2023-10-06T04:58:46.992 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T04:58:46.992 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/orig.config.yaml 2023-10-06T04:58:47.038 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T04:58:47.134 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} email: yweinste@redhat.com first_in_suite: false job_id: '7412322' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKBa3lfTt2DZu9P4yZRPDyvyQf+QSD5SAI+sU0pgiHdmC0MYDM0qgHDrd8YHKd1B8IXkPBZAihL7kQmYTYjiR1Y= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIWK9WT+5apjDhQm97J+fYPfqyp6XWshyveO/zIdWuVysyXRHbrCqwG5j7SWBKH3wdTQQOfl3zYtPbrFPgBzFrQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T04:58:47.134 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T04:58:47.135 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T04:58:47.135 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T04:58:47.136 INFO:teuthology.task.internal:Checking packages... 2023-10-06T04:58:47.159 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T04:58:47.159 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T04:58:47.159 INFO:teuthology.packaging:ref: None 2023-10-06T04:58:47.159 INFO:teuthology.packaging:tag: None 2023-10-06T04:58:47.159 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:58:47.159 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:58:47.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:58:47.296 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274.g68e53a4a 2023-10-06T04:58:47.296 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T04:58:47.306 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T04:58:47.306 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T04:58:47.312 INFO:teuthology.task.internal:Saving configuration 2023-10-06T04:58:47.332 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T04:58:47.339 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:58:47.364 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:53:42.380397', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKBa3lfTt2DZu9P4yZRPDyvyQf+QSD5SAI+sU0pgiHdmC0MYDM0qgHDrd8YHKd1B8IXkPBZAihL7kQmYTYjiR1Y='} 2023-10-06T04:58:47.388 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:53:42.381794', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIWK9WT+5apjDhQm97J+fYPfqyp6XWshyveO/zIdWuVysyXRHbrCqwG5j7SWBKH3wdTQQOfl3zYtPbrFPgBzFrQ='} 2023-10-06T04:58:47.388 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T04:58:47.394 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T04:58:47.394 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T04:58:47.394 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T04:58:47.459 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f41f38bbb80>, signals=[15]) 2023-10-06T04:58:47.459 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T04:58:47.465 INFO:teuthology.task.internal:Opening connections... 2023-10-06T04:58:47.465 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T04:58:47.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:58:47.537 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T04:58:47.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:58:47.605 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T04:58:47.611 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2023-10-06T04:58:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-10-06T04:58:47.635 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-10-06T04:58:47.697 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2023-10-06T04:58:47.697 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8" 2023-10-06T04:58:47.697 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2023-10-06T04:58:47.697 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:58:47.698 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T04:58:47.699 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-10-06T04:58:47.723 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2023-10-06T04:58:47.745 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2023-10-06T04:58:47.746 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="8" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="8" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T04:58:47.807 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2023-10-06T04:58:47.808 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:58:47.808 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:58:47.808 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T04:58:47.808 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-10-06T04:58:47.831 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T04:58:47.841 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T04:58:47.847 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T04:58:47.847 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:58:47.850 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:58:47.870 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T04:58:47.877 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T04:58:47.877 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:58:47.912 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:58:47.949 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T04:58:48.000 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T04:58:48.015 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-06T04:58:48.015 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-06T04:58:48.015 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:58:48.016 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:58:48.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:58:48.036 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2023-10-06T04:58:48.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:58:48.038 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-10-06T04:58:48.098 INFO:teuthology.orchestra.run.smithi144.stdout:4.18.0-489.el8.x86_64 2023-10-06T04:58:48.098 INFO:teuthology.task.kernel:Running kernel on smithi144: 4.18.0-489.el8.x86_64 2023-10-06T04:58:48.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2023-10-06T04:58:48.099 INFO:teuthology.orchestra.run.smithi045.stdout:4.18.0-489.el8.x86_64 2023-10-06T04:58:48.100 INFO:teuthology.task.kernel:Running kernel on smithi045: 4.18.0-489.el8.x86_64 2023-10-06T04:58:48.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2023-10-06T04:58:52.382 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - AppStream 19 MB/s | 33 MB 00:01 2023-10-06T04:58:54.747 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - AppStream 8.4 MB/s | 33 MB 00:03 2023-10-06T04:59:03.048 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 48 MB 00:04 2023-10-06T04:59:04.896 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 48 MB 00:04 2023-10-06T04:59:12.138 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - Extras 58 kB/s | 18 kB 00:00 2023-10-06T04:59:12.555 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - Extras common packages 29 kB/s | 6.9 kB 00:00 2023-10-06T04:59:12.857 INFO:teuthology.orchestra.run.smithi144.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-10-06T04:59:13.614 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - Extras 5.5 kB/s | 18 kB 00:03 2023-10-06T04:59:14.048 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - Extras common packages 29 kB/s | 6.9 kB 00:00 2023-10-06T04:59:14.350 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-06T04:59:14.993 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 16 MB 00:01 2023-10-06T04:59:15.294 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-10-06T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 397 kB/s | 24 kB 00:00 2023-10-06T04:59:19.367 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 322 kB/s | 24 kB 00:00 2023-10-06T04:59:22.240 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-06T04:59:22.240 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-06T04:59:22.495 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2023-10-06T04:59:22.495 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2023-10-06T04:59:22.495 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2023-10-06T04:59:22.496 INFO:teuthology.orchestra.run.smithi144.stdout:Install 3 Packages 2023-10-06T04:59:22.497 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T04:59:22.497 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 88 M 2023-10-06T04:59:22.497 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 96 M 2023-10-06T04:59:22.497 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2023-10-06T04:59:23.116 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-06T04:59:23.116 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-06T04:59:23.387 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-06T04:59:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T04:59:23.389 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-10-06T04:59:23.389 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-10-06T04:59:23.389 INFO:teuthology.orchestra.run.smithi045.stdout:Install 3 Packages 2023-10-06T04:59:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T04:59:23.389 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 88 M 2023-10-06T04:59:23.389 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 96 M 2023-10-06T04:59:23.390 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-10-06T04:59:24.937 INFO:teuthology.orchestra.run.smithi144.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 4.2 MB/s | 10 MB 00:02 2023-10-06T04:59:25.959 INFO:teuthology.orchestra.run.smithi045.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 4.2 MB/s | 10 MB 00:02 2023-10-06T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 9.0 MB/s | 35 MB 00:03 2023-10-06T04:59:26.892 INFO:teuthology.orchestra.run.smithi144.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 9.8 MB/s | 43 MB 00:04 2023-10-06T04:59:26.892 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:59:26.892 INFO:teuthology.orchestra.run.smithi144.stdout:Total 20 MB/s | 88 MB 00:04 2023-10-06T04:59:27.560 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2023-10-06T04:59:27.668 INFO:teuthology.orchestra.run.smithi045.stdout:(2/3): kernel-core-4.18.0-514.el8.x86_64.rpm 10 MB/s | 43 MB 00:04 2023-10-06T04:59:27.974 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2023-10-06T04:59:27.974 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2023-10-06T04:59:29.200 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2023-10-06T04:59:29.228 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2023-10-06T04:59:32.038 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2023-10-06T04:59:32.623 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:59:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:59:36.598 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:59:39.976 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:59:40.285 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:01.997 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:02.019 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:02.382 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:02.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-06T05:00:02.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:00:03.079 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2023-10-06T05:00:03.490 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T05:00:03.708 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T05:00:03.708 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T05:00:03.708 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-06T05:00:03.708 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T05:00:03.708 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T05:00:03.708 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-06T05:00:03.709 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T05:00:03.709 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T05:00:03.917 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T05:00:03.917 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T05:00:03.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.02 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 129.el8 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Group : System Environment/Base 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 9265586 2023-10-06T05:00:03.953 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Relocations : (not relocatable) 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : CentOS BuildSystem 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T05:00:03.954 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T05:00:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T05:00:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2023-10-06T05:00:03.955 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:00:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T05:00:03.956 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T05:00:03.956 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 4.18.0-514.el8.x86_64 2023-10-06T05:00:03.956 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T05:00:05.072 INFO:teuthology.orchestra.run.smithi045.stdout:(3/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 874 kB/s | 35 MB 00:41 2023-10-06T05:00:05.072 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-10-06T05:00:05.072 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.1 MB/s | 88 MB 00:41 2023-10-06T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2023-10-06T05:00:05.777 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-10-06T05:00:06.232 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-10-06T05:00:06.232 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-10-06T05:00:07.449 INFO:teuthology.orchestra.run.smithi144.stderr:done 2023-10-06T05:00:07.450 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-10-06T05:00:07.479 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.EpTkrb50g7 2023-10-06T05:00:07.480 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EpTkrb50g7 2023-10-06T05:00:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-10-06T05:00:07.549 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-10-06T05:00:07.554 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.EpTkrb50g7 2023-10-06T05:00:07.699 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.EpTkrb50g7 is 6KB 2023-10-06T05:00:07.721 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.EpTkrb50g7 2023-10-06T05:00:07.739 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T05:00:07.804 INFO:teuthology.orchestra.run.smithi144.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-06T05:00:07.805 INFO:teuthology.orchestra.run.smithi144.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-06T05:00:07.805 INFO:teuthology.orchestra.run.smithi144.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-06T05:00:07.805 INFO:teuthology.orchestra.run.smithi144.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-06T05:00:07.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-06T05:00:07.963 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2023-10-06T05:00:10.477 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-10-06T05:00:11.062 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T05:00:14.824 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T05:00:15.203 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T05:00:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T05:00:19.146 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:37.966 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T05:00:37.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-06T05:00:37.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:00:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-06T05:00:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-06T05:00:43.782 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T05:00:43.782 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:00:43.782 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-10-06T05:00:43.782 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-06T05:00:43.782 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-06T05:00:43.782 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:00:43.782 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-10-06T05:00:43.926 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T05:00:44.150 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T05:00:44.150 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T05:00:44.151 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-06T05:00:44.151 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T05:00:44.151 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T05:00:44.151 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-06T05:00:44.151 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T05:00:44.151 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T05:00:44.369 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T05:00:44.369 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T05:00:44.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.02 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 129.el8 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Group : System Environment/Base 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 9265586 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-06T05:00:44.404 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:Relocations : (not relocatable) 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : CentOS BuildSystem 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T05:00:44.405 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T05:00:44.406 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2023-10-06T05:00:44.406 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:00:44.406 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T05:00:44.407 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T05:00:44.407 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 4.18.0-514.el8.x86_64 2023-10-06T05:00:44.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T05:00:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2023-10-06T05:00:47.898 INFO:teuthology.orchestra.run.smithi045.stderr:done 2023-10-06T05:00:47.899 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2023-10-06T05:00:47.920 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.kVIGUapKmx 2023-10-06T05:00:47.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kVIGUapKmx 2023-10-06T05:00:47.992 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.kVIGUapKmx 2023-10-06T05:00:48.135 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.kVIGUapKmx is 6KB 2023-10-06T05:00:48.183 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.kVIGUapKmx 2023-10-06T05:00:48.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T05:00:48.271 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-06T05:00:48.271 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-06T05:00:48.271 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-06T05:00:48.271 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-06T05:00:48.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-06T05:00:48.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2023-10-06T05:00:56.314 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-10-06T05:01:05.322 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-06T05:01:05.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:08.378 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-10-06T05:01:18.438 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T05:01:18.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-06T05:01:18.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:20.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-06T05:01:20.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:36.762 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-10-06T05:01:45.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-06T05:01:45.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:52.983 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-10-06T05:01:54.193 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-06T05:01:54.193 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-514.el8.x86_64"... 2023-10-06T05:01:54.193 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2023-10-06T05:01:54.211 INFO:teuthology.orchestra.run.smithi144.stdout:4.18.0-514.el8.x86_64 2023-10-06T05:01:54.211 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-06T05:01:54.211 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T05:01:54.211 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T05:01:55.212 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T05:01:55.213 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T05:01:55.281 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2023-10-06T05:01:55.293 DEBUG:teuthology.parallel:result is None 2023-10-06T05:02:45.773 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T05:02:57.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-06T05:02:57.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:02:58.039 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-10-06T05:02:58.371 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-06T05:02:58.372 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-10-06T05:02:58.372 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-10-06T05:02:58.394 INFO:teuthology.orchestra.run.smithi045.stdout:4.18.0-514.el8.x86_64 2023-10-06T05:02:58.394 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-06T05:02:58.395 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T05:02:58.395 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T05:02:59.396 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T05:02:59.396 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T05:02:59.475 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2023-10-06T05:02:59.490 DEBUG:teuthology.parallel:result is None 2023-10-06T05:02:59.490 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T05:02:59.500 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T05:02:59.501 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T05:02:59.503 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T05:02:59.525 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T05:02:59.537 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T05:02:59.550 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T05:02:59.551 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T05:02:59.567 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T05:02:59.608 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T05:02:59.619 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T05:02:59.620 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T05:02:59.637 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T05:02:59.689 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:59.690 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:59.701 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:59.703 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:59.705 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T05:02:59.716 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T05:02:59.716 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T05:02:59.718 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T05:02:59.751 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T05:02:59.764 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T05:02:59.764 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T05:02:59.794 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T05:02:59.814 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:02:59.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:02:59.969 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:03:00.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:03:00.071 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:03:00.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T05:03:00.147 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:03:00.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:03:00.245 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:03:00.329 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:03:00.376 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:03:00.377 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T05:03:00.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2023-10-06T05:03:00.450 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2023-10-06T05:03:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:03:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:03:00.937 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T05:03:00.945 INFO:teuthology.task.internal:Starting timer... 2023-10-06T05:03:00.945 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T05:03:00.962 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T05:03:00.976 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T05:03:00.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2023-10-06T05:03:01.065 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2023-10-06T05:03:01.066 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2023-10-06T05:03:01.141 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2023-10-06T05:03:01.142 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T05:03:01.143 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2023-10-06T05:03:01.191 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2023-10-06T05:03:01.191 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2023-10-06T05:03:01.247 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2023-10-06T05:03:01.247 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2023-10-06T05:03:01.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:03:01.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:03:01.285 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:03:01.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:03:01.312 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T05:03:01.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2023-10-06T05:03:01.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2023-10-06T05:03:01.394 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T05:03:01.403 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T05:03:01.403 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T05:03:01.419 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T05:03:01.421 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T05:11:12.260 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2023-10-06T05:11:12.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-06T05:11:12.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:11:12.336 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-10-06T05:11:12.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-10-06T05:11:12.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-06T05:11:12.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:11:12.485 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-10-06T05:11:12.554 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-06T05:11:12.554 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T05:11:12.566 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T05:11:12.566 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T05:11:12.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:11:12.569 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T05:11:12.569 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:11:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T05:11:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T05:11:12.629 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T05:11:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T05:11:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2023-10-06T05:11:12.712 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2023-10-06T05:11:12.719 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2023-10-06T05:11:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T05:11:12.731 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2023-10-06T05:11:12.744 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T05:11:12.745 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T05:11:12.759 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T05:11:12.820 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-10-06T05:11:12.824 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:11:12.824 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-10-06T05:11:12.824 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.824 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.824 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.824 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.826 INFO:teuthology.orchestra.run.smithi144.stderr:bash: ntpq: command not found 2023-10-06T05:11:12.829 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:11:12.829 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2023-10-06T05:11:12.829 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.829 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.829 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.829 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:11:12.830 INFO:teuthology.run_tasks:Running task install... 2023-10-06T05:11:12.839 DEBUG:teuthology.task.install:project ceph 2023-10-06T05:11:12.839 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T05:11:12.839 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T05:11:12.839 INFO:teuthology.task.install:Using flavor: default 2023-10-06T05:11:12.845 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T05:11:12.845 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T05:11:12.845 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T05:11:12.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:11:12.847 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T05:11:12.848 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:11:12.989 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T05:11:12.989 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T05:11:13.082 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T05:11:13.083 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T05:11:13.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T05:11:13.099 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:11:13.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T05:11:13.135 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T05:11:13.135 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T05:11:13.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T05:11:13.253 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T05:11:13.254 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:11:13.254 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T05:11:13.283 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T05:11:13.284 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T05:11:13.302 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T05:11:13.335 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2023-10-06T05:11:13.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2023-10-06T05:11:13.353 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T05:11:13.432 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T05:11:13.499 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2023-10-06T05:11:13.501 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2023-10-06T05:11:13.653 INFO:teuthology.orchestra.run.smithi045.stdout:76 files removed 2023-10-06T05:11:13.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T05:11:13.769 INFO:teuthology.orchestra.run.smithi144.stdout:76 files removed 2023-10-06T05:11:13.796 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T05:11:14.589 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 183 kB/s | 78 kB 00:00 2023-10-06T05:11:14.604 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 198 kB/s | 78 kB 00:00 2023-10-06T05:11:14.931 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 31 kB/s | 9.2 kB 00:00 2023-10-06T05:11:14.972 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 28 kB/s | 9.2 kB 00:00 2023-10-06T05:11:15.272 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 5.3 kB/s | 1.4 kB 00:00 2023-10-06T05:11:15.290 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 4.4 kB/s | 1.4 kB 00:00 2023-10-06T05:11:17.522 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - AppStream 15 MB/s | 33 MB 00:02 2023-10-06T05:11:18.407 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - AppStream 11 MB/s | 33 MB 00:03 2023-10-06T05:11:28.608 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - BaseOS 9.7 MB/s | 48 MB 00:04 2023-10-06T05:11:34.807 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - BaseOS 4.9 MB/s | 48 MB 00:09 2023-10-06T05:11:35.940 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - Extras 58 kB/s | 18 kB 00:00 2023-10-06T05:11:36.690 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 6.9 kB 00:00 2023-10-06T05:11:37.850 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 8 - PowerTools 6.5 MB/s | 6.5 MB 00:01 2023-10-06T05:11:39.276 INFO:teuthology.orchestra.run.smithi144.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-06T05:11:41.496 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2023-10-06T05:11:42.582 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - Extras 103 kB/s | 18 kB 00:00 2023-10-06T05:11:43.231 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - Extras common packages 15 kB/s | 6.9 kB 00:00 2023-10-06T05:11:44.309 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - PowerTools 7.1 MB/s | 6.5 MB 00:00 2023-10-06T05:11:45.359 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 306 kB/s | 24 kB 00:00 2023-10-06T05:11:45.861 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-10-06T05:11:46.970 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-06T05:11:49.590 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:49.591 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:49.604 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-06T05:11:49.604 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-06T05:11:49.605 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-06T05:11:49.606 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-06T05:11:49.607 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T05:11:49.609 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T05:11:49.612 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T05:11:49.613 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T05:11:49.830 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2023-10-06T05:11:49.843 INFO:teuthology.orchestra.run.smithi144.stdout:==================================================================================================== 2023-10-06T05:11:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2023-10-06T05:11:49.843 INFO:teuthology.orchestra.run.smithi144.stdout:==================================================================================================== 2023-10-06T05:11:49.843 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T05:11:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-06T05:11:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T05:11:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-06T05:11:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-06T05:11:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T05:11:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T05:11:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-06T05:11:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T05:11:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T05:11:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T05:11:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T05:11:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T05:11:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2023-10-06T05:11:49.857 INFO:teuthology.orchestra.run.smithi144.stdout:==================================================================================================== 2023-10-06T05:11:49.858 INFO:teuthology.orchestra.run.smithi144.stdout:Install 123 Packages 2023-10-06T05:11:49.858 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 16 Packages 2023-10-06T05:11:49.858 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:11:49.858 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 209 M 2023-10-06T05:11:49.858 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2023-10-06T05:11:50.561 INFO:teuthology.orchestra.run.smithi144.stdout:(1/139): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-10-06T05:11:51.245 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 335 kB/s | 24 kB 00:00 2023-10-06T05:11:51.653 INFO:teuthology.orchestra.run.smithi144.stdout:(2/139): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 804 kB/s | 877 kB 00:01 2023-10-06T05:11:51.912 INFO:teuthology.orchestra.run.smithi144.stdout:(3/139): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 3.3 MB/s | 5.2 MB 00:01 2023-10-06T05:11:51.937 INFO:teuthology.orchestra.run.smithi144.stdout:(4/139): ceph-immutable-object-cache-17.2.6-127 570 kB/s | 162 kB 00:00 2023-10-06T05:11:52.596 INFO:teuthology.orchestra.run.smithi144.stdout:(5/139): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 3.4 MB/s | 2.3 MB 00:00 2023-10-06T05:11:53.563 INFO:teuthology.orchestra.run.smithi144.stdout:(6/139): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 4.1 MB/s | 3.9 MB 00:00 2023-10-06T05:11:53.804 INFO:teuthology.orchestra.run.smithi144.stdout:(7/139): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 861 kB/s | 1.6 MB 00:01 2023-10-06T05:11:55.914 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:55.915 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:11:55.933 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-06T05:11:55.933 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-06T05:11:55.934 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-06T05:11:55.934 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-06T05:11:55.934 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T05:11:55.936 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T05:11:55.939 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T05:11:55.941 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T05:11:56.179 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-10-06T05:11:56.194 INFO:teuthology.orchestra.run.smithi045.stdout:==================================================================================================== 2023-10-06T05:11:56.194 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-10-06T05:11:56.194 INFO:teuthology.orchestra.run.smithi045.stdout:==================================================================================================== 2023-10-06T05:11:56.194 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-10-06T05:11:56.194 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:56.195 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-06T05:11:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-06T05:11:56.197 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-06T05:11:56.198 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:11:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-06T05:11:56.200 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-06T05:11:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T05:11:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-06T05:11:56.203 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T05:11:56.204 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T05:11:56.205 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-06T05:11:56.206 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-06T05:11:56.207 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:11:56.207 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-10-06T05:11:56.207 INFO:teuthology.orchestra.run.smithi045.stdout:==================================================================================================== 2023-10-06T05:11:56.207 INFO:teuthology.orchestra.run.smithi045.stdout:Install 123 Packages 2023-10-06T05:11:56.207 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 16 Packages 2023-10-06T05:11:56.207 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:11:56.207 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 209 M 2023-10-06T05:11:56.208 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-10-06T05:11:56.821 INFO:teuthology.orchestra.run.smithi045.stdout:(1/139): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-10-06T05:11:57.929 INFO:teuthology.orchestra.run.smithi144.stdout:(8/139): ceph-osd-17.2.6-1274.g68e53a4a.el8.x86 3.8 MB/s | 17 MB 00:04 2023-10-06T05:11:58.021 INFO:teuthology.orchestra.run.smithi144.stdout:(9/139): ceph-selinux-17.2.6-1274.g68e53a4a.el8 266 kB/s | 24 kB 00:00 2023-10-06T05:11:58.080 INFO:teuthology.orchestra.run.smithi045.stdout:(2/139): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 697 kB/s | 877 kB 00:01 2023-10-06T05:11:58.314 INFO:teuthology.orchestra.run.smithi045.stdout:(3/139): ceph-immutable-object-cache-17.2.6-127 693 kB/s | 162 kB 00:00 2023-10-06T05:12:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:(4/139): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-10-06T05:12:01.897 INFO:teuthology.orchestra.run.smithi045.stdout:(5/139): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 1.2 MB/s | 1.6 MB 00:01 2023-10-06T05:12:02.598 INFO:teuthology.orchestra.run.smithi045.stdout:(6/139): ceph-common-17.2.6-1274.g68e53a4a.el8. 3.9 MB/s | 23 MB 00:06 2023-10-06T05:12:03.523 INFO:teuthology.orchestra.run.smithi045.stdout:(7/139): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 754 kB/s | 5.2 MB 00:07 2023-10-06T05:12:04.065 INFO:teuthology.orchestra.run.smithi045.stdout:(8/139): ceph-osd-17.2.6-1274.g68e53a4a.el8.x86 11 MB/s | 17 MB 00:01 2023-10-06T05:12:04.087 INFO:teuthology.orchestra.run.smithi144.stdout:(10/139): ceph-radosgw-17.2.6-1274.g68e53a4a.el 1.6 MB/s | 17 MB 00:10 2023-10-06T05:12:04.148 INFO:teuthology.orchestra.run.smithi045.stdout:(9/139): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 1.7 MB/s | 3.9 MB 00:02 2023-10-06T05:12:04.174 INFO:teuthology.orchestra.run.smithi045.stdout:(10/139): ceph-selinux-17.2.6-1274.g68e53a4a.el 224 kB/s | 24 kB 00:00 2023-10-06T05:12:04.196 INFO:teuthology.orchestra.run.smithi144.stdout:(11/139): cephfs-mirror-17.2.6-1274.g68e53a4a.e 2.3 MB/s | 255 kB 00:00 2023-10-06T05:12:04.249 INFO:teuthology.orchestra.run.smithi045.stdout:(11/139): cephfs-mirror-17.2.6-1274.g68e53a4a.e 3.3 MB/s | 255 kB 00:00 2023-10-06T05:12:04.263 INFO:teuthology.orchestra.run.smithi144.stdout:(12/139): libcephfs-devel-17.2.6-1274.g68e53a4a 373 kB/s | 25 kB 00:00 2023-10-06T05:12:04.308 INFO:teuthology.orchestra.run.smithi045.stdout:(12/139): libcephfs-devel-17.2.6-1274.g68e53a4a 427 kB/s | 25 kB 00:00 2023-10-06T05:12:04.433 INFO:teuthology.orchestra.run.smithi045.stdout:(13/139): libcephfs2-17.2.6-1274.g68e53a4a.el8. 6.2 MB/s | 797 kB 00:00 2023-10-06T05:12:04.508 INFO:teuthology.orchestra.run.smithi045.stdout:(14/139): libcephsqlite-17.2.6-1274.g68e53a4a.e 2.3 MB/s | 175 kB 00:00 2023-10-06T05:12:04.530 INFO:teuthology.orchestra.run.smithi144.stdout:(13/139): libcephfs2-17.2.6-1274.g68e53a4a.el8. 2.9 MB/s | 797 kB 00:00 2023-10-06T05:12:04.594 INFO:teuthology.orchestra.run.smithi045.stdout:(15/139): librados-devel-17.2.6-1274.g68e53a4a. 1.5 MB/s | 130 kB 00:00 2023-10-06T05:12:04.672 INFO:teuthology.orchestra.run.smithi144.stdout:(14/139): libcephsqlite-17.2.6-1274.g68e53a4a.e 1.2 MB/s | 175 kB 00:00 2023-10-06T05:12:04.700 INFO:teuthology.orchestra.run.smithi045.stdout:(16/139): libradosstriper1-17.2.6-1274.g68e53a4 4.7 MB/s | 506 kB 00:00 2023-10-06T05:12:04.747 INFO:teuthology.orchestra.run.smithi144.stdout:(15/139): librados-devel-17.2.6-1274.g68e53a4a. 1.7 MB/s | 130 kB 00:00 2023-10-06T05:12:05.031 INFO:teuthology.orchestra.run.smithi144.stdout:(16/139): ceph-common-17.2.6-1274.g68e53a4a.el8 1.6 MB/s | 23 MB 00:14 2023-10-06T05:12:05.139 INFO:teuthology.orchestra.run.smithi144.stdout:(17/139): libradosstriper1-17.2.6-1274.g68e53a4 1.3 MB/s | 506 kB 00:00 2023-10-06T05:12:05.256 INFO:teuthology.orchestra.run.smithi144.stdout:(18/139): python3-ceph-argparse-17.2.6-1274.g68 382 kB/s | 45 kB 00:00 2023-10-06T05:12:05.357 INFO:teuthology.orchestra.run.smithi144.stdout:(19/139): python3-ceph-common-17.2.6-1274.g68e5 1.1 MB/s | 115 kB 00:00 2023-10-06T05:12:05.392 INFO:teuthology.orchestra.run.smithi045.stdout:(17/139): librgw2-17.2.6-1274.g68e53a4a.el8.x86 8.0 MB/s | 5.6 MB 00:00 2023-10-06T05:12:05.476 INFO:teuthology.orchestra.run.smithi045.stdout:(18/139): python3-ceph-argparse-17.2.6-1274.g68 537 kB/s | 45 kB 00:00 2023-10-06T05:12:05.515 INFO:teuthology.orchestra.run.smithi144.stdout:(20/139): python3-cephfs-17.2.6-1274.g68e53a4a. 1.2 MB/s | 190 kB 00:00 2023-10-06T05:12:05.559 INFO:teuthology.orchestra.run.smithi045.stdout:(19/139): python3-ceph-common-17.2.6-1274.g68e5 1.4 MB/s | 115 kB 00:00 2023-10-06T05:12:05.643 INFO:teuthology.orchestra.run.smithi045.stdout:(20/139): python3-cephfs-17.2.6-1274.g68e53a4a. 2.2 MB/s | 190 kB 00:00 2023-10-06T05:12:05.691 INFO:teuthology.orchestra.run.smithi144.stdout:(21/139): python3-rados-17.2.6-1274.g68e53a4a.e 2.1 MB/s | 371 kB 00:00 2023-10-06T05:12:05.747 INFO:teuthology.orchestra.run.smithi045.stdout:(21/139): python3-rados-17.2.6-1274.g68e53a4a.e 3.5 MB/s | 371 kB 00:00 2023-10-06T05:12:05.844 INFO:teuthology.orchestra.run.smithi045.stdout:(22/139): python3-rbd-17.2.6-1274.g68e53a4a.el8 3.6 MB/s | 358 kB 00:00 2023-10-06T05:12:05.874 INFO:teuthology.orchestra.run.smithi144.stdout:(22/139): python3-rbd-17.2.6-1274.g68e53a4a.el8 1.9 MB/s | 358 kB 00:00 2023-10-06T05:12:05.919 INFO:teuthology.orchestra.run.smithi045.stdout:(23/139): python3-rgw-17.2.6-1274.g68e53a4a.el8 1.4 MB/s | 107 kB 00:00 2023-10-06T05:12:05.958 INFO:teuthology.orchestra.run.smithi144.stdout:(23/139): python3-rgw-17.2.6-1274.g68e53a4a.el8 1.3 MB/s | 107 kB 00:00 2023-10-06T05:12:05.995 INFO:teuthology.orchestra.run.smithi045.stdout:(24/139): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.2 MB/s | 91 kB 00:00 2023-10-06T05:12:06.025 INFO:teuthology.orchestra.run.smithi144.stdout:(24/139): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.3 MB/s | 91 kB 00:00 2023-10-06T05:12:06.362 INFO:teuthology.orchestra.run.smithi045.stdout:(25/139): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 9.4 MB/s | 3.5 MB 00:00 2023-10-06T05:12:06.437 INFO:teuthology.orchestra.run.smithi045.stdout:(26/139): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 2.3 MB/s | 173 kB 00:00 2023-10-06T05:12:06.529 INFO:teuthology.orchestra.run.smithi045.stdout:(27/139): ceph-grafana-dashboards-17.2.6-1274.g 252 kB/s | 23 kB 00:00 2023-10-06T05:12:06.596 INFO:teuthology.orchestra.run.smithi045.stdout:(28/139): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 1.8 MB/s | 121 kB 00:00 2023-10-06T05:12:06.804 INFO:teuthology.orchestra.run.smithi045.stdout:(29/139): ceph-mgr-dashboard-17.2.6-1274.g68e53 8.1 MB/s | 1.7 MB 00:00 2023-10-06T05:12:06.975 INFO:teuthology.orchestra.run.smithi144.stdout:(25/139): librgw2-17.2.6-1274.g68e53a4a.el8.x86 2.9 MB/s | 5.6 MB 00:01 2023-10-06T05:12:07.075 INFO:teuthology.orchestra.run.smithi144.stdout:(26/139): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 1.7 MB/s | 173 kB 00:00 2023-10-06T05:12:07.176 INFO:teuthology.orchestra.run.smithi144.stdout:(27/139): ceph-grafana-dashboards-17.2.6-1274.g 230 kB/s | 23 kB 00:00 2023-10-06T05:12:07.268 INFO:teuthology.orchestra.run.smithi144.stdout:(28/139): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 1.3 MB/s | 121 kB 00:00 2023-10-06T05:12:07.360 INFO:teuthology.orchestra.run.smithi144.stdout:(29/139): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 2.6 MB/s | 3.5 MB 00:01 2023-10-06T05:12:07.505 INFO:teuthology.orchestra.run.smithi045.stdout:(30/139): ceph-mgr-diskprediction-local-17.2.6- 11 MB/s | 7.4 MB 00:00 2023-10-06T05:12:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1274.g68 2.8 MB/s | 238 kB 00:00 2023-10-06T05:12:07.672 INFO:teuthology.orchestra.run.smithi045.stdout:(32/139): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 569 kB/s | 47 kB 00:00 2023-10-06T05:12:07.756 INFO:teuthology.orchestra.run.smithi045.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1274.g6 174 kB/s | 14 kB 00:00 2023-10-06T05:12:07.827 INFO:teuthology.orchestra.run.smithi144.stdout:(30/139): ceph-mgr-dashboard-17.2.6-1274.g68e53 3.0 MB/s | 1.7 MB 00:00 2023-10-06T05:12:07.839 INFO:teuthology.orchestra.run.smithi045.stdout:(34/139): ceph-volume-17.2.6-1274.g68e53a4a.el8 3.1 MB/s | 267 kB 00:00 2023-10-06T05:12:07.915 INFO:teuthology.orchestra.run.smithi045.stdout:(35/139): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.0 MB/s | 80 kB 00:00 2023-10-06T05:12:07.944 INFO:teuthology.orchestra.run.smithi144.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1274.g68 2.0 MB/s | 238 kB 00:00 2023-10-06T05:12:07.973 INFO:teuthology.orchestra.run.smithi045.stdout:(36/139): cephfs-top-17.2.6-1274.g68e53a4a.el8. 306 kB/s | 18 kB 00:00 2023-10-06T05:12:08.011 INFO:teuthology.orchestra.run.smithi144.stdout:(32/139): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 712 kB/s | 47 kB 00:00 2023-10-06T05:12:08.069 INFO:teuthology.orchestra.run.smithi144.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1274.g6 249 kB/s | 14 kB 00:00 2023-10-06T05:12:08.195 INFO:teuthology.orchestra.run.smithi144.stdout:(34/139): ceph-volume-17.2.6-1274.g68e53a4a.el8 2.1 MB/s | 267 kB 00:00 2023-10-06T05:12:08.278 INFO:teuthology.orchestra.run.smithi144.stdout:(35/139): cephadm-17.2.6-1274.g68e53a4a.el8.noa 967 kB/s | 80 kB 00:00 2023-10-06T05:12:08.345 INFO:teuthology.orchestra.run.smithi144.stdout:(36/139): cephfs-top-17.2.6-1274.g68e53a4a.el8. 267 kB/s | 18 kB 00:00 2023-10-06T05:12:08.424 INFO:teuthology.orchestra.run.smithi045.stdout:(37/139): automake-1.16.1-8.el8.noarch.rpm 1.6 MB/s | 740 kB 00:00 2023-10-06T05:12:08.616 INFO:teuthology.orchestra.run.smithi045.stdout:(38/139): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2023-10-06T05:12:08.729 INFO:teuthology.orchestra.run.smithi144.stdout:(37/139): automake-1.16.1-8.el8.noarch.rpm 1.9 MB/s | 740 kB 00:00 2023-10-06T05:12:08.733 INFO:teuthology.orchestra.run.smithi045.stdout:(39/139): flex-2.6.1-9.el8.x86_64.rpm 2.7 MB/s | 320 kB 00:00 2023-10-06T05:12:08.862 INFO:teuthology.orchestra.run.smithi144.stdout:(38/139): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2023-10-06T05:12:08.883 INFO:teuthology.orchestra.run.smithi045.stdout:(40/139): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2023-10-06T05:12:08.963 INFO:teuthology.orchestra.run.smithi144.stdout:(39/139): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-10-06T05:12:09.017 INFO:teuthology.orchestra.run.smithi045.stdout:(41/139): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-10-06T05:12:09.038 INFO:teuthology.orchestra.run.smithi144.stdout:(40/139): jq-1.6-7.el8.x86_64.rpm 2.6 MB/s | 202 kB 00:00 2023-10-06T05:12:09.113 INFO:teuthology.orchestra.run.smithi144.stdout:(41/139): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2023-10-06T05:12:09.208 INFO:teuthology.orchestra.run.smithi045.stdout:(42/139): libtool-2.4.6-25.el8.x86_64.rpm 3.6 MB/s | 709 kB 00:00 2023-10-06T05:12:09.272 INFO:teuthology.orchestra.run.smithi144.stdout:(42/139): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2023-10-06T05:12:09.364 INFO:teuthology.orchestra.run.smithi144.stdout:(43/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2023-10-06T05:12:09.367 INFO:teuthology.orchestra.run.smithi045.stdout:(43/139): ceph-radosgw-17.2.6-1274.g68e53a4a.el 2.9 MB/s | 17 MB 00:05 2023-10-06T05:12:09.393 INFO:teuthology.orchestra.run.smithi045.stdout:(44/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2023-10-06T05:12:09.448 INFO:teuthology.orchestra.run.smithi144.stdout:(44/139): ceph-mgr-diskprediction-local-17.2.6- 3.5 MB/s | 7.4 MB 00:02 2023-10-06T05:12:09.473 INFO:teuthology.orchestra.run.smithi144.stdout:(45/139): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-10-06T05:12:09.660 INFO:teuthology.orchestra.run.smithi045.stdout:(45/139): oniguruma-6.8.2-2.el8.x86_64.rpm 642 kB/s | 187 kB 00:00 2023-10-06T05:12:09.982 INFO:teuthology.orchestra.run.smithi144.stdout:(46/139): openblas-threads-0.3.15-4.el8.x86_64. 9.6 MB/s | 4.9 MB 00:00 2023-10-06T05:12:10.344 INFO:teuthology.orchestra.run.smithi045.stdout:(46/139): openblas-0.3.15-4.el8.x86_64.rpm 5.0 MB/s | 4.7 MB 00:00 2023-10-06T05:12:10.590 INFO:teuthology.orchestra.run.smithi144.stdout:(47/139): ceph-test-17.2.6-1274.g68e53a4a.el8.x 4.1 MB/s | 52 MB 00:12 2023-10-06T05:12:10.674 INFO:teuthology.orchestra.run.smithi144.stdout:(48/139): python3-babel-2.5.1-7.el8.noarch.rpm 6.9 MB/s | 4.8 MB 00:00 2023-10-06T05:12:10.766 INFO:teuthology.orchestra.run.smithi144.stdout:(49/139): python3-jsonpatch-1.21-2.el8.noarch.r 290 kB/s | 27 kB 00:00 2023-10-06T05:12:10.858 INFO:teuthology.orchestra.run.smithi144.stdout:(50/139): python3-jsonpointer-1.10-11.el8.noarc 217 kB/s | 20 kB 00:00 2023-10-06T05:12:10.942 INFO:teuthology.orchestra.run.smithi144.stdout:(51/139): python3-mako-1.0.6-14.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2023-10-06T05:12:11.025 INFO:teuthology.orchestra.run.smithi144.stdout:(52/139): python3-markupsafe-0.23-19.el8.x86_64 466 kB/s | 39 kB 00:00 2023-10-06T05:12:11.059 INFO:teuthology.orchestra.run.smithi144.stdout:(53/139): python3-jinja2-2.10.1-3.el8.noarch.rp 1.1 MB/s | 538 kB 00:00 2023-10-06T05:12:11.085 INFO:teuthology.orchestra.run.smithi045.stdout:(47/139): openblas-threads-0.3.15-4.el8.x86_64. 3.4 MB/s | 4.9 MB 00:01 2023-10-06T05:12:11.143 INFO:teuthology.orchestra.run.smithi144.stdout:(54/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.6 MB/s | 225 kB 00:00 2023-10-06T05:12:11.226 INFO:teuthology.orchestra.run.smithi144.stdout:(55/139): python3-prettytable-0.7.2-14.el8.noar 527 kB/s | 44 kB 00:00 2023-10-06T05:12:11.236 INFO:teuthology.orchestra.run.smithi045.stdout:(48/139): python3-jinja2-2.10.1-3.el8.noarch.rp 3.5 MB/s | 538 kB 00:00 2023-10-06T05:12:11.293 INFO:teuthology.orchestra.run.smithi144.stdout:(56/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2023-10-06T05:12:11.319 INFO:teuthology.orchestra.run.smithi045.stdout:(49/139): python3-jsonpatch-1.21-2.el8.noarch.r 319 kB/s | 27 kB 00:00 2023-10-06T05:12:11.368 INFO:teuthology.orchestra.run.smithi144.stdout:(57/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.7 MB/s | 126 kB 00:00 2023-10-06T05:12:11.386 INFO:teuthology.orchestra.run.smithi045.stdout:(50/139): python3-babel-2.5.1-7.el8.noarch.rpm 4.6 MB/s | 4.8 MB 00:01 2023-10-06T05:12:11.411 INFO:teuthology.orchestra.run.smithi045.stdout:(51/139): python3-jsonpointer-1.10-11.el8.noarc 217 kB/s | 20 kB 00:00 2023-10-06T05:12:11.444 INFO:teuthology.orchestra.run.smithi144.stdout:(58/139): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-10-06T05:12:11.470 INFO:teuthology.orchestra.run.smithi045.stdout:(52/139): python3-mako-1.0.6-14.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2023-10-06T05:12:11.496 INFO:teuthology.orchestra.run.smithi045.stdout:(53/139): python3-markupsafe-0.23-19.el8.x86_64 463 kB/s | 39 kB 00:00 2023-10-06T05:12:11.502 INFO:teuthology.orchestra.run.smithi144.stdout:(59/139): python3-numpy-1.14.3-10.el8.x86_64.rp 7.8 MB/s | 3.7 MB 00:00 2023-10-06T05:12:11.519 INFO:teuthology.orchestra.run.smithi144.stdout:(60/139): python3-pytz-2017.2-11.el8.noarch.rpm 759 kB/s | 57 kB 00:00 2023-10-06T05:12:11.579 INFO:teuthology.orchestra.run.smithi045.stdout:(54/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.6 MB/s | 225 kB 00:00 2023-10-06T05:12:11.656 INFO:teuthology.orchestra.run.smithi045.stdout:(55/139): python3-prettytable-0.7.2-14.el8.noar 586 kB/s | 44 kB 00:00 2023-10-06T05:12:11.661 INFO:teuthology.orchestra.run.smithi144.stdout:(61/139): python3-werkzeug-0.12.2-4.el8.noarch. 3.2 MB/s | 457 kB 00:00 2023-10-06T05:12:11.730 INFO:teuthology.orchestra.run.smithi045.stdout:(56/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.4 MB/s | 103 kB 00:00 2023-10-06T05:12:11.762 INFO:teuthology.orchestra.run.smithi144.stdout:(62/139): socat-1.7.4.1-1.el8.x86_64.rpm 3.1 MB/s | 323 kB 00:00 2023-10-06T05:12:11.829 INFO:teuthology.orchestra.run.smithi045.stdout:(57/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.3 MB/s | 126 kB 00:00 2023-10-06T05:12:11.862 INFO:teuthology.orchestra.run.smithi144.stdout:(63/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 707 kB/s | 71 kB 00:00 2023-10-06T05:12:11.914 INFO:teuthology.orchestra.run.smithi045.stdout:(58/139): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-10-06T05:12:11.989 INFO:teuthology.orchestra.run.smithi045.stdout:(59/139): python3-pytz-2017.2-11.el8.noarch.rpm 764 kB/s | 57 kB 00:00 2023-10-06T05:12:12.370 INFO:teuthology.orchestra.run.smithi144.stdout:(64/139): python3-scipy-1.0.0-21.module_el8.5.0 16 MB/s | 14 MB 00:00 2023-10-06T05:12:12.440 INFO:teuthology.orchestra.run.smithi045.stdout:(60/139): python3-numpy-1.14.3-10.el8.x86_64.rp 3.9 MB/s | 3.7 MB 00:00 2023-10-06T05:12:12.521 INFO:teuthology.orchestra.run.smithi144.stdout:(65/139): elfutils-libelf-devel-0.189-3.el8.x86 94 kB/s | 62 kB 00:00 2023-10-06T05:12:12.607 INFO:teuthology.orchestra.run.smithi045.stdout:(61/139): python3-werkzeug-0.12.2-4.el8.noarch. 2.7 MB/s | 457 kB 00:00 2023-10-06T05:12:12.757 INFO:teuthology.orchestra.run.smithi045.stdout:(62/139): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2023-10-06T05:12:13.081 INFO:teuthology.orchestra.run.smithi144.stdout:(66/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 86 kB/s | 48 kB 00:00 2023-10-06T05:12:13.113 INFO:teuthology.orchestra.run.smithi144.stdout:(67/139): fuse-2.9.7-17.el8.x86_64.rpm 112 kB/s | 83 kB 00:00 2023-10-06T05:12:13.274 INFO:teuthology.orchestra.run.smithi045.stdout:(63/139): ceph-test-17.2.6-1274.g68e53a4a.el8.x 5.7 MB/s | 52 MB 00:09 2023-10-06T05:12:13.300 INFO:teuthology.orchestra.run.smithi045.stdout:(64/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 130 kB/s | 71 kB 00:00 2023-10-06T05:12:13.538 INFO:teuthology.orchestra.run.smithi144.stdout:(68/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 44 kB/s | 19 kB 00:00 2023-10-06T05:12:13.664 INFO:teuthology.orchestra.run.smithi144.stdout:(69/139): openblas-0.3.15-4.el8.x86_64.rpm 1.1 MB/s | 4.7 MB 00:04 2023-10-06T05:12:13.917 INFO:teuthology.orchestra.run.smithi045.stdout:(65/139): python3-scipy-1.0.0-21.module_el8.5.0 7.1 MB/s | 14 MB 00:01 2023-10-06T05:12:14.081 INFO:teuthology.orchestra.run.smithi144.stdout:(70/139): libcom_err-devel-1.45.6-5.el8.x86_64. 71 kB/s | 39 kB 00:00 2023-10-06T05:12:14.147 INFO:teuthology.orchestra.run.smithi045.stdout:(66/139): elfutils-libelf-devel-0.189-3.el8.x86 71 kB/s | 62 kB 00:00 2023-10-06T05:12:14.175 INFO:teuthology.orchestra.run.smithi045.stdout:(67/139): fuse-2.9.7-17.el8.x86_64.rpm 95 kB/s | 83 kB 00:00 2023-10-06T05:12:14.423 INFO:teuthology.orchestra.run.smithi144.stdout:(71/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 418 kB/s | 561 kB 00:01 2023-10-06T05:12:14.551 INFO:teuthology.orchestra.run.smithi045.stdout:(68/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 76 kB/s | 48 kB 00:00 2023-10-06T05:12:14.576 INFO:teuthology.orchestra.run.smithi045.stdout:(69/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 47 kB/s | 19 kB 00:00 2023-10-06T05:12:14.948 INFO:teuthology.orchestra.run.smithi144.stdout:(72/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 89 kB/s | 47 kB 00:00 2023-10-06T05:12:15.015 INFO:teuthology.orchestra.run.smithi144.stdout:(73/139): libquadmath-8.5.0-20.el8.x86_64.rpm 185 kB/s | 172 kB 00:00 2023-10-06T05:12:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:(70/139): libcom_err-devel-1.45.6-5.el8.x86_64. 75 kB/s | 39 kB 00:00 2023-10-06T05:12:15.140 INFO:teuthology.orchestra.run.smithi144.stdout:(74/139): libgfortran-8.5.0-20.el8.x86_64.rpm 437 kB/s | 645 kB 00:01 2023-10-06T05:12:15.653 INFO:teuthology.orchestra.run.smithi045.stdout:(71/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 372 kB/s | 561 kB 00:01 2023-10-06T05:12:15.848 INFO:teuthology.orchestra.run.smithi144.stdout:(75/139): libsepol-devel-2.9-3.el8.x86_64.rpm 104 kB/s | 87 kB 00:00 2023-10-06T05:12:15.957 INFO:teuthology.orchestra.run.smithi144.stdout:(76/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 122 kB/s | 100 kB 00:00 2023-10-06T05:12:16.023 INFO:teuthology.orchestra.run.smithi045.stdout:(72/139): libquadmath-8.5.0-20.el8.x86_64.rpm 181 kB/s | 172 kB 00:00 2023-10-06T05:12:16.041 INFO:teuthology.orchestra.run.smithi144.stdout:(77/139): libselinux-devel-2.9-8.el8.x86_64.rpm 183 kB/s | 200 kB 00:01 2023-10-06T05:12:16.085 INFO:teuthology.orchestra.run.smithi045.stdout:(73/139): libgfortran-8.5.0-20.el8.x86_64.rpm 427 kB/s | 645 kB 00:01 2023-10-06T05:12:16.177 INFO:teuthology.orchestra.run.smithi045.stdout:(74/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 89 kB/s | 47 kB 00:00 2023-10-06T05:12:16.266 INFO:teuthology.orchestra.run.smithi144.stdout:(78/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 43 kB/s | 18 kB 00:00 2023-10-06T05:12:16.500 INFO:teuthology.orchestra.run.smithi144.stdout:(79/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 81 kB/s | 44 kB 00:00 2023-10-06T05:12:16.789 INFO:teuthology.orchestra.run.smithi045.stdout:(75/139): libsepol-devel-2.9-3.el8.x86_64.rpm 124 kB/s | 87 kB 00:00 2023-10-06T05:12:16.936 INFO:teuthology.orchestra.run.smithi045.stdout:(76/139): libselinux-devel-2.9-8.el8.x86_64.rpm 219 kB/s | 200 kB 00:00 2023-10-06T05:12:16.961 INFO:teuthology.orchestra.run.smithi045.stdout:(77/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 127 kB/s | 100 kB 00:00 2023-10-06T05:12:17.212 INFO:teuthology.orchestra.run.smithi045.stdout:(78/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 43 kB/s | 18 kB 00:00 2023-10-06T05:12:17.462 INFO:teuthology.orchestra.run.smithi045.stdout:(79/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 83 kB/s | 44 kB 00:00 2023-10-06T05:12:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:(80/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 229 kB/s | 229 kB 00:00 2023-10-06T05:12:17.592 INFO:teuthology.orchestra.run.smithi144.stdout:(81/139): pcre2-devel-10.32-3.el8.x86_64.rpm 456 kB/s | 605 kB 00:01 2023-10-06T05:12:18.092 INFO:teuthology.orchestra.run.smithi144.stdout:(82/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 1.5 MB/s | 3.2 MB 00:02 2023-10-06T05:12:18.470 INFO:teuthology.orchestra.run.smithi045.stdout:(80/139): pcre2-devel-10.32-3.el8.x86_64.rpm 481 kB/s | 605 kB 00:01 2023-10-06T05:12:18.496 INFO:teuthology.orchestra.run.smithi045.stdout:(81/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 222 kB/s | 229 kB 00:01 2023-10-06T05:12:18.509 INFO:teuthology.orchestra.run.smithi144.stdout:(83/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 219 kB/s | 220 kB 00:01 2023-10-06T05:12:18.668 INFO:teuthology.orchestra.run.smithi144.stdout:(84/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 231 kB/s | 249 kB 00:01 2023-10-06T05:12:18.830 INFO:teuthology.orchestra.run.smithi045.stdout:(82/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 1.7 MB/s | 3.2 MB 00:01 2023-10-06T05:12:19.084 INFO:teuthology.orchestra.run.smithi144.stdout:(85/139): python3-jwt-1.6.1-2.el8.noarch.rpm 74 kB/s | 43 kB 00:00 2023-10-06T05:12:19.511 INFO:teuthology.orchestra.run.smithi045.stdout:(83/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 212 kB/s | 220 kB 00:01 2023-10-06T05:12:19.568 INFO:teuthology.orchestra.run.smithi144.stdout:(86/139): python3-cryptography-3.2.1-6.el8.x86_ 434 kB/s | 641 kB 00:01 2023-10-06T05:12:19.572 INFO:teuthology.orchestra.run.smithi045.stdout:(84/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 231 kB/s | 249 kB 00:01 2023-10-06T05:12:19.593 INFO:teuthology.orchestra.run.smithi144.stdout:(87/139): python3-oauthlib-2.1.0-1.el8.noarch.r 167 kB/s | 155 kB 00:00 2023-10-06T05:12:19.877 INFO:teuthology.orchestra.run.smithi144.stdout:(88/139): python3-pycparser-2.14-14.el8.noarch. 138 kB/s | 109 kB 00:00 2023-10-06T05:12:20.030 INFO:teuthology.orchestra.run.smithi045.stdout:(85/139): python3-jwt-1.6.1-2.el8.noarch.rpm 82 kB/s | 43 kB 00:00 2023-10-06T05:12:20.127 INFO:teuthology.orchestra.run.smithi144.stdout:(89/139): python3-requests-oauthlib-1.0.0-1.el8 80 kB/s | 43 kB 00:00 2023-10-06T05:12:20.130 INFO:teuthology.orchestra.run.smithi045.stdout:(86/139): python3-cryptography-3.2.1-6.el8.x86_ 493 kB/s | 641 kB 00:01 2023-10-06T05:12:20.252 INFO:teuthology.orchestra.run.smithi144.stdout:(90/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 113 kB/s | 74 kB 00:00 2023-10-06T05:12:20.394 INFO:teuthology.orchestra.run.smithi144.stdout:(91/139): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-10-06T05:12:20.419 INFO:teuthology.orchestra.run.smithi144.stdout:(92/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 107 kB/s | 58 kB 00:00 2023-10-06T05:12:20.445 INFO:teuthology.orchestra.run.smithi144.stdout:(93/139): liboath-2.6.2-3.el8.x86_64.rpm 306 kB/s | 59 kB 00:00 2023-10-06T05:12:20.470 INFO:teuthology.orchestra.run.smithi144.stdout:(94/139): libunwind-1.3.1-3.el8.x86_64.rpm 992 kB/s | 75 kB 00:00 2023-10-06T05:12:20.497 INFO:teuthology.orchestra.run.smithi045.stdout:(87/139): python3-oauthlib-2.1.0-1.el8.noarch.r 167 kB/s | 155 kB 00:00 2023-10-06T05:12:20.537 INFO:teuthology.orchestra.run.smithi144.stdout:(95/139): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-10-06T05:12:20.596 INFO:teuthology.orchestra.run.smithi144.stdout:(96/139): python3-asyncssh-2.7.0-2.el8.noarch.r 2.6 MB/s | 476 kB 00:00 2023-10-06T05:12:20.645 INFO:teuthology.orchestra.run.smithi144.stdout:(97/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 218 kB/s | 44 kB 00:00 2023-10-06T05:12:20.698 INFO:teuthology.orchestra.run.smithi045.stdout:(88/139): python3-requests-oauthlib-1.0.0-1.el8 81 kB/s | 43 kB 00:00 2023-10-06T05:12:20.699 INFO:teuthology.orchestra.run.smithi144.stdout:(98/139): python3-cachetools-3.1.1-4.el8.noarch 246 kB/s | 33 kB 00:00 2023-10-06T05:12:20.705 INFO:teuthology.orchestra.run.smithi144.stdout:(99/139): python3-certifi-2018.10.15-7.el8.noar 149 kB/s | 16 kB 00:00 2023-10-06T05:12:20.764 INFO:teuthology.orchestra.run.smithi144.stdout:(100/139): python3-google-auth-1.1.1-10.el8.noa 1.4 MB/s | 81 kB 00:00 2023-10-06T05:12:20.789 INFO:teuthology.orchestra.run.smithi144.stdout:(101/139): python3-cheroot-8.5.2-1.el8.noarch.r 1.2 MB/s | 173 kB 00:00 2023-10-06T05:12:20.789 INFO:teuthology.orchestra.run.smithi045.stdout:(89/139): python3-pycparser-2.14-14.el8.noarch. 144 kB/s | 109 kB 00:00 2023-10-06T05:12:20.822 INFO:teuthology.orchestra.run.smithi144.stdout:(102/139): python3-cherrypy-18.4.0-1.el8.noarch 3.2 MB/s | 384 kB 00:00 2023-10-06T05:12:20.848 INFO:teuthology.orchestra.run.smithi144.stdout:(103/139): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-10-06T05:12:20.873 INFO:teuthology.orchestra.run.smithi144.stdout:(104/139): python3-jaraco-functools-2.0-4.el8.n 209 kB/s | 18 kB 00:00 2023-10-06T05:12:20.907 INFO:teuthology.orchestra.run.smithi144.stdout:(105/139): python3-logutils-0.3.5-11.el8.noarch 833 kB/s | 49 kB 00:00 2023-10-06T05:12:20.940 INFO:teuthology.orchestra.run.smithi144.stdout:(106/139): python3-more-itertools-7.2.0-3.el8.n 879 kB/s | 59 kB 00:00 2023-10-06T05:12:20.982 INFO:teuthology.orchestra.run.smithi144.stdout:(107/139): python3-kubernetes-11.0.0-6.el8.noar 7.3 MB/s | 1.2 MB 00:00 2023-10-06T05:12:21.007 INFO:teuthology.orchestra.run.smithi144.stdout:(108/139): python3-natsort-7.1.1-2.el8.noarch.r 600 kB/s | 60 kB 00:00 2023-10-06T05:12:21.041 INFO:teuthology.orchestra.run.smithi144.stdout:(109/139): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2023-10-06T05:12:21.091 INFO:teuthology.orchestra.run.smithi144.stdout:(110/139): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-10-06T05:12:21.117 INFO:teuthology.orchestra.run.smithi144.stdout:(111/139): python3-repoze-lru-0.7-6.el8.noarch. 309 kB/s | 34 kB 00:00 2023-10-06T05:12:21.131 INFO:teuthology.orchestra.run.smithi045.stdout:(90/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 118 kB/s | 74 kB 00:00 2023-10-06T05:12:21.142 INFO:teuthology.orchestra.run.smithi144.stdout:(112/139): python3-routes-2.4.1-12.el8.noarch.r 1.9 MB/s | 196 kB 00:00 2023-10-06T05:12:21.167 INFO:teuthology.orchestra.run.smithi144.stdout:(113/139): python3-rsa-4.9-2.el8.noarch.rpm 849 kB/s | 64 kB 00:00 2023-10-06T05:12:21.193 INFO:teuthology.orchestra.run.smithi144.stdout:(114/139): python3-simplegeneric-0.8.1-17.el8.n 247 kB/s | 19 kB 00:00 2023-10-06T05:12:21.218 INFO:teuthology.orchestra.run.smithi144.stdout:(115/139): python3-singledispatch-3.4.0.3-18.el 317 kB/s | 24 kB 00:00 2023-10-06T05:12:21.243 INFO:teuthology.orchestra.run.smithi144.stdout:(116/139): python3-tempora-1.14.1-5.el8.noarch. 387 kB/s | 29 kB 00:00 2023-10-06T05:12:21.268 INFO:teuthology.orchestra.run.smithi144.stdout:(117/139): python3-trustme-0.6.0-4.el8.noarch.r 364 kB/s | 27 kB 00:00 2023-10-06T05:12:21.294 INFO:teuthology.orchestra.run.smithi144.stdout:(118/139): python3-waitress-1.4.3-1.el8.noarch. 3.3 MB/s | 255 kB 00:00 2023-10-06T05:12:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:(91/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 88 kB/s | 58 kB 00:00 2023-10-06T05:12:21.319 INFO:teuthology.orchestra.run.smithi144.stdout:(119/139): python3-webob-1.8.5-1.el8.1.noarch.r 3.3 MB/s | 251 kB 00:00 2023-10-06T05:12:21.344 INFO:teuthology.orchestra.run.smithi144.stdout:(120/139): python3-websocket-client-0.56.0-5.el 811 kB/s | 61 kB 00:00 2023-10-06T05:12:21.369 INFO:teuthology.orchestra.run.smithi144.stdout:(121/139): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2023-10-06T05:12:21.374 INFO:teuthology.orchestra.run.smithi045.stdout:(92/139): gperftools-libs-2.7-9.el8.x86_64.rpm 524 kB/s | 306 kB 00:00 2023-10-06T05:12:21.395 INFO:teuthology.orchestra.run.smithi144.stdout:(122/139): python3-zc-lockfile-2.0-2.el8.noarch 304 kB/s | 23 kB 00:00 2023-10-06T05:12:21.516 INFO:teuthology.orchestra.run.smithi045.stdout:(93/139): liboath-2.6.2-3.el8.x86_64.rpm 153 kB/s | 59 kB 00:00 2023-10-06T05:12:21.537 INFO:teuthology.orchestra.run.smithi144.stdout:(123/139): thrift-0.13.0-2.el8.x86_64.rpm 9.0 MB/s | 1.7 MB 00:00 2023-10-06T05:12:21.591 INFO:teuthology.orchestra.run.smithi045.stdout:(94/139): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-10-06T05:12:21.667 INFO:teuthology.orchestra.run.smithi045.stdout:(95/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 290 kB/s | 44 kB 00:00 2023-10-06T05:12:21.708 INFO:teuthology.orchestra.run.smithi045.stdout:(96/139): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2023-10-06T05:12:21.734 INFO:teuthology.orchestra.run.smithi045.stdout:(97/139): libunwind-1.3.1-3.el8.x86_64.rpm 179 kB/s | 75 kB 00:00 2023-10-06T05:12:21.834 INFO:teuthology.orchestra.run.smithi045.stdout:(98/139): python3-cachetools-3.1.1-4.el8.noarch 197 kB/s | 33 kB 00:00 2023-10-06T05:12:21.837 INFO:teuthology.orchestra.run.smithi144.stdout:(124/139): librbd1-17.2.6-1274.g68e53a4a.el8.x8 8.0 MB/s | 3.5 MB 00:00 2023-10-06T05:12:21.851 INFO:teuthology.orchestra.run.smithi045.stdout:(99/139): python3-certifi-2018.10.15-7.el8.noar 114 kB/s | 16 kB 00:00 2023-10-06T05:12:21.960 INFO:teuthology.orchestra.run.smithi045.stdout:(100/139): python3-google-auth-1.1.1-10.el8.noa 751 kB/s | 81 kB 00:00 2023-10-06T05:12:21.993 INFO:teuthology.orchestra.run.smithi045.stdout:(101/139): python3-cheroot-8.5.2-1.el8.noarch.r 666 kB/s | 173 kB 00:00 2023-10-06T05:12:22.060 INFO:teuthology.orchestra.run.smithi045.stdout:(102/139): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-10-06T05:12:22.094 INFO:teuthology.orchestra.run.smithi045.stdout:(103/139): python3-jaraco-functools-2.0-4.el8.n 175 kB/s | 18 kB 00:00 2023-10-06T05:12:22.194 INFO:teuthology.orchestra.run.smithi045.stdout:(104/139): python3-cherrypy-18.4.0-1.el8.noarch 1.0 MB/s | 384 kB 00:00 2023-10-06T05:12:22.196 INFO:teuthology.orchestra.run.smithi144.stdout:(125/139): librados2-17.2.6-1274.g68e53a4a.el8. 4.1 MB/s | 3.4 MB 00:00 2023-10-06T05:12:22.211 INFO:teuthology.orchestra.run.smithi045.stdout:(105/139): python3-logutils-0.3.5-11.el8.noarch 416 kB/s | 49 kB 00:00 2023-10-06T05:12:22.278 INFO:teuthology.orchestra.run.smithi045.stdout:(106/139): python3-kubernetes-11.0.0-6.el8.noar 5.4 MB/s | 1.2 MB 00:00 2023-10-06T05:12:22.304 INFO:teuthology.orchestra.run.smithi045.stdout:(107/139): python3-more-itertools-7.2.0-3.el8.n 540 kB/s | 59 kB 00:00 2023-10-06T05:12:22.346 INFO:teuthology.orchestra.run.smithi144.stdout:(126/139): NetworkManager-initscripts-updown-1. 177 kB/s | 143 kB 00:00 2023-10-06T05:12:22.396 INFO:teuthology.orchestra.run.smithi045.stdout:(108/139): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-10-06T05:12:22.412 INFO:teuthology.orchestra.run.smithi045.stdout:(109/139): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-10-06T05:12:22.454 INFO:teuthology.orchestra.run.smithi045.stdout:(110/139): python3-natsort-7.1.1-2.el8.noarch.r 248 kB/s | 60 kB 00:00 2023-10-06T05:12:22.496 INFO:teuthology.orchestra.run.smithi144.stdout:(127/139): elfutils-debuginfod-client-0.189-3.e 115 kB/s | 76 kB 00:00 2023-10-06T05:12:22.505 INFO:teuthology.orchestra.run.smithi045.stdout:(111/139): python3-repoze-lru-0.7-6.el8.noarch. 309 kB/s | 34 kB 00:00 2023-10-06T05:12:22.563 INFO:teuthology.orchestra.run.smithi045.stdout:(112/139): python3-routes-2.4.1-12.el8.noarch.r 1.3 MB/s | 196 kB 00:00 2023-10-06T05:12:22.588 INFO:teuthology.orchestra.run.smithi045.stdout:(113/139): python3-rsa-4.9-2.el8.noarch.rpm 480 kB/s | 64 kB 00:00 2023-10-06T05:12:22.614 INFO:teuthology.orchestra.run.smithi045.stdout:(114/139): python3-simplegeneric-0.8.1-17.el8.n 172 kB/s | 19 kB 00:00 2023-10-06T05:12:22.672 INFO:teuthology.orchestra.run.smithi045.stdout:(115/139): python3-singledispatch-3.4.0.3-18.el 220 kB/s | 24 kB 00:00 2023-10-06T05:12:22.706 INFO:teuthology.orchestra.run.smithi045.stdout:(116/139): python3-tempora-1.14.1-5.el8.noarch. 249 kB/s | 29 kB 00:00 2023-10-06T05:12:22.731 INFO:teuthology.orchestra.run.smithi045.stdout:(117/139): python3-trustme-0.6.0-4.el8.noarch.r 235 kB/s | 27 kB 00:00 2023-10-06T05:12:22.815 INFO:teuthology.orchestra.run.smithi045.stdout:(118/139): python3-waitress-1.4.3-1.el8.noarch. 1.8 MB/s | 255 kB 00:00 2023-10-06T05:12:22.840 INFO:teuthology.orchestra.run.smithi045.stdout:(119/139): python3-websocket-client-0.56.0-5.el 563 kB/s | 61 kB 00:00 2023-10-06T05:12:22.891 INFO:teuthology.orchestra.run.smithi045.stdout:(120/139): python3-webob-1.8.5-1.el8.1.noarch.r 1.3 MB/s | 251 kB 00:00 2023-10-06T05:12:22.941 INFO:teuthology.orchestra.run.smithi045.stdout:(121/139): python3-webtest-2.0.33-1.el8.noarch. 681 kB/s | 85 kB 00:00 2023-10-06T05:12:22.958 INFO:teuthology.orchestra.run.smithi045.stdout:(122/139): python3-zc-lockfile-2.0-2.el8.noarch 196 kB/s | 23 kB 00:00 2023-10-06T05:12:23.258 INFO:teuthology.orchestra.run.smithi045.stdout:(123/139): thrift-0.13.0-2.el8.x86_64.rpm 4.7 MB/s | 1.7 MB 00:00 2023-10-06T05:12:23.263 INFO:teuthology.orchestra.run.smithi144.stdout:(128/139): elfutils-libelf-0.189-3.el8.x86_64.r 217 kB/s | 232 kB 00:01 2023-10-06T05:12:23.359 INFO:teuthology.orchestra.run.smithi045.stdout:(124/139): librados2-17.2.6-1274.g68e53a4a.el8. 8.2 MB/s | 3.4 MB 00:00 2023-10-06T05:12:23.439 INFO:teuthology.orchestra.run.smithi144.stdout:(129/139): elfutils-libs-0.189-3.el8.x86_64.rpm 277 kB/s | 303 kB 00:01 2023-10-06T05:12:23.576 INFO:teuthology.orchestra.run.smithi045.stdout:(125/139): librbd1-17.2.6-1274.g68e53a4a.el8.x8 5.7 MB/s | 3.5 MB 00:00 2023-10-06T05:12:24.001 INFO:teuthology.orchestra.run.smithi045.stdout:(126/139): elfutils-debuginfod-client-0.189-3.e 118 kB/s | 76 kB 00:00 2023-10-06T05:12:24.026 INFO:teuthology.orchestra.run.smithi045.stdout:(127/139): NetworkManager-initscripts-updown-1. 187 kB/s | 143 kB 00:00 2023-10-06T05:12:24.122 INFO:teuthology.orchestra.run.smithi144.stdout:(130/139): iproute-6.2.0-5.el8.x86_64.rpm 525 kB/s | 854 kB 00:01 2023-10-06T05:12:24.306 INFO:teuthology.orchestra.run.smithi144.stdout:(131/139): libblkid-2.32.1-43.el8.x86_64.rpm 212 kB/s | 220 kB 00:01 2023-10-06T05:12:24.473 INFO:teuthology.orchestra.run.smithi144.stdout:(132/139): libfdisk-2.32.1-43.el8.x86_64.rpm 245 kB/s | 254 kB 00:01 2023-10-06T05:12:24.585 INFO:teuthology.orchestra.run.smithi045.stdout:(128/139): elfutils-libelf-0.189-3.el8.x86_64.r 230 kB/s | 232 kB 00:01 2023-10-06T05:12:25.114 INFO:teuthology.orchestra.run.smithi144.stdout:(133/139): libsmartcols-2.32.1-43.el8.x86_64.rp 222 kB/s | 179 kB 00:00 2023-10-06T05:12:25.144 INFO:teuthology.orchestra.run.smithi045.stdout:(129/139): elfutils-libs-0.189-3.el8.x86_64.rpm 265 kB/s | 303 kB 00:01 2023-10-06T05:12:25.148 INFO:teuthology.orchestra.run.smithi144.stdout:(134/139): libmount-2.32.1-43.el8.x86_64.rpm 231 kB/s | 236 kB 00:01 2023-10-06T05:12:25.273 INFO:teuthology.orchestra.run.smithi144.stdout:(135/139): libuuid-2.32.1-43.el8.x86_64.rpm 124 kB/s | 99 kB 00:00 2023-10-06T05:12:25.472 INFO:teuthology.orchestra.run.smithi045.stdout:(130/139): iproute-6.2.0-5.el8.x86_64.rpm 592 kB/s | 854 kB 00:01 2023-10-06T05:12:25.636 INFO:teuthology.orchestra.run.smithi045.stdout:(131/139): libblkid-2.32.1-43.el8.x86_64.rpm 210 kB/s | 220 kB 00:01 2023-10-06T05:12:25.907 INFO:teuthology.orchestra.run.smithi144.stdout:(136/139): util-linux-user-2.32.1-43.el8.x86_64 135 kB/s | 103 kB 00:00 2023-10-06T05:12:26.169 INFO:teuthology.orchestra.run.smithi045.stdout:(132/139): libfdisk-2.32.1-43.el8.x86_64.rpm 247 kB/s | 254 kB 00:01 2023-10-06T05:12:26.466 INFO:teuthology.orchestra.run.smithi144.stdout:(137/139): xfsdump-3.1.8-6.el8.x86_64.rpm 280 kB/s | 334 kB 00:01 2023-10-06T05:12:26.503 INFO:teuthology.orchestra.run.smithi045.stdout:(133/139): libmount-2.32.1-43.el8.x86_64.rpm 229 kB/s | 236 kB 00:01 2023-10-06T05:12:26.537 INFO:teuthology.orchestra.run.smithi045.stdout:(134/139): libsmartcols-2.32.1-43.el8.x86_64.rp 199 kB/s | 179 kB 00:00 2023-10-06T05:12:26.820 INFO:teuthology.orchestra.run.smithi045.stdout:(135/139): libuuid-2.32.1-43.el8.x86_64.rpm 152 kB/s | 99 kB 00:00 2023-10-06T05:12:27.149 INFO:teuthology.orchestra.run.smithi144.stdout:(138/139): util-linux-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 2.5 MB 00:02 2023-10-06T05:12:27.195 INFO:teuthology.orchestra.run.smithi045.stdout:(136/139): util-linux-user-2.32.1-43.el8.x86_64 156 kB/s | 103 kB 00:00 2023-10-06T05:12:27.507 INFO:teuthology.orchestra.run.smithi144.stdout:(139/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 692 kB/s | 1.1 MB 00:01 2023-10-06T05:12:27.511 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2023-10-06T05:12:27.511 INFO:teuthology.orchestra.run.smithi144.stdout:Total 5.6 MB/s | 209 MB 00:37 2023-10-06T05:12:27.854 INFO:teuthology.orchestra.run.smithi045.stdout:(137/139): xfsdump-3.1.8-6.el8.x86_64.rpm 323 kB/s | 334 kB 00:01 2023-10-06T05:12:28.262 INFO:teuthology.orchestra.run.smithi045.stdout:(138/139): util-linux-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 2.5 MB 00:01 2023-10-06T05:12:28.366 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2023-10-06T05:12:28.538 INFO:teuthology.orchestra.run.smithi045.stdout:(139/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 826 kB/s | 1.1 MB 00:01 2023-10-06T05:12:28.541 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-10-06T05:12:28.542 INFO:teuthology.orchestra.run.smithi045.stdout:Total 6.5 MB/s | 209 MB 00:32 2023-10-06T05:12:28.797 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2023-10-06T05:12:28.797 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2023-10-06T05:12:29.419 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-10-06T05:12:29.902 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-10-06T05:12:29.902 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-10-06T05:12:30.206 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2023-10-06T05:12:30.209 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2023-10-06T05:12:31.428 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-10-06T05:12:31.432 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-10-06T05:12:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2023-10-06T05:12:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-06T05:12:35.785 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:12:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:12:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:12:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:12:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-06T05:12:36.631 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-06T05:12:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-06T05:12:36.899 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-10-06T05:12:36.954 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-06T05:12:37.077 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-06T05:12:37.127 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:12:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:12:37.235 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:12:37.410 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:12:37.473 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:12:37.637 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:12:37.659 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-06T05:12:37.860 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:12:38.074 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-06T05:12:38.250 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:12:38.257 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-06T05:12:38.435 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:12:38.447 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-06T05:12:38.560 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:12:38.652 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:12:38.760 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:12:38.856 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:12:39.072 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:12:39.074 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:12:39.285 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:12:39.293 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:12:39.723 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:12:39.838 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:12:39.910 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:12:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:12:40.050 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:12:40.239 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:12:40.260 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:12:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:12:40.498 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:12:40.572 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:12:40.604 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:12:40.735 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:12:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-06T05:12:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-06T05:12:41.118 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:12:41.206 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:12:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:12:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:12:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:12:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:12:41.665 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-06T05:12:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:12:41.827 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-06T05:12:41.976 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:12:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:12:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:12:42.135 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:12:42.267 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:12:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-06T05:12:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-06T05:12:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:12:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:12:42.853 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:12:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:12:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:12:43.076 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:12:43.242 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-06T05:12:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-06T05:12:43.559 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:12:43.690 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:12:43.699 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:12:43.876 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:12:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:12:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:12:44.604 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:12:44.768 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:12:44.929 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-06T05:12:45.148 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-06T05:12:45.305 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-06T05:12:45.449 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:12:45.553 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-06T05:12:45.618 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:12:45.757 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 29/155 2023-10-06T05:12:45.970 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 30/155 2023-10-06T05:12:46.185 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 31/155 2023-10-06T05:12:46.393 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:12:46.877 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-06T05:12:47.188 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-06T05:12:47.220 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-06T05:12:47.409 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-06T05:12:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-06T05:12:47.621 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-06T05:12:47.839 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-06T05:12:47.850 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 29/155 2023-10-06T05:12:48.014 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 30/155 2023-10-06T05:12:48.033 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:12:48.252 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 31/155 2023-10-06T05:12:49.276 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-06T05:12:49.644 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-06T05:12:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:12:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-06T05:12:50.137 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-06T05:12:50.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:12:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-06T05:12:50.544 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-06T05:12:50.759 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/155 2023-10-06T05:12:50.979 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 40/155 2023-10-06T05:12:51.233 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 41/155 2023-10-06T05:12:51.458 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-06T05:12:51.658 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-06T05:12:51.891 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:12:51.973 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:12:52.166 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:12:52.266 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:12:52.269 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:12:52.395 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-06T05:12:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:12:52.589 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-06T05:12:52.802 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-06T05:12:52.980 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-06T05:12:53.125 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:12:53.222 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/155 2023-10-06T05:12:53.383 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:12:53.398 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:12:53.399 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 40/155 2023-10-06T05:12:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-06T05:12:53.681 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 41/155 2023-10-06T05:12:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-06T05:12:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-06T05:12:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-06T05:12:54.118 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-06T05:12:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-06T05:12:54.341 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:12:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:12:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-06T05:12:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-06T05:12:54.624 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:12:54.725 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:12:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-06T05:12:54.870 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-06T05:12:55.053 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-06T05:12:55.058 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:12:55.244 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-06T05:12:55.405 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-06T05:12:55.609 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:12:55.611 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-06T05:12:55.763 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-06T05:12:55.858 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:12:55.877 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:12:55.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-06T05:12:56.050 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-06T05:12:56.139 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-06T05:12:56.341 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-06T05:12:56.351 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-06T05:12:56.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-06T05:12:56.607 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-06T05:12:56.730 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-06T05:12:56.826 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-06T05:12:56.879 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-06T05:12:57.098 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-06T05:12:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-06T05:12:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-06T05:12:57.457 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-06T05:12:57.514 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-06T05:12:57.665 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-06T05:12:57.669 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-06T05:12:57.856 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-06T05:12:57.929 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-06T05:12:58.013 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-06T05:12:58.152 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-06T05:12:58.214 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-06T05:12:58.358 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-06T05:12:58.555 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-06T05:12:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-06T05:12:58.814 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-06T05:12:58.835 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-06T05:12:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-06T05:12:59.095 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-06T05:12:59.338 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-06T05:12:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-06T05:12:59.578 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-06T05:12:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-06T05:12:59.901 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-06T05:13:00.120 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-06T05:13:00.319 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-06T05:13:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-06T05:13:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-06T05:13:01.048 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-06T05:13:01.050 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-06T05:13:01.314 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-06T05:13:01.348 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-06T05:13:01.550 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-06T05:13:01.556 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-06T05:13:01.861 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-06T05:13:01.883 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-06T05:13:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-06T05:13:02.140 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-06T05:13:02.400 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-06T05:13:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:13:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-06T05:13:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-06T05:13:03.260 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-06T05:13:03.349 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-06T05:13:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-06T05:13:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-06T05:13:03.833 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-06T05:13:03.856 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-06T05:13:04.084 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-06T05:13:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-06T05:13:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-06T05:13:04.438 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-06T05:13:04.449 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-06T05:13:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-06T05:13:04.682 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-06T05:13:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-06T05:13:04.903 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-06T05:13:05.127 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-06T05:13:05.145 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:13:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-06T05:13:05.460 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-06T05:13:06.183 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-06T05:13:06.184 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-06T05:13:06.264 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-06T05:13:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-06T05:13:06.765 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-06T05:13:06.986 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-06T05:13:07.381 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-06T05:13:07.385 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-06T05:13:07.626 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-06T05:13:07.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-06T05:13:08.083 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-06T05:13:08.231 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:13:08.585 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-06T05:13:08.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-06T05:13:08.939 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:13:09.526 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:13:09.790 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:13:09.895 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:13:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:13:11.433 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:13:12.047 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:13:12.697 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:13:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:13:13.172 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:13:13.338 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /dev 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /run 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2023-10-06T05:13:28.873 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2023-10-06T05:13:28.874 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2023-10-06T05:13:28.874 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /dev 2023-10-06T05:13:28.874 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:13:29.226 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:13:30.705 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:13:30.929 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:13:31.149 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:13:31.334 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:13:31.724 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:13:31.901 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:13:31.925 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:13:32.124 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-06T05:13:32.320 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-06T05:13:32.500 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:13:32.584 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:13:32.833 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-06T05:13:33.050 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-06T05:13:33.580 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:13:33.796 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:13:33.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 110/155 2023-10-06T05:13:34.111 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 111/155 2023-10-06T05:13:34.317 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:13:34.527 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:13:34.727 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /run 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2023-10-06T05:13:34.732 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-10-06T05:13:34.733 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:13:35.119 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:13:35.695 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:13:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:13:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:13:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 115/155 2023-10-06T05:13:36.660 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:13:36.697 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:13:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:13:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:13:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:13:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:13:37.105 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 118/155 2023-10-06T05:13:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:13:37.249 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:13:37.385 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:13:37.817 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:13:38.010 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:13:38.040 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:13:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-06T05:13:38.557 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-06T05:13:38.718 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:13:38.805 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:13:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-06T05:13:39.485 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-06T05:13:39.995 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:13:40.191 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:13:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 110/155 2023-10-06T05:13:40.509 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 111/155 2023-10-06T05:13:40.732 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:13:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:13:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:13:42.132 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:13:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:13:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:13:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:13:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 115/155 2023-10-06T05:13:43.130 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:13:43.172 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:13:43.350 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:13:43.365 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:13:43.600 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 118/155 2023-10-06T05:13:43.784 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:13:43.851 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 120/155 2023-10-06T05:13:44.084 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:13:44.206 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:13:44.373 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:13:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:13:44.607 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:13:44.953 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:13:45.140 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:13:45.236 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:13:46.129 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 125/155 2023-10-06T05:13:46.444 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-06T05:13:46.687 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-06T05:13:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-06T05:13:47.135 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-06T05:13:47.377 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 130/155 2023-10-06T05:13:47.628 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-06T05:13:47.816 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 132/155 2023-10-06T05:13:48.031 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 133/155 2023-10-06T05:13:48.322 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 134/155 2023-10-06T05:13:48.616 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-06T05:13:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:13:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:13:49.087 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-06T05:13:49.184 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:13:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:13:49.411 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:13:49.622 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:13:49.636 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:13:49.840 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-06T05:13:49.996 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-06T05:13:50.168 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-06T05:13:50.168 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:13:50.417 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:13:50.417 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:13:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:13:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:13:50.776 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:13:50.776 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:13:51.037 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:13:51.059 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 120/155 2023-10-06T05:13:51.234 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-06T05:13:51.270 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:13:51.362 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:13:51.422 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-06T05:13:51.566 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:13:51.615 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:13:51.632 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:13:51.632 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:13:51.775 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:13:51.883 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:13:51.883 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:13:52.100 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:13:52.100 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:13:52.160 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:13:52.333 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:13:52.333 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:13:52.341 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:13:52.447 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:13:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:13:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:13:52.750 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:13:52.923 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:13:53.451 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 125/155 2023-10-06T05:13:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:13:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/155 2023-10-06T05:13:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/155 2023-10-06T05:13:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/155 2023-10-06T05:13:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/155 2023-10-06T05:13:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/155 2023-10-06T05:13:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/155 2023-10-06T05:13:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/155 2023-10-06T05:13:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:13:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:13:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:13:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/155 2023-10-06T05:13:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/155 2023-10-06T05:13:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/155 2023-10-06T05:13:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/155 2023-10-06T05:13:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/155 2023-10-06T05:13:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/155 2023-10-06T05:13:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/155 2023-10-06T05:13:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/155 2023-10-06T05:13:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/155 2023-10-06T05:13:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/155 2023-10-06T05:13:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/155 2023-10-06T05:13:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/155 2023-10-06T05:13:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-06T05:13:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-06T05:13:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-06T05:13:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-06T05:13:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-06T05:13:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-06T05:13:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-06T05:13:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-06T05:13:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:13:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-06T05:13:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 126/155 2023-10-06T05:13:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-06T05:13:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-06T05:13:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-06T05:13:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-06T05:13:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-06T05:13:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-06T05:13:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-06T05:13:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-06T05:13:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-06T05:13:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-06T05:13:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-06T05:13:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-06T05:13:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-06T05:13:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-06T05:13:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-06T05:13:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-06T05:13:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-06T05:13:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-06T05:13:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-06T05:13:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-06T05:13:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-06T05:13:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-06T05:13:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:13:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-06T05:13:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:13:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-06T05:13:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-06T05:13:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-06T05:13:53.755 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-06T05:13:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-06T05:13:54.179 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-06T05:13:54.345 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-06T05:13:54.638 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 130/155 2023-10-06T05:13:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-06T05:13:55.076 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 132/155 2023-10-06T05:13:55.259 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 133/155 2023-10-06T05:13:55.572 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 134/155 2023-10-06T05:13:55.727 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-06T05:13:55.727 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:13:55.727 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2023-10-06T05:13:55.728 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-06T05:13:55.728 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-06T05:13:55.728 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-06T05:13:55.728 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-06T05:13:55.728 INFO:teuthology.orchestra.run.smithi144.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-06T05:13:55.728 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.728 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-06T05:13:55.729 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.1-8.el8.noarch 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.730 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.731 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-7.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-06T05:13:55.733 INFO:teuthology.orchestra.run.smithi144.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-06T05:13:55.734 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-06T05:13:55.735 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T05:13:55.736 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T05:13:55.737 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T05:13:55.738 INFO:teuthology.orchestra.run.smithi144.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:13:55.739 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T05:13:55.740 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T05:13:55.740 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-06T05:13:55.740 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T05:13:55.740 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-06T05:13:55.740 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:13:55.740 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2023-10-06T05:13:55.868 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-06T05:13:56.012 DEBUG:teuthology.parallel:result is None 2023-10-06T05:13:56.081 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:13:56.095 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:13:56.288 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-06T05:13:56.525 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:13:56.613 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:13:56.849 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:13:56.863 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:13:57.025 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-06T05:13:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-06T05:13:57.395 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-06T05:13:57.395 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:13:57.668 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:13:57.669 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:13:57.884 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:13:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:13:58.031 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:13:58.031 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:13:58.264 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:13:58.467 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-06T05:13:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-06T05:13:58.859 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:13:58.859 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:13:59.068 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:13:59.068 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:13:59.293 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:13:59.293 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:13:59.485 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:13:59.485 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:13:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:13:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:13:59.909 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:14:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:14:00.721 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:14:00.721 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/155 2023-10-06T05:14:00.721 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/155 2023-10-06T05:14:00.721 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/155 2023-10-06T05:14:00.722 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/155 2023-10-06T05:14:00.723 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/155 2023-10-06T05:14:00.724 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/155 2023-10-06T05:14:00.734 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/155 2023-10-06T05:14:00.734 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/155 2023-10-06T05:14:00.734 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/155 2023-10-06T05:14:00.734 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/155 2023-10-06T05:14:00.734 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-06T05:14:00.735 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-06T05:14:00.736 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-06T05:14:00.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:14:00.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-06T05:14:00.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-06T05:14:00.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-06T05:14:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 126/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-06T05:14:00.742 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-06T05:14:00.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-06T05:14:00.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-06T05:14:03.016 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-06T05:14:03.016 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:03.016 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2023-10-06T05:14:03.016 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-06T05:14:03.016 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-06T05:14:03.016 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-06T05:14:03.016 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-06T05:14:03.017 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.1-8.el8.noarch 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.018 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: jq-1.6-7.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-06T05:14:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.021 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T05:14:03.022 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T05:14:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T05:14:03.024 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T05:14:03.025 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T05:14:03.026 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T05:14:03.027 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T05:14:03.028 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T05:14:03.029 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-06T05:14:03.030 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:03.030 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-10-06T05:14:03.234 DEBUG:teuthology.parallel:result is None 2023-10-06T05:14:03.235 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:14:03.490 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:14:03.521 INFO:teuthology.orchestra.run.smithi045.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T05:14:03.521 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T05:14:03.522 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T05:14:03.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:14:03.741 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:14:03.770 INFO:teuthology.orchestra.run.smithi144.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T05:14:03.770 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T05:14:03.770 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T05:14:03.770 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T05:14:03.771 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:03.771 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T05:14:03.810 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:03.810 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T05:14:03.845 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T05:14:03.845 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:03.846 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T05:14:03.885 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T05:14:03.957 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:03.957 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T05:14:03.989 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T05:14:04.060 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T05:14:04.060 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:04.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T05:14:04.094 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T05:14:04.167 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:04.167 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T05:14:04.198 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T05:14:04.268 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T05:14:04.268 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:04.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T05:14:04.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T05:14:04.379 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:04.379 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T05:14:04.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T05:14:04.480 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T05:14:04.542 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T05:14:04.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2023-10-06T05:14:04.545 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2023-10-06T05:14:04.577 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T05:14:04.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T05:14:04.621 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T05:14:04.651 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T05:14:04.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T05:14:04.694 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T05:14:04.731 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T05:14:04.731 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T05:14:04.731 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T05:14:04.732 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:04.732 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T05:14:04.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:04.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:04.949 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:04.949 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T05:14:04.970 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:05.089 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:05.124 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T05:14:05.124 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T05:14:05.124 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412322', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKBa3lfTt2DZu9P4yZRPDyvyQf+QSD5SAI+sU0pgiHdmC0MYDM0qgHDrd8YHKd1B8IXkPBZAihL7kQmYTYjiR1Y=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIWK9WT+5apjDhQm97J+fYPfqyp6XWshyveO/zIdWuVysyXRHbrCqwG5j7SWBKH3wdTQQOfl3zYtPbrFPgBzFrQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T05:14:05.125 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T05:14:05.127 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T05:14:05.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T05:14:05.189 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T05:14:05.245 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:05.246 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T05:14:05.284 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T05:14:05.285 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2023-10-06T05:14:05.347 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T05:14:05.347 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:05.347 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 386848 Links: 1 2023-10-06T05:14:05.347 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:05.348 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:05.348 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-06 05:14:00.329408499 +0000 2023-10-06T05:14:05.348 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-06 05:10:33.674302588 +0000 2023-10-06T05:14:05.348 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-06 05:10:33.674302588 +0000 2023-10-06T05:14:05.348 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-10-06T05:14:05.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T05:14:05.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-06T05:14:05.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-06T05:14:05.420 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000123108 s, 4.2 MB/s 2023-10-06T05:14:05.421 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T05:14:05.486 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 395489 Links: 1 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-06 05:14:00.329408499 +0000 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-06 05:10:34.268293665 +0000 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-06 05:10:34.268293665 +0000 2023-10-06T05:14:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-10-06T05:14:05.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T05:14:05.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-06T05:14:05.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-06T05:14:05.624 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000106457 s, 4.8 MB/s 2023-10-06T05:14:05.625 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T05:14:05.690 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2023-10-06T05:14:05.753 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T05:14:05.753 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:05.753 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 395663 Links: 1 2023-10-06T05:14:05.753 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:05.753 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:05.753 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-06 05:14:00.329408499 +0000 2023-10-06T05:14:05.753 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-06 05:10:34.855284848 +0000 2023-10-06T05:14:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-06 05:10:34.855284848 +0000 2023-10-06T05:14:05.754 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-10-06T05:14:05.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T05:14:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-06T05:14:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-06T05:14:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000113794 s, 4.5 MB/s 2023-10-06T05:14:05.827 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T05:14:05.892 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2023-10-06T05:14:05.954 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T05:14:05.954 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:05.954 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 395746 Links: 1 2023-10-06T05:14:05.955 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:05.955 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:05.955 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-06 05:14:00.329408499 +0000 2023-10-06T05:14:05.955 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-06 05:10:35.444276000 +0000 2023-10-06T05:14:05.955 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-06 05:10:35.444276000 +0000 2023-10-06T05:14:05.955 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-10-06T05:14:05.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T05:14:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-10-06T05:14:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-10-06T05:14:06.027 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000116434 s, 4.4 MB/s 2023-10-06T05:14:06.028 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T05:14:06.093 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:06.093 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:06.093 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T05:14:06.114 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T05:14:06.115 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2023-10-06T05:14:06.175 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T05:14:06.175 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 390688 Links: 1 2023-10-06T05:14:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-06 05:13:53.141106434 +0000 2023-10-06T05:14:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-06 05:09:50.268204895 +0000 2023-10-06T05:14:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-06 05:09:50.268204895 +0000 2023-10-06T05:14:06.176 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-10-06T05:14:06.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T05:14:06.244 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-06T05:14:06.244 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-06T05:14:06.245 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000242754 s, 2.1 MB/s 2023-10-06T05:14:06.245 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T05:14:06.307 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2023-10-06T05:14:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 398468 Links: 1 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-06 05:13:53.141106434 +0000 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-06 05:09:50.791196572 +0000 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-06 05:09:50.791196572 +0000 2023-10-06T05:14:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-10-06T05:14:06.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T05:14:06.435 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-06T05:14:06.435 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-06T05:14:06.436 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000204873 s, 2.5 MB/s 2023-10-06T05:14:06.436 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T05:14:06.497 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2023-10-06T05:14:06.556 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T05:14:06.556 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:06.556 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 390805 Links: 1 2023-10-06T05:14:06.556 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:06.556 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:06.556 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-06 05:13:53.141106434 +0000 2023-10-06T05:14:06.556 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-06 05:09:51.309188329 +0000 2023-10-06T05:14:06.557 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-06 05:09:51.309188329 +0000 2023-10-06T05:14:06.557 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-10-06T05:14:06.557 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T05:14:06.622 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-06T05:14:06.622 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-06T05:14:06.623 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000231682 s, 2.2 MB/s 2023-10-06T05:14:06.624 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T05:14:06.683 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2023-10-06T05:14:06.741 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T05:14:06.741 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:14:06.742 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 397527 Links: 1 2023-10-06T05:14:06.742 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:14:06.742 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:14:06.742 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-06 05:13:53.141106434 +0000 2023-10-06T05:14:06.742 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-06 05:09:51.826180102 +0000 2023-10-06T05:14:06.742 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-06 05:09:51.826180102 +0000 2023-10-06T05:14:06.742 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-10-06T05:14:06.742 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T05:14:06.807 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-06T05:14:06.807 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-06T05:14:06.807 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000212969 s, 2.4 MB/s 2023-10-06T05:14:06.808 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T05:14:06.867 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:06.867 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T05:14:06.867 INFO:tasks.ceph:Generating config... 2023-10-06T05:14:06.868 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T05:14:06.868 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T05:14:06.868 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T05:14:06.869 INFO:tasks.ceph:[client] pid file = 2023-10-06T05:14:06.869 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T05:14:06.869 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T05:14:06.869 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T05:14:06.870 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T05:14:06.870 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T05:14:06.870 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T05:14:06.870 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T05:14:06.870 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T05:14:06.870 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T05:14:06.870 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T05:14:06.870 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T05:14:06.870 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-06T05:14:06.870 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-06T05:14:06.870 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T05:14:06.871 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T05:14:06.871 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T05:14:06.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T05:14:06.938 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T05:14:06.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T05:14:06.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:14:07.025 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.45'), ('mon.b', '172.21.15.144'), ('mon.c', '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]')] 2023-10-06T05:14:07.025 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.45,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T05:14:07.026 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:07.026 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T05:14:07.089 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.45 --add b 172.21.15.144 --addv c '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:07.184 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:07.184 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: generated fsid 158572ea-864f-4150-9c7e-6d3b4fe3c21a 2023-10-06T05:14:07.184 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = octopus 2023-10-06T05:14:07.184 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 0 2023-10-06T05:14:07.184 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 158572ea-864f-4150-9c7e-6d3b4fe3c21a 2023-10-06T05:14:07.184 INFO:teuthology.orchestra.run.smithi045.stdout:last_changed 2023-10-06T05:14:07.183574+0000 2023-10-06T05:14:07.184 INFO:teuthology.orchestra.run.smithi045.stdout:created 2023-10-06T05:14:07.183574+0000 2023-10-06T05:14:07.185 INFO:teuthology.orchestra.run.smithi045.stdout:min_mon_release 15 (octopus) 2023-10-06T05:14:07.185 INFO:teuthology.orchestra.run.smithi045.stdout:election_strategy: 1 2023-10-06T05:14:07.185 INFO:teuthology.orchestra.run.smithi045.stdout:0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2023-10-06T05:14:07.185 INFO:teuthology.orchestra.run.smithi045.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2023-10-06T05:14:07.185 INFO:teuthology.orchestra.run.smithi045.stdout:2: [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] mon.c 2023-10-06T05:14:07.185 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T05:14:07.186 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T05:14:07.207 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 158572ea-864f-4150-9c7e-6d3b4fe3c21a... 2023-10-06T05:14:07.208 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:14:07.249 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:14:07.295 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2023-10-06T05:14:07.296 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.297 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2023-10-06T05:14:07.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2023-10-06T05:14:07.299 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.45,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 158572ea-864f-4150-9c7e-6d3b4fe3c21a 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.300 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2023-10-06T05:14:07.301 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2023-10-06T05:14:07.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 20 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 20 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2023-10-06T05:14:07.303 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 10 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-10-06T05:14:07.304 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-10-06T05:14:07.305 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:14:07.306 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2023-10-06T05:14:07.307 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-10-06T05:14:07.308 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2023-10-06T05:14:07.309 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2023-10-06T05:14:07.310 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2023-10-06T05:14:07.310 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2023-10-06T05:14:07.310 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:14:07.310 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2023-10-06T05:14:07.310 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.310 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.311 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:14:07.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2023-10-06T05:14:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 158572ea-864f-4150-9c7e-6d3b4fe3c21a 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2023-10-06T05:14:07.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2023-10-06T05:14:07.316 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2023-10-06T05:14:07.317 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2023-10-06T05:14:07.318 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:14:07.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-10-06T05:14:07.320 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-10-06T05:14:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2023-10-06T05:14:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2023-10-06T05:14:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2023-10-06T05:14:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = True 2023-10-06T05:14:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2023-10-06T05:14:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2023-10-06T05:14:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2023-10-06T05:14:07.323 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T05:14:07.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T05:14:07.429 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T05:14:07.430 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:07.430 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T05:14:07.451 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:07.451 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T05:14:07.514 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:14:07.514 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:07.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T05:14:07.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:14:07.601 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:07.601 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:07.664 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:14:07.664 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:07.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T05:14:07.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:14:07.702 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:07.702 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:07.760 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T05:14:07.760 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T05:14:07.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T05:14:07.824 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T05:14:07.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T05:14:07.885 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T05:14:07.888 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T05:14:07.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T05:14:07.950 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T05:14:07.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T05:14:07.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T05:14:08.087 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T05:14:08.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T05:14:08.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T05:14:08.172 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T05:14:08.175 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T05:14:08.201 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T05:14:08.291 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T05:14:08.293 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T05:14:08.321 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T05:14:08.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T05:14:08.373 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T05:14:08.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T05:14:08.437 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T05:14:08.452 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T05:14:08.492 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T05:14:08.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T05:14:08.585 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T05:14:08.597 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T05:14:08.598 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T05:14:08.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T05:14:08.633 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:08.633 INFO:tasks.ceph:role: osd.0 2023-10-06T05:14:08.633 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:14:08.633 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T05:14:08.708 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:08.708 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:08.708 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:08.709 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:08.709 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:08.709 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:08.709 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:08.709 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:08.709 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:08.709 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:08.731 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-10-06T05:14:08.733 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-10-06T05:14:08.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T05:14:08.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T05:14:08.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T05:14:08.935 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:08.936 INFO:tasks.ceph:role: osd.1 2023-10-06T05:14:08.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:14:08.936 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T05:14:09.010 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:09.011 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:09.033 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-10-06T05:14:09.036 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-10-06T05:14:09.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T05:14:09.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T05:14:09.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T05:14:09.235 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:09.235 INFO:tasks.ceph:role: osd.2 2023-10-06T05:14:09.235 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:14:09.235 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T05:14:09.310 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:09.310 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:09.310 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:09.311 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:09.311 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:09.311 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:09.311 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:09.311 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:09.311 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:09.311 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:09.333 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-10-06T05:14:09.335 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-10-06T05:14:09.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T05:14:09.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T05:14:09.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T05:14:09.537 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:09.537 INFO:tasks.ceph:role: osd.3 2023-10-06T05:14:09.537 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:14:09.537 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:09.613 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:09.614 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:09.614 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:09.636 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-10-06T05:14:09.638 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-10-06T05:14:09.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T05:14:09.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T05:14:09.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:10.074+0000 7f17714f4540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T05:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:10.075+0000 7f17714f4540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T05:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:10.075+0000 7f17714f4540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:10.075+0000 7f17714f4540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:10.075+0000 7f17714f4540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:10.075+0000 7f17714f4540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T05:14:10.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:10.599+0000 7f17714f4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:11.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:11.361+0000 7f17714f4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:12.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:12.125+0000 7f17714f4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:12.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T05:14:12.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:13.058+0000 7fec823d5540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T05:14:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:13.058+0000 7fec823d5540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T05:14:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:13.058+0000 7fec823d5540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:14:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:13.058+0000 7fec823d5540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:14:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:13.058+0000 7fec823d5540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:14:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:13.058+0000 7fec823d5540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T05:14:13.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:13.589+0000 7fec823d5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:14.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:14.351+0000 7fec823d5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:15.115+0000 7fec823d5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:15.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T05:14:15.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:16.051+0000 7f5c7e080540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T05:14:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:16.051+0000 7f5c7e080540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T05:14:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:16.051+0000 7f5c7e080540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:14:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:16.051+0000 7f5c7e080540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:14:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:16.051+0000 7f5c7e080540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:14:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:16.052+0000 7f5c7e080540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T05:14:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:16.575+0000 7f5c7e080540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:17.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:17.337+0000 7f5c7e080540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:18.103+0000 7f5c7e080540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:18.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T05:14:18.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:19.031+0000 7f8b0dc04540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T05:14:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:19.031+0000 7f8b0dc04540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T05:14:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:19.031+0000 7f8b0dc04540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:14:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:19.031+0000 7f8b0dc04540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:14:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:19.031+0000 7f8b0dc04540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:14:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:19.031+0000 7f8b0dc04540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T05:14:19.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:19.556+0000 7f8b0dc04540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:20.317+0000 7f8b0dc04540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:21.080+0000 7f8b0dc04540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:21.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T05:14:21.911 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T05:14:21.947 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:21.947 INFO:tasks.ceph:role: osd.4 2023-10-06T05:14:21.948 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:14:21.948 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T05:14:22.026 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:22.026 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:22.026 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:22.026 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:22.027 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:22.027 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:22.027 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:22.027 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:22.027 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:22.027 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:22.449 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-06T05:14:22.451 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-06T05:14:22.451 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T05:14:22.636 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T05:14:22.673 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T05:14:22.743 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:22.743 INFO:tasks.ceph:role: osd.5 2023-10-06T05:14:22.744 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:14:22.744 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:22.821 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:22.822 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:23.246 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-06T05:14:23.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-06T05:14:23.248 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T05:14:23.361 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T05:14:23.395 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T05:14:23.464 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:23.464 INFO:tasks.ceph:role: osd.6 2023-10-06T05:14:23.464 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:14:23.465 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T05:14:23.539 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:23.540 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:23.986 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-06T05:14:23.987 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-06T05:14:23.988 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T05:14:24.166 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T05:14:24.198 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T05:14:24.266 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:14:24.266 INFO:tasks.ceph:role: osd.7 2023-10-06T05:14:24.266 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:14:24.266 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T05:14:24.341 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:14:24.341 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:14:24.341 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:14:24.341 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:14:24.341 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:14:24.342 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:14:24.342 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:14:24.342 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:14:24.342 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:14:24.342 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:14:24.788 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-06T05:14:24.789 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-06T05:14:24.789 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T05:14:24.958 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T05:14:24.991 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:25.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:25.081+0000 7f53d05dd540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T05:14:25.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:25.081+0000 7f53d05dd540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T05:14:25.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:25.081+0000 7f53d05dd540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:14:25.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:25.081+0000 7f53d05dd540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:14:25.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:25.081+0000 7f53d05dd540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:14:25.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:25.082+0000 7f53d05dd540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T05:14:25.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:25.607+0000 7f53d05dd540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:26.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:26.377+0000 7f53d05dd540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:27.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:27.140+0000 7f53d05dd540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:27.936 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T05:14:27.969 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:28.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:28.057+0000 7f7f22a48540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T05:14:28.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:28.057+0000 7f7f22a48540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T05:14:28.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:28.057+0000 7f7f22a48540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:14:28.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:28.057+0000 7f7f22a48540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:14:28.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:28.057+0000 7f7f22a48540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:14:28.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:28.058+0000 7f7f22a48540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T05:14:28.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:28.583+0000 7f7f22a48540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:29.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:29.352+0000 7f7f22a48540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:30.116 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:30.114+0000 7f7f22a48540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:30.914 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T05:14:30.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:31.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:31.034+0000 7f8fe7dea540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T05:14:31.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:31.034+0000 7f8fe7dea540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T05:14:31.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:31.034+0000 7f8fe7dea540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:14:31.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:31.034+0000 7f8fe7dea540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:14:31.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:31.034+0000 7f8fe7dea540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:14:31.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:31.035+0000 7f8fe7dea540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T05:14:31.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:31.561+0000 7f8fe7dea540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:32.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:32.325+0000 7f8fe7dea540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:33.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:33.091+0000 7f8fe7dea540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:33.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T05:14:33.925 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:34.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:34.014+0000 7faf7e6b9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T05:14:34.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:34.015+0000 7faf7e6b9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T05:14:34.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:34.015+0000 7faf7e6b9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:14:34.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:34.015+0000 7faf7e6b9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:14:34.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:34.015+0000 7faf7e6b9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:14:34.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:34.015+0000 7faf7e6b9540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T05:14:34.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:34.540+0000 7faf7e6b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:35.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:35.129 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:14:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:35.302+0000 7faf7e6b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:36.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:36.067+0000 7faf7e6b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:14:36.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T05:14:36.893 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T05:14:36.893 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:36.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T05:14:36.928 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:36.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T05:14:37.003 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:37.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T05:14:37.078 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:37.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T05:14:37.153 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:37.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T05:14:37.229 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:37.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T05:14:37.303 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:37.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T05:14:37.378 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.379 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T05:14:37.411 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.411 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T05:14:37.482 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T05:14:37.553 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.553 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T05:14:37.622 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.622 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T05:14:37.691 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.691 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T05:14:37.760 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.760 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T05:14:37.827 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:37.827 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T05:14:37.851 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:37.851 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T05:14:37.913 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.914 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T05:14:37.931 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:37.932 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T05:14:37.989 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T05:14:37.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T05:14:37.992 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T05:14:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2023-10-06T05:14:38.032 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg== 2023-10-06T05:14:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2023-10-06T05:14:38.032 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw== 2023-10-06T05:14:38.032 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg== 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg== 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg== 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w== 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw== 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A== 2023-10-06T05:14:38.033 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w== 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw== 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA== 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ== 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw== 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2023-10-06T05:14:38.034 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA== 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw== 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:[client.2] 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g== 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw== 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:[client.3] 2023-10-06T05:14:38.035 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w== 2023-10-06T05:14:38.054 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2023-10-06T05:14:38.054 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg== 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw== 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg== 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg== 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg== 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w== 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2023-10-06T05:14:38.055 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw== 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A== 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w== 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw== 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA== 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ== 2023-10-06T05:14:38.056 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw== 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA== 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw== 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout:[client.2] 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g== 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw== 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout:[client.3] 2023-10-06T05:14:38.057 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w== 2023-10-06T05:14:38.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:14:38.079 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:14:38.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.150 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.214 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.275 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.332 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.388 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.444 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:14:38.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:14:38.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.555 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.614 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:14:38.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.673 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.728 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.783 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:14:38.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:38.896 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:38.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:38.955 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:39.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:39.013 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:39.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:39.072 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:14:39.129 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T05:14:39.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T05:14:39.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:14:39.262 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T05:14:39.294 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T05:14:39.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:14:39.416 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T05:14:39.443 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T05:14:39.510 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:14:39.604 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T05:14:40.523 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:40.589 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:14:40.609 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T05:14:40.610 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T05:14:40.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T05:14:40.652 INFO:tasks.ceph.mon.a:Started 2023-10-06T05:14:40.652 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T05:14:40.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T05:14:40.655 INFO:tasks.ceph.mon.b:Started 2023-10-06T05:14:40.655 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T05:14:40.655 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T05:14:40.658 INFO:tasks.ceph.mon.c:Started 2023-10-06T05:14:40.658 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T05:14:40.658 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T05:14:40.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T05:14:40.660 INFO:tasks.ceph.mgr.y:Started 2023-10-06T05:14:40.660 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T05:14:40.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T05:14:40.662 INFO:tasks.ceph.mgr.x:Started 2023-10-06T05:14:40.663 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T05:14:40.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T05:14:40.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.802+0000 7f0ac5f0b700 1 Processor -- start 2023-10-06T05:14:40.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.802+0000 7f0ac5f0b700 1 -- start start 2023-10-06T05:14:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0ac5f0b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0ac00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0ac5f0b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 0x7f0ac00c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0ac5f0b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00c3b50 con 0x7f0ac00aaae0 2023-10-06T05:14:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0ac5f0b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac0157700 con 0x7f0ac00ccb70 2023-10-06T05:14:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0ac5f0b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c98e0 con 0x7f0ac0157330 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 0x7f0ac00c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abeffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0ac00aaae0 0x7f0ac00c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40048/0 (socket says 172.21.15.45:40048) 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abeffd700 1 -- 172.21.15.45:0/385969953 learned_addr learned my addr 172.21.15.45:0/385969953 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0ac00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abffff700 1 -- 172.21.15.45:0/385969953 >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 msgr2=0x7f0ac00c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abffff700 1 -- 172.21.15.45:0/385969953 >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 msgr2=0x7f0ac00c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abffff700 1 --2- 172.21.15.45:0/385969953 >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 0x7f0ac00c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abf7fe700 1 -- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 msgr2=0x7f0ac00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abf7fe700 1 -- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 msgr2=0x7f0ac00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abffff700 1 --2- 172.21.15.45:0/385969953 >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 0x7f0ac00c9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0ac00ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:14:40.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.803+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0ac00ce8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:14:40.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194279489 0 0) 0x7f0ac00c3b50 con 0x7f0ac00aaae0 2023-10-06T05:14:40.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa8003570 con 0x7f0ac00aaae0 2023-10-06T05:14:40.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3411080507 0 0) 0x7f0aa8003570 con 0x7f0ac00aaae0 2023-10-06T05:14:40.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 msgr2=0x7f0ac00ce8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:14:40.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0ac00ce8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 msgr2=0x7f0ac00c9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 --2- 172.21.15.45:0/385969953 >> v2:172.21.15.45:3300/0 conn(0x7f0ac00ccb70 0x7f0ac00c9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00c9bc0 con 0x7f0ac00aaae0 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f0ab4001470 con 0x7f0ac00aaae0 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 >> v1:172.21.15.45:6789/0 conn(0x7f0ac00aaae0 legacy=0x7f0ac00c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0aa8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa8004090 0x7f0aa8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0157700 con 0x7f0aa8004d50 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c98e0 con 0x7f0aa8004090 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa800bbf0 con 0x7f0ac0157330 2023-10-06T05:14:40.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.811+0000 7f0abffff700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abeffd700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa8004090 0x7f0aa8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0aa8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abffff700 1 -- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 msgr2=0x7f0aa8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abffff700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac0157330 0x7f0aa8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abffff700 1 -- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa8004090 msgr2=0x7f0aa8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abffff700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa8004090 0x7f0aa8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abffff700 1 -- 172.21.15.45:0/385969953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8003570 con 0x7f0aa8004d50 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abffff700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8007440 tx=0x7f0ab8009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85efa7945a30bdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8009a30 con 0x7f0aa8004d50 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8007cf0 con 0x7f0aa8004d50 2023-10-06T05:14:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.812+0000 7f0abe7fc700 1 -- 172.21.15.45:0/385969953 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8014650 con 0x7f0aa8004d50 2023-10-06T05:14:40.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.813+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/385969953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8004d50 msgr2=0x7f0aa80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.813+0000 7f0ac5f0b700 1 --2- 172.21.15.45:0/385969953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8007440 tx=0x7f0ab8009720 comp rx=0 tx=0).stop 2023-10-06T05:14:40.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.813+0000 7f0abffff700 1 -- 172.21.15.45:0/385969953 reap_dead start 2023-10-06T05:14:40.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.813+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/385969953 shutdown_connections 2023-10-06T05:14:40.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.813+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/385969953 >> 172.21.15.45:0/385969953 conn(0x7f0ac0067e00 msgr2=0x7f0ac00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:40.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.813+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/385969953 shutdown_connections 2023-10-06T05:14:40.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.813+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/385969953 wait complete. 2023-10-06T05:14:40.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.814+0000 7f0ac5f0b700 1 Processor -- start 2023-10-06T05:14:40.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.814+0000 7f0ac5f0b700 1 -- start start 2023-10-06T05:14:40.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0ac5f0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac00b3970 0x7f0ac00aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0ac5f0b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac016b220 0x7f0ac00abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0ac5f0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ac3e0 0x7f0ac00ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0ac5f0b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00ac850 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0ac5f0b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b1b90 con 0x7f0ac00b3970 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0ac5f0b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00af2f0 con 0x7f0ac016b220 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0abffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ac3e0 0x7f0ac00ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0abffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ac3e0 0x7f0ac00ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48218/0 (socket says 172.21.15.45:48218) 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0abffff700 1 -- 172.21.15.45:0/2492229731 learned_addr learned my addr 172.21.15.45:0/2492229731 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0abf7fe700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac00b3970 0x7f0ac00aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.815+0000 7f0abeffd700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac016b220 0x7f0ac00abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.816+0000 7f0abffff700 1 -- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac016b220 msgr2=0x7f0ac00abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.816+0000 7f0abffff700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac016b220 0x7f0ac00abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.816+0000 7f0abffff700 1 -- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac00b3970 msgr2=0x7f0ac00aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.816+0000 7f0abffff700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac00b3970 0x7f0ac00aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.816+0000 7f0abffff700 1 -- 172.21.15.45:0/2492229731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00af5d0 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.816+0000 7f0abffff700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ac3e0 0x7f0ac00ad520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8014000 tx=0x7f0ab8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.817+0000 7f0aa77fe700 1 -- 172.21.15.45:0/2492229731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8009a30 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.817+0000 7f0aa77fe700 1 -- 172.21.15.45:0/2492229731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8007e60 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.817+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac00af8b0 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.817+0000 7f0aa77fe700 1 -- 172.21.15.45:0/2492229731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8013cb0 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.817+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac00b3d40 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.818+0000 7f0aa77fe700 1 -- 172.21.15.45:0/2492229731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0ab801c040 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.818+0000 7f0aa77fe700 1 -- 172.21.15.45:0/2492229731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0ab80209e0 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.819+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac00b1d00 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.821+0000 7f0aa77fe700 1 -- 172.21.15.45:0/2492229731 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0ab8018020 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.902+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0ac006e350 con 0x7f0ac00ac3e0 2023-10-06T05:14:40.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.904+0000 7f0aa77fe700 1 -- 172.21.15.45:0/2492229731 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0ab8020bf0 con 0x7f0ac00ac3e0 2023-10-06T05:14:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ac3e0 msgr2=0x7f0ac00ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ac3e0 0x7f0ac00ad520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8014000 tx=0x7f0ab8017000 comp rx=0 tx=0).stop 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 shutdown_connections 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ac00ac3e0 0x7f0ac00ad520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac00b3970 0x7f0ac00aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 --2- 172.21.15.45:0/2492229731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac016b220 0x7f0ac00abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 >> 172.21.15.45:0/2492229731 conn(0x7f0ac0067e00 msgr2=0x7f0ac014cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 shutdown_connections 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:40.905+0000 7f0ac5f0b700 1 -- 172.21.15.45:0/2492229731 wait complete. 2023-10-06T05:14:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2023-10-06T05:14:41.048 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T05:14:41.049 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:41.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T05:14:41.050 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:40.951+0000 7fe58f2f31c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:14:41.058 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:41.056+0000 7f0fa8f831c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:14:41.365 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:41.216+0000 7f0fa8f831c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:14:41.366 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:41.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T05:14:41.367 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:41.153+0000 7fe58f2f31c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:14:41.439 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:41.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T05:14:41.511 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-10-06T05:14:41.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T05:14:41.560 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:41.559+0000 7f0fa8f831c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:14:41.579 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:41.578+0000 7fe58f2f31c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:14:41.582 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:41.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T05:14:41.609 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:41.610 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T05:14:41.677 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:41.677 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T05:14:41.744 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-06T05:14:41.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T05:14:41.812 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8 0 2023-10-06T05:14:41.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.970+0000 7fd22ad7e700 1 Processor -- start 2023-10-06T05:14:41.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.970+0000 7fd22ad7e700 1 -- start start 2023-10-06T05:14:41.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22ad7e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd2240cc840 0x7fd2240c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22ad7e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd2240affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22ad7e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2240ccca0 con 0x7fd22413f900 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22ad7e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2240c9630 con 0x7fd2240cc840 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22ad7e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240ca190 con 0x7fd2240c9d50 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd223fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd2240cc840 0x7fd2240c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22931b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd2240affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22931b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd2240affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44580/0 (socket says 172.21.15.144:44580) 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd223fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd2240cc840 0x7fd2240c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38630/0 (socket says 172.21.15.144:38630) 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22931b700 1 -- 172.21.15.144:0/1095880836 learned_addr learned my addr 172.21.15.144:0/1095880836 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd228b1a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd22413f900 0x7fd2240c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:57410/0 (socket says 172.21.15.144:57410) 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22931b700 1 -- 172.21.15.144:0/1095880836 >> v2:172.21.15.144:3300/0 conn(0x7fd2240cc840 msgr2=0x7fd2240c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22931b700 1 --2- 172.21.15.144:0/1095880836 >> v2:172.21.15.144:3300/0 conn(0x7fd2240cc840 0x7fd2240c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22931b700 1 -- 172.21.15.144:0/1095880836 >> v1:172.21.15.144:6789/0 conn(0x7fd22413f900 legacy=0x7fd2240c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd22931b700 1 -- 172.21.15.144:0/1095880836 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240cb550 con 0x7fd2240c9d50 2023-10-06T05:14:41.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd2237fe700 1 -- 172.21.15.144:0/1095880836 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325065606 0 0) 0x7fd218001110 con 0x7fd22413f900 2023-10-06T05:14:41.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.972+0000 7fd223fff700 1 --2- 172.21.15.144:0/1095880836 >> v2:172.21.15.144:3300/0 conn(0x7fd2240cc840 0x7fd2240c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:41.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.973+0000 7fd22931b700 1 --2- 172.21.15.144:0/1095880836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd2240affa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009f20 tx=0x7fd21c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6c2a9a2f122484 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.973+0000 7fd2237fe700 1 -- 172.21.15.144:0/1095880836 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c00e070 con 0x7fd2240c9d50 2023-10-06T05:14:41.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.973+0000 7fd2237fe700 1 -- 172.21.15.144:0/1095880836 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21c00b9d0 con 0x7fd2240c9d50 2023-10-06T05:14:41.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.973+0000 7fd2237fe700 1 -- 172.21.15.144:0/1095880836 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0092e0 con 0x7fd2240c9d50 2023-10-06T05:14:41.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.973+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1095880836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 msgr2=0x7fd2240affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.973+0000 7fd22ad7e700 1 --2- 172.21.15.144:0/1095880836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd2240affa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009f20 tx=0x7fd21c009840 comp rx=0 tx=0).stop 2023-10-06T05:14:41.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.974+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1095880836 shutdown_connections 2023-10-06T05:14:41.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.974+0000 7fd22ad7e700 1 --2- 172.21.15.144:0/1095880836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd2240affa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.974+0000 7fd22ad7e700 1 --2- 172.21.15.144:0/1095880836 >> v2:172.21.15.144:3300/0 conn(0x7fd2240cc840 0x7fd2240c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.974+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1095880836 >> 172.21.15.144:0/1095880836 conn(0x7fd2240691c0 msgr2=0x7fd2240cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:41.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.974+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1095880836 shutdown_connections 2023-10-06T05:14:41.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.974+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1095880836 wait complete. 2023-10-06T05:14:41.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.975+0000 7fd22ad7e700 1 Processor -- start 2023-10-06T05:14:41.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.976+0000 7fd22ad7e700 1 -- start start 2023-10-06T05:14:41.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22ad7e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd224155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22ad7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240cc840 0x7fd224139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22ad7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd22413f900 0x7fd2240c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22ad7e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd224138960 con 0x7fd22413f900 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22ad7e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd224155c40 con 0x7fd2240cc840 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22ad7e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd224137d50 con 0x7fd2240c9d50 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd223fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240cc840 0x7fd224139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd224155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd223fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240cc840 0x7fd224139440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38644/0 (socket says 172.21.15.144:38644) 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd224155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44592/0 (socket says 172.21.15.144:44592) 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd223fff700 1 -- 172.21.15.144:0/1325188856 learned_addr learned my addr 172.21.15.144:0/1325188856 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22931b700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd22413f900 0x7fd2240c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 -- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240cc840 msgr2=0x7fd224139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240cc840 0x7fd224139440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 -- 172.21.15.144:0/1325188856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd22413f900 msgr2=0x7fd2240c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd22413f900 0x7fd2240c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 -- 172.21.15.144:0/1325188856 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210009420 con 0x7fd2240c9d50 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd223fff700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240cc840 0x7fd224139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd22931b700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd22413f900 0x7fd2240c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.977+0000 7fd228b1a700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd224155700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c4f0 tx=0x7fd218007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.978+0000 7fd220ff9700 1 -- 172.21.15.144:0/1325188856 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218005a70 con 0x7fd2240c9d50 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.978+0000 7fd220ff9700 1 -- 172.21.15.144:0/1325188856 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd218005bd0 con 0x7fd2240c9d50 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.978+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c009ae0 con 0x7fd2240c9d50 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.978+0000 7fd220ff9700 1 -- 172.21.15.144:0/1325188856 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180070d0 con 0x7fd2240c9d50 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.978+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd224138310 con 0x7fd2240c9d50 2023-10-06T05:14:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.978+0000 7fd220ff9700 1 -- 172.21.15.144:0/1325188856 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd21801c440 con 0x7fd2240c9d50 2023-10-06T05:14:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.978+0000 7fd220ff9700 1 -- 172.21.15.144:0/1325188856 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd218012750 con 0x7fd2240c9d50 2023-10-06T05:14:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.979+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22406e2d0 con 0x7fd2240c9d50 2023-10-06T05:14:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:41.981+0000 7fd220ff9700 1 -- 172.21.15.144:0/1325188856 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd218017070 con 0x7fd2240c9d50 2023-10-06T05:14:42.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.063+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8", "id": 0} v 0) v1 -- 0x7fd2240cb830 con 0x7fd2240c9d50 2023-10-06T05:14:42.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.067+0000 7fd220ff9700 1 -- 172.21.15.144:0/1325188856 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd2240ccca0 con 0x7fd2240c9d50 2023-10-06T05:14:42.069 INFO:teuthology.orchestra.run.smithi144.stdout:0 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 msgr2=0x7fd224155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd224155700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c4f0 tx=0x7fd218007ef0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 shutdown_connections 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd22413f900 0x7fd2240c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2240c9d50 0x7fd224155700 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 --2- 172.21.15.144:0/1325188856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240cc840 0x7fd224139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 >> 172.21.15.144:0/1325188856 conn(0x7fd2240691c0 msgr2=0x7fd22413e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 shutdown_connections 2023-10-06T05:14:42.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.068+0000 7fd22ad7e700 1 -- 172.21.15.144:0/1325188856 wait complete. 2023-10-06T05:14:42.081 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 8efcdf9d-7f7d-4c07-98f5-1d752cb42495 1 2023-10-06T05:14:42.093 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:42.092+0000 7f0fa8f831c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:14:42.151 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:42.150+0000 7fe58f2f31c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:14:42.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.197+0000 7f211de7f700 1 Processor -- start 2023-10-06T05:14:42.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.197+0000 7f211de7f700 1 -- start start 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f211de7f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f211813f900 0x7f21180cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f211de7f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c8de0 0x7f21180c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f211de7f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21180c9630 con 0x7f21180c9d50 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f211de7f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21180ca150 con 0x7f211813f900 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f211de7f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21180cb550 con 0x7f21180c8de0 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f21177fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f211813f900 0x7f21180cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c8de0 0x7f21180c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f21177fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f211813f900 0x7f21180cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38646/0 (socket says 172.21.15.144:38646) 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c8de0 0x7f21180c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44608/0 (socket says 172.21.15.144:44608) 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f21177fe700 1 -- 172.21.15.144:0/951056933 learned_addr learned my addr 172.21.15.144:0/951056933 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f21177fe700 1 -- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c8de0 msgr2=0x7f21180c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f21177fe700 1 --2- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c8de0 0x7f21180c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f21177fe700 1 -- 172.21.15.144:0/951056933 >> v1:172.21.15.45:6789/0 conn(0x7f21180c9d50 legacy=0x7f211806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f21177fe700 1 -- 172.21.15.144:0/951056933 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21180cb830 con 0x7f211813f900 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.198+0000 7f2116ffd700 1 --2- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c8de0 0x7f21180c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f21177fe700 1 --2- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f211813f900 0x7f21180cc840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2108009f20 tx=0x7f2108009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c2b3d49651154b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f21167fc700 1 -- 172.21.15.144:0/951056933 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210800e070 con 0x7f211813f900 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f21167fc700 1 -- 172.21.15.144:0/951056933 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210800b9d0 con 0x7f211813f900 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f21167fc700 1 -- 172.21.15.144:0/951056933 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080092e0 con 0x7f211813f900 2023-10-06T05:14:42.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 -- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f211813f900 msgr2=0x7f21180cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 --2- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f211813f900 0x7f21180cc840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2108009f20 tx=0x7f2108009840 comp rx=0 tx=0).stop 2023-10-06T05:14:42.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 -- 172.21.15.144:0/951056933 shutdown_connections 2023-10-06T05:14:42.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 --2- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f211813f900 0x7f21180cc840 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 --2- 172.21.15.144:0/951056933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c8de0 0x7f21180c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 -- 172.21.15.144:0/951056933 >> 172.21.15.144:0/951056933 conn(0x7f21180691c0 msgr2=0x7f21180cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 -- 172.21.15.144:0/951056933 shutdown_connections 2023-10-06T05:14:42.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.199+0000 7f211de7f700 1 -- 172.21.15.144:0/951056933 wait complete. 2023-10-06T05:14:42.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.201+0000 7f211de7f700 1 Processor -- start 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.201+0000 7f211de7f700 1 -- start start 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f211de7f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21180c8de0 0x7f211813c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f211de7f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c9d50 0x7f211813b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f211de7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211813f900 0x7f211813bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f211de7f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211813b540 con 0x7f211813f900 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f211de7f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21180c5ed0 con 0x7f21180c8de0 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f211de7f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21180c61b0 con 0x7f21180c9d50 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c9d50 0x7f211813b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f21177fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21180c8de0 0x7f211813c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c9d50 0x7f211813b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44618/0 (socket says 172.21.15.144:44618) 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 -- 172.21.15.144:0/1517864259 learned_addr learned my addr 172.21.15.144:0/1517864259 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2117fff700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211813f900 0x7f211813bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 -- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21180c8de0 msgr2=0x7f211813c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21180c8de0 0x7f211813c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 -- 172.21.15.144:0/1517864259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211813f900 msgr2=0x7f211813bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211813f900 0x7f211813bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2116ffd700 1 -- 172.21.15.144:0/1517864259 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100009420 con 0x7f21180c9d50 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f21177fe700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21180c8de0 0x7f211813c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.202+0000 7f2117fff700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211813f900 0x7f211813bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f2116ffd700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c9d50 0x7f211813b000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f210000ec80 tx=0x7f21000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f210f7fe700 1 -- 172.21.15.144:0/1517864259 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210000c8f0 con 0x7f21180c9d50 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f210f7fe700 1 -- 172.21.15.144:0/1517864259 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210000ca50 con 0x7f21180c9d50 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2108009ae0 con 0x7f21180c9d50 2023-10-06T05:14:42.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f210f7fe700 1 -- 172.21.15.144:0/1517864259 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210000cdd0 con 0x7f21180c9d50 2023-10-06T05:14:42.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21180c6770 con 0x7f21180c9d50 2023-10-06T05:14:42.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f210f7fe700 1 -- 172.21.15.144:0/1517864259 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2100013900 con 0x7f21180c9d50 2023-10-06T05:14:42.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.203+0000 7f210f7fe700 1 -- 172.21.15.144:0/1517864259 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f210001cb20 con 0x7f21180c9d50 2023-10-06T05:14:42.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.204+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211806e2d0 con 0x7f21180c9d50 2023-10-06T05:14:42.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.206+0000 7f210f7fe700 1 -- 172.21.15.144:0/1517864259 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2100021060 con 0x7f21180c9d50 2023-10-06T05:14:42.238 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:42.236+0000 7f0fa8f831c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:14:42.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.289+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8efcdf9d-7f7d-4c07-98f5-1d752cb42495", "id": 1} v 0) v1 -- 0x7f21180cbb10 con 0x7f21180c9d50 2023-10-06T05:14:42.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.293+0000 7f210f7fe700 1 -- 172.21.15.144:0/1517864259 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8efcdf9d-7f7d-4c07-98f5-1d752cb42495", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2100017070 con 0x7f21180c9d50 2023-10-06T05:14:42.294 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c9d50 msgr2=0x7f211813b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c9d50 0x7f211813b000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f210000ec80 tx=0x7f21000091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 shutdown_connections 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211813f900 0x7f211813bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21180c8de0 0x7f211813c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 --2- 172.21.15.144:0/1517864259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21180c9d50 0x7f211813b000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 >> 172.21.15.144:0/1517864259 conn(0x7f21180691c0 msgr2=0x7f21180c29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 shutdown_connections 2023-10-06T05:14:42.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.294+0000 7f211de7f700 1 -- 172.21.15.144:0/1517864259 wait complete. 2023-10-06T05:14:42.306 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:42.305+0000 7fe58f2f31c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:14:42.307 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 0540e79e-e321-4dcf-8ed6-1d023b47ab28 2 2023-10-06T05:14:42.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.466+0000 7f4a2810e700 1 Processor -- start 2023-10-06T05:14:42.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.466+0000 7f4a2810e700 1 -- start start 2023-10-06T05:14:42.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a2810e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a201406e0 0x7f4a2006c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a2810e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a200c9940 con 0x7f4a2006cd60 2023-10-06T05:14:42.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a2810e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2006d230 con 0x7f4a20140ab0 2023-10-06T05:14:42.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a2810e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2006d510 con 0x7f4a201406e0 2023-10-06T05:14:42.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a266ab700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4a20140ab0 0x7f4a200c9dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:57420/0 (socket says 172.21.15.144:57420) 2023-10-06T05:14:42.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a266ab700 1 -- 172.21.15.144:0/2389748159 learned_addr learned my addr 172.21.15.144:0/2389748159 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-10-06T05:14:42.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a25eaa700 1 --2- 172.21.15.144:0/2389748159 >> v2:172.21.15.45:3300/0 conn(0x7f4a201406e0 0x7f4a2006c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a24ea8700 1 -- 172.21.15.144:0/2389748159 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476398876 0 0) 0x7f4a2006d230 con 0x7f4a20140ab0 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a24ea8700 1 -- 172.21.15.144:0/2389748159 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a0c003570 con 0x7f4a20140ab0 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a24ea8700 1 -- 172.21.15.144:0/2389748159 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099256867 0 0) 0x7f4a200c9940 con 0x7f4a2006cd60 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a24ea8700 1 -- 172.21.15.144:0/2389748159 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a2006d230 con 0x7f4a2006cd60 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a25eaa700 1 -- 172.21.15.144:0/2389748159 >> v1:172.21.15.144:6789/0 conn(0x7f4a20140ab0 legacy=0x7f4a200c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.467+0000 7f4a25eaa700 1 -- 172.21.15.144:0/2389748159 >> v1:172.21.15.45:6789/0 conn(0x7f4a2006cd60 legacy=0x7f4a200acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.468+0000 7f4a25eaa700 1 -- 172.21.15.144:0/2389748159 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2006d7f0 con 0x7f4a201406e0 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.468+0000 7f4a25eaa700 1 --2- 172.21.15.144:0/2389748159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a201406e0 0x7f4a2006c820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0098e0 tx=0x7f4a1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1bd62bfca63758 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.468+0000 7f4a24ea8700 1 -- 172.21.15.144:0/2389748159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c011630 con 0x7f4a201406e0 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.468+0000 7f4a24ea8700 1 -- 172.21.15.144:0/2389748159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c011790 con 0x7f4a201406e0 2023-10-06T05:14:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.468+0000 7f4a24ea8700 1 -- 172.21.15.144:0/2389748159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c011b30 con 0x7f4a201406e0 2023-10-06T05:14:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.468+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389748159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a201406e0 msgr2=0x7f4a2006c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.468+0000 7f4a2810e700 1 --2- 172.21.15.144:0/2389748159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a201406e0 0x7f4a2006c820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0098e0 tx=0x7f4a1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.469+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389748159 shutdown_connections 2023-10-06T05:14:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.469+0000 7f4a2810e700 1 --2- 172.21.15.144:0/2389748159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a201406e0 0x7f4a2006c820 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.469+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389748159 >> 172.21.15.144:0/2389748159 conn(0x7f4a200691c0 msgr2=0x7f4a200cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.469+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389748159 shutdown_connections 2023-10-06T05:14:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.469+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389748159 wait complete. 2023-10-06T05:14:42.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 Processor -- start 2023-10-06T05:14:42.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 -- start start 2023-10-06T05:14:42.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a2006cd60 0x7f4a2016d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a201406e0 0x7f4a20169460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a20140ab0 0x7f4a201568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200c2ab0 con 0x7f4a20140ab0 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a20169d60 con 0x7f4a201406e0 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.470+0000 7f4a2810e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2016a040 con 0x7f4a2006cd60 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a2006cd60 0x7f4a2016d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a256a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a201406e0 0x7f4a20169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a2006cd60 0x7f4a2016d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44630/0 (socket says 172.21.15.144:44630) 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 -- 172.21.15.144:0/2389184145 learned_addr learned my addr 172.21.15.144:0/2389184145 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a256a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a201406e0 0x7f4a20169460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38658/0 (socket says 172.21.15.144:38658) 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a266ab700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a20140ab0 0x7f4a201568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 -- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a201406e0 msgr2=0x7f4a20169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a201406e0 0x7f4a20169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 -- 172.21.15.144:0/2389184145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a20140ab0 msgr2=0x7f4a201568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a20140ab0 0x7f4a201568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 -- 172.21.15.144:0/2389184145 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c009420 con 0x7f4a2006cd60 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a256a9700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a201406e0 0x7f4a20169460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a25eaa700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a2006cd60 0x7f4a2016d620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a2006c4b0 tx=0x7f4a1c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a15ffb700 1 -- 172.21.15.144:0/2389184145 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c01f070 con 0x7f4a2006cd60 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a15ffb700 1 -- 172.21.15.144:0/2389184145 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c000bc0 con 0x7f4a2006cd60 2023-10-06T05:14:42.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2016a320 con 0x7f4a2006cd60 2023-10-06T05:14:42.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.471+0000 7f4a15ffb700 1 -- 172.21.15.144:0/2389184145 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c025420 con 0x7f4a2006cd60 2023-10-06T05:14:42.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.472+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a201699a0 con 0x7f4a2006cd60 2023-10-06T05:14:42.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.472+0000 7f4a15ffb700 1 -- 172.21.15.144:0/2389184145 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4a1c023070 con 0x7f4a2006cd60 2023-10-06T05:14:42.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.472+0000 7f4a15ffb700 1 -- 172.21.15.144:0/2389184145 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f4a1c02e7e0 con 0x7f4a2006cd60 2023-10-06T05:14:42.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.472+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2006e2d0 con 0x7f4a2006cd60 2023-10-06T05:14:42.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.474+0000 7f4a15ffb700 1 -- 172.21.15.144:0/2389184145 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4a1c02a070 con 0x7f4a2006cd60 2023-10-06T05:14:42.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.556+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0540e79e-e321-4dcf-8ed6-1d023b47ab28", "id": 2} v 0) v1 -- 0x7f4a2004bb40 con 0x7f4a2006cd60 2023-10-06T05:14:42.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.560+0000 7f4a15ffb700 1 -- 172.21.15.144:0/2389184145 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0540e79e-e321-4dcf-8ed6-1d023b47ab28", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4a1c015020 con 0x7f4a2006cd60 2023-10-06T05:14:42.562 INFO:teuthology.orchestra.run.smithi144.stdout:2 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a2006cd60 msgr2=0x7f4a2016d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a2006cd60 0x7f4a2016d620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a2006c4b0 tx=0x7f4a1c017000 comp rx=0 tx=0).stop 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 shutdown_connections 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a20140ab0 0x7f4a201568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a2006cd60 0x7f4a2016d620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 --2- 172.21.15.144:0/2389184145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a201406e0 0x7f4a20169460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 >> 172.21.15.144:0/2389184145 conn(0x7f4a200691c0 msgr2=0x7f4a200adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 shutdown_connections 2023-10-06T05:14:42.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.561+0000 7f4a2810e700 1 -- 172.21.15.144:0/2389184145 wait complete. 2023-10-06T05:14:42.574 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 7500eeb7-53cf-41c3-bc9d-6bc9a6165384 3 2023-10-06T05:14:42.608 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:42.607+0000 7f0fa8f831c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:14:42.694 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:42.692+0000 7fe58f2f31c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:14:42.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.695+0000 7fe2e7890700 1 Processor -- start 2023-10-06T05:14:42.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.695+0000 7fe2e7890700 1 -- start start 2023-10-06T05:14:42.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e7890700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2e006ae40 0x7fe2e015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e7890700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e0067d80 con 0x7fe2e015c6f0 2023-10-06T05:14:42.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e7890700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e015cb70 con 0x7fe2e006aa70 2023-10-06T05:14:42.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e7890700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e00ac620 con 0x7fe2e006ae40 2023-10-06T05:14:42.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e4e2b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2e006ae40 0x7fe2e015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e4e2b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2e006ae40 0x7fe2e015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38672/0 (socket says 172.21.15.144:38672) 2023-10-06T05:14:42.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e562c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe2e006aa70 0x7fe2e00c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:57426/0 (socket says 172.21.15.144:57426) 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e4e2b700 1 -- 172.21.15.144:0/1403336573 learned_addr learned my addr 172.21.15.144:0/1403336573 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e4e2b700 1 -- 172.21.15.144:0/1403336573 >> v1:172.21.15.144:6789/0 conn(0x7fe2e006aa70 legacy=0x7fe2e00c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e4e2b700 1 -- 172.21.15.144:0/1403336573 >> v1:172.21.15.45:6789/0 conn(0x7fe2e015c6f0 legacy=0x7fe2e00aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e4e2b700 1 -- 172.21.15.144:0/1403336573 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00c75b0 con 0x7fe2e006ae40 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2e4e2b700 1 --2- 172.21.15.144:0/1403336573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006ae40 0x7fe2e015c1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00099b0 tx=0x7fe2d0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4801d757c1bb5640 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2d7fff700 1 -- 172.21.15.144:0/1403336573 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d000b860 con 0x7fe2e006ae40 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2d7fff700 1 -- 172.21.15.144:0/1403336573 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d000b9c0 con 0x7fe2e006ae40 2023-10-06T05:14:42.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.696+0000 7fe2d7fff700 1 -- 172.21.15.144:0/1403336573 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d000bd60 con 0x7fe2e006ae40 2023-10-06T05:14:42.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.697+0000 7fe2e7890700 1 -- 172.21.15.144:0/1403336573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006ae40 msgr2=0x7fe2e015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.697+0000 7fe2e7890700 1 --2- 172.21.15.144:0/1403336573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006ae40 0x7fe2e015c1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00099b0 tx=0x7fe2d0009250 comp rx=0 tx=0).stop 2023-10-06T05:14:42.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.697+0000 7fe2e7890700 1 -- 172.21.15.144:0/1403336573 shutdown_connections 2023-10-06T05:14:42.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.697+0000 7fe2e7890700 1 --2- 172.21.15.144:0/1403336573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006ae40 0x7fe2e015c1b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.697+0000 7fe2e7890700 1 -- 172.21.15.144:0/1403336573 >> 172.21.15.144:0/1403336573 conn(0x7fe2e00cc570 msgr2=0x7fe2e00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.697+0000 7fe2e7890700 1 -- 172.21.15.144:0/1403336573 shutdown_connections 2023-10-06T05:14:42.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.697+0000 7fe2e7890700 1 -- 172.21.15.144:0/1403336573 wait complete. 2023-10-06T05:14:42.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.699+0000 7fe2e7890700 1 Processor -- start 2023-10-06T05:14:42.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.699+0000 7fe2e7890700 1 -- start start 2023-10-06T05:14:42.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e7890700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006aa70 0x7fe2e00c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e7890700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e006ae40 0x7fe2e00c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e7890700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e015c6f0 0x7fe2e01489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e7890700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00c63e0 con 0x7fe2e006ae40 2023-10-06T05:14:42.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e7890700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00c4bb0 con 0x7fe2e006aa70 2023-10-06T05:14:42.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e7890700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00c5a10 con 0x7fe2e015c6f0 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006aa70 0x7fe2e00c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006aa70 0x7fe2e00c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38678/0 (socket says 172.21.15.144:38678) 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 -- 172.21.15.144:0/2411458358 learned_addr learned my addr 172.21.15.144:0/2411458358 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e5e2d700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e015c6f0 0x7fe2e01489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e4e2b700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e006ae40 0x7fe2e00c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 -- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e015c6f0 msgr2=0x7fe2e01489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e015c6f0 0x7fe2e01489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 -- 172.21.15.144:0/2411458358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e006ae40 msgr2=0x7fe2e00c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e006ae40 0x7fe2e00c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 -- 172.21.15.144:0/2411458358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d00094f0 con 0x7fe2e006aa70 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2e562c700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006aa70 0x7fe2e00c5ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c5c0 tx=0x7fe2dc007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.700+0000 7fe2d57fa700 1 -- 172.21.15.144:0/2411458358 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc005a70 con 0x7fe2e006aa70 2023-10-06T05:14:42.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.701+0000 7fe2d57fa700 1 -- 172.21.15.144:0/2411458358 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc005bd0 con 0x7fe2e006aa70 2023-10-06T05:14:42.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.701+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00c4e00 con 0x7fe2e006aa70 2023-10-06T05:14:42.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.701+0000 7fe2d57fa700 1 -- 172.21.15.144:0/2411458358 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0070d0 con 0x7fe2e006aa70 2023-10-06T05:14:42.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.701+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00c5190 con 0x7fe2e006aa70 2023-10-06T05:14:42.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.701+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e006e300 con 0x7fe2e006aa70 2023-10-06T05:14:42.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.702+0000 7fe2d57fa700 1 -- 172.21.15.144:0/2411458358 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe2dc01c440 con 0x7fe2e006aa70 2023-10-06T05:14:42.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.702+0000 7fe2d57fa700 1 -- 172.21.15.144:0/2411458358 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fe2dc0076c0 con 0x7fe2e006aa70 2023-10-06T05:14:42.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.705+0000 7fe2d57fa700 1 -- 172.21.15.144:0/2411458358 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe2dc017070 con 0x7fe2e006aa70 2023-10-06T05:14:42.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.789+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7500eeb7-53cf-41c3-bc9d-6bc9a6165384", "id": 3} v 0) v1 -- 0x7fe2e00c7890 con 0x7fe2e006aa70 2023-10-06T05:14:42.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.793+0000 7fe2d57fa700 1 -- 172.21.15.144:0/2411458358 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7500eeb7-53cf-41c3-bc9d-6bc9a6165384", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe2dc01c6f0 con 0x7fe2e006aa70 2023-10-06T05:14:42.794 INFO:teuthology.orchestra.run.smithi144.stdout:3 2023-10-06T05:14:42.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006aa70 msgr2=0x7fe2e00c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006aa70 0x7fe2e00c5ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c5c0 tx=0x7fe2dc007f70 comp rx=0 tx=0).stop 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 shutdown_connections 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e015c6f0 0x7fe2e01489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006aa70 0x7fe2e00c5ea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 --2- 172.21.15.144:0/2411458358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e006ae40 0x7fe2e00c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 >> 172.21.15.144:0/2411458358 conn(0x7fe2e00cc570 msgr2=0x7fe2e00c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 shutdown_connections 2023-10-06T05:14:42.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.794+0000 7fe2e7890700 1 -- 172.21.15.144:0/2411458358 wait complete. 2023-10-06T05:14:42.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 8fb00fd5-c06a-4ce9-adba-cf22ef411339 4 2023-10-06T05:14:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.962+0000 7f328c397700 1 Processor -- start 2023-10-06T05:14:42.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.962+0000 7f328c397700 1 -- start start 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f328c397700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3284159150 0x7f32840ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f328c397700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3284159520 0x7f328415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f328c397700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32840c9c90 con 0x7f328415d3d0 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f328c397700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328415d8e0 con 0x7f3284159150 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f328c397700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328415dbc0 con 0x7f3284159520 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f3289932700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3284159520 0x7f328415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f3289932700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3284159520 0x7f328415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44660/0 (socket says 172.21.15.144:44660) 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f3289932700 1 -- 172.21.15.144:0/1237650861 learned_addr learned my addr 172.21.15.144:0/1237650861 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:42.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f328a133700 1 --2- 172.21.15.144:0/1237650861 >> v2:172.21.15.45:3300/0 conn(0x7f3284159150 0x7f32840ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f3289932700 1 -- 172.21.15.144:0/1237650861 >> v2:172.21.15.45:3300/0 conn(0x7f3284159150 msgr2=0x7f32840ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f3289932700 1 --2- 172.21.15.144:0/1237650861 >> v2:172.21.15.45:3300/0 conn(0x7f3284159150 0x7f32840ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f3289932700 1 -- 172.21.15.144:0/1237650861 >> v1:172.21.15.45:6789/0 conn(0x7f328415d3d0 legacy=0x7f3284067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.963+0000 7f3289932700 1 -- 172.21.15.144:0/1237650861 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840c9920 con 0x7f3284159520 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f3289932700 1 --2- 172.21.15.144:0/1237650861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3284159520 0x7f328415ce90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32740099b0 tx=0x7f3274009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e33c1e1f72d746e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f3289131700 1 -- 172.21.15.144:0/1237650861 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327400b860 con 0x7f3284159520 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f3289131700 1 -- 172.21.15.144:0/1237650861 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327400b9c0 con 0x7f3284159520 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f3289131700 1 -- 172.21.15.144:0/1237650861 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327400bd60 con 0x7f3284159520 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f328c397700 1 -- 172.21.15.144:0/1237650861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3284159520 msgr2=0x7f328415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f328c397700 1 --2- 172.21.15.144:0/1237650861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3284159520 0x7f328415ce90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32740099b0 tx=0x7f3274009250 comp rx=0 tx=0).stop 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f328c397700 1 -- 172.21.15.144:0/1237650861 shutdown_connections 2023-10-06T05:14:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f328c397700 1 --2- 172.21.15.144:0/1237650861 >> v2:172.21.15.45:3300/0 conn(0x7f3284159150 0x7f32840ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f328c397700 1 --2- 172.21.15.144:0/1237650861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3284159520 0x7f328415ce90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f328c397700 1 -- 172.21.15.144:0/1237650861 >> 172.21.15.144:0/1237650861 conn(0x7f32840691c0 msgr2=0x7f32840cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:42.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.964+0000 7f328c397700 1 -- 172.21.15.144:0/1237650861 shutdown_connections 2023-10-06T05:14:42.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.965+0000 7f328c397700 1 -- 172.21.15.144:0/1237650861 wait complete. 2023-10-06T05:14:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 Processor -- start 2023-10-06T05:14:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 -- start start 2023-10-06T05:14:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3284159150 0x7f328415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415d3d0 0x7f3284159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284159e40 0x7f3284158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840c4790 con 0x7f3284159150 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328415ac20 con 0x7f3284159e40 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328c397700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840c4a50 con 0x7f328415d3d0 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f3289932700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415d3d0 0x7f3284159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328a934700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284159e40 0x7f3284158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f3289932700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415d3d0 0x7f3284159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44670/0 (socket says 172.21.15.144:44670) 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f3289932700 1 -- 172.21.15.144:0/26608992 learned_addr learned my addr 172.21.15.144:0/26608992 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.966+0000 7f328a133700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3284159150 0x7f328415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f3289932700 1 -- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284159e40 msgr2=0x7f3284158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f3289932700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284159e40 0x7f3284158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f3289932700 1 -- 172.21.15.144:0/26608992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3284159150 msgr2=0x7f328415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f3289932700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3284159150 0x7f328415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f3289932700 1 -- 172.21.15.144:0/26608992 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32740094f0 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f328a934700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284159e40 0x7f3284158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f3289932700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415d3d0 0x7f3284159900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f328415dd00 tx=0x7f3274017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f327a7fc700 1 -- 172.21.15.144:0/26608992 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274011e40 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f327a7fc700 1 -- 172.21.15.144:0/26608992 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3274024440 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328415a380 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f327a7fc700 1 -- 172.21.15.144:0/26608992 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274024800 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328415a940 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f327a7fc700 1 -- 172.21.15.144:0/26608992 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f327402d440 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.967+0000 7f327a7fc700 1 -- 172.21.15.144:0/26608992 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f3274024df0 con 0x7f328415d3d0 2023-10-06T05:14:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.968+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32840c4bc0 con 0x7f328415d3d0 2023-10-06T05:14:42.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:42.970+0000 7f327a7fc700 1 -- 172.21.15.144:0/26608992 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3274015020 con 0x7f328415d3d0 2023-10-06T05:14:43.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.052+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8fb00fd5-c06a-4ce9-adba-cf22ef411339", "id": 4} v 0) v1 -- 0x7f328406e300 con 0x7f328415d3d0 2023-10-06T05:14:43.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.056+0000 7f327a7fc700 1 -- 172.21.15.144:0/26608992 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8fb00fd5-c06a-4ce9-adba-cf22ef411339", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f327402d880 con 0x7f328415d3d0 2023-10-06T05:14:43.057 INFO:teuthology.orchestra.run.smithi144.stdout:4 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415d3d0 msgr2=0x7f3284159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415d3d0 0x7f3284159900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f328415dd00 tx=0x7f3274017000 comp rx=0 tx=0).stop 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 shutdown_connections 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284159e40 0x7f3284158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3284159150 0x7f328415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 --2- 172.21.15.144:0/26608992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415d3d0 0x7f3284159900 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 >> 172.21.15.144:0/26608992 conn(0x7f32840691c0 msgr2=0x7f32840c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 shutdown_connections 2023-10-06T05:14:43.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.057+0000 7f328c397700 1 -- 172.21.15.144:0/26608992 wait complete. 2023-10-06T05:14:43.070 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new b9b96788-94eb-4910-97ca-3469eb4fd873 5 2023-10-06T05:14:43.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.187+0000 7f6dd4570700 1 Processor -- start 2023-10-06T05:14:43.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.187+0000 7f6dd4570700 1 -- start start 2023-10-06T05:14:43.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd4570700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6dcc159150 0x7f6dcc0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd4570700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc0c9c90 con 0x7f6dcc067620 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd4570700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc15d8e0 con 0x7f6dcc159520 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd4570700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc15dbc0 con 0x7f6dcc159150 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd1b0b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6dcc159520 0x7f6dcc15ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:57440/0 (socket says 172.21.15.144:57440) 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd1b0b700 1 -- 172.21.15.144:0/994672085 learned_addr learned my addr 172.21.15.144:0/994672085 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd230c700 1 --2- 172.21.15.144:0/994672085 >> v2:172.21.15.45:3300/0 conn(0x7f6dcc159150 0x7f6dcc0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd130a700 1 -- 172.21.15.144:0/994672085 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884447658 0 0) 0x7f6dcc15d8e0 con 0x7f6dcc159520 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.188+0000 7f6dd130a700 1 -- 172.21.15.144:0/994672085 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db8003570 con 0x7f6dcc159520 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd130a700 1 -- 172.21.15.144:0/994672085 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846378568 0 0) 0x7f6dcc0c9c90 con 0x7f6dcc067620 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd130a700 1 -- 172.21.15.144:0/994672085 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dcc15d8e0 con 0x7f6dcc067620 2023-10-06T05:14:43.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd230c700 1 -- 172.21.15.144:0/994672085 >> v1:172.21.15.144:6789/0 conn(0x7f6dcc159520 legacy=0x7f6dcc15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd230c700 1 -- 172.21.15.144:0/994672085 >> v1:172.21.15.45:6789/0 conn(0x7f6dcc067620 legacy=0x7f6dcc067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd230c700 1 -- 172.21.15.144:0/994672085 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0c9920 con 0x7f6dcc159150 2023-10-06T05:14:43.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd230c700 1 --2- 172.21.15.144:0/994672085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc159150 0x7f6dcc0ca120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80098e0 tx=0x7f6dc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2d325959d893c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd130a700 1 -- 172.21.15.144:0/994672085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8011630 con 0x7f6dcc159150 2023-10-06T05:14:43.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd130a700 1 -- 172.21.15.144:0/994672085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc8011790 con 0x7f6dcc159150 2023-10-06T05:14:43.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.189+0000 7f6dd130a700 1 -- 172.21.15.144:0/994672085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8011b30 con 0x7f6dcc159150 2023-10-06T05:14:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.190+0000 7f6dd4570700 1 -- 172.21.15.144:0/994672085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc159150 msgr2=0x7f6dcc0ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.190+0000 7f6dd4570700 1 --2- 172.21.15.144:0/994672085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc159150 0x7f6dcc0ca120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80098e0 tx=0x7f6dc80091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.190+0000 7f6dd4570700 1 -- 172.21.15.144:0/994672085 shutdown_connections 2023-10-06T05:14:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.190+0000 7f6dd4570700 1 --2- 172.21.15.144:0/994672085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc159150 0x7f6dcc0ca120 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.190+0000 7f6dd4570700 1 -- 172.21.15.144:0/994672085 >> 172.21.15.144:0/994672085 conn(0x7f6dcc0691c0 msgr2=0x7f6dcc0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.190+0000 7f6dd4570700 1 -- 172.21.15.144:0/994672085 shutdown_connections 2023-10-06T05:14:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.190+0000 7f6dd4570700 1 -- 172.21.15.144:0/994672085 wait complete. 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.191+0000 7f6dd4570700 1 Processor -- start 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.191+0000 7f6dd4570700 1 -- start start 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dcc067620 0x7f6dcc0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dcc159150 0x7f6dcc156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc156d80 0x7f6dcc15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc169c00 con 0x7f6dcc156d80 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc157220 con 0x7f6dcc159150 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc169e10 con 0x7f6dcc067620 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd1b0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dcc159150 0x7f6dcc156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd230c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dcc067620 0x7f6dcc0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd1b0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dcc159150 0x7f6dcc156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38696/0 (socket says 172.21.15.144:38696) 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd1b0b700 1 -- 172.21.15.144:0/3144737651 learned_addr learned my addr 172.21.15.144:0/3144737651 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd2b0d700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc156d80 0x7f6dcc15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd230c700 1 -- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dcc159150 msgr2=0x7f6dcc156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd230c700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dcc159150 0x7f6dcc156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd230c700 1 -- 172.21.15.144:0/3144737651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc156d80 msgr2=0x7f6dcc15b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd230c700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc156d80 0x7f6dcc15b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd230c700 1 -- 172.21.15.144:0/3144737651 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8009420 con 0x7f6dcc067620 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd1b0b700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dcc159150 0x7f6dcc156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd230c700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dcc067620 0x7f6dcc0c66f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8016000 tx=0x7f6dc8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dc27fc700 1 -- 172.21.15.144:0/3144737651 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc801f070 con 0x7f6dcc067620 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dc27fc700 1 -- 172.21.15.144:0/3144737651 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc8000bc0 con 0x7f6dcc067620 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc16a0f0 con 0x7f6dcc067620 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dc27fc700 1 -- 172.21.15.144:0/3144737651 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8025420 con 0x7f6dcc067620 2023-10-06T05:14:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.192+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc16a6b0 con 0x7f6dcc067620 2023-10-06T05:14:43.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.193+0000 7f6dc27fc700 1 -- 172.21.15.144:0/3144737651 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6dc8023070 con 0x7f6dcc067620 2023-10-06T05:14:43.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.193+0000 7f6dc27fc700 1 -- 172.21.15.144:0/3144737651 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f6dc802e930 con 0x7f6dcc067620 2023-10-06T05:14:43.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.193+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc06e2d0 con 0x7f6dcc067620 2023-10-06T05:14:43.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.195+0000 7f6dc27fc700 1 -- 172.21.15.144:0/3144737651 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f6dc802a070 con 0x7f6dcc067620 2023-10-06T05:14:43.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.278+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b9b96788-94eb-4910-97ca-3469eb4fd873", "id": 5} v 0) v1 -- 0x7f6dcc0ca660 con 0x7f6dcc067620 2023-10-06T05:14:43.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.282+0000 7f6dc27fc700 1 -- 172.21.15.144:0/3144737651 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b9b96788-94eb-4910-97ca-3469eb4fd873", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6dc8015020 con 0x7f6dcc067620 2023-10-06T05:14:43.284 INFO:teuthology.orchestra.run.smithi144.stdout:5 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dcc067620 msgr2=0x7f6dcc0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dcc067620 0x7f6dcc0c66f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8016000 tx=0x7f6dc8017000 comp rx=0 tx=0).stop 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 shutdown_connections 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc156d80 0x7f6dcc15b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dcc067620 0x7f6dcc0c66f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 --2- 172.21.15.144:0/3144737651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dcc159150 0x7f6dcc156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 >> 172.21.15.144:0/3144737651 conn(0x7f6dcc0691c0 msgr2=0x7f6dcc0cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 shutdown_connections 2023-10-06T05:14:43.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.283+0000 7f6dd4570700 1 -- 172.21.15.144:0/3144737651 wait complete. 2023-10-06T05:14:43.296 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new fb1fe043-d4b9-4353-b923-f39108e85b1b 6 2023-10-06T05:14:43.369 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:43.367+0000 7f0fa8f831c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:14:43.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.412+0000 7fc11dab9700 1 Processor -- start 2023-10-06T05:14:43.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.412+0000 7fc11dab9700 1 -- start start 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc11dab9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc118157330 0x7fc1180ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc11dab9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1180ccb70 0x7fc1180c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc11dab9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1180c3b50 con 0x7fc1180aaae0 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc11dab9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc118157700 con 0x7fc1180ccb70 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc11dab9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180c98e0 con 0x7fc118157330 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc118157330 0x7fc1180ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc118157330 0x7fc1180ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44682/0 (socket says 172.21.15.144:44682) 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 -- 172.21.15.144:0/1820849706 learned_addr learned my addr 172.21.15.144:0/1820849706 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc117fff700 1 --2- 172.21.15.144:0/1820849706 >> v2:172.21.15.45:3300/0 conn(0x7fc1180ccb70 0x7fc1180c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 -- 172.21.15.144:0/1820849706 >> v2:172.21.15.45:3300/0 conn(0x7fc1180ccb70 msgr2=0x7fc1180c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 --2- 172.21.15.144:0/1820849706 >> v2:172.21.15.45:3300/0 conn(0x7fc1180ccb70 0x7fc1180c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 -- 172.21.15.144:0/1820849706 >> v1:172.21.15.45:6789/0 conn(0x7fc1180aaae0 legacy=0x7fc1180c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 -- 172.21.15.144:0/1820849706 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180c9bc0 con 0x7fc118157330 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.414+0000 7fc1177fe700 1 --2- 172.21.15.144:0/1820849706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc118157330 0x7fc1180ce8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc108009990 tx=0x7fc108009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac26e617ab08c41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc1167fc700 1 -- 172.21.15.144:0/1820849706 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108005c10 con 0x7fc118157330 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc1167fc700 1 -- 172.21.15.144:0/1820849706 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc108005d70 con 0x7fc118157330 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc1167fc700 1 -- 172.21.15.144:0/1820849706 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800b880 con 0x7fc118157330 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 -- 172.21.15.144:0/1820849706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc118157330 msgr2=0x7fc1180ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 --2- 172.21.15.144:0/1820849706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc118157330 0x7fc1180ce8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc108009990 tx=0x7fc108009220 comp rx=0 tx=0).stop 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 -- 172.21.15.144:0/1820849706 shutdown_connections 2023-10-06T05:14:43.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 --2- 172.21.15.144:0/1820849706 >> v2:172.21.15.45:3300/0 conn(0x7fc1180ccb70 0x7fc1180c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 --2- 172.21.15.144:0/1820849706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc118157330 0x7fc1180ce8b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 -- 172.21.15.144:0/1820849706 >> 172.21.15.144:0/1820849706 conn(0x7fc118067e00 msgr2=0x7fc1180cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 -- 172.21.15.144:0/1820849706 shutdown_connections 2023-10-06T05:14:43.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.415+0000 7fc11dab9700 1 -- 172.21.15.144:0/1820849706 wait complete. 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.416+0000 7fc11dab9700 1 Processor -- start 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc11dab9700 1 -- start start 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc11dab9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1180aaae0 0x7fc118068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc11dab9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1180ccb70 0x7fc1181592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc11dab9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc118157330 0x7fc1180afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc11dab9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118069970 con 0x7fc1180aaae0 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc11dab9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181597f0 con 0x7fc118157330 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc11dab9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc118068d60 con 0x7fc1180ccb70 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc116ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1180ccb70 0x7fc1181592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc116ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1180ccb70 0x7fc1181592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44698/0 (socket says 172.21.15.144:44698) 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc116ffd700 1 -- 172.21.15.144:0/4183281702 learned_addr learned my addr 172.21.15.144:0/4183281702 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc117fff700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc118157330 0x7fc1180afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.417+0000 7fc1177fe700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1180aaae0 0x7fc118068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc116ffd700 1 -- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc118157330 msgr2=0x7fc1180afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc116ffd700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc118157330 0x7fc1180afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc116ffd700 1 -- 172.21.15.144:0/4183281702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1180aaae0 msgr2=0x7fc118068730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc116ffd700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1180aaae0 0x7fc118068730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc116ffd700 1 -- 172.21.15.144:0/4183281702 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108009480 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc116ffd700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1180ccb70 0x7fc1181592b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c4e0 tx=0x7fc10c00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc0ff7fe700 1 -- 172.21.15.144:0/4183281702 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c011590 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc0ff7fe700 1 -- 172.21.15.144:0/4183281702 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c011710 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc118069040 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc0ff7fe700 1 -- 172.21.15.144:0/4183281702 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c011ad0 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc118069600 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc0ff7fe700 1 -- 172.21.15.144:0/4183281702 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc10c01c6c0 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.418+0000 7fc0ff7fe700 1 -- 172.21.15.144:0/4183281702 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fc10c012a20 con 0x7fc1180ccb70 2023-10-06T05:14:43.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.419+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11806fa60 con 0x7fc1180ccb70 2023-10-06T05:14:43.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.421+0000 7fc0ff7fe700 1 -- 172.21.15.144:0/4183281702 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc10c017070 con 0x7fc1180ccb70 2023-10-06T05:14:43.445 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:43.444+0000 7f0fa8f831c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:14:43.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.503+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb1fe043-d4b9-4353-b923-f39108e85b1b", "id": 6} v 0) v1 -- 0x7fc11806e350 con 0x7fc1180ccb70 2023-10-06T05:14:43.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.507+0000 7fc0ff7fe700 1 -- 172.21.15.144:0/4183281702 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb1fe043-d4b9-4353-b923-f39108e85b1b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc10c01c970 con 0x7fc1180ccb70 2023-10-06T05:14:43.508 INFO:teuthology.orchestra.run.smithi144.stdout:6 2023-10-06T05:14:43.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1180ccb70 msgr2=0x7fc1181592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1180ccb70 0x7fc1181592b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c4e0 tx=0x7fc10c00c7f0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 shutdown_connections 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc118157330 0x7fc1180afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1180aaae0 0x7fc118068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 --2- 172.21.15.144:0/4183281702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1180ccb70 0x7fc1181592b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 >> 172.21.15.144:0/4183281702 conn(0x7fc118067e00 msgr2=0x7fc118155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 shutdown_connections 2023-10-06T05:14:43.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.508+0000 7fc11dab9700 1 -- 172.21.15.144:0/4183281702 wait complete. 2023-10-06T05:14:43.513 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:43.511+0000 7fe58f2f31c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:14:43.521 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 818a9fdc-9978-46ce-8993-1f4ff8d9c50a 7 2023-10-06T05:14:43.585 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:43.584+0000 7f0fa8f831c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:14:43.593 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:43.591+0000 7fe58f2f31c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:14:43.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.637+0000 7f14e6304700 1 Processor -- start 2023-10-06T05:14:43.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.637+0000 7f14e6304700 1 -- start start 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.637+0000 7f14e6304700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.637+0000 7f14e6304700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14e00c7050 0x7f14e015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.637+0000 7f14e6304700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e0067d80 con 0x7f14e006ae40 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14e6304700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e00ac620 con 0x7f14e00c7050 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14e6304700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00c75b0 con 0x7f14e006aa70 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14df7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14e00c7050 0x7f14e015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e00c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44700/0 (socket says 172.21.15.144:44700) 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14df7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14e00c7050 0x7f14e015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38716/0 (socket says 172.21.15.144:38716) 2023-10-06T05:14:43.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 -- 172.21.15.144:0/334051145 learned_addr learned my addr 172.21.15.144:0/334051145 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 -- 172.21.15.144:0/334051145 >> v2:172.21.15.144:3300/0 conn(0x7f14e00c7050 msgr2=0x7f14e015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 --2- 172.21.15.144:0/334051145 >> v2:172.21.15.144:3300/0 conn(0x7f14e00c7050 0x7f14e015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 -- 172.21.15.144:0/334051145 >> v1:172.21.15.45:6789/0 conn(0x7f14e006ae40 legacy=0x7f14e006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 -- 172.21.15.144:0/334051145 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e00c7890 con 0x7f14e006aa70 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14df7fe700 1 --2- 172.21.15.144:0/334051145 >> v2:172.21.15.144:3300/0 conn(0x7f14e00c7050 0x7f14e015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14dffff700 1 --2- 172.21.15.144:0/334051145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e00c6b10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f14d00099b0 tx=0x7f14d0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28a690a8fc61493 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14deffd700 1 -- 172.21.15.144:0/334051145 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0009ea0 con 0x7f14e006aa70 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14deffd700 1 -- 172.21.15.144:0/334051145 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d000b860 con 0x7f14e006aa70 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.638+0000 7f14deffd700 1 -- 172.21.15.144:0/334051145 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d000bc00 con 0x7f14e006aa70 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 -- 172.21.15.144:0/334051145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 msgr2=0x7f14e00c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 --2- 172.21.15.144:0/334051145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e00c6b10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f14d00099b0 tx=0x7f14d0009250 comp rx=0 tx=0).stop 2023-10-06T05:14:43.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 -- 172.21.15.144:0/334051145 shutdown_connections 2023-10-06T05:14:43.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 --2- 172.21.15.144:0/334051145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e00c6b10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 --2- 172.21.15.144:0/334051145 >> v2:172.21.15.144:3300/0 conn(0x7f14e00c7050 0x7f14e015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 -- 172.21.15.144:0/334051145 >> 172.21.15.144:0/334051145 conn(0x7f14e00cc570 msgr2=0x7f14e00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 -- 172.21.15.144:0/334051145 shutdown_connections 2023-10-06T05:14:43.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.639+0000 7f14e6304700 1 -- 172.21.15.144:0/334051145 wait complete. 2023-10-06T05:14:43.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.640+0000 7f14e6304700 1 Processor -- start 2023-10-06T05:14:43.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.640+0000 7f14e6304700 1 -- start start 2023-10-06T05:14:43.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14e6304700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e014d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14e6304700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e006ae40 0x7f14e015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14e6304700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e00c7050 0x7f14e016cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14e6304700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01563c0 con 0x7f14e00c7050 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14e6304700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e016d9c0 con 0x7f14e006ae40 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14e6304700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00c5a30 con 0x7f14e006aa70 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e006ae40 0x7f14e015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14dffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e014d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e006ae40 0x7f14e015ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38730/0 (socket says 172.21.15.144:38730) 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 -- 172.21.15.144:0/1103189915 learned_addr learned my addr 172.21.15.144:0/1103189915 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14dffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e014d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44704/0 (socket says 172.21.15.144:44704) 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14e48a1700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e00c7050 0x7f14e016cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 -- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 msgr2=0x7f14e014d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e014d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 -- 172.21.15.144:0/1103189915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e00c7050 msgr2=0x7f14e016cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e00c7050 0x7f14e016cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 -- 172.21.15.144:0/1103189915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d4009420 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14dffff700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e014d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.641+0000 7f14df7fe700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e006ae40 0x7f14e015ac40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f14d400ec80 tx=0x7f14d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.642+0000 7f14c7fff700 1 -- 172.21.15.144:0/1103189915 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400c850 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.642+0000 7f14c7fff700 1 -- 172.21.15.144:0/1103189915 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d400c9b0 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.642+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d00094f0 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.642+0000 7f14c7fff700 1 -- 172.21.15.144:0/1103189915 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400cd30 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.642+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e00c5ff0 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.642+0000 7f14c7fff700 1 -- 172.21.15.144:0/1103189915 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f14d4005c80 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.642+0000 7f14c7fff700 1 -- 172.21.15.144:0/1103189915 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f14d4013680 con 0x7f14e006ae40 2023-10-06T05:14:43.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.643+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e0156530 con 0x7f14e006ae40 2023-10-06T05:14:43.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.645+0000 7f14c7fff700 1 -- 172.21.15.144:0/1103189915 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f14d4021060 con 0x7f14e006ae40 2023-10-06T05:14:43.656 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:43.654+0000 7f0fa8f831c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:14:43.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.726+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "818a9fdc-9978-46ce-8993-1f4ff8d9c50a", "id": 7} v 0) v1 -- 0x7f14e00c7b70 con 0x7f14e006ae40 2023-10-06T05:14:43.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.729+0000 7f14c7fff700 1 -- 172.21.15.144:0/1103189915 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "818a9fdc-9978-46ce-8993-1f4ff8d9c50a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f14d4029020 con 0x7f14e006ae40 2023-10-06T05:14:43.731 INFO:teuthology.orchestra.run.smithi144.stdout:7 2023-10-06T05:14:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.730+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e006ae40 msgr2=0x7f14e015ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.730+0000 7f14e6304700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e006ae40 0x7f14e015ac40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f14d400ec80 tx=0x7f14d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.730+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 shutdown_connections 2023-10-06T05:14:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.730+0000 7f14e6304700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e00c7050 0x7f14e016cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.730+0000 7f14e6304700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14e006aa70 0x7f14e014d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.730+0000 7f14e6304700 1 --2- 172.21.15.144:0/1103189915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e006ae40 0x7f14e015ac40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.730+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 >> 172.21.15.144:0/1103189915 conn(0x7f14e00cc570 msgr2=0x7f14e014eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.731+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 shutdown_connections 2023-10-06T05:14:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-06T05:14:43.731+0000 7f14e6304700 1 -- 172.21.15.144:0/1103189915 wait complete. 2023-10-06T05:14:43.745 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T05:14:43.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T05:14:43.745 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:43.743+0000 7fe58f2f31c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:14:43.747 INFO:tasks.ceph.osd.0:Started 2023-10-06T05:14:43.747 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T05:14:43.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T05:14:43.749 INFO:tasks.ceph.osd.1:Started 2023-10-06T05:14:43.749 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T05:14:43.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T05:14:43.751 INFO:tasks.ceph.osd.2:Started 2023-10-06T05:14:43.751 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T05:14:43.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T05:14:43.753 INFO:tasks.ceph.osd.3:Started 2023-10-06T05:14:43.754 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T05:14:43.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T05:14:43.756 INFO:tasks.ceph.osd.4:Started 2023-10-06T05:14:43.756 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T05:14:43.756 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T05:14:43.758 INFO:tasks.ceph.osd.5:Started 2023-10-06T05:14:43.758 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T05:14:43.758 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T05:14:43.760 INFO:tasks.ceph.osd.6:Started 2023-10-06T05:14:43.760 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T05:14:43.760 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T05:14:43.762 INFO:tasks.ceph.osd.7:Started 2023-10-06T05:14:43.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:43.859 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:43.858+0000 7f0fa8f831c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:14:43.870 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:43.868+0000 7fe58f2f31c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:14:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.935+0000 7f533e736700 1 Processor -- start 2023-10-06T05:14:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.935+0000 7f533e736700 1 -- start start 2023-10-06T05:14:43.938 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:43.936+0000 7f0fa8f831c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:14:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.936+0000 7f533e736700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5338159170 0x7f5338159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.936+0000 7f533e736700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f533815d040 0x7f5338068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.936+0000 7f533e736700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53380ca6d0 con 0x7f5338069b90 2023-10-06T05:14:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.936+0000 7f533e736700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533815d550 con 0x7f533815d040 2023-10-06T05:14:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.936+0000 7f533e736700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533815d810 con 0x7f5338159170 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5337fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5338159170 0x7f5338159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f53377fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f533815d040 0x7f5338068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f533ccd3700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5338069b90 0x7f53380aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:40482/0 (socket says 172.21.15.45:40482) 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f533ccd3700 1 -- 172.21.15.45:0/720710143 learned_addr learned my addr 172.21.15.45:0/720710143 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5336ffd700 1 -- 172.21.15.45:0/720710143 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270237937 0 0) 0x7f53380ca6d0 con 0x7f5338069b90 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5336ffd700 1 -- 172.21.15.45:0/720710143 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5320003570 con 0x7f5338069b90 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5337fff700 1 -- 172.21.15.45:0/720710143 >> v2:172.21.15.144:3300/0 conn(0x7f533815d040 msgr2=0x7f5338068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5337fff700 1 --2- 172.21.15.45:0/720710143 >> v2:172.21.15.144:3300/0 conn(0x7f533815d040 0x7f5338068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5337fff700 1 -- 172.21.15.45:0/720710143 >> v1:172.21.15.144:6789/0 conn(0x7f5338069b90 legacy=0x7f53380aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5337fff700 1 -- 172.21.15.45:0/720710143 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533815daf0 con 0x7f5338159170 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f53377fe700 1 --2- 172.21.15.45:0/720710143 >> v2:172.21.15.144:3300/0 conn(0x7f533815d040 0x7f5338068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5337fff700 1 --2- 172.21.15.45:0/720710143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5338159170 0x7f5338159540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53280098e0 tx=0x7f53280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b294fc4a5c2dd168 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5336ffd700 1 -- 172.21.15.45:0/720710143 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328011630 con 0x7f5338159170 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5336ffd700 1 -- 172.21.15.45:0/720710143 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5328011790 con 0x7f5338159170 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.937+0000 7f5336ffd700 1 -- 172.21.15.45:0/720710143 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328011b30 con 0x7f5338159170 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 -- 172.21.15.45:0/720710143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5338159170 msgr2=0x7f5338159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 --2- 172.21.15.45:0/720710143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5338159170 0x7f5338159540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53280098e0 tx=0x7f53280091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 -- 172.21.15.45:0/720710143 shutdown_connections 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 --2- 172.21.15.45:0/720710143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5338159170 0x7f5338159540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 --2- 172.21.15.45:0/720710143 >> v2:172.21.15.144:3300/0 conn(0x7f533815d040 0x7f5338068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 -- 172.21.15.45:0/720710143 >> 172.21.15.45:0/720710143 conn(0x7f533806f7c0 msgr2=0x7f53380d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 -- 172.21.15.45:0/720710143 shutdown_connections 2023-10-06T05:14:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.938+0000 7f533e736700 1 -- 172.21.15.45:0/720710143 wait complete. 2023-10-06T05:14:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.939+0000 7f533e736700 1 Processor -- start 2023-10-06T05:14:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533e736700 1 -- start start 2023-10-06T05:14:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533e736700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5338069b90 0x7f5338159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533e736700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5338159170 0x7f5338150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533e736700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f533815d040 0x7f533816cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533e736700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533815cdf0 con 0x7f5338069b90 2023-10-06T05:14:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533e736700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53380b33f0 con 0x7f5338159170 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533e736700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533815c580 con 0x7f533815d040 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f5337fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5338069b90 0x7f5338159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f5337fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5338069b90 0x7f5338159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48248/0 (socket says 172.21.15.45:48248) 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f5337fff700 1 -- 172.21.15.45:0/3855817543 learned_addr learned my addr 172.21.15.45:0/3855817543 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f533ccd3700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f533815d040 0x7f533816cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.940+0000 7f53377fe700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5338159170 0x7f5338150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533ccd3700 1 -- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5338159170 msgr2=0x7f5338150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533ccd3700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5338159170 0x7f5338150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533ccd3700 1 -- 172.21.15.45:0/3855817543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5338069b90 msgr2=0x7f5338159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533ccd3700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5338069b90 0x7f5338159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533ccd3700 1 -- 172.21.15.45:0/3855817543 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c009420 con 0x7f533815d040 2023-10-06T05:14:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f5337fff700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5338069b90 0x7f5338159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533ccd3700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f533815d040 0x7f533816cfe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f533000cbc0 tx=0x7f5330001a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f531ffff700 1 -- 172.21.15.45:0/3855817543 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533000a7e0 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f531ffff700 1 -- 172.21.15.45:0/3855817543 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533000a940 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f531ffff700 1 -- 172.21.15.45:0/3855817543 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533000acc0 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5328009420 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533815cb10 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f531ffff700 1 -- 172.21.15.45:0/3855817543 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5330007300 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.941+0000 7f531ffff700 1 -- 172.21.15.45:0/3855817543 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f5330012ae0 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.942+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5338063b50 con 0x7f533815d040 2023-10-06T05:14:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:43.945+0000 7f531ffff700 1 -- 172.21.15.45:0/3855817543 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5330012dc0 con 0x7f533815d040 2023-10-06T05:14:44.015 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:44.013+0000 7f0fa8f831c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:14:44.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.019+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53380b3530 con 0x7f533815d040 2023-10-06T05:14:44.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.020+0000 7f531ffff700 1 -- 172.21.15.45:0/3855817543 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f53300075b0 con 0x7f533815d040 2023-10-06T05:14:44.022 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:44.022 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:14:43.727684+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f533815d040 msgr2=0x7f533816cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f533815d040 0x7f533816cfe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f533000cbc0 tx=0x7f5330001a20 comp rx=0 tx=0).stop 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 shutdown_connections 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f533815d040 0x7f533816cfe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5338069b90 0x7f5338159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 --2- 172.21.15.45:0/3855817543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5338159170 0x7f5338150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 >> 172.21.15.45:0/3855817543 conn(0x7f533806f7c0 msgr2=0x7f53380cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 shutdown_connections 2023-10-06T05:14:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:44.021+0000 7f533e736700 1 -- 172.21.15.45:0/3855817543 wait complete. 2023-10-06T05:14:44.037 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T05:14:44.037 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T05:14:44.078 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:44.076+0000 7fe58f2f31c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:14:44.091 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:44.090+0000 7f0fa8f831c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:14:44.124 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-10-06T05:14:44.122+0000 7f49056ef540 -1 Falling back to public interface 2023-10-06T05:14:44.164 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:44.162+0000 7fe58f2f31c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:14:44.249 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:44.247+0000 7fe58f2f31c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:14:44.269 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:44.268+0000 7f0fa8f831c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:14:44.348 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:44.346+0000 7f0fa8f831c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:14:44.353 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:44.352+0000 7fe58f2f31c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:14:44.355 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-06T05:14:44.354+0000 7f7d6695e540 -1 Falling back to public interface 2023-10-06T05:14:44.376 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-06T05:14:44.375+0000 7fa597cf0540 -1 Falling back to public interface 2023-10-06T05:14:44.377 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-10-06T05:14:44.375+0000 7f7c37ff6540 -1 Falling back to public interface 2023-10-06T05:14:44.385 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-06T05:14:44.383+0000 7ff41ac0f540 -1 Falling back to public interface 2023-10-06T05:14:44.388 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-06T05:14:44.386+0000 7fb755c5c540 -1 Falling back to public interface 2023-10-06T05:14:44.393 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-10-06T05:14:44.391+0000 7fbfd6043540 -1 Falling back to public interface 2023-10-06T05:14:44.401 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-10-06T05:14:44.399+0000 7f4a1b057540 -1 Falling back to public interface 2023-10-06T05:14:44.553 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:44.551+0000 7fe58f2f31c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:14:44.638 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:44.637+0000 7f0fa8f831c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:14:44.642 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:44.640+0000 7fe58f2f31c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:14:44.709 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:44.707+0000 7f0fa8f831c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:44.842 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:44.966 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.964+0000 7fb38fe75700 1 Processor -- start 2023-10-06T05:14:44.966 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.964+0000 7fb38fe75700 1 -- start start 2023-10-06T05:14:44.967 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38fe75700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb3880ccfe0 0x7fb3880ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38fe75700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880cf530 0x7fb3880b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38fe75700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3880b0640 con 0x7fb3880af450 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38fe75700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880af820 con 0x7fb3880ccfe0 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38fe75700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb388159670 con 0x7fb3880cf530 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38d410700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb3880ccfe0 0x7fb3880ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38dc11700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb3880af450 0x7fb3880cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40082/0 (socket says 172.21.15.45:40082) 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38dc11700 1 -- 172.21.15.45:0/3795563052 learned_addr learned my addr 172.21.15.45:0/3795563052 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38e412700 1 --2- 172.21.15.45:0/3795563052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880cf530 0x7fb3880b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38d410700 1 -- 172.21.15.45:0/3795563052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880cf530 msgr2=0x7fb3880b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38d410700 1 --2- 172.21.15.45:0/3795563052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880cf530 0x7fb3880b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.968 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38d410700 1 -- 172.21.15.45:0/3795563052 >> v1:172.21.15.45:6789/0 conn(0x7fb3880af450 legacy=0x7fb3880cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.969 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38d410700 1 -- 172.21.15.45:0/3795563052 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880cfd50 con 0x7fb3880ccfe0 2023-10-06T05:14:44.969 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.965+0000 7fb38cc0f700 1 -- 172.21.15.45:0/3795563052 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179634371 0 0) 0x7fb3880b0640 con 0x7fb3880af450 2023-10-06T05:14:44.969 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38d410700 1 --2- 172.21.15.45:0/3795563052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880ccfe0 0x7fb3880ceff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb3780098e0 tx=0x7fb3780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c93f5a3d9b6ce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.969 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38cc0f700 1 -- 172.21.15.45:0/3795563052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800b640 con 0x7fb3880ccfe0 2023-10-06T05:14:44.969 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38cc0f700 1 -- 172.21.15.45:0/3795563052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800b7a0 con 0x7fb3880ccfe0 2023-10-06T05:14:44.969 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38cc0f700 1 -- 172.21.15.45:0/3795563052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800bb40 con 0x7fb3880ccfe0 2023-10-06T05:14:44.971 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38fe75700 1 -- 172.21.15.45:0/3795563052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880ccfe0 msgr2=0x7fb3880ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.971 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38fe75700 1 --2- 172.21.15.45:0/3795563052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880ccfe0 0x7fb3880ceff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb3780098e0 tx=0x7fb3780091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.971 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38fe75700 1 -- 172.21.15.45:0/3795563052 shutdown_connections 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38fe75700 1 --2- 172.21.15.45:0/3795563052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880cf530 0x7fb3880b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.966+0000 7fb38fe75700 1 --2- 172.21.15.45:0/3795563052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880ccfe0 0x7fb3880ceff0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.967+0000 7fb38fe75700 1 -- 172.21.15.45:0/3795563052 >> 172.21.15.45:0/3795563052 conn(0x7fb38806ed50 msgr2=0x7fb38806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.967+0000 7fb38fe75700 1 -- 172.21.15.45:0/3795563052 shutdown_connections 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.967+0000 7fb38fe75700 1 -- 172.21.15.45:0/3795563052 wait complete. 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.968+0000 7fb38fe75700 1 Processor -- start 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.968+0000 7fb38fe75700 1 -- start start 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38fe75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880af450 0x7fb388155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38fe75700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880ccfe0 0x7fb38815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38fe75700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3880cf530 0x7fb3880c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38fe75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb388155e70 con 0x7fb3880af450 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38fe75700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38815ad10 con 0x7fb3880cf530 2023-10-06T05:14:44.972 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38fe75700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb388159c70 con 0x7fb3880ccfe0 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880af450 0x7fb388155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880af450 0x7fb388155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48260/0 (socket says 172.21.15.45:48260) 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 -- 172.21.15.45:0/3807877383 learned_addr learned my addr 172.21.15.45:0/3807877383 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38d410700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880ccfe0 0x7fb38815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38e412700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3880cf530 0x7fb3880c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 -- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880ccfe0 msgr2=0x7fb38815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880ccfe0 0x7fb38815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 -- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3880cf530 msgr2=0x7fb3880c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3880cf530 0x7fb3880c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 -- 172.21.15.45:0/3807877383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378009420 con 0x7fb3880af450 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38d410700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880ccfe0 0x7fb38815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:44.973 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb38dc11700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880af450 0x7fb388155930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c7e0 tx=0x7fb384007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:44.974 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fe58f2f31c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb37dffb700 1 -- 172.21.15.45:0/3807877383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384005b30 con 0x7fb3880af450 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.969+0000 7fb37dffb700 1 -- 172.21.15.45:0/3807877383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb384005c90 con 0x7fb3880af450 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.970+0000 7fb37dffb700 1 -- 172.21.15.45:0/3807877383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384007300 con 0x7fb3880af450 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.970+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb388159f50 con 0x7fb3880af450 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.970+0000 7fb37dffb700 1 -- 172.21.15.45:0/3807877383 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb384012400 con 0x7fb3880af450 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.970+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38815a510 con 0x7fb3880af450 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.970+0000 7fb37dffb700 1 -- 172.21.15.45:0/3807877383 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fb384005e00 con 0x7fb3880af450 2023-10-06T05:14:44.975 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.971+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38804c380 con 0x7fb3880af450 2023-10-06T05:14:44.976 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:44.973+0000 7fb37dffb700 1 -- 172.21.15.45:0/3807877383 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb384017070 con 0x7fb3880af450 2023-10-06T05:14:45.066 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:45.064+0000 7fe58f2f31c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:45.069 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.068+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3880afff0 con 0x7fb3880af450 2023-10-06T05:14:45.070 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.069+0000 7fb37dffb700 1 -- 172.21.15.45:0/3807877383 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fb3840128e0 con 0x7fb3880af450 2023-10-06T05:14:45.070 INFO:teuthology.misc.health.smithi045.stdout: 2023-10-06T05:14:45.071 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":10,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:14:43.727684+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880af450 msgr2=0x7fb388155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880af450 0x7fb388155930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c7e0 tx=0x7fb384007df0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 shutdown_connections 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3880cf530 0x7fb3880c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3880af450 0x7fb388155930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 --2- 172.21.15.45:0/3807877383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3880ccfe0 0x7fb38815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 >> 172.21.15.45:0/3807877383 conn(0x7fb38806ed50 msgr2=0x7fb388157760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 shutdown_connections 2023-10-06T05:14:45.072 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:45.070+0000 7fb38fe75700 1 -- 172.21.15.45:0/3807877383 wait complete. 2023-10-06T05:14:45.083 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T05:14:45.212 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:45.211+0000 7f0fa8f831c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:14:45.530 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:45.528+0000 7f0fa8f831c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:14:45.638 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:45.636+0000 7fe58f2f31c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:14:45.669 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:45.667+0000 7f0fa8f831c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:45.996 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:45.994+0000 7fe58f2f31c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:14:46.025 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:46.023+0000 7f0fa8f831c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:14:46.154 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:46.152+0000 7fe58f2f31c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:46.168 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:46.167+0000 7f0fa8f831c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:46.316 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:46.315+0000 7f0fa8f831c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:14:46.416 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:46.415+0000 7f0fa8f831c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:46.559 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:46.558+0000 7fe58f2f31c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:14:46.723 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:46.721+0000 7fe58f2f31c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:14:46.900 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:46.899+0000 7fe58f2f31c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:14:47.018 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-10-06T05:14:47.016+0000 7fe58f2f31c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:14:47.442 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:47.441+0000 7f0f6d663700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T05:14:47.443 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-06T05:14:47.442+0000 7f0f6d663700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T05:14:47.620 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-10-06T05:14:47.619+0000 7f49056ef540 -1 osd.1 0 log_to_monitors true 2023-10-06T05:14:47.844 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-06T05:14:47.843+0000 7f7d6695e540 -1 osd.6 0 log_to_monitors true 2023-10-06T05:14:47.874 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-06T05:14:47.873+0000 7ff41ac0f540 -1 osd.5 0 log_to_monitors true 2023-10-06T05:14:47.886 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-10-06T05:14:47.884+0000 7fbfd6043540 -1 osd.0 0 log_to_monitors true 2023-10-06T05:14:47.901 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-06T05:14:47.899+0000 7fa597cf0540 -1 osd.7 0 log_to_monitors true 2023-10-06T05:14:47.908 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-10-06T05:14:47.907+0000 7f7c37ff6540 -1 osd.2 0 log_to_monitors true 2023-10-06T05:14:47.915 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-06T05:14:47.914+0000 7fb755c5c540 -1 osd.4 0 log_to_monitors true 2023-10-06T05:14:47.930 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-10-06T05:14:47.929+0000 7f4a1b057540 -1 osd.3 0 log_to_monitors true 2023-10-06T05:14:50.801 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-10-06T05:14:50.798+0000 7f4a11c5c700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T05:14:50.814 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-10-06T05:14:50.813+0000 7f4a0aa53700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:50.816 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-10-06T05:14:50.814+0000 7f7c2ebfb700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T05:14:50.820 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-10-06T05:14:50.818+0000 7f7c279f2700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:50.855 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-10-06T05:14:50.854+0000 7fbfccc48700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T05:14:50.861 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-10-06T05:14:50.859+0000 7fbfc5a3f700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:50.873 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-10-06T05:14:50.871+0000 7f48fc2f4700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T05:14:50.895 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-10-06T05:14:50.893+0000 7f48f50eb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:51.446 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-06T05:14:51.441+0000 7ff411814700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T05:14:51.459 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-06T05:14:51.458+0000 7ff40a60b700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:51.468 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-06T05:14:51.467+0000 7f7d5d563700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T05:14:51.477 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-06T05:14:51.475+0000 7f7d5635a700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:51.492 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-06T05:14:51.491+0000 7fa58e8f5700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T05:14:51.498 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-06T05:14:51.496+0000 7fa5876ec700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:51.730 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-06T05:14:51.728+0000 7fb74c861700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T05:14:51.733 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-06T05:14:51.732+0000 7fb745658700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:14:51.890 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:52.021 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.019+0000 7fa0cc406700 1 Processor -- start 2023-10-06T05:14:52.021 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.019+0000 7fa0cc406700 1 -- start start 2023-10-06T05:14:52.023 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.020+0000 7fa0cc406700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa0c40af450 0x7fa0c40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.023 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.020+0000 7fa0cc406700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cce10 0x7fa0c40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.023 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.020+0000 7fa0cc406700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40b0640 con 0x7fa0c40cf530 2023-10-06T05:14:52.023 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.020+0000 7fa0cc406700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40af820 con 0x7fa0c40af450 2023-10-06T05:14:52.023 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.020+0000 7fa0cc406700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4159670 con 0x7fa0c40cce10 2023-10-06T05:14:52.023 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.020+0000 7fa0ca9a3700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa0c40cf530 0x7fa0c4068c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33712/0 (socket says 172.21.15.45:33712) 2023-10-06T05:14:52.023 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.020+0000 7fa0ca9a3700 1 -- 172.21.15.45:0/2862382226 learned_addr learned my addr 172.21.15.45:0/2862382226 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0ca1a2700 1 --2- 172.21.15.45:0/2862382226 >> v2:172.21.15.144:3300/0 conn(0x7fa0c40af450 0x7fa0c40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c99a1700 1 --2- 172.21.15.45:0/2862382226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cce10 0x7fa0c40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c91a0700 1 -- 172.21.15.45:0/2862382226 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409467718 0 0) 0x7fa0c40b0640 con 0x7fa0c40cf530 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c91a0700 1 -- 172.21.15.45:0/2862382226 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b0003570 con 0x7fa0c40cf530 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c99a1700 1 -- 172.21.15.45:0/2862382226 >> v2:172.21.15.144:3300/0 conn(0x7fa0c40af450 msgr2=0x7fa0c40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c99a1700 1 --2- 172.21.15.45:0/2862382226 >> v2:172.21.15.144:3300/0 conn(0x7fa0c40af450 0x7fa0c40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c99a1700 1 -- 172.21.15.45:0/2862382226 >> v1:172.21.15.45:6789/0 conn(0x7fa0c40cf530 legacy=0x7fa0c4068c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c99a1700 1 -- 172.21.15.45:0/2862382226 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40cfd50 con 0x7fa0c40cce10 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0ca1a2700 1 --2- 172.21.15.45:0/2862382226 >> v2:172.21.15.144:3300/0 conn(0x7fa0c40af450 0x7fa0c40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c99a1700 1 --2- 172.21.15.45:0/2862382226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cce10 0x7fa0c40ceff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80098e0 tx=0x7fa0b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9398073175c944 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c91a0700 1 -- 172.21.15.45:0/2862382226 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011630 con 0x7fa0c40cce10 2023-10-06T05:14:52.024 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c91a0700 1 -- 172.21.15.45:0/2862382226 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8011790 con 0x7fa0c40cce10 2023-10-06T05:14:52.025 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.021+0000 7fa0c91a0700 1 -- 172.21.15.45:0/2862382226 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011b10 con 0x7fa0c40cce10 2023-10-06T05:14:52.027 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 -- 172.21.15.45:0/2862382226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cce10 msgr2=0x7fa0c40ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.027 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 --2- 172.21.15.45:0/2862382226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cce10 0x7fa0c40ceff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80098e0 tx=0x7fa0b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.027 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 -- 172.21.15.45:0/2862382226 shutdown_connections 2023-10-06T05:14:52.027 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 --2- 172.21.15.45:0/2862382226 >> v2:172.21.15.144:3300/0 conn(0x7fa0c40af450 0x7fa0c40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.027 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 --2- 172.21.15.45:0/2862382226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cce10 0x7fa0c40ceff0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.027 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 -- 172.21.15.45:0/2862382226 >> 172.21.15.45:0/2862382226 conn(0x7fa0c406ed50 msgr2=0x7fa0c406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 -- 172.21.15.45:0/2862382226 shutdown_connections 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.022+0000 7fa0cc406700 1 -- 172.21.15.45:0/2862382226 wait complete. 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.023+0000 7fa0cc406700 1 Processor -- start 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.023+0000 7fa0cc406700 1 -- start start 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0cc406700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40af450 0x7fa0c40b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0cc406700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cce10 0x7fa0c41530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0cc406700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40cf530 0x7fa0c40b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0cc406700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c82e0 con 0x7fa0c40cce10 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0cc406700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b3750 con 0x7fa0c40cf530 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0cc406700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b3a30 con 0x7fa0c40af450 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0c99a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cce10 0x7fa0c41530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.028 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0c99a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cce10 0x7fa0c41530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34536/0 (socket says 172.21.15.45:34536) 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0c99a1700 1 -- 172.21.15.45:0/3411137089 learned_addr learned my addr 172.21.15.45:0/3411137089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0ca9a3700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40cf530 0x7fa0c40b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.024+0000 7fa0ca1a2700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40af450 0x7fa0c40b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0c99a1700 1 -- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40af450 msgr2=0x7fa0c40b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0c99a1700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40af450 0x7fa0c40b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0c99a1700 1 -- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40cf530 msgr2=0x7fa0c40b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0c99a1700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40cf530 0x7fa0c40b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0c99a1700 1 -- 172.21.15.45:0/3411137089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c0009420 con 0x7fa0c40cce10 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0c99a1700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cce10 0x7fa0c41530e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0c41597b0 tx=0x7fa0b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0b67fc700 1 -- 172.21.15.45:0/3411137089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011900 con 0x7fa0c40cce10 2023-10-06T05:14:52.029 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0b67fc700 1 -- 172.21.15.45:0/3411137089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8011a60 con 0x7fa0c40cce10 2023-10-06T05:14:52.030 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0b67fc700 1 -- 172.21.15.45:0/3411137089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b801a680 con 0x7fa0c40cce10 2023-10-06T05:14:52.030 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b8009420 con 0x7fa0c40cce10 2023-10-06T05:14:52.030 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.025+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40b3ff0 con 0x7fa0c40cce10 2023-10-06T05:14:52.031 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.026+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c40c84e0 con 0x7fa0c40cce10 2023-10-06T05:14:52.031 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.026+0000 7fa0b67fc700 1 -- 172.21.15.45:0/3411137089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fa0b801aa30 con 0x7fa0c40cce10 2023-10-06T05:14:52.031 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.027+0000 7fa0b67fc700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a8066ea0 0x7fa0a8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.031 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.027+0000 7fa0b67fc700 1 -- 172.21.15.45:0/3411137089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fa0b8085b90 con 0x7fa0c40cce10 2023-10-06T05:14:52.031 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.027+0000 7fa0ca1a2700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a8066ea0 0x7fa0a8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.031 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.027+0000 7fa0ca1a2700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a8066ea0 0x7fa0a8069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009f40 tx=0x7fa0c00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:52.033 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.032+0000 7fa0b67fc700 1 -- 172.21.15.45:0/3411137089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0c40c84e0 con 0x7fa0c40cce10 2023-10-06T05:14:52.236 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.234+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0c40c84b0 con 0x7fa0c40cce10 2023-10-06T05:14:52.237 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.235+0000 7fa0b67fc700 1 -- 172.21.15.45:0/3411137089 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10492 (secure 0 0 0) 0x7fa0b80533b0 con 0x7fa0c40cce10 2023-10-06T05:14:52.237 INFO:teuthology.misc.health.smithi045.stdout: 2023-10-06T05:14:52.237 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":13,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:14:51.823626+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:52.239 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a8066ea0 msgr2=0x7fa0a8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.239 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a8066ea0 0x7fa0a8069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009f40 tx=0x7fa0c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.239 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cce10 msgr2=0x7fa0c41530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.239 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cce10 0x7fa0c41530e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0c41597b0 tx=0x7fa0b8017000 comp rx=0 tx=0).stop 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 shutdown_connections 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a8066ea0 0x7fa0a8069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40cf530 0x7fa0c40b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40af450 0x7fa0c40b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 --2- 172.21.15.45:0/3411137089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cce10 0x7fa0c41530e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 >> 172.21.15.45:0/3411137089 conn(0x7fa0c406ed50 msgr2=0x7fa0c406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 shutdown_connections 2023-10-06T05:14:52.240 INFO:teuthology.misc.health.smithi045.stderr:2023-10-06T05:14:52.237+0000 7fa0cc406700 1 -- 172.21.15.45:0/3411137089 wait complete. 2023-10-06T05:14:52.251 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T05:14:52.252 INFO:tasks.ceph:Creating RBD pool 2023-10-06T05:14:52.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T05:14:52.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.381+0000 7f37506cd700 1 Processor -- start 2023-10-06T05:14:52.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.381+0000 7f37506cd700 1 -- start start 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f37506cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f37506cd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f374815c8c0 0x7f37480aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f37506cd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3748067d80 con 0x7f37480c7050 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f37506cd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37480ac620 con 0x7f374815c8c0 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f37506cd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c75b0 con 0x7f374806aa70 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f374ec6a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f374815c8c0 0x7f37480aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f374ec6a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f374815c8c0 0x7f37480aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34552/0 (socket says 172.21.15.45:34552) 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.382+0000 7f374ec6a700 1 -- 172.21.15.45:0/3634724705 learned_addr learned my addr 172.21.15.45:0/3634724705 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374e469700 1 --2- 172.21.15.45:0/3634724705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374ec6a700 1 -- 172.21.15.45:0/3634724705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 msgr2=0x7f37480c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374ec6a700 1 --2- 172.21.15.45:0/3634724705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374ec6a700 1 -- 172.21.15.45:0/3634724705 >> v1:172.21.15.144:6789/0 conn(0x7f37480c7050 legacy=0x7f374815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374ec6a700 1 -- 172.21.15.45:0/3634724705 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480c7890 con 0x7f374815c8c0 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374e469700 1 --2- 172.21.15.45:0/3634724705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374ec6a700 1 --2- 172.21.15.45:0/3634724705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480aca70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3740009f20 tx=0x7f3740009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af44a4eaca60044 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374d467700 1 -- 172.21.15.45:0/3634724705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374000e070 con 0x7f374815c8c0 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374d467700 1 -- 172.21.15.45:0/3634724705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374000ba80 con 0x7f374815c8c0 2023-10-06T05:14:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.383+0000 7f374d467700 1 -- 172.21.15.45:0/3634724705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400092e0 con 0x7f374815c8c0 2023-10-06T05:14:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 -- 172.21.15.45:0/3634724705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 msgr2=0x7f37480aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 --2- 172.21.15.45:0/3634724705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480aca70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3740009f20 tx=0x7f3740009840 comp rx=0 tx=0).stop 2023-10-06T05:14:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 -- 172.21.15.45:0/3634724705 shutdown_connections 2023-10-06T05:14:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 --2- 172.21.15.45:0/3634724705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480aca70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 --2- 172.21.15.45:0/3634724705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 -- 172.21.15.45:0/3634724705 >> 172.21.15.45:0/3634724705 conn(0x7f37480cc570 msgr2=0x7f37480cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 -- 172.21.15.45:0/3634724705 shutdown_connections 2023-10-06T05:14:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.384+0000 7f37506cd700 1 -- 172.21.15.45:0/3634724705 wait complete. 2023-10-06T05:14:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.385+0000 7f37506cd700 1 Processor -- start 2023-10-06T05:14:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.386+0000 7f37506cd700 1 -- start start 2023-10-06T05:14:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.386+0000 7f37506cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.386+0000 7f37506cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37480c7050 0x7f3748154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.386+0000 7f37506cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f37506cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c6430 con 0x7f374815c8c0 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f37506cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c4600 con 0x7f37480c7050 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f37506cd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37480afcf0 con 0x7f374806aa70 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34556/0 (socket says 172.21.15.45:34556) 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 -- 172.21.15.45:0/3269618695 learned_addr learned my addr 172.21.15.45:0/3269618695 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374e469700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374dc68700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37480c7050 0x7f3748154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 -- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 msgr2=0x7f37480c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 -- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37480c7050 msgr2=0x7f3748154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37480c7050 0x7f3748154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 -- 172.21.15.45:0/3269618695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3740009ae0 con 0x7f374815c8c0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f374ec6a700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480c5ef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3740008950 tx=0x7f3740018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f373e7fc700 1 -- 172.21.15.45:0/3269618695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400032f0 con 0x7f374815c8c0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f373e7fc700 1 -- 172.21.15.45:0/3269618695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3740008d40 con 0x7f374815c8c0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37480a8e80 con 0x7f374815c8c0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f373e7fc700 1 -- 172.21.15.45:0/3269618695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740025690 con 0x7f374815c8c0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.387+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37480af6a0 con 0x7f374815c8c0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.388+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374806e300 con 0x7f374815c8c0 2023-10-06T05:14:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.389+0000 7f373e7fc700 1 -- 172.21.15.45:0/3269618695 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f3740003040 con 0x7f374815c8c0 2023-10-06T05:14:52.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.389+0000 7f373e7fc700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f372c066ea0 0x7f372c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.389+0000 7f373e7fc700 1 -- 172.21.15.45:0/3269618695 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f3740085a80 con 0x7f374815c8c0 2023-10-06T05:14:52.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.389+0000 7f374e469700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f372c066ea0 0x7f372c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.390+0000 7f374e469700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f372c066ea0 0x7f372c069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3744006f00 tx=0x7f3744008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.394+0000 7f373e7fc700 1 -- 172.21.15.45:0/3269618695 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3740087e60 con 0x7f374815c8c0 2023-10-06T05:14:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.600+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f37480afe30 con 0x7f374815c8c0 2023-10-06T05:14:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.828+0000 7f373e7fc700 1 -- 172.21.15.45:0/3269618695 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f37400532a0 con 0x7f374815c8c0 2023-10-06T05:14:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f372c066ea0 msgr2=0x7f372c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f372c066ea0 0x7f372c069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3744006f00 tx=0x7f3744008040 comp rx=0 tx=0).stop 2023-10-06T05:14:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 msgr2=0x7f37480c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480c5ef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3740008950 tx=0x7f3740018000 comp rx=0 tx=0).stop 2023-10-06T05:14:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 shutdown_connections 2023-10-06T05:14:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f372c066ea0 0x7f372c069350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f374815c8c0 0x7f37480c5ef0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f374806aa70 0x7f37480c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 --2- 172.21.15.45:0/3269618695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37480c7050 0x7f3748154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 >> 172.21.15.45:0/3269618695 conn(0x7f37480cc570 msgr2=0x7f37480ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 shutdown_connections 2023-10-06T05:14:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.832+0000 7f37506cd700 1 -- 172.21.15.45:0/3269618695 wait complete. 2023-10-06T05:14:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'rbd' created 2023-10-06T05:14:52.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T05:14:52.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.987+0000 7f2fae15f700 1 Processor -- start 2023-10-06T05:14:52.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.987+0000 7f2fae15f700 1 -- start start 2023-10-06T05:14:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.987+0000 7f2fae15f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 0x7f2fa8161c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fae15f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa80cbfc0 con 0x7f2fa80ce740 2023-10-06T05:14:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fae15f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa8155050 con 0x7f2fa8154c80 2023-10-06T05:14:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fae15f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81585e0 con 0x7f2fa80cd460 2023-10-06T05:14:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa6ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2fa80ce740 0x7f2fa80ca7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33714/0 (socket says 172.21.15.45:33714) 2023-10-06T05:14:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa6ffd700 1 -- 172.21.15.45:0/4204911525 learned_addr learned my addr 172.21.15.45:0/4204911525 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa7fff700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 0x7f2fa8161c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953188242 0 0) 0x7f2fa80cbfc0 con 0x7f2fa80ce740 2023-10-06T05:14:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f90003570 con 0x7f2fa80ce740 2023-10-06T05:14:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688495873 0 0) 0x7f2fa8155050 con 0x7f2fa8154c80 2023-10-06T05:14:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa80cbfc0 con 0x7f2fa8154c80 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2264583931 0 0) 0x7f2f90003570 con 0x7f2fa80ce740 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 msgr2=0x7f2fa8161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 0x7f2fa8161c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 >> v1:172.21.15.144:6789/0 conn(0x7f2fa8154c80 legacy=0x7f2fa8157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8157d50 con 0x7f2fa80ce740 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f2f9c001470 con 0x7f2fa80ce740 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 >> v1:172.21.15.45:6789/0 conn(0x7f2fa80ce740 legacy=0x7f2fa80ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90003c20 0x7f2f90004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90004810 0x7f2f90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 0x7f2f900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81585e0 con 0x7f2f90003c20 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9000bb70 con 0x7f2f90004810 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.988+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9000bd30 con 0x7f2fa80cd460 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa77fe700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90003c20 0x7f2f90004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa6ffd700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90004810 0x7f2f90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa7fff700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 0x7f2f900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa77fe700 1 -- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 msgr2=0x7f2f900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa77fe700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80cd460 0x7f2f900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa77fe700 1 -- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90004810 msgr2=0x7f2f90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa77fe700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90004810 0x7f2f90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa77fe700 1 -- 172.21.15.45:0/4204911525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90003570 con 0x7f2f90003c20 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa77fe700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90003c20 0x7f2f90004010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f98001a50 tx=0x7f2f98007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae05dacf4cae7d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800c5d0 con 0x7f2f90003c20 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9800c730 con 0x7f2f90003c20 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.989+0000 7f2fa67fc700 1 -- 172.21.15.45:0/4204911525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800cad0 con 0x7f2f90003c20 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.990+0000 7f2fae15f700 1 -- 172.21.15.45:0/4204911525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90003c20 msgr2=0x7f2f90004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.990+0000 7f2fae15f700 1 --2- 172.21.15.45:0/4204911525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90003c20 0x7f2f90004010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f98001a50 tx=0x7f2f98007d30 comp rx=0 tx=0).stop 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.990+0000 7f2fa7fff700 1 -- 172.21.15.45:0/4204911525 reap_dead start 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.990+0000 7f2fae15f700 1 -- 172.21.15.45:0/4204911525 shutdown_connections 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.990+0000 7f2fae15f700 1 -- 172.21.15.45:0/4204911525 >> 172.21.15.45:0/4204911525 conn(0x7f2fa80680b0 msgr2=0x7f2fa80cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.990+0000 7f2fae15f700 1 -- 172.21.15.45:0/4204911525 shutdown_connections 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.990+0000 7f2fae15f700 1 -- 172.21.15.45:0/4204911525 wait complete. 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.991+0000 7f2fae15f700 1 Processor -- start 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.991+0000 7f2fae15f700 1 -- start start 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fae15f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f90003c20 0x7f2fa80c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fae15f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fa81696d0 0x7f2fa816c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fae15f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa806a150 0x7f2fa806bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fae15f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80c7e50 con 0x7f2fa81696d0 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fae15f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8169b40 con 0x7f2fa806a150 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fae15f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa816ca00 con 0x7f2f90003c20 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fa81696d0 0x7f2fa816c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fa81696d0 0x7f2fa816c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34572/0 (socket says 172.21.15.45:34572) 2023-10-06T05:14:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 -- 172.21.15.45:0/676787385 learned_addr learned my addr 172.21.15.45:0/676787385 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa77fe700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f90003c20 0x7f2fa80c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa7fff700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa806a150 0x7f2fa806bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 -- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f90003c20 msgr2=0x7f2fa80c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f90003c20 0x7f2fa80c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 -- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa806a150 msgr2=0x7f2fa806bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa806a150 0x7f2fa806bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.992+0000 7f2fa6ffd700 1 -- 172.21.15.45:0/676787385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa80c75b0 con 0x7f2fa81696d0 2023-10-06T05:14:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.993+0000 7f2fa6ffd700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fa81696d0 0x7f2fa816c4c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0073c0 tx=0x7f2f9c001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.993+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/676787385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0010b0 con 0x7f2fa81696d0 2023-10-06T05:14:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.993+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/676787385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c00f640 con 0x7f2fa81696d0 2023-10-06T05:14:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.993+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa80c7890 con 0x7f2fa81696d0 2023-10-06T05:14:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.993+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/676787385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00f9c0 con 0x7f2fa81696d0 2023-10-06T05:14:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.993+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa8068860 con 0x7f2fa81696d0 2023-10-06T05:14:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.994+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa8068a80 con 0x7f2fa81696d0 2023-10-06T05:14:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.994+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/676787385 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f2f9c00fd30 con 0x7f2fa81696d0 2023-10-06T05:14:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.995+0000 7f2f8f7fe700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f94066ea0 0x7f2f94069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.995+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/676787385 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5184+0+0 (secure 0 0 0) 0x7f2f9c083cb0 con 0x7f2fa81696d0 2023-10-06T05:14:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.995+0000 7f2fa77fe700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f94066ea0 0x7f2f94069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:52.996+0000 7f2fa77fe700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f94066ea0 0x7f2f94069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f98001ed0 tx=0x7f2f98015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:53.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.000+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/676787385 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f9c04e5a0 con 0x7f2fa81696d0 2023-10-06T05:14:53.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.205+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2fa804c380 con 0x7f2fa81696d0 2023-10-06T05:14:53.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.832+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/676787385 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2f9c01a0c0 con 0x7f2fa81696d0 2023-10-06T05:14:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.835+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f94066ea0 msgr2=0x7f2f94069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.835+0000 7f2fae15f700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f94066ea0 0x7f2f94069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f98001ed0 tx=0x7f2f98015000 comp rx=0 tx=0).stop 2023-10-06T05:14:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.835+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fa81696d0 msgr2=0x7f2fa816c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.835+0000 7f2fae15f700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fa81696d0 0x7f2fa816c4c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0073c0 tx=0x7f2f9c001df0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 shutdown_connections 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f94066ea0 0x7f2f94069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa806a150 0x7f2fa806bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f90003c20 0x7f2fa80c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 --2- 172.21.15.45:0/676787385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fa81696d0 0x7f2fa816c4c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 >> 172.21.15.45:0/676787385 conn(0x7f2fa80680b0 msgr2=0x7f2fa8153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 shutdown_connections 2023-10-06T05:14:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:53.836+0000 7f2fae15f700 1 -- 172.21.15.45:0/676787385 wait complete. 2023-10-06T05:14:53.840 INFO:teuthology.orchestra.run.smithi045.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T05:14:53.857 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T05:14:53.857 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:14:53.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:14:53.859 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:14:53.860 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:14:53.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:14:53.862 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:14:53.862 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:14:53.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:14:53.865 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:14:53.865 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:14:53.865 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:14:53.867 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:14:53.867 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T05:14:53.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:53.949 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:14:53.949 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:14:53.951 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:14:53.951 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:14:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.009+0000 7fe98a774700 1 Processor -- start 2023-10-06T05:14:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.009+0000 7fe98a774700 1 -- start start 2023-10-06T05:14:54.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe98a774700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe984156040 0x7fe9840ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe98a774700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe984067620 con 0x7fe984155c70 2023-10-06T05:14:54.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe98a774700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe984067900 con 0x7fe9840ae6d0 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe98a774700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe984067be0 con 0x7fe984156040 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe983fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe984155c70 0x7fe9840689a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33730/0 (socket says 172.21.15.45:33730) 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe983fff700 1 -- 172.21.15.45:0/2920345945 learned_addr learned my addr 172.21.15.45:0/2920345945 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe9837fe700 1 --2- 172.21.15.45:0/2920345945 >> v2:172.21.15.144:3300/0 conn(0x7fe984156040 0x7fe9840ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe982ffd700 1 -- 172.21.15.45:0/2920345945 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346594036 0 0) 0x7fe984067620 con 0x7fe984155c70 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe982ffd700 1 -- 172.21.15.45:0/2920345945 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe96c003570 con 0x7fe984155c70 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe982ffd700 1 -- 172.21.15.45:0/2920345945 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329712872 0 0) 0x7fe984067900 con 0x7fe9840ae6d0 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe982ffd700 1 -- 172.21.15.45:0/2920345945 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe984067620 con 0x7fe9840ae6d0 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe9837fe700 1 -- 172.21.15.45:0/2920345945 >> v1:172.21.15.144:6789/0 conn(0x7fe9840ae6d0 legacy=0x7fe98406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe9837fe700 1 -- 172.21.15.45:0/2920345945 >> v1:172.21.15.45:6789/0 conn(0x7fe984155c70 legacy=0x7fe9840689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.010+0000 7fe9837fe700 1 -- 172.21.15.45:0/2920345945 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984067ec0 con 0x7fe984156040 2023-10-06T05:14:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.011+0000 7fe9837fe700 1 --2- 172.21.15.45:0/2920345945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe984156040 0x7fe9840ae190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098e0 tx=0x7fe9780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a66ed1e2d54bca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.011+0000 7fe982ffd700 1 -- 172.21.15.45:0/2920345945 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011630 con 0x7fe984156040 2023-10-06T05:14:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.011+0000 7fe982ffd700 1 -- 172.21.15.45:0/2920345945 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978011790 con 0x7fe984156040 2023-10-06T05:14:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.011+0000 7fe982ffd700 1 -- 172.21.15.45:0/2920345945 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011b30 con 0x7fe984156040 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.011+0000 7fe98a774700 1 -- 172.21.15.45:0/2920345945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe984156040 msgr2=0x7fe9840ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.011+0000 7fe98a774700 1 --2- 172.21.15.45:0/2920345945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe984156040 0x7fe9840ae190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098e0 tx=0x7fe9780091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.012+0000 7fe98a774700 1 -- 172.21.15.45:0/2920345945 shutdown_connections 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.012+0000 7fe98a774700 1 --2- 172.21.15.45:0/2920345945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe984156040 0x7fe9840ae190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.012+0000 7fe98a774700 1 -- 172.21.15.45:0/2920345945 >> 172.21.15.45:0/2920345945 conn(0x7fe9840cc5a0 msgr2=0x7fe9840cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.012+0000 7fe98a774700 1 -- 172.21.15.45:0/2920345945 shutdown_connections 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.012+0000 7fe98a774700 1 -- 172.21.15.45:0/2920345945 wait complete. 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.013+0000 7fe98a774700 1 Processor -- start 2023-10-06T05:14:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.013+0000 7fe98a774700 1 -- start start 2023-10-06T05:14:54.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe98a774700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840ae6d0 0x7fe984151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe98a774700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984155c70 0x7fe9840c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe98a774700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe984156040 0x7fe9840c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe98a774700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840a4650 con 0x7fe984156040 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe98a774700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840a4930 con 0x7fe9840ae6d0 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe98a774700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840a4c10 con 0x7fe984155c70 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe988d11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe984156040 0x7fe9840c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840ae6d0 0x7fe984151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840ae6d0 0x7fe984151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:34420/0 (socket says 172.21.15.45:34420) 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe988d11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe984156040 0x7fe9840c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34616/0 (socket says 172.21.15.45:34616) 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 -- 172.21.15.45:0/389710614 learned_addr learned my addr 172.21.15.45:0/389710614 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe9837fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984155c70 0x7fe9840c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 -- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984155c70 msgr2=0x7fe9840c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984155c70 0x7fe9840c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 -- 172.21.15.45:0/389710614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe984156040 msgr2=0x7fe9840c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe984156040 0x7fe9840c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe983fff700 1 -- 172.21.15.45:0/389710614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009420 con 0x7fe9840ae6d0 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.014+0000 7fe988d11700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe984156040 0x7fe9840c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.015+0000 7fe983fff700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840ae6d0 0x7fe984151a20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c790 tx=0x7fe974007e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.015+0000 7fe96bfff700 1 -- 172.21.15.45:0/389710614 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974005b30 con 0x7fe9840ae6d0 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.015+0000 7fe96bfff700 1 -- 172.21.15.45:0/389710614 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe974005cd0 con 0x7fe9840ae6d0 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.015+0000 7fe96bfff700 1 -- 172.21.15.45:0/389710614 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974007320 con 0x7fe9840ae6d0 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.015+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9840a4ef0 con 0x7fe9840ae6d0 2023-10-06T05:14:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.015+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9840a5410 con 0x7fe9840ae6d0 2023-10-06T05:14:54.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.016+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9840a4790 con 0x7fe9840ae6d0 2023-10-06T05:14:54.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.016+0000 7fe96bfff700 1 -- 172.21.15.45:0/389710614 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fe974012400 con 0x7fe9840ae6d0 2023-10-06T05:14:54.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.017+0000 7fe96bfff700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe970066ea0 0x7fe970069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.017+0000 7fe96bfff700 1 -- 172.21.15.45:0/389710614 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe9740835e0 con 0x7fe9840ae6d0 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.017+0000 7fe9837fe700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe970066ea0 0x7fe970069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.018+0000 7fe9837fe700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe970066ea0 0x7fe970069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe9780091e0 tx=0x7fe978017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.022+0000 7fe96bfff700 1 -- 172.21.15.45:0/389710614 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe97404b5a0 con 0x7fe9840ae6d0 2023-10-06T05:14:54.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.230+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9840cc0e0 con 0x7fe9840ae6d0 2023-10-06T05:14:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.231+0000 7fe96bfff700 1 -- 172.21.15.45:0/389710614 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13769 (secure 0 0 0) 0x7fe9840a4790 con 0x7fe9840ae6d0 2023-10-06T05:14:54.233 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:54.233 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":15,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:14:53.827315+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe970066ea0 msgr2=0x7fe970069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe970066ea0 0x7fe970069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe9780091e0 tx=0x7fe978017000 comp rx=0 tx=0).stop 2023-10-06T05:14:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840ae6d0 msgr2=0x7fe984151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840ae6d0 0x7fe984151a20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c790 tx=0x7fe974007e80 comp rx=0 tx=0).stop 2023-10-06T05:14:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 shutdown_connections 2023-10-06T05:14:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe970066ea0 0x7fe970069350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe984156040 0x7fe9840c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840ae6d0 0x7fe984151a20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 --2- 172.21.15.45:0/389710614 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984155c70 0x7fe9840c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 >> 172.21.15.45:0/389710614 conn(0x7fe9840cc5a0 msgr2=0x7fe9841526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 shutdown_connections 2023-10-06T05:14:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.234+0000 7fe98a774700 1 -- 172.21.15.45:0/389710614 wait complete. 2023-10-06T05:14:54.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:14:54.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.384+0000 7f3696465700 1 Processor -- start 2023-10-06T05:14:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.384+0000 7f3696465700 1 -- start start 2023-10-06T05:14:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.385+0000 7f3696465700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f36900c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.385+0000 7f3696465700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3690069ed0 0x7f36900cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.385+0000 7f3696465700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36900ce6a0 con 0x7f36900b0610 2023-10-06T05:14:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.385+0000 7f3696465700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36900690f0 con 0x7f3690069ed0 2023-10-06T05:14:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.385+0000 7f3696465700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36900ceeb0 con 0x7f36900c3db0 2023-10-06T05:14:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f3694a02700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3690069ed0 0x7f36900cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f368ffff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f36900b0610 0x7f36900697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:44036/0 (socket says 172.21.15.45:44036) 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f368ffff700 1 -- 172.21.15.45:0/114407331 learned_addr learned my addr 172.21.15.45:0/114407331 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f368f7fe700 1 --2- 172.21.15.45:0/114407331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f36900c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f3694a02700 1 -- 172.21.15.45:0/114407331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 msgr2=0x7f36900c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f3694a02700 1 --2- 172.21.15.45:0/114407331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f36900c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f3694a02700 1 -- 172.21.15.45:0/114407331 >> v1:172.21.15.144:6789/0 conn(0x7f36900b0610 legacy=0x7f36900697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f3694a02700 1 -- 172.21.15.45:0/114407331 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3690159880 con 0x7f3690069ed0 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f368effd700 1 -- 172.21.15.45:0/114407331 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459936094 0 0) 0x7f36900ce6a0 con 0x7f36900b0610 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f3694a02700 1 --2- 172.21.15.45:0/114407331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f36900cc350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3688009e90 tx=0x7f3688009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51c1427b8711287 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f368effd700 1 -- 172.21.15.45:0/114407331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800e070 con 0x7f3690069ed0 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f368effd700 1 -- 172.21.15.45:0/114407331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368800bbf0 con 0x7f3690069ed0 2023-10-06T05:14:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.386+0000 7f368effd700 1 -- 172.21.15.45:0/114407331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688012430 con 0x7f3690069ed0 2023-10-06T05:14:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 -- 172.21.15.45:0/114407331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 msgr2=0x7f36900cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 --2- 172.21.15.45:0/114407331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f36900cc350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3688009e90 tx=0x7f3688009780 comp rx=0 tx=0).stop 2023-10-06T05:14:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 -- 172.21.15.45:0/114407331 shutdown_connections 2023-10-06T05:14:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 --2- 172.21.15.45:0/114407331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f36900cc350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 --2- 172.21.15.45:0/114407331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f36900c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 -- 172.21.15.45:0/114407331 >> 172.21.15.45:0/114407331 conn(0x7f3690067e00 msgr2=0x7f36900cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 -- 172.21.15.45:0/114407331 shutdown_connections 2023-10-06T05:14:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.387+0000 7f3696465700 1 -- 172.21.15.45:0/114407331 wait complete. 2023-10-06T05:14:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.388+0000 7f3696465700 1 Processor -- start 2023-10-06T05:14:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.388+0000 7f3696465700 1 -- start start 2023-10-06T05:14:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f3696465700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f369014ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f3696465700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36900b0610 0x7f36900b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f3696465700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f369015f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f3696465700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369015d800 con 0x7f3690069ed0 2023-10-06T05:14:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f3696465700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36900c6360 con 0x7f36900b0610 2023-10-06T05:14:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f3696465700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369015cef0 con 0x7f36900c3db0 2023-10-06T05:14:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f368ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f369014ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f368ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f369014ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34630/0 (socket says 172.21.15.45:34630) 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f368ffff700 1 -- 172.21.15.45:0/2001399066 learned_addr learned my addr 172.21.15.45:0/2001399066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f368f7fe700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36900b0610 0x7f36900b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.389+0000 7f3694a02700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f369015f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f368ffff700 1 -- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 msgr2=0x7f369015f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f368ffff700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f369015f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f368ffff700 1 -- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36900b0610 msgr2=0x7f36900b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f368ffff700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36900b0610 0x7f36900b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f368ffff700 1 -- 172.21.15.45:0/2001399066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3688009a20 con 0x7f3690069ed0 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f3694a02700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f369015f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f368ffff700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f369014ad40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f368000c7e0 tx=0x7f3680007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f3677fff700 1 -- 172.21.15.45:0/2001399066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680005b80 con 0x7f3690069ed0 2023-10-06T05:14:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f3677fff700 1 -- 172.21.15.45:0/2001399066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3680005ce0 con 0x7f3690069ed0 2023-10-06T05:14:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36900b36a0 con 0x7f3690069ed0 2023-10-06T05:14:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f3677fff700 1 -- 172.21.15.45:0/2001399066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680007320 con 0x7f3690069ed0 2023-10-06T05:14:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.390+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369015d490 con 0x7f3690069ed0 2023-10-06T05:14:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.391+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369006e350 con 0x7f3690069ed0 2023-10-06T05:14:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.392+0000 7f3677fff700 1 -- 172.21.15.45:0/2001399066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f3680007690 con 0x7f3690069ed0 2023-10-06T05:14:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.392+0000 7f3677fff700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f367c066ea0 0x7f367c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.392+0000 7f3677fff700 1 -- 172.21.15.45:0/2001399066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f36800837b0 con 0x7f3690069ed0 2023-10-06T05:14:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.392+0000 7f368f7fe700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f367c066ea0 0x7f367c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.393+0000 7f368f7fe700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f367c066ea0 0x7f367c069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36900cb4e0 tx=0x7f36840091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.397+0000 7f3677fff700 1 -- 172.21.15.45:0/2001399066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3680050cf0 con 0x7f3690069ed0 2023-10-06T05:14:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.606+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f369004bb40 con 0x7f3690069ed0 2023-10-06T05:14:54.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.606+0000 7f3677fff700 1 -- 172.21.15.45:0/2001399066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36800170a0 con 0x7f3690069ed0 2023-10-06T05:14:54.608 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f367c066ea0 msgr2=0x7f367c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f367c066ea0 0x7f367c069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36900cb4e0 tx=0x7f36840091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 msgr2=0x7f369014ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f369014ad40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f368000c7e0 tx=0x7f3680007e00 comp rx=0 tx=0).stop 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 shutdown_connections 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f367c066ea0 0x7f367c069350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900c3db0 0x7f369015f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3690069ed0 0x7f369014ad40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.608+0000 7f3696465700 1 --2- 172.21.15.45:0/2001399066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36900b0610 0x7f36900b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.609+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 >> 172.21.15.45:0/2001399066 conn(0x7f3690067e00 msgr2=0x7f36900ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:54.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.609+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 shutdown_connections 2023-10-06T05:14:54.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.609+0000 7f3696465700 1 -- 172.21.15.45:0/2001399066 wait complete. 2023-10-06T05:14:54.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.757+0000 7ff76a47b700 1 Processor -- start 2023-10-06T05:14:54.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.757+0000 7ff76a47b700 1 -- start start 2023-10-06T05:14:54.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff76a47b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff76406ae40 0x7ff76415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff76a47b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff76415c6f0 0x7ff7640aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff76a47b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff764067d80 con 0x7ff76406aa70 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff76a47b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7640ac620 con 0x7ff76415c6f0 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff76a47b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76415cb70 con 0x7ff76406ae40 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff76415c6f0 0x7ff7640aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff76415c6f0 0x7ff7640aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34636/0 (socket says 172.21.15.45:34636) 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 -- 172.21.15.45:0/1133320840 learned_addr learned my addr 172.21.15.45:0/1133320840 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff7637fe700 1 --2- 172.21.15.45:0/1133320840 >> v2:172.21.15.144:3300/0 conn(0x7ff76406ae40 0x7ff76415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 -- 172.21.15.45:0/1133320840 >> v2:172.21.15.144:3300/0 conn(0x7ff76406ae40 msgr2=0x7ff76415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 --2- 172.21.15.45:0/1133320840 >> v2:172.21.15.144:3300/0 conn(0x7ff76406ae40 0x7ff76415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 -- 172.21.15.45:0/1133320840 >> v1:172.21.15.144:6789/0 conn(0x7ff76406aa70 legacy=0x7ff7640c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 -- 172.21.15.45:0/1133320840 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7640c75b0 con 0x7ff76415c6f0 2023-10-06T05:14:54.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.758+0000 7ff768a18700 1 --2- 172.21.15.45:0/1133320840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76415c6f0 0x7ff7640aca70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7640cd5e0 tx=0x7ff75c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d32266a7a28f011 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff762ffd700 1 -- 172.21.15.45:0/1133320840 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00e070 con 0x7ff76415c6f0 2023-10-06T05:14:54.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff762ffd700 1 -- 172.21.15.45:0/1133320840 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c00ba20 con 0x7ff76415c6f0 2023-10-06T05:14:54.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff762ffd700 1 -- 172.21.15.45:0/1133320840 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0092e0 con 0x7ff76415c6f0 2023-10-06T05:14:54.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 -- 172.21.15.45:0/1133320840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76415c6f0 msgr2=0x7ff7640aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1133320840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76415c6f0 0x7ff7640aca70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7640cd5e0 tx=0x7ff75c009840 comp rx=0 tx=0).stop 2023-10-06T05:14:54.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 -- 172.21.15.45:0/1133320840 shutdown_connections 2023-10-06T05:14:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1133320840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76415c6f0 0x7ff7640aca70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1133320840 >> v2:172.21.15.144:3300/0 conn(0x7ff76406ae40 0x7ff76415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 -- 172.21.15.45:0/1133320840 >> 172.21.15.45:0/1133320840 conn(0x7ff7640cc570 msgr2=0x7ff7640cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 -- 172.21.15.45:0/1133320840 shutdown_connections 2023-10-06T05:14:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.759+0000 7ff76a47b700 1 -- 172.21.15.45:0/1133320840 wait complete. 2023-10-06T05:14:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.760+0000 7ff76a47b700 1 Processor -- start 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff76a47b700 1 -- start start 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff76a47b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff76406aa70 0x7ff764157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff76a47b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76406ae40 0x7ff764159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff76a47b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415c6f0 0x7ff76414b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff76a47b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640c0320 con 0x7ff76406ae40 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff76a47b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76415a2d0 con 0x7ff76415c6f0 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff76a47b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640afaa0 con 0x7ff76406aa70 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff7637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76406ae40 0x7ff764159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff7637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76406ae40 0x7ff764159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34648/0 (socket says 172.21.15.45:34648) 2023-10-06T05:14:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff7637fe700 1 -- 172.21.15.45:0/1365104918 learned_addr learned my addr 172.21.15.45:0/1365104918 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.761+0000 7ff768a18700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415c6f0 0x7ff76414b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff763fff700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff76406aa70 0x7ff764157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff7637fe700 1 -- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff76406aa70 msgr2=0x7ff764157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff7637fe700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff76406aa70 0x7ff764157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff7637fe700 1 -- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415c6f0 msgr2=0x7ff76414b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff7637fe700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415c6f0 0x7ff76414b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff7637fe700 1 -- 172.21.15.45:0/1365104918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c009ae0 con 0x7ff76406ae40 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff768a18700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415c6f0 0x7ff76414b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff7637fe700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76406ae40 0x7ff764159810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff75800ecb0 tx=0x7ff7580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff74bfff700 1 -- 172.21.15.45:0/1365104918 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75800c850 con 0x7ff76406ae40 2023-10-06T05:14:54.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff74bfff700 1 -- 172.21.15.45:0/1365104918 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75800c9b0 con 0x7ff76406ae40 2023-10-06T05:14:54.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff764159d50 con 0x7ff76406ae40 2023-10-06T05:14:54.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff74bfff700 1 -- 172.21.15.45:0/1365104918 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75800cd50 con 0x7ff76406ae40 2023-10-06T05:14:54.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.762+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7640af450 con 0x7ff76406ae40 2023-10-06T05:14:54.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.763+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76406e300 con 0x7ff76406ae40 2023-10-06T05:14:54.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.764+0000 7ff74bfff700 1 -- 172.21.15.45:0/1365104918 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7ff758013680 con 0x7ff76406ae40 2023-10-06T05:14:54.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.764+0000 7ff74bfff700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff750066ea0 0x7ff750069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:54.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.764+0000 7ff74bfff700 1 -- 172.21.15.45:0/1365104918 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff7580844b0 con 0x7ff76406ae40 2023-10-06T05:14:54.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.764+0000 7ff763fff700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff750066ea0 0x7ff750069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:54.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.765+0000 7ff763fff700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff750066ea0 0x7ff750069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff754003a80 tx=0x7ff754007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:54.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.769+0000 7ff74bfff700 1 -- 172.21.15.45:0/1365104918 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7580519f0 con 0x7ff76406ae40 2023-10-06T05:14:54.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.975+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff7640afbe0 con 0x7ff76406ae40 2023-10-06T05:14:54.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.976+0000 7ff74bfff700 1 -- 172.21.15.45:0/1365104918 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff758055010 con 0x7ff76406ae40 2023-10-06T05:14:54.978 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:14:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff750066ea0 msgr2=0x7ff750069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff750066ea0 0x7ff750069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff754003a80 tx=0x7ff754007ec0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76406ae40 msgr2=0x7ff764159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76406ae40 0x7ff764159810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff75800ecb0 tx=0x7ff7580091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 shutdown_connections 2023-10-06T05:14:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff750066ea0 0x7ff750069350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415c6f0 0x7ff76414b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff76406aa70 0x7ff764157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 --2- 172.21.15.45:0/1365104918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff76406ae40 0x7ff764159810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 >> 172.21.15.45:0/1365104918 conn(0x7ff7640cc570 msgr2=0x7ff7640ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 shutdown_connections 2023-10-06T05:14:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:54.978+0000 7ff76a47b700 1 -- 172.21.15.45:0/1365104918 wait complete. 2023-10-06T05:14:54.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:14:55.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.130+0000 7fe6f41e7700 1 Processor -- start 2023-10-06T05:14:55.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.130+0000 7fe6f41e7700 1 -- start start 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f41e7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6ec0c8ff0 0x7fe6ec0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f41e7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ec15b000 con 0x7fe6ec157200 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f41e7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ec15b2e0 con 0x7fe6ec0caad0 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f41e7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ec15b5c0 con 0x7fe6ec0c8ff0 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f1f83700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6ec0c8ff0 0x7fe6ec0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f2784700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe6ec157200 0x7fe6ec0cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33736/0 (socket says 172.21.15.45:33736) 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f1f83700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6ec0c8ff0 0x7fe6ec0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34662/0 (socket says 172.21.15.45:34662) 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f2784700 1 -- 172.21.15.45:0/3019035191 learned_addr learned my addr 172.21.15.45:0/3019035191 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f1f83700 1 -- 172.21.15.45:0/3019035191 >> v1:172.21.15.144:6789/0 conn(0x7fe6ec0caad0 legacy=0x7fe6ec156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f0f81700 1 -- 172.21.15.45:0/3019035191 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240957025 0 0) 0x7fe6ec15b2e0 con 0x7fe6ec0caad0 2023-10-06T05:14:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.131+0000 7fe6f1f83700 1 -- 172.21.15.45:0/3019035191 >> v1:172.21.15.45:6789/0 conn(0x7fe6ec157200 legacy=0x7fe6ec0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f1f83700 1 -- 172.21.15.45:0/3019035191 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec15b8a0 con 0x7fe6ec0c8ff0 2023-10-06T05:14:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f1f83700 1 --2- 172.21.15.45:0/3019035191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec0c8ff0 0x7fe6ec0c93c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80098e0 tx=0x7fe6e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbc3d209fb307c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f0f81700 1 -- 172.21.15.45:0/3019035191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8009e80 con 0x7fe6ec0c8ff0 2023-10-06T05:14:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f0f81700 1 -- 172.21.15.45:0/3019035191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e800b760 con 0x7fe6ec0c8ff0 2023-10-06T05:14:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f0f81700 1 -- 172.21.15.45:0/3019035191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800bae0 con 0x7fe6ec0c8ff0 2023-10-06T05:14:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3019035191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec0c8ff0 msgr2=0x7fe6ec0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3019035191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec0c8ff0 0x7fe6ec0c93c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80098e0 tx=0x7fe6e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3019035191 shutdown_connections 2023-10-06T05:14:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.132+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3019035191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec0c8ff0 0x7fe6ec0c93c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.133+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3019035191 >> 172.21.15.45:0/3019035191 conn(0x7fe6ec0cdf50 msgr2=0x7fe6ec0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.133+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3019035191 shutdown_connections 2023-10-06T05:14:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.133+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3019035191 wait complete. 2023-10-06T05:14:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.134+0000 7fe6f41e7700 1 Processor -- start 2023-10-06T05:14:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.134+0000 7fe6f41e7700 1 -- start start 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f41e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6ec0c8ff0 0x7fe6ec159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f41e7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6ec0caad0 0x7fe6ec0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f41e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec157200 0x7fe6ec0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f41e7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0c7c00 con 0x7fe6ec157200 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f41e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0ae550 con 0x7fe6ec0c8ff0 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f41e7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0ae7f0 con 0x7fe6ec0caad0 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec157200 0x7fe6ec0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec157200 0x7fe6ec0c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34678/0 (socket says 172.21.15.45:34678) 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 -- 172.21.15.45:0/3656065223 learned_addr learned my addr 172.21.15.45:0/3656065223 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f1f83700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6ec0c8ff0 0x7fe6ec159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f1782700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6ec0caad0 0x7fe6ec0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 -- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6ec0caad0 msgr2=0x7fe6ec0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6ec0caad0 0x7fe6ec0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 -- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6ec0c8ff0 msgr2=0x7fe6ec159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6ec0c8ff0 0x7fe6ec159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.135+0000 7fe6f2784700 1 -- 172.21.15.45:0/3656065223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8009420 con 0x7fe6ec157200 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.136+0000 7fe6f2784700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec157200 0x7fe6ec0c76c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400cd50 tx=0x7fe6e4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.136+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ce50 con 0x7fe6ec157200 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.136+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400aac0 con 0x7fe6ec157200 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.136+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ec0aead0 con 0x7fe6ec157200 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.136+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ae80 con 0x7fe6ec157200 2023-10-06T05:14:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.136+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec0af090 con 0x7fe6ec157200 2023-10-06T05:14:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.137+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ec0af2b0 con 0x7fe6ec157200 2023-10-06T05:14:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.137+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fe6e4007300 con 0x7fe6ec157200 2023-10-06T05:14:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.138+0000 7fe6e1ffb700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe6d0066ea0 0x7fe6d0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.138+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe6e40845e0 con 0x7fe6ec157200 2023-10-06T05:14:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.138+0000 7fe6f1f83700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe6d0066ea0 0x7fe6d0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.138+0000 7fe6f1f83700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe6d0066ea0 0x7fe6d0069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80030c0 tx=0x7fe6e8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.143+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6e4051ba0 con 0x7fe6ec157200 2023-10-06T05:14:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.202+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6e4051f70 con 0x7fe6ec157200 2023-10-06T05:14:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.395+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6ec04bb40 con 0x7fe6ec157200 2023-10-06T05:14:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.396+0000 7fe6e1ffb700 1 -- 172.21.15.45:0/3656065223 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7fe6e4055630 con 0x7fe6ec157200 2023-10-06T05:14:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:55.398 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-06T05:14:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe6d0066ea0 msgr2=0x7fe6d0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe6d0066ea0 0x7fe6d0069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80030c0 tx=0x7fe6e8016000 comp rx=0 tx=0).stop 2023-10-06T05:14:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec157200 msgr2=0x7fe6ec0c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec157200 0x7fe6ec0c76c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400cd50 tx=0x7fe6e4001ab0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 shutdown_connections 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe6d0066ea0 0x7fe6d0069350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec157200 0x7fe6ec0c76c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6ec0c8ff0 0x7fe6ec159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 --2- 172.21.15.45:0/3656065223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6ec0caad0 0x7fe6ec0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 >> 172.21.15.45:0/3656065223 conn(0x7fe6ec0cdf50 msgr2=0x7fe6ec14a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 shutdown_connections 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.398+0000 7fe6f41e7700 1 -- 172.21.15.45:0/3656065223 wait complete. 2023-10-06T05:14:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2023-10-06T05:14:55.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:14:55.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.544+0000 7fbb96058700 1 Processor -- start 2023-10-06T05:14:55.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.544+0000 7fbb96058700 1 -- start start 2023-10-06T05:14:55.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb96058700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbb901539f0 0x7fbb9015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb96058700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb9015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb96058700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb900ce690 con 0x7fbb90153620 2023-10-06T05:14:55.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb96058700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9015a7a0 con 0x7fbb901539f0 2023-10-06T05:14:55.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb96058700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900ce3f0 con 0x7fbb900ccfc0 2023-10-06T05:14:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb8ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb9015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb8effd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbb901539f0 0x7fbb9015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb8effd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbb901539f0 0x7fbb9015a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:34450/0 (socket says 172.21.15.45:34450) 2023-10-06T05:14:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb8ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb9015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:37880/0 (socket says 172.21.15.45:37880) 2023-10-06T05:14:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb8effd700 1 -- 172.21.15.45:0/178586905 learned_addr learned my addr 172.21.15.45:0/178586905 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.545+0000 7fbb8f7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbb90153620 0x7fbb900cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:44076/0 (socket says 172.21.15.45:44076) 2023-10-06T05:14:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8ffff700 1 -- 172.21.15.45:0/178586905 >> v2:172.21.15.144:3300/0 conn(0x7fbb901539f0 msgr2=0x7fbb9015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8ffff700 1 --2- 172.21.15.45:0/178586905 >> v2:172.21.15.144:3300/0 conn(0x7fbb901539f0 0x7fbb9015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8ffff700 1 -- 172.21.15.45:0/178586905 >> v1:172.21.15.144:6789/0 conn(0x7fbb90153620 legacy=0x7fbb900cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8ffff700 1 -- 172.21.15.45:0/178586905 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9015d510 con 0x7fbb900ccfc0 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8effd700 1 --2- 172.21.15.45:0/178586905 >> v2:172.21.15.144:3300/0 conn(0x7fbb901539f0 0x7fbb9015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8ffff700 1 --2- 172.21.15.45:0/178586905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb9015cdc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009e60 tx=0x7fbb88009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4567288ed993f27a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8e7fc700 1 -- 172.21.15.45:0/178586905 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800e070 con 0x7fbb900ccfc0 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8e7fc700 1 -- 172.21.15.45:0/178586905 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8800bb60 con 0x7fbb900ccfc0 2023-10-06T05:14:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.546+0000 7fbb8e7fc700 1 -- 172.21.15.45:0/178586905 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88012430 con 0x7fbb900ccfc0 2023-10-06T05:14:55.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 -- 172.21.15.45:0/178586905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 msgr2=0x7fbb9015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 --2- 172.21.15.45:0/178586905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb9015cdc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009e60 tx=0x7fbb88009780 comp rx=0 tx=0).stop 2023-10-06T05:14:55.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 -- 172.21.15.45:0/178586905 shutdown_connections 2023-10-06T05:14:55.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 --2- 172.21.15.45:0/178586905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb9015cdc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 --2- 172.21.15.45:0/178586905 >> v2:172.21.15.144:3300/0 conn(0x7fbb901539f0 0x7fbb9015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 -- 172.21.15.45:0/178586905 >> 172.21.15.45:0/178586905 conn(0x7fbb9006ed50 msgr2=0x7fbb9006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 -- 172.21.15.45:0/178586905 shutdown_connections 2023-10-06T05:14:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.547+0000 7fbb96058700 1 -- 172.21.15.45:0/178586905 wait complete. 2023-10-06T05:14:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.548+0000 7fbb96058700 1 Processor -- start 2023-10-06T05:14:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.548+0000 7fbb96058700 1 -- start start 2023-10-06T05:14:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb96058700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb90155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb96058700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb90153620 0x7fbb900c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb96058700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb90154390 0x7fbb900c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb96058700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900b46c0 con 0x7fbb90153620 2023-10-06T05:14:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb96058700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb901562f0 con 0x7fbb90154390 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb96058700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb901565d0 con 0x7fbb900ccfc0 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb90153620 0x7fbb900c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb90153620 0x7fbb900c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34690/0 (socket says 172.21.15.45:34690) 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb8effd700 1 -- 172.21.15.45:0/3341695412 learned_addr learned my addr 172.21.15.45:0/3341695412 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb8f7fe700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb90155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.549+0000 7fbb8ffff700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb90154390 0x7fbb900c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb8effd700 1 -- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 msgr2=0x7fbb90155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb8effd700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb90155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb8effd700 1 -- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb90154390 msgr2=0x7fbb900c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb8effd700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb90154390 0x7fbb900c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb8effd700 1 -- 172.21.15.45:0/3341695412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840094f0 con 0x7fbb90153620 2023-10-06T05:14:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb8effd700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb90153620 0x7fbb900c59e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400ec80 tx=0x7fbb84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb777fe700 1 -- 172.21.15.45:0/3341695412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400c850 con 0x7fbb90153620 2023-10-06T05:14:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb777fe700 1 -- 172.21.15.45:0/3341695412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400c9b0 con 0x7fbb90153620 2023-10-06T05:14:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb88009a20 con 0x7fbb90153620 2023-10-06T05:14:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb777fe700 1 -- 172.21.15.45:0/3341695412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400cd30 con 0x7fbb90153620 2023-10-06T05:14:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.550+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb90156b90 con 0x7fbb90153620 2023-10-06T05:14:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.551+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb900b4830 con 0x7fbb90153620 2023-10-06T05:14:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.552+0000 7fbb777fe700 1 -- 172.21.15.45:0/3341695412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb84013680 con 0x7fbb90153620 2023-10-06T05:14:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.552+0000 7fbb777fe700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbb7c066f30 0x7fbb7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.552+0000 7fbb777fe700 1 -- 172.21.15.45:0/3341695412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbb84084550 con 0x7fbb90153620 2023-10-06T05:14:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.552+0000 7fbb8f7fe700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbb7c066f30 0x7fbb7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.553+0000 7fbb8f7fe700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbb7c066f30 0x7fbb7c0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb80004fd0 tx=0x7fbb80007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.557+0000 7fbb777fe700 1 -- 172.21.15.45:0/3341695412 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb840519b0 con 0x7fbb90153620 2023-10-06T05:14:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.757+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb900b4490 con 0x7fbb90153620 2023-10-06T05:14:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.758+0000 7fbb777fe700 1 -- 172.21.15.45:0/3341695412 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13743 (secure 0 0 0) 0x7fbb84054fd0 con 0x7fbb90153620 2023-10-06T05:14:55.760 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:55.760 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":16,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:14:54.912639+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:14:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbb7c066f30 msgr2=0x7fbb7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbb7c066f30 0x7fbb7c0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb80004fd0 tx=0x7fbb80007eb0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb90153620 msgr2=0x7fbb900c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb90153620 0x7fbb900c59e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400ec80 tx=0x7fbb84009250 comp rx=0 tx=0).stop 2023-10-06T05:14:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 shutdown_connections 2023-10-06T05:14:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbb7c066f30 0x7fbb7c0693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb90154390 0x7fbb900c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb900ccfc0 0x7fbb90155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 --2- 172.21.15.45:0/3341695412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb90153620 0x7fbb900c59e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 >> 172.21.15.45:0/3341695412 conn(0x7fbb9006ed50 msgr2=0x7fbb90151e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:55.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 shutdown_connections 2023-10-06T05:14:55.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.760+0000 7fbb96058700 1 -- 172.21.15.45:0/3341695412 wait complete. 2023-10-06T05:14:55.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:14:55.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:14:55.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.909+0000 7f71db93b700 1 Processor -- start 2023-10-06T05:14:55.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.909+0000 7f71db93b700 1 -- start start 2023-10-06T05:14:55.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71db93b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71d40b0610 0x7f71d40c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71db93b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40ce6a0 con 0x7f71d4069ed0 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71db93b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d4159880 con 0x7f71d40c3db0 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71db93b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d40690f0 con 0x7f71d40b0610 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71d9ed8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f71d4069ed0 0x7f71d40cc350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33744/0 (socket says 172.21.15.45:33744) 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71d9ed8700 1 -- 172.21.15.45:0/2214216007 learned_addr learned my addr 172.21.15.45:0/2214216007 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71d96d7700 1 --2- 172.21.15.45:0/2214216007 >> v2:172.21.15.144:3300/0 conn(0x7f71d40b0610 0x7f71d40c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71cbfff700 1 -- 172.21.15.45:0/2214216007 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301489830 0 0) 0x7f71d40ce6a0 con 0x7f71d4069ed0 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71cbfff700 1 -- 172.21.15.45:0/2214216007 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c0003570 con 0x7f71d4069ed0 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71d96d7700 1 -- 172.21.15.45:0/2214216007 >> v1:172.21.15.144:6789/0 conn(0x7f71d40c3db0 legacy=0x7f71d40697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.910+0000 7f71cbfff700 1 -- 172.21.15.45:0/2214216007 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 977989371 0 0) 0x7f71c0003570 con 0x7f71d4069ed0 2023-10-06T05:14:55.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71d96d7700 1 -- 172.21.15.45:0/2214216007 >> v1:172.21.15.45:6789/0 conn(0x7f71d4069ed0 legacy=0x7f71d40cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71d96d7700 1 -- 172.21.15.45:0/2214216007 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40692e0 con 0x7f71d40b0610 2023-10-06T05:14:55.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71d96d7700 1 --2- 172.21.15.45:0/2214216007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 0x7f71d40c85d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71d00098d0 tx=0x7f71d0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c65c1072731b5fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71cbfff700 1 -- 172.21.15.45:0/2214216007 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0017f0 con 0x7f71d40b0610 2023-10-06T05:14:55.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71cbfff700 1 -- 172.21.15.45:0/2214216007 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d0011420 con 0x7f71d40b0610 2023-10-06T05:14:55.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71cbfff700 1 -- 172.21.15.45:0/2214216007 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d00117e0 con 0x7f71d40b0610 2023-10-06T05:14:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71db93b700 1 -- 172.21.15.45:0/2214216007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 msgr2=0x7f71d40c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71db93b700 1 --2- 172.21.15.45:0/2214216007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 0x7f71d40c85d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71d00098d0 tx=0x7f71d0009120 comp rx=0 tx=0).stop 2023-10-06T05:14:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71db93b700 1 -- 172.21.15.45:0/2214216007 shutdown_connections 2023-10-06T05:14:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.911+0000 7f71db93b700 1 --2- 172.21.15.45:0/2214216007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 0x7f71d40c85d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.912+0000 7f71db93b700 1 -- 172.21.15.45:0/2214216007 >> 172.21.15.45:0/2214216007 conn(0x7f71d4067e00 msgr2=0x7f71d40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.912+0000 7f71db93b700 1 -- 172.21.15.45:0/2214216007 shutdown_connections 2023-10-06T05:14:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.912+0000 7f71db93b700 1 -- 172.21.15.45:0/2214216007 wait complete. 2023-10-06T05:14:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.913+0000 7f71db93b700 1 Processor -- start 2023-10-06T05:14:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.913+0000 7f71db93b700 1 -- start start 2023-10-06T05:14:55.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71db93b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d4069ed0 0x7f71d416d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71db93b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 0x7f71d4156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71db93b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71d40c3db0 0x7f71d416a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71db93b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d416dec0 con 0x7f71d4069ed0 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71db93b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4169d50 con 0x7f71d40b0610 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71db93b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40b1050 con 0x7f71d40c3db0 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71d96d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d4069ed0 0x7f71d416d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71d96d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d4069ed0 0x7f71d416d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34698/0 (socket says 172.21.15.45:34698) 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71d96d7700 1 -- 172.21.15.45:0/4051423153 learned_addr learned my addr 172.21.15.45:0/4051423153 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71d8ed6700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 0x7f71d4156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.914+0000 7f71d9ed8700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71d40c3db0 0x7f71d416a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.915+0000 7f71d96d7700 1 -- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71d40c3db0 msgr2=0x7f71d416a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.915+0000 7f71d96d7700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71d40c3db0 0x7f71d416a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.915+0000 7f71d96d7700 1 -- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 msgr2=0x7f71d4156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.915+0000 7f71d96d7700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 0x7f71d4156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.915+0000 7f71d96d7700 1 -- 172.21.15.45:0/4051423153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d00093c0 con 0x7f71d4069ed0 2023-10-06T05:14:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.915+0000 7f71d96d7700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d4069ed0 0x7f71d416d980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71d406bac0 tx=0x7f71d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.917+0000 7f71c97fa700 1 -- 172.21.15.45:0/4051423153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0010a40 con 0x7f71d4069ed0 2023-10-06T05:14:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.917+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d40b1330 con 0x7f71d4069ed0 2023-10-06T05:14:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.917+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d4156fe0 con 0x7f71d4069ed0 2023-10-06T05:14:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.917+0000 7f71c97fa700 1 -- 172.21.15.45:0/4051423153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d0010db0 con 0x7f71d4069ed0 2023-10-06T05:14:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.917+0000 7f71c97fa700 1 -- 172.21.15.45:0/4051423153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0005a00 con 0x7f71d4069ed0 2023-10-06T05:14:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.918+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b40052b0 con 0x7f71d4069ed0 2023-10-06T05:14:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.919+0000 7f71c97fa700 1 -- 172.21.15.45:0/4051423153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71d0006030 con 0x7f71d4069ed0 2023-10-06T05:14:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.919+0000 7f71c97fa700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71b8066f30 0x7f71b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.920+0000 7f71c97fa700 1 -- 172.21.15.45:0/4051423153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f71d0086720 con 0x7f71d4069ed0 2023-10-06T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.920+0000 7f71d8ed6700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71b8066f30 0x7f71b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.920+0000 7f71d8ed6700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71b8066f30 0x7f71b80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71d40b3d20 tx=0x7f71c4007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:55.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:55.923+0000 7f71c97fa700 1 -- 172.21.15.45:0/4051423153 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71d0053ae0 con 0x7f71d4069ed0 2023-10-06T05:14:56.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.126+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71b4005ae0 con 0x7f71d4069ed0 2023-10-06T05:14:56.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.127+0000 7f71c97fa700 1 -- 172.21.15.45:0/4051423153 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71d0057100 con 0x7f71d4069ed0 2023-10-06T05:14:56.129 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:14:56.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71b8066f30 msgr2=0x7f71b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71b8066f30 0x7f71b80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71d40b3d20 tx=0x7f71c4007ec0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d4069ed0 msgr2=0x7f71d416d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d4069ed0 0x7f71d416d980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71d406bac0 tx=0x7f71d0017000 comp rx=0 tx=0).stop 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 shutdown_connections 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71b8066f30 0x7f71b80693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71d40c3db0 0x7f71d416a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71d4069ed0 0x7f71d416d980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 --2- 172.21.15.45:0/4051423153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d40b0610 0x7f71d4156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 >> 172.21.15.45:0/4051423153 conn(0x7f71d4067e00 msgr2=0x7f71d4158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 shutdown_connections 2023-10-06T05:14:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.129+0000 7f71db93b700 1 -- 172.21.15.45:0/4051423153 wait complete. 2023-10-06T05:14:56.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:14:56.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.277+0000 7fc3c7969700 1 Processor -- start 2023-10-06T05:14:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.277+0000 7fc3c7969700 1 -- start start 2023-10-06T05:14:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c7969700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc3c0067d30 0x7fc3c0168640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c7969700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c00c8770 con 0x7fc3c0147210 2023-10-06T05:14:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c7969700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c00cbd60 con 0x7fc3c0146e40 2023-10-06T05:14:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c7969700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c00cc030 con 0x7fc3c0067d30 2023-10-06T05:14:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c4f04700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc3c0147210 0x7fc3c0067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33752/0 (socket says 172.21.15.45:33752) 2023-10-06T05:14:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c4f04700 1 -- 172.21.15.45:0/3173029694 learned_addr learned my addr 172.21.15.45:0/3173029694 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c5f06700 1 --2- 172.21.15.45:0/3173029694 >> v2:172.21.15.144:3300/0 conn(0x7fc3c0067d30 0x7fc3c0168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3b3fff700 1 -- 172.21.15.45:0/3173029694 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205328082 0 0) 0x7fc3c00c8770 con 0x7fc3c0147210 2023-10-06T05:14:56.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3b3fff700 1 -- 172.21.15.45:0/3173029694 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3ac003570 con 0x7fc3c0147210 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3b3fff700 1 -- 172.21.15.45:0/3173029694 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726599683 0 0) 0x7fc3ac003570 con 0x7fc3c0147210 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c5f06700 1 -- 172.21.15.45:0/3173029694 >> v1:172.21.15.144:6789/0 conn(0x7fc3c0146e40 legacy=0x7fc3c00cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c5f06700 1 -- 172.21.15.45:0/3173029694 >> v1:172.21.15.45:6789/0 conn(0x7fc3c0147210 legacy=0x7fc3c0067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.278+0000 7fc3c5f06700 1 -- 172.21.15.45:0/3173029694 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00c8120 con 0x7fc3c0067d30 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3c5f06700 1 --2- 172.21.15.45:0/3173029694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 0x7fc3c0168640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009e60 tx=0x7fc3b8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8400dd1899a03d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3b3fff700 1 -- 172.21.15.45:0/3173029694 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b800e070 con 0x7fc3c0067d30 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3b3fff700 1 -- 172.21.15.45:0/3173029694 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b800b850 con 0x7fc3c0067d30 2023-10-06T05:14:56.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3b3fff700 1 -- 172.21.15.45:0/3173029694 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4001470 con 0x7fc3c0067d30 2023-10-06T05:14:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3c7969700 1 -- 172.21.15.45:0/3173029694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 msgr2=0x7fc3c0168640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3c7969700 1 --2- 172.21.15.45:0/3173029694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 0x7fc3c0168640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009e60 tx=0x7fc3b8009780 comp rx=0 tx=0).stop 2023-10-06T05:14:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3c7969700 1 -- 172.21.15.45:0/3173029694 shutdown_connections 2023-10-06T05:14:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3c7969700 1 --2- 172.21.15.45:0/3173029694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 0x7fc3c0168640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3c7969700 1 -- 172.21.15.45:0/3173029694 >> 172.21.15.45:0/3173029694 conn(0x7fc3c00691c0 msgr2=0x7fc3c00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.279+0000 7fc3c7969700 1 -- 172.21.15.45:0/3173029694 shutdown_connections 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.280+0000 7fc3c7969700 1 -- 172.21.15.45:0/3173029694 wait complete. 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.281+0000 7fc3c7969700 1 Processor -- start 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.281+0000 7fc3c7969700 1 -- start start 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c7969700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 0x7fc3c016d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c7969700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c0146e40 0x7fc3c0141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c7969700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3c0147210 0x7fc3c00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c7969700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0141dd0 con 0x7fc3c0146e40 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c7969700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c01420b0 con 0x7fc3c0067d30 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c7969700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0142390 con 0x7fc3c0147210 2023-10-06T05:14:56.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c0146e40 0x7fc3c0141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c0146e40 0x7fc3c0141360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34710/0 (socket says 172.21.15.45:34710) 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 -- 172.21.15.45:0/1579787245 learned_addr learned my addr 172.21.15.45:0/1579787245 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c5705700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 0x7fc3c016d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c5f06700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3c0147210 0x7fc3c00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 -- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3c0147210 msgr2=0x7fc3c00c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3c0147210 0x7fc3c00c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 -- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 msgr2=0x7fc3c016d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 0x7fc3c016d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 -- 172.21.15.45:0/1579787245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8009a20 con 0x7fc3c0146e40 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c5f06700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3c0147210 0x7fc3c00c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c4f04700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c0146e40 0x7fc3c0141360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400cb90 tx=0x7fc3b4001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3b17fa700 1 -- 172.21.15.45:0/1579787245 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4001d10 con 0x7fc3c0146e40 2023-10-06T05:14:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3b17fa700 1 -- 172.21.15.45:0/1579787245 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b4007360 con 0x7fc3c0146e40 2023-10-06T05:14:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c00afd60 con 0x7fc3c0146e40 2023-10-06T05:14:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.282+0000 7fc3b17fa700 1 -- 172.21.15.45:0/1579787245 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4004830 con 0x7fc3c0146e40 2023-10-06T05:14:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.283+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c016d810 con 0x7fc3c0146e40 2023-10-06T05:14:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.283+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c006e2d0 con 0x7fc3c0146e40 2023-10-06T05:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.284+0000 7fc3b17fa700 1 -- 172.21.15.45:0/1579787245 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3b4004ba0 con 0x7fc3c0146e40 2023-10-06T05:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.284+0000 7fc3b17fa700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc3a4066f30 0x7fc3a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.284+0000 7fc3b17fa700 1 -- 172.21.15.45:0/1579787245 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc3b40839d0 con 0x7fc3c0146e40 2023-10-06T05:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.284+0000 7fc3c5705700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc3a4066f30 0x7fc3a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.285+0000 7fc3c5705700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc3a4066f30 0x7fc3a40693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc003a80 tx=0x7fc3bc007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.289+0000 7fc3b17fa700 1 -- 172.21.15.45:0/1579787245 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3b4050eb0 con 0x7fc3c0146e40 2023-10-06T05:14:56.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.489+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3c004bb40 con 0x7fc3c0146e40 2023-10-06T05:14:56.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.489+0000 7fc3b17fa700 1 -- 172.21.15.45:0/1579787245 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc3b4018020 con 0x7fc3c0146e40 2023-10-06T05:14:56.491 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:14:56.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc3a4066f30 msgr2=0x7fc3a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc3a4066f30 0x7fc3a40693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc003a80 tx=0x7fc3bc007ec0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c0146e40 msgr2=0x7fc3c0141360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c0146e40 0x7fc3c0141360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400cb90 tx=0x7fc3b4001aa0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 shutdown_connections 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc3a4066f30 0x7fc3a40693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3c0147210 0x7fc3c00c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3c0067d30 0x7fc3c016d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 --2- 172.21.15.45:0/1579787245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c0146e40 0x7fc3c0141360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 >> 172.21.15.45:0/1579787245 conn(0x7fc3c00691c0 msgr2=0x7fc3c00c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.491+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 shutdown_connections 2023-10-06T05:14:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.492+0000 7fc3c7969700 1 -- 172.21.15.45:0/1579787245 wait complete. 2023-10-06T05:14:56.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:14:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.637+0000 7fa7d90d3700 1 Processor -- start 2023-10-06T05:14:56.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.637+0000 7fa7d90d3700 1 -- start start 2023-10-06T05:14:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d90d3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7d4156ce0 0x7fa7d41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d90d3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d406bad0 0x7fa7d415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d90d3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa7d40afbe0 0x7fa7d40ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d90d3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d415b150 con 0x7fa7d4156ce0 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d90d3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d406bf40 con 0x7fa7d40afbe0 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d90d3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d415b380 con 0x7fa7d406bad0 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d2d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7d4156ce0 0x7fa7d41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d2d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7d4156ce0 0x7fa7d41570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42300/0 (socket says 172.21.15.45:42300) 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.638+0000 7fa7d2d9d700 1 -- 172.21.15.45:0/1756963713 learned_addr learned my addr 172.21.15.45:0/1756963713 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d359e700 1 --2- 172.21.15.45:0/1756963713 >> v2:172.21.15.144:3300/0 conn(0x7fa7d40afbe0 0x7fa7d40ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d259c700 1 --2- 172.21.15.45:0/1756963713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d406bad0 0x7fa7d415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d2d9d700 1 -- 172.21.15.45:0/1756963713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d406bad0 msgr2=0x7fa7d415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:56.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d2d9d700 1 --2- 172.21.15.45:0/1756963713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d406bad0 0x7fa7d415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d2d9d700 1 -- 172.21.15.45:0/1756963713 >> v2:172.21.15.144:3300/0 conn(0x7fa7d40afbe0 msgr2=0x7fa7d40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d2d9d700 1 --2- 172.21.15.45:0/1756963713 >> v2:172.21.15.144:3300/0 conn(0x7fa7d40afbe0 0x7fa7d40ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d2d9d700 1 -- 172.21.15.45:0/1756963713 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d415b660 con 0x7fa7d4156ce0 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d2d9d700 1 --2- 172.21.15.45:0/1756963713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41570b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40aec60 tx=0x7fa7c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e666a2931ef53ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d1d9b700 1 -- 172.21.15.45:0/1756963713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c400b860 con 0x7fa7d4156ce0 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d1d9b700 1 -- 172.21.15.45:0/1756963713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c400b9c0 con 0x7fa7d4156ce0 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.639+0000 7fa7d1d9b700 1 -- 172.21.15.45:0/1756963713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c400bd60 con 0x7fa7d4156ce0 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 -- 172.21.15.45:0/1756963713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 msgr2=0x7fa7d41570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/1756963713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41570b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40aec60 tx=0x7fa7c4009250 comp rx=0 tx=0).stop 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 -- 172.21.15.45:0/1756963713 shutdown_connections 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/1756963713 >> v2:172.21.15.144:3300/0 conn(0x7fa7d40afbe0 0x7fa7d40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/1756963713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41570b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/1756963713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d406bad0 0x7fa7d415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 -- 172.21.15.45:0/1756963713 >> 172.21.15.45:0/1756963713 conn(0x7fa7d406f090 msgr2=0x7fa7d406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 -- 172.21.15.45:0/1756963713 shutdown_connections 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.640+0000 7fa7d90d3700 1 -- 172.21.15.45:0/1756963713 wait complete. 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.641+0000 7fa7d90d3700 1 Processor -- start 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.641+0000 7fa7d90d3700 1 -- start start 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d90d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d406bad0 0x7fa7d40c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d90d3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d40afbe0 0x7fa7d40ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d90d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d90d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4167c80 con 0x7fa7d4156ce0 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d90d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4157b50 con 0x7fa7d406bad0 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d90d3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4157820 con 0x7fa7d40afbe0 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42308/0 (socket says 172.21.15.45:42308) 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 -- 172.21.15.45:0/2388575325 learned_addr learned my addr 172.21.15.45:0/2388575325 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d2d9d700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d406bad0 0x7fa7d40c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 -- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d40afbe0 msgr2=0x7fa7d40ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d259c700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d40afbe0 0x7fa7d40ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d40afbe0 0x7fa7d40ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 -- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d406bad0 msgr2=0x7fa7d40c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d406bad0 0x7fa7d40c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d359e700 1 -- 172.21.15.45:0/2388575325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40094f0 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.642+0000 7fa7d2d9d700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d406bad0 0x7fa7d40c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.643+0000 7fa7d359e700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41596d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009fa0 tx=0x7fa7cc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.643+0000 7fa7baffd700 1 -- 172.21.15.45:0/2388575325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00ed20 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.643+0000 7fa7baffd700 1 -- 172.21.15.45:0/2388575325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc00ee80 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.643+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d415a850 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.643+0000 7fa7baffd700 1 -- 172.21.15.45:0/2388575325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc014670 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.643+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d40c5500 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.644+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d406e2f0 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.644+0000 7fa7baffd700 1 -- 172.21.15.45:0/2388575325 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7cc0149e0 con 0x7fa7d4156ce0 2023-10-06T05:14:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.645+0000 7fa7baffd700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7c0066f30 0x7fa7c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.645+0000 7fa7baffd700 1 -- 172.21.15.45:0/2388575325 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa7cc084ac0 con 0x7fa7d4156ce0 2023-10-06T05:14:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.645+0000 7fa7d2d9d700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7c0066f30 0x7fa7c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.645+0000 7fa7d2d9d700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7c0066f30 0x7fa7c00693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40aec60 tx=0x7fa7c4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.649+0000 7fa7baffd700 1 -- 172.21.15.45:0/2388575325 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7cc051f20 con 0x7fa7d4156ce0 2023-10-06T05:14:56.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.853+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa7d40b24f0 con 0x7fa7d4156ce0 2023-10-06T05:14:56.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.854+0000 7fa7baffd700 1 -- 172.21.15.45:0/2388575325 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa7cc084e60 con 0x7fa7d4156ce0 2023-10-06T05:14:56.856 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:14:56.856 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:14:56.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:14:56.856 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-06T05:14:56.856 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-06T05:14:56.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:14:56.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:14:56.857 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-10-06T05:14:56.857 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-10-06T05:14:56.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:14:56.857 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:14:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7c0066f30 msgr2=0x7fa7c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7c0066f30 0x7fa7c00693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40aec60 tx=0x7fa7c4017000 comp rx=0 tx=0).stop 2023-10-06T05:14:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 msgr2=0x7fa7d41596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41596d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009fa0 tx=0x7fa7cc009900 comp rx=0 tx=0).stop 2023-10-06T05:14:56.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 shutdown_connections 2023-10-06T05:14:56.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7c0066f30 0x7fa7c00693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7d4156ce0 0x7fa7d41596d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d406bad0 0x7fa7d40c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 --2- 172.21.15.45:0/2388575325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7d40afbe0 0x7fa7d40ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:56.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 >> 172.21.15.45:0/2388575325 conn(0x7fa7d406f090 msgr2=0x7fa7d4068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:56.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 shutdown_connections 2023-10-06T05:14:56.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:56.856+0000 7fa7d90d3700 1 -- 172.21.15.45:0/2388575325 wait complete. 2023-10-06T05:14:56.871 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:14:56.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:14:57.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.007+0000 7f7c3fb97700 1 Processor -- start 2023-10-06T05:14:57.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.007+0000 7f7c3fb97700 1 -- start start 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.008+0000 7f7c3fb97700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3815d5a0 0x7f7c380c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.008+0000 7f7c3fb97700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c38068040 con 0x7f7c381590c0 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.008+0000 7f7c3fb97700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3815dcf0 con 0x7f7c38159490 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.008+0000 7f7c3fb97700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38067750 con 0x7f7c3815d5a0 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.008+0000 7f7c3d933700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7c381590c0 0x7f7c380ac5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44536/0 (socket says 172.21.15.45:44536) 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.008+0000 7f7c3d933700 1 -- 172.21.15.45:0/292553519 learned_addr learned my addr 172.21.15.45:0/292553519 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.008+0000 7f7c3e134700 1 --2- 172.21.15.45:0/292553519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3815d5a0 0x7f7c380c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3c931700 1 -- 172.21.15.45:0/292553519 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730715435 0 0) 0x7f7c38068040 con 0x7f7c381590c0 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3c931700 1 -- 172.21.15.45:0/292553519 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c24003570 con 0x7f7c381590c0 2023-10-06T05:14:57.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3c931700 1 -- 172.21.15.45:0/292553519 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514685008 0 0) 0x7f7c3815dcf0 con 0x7f7c38159490 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3c931700 1 -- 172.21.15.45:0/292553519 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c38068040 con 0x7f7c38159490 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3e134700 1 -- 172.21.15.45:0/292553519 >> v1:172.21.15.144:6789/0 conn(0x7f7c38159490 legacy=0x7f7c3815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3e134700 1 -- 172.21.15.45:0/292553519 >> v1:172.21.15.45:6789/0 conn(0x7f7c381590c0 legacy=0x7f7c380ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3e134700 1 -- 172.21.15.45:0/292553519 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380679f0 con 0x7f7c3815d5a0 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3e134700 1 --2- 172.21.15.45:0/292553519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3815d5a0 0x7f7c380c74f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009e90 tx=0x7f7c30009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18e9aa42c5d5250 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3c931700 1 -- 172.21.15.45:0/292553519 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000e070 con 0x7f7c3815d5a0 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3c931700 1 -- 172.21.15.45:0/292553519 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3000bb80 con 0x7f7c3815d5a0 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.009+0000 7f7c3c931700 1 -- 172.21.15.45:0/292553519 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30012430 con 0x7f7c3815d5a0 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.010+0000 7f7c3fb97700 1 -- 172.21.15.45:0/292553519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3815d5a0 msgr2=0x7f7c380c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.010+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/292553519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3815d5a0 0x7f7c380c74f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009e90 tx=0x7f7c30009780 comp rx=0 tx=0).stop 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.010+0000 7f7c3fb97700 1 -- 172.21.15.45:0/292553519 shutdown_connections 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.010+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/292553519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3815d5a0 0x7f7c380c74f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.010+0000 7f7c3fb97700 1 -- 172.21.15.45:0/292553519 >> 172.21.15.45:0/292553519 conn(0x7f7c380cd060 msgr2=0x7f7c380c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.010+0000 7f7c3fb97700 1 -- 172.21.15.45:0/292553519 shutdown_connections 2023-10-06T05:14:57.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.010+0000 7f7c3fb97700 1 -- 172.21.15.45:0/292553519 wait complete. 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.011+0000 7f7c3fb97700 1 Processor -- start 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.012+0000 7f7c3fb97700 1 -- start start 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3fb97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c381590c0 0x7f7c38155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3fb97700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c38159490 0x7f7c3815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3fb97700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3815d5a0 0x7f7c38154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3fb97700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c1320 con 0x7f7c3815d5a0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3fb97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c1ea0 con 0x7f7c381590c0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3fb97700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c1570 con 0x7f7c38159490 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3815d5a0 0x7f7c38154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3815d5a0 0x7f7c38154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42316/0 (socket says 172.21.15.45:42316) 2023-10-06T05:14:57.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 -- 172.21.15.45:0/2249068401 learned_addr learned my addr 172.21.15.45:0/2249068401 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3d933700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c381590c0 0x7f7c38155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3d132700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c38159490 0x7f7c3815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 -- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c38159490 msgr2=0x7f7c3815b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c38159490 0x7f7c3815b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 -- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c381590c0 msgr2=0x7f7c38155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c381590c0 0x7f7c38155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 -- 172.21.15.45:0/2249068401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009a20 con 0x7f7c3815d5a0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3d933700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c381590c0 0x7f7c38155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3e134700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3815d5a0 0x7f7c38154640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c30013000 tx=0x7f7c30019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c2dffb700 1 -- 172.21.15.45:0/2249068401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3001b420 con 0x7f7c3815d5a0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c2dffb700 1 -- 172.21.15.45:0/2249068401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3001b5a0 con 0x7f7c3815d5a0 2023-10-06T05:14:57.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c2dffb700 1 -- 172.21.15.45:0/2249068401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3001b960 con 0x7f7c3815d5a0 2023-10-06T05:14:57.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.013+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c380c1850 con 0x7f7c3815d5a0 2023-10-06T05:14:57.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.014+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c38156160 con 0x7f7c3815d5a0 2023-10-06T05:14:57.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.015+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c380c2050 con 0x7f7c3815d5a0 2023-10-06T05:14:57.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.015+0000 7f7c2dffb700 1 -- 172.21.15.45:0/2249068401 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c3001c400 con 0x7f7c3815d5a0 2023-10-06T05:14:57.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.015+0000 7f7c2dffb700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c1c066f30 0x7f7c1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:57.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.016+0000 7f7c2dffb700 1 -- 172.21.15.45:0/2249068401 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7c30085ce0 con 0x7f7c3815d5a0 2023-10-06T05:14:57.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.016+0000 7f7c3d933700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c1c066f30 0x7f7c1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:57.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.016+0000 7f7c3d933700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c1c066f30 0x7f7c1c0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c340058d0 tx=0x7f7c34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:57.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.020+0000 7f7c2dffb700 1 -- 172.21.15.45:0/2249068401 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c300530a0 con 0x7f7c3815d5a0 2023-10-06T05:14:57.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.221+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7c3804c380 con 0x7f7c3815d5a0 2023-10-06T05:14:57.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.927+0000 7f7c2dffb700 1 -- 172.21.15.45:0/2249068401 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7c300566c0 con 0x7f7c3815d5a0 2023-10-06T05:14:57.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.930+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c1c066f30 msgr2=0x7f7c1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.930+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c1c066f30 0x7f7c1c0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c340058d0 tx=0x7f7c34009040 comp rx=0 tx=0).stop 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.930+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3815d5a0 msgr2=0x7f7c38154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.930+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3815d5a0 0x7f7c38154640 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c30013000 tx=0x7f7c30019000 comp rx=0 tx=0).stop 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 shutdown_connections 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c1c066f30 0x7f7c1c0693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3815d5a0 0x7f7c38154640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c381590c0 0x7f7c38155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 --2- 172.21.15.45:0/2249068401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c38159490 0x7f7c3815b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 >> 172.21.15.45:0/2249068401 conn(0x7f7c380cd060 msgr2=0x7f7c380cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 shutdown_connections 2023-10-06T05:14:57.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:57.931+0000 7f7c3fb97700 1 -- 172.21.15.45:0/2249068401 wait complete. 2023-10-06T05:14:57.934 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-06T05:14:57.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-06T05:14:58.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.092+0000 7eff00487700 1 Processor -- start 2023-10-06T05:14:58.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.092+0000 7eff00487700 1 -- start start 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.093+0000 7eff00487700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efef80ccb70 0x7efef80aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7eff00487700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80c96a0 con 0x7efef80c8b70 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7eff00487700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80c3b50 con 0x7efef8157330 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7eff00487700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef8157700 con 0x7efef80ccb70 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefda22700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efef80ccb70 0x7efef80aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefea24700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efef80c8b70 0x7efef80c8f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44550/0 (socket says 172.21.15.45:44550) 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefea24700 1 -- 172.21.15.45:0/3136773588 learned_addr learned my addr 172.21.15.45:0/3136773588 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefda22700 1 -- 172.21.15.45:0/3136773588 >> v1:172.21.15.144:6789/0 conn(0x7efef8157330 legacy=0x7efef80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefda22700 1 -- 172.21.15.45:0/3136773588 >> v1:172.21.15.45:6789/0 conn(0x7efef80c8b70 legacy=0x7efef80c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefda22700 1 -- 172.21.15.45:0/3136773588 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80c98e0 con 0x7efef80ccb70 2023-10-06T05:14:58.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefda22700 1 --2- 172.21.15.45:0/3136773588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80ccb70 0x7efef80aaae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efeec0098e0 tx=0x7efeec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7a0ae17ad9dcad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefd221700 1 -- 172.21.15.45:0/3136773588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec009e80 con 0x7efef80ccb70 2023-10-06T05:14:58.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefd221700 1 -- 172.21.15.45:0/3136773588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec00b860 con 0x7efef80ccb70 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.094+0000 7efefd221700 1 -- 172.21.15.45:0/3136773588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec00bbe0 con 0x7efef80ccb70 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.095+0000 7eff00487700 1 -- 172.21.15.45:0/3136773588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80ccb70 msgr2=0x7efef80aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.095+0000 7eff00487700 1 --2- 172.21.15.45:0/3136773588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80ccb70 0x7efef80aaae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efeec0098e0 tx=0x7efeec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.095+0000 7eff00487700 1 -- 172.21.15.45:0/3136773588 shutdown_connections 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.095+0000 7eff00487700 1 --2- 172.21.15.45:0/3136773588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80ccb70 0x7efef80aaae0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.095+0000 7eff00487700 1 -- 172.21.15.45:0/3136773588 >> 172.21.15.45:0/3136773588 conn(0x7efef8067e00 msgr2=0x7efef80cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.095+0000 7eff00487700 1 -- 172.21.15.45:0/3136773588 shutdown_connections 2023-10-06T05:14:58.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.095+0000 7eff00487700 1 -- 172.21.15.45:0/3136773588 wait complete. 2023-10-06T05:14:58.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.096+0000 7eff00487700 1 Processor -- start 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.096+0000 7eff00487700 1 -- start start 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7eff00487700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80c8b70 0x7efef8148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7eff00487700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef80ccb70 0x7efef80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7eff00487700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef8157330 0x7efef816aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7eff00487700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80afef0 con 0x7efef80c8b70 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7eff00487700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80af2e0 con 0x7efef8157330 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7eff00487700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80af5c0 con 0x7efef80ccb70 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7efefe223700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80c8b70 0x7efef8148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7efefe223700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80c8b70 0x7efef8148cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42346/0 (socket says 172.21.15.45:42346) 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.097+0000 7efefe223700 1 -- 172.21.15.45:0/2722829249 learned_addr learned my addr 172.21.15.45:0/2722829249 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:58.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefea24700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef8157330 0x7efef816aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefda22700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef80ccb70 0x7efef80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefe223700 1 -- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef80ccb70 msgr2=0x7efef80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefe223700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef80ccb70 0x7efef80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefe223700 1 -- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef8157330 msgr2=0x7efef816aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefe223700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef8157330 0x7efef816aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefe223700 1 -- 172.21.15.45:0/2722829249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec009420 con 0x7efef80c8b70 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefea24700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef8157330 0x7efef816aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefda22700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef80ccb70 0x7efef80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efefe223700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80c8b70 0x7efef8148cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efef400c7c0 tx=0x7efef4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efeea7fc700 1 -- 172.21.15.45:0/2722829249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4011590 con 0x7efef80c8b70 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efeea7fc700 1 -- 172.21.15.45:0/2722829249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef4011710 con 0x7efef80c8b70 2023-10-06T05:14:58.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef80af8a0 con 0x7efef80c8b70 2023-10-06T05:14:58.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7efeea7fc700 1 -- 172.21.15.45:0/2722829249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4011ad0 con 0x7efef80c8b70 2023-10-06T05:14:58.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.098+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef80b0380 con 0x7efef80c8b70 2023-10-06T05:14:58.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.099+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef806e350 con 0x7efef80c8b70 2023-10-06T05:14:58.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.100+0000 7efeea7fc700 1 -- 172.21.15.45:0/2722829249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efef401b3b0 con 0x7efef80c8b70 2023-10-06T05:14:58.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.100+0000 7efeea7fc700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efedc066f30 0x7efedc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:58.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.100+0000 7efeea7fc700 1 -- 172.21.15.45:0/2722829249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5576+0+0 (secure 0 0 0) 0x7efef4083d90 con 0x7efef80c8b70 2023-10-06T05:14:58.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.101+0000 7efefda22700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efedc066f30 0x7efedc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:58.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.101+0000 7efefda22700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efedc066f30 0x7efedc0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efeec0093f0 tx=0x7efeec017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:58.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.104+0000 7efeea7fc700 1 -- 172.21.15.45:0/2722829249 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efef40511b0 con 0x7efef80c8b70 2023-10-06T05:14:58.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.306+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7efef80c9bc0 con 0x7efef80c8b70 2023-10-06T05:14:58.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.936+0000 7efeea7fc700 1 -- 172.21.15.45:0/2722829249 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7efef40170d0 con 0x7efef80c8b70 2023-10-06T05:14:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efedc066f30 msgr2=0x7efedc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efedc066f30 0x7efedc0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efeec0093f0 tx=0x7efeec017000 comp rx=0 tx=0).stop 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80c8b70 msgr2=0x7efef8148cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80c8b70 0x7efef8148cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efef400c7c0 tx=0x7efef4007f70 comp rx=0 tx=0).stop 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 shutdown_connections 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efedc066f30 0x7efedc0693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef8157330 0x7efef816aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef80c8b70 0x7efef8148cc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 --2- 172.21.15.45:0/2722829249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef80ccb70 0x7efef80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.940+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 >> 172.21.15.45:0/2722829249 conn(0x7efef8067e00 msgr2=0x7efef80c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.941+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 shutdown_connections 2023-10-06T05:14:58.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:58.943+0000 7eff00487700 1 -- 172.21.15.45:0/2722829249 wait complete. 2023-10-06T05:14:58.957 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-06T05:14:58.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:14:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.108+0000 7fc34d0de700 1 Processor -- start 2023-10-06T05:14:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.108+0000 7fc34d0de700 1 -- start start 2023-10-06T05:14:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc34d0de700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3480b0610 0x7fc3480c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc34d0de700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c3db0 0x7fc3480697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc34d0de700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3480ce6a0 con 0x7fc348069d00 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc34d0de700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3480690f0 con 0x7fc3480b0610 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc34d0de700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480ceeb0 con 0x7fc3480c3db0 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc346d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3480b0610 0x7fc3480c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc346d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3480b0610 0x7fc3480c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42362/0 (socket says 172.21.15.45:42362) 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.109+0000 7fc346d9d700 1 -- 172.21.15.45:0/1643003804 learned_addr learned my addr 172.21.15.45:0/1643003804 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc34659c700 1 --2- 172.21.15.45:0/1643003804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c3db0 0x7fc3480697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc346d9d700 1 -- 172.21.15.45:0/1643003804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c3db0 msgr2=0x7fc3480697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc346d9d700 1 --2- 172.21.15.45:0/1643003804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c3db0 0x7fc3480697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc346d9d700 1 -- 172.21.15.45:0/1643003804 >> v1:172.21.15.144:6789/0 conn(0x7fc348069d00 legacy=0x7fc3480c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc346d9d700 1 -- 172.21.15.45:0/1643003804 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348159880 con 0x7fc3480b0610 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc346d9d700 1 --2- 172.21.15.45:0/1643003804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c85d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099c0 tx=0x7fc338009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f0ab2a6d294159 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc345d9b700 1 -- 172.21.15.45:0/1643003804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338005c10 con 0x7fc3480b0610 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc345d9b700 1 -- 172.21.15.45:0/1643003804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338005d70 con 0x7fc3480b0610 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.110+0000 7fc345d9b700 1 -- 172.21.15.45:0/1643003804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800b840 con 0x7fc3480b0610 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 -- 172.21.15.45:0/1643003804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 msgr2=0x7fc3480c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 --2- 172.21.15.45:0/1643003804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c85d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc3380099c0 tx=0x7fc338009220 comp rx=0 tx=0).stop 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 -- 172.21.15.45:0/1643003804 shutdown_connections 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 --2- 172.21.15.45:0/1643003804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c85d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 --2- 172.21.15.45:0/1643003804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c3db0 0x7fc3480697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 -- 172.21.15.45:0/1643003804 >> 172.21.15.45:0/1643003804 conn(0x7fc348067e00 msgr2=0x7fc3480cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 -- 172.21.15.45:0/1643003804 shutdown_connections 2023-10-06T05:14:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.111+0000 7fc34d0de700 1 -- 172.21.15.45:0/1643003804 wait complete. 2023-10-06T05:14:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.112+0000 7fc34d0de700 1 Processor -- start 2023-10-06T05:14:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.112+0000 7fc34d0de700 1 -- start start 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34d0de700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc348069d00 0x7fc3480c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34d0de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34d0de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c3db0 0x7fc348169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34d0de700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c76f0 con 0x7fc3480b0610 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34d0de700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480b2e60 con 0x7fc3480c3db0 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34d0de700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c5300 con 0x7fc348069d00 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42372/0 (socket says 172.21.15.45:42372) 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 -- 172.21.15.45:0/333833410 learned_addr learned my addr 172.21.15.45:0/333833410 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc346d9d700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc348069d00 0x7fc3480c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34759e700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c3db0 0x7fc348169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 -- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc348069d00 msgr2=0x7fc3480c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc348069d00 0x7fc3480c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 -- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c3db0 msgr2=0x7fc348169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c3db0 0x7fc348169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34659c700 1 -- 172.21.15.45:0/333833410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009480 con 0x7fc3480b0610 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc34759e700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c3db0 0x7fc348169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.113+0000 7fc346d9d700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc348069d00 0x7fc3480c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.114+0000 7fc34659c700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c71b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc33000d890 tx=0x7fc330008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.114+0000 7fc33effd700 1 -- 172.21.15.45:0/333833410 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330009610 con 0x7fc3480b0610 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.114+0000 7fc33effd700 1 -- 172.21.15.45:0/333833410 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc330009770 con 0x7fc3480b0610 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.114+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3480c78e0 con 0x7fc3480b0610 2023-10-06T05:14:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.114+0000 7fc33effd700 1 -- 172.21.15.45:0/333833410 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330009b30 con 0x7fc3480b0610 2023-10-06T05:14:59.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.114+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3480c7ea0 con 0x7fc3480b0610 2023-10-06T05:14:59.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.115+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34815cc90 con 0x7fc3480b0610 2023-10-06T05:14:59.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.115+0000 7fc33effd700 1 -- 172.21.15.45:0/333833410 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc33001e460 con 0x7fc3480b0610 2023-10-06T05:14:59.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.116+0000 7fc33effd700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc328066f30 0x7fc3280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:14:59.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.116+0000 7fc33effd700 1 -- 172.21.15.45:0/333833410 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fc330084c30 con 0x7fc3480b0610 2023-10-06T05:14:59.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.116+0000 7fc346d9d700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc328066f30 0x7fc3280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:14:59.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.117+0000 7fc346d9d700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc328066f30 0x7fc3280693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3480712e0 tx=0x7fc338016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:14:59.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.121+0000 7fc33effd700 1 -- 172.21.15.45:0/333833410 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc330051da0 con 0x7fc3480b0610 2023-10-06T05:14:59.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.367+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc34804bb40 con 0x7fc3480b0610 2023-10-06T05:14:59.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.940+0000 7fc33effd700 1 -- 172.21.15.45:0/333833410 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc3300553c0 con 0x7fc3480b0610 2023-10-06T05:14:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc328066f30 msgr2=0x7fc3280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc328066f30 0x7fc3280693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3480712e0 tx=0x7fc338016000 comp rx=0 tx=0).stop 2023-10-06T05:14:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 msgr2=0x7fc3480c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:14:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c71b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc33000d890 tx=0x7fc330008040 comp rx=0 tx=0).stop 2023-10-06T05:14:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 shutdown_connections 2023-10-06T05:14:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc328066f30 0x7fc3280693e0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3480712e0 tx=0x7fc338016000 comp rx=0 tx=0).stop 2023-10-06T05:14:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c3db0 0x7fc348169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc348069d00 0x7fc3480c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:14:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 --2- 172.21.15.45:0/333833410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480b0610 0x7fc3480c71b0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc33000d890 tx=0x7fc330008040 comp rx=0 tx=0).stop 2023-10-06T05:14:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.947+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 >> 172.21.15.45:0/333833410 conn(0x7fc348067e00 msgr2=0x7fc348068940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:14:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.948+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 shutdown_connections 2023-10-06T05:14:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:14:59.951+0000 7fc34d0de700 1 -- 172.21.15.45:0/333833410 wait complete. 2023-10-06T05:14:59.962 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T05:14:59.983 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-06T05:14:59.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-06T05:15:00.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.121+0000 7fc070276700 1 Processor -- start 2023-10-06T05:15:00.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.121+0000 7fc070276700 1 -- start start 2023-10-06T05:15:00.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.122+0000 7fc070276700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc0680c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.122+0000 7fc070276700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0680ad1f0 0x7fc0681404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc070276700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680ca7f0 con 0x7fc068067bb0 2023-10-06T05:15:00.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc070276700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc068068000 con 0x7fc0680ad1f0 2023-10-06T05:15:00.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc070276700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680ccdf0 con 0x7fc0680677e0 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06d811700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc068067bb0 0x7fc0680c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44566/0 (socket says 172.21.15.45:44566) 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06d811700 1 -- 172.21.15.45:0/3021266343 learned_addr learned my addr 172.21.15.45:0/3021266343 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e012700 1 --2- 172.21.15.45:0/3021266343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc0680c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e813700 1 --2- 172.21.15.45:0/3021266343 >> v2:172.21.15.144:3300/0 conn(0x7fc0680ad1f0 0x7fc0681404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06d010700 1 -- 172.21.15.45:0/3021266343 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754136351 0 0) 0x7fc0680ca7f0 con 0x7fc068067bb0 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06d010700 1 -- 172.21.15.45:0/3021266343 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc054003570 con 0x7fc068067bb0 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e012700 1 -- 172.21.15.45:0/3021266343 >> v2:172.21.15.144:3300/0 conn(0x7fc0680ad1f0 msgr2=0x7fc0681404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e012700 1 --2- 172.21.15.45:0/3021266343 >> v2:172.21.15.144:3300/0 conn(0x7fc0680ad1f0 0x7fc0681404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06d010700 1 -- 172.21.15.45:0/3021266343 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468186349 0 0) 0x7fc054003570 con 0x7fc068067bb0 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e012700 1 -- 172.21.15.45:0/3021266343 >> v1:172.21.15.45:6789/0 conn(0x7fc068067bb0 legacy=0x7fc0680c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e012700 1 -- 172.21.15.45:0/3021266343 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680c9f50 con 0x7fc0680677e0 2023-10-06T05:15:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e813700 1 --2- 172.21.15.45:0/3021266343 >> v2:172.21.15.144:3300/0 conn(0x7fc0680ad1f0 0x7fc0681404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.123+0000 7fc06e012700 1 --2- 172.21.15.45:0/3021266343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc0680c7500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0580098e0 tx=0x7fc0580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbfb483e6c9eb22c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc06d010700 1 -- 172.21.15.45:0/3021266343 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640017f0 con 0x7fc0680677e0 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc06d010700 1 -- 172.21.15.45:0/3021266343 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058011420 con 0x7fc0680677e0 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc06d010700 1 -- 172.21.15.45:0/3021266343 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580117a0 con 0x7fc0680677e0 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 -- 172.21.15.45:0/3021266343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 msgr2=0x7fc0680c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 --2- 172.21.15.45:0/3021266343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc0680c7500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0580098e0 tx=0x7fc0580091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 -- 172.21.15.45:0/3021266343 shutdown_connections 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 --2- 172.21.15.45:0/3021266343 >> v2:172.21.15.144:3300/0 conn(0x7fc0680ad1f0 0x7fc0681404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 --2- 172.21.15.45:0/3021266343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc0680c7500 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 -- 172.21.15.45:0/3021266343 >> 172.21.15.45:0/3021266343 conn(0x7fc068068730 msgr2=0x7fc068068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 -- 172.21.15.45:0/3021266343 shutdown_connections 2023-10-06T05:15:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.124+0000 7fc070276700 1 -- 172.21.15.45:0/3021266343 wait complete. 2023-10-06T05:15:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.126+0000 7fc070276700 1 Processor -- start 2023-10-06T05:15:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.126+0000 7fc070276700 1 -- start start 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc070276700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc068155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc070276700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc068067bb0 0x7fc068167e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc070276700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0680ad1f0 0x7fc06813a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc070276700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068168350 con 0x7fc068067bb0 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc070276700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06813aad0 con 0x7fc0680ad1f0 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc070276700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc068167980 con 0x7fc0680677e0 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc06d811700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc068067bb0 0x7fc068167e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc06d811700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc068067bb0 0x7fc068167e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42386/0 (socket says 172.21.15.45:42386) 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc06d811700 1 -- 172.21.15.45:0/537488361 learned_addr learned my addr 172.21.15.45:0/537488361 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc06e012700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc068155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.127+0000 7fc06e813700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0680ad1f0 0x7fc06813a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc06d811700 1 -- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 msgr2=0x7fc068155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc06d811700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc068155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc06d811700 1 -- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0680ad1f0 msgr2=0x7fc06813a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc06d811700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0680ad1f0 0x7fc06813a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc06d811700 1 -- 172.21.15.45:0/537488361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009420 con 0x7fc068067bb0 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc06d811700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc068067bb0 0x7fc068167e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc06400ccf0 tx=0x7fc064001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc05e7fc700 1 -- 172.21.15.45:0/537488361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064004dc0 con 0x7fc068067bb0 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc05e7fc700 1 -- 172.21.15.45:0/537488361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06400a710 con 0x7fc068067bb0 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc058009420 con 0x7fc068067bb0 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc05e7fc700 1 -- 172.21.15.45:0/537488361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400aad0 con 0x7fc068067bb0 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.128+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0681673e0 con 0x7fc068067bb0 2023-10-06T05:15:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.129+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06804c380 con 0x7fc068067bb0 2023-10-06T05:15:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.129+0000 7fc05e7fc700 1 -- 172.21.15.45:0/537488361 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc064007150 con 0x7fc068067bb0 2023-10-06T05:15:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.130+0000 7fc05e7fc700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc04c066f30 0x7fc04c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.130+0000 7fc05e7fc700 1 -- 172.21.15.45:0/537488361 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc0640840b0 con 0x7fc068067bb0 2023-10-06T05:15:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.130+0000 7fc06e012700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc04c066f30 0x7fc04c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.131+0000 7fc06e012700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc04c066f30 0x7fc04c0693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc058003080 tx=0x7fc058016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.134+0000 7fc05e7fc700 1 -- 172.21.15.45:0/537488361 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc064051300 con 0x7fc068067bb0 2023-10-06T05:15:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.349+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fc0680b2960 con 0x7fc068067bb0 2023-10-06T05:15:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.952+0000 7fc05e7fc700 1 -- 172.21.15.45:0/537488361 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc064054920 con 0x7fc068067bb0 2023-10-06T05:15:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.955+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc04c066f30 msgr2=0x7fc04c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.955+0000 7fc070276700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc04c066f30 0x7fc04c0693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc058003080 tx=0x7fc058016000 comp rx=0 tx=0).stop 2023-10-06T05:15:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.955+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc068067bb0 msgr2=0x7fc068167e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.955+0000 7fc070276700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc068067bb0 0x7fc068167e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc06400ccf0 tx=0x7fc064001670 comp rx=0 tx=0).stop 2023-10-06T05:15:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 shutdown_connections 2023-10-06T05:15:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc04c066f30 0x7fc04c0693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0680ad1f0 0x7fc06813a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0680677e0 0x7fc068155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 --2- 172.21.15.45:0/537488361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc068067bb0 0x7fc068167e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 >> 172.21.15.45:0/537488361 conn(0x7fc068068730 msgr2=0x7fc06813f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 shutdown_connections 2023-10-06T05:15:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:00.956+0000 7fc070276700 1 -- 172.21.15.45:0/537488361 wait complete. 2023-10-06T05:15:00.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:15:01.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.116+0000 7fee08aaf700 1 Processor -- start 2023-10-06T05:15:01.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.116+0000 7fee08aaf700 1 -- start start 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee08aaf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee0415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee08aaf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee0415c6f0 0x7fee040aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee08aaf700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee04067d80 con 0x7fee0406aa70 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee08aaf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee040ac620 con 0x7fee0415c6f0 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee08aaf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0415cb70 con 0x7fee0406ae40 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee02d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee0415c6f0 0x7fee040aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee01d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee0415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee0259c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fee0406aa70 0x7fee040c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:45280/0 (socket says 172.21.15.45:45280) 2023-10-06T05:15:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.117+0000 7fee0259c700 1 -- 172.21.15.45:0/485344687 learned_addr learned my addr 172.21.15.45:0/485344687 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee02d9d700 1 -- 172.21.15.45:0/485344687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 msgr2=0x7fee0415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee02d9d700 1 --2- 172.21.15.45:0/485344687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee0415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee02d9d700 1 -- 172.21.15.45:0/485344687 >> v1:172.21.15.144:6789/0 conn(0x7fee0406aa70 legacy=0x7fee040c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee02d9d700 1 -- 172.21.15.45:0/485344687 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040c75b0 con 0x7fee0415c6f0 2023-10-06T05:15:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee02d9d700 1 --2- 172.21.15.45:0/485344687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 0x7fee040aca70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009e60 tx=0x7fedfc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d127135377c570 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee0159a700 1 -- 172.21.15.45:0/485344687 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00e070 con 0x7fee0415c6f0 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee0159a700 1 -- 172.21.15.45:0/485344687 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc00bb60 con 0x7fee0415c6f0 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee0159a700 1 -- 172.21.15.45:0/485344687 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc012430 con 0x7fee0415c6f0 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee08aaf700 1 -- 172.21.15.45:0/485344687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 msgr2=0x7fee040aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.118+0000 7fee08aaf700 1 --2- 172.21.15.45:0/485344687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 0x7fee040aca70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009e60 tx=0x7fedfc009780 comp rx=0 tx=0).stop 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.119+0000 7fee08aaf700 1 -- 172.21.15.45:0/485344687 shutdown_connections 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.119+0000 7fee08aaf700 1 --2- 172.21.15.45:0/485344687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 0x7fee040aca70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.119+0000 7fee08aaf700 1 --2- 172.21.15.45:0/485344687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee0415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.119+0000 7fee08aaf700 1 -- 172.21.15.45:0/485344687 >> 172.21.15.45:0/485344687 conn(0x7fee040cc570 msgr2=0x7fee040cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.119+0000 7fee08aaf700 1 -- 172.21.15.45:0/485344687 shutdown_connections 2023-10-06T05:15:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.119+0000 7fee08aaf700 1 -- 172.21.15.45:0/485344687 wait complete. 2023-10-06T05:15:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.120+0000 7fee08aaf700 1 Processor -- start 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.120+0000 7fee08aaf700 1 -- start start 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee08aaf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0406aa70 0x7fee04155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee08aaf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee040cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee08aaf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 0x7fee040cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee08aaf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee040b1f40 con 0x7fee0406aa70 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee08aaf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee040a8fb0 con 0x7fee0415c6f0 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee08aaf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee04155fe0 con 0x7fee0406ae40 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0406aa70 0x7fee04155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0406aa70 0x7fee04155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42388/0 (socket says 172.21.15.45:42388) 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 -- 172.21.15.45:0/2185726136 learned_addr learned my addr 172.21.15.45:0/2185726136 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee02d9d700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 0x7fee040cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee01d9b700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee040cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 -- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 msgr2=0x7fee040cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee040cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 -- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 msgr2=0x7fee040cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 0x7fee040cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 -- 172.21.15.45:0/2185726136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc009a20 con 0x7fee0406aa70 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.121+0000 7fee0259c700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0406aa70 0x7fee04155aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c7c0 tx=0x7fedf4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.122+0000 7fedf27fc700 1 -- 172.21.15.45:0/2185726136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4005b30 con 0x7fee0406aa70 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.122+0000 7fedf27fc700 1 -- 172.21.15.45:0/2185726136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf4005c90 con 0x7fee0406aa70 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.122+0000 7fedf27fc700 1 -- 172.21.15.45:0/2185726136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4004a80 con 0x7fee0406aa70 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.122+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee040b1c60 con 0x7fee0406aa70 2023-10-06T05:15:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.122+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee040a8c40 con 0x7fee0406aa70 2023-10-06T05:15:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.123+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0406e300 con 0x7fee0406aa70 2023-10-06T05:15:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.123+0000 7fedf27fc700 1 -- 172.21.15.45:0/2185726136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedf40070d0 con 0x7fee0406aa70 2023-10-06T05:15:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.123+0000 7fedf27fc700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fede4066f30 0x7fede40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.124+0000 7fedf27fc700 1 -- 172.21.15.45:0/2185726136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fedf4083a60 con 0x7fee0406aa70 2023-10-06T05:15:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.124+0000 7fee01d9b700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fede4066f30 0x7fede40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.124+0000 7fee01d9b700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fede4066f30 0x7fede40693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee040cbef0 tx=0x7fedf80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:01.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.128+0000 7fedf27fc700 1 -- 172.21.15.45:0/2185726136 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedf404b5a0 con 0x7fee0406aa70 2023-10-06T05:15:01.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:01.332+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fee040c7890 con 0x7fee0406aa70 2023-10-06T05:15:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.956+0000 7fedf27fc700 1 -- 172.21.15.45:0/2185726136 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fedf4054520 con 0x7fee0406aa70 2023-10-06T05:15:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fede4066f30 msgr2=0x7fede40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fede4066f30 0x7fede40693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee040cbef0 tx=0x7fedf80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0406aa70 msgr2=0x7fee04155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0406aa70 0x7fee04155aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c7c0 tx=0x7fedf4007f70 comp rx=0 tx=0).stop 2023-10-06T05:15:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 shutdown_connections 2023-10-06T05:15:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fede4066f30 0x7fede40693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee0415c6f0 0x7fee040cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0406aa70 0x7fee04155aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 --2- 172.21.15.45:0/2185726136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee0406ae40 0x7fee040cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.963+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 >> 172.21.15.45:0/2185726136 conn(0x7fee040cc570 msgr2=0x7fee0415bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:02.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.967+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 shutdown_connections 2023-10-06T05:15:02.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:02.967+0000 7fee08aaf700 1 -- 172.21.15.45:0/2185726136 wait complete. 2023-10-06T05:15:02.970 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' created 2023-10-06T05:15:02.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-06T05:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.121+0000 7fd2d57e0700 1 Processor -- start 2023-10-06T05:15:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.121+0000 7fd2d57e0700 1 -- start start 2023-10-06T05:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2d57e0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2d00afcd0 0x7fd2d00cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2d57e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d00cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2d57e0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d00b0140 con 0x7fd2d00b2340 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2d57e0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d00b0420 con 0x7fd2d00afcd0 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2d57e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d00b0700 con 0x7fd2d015cb80 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2ce7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2d00afcd0 0x7fd2d00cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2ce7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2d00afcd0 0x7fd2d00cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42404/0 (socket says 172.21.15.45:42404) 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.122+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1111115007 learned_addr learned my addr 172.21.15.45:0/1111115007 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2cf7fe700 1 --2- 172.21.15.45:0/1111115007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d00cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1111115007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 msgr2=0x7fd2d00cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2ce7fc700 1 --2- 172.21.15.45:0/1111115007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d00cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1111115007 >> v1:172.21.15.144:6789/0 conn(0x7fd2d00b2340 legacy=0x7fd2d015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1111115007 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d00b09e0 con 0x7fd2d00afcd0 2023-10-06T05:15:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2ce7fc700 1 --2- 172.21.15.45:0/1111115007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00afcd0 0x7fd2d00cc390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80099b0 tx=0x7fd2b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916ea6a929470892 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2cdffb700 1 -- 172.21.15.45:0/1111115007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800b8f0 con 0x7fd2d00afcd0 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2cdffb700 1 -- 172.21.15.45:0/1111115007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b800ba50 con 0x7fd2d00afcd0 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.123+0000 7fd2cdffb700 1 -- 172.21.15.45:0/1111115007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800bdf0 con 0x7fd2d00afcd0 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1111115007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00afcd0 msgr2=0x7fd2d00cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1111115007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00afcd0 0x7fd2d00cc390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80099b0 tx=0x7fd2b8009250 comp rx=0 tx=0).stop 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1111115007 shutdown_connections 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1111115007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d00cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1111115007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00afcd0 0x7fd2d00cc390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1111115007 >> 172.21.15.45:0/1111115007 conn(0x7fd2d006f820 msgr2=0x7fd2d00d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1111115007 shutdown_connections 2023-10-06T05:15:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.124+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1111115007 wait complete. 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.125+0000 7fd2d57e0700 1 Processor -- start 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.125+0000 7fd2d57e0700 1 -- start start 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2d57e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d00afcd0 0x7fd2d015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2d57e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00b2340 0x7fd2d01484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2d57e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d0148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2d57e0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d016b490 con 0x7fd2d00b2340 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2d57e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d016b6e0 con 0x7fd2d00afcd0 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2d57e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d016b9c0 con 0x7fd2d015cb80 2023-10-06T05:15:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00b2340 0x7fd2d01484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00b2340 0x7fd2d01484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42420/0 (socket says 172.21.15.45:42420) 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1273792024 learned_addr learned my addr 172.21.15.45:0/1273792024 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2cf7fe700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d0148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ceffd700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d00afcd0 0x7fd2d015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 msgr2=0x7fd2d0148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d0148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d00afcd0 msgr2=0x7fd2d015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d00afcd0 0x7fd2d015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ce7fc700 1 -- 172.21.15.45:0/1273792024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80094f0 con 0x7fd2d00b2340 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2cf7fe700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d0148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.126+0000 7fd2ceffd700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d00afcd0 0x7fd2d015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.127+0000 7fd2ce7fc700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00b2340 0x7fd2d01484f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0088c40 tx=0x7fd2b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.127+0000 7fd2c6ffd700 1 -- 172.21.15.45:0/1273792024 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8024440 con 0x7fd2d00b2340 2023-10-06T05:15:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.127+0000 7fd2c6ffd700 1 -- 172.21.15.45:0/1273792024 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b80245a0 con 0x7fd2d00b2340 2023-10-06T05:15:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.127+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d016bca0 con 0x7fd2d00b2340 2023-10-06T05:15:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.127+0000 7fd2c6ffd700 1 -- 172.21.15.45:0/1273792024 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8024940 con 0x7fd2d00b2340 2023-10-06T05:15:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.127+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d015baf0 con 0x7fd2d00b2340 2023-10-06T05:15:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.128+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d016c1c0 con 0x7fd2d00b2340 2023-10-06T05:15:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.128+0000 7fd2c6ffd700 1 -- 172.21.15.45:0/1273792024 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2b801a420 con 0x7fd2d00b2340 2023-10-06T05:15:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.129+0000 7fd2c6ffd700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.129+0000 7fd2c6ffd700 1 -- 172.21.15.45:0/1273792024 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fd2b80867b0 con 0x7fd2d00b2340 2023-10-06T05:15:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.129+0000 7fd2ceffd700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.130+0000 7fd2ceffd700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001840 tx=0x7fd2c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.134+0000 7fd2c6ffd700 1 -- 172.21.15.45:0/1273792024 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2d016c1c0 con 0x7fd2d00b2340 2023-10-06T05:15:03.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.334+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd2d016c1c0 con 0x7fd2d00b2340 2023-10-06T05:15:03.960 INFO:tasks.ceph.mds.d.smithi144.stderr:2023-10-06T05:15:03.957+0000 7f6148125700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:15:03.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.958+0000 7fd2c6ffd700 1 -- 172.21.15.45:0/1273792024 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd2d016c1c0 con 0x7fd2d00b2340 2023-10-06T05:15:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.963+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 msgr2=0x7fd2b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.963+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001840 tx=0x7fd2c0009040 comp rx=0 tx=0).stop 2023-10-06T05:15:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.963+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00b2340 msgr2=0x7fd2d01484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:03.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.963+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00b2340 0x7fd2d01484f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0088c40 tx=0x7fd2b8017000 comp rx=0 tx=0).stop 2023-10-06T05:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.967+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 shutdown_connections 2023-10-06T05:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.967+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.967+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2d015cb80 0x7fd2d0148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.967+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d00afcd0 0x7fd2d015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.967+0000 7fd2d57e0700 1 --2- 172.21.15.45:0/1273792024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d00b2340 0x7fd2d01484f0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0088c40 tx=0x7fd2b8017000 comp rx=0 tx=0).stop 2023-10-06T05:15:03.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.967+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 >> 172.21.15.45:0/1273792024 conn(0x7fd2d006f820 msgr2=0x7fd2d00d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.974+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 shutdown_connections 2023-10-06T05:15:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:03.974+0000 7fd2d57e0700 1 -- 172.21.15.45:0/1273792024 wait complete. 2023-10-06T05:15:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:set pool 5 allow_ec_overwrites to true 2023-10-06T05:15:04.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-06T05:15:04.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.136+0000 7ff0d0edb700 1 Processor -- start 2023-10-06T05:15:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.136+0000 7ff0d0edb700 1 -- start start 2023-10-06T05:15:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0d0edb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff0cc0b1cb0 0x7ff0cc06a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0d0edb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0cc06ad60 0x7ff0cc0ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0d0edb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc06b1c0 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0d0edb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc068750 con 0x7ff0cc06ad60 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0d0edb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc067860 con 0x7ff0cc0b1cb0 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0cad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0cc06ad60 0x7ff0cc0ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0ca59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff0cc0b18e0 0x7ff0cc15b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44568/0 (socket says 172.21.15.45:44568) 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0ca59c700 1 -- 172.21.15.45:0/1935398107 learned_addr learned my addr 172.21.15.45:0/1935398107 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0c9d9b700 1 --2- 172.21.15.45:0/1935398107 >> v2:172.21.15.144:3300/0 conn(0x7ff0cc0b1cb0 0x7ff0cc06a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0cad9d700 1 -- 172.21.15.45:0/1935398107 >> v2:172.21.15.144:3300/0 conn(0x7ff0cc0b1cb0 msgr2=0x7ff0cc06a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0cad9d700 1 --2- 172.21.15.45:0/1935398107 >> v2:172.21.15.144:3300/0 conn(0x7ff0cc0b1cb0 0x7ff0cc06a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0cad9d700 1 -- 172.21.15.45:0/1935398107 >> v1:172.21.15.45:6789/0 conn(0x7ff0cc0b18e0 legacy=0x7ff0cc15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0cad9d700 1 -- 172.21.15.45:0/1935398107 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc067b40 con 0x7ff0cc06ad60 2023-10-06T05:15:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0c959a700 1 -- 172.21.15.45:0/1935398107 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780546152 0 0) 0x7ff0bc001110 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.137+0000 7ff0cad9d700 1 --2- 172.21.15.45:0/1935398107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc06ad60 0x7ff0cc0ab9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0c8db0 tx=0x7ff0c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43091a5d8920c2e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0c959a700 1 -- 172.21.15.45:0/1935398107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400e070 con 0x7ff0cc06ad60 2023-10-06T05:15:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0c959a700 1 -- 172.21.15.45:0/1935398107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c400bc30 con 0x7ff0cc06ad60 2023-10-06T05:15:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0c959a700 1 -- 172.21.15.45:0/1935398107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4012430 con 0x7ff0cc06ad60 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 -- 172.21.15.45:0/1935398107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc06ad60 msgr2=0x7ff0cc0ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/1935398107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc06ad60 0x7ff0cc0ab9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0c8db0 tx=0x7ff0c4009780 comp rx=0 tx=0).stop 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 -- 172.21.15.45:0/1935398107 shutdown_connections 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/1935398107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc06ad60 0x7ff0cc0ab9d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/1935398107 >> v2:172.21.15.144:3300/0 conn(0x7ff0cc0b1cb0 0x7ff0cc06a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 -- 172.21.15.45:0/1935398107 >> 172.21.15.45:0/1935398107 conn(0x7ff0cc0cc570 msgr2=0x7ff0cc0cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 -- 172.21.15.45:0/1935398107 shutdown_connections 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.138+0000 7ff0d0edb700 1 -- 172.21.15.45:0/1935398107 wait complete. 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 Processor -- start 2023-10-06T05:15:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 -- start start 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0cc06ad60 0x7ff0cc0aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc0b18e0 0x7ff0cc153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0cc0b1cb0 0x7ff0cc0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0aa850 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc16d500 con 0x7ff0cc0b1cb0 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0d0edb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc16cc60 con 0x7ff0cc06ad60 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.140+0000 7ff0c9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc0b18e0 0x7ff0cc153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc0b18e0 0x7ff0cc153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42438/0 (socket says 172.21.15.45:42438) 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 -- 172.21.15.45:0/3659702440 learned_addr learned my addr 172.21.15.45:0/3659702440 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0cad9d700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0cc0b1cb0 0x7ff0cc0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0ca59c700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0cc06ad60 0x7ff0cc0aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 -- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0cc06ad60 msgr2=0x7ff0cc0aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0cc06ad60 0x7ff0cc0aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 -- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0cc0b1cb0 msgr2=0x7ff0cc0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0cc0b1cb0 0x7ff0cc0cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 -- 172.21.15.45:0/3659702440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009a20 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0ca59c700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0cc06ad60 0x7ff0cc0aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0c9d9b700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc0b18e0 0x7ff0cc153590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000d860 tx=0x7ff0c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0ba7fc700 1 -- 172.21.15.45:0/3659702440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00125f0 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0ba7fc700 1 -- 172.21.15.45:0/3659702440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0012750 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0ba7fc700 1 -- 172.21.15.45:0/3659702440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0012ad0 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc0ab290 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.141+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc16ceb0 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.142+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc16a160 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.143+0000 7ff0ba7fc700 1 -- 172.21.15.45:0/3659702440 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0c001e460 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.143+0000 7ff0ba7fc700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0ac066f30 0x7ff0ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.143+0000 7ff0ba7fc700 1 -- 172.21.15.45:0/3659702440 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6777+0+0 (secure 0 0 0) 0x7ff0c0084fc0 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.143+0000 7ff0ca59c700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0ac066f30 0x7ff0ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.144+0000 7ff0ca59c700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0ac066f30 0x7ff0ac0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0018c0 tx=0x7ff0bc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.148+0000 7ff0ba7fc700 1 -- 172.21.15.45:0/3659702440 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0cc16a160 con 0x7ff0cc0b18e0 2023-10-06T05:15:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:04.402+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7ff0cc16d6b0 con 0x7ff0cc0b18e0 2023-10-06T05:15:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.006+0000 7ff0ba7fc700 1 -- 172.21.15.45:0/3659702440 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7ff0cc16d6b0 con 0x7ff0cc0b18e0 2023-10-06T05:15:05.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0ac066f30 msgr2=0x7ff0ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0ac066f30 0x7ff0ac0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0018c0 tx=0x7ff0bc009040 comp rx=0 tx=0).stop 2023-10-06T05:15:05.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc0b18e0 msgr2=0x7ff0cc153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc0b18e0 0x7ff0cc153590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000d860 tx=0x7ff0c0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 shutdown_connections 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0ac066f30 0x7ff0ac0693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0cc0b1cb0 0x7ff0cc0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0cc06ad60 0x7ff0cc0aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 --2- 172.21.15.45:0/3659702440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0cc0b18e0 0x7ff0cc153590 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 >> 172.21.15.45:0/3659702440 conn(0x7ff0cc0cc570 msgr2=0x7ff0cc0cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 shutdown_connections 2023-10-06T05:15:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.009+0000 7ff0d0edb700 1 -- 172.21.15.45:0/3659702440 wait complete. 2023-10-06T05:15:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:added data pool 5 to fsmap 2023-10-06T05:15:05.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.160+0000 7f4a7e8dc700 1 Processor -- start 2023-10-06T05:15:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.160+0000 7f4a7e8dc700 1 -- start start 2023-10-06T05:15:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7e8dc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4a78067830 0x7f4a7806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7e8dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a7806add0 0x7f4a780abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7e8dc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a780b2120 con 0x7f4a7815a490 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7e8dc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a780ae020 con 0x7f4a7806add0 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7e8dc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a78067ca0 con 0x7f4a78067830 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7ce79700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a7806add0 0x7f4a780abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7ce79700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a7806add0 0x7f4a780abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42452/0 (socket says 172.21.15.45:42452) 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7ce79700 1 -- 172.21.15.45:0/3445911526 learned_addr learned my addr 172.21.15.45:0/3445911526 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a777fe700 1 --2- 172.21.15.45:0/3445911526 >> v2:172.21.15.144:3300/0 conn(0x7f4a78067830 0x7f4a7806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7ce79700 1 -- 172.21.15.45:0/3445911526 >> v2:172.21.15.144:3300/0 conn(0x7f4a78067830 msgr2=0x7f4a7806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7ce79700 1 --2- 172.21.15.45:0/3445911526 >> v2:172.21.15.144:3300/0 conn(0x7f4a78067830 0x7f4a7806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7ce79700 1 -- 172.21.15.45:0/3445911526 >> v1:172.21.15.144:6789/0 conn(0x7f4a7815a490 legacy=0x7f4a7815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a7ce79700 1 -- 172.21.15.45:0/3445911526 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7806b230 con 0x7f4a7806add0 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.161+0000 7f4a777fe700 1 --2- 172.21.15.45:0/3445911526 >> v2:172.21.15.144:3300/0 conn(0x7f4a78067830 0x7f4a7806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7ce79700 1 --2- 172.21.15.45:0/3445911526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7806add0 0x7f4a780abc60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a7806b610 tx=0x7f4a70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7673a1a7692a468 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a76ffd700 1 -- 172.21.15.45:0/3445911526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000e070 con 0x7f4a7806add0 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a76ffd700 1 -- 172.21.15.45:0/3445911526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7000ba10 con 0x7f4a7806add0 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a76ffd700 1 -- 172.21.15.45:0/3445911526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a700092e0 con 0x7f4a7806add0 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/3445911526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7806add0 msgr2=0x7f4a780abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/3445911526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7806add0 0x7f4a780abc60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a7806b610 tx=0x7f4a70009840 comp rx=0 tx=0).stop 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/3445911526 shutdown_connections 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/3445911526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7806add0 0x7f4a780abc60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/3445911526 >> v2:172.21.15.144:3300/0 conn(0x7f4a78067830 0x7f4a7806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/3445911526 >> 172.21.15.45:0/3445911526 conn(0x7f4a780cc5a0 msgr2=0x7f4a780cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/3445911526 shutdown_connections 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.162+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/3445911526 wait complete. 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.164+0000 7f4a7e8dc700 1 Processor -- start 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.164+0000 7f4a7e8dc700 1 -- start start 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a78067830 0x7f4a780aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a7806add0 0x7f4a78169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7815a490 0x7f4a7816a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780c6b40 con 0x7f4a78067830 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780c1dd0 con 0x7f4a7815a490 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780ab4c0 con 0x7f4a7806add0 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a78067830 0x7f4a780aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a78067830 0x7f4a780aaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42464/0 (socket says 172.21.15.45:42464) 2023-10-06T05:15:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 -- 172.21.15.45:0/746460969 learned_addr learned my addr 172.21.15.45:0/746460969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7ce79700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7815a490 0x7f4a7816a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a777fe700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a7806add0 0x7f4a78169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 -- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a7806add0 msgr2=0x7f4a78169b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a7806add0 0x7f4a78169b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 -- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7815a490 msgr2=0x7f4a7816a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7815a490 0x7f4a7816a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 -- 172.21.15.45:0/746460969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70009ae0 con 0x7f4a78067830 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a77fff700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a78067830 0x7f4a780aaf80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c5d0 tx=0x7f4a68007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a6ffff700 1 -- 172.21.15.45:0/746460969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68011590 con 0x7f4a78067830 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a6ffff700 1 -- 172.21.15.45:0/746460969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a68011710 con 0x7f4a78067830 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a780a9c30 con 0x7f4a78067830 2023-10-06T05:15:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a6ffff700 1 -- 172.21.15.45:0/746460969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68011ad0 con 0x7f4a78067830 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.165+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a780c1a60 con 0x7f4a78067830 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.166+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a780c18c0 con 0x7f4a78067830 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.167+0000 7f4a6ffff700 1 -- 172.21.15.45:0/746460969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a6801c460 con 0x7f4a78067830 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.167+0000 7f4a6ffff700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a58066f60 0x7f4a58069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.167+0000 7f4a6ffff700 1 -- 172.21.15.45:0/746460969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4a68084750 con 0x7f4a78067830 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.167+0000 7f4a777fe700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a58066f60 0x7f4a58069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.168+0000 7f4a777fe700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a58066f60 0x7f4a58069410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a780a8d10 tx=0x7f4a60008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.172+0000 7f4a6ffff700 1 -- 172.21.15.45:0/746460969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a780c18c0 con 0x7f4a78067830 2023-10-06T05:15:05.213 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:05.216 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.423+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a780c18c0 con 0x7f4a78067830 2023-10-06T05:15:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.424+0000 7f4a6ffff700 1 -- 172.21.15.45:0/746460969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f4a780c18c0 con 0x7f4a78067830 2023-10-06T05:15:05.426 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:05.426 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:04.996979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a58066f60 msgr2=0x7f4a58069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a58066f60 0x7f4a58069410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a780a8d10 tx=0x7f4a60008040 comp rx=0 tx=0).stop 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a78067830 msgr2=0x7f4a780aaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a78067830 0x7f4a780aaf80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c5d0 tx=0x7f4a68007f60 comp rx=0 tx=0).stop 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 shutdown_connections 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a58066f60 0x7f4a58069410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7815a490 0x7f4a7816a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a78067830 0x7f4a780aaf80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 --2- 172.21.15.45:0/746460969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a7806add0 0x7f4a78169b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 >> 172.21.15.45:0/746460969 conn(0x7f4a780cc5a0 msgr2=0x7f4a7806e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 shutdown_connections 2023-10-06T05:15:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.426+0000 7f4a7e8dc700 1 -- 172.21.15.45:0/746460969 wait complete. 2023-10-06T05:15:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2023-10-06T05:15:05.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:05.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.571+0000 7fdd57f5d700 1 Processor -- start 2023-10-06T05:15:05.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.571+0000 7fdd57f5d700 1 -- start start 2023-10-06T05:15:05.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd57f5d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd5006c730 0x7fdd50156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd57f5d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5006d1d0 con 0x7fdd5015b950 2023-10-06T05:15:05.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd57f5d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd50068f60 con 0x7fdd5015ae70 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd57f5d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5006cc10 con 0x7fdd5006c730 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd554f8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd5006c730 0x7fdd50156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd564fa700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdd5015b950 0x7fdd50156fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44582/0 (socket says 172.21.15.45:44582) 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd564fa700 1 -- 172.21.15.45:0/745070591 learned_addr learned my addr 172.21.15.45:0/745070591 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd554f8700 1 -- 172.21.15.45:0/745070591 >> v1:172.21.15.144:6789/0 conn(0x7fdd5015ae70 legacy=0x7fdd5015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd554f8700 1 -- 172.21.15.45:0/745070591 >> v1:172.21.15.45:6789/0 conn(0x7fdd5015b950 legacy=0x7fdd50156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd554f8700 1 -- 172.21.15.45:0/745070591 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5006cef0 con 0x7fdd5006c730 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.572+0000 7fdd554f8700 1 --2- 172.21.15.45:0/745070591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5006c730 0x7fdd50156a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd400098e0 tx=0x7fdd400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5fe5c5e64e74a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd54cf7700 1 -- 172.21.15.45:0/745070591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40009e80 con 0x7fdd5006c730 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd54cf7700 1 -- 172.21.15.45:0/745070591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4000b860 con 0x7fdd5006c730 2023-10-06T05:15:05.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd54cf7700 1 -- 172.21.15.45:0/745070591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4000bbe0 con 0x7fdd5006c730 2023-10-06T05:15:05.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd57f5d700 1 -- 172.21.15.45:0/745070591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5006c730 msgr2=0x7fdd50156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/745070591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5006c730 0x7fdd50156a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd400098e0 tx=0x7fdd400091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd57f5d700 1 -- 172.21.15.45:0/745070591 shutdown_connections 2023-10-06T05:15:05.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/745070591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5006c730 0x7fdd50156a60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd57f5d700 1 -- 172.21.15.45:0/745070591 >> 172.21.15.45:0/745070591 conn(0x7fdd5006a870 msgr2=0x7fdd5006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd57f5d700 1 -- 172.21.15.45:0/745070591 shutdown_connections 2023-10-06T05:15:05.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.573+0000 7fdd57f5d700 1 -- 172.21.15.45:0/745070591 wait complete. 2023-10-06T05:15:05.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.574+0000 7fdd57f5d700 1 Processor -- start 2023-10-06T05:15:05.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd57f5d700 1 -- start start 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd57f5d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5006c730 0x7fdd500caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd57f5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5015ae70 0x7fdd500aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd57f5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5015b950 0x7fdd500a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd57f5d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500cb7f0 con 0x7fdd5015b950 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd57f5d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50169d80 con 0x7fdd5015ae70 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd57f5d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500cb020 con 0x7fdd5006c730 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd564fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5015b950 0x7fdd500a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd564fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5015b950 0x7fdd500a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42482/0 (socket says 172.21.15.45:42482) 2023-10-06T05:15:05.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd564fa700 1 -- 172.21.15.45:0/2679026253 learned_addr learned my addr 172.21.15.45:0/2679026253 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.575+0000 7fdd554f8700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5015ae70 0x7fdd500aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd55cf9700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5006c730 0x7fdd500caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd564fa700 1 -- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5006c730 msgr2=0x7fdd500caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd564fa700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5006c730 0x7fdd500caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd564fa700 1 -- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5015ae70 msgr2=0x7fdd500aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd564fa700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5015ae70 0x7fdd500aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd564fa700 1 -- 172.21.15.45:0/2679026253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40009420 con 0x7fdd5015b950 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd564fa700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5015b950 0x7fdd500a9f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800cbc0 tx=0x7fdd480019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd45ffb700 1 -- 172.21.15.45:0/2679026253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800a7e0 con 0x7fdd5015b950 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd45ffb700 1 -- 172.21.15.45:0/2679026253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800a940 con 0x7fdd5015b950 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd50154850 con 0x7fdd5015b950 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd45ffb700 1 -- 172.21.15.45:0/2679026253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800acc0 con 0x7fdd5015b950 2023-10-06T05:15:05.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.576+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd501550f0 con 0x7fdd5015b950 2023-10-06T05:15:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.577+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd50154c50 con 0x7fdd5015b950 2023-10-06T05:15:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.578+0000 7fdd45ffb700 1 -- 172.21.15.45:0/2679026253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd48007300 con 0x7fdd5015b950 2023-10-06T05:15:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.578+0000 7fdd45ffb700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd34066f30 0x7fdd340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.578+0000 7fdd45ffb700 1 -- 172.21.15.45:0/2679026253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdd48084ba0 con 0x7fdd5015b950 2023-10-06T05:15:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.578+0000 7fdd55cf9700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd34066f30 0x7fdd340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.579+0000 7fdd55cf9700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd34066f30 0x7fdd340693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c003a80 tx=0x7fdd4c007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.583+0000 7fdd45ffb700 1 -- 172.21.15.45:0/2679026253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd48051a40 con 0x7fdd5015b950 2023-10-06T05:15:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.788+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd50169ef0 con 0x7fdd5015b950 2023-10-06T05:15:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.789+0000 7fdd45ffb700 1 -- 172.21.15.45:0/2679026253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fdd48055060 con 0x7fdd5015b950 2023-10-06T05:15:05.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:05.791 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd34066f30 msgr2=0x7fdd340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd34066f30 0x7fdd340693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c003a80 tx=0x7fdd4c007ec0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5015b950 msgr2=0x7fdd500a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5015b950 0x7fdd500a9f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800cbc0 tx=0x7fdd480019c0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 shutdown_connections 2023-10-06T05:15:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd34066f30 0x7fdd340693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5015b950 0x7fdd500a9f70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5006c730 0x7fdd500caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 --2- 172.21.15.45:0/2679026253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5015ae70 0x7fdd500aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 >> 172.21.15.45:0/2679026253 conn(0x7fdd5006a870 msgr2=0x7fdd5006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 shutdown_connections 2023-10-06T05:15:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.791+0000 7fdd57f5d700 1 -- 172.21.15.45:0/2679026253 wait complete. 2023-10-06T05:15:05.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:05.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.937+0000 7f0641499700 1 Processor -- start 2023-10-06T05:15:05.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.937+0000 7f0641499700 1 -- start start 2023-10-06T05:15:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f0641499700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f063c155440 0x7f063c155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f0641499700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f0641499700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c0c8080 con 0x7f063c06a890 2023-10-06T05:15:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f0641499700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063c068cd0 con 0x7f063c155440 2023-10-06T05:15:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f0641499700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063c156050 con 0x7f063c06e6e0 2023-10-06T05:15:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f063affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f063c155440 0x7f063c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f063b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f063c06a890 0x7f063c1595e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44586/0 (socket says 172.21.15.45:44586) 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.938+0000 7f063b7fe700 1 -- 172.21.15.45:0/1845626668 learned_addr learned my addr 172.21.15.45:0/1845626668 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f063a7fc700 1 --2- 172.21.15.45:0/1845626668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f063affd700 1 -- 172.21.15.45:0/1845626668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 msgr2=0x7f063c0c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f063affd700 1 --2- 172.21.15.45:0/1845626668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f063affd700 1 -- 172.21.15.45:0/1845626668 >> v1:172.21.15.45:6789/0 conn(0x7f063c06a890 legacy=0x7f063c1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f063affd700 1 -- 172.21.15.45:0/1845626668 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c156330 con 0x7f063c155440 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f0639ffb700 1 -- 172.21.15.45:0/1845626668 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794641343 0 0) 0x7f063c0c8080 con 0x7f063c06a890 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f063affd700 1 --2- 172.21.15.45:0/1845626668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c155440 0x7f063c155810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f062c00aa90 tx=0x7f062c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f8cfddae4bb9d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f0639ffb700 1 -- 172.21.15.45:0/1845626668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00ca40 con 0x7f063c155440 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f0639ffb700 1 -- 172.21.15.45:0/1845626668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c00cba0 con 0x7f063c155440 2023-10-06T05:15:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.939+0000 7f0639ffb700 1 -- 172.21.15.45:0/1845626668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c018400 con 0x7f063c155440 2023-10-06T05:15:05.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 -- 172.21.15.45:0/1845626668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c155440 msgr2=0x7f063c155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 --2- 172.21.15.45:0/1845626668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c155440 0x7f063c155810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f062c00aa90 tx=0x7f062c00a330 comp rx=0 tx=0).stop 2023-10-06T05:15:05.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 -- 172.21.15.45:0/1845626668 shutdown_connections 2023-10-06T05:15:05.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 --2- 172.21.15.45:0/1845626668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c155440 0x7f063c155810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 --2- 172.21.15.45:0/1845626668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 -- 172.21.15.45:0/1845626668 >> 172.21.15.45:0/1845626668 conn(0x7f063c0cc5a0 msgr2=0x7f063c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 -- 172.21.15.45:0/1845626668 shutdown_connections 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.940+0000 7f0641499700 1 -- 172.21.15.45:0/1845626668 wait complete. 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.941+0000 7f0641499700 1 Processor -- start 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.941+0000 7f0641499700 1 -- start start 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f0641499700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c06a890 0x7f063c1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f0641499700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f0641499700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f063c155440 0x7f063c0a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f0641499700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c0b0770 con 0x7f063c06a890 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f0641499700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c0afe10 con 0x7f063c155440 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f0641499700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063c0b00f0 con 0x7f063c06e6e0 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f063affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c06a890 0x7f063c1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f063affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c06a890 0x7f063c1549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42500/0 (socket says 172.21.15.45:42500) 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f063affd700 1 -- 172.21.15.45:0/3749174667 learned_addr learned my addr 172.21.15.45:0/3749174667 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f063a7fc700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.942+0000 7f063b7fe700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f063c155440 0x7f063c0a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f063affd700 1 -- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 msgr2=0x7f063c0af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f063affd700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f063affd700 1 -- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f063c155440 msgr2=0x7f063c0a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f063affd700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f063c155440 0x7f063c0a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f063affd700 1 -- 172.21.15.45:0/3749174667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c00a5d0 con 0x7f063c06a890 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f063affd700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c06a890 0x7f063c1549a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f062c00a5a0 tx=0x7f062c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f0622ffd700 1 -- 172.21.15.45:0/3749174667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00ce60 con 0x7f063c06a890 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f0622ffd700 1 -- 172.21.15.45:0/3749174667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c010600 con 0x7f063c06a890 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f0622ffd700 1 -- 172.21.15.45:0/3749174667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0109a0 con 0x7f063c06a890 2023-10-06T05:15:05.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c0b09c0 con 0x7f063c06a890 2023-10-06T05:15:05.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.943+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c0c4000 con 0x7f063c06a890 2023-10-06T05:15:05.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.944+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063c0b0ee0 con 0x7f063c06a890 2023-10-06T05:15:05.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.945+0000 7f0622ffd700 1 -- 172.21.15.45:0/3749174667 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f062c010d50 con 0x7f063c06a890 2023-10-06T05:15:05.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.945+0000 7f0622ffd700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0628066f30 0x7f06280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:05.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.945+0000 7f0622ffd700 1 -- 172.21.15.45:0/3749174667 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f062c085e70 con 0x7f063c06a890 2023-10-06T05:15:05.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.945+0000 7f063a7fc700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0628066f30 0x7f06280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:05.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.946+0000 7f063a7fc700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0628066f30 0x7f06280693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0630006fd0 tx=0x7f0630008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:05.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:05.950+0000 7f0622ffd700 1 -- 172.21.15.45:0/3749174667 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f062c052c90 con 0x7f063c06a890 2023-10-06T05:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.154+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f063c0cc0e0 con 0x7f063c06a890 2023-10-06T05:15:06.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.155+0000 7f0622ffd700 1 -- 172.21.15.45:0/3749174667 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f062c026090 con 0x7f063c06a890 2023-10-06T05:15:06.159 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:06.159 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0628066f30 msgr2=0x7f06280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0628066f30 0x7f06280693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0630006fd0 tx=0x7f0630008040 comp rx=0 tx=0).stop 2023-10-06T05:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c06a890 msgr2=0x7f063c1549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c06a890 0x7f063c1549a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f062c00a5a0 tx=0x7f062c01a000 comp rx=0 tx=0).stop 2023-10-06T05:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 shutdown_connections 2023-10-06T05:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0628066f30 0x7f06280693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f063c155440 0x7f063c0a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c06a890 0x7f063c1549a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 --2- 172.21.15.45:0/3749174667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063c06e6e0 0x7f063c0af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 >> 172.21.15.45:0/3749174667 conn(0x7f063c0cc5a0 msgr2=0x7f063c070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 shutdown_connections 2023-10-06T05:15:06.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.157+0000 7f0641499700 1 -- 172.21.15.45:0/3749174667 wait complete. 2023-10-06T05:15:06.185 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:06.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.315+0000 7f608b929700 1 Processor -- start 2023-10-06T05:15:06.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.315+0000 7f608b929700 1 -- start start 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f608b929700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6084159200 0x7f60840c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f608b929700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60841595d0 0x7f608406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f608b929700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60840c7d00 con 0x7f608406b5c0 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f608b929700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60840c7780 con 0x7f60841595d0 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f608b929700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60840c86d0 con 0x7f6084159200 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60841595d0 0x7f608406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60841595d0 0x7f608406b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42508/0 (socket says 172.21.15.45:42508) 2023-10-06T05:15:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 -- 172.21.15.45:0/3249002973 learned_addr learned my addr 172.21.15.45:0/3249002973 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f60896c5700 1 --2- 172.21.15.45:0/3249002973 >> v2:172.21.15.144:3300/0 conn(0x7f6084159200 0x7f60840c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 -- 172.21.15.45:0/3249002973 >> v2:172.21.15.144:3300/0 conn(0x7f6084159200 msgr2=0x7f60840c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 --2- 172.21.15.45:0/3249002973 >> v2:172.21.15.144:3300/0 conn(0x7f6084159200 0x7f60840c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 -- 172.21.15.45:0/3249002973 >> v1:172.21.15.144:6789/0 conn(0x7f608406b5c0 legacy=0x7f60840ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 -- 172.21.15.45:0/3249002973 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608406bb40 con 0x7f60841595d0 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.316+0000 7f6088ec4700 1 --2- 172.21.15.45:0/3249002973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f608406b080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f608406ae50 tx=0x7f6074009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570ff3d1d462509b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f607bfff700 1 -- 172.21.15.45:0/3249002973 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074009e70 con 0x7f60841595d0 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f607bfff700 1 -- 172.21.15.45:0/3249002973 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607400b860 con 0x7f60841595d0 2023-10-06T05:15:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f607bfff700 1 -- 172.21.15.45:0/3249002973 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400bc00 con 0x7f60841595d0 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 -- 172.21.15.45:0/3249002973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 msgr2=0x7f608406b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 --2- 172.21.15.45:0/3249002973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f608406b080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f608406ae50 tx=0x7f6074009250 comp rx=0 tx=0).stop 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 -- 172.21.15.45:0/3249002973 shutdown_connections 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 --2- 172.21.15.45:0/3249002973 >> v2:172.21.15.144:3300/0 conn(0x7f6084159200 0x7f60840c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 --2- 172.21.15.45:0/3249002973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f608406b080 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 -- 172.21.15.45:0/3249002973 >> 172.21.15.45:0/3249002973 conn(0x7f608406f090 msgr2=0x7f608406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 -- 172.21.15.45:0/3249002973 shutdown_connections 2023-10-06T05:15:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.317+0000 7f608b929700 1 -- 172.21.15.45:0/3249002973 wait complete. 2023-10-06T05:15:06.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.318+0000 7f608b929700 1 Processor -- start 2023-10-06T05:15:06.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.319+0000 7f608b929700 1 -- start start 2023-10-06T05:15:06.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f608b929700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608406b5c0 0x7f60841677e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f608b929700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6084159200 0x7f608416ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f608b929700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f60840b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f608b929700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60840c67c0 con 0x7f60841595d0 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f608b929700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60840b1210 con 0x7f608406b5c0 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f608b929700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608416b250 con 0x7f6084159200 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f60840b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f60840b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42522/0 (socket says 172.21.15.45:42522) 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 -- 172.21.15.45:0/293347749 learned_addr learned my addr 172.21.15.45:0/293347749 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6088ec4700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6084159200 0x7f608416ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f60896c5700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608406b5c0 0x7f60841677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 -- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6084159200 msgr2=0x7f608416ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6084159200 0x7f608416ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 -- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608406b5c0 msgr2=0x7f60841677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608406b5c0 0x7f60841677e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 -- 172.21.15.45:0/293347749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740094f0 con 0x7f60841595d0 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.320+0000 7f6089ec6700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f60840b1c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f607c00c910 tx=0x7f607c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.321+0000 7f60797fa700 1 -- 172.21.15.45:0/293347749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c012630 con 0x7f60841595d0 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.321+0000 7f60797fa700 1 -- 172.21.15.45:0/293347749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c012790 con 0x7f60841595d0 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.321+0000 7f60797fa700 1 -- 172.21.15.45:0/293347749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c012b50 con 0x7f60841595d0 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.321+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60840b0600 con 0x7f60841595d0 2023-10-06T05:15:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.321+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60840b0bc0 con 0x7f60841595d0 2023-10-06T05:15:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.322+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608406b080 con 0x7f60841595d0 2023-10-06T05:15:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.323+0000 7f60797fa700 1 -- 172.21.15.45:0/293347749 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f607c024420 con 0x7f60841595d0 2023-10-06T05:15:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.324+0000 7f60797fa700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6068066f30 0x7f60680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.324+0000 7f60797fa700 1 -- 172.21.15.45:0/293347749 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f607c0853a0 con 0x7f60841595d0 2023-10-06T05:15:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.324+0000 7f60896c5700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6068066f30 0x7f60680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.325+0000 7f60896c5700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6068066f30 0x7f60680693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6080009db0 tx=0x7f60800091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.328+0000 7f60797fa700 1 -- 172.21.15.45:0/293347749 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f607c055bb0 con 0x7f60841595d0 2023-10-06T05:15:06.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.581+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f608406be20 con 0x7f60841595d0 2023-10-06T05:15:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.581+0000 7f60797fa700 1 -- 172.21.15.45:0/293347749 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f607c055740 con 0x7f60841595d0 2023-10-06T05:15:06.584 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:06.584 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:04.996979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6068066f30 msgr2=0x7f60680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6068066f30 0x7f60680693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6080009db0 tx=0x7f60800091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 msgr2=0x7f60840b1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f60840b1c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f607c00c910 tx=0x7f607c001710 comp rx=0 tx=0).stop 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 shutdown_connections 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6068066f30 0x7f60680693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60841595d0 0x7f60840b1c50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608406b5c0 0x7f60841677e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 --2- 172.21.15.45:0/293347749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6084159200 0x7f608416ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 >> 172.21.15.45:0/293347749 conn(0x7f608406f090 msgr2=0x7f60840af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 shutdown_connections 2023-10-06T05:15:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.584+0000 7f608b929700 1 -- 172.21.15.45:0/293347749 wait complete. 2023-10-06T05:15:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2023-10-06T05:15:06.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.729+0000 7faa8e9c6700 1 Processor -- start 2023-10-06T05:15:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.729+0000 7faa8e9c6700 1 -- start start 2023-10-06T05:15:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa8e9c6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa880c93c0 0x7faa88156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa8e9c6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa88157030 0x7faa880cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa8e9c6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8815b000 con 0x7faa880c8ff0 2023-10-06T05:15:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa8e9c6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa880c9830 con 0x7faa880c93c0 2023-10-06T05:15:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa8e9c6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8815b2e0 con 0x7faa88157030 2023-10-06T05:15:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa880c93c0 0x7faa88156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa880c93c0 0x7faa88156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58234/0 (socket says 172.21.15.45:58234) 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 -- 172.21.15.45:0/142652050 learned_addr learned my addr 172.21.15.45:0/142652050 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa8cf63700 1 --2- 172.21.15.45:0/142652050 >> v2:172.21.15.144:3300/0 conn(0x7faa88157030 0x7faa880cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 -- 172.21.15.45:0/142652050 >> v2:172.21.15.144:3300/0 conn(0x7faa88157030 msgr2=0x7faa880cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 --2- 172.21.15.45:0/142652050 >> v2:172.21.15.144:3300/0 conn(0x7faa88157030 0x7faa880cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 -- 172.21.15.45:0/142652050 >> v1:172.21.15.144:6789/0 conn(0x7faa880c8ff0 legacy=0x7faa880caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 -- 172.21.15.45:0/142652050 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8815b5c0 con 0x7faa880c93c0 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa877fe700 1 --2- 172.21.15.45:0/142652050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa880c93c0 0x7faa88156af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa8815abf0 tx=0x7faa7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ec4e0d2440f4e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa86ffd700 1 -- 172.21.15.45:0/142652050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00b860 con 0x7faa880c93c0 2023-10-06T05:15:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa86ffd700 1 -- 172.21.15.45:0/142652050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c00b9c0 con 0x7faa880c93c0 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.730+0000 7faa86ffd700 1 -- 172.21.15.45:0/142652050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00bd60 con 0x7faa880c93c0 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 -- 172.21.15.45:0/142652050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa880c93c0 msgr2=0x7faa88156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/142652050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa880c93c0 0x7faa88156af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa8815abf0 tx=0x7faa7c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 -- 172.21.15.45:0/142652050 shutdown_connections 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/142652050 >> v2:172.21.15.144:3300/0 conn(0x7faa88157030 0x7faa880cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/142652050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa880c93c0 0x7faa88156af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 -- 172.21.15.45:0/142652050 >> 172.21.15.45:0/142652050 conn(0x7faa880cdf50 msgr2=0x7faa880ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 -- 172.21.15.45:0/142652050 shutdown_connections 2023-10-06T05:15:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.731+0000 7faa8e9c6700 1 -- 172.21.15.45:0/142652050 wait complete. 2023-10-06T05:15:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.733+0000 7faa8e9c6700 1 Processor -- start 2023-10-06T05:15:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.733+0000 7faa8e9c6700 1 -- start start 2023-10-06T05:15:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8e9c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa880c8ff0 0x7faa88149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8e9c6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa880c93c0 0x7faa880c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8e9c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa88157030 0x7faa880c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8e9c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa880c6900 con 0x7faa88157030 2023-10-06T05:15:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8e9c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8814a3a0 con 0x7faa880c8ff0 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8e9c6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa880c5cf0 con 0x7faa880c93c0 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa88157030 0x7faa880c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa88157030 0x7faa880c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58242/0 (socket says 172.21.15.45:58242) 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 -- 172.21.15.45:0/4011396721 learned_addr learned my addr 172.21.15.45:0/4011396721 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa87fff700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa880c8ff0 0x7faa88149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa877fe700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa880c93c0 0x7faa880c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 -- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa880c93c0 msgr2=0x7faa880c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa880c93c0 0x7faa880c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 -- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa880c8ff0 msgr2=0x7faa88149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa880c8ff0 0x7faa88149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.734+0000 7faa8cf63700 1 -- 172.21.15.45:0/4011396721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0094f0 con 0x7faa88157030 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.735+0000 7faa8cf63700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa88157030 0x7faa880c7880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa80009fa0 tx=0x7faa80009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.735+0000 7faa6ffff700 1 -- 172.21.15.45:0/4011396721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000ebc0 con 0x7faa88157030 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.735+0000 7faa6ffff700 1 -- 172.21.15.45:0/4011396721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8000ed20 con 0x7faa88157030 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.735+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa880c5fd0 con 0x7faa88157030 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.735+0000 7faa6ffff700 1 -- 172.21.15.45:0/4011396721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80014670 con 0x7faa88157030 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.735+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa880c6590 con 0x7faa88157030 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.736+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8806f410 con 0x7faa88157030 2023-10-06T05:15:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.736+0000 7faa6ffff700 1 -- 172.21.15.45:0/4011396721 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa800149e0 con 0x7faa88157030 2023-10-06T05:15:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.737+0000 7faa6ffff700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa74066f30 0x7faa740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.737+0000 7faa6ffff700 1 -- 172.21.15.45:0/4011396721 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7faa80084fd0 con 0x7faa88157030 2023-10-06T05:15:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.737+0000 7faa87fff700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa74066f30 0x7faa740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.738+0000 7faa87fff700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa74066f30 0x7faa740693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa78000c00 tx=0x7faa78007ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:06.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.742+0000 7faa6ffff700 1 -- 172.21.15.45:0/4011396721 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa80051df0 con 0x7faa88157030 2023-10-06T05:15:06.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.947+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa880c6a40 con 0x7faa88157030 2023-10-06T05:15:06.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.948+0000 7faa6ffff700 1 -- 172.21.15.45:0/4011396721 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7faa880c6a40 con 0x7faa88157030 2023-10-06T05:15:06.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:06.950 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa74066f30 msgr2=0x7faa740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa74066f30 0x7faa740693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa78000c00 tx=0x7faa78007ea0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa88157030 msgr2=0x7faa880c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa88157030 0x7faa880c7880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa80009fa0 tx=0x7faa80009900 comp rx=0 tx=0).stop 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 shutdown_connections 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa74066f30 0x7faa740693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa88157030 0x7faa880c7880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa880c8ff0 0x7faa88149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 --2- 172.21.15.45:0/4011396721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa880c93c0 0x7faa880c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 >> 172.21.15.45:0/4011396721 conn(0x7faa880cdf50 msgr2=0x7faa8814acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 shutdown_connections 2023-10-06T05:15:06.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:06.950+0000 7faa8e9c6700 1 -- 172.21.15.45:0/4011396721 wait complete. 2023-10-06T05:15:07.367 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:14:59.369246+0000', 'modified': '2023-10-06T05:15:04.996979+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6837/1292454917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 1292454917}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1292454917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2023-10-06T05:15:07.367 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:15:07.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:15:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.508+0000 7f5a1a7d2700 1 Processor -- start 2023-10-06T05:15:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.508+0000 7f5a1a7d2700 1 -- start start 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a1a7d2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5a14159200 0x7f5a140c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a1a7d2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a140ccdc0 0x7f5a140cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a1a7d2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a140c7d00 con 0x7f5a141595d0 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a1a7d2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140c7780 con 0x7f5a14159200 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a1a7d2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140cd720 con 0x7f5a140ccdc0 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a137fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5a141595d0 0x7f5a1406b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34264/0 (socket says 172.21.15.45:34264) 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a137fe700 1 -- 172.21.15.45:0/3396404632 learned_addr learned my addr 172.21.15.45:0/3396404632 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a13fff700 1 --2- 172.21.15.45:0/3396404632 >> v2:172.21.15.144:3300/0 conn(0x7f5a14159200 0x7f5a140c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/3396404632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a140ccdc0 0x7f5a140cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a12ffd700 1 -- 172.21.15.45:0/3396404632 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582100439 0 0) 0x7f5a140c7d00 con 0x7f5a141595d0 2023-10-06T05:15:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.509+0000 7f5a12ffd700 1 -- 172.21.15.45:0/3396404632 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a00003570 con 0x7f5a141595d0 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a12ffd700 1 -- 172.21.15.45:0/3396404632 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557945222 0 0) 0x7f5a00003570 con 0x7f5a141595d0 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a18d6f700 1 -- 172.21.15.45:0/3396404632 >> v2:172.21.15.144:3300/0 conn(0x7f5a14159200 msgr2=0x7f5a140c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/3396404632 >> v2:172.21.15.144:3300/0 conn(0x7f5a14159200 0x7f5a140c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a18d6f700 1 -- 172.21.15.45:0/3396404632 >> v1:172.21.15.45:6789/0 conn(0x7f5a141595d0 legacy=0x7f5a1406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a18d6f700 1 -- 172.21.15.45:0/3396404632 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140c86d0 con 0x7f5a140ccdc0 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a13fff700 1 --2- 172.21.15.45:0/3396404632 >> v2:172.21.15.144:3300/0 conn(0x7f5a14159200 0x7f5a140c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/3396404632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a140ccdc0 0x7f5a140cd1e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009e60 tx=0x7f5a0c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3545c9f79c86c1fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a12ffd700 1 -- 172.21.15.45:0/3396404632 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00e070 con 0x7f5a140ccdc0 2023-10-06T05:15:07.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a12ffd700 1 -- 172.21.15.45:0/3396404632 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c00b7a0 con 0x7f5a140ccdc0 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.510+0000 7f5a12ffd700 1 -- 172.21.15.45:0/3396404632 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc001470 con 0x7f5a140ccdc0 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/3396404632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a140ccdc0 msgr2=0x7f5a140cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/3396404632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a140ccdc0 0x7f5a140cd1e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009e60 tx=0x7f5a0c009780 comp rx=0 tx=0).stop 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/3396404632 shutdown_connections 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/3396404632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a140ccdc0 0x7f5a140cd1e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/3396404632 >> v2:172.21.15.144:3300/0 conn(0x7f5a14159200 0x7f5a140c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/3396404632 >> 172.21.15.45:0/3396404632 conn(0x7f5a1406f090 msgr2=0x7f5a1406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/3396404632 shutdown_connections 2023-10-06T05:15:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.511+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/3396404632 wait complete. 2023-10-06T05:15:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.512+0000 7f5a1a7d2700 1 Processor -- start 2023-10-06T05:15:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.513+0000 7f5a1a7d2700 1 -- start start 2023-10-06T05:15:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a140ccdc0 0x7f5a1414e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a14159200 0x7f5a1415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a141595d0 0x7f5a140c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140c0890 con 0x7f5a140ccdc0 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140b0020 con 0x7f5a14159200 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1415bdb0 con 0x7f5a141595d0 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a140ccdc0 0x7f5a1414e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a140ccdc0 0x7f5a1414e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58256/0 (socket says 172.21.15.45:58256) 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 -- 172.21.15.45:0/380916591 learned_addr learned my addr 172.21.15.45:0/380916591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a137fe700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a14159200 0x7f5a1415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a141595d0 0x7f5a140c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 -- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a141595d0 msgr2=0x7f5a140c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a141595d0 0x7f5a140c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 -- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a14159200 msgr2=0x7f5a1415b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a14159200 0x7f5a1415b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 -- 172.21.15.45:0/380916591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a04009420 con 0x7f5a140ccdc0 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a137fe700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a14159200 0x7f5a1415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a141595d0 0x7f5a140c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a13fff700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a140ccdc0 0x7f5a1414e510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400ec80 tx=0x7f5a040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a0bfff700 1 -- 172.21.15.45:0/380916591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400c9d0 con 0x7f5a140ccdc0 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a0bfff700 1 -- 172.21.15.45:0/380916591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0400cb30 con 0x7f5a140ccdc0 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c009a20 con 0x7f5a140ccdc0 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a0bfff700 1 -- 172.21.15.45:0/380916591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400ceb0 con 0x7f5a140ccdc0 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.514+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1415c370 con 0x7f5a140ccdc0 2023-10-06T05:15:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.516+0000 7f5a0bfff700 1 -- 172.21.15.45:0/380916591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a04013680 con 0x7f5a140ccdc0 2023-10-06T05:15:07.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.516+0000 7f5a0bfff700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59f4066f30 0x7f59f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.517+0000 7f5a0bfff700 1 -- 172.21.15.45:0/380916591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5a04084ef0 con 0x7f5a140ccdc0 2023-10-06T05:15:07.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.517+0000 7f5a137fe700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59f4066f30 0x7f59f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.517+0000 7f5a137fe700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59f4066f30 0x7f59f40693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0011e0 tx=0x7f59fc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.519+0000 7f5a23ae6b80 1 Processor -- start 2023-10-06T05:15:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.519+0000 7f5a23ae6b80 1 -- start start 2023-10-06T05:15:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f72835d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f728365f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f7283664d0 0x55f728366940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728230310 con 0x55f72835d450 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72826d130 con 0x55f728342f40 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f72826aaf0 con 0x55f7283664d0 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f72835d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f72835d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58266/0 (socket says 172.21.15.45:58266) 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 -- 172.21.15.45:0/1515944370 learned_addr learned my addr 172.21.15.45:0/1515944370 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a137fe700 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f728365f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f7283664d0 0x55f728366940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 -- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f7283664d0 msgr2=0x55f728366940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f7283664d0 0x55f728366940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 -- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 msgr2=0x55f728365f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f728365f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 -- 172.21.15.45:0/1515944370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f72806c790 con 0x55f72835d450 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f72835d820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5a04051e30 tx=0x7f5a04082e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f5b275535977b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.520+0000 7f5a097fa700 1 -- 172.21.15.45:0/1515944370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04058c00 con 0x55f72835d450 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a097fa700 1 -- 172.21.15.45:0/1515944370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0405c8d0 con 0x55f72835d450 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a097fa700 1 -- 172.21.15.45:0/1515944370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04053a70 con 0x55f72835d450 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1515944370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 msgr2=0x55f72835d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f72835d820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5a04051e30 tx=0x7f5a04082e80 comp rx=0 tx=0).stop 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1515944370 shutdown_connections 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f7283664d0 0x55f728366940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f72835d820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1515944370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f728365f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1515944370 >> 172.21.15.45:0/1515944370 conn(0x55f728360880 msgr2=0x55f728360c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1515944370 shutdown_connections 2023-10-06T05:15:07.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.521+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1515944370 wait complete. 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.522+0000 7f5a23ae6b80 1 -- start start 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.522+0000 7f5a23ae6b80 1 Processor -- start 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.523+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f72833b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f72833bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f7283664d0 0x55f72835e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7282668e0 con 0x55f7283664d0 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728212980 con 0x55f728342f40 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f728212620 con 0x55f72835d450 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a23ae6b80 10 client.0 _ll_register_callbacks cb 0x55f728075a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:07.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f7283664d0 0x55f72835e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f7283664d0 0x55f72835e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58278/0 (socket says 172.21.15.45:58278) 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 -- 172.21.15.45:0/1791773186 learned_addr learned my addr 172.21.15.45:0/1791773186 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f72833b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a137fe700 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f72833bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 -- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 msgr2=0x55f72833bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f72833bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 -- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 msgr2=0x55f72833b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f72833b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 -- 172.21.15.45:0/1791773186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1415c370 con 0x55f7283664d0 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f7283664d0 0x55f72835e910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a140c78c0 tx=0x7f5a0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae8f4933c7eecdd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f59f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:15:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f59f27fc700 1 -- 172.21.15.45:0/1791773186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c023070 con 0x55f7283664d0 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f59f27fc700 1 -- 172.21.15.45:0/1791773186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5a0c012a80 con 0x55f7283664d0 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.524+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f72830ad60 con 0x55f7283664d0 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f59f27fc700 1 -- 172.21.15.45:0/1791773186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a0c012e50 con 0x55f7283664d0 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 10 client.4431 fetch_fsmap learned FSMap version 6 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55f7281590a0 con 0x55f7283664d0 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f59f27fc700 1 -- 172.21.15.45:0/1791773186 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f72830ad60 con 0x55f7283664d0 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f59f27fc700 1 -- 172.21.15.45:0/1791773186 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f5a0c00b050 con 0x55f7283664d0 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 10 client.4431 fetch_fsmap finished waiting for FSMap version 6 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 10 client.4431 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 20 client.4431 populate_metadata read hostname 'smithi045' 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f72835efa0 0x55f72835f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 4 client.4431 mds_command: new command op to 4276 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 --> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7282e7b30 con 0x55f72835efa0 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.525+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f72835efa0 0x55f72835f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.526+0000 7f59f27fc700 10 client.4431 ms_handle_connect on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.526+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f72835efa0 0x55f72835f370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd220c6db83adbd server_cookie=d1170500b83a1a9b in_seq=0 out_seq=0 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.526+0000 7f59f27fc700 1 -- 172.21.15.45:0/1791773186 <== mds.0 v2:172.21.15.144:6835/1292454917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f7282e7b30 con 0x55f72835efa0 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.526+0000 7f59f27fc700 10 client.4431 handle_command_reply: tid=0 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.526+0000 7f5a23ae6b80 1 client.4431 shutdown 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.526+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f7283664d0 msgr2=0x55f72835e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.526+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f7283664d0 0x55f72835e910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a140c78c0 tx=0x7f5a0c018000 comp rx=0 tx=0).stop 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 shutdown_connections 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f7283664d0 0x55f72835e910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f72835efa0 0x55f72835f370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f728342f40 0x55f72833b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1791773186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f72833bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f59f27fc700 0 client.4431 ms_handle_reset on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 >> 172.21.15.45:0/1791773186 conn(0x55f728360880 msgr2=0x55f728312c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 shutdown_connections 2023-10-06T05:15:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1791773186 wait complete. 2023-10-06T05:15:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.527+0000 7f5a23ae6b80 20 client.4431 trim_cache size 0 max 16384 2023-10-06T05:15:07.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.544+0000 7f5a23ae6b80 1 Processor -- start 2023-10-06T05:15:07.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.545+0000 7f5a23ae6b80 1 -- start start 2023-10-06T05:15:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.545+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f728342f40 0x55f728312440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.545+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f7283437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.545+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728360e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.545+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7280fe1e0 con 0x55f728342f40 2023-10-06T05:15:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.545+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728340710 con 0x55f7283664d0 2023-10-06T05:15:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.545+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f72835d920 con 0x55f72835d450 2023-10-06T05:15:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f728342f40 0x55f728312440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f728342f40 0x55f728312440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58290/0 (socket says 172.21.15.45:58290) 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 -- 172.21.15.45:0/2665317626 learned_addr learned my addr 172.21.15.45:0/2665317626 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728360e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a137fe700 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f7283437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 -- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 msgr2=0x55f7283437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f7283437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 -- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 msgr2=0x55f728360e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728360e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 -- 172.21.15.45:0/2665317626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7282353c0 con 0x55f728342f40 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728360e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a13fff700 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f728342f40 0x55f728312440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a04055320 tx=0x7f5a04063250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d110011d9f35d1f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a09ffb700 1 -- 172.21.15.45:0/2665317626 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04058b80 con 0x55f728342f40 2023-10-06T05:15:07.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a09ffb700 1 -- 172.21.15.45:0/2665317626 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a04013470 con 0x55f728342f40 2023-10-06T05:15:07.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.546+0000 7f5a09ffb700 1 -- 172.21.15.45:0/2665317626 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04053a70 con 0x55f728342f40 2023-10-06T05:15:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/2665317626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f728342f40 msgr2=0x55f728312440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f728342f40 0x55f728312440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a04055320 tx=0x7f5a04063250 comp rx=0 tx=0).stop 2023-10-06T05:15:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/2665317626 shutdown_connections 2023-10-06T05:15:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728360e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f728342f40 0x55f728312440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/2665317626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f72835d450 0x55f7283437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/2665317626 >> 172.21.15.45:0/2665317626 conn(0x55f728360880 msgr2=0x55f72835e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/2665317626 shutdown_connections 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.547+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/2665317626 wait complete. 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 -- start start 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 Processor -- start 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f728342f40 0x55f72833a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f728362710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728335940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728363430 con 0x55f72835d450 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728339ca0 con 0x55f7283664d0 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.548+0000 7f5a23ae6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f728360df0 con 0x55f728342f40 2023-10-06T05:15:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a23ae6b80 10 client.0 _ll_register_callbacks cb 0x55f728075a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f728362710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f728362710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58296/0 (socket says 172.21.15.45:58296) 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 -- 172.21.15.45:0/1223081362 learned_addr learned my addr 172.21.15.45:0/1223081362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a18d6f700 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728335940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f728342f40 0x55f72833a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 -- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f728342f40 msgr2=0x55f72833a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f728342f40 0x55f72833a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 -- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 msgr2=0x55f728335940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728335940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 -- 172.21.15.45:0/1223081362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f72806c790 con 0x55f72835d450 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a137fe700 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f728362710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55f72833f9b0 tx=0x7f59fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71337be4117618c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f59f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f59f27fc700 1 -- 172.21.15.45:0/1223081362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc005bc0 con 0x55f72835d450 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f59f27fc700 1 -- 172.21.15.45:0/1223081362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f59fc00dd30 con 0x55f72835d450 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a0405c8d0 con 0x55f72835d450 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f59f27fc700 1 -- 172.21.15.45:0/1223081362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59fc012620 con 0x55f72835d450 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a23ae6b80 10 client.4437 fetch_fsmap learned FSMap version 6 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.549+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55f7280cd1d0 con 0x55f72835d450 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f59f27fc700 1 -- 172.21.15.45:0/1223081362 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0405c8d0 con 0x55f72835d450 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f59f27fc700 1 -- 172.21.15.45:0/1223081362 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 6) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f59fc012780 con 0x55f72835d450 2023-10-06T05:15:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a23ae6b80 10 client.4437 fetch_fsmap finished waiting for FSMap version 6 2023-10-06T05:15:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a23ae6b80 10 client.4437 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:15:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a23ae6b80 20 client.4437 populate_metadata read hostname 'smithi045' 2023-10-06T05:15:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f728335e80 0x55f72833c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a23ae6b80 4 client.4437 mds_command: new command op to 4276 tid=0[{"prefix": "status"}] 2023-10-06T05:15:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 --> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f7282098d0 con 0x55f728335e80 2023-10-06T05:15:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f728335e80 0x55f72833c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f59f27fc700 10 client.4437 ms_handle_connect on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.550+0000 7f5a13fff700 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f728335e80 0x55f72833c3c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00df6428910ff18 server_cookie=7dba6c863d6dfb97 in_seq=0 out_seq=0 2023-10-06T05:15:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f59f27fc700 1 -- 172.21.15.45:0/1223081362 <== mds.0 v2:172.21.15.144:6835/1292454917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55f7282098d0 con 0x55f728335e80 2023-10-06T05:15:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f59f27fc700 10 client.4437 handle_command_reply: tid=0 2023-10-06T05:15:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 client.4437 shutdown 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 msgr2=0x55f728362710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f728362710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55f72833f9b0 tx=0x7f59fc012000 comp rx=0 tx=0).stop 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 shutdown_connections 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f728335e80 0x55f72833c3c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7283664d0 0x55f728335940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f59f27fc700 0 client.4437 ms_handle_reset on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f728342f40 0x55f72833a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 --2- 172.21.15.45:0/1223081362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f72835d450 0x55f728362710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 >> 172.21.15.45:0/1223081362 conn(0x55f728360880 msgr2=0x55f72835e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 shutdown_connections 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 1 -- 172.21.15.45:0/1223081362 wait complete. 2023-10-06T05:15:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a23ae6b80 20 client.4437 trim_cache size 0 max 16384 2023-10-06T05:15:07.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:15:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:15:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:15:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4276, 2023-10-06T05:15:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:15:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:15:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:15:07.562 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 7.6140447350000002, 2023-10-06T05:15:07.562 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 6, 2023-10-06T05:15:07.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 24, 2023-10-06T05:15:07.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 24, 2023-10-06T05:15:07.562 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 13.601503182 2023-10-06T05:15:07.562 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59f4066f30 msgr2=0x7f59f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59f4066f30 0x7f59f40693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0011e0 tx=0x7f59fc009040 comp rx=0 tx=0).stop 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a140ccdc0 msgr2=0x7f5a1414e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a140ccdc0 0x7f5a1414e510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400ec80 tx=0x7f5a040091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.551+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 shutdown_connections 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.552+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59f4066f30 0x7f59f40693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.552+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a141595d0 0x7f5a140c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.552+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a140ccdc0 0x7f5a1414e510 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.552+0000 7f5a1a7d2700 1 --2- 172.21.15.45:0/380916591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a14159200 0x7f5a1415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.552+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 >> 172.21.15.45:0/380916591 conn(0x7f5a1406f090 msgr2=0x7f5a140adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.552+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 shutdown_connections 2023-10-06T05:15:07.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.552+0000 7f5a1a7d2700 1 -- 172.21.15.45:0/380916591 wait complete. 2023-10-06T05:15:07.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:15:07.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.697+0000 7f72779a7700 1 Processor -- start 2023-10-06T05:15:07.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.697+0000 7f72779a7700 1 -- start start 2023-10-06T05:15:07.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f72779a7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7270155440 0x7f7270155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f72779a7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f727006e6e0 0x7f72700c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f72779a7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72700c8080 con 0x7f72700631f0 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f72779a7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7270068cd0 con 0x7f727006e6e0 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f72779a7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7270156050 con 0x7f7270155440 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7274f42700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f727006e6e0 0x7f72700c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7274f42700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f727006e6e0 0x7f72700c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58300/0 (socket says 172.21.15.45:58300) 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7274f42700 1 -- 172.21.15.45:0/2147116996 learned_addr learned my addr 172.21.15.45:0/2147116996 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7275743700 1 --2- 172.21.15.45:0/2147116996 >> v2:172.21.15.144:3300/0 conn(0x7f7270155440 0x7f7270155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7274f42700 1 -- 172.21.15.45:0/2147116996 >> v2:172.21.15.144:3300/0 conn(0x7f7270155440 msgr2=0x7f7270155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7274f42700 1 --2- 172.21.15.45:0/2147116996 >> v2:172.21.15.144:3300/0 conn(0x7f7270155440 0x7f7270155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7274f42700 1 -- 172.21.15.45:0/2147116996 >> v1:172.21.15.144:6789/0 conn(0x7f72700631f0 legacy=0x7f727006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7274f42700 1 -- 172.21.15.45:0/2147116996 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270156330 con 0x7f727006e6e0 2023-10-06T05:15:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.698+0000 7f7275743700 1 --2- 172.21.15.45:0/2147116996 >> v2:172.21.15.144:3300/0 conn(0x7f7270155440 0x7f7270155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f7274f42700 1 --2- 172.21.15.45:0/2147116996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c8320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7270067a90 tx=0x7f7260009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d917c43fea517a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f7267fff700 1 -- 172.21.15.45:0/2147116996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726000b820 con 0x7f727006e6e0 2023-10-06T05:15:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f7267fff700 1 -- 172.21.15.45:0/2147116996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726000b980 con 0x7f727006e6e0 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f7267fff700 1 -- 172.21.15.45:0/2147116996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726000bd20 con 0x7f727006e6e0 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 -- 172.21.15.45:0/2147116996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 msgr2=0x7f72700c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 --2- 172.21.15.45:0/2147116996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c8320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7270067a90 tx=0x7f7260009250 comp rx=0 tx=0).stop 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 -- 172.21.15.45:0/2147116996 shutdown_connections 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 --2- 172.21.15.45:0/2147116996 >> v2:172.21.15.144:3300/0 conn(0x7f7270155440 0x7f7270155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 --2- 172.21.15.45:0/2147116996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c8320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 -- 172.21.15.45:0/2147116996 >> 172.21.15.45:0/2147116996 conn(0x7f72700cc5a0 msgr2=0x7f72700cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 -- 172.21.15.45:0/2147116996 shutdown_connections 2023-10-06T05:15:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.699+0000 7f72779a7700 1 -- 172.21.15.45:0/2147116996 wait complete. 2023-10-06T05:15:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.701+0000 7f72779a7700 1 Processor -- start 2023-10-06T05:15:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.701+0000 7f72779a7700 1 -- start start 2023-10-06T05:15:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72779a7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72700631f0 0x7f72701697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72779a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72779a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7270155440 0x7f72700c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72779a7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700ab970 con 0x7f727006e6e0 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72779a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727016cff0 con 0x7f7270155440 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72779a7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7270169ed0 con 0x7f72700631f0 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58308/0 (socket says 172.21.15.45:58308) 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 -- 172.21.15.45:0/470152977 learned_addr learned my addr 172.21.15.45:0/470152977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7275f44700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7270155440 0x7f72700c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7275743700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72700631f0 0x7f72701697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 -- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72700631f0 msgr2=0x7f72701697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72700631f0 0x7f72701697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 -- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7270155440 msgr2=0x7f72700c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7270155440 0x7f72700c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 -- 172.21.15.45:0/470152977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600094f0 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f7274f42700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c4a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f72600030c0 tx=0x7f7260016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72657fa700 1 -- 172.21.15.45:0/470152977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260011d00 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72657fa700 1 -- 172.21.15.45:0/470152977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260020070 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72657fa700 1 -- 172.21.15.45:0/470152977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726001b6a0 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.702+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727016a1b0 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.703+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72700cafb0 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.704+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727016cdc0 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.704+0000 7f72657fa700 1 -- 172.21.15.45:0/470152977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7260022030 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.704+0000 7f72657fa700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7254066f30 0x7f72540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.704+0000 7f72657fa700 1 -- 172.21.15.45:0/470152977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7260086830 con 0x7f727006e6e0 2023-10-06T05:15:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.704+0000 7f7275743700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7254066f30 0x7f72540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:07.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.705+0000 7f7275743700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7254066f30 0x7f72540693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f726c006fd0 tx=0x7f726c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:07.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.709+0000 7f72657fa700 1 -- 172.21.15.45:0/470152977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7260053650 con 0x7f727006e6e0 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.957+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f727016d160 con 0x7f727006e6e0 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.957+0000 7f72657fa700 1 -- 172.21.15.45:0/470152977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7260011e60 con 0x7f727006e6e0 2023-10-06T05:15:07.959 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7254066f30 msgr2=0x7f72540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7254066f30 0x7f72540693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f726c006fd0 tx=0x7f726c008040 comp rx=0 tx=0).stop 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 msgr2=0x7f72700c4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c4a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f72600030c0 tx=0x7f7260016000 comp rx=0 tx=0).stop 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 shutdown_connections 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7254066f30 0x7f72540693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7270155440 0x7f72700c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72700631f0 0x7f72701697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 --2- 172.21.15.45:0/470152977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006e6e0 0x7f72700c4a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 >> 172.21.15.45:0/470152977 conn(0x7f72700cc5a0 msgr2=0x7f72700c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 shutdown_connections 2023-10-06T05:15:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:07.960+0000 7f72779a7700 1 -- 172.21.15.45:0/470152977 wait complete. 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:15:07.976 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T05:15:07.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:07.977 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:07.997 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42433sec preferred_lft 42433sec 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:07.998 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:15:08.001 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T05:15:08.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.001 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:08.001 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:08.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2023-10-06T05:15:08.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2023-10-06T05:15:08.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2023-10-06T05:15:08.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T05:15:08.002 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:08.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.184 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:15:08.222 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:15:08.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.223 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:15:08.313 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:15:08.313 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:15:08.313 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:15:08.313 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:15:08.313 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:15:08.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.314 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:08.314 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:08.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:15:08.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:15:08.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:15:08.314 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:08.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.603 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:15:08.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.626 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-10-06T05:15:08.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.689 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:08.689 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:08.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T05:15:08.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-06T05:15:08.690 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:08.800 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-06T05:15:08.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:08.800 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:08.800 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:08.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-06T05:15:08.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T05:15:08.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-06T05:15:08.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-06T05:15:08.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-06T05:15:08.801 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:09.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:09.034 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:09.034 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:09.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:15:09.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T05:15:09.035 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:09.146 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-06T05:15:09.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:09.146 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-06T05:15:09.167 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T05:15:09.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:09.168 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-06T05:15:09.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:15:09.306 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:09.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:09.382 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:09.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:09.383 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:15:09.447 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:15:09.447 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-06T05:15:09.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:15:09.525 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:09.525 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:09.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:2023-10-06T05:15:09.592+0000 7f9d97fec540 -1 init, newargv = 0x556c7337a0d0 newargc=15 2023-10-06T05:15:09.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[127390]: starting ceph client 2023-10-06T05:15:09.598 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:09.599 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:09.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:09.600 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:15:09.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[127390]: starting fuse 2023-10-06T05:15:09.623 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:15:09.623 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:15:09.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:15:09.623 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:15:09.623 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:15:09.623 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:15:09.623 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:15:09.624 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:09.624 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:15:09.624 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-06T05:15:09.624 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:15:09.624 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-06T05:15:09.624 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:15:09.625 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:15:09.626 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:15:09.626 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:15:09.626 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:09.626 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.admin")) 2023-10-06T05:15:09.626 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:15:09.760 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.admin.127390.asok 2023-10-06T05:15:09.941 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.127390.asok 2023-10-06T05:15:09.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:09.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.127390.asok status 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "admin", 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "127390", 2023-10-06T05:15:10.117 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4452, 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4452 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:15:10.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1444074685 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1444074685 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.4452 192.168.0.1:0/1444074685", 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1444074685", 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:15:10.119 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 6, 2023-10-06T05:15:10.120 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 24, 2023-10-06T05:15:10.120 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:15:10.120 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:15:10.120 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:15:10.120 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:15:10.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:10.128 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:15:10.128 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-06T05:15:10.148 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:15:10.148 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-06T05:15:10.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:10.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-06T05:15:10.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:10.222 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-06T05:15:10.392 INFO:teuthology.orchestra.run.smithi045.stdout:# file: . 2023-10-06T05:15:10.392 INFO:teuthology.orchestra.run.smithi045.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-06T05:15:10.392 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:10.392 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:15:10.393 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:15:10.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-06T05:15:10.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[127390]: fuse finished with error 0 and tester_r 0 2023-10-06T05:15:10.444 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:15:16.546 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:15:16.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:16.547 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-06T05:15:16.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:16.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.756+0000 7fcc33983700 1 Processor -- start 2023-10-06T05:15:16.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.756+0000 7fcc33983700 1 -- start start 2023-10-06T05:15:16.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc33983700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc2c0692d0 0x7fcc2c159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc33983700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c15d040 0x7fcc2c0aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc33983700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c0ca6d0 con 0x7fcc2c159170 2023-10-06T05:15:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc33983700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c15d550 con 0x7fcc2c0692d0 2023-10-06T05:15:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc33983700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c15d810 con 0x7fcc2c15d040 2023-10-06T05:15:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc30f1e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc2c0692d0 0x7fcc2c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc30f1e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc2c0692d0 0x7fcc2c159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52970/0 (socket says 172.21.15.45:52970) 2023-10-06T05:15:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.757+0000 7fcc30f1e700 1 -- 172.21.15.45:0/426102097 learned_addr learned my addr 172.21.15.45:0/426102097 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc31f20700 1 --2- 172.21.15.45:0/426102097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c15d040 0x7fcc2c0aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc30f1e700 1 -- 172.21.15.45:0/426102097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c15d040 msgr2=0x7fcc2c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:16.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc30f1e700 1 --2- 172.21.15.45:0/426102097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c15d040 0x7fcc2c0aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc30f1e700 1 -- 172.21.15.45:0/426102097 >> v1:172.21.15.144:6789/0 conn(0x7fcc2c159170 legacy=0x7fcc2c068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc30f1e700 1 -- 172.21.15.45:0/426102097 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c15daf0 con 0x7fcc2c0692d0 2023-10-06T05:15:16.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc30f1e700 1 --2- 172.21.15.45:0/426102097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c0692d0 0x7fcc2c159540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0099b0 tx=0x7fcc1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148157641d1d913 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:16.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc23fff700 1 -- 172.21.15.45:0/426102097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c009ea0 con 0x7fcc2c0692d0 2023-10-06T05:15:16.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc23fff700 1 -- 172.21.15.45:0/426102097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00b860 con 0x7fcc2c0692d0 2023-10-06T05:15:16.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.758+0000 7fcc23fff700 1 -- 172.21.15.45:0/426102097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00bc00 con 0x7fcc2c0692d0 2023-10-06T05:15:16.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 -- 172.21.15.45:0/426102097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c0692d0 msgr2=0x7fcc2c159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 --2- 172.21.15.45:0/426102097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c0692d0 0x7fcc2c159540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0099b0 tx=0x7fcc1c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:16.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 -- 172.21.15.45:0/426102097 shutdown_connections 2023-10-06T05:15:16.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 --2- 172.21.15.45:0/426102097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c15d040 0x7fcc2c0aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 --2- 172.21.15.45:0/426102097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c0692d0 0x7fcc2c159540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 -- 172.21.15.45:0/426102097 >> 172.21.15.45:0/426102097 conn(0x7fcc2c06f7c0 msgr2=0x7fcc2c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 -- 172.21.15.45:0/426102097 shutdown_connections 2023-10-06T05:15:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.759+0000 7fcc33983700 1 -- 172.21.15.45:0/426102097 wait complete. 2023-10-06T05:15:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.760+0000 7fcc33983700 1 Processor -- start 2023-10-06T05:15:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.760+0000 7fcc33983700 1 -- start start 2023-10-06T05:15:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc33983700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c0692d0 0x7fcc2c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc33983700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2c159170 0x7fcc2c149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc33983700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c15d040 0x7fcc2c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc33983700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c149f40 con 0x7fcc2c15d040 2023-10-06T05:15:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc33983700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c14a220 con 0x7fcc2c159170 2023-10-06T05:15:16.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc33983700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c14a500 con 0x7fcc2c0692d0 2023-10-06T05:15:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c15d040 0x7fcc2c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c15d040 0x7fcc2c1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52974/0 (socket says 172.21.15.45:52974) 2023-10-06T05:15:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 -- 172.21.15.45:0/1579712777 learned_addr learned my addr 172.21.15.45:0/1579712777 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc30f1e700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2c159170 0x7fcc2c149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc3171f700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c0692d0 0x7fcc2c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 -- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c0692d0 msgr2=0x7fcc2c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c0692d0 0x7fcc2c15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 -- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2c159170 msgr2=0x7fcc2c149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2c159170 0x7fcc2c149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.761+0000 7fcc31f20700 1 -- 172.21.15.45:0/1579712777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0094f0 con 0x7fcc2c15d040 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.762+0000 7fcc31f20700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c15d040 0x7fcc2c1571d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009df0 tx=0x7fcc2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.762+0000 7fcc217fa700 1 -- 172.21.15.45:0/1579712777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400cb50 con 0x7fcc2c15d040 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.762+0000 7fcc217fa700 1 -- 172.21.15.45:0/1579712777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400f040 con 0x7fcc2c15d040 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.762+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c14a7e0 con 0x7fcc2c15d040 2023-10-06T05:15:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.762+0000 7fcc217fa700 1 -- 172.21.15.45:0/1579712777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24014480 con 0x7fcc2c15d040 2023-10-06T05:15:16.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.762+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c157710 con 0x7fcc2c15d040 2023-10-06T05:15:16.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.763+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c063b50 con 0x7fcc2c15d040 2023-10-06T05:15:16.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.763+0000 7fcc217fa700 1 -- 172.21.15.45:0/1579712777 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc24014830 con 0x7fcc2c15d040 2023-10-06T05:15:16.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.764+0000 7fcc217fa700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc10066f30 0x7fcc100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:16.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.764+0000 7fcc217fa700 1 -- 172.21.15.45:0/1579712777 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcc24085300 con 0x7fcc2c15d040 2023-10-06T05:15:16.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.764+0000 7fcc3171f700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc10066f30 0x7fcc100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:16.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.765+0000 7fcc3171f700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc10066f30 0x7fcc100693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc280037c0 tx=0x7fcc28009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:16.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.769+0000 7fcc217fa700 1 -- 172.21.15.45:0/1579712777 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc24052120 con 0x7fcc2c15d040 2023-10-06T05:15:16.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.972+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc2c15ddd0 con 0x7fcc2c15d040 2023-10-06T05:15:16.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.973+0000 7fcc217fa700 1 -- 172.21.15.45:0/1579712777 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fcc24055740 con 0x7fcc2c15d040 2023-10-06T05:15:16.975 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:16.976 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:16.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc10066f30 msgr2=0x7fcc100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc10066f30 0x7fcc100693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc280037c0 tx=0x7fcc28009040 comp rx=0 tx=0).stop 2023-10-06T05:15:16.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c15d040 msgr2=0x7fcc2c1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:16.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c15d040 0x7fcc2c1571d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009df0 tx=0x7fcc2400b040 comp rx=0 tx=0).stop 2023-10-06T05:15:16.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 shutdown_connections 2023-10-06T05:15:16.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc10066f30 0x7fcc100693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c15d040 0x7fcc2c1571d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2c0692d0 0x7fcc2c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 --2- 172.21.15.45:0/1579712777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2c159170 0x7fcc2c149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:16.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 >> 172.21.15.45:0/1579712777 conn(0x7fcc2c06f7c0 msgr2=0x7fcc2c0b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:16.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 shutdown_connections 2023-10-06T05:15:16.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:16.976+0000 7fcc33983700 1 -- 172.21.15.45:0/1579712777 wait complete. 2023-10-06T05:15:16.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:17.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.122+0000 7f08d8ac2700 1 Processor -- start 2023-10-06T05:15:17.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.122+0000 7f08d8ac2700 1 -- start start 2023-10-06T05:15:17.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d8ac2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40c8ff0 0x7f08d40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d8ac2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f08d4157200 0x7f08d40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d8ac2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d415b000 con 0x7f08d40caad0 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d8ac2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d415b2e0 con 0x7f08d4157200 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d8ac2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d415b5c0 con 0x7f08d40c8ff0 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d1d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f08d40caad0 0x7f08d4156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58470/0 (socket says 172.21.15.45:58470) 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d1d9b700 1 -- 172.21.15.45:0/1506976083 learned_addr learned my addr 172.21.15.45:0/1506976083 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d2d9d700 1 --2- 172.21.15.45:0/1506976083 >> v2:172.21.15.144:3300/0 conn(0x7f08d4157200 0x7f08d40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d259c700 1 --2- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40c8ff0 0x7f08d40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d159a700 1 -- 172.21.15.45:0/1506976083 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769355681 0 0) 0x7f08d415b000 con 0x7f08d40caad0 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d159a700 1 -- 172.21.15.45:0/1506976083 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c0003570 con 0x7f08d40caad0 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d2d9d700 1 -- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40c8ff0 msgr2=0x7f08d40c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d2d9d700 1 --2- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40c8ff0 0x7f08d40c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d2d9d700 1 -- 172.21.15.45:0/1506976083 >> v1:172.21.15.45:6789/0 conn(0x7f08d40caad0 legacy=0x7f08d4156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d159a700 1 -- 172.21.15.45:0/1506976083 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2411681138 0 0) 0x7f08c0003570 con 0x7f08d40caad0 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d2d9d700 1 -- 172.21.15.45:0/1506976083 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d415b8a0 con 0x7f08d4157200 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.123+0000 7f08d259c700 1 --2- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40c8ff0 0x7f08d40c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.124+0000 7f08d2d9d700 1 --2- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d4157200 0x7f08d40caf40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009e60 tx=0x7f08cc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1999e1e32513122d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.124+0000 7f08d159a700 1 -- 172.21.15.45:0/1506976083 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00e070 con 0x7f08d4157200 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.124+0000 7f08d159a700 1 -- 172.21.15.45:0/1506976083 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc00b7a0 con 0x7f08d4157200 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.124+0000 7f08d159a700 1 -- 172.21.15.45:0/1506976083 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc001470 con 0x7f08d4157200 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.124+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d4157200 msgr2=0x7f08d40caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.124+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d4157200 0x7f08d40caf40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009e60 tx=0x7f08cc009780 comp rx=0 tx=0).stop 2023-10-06T05:15:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.125+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1506976083 shutdown_connections 2023-10-06T05:15:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.125+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d4157200 0x7f08d40caf40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.125+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1506976083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40c8ff0 0x7f08d40c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.125+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1506976083 >> 172.21.15.45:0/1506976083 conn(0x7f08d40cdf50 msgr2=0x7f08d40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.125+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1506976083 shutdown_connections 2023-10-06T05:15:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.125+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1506976083 wait complete. 2023-10-06T05:15:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.126+0000 7f08d8ac2700 1 Processor -- start 2023-10-06T05:15:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.126+0000 7f08d8ac2700 1 -- start start 2023-10-06T05:15:17.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d8ac2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d40c8ff0 0x7f08d40aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d8ac2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40caad0 0x7f08d40c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d8ac2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08d4157200 0x7f08d414e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d8ac2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40af330 con 0x7f08d4157200 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d8ac2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40af610 con 0x7f08d40c8ff0 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d8ac2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40af8f0 con 0x7f08d40caad0 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d40c8ff0 0x7f08d40aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d1d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40caad0 0x7f08d40c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d40c8ff0 0x7f08d40aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47178/0 (socket says 172.21.15.45:47178) 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 -- 172.21.15.45:0/1006328383 learned_addr learned my addr 172.21.15.45:0/1006328383 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d2d9d700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08d4157200 0x7f08d414e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 -- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40caad0 msgr2=0x7f08d40c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40caad0 0x7f08d40c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 -- 172.21.15.45:0/1006328383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08d4157200 msgr2=0x7f08d414e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08d4157200 0x7f08d414e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 -- 172.21.15.45:0/1006328383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c4009420 con 0x7f08d40c8ff0 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.127+0000 7f08d259c700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d40c8ff0 0x7f08d40aed20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08c400ec80 tx=0x7f08c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.128+0000 7f08ca7fc700 1 -- 172.21.15.45:0/1006328383 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400c920 con 0x7f08d40c8ff0 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.128+0000 7f08ca7fc700 1 -- 172.21.15.45:0/1006328383 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c400ca80 con 0x7f08d40c8ff0 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.128+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc009a20 con 0x7f08d40c8ff0 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.128+0000 7f08ca7fc700 1 -- 172.21.15.45:0/1006328383 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400ce00 con 0x7f08d40c8ff0 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.128+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d40c5f20 con 0x7f08d40c8ff0 2023-10-06T05:15:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.129+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d40caf00 con 0x7f08d40c8ff0 2023-10-06T05:15:17.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.129+0000 7f08ca7fc700 1 -- 172.21.15.45:0/1006328383 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08c4013680 con 0x7f08d40c8ff0 2023-10-06T05:15:17.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.130+0000 7f08ca7fc700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08b4066f30 0x7f08b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.130+0000 7f08ca7fc700 1 -- 172.21.15.45:0/1006328383 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f08c4084d60 con 0x7f08d40c8ff0 2023-10-06T05:15:17.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.130+0000 7f08d1d9b700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08b4066f30 0x7f08b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.130+0000 7f08d1d9b700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08b4066f30 0x7f08b40693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08d40af260 tx=0x7f08bc001aa0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.135+0000 7f08ca7fc700 1 -- 172.21.15.45:0/1006328383 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08c4051b80 con 0x7f08d40c8ff0 2023-10-06T05:15:17.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.338+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08d404bb40 con 0x7f08d40c8ff0 2023-10-06T05:15:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.340+0000 7f08ca7fc700 1 -- 172.21.15.45:0/1006328383 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f08c40551a0 con 0x7f08d40c8ff0 2023-10-06T05:15:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:17.342 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:17.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08b4066f30 msgr2=0x7f08b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08b4066f30 0x7f08b40693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08d40af260 tx=0x7f08bc001aa0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d40c8ff0 msgr2=0x7f08d40aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d40c8ff0 0x7f08d40aed20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08c400ec80 tx=0x7f08c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 shutdown_connections 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08b4066f30 0x7f08b40693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08d4157200 0x7f08d414e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d40c8ff0 0x7f08d40aed20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 --2- 172.21.15.45:0/1006328383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08d40caad0 0x7f08d40c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 >> 172.21.15.45:0/1006328383 conn(0x7f08d40cdf50 msgr2=0x7f08d406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 shutdown_connections 2023-10-06T05:15:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.342+0000 7f08d8ac2700 1 -- 172.21.15.45:0/1006328383 wait complete. 2023-10-06T05:15:17.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:15:17.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.491+0000 7fd9e6696700 1 Processor -- start 2023-10-06T05:15:17.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.491+0000 7fd9e6696700 1 -- start start 2023-10-06T05:15:17.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9e6696700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd9e006aa70 0x7fd9e00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9e6696700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9e6696700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e0067d80 con 0x7fd9e00c7050 2023-10-06T05:15:17.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9e6696700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00ac620 con 0x7fd9e006aa70 2023-10-06T05:15:17.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9e6696700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c75b0 con 0x7fd9e015c8c0 2023-10-06T05:15:17.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9df7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd9e00c7050 0x7fd9e015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58478/0 (socket says 172.21.15.45:58478) 2023-10-06T05:15:17.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9df7fe700 1 -- 172.21.15.45:0/2168224928 learned_addr learned my addr 172.21.15.45:0/2168224928 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9dffff700 1 --2- 172.21.15.45:0/2168224928 >> v2:172.21.15.144:3300/0 conn(0x7fd9e006aa70 0x7fd9e00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.492+0000 7fd9e4c33700 1 --2- 172.21.15.45:0/2168224928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9deffd700 1 -- 172.21.15.45:0/2168224928 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652672738 0 0) 0x7fd9e0067d80 con 0x7fd9e00c7050 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9deffd700 1 -- 172.21.15.45:0/2168224928 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c8003570 con 0x7fd9e00c7050 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9e4c33700 1 -- 172.21.15.45:0/2168224928 >> v2:172.21.15.144:3300/0 conn(0x7fd9e006aa70 msgr2=0x7fd9e00c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9e4c33700 1 --2- 172.21.15.45:0/2168224928 >> v2:172.21.15.144:3300/0 conn(0x7fd9e006aa70 0x7fd9e00c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9e4c33700 1 -- 172.21.15.45:0/2168224928 >> v1:172.21.15.45:6789/0 conn(0x7fd9e00c7050 legacy=0x7fd9e015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9e4c33700 1 -- 172.21.15.45:0/2168224928 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00c7890 con 0x7fd9e015c8c0 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9e4c33700 1 --2- 172.21.15.45:0/2168224928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00aca70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009e60 tx=0x7fd9d8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89137ef8999e357b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9deffd700 1 -- 172.21.15.45:0/2168224928 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800e070 con 0x7fd9e015c8c0 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9deffd700 1 -- 172.21.15.45:0/2168224928 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d800bab0 con 0x7fd9e015c8c0 2023-10-06T05:15:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.493+0000 7fd9deffd700 1 -- 172.21.15.45:0/2168224928 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800be70 con 0x7fd9e015c8c0 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 -- 172.21.15.45:0/2168224928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 msgr2=0x7fd9e00aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2168224928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00aca70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009e60 tx=0x7fd9d8009780 comp rx=0 tx=0).stop 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 -- 172.21.15.45:0/2168224928 shutdown_connections 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2168224928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00aca70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2168224928 >> v2:172.21.15.144:3300/0 conn(0x7fd9e006aa70 0x7fd9e00c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 -- 172.21.15.45:0/2168224928 >> 172.21.15.45:0/2168224928 conn(0x7fd9e00cc570 msgr2=0x7fd9e00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 -- 172.21.15.45:0/2168224928 shutdown_connections 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.494+0000 7fd9e6696700 1 -- 172.21.15.45:0/2168224928 wait complete. 2023-10-06T05:15:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.495+0000 7fd9e6696700 1 Processor -- start 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.495+0000 7fd9e6696700 1 -- start start 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9e6696700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e006aa70 0x7fd9e00b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9e6696700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e00c7050 0x7fd9e00a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9e6696700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9e6696700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00b1690 con 0x7fd9e006aa70 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9e6696700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00b0a80 con 0x7fd9e00c7050 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9e6696700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00b0d60 con 0x7fd9e015c8c0 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e006aa70 0x7fd9e00b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e006aa70 0x7fd9e00b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52980/0 (socket says 172.21.15.45:52980) 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 -- 172.21.15.45:0/2149811906 learned_addr learned my addr 172.21.15.45:0/2149811906 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9df7fe700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e00c7050 0x7fd9e00a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9e4c33700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 -- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 msgr2=0x7fd9e00a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 -- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e00c7050 msgr2=0x7fd9e00a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e00c7050 0x7fd9e00a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.496+0000 7fd9dffff700 1 -- 172.21.15.45:0/2149811906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8009a20 con 0x7fd9e006aa70 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.497+0000 7fd9dffff700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e006aa70 0x7fd9e00b04a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000ec80 tx=0x7fd9d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.497+0000 7fd9c7fff700 1 -- 172.21.15.45:0/2149811906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000c850 con 0x7fd9e006aa70 2023-10-06T05:15:17.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.497+0000 7fd9c7fff700 1 -- 172.21.15.45:0/2149811906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d000c9b0 con 0x7fd9e006aa70 2023-10-06T05:15:17.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.497+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e00b1040 con 0x7fd9e006aa70 2023-10-06T05:15:17.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.497+0000 7fd9c7fff700 1 -- 172.21.15.45:0/2149811906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000cd50 con 0x7fd9e006aa70 2023-10-06T05:15:17.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.497+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e00b1b20 con 0x7fd9e006aa70 2023-10-06T05:15:17.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.498+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e006e300 con 0x7fd9e006aa70 2023-10-06T05:15:17.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.498+0000 7fd9c7fff700 1 -- 172.21.15.45:0/2149811906 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9d0013680 con 0x7fd9e006aa70 2023-10-06T05:15:17.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.499+0000 7fd9c7fff700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9cc066f30 0x7fd9cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.499+0000 7fd9c7fff700 1 -- 172.21.15.45:0/2149811906 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd9d0084c00 con 0x7fd9e006aa70 2023-10-06T05:15:17.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.499+0000 7fd9df7fe700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9cc066f30 0x7fd9cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.499+0000 7fd9df7fe700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9cc066f30 0x7fd9cc0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd9e015baa0 tx=0x7fd9d4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.503+0000 7fd9c7fff700 1 -- 172.21.15.45:0/2149811906 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9d008b080 con 0x7fd9e006aa70 2023-10-06T05:15:17.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.750+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9e00b1970 con 0x7fd9e006aa70 2023-10-06T05:15:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.783+0000 7fd9c7fff700 1 -- 172.21.15.45:0/2149811906 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd9d0051a20 con 0x7fd9e006aa70 2023-10-06T05:15:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9cc066f30 msgr2=0x7fd9cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9cc066f30 0x7fd9cc0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd9e015baa0 tx=0x7fd9d4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e006aa70 msgr2=0x7fd9e00b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e006aa70 0x7fd9e00b04a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000ec80 tx=0x7fd9d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 shutdown_connections 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9cc066f30 0x7fd9cc0693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9e015c8c0 0x7fd9e00a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e006aa70 0x7fd9e00b04a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 --2- 172.21.15.45:0/2149811906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e00c7050 0x7fd9e00a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 >> 172.21.15.45:0/2149811906 conn(0x7fd9e00cc570 msgr2=0x7fd9e0156b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 shutdown_connections 2023-10-06T05:15:17.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.785+0000 7fd9e6696700 1 -- 172.21.15.45:0/2149811906 wait complete. 2023-10-06T05:15:17.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:15:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.935+0000 7f9b264ad700 1 Processor -- start 2023-10-06T05:15:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.935+0000 7f9b264ad700 1 -- start start 2023-10-06T05:15:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.936+0000 7f9b264ad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b200b0610 0x7f9b200c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.936+0000 7f9b264ad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b20069ed0 0x7f9b200cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.936+0000 7f9b264ad700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b200ce6a0 con 0x7f9b200c3db0 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.936+0000 7f9b264ad700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200690f0 con 0x7f9b200b0610 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.936+0000 7f9b264ad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200ceeb0 con 0x7f9b20069ed0 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b24a4a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b20069ed0 0x7f9b200cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b200b0610 0x7f9b200c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1f7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9b200c3db0 0x7f9b200697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50746/0 (socket says 172.21.15.45:50746) 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1f7fe700 1 -- 172.21.15.45:0/3133059859 learned_addr learned my addr 172.21.15.45:0/3133059859 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b24a4a700 1 --2- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b20069ed0 0x7f9b200cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33700/0 (socket says 172.21.15.45:33700) 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1ffff700 1 -- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b20069ed0 msgr2=0x7f9b200cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1ffff700 1 --2- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b20069ed0 0x7f9b200cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1ffff700 1 -- 172.21.15.45:0/3133059859 >> v1:172.21.15.144:6789/0 conn(0x7f9b200c3db0 legacy=0x7f9b200697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1ffff700 1 -- 172.21.15.45:0/3133059859 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b20159880 con 0x7f9b200b0610 2023-10-06T05:15:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1effd700 1 -- 172.21.15.45:0/3133059859 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871148498 0 0) 0x7f9b200ce6a0 con 0x7f9b200c3db0 2023-10-06T05:15:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.937+0000 7f9b1ffff700 1 --2- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b200b0610 0x7f9b200c85d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009900 tx=0x7f9b10009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e39fcb04911acc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b1effd700 1 -- 172.21.15.45:0/3133059859 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10005c10 con 0x7f9b200b0610 2023-10-06T05:15:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b1effd700 1 -- 172.21.15.45:0/3133059859 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b10005d70 con 0x7f9b200b0610 2023-10-06T05:15:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b1effd700 1 -- 172.21.15.45:0/3133059859 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10006260 con 0x7f9b200b0610 2023-10-06T05:15:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 -- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b200b0610 msgr2=0x7f9b200c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 --2- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b200b0610 0x7f9b200c85d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009900 tx=0x7f9b10009120 comp rx=0 tx=0).stop 2023-10-06T05:15:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 -- 172.21.15.45:0/3133059859 shutdown_connections 2023-10-06T05:15:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 --2- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b20069ed0 0x7f9b200cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 --2- 172.21.15.45:0/3133059859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b200b0610 0x7f9b200c85d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 -- 172.21.15.45:0/3133059859 >> 172.21.15.45:0/3133059859 conn(0x7f9b20067e00 msgr2=0x7f9b200cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 -- 172.21.15.45:0/3133059859 shutdown_connections 2023-10-06T05:15:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.938+0000 7f9b264ad700 1 -- 172.21.15.45:0/3133059859 wait complete. 2023-10-06T05:15:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.940+0000 7f9b264ad700 1 Processor -- start 2023-10-06T05:15:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.940+0000 7f9b264ad700 1 -- start start 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b264ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b20069ed0 0x7f9b2015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b264ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b200b0610 0x7f9b200b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b264ad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b200c3db0 0x7f9b200b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b264ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2009ba70 con 0x7f9b200b0610 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b264ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2015adc0 con 0x7f9b20069ed0 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b264ad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2015b0a0 con 0x7f9b200c3db0 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b1f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b200b0610 0x7f9b200b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b1f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b200b0610 0x7f9b200b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52988/0 (socket says 172.21.15.45:52988) 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b1f7fe700 1 -- 172.21.15.45:0/2563234526 learned_addr learned my addr 172.21.15.45:0/2563234526 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b1ffff700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b20069ed0 0x7f9b2015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.941+0000 7f9b24a4a700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b200c3db0 0x7f9b200b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b1f7fe700 1 -- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b200c3db0 msgr2=0x7f9b200b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b1f7fe700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b200c3db0 0x7f9b200b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b1f7fe700 1 -- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b20069ed0 msgr2=0x7f9b2015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b1f7fe700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b20069ed0 0x7f9b2015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b1f7fe700 1 -- 172.21.15.45:0/2563234526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100093c0 con 0x7f9b200b0610 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b1f7fe700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b200b0610 0x7f9b200b1bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c5b0 tx=0x7f9b14001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b07fff700 1 -- 172.21.15.45:0/2563234526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14005b30 con 0x7f9b200b0610 2023-10-06T05:15:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b07fff700 1 -- 172.21.15.45:0/2563234526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b14005c90 con 0x7f9b200b0610 2023-10-06T05:15:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2015b380 con 0x7f9b200b0610 2023-10-06T05:15:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b07fff700 1 -- 172.21.15.45:0/2563234526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14007300 con 0x7f9b200b0610 2023-10-06T05:15:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.942+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2015b9c0 con 0x7f9b200b0610 2023-10-06T05:15:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.943+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2015bb50 con 0x7f9b200b0610 2023-10-06T05:15:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.944+0000 7f9b07fff700 1 -- 172.21.15.45:0/2563234526 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b1401b3b0 con 0x7f9b200b0610 2023-10-06T05:15:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.944+0000 7f9b07fff700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b0c066f30 0x7f9b0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.944+0000 7f9b07fff700 1 -- 172.21.15.45:0/2563234526 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9b14083dd0 con 0x7f9b200b0610 2023-10-06T05:15:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.944+0000 7f9b1ffff700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b0c066f30 0x7f9b0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.945+0000 7f9b1ffff700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b0c066f30 0x7f9b0c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009160 tx=0x7f9b10016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:17.949+0000 7f9b07fff700 1 -- 172.21.15.45:0/2563234526 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b14050bf0 con 0x7f9b200b0610 2023-10-06T05:15:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.198+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9b2006e350 con 0x7f9b200b0610 2023-10-06T05:15:18.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.788+0000 7f9b07fff700 1 -- 172.21.15.45:0/2563234526 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9b140850c0 con 0x7f9b200b0610 2023-10-06T05:15:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b0c066f30 msgr2=0x7f9b0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b0c066f30 0x7f9b0c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009160 tx=0x7f9b10016000 comp rx=0 tx=0).stop 2023-10-06T05:15:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b200b0610 msgr2=0x7f9b200b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b200b0610 0x7f9b200b1bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c5b0 tx=0x7f9b14001070 comp rx=0 tx=0).stop 2023-10-06T05:15:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 shutdown_connections 2023-10-06T05:15:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b0c066f30 0x7f9b0c0693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b200c3db0 0x7f9b200b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b20069ed0 0x7f9b2015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 --2- 172.21.15.45:0/2563234526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b200b0610 0x7f9b200b1bb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 >> 172.21.15.45:0/2563234526 conn(0x7f9b20067e00 msgr2=0x7f9b200c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 shutdown_connections 2023-10-06T05:15:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.791+0000 7f9b264ad700 1 -- 172.21.15.45:0/2563234526 wait complete. 2023-10-06T05:15:18.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:18.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.943+0000 7f1947b53700 1 Processor -- start 2023-10-06T05:15:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.943+0000 7f1947b53700 1 -- start start 2023-10-06T05:15:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f1947b53700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153a20 0x7f194006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f1947b53700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f194006add0 0x7f1940068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f1947b53700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19400676c0 con 0x7f1940153650 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f1947b53700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194006b250 con 0x7f194006add0 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f1947b53700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19400679a0 con 0x7f1940153a20 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19460f0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f194006add0 0x7f1940068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19458ef700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1940153650 0x7f194015a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58494/0 (socket says 172.21.15.45:58494) 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19458ef700 1 -- 172.21.15.45:0/1718807228 learned_addr learned my addr 172.21.15.45:0/1718807228 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19450ee700 1 --2- 172.21.15.45:0/1718807228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153a20 0x7f194006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19460f0700 1 -- 172.21.15.45:0/1718807228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153a20 msgr2=0x7f194006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19460f0700 1 --2- 172.21.15.45:0/1718807228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153a20 0x7f194006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19460f0700 1 -- 172.21.15.45:0/1718807228 >> v1:172.21.15.45:6789/0 conn(0x7f1940153650 legacy=0x7f194015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19448ed700 1 -- 172.21.15.45:0/1718807228 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148893637 0 0) 0x7f19400676c0 con 0x7f1940153650 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19460f0700 1 -- 172.21.15.45:0/1718807228 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1940067c80 con 0x7f194006add0 2023-10-06T05:15:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.944+0000 7f19460f0700 1 --2- 172.21.15.45:0/1718807228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f1940068480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19380098e0 tx=0x7f19380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9441076421ce584 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f19448ed700 1 -- 172.21.15.45:0/1718807228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800b640 con 0x7f194006add0 2023-10-06T05:15:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f19448ed700 1 -- 172.21.15.45:0/1718807228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193800b7a0 con 0x7f194006add0 2023-10-06T05:15:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f19448ed700 1 -- 172.21.15.45:0/1718807228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800bb40 con 0x7f194006add0 2023-10-06T05:15:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 -- 172.21.15.45:0/1718807228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 msgr2=0x7f1940068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 --2- 172.21.15.45:0/1718807228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f1940068480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19380098e0 tx=0x7f19380091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 -- 172.21.15.45:0/1718807228 shutdown_connections 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 --2- 172.21.15.45:0/1718807228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f1940068480 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 --2- 172.21.15.45:0/1718807228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153a20 0x7f194006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 -- 172.21.15.45:0/1718807228 >> 172.21.15.45:0/1718807228 conn(0x7f19400cc5a0 msgr2=0x7f19400cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 -- 172.21.15.45:0/1718807228 shutdown_connections 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.945+0000 7f1947b53700 1 -- 172.21.15.45:0/1718807228 wait complete. 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.947+0000 7f1947b53700 1 Processor -- start 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.947+0000 7f1947b53700 1 -- start start 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.947+0000 7f1947b53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f19400c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.947+0000 7f1947b53700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153650 0x7f19400c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.947+0000 7f1947b53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1940153a20 0x7f194016a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.947+0000 7f1947b53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940169900 con 0x7f194006add0 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.947+0000 7f1947b53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940169be0 con 0x7f1940153a20 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f1947b53700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1940169ec0 con 0x7f1940153650 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f19400c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f19400c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53010/0 (socket says 172.21.15.45:53010) 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 -- 172.21.15.45:0/31900186 learned_addr learned my addr 172.21.15.45:0/31900186 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19460f0700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1940153a20 0x7f194016a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19450ee700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153650 0x7f19400c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 -- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153650 msgr2=0x7f19400c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153650 0x7f19400c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 -- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1940153a20 msgr2=0x7f194016a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1940153a20 0x7f194016a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 -- 172.21.15.45:0/31900186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1938009420 con 0x7f194006add0 2023-10-06T05:15:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f19458ef700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f19400c3e70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f193c00cd80 tx=0x7f193c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f1935ffb700 1 -- 172.21.15.45:0/31900186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00aa60 con 0x7f194006add0 2023-10-06T05:15:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f1935ffb700 1 -- 172.21.15.45:0/31900186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c00abc0 con 0x7f194006add0 2023-10-06T05:15:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194016a1a0 con 0x7f194006add0 2023-10-06T05:15:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.948+0000 7f1935ffb700 1 -- 172.21.15.45:0/31900186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0070f0 con 0x7f194006add0 2023-10-06T05:15:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.949+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19400c43b0 con 0x7f194006add0 2023-10-06T05:15:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.950+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194016ce90 con 0x7f194006add0 2023-10-06T05:15:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.950+0000 7f1935ffb700 1 -- 172.21.15.45:0/31900186 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f193c0074e0 con 0x7f194006add0 2023-10-06T05:15:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.950+0000 7f1935ffb700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1924066f30 0x7f19240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.950+0000 7f1935ffb700 1 -- 172.21.15.45:0/31900186 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f193c084d70 con 0x7f194006add0 2023-10-06T05:15:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.950+0000 7f19450ee700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1924066f30 0x7f19240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:18.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.951+0000 7f19450ee700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1924066f30 0x7f19240693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f19400aeab0 tx=0x7f19300091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:18.955+0000 7f1935ffb700 1 -- 172.21.15.45:0/31900186 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f194016ce90 con 0x7f194006add0 2023-10-06T05:15:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.209+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f194016ce90 con 0x7f194006add0 2023-10-06T05:15:19.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.209+0000 7f1935ffb700 1 -- 172.21.15.45:0/31900186 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f194016ce90 con 0x7f194006add0 2023-10-06T05:15:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:19.211 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1924066f30 msgr2=0x7f19240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1924066f30 0x7f19240693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f19400aeab0 tx=0x7f19300091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 msgr2=0x7f19400c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f19400c3e70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f193c00cd80 tx=0x7f193c001ae0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 shutdown_connections 2023-10-06T05:15:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1924066f30 0x7f19240693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1940153a20 0x7f194016a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f194006add0 0x7f19400c3e70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 --2- 172.21.15.45:0/31900186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1940153650 0x7f19400c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 >> 172.21.15.45:0/31900186 conn(0x7f19400cc5a0 msgr2=0x7f19400cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 shutdown_connections 2023-10-06T05:15:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.212+0000 7f1947b53700 1 -- 172.21.15.45:0/31900186 wait complete. 2023-10-06T05:15:19.216 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:19.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.357+0000 7f45112cd700 1 Processor -- start 2023-10-06T05:15:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.357+0000 7f45112cd700 1 -- start start 2023-10-06T05:15:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f45112cd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f450c0692d0 0x7f450c159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f45112cd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f450c15d040 0x7f450c0aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f45112cd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c0ca6d0 con 0x7f450c159170 2023-10-06T05:15:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f45112cd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450c15d550 con 0x7f450c0692d0 2023-10-06T05:15:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f45112cd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450c15d810 con 0x7f450c15d040 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f450c0692d0 0x7f450c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f450c159170 0x7f450c068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58502/0 (socket says 172.21.15.45:58502) 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450affd700 1 -- 172.21.15.45:0/1997357674 learned_addr learned my addr 172.21.15.45:0/1997357674 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450b7fe700 1 --2- 172.21.15.45:0/1997357674 >> v2:172.21.15.144:3300/0 conn(0x7f450c15d040 0x7f450c0aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450a7fc700 1 -- 172.21.15.45:0/1997357674 >> v2:172.21.15.144:3300/0 conn(0x7f450c15d040 msgr2=0x7f450c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450a7fc700 1 --2- 172.21.15.45:0/1997357674 >> v2:172.21.15.144:3300/0 conn(0x7f450c15d040 0x7f450c0aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450a7fc700 1 -- 172.21.15.45:0/1997357674 >> v1:172.21.15.45:6789/0 conn(0x7f450c159170 legacy=0x7f450c068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.358+0000 7f450a7fc700 1 -- 172.21.15.45:0/1997357674 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c15daf0 con 0x7f450c0692d0 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f450a7fc700 1 --2- 172.21.15.45:0/1997357674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c159540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f450c0c70c0 tx=0x7f44f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a328a6bef71a3be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f4509ffb700 1 -- 172.21.15.45:0/1997357674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400b6b0 con 0x7f450c0692d0 2023-10-06T05:15:19.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f4509ffb700 1 -- 172.21.15.45:0/1997357674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f400b810 con 0x7f450c0692d0 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f4509ffb700 1 -- 172.21.15.45:0/1997357674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400bbb0 con 0x7f450c0692d0 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f45112cd700 1 -- 172.21.15.45:0/1997357674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 msgr2=0x7f450c159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f45112cd700 1 --2- 172.21.15.45:0/1997357674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c159540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f450c0c70c0 tx=0x7f44f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f45112cd700 1 -- 172.21.15.45:0/1997357674 shutdown_connections 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f45112cd700 1 --2- 172.21.15.45:0/1997357674 >> v2:172.21.15.144:3300/0 conn(0x7f450c15d040 0x7f450c0aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f45112cd700 1 --2- 172.21.15.45:0/1997357674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c159540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.359+0000 7f45112cd700 1 -- 172.21.15.45:0/1997357674 >> 172.21.15.45:0/1997357674 conn(0x7f450c06f7c0 msgr2=0x7f450c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.360+0000 7f45112cd700 1 -- 172.21.15.45:0/1997357674 shutdown_connections 2023-10-06T05:15:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.360+0000 7f45112cd700 1 -- 172.21.15.45:0/1997357674 wait complete. 2023-10-06T05:15:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.361+0000 7f45112cd700 1 Processor -- start 2023-10-06T05:15:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.361+0000 7f45112cd700 1 -- start start 2023-10-06T05:15:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f45112cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f45112cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c159170 0x7f450c16d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f45112cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f450c15d040 0x7f450c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f45112cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c16e0a0 con 0x7f450c0692d0 2023-10-06T05:15:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f45112cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c16d800 con 0x7f450c159170 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f45112cd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c16dae0 con 0x7f450c15d040 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c0c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53024/0 (socket says 172.21.15.45:53024) 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 -- 172.21.15.45:0/3199393301 learned_addr learned my addr 172.21.15.45:0/3199393301 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450a7fc700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c159170 0x7f450c16d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450b7fe700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f450c15d040 0x7f450c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 -- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f450c15d040 msgr2=0x7f450c156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f450c15d040 0x7f450c156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 -- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c159170 msgr2=0x7f450c16d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c159170 0x7f450c16d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.362+0000 7f450affd700 1 -- 172.21.15.45:0/3199393301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4009420 con 0x7f450c0692d0 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.363+0000 7f450affd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c0c83c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c7c0 tx=0x7f44fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.363+0000 7f4502ffd700 1 -- 172.21.15.45:0/3199393301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc005b30 con 0x7f450c0692d0 2023-10-06T05:15:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.363+0000 7f4502ffd700 1 -- 172.21.15.45:0/3199393301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc005c90 con 0x7f450c0692d0 2023-10-06T05:15:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.363+0000 7f4502ffd700 1 -- 172.21.15.45:0/3199393301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004a80 con 0x7f450c0692d0 2023-10-06T05:15:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.363+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c16ddc0 con 0x7f450c0692d0 2023-10-06T05:15:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.363+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c1569b0 con 0x7f450c0692d0 2023-10-06T05:15:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.364+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c063b50 con 0x7f450c0692d0 2023-10-06T05:15:19.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.364+0000 7f4502ffd700 1 -- 172.21.15.45:0/3199393301 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44fc0070d0 con 0x7f450c0692d0 2023-10-06T05:15:19.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.365+0000 7f4502ffd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44ec066f30 0x7f44ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.365+0000 7f4502ffd700 1 -- 172.21.15.45:0/3199393301 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f44fc083c70 con 0x7f450c0692d0 2023-10-06T05:15:19.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.365+0000 7f450a7fc700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44ec066f30 0x7f44ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.365+0000 7f450a7fc700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44ec066f30 0x7f44ec0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f450c0696d0 tx=0x7f44f4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.369+0000 7f4502ffd700 1 -- 172.21.15.45:0/3199393301 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44fc04b5a0 con 0x7f450c0692d0 2023-10-06T05:15:19.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.572+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f450c15ddd0 con 0x7f450c0692d0 2023-10-06T05:15:19.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.574+0000 7f4502ffd700 1 -- 172.21.15.45:0/3199393301 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f44fc050a90 con 0x7f450c0692d0 2023-10-06T05:15:19.576 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:19.576 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44ec066f30 msgr2=0x7f44ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44ec066f30 0x7f44ec0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f450c0696d0 tx=0x7f44f4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 msgr2=0x7f450c0c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c0c83c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c7c0 tx=0x7f44fc007f70 comp rx=0 tx=0).stop 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 shutdown_connections 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44ec066f30 0x7f44ec0693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f450c15d040 0x7f450c156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0692d0 0x7f450c0c83c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 --2- 172.21.15.45:0/3199393301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c159170 0x7f450c16d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 >> 172.21.15.45:0/3199393301 conn(0x7f450c06f7c0 msgr2=0x7f450c0aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 shutdown_connections 2023-10-06T05:15:19.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.576+0000 7f45112cd700 1 -- 172.21.15.45:0/3199393301 wait complete. 2023-10-06T05:15:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:15:19.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:19.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.722+0000 7f7ea1fb1700 1 Processor -- start 2023-10-06T05:15:19.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.722+0000 7f7ea1fb1700 1 -- start start 2023-10-06T05:15:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.722+0000 7f7ea1fb1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e9c153a20 0x7f7e9c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.722+0000 7f7ea1fb1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c06add0 0x7f7e9c068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.722+0000 7f7ea1fb1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0676c0 con 0x7f7e9c153650 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.722+0000 7f7ea1fb1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c06b270 con 0x7f7e9c153a20 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.722+0000 7f7ea1fb1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0679a0 con 0x7f7e9c06add0 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e9c153a20 0x7f7e9c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c06add0 0x7f7e9c068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e9c153a20 0x7f7e9c06a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47244/0 (socket says 172.21.15.45:47244) 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 -- 172.21.15.45:0/2552270952 learned_addr learned my addr 172.21.15.45:0/2552270952 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 -- 172.21.15.45:0/2552270952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c06add0 msgr2=0x7f7e9c068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 --2- 172.21.15.45:0/2552270952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c06add0 0x7f7e9c068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 -- 172.21.15.45:0/2552270952 >> v1:172.21.15.144:6789/0 conn(0x7f7e9c153650 legacy=0x7f7e9c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/2552270952 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985720159 0 0) 0x7f7e9c0676c0 con 0x7f7e9c153650 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 -- 172.21.15.45:0/2552270952 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c067c80 con 0x7f7e9c153a20 2023-10-06T05:15:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.723+0000 7f7e9affd700 1 --2- 172.21.15.45:0/2552270952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 0x7f7e9c06a890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e840099b0 tx=0x7f7e84009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622168ed765b4868 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/2552270952 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84009ea0 con 0x7f7e9c153a20 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/2552270952 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8400b860 con 0x7f7e9c153a20 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/2552270952 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400bc00 con 0x7f7e9c153a20 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/2552270952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 msgr2=0x7f7e9c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/2552270952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 0x7f7e9c06a890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e840099b0 tx=0x7f7e84009250 comp rx=0 tx=0).stop 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/2552270952 shutdown_connections 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/2552270952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c06add0 0x7f7e9c068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/2552270952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 0x7f7e9c06a890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/2552270952 >> 172.21.15.45:0/2552270952 conn(0x7f7e9c0cc5a0 msgr2=0x7f7e9c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/2552270952 shutdown_connections 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.724+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/2552270952 wait complete. 2023-10-06T05:15:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.725+0000 7f7ea1fb1700 1 Processor -- start 2023-10-06T05:15:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7ea1fb1700 1 -- start start 2023-10-06T05:15:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7ea1fb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9c06add0 0x7f7e9c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7ea1fb1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c153650 0x7f7e9c156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7ea1fb1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 0x7f7e9c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7ea1fb1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0b0fc0 con 0x7f7e9c06add0 2023-10-06T05:15:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7ea1fb1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c156ca0 con 0x7f7e9c153a20 2023-10-06T05:15:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7ea1fb1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0b03b0 con 0x7f7e9c153650 2023-10-06T05:15:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7e9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9c06add0 0x7f7e9c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7e9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9c06add0 0x7f7e9c0afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53030/0 (socket says 172.21.15.45:53030) 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.726+0000 7f7e9b7fe700 1 -- 172.21.15.45:0/3229329089 learned_addr learned my addr 172.21.15.45:0/3229329089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9affd700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c153650 0x7f7e9c156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9bfff700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 0x7f7e9c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9b7fe700 1 -- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c153650 msgr2=0x7f7e9c156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9b7fe700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c153650 0x7f7e9c156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9b7fe700 1 -- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 msgr2=0x7f7e9c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9b7fe700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 0x7f7e9c0b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9b7fe700 1 -- 172.21.15.45:0/3229329089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e840094f0 con 0x7f7e9c06add0 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e9b7fe700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9c06add0 0x7f7e9c0afd30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c7e0 tx=0x7f7e8c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e937fe700 1 -- 172.21.15.45:0/3229329089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0105c0 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e937fe700 1 -- 172.21.15.45:0/3229329089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c010740 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0b0690 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7e937fe700 1 -- 172.21.15.45:0/3229329089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c010b00 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.727+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0b0c50 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.728+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c0cc0e0 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.729+0000 7f7e937fe700 1 -- 172.21.15.45:0/3229329089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e8c01b3b0 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.729+0000 7f7e937fe700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e7c066f30 0x7f7e7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.729+0000 7f7e937fe700 1 -- 172.21.15.45:0/3229329089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7e8c083f30 con 0x7f7e9c06add0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.729+0000 7f7e9affd700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e7c066f30 0x7f7e7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.730+0000 7f7e9affd700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e7c066f30 0x7f7e7c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e840099b0 tx=0x7f7e84017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:19.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.734+0000 7f7e937fe700 1 -- 172.21.15.45:0/3229329089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e8c050d50 con 0x7f7e9c06add0 2023-10-06T05:15:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.931+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e9c0b1100 con 0x7f7e7c066f30 2023-10-06T05:15:19.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.943+0000 7f7e937fe700 1 -- 172.21.15.45:0/3229329089 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367375 (secure 0 0 0) 0x7f7e8400b9d0 con 0x7f7e7c066f30 2023-10-06T05:15:19.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:19.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e7c066f30 msgr2=0x7f7e7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e7c066f30 0x7f7e7c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e840099b0 tx=0x7f7e84017000 comp rx=0 tx=0).stop 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9c06add0 msgr2=0x7f7e9c0afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9c06add0 0x7f7e9c0afd30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c7e0 tx=0x7f7e8c007df0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 shutdown_connections 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e7c066f30 0x7f7e7c0693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e9c153a20 0x7f7e9c0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9c06add0 0x7f7e9c0afd30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 --2- 172.21.15.45:0/3229329089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e9c153650 0x7f7e9c156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:19.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 >> 172.21.15.45:0/3229329089 conn(0x7f7e9c0cc5a0 msgr2=0x7f7e9c15a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:19.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 shutdown_connections 2023-10-06T05:15:19.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:19.947+0000 7f7ea1fb1700 1 -- 172.21.15.45:0/3229329089 wait complete. 2023-10-06T05:15:19.955 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:15:19.962 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-06T05:15:18.444499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003519"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.969110+0000","last_change":"2023-10-06T05:14:59.967201+0000","last_active":"2023-10-06T05:15:03.969110+0000","last_peered":"2023-10-06T05:15:03.969110+0000","last_clean":"2023-10-06T05:15:03.969110+0000","last_became_active":"2023-10-06T05:14:59.966818+0000","last_became_peered":"2023-10-06T05:14:59.966818+0000","last_unstale":"2023-10-06T05:15:03.969110+0000","last_undegraded":"2023-10-06T05:15:03.969110+0000","last_fullsized":"2023-10-06T05:15:03.969110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963221+0000","last_change":"2023-10-06T05:14:59.972677+0000","last_active":"2023-10-06T05:15:03.963221+0000","last_peered":"2023-10-06T05:15:03.963221+0000","last_clean":"2023-10-06T05:15:03.963221+0000","last_became_active":"2023-10-06T05:14:59.972435+0000","last_became_peered":"2023-10-06T05:14:59.972435+0000","last_unstale":"2023-10-06T05:15:03.963221+0000","last_undegraded":"2023-10-06T05:15:03.963221+0000","last_fullsized":"2023-10-06T05:15:03.963221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960554+0000","last_change":"2023-10-06T05:14:59.955617+0000","last_active":"2023-10-06T05:15:03.960554+0000","last_peered":"2023-10-06T05:15:03.960554+0000","last_clean":"2023-10-06T05:15:03.960554+0000","last_became_active":"2023-10-06T05:14:59.951585+0000","last_became_peered":"2023-10-06T05:14:59.951585+0000","last_unstale":"2023-10-06T05:15:03.960554+0000","last_undegraded":"2023-10-06T05:15:03.960554+0000","last_fullsized":"2023-10-06T05:15:03.960554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:56.835787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975703+0000","last_change":"2023-10-06T05:14:59.962591+0000","last_active":"2023-10-06T05:15:03.975703+0000","last_peered":"2023-10-06T05:15:03.975703+0000","last_clean":"2023-10-06T05:15:03.975703+0000","last_became_active":"2023-10-06T05:14:59.962219+0000","last_became_peered":"2023-10-06T05:14:59.962219+0000","last_unstale":"2023-10-06T05:15:03.975703+0000","last_undegraded":"2023-10-06T05:15:03.975703+0000","last_fullsized":"2023-10-06T05:15:03.975703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:11.373603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967261+0000","last_change":"2023-10-06T05:14:59.971703+0000","last_active":"2023-10-06T05:15:03.967261+0000","last_peered":"2023-10-06T05:15:03.967261+0000","last_clean":"2023-10-06T05:15:03.967261+0000","last_became_active":"2023-10-06T05:14:59.971083+0000","last_became_peered":"2023-10-06T05:14:59.971083+0000","last_unstale":"2023-10-06T05:15:03.967261+0000","last_undegraded":"2023-10-06T05:15:03.967261+0000","last_fullsized":"2023-10-06T05:15:03.967261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961235+0000","last_change":"2023-10-06T05:14:59.952376+0000","last_active":"2023-10-06T05:15:03.961235+0000","last_peered":"2023-10-06T05:15:03.961235+0000","last_clean":"2023-10-06T05:15:03.961235+0000","last_became_active":"2023-10-06T05:14:59.952101+0000","last_became_peered":"2023-10-06T05:14:59.952101+0000","last_unstale":"2023-10-06T05:15:03.961235+0000","last_undegraded":"2023-10-06T05:15:03.961235+0000","last_fullsized":"2023-10-06T05:15:03.961235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:46.640934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965050+0000","last_change":"2023-10-06T05:14:59.958503+0000","last_active":"2023-10-06T05:15:03.965050+0000","last_peered":"2023-10-06T05:15:03.965050+0000","last_clean":"2023-10-06T05:15:03.965050+0000","last_became_active":"2023-10-06T05:14:59.957962+0000","last_became_peered":"2023-10-06T05:14:59.957962+0000","last_unstale":"2023-10-06T05:15:03.965050+0000","last_undegraded":"2023-10-06T05:15:03.965050+0000","last_fullsized":"2023-10-06T05:15:03.965050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961531+0000","last_change":"2023-10-06T05:14:59.966704+0000","last_active":"2023-10-06T05:15:03.961531+0000","last_peered":"2023-10-06T05:15:03.961531+0000","last_clean":"2023-10-06T05:15:03.961531+0000","last_became_active":"2023-10-06T05:14:59.966378+0000","last_became_peered":"2023-10-06T05:14:59.966378+0000","last_unstale":"2023-10-06T05:15:03.961531+0000","last_undegraded":"2023-10-06T05:15:03.961531+0000","last_fullsized":"2023-10-06T05:15:03.961531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970103+0000","last_change":"2023-10-06T05:14:59.967328+0000","last_active":"2023-10-06T05:15:03.970103+0000","last_peered":"2023-10-06T05:15:03.970103+0000","last_clean":"2023-10-06T05:15:03.970103+0000","last_became_active":"2023-10-06T05:14:59.966868+0000","last_became_peered":"2023-10-06T05:14:59.966868+0000","last_unstale":"2023-10-06T05:15:03.970103+0000","last_undegraded":"2023-10-06T05:15:03.970103+0000","last_fullsized":"2023-10-06T05:15:03.970103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963384+0000","last_change":"2023-10-06T05:14:59.966488+0000","last_active":"2023-10-06T05:15:03.963384+0000","last_peered":"2023-10-06T05:15:03.963384+0000","last_clean":"2023-10-06T05:15:03.963384+0000","last_became_active":"2023-10-06T05:14:59.966153+0000","last_became_peered":"2023-10-06T05:14:59.966153+0000","last_unstale":"2023-10-06T05:15:03.963384+0000","last_undegraded":"2023-10-06T05:15:03.963384+0000","last_fullsized":"2023-10-06T05:15:03.963384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960401+0000","last_change":"2023-10-06T05:14:59.956939+0000","last_active":"2023-10-06T05:15:03.960401+0000","last_peered":"2023-10-06T05:15:03.960401+0000","last_clean":"2023-10-06T05:15:03.960401+0000","last_became_active":"2023-10-06T05:14:59.956467+0000","last_became_peered":"2023-10-06T05:14:59.956467+0000","last_unstale":"2023-10-06T05:15:03.960401+0000","last_undegraded":"2023-10-06T05:15:03.960401+0000","last_fullsized":"2023-10-06T05:15:03.960401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968295+0000","last_change":"2023-10-06T05:14:59.965955+0000","last_active":"2023-10-06T05:15:03.968295+0000","last_peered":"2023-10-06T05:15:03.968295+0000","last_clean":"2023-10-06T05:15:03.968295+0000","last_became_active":"2023-10-06T05:14:59.965676+0000","last_became_peered":"2023-10-06T05:14:59.965676+0000","last_unstale":"2023-10-06T05:15:03.968295+0000","last_undegraded":"2023-10-06T05:15:03.968295+0000","last_fullsized":"2023-10-06T05:15:03.968295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:39.098260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968290+0000","last_change":"2023-10-06T05:14:59.971318+0000","last_active":"2023-10-06T05:15:03.968290+0000","last_peered":"2023-10-06T05:15:03.968290+0000","last_clean":"2023-10-06T05:15:03.968290+0000","last_became_active":"2023-10-06T05:14:59.971044+0000","last_became_peered":"2023-10-06T05:14:59.971044+0000","last_unstale":"2023-10-06T05:15:03.968290+0000","last_undegraded":"2023-10-06T05:15:03.968290+0000","last_fullsized":"2023-10-06T05:15:03.968290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:17.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962264+0000","last_change":"2023-10-06T05:14:59.962752+0000","last_active":"2023-10-06T05:15:03.962264+0000","last_peered":"2023-10-06T05:15:03.962264+0000","last_clean":"2023-10-06T05:15:03.962264+0000","last_became_active":"2023-10-06T05:14:59.962473+0000","last_became_peered":"2023-10-06T05:14:59.962473+0000","last_unstale":"2023-10-06T05:15:03.962264+0000","last_undegraded":"2023-10-06T05:15:03.962264+0000","last_fullsized":"2023-10-06T05:15:03.962264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964859+0000","last_change":"2023-10-06T05:14:59.958686+0000","last_active":"2023-10-06T05:15:03.964859+0000","last_peered":"2023-10-06T05:15:03.964859+0000","last_clean":"2023-10-06T05:15:03.964859+0000","last_became_active":"2023-10-06T05:14:59.958258+0000","last_became_peered":"2023-10-06T05:14:59.958258+0000","last_unstale":"2023-10-06T05:15:03.964859+0000","last_undegraded":"2023-10-06T05:15:03.964859+0000","last_fullsized":"2023-10-06T05:15:03.964859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960536+0000","last_change":"2023-10-06T05:14:59.956754+0000","last_active":"2023-10-06T05:15:03.960536+0000","last_peered":"2023-10-06T05:15:03.960536+0000","last_clean":"2023-10-06T05:15:03.960536+0000","last_became_active":"2023-10-06T05:14:59.951445+0000","last_became_peered":"2023-10-06T05:14:59.951445+0000","last_unstale":"2023-10-06T05:15:03.960536+0000","last_undegraded":"2023-10-06T05:15:03.960536+0000","last_fullsized":"2023-10-06T05:15:03.960536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:43.881359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970204+0000","last_change":"2023-10-06T05:14:59.967155+0000","last_active":"2023-10-06T05:15:03.970204+0000","last_peered":"2023-10-06T05:15:03.970204+0000","last_clean":"2023-10-06T05:15:03.970204+0000","last_became_active":"2023-10-06T05:14:59.966533+0000","last_became_peered":"2023-10-06T05:14:59.966533+0000","last_unstale":"2023-10-06T05:15:03.970204+0000","last_undegraded":"2023-10-06T05:15:03.970204+0000","last_fullsized":"2023-10-06T05:15:03.970204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960993+0000","last_change":"2023-10-06T05:14:59.949784+0000","last_active":"2023-10-06T05:15:03.960993+0000","last_peered":"2023-10-06T05:15:03.960993+0000","last_clean":"2023-10-06T05:15:03.960993+0000","last_became_active":"2023-10-06T05:14:59.949540+0000","last_became_peered":"2023-10-06T05:14:59.949540+0000","last_unstale":"2023-10-06T05:15:03.960993+0000","last_undegraded":"2023-10-06T05:15:03.960993+0000","last_fullsized":"2023-10-06T05:15:03.960993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:55.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967686+0000","last_change":"2023-10-06T05:14:59.954874+0000","last_active":"2023-10-06T05:15:03.967686+0000","last_peered":"2023-10-06T05:15:03.967686+0000","last_clean":"2023-10-06T05:15:03.967686+0000","last_became_active":"2023-10-06T05:14:59.954596+0000","last_became_peered":"2023-10-06T05:14:59.954596+0000","last_unstale":"2023-10-06T05:15:03.967686+0000","last_undegraded":"2023-10-06T05:15:03.967686+0000","last_fullsized":"2023-10-06T05:15:03.967686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:12.617791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959901+0000","last_change":"2023-10-06T05:14:59.964700+0000","last_active":"2023-10-06T05:15:03.959901+0000","last_peered":"2023-10-06T05:15:03.959901+0000","last_clean":"2023-10-06T05:15:03.959901+0000","last_became_active":"2023-10-06T05:14:59.964122+0000","last_became_peered":"2023-10-06T05:14:59.964122+0000","last_unstale":"2023-10-06T05:15:03.959901+0000","last_undegraded":"2023-10-06T05:15:03.959901+0000","last_fullsized":"2023-10-06T05:15:03.959901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.174130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965374+0000","last_change":"2023-10-06T05:14:59.971088+0000","last_active":"2023-10-06T05:15:03.965374+0000","last_peered":"2023-10-06T05:15:03.965374+0000","last_clean":"2023-10-06T05:15:03.965374+0000","last_became_active":"2023-10-06T05:14:59.970833+0000","last_became_peered":"2023-10-06T05:14:59.970833+0000","last_unstale":"2023-10-06T05:15:03.965374+0000","last_undegraded":"2023-10-06T05:15:03.965374+0000","last_fullsized":"2023-10-06T05:15:03.965374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963287+0000","last_change":"2023-10-06T05:14:59.959301+0000","last_active":"2023-10-06T05:15:03.963287+0000","last_peered":"2023-10-06T05:15:03.963287+0000","last_clean":"2023-10-06T05:15:03.963287+0000","last_became_active":"2023-10-06T05:14:59.958848+0000","last_became_peered":"2023-10-06T05:14:59.958848+0000","last_unstale":"2023-10-06T05:15:03.963287+0000","last_undegraded":"2023-10-06T05:15:03.963287+0000","last_fullsized":"2023-10-06T05:15:03.963287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963428+0000","last_change":"2023-10-06T05:14:59.964849+0000","last_active":"2023-10-06T05:15:03.963428+0000","last_peered":"2023-10-06T05:15:03.963428+0000","last_clean":"2023-10-06T05:15:03.963428+0000","last_became_active":"2023-10-06T05:14:59.964525+0000","last_became_peered":"2023-10-06T05:14:59.964525+0000","last_unstale":"2023-10-06T05:15:03.963428+0000","last_undegraded":"2023-10-06T05:15:03.963428+0000","last_fullsized":"2023-10-06T05:15:03.963428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964605+0000","last_change":"2023-10-06T05:14:59.968524+0000","last_active":"2023-10-06T05:15:03.964605+0000","last_peered":"2023-10-06T05:15:03.964605+0000","last_clean":"2023-10-06T05:15:03.964605+0000","last_became_active":"2023-10-06T05:14:59.968292+0000","last_became_peered":"2023-10-06T05:14:59.968292+0000","last_unstale":"2023-10-06T05:15:03.964605+0000","last_undegraded":"2023-10-06T05:15:03.964605+0000","last_fullsized":"2023-10-06T05:15:03.964605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979056+0000","last_change":"2023-10-06T05:14:59.957743+0000","last_active":"2023-10-06T05:15:03.979056+0000","last_peered":"2023-10-06T05:15:03.979056+0000","last_clean":"2023-10-06T05:15:03.979056+0000","last_became_active":"2023-10-06T05:14:59.957472+0000","last_became_peered":"2023-10-06T05:14:59.957472+0000","last_unstale":"2023-10-06T05:15:03.979056+0000","last_undegraded":"2023-10-06T05:15:03.979056+0000","last_fullsized":"2023-10-06T05:15:03.979056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977658+0000","last_change":"2023-10-06T05:14:59.953807+0000","last_active":"2023-10-06T05:15:03.977658+0000","last_peered":"2023-10-06T05:15:03.977658+0000","last_clean":"2023-10-06T05:15:03.977658+0000","last_became_active":"2023-10-06T05:14:59.953532+0000","last_became_peered":"2023-10-06T05:14:59.953532+0000","last_unstale":"2023-10-06T05:15:03.977658+0000","last_undegraded":"2023-10-06T05:15:03.977658+0000","last_fullsized":"2023-10-06T05:15:03.977658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:39.399842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962861+0000","last_change":"2023-10-06T05:14:59.968101+0000","last_active":"2023-10-06T05:15:03.962861+0000","last_peered":"2023-10-06T05:15:03.962861+0000","last_clean":"2023-10-06T05:15:03.962861+0000","last_became_active":"2023-10-06T05:14:59.967317+0000","last_became_peered":"2023-10-06T05:14:59.967317+0000","last_unstale":"2023-10-06T05:15:03.962861+0000","last_undegraded":"2023-10-06T05:15:03.962861+0000","last_fullsized":"2023-10-06T05:15:03.962861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987821+0000","last_change":"2023-10-06T05:15:03.983811+0000","last_active":"2023-10-06T05:15:03.987821+0000","last_peered":"2023-10-06T05:15:03.987821+0000","last_clean":"2023-10-06T05:15:03.987821+0000","last_became_active":"2023-10-06T05:15:03.983468+0000","last_became_peered":"2023-10-06T05:15:03.983468+0000","last_unstale":"2023-10-06T05:15:03.987821+0000","last_undegraded":"2023-10-06T05:15:03.987821+0000","last_fullsized":"2023-10-06T05:15:03.987821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:02.315455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961143+0000","last_change":"2023-10-06T05:14:58.958395+0000","last_active":"2023-10-06T05:15:03.961143+0000","last_peered":"2023-10-06T05:15:03.961143+0000","last_clean":"2023-10-06T05:15:03.961143+0000","last_became_active":"2023-10-06T05:14:58.957979+0000","last_became_peered":"2023-10-06T05:14:58.957979+0000","last_unstale":"2023-10-06T05:15:03.961143+0000","last_undegraded":"2023-10-06T05:15:03.961143+0000","last_fullsized":"2023-10-06T05:15:03.961143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965650+0000","last_change":"2023-10-06T05:14:59.960654+0000","last_active":"2023-10-06T05:15:03.965650+0000","last_peered":"2023-10-06T05:15:03.965650+0000","last_clean":"2023-10-06T05:15:03.965650+0000","last_became_active":"2023-10-06T05:14:59.960359+0000","last_became_peered":"2023-10-06T05:14:59.960359+0000","last_unstale":"2023-10-06T05:15:03.965650+0000","last_undegraded":"2023-10-06T05:15:03.965650+0000","last_fullsized":"2023-10-06T05:15:03.965650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987166+0000","last_change":"2023-10-06T05:15:03.980797+0000","last_active":"2023-10-06T05:15:03.987166+0000","last_peered":"2023-10-06T05:15:03.987166+0000","last_clean":"2023-10-06T05:15:03.987166+0000","last_became_active":"2023-10-06T05:15:03.980441+0000","last_became_peered":"2023-10-06T05:15:03.980441+0000","last_unstale":"2023-10-06T05:15:03.987166+0000","last_undegraded":"2023-10-06T05:15:03.987166+0000","last_fullsized":"2023-10-06T05:15:03.987166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:59.153783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972385+0000","last_change":"2023-10-06T05:14:58.967470+0000","last_active":"2023-10-06T05:15:03.972385+0000","last_peered":"2023-10-06T05:15:03.972385+0000","last_clean":"2023-10-06T05:15:03.972385+0000","last_became_active":"2023-10-06T05:14:58.967143+0000","last_became_peered":"2023-10-06T05:14:58.967143+0000","last_unstale":"2023-10-06T05:15:03.972385+0000","last_undegraded":"2023-10-06T05:15:03.972385+0000","last_fullsized":"2023-10-06T05:15:03.972385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970567+0000","last_change":"2023-10-06T05:14:59.961217+0000","last_active":"2023-10-06T05:15:03.970567+0000","last_peered":"2023-10-06T05:15:03.970567+0000","last_clean":"2023-10-06T05:15:03.970567+0000","last_became_active":"2023-10-06T05:14:59.960909+0000","last_became_peered":"2023-10-06T05:14:59.960909+0000","last_unstale":"2023-10-06T05:15:03.970567+0000","last_undegraded":"2023-10-06T05:15:03.970567+0000","last_fullsized":"2023-10-06T05:15:03.970567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995384+0000","last_change":"2023-10-06T05:15:03.993702+0000","last_active":"2023-10-06T05:15:03.995384+0000","last_peered":"2023-10-06T05:15:03.995384+0000","last_clean":"2023-10-06T05:15:03.995384+0000","last_became_active":"2023-10-06T05:15:03.993170+0000","last_became_peered":"2023-10-06T05:15:03.993170+0000","last_unstale":"2023-10-06T05:15:03.995384+0000","last_undegraded":"2023-10-06T05:15:03.995384+0000","last_fullsized":"2023-10-06T05:15:03.995384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964634+0000","last_change":"2023-10-06T05:14:58.964287+0000","last_active":"2023-10-06T05:15:03.964634+0000","last_peered":"2023-10-06T05:15:03.964634+0000","last_clean":"2023-10-06T05:15:03.964634+0000","last_became_active":"2023-10-06T05:14:58.963981+0000","last_became_peered":"2023-10-06T05:14:58.963981+0000","last_unstale":"2023-10-06T05:15:03.964634+0000","last_undegraded":"2023-10-06T05:15:03.964634+0000","last_fullsized":"2023-10-06T05:15:03.964634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975859+0000","last_change":"2023-10-06T05:14:59.964080+0000","last_active":"2023-10-06T05:15:03.975859+0000","last_peered":"2023-10-06T05:15:03.975859+0000","last_clean":"2023-10-06T05:15:03.975859+0000","last_became_active":"2023-10-06T05:14:59.963849+0000","last_became_peered":"2023-10-06T05:14:59.963849+0000","last_unstale":"2023-10-06T05:15:03.975859+0000","last_undegraded":"2023-10-06T05:15:03.975859+0000","last_fullsized":"2023-10-06T05:15:03.975859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:13.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.988935+0000","last_change":"2023-10-06T05:15:03.985094+0000","last_active":"2023-10-06T05:15:03.988935+0000","last_peered":"2023-10-06T05:15:03.988935+0000","last_clean":"2023-10-06T05:15:03.988935+0000","last_became_active":"2023-10-06T05:15:03.984516+0000","last_became_peered":"2023-10-06T05:15:03.984516+0000","last_unstale":"2023-10-06T05:15:03.988935+0000","last_undegraded":"2023-10-06T05:15:03.988935+0000","last_fullsized":"2023-10-06T05:15:03.988935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:09.008217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970313+0000","last_change":"2023-10-06T05:14:58.970639+0000","last_active":"2023-10-06T05:15:03.970313+0000","last_peered":"2023-10-06T05:15:03.970313+0000","last_clean":"2023-10-06T05:15:03.970313+0000","last_became_active":"2023-10-06T05:14:58.970183+0000","last_became_peered":"2023-10-06T05:14:58.970183+0000","last_unstale":"2023-10-06T05:15:03.970313+0000","last_undegraded":"2023-10-06T05:15:03.970313+0000","last_fullsized":"2023-10-06T05:15:03.970313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964542+0000","last_change":"2023-10-06T05:14:59.962878+0000","last_active":"2023-10-06T05:15:03.964542+0000","last_peered":"2023-10-06T05:15:03.964542+0000","last_clean":"2023-10-06T05:15:03.964542+0000","last_became_active":"2023-10-06T05:14:59.962557+0000","last_became_peered":"2023-10-06T05:14:59.962557+0000","last_unstale":"2023-10-06T05:15:03.964542+0000","last_undegraded":"2023-10-06T05:15:03.964542+0000","last_fullsized":"2023-10-06T05:15:03.964542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.984338+0000","last_change":"2023-10-06T05:15:03.979469+0000","last_active":"2023-10-06T05:15:03.984338+0000","last_peered":"2023-10-06T05:15:03.984338+0000","last_clean":"2023-10-06T05:15:03.984338+0000","last_became_active":"2023-10-06T05:15:03.979016+0000","last_became_peered":"2023-10-06T05:15:03.979016+0000","last_unstale":"2023-10-06T05:15:03.984338+0000","last_undegraded":"2023-10-06T05:15:03.984338+0000","last_fullsized":"2023-10-06T05:15:03.984338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963303+0000","last_change":"2023-10-06T05:14:58.953556+0000","last_active":"2023-10-06T05:15:03.963303+0000","last_peered":"2023-10-06T05:15:03.963303+0000","last_clean":"2023-10-06T05:15:03.963303+0000","last_became_active":"2023-10-06T05:14:58.953280+0000","last_became_peered":"2023-10-06T05:14:58.953280+0000","last_unstale":"2023-10-06T05:15:03.963303+0000","last_undegraded":"2023-10-06T05:15:03.963303+0000","last_fullsized":"2023-10-06T05:15:03.963303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963541+0000","last_change":"2023-10-06T05:14:59.969909+0000","last_active":"2023-10-06T05:15:03.963541+0000","last_peered":"2023-10-06T05:15:03.963541+0000","last_clean":"2023-10-06T05:15:03.963541+0000","last_became_active":"2023-10-06T05:14:59.967612+0000","last_became_peered":"2023-10-06T05:14:59.967612+0000","last_unstale":"2023-10-06T05:15:03.963541+0000","last_undegraded":"2023-10-06T05:15:03.963541+0000","last_fullsized":"2023-10-06T05:15:03.963541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994156+0000","last_change":"2023-10-06T05:15:03.984187+0000","last_active":"2023-10-06T05:15:03.994156+0000","last_peered":"2023-10-06T05:15:03.994156+0000","last_clean":"2023-10-06T05:15:03.994156+0000","last_became_active":"2023-10-06T05:15:03.983778+0000","last_became_peered":"2023-10-06T05:15:03.983778+0000","last_unstale":"2023-10-06T05:15:03.994156+0000","last_undegraded":"2023-10-06T05:15:03.994156+0000","last_fullsized":"2023-10-06T05:15:03.994156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:24.403669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967440+0000","last_change":"2023-10-06T05:14:58.971420+0000","last_active":"2023-10-06T05:15:03.967440+0000","last_peered":"2023-10-06T05:15:03.967440+0000","last_clean":"2023-10-06T05:15:03.967440+0000","last_became_active":"2023-10-06T05:14:58.971181+0000","last_became_peered":"2023-10-06T05:14:58.971181+0000","last_unstale":"2023-10-06T05:15:03.967440+0000","last_undegraded":"2023-10-06T05:15:03.967440+0000","last_fullsized":"2023-10-06T05:15:03.967440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973790+0000","last_change":"2023-10-06T05:14:59.966072+0000","last_active":"2023-10-06T05:15:03.973790+0000","last_peered":"2023-10-06T05:15:03.973790+0000","last_clean":"2023-10-06T05:15:03.973790+0000","last_became_active":"2023-10-06T05:14:59.965368+0000","last_became_peered":"2023-10-06T05:14:59.965368+0000","last_unstale":"2023-10-06T05:15:03.973790+0000","last_undegraded":"2023-10-06T05:15:03.973790+0000","last_fullsized":"2023-10-06T05:15:03.973790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990856+0000","last_change":"2023-10-06T05:15:03.989524+0000","last_active":"2023-10-06T05:15:03.990856+0000","last_peered":"2023-10-06T05:15:03.990856+0000","last_clean":"2023-10-06T05:15:03.990856+0000","last_became_active":"2023-10-06T05:15:03.982518+0000","last_became_peered":"2023-10-06T05:15:03.982518+0000","last_unstale":"2023-10-06T05:15:03.990856+0000","last_undegraded":"2023-10-06T05:15:03.990856+0000","last_fullsized":"2023-10-06T05:15:03.990856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T05:15:10.388156+0000","last_change":"2023-10-06T05:14:58.956195+0000","last_active":"2023-10-06T05:15:10.388156+0000","last_peered":"2023-10-06T05:15:10.388156+0000","last_clean":"2023-10-06T05:15:10.388156+0000","last_became_active":"2023-10-06T05:14:58.955906+0000","last_became_peered":"2023-10-06T05:14:58.955906+0000","last_unstale":"2023-10-06T05:15:10.388156+0000","last_undegraded":"2023-10-06T05:15:10.388156+0000","last_fullsized":"2023-10-06T05:15:10.388156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.952535+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.952313+0000","last_became_peered":"2023-10-06T05:14:59.952313+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987354+0000","last_change":"2023-10-06T05:15:03.982183+0000","last_active":"2023-10-06T05:15:03.987354+0000","last_peered":"2023-10-06T05:15:03.987354+0000","last_clean":"2023-10-06T05:15:03.987354+0000","last_became_active":"2023-10-06T05:15:03.981849+0000","last_became_peered":"2023-10-06T05:15:03.981849+0000","last_unstale":"2023-10-06T05:15:03.987354+0000","last_undegraded":"2023-10-06T05:15:03.987354+0000","last_fullsized":"2023-10-06T05:15:03.987354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965015+0000","last_change":"2023-10-06T05:14:58.959548+0000","last_active":"2023-10-06T05:15:03.965015+0000","last_peered":"2023-10-06T05:15:03.965015+0000","last_clean":"2023-10-06T05:15:03.965015+0000","last_became_active":"2023-10-06T05:14:58.959270+0000","last_became_peered":"2023-10-06T05:14:58.959270+0000","last_unstale":"2023-10-06T05:15:03.965015+0000","last_undegraded":"2023-10-06T05:15:03.965015+0000","last_fullsized":"2023-10-06T05:15:03.965015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:02.018618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968411+0000","last_change":"2023-10-06T05:14:59.973960+0000","last_active":"2023-10-06T05:15:03.968411+0000","last_peered":"2023-10-06T05:15:03.968411+0000","last_clean":"2023-10-06T05:15:03.968411+0000","last_became_active":"2023-10-06T05:14:59.973622+0000","last_became_peered":"2023-10-06T05:14:59.973622+0000","last_unstale":"2023-10-06T05:15:03.968411+0000","last_undegraded":"2023-10-06T05:15:03.968411+0000","last_fullsized":"2023-10-06T05:15:03.968411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994166+0000","last_change":"2023-10-06T05:15:03.985900+0000","last_active":"2023-10-06T05:15:03.994166+0000","last_peered":"2023-10-06T05:15:03.994166+0000","last_clean":"2023-10-06T05:15:03.994166+0000","last_became_active":"2023-10-06T05:15:03.985484+0000","last_became_peered":"2023-10-06T05:15:03.985484+0000","last_unstale":"2023-10-06T05:15:03.994166+0000","last_undegraded":"2023-10-06T05:15:03.994166+0000","last_fullsized":"2023-10-06T05:15:03.994166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:50.069708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963167+0000","last_change":"2023-10-06T05:14:58.968185+0000","last_active":"2023-10-06T05:15:03.963167+0000","last_peered":"2023-10-06T05:15:03.963167+0000","last_clean":"2023-10-06T05:15:03.963167+0000","last_became_active":"2023-10-06T05:14:58.967895+0000","last_became_peered":"2023-10-06T05:14:58.967895+0000","last_unstale":"2023-10-06T05:15:03.963167+0000","last_undegraded":"2023-10-06T05:15:03.963167+0000","last_fullsized":"2023-10-06T05:15:03.963167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962815+0000","last_change":"2023-10-06T05:14:59.967384+0000","last_active":"2023-10-06T05:15:03.962815+0000","last_peered":"2023-10-06T05:15:03.962815+0000","last_clean":"2023-10-06T05:15:03.962815+0000","last_became_active":"2023-10-06T05:14:59.967131+0000","last_became_peered":"2023-10-06T05:14:59.967131+0000","last_unstale":"2023-10-06T05:15:03.962815+0000","last_undegraded":"2023-10-06T05:15:03.962815+0000","last_fullsized":"2023-10-06T05:15:03.962815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994086+0000","last_change":"2023-10-06T05:15:03.985192+0000","last_active":"2023-10-06T05:15:03.994086+0000","last_peered":"2023-10-06T05:15:03.994086+0000","last_clean":"2023-10-06T05:15:03.994086+0000","last_became_active":"2023-10-06T05:15:03.984600+0000","last_became_peered":"2023-10-06T05:15:03.984600+0000","last_unstale":"2023-10-06T05:15:03.994086+0000","last_undegraded":"2023-10-06T05:15:03.994086+0000","last_fullsized":"2023-10-06T05:15:03.994086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964960+0000","last_change":"2023-10-06T05:14:58.959794+0000","last_active":"2023-10-06T05:15:03.964960+0000","last_peered":"2023-10-06T05:15:03.964960+0000","last_clean":"2023-10-06T05:15:03.964960+0000","last_became_active":"2023-10-06T05:14:58.959493+0000","last_became_peered":"2023-10-06T05:14:58.959493+0000","last_unstale":"2023-10-06T05:15:03.964960+0000","last_undegraded":"2023-10-06T05:15:03.964960+0000","last_fullsized":"2023-10-06T05:15:03.964960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:10.366367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970679+0000","last_change":"2023-10-06T05:14:59.964236+0000","last_active":"2023-10-06T05:15:03.970679+0000","last_peered":"2023-10-06T05:15:03.970679+0000","last_clean":"2023-10-06T05:15:03.970679+0000","last_became_active":"2023-10-06T05:14:59.963902+0000","last_became_peered":"2023-10-06T05:14:59.963902+0000","last_unstale":"2023-10-06T05:15:03.970679+0000","last_undegraded":"2023-10-06T05:15:03.970679+0000","last_fullsized":"2023-10-06T05:15:03.970679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987391+0000","last_change":"2023-10-06T05:15:03.982595+0000","last_active":"2023-10-06T05:15:03.987391+0000","last_peered":"2023-10-06T05:15:03.987391+0000","last_clean":"2023-10-06T05:15:03.987391+0000","last_became_active":"2023-10-06T05:15:03.982200+0000","last_became_peered":"2023-10-06T05:15:03.982200+0000","last_unstale":"2023-10-06T05:15:03.987391+0000","last_undegraded":"2023-10-06T05:15:03.987391+0000","last_fullsized":"2023-10-06T05:15:03.987391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:20.022766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974038+0000","last_change":"2023-10-06T05:14:58.961380+0000","last_active":"2023-10-06T05:15:03.974038+0000","last_peered":"2023-10-06T05:15:03.974038+0000","last_clean":"2023-10-06T05:15:03.974038+0000","last_became_active":"2023-10-06T05:14:58.961038+0000","last_became_peered":"2023-10-06T05:14:58.961038+0000","last_unstale":"2023-10-06T05:15:03.974038+0000","last_undegraded":"2023-10-06T05:15:03.974038+0000","last_fullsized":"2023-10-06T05:15:03.974038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964625+0000","last_change":"2023-10-06T05:14:59.967979+0000","last_active":"2023-10-06T05:15:03.964625+0000","last_peered":"2023-10-06T05:15:03.964625+0000","last_clean":"2023-10-06T05:15:03.964625+0000","last_became_active":"2023-10-06T05:14:59.964930+0000","last_became_peered":"2023-10-06T05:14:59.964930+0000","last_unstale":"2023-10-06T05:15:03.964625+0000","last_undegraded":"2023-10-06T05:15:03.964625+0000","last_fullsized":"2023-10-06T05:15:03.964625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.981489+0000","last_change":"2023-10-06T05:15:03.978144+0000","last_active":"2023-10-06T05:15:03.981489+0000","last_peered":"2023-10-06T05:15:03.981489+0000","last_clean":"2023-10-06T05:15:03.981489+0000","last_became_active":"2023-10-06T05:15:03.977784+0000","last_became_peered":"2023-10-06T05:15:03.977784+0000","last_unstale":"2023-10-06T05:15:03.981489+0000","last_undegraded":"2023-10-06T05:15:03.981489+0000","last_fullsized":"2023-10-06T05:15:03.981489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967621+0000","last_change":"2023-10-06T05:14:58.959617+0000","last_active":"2023-10-06T05:15:03.967621+0000","last_peered":"2023-10-06T05:15:03.967621+0000","last_clean":"2023-10-06T05:15:03.967621+0000","last_became_active":"2023-10-06T05:14:58.959350+0000","last_became_peered":"2023-10-06T05:14:58.959350+0000","last_unstale":"2023-10-06T05:15:03.967621+0000","last_undegraded":"2023-10-06T05:15:03.967621+0000","last_fullsized":"2023-10-06T05:15:03.967621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:12.695218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961420+0000","last_change":"2023-10-06T05:14:59.967832+0000","last_active":"2023-10-06T05:15:03.961420+0000","last_peered":"2023-10-06T05:15:03.961420+0000","last_clean":"2023-10-06T05:15:03.961420+0000","last_became_active":"2023-10-06T05:14:59.967192+0000","last_became_peered":"2023-10-06T05:14:59.967192+0000","last_unstale":"2023-10-06T05:15:03.961420+0000","last_undegraded":"2023-10-06T05:15:03.961420+0000","last_fullsized":"2023-10-06T05:15:03.961420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995012+0000","last_change":"2023-10-06T05:15:03.993765+0000","last_active":"2023-10-06T05:15:03.995012+0000","last_peered":"2023-10-06T05:15:03.995012+0000","last_clean":"2023-10-06T05:15:03.995012+0000","last_became_active":"2023-10-06T05:15:03.993371+0000","last_became_peered":"2023-10-06T05:15:03.993371+0000","last_unstale":"2023-10-06T05:15:03.995012+0000","last_undegraded":"2023-10-06T05:15:03.995012+0000","last_fullsized":"2023-10-06T05:15:03.995012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:00.861115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967256+0000","last_change":"2023-10-06T05:14:58.962483+0000","last_active":"2023-10-06T05:15:03.967256+0000","last_peered":"2023-10-06T05:15:03.967256+0000","last_clean":"2023-10-06T05:15:03.967256+0000","last_became_active":"2023-10-06T05:14:58.962203+0000","last_became_peered":"2023-10-06T05:14:58.962203+0000","last_unstale":"2023-10-06T05:15:03.967256+0000","last_undegraded":"2023-10-06T05:15:03.967256+0000","last_fullsized":"2023-10-06T05:15:03.967256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:34.275033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960934+0000","last_change":"2023-10-06T05:14:59.956925+0000","last_active":"2023-10-06T05:15:03.960934+0000","last_peered":"2023-10-06T05:15:03.960934+0000","last_clean":"2023-10-06T05:15:03.960934+0000","last_became_active":"2023-10-06T05:14:59.956653+0000","last_became_peered":"2023-10-06T05:14:59.956653+0000","last_unstale":"2023-10-06T05:15:03.960934+0000","last_undegraded":"2023-10-06T05:15:03.960934+0000","last_fullsized":"2023-10-06T05:15:03.960934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985561+0000","last_change":"2023-10-06T05:15:03.983559+0000","last_active":"2023-10-06T05:15:03.985561+0000","last_peered":"2023-10-06T05:15:03.985561+0000","last_clean":"2023-10-06T05:15:03.985561+0000","last_became_active":"2023-10-06T05:15:03.983123+0000","last_became_peered":"2023-10-06T05:15:03.983123+0000","last_unstale":"2023-10-06T05:15:03.985561+0000","last_undegraded":"2023-10-06T05:15:03.985561+0000","last_fullsized":"2023-10-06T05:15:03.985561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:50.666262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967619+0000","last_change":"2023-10-06T05:14:58.956636+0000","last_active":"2023-10-06T05:15:03.967619+0000","last_peered":"2023-10-06T05:15:03.967619+0000","last_clean":"2023-10-06T05:15:03.967619+0000","last_became_active":"2023-10-06T05:14:58.956436+0000","last_became_peered":"2023-10-06T05:14:58.956436+0000","last_unstale":"2023-10-06T05:15:03.967619+0000","last_undegraded":"2023-10-06T05:15:03.967619+0000","last_fullsized":"2023-10-06T05:15:03.967619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:45.319345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975492+0000","last_change":"2023-10-06T05:14:59.962010+0000","last_active":"2023-10-06T05:15:03.975492+0000","last_peered":"2023-10-06T05:15:03.975492+0000","last_clean":"2023-10-06T05:15:03.975492+0000","last_became_active":"2023-10-06T05:14:59.961837+0000","last_became_peered":"2023-10-06T05:14:59.961837+0000","last_unstale":"2023-10-06T05:15:03.975492+0000","last_undegraded":"2023-10-06T05:15:03.975492+0000","last_fullsized":"2023-10-06T05:15:03.975492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:46.326581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985590+0000","last_change":"2023-10-06T05:15:03.983998+0000","last_active":"2023-10-06T05:15:03.985590+0000","last_peered":"2023-10-06T05:15:03.985590+0000","last_clean":"2023-10-06T05:15:03.985590+0000","last_became_active":"2023-10-06T05:15:03.983598+0000","last_became_peered":"2023-10-06T05:15:03.983598+0000","last_unstale":"2023-10-06T05:15:03.985590+0000","last_undegraded":"2023-10-06T05:15:03.985590+0000","last_fullsized":"2023-10-06T05:15:03.985590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:01.199458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961576+0000","last_change":"2023-10-06T05:14:58.969438+0000","last_active":"2023-10-06T05:15:03.961576+0000","last_peered":"2023-10-06T05:15:03.961576+0000","last_clean":"2023-10-06T05:15:03.961576+0000","last_became_active":"2023-10-06T05:14:58.969214+0000","last_became_peered":"2023-10-06T05:14:58.969214+0000","last_unstale":"2023-10-06T05:15:03.961576+0000","last_undegraded":"2023-10-06T05:15:03.961576+0000","last_fullsized":"2023-10-06T05:15:03.961576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966944+0000","last_change":"2023-10-06T05:14:59.958303+0000","last_active":"2023-10-06T05:15:03.966944+0000","last_peered":"2023-10-06T05:15:03.966944+0000","last_clean":"2023-10-06T05:15:03.966944+0000","last_became_active":"2023-10-06T05:14:59.957997+0000","last_became_peered":"2023-10-06T05:14:59.957997+0000","last_unstale":"2023-10-06T05:15:03.966944+0000","last_undegraded":"2023-10-06T05:15:03.966944+0000","last_fullsized":"2023-10-06T05:15:03.966944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.982560+0000","last_change":"2023-10-06T05:15:03.976634+0000","last_active":"2023-10-06T05:15:03.982560+0000","last_peered":"2023-10-06T05:15:03.982560+0000","last_clean":"2023-10-06T05:15:03.982560+0000","last_became_active":"2023-10-06T05:15:03.976257+0000","last_became_peered":"2023-10-06T05:15:03.976257+0000","last_unstale":"2023-10-06T05:15:03.982560+0000","last_undegraded":"2023-10-06T05:15:03.982560+0000","last_fullsized":"2023-10-06T05:15:03.982560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:47.906687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964042+0000","last_change":"2023-10-06T05:14:58.969408+0000","last_active":"2023-10-06T05:15:03.964042+0000","last_peered":"2023-10-06T05:15:03.964042+0000","last_clean":"2023-10-06T05:15:03.964042+0000","last_became_active":"2023-10-06T05:14:58.969070+0000","last_became_peered":"2023-10-06T05:14:58.969070+0000","last_unstale":"2023-10-06T05:15:03.964042+0000","last_undegraded":"2023-10-06T05:15:03.964042+0000","last_fullsized":"2023-10-06T05:15:03.964042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959678+0000","last_change":"2023-10-06T05:14:59.969503+0000","last_active":"2023-10-06T05:15:03.959678+0000","last_peered":"2023-10-06T05:15:03.959678+0000","last_clean":"2023-10-06T05:15:03.959678+0000","last_became_active":"2023-10-06T05:14:59.969109+0000","last_became_peered":"2023-10-06T05:14:59.969109+0000","last_unstale":"2023-10-06T05:15:03.959678+0000","last_undegraded":"2023-10-06T05:15:03.959678+0000","last_fullsized":"2023-10-06T05:15:03.959678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:59.890813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.989045+0000","last_change":"2023-10-06T05:15:03.982573+0000","last_active":"2023-10-06T05:15:03.989045+0000","last_peered":"2023-10-06T05:15:03.989045+0000","last_clean":"2023-10-06T05:15:03.989045+0000","last_became_active":"2023-10-06T05:15:03.982173+0000","last_became_peered":"2023-10-06T05:15:03.982173+0000","last_unstale":"2023-10-06T05:15:03.989045+0000","last_undegraded":"2023-10-06T05:15:03.989045+0000","last_fullsized":"2023-10-06T05:15:03.989045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.921378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975586+0000","last_change":"2023-10-06T05:14:58.962379+0000","last_active":"2023-10-06T05:15:03.975586+0000","last_peered":"2023-10-06T05:15:03.975586+0000","last_clean":"2023-10-06T05:15:03.975586+0000","last_became_active":"2023-10-06T05:14:58.962093+0000","last_became_peered":"2023-10-06T05:14:58.962093+0000","last_unstale":"2023-10-06T05:15:03.975586+0000","last_undegraded":"2023-10-06T05:15:03.975586+0000","last_fullsized":"2023-10-06T05:15:03.975586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960163+0000","last_change":"2023-10-06T05:14:59.951502+0000","last_active":"2023-10-06T05:15:03.960163+0000","last_peered":"2023-10-06T05:15:03.960163+0000","last_clean":"2023-10-06T05:15:03.960163+0000","last_became_active":"2023-10-06T05:14:59.951197+0000","last_became_peered":"2023-10-06T05:14:59.951197+0000","last_unstale":"2023-10-06T05:15:03.960163+0000","last_undegraded":"2023-10-06T05:15:03.960163+0000","last_fullsized":"2023-10-06T05:15:03.960163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978919+0000","last_change":"2023-10-06T05:15:03.977325+0000","last_active":"2023-10-06T05:15:03.978919+0000","last_peered":"2023-10-06T05:15:03.978919+0000","last_clean":"2023-10-06T05:15:03.978919+0000","last_became_active":"2023-10-06T05:15:03.976844+0000","last_became_peered":"2023-10-06T05:15:03.976844+0000","last_unstale":"2023-10-06T05:15:03.978919+0000","last_undegraded":"2023-10-06T05:15:03.978919+0000","last_fullsized":"2023-10-06T05:15:03.978919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:47.153239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960726+0000","last_change":"2023-10-06T05:14:58.958227+0000","last_active":"2023-10-06T05:15:03.960726+0000","last_peered":"2023-10-06T05:15:03.960726+0000","last_clean":"2023-10-06T05:15:03.960726+0000","last_became_active":"2023-10-06T05:14:58.957924+0000","last_became_peered":"2023-10-06T05:14:58.957924+0000","last_unstale":"2023-10-06T05:15:03.960726+0000","last_undegraded":"2023-10-06T05:15:03.960726+0000","last_fullsized":"2023-10-06T05:15:03.960726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961096+0000","last_change":"2023-10-06T05:14:59.961306+0000","last_active":"2023-10-06T05:15:03.961096+0000","last_peered":"2023-10-06T05:15:03.961096+0000","last_clean":"2023-10-06T05:15:03.961096+0000","last_became_active":"2023-10-06T05:14:59.961025+0000","last_became_peered":"2023-10-06T05:14:59.961025+0000","last_unstale":"2023-10-06T05:15:03.961096+0000","last_undegraded":"2023-10-06T05:15:03.961096+0000","last_fullsized":"2023-10-06T05:15:03.961096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987866+0000","last_change":"2023-10-06T05:15:03.983777+0000","last_active":"2023-10-06T05:15:03.987866+0000","last_peered":"2023-10-06T05:15:03.987866+0000","last_clean":"2023-10-06T05:15:03.987866+0000","last_became_active":"2023-10-06T05:15:03.983380+0000","last_became_peered":"2023-10-06T05:15:03.983380+0000","last_unstale":"2023-10-06T05:15:03.987866+0000","last_undegraded":"2023-10-06T05:15:03.987866+0000","last_fullsized":"2023-10-06T05:15:03.987866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:21.551659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967787+0000","last_change":"2023-10-06T05:14:58.966032+0000","last_active":"2023-10-06T05:15:03.967787+0000","last_peered":"2023-10-06T05:15:03.967787+0000","last_clean":"2023-10-06T05:15:03.967787+0000","last_became_active":"2023-10-06T05:14:58.965697+0000","last_became_peered":"2023-10-06T05:14:58.965697+0000","last_unstale":"2023-10-06T05:15:03.967787+0000","last_undegraded":"2023-10-06T05:15:03.967787+0000","last_fullsized":"2023-10-06T05:15:03.967787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979242+0000","last_change":"2023-10-06T05:14:59.966425+0000","last_active":"2023-10-06T05:15:03.979242+0000","last_peered":"2023-10-06T05:15:03.979242+0000","last_clean":"2023-10-06T05:15:03.979242+0000","last_became_active":"2023-10-06T05:14:59.966122+0000","last_became_peered":"2023-10-06T05:14:59.966122+0000","last_unstale":"2023-10-06T05:15:03.979242+0000","last_undegraded":"2023-10-06T05:15:03.979242+0000","last_fullsized":"2023-10-06T05:15:03.979242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.983132+0000","last_change":"2023-10-06T05:15:03.979017+0000","last_active":"2023-10-06T05:15:03.983132+0000","last_peered":"2023-10-06T05:15:03.983132+0000","last_clean":"2023-10-06T05:15:03.983132+0000","last_became_active":"2023-10-06T05:15:03.978644+0000","last_became_peered":"2023-10-06T05:15:03.978644+0000","last_unstale":"2023-10-06T05:15:03.983132+0000","last_undegraded":"2023-10-06T05:15:03.983132+0000","last_fullsized":"2023-10-06T05:15:03.983132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973620+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:15:03.973620+0000","last_peered":"2023-10-06T05:15:03.973620+0000","last_clean":"2023-10-06T05:15:03.973620+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:15:03.973620+0000","last_undegraded":"2023-10-06T05:15:03.973620+0000","last_fullsized":"2023-10-06T05:15:03.973620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962773+0000","last_change":"2023-10-06T05:14:58.969546+0000","last_active":"2023-10-06T05:15:03.962773+0000","last_peered":"2023-10-06T05:15:03.962773+0000","last_clean":"2023-10-06T05:15:03.962773+0000","last_became_active":"2023-10-06T05:14:58.969281+0000","last_became_peered":"2023-10-06T05:14:58.969281+0000","last_unstale":"2023-10-06T05:15:03.962773+0000","last_undegraded":"2023-10-06T05:15:03.962773+0000","last_fullsized":"2023-10-06T05:15:03.962773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973704+0000","last_change":"2023-10-06T05:14:59.965859+0000","last_active":"2023-10-06T05:15:03.973704+0000","last_peered":"2023-10-06T05:15:03.973704+0000","last_clean":"2023-10-06T05:15:03.973704+0000","last_became_active":"2023-10-06T05:14:59.964784+0000","last_became_peered":"2023-10-06T05:14:59.964784+0000","last_unstale":"2023-10-06T05:15:03.973704+0000","last_undegraded":"2023-10-06T05:15:03.973704+0000","last_fullsized":"2023-10-06T05:15:03.973704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985105+0000","last_change":"2023-10-06T05:15:03.982375+0000","last_active":"2023-10-06T05:15:03.985105+0000","last_peered":"2023-10-06T05:15:03.985105+0000","last_clean":"2023-10-06T05:15:03.985105+0000","last_became_active":"2023-10-06T05:15:03.981983+0000","last_became_peered":"2023-10-06T05:15:03.981983+0000","last_unstale":"2023-10-06T05:15:03.985105+0000","last_undegraded":"2023-10-06T05:15:03.985105+0000","last_fullsized":"2023-10-06T05:15:03.985105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962357+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:15:03.962357+0000","last_peered":"2023-10-06T05:15:03.962357+0000","last_clean":"2023-10-06T05:15:03.962357+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:15:03.962357+0000","last_undegraded":"2023-10-06T05:15:03.962357+0000","last_fullsized":"2023-10-06T05:15:03.962357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959855+0000","last_change":"2023-10-06T05:14:58.966374+0000","last_active":"2023-10-06T05:15:03.959855+0000","last_peered":"2023-10-06T05:15:03.959855+0000","last_clean":"2023-10-06T05:15:03.959855+0000","last_became_active":"2023-10-06T05:14:58.966124+0000","last_became_peered":"2023-10-06T05:14:58.966124+0000","last_unstale":"2023-10-06T05:15:03.959855+0000","last_undegraded":"2023-10-06T05:15:03.959855+0000","last_fullsized":"2023-10-06T05:15:03.959855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973548+0000","last_change":"2023-10-06T05:14:59.953791+0000","last_active":"2023-10-06T05:15:03.973548+0000","last_peered":"2023-10-06T05:15:03.973548+0000","last_clean":"2023-10-06T05:15:03.973548+0000","last_became_active":"2023-10-06T05:14:59.953491+0000","last_became_peered":"2023-10-06T05:14:59.953491+0000","last_unstale":"2023-10-06T05:15:03.973548+0000","last_undegraded":"2023-10-06T05:15:03.973548+0000","last_fullsized":"2023-10-06T05:15:03.973548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994360+0000","last_change":"2023-10-06T05:15:03.993204+0000","last_active":"2023-10-06T05:15:03.994360+0000","last_peered":"2023-10-06T05:15:03.994360+0000","last_clean":"2023-10-06T05:15:03.994360+0000","last_became_active":"2023-10-06T05:15:03.992794+0000","last_became_peered":"2023-10-06T05:15:03.992794+0000","last_unstale":"2023-10-06T05:15:03.994360+0000","last_undegraded":"2023-10-06T05:15:03.994360+0000","last_fullsized":"2023-10-06T05:15:03.994360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:53.385235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967975+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:15:03.967975+0000","last_peered":"2023-10-06T05:15:03.967975+0000","last_clean":"2023-10-06T05:15:03.967975+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:15:03.967975+0000","last_undegraded":"2023-10-06T05:15:03.967975+0000","last_fullsized":"2023-10-06T05:15:03.967975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972830+0000","last_change":"2023-10-06T05:14:58.954417+0000","last_active":"2023-10-06T05:15:03.972830+0000","last_peered":"2023-10-06T05:15:03.972830+0000","last_clean":"2023-10-06T05:15:03.972830+0000","last_became_active":"2023-10-06T05:14:58.954052+0000","last_became_peered":"2023-10-06T05:14:58.954052+0000","last_unstale":"2023-10-06T05:15:03.972830+0000","last_undegraded":"2023-10-06T05:15:03.972830+0000","last_fullsized":"2023-10-06T05:15:03.972830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979150+0000","last_change":"2023-10-06T05:14:59.963695+0000","last_active":"2023-10-06T05:15:03.979150+0000","last_peered":"2023-10-06T05:15:03.979150+0000","last_clean":"2023-10-06T05:15:03.979150+0000","last_became_active":"2023-10-06T05:14:59.963426+0000","last_became_peered":"2023-10-06T05:14:59.963426+0000","last_unstale":"2023-10-06T05:15:03.979150+0000","last_undegraded":"2023-10-06T05:15:03.979150+0000","last_fullsized":"2023-10-06T05:15:03.979150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987182+0000","last_change":"2023-10-06T05:15:03.985907+0000","last_active":"2023-10-06T05:15:03.987182+0000","last_peered":"2023-10-06T05:15:03.987182+0000","last_clean":"2023-10-06T05:15:03.987182+0000","last_became_active":"2023-10-06T05:15:03.985534+0000","last_became_peered":"2023-10-06T05:15:03.985534+0000","last_unstale":"2023-10-06T05:15:03.987182+0000","last_undegraded":"2023-10-06T05:15:03.987182+0000","last_fullsized":"2023-10-06T05:15:03.987182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961919+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:15:03.961919+0000","last_peered":"2023-10-06T05:15:03.961919+0000","last_clean":"2023-10-06T05:15:03.961919+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:15:03.961919+0000","last_undegraded":"2023-10-06T05:15:03.961919+0000","last_fullsized":"2023-10-06T05:15:03.961919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964512+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:15:03.964512+0000","last_peered":"2023-10-06T05:15:03.964512+0000","last_clean":"2023-10-06T05:15:03.964512+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:15:03.964512+0000","last_undegraded":"2023-10-06T05:15:03.964512+0000","last_fullsized":"2023-10-06T05:15:03.964512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963731+0000","last_change":"2023-10-06T05:14:58.965199+0000","last_active":"2023-10-06T05:15:03.963731+0000","last_peered":"2023-10-06T05:15:03.963731+0000","last_clean":"2023-10-06T05:15:03.963731+0000","last_became_active":"2023-10-06T05:14:58.964902+0000","last_became_peered":"2023-10-06T05:14:58.964902+0000","last_unstale":"2023-10-06T05:15:03.963731+0000","last_undegraded":"2023-10-06T05:15:03.963731+0000","last_fullsized":"2023-10-06T05:15:03.963731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975967+0000","last_change":"2023-10-06T05:14:59.953471+0000","last_active":"2023-10-06T05:15:03.975967+0000","last_peered":"2023-10-06T05:15:03.975967+0000","last_clean":"2023-10-06T05:15:03.975967+0000","last_became_active":"2023-10-06T05:14:59.953166+0000","last_became_peered":"2023-10-06T05:14:59.953166+0000","last_unstale":"2023-10-06T05:15:03.975967+0000","last_undegraded":"2023-10-06T05:15:03.975967+0000","last_fullsized":"2023-10-06T05:15:03.975967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:35.282268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987431+0000","last_change":"2023-10-06T05:15:03.982741+0000","last_active":"2023-10-06T05:15:03.987431+0000","last_peered":"2023-10-06T05:15:03.987431+0000","last_clean":"2023-10-06T05:15:03.987431+0000","last_became_active":"2023-10-06T05:15:03.982333+0000","last_became_peered":"2023-10-06T05:15:03.982333+0000","last_unstale":"2023-10-06T05:15:03.987431+0000","last_undegraded":"2023-10-06T05:15:03.987431+0000","last_fullsized":"2023-10-06T05:15:03.987431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:07.926945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964687+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:15:03.964687+0000","last_peered":"2023-10-06T05:15:03.964687+0000","last_clean":"2023-10-06T05:15:03.964687+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:15:03.964687+0000","last_undegraded":"2023-10-06T05:15:03.964687+0000","last_fullsized":"2023-10-06T05:15:03.964687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965134+0000","last_change":"2023-10-06T05:14:58.970767+0000","last_active":"2023-10-06T05:15:03.965134+0000","last_peered":"2023-10-06T05:15:03.965134+0000","last_clean":"2023-10-06T05:15:03.965134+0000","last_became_active":"2023-10-06T05:14:58.970523+0000","last_became_peered":"2023-10-06T05:14:58.970523+0000","last_unstale":"2023-10-06T05:15:03.965134+0000","last_undegraded":"2023-10-06T05:15:03.965134+0000","last_fullsized":"2023-10-06T05:15:03.965134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970560+0000","last_change":"2023-10-06T05:14:59.958130+0000","last_active":"2023-10-06T05:15:03.970560+0000","last_peered":"2023-10-06T05:15:03.970560+0000","last_clean":"2023-10-06T05:15:03.970560+0000","last_became_active":"2023-10-06T05:14:59.957826+0000","last_became_peered":"2023-10-06T05:14:59.957826+0000","last_unstale":"2023-10-06T05:15:03.970560+0000","last_undegraded":"2023-10-06T05:15:03.970560+0000","last_fullsized":"2023-10-06T05:15:03.970560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987228+0000","last_change":"2023-10-06T05:15:03.981078+0000","last_active":"2023-10-06T05:15:03.987228+0000","last_peered":"2023-10-06T05:15:03.987228+0000","last_clean":"2023-10-06T05:15:03.987228+0000","last_became_active":"2023-10-06T05:15:03.980645+0000","last_became_peered":"2023-10-06T05:15:03.980645+0000","last_unstale":"2023-10-06T05:15:03.987228+0000","last_undegraded":"2023-10-06T05:15:03.987228+0000","last_fullsized":"2023-10-06T05:15:03.987228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.847066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967525+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:15:03.967525+0000","last_peered":"2023-10-06T05:15:03.967525+0000","last_clean":"2023-10-06T05:15:03.967525+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:15:03.967525+0000","last_undegraded":"2023-10-06T05:15:03.967525+0000","last_fullsized":"2023-10-06T05:15:03.967525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973787+0000","last_change":"2023-10-06T05:14:58.954390+0000","last_active":"2023-10-06T05:15:03.973787+0000","last_peered":"2023-10-06T05:15:03.973787+0000","last_clean":"2023-10-06T05:15:03.973787+0000","last_became_active":"2023-10-06T05:14:58.954003+0000","last_became_peered":"2023-10-06T05:14:58.954003+0000","last_unstale":"2023-10-06T05:15:03.973787+0000","last_undegraded":"2023-10-06T05:15:03.973787+0000","last_fullsized":"2023-10-06T05:15:03.973787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967163+0000","last_change":"2023-10-06T05:14:59.965298+0000","last_active":"2023-10-06T05:15:03.967163+0000","last_peered":"2023-10-06T05:15:03.967163+0000","last_clean":"2023-10-06T05:15:03.967163+0000","last_became_active":"2023-10-06T05:14:59.964967+0000","last_became_peered":"2023-10-06T05:14:59.964967+0000","last_unstale":"2023-10-06T05:15:03.967163+0000","last_undegraded":"2023-10-06T05:15:03.967163+0000","last_fullsized":"2023-10-06T05:15:03.967163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978732+0000","last_change":"2023-10-06T05:15:03.976855+0000","last_active":"2023-10-06T05:15:03.978732+0000","last_peered":"2023-10-06T05:15:03.978732+0000","last_clean":"2023-10-06T05:15:03.978732+0000","last_became_active":"2023-10-06T05:15:03.976461+0000","last_became_peered":"2023-10-06T05:15:03.976461+0000","last_unstale":"2023-10-06T05:15:03.978732+0000","last_undegraded":"2023-10-06T05:15:03.978732+0000","last_fullsized":"2023-10-06T05:15:03.978732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:45.979196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964732+0000","last_change":"2023-10-06T05:14:58.970171+0000","last_active":"2023-10-06T05:15:03.964732+0000","last_peered":"2023-10-06T05:15:03.964732+0000","last_clean":"2023-10-06T05:15:03.964732+0000","last_became_active":"2023-10-06T05:14:58.969914+0000","last_became_peered":"2023-10-06T05:14:58.969914+0000","last_unstale":"2023-10-06T05:15:03.964732+0000","last_undegraded":"2023-10-06T05:15:03.964732+0000","last_fullsized":"2023-10-06T05:15:03.964732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977482+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:15:03.977482+0000","last_peered":"2023-10-06T05:15:03.977482+0000","last_clean":"2023-10-06T05:15:03.977482+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:15:03.977482+0000","last_undegraded":"2023-10-06T05:15:03.977482+0000","last_fullsized":"2023-10-06T05:15:03.977482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964253+0000","last_change":"2023-10-06T05:14:59.964377+0000","last_active":"2023-10-06T05:15:03.964253+0000","last_peered":"2023-10-06T05:15:03.964253+0000","last_clean":"2023-10-06T05:15:03.964253+0000","last_became_active":"2023-10-06T05:14:59.964058+0000","last_became_peered":"2023-10-06T05:14:59.964058+0000","last_unstale":"2023-10-06T05:15:03.964253+0000","last_undegraded":"2023-10-06T05:15:03.964253+0000","last_fullsized":"2023-10-06T05:15:03.964253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994304+0000","last_change":"2023-10-06T05:15:03.991574+0000","last_active":"2023-10-06T05:15:03.994304+0000","last_peered":"2023-10-06T05:15:03.994304+0000","last_clean":"2023-10-06T05:15:03.994304+0000","last_became_active":"2023-10-06T05:15:03.991171+0000","last_became_peered":"2023-10-06T05:15:03.991171+0000","last_unstale":"2023-10-06T05:15:03.994304+0000","last_undegraded":"2023-10-06T05:15:03.994304+0000","last_fullsized":"2023-10-06T05:15:03.994304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975779+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:15:03.975779+0000","last_peered":"2023-10-06T05:15:03.975779+0000","last_clean":"2023-10-06T05:15:03.975779+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:15:03.975779+0000","last_undegraded":"2023-10-06T05:15:03.975779+0000","last_fullsized":"2023-10-06T05:15:03.975779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970480+0000","last_change":"2023-10-06T05:14:58.966696+0000","last_active":"2023-10-06T05:15:03.970480+0000","last_peered":"2023-10-06T05:15:03.970480+0000","last_clean":"2023-10-06T05:15:03.970480+0000","last_became_active":"2023-10-06T05:14:58.966398+0000","last_became_peered":"2023-10-06T05:14:58.966398+0000","last_unstale":"2023-10-06T05:15:03.970480+0000","last_undegraded":"2023-10-06T05:15:03.970480+0000","last_fullsized":"2023-10-06T05:15:03.970480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960837+0000","last_change":"2023-10-06T05:14:59.967931+0000","last_active":"2023-10-06T05:15:03.960837+0000","last_peered":"2023-10-06T05:15:03.960837+0000","last_clean":"2023-10-06T05:15:03.960837+0000","last_became_active":"2023-10-06T05:14:59.967681+0000","last_became_peered":"2023-10-06T05:14:59.967681+0000","last_unstale":"2023-10-06T05:15:03.960837+0000","last_undegraded":"2023-10-06T05:15:03.960837+0000","last_fullsized":"2023-10-06T05:15:03.960837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990296+0000","last_change":"2023-10-06T05:15:03.989021+0000","last_active":"2023-10-06T05:15:03.990296+0000","last_peered":"2023-10-06T05:15:03.990296+0000","last_clean":"2023-10-06T05:15:03.990296+0000","last_became_active":"2023-10-06T05:15:03.988411+0000","last_became_peered":"2023-10-06T05:15:03.988411+0000","last_unstale":"2023-10-06T05:15:03.990296+0000","last_undegraded":"2023-10-06T05:15:03.990296+0000","last_fullsized":"2023-10-06T05:15:03.990296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967052+0000","last_change":"2023-10-06T05:14:58.970840+0000","last_active":"2023-10-06T05:15:03.967052+0000","last_peered":"2023-10-06T05:15:03.967052+0000","last_clean":"2023-10-06T05:15:03.967052+0000","last_became_active":"2023-10-06T05:14:58.970619+0000","last_became_peered":"2023-10-06T05:14:58.970619+0000","last_unstale":"2023-10-06T05:15:03.967052+0000","last_undegraded":"2023-10-06T05:15:03.967052+0000","last_fullsized":"2023-10-06T05:15:03.967052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968655+0000","last_change":"2023-10-06T05:14:59.955830+0000","last_active":"2023-10-06T05:15:03.968655+0000","last_peered":"2023-10-06T05:15:03.968655+0000","last_clean":"2023-10-06T05:15:03.968655+0000","last_became_active":"2023-10-06T05:14:59.955566+0000","last_became_peered":"2023-10-06T05:14:59.955566+0000","last_unstale":"2023-10-06T05:15:03.968655+0000","last_undegraded":"2023-10-06T05:15:03.968655+0000","last_fullsized":"2023-10-06T05:15:03.968655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:47.738141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987078+0000","last_change":"2023-10-06T05:15:03.984831+0000","last_active":"2023-10-06T05:15:03.987078+0000","last_peered":"2023-10-06T05:15:03.987078+0000","last_clean":"2023-10-06T05:15:03.987078+0000","last_became_active":"2023-10-06T05:15:03.984436+0000","last_became_peered":"2023-10-06T05:15:03.984436+0000","last_unstale":"2023-10-06T05:15:03.987078+0000","last_undegraded":"2023-10-06T05:15:03.987078+0000","last_fullsized":"2023-10-06T05:15:03.987078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:10.551105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961685+0000","last_change":"2023-10-06T05:14:58.956457+0000","last_active":"2023-10-06T05:15:03.961685+0000","last_peered":"2023-10-06T05:15:03.961685+0000","last_clean":"2023-10-06T05:15:03.961685+0000","last_became_active":"2023-10-06T05:14:58.956165+0000","last_became_peered":"2023-10-06T05:14:58.956165+0000","last_unstale":"2023-10-06T05:15:03.961685+0000","last_undegraded":"2023-10-06T05:15:03.961685+0000","last_fullsized":"2023-10-06T05:15:03.961685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.968300+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.967965+0000","last_became_peered":"2023-10-06T05:14:59.967965+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:35.451365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978224+0000","last_change":"2023-10-06T05:15:03.976752+0000","last_active":"2023-10-06T05:15:03.978224+0000","last_peered":"2023-10-06T05:15:03.978224+0000","last_clean":"2023-10-06T05:15:03.978224+0000","last_became_active":"2023-10-06T05:15:03.976322+0000","last_became_peered":"2023-10-06T05:15:03.976322+0000","last_unstale":"2023-10-06T05:15:03.978224+0000","last_undegraded":"2023-10-06T05:15:03.978224+0000","last_fullsized":"2023-10-06T05:15:03.978224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962719+0000","last_change":"2023-10-06T05:14:58.955881+0000","last_active":"2023-10-06T05:15:03.962719+0000","last_peered":"2023-10-06T05:15:03.962719+0000","last_clean":"2023-10-06T05:15:03.962719+0000","last_became_active":"2023-10-06T05:14:58.955624+0000","last_became_peered":"2023-10-06T05:14:58.955624+0000","last_unstale":"2023-10-06T05:15:03.962719+0000","last_undegraded":"2023-10-06T05:15:03.962719+0000","last_fullsized":"2023-10-06T05:15:03.962719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:16.365139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960721+0000","last_change":"2023-10-06T05:14:59.968565+0000","last_active":"2023-10-06T05:15:03.960721+0000","last_peered":"2023-10-06T05:15:03.960721+0000","last_clean":"2023-10-06T05:15:03.960721+0000","last_became_active":"2023-10-06T05:14:59.965728+0000","last_became_peered":"2023-10-06T05:14:59.965728+0000","last_unstale":"2023-10-06T05:15:03.960721+0000","last_undegraded":"2023-10-06T05:15:03.960721+0000","last_fullsized":"2023-10-06T05:15:03.960721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:49.359907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977840+0000","last_change":"2023-10-06T05:15:03.975283+0000","last_active":"2023-10-06T05:15:03.977840+0000","last_peered":"2023-10-06T05:15:03.977840+0000","last_clean":"2023-10-06T05:15:03.977840+0000","last_became_active":"2023-10-06T05:15:03.974903+0000","last_became_peered":"2023-10-06T05:15:03.974903+0000","last_unstale":"2023-10-06T05:15:03.977840+0000","last_undegraded":"2023-10-06T05:15:03.977840+0000","last_fullsized":"2023-10-06T05:15:03.977840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:18.853022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966753+0000","last_change":"2023-10-06T05:14:58.961971+0000","last_active":"2023-10-06T05:15:03.966753+0000","last_peered":"2023-10-06T05:15:03.966753+0000","last_clean":"2023-10-06T05:15:03.966753+0000","last_became_active":"2023-10-06T05:14:58.961689+0000","last_became_peered":"2023-10-06T05:14:58.961689+0000","last_unstale":"2023-10-06T05:15:03.966753+0000","last_undegraded":"2023-10-06T05:15:03.966753+0000","last_fullsized":"2023-10-06T05:15:03.966753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963640+0000","last_change":"2023-10-06T05:14:59.959821+0000","last_active":"2023-10-06T05:15:03.963640+0000","last_peered":"2023-10-06T05:15:03.963640+0000","last_clean":"2023-10-06T05:15:03.963640+0000","last_became_active":"2023-10-06T05:14:59.959543+0000","last_became_peered":"2023-10-06T05:14:59.959543+0000","last_unstale":"2023-10-06T05:15:03.963640+0000","last_undegraded":"2023-10-06T05:15:03.963640+0000","last_fullsized":"2023-10-06T05:15:03.963640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.991469+0000","last_change":"2023-10-06T05:15:03.990597+0000","last_active":"2023-10-06T05:15:03.991469+0000","last_peered":"2023-10-06T05:15:03.991469+0000","last_clean":"2023-10-06T05:15:03.991469+0000","last_became_active":"2023-10-06T05:15:03.990258+0000","last_became_peered":"2023-10-06T05:15:03.990258+0000","last_unstale":"2023-10-06T05:15:03.991469+0000","last_undegraded":"2023-10-06T05:15:03.991469+0000","last_fullsized":"2023-10-06T05:15:03.991469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970366+0000","last_change":"2023-10-06T05:14:58.970911+0000","last_active":"2023-10-06T05:15:03.970366+0000","last_peered":"2023-10-06T05:15:03.970366+0000","last_clean":"2023-10-06T05:15:03.970366+0000","last_became_active":"2023-10-06T05:14:58.970439+0000","last_became_peered":"2023-10-06T05:14:58.970439+0000","last_unstale":"2023-10-06T05:15:03.970366+0000","last_undegraded":"2023-10-06T05:15:03.970366+0000","last_fullsized":"2023-10-06T05:15:03.970366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960109+0000","last_change":"2023-10-06T05:14:59.961878+0000","last_active":"2023-10-06T05:15:03.960109+0000","last_peered":"2023-10-06T05:15:03.960109+0000","last_clean":"2023-10-06T05:15:03.960109+0000","last_became_active":"2023-10-06T05:14:59.961617+0000","last_became_peered":"2023-10-06T05:14:59.961617+0000","last_unstale":"2023-10-06T05:15:03.960109+0000","last_undegraded":"2023-10-06T05:15:03.960109+0000","last_fullsized":"2023-10-06T05:15:03.960109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974134+0000","last_change":"2023-10-06T05:14:59.960593+0000","last_active":"2023-10-06T05:15:03.974134+0000","last_peered":"2023-10-06T05:15:03.974134+0000","last_clean":"2023-10-06T05:15:03.974134+0000","last_became_active":"2023-10-06T05:14:59.960329+0000","last_became_peered":"2023-10-06T05:14:59.960329+0000","last_unstale":"2023-10-06T05:15:03.974134+0000","last_undegraded":"2023-10-06T05:15:03.974134+0000","last_fullsized":"2023-10-06T05:15:03.974134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963791+0000","last_change":"2023-10-06T05:14:59.966932+0000","last_active":"2023-10-06T05:15:03.963791+0000","last_peered":"2023-10-06T05:15:03.963791+0000","last_clean":"2023-10-06T05:15:03.963791+0000","last_became_active":"2023-10-06T05:14:59.965101+0000","last_became_peered":"2023-10-06T05:14:59.965101+0000","last_unstale":"2023-10-06T05:15:03.963791+0000","last_undegraded":"2023-10-06T05:15:03.963791+0000","last_fullsized":"2023-10-06T05:15:03.963791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961190+0000","last_change":"2023-10-06T05:14:59.961912+0000","last_active":"2023-10-06T05:15:03.961190+0000","last_peered":"2023-10-06T05:15:03.961190+0000","last_clean":"2023-10-06T05:15:03.961190+0000","last_became_active":"2023-10-06T05:14:59.961658+0000","last_became_peered":"2023-10-06T05:14:59.961658+0000","last_unstale":"2023-10-06T05:15:03.961190+0000","last_undegraded":"2023-10-06T05:15:03.961190+0000","last_fullsized":"2023-10-06T05:15:03.961190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963927+0000","last_change":"2023-10-06T05:14:59.968762+0000","last_active":"2023-10-06T05:15:03.963927+0000","last_peered":"2023-10-06T05:15:03.963927+0000","last_clean":"2023-10-06T05:15:03.963927+0000","last_became_active":"2023-10-06T05:14:59.968111+0000","last_became_peered":"2023-10-06T05:14:59.968111+0000","last_unstale":"2023-10-06T05:15:03.963927+0000","last_undegraded":"2023-10-06T05:15:03.963927+0000","last_fullsized":"2023-10-06T05:15:03.963927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973699+0000","last_change":"2023-10-06T05:14:59.958480+0000","last_active":"2023-10-06T05:15:03.973699+0000","last_peered":"2023-10-06T05:15:03.973699+0000","last_clean":"2023-10-06T05:15:03.973699+0000","last_became_active":"2023-10-06T05:14:59.958139+0000","last_became_peered":"2023-10-06T05:14:59.958139+0000","last_unstale":"2023-10-06T05:15:03.973699+0000","last_undegraded":"2023-10-06T05:15:03.973699+0000","last_fullsized":"2023-10-06T05:15:03.973699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:03.025853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":695352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":103167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:19.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:20.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.135+0000 7f3007318700 1 Processor -- start 2023-10-06T05:15:20.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.135+0000 7f3007318700 1 -- start start 2023-10-06T05:15:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f3007318700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3000155cd0 0x7f300015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f3007318700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30001560a0 0x7f30000adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f3007318700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f30000ccdc0 0x7f30000cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f3007318700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30000cd740 con 0x7f3000155cd0 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f3007318700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30000ae8d0 con 0x7f30000ccdc0 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f3007318700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30000aebb0 con 0x7f30001560a0 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f30050b4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3000155cd0 0x7f300015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f30050b4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3000155cd0 0x7f300015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53038/0 (socket says 172.21.15.45:53038) 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f30050b4700 1 -- 172.21.15.45:0/925890698 learned_addr learned my addr 172.21.15.45:0/925890698 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f30048b3700 1 --2- 172.21.15.45:0/925890698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30001560a0 0x7f30000adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.136+0000 7f30058b5700 1 --2- 172.21.15.45:0/925890698 >> v2:172.21.15.144:3300/0 conn(0x7f30000ccdc0 0x7f30000cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f30050b4700 1 -- 172.21.15.45:0/925890698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30001560a0 msgr2=0x7f30000adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f30050b4700 1 --2- 172.21.15.45:0/925890698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30001560a0 0x7f30000adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f30050b4700 1 -- 172.21.15.45:0/925890698 >> v2:172.21.15.144:3300/0 conn(0x7f30000ccdc0 msgr2=0x7f30000cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f30050b4700 1 --2- 172.21.15.45:0/925890698 >> v2:172.21.15.144:3300/0 conn(0x7f30000ccdc0 0x7f30000cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f30050b4700 1 -- 172.21.15.45:0/925890698 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30000aee90 con 0x7f3000155cd0 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f30048b3700 1 --2- 172.21.15.45:0/925890698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30001560a0 0x7f30000adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f30050b4700 1 --2- 172.21.15.45:0/925890698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000155cd0 0x7f300015a730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30000af930 tx=0x7f2ffc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da46a6d33ff3966 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f2ff7fff700 1 -- 172.21.15.45:0/925890698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00dc50 con 0x7f3000155cd0 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f2ff7fff700 1 -- 172.21.15.45:0/925890698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc00ddb0 con 0x7f3000155cd0 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.137+0000 7f2ff7fff700 1 -- 172.21.15.45:0/925890698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0196d0 con 0x7f3000155cd0 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 -- 172.21.15.45:0/925890698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000155cd0 msgr2=0x7f300015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 --2- 172.21.15.45:0/925890698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000155cd0 0x7f300015a730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30000af930 tx=0x7f2ffc00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 -- 172.21.15.45:0/925890698 shutdown_connections 2023-10-06T05:15:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 --2- 172.21.15.45:0/925890698 >> v2:172.21.15.144:3300/0 conn(0x7f30000ccdc0 0x7f30000cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 --2- 172.21.15.45:0/925890698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3000155cd0 0x7f300015a730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 --2- 172.21.15.45:0/925890698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30001560a0 0x7f30000adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 -- 172.21.15.45:0/925890698 >> 172.21.15.45:0/925890698 conn(0x7f300006f090 msgr2=0x7f300006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 -- 172.21.15.45:0/925890698 shutdown_connections 2023-10-06T05:15:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.138+0000 7f3007318700 1 -- 172.21.15.45:0/925890698 wait complete. 2023-10-06T05:15:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.139+0000 7f3007318700 1 Processor -- start 2023-10-06T05:15:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.139+0000 7f3007318700 1 -- start start 2023-10-06T05:15:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f3007318700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30000ccdc0 0x7f3000164e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f3007318700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3000155cd0 0x7f3000157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f3007318700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30001560a0 0x7f30000b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f3007318700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30000b0300 con 0x7f30001560a0 2023-10-06T05:15:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f3007318700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3000158430 con 0x7f3000155cd0 2023-10-06T05:15:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f3007318700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30000afd40 con 0x7f30000ccdc0 2023-10-06T05:15:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30001560a0 0x7f30000b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30001560a0 0x7f30000b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53048/0 (socket says 172.21.15.45:53048) 2023-10-06T05:15:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 -- 172.21.15.45:0/766232955 learned_addr learned my addr 172.21.15.45:0/766232955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30048b3700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3000155cd0 0x7f3000157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30050b4700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30000ccdc0 0x7f3000164e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 -- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30000ccdc0 msgr2=0x7f3000164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30000ccdc0 0x7f3000164e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 -- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3000155cd0 msgr2=0x7f3000157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3000155cd0 0x7f3000157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30058b5700 1 -- 172.21.15.45:0/766232955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc00b790 con 0x7f30001560a0 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.140+0000 7f30050b4700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30000ccdc0 0x7f3000164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.141+0000 7f30058b5700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30001560a0 0x7f30000b2470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009fa0 tx=0x7f2ff8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.141+0000 7f2ff57fa700 1 -- 172.21.15.45:0/766232955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff800ec70 con 0x7f30001560a0 2023-10-06T05:15:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.141+0000 7f2ff57fa700 1 -- 172.21.15.45:0/766232955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff800edd0 con 0x7f30001560a0 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.141+0000 7f2ff57fa700 1 -- 172.21.15.45:0/766232955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8014670 con 0x7f30001560a0 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.141+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30000b0020 con 0x7f30001560a0 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.141+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30000c4420 con 0x7f30001560a0 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.142+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30000c3fa0 con 0x7f30001560a0 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.142+0000 7f2ff57fa700 1 -- 172.21.15.45:0/766232955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ff80149e0 con 0x7f30001560a0 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.143+0000 7f2ff57fa700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fe4066f30 0x7f2fe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.143+0000 7f2ff57fa700 1 -- 172.21.15.45:0/766232955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2ff8085070 con 0x7f30001560a0 2023-10-06T05:15:20.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.143+0000 7f30050b4700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fe4066f30 0x7f2fe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.144+0000 7f30050b4700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fe4066f30 0x7f2fe40693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30000af930 tx=0x7f2ffc01b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.147+0000 7f2ff57fa700 1 -- 172.21.15.45:0/766232955 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30000c3fa0 con 0x7f30001560a0 2023-10-06T05:15:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.344+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30000c3fa0 con 0x7f2fe4066f30 2023-10-06T05:15:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.354+0000 7f2ff57fa700 1 -- 172.21.15.45:0/766232955 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367375 (secure 0 0 0) 0x7f30000c3fa0 con 0x7f2fe4066f30 2023-10-06T05:15:20.360 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fe4066f30 msgr2=0x7f2fe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fe4066f30 0x7f2fe40693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30000af930 tx=0x7f2ffc01b000 comp rx=0 tx=0).stop 2023-10-06T05:15:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30001560a0 msgr2=0x7f30000b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30001560a0 0x7f30000b2470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009fa0 tx=0x7f2ff8009900 comp rx=0 tx=0).stop 2023-10-06T05:15:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 shutdown_connections 2023-10-06T05:15:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fe4066f30 0x7f2fe40693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30001560a0 0x7f30000b2470 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30000ccdc0 0x7f3000164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 --2- 172.21.15.45:0/766232955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3000155cd0 0x7f3000157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 >> 172.21.15.45:0/766232955 conn(0x7f300006f090 msgr2=0x7f3000152e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 shutdown_connections 2023-10-06T05:15:20.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.358+0000 7f3007318700 1 -- 172.21.15.45:0/766232955 wait complete. 2023-10-06T05:15:20.365 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:15:20.373 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-06T05:15:18.444499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003519"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.969110+0000","last_change":"2023-10-06T05:14:59.967201+0000","last_active":"2023-10-06T05:15:03.969110+0000","last_peered":"2023-10-06T05:15:03.969110+0000","last_clean":"2023-10-06T05:15:03.969110+0000","last_became_active":"2023-10-06T05:14:59.966818+0000","last_became_peered":"2023-10-06T05:14:59.966818+0000","last_unstale":"2023-10-06T05:15:03.969110+0000","last_undegraded":"2023-10-06T05:15:03.969110+0000","last_fullsized":"2023-10-06T05:15:03.969110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963221+0000","last_change":"2023-10-06T05:14:59.972677+0000","last_active":"2023-10-06T05:15:03.963221+0000","last_peered":"2023-10-06T05:15:03.963221+0000","last_clean":"2023-10-06T05:15:03.963221+0000","last_became_active":"2023-10-06T05:14:59.972435+0000","last_became_peered":"2023-10-06T05:14:59.972435+0000","last_unstale":"2023-10-06T05:15:03.963221+0000","last_undegraded":"2023-10-06T05:15:03.963221+0000","last_fullsized":"2023-10-06T05:15:03.963221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960554+0000","last_change":"2023-10-06T05:14:59.955617+0000","last_active":"2023-10-06T05:15:03.960554+0000","last_peered":"2023-10-06T05:15:03.960554+0000","last_clean":"2023-10-06T05:15:03.960554+0000","last_became_active":"2023-10-06T05:14:59.951585+0000","last_became_peered":"2023-10-06T05:14:59.951585+0000","last_unstale":"2023-10-06T05:15:03.960554+0000","last_undegraded":"2023-10-06T05:15:03.960554+0000","last_fullsized":"2023-10-06T05:15:03.960554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:56.835787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975703+0000","last_change":"2023-10-06T05:14:59.962591+0000","last_active":"2023-10-06T05:15:03.975703+0000","last_peered":"2023-10-06T05:15:03.975703+0000","last_clean":"2023-10-06T05:15:03.975703+0000","last_became_active":"2023-10-06T05:14:59.962219+0000","last_became_peered":"2023-10-06T05:14:59.962219+0000","last_unstale":"2023-10-06T05:15:03.975703+0000","last_undegraded":"2023-10-06T05:15:03.975703+0000","last_fullsized":"2023-10-06T05:15:03.975703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:11.373603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967261+0000","last_change":"2023-10-06T05:14:59.971703+0000","last_active":"2023-10-06T05:15:03.967261+0000","last_peered":"2023-10-06T05:15:03.967261+0000","last_clean":"2023-10-06T05:15:03.967261+0000","last_became_active":"2023-10-06T05:14:59.971083+0000","last_became_peered":"2023-10-06T05:14:59.971083+0000","last_unstale":"2023-10-06T05:15:03.967261+0000","last_undegraded":"2023-10-06T05:15:03.967261+0000","last_fullsized":"2023-10-06T05:15:03.967261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961235+0000","last_change":"2023-10-06T05:14:59.952376+0000","last_active":"2023-10-06T05:15:03.961235+0000","last_peered":"2023-10-06T05:15:03.961235+0000","last_clean":"2023-10-06T05:15:03.961235+0000","last_became_active":"2023-10-06T05:14:59.952101+0000","last_became_peered":"2023-10-06T05:14:59.952101+0000","last_unstale":"2023-10-06T05:15:03.961235+0000","last_undegraded":"2023-10-06T05:15:03.961235+0000","last_fullsized":"2023-10-06T05:15:03.961235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:46.640934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965050+0000","last_change":"2023-10-06T05:14:59.958503+0000","last_active":"2023-10-06T05:15:03.965050+0000","last_peered":"2023-10-06T05:15:03.965050+0000","last_clean":"2023-10-06T05:15:03.965050+0000","last_became_active":"2023-10-06T05:14:59.957962+0000","last_became_peered":"2023-10-06T05:14:59.957962+0000","last_unstale":"2023-10-06T05:15:03.965050+0000","last_undegraded":"2023-10-06T05:15:03.965050+0000","last_fullsized":"2023-10-06T05:15:03.965050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961531+0000","last_change":"2023-10-06T05:14:59.966704+0000","last_active":"2023-10-06T05:15:03.961531+0000","last_peered":"2023-10-06T05:15:03.961531+0000","last_clean":"2023-10-06T05:15:03.961531+0000","last_became_active":"2023-10-06T05:14:59.966378+0000","last_became_peered":"2023-10-06T05:14:59.966378+0000","last_unstale":"2023-10-06T05:15:03.961531+0000","last_undegraded":"2023-10-06T05:15:03.961531+0000","last_fullsized":"2023-10-06T05:15:03.961531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970103+0000","last_change":"2023-10-06T05:14:59.967328+0000","last_active":"2023-10-06T05:15:03.970103+0000","last_peered":"2023-10-06T05:15:03.970103+0000","last_clean":"2023-10-06T05:15:03.970103+0000","last_became_active":"2023-10-06T05:14:59.966868+0000","last_became_peered":"2023-10-06T05:14:59.966868+0000","last_unstale":"2023-10-06T05:15:03.970103+0000","last_undegraded":"2023-10-06T05:15:03.970103+0000","last_fullsized":"2023-10-06T05:15:03.970103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963384+0000","last_change":"2023-10-06T05:14:59.966488+0000","last_active":"2023-10-06T05:15:03.963384+0000","last_peered":"2023-10-06T05:15:03.963384+0000","last_clean":"2023-10-06T05:15:03.963384+0000","last_became_active":"2023-10-06T05:14:59.966153+0000","last_became_peered":"2023-10-06T05:14:59.966153+0000","last_unstale":"2023-10-06T05:15:03.963384+0000","last_undegraded":"2023-10-06T05:15:03.963384+0000","last_fullsized":"2023-10-06T05:15:03.963384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960401+0000","last_change":"2023-10-06T05:14:59.956939+0000","last_active":"2023-10-06T05:15:03.960401+0000","last_peered":"2023-10-06T05:15:03.960401+0000","last_clean":"2023-10-06T05:15:03.960401+0000","last_became_active":"2023-10-06T05:14:59.956467+0000","last_became_peered":"2023-10-06T05:14:59.956467+0000","last_unstale":"2023-10-06T05:15:03.960401+0000","last_undegraded":"2023-10-06T05:15:03.960401+0000","last_fullsized":"2023-10-06T05:15:03.960401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968295+0000","last_change":"2023-10-06T05:14:59.965955+0000","last_active":"2023-10-06T05:15:03.968295+0000","last_peered":"2023-10-06T05:15:03.968295+0000","last_clean":"2023-10-06T05:15:03.968295+0000","last_became_active":"2023-10-06T05:14:59.965676+0000","last_became_peered":"2023-10-06T05:14:59.965676+0000","last_unstale":"2023-10-06T05:15:03.968295+0000","last_undegraded":"2023-10-06T05:15:03.968295+0000","last_fullsized":"2023-10-06T05:15:03.968295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:39.098260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968290+0000","last_change":"2023-10-06T05:14:59.971318+0000","last_active":"2023-10-06T05:15:03.968290+0000","last_peered":"2023-10-06T05:15:03.968290+0000","last_clean":"2023-10-06T05:15:03.968290+0000","last_became_active":"2023-10-06T05:14:59.971044+0000","last_became_peered":"2023-10-06T05:14:59.971044+0000","last_unstale":"2023-10-06T05:15:03.968290+0000","last_undegraded":"2023-10-06T05:15:03.968290+0000","last_fullsized":"2023-10-06T05:15:03.968290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:17.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962264+0000","last_change":"2023-10-06T05:14:59.962752+0000","last_active":"2023-10-06T05:15:03.962264+0000","last_peered":"2023-10-06T05:15:03.962264+0000","last_clean":"2023-10-06T05:15:03.962264+0000","last_became_active":"2023-10-06T05:14:59.962473+0000","last_became_peered":"2023-10-06T05:14:59.962473+0000","last_unstale":"2023-10-06T05:15:03.962264+0000","last_undegraded":"2023-10-06T05:15:03.962264+0000","last_fullsized":"2023-10-06T05:15:03.962264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964859+0000","last_change":"2023-10-06T05:14:59.958686+0000","last_active":"2023-10-06T05:15:03.964859+0000","last_peered":"2023-10-06T05:15:03.964859+0000","last_clean":"2023-10-06T05:15:03.964859+0000","last_became_active":"2023-10-06T05:14:59.958258+0000","last_became_peered":"2023-10-06T05:14:59.958258+0000","last_unstale":"2023-10-06T05:15:03.964859+0000","last_undegraded":"2023-10-06T05:15:03.964859+0000","last_fullsized":"2023-10-06T05:15:03.964859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960536+0000","last_change":"2023-10-06T05:14:59.956754+0000","last_active":"2023-10-06T05:15:03.960536+0000","last_peered":"2023-10-06T05:15:03.960536+0000","last_clean":"2023-10-06T05:15:03.960536+0000","last_became_active":"2023-10-06T05:14:59.951445+0000","last_became_peered":"2023-10-06T05:14:59.951445+0000","last_unstale":"2023-10-06T05:15:03.960536+0000","last_undegraded":"2023-10-06T05:15:03.960536+0000","last_fullsized":"2023-10-06T05:15:03.960536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:43.881359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970204+0000","last_change":"2023-10-06T05:14:59.967155+0000","last_active":"2023-10-06T05:15:03.970204+0000","last_peered":"2023-10-06T05:15:03.970204+0000","last_clean":"2023-10-06T05:15:03.970204+0000","last_became_active":"2023-10-06T05:14:59.966533+0000","last_became_peered":"2023-10-06T05:14:59.966533+0000","last_unstale":"2023-10-06T05:15:03.970204+0000","last_undegraded":"2023-10-06T05:15:03.970204+0000","last_fullsized":"2023-10-06T05:15:03.970204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960993+0000","last_change":"2023-10-06T05:14:59.949784+0000","last_active":"2023-10-06T05:15:03.960993+0000","last_peered":"2023-10-06T05:15:03.960993+0000","last_clean":"2023-10-06T05:15:03.960993+0000","last_became_active":"2023-10-06T05:14:59.949540+0000","last_became_peered":"2023-10-06T05:14:59.949540+0000","last_unstale":"2023-10-06T05:15:03.960993+0000","last_undegraded":"2023-10-06T05:15:03.960993+0000","last_fullsized":"2023-10-06T05:15:03.960993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:55.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967686+0000","last_change":"2023-10-06T05:14:59.954874+0000","last_active":"2023-10-06T05:15:03.967686+0000","last_peered":"2023-10-06T05:15:03.967686+0000","last_clean":"2023-10-06T05:15:03.967686+0000","last_became_active":"2023-10-06T05:14:59.954596+0000","last_became_peered":"2023-10-06T05:14:59.954596+0000","last_unstale":"2023-10-06T05:15:03.967686+0000","last_undegraded":"2023-10-06T05:15:03.967686+0000","last_fullsized":"2023-10-06T05:15:03.967686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:12.617791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959901+0000","last_change":"2023-10-06T05:14:59.964700+0000","last_active":"2023-10-06T05:15:03.959901+0000","last_peered":"2023-10-06T05:15:03.959901+0000","last_clean":"2023-10-06T05:15:03.959901+0000","last_became_active":"2023-10-06T05:14:59.964122+0000","last_became_peered":"2023-10-06T05:14:59.964122+0000","last_unstale":"2023-10-06T05:15:03.959901+0000","last_undegraded":"2023-10-06T05:15:03.959901+0000","last_fullsized":"2023-10-06T05:15:03.959901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.174130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965374+0000","last_change":"2023-10-06T05:14:59.971088+0000","last_active":"2023-10-06T05:15:03.965374+0000","last_peered":"2023-10-06T05:15:03.965374+0000","last_clean":"2023-10-06T05:15:03.965374+0000","last_became_active":"2023-10-06T05:14:59.970833+0000","last_became_peered":"2023-10-06T05:14:59.970833+0000","last_unstale":"2023-10-06T05:15:03.965374+0000","last_undegraded":"2023-10-06T05:15:03.965374+0000","last_fullsized":"2023-10-06T05:15:03.965374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963287+0000","last_change":"2023-10-06T05:14:59.959301+0000","last_active":"2023-10-06T05:15:03.963287+0000","last_peered":"2023-10-06T05:15:03.963287+0000","last_clean":"2023-10-06T05:15:03.963287+0000","last_became_active":"2023-10-06T05:14:59.958848+0000","last_became_peered":"2023-10-06T05:14:59.958848+0000","last_unstale":"2023-10-06T05:15:03.963287+0000","last_undegraded":"2023-10-06T05:15:03.963287+0000","last_fullsized":"2023-10-06T05:15:03.963287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963428+0000","last_change":"2023-10-06T05:14:59.964849+0000","last_active":"2023-10-06T05:15:03.963428+0000","last_peered":"2023-10-06T05:15:03.963428+0000","last_clean":"2023-10-06T05:15:03.963428+0000","last_became_active":"2023-10-06T05:14:59.964525+0000","last_became_peered":"2023-10-06T05:14:59.964525+0000","last_unstale":"2023-10-06T05:15:03.963428+0000","last_undegraded":"2023-10-06T05:15:03.963428+0000","last_fullsized":"2023-10-06T05:15:03.963428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964605+0000","last_change":"2023-10-06T05:14:59.968524+0000","last_active":"2023-10-06T05:15:03.964605+0000","last_peered":"2023-10-06T05:15:03.964605+0000","last_clean":"2023-10-06T05:15:03.964605+0000","last_became_active":"2023-10-06T05:14:59.968292+0000","last_became_peered":"2023-10-06T05:14:59.968292+0000","last_unstale":"2023-10-06T05:15:03.964605+0000","last_undegraded":"2023-10-06T05:15:03.964605+0000","last_fullsized":"2023-10-06T05:15:03.964605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979056+0000","last_change":"2023-10-06T05:14:59.957743+0000","last_active":"2023-10-06T05:15:03.979056+0000","last_peered":"2023-10-06T05:15:03.979056+0000","last_clean":"2023-10-06T05:15:03.979056+0000","last_became_active":"2023-10-06T05:14:59.957472+0000","last_became_peered":"2023-10-06T05:14:59.957472+0000","last_unstale":"2023-10-06T05:15:03.979056+0000","last_undegraded":"2023-10-06T05:15:03.979056+0000","last_fullsized":"2023-10-06T05:15:03.979056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977658+0000","last_change":"2023-10-06T05:14:59.953807+0000","last_active":"2023-10-06T05:15:03.977658+0000","last_peered":"2023-10-06T05:15:03.977658+0000","last_clean":"2023-10-06T05:15:03.977658+0000","last_became_active":"2023-10-06T05:14:59.953532+0000","last_became_peered":"2023-10-06T05:14:59.953532+0000","last_unstale":"2023-10-06T05:15:03.977658+0000","last_undegraded":"2023-10-06T05:15:03.977658+0000","last_fullsized":"2023-10-06T05:15:03.977658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:39.399842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962861+0000","last_change":"2023-10-06T05:14:59.968101+0000","last_active":"2023-10-06T05:15:03.962861+0000","last_peered":"2023-10-06T05:15:03.962861+0000","last_clean":"2023-10-06T05:15:03.962861+0000","last_became_active":"2023-10-06T05:14:59.967317+0000","last_became_peered":"2023-10-06T05:14:59.967317+0000","last_unstale":"2023-10-06T05:15:03.962861+0000","last_undegraded":"2023-10-06T05:15:03.962861+0000","last_fullsized":"2023-10-06T05:15:03.962861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987821+0000","last_change":"2023-10-06T05:15:03.983811+0000","last_active":"2023-10-06T05:15:03.987821+0000","last_peered":"2023-10-06T05:15:03.987821+0000","last_clean":"2023-10-06T05:15:03.987821+0000","last_became_active":"2023-10-06T05:15:03.983468+0000","last_became_peered":"2023-10-06T05:15:03.983468+0000","last_unstale":"2023-10-06T05:15:03.987821+0000","last_undegraded":"2023-10-06T05:15:03.987821+0000","last_fullsized":"2023-10-06T05:15:03.987821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:02.315455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961143+0000","last_change":"2023-10-06T05:14:58.958395+0000","last_active":"2023-10-06T05:15:03.961143+0000","last_peered":"2023-10-06T05:15:03.961143+0000","last_clean":"2023-10-06T05:15:03.961143+0000","last_became_active":"2023-10-06T05:14:58.957979+0000","last_became_peered":"2023-10-06T05:14:58.957979+0000","last_unstale":"2023-10-06T05:15:03.961143+0000","last_undegraded":"2023-10-06T05:15:03.961143+0000","last_fullsized":"2023-10-06T05:15:03.961143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965650+0000","last_change":"2023-10-06T05:14:59.960654+0000","last_active":"2023-10-06T05:15:03.965650+0000","last_peered":"2023-10-06T05:15:03.965650+0000","last_clean":"2023-10-06T05:15:03.965650+0000","last_became_active":"2023-10-06T05:14:59.960359+0000","last_became_peered":"2023-10-06T05:14:59.960359+0000","last_unstale":"2023-10-06T05:15:03.965650+0000","last_undegraded":"2023-10-06T05:15:03.965650+0000","last_fullsized":"2023-10-06T05:15:03.965650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987166+0000","last_change":"2023-10-06T05:15:03.980797+0000","last_active":"2023-10-06T05:15:03.987166+0000","last_peered":"2023-10-06T05:15:03.987166+0000","last_clean":"2023-10-06T05:15:03.987166+0000","last_became_active":"2023-10-06T05:15:03.980441+0000","last_became_peered":"2023-10-06T05:15:03.980441+0000","last_unstale":"2023-10-06T05:15:03.987166+0000","last_undegraded":"2023-10-06T05:15:03.987166+0000","last_fullsized":"2023-10-06T05:15:03.987166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:59.153783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972385+0000","last_change":"2023-10-06T05:14:58.967470+0000","last_active":"2023-10-06T05:15:03.972385+0000","last_peered":"2023-10-06T05:15:03.972385+0000","last_clean":"2023-10-06T05:15:03.972385+0000","last_became_active":"2023-10-06T05:14:58.967143+0000","last_became_peered":"2023-10-06T05:14:58.967143+0000","last_unstale":"2023-10-06T05:15:03.972385+0000","last_undegraded":"2023-10-06T05:15:03.972385+0000","last_fullsized":"2023-10-06T05:15:03.972385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970567+0000","last_change":"2023-10-06T05:14:59.961217+0000","last_active":"2023-10-06T05:15:03.970567+0000","last_peered":"2023-10-06T05:15:03.970567+0000","last_clean":"2023-10-06T05:15:03.970567+0000","last_became_active":"2023-10-06T05:14:59.960909+0000","last_became_peered":"2023-10-06T05:14:59.960909+0000","last_unstale":"2023-10-06T05:15:03.970567+0000","last_undegraded":"2023-10-06T05:15:03.970567+0000","last_fullsized":"2023-10-06T05:15:03.970567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995384+0000","last_change":"2023-10-06T05:15:03.993702+0000","last_active":"2023-10-06T05:15:03.995384+0000","last_peered":"2023-10-06T05:15:03.995384+0000","last_clean":"2023-10-06T05:15:03.995384+0000","last_became_active":"2023-10-06T05:15:03.993170+0000","last_became_peered":"2023-10-06T05:15:03.993170+0000","last_unstale":"2023-10-06T05:15:03.995384+0000","last_undegraded":"2023-10-06T05:15:03.995384+0000","last_fullsized":"2023-10-06T05:15:03.995384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964634+0000","last_change":"2023-10-06T05:14:58.964287+0000","last_active":"2023-10-06T05:15:03.964634+0000","last_peered":"2023-10-06T05:15:03.964634+0000","last_clean":"2023-10-06T05:15:03.964634+0000","last_became_active":"2023-10-06T05:14:58.963981+0000","last_became_peered":"2023-10-06T05:14:58.963981+0000","last_unstale":"2023-10-06T05:15:03.964634+0000","last_undegraded":"2023-10-06T05:15:03.964634+0000","last_fullsized":"2023-10-06T05:15:03.964634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975859+0000","last_change":"2023-10-06T05:14:59.964080+0000","last_active":"2023-10-06T05:15:03.975859+0000","last_peered":"2023-10-06T05:15:03.975859+0000","last_clean":"2023-10-06T05:15:03.975859+0000","last_became_active":"2023-10-06T05:14:59.963849+0000","last_became_peered":"2023-10-06T05:14:59.963849+0000","last_unstale":"2023-10-06T05:15:03.975859+0000","last_undegraded":"2023-10-06T05:15:03.975859+0000","last_fullsized":"2023-10-06T05:15:03.975859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:13.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.988935+0000","last_change":"2023-10-06T05:15:03.985094+0000","last_active":"2023-10-06T05:15:03.988935+0000","last_peered":"2023-10-06T05:15:03.988935+0000","last_clean":"2023-10-06T05:15:03.988935+0000","last_became_active":"2023-10-06T05:15:03.984516+0000","last_became_peered":"2023-10-06T05:15:03.984516+0000","last_unstale":"2023-10-06T05:15:03.988935+0000","last_undegraded":"2023-10-06T05:15:03.988935+0000","last_fullsized":"2023-10-06T05:15:03.988935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:09.008217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970313+0000","last_change":"2023-10-06T05:14:58.970639+0000","last_active":"2023-10-06T05:15:03.970313+0000","last_peered":"2023-10-06T05:15:03.970313+0000","last_clean":"2023-10-06T05:15:03.970313+0000","last_became_active":"2023-10-06T05:14:58.970183+0000","last_became_peered":"2023-10-06T05:14:58.970183+0000","last_unstale":"2023-10-06T05:15:03.970313+0000","last_undegraded":"2023-10-06T05:15:03.970313+0000","last_fullsized":"2023-10-06T05:15:03.970313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964542+0000","last_change":"2023-10-06T05:14:59.962878+0000","last_active":"2023-10-06T05:15:03.964542+0000","last_peered":"2023-10-06T05:15:03.964542+0000","last_clean":"2023-10-06T05:15:03.964542+0000","last_became_active":"2023-10-06T05:14:59.962557+0000","last_became_peered":"2023-10-06T05:14:59.962557+0000","last_unstale":"2023-10-06T05:15:03.964542+0000","last_undegraded":"2023-10-06T05:15:03.964542+0000","last_fullsized":"2023-10-06T05:15:03.964542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.984338+0000","last_change":"2023-10-06T05:15:03.979469+0000","last_active":"2023-10-06T05:15:03.984338+0000","last_peered":"2023-10-06T05:15:03.984338+0000","last_clean":"2023-10-06T05:15:03.984338+0000","last_became_active":"2023-10-06T05:15:03.979016+0000","last_became_peered":"2023-10-06T05:15:03.979016+0000","last_unstale":"2023-10-06T05:15:03.984338+0000","last_undegraded":"2023-10-06T05:15:03.984338+0000","last_fullsized":"2023-10-06T05:15:03.984338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963303+0000","last_change":"2023-10-06T05:14:58.953556+0000","last_active":"2023-10-06T05:15:03.963303+0000","last_peered":"2023-10-06T05:15:03.963303+0000","last_clean":"2023-10-06T05:15:03.963303+0000","last_became_active":"2023-10-06T05:14:58.953280+0000","last_became_peered":"2023-10-06T05:14:58.953280+0000","last_unstale":"2023-10-06T05:15:03.963303+0000","last_undegraded":"2023-10-06T05:15:03.963303+0000","last_fullsized":"2023-10-06T05:15:03.963303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963541+0000","last_change":"2023-10-06T05:14:59.969909+0000","last_active":"2023-10-06T05:15:03.963541+0000","last_peered":"2023-10-06T05:15:03.963541+0000","last_clean":"2023-10-06T05:15:03.963541+0000","last_became_active":"2023-10-06T05:14:59.967612+0000","last_became_peered":"2023-10-06T05:14:59.967612+0000","last_unstale":"2023-10-06T05:15:03.963541+0000","last_undegraded":"2023-10-06T05:15:03.963541+0000","last_fullsized":"2023-10-06T05:15:03.963541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994156+0000","last_change":"2023-10-06T05:15:03.984187+0000","last_active":"2023-10-06T05:15:03.994156+0000","last_peered":"2023-10-06T05:15:03.994156+0000","last_clean":"2023-10-06T05:15:03.994156+0000","last_became_active":"2023-10-06T05:15:03.983778+0000","last_became_peered":"2023-10-06T05:15:03.983778+0000","last_unstale":"2023-10-06T05:15:03.994156+0000","last_undegraded":"2023-10-06T05:15:03.994156+0000","last_fullsized":"2023-10-06T05:15:03.994156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:24.403669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967440+0000","last_change":"2023-10-06T05:14:58.971420+0000","last_active":"2023-10-06T05:15:03.967440+0000","last_peered":"2023-10-06T05:15:03.967440+0000","last_clean":"2023-10-06T05:15:03.967440+0000","last_became_active":"2023-10-06T05:14:58.971181+0000","last_became_peered":"2023-10-06T05:14:58.971181+0000","last_unstale":"2023-10-06T05:15:03.967440+0000","last_undegraded":"2023-10-06T05:15:03.967440+0000","last_fullsized":"2023-10-06T05:15:03.967440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973790+0000","last_change":"2023-10-06T05:14:59.966072+0000","last_active":"2023-10-06T05:15:03.973790+0000","last_peered":"2023-10-06T05:15:03.973790+0000","last_clean":"2023-10-06T05:15:03.973790+0000","last_became_active":"2023-10-06T05:14:59.965368+0000","last_became_peered":"2023-10-06T05:14:59.965368+0000","last_unstale":"2023-10-06T05:15:03.973790+0000","last_undegraded":"2023-10-06T05:15:03.973790+0000","last_fullsized":"2023-10-06T05:15:03.973790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990856+0000","last_change":"2023-10-06T05:15:03.989524+0000","last_active":"2023-10-06T05:15:03.990856+0000","last_peered":"2023-10-06T05:15:03.990856+0000","last_clean":"2023-10-06T05:15:03.990856+0000","last_became_active":"2023-10-06T05:15:03.982518+0000","last_became_peered":"2023-10-06T05:15:03.982518+0000","last_unstale":"2023-10-06T05:15:03.990856+0000","last_undegraded":"2023-10-06T05:15:03.990856+0000","last_fullsized":"2023-10-06T05:15:03.990856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T05:15:10.388156+0000","last_change":"2023-10-06T05:14:58.956195+0000","last_active":"2023-10-06T05:15:10.388156+0000","last_peered":"2023-10-06T05:15:10.388156+0000","last_clean":"2023-10-06T05:15:10.388156+0000","last_became_active":"2023-10-06T05:14:58.955906+0000","last_became_peered":"2023-10-06T05:14:58.955906+0000","last_unstale":"2023-10-06T05:15:10.388156+0000","last_undegraded":"2023-10-06T05:15:10.388156+0000","last_fullsized":"2023-10-06T05:15:10.388156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.952535+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.952313+0000","last_became_peered":"2023-10-06T05:14:59.952313+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987354+0000","last_change":"2023-10-06T05:15:03.982183+0000","last_active":"2023-10-06T05:15:03.987354+0000","last_peered":"2023-10-06T05:15:03.987354+0000","last_clean":"2023-10-06T05:15:03.987354+0000","last_became_active":"2023-10-06T05:15:03.981849+0000","last_became_peered":"2023-10-06T05:15:03.981849+0000","last_unstale":"2023-10-06T05:15:03.987354+0000","last_undegraded":"2023-10-06T05:15:03.987354+0000","last_fullsized":"2023-10-06T05:15:03.987354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965015+0000","last_change":"2023-10-06T05:14:58.959548+0000","last_active":"2023-10-06T05:15:03.965015+0000","last_peered":"2023-10-06T05:15:03.965015+0000","last_clean":"2023-10-06T05:15:03.965015+0000","last_became_active":"2023-10-06T05:14:58.959270+0000","last_became_peered":"2023-10-06T05:14:58.959270+0000","last_unstale":"2023-10-06T05:15:03.965015+0000","last_undegraded":"2023-10-06T05:15:03.965015+0000","last_fullsized":"2023-10-06T05:15:03.965015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:02.018618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968411+0000","last_change":"2023-10-06T05:14:59.973960+0000","last_active":"2023-10-06T05:15:03.968411+0000","last_peered":"2023-10-06T05:15:03.968411+0000","last_clean":"2023-10-06T05:15:03.968411+0000","last_became_active":"2023-10-06T05:14:59.973622+0000","last_became_peered":"2023-10-06T05:14:59.973622+0000","last_unstale":"2023-10-06T05:15:03.968411+0000","last_undegraded":"2023-10-06T05:15:03.968411+0000","last_fullsized":"2023-10-06T05:15:03.968411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994166+0000","last_change":"2023-10-06T05:15:03.985900+0000","last_active":"2023-10-06T05:15:03.994166+0000","last_peered":"2023-10-06T05:15:03.994166+0000","last_clean":"2023-10-06T05:15:03.994166+0000","last_became_active":"2023-10-06T05:15:03.985484+0000","last_became_peered":"2023-10-06T05:15:03.985484+0000","last_unstale":"2023-10-06T05:15:03.994166+0000","last_undegraded":"2023-10-06T05:15:03.994166+0000","last_fullsized":"2023-10-06T05:15:03.994166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:50.069708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963167+0000","last_change":"2023-10-06T05:14:58.968185+0000","last_active":"2023-10-06T05:15:03.963167+0000","last_peered":"2023-10-06T05:15:03.963167+0000","last_clean":"2023-10-06T05:15:03.963167+0000","last_became_active":"2023-10-06T05:14:58.967895+0000","last_became_peered":"2023-10-06T05:14:58.967895+0000","last_unstale":"2023-10-06T05:15:03.963167+0000","last_undegraded":"2023-10-06T05:15:03.963167+0000","last_fullsized":"2023-10-06T05:15:03.963167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962815+0000","last_change":"2023-10-06T05:14:59.967384+0000","last_active":"2023-10-06T05:15:03.962815+0000","last_peered":"2023-10-06T05:15:03.962815+0000","last_clean":"2023-10-06T05:15:03.962815+0000","last_became_active":"2023-10-06T05:14:59.967131+0000","last_became_peered":"2023-10-06T05:14:59.967131+0000","last_unstale":"2023-10-06T05:15:03.962815+0000","last_undegraded":"2023-10-06T05:15:03.962815+0000","last_fullsized":"2023-10-06T05:15:03.962815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994086+0000","last_change":"2023-10-06T05:15:03.985192+0000","last_active":"2023-10-06T05:15:03.994086+0000","last_peered":"2023-10-06T05:15:03.994086+0000","last_clean":"2023-10-06T05:15:03.994086+0000","last_became_active":"2023-10-06T05:15:03.984600+0000","last_became_peered":"2023-10-06T05:15:03.984600+0000","last_unstale":"2023-10-06T05:15:03.994086+0000","last_undegraded":"2023-10-06T05:15:03.994086+0000","last_fullsized":"2023-10-06T05:15:03.994086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964960+0000","last_change":"2023-10-06T05:14:58.959794+0000","last_active":"2023-10-06T05:15:03.964960+0000","last_peered":"2023-10-06T05:15:03.964960+0000","last_clean":"2023-10-06T05:15:03.964960+0000","last_became_active":"2023-10-06T05:14:58.959493+0000","last_became_peered":"2023-10-06T05:14:58.959493+0000","last_unstale":"2023-10-06T05:15:03.964960+0000","last_undegraded":"2023-10-06T05:15:03.964960+0000","last_fullsized":"2023-10-06T05:15:03.964960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:10.366367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970679+0000","last_change":"2023-10-06T05:14:59.964236+0000","last_active":"2023-10-06T05:15:03.970679+0000","last_peered":"2023-10-06T05:15:03.970679+0000","last_clean":"2023-10-06T05:15:03.970679+0000","last_became_active":"2023-10-06T05:14:59.963902+0000","last_became_peered":"2023-10-06T05:14:59.963902+0000","last_unstale":"2023-10-06T05:15:03.970679+0000","last_undegraded":"2023-10-06T05:15:03.970679+0000","last_fullsized":"2023-10-06T05:15:03.970679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987391+0000","last_change":"2023-10-06T05:15:03.982595+0000","last_active":"2023-10-06T05:15:03.987391+0000","last_peered":"2023-10-06T05:15:03.987391+0000","last_clean":"2023-10-06T05:15:03.987391+0000","last_became_active":"2023-10-06T05:15:03.982200+0000","last_became_peered":"2023-10-06T05:15:03.982200+0000","last_unstale":"2023-10-06T05:15:03.987391+0000","last_undegraded":"2023-10-06T05:15:03.987391+0000","last_fullsized":"2023-10-06T05:15:03.987391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:20.022766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974038+0000","last_change":"2023-10-06T05:14:58.961380+0000","last_active":"2023-10-06T05:15:03.974038+0000","last_peered":"2023-10-06T05:15:03.974038+0000","last_clean":"2023-10-06T05:15:03.974038+0000","last_became_active":"2023-10-06T05:14:58.961038+0000","last_became_peered":"2023-10-06T05:14:58.961038+0000","last_unstale":"2023-10-06T05:15:03.974038+0000","last_undegraded":"2023-10-06T05:15:03.974038+0000","last_fullsized":"2023-10-06T05:15:03.974038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964625+0000","last_change":"2023-10-06T05:14:59.967979+0000","last_active":"2023-10-06T05:15:03.964625+0000","last_peered":"2023-10-06T05:15:03.964625+0000","last_clean":"2023-10-06T05:15:03.964625+0000","last_became_active":"2023-10-06T05:14:59.964930+0000","last_became_peered":"2023-10-06T05:14:59.964930+0000","last_unstale":"2023-10-06T05:15:03.964625+0000","last_undegraded":"2023-10-06T05:15:03.964625+0000","last_fullsized":"2023-10-06T05:15:03.964625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.981489+0000","last_change":"2023-10-06T05:15:03.978144+0000","last_active":"2023-10-06T05:15:03.981489+0000","last_peered":"2023-10-06T05:15:03.981489+0000","last_clean":"2023-10-06T05:15:03.981489+0000","last_became_active":"2023-10-06T05:15:03.977784+0000","last_became_peered":"2023-10-06T05:15:03.977784+0000","last_unstale":"2023-10-06T05:15:03.981489+0000","last_undegraded":"2023-10-06T05:15:03.981489+0000","last_fullsized":"2023-10-06T05:15:03.981489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967621+0000","last_change":"2023-10-06T05:14:58.959617+0000","last_active":"2023-10-06T05:15:03.967621+0000","last_peered":"2023-10-06T05:15:03.967621+0000","last_clean":"2023-10-06T05:15:03.967621+0000","last_became_active":"2023-10-06T05:14:58.959350+0000","last_became_peered":"2023-10-06T05:14:58.959350+0000","last_unstale":"2023-10-06T05:15:03.967621+0000","last_undegraded":"2023-10-06T05:15:03.967621+0000","last_fullsized":"2023-10-06T05:15:03.967621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:12.695218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961420+0000","last_change":"2023-10-06T05:14:59.967832+0000","last_active":"2023-10-06T05:15:03.961420+0000","last_peered":"2023-10-06T05:15:03.961420+0000","last_clean":"2023-10-06T05:15:03.961420+0000","last_became_active":"2023-10-06T05:14:59.967192+0000","last_became_peered":"2023-10-06T05:14:59.967192+0000","last_unstale":"2023-10-06T05:15:03.961420+0000","last_undegraded":"2023-10-06T05:15:03.961420+0000","last_fullsized":"2023-10-06T05:15:03.961420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995012+0000","last_change":"2023-10-06T05:15:03.993765+0000","last_active":"2023-10-06T05:15:03.995012+0000","last_peered":"2023-10-06T05:15:03.995012+0000","last_clean":"2023-10-06T05:15:03.995012+0000","last_became_active":"2023-10-06T05:15:03.993371+0000","last_became_peered":"2023-10-06T05:15:03.993371+0000","last_unstale":"2023-10-06T05:15:03.995012+0000","last_undegraded":"2023-10-06T05:15:03.995012+0000","last_fullsized":"2023-10-06T05:15:03.995012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:00.861115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967256+0000","last_change":"2023-10-06T05:14:58.962483+0000","last_active":"2023-10-06T05:15:03.967256+0000","last_peered":"2023-10-06T05:15:03.967256+0000","last_clean":"2023-10-06T05:15:03.967256+0000","last_became_active":"2023-10-06T05:14:58.962203+0000","last_became_peered":"2023-10-06T05:14:58.962203+0000","last_unstale":"2023-10-06T05:15:03.967256+0000","last_undegraded":"2023-10-06T05:15:03.967256+0000","last_fullsized":"2023-10-06T05:15:03.967256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:34.275033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960934+0000","last_change":"2023-10-06T05:14:59.956925+0000","last_active":"2023-10-06T05:15:03.960934+0000","last_peered":"2023-10-06T05:15:03.960934+0000","last_clean":"2023-10-06T05:15:03.960934+0000","last_became_active":"2023-10-06T05:14:59.956653+0000","last_became_peered":"2023-10-06T05:14:59.956653+0000","last_unstale":"2023-10-06T05:15:03.960934+0000","last_undegraded":"2023-10-06T05:15:03.960934+0000","last_fullsized":"2023-10-06T05:15:03.960934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985561+0000","last_change":"2023-10-06T05:15:03.983559+0000","last_active":"2023-10-06T05:15:03.985561+0000","last_peered":"2023-10-06T05:15:03.985561+0000","last_clean":"2023-10-06T05:15:03.985561+0000","last_became_active":"2023-10-06T05:15:03.983123+0000","last_became_peered":"2023-10-06T05:15:03.983123+0000","last_unstale":"2023-10-06T05:15:03.985561+0000","last_undegraded":"2023-10-06T05:15:03.985561+0000","last_fullsized":"2023-10-06T05:15:03.985561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:50.666262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967619+0000","last_change":"2023-10-06T05:14:58.956636+0000","last_active":"2023-10-06T05:15:03.967619+0000","last_peered":"2023-10-06T05:15:03.967619+0000","last_clean":"2023-10-06T05:15:03.967619+0000","last_became_active":"2023-10-06T05:14:58.956436+0000","last_became_peered":"2023-10-06T05:14:58.956436+0000","last_unstale":"2023-10-06T05:15:03.967619+0000","last_undegraded":"2023-10-06T05:15:03.967619+0000","last_fullsized":"2023-10-06T05:15:03.967619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:45.319345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975492+0000","last_change":"2023-10-06T05:14:59.962010+0000","last_active":"2023-10-06T05:15:03.975492+0000","last_peered":"2023-10-06T05:15:03.975492+0000","last_clean":"2023-10-06T05:15:03.975492+0000","last_became_active":"2023-10-06T05:14:59.961837+0000","last_became_peered":"2023-10-06T05:14:59.961837+0000","last_unstale":"2023-10-06T05:15:03.975492+0000","last_undegraded":"2023-10-06T05:15:03.975492+0000","last_fullsized":"2023-10-06T05:15:03.975492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:46.326581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985590+0000","last_change":"2023-10-06T05:15:03.983998+0000","last_active":"2023-10-06T05:15:03.985590+0000","last_peered":"2023-10-06T05:15:03.985590+0000","last_clean":"2023-10-06T05:15:03.985590+0000","last_became_active":"2023-10-06T05:15:03.983598+0000","last_became_peered":"2023-10-06T05:15:03.983598+0000","last_unstale":"2023-10-06T05:15:03.985590+0000","last_undegraded":"2023-10-06T05:15:03.985590+0000","last_fullsized":"2023-10-06T05:15:03.985590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:01.199458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961576+0000","last_change":"2023-10-06T05:14:58.969438+0000","last_active":"2023-10-06T05:15:03.961576+0000","last_peered":"2023-10-06T05:15:03.961576+0000","last_clean":"2023-10-06T05:15:03.961576+0000","last_became_active":"2023-10-06T05:14:58.969214+0000","last_became_peered":"2023-10-06T05:14:58.969214+0000","last_unstale":"2023-10-06T05:15:03.961576+0000","last_undegraded":"2023-10-06T05:15:03.961576+0000","last_fullsized":"2023-10-06T05:15:03.961576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966944+0000","last_change":"2023-10-06T05:14:59.958303+0000","last_active":"2023-10-06T05:15:03.966944+0000","last_peered":"2023-10-06T05:15:03.966944+0000","last_clean":"2023-10-06T05:15:03.966944+0000","last_became_active":"2023-10-06T05:14:59.957997+0000","last_became_peered":"2023-10-06T05:14:59.957997+0000","last_unstale":"2023-10-06T05:15:03.966944+0000","last_undegraded":"2023-10-06T05:15:03.966944+0000","last_fullsized":"2023-10-06T05:15:03.966944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.982560+0000","last_change":"2023-10-06T05:15:03.976634+0000","last_active":"2023-10-06T05:15:03.982560+0000","last_peered":"2023-10-06T05:15:03.982560+0000","last_clean":"2023-10-06T05:15:03.982560+0000","last_became_active":"2023-10-06T05:15:03.976257+0000","last_became_peered":"2023-10-06T05:15:03.976257+0000","last_unstale":"2023-10-06T05:15:03.982560+0000","last_undegraded":"2023-10-06T05:15:03.982560+0000","last_fullsized":"2023-10-06T05:15:03.982560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:47.906687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964042+0000","last_change":"2023-10-06T05:14:58.969408+0000","last_active":"2023-10-06T05:15:03.964042+0000","last_peered":"2023-10-06T05:15:03.964042+0000","last_clean":"2023-10-06T05:15:03.964042+0000","last_became_active":"2023-10-06T05:14:58.969070+0000","last_became_peered":"2023-10-06T05:14:58.969070+0000","last_unstale":"2023-10-06T05:15:03.964042+0000","last_undegraded":"2023-10-06T05:15:03.964042+0000","last_fullsized":"2023-10-06T05:15:03.964042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959678+0000","last_change":"2023-10-06T05:14:59.969503+0000","last_active":"2023-10-06T05:15:03.959678+0000","last_peered":"2023-10-06T05:15:03.959678+0000","last_clean":"2023-10-06T05:15:03.959678+0000","last_became_active":"2023-10-06T05:14:59.969109+0000","last_became_peered":"2023-10-06T05:14:59.969109+0000","last_unstale":"2023-10-06T05:15:03.959678+0000","last_undegraded":"2023-10-06T05:15:03.959678+0000","last_fullsized":"2023-10-06T05:15:03.959678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:59.890813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.989045+0000","last_change":"2023-10-06T05:15:03.982573+0000","last_active":"2023-10-06T05:15:03.989045+0000","last_peered":"2023-10-06T05:15:03.989045+0000","last_clean":"2023-10-06T05:15:03.989045+0000","last_became_active":"2023-10-06T05:15:03.982173+0000","last_became_peered":"2023-10-06T05:15:03.982173+0000","last_unstale":"2023-10-06T05:15:03.989045+0000","last_undegraded":"2023-10-06T05:15:03.989045+0000","last_fullsized":"2023-10-06T05:15:03.989045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.921378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975586+0000","last_change":"2023-10-06T05:14:58.962379+0000","last_active":"2023-10-06T05:15:03.975586+0000","last_peered":"2023-10-06T05:15:03.975586+0000","last_clean":"2023-10-06T05:15:03.975586+0000","last_became_active":"2023-10-06T05:14:58.962093+0000","last_became_peered":"2023-10-06T05:14:58.962093+0000","last_unstale":"2023-10-06T05:15:03.975586+0000","last_undegraded":"2023-10-06T05:15:03.975586+0000","last_fullsized":"2023-10-06T05:15:03.975586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960163+0000","last_change":"2023-10-06T05:14:59.951502+0000","last_active":"2023-10-06T05:15:03.960163+0000","last_peered":"2023-10-06T05:15:03.960163+0000","last_clean":"2023-10-06T05:15:03.960163+0000","last_became_active":"2023-10-06T05:14:59.951197+0000","last_became_peered":"2023-10-06T05:14:59.951197+0000","last_unstale":"2023-10-06T05:15:03.960163+0000","last_undegraded":"2023-10-06T05:15:03.960163+0000","last_fullsized":"2023-10-06T05:15:03.960163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978919+0000","last_change":"2023-10-06T05:15:03.977325+0000","last_active":"2023-10-06T05:15:03.978919+0000","last_peered":"2023-10-06T05:15:03.978919+0000","last_clean":"2023-10-06T05:15:03.978919+0000","last_became_active":"2023-10-06T05:15:03.976844+0000","last_became_peered":"2023-10-06T05:15:03.976844+0000","last_unstale":"2023-10-06T05:15:03.978919+0000","last_undegraded":"2023-10-06T05:15:03.978919+0000","last_fullsized":"2023-10-06T05:15:03.978919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:47.153239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960726+0000","last_change":"2023-10-06T05:14:58.958227+0000","last_active":"2023-10-06T05:15:03.960726+0000","last_peered":"2023-10-06T05:15:03.960726+0000","last_clean":"2023-10-06T05:15:03.960726+0000","last_became_active":"2023-10-06T05:14:58.957924+0000","last_became_peered":"2023-10-06T05:14:58.957924+0000","last_unstale":"2023-10-06T05:15:03.960726+0000","last_undegraded":"2023-10-06T05:15:03.960726+0000","last_fullsized":"2023-10-06T05:15:03.960726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961096+0000","last_change":"2023-10-06T05:14:59.961306+0000","last_active":"2023-10-06T05:15:03.961096+0000","last_peered":"2023-10-06T05:15:03.961096+0000","last_clean":"2023-10-06T05:15:03.961096+0000","last_became_active":"2023-10-06T05:14:59.961025+0000","last_became_peered":"2023-10-06T05:14:59.961025+0000","last_unstale":"2023-10-06T05:15:03.961096+0000","last_undegraded":"2023-10-06T05:15:03.961096+0000","last_fullsized":"2023-10-06T05:15:03.961096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987866+0000","last_change":"2023-10-06T05:15:03.983777+0000","last_active":"2023-10-06T05:15:03.987866+0000","last_peered":"2023-10-06T05:15:03.987866+0000","last_clean":"2023-10-06T05:15:03.987866+0000","last_became_active":"2023-10-06T05:15:03.983380+0000","last_became_peered":"2023-10-06T05:15:03.983380+0000","last_unstale":"2023-10-06T05:15:03.987866+0000","last_undegraded":"2023-10-06T05:15:03.987866+0000","last_fullsized":"2023-10-06T05:15:03.987866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:21.551659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967787+0000","last_change":"2023-10-06T05:14:58.966032+0000","last_active":"2023-10-06T05:15:03.967787+0000","last_peered":"2023-10-06T05:15:03.967787+0000","last_clean":"2023-10-06T05:15:03.967787+0000","last_became_active":"2023-10-06T05:14:58.965697+0000","last_became_peered":"2023-10-06T05:14:58.965697+0000","last_unstale":"2023-10-06T05:15:03.967787+0000","last_undegraded":"2023-10-06T05:15:03.967787+0000","last_fullsized":"2023-10-06T05:15:03.967787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979242+0000","last_change":"2023-10-06T05:14:59.966425+0000","last_active":"2023-10-06T05:15:03.979242+0000","last_peered":"2023-10-06T05:15:03.979242+0000","last_clean":"2023-10-06T05:15:03.979242+0000","last_became_active":"2023-10-06T05:14:59.966122+0000","last_became_peered":"2023-10-06T05:14:59.966122+0000","last_unstale":"2023-10-06T05:15:03.979242+0000","last_undegraded":"2023-10-06T05:15:03.979242+0000","last_fullsized":"2023-10-06T05:15:03.979242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.983132+0000","last_change":"2023-10-06T05:15:03.979017+0000","last_active":"2023-10-06T05:15:03.983132+0000","last_peered":"2023-10-06T05:15:03.983132+0000","last_clean":"2023-10-06T05:15:03.983132+0000","last_became_active":"2023-10-06T05:15:03.978644+0000","last_became_peered":"2023-10-06T05:15:03.978644+0000","last_unstale":"2023-10-06T05:15:03.983132+0000","last_undegraded":"2023-10-06T05:15:03.983132+0000","last_fullsized":"2023-10-06T05:15:03.983132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973620+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:15:03.973620+0000","last_peered":"2023-10-06T05:15:03.973620+0000","last_clean":"2023-10-06T05:15:03.973620+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:15:03.973620+0000","last_undegraded":"2023-10-06T05:15:03.973620+0000","last_fullsized":"2023-10-06T05:15:03.973620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962773+0000","last_change":"2023-10-06T05:14:58.969546+0000","last_active":"2023-10-06T05:15:03.962773+0000","last_peered":"2023-10-06T05:15:03.962773+0000","last_clean":"2023-10-06T05:15:03.962773+0000","last_became_active":"2023-10-06T05:14:58.969281+0000","last_became_peered":"2023-10-06T05:14:58.969281+0000","last_unstale":"2023-10-06T05:15:03.962773+0000","last_undegraded":"2023-10-06T05:15:03.962773+0000","last_fullsized":"2023-10-06T05:15:03.962773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973704+0000","last_change":"2023-10-06T05:14:59.965859+0000","last_active":"2023-10-06T05:15:03.973704+0000","last_peered":"2023-10-06T05:15:03.973704+0000","last_clean":"2023-10-06T05:15:03.973704+0000","last_became_active":"2023-10-06T05:14:59.964784+0000","last_became_peered":"2023-10-06T05:14:59.964784+0000","last_unstale":"2023-10-06T05:15:03.973704+0000","last_undegraded":"2023-10-06T05:15:03.973704+0000","last_fullsized":"2023-10-06T05:15:03.973704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985105+0000","last_change":"2023-10-06T05:15:03.982375+0000","last_active":"2023-10-06T05:15:03.985105+0000","last_peered":"2023-10-06T05:15:03.985105+0000","last_clean":"2023-10-06T05:15:03.985105+0000","last_became_active":"2023-10-06T05:15:03.981983+0000","last_became_peered":"2023-10-06T05:15:03.981983+0000","last_unstale":"2023-10-06T05:15:03.985105+0000","last_undegraded":"2023-10-06T05:15:03.985105+0000","last_fullsized":"2023-10-06T05:15:03.985105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962357+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:15:03.962357+0000","last_peered":"2023-10-06T05:15:03.962357+0000","last_clean":"2023-10-06T05:15:03.962357+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:15:03.962357+0000","last_undegraded":"2023-10-06T05:15:03.962357+0000","last_fullsized":"2023-10-06T05:15:03.962357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959855+0000","last_change":"2023-10-06T05:14:58.966374+0000","last_active":"2023-10-06T05:15:03.959855+0000","last_peered":"2023-10-06T05:15:03.959855+0000","last_clean":"2023-10-06T05:15:03.959855+0000","last_became_active":"2023-10-06T05:14:58.966124+0000","last_became_peered":"2023-10-06T05:14:58.966124+0000","last_unstale":"2023-10-06T05:15:03.959855+0000","last_undegraded":"2023-10-06T05:15:03.959855+0000","last_fullsized":"2023-10-06T05:15:03.959855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973548+0000","last_change":"2023-10-06T05:14:59.953791+0000","last_active":"2023-10-06T05:15:03.973548+0000","last_peered":"2023-10-06T05:15:03.973548+0000","last_clean":"2023-10-06T05:15:03.973548+0000","last_became_active":"2023-10-06T05:14:59.953491+0000","last_became_peered":"2023-10-06T05:14:59.953491+0000","last_unstale":"2023-10-06T05:15:03.973548+0000","last_undegraded":"2023-10-06T05:15:03.973548+0000","last_fullsized":"2023-10-06T05:15:03.973548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994360+0000","last_change":"2023-10-06T05:15:03.993204+0000","last_active":"2023-10-06T05:15:03.994360+0000","last_peered":"2023-10-06T05:15:03.994360+0000","last_clean":"2023-10-06T05:15:03.994360+0000","last_became_active":"2023-10-06T05:15:03.992794+0000","last_became_peered":"2023-10-06T05:15:03.992794+0000","last_unstale":"2023-10-06T05:15:03.994360+0000","last_undegraded":"2023-10-06T05:15:03.994360+0000","last_fullsized":"2023-10-06T05:15:03.994360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:53.385235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967975+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:15:03.967975+0000","last_peered":"2023-10-06T05:15:03.967975+0000","last_clean":"2023-10-06T05:15:03.967975+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:15:03.967975+0000","last_undegraded":"2023-10-06T05:15:03.967975+0000","last_fullsized":"2023-10-06T05:15:03.967975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972830+0000","last_change":"2023-10-06T05:14:58.954417+0000","last_active":"2023-10-06T05:15:03.972830+0000","last_peered":"2023-10-06T05:15:03.972830+0000","last_clean":"2023-10-06T05:15:03.972830+0000","last_became_active":"2023-10-06T05:14:58.954052+0000","last_became_peered":"2023-10-06T05:14:58.954052+0000","last_unstale":"2023-10-06T05:15:03.972830+0000","last_undegraded":"2023-10-06T05:15:03.972830+0000","last_fullsized":"2023-10-06T05:15:03.972830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979150+0000","last_change":"2023-10-06T05:14:59.963695+0000","last_active":"2023-10-06T05:15:03.979150+0000","last_peered":"2023-10-06T05:15:03.979150+0000","last_clean":"2023-10-06T05:15:03.979150+0000","last_became_active":"2023-10-06T05:14:59.963426+0000","last_became_peered":"2023-10-06T05:14:59.963426+0000","last_unstale":"2023-10-06T05:15:03.979150+0000","last_undegraded":"2023-10-06T05:15:03.979150+0000","last_fullsized":"2023-10-06T05:15:03.979150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987182+0000","last_change":"2023-10-06T05:15:03.985907+0000","last_active":"2023-10-06T05:15:03.987182+0000","last_peered":"2023-10-06T05:15:03.987182+0000","last_clean":"2023-10-06T05:15:03.987182+0000","last_became_active":"2023-10-06T05:15:03.985534+0000","last_became_peered":"2023-10-06T05:15:03.985534+0000","last_unstale":"2023-10-06T05:15:03.987182+0000","last_undegraded":"2023-10-06T05:15:03.987182+0000","last_fullsized":"2023-10-06T05:15:03.987182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961919+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:15:03.961919+0000","last_peered":"2023-10-06T05:15:03.961919+0000","last_clean":"2023-10-06T05:15:03.961919+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:15:03.961919+0000","last_undegraded":"2023-10-06T05:15:03.961919+0000","last_fullsized":"2023-10-06T05:15:03.961919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964512+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:15:03.964512+0000","last_peered":"2023-10-06T05:15:03.964512+0000","last_clean":"2023-10-06T05:15:03.964512+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:15:03.964512+0000","last_undegraded":"2023-10-06T05:15:03.964512+0000","last_fullsized":"2023-10-06T05:15:03.964512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963731+0000","last_change":"2023-10-06T05:14:58.965199+0000","last_active":"2023-10-06T05:15:03.963731+0000","last_peered":"2023-10-06T05:15:03.963731+0000","last_clean":"2023-10-06T05:15:03.963731+0000","last_became_active":"2023-10-06T05:14:58.964902+0000","last_became_peered":"2023-10-06T05:14:58.964902+0000","last_unstale":"2023-10-06T05:15:03.963731+0000","last_undegraded":"2023-10-06T05:15:03.963731+0000","last_fullsized":"2023-10-06T05:15:03.963731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975967+0000","last_change":"2023-10-06T05:14:59.953471+0000","last_active":"2023-10-06T05:15:03.975967+0000","last_peered":"2023-10-06T05:15:03.975967+0000","last_clean":"2023-10-06T05:15:03.975967+0000","last_became_active":"2023-10-06T05:14:59.953166+0000","last_became_peered":"2023-10-06T05:14:59.953166+0000","last_unstale":"2023-10-06T05:15:03.975967+0000","last_undegraded":"2023-10-06T05:15:03.975967+0000","last_fullsized":"2023-10-06T05:15:03.975967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:35.282268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987431+0000","last_change":"2023-10-06T05:15:03.982741+0000","last_active":"2023-10-06T05:15:03.987431+0000","last_peered":"2023-10-06T05:15:03.987431+0000","last_clean":"2023-10-06T05:15:03.987431+0000","last_became_active":"2023-10-06T05:15:03.982333+0000","last_became_peered":"2023-10-06T05:15:03.982333+0000","last_unstale":"2023-10-06T05:15:03.987431+0000","last_undegraded":"2023-10-06T05:15:03.987431+0000","last_fullsized":"2023-10-06T05:15:03.987431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:07.926945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964687+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:15:03.964687+0000","last_peered":"2023-10-06T05:15:03.964687+0000","last_clean":"2023-10-06T05:15:03.964687+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:15:03.964687+0000","last_undegraded":"2023-10-06T05:15:03.964687+0000","last_fullsized":"2023-10-06T05:15:03.964687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965134+0000","last_change":"2023-10-06T05:14:58.970767+0000","last_active":"2023-10-06T05:15:03.965134+0000","last_peered":"2023-10-06T05:15:03.965134+0000","last_clean":"2023-10-06T05:15:03.965134+0000","last_became_active":"2023-10-06T05:14:58.970523+0000","last_became_peered":"2023-10-06T05:14:58.970523+0000","last_unstale":"2023-10-06T05:15:03.965134+0000","last_undegraded":"2023-10-06T05:15:03.965134+0000","last_fullsized":"2023-10-06T05:15:03.965134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970560+0000","last_change":"2023-10-06T05:14:59.958130+0000","last_active":"2023-10-06T05:15:03.970560+0000","last_peered":"2023-10-06T05:15:03.970560+0000","last_clean":"2023-10-06T05:15:03.970560+0000","last_became_active":"2023-10-06T05:14:59.957826+0000","last_became_peered":"2023-10-06T05:14:59.957826+0000","last_unstale":"2023-10-06T05:15:03.970560+0000","last_undegraded":"2023-10-06T05:15:03.970560+0000","last_fullsized":"2023-10-06T05:15:03.970560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987228+0000","last_change":"2023-10-06T05:15:03.981078+0000","last_active":"2023-10-06T05:15:03.987228+0000","last_peered":"2023-10-06T05:15:03.987228+0000","last_clean":"2023-10-06T05:15:03.987228+0000","last_became_active":"2023-10-06T05:15:03.980645+0000","last_became_peered":"2023-10-06T05:15:03.980645+0000","last_unstale":"2023-10-06T05:15:03.987228+0000","last_undegraded":"2023-10-06T05:15:03.987228+0000","last_fullsized":"2023-10-06T05:15:03.987228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.847066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967525+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:15:03.967525+0000","last_peered":"2023-10-06T05:15:03.967525+0000","last_clean":"2023-10-06T05:15:03.967525+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:15:03.967525+0000","last_undegraded":"2023-10-06T05:15:03.967525+0000","last_fullsized":"2023-10-06T05:15:03.967525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973787+0000","last_change":"2023-10-06T05:14:58.954390+0000","last_active":"2023-10-06T05:15:03.973787+0000","last_peered":"2023-10-06T05:15:03.973787+0000","last_clean":"2023-10-06T05:15:03.973787+0000","last_became_active":"2023-10-06T05:14:58.954003+0000","last_became_peered":"2023-10-06T05:14:58.954003+0000","last_unstale":"2023-10-06T05:15:03.973787+0000","last_undegraded":"2023-10-06T05:15:03.973787+0000","last_fullsized":"2023-10-06T05:15:03.973787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967163+0000","last_change":"2023-10-06T05:14:59.965298+0000","last_active":"2023-10-06T05:15:03.967163+0000","last_peered":"2023-10-06T05:15:03.967163+0000","last_clean":"2023-10-06T05:15:03.967163+0000","last_became_active":"2023-10-06T05:14:59.964967+0000","last_became_peered":"2023-10-06T05:14:59.964967+0000","last_unstale":"2023-10-06T05:15:03.967163+0000","last_undegraded":"2023-10-06T05:15:03.967163+0000","last_fullsized":"2023-10-06T05:15:03.967163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978732+0000","last_change":"2023-10-06T05:15:03.976855+0000","last_active":"2023-10-06T05:15:03.978732+0000","last_peered":"2023-10-06T05:15:03.978732+0000","last_clean":"2023-10-06T05:15:03.978732+0000","last_became_active":"2023-10-06T05:15:03.976461+0000","last_became_peered":"2023-10-06T05:15:03.976461+0000","last_unstale":"2023-10-06T05:15:03.978732+0000","last_undegraded":"2023-10-06T05:15:03.978732+0000","last_fullsized":"2023-10-06T05:15:03.978732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:45.979196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964732+0000","last_change":"2023-10-06T05:14:58.970171+0000","last_active":"2023-10-06T05:15:03.964732+0000","last_peered":"2023-10-06T05:15:03.964732+0000","last_clean":"2023-10-06T05:15:03.964732+0000","last_became_active":"2023-10-06T05:14:58.969914+0000","last_became_peered":"2023-10-06T05:14:58.969914+0000","last_unstale":"2023-10-06T05:15:03.964732+0000","last_undegraded":"2023-10-06T05:15:03.964732+0000","last_fullsized":"2023-10-06T05:15:03.964732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977482+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:15:03.977482+0000","last_peered":"2023-10-06T05:15:03.977482+0000","last_clean":"2023-10-06T05:15:03.977482+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:15:03.977482+0000","last_undegraded":"2023-10-06T05:15:03.977482+0000","last_fullsized":"2023-10-06T05:15:03.977482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964253+0000","last_change":"2023-10-06T05:14:59.964377+0000","last_active":"2023-10-06T05:15:03.964253+0000","last_peered":"2023-10-06T05:15:03.964253+0000","last_clean":"2023-10-06T05:15:03.964253+0000","last_became_active":"2023-10-06T05:14:59.964058+0000","last_became_peered":"2023-10-06T05:14:59.964058+0000","last_unstale":"2023-10-06T05:15:03.964253+0000","last_undegraded":"2023-10-06T05:15:03.964253+0000","last_fullsized":"2023-10-06T05:15:03.964253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994304+0000","last_change":"2023-10-06T05:15:03.991574+0000","last_active":"2023-10-06T05:15:03.994304+0000","last_peered":"2023-10-06T05:15:03.994304+0000","last_clean":"2023-10-06T05:15:03.994304+0000","last_became_active":"2023-10-06T05:15:03.991171+0000","last_became_peered":"2023-10-06T05:15:03.991171+0000","last_unstale":"2023-10-06T05:15:03.994304+0000","last_undegraded":"2023-10-06T05:15:03.994304+0000","last_fullsized":"2023-10-06T05:15:03.994304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975779+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:15:03.975779+0000","last_peered":"2023-10-06T05:15:03.975779+0000","last_clean":"2023-10-06T05:15:03.975779+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:15:03.975779+0000","last_undegraded":"2023-10-06T05:15:03.975779+0000","last_fullsized":"2023-10-06T05:15:03.975779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970480+0000","last_change":"2023-10-06T05:14:58.966696+0000","last_active":"2023-10-06T05:15:03.970480+0000","last_peered":"2023-10-06T05:15:03.970480+0000","last_clean":"2023-10-06T05:15:03.970480+0000","last_became_active":"2023-10-06T05:14:58.966398+0000","last_became_peered":"2023-10-06T05:14:58.966398+0000","last_unstale":"2023-10-06T05:15:03.970480+0000","last_undegraded":"2023-10-06T05:15:03.970480+0000","last_fullsized":"2023-10-06T05:15:03.970480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960837+0000","last_change":"2023-10-06T05:14:59.967931+0000","last_active":"2023-10-06T05:15:03.960837+0000","last_peered":"2023-10-06T05:15:03.960837+0000","last_clean":"2023-10-06T05:15:03.960837+0000","last_became_active":"2023-10-06T05:14:59.967681+0000","last_became_peered":"2023-10-06T05:14:59.967681+0000","last_unstale":"2023-10-06T05:15:03.960837+0000","last_undegraded":"2023-10-06T05:15:03.960837+0000","last_fullsized":"2023-10-06T05:15:03.960837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990296+0000","last_change":"2023-10-06T05:15:03.989021+0000","last_active":"2023-10-06T05:15:03.990296+0000","last_peered":"2023-10-06T05:15:03.990296+0000","last_clean":"2023-10-06T05:15:03.990296+0000","last_became_active":"2023-10-06T05:15:03.988411+0000","last_became_peered":"2023-10-06T05:15:03.988411+0000","last_unstale":"2023-10-06T05:15:03.990296+0000","last_undegraded":"2023-10-06T05:15:03.990296+0000","last_fullsized":"2023-10-06T05:15:03.990296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967052+0000","last_change":"2023-10-06T05:14:58.970840+0000","last_active":"2023-10-06T05:15:03.967052+0000","last_peered":"2023-10-06T05:15:03.967052+0000","last_clean":"2023-10-06T05:15:03.967052+0000","last_became_active":"2023-10-06T05:14:58.970619+0000","last_became_peered":"2023-10-06T05:14:58.970619+0000","last_unstale":"2023-10-06T05:15:03.967052+0000","last_undegraded":"2023-10-06T05:15:03.967052+0000","last_fullsized":"2023-10-06T05:15:03.967052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968655+0000","last_change":"2023-10-06T05:14:59.955830+0000","last_active":"2023-10-06T05:15:03.968655+0000","last_peered":"2023-10-06T05:15:03.968655+0000","last_clean":"2023-10-06T05:15:03.968655+0000","last_became_active":"2023-10-06T05:14:59.955566+0000","last_became_peered":"2023-10-06T05:14:59.955566+0000","last_unstale":"2023-10-06T05:15:03.968655+0000","last_undegraded":"2023-10-06T05:15:03.968655+0000","last_fullsized":"2023-10-06T05:15:03.968655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:47.738141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987078+0000","last_change":"2023-10-06T05:15:03.984831+0000","last_active":"2023-10-06T05:15:03.987078+0000","last_peered":"2023-10-06T05:15:03.987078+0000","last_clean":"2023-10-06T05:15:03.987078+0000","last_became_active":"2023-10-06T05:15:03.984436+0000","last_became_peered":"2023-10-06T05:15:03.984436+0000","last_unstale":"2023-10-06T05:15:03.987078+0000","last_undegraded":"2023-10-06T05:15:03.987078+0000","last_fullsized":"2023-10-06T05:15:03.987078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:10.551105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961685+0000","last_change":"2023-10-06T05:14:58.956457+0000","last_active":"2023-10-06T05:15:03.961685+0000","last_peered":"2023-10-06T05:15:03.961685+0000","last_clean":"2023-10-06T05:15:03.961685+0000","last_became_active":"2023-10-06T05:14:58.956165+0000","last_became_peered":"2023-10-06T05:14:58.956165+0000","last_unstale":"2023-10-06T05:15:03.961685+0000","last_undegraded":"2023-10-06T05:15:03.961685+0000","last_fullsized":"2023-10-06T05:15:03.961685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.968300+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.967965+0000","last_became_peered":"2023-10-06T05:14:59.967965+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:35.451365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978224+0000","last_change":"2023-10-06T05:15:03.976752+0000","last_active":"2023-10-06T05:15:03.978224+0000","last_peered":"2023-10-06T05:15:03.978224+0000","last_clean":"2023-10-06T05:15:03.978224+0000","last_became_active":"2023-10-06T05:15:03.976322+0000","last_became_peered":"2023-10-06T05:15:03.976322+0000","last_unstale":"2023-10-06T05:15:03.978224+0000","last_undegraded":"2023-10-06T05:15:03.978224+0000","last_fullsized":"2023-10-06T05:15:03.978224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962719+0000","last_change":"2023-10-06T05:14:58.955881+0000","last_active":"2023-10-06T05:15:03.962719+0000","last_peered":"2023-10-06T05:15:03.962719+0000","last_clean":"2023-10-06T05:15:03.962719+0000","last_became_active":"2023-10-06T05:14:58.955624+0000","last_became_peered":"2023-10-06T05:14:58.955624+0000","last_unstale":"2023-10-06T05:15:03.962719+0000","last_undegraded":"2023-10-06T05:15:03.962719+0000","last_fullsized":"2023-10-06T05:15:03.962719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:16.365139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960721+0000","last_change":"2023-10-06T05:14:59.968565+0000","last_active":"2023-10-06T05:15:03.960721+0000","last_peered":"2023-10-06T05:15:03.960721+0000","last_clean":"2023-10-06T05:15:03.960721+0000","last_became_active":"2023-10-06T05:14:59.965728+0000","last_became_peered":"2023-10-06T05:14:59.965728+0000","last_unstale":"2023-10-06T05:15:03.960721+0000","last_undegraded":"2023-10-06T05:15:03.960721+0000","last_fullsized":"2023-10-06T05:15:03.960721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:49.359907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977840+0000","last_change":"2023-10-06T05:15:03.975283+0000","last_active":"2023-10-06T05:15:03.977840+0000","last_peered":"2023-10-06T05:15:03.977840+0000","last_clean":"2023-10-06T05:15:03.977840+0000","last_became_active":"2023-10-06T05:15:03.974903+0000","last_became_peered":"2023-10-06T05:15:03.974903+0000","last_unstale":"2023-10-06T05:15:03.977840+0000","last_undegraded":"2023-10-06T05:15:03.977840+0000","last_fullsized":"2023-10-06T05:15:03.977840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:18.853022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966753+0000","last_change":"2023-10-06T05:14:58.961971+0000","last_active":"2023-10-06T05:15:03.966753+0000","last_peered":"2023-10-06T05:15:03.966753+0000","last_clean":"2023-10-06T05:15:03.966753+0000","last_became_active":"2023-10-06T05:14:58.961689+0000","last_became_peered":"2023-10-06T05:14:58.961689+0000","last_unstale":"2023-10-06T05:15:03.966753+0000","last_undegraded":"2023-10-06T05:15:03.966753+0000","last_fullsized":"2023-10-06T05:15:03.966753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963640+0000","last_change":"2023-10-06T05:14:59.959821+0000","last_active":"2023-10-06T05:15:03.963640+0000","last_peered":"2023-10-06T05:15:03.963640+0000","last_clean":"2023-10-06T05:15:03.963640+0000","last_became_active":"2023-10-06T05:14:59.959543+0000","last_became_peered":"2023-10-06T05:14:59.959543+0000","last_unstale":"2023-10-06T05:15:03.963640+0000","last_undegraded":"2023-10-06T05:15:03.963640+0000","last_fullsized":"2023-10-06T05:15:03.963640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.991469+0000","last_change":"2023-10-06T05:15:03.990597+0000","last_active":"2023-10-06T05:15:03.991469+0000","last_peered":"2023-10-06T05:15:03.991469+0000","last_clean":"2023-10-06T05:15:03.991469+0000","last_became_active":"2023-10-06T05:15:03.990258+0000","last_became_peered":"2023-10-06T05:15:03.990258+0000","last_unstale":"2023-10-06T05:15:03.991469+0000","last_undegraded":"2023-10-06T05:15:03.991469+0000","last_fullsized":"2023-10-06T05:15:03.991469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970366+0000","last_change":"2023-10-06T05:14:58.970911+0000","last_active":"2023-10-06T05:15:03.970366+0000","last_peered":"2023-10-06T05:15:03.970366+0000","last_clean":"2023-10-06T05:15:03.970366+0000","last_became_active":"2023-10-06T05:14:58.970439+0000","last_became_peered":"2023-10-06T05:14:58.970439+0000","last_unstale":"2023-10-06T05:15:03.970366+0000","last_undegraded":"2023-10-06T05:15:03.970366+0000","last_fullsized":"2023-10-06T05:15:03.970366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960109+0000","last_change":"2023-10-06T05:14:59.961878+0000","last_active":"2023-10-06T05:15:03.960109+0000","last_peered":"2023-10-06T05:15:03.960109+0000","last_clean":"2023-10-06T05:15:03.960109+0000","last_became_active":"2023-10-06T05:14:59.961617+0000","last_became_peered":"2023-10-06T05:14:59.961617+0000","last_unstale":"2023-10-06T05:15:03.960109+0000","last_undegraded":"2023-10-06T05:15:03.960109+0000","last_fullsized":"2023-10-06T05:15:03.960109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974134+0000","last_change":"2023-10-06T05:14:59.960593+0000","last_active":"2023-10-06T05:15:03.974134+0000","last_peered":"2023-10-06T05:15:03.974134+0000","last_clean":"2023-10-06T05:15:03.974134+0000","last_became_active":"2023-10-06T05:14:59.960329+0000","last_became_peered":"2023-10-06T05:14:59.960329+0000","last_unstale":"2023-10-06T05:15:03.974134+0000","last_undegraded":"2023-10-06T05:15:03.974134+0000","last_fullsized":"2023-10-06T05:15:03.974134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963791+0000","last_change":"2023-10-06T05:14:59.966932+0000","last_active":"2023-10-06T05:15:03.963791+0000","last_peered":"2023-10-06T05:15:03.963791+0000","last_clean":"2023-10-06T05:15:03.963791+0000","last_became_active":"2023-10-06T05:14:59.965101+0000","last_became_peered":"2023-10-06T05:14:59.965101+0000","last_unstale":"2023-10-06T05:15:03.963791+0000","last_undegraded":"2023-10-06T05:15:03.963791+0000","last_fullsized":"2023-10-06T05:15:03.963791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961190+0000","last_change":"2023-10-06T05:14:59.961912+0000","last_active":"2023-10-06T05:15:03.961190+0000","last_peered":"2023-10-06T05:15:03.961190+0000","last_clean":"2023-10-06T05:15:03.961190+0000","last_became_active":"2023-10-06T05:14:59.961658+0000","last_became_peered":"2023-10-06T05:14:59.961658+0000","last_unstale":"2023-10-06T05:15:03.961190+0000","last_undegraded":"2023-10-06T05:15:03.961190+0000","last_fullsized":"2023-10-06T05:15:03.961190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963927+0000","last_change":"2023-10-06T05:14:59.968762+0000","last_active":"2023-10-06T05:15:03.963927+0000","last_peered":"2023-10-06T05:15:03.963927+0000","last_clean":"2023-10-06T05:15:03.963927+0000","last_became_active":"2023-10-06T05:14:59.968111+0000","last_became_peered":"2023-10-06T05:14:59.968111+0000","last_unstale":"2023-10-06T05:15:03.963927+0000","last_undegraded":"2023-10-06T05:15:03.963927+0000","last_fullsized":"2023-10-06T05:15:03.963927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973699+0000","last_change":"2023-10-06T05:14:59.958480+0000","last_active":"2023-10-06T05:15:03.973699+0000","last_peered":"2023-10-06T05:15:03.973699+0000","last_clean":"2023-10-06T05:15:03.973699+0000","last_became_active":"2023-10-06T05:14:59.958139+0000","last_became_peered":"2023-10-06T05:14:59.958139+0000","last_unstale":"2023-10-06T05:15:03.973699+0000","last_undegraded":"2023-10-06T05:15:03.973699+0000","last_fullsized":"2023-10-06T05:15:03.973699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:03.025853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":695352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":103167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:20.382 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:15:20.382 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T05:15:20.382 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T05:15:20.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T05:15:20.383 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T05:15:20.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.548+0000 7f68f4def700 1 Processor -- start 2023-10-06T05:15:20.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.548+0000 7f68f4def700 1 -- start start 2023-10-06T05:15:20.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68f4def700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68f01560a0 0x7f68f015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68f4def700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f00ae8d0 con 0x7f68f0155cd0 2023-10-06T05:15:20.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68f4def700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f00aebb0 con 0x7f68f00ccdc0 2023-10-06T05:15:20.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68f4def700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f00ae6b0 con 0x7f68f01560a0 2023-10-06T05:15:20.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68edd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68f01560a0 0x7f68f015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68ee59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68f0155cd0 0x7f68f00adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58506/0 (socket says 172.21.15.45:58506) 2023-10-06T05:15:20.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68ee59c700 1 -- 172.21.15.45:0/2613110380 learned_addr learned my addr 172.21.15.45:0/2613110380 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:20.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68edd9b700 1 -- 172.21.15.45:0/2613110380 >> v1:172.21.15.144:6789/0 conn(0x7f68f00ccdc0 legacy=0x7f68f00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68edd9b700 1 -- 172.21.15.45:0/2613110380 >> v1:172.21.15.45:6789/0 conn(0x7f68f0155cd0 legacy=0x7f68f00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.549+0000 7f68edd9b700 1 -- 172.21.15.45:0/2613110380 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00aee90 con 0x7f68f01560a0 2023-10-06T05:15:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68edd9b700 1 --2- 172.21.15.45:0/2613110380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f015a730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f68e00098e0 tx=0x7f68e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ddbd2297e41615 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68ed59a700 1 -- 172.21.15.45:0/2613110380 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0009e80 con 0x7f68f01560a0 2023-10-06T05:15:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68ed59a700 1 -- 172.21.15.45:0/2613110380 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e000b860 con 0x7f68f01560a0 2023-10-06T05:15:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68ed59a700 1 -- 172.21.15.45:0/2613110380 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e000bbe0 con 0x7f68f01560a0 2023-10-06T05:15:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68f4def700 1 -- 172.21.15.45:0/2613110380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 msgr2=0x7f68f015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68f4def700 1 --2- 172.21.15.45:0/2613110380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f015a730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f68e00098e0 tx=0x7f68e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68f4def700 1 -- 172.21.15.45:0/2613110380 shutdown_connections 2023-10-06T05:15:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68f4def700 1 --2- 172.21.15.45:0/2613110380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f015a730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68f4def700 1 -- 172.21.15.45:0/2613110380 >> 172.21.15.45:0/2613110380 conn(0x7f68f006f090 msgr2=0x7f68f006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.550+0000 7f68f4def700 1 -- 172.21.15.45:0/2613110380 shutdown_connections 2023-10-06T05:15:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.551+0000 7f68f4def700 1 -- 172.21.15.45:0/2613110380 wait complete. 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.552+0000 7f68f4def700 1 Processor -- start 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.552+0000 7f68f4def700 1 -- start start 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.552+0000 7f68f4def700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68f00ccdc0 0x7f68f0158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.552+0000 7f68f4def700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f0155cd0 0x7f68f00c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.552+0000 7f68f4def700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f0167850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68f4def700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f0155450 con 0x7f68f01560a0 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68f4def700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f015a870 con 0x7f68f0155cd0 2023-10-06T05:15:20.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68f4def700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f0158c60 con 0x7f68f00ccdc0 2023-10-06T05:15:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f0167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f0167850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53070/0 (socket says 172.21.15.45:53070) 2023-10-06T05:15:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 -- 172.21.15.45:0/594865597 learned_addr learned my addr 172.21.15.45:0/594865597 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68edd9b700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f0155cd0 0x7f68f00c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68ee59c700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68f00ccdc0 0x7f68f0158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 -- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68f00ccdc0 msgr2=0x7f68f0158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68f00ccdc0 0x7f68f0158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 -- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f0155cd0 msgr2=0x7f68f00c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f0155cd0 0x7f68f00c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 -- 172.21.15.45:0/594865597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e0009420 con 0x7f68f01560a0 2023-10-06T05:15:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68eed9d700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f0167850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c810 tx=0x7f68e8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68e67fc700 1 -- 172.21.15.45:0/594865597 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8012630 con 0x7f68f01560a0 2023-10-06T05:15:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68e67fc700 1 -- 172.21.15.45:0/594865597 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80127b0 con 0x7f68f01560a0 2023-10-06T05:15:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f0158f40 con 0x7f68f01560a0 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68e67fc700 1 -- 172.21.15.45:0/594865597 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8012b70 con 0x7f68f01560a0 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.553+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f015a500 con 0x7f68f01560a0 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.554+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f006e2f0 con 0x7f68f01560a0 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.555+0000 7f68e67fc700 1 -- 172.21.15.45:0/594865597 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68e8003020 con 0x7f68f01560a0 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.556+0000 7f68e67fc700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68d0066f60 0x7f68d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.556+0000 7f68e67fc700 1 -- 172.21.15.45:0/594865597 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f68e8084d50 con 0x7f68f01560a0 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.556+0000 7f68ee59c700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68d0066f60 0x7f68d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.556+0000 7f68ee59c700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68d0066f60 0x7f68d0069410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68f00b2610 tx=0x7f68d8007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.560+0000 7f68e67fc700 1 -- 172.21.15.45:0/594865597 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68e8051b50 con 0x7f68f01560a0 2023-10-06T05:15:20.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.822+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f68f00af170 con 0x7f68f01560a0 2023-10-06T05:15:20.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.825+0000 7f68e67fc700 1 -- 172.21.15.45:0/594865597 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163709 (secure 0 0 0) 0x7f68e804ff90 con 0x7f68f01560a0 2023-10-06T05:15:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:20.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68d0066f60 msgr2=0x7f68d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68d0066f60 0x7f68d0069410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68f00b2610 tx=0x7f68d8007eb0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 msgr2=0x7f68f0167850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f0167850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c810 tx=0x7f68e8001650 comp rx=0 tx=0).stop 2023-10-06T05:15:20.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 shutdown_connections 2023-10-06T05:15:20.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68d0066f60 0x7f68d0069410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f01560a0 0x7f68f0167850 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68f00ccdc0 0x7f68f0158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 --2- 172.21.15.45:0/594865597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f0155cd0 0x7f68f00c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 >> 172.21.15.45:0/594865597 conn(0x7f68f006f090 msgr2=0x7f68f00cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 shutdown_connections 2023-10-06T05:15:20.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.828+0000 7f68f4def700 1 -- 172.21.15.45:0/594865597 wait complete. 2023-10-06T05:15:20.843 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":119196},{"type":"v1","addr":"172.21.15.144:6833","nonce":119196}]},"active_addr":"172.21.15.144:6833/119196","active_change":"2023-10-06T05:14:46.429551+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":1681422965}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":2919765177}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":4267252573}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":1049811292}]}]} 2023-10-06T05:15:20.847 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T05:15:20.848 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T05:15:20.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.978+0000 7fb5df712700 1 Processor -- start 2023-10-06T05:15:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.979+0000 7fb5df712700 1 -- start start 2023-10-06T05:15:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.979+0000 7fb5df712700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5d8155cd0 0x7fb5d815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.979+0000 7fb5df712700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb5d80ccdc0 0x7fb5d80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.979+0000 7fb5df712700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d80cd700 con 0x7fb5d81560a0 2023-10-06T05:15:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.979+0000 7fb5df712700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80ae6b0 con 0x7fb5d8155cd0 2023-10-06T05:15:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.979+0000 7fb5df712700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80ae8d0 con 0x7fb5d80ccdc0 2023-10-06T05:15:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dccad700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb5d81560a0 0x7fb5d80adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58516/0 (socket says 172.21.15.45:58516) 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dd4ae700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5d8155cd0 0x7fb5d815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dccad700 1 -- 172.21.15.45:0/920454866 learned_addr learned my addr 172.21.15.45:0/920454866 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5ddcaf700 1 --2- 172.21.15.45:0/920454866 >> v2:172.21.15.144:3300/0 conn(0x7fb5d80ccdc0 0x7fb5d80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5cffff700 1 -- 172.21.15.45:0/920454866 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44842740 0 0) 0x7fb5d80cd700 con 0x7fb5d81560a0 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5cffff700 1 -- 172.21.15.45:0/920454866 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c4003570 con 0x7fb5d81560a0 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dd4ae700 1 -- 172.21.15.45:0/920454866 >> v2:172.21.15.144:3300/0 conn(0x7fb5d80ccdc0 msgr2=0x7fb5d80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dd4ae700 1 --2- 172.21.15.45:0/920454866 >> v2:172.21.15.144:3300/0 conn(0x7fb5d80ccdc0 0x7fb5d80cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dd4ae700 1 -- 172.21.15.45:0/920454866 >> v1:172.21.15.45:6789/0 conn(0x7fb5d81560a0 legacy=0x7fb5d80adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dd4ae700 1 -- 172.21.15.45:0/920454866 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80aebb0 con 0x7fb5d8155cd0 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5ddcaf700 1 --2- 172.21.15.45:0/920454866 >> v2:172.21.15.144:3300/0 conn(0x7fb5d80ccdc0 0x7fb5d80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5dd4ae700 1 --2- 172.21.15.45:0/920454866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d815a730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80c3dd0 tx=0x7fb5d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa73272ad2582677 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5cffff700 1 -- 172.21.15.45:0/920454866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4011630 con 0x7fb5d8155cd0 2023-10-06T05:15:20.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5cffff700 1 -- 172.21.15.45:0/920454866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d4011790 con 0x7fb5d8155cd0 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.980+0000 7fb5cffff700 1 -- 172.21.15.45:0/920454866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4011b30 con 0x7fb5d8155cd0 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 -- 172.21.15.45:0/920454866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 msgr2=0x7fb5d815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 --2- 172.21.15.45:0/920454866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d815a730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80c3dd0 tx=0x7fb5d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 -- 172.21.15.45:0/920454866 shutdown_connections 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 --2- 172.21.15.45:0/920454866 >> v2:172.21.15.144:3300/0 conn(0x7fb5d80ccdc0 0x7fb5d80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 --2- 172.21.15.45:0/920454866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d815a730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 -- 172.21.15.45:0/920454866 >> 172.21.15.45:0/920454866 conn(0x7fb5d806f090 msgr2=0x7fb5d806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 -- 172.21.15.45:0/920454866 shutdown_connections 2023-10-06T05:15:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.981+0000 7fb5df712700 1 -- 172.21.15.45:0/920454866 wait complete. 2023-10-06T05:15:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.983+0000 7fb5df712700 1 Processor -- start 2023-10-06T05:15:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.983+0000 7fb5df712700 1 -- start start 2023-10-06T05:15:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5df712700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d80ccdc0 0x7fb5d8152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5df712700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d8156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5df712700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d81560a0 0x7fb5d80c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5df712700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80c3ee0 con 0x7fb5d8155cd0 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5df712700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80c41c0 con 0x7fb5d80ccdc0 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5df712700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80c44a0 con 0x7fb5d81560a0 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5dccad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d8156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5dccad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d8156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53082/0 (socket says 172.21.15.45:53082) 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5dccad700 1 -- 172.21.15.45:0/24570715 learned_addr learned my addr 172.21.15.45:0/24570715 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5dd4ae700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d80ccdc0 0x7fb5d8152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.984+0000 7fb5ddcaf700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d81560a0 0x7fb5d80c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5dccad700 1 -- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d81560a0 msgr2=0x7fb5d80c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5dccad700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d81560a0 0x7fb5d80c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5dccad700 1 -- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d80ccdc0 msgr2=0x7fb5d8152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5dccad700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d80ccdc0 0x7fb5d8152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5dccad700 1 -- 172.21.15.45:0/24570715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d4009420 con 0x7fb5d8155cd0 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5ddcaf700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d81560a0 0x7fb5d80c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5dccad700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d8156540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c590 tx=0x7fb5c8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5cd7fa700 1 -- 172.21.15.45:0/24570715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8005cd0 con 0x7fb5d8155cd0 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5cd7fa700 1 -- 172.21.15.45:0/24570715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8005e50 con 0x7fb5d8155cd0 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d80c4780 con 0x7fb5d8155cd0 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5cd7fa700 1 -- 172.21.15.45:0/24570715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8007320 con 0x7fb5d8155cd0 2023-10-06T05:15:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.985+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d8152860 con 0x7fb5d8155cd0 2023-10-06T05:15:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.986+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d806e2f0 con 0x7fb5d8155cd0 2023-10-06T05:15:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.986+0000 7fb5cd7fa700 1 -- 172.21.15.45:0/24570715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5c801b3b0 con 0x7fb5d8155cd0 2023-10-06T05:15:20.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.987+0000 7fb5cd7fa700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5bc066f30 0x7fb5bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:20.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.987+0000 7fb5cd7fa700 1 -- 172.21.15.45:0/24570715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb5c8084070 con 0x7fb5d8155cd0 2023-10-06T05:15:20.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.987+0000 7fb5dd4ae700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5bc066f30 0x7fb5bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:20.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.988+0000 7fb5dd4ae700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5bc066f30 0x7fb5bc0693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40093f0 tx=0x7fb5d4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:20.991+0000 7fb5cd7fa700 1 -- 172.21.15.45:0/24570715 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5c8050e90 con 0x7fb5d8155cd0 2023-10-06T05:15:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.197+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5d80aee90 con 0x7fb5d8155cd0 2023-10-06T05:15:21.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.198+0000 7fb5cd7fa700 1 -- 172.21.15.45:0/24570715 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fb5c8017020 con 0x7fb5d8155cd0 2023-10-06T05:15:21.200 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:21.201 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5bc066f30 msgr2=0x7fb5bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5bc066f30 0x7fb5bc0693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40093f0 tx=0x7fb5d4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 msgr2=0x7fb5d8156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d8156540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c590 tx=0x7fb5c8007e00 comp rx=0 tx=0).stop 2023-10-06T05:15:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 shutdown_connections 2023-10-06T05:15:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5bc066f30 0x7fb5bc0693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d81560a0 0x7fb5d80c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d80ccdc0 0x7fb5d8152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 --2- 172.21.15.45:0/24570715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8155cd0 0x7fb5d8156540 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 >> 172.21.15.45:0/24570715 conn(0x7fb5d806f090 msgr2=0x7fb5d8154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 shutdown_connections 2023-10-06T05:15:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.200+0000 7fb5df712700 1 -- 172.21.15.45:0/24570715 wait complete. 2023-10-06T05:15:21.217 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T05:15:21.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:21.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.348+0000 7f8c51d05700 1 Processor -- start 2023-10-06T05:15:21.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.348+0000 7f8c51d05700 1 -- start start 2023-10-06T05:15:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c51d05700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8c4c153650 0x7f8c4c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c51d05700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c0676c0 con 0x7f8c4c06afa0 2023-10-06T05:15:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c51d05700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c0679a0 con 0x7f8c4c153a20 2023-10-06T05:15:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c51d05700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4c067c80 con 0x7f8c4c153650 2023-10-06T05:15:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c4bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8c4c06afa0 0x7f8c4c068480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58520/0 (socket says 172.21.15.45:58520) 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c4bfff700 1 -- 172.21.15.45:0/2004828891 learned_addr learned my addr 172.21.15.45:0/2004828891 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c4b7fe700 1 --2- 172.21.15.45:0/2004828891 >> v2:172.21.15.144:3300/0 conn(0x7f8c4c153650 0x7f8c4c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c4a7fc700 1 -- 172.21.15.45:0/2004828891 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602213204 0 0) 0x7f8c4c0676c0 con 0x7f8c4c06afa0 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c4a7fc700 1 -- 172.21.15.45:0/2004828891 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c38003570 con 0x7f8c4c06afa0 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c4a7fc700 1 -- 172.21.15.45:0/2004828891 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164763117 0 0) 0x7f8c38003570 con 0x7f8c4c06afa0 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.349+0000 7f8c4b7fe700 1 -- 172.21.15.45:0/2004828891 >> v1:172.21.15.144:6789/0 conn(0x7f8c4c153a20 legacy=0x7f8c4c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c4b7fe700 1 -- 172.21.15.45:0/2004828891 >> v1:172.21.15.45:6789/0 conn(0x7f8c4c06afa0 legacy=0x7f8c4c068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c4b7fe700 1 -- 172.21.15.45:0/2004828891 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c067f60 con 0x7f8c4c153650 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c4b7fe700 1 --2- 172.21.15.45:0/2004828891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153650 0x7f8c4c15a930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098e0 tx=0x7f8c3c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c080617356fd5a30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c4a7fc700 1 -- 172.21.15.45:0/2004828891 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c440017f0 con 0x7f8c4c153650 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c4a7fc700 1 -- 172.21.15.45:0/2004828891 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c011420 con 0x7f8c4c153650 2023-10-06T05:15:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c4a7fc700 1 -- 172.21.15.45:0/2004828891 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0117c0 con 0x7f8c4c153650 2023-10-06T05:15:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c51d05700 1 -- 172.21.15.45:0/2004828891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153650 msgr2=0x7f8c4c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.350+0000 7f8c51d05700 1 --2- 172.21.15.45:0/2004828891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153650 0x7f8c4c15a930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098e0 tx=0x7f8c3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.351+0000 7f8c51d05700 1 -- 172.21.15.45:0/2004828891 shutdown_connections 2023-10-06T05:15:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.351+0000 7f8c51d05700 1 --2- 172.21.15.45:0/2004828891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153650 0x7f8c4c15a930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.351+0000 7f8c51d05700 1 -- 172.21.15.45:0/2004828891 >> 172.21.15.45:0/2004828891 conn(0x7f8c4c0cc5a0 msgr2=0x7f8c4c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.351+0000 7f8c51d05700 1 -- 172.21.15.45:0/2004828891 shutdown_connections 2023-10-06T05:15:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.351+0000 7f8c51d05700 1 -- 172.21.15.45:0/2004828891 wait complete. 2023-10-06T05:15:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.352+0000 7f8c51d05700 1 Processor -- start 2023-10-06T05:15:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.352+0000 7f8c51d05700 1 -- start start 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c51d05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c4c06afa0 0x7f8c4c0ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c51d05700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4c153650 0x7f8c4c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c51d05700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153a20 0x7f8c4c0ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c51d05700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0c1c70 con 0x7f8c4c06afa0 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c51d05700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0c1f50 con 0x7f8c4c153a20 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c51d05700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0c2230 con 0x7f8c4c153650 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c4c06afa0 0x7f8c4c0ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c4c06afa0 0x7f8c4c0ab1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53090/0 (socket says 172.21.15.45:53090) 2023-10-06T05:15:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 -- 172.21.15.45:0/3952935204 learned_addr learned my addr 172.21.15.45:0/3952935204 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4bfff700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153a20 0x7f8c4c0ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4affd700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4c153650 0x7f8c4c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 -- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4c153650 msgr2=0x7f8c4c0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4c153650 0x7f8c4c0b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 -- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153a20 msgr2=0x7f8c4c0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153a20 0x7f8c4c0ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 -- 172.21.15.45:0/3952935204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c009420 con 0x7f8c4c06afa0 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.353+0000 7f8c4b7fe700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c4c06afa0 0x7f8c4c0ab1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0091e0 tx=0x7f8c3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.354+0000 7f8c437fe700 1 -- 172.21.15.45:0/3952935204 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c01cbe0 con 0x7f8c4c06afa0 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.354+0000 7f8c437fe700 1 -- 172.21.15.45:0/3952935204 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c01cd40 con 0x7f8c4c06afa0 2023-10-06T05:15:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.354+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c0c2510 con 0x7f8c4c06afa0 2023-10-06T05:15:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.354+0000 7f8c437fe700 1 -- 172.21.15.45:0/3952935204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c01a6a0 con 0x7f8c4c06afa0 2023-10-06T05:15:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.354+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c0b2140 con 0x7f8c4c06afa0 2023-10-06T05:15:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.355+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c0b2330 con 0x7f8c4c06afa0 2023-10-06T05:15:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.355+0000 7f8c437fe700 1 -- 172.21.15.45:0/3952935204 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c3c01aa90 con 0x7f8c4c06afa0 2023-10-06T05:15:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.356+0000 7f8c437fe700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c2c066f30 0x7f8c2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.356+0000 7f8c437fe700 1 -- 172.21.15.45:0/3952935204 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8c3c0863c0 con 0x7f8c4c06afa0 2023-10-06T05:15:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.356+0000 7f8c4affd700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c2c066f30 0x7f8c2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.356+0000 7f8c4affd700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c2c066f30 0x7f8c2c0693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0c13c0 tx=0x7f8c34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.360+0000 7f8c437fe700 1 -- 172.21.15.45:0/3952935204 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c4c0b2330 con 0x7f8c4c06afa0 2023-10-06T05:15:21.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.564+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c4c0b2330 con 0x7f8c4c06afa0 2023-10-06T05:15:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.565+0000 7f8c437fe700 1 -- 172.21.15.45:0/3952935204 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f8c4c0b2330 con 0x7f8c4c06afa0 2023-10-06T05:15:21.567 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:21.567 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:21.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c2c066f30 msgr2=0x7f8c2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c2c066f30 0x7f8c2c0693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0c13c0 tx=0x7f8c34009040 comp rx=0 tx=0).stop 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c4c06afa0 msgr2=0x7f8c4c0ab1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c4c06afa0 0x7f8c4c0ab1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0091e0 tx=0x7f8c3c016000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 shutdown_connections 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c2c066f30 0x7f8c2c0693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4c153a20 0x7f8c4c0ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c4c06afa0 0x7f8c4c0ab1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 --2- 172.21.15.45:0/3952935204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4c153650 0x7f8c4c0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 >> 172.21.15.45:0/3952935204 conn(0x7f8c4c0cc5a0 msgr2=0x7f8c4c15b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 shutdown_connections 2023-10-06T05:15:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.567+0000 7f8c51d05700 1 -- 172.21.15.45:0/3952935204 wait complete. 2023-10-06T05:15:21.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T05:15:21.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T05:15:21.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T05:15:21.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T05:15:21.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T05:15:21.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T05:15:21.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T05:15:21.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T05:15:21.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.817+0000 7f84426fb700 1 Processor -- start 2023-10-06T05:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.817+0000 7f84426fb700 1 -- start start 2023-10-06T05:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.819+0000 7f84426fb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f843c0adf10 0x7f843c161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.819+0000 7f84426fb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c06b080 con 0x7f843c0c0170 2023-10-06T05:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.819+0000 7f84426fb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c06b2d0 con 0x7f843c0cd4d0 2023-10-06T05:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.819+0000 7f84426fb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c06b5b0 con 0x7f843c0adf10 2023-10-06T05:15:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.819+0000 7f843bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f843c0c0170 0x7f843c0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58532/0 (socket says 172.21.15.45:58532) 2023-10-06T05:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.819+0000 7f843bfff700 1 -- 172.21.15.45:0/3888727220 learned_addr learned my addr 172.21.15.45:0/3888727220 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.820+0000 7f5ad68e6700 1 Processor -- start 2023-10-06T05:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.820+0000 7f5ad68e6700 1 -- start start 2023-10-06T05:15:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.820+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706943847 0 0) 0x7f843c06b080 con 0x7f843c0c0170 2023-10-06T05:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.820+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8424003570 con 0x7f843c0c0170 2023-10-06T05:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.820+0000 7f8440c98700 1 --2- 172.21.15.45:0/3888727220 >> v2:172.21.15.144:3300/0 conn(0x7f843c0adf10 0x7f843c161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3524198694 0 0) 0x7f8424003570 con 0x7f843c0c0170 2023-10-06T05:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 >> v2:172.21.15.144:3300/0 conn(0x7f843c0adf10 msgr2=0x7f843c161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 --2- 172.21.15.45:0/3888727220 >> v2:172.21.15.144:3300/0 conn(0x7f843c0adf10 0x7f843c161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 >> v1:172.21.15.144:6789/0 conn(0x7f843c0cd4d0 legacy=0x7f843c161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f0083fff700 1 Processor -- start 2023-10-06T05:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c06b890 con 0x7f843c0c0170 2023-10-06T05:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f842c001470 con 0x7f843c0c0170 2023-10-06T05:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 >> v1:172.21.15.45:6789/0 conn(0x7f843c0c0170 legacy=0x7f843c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f5ad68e6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad0146f80 0x7f5ad00afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f5ad68e6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad015f1c0 con 0x7f5ad00aff60 2023-10-06T05:15:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f5ad68e6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad00b03b0 con 0x7f5ad006a6f0 2023-10-06T05:15:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7ff5c2d30700 1 Processor -- start 2023-10-06T05:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8424003a60 0x7f8424003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8424004810 0x7f8424004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424005210 0x7f842400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c06b5b0 con 0x7f8424004810 2023-10-06T05:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842400c1e0 con 0x7f8424005210 2023-10-06T05:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842400c3a0 con 0x7f8424003a60 2023-10-06T05:15:21.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f5ad68e6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad006ad10 con 0x7f5ad0146f80 2023-10-06T05:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f843bfff700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8424003a60 0x7f8424003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f0083fff700 1 -- start start 2023-10-06T05:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f5acffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad0146f80 0x7f5ad00afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f5acffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad0146f80 0x7f5ad00afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33800/0 (socket says 172.21.15.45:33800) 2023-10-06T05:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7f5acffff700 1 -- 172.21.15.45:0/2975132467 learned_addr learned my addr 172.21.15.45:0/2975132467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f5acffff700 1 -- 172.21.15.45:0/2975132467 >> v1:172.21.15.144:6789/0 conn(0x7f5ad006a6f0 legacy=0x7f5ad015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f5acffff700 1 -- 172.21.15.45:0/2975132467 >> v1:172.21.15.45:6789/0 conn(0x7f5ad00aff60 legacy=0x7f5ad0069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f5acffff700 1 -- 172.21.15.45:0/2975132467 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad006aff0 con 0x7f5ad0146f80 2023-10-06T05:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.821+0000 7ff5c2d30700 1 -- start start 2023-10-06T05:15:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f5acffff700 1 --2- 172.21.15.45:0/2975132467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad0146f80 0x7f5ad00afa20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00099e0 tx=0x7f5ac0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18761403fe602fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f0083fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f008406de30 0x7f0084154b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f0083fff700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0084158640 con 0x7f008406c9a0 2023-10-06T05:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f0083fff700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0084158890 con 0x7f0084154760 2023-10-06T05:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f0083fff700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00840c9590 con 0x7f008406de30 2023-10-06T05:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f0082ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0084154760 0x7f00840c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50802/0 (socket says 172.21.15.45:50802) 2023-10-06T05:15:21.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f0082ffd700 1 -- 172.21.15.45:0/3997271562 learned_addr learned my addr 172.21.15.45:0/3997271562 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f843b7fe700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8424004810 0x7f8424004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f8440c98700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424005210 0x7f842400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f843b7fe700 1 -- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8424003a60 msgr2=0x7f8424003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f843b7fe700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8424003a60 0x7f8424003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f843b7fe700 1 -- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424005210 msgr2=0x7f842400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f843b7fe700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424005210 0x7f842400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f843b7fe700 1 -- 172.21.15.45:0/3888727220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c06b890 con 0x7f8424004810 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.822+0000 7f8440c98700 1 -- 172.21.15.45:0/3888727220 reap_dead start 2023-10-06T05:15:21.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f843b7fe700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8424004810 0x7f8424004c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8424004790 tx=0x7f8430007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bef36e4bc5fa7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430001230 con 0x7f8424004810 2023-10-06T05:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430012400 con 0x7f8424004810 2023-10-06T05:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f843affd700 1 -- 172.21.15.45:0/3888727220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300127a0 con 0x7f8424004810 2023-10-06T05:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f5aceffd700 1 -- 172.21.15.45:0/2975132467 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000b7e0 con 0x7f5ad0146f80 2023-10-06T05:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f5aceffd700 1 -- 172.21.15.45:0/2975132467 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac000b940 con 0x7f5ad0146f80 2023-10-06T05:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c2d30700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff5bc067620 0x7ff5bc06e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c2d30700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06ec90 0x7ff5bc0af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c2d30700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc0686d0 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f5aceffd700 1 -- 172.21.15.45:0/2975132467 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000bce0 con 0x7f5ad0146f80 2023-10-06T05:15:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f8519c58700 1 Processor -- start 2023-10-06T05:15:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c2d30700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc06f110 con 0x7ff5bc067620 2023-10-06T05:15:21.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c2d30700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc067ac0 con 0x7ff5bc06ec90 2023-10-06T05:15:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f8519c58700 1 -- start start 2023-10-06T05:15:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79137464 0 0) 0x7f0084158890 con 0x7f0084154760 2023-10-06T05:15:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f006c003570 con 0x7f0084154760 2023-10-06T05:15:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341888322 0 0) 0x7f0084158640 con 0x7f008406c9a0 2023-10-06T05:15:21.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0084158890 con 0x7f008406c9a0 2023-10-06T05:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c1d2e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff5bc15d4b0 0x7ff5bc15d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50810/0 (socket says 172.21.15.45:50810) 2023-10-06T05:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c1d2e700 1 -- 172.21.15.45:0/2478266221 learned_addr learned my addr 172.21.15.45:0/2478266221 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c152d700 1 --2- 172.21.15.45:0/2478266221 >> v2:172.21.15.144:3300/0 conn(0x7ff5bc067620 0x7ff5bc06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c252f700 1 --2- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06ec90 0x7ff5bc0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.823+0000 7f7a48411700 1 Processor -- start 2023-10-06T05:15:21.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7f7a48411700 1 -- start start 2023-10-06T05:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c152d700 1 -- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06ec90 msgr2=0x7ff5bc0af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c152d700 1 --2- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06ec90 0x7ff5bc0af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c152d700 1 -- 172.21.15.45:0/2478266221 >> v1:172.21.15.144:6789/0 conn(0x7ff5bc15d4b0 legacy=0x7ff5bc15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.824+0000 7ff5c152d700 1 -- 172.21.15.45:0/2478266221 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc067da0 con 0x7ff5bc067620 2023-10-06T05:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f84426fb700 1 -- 172.21.15.45:0/3888727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8424004810 msgr2=0x7f8424004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f84426fb700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8424004810 0x7f8424004c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8424004790 tx=0x7f8430007cf0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f5ad68e6700 1 -- 172.21.15.45:0/2975132467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad0146f80 msgr2=0x7f5ad00afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f5ad68e6700 1 --2- 172.21.15.45:0/2975132467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad0146f80 0x7f5ad00afa20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00099e0 tx=0x7f5ac0009250 comp rx=0 tx=0).stop 2023-10-06T05:15:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7ff5c152d700 1 --2- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 0x7ff5bc06e750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80098e0 tx=0x7ff5b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de3b722aa8b4006 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f84426fb700 1 -- 172.21.15.45:0/3888727220 shutdown_connections 2023-10-06T05:15:21.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f84426fb700 1 --2- 172.21.15.45:0/3888727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8424004810 0x7f8424004c80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f84426fb700 1 -- 172.21.15.45:0/3888727220 >> 172.21.15.45:0/3888727220 conn(0x7f843c06f090 msgr2=0x7f843c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f84426fb700 1 -- 172.21.15.45:0/3888727220 shutdown_connections 2023-10-06T05:15:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f84426fb700 1 -- 172.21.15.45:0/3888727220 wait complete. 2023-10-06T05:15:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f5ad68e6700 1 -- 172.21.15.45:0/2975132467 shutdown_connections 2023-10-06T05:15:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f5ad68e6700 1 --2- 172.21.15.45:0/2975132467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad0146f80 0x7f5ad00afa20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f5ad68e6700 1 -- 172.21.15.45:0/2975132467 >> 172.21.15.45:0/2975132467 conn(0x7f5ad0067620 msgr2=0x7f5ad0067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f8519c58700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85140c7f90 0x7f8514159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f8519c58700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85140af1f0 con 0x7f8514155590 2023-10-06T05:15:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f8519c58700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8514159bf0 con 0x7f85140c8360 2023-10-06T05:15:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f8519c58700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85140cbf80 con 0x7f85140c7f90 2023-10-06T05:15:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f85137fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85140c7f90 0x7f8514159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f85137fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85140c7f90 0x7f8514159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53138/0 (socket says 172.21.15.45:53138) 2023-10-06T05:15:21.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f85137fe700 1 -- 172.21.15.45:0/3377233237 learned_addr learned my addr 172.21.15.45:0/3377233237 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7ff5c0d2c700 1 -- 172.21.15.45:0/2478266221 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8011630 con 0x7ff5bc067620 2023-10-06T05:15:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7ff5c0d2c700 1 -- 172.21.15.45:0/2478266221 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b8011790 con 0x7ff5bc067620 2023-10-06T05:15:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7ff5c0d2c700 1 -- 172.21.15.45:0/2478266221 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8011b30 con 0x7ff5bc067620 2023-10-06T05:15:21.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7ff5c2d30700 1 -- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 msgr2=0x7ff5bc06e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7ff5c2d30700 1 --2- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 0x7ff5bc06e750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80098e0 tx=0x7ff5b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7ff5c2d30700 1 -- 172.21.15.45:0/2478266221 shutdown_connections 2023-10-06T05:15:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7ff5c2d30700 1 --2- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06ec90 0x7ff5bc0af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7ff5c2d30700 1 --2- 172.21.15.45:0/2478266221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 0x7ff5bc06e750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7ff5c2d30700 1 -- 172.21.15.45:0/2478266221 >> 172.21.15.45:0/2478266221 conn(0x7ff5bc0ccf10 msgr2=0x7ff5bc0aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f85137fe700 1 -- 172.21.15.45:0/3377233237 >> v1:172.21.15.144:6789/0 conn(0x7f85140c8360 legacy=0x7f8514154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f85137fe700 1 -- 172.21.15.45:0/3377233237 >> v1:172.21.15.45:6789/0 conn(0x7f8514155590 legacy=0x7f85140af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f85137fe700 1 -- 172.21.15.45:0/3377233237 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8514159ed0 con 0x7f85140c7f90 2023-10-06T05:15:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7ff5c2d30700 1 -- 172.21.15.45:0/2478266221 shutdown_connections 2023-10-06T05:15:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7f85137fe700 1 --2- 172.21.15.45:0/3377233237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85140c7f90 0x7f8514159010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f850400d280 tx=0x7f85040076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f570947f8edb79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7f5ad68e6700 1 -- 172.21.15.45:0/2975132467 shutdown_connections 2023-10-06T05:15:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7f5ad68e6700 1 -- 172.21.15.45:0/2975132467 wait complete. 2023-10-06T05:15:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7f84426fb700 1 Processor -- start 2023-10-06T05:15:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7f84426fb700 1 -- start start 2023-10-06T05:15:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7f5ad68e6700 1 Processor -- start 2023-10-06T05:15:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7ff5c2d30700 1 -- 172.21.15.45:0/2478266221 wait complete. 2023-10-06T05:15:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7ff5c2d30700 1 Processor -- start 2023-10-06T05:15:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f00837fe700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f008406de30 0x7f0084154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1529414700 0 0) 0x7f006c003570 con 0x7f0084154760 2023-10-06T05:15:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f008406de30 msgr2=0x7f0084154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f008406de30 0x7f0084154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 >> v1:172.21.15.45:6789/0 conn(0x7f008406c9a0 legacy=0x7f008406d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084158b70 con 0x7f0084154760 2023-10-06T05:15:21.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f007c001720 con 0x7f0084154760 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 >> v1:172.21.15.144:6789/0 conn(0x7f0084154760 legacy=0x7f00840c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f006c004cd0 0x7f006c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f006c0037b0 0x7f006c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c009a50 0x7f006c0041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00840c9590 con 0x7f006c009a50 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c009e70 con 0x7f006c0037b0 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c00a030 con 0x7f006c004cd0 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f0082ffd700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f006c004cd0 0x7f006c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.825+0000 7f00837fe700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c009a50 0x7f006c0041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0082ffd700 1 -- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f006c0037b0 msgr2=0x7f006c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0082ffd700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f006c0037b0 0x7f006c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0082ffd700 1 -- 172.21.15.45:0/3997271562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c009a50 msgr2=0x7f006c0041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0082ffd700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c009a50 0x7f006c0041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0082ffd700 1 -- 172.21.15.45:0/3997271562 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084158b70 con 0x7f006c004cd0 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0082ffd700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f006c004cd0 0x7f006c0050c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f007c00a400 tx=0x7f007c002130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afad78508cdb58c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0070f0 con 0x7f006c004cd0 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c007250 con 0x7f006c004cd0 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.826+0000 7f0081ffb700 1 -- 172.21.15.45:0/3997271562 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c007610 con 0x7f006c004cd0 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7f0083fff700 1 -- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f006c004cd0 msgr2=0x7f006c0050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7f0083fff700 1 --2- 172.21.15.45:0/3997271562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f006c004cd0 0x7f006c0050c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f007c00a400 tx=0x7f007c002130 comp rx=0 tx=0).stop 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.827+0000 7f00837fe700 1 -- 172.21.15.45:0/3997271562 reap_dead start 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7f0083fff700 1 -- 172.21.15.45:0/3997271562 shutdown_connections 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7f0083fff700 1 -- 172.21.15.45:0/3997271562 >> 172.21.15.45:0/3997271562 conn(0x7f00840ce800 msgr2=0x7f008406bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7f0083fff700 1 -- 172.21.15.45:0/3997271562 shutdown_connections 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.828+0000 7f0083fff700 1 -- 172.21.15.45:0/3997271562 wait complete. 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f84426fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424004810 0x7f843c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f84426fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f843c0c5cc0 0x7f843c14fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f84426fb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f843c0c6130 0x7f843c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f84426fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c1507f0 con 0x7f843c0c5cc0 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f84426fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c150ad0 con 0x7f8424004810 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f84426fb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c151930 con 0x7f843c0c6130 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f843bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424004810 0x7f843c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f843bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424004810 0x7f843c1521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47378/0 (socket says 172.21.15.45:47378) 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f843bfff700 1 -- 172.21.15.45:0/1422957083 learned_addr learned my addr 172.21.15.45:0/1422957083 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f8440c98700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f843c0c6130 0x7f843c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f5ad68e6700 1 -- start start 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5ad68e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad006a6f0 0x7f5ad016dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5ad68e6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad00aff60 0x7f5ad015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5ad68e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0146f80 0x7f5ad0144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5ad68e6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad016e1f0 con 0x7f5ad0146f80 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5ad68e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad00a5bb0 con 0x7f5ad006a6f0 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5ad68e6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad00aaff0 con 0x7f5ad00aff60 2023-10-06T05:15:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5acffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad006a6f0 0x7f5ad016dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5acf7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad00aff60 0x7f5ad015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5acffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad006a6f0 0x7f5ad016dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47392/0 (socket says 172.21.15.45:47392) 2023-10-06T05:15:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f5acffff700 1 -- 172.21.15.45:0/4270873643 learned_addr learned my addr 172.21.15.45:0/4270873643 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f0083fff700 1 Processor -- start 2023-10-06T05:15:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7ff5c2d30700 1 -- start start 2023-10-06T05:15:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7ff5c2d30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 0x7ff5bc15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7ff5c2d30700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5bc06ec90 0x7ff5bc16e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7ff5c2d30700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15d4b0 0x7ff5bc0aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7ff5c2d30700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0abc70 con 0x7ff5bc06ec90 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7ff5c2d30700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc16ed30 con 0x7ff5bc067620 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7ff5c2d30700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0aaff0 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7ff5c252f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15d4b0 0x7ff5bc0aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7ff5c252f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15d4b0 0x7ff5bc0aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33870/0 (socket says 172.21.15.45:33870) 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7ff5c252f700 1 -- 172.21.15.45:0/1868612478 learned_addr learned my addr 172.21.15.45:0/1868612478 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f8cb45a8700 1 Processor -- start 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f8cb45a8700 1 -- start start 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a48411700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a400b0130 0x7f7a40069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a48411700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a4006a520 0x7f7a4015f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a48411700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4015f1c0 con 0x7f7a40146f80 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a48411700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4006ad10 con 0x7f7a400b0130 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a48411700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4006aff0 con 0x7f7a4006a520 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a469ae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a4006a520 0x7f7a4015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a469ae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a4006a520 0x7f7a4015f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47362/0 (socket says 172.21.15.45:47362) 2023-10-06T05:15:21.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a469ae700 1 -- 172.21.15.45:0/162263291 learned_addr learned my addr 172.21.15.45:0/162263291 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.829+0000 7f7a459ac700 1 --2- 172.21.15.45:0/162263291 >> v2:172.21.15.45:3300/0 conn(0x7f7a400b0130 0x7f7a40069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f7a469ae700 1 -- 172.21.15.45:0/162263291 >> v2:172.21.15.45:3300/0 conn(0x7f7a400b0130 msgr2=0x7f7a40069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f7a469ae700 1 --2- 172.21.15.45:0/162263291 >> v2:172.21.15.45:3300/0 conn(0x7f7a400b0130 0x7f7a40069fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f7a469ae700 1 -- 172.21.15.45:0/162263291 >> v1:172.21.15.45:6789/0 conn(0x7f7a40146f80 legacy=0x7f7a400afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f7a469ae700 1 -- 172.21.15.45:0/162263291 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4006b2d0 con 0x7f7a4006a520 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f7a469ae700 1 --2- 172.21.15.45:0/162263291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015f650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a40161140 tx=0x7f7a3c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8ff65320a9f9796 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f7a451ab700 1 -- 172.21.15.45:0/162263291 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00b820 con 0x7f7a4006a520 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f7a451ab700 1 -- 172.21.15.45:0/162263291 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00b980 con 0x7f7a4006a520 2023-10-06T05:15:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f7a451ab700 1 -- 172.21.15.45:0/162263291 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00bd20 con 0x7f7a4006a520 2023-10-06T05:15:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f5acf7fe700 1 -- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad006a6f0 msgr2=0x7f5ad016dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f5acf7fe700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad006a6f0 0x7f5ad016dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f5acf7fe700 1 -- 172.21.15.45:0/4270873643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0146f80 msgr2=0x7f5ad0144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f5acf7fe700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad0146f80 0x7f5ad0144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f5acf7fe700 1 -- 172.21.15.45:0/4270873643 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac00094f0 con 0x7f5ad00aff60 2023-10-06T05:15:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7ff5c152d700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5bc06ec90 0x7ff5bc16e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7fb2dda49700 1 Processor -- start 2023-10-06T05:15:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7ff5c1d2e700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 0x7ff5bc15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7ff5c252f700 1 -- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 msgr2=0x7ff5bc15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7ff5c252f700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc067620 0x7ff5bc15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7ff5c252f700 1 -- 172.21.15.45:0/1868612478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5bc06ec90 msgr2=0x7ff5bc16e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7ff5c252f700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5bc06ec90 0x7ff5bc16e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7ff5c252f700 1 -- 172.21.15.45:0/1868612478 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8009420 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f7a48411700 1 -- 172.21.15.45:0/162263291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 msgr2=0x7f7a4015f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f7a48411700 1 --2- 172.21.15.45:0/162263291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015f650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a40161140 tx=0x7f7a3c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f0083fff700 1 -- start start 2023-10-06T05:15:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f0083fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c004cd0 0x7f00840cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f0083fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084148720 0x7f00841563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f0083fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00841568e0 0x7f0084153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f0083fff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0084154290 con 0x7f006c004cd0 2023-10-06T05:15:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f0083fff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0084156d50 con 0x7f0084148720 2023-10-06T05:15:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f0083fff700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00840aefa0 con 0x7f00841568e0 2023-10-06T05:15:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f00827fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084148720 0x7f00841563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f00827fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084148720 0x7f00841563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47414/0 (socket says 172.21.15.45:47414) 2023-10-06T05:15:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f00827fc700 1 -- 172.21.15.45:0/3755284696 learned_addr learned my addr 172.21.15.45:0/3755284696 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f00837fe700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00841568e0 0x7f0084153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f5acf7fe700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad00aff60 0x7f5ad015b670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800bb00 tx=0x7f5ac8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f843bfff700 1 -- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f843c0c6130 msgr2=0x7f843c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f843bfff700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f843c0c6130 0x7f843c1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f843bfff700 1 -- 172.21.15.45:0/1422957083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f843c0c5cc0 msgr2=0x7f843c14fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f843bfff700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f843c0c5cc0 0x7f843c14fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f843bfff700 1 -- 172.21.15.45:0/1422957083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c1526f0 con 0x7f8424004810 2023-10-06T05:15:21.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7ff5c252f700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15d4b0 0x7ff5bc0aca20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000ea80 tx=0x7ff5b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800c840 con 0x7f5ad00aff60 2023-10-06T05:15:21.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac800c9a0 con 0x7f5ad00aff60 2023-10-06T05:15:21.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800cd60 con 0x7f5ad00aff60 2023-10-06T05:15:21.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad00ab2d0 con 0x7f5ad00aff60 2023-10-06T05:15:21.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad00a58c0 con 0x7f5ad00aff60 2023-10-06T05:15:21.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f0082ffd700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c004cd0 0x7f00840cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f00827fc700 1 -- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00841568e0 msgr2=0x7f0084153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f00827fc700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00841568e0 0x7f0084153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f00827fc700 1 -- 172.21.15.45:0/3755284696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c004cd0 msgr2=0x7f00840cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7fb2dda49700 1 -- start start 2023-10-06T05:15:21.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f00827fc700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c004cd0 0x7f00840cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f00827fc700 1 -- 172.21.15.45:0/3755284696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084153cd0 con 0x7f0084148720 2023-10-06T05:15:21.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.830+0000 7f8cb45a8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8cac06c050 0x7f8cac0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb45a8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cac1553b0 0x7f8cac0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb45a8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac0aeb00 con 0x7f8cac154fe0 2023-10-06T05:15:21.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb45a8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0aed50 con 0x7f8cac1553b0 2023-10-06T05:15:21.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb45a8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0af030 con 0x7f8cac06c050 2023-10-06T05:15:21.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb2344700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8cac154fe0 0x7f8cac06a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50834/0 (socket says 172.21.15.45:50834) 2023-10-06T05:15:21.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb2344700 1 -- 172.21.15.45:0/4033842104 learned_addr learned my addr 172.21.15.45:0/4033842104 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:21.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb1b43700 1 --2- 172.21.15.45:0/4033842104 >> v2:172.21.15.144:3300/0 conn(0x7f8cac06c050 0x7f8cac0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb2b45700 1 --2- 172.21.15.45:0/4033842104 >> v2:172.21.15.45:3300/0 conn(0x7f8cac1553b0 0x7f8cac0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567797174 0 0) 0x7f8cac0aeb00 con 0x7f8cac154fe0 2023-10-06T05:15:21.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f00827fc700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084148720 0x7f00841563a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f007800c790 tx=0x7f0078007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.831+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c98003570 con 0x7f8cac154fe0 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2861136504 0 0) 0x7f8c98003570 con 0x7f8cac154fe0 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 >> v2:172.21.15.144:3300/0 conn(0x7f8cac06c050 msgr2=0x7f8cac0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 --2- 172.21.15.45:0/4033842104 >> v2:172.21.15.144:3300/0 conn(0x7f8cac06c050 0x7f8cac0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 >> v2:172.21.15.45:3300/0 conn(0x7f8cac1553b0 msgr2=0x7f8cac0b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 --2- 172.21.15.45:0/4033842104 >> v2:172.21.15.45:3300/0 conn(0x7f8cac1553b0 0x7f8cac0b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0af310 con 0x7f8cac154fe0 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f8ca8001470 con 0x7f8cac154fe0 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 >> v1:172.21.15.144:6789/0 conn(0x7f8cac154fe0 legacy=0x7f8cac06a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c98003ae0 0x7f8c98003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c98004810 0x7f8c98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8c9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0aed50 con 0x7f8c98003ae0 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0af030 con 0x7f8c98004810 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9800c100 con 0x7f8c980051c0 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb2b45700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8c9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb1b43700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c98004810 0x7f8c98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb2344700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c98003ae0 0x7f8c98003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb2b45700 1 -- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c98004810 msgr2=0x7f8c98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb2b45700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c98004810 0x7f8c98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb2b45700 1 -- 172.21.15.45:0/4033842104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c98003ae0 msgr2=0x7f8c98003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb2b45700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c98003ae0 0x7f8c98003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.832+0000 7f8cb2b45700 1 -- 172.21.15.45:0/4033842104 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0af310 con 0x7f8c980051c0 2023-10-06T05:15:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f8cb2b45700 1 -- 172.21.15.45:0/4033842104 reap_dead start 2023-10-06T05:15:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f8cb2344700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c98003ae0 0x7f8c98003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f8cb2b45700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8c9800ba60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4007480 tx=0x7f8ca4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36282b1335a828fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078012630 con 0x7f0084148720 2023-10-06T05:15:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f8512ffd700 1 -- 172.21.15.45:0/3377233237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504010070 con 0x7f85140c7f90 2023-10-06T05:15:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f8512ffd700 1 -- 172.21.15.45:0/3377233237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850400dd10 con 0x7f85140c7f90 2023-10-06T05:15:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f8512ffd700 1 -- 172.21.15.45:0/3377233237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400bea0 con 0x7f85140c7f90 2023-10-06T05:15:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7fb2dda49700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d8068130 0x7fb2d80c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7fb2dda49700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d80af1f0 con 0x7fb2d8159720 2023-10-06T05:15:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7fb2dda49700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d8159bf0 con 0x7fb2d80c7f00 2023-10-06T05:15:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7fb2dda49700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80cbf80 con 0x7fb2d8068130 2023-10-06T05:15:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d67fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb2d8159720 0x7fb2d8154e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58590/0 (socket says 172.21.15.45:58590) 2023-10-06T05:15:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d67fc700 1 -- 172.21.15.45:0/1769753755 learned_addr learned my addr 172.21.15.45:0/1769753755 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:21.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000c850 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b000c9d0 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000cd90 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4007d30 con 0x7f8c980051c0 2023-10-06T05:15:21.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca400f040 con 0x7f8c980051c0 2023-10-06T05:15:21.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f8cb1342700 1 -- 172.21.15.45:0/4033842104 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4013640 con 0x7f8c980051c0 2023-10-06T05:15:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f7a48411700 1 -- 172.21.15.45:0/162263291 shutdown_connections 2023-10-06T05:15:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f7a48411700 1 --2- 172.21.15.45:0/162263291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015f650 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f7a48411700 1 --2- 172.21.15.45:0/162263291 >> v2:172.21.15.45:3300/0 conn(0x7f7a400b0130 0x7f7a40069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f7a48411700 1 -- 172.21.15.45:0/162263291 >> 172.21.15.45:0/162263291 conn(0x7f7a40067620 msgr2=0x7f7a40067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0078012790 con 0x7f0084148720 2023-10-06T05:15:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.833+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078012b30 con 0x7f0084148720 2023-10-06T05:15:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f843bfff700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424004810 0x7f843c1521b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f842c001ac0 tx=0x7f842c007940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc0ab2d0 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c018030 con 0x7f8424004810 2023-10-06T05:15:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c004e00 con 0x7f8424004810 2023-10-06T05:15:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c150d20 con 0x7f8424004810 2023-10-06T05:15:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00feb0 con 0x7f8424004810 2023-10-06T05:15:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c1512e0 con 0x7f8424004810 2023-10-06T05:15:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f843c06e2f0 con 0x7f8424004810 2023-10-06T05:15:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ad006cec0 con 0x7f5ad00aff60 2023-10-06T05:15:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d77fe700 1 --2- 172.21.15.45:0/1769753755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d8068130 0x7fb2d80c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d5ffb700 1 -- 172.21.15.45:0/1769753755 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084038248 0 0) 0x7fb2d80af1f0 con 0x7fb2d8159720 2023-10-06T05:15:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d5ffb700 1 -- 172.21.15.45:0/1769753755 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c0003570 con 0x7fb2d8159720 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d5ffb700 1 -- 172.21.15.45:0/1769753755 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576413679 0 0) 0x7fb2d8159bf0 con 0x7fb2d80c7f00 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d5ffb700 1 -- 172.21.15.45:0/1769753755 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d80af1f0 con 0x7fb2d80c7f00 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d77fe700 1 -- 172.21.15.45:0/1769753755 >> v1:172.21.15.144:6789/0 conn(0x7fb2d80c7f00 legacy=0x7fb2d8159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d77fe700 1 -- 172.21.15.45:0/1769753755 >> v1:172.21.15.45:6789/0 conn(0x7fb2d8159720 legacy=0x7fb2d8154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7fb2d77fe700 1 -- 172.21.15.45:0/1769753755 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8159ed0 con 0x7fb2d8068130 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7fb2d77fe700 1 --2- 172.21.15.45:0/1769753755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d8068130 0x7fb2d80c82d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009910 tx=0x7fb2cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc6c5dc9eb7209 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7fb2d5ffb700 1 -- 172.21.15.45:0/1769753755 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc011630 con 0x7fb2d8068130 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7fb2d5ffb700 1 -- 172.21.15.45:0/1769753755 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc011790 con 0x7fb2d8068130 2023-10-06T05:15:21.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7fb2d5ffb700 1 -- 172.21.15.45:0/1769753755 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc011b30 con 0x7fb2d8068130 2023-10-06T05:15:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.834+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0084153fb0 con 0x7f0084148720 2023-10-06T05:15:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00840af520 con 0x7f0084148720 2023-10-06T05:15:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc0ab890 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5bc064a60 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8cb45a8700 1 -- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 msgr2=0x7f8c9800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8cb45a8700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8c9800ba60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4007480 tx=0x7f8ca4009780 comp rx=0 tx=0).stop 2023-10-06T05:15:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8cb45a8700 1 -- 172.21.15.45:0/4033842104 shutdown_connections 2023-10-06T05:15:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8cb45a8700 1 --2- 172.21.15.45:0/4033842104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8c9800ba60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8cb45a8700 1 -- 172.21.15.45:0/4033842104 >> 172.21.15.45:0/4033842104 conn(0x7f8cac067620 msgr2=0x7f8cac067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8cb45a8700 1 -- 172.21.15.45:0/4033842104 shutdown_connections 2023-10-06T05:15:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8cb45a8700 1 -- 172.21.15.45:0/4033842104 wait complete. 2023-10-06T05:15:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f7a48411700 1 -- 172.21.15.45:0/162263291 shutdown_connections 2023-10-06T05:15:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f7a48411700 1 -- 172.21.15.45:0/162263291 wait complete. 2023-10-06T05:15:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8519c58700 1 -- 172.21.15.45:0/3377233237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85140c7f90 msgr2=0x7f8514159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.835+0000 7f8519c58700 1 --2- 172.21.15.45:0/3377233237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85140c7f90 0x7f8514159010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f850400d280 tx=0x7f85040076f0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7f8519c58700 1 -- 172.21.15.45:0/3377233237 shutdown_connections 2023-10-06T05:15:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7f8519c58700 1 --2- 172.21.15.45:0/3377233237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85140c7f90 0x7f8514159010 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7f8519c58700 1 -- 172.21.15.45:0/3377233237 >> 172.21.15.45:0/3377233237 conn(0x7f8514067620 msgr2=0x7f8514067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7fb2dda49700 1 -- 172.21.15.45:0/1769753755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d8068130 msgr2=0x7fb2d80c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7fb2dda49700 1 --2- 172.21.15.45:0/1769753755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d8068130 0x7fb2d80c82d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009910 tx=0x7fb2cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.841+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f842c01b070 con 0x7f8424004810 2023-10-06T05:15:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.841+0000 7f8423fff700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8428067080 0x7f8428069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.841+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f842c019070 con 0x7f8424004810 2023-10-06T05:15:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8423fff700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f8428070d50 0x7f8428073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8428073850 con 0x7f8428070d50 2023-10-06T05:15:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f842c085660 con 0x7f8424004810 2023-10-06T05:15:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.836+0000 7f8519c58700 1 -- 172.21.15.45:0/3377233237 shutdown_connections 2023-10-06T05:15:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7f8519c58700 1 -- 172.21.15.45:0/3377233237 wait complete. 2023-10-06T05:15:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8519c58700 1 Processor -- start 2023-10-06T05:15:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7fb2dda49700 1 -- 172.21.15.45:0/1769753755 shutdown_connections 2023-10-06T05:15:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7fb2dda49700 1 --2- 172.21.15.45:0/1769753755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d8068130 0x7fb2d80c82d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7fb2dda49700 1 -- 172.21.15.45:0/1769753755 >> 172.21.15.45:0/1769753755 conn(0x7fb2d8067620 msgr2=0x7fb2d8067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7fb2dda49700 1 -- 172.21.15.45:0/1769753755 shutdown_connections 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7fb2dda49700 1 -- 172.21.15.45:0/1769753755 wait complete. 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0084064a60 con 0x7f0084148720 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f007801b400 con 0x7f0084148720 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f006affd700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0070066fb0 0x7f0070069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0078084b80 con 0x7f0084148720 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f006affd700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f0070070c80 0x7f00700730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0070073780 con 0x7f0070070c80 2023-10-06T05:15:21.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0078089220 con 0x7f0084148720 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f0082ffd700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0070066fb0 0x7f0070069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f00837fe700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f0070070c80 0x7f00700730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f00837fe700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f0070070c80 0x7f00700730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f0082ffd700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0070066fb0 0x7f0070069460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f007c000f80 tx=0x7f007c035000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.841+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0070073780 con 0x7f0070070c80 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7f8cb45a8700 1 Processor -- start 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7f8cb45a8700 1 -- start start 2023-10-06T05:15:21.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb45a8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8cac147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb45a8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cac14b100 0x7f8cac134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb45a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cac14b570 0x7f8cac134660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb45a8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac147b70 con 0x7f8cac14b100 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb45a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac14ac70 con 0x7f8cac14b570 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb45a8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac14a060 con 0x7f8c980051c0 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb2344700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8cac147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb2b45700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cac14b570 0x7f8cac134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb2344700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8cac147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33914/0 (socket says 172.21.15.45:33914) 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb2344700 1 -- 172.21.15.45:0/832307708 learned_addr learned my addr 172.21.15.45:0/832307708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f8cb1b43700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cac14b100 0x7f8cac134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8cb2344700 1 -- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cac14b570 msgr2=0x7f8cac134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8cb2344700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cac14b570 0x7f8cac134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8cb2344700 1 -- 172.21.15.45:0/832307708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cac14b100 msgr2=0x7f8cac134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8cb2344700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cac14b100 0x7f8cac134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8cb2344700 1 -- 172.21.15.45:0/832307708 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac14a340 con 0x7f8c980051c0 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8cb1b43700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cac14b100 0x7f8cac134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f8cb2344700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8cac147040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800ec30 tx=0x7f8ca80014f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8018070 con 0x7f8c980051c0 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca80061a0 con 0x7f8c980051c0 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8006560 con 0x7f8c980051c0 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac14a620 con 0x7f8c980051c0 2023-10-06T05:15:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac147580 con 0x7f8c980051c0 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.841+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cac06cec0 con 0x7f8c980051c0 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ca8004a40 con 0x7f8c980051c0 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7f7a48411700 1 Processor -- start 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.837+0000 7f7a48411700 1 -- start start 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f7a48411700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f7a48411700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a400b0130 0x7f7a4016d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f7a48411700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a40146f80 0x7f7a4016d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f7a48411700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400cc060 con 0x7f7a40146f80 2023-10-06T05:15:21.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f7a48411700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4016dfa0 con 0x7f7a4006a520 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.838+0000 7f7a48411700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4016e280 con 0x7f7a400b0130 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a459ac700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a400b0130 0x7f7a4016d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47432/0 (socket says 172.21.15.45:47432) 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 -- 172.21.15.45:0/974124887 learned_addr learned my addr 172.21.15.45:0/974124887 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 -- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a400b0130 msgr2=0x7f7a4016d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a400b0130 0x7f7a4016d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 -- 172.21.15.45:0/974124887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a40146f80 msgr2=0x7f7a4016d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a40146f80 0x7f7a4016d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 -- 172.21.15.45:0/974124887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c0094f0 con 0x7f7a4006a520 2023-10-06T05:15:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.839+0000 7f7a461ad700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015cdf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800bbb0 tx=0x7f7a38009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380092e0 con 0x7f7a4006a520 2023-10-06T05:15:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3800c6f0 con 0x7f7a4006a520 2023-10-06T05:15:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800ca90 con 0x7f7a4006a520 2023-10-06T05:15:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7fb2dda49700 1 Processor -- start 2023-10-06T05:15:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f843b7fe700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8428067080 0x7f8428069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a400a44a0 con 0x7f7a4006a520 2023-10-06T05:15:21.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.840+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a400a4a60 con 0x7f7a4006a520 2023-10-06T05:15:21.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a4006cec0 con 0x7f7a4006a520 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8ca27fc700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c90066fb0 0x7f8c90069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8cb1b43700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c90066fb0 0x7f8c90069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8ca8084580 con 0x7f8c980051c0 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8ca27fc700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f8c90070c80 0x7f8c900730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c90073780 con 0x7f8c90070c80 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ca8086b20 con 0x7f8c980051c0 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8cb2b45700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f8c90070c80 0x7f8c900730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8cb1b43700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c90066fb0 0x7f8c90069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b0b0 tx=0x7f8c9c00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7fb2dda49700 1 -- start start 2023-10-06T05:15:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7fb2dda49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d8068130 0x7fb2d81453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7fb2dda49700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d80c7f00 0x7fb2d8145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7fb2dda49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8159720 0x7fb2d80ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7fb2dda49700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8147b40 con 0x7fb2d8159720 2023-10-06T05:15:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7fb2dda49700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8146460 con 0x7fb2d8068130 2023-10-06T05:15:21.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7fb2dda49700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8146740 con 0x7fb2d80c7f00 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8cb2b45700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f8c90070c80 0x7f8c900730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.842+0000 7f8519c58700 1 -- start start 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8519c58700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85140c7f90 0x7f85140af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8519c58700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85140c8360 0x7f8514153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8519c58700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514155590 0x7f85140aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8519c58700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514179d00 con 0x7f85140c7f90 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8519c58700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514179fe0 con 0x7f8514155590 2023-10-06T05:15:21.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f8519c58700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851417a2c0 con 0x7f85140c8360 2023-10-06T05:15:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.843+0000 7f843b7fe700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8428067080 0x7f8428069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f843c153f60 tx=0x7f8430015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85140c8360 0x7f8514153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85140c8360 0x7f8514153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33938/0 (socket says 172.21.15.45:33938) 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 -- 172.21.15.45:0/3516564777 learned_addr learned my addr 172.21.15.45:0/3516564777 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f8513fff700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514155590 0x7f85140aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 -- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514155590 msgr2=0x7f85140aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514155590 0x7f85140aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 -- 172.21.15.45:0/3516564777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85140c7f90 msgr2=0x7f85140af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85140c7f90 0x7f85140af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 -- 172.21.15.45:0/3516564777 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504007990 con 0x7f85140c8360 2023-10-06T05:15:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f8513fff700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514155590 0x7f85140aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8c90073780 con 0x7f8c90070c80 2023-10-06T05:15:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850affd700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85140c8360 0x7f8514153680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c810 tx=0x7f84fc007ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc012630 con 0x7f85140c8360 2023-10-06T05:15:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc0127b0 con 0x7f85140c8360 2023-10-06T05:15:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc012b70 con 0x7f85140c8360 2023-10-06T05:15:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851417a5a0 con 0x7f85140c8360 2023-10-06T05:15:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851414e410 con 0x7f85140c8360 2023-10-06T05:15:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.844+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a38012070 con 0x7f7a4006a520 2023-10-06T05:15:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f7a367fc700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a24066f80 0x7f7a24069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7a380847e0 con 0x7f7a4006a520 2023-10-06T05:15:21.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f7a367fc700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f7a24070c50 0x7f7a24073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a24073750 con 0x7f7a24070c50 2023-10-06T05:15:21.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a38084b10 con 0x7f7a4006a520 2023-10-06T05:15:21.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f7a459ac700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a24066f80 0x7f7a24069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f7a469ae700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f7a24070c50 0x7f7a24073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f851406cec0 con 0x7f85140c8360 2023-10-06T05:15:21.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f7a469ae700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f7a24070c50 0x7f7a24073080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8159720 0x7fb2d80ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8159720 0x7fb2d80ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53228/0 (socket says 172.21.15.45:53228) 2023-10-06T05:15:21.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 -- 172.21.15.45:0/2269862003 learned_addr learned my addr 172.21.15.45:0/2269862003 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:21.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f7a459ac700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a24066f80 0x7f7a24069430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d67fc700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d80c7f00 0x7fb2d8145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d6ffd700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d8068130 0x7fb2d81453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.845+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ac8006d50 con 0x7f5ad00aff60 2023-10-06T05:15:21.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f5ab7fff700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5abc066fb0 0x7f5abc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5ac8084800 con 0x7f5ad00aff60 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f5ab7fff700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f5abc070c80 0x7f5abc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5abc073780 con 0x7f5abc070c80 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ac8086da0 con 0x7f5ad00aff60 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 -- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d80c7f00 msgr2=0x7fb2d8145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d80c7f00 0x7fb2d8145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 -- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d8068130 msgr2=0x7fb2d81453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d8068130 0x7fb2d81453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7fb2d77fe700 1 -- 172.21.15.45:0/2269862003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc009420 con 0x7fb2d8159720 2023-10-06T05:15:21.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f5ad4e83700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f5abc070c80 0x7f5abc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7fb2d77fe700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8159720 0x7fb2d80ad380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009fd0 tx=0x7fb2cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.846+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84fc01b400 con 0x7f85140c8360 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7f850bfff700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84f4066f30 0x7f84f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f84fc084960 con 0x7f85140c8360 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7f850bfff700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7f84f4070c00 0x7f84f4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84f4073700 con 0x7f84f4070c00 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84fc084c50 con 0x7f85140c8360 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7f85137fe700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84f4066f30 0x7f84f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7f8440c98700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f8428070d50 0x7f8428073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7f85137fe700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84f4066f30 0x7f84f40693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8504007910 tx=0x7f850401a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7f8440c98700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f8428070d50 0x7f8428073180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7f8513fff700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7f84f4070c00 0x7f84f4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7f5ad4e83700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f5abc070c80 0x7f5abc0730b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8428073850 con 0x7f8428070d50 2023-10-06T05:15:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7f8513fff700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7f84f4070c00 0x7f84f4073030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc01be60 con 0x7fb2d8159720 2023-10-06T05:15:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc020070 con 0x7fb2d8159720 2023-10-06T05:15:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0276b0 con 0x7fb2d8159720 2023-10-06T05:15:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.847+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5b0013680 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7ff5adffb700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5a0066fb0 0x7ff5a0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff5b00850b0 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7ff5adffb700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7ff5a0070c80 0x7ff5a00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5a0073780 con 0x7ff5a0070c80 2023-10-06T05:15:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.848+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b00853e0 con 0x7ff5bc15d4b0 2023-10-06T05:15:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.852+0000 7ff5c152d700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7ff5a0070c80 0x7ff5a00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.852+0000 7ff5c1d2e700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5a0066fb0 0x7ff5a0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d81477d0 con 0x7fb2d8159720 2023-10-06T05:15:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d81460c0 con 0x7fb2d8159720 2023-10-06T05:15:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.850+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2d806cec0 con 0x7fb2d8159720 2023-10-06T05:15:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.853+0000 7ff5c1d2e700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5a0066fb0 0x7ff5a0069460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4007f10 tx=0x7ff5b40073e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.853+0000 7ff5c152d700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7ff5a0070c80 0x7ff5a00730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.849+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f5abc073780 con 0x7f5abc070c80 2023-10-06T05:15:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.852+0000 7f5acffff700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5abc066fb0 0x7f5abc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.853+0000 7f5acffff700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5abc066fb0 0x7f5abc069460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0003080 tx=0x7f5ac0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.853+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f84f4073700 con 0x7f84f4070c00 2023-10-06T05:15:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.854+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff5a0073780 con 0x7ff5a0070c80 2023-10-06T05:15:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.850+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7a24073750 con 0x7f7a24070c50 2023-10-06T05:15:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.864+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cac04bb40 con 0x7f8c90070c80 2023-10-06T05:15:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.869+0000 7f8ca27fc700 1 -- 172.21.15.45:0/832307708 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8cac04bb40 con 0x7f8c90070c80 2023-10-06T05:15:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.869+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ad006b2d0 con 0x7f5abc070c80 2023-10-06T05:15:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.869+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f8c90070c80 msgr2=0x7f8c900730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.869+0000 7f8cb45a8700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f8c90070c80 0x7f8c900730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c90066fb0 msgr2=0x7f8c90069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c90066fb0 0x7f8c90069460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b0b0 tx=0x7f8c9c00b040 comp rx=0 tx=0).stop 2023-10-06T05:15:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 msgr2=0x7f8cac147040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 --2- 172.21.15.45:0/832307708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c980051c0 0x7f8cac147040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800ec30 tx=0x7f8ca80014f0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb2b45700 1 -- 172.21.15.45:0/832307708 reap_dead start 2023-10-06T05:15:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 shutdown_connections 2023-10-06T05:15:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 >> 172.21.15.45:0/832307708 conn(0x7f8cac067620 msgr2=0x7f8cac0caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f5ab7fff700 1 -- 172.21.15.45:0/4270873643 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ad006b2d0 con 0x7f5abc070c80 2023-10-06T05:15:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 shutdown_connections 2023-10-06T05:15:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f8cb45a8700 1 -- 172.21.15.45:0/832307708 wait complete. 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f5abc070c80 msgr2=0x7f5abc0730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.870+0000 7f5ad68e6700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f5abc070c80 0x7f5abc0730b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.871+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5abc066fb0 msgr2=0x7f5abc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.871+0000 7f5ad68e6700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5abc066fb0 0x7f5abc069460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0003080 tx=0x7f5ac0016000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.871+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad00aff60 msgr2=0x7f5ad015b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.871+0000 7f5ad68e6700 1 --2- 172.21.15.45:0/4270873643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad00aff60 0x7f5ad015b670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800bb00 tx=0x7f5ac8009780 comp rx=0 tx=0).stop 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.871+0000 7f5ad4e83700 1 -- 172.21.15.45:0/4270873643 reap_dead start 2023-10-06T05:15:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.873+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 shutdown_connections 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.873+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 >> 172.21.15.45:0/4270873643 conn(0x7f5ad0067620 msgr2=0x7f5ad00ad100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.874+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 shutdown_connections 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.874+0000 7f5ad68e6700 1 -- 172.21.15.45:0/4270873643 wait complete. 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.874+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2cc01c450 con 0x7fb2d8159720 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.874+0000 7fb2beffd700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb2c4066fb0 0x7fb2c4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.874+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb2cc022070 con 0x7fb2d8159720 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.874+0000 7fb2beffd700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fb2c4070c80 0x7fb2c40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.875+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2c4073780 con 0x7fb2c4070c80 2023-10-06T05:15:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.875+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2cc086f20 con 0x7fb2d8159720 2023-10-06T05:15:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.875+0000 7fb2d67fc700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fb2c4070c80 0x7fb2c40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.875+0000 7fb2d6ffd700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb2c4066fb0 0x7fb2c4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7fb2d67fc700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fb2c4070c80 0x7fb2c40730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7fb2d6ffd700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb2c4066fb0 0x7fb2c4069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0007fd0 tx=0x7fb2d00016c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.874+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f00840ad720 con 0x7f0070070c80 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.875+0000 7f006affd700 1 -- 172.21.15.45:0/3755284696 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f00840ad720 con 0x7f0070070c80 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.875+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f0070070c80 msgr2=0x7f00700730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.875+0000 7f0083fff700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f0070070c80 0x7f00700730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0070066fb0 msgr2=0x7f0070069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0070066fb0 0x7f0070069460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f007c000f80 tx=0x7f007c035000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084148720 msgr2=0x7f00841563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 --2- 172.21.15.45:0/3755284696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084148720 0x7f00841563a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f007800c790 tx=0x7f0078007f40 comp rx=0 tx=0).stop 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f00837fe700 1 -- 172.21.15.45:0/3755284696 reap_dead start 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 shutdown_connections 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 >> 172.21.15.45:0/3755284696 conn(0x7f00840ce800 msgr2=0x7f0084151560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 shutdown_connections 2023-10-06T05:15:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.876+0000 7f0083fff700 1 -- 172.21.15.45:0/3755284696 wait complete. 2023-10-06T05:15:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.882+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a4015ee50 con 0x7f7a24070c50 2023-10-06T05:15:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.882+0000 7f7a367fc700 1 -- 172.21.15.45:0/974124887 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7a4015ee50 con 0x7f7a24070c50 2023-10-06T05:15:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.883+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== osd.3 v2:172.21.15.45:6824/119762 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb2c4073780 con 0x7fb2c4070c80 2023-10-06T05:15:21.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.886+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f7a24070c50 msgr2=0x7f7a24073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.886+0000 7f7a48411700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f7a24070c50 0x7f7a24073080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a24066f80 msgr2=0x7f7a24069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f7a48411700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a24066f80 0x7f7a24069430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30008040 comp rx=0 tx=0).stop 2023-10-06T05:15:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 msgr2=0x7f7a4015cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f7a48411700 1 --2- 172.21.15.45:0/974124887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4006a520 0x7f7a4015cdf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800bbb0 tx=0x7f7a38009840 comp rx=0 tx=0).stop 2023-10-06T05:15:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f7a469ae700 1 -- 172.21.15.45:0/974124887 reap_dead start 2023-10-06T05:15:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 shutdown_connections 2023-10-06T05:15:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 >> 172.21.15.45:0/974124887 conn(0x7f7a40067620 msgr2=0x7f7a400ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.885+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85140aeba0 con 0x7f84f4070c00 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.886+0000 7f850bfff700 1 -- 172.21.15.45:0/3516564777 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85140aeba0 con 0x7f84f4070c00 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7f84f4070c00 msgr2=0x7f84f4073030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7f84f4070c00 0x7f84f4073030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84f4066f30 msgr2=0x7f84f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84f4066f30 0x7f84f40693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8504007910 tx=0x7f850401a000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85140c8360 msgr2=0x7f8514153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 --2- 172.21.15.45:0/3516564777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85140c8360 0x7f8514153680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c810 tx=0x7f84fc007ee0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8513fff700 1 -- 172.21.15.45:0/3516564777 reap_dead start 2023-10-06T05:15:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 shutdown_connections 2023-10-06T05:15:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.888+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 >> 172.21.15.45:0/3516564777 conn(0x7f8514067620 msgr2=0x7f851415a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.889+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 shutdown_connections 2023-10-06T05:15:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.889+0000 7f8519c58700 1 -- 172.21.15.45:0/3516564777 wait complete. 2023-10-06T05:15:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.887+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f843c04bb40 con 0x7f8428070d50 2023-10-06T05:15:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.891+0000 7f8423fff700 1 -- 172.21.15.45:0/1422957083 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f843c04bb40 con 0x7f8428070d50 2023-10-06T05:15:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 shutdown_connections 2023-10-06T05:15:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f8428070d50 msgr2=0x7f8428073180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f84426fb700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f8428070d50 0x7f8428073180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8428067080 msgr2=0x7f8428069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f84426fb700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8428067080 0x7f8428069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f843c153f60 tx=0x7f8430015000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424004810 msgr2=0x7f843c1521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f84426fb700 1 --2- 172.21.15.45:0/1422957083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8424004810 0x7f843c1521b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f842c001ac0 tx=0x7f842c007940 comp rx=0 tx=0).stop 2023-10-06T05:15:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.892+0000 7f8440c98700 1 -- 172.21.15.45:0/1422957083 reap_dead start 2023-10-06T05:15:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.893+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 shutdown_connections 2023-10-06T05:15:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.893+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 >> 172.21.15.45:0/1422957083 conn(0x7f843c06f090 msgr2=0x7f843c0ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.918 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2023-10-06T05:15:21.919 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.893+0000 7f7a48411700 1 -- 172.21.15.45:0/974124887 wait complete. 2023-10-06T05:15:21.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T05:15:21.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T05:15:21.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.894+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 shutdown_connections 2023-10-06T05:15:21.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.894+0000 7f84426fb700 1 -- 172.21.15.45:0/1422957083 wait complete. 2023-10-06T05:15:21.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.894+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5bc068080 con 0x7ff5a0070c80 2023-10-06T05:15:21.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.895+0000 7ff5adffb700 1 -- 172.21.15.45:0/1868612478 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5bc068080 con 0x7ff5a0070c80 2023-10-06T05:15:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.895+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7ff5a0070c80 msgr2=0x7ff5a00730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.895+0000 7ff5c2d30700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7ff5a0070c80 0x7ff5a00730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.896+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5a0066fb0 msgr2=0x7ff5a0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.896+0000 7ff5c2d30700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5a0066fb0 0x7ff5a0069460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4007f10 tx=0x7ff5b40073e0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.896+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15d4b0 msgr2=0x7ff5bc0aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.896+0000 7ff5c2d30700 1 --2- 172.21.15.45:0/1868612478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15d4b0 0x7ff5bc0aca20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000ea80 tx=0x7ff5b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.896+0000 7ff5c252f700 1 -- 172.21.15.45:0/1868612478 reap_dead start 2023-10-06T05:15:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.896+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 shutdown_connections 2023-10-06T05:15:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.896+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 >> 172.21.15.45:0/1868612478 conn(0x7ff5bc0ccf10 msgr2=0x7ff5bc1610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.923 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:21.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T05:15:21.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.898+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 shutdown_connections 2023-10-06T05:15:21.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.898+0000 7ff5c2d30700 1 -- 172.21.15.45:0/1868612478 wait complete. 2023-10-06T05:15:21.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.906+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2d8146880 con 0x7fb2c4070c80 2023-10-06T05:15:21.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.906+0000 7fb2beffd700 1 -- 172.21.15.45:0/2269862003 <== osd.3 v2:172.21.15.45:6824/119762 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb2d8146880 con 0x7fb2c4070c80 2023-10-06T05:15:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.907+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fb2c4070c80 msgr2=0x7fb2c40730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.907+0000 7fb2dda49700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fb2c4070c80 0x7fb2c40730b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.907+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb2c4066fb0 msgr2=0x7fb2c4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.907+0000 7fb2dda49700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb2c4066fb0 0x7fb2c4069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0007fd0 tx=0x7fb2d00016c0 comp rx=0 tx=0).stop 2023-10-06T05:15:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.907+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8159720 msgr2=0x7fb2d80ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.907+0000 7fb2dda49700 1 --2- 172.21.15.45:0/2269862003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d8159720 0x7fb2d80ad380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009fd0 tx=0x7fb2cc016000 comp rx=0 tx=0).stop 2023-10-06T05:15:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.907+0000 7fb2d77fe700 1 -- 172.21.15.45:0/2269862003 reap_dead start 2023-10-06T05:15:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.915+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 shutdown_connections 2023-10-06T05:15:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.915+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 >> 172.21.15.45:0/2269862003 conn(0x7fb2d8067620 msgr2=0x7fb2d814eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.915+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 shutdown_connections 2023-10-06T05:15:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:21.915+0000 7fb2dda49700 1 -- 172.21.15.45:0/2269862003 wait complete. 2023-10-06T05:15:21.927 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2023-10-06T05:15:21.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T05:15:21.928 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2023-10-06T05:15:21.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T05:15:21.929 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:21.930 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T05:15:21.931 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:21.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T05:15:21.932 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:21.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T05:15:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.131+0000 7f2928e86700 1 Processor -- start 2023-10-06T05:15:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.132+0000 7f2928e86700 1 -- start start 2023-10-06T05:15:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.134+0000 7f2928e86700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29240c7f90 0x7f2924159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.134+0000 7f2928e86700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 0x7f2924159550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.134+0000 7f2928e86700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2924154e80 0x7f2924155300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.134+0000 7f2928e86700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240cbf80 con 0x7f29240c7f90 2023-10-06T05:15:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.134+0000 7f2928e86700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240af1f0 con 0x7f2924154e80 2023-10-06T05:15:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.134+0000 7f2928e86700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2924159bf0 con 0x7f29240c8360 2023-10-06T05:15:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.135+0000 7f2922d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2924154e80 0x7f2924155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.135+0000 7f292259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29240c7f90 0x7f2924159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.135+0000 7f292259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29240c7f90 0x7f2924159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53258/0 (socket says 172.21.15.45:53258) 2023-10-06T05:15:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.135+0000 7f292259c700 1 -- 172.21.15.45:0/1264233386 learned_addr learned my addr 172.21.15.45:0/1264233386 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.136+0000 7f292259c700 1 -- 172.21.15.45:0/1264233386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 msgr2=0x7f2924159550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:15:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.136+0000 7f292259c700 1 --2- 172.21.15.45:0/1264233386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 0x7f2924159550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.136+0000 7f292259c700 1 -- 172.21.15.45:0/1264233386 >> v2:172.21.15.144:3300/0 conn(0x7f2924154e80 msgr2=0x7f2924155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.136+0000 7f292259c700 1 --2- 172.21.15.45:0/1264233386 >> v2:172.21.15.144:3300/0 conn(0x7f2924154e80 0x7f2924155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.136+0000 7f292259c700 1 -- 172.21.15.45:0/1264233386 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924159ed0 con 0x7f29240c7f90 2023-10-06T05:15:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.136+0000 7f292259c700 1 --2- 172.21.15.45:0/1264233386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29240c7f90 0x7f2924159010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f29240af160 tx=0x7f2914009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba99f102b71d45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.137+0000 7f292159a700 1 -- 172.21.15.45:0/1264233386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291400c970 con 0x7f29240c7f90 2023-10-06T05:15:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.137+0000 7f292159a700 1 -- 172.21.15.45:0/1264233386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291400cad0 con 0x7f29240c7f90 2023-10-06T05:15:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.137+0000 7f292159a700 1 -- 172.21.15.45:0/1264233386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291400ce50 con 0x7f29240c7f90 2023-10-06T05:15:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.138+0000 7f2928e86700 1 -- 172.21.15.45:0/1264233386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29240c7f90 msgr2=0x7f2924159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.138+0000 7f2928e86700 1 --2- 172.21.15.45:0/1264233386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29240c7f90 0x7f2924159010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f29240af160 tx=0x7f2914009ae0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.139+0000 7f2928e86700 1 -- 172.21.15.45:0/1264233386 shutdown_connections 2023-10-06T05:15:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.139+0000 7f2928e86700 1 --2- 172.21.15.45:0/1264233386 >> v2:172.21.15.144:3300/0 conn(0x7f2924154e80 0x7f2924155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.139+0000 7f2928e86700 1 --2- 172.21.15.45:0/1264233386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29240c7f90 0x7f2924159010 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.139+0000 7f2928e86700 1 --2- 172.21.15.45:0/1264233386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 0x7f2924159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.139+0000 7f2928e86700 1 -- 172.21.15.45:0/1264233386 >> 172.21.15.45:0/1264233386 conn(0x7f2924067620 msgr2=0x7f2924067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.139+0000 7f2928e86700 1 -- 172.21.15.45:0/1264233386 shutdown_connections 2023-10-06T05:15:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.139+0000 7f2928e86700 1 -- 172.21.15.45:0/1264233386 wait complete. 2023-10-06T05:15:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.141+0000 7f2928e86700 1 Processor -- start 2023-10-06T05:15:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.141+0000 7f2928e86700 1 -- start start 2023-10-06T05:15:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2928e86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29240c7f90 0x7f2924171a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2928e86700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 0x7f2924171f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2928e86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2924154e80 0x7f2924170500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2928e86700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2924155710 con 0x7f2924154e80 2023-10-06T05:15:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2928e86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29241717b0 con 0x7f29240c7f90 2023-10-06T05:15:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2928e86700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29241711f0 con 0x7f29240c8360 2023-10-06T05:15:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2922d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2924154e80 0x7f2924170500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2922d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2924154e80 0x7f2924170500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53262/0 (socket says 172.21.15.45:53262) 2023-10-06T05:15:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.142+0000 7f2922d9d700 1 -- 172.21.15.45:0/865147469 learned_addr learned my addr 172.21.15.45:0/865147469 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.143+0000 7f2922d9d700 1 -- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 msgr2=0x7f2924171f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.143+0000 7f2922d9d700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 0x7f2924171f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.143+0000 7f2922d9d700 1 -- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29240c7f90 msgr2=0x7f2924171a40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.143+0000 7f2922d9d700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29240c7f90 0x7f2924171a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.143+0000 7f2922d9d700 1 -- 172.21.15.45:0/865147469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914009d80 con 0x7f2924154e80 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.143+0000 7f2922d9d700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2924154e80 0x7f2924170500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f291c00cdb0 tx=0x7f291c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.143+0000 7f29127fc700 1 -- 172.21.15.45:0/865147469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c00aa80 con 0x7f2924154e80 2023-10-06T05:15:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.144+0000 7f29127fc700 1 -- 172.21.15.45:0/865147469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c00abe0 con 0x7f2924154e80 2023-10-06T05:15:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.144+0000 7f29127fc700 1 -- 172.21.15.45:0/865147469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c017430 con 0x7f2924154e80 2023-10-06T05:15:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.144+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29241714d0 con 0x7f2924154e80 2023-10-06T05:15:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.144+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2924145b20 con 0x7f2924154e80 2023-10-06T05:15:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.146+0000 7f29127fc700 1 -- 172.21.15.45:0/865147469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f291c003020 con 0x7f2924154e80 2023-10-06T05:15:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.146+0000 7f29127fc700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2904066f30 0x7f29040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.147+0000 7f29127fc700 1 -- 172.21.15.45:0/865147469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f291c013070 con 0x7f2924154e80 2023-10-06T05:15:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.147+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29080052b0 con 0x7f2924154e80 2023-10-06T05:15:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.149+0000 7f292259c700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2904066f30 0x7f29040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.154+0000 7f292259c700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2904066f30 0x7f29040693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29240af160 tx=0x7f2914018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.155+0000 7f29127fc700 1 -- 172.21.15.45:0/865147469 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f291c055750 con 0x7f2924154e80 2023-10-06T05:15:22.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.156+0000 7f4349d55700 1 Processor -- start 2023-10-06T05:15:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.156+0000 7f4349d55700 1 -- start start 2023-10-06T05:15:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f4349d55700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43440c7f90 0x7f4344159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f4349d55700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4344155590 0x7f4344159550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f4349d55700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43440af1f0 con 0x7f43440c8360 2023-10-06T05:15:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f4349d55700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43440cbf80 con 0x7f4344155590 2023-10-06T05:15:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f4349d55700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4344159bf0 con 0x7f43440c7f90 2023-10-06T05:15:22.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.159+0000 7f58b200a700 1 Processor -- start 2023-10-06T05:15:22.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.159+0000 7f58b200a700 1 -- start start 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58b200a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58ac0c0170 0x7f58ac0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58b200a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f58ac161c20 0x7f58ac0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58b200a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac162060 con 0x7f58ac0cd300 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58b200a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac0cd7a0 con 0x7f58ac0c0170 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58b200a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac06b080 con 0x7f58ac161c20 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58abfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f58ac161c20 0x7f58ac0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58abfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f58ac161c20 0x7f58ac0acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47478/0 (socket says 172.21.15.45:47478) 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58abfff700 1 -- 172.21.15.45:0/1503319578 learned_addr learned my addr 172.21.15.45:0/1503319578 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.160+0000 7f58aaffd700 1 --1- 172.21.15.45:0/1503319578 >> v1:172.21.15.144:6789/0 conn(0x7f58ac0cd300 0x7f58ac161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50844/0 (socket says 172.21.15.45:50844) 2023-10-06T05:15:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f43437fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43440c7f90 0x7f4344159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f43437fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43440c7f90 0x7f4344159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47462/0 (socket says 172.21.15.45:47462) 2023-10-06T05:15:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f43437fe700 1 -- 172.21.15.45:0/453439188 learned_addr learned my addr 172.21.15.45:0/453439188 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f58abfff700 1 -- 172.21.15.45:0/1503319578 >> v2:172.21.15.45:3300/0 conn(0x7f58ac0c0170 msgr2=0x7f58ac0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f58abfff700 1 --2- 172.21.15.45:0/1503319578 >> v2:172.21.15.45:3300/0 conn(0x7f58ac0c0170 0x7f58ac0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f58abfff700 1 -- 172.21.15.45:0/1503319578 >> v1:172.21.15.144:6789/0 conn(0x7f58ac0cd300 legacy=0x7f58ac161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f58abfff700 1 -- 172.21.15.45:0/1503319578 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac06b2d0 con 0x7f58ac161c20 2023-10-06T05:15:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f43437fe700 1 -- 172.21.15.45:0/453439188 >> v2:172.21.15.45:3300/0 conn(0x7f4344155590 msgr2=0x7f4344159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f43437fe700 1 --2- 172.21.15.45:0/453439188 >> v2:172.21.15.45:3300/0 conn(0x7f4344155590 0x7f4344159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f43437fe700 1 -- 172.21.15.45:0/453439188 >> v1:172.21.15.144:6789/0 conn(0x7f43440c8360 legacy=0x7f4344154e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:15:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f43437fe700 1 -- 172.21.15.45:0/453439188 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344159ed0 con 0x7f43440c7f90 2023-10-06T05:15:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.162+0000 7f58abfff700 1 --2- 172.21.15.45:0/1503319578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac161c20 0x7f58ac0acf00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f58ac06ae50 tx=0x7f58a4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1675a939339bafd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f43437fe700 1 --2- 172.21.15.45:0/453439188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c7f90 0x7f4344159010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f434406d870 tx=0x7f43340082d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66900f5edd837e43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f58aa7fc700 1 -- 172.21.15.45:0/1503319578 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a400e070 con 0x7f58ac161c20 2023-10-06T05:15:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f58aa7fc700 1 -- 172.21.15.45:0/1503319578 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a400ba50 con 0x7f58ac161c20 2023-10-06T05:15:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f58aa7fc700 1 -- 172.21.15.45:0/1503319578 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40092e0 con 0x7f58ac161c20 2023-10-06T05:15:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f40426d0700 1 Processor -- start 2023-10-06T05:15:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f43427fc700 1 -- 172.21.15.45:0/453439188 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334010070 con 0x7f43440c7f90 2023-10-06T05:15:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f43427fc700 1 -- 172.21.15.45:0/453439188 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433400dac0 con 0x7f43440c7f90 2023-10-06T05:15:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f43427fc700 1 -- 172.21.15.45:0/453439188 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433400de60 con 0x7f43440c7f90 2023-10-06T05:15:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.157+0000 7f3d4625e700 1 Processor -- start 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.157+0000 7f3d4625e700 1 -- start start 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f3d4625e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d400aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f3d4625e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d400af3e0 0x7f3d401668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f3d4625e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d400afc40 con 0x7f3d40165f60 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f3d4625e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d400b13e0 con 0x7f3d400af3e0 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.158+0000 7f3d4625e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d400aff20 con 0x7f3d400cc3c0 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.159+0000 7f3d3effd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d400aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.159+0000 7f3d3effd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d400aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33980/0 (socket says 172.21.15.45:33980) 2023-10-06T05:15:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.159+0000 7f3d3effd700 1 -- 172.21.15.45:0/2737536830 learned_addr learned my addr 172.21.15.45:0/2737536830 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.161+0000 7f3d3ffff700 1 --2- 172.21.15.45:0/2737536830 >> v2:172.21.15.45:3300/0 conn(0x7f3d400af3e0 0x7f3d401668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.163+0000 7f40426d0700 1 -- start start 2023-10-06T05:15:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f58b200a700 1 -- 172.21.15.45:0/1503319578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac161c20 msgr2=0x7f58ac0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f58b200a700 1 --2- 172.21.15.45:0/1503319578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac161c20 0x7f58ac0acf00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f58ac06ae50 tx=0x7f58a4009840 comp rx=0 tx=0).stop 2023-10-06T05:15:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f40426d0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f403c06afa0 0x7f403c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f40426d0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0ac8d0 con 0x7f403c15d890 2023-10-06T05:15:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f40426d0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0c81a0 con 0x7f403c15d4c0 2023-10-06T05:15:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f40426d0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c0ac5a0 con 0x7f403c06afa0 2023-10-06T05:15:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f4349d55700 1 -- 172.21.15.45:0/453439188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c7f90 msgr2=0x7f4344159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f4349d55700 1 --2- 172.21.15.45:0/453439188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c7f90 0x7f4344159010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f434406d870 tx=0x7f43340082d0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f3d3ffff700 1 -- 172.21.15.45:0/2737536830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 msgr2=0x7f3d400aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f3d3ffff700 1 --2- 172.21.15.45:0/2737536830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d400aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f3d3ffff700 1 -- 172.21.15.45:0/2737536830 >> v1:172.21.15.45:6789/0 conn(0x7f3d40165f60 legacy=0x7f3d400cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.164+0000 7f3d3ffff700 1 -- 172.21.15.45:0/2737536830 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d400b0200 con 0x7f3d400af3e0 2023-10-06T05:15:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f4040c6d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f403c15d890 0x7f403c0c70f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58616/0 (socket says 172.21.15.45:58616) 2023-10-06T05:15:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f4040c6d700 1 -- 172.21.15.45:0/4048103532 learned_addr learned my addr 172.21.15.45:0/4048103532 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f3d3ffff700 1 --2- 172.21.15.45:0/2737536830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d400af3e0 0x7f3d401668f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007cd0 tx=0x7f3d38005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efd02e2ed9f6d3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f58b200a700 1 -- 172.21.15.45:0/1503319578 shutdown_connections 2023-10-06T05:15:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f58b200a700 1 --2- 172.21.15.45:0/1503319578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac161c20 0x7f58ac0acf00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f58b200a700 1 --2- 172.21.15.45:0/1503319578 >> v2:172.21.15.45:3300/0 conn(0x7f58ac0c0170 0x7f58ac0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f58b200a700 1 -- 172.21.15.45:0/1503319578 >> 172.21.15.45:0/1503319578 conn(0x7f58ac06f090 msgr2=0x7f58ac06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f4349d55700 1 -- 172.21.15.45:0/453439188 shutdown_connections 2023-10-06T05:15:22.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f4349d55700 1 --2- 172.21.15.45:0/453439188 >> v2:172.21.15.45:3300/0 conn(0x7f4344155590 0x7f4344159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f4349d55700 1 --2- 172.21.15.45:0/453439188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c7f90 0x7f4344159010 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.165+0000 7f4349d55700 1 -- 172.21.15.45:0/453439188 >> 172.21.15.45:0/453439188 conn(0x7f4344067620 msgr2=0x7f4344067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f403affd700 1 -- 172.21.15.45:0/4048103532 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216513777 0 0) 0x7f403c0ac8d0 con 0x7f403c15d890 2023-10-06T05:15:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f403affd700 1 -- 172.21.15.45:0/4048103532 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4024003570 con 0x7f403c15d890 2023-10-06T05:15:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f3d3e7fc700 1 -- 172.21.15.45:0/2737536830 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38015410 con 0x7f3d400af3e0 2023-10-06T05:15:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f3d3e7fc700 1 -- 172.21.15.45:0/2737536830 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d38015570 con 0x7f3d400af3e0 2023-10-06T05:15:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f3d3e7fc700 1 -- 172.21.15.45:0/2737536830 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d380158f0 con 0x7f3d400af3e0 2023-10-06T05:15:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f58b200a700 1 -- 172.21.15.45:0/1503319578 shutdown_connections 2023-10-06T05:15:22.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f4349d55700 1 -- 172.21.15.45:0/453439188 shutdown_connections 2023-10-06T05:15:22.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f3d4625e700 1 -- 172.21.15.45:0/2737536830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d400af3e0 msgr2=0x7f3d401668f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f3d4625e700 1 --2- 172.21.15.45:0/2737536830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d400af3e0 0x7f3d401668f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007cd0 tx=0x7f3d38005ef0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f403b7fe700 1 --2- 172.21.15.45:0/4048103532 >> v2:172.21.15.45:3300/0 conn(0x7f403c06afa0 0x7f403c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f403b7fe700 1 -- 172.21.15.45:0/4048103532 >> v1:172.21.15.144:6789/0 conn(0x7f403c15d4c0 legacy=0x7f403c06a890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:15:22.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f403b7fe700 1 -- 172.21.15.45:0/4048103532 >> v1:172.21.15.45:6789/0 conn(0x7f403c15d890 legacy=0x7f403c0c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f403b7fe700 1 -- 172.21.15.45:0/4048103532 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0c7900 con 0x7f403c06afa0 2023-10-06T05:15:22.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f403b7fe700 1 --2- 172.21.15.45:0/4048103532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c06afa0 0x7f403c0c6bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f402c0098e0 tx=0x7f402c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff1596e5c8b2db3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f3d4625e700 1 -- 172.21.15.45:0/2737536830 shutdown_connections 2023-10-06T05:15:22.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f3d4625e700 1 --2- 172.21.15.45:0/2737536830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d400af3e0 0x7f3d401668f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f3d4625e700 1 --2- 172.21.15.45:0/2737536830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d400aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f3d4625e700 1 -- 172.21.15.45:0/2737536830 >> 172.21.15.45:0/2737536830 conn(0x7f3d40067620 msgr2=0x7f3d40067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f403affd700 1 -- 172.21.15.45:0/4048103532 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c011630 con 0x7f403c06afa0 2023-10-06T05:15:22.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.168+0000 7f403affd700 1 -- 172.21.15.45:0/4048103532 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c011790 con 0x7f403c06afa0 2023-10-06T05:15:22.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.168+0000 7f403affd700 1 -- 172.21.15.45:0/4048103532 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c011b50 con 0x7f403c06afa0 2023-10-06T05:15:22.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.166+0000 7f4349d55700 1 -- 172.21.15.45:0/453439188 wait complete. 2023-10-06T05:15:22.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.168+0000 7f4349d55700 1 Processor -- start 2023-10-06T05:15:22.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.168+0000 7f40426d0700 1 -- 172.21.15.45:0/4048103532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c06afa0 msgr2=0x7f403c0c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.169+0000 7f40426d0700 1 --2- 172.21.15.45:0/4048103532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c06afa0 0x7f403c0c6bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f402c0098e0 tx=0x7f402c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.169+0000 7f40426d0700 1 -- 172.21.15.45:0/4048103532 shutdown_connections 2023-10-06T05:15:22.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.169+0000 7f40426d0700 1 --2- 172.21.15.45:0/4048103532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c06afa0 0x7f403c0c6bb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.169+0000 7f40426d0700 1 -- 172.21.15.45:0/4048103532 >> 172.21.15.45:0/4048103532 conn(0x7f403c0cc5a0 msgr2=0x7f403c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.167+0000 7f58b200a700 1 -- 172.21.15.45:0/1503319578 wait complete. 2023-10-06T05:15:22.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.169+0000 7f58b200a700 1 Processor -- start 2023-10-06T05:15:22.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.169+0000 7f58b200a700 1 -- start start 2023-10-06T05:15:22.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.171+0000 7fc0e5c9e700 1 Processor -- start 2023-10-06T05:15:22.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.171+0000 7fc0e5c9e700 1 -- start start 2023-10-06T05:15:22.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.171+0000 7f4349d55700 1 -- start start 2023-10-06T05:15:22.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f3d4625e700 1 -- 172.21.15.45:0/2737536830 shutdown_connections 2023-10-06T05:15:22.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f40426d0700 1 -- 172.21.15.45:0/4048103532 shutdown_connections 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f4349d55700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43440c7f90 0x7f4344150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f4349d55700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c8360 0x7f4344158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f4349d55700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4344155590 0x7f43440aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f4349d55700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43441512a0 con 0x7f4344155590 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f4349d55700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4344158170 con 0x7f43440c8360 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f4349d55700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4344157560 con 0x7f43440c7f90 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f43437fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43440c7f90 0x7f4344150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f43437fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43440c7f90 0x7f4344150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33994/0 (socket says 172.21.15.45:33994) 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f43437fe700 1 -- 172.21.15.45:0/404888767 learned_addr learned my addr 172.21.15.45:0/404888767 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f4342ffd700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c8360 0x7f4344158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0e5c9e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0e0155d70 0x7fc0e0068910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0e5c9e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0156140 0x7fc0e006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0e5c9e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e0067620 con 0x7fc0e006ec20 2023-10-06T05:15:22.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0e5c9e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e006f0c0 con 0x7fc0e0155d70 2023-10-06T05:15:22.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0e5c9e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0067870 con 0x7fc0e0156140 2023-10-06T05:15:22.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0156140 0x7fc0e006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0156140 0x7fc0e006e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34006/0 (socket says 172.21.15.45:34006) 2023-10-06T05:15:22.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0deffd700 1 -- 172.21.15.45:0/1434063076 learned_addr learned my addr 172.21.15.45:0/1434063076 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f40426d0700 1 -- 172.21.15.45:0/4048103532 wait complete. 2023-10-06T05:15:22.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f40426d0700 1 Processor -- start 2023-10-06T05:15:22.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0deffd700 1 -- 172.21.15.45:0/1434063076 >> v2:172.21.15.45:3300/0 conn(0x7fc0e0155d70 msgr2=0x7fc0e0068910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:15:22.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0deffd700 1 --2- 172.21.15.45:0/1434063076 >> v2:172.21.15.45:3300/0 conn(0x7fc0e0155d70 0x7fc0e0068910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7fc0deffd700 1 -- 172.21.15.45:0/1434063076 >> v1:172.21.15.45:6789/0 conn(0x7fc0e006ec20 legacy=0x7fc0e00ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.174+0000 7fc0deffd700 1 -- 172.21.15.45:0/1434063076 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0067b50 con 0x7fc0e0156140 2023-10-06T05:15:22.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f4342ffd700 1 -- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43440c7f90 msgr2=0x7f4344150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f4342ffd700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43440c7f90 0x7f4344150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f4342ffd700 1 -- 172.21.15.45:0/404888767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4344155590 msgr2=0x7f43440aac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f4342ffd700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4344155590 0x7f43440aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f4342ffd700 1 -- 172.21.15.45:0/404888767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334008570 con 0x7f43440c8360 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f4342ffd700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c8360 0x7f4344158600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f433800b5b0 tx=0x7f43380090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.174+0000 7f432b7fe700 1 -- 172.21.15.45:0/404888767 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800f510 con 0x7f43440c8360 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.174+0000 7f432b7fe700 1 -- 172.21.15.45:0/404888767 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433800f670 con 0x7f43440c8360 2023-10-06T05:15:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.174+0000 7f432b7fe700 1 -- 172.21.15.45:0/404888767 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800fa10 con 0x7f43440c8360 2023-10-06T05:15:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.174+0000 7f40426d0700 1 -- start start 2023-10-06T05:15:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f40426d0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f403c06afa0 0x7f403c0ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f40426d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c15d890 0x7f403c0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f40426d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f403c15bf90 0x7f403c0c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f40426d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a4c50 con 0x7f403c15d890 2023-10-06T05:15:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f40426d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a49f0 con 0x7f403c15bf90 2023-10-06T05:15:22.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f40426d0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c177eb0 con 0x7f403c06afa0 2023-10-06T05:15:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.173+0000 7f3d4625e700 1 -- 172.21.15.45:0/2737536830 wait complete. 2023-10-06T05:15:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f3d4625e700 1 Processor -- start 2023-10-06T05:15:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f403b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c15d890 0x7f403c0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f403b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c15d890 0x7f403c0a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53326/0 (socket says 172.21.15.45:53326) 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f403b7fe700 1 -- 172.21.15.45:0/663349194 learned_addr learned my addr 172.21.15.45:0/663349194 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f4040c6d700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f403c15bf90 0x7f403c0c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f403b7fe700 1 -- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f403c06afa0 msgr2=0x7f403c0ae320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f403b7fe700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f403c06afa0 0x7f403c0ae320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f403b7fe700 1 -- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f403c15bf90 msgr2=0x7f403c0c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f403b7fe700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f403c15bf90 0x7f403c0c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f403b7fe700 1 -- 172.21.15.45:0/663349194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c009420 con 0x7f403c15d890 2023-10-06T05:15:22.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.171+0000 7f58b200a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58ac0c0170 0x7f58ac1485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.172+0000 7f58b200a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac0cd300 0x7f58ac0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f58aaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac0cd300 0x7f58ac0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f58aaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac0cd300 0x7f58ac0b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47492/0 (socket says 172.21.15.45:47492) 2023-10-06T05:15:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f58b200a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58ac161c20 0x7f58ac0c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f58b200a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0b2240 con 0x7f58ac0c0170 2023-10-06T05:15:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f58b200a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac148af0 con 0x7f58ac0cd300 2023-10-06T05:15:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f58b200a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0b1630 con 0x7f58ac161c20 2023-10-06T05:15:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f403b7fe700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c15d890 0x7f403c0a44b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f402c0091e0 tx=0x7f402c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.176+0000 7f58abfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58ac161c20 0x7f58ac0c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f58abfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58ac161c20 0x7f58ac0c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34016/0 (socket says 172.21.15.45:34016) 2023-10-06T05:15:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f58abfff700 1 -- 172.21.15.45:0/1686059939 learned_addr learned my addr 172.21.15.45:0/1686059939 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.175+0000 7f3d4625e700 1 -- start start 2023-10-06T05:15:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d4625e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d400af3e0 0x7f3d400855d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d4625e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d40160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d4625e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40165f60 0x7f3d400a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d4625e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d401619c0 con 0x7f3d40165f60 2023-10-06T05:15:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d4625e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d40161ca0 con 0x7f3d400af3e0 2023-10-06T05:15:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d4625e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d40161f80 con 0x7f3d400cc3c0 2023-10-06T05:15:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.171+0000 7f58ab7fe700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58ac0c0170 0x7f58ac1485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3effd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d40160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4344157840 con 0x7f43440c8360 2023-10-06T05:15:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4344157e00 con 0x7f43440c8360 2023-10-06T05:15:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7fc0deffd700 1 --2- 172.21.15.45:0/1434063076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0156140 0x7fc0e006e6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400bc80 tx=0x7fc0d400b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4709344441ebc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d400af3e0 0x7f3d400855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3effd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d40160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34030/0 (socket says 172.21.15.45:34030) 2023-10-06T05:15:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d400af3e0 0x7f3d400855d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47526/0 (socket says 172.21.15.45:47526) 2023-10-06T05:15:22.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3effd700 1 -- 172.21.15.45:0/1363363117 learned_addr learned my addr 172.21.15.45:0/1363363117 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f58ab7fe700 1 -- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58ac161c20 msgr2=0x7f58ac0c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f58ab7fe700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58ac161c20 0x7f58ac0c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f58ab7fe700 1 -- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac0cd300 msgr2=0x7f58ac0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f58ab7fe700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac0cd300 0x7f58ac0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f58ab7fe700 1 -- 172.21.15.45:0/1686059939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a4009ae0 con 0x7f58ac0c0170 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f58ab7fe700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58ac0c0170 0x7f58ac1485b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f589c00b600 tx=0x7f589c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f4023fff700 1 -- 172.21.15.45:0/663349194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c01bdb0 con 0x7f403c15d890 2023-10-06T05:15:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f4023fff700 1 -- 172.21.15.45:0/663349194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c020070 con 0x7f403c15d890 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f4023fff700 1 -- 172.21.15.45:0/663349194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0276d0 con 0x7f403c15d890 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3ffff700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40165f60 0x7f3d400a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3f7fe700 1 -- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 msgr2=0x7f3d40160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3f7fe700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d40160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3f7fe700 1 -- 172.21.15.45:0/1363363117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40165f60 msgr2=0x7f3d400a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7f3d3f7fe700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40165f60 0x7f3d400a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f3d3f7fe700 1 -- 172.21.15.45:0/1363363117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d38007810 con 0x7f3d400af3e0 2023-10-06T05:15:22.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.178+0000 7f3d3ffff700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40165f60 0x7f3d400a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7fc0de7fc700 1 -- 172.21.15.45:0/1434063076 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400be90 con 0x7fc0e0156140 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7fc0de7fc700 1 -- 172.21.15.45:0/1434063076 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d400dd80 con 0x7fc0e0156140 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.177+0000 7fc0de7fc700 1 -- 172.21.15.45:0/1434063076 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40106a0 con 0x7fc0e0156140 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/1434063076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0156140 msgr2=0x7fc0e006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/1434063076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0156140 0x7fc0e006e6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400bc80 tx=0x7fc0d400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/1434063076 shutdown_connections 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/1434063076 >> v2:172.21.15.45:3300/0 conn(0x7fc0e0155d70 0x7fc0e0068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/1434063076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0156140 0x7fc0e006e6e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/1434063076 >> 172.21.15.45:0/1434063076 conn(0x7fc0e00cc5a0 msgr2=0x7fc0e00cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f58937fe700 1 -- 172.21.15.45:0/1686059939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00f510 con 0x7f58ac0c0170 2023-10-06T05:15:22.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac0b1910 con 0x7f58ac0c0170 2023-10-06T05:15:22.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f58937fe700 1 -- 172.21.15.45:0/1686059939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c00f670 con 0x7f58ac0c0170 2023-10-06T05:15:22.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f58937fe700 1 -- 172.21.15.45:0/1686059939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00fa30 con 0x7f58ac0c0170 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac0b1ed0 con 0x7f58ac0c0170 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.181+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac06e2f0 con 0x7f58ac0c0170 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.182+0000 7f58937fe700 1 -- 172.21.15.45:0/1686059939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f589c01a400 con 0x7f58ac0c0170 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f58937fe700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5898066fb0 0x7f5898069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f58937fe700 1 -- 172.21.15.45:0/1686059939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f589c084490 con 0x7f58ac0c0170 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f432b7fe700 1 -- 172.21.15.45:0/404888767 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f433801a3b0 con 0x7f43440c8360 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.180+0000 7f432b7fe700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.180+0000 7f432b7fe700 1 -- 172.21.15.45:0/404888767 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4338083d90 con 0x7f43440c8360 2023-10-06T05:15:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.181+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43441582e0 con 0x7f43440c8360 2023-10-06T05:15:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f43437fe700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c178190 con 0x7f403c15d890 2023-10-06T05:15:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c178750 con 0x7f403c15d890 2023-10-06T05:15:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.181+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/1434063076 shutdown_connections 2023-10-06T05:15:22.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.181+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/1434063076 wait complete. 2023-10-06T05:15:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f3d3f7fe700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d400af3e0 0x7f3d400855d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000c590 tx=0x7f3d30007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f3d277fe700 1 -- 172.21.15.45:0/1363363117 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30011590 con 0x7f3d400af3e0 2023-10-06T05:15:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f3d277fe700 1 -- 172.21.15.45:0/1363363117 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d30011710 con 0x7f3d400af3e0 2023-10-06T05:15:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.179+0000 7f3d277fe700 1 -- 172.21.15.45:0/1363363117 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30011ad0 con 0x7f3d400af3e0 2023-10-06T05:15:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.180+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d40162260 con 0x7f3d400af3e0 2023-10-06T05:15:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.180+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d40085b10 con 0x7f3d400af3e0 2023-10-06T05:15:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f3d277fe700 1 -- 172.21.15.45:0/1363363117 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d3001b3b0 con 0x7f3d400af3e0 2023-10-06T05:15:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d40162780 con 0x7f3d400af3e0 2023-10-06T05:15:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f58aaffd700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5898066fb0 0x7f5898069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7fc0e5c9e700 1 Processor -- start 2023-10-06T05:15:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.183+0000 7f3d277fe700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d28067160 0x7f3d28069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.184+0000 7f3d277fe700 1 -- 172.21.15.45:0/1363363117 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3d30083e90 con 0x7f3d400af3e0 2023-10-06T05:15:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.184+0000 7f43437fe700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4334006e80 tx=0x7f433401b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c0cc0e0 con 0x7f403c15d890 2023-10-06T05:15:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.184+0000 7fc0e5c9e700 1 -- start start 2023-10-06T05:15:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0e5c9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e006ec20 0x7fc0e0149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0e5c9e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0155d70 0x7fc0e0164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0e5c9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0e0156140 0x7fc0e0165400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0e5c9e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0149fe0 con 0x7fc0e0156140 2023-10-06T05:15:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0e5c9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e01572b0 con 0x7fc0e006ec20 2023-10-06T05:15:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.184+0000 7f3d3effd700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d28067160 0x7f3d28069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7f3d3effd700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d28067160 0x7f3d28069610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d40169210 tx=0x7f3d34008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0e5c9e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00afc50 con 0x7fc0e0155d70 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0155d70 0x7fc0e0164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0155d70 0x7fc0e0164ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34040/0 (socket says 172.21.15.45:34040) 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0deffd700 1 -- 172.21.15.45:0/643693562 learned_addr learned my addr 172.21.15.45:0/643693562 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.185+0000 7fc0df7fe700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e006ec20 0x7fc0e0149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0deffd700 1 -- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e006ec20 msgr2=0x7fc0e0149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0deffd700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e006ec20 0x7fc0e0149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0deffd700 1 -- 172.21.15.45:0/643693562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0e0156140 msgr2=0x7fc0e0165400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0deffd700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0e0156140 0x7fc0e0165400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0deffd700 1 -- 172.21.15.45:0/643693562 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d400b790 con 0x7fc0e0155d70 2023-10-06T05:15:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0deffd700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0155d70 0x7fc0e0164ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b530 tx=0x7fc0d401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0c77fe700 1 -- 172.21.15.45:0/643693562 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4018780 con 0x7fc0e0155d70 2023-10-06T05:15:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0c77fe700 1 -- 172.21.15.45:0/643693562 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d40188e0 con 0x7fc0e0155d70 2023-10-06T05:15:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0c77fe700 1 -- 172.21.15.45:0/643693562 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4018ca0 con 0x7fc0e0155d70 2023-10-06T05:15:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e00aff30 con 0x7fc0e0155d70 2023-10-06T05:15:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.186+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e00b04f0 con 0x7fc0e0155d70 2023-10-06T05:15:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.187+0000 7f432b7fe700 1 -- 172.21.15.45:0/404888767 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4338054620 con 0x7f43440c8360 2023-10-06T05:15:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.189+0000 7f4023fff700 1 -- 172.21.15.45:0/663349194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f402c027a80 con 0x7f403c15d890 2023-10-06T05:15:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.189+0000 7f4023fff700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4024067480 0x7f4024069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.188+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c00052b0 con 0x7fc0e0155d70 2023-10-06T05:15:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.191+0000 7f403bfff700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4024067480 0x7f4024069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.191+0000 7f403bfff700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4024067480 0x7f4024069930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f403c06e290 tx=0x7f4030006c60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.191+0000 7f3d277fe700 1 -- 172.21.15.45:0/1363363117 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d40162780 con 0x7f3d400af3e0 2023-10-06T05:15:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.191+0000 7f58aaffd700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5898066fb0 0x7f5898069460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f58a0007fd0 tx=0x7f58a0007d70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.193+0000 7f58937fe700 1 -- 172.21.15.45:0/1686059939 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f589c051290 con 0x7f58ac0c0170 2023-10-06T05:15:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.193+0000 7f4023fff700 1 -- 172.21.15.45:0/663349194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f402c022070 con 0x7f403c15d890 2023-10-06T05:15:22.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.198+0000 7f81b6bd8700 1 Processor -- start 2023-10-06T05:15:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.199+0000 7f4023fff700 1 -- 172.21.15.45:0/663349194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f402c0b05a0 con 0x7f403c15d890 2023-10-06T05:15:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.198+0000 7f81b6bd8700 1 -- start start 2023-10-06T05:15:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7f81b6bd8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f81b015d6a0 0x7f81b00ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7f81b6bd8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f81b00b3610 0x7f81b00cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7f81b6bd8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b00cbc50 con 0x7f81b00cc0e0 2023-10-06T05:15:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7f81b6bd8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b00cb040 con 0x7f81b015d6a0 2023-10-06T05:15:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7f81b6bd8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b00cb320 con 0x7f81b00b3610 2023-10-06T05:15:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.199+0000 7fc0c77fe700 1 -- 172.21.15.45:0/643693562 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0d402c450 con 0x7fc0e0155d70 2023-10-06T05:15:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.199+0000 7fc0c77fe700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc066fb0 0x7fc0cc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.199+0000 7fc0c77fe700 1 -- 172.21.15.45:0/643693562 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc0d4014070 con 0x7fc0e0155d70 2023-10-06T05:15:22.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7fc0c77fe700 1 -- 172.21.15.45:0/643693562 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0d40889e0 con 0x7fc0e0155d70 2023-10-06T05:15:22.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7f81a7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f81b00cc0e0 0x7f81b00b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58630/0 (socket says 172.21.15.45:58630) 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.200+0000 7f81a7fff700 1 -- 172.21.15.45:0/310712078 learned_addr learned my addr 172.21.15.45:0/310712078 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.201+0000 7f81b5175700 1 --2- 172.21.15.45:0/310712078 >> v2:172.21.15.144:3300/0 conn(0x7f81b00b3610 0x7f81b00cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.201+0000 7f81b4974700 1 --2- 172.21.15.45:0/310712078 >> v2:172.21.15.45:3300/0 conn(0x7f81b015d6a0 0x7f81b00ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.201+0000 7f81b4974700 1 -- 172.21.15.45:0/310712078 >> v2:172.21.15.144:3300/0 conn(0x7f81b00b3610 msgr2=0x7f81b00cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.201+0000 7f81b4974700 1 --2- 172.21.15.45:0/310712078 >> v2:172.21.15.144:3300/0 conn(0x7f81b00b3610 0x7f81b00cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.201+0000 7f81b4974700 1 -- 172.21.15.45:0/310712078 >> v1:172.21.15.45:6789/0 conn(0x7f81b00cc0e0 legacy=0x7f81b00b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.201+0000 7f81b4974700 1 -- 172.21.15.45:0/310712078 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00cb600 con 0x7f81b015d6a0 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.202+0000 7f81b4974700 1 --2- 172.21.15.45:0/310712078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b015d6a0 0x7f81b00ca6d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ca660 tx=0x7f81a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f53df1e3982b15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.202+0000 7f81a77fe700 1 -- 172.21.15.45:0/310712078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0011630 con 0x7f81b015d6a0 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.202+0000 7f81a77fe700 1 -- 172.21.15.45:0/310712078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a00117b0 con 0x7f81b015d6a0 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.202+0000 7f81a77fe700 1 -- 172.21.15.45:0/310712078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0011b70 con 0x7f81b015d6a0 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.203+0000 7f81b6bd8700 1 -- 172.21.15.45:0/310712078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b015d6a0 msgr2=0x7f81b00ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.204+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/310712078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b015d6a0 0x7f81b00ca6d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ca660 tx=0x7f81a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.210+0000 7f81b6bd8700 1 -- 172.21.15.45:0/310712078 shutdown_connections 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.210+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/310712078 >> v2:172.21.15.144:3300/0 conn(0x7f81b00b3610 0x7f81b00cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.210+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/310712078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b015d6a0 0x7f81b00ca6d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.210+0000 7f81b6bd8700 1 -- 172.21.15.45:0/310712078 >> 172.21.15.45:0/310712078 conn(0x7f81b006f7c0 msgr2=0x7f81b00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.210+0000 7f81b6bd8700 1 -- 172.21.15.45:0/310712078 shutdown_connections 2023-10-06T05:15:22.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.211+0000 7fc0df7fe700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc066fb0 0x7fc0cc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.210+0000 7f81b6bd8700 1 -- 172.21.15.45:0/310712078 wait complete. 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.212+0000 7f81b6bd8700 1 Processor -- start 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.213+0000 7f81b6bd8700 1 -- start start 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.214+0000 7f81b6bd8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b00b3610 0x7f81b00c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.214+0000 7f81b6bd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b00cc0e0 0x7f81b015ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.214+0000 7f81b6bd8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b015d6a0 0x7f81b015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.214+0000 7f81b6bd8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00c38a0 con 0x7f81b00cc0e0 2023-10-06T05:15:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.214+0000 7f81b6bd8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00c7d50 con 0x7f81b015d6a0 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.214+0000 7f81b6bd8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015ec30 con 0x7f81b00b3610 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.215+0000 7f81a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b00cc0e0 0x7f81b015ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.215+0000 7f81a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b00cc0e0 0x7f81b015ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53360/0 (socket says 172.21.15.45:53360) 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.215+0000 7f81a7fff700 1 -- 172.21.15.45:0/2782854599 learned_addr learned my addr 172.21.15.45:0/2782854599 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.215+0000 7f81a7fff700 1 -- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b00b3610 msgr2=0x7f81b00c9b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.215+0000 7f81a7fff700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b00b3610 0x7f81b00c9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.216+0000 7f81a7fff700 1 -- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b015d6a0 msgr2=0x7f81b015f4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.216+0000 7f81a7fff700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b015d6a0 0x7f81b015f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.216+0000 7f81a7fff700 1 -- 172.21.15.45:0/2782854599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0009420 con 0x7f81b00cc0e0 2023-10-06T05:15:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.216+0000 7f81a7fff700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b00cc0e0 0x7f81b015ef90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c790 tx=0x7f81ac007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.217+0000 7f81a4ff9700 1 -- 172.21.15.45:0/2782854599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac005b30 con 0x7f81b00cc0e0 2023-10-06T05:15:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.217+0000 7f81a4ff9700 1 -- 172.21.15.45:0/2782854599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac005c90 con 0x7f81b00cc0e0 2023-10-06T05:15:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.217+0000 7f81a4ff9700 1 -- 172.21.15.45:0/2782854599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac007340 con 0x7f81b00cc0e0 2023-10-06T05:15:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.217+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b00c7140 con 0x7f81b00cc0e0 2023-10-06T05:15:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.217+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b00c7700 con 0x7f81b00cc0e0 2023-10-06T05:15:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.223+0000 7f81a4ff9700 1 -- 172.21.15.45:0/2782854599 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f81ac01b3b0 con 0x7f81b00cc0e0 2023-10-06T05:15:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.224+0000 7f81a4ff9700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f81980671e0 0x7f8198069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.224+0000 7f81a4ff9700 1 -- 172.21.15.45:0/2782854599 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f81ac084010 con 0x7f81b00cc0e0 2023-10-06T05:15:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.225+0000 7f81b4974700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f81980671e0 0x7f8198069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.225+0000 7f81b4974700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f81980671e0 0x7f8198069690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ca660 tx=0x7f81a0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.230+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81900052b0 con 0x7f81b00cc0e0 2023-10-06T05:15:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.224+0000 7fc0df7fe700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc066fb0 0x7fc0cc069460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0006fd0 tx=0x7fc0d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.240+0000 7f81a4ff9700 1 -- 172.21.15.45:0/2782854599 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81ac050e90 con 0x7f81b00cc0e0 2023-10-06T05:15:22.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.258+0000 7f6705c27700 1 Processor -- start 2023-10-06T05:15:22.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.258+0000 7f6705c27700 1 -- start start 2023-10-06T05:15:22.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.259+0000 7f6705c27700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6700146f80 0x7f67000afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.259+0000 7f6705c27700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f670006a6f0 0x7f670015f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.259+0000 7f6705c27700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670015f1c0 con 0x7f67000aff60 2023-10-06T05:15:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.259+0000 7f6705c27700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670006ab20 con 0x7f670006a6f0 2023-10-06T05:15:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.259+0000 7f6705c27700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b03e0 con 0x7f6700146f80 2023-10-06T05:15:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.260+0000 7f66ff7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6700146f80 0x7f67000afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.260+0000 7f66ff7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6700146f80 0x7f67000afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47572/0 (socket says 172.21.15.45:47572) 2023-10-06T05:15:22.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.260+0000 7f66ff7fe700 1 -- 172.21.15.45:0/2255425511 learned_addr learned my addr 172.21.15.45:0/2255425511 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.262+0000 7f66fffff700 1 --2- 172.21.15.45:0/2255425511 >> v2:172.21.15.45:3300/0 conn(0x7f670006a6f0 0x7f670015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.263+0000 7f66fffff700 1 -- 172.21.15.45:0/2255425511 >> v2:172.21.15.144:3300/0 conn(0x7f6700146f80 msgr2=0x7f67000afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.263+0000 7f66fffff700 1 --2- 172.21.15.45:0/2255425511 >> v2:172.21.15.144:3300/0 conn(0x7f6700146f80 0x7f67000afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.263+0000 7f66fffff700 1 -- 172.21.15.45:0/2255425511 >> v1:172.21.15.144:6789/0 conn(0x7f67000aff60 legacy=0x7f6700069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.263+0000 7f66fffff700 1 -- 172.21.15.45:0/2255425511 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670006ad10 con 0x7f670006a6f0 2023-10-06T05:15:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.263+0000 7f66fffff700 1 --2- 172.21.15.45:0/2255425511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f670006a6f0 0x7f670015f650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f670006b590 tx=0x7f66f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891dad00521d5a30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.264+0000 7f66fe7fc700 1 -- 172.21.15.45:0/2255425511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0011630 con 0x7f670006a6f0 2023-10-06T05:15:22.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.264+0000 7f66fe7fc700 1 -- 172.21.15.45:0/2255425511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f00117b0 con 0x7f670006a6f0 2023-10-06T05:15:22.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.264+0000 7f66fe7fc700 1 -- 172.21.15.45:0/2255425511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0011b70 con 0x7f670006a6f0 2023-10-06T05:15:22.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.265+0000 7f6705c27700 1 -- 172.21.15.45:0/2255425511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f670006a6f0 msgr2=0x7f670015f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.265+0000 7f6705c27700 1 --2- 172.21.15.45:0/2255425511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f670006a6f0 0x7f670015f650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f670006b590 tx=0x7f66f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.266+0000 7f6705c27700 1 -- 172.21.15.45:0/2255425511 shutdown_connections 2023-10-06T05:15:22.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.266+0000 7f6705c27700 1 --2- 172.21.15.45:0/2255425511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f670006a6f0 0x7f670015f650 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.266+0000 7f6705c27700 1 --2- 172.21.15.45:0/2255425511 >> v2:172.21.15.144:3300/0 conn(0x7f6700146f80 0x7f67000afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.266+0000 7f6705c27700 1 -- 172.21.15.45:0/2255425511 >> 172.21.15.45:0/2255425511 conn(0x7f6700067620 msgr2=0x7f6700067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.266+0000 7f6705c27700 1 -- 172.21.15.45:0/2255425511 shutdown_connections 2023-10-06T05:15:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.275+0000 7f6705c27700 1 -- 172.21.15.45:0/2255425511 wait complete. 2023-10-06T05:15:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.277+0000 7f6705c27700 1 Processor -- start 2023-10-06T05:15:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.278+0000 7f6705c27700 1 -- start start 2023-10-06T05:15:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.279+0000 7f6705c27700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f670006a6f0 0x7f6700146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.279+0000 7f6705c27700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6700146f80 0x7f67000a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.279+0000 7f6705c27700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67000aa510 0x7f670013c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.279+0000 7f6705c27700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700146850 con 0x7f6700146f80 2023-10-06T05:15:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.279+0000 7f6705c27700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000cc4a0 con 0x7f67000aa510 2023-10-06T05:15:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.279+0000 7f6705c27700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6700144130 con 0x7f670006a6f0 2023-10-06T05:15:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.280+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f670006a6f0 0x7f6700146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.280+0000 7f66fffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67000aa510 0x7f670013c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.280+0000 7f66fffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67000aa510 0x7f670013c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47588/0 (socket says 172.21.15.45:47588) 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.280+0000 7f66fffff700 1 -- 172.21.15.45:0/2990200310 learned_addr learned my addr 172.21.15.45:0/2990200310 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.280+0000 7f66feffd700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6700146f80 0x7f67000a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.281+0000 7f66fffff700 1 -- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f670006a6f0 msgr2=0x7f6700146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.281+0000 7f66fffff700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f670006a6f0 0x7f6700146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.281+0000 7f66fffff700 1 -- 172.21.15.45:0/2990200310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6700146f80 msgr2=0x7f67000a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.281+0000 7f66fffff700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6700146f80 0x7f67000a9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.281+0000 7f66fffff700 1 -- 172.21.15.45:0/2990200310 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0009420 con 0x7f67000aa510 2023-10-06T05:15:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.283+0000 7f66fffff700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67000aa510 0x7f670013c2b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009f80 tx=0x7f66f0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.283+0000 7f66e77fe700 1 -- 172.21.15.45:0/2990200310 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f001f070 con 0x7f67000aa510 2023-10-06T05:15:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.284+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67000aa970 con 0x7f67000aa510 2023-10-06T05:15:22.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.284+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67000a6340 con 0x7f67000aa510 2023-10-06T05:15:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.285+0000 7f66e77fe700 1 -- 172.21.15.45:0/2990200310 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0011920 con 0x7f67000aa510 2023-10-06T05:15:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.285+0000 7f66e77fe700 1 -- 172.21.15.45:0/2990200310 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0025420 con 0x7f67000aa510 2023-10-06T05:15:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.288+0000 7f66e77fe700 1 -- 172.21.15.45:0/2990200310 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66f0023070 con 0x7f67000aa510 2023-10-06T05:15:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.288+0000 7f66e77fe700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66ec066fb0 0x7f66ec069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.289+0000 7f66e77fe700 1 -- 172.21.15.45:0/2990200310 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f66f0085f60 con 0x7f67000aa510 2023-10-06T05:15:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.289+0000 7f66ff7fe700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66ec066fb0 0x7f66ec069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:22.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.290+0000 7f66ff7fe700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66ec066fb0 0x7f66ec069460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66f80072f0 tx=0x7f66f800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:22.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.298+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e00052b0 con 0x7f67000aa510 2023-10-06T05:15:22.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.303+0000 7f66e77fe700 1 -- 172.21.15.45:0/2990200310 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66f00b15a0 con 0x7f67000aa510 2023-10-06T05:15:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.584+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2908005560 con 0x7f2924154e80 2023-10-06T05:15:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.585+0000 7f29127fc700 1 -- 172.21.15.45:0/865147469 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f291c0552e0 con 0x7f2924154e80 2023-10-06T05:15:22.590 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2023-10-06T05:15:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2904066f30 msgr2=0x7f29040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2904066f30 0x7f29040693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29240af160 tx=0x7f2914018000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2924154e80 msgr2=0x7f2924170500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2924154e80 0x7f2924170500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f291c00cdb0 tx=0x7f291c0077d0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 shutdown_connections 2023-10-06T05:15:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2904066f30 0x7f29040693e0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29240af160 tx=0x7f2914018000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2924154e80 0x7f2924170500 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29240c7f90 0x7f2924171a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 --2- 172.21.15.45:0/865147469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29240c8360 0x7f2924171f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.589+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 >> 172.21.15.45:0/865147469 conn(0x7f2924067620 msgr2=0x7f29240c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.593+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 shutdown_connections 2023-10-06T05:15:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.593+0000 7f2928e86700 1 -- 172.21.15.45:0/865147469 wait complete. 2023-10-06T05:15:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.607+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f403c04bb40 con 0x7f403c15d890 2023-10-06T05:15:22.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.607+0000 7f4023fff700 1 -- 172.21.15.45:0/663349194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f402c057940 con 0x7f403c15d890 2023-10-06T05:15:22.609 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2023-10-06T05:15:22.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-10-06T05:15:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.610+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3d40161de0 con 0x7f3d400af3e0 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4024067480 msgr2=0x7f4024069930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4024067480 0x7f4024069930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f403c06e290 tx=0x7f4030006c60 comp rx=0 tx=0).stop 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c15d890 msgr2=0x7f403c0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c15d890 0x7f403c0a44b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f402c0091e0 tx=0x7f402c016000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 shutdown_connections 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4024067480 0x7f4024069930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f403c15bf90 0x7f403c0c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f403c06afa0 0x7f403c0ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 --2- 172.21.15.45:0/663349194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f403c15d890 0x7f403c0a44b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 >> 172.21.15.45:0/663349194 conn(0x7f403c0cc5a0 msgr2=0x7f403c0cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f3d277fe700 1 -- 172.21.15.45:0/1363363117 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3d40161de0 con 0x7f3d400af3e0 2023-10-06T05:15:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 shutdown_connections 2023-10-06T05:15:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.611+0000 7f40426d0700 1 -- 172.21.15.45:0/663349194 wait complete. 2023-10-06T05:15:22.615 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2023-10-06T05:15:22.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.612+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f58ac04bb40 con 0x7f58ac0c0170 2023-10-06T05:15:22.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.613+0000 7f58937fe700 1 -- 172.21.15.45:0/1686059939 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f589c0548b0 con 0x7f58ac0c0170 2023-10-06T05:15:22.616 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2023-10-06T05:15:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.614+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f43440aeba0 con 0x7f43440c8360 2023-10-06T05:15:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.614+0000 7f432b7fe700 1 -- 172.21.15.45:0/404888767 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4338016030 con 0x7f43440c8360 2023-10-06T05:15:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.617+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5898066fb0 msgr2=0x7f5898069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.617+0000 7f58b200a700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5898066fb0 0x7f5898069460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f58a0007fd0 tx=0x7f58a0007d70 comp rx=0 tx=0).stop 2023-10-06T05:15:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.617+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58ac0c0170 msgr2=0x7f58ac1485b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.617+0000 7f58b200a700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58ac0c0170 0x7f58ac1485b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f589c00b600 tx=0x7f589c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.620 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:22.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 shutdown_connections 2023-10-06T05:15:22.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f58b200a700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5898066fb0 0x7f5898069460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f58b200a700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58ac161c20 0x7f58ac0c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f58b200a700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58ac0c0170 0x7f58ac1485b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f58b200a700 1 --2- 172.21.15.45:0/1686059939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58ac0cd300 0x7f58ac0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 >> 172.21.15.45:0/1686059939 conn(0x7f58ac06f090 msgr2=0x7f58ac070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d28067160 msgr2=0x7f3d28069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d28067160 0x7f3d28069610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d40169210 tx=0x7f3d34008040 comp rx=0 tx=0).stop 2023-10-06T05:15:22.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d400af3e0 msgr2=0x7f3d400855d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d400af3e0 0x7f3d400855d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000c590 tx=0x7f3d30007ec0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 msgr2=0x7f43300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4334006e80 tx=0x7f433401b000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c8360 msgr2=0x7f4344158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c8360 0x7f4344158600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f433800b5b0 tx=0x7f43380090f0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 shutdown_connections 2023-10-06T05:15:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d28067160 0x7f3d28069610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40165f60 0x7f3d400a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d400af3e0 0x7f3d400855d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 --2- 172.21.15.45:0/1363363117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d400cc3c0 0x7f3d40160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 >> 172.21.15.45:0/1363363117 conn(0x7f3d40067620 msgr2=0x7f3d400a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 shutdown_connections 2023-10-06T05:15:22.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4344155590 0x7f43440aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43440c7f90 0x7f4344150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 --2- 172.21.15.45:0/404888767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43440c8360 0x7f4344158600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 >> 172.21.15.45:0/404888767 conn(0x7f4344067620 msgr2=0x7f43440679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 shutdown_connections 2023-10-06T05:15:22.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.618+0000 7f4349d55700 1 -- 172.21.15.45:0/404888767 wait complete. 2023-10-06T05:15:22.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.620+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 shutdown_connections 2023-10-06T05:15:22.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.620+0000 7f3d4625e700 1 -- 172.21.15.45:0/1363363117 wait complete. 2023-10-06T05:15:22.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.620+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc0c0005560 con 0x7fc0e0155d70 2023-10-06T05:15:22.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.621+0000 7fc0c77fe700 1 -- 172.21.15.45:0/643693562 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc0d4053440 con 0x7fc0e0155d70 2023-10-06T05:15:22.626 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:22.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.622+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 shutdown_connections 2023-10-06T05:15:22.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.622+0000 7f58b200a700 1 -- 172.21.15.45:0/1686059939 wait complete. 2023-10-06T05:15:22.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.626+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc066fb0 msgr2=0x7fc0cc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.626+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc066fb0 0x7fc0cc069460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0006fd0 tx=0x7fc0d0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:22.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.626+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0155d70 msgr2=0x7fc0e0164ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.626+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0155d70 0x7fc0e0164ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b530 tx=0x7fc0d401a000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-06T05:15:22.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.631+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 shutdown_connections 2023-10-06T05:15:22.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.631+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc066fb0 0x7fc0cc069460 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0006fd0 tx=0x7fc0d0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:22.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.631+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0e0156140 0x7fc0e0165400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.631+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e006ec20 0x7fc0e0149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.631+0000 7fc0e5c9e700 1 --2- 172.21.15.45:0/643693562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e0155d70 0x7fc0e0164ec0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.631+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 >> 172.21.15.45:0/643693562 conn(0x7fc0e00cc5a0 msgr2=0x7fc0e006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.633+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 shutdown_connections 2023-10-06T05:15:22.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.633+0000 7fc0e5c9e700 1 -- 172.21.15.45:0/643693562 wait complete. 2023-10-06T05:15:22.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-10-06T05:15:22.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-10-06T05:15:22.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-10-06T05:15:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.654+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8190005560 con 0x7f81b00cc0e0 2023-10-06T05:15:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.655+0000 7f81a4ff9700 1 -- 172.21.15.45:0/2782854599 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f81ac017030 con 0x7f81b00cc0e0 2023-10-06T05:15:22.657 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:22.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-10-06T05:15:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f81980671e0 msgr2=0x7f8198069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f81980671e0 0x7f8198069690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ca660 tx=0x7f81a0016000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b00cc0e0 msgr2=0x7f81b015ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b00cc0e0 0x7f81b015ef90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c790 tx=0x7f81ac007ef0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 shutdown_connections 2023-10-06T05:15:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f81980671e0 0x7f8198069690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b015d6a0 0x7f81b015f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b00b3610 0x7f81b00c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 --2- 172.21.15.45:0/2782854599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b00cc0e0 0x7f81b015ef90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 >> 172.21.15.45:0/2782854599 conn(0x7f81b006f7c0 msgr2=0x7f81b00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 shutdown_connections 2023-10-06T05:15:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.657+0000 7f81b6bd8700 1 -- 172.21.15.45:0/2782854599 wait complete. 2023-10-06T05:15:22.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-10-06T05:15:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.682+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f66e0005560 con 0x7f67000aa510 2023-10-06T05:15:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.683+0000 7f66e77fe700 1 -- 172.21.15.45:0/2990200310 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66f002a080 con 0x7f67000aa510 2023-10-06T05:15:22.685 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2023-10-06T05:15:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66ec066fb0 msgr2=0x7f66ec069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66ec066fb0 0x7f66ec069460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66f80072f0 tx=0x7f66f800b040 comp rx=0 tx=0).stop 2023-10-06T05:15:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67000aa510 msgr2=0x7f670013c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67000aa510 0x7f670013c2b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009f80 tx=0x7f66f0017000 comp rx=0 tx=0).stop 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 shutdown_connections 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66ec066fb0 0x7f66ec069460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67000aa510 0x7f670013c2b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f670006a6f0 0x7f6700146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 --2- 172.21.15.45:0/2990200310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6700146f80 0x7f67000a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 >> 172.21.15.45:0/2990200310 conn(0x7f6700067620 msgr2=0x7f67000ab740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 shutdown_connections 2023-10-06T05:15:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:22.685+0000 7f6705c27700 1 -- 172.21.15.45:0/2990200310 wait complete. 2023-10-06T05:15:22.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-10-06T05:15:23.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T05:15:23.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T05:15:23.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T05:15:23.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T05:15:23.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T05:15:23.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T05:15:23.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T05:15:23.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T05:15:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.829+0000 7ffb8a0c3700 1 Processor -- start 2023-10-06T05:15:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.829+0000 7ffb8a0c3700 1 -- start start 2023-10-06T05:15:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb8a0c3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb8415d770 0x7ffb840cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb8a0c3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb8415e930 0x7ffb840cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb8a0c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb84069fe0 0x7ffb8406a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb8a0c3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8415e4a0 con 0x7ffb84069fe0 2023-10-06T05:15:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb8a0c3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8406aca0 con 0x7ffb8415d770 2023-10-06T05:15:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb8a0c3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8415de50 con 0x7ffb8415e930 2023-10-06T05:15:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb837fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb8415d770 0x7ffb840cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb837fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb8415d770 0x7ffb840cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47590/0 (socket says 172.21.15.45:47590) 2023-10-06T05:15:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.831+0000 7ffb837fe700 1 -- 172.21.15.45:0/3500823079 learned_addr learned my addr 172.21.15.45:0/3500823079 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb83fff700 1 --2- 172.21.15.45:0/3500823079 >> v2:172.21.15.45:3300/0 conn(0x7ffb84069fe0 0x7ffb8406a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb837fe700 1 -- 172.21.15.45:0/3500823079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb8415e930 msgr2=0x7ffb840cc1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:15:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb837fe700 1 --2- 172.21.15.45:0/3500823079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb8415e930 0x7ffb840cc1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb837fe700 1 -- 172.21.15.45:0/3500823079 >> v2:172.21.15.45:3300/0 conn(0x7ffb84069fe0 msgr2=0x7ffb8406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb837fe700 1 --2- 172.21.15.45:0/3500823079 >> v2:172.21.15.45:3300/0 conn(0x7ffb84069fe0 0x7ffb8406a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb837fe700 1 -- 172.21.15.45:0/3500823079 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8415e130 con 0x7ffb8415d770 2023-10-06T05:15:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb837fe700 1 --2- 172.21.15.45:0/3500823079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415d770 0x7ffb840cbcb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb8406c470 tx=0x7ffb78009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca59a8585f15a20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb827fc700 1 -- 172.21.15.45:0/3500823079 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800b860 con 0x7ffb8415d770 2023-10-06T05:15:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.832+0000 7ffb827fc700 1 -- 172.21.15.45:0/3500823079 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7800b9c0 con 0x7ffb8415d770 2023-10-06T05:15:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.833+0000 7ffb827fc700 1 -- 172.21.15.45:0/3500823079 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800bd60 con 0x7ffb8415d770 2023-10-06T05:15:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3500823079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415d770 msgr2=0x7ffb840cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3500823079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415d770 0x7ffb840cbcb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb8406c470 tx=0x7ffb78009250 comp rx=0 tx=0).stop 2023-10-06T05:15:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3500823079 shutdown_connections 2023-10-06T05:15:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3500823079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415d770 0x7ffb840cbcb0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb8406c470 tx=0x7ffb78009250 comp rx=0 tx=0).stop 2023-10-06T05:15:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3500823079 >> v2:172.21.15.45:3300/0 conn(0x7ffb84069fe0 0x7ffb8406a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3500823079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb8415e930 0x7ffb840cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3500823079 >> 172.21.15.45:0/3500823079 conn(0x7ffb84067620 msgr2=0x7ffb84067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3500823079 shutdown_connections 2023-10-06T05:15:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.834+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3500823079 wait complete. 2023-10-06T05:15:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.836+0000 7ffb8a0c3700 1 Processor -- start 2023-10-06T05:15:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.836+0000 7ffb8a0c3700 1 -- start start 2023-10-06T05:15:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.837+0000 7ffb8a0c3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb84069fe0 0x7ffb84167880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb8a0c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415e930 0x7ffb84148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb8a0c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb841494b0 0x7ffb84165020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb8a0c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84161570 con 0x7ffb841494b0 2023-10-06T05:15:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb8a0c3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84149ca0 con 0x7ffb8415e930 2023-10-06T05:15:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb8a0c3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84149f80 con 0x7ffb84069fe0 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb841494b0 0x7ffb84165020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb841494b0 0x7ffb84165020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53386/0 (socket says 172.21.15.45:53386) 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.838+0000 7ffb83fff700 1 -- 172.21.15.45:0/3196184513 learned_addr learned my addr 172.21.15.45:0/3196184513 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.839+0000 7ffb82ffd700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415e930 0x7ffb84148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.839+0000 7ffb83fff700 1 -- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb84069fe0 msgr2=0x7ffb84167880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.839+0000 7ffb83fff700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb84069fe0 0x7ffb84167880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.839+0000 7ffb83fff700 1 -- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415e930 msgr2=0x7ffb84148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.839+0000 7ffb83fff700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415e930 0x7ffb84148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.839+0000 7ffb83fff700 1 -- 172.21.15.45:0/3196184513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780094f0 con 0x7ffb841494b0 2023-10-06T05:15:23.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.839+0000 7ffb83fff700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb841494b0 0x7ffb84165020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009fa0 tx=0x7ffb7c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.844+0000 7ffb6b7fe700 1 -- 172.21.15.45:0/3196184513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00ed20 con 0x7ffb841494b0 2023-10-06T05:15:23.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.844+0000 7ffb6b7fe700 1 -- 172.21.15.45:0/3196184513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c00ee80 con 0x7ffb841494b0 2023-10-06T05:15:23.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.844+0000 7ffb6b7fe700 1 -- 172.21.15.45:0/3196184513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c014650 con 0x7ffb841494b0 2023-10-06T05:15:23.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.845+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8414a260 con 0x7ffb841494b0 2023-10-06T05:15:23.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.845+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb84149910 con 0x7ffb841494b0 2023-10-06T05:15:23.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.851+0000 7ffb6b7fe700 1 -- 172.21.15.45:0/3196184513 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb7c014a00 con 0x7ffb841494b0 2023-10-06T05:15:23.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.852+0000 7ffb6b7fe700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb70067080 0x7ffb70069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.852+0000 7ffb6b7fe700 1 -- 172.21.15.45:0/3196184513 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ffb7c085130 con 0x7ffb841494b0 2023-10-06T05:15:23.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.853+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb640052b0 con 0x7ffb841494b0 2023-10-06T05:15:23.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.858+0000 7ffb6b7fe700 1 -- 172.21.15.45:0/3196184513 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb7c051f30 con 0x7ffb841494b0 2023-10-06T05:15:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.862+0000 7ffb837fe700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb70067080 0x7ffb70069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.871+0000 7ffb837fe700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb70067080 0x7ffb70069530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb8406c470 tx=0x7ffb78017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.873+0000 7f425062c700 1 Processor -- start 2023-10-06T05:15:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.874+0000 7f425062c700 1 -- start start 2023-10-06T05:15:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f425062c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4248154e40 0x7f424806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f425062c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f42480aa8a0 0x7f42480b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f425062c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424815b110 con 0x7f4248154a70 2023-10-06T05:15:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f425062c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42480ab5d0 con 0x7f42480aa8a0 2023-10-06T05:15:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f425062c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480ab010 con 0x7f4248154e40 2023-10-06T05:15:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f424e3c8700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4248154a70 0x7f424815b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50860/0 (socket says 172.21.15.45:50860) 2023-10-06T05:15:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f424e3c8700 1 -- 172.21.15.45:0/858986768 learned_addr learned my addr 172.21.15.45:0/858986768 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:23.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33075901 0 0) 0x7f424815b110 con 0x7f4248154a70 2023-10-06T05:15:23.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.876+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4234003570 con 0x7f4248154a70 2023-10-06T05:15:23.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594792651 0 0) 0x7f4234003570 con 0x7f4248154a70 2023-10-06T05:15:23.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4248154e40 msgr2=0x7f424806ee60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:15:23.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4248154e40 0x7f424806ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 >> v2:172.21.15.144:3300/0 conn(0x7f42480aa8a0 msgr2=0x7f42480b0e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 --2- 172.21.15.45:0/858986768 >> v2:172.21.15.144:3300/0 conn(0x7f42480aa8a0 0x7f42480b0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480ab2f0 con 0x7f4248154a70 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f4244001470 con 0x7f4248154a70 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 >> v1:172.21.15.144:6789/0 conn(0x7f4248154a70 legacy=0x7f424815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4234003ae0 0x7f4234003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234004810 0x7f4234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42340053c0 0x7f4234014290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480ab5d0 con 0x7f4234003ae0 2023-10-06T05:15:23.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480ab010 con 0x7f42340053c0 2023-10-06T05:15:23.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4234014bf0 con 0x7f4234004810 2023-10-06T05:15:23.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.877+0000 7f424e3c8700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4234003ae0 0x7f4234003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424ebc9700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42340053c0 0x7f4234014290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424dbc7700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234004810 0x7f4234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424e3c8700 1 -- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234004810 msgr2=0x7f4234004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424e3c8700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234004810 0x7f4234004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424e3c8700 1 -- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42340053c0 msgr2=0x7f4234014290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424e3c8700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42340053c0 0x7f4234014290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424e3c8700 1 -- 172.21.15.45:0/858986768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480ab2f0 con 0x7f4234003ae0 2023-10-06T05:15:23.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424dbc7700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234004810 0x7f4234004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.878+0000 7f424e3c8700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4234003ae0 0x7f4234003ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f424400c410 tx=0x7f4244007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446d994c5d2180c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.879+0000 7f9e8e457700 1 Processor -- start 2023-10-06T05:15:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.879+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244007c70 con 0x7f4234003ae0 2023-10-06T05:15:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.879+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424400ccc0 con 0x7f4234003ae0 2023-10-06T05:15:23.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.879+0000 7f424d3c6700 1 -- 172.21.15.45:0/858986768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244005b50 con 0x7f4234003ae0 2023-10-06T05:15:23.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.879+0000 7f9e8e457700 1 -- start start 2023-10-06T05:15:23.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f9e8e457700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e8815e930 0x7f9e88069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f9e8e457700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e8806a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f9e8e457700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8806aa60 con 0x7f9e8815d770 2023-10-06T05:15:23.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f9e8e457700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8815e4a0 con 0x7f9e8815e930 2023-10-06T05:15:23.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f9e8e457700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8806aca0 con 0x7f9e880cc3c0 2023-10-06T05:15:23.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f425062c700 1 -- 172.21.15.45:0/858986768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4234003ae0 msgr2=0x7f4234003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f425062c700 1 --2- 172.21.15.45:0/858986768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4234003ae0 0x7f4234003ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f424400c410 tx=0x7f4244007940 comp rx=0 tx=0).stop 2023-10-06T05:15:23.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f424ebc9700 1 -- 172.21.15.45:0/858986768 reap_dead start 2023-10-06T05:15:23.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f425062c700 1 -- 172.21.15.45:0/858986768 shutdown_connections 2023-10-06T05:15:23.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f425062c700 1 -- 172.21.15.45:0/858986768 >> 172.21.15.45:0/858986768 conn(0x7f4248067620 msgr2=0x7f4248067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f425062c700 1 -- 172.21.15.45:0/858986768 shutdown_connections 2023-10-06T05:15:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.880+0000 7f9e87fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9e8815d770 0x7f9e880cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50862/0 (socket says 172.21.15.45:50862) 2023-10-06T05:15:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e87fff700 1 -- 172.21.15.45:0/825592770 learned_addr learned my addr 172.21.15.45:0/825592770 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e8c9f4700 1 --2- 172.21.15.45:0/825592770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e8806a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e877fe700 1 --2- 172.21.15.45:0/825592770 >> v2:172.21.15.45:3300/0 conn(0x7f9e8815e930 0x7f9e88069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e8c9f4700 1 -- 172.21.15.45:0/825592770 >> v2:172.21.15.45:3300/0 conn(0x7f9e8815e930 msgr2=0x7f9e88069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e8c9f4700 1 --2- 172.21.15.45:0/825592770 >> v2:172.21.15.45:3300/0 conn(0x7f9e8815e930 0x7f9e88069fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e8c9f4700 1 -- 172.21.15.45:0/825592770 >> v1:172.21.15.144:6789/0 conn(0x7f9e8815d770 legacy=0x7f9e880cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e8c9f4700 1 -- 172.21.15.45:0/825592770 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8815de50 con 0x7f9e880cc3c0 2023-10-06T05:15:23.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f9e8c9f4700 1 --2- 172.21.15.45:0/825592770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e8806a520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e780098e0 tx=0x7f9e780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16856cdec947ee37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.881+0000 7f425062c700 1 -- 172.21.15.45:0/858986768 wait complete. 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.882+0000 7f425062c700 1 Processor -- start 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.883+0000 7f425062c700 1 -- start start 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f425062c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234003ae0 0x7f4248166010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f425062c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424814f8f0 0x7f424814a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f425062c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424814a6c0 0x7f424814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f425062c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248166550 con 0x7f424814a6c0 2023-10-06T05:15:23.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f425062c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248168fd0 con 0x7f424814f8f0 2023-10-06T05:15:23.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f425062c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42481692b0 con 0x7f4234003ae0 2023-10-06T05:15:23.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f424ebc9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424814a6c0 0x7f424814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f424ebc9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424814a6c0 0x7f424814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53422/0 (socket says 172.21.15.45:53422) 2023-10-06T05:15:23.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f424ebc9700 1 -- 172.21.15.45:0/1660093065 learned_addr learned my addr 172.21.15.45:0/1660093065 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.883+0000 7f67f7f73700 1 Processor -- start 2023-10-06T05:15:23.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.883+0000 7f67f7f73700 1 -- start start 2023-10-06T05:15:23.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f67f7f73700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f67f00b0130 0x7f67f0069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f67f7f73700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f015f1c0 con 0x7f67f006a520 2023-10-06T05:15:23.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f67f7f73700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f006ad10 con 0x7f67f0146f80 2023-10-06T05:15:23.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f67f7f73700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f006aff0 con 0x7f67f00b0130 2023-10-06T05:15:23.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.884+0000 7f67f5d0f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f67f0146f80 0x7f67f00afa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50864/0 (socket says 172.21.15.45:50864) 2023-10-06T05:15:23.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f5d0f700 1 -- 172.21.15.45:0/1810301698 learned_addr learned my addr 172.21.15.45:0/1810301698 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:23.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f550e700 1 --2- 172.21.15.45:0/1810301698 >> v2:172.21.15.45:3300/0 conn(0x7f67f00b0130 0x7f67f0069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f424ebc9700 1 -- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234003ae0 msgr2=0x7f4248166010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:15:23.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f424ebc9700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234003ae0 0x7f4248166010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f424ebc9700 1 -- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424814f8f0 msgr2=0x7f424814a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f424ebc9700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424814f8f0 0x7f424814a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f424ebc9700 1 -- 172.21.15.45:0/1660093065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248169590 con 0x7f424814a6c0 2023-10-06T05:15:23.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816613697 0 0) 0x7f67f006ad10 con 0x7f67f0146f80 2023-10-06T05:15:23.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67dc003570 con 0x7f67f0146f80 2023-10-06T05:15:23.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202354772 0 0) 0x7f67f015f1c0 con 0x7f67f006a520 2023-10-06T05:15:23.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67f006ad10 con 0x7f67f006a520 2023-10-06T05:15:23.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1084277299 0 0) 0x7f67dc003570 con 0x7f67f0146f80 2023-10-06T05:15:23.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 >> v2:172.21.15.45:3300/0 conn(0x7f67f00b0130 msgr2=0x7f67f0069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 --2- 172.21.15.45:0/1810301698 >> v2:172.21.15.45:3300/0 conn(0x7f67f00b0130 0x7f67f0069fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 >> v1:172.21.15.45:6789/0 conn(0x7f67f006a520 legacy=0x7f67f015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f006b2d0 con 0x7f67f0146f80 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.885+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f67ec001470 con 0x7f67f0146f80 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 >> v1:172.21.15.144:6789/0 conn(0x7f67f0146f80 legacy=0x7f67f00afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f4d0d700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67dc003c40 0x7f67dc004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f4d0d700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67dc004810 0x7f67dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f4d0d700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc005210 0x7f67dc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f006aff0 con 0x7f67dc005210 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc00c130 con 0x7f67dc004810 2023-10-06T05:15:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc00c2c0 con 0x7f67dc003c40 2023-10-06T05:15:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f550e700 1 --2- 172.21.15.45:0/1810301698 >> v2:172.21.15.45:3300/0 conn(0x7f67f00b0130 0x7f67f0069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67dc003c40 0x7f67dc004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f6510700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc005210 0x7f67dc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f5d0f700 1 -- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67dc004810 msgr2=0x7f67dc004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:15:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67dc004810 0x7f67dc004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f5d0f700 1 -- 172.21.15.45:0/1810301698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc005210 msgr2=0x7f67dc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc005210 0x7f67dc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f424ebc9700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424814a6c0 0x7f424814b150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f424815b080 tx=0x7f42400095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f5d0f700 1 -- 172.21.15.45:0/1810301698 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc003570 con 0x7f67dc003c40 2023-10-06T05:15:23.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f6510700 1 -- 172.21.15.45:0/1810301698 reap_dead start 2023-10-06T05:15:23.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.886+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67dc003c40 0x7f67dc004030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00c210 tx=0x7f67ec001d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13739a21c74a496f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.887+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec00cb00 con 0x7f67dc003c40 2023-10-06T05:15:23.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.887+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec00cca0 con 0x7f67dc003c40 2023-10-06T05:15:23.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.887+0000 7f67f4d0d700 1 -- 172.21.15.45:0/1810301698 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec005b70 con 0x7f67dc003c40 2023-10-06T05:15:23.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.887+0000 7f67f7f73700 1 -- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67dc003c40 msgr2=0x7f67dc004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.887+0000 7f67f7f73700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67dc003c40 0x7f67dc004030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00c210 tx=0x7f67ec001d40 comp rx=0 tx=0).stop 2023-10-06T05:15:23.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.890+0000 7fdf9a1e1700 1 Processor -- start 2023-10-06T05:15:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.889+0000 7f423e7fc700 1 -- 172.21.15.45:0/1660093065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011070 con 0x7f424814a6c0 2023-10-06T05:15:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.889+0000 7f423e7fc700 1 -- 172.21.15.45:0/1660093065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240015410 con 0x7f424814a6c0 2023-10-06T05:15:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.889+0000 7f423e7fc700 1 -- 172.21.15.45:0/1660093065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400157d0 con 0x7f424814a6c0 2023-10-06T05:15:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.889+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4248169870 con 0x7f424814a6c0 2023-10-06T05:15:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.889+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424814adb0 con 0x7f424814a6c0 2023-10-06T05:15:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.890+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42300052b0 con 0x7f424814a6c0 2023-10-06T05:15:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.890+0000 7fdf9a1e1700 1 -- start start 2023-10-06T05:15:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.891+0000 7fdf9a1e1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdf9415d4c0 0x7fdf9406a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.891+0000 7fdf937fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdf9415d4c0 0x7fdf9406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.891+0000 7fdf937fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdf9415d4c0 0x7fdf9406a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47650/0 (socket says 172.21.15.45:47650) 2023-10-06T05:15:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.892+0000 7fdf92ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdf9415d890 0x7fdf940c6bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58648/0 (socket says 172.21.15.45:58648) 2023-10-06T05:15:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.892+0000 7fdf937fe700 1 -- 172.21.15.45:0/312550264 learned_addr learned my addr 172.21.15.45:0/312550264 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.892+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf940c76c0 con 0x7fdf9415d890 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.892+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf940ac8d0 con 0x7fdf940c72c0 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.892+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf940c81a0 con 0x7fdf9415d4c0 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.893+0000 7fdf937fe700 1 -- 172.21.15.45:0/312550264 >> v1:172.21.15.144:6789/0 conn(0x7fdf940c72c0 legacy=0x7fdf940bf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.893+0000 7fdf927fc700 1 -- 172.21.15.45:0/312550264 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257959086 0 0) 0x7fdf940ac8d0 con 0x7fdf940c72c0 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.893+0000 7fdf937fe700 1 -- 172.21.15.45:0/312550264 >> v1:172.21.15.45:6789/0 conn(0x7fdf9415d890 legacy=0x7fdf940c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.893+0000 7fdf937fe700 1 -- 172.21.15.45:0/312550264 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940ac5a0 con 0x7fdf9415d4c0 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.894+0000 7fdf937fe700 1 --2- 172.21.15.45:0/312550264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf9406a890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdf840099b0 tx=0x7fdf84009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e39393834d286870 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.894+0000 7fdf927fc700 1 -- 172.21.15.45:0/312550264 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400b860 con 0x7fdf9415d4c0 2023-10-06T05:15:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.895+0000 7fdf927fc700 1 -- 172.21.15.45:0/312550264 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8400b9c0 con 0x7fdf9415d4c0 2023-10-06T05:15:23.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.895+0000 7fdf927fc700 1 -- 172.21.15.45:0/312550264 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400bd60 con 0x7fdf9415d4c0 2023-10-06T05:15:23.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.895+0000 7f9e86ffd700 1 -- 172.21.15.45:0/825592770 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78009e80 con 0x7f9e880cc3c0 2023-10-06T05:15:23.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.895+0000 7f9e86ffd700 1 -- 172.21.15.45:0/825592770 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7800b760 con 0x7f9e880cc3c0 2023-10-06T05:15:23.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.895+0000 7f9e86ffd700 1 -- 172.21.15.45:0/825592770 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800bb00 con 0x7f9e880cc3c0 2023-10-06T05:15:23.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7f9e8e457700 1 -- 172.21.15.45:0/825592770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 msgr2=0x7f9e8806a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7f9e8e457700 1 --2- 172.21.15.45:0/825592770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e8806a520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e780098e0 tx=0x7f9e780091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 msgr2=0x7fdf9406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/312550264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf9406a890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdf840099b0 tx=0x7fdf84009250 comp rx=0 tx=0).stop 2023-10-06T05:15:23.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7f9e8e457700 1 -- 172.21.15.45:0/825592770 shutdown_connections 2023-10-06T05:15:23.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7f9e8e457700 1 --2- 172.21.15.45:0/825592770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e8806a520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7f9e8e457700 1 --2- 172.21.15.45:0/825592770 >> v2:172.21.15.45:3300/0 conn(0x7f9e8815e930 0x7f9e88069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7f9e8e457700 1 -- 172.21.15.45:0/825592770 >> 172.21.15.45:0/825592770 conn(0x7f9e88067620 msgr2=0x7f9e88067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 shutdown_connections 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/312550264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf9406a890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 >> 172.21.15.45:0/312550264 conn(0x7fdf940cc5a0 msgr2=0x7fdf940cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 shutdown_connections 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.896+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/312550264 wait complete. 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.898+0000 7fdf9a1e1700 1 Processor -- start 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.899+0000 7fdf9a1e1700 1 -- start start 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf9a1e1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c72c0 0x7fdf9416de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf9a1e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf940afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf9a1e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9415d890 0x7fdf940b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf9a1e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf940c4360 con 0x7fdf9415d890 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf9a1e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf940c3750 con 0x7fdf9415d4c0 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf9a1e1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf940c3a30 con 0x7fdf940c72c0 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf93fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9415d890 0x7fdf940b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf93fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9415d890 0x7fdf940b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53444/0 (socket says 172.21.15.45:53444) 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf93fff700 1 -- 172.21.15.45:0/3002677711 learned_addr learned my addr 172.21.15.45:0/3002677711 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.900+0000 7fdf92ffd700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf940afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf937fe700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c72c0 0x7fdf9416de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf93fff700 1 -- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c72c0 msgr2=0x7fdf9416de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf93fff700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c72c0 0x7fdf9416de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf93fff700 1 -- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 msgr2=0x7fdf940afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf93fff700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf940afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf93fff700 1 -- 172.21.15.45:0/3002677711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf840094f0 con 0x7fdf9415d890 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf92ffd700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf940afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.901+0000 7fdf93fff700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9415d890 0x7fdf940b0e50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c001560 tx=0x7fdf8c001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.902+0000 7fdf8b7fe700 1 -- 172.21.15.45:0/3002677711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00a7e0 con 0x7fdf9415d890 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.902+0000 7fdf8b7fe700 1 -- 172.21.15.45:0/3002677711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c00a940 con 0x7fdf9415d890 2023-10-06T05:15:23.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.902+0000 7fdf8b7fe700 1 -- 172.21.15.45:0/3002677711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00ad00 con 0x7fdf9415d890 2023-10-06T05:15:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.902+0000 7f423e7fc700 1 -- 172.21.15.45:0/1660093065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4240015b80 con 0x7f424814a6c0 2023-10-06T05:15:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.903+0000 7f423e7fc700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f422c067080 0x7f422c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.903+0000 7f423e7fc700 1 -- 172.21.15.45:0/1660093065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4240085360 con 0x7f424814a6c0 2023-10-06T05:15:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.903+0000 7f424e3c8700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f422c067080 0x7f422c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.903+0000 7f423e7fc700 1 -- 172.21.15.45:0/1660093065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4240085750 con 0x7f424814a6c0 2023-10-06T05:15:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.903+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf940c3d10 con 0x7fdf9415d890 2023-10-06T05:15:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.903+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9416e340 con 0x7fdf9415d890 2023-10-06T05:15:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.904+0000 7f424e3c8700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f422c067080 0x7f422c069530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f42480ad170 tx=0x7f4244015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.904+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf940cc0e0 con 0x7fdf9415d890 2023-10-06T05:15:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.904+0000 7f67f7f73700 1 -- 172.21.15.45:0/1810301698 shutdown_connections 2023-10-06T05:15:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.904+0000 7f67f7f73700 1 --2- 172.21.15.45:0/1810301698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67dc003c40 0x7f67dc004030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.904+0000 7f67f7f73700 1 -- 172.21.15.45:0/1810301698 >> 172.21.15.45:0/1810301698 conn(0x7f67f0067620 msgr2=0x7f67f0067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.905+0000 7f67f7f73700 1 -- 172.21.15.45:0/1810301698 shutdown_connections 2023-10-06T05:15:23.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.905+0000 7fdf8b7fe700 1 -- 172.21.15.45:0/3002677711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf8c003030 con 0x7fdf9415d890 2023-10-06T05:15:23.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.906+0000 7fdf8b7fe700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf74067000 0x7fdf740694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.906+0000 7fdf8b7fe700 1 -- 172.21.15.45:0/3002677711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdf8c0849b0 con 0x7fdf9415d890 2023-10-06T05:15:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.910+0000 7f67f7f73700 1 -- 172.21.15.45:0/1810301698 wait complete. 2023-10-06T05:15:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.910+0000 7fdf8b7fe700 1 -- 172.21.15.45:0/3002677711 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf8c0517b0 con 0x7fdf9415d890 2023-10-06T05:15:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.911+0000 7fdf937fe700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf74067000 0x7fdf740694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.912+0000 7f67f7f73700 1 Processor -- start 2023-10-06T05:15:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.912+0000 7f67f7f73700 1 -- start start 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f7f73700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003c40 0x7f67f0155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f7f73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01562e0 0x7f67f0134a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f7f73700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f0134f50 0x7f67f015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f7f73700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f0156160 con 0x7f67dc003c40 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f7f73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f015c350 con 0x7f67f01562e0 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f7f73700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f01353c0 con 0x7f67f0134f50 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.911+0000 7fdf937fe700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf74067000 0x7fdf740694b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf840099b0 tx=0x7fdf84017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f9e8e457700 1 -- 172.21.15.45:0/825592770 shutdown_connections 2023-10-06T05:15:23.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f6510700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f0134f50 0x7f67f015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f6510700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f0134f50 0x7f67f015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34098/0 (socket says 172.21.15.45:34098) 2023-10-06T05:15:23.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f6510700 1 -- 172.21.15.45:0/2740822853 learned_addr learned my addr 172.21.15.45:0/2740822853 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f9e8e457700 1 -- 172.21.15.45:0/825592770 wait complete. 2023-10-06T05:15:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.915+0000 7f9e8e457700 1 Processor -- start 2023-10-06T05:15:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.913+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003c40 0x7f67f0155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.914+0000 7f67f5d0f700 1 -- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f0134f50 msgr2=0x7f67f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.914+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f0134f50 0x7f67f015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.914+0000 7f67f5d0f700 1 -- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01562e0 msgr2=0x7f67f0134a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:15:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.914+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01562e0 0x7f67f0134a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.914+0000 7f67f5d0f700 1 -- 172.21.15.45:0/2740822853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0156ac0 con 0x7f67dc003c40 2023-10-06T05:15:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.914+0000 7f67f5d0f700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003c40 0x7f67f0155c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67ec014000 tx=0x7f67ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.916+0000 7f9e8e457700 1 -- start start 2023-10-06T05:15:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e8e457700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e880a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e8e457700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8815d770 0x7f9e880a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e8e457700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e8815e930 0x7f9e8816df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e8e457700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880a7f30 con 0x7f9e8815e930 2023-10-06T05:15:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e8e457700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880a7070 con 0x7f9e8815d770 2023-10-06T05:15:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e8e457700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880a7320 con 0x7f9e880cc3c0 2023-10-06T05:15:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8815d770 0x7f9e880a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e880a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8815d770 0x7f9e880a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47668/0 (socket says 172.21.15.45:47668) 2023-10-06T05:15:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.917+0000 7f9e877fe700 1 -- 172.21.15.45:0/3674181305 learned_addr learned my addr 172.21.15.45:0/3674181305 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e87fff700 1 -- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8815d770 msgr2=0x7f9e880a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e87fff700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8815d770 0x7f9e880a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e87fff700 1 -- 172.21.15.45:0/3674181305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e8815e930 msgr2=0x7f9e8816df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e87fff700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e8815e930 0x7f9e8816df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e87fff700 1 -- 172.21.15.45:0/3674181305 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e78009420 con 0x7f9e880cc3c0 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e877fe700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8815d770 0x7f9e880a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e87fff700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e880a65f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c001eb0 tx=0x7f9e7c007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e6ffff700 1 -- 172.21.15.45:0/3674181305 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c005b30 con 0x7f9e880cc3c0 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e6ffff700 1 -- 172.21.15.45:0/3674181305 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c005cb0 con 0x7f9e880cc3c0 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e6ffff700 1 -- 172.21.15.45:0/3674181305 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c007340 con 0x7f9e880cc3c0 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e880a7600 con 0x7f9e880cc3c0 2023-10-06T05:15:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.918+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e880a7bc0 con 0x7f9e880cc3c0 2023-10-06T05:15:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.921+0000 7f67e5ffb700 1 -- 172.21.15.45:0/2740822853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec000c90 con 0x7f67dc003c40 2023-10-06T05:15:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.921+0000 7f67e5ffb700 1 -- 172.21.15.45:0/2740822853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec0070d0 con 0x7f67dc003c40 2023-10-06T05:15:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.921+0000 7f67e5ffb700 1 -- 172.21.15.45:0/2740822853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec007470 con 0x7f67dc003c40 2023-10-06T05:15:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.921+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e680052b0 con 0x7f9e880cc3c0 2023-10-06T05:15:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.922+0000 7f9e6ffff700 1 -- 172.21.15.45:0/3674181305 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e7c01b400 con 0x7f9e880cc3c0 2023-10-06T05:15:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.922+0000 7f9e6ffff700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e74066fb0 0x7f9e74069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.922+0000 7f9e6ffff700 1 -- 172.21.15.45:0/3674181305 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9e7c084470 con 0x7f9e880cc3c0 2023-10-06T05:15:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.924+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f0156da0 con 0x7f67dc003c40 2023-10-06T05:15:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.924+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f0157360 con 0x7f67dc003c40 2023-10-06T05:15:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.926+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f006cec0 con 0x7f67dc003c40 2023-10-06T05:15:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.927+0000 7f9e877fe700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e74066fb0 0x7f9e74069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.927+0000 7f9e6ffff700 1 -- 172.21.15.45:0/3674181305 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e7c051270 con 0x7f9e880cc3c0 2023-10-06T05:15:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.928+0000 7f67e5ffb700 1 -- 172.21.15.45:0/2740822853 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67ec01e430 con 0x7f67dc003c40 2023-10-06T05:15:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.929+0000 7f67e5ffb700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f67dc068aa0 0x7f67dc06af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.929+0000 7f67e5ffb700 1 -- 172.21.15.45:0/2740822853 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f67ec0854a0 con 0x7f67dc003c40 2023-10-06T05:15:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.934+0000 7f9e877fe700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e74066fb0 0x7f9e74069460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e880aa230 tx=0x7f9e80009880 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.935+0000 7f67f550e700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f67dc068aa0 0x7f67dc06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.935+0000 7f67f550e700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f67dc068aa0 0x7f67dc06af50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67f006e8d0 tx=0x7f67e00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.938+0000 7f67e5ffb700 1 -- 172.21.15.45:0/2740822853 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67ec0522a0 con 0x7f67dc003c40 2023-10-06T05:15:23.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.958+0000 7ff45e57b700 1 Processor -- start 2023-10-06T05:15:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.958+0000 7ff45e57b700 1 -- start start 2023-10-06T05:15:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff45e57b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4580adf10 0x7ff458161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff45e57b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45806b080 con 0x7ff4580cd4d0 2023-10-06T05:15:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff45e57b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45806b2d0 con 0x7ff4580c0170 2023-10-06T05:15:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff45e57b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45806b5b0 con 0x7ff4580adf10 2023-10-06T05:15:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff457fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff4580c0170 0x7ff4580ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50888/0 (socket says 172.21.15.45:50888) 2023-10-06T05:15:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff457fff700 1 -- 172.21.15.45:0/1040165937 learned_addr learned my addr 172.21.15.45:0/1040165937 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830281159 0 0) 0x7ff45806b2d0 con 0x7ff4580c0170 2023-10-06T05:15:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.960+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff444003570 con 0x7ff4580c0170 2023-10-06T05:15:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 217116932 0 0) 0x7ff444003570 con 0x7ff4580c0170 2023-10-06T05:15:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 >> v2:172.21.15.45:3300/0 conn(0x7ff4580adf10 msgr2=0x7ff458161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 --2- 172.21.15.45:0/1040165937 >> v2:172.21.15.45:3300/0 conn(0x7ff4580adf10 0x7ff458161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 >> v1:172.21.15.45:6789/0 conn(0x7ff4580cd4d0 legacy=0x7ff458161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45806b890 con 0x7ff4580c0170 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7ff4500017f0 con 0x7ff4580c0170 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 >> v1:172.21.15.144:6789/0 conn(0x7ff4580c0170 legacy=0x7ff4580ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff444003b30 0x7ff444003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff444004810 0x7ff444004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff444005410 0x7ff44400bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45806b5b0 con 0x7ff444004810 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44400c3a0 con 0x7ff444003b30 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44400c560 con 0x7ff444005410 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff457fff700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff444003b30 0x7ff444003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.961+0000 7ff4577fe700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff444004810 0x7ff444004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.962+0000 7ff4577fe700 1 -- 172.21.15.45:0/1040165937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff444005410 msgr2=0x7ff44400bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.962+0000 7ff4577fe700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff444005410 0x7ff44400bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.962+0000 7ff4577fe700 1 -- 172.21.15.45:0/1040165937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff444003b30 msgr2=0x7ff444003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.962+0000 7ff4577fe700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff444003b30 0x7ff444003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.962+0000 7ff4577fe700 1 -- 172.21.15.45:0/1040165937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444003570 con 0x7ff444004810 2023-10-06T05:15:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.962+0000 7ff4577fe700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff444004810 0x7ff444004c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff448001790 tx=0x7ff448007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00a4193e28916ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.962+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448012400 con 0x7ff444004810 2023-10-06T05:15:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.963+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff448012560 con 0x7ff444004810 2023-10-06T05:15:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.963+0000 7ff456ffd700 1 -- 172.21.15.45:0/1040165937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448012920 con 0x7ff444004810 2023-10-06T05:15:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.964+0000 7ff45e57b700 1 -- 172.21.15.45:0/1040165937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff444004810 msgr2=0x7ff444004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.964+0000 7ff45e57b700 1 --2- 172.21.15.45:0/1040165937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff444004810 0x7ff444004c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff448001790 tx=0x7ff448007d10 comp rx=0 tx=0).stop 2023-10-06T05:15:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.965+0000 7ff45cb18700 1 -- 172.21.15.45:0/1040165937 reap_dead start 2023-10-06T05:15:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.965+0000 7ff45e57b700 1 -- 172.21.15.45:0/1040165937 shutdown_connections 2023-10-06T05:15:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.965+0000 7ff45e57b700 1 -- 172.21.15.45:0/1040165937 >> 172.21.15.45:0/1040165937 conn(0x7ff45806f090 msgr2=0x7ff45806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.966+0000 7ff45e57b700 1 -- 172.21.15.45:0/1040165937 shutdown_connections 2023-10-06T05:15:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.967+0000 7ff45e57b700 1 -- 172.21.15.45:0/1040165937 wait complete. 2023-10-06T05:15:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.968+0000 7ff45e57b700 1 Processor -- start 2023-10-06T05:15:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.969+0000 7ff45e57b700 1 -- start start 2023-10-06T05:15:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff45e57b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff45815a550 0x7ff458163500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff45e57b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff458163a40 0x7ff4580b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff45e57b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4580b1c70 0x7ff4580afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff45e57b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580c0650 con 0x7ff4580b1c70 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff45e57b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff458163eb0 con 0x7ff45815a550 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff45e57b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580b2a60 con 0x7ff458163a40 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff4577fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff458163a40 0x7ff4580b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.970+0000 7ff4577fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff458163a40 0x7ff4580b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34128/0 (socket says 172.21.15.45:34128) 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.971+0000 7ff4577fe700 1 -- 172.21.15.45:0/3522294043 learned_addr learned my addr 172.21.15.45:0/3522294043 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.971+0000 7ff45cb18700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4580b1c70 0x7ff4580afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.971+0000 7ff4577fe700 1 -- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff45815a550 msgr2=0x7ff458163500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:15:23.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.971+0000 7ff4577fe700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff45815a550 0x7ff458163500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.971+0000 7ff4577fe700 1 -- 172.21.15.45:0/3522294043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4580b1c70 msgr2=0x7ff4580afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.971+0000 7ff4577fe700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4580b1c70 0x7ff4580afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.971+0000 7ff4577fe700 1 -- 172.21.15.45:0/3522294043 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4580c42e0 con 0x7ff458163a40 2023-10-06T05:15:23.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7ff4577fe700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff458163a40 0x7ff4580b1730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff44400c370 tx=0x7ff448015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7fab7c961700 1 Processor -- start 2023-10-06T05:15:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7fab7c961700 1 -- start start 2023-10-06T05:15:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7ff44ffff700 1 -- 172.21.15.45:0/3522294043 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011b10 con 0x7ff458163a40 2023-10-06T05:15:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7ff44ffff700 1 -- 172.21.15.45:0/3522294043 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff448011c70 con 0x7ff458163a40 2023-10-06T05:15:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7ff44ffff700 1 -- 172.21.15.45:0/3522294043 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44801e430 con 0x7ff458163a40 2023-10-06T05:15:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45815a920 con 0x7ff458163a40 2023-10-06T05:15:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.972+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4580b2780 con 0x7ff458163a40 2023-10-06T05:15:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7c961700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab7806aa80 0x7fab7806ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7c961700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab78158f80 0x7fab781593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7c961700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab78158af0 con 0x7fab78157690 2023-10-06T05:15:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7c961700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab78157ee0 con 0x7fab78158f80 2023-10-06T05:15:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7c961700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab781581c0 con 0x7fab7806aa80 2023-10-06T05:15:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7659c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab7806aa80 0x7fab7806ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7659c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab7806aa80 0x7fab7806ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34140/0 (socket says 172.21.15.45:34140) 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7659c700 1 -- 172.21.15.45:0/2355060314 learned_addr learned my addr 172.21.15.45:0/2355060314 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7659c700 1 -- 172.21.15.45:0/2355060314 >> v2:172.21.15.45:3300/0 conn(0x7fab78158f80 msgr2=0x7fab781593e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7659c700 1 --2- 172.21.15.45:0/2355060314 >> v2:172.21.15.45:3300/0 conn(0x7fab78158f80 0x7fab781593e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7659c700 1 -- 172.21.15.45:0/2355060314 >> v1:172.21.15.144:6789/0 conn(0x7fab78157690 legacy=0x7fab78154100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.974+0000 7fab7659c700 1 -- 172.21.15.45:0/2355060314 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab781584a0 con 0x7fab7806aa80 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.975+0000 7fab7659c700 1 --2- 172.21.15.45:0/2355060314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab7806aa80 0x7fab7806ae50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab70009f50 tx=0x7fab70009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf88b367d154e5b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.975+0000 7fab7559a700 1 -- 172.21.15.45:0/2355060314 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7000e070 con 0x7fab7806aa80 2023-10-06T05:15:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.975+0000 7fab7559a700 1 -- 172.21.15.45:0/2355060314 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7000ba30 con 0x7fab7806aa80 2023-10-06T05:15:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.975+0000 7fab7559a700 1 -- 172.21.15.45:0/2355060314 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700092e0 con 0x7fab7806aa80 2023-10-06T05:15:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.978+0000 7fab7c961700 1 -- 172.21.15.45:0/2355060314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab7806aa80 msgr2=0x7fab7806ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.978+0000 7fab7c961700 1 --2- 172.21.15.45:0/2355060314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab7806aa80 0x7fab7806ae50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab70009f50 tx=0x7fab70009840 comp rx=0 tx=0).stop 2023-10-06T05:15:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.978+0000 7fab7c961700 1 -- 172.21.15.45:0/2355060314 shutdown_connections 2023-10-06T05:15:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.978+0000 7fab7c961700 1 --2- 172.21.15.45:0/2355060314 >> v2:172.21.15.45:3300/0 conn(0x7fab78158f80 0x7fab781593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.978+0000 7fab7c961700 1 --2- 172.21.15.45:0/2355060314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab7806aa80 0x7fab7806ae50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.978+0000 7fab7c961700 1 -- 172.21.15.45:0/2355060314 >> 172.21.15.45:0/2355060314 conn(0x7fab78067620 msgr2=0x7fab78067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.979+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45806e2f0 con 0x7ff458163a40 2023-10-06T05:15:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.981+0000 7fab7c961700 1 -- 172.21.15.45:0/2355060314 shutdown_connections 2023-10-06T05:15:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.980+0000 7ff44ffff700 1 -- 172.21.15.45:0/3522294043 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff44801e860 con 0x7ff458163a40 2023-10-06T05:15:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.981+0000 7ff44ffff700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff438066f30 0x7ff4380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.981+0000 7ff44ffff700 1 -- 172.21.15.45:0/3522294043 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff4480862d0 con 0x7ff458163a40 2023-10-06T05:15:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.981+0000 7ff457fff700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff438066f30 0x7ff4380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.981+0000 7fab7c961700 1 -- 172.21.15.45:0/2355060314 wait complete. 2023-10-06T05:15:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.983+0000 7fab7c961700 1 Processor -- start 2023-10-06T05:15:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.981+0000 7ff457fff700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff438066f30 0x7ff4380693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff4500013d0 tx=0x7ff4580aca70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.984+0000 7fab7c961700 1 -- start start 2023-10-06T05:15:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.985+0000 7fab7c961700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7806aa80 0x7fab78149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.985+0000 7fab7c961700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab78158f80 0x7fab7814aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.985+0000 7fab7c961700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7816ddd0 0x7fab7806df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.985+0000 7fab7c961700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78149e60 con 0x7fab7806aa80 2023-10-06T05:15:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.985+0000 7fab7c961700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7816efa0 con 0x7fab7816ddd0 2023-10-06T05:15:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.985+0000 7fab7c961700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab780cbe10 con 0x7fab78158f80 2023-10-06T05:15:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.985+0000 7ff44ffff700 1 -- 172.21.15.45:0/3522294043 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff448053110 con 0x7ff458163a40 2023-10-06T05:15:23.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7fab7659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7806aa80 0x7fab78149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7fab7659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7806aa80 0x7fab78149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53494/0 (socket says 172.21.15.45:53494) 2023-10-06T05:15:23.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7fab7659c700 1 -- 172.21.15.45:0/780140699 learned_addr learned my addr 172.21.15.45:0/780140699 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:23.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7fab76d9d700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7816ddd0 0x7fab7806df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7fab75d9b700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab78158f80 0x7fab7814aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.994+0000 7fab7659c700 1 -- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab78158f80 msgr2=0x7fab7814aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.994+0000 7fab7659c700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab78158f80 0x7fab7814aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.994+0000 7fab7659c700 1 -- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7816ddd0 msgr2=0x7fab7806df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.994+0000 7fab7659c700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7816ddd0 0x7fab7806df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.994+0000 7fab7659c700 1 -- 172.21.15.45:0/780140699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70009ae0 con 0x7fab7806aa80 2023-10-06T05:15:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.994+0000 7fab7659c700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7806aa80 0x7fab78149920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab78158300 tx=0x7fab70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:23.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.995+0000 7fab667fc700 1 -- 172.21.15.45:0/780140699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70012950 con 0x7fab7806aa80 2023-10-06T05:15:23.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.995+0000 7fab667fc700 1 -- 172.21.15.45:0/780140699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab70012ab0 con 0x7fab7806aa80 2023-10-06T05:15:23.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.995+0000 7fab667fc700 1 -- 172.21.15.45:0/780140699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70012e70 con 0x7fab7806aa80 2023-10-06T05:15:23.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.995+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7816e290 con 0x7fab7806aa80 2023-10-06T05:15:23.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.995+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7816e850 con 0x7fab7806aa80 2023-10-06T05:15:23.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.996+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c0052b0 con 0x7fab7806aa80 2023-10-06T05:15:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.986+0000 7f3a8e0bc700 1 Processor -- start 2023-10-06T05:15:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.986+0000 7f3a8e0bc700 1 -- start start 2023-10-06T05:15:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.988+0000 7f3a8e0bc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3a880cc3c0 0x7f3a880aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.988+0000 7f3a8e0bc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a880af3e0 0x7f3a881668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.988+0000 7f3a8e0bc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a880afc40 con 0x7f3a88165f60 2023-10-06T05:15:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.988+0000 7f3a8e0bc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880b13e0 con 0x7f3a880af3e0 2023-10-06T05:15:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.988+0000 7f3a8e0bc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880aff20 con 0x7f3a880cc3c0 2023-10-06T05:15:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7f3a87fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a880af3e0 0x7f3a881668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7f3a87fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a880af3e0 0x7f3a881668f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53496/0 (socket says 172.21.15.45:53496) 2023-10-06T05:15:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7f3a87fff700 1 -- 172.21.15.45:0/2882216411 learned_addr learned my addr 172.21.15.45:0/2882216411 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.994+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348732100 0 0) 0x7f3a880afc40 con 0x7f3a88165f60 2023-10-06T05:15:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.993+0000 7f3a86ffd700 1 --2- 172.21.15.45:0/2882216411 >> v2:172.21.15.144:3300/0 conn(0x7f3a880cc3c0 0x7f3a880aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.997+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a700039c0 con 0x7f3a88165f60 2023-10-06T05:15:24.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789613353 0 0) 0x7f3a700039c0 con 0x7f3a88165f60 2023-10-06T05:15:24.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 >> v2:172.21.15.144:3300/0 conn(0x7f3a880cc3c0 msgr2=0x7f3a880aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:24.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 --2- 172.21.15.45:0/2882216411 >> v2:172.21.15.144:3300/0 conn(0x7f3a880cc3c0 0x7f3a880aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 >> v2:172.21.15.45:3300/0 conn(0x7f3a880af3e0 msgr2=0x7f3a881668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 --2- 172.21.15.45:0/2882216411 >> v2:172.21.15.45:3300/0 conn(0x7f3a880af3e0 0x7f3a881668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880b0200 con 0x7f3a88165f60 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f3a7c001470 con 0x7f3a88165f60 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 >> v1:172.21.15.144:6789/0 conn(0x7f3a88165f60 legacy=0x7f3a880cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a70003c00 0x7f3a70004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700045a0 0x7f3a70004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a700051a0 0x7f3a7000beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a880aff20 con 0x7f3a700051a0 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a700058f0 con 0x7f3a700045a0 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a70005ab0 con 0x7f3a70003c00 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:23.999+0000 7f3a87fff700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a700051a0 0x7f3a7000beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7f3a87fff700 1 -- 172.21.15.45:0/2882216411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a70003c00 msgr2=0x7f3a70004010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7f3a87fff700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a70003c00 0x7f3a70004010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7f3a87fff700 1 -- 172.21.15.45:0/2882216411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700045a0 msgr2=0x7f3a70004c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7f3a87fff700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700045a0 0x7f3a70004c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7f3a87fff700 1 -- 172.21.15.45:0/2882216411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880b0200 con 0x7f3a700051a0 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7f3a87fff700 1 -- 172.21.15.45:0/2882216411 reap_dead start 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7f3a87fff700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a700051a0 0x7f3a7000beb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009da0 tx=0x7f3a800091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc03b87180902d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7fab667fc700 1 -- 172.21.15.45:0/780140699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab7001e640 con 0x7fab7806aa80 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7fab667fc700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab58067080 0x7fab58069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.000+0000 7fab667fc700 1 -- 172.21.15.45:0/780140699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fab70086e00 con 0x7fab7806aa80 2023-10-06T05:15:24.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.006+0000 7fab75d9b700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab58067080 0x7fab58069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.007+0000 7fab667fc700 1 -- 172.21.15.45:0/780140699 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab70053c80 con 0x7fab7806aa80 2023-10-06T05:15:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.007+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80009790 con 0x7f3a700051a0 2023-10-06T05:15:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.007+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000f040 con 0x7f3a700051a0 2023-10-06T05:15:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.007+0000 7f3a867fc700 1 -- 172.21.15.45:0/2882216411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80007da0 con 0x7f3a700051a0 2023-10-06T05:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.008+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/2882216411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a700051a0 msgr2=0x7f3a7000beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.008+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a700051a0 0x7f3a7000beb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009da0 tx=0x7f3a800091c0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.009+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/2882216411 shutdown_connections 2023-10-06T05:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.009+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/2882216411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a700051a0 0x7f3a7000beb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.009+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/2882216411 >> 172.21.15.45:0/2882216411 conn(0x7f3a88067620 msgr2=0x7f3a88067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.012+0000 7fab75d9b700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab58067080 0x7fab58069530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab780cbb00 tx=0x7fab680090f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.009+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/2882216411 shutdown_connections 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.009+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/2882216411 wait complete. 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.011+0000 7f3a8e0bc700 1 Processor -- start 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.012+0000 7f3a8e0bc700 1 -- start start 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.013+0000 7f3a8e0bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700051a0 0x7f3a8816ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.013+0000 7f3a8e0bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8815c740 0x7f3a880ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.013+0000 7f3a8e0bc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a8815a790 0x7f3a8815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.013+0000 7f3a8e0bc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8815e0a0 con 0x7f3a8815c740 2023-10-06T05:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.013+0000 7f3a8e0bc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8815dc10 con 0x7f3a700051a0 2023-10-06T05:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.013+0000 7f3a8e0bc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8815b120 con 0x7f3a8815a790 2023-10-06T05:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.014+0000 7f3a87fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a8815a790 0x7f3a8815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.014+0000 7f3a877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700051a0 0x7f3a8816ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8815c740 0x7f3a880ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8815c740 0x7f3a880ad000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53512/0 (socket says 172.21.15.45:53512) 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 -- 172.21.15.45:0/3393518215 learned_addr learned my addr 172.21.15.45:0/3393518215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 -- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a8815a790 msgr2=0x7f3a8815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a8815a790 0x7f3a8815abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 -- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700051a0 msgr2=0x7f3a8816ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700051a0 0x7f3a8816ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.015+0000 7f3a86ffd700 1 -- 172.21.15.45:0/3393518215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8815d000 con 0x7f3a8815c740 2023-10-06T05:15:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.016+0000 7f3a86ffd700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8815c740 0x7f3a880ad000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009a00 tx=0x7f3a78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.016+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/3393518215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800c920 con 0x7f3a8815c740 2023-10-06T05:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.016+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/3393518215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7800ca80 con 0x7f3a8815c740 2023-10-06T05:15:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.016+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/3393518215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800ce40 con 0x7f3a8815c740 2023-10-06T05:15:24.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.016+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8815d2e0 con 0x7f3a8815c740 2023-10-06T05:15:24.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.016+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8815d8a0 con 0x7f3a8815c740 2023-10-06T05:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.022+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/3393518215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a78011600 con 0x7f3a8815c740 2023-10-06T05:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.022+0000 7f3a6f7fe700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a74067080 0x7f3a74069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.022+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/3393518215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3a78084420 con 0x7f3a8815c740 2023-10-06T05:15:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.023+0000 7f3a877fe700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a74067080 0x7f3a74069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.024+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a680052b0 con 0x7f3a8815c740 2023-10-06T05:15:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.034+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/3393518215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a78051220 con 0x7f3a8815c740 2023-10-06T05:15:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.034+0000 7f3a877fe700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a74067080 0x7f3a74069530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c001660 tx=0x7f3a7c0083b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.289+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffb64005560 con 0x7ffb841494b0 2023-10-06T05:15:24.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.290+0000 7ffb6b7fe700 1 -- 172.21.15.45:0/3196184513 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb7c014cb0 con 0x7ffb841494b0 2023-10-06T05:15:24.295 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:24.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb70067080 msgr2=0x7ffb70069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb70067080 0x7ffb70069530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb8406c470 tx=0x7ffb78017000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb841494b0 msgr2=0x7ffb84165020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb841494b0 0x7ffb84165020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009fa0 tx=0x7ffb7c009900 comp rx=0 tx=0).stop 2023-10-06T05:15:24.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 shutdown_connections 2023-10-06T05:15:24.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb70067080 0x7ffb70069530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb841494b0 0x7ffb84165020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb84069fe0 0x7ffb84167880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 --2- 172.21.15.45:0/3196184513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb8415e930 0x7ffb84148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 >> 172.21.15.45:0/3196184513 conn(0x7ffb84067620 msgr2=0x7ffb8414dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.294+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 shutdown_connections 2023-10-06T05:15:24.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.295+0000 7ffb8a0c3700 1 -- 172.21.15.45:0/3196184513 wait complete. 2023-10-06T05:15:24.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-06T05:15:24.317 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.326+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdf940c44a0 con 0x7fdf9415d890 2023-10-06T05:15:24.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.327+0000 7fdf8b7fe700 1 -- 172.21.15.45:0/3002677711 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf940c44a0 con 0x7fdf9415d890 2023-10-06T05:15:24.329 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2023-10-06T05:15:24.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4230005830 con 0x7f424814a6c0 2023-10-06T05:15:24.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf74067000 msgr2=0x7fdf740694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf74067000 0x7fdf740694b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf840099b0 tx=0x7fdf84017000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9415d890 msgr2=0x7fdf940b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9415d890 0x7fdf940b0e50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c001560 tx=0x7fdf8c001770 comp rx=0 tx=0).stop 2023-10-06T05:15:24.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7f423e7fc700 1 -- 172.21.15.45:0/1660093065 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4240052290 con 0x7f424814a6c0 2023-10-06T05:15:24.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 shutdown_connections 2023-10-06T05:15:24.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf74067000 0x7fdf740694b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9415d890 0x7fdf940b0e50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c72c0 0x7fdf9416de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 --2- 172.21.15.45:0/3002677711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9415d4c0 0x7fdf940afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.330+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 >> 172.21.15.45:0/3002677711 conn(0x7fdf940cc5a0 msgr2=0x7fdf9415fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.333 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:24.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.331+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 shutdown_connections 2023-10-06T05:15:24.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.332+0000 7fdf9a1e1700 1 -- 172.21.15.45:0/3002677711 wait complete. 2023-10-06T05:15:24.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f422c067080 msgr2=0x7f422c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f422c067080 0x7f422c069530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f42480ad170 tx=0x7f4244015000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424814a6c0 msgr2=0x7f424814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424814a6c0 0x7f424814b150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f424815b080 tx=0x7f42400095f0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 shutdown_connections 2023-10-06T05:15:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f422c067080 0x7f422c069530 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424814a6c0 0x7f424814b150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4234003ae0 0x7f4248166010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.334+0000 7f425062c700 1 --2- 172.21.15.45:0/1660093065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424814f8f0 0x7f424814a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.335+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 >> 172.21.15.45:0/1660093065 conn(0x7f4248067620 msgr2=0x7f4248067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.335+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 shutdown_connections 2023-10-06T05:15:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.335+0000 7f425062c700 1 -- 172.21.15.45:0/1660093065 wait complete. 2023-10-06T05:15:24.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.337+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f67f015ee50 con 0x7f67dc003c40 2023-10-06T05:15:24.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.338+0000 7f67e5ffb700 1 -- 172.21.15.45:0/2740822853 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67ec0558c0 con 0x7f67dc003c40 2023-10-06T05:15:24.340 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.339+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9e68005560 con 0x7f9e880cc3c0 2023-10-06T05:15:24.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.341+0000 7f9e6ffff700 1 -- 172.21.15.45:0/3674181305 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e7c054890 con 0x7f9e880cc3c0 2023-10-06T05:15:24.347 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f67dc068aa0 msgr2=0x7f67dc06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f67dc068aa0 0x7f67dc06af50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67f006e8d0 tx=0x7f67e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003c40 msgr2=0x7f67f0155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003c40 0x7f67f0155c20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67ec014000 tx=0x7f67ec015000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 shutdown_connections 2023-10-06T05:15:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f67dc068aa0 0x7f67dc06af50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f0134f50 0x7f67f015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003c40 0x7f67f0155c20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 --2- 172.21.15.45:0/2740822853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01562e0 0x7f67f0134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 >> 172.21.15.45:0/2740822853 conn(0x7f67f0067620 msgr2=0x7f67f006df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 shutdown_connections 2023-10-06T05:15:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.346+0000 7f67f7f73700 1 -- 172.21.15.45:0/2740822853 wait complete. 2023-10-06T05:15:24.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e74066fb0 msgr2=0x7f9e74069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e74066fb0 0x7f9e74069460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e880aa230 tx=0x7f9e80009880 comp rx=0 tx=0).stop 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 msgr2=0x7f9e880a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e880a65f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c001eb0 tx=0x7f9e7c007e40 comp rx=0 tx=0).stop 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 shutdown_connections 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e74066fb0 0x7f9e74069460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e8815e930 0x7f9e8816df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e880cc3c0 0x7f9e880a65f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 --2- 172.21.15.45:0/3674181305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8815d770 0x7f9e880a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.350+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 >> 172.21.15.45:0/3674181305 conn(0x7f9e88067620 msgr2=0x7f9e8815ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-10-06T05:15:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.357+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 shutdown_connections 2023-10-06T05:15:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.357+0000 7f9e8e457700 1 -- 172.21.15.45:0/3674181305 wait complete. 2023-10-06T05:15:24.360 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-06T05:15:24.362 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-06T05:15:24.366 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T05:15:24.377 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.390+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff4580bfd00 con 0x7ff458163a40 2023-10-06T05:15:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.391+0000 7ff44ffff700 1 -- 172.21.15.45:0/3522294043 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff448017020 con 0x7ff458163a40 2023-10-06T05:15:24.393 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2023-10-06T05:15:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff438066f30 msgr2=0x7ff4380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff438066f30 0x7ff4380693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff4500013d0 tx=0x7ff4580aca70 comp rx=0 tx=0).stop 2023-10-06T05:15:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff458163a40 msgr2=0x7ff4580b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff458163a40 0x7ff4580b1730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff44400c370 tx=0x7ff448015000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 shutdown_connections 2023-10-06T05:15:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff438066f30 0x7ff4380693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4580b1c70 0x7ff4580afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff45815a550 0x7ff458163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 --2- 172.21.15.45:0/3522294043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff458163a40 0x7ff4580b1730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 >> 172.21.15.45:0/3522294043 conn(0x7ff45806f090 msgr2=0x7ff45815ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 shutdown_connections 2023-10-06T05:15:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.393+0000 7ff45e57b700 1 -- 172.21.15.45:0/3522294043 wait complete. 2023-10-06T05:15:24.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.396+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fab5c005830 con 0x7fab7806aa80 2023-10-06T05:15:24.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.396+0000 7fab667fc700 1 -- 172.21.15.45:0/780140699 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab700572a0 con 0x7fab7806aa80 2023-10-06T05:15:24.398 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2023-10-06T05:15:24.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.398+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3a68005560 con 0x7f3a8815c740 2023-10-06T05:15:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.398+0000 7f3a6f7fe700 1 -- 172.21.15.45:0/3393518215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a78020020 con 0x7f3a8815c740 2023-10-06T05:15:24.400 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab58067080 msgr2=0x7fab58069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab58067080 0x7fab58069530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab780cbb00 tx=0x7fab680090f0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7806aa80 msgr2=0x7fab78149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7806aa80 0x7fab78149920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab78158300 tx=0x7fab70017000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 shutdown_connections 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab58067080 0x7fab58069530 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7816ddd0 0x7fab7806df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7806aa80 0x7fab78149920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 --2- 172.21.15.45:0/780140699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab78158f80 0x7fab7814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 >> 172.21.15.45:0/780140699 conn(0x7fab78067620 msgr2=0x7fab78150c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 shutdown_connections 2023-10-06T05:15:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.399+0000 7fab7c961700 1 -- 172.21.15.45:0/780140699 wait complete. 2023-10-06T05:15:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a74067080 msgr2=0x7f3a74069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a74067080 0x7f3a74069530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c001660 tx=0x7f3a7c0083b0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8815c740 msgr2=0x7f3a880ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8815c740 0x7f3a880ad000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009a00 tx=0x7f3a78008040 comp rx=0 tx=0).stop 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 shutdown_connections 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a74067080 0x7f3a74069530 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a8815a790 0x7f3a8815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a700051a0 0x7f3a8816ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 --2- 172.21.15.45:0/3393518215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8815c740 0x7f3a880ad000 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.400+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 >> 172.21.15.45:0/3393518215 conn(0x7f3a88067620 msgr2=0x7f3a8806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.401+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 shutdown_connections 2023-10-06T05:15:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.401+0000 7f3a8e0bc700 1 -- 172.21.15.45:0/3393518215 wait complete. 2023-10-06T05:15:24.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-10-06T05:15:24.414 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-10-06T05:15:24.417 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-10-06T05:15:24.418 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:24.418 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:15:24.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:24.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.592+0000 7fba2f73d700 1 Processor -- start 2023-10-06T05:15:24.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.592+0000 7fba2f73d700 1 -- start start 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2f73d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba28158f60 0x7fba28159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2f73d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fba280cc390 0x7fba280cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2f73d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba280cb530 con 0x7fba2806a060 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2f73d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba280ccd50 con 0x7fba280cc390 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2f73d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba28069bd0 con 0x7fba28158f60 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2ccd8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fba2806a060 0x7fba280cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58664/0 (socket says 172.21.15.45:58664) 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2ccd8700 1 -- 172.21.15.45:0/1251324491 learned_addr learned my addr 172.21.15.45:0/1251324491 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2dcda700 1 --2- 172.21.15.45:0/1251324491 >> v2:172.21.15.144:3300/0 conn(0x7fba280cc390 0x7fba280cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.593+0000 7fba2d4d9700 1 --2- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba28158f60 0x7fba28159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba1ffff700 1 -- 172.21.15.45:0/1251324491 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303596118 0 0) 0x7fba18001110 con 0x7fba2806a060 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba1ffff700 1 -- 172.21.15.45:0/1251324491 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba14003570 con 0x7fba2806a060 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba2dcda700 1 -- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba28158f60 msgr2=0x7fba28159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba2dcda700 1 --2- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba28158f60 0x7fba28159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba2dcda700 1 -- 172.21.15.45:0/1251324491 >> v1:172.21.15.45:6789/0 conn(0x7fba2806a060 legacy=0x7fba280cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba2dcda700 1 -- 172.21.15.45:0/1251324491 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2806a4d0 con 0x7fba280cc390 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba2d4d9700 1 --2- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba28158f60 0x7fba28159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba2dcda700 1 --2- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba280cc390 0x7fba280cc810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba20009e60 tx=0x7fba20009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69afafb373d94e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba1ffff700 1 -- 172.21.15.45:0/1251324491 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2000e070 con 0x7fba280cc390 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba1ffff700 1 -- 172.21.15.45:0/1251324491 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2000bab0 con 0x7fba280cc390 2023-10-06T05:15:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.594+0000 7fba1ffff700 1 -- 172.21.15.45:0/1251324491 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2000be30 con 0x7fba280cc390 2023-10-06T05:15:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 -- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba280cc390 msgr2=0x7fba280cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 --2- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba280cc390 0x7fba280cc810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fba20009e60 tx=0x7fba20009780 comp rx=0 tx=0).stop 2023-10-06T05:15:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 -- 172.21.15.45:0/1251324491 shutdown_connections 2023-10-06T05:15:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 --2- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba280cc390 0x7fba280cc810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 --2- 172.21.15.45:0/1251324491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba28158f60 0x7fba28159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 -- 172.21.15.45:0/1251324491 >> 172.21.15.45:0/1251324491 conn(0x7fba280cdf50 msgr2=0x7fba280ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 -- 172.21.15.45:0/1251324491 shutdown_connections 2023-10-06T05:15:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.595+0000 7fba2f73d700 1 -- 172.21.15.45:0/1251324491 wait complete. 2023-10-06T05:15:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.596+0000 7fba2f73d700 1 Processor -- start 2023-10-06T05:15:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.596+0000 7fba2f73d700 1 -- start start 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2f73d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2806a060 0x7fba2815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2f73d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba280cc390 0x7fba2815bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2f73d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba28158f60 0x7fba2815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2f73d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba280c5a40 con 0x7fba28158f60 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2f73d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2815dea0 con 0x7fba2806a060 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2f73d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2815d600 con 0x7fba280cc390 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba28158f60 0x7fba2815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba28158f60 0x7fba2815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53526/0 (socket says 172.21.15.45:53526) 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 -- 172.21.15.45:0/3466288673 learned_addr learned my addr 172.21.15.45:0/3466288673 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:24.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2ccd8700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba280cc390 0x7fba2815bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2d4d9700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2806a060 0x7fba2815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 -- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba280cc390 msgr2=0x7fba2815bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba280cc390 0x7fba2815bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 -- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2806a060 msgr2=0x7fba2815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2806a060 0x7fba2815cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 -- 172.21.15.45:0/3466288673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009420 con 0x7fba28158f60 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2dcda700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba28158f60 0x7fba2815c4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba280c4f80 tx=0x7fba20019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba1d7fa700 1 -- 172.21.15.45:0/3466288673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20012d60 con 0x7fba28158f60 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba1d7fa700 1 -- 172.21.15.45:0/3466288673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba20021070 con 0x7fba28158f60 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba20009a20 con 0x7fba28158f60 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.598+0000 7fba1d7fa700 1 -- 172.21.15.45:0/3466288673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20026680 con 0x7fba28158f60 2023-10-06T05:15:24.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.599+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2815dbc0 con 0x7fba28158f60 2023-10-06T05:15:24.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.600+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2806f410 con 0x7fba28158f60 2023-10-06T05:15:24.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.600+0000 7fba1d7fa700 1 -- 172.21.15.45:0/3466288673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba20023030 con 0x7fba28158f60 2023-10-06T05:15:24.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.600+0000 7fba1d7fa700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba0c066f30 0x7fba0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:24.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.600+0000 7fba1d7fa700 1 -- 172.21.15.45:0/3466288673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fba20086a90 con 0x7fba28158f60 2023-10-06T05:15:24.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.601+0000 7fba2d4d9700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba0c066f30 0x7fba0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:24.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.601+0000 7fba2d4d9700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba0c066f30 0x7fba0c0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba240093f0 tx=0x7fba24008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:24.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.605+0000 7fba1d7fa700 1 -- 172.21.15.45:0/3466288673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba200538b0 con 0x7fba28158f60 2023-10-06T05:15:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.803+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba28068fc0 con 0x7fba0c066f30 2023-10-06T05:15:24.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.815+0000 7fba1d7fa700 1 -- 172.21.15.45:0/3466288673 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367296 (secure 0 0 0) 0x7fba28068fc0 con 0x7fba0c066f30 2023-10-06T05:15:24.822 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:24.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.818+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba0c066f30 msgr2=0x7fba0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.818+0000 7fba2f73d700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba0c066f30 0x7fba0c0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba240093f0 tx=0x7fba24008040 comp rx=0 tx=0).stop 2023-10-06T05:15:24.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.818+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba28158f60 msgr2=0x7fba2815c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:24.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.818+0000 7fba2f73d700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba28158f60 0x7fba2815c4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba280c4f80 tx=0x7fba20019000 comp rx=0 tx=0).stop 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 shutdown_connections 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba0c066f30 0x7fba0c0693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba28158f60 0x7fba2815c4c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2806a060 0x7fba2815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 --2- 172.21.15.45:0/3466288673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba280cc390 0x7fba2815bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 >> 172.21.15.45:0/3466288673 conn(0x7fba280cdf50 msgr2=0x7fba280ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 shutdown_connections 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:24.819+0000 7fba2f73d700 1 -- 172.21.15.45:0/3466288673 wait complete. 2023-10-06T05:15:24.826 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:15:24.835 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-06T05:15:24.446145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003346"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.969110+0000","last_change":"2023-10-06T05:14:59.967201+0000","last_active":"2023-10-06T05:15:03.969110+0000","last_peered":"2023-10-06T05:15:03.969110+0000","last_clean":"2023-10-06T05:15:03.969110+0000","last_became_active":"2023-10-06T05:14:59.966818+0000","last_became_peered":"2023-10-06T05:14:59.966818+0000","last_unstale":"2023-10-06T05:15:03.969110+0000","last_undegraded":"2023-10-06T05:15:03.969110+0000","last_fullsized":"2023-10-06T05:15:03.969110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963221+0000","last_change":"2023-10-06T05:14:59.972677+0000","last_active":"2023-10-06T05:15:03.963221+0000","last_peered":"2023-10-06T05:15:03.963221+0000","last_clean":"2023-10-06T05:15:03.963221+0000","last_became_active":"2023-10-06T05:14:59.972435+0000","last_became_peered":"2023-10-06T05:14:59.972435+0000","last_unstale":"2023-10-06T05:15:03.963221+0000","last_undegraded":"2023-10-06T05:15:03.963221+0000","last_fullsized":"2023-10-06T05:15:03.963221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960554+0000","last_change":"2023-10-06T05:14:59.955617+0000","last_active":"2023-10-06T05:15:03.960554+0000","last_peered":"2023-10-06T05:15:03.960554+0000","last_clean":"2023-10-06T05:15:03.960554+0000","last_became_active":"2023-10-06T05:14:59.951585+0000","last_became_peered":"2023-10-06T05:14:59.951585+0000","last_unstale":"2023-10-06T05:15:03.960554+0000","last_undegraded":"2023-10-06T05:15:03.960554+0000","last_fullsized":"2023-10-06T05:15:03.960554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:56.835787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975703+0000","last_change":"2023-10-06T05:14:59.962591+0000","last_active":"2023-10-06T05:15:03.975703+0000","last_peered":"2023-10-06T05:15:03.975703+0000","last_clean":"2023-10-06T05:15:03.975703+0000","last_became_active":"2023-10-06T05:14:59.962219+0000","last_became_peered":"2023-10-06T05:14:59.962219+0000","last_unstale":"2023-10-06T05:15:03.975703+0000","last_undegraded":"2023-10-06T05:15:03.975703+0000","last_fullsized":"2023-10-06T05:15:03.975703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:11.373603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967261+0000","last_change":"2023-10-06T05:14:59.971703+0000","last_active":"2023-10-06T05:15:03.967261+0000","last_peered":"2023-10-06T05:15:03.967261+0000","last_clean":"2023-10-06T05:15:03.967261+0000","last_became_active":"2023-10-06T05:14:59.971083+0000","last_became_peered":"2023-10-06T05:14:59.971083+0000","last_unstale":"2023-10-06T05:15:03.967261+0000","last_undegraded":"2023-10-06T05:15:03.967261+0000","last_fullsized":"2023-10-06T05:15:03.967261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961235+0000","last_change":"2023-10-06T05:14:59.952376+0000","last_active":"2023-10-06T05:15:03.961235+0000","last_peered":"2023-10-06T05:15:03.961235+0000","last_clean":"2023-10-06T05:15:03.961235+0000","last_became_active":"2023-10-06T05:14:59.952101+0000","last_became_peered":"2023-10-06T05:14:59.952101+0000","last_unstale":"2023-10-06T05:15:03.961235+0000","last_undegraded":"2023-10-06T05:15:03.961235+0000","last_fullsized":"2023-10-06T05:15:03.961235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:46.640934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965050+0000","last_change":"2023-10-06T05:14:59.958503+0000","last_active":"2023-10-06T05:15:03.965050+0000","last_peered":"2023-10-06T05:15:03.965050+0000","last_clean":"2023-10-06T05:15:03.965050+0000","last_became_active":"2023-10-06T05:14:59.957962+0000","last_became_peered":"2023-10-06T05:14:59.957962+0000","last_unstale":"2023-10-06T05:15:03.965050+0000","last_undegraded":"2023-10-06T05:15:03.965050+0000","last_fullsized":"2023-10-06T05:15:03.965050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961531+0000","last_change":"2023-10-06T05:14:59.966704+0000","last_active":"2023-10-06T05:15:03.961531+0000","last_peered":"2023-10-06T05:15:03.961531+0000","last_clean":"2023-10-06T05:15:03.961531+0000","last_became_active":"2023-10-06T05:14:59.966378+0000","last_became_peered":"2023-10-06T05:14:59.966378+0000","last_unstale":"2023-10-06T05:15:03.961531+0000","last_undegraded":"2023-10-06T05:15:03.961531+0000","last_fullsized":"2023-10-06T05:15:03.961531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970103+0000","last_change":"2023-10-06T05:14:59.967328+0000","last_active":"2023-10-06T05:15:03.970103+0000","last_peered":"2023-10-06T05:15:03.970103+0000","last_clean":"2023-10-06T05:15:03.970103+0000","last_became_active":"2023-10-06T05:14:59.966868+0000","last_became_peered":"2023-10-06T05:14:59.966868+0000","last_unstale":"2023-10-06T05:15:03.970103+0000","last_undegraded":"2023-10-06T05:15:03.970103+0000","last_fullsized":"2023-10-06T05:15:03.970103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963384+0000","last_change":"2023-10-06T05:14:59.966488+0000","last_active":"2023-10-06T05:15:03.963384+0000","last_peered":"2023-10-06T05:15:03.963384+0000","last_clean":"2023-10-06T05:15:03.963384+0000","last_became_active":"2023-10-06T05:14:59.966153+0000","last_became_peered":"2023-10-06T05:14:59.966153+0000","last_unstale":"2023-10-06T05:15:03.963384+0000","last_undegraded":"2023-10-06T05:15:03.963384+0000","last_fullsized":"2023-10-06T05:15:03.963384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960401+0000","last_change":"2023-10-06T05:14:59.956939+0000","last_active":"2023-10-06T05:15:03.960401+0000","last_peered":"2023-10-06T05:15:03.960401+0000","last_clean":"2023-10-06T05:15:03.960401+0000","last_became_active":"2023-10-06T05:14:59.956467+0000","last_became_peered":"2023-10-06T05:14:59.956467+0000","last_unstale":"2023-10-06T05:15:03.960401+0000","last_undegraded":"2023-10-06T05:15:03.960401+0000","last_fullsized":"2023-10-06T05:15:03.960401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968295+0000","last_change":"2023-10-06T05:14:59.965955+0000","last_active":"2023-10-06T05:15:03.968295+0000","last_peered":"2023-10-06T05:15:03.968295+0000","last_clean":"2023-10-06T05:15:03.968295+0000","last_became_active":"2023-10-06T05:14:59.965676+0000","last_became_peered":"2023-10-06T05:14:59.965676+0000","last_unstale":"2023-10-06T05:15:03.968295+0000","last_undegraded":"2023-10-06T05:15:03.968295+0000","last_fullsized":"2023-10-06T05:15:03.968295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:39.098260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968290+0000","last_change":"2023-10-06T05:14:59.971318+0000","last_active":"2023-10-06T05:15:03.968290+0000","last_peered":"2023-10-06T05:15:03.968290+0000","last_clean":"2023-10-06T05:15:03.968290+0000","last_became_active":"2023-10-06T05:14:59.971044+0000","last_became_peered":"2023-10-06T05:14:59.971044+0000","last_unstale":"2023-10-06T05:15:03.968290+0000","last_undegraded":"2023-10-06T05:15:03.968290+0000","last_fullsized":"2023-10-06T05:15:03.968290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:17.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962264+0000","last_change":"2023-10-06T05:14:59.962752+0000","last_active":"2023-10-06T05:15:03.962264+0000","last_peered":"2023-10-06T05:15:03.962264+0000","last_clean":"2023-10-06T05:15:03.962264+0000","last_became_active":"2023-10-06T05:14:59.962473+0000","last_became_peered":"2023-10-06T05:14:59.962473+0000","last_unstale":"2023-10-06T05:15:03.962264+0000","last_undegraded":"2023-10-06T05:15:03.962264+0000","last_fullsized":"2023-10-06T05:15:03.962264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964859+0000","last_change":"2023-10-06T05:14:59.958686+0000","last_active":"2023-10-06T05:15:03.964859+0000","last_peered":"2023-10-06T05:15:03.964859+0000","last_clean":"2023-10-06T05:15:03.964859+0000","last_became_active":"2023-10-06T05:14:59.958258+0000","last_became_peered":"2023-10-06T05:14:59.958258+0000","last_unstale":"2023-10-06T05:15:03.964859+0000","last_undegraded":"2023-10-06T05:15:03.964859+0000","last_fullsized":"2023-10-06T05:15:03.964859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960536+0000","last_change":"2023-10-06T05:14:59.956754+0000","last_active":"2023-10-06T05:15:03.960536+0000","last_peered":"2023-10-06T05:15:03.960536+0000","last_clean":"2023-10-06T05:15:03.960536+0000","last_became_active":"2023-10-06T05:14:59.951445+0000","last_became_peered":"2023-10-06T05:14:59.951445+0000","last_unstale":"2023-10-06T05:15:03.960536+0000","last_undegraded":"2023-10-06T05:15:03.960536+0000","last_fullsized":"2023-10-06T05:15:03.960536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:43.881359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970204+0000","last_change":"2023-10-06T05:14:59.967155+0000","last_active":"2023-10-06T05:15:03.970204+0000","last_peered":"2023-10-06T05:15:03.970204+0000","last_clean":"2023-10-06T05:15:03.970204+0000","last_became_active":"2023-10-06T05:14:59.966533+0000","last_became_peered":"2023-10-06T05:14:59.966533+0000","last_unstale":"2023-10-06T05:15:03.970204+0000","last_undegraded":"2023-10-06T05:15:03.970204+0000","last_fullsized":"2023-10-06T05:15:03.970204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960993+0000","last_change":"2023-10-06T05:14:59.949784+0000","last_active":"2023-10-06T05:15:03.960993+0000","last_peered":"2023-10-06T05:15:03.960993+0000","last_clean":"2023-10-06T05:15:03.960993+0000","last_became_active":"2023-10-06T05:14:59.949540+0000","last_became_peered":"2023-10-06T05:14:59.949540+0000","last_unstale":"2023-10-06T05:15:03.960993+0000","last_undegraded":"2023-10-06T05:15:03.960993+0000","last_fullsized":"2023-10-06T05:15:03.960993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:55.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967686+0000","last_change":"2023-10-06T05:14:59.954874+0000","last_active":"2023-10-06T05:15:03.967686+0000","last_peered":"2023-10-06T05:15:03.967686+0000","last_clean":"2023-10-06T05:15:03.967686+0000","last_became_active":"2023-10-06T05:14:59.954596+0000","last_became_peered":"2023-10-06T05:14:59.954596+0000","last_unstale":"2023-10-06T05:15:03.967686+0000","last_undegraded":"2023-10-06T05:15:03.967686+0000","last_fullsized":"2023-10-06T05:15:03.967686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:12.617791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959901+0000","last_change":"2023-10-06T05:14:59.964700+0000","last_active":"2023-10-06T05:15:03.959901+0000","last_peered":"2023-10-06T05:15:03.959901+0000","last_clean":"2023-10-06T05:15:03.959901+0000","last_became_active":"2023-10-06T05:14:59.964122+0000","last_became_peered":"2023-10-06T05:14:59.964122+0000","last_unstale":"2023-10-06T05:15:03.959901+0000","last_undegraded":"2023-10-06T05:15:03.959901+0000","last_fullsized":"2023-10-06T05:15:03.959901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.174130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965374+0000","last_change":"2023-10-06T05:14:59.971088+0000","last_active":"2023-10-06T05:15:03.965374+0000","last_peered":"2023-10-06T05:15:03.965374+0000","last_clean":"2023-10-06T05:15:03.965374+0000","last_became_active":"2023-10-06T05:14:59.970833+0000","last_became_peered":"2023-10-06T05:14:59.970833+0000","last_unstale":"2023-10-06T05:15:03.965374+0000","last_undegraded":"2023-10-06T05:15:03.965374+0000","last_fullsized":"2023-10-06T05:15:03.965374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963287+0000","last_change":"2023-10-06T05:14:59.959301+0000","last_active":"2023-10-06T05:15:03.963287+0000","last_peered":"2023-10-06T05:15:03.963287+0000","last_clean":"2023-10-06T05:15:03.963287+0000","last_became_active":"2023-10-06T05:14:59.958848+0000","last_became_peered":"2023-10-06T05:14:59.958848+0000","last_unstale":"2023-10-06T05:15:03.963287+0000","last_undegraded":"2023-10-06T05:15:03.963287+0000","last_fullsized":"2023-10-06T05:15:03.963287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963428+0000","last_change":"2023-10-06T05:14:59.964849+0000","last_active":"2023-10-06T05:15:03.963428+0000","last_peered":"2023-10-06T05:15:03.963428+0000","last_clean":"2023-10-06T05:15:03.963428+0000","last_became_active":"2023-10-06T05:14:59.964525+0000","last_became_peered":"2023-10-06T05:14:59.964525+0000","last_unstale":"2023-10-06T05:15:03.963428+0000","last_undegraded":"2023-10-06T05:15:03.963428+0000","last_fullsized":"2023-10-06T05:15:03.963428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964605+0000","last_change":"2023-10-06T05:14:59.968524+0000","last_active":"2023-10-06T05:15:03.964605+0000","last_peered":"2023-10-06T05:15:03.964605+0000","last_clean":"2023-10-06T05:15:03.964605+0000","last_became_active":"2023-10-06T05:14:59.968292+0000","last_became_peered":"2023-10-06T05:14:59.968292+0000","last_unstale":"2023-10-06T05:15:03.964605+0000","last_undegraded":"2023-10-06T05:15:03.964605+0000","last_fullsized":"2023-10-06T05:15:03.964605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979056+0000","last_change":"2023-10-06T05:14:59.957743+0000","last_active":"2023-10-06T05:15:03.979056+0000","last_peered":"2023-10-06T05:15:03.979056+0000","last_clean":"2023-10-06T05:15:03.979056+0000","last_became_active":"2023-10-06T05:14:59.957472+0000","last_became_peered":"2023-10-06T05:14:59.957472+0000","last_unstale":"2023-10-06T05:15:03.979056+0000","last_undegraded":"2023-10-06T05:15:03.979056+0000","last_fullsized":"2023-10-06T05:15:03.979056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977658+0000","last_change":"2023-10-06T05:14:59.953807+0000","last_active":"2023-10-06T05:15:03.977658+0000","last_peered":"2023-10-06T05:15:03.977658+0000","last_clean":"2023-10-06T05:15:03.977658+0000","last_became_active":"2023-10-06T05:14:59.953532+0000","last_became_peered":"2023-10-06T05:14:59.953532+0000","last_unstale":"2023-10-06T05:15:03.977658+0000","last_undegraded":"2023-10-06T05:15:03.977658+0000","last_fullsized":"2023-10-06T05:15:03.977658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:39.399842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962861+0000","last_change":"2023-10-06T05:14:59.968101+0000","last_active":"2023-10-06T05:15:03.962861+0000","last_peered":"2023-10-06T05:15:03.962861+0000","last_clean":"2023-10-06T05:15:03.962861+0000","last_became_active":"2023-10-06T05:14:59.967317+0000","last_became_peered":"2023-10-06T05:14:59.967317+0000","last_unstale":"2023-10-06T05:15:03.962861+0000","last_undegraded":"2023-10-06T05:15:03.962861+0000","last_fullsized":"2023-10-06T05:15:03.962861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987821+0000","last_change":"2023-10-06T05:15:03.983811+0000","last_active":"2023-10-06T05:15:03.987821+0000","last_peered":"2023-10-06T05:15:03.987821+0000","last_clean":"2023-10-06T05:15:03.987821+0000","last_became_active":"2023-10-06T05:15:03.983468+0000","last_became_peered":"2023-10-06T05:15:03.983468+0000","last_unstale":"2023-10-06T05:15:03.987821+0000","last_undegraded":"2023-10-06T05:15:03.987821+0000","last_fullsized":"2023-10-06T05:15:03.987821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:02.315455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961143+0000","last_change":"2023-10-06T05:14:58.958395+0000","last_active":"2023-10-06T05:15:03.961143+0000","last_peered":"2023-10-06T05:15:03.961143+0000","last_clean":"2023-10-06T05:15:03.961143+0000","last_became_active":"2023-10-06T05:14:58.957979+0000","last_became_peered":"2023-10-06T05:14:58.957979+0000","last_unstale":"2023-10-06T05:15:03.961143+0000","last_undegraded":"2023-10-06T05:15:03.961143+0000","last_fullsized":"2023-10-06T05:15:03.961143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965650+0000","last_change":"2023-10-06T05:14:59.960654+0000","last_active":"2023-10-06T05:15:03.965650+0000","last_peered":"2023-10-06T05:15:03.965650+0000","last_clean":"2023-10-06T05:15:03.965650+0000","last_became_active":"2023-10-06T05:14:59.960359+0000","last_became_peered":"2023-10-06T05:14:59.960359+0000","last_unstale":"2023-10-06T05:15:03.965650+0000","last_undegraded":"2023-10-06T05:15:03.965650+0000","last_fullsized":"2023-10-06T05:15:03.965650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987166+0000","last_change":"2023-10-06T05:15:03.980797+0000","last_active":"2023-10-06T05:15:03.987166+0000","last_peered":"2023-10-06T05:15:03.987166+0000","last_clean":"2023-10-06T05:15:03.987166+0000","last_became_active":"2023-10-06T05:15:03.980441+0000","last_became_peered":"2023-10-06T05:15:03.980441+0000","last_unstale":"2023-10-06T05:15:03.987166+0000","last_undegraded":"2023-10-06T05:15:03.987166+0000","last_fullsized":"2023-10-06T05:15:03.987166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:59.153783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972385+0000","last_change":"2023-10-06T05:14:58.967470+0000","last_active":"2023-10-06T05:15:03.972385+0000","last_peered":"2023-10-06T05:15:03.972385+0000","last_clean":"2023-10-06T05:15:03.972385+0000","last_became_active":"2023-10-06T05:14:58.967143+0000","last_became_peered":"2023-10-06T05:14:58.967143+0000","last_unstale":"2023-10-06T05:15:03.972385+0000","last_undegraded":"2023-10-06T05:15:03.972385+0000","last_fullsized":"2023-10-06T05:15:03.972385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970567+0000","last_change":"2023-10-06T05:14:59.961217+0000","last_active":"2023-10-06T05:15:03.970567+0000","last_peered":"2023-10-06T05:15:03.970567+0000","last_clean":"2023-10-06T05:15:03.970567+0000","last_became_active":"2023-10-06T05:14:59.960909+0000","last_became_peered":"2023-10-06T05:14:59.960909+0000","last_unstale":"2023-10-06T05:15:03.970567+0000","last_undegraded":"2023-10-06T05:15:03.970567+0000","last_fullsized":"2023-10-06T05:15:03.970567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995384+0000","last_change":"2023-10-06T05:15:03.993702+0000","last_active":"2023-10-06T05:15:03.995384+0000","last_peered":"2023-10-06T05:15:03.995384+0000","last_clean":"2023-10-06T05:15:03.995384+0000","last_became_active":"2023-10-06T05:15:03.993170+0000","last_became_peered":"2023-10-06T05:15:03.993170+0000","last_unstale":"2023-10-06T05:15:03.995384+0000","last_undegraded":"2023-10-06T05:15:03.995384+0000","last_fullsized":"2023-10-06T05:15:03.995384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964634+0000","last_change":"2023-10-06T05:14:58.964287+0000","last_active":"2023-10-06T05:15:03.964634+0000","last_peered":"2023-10-06T05:15:03.964634+0000","last_clean":"2023-10-06T05:15:03.964634+0000","last_became_active":"2023-10-06T05:14:58.963981+0000","last_became_peered":"2023-10-06T05:14:58.963981+0000","last_unstale":"2023-10-06T05:15:03.964634+0000","last_undegraded":"2023-10-06T05:15:03.964634+0000","last_fullsized":"2023-10-06T05:15:03.964634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975859+0000","last_change":"2023-10-06T05:14:59.964080+0000","last_active":"2023-10-06T05:15:03.975859+0000","last_peered":"2023-10-06T05:15:03.975859+0000","last_clean":"2023-10-06T05:15:03.975859+0000","last_became_active":"2023-10-06T05:14:59.963849+0000","last_became_peered":"2023-10-06T05:14:59.963849+0000","last_unstale":"2023-10-06T05:15:03.975859+0000","last_undegraded":"2023-10-06T05:15:03.975859+0000","last_fullsized":"2023-10-06T05:15:03.975859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:13.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.988935+0000","last_change":"2023-10-06T05:15:03.985094+0000","last_active":"2023-10-06T05:15:03.988935+0000","last_peered":"2023-10-06T05:15:03.988935+0000","last_clean":"2023-10-06T05:15:03.988935+0000","last_became_active":"2023-10-06T05:15:03.984516+0000","last_became_peered":"2023-10-06T05:15:03.984516+0000","last_unstale":"2023-10-06T05:15:03.988935+0000","last_undegraded":"2023-10-06T05:15:03.988935+0000","last_fullsized":"2023-10-06T05:15:03.988935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:09.008217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970313+0000","last_change":"2023-10-06T05:14:58.970639+0000","last_active":"2023-10-06T05:15:03.970313+0000","last_peered":"2023-10-06T05:15:03.970313+0000","last_clean":"2023-10-06T05:15:03.970313+0000","last_became_active":"2023-10-06T05:14:58.970183+0000","last_became_peered":"2023-10-06T05:14:58.970183+0000","last_unstale":"2023-10-06T05:15:03.970313+0000","last_undegraded":"2023-10-06T05:15:03.970313+0000","last_fullsized":"2023-10-06T05:15:03.970313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964542+0000","last_change":"2023-10-06T05:14:59.962878+0000","last_active":"2023-10-06T05:15:03.964542+0000","last_peered":"2023-10-06T05:15:03.964542+0000","last_clean":"2023-10-06T05:15:03.964542+0000","last_became_active":"2023-10-06T05:14:59.962557+0000","last_became_peered":"2023-10-06T05:14:59.962557+0000","last_unstale":"2023-10-06T05:15:03.964542+0000","last_undegraded":"2023-10-06T05:15:03.964542+0000","last_fullsized":"2023-10-06T05:15:03.964542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.984338+0000","last_change":"2023-10-06T05:15:03.979469+0000","last_active":"2023-10-06T05:15:03.984338+0000","last_peered":"2023-10-06T05:15:03.984338+0000","last_clean":"2023-10-06T05:15:03.984338+0000","last_became_active":"2023-10-06T05:15:03.979016+0000","last_became_peered":"2023-10-06T05:15:03.979016+0000","last_unstale":"2023-10-06T05:15:03.984338+0000","last_undegraded":"2023-10-06T05:15:03.984338+0000","last_fullsized":"2023-10-06T05:15:03.984338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963303+0000","last_change":"2023-10-06T05:14:58.953556+0000","last_active":"2023-10-06T05:15:03.963303+0000","last_peered":"2023-10-06T05:15:03.963303+0000","last_clean":"2023-10-06T05:15:03.963303+0000","last_became_active":"2023-10-06T05:14:58.953280+0000","last_became_peered":"2023-10-06T05:14:58.953280+0000","last_unstale":"2023-10-06T05:15:03.963303+0000","last_undegraded":"2023-10-06T05:15:03.963303+0000","last_fullsized":"2023-10-06T05:15:03.963303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963541+0000","last_change":"2023-10-06T05:14:59.969909+0000","last_active":"2023-10-06T05:15:03.963541+0000","last_peered":"2023-10-06T05:15:03.963541+0000","last_clean":"2023-10-06T05:15:03.963541+0000","last_became_active":"2023-10-06T05:14:59.967612+0000","last_became_peered":"2023-10-06T05:14:59.967612+0000","last_unstale":"2023-10-06T05:15:03.963541+0000","last_undegraded":"2023-10-06T05:15:03.963541+0000","last_fullsized":"2023-10-06T05:15:03.963541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994156+0000","last_change":"2023-10-06T05:15:03.984187+0000","last_active":"2023-10-06T05:15:03.994156+0000","last_peered":"2023-10-06T05:15:03.994156+0000","last_clean":"2023-10-06T05:15:03.994156+0000","last_became_active":"2023-10-06T05:15:03.983778+0000","last_became_peered":"2023-10-06T05:15:03.983778+0000","last_unstale":"2023-10-06T05:15:03.994156+0000","last_undegraded":"2023-10-06T05:15:03.994156+0000","last_fullsized":"2023-10-06T05:15:03.994156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:24.403669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967440+0000","last_change":"2023-10-06T05:14:58.971420+0000","last_active":"2023-10-06T05:15:03.967440+0000","last_peered":"2023-10-06T05:15:03.967440+0000","last_clean":"2023-10-06T05:15:03.967440+0000","last_became_active":"2023-10-06T05:14:58.971181+0000","last_became_peered":"2023-10-06T05:14:58.971181+0000","last_unstale":"2023-10-06T05:15:03.967440+0000","last_undegraded":"2023-10-06T05:15:03.967440+0000","last_fullsized":"2023-10-06T05:15:03.967440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973790+0000","last_change":"2023-10-06T05:14:59.966072+0000","last_active":"2023-10-06T05:15:03.973790+0000","last_peered":"2023-10-06T05:15:03.973790+0000","last_clean":"2023-10-06T05:15:03.973790+0000","last_became_active":"2023-10-06T05:14:59.965368+0000","last_became_peered":"2023-10-06T05:14:59.965368+0000","last_unstale":"2023-10-06T05:15:03.973790+0000","last_undegraded":"2023-10-06T05:15:03.973790+0000","last_fullsized":"2023-10-06T05:15:03.973790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990856+0000","last_change":"2023-10-06T05:15:03.989524+0000","last_active":"2023-10-06T05:15:03.990856+0000","last_peered":"2023-10-06T05:15:03.990856+0000","last_clean":"2023-10-06T05:15:03.990856+0000","last_became_active":"2023-10-06T05:15:03.982518+0000","last_became_peered":"2023-10-06T05:15:03.982518+0000","last_unstale":"2023-10-06T05:15:03.990856+0000","last_undegraded":"2023-10-06T05:15:03.990856+0000","last_fullsized":"2023-10-06T05:15:03.990856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T05:15:10.388156+0000","last_change":"2023-10-06T05:14:58.956195+0000","last_active":"2023-10-06T05:15:10.388156+0000","last_peered":"2023-10-06T05:15:10.388156+0000","last_clean":"2023-10-06T05:15:10.388156+0000","last_became_active":"2023-10-06T05:14:58.955906+0000","last_became_peered":"2023-10-06T05:14:58.955906+0000","last_unstale":"2023-10-06T05:15:10.388156+0000","last_undegraded":"2023-10-06T05:15:10.388156+0000","last_fullsized":"2023-10-06T05:15:10.388156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.952535+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.952313+0000","last_became_peered":"2023-10-06T05:14:59.952313+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987354+0000","last_change":"2023-10-06T05:15:03.982183+0000","last_active":"2023-10-06T05:15:03.987354+0000","last_peered":"2023-10-06T05:15:03.987354+0000","last_clean":"2023-10-06T05:15:03.987354+0000","last_became_active":"2023-10-06T05:15:03.981849+0000","last_became_peered":"2023-10-06T05:15:03.981849+0000","last_unstale":"2023-10-06T05:15:03.987354+0000","last_undegraded":"2023-10-06T05:15:03.987354+0000","last_fullsized":"2023-10-06T05:15:03.987354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965015+0000","last_change":"2023-10-06T05:14:58.959548+0000","last_active":"2023-10-06T05:15:03.965015+0000","last_peered":"2023-10-06T05:15:03.965015+0000","last_clean":"2023-10-06T05:15:03.965015+0000","last_became_active":"2023-10-06T05:14:58.959270+0000","last_became_peered":"2023-10-06T05:14:58.959270+0000","last_unstale":"2023-10-06T05:15:03.965015+0000","last_undegraded":"2023-10-06T05:15:03.965015+0000","last_fullsized":"2023-10-06T05:15:03.965015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:02.018618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968411+0000","last_change":"2023-10-06T05:14:59.973960+0000","last_active":"2023-10-06T05:15:03.968411+0000","last_peered":"2023-10-06T05:15:03.968411+0000","last_clean":"2023-10-06T05:15:03.968411+0000","last_became_active":"2023-10-06T05:14:59.973622+0000","last_became_peered":"2023-10-06T05:14:59.973622+0000","last_unstale":"2023-10-06T05:15:03.968411+0000","last_undegraded":"2023-10-06T05:15:03.968411+0000","last_fullsized":"2023-10-06T05:15:03.968411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994166+0000","last_change":"2023-10-06T05:15:03.985900+0000","last_active":"2023-10-06T05:15:03.994166+0000","last_peered":"2023-10-06T05:15:03.994166+0000","last_clean":"2023-10-06T05:15:03.994166+0000","last_became_active":"2023-10-06T05:15:03.985484+0000","last_became_peered":"2023-10-06T05:15:03.985484+0000","last_unstale":"2023-10-06T05:15:03.994166+0000","last_undegraded":"2023-10-06T05:15:03.994166+0000","last_fullsized":"2023-10-06T05:15:03.994166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:50.069708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963167+0000","last_change":"2023-10-06T05:14:58.968185+0000","last_active":"2023-10-06T05:15:03.963167+0000","last_peered":"2023-10-06T05:15:03.963167+0000","last_clean":"2023-10-06T05:15:03.963167+0000","last_became_active":"2023-10-06T05:14:58.967895+0000","last_became_peered":"2023-10-06T05:14:58.967895+0000","last_unstale":"2023-10-06T05:15:03.963167+0000","last_undegraded":"2023-10-06T05:15:03.963167+0000","last_fullsized":"2023-10-06T05:15:03.963167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962815+0000","last_change":"2023-10-06T05:14:59.967384+0000","last_active":"2023-10-06T05:15:03.962815+0000","last_peered":"2023-10-06T05:15:03.962815+0000","last_clean":"2023-10-06T05:15:03.962815+0000","last_became_active":"2023-10-06T05:14:59.967131+0000","last_became_peered":"2023-10-06T05:14:59.967131+0000","last_unstale":"2023-10-06T05:15:03.962815+0000","last_undegraded":"2023-10-06T05:15:03.962815+0000","last_fullsized":"2023-10-06T05:15:03.962815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994086+0000","last_change":"2023-10-06T05:15:03.985192+0000","last_active":"2023-10-06T05:15:03.994086+0000","last_peered":"2023-10-06T05:15:03.994086+0000","last_clean":"2023-10-06T05:15:03.994086+0000","last_became_active":"2023-10-06T05:15:03.984600+0000","last_became_peered":"2023-10-06T05:15:03.984600+0000","last_unstale":"2023-10-06T05:15:03.994086+0000","last_undegraded":"2023-10-06T05:15:03.994086+0000","last_fullsized":"2023-10-06T05:15:03.994086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964960+0000","last_change":"2023-10-06T05:14:58.959794+0000","last_active":"2023-10-06T05:15:03.964960+0000","last_peered":"2023-10-06T05:15:03.964960+0000","last_clean":"2023-10-06T05:15:03.964960+0000","last_became_active":"2023-10-06T05:14:58.959493+0000","last_became_peered":"2023-10-06T05:14:58.959493+0000","last_unstale":"2023-10-06T05:15:03.964960+0000","last_undegraded":"2023-10-06T05:15:03.964960+0000","last_fullsized":"2023-10-06T05:15:03.964960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:10.366367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970679+0000","last_change":"2023-10-06T05:14:59.964236+0000","last_active":"2023-10-06T05:15:03.970679+0000","last_peered":"2023-10-06T05:15:03.970679+0000","last_clean":"2023-10-06T05:15:03.970679+0000","last_became_active":"2023-10-06T05:14:59.963902+0000","last_became_peered":"2023-10-06T05:14:59.963902+0000","last_unstale":"2023-10-06T05:15:03.970679+0000","last_undegraded":"2023-10-06T05:15:03.970679+0000","last_fullsized":"2023-10-06T05:15:03.970679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987391+0000","last_change":"2023-10-06T05:15:03.982595+0000","last_active":"2023-10-06T05:15:03.987391+0000","last_peered":"2023-10-06T05:15:03.987391+0000","last_clean":"2023-10-06T05:15:03.987391+0000","last_became_active":"2023-10-06T05:15:03.982200+0000","last_became_peered":"2023-10-06T05:15:03.982200+0000","last_unstale":"2023-10-06T05:15:03.987391+0000","last_undegraded":"2023-10-06T05:15:03.987391+0000","last_fullsized":"2023-10-06T05:15:03.987391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:20.022766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974038+0000","last_change":"2023-10-06T05:14:58.961380+0000","last_active":"2023-10-06T05:15:03.974038+0000","last_peered":"2023-10-06T05:15:03.974038+0000","last_clean":"2023-10-06T05:15:03.974038+0000","last_became_active":"2023-10-06T05:14:58.961038+0000","last_became_peered":"2023-10-06T05:14:58.961038+0000","last_unstale":"2023-10-06T05:15:03.974038+0000","last_undegraded":"2023-10-06T05:15:03.974038+0000","last_fullsized":"2023-10-06T05:15:03.974038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964625+0000","last_change":"2023-10-06T05:14:59.967979+0000","last_active":"2023-10-06T05:15:03.964625+0000","last_peered":"2023-10-06T05:15:03.964625+0000","last_clean":"2023-10-06T05:15:03.964625+0000","last_became_active":"2023-10-06T05:14:59.964930+0000","last_became_peered":"2023-10-06T05:14:59.964930+0000","last_unstale":"2023-10-06T05:15:03.964625+0000","last_undegraded":"2023-10-06T05:15:03.964625+0000","last_fullsized":"2023-10-06T05:15:03.964625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.981489+0000","last_change":"2023-10-06T05:15:03.978144+0000","last_active":"2023-10-06T05:15:03.981489+0000","last_peered":"2023-10-06T05:15:03.981489+0000","last_clean":"2023-10-06T05:15:03.981489+0000","last_became_active":"2023-10-06T05:15:03.977784+0000","last_became_peered":"2023-10-06T05:15:03.977784+0000","last_unstale":"2023-10-06T05:15:03.981489+0000","last_undegraded":"2023-10-06T05:15:03.981489+0000","last_fullsized":"2023-10-06T05:15:03.981489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967621+0000","last_change":"2023-10-06T05:14:58.959617+0000","last_active":"2023-10-06T05:15:03.967621+0000","last_peered":"2023-10-06T05:15:03.967621+0000","last_clean":"2023-10-06T05:15:03.967621+0000","last_became_active":"2023-10-06T05:14:58.959350+0000","last_became_peered":"2023-10-06T05:14:58.959350+0000","last_unstale":"2023-10-06T05:15:03.967621+0000","last_undegraded":"2023-10-06T05:15:03.967621+0000","last_fullsized":"2023-10-06T05:15:03.967621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:12.695218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961420+0000","last_change":"2023-10-06T05:14:59.967832+0000","last_active":"2023-10-06T05:15:03.961420+0000","last_peered":"2023-10-06T05:15:03.961420+0000","last_clean":"2023-10-06T05:15:03.961420+0000","last_became_active":"2023-10-06T05:14:59.967192+0000","last_became_peered":"2023-10-06T05:14:59.967192+0000","last_unstale":"2023-10-06T05:15:03.961420+0000","last_undegraded":"2023-10-06T05:15:03.961420+0000","last_fullsized":"2023-10-06T05:15:03.961420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995012+0000","last_change":"2023-10-06T05:15:03.993765+0000","last_active":"2023-10-06T05:15:03.995012+0000","last_peered":"2023-10-06T05:15:03.995012+0000","last_clean":"2023-10-06T05:15:03.995012+0000","last_became_active":"2023-10-06T05:15:03.993371+0000","last_became_peered":"2023-10-06T05:15:03.993371+0000","last_unstale":"2023-10-06T05:15:03.995012+0000","last_undegraded":"2023-10-06T05:15:03.995012+0000","last_fullsized":"2023-10-06T05:15:03.995012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:00.861115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967256+0000","last_change":"2023-10-06T05:14:58.962483+0000","last_active":"2023-10-06T05:15:03.967256+0000","last_peered":"2023-10-06T05:15:03.967256+0000","last_clean":"2023-10-06T05:15:03.967256+0000","last_became_active":"2023-10-06T05:14:58.962203+0000","last_became_peered":"2023-10-06T05:14:58.962203+0000","last_unstale":"2023-10-06T05:15:03.967256+0000","last_undegraded":"2023-10-06T05:15:03.967256+0000","last_fullsized":"2023-10-06T05:15:03.967256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:34.275033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960934+0000","last_change":"2023-10-06T05:14:59.956925+0000","last_active":"2023-10-06T05:15:03.960934+0000","last_peered":"2023-10-06T05:15:03.960934+0000","last_clean":"2023-10-06T05:15:03.960934+0000","last_became_active":"2023-10-06T05:14:59.956653+0000","last_became_peered":"2023-10-06T05:14:59.956653+0000","last_unstale":"2023-10-06T05:15:03.960934+0000","last_undegraded":"2023-10-06T05:15:03.960934+0000","last_fullsized":"2023-10-06T05:15:03.960934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985561+0000","last_change":"2023-10-06T05:15:03.983559+0000","last_active":"2023-10-06T05:15:03.985561+0000","last_peered":"2023-10-06T05:15:03.985561+0000","last_clean":"2023-10-06T05:15:03.985561+0000","last_became_active":"2023-10-06T05:15:03.983123+0000","last_became_peered":"2023-10-06T05:15:03.983123+0000","last_unstale":"2023-10-06T05:15:03.985561+0000","last_undegraded":"2023-10-06T05:15:03.985561+0000","last_fullsized":"2023-10-06T05:15:03.985561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:50.666262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967619+0000","last_change":"2023-10-06T05:14:58.956636+0000","last_active":"2023-10-06T05:15:03.967619+0000","last_peered":"2023-10-06T05:15:03.967619+0000","last_clean":"2023-10-06T05:15:03.967619+0000","last_became_active":"2023-10-06T05:14:58.956436+0000","last_became_peered":"2023-10-06T05:14:58.956436+0000","last_unstale":"2023-10-06T05:15:03.967619+0000","last_undegraded":"2023-10-06T05:15:03.967619+0000","last_fullsized":"2023-10-06T05:15:03.967619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:45.319345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975492+0000","last_change":"2023-10-06T05:14:59.962010+0000","last_active":"2023-10-06T05:15:03.975492+0000","last_peered":"2023-10-06T05:15:03.975492+0000","last_clean":"2023-10-06T05:15:03.975492+0000","last_became_active":"2023-10-06T05:14:59.961837+0000","last_became_peered":"2023-10-06T05:14:59.961837+0000","last_unstale":"2023-10-06T05:15:03.975492+0000","last_undegraded":"2023-10-06T05:15:03.975492+0000","last_fullsized":"2023-10-06T05:15:03.975492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:46.326581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985590+0000","last_change":"2023-10-06T05:15:03.983998+0000","last_active":"2023-10-06T05:15:03.985590+0000","last_peered":"2023-10-06T05:15:03.985590+0000","last_clean":"2023-10-06T05:15:03.985590+0000","last_became_active":"2023-10-06T05:15:03.983598+0000","last_became_peered":"2023-10-06T05:15:03.983598+0000","last_unstale":"2023-10-06T05:15:03.985590+0000","last_undegraded":"2023-10-06T05:15:03.985590+0000","last_fullsized":"2023-10-06T05:15:03.985590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:01.199458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961576+0000","last_change":"2023-10-06T05:14:58.969438+0000","last_active":"2023-10-06T05:15:03.961576+0000","last_peered":"2023-10-06T05:15:03.961576+0000","last_clean":"2023-10-06T05:15:03.961576+0000","last_became_active":"2023-10-06T05:14:58.969214+0000","last_became_peered":"2023-10-06T05:14:58.969214+0000","last_unstale":"2023-10-06T05:15:03.961576+0000","last_undegraded":"2023-10-06T05:15:03.961576+0000","last_fullsized":"2023-10-06T05:15:03.961576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966944+0000","last_change":"2023-10-06T05:14:59.958303+0000","last_active":"2023-10-06T05:15:03.966944+0000","last_peered":"2023-10-06T05:15:03.966944+0000","last_clean":"2023-10-06T05:15:03.966944+0000","last_became_active":"2023-10-06T05:14:59.957997+0000","last_became_peered":"2023-10-06T05:14:59.957997+0000","last_unstale":"2023-10-06T05:15:03.966944+0000","last_undegraded":"2023-10-06T05:15:03.966944+0000","last_fullsized":"2023-10-06T05:15:03.966944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.982560+0000","last_change":"2023-10-06T05:15:03.976634+0000","last_active":"2023-10-06T05:15:03.982560+0000","last_peered":"2023-10-06T05:15:03.982560+0000","last_clean":"2023-10-06T05:15:03.982560+0000","last_became_active":"2023-10-06T05:15:03.976257+0000","last_became_peered":"2023-10-06T05:15:03.976257+0000","last_unstale":"2023-10-06T05:15:03.982560+0000","last_undegraded":"2023-10-06T05:15:03.982560+0000","last_fullsized":"2023-10-06T05:15:03.982560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:47.906687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964042+0000","last_change":"2023-10-06T05:14:58.969408+0000","last_active":"2023-10-06T05:15:03.964042+0000","last_peered":"2023-10-06T05:15:03.964042+0000","last_clean":"2023-10-06T05:15:03.964042+0000","last_became_active":"2023-10-06T05:14:58.969070+0000","last_became_peered":"2023-10-06T05:14:58.969070+0000","last_unstale":"2023-10-06T05:15:03.964042+0000","last_undegraded":"2023-10-06T05:15:03.964042+0000","last_fullsized":"2023-10-06T05:15:03.964042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959678+0000","last_change":"2023-10-06T05:14:59.969503+0000","last_active":"2023-10-06T05:15:03.959678+0000","last_peered":"2023-10-06T05:15:03.959678+0000","last_clean":"2023-10-06T05:15:03.959678+0000","last_became_active":"2023-10-06T05:14:59.969109+0000","last_became_peered":"2023-10-06T05:14:59.969109+0000","last_unstale":"2023-10-06T05:15:03.959678+0000","last_undegraded":"2023-10-06T05:15:03.959678+0000","last_fullsized":"2023-10-06T05:15:03.959678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:59.890813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.989045+0000","last_change":"2023-10-06T05:15:03.982573+0000","last_active":"2023-10-06T05:15:03.989045+0000","last_peered":"2023-10-06T05:15:03.989045+0000","last_clean":"2023-10-06T05:15:03.989045+0000","last_became_active":"2023-10-06T05:15:03.982173+0000","last_became_peered":"2023-10-06T05:15:03.982173+0000","last_unstale":"2023-10-06T05:15:03.989045+0000","last_undegraded":"2023-10-06T05:15:03.989045+0000","last_fullsized":"2023-10-06T05:15:03.989045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.921378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975586+0000","last_change":"2023-10-06T05:14:58.962379+0000","last_active":"2023-10-06T05:15:03.975586+0000","last_peered":"2023-10-06T05:15:03.975586+0000","last_clean":"2023-10-06T05:15:03.975586+0000","last_became_active":"2023-10-06T05:14:58.962093+0000","last_became_peered":"2023-10-06T05:14:58.962093+0000","last_unstale":"2023-10-06T05:15:03.975586+0000","last_undegraded":"2023-10-06T05:15:03.975586+0000","last_fullsized":"2023-10-06T05:15:03.975586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960163+0000","last_change":"2023-10-06T05:14:59.951502+0000","last_active":"2023-10-06T05:15:03.960163+0000","last_peered":"2023-10-06T05:15:03.960163+0000","last_clean":"2023-10-06T05:15:03.960163+0000","last_became_active":"2023-10-06T05:14:59.951197+0000","last_became_peered":"2023-10-06T05:14:59.951197+0000","last_unstale":"2023-10-06T05:15:03.960163+0000","last_undegraded":"2023-10-06T05:15:03.960163+0000","last_fullsized":"2023-10-06T05:15:03.960163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978919+0000","last_change":"2023-10-06T05:15:03.977325+0000","last_active":"2023-10-06T05:15:03.978919+0000","last_peered":"2023-10-06T05:15:03.978919+0000","last_clean":"2023-10-06T05:15:03.978919+0000","last_became_active":"2023-10-06T05:15:03.976844+0000","last_became_peered":"2023-10-06T05:15:03.976844+0000","last_unstale":"2023-10-06T05:15:03.978919+0000","last_undegraded":"2023-10-06T05:15:03.978919+0000","last_fullsized":"2023-10-06T05:15:03.978919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:47.153239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960726+0000","last_change":"2023-10-06T05:14:58.958227+0000","last_active":"2023-10-06T05:15:03.960726+0000","last_peered":"2023-10-06T05:15:03.960726+0000","last_clean":"2023-10-06T05:15:03.960726+0000","last_became_active":"2023-10-06T05:14:58.957924+0000","last_became_peered":"2023-10-06T05:14:58.957924+0000","last_unstale":"2023-10-06T05:15:03.960726+0000","last_undegraded":"2023-10-06T05:15:03.960726+0000","last_fullsized":"2023-10-06T05:15:03.960726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961096+0000","last_change":"2023-10-06T05:14:59.961306+0000","last_active":"2023-10-06T05:15:03.961096+0000","last_peered":"2023-10-06T05:15:03.961096+0000","last_clean":"2023-10-06T05:15:03.961096+0000","last_became_active":"2023-10-06T05:14:59.961025+0000","last_became_peered":"2023-10-06T05:14:59.961025+0000","last_unstale":"2023-10-06T05:15:03.961096+0000","last_undegraded":"2023-10-06T05:15:03.961096+0000","last_fullsized":"2023-10-06T05:15:03.961096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987866+0000","last_change":"2023-10-06T05:15:03.983777+0000","last_active":"2023-10-06T05:15:03.987866+0000","last_peered":"2023-10-06T05:15:03.987866+0000","last_clean":"2023-10-06T05:15:03.987866+0000","last_became_active":"2023-10-06T05:15:03.983380+0000","last_became_peered":"2023-10-06T05:15:03.983380+0000","last_unstale":"2023-10-06T05:15:03.987866+0000","last_undegraded":"2023-10-06T05:15:03.987866+0000","last_fullsized":"2023-10-06T05:15:03.987866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:21.551659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967787+0000","last_change":"2023-10-06T05:14:58.966032+0000","last_active":"2023-10-06T05:15:03.967787+0000","last_peered":"2023-10-06T05:15:03.967787+0000","last_clean":"2023-10-06T05:15:03.967787+0000","last_became_active":"2023-10-06T05:14:58.965697+0000","last_became_peered":"2023-10-06T05:14:58.965697+0000","last_unstale":"2023-10-06T05:15:03.967787+0000","last_undegraded":"2023-10-06T05:15:03.967787+0000","last_fullsized":"2023-10-06T05:15:03.967787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979242+0000","last_change":"2023-10-06T05:14:59.966425+0000","last_active":"2023-10-06T05:15:03.979242+0000","last_peered":"2023-10-06T05:15:03.979242+0000","last_clean":"2023-10-06T05:15:03.979242+0000","last_became_active":"2023-10-06T05:14:59.966122+0000","last_became_peered":"2023-10-06T05:14:59.966122+0000","last_unstale":"2023-10-06T05:15:03.979242+0000","last_undegraded":"2023-10-06T05:15:03.979242+0000","last_fullsized":"2023-10-06T05:15:03.979242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.983132+0000","last_change":"2023-10-06T05:15:03.979017+0000","last_active":"2023-10-06T05:15:03.983132+0000","last_peered":"2023-10-06T05:15:03.983132+0000","last_clean":"2023-10-06T05:15:03.983132+0000","last_became_active":"2023-10-06T05:15:03.978644+0000","last_became_peered":"2023-10-06T05:15:03.978644+0000","last_unstale":"2023-10-06T05:15:03.983132+0000","last_undegraded":"2023-10-06T05:15:03.983132+0000","last_fullsized":"2023-10-06T05:15:03.983132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973620+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:15:03.973620+0000","last_peered":"2023-10-06T05:15:03.973620+0000","last_clean":"2023-10-06T05:15:03.973620+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:15:03.973620+0000","last_undegraded":"2023-10-06T05:15:03.973620+0000","last_fullsized":"2023-10-06T05:15:03.973620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962773+0000","last_change":"2023-10-06T05:14:58.969546+0000","last_active":"2023-10-06T05:15:03.962773+0000","last_peered":"2023-10-06T05:15:03.962773+0000","last_clean":"2023-10-06T05:15:03.962773+0000","last_became_active":"2023-10-06T05:14:58.969281+0000","last_became_peered":"2023-10-06T05:14:58.969281+0000","last_unstale":"2023-10-06T05:15:03.962773+0000","last_undegraded":"2023-10-06T05:15:03.962773+0000","last_fullsized":"2023-10-06T05:15:03.962773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973704+0000","last_change":"2023-10-06T05:14:59.965859+0000","last_active":"2023-10-06T05:15:03.973704+0000","last_peered":"2023-10-06T05:15:03.973704+0000","last_clean":"2023-10-06T05:15:03.973704+0000","last_became_active":"2023-10-06T05:14:59.964784+0000","last_became_peered":"2023-10-06T05:14:59.964784+0000","last_unstale":"2023-10-06T05:15:03.973704+0000","last_undegraded":"2023-10-06T05:15:03.973704+0000","last_fullsized":"2023-10-06T05:15:03.973704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985105+0000","last_change":"2023-10-06T05:15:03.982375+0000","last_active":"2023-10-06T05:15:03.985105+0000","last_peered":"2023-10-06T05:15:03.985105+0000","last_clean":"2023-10-06T05:15:03.985105+0000","last_became_active":"2023-10-06T05:15:03.981983+0000","last_became_peered":"2023-10-06T05:15:03.981983+0000","last_unstale":"2023-10-06T05:15:03.985105+0000","last_undegraded":"2023-10-06T05:15:03.985105+0000","last_fullsized":"2023-10-06T05:15:03.985105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962357+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:15:03.962357+0000","last_peered":"2023-10-06T05:15:03.962357+0000","last_clean":"2023-10-06T05:15:03.962357+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:15:03.962357+0000","last_undegraded":"2023-10-06T05:15:03.962357+0000","last_fullsized":"2023-10-06T05:15:03.962357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959855+0000","last_change":"2023-10-06T05:14:58.966374+0000","last_active":"2023-10-06T05:15:03.959855+0000","last_peered":"2023-10-06T05:15:03.959855+0000","last_clean":"2023-10-06T05:15:03.959855+0000","last_became_active":"2023-10-06T05:14:58.966124+0000","last_became_peered":"2023-10-06T05:14:58.966124+0000","last_unstale":"2023-10-06T05:15:03.959855+0000","last_undegraded":"2023-10-06T05:15:03.959855+0000","last_fullsized":"2023-10-06T05:15:03.959855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973548+0000","last_change":"2023-10-06T05:14:59.953791+0000","last_active":"2023-10-06T05:15:03.973548+0000","last_peered":"2023-10-06T05:15:03.973548+0000","last_clean":"2023-10-06T05:15:03.973548+0000","last_became_active":"2023-10-06T05:14:59.953491+0000","last_became_peered":"2023-10-06T05:14:59.953491+0000","last_unstale":"2023-10-06T05:15:03.973548+0000","last_undegraded":"2023-10-06T05:15:03.973548+0000","last_fullsized":"2023-10-06T05:15:03.973548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994360+0000","last_change":"2023-10-06T05:15:03.993204+0000","last_active":"2023-10-06T05:15:03.994360+0000","last_peered":"2023-10-06T05:15:03.994360+0000","last_clean":"2023-10-06T05:15:03.994360+0000","last_became_active":"2023-10-06T05:15:03.992794+0000","last_became_peered":"2023-10-06T05:15:03.992794+0000","last_unstale":"2023-10-06T05:15:03.994360+0000","last_undegraded":"2023-10-06T05:15:03.994360+0000","last_fullsized":"2023-10-06T05:15:03.994360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:53.385235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967975+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:15:03.967975+0000","last_peered":"2023-10-06T05:15:03.967975+0000","last_clean":"2023-10-06T05:15:03.967975+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:15:03.967975+0000","last_undegraded":"2023-10-06T05:15:03.967975+0000","last_fullsized":"2023-10-06T05:15:03.967975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972830+0000","last_change":"2023-10-06T05:14:58.954417+0000","last_active":"2023-10-06T05:15:03.972830+0000","last_peered":"2023-10-06T05:15:03.972830+0000","last_clean":"2023-10-06T05:15:03.972830+0000","last_became_active":"2023-10-06T05:14:58.954052+0000","last_became_peered":"2023-10-06T05:14:58.954052+0000","last_unstale":"2023-10-06T05:15:03.972830+0000","last_undegraded":"2023-10-06T05:15:03.972830+0000","last_fullsized":"2023-10-06T05:15:03.972830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979150+0000","last_change":"2023-10-06T05:14:59.963695+0000","last_active":"2023-10-06T05:15:03.979150+0000","last_peered":"2023-10-06T05:15:03.979150+0000","last_clean":"2023-10-06T05:15:03.979150+0000","last_became_active":"2023-10-06T05:14:59.963426+0000","last_became_peered":"2023-10-06T05:14:59.963426+0000","last_unstale":"2023-10-06T05:15:03.979150+0000","last_undegraded":"2023-10-06T05:15:03.979150+0000","last_fullsized":"2023-10-06T05:15:03.979150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987182+0000","last_change":"2023-10-06T05:15:03.985907+0000","last_active":"2023-10-06T05:15:03.987182+0000","last_peered":"2023-10-06T05:15:03.987182+0000","last_clean":"2023-10-06T05:15:03.987182+0000","last_became_active":"2023-10-06T05:15:03.985534+0000","last_became_peered":"2023-10-06T05:15:03.985534+0000","last_unstale":"2023-10-06T05:15:03.987182+0000","last_undegraded":"2023-10-06T05:15:03.987182+0000","last_fullsized":"2023-10-06T05:15:03.987182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961919+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:15:03.961919+0000","last_peered":"2023-10-06T05:15:03.961919+0000","last_clean":"2023-10-06T05:15:03.961919+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:15:03.961919+0000","last_undegraded":"2023-10-06T05:15:03.961919+0000","last_fullsized":"2023-10-06T05:15:03.961919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964512+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:15:03.964512+0000","last_peered":"2023-10-06T05:15:03.964512+0000","last_clean":"2023-10-06T05:15:03.964512+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:15:03.964512+0000","last_undegraded":"2023-10-06T05:15:03.964512+0000","last_fullsized":"2023-10-06T05:15:03.964512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963731+0000","last_change":"2023-10-06T05:14:58.965199+0000","last_active":"2023-10-06T05:15:03.963731+0000","last_peered":"2023-10-06T05:15:03.963731+0000","last_clean":"2023-10-06T05:15:03.963731+0000","last_became_active":"2023-10-06T05:14:58.964902+0000","last_became_peered":"2023-10-06T05:14:58.964902+0000","last_unstale":"2023-10-06T05:15:03.963731+0000","last_undegraded":"2023-10-06T05:15:03.963731+0000","last_fullsized":"2023-10-06T05:15:03.963731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975967+0000","last_change":"2023-10-06T05:14:59.953471+0000","last_active":"2023-10-06T05:15:03.975967+0000","last_peered":"2023-10-06T05:15:03.975967+0000","last_clean":"2023-10-06T05:15:03.975967+0000","last_became_active":"2023-10-06T05:14:59.953166+0000","last_became_peered":"2023-10-06T05:14:59.953166+0000","last_unstale":"2023-10-06T05:15:03.975967+0000","last_undegraded":"2023-10-06T05:15:03.975967+0000","last_fullsized":"2023-10-06T05:15:03.975967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:35.282268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987431+0000","last_change":"2023-10-06T05:15:03.982741+0000","last_active":"2023-10-06T05:15:03.987431+0000","last_peered":"2023-10-06T05:15:03.987431+0000","last_clean":"2023-10-06T05:15:03.987431+0000","last_became_active":"2023-10-06T05:15:03.982333+0000","last_became_peered":"2023-10-06T05:15:03.982333+0000","last_unstale":"2023-10-06T05:15:03.987431+0000","last_undegraded":"2023-10-06T05:15:03.987431+0000","last_fullsized":"2023-10-06T05:15:03.987431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:07.926945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964687+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:15:03.964687+0000","last_peered":"2023-10-06T05:15:03.964687+0000","last_clean":"2023-10-06T05:15:03.964687+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:15:03.964687+0000","last_undegraded":"2023-10-06T05:15:03.964687+0000","last_fullsized":"2023-10-06T05:15:03.964687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965134+0000","last_change":"2023-10-06T05:14:58.970767+0000","last_active":"2023-10-06T05:15:03.965134+0000","last_peered":"2023-10-06T05:15:03.965134+0000","last_clean":"2023-10-06T05:15:03.965134+0000","last_became_active":"2023-10-06T05:14:58.970523+0000","last_became_peered":"2023-10-06T05:14:58.970523+0000","last_unstale":"2023-10-06T05:15:03.965134+0000","last_undegraded":"2023-10-06T05:15:03.965134+0000","last_fullsized":"2023-10-06T05:15:03.965134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970560+0000","last_change":"2023-10-06T05:14:59.958130+0000","last_active":"2023-10-06T05:15:03.970560+0000","last_peered":"2023-10-06T05:15:03.970560+0000","last_clean":"2023-10-06T05:15:03.970560+0000","last_became_active":"2023-10-06T05:14:59.957826+0000","last_became_peered":"2023-10-06T05:14:59.957826+0000","last_unstale":"2023-10-06T05:15:03.970560+0000","last_undegraded":"2023-10-06T05:15:03.970560+0000","last_fullsized":"2023-10-06T05:15:03.970560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987228+0000","last_change":"2023-10-06T05:15:03.981078+0000","last_active":"2023-10-06T05:15:03.987228+0000","last_peered":"2023-10-06T05:15:03.987228+0000","last_clean":"2023-10-06T05:15:03.987228+0000","last_became_active":"2023-10-06T05:15:03.980645+0000","last_became_peered":"2023-10-06T05:15:03.980645+0000","last_unstale":"2023-10-06T05:15:03.987228+0000","last_undegraded":"2023-10-06T05:15:03.987228+0000","last_fullsized":"2023-10-06T05:15:03.987228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.847066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967525+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:15:03.967525+0000","last_peered":"2023-10-06T05:15:03.967525+0000","last_clean":"2023-10-06T05:15:03.967525+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:15:03.967525+0000","last_undegraded":"2023-10-06T05:15:03.967525+0000","last_fullsized":"2023-10-06T05:15:03.967525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973787+0000","last_change":"2023-10-06T05:14:58.954390+0000","last_active":"2023-10-06T05:15:03.973787+0000","last_peered":"2023-10-06T05:15:03.973787+0000","last_clean":"2023-10-06T05:15:03.973787+0000","last_became_active":"2023-10-06T05:14:58.954003+0000","last_became_peered":"2023-10-06T05:14:58.954003+0000","last_unstale":"2023-10-06T05:15:03.973787+0000","last_undegraded":"2023-10-06T05:15:03.973787+0000","last_fullsized":"2023-10-06T05:15:03.973787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967163+0000","last_change":"2023-10-06T05:14:59.965298+0000","last_active":"2023-10-06T05:15:03.967163+0000","last_peered":"2023-10-06T05:15:03.967163+0000","last_clean":"2023-10-06T05:15:03.967163+0000","last_became_active":"2023-10-06T05:14:59.964967+0000","last_became_peered":"2023-10-06T05:14:59.964967+0000","last_unstale":"2023-10-06T05:15:03.967163+0000","last_undegraded":"2023-10-06T05:15:03.967163+0000","last_fullsized":"2023-10-06T05:15:03.967163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978732+0000","last_change":"2023-10-06T05:15:03.976855+0000","last_active":"2023-10-06T05:15:03.978732+0000","last_peered":"2023-10-06T05:15:03.978732+0000","last_clean":"2023-10-06T05:15:03.978732+0000","last_became_active":"2023-10-06T05:15:03.976461+0000","last_became_peered":"2023-10-06T05:15:03.976461+0000","last_unstale":"2023-10-06T05:15:03.978732+0000","last_undegraded":"2023-10-06T05:15:03.978732+0000","last_fullsized":"2023-10-06T05:15:03.978732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:45.979196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964732+0000","last_change":"2023-10-06T05:14:58.970171+0000","last_active":"2023-10-06T05:15:03.964732+0000","last_peered":"2023-10-06T05:15:03.964732+0000","last_clean":"2023-10-06T05:15:03.964732+0000","last_became_active":"2023-10-06T05:14:58.969914+0000","last_became_peered":"2023-10-06T05:14:58.969914+0000","last_unstale":"2023-10-06T05:15:03.964732+0000","last_undegraded":"2023-10-06T05:15:03.964732+0000","last_fullsized":"2023-10-06T05:15:03.964732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977482+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:15:03.977482+0000","last_peered":"2023-10-06T05:15:03.977482+0000","last_clean":"2023-10-06T05:15:03.977482+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:15:03.977482+0000","last_undegraded":"2023-10-06T05:15:03.977482+0000","last_fullsized":"2023-10-06T05:15:03.977482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964253+0000","last_change":"2023-10-06T05:14:59.964377+0000","last_active":"2023-10-06T05:15:03.964253+0000","last_peered":"2023-10-06T05:15:03.964253+0000","last_clean":"2023-10-06T05:15:03.964253+0000","last_became_active":"2023-10-06T05:14:59.964058+0000","last_became_peered":"2023-10-06T05:14:59.964058+0000","last_unstale":"2023-10-06T05:15:03.964253+0000","last_undegraded":"2023-10-06T05:15:03.964253+0000","last_fullsized":"2023-10-06T05:15:03.964253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994304+0000","last_change":"2023-10-06T05:15:03.991574+0000","last_active":"2023-10-06T05:15:03.994304+0000","last_peered":"2023-10-06T05:15:03.994304+0000","last_clean":"2023-10-06T05:15:03.994304+0000","last_became_active":"2023-10-06T05:15:03.991171+0000","last_became_peered":"2023-10-06T05:15:03.991171+0000","last_unstale":"2023-10-06T05:15:03.994304+0000","last_undegraded":"2023-10-06T05:15:03.994304+0000","last_fullsized":"2023-10-06T05:15:03.994304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975779+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:15:03.975779+0000","last_peered":"2023-10-06T05:15:03.975779+0000","last_clean":"2023-10-06T05:15:03.975779+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:15:03.975779+0000","last_undegraded":"2023-10-06T05:15:03.975779+0000","last_fullsized":"2023-10-06T05:15:03.975779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970480+0000","last_change":"2023-10-06T05:14:58.966696+0000","last_active":"2023-10-06T05:15:03.970480+0000","last_peered":"2023-10-06T05:15:03.970480+0000","last_clean":"2023-10-06T05:15:03.970480+0000","last_became_active":"2023-10-06T05:14:58.966398+0000","last_became_peered":"2023-10-06T05:14:58.966398+0000","last_unstale":"2023-10-06T05:15:03.970480+0000","last_undegraded":"2023-10-06T05:15:03.970480+0000","last_fullsized":"2023-10-06T05:15:03.970480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960837+0000","last_change":"2023-10-06T05:14:59.967931+0000","last_active":"2023-10-06T05:15:03.960837+0000","last_peered":"2023-10-06T05:15:03.960837+0000","last_clean":"2023-10-06T05:15:03.960837+0000","last_became_active":"2023-10-06T05:14:59.967681+0000","last_became_peered":"2023-10-06T05:14:59.967681+0000","last_unstale":"2023-10-06T05:15:03.960837+0000","last_undegraded":"2023-10-06T05:15:03.960837+0000","last_fullsized":"2023-10-06T05:15:03.960837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990296+0000","last_change":"2023-10-06T05:15:03.989021+0000","last_active":"2023-10-06T05:15:03.990296+0000","last_peered":"2023-10-06T05:15:03.990296+0000","last_clean":"2023-10-06T05:15:03.990296+0000","last_became_active":"2023-10-06T05:15:03.988411+0000","last_became_peered":"2023-10-06T05:15:03.988411+0000","last_unstale":"2023-10-06T05:15:03.990296+0000","last_undegraded":"2023-10-06T05:15:03.990296+0000","last_fullsized":"2023-10-06T05:15:03.990296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967052+0000","last_change":"2023-10-06T05:14:58.970840+0000","last_active":"2023-10-06T05:15:03.967052+0000","last_peered":"2023-10-06T05:15:03.967052+0000","last_clean":"2023-10-06T05:15:03.967052+0000","last_became_active":"2023-10-06T05:14:58.970619+0000","last_became_peered":"2023-10-06T05:14:58.970619+0000","last_unstale":"2023-10-06T05:15:03.967052+0000","last_undegraded":"2023-10-06T05:15:03.967052+0000","last_fullsized":"2023-10-06T05:15:03.967052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968655+0000","last_change":"2023-10-06T05:14:59.955830+0000","last_active":"2023-10-06T05:15:03.968655+0000","last_peered":"2023-10-06T05:15:03.968655+0000","last_clean":"2023-10-06T05:15:03.968655+0000","last_became_active":"2023-10-06T05:14:59.955566+0000","last_became_peered":"2023-10-06T05:14:59.955566+0000","last_unstale":"2023-10-06T05:15:03.968655+0000","last_undegraded":"2023-10-06T05:15:03.968655+0000","last_fullsized":"2023-10-06T05:15:03.968655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:47.738141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987078+0000","last_change":"2023-10-06T05:15:03.984831+0000","last_active":"2023-10-06T05:15:03.987078+0000","last_peered":"2023-10-06T05:15:03.987078+0000","last_clean":"2023-10-06T05:15:03.987078+0000","last_became_active":"2023-10-06T05:15:03.984436+0000","last_became_peered":"2023-10-06T05:15:03.984436+0000","last_unstale":"2023-10-06T05:15:03.987078+0000","last_undegraded":"2023-10-06T05:15:03.987078+0000","last_fullsized":"2023-10-06T05:15:03.987078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:10.551105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961685+0000","last_change":"2023-10-06T05:14:58.956457+0000","last_active":"2023-10-06T05:15:03.961685+0000","last_peered":"2023-10-06T05:15:03.961685+0000","last_clean":"2023-10-06T05:15:03.961685+0000","last_became_active":"2023-10-06T05:14:58.956165+0000","last_became_peered":"2023-10-06T05:14:58.956165+0000","last_unstale":"2023-10-06T05:15:03.961685+0000","last_undegraded":"2023-10-06T05:15:03.961685+0000","last_fullsized":"2023-10-06T05:15:03.961685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.968300+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.967965+0000","last_became_peered":"2023-10-06T05:14:59.967965+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:35.451365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978224+0000","last_change":"2023-10-06T05:15:03.976752+0000","last_active":"2023-10-06T05:15:03.978224+0000","last_peered":"2023-10-06T05:15:03.978224+0000","last_clean":"2023-10-06T05:15:03.978224+0000","last_became_active":"2023-10-06T05:15:03.976322+0000","last_became_peered":"2023-10-06T05:15:03.976322+0000","last_unstale":"2023-10-06T05:15:03.978224+0000","last_undegraded":"2023-10-06T05:15:03.978224+0000","last_fullsized":"2023-10-06T05:15:03.978224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962719+0000","last_change":"2023-10-06T05:14:58.955881+0000","last_active":"2023-10-06T05:15:03.962719+0000","last_peered":"2023-10-06T05:15:03.962719+0000","last_clean":"2023-10-06T05:15:03.962719+0000","last_became_active":"2023-10-06T05:14:58.955624+0000","last_became_peered":"2023-10-06T05:14:58.955624+0000","last_unstale":"2023-10-06T05:15:03.962719+0000","last_undegraded":"2023-10-06T05:15:03.962719+0000","last_fullsized":"2023-10-06T05:15:03.962719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:16.365139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960721+0000","last_change":"2023-10-06T05:14:59.968565+0000","last_active":"2023-10-06T05:15:03.960721+0000","last_peered":"2023-10-06T05:15:03.960721+0000","last_clean":"2023-10-06T05:15:03.960721+0000","last_became_active":"2023-10-06T05:14:59.965728+0000","last_became_peered":"2023-10-06T05:14:59.965728+0000","last_unstale":"2023-10-06T05:15:03.960721+0000","last_undegraded":"2023-10-06T05:15:03.960721+0000","last_fullsized":"2023-10-06T05:15:03.960721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:49.359907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977840+0000","last_change":"2023-10-06T05:15:03.975283+0000","last_active":"2023-10-06T05:15:03.977840+0000","last_peered":"2023-10-06T05:15:03.977840+0000","last_clean":"2023-10-06T05:15:03.977840+0000","last_became_active":"2023-10-06T05:15:03.974903+0000","last_became_peered":"2023-10-06T05:15:03.974903+0000","last_unstale":"2023-10-06T05:15:03.977840+0000","last_undegraded":"2023-10-06T05:15:03.977840+0000","last_fullsized":"2023-10-06T05:15:03.977840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:18.853022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966753+0000","last_change":"2023-10-06T05:14:58.961971+0000","last_active":"2023-10-06T05:15:03.966753+0000","last_peered":"2023-10-06T05:15:03.966753+0000","last_clean":"2023-10-06T05:15:03.966753+0000","last_became_active":"2023-10-06T05:14:58.961689+0000","last_became_peered":"2023-10-06T05:14:58.961689+0000","last_unstale":"2023-10-06T05:15:03.966753+0000","last_undegraded":"2023-10-06T05:15:03.966753+0000","last_fullsized":"2023-10-06T05:15:03.966753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963640+0000","last_change":"2023-10-06T05:14:59.959821+0000","last_active":"2023-10-06T05:15:03.963640+0000","last_peered":"2023-10-06T05:15:03.963640+0000","last_clean":"2023-10-06T05:15:03.963640+0000","last_became_active":"2023-10-06T05:14:59.959543+0000","last_became_peered":"2023-10-06T05:14:59.959543+0000","last_unstale":"2023-10-06T05:15:03.963640+0000","last_undegraded":"2023-10-06T05:15:03.963640+0000","last_fullsized":"2023-10-06T05:15:03.963640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.991469+0000","last_change":"2023-10-06T05:15:03.990597+0000","last_active":"2023-10-06T05:15:03.991469+0000","last_peered":"2023-10-06T05:15:03.991469+0000","last_clean":"2023-10-06T05:15:03.991469+0000","last_became_active":"2023-10-06T05:15:03.990258+0000","last_became_peered":"2023-10-06T05:15:03.990258+0000","last_unstale":"2023-10-06T05:15:03.991469+0000","last_undegraded":"2023-10-06T05:15:03.991469+0000","last_fullsized":"2023-10-06T05:15:03.991469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970366+0000","last_change":"2023-10-06T05:14:58.970911+0000","last_active":"2023-10-06T05:15:03.970366+0000","last_peered":"2023-10-06T05:15:03.970366+0000","last_clean":"2023-10-06T05:15:03.970366+0000","last_became_active":"2023-10-06T05:14:58.970439+0000","last_became_peered":"2023-10-06T05:14:58.970439+0000","last_unstale":"2023-10-06T05:15:03.970366+0000","last_undegraded":"2023-10-06T05:15:03.970366+0000","last_fullsized":"2023-10-06T05:15:03.970366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960109+0000","last_change":"2023-10-06T05:14:59.961878+0000","last_active":"2023-10-06T05:15:03.960109+0000","last_peered":"2023-10-06T05:15:03.960109+0000","last_clean":"2023-10-06T05:15:03.960109+0000","last_became_active":"2023-10-06T05:14:59.961617+0000","last_became_peered":"2023-10-06T05:14:59.961617+0000","last_unstale":"2023-10-06T05:15:03.960109+0000","last_undegraded":"2023-10-06T05:15:03.960109+0000","last_fullsized":"2023-10-06T05:15:03.960109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974134+0000","last_change":"2023-10-06T05:14:59.960593+0000","last_active":"2023-10-06T05:15:03.974134+0000","last_peered":"2023-10-06T05:15:03.974134+0000","last_clean":"2023-10-06T05:15:03.974134+0000","last_became_active":"2023-10-06T05:14:59.960329+0000","last_became_peered":"2023-10-06T05:14:59.960329+0000","last_unstale":"2023-10-06T05:15:03.974134+0000","last_undegraded":"2023-10-06T05:15:03.974134+0000","last_fullsized":"2023-10-06T05:15:03.974134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963791+0000","last_change":"2023-10-06T05:14:59.966932+0000","last_active":"2023-10-06T05:15:03.963791+0000","last_peered":"2023-10-06T05:15:03.963791+0000","last_clean":"2023-10-06T05:15:03.963791+0000","last_became_active":"2023-10-06T05:14:59.965101+0000","last_became_peered":"2023-10-06T05:14:59.965101+0000","last_unstale":"2023-10-06T05:15:03.963791+0000","last_undegraded":"2023-10-06T05:15:03.963791+0000","last_fullsized":"2023-10-06T05:15:03.963791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961190+0000","last_change":"2023-10-06T05:14:59.961912+0000","last_active":"2023-10-06T05:15:03.961190+0000","last_peered":"2023-10-06T05:15:03.961190+0000","last_clean":"2023-10-06T05:15:03.961190+0000","last_became_active":"2023-10-06T05:14:59.961658+0000","last_became_peered":"2023-10-06T05:14:59.961658+0000","last_unstale":"2023-10-06T05:15:03.961190+0000","last_undegraded":"2023-10-06T05:15:03.961190+0000","last_fullsized":"2023-10-06T05:15:03.961190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963927+0000","last_change":"2023-10-06T05:14:59.968762+0000","last_active":"2023-10-06T05:15:03.963927+0000","last_peered":"2023-10-06T05:15:03.963927+0000","last_clean":"2023-10-06T05:15:03.963927+0000","last_became_active":"2023-10-06T05:14:59.968111+0000","last_became_peered":"2023-10-06T05:14:59.968111+0000","last_unstale":"2023-10-06T05:15:03.963927+0000","last_undegraded":"2023-10-06T05:15:03.963927+0000","last_fullsized":"2023-10-06T05:15:03.963927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973699+0000","last_change":"2023-10-06T05:14:59.958480+0000","last_active":"2023-10-06T05:15:03.973699+0000","last_peered":"2023-10-06T05:15:03.973699+0000","last_clean":"2023-10-06T05:15:03.973699+0000","last_became_active":"2023-10-06T05:14:59.958139+0000","last_became_peered":"2023-10-06T05:14:59.958139+0000","last_unstale":"2023-10-06T05:15:03.973699+0000","last_undegraded":"2023-10-06T05:15:03.973699+0000","last_fullsized":"2023-10-06T05:15:03.973699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:03.025853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0490000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":695352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.571}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":103167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:24.844 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:15:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.008+0000 7fab24fe1700 1 Processor -- start 2023-10-06T05:15:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.008+0000 7fab24fe1700 1 -- start start 2023-10-06T05:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab24fe1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab2015a490 0x7fab20067830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab24fe1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015b5e0 0x7fab2006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab24fe1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab200b2120 con 0x7fab2006add0 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab24fe1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab200ae020 con 0x7fab2015a490 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab24fe1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2006b250 con 0x7fab2015b5e0 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab2015a490 0x7fab20067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab2015a490 0x7fab20067830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53530/0 (socket says 172.21.15.45:53530) 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 -- 172.21.15.45:0/1898080783 learned_addr learned my addr 172.21.15.45:0/1898080783 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1dd9b700 1 --2- 172.21.15.45:0/1898080783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015b5e0 0x7fab2006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 -- 172.21.15.45:0/1898080783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015b5e0 msgr2=0x7fab2006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 --2- 172.21.15.45:0/1898080783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015b5e0 0x7fab2006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 -- 172.21.15.45:0/1898080783 >> v1:172.21.15.144:6789/0 conn(0x7fab2006add0 legacy=0x7fab2006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 -- 172.21.15.45:0/1898080783 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab200ab7d0 con 0x7fab2015a490 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1dd9b700 1 --2- 172.21.15.45:0/1898080783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015b5e0 0x7fab2006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.009+0000 7fab1e59c700 1 --2- 172.21.15.45:0/1898080783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2015a490 0x7fab20067830 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fab140099b0 tx=0x7fab14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5554ac9dfed01dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab1d59a700 1 -- 172.21.15.45:0/1898080783 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400b860 con 0x7fab2015a490 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab1d59a700 1 -- 172.21.15.45:0/1898080783 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1400b9c0 con 0x7fab2015a490 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab1d59a700 1 -- 172.21.15.45:0/1898080783 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400bd60 con 0x7fab2015a490 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 -- 172.21.15.45:0/1898080783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2015a490 msgr2=0x7fab20067830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 --2- 172.21.15.45:0/1898080783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2015a490 0x7fab20067830 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fab140099b0 tx=0x7fab14009250 comp rx=0 tx=0).stop 2023-10-06T05:15:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 -- 172.21.15.45:0/1898080783 shutdown_connections 2023-10-06T05:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 --2- 172.21.15.45:0/1898080783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2015a490 0x7fab20067830 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 --2- 172.21.15.45:0/1898080783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015b5e0 0x7fab2006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 -- 172.21.15.45:0/1898080783 >> 172.21.15.45:0/1898080783 conn(0x7fab200cc5a0 msgr2=0x7fab200cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 -- 172.21.15.45:0/1898080783 shutdown_connections 2023-10-06T05:15:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.010+0000 7fab24fe1700 1 -- 172.21.15.45:0/1898080783 wait complete. 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 Processor -- start 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 -- start start 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2006add0 0x7fab200af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015a490 0x7fab200afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab2015b5e0 0x7fab200a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab20158a30 con 0x7fab2006add0 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab200ab200 con 0x7fab2015b5e0 2023-10-06T05:15:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab24fe1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab200aa5f0 con 0x7fab2015a490 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab1e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2006add0 0x7fab200af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab1e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2006add0 0x7fab200af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53544/0 (socket says 172.21.15.45:53544) 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.012+0000 7fab1e59c700 1 -- 172.21.15.45:0/551551363 learned_addr learned my addr 172.21.15.45:0/551551363 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1ed9d700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab2015b5e0 0x7fab200a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1dd9b700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015a490 0x7fab200afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1e59c700 1 -- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015a490 msgr2=0x7fab200afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1e59c700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015a490 0x7fab200afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1e59c700 1 -- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab2015b5e0 msgr2=0x7fab200a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1e59c700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab2015b5e0 0x7fab200a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1e59c700 1 -- 172.21.15.45:0/551551363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140094f0 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1dd9b700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015a490 0x7fab200afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab1e59c700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2006add0 0x7fab200af610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fab14009290 tx=0x7fab14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab0e7fc700 1 -- 172.21.15.45:0/551551363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14026480 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab0e7fc700 1 -- 172.21.15.45:0/551551363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1401f070 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab0e7fc700 1 -- 172.21.15.45:0/551551363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14026830 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab200aa8d0 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.013+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab200aae90 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.014+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab200cc0e0 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.015+0000 7fab0e7fc700 1 -- 172.21.15.45:0/551551363 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab14026ba0 con 0x7fab2006add0 2023-10-06T05:15:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.015+0000 7fab0e7fc700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab00066f00 0x7fab000693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.015+0000 7fab0e7fc700 1 -- 172.21.15.45:0/551551363 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fab14021070 con 0x7fab2006add0 2023-10-06T05:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.015+0000 7fab1dd9b700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab00066f00 0x7fab000693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.016+0000 7fab1dd9b700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab00066f00 0x7fab000693b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fab10006f00 tx=0x7fab10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.020+0000 7fab0e7fc700 1 -- 172.21.15.45:0/551551363 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab14053590 con 0x7fab2006add0 2023-10-06T05:15:25.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.217+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab20067e80 con 0x7fab00066f00 2023-10-06T05:15:25.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.228+0000 7fab0e7fc700 1 -- 172.21.15.45:0/551551363 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367296 (secure 0 0 0) 0x7fab20067e80 con 0x7fab00066f00 2023-10-06T05:15:25.234 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab00066f00 msgr2=0x7fab000693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab00066f00 0x7fab000693b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fab10006f00 tx=0x7fab10008040 comp rx=0 tx=0).stop 2023-10-06T05:15:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2006add0 msgr2=0x7fab200af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2006add0 0x7fab200af610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fab14009290 tx=0x7fab14017000 comp rx=0 tx=0).stop 2023-10-06T05:15:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 shutdown_connections 2023-10-06T05:15:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab00066f00 0x7fab000693b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab2015b5e0 0x7fab200a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2006add0 0x7fab200af610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 --2- 172.21.15.45:0/551551363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab2015a490 0x7fab200afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 >> 172.21.15.45:0/551551363 conn(0x7fab200cc5a0 msgr2=0x7fab200cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 shutdown_connections 2023-10-06T05:15:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.232+0000 7fab24fe1700 1 -- 172.21.15.45:0/551551363 wait complete. 2023-10-06T05:15:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:15:25.247 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-06T05:15:24.446145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381508,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593212,"statfs":{"total":773094113280,"available":770655449088,"internally_reserved":0,"allocated":3280896,"data_stored":2023793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2179,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003346"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.969110+0000","last_change":"2023-10-06T05:14:59.967201+0000","last_active":"2023-10-06T05:15:03.969110+0000","last_peered":"2023-10-06T05:15:03.969110+0000","last_clean":"2023-10-06T05:15:03.969110+0000","last_became_active":"2023-10-06T05:14:59.966818+0000","last_became_peered":"2023-10-06T05:14:59.966818+0000","last_unstale":"2023-10-06T05:15:03.969110+0000","last_undegraded":"2023-10-06T05:15:03.969110+0000","last_fullsized":"2023-10-06T05:15:03.969110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963221+0000","last_change":"2023-10-06T05:14:59.972677+0000","last_active":"2023-10-06T05:15:03.963221+0000","last_peered":"2023-10-06T05:15:03.963221+0000","last_clean":"2023-10-06T05:15:03.963221+0000","last_became_active":"2023-10-06T05:14:59.972435+0000","last_became_peered":"2023-10-06T05:14:59.972435+0000","last_unstale":"2023-10-06T05:15:03.963221+0000","last_undegraded":"2023-10-06T05:15:03.963221+0000","last_fullsized":"2023-10-06T05:15:03.963221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960554+0000","last_change":"2023-10-06T05:14:59.955617+0000","last_active":"2023-10-06T05:15:03.960554+0000","last_peered":"2023-10-06T05:15:03.960554+0000","last_clean":"2023-10-06T05:15:03.960554+0000","last_became_active":"2023-10-06T05:14:59.951585+0000","last_became_peered":"2023-10-06T05:14:59.951585+0000","last_unstale":"2023-10-06T05:15:03.960554+0000","last_undegraded":"2023-10-06T05:15:03.960554+0000","last_fullsized":"2023-10-06T05:15:03.960554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:56.835787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975703+0000","last_change":"2023-10-06T05:14:59.962591+0000","last_active":"2023-10-06T05:15:03.975703+0000","last_peered":"2023-10-06T05:15:03.975703+0000","last_clean":"2023-10-06T05:15:03.975703+0000","last_became_active":"2023-10-06T05:14:59.962219+0000","last_became_peered":"2023-10-06T05:14:59.962219+0000","last_unstale":"2023-10-06T05:15:03.975703+0000","last_undegraded":"2023-10-06T05:15:03.975703+0000","last_fullsized":"2023-10-06T05:15:03.975703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:11.373603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967261+0000","last_change":"2023-10-06T05:14:59.971703+0000","last_active":"2023-10-06T05:15:03.967261+0000","last_peered":"2023-10-06T05:15:03.967261+0000","last_clean":"2023-10-06T05:15:03.967261+0000","last_became_active":"2023-10-06T05:14:59.971083+0000","last_became_peered":"2023-10-06T05:14:59.971083+0000","last_unstale":"2023-10-06T05:15:03.967261+0000","last_undegraded":"2023-10-06T05:15:03.967261+0000","last_fullsized":"2023-10-06T05:15:03.967261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961235+0000","last_change":"2023-10-06T05:14:59.952376+0000","last_active":"2023-10-06T05:15:03.961235+0000","last_peered":"2023-10-06T05:15:03.961235+0000","last_clean":"2023-10-06T05:15:03.961235+0000","last_became_active":"2023-10-06T05:14:59.952101+0000","last_became_peered":"2023-10-06T05:14:59.952101+0000","last_unstale":"2023-10-06T05:15:03.961235+0000","last_undegraded":"2023-10-06T05:15:03.961235+0000","last_fullsized":"2023-10-06T05:15:03.961235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:46.640934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965050+0000","last_change":"2023-10-06T05:14:59.958503+0000","last_active":"2023-10-06T05:15:03.965050+0000","last_peered":"2023-10-06T05:15:03.965050+0000","last_clean":"2023-10-06T05:15:03.965050+0000","last_became_active":"2023-10-06T05:14:59.957962+0000","last_became_peered":"2023-10-06T05:14:59.957962+0000","last_unstale":"2023-10-06T05:15:03.965050+0000","last_undegraded":"2023-10-06T05:15:03.965050+0000","last_fullsized":"2023-10-06T05:15:03.965050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961531+0000","last_change":"2023-10-06T05:14:59.966704+0000","last_active":"2023-10-06T05:15:03.961531+0000","last_peered":"2023-10-06T05:15:03.961531+0000","last_clean":"2023-10-06T05:15:03.961531+0000","last_became_active":"2023-10-06T05:14:59.966378+0000","last_became_peered":"2023-10-06T05:14:59.966378+0000","last_unstale":"2023-10-06T05:15:03.961531+0000","last_undegraded":"2023-10-06T05:15:03.961531+0000","last_fullsized":"2023-10-06T05:15:03.961531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970103+0000","last_change":"2023-10-06T05:14:59.967328+0000","last_active":"2023-10-06T05:15:03.970103+0000","last_peered":"2023-10-06T05:15:03.970103+0000","last_clean":"2023-10-06T05:15:03.970103+0000","last_became_active":"2023-10-06T05:14:59.966868+0000","last_became_peered":"2023-10-06T05:14:59.966868+0000","last_unstale":"2023-10-06T05:15:03.970103+0000","last_undegraded":"2023-10-06T05:15:03.970103+0000","last_fullsized":"2023-10-06T05:15:03.970103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963384+0000","last_change":"2023-10-06T05:14:59.966488+0000","last_active":"2023-10-06T05:15:03.963384+0000","last_peered":"2023-10-06T05:15:03.963384+0000","last_clean":"2023-10-06T05:15:03.963384+0000","last_became_active":"2023-10-06T05:14:59.966153+0000","last_became_peered":"2023-10-06T05:14:59.966153+0000","last_unstale":"2023-10-06T05:15:03.963384+0000","last_undegraded":"2023-10-06T05:15:03.963384+0000","last_fullsized":"2023-10-06T05:15:03.963384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960401+0000","last_change":"2023-10-06T05:14:59.956939+0000","last_active":"2023-10-06T05:15:03.960401+0000","last_peered":"2023-10-06T05:15:03.960401+0000","last_clean":"2023-10-06T05:15:03.960401+0000","last_became_active":"2023-10-06T05:14:59.956467+0000","last_became_peered":"2023-10-06T05:14:59.956467+0000","last_unstale":"2023-10-06T05:15:03.960401+0000","last_undegraded":"2023-10-06T05:15:03.960401+0000","last_fullsized":"2023-10-06T05:15:03.960401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968295+0000","last_change":"2023-10-06T05:14:59.965955+0000","last_active":"2023-10-06T05:15:03.968295+0000","last_peered":"2023-10-06T05:15:03.968295+0000","last_clean":"2023-10-06T05:15:03.968295+0000","last_became_active":"2023-10-06T05:14:59.965676+0000","last_became_peered":"2023-10-06T05:14:59.965676+0000","last_unstale":"2023-10-06T05:15:03.968295+0000","last_undegraded":"2023-10-06T05:15:03.968295+0000","last_fullsized":"2023-10-06T05:15:03.968295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:39.098260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968290+0000","last_change":"2023-10-06T05:14:59.971318+0000","last_active":"2023-10-06T05:15:03.968290+0000","last_peered":"2023-10-06T05:15:03.968290+0000","last_clean":"2023-10-06T05:15:03.968290+0000","last_became_active":"2023-10-06T05:14:59.971044+0000","last_became_peered":"2023-10-06T05:14:59.971044+0000","last_unstale":"2023-10-06T05:15:03.968290+0000","last_undegraded":"2023-10-06T05:15:03.968290+0000","last_fullsized":"2023-10-06T05:15:03.968290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:17.372375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962264+0000","last_change":"2023-10-06T05:14:59.962752+0000","last_active":"2023-10-06T05:15:03.962264+0000","last_peered":"2023-10-06T05:15:03.962264+0000","last_clean":"2023-10-06T05:15:03.962264+0000","last_became_active":"2023-10-06T05:14:59.962473+0000","last_became_peered":"2023-10-06T05:14:59.962473+0000","last_unstale":"2023-10-06T05:15:03.962264+0000","last_undegraded":"2023-10-06T05:15:03.962264+0000","last_fullsized":"2023-10-06T05:15:03.962264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964859+0000","last_change":"2023-10-06T05:14:59.958686+0000","last_active":"2023-10-06T05:15:03.964859+0000","last_peered":"2023-10-06T05:15:03.964859+0000","last_clean":"2023-10-06T05:15:03.964859+0000","last_became_active":"2023-10-06T05:14:59.958258+0000","last_became_peered":"2023-10-06T05:14:59.958258+0000","last_unstale":"2023-10-06T05:15:03.964859+0000","last_undegraded":"2023-10-06T05:15:03.964859+0000","last_fullsized":"2023-10-06T05:15:03.964859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960536+0000","last_change":"2023-10-06T05:14:59.956754+0000","last_active":"2023-10-06T05:15:03.960536+0000","last_peered":"2023-10-06T05:15:03.960536+0000","last_clean":"2023-10-06T05:15:03.960536+0000","last_became_active":"2023-10-06T05:14:59.951445+0000","last_became_peered":"2023-10-06T05:14:59.951445+0000","last_unstale":"2023-10-06T05:15:03.960536+0000","last_undegraded":"2023-10-06T05:15:03.960536+0000","last_fullsized":"2023-10-06T05:15:03.960536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:43.881359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970204+0000","last_change":"2023-10-06T05:14:59.967155+0000","last_active":"2023-10-06T05:15:03.970204+0000","last_peered":"2023-10-06T05:15:03.970204+0000","last_clean":"2023-10-06T05:15:03.970204+0000","last_became_active":"2023-10-06T05:14:59.966533+0000","last_became_peered":"2023-10-06T05:14:59.966533+0000","last_unstale":"2023-10-06T05:15:03.970204+0000","last_undegraded":"2023-10-06T05:15:03.970204+0000","last_fullsized":"2023-10-06T05:15:03.970204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960993+0000","last_change":"2023-10-06T05:14:59.949784+0000","last_active":"2023-10-06T05:15:03.960993+0000","last_peered":"2023-10-06T05:15:03.960993+0000","last_clean":"2023-10-06T05:15:03.960993+0000","last_became_active":"2023-10-06T05:14:59.949540+0000","last_became_peered":"2023-10-06T05:14:59.949540+0000","last_unstale":"2023-10-06T05:15:03.960993+0000","last_undegraded":"2023-10-06T05:15:03.960993+0000","last_fullsized":"2023-10-06T05:15:03.960993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:55.128455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967686+0000","last_change":"2023-10-06T05:14:59.954874+0000","last_active":"2023-10-06T05:15:03.967686+0000","last_peered":"2023-10-06T05:15:03.967686+0000","last_clean":"2023-10-06T05:15:03.967686+0000","last_became_active":"2023-10-06T05:14:59.954596+0000","last_became_peered":"2023-10-06T05:14:59.954596+0000","last_unstale":"2023-10-06T05:15:03.967686+0000","last_undegraded":"2023-10-06T05:15:03.967686+0000","last_fullsized":"2023-10-06T05:15:03.967686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:12.617791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959901+0000","last_change":"2023-10-06T05:14:59.964700+0000","last_active":"2023-10-06T05:15:03.959901+0000","last_peered":"2023-10-06T05:15:03.959901+0000","last_clean":"2023-10-06T05:15:03.959901+0000","last_became_active":"2023-10-06T05:14:59.964122+0000","last_became_peered":"2023-10-06T05:14:59.964122+0000","last_unstale":"2023-10-06T05:15:03.959901+0000","last_undegraded":"2023-10-06T05:15:03.959901+0000","last_fullsized":"2023-10-06T05:15:03.959901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.174130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965374+0000","last_change":"2023-10-06T05:14:59.971088+0000","last_active":"2023-10-06T05:15:03.965374+0000","last_peered":"2023-10-06T05:15:03.965374+0000","last_clean":"2023-10-06T05:15:03.965374+0000","last_became_active":"2023-10-06T05:14:59.970833+0000","last_became_peered":"2023-10-06T05:14:59.970833+0000","last_unstale":"2023-10-06T05:15:03.965374+0000","last_undegraded":"2023-10-06T05:15:03.965374+0000","last_fullsized":"2023-10-06T05:15:03.965374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963287+0000","last_change":"2023-10-06T05:14:59.959301+0000","last_active":"2023-10-06T05:15:03.963287+0000","last_peered":"2023-10-06T05:15:03.963287+0000","last_clean":"2023-10-06T05:15:03.963287+0000","last_became_active":"2023-10-06T05:14:59.958848+0000","last_became_peered":"2023-10-06T05:14:59.958848+0000","last_unstale":"2023-10-06T05:15:03.963287+0000","last_undegraded":"2023-10-06T05:15:03.963287+0000","last_fullsized":"2023-10-06T05:15:03.963287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963428+0000","last_change":"2023-10-06T05:14:59.964849+0000","last_active":"2023-10-06T05:15:03.963428+0000","last_peered":"2023-10-06T05:15:03.963428+0000","last_clean":"2023-10-06T05:15:03.963428+0000","last_became_active":"2023-10-06T05:14:59.964525+0000","last_became_peered":"2023-10-06T05:14:59.964525+0000","last_unstale":"2023-10-06T05:15:03.963428+0000","last_undegraded":"2023-10-06T05:15:03.963428+0000","last_fullsized":"2023-10-06T05:15:03.963428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964605+0000","last_change":"2023-10-06T05:14:59.968524+0000","last_active":"2023-10-06T05:15:03.964605+0000","last_peered":"2023-10-06T05:15:03.964605+0000","last_clean":"2023-10-06T05:15:03.964605+0000","last_became_active":"2023-10-06T05:14:59.968292+0000","last_became_peered":"2023-10-06T05:14:59.968292+0000","last_unstale":"2023-10-06T05:15:03.964605+0000","last_undegraded":"2023-10-06T05:15:03.964605+0000","last_fullsized":"2023-10-06T05:15:03.964605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979056+0000","last_change":"2023-10-06T05:14:59.957743+0000","last_active":"2023-10-06T05:15:03.979056+0000","last_peered":"2023-10-06T05:15:03.979056+0000","last_clean":"2023-10-06T05:15:03.979056+0000","last_became_active":"2023-10-06T05:14:59.957472+0000","last_became_peered":"2023-10-06T05:14:59.957472+0000","last_unstale":"2023-10-06T05:15:03.979056+0000","last_undegraded":"2023-10-06T05:15:03.979056+0000","last_fullsized":"2023-10-06T05:15:03.979056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977658+0000","last_change":"2023-10-06T05:14:59.953807+0000","last_active":"2023-10-06T05:15:03.977658+0000","last_peered":"2023-10-06T05:15:03.977658+0000","last_clean":"2023-10-06T05:15:03.977658+0000","last_became_active":"2023-10-06T05:14:59.953532+0000","last_became_peered":"2023-10-06T05:14:59.953532+0000","last_unstale":"2023-10-06T05:15:03.977658+0000","last_undegraded":"2023-10-06T05:15:03.977658+0000","last_fullsized":"2023-10-06T05:15:03.977658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:39.399842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962861+0000","last_change":"2023-10-06T05:14:59.968101+0000","last_active":"2023-10-06T05:15:03.962861+0000","last_peered":"2023-10-06T05:15:03.962861+0000","last_clean":"2023-10-06T05:15:03.962861+0000","last_became_active":"2023-10-06T05:14:59.967317+0000","last_became_peered":"2023-10-06T05:14:59.967317+0000","last_unstale":"2023-10-06T05:15:03.962861+0000","last_undegraded":"2023-10-06T05:15:03.962861+0000","last_fullsized":"2023-10-06T05:15:03.962861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987821+0000","last_change":"2023-10-06T05:15:03.983811+0000","last_active":"2023-10-06T05:15:03.987821+0000","last_peered":"2023-10-06T05:15:03.987821+0000","last_clean":"2023-10-06T05:15:03.987821+0000","last_became_active":"2023-10-06T05:15:03.983468+0000","last_became_peered":"2023-10-06T05:15:03.983468+0000","last_unstale":"2023-10-06T05:15:03.987821+0000","last_undegraded":"2023-10-06T05:15:03.987821+0000","last_fullsized":"2023-10-06T05:15:03.987821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:02.315455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961143+0000","last_change":"2023-10-06T05:14:58.958395+0000","last_active":"2023-10-06T05:15:03.961143+0000","last_peered":"2023-10-06T05:15:03.961143+0000","last_clean":"2023-10-06T05:15:03.961143+0000","last_became_active":"2023-10-06T05:14:58.957979+0000","last_became_peered":"2023-10-06T05:14:58.957979+0000","last_unstale":"2023-10-06T05:15:03.961143+0000","last_undegraded":"2023-10-06T05:15:03.961143+0000","last_fullsized":"2023-10-06T05:15:03.961143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965650+0000","last_change":"2023-10-06T05:14:59.960654+0000","last_active":"2023-10-06T05:15:03.965650+0000","last_peered":"2023-10-06T05:15:03.965650+0000","last_clean":"2023-10-06T05:15:03.965650+0000","last_became_active":"2023-10-06T05:14:59.960359+0000","last_became_peered":"2023-10-06T05:14:59.960359+0000","last_unstale":"2023-10-06T05:15:03.965650+0000","last_undegraded":"2023-10-06T05:15:03.965650+0000","last_fullsized":"2023-10-06T05:15:03.965650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987166+0000","last_change":"2023-10-06T05:15:03.980797+0000","last_active":"2023-10-06T05:15:03.987166+0000","last_peered":"2023-10-06T05:15:03.987166+0000","last_clean":"2023-10-06T05:15:03.987166+0000","last_became_active":"2023-10-06T05:15:03.980441+0000","last_became_peered":"2023-10-06T05:15:03.980441+0000","last_unstale":"2023-10-06T05:15:03.987166+0000","last_undegraded":"2023-10-06T05:15:03.987166+0000","last_fullsized":"2023-10-06T05:15:03.987166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:59.153783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972385+0000","last_change":"2023-10-06T05:14:58.967470+0000","last_active":"2023-10-06T05:15:03.972385+0000","last_peered":"2023-10-06T05:15:03.972385+0000","last_clean":"2023-10-06T05:15:03.972385+0000","last_became_active":"2023-10-06T05:14:58.967143+0000","last_became_peered":"2023-10-06T05:14:58.967143+0000","last_unstale":"2023-10-06T05:15:03.972385+0000","last_undegraded":"2023-10-06T05:15:03.972385+0000","last_fullsized":"2023-10-06T05:15:03.972385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970567+0000","last_change":"2023-10-06T05:14:59.961217+0000","last_active":"2023-10-06T05:15:03.970567+0000","last_peered":"2023-10-06T05:15:03.970567+0000","last_clean":"2023-10-06T05:15:03.970567+0000","last_became_active":"2023-10-06T05:14:59.960909+0000","last_became_peered":"2023-10-06T05:14:59.960909+0000","last_unstale":"2023-10-06T05:15:03.970567+0000","last_undegraded":"2023-10-06T05:15:03.970567+0000","last_fullsized":"2023-10-06T05:15:03.970567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995384+0000","last_change":"2023-10-06T05:15:03.993702+0000","last_active":"2023-10-06T05:15:03.995384+0000","last_peered":"2023-10-06T05:15:03.995384+0000","last_clean":"2023-10-06T05:15:03.995384+0000","last_became_active":"2023-10-06T05:15:03.993170+0000","last_became_peered":"2023-10-06T05:15:03.993170+0000","last_unstale":"2023-10-06T05:15:03.995384+0000","last_undegraded":"2023-10-06T05:15:03.995384+0000","last_fullsized":"2023-10-06T05:15:03.995384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964634+0000","last_change":"2023-10-06T05:14:58.964287+0000","last_active":"2023-10-06T05:15:03.964634+0000","last_peered":"2023-10-06T05:15:03.964634+0000","last_clean":"2023-10-06T05:15:03.964634+0000","last_became_active":"2023-10-06T05:14:58.963981+0000","last_became_peered":"2023-10-06T05:14:58.963981+0000","last_unstale":"2023-10-06T05:15:03.964634+0000","last_undegraded":"2023-10-06T05:15:03.964634+0000","last_fullsized":"2023-10-06T05:15:03.964634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975859+0000","last_change":"2023-10-06T05:14:59.964080+0000","last_active":"2023-10-06T05:15:03.975859+0000","last_peered":"2023-10-06T05:15:03.975859+0000","last_clean":"2023-10-06T05:15:03.975859+0000","last_became_active":"2023-10-06T05:14:59.963849+0000","last_became_peered":"2023-10-06T05:14:59.963849+0000","last_unstale":"2023-10-06T05:15:03.975859+0000","last_undegraded":"2023-10-06T05:15:03.975859+0000","last_fullsized":"2023-10-06T05:15:03.975859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:13.702454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.988935+0000","last_change":"2023-10-06T05:15:03.985094+0000","last_active":"2023-10-06T05:15:03.988935+0000","last_peered":"2023-10-06T05:15:03.988935+0000","last_clean":"2023-10-06T05:15:03.988935+0000","last_became_active":"2023-10-06T05:15:03.984516+0000","last_became_peered":"2023-10-06T05:15:03.984516+0000","last_unstale":"2023-10-06T05:15:03.988935+0000","last_undegraded":"2023-10-06T05:15:03.988935+0000","last_fullsized":"2023-10-06T05:15:03.988935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:09.008217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970313+0000","last_change":"2023-10-06T05:14:58.970639+0000","last_active":"2023-10-06T05:15:03.970313+0000","last_peered":"2023-10-06T05:15:03.970313+0000","last_clean":"2023-10-06T05:15:03.970313+0000","last_became_active":"2023-10-06T05:14:58.970183+0000","last_became_peered":"2023-10-06T05:14:58.970183+0000","last_unstale":"2023-10-06T05:15:03.970313+0000","last_undegraded":"2023-10-06T05:15:03.970313+0000","last_fullsized":"2023-10-06T05:15:03.970313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964542+0000","last_change":"2023-10-06T05:14:59.962878+0000","last_active":"2023-10-06T05:15:03.964542+0000","last_peered":"2023-10-06T05:15:03.964542+0000","last_clean":"2023-10-06T05:15:03.964542+0000","last_became_active":"2023-10-06T05:14:59.962557+0000","last_became_peered":"2023-10-06T05:14:59.962557+0000","last_unstale":"2023-10-06T05:15:03.964542+0000","last_undegraded":"2023-10-06T05:15:03.964542+0000","last_fullsized":"2023-10-06T05:15:03.964542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.984338+0000","last_change":"2023-10-06T05:15:03.979469+0000","last_active":"2023-10-06T05:15:03.984338+0000","last_peered":"2023-10-06T05:15:03.984338+0000","last_clean":"2023-10-06T05:15:03.984338+0000","last_became_active":"2023-10-06T05:15:03.979016+0000","last_became_peered":"2023-10-06T05:15:03.979016+0000","last_unstale":"2023-10-06T05:15:03.984338+0000","last_undegraded":"2023-10-06T05:15:03.984338+0000","last_fullsized":"2023-10-06T05:15:03.984338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963303+0000","last_change":"2023-10-06T05:14:58.953556+0000","last_active":"2023-10-06T05:15:03.963303+0000","last_peered":"2023-10-06T05:15:03.963303+0000","last_clean":"2023-10-06T05:15:03.963303+0000","last_became_active":"2023-10-06T05:14:58.953280+0000","last_became_peered":"2023-10-06T05:14:58.953280+0000","last_unstale":"2023-10-06T05:15:03.963303+0000","last_undegraded":"2023-10-06T05:15:03.963303+0000","last_fullsized":"2023-10-06T05:15:03.963303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963541+0000","last_change":"2023-10-06T05:14:59.969909+0000","last_active":"2023-10-06T05:15:03.963541+0000","last_peered":"2023-10-06T05:15:03.963541+0000","last_clean":"2023-10-06T05:15:03.963541+0000","last_became_active":"2023-10-06T05:14:59.967612+0000","last_became_peered":"2023-10-06T05:14:59.967612+0000","last_unstale":"2023-10-06T05:15:03.963541+0000","last_undegraded":"2023-10-06T05:15:03.963541+0000","last_fullsized":"2023-10-06T05:15:03.963541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994156+0000","last_change":"2023-10-06T05:15:03.984187+0000","last_active":"2023-10-06T05:15:03.994156+0000","last_peered":"2023-10-06T05:15:03.994156+0000","last_clean":"2023-10-06T05:15:03.994156+0000","last_became_active":"2023-10-06T05:15:03.983778+0000","last_became_peered":"2023-10-06T05:15:03.983778+0000","last_unstale":"2023-10-06T05:15:03.994156+0000","last_undegraded":"2023-10-06T05:15:03.994156+0000","last_fullsized":"2023-10-06T05:15:03.994156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:24.403669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967440+0000","last_change":"2023-10-06T05:14:58.971420+0000","last_active":"2023-10-06T05:15:03.967440+0000","last_peered":"2023-10-06T05:15:03.967440+0000","last_clean":"2023-10-06T05:15:03.967440+0000","last_became_active":"2023-10-06T05:14:58.971181+0000","last_became_peered":"2023-10-06T05:14:58.971181+0000","last_unstale":"2023-10-06T05:15:03.967440+0000","last_undegraded":"2023-10-06T05:15:03.967440+0000","last_fullsized":"2023-10-06T05:15:03.967440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973790+0000","last_change":"2023-10-06T05:14:59.966072+0000","last_active":"2023-10-06T05:15:03.973790+0000","last_peered":"2023-10-06T05:15:03.973790+0000","last_clean":"2023-10-06T05:15:03.973790+0000","last_became_active":"2023-10-06T05:14:59.965368+0000","last_became_peered":"2023-10-06T05:14:59.965368+0000","last_unstale":"2023-10-06T05:15:03.973790+0000","last_undegraded":"2023-10-06T05:15:03.973790+0000","last_fullsized":"2023-10-06T05:15:03.973790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990856+0000","last_change":"2023-10-06T05:15:03.989524+0000","last_active":"2023-10-06T05:15:03.990856+0000","last_peered":"2023-10-06T05:15:03.990856+0000","last_clean":"2023-10-06T05:15:03.990856+0000","last_became_active":"2023-10-06T05:15:03.982518+0000","last_became_peered":"2023-10-06T05:15:03.982518+0000","last_unstale":"2023-10-06T05:15:03.990856+0000","last_undegraded":"2023-10-06T05:15:03.990856+0000","last_fullsized":"2023-10-06T05:15:03.990856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'9","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-06T05:15:10.388156+0000","last_change":"2023-10-06T05:14:58.956195+0000","last_active":"2023-10-06T05:15:10.388156+0000","last_peered":"2023-10-06T05:15:10.388156+0000","last_clean":"2023-10-06T05:15:10.388156+0000","last_became_active":"2023-10-06T05:14:58.955906+0000","last_became_peered":"2023-10-06T05:14:58.955906+0000","last_unstale":"2023-10-06T05:15:10.388156+0000","last_undegraded":"2023-10-06T05:15:10.388156+0000","last_fullsized":"2023-10-06T05:15:10.388156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.952535+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.952313+0000","last_became_peered":"2023-10-06T05:14:59.952313+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987354+0000","last_change":"2023-10-06T05:15:03.982183+0000","last_active":"2023-10-06T05:15:03.987354+0000","last_peered":"2023-10-06T05:15:03.987354+0000","last_clean":"2023-10-06T05:15:03.987354+0000","last_became_active":"2023-10-06T05:15:03.981849+0000","last_became_peered":"2023-10-06T05:15:03.981849+0000","last_unstale":"2023-10-06T05:15:03.987354+0000","last_undegraded":"2023-10-06T05:15:03.987354+0000","last_fullsized":"2023-10-06T05:15:03.987354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965015+0000","last_change":"2023-10-06T05:14:58.959548+0000","last_active":"2023-10-06T05:15:03.965015+0000","last_peered":"2023-10-06T05:15:03.965015+0000","last_clean":"2023-10-06T05:15:03.965015+0000","last_became_active":"2023-10-06T05:14:58.959270+0000","last_became_peered":"2023-10-06T05:14:58.959270+0000","last_unstale":"2023-10-06T05:15:03.965015+0000","last_undegraded":"2023-10-06T05:15:03.965015+0000","last_fullsized":"2023-10-06T05:15:03.965015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:02.018618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968411+0000","last_change":"2023-10-06T05:14:59.973960+0000","last_active":"2023-10-06T05:15:03.968411+0000","last_peered":"2023-10-06T05:15:03.968411+0000","last_clean":"2023-10-06T05:15:03.968411+0000","last_became_active":"2023-10-06T05:14:59.973622+0000","last_became_peered":"2023-10-06T05:14:59.973622+0000","last_unstale":"2023-10-06T05:15:03.968411+0000","last_undegraded":"2023-10-06T05:15:03.968411+0000","last_fullsized":"2023-10-06T05:15:03.968411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994166+0000","last_change":"2023-10-06T05:15:03.985900+0000","last_active":"2023-10-06T05:15:03.994166+0000","last_peered":"2023-10-06T05:15:03.994166+0000","last_clean":"2023-10-06T05:15:03.994166+0000","last_became_active":"2023-10-06T05:15:03.985484+0000","last_became_peered":"2023-10-06T05:15:03.985484+0000","last_unstale":"2023-10-06T05:15:03.994166+0000","last_undegraded":"2023-10-06T05:15:03.994166+0000","last_fullsized":"2023-10-06T05:15:03.994166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:50.069708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963167+0000","last_change":"2023-10-06T05:14:58.968185+0000","last_active":"2023-10-06T05:15:03.963167+0000","last_peered":"2023-10-06T05:15:03.963167+0000","last_clean":"2023-10-06T05:15:03.963167+0000","last_became_active":"2023-10-06T05:14:58.967895+0000","last_became_peered":"2023-10-06T05:14:58.967895+0000","last_unstale":"2023-10-06T05:15:03.963167+0000","last_undegraded":"2023-10-06T05:15:03.963167+0000","last_fullsized":"2023-10-06T05:15:03.963167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962815+0000","last_change":"2023-10-06T05:14:59.967384+0000","last_active":"2023-10-06T05:15:03.962815+0000","last_peered":"2023-10-06T05:15:03.962815+0000","last_clean":"2023-10-06T05:15:03.962815+0000","last_became_active":"2023-10-06T05:14:59.967131+0000","last_became_peered":"2023-10-06T05:14:59.967131+0000","last_unstale":"2023-10-06T05:15:03.962815+0000","last_undegraded":"2023-10-06T05:15:03.962815+0000","last_fullsized":"2023-10-06T05:15:03.962815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994086+0000","last_change":"2023-10-06T05:15:03.985192+0000","last_active":"2023-10-06T05:15:03.994086+0000","last_peered":"2023-10-06T05:15:03.994086+0000","last_clean":"2023-10-06T05:15:03.994086+0000","last_became_active":"2023-10-06T05:15:03.984600+0000","last_became_peered":"2023-10-06T05:15:03.984600+0000","last_unstale":"2023-10-06T05:15:03.994086+0000","last_undegraded":"2023-10-06T05:15:03.994086+0000","last_fullsized":"2023-10-06T05:15:03.994086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964960+0000","last_change":"2023-10-06T05:14:58.959794+0000","last_active":"2023-10-06T05:15:03.964960+0000","last_peered":"2023-10-06T05:15:03.964960+0000","last_clean":"2023-10-06T05:15:03.964960+0000","last_became_active":"2023-10-06T05:14:58.959493+0000","last_became_peered":"2023-10-06T05:14:58.959493+0000","last_unstale":"2023-10-06T05:15:03.964960+0000","last_undegraded":"2023-10-06T05:15:03.964960+0000","last_fullsized":"2023-10-06T05:15:03.964960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:10.366367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970679+0000","last_change":"2023-10-06T05:14:59.964236+0000","last_active":"2023-10-06T05:15:03.970679+0000","last_peered":"2023-10-06T05:15:03.970679+0000","last_clean":"2023-10-06T05:15:03.970679+0000","last_became_active":"2023-10-06T05:14:59.963902+0000","last_became_peered":"2023-10-06T05:14:59.963902+0000","last_unstale":"2023-10-06T05:15:03.970679+0000","last_undegraded":"2023-10-06T05:15:03.970679+0000","last_fullsized":"2023-10-06T05:15:03.970679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:27.793444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987391+0000","last_change":"2023-10-06T05:15:03.982595+0000","last_active":"2023-10-06T05:15:03.987391+0000","last_peered":"2023-10-06T05:15:03.987391+0000","last_clean":"2023-10-06T05:15:03.987391+0000","last_became_active":"2023-10-06T05:15:03.982200+0000","last_became_peered":"2023-10-06T05:15:03.982200+0000","last_unstale":"2023-10-06T05:15:03.987391+0000","last_undegraded":"2023-10-06T05:15:03.987391+0000","last_fullsized":"2023-10-06T05:15:03.987391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:20.022766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974038+0000","last_change":"2023-10-06T05:14:58.961380+0000","last_active":"2023-10-06T05:15:03.974038+0000","last_peered":"2023-10-06T05:15:03.974038+0000","last_clean":"2023-10-06T05:15:03.974038+0000","last_became_active":"2023-10-06T05:14:58.961038+0000","last_became_peered":"2023-10-06T05:14:58.961038+0000","last_unstale":"2023-10-06T05:15:03.974038+0000","last_undegraded":"2023-10-06T05:15:03.974038+0000","last_fullsized":"2023-10-06T05:15:03.974038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964625+0000","last_change":"2023-10-06T05:14:59.967979+0000","last_active":"2023-10-06T05:15:03.964625+0000","last_peered":"2023-10-06T05:15:03.964625+0000","last_clean":"2023-10-06T05:15:03.964625+0000","last_became_active":"2023-10-06T05:14:59.964930+0000","last_became_peered":"2023-10-06T05:14:59.964930+0000","last_unstale":"2023-10-06T05:15:03.964625+0000","last_undegraded":"2023-10-06T05:15:03.964625+0000","last_fullsized":"2023-10-06T05:15:03.964625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:36.979941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.981489+0000","last_change":"2023-10-06T05:15:03.978144+0000","last_active":"2023-10-06T05:15:03.981489+0000","last_peered":"2023-10-06T05:15:03.981489+0000","last_clean":"2023-10-06T05:15:03.981489+0000","last_became_active":"2023-10-06T05:15:03.977784+0000","last_became_peered":"2023-10-06T05:15:03.977784+0000","last_unstale":"2023-10-06T05:15:03.981489+0000","last_undegraded":"2023-10-06T05:15:03.981489+0000","last_fullsized":"2023-10-06T05:15:03.981489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967621+0000","last_change":"2023-10-06T05:14:58.959617+0000","last_active":"2023-10-06T05:15:03.967621+0000","last_peered":"2023-10-06T05:15:03.967621+0000","last_clean":"2023-10-06T05:15:03.967621+0000","last_became_active":"2023-10-06T05:14:58.959350+0000","last_became_peered":"2023-10-06T05:14:58.959350+0000","last_unstale":"2023-10-06T05:15:03.967621+0000","last_undegraded":"2023-10-06T05:15:03.967621+0000","last_fullsized":"2023-10-06T05:15:03.967621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:12.695218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961420+0000","last_change":"2023-10-06T05:14:59.967832+0000","last_active":"2023-10-06T05:15:03.961420+0000","last_peered":"2023-10-06T05:15:03.961420+0000","last_clean":"2023-10-06T05:15:03.961420+0000","last_became_active":"2023-10-06T05:14:59.967192+0000","last_became_peered":"2023-10-06T05:14:59.967192+0000","last_unstale":"2023-10-06T05:15:03.961420+0000","last_undegraded":"2023-10-06T05:15:03.961420+0000","last_fullsized":"2023-10-06T05:15:03.961420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.995012+0000","last_change":"2023-10-06T05:15:03.993765+0000","last_active":"2023-10-06T05:15:03.995012+0000","last_peered":"2023-10-06T05:15:03.995012+0000","last_clean":"2023-10-06T05:15:03.995012+0000","last_became_active":"2023-10-06T05:15:03.993371+0000","last_became_peered":"2023-10-06T05:15:03.993371+0000","last_unstale":"2023-10-06T05:15:03.995012+0000","last_undegraded":"2023-10-06T05:15:03.995012+0000","last_fullsized":"2023-10-06T05:15:03.995012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:00.861115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967256+0000","last_change":"2023-10-06T05:14:58.962483+0000","last_active":"2023-10-06T05:15:03.967256+0000","last_peered":"2023-10-06T05:15:03.967256+0000","last_clean":"2023-10-06T05:15:03.967256+0000","last_became_active":"2023-10-06T05:14:58.962203+0000","last_became_peered":"2023-10-06T05:14:58.962203+0000","last_unstale":"2023-10-06T05:15:03.967256+0000","last_undegraded":"2023-10-06T05:15:03.967256+0000","last_fullsized":"2023-10-06T05:15:03.967256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:34.275033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960934+0000","last_change":"2023-10-06T05:14:59.956925+0000","last_active":"2023-10-06T05:15:03.960934+0000","last_peered":"2023-10-06T05:15:03.960934+0000","last_clean":"2023-10-06T05:15:03.960934+0000","last_became_active":"2023-10-06T05:14:59.956653+0000","last_became_peered":"2023-10-06T05:14:59.956653+0000","last_unstale":"2023-10-06T05:15:03.960934+0000","last_undegraded":"2023-10-06T05:15:03.960934+0000","last_fullsized":"2023-10-06T05:15:03.960934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985561+0000","last_change":"2023-10-06T05:15:03.983559+0000","last_active":"2023-10-06T05:15:03.985561+0000","last_peered":"2023-10-06T05:15:03.985561+0000","last_clean":"2023-10-06T05:15:03.985561+0000","last_became_active":"2023-10-06T05:15:03.983123+0000","last_became_peered":"2023-10-06T05:15:03.983123+0000","last_unstale":"2023-10-06T05:15:03.985561+0000","last_undegraded":"2023-10-06T05:15:03.985561+0000","last_fullsized":"2023-10-06T05:15:03.985561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:50.666262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967619+0000","last_change":"2023-10-06T05:14:58.956636+0000","last_active":"2023-10-06T05:15:03.967619+0000","last_peered":"2023-10-06T05:15:03.967619+0000","last_clean":"2023-10-06T05:15:03.967619+0000","last_became_active":"2023-10-06T05:14:58.956436+0000","last_became_peered":"2023-10-06T05:14:58.956436+0000","last_unstale":"2023-10-06T05:15:03.967619+0000","last_undegraded":"2023-10-06T05:15:03.967619+0000","last_fullsized":"2023-10-06T05:15:03.967619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:45.319345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975492+0000","last_change":"2023-10-06T05:14:59.962010+0000","last_active":"2023-10-06T05:15:03.975492+0000","last_peered":"2023-10-06T05:15:03.975492+0000","last_clean":"2023-10-06T05:15:03.975492+0000","last_became_active":"2023-10-06T05:14:59.961837+0000","last_became_peered":"2023-10-06T05:14:59.961837+0000","last_unstale":"2023-10-06T05:15:03.975492+0000","last_undegraded":"2023-10-06T05:15:03.975492+0000","last_fullsized":"2023-10-06T05:15:03.975492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:46.326581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985590+0000","last_change":"2023-10-06T05:15:03.983998+0000","last_active":"2023-10-06T05:15:03.985590+0000","last_peered":"2023-10-06T05:15:03.985590+0000","last_clean":"2023-10-06T05:15:03.985590+0000","last_became_active":"2023-10-06T05:15:03.983598+0000","last_became_peered":"2023-10-06T05:15:03.983598+0000","last_unstale":"2023-10-06T05:15:03.985590+0000","last_undegraded":"2023-10-06T05:15:03.985590+0000","last_fullsized":"2023-10-06T05:15:03.985590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:01.199458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961576+0000","last_change":"2023-10-06T05:14:58.969438+0000","last_active":"2023-10-06T05:15:03.961576+0000","last_peered":"2023-10-06T05:15:03.961576+0000","last_clean":"2023-10-06T05:15:03.961576+0000","last_became_active":"2023-10-06T05:14:58.969214+0000","last_became_peered":"2023-10-06T05:14:58.969214+0000","last_unstale":"2023-10-06T05:15:03.961576+0000","last_undegraded":"2023-10-06T05:15:03.961576+0000","last_fullsized":"2023-10-06T05:15:03.961576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966944+0000","last_change":"2023-10-06T05:14:59.958303+0000","last_active":"2023-10-06T05:15:03.966944+0000","last_peered":"2023-10-06T05:15:03.966944+0000","last_clean":"2023-10-06T05:15:03.966944+0000","last_became_active":"2023-10-06T05:14:59.957997+0000","last_became_peered":"2023-10-06T05:14:59.957997+0000","last_unstale":"2023-10-06T05:15:03.966944+0000","last_undegraded":"2023-10-06T05:15:03.966944+0000","last_fullsized":"2023-10-06T05:15:03.966944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.982560+0000","last_change":"2023-10-06T05:15:03.976634+0000","last_active":"2023-10-06T05:15:03.982560+0000","last_peered":"2023-10-06T05:15:03.982560+0000","last_clean":"2023-10-06T05:15:03.982560+0000","last_became_active":"2023-10-06T05:15:03.976257+0000","last_became_peered":"2023-10-06T05:15:03.976257+0000","last_unstale":"2023-10-06T05:15:03.982560+0000","last_undegraded":"2023-10-06T05:15:03.982560+0000","last_fullsized":"2023-10-06T05:15:03.982560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:47.906687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964042+0000","last_change":"2023-10-06T05:14:58.969408+0000","last_active":"2023-10-06T05:15:03.964042+0000","last_peered":"2023-10-06T05:15:03.964042+0000","last_clean":"2023-10-06T05:15:03.964042+0000","last_became_active":"2023-10-06T05:14:58.969070+0000","last_became_peered":"2023-10-06T05:14:58.969070+0000","last_unstale":"2023-10-06T05:15:03.964042+0000","last_undegraded":"2023-10-06T05:15:03.964042+0000","last_fullsized":"2023-10-06T05:15:03.964042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959678+0000","last_change":"2023-10-06T05:14:59.969503+0000","last_active":"2023-10-06T05:15:03.959678+0000","last_peered":"2023-10-06T05:15:03.959678+0000","last_clean":"2023-10-06T05:15:03.959678+0000","last_became_active":"2023-10-06T05:14:59.969109+0000","last_became_peered":"2023-10-06T05:14:59.969109+0000","last_unstale":"2023-10-06T05:15:03.959678+0000","last_undegraded":"2023-10-06T05:15:03.959678+0000","last_fullsized":"2023-10-06T05:15:03.959678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:59.890813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.989045+0000","last_change":"2023-10-06T05:15:03.982573+0000","last_active":"2023-10-06T05:15:03.989045+0000","last_peered":"2023-10-06T05:15:03.989045+0000","last_clean":"2023-10-06T05:15:03.989045+0000","last_became_active":"2023-10-06T05:15:03.982173+0000","last_became_peered":"2023-10-06T05:15:03.982173+0000","last_unstale":"2023-10-06T05:15:03.989045+0000","last_undegraded":"2023-10-06T05:15:03.989045+0000","last_fullsized":"2023-10-06T05:15:03.989045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:57.921378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975586+0000","last_change":"2023-10-06T05:14:58.962379+0000","last_active":"2023-10-06T05:15:03.975586+0000","last_peered":"2023-10-06T05:15:03.975586+0000","last_clean":"2023-10-06T05:15:03.975586+0000","last_became_active":"2023-10-06T05:14:58.962093+0000","last_became_peered":"2023-10-06T05:14:58.962093+0000","last_unstale":"2023-10-06T05:15:03.975586+0000","last_undegraded":"2023-10-06T05:15:03.975586+0000","last_fullsized":"2023-10-06T05:15:03.975586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:59.627806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960163+0000","last_change":"2023-10-06T05:14:59.951502+0000","last_active":"2023-10-06T05:15:03.960163+0000","last_peered":"2023-10-06T05:15:03.960163+0000","last_clean":"2023-10-06T05:15:03.960163+0000","last_became_active":"2023-10-06T05:14:59.951197+0000","last_became_peered":"2023-10-06T05:14:59.951197+0000","last_unstale":"2023-10-06T05:15:03.960163+0000","last_undegraded":"2023-10-06T05:15:03.960163+0000","last_fullsized":"2023-10-06T05:15:03.960163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:39.821738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978919+0000","last_change":"2023-10-06T05:15:03.977325+0000","last_active":"2023-10-06T05:15:03.978919+0000","last_peered":"2023-10-06T05:15:03.978919+0000","last_clean":"2023-10-06T05:15:03.978919+0000","last_became_active":"2023-10-06T05:15:03.976844+0000","last_became_peered":"2023-10-06T05:15:03.976844+0000","last_unstale":"2023-10-06T05:15:03.978919+0000","last_undegraded":"2023-10-06T05:15:03.978919+0000","last_fullsized":"2023-10-06T05:15:03.978919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:47.153239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960726+0000","last_change":"2023-10-06T05:14:58.958227+0000","last_active":"2023-10-06T05:15:03.960726+0000","last_peered":"2023-10-06T05:15:03.960726+0000","last_clean":"2023-10-06T05:15:03.960726+0000","last_became_active":"2023-10-06T05:14:58.957924+0000","last_became_peered":"2023-10-06T05:14:58.957924+0000","last_unstale":"2023-10-06T05:15:03.960726+0000","last_undegraded":"2023-10-06T05:15:03.960726+0000","last_fullsized":"2023-10-06T05:15:03.960726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961096+0000","last_change":"2023-10-06T05:14:59.961306+0000","last_active":"2023-10-06T05:15:03.961096+0000","last_peered":"2023-10-06T05:15:03.961096+0000","last_clean":"2023-10-06T05:15:03.961096+0000","last_became_active":"2023-10-06T05:14:59.961025+0000","last_became_peered":"2023-10-06T05:14:59.961025+0000","last_unstale":"2023-10-06T05:15:03.961096+0000","last_undegraded":"2023-10-06T05:15:03.961096+0000","last_fullsized":"2023-10-06T05:15:03.961096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987866+0000","last_change":"2023-10-06T05:15:03.983777+0000","last_active":"2023-10-06T05:15:03.987866+0000","last_peered":"2023-10-06T05:15:03.987866+0000","last_clean":"2023-10-06T05:15:03.987866+0000","last_became_active":"2023-10-06T05:15:03.983380+0000","last_became_peered":"2023-10-06T05:15:03.983380+0000","last_unstale":"2023-10-06T05:15:03.987866+0000","last_undegraded":"2023-10-06T05:15:03.987866+0000","last_fullsized":"2023-10-06T05:15:03.987866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:21.551659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967787+0000","last_change":"2023-10-06T05:14:58.966032+0000","last_active":"2023-10-06T05:15:03.967787+0000","last_peered":"2023-10-06T05:15:03.967787+0000","last_clean":"2023-10-06T05:15:03.967787+0000","last_became_active":"2023-10-06T05:14:58.965697+0000","last_became_peered":"2023-10-06T05:14:58.965697+0000","last_unstale":"2023-10-06T05:15:03.967787+0000","last_undegraded":"2023-10-06T05:15:03.967787+0000","last_fullsized":"2023-10-06T05:15:03.967787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979242+0000","last_change":"2023-10-06T05:14:59.966425+0000","last_active":"2023-10-06T05:15:03.979242+0000","last_peered":"2023-10-06T05:15:03.979242+0000","last_clean":"2023-10-06T05:15:03.979242+0000","last_became_active":"2023-10-06T05:14:59.966122+0000","last_became_peered":"2023-10-06T05:14:59.966122+0000","last_unstale":"2023-10-06T05:15:03.979242+0000","last_undegraded":"2023-10-06T05:15:03.979242+0000","last_fullsized":"2023-10-06T05:15:03.979242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.983132+0000","last_change":"2023-10-06T05:15:03.979017+0000","last_active":"2023-10-06T05:15:03.983132+0000","last_peered":"2023-10-06T05:15:03.983132+0000","last_clean":"2023-10-06T05:15:03.983132+0000","last_became_active":"2023-10-06T05:15:03.978644+0000","last_became_peered":"2023-10-06T05:15:03.978644+0000","last_unstale":"2023-10-06T05:15:03.983132+0000","last_undegraded":"2023-10-06T05:15:03.983132+0000","last_fullsized":"2023-10-06T05:15:03.983132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973620+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:15:03.973620+0000","last_peered":"2023-10-06T05:15:03.973620+0000","last_clean":"2023-10-06T05:15:03.973620+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:15:03.973620+0000","last_undegraded":"2023-10-06T05:15:03.973620+0000","last_fullsized":"2023-10-06T05:15:03.973620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962773+0000","last_change":"2023-10-06T05:14:58.969546+0000","last_active":"2023-10-06T05:15:03.962773+0000","last_peered":"2023-10-06T05:15:03.962773+0000","last_clean":"2023-10-06T05:15:03.962773+0000","last_became_active":"2023-10-06T05:14:58.969281+0000","last_became_peered":"2023-10-06T05:14:58.969281+0000","last_unstale":"2023-10-06T05:15:03.962773+0000","last_undegraded":"2023-10-06T05:15:03.962773+0000","last_fullsized":"2023-10-06T05:15:03.962773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973704+0000","last_change":"2023-10-06T05:14:59.965859+0000","last_active":"2023-10-06T05:15:03.973704+0000","last_peered":"2023-10-06T05:15:03.973704+0000","last_clean":"2023-10-06T05:15:03.973704+0000","last_became_active":"2023-10-06T05:14:59.964784+0000","last_became_peered":"2023-10-06T05:14:59.964784+0000","last_unstale":"2023-10-06T05:15:03.973704+0000","last_undegraded":"2023-10-06T05:15:03.973704+0000","last_fullsized":"2023-10-06T05:15:03.973704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.985105+0000","last_change":"2023-10-06T05:15:03.982375+0000","last_active":"2023-10-06T05:15:03.985105+0000","last_peered":"2023-10-06T05:15:03.985105+0000","last_clean":"2023-10-06T05:15:03.985105+0000","last_became_active":"2023-10-06T05:15:03.981983+0000","last_became_peered":"2023-10-06T05:15:03.981983+0000","last_unstale":"2023-10-06T05:15:03.985105+0000","last_undegraded":"2023-10-06T05:15:03.985105+0000","last_fullsized":"2023-10-06T05:15:03.985105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:31.485623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962357+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:15:03.962357+0000","last_peered":"2023-10-06T05:15:03.962357+0000","last_clean":"2023-10-06T05:15:03.962357+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:15:03.962357+0000","last_undegraded":"2023-10-06T05:15:03.962357+0000","last_fullsized":"2023-10-06T05:15:03.962357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.959855+0000","last_change":"2023-10-06T05:14:58.966374+0000","last_active":"2023-10-06T05:15:03.959855+0000","last_peered":"2023-10-06T05:15:03.959855+0000","last_clean":"2023-10-06T05:15:03.959855+0000","last_became_active":"2023-10-06T05:14:58.966124+0000","last_became_peered":"2023-10-06T05:14:58.966124+0000","last_unstale":"2023-10-06T05:15:03.959855+0000","last_undegraded":"2023-10-06T05:15:03.959855+0000","last_fullsized":"2023-10-06T05:15:03.959855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973548+0000","last_change":"2023-10-06T05:14:59.953791+0000","last_active":"2023-10-06T05:15:03.973548+0000","last_peered":"2023-10-06T05:15:03.973548+0000","last_clean":"2023-10-06T05:15:03.973548+0000","last_became_active":"2023-10-06T05:14:59.953491+0000","last_became_peered":"2023-10-06T05:14:59.953491+0000","last_unstale":"2023-10-06T05:15:03.973548+0000","last_undegraded":"2023-10-06T05:15:03.973548+0000","last_fullsized":"2023-10-06T05:15:03.973548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:32.203112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994360+0000","last_change":"2023-10-06T05:15:03.993204+0000","last_active":"2023-10-06T05:15:03.994360+0000","last_peered":"2023-10-06T05:15:03.994360+0000","last_clean":"2023-10-06T05:15:03.994360+0000","last_became_active":"2023-10-06T05:15:03.992794+0000","last_became_peered":"2023-10-06T05:15:03.992794+0000","last_unstale":"2023-10-06T05:15:03.994360+0000","last_undegraded":"2023-10-06T05:15:03.994360+0000","last_fullsized":"2023-10-06T05:15:03.994360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:53.385235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967975+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:15:03.967975+0000","last_peered":"2023-10-06T05:15:03.967975+0000","last_clean":"2023-10-06T05:15:03.967975+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:15:03.967975+0000","last_undegraded":"2023-10-06T05:15:03.967975+0000","last_fullsized":"2023-10-06T05:15:03.967975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.972830+0000","last_change":"2023-10-06T05:14:58.954417+0000","last_active":"2023-10-06T05:15:03.972830+0000","last_peered":"2023-10-06T05:15:03.972830+0000","last_clean":"2023-10-06T05:15:03.972830+0000","last_became_active":"2023-10-06T05:14:58.954052+0000","last_became_peered":"2023-10-06T05:14:58.954052+0000","last_unstale":"2023-10-06T05:15:03.972830+0000","last_undegraded":"2023-10-06T05:15:03.972830+0000","last_fullsized":"2023-10-06T05:15:03.972830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.979150+0000","last_change":"2023-10-06T05:14:59.963695+0000","last_active":"2023-10-06T05:15:03.979150+0000","last_peered":"2023-10-06T05:15:03.979150+0000","last_clean":"2023-10-06T05:15:03.979150+0000","last_became_active":"2023-10-06T05:14:59.963426+0000","last_became_peered":"2023-10-06T05:14:59.963426+0000","last_unstale":"2023-10-06T05:15:03.979150+0000","last_undegraded":"2023-10-06T05:15:03.979150+0000","last_fullsized":"2023-10-06T05:15:03.979150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:00.456603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987182+0000","last_change":"2023-10-06T05:15:03.985907+0000","last_active":"2023-10-06T05:15:03.987182+0000","last_peered":"2023-10-06T05:15:03.987182+0000","last_clean":"2023-10-06T05:15:03.987182+0000","last_became_active":"2023-10-06T05:15:03.985534+0000","last_became_peered":"2023-10-06T05:15:03.985534+0000","last_unstale":"2023-10-06T05:15:03.987182+0000","last_undegraded":"2023-10-06T05:15:03.987182+0000","last_fullsized":"2023-10-06T05:15:03.987182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961919+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:15:03.961919+0000","last_peered":"2023-10-06T05:15:03.961919+0000","last_clean":"2023-10-06T05:15:03.961919+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:15:03.961919+0000","last_undegraded":"2023-10-06T05:15:03.961919+0000","last_fullsized":"2023-10-06T05:15:03.961919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964512+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:15:03.964512+0000","last_peered":"2023-10-06T05:15:03.964512+0000","last_clean":"2023-10-06T05:15:03.964512+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:15:03.964512+0000","last_undegraded":"2023-10-06T05:15:03.964512+0000","last_fullsized":"2023-10-06T05:15:03.964512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963731+0000","last_change":"2023-10-06T05:14:58.965199+0000","last_active":"2023-10-06T05:15:03.963731+0000","last_peered":"2023-10-06T05:15:03.963731+0000","last_clean":"2023-10-06T05:15:03.963731+0000","last_became_active":"2023-10-06T05:14:58.964902+0000","last_became_peered":"2023-10-06T05:14:58.964902+0000","last_unstale":"2023-10-06T05:15:03.963731+0000","last_undegraded":"2023-10-06T05:15:03.963731+0000","last_fullsized":"2023-10-06T05:15:03.963731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:46.730905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975967+0000","last_change":"2023-10-06T05:14:59.953471+0000","last_active":"2023-10-06T05:15:03.975967+0000","last_peered":"2023-10-06T05:15:03.975967+0000","last_clean":"2023-10-06T05:15:03.975967+0000","last_became_active":"2023-10-06T05:14:59.953166+0000","last_became_peered":"2023-10-06T05:14:59.953166+0000","last_unstale":"2023-10-06T05:15:03.975967+0000","last_undegraded":"2023-10-06T05:15:03.975967+0000","last_fullsized":"2023-10-06T05:15:03.975967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:35.282268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987431+0000","last_change":"2023-10-06T05:15:03.982741+0000","last_active":"2023-10-06T05:15:03.987431+0000","last_peered":"2023-10-06T05:15:03.987431+0000","last_clean":"2023-10-06T05:15:03.987431+0000","last_became_active":"2023-10-06T05:15:03.982333+0000","last_became_peered":"2023-10-06T05:15:03.982333+0000","last_unstale":"2023-10-06T05:15:03.987431+0000","last_undegraded":"2023-10-06T05:15:03.987431+0000","last_fullsized":"2023-10-06T05:15:03.987431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:07.926945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964687+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:15:03.964687+0000","last_peered":"2023-10-06T05:15:03.964687+0000","last_clean":"2023-10-06T05:15:03.964687+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:15:03.964687+0000","last_undegraded":"2023-10-06T05:15:03.964687+0000","last_fullsized":"2023-10-06T05:15:03.964687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.965134+0000","last_change":"2023-10-06T05:14:58.970767+0000","last_active":"2023-10-06T05:15:03.965134+0000","last_peered":"2023-10-06T05:15:03.965134+0000","last_clean":"2023-10-06T05:15:03.965134+0000","last_became_active":"2023-10-06T05:14:58.970523+0000","last_became_peered":"2023-10-06T05:14:58.970523+0000","last_unstale":"2023-10-06T05:15:03.965134+0000","last_undegraded":"2023-10-06T05:15:03.965134+0000","last_fullsized":"2023-10-06T05:15:03.965134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:11.610555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970560+0000","last_change":"2023-10-06T05:14:59.958130+0000","last_active":"2023-10-06T05:15:03.970560+0000","last_peered":"2023-10-06T05:15:03.970560+0000","last_clean":"2023-10-06T05:15:03.970560+0000","last_became_active":"2023-10-06T05:14:59.957826+0000","last_became_peered":"2023-10-06T05:14:59.957826+0000","last_unstale":"2023-10-06T05:15:03.970560+0000","last_undegraded":"2023-10-06T05:15:03.970560+0000","last_fullsized":"2023-10-06T05:15:03.970560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:21.677280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987228+0000","last_change":"2023-10-06T05:15:03.981078+0000","last_active":"2023-10-06T05:15:03.987228+0000","last_peered":"2023-10-06T05:15:03.987228+0000","last_clean":"2023-10-06T05:15:03.987228+0000","last_became_active":"2023-10-06T05:15:03.980645+0000","last_became_peered":"2023-10-06T05:15:03.980645+0000","last_unstale":"2023-10-06T05:15:03.987228+0000","last_undegraded":"2023-10-06T05:15:03.987228+0000","last_fullsized":"2023-10-06T05:15:03.987228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.847066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967525+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:15:03.967525+0000","last_peered":"2023-10-06T05:15:03.967525+0000","last_clean":"2023-10-06T05:15:03.967525+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:15:03.967525+0000","last_undegraded":"2023-10-06T05:15:03.967525+0000","last_fullsized":"2023-10-06T05:15:03.967525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973787+0000","last_change":"2023-10-06T05:14:58.954390+0000","last_active":"2023-10-06T05:15:03.973787+0000","last_peered":"2023-10-06T05:15:03.973787+0000","last_clean":"2023-10-06T05:15:03.973787+0000","last_became_active":"2023-10-06T05:14:58.954003+0000","last_became_peered":"2023-10-06T05:14:58.954003+0000","last_unstale":"2023-10-06T05:15:03.973787+0000","last_undegraded":"2023-10-06T05:15:03.973787+0000","last_fullsized":"2023-10-06T05:15:03.973787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967163+0000","last_change":"2023-10-06T05:14:59.965298+0000","last_active":"2023-10-06T05:15:03.967163+0000","last_peered":"2023-10-06T05:15:03.967163+0000","last_clean":"2023-10-06T05:15:03.967163+0000","last_became_active":"2023-10-06T05:14:59.964967+0000","last_became_peered":"2023-10-06T05:14:59.964967+0000","last_unstale":"2023-10-06T05:15:03.967163+0000","last_undegraded":"2023-10-06T05:15:03.967163+0000","last_fullsized":"2023-10-06T05:15:03.967163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.636286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978732+0000","last_change":"2023-10-06T05:15:03.976855+0000","last_active":"2023-10-06T05:15:03.978732+0000","last_peered":"2023-10-06T05:15:03.978732+0000","last_clean":"2023-10-06T05:15:03.978732+0000","last_became_active":"2023-10-06T05:15:03.976461+0000","last_became_peered":"2023-10-06T05:15:03.976461+0000","last_unstale":"2023-10-06T05:15:03.978732+0000","last_undegraded":"2023-10-06T05:15:03.978732+0000","last_fullsized":"2023-10-06T05:15:03.978732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:45.979196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964732+0000","last_change":"2023-10-06T05:14:58.970171+0000","last_active":"2023-10-06T05:15:03.964732+0000","last_peered":"2023-10-06T05:15:03.964732+0000","last_clean":"2023-10-06T05:15:03.964732+0000","last_became_active":"2023-10-06T05:14:58.969914+0000","last_became_peered":"2023-10-06T05:14:58.969914+0000","last_unstale":"2023-10-06T05:15:03.964732+0000","last_undegraded":"2023-10-06T05:15:03.964732+0000","last_fullsized":"2023-10-06T05:15:03.964732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:01.980075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977482+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:15:03.977482+0000","last_peered":"2023-10-06T05:15:03.977482+0000","last_clean":"2023-10-06T05:15:03.977482+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:15:03.977482+0000","last_undegraded":"2023-10-06T05:15:03.977482+0000","last_fullsized":"2023-10-06T05:15:03.977482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.964253+0000","last_change":"2023-10-06T05:14:59.964377+0000","last_active":"2023-10-06T05:15:03.964253+0000","last_peered":"2023-10-06T05:15:03.964253+0000","last_clean":"2023-10-06T05:15:03.964253+0000","last_became_active":"2023-10-06T05:14:59.964058+0000","last_became_peered":"2023-10-06T05:14:59.964058+0000","last_unstale":"2023-10-06T05:15:03.964253+0000","last_undegraded":"2023-10-06T05:15:03.964253+0000","last_fullsized":"2023-10-06T05:15:03.964253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.801516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.994304+0000","last_change":"2023-10-06T05:15:03.991574+0000","last_active":"2023-10-06T05:15:03.994304+0000","last_peered":"2023-10-06T05:15:03.994304+0000","last_clean":"2023-10-06T05:15:03.994304+0000","last_became_active":"2023-10-06T05:15:03.991171+0000","last_became_peered":"2023-10-06T05:15:03.991171+0000","last_unstale":"2023-10-06T05:15:03.994304+0000","last_undegraded":"2023-10-06T05:15:03.994304+0000","last_fullsized":"2023-10-06T05:15:03.994304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.156992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.975779+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:15:03.975779+0000","last_peered":"2023-10-06T05:15:03.975779+0000","last_clean":"2023-10-06T05:15:03.975779+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:15:03.975779+0000","last_undegraded":"2023-10-06T05:15:03.975779+0000","last_fullsized":"2023-10-06T05:15:03.975779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970480+0000","last_change":"2023-10-06T05:14:58.966696+0000","last_active":"2023-10-06T05:15:03.970480+0000","last_peered":"2023-10-06T05:15:03.970480+0000","last_clean":"2023-10-06T05:15:03.970480+0000","last_became_active":"2023-10-06T05:14:58.966398+0000","last_became_peered":"2023-10-06T05:14:58.966398+0000","last_unstale":"2023-10-06T05:15:03.970480+0000","last_undegraded":"2023-10-06T05:15:03.970480+0000","last_fullsized":"2023-10-06T05:15:03.970480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:00.975481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960837+0000","last_change":"2023-10-06T05:14:59.967931+0000","last_active":"2023-10-06T05:15:03.960837+0000","last_peered":"2023-10-06T05:15:03.960837+0000","last_clean":"2023-10-06T05:15:03.960837+0000","last_became_active":"2023-10-06T05:14:59.967681+0000","last_became_peered":"2023-10-06T05:14:59.967681+0000","last_unstale":"2023-10-06T05:15:03.960837+0000","last_undegraded":"2023-10-06T05:15:03.960837+0000","last_fullsized":"2023-10-06T05:15:03.960837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:16.784926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.990296+0000","last_change":"2023-10-06T05:15:03.989021+0000","last_active":"2023-10-06T05:15:03.990296+0000","last_peered":"2023-10-06T05:15:03.990296+0000","last_clean":"2023-10-06T05:15:03.990296+0000","last_became_active":"2023-10-06T05:15:03.988411+0000","last_became_peered":"2023-10-06T05:15:03.988411+0000","last_unstale":"2023-10-06T05:15:03.990296+0000","last_undegraded":"2023-10-06T05:15:03.990296+0000","last_fullsized":"2023-10-06T05:15:03.990296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:46.905684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.967052+0000","last_change":"2023-10-06T05:14:58.970840+0000","last_active":"2023-10-06T05:15:03.967052+0000","last_peered":"2023-10-06T05:15:03.967052+0000","last_clean":"2023-10-06T05:15:03.967052+0000","last_became_active":"2023-10-06T05:14:58.970619+0000","last_became_peered":"2023-10-06T05:14:58.970619+0000","last_unstale":"2023-10-06T05:15:03.967052+0000","last_undegraded":"2023-10-06T05:15:03.967052+0000","last_fullsized":"2023-10-06T05:15:03.967052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:38.091025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.968655+0000","last_change":"2023-10-06T05:14:59.955830+0000","last_active":"2023-10-06T05:15:03.968655+0000","last_peered":"2023-10-06T05:15:03.968655+0000","last_clean":"2023-10-06T05:15:03.968655+0000","last_became_active":"2023-10-06T05:14:59.955566+0000","last_became_peered":"2023-10-06T05:14:59.955566+0000","last_unstale":"2023-10-06T05:15:03.968655+0000","last_undegraded":"2023-10-06T05:15:03.968655+0000","last_fullsized":"2023-10-06T05:15:03.968655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:47.738141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.987078+0000","last_change":"2023-10-06T05:15:03.984831+0000","last_active":"2023-10-06T05:15:03.987078+0000","last_peered":"2023-10-06T05:15:03.987078+0000","last_clean":"2023-10-06T05:15:03.987078+0000","last_became_active":"2023-10-06T05:15:03.984436+0000","last_became_peered":"2023-10-06T05:15:03.984436+0000","last_unstale":"2023-10-06T05:15:03.987078+0000","last_undegraded":"2023-10-06T05:15:03.987078+0000","last_fullsized":"2023-10-06T05:15:03.987078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:10.551105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961685+0000","last_change":"2023-10-06T05:14:58.956457+0000","last_active":"2023-10-06T05:15:03.961685+0000","last_peered":"2023-10-06T05:15:03.961685+0000","last_clean":"2023-10-06T05:15:03.961685+0000","last_became_active":"2023-10-06T05:14:58.956165+0000","last_became_peered":"2023-10-06T05:14:58.956165+0000","last_unstale":"2023-10-06T05:15:03.961685+0000","last_undegraded":"2023-10-06T05:15:03.961685+0000","last_fullsized":"2023-10-06T05:15:03.961685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:01.396223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960037+0000","last_change":"2023-10-06T05:14:59.968300+0000","last_active":"2023-10-06T05:15:03.960037+0000","last_peered":"2023-10-06T05:15:03.960037+0000","last_clean":"2023-10-06T05:15:03.960037+0000","last_became_active":"2023-10-06T05:14:59.967965+0000","last_became_peered":"2023-10-06T05:14:59.967965+0000","last_unstale":"2023-10-06T05:15:03.960037+0000","last_undegraded":"2023-10-06T05:15:03.960037+0000","last_fullsized":"2023-10-06T05:15:03.960037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:35.451365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.978224+0000","last_change":"2023-10-06T05:15:03.976752+0000","last_active":"2023-10-06T05:15:03.978224+0000","last_peered":"2023-10-06T05:15:03.978224+0000","last_clean":"2023-10-06T05:15:03.978224+0000","last_became_active":"2023-10-06T05:15:03.976322+0000","last_became_peered":"2023-10-06T05:15:03.976322+0000","last_unstale":"2023-10-06T05:15:03.978224+0000","last_undegraded":"2023-10-06T05:15:03.978224+0000","last_fullsized":"2023-10-06T05:15:03.978224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:43.931516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.962719+0000","last_change":"2023-10-06T05:14:58.955881+0000","last_active":"2023-10-06T05:15:03.962719+0000","last_peered":"2023-10-06T05:15:03.962719+0000","last_clean":"2023-10-06T05:15:03.962719+0000","last_became_active":"2023-10-06T05:14:58.955624+0000","last_became_peered":"2023-10-06T05:14:58.955624+0000","last_unstale":"2023-10-06T05:15:03.962719+0000","last_undegraded":"2023-10-06T05:15:03.962719+0000","last_fullsized":"2023-10-06T05:15:03.962719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:16.365139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960721+0000","last_change":"2023-10-06T05:14:59.968565+0000","last_active":"2023-10-06T05:15:03.960721+0000","last_peered":"2023-10-06T05:15:03.960721+0000","last_clean":"2023-10-06T05:15:03.960721+0000","last_became_active":"2023-10-06T05:14:59.965728+0000","last_became_peered":"2023-10-06T05:14:59.965728+0000","last_unstale":"2023-10-06T05:15:03.960721+0000","last_undegraded":"2023-10-06T05:15:03.960721+0000","last_fullsized":"2023-10-06T05:15:03.960721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:49.359907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.977840+0000","last_change":"2023-10-06T05:15:03.975283+0000","last_active":"2023-10-06T05:15:03.977840+0000","last_peered":"2023-10-06T05:15:03.977840+0000","last_clean":"2023-10-06T05:15:03.977840+0000","last_became_active":"2023-10-06T05:15:03.974903+0000","last_became_peered":"2023-10-06T05:15:03.974903+0000","last_unstale":"2023-10-06T05:15:03.977840+0000","last_undegraded":"2023-10-06T05:15:03.977840+0000","last_fullsized":"2023-10-06T05:15:03.977840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:18.853022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.966753+0000","last_change":"2023-10-06T05:14:58.961971+0000","last_active":"2023-10-06T05:15:03.966753+0000","last_peered":"2023-10-06T05:15:03.966753+0000","last_clean":"2023-10-06T05:15:03.966753+0000","last_became_active":"2023-10-06T05:14:58.961689+0000","last_became_peered":"2023-10-06T05:14:58.961689+0000","last_unstale":"2023-10-06T05:15:03.966753+0000","last_undegraded":"2023-10-06T05:15:03.966753+0000","last_fullsized":"2023-10-06T05:15:03.966753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:47.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963640+0000","last_change":"2023-10-06T05:14:59.959821+0000","last_active":"2023-10-06T05:15:03.963640+0000","last_peered":"2023-10-06T05:15:03.963640+0000","last_clean":"2023-10-06T05:15:03.963640+0000","last_became_active":"2023-10-06T05:14:59.959543+0000","last_became_peered":"2023-10-06T05:14:59.959543+0000","last_unstale":"2023-10-06T05:15:03.963640+0000","last_undegraded":"2023-10-06T05:15:03.963640+0000","last_fullsized":"2023-10-06T05:15:03.963640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.991469+0000","last_change":"2023-10-06T05:15:03.990597+0000","last_active":"2023-10-06T05:15:03.991469+0000","last_peered":"2023-10-06T05:15:03.991469+0000","last_clean":"2023-10-06T05:15:03.991469+0000","last_became_active":"2023-10-06T05:15:03.990258+0000","last_became_peered":"2023-10-06T05:15:03.990258+0000","last_unstale":"2023-10-06T05:15:03.991469+0000","last_undegraded":"2023-10-06T05:15:03.991469+0000","last_fullsized":"2023-10-06T05:15:03.991469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:15:02.953018+0000","last_clean_scrub_stamp":"2023-10-06T05:15:02.953018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.696122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.970366+0000","last_change":"2023-10-06T05:14:58.970911+0000","last_active":"2023-10-06T05:15:03.970366+0000","last_peered":"2023-10-06T05:15:03.970366+0000","last_clean":"2023-10-06T05:15:03.970366+0000","last_became_active":"2023-10-06T05:14:58.970439+0000","last_became_peered":"2023-10-06T05:14:58.970439+0000","last_unstale":"2023-10-06T05:15:03.970366+0000","last_undegraded":"2023-10-06T05:15:03.970366+0000","last_fullsized":"2023-10-06T05:15:03.970366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:57.920454+0000","last_clean_scrub_stamp":"2023-10-06T05:14:57.920454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:03.399022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.960109+0000","last_change":"2023-10-06T05:14:59.961878+0000","last_active":"2023-10-06T05:15:03.960109+0000","last_peered":"2023-10-06T05:15:03.960109+0000","last_clean":"2023-10-06T05:15:03.960109+0000","last_became_active":"2023-10-06T05:14:59.961617+0000","last_became_peered":"2023-10-06T05:14:59.961617+0000","last_unstale":"2023-10-06T05:15:03.960109+0000","last_undegraded":"2023-10-06T05:15:03.960109+0000","last_fullsized":"2023-10-06T05:15:03.960109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.974134+0000","last_change":"2023-10-06T05:14:59.960593+0000","last_active":"2023-10-06T05:15:03.974134+0000","last_peered":"2023-10-06T05:15:03.974134+0000","last_clean":"2023-10-06T05:15:03.974134+0000","last_became_active":"2023-10-06T05:14:59.960329+0000","last_became_peered":"2023-10-06T05:14:59.960329+0000","last_unstale":"2023-10-06T05:15:03.974134+0000","last_undegraded":"2023-10-06T05:15:03.974134+0000","last_fullsized":"2023-10-06T05:15:03.974134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:02.403458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963791+0000","last_change":"2023-10-06T05:14:59.966932+0000","last_active":"2023-10-06T05:15:03.963791+0000","last_peered":"2023-10-06T05:15:03.963791+0000","last_clean":"2023-10-06T05:15:03.963791+0000","last_became_active":"2023-10-06T05:14:59.965101+0000","last_became_peered":"2023-10-06T05:14:59.965101+0000","last_unstale":"2023-10-06T05:15:03.963791+0000","last_undegraded":"2023-10-06T05:15:03.963791+0000","last_fullsized":"2023-10-06T05:15:03.963791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.406317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.961190+0000","last_change":"2023-10-06T05:14:59.961912+0000","last_active":"2023-10-06T05:15:03.961190+0000","last_peered":"2023-10-06T05:15:03.961190+0000","last_clean":"2023-10-06T05:15:03.961190+0000","last_became_active":"2023-10-06T05:14:59.961658+0000","last_became_peered":"2023-10-06T05:14:59.961658+0000","last_unstale":"2023-10-06T05:15:03.961190+0000","last_undegraded":"2023-10-06T05:15:03.961190+0000","last_fullsized":"2023-10-06T05:15:03.961190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:04.303466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.963927+0000","last_change":"2023-10-06T05:14:59.968762+0000","last_active":"2023-10-06T05:15:03.963927+0000","last_peered":"2023-10-06T05:15:03.963927+0000","last_clean":"2023-10-06T05:15:03.963927+0000","last_became_active":"2023-10-06T05:14:59.968111+0000","last_became_peered":"2023-10-06T05:14:59.968111+0000","last_unstale":"2023-10-06T05:15:03.963927+0000","last_undegraded":"2023-10-06T05:15:03.963927+0000","last_fullsized":"2023-10-06T05:15:03.963927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:04.982889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T05:15:03.973699+0000","last_change":"2023-10-06T05:14:59.958480+0000","last_active":"2023-10-06T05:15:03.973699+0000","last_peered":"2023-10-06T05:15:03.973699+0000","last_clean":"2023-10-06T05:15:03.973699+0000","last_became_active":"2023-10-06T05:14:59.958139+0000","last_became_peered":"2023-10-06T05:14:59.958139+0000","last_unstale":"2023-10-06T05:15:03.973699+0000","last_undegraded":"2023-10-06T05:15:03.973699+0000","last_fullsized":"2023-10-06T05:15:03.973699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:58.927690+0000","last_clean_scrub_stamp":"2023-10-06T05:14:58.927690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:03.025853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4485,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":698729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0490000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":695352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":104430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.571}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":103167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":274432,"data_stored":108201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":105054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:15:25.255 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:15:25.255 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T05:15:25.255 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T05:15:25.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:15:25.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.418+0000 7feecff64700 1 Processor -- start 2023-10-06T05:15:25.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.419+0000 7feecff64700 1 -- start start 2023-10-06T05:15:25.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecff64700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec80c8ff0 0x7feec80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecff64700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec815b000 con 0x7feec80caad0 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecff64700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec815b2e0 con 0x7feec8157200 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecff64700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec815b5c0 con 0x7feec80c8ff0 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecd4ff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feec80caad0 0x7feec8156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58674/0 (socket says 172.21.15.45:58674) 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecd4ff700 1 -- 172.21.15.45:0/2932104917 learned_addr learned my addr 172.21.15.45:0/2932104917 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecdd00700 1 --2- 172.21.15.45:0/2932104917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec80c8ff0 0x7feec80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecccfe700 1 -- 172.21.15.45:0/2932104917 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160740811 0 0) 0x7feec815b000 con 0x7feec80caad0 2023-10-06T05:15:25.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecccfe700 1 -- 172.21.15.45:0/2932104917 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb4003570 con 0x7feec80caad0 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecccfe700 1 -- 172.21.15.45:0/2932104917 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520683779 0 0) 0x7feec815b2e0 con 0x7feec8157200 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecccfe700 1 -- 172.21.15.45:0/2932104917 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec815b000 con 0x7feec8157200 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecdd00700 1 -- 172.21.15.45:0/2932104917 >> v1:172.21.15.144:6789/0 conn(0x7feec8157200 legacy=0x7feec80cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecdd00700 1 -- 172.21.15.45:0/2932104917 >> v1:172.21.15.45:6789/0 conn(0x7feec80caad0 legacy=0x7feec8156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.420+0000 7feecdd00700 1 -- 172.21.15.45:0/2932104917 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec815b8a0 con 0x7feec80c8ff0 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecdd00700 1 --2- 172.21.15.45:0/2932104917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec80c8ff0 0x7feec80c93c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feec4009910 tx=0x7feec40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12e023f02764a44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecccfe700 1 -- 172.21.15.45:0/2932104917 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011630 con 0x7feec80c8ff0 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecccfe700 1 -- 172.21.15.45:0/2932104917 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4011790 con 0x7feec80c8ff0 2023-10-06T05:15:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecccfe700 1 -- 172.21.15.45:0/2932104917 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011b10 con 0x7feec80c8ff0 2023-10-06T05:15:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecff64700 1 -- 172.21.15.45:0/2932104917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec80c8ff0 msgr2=0x7feec80c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecff64700 1 --2- 172.21.15.45:0/2932104917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec80c8ff0 0x7feec80c93c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feec4009910 tx=0x7feec40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecff64700 1 -- 172.21.15.45:0/2932104917 shutdown_connections 2023-10-06T05:15:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecff64700 1 --2- 172.21.15.45:0/2932104917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec80c8ff0 0x7feec80c93c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecff64700 1 -- 172.21.15.45:0/2932104917 >> 172.21.15.45:0/2932104917 conn(0x7feec80cdf50 msgr2=0x7feec80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.421+0000 7feecff64700 1 -- 172.21.15.45:0/2932104917 shutdown_connections 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.422+0000 7feecff64700 1 -- 172.21.15.45:0/2932104917 wait complete. 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.423+0000 7feecff64700 1 Processor -- start 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.423+0000 7feecff64700 1 -- start start 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.423+0000 7feecff64700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feec80c8ff0 0x7feec80c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.423+0000 7feecff64700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feec80caad0 0x7feec80b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.423+0000 7feecff64700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec8157200 0x7feec814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecff64700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec80b25a0 con 0x7feec80caad0 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecff64700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec8169890 con 0x7feec80c8ff0 2023-10-06T05:15:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecff64700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec8169b70 con 0x7feec8157200 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feec80caad0 0x7feec80b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feec80caad0 0x7feec80b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53548/0 (socket says 172.21.15.45:53548) 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 -- 172.21.15.45:0/17642695 learned_addr learned my addr 172.21.15.45:0/17642695 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecdd00700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feec80c8ff0 0x7feec80c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feece501700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec8157200 0x7feec814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 -- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec8157200 msgr2=0x7feec814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec8157200 0x7feec814e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 -- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feec80c8ff0 msgr2=0x7feec80c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feec80c8ff0 0x7feec80c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 -- 172.21.15.45:0/17642695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4009420 con 0x7feec80caad0 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecd4ff700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feec80caad0 0x7feec80b2060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c790 tx=0x7feeb8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feebdffb700 1 -- 172.21.15.45:0/17642695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8005b30 con 0x7feec80caad0 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feebdffb700 1 -- 172.21.15.45:0/17642695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8005c90 con 0x7feec80caad0 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec8169e50 con 0x7feec80caad0 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.424+0000 7feebdffb700 1 -- 172.21.15.45:0/17642695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8007300 con 0x7feec80caad0 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.425+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec816a490 con 0x7feec80caad0 2023-10-06T05:15:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.426+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec8169600 con 0x7feec80caad0 2023-10-06T05:15:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.426+0000 7feebdffb700 1 -- 172.21.15.45:0/17642695 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feeb801b3b0 con 0x7feec80caad0 2023-10-06T05:15:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.427+0000 7feebdffb700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feeac066f30 0x7feeac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.427+0000 7feebdffb700 1 -- 172.21.15.45:0/17642695 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7feeb8083e60 con 0x7feec80caad0 2023-10-06T05:15:25.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.427+0000 7feecdd00700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feeac066f30 0x7feeac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.427+0000 7feecdd00700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feeac066f30 0x7feeac0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feec4009860 tx=0x7feec4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.431+0000 7feebdffb700 1 -- 172.21.15.45:0/17642695 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeb8050c80 con 0x7feec80caad0 2023-10-06T05:15:25.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.677+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feec804bb40 con 0x7feec80caad0 2023-10-06T05:15:25.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.678+0000 7feebdffb700 1 -- 172.21.15.45:0/17642695 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7feeb8017080 con 0x7feec80caad0 2023-10-06T05:15:25.680 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:25.680 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:15:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feeac066f30 msgr2=0x7feeac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feeac066f30 0x7feeac0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feec4009860 tx=0x7feec4017000 comp rx=0 tx=0).stop 2023-10-06T05:15:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feec80caad0 msgr2=0x7feec80b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feec80caad0 0x7feec80b2060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c790 tx=0x7feeb8007ea0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 shutdown_connections 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feeac066f30 0x7feeac0693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feec8157200 0x7feec814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feec80c8ff0 0x7feec80c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 --2- 172.21.15.45:0/17642695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feec80caad0 0x7feec80b2060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 >> 172.21.15.45:0/17642695 conn(0x7feec80cdf50 msgr2=0x7feec8150470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 shutdown_connections 2023-10-06T05:15:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.680+0000 7feecff64700 1 -- 172.21.15.45:0/17642695 wait complete. 2023-10-06T05:15:25.695 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T05:15:25.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:25.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.826+0000 7fa1540f9700 1 Processor -- start 2023-10-06T05:15:25.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.827+0000 7fa1540f9700 1 -- start start 2023-10-06T05:15:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa1540f9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa14c0c8ff0 0x7fa14c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa1540f9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c157200 0x7fa14c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa1540f9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c15b000 con 0x7fa14c0caad0 2023-10-06T05:15:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa1540f9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c15b2e0 con 0x7fa14c0c8ff0 2023-10-06T05:15:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa1540f9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c15b5c0 con 0x7fa14c157200 2023-10-06T05:15:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa151e95700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa14c0c8ff0 0x7fa14c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa152696700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c157200 0x7fa14c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa151694700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa14c0caad0 0x7fa14c156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50930/0 (socket says 172.21.15.45:50930) 2023-10-06T05:15:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa151694700 1 -- 172.21.15.45:0/817528007 learned_addr learned my addr 172.21.15.45:0/817528007 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa152696700 1 -- 172.21.15.45:0/817528007 >> v2:172.21.15.144:3300/0 conn(0x7fa14c0c8ff0 msgr2=0x7fa14c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa152696700 1 --2- 172.21.15.45:0/817528007 >> v2:172.21.15.144:3300/0 conn(0x7fa14c0c8ff0 0x7fa14c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.828+0000 7fa152696700 1 -- 172.21.15.45:0/817528007 >> v1:172.21.15.144:6789/0 conn(0x7fa14c0caad0 legacy=0x7fa14c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.829+0000 7fa152696700 1 -- 172.21.15.45:0/817528007 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c15b8a0 con 0x7fa14c157200 2023-10-06T05:15:25.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.829+0000 7fa151e95700 1 --2- 172.21.15.45:0/817528007 >> v2:172.21.15.144:3300/0 conn(0x7fa14c0c8ff0 0x7fa14c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:25.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.829+0000 7fa152696700 1 --2- 172.21.15.45:0/817528007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c157200 0x7fa14c0caf40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa144009e60 tx=0x7fa144009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d9c36655f0fab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.831+0000 7fa150e93700 1 -- 172.21.15.45:0/817528007 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400e070 con 0x7fa14c157200 2023-10-06T05:15:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.831+0000 7fa150e93700 1 -- 172.21.15.45:0/817528007 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14400bb60 con 0x7fa14c157200 2023-10-06T05:15:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.831+0000 7fa150e93700 1 -- 172.21.15.45:0/817528007 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144012430 con 0x7fa14c157200 2023-10-06T05:15:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 -- 172.21.15.45:0/817528007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c157200 msgr2=0x7fa14c0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 --2- 172.21.15.45:0/817528007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c157200 0x7fa14c0caf40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa144009e60 tx=0x7fa144009780 comp rx=0 tx=0).stop 2023-10-06T05:15:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 -- 172.21.15.45:0/817528007 shutdown_connections 2023-10-06T05:15:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 --2- 172.21.15.45:0/817528007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c157200 0x7fa14c0caf40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 --2- 172.21.15.45:0/817528007 >> v2:172.21.15.144:3300/0 conn(0x7fa14c0c8ff0 0x7fa14c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 -- 172.21.15.45:0/817528007 >> 172.21.15.45:0/817528007 conn(0x7fa14c0cdf50 msgr2=0x7fa14c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 -- 172.21.15.45:0/817528007 shutdown_connections 2023-10-06T05:15:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.832+0000 7fa1540f9700 1 -- 172.21.15.45:0/817528007 wait complete. 2023-10-06T05:15:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.833+0000 7fa1540f9700 1 Processor -- start 2023-10-06T05:15:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.833+0000 7fa1540f9700 1 -- start start 2023-10-06T05:15:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa1540f9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c0c8ff0 0x7fa14c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa1540f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c0caad0 0x7fa14c0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa1540f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14c157200 0x7fa14c0c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa1540f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0b3c30 con 0x7fa14c0caad0 2023-10-06T05:15:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa1540f9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0aeaa0 con 0x7fa14c157200 2023-10-06T05:15:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa1540f9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0c70e0 con 0x7fa14c0c8ff0 2023-10-06T05:15:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151694700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c0caad0 0x7fa14c0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151694700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c0caad0 0x7fa14c0b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53562/0 (socket says 172.21.15.45:53562) 2023-10-06T05:15:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151694700 1 -- 172.21.15.45:0/3433969052 learned_addr learned my addr 172.21.15.45:0/3433969052 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa152696700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14c157200 0x7fa14c0c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151e95700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c0c8ff0 0x7fa14c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151694700 1 -- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c0c8ff0 msgr2=0x7fa14c15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151694700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c0c8ff0 0x7fa14c15a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151694700 1 -- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14c157200 msgr2=0x7fa14c0c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.834+0000 7fa151694700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14c157200 0x7fa14c0c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.835+0000 7fa151694700 1 -- 172.21.15.45:0/3433969052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa148009420 con 0x7fa14c0caad0 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.835+0000 7fa151694700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c0caad0 0x7fa14c0b36f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7c0 tx=0x7fa13c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.835+0000 7fa141ffb700 1 -- 172.21.15.45:0/3433969052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c005b30 con 0x7fa14c0caad0 2023-10-06T05:15:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.835+0000 7fa141ffb700 1 -- 172.21.15.45:0/3433969052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c005c90 con 0x7fa14c0caad0 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.835+0000 7fa141ffb700 1 -- 172.21.15.45:0/3433969052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c004a80 con 0x7fa14c0caad0 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.835+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa144009a20 con 0x7fa14c0caad0 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.835+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c0c7660 con 0x7fa14c0caad0 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.836+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c06f410 con 0x7fa14c0caad0 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.836+0000 7fa141ffb700 1 -- 172.21.15.45:0/3433969052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa13c0070d0 con 0x7fa14c0caad0 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.837+0000 7fa141ffb700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa130066f30 0x7fa1300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.837+0000 7fa141ffb700 1 -- 172.21.15.45:0/3433969052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa13c083c70 con 0x7fa14c0caad0 2023-10-06T05:15:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.837+0000 7fa151e95700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa130066f30 0x7fa1300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:25.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.837+0000 7fa151e95700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa130066f30 0x7fa1300693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa148009f40 tx=0x7fa1480091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:25.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:25.841+0000 7fa141ffb700 1 -- 172.21.15.45:0/3433969052 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa13c04b5a0 con 0x7fa14c0caad0 2023-10-06T05:15:26.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.043+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa14c04bb40 con 0x7fa14c0caad0 2023-10-06T05:15:26.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.044+0000 7fa141ffb700 1 -- 172.21.15.45:0/3433969052 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fa13c050a90 con 0x7fa14c0caad0 2023-10-06T05:15:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:26.046 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa130066f30 msgr2=0x7fa1300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa130066f30 0x7fa1300693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa148009f40 tx=0x7fa1480091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c0caad0 msgr2=0x7fa14c0b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c0caad0 0x7fa14c0b36f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7c0 tx=0x7fa13c007f70 comp rx=0 tx=0).stop 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 shutdown_connections 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa130066f30 0x7fa1300693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14c157200 0x7fa14c0c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14c0c8ff0 0x7fa14c15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 --2- 172.21.15.45:0/3433969052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c0caad0 0x7fa14c0b36f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 >> 172.21.15.45:0/3433969052 conn(0x7fa14c0cdf50 msgr2=0x7fa14c14a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 shutdown_connections 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.046+0000 7fa1540f9700 1 -- 172.21.15.45:0/3433969052 wait complete. 2023-10-06T05:15:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:57.253547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:58.338382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T05:15:01.988547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:26.062 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.195+0000 7f2471f3e700 1 Processor -- start 2023-10-06T05:15:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.195+0000 7f2471f3e700 1 -- start start 2023-10-06T05:15:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.196+0000 7f2471f3e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f246c157330 0x7f246c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.196+0000 7f2471f3e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0ccb70 0x7f246c0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.196+0000 7f2471f3e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246c0c3b50 con 0x7f246c0aaae0 2023-10-06T05:15:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.196+0000 7f2471f3e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246c157700 con 0x7f246c157330 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.196+0000 7f2471f3e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0c98e0 con 0x7f246c0ccb70 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.196+0000 7f246affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f246c0aaae0 0x7f246c0c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58680/0 (socket says 172.21.15.45:58680) 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246affd700 1 -- 172.21.15.45:0/3337882925 learned_addr learned my addr 172.21.15.45:0/3337882925 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246b7fe700 1 --2- 172.21.15.45:0/3337882925 >> v2:172.21.15.144:3300/0 conn(0x7f246c157330 0x7f246c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246bfff700 1 --2- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0ccb70 0x7f246c0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246a7fc700 1 -- 172.21.15.45:0/3337882925 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132898123 0 0) 0x7f246c0c3b50 con 0x7f246c0aaae0 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246a7fc700 1 -- 172.21.15.45:0/3337882925 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2458003570 con 0x7f246c0aaae0 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246b7fe700 1 -- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0ccb70 msgr2=0x7f246c0c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246b7fe700 1 --2- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0ccb70 0x7f246c0c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246b7fe700 1 -- 172.21.15.45:0/3337882925 >> v1:172.21.15.45:6789/0 conn(0x7f246c0aaae0 legacy=0x7f246c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246b7fe700 1 -- 172.21.15.45:0/3337882925 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c0c9bc0 con 0x7f246c157330 2023-10-06T05:15:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246bfff700 1 --2- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0ccb70 0x7f246c0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246b7fe700 1 --2- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c157330 0x7f246c0ce8b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f245c0098d0 tx=0x7f245c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda07d81f0a1a3df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.197+0000 7f246a7fc700 1 -- 172.21.15.45:0/3337882925 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c005c10 con 0x7f246c157330 2023-10-06T05:15:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f246a7fc700 1 -- 172.21.15.45:0/3337882925 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c005d70 con 0x7f246c157330 2023-10-06T05:15:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f246a7fc700 1 -- 172.21.15.45:0/3337882925 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c006260 con 0x7f246c157330 2023-10-06T05:15:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 -- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c157330 msgr2=0x7f246c0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 --2- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c157330 0x7f246c0ce8b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f245c0098d0 tx=0x7f245c009120 comp rx=0 tx=0).stop 2023-10-06T05:15:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 -- 172.21.15.45:0/3337882925 shutdown_connections 2023-10-06T05:15:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 --2- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0ccb70 0x7f246c0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 --2- 172.21.15.45:0/3337882925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c157330 0x7f246c0ce8b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 -- 172.21.15.45:0/3337882925 >> 172.21.15.45:0/3337882925 conn(0x7f246c067e00 msgr2=0x7f246c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 -- 172.21.15.45:0/3337882925 shutdown_connections 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.198+0000 7f2471f3e700 1 -- 172.21.15.45:0/3337882925 wait complete. 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.199+0000 7f2471f3e700 1 Processor -- start 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f2471f3e700 1 -- start start 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f2471f3e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0aaae0 0x7f246c14cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f2471f3e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c0ccb70 0x7f246c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f2471f3e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f246c157330 0x7f246c0ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f2471f3e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c155500 con 0x7f246c157330 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f2471f3e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c154930 con 0x7f246c0ccb70 2023-10-06T05:15:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f2471f3e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c154bd0 con 0x7f246c0aaae0 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.200+0000 7f246bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f246c157330 0x7f246c0ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f246c157330 0x7f246c0ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53576/0 (socket says 172.21.15.45:53576) 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 -- 172.21.15.45:0/2086288186 learned_addr learned my addr 172.21.15.45:0/2086288186 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246b7fe700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0aaae0 0x7f246c14cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246affd700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c0ccb70 0x7f246c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 -- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0aaae0 msgr2=0x7f246c14cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0aaae0 0x7f246c14cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 -- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c0ccb70 msgr2=0x7f246c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c0ccb70 0x7f246c156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 -- 172.21.15.45:0/2086288186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464009a20 con 0x7f246c157330 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f246bfff700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f246c157330 0x7f246c0ac0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2464009fc0 tx=0x7f246400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f24637fe700 1 -- 172.21.15.45:0/2086288186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400cbd0 con 0x7f246c157330 2023-10-06T05:15:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f24637fe700 1 -- 172.21.15.45:0/2086288186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246400f040 con 0x7f246c157330 2023-10-06T05:15:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f24637fe700 1 -- 172.21.15.45:0/2086288186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464014480 con 0x7f246c157330 2023-10-06T05:15:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c0093c0 con 0x7f246c157330 2023-10-06T05:15:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.201+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246c155190 con 0x7f246c157330 2023-10-06T05:15:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.202+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246c155890 con 0x7f246c157330 2023-10-06T05:15:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.203+0000 7f24637fe700 1 -- 172.21.15.45:0/2086288186 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2464003020 con 0x7f246c157330 2023-10-06T05:15:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.203+0000 7f24637fe700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f244c066f30 0x7f244c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.203+0000 7f24637fe700 1 -- 172.21.15.45:0/2086288186 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f24640852a0 con 0x7f246c157330 2023-10-06T05:15:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.203+0000 7f246b7fe700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f244c066f30 0x7f244c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.204+0000 7f246b7fe700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f244c066f30 0x7f244c0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f245c009160 tx=0x7f245c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.208+0000 7f24637fe700 1 -- 172.21.15.45:0/2086288186 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24640520c0 con 0x7f246c157330 2023-10-06T05:15:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.412+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f246c06e350 con 0x7f246c157330 2023-10-06T05:15:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.413+0000 7f24637fe700 1 -- 172.21.15.45:0/2086288186 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24640556e0 con 0x7f246c157330 2023-10-06T05:15:26.415 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f244c066f30 msgr2=0x7f244c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f244c066f30 0x7f244c0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f245c009160 tx=0x7f245c016000 comp rx=0 tx=0).stop 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f246c157330 msgr2=0x7f246c0ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f246c157330 0x7f246c0ac0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2464009fc0 tx=0x7f246400b040 comp rx=0 tx=0).stop 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 shutdown_connections 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f244c066f30 0x7f244c0693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f246c157330 0x7f246c0ac0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c0aaae0 0x7f246c14cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 --2- 172.21.15.45:0/2086288186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c0ccb70 0x7f246c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 >> 172.21.15.45:0/2086288186 conn(0x7f246c067e00 msgr2=0x7f246c0cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:26.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 shutdown_connections 2023-10-06T05:15:26.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.415+0000 7f2471f3e700 1 -- 172.21.15.45:0/2086288186 wait complete. 2023-10-06T05:15:26.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.563+0000 7fbd56949700 1 Processor -- start 2023-10-06T05:15:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.563+0000 7fbd56949700 1 -- start start 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd56949700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50159150 0x7fbd500ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd56949700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbd50159520 0x7fbd5015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd56949700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd500c9c90 con 0x7fbd5015d3d0 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd56949700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5015d8e0 con 0x7fbd50159520 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd56949700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5015dbc0 con 0x7fbd50159150 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd4ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50159150 0x7fbd500ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd4f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbd50159520 0x7fbd5015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd4ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50159150 0x7fbd500ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34406/0 (socket says 172.21.15.45:34406) 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd4ffff700 1 -- 172.21.15.45:0/830385681 learned_addr learned my addr 172.21.15.45:0/830385681 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.564+0000 7fbd54ee6700 1 --1- 172.21.15.45:0/830385681 >> v1:172.21.15.144:6789/0 conn(0x7fbd5015d3d0 0x7fbd50067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:58076/0 (socket says 172.21.15.45:58076) 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4ffff700 1 -- 172.21.15.45:0/830385681 >> v2:172.21.15.144:3300/0 conn(0x7fbd50159520 msgr2=0x7fbd5015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4ffff700 1 --2- 172.21.15.45:0/830385681 >> v2:172.21.15.144:3300/0 conn(0x7fbd50159520 0x7fbd5015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4ffff700 1 -- 172.21.15.45:0/830385681 >> v1:172.21.15.144:6789/0 conn(0x7fbd5015d3d0 legacy=0x7fbd50067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4ffff700 1 -- 172.21.15.45:0/830385681 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500c9920 con 0x7fbd50159150 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4ffff700 1 --2- 172.21.15.45:0/830385681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50159150 0x7fbd500ca120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd400099b0 tx=0x7fbd40009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e28fcc3913271c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4effd700 1 -- 172.21.15.45:0/830385681 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40009ea0 con 0x7fbd50159150 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4effd700 1 -- 172.21.15.45:0/830385681 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4000b860 con 0x7fbd50159150 2023-10-06T05:15:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd4effd700 1 -- 172.21.15.45:0/830385681 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000bc00 con 0x7fbd50159150 2023-10-06T05:15:26.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd56949700 1 -- 172.21.15.45:0/830385681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50159150 msgr2=0x7fbd500ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd56949700 1 --2- 172.21.15.45:0/830385681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50159150 0x7fbd500ca120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd400099b0 tx=0x7fbd40009250 comp rx=0 tx=0).stop 2023-10-06T05:15:26.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd56949700 1 -- 172.21.15.45:0/830385681 shutdown_connections 2023-10-06T05:15:26.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.565+0000 7fbd56949700 1 --2- 172.21.15.45:0/830385681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50159150 0x7fbd500ca120 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.566+0000 7fbd56949700 1 --2- 172.21.15.45:0/830385681 >> v2:172.21.15.144:3300/0 conn(0x7fbd50159520 0x7fbd5015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.566+0000 7fbd56949700 1 -- 172.21.15.45:0/830385681 >> 172.21.15.45:0/830385681 conn(0x7fbd500691c0 msgr2=0x7fbd500cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.566+0000 7fbd56949700 1 -- 172.21.15.45:0/830385681 shutdown_connections 2023-10-06T05:15:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.566+0000 7fbd56949700 1 -- 172.21.15.45:0/830385681 wait complete. 2023-10-06T05:15:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.567+0000 7fbd56949700 1 Processor -- start 2023-10-06T05:15:26.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.567+0000 7fbd56949700 1 -- start start 2023-10-06T05:15:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.568+0000 7fbd56949700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd50159150 0x7fbd501478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.568+0000 7fbd56949700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd5015d3d0 0x7fbd500c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.568+0000 7fbd56949700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd500c64e0 0x7fbd500c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.568+0000 7fbd56949700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd50148410 con 0x7fbd500c64e0 2023-10-06T05:15:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.568+0000 7fbd56949700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5015a210 con 0x7fbd50159150 2023-10-06T05:15:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.568+0000 7fbd56949700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd50159900 con 0x7fbd5015d3d0 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd500c64e0 0x7fbd500c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd4ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd50159150 0x7fbd501478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd500c64e0 0x7fbd500c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47630/0 (socket says 172.21.15.45:47630) 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 -- 172.21.15.45:0/1560122268 learned_addr learned my addr 172.21.15.45:0/1560122268 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd4f7fe700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd5015d3d0 0x7fbd500c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 -- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd5015d3d0 msgr2=0x7fbd500c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd5015d3d0 0x7fbd500c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 -- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd50159150 msgr2=0x7fbd501478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd50159150 0x7fbd501478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 -- 172.21.15.45:0/1560122268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd400094f0 con 0x7fbd500c64e0 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd4ffff700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd50159150 0x7fbd501478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:26.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd54ee6700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd500c64e0 0x7fbd500c8060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800cc70 tx=0x7fbd48001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd47fff700 1 -- 172.21.15.45:0/1560122268 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48005df0 con 0x7fbd500c64e0 2023-10-06T05:15:26.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd47fff700 1 -- 172.21.15.45:0/1560122268 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4800a620 con 0x7fbd500c64e0 2023-10-06T05:15:26.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd500c6a20 con 0x7fbd500c64e0 2023-10-06T05:15:26.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd47fff700 1 -- 172.21.15.45:0/1560122268 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4800a9a0 con 0x7fbd500c64e0 2023-10-06T05:15:26.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.569+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd50159ea0 con 0x7fbd500c64e0 2023-10-06T05:15:26.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.570+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5015a3a0 con 0x7fbd500c64e0 2023-10-06T05:15:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.571+0000 7fbd47fff700 1 -- 172.21.15.45:0/1560122268 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd4800ad50 con 0x7fbd500c64e0 2023-10-06T05:15:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.571+0000 7fbd47fff700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd30066f60 0x7fbd30069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.571+0000 7fbd47fff700 1 -- 172.21.15.45:0/1560122268 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbd48084940 con 0x7fbd500c64e0 2023-10-06T05:15:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.572+0000 7fbd4ffff700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd30066f60 0x7fbd30069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.572+0000 7fbd4ffff700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd30066f60 0x7fbd30069410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd400099b0 tx=0x7fbd40017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.576+0000 7fbd47fff700 1 -- 172.21.15.45:0/1560122268 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd48051740 con 0x7fbd500c64e0 2023-10-06T05:15:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.780+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd50147e40 con 0x7fbd500c64e0 2023-10-06T05:15:26.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.781+0000 7fbd47fff700 1 -- 172.21.15.45:0/1560122268 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd48054d60 con 0x7fbd500c64e0 2023-10-06T05:15:26.783 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:15:26.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd30066f60 msgr2=0x7fbd30069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd30066f60 0x7fbd30069410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd400099b0 tx=0x7fbd40017000 comp rx=0 tx=0).stop 2023-10-06T05:15:26.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd500c64e0 msgr2=0x7fbd500c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd500c64e0 0x7fbd500c8060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800cc70 tx=0x7fbd48001a10 comp rx=0 tx=0).stop 2023-10-06T05:15:26.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 shutdown_connections 2023-10-06T05:15:26.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd30066f60 0x7fbd30069410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd500c64e0 0x7fbd500c8060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd50159150 0x7fbd501478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 --2- 172.21.15.45:0/1560122268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd5015d3d0 0x7fbd500c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 >> 172.21.15.45:0/1560122268 conn(0x7fbd500691c0 msgr2=0x7fbd500c1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:26.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 shutdown_connections 2023-10-06T05:15:26.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.783+0000 7fbd56949700 1 -- 172.21.15.45:0/1560122268 wait complete. 2023-10-06T05:15:26.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.931+0000 7fc48513e700 1 Processor -- start 2023-10-06T05:15:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.931+0000 7fc48513e700 1 -- start start 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc48513e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc48013f900 0x7fc4800cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc48513e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4800c8de0 0x7fc4800c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc48513e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4800c9630 con 0x7fc4800c9d50 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc48513e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4800ca150 con 0x7fc48013f900 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc48513e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800cb550 con 0x7fc4800c8de0 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc47ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc48013f900 0x7fc4800cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc47ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc48013f900 0x7fc4800cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47634/0 (socket says 172.21.15.45:47634) 2023-10-06T05:15:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.932+0000 7fc47ed9d700 1 -- 172.21.15.45:0/3358082704 learned_addr learned my addr 172.21.15.45:0/3358082704 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47e59c700 1 --2- 172.21.15.45:0/3358082704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4800c8de0 0x7fc4800c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47ed9d700 1 -- 172.21.15.45:0/3358082704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4800c8de0 msgr2=0x7fc4800c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47ed9d700 1 --2- 172.21.15.45:0/3358082704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4800c8de0 0x7fc4800c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47ed9d700 1 -- 172.21.15.45:0/3358082704 >> v1:172.21.15.144:6789/0 conn(0x7fc4800c9d50 legacy=0x7fc48006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47ed9d700 1 -- 172.21.15.45:0/3358082704 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800cb830 con 0x7fc48013f900 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47ed9d700 1 --2- 172.21.15.45:0/3358082704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc48013f900 0x7fc4800cc840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4700099b0 tx=0x7fc470009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4dce6e1d20fbad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47dd9b700 1 -- 172.21.15.45:0/3358082704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000b860 con 0x7fc48013f900 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47dd9b700 1 -- 172.21.15.45:0/3358082704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000b9c0 con 0x7fc48013f900 2023-10-06T05:15:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.933+0000 7fc47dd9b700 1 -- 172.21.15.45:0/3358082704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000bd60 con 0x7fc48013f900 2023-10-06T05:15:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 -- 172.21.15.45:0/3358082704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc48013f900 msgr2=0x7fc4800cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 --2- 172.21.15.45:0/3358082704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc48013f900 0x7fc4800cc840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4700099b0 tx=0x7fc470009250 comp rx=0 tx=0).stop 2023-10-06T05:15:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 -- 172.21.15.45:0/3358082704 shutdown_connections 2023-10-06T05:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 --2- 172.21.15.45:0/3358082704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc48013f900 0x7fc4800cc840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 --2- 172.21.15.45:0/3358082704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4800c8de0 0x7fc4800c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 -- 172.21.15.45:0/3358082704 >> 172.21.15.45:0/3358082704 conn(0x7fc4800691c0 msgr2=0x7fc4800cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 -- 172.21.15.45:0/3358082704 shutdown_connections 2023-10-06T05:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.934+0000 7fc48513e700 1 -- 172.21.15.45:0/3358082704 wait complete. 2023-10-06T05:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.935+0000 7fc48513e700 1 Processor -- start 2023-10-06T05:15:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.935+0000 7fc48513e700 1 -- start start 2023-10-06T05:15:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc48513e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800c8de0 0x7fc480155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc48513e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4800c9d50 0x7fc480135d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc48513e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48013f900 0x7fc4800c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc48513e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4801367e0 con 0x7fc4800c8de0 2023-10-06T05:15:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc48513e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc480136ac0 con 0x7fc4800c9d50 2023-10-06T05:15:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc48513e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc480136da0 con 0x7fc48013f900 2023-10-06T05:15:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc47ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800c8de0 0x7fc480155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc47ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800c8de0 0x7fc480155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47640/0 (socket says 172.21.15.45:47640) 2023-10-06T05:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc47ed9d700 1 -- 172.21.15.45:0/910110541 learned_addr learned my addr 172.21.15.45:0/910110541 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc47e59c700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4800c9d50 0x7fc480135d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc47f59e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48013f900 0x7fc4800c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc47ed9d700 1 -- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48013f900 msgr2=0x7fc4800c2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.936+0000 7fc47ed9d700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48013f900 0x7fc4800c2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc47ed9d700 1 -- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4800c9d50 msgr2=0x7fc480135d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc47ed9d700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4800c9d50 0x7fc480135d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc47ed9d700 1 -- 172.21.15.45:0/910110541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700094f0 con 0x7fc4800c8de0 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc47f59e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48013f900 0x7fc4800c2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc47ed9d700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800c8de0 0x7fc480155540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc48006e930 tx=0x7fc470017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc476ffd700 1 -- 172.21.15.45:0/910110541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470024440 con 0x7fc4800c8de0 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc476ffd700 1 -- 172.21.15.45:0/910110541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4700245a0 con 0x7fc4800c8de0 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc480137080 con 0x7fc4800c8de0 2023-10-06T05:15:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc476ffd700 1 -- 172.21.15.45:0/910110541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470024940 con 0x7fc4800c8de0 2023-10-06T05:15:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.937+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc480137e90 con 0x7fc4800c8de0 2023-10-06T05:15:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.938+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48013a920 con 0x7fc4800c8de0 2023-10-06T05:15:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.939+0000 7fc476ffd700 1 -- 172.21.15.45:0/910110541 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc47001a420 con 0x7fc4800c8de0 2023-10-06T05:15:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.939+0000 7fc476ffd700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc460066f30 0x7fc4600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.939+0000 7fc476ffd700 1 -- 172.21.15.45:0/910110541 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc4700866f0 con 0x7fc4800c8de0 2023-10-06T05:15:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.939+0000 7fc47e59c700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc460066f30 0x7fc4600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.940+0000 7fc47e59c700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc460066f30 0x7fc4600693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc480139460 tx=0x7fc468008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:26.943+0000 7fc476ffd700 1 -- 172.21.15.45:0/910110541 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc470053510 con 0x7fc4800c8de0 2023-10-06T05:15:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.147+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4800cbb10 con 0x7fc4800c8de0 2023-10-06T05:15:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.148+0000 7fc476ffd700 1 -- 172.21.15.45:0/910110541 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc470056b30 con 0x7fc4800c8de0 2023-10-06T05:15:27.149 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc460066f30 msgr2=0x7fc4600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc460066f30 0x7fc4600693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc480139460 tx=0x7fc468008040 comp rx=0 tx=0).stop 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800c8de0 msgr2=0x7fc480155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800c8de0 0x7fc480155540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc48006e930 tx=0x7fc470017000 comp rx=0 tx=0).stop 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 shutdown_connections 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc460066f30 0x7fc4600693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48013f900 0x7fc4800c2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800c8de0 0x7fc480155540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 --2- 172.21.15.45:0/910110541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4800c9d50 0x7fc480135d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 >> 172.21.15.45:0/910110541 conn(0x7fc4800691c0 msgr2=0x7fc4800c3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 shutdown_connections 2023-10-06T05:15:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.150+0000 7fc48513e700 1 -- 172.21.15.45:0/910110541 wait complete. 2023-10-06T05:15:27.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:27.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.297+0000 7fbfa1db7700 1 Processor -- start 2023-10-06T05:15:27.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.297+0000 7fbfa1db7700 1 -- start start 2023-10-06T05:15:27.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbfa1db7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c159520 0x7fbf9c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbfa1db7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbf9c15d3d0 0x7fbf9c067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbfa1db7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0c9c90 con 0x7fbf9c159150 2023-10-06T05:15:27.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbfa1db7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c15d8e0 con 0x7fbf9c15d3d0 2023-10-06T05:15:27.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbfa1db7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c15dbc0 con 0x7fbf9c159520 2023-10-06T05:15:27.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbf9b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf9c159150 0x7fbf9c0ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50360/0 (socket says 172.21.15.45:50360) 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbf9b7fe700 1 -- 172.21.15.45:0/1489965292 learned_addr learned my addr 172.21.15.45:0/1489965292 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/1489965292 >> v2:172.21.15.144:3300/0 conn(0x7fbf9c15d3d0 0x7fbf9c067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbf9affd700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c159520 0x7fbf9c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.298+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174536238 0 0) 0x7fbf9c0c9c90 con 0x7fbf9c159150 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf88003570 con 0x7fbf9c159150 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2438829676 0 0) 0x7fbf88003570 con 0x7fbf9c159150 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c159520 msgr2=0x7fbf9c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c159520 0x7fbf9c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 >> v2:172.21.15.144:3300/0 conn(0x7fbf9c15d3d0 msgr2=0x7fbf9c067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 --2- 172.21.15.45:0/1489965292 >> v2:172.21.15.144:3300/0 conn(0x7fbf9c15d3d0 0x7fbf9c067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0c9920 con 0x7fbf9c159150 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fbf8c001470 con 0x7fbf9c159150 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/1489965292 >> v2:172.21.15.144:3300/0 conn(0x7fbf9c15d3d0 0x7fbf9c067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9affd700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c159520 0x7fbf9c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 >> v1:172.21.15.45:6789/0 conn(0x7fbf9c159150 legacy=0x7fbf9c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88003a60 0x7fbf88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf88004810 0x7fbf88004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf88005140 0x7fbf8800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c15d8e0 con 0x7fbf88005140 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c15dbc0 con 0x7fbf88003a60 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88004670 con 0x7fbf88004810 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf88005140 0x7fbf8800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9b7fe700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88003a60 0x7fbf88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9affd700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf88004810 0x7fbf88004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9bfff700 1 -- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf88004810 msgr2=0x7fbf88004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf88004810 0x7fbf88004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9bfff700 1 -- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88003a60 msgr2=0x7fbf88003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88003a60 0x7fbf88003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.299+0000 7fbf9bfff700 1 -- 172.21.15.45:0/1489965292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf84009420 con 0x7fbf88005140 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf88005140 0x7fbf8800ba60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c15d810 tx=0x7fbf9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a360cf3e440b56b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400ec10 con 0x7fbf88005140 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9400f040 con 0x7fbf88005140 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbf9a7fc700 1 -- 172.21.15.45:0/1489965292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94014480 con 0x7fbf88005140 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbfa1db7700 1 -- 172.21.15.45:0/1489965292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf88005140 msgr2=0x7fbf8800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbfa1db7700 1 --2- 172.21.15.45:0/1489965292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf88005140 0x7fbf8800ba60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c15d810 tx=0x7fbf9400b040 comp rx=0 tx=0).stop 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbf9bfff700 1 -- 172.21.15.45:0/1489965292 reap_dead start 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbfa1db7700 1 -- 172.21.15.45:0/1489965292 shutdown_connections 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbfa1db7700 1 -- 172.21.15.45:0/1489965292 >> 172.21.15.45:0/1489965292 conn(0x7fbf9c0691c0 msgr2=0x7fbf9c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbfa1db7700 1 -- 172.21.15.45:0/1489965292 shutdown_connections 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.300+0000 7fbfa1db7700 1 -- 172.21.15.45:0/1489965292 wait complete. 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.302+0000 7fbfa1db7700 1 Processor -- start 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.302+0000 7fbfa1db7700 1 -- start start 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.302+0000 7fbfa1db7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88005140 0x7fbf9c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbfa1db7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0b1d20 0x7fbf9c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbfa1db7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c0cba20 0x7fbf9c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbfa1db7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0b2aa0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbfa1db7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c16cfd0 con 0x7fbf88005140 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbfa1db7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0b2200 con 0x7fbf9c0cba20 2023-10-06T05:15:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0b1d20 0x7fbf9c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0b1d20 0x7fbf9c0cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47652/0 (socket says 172.21.15.45:47652) 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 -- 172.21.15.45:0/3151104723 learned_addr learned my addr 172.21.15.45:0/3151104723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9b7fe700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88005140 0x7fbf9c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c0cba20 0x7fbf9c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 -- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c0cba20 msgr2=0x7fbf9c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c0cba20 0x7fbf9c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 -- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88005140 msgr2=0x7fbf9c0b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88005140 0x7fbf9c0b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 -- 172.21.15.45:0/3151104723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0b24e0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9b7fe700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88005140 0x7fbf9c0b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf9affd700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0b1d20 0x7fbf9c0cb4e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf84011da0 tx=0x7fbf840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf937fe700 1 -- 172.21.15.45:0/3151104723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400fcd0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf937fe700 1 -- 172.21.15.45:0/3151104723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8400fe30 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c0b27c0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.303+0000 7fbf937fe700 1 -- 172.21.15.45:0/3151104723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8401d6d0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.304+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c153890 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.305+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c16cda0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.305+0000 7fbf937fe700 1 -- 172.21.15.45:0/3151104723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf8401da80 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.305+0000 7fbf937fe700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.305+0000 7fbf937fe700 1 -- 172.21.15.45:0/3151104723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbf84018070 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.306+0000 7fbf9b7fe700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.306+0000 7fbf9b7fe700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbf7c066f30 0x7fbf7c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000e10 tx=0x7fbf8c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.310+0000 7fbf937fe700 1 -- 172.21.15.45:0/3151104723 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf9c16cda0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.513+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf9c16cda0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.513+0000 7fbf937fe700 1 -- 172.21.15.45:0/3151104723 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf9c16cda0 con 0x7fbf9c0b1d20 2023-10-06T05:15:27.515 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbf7c066f30 msgr2=0x7fbf7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbf7c066f30 0x7fbf7c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000e10 tx=0x7fbf8c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0b1d20 msgr2=0x7fbf9c0cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0b1d20 0x7fbf9c0cb4e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf84011da0 tx=0x7fbf840091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 shutdown_connections 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf9c0cba20 0x7fbf9c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88005140 0x7fbf9c0b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 --2- 172.21.15.45:0/3151104723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0b1d20 0x7fbf9c0cb4e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 >> 172.21.15.45:0/3151104723 conn(0x7fbf9c0691c0 msgr2=0x7fbf9c0c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:27.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 shutdown_connections 2023-10-06T05:15:27.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.516+0000 7fbfa1db7700 1 -- 172.21.15.45:0/3151104723 wait complete. 2023-10-06T05:15:27.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T05:15:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.664+0000 7f3d89a94700 1 Processor -- start 2023-10-06T05:15:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.664+0000 7f3d89a94700 1 -- start start 2023-10-06T05:15:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d89a94700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d840b0610 0x7f3d840c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d89a94700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d840ce6a0 con 0x7f3d84069ed0 2023-10-06T05:15:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d89a94700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d84159880 con 0x7f3d840c3db0 2023-10-06T05:15:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d89a94700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d840690f0 con 0x7f3d840b0610 2023-10-06T05:15:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d837fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d840b0610 0x7f3d840c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d83fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3d84069ed0 0x7f3d840cc350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50364/0 (socket says 172.21.15.45:50364) 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d83fff700 1 -- 172.21.15.45:0/2609594307 learned_addr learned my addr 172.21.15.45:0/2609594307 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d837fe700 1 -- 172.21.15.45:0/2609594307 >> v1:172.21.15.144:6789/0 conn(0x7f3d840c3db0 legacy=0x7f3d840697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d837fe700 1 -- 172.21.15.45:0/2609594307 >> v1:172.21.15.45:6789/0 conn(0x7f3d84069ed0 legacy=0x7f3d840cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d837fe700 1 -- 172.21.15.45:0/2609594307 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840692e0 con 0x7f3d840b0610 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d827fc700 1 -- 172.21.15.45:0/2609594307 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932077182 0 0) 0x7f3d840ce6a0 con 0x7f3d84069ed0 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.665+0000 7f3d837fe700 1 --2- 172.21.15.45:0/2609594307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d840b0610 0x7f3d840c85d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d740098d0 tx=0x7f3d74009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca95022f4f340edf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d827fc700 1 -- 172.21.15.45:0/2609594307 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74009e70 con 0x7f3d840b0610 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d827fc700 1 -- 172.21.15.45:0/2609594307 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74005c10 con 0x7f3d840b0610 2023-10-06T05:15:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d827fc700 1 -- 172.21.15.45:0/2609594307 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400b650 con 0x7f3d840b0610 2023-10-06T05:15:27.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d89a94700 1 -- 172.21.15.45:0/2609594307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d840b0610 msgr2=0x7f3d840c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d89a94700 1 --2- 172.21.15.45:0/2609594307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d840b0610 0x7f3d840c85d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d740098d0 tx=0x7f3d74009120 comp rx=0 tx=0).stop 2023-10-06T05:15:27.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d89a94700 1 -- 172.21.15.45:0/2609594307 shutdown_connections 2023-10-06T05:15:27.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d89a94700 1 --2- 172.21.15.45:0/2609594307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d840b0610 0x7f3d840c85d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d89a94700 1 -- 172.21.15.45:0/2609594307 >> 172.21.15.45:0/2609594307 conn(0x7f3d84067e00 msgr2=0x7f3d840cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:27.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d89a94700 1 -- 172.21.15.45:0/2609594307 shutdown_connections 2023-10-06T05:15:27.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.666+0000 7f3d89a94700 1 -- 172.21.15.45:0/2609594307 wait complete. 2023-10-06T05:15:27.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.668+0000 7f3d89a94700 1 Processor -- start 2023-10-06T05:15:27.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.668+0000 7f3d89a94700 1 -- start start 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.668+0000 7f3d89a94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d84069ed0 0x7f3d840b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.668+0000 7f3d89a94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d840b0610 0x7f3d840c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d89a94700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d840c3db0 0x7f3d8416ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d89a94700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8415b260 con 0x7f3d84069ed0 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d89a94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8415b850 con 0x7f3d840b0610 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d89a94700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8416d8d0 con 0x7f3d840c3db0 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d84069ed0 0x7f3d840b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d84069ed0 0x7f3d840b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47674/0 (socket says 172.21.15.45:47674) 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 -- 172.21.15.45:0/1251996359 learned_addr learned my addr 172.21.15.45:0/1251996359 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d83fff700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d840c3db0 0x7f3d8416ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d82ffd700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d840b0610 0x7f3d840c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 -- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d840c3db0 msgr2=0x7f3d8416ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d840c3db0 0x7f3d8416ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 -- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d840b0610 msgr2=0x7f3d840c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d840b0610 0x7f3d840c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 -- 172.21.15.45:0/1251996359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740093c0 con 0x7f3d84069ed0 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d83fff700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d840c3db0 0x7f3d8416ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d82ffd700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d840b0610 0x7f3d840c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d837fe700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d84069ed0 0x7f3d840b30d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009160 tx=0x7f3d74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d7b7fe700 1 -- 172.21.15.45:0/1251996359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74006460 con 0x7f3d84069ed0 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d7b7fe700 1 -- 172.21.15.45:0/1251996359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d740065c0 con 0x7f3d84069ed0 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d7b7fe700 1 -- 172.21.15.45:0/1251996359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7401b480 con 0x7f3d84069ed0 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.669+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8415baf0 con 0x7f3d84069ed0 2023-10-06T05:15:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.670+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8416dbb0 con 0x7f3d84069ed0 2023-10-06T05:15:27.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.670+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8406e350 con 0x7f3d84069ed0 2023-10-06T05:15:27.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.671+0000 7f3d7b7fe700 1 -- 172.21.15.45:0/1251996359 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d7401b830 con 0x7f3d84069ed0 2023-10-06T05:15:27.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.671+0000 7f3d7b7fe700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:27.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.671+0000 7f3d7b7fe700 1 -- 172.21.15.45:0/1251996359 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3d74086c60 con 0x7f3d84069ed0 2023-10-06T05:15:27.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.672+0000 7f3d82ffd700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:27.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.672+0000 7f3d82ffd700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d64066f30 0x7f3d640693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c007fd0 tx=0x7f3d6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.676+0000 7f3d7b7fe700 1 -- 172.21.15.45:0/1251996359 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d740539e0 con 0x7f3d84069ed0 2023-10-06T05:15:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.879+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3d8404bb40 con 0x7f3d84069ed0 2023-10-06T05:15:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.879+0000 7f3d7b7fe700 1 -- 172.21.15.45:0/1251996359 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3d74057000 con 0x7f3d84069ed0 2023-10-06T05:15:27.881 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d64066f30 msgr2=0x7f3d640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d64066f30 0x7f3d640693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c007fd0 tx=0x7f3d6c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d84069ed0 msgr2=0x7f3d840b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d84069ed0 0x7f3d840b30d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009160 tx=0x7f3d74016000 comp rx=0 tx=0).stop 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 shutdown_connections 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d840c3db0 0x7f3d8416ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d84069ed0 0x7f3d840b30d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 --2- 172.21.15.45:0/1251996359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d840b0610 0x7f3d840c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:27.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 >> 172.21.15.45:0/1251996359 conn(0x7f3d84067e00 msgr2=0x7f3d84158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 shutdown_connections 2023-10-06T05:15:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:27.882+0000 7f3d89a94700 1 -- 172.21.15.45:0/1251996359 wait complete. 2023-10-06T05:15:27.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.030+0000 7f50403bd700 1 Processor -- start 2023-10-06T05:15:28.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.030+0000 7f50403bd700 1 -- start start 2023-10-06T05:15:28.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f50403bd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5038153650 0x7f503815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f50403bd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5038153a20 0x7f503806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f50403bd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380676c0 con 0x7f503806add0 2023-10-06T05:15:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f50403bd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503806b270 con 0x7f5038153650 2023-10-06T05:15:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f50403bd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50380679a0 con 0x7f5038153a20 2023-10-06T05:15:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e95a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f503806add0 0x7f5038068480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50380/0 (socket says 172.21.15.45:50380) 2023-10-06T05:15:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e159700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5038153650 0x7f503815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e95a700 1 -- 172.21.15.45:0/2296706755 learned_addr learned my addr 172.21.15.45:0/2296706755 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503d958700 1 --2- 172.21.15.45:0/2296706755 >> v2:172.21.15.144:3300/0 conn(0x7f5038153a20 0x7f503806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e159700 1 -- 172.21.15.45:0/2296706755 >> v2:172.21.15.144:3300/0 conn(0x7f5038153a20 msgr2=0x7f503806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e159700 1 --2- 172.21.15.45:0/2296706755 >> v2:172.21.15.144:3300/0 conn(0x7f5038153a20 0x7f503806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e159700 1 -- 172.21.15.45:0/2296706755 >> v1:172.21.15.45:6789/0 conn(0x7f503806add0 legacy=0x7f5038068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e159700 1 -- 172.21.15.45:0/2296706755 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5038067c80 con 0x7f5038153650 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503d157700 1 -- 172.21.15.45:0/2296706755 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620752183 0 0) 0x7f50380676c0 con 0x7f503806add0 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.031+0000 7f503e159700 1 --2- 172.21.15.45:0/2296706755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153650 0x7f503815a930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50380b2660 tx=0x7f50340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98023ddb167c3a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f503d157700 1 -- 172.21.15.45:0/2296706755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034011630 con 0x7f5038153650 2023-10-06T05:15:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f503d157700 1 -- 172.21.15.45:0/2296706755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034011790 con 0x7f5038153650 2023-10-06T05:15:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f503d157700 1 -- 172.21.15.45:0/2296706755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034011b30 con 0x7f5038153650 2023-10-06T05:15:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 -- 172.21.15.45:0/2296706755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153650 msgr2=0x7f503815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 --2- 172.21.15.45:0/2296706755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153650 0x7f503815a930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50380b2660 tx=0x7f50340091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 -- 172.21.15.45:0/2296706755 shutdown_connections 2023-10-06T05:15:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 --2- 172.21.15.45:0/2296706755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153650 0x7f503815a930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 --2- 172.21.15.45:0/2296706755 >> v2:172.21.15.144:3300/0 conn(0x7f5038153a20 0x7f503806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 -- 172.21.15.45:0/2296706755 >> 172.21.15.45:0/2296706755 conn(0x7f50380cc5a0 msgr2=0x7f50380cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 -- 172.21.15.45:0/2296706755 shutdown_connections 2023-10-06T05:15:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.032+0000 7f50403bd700 1 -- 172.21.15.45:0/2296706755 wait complete. 2023-10-06T05:15:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 Processor -- start 2023-10-06T05:15:28.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 -- start start 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f503806add0 0x7f5038155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5038153650 0x7f5038156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153a20 0x7f50380c1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50380c13c0 con 0x7f5038153a20 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503816ac10 con 0x7f503806add0 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.034+0000 7f50403bd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5038156d00 con 0x7f5038153650 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153a20 0x7f50380c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153a20 0x7f50380c1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47702/0 (socket says 172.21.15.45:47702) 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 -- 172.21.15.45:0/189136608 learned_addr learned my addr 172.21.15.45:0/189136608 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e159700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f503806add0 0x7f5038155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503d958700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5038153650 0x7f5038156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 -- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5038153650 msgr2=0x7f5038156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5038153650 0x7f5038156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 -- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f503806add0 msgr2=0x7f5038155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f503806add0 0x7f5038155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 -- 172.21.15.45:0/189136608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5034009420 con 0x7f5038153a20 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f503e95a700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153a20 0x7f50380c1850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f503000cbf0 tx=0x7f50300019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f502e7fc700 1 -- 172.21.15.45:0/189136608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503000a8b0 con 0x7f5038153a20 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f502e7fc700 1 -- 172.21.15.45:0/189136608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503000aa10 con 0x7f5038153a20 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f502e7fc700 1 -- 172.21.15.45:0/189136608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503000add0 con 0x7f5038153a20 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5038156740 con 0x7f5038153a20 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.035+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50380c10d0 con 0x7f5038153a20 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.036+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50380c1570 con 0x7f5038153a20 2023-10-06T05:15:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.037+0000 7f502e7fc700 1 -- 172.21.15.45:0/189136608 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5030007300 con 0x7f5038153a20 2023-10-06T05:15:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.037+0000 7f502e7fc700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f501c066f30 0x7f501c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.037+0000 7f502e7fc700 1 -- 172.21.15.45:0/189136608 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5030084c90 con 0x7f5038153a20 2023-10-06T05:15:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.037+0000 7f503e159700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f501c066f30 0x7f501c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.038+0000 7f503e159700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f501c066f30 0x7f501c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50340093f0 tx=0x7f5034017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.042+0000 7f502e7fc700 1 -- 172.21.15.45:0/189136608 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50380c1570 con 0x7f5038153a20 2023-10-06T05:15:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.293+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50380cc0e0 con 0x7f5038153a20 2023-10-06T05:15:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.294+0000 7f502e7fc700 1 -- 172.21.15.45:0/189136608 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f5030051b30 con 0x7f5038153a20 2023-10-06T05:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f501c066f30 msgr2=0x7f501c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f501c066f30 0x7f501c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50340093f0 tx=0x7f5034017000 comp rx=0 tx=0).stop 2023-10-06T05:15:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153a20 msgr2=0x7f50380c1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153a20 0x7f50380c1850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f503000cbf0 tx=0x7f50300019b0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 shutdown_connections 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f501c066f30 0x7f501c0693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5038153a20 0x7f50380c1850 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f503806add0 0x7f5038155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 --2- 172.21.15.45:0/189136608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5038153650 0x7f5038156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 >> 172.21.15.45:0/189136608 conn(0x7f50380cc5a0 msgr2=0x7f50380cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 shutdown_connections 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.296+0000 7f50403bd700 1 -- 172.21.15.45:0/189136608 wait complete. 2023-10-06T05:15:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:28.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:28.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.442+0000 7f2e444d3700 1 Processor -- start 2023-10-06T05:15:28.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.442+0000 7f2e444d3700 1 -- start start 2023-10-06T05:15:28.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.443+0000 7f2e444d3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2e3c158f60 0x7f2e3c159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.443+0000 7f2e444d3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c0cb530 con 0x7f2e3c0cc390 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.443+0000 7f2e444d3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c069bd0 con 0x7f2e3c06a060 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.443+0000 7f2e444d3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c06a500 con 0x7f2e3c158f60 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.443+0000 7f2e42a70700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2e3c0cc390 0x7f2e3c0c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50394/0 (socket says 172.21.15.45:50394) 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.443+0000 7f2e42a70700 1 -- 172.21.15.45:0/4163992534 learned_addr learned my addr 172.21.15.45:0/4163992534 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4226f700 1 --2- 172.21.15.45:0/4163992534 >> v2:172.21.15.144:3300/0 conn(0x7f2e3c158f60 0x7f2e3c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4126d700 1 -- 172.21.15.45:0/4163992534 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592052904 0 0) 0x7f2e34001490 con 0x7f2e3c0cc390 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4126d700 1 -- 172.21.15.45:0/4163992534 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e28003570 con 0x7f2e3c0cc390 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4226f700 1 -- 172.21.15.45:0/4163992534 >> v1:172.21.15.144:6789/0 conn(0x7f2e3c06a060 legacy=0x7f2e3c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4226f700 1 -- 172.21.15.45:0/4163992534 >> v1:172.21.15.45:6789/0 conn(0x7f2e3c0cc390 legacy=0x7f2e3c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4126d700 1 -- 172.21.15.45:0/4163992534 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398968 0 0) 0x7f2e3c069bd0 con 0x7f2e3c06a060 2023-10-06T05:15:28.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4226f700 1 -- 172.21.15.45:0/4163992534 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c068fc0 con 0x7f2e3c158f60 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4226f700 1 --2- 172.21.15.45:0/4163992534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 0x7f2e3c159330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2e300098e0 tx=0x7f2e300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6af0fa8439dbac18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4126d700 1 -- 172.21.15.45:0/4163992534 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000b640 con 0x7f2e3c158f60 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4126d700 1 -- 172.21.15.45:0/4163992534 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3000b7a0 con 0x7f2e3c158f60 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.444+0000 7f2e4126d700 1 -- 172.21.15.45:0/4163992534 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000bb40 con 0x7f2e3c158f60 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.445+0000 7f2e444d3700 1 -- 172.21.15.45:0/4163992534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 msgr2=0x7f2e3c159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.445+0000 7f2e444d3700 1 --2- 172.21.15.45:0/4163992534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 0x7f2e3c159330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2e300098e0 tx=0x7f2e300091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.445+0000 7f2e444d3700 1 -- 172.21.15.45:0/4163992534 shutdown_connections 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.445+0000 7f2e444d3700 1 --2- 172.21.15.45:0/4163992534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 0x7f2e3c159330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.445+0000 7f2e444d3700 1 -- 172.21.15.45:0/4163992534 >> 172.21.15.45:0/4163992534 conn(0x7f2e3c0cdf50 msgr2=0x7f2e3c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.445+0000 7f2e444d3700 1 -- 172.21.15.45:0/4163992534 shutdown_connections 2023-10-06T05:15:28.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.445+0000 7f2e444d3700 1 -- 172.21.15.45:0/4163992534 wait complete. 2023-10-06T05:15:28.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.446+0000 7f2e444d3700 1 Processor -- start 2023-10-06T05:15:28.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.446+0000 7f2e444d3700 1 -- start start 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e444d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c06a060 0x7f2e3c144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e444d3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e3c0cc390 0x7f2e3c169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e444d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 0x7f2e3c0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e444d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c144f90 con 0x7f2e3c06a060 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e444d3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c16a1e0 con 0x7f2e3c158f60 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e444d3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c16a4c0 con 0x7f2e3c0cc390 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e4226f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c06a060 0x7f2e3c144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e4226f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c06a060 0x7f2e3c144a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47718/0 (socket says 172.21.15.45:47718) 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e4226f700 1 -- 172.21.15.45:0/2680836044 learned_addr learned my addr 172.21.15.45:0/2680836044 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e42a70700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 0x7f2e3c0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.447+0000 7f2e41a6e700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e3c0cc390 0x7f2e3c169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e4226f700 1 -- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e3c0cc390 msgr2=0x7f2e3c169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e4226f700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e3c0cc390 0x7f2e3c169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e4226f700 1 -- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 msgr2=0x7f2e3c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e4226f700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 0x7f2e3c0b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e4226f700 1 -- 172.21.15.45:0/2680836044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30009420 con 0x7f2e3c06a060 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e4226f700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c06a060 0x7f2e3c144a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0c4f80 tx=0x7f2e30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e2e7fc700 1 -- 172.21.15.45:0/2680836044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3001f070 con 0x7f2e3c06a060 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e2e7fc700 1 -- 172.21.15.45:0/2680836044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e30002f70 con 0x7f2e3c06a060 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c0b20a0 con 0x7f2e3c06a060 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e2e7fc700 1 -- 172.21.15.45:0/2680836044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000bb40 con 0x7f2e3c06a060 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.448+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c153fe0 con 0x7f2e3c06a060 2023-10-06T05:15:28.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.449+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c0b1f00 con 0x7f2e3c06a060 2023-10-06T05:15:28.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.449+0000 7f2e2e7fc700 1 -- 172.21.15.45:0/2680836044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e30011630 con 0x7f2e3c06a060 2023-10-06T05:15:28.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.450+0000 7f2e2e7fc700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e20066f30 0x7f2e200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.450+0000 7f2e2e7fc700 1 -- 172.21.15.45:0/2680836044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2e30085b70 con 0x7f2e3c06a060 2023-10-06T05:15:28.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.450+0000 7f2e41a6e700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e20066f30 0x7f2e200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.451+0000 7f2e41a6e700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e20066f30 0x7f2e200693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0c6ec0 tx=0x7f2e38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.454+0000 7f2e2e7fc700 1 -- 172.21.15.45:0/2680836044 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e3c0b1f00 con 0x7f2e3c06a060 2023-10-06T05:15:28.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.656+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e3c0b1f00 con 0x7f2e3c06a060 2023-10-06T05:15:28.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.657+0000 7f2e2e7fc700 1 -- 172.21.15.45:0/2680836044 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f2e3c0b1f00 con 0x7f2e3c06a060 2023-10-06T05:15:28.659 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:28.659 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e20066f30 msgr2=0x7f2e200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e20066f30 0x7f2e200693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0c6ec0 tx=0x7f2e38009040 comp rx=0 tx=0).stop 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c06a060 msgr2=0x7f2e3c144a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c06a060 0x7f2e3c144a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0c4f80 tx=0x7f2e30017000 comp rx=0 tx=0).stop 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 shutdown_connections 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e20066f30 0x7f2e200693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3c158f60 0x7f2e3c0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e3c06a060 0x7f2e3c144a50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 --2- 172.21.15.45:0/2680836044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e3c0cc390 0x7f2e3c169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 >> 172.21.15.45:0/2680836044 conn(0x7f2e3c0cdf50 msgr2=0x7f2e3c158130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 shutdown_connections 2023-10-06T05:15:28.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.659+0000 7f2e444d3700 1 -- 172.21.15.45:0/2680836044 wait complete. 2023-10-06T05:15:28.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:57.253547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:58.338382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T05:15:01.988547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:28.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.808+0000 7fec3c1f9700 1 Processor -- start 2023-10-06T05:15:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.808+0000 7fec3c1f9700 1 -- start start 2023-10-06T05:15:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3c1f9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec3406ee30 0x7fec34068c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3c1f9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec34067be0 con 0x7fec34067620 2023-10-06T05:15:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3c1f9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec34067ec0 con 0x7fec34155e60 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3c1f9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340681a0 con 0x7fec3406ee30 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec39794700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fec34067620 0x7fec3406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50398/0 (socket says 172.21.15.45:50398) 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec39794700 1 -- 172.21.15.45:0/2914985722 learned_addr learned my addr 172.21.15.45:0/2914985722 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3a796700 1 --2- 172.21.15.45:0/2914985722 >> v2:172.21.15.144:3300/0 conn(0x7fec3406ee30 0x7fec34068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec38f93700 1 -- 172.21.15.45:0/2914985722 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661650686 0 0) 0x7fec34067be0 con 0x7fec34067620 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec38f93700 1 -- 172.21.15.45:0/2914985722 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec20003570 con 0x7fec34067620 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec38f93700 1 -- 172.21.15.45:0/2914985722 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160546378 0 0) 0x7fec34067ec0 con 0x7fec34155e60 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec38f93700 1 -- 172.21.15.45:0/2914985722 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec34067be0 con 0x7fec34155e60 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3a796700 1 -- 172.21.15.45:0/2914985722 >> v1:172.21.15.144:6789/0 conn(0x7fec34155e60 legacy=0x7fec340ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3a796700 1 -- 172.21.15.45:0/2914985722 >> v1:172.21.15.45:6789/0 conn(0x7fec34067620 legacy=0x7fec3406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.809+0000 7fec3a796700 1 -- 172.21.15.45:0/2914985722 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34068480 con 0x7fec3406ee30 2023-10-06T05:15:28.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3a796700 1 --2- 172.21.15.45:0/2914985722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3406ee30 0x7fec34068c80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009e60 tx=0x7fec2c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b94b278f06afe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec38f93700 1 -- 172.21.15.45:0/2914985722 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00bab0 con 0x7fec3406ee30 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec38f93700 1 -- 172.21.15.45:0/2914985722 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c00bc10 con 0x7fec3406ee30 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec38f93700 1 -- 172.21.15.45:0/2914985722 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c011460 con 0x7fec3406ee30 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3c1f9700 1 -- 172.21.15.45:0/2914985722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3406ee30 msgr2=0x7fec34068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/2914985722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3406ee30 0x7fec34068c80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009e60 tx=0x7fec2c009780 comp rx=0 tx=0).stop 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3c1f9700 1 -- 172.21.15.45:0/2914985722 shutdown_connections 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/2914985722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3406ee30 0x7fec34068c80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3c1f9700 1 -- 172.21.15.45:0/2914985722 >> 172.21.15.45:0/2914985722 conn(0x7fec340cc570 msgr2=0x7fec340cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3c1f9700 1 -- 172.21.15.45:0/2914985722 shutdown_connections 2023-10-06T05:15:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.810+0000 7fec3c1f9700 1 -- 172.21.15.45:0/2914985722 wait complete. 2023-10-06T05:15:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 Processor -- start 2023-10-06T05:15:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 -- start start 2023-10-06T05:15:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec34067620 0x7fec340c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3406ee30 0x7fec340afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec34155e60 0x7fec340b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec341651c0 con 0x7fec3406ee30 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec340c26d0 con 0x7fec34067620 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec3c1f9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340c29b0 con 0x7fec34155e60 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.812+0000 7fec39794700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3406ee30 0x7fec340afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3406ee30 0x7fec340afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47724/0 (socket says 172.21.15.45:47724) 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 -- 172.21.15.45:0/856664707 learned_addr learned my addr 172.21.15.45:0/856664707 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39f95700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec34067620 0x7fec340c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec3a796700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec34155e60 0x7fec340b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 -- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec34155e60 msgr2=0x7fec340b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec34155e60 0x7fec340b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 -- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec34067620 msgr2=0x7fec340c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec34067620 0x7fec340c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 -- 172.21.15.45:0/856664707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c009a20 con 0x7fec3406ee30 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec39794700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3406ee30 0x7fec340afbb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c790 tx=0x7fec24007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec29ffb700 1 -- 172.21.15.45:0/856664707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24005b30 con 0x7fec3406ee30 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec29ffb700 1 -- 172.21.15.45:0/856664707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24005cb0 con 0x7fec3406ee30 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec340c2c90 con 0x7fec3406ee30 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec29ffb700 1 -- 172.21.15.45:0/856664707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24007320 con 0x7fec3406ee30 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.813+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec340c23e0 con 0x7fec3406ee30 2023-10-06T05:15:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.814+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec341653e0 con 0x7fec3406ee30 2023-10-06T05:15:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.815+0000 7fec29ffb700 1 -- 172.21.15.45:0/856664707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec2401b3b0 con 0x7fec3406ee30 2023-10-06T05:15:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.815+0000 7fec29ffb700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.815+0000 7fec29ffb700 1 -- 172.21.15.45:0/856664707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fec24083e90 con 0x7fec3406ee30 2023-10-06T05:15:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.816+0000 7fec39f95700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.816+0000 7fec39f95700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec30003870 tx=0x7fec30009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:28.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:28.820+0000 7fec29ffb700 1 -- 172.21.15.45:0/856664707 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec341653e0 con 0x7fec3406ee30 2023-10-06T05:15:29.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.023+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec341653e0 con 0x7fec3406ee30 2023-10-06T05:15:29.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.023+0000 7fec29ffb700 1 -- 172.21.15.45:0/856664707 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec341653e0 con 0x7fec3406ee30 2023-10-06T05:15:29.025 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 msgr2=0x7fec180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec30003870 tx=0x7fec30009040 comp rx=0 tx=0).stop 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3406ee30 msgr2=0x7fec340afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3406ee30 0x7fec340afbb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c790 tx=0x7fec24007ea0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 shutdown_connections 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec34155e60 0x7fec340b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec34067620 0x7fec340c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 --2- 172.21.15.45:0/856664707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec3406ee30 0x7fec340afbb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 >> 172.21.15.45:0/856664707 conn(0x7fec340cc570 msgr2=0x7fec340b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 shutdown_connections 2023-10-06T05:15:29.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.026+0000 7fec3c1f9700 1 -- 172.21.15.45:0/856664707 wait complete. 2023-10-06T05:15:29.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.174+0000 7f98b9e32700 1 Processor -- start 2023-10-06T05:15:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.174+0000 7f98b9e32700 1 -- start start 2023-10-06T05:15:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.175+0000 7f98b9e32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b40ccb70 0x7f98b40aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.175+0000 7f98b9e32700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f98b40c8b70 0x7f98b40c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.175+0000 7f98b9e32700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b40c3b50 con 0x7f98b4157330 2023-10-06T05:15:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.175+0000 7f98b9e32700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b4157700 con 0x7f98b40c8b70 2023-10-06T05:15:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.175+0000 7f98b9e32700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40c98e0 con 0x7f98b40ccb70 2023-10-06T05:15:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b40ccb70 0x7f98b40aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f98b40c8b70 0x7f98b40c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b40ccb70 0x7f98b40aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34498/0 (socket says 172.21.15.45:34498) 2023-10-06T05:15:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f98b40c8b70 0x7f98b40c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:39782/0 (socket says 172.21.15.45:39782) 2023-10-06T05:15:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b2ffd700 1 -- 172.21.15.45:0/720003157 learned_addr learned my addr 172.21.15.45:0/720003157 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b37fe700 1 --1- 172.21.15.45:0/720003157 >> v1:172.21.15.144:6789/0 conn(0x7f98b4157330 0x7f98b40ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:58128/0 (socket says 172.21.15.45:58128) 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b3fff700 1 -- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b40ccb70 msgr2=0x7f98b40aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b3fff700 1 --2- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b40ccb70 0x7f98b40aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b3fff700 1 -- 172.21.15.45:0/720003157 >> v1:172.21.15.144:6789/0 conn(0x7f98b4157330 legacy=0x7f98b40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b27fc700 1 -- 172.21.15.45:0/720003157 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050299864 0 0) 0x7f98b40c3b50 con 0x7f98b4157330 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b3fff700 1 -- 172.21.15.45:0/720003157 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40c9bc0 con 0x7f98b40c8b70 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b2ffd700 1 --2- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b40ccb70 0x7f98b40aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b3fff700 1 --2- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 0x7f98b40c8ff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00f300 tx=0x7f98ac00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3005876435e20376 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b27fc700 1 -- 172.21.15.45:0/720003157 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac012070 con 0x7f98b40c8b70 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b27fc700 1 -- 172.21.15.45:0/720003157 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac00d0d0 con 0x7f98b40c8b70 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.176+0000 7f98b27fc700 1 -- 172.21.15.45:0/720003157 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00d490 con 0x7f98b40c8b70 2023-10-06T05:15:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 -- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 msgr2=0x7f98b40c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 --2- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 0x7f98b40c8ff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00f300 tx=0x7f98ac00db90 comp rx=0 tx=0).stop 2023-10-06T05:15:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 -- 172.21.15.45:0/720003157 shutdown_connections 2023-10-06T05:15:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 --2- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 0x7f98b40c8ff0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 --2- 172.21.15.45:0/720003157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b40ccb70 0x7f98b40aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 -- 172.21.15.45:0/720003157 >> 172.21.15.45:0/720003157 conn(0x7f98b4067e00 msgr2=0x7f98b40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 -- 172.21.15.45:0/720003157 shutdown_connections 2023-10-06T05:15:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.177+0000 7f98b9e32700 1 -- 172.21.15.45:0/720003157 wait complete. 2023-10-06T05:15:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.178+0000 7f98b9e32700 1 Processor -- start 2023-10-06T05:15:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b9e32700 1 -- start start 2023-10-06T05:15:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b9e32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 0x7f98b40b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b9e32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b40ccb70 0x7f98b4158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b9e32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b4157330 0x7f98b4153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b9e32700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40b3650 con 0x7f98b40ccb70 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b9e32700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4159d60 con 0x7f98b40c8b70 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b9e32700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4158eb0 con 0x7f98b4157330 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b40ccb70 0x7f98b4158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b40ccb70 0x7f98b4158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47728/0 (socket says 172.21.15.45:47728) 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.179+0000 7f98b2ffd700 1 -- 172.21.15.45:0/2393945092 learned_addr learned my addr 172.21.15.45:0/2393945092 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b37fe700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 0x7f98b40b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b3fff700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b4157330 0x7f98b4153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b2ffd700 1 -- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b4157330 msgr2=0x7f98b4153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b2ffd700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b4157330 0x7f98b4153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b2ffd700 1 -- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 msgr2=0x7f98b40b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b2ffd700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 0x7f98b40b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b2ffd700 1 -- 172.21.15.45:0/2393945092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0094f0 con 0x7f98b40ccb70 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b2ffd700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b40ccb70 0x7f98b4158970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f989c00ec50 tx=0x7f989c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98ab7fe700 1 -- 172.21.15.45:0/2393945092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00c850 con 0x7f98b40ccb70 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98ab7fe700 1 -- 172.21.15.45:0/2393945092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00c9b0 con 0x7f98b40ccb70 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac00de30 con 0x7f98b40ccb70 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98ab7fe700 1 -- 172.21.15.45:0/2393945092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00cd50 con 0x7f98b40ccb70 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.180+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b4159430 con 0x7f98b40ccb70 2023-10-06T05:15:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.181+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b4159ef0 con 0x7f98b40ccb70 2023-10-06T05:15:29.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.182+0000 7f98ab7fe700 1 -- 172.21.15.45:0/2393945092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f989c013680 con 0x7f98b40ccb70 2023-10-06T05:15:29.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.182+0000 7f98ab7fe700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9894066f30 0x7f98940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.182+0000 7f98ab7fe700 1 -- 172.21.15.45:0/2393945092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f989c084c30 con 0x7f98b40ccb70 2023-10-06T05:15:29.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.182+0000 7f98b37fe700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9894066f30 0x7f98940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.183+0000 7f98b37fe700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9894066f30 0x7f98940693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98a4001230 tx=0x7f98a4007d20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.187+0000 7f98ab7fe700 1 -- 172.21.15.45:0/2393945092 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f989c051a50 con 0x7f98b40ccb70 2023-10-06T05:15:29.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.390+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98b406e350 con 0x7f98b40ccb70 2023-10-06T05:15:29.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.390+0000 7f98ab7fe700 1 -- 172.21.15.45:0/2393945092 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f989c055070 con 0x7f98b40ccb70 2023-10-06T05:15:29.392 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:15:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9894066f30 msgr2=0x7f98940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9894066f30 0x7f98940693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98a4001230 tx=0x7f98a4007d20 comp rx=0 tx=0).stop 2023-10-06T05:15:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b40ccb70 msgr2=0x7f98b4158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b40ccb70 0x7f98b4158970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f989c00ec50 tx=0x7f989c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 shutdown_connections 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9894066f30 0x7f98940693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b4157330 0x7f98b4153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b40c8b70 0x7f98b40b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 --2- 172.21.15.45:0/2393945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b40ccb70 0x7f98b4158970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 >> 172.21.15.45:0/2393945092 conn(0x7f98b4067e00 msgr2=0x7f98b40c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 shutdown_connections 2023-10-06T05:15:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.392+0000 7f98b9e32700 1 -- 172.21.15.45:0/2393945092 wait complete. 2023-10-06T05:15:29.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.541+0000 7fd53d43c700 1 Processor -- start 2023-10-06T05:15:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.541+0000 7fd53d43c700 1 -- start start 2023-10-06T05:15:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd53d43c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5380d0680 0x7fd53815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd53d43c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd53815c6f0 0x7fd5380caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd53d43c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5380cd740 con 0x7fd5380d02b0 2023-10-06T05:15:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd53d43c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5380d0ad0 con 0x7fd5380d0680 2023-10-06T05:15:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd53d43c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5380ccb30 con 0x7fd53815c6f0 2023-10-06T05:15:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd5367fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5380d0680 0x7fd53815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd536ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd5380d02b0 0x7fd53815d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50412/0 (socket says 172.21.15.45:50412) 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd536ffd700 1 -- 172.21.15.45:0/2776760036 learned_addr learned my addr 172.21.15.45:0/2776760036 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd5377fe700 1 --2- 172.21.15.45:0/2776760036 >> v2:172.21.15.144:3300/0 conn(0x7fd53815c6f0 0x7fd5380caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd5367fc700 1 -- 172.21.15.45:0/2776760036 >> v2:172.21.15.144:3300/0 conn(0x7fd53815c6f0 msgr2=0x7fd5380caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd5367fc700 1 --2- 172.21.15.45:0/2776760036 >> v2:172.21.15.144:3300/0 conn(0x7fd53815c6f0 0x7fd5380caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd5367fc700 1 -- 172.21.15.45:0/2776760036 >> v1:172.21.15.45:6789/0 conn(0x7fd5380d02b0 legacy=0x7fd53815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd5367fc700 1 -- 172.21.15.45:0/2776760036 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380cce10 con 0x7fd5380d0680 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd535ffb700 1 -- 172.21.15.45:0/2776760036 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887183191 0 0) 0x7fd5380cd740 con 0x7fd5380d02b0 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd5367fc700 1 --2- 172.21.15.45:0/2776760036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d0680 0x7fd53815c1b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5280098e0 tx=0x7fd5280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1c6268ba91947c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.542+0000 7fd535ffb700 1 -- 172.21.15.45:0/2776760036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528009e80 con 0x7fd5380d0680 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd535ffb700 1 -- 172.21.15.45:0/2776760036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52800b820 con 0x7fd5380d0680 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd535ffb700 1 -- 172.21.15.45:0/2776760036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52800bba0 con 0x7fd5380d0680 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 -- 172.21.15.45:0/2776760036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d0680 msgr2=0x7fd53815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 --2- 172.21.15.45:0/2776760036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d0680 0x7fd53815c1b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5280098e0 tx=0x7fd5280091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 -- 172.21.15.45:0/2776760036 shutdown_connections 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 --2- 172.21.15.45:0/2776760036 >> v2:172.21.15.144:3300/0 conn(0x7fd53815c6f0 0x7fd5380caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 --2- 172.21.15.45:0/2776760036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d0680 0x7fd53815c1b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 -- 172.21.15.45:0/2776760036 >> 172.21.15.45:0/2776760036 conn(0x7fd53806ed50 msgr2=0x7fd53806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 -- 172.21.15.45:0/2776760036 shutdown_connections 2023-10-06T05:15:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.543+0000 7fd53d43c700 1 -- 172.21.15.45:0/2776760036 wait complete. 2023-10-06T05:15:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 Processor -- start 2023-10-06T05:15:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 -- start start 2023-10-06T05:15:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d02b0 0x7fd538158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5380d0680 0x7fd5380a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd53815c6f0 0x7fd5380a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53816a500 con 0x7fd5380d02b0 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c7170 con 0x7fd5380d0680 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd53d43c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c7fd0 con 0x7fd53815c6f0 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.545+0000 7fd536ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d02b0 0x7fd538158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d02b0 0x7fd538158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47750/0 (socket says 172.21.15.45:47750) 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 -- 172.21.15.45:0/1021764004 learned_addr learned my addr 172.21.15.45:0/1021764004 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd5367fc700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5380d0680 0x7fd5380a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd5377fe700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd53815c6f0 0x7fd5380a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 -- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd53815c6f0 msgr2=0x7fd5380a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd53815c6f0 0x7fd5380a78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 -- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5380d0680 msgr2=0x7fd5380a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5380d0680 0x7fd5380a73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 -- 172.21.15.45:0/1021764004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd528009420 con 0x7fd5380d02b0 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd536ffd700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d02b0 0x7fd538158fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c810 tx=0x7fd520007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd52effd700 1 -- 172.21.15.45:0/1021764004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520005b30 con 0x7fd5380d02b0 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd52effd700 1 -- 172.21.15.45:0/1021764004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd520005c90 con 0x7fd5380d02b0 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5380c76a0 con 0x7fd5380d02b0 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd52effd700 1 -- 172.21.15.45:0/1021764004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520007300 con 0x7fd5380d02b0 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.546+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5380c7c60 con 0x7fd5380d02b0 2023-10-06T05:15:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.547+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd538088730 con 0x7fd5380d02b0 2023-10-06T05:15:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.548+0000 7fd52effd700 1 -- 172.21.15.45:0/1021764004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd52001b3b0 con 0x7fd5380d02b0 2023-10-06T05:15:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.548+0000 7fd52effd700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd518066f30 0x7fd5180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.549+0000 7fd52effd700 1 -- 172.21.15.45:0/1021764004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd520083e30 con 0x7fd5380d02b0 2023-10-06T05:15:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.549+0000 7fd5367fc700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd518066f30 0x7fd5180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.549+0000 7fd5367fc700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd518066f30 0x7fd5180693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5280093f0 tx=0x7fd528017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.553+0000 7fd52effd700 1 -- 172.21.15.45:0/1021764004 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd520050c50 con 0x7fd5380d02b0 2023-10-06T05:15:29.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.753+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5380cd0f0 con 0x7fd5380d02b0 2023-10-06T05:15:29.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.753+0000 7fd52effd700 1 -- 172.21.15.45:0/1021764004 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd520001d60 con 0x7fd5380d02b0 2023-10-06T05:15:29.755 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:29.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd518066f30 msgr2=0x7fd5180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd518066f30 0x7fd5180693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5280093f0 tx=0x7fd528017000 comp rx=0 tx=0).stop 2023-10-06T05:15:29.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d02b0 msgr2=0x7fd538158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d02b0 0x7fd538158fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c810 tx=0x7fd520007df0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 shutdown_connections 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd518066f30 0x7fd5180693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd53815c6f0 0x7fd5380a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5380d02b0 0x7fd538158fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 --2- 172.21.15.45:0/1021764004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5380d0680 0x7fd5380a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 >> 172.21.15.45:0/1021764004 conn(0x7fd53806ed50 msgr2=0x7fd53815a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.755+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 shutdown_connections 2023-10-06T05:15:29.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.756+0000 7fd53d43c700 1 -- 172.21.15.45:0/1021764004 wait complete. 2023-10-06T05:15:29.771 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:29.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.904+0000 7f98d7f24700 1 Processor -- start 2023-10-06T05:15:29.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.904+0000 7f98d7f24700 1 -- start start 2023-10-06T05:15:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d7f24700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d00adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d7f24700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98d00c9730 0x7f98d00c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d7f24700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00ca0c0 con 0x7f98d006bf20 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d7f24700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d00adff0 con 0x7f98d00c9730 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d7f24700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00684a0 con 0x7f98d013f0a0 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d54bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98d00c9730 0x7f98d00c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d64c1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f98d006bf20 0x7f98d006c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50422/0 (socket says 172.21.15.45:50422) 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.905+0000 7f98d64c1700 1 -- 172.21.15.45:0/1577904952 learned_addr learned my addr 172.21.15.45:0/1577904952 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d5cc0700 1 --2- 172.21.15.45:0/1577904952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d00adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d54bf700 1 -- 172.21.15.45:0/1577904952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 msgr2=0x7f98d00adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d54bf700 1 --2- 172.21.15.45:0/1577904952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d00adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d54bf700 1 -- 172.21.15.45:0/1577904952 >> v1:172.21.15.45:6789/0 conn(0x7f98d006bf20 legacy=0x7f98d006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d4cbe700 1 -- 172.21.15.45:0/1577904952 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372155693 0 0) 0x7f98d00ca0c0 con 0x7f98d006bf20 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d54bf700 1 -- 172.21.15.45:0/1577904952 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d0068780 con 0x7f98d00c9730 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d54bf700 1 --2- 172.21.15.45:0/1577904952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d00c9730 0x7f98d00c9b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0098e0 tx=0x7f98cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63f60d29ce07c7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d4cbe700 1 -- 172.21.15.45:0/1577904952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00b640 con 0x7f98d00c9730 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d4cbe700 1 -- 172.21.15.45:0/1577904952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc00b7a0 con 0x7f98d00c9730 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.906+0000 7f98d4cbe700 1 -- 172.21.15.45:0/1577904952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00bb40 con 0x7f98d00c9730 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 -- 172.21.15.45:0/1577904952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d00c9730 msgr2=0x7f98d00c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1577904952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d00c9730 0x7f98d00c9b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0098e0 tx=0x7f98cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 -- 172.21.15.45:0/1577904952 shutdown_connections 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1577904952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d00adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1577904952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d00c9730 0x7f98d00c9b80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 -- 172.21.15.45:0/1577904952 >> 172.21.15.45:0/1577904952 conn(0x7f98d00cd080 msgr2=0x7f98d00c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 -- 172.21.15.45:0/1577904952 shutdown_connections 2023-10-06T05:15:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.907+0000 7f98d7f24700 1 -- 172.21.15.45:0/1577904952 wait complete. 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.908+0000 7f98d7f24700 1 Processor -- start 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.908+0000 7f98d7f24700 1 -- start start 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d7f24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d006bf20 0x7f98d0158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d7f24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d00c9730 0x7f98d0158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d7f24700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d0153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d7f24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d013d680 con 0x7f98d006bf20 2023-10-06T05:15:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d7f24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d013ca70 con 0x7f98d00c9730 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d7f24700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d013cd50 con 0x7f98d013f0a0 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d006bf20 0x7f98d0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d006bf20 0x7f98d0158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47774/0 (socket says 172.21.15.45:47774) 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 -- 172.21.15.45:0/1381377966 learned_addr learned my addr 172.21.15.45:0/1381377966 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d64c1700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d0153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d54bf700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d00c9730 0x7f98d0158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 -- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 msgr2=0x7f98d0153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d0153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 -- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d00c9730 msgr2=0x7f98d0158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d00c9730 0x7f98d0158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d5cc0700 1 -- 172.21.15.45:0/1381377966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009420 con 0x7f98d006bf20 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.909+0000 7f98d54bf700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d00c9730 0x7f98d0158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.910+0000 7f98d5cc0700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d006bf20 0x7f98d0158010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e930 tx=0x7f98c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.910+0000 7f98c1ffb700 1 -- 172.21.15.45:0/1381377966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4013680 con 0x7f98d006bf20 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.910+0000 7f98c1ffb700 1 -- 172.21.15.45:0/1381377966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c40137e0 con 0x7f98d006bf20 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.910+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d013d030 con 0x7f98d006bf20 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.910+0000 7f98c1ffb700 1 -- 172.21.15.45:0/1381377966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4013b80 con 0x7f98d006bf20 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.910+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d0131030 con 0x7f98d006bf20 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.911+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d006da90 con 0x7f98d006bf20 2023-10-06T05:15:29.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.912+0000 7f98c1ffb700 1 -- 172.21.15.45:0/1381377966 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98c400c460 con 0x7f98d006bf20 2023-10-06T05:15:29.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.912+0000 7f98c1ffb700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:29.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.912+0000 7f98c1ffb700 1 -- 172.21.15.45:0/1381377966 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f98c4084af0 con 0x7f98d006bf20 2023-10-06T05:15:29.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.912+0000 7f98d54bf700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:29.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.913+0000 7f98d54bf700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98b4066f30 0x7f98b40693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0093f0 tx=0x7f98cc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:29.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:29.917+0000 7f98c1ffb700 1 -- 172.21.15.45:0/1381377966 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98c4051910 con 0x7f98d006bf20 2023-10-06T05:15:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.120+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98d00ca030 con 0x7f98d006bf20 2023-10-06T05:15:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.120+0000 7f98c1ffb700 1 -- 172.21.15.45:0/1381377966 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98c4054f30 con 0x7f98d006bf20 2023-10-06T05:15:30.122 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:15:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98b4066f30 msgr2=0x7f98b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98b4066f30 0x7f98b40693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0093f0 tx=0x7f98cc017000 comp rx=0 tx=0).stop 2023-10-06T05:15:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d006bf20 msgr2=0x7f98d0158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d006bf20 0x7f98d0158010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e930 tx=0x7f98c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 shutdown_connections 2023-10-06T05:15:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d013f0a0 0x7f98d0153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98d006bf20 0x7f98d0158010 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 --2- 172.21.15.45:0/1381377966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d00c9730 0x7f98d0158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 >> 172.21.15.45:0/1381377966 conn(0x7f98d00cd080 msgr2=0x7f98d00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 shutdown_connections 2023-10-06T05:15:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.123+0000 7f98d7f24700 1 -- 172.21.15.45:0/1381377966 wait complete. 2023-10-06T05:15:30.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T05:15:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.271+0000 7f656d20e700 1 Processor -- start 2023-10-06T05:15:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.271+0000 7f656d20e700 1 -- start start 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f656d20e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65680c9f20 0x7f6568160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f656d20e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65680ccca0 con 0x7f656813f900 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f656d20e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65680c9630 con 0x7f65680cc840 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f656d20e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65680cb550 con 0x7f65680c9f20 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f6566d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f656813f900 0x7f65680c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50436/0 (socket says 172.21.15.45:50436) 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f6566d9d700 1 -- 172.21.15.45:0/4274056471 learned_addr learned my addr 172.21.15.45:0/4274056471 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f656759e700 1 --2- 172.21.15.45:0/4274056471 >> v2:172.21.15.144:3300/0 conn(0x7f65680c9f20 0x7f6568160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f656659c700 1 -- 172.21.15.45:0/4274056471 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926040516 0 0) 0x7f6558001110 con 0x7f656813f900 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.272+0000 7f656659c700 1 -- 172.21.15.45:0/4274056471 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6554003570 con 0x7f656813f900 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.273+0000 7f656759e700 1 -- 172.21.15.45:0/4274056471 >> v1:172.21.15.144:6789/0 conn(0x7f65680cc840 legacy=0x7f65680c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.273+0000 7f656759e700 1 -- 172.21.15.45:0/4274056471 >> v1:172.21.15.45:6789/0 conn(0x7f656813f900 legacy=0x7f65680c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.273+0000 7f656759e700 1 -- 172.21.15.45:0/4274056471 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680cb830 con 0x7f65680c9f20 2023-10-06T05:15:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.273+0000 7f656759e700 1 --2- 172.21.15.45:0/4274056471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680c9f20 0x7f6568160c90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e60 tx=0x7f6560009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce350db9415f067b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.273+0000 7f656659c700 1 -- 172.21.15.45:0/4274056471 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000e070 con 0x7f65680c9f20 2023-10-06T05:15:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.273+0000 7f656659c700 1 -- 172.21.15.45:0/4274056471 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656000bb60 con 0x7f65680c9f20 2023-10-06T05:15:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.273+0000 7f656659c700 1 -- 172.21.15.45:0/4274056471 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560012430 con 0x7f65680c9f20 2023-10-06T05:15:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.274+0000 7f656d20e700 1 -- 172.21.15.45:0/4274056471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680c9f20 msgr2=0x7f6568160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.274+0000 7f656d20e700 1 --2- 172.21.15.45:0/4274056471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680c9f20 0x7f6568160c90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6560009e60 tx=0x7f6560009780 comp rx=0 tx=0).stop 2023-10-06T05:15:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.274+0000 7f656d20e700 1 -- 172.21.15.45:0/4274056471 shutdown_connections 2023-10-06T05:15:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.274+0000 7f656d20e700 1 --2- 172.21.15.45:0/4274056471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680c9f20 0x7f6568160c90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.274+0000 7f656d20e700 1 -- 172.21.15.45:0/4274056471 >> 172.21.15.45:0/4274056471 conn(0x7f65680691c0 msgr2=0x7f65680cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.274+0000 7f656d20e700 1 -- 172.21.15.45:0/4274056471 shutdown_connections 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.274+0000 7f656d20e700 1 -- 172.21.15.45:0/4274056471 wait complete. 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.275+0000 7f656d20e700 1 Processor -- start 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.275+0000 7f656d20e700 1 -- start start 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f656d20e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9f20 0x7f65680c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f656d20e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680cc840 0x7f65681552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f656d20e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f656813f900 0x7f6568155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f656d20e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681661e0 con 0x7f65680c9f20 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f656d20e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681689c0 con 0x7f65680cc840 2023-10-06T05:15:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f656d20e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6568168ca0 con 0x7f656813f900 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9f20 0x7f65680c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9f20 0x7f65680c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47780/0 (socket says 172.21.15.45:47780) 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 -- 172.21.15.45:0/1251780508 learned_addr learned my addr 172.21.15.45:0/1251780508 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f656759e700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f656813f900 0x7f6568155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f655e59c700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680cc840 0x7f65681552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 -- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f656813f900 msgr2=0x7f6568155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f656813f900 0x7f6568155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 -- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680cc840 msgr2=0x7f65681552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680cc840 0x7f65681552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 -- 172.21.15.45:0/1251780508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560009a20 con 0x7f65680c9f20 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f655e59c700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680cc840 0x7f65681552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.276+0000 7f6566d9d700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9f20 0x7f65680c1e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f655800c590 tx=0x7f6558007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.277+0000 7f655f7fe700 1 -- 172.21.15.45:0/1251780508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558005c00 con 0x7f65680c9f20 2023-10-06T05:15:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.277+0000 7f655f7fe700 1 -- 172.21.15.45:0/1251780508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6558005d80 con 0x7f65680c9f20 2023-10-06T05:15:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.277+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6568168f80 con 0x7f65680c9f20 2023-10-06T05:15:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.277+0000 7f655f7fe700 1 -- 172.21.15.45:0/1251780508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558007320 con 0x7f65680c9f20 2023-10-06T05:15:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.277+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65680c2370 con 0x7f65680c9f20 2023-10-06T05:15:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.278+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65681694a0 con 0x7f65680c9f20 2023-10-06T05:15:30.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.278+0000 7f655f7fe700 1 -- 172.21.15.45:0/1251780508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f655801b3b0 con 0x7f65680c9f20 2023-10-06T05:15:30.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.279+0000 7f655f7fe700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6548066f30 0x7f65480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.279+0000 7f655f7fe700 1 -- 172.21.15.45:0/1251780508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6558083da0 con 0x7f65680c9f20 2023-10-06T05:15:30.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.279+0000 7f655e59c700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6548066f30 0x7f65480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.279+0000 7f655e59c700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6548066f30 0x7f65480693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6550007fd0 tx=0x7f6550009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.283+0000 7f655f7fe700 1 -- 172.21.15.45:0/1251780508 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65681694a0 con 0x7f65680c9f20 2023-10-06T05:15:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.483+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f65680ccca0 con 0x7f65680c9f20 2023-10-06T05:15:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.483+0000 7f655f7fe700 1 -- 172.21.15.45:0/1251780508 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f65680ccca0 con 0x7f65680c9f20 2023-10-06T05:15:30.485 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6548066f30 msgr2=0x7f65480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6548066f30 0x7f65480693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6550007fd0 tx=0x7f6550009040 comp rx=0 tx=0).stop 2023-10-06T05:15:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9f20 msgr2=0x7f65680c1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9f20 0x7f65680c1e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f655800c590 tx=0x7f6558007c70 comp rx=0 tx=0).stop 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 shutdown_connections 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6548066f30 0x7f65480693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f656813f900 0x7f6568155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9f20 0x7f65680c1e30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 --2- 172.21.15.45:0/1251780508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65680cc840 0x7f65681552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 >> 172.21.15.45:0/1251780508 conn(0x7f65680691c0 msgr2=0x7f65680c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 shutdown_connections 2023-10-06T05:15:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.485+0000 7f656d20e700 1 -- 172.21.15.45:0/1251780508 wait complete. 2023-10-06T05:15:30.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.631+0000 7f68d4ae3700 1 Processor -- start 2023-10-06T05:15:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.631+0000 7f68d4ae3700 1 -- start start 2023-10-06T05:15:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68d4ae3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0159200 0x7f68d00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68d4ae3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f68d01595d0 0x7f68d006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68d4ae3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68d006b5c0 0x7f68d00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68d4ae3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00c7780 con 0x7f68d006b5c0 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68d4ae3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00c7d00 con 0x7f68d01595d0 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68d4ae3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00c86d0 con 0x7f68d0159200 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68d006b5c0 0x7f68d00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68d006b5c0 0x7f68d00ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47784/0 (socket says 172.21.15.45:47784) 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 -- 172.21.15.45:0/2672964203 learned_addr learned my addr 172.21.15.45:0/2672964203 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ce59c700 1 --2- 172.21.15.45:0/2672964203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0159200 0x7f68d00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68cdd9b700 1 --2- 172.21.15.45:0/2672964203 >> v2:172.21.15.144:3300/0 conn(0x7f68d01595d0 0x7f68d006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 -- 172.21.15.45:0/2672964203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0159200 msgr2=0x7f68d00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 --2- 172.21.15.45:0/2672964203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0159200 0x7f68d00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 -- 172.21.15.45:0/2672964203 >> v2:172.21.15.144:3300/0 conn(0x7f68d01595d0 msgr2=0x7f68d006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 --2- 172.21.15.45:0/2672964203 >> v2:172.21.15.144:3300/0 conn(0x7f68d01595d0 0x7f68d006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 -- 172.21.15.45:0/2672964203 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d006bb40 con 0x7f68d006b5c0 2023-10-06T05:15:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.632+0000 7f68ced9d700 1 --2- 172.21.15.45:0/2672964203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d006b5c0 0x7f68d00ccdc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68d006ac20 tx=0x7f68c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d536bae3a377db74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68cd59a700 1 -- 172.21.15.45:0/2672964203 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800e070 con 0x7f68d006b5c0 2023-10-06T05:15:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68cd59a700 1 -- 172.21.15.45:0/2672964203 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c800b970 con 0x7f68d006b5c0 2023-10-06T05:15:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68cd59a700 1 -- 172.21.15.45:0/2672964203 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80092e0 con 0x7f68d006b5c0 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2672964203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d006b5c0 msgr2=0x7f68d00ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2672964203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d006b5c0 0x7f68d00ccdc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68d006ac20 tx=0x7f68c8009840 comp rx=0 tx=0).stop 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2672964203 shutdown_connections 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2672964203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d006b5c0 0x7f68d00ccdc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2672964203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0159200 0x7f68d00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2672964203 >> v2:172.21.15.144:3300/0 conn(0x7f68d01595d0 0x7f68d006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2672964203 >> 172.21.15.45:0/2672964203 conn(0x7f68d006f090 msgr2=0x7f68d006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2672964203 shutdown_connections 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.633+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2672964203 wait complete. 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.634+0000 7f68d4ae3700 1 Processor -- start 2023-10-06T05:15:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.635+0000 7f68d4ae3700 1 -- start start 2023-10-06T05:15:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.635+0000 7f68d4ae3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006b5c0 0x7f68d00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68d4ae3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0159200 0x7f68d00b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68d4ae3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d01595d0 0x7f68d00b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68d4ae3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00b1f40 con 0x7f68d01595d0 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68d4ae3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00b1030 con 0x7f68d0159200 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68d4ae3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d016a900 con 0x7f68d006b5c0 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d01595d0 0x7f68d00b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d01595d0 0x7f68d00b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47792/0 (socket says 172.21.15.45:47792) 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 -- 172.21.15.45:0/2673897810 learned_addr learned my addr 172.21.15.45:0/2673897810 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ce59c700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006b5c0 0x7f68d00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68cdd9b700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0159200 0x7f68d00b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 -- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006b5c0 msgr2=0x7f68d00cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006b5c0 0x7f68d00cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 -- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0159200 msgr2=0x7f68d00b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0159200 0x7f68d00b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 -- 172.21.15.45:0/2673897810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8009ae0 con 0x7f68d01595d0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68ced9d700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d01595d0 0x7f68d00b1a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68d006aed0 tx=0x7f68c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68be7fc700 1 -- 172.21.15.45:0/2673897810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8008ad0 con 0x7f68d01595d0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.636+0000 7f68be7fc700 1 -- 172.21.15.45:0/2673897810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c8008c50 con 0x7f68d01595d0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.637+0000 7f68be7fc700 1 -- 172.21.15.45:0/2673897810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8026460 con 0x7f68d01595d0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.637+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d00b0700 con 0x7f68d01595d0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.637+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d00b0cc0 con 0x7f68d01595d0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.638+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d00b11e0 con 0x7f68d01595d0 2023-10-06T05:15:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.638+0000 7f68be7fc700 1 -- 172.21.15.45:0/2673897810 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68c8003040 con 0x7f68d01595d0 2023-10-06T05:15:30.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.638+0000 7f68be7fc700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68b0066f30 0x7f68b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:30.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.638+0000 7f68be7fc700 1 -- 172.21.15.45:0/2673897810 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f68c8086160 con 0x7f68d01595d0 2023-10-06T05:15:30.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.639+0000 7f68ce59c700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68b0066f30 0x7f68b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:30.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.639+0000 7f68ce59c700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68b0066f30 0x7f68b00693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f68d00af810 tx=0x7f68c00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:30.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.643+0000 7f68be7fc700 1 -- 172.21.15.45:0/2673897810 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68d00b11e0 con 0x7f68d01595d0 2023-10-06T05:15:30.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.899+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68d00b0e50 con 0x7f68d01595d0 2023-10-06T05:15:30.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.900+0000 7f68be7fc700 1 -- 172.21.15.45:0/2673897810 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f68c8053030 con 0x7f68d01595d0 2023-10-06T05:15:30.902 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:30.902 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:30.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68b0066f30 msgr2=0x7f68b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68b0066f30 0x7f68b00693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f68d00af810 tx=0x7f68c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d01595d0 msgr2=0x7f68d00b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d01595d0 0x7f68d00b1a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68d006aed0 tx=0x7f68c8018000 comp rx=0 tx=0).stop 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 shutdown_connections 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68b0066f30 0x7f68b00693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d01595d0 0x7f68d00b1a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006b5c0 0x7f68d00cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 --2- 172.21.15.45:0/2673897810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0159200 0x7f68d00b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.902+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 >> 172.21.15.45:0/2673897810 conn(0x7f68d006f090 msgr2=0x7f68d00c9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.903+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 shutdown_connections 2023-10-06T05:15:30.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:30.903+0000 7f68d4ae3700 1 -- 172.21.15.45:0/2673897810 wait complete. 2023-10-06T05:15:30.906 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:30.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:31.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.049+0000 7fb6eb91a700 1 Processor -- start 2023-10-06T05:15:31.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.049+0000 7fb6eb91a700 1 -- start start 2023-10-06T05:15:31.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6eb91a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6e41570b0 0x7fb6e406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6eb91a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6e415ac10 0x7fb6e40ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6eb91a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e415b380 con 0x7fb6e4156ce0 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6eb91a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e406c010 con 0x7fb6e41570b0 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6eb91a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e40b02f0 con 0x7fb6e415ac10 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6e8eb5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6e41570b0 0x7fb6e406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6e96b6700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb6e4156ce0 0x7fb6e40afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50444/0 (socket says 172.21.15.45:50444) 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6e96b6700 1 -- 172.21.15.45:0/3702234161 learned_addr learned my addr 172.21.15.45:0/3702234161 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.050+0000 7fb6e9eb7700 1 --2- 172.21.15.45:0/3702234161 >> v2:172.21.15.144:3300/0 conn(0x7fb6e415ac10 0x7fb6e40ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e8eb5700 1 -- 172.21.15.45:0/3702234161 >> v2:172.21.15.144:3300/0 conn(0x7fb6e415ac10 msgr2=0x7fb6e40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e8eb5700 1 --2- 172.21.15.45:0/3702234161 >> v2:172.21.15.144:3300/0 conn(0x7fb6e415ac10 0x7fb6e40ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e8eb5700 1 -- 172.21.15.45:0/3702234161 >> v1:172.21.15.45:6789/0 conn(0x7fb6e4156ce0 legacy=0x7fb6e40afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e8eb5700 1 -- 172.21.15.45:0/3702234161 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e415b660 con 0x7fb6e41570b0 2023-10-06T05:15:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e3fff700 1 -- 172.21.15.45:0/3702234161 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726003312 0 0) 0x7fb6e415b380 con 0x7fb6e4156ce0 2023-10-06T05:15:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e8eb5700 1 --2- 172.21.15.45:0/3702234161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e41570b0 0x7fb6e406bad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40ae6d0 tx=0x7fb6d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91915a07754c0a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e3fff700 1 -- 172.21.15.45:0/3702234161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8011630 con 0x7fb6e41570b0 2023-10-06T05:15:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e3fff700 1 -- 172.21.15.45:0/3702234161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8011790 con 0x7fb6e41570b0 2023-10-06T05:15:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.051+0000 7fb6e3fff700 1 -- 172.21.15.45:0/3702234161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8011b30 con 0x7fb6e41570b0 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 -- 172.21.15.45:0/3702234161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e41570b0 msgr2=0x7fb6e406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/3702234161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e41570b0 0x7fb6e406bad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40ae6d0 tx=0x7fb6d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 -- 172.21.15.45:0/3702234161 shutdown_connections 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/3702234161 >> v2:172.21.15.144:3300/0 conn(0x7fb6e415ac10 0x7fb6e40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/3702234161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e41570b0 0x7fb6e406bad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 -- 172.21.15.45:0/3702234161 >> 172.21.15.45:0/3702234161 conn(0x7fb6e406f090 msgr2=0x7fb6e406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 -- 172.21.15.45:0/3702234161 shutdown_connections 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.052+0000 7fb6eb91a700 1 -- 172.21.15.45:0/3702234161 wait complete. 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.053+0000 7fb6eb91a700 1 Processor -- start 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.053+0000 7fb6eb91a700 1 -- start start 2023-10-06T05:15:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6eb91a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6e4156ce0 0x7fb6e40c0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6eb91a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6e41570b0 0x7fb6e4154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6eb91a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e415ac10 0x7fb6e4067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6eb91a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40c1510 con 0x7fb6e415ac10 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6eb91a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40abc70 con 0x7fb6e4156ce0 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6eb91a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4168480 con 0x7fb6e41570b0 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e415ac10 0x7fb6e4067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e415ac10 0x7fb6e4067f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47818/0 (socket says 172.21.15.45:47818) 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 -- 172.21.15.45:0/2188330817 learned_addr learned my addr 172.21.15.45:0/2188330817 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e96b6700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6e4156ce0 0x7fb6e40c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e8eb5700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6e41570b0 0x7fb6e4154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 -- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6e41570b0 msgr2=0x7fb6e4154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6e41570b0 0x7fb6e4154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 -- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6e4156ce0 msgr2=0x7fb6e40c0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6e4156ce0 0x7fb6e40c0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e9eb7700 1 -- 172.21.15.45:0/2188330817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8009420 con 0x7fb6e415ac10 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e96b6700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6e4156ce0 0x7fb6e40c0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.054+0000 7fb6e8eb5700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6e41570b0 0x7fb6e4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.055+0000 7fb6e9eb7700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e415ac10 0x7fb6e4067f70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00ea50 tx=0x7fb6dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.055+0000 7fb6e17fa700 1 -- 172.21.15.45:0/2188330817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00c920 con 0x7fb6e415ac10 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.055+0000 7fb6e17fa700 1 -- 172.21.15.45:0/2188330817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc00caa0 con 0x7fb6e415ac10 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.055+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e40abf10 con 0x7fb6e415ac10 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.055+0000 7fb6e17fa700 1 -- 172.21.15.45:0/2188330817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00ce60 con 0x7fb6e415ac10 2023-10-06T05:15:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.055+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e40c1230 con 0x7fb6e415ac10 2023-10-06T05:15:31.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.056+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e40ac0a0 con 0x7fb6e415ac10 2023-10-06T05:15:31.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.056+0000 7fb6e17fa700 1 -- 172.21.15.45:0/2188330817 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6dc013680 con 0x7fb6e415ac10 2023-10-06T05:15:31.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.057+0000 7fb6e17fa700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6d0066f60 0x7fb6d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.057+0000 7fb6e17fa700 1 -- 172.21.15.45:0/2188330817 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb6dc084d70 con 0x7fb6e415ac10 2023-10-06T05:15:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.057+0000 7fb6e96b6700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6d0066f60 0x7fb6d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.057+0000 7fb6e96b6700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6d0066f60 0x7fb6d0069410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400d150 tx=0x7fb6d4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.061+0000 7fb6e17fa700 1 -- 172.21.15.45:0/2188330817 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6dc051b90 con 0x7fb6e415ac10 2023-10-06T05:15:31.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.268+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6e40c6e70 con 0x7fb6e415ac10 2023-10-06T05:15:31.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.269+0000 7fb6e17fa700 1 -- 172.21.15.45:0/2188330817 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fb6dc0551b0 con 0x7fb6e415ac10 2023-10-06T05:15:31.271 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:31.271 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6d0066f60 msgr2=0x7fb6d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6d0066f60 0x7fb6d0069410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400d150 tx=0x7fb6d4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e415ac10 msgr2=0x7fb6e4067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e415ac10 0x7fb6e4067f70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00ea50 tx=0x7fb6dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 shutdown_connections 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6d0066f60 0x7fb6d0069410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e415ac10 0x7fb6e4067f70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6e4156ce0 0x7fb6e40c0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 --2- 172.21.15.45:0/2188330817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6e41570b0 0x7fb6e4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 >> 172.21.15.45:0/2188330817 conn(0x7fb6e406f090 msgr2=0x7fb6e41555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 shutdown_connections 2023-10-06T05:15:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.271+0000 7fb6eb91a700 1 -- 172.21.15.45:0/2188330817 wait complete. 2023-10-06T05:15:31.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.417+0000 7f6fd0ba0700 1 Processor -- start 2023-10-06T05:15:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.417+0000 7f6fd0ba0700 1 -- start start 2023-10-06T05:15:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.418+0000 7f6fd0ba0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6fcc158f60 0x7f6fcc159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fd0ba0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0cb530 con 0x7f6fcc06a060 2023-10-06T05:15:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fd0ba0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc069bd0 con 0x7f6fcc0cc390 2023-10-06T05:15:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fd0ba0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc06a500 con 0x7f6fcc158f60 2023-10-06T05:15:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fc9d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6fcc06a060 0x7f6fcc0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50460/0 (socket says 172.21.15.45:50460) 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fc9d9b700 1 -- 172.21.15.45:0/3311185818 learned_addr learned my addr 172.21.15.45:0/3311185818 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fca59c700 1 --2- 172.21.15.45:0/3311185818 >> v2:172.21.15.144:3300/0 conn(0x7f6fcc158f60 0x7f6fcc159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fc959a700 1 -- 172.21.15.45:0/3311185818 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545671218 0 0) 0x7f6fc0001110 con 0x7f6fcc06a060 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fc959a700 1 -- 172.21.15.45:0/3311185818 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb4003570 con 0x7f6fcc06a060 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fca59c700 1 -- 172.21.15.45:0/3311185818 >> v1:172.21.15.144:6789/0 conn(0x7f6fcc0cc390 legacy=0x7f6fcc0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fca59c700 1 -- 172.21.15.45:0/3311185818 >> v1:172.21.15.45:6789/0 conn(0x7f6fcc06a060 legacy=0x7f6fcc0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fc959a700 1 -- 172.21.15.45:0/3311185818 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445766205 0 0) 0x7f6fb4003570 con 0x7f6fcc06a060 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fca59c700 1 -- 172.21.15.45:0/3311185818 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc068fc0 con 0x7f6fcc158f60 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.419+0000 7f6fca59c700 1 --2- 172.21.15.45:0/3311185818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 0x7f6fcc159330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0098e0 tx=0x7f6fbc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a8a70b303c9ab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fc959a700 1 -- 172.21.15.45:0/3311185818 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0001600 con 0x7f6fcc158f60 2023-10-06T05:15:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fc959a700 1 -- 172.21.15.45:0/3311185818 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc011420 con 0x7f6fcc158f60 2023-10-06T05:15:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fc959a700 1 -- 172.21.15.45:0/3311185818 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0117c0 con 0x7f6fcc158f60 2023-10-06T05:15:31.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3311185818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 msgr2=0x7f6fcc159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3311185818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 0x7f6fcc159330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0098e0 tx=0x7f6fbc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3311185818 shutdown_connections 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3311185818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 0x7f6fcc159330 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3311185818 >> 172.21.15.45:0/3311185818 conn(0x7f6fcc0cdf50 msgr2=0x7f6fcc0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3311185818 shutdown_connections 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.420+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3311185818 wait complete. 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 Processor -- start 2023-10-06T05:15:31.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 -- start start 2023-10-06T05:15:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fcc06a060 0x7f6fcc1697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fcc0cc390 0x7f6fcc0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 0x7f6fcc0ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0a8f60 con 0x7f6fcc0cc390 2023-10-06T05:15:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc169ce0 con 0x7f6fcc158f60 2023-10-06T05:15:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.422+0000 7f6fd0ba0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0a8c10 con 0x7f6fcc06a060 2023-10-06T05:15:31.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fcc0cc390 0x7f6fcc0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fcc0cc390 0x7f6fcc0b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47828/0 (socket says 172.21.15.45:47828) 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 -- 172.21.15.45:0/3892683652 learned_addr learned my addr 172.21.15.45:0/3892683652 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fca59c700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fcc06a060 0x7f6fcc1697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fcad9d700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 0x7f6fcc0ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 -- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fcc06a060 msgr2=0x7f6fcc1697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fcc06a060 0x7f6fcc1697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 -- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 msgr2=0x7f6fcc0ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 0x7f6fcc0ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 -- 172.21.15.45:0/3892683652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc009420 con 0x7f6fcc0cc390 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fc9d9b700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fcc0cc390 0x7f6fcc0b01c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c980 tx=0x7f6fc00013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fba7fc700 1 -- 172.21.15.45:0/3892683652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0007360 con 0x7f6fcc0cc390 2023-10-06T05:15:31.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fba7fc700 1 -- 172.21.15.45:0/3892683652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc00074c0 con 0x7f6fcc0cc390 2023-10-06T05:15:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fba7fc700 1 -- 172.21.15.45:0/3892683652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0004830 con 0x7f6fcc0cc390 2023-10-06T05:15:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc169f80 con 0x7f6fcc0cc390 2023-10-06T05:15:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.423+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc16a540 con 0x7f6fcc0cc390 2023-10-06T05:15:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.425+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc0ab960 con 0x7f6fcc0cc390 2023-10-06T05:15:31.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.425+0000 7f6fba7fc700 1 -- 172.21.15.45:0/3892683652 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6fc0004be0 con 0x7f6fcc0cc390 2023-10-06T05:15:31.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.426+0000 7f6fba7fc700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:31.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.426+0000 7f6fba7fc700 1 -- 172.21.15.45:0/3892683652 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6fc0083ca0 con 0x7f6fcc0cc390 2023-10-06T05:15:31.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.426+0000 7f6fca59c700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:31.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.426+0000 7f6fca59c700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fac066f30 0x7f6fac0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0091e0 tx=0x7f6fbc016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:31.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.430+0000 7f6fba7fc700 1 -- 172.21.15.45:0/3892683652 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fc0050b40 con 0x7f6fcc0cc390 2023-10-06T05:15:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.684+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fcc0cb530 con 0x7f6fcc0cc390 2023-10-06T05:15:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.684+0000 7f6fba7fc700 1 -- 172.21.15.45:0/3892683652 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f6fcc0cb530 con 0x7f6fcc0cc390 2023-10-06T05:15:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:31.687 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:31.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fac066f30 msgr2=0x7f6fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fac066f30 0x7f6fac0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0091e0 tx=0x7f6fbc016000 comp rx=0 tx=0).stop 2023-10-06T05:15:31.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fcc0cc390 msgr2=0x7f6fcc0b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:31.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fcc0cc390 0x7f6fcc0b01c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c980 tx=0x7f6fc00013c0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 shutdown_connections 2023-10-06T05:15:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fcc158f60 0x7f6fcc0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fcc06a060 0x7f6fcc1697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 --2- 172.21.15.45:0/3892683652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fcc0cc390 0x7f6fcc0b01c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 >> 172.21.15.45:0/3892683652 conn(0x7f6fcc0cdf50 msgr2=0x7f6fcc0ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 shutdown_connections 2023-10-06T05:15:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:31.687+0000 7f6fd0ba0700 1 -- 172.21.15.45:0/3892683652 wait complete. 2023-10-06T05:15:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:32.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:14:59.369246+0000', 'modified': '2023-10-06T05:15:18.783356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6837/1292454917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 1292454917}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1292454917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:15:32.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:15:32.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:15:32.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.246+0000 7f4ee0a7f700 1 Processor -- start 2023-10-06T05:15:32.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.247+0000 7f4ee0a7f700 1 -- start start 2023-10-06T05:15:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ee0a7f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ee0a7f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4edc0c70f0 0x7f4edc0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ee0a7f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0ac5a0 con 0x7f4edc06f070 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ee0a7f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc159690 con 0x7f4edc0c70f0 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ee0a7f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0c78e0 con 0x7f4edc06afa0 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4edad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4edc0c70f0 0x7f4edc0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc0c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34584/0 (socket says 172.21.15.45:34584) 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4edad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4edc0c70f0 0x7f4edc0acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:39898/0 (socket says 172.21.15.45:39898) 2023-10-06T05:15:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/1635729846 learned_addr learned my addr 172.21.15.45:0/1635729846 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4eda59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4edc06f070 0x7f4edc06a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:58164/0 (socket says 172.21.15.45:58164) 2023-10-06T05:15:32.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/1635729846 >> v2:172.21.15.144:3300/0 conn(0x7f4edc0c70f0 msgr2=0x7f4edc0acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/1635729846 >> v2:172.21.15.144:3300/0 conn(0x7f4edc0c70f0 0x7f4edc0acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/1635729846 >> v1:172.21.15.144:6789/0 conn(0x7f4edc06f070 legacy=0x7f4edc06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/1635729846 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0c7bc0 con 0x7f4edc06afa0 2023-10-06T05:15:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.248+0000 7f4edad9d700 1 --2- 172.21.15.45:0/1635729846 >> v2:172.21.15.144:3300/0 conn(0x7f4edc0c70f0 0x7f4edc0acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/1635729846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc0c6bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00099b0 tx=0x7f4ed0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ba731c66d0466f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ed959a700 1 -- 172.21.15.45:0/1635729846 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0009ea0 con 0x7f4edc06afa0 2023-10-06T05:15:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ed959a700 1 -- 172.21.15.45:0/1635729846 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed000b860 con 0x7f4edc06afa0 2023-10-06T05:15:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ed959a700 1 -- 172.21.15.45:0/1635729846 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000bc00 con 0x7f4edc06afa0 2023-10-06T05:15:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/1635729846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 msgr2=0x7f4edc0c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/1635729846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc0c6bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00099b0 tx=0x7f4ed0009250 comp rx=0 tx=0).stop 2023-10-06T05:15:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/1635729846 shutdown_connections 2023-10-06T05:15:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/1635729846 >> v2:172.21.15.144:3300/0 conn(0x7f4edc0c70f0 0x7f4edc0acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/1635729846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc0c6bb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/1635729846 >> 172.21.15.45:0/1635729846 conn(0x7f4edc0cc5a0 msgr2=0x7f4edc0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/1635729846 shutdown_connections 2023-10-06T05:15:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.249+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/1635729846 wait complete. 2023-10-06T05:15:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 Processor -- start 2023-10-06T05:15:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 -- start start 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc06f070 0x7f4edc15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0c70f0 0x7f4edc0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0af8a0 con 0x7f4edc0c70f0 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0afb80 con 0x7f4edc06f070 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.251+0000 7f4ee0a7f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0afe60 con 0x7f4edc06afa0 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4edad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0c70f0 0x7f4edc0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:34588/0 (socket says 172.21.15.45:34588) 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4edad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0c70f0 0x7f4edc0c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47842/0 (socket says 172.21.15.45:47842) 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 -- 172.21.15.45:0/94588316 learned_addr learned my addr 172.21.15.45:0/94588316 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc06f070 0x7f4edc15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 -- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc06f070 msgr2=0x7f4edc15c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc06f070 0x7f4edc15c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 -- 172.21.15.45:0/94588316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0c70f0 msgr2=0x7f4edc0c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0c70f0 0x7f4edc0c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 -- 172.21.15.45:0/94588316 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4009a20 con 0x7f4edc06afa0 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eda59c700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc157bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00c7c0 tx=0x7f4ecc007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eca7fc700 1 -- 172.21.15.45:0/94588316 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc005a70 con 0x7f4edc06afa0 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eca7fc700 1 -- 172.21.15.45:0/94588316 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc005bd0 con 0x7f4edc06afa0 2023-10-06T05:15:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed00094f0 con 0x7f4edc06afa0 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4eca7fc700 1 -- 172.21.15.45:0/94588316 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0070d0 con 0x7f4edc06afa0 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.252+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc158130 con 0x7f4edc06afa0 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.254+0000 7f4eca7fc700 1 -- 172.21.15.45:0/94588316 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ecc007480 con 0x7f4edc06afa0 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.254+0000 7f4eca7fc700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ebc066f30 0x7f4ebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.254+0000 7f4eca7fc700 1 -- 172.21.15.45:0/94588316 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4ecc083e30 con 0x7f4edc06afa0 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.255+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ebc066f30 0x7f4ebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.255+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ebc066f30 0x7f4ebc0693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00099b0 tx=0x7f4ed0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.256+0000 7f4ee9d93b80 1 Processor -- start 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.256+0000 7f4ee9d93b80 1 -- start start 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa20820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa29940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02a8f3310 con 0x55f02aa20450 2023-10-06T05:15:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02a930130 con 0x55f02aa294d0 2023-10-06T05:15:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f02a92daf0 con 0x55f02aa05f40 2023-10-06T05:15:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4eda59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa20820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4eda59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa20820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47858/0 (socket says 172.21.15.45:47858) 2023-10-06T05:15:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4eda59c700 1 -- 172.21.15.45:0/901711560 learned_addr learned my addr 172.21.15.45:0/901711560 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4edad9d700 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa29940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.257+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4eda59c700 1 -- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 msgr2=0x55f02aa28f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4eda59c700 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa28f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4eda59c700 1 -- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 msgr2=0x55f02aa29940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4eda59c700 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa29940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4eda59c700 1 -- 172.21.15.45:0/901711560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f02a72f790 con 0x55f02aa20450 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa28f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4eda59c700 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa20820 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc081e30 tx=0x7f4ecc081dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cf25e76c4a3cc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4ebb7fe700 1 -- 172.21.15.45:0/901711560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc055030 con 0x55f02aa20450 2023-10-06T05:15:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4ebb7fe700 1 -- 172.21.15.45:0/901711560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc06ae90 con 0x55f02aa20450 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.258+0000 7f4ebb7fe700 1 -- 172.21.15.45:0/901711560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc054930 con 0x55f02aa20450 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/901711560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 msgr2=0x55f02aa20820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa20820 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc081e30 tx=0x7f4ecc081dc0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/901711560 shutdown_connections 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa29940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa20820 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/901711560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa28f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/901711560 >> 172.21.15.45:0/901711560 conn(0x55f02aa23880 msgr2=0x55f02aa23c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/901711560 shutdown_connections 2023-10-06T05:15:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.259+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/901711560 wait complete. 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.260+0000 7f4ee9d93b80 1 -- start start 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.260+0000 7f4ee9d93b80 1 Processor -- start 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02a9fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02a9fb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02a9298e0 con 0x55f02aa20450 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02a8d5980 con 0x55f02aa294d0 2023-10-06T05:15:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f02a8d5620 con 0x55f02aa05f40 2023-10-06T05:15:32.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ee9d93b80 10 client.0 _ll_register_callbacks cb 0x55f02a738a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:32.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02a9fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02a9fabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47862/0 (socket says 172.21.15.45:47862) 2023-10-06T05:15:32.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/520360046 learned_addr learned my addr 172.21.15.45:0/520360046 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4eda59c700 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4edad9d700 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02a9fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 msgr2=0x55f02aa230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 msgr2=0x55f02a9fb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02a9fb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/520360046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc158130 con 0x55f02aa20450 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02a9fabc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ffe6ef0528790f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.261+0000 7f4eb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:15:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/520360046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed002e070 con 0x55f02aa20450 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f02a9cdd60 con 0x55f02aa20450 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/520360046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4ed0011420 con 0x55f02aa20450 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/520360046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed0021070 con 0x55f02aa20450 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 10 client.4761 fetch_fsmap learned FSMap version 8 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55f02a81c0a0 con 0x55f02aa20450 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/520360046 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f02a9cdd60 con 0x55f02aa20450 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/520360046 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f4ed00117f0 con 0x55f02aa20450 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 10 client.4761 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 10 client.4761 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:15:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 20 client.4761 populate_metadata read hostname 'smithi045' 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa21a60 0x55f02aa21e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 4 client.4761 mds_command: new command op to 4276 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 --> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f02a9aab30 con 0x55f02aa21a60 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.262+0000 7f4eda59c700 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa21a60 0x55f02aa21e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.263+0000 7f4eb8ff9700 10 client.4761 ms_handle_connect on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.263+0000 7f4eda59c700 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa21a60 0x55f02aa21e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2a166921a59950 server_cookie=7eafbb2d8b966f7e in_seq=0 out_seq=0 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.263+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/520360046 <== mds.0 v2:172.21.15.144:6835/1292454917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f02a9aab30 con 0x55f02aa21a60 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.263+0000 7f4eb8ff9700 10 client.4761 handle_command_reply: tid=0 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 client.4761 shutdown 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 msgr2=0x55f02a9fabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02a9fabc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0021000 comp rx=0 tx=0).stop 2023-10-06T05:15:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 shutdown_connections 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa21a60 0x55f02aa21e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02a9fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02aa230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4eb8ff9700 0 client.4761 ms_handle_reset on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/520360046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02a9fabc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 >> 172.21.15.45:0/520360046 conn(0x55f02aa23880 msgr2=0x55f02a9d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 shutdown_connections 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/520360046 wait complete. 2023-10-06T05:15:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.264+0000 7f4ee9d93b80 20 client.4761 trim_cache size 0 max 16384 2023-10-06T05:15:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 Processor -- start 2023-10-06T05:15:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 -- start start 2023-10-06T05:15:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa01360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02a7c11e0 con 0x55f02aa20450 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02a9d72f0 con 0x55f02aa294d0 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.282+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f02a9fd7b0 con 0x55f02aa05f40 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa01360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa01360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47864/0 (socket says 172.21.15.45:47864) 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/890785704 learned_addr learned my addr 172.21.15.45:0/890785704 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4eda59c700 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4edad9d700 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 msgr2=0x55f02a9f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 msgr2=0x55f02aa018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 -- 172.21.15.45:0/890785704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f02a8f83c0 con 0x55f02aa20450 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4edad9d700 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa01360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000c010 tx=0x7f4ed002b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af3993271d083fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ebbfff700 1 -- 172.21.15.45:0/890785704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0021070 con 0x55f02aa20450 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ebbfff700 1 -- 172.21.15.45:0/890785704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed001aa90 con 0x55f02aa20450 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.283+0000 7f4ebbfff700 1 -- 172.21.15.45:0/890785704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed002e040 con 0x55f02aa20450 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/890785704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 msgr2=0x55f02aa01360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa01360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000c010 tx=0x7f4ed002b050 comp rx=0 tx=0).stop 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/890785704 shutdown_connections 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa294d0 0x55f02aa018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/890785704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa20450 0x55f02aa01360 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/890785704 >> 172.21.15.45:0/890785704 conn(0x55f02aa23880 msgr2=0x55f02aa230c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/890785704 shutdown_connections 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/890785704 wait complete. 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.284+0000 7f4ee9d93b80 1 -- start start 2023-10-06T05:15:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.285+0000 7f4ee9d93b80 1 Processor -- start 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.285+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa20450 0x55f02a9fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ee9d93b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa294d0 0x55f02aa3be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02aa859c0 con 0x55f02aa294d0 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02aa29020 con 0x55f02aa20450 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ee9d93b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f02aa41d20 con 0x55f02aa05f40 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ee9d93b80 10 client.0 _ll_register_callbacks cb 0x55f02a738a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa294d0 0x55f02aa3be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa294d0 0x55f02aa3be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47878/0 (socket says 172.21.15.45:47878) 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 -- 172.21.15.45:0/2789957051 learned_addr learned my addr 172.21.15.45:0/2789957051 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4eda59c700 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ed9d9b700 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa20450 0x55f02a9fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 -- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 msgr2=0x55f02a9fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 -- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa20450 msgr2=0x55f02a9fdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa20450 0x55f02a9fdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 -- 172.21.15.45:0/2789957051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f02a81c0a0 con 0x55f02aa294d0 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4edad9d700 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa294d0 0x55f02aa3be80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40186d0 tx=0x7f4ed400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39a0eb0400f55f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4eb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:15:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/2789957051 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400e040 con 0x55f02aa294d0 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/2789957051 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4ed40155a0 con 0x55f02aa294d0 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.286+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ecc06ae90 con 0x55f02aa294d0 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/2789957051 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed4015970 con 0x55f02aa294d0 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 10 client.4767 fetch_fsmap learned FSMap version 8 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55f02a7901d0 con 0x55f02aa294d0 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/2789957051 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc06ae90 con 0x55f02aa294d0 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/2789957051 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 8) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f4ed4015ad0 con 0x55f02aa294d0 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 10 client.4767 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T05:15:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 10 client.4767 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 20 client.4767 populate_metadata read hostname 'smithi045' 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa3c440 0x55f02aa3c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 4 client.4767 mds_command: new command op to 4276 tid=0[{"prefix": "status"}] 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 --> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f02a8cc8d0 con 0x55f02aa3c440 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4eda59c700 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa3c440 0x55f02aa3c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4eb8ff9700 10 client.4767 ms_handle_connect on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.287+0000 7f4eda59c700 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa3c440 0x55f02aa3c810 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e74d968f92d963 server_cookie=d41f58cb5b833766 in_seq=0 out_seq=0 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4eb8ff9700 1 -- 172.21.15.45:0/2789957051 <== mds.0 v2:172.21.15.144:6835/1292454917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55f02a8cc8d0 con 0x55f02aa3c440 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4eb8ff9700 10 client.4767 handle_command_reply: tid=0 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 client.4767 shutdown 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa294d0 msgr2=0x55f02aa3be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa294d0 0x55f02aa3be80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40186d0 tx=0x7f4ed400b040 comp rx=0 tx=0).stop 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 shutdown_connections 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f02aa294d0 0x55f02aa3be80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f02aa05f40 0x55f02a9fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:6835/1292454917,v1:172.21.15.144:6837/1292454917] conn(0x55f02aa3c440 0x55f02aa3c810 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 --2- 172.21.15.45:0/2789957051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f02aa20450 0x55f02a9fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4eb8ff9700 0 client.4767 ms_handle_reset on v2:172.21.15.144:6835/1292454917 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 >> 172.21.15.45:0/2789957051 conn(0x55f02aa23880 msgr2=0x55f02a9f9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 shutdown_connections 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 1 -- 172.21.15.45:0/2789957051 wait complete. 2023-10-06T05:15:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.288+0000 7f4ee9d93b80 20 client.4767 trim_cache size 0 max 16384 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4276, 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:15:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 32.351204494000001, 2023-10-06T05:15:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 8, 2023-10-06T05:15:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 24, 2023-10-06T05:15:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 24, 2023-10-06T05:15:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 38.338663218999997 2023-10-06T05:15:32.299 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:15:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ebc066f30 msgr2=0x7f4ebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ebc066f30 0x7f4ebc0693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00099b0 tx=0x7f4ed0017000 comp rx=0 tx=0).stop 2023-10-06T05:15:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 msgr2=0x7f4edc157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc157bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00c7c0 tx=0x7f4ecc007f70 comp rx=0 tx=0).stop 2023-10-06T05:15:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 shutdown_connections 2023-10-06T05:15:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ebc066f30 0x7f4ebc0693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0c70f0 0x7f4edc0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc06afa0 0x7f4edc157bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 --2- 172.21.15.45:0/94588316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc06f070 0x7f4edc15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 >> 172.21.15.45:0/94588316 conn(0x7f4edc0cc5a0 msgr2=0x7f4edc068600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 shutdown_connections 2023-10-06T05:15:32.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.289+0000 7f4ee0a7f700 1 -- 172.21.15.45:0/94588316 wait complete. 2023-10-06T05:15:32.304 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-06T05:15:32.314 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-06T05:15:32.314 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T05:15:32.314 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-06T05:15:32.314 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-06T05:15:32.315 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-06T05:15:32.315 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-06T05:15:32.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.315 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-06T05:15:32.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.338 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-10-06T05:15:32.411 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2023-10-06T05:15:32.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:15:32.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.413 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:15:32.435 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-06T05:15:32.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.436 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T05:15:32.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:32.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-10-06T05:15:32.621 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-06T05:15:32.622 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:15:32.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.761+0000 7f4d6aa5d700 1 Processor -- start 2023-10-06T05:15:32.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.761+0000 7f4d6aa5d700 1 -- start start 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d6aa5d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d640cc3c0 0x7f4d6415a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d6aa5d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d6406a210 0x7f4d6406a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d6aa5d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d640aed10 con 0x7f4d64153720 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d6aa5d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640a8ac0 con 0x7f4d640cc3c0 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d6aa5d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6406af80 con 0x7f4d6406a210 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d637fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d640cc3c0 0x7f4d6415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d637fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d640cc3c0 0x7f4d6415a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47884/0 (socket says 172.21.15.45:47884) 2023-10-06T05:15:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.762+0000 7f4d637fe700 1 -- 172.21.15.45:0/3275149694 learned_addr learned my addr 172.21.15.45:0/3275149694 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d68ffa700 1 --2- 172.21.15.45:0/3275149694 >> v2:172.21.15.144:3300/0 conn(0x7f4d6406a210 0x7f4d6406a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d637fe700 1 -- 172.21.15.45:0/3275149694 >> v2:172.21.15.144:3300/0 conn(0x7f4d6406a210 msgr2=0x7f4d6406a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d637fe700 1 --2- 172.21.15.45:0/3275149694 >> v2:172.21.15.144:3300/0 conn(0x7f4d6406a210 0x7f4d6406a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d637fe700 1 -- 172.21.15.45:0/3275149694 >> v1:172.21.15.144:6789/0 conn(0x7f4d64153720 legacy=0x7f4d640cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d637fe700 1 -- 172.21.15.45:0/3275149694 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6406b260 con 0x7f4d640cc3c0 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d637fe700 1 --2- 172.21.15.45:0/3275149694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d640cc3c0 0x7f4d6415a970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d640c7ef0 tx=0x7f4d54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0cfc248cc8e29b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d62ffd700 1 -- 172.21.15.45:0/3275149694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400b8b0 con 0x7f4d640cc3c0 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d62ffd700 1 -- 172.21.15.45:0/3275149694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5400ba10 con 0x7f4d640cc3c0 2023-10-06T05:15:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.763+0000 7f4d62ffd700 1 -- 172.21.15.45:0/3275149694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400bdb0 con 0x7f4d640cc3c0 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/3275149694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d640cc3c0 msgr2=0x7f4d6415a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/3275149694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d640cc3c0 0x7f4d6415a970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d640c7ef0 tx=0x7f4d54009250 comp rx=0 tx=0).stop 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/3275149694 shutdown_connections 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/3275149694 >> v2:172.21.15.144:3300/0 conn(0x7f4d6406a210 0x7f4d6406a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/3275149694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d640cc3c0 0x7f4d6415a970 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/3275149694 >> 172.21.15.45:0/3275149694 conn(0x7f4d64067620 msgr2=0x7f4d64067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/3275149694 shutdown_connections 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.764+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/3275149694 wait complete. 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.765+0000 7f4d6aa5d700 1 Processor -- start 2023-10-06T05:15:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.765+0000 7f4d6aa5d700 1 -- start start 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d6aa5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6406a210 0x7f4d6415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d6aa5d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d640cc3c0 0x7f4d64153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d6aa5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64153720 0x7f4d64169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d6aa5d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64154b60 con 0x7f4d6406a210 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d6aa5d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d641593f0 con 0x7f4d64153720 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d6aa5d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64158e30 con 0x7f4d640cc3c0 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d63fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6406a210 0x7f4d6415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d63fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6406a210 0x7f4d6415bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47888/0 (socket says 172.21.15.45:47888) 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d63fff700 1 -- 172.21.15.45:0/1609352593 learned_addr learned my addr 172.21.15.45:0/1609352593 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d68ffa700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64153720 0x7f4d64169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.766+0000 7f4d637fe700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d640cc3c0 0x7f4d64153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d63fff700 1 -- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d640cc3c0 msgr2=0x7f4d64153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d63fff700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d640cc3c0 0x7f4d64153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d63fff700 1 -- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64153720 msgr2=0x7f4d64169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d63fff700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64153720 0x7f4d64169890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d63fff700 1 -- 172.21.15.45:0/1609352593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540094f0 con 0x7f4d6406a210 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d63fff700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6406a210 0x7f4d6415bd70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00b5e0 tx=0x7f4d4c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d5bfff700 1 -- 172.21.15.45:0/1609352593 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00f510 con 0x7f4d6406a210 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d5bfff700 1 -- 172.21.15.45:0/1609352593 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c00f670 con 0x7f4d6406a210 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d64159110 con 0x7f4d6406a210 2023-10-06T05:15:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d5bfff700 1 -- 172.21.15.45:0/1609352593 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00fa30 con 0x7f4d6406a210 2023-10-06T05:15:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.767+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d64154880 con 0x7f4d6406a210 2023-10-06T05:15:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.768+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d640c9520 con 0x7f4d6406a210 2023-10-06T05:15:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.769+0000 7f4d5bfff700 1 -- 172.21.15.45:0/1609352593 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d4c01a400 con 0x7f4d6406a210 2023-10-06T05:15:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.769+0000 7f4d5bfff700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d44066f30 0x7f4d440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.769+0000 7f4d5bfff700 1 -- 172.21.15.45:0/1609352593 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4d4c0843c0 con 0x7f4d6406a210 2023-10-06T05:15:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.769+0000 7f4d637fe700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d44066f30 0x7f4d440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.770+0000 7f4d637fe700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d44066f30 0x7f4d440693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d640c7ef0 tx=0x7f4d54016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:32.773+0000 7f4d5bfff700 1 -- 172.21.15.45:0/1609352593 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d640c9520 con 0x7f4d6406a210 2023-10-06T05:15:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.024+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d640c9520 con 0x7f4d6406a210 2023-10-06T05:15:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.024+0000 7f4d5bfff700 1 -- 172.21.15.45:0/1609352593 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4d640c9520 con 0x7f4d6406a210 2023-10-06T05:15:33.026 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T05:15:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d44066f30 msgr2=0x7f4d440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d44066f30 0x7f4d440693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d640c7ef0 tx=0x7f4d54016000 comp rx=0 tx=0).stop 2023-10-06T05:15:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6406a210 msgr2=0x7f4d6415bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6406a210 0x7f4d6415bd70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00b5e0 tx=0x7f4d4c009120 comp rx=0 tx=0).stop 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 shutdown_connections 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d44066f30 0x7f4d440693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64153720 0x7f4d64169890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6406a210 0x7f4d6415bd70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 --2- 172.21.15.45:0/1609352593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d640cc3c0 0x7f4d64153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.026+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 >> 172.21.15.45:0/1609352593 conn(0x7f4d64067620 msgr2=0x7f4d64070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.027+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 shutdown_connections 2023-10-06T05:15:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:33.027+0000 7f4d6aa5d700 1 -- 172.21.15.45:0/1609352593 wait complete. 2023-10-06T05:15:33.041 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:15:33.041 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:15:33.042 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:15:33.042 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:15:33.042 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:33.042 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:15:33.042 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:15:33.042 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:15:33.042 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:15:33.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.042 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:15:33.063 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.064 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:15:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42408sec preferred_lft 42408sec 2023-10-06T05:15:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:15:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:33.066 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:15:33.066 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T05:15:33.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.067 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:33.067 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:33.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2023-10-06T05:15:33.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2023-10-06T05:15:33.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2023-10-06T05:15:33.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T05:15:33.067 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:33.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.194 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:15:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:15:33.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.226 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:15:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:15:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:15:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:15:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:15:33.291 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:15:33.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.292 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:33.292 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:33.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:15:33.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:15:33.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:15:33.292 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:33.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.415 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:15:33.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.437 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-10-06T05:15:33.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.501 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:33.501 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:33.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:15:33.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T05:15:33.501 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:33.612 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T05:15:33.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T05:15:33.613 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:33.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:33.842 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:33.842 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:33.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:15:33.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T05:15:33.842 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:33.953 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:15:33.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:33.953 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:33.973 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:15:33.974 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:33.974 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:34.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:15:34.112 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:34.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:34.187 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:34.188 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:34.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:34.189 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:15:34.253 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:15:34.253 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:15:34.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:15:34.330 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:34.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:34.397 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:15:34.394+0000 7f8924034540 -1 init, newargv = 0x55b5a6d4c5d0 newargc=15 2023-10-06T05:15:34.397 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[131070]: starting ceph client 2023-10-06T05:15:34.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[131070]: starting fuse 2023-10-06T05:15:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:34.404 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:34.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:34.405 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:15:34.465 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:15:34.466 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:15:34.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:15:34.466 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:15:34.466 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:15:34.466 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:15:34.466 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:15:34.466 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:34.466 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:15:34.467 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:15:34.468 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:15:34.468 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:15:34.468 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:15:34.468 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:34.468 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:15:34.468 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:15:34.607 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.131070.asok 2023-10-06T05:15:34.787 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.131070.asok 2023-10-06T05:15:34.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:34.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.131070.asok status 2023-10-06T05:15:34.963 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:15:34.963 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:15:34.963 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:15:34.963 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:15:34.963 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:15:34.963 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "131070", 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4779, 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4779 2023-10-06T05:15:34.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 759524471 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 759524471 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:34.965 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.4779 192.168.0.1:0/759524471", 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/759524471", 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 8, 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 24, 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:15:34.966 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:15:34.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:15:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.102+0000 7fda34ad2700 1 Processor -- start 2023-10-06T05:15:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.102+0000 7fda34ad2700 1 -- start start 2023-10-06T05:15:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda34ad2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 0x7fda3015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda34ad2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda300ccdc0 0x7fda300cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda34ad2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda300cd740 con 0x7fda301560a0 2023-10-06T05:15:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda34ad2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda300ae6b0 con 0x7fda300ccdc0 2023-10-06T05:15:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda34ad2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda300ae8d0 con 0x7fda30155cd0 2023-10-06T05:15:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda2dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda301560a0 0x7fda300adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50466/0 (socket says 172.21.15.45:50466) 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda2dd9b700 1 -- 172.21.15.45:0/2534897677 learned_addr learned my addr 172.21.15.45:0/2534897677 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda2e59c700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 0x7fda3015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda2ed9d700 1 --2- 172.21.15.45:0/2534897677 >> v2:172.21.15.144:3300/0 conn(0x7fda300ccdc0 0x7fda300cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918381279 0 0) 0x7fda300cd740 con 0x7fda301560a0 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.103+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda1c003570 con 0x7fda301560a0 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3154504043 0 0) 0x7fda1c003570 con 0x7fda301560a0 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 msgr2=0x7fda3015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 0x7fda3015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 >> v2:172.21.15.144:3300/0 conn(0x7fda300ccdc0 msgr2=0x7fda300cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 --2- 172.21.15.45:0/2534897677 >> v2:172.21.15.144:3300/0 conn(0x7fda300ccdc0 0x7fda300cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda300aebb0 con 0x7fda301560a0 2023-10-06T05:15:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fda18001470 con 0x7fda301560a0 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 >> v1:172.21.15.45:6789/0 conn(0x7fda301560a0 legacy=0x7fda300adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 0x7fda1c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda1c003fe0 0x7fda1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda1c004d50 0x7fda1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300ae6b0 con 0x7fda1c003fe0 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300ae8d0 con 0x7fda1c004d50 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c00ba90 con 0x7fda30155cd0 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2e59c700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 0x7fda1c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2dd9b700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda1c003fe0 0x7fda1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2ed9d700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda1c004d50 0x7fda1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2dd9b700 1 -- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 msgr2=0x7fda1c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2dd9b700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 0x7fda1c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2dd9b700 1 -- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda1c004d50 msgr2=0x7fda1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2dd9b700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda1c004d50 0x7fda1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2dd9b700 1 -- 172.21.15.45:0/2534897677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c003570 con 0x7fda1c003fe0 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.104+0000 7fda2e59c700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda30155cd0 0x7fda1c003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda2dd9b700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda1c003fe0 0x7fda1c004810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c3c0 tx=0x7fda18001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c870ea4eb88d4448 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800cc30 con 0x7fda1c003fe0 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1800cdd0 con 0x7fda1c003fe0 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda2d59a700 1 -- 172.21.15.45:0/2534897677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18005b70 con 0x7fda1c003fe0 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda34ad2700 1 -- 172.21.15.45:0/2534897677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda1c003fe0 msgr2=0x7fda1c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda34ad2700 1 --2- 172.21.15.45:0/2534897677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda1c003fe0 0x7fda1c004810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c3c0 tx=0x7fda18001e90 comp rx=0 tx=0).stop 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda2ed9d700 1 -- 172.21.15.45:0/2534897677 reap_dead start 2023-10-06T05:15:35.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda34ad2700 1 -- 172.21.15.45:0/2534897677 shutdown_connections 2023-10-06T05:15:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda34ad2700 1 -- 172.21.15.45:0/2534897677 >> 172.21.15.45:0/2534897677 conn(0x7fda3006f090 msgr2=0x7fda3006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda34ad2700 1 -- 172.21.15.45:0/2534897677 shutdown_connections 2023-10-06T05:15:35.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.105+0000 7fda34ad2700 1 -- 172.21.15.45:0/2534897677 wait complete. 2023-10-06T05:15:35.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 Processor -- start 2023-10-06T05:15:35.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 -- start start 2023-10-06T05:15:35.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1c003fe0 0x7fda300a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda300a4bb0 0x7fda30169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3016a030 0x7fda300caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3016c9d0 con 0x7fda300a4bb0 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300cb770 con 0x7fda3016a030 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.107+0000 7fda34ad2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda300cba50 con 0x7fda1c003fe0 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda300a4bb0 0x7fda30169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda300a4bb0 0x7fda30169af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47912/0 (socket says 172.21.15.45:47912) 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 -- 172.21.15.45:0/1967489500 learned_addr learned my addr 172.21.15.45:0/1967489500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2e59c700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1c003fe0 0x7fda300a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2ed9d700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3016a030 0x7fda300caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 -- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1c003fe0 msgr2=0x7fda300a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1c003fe0 0x7fda300a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 -- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3016a030 msgr2=0x7fda300caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3016a030 0x7fda300caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 -- 172.21.15.45:0/1967489500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda300cbd30 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda2dd9b700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda300a4bb0 0x7fda30169af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda18011000 tx=0x7fda18015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda267fc700 1 -- 172.21.15.45:0/1967489500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800c9e0 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda267fc700 1 -- 172.21.15.45:0/1967489500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18001bd0 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda267fc700 1 -- 172.21.15.45:0/1967489500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18007530 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda300cc010 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.108+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda300cc2f0 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.109+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3016a690 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.110+0000 7fda267fc700 1 -- 172.21.15.45:0/1967489500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda18019030 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.110+0000 7fda267fc700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda10066f30 0x7fda100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.110+0000 7fda267fc700 1 -- 172.21.15.45:0/1967489500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fda18085b60 con 0x7fda300a4bb0 2023-10-06T05:15:35.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.110+0000 7fda2e59c700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda10066f30 0x7fda100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:35.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.111+0000 7fda2e59c700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda10066f30 0x7fda100693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda20006dc0 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:35.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.115+0000 7fda267fc700 1 -- 172.21.15.45:0/1967489500 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda3016a690 con 0x7fda300a4bb0 2023-10-06T05:15:35.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.364+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fda3016a690 con 0x7fda300a4bb0 2023-10-06T05:15:35.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.365+0000 7fda267fc700 1 -- 172.21.15.45:0/1967489500 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fda3016a690 con 0x7fda300a4bb0 2023-10-06T05:15:35.368 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T05:15:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda10066f30 msgr2=0x7fda100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda10066f30 0x7fda100693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda20006dc0 tx=0x7fda20008040 comp rx=0 tx=0).stop 2023-10-06T05:15:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda300a4bb0 msgr2=0x7fda30169af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda300a4bb0 0x7fda30169af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda18011000 tx=0x7fda18015000 comp rx=0 tx=0).stop 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 shutdown_connections 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda10066f30 0x7fda100693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3016a030 0x7fda300caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1c003fe0 0x7fda300a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 --2- 172.21.15.45:0/1967489500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda300a4bb0 0x7fda30169af0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 >> 172.21.15.45:0/1967489500 conn(0x7fda3006f090 msgr2=0x7fda300c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 shutdown_connections 2023-10-06T05:15:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:35.367+0000 7fda34ad2700 1 -- 172.21.15.45:0/1967489500 wait complete. 2023-10-06T05:15:35.383 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:15:35.383 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:15:35.383 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:15:35.383 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T05:15:35.384 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T05:15:35.384 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:15:35.384 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:15:35.384 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:15:35.384 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T05:15:35.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.384 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:15:35.405 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:35.405 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:35.405 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:35.405 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:15:35.406 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42406sec preferred_lft 42406sec 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:15:35.407 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.408 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:15:35.408 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:15:35.408 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link tentative 2023-10-06T05:15:35.408 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.408 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:15:35.477 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:15:35.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.482 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:15:35.547 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:15:35.548 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:15:35.548 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:15:35.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.548 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:35.548 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:35.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:15:35.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:15:35.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:15:35.548 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:35.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:35.591 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:35.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.670 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:15:35.733 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:15:35.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.734 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-10-06T05:15:35.797 INFO:teuthology.orchestra.run.smithi045.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.798 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:35.798 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:35.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:15:35.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T05:15:35.798 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:35.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:15:35.953 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:35.953 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:35.953 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:35.953 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.953 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:35.953 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.954 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:35.954 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether e6:50:de:a9:a0:c4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:15:35.954 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:15:35.954 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.954 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::e450:deff:fea9:a0c4/64 scope link 2023-10-06T05:15:35.954 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:35.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:35.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:36.008 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether e6:50:de:a9:a0:c4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:15:36.009 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:15:36.009 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:36.009 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::e450:deff:fea9:a0c4/64 scope link 2023-10-06T05:15:36.009 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:36.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T05:15:36.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:36.024 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:36.025 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:36.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T05:15:36.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T05:15:36.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T05:15:36.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T05:15:36.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T05:15:36.025 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:36.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:36.231 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:15:36.231 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:15:36.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.1 up 2023-10-06T05:15:36.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T05:15:36.231 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:15:36.343 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-06T05:15:36.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:36.343 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T05:15:36.364 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T05:15:36.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:36.364 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T05:15:36.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:15:36.502 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:36.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:36.578 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:36.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:36.579 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:15:36.643 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:15:36.643 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-10-06T05:15:36.643 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-06T05:15:36.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:15:36.722 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:36.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:36.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi045.stderr:2023-10-06T05:15:36.784+0000 7ff900a32540 -1 init, newargv = 0x55ce57880060 newargc=15 2023-10-06T05:15:36.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi045.stderr:ceph-fuse[131816]: starting ceph client 2023-10-06T05:15:36.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi045.stderr:ceph-fuse[131816]: starting fuse 2023-10-06T05:15:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:36.795 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:36.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:36.796 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:15:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:15:36.858 INFO:teuthology.orchestra.run.smithi045.stdout:48 2023-10-06T05:15:36.858 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-10-06T05:15:36.858 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:15:36.858 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:15:36.858 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:15:36.858 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:36.859 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.2")) 2023-10-06T05:15:36.860 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:15:36.999 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.2.131816.asok 2023-10-06T05:15:37.181 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.131816.asok 2023-10-06T05:15:37.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:37.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.131816.asok status 2023-10-06T05:15:37.356 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "2", 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "131816", 2023-10-06T05:15:37.357 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4794, 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4794 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:37.358 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.2:0", 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 582712581 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.2:0", 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 582712581 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:15:37.359 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.4794 192.168.0.2:0/582712581", 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.2:0/582712581", 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 8, 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 24, 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:15:37.360 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:15:37.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:15:37.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.497+0000 7f0c9e9f7700 1 Processor -- start 2023-10-06T05:15:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.497+0000 7f0c9e9f7700 1 -- start start 2023-10-06T05:15:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.497+0000 7f0c9e9f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c980b1060 0x7f0c98069f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.497+0000 7f0c9e9f7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c98140990 0x7f0c9806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.497+0000 7f0c9e9f7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9806b1f0 con 0x7f0c980b1430 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.497+0000 7f0c9e9f7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9806f270 con 0x7f0c98140990 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c9e9f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9806a5e0 con 0x7f0c980b1060 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c980b1060 0x7f0c98069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c9cf94700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c98140990 0x7f0c9806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c9cf94700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c98140990 0x7f0c9806ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:37346/0 (socket says 172.21.15.45:37346) 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c980b1060 0x7f0c98069f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:59800/0 (socket says 172.21.15.45:59800) 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c9cf94700 1 -- 172.21.15.45:0/2467541091 learned_addr learned my addr 172.21.15.45:0/2467541091 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c977fe700 1 --1- 172.21.15.45:0/2467541091 >> v1:172.21.15.144:6789/0 conn(0x7f0c980b1430 0x7f0c98140280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:35676/0 (socket says 172.21.15.45:35676) 2023-10-06T05:15:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c97fff700 1 -- 172.21.15.45:0/2467541091 >> v2:172.21.15.144:3300/0 conn(0x7f0c98140990 msgr2=0x7f0c9806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c97fff700 1 --2- 172.21.15.45:0/2467541091 >> v2:172.21.15.144:3300/0 conn(0x7f0c98140990 0x7f0c9806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c97fff700 1 -- 172.21.15.45:0/2467541091 >> v1:172.21.15.144:6789/0 conn(0x7f0c980b1430 legacy=0x7f0c98140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c97fff700 1 -- 172.21.15.45:0/2467541091 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9806a8c0 con 0x7f0c980b1060 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.498+0000 7f0c9cf94700 1 --2- 172.21.15.45:0/2467541091 >> v2:172.21.15.144:3300/0 conn(0x7f0c98140990 0x7f0c9806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c97fff700 1 --2- 172.21.15.45:0/2467541091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c980b1060 0x7f0c98069f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800dd10 tx=0x7f0c8800d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b98560d4611d866a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c96ffd700 1 -- 172.21.15.45:0/2467541091 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88012070 con 0x7f0c980b1060 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c96ffd700 1 -- 172.21.15.45:0/2467541091 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8800de70 con 0x7f0c980b1060 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c96ffd700 1 -- 172.21.15.45:0/2467541091 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800fb60 con 0x7f0c980b1060 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/2467541091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c980b1060 msgr2=0x7f0c98069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/2467541091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c980b1060 0x7f0c98069f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800dd10 tx=0x7f0c8800d630 comp rx=0 tx=0).stop 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/2467541091 shutdown_connections 2023-10-06T05:15:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/2467541091 >> v2:172.21.15.144:3300/0 conn(0x7f0c98140990 0x7f0c9806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/2467541091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c980b1060 0x7f0c98069f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/2467541091 >> 172.21.15.45:0/2467541091 conn(0x7f0c98067620 msgr2=0x7f0c98067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/2467541091 shutdown_connections 2023-10-06T05:15:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.499+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/2467541091 wait complete. 2023-10-06T05:15:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.500+0000 7f0c9e9f7700 1 Processor -- start 2023-10-06T05:15:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.501+0000 7f0c9e9f7700 1 -- start start 2023-10-06T05:15:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.501+0000 7f0c9e9f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c980b1060 0x7f0c98165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.501+0000 7f0c9e9f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c980b1430 0x7f0c980c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.501+0000 7f0c9e9f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c98140990 0x7f0c980c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c9e9f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980c6f40 con 0x7f0c980b1060 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c9e9f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980c5770 con 0x7f0c980b1430 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c9e9f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981686e0 con 0x7f0c98140990 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c980b1060 0x7f0c98165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c980b1060 0x7f0c98165300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52006/0 (socket says 172.21.15.45:52006) 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 -- 172.21.15.45:0/1729371909 learned_addr learned my addr 172.21.15.45:0/1729371909 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c9cf94700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c98140990 0x7f0c980c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c977fe700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c980b1430 0x7f0c980c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 -- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c98140990 msgr2=0x7f0c980c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c98140990 0x7f0c980c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 -- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c980b1430 msgr2=0x7f0c980c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c980b1430 0x7f0c980c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 -- 172.21.15.45:0/1729371909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009ae0 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c97fff700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c980b1060 0x7f0c98165300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800f6c0 tx=0x7f0c8801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.502+0000 7f0c7ffff700 1 -- 172.21.15.45:0/1729371909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800c840 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.503+0000 7f0c7ffff700 1 -- 172.21.15.45:0/1729371909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8800c9a0 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.503+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8800d8d0 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.503+0000 7f0c7ffff700 1 -- 172.21.15.45:0/1729371909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800cd40 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.503+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c980c4190 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.504+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9806cec0 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.504+0000 7f0c7ffff700 1 -- 172.21.15.45:0/1729371909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c88020620 con 0x7f0c980b1060 2023-10-06T05:15:37.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.505+0000 7f0c7ffff700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c84066f30 0x7f0c840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.505+0000 7f0c7ffff700 1 -- 172.21.15.45:0/1729371909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0c88089e40 con 0x7f0c980b1060 2023-10-06T05:15:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.505+0000 7f0c977fe700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c84066f30 0x7f0c840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.507+0000 7f0c977fe700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c84066f30 0x7f0c840693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c980af730 tx=0x7f0c8c007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.510+0000 7f0c7ffff700 1 -- 172.21.15.45:0/1729371909 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c88056c60 con 0x7f0c980b1060 2023-10-06T05:15:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.754+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0c9806aba0 con 0x7f0c980b1060 2023-10-06T05:15:37.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.755+0000 7f0c7ffff700 1 -- 172.21.15.45:0/1729371909 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0c88024070 con 0x7f0c980b1060 2023-10-06T05:15:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T05:15:37.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c84066f30 msgr2=0x7f0c840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c84066f30 0x7f0c840693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c980af730 tx=0x7f0c8c007eb0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c980b1060 msgr2=0x7f0c98165300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:37.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c980b1060 0x7f0c98165300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800f6c0 tx=0x7f0c8801c000 comp rx=0 tx=0).stop 2023-10-06T05:15:37.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 shutdown_connections 2023-10-06T05:15:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c84066f30 0x7f0c840693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c98140990 0x7f0c980c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c980b1060 0x7f0c98165300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 --2- 172.21.15.45:0/1729371909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c980b1430 0x7f0c980c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 >> 172.21.15.45:0/1729371909 conn(0x7f0c98067620 msgr2=0x7f0c980c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 shutdown_connections 2023-10-06T05:15:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:37.757+0000 7f0c9e9f7700 1 -- 172.21.15.45:0/1729371909 wait complete. 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:15:37.773 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:15:37.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:37.774 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-06T05:15:37.796 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:37.797 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42363sec preferred_lft 42363sec 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-06T05:15:37.798 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:37.799 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T05:15:37.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2023-10-06T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2023-10-06T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2023-10-06T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:37.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:37.985 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:15:38.019 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-06T05:15:38.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:38.020 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-06T05:15:38.107 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-06T05:15:38.107 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:15:38.107 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:15:38.107 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:15:38.107 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:15:38.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:38.108 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:38.108 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:38.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:15:38.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:15:38.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:15:38.108 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:38.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:38.351 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-06T05:15:38.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:38.369 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2023-10-06T05:15:38.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:38.428 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:38.428 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:38.428 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:15:38.428 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T05:15:38.428 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:38.523 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T05:15:38.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:38.523 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:38.523 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:38.523 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T05:15:38.523 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T05:15:38.523 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T05:15:38.523 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T05:15:38.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T05:15:38.524 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:38.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:38.739 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:38.739 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:38.739 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-06T05:15:38.739 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T05:15:38.739 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:38.837 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T05:15:38.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:38.837 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:38.855 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:15:38.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:38.855 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:38.913 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2023-10-06T05:15:38.981 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:38.981 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:39.048 INFO:teuthology.orchestra.run.smithi144.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:39.049 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:39.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:39.049 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2023-10-06T05:15:39.107 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:15:39.107 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T05:15:39.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2023-10-06T05:15:39.176 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:39.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:39.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:2023-10-06T05:15:39.231+0000 7f4d7aa97540 -1 init, newargv = 0x563d8fa56900 newargc=15 2023-10-06T05:15:39.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[126362]: starting ceph client 2023-10-06T05:15:39.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[126362]: starting fuse 2023-10-06T05:15:39.245 INFO:teuthology.orchestra.run.smithi144.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:39.246 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:39.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:39.247 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2023-10-06T05:15:39.305 INFO:teuthology.orchestra.run.smithi144.stdout:46 2023-10-06T05:15:39.305 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T05:15:39.306 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:15:39.306 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2023-10-06T05:15:39.306 DEBUG:teuthology.orchestra.run.smithi144:> import re 2023-10-06T05:15:39.306 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-10-06T05:15:39.306 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2023-10-06T05:15:39.307 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:39.307 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2023-10-06T05:15:39.307 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T05:15:39.307 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2023-10-06T05:15:39.307 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T05:15:39.307 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:39.308 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2023-10-06T05:15:39.308 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2023-10-06T05:15:39.308 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2023-10-06T05:15:39.308 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2023-10-06T05:15:39.308 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:39.309 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2023-10-06T05:15:39.309 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:15:39.309 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:15:39.309 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:15:39.309 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2023-10-06T05:15:39.309 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2023-10-06T05:15:39.310 DEBUG:teuthology.orchestra.run.smithi144:> return f 2023-10-06T05:15:39.310 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:15:39.310 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:39.310 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.1")) 2023-10-06T05:15:39.310 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-10-06T05:15:39.428 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.1.126362.asok 2023-10-06T05:15:39.610 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.126362.asok 2023-10-06T05:15:39.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:39.611 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.126362.asok status 2023-10-06T05:15:39.765 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "1", 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "126362", 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:39.766 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 4819, 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 4819 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:15:39.767 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 29996100 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 29996100 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.4819 192.168.0.1:0/29996100", 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.1:0/29996100", 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2023-10-06T05:15:39.768 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 8, 2023-10-06T05:15:39.769 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 24, 2023-10-06T05:15:39.769 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:15:39.769 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2023-10-06T05:15:39.769 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2023-10-06T05:15:39.769 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-10-06T05:15:39.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:15:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.915+0000 7f7cc1321700 1 Processor -- start 2023-10-06T05:15:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.915+0000 7f7cc1321700 1 -- start start 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.916+0000 7f7cc1321700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cbc159170 0x7f7cbc159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.916+0000 7f7cc1321700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc0ca6d0 con 0x7f7cbc0692d0 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.916+0000 7f7cc1321700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc15d550 con 0x7f7cbc15d040 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.916+0000 7f7cc1321700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cbc15d810 con 0x7f7cbc159170 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.916+0000 7f7cbb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7cbc0692d0 0x7f7cbc0aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46272/0 (socket says 172.21.15.45:46272) 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.916+0000 7f7cbb7fe700 1 -- 172.21.15.45:0/336263808 learned_addr learned my addr 172.21.15.45:0/336263808 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.916+0000 7f7cbaffd700 1 --2- 172.21.15.45:0/336263808 >> v2:172.21.15.144:3300/0 conn(0x7f7cbc159170 0x7f7cbc159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cb9ffb700 1 -- 172.21.15.45:0/336263808 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298628197 0 0) 0x7f7cbc0ca6d0 con 0x7f7cbc0692d0 2023-10-06T05:15:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cb9ffb700 1 -- 172.21.15.45:0/336263808 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca8003570 con 0x7f7cbc0692d0 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cb9ffb700 1 -- 172.21.15.45:0/336263808 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310822460 0 0) 0x7f7cbc15d550 con 0x7f7cbc15d040 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cb9ffb700 1 -- 172.21.15.45:0/336263808 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cbc0ca6d0 con 0x7f7cbc15d040 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cbaffd700 1 -- 172.21.15.45:0/336263808 >> v1:172.21.15.144:6789/0 conn(0x7f7cbc15d040 legacy=0x7f7cbc068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cbaffd700 1 -- 172.21.15.45:0/336263808 >> v1:172.21.15.45:6789/0 conn(0x7f7cbc0692d0 legacy=0x7f7cbc0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cbaffd700 1 -- 172.21.15.45:0/336263808 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc15daf0 con 0x7f7cbc159170 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cbaffd700 1 --2- 172.21.15.45:0/336263808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc159170 0x7f7cbc159540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0098e0 tx=0x7f7cac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8499330b3969b5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cb9ffb700 1 -- 172.21.15.45:0/336263808 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac011630 con 0x7f7cbc159170 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cb9ffb700 1 -- 172.21.15.45:0/336263808 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac011790 con 0x7f7cbc159170 2023-10-06T05:15:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.917+0000 7f7cb9ffb700 1 -- 172.21.15.45:0/336263808 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac011b30 con 0x7f7cbc159170 2023-10-06T05:15:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.918+0000 7f7cc1321700 1 -- 172.21.15.45:0/336263808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc159170 msgr2=0x7f7cbc159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.918+0000 7f7cc1321700 1 --2- 172.21.15.45:0/336263808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc159170 0x7f7cbc159540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0098e0 tx=0x7f7cac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.918+0000 7f7cc1321700 1 -- 172.21.15.45:0/336263808 shutdown_connections 2023-10-06T05:15:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.918+0000 7f7cc1321700 1 --2- 172.21.15.45:0/336263808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc159170 0x7f7cbc159540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.918+0000 7f7cc1321700 1 -- 172.21.15.45:0/336263808 >> 172.21.15.45:0/336263808 conn(0x7f7cbc06f7c0 msgr2=0x7f7cbc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.918+0000 7f7cc1321700 1 -- 172.21.15.45:0/336263808 shutdown_connections 2023-10-06T05:15:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.918+0000 7f7cc1321700 1 -- 172.21.15.45:0/336263808 wait complete. 2023-10-06T05:15:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.919+0000 7f7cc1321700 1 Processor -- start 2023-10-06T05:15:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.919+0000 7f7cc1321700 1 -- start start 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cc1321700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cbc0692d0 0x7f7cbc0a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cc1321700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cbc159170 0x7f7cbc169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cc1321700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc15d040 0x7f7cbc169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cc1321700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc0c9450 con 0x7f7cbc0692d0 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cc1321700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc0a6f20 con 0x7f7cbc15d040 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cc1321700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc15a6e0 con 0x7f7cbc159170 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cbaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cbc0692d0 0x7f7cbc0a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cbaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cbc0692d0 0x7f7cbc0a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52018/0 (socket says 172.21.15.45:52018) 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cbaffd700 1 -- 172.21.15.45:0/3023429947 learned_addr learned my addr 172.21.15.45:0/3023429947 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cbb7fe700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc15d040 0x7f7cbc169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.920+0000 7f7cba7fc700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cbc159170 0x7f7cbc169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cbaffd700 1 -- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cbc159170 msgr2=0x7f7cbc169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cbaffd700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cbc159170 0x7f7cbc169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cbaffd700 1 -- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc15d040 msgr2=0x7f7cbc169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cbaffd700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc15d040 0x7f7cbc169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cbaffd700 1 -- 172.21.15.45:0/3023429947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac009420 con 0x7f7cbc0692d0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cbaffd700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cbc0692d0 0x7f7cbc0a69e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0d0ed0 tx=0x7f7cac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cb2ffd700 1 -- 172.21.15.45:0/3023429947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac01f070 con 0x7f7cbc0692d0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cb2ffd700 1 -- 172.21.15.45:0/3023429947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac011900 con 0x7f7cbc0692d0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc0c8e00 con 0x7f7cbc0692d0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cb2ffd700 1 -- 172.21.15.45:0/3023429947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac023410 con 0x7f7cbc0692d0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.921+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc0c8ac0 con 0x7f7cbc0692d0 2023-10-06T05:15:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.922+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cbc063b50 con 0x7f7cbc0692d0 2023-10-06T05:15:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.922+0000 7f7cb2ffd700 1 -- 172.21.15.45:0/3023429947 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cac02c3b0 con 0x7f7cbc0692d0 2023-10-06T05:15:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.923+0000 7f7cb2ffd700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.923+0000 7f7cb2ffd700 1 -- 172.21.15.45:0/3023429947 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7cac085b50 con 0x7f7cbc0692d0 2023-10-06T05:15:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.923+0000 7f7cba7fc700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.923+0000 7f7cba7fc700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc15d010 tx=0x7f7ca4007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:39.927+0000 7f7cb2ffd700 1 -- 172.21.15.45:0/3023429947 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cac052970 con 0x7f7cbc0692d0 2023-10-06T05:15:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.178+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7cbc15ddd0 con 0x7f7cbc0692d0 2023-10-06T05:15:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.178+0000 7f7cb2ffd700 1 -- 172.21.15.45:0/3023429947 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7cac028070 con 0x7f7cbc0692d0 2023-10-06T05:15:40.180 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 msgr2=0x7f7c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc15d010 tx=0x7f7ca4007d50 comp rx=0 tx=0).stop 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cbc0692d0 msgr2=0x7f7cbc0a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cbc0692d0 0x7f7cbc0a69e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0d0ed0 tx=0x7f7cac017000 comp rx=0 tx=0).stop 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 shutdown_connections 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cbc15d040 0x7f7cbc169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cbc0692d0 0x7f7cbc0a69e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.180+0000 7f7cc1321700 1 --2- 172.21.15.45:0/3023429947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cbc159170 0x7f7cbc169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.181+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 >> 172.21.15.45:0/3023429947 conn(0x7f7cbc06f7c0 msgr2=0x7f7cbc0cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:40.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.181+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 shutdown_connections 2023-10-06T05:15:40.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:40.181+0000 7f7cc1321700 1 -- 172.21.15.45:0/3023429947 wait complete. 2023-10-06T05:15:40.195 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:15:40.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:15:40.197 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-06T05:15:40.197 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T05:15:40.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T05:15:40.197 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:15:40.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:15:40.197 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:15:40.197 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T05:15:40.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.198 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-06T05:15:40.217 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:40.217 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:40.217 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:40.218 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42361sec preferred_lft 42361sec 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 22:88:4a:a1:8d:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::98d9:45ff:fe30:9c6e/64 scope link 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.219 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:15:40.220 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 22:88:4a:a1:8d:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:15:40.220 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::2088:4aff:fea1:8d14/64 scope link tentative 2023-10-06T05:15:40.220 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.220 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:15:40.284 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-06T05:15:40.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.285 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-06T05:15:40.345 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-06T05:15:40.345 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:15:40.345 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:15:40.345 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:15:40.345 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:15:40.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.346 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:40.346 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:40.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:15:40.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:15:40.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:15:40.346 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:40.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.450 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-06T05:15:40.468 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:15:40.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.469 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2023-10-06T05:15:40.526 INFO:teuthology.orchestra.run.smithi144.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T05:15:40.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.527 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:40.527 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:40.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:15:40.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T05:15:40.527 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:40.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.622 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T05:15:40.656 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 6a:db:14:1d:c9:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::68db:14ff:fe1d:c944/64 scope link 2023-10-06T05:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.674 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T05:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-06T05:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 6a:db:14:1d:c9:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::68db:14ff:fe1d:c944/64 scope link 2023-10-06T05:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:15:40.768 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T05:15:40.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:40.768 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:40.768 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:40.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T05:15:40.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T05:15:40.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T05:15:40.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T05:15:40.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T05:15:40.769 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:41.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:41.025 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-06T05:15:41.025 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-06T05:15:41.025 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.1 up 2023-10-06T05:15:41.025 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T05:15:41.025 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-06T05:15:41.122 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-06T05:15:41.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:41.122 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T05:15:41.140 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T05:15:41.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:15:41.140 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T05:15:41.197 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2023-10-06T05:15:41.264 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:41.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:41.330 INFO:teuthology.orchestra.run.smithi144.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:41.331 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:41.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:41.332 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2023-10-06T05:15:41.390 INFO:teuthology.orchestra.run.smithi144.stdout:46 2023-10-06T05:15:41.391 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-10-06T05:15:41.391 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-06T05:15:41.433 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2023-10-06T05:15:41.460 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:15:41.460 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:15:41.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:2023-10-06T05:15:41.514+0000 7f83c8518540 -1 init, newargv = 0x557353276880 newargc=15 2023-10-06T05:15:41.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:ceph-fuse[127037]: starting ceph client 2023-10-06T05:15:41.520 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:ceph-fuse[127037]: starting fuse 2023-10-06T05:15:41.529 INFO:teuthology.orchestra.run.smithi144.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:15:41.529 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:15:41.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:41.530 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2023-10-06T05:15:41.588 INFO:teuthology.orchestra.run.smithi144.stdout:46 2023-10-06T05:15:41.588 INFO:teuthology.orchestra.run.smithi144.stdout:48 2023-10-06T05:15:41.588 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> import re 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-06T05:15:41.589 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2023-10-06T05:15:41.590 DEBUG:teuthology.orchestra.run.smithi144:> return f 2023-10-06T05:15:41.591 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:15:41.591 DEBUG:teuthology.orchestra.run.smithi144:> 2023-10-06T05:15:41.591 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.3")) 2023-10-06T05:15:41.591 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-10-06T05:15:41.711 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.3.127037.asok 2023-10-06T05:15:41.894 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.127037.asok 2023-10-06T05:15:41.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:41.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.127037.asok status 2023-10-06T05:15:42.048 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-10-06T05:15:42.048 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2023-10-06T05:15:42.048 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:15:42.048 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:15:42.048 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "3", 2023-10-06T05:15:42.048 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2023-10-06T05:15:42.048 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "127037", 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 4622, 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 4622 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2023-10-06T05:15:42.049 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.2:0", 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 2622221003 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.2:0", 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 2622221003 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.4622 192.168.0.2:0/2622221003", 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.2:0/2622221003", 2023-10-06T05:15:42.050 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2023-10-06T05:15:42.051 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 8, 2023-10-06T05:15:42.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 24, 2023-10-06T05:15:42.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:15:42.051 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2023-10-06T05:15:42.051 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2023-10-06T05:15:42.051 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-10-06T05:15:42.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.058 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:15:42.058 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:42.082 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:15:42.082 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:42.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:42.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.159 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:15:42.159 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-06T05:15:42.222 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:15:42.223 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-06T05:15:42.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T05:15:42.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.299 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2023-10-06T05:15:42.299 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:42.318 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2023-10-06T05:15:42.318 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:42.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.318 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:42.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.386 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2023-10-06T05:15:42.386 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-06T05:15:42.445 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2023-10-06T05:15:42.446 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-06T05:15:42.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:15:42.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T05:15:42.513 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T05:15:42.532 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.671+0000 7f2e08a06700 1 Processor -- start 2023-10-06T05:15:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.671+0000 7f2e08a06700 1 -- start start 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.672+0000 7f2e08a06700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e041570b0 0x7f2e0406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.672+0000 7f2e08a06700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0415b380 con 0x7f2e04156ce0 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.672+0000 7f2e08a06700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0415b660 con 0x7f2e0415ac10 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.672+0000 7f2e08a06700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0406c010 con 0x7f2e041570b0 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.672+0000 7f2e01d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e041570b0 0x7f2e0406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.672+0000 7f2e0259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2e04156ce0 0x7f2e040afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46286/0 (socket says 172.21.15.45:46286) 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.672+0000 7f2e0259c700 1 -- 172.21.15.45:0/2328699625 learned_addr learned my addr 172.21.15.45:0/2328699625 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.673+0000 7f2e01d9b700 1 -- 172.21.15.45:0/2328699625 >> v1:172.21.15.144:6789/0 conn(0x7f2e0415ac10 legacy=0x7f2e040ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.673+0000 7f2e01d9b700 1 -- 172.21.15.45:0/2328699625 >> v1:172.21.15.45:6789/0 conn(0x7f2e04156ce0 legacy=0x7f2e040afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.673+0000 7f2e01d9b700 1 -- 172.21.15.45:0/2328699625 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0415b940 con 0x7f2e041570b0 2023-10-06T05:15:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.673+0000 7f2e01d9b700 1 --2- 172.21.15.45:0/2328699625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e0406bad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098e0 tx=0x7f2df80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f771cd8cc67beb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.673+0000 7f2e0159a700 1 -- 172.21.15.45:0/2328699625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8009e80 con 0x7f2e041570b0 2023-10-06T05:15:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.673+0000 7f2e0159a700 1 -- 172.21.15.45:0/2328699625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df800b820 con 0x7f2e041570b0 2023-10-06T05:15:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.673+0000 7f2e0159a700 1 -- 172.21.15.45:0/2328699625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800bba0 con 0x7f2e041570b0 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.674+0000 7f2e08a06700 1 -- 172.21.15.45:0/2328699625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 msgr2=0x7f2e0406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.674+0000 7f2e08a06700 1 --2- 172.21.15.45:0/2328699625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e0406bad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098e0 tx=0x7f2df80091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.674+0000 7f2e08a06700 1 -- 172.21.15.45:0/2328699625 shutdown_connections 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.674+0000 7f2e08a06700 1 --2- 172.21.15.45:0/2328699625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e0406bad0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.674+0000 7f2e08a06700 1 -- 172.21.15.45:0/2328699625 >> 172.21.15.45:0/2328699625 conn(0x7f2e0406f090 msgr2=0x7f2e0406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.674+0000 7f2e08a06700 1 -- 172.21.15.45:0/2328699625 shutdown_connections 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.674+0000 7f2e08a06700 1 -- 172.21.15.45:0/2328699625 wait complete. 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.675+0000 7f2e08a06700 1 Processor -- start 2023-10-06T05:15:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.675+0000 7f2e08a06700 1 -- start start 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e08a06700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e04156ce0 0x7f2e04169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e08a06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e040c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e08a06700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e0415ac10 0x7f2e04154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e08a06700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0416bc20 con 0x7f2e041570b0 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e08a06700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0416ad50 con 0x7f2e04156ce0 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e08a06700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04169a40 con 0x7f2e0415ac10 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e040c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e040c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52030/0 (socket says 172.21.15.45:52030) 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 -- 172.21.15.45:0/283294997 learned_addr learned my addr 172.21.15.45:0/283294997 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e0259c700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e04156ce0 0x7f2e04169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 -- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e0415ac10 msgr2=0x7f2e04154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e02d9d700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e0415ac10 0x7f2e04154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e0415ac10 0x7f2e04154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 -- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e04156ce0 msgr2=0x7f2e04169500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e04156ce0 0x7f2e04169500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 -- 172.21.15.45:0/283294997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8009420 con 0x7f2e041570b0 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e0259c700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e04156ce0 0x7f2e04169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e02d9d700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e0415ac10 0x7f2e04154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2e01d9b700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e040c8e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2e0406aed0 tx=0x7f2df8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.676+0000 7f2df27fc700 1 -- 172.21.15.45:0/283294997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800bae0 con 0x7f2e041570b0 2023-10-06T05:15:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.677+0000 7f2df27fc700 1 -- 172.21.15.45:0/283294997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8020070 con 0x7f2e041570b0 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.677+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0416afa0 con 0x7f2e041570b0 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.677+0000 7f2df27fc700 1 -- 172.21.15.45:0/283294997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df801b6a0 con 0x7f2e041570b0 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.677+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0416b560 con 0x7f2e041570b0 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.678+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0416bd60 con 0x7f2e041570b0 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.678+0000 7f2df27fc700 1 -- 172.21.15.45:0/283294997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2df8022030 con 0x7f2e041570b0 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.678+0000 7f2df27fc700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2de4066f30 0x7f2de40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.679+0000 7f2df27fc700 1 -- 172.21.15.45:0/283294997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2df8086a60 con 0x7f2e041570b0 2023-10-06T05:15:42.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.679+0000 7f2e0259c700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2de4066f30 0x7f2de40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:42.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.679+0000 7f2e0259c700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2de4066f30 0x7f2de40693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2df4001820 tx=0x7f2df4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:42.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.683+0000 7f2df27fc700 1 -- 172.21.15.45:0/283294997 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e0416bd60 con 0x7f2e041570b0 2023-10-06T05:15:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.888+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e0416bd60 con 0x7f2e041570b0 2023-10-06T05:15:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.889+0000 7f2df27fc700 1 -- 172.21.15.45:0/283294997 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f2e0416bd60 con 0x7f2e041570b0 2023-10-06T05:15:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:42.892 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2de4066f30 msgr2=0x7f2de40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2de4066f30 0x7f2de40693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2df4001820 tx=0x7f2df4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 msgr2=0x7f2e040c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e040c8e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2e0406aed0 tx=0x7f2df8016000 comp rx=0 tx=0).stop 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 shutdown_connections 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2de4066f30 0x7f2de40693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e0415ac10 0x7f2e04154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e04156ce0 0x7f2e04169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 --2- 172.21.15.45:0/283294997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e041570b0 0x7f2e040c8e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.891+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 >> 172.21.15.45:0/283294997 conn(0x7f2e0406f090 msgr2=0x7f2e04070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.892+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 shutdown_connections 2023-10-06T05:15:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:42.892+0000 7f2e08a06700 1 -- 172.21.15.45:0/283294997 wait complete. 2023-10-06T05:15:42.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:57.253547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:58.338382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T05:15:01.988547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:42.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:43.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.040+0000 7f01f96ce700 1 Processor -- start 2023-10-06T05:15:43.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.041+0000 7f01f96ce700 1 -- start start 2023-10-06T05:15:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.041+0000 7f01f96ce700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f01f4157330 0x7f01f40ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.041+0000 7f01f96ce700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40aaae0 0x7f01f40c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.041+0000 7f01f96ce700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01f40c90b0 0x7f01f40c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.041+0000 7f01f96ce700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f40c3b50 con 0x7f01f40c90b0 2023-10-06T05:15:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.041+0000 7f01f96ce700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f4157700 con 0x7f01f4157330 2023-10-06T05:15:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.041+0000 7f01f96ce700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40c98e0 con 0x7f01f40aaae0 2023-10-06T05:15:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01f40c90b0 0x7f01f40c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01f40c90b0 0x7f01f40c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52038/0 (socket says 172.21.15.45:52038) 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 -- 172.21.15.45:0/3005024820 learned_addr learned my addr 172.21.15.45:0/3005024820 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f27fc700 1 --2- 172.21.15.45:0/3005024820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40aaae0 0x7f01f40c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f2ffd700 1 --2- 172.21.15.45:0/3005024820 >> v2:172.21.15.144:3300/0 conn(0x7f01f4157330 0x7f01f40ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 -- 172.21.15.45:0/3005024820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40aaae0 msgr2=0x7f01f40c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 --2- 172.21.15.45:0/3005024820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40aaae0 0x7f01f40c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 -- 172.21.15.45:0/3005024820 >> v2:172.21.15.144:3300/0 conn(0x7f01f4157330 msgr2=0x7f01f40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 --2- 172.21.15.45:0/3005024820 >> v2:172.21.15.144:3300/0 conn(0x7f01f4157330 0x7f01f40ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 -- 172.21.15.45:0/3005024820 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40c9bc0 con 0x7f01f40c90b0 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f37fe700 1 --2- 172.21.15.45:0/3005024820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40c90b0 0x7f01f40c3fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f01f40cd1d0 tx=0x7f01ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109e3374420fcfe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f1ffb700 1 -- 172.21.15.45:0/3005024820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00e070 con 0x7f01f40c90b0 2023-10-06T05:15:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f1ffb700 1 -- 172.21.15.45:0/3005024820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00b970 con 0x7f01f40c90b0 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.042+0000 7f01f1ffb700 1 -- 172.21.15.45:0/3005024820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0092e0 con 0x7f01f40c90b0 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 -- 172.21.15.45:0/3005024820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40c90b0 msgr2=0x7f01f40c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3005024820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40c90b0 0x7f01f40c3fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f01f40cd1d0 tx=0x7f01ec009840 comp rx=0 tx=0).stop 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 -- 172.21.15.45:0/3005024820 shutdown_connections 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3005024820 >> v2:172.21.15.144:3300/0 conn(0x7f01f4157330 0x7f01f40ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3005024820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40c90b0 0x7f01f40c3fe0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3005024820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40aaae0 0x7f01f40c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 -- 172.21.15.45:0/3005024820 >> 172.21.15.45:0/3005024820 conn(0x7f01f4067e00 msgr2=0x7f01f40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 -- 172.21.15.45:0/3005024820 shutdown_connections 2023-10-06T05:15:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.043+0000 7f01f96ce700 1 -- 172.21.15.45:0/3005024820 wait complete. 2023-10-06T05:15:43.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.044+0000 7f01f96ce700 1 Processor -- start 2023-10-06T05:15:43.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.044+0000 7f01f96ce700 1 -- start start 2023-10-06T05:15:43.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f96ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40aaae0 0x7f01f41563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f96ce700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40c90b0 0x7f01f40cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f96ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f4157330 0x7f01f40ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f96ce700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f416acc0 con 0x7f01f40aaae0 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f96ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4167c20 con 0x7f01f4157330 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f96ce700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f416a670 con 0x7f01f40c90b0 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40aaae0 0x7f01f41563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40aaae0 0x7f01f41563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52042/0 (socket says 172.21.15.45:52042) 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f2ffd700 1 -- 172.21.15.45:0/3242265777 learned_addr learned my addr 172.21.15.45:0/3242265777 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f37fe700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f4157330 0x7f01f40ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.045+0000 7f01f27fc700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40c90b0 0x7f01f40cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f2ffd700 1 -- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40c90b0 msgr2=0x7f01f40cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f2ffd700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40c90b0 0x7f01f40cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f2ffd700 1 -- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f4157330 msgr2=0x7f01f40ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f2ffd700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f4157330 0x7f01f40ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f2ffd700 1 -- 172.21.15.45:0/3242265777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec009ae0 con 0x7f01f40aaae0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f2ffd700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40aaae0 0x7f01f41563c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f01e400ec80 tx=0x7f01e4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01eaffd700 1 -- 172.21.15.45:0/3242265777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e400c850 con 0x7f01f40aaae0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01eaffd700 1 -- 172.21.15.45:0/3242265777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e400c9b0 con 0x7f01f40aaae0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f416a950 con 0x7f01f40aaae0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01eaffd700 1 -- 172.21.15.45:0/3242265777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e400cd30 con 0x7f01f40aaae0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.046+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f4166ef0 con 0x7f01f40aaae0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.047+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f416afa0 con 0x7f01f40aaae0 2023-10-06T05:15:43.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.048+0000 7f01eaffd700 1 -- 172.21.15.45:0/3242265777 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01e4013680 con 0x7f01f40aaae0 2023-10-06T05:15:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.048+0000 7f01eaffd700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01d4066f30 0x7f01d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.048+0000 7f01eaffd700 1 -- 172.21.15.45:0/3242265777 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f01e4084ee0 con 0x7f01f40aaae0 2023-10-06T05:15:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.048+0000 7f01f27fc700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01d4066f30 0x7f01d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.049+0000 7f01f27fc700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01d4066f30 0x7f01d40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f01f406c470 tx=0x7f01dc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.052+0000 7f01eaffd700 1 -- 172.21.15.45:0/3242265777 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01e4051d00 con 0x7f01f40aaae0 2023-10-06T05:15:43.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.253+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01f4063500 con 0x7f01f40aaae0 2023-10-06T05:15:43.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.254+0000 7f01eaffd700 1 -- 172.21.15.45:0/3242265777 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01e40210a0 con 0x7f01f40aaae0 2023-10-06T05:15:43.255 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:15:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01d4066f30 msgr2=0x7f01d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01d4066f30 0x7f01d40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f01f406c470 tx=0x7f01dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40aaae0 msgr2=0x7f01f41563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40aaae0 0x7f01f41563c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f01e400ec80 tx=0x7f01e4009120 comp rx=0 tx=0).stop 2023-10-06T05:15:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 shutdown_connections 2023-10-06T05:15:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01d4066f30 0x7f01d40693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f4157330 0x7f01f40ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f40aaae0 0x7f01f41563c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 --2- 172.21.15.45:0/3242265777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f40c90b0 0x7f01f40cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 >> 172.21.15.45:0/3242265777 conn(0x7f01f4067e00 msgr2=0x7f01f414cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 shutdown_connections 2023-10-06T05:15:43.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.256+0000 7f01f96ce700 1 -- 172.21.15.45:0/3242265777 wait complete. 2023-10-06T05:15:43.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:43.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.404+0000 7f8387196700 1 Processor -- start 2023-10-06T05:15:43.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.404+0000 7f8387196700 1 -- start start 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.405+0000 7f8387196700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8380159520 0x7f838015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.405+0000 7f8387196700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83800c9c90 con 0x7f8380159150 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.405+0000 7f8387196700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838015d8e0 con 0x7f838015d3d0 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.405+0000 7f8387196700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838015dbc0 con 0x7f8380159520 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.405+0000 7f837ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8380159520 0x7f838015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.405+0000 7f8384f32700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8380159150 0x7f83800ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46292/0 (socket says 172.21.15.45:46292) 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.405+0000 7f8384f32700 1 -- 172.21.15.45:0/2885428093 learned_addr learned my addr 172.21.15.45:0/2885428093 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837ffff700 1 -- 172.21.15.45:0/2885428093 >> v1:172.21.15.144:6789/0 conn(0x7f838015d3d0 legacy=0x7f8380067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837ffff700 1 -- 172.21.15.45:0/2885428093 >> v1:172.21.15.45:6789/0 conn(0x7f8380159150 legacy=0x7f83800ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837ffff700 1 -- 172.21.15.45:0/2885428093 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83800c9920 con 0x7f8380159520 2023-10-06T05:15:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837f7fe700 1 -- 172.21.15.45:0/2885428093 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533687978 0 0) 0x7f83800c9c90 con 0x7f8380159150 2023-10-06T05:15:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837ffff700 1 --2- 172.21.15.45:0/2885428093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159520 0x7f838015ce90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f836c0098e0 tx=0x7f836c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855c0891a4ff42e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837f7fe700 1 -- 172.21.15.45:0/2885428093 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c009e80 con 0x7f8380159520 2023-10-06T05:15:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837f7fe700 1 -- 172.21.15.45:0/2885428093 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c00b860 con 0x7f8380159520 2023-10-06T05:15:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.406+0000 7f837f7fe700 1 -- 172.21.15.45:0/2885428093 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c00bbe0 con 0x7f8380159520 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.407+0000 7f8387196700 1 -- 172.21.15.45:0/2885428093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159520 msgr2=0x7f838015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.407+0000 7f8387196700 1 --2- 172.21.15.45:0/2885428093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159520 0x7f838015ce90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f836c0098e0 tx=0x7f836c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.407+0000 7f8387196700 1 -- 172.21.15.45:0/2885428093 shutdown_connections 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.407+0000 7f8387196700 1 --2- 172.21.15.45:0/2885428093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159520 0x7f838015ce90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.407+0000 7f8387196700 1 -- 172.21.15.45:0/2885428093 >> 172.21.15.45:0/2885428093 conn(0x7f83800691c0 msgr2=0x7f83800cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.407+0000 7f8387196700 1 -- 172.21.15.45:0/2885428093 shutdown_connections 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.407+0000 7f8387196700 1 -- 172.21.15.45:0/2885428093 wait complete. 2023-10-06T05:15:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.408+0000 7f8387196700 1 Processor -- start 2023-10-06T05:15:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.409+0000 7f8387196700 1 -- start start 2023-10-06T05:15:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.409+0000 7f8387196700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159150 0x7f838016a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.409+0000 7f8387196700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f838015d3d0 0x7f8380159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8387196700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83800c3ef0 0x7f83800c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8387196700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838016aad0 con 0x7f8380159150 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8387196700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8380159e40 con 0x7f838015d3d0 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8387196700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838015b630 con 0x7f83800c3ef0 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159150 0x7f838016a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159150 0x7f838016a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52056/0 (socket says 172.21.15.45:52056) 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 -- 172.21.15.45:0/1272659047 learned_addr learned my addr 172.21.15.45:0/1272659047 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f837ffff700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f838015d3d0 0x7f8380159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8385733700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83800c3ef0 0x7f83800c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 -- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83800c3ef0 msgr2=0x7f83800c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83800c3ef0 0x7f83800c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 -- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f838015d3d0 msgr2=0x7f8380159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f838015d3d0 0x7f8380159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 -- 172.21.15.45:0/1272659047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c009420 con 0x7f8380159150 2023-10-06T05:15:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8384f32700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159150 0x7f838016a590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f837400c7e0 tx=0x7f8374007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f837cff9700 1 -- 172.21.15.45:0/1272659047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374005b30 con 0x7f8380159150 2023-10-06T05:15:43.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f837cff9700 1 -- 172.21.15.45:0/1272659047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8374005c90 con 0x7f8380159150 2023-10-06T05:15:43.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838015a050 con 0x7f8380159150 2023-10-06T05:15:43.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.410+0000 7f837cff9700 1 -- 172.21.15.45:0/1272659047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374007300 con 0x7f8380159150 2023-10-06T05:15:43.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.411+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838015a610 con 0x7f8380159150 2023-10-06T05:15:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.411+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838006e2d0 con 0x7f8380159150 2023-10-06T05:15:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.412+0000 7f837cff9700 1 -- 172.21.15.45:0/1272659047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f837401b3b0 con 0x7f8380159150 2023-10-06T05:15:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.412+0000 7f837cff9700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8364066f30 0x7f83640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.412+0000 7f837cff9700 1 -- 172.21.15.45:0/1272659047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8374083f40 con 0x7f8380159150 2023-10-06T05:15:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.413+0000 7f837ffff700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8364066f30 0x7f83640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.413+0000 7f837ffff700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8364066f30 0x7f83640693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f836c0030c0 tx=0x7f836c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.417+0000 7f837cff9700 1 -- 172.21.15.45:0/1272659047 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8374050cc0 con 0x7f8380159150 2023-10-06T05:15:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.615+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f83800cdba0 con 0x7f8380159150 2023-10-06T05:15:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.616+0000 7f837cff9700 1 -- 172.21.15.45:0/1272659047 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8374017020 con 0x7f8380159150 2023-10-06T05:15:43.618 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8364066f30 msgr2=0x7f83640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8364066f30 0x7f83640693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f836c0030c0 tx=0x7f836c016000 comp rx=0 tx=0).stop 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159150 msgr2=0x7f838016a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159150 0x7f838016a590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f837400c7e0 tx=0x7f8374007df0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 shutdown_connections 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8364066f30 0x7f83640693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83800c3ef0 0x7f83800c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8380159150 0x7f838016a590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 --2- 172.21.15.45:0/1272659047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f838015d3d0 0x7f8380159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 >> 172.21.15.45:0/1272659047 conn(0x7f83800691c0 msgr2=0x7f83800ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 shutdown_connections 2023-10-06T05:15:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.618+0000 7f8387196700 1 -- 172.21.15.45:0/1272659047 wait complete. 2023-10-06T05:15:43.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:43.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.765+0000 7fb57a0f5700 1 Processor -- start 2023-10-06T05:15:43.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.765+0000 7fb57a0f5700 1 -- start start 2023-10-06T05:15:43.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb57a0f5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb57406aa70 0x7fb5740c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb57a0f5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb57415c8c0 0x7fb5740aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb57a0f5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb574067d80 con 0x7fb5740c7050 2023-10-06T05:15:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb57a0f5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740ac620 con 0x7fb57406aa70 2023-10-06T05:15:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb57a0f5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740c75b0 con 0x7fb57415c8c0 2023-10-06T05:15:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb5737fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb57406aa70 0x7fb5740c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb572ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb5740c7050 0x7fb57415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46300/0 (socket says 172.21.15.45:46300) 2023-10-06T05:15:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb572ffd700 1 -- 172.21.15.45:0/1188026082 learned_addr learned my addr 172.21.15.45:0/1188026082 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb573fff700 1 --2- 172.21.15.45:0/1188026082 >> v2:172.21.15.144:3300/0 conn(0x7fb57415c8c0 0x7fb5740aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb5737fe700 1 -- 172.21.15.45:0/1188026082 >> v2:172.21.15.144:3300/0 conn(0x7fb57415c8c0 msgr2=0x7fb5740aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb5737fe700 1 --2- 172.21.15.45:0/1188026082 >> v2:172.21.15.144:3300/0 conn(0x7fb57415c8c0 0x7fb5740aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb5737fe700 1 -- 172.21.15.45:0/1188026082 >> v1:172.21.15.45:6789/0 conn(0x7fb5740c7050 legacy=0x7fb57415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb5737fe700 1 -- 172.21.15.45:0/1188026082 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740c7890 con 0x7fb57406aa70 2023-10-06T05:15:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.766+0000 7fb5727fc700 1 -- 172.21.15.45:0/1188026082 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603582948 0 0) 0x7fb574067d80 con 0x7fb5740c7050 2023-10-06T05:15:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb5737fe700 1 --2- 172.21.15.45:0/1188026082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb5740c6b10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5740c8260 tx=0x7fb5640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916b8fcb13aed263 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb5727fc700 1 -- 172.21.15.45:0/1188026082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400b6b0 con 0x7fb57406aa70 2023-10-06T05:15:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb5727fc700 1 -- 172.21.15.45:0/1188026082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56400b810 con 0x7fb57406aa70 2023-10-06T05:15:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb5727fc700 1 -- 172.21.15.45:0/1188026082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400bbb0 con 0x7fb57406aa70 2023-10-06T05:15:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 -- 172.21.15.45:0/1188026082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 msgr2=0x7fb5740c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/1188026082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb5740c6b10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5740c8260 tx=0x7fb5640091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 -- 172.21.15.45:0/1188026082 shutdown_connections 2023-10-06T05:15:43.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/1188026082 >> v2:172.21.15.144:3300/0 conn(0x7fb57415c8c0 0x7fb5740aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/1188026082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb5740c6b10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 -- 172.21.15.45:0/1188026082 >> 172.21.15.45:0/1188026082 conn(0x7fb5740cc570 msgr2=0x7fb5740cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 -- 172.21.15.45:0/1188026082 shutdown_connections 2023-10-06T05:15:43.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.767+0000 7fb57a0f5700 1 -- 172.21.15.45:0/1188026082 wait complete. 2023-10-06T05:15:43.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.769+0000 7fb57a0f5700 1 Processor -- start 2023-10-06T05:15:43.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.769+0000 7fb57a0f5700 1 -- start start 2023-10-06T05:15:43.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb57a0f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb574159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb57a0f5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5740c7050 0x7fb574155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb57a0f5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57415c8c0 0x7fb574158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb57a0f5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57415a4d0 con 0x7fb57406aa70 2023-10-06T05:15:43.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb57a0f5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740c1310 con 0x7fb57415c8c0 2023-10-06T05:15:43.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb57a0f5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740c15f0 con 0x7fb5740c7050 2023-10-06T05:15:43.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb574159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb574159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52068/0 (socket says 172.21.15.45:52068) 2023-10-06T05:15:43.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 -- 172.21.15.45:0/3672428205 learned_addr learned my addr 172.21.15.45:0/3672428205 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:43.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb572ffd700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5740c7050 0x7fb574155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb573fff700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57415c8c0 0x7fb574158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 -- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5740c7050 msgr2=0x7fb574155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5740c7050 0x7fb574155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 -- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57415c8c0 msgr2=0x7fb574158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57415c8c0 0x7fb574158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 -- 172.21.15.45:0/3672428205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564009420 con 0x7fb57406aa70 2023-10-06T05:15:43.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb5737fe700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb574159f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb57406fbf0 tx=0x7fb564017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb56b7fe700 1 -- 172.21.15.45:0/3672428205 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011420 con 0x7fb57406aa70 2023-10-06T05:15:43.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb56b7fe700 1 -- 172.21.15.45:0/3672428205 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564011580 con 0x7fb57406aa70 2023-10-06T05:15:43.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5740c18d0 con 0x7fb57406aa70 2023-10-06T05:15:43.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.770+0000 7fb56b7fe700 1 -- 172.21.15.45:0/3672428205 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011940 con 0x7fb57406aa70 2023-10-06T05:15:43.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.771+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5740c1e90 con 0x7fb57406aa70 2023-10-06T05:15:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.771+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740c2080 con 0x7fb57406aa70 2023-10-06T05:15:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.772+0000 7fb56b7fe700 1 -- 172.21.15.45:0/3672428205 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb564011cf0 con 0x7fb57406aa70 2023-10-06T05:15:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.772+0000 7fb56b7fe700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb554066f30 0x7fb5540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.772+0000 7fb56b7fe700 1 -- 172.21.15.45:0/3672428205 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb564085e50 con 0x7fb57406aa70 2023-10-06T05:15:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.773+0000 7fb572ffd700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb554066f30 0x7fb5540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.773+0000 7fb572ffd700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb554066f30 0x7fb5540693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb57415ba30 tx=0x7fb55c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.777+0000 7fb56b7fe700 1 -- 172.21.15.45:0/3672428205 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb564052c70 con 0x7fb57406aa70 2023-10-06T05:15:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.981+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5740c7b70 con 0x7fb57406aa70 2023-10-06T05:15:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.982+0000 7fb56b7fe700 1 -- 172.21.15.45:0/3672428205 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb564015020 con 0x7fb57406aa70 2023-10-06T05:15:43.984 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb554066f30 msgr2=0x7fb5540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb554066f30 0x7fb5540693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb57415ba30 tx=0x7fb55c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 msgr2=0x7fb574159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb574159f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb57406fbf0 tx=0x7fb564017000 comp rx=0 tx=0).stop 2023-10-06T05:15:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 shutdown_connections 2023-10-06T05:15:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb554066f30 0x7fb5540693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57415c8c0 0x7fb574158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb57406aa70 0x7fb574159f90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 --2- 172.21.15.45:0/3672428205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5740c7050 0x7fb574155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:43.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 >> 172.21.15.45:0/3672428205 conn(0x7fb5740cc570 msgr2=0x7fb5740acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:43.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 shutdown_connections 2023-10-06T05:15:43.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:43.985+0000 7fb57a0f5700 1 -- 172.21.15.45:0/3672428205 wait complete. 2023-10-06T05:15:44.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.135+0000 7f8fae997700 1 Processor -- start 2023-10-06T05:15:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.135+0000 7f8fae997700 1 -- start start 2023-10-06T05:15:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fae997700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fa8153840 0x7f8fa815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fae997700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fa815cd70 0x7f8fa80ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fae997700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa80ce770 con 0x7f8fa80cd060 2023-10-06T05:15:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fae997700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa8153c10 con 0x7f8fa815cd70 2023-10-06T05:15:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fae997700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa80cd450 con 0x7f8fa8153840 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fa815cd70 0x7f8fa80ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fa815cd70 0x7f8fa80ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52078/0 (socket says 172.21.15.45:52078) 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 -- 172.21.15.45:0/1869360166 learned_addr learned my addr 172.21.15.45:0/1869360166 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa7fff700 1 --2- 172.21.15.45:0/1869360166 >> v2:172.21.15.144:3300/0 conn(0x7f8fa8153840 0x7f8fa815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 -- 172.21.15.45:0/1869360166 >> v2:172.21.15.144:3300/0 conn(0x7f8fa8153840 msgr2=0x7f8fa815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 --2- 172.21.15.45:0/1869360166 >> v2:172.21.15.144:3300/0 conn(0x7f8fa8153840 0x7f8fa815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 -- 172.21.15.45:0/1869360166 >> v1:172.21.15.144:6789/0 conn(0x7f8fa80cd060 legacy=0x7f8fa8068bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 -- 172.21.15.45:0/1869360166 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa815d650 con 0x7f8fa815cd70 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa7fff700 1 --2- 172.21.15.45:0/1869360166 >> v2:172.21.15.144:3300/0 conn(0x7f8fa8153840 0x7f8fa815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.136+0000 7f8fa77fe700 1 --2- 172.21.15.45:0/1869360166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa815cd70 0x7f8fa80ccb20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80caf40 tx=0x7f8f9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2286ccb4f5de67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fa6ffd700 1 -- 172.21.15.45:0/1869360166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00b8b0 con 0x7f8fa815cd70 2023-10-06T05:15:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fa6ffd700 1 -- 172.21.15.45:0/1869360166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c00ba10 con 0x7f8fa815cd70 2023-10-06T05:15:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fa6ffd700 1 -- 172.21.15.45:0/1869360166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00bdb0 con 0x7f8fa815cd70 2023-10-06T05:15:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 -- 172.21.15.45:0/1869360166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa815cd70 msgr2=0x7f8fa80ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 --2- 172.21.15.45:0/1869360166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa815cd70 0x7f8fa80ccb20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80caf40 tx=0x7f8f9c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:44.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 -- 172.21.15.45:0/1869360166 shutdown_connections 2023-10-06T05:15:44.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 --2- 172.21.15.45:0/1869360166 >> v2:172.21.15.144:3300/0 conn(0x7f8fa8153840 0x7f8fa815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 --2- 172.21.15.45:0/1869360166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa815cd70 0x7f8fa80ccb20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 -- 172.21.15.45:0/1869360166 >> 172.21.15.45:0/1869360166 conn(0x7f8fa806ed50 msgr2=0x7f8fa806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:44.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 -- 172.21.15.45:0/1869360166 shutdown_connections 2023-10-06T05:15:44.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.137+0000 7f8fae997700 1 -- 172.21.15.45:0/1869360166 wait complete. 2023-10-06T05:15:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.138+0000 7f8fae997700 1 Processor -- start 2023-10-06T05:15:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fae997700 1 -- start start 2023-10-06T05:15:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fae997700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa80cd060 0x7f8fa80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fae997700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fa8153840 0x7f8fa8158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fae997700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fa815cd70 0x7f8fa8150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fae997700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa81591b0 con 0x7f8fa80cd060 2023-10-06T05:15:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fae997700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80c8640 con 0x7f8fa8153840 2023-10-06T05:15:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fae997700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80b5620 con 0x7f8fa815cd70 2023-10-06T05:15:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.139+0000 7f8fa7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa80cd060 0x7f8fa80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa80cd060 0x7f8fa80b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52088/0 (socket says 172.21.15.45:52088) 2023-10-06T05:15:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 -- 172.21.15.45:0/2041218189 learned_addr learned my addr 172.21.15.45:0/2041218189 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8facf34700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fa815cd70 0x7f8fa8150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa77fe700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fa8153840 0x7f8fa8158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 -- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fa815cd70 msgr2=0x7f8fa8150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fa815cd70 0x7f8fa8150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 -- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fa8153840 msgr2=0x7f8fa8158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fa8153840 0x7f8fa8158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 -- 172.21.15.45:0/2041218189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c0094f0 con 0x7f8fa80cd060 2023-10-06T05:15:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fa7fff700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa80cd060 0x7f8fa80b4f90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800c830 tx=0x7f8f98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8f8ffff700 1 -- 172.21.15.45:0/2041218189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800a680 con 0x7f8fa80cd060 2023-10-06T05:15:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8f8ffff700 1 -- 172.21.15.45:0/2041218189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9800a7e0 con 0x7f8fa80cd060 2023-10-06T05:15:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa80c7d10 con 0x7f8fa80cd060 2023-10-06T05:15:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8f8ffff700 1 -- 172.21.15.45:0/2041218189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800aba0 con 0x7f8fa80cd060 2023-10-06T05:15:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.140+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa80c82d0 con 0x7f8fa80cd060 2023-10-06T05:15:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.141+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa8088a60 con 0x7f8fa80cd060 2023-10-06T05:15:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.142+0000 7f8f8ffff700 1 -- 172.21.15.45:0/2041218189 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f980113f0 con 0x7f8fa80cd060 2023-10-06T05:15:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.142+0000 7f8f8ffff700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f94066f30 0x7f8f940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.142+0000 7f8f8ffff700 1 -- 172.21.15.45:0/2041218189 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8f98084e90 con 0x7f8fa80cd060 2023-10-06T05:15:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.143+0000 7f8fa77fe700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f94066f30 0x7f8f940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.143+0000 7f8fa77fe700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f94066f30 0x7f8f940693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80caf40 tx=0x7f8f9c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.147+0000 7f8f8ffff700 1 -- 172.21.15.45:0/2041218189 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f98051cb0 con 0x7f8fa80cd060 2023-10-06T05:15:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.349+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fa815d930 con 0x7f8fa80cd060 2023-10-06T05:15:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.349+0000 7f8f8ffff700 1 -- 172.21.15.45:0/2041218189 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f980150a0 con 0x7f8fa80cd060 2023-10-06T05:15:44.352 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:15:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.351+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f94066f30 msgr2=0x7f8f940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.351+0000 7f8fae997700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f94066f30 0x7f8f940693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80caf40 tx=0x7f8f9c016000 comp rx=0 tx=0).stop 2023-10-06T05:15:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.351+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa80cd060 msgr2=0x7f8fa80b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.351+0000 7f8fae997700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa80cd060 0x7f8fa80b4f90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800c830 tx=0x7f8f98008040 comp rx=0 tx=0).stop 2023-10-06T05:15:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 shutdown_connections 2023-10-06T05:15:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f94066f30 0x7f8f940693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fa815cd70 0x7f8fa8150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fa80cd060 0x7f8fa80b4f90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 --2- 172.21.15.45:0/2041218189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fa8153840 0x7f8fa8158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 >> 172.21.15.45:0/2041218189 conn(0x7f8fa806ed50 msgr2=0x7f8fa815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 shutdown_connections 2023-10-06T05:15:44.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.352+0000 7f8fae997700 1 -- 172.21.15.45:0/2041218189 wait complete. 2023-10-06T05:15:44.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T05:15:44.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.501+0000 7f17300ff700 1 Processor -- start 2023-10-06T05:15:44.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.501+0000 7f17300ff700 1 -- start start 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f17300ff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f172806ae40 0x7f172815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f17300ff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f172815c6f0 0x7f17280aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f17300ff700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1728067d80 con 0x7f172806aa70 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f17300ff700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17280ac620 con 0x7f172815c6f0 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f17300ff700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172815cb70 con 0x7f172806ae40 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f172e69c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f172815c6f0 0x7f17280aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f172de9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f172806aa70 0x7f17280c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46316/0 (socket says 172.21.15.45:46316) 2023-10-06T05:15:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.502+0000 7f172de9b700 1 -- 172.21.15.45:0/2466062322 learned_addr learned my addr 172.21.15.45:0/2466062322 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172d69a700 1 --2- 172.21.15.45:0/2466062322 >> v2:172.21.15.144:3300/0 conn(0x7f172806ae40 0x7f172815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172e69c700 1 -- 172.21.15.45:0/2466062322 >> v2:172.21.15.144:3300/0 conn(0x7f172806ae40 msgr2=0x7f172815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172e69c700 1 --2- 172.21.15.45:0/2466062322 >> v2:172.21.15.144:3300/0 conn(0x7f172806ae40 0x7f172815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172e69c700 1 -- 172.21.15.45:0/2466062322 >> v1:172.21.15.45:6789/0 conn(0x7f172806aa70 legacy=0x7f17280c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172e69c700 1 -- 172.21.15.45:0/2466062322 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280c75b0 con 0x7f172815c6f0 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172ce99700 1 -- 172.21.15.45:0/2466062322 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826138116 0 0) 0x7f1728067d80 con 0x7f172806aa70 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172e69c700 1 --2- 172.21.15.45:0/2466062322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172815c6f0 0x7f17280aca70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17280cd5e0 tx=0x7f1720009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dadfaf93bd24e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172ce99700 1 -- 172.21.15.45:0/2466062322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000e070 con 0x7f172815c6f0 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172ce99700 1 -- 172.21.15.45:0/2466062322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172000bc30 con 0x7f172815c6f0 2023-10-06T05:15:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.503+0000 7f172ce99700 1 -- 172.21.15.45:0/2466062322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720012430 con 0x7f172815c6f0 2023-10-06T05:15:44.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 -- 172.21.15.45:0/2466062322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172815c6f0 msgr2=0x7f17280aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 --2- 172.21.15.45:0/2466062322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172815c6f0 0x7f17280aca70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17280cd5e0 tx=0x7f1720009780 comp rx=0 tx=0).stop 2023-10-06T05:15:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 -- 172.21.15.45:0/2466062322 shutdown_connections 2023-10-06T05:15:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 --2- 172.21.15.45:0/2466062322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172815c6f0 0x7f17280aca70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 --2- 172.21.15.45:0/2466062322 >> v2:172.21.15.144:3300/0 conn(0x7f172806ae40 0x7f172815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 -- 172.21.15.45:0/2466062322 >> 172.21.15.45:0/2466062322 conn(0x7f17280cc570 msgr2=0x7f17280cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 -- 172.21.15.45:0/2466062322 shutdown_connections 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.504+0000 7f17300ff700 1 -- 172.21.15.45:0/2466062322 wait complete. 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.505+0000 7f17300ff700 1 Processor -- start 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.505+0000 7f17300ff700 1 -- start start 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f17300ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172806aa70 0x7f1728158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f17300ff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172806ae40 0x7f17280b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f17300ff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f172815c6f0 0x7f17280b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f17300ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280c5870 con 0x7f172806aa70 2023-10-06T05:15:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f17300ff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280c6390 con 0x7f172806ae40 2023-10-06T05:15:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f17300ff700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17280c5af0 con 0x7f172815c6f0 2023-10-06T05:15:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172806aa70 0x7f1728158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172806aa70 0x7f1728158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52102/0 (socket says 172.21.15.45:52102) 2023-10-06T05:15:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 -- 172.21.15.45:0/1791452000 learned_addr learned my addr 172.21.15.45:0/1791452000 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172e69c700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f172815c6f0 0x7f17280b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172d69a700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172806ae40 0x7f17280b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 -- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f172815c6f0 msgr2=0x7f17280b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f172815c6f0 0x7f17280b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 -- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172806ae40 msgr2=0x7f17280b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172806ae40 0x7f17280b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.506+0000 7f172de9b700 1 -- 172.21.15.45:0/1791452000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1720009a20 con 0x7f172806aa70 2023-10-06T05:15:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.507+0000 7f172de9b700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172806aa70 0x7f1728158de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f172400c7e0 tx=0x7f1724007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.507+0000 7f171dffb700 1 -- 172.21.15.45:0/1791452000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724005b30 con 0x7f172806aa70 2023-10-06T05:15:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.507+0000 7f171dffb700 1 -- 172.21.15.45:0/1791452000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724005c90 con 0x7f172806aa70 2023-10-06T05:15:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.507+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17280c5dd0 con 0x7f172806aa70 2023-10-06T05:15:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.507+0000 7f171dffb700 1 -- 172.21.15.45:0/1791452000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724007300 con 0x7f172806aa70 2023-10-06T05:15:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.507+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17280c0050 con 0x7f172806aa70 2023-10-06T05:15:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.508+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17280c0270 con 0x7f172806aa70 2023-10-06T05:15:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.508+0000 7f171dffb700 1 -- 172.21.15.45:0/1791452000 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f172401b400 con 0x7f172806aa70 2023-10-06T05:15:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.509+0000 7f171dffb700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f170c066f30 0x7f170c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.509+0000 7f171dffb700 1 -- 172.21.15.45:0/1791452000 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1724084470 con 0x7f172806aa70 2023-10-06T05:15:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.509+0000 7f172d69a700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f170c066f30 0x7f170c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.510+0000 7f172d69a700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f170c066f30 0x7f170c0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f172806b660 tx=0x7f1718008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.513+0000 7f171dffb700 1 -- 172.21.15.45:0/1791452000 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17240513c0 con 0x7f172806aa70 2023-10-06T05:15:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.714+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f17280c7890 con 0x7f172806aa70 2023-10-06T05:15:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.714+0000 7f171dffb700 1 -- 172.21.15.45:0/1791452000 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1724007730 con 0x7f172806aa70 2023-10-06T05:15:44.716 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f170c066f30 msgr2=0x7f170c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f170c066f30 0x7f170c0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f172806b660 tx=0x7f1718008040 comp rx=0 tx=0).stop 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172806aa70 msgr2=0x7f1728158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172806aa70 0x7f1728158de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f172400c7e0 tx=0x7f1724007df0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 shutdown_connections 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f170c066f30 0x7f170c0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f172815c6f0 0x7f17280b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172806aa70 0x7f1728158de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 --2- 172.21.15.45:0/1791452000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172806ae40 0x7f17280b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 >> 172.21.15.45:0/1791452000 conn(0x7f17280cc570 msgr2=0x7f172815ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 shutdown_connections 2023-10-06T05:15:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.716+0000 7f17300ff700 1 -- 172.21.15.45:0/1791452000 wait complete. 2023-10-06T05:15:44.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.862+0000 7f54fb456700 1 Processor -- start 2023-10-06T05:15:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.862+0000 7f54fb456700 1 -- start start 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54fb456700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f54f4153620 0x7f54f415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54fb456700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f40ccfc0 0x7f54f415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54fb456700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f40ce690 con 0x7f54f41539f0 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54fb456700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f415a7a0 con 0x7f54f40ccfc0 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54fb456700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f40ce3f0 con 0x7f54f4153620 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f99f3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f40ccfc0 0x7f54f415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f99f3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f40ccfc0 0x7f54f415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52112/0 (socket says 172.21.15.45:52112) 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f99f3700 1 -- 172.21.15.45:0/4000394201 learned_addr learned my addr 172.21.15.45:0/4000394201 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f99f3700 1 -- 172.21.15.45:0/4000394201 >> v2:172.21.15.144:3300/0 conn(0x7f54f4153620 msgr2=0x7f54f415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f91f2700 1 --2- 172.21.15.45:0/4000394201 >> v2:172.21.15.144:3300/0 conn(0x7f54f4153620 0x7f54f415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f99f3700 1 --2- 172.21.15.45:0/4000394201 >> v2:172.21.15.144:3300/0 conn(0x7f54f4153620 0x7f54f415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f99f3700 1 -- 172.21.15.45:0/4000394201 >> v1:172.21.15.144:6789/0 conn(0x7f54f41539f0 legacy=0x7f54f40cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f99f3700 1 -- 172.21.15.45:0/4000394201 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f415d510 con 0x7f54f40ccfc0 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.863+0000 7f54f91f2700 1 --2- 172.21.15.45:0/4000394201 >> v2:172.21.15.144:3300/0 conn(0x7f54f4153620 0x7f54f415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54f99f3700 1 --2- 172.21.15.45:0/4000394201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f40ccfc0 0x7f54f415cdc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f54f40c6f30 tx=0x7f54ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd647c2c6a9e7a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54ebfff700 1 -- 172.21.15.45:0/4000394201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00e070 con 0x7f54f40ccfc0 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54ebfff700 1 -- 172.21.15.45:0/4000394201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00ba10 con 0x7f54f40ccfc0 2023-10-06T05:15:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54ebfff700 1 -- 172.21.15.45:0/4000394201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0092e0 con 0x7f54f40ccfc0 2023-10-06T05:15:44.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54fb456700 1 -- 172.21.15.45:0/4000394201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f40ccfc0 msgr2=0x7f54f415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54fb456700 1 --2- 172.21.15.45:0/4000394201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f40ccfc0 0x7f54f415cdc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f54f40c6f30 tx=0x7f54ec009840 comp rx=0 tx=0).stop 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54fb456700 1 -- 172.21.15.45:0/4000394201 shutdown_connections 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54fb456700 1 --2- 172.21.15.45:0/4000394201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f40ccfc0 0x7f54f415cdc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54fb456700 1 --2- 172.21.15.45:0/4000394201 >> v2:172.21.15.144:3300/0 conn(0x7f54f4153620 0x7f54f415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.864+0000 7f54fb456700 1 -- 172.21.15.45:0/4000394201 >> 172.21.15.45:0/4000394201 conn(0x7f54f406ed50 msgr2=0x7f54f406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.865+0000 7f54fb456700 1 -- 172.21.15.45:0/4000394201 shutdown_connections 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.865+0000 7f54fb456700 1 -- 172.21.15.45:0/4000394201 wait complete. 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.866+0000 7f54fb456700 1 Processor -- start 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.866+0000 7f54fb456700 1 -- start start 2023-10-06T05:15:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54fb456700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f40ccfc0 0x7f54f416d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54fb456700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f4153620 0x7f54f414e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54fb456700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f414f170 0x7f54f4158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54fb456700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41518e0 con 0x7f54f4153620 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54fb456700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f416d540 con 0x7f54f414f170 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54fb456700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f40b5950 con 0x7f54f40ccfc0 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54f89f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f4153620 0x7f54f414e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54f89f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f4153620 0x7f54f414e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52118/0 (socket says 172.21.15.45:52118) 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54f89f1700 1 -- 172.21.15.45:0/137572879 learned_addr learned my addr 172.21.15.45:0/137572879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54f99f3700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f414f170 0x7f54f4158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.867+0000 7f54f91f2700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f40ccfc0 0x7f54f416d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.868+0000 7f54f89f1700 1 -- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f40ccfc0 msgr2=0x7f54f416d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.868+0000 7f54f89f1700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f40ccfc0 0x7f54f416d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.868+0000 7f54f89f1700 1 -- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f414f170 msgr2=0x7f54f4158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:44.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.868+0000 7f54f89f1700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f414f170 0x7f54f4158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:44.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.868+0000 7f54f89f1700 1 -- 172.21.15.45:0/137572879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec009ae0 con 0x7f54f4153620 2023-10-06T05:15:44.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.868+0000 7f54f89f1700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f4153620 0x7f54f414e9a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f54e400b610 tx=0x7f54e40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.870+0000 7f54e97fa700 1 -- 172.21.15.45:0/137572879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400f510 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.870+0000 7f54e97fa700 1 -- 172.21.15.45:0/137572879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e400f6b0 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.870+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f40b5c30 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.870+0000 7f54e97fa700 1 -- 172.21.15.45:0/137572879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400fa70 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.870+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f416d1d0 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.871+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f4088a60 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.871+0000 7f54e97fa700 1 -- 172.21.15.45:0/137572879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54e401a400 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.872+0000 7f54e97fa700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54d8066f30 0x7f54d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.872+0000 7f54e97fa700 1 -- 172.21.15.45:0/137572879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f54e4084470 con 0x7f54f4153620 2023-10-06T05:15:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.872+0000 7f54f91f2700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54d8066f30 0x7f54d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.872+0000 7f54f91f2700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54d8066f30 0x7f54d80693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54f40b36d0 tx=0x7f54f0006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:44.878+0000 7f54e97fa700 1 -- 172.21.15.45:0/137572879 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54e4051270 con 0x7f54f4153620 2023-10-06T05:15:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.080+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54f404bb40 con 0x7f54f4153620 2023-10-06T05:15:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.081+0000 7f54e97fa700 1 -- 172.21.15.45:0/137572879 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f54e40091f0 con 0x7f54f4153620 2023-10-06T05:15:45.084 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:45.084 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:04.403597+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:45.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.083+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54d8066f30 msgr2=0x7f54d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.083+0000 7f54fb456700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54d8066f30 0x7f54d80693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54f40b36d0 tx=0x7f54f0006f90 comp rx=0 tx=0).stop 2023-10-06T05:15:45.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.083+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f4153620 msgr2=0x7f54f414e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.083+0000 7f54fb456700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f4153620 0x7f54f414e9a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f54e400b610 tx=0x7f54e40090f0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 shutdown_connections 2023-10-06T05:15:45.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54d8066f30 0x7f54d80693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f414f170 0x7f54f4158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f40ccfc0 0x7f54f416d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 --2- 172.21.15.45:0/137572879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f4153620 0x7f54f414e9a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 >> 172.21.15.45:0/137572879 conn(0x7f54f406ed50 msgr2=0x7f54f4152820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 shutdown_connections 2023-10-06T05:15:45.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.084+0000 7f54fb456700 1 -- 172.21.15.45:0/137572879 wait complete. 2023-10-06T05:15:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:57.253547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:58.338382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T05:15:01.988547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:45.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.235+0000 7f79959c0700 1 Processor -- start 2023-10-06T05:15:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.235+0000 7f79959c0700 1 -- start start 2023-10-06T05:15:45.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f79959c0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f79900d02b0 0x7f79900d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f79959c0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f799015d820 0x7f799015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f79959c0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79900cd740 con 0x7f799015c6f0 2023-10-06T05:15:45.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f79959c0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79900ccb30 con 0x7f79900d02b0 2023-10-06T05:15:45.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f79959c0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79900cce10 con 0x7f799015d820 2023-10-06T05:15:45.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f798e7fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f799015d820 0x7f799015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f798f7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f799015c6f0 0x7f79900afd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:35740/0 (socket says 172.21.15.45:35740) 2023-10-06T05:15:45.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f798effd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f79900d02b0 0x7f79900d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.236+0000 7f798f7fe700 1 -- 172.21.15.45:0/1249124563 learned_addr learned my addr 172.21.15.45:0/1249124563 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:45.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798dffb700 1 -- 172.21.15.45:0/1249124563 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14745127 0 0) 0x7f79900cd740 con 0x7f799015c6f0 2023-10-06T05:15:45.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798dffb700 1 -- 172.21.15.45:0/1249124563 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7978003570 con 0x7f799015c6f0 2023-10-06T05:15:45.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798effd700 1 -- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f799015d820 msgr2=0x7f799015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:45.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798effd700 1 --2- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f799015d820 0x7f799015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798effd700 1 -- 172.21.15.45:0/1249124563 >> v1:172.21.15.144:6789/0 conn(0x7f799015c6f0 legacy=0x7f79900afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798effd700 1 -- 172.21.15.45:0/1249124563 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900cd0f0 con 0x7f79900d02b0 2023-10-06T05:15:45.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798e7fc700 1 --2- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f799015d820 0x7f799015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:45.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798effd700 1 --2- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79900d02b0 0x7f79900d0680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7980009910 tx=0x7f79800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5217f83bff2d344 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798dffb700 1 -- 172.21.15.45:0/1249124563 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980011630 con 0x7f79900d02b0 2023-10-06T05:15:45.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798dffb700 1 -- 172.21.15.45:0/1249124563 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7980011790 con 0x7f79900d02b0 2023-10-06T05:15:45.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.237+0000 7f798dffb700 1 -- 172.21.15.45:0/1249124563 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980011b30 con 0x7f79900d02b0 2023-10-06T05:15:45.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 -- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79900d02b0 msgr2=0x7f79900d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 --2- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79900d02b0 0x7f79900d0680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7980009910 tx=0x7f79800091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 -- 172.21.15.45:0/1249124563 shutdown_connections 2023-10-06T05:15:45.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 --2- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79900d02b0 0x7f79900d0680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 --2- 172.21.15.45:0/1249124563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f799015d820 0x7f799015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 -- 172.21.15.45:0/1249124563 >> 172.21.15.45:0/1249124563 conn(0x7f799006ed50 msgr2=0x7f799006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 -- 172.21.15.45:0/1249124563 shutdown_connections 2023-10-06T05:15:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.238+0000 7f79959c0700 1 -- 172.21.15.45:0/1249124563 wait complete. 2023-10-06T05:15:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.239+0000 7f79959c0700 1 Processor -- start 2023-10-06T05:15:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.239+0000 7f79959c0700 1 -- start start 2023-10-06T05:15:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f79959c0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79900d02b0 0x7f79900c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f79959c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f799015c6f0 0x7f799016c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f79959c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799015d820 0x7f79900b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f79959c0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7990150f60 con 0x7f799015d820 2023-10-06T05:15:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f79959c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7990169970 con 0x7f799015c6f0 2023-10-06T05:15:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f79959c0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7990169c50 con 0x7f79900d02b0 2023-10-06T05:15:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f798f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799015d820 0x7f79900b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f798f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799015d820 0x7f79900b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52122/0 (socket says 172.21.15.45:52122) 2023-10-06T05:15:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f798f7fe700 1 -- 172.21.15.45:0/234759636 learned_addr learned my addr 172.21.15.45:0/234759636 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f798effd700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79900d02b0 0x7f79900c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.240+0000 7f798e7fc700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f799015c6f0 0x7f799016c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798f7fe700 1 -- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79900d02b0 msgr2=0x7f79900c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798f7fe700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79900d02b0 0x7f79900c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798f7fe700 1 -- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f799015c6f0 msgr2=0x7f799016c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798f7fe700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f799015c6f0 0x7f799016c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798f7fe700 1 -- 172.21.15.45:0/234759636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7984009420 con 0x7f799015d820 2023-10-06T05:15:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798effd700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79900d02b0 0x7f79900c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798e7fc700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f799015c6f0 0x7f799016c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f798f7fe700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799015d820 0x7f79900b46e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7988001460 tx=0x7f7988001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f7976ffd700 1 -- 172.21.15.45:0/234759636 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798800ce80 con 0x7f799015d820 2023-10-06T05:15:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f7976ffd700 1 -- 172.21.15.45:0/234759636 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798800ab70 con 0x7f799015d820 2023-10-06T05:15:45.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7980009420 con 0x7f799015d820 2023-10-06T05:15:45.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f7976ffd700 1 -- 172.21.15.45:0/234759636 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79880070f0 con 0x7f799015d820 2023-10-06T05:15:45.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.241+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799016a210 con 0x7f799015d820 2023-10-06T05:15:45.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.242+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799016a430 con 0x7f799015d820 2023-10-06T05:15:45.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.242+0000 7f7976ffd700 1 -- 172.21.15.45:0/234759636 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7988007520 con 0x7f799015d820 2023-10-06T05:15:45.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.243+0000 7f7976ffd700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f797c066f30 0x7f797c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.243+0000 7f7976ffd700 1 -- 172.21.15.45:0/234759636 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f7988084ea0 con 0x7f799015d820 2023-10-06T05:15:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.243+0000 7f798effd700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f797c066f30 0x7f797c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.244+0000 7f798effd700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f797c066f30 0x7f797c0693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f79800091e0 tx=0x7f7980017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.247+0000 7f7976ffd700 1 -- 172.21.15.45:0/234759636 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7988051cb0 con 0x7f799015d820 2023-10-06T05:15:45.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.451+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79900cd3d0 con 0x7f799015d820 2023-10-06T05:15:45.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.451+0000 7f7976ffd700 1 -- 172.21.15.45:0/234759636 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79880552d0 con 0x7f799015d820 2023-10-06T05:15:45.453 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f797c066f30 msgr2=0x7f797c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f797c066f30 0x7f797c0693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f79800091e0 tx=0x7f7980017000 comp rx=0 tx=0).stop 2023-10-06T05:15:45.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799015d820 msgr2=0x7f79900b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799015d820 0x7f79900b46e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7988001460 tx=0x7f7988001ae0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 shutdown_connections 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f797c066f30 0x7f797c0693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799015d820 0x7f79900b46e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79900d02b0 0x7f79900c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 --2- 172.21.15.45:0/234759636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f799015c6f0 0x7f799016c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.453+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 >> 172.21.15.45:0/234759636 conn(0x7f799006ed50 msgr2=0x7f799015cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.454+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 shutdown_connections 2023-10-06T05:15:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.454+0000 7f79959c0700 1 -- 172.21.15.45:0/234759636 wait complete. 2023-10-06T05:15:45.469 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:45.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.601+0000 7fcb9a7b0700 1 Processor -- start 2023-10-06T05:15:45.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.601+0000 7fcb9a7b0700 1 -- start start 2023-10-06T05:15:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.602+0000 7fcb9a7b0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb9406a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.602+0000 7fcb9a7b0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb94159ea0 0x7fcb9406ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb9a7b0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb94159a10 con 0x7fcb9406e720 2023-10-06T05:15:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb9a7b0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb940ac2e0 con 0x7fcb94159ea0 2023-10-06T05:15:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb9a7b0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9406f410 con 0x7fcb94158a60 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb937fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb94159ea0 0x7fcb9406ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb98d4d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcb9406e720 0x7fcb9406eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46324/0 (socket says 172.21.15.45:46324) 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb98d4d700 1 -- 172.21.15.45:0/593375152 learned_addr learned my addr 172.21.15.45:0/593375152 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb93fff700 1 --2- 172.21.15.45:0/593375152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb9406a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb937fe700 1 -- 172.21.15.45:0/593375152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 msgr2=0x7fcb9406a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb937fe700 1 --2- 172.21.15.45:0/593375152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb9406a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb937fe700 1 -- 172.21.15.45:0/593375152 >> v1:172.21.15.45:6789/0 conn(0x7fcb9406e720 legacy=0x7fcb9406eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb92ffd700 1 -- 172.21.15.45:0/593375152 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004968728 0 0) 0x7fcb94159a10 con 0x7fcb9406e720 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb937fe700 1 -- 172.21.15.45:0/593375152 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb941593c0 con 0x7fcb94159ea0 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb937fe700 1 --2- 172.21.15.45:0/593375152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb94159ea0 0x7fcb9406ad60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00aa90 tx=0x7fcb7c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce47217c06daf6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.603+0000 7fcb92ffd700 1 -- 172.21.15.45:0/593375152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00c970 con 0x7fcb94159ea0 2023-10-06T05:15:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb92ffd700 1 -- 172.21.15.45:0/593375152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00cad0 con 0x7fcb94159ea0 2023-10-06T05:15:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb92ffd700 1 -- 172.21.15.45:0/593375152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00ce70 con 0x7fcb94159ea0 2023-10-06T05:15:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/593375152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb94159ea0 msgr2=0x7fcb9406ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/593375152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb94159ea0 0x7fcb9406ad60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00aa90 tx=0x7fcb7c00a330 comp rx=0 tx=0).stop 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/593375152 shutdown_connections 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/593375152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb9406a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/593375152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb94159ea0 0x7fcb9406ad60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/593375152 >> 172.21.15.45:0/593375152 conn(0x7fcb940cc570 msgr2=0x7fcb940cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/593375152 shutdown_connections 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.604+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/593375152 wait complete. 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.605+0000 7fcb9a7b0700 1 Processor -- start 2023-10-06T05:15:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb9a7b0700 1 -- start start 2023-10-06T05:15:45.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb9a7b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb9406e720 0x7fcb94142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb9a7b0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb94168990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb9a7b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb94159ea0 0x7fcb94168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb9a7b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940b1000 con 0x7fcb9406e720 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb9a7b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb941665f0 con 0x7fcb94159ea0 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb9a7b0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940b0a40 con 0x7fcb94158a60 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.606+0000 7fcb93fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb9406e720 0x7fcb94142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb9406e720 0x7fcb94142090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52152/0 (socket says 172.21.15.45:52152) 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 -- 172.21.15.45:0/758992867 learned_addr learned my addr 172.21.15.45:0/758992867 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb937fe700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb94168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb98d4d700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb94159ea0 0x7fcb94168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 -- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 msgr2=0x7fcb94168990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb94168990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 -- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb94159ea0 msgr2=0x7fcb94168ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb94159ea0 0x7fcb94168ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 -- 172.21.15.45:0/758992867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c00a5d0 con 0x7fcb9406e720 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb98d4d700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb94159ea0 0x7fcb94168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb93fff700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb9406e720 0x7fcb94142090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ec80 tx=0x7fcb840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb8bfff700 1 -- 172.21.15.45:0/758992867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400c850 con 0x7fcb9406e720 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb8bfff700 1 -- 172.21.15.45:0/758992867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8400c9b0 con 0x7fcb9406e720 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb940b0d20 con 0x7fcb9406e720 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb8bfff700 1 -- 172.21.15.45:0/758992867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400cd50 con 0x7fcb9406e720 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.607+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb941425d0 con 0x7fcb9406e720 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.608+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9406e300 con 0x7fcb9406e720 2023-10-06T05:15:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.609+0000 7fcb8bfff700 1 -- 172.21.15.45:0/758992867 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb84013680 con 0x7fcb9406e720 2023-10-06T05:15:45.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.609+0000 7fcb8bfff700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb74066f30 0x7fcb740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.609+0000 7fcb8bfff700 1 -- 172.21.15.45:0/758992867 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcb84084c40 con 0x7fcb9406e720 2023-10-06T05:15:45.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.609+0000 7fcb937fe700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb74066f30 0x7fcb740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.610+0000 7fcb937fe700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb74066f30 0x7fcb740693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00a5a0 tx=0x7fcb7c01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.614+0000 7fcb8bfff700 1 -- 172.21.15.45:0/758992867 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb84051a60 con 0x7fcb9406e720 2023-10-06T05:15:45.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.813+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcb9404bb40 con 0x7fcb9406e720 2023-10-06T05:15:45.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.813+0000 7fcb8bfff700 1 -- 172.21.15.45:0/758992867 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcb84055080 con 0x7fcb9406e720 2023-10-06T05:15:45.815 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb74066f30 msgr2=0x7fcb740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb74066f30 0x7fcb740693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00a5a0 tx=0x7fcb7c01a000 comp rx=0 tx=0).stop 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb9406e720 msgr2=0x7fcb94142090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb9406e720 0x7fcb94142090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ec80 tx=0x7fcb840091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 shutdown_connections 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb74066f30 0x7fcb740693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb94159ea0 0x7fcb94168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb9406e720 0x7fcb94142090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 --2- 172.21.15.45:0/758992867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb94158a60 0x7fcb94168990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 >> 172.21.15.45:0/758992867 conn(0x7fcb940cc570 msgr2=0x7fcb940c8060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 shutdown_connections 2023-10-06T05:15:45.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.816+0000 7fcb9a7b0700 1 -- 172.21.15.45:0/758992867 wait complete. 2023-10-06T05:15:45.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.965+0000 7f2a70d37700 1 Processor -- start 2023-10-06T05:15:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.965+0000 7f2a70d37700 1 -- start start 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.966+0000 7f2a70d37700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 0x7f2a6c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.966+0000 7f2a70d37700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c0c9940 con 0x7f2a6c06cd60 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.966+0000 7f2a70d37700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c06d230 con 0x7f2a6c140ab0 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.966+0000 7f2a70d37700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c06d510 con 0x7f2a6c1406e0 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a69d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2a6c06cd60 0x7f2a6c0acfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46338/0 (socket says 172.21.15.45:46338) 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a69d9b700 1 -- 172.21.15.45:0/2659954608 learned_addr learned my addr 172.21.15.45:0/2659954608 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 0x7f2a6c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145941594 0 0) 0x7f2a6c0c9940 con 0x7f2a6c06cd60 2023-10-06T05:15:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a54003570 con 0x7f2a6c06cd60 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 770047906 0 0) 0x7f2a54003570 con 0x7f2a6c06cd60 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 msgr2=0x7f2a6c06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 0x7f2a6c06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 >> v1:172.21.15.144:6789/0 conn(0x7f2a6c140ab0 legacy=0x7f2a6c0c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c06d7f0 con 0x7f2a6c06cd60 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f2a60001470 con 0x7f2a6c06cd60 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 >> v1:172.21.15.45:6789/0 conn(0x7f2a6c06cd60 legacy=0x7f2a6c0acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a54003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 0x7f2a54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a54004d50 0x7f2a540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c06d510 con 0x7f2a54003a90 2023-10-06T05:15:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5400bb70 con 0x7f2a54004d50 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5400bd30 con 0x7f2a6c1406e0 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.967+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a54003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a69d9b700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 0x7f2a54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6ad9d700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a54004d50 0x7f2a540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6a59c700 1 -- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 msgr2=0x7f2a54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c1406e0 0x7f2a54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6a59c700 1 -- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a54004d50 msgr2=0x7f2a540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a54004d50 0x7f2a540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6a59c700 1 -- 172.21.15.45:0/2659954608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54003570 con 0x7f2a54003a90 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a54003ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009c00 tx=0x7f2a5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbe132012bf60d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00e710 con 0x7f2a54003a90 2023-10-06T05:15:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c00e870 con 0x7f2a54003a90 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.968+0000 7f2a6959a700 1 -- 172.21.15.45:0/2659954608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00ec30 con 0x7f2a54003a90 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.969+0000 7f2a70d37700 1 -- 172.21.15.45:0/2659954608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 msgr2=0x7f2a54003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.969+0000 7f2a70d37700 1 --2- 172.21.15.45:0/2659954608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a54003ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009c00 tx=0x7f2a5c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.969+0000 7f2a6ad9d700 1 -- 172.21.15.45:0/2659954608 reap_dead start 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.969+0000 7f2a70d37700 1 -- 172.21.15.45:0/2659954608 shutdown_connections 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.969+0000 7f2a70d37700 1 -- 172.21.15.45:0/2659954608 >> 172.21.15.45:0/2659954608 conn(0x7f2a6c0691c0 msgr2=0x7f2a6c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.969+0000 7f2a70d37700 1 -- 172.21.15.45:0/2659954608 shutdown_connections 2023-10-06T05:15:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.969+0000 7f2a70d37700 1 -- 172.21.15.45:0/2659954608 wait complete. 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.970+0000 7f2a70d37700 1 Processor -- start 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.970+0000 7f2a70d37700 1 -- start start 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a70d37700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a6c130670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a70d37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a6c15ab20 0x7f2a6c0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a70d37700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c158540 0x7f2a6c0acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a70d37700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c15bcf0 con 0x7f2a54003a90 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a70d37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c130bb0 con 0x7f2a6c15ab20 2023-10-06T05:15:45.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a70d37700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c15b0e0 con 0x7f2a6c158540 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a6c130670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a6c130670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52170/0 (socket says 172.21.15.45:52170) 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 -- 172.21.15.45:0/671659995 learned_addr learned my addr 172.21.15.45:0/671659995 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a69d9b700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a6c15ab20 0x7f2a6c0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6ad9d700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c158540 0x7f2a6c0acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 -- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c158540 msgr2=0x7f2a6c0acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c158540 0x7f2a6c0acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 -- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a6c15ab20 msgr2=0x7f2a6c0c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a6c15ab20 0x7f2a6c0c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:45.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a6a59c700 1 -- 172.21.15.45:0/671659995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c15b3c0 con 0x7f2a54003a90 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.971+0000 7f2a69d9b700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a6c15ab20 0x7f2a6c0c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.972+0000 7f2a6a59c700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a6c130670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c013ef0 tx=0x7f2a5c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.972+0000 7f2a5a7fc700 1 -- 172.21.15.45:0/671659995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c022070 con 0x7f2a54003a90 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.972+0000 7f2a5a7fc700 1 -- 172.21.15.45:0/671659995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c00e9e0 con 0x7f2a54003a90 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.972+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c15b6a0 con 0x7f2a54003a90 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.972+0000 7f2a5a7fc700 1 -- 172.21.15.45:0/671659995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c027470 con 0x7f2a54003a90 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.972+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c0c4430 con 0x7f2a54003a90 2023-10-06T05:15:45.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.973+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c06e2d0 con 0x7f2a54003a90 2023-10-06T05:15:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.973+0000 7f2a5a7fc700 1 -- 172.21.15.45:0/671659995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a5c01c450 con 0x7f2a54003a90 2023-10-06T05:15:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.974+0000 7f2a5a7fc700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a4c066f30 0x7f2a4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.974+0000 7f2a5a7fc700 1 -- 172.21.15.45:0/671659995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f2a5c0865c0 con 0x7f2a54003a90 2023-10-06T05:15:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.974+0000 7f2a69d9b700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a4c066f30 0x7f2a4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.975+0000 7f2a69d9b700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a4c066f30 0x7f2a4c0693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a600010e0 tx=0x7f2a60009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:45.978+0000 7f2a5a7fc700 1 -- 172.21.15.45:0/671659995 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a5c053340 con 0x7f2a54003a90 2023-10-06T05:15:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.176+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a6c15bfd0 con 0x7f2a54003a90 2023-10-06T05:15:46.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.177+0000 7f2a5a7fc700 1 -- 172.21.15.45:0/671659995 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a5c056960 con 0x7f2a54003a90 2023-10-06T05:15:46.179 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a4c066f30 msgr2=0x7f2a4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a4c066f30 0x7f2a4c0693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a600010e0 tx=0x7f2a60009040 comp rx=0 tx=0).stop 2023-10-06T05:15:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 msgr2=0x7f2a6c130670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a6c130670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c013ef0 tx=0x7f2a5c01a000 comp rx=0 tx=0).stop 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 shutdown_connections 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a4c066f30 0x7f2a4c0693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a6c158540 0x7f2a6c0acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54003a90 0x7f2a6c130670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 --2- 172.21.15.45:0/671659995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a6c15ab20 0x7f2a6c0c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 >> 172.21.15.45:0/671659995 conn(0x7f2a6c0691c0 msgr2=0x7f2a6c0ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 shutdown_connections 2023-10-06T05:15:46.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.179+0000 7f2a70d37700 1 -- 172.21.15.45:0/671659995 wait complete. 2023-10-06T05:15:46.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:46.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.328+0000 7f772ce07700 1 Processor -- start 2023-10-06T05:15:46.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.328+0000 7f772ce07700 1 -- start start 2023-10-06T05:15:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.329+0000 7f772ce07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f77280c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.329+0000 7f772ce07700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f772815c8c0 0x7f77280aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.329+0000 7f772ce07700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7728067d80 con 0x7f77280c7050 2023-10-06T05:15:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.329+0000 7f772ce07700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77280ac620 con 0x7f772815c8c0 2023-10-06T05:15:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.329+0000 7f772ce07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77280c75b0 con 0x7f772806aa70 2023-10-06T05:15:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.329+0000 7f7726d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f772815c8c0 0x7f77280aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.329+0000 7f7725d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f77280c7050 0x7f772815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46344/0 (socket says 172.21.15.45:46344) 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f7725d9b700 1 -- 172.21.15.45:0/259055666 learned_addr learned my addr 172.21.15.45:0/259055666 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f772659c700 1 --2- 172.21.15.45:0/259055666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f77280c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f7726d9d700 1 -- 172.21.15.45:0/259055666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 msgr2=0x7f77280c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f7726d9d700 1 --2- 172.21.15.45:0/259055666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f77280c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f7726d9d700 1 -- 172.21.15.45:0/259055666 >> v1:172.21.15.45:6789/0 conn(0x7f77280c7050 legacy=0x7f772815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f7726d9d700 1 -- 172.21.15.45:0/259055666 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280c7890 con 0x7f772815c8c0 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f772559a700 1 -- 172.21.15.45:0/259055666 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921639385 0 0) 0x7f7728067d80 con 0x7f77280c7050 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f7726d9d700 1 --2- 172.21.15.45:0/259055666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f772815c8c0 0x7f77280aca70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7720009e60 tx=0x7f7720009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263971c7f1d4e652 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f772559a700 1 -- 172.21.15.45:0/259055666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000e070 con 0x7f772815c8c0 2023-10-06T05:15:46.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f772559a700 1 -- 172.21.15.45:0/259055666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000bc30 con 0x7f772815c8c0 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.330+0000 7f772559a700 1 -- 172.21.15.45:0/259055666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720012430 con 0x7f772815c8c0 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 -- 172.21.15.45:0/259055666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f772815c8c0 msgr2=0x7f77280aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 --2- 172.21.15.45:0/259055666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f772815c8c0 0x7f77280aca70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7720009e60 tx=0x7f7720009780 comp rx=0 tx=0).stop 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 -- 172.21.15.45:0/259055666 shutdown_connections 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 --2- 172.21.15.45:0/259055666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f772815c8c0 0x7f77280aca70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 --2- 172.21.15.45:0/259055666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f77280c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 -- 172.21.15.45:0/259055666 >> 172.21.15.45:0/259055666 conn(0x7f77280cc570 msgr2=0x7f77280cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 -- 172.21.15.45:0/259055666 shutdown_connections 2023-10-06T05:15:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.331+0000 7f772ce07700 1 -- 172.21.15.45:0/259055666 wait complete. 2023-10-06T05:15:46.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.332+0000 7f772ce07700 1 Processor -- start 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.332+0000 7f772ce07700 1 -- start start 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f772ce07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f7728156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f772ce07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77280c7050 0x7f77280c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f772ce07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f772815c8c0 0x7f77280c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f772ce07700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77280c4d00 con 0x7f77280c7050 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f772ce07700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77280b15c0 con 0x7f772815c8c0 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f772ce07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77280c40f0 con 0x7f772806aa70 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77280c7050 0x7f77280c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77280c7050 0x7f77280c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52186/0 (socket says 172.21.15.45:52186) 2023-10-06T05:15:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 -- 172.21.15.45:0/373514024 learned_addr learned my addr 172.21.15.45:0/373514024 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f772659c700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f7728156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7726d9d700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f772815c8c0 0x7f77280c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 -- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 msgr2=0x7f7728156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f7728156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 -- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f772815c8c0 msgr2=0x7f77280c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f772815c8c0 0x7f77280c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.333+0000 7f7725d9b700 1 -- 172.21.15.45:0/373514024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7720009a20 con 0x7f77280c7050 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.334+0000 7f7725d9b700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77280c7050 0x7f77280c3580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c7e0 tx=0x7f771c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.334+0000 7f77167fc700 1 -- 172.21.15.45:0/373514024 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c005b30 con 0x7f77280c7050 2023-10-06T05:15:46.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.334+0000 7f77167fc700 1 -- 172.21.15.45:0/373514024 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c005c90 con 0x7f77280c7050 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.334+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77280c43d0 con 0x7f77280c7050 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.334+0000 7f77167fc700 1 -- 172.21.15.45:0/373514024 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c007300 con 0x7f77280c7050 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.334+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77280c4990 con 0x7f77280c7050 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.335+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772806e300 con 0x7f77280c7050 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.335+0000 7f77167fc700 1 -- 172.21.15.45:0/373514024 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f771c01b3b0 con 0x7f77280c7050 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.336+0000 7f77167fc700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7708066f30 0x7f77080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.336+0000 7f77167fc700 1 -- 172.21.15.45:0/373514024 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f771c083ea0 con 0x7f77280c7050 2023-10-06T05:15:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.336+0000 7f772659c700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7708066f30 0x7f77080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.337+0000 7f772659c700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7708066f30 0x7f77080693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7718006fd0 tx=0x7f7718008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.341+0000 7f77167fc700 1 -- 172.21.15.45:0/373514024 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f771c050cc0 con 0x7f77280c7050 2023-10-06T05:15:46.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.544+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77280c7b70 con 0x7f77280c7050 2023-10-06T05:15:46.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.545+0000 7f77167fc700 1 -- 172.21.15.45:0/373514024 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f771c017020 con 0x7f77280c7050 2023-10-06T05:15:46.546 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:15:46.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7708066f30 msgr2=0x7f77080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7708066f30 0x7f77080693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7718006fd0 tx=0x7f7718008040 comp rx=0 tx=0).stop 2023-10-06T05:15:46.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77280c7050 msgr2=0x7f77280c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77280c7050 0x7f77280c3580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c7e0 tx=0x7f771c007df0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 shutdown_connections 2023-10-06T05:15:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7708066f30 0x7f77080693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f772815c8c0 0x7f77280c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f772806aa70 0x7f7728156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 --2- 172.21.15.45:0/373514024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77280c7050 0x7f77280c3580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 >> 172.21.15.45:0/373514024 conn(0x7f77280cc570 msgr2=0x7f7728070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 shutdown_connections 2023-10-06T05:15:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.547+0000 7f772ce07700 1 -- 172.21.15.45:0/373514024 wait complete. 2023-10-06T05:15:46.562 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T05:15:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.694+0000 7f9bfba79700 1 Processor -- start 2023-10-06T05:15:46.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.694+0000 7f9bfba79700 1 -- start start 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfba79700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf4147210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfba79700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9bf40cb650 0x7f9bf4067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfba79700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9bf40cbac0 0x7f9bf4067b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfba79700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40c8770 con 0x7f9bf40cbac0 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfba79700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40cc030 con 0x7f9bf40cb650 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfba79700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40c8120 con 0x7f9bf4146e40 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfa016700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9bf40cbac0 0x7f9bf4067b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfa016700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9bf40cbac0 0x7f9bf4067b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37724/0 (socket says 172.21.15.45:37724) 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.695+0000 7f9bfa016700 1 -- 172.21.15.45:0/4134604363 learned_addr learned my addr 172.21.15.45:0/4134604363 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bf9014700 1 --2- 172.21.15.45:0/4134604363 >> v2:172.21.15.144:3300/0 conn(0x7f9bf40cb650 0x7f9bf4067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bf9815700 1 --2- 172.21.15.45:0/4134604363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf4147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bfa016700 1 -- 172.21.15.45:0/4134604363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 msgr2=0x7f9bf4147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bfa016700 1 --2- 172.21.15.45:0/4134604363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf4147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bfa016700 1 -- 172.21.15.45:0/4134604363 >> v2:172.21.15.144:3300/0 conn(0x7f9bf40cb650 msgr2=0x7f9bf4067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bfa016700 1 --2- 172.21.15.45:0/4134604363 >> v2:172.21.15.144:3300/0 conn(0x7f9bf40cb650 0x7f9bf4067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bfa016700 1 -- 172.21.15.45:0/4134604363 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40c8400 con 0x7f9bf40cbac0 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bf9014700 1 --2- 172.21.15.45:0/4134604363 >> v2:172.21.15.144:3300/0 conn(0x7f9bf40cb650 0x7f9bf4067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bfa016700 1 --2- 172.21.15.45:0/4134604363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cbac0 0x7f9bf4067b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9bf406eaa0 tx=0x7f9bec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7578f404d9d6d930 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bf8813700 1 -- 172.21.15.45:0/4134604363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00e070 con 0x7f9bf40cbac0 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bf8813700 1 -- 172.21.15.45:0/4134604363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec00ba50 con 0x7f9bf40cbac0 2023-10-06T05:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.696+0000 7f9bf8813700 1 -- 172.21.15.45:0/4134604363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0092e0 con 0x7f9bf40cbac0 2023-10-06T05:15:46.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 -- 172.21.15.45:0/4134604363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cbac0 msgr2=0x7f9bf4067b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 --2- 172.21.15.45:0/4134604363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cbac0 0x7f9bf4067b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9bf406eaa0 tx=0x7f9bec009840 comp rx=0 tx=0).stop 2023-10-06T05:15:46.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 -- 172.21.15.45:0/4134604363 shutdown_connections 2023-10-06T05:15:46.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 --2- 172.21.15.45:0/4134604363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cbac0 0x7f9bf4067b60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 --2- 172.21.15.45:0/4134604363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf4147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 --2- 172.21.15.45:0/4134604363 >> v2:172.21.15.144:3300/0 conn(0x7f9bf40cb650 0x7f9bf4067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 -- 172.21.15.45:0/4134604363 >> 172.21.15.45:0/4134604363 conn(0x7f9bf40691c0 msgr2=0x7f9bf40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 -- 172.21.15.45:0/4134604363 shutdown_connections 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.697+0000 7f9bfba79700 1 -- 172.21.15.45:0/4134604363 wait complete. 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.698+0000 7f9bfba79700 1 Processor -- start 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.698+0000 7f9bfba79700 1 -- start start 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bfba79700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cb650 0x7f9bf40af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bfba79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf40cbac0 0x7f9bf416d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bfba79700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf40c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bfba79700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40c4260 con 0x7f9bf40cb650 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bfba79700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40af7c0 con 0x7f9bf40cbac0 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bfba79700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf416e070 con 0x7f9bf4146e40 2023-10-06T05:15:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cb650 0x7f9bf40af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cb650 0x7f9bf40af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37740/0 (socket says 172.21.15.45:37740) 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 -- 172.21.15.45:0/653664275 learned_addr learned my addr 172.21.15.45:0/653664275 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bfa016700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf40c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9014700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf40cbac0 0x7f9bf416d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 -- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 msgr2=0x7f9bf40c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf40c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 -- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf40cbac0 msgr2=0x7f9bf416d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf40cbac0 0x7f9bf416d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9815700 1 -- 172.21.15.45:0/653664275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009ae0 con 0x7f9bf40cb650 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.699+0000 7f9bf9014700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf40cbac0 0x7f9bf416d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.700+0000 7f9bf9815700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cb650 0x7f9bf40af280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000d860 tx=0x7f9bf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.700+0000 7f9be5ffb700 1 -- 172.21.15.45:0/653664275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00125f0 con 0x7f9bf40cb650 2023-10-06T05:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.700+0000 7f9be5ffb700 1 -- 172.21.15.45:0/653664275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0012750 con 0x7f9bf40cb650 2023-10-06T05:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.700+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf416e350 con 0x7f9bf40cb650 2023-10-06T05:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.700+0000 7f9be5ffb700 1 -- 172.21.15.45:0/653664275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0012b10 con 0x7f9bf40cb650 2023-10-06T05:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.700+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf40c3ef0 con 0x7f9bf40cb650 2023-10-06T05:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.701+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf406e2d0 con 0x7f9bf40cb650 2023-10-06T05:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.701+0000 7f9be5ffb700 1 -- 172.21.15.45:0/653664275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bf001e460 con 0x7f9bf40cb650 2023-10-06T05:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.702+0000 7f9be5ffb700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9bd8066f30 0x7f9bd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.702+0000 7f9be5ffb700 1 -- 172.21.15.45:0/653664275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9bf0084f20 con 0x7f9bf40cb650 2023-10-06T05:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.702+0000 7f9bf9014700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9bd8066f30 0x7f9bd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.702+0000 7f9bf9014700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9bd8066f30 0x7f9bd80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006f00 tx=0x7f9be8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.706+0000 7f9be5ffb700 1 -- 172.21.15.45:0/653664275 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bf0051d40 con 0x7f9bf40cb650 2023-10-06T05:15:46.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.911+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9bf41619c0 con 0x7f9bf40cb650 2023-10-06T05:15:46.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.912+0000 7f9be5ffb700 1 -- 172.21.15.45:0/653664275 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9bf0055360 con 0x7f9bf40cb650 2023-10-06T05:15:46.914 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9bd8066f30 msgr2=0x7f9bd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9bd8066f30 0x7f9bd80693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006f00 tx=0x7f9be8008040 comp rx=0 tx=0).stop 2023-10-06T05:15:46.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cb650 msgr2=0x7f9bf40af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cb650 0x7f9bf40af280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000d860 tx=0x7f9bf0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 shutdown_connections 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9bd8066f30 0x7f9bd80693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bf4146e40 0x7f9bf40c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bf40cb650 0x7f9bf40af280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 --2- 172.21.15.45:0/653664275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf40cbac0 0x7f9bf416d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 >> 172.21.15.45:0/653664275 conn(0x7f9bf40691c0 msgr2=0x7f9bf40c2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 shutdown_connections 2023-10-06T05:15:46.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:46.914+0000 7f9bfba79700 1 -- 172.21.15.45:0/653664275 wait complete. 2023-10-06T05:15:46.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.063+0000 7ff11586f700 1 Processor -- start 2023-10-06T05:15:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.063+0000 7ff11586f700 1 -- start start 2023-10-06T05:15:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff11586f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff110159170 0x7ff110159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff11586f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff11015d040 0x7ff110068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff11586f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1100ca6d0 con 0x7ff110069b90 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff11586f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11015d550 con 0x7ff11015d040 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff11586f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11015d810 con 0x7ff110159170 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff10e7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff11015d040 0x7ff110068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff10f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff110069b90 0x7ff1100aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49672/0 (socket says 172.21.15.45:49672) 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff10f7fe700 1 -- 172.21.15.45:0/3715874195 learned_addr learned my addr 172.21.15.45:0/3715874195 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.064+0000 7ff10effd700 1 --2- 172.21.15.45:0/3715874195 >> v2:172.21.15.144:3300/0 conn(0x7ff110159170 0x7ff110159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10e7fc700 1 -- 172.21.15.45:0/3715874195 >> v2:172.21.15.144:3300/0 conn(0x7ff110159170 msgr2=0x7ff110159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10e7fc700 1 --2- 172.21.15.45:0/3715874195 >> v2:172.21.15.144:3300/0 conn(0x7ff110159170 0x7ff110159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10e7fc700 1 -- 172.21.15.45:0/3715874195 >> v1:172.21.15.45:6789/0 conn(0x7ff110069b90 legacy=0x7ff1100aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10e7fc700 1 -- 172.21.15.45:0/3715874195 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11015daf0 con 0x7ff11015d040 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10dffb700 1 -- 172.21.15.45:0/3715874195 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922116033 0 0) 0x7ff1100ca6d0 con 0x7ff110069b90 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10e7fc700 1 --2- 172.21.15.45:0/3715874195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff110068bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff110068a90 tx=0x7ff1040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd9d34558167285 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10dffb700 1 -- 172.21.15.45:0/3715874195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104011630 con 0x7ff11015d040 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10dffb700 1 -- 172.21.15.45:0/3715874195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff104011790 con 0x7ff11015d040 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff10dffb700 1 -- 172.21.15.45:0/3715874195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104011b30 con 0x7ff11015d040 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.065+0000 7ff11586f700 1 -- 172.21.15.45:0/3715874195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 msgr2=0x7ff110068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.066+0000 7ff11586f700 1 --2- 172.21.15.45:0/3715874195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff110068bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff110068a90 tx=0x7ff1040091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.066+0000 7ff11586f700 1 -- 172.21.15.45:0/3715874195 shutdown_connections 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.066+0000 7ff11586f700 1 --2- 172.21.15.45:0/3715874195 >> v2:172.21.15.144:3300/0 conn(0x7ff110159170 0x7ff110159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.066+0000 7ff11586f700 1 --2- 172.21.15.45:0/3715874195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff110068bc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.066+0000 7ff11586f700 1 -- 172.21.15.45:0/3715874195 >> 172.21.15.45:0/3715874195 conn(0x7ff11006f7c0 msgr2=0x7ff1100d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.066+0000 7ff11586f700 1 -- 172.21.15.45:0/3715874195 shutdown_connections 2023-10-06T05:15:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.066+0000 7ff11586f700 1 -- 172.21.15.45:0/3715874195 wait complete. 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.067+0000 7ff11586f700 1 Processor -- start 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.067+0000 7ff11586f700 1 -- start start 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff11586f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110069b90 0x7ff1101566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff11586f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff110159170 0x7ff1100ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff11586f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff1100b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff11586f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015c3c0 con 0x7ff11015d040 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff11586f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015b740 con 0x7ff110159170 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff11586f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015ba20 con 0x7ff110069b90 2023-10-06T05:15:47.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff1100b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff1100b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37746/0 (socket says 172.21.15.45:37746) 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 -- 172.21.15.45:0/128736223 learned_addr learned my addr 172.21.15.45:0/128736223 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10effd700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110069b90 0x7ff1101566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10e7fc700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff110159170 0x7ff1100ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 -- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110069b90 msgr2=0x7ff1101566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110069b90 0x7ff1101566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 -- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff110159170 msgr2=0x7ff1100ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff110159170 0x7ff1100ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10f7fe700 1 -- 172.21.15.45:0/128736223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff104009420 con 0x7ff11015d040 2023-10-06T05:15:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.068+0000 7ff10effd700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110069b90 0x7ff1101566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.069+0000 7ff10f7fe700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff1100b2110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff10800cc90 tx=0x7ff108001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.069+0000 7ff0f6ffd700 1 -- 172.21.15.45:0/128736223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10800cea0 con 0x7ff11015d040 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.069+0000 7ff0f6ffd700 1 -- 172.21.15.45:0/128736223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10800ab70 con 0x7ff11015d040 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.069+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11015bd00 con 0x7ff11015d040 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.069+0000 7ff0f6ffd700 1 -- 172.21.15.45:0/128736223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080070f0 con 0x7ff11015d040 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.069+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1100b3a70 con 0x7ff11015d040 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.070+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11015c500 con 0x7ff11015d040 2023-10-06T05:15:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.070+0000 7ff0f6ffd700 1 -- 172.21.15.45:0/128736223 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff108007460 con 0x7ff11015d040 2023-10-06T05:15:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.071+0000 7ff0f6ffd700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0fc066f30 0x7ff0fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.071+0000 7ff0f6ffd700 1 -- 172.21.15.45:0/128736223 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6867+0+0 (secure 0 0 0) 0x7ff108084e00 con 0x7ff11015d040 2023-10-06T05:15:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.071+0000 7ff10effd700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0fc066f30 0x7ff0fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.072+0000 7ff10effd700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0fc066f30 0x7ff0fc0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff100009d80 tx=0x7ff100008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.075+0000 7ff0f6ffd700 1 -- 172.21.15.45:0/128736223 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff108051c60 con 0x7ff11015d040 2023-10-06T05:15:47.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.277+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff110063b50 con 0x7ff11015d040 2023-10-06T05:15:47.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.278+0000 7ff0f6ffd700 1 -- 172.21.15.45:0/128736223 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18899 (secure 0 0 0) 0x7ff108055280 con 0x7ff11015d040 2023-10-06T05:15:47.280 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:47.281 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:46.873705+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0fc066f30 msgr2=0x7ff0fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0fc066f30 0x7ff0fc0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff100009d80 tx=0x7ff100008040 comp rx=0 tx=0).stop 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 msgr2=0x7ff1100b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff1100b2110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff10800cc90 tx=0x7ff108001a80 comp rx=0 tx=0).stop 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 shutdown_connections 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0fc066f30 0x7ff0fc0693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff11015d040 0x7ff1100b2110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110069b90 0x7ff1101566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 --2- 172.21.15.45:0/128736223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff110159170 0x7ff1100ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 >> 172.21.15.45:0/128736223 conn(0x7ff11006f7c0 msgr2=0x7ff110158600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 shutdown_connections 2023-10-06T05:15:47.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.281+0000 7ff11586f700 1 -- 172.21.15.45:0/128736223 wait complete. 2023-10-06T05:15:47.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:57.253547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:58.338382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T05:15:01.988547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:47.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.430+0000 7f5eb8a85700 1 Processor -- start 2023-10-06T05:15:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.430+0000 7f5eb8a85700 1 -- start start 2023-10-06T05:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb8a85700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5eb413f900 0x7f5eb40cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb8a85700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5eb40c8de0 0x7f5eb40c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb8a85700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb40c9630 con 0x7f5eb40c9d50 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb8a85700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb40ca150 con 0x7f5eb413f900 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb8a85700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb40cb550 con 0x7f5eb40c8de0 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5eb413f900 0x7f5eb40cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5eb413f900 0x7f5eb40cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37758/0 (socket says 172.21.15.45:37758) 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb259c700 1 -- 172.21.15.45:0/3523976821 learned_addr learned my addr 172.21.15.45:0/3523976821 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb1d9b700 1 --2- 172.21.15.45:0/3523976821 >> v2:172.21.15.144:3300/0 conn(0x7f5eb40c8de0 0x7f5eb40c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb259c700 1 -- 172.21.15.45:0/3523976821 >> v2:172.21.15.144:3300/0 conn(0x7f5eb40c8de0 msgr2=0x7f5eb40c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb259c700 1 --2- 172.21.15.45:0/3523976821 >> v2:172.21.15.144:3300/0 conn(0x7f5eb40c8de0 0x7f5eb40c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.432+0000 7f5eb259c700 1 -- 172.21.15.45:0/3523976821 >> v1:172.21.15.144:6789/0 conn(0x7f5eb40c9d50 legacy=0x7f5eb406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb259c700 1 -- 172.21.15.45:0/3523976821 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40cb830 con 0x7f5eb413f900 2023-10-06T05:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb1d9b700 1 --2- 172.21.15.45:0/3523976821 >> v2:172.21.15.144:3300/0 conn(0x7f5eb40c8de0 0x7f5eb40c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb259c700 1 --2- 172.21.15.45:0/3523976821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb413f900 0x7f5eb40cc840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40cc620 tx=0x7f5ea800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6073e4632f99052 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb159a700 1 -- 172.21.15.45:0/3523976821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800dc50 con 0x7f5eb413f900 2023-10-06T05:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb159a700 1 -- 172.21.15.45:0/3523976821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800ddb0 con 0x7f5eb413f900 2023-10-06T05:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb159a700 1 -- 172.21.15.45:0/3523976821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80196d0 con 0x7f5eb413f900 2023-10-06T05:15:47.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb8a85700 1 -- 172.21.15.45:0/3523976821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb413f900 msgr2=0x7f5eb40cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.433+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/3523976821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb413f900 0x7f5eb40cc840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40cc620 tx=0x7f5ea800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.434+0000 7f5eb8a85700 1 -- 172.21.15.45:0/3523976821 shutdown_connections 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.434+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/3523976821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb413f900 0x7f5eb40cc840 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.434+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/3523976821 >> v2:172.21.15.144:3300/0 conn(0x7f5eb40c8de0 0x7f5eb40c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.434+0000 7f5eb8a85700 1 -- 172.21.15.45:0/3523976821 >> 172.21.15.45:0/3523976821 conn(0x7f5eb40691c0 msgr2=0x7f5eb40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.434+0000 7f5eb8a85700 1 -- 172.21.15.45:0/3523976821 shutdown_connections 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.434+0000 7f5eb8a85700 1 -- 172.21.15.45:0/3523976821 wait complete. 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.435+0000 7f5eb8a85700 1 Processor -- start 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.435+0000 7f5eb8a85700 1 -- start start 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eb40c8de0 0x7f5eb41594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb40c9d50 0x7f5eb413bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb413f900 0x7f5eb4156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4156c60 con 0x7f5eb40c9d50 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4155b50 con 0x7f5eb413f900 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4159ac0 con 0x7f5eb40c8de0 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb40c9d50 0x7f5eb413bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb40c9d50 0x7f5eb413bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37772/0 (socket says 172.21.15.45:37772) 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 -- 172.21.15.45:0/4236379583 learned_addr learned my addr 172.21.15.45:0/4236379583 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb259c700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eb40c8de0 0x7f5eb41594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb2d9d700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb413f900 0x7f5eb4156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 -- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eb40c8de0 msgr2=0x7f5eb41594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eb40c8de0 0x7f5eb41594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 -- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb413f900 msgr2=0x7f5eb4156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb413f900 0x7f5eb4156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 -- 172.21.15.45:0/4236379583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea800b790 con 0x7f5eb40c9d50 2023-10-06T05:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb2d9d700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb413f900 0x7f5eb4156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb1d9b700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb40c9d50 0x7f5eb413bb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e8d0 tx=0x7f5ea4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5ea27fc700 1 -- 172.21.15.45:0/4236379583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400ee70 con 0x7f5eb40c9d50 2023-10-06T05:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5ea27fc700 1 -- 172.21.15.45:0/4236379583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea400c670 con 0x7f5eb40c9d50 2023-10-06T05:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb4159da0 con 0x7f5eb40c9d50 2023-10-06T05:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5ea27fc700 1 -- 172.21.15.45:0/4236379583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400ca10 con 0x7f5eb40c9d50 2023-10-06T05:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.436+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb415a360 con 0x7f5eb40c9d50 2023-10-06T05:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.437+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb406e2d0 con 0x7f5eb40c9d50 2023-10-06T05:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.438+0000 7f5ea27fc700 1 -- 172.21.15.45:0/4236379583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ea4013410 con 0x7f5eb40c9d50 2023-10-06T05:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.438+0000 7f5ea27fc700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5e94066f30 0x7f5e940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.438+0000 7f5ea27fc700 1 -- 172.21.15.45:0/4236379583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f5ea4084840 con 0x7f5eb40c9d50 2023-10-06T05:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.439+0000 7f5eb259c700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5e94066f30 0x7f5e940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.439+0000 7f5eb259c700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5e94066f30 0x7f5e940693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40cc620 tx=0x7f5ea801b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.443+0000 7f5ea27fc700 1 -- 172.21.15.45:0/4236379583 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ea4051620 con 0x7f5eb40c9d50 2023-10-06T05:15:47.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.645+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5eb40cbb10 con 0x7f5eb40c9d50 2023-10-06T05:15:47.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.645+0000 7f5ea27fc700 1 -- 172.21.15.45:0/4236379583 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ea4054c40 con 0x7f5eb40c9d50 2023-10-06T05:15:47.648 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:15:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5e94066f30 msgr2=0x7f5e940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5e94066f30 0x7f5e940693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40cc620 tx=0x7f5ea801b000 comp rx=0 tx=0).stop 2023-10-06T05:15:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb40c9d50 msgr2=0x7f5eb413bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb40c9d50 0x7f5eb413bb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e8d0 tx=0x7f5ea4008040 comp rx=0 tx=0).stop 2023-10-06T05:15:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 shutdown_connections 2023-10-06T05:15:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5e94066f30 0x7f5e940693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb413f900 0x7f5eb4156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eb40c8de0 0x7f5eb41594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 --2- 172.21.15.45:0/4236379583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb40c9d50 0x7f5eb413bb80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 >> 172.21.15.45:0/4236379583 conn(0x7f5eb40691c0 msgr2=0x7f5eb413e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 shutdown_connections 2023-10-06T05:15:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.648+0000 7f5eb8a85700 1 -- 172.21.15.45:0/4236379583 wait complete. 2023-10-06T05:15:47.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.795+0000 7f10a13a0700 1 Processor -- start 2023-10-06T05:15:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.795+0000 7f10a13a0700 1 -- start start 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f10a13a0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f109c06dff0 0x7f109c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f10a13a0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f109c06a070 0x7f109c06a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f10a13a0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0ac230 con 0x7f109c157140 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f10a13a0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0aba00 con 0x7f109c06dff0 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f10a13a0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0acdd0 con 0x7f109c06a070 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f109a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f109c06dff0 0x7f109c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f109a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f109c06dff0 0x7f109c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37788/0 (socket says 172.21.15.45:37788) 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.796+0000 7f109a7fc700 1 -- 172.21.15.45:0/3998485141 learned_addr learned my addr 172.21.15.45:0/3998485141 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f109b7fe700 1 --2- 172.21.15.45:0/3998485141 >> v2:172.21.15.144:3300/0 conn(0x7f109c06a070 0x7f109c06a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f109a7fc700 1 -- 172.21.15.45:0/3998485141 >> v2:172.21.15.144:3300/0 conn(0x7f109c06a070 msgr2=0x7f109c06a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f109a7fc700 1 --2- 172.21.15.45:0/3998485141 >> v2:172.21.15.144:3300/0 conn(0x7f109c06a070 0x7f109c06a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f109a7fc700 1 -- 172.21.15.45:0/3998485141 >> v1:172.21.15.144:6789/0 conn(0x7f109c157140 legacy=0x7f109c0ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f109a7fc700 1 -- 172.21.15.45:0/3998485141 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c06e980 con 0x7f109c06dff0 2023-10-06T05:15:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f109a7fc700 1 --2- 172.21.15.45:0/3998485141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06dff0 0x7f109c06e440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f109c0cce10 tx=0x7f1084009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7438995a0ed72179 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f1099ffb700 1 -- 172.21.15.45:0/3998485141 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400b820 con 0x7f109c06dff0 2023-10-06T05:15:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f1099ffb700 1 -- 172.21.15.45:0/3998485141 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108400b980 con 0x7f109c06dff0 2023-10-06T05:15:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.797+0000 7f1099ffb700 1 -- 172.21.15.45:0/3998485141 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400bd20 con 0x7f109c06dff0 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 -- 172.21.15.45:0/3998485141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06dff0 msgr2=0x7f109c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 --2- 172.21.15.45:0/3998485141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06dff0 0x7f109c06e440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f109c0cce10 tx=0x7f1084009250 comp rx=0 tx=0).stop 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 -- 172.21.15.45:0/3998485141 shutdown_connections 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 --2- 172.21.15.45:0/3998485141 >> v2:172.21.15.144:3300/0 conn(0x7f109c06a070 0x7f109c06a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 --2- 172.21.15.45:0/3998485141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06dff0 0x7f109c06e440 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 -- 172.21.15.45:0/3998485141 >> 172.21.15.45:0/3998485141 conn(0x7f109c0680f0 msgr2=0x7f109c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 -- 172.21.15.45:0/3998485141 shutdown_connections 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.798+0000 7f10a13a0700 1 -- 172.21.15.45:0/3998485141 wait complete. 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.799+0000 7f10a13a0700 1 Processor -- start 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.799+0000 7f10a13a0700 1 -- start start 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f10a13a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06a070 0x7f109c168a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f10a13a0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f109c06dff0 0x7f109c16a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f10a13a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c157140 0x7f109c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f10a13a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c168840 con 0x7f109c06a070 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f10a13a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c168560 con 0x7f109c157140 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f10a13a0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0c3ed0 con 0x7f109c06dff0 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06a070 0x7f109c168a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06a070 0x7f109c168a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37790/0 (socket says 172.21.15.45:37790) 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 -- 172.21.15.45:0/4037059077 learned_addr learned my addr 172.21.15.45:0/4037059077 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109a7fc700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f109c06dff0 0x7f109c16a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109b7fe700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c157140 0x7f109c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 -- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f109c06dff0 msgr2=0x7f109c16a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f109c06dff0 0x7f109c16a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 -- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c157140 msgr2=0x7f109c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c157140 0x7f109c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.800+0000 7f109affd700 1 -- 172.21.15.45:0/4037059077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840094f0 con 0x7f109c06a070 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.801+0000 7f109affd700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06a070 0x7f109c168a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f108c00b5e0 tx=0x7f108c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.801+0000 7f1092ffd700 1 -- 172.21.15.45:0/4037059077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00f510 con 0x7f109c06a070 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.801+0000 7f1092ffd700 1 -- 172.21.15.45:0/4037059077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c00f670 con 0x7f109c06a070 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.801+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c0c41b0 con 0x7f109c06a070 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.801+0000 7f1092ffd700 1 -- 172.21.15.45:0/4037059077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00fa30 con 0x7f109c06a070 2023-10-06T05:15:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.801+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c15b700 con 0x7f109c06a070 2023-10-06T05:15:47.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.802+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c06cc70 con 0x7f109c06a070 2023-10-06T05:15:47.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.802+0000 7f1092ffd700 1 -- 172.21.15.45:0/4037059077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f108c01a400 con 0x7f109c06a070 2023-10-06T05:15:47.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.803+0000 7f1092ffd700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f107c066f30 0x7f107c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:47.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.803+0000 7f1092ffd700 1 -- 172.21.15.45:0/4037059077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f108c0843d0 con 0x7f109c06a070 2023-10-06T05:15:47.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.803+0000 7f109a7fc700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f107c066f30 0x7f107c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:47.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.803+0000 7f109a7fc700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f107c066f30 0x7f107c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f109c0cce10 tx=0x7f1084016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:47.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:47.807+0000 7f1092ffd700 1 -- 172.21.15.45:0/4037059077 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f108c051230 con 0x7f109c06a070 2023-10-06T05:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.016+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f109c158520 con 0x7f109c06a070 2023-10-06T05:15:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.016+0000 7f1092ffd700 1 -- 172.21.15.45:0/4037059077 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f108c054850 con 0x7f109c06a070 2023-10-06T05:15:48.018 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f107c066f30 msgr2=0x7f107c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f107c066f30 0x7f107c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f109c0cce10 tx=0x7f1084016000 comp rx=0 tx=0).stop 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06a070 msgr2=0x7f109c168a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06a070 0x7f109c168a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f108c00b5e0 tx=0x7f108c009120 comp rx=0 tx=0).stop 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 shutdown_connections 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f107c066f30 0x7f107c0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109c157140 0x7f109c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f109c06a070 0x7f109c168a20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 --2- 172.21.15.45:0/4037059077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f109c06dff0 0x7f109c16a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 >> 172.21.15.45:0/4037059077 conn(0x7f109c0680f0 msgr2=0x7f109c0ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 shutdown_connections 2023-10-06T05:15:48.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.019+0000 7f10a13a0700 1 -- 172.21.15.45:0/4037059077 wait complete. 2023-10-06T05:15:48.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.169+0000 7fcc2c2eb700 1 Processor -- start 2023-10-06T05:15:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.169+0000 7fcc2c2eb700 1 -- start start 2023-10-06T05:15:48.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2c2eb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc2406ae40 0x7fcc2415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2c2eb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc2415c6f0 0x7fcc240aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2c2eb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc24067d80 con 0x7fcc2406aa70 2023-10-06T05:15:48.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2c2eb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc240ac620 con 0x7fcc2415c6f0 2023-10-06T05:15:48.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2c2eb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2415cb70 con 0x7fcc2406ae40 2023-10-06T05:15:48.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2a888700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc2415c6f0 0x7fcc240aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2a087700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcc2406aa70 0x7fcc240c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49680/0 (socket says 172.21.15.45:49680) 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.170+0000 7fcc2a087700 1 -- 172.21.15.45:0/1731861969 learned_addr learned my addr 172.21.15.45:0/1731861969 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc29886700 1 --2- 172.21.15.45:0/1731861969 >> v2:172.21.15.144:3300/0 conn(0x7fcc2406ae40 0x7fcc2415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc2a888700 1 -- 172.21.15.45:0/1731861969 >> v2:172.21.15.144:3300/0 conn(0x7fcc2406ae40 msgr2=0x7fcc2415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc2a888700 1 --2- 172.21.15.45:0/1731861969 >> v2:172.21.15.144:3300/0 conn(0x7fcc2406ae40 0x7fcc2415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc2a888700 1 -- 172.21.15.45:0/1731861969 >> v1:172.21.15.45:6789/0 conn(0x7fcc2406aa70 legacy=0x7fcc240c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc2a888700 1 -- 172.21.15.45:0/1731861969 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240c75b0 con 0x7fcc2415c6f0 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc29085700 1 -- 172.21.15.45:0/1731861969 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195150093 0 0) 0x7fcc24067d80 con 0x7fcc2406aa70 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc29886700 1 --2- 172.21.15.45:0/1731861969 >> v2:172.21.15.144:3300/0 conn(0x7fcc2406ae40 0x7fcc2415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc2a888700 1 --2- 172.21.15.45:0/1731861969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2415c6f0 0x7fcc240aca70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcc240cd5e0 tx=0x7fcc1c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec218e756d9d12d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc29085700 1 -- 172.21.15.45:0/1731861969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00e070 con 0x7fcc2415c6f0 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc29085700 1 -- 172.21.15.45:0/1731861969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00bc30 con 0x7fcc2415c6f0 2023-10-06T05:15:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.171+0000 7fcc29085700 1 -- 172.21.15.45:0/1731861969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c012430 con 0x7fcc2415c6f0 2023-10-06T05:15:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/1731861969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2415c6f0 msgr2=0x7fcc240aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/1731861969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2415c6f0 0x7fcc240aca70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcc240cd5e0 tx=0x7fcc1c009780 comp rx=0 tx=0).stop 2023-10-06T05:15:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/1731861969 shutdown_connections 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/1731861969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2415c6f0 0x7fcc240aca70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/1731861969 >> v2:172.21.15.144:3300/0 conn(0x7fcc2406ae40 0x7fcc2415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/1731861969 >> 172.21.15.45:0/1731861969 conn(0x7fcc240cc570 msgr2=0x7fcc240cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/1731861969 shutdown_connections 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.172+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/1731861969 wait complete. 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.173+0000 7fcc2c2eb700 1 Processor -- start 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.173+0000 7fcc2c2eb700 1 -- start start 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2c2eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406aa70 0x7fcc240c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2c2eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2406ae40 0x7fcc240c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2c2eb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2415c6f0 0x7fcc240c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2c2eb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240c4cd0 con 0x7fcc2406ae40 2023-10-06T05:15:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2c2eb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240c5ac0 con 0x7fcc2406aa70 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2c2eb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240c4eb0 con 0x7fcc2415c6f0 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2406ae40 0x7fcc240c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2406ae40 0x7fcc240c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37810/0 (socket says 172.21.15.45:37810) 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 -- 172.21.15.45:0/3901448823 learned_addr learned my addr 172.21.15.45:0/3901448823 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2a087700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406aa70 0x7fcc240c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2a888700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2415c6f0 0x7fcc240c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 -- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2415c6f0 msgr2=0x7fcc240c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2415c6f0 0x7fcc240c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 -- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406aa70 msgr2=0x7fcc240c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406aa70 0x7fcc240c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc29886700 1 -- 172.21.15.45:0/3901448823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009a20 con 0x7fcc2406ae40 2023-10-06T05:15:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2a087700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406aa70 0x7fcc240c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.174+0000 7fcc2a888700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2415c6f0 0x7fcc240c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.175+0000 7fcc29886700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2406ae40 0x7fcc240c3f90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e8d0 tx=0x7fcc14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.175+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/3901448823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400ee70 con 0x7fcc2406ae40 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.175+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/3901448823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1400c5e0 con 0x7fcc2406ae40 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.175+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc240c5190 con 0x7fcc2406ae40 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.175+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/3901448823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400c980 con 0x7fcc2406ae40 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.175+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc240c5750 con 0x7fcc2406ae40 2023-10-06T05:15:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.176+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2406b320 con 0x7fcc2406ae40 2023-10-06T05:15:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.176+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/3901448823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc14013410 con 0x7fcc2406ae40 2023-10-06T05:15:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.177+0000 7fcc1a7fc700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc08066f30 0x7fcc080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.177+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/3901448823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fcc14084820 con 0x7fcc2406ae40 2023-10-06T05:15:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.177+0000 7fcc2a087700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc08066f30 0x7fcc080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.177+0000 7fcc2a087700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc08066f30 0x7fcc080693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc200011e0 tx=0x7fcc20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.181+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/3901448823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc14051600 con 0x7fcc2406ae40 2023-10-06T05:15:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.388+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc240c5c00 con 0x7fcc2406ae40 2023-10-06T05:15:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.388+0000 7fcc1a7fc700 1 -- 172.21.15.45:0/3901448823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc14054c20 con 0x7fcc2406ae40 2023-10-06T05:15:48.391 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc08066f30 msgr2=0x7fcc080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc08066f30 0x7fcc080693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc200011e0 tx=0x7fcc20009040 comp rx=0 tx=0).stop 2023-10-06T05:15:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2406ae40 msgr2=0x7fcc240c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2406ae40 0x7fcc240c3f90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e8d0 tx=0x7fcc14008040 comp rx=0 tx=0).stop 2023-10-06T05:15:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 shutdown_connections 2023-10-06T05:15:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc08066f30 0x7fcc080693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc2415c6f0 0x7fcc240c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406aa70 0x7fcc240c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 --2- 172.21.15.45:0/3901448823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2406ae40 0x7fcc240c3f90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 >> 172.21.15.45:0/3901448823 conn(0x7fcc240cc570 msgr2=0x7fcc240ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 shutdown_connections 2023-10-06T05:15:48.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.391+0000 7fcc2c2eb700 1 -- 172.21.15.45:0/3901448823 wait complete. 2023-10-06T05:15:48.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:48.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.540+0000 7fa2279d6700 1 Processor -- start 2023-10-06T05:15:48.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.540+0000 7fa2279d6700 1 -- start start 2023-10-06T05:15:48.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa2279d6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa22006ae40 0x7fa22015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa2279d6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa2279d6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa220067d80 con 0x7fa22006aa70 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa2279d6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2200ac620 con 0x7fa22006ae40 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa2279d6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22015cb70 con 0x7fa22015c6f0 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa224f71700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa22006ae40 0x7fa22015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa224f71700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa22006ae40 0x7fa22015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37816/0 (socket says 172.21.15.45:37816) 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa225772700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa22006aa70 0x7fa2200c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49694/0 (socket says 172.21.15.45:49694) 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa224f71700 1 -- 172.21.15.45:0/1229914560 learned_addr learned my addr 172.21.15.45:0/1229914560 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.541+0000 7fa225f73700 1 --2- 172.21.15.45:0/1229914560 >> v2:172.21.15.144:3300/0 conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa224f71700 1 -- 172.21.15.45:0/1229914560 >> v2:172.21.15.144:3300/0 conn(0x7fa22015c6f0 msgr2=0x7fa2200aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa224f71700 1 --2- 172.21.15.45:0/1229914560 >> v2:172.21.15.144:3300/0 conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa224f71700 1 -- 172.21.15.45:0/1229914560 >> v1:172.21.15.45:6789/0 conn(0x7fa22006aa70 legacy=0x7fa2200c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa224f71700 1 -- 172.21.15.45:0/1229914560 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2200c75b0 con 0x7fa22006ae40 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa217fff700 1 -- 172.21.15.45:0/1229914560 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935499968 0 0) 0x7fa220067d80 con 0x7fa22006aa70 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa225f73700 1 --2- 172.21.15.45:0/1229914560 >> v2:172.21.15.144:3300/0 conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa224f71700 1 --2- 172.21.15.45:0/1229914560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa22015c1b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa2200c8ef0 tx=0x7fa210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec9bc94c1d133b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa217fff700 1 -- 172.21.15.45:0/1229914560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21000b860 con 0x7fa22006ae40 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa217fff700 1 -- 172.21.15.45:0/1229914560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21000b9c0 con 0x7fa22006ae40 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.542+0000 7fa217fff700 1 -- 172.21.15.45:0/1229914560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21000bd60 con 0x7fa22006ae40 2023-10-06T05:15:48.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 -- 172.21.15.45:0/1229914560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 msgr2=0x7fa22015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1229914560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa22015c1b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa2200c8ef0 tx=0x7fa210009250 comp rx=0 tx=0).stop 2023-10-06T05:15:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 -- 172.21.15.45:0/1229914560 shutdown_connections 2023-10-06T05:15:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1229914560 >> v2:172.21.15.144:3300/0 conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1229914560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa22015c1b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 -- 172.21.15.45:0/1229914560 >> 172.21.15.45:0/1229914560 conn(0x7fa2200cc570 msgr2=0x7fa2200cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 -- 172.21.15.45:0/1229914560 shutdown_connections 2023-10-06T05:15:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.543+0000 7fa2279d6700 1 -- 172.21.15.45:0/1229914560 wait complete. 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.544+0000 7fa2279d6700 1 Processor -- start 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.544+0000 7fa2279d6700 1 -- start start 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa2279d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa22006aa70 0x7fa220156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa2279d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa2200c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa2279d6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa2279d6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2200c5f60 con 0x7fa22006ae40 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa2279d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2200c5350 con 0x7fa22006aa70 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa2279d6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2200c5630 con 0x7fa22015c6f0 2023-10-06T05:15:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa2200c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa2200c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37830/0 (socket says 172.21.15.45:37830) 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 -- 172.21.15.45:0/1868207450 learned_addr learned my addr 172.21.15.45:0/1868207450 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa225772700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa22006aa70 0x7fa220156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa225f73700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 -- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa22015c6f0 msgr2=0x7fa2200aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 -- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa22006aa70 msgr2=0x7fa220156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa22006aa70 0x7fa220156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.545+0000 7fa224f71700 1 -- 172.21.15.45:0/1868207450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100094f0 con 0x7fa22006ae40 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.546+0000 7fa224f71700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa2200c49b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa2100090b0 tx=0x7fa210017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.546+0000 7fa2157fa700 1 -- 172.21.15.45:0/1868207450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210011e60 con 0x7fa22006ae40 2023-10-06T05:15:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.546+0000 7fa2157fa700 1 -- 172.21.15.45:0/1868207450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21001f070 con 0x7fa22006ae40 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.546+0000 7fa2157fa700 1 -- 172.21.15.45:0/1868207450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21001a890 con 0x7fa22006ae40 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.546+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2200c5910 con 0x7fa22006ae40 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.546+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2201574c0 con 0x7fa22006ae40 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.547+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2200c0ec0 con 0x7fa22006ae40 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.547+0000 7fa2157fa700 1 -- 172.21.15.45:0/1868207450 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa21001ac00 con 0x7fa22006ae40 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.548+0000 7fa2157fa700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa204066f60 0x7fa204069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.548+0000 7fa2157fa700 1 -- 172.21.15.45:0/1868207450 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fa210021070 con 0x7fa22006ae40 2023-10-06T05:15:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.548+0000 7fa225772700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa204066f60 0x7fa204069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.548+0000 7fa225772700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa204066f60 0x7fa204069410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa21c007fd0 tx=0x7fa21c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.552+0000 7fa2157fa700 1 -- 172.21.15.45:0/1868207450 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa210053700 con 0x7fa22006ae40 2023-10-06T05:15:48.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.756+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2200c7890 con 0x7fa22006ae40 2023-10-06T05:15:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.756+0000 7fa2157fa700 1 -- 172.21.15.45:0/1868207450 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa21002b020 con 0x7fa22006ae40 2023-10-06T05:15:48.758 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa204066f60 msgr2=0x7fa204069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa204066f60 0x7fa204069410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa21c007fd0 tx=0x7fa21c009040 comp rx=0 tx=0).stop 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 msgr2=0x7fa2200c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa2200c49b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa2100090b0 tx=0x7fa210017000 comp rx=0 tx=0).stop 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 shutdown_connections 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa204066f60 0x7fa204069410 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa22015c6f0 0x7fa2200aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa22006aa70 0x7fa220156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 --2- 172.21.15.45:0/1868207450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22006ae40 0x7fa2200c49b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 >> 172.21.15.45:0/1868207450 conn(0x7fa2200cc570 msgr2=0x7fa220157ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 shutdown_connections 2023-10-06T05:15:48.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.759+0000 7fa2279d6700 1 -- 172.21.15.45:0/1868207450 wait complete. 2023-10-06T05:15:48.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T05:15:48.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.908+0000 7fb855f74700 1 Processor -- start 2023-10-06T05:15:48.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.908+0000 7fb855f74700 1 -- start start 2023-10-06T05:15:48.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb855f74700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb850157330 0x7fb8500ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb855f74700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8500ccb70 0x7fb8500c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb855f74700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8500c3b50 con 0x7fb8500aaae0 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb855f74700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb850157700 con 0x7fb8500ccb70 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb855f74700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8500c98e0 con 0x7fb850157330 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8500ccb70 0x7fb8500c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8500ccb70 0x7fb8500c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37844/0 (socket says 172.21.15.45:37844) 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb8500aaae0 0x7fb8500c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49706/0 (socket says 172.21.15.45:49706) 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 -- 172.21.15.45:0/2336916352 learned_addr learned my addr 172.21.15.45:0/2336916352 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84f7fe700 1 --2- 172.21.15.45:0/2336916352 >> v2:172.21.15.144:3300/0 conn(0x7fb850157330 0x7fb8500ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 -- 172.21.15.45:0/2336916352 >> v2:172.21.15.144:3300/0 conn(0x7fb850157330 msgr2=0x7fb8500ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 --2- 172.21.15.45:0/2336916352 >> v2:172.21.15.144:3300/0 conn(0x7fb850157330 0x7fb8500ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 -- 172.21.15.45:0/2336916352 >> v1:172.21.15.45:6789/0 conn(0x7fb8500aaae0 legacy=0x7fb8500c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84e7fc700 1 -- 172.21.15.45:0/2336916352 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293884307 0 0) 0x7fb8500c3b50 con 0x7fb8500aaae0 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 -- 172.21.15.45:0/2336916352 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500c9bc0 con 0x7fb8500ccb70 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.909+0000 7fb84ffff700 1 --2- 172.21.15.45:0/2336916352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c9280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb8500cd1d0 tx=0x7fb848009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fceb16b36b1cfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb84e7fc700 1 -- 172.21.15.45:0/2336916352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800e070 con 0x7fb8500ccb70 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb84e7fc700 1 -- 172.21.15.45:0/2336916352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84800ba20 con 0x7fb8500ccb70 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb84e7fc700 1 -- 172.21.15.45:0/2336916352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480092e0 con 0x7fb8500ccb70 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 -- 172.21.15.45:0/2336916352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 msgr2=0x7fb8500c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 --2- 172.21.15.45:0/2336916352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c9280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb8500cd1d0 tx=0x7fb848009840 comp rx=0 tx=0).stop 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 -- 172.21.15.45:0/2336916352 shutdown_connections 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 --2- 172.21.15.45:0/2336916352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c9280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 --2- 172.21.15.45:0/2336916352 >> v2:172.21.15.144:3300/0 conn(0x7fb850157330 0x7fb8500ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 -- 172.21.15.45:0/2336916352 >> 172.21.15.45:0/2336916352 conn(0x7fb850067e00 msgr2=0x7fb8500cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:48.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 -- 172.21.15.45:0/2336916352 shutdown_connections 2023-10-06T05:15:48.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.910+0000 7fb855f74700 1 -- 172.21.15.45:0/2336916352 wait complete. 2023-10-06T05:15:48.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.912+0000 7fb855f74700 1 Processor -- start 2023-10-06T05:15:48.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.912+0000 7fb855f74700 1 -- start start 2023-10-06T05:15:48.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb855f74700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8500aaae0 0x7fb85014cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb855f74700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb855f74700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb850157330 0x7fb8500ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb855f74700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500c7550 con 0x7fb8500ccb70 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb855f74700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500b1790 con 0x7fb850157330 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb855f74700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500b0e60 con 0x7fb8500aaae0 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb84effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb84effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37846/0 (socket says 172.21.15.45:37846) 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb84effd700 1 -- 172.21.15.45:0/2208768154 learned_addr learned my addr 172.21.15.45:0/2208768154 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb84ffff700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb850157330 0x7fb8500ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.913+0000 7fb84f7fe700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8500aaae0 0x7fb85014cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb84effd700 1 -- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8500aaae0 msgr2=0x7fb85014cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb84effd700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8500aaae0 0x7fb85014cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb84effd700 1 -- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb850157330 msgr2=0x7fb8500ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb84effd700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb850157330 0x7fb8500ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb84effd700 1 -- 172.21.15.45:0/2208768154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb848009ae0 con 0x7fb8500ccb70 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb84ffff700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb850157330 0x7fb8500ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb84effd700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c6c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c5b0 tx=0x7fb844001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb8377fe700 1 -- 172.21.15.45:0/2208768154 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844005a70 con 0x7fb8500ccb70 2023-10-06T05:15:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb8377fe700 1 -- 172.21.15.45:0/2208768154 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb844005bd0 con 0x7fb8500ccb70 2023-10-06T05:15:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8500b1140 con 0x7fb8500ccb70 2023-10-06T05:15:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb8377fe700 1 -- 172.21.15.45:0/2208768154 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440070d0 con 0x7fb8500ccb70 2023-10-06T05:15:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.914+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85014d170 con 0x7fb8500ccb70 2023-10-06T05:15:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.915+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8500b1900 con 0x7fb8500ccb70 2023-10-06T05:15:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.915+0000 7fb8377fe700 1 -- 172.21.15.45:0/2208768154 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb844007480 con 0x7fb8500ccb70 2023-10-06T05:15:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.916+0000 7fb8377fe700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb83c066f30 0x7fb83c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:48.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.916+0000 7fb8377fe700 1 -- 172.21.15.45:0/2208768154 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fb844083ee0 con 0x7fb8500ccb70 2023-10-06T05:15:48.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.916+0000 7fb84f7fe700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb83c066f30 0x7fb83c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:48.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.917+0000 7fb84f7fe700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb83c066f30 0x7fb83c0693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb840009d30 tx=0x7fb840008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:48.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:48.920+0000 7fb8377fe700 1 -- 172.21.15.45:0/2208768154 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb844050cc0 con 0x7fb8500ccb70 2023-10-06T05:15:49.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.123+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb85006e350 con 0x7fb8500ccb70 2023-10-06T05:15:49.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.123+0000 7fb8377fe700 1 -- 172.21.15.45:0/2208768154 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb844017020 con 0x7fb8500ccb70 2023-10-06T05:15:49.126 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb83c066f30 msgr2=0x7fb83c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb83c066f30 0x7fb83c0693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb840009d30 tx=0x7fb840008040 comp rx=0 tx=0).stop 2023-10-06T05:15:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 msgr2=0x7fb8500c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c6c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c5b0 tx=0x7fb844001040 comp rx=0 tx=0).stop 2023-10-06T05:15:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 shutdown_connections 2023-10-06T05:15:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb83c066f30 0x7fb83c0693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb850157330 0x7fb8500ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8500aaae0 0x7fb85014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 --2- 172.21.15.45:0/2208768154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8500ccb70 0x7fb8500c6c50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 >> 172.21.15.45:0/2208768154 conn(0x7fb850067e00 msgr2=0x7fb8500c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 shutdown_connections 2023-10-06T05:15:49.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.126+0000 7fb855f74700 1 -- 172.21.15.45:0/2208768154 wait complete. 2023-10-06T05:15:49.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.271+0000 7f2456d7d700 1 Processor -- start 2023-10-06T05:15:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.272+0000 7f2456d7d700 1 -- start start 2023-10-06T05:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.272+0000 7f2456d7d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2450158f60 0x7f2450159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.272+0000 7f2456d7d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f245006a060 0x7f24500cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.272+0000 7f2456d7d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2450069bd0 con 0x7f24500cc390 2023-10-06T05:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.272+0000 7f2456d7d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f245006a4d0 con 0x7f245006a060 2023-10-06T05:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.272+0000 7f2456d7d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2450068fc0 con 0x7f2450158f60 2023-10-06T05:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f245006a060 0x7f24500cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f245531a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f24500cc390 0x7f24500c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49722/0 (socket says 172.21.15.45:49722) 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f245531a700 1 -- 172.21.15.45:0/115685268 learned_addr learned my addr 172.21.15.45:0/115685268 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f2454b19700 1 --2- 172.21.15.45:0/115685268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2450158f60 0x7f2450159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244ffff700 1 -- 172.21.15.45:0/115685268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2450158f60 msgr2=0x7f2450159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244ffff700 1 --2- 172.21.15.45:0/115685268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2450158f60 0x7f2450159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244ffff700 1 -- 172.21.15.45:0/115685268 >> v1:172.21.15.45:6789/0 conn(0x7f24500cc390 legacy=0x7f24500c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244ffff700 1 -- 172.21.15.45:0/115685268 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500692a0 con 0x7f245006a060 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244f7fe700 1 -- 172.21.15.45:0/115685268 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319529318 0 0) 0x7f2450069bd0 con 0x7f24500cc390 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f2454b19700 1 --2- 172.21.15.45:0/115685268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2450158f60 0x7f2450159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244ffff700 1 --2- 172.21.15.45:0/115685268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f24500cae20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f243c0098e0 tx=0x7f243c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405d645ce55ba0db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244f7fe700 1 -- 172.21.15.45:0/115685268 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c009e80 con 0x7f245006a060 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244f7fe700 1 -- 172.21.15.45:0/115685268 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c00b860 con 0x7f245006a060 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.273+0000 7f244f7fe700 1 -- 172.21.15.45:0/115685268 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00bbe0 con 0x7f245006a060 2023-10-06T05:15:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 -- 172.21.15.45:0/115685268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 msgr2=0x7f24500cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 --2- 172.21.15.45:0/115685268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f24500cae20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f243c0098e0 tx=0x7f243c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 -- 172.21.15.45:0/115685268 shutdown_connections 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 --2- 172.21.15.45:0/115685268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2450158f60 0x7f2450159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 --2- 172.21.15.45:0/115685268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f24500cae20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 -- 172.21.15.45:0/115685268 >> 172.21.15.45:0/115685268 conn(0x7f24500cdf50 msgr2=0x7f24500ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 -- 172.21.15.45:0/115685268 shutdown_connections 2023-10-06T05:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.274+0000 7f2456d7d700 1 -- 172.21.15.45:0/115685268 wait complete. 2023-10-06T05:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.275+0000 7f2456d7d700 1 Processor -- start 2023-10-06T05:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.275+0000 7f2456d7d700 1 -- start start 2023-10-06T05:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2456d7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f245016d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2456d7d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24500cc390 0x7f2450155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2456d7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2450158f60 0x7f245015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2456d7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245016de70 con 0x7f245006a060 2023-10-06T05:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2456d7d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450169d00 con 0x7f2450158f60 2023-10-06T05:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2456d7d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2450169fe0 con 0x7f24500cc390 2023-10-06T05:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2454b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f245016d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2454b19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f245016d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37864/0 (socket says 172.21.15.45:37864) 2023-10-06T05:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.276+0000 7f2454b19700 1 -- 172.21.15.45:0/810652707 learned_addr learned my addr 172.21.15.45:0/810652707 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f244ffff700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24500cc390 0x7f2450155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f245531a700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2450158f60 0x7f245015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2454b19700 1 -- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24500cc390 msgr2=0x7f2450155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2454b19700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24500cc390 0x7f2450155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2454b19700 1 -- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2450158f60 msgr2=0x7f245015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2454b19700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2450158f60 0x7f245015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2454b19700 1 -- 172.21.15.45:0/810652707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c009420 con 0x7f245006a060 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2454b19700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f245016d930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f244400d860 tx=0x7f2444008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f244cff9700 1 -- 172.21.15.45:0/810652707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440125f0 con 0x7f245006a060 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f244cff9700 1 -- 172.21.15.45:0/810652707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444012750 con 0x7f245006a060 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f244cff9700 1 -- 172.21.15.45:0/810652707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444012ad0 con 0x7f245006a060 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245016a2c0 con 0x7f245006a060 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.277+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245016a880 con 0x7f245006a060 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.278+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245015ebd0 con 0x7f245006a060 2023-10-06T05:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.278+0000 7f244cff9700 1 -- 172.21.15.45:0/810652707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2444014430 con 0x7f245006a060 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.279+0000 7f244cff9700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2434066f30 0x7f24340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.279+0000 7f244cff9700 1 -- 172.21.15.45:0/810652707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f2444084e10 con 0x7f245006a060 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.279+0000 7f244ffff700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2434066f30 0x7f24340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.280+0000 7f244ffff700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2434066f30 0x7f24340693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f243c0091e0 tx=0x7f243c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.283+0000 7f244cff9700 1 -- 172.21.15.45:0/810652707 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2444055680 con 0x7f245006a060 2023-10-06T05:15:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.489+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24500c3ef0 con 0x7f245006a060 2023-10-06T05:15:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.490+0000 7f244cff9700 1 -- 172.21.15.45:0/810652707 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18899 (secure 0 0 0) 0x7f2444055210 con 0x7f245006a060 2023-10-06T05:15:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:49.493 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:47.878034+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2434066f30 msgr2=0x7f24340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2434066f30 0x7f24340693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f243c0091e0 tx=0x7f243c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 msgr2=0x7f245016d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f245016d930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f244400d860 tx=0x7f2444008040 comp rx=0 tx=0).stop 2023-10-06T05:15:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 shutdown_connections 2023-10-06T05:15:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2434066f30 0x7f24340693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2450158f60 0x7f245015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f245006a060 0x7f245016d930 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 --2- 172.21.15.45:0/810652707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24500cc390 0x7f2450155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 >> 172.21.15.45:0/810652707 conn(0x7f24500cdf50 msgr2=0x7f2450070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 shutdown_connections 2023-10-06T05:15:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.493+0000 7f2456d7d700 1 -- 172.21.15.45:0/810652707 wait complete. 2023-10-06T05:15:49.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:57.253547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:58.338382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T05:15:01.988547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:49.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:49.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.642+0000 7f41254be700 1 Processor -- start 2023-10-06T05:15:49.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.642+0000 7f41254be700 1 -- start start 2023-10-06T05:15:49.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f41254be700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f41200d0680 0x7f412015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f41254be700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f412015c6f0 0x7f41200caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f41254be700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41200cd740 con 0x7f41200d02b0 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f41254be700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41200d0ad0 con 0x7f41200d0680 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f41254be700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41200ccb30 con 0x7f412015c6f0 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411f7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f412015c6f0 0x7f41200caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411e7fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f41200d0680 0x7f412015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411effd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f41200d02b0 0x7f412015d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:58788/0 (socket says 172.21.15.45:58788) 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411effd700 1 -- 172.21.15.45:0/1494043328 learned_addr learned my addr 172.21.15.45:0/1494043328 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411f7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f412015c6f0 0x7f41200caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:35852/0 (socket says 172.21.15.45:35852) 2023-10-06T05:15:49.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411f7fe700 1 -- 172.21.15.45:0/1494043328 >> v2:172.21.15.144:3300/0 conn(0x7f41200d0680 msgr2=0x7f412015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411f7fe700 1 --2- 172.21.15.45:0/1494043328 >> v2:172.21.15.144:3300/0 conn(0x7f41200d0680 0x7f412015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.643+0000 7f411f7fe700 1 -- 172.21.15.45:0/1494043328 >> v1:172.21.15.144:6789/0 conn(0x7f41200d02b0 legacy=0x7f412015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f411f7fe700 1 -- 172.21.15.45:0/1494043328 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200cce10 con 0x7f412015c6f0 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f411e7fc700 1 --2- 172.21.15.45:0/1494043328 >> v2:172.21.15.144:3300/0 conn(0x7f41200d0680 0x7f412015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f411f7fe700 1 --2- 172.21.15.45:0/1494043328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f412015c6f0 0x7f41200caf10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4118009e60 tx=0x7f4118009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83388f1d5f978786 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f411dffb700 1 -- 172.21.15.45:0/1494043328 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800e070 con 0x7f412015c6f0 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f411dffb700 1 -- 172.21.15.45:0/1494043328 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411800bb60 con 0x7f412015c6f0 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f411dffb700 1 -- 172.21.15.45:0/1494043328 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118012430 con 0x7f412015c6f0 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f41254be700 1 -- 172.21.15.45:0/1494043328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f412015c6f0 msgr2=0x7f41200caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.644+0000 7f41254be700 1 --2- 172.21.15.45:0/1494043328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f412015c6f0 0x7f41200caf10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4118009e60 tx=0x7f4118009780 comp rx=0 tx=0).stop 2023-10-06T05:15:49.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.645+0000 7f41254be700 1 -- 172.21.15.45:0/1494043328 shutdown_connections 2023-10-06T05:15:49.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.645+0000 7f41254be700 1 --2- 172.21.15.45:0/1494043328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f412015c6f0 0x7f41200caf10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.645+0000 7f41254be700 1 --2- 172.21.15.45:0/1494043328 >> v2:172.21.15.144:3300/0 conn(0x7f41200d0680 0x7f412015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.645+0000 7f41254be700 1 -- 172.21.15.45:0/1494043328 >> 172.21.15.45:0/1494043328 conn(0x7f412006ed50 msgr2=0x7f412006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.645+0000 7f41254be700 1 -- 172.21.15.45:0/1494043328 shutdown_connections 2023-10-06T05:15:49.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.645+0000 7f41254be700 1 -- 172.21.15.45:0/1494043328 wait complete. 2023-10-06T05:15:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.646+0000 7f41254be700 1 Processor -- start 2023-10-06T05:15:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.646+0000 7f41254be700 1 -- start start 2023-10-06T05:15:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f41254be700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200d02b0 0x7f41200c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f41254be700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41200d0680 0x7f41200cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f41254be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f412015c6f0 0x7f41200b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f41254be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200c6790 con 0x7f412015c6f0 2023-10-06T05:15:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f41254be700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200c6fa0 con 0x7f41200d02b0 2023-10-06T05:15:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f41254be700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41200b35f0 con 0x7f41200d0680 2023-10-06T05:15:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f412015c6f0 0x7f41200b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f412015c6f0 0x7f41200b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37878/0 (socket says 172.21.15.45:37878) 2023-10-06T05:15:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 -- 172.21.15.45:0/1488523520 learned_addr learned my addr 172.21.15.45:0/1488523520 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411e7fc700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41200d0680 0x7f41200cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411effd700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200d02b0 0x7f41200c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 -- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41200d0680 msgr2=0x7f41200cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41200d0680 0x7f41200cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 -- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200d02b0 msgr2=0x7f41200c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200d02b0 0x7f41200c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 -- 172.21.15.45:0/1488523520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4108009420 con 0x7f412015c6f0 2023-10-06T05:15:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.647+0000 7f411f7fe700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f412015c6f0 0x7f41200b3a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f411800b6c0 tx=0x7f4118019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.648+0000 7f4116ffd700 1 -- 172.21.15.45:0/1488523520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118012e90 con 0x7f412015c6f0 2023-10-06T05:15:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.648+0000 7f4116ffd700 1 -- 172.21.15.45:0/1488523520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118027470 con 0x7f412015c6f0 2023-10-06T05:15:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.648+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4118009a20 con 0x7f412015c6f0 2023-10-06T05:15:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.648+0000 7f4116ffd700 1 -- 172.21.15.45:0/1488523520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118027850 con 0x7f412015c6f0 2023-10-06T05:15:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.648+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41200c6cc0 con 0x7f412015c6f0 2023-10-06T05:15:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.649+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4120088730 con 0x7f412015c6f0 2023-10-06T05:15:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.649+0000 7f4116ffd700 1 -- 172.21.15.45:0/1488523520 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4118024070 con 0x7f412015c6f0 2023-10-06T05:15:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.650+0000 7f4116ffd700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4100066f30 0x7f41000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.650+0000 7f4116ffd700 1 -- 172.21.15.45:0/1488523520 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f4118087450 con 0x7f412015c6f0 2023-10-06T05:15:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.650+0000 7f411effd700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4100066f30 0x7f41000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.650+0000 7f411effd700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4100066f30 0x7f41000693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4110007130 tx=0x7f4110007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.654+0000 7f4116ffd700 1 -- 172.21.15.45:0/1488523520 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4118054230 con 0x7f412015c6f0 2023-10-06T05:15:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.856+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41200b3730 con 0x7f412015c6f0 2023-10-06T05:15:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.856+0000 7f4116ffd700 1 -- 172.21.15.45:0/1488523520 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4118057850 con 0x7f412015c6f0 2023-10-06T05:15:49.859 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:15:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4100066f30 msgr2=0x7f41000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4100066f30 0x7f41000693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4110007130 tx=0x7f4110007eb0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f412015c6f0 msgr2=0x7f41200b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f412015c6f0 0x7f41200b3a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f411800b6c0 tx=0x7f4118019000 comp rx=0 tx=0).stop 2023-10-06T05:15:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 shutdown_connections 2023-10-06T05:15:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4100066f30 0x7f41000693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f412015c6f0 0x7f41200b3a80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200d02b0 0x7f41200c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 --2- 172.21.15.45:0/1488523520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41200d0680 0x7f41200cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 >> 172.21.15.45:0/1488523520 conn(0x7f412006ed50 msgr2=0x7f41200b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 shutdown_connections 2023-10-06T05:15:49.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:49.859+0000 7f41254be700 1 -- 172.21.15.45:0/1488523520 wait complete. 2023-10-06T05:15:49.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.006+0000 7fdee40fb700 1 Processor -- start 2023-10-06T05:15:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.007+0000 7fdee40fb700 1 -- start start 2023-10-06T05:15:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.007+0000 7fdee40fb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdedc0c3db0 0x7fdedc0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.007+0000 7fdee40fb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdedc069ed0 0x7fdedc0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.007+0000 7fdee40fb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdedc0ce6a0 con 0x7fdedc0b0610 2023-10-06T05:15:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.007+0000 7fdee40fb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdedc0690f0 con 0x7fdedc0c3db0 2023-10-06T05:15:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.007+0000 7fdee40fb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdedc0ceeb0 con 0x7fdedc069ed0 2023-10-06T05:15:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdedc0c3db0 0x7fdedc0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdedc0c3db0 0x7fdedc0c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37880/0 (socket says 172.21.15.45:37880) 2023-10-06T05:15:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 -- 172.21.15.45:0/3456182311 learned_addr learned my addr 172.21.15.45:0/3456182311 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee2698700 1 --2- 172.21.15.45:0/3456182311 >> v2:172.21.15.144:3300/0 conn(0x7fdedc069ed0 0x7fdedc0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 -- 172.21.15.45:0/3456182311 >> v2:172.21.15.144:3300/0 conn(0x7fdedc069ed0 msgr2=0x7fdedc0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 --2- 172.21.15.45:0/3456182311 >> v2:172.21.15.144:3300/0 conn(0x7fdedc069ed0 0x7fdedc0cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 -- 172.21.15.45:0/3456182311 >> v1:172.21.15.144:6789/0 conn(0x7fdedc0b0610 legacy=0x7fdedc0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 -- 172.21.15.45:0/3456182311 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc159880 con 0x7fdedc0c3db0 2023-10-06T05:15:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee1696700 1 --2- 172.21.15.45:0/3456182311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0c3db0 0x7fdedc0c85d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0886e0 tx=0x7fded0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b0402ed3cd02c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.008+0000 7fdee0e95700 1 -- 172.21.15.45:0/3456182311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000b820 con 0x7fdedc0c3db0 2023-10-06T05:15:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee0e95700 1 -- 172.21.15.45:0/3456182311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded000b980 con 0x7fdedc0c3db0 2023-10-06T05:15:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee0e95700 1 -- 172.21.15.45:0/3456182311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000bd20 con 0x7fdedc0c3db0 2023-10-06T05:15:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 -- 172.21.15.45:0/3456182311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0c3db0 msgr2=0x7fdedc0c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 --2- 172.21.15.45:0/3456182311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0c3db0 0x7fdedc0c85d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0886e0 tx=0x7fded0009250 comp rx=0 tx=0).stop 2023-10-06T05:15:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 -- 172.21.15.45:0/3456182311 shutdown_connections 2023-10-06T05:15:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 --2- 172.21.15.45:0/3456182311 >> v2:172.21.15.144:3300/0 conn(0x7fdedc069ed0 0x7fdedc0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 --2- 172.21.15.45:0/3456182311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0c3db0 0x7fdedc0c85d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 -- 172.21.15.45:0/3456182311 >> 172.21.15.45:0/3456182311 conn(0x7fdedc067e00 msgr2=0x7fdedc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:50.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 -- 172.21.15.45:0/3456182311 shutdown_connections 2023-10-06T05:15:50.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.009+0000 7fdee40fb700 1 -- 172.21.15.45:0/3456182311 wait complete. 2023-10-06T05:15:50.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.010+0000 7fdee40fb700 1 Processor -- start 2023-10-06T05:15:50.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.010+0000 7fdee40fb700 1 -- start start 2023-10-06T05:15:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee40fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdedc069ed0 0x7fdedc15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee40fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0b0610 0x7fdedc0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee40fb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdedc0c3db0 0x7fdedc0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee40fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc15c900 con 0x7fdedc0b0610 2023-10-06T05:15:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee40fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc15cbe0 con 0x7fdedc069ed0 2023-10-06T05:15:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee40fb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc15cec0 con 0x7fdedc0c3db0 2023-10-06T05:15:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee1696700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0b0610 0x7fdedc0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee1696700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0b0610 0x7fdedc0ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37884/0 (socket says 172.21.15.45:37884) 2023-10-06T05:15:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee1696700 1 -- 172.21.15.45:0/240252552 learned_addr learned my addr 172.21.15.45:0/240252552 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee1e97700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdedc069ed0 0x7fdedc15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.011+0000 7fdee2698700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdedc0c3db0 0x7fdedc0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee1696700 1 -- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdedc0c3db0 msgr2=0x7fdedc0c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee1696700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdedc0c3db0 0x7fdedc0c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee1696700 1 -- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdedc069ed0 msgr2=0x7fdedc15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee1696700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdedc069ed0 0x7fdedc15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee1696700 1 -- 172.21.15.45:0/240252552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00094f0 con 0x7fdedc0b0610 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee2698700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdedc0c3db0 0x7fdedc0c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee1696700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0b0610 0x7fdedc0ccb40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fded00094c0 tx=0x7fded0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdecdffb700 1 -- 172.21.15.45:0/240252552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000bd00 con 0x7fdedc0b0610 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdecdffb700 1 -- 172.21.15.45:0/240252552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0024440 con 0x7fdedc0b0610 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc15d1a0 con 0x7fdedc0b0610 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdecdffb700 1 -- 172.21.15.45:0/240252552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00247c0 con 0x7fdedc0b0610 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.012+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdedc15c4f0 con 0x7fdedc0b0610 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.013+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdedc15d6c0 con 0x7fdedc0b0610 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.014+0000 7fdecdffb700 1 -- 172.21.15.45:0/240252552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fded002d440 con 0x7fdedc0b0610 2023-10-06T05:15:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.014+0000 7fdecdffb700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdec0066f30 0x7fdec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.014+0000 7fdecdffb700 1 -- 172.21.15.45:0/240252552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fded0086390 con 0x7fdedc0b0610 2023-10-06T05:15:50.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.014+0000 7fdee1e97700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdec0066f30 0x7fdec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.015+0000 7fdee1e97700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdec0066f30 0x7fdec00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fded8007fd0 tx=0x7fded8007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.019+0000 7fdecdffb700 1 -- 172.21.15.45:0/240252552 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdedc15d6c0 con 0x7fdedc0b0610 2023-10-06T05:15:50.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.220+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdedc15d6c0 con 0x7fdedc0b0610 2023-10-06T05:15:50.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.220+0000 7fdecdffb700 1 -- 172.21.15.45:0/240252552 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdedc15d6c0 con 0x7fdedc0b0610 2023-10-06T05:15:50.222 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:15:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdec0066f30 msgr2=0x7fdec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdec0066f30 0x7fdec00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fded8007fd0 tx=0x7fded8007eb0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0b0610 msgr2=0x7fdedc0ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0b0610 0x7fdedc0ccb40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fded00094c0 tx=0x7fded0017000 comp rx=0 tx=0).stop 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 shutdown_connections 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdec0066f30 0x7fdec00693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdedc0c3db0 0x7fdedc0c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdedc069ed0 0x7fdedc15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 --2- 172.21.15.45:0/240252552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0b0610 0x7fdedc0ccb40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 >> 172.21.15.45:0/240252552 conn(0x7fdedc067e00 msgr2=0x7fdedc0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 shutdown_connections 2023-10-06T05:15:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.222+0000 7fdee40fb700 1 -- 172.21.15.45:0/240252552 wait complete. 2023-10-06T05:15:50.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.370+0000 7f59870ba700 1 Processor -- start 2023-10-06T05:15:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.370+0000 7f59870ba700 1 -- start start 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f59870ba700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f598013f900 0x7f59800cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f59870ba700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59800c9f20 0x7f59800affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f59870ba700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800c9630 con 0x7f59800c8de0 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f59870ba700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59800cb550 con 0x7f598013f900 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f59870ba700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800cb830 con 0x7f59800c9f20 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f5984e56700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f598013f900 0x7f59800cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f597ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59800c8de0 0x7f59800c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49726/0 (socket says 172.21.15.45:49726) 2023-10-06T05:15:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.371+0000 7f597ffff700 1 -- 172.21.15.45:0/1692882516 learned_addr learned my addr 172.21.15.45:0/1692882516 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f5985657700 1 --2- 172.21.15.45:0/1692882516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59800c9f20 0x7f59800affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f5984e56700 1 -- 172.21.15.45:0/1692882516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59800c9f20 msgr2=0x7f59800affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f5984e56700 1 --2- 172.21.15.45:0/1692882516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59800c9f20 0x7f59800affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f5984e56700 1 -- 172.21.15.45:0/1692882516 >> v1:172.21.15.45:6789/0 conn(0x7f59800c8de0 legacy=0x7f59800c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f597f7fe700 1 -- 172.21.15.45:0/1692882516 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481285930 0 0) 0x7f59800c9630 con 0x7f59800c8de0 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f5984e56700 1 -- 172.21.15.45:0/1692882516 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800cbb10 con 0x7f598013f900 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f5984e56700 1 --2- 172.21.15.45:0/1692882516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598013f900 0x7f59800cc840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59700098e0 tx=0x7f59700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16d868a35853b42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f597f7fe700 1 -- 172.21.15.45:0/1692882516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970009e80 con 0x7f598013f900 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f597f7fe700 1 -- 172.21.15.45:0/1692882516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597000b860 con 0x7f598013f900 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.372+0000 7f597f7fe700 1 -- 172.21.15.45:0/1692882516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597000bbe0 con 0x7f598013f900 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 -- 172.21.15.45:0/1692882516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598013f900 msgr2=0x7f59800cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 --2- 172.21.15.45:0/1692882516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598013f900 0x7f59800cc840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59700098e0 tx=0x7f59700091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 -- 172.21.15.45:0/1692882516 shutdown_connections 2023-10-06T05:15:50.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 --2- 172.21.15.45:0/1692882516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59800c9f20 0x7f59800affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 --2- 172.21.15.45:0/1692882516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598013f900 0x7f59800cc840 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 -- 172.21.15.45:0/1692882516 >> 172.21.15.45:0/1692882516 conn(0x7f59800691c0 msgr2=0x7f59800cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:50.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 -- 172.21.15.45:0/1692882516 shutdown_connections 2023-10-06T05:15:50.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.373+0000 7f59870ba700 1 -- 172.21.15.45:0/1692882516 wait complete. 2023-10-06T05:15:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.374+0000 7f59870ba700 1 Processor -- start 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.374+0000 7f59870ba700 1 -- start start 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f59870ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59800c8de0 0x7f5980133ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f59870ba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59800c9f20 0x7f5980138da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f59870ba700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f598013f900 0x7f598015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f59870ba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598013a350 con 0x7f59800c9f20 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f59870ba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980134020 con 0x7f59800c8de0 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f59870ba700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5980139740 con 0x7f598013f900 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f597ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59800c9f20 0x7f5980138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f597ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59800c9f20 0x7f5980138da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37896/0 (socket says 172.21.15.45:37896) 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f597ffff700 1 -- 172.21.15.45:0/3546099468 learned_addr learned my addr 172.21.15.45:0/3546099468 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f5985657700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f598013f900 0x7f598015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.375+0000 7f5984e56700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59800c8de0 0x7f5980133ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597ffff700 1 -- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f598013f900 msgr2=0x7f598015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597ffff700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f598013f900 0x7f598015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597ffff700 1 -- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59800c8de0 msgr2=0x7f5980133ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597ffff700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59800c8de0 0x7f5980133ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597ffff700 1 -- 172.21.15.45:0/3546099468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009420 con 0x7f59800c9f20 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f5984e56700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59800c8de0 0x7f5980133ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597ffff700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59800c9f20 0x7f5980138da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f596800c7e0 tx=0x7f5968007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597cff9700 1 -- 172.21.15.45:0/3546099468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968005b30 con 0x7f59800c9f20 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597cff9700 1 -- 172.21.15.45:0/3546099468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968005c90 con 0x7f59800c9f20 2023-10-06T05:15:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5980139a20 con 0x7f59800c9f20 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f597cff9700 1 -- 172.21.15.45:0/3546099468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968007300 con 0x7f59800c9f20 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.376+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5980139fe0 con 0x7f59800c9f20 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.377+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598013a500 con 0x7f59800c9f20 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.377+0000 7f597cff9700 1 -- 172.21.15.45:0/3546099468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f596801b400 con 0x7f59800c9f20 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.378+0000 7f597cff9700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5960066f30 0x7f59600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.378+0000 7f597cff9700 1 -- 172.21.15.45:0/3546099468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f5968084470 con 0x7f59800c9f20 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.378+0000 7f5984e56700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5960066f30 0x7f59600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.379+0000 7f5984e56700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5960066f30 0x7f59600693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5970003080 tx=0x7f5970017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.382+0000 7f597cff9700 1 -- 172.21.15.45:0/3546099468 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f598013a500 con 0x7f59800c9f20 2023-10-06T05:15:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.583+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f598013a490 con 0x7f59800c9f20 2023-10-06T05:15:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.583+0000 7f597cff9700 1 -- 172.21.15.45:0/3546099468 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59680512d0 con 0x7f59800c9f20 2023-10-06T05:15:50.585 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:50.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5960066f30 msgr2=0x7f59600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5960066f30 0x7f59600693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5970003080 tx=0x7f5970017000 comp rx=0 tx=0).stop 2023-10-06T05:15:50.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59800c9f20 msgr2=0x7f5980138da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59800c9f20 0x7f5980138da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f596800c7e0 tx=0x7f5968007df0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 shutdown_connections 2023-10-06T05:15:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5960066f30 0x7f59600693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f598013f900 0x7f598015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59800c8de0 0x7f5980133ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 --2- 172.21.15.45:0/3546099468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59800c9f20 0x7f5980138da0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 >> 172.21.15.45:0/3546099468 conn(0x7f59800691c0 msgr2=0x7f598013e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 shutdown_connections 2023-10-06T05:15:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.585+0000 7f59870ba700 1 -- 172.21.15.45:0/3546099468 wait complete. 2023-10-06T05:15:50.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.733+0000 7f31f7019700 1 Processor -- start 2023-10-06T05:15:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.733+0000 7f31f7019700 1 -- start start 2023-10-06T05:15:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.734+0000 7f31f7019700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.734+0000 7f31f7019700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f00c9940 con 0x7f31f01406e0 2023-10-06T05:15:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.734+0000 7f31f7019700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f006d230 con 0x7f31f00ad4e0 2023-10-06T05:15:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.734+0000 7f31f7019700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f006d510 con 0x7f31f0140ab0 2023-10-06T05:15:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.734+0000 7f31f4db5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f31f01406e0 0x7f31f006c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49742/0 (socket says 172.21.15.45:49742) 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.734+0000 7f31f4db5700 1 -- 172.21.15.45:0/1477433435 learned_addr learned my addr 172.21.15.45:0/1477433435 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31effff700 1 --2- 172.21.15.45:0/1477433435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31ef7fe700 1 -- 172.21.15.45:0/1477433435 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208903716 0 0) 0x7f31f00c9940 con 0x7f31f01406e0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31ef7fe700 1 -- 172.21.15.45:0/1477433435 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31e0003570 con 0x7f31f01406e0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31ef7fe700 1 -- 172.21.15.45:0/1477433435 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178080600 0 0) 0x7f31f006d230 con 0x7f31f00ad4e0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31ef7fe700 1 -- 172.21.15.45:0/1477433435 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f00c9940 con 0x7f31f00ad4e0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31effff700 1 -- 172.21.15.45:0/1477433435 >> v1:172.21.15.144:6789/0 conn(0x7f31f00ad4e0 legacy=0x7f31f00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31effff700 1 -- 172.21.15.45:0/1477433435 >> v1:172.21.15.45:6789/0 conn(0x7f31f01406e0 legacy=0x7f31f006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31effff700 1 -- 172.21.15.45:0/1477433435 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f006d7f0 con 0x7f31f0140ab0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31effff700 1 --2- 172.21.15.45:0/1477433435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00acfa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009910 tx=0x7f31dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb163f7076f0aed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31ef7fe700 1 -- 172.21.15.45:0/1477433435 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc011630 con 0x7f31f0140ab0 2023-10-06T05:15:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31ef7fe700 1 -- 172.21.15.45:0/1477433435 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc011790 con 0x7f31f0140ab0 2023-10-06T05:15:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.735+0000 7f31ef7fe700 1 -- 172.21.15.45:0/1477433435 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc011b10 con 0x7f31f0140ab0 2023-10-06T05:15:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.736+0000 7f31f7019700 1 -- 172.21.15.45:0/1477433435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 msgr2=0x7f31f00acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.736+0000 7f31f7019700 1 --2- 172.21.15.45:0/1477433435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00acfa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009910 tx=0x7f31dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.736+0000 7f31f7019700 1 -- 172.21.15.45:0/1477433435 shutdown_connections 2023-10-06T05:15:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.736+0000 7f31f7019700 1 --2- 172.21.15.45:0/1477433435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00acfa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.736+0000 7f31f7019700 1 -- 172.21.15.45:0/1477433435 >> 172.21.15.45:0/1477433435 conn(0x7f31f00691c0 msgr2=0x7f31f00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.736+0000 7f31f7019700 1 -- 172.21.15.45:0/1477433435 shutdown_connections 2023-10-06T05:15:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.736+0000 7f31f7019700 1 -- 172.21.15.45:0/1477433435 wait complete. 2023-10-06T05:15:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.737+0000 7f31f7019700 1 Processor -- start 2023-10-06T05:15:50.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31f7019700 1 -- start start 2023-10-06T05:15:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31f7019700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f00ad4e0 0x7f31f0156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31f7019700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f01406e0 0x7f31f00c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31f7019700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31f7019700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0157040 con 0x7f31f01406e0 2023-10-06T05:15:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31f7019700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f016b1b0 con 0x7f31f00ad4e0 2023-10-06T05:15:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31f7019700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f016b490 con 0x7f31f0140ab0 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31effff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f01406e0 0x7f31f00c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31effff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f01406e0 0x7f31f00c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37900/0 (socket says 172.21.15.45:37900) 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.738+0000 7f31effff700 1 -- 172.21.15.45:0/876331634 learned_addr learned my addr 172.21.15.45:0/876331634 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31f4db5700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f00ad4e0 0x7f31f0156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31f55b6700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31effff700 1 -- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 msgr2=0x7f31f00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31effff700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31effff700 1 -- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f00ad4e0 msgr2=0x7f31f0156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31effff700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f00ad4e0 0x7f31f0156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31effff700 1 -- 172.21.15.45:0/876331634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc009420 con 0x7f31f01406e0 2023-10-06T05:15:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31effff700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f01406e0 0x7f31f00c4460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0091e0 tx=0x7f31dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31ecff9700 1 -- 172.21.15.45:0/876331634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc000c90 con 0x7f31f01406e0 2023-10-06T05:15:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31ecff9700 1 -- 172.21.15.45:0/876331634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc011900 con 0x7f31f01406e0 2023-10-06T05:15:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f016b770 con 0x7f31f01406e0 2023-10-06T05:15:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31ecff9700 1 -- 172.21.15.45:0/876331634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc026660 con 0x7f31f01406e0 2023-10-06T05:15:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.739+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f016bd30 con 0x7f31f01406e0 2023-10-06T05:15:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.741+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f006e2d0 con 0x7f31f01406e0 2023-10-06T05:15:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.741+0000 7f31ecff9700 1 -- 172.21.15.45:0/876331634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31dc023030 con 0x7f31f01406e0 2023-10-06T05:15:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.741+0000 7f31ecff9700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.742+0000 7f31ecff9700 1 -- 172.21.15.45:0/876331634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f31dc086130 con 0x7f31f01406e0 2023-10-06T05:15:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.742+0000 7f31f4db5700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.742+0000 7f31f4db5700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f31e4007fd0 tx=0x7f31e4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.746+0000 7f31ecff9700 1 -- 172.21.15.45:0/876331634 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31dc052e70 con 0x7f31f01406e0 2023-10-06T05:15:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.949+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31f00acfa0 con 0x7f31f01406e0 2023-10-06T05:15:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.949+0000 7f31ecff9700 1 -- 172.21.15.45:0/876331634 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f31dc0150a0 con 0x7f31f01406e0 2023-10-06T05:15:50.951 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:15:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f31d4066f30 msgr2=0x7f31d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f31e4007fd0 tx=0x7f31e4009040 comp rx=0 tx=0).stop 2023-10-06T05:15:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f01406e0 msgr2=0x7f31f00c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f01406e0 0x7f31f00c4460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0091e0 tx=0x7f31dc017000 comp rx=0 tx=0).stop 2023-10-06T05:15:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 shutdown_connections 2023-10-06T05:15:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31f0140ab0 0x7f31f00c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f00ad4e0 0x7f31f0156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 --2- 172.21.15.45:0/876331634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f01406e0 0x7f31f00c4460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.951+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 >> 172.21.15.45:0/876331634 conn(0x7f31f00691c0 msgr2=0x7f31f00c1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.952+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 shutdown_connections 2023-10-06T05:15:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:50.952+0000 7f31f7019700 1 -- 172.21.15.45:0/876331634 wait complete. 2023-10-06T05:15:50.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T05:15:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.099+0000 7f300fb32700 1 Processor -- start 2023-10-06T05:15:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.099+0000 7f300fb32700 1 -- start start 2023-10-06T05:15:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.100+0000 7f300fb32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f30080acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.100+0000 7f300fb32700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f30080ad4e0 0x7f30080c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.100+0000 7f300fb32700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30080c9940 con 0x7f30081406e0 2023-10-06T05:15:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.100+0000 7f300fb32700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300806d230 con 0x7f30080ad4e0 2023-10-06T05:15:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.100+0000 7f300fb32700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300806d510 con 0x7f3008140ab0 2023-10-06T05:15:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300d0cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f30080acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f30080ad4e0 0x7f30080c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f30080ad4e0 0x7f30080c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:58302/0 (socket says 172.21.15.45:58302) 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 -- 172.21.15.45:0/3484155176 learned_addr learned my addr 172.21.15.45:0/3484155176 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 -- 172.21.15.45:0/3484155176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 msgr2=0x7f30080acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 --2- 172.21.15.45:0/3484155176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f30080acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 -- 172.21.15.45:0/3484155176 >> v1:172.21.15.144:6789/0 conn(0x7f30081406e0 legacy=0x7f300806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 -- 172.21.15.45:0/3484155176 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300806d7f0 con 0x7f30080ad4e0 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300c8cc700 1 -- 172.21.15.45:0/3484155176 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226958322 0 0) 0x7f30080c9940 con 0x7f30081406e0 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.101+0000 7f300e0cf700 1 --2- 172.21.15.45:0/3484155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f30080c9dd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3000009f20 tx=0x7f3000009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=994a046d2fab35b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300c8cc700 1 -- 172.21.15.45:0/3484155176 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300000e070 con 0x7f30080ad4e0 2023-10-06T05:15:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300c8cc700 1 -- 172.21.15.45:0/3484155176 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300000b9d0 con 0x7f30080ad4e0 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300c8cc700 1 -- 172.21.15.45:0/3484155176 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000092e0 con 0x7f30080ad4e0 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 -- 172.21.15.45:0/3484155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 msgr2=0x7f30080c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 --2- 172.21.15.45:0/3484155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f30080c9dd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3000009f20 tx=0x7f3000009840 comp rx=0 tx=0).stop 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 -- 172.21.15.45:0/3484155176 shutdown_connections 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 --2- 172.21.15.45:0/3484155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f30080c9dd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 --2- 172.21.15.45:0/3484155176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f30080acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 -- 172.21.15.45:0/3484155176 >> 172.21.15.45:0/3484155176 conn(0x7f30080691c0 msgr2=0x7f30080cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 -- 172.21.15.45:0/3484155176 shutdown_connections 2023-10-06T05:15:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.102+0000 7f300fb32700 1 -- 172.21.15.45:0/3484155176 wait complete. 2023-10-06T05:15:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.103+0000 7f300fb32700 1 Processor -- start 2023-10-06T05:15:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.103+0000 7f300fb32700 1 -- start start 2023-10-06T05:15:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300fb32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f3008156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300fb32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30081406e0 0x7f30080c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300fb32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f300815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300fb32700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30080c41a0 con 0x7f30081406e0 2023-10-06T05:15:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300fb32700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30080c5ed0 con 0x7f30080ad4e0 2023-10-06T05:15:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300fb32700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30080c61b0 con 0x7f3008140ab0 2023-10-06T05:15:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300d0cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30081406e0 0x7f30080c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300e0cf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f300815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300d8ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f3008156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300e0cf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f300815bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:35938/0 (socket says 172.21.15.45:35938) 2023-10-06T05:15:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300d0cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30081406e0 0x7f30080c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37902/0 (socket says 172.21.15.45:37902) 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.104+0000 7f300e0cf700 1 -- 172.21.15.45:0/1806724542 learned_addr learned my addr 172.21.15.45:0/1806724542 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300d0cd700 1 -- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 msgr2=0x7f300815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300d0cd700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f300815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300d0cd700 1 -- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 msgr2=0x7f3008156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300d0cd700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f3008156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300d0cd700 1 -- 172.21.15.45:0/1806724542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009ae0 con 0x7f30081406e0 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300d8ce700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f3008156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300d0cd700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30081406e0 0x7f30080c5660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f300400e900 tx=0x7f30040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f2ffdffb700 1 -- 172.21.15.45:0/1806724542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004013680 con 0x7f30081406e0 2023-10-06T05:15:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f2ffdffb700 1 -- 172.21.15.45:0/1806724542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040137e0 con 0x7f30081406e0 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30080c6490 con 0x7f30081406e0 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f2ffdffb700 1 -- 172.21.15.45:0/1806724542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004013b80 con 0x7f30081406e0 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.105+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30080c5ba0 con 0x7f30081406e0 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.106+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30080b0c40 con 0x7f30081406e0 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.107+0000 7f2ffdffb700 1 -- 172.21.15.45:0/1806724542 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30040123d0 con 0x7f30081406e0 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.107+0000 7f2ffdffb700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fec066f30 0x7f2fec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.107+0000 7f2ffdffb700 1 -- 172.21.15.45:0/1806724542 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f3004084a70 con 0x7f30081406e0 2023-10-06T05:15:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.108+0000 7f300d8ce700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fec066f30 0x7f2fec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.108+0000 7f300d8ce700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fec066f30 0x7f2fec0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8007fd0 tx=0x7f2ff8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.112+0000 7f2ffdffb700 1 -- 172.21.15.45:0/1806724542 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30080b0c40 con 0x7f30081406e0 2023-10-06T05:15:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.312+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f30080b0c40 con 0x7f30081406e0 2023-10-06T05:15:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.313+0000 7f2ffdffb700 1 -- 172.21.15.45:0/1806724542 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f30080b0c40 con 0x7f30081406e0 2023-10-06T05:15:51.315 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fec066f30 msgr2=0x7f2fec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fec066f30 0x7f2fec0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8007fd0 tx=0x7f2ff8009040 comp rx=0 tx=0).stop 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30081406e0 msgr2=0x7f30080c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30081406e0 0x7f30080c5660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f300400e900 tx=0x7f30040091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 shutdown_connections 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2fec066f30 0x7f2fec0693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3008140ab0 0x7f300815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30080ad4e0 0x7f3008156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 --2- 172.21.15.45:0/1806724542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30081406e0 0x7f30080c5660 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 >> 172.21.15.45:0/1806724542 conn(0x7f30080691c0 msgr2=0x7f30080ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 shutdown_connections 2023-10-06T05:15:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.315+0000 7f300fb32700 1 -- 172.21.15.45:0/1806724542 wait complete. 2023-10-06T05:15:51.370 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2023-10-06T05:15:51.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2023-10-06T05:15:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.503+0000 7f3761b5a700 1 Processor -- start 2023-10-06T05:15:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.503+0000 7f3761b5a700 1 -- start start 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f3761b5a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f375c156ce0 0x7f375c1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f3761b5a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f3761b5a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c06bf70 con 0x7f375c06bad0 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f3761b5a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c0b02f0 con 0x7f375c156ce0 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f3761b5a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c15b380 con 0x7f375c15ac10 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f375b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f375c156ce0 0x7f375c1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f375affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f375c06bad0 0x7f375c0afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49744/0 (socket says 172.21.15.45:49744) 2023-10-06T05:15:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f375affd700 1 -- 172.21.15.45:0/2643834347 learned_addr learned my addr 172.21.15.45:0/2643834347 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f375bfff700 1 --2- 172.21.15.45:0/2643834347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.504+0000 7f375a7fc700 1 -- 172.21.15.45:0/2643834347 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341725285 0 0) 0x7f375c06bf70 con 0x7f375c06bad0 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375a7fc700 1 -- 172.21.15.45:0/2643834347 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3748003570 con 0x7f375c06bad0 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375b7fe700 1 -- 172.21.15.45:0/2643834347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 msgr2=0x7f375c0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375b7fe700 1 --2- 172.21.15.45:0/2643834347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c0ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375b7fe700 1 -- 172.21.15.45:0/2643834347 >> v1:172.21.15.45:6789/0 conn(0x7f375c06bad0 legacy=0x7f375c0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375b7fe700 1 -- 172.21.15.45:0/2643834347 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c15b660 con 0x7f375c156ce0 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375bfff700 1 --2- 172.21.15.45:0/2643834347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375b7fe700 1 --2- 172.21.15.45:0/2643834347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c156ce0 0x7f375c1570b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f374c0098e0 tx=0x7f374c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d2ed37a85eb0da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375a7fc700 1 -- 172.21.15.45:0/2643834347 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c011630 con 0x7f375c156ce0 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375a7fc700 1 -- 172.21.15.45:0/2643834347 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c011790 con 0x7f375c156ce0 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f375a7fc700 1 -- 172.21.15.45:0/2643834347 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c011b30 con 0x7f375c156ce0 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f3761b5a700 1 -- 172.21.15.45:0/2643834347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c156ce0 msgr2=0x7f375c1570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.505+0000 7f3761b5a700 1 --2- 172.21.15.45:0/2643834347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c156ce0 0x7f375c1570b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f374c0098e0 tx=0x7f374c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.506+0000 7f3761b5a700 1 -- 172.21.15.45:0/2643834347 shutdown_connections 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.506+0000 7f3761b5a700 1 --2- 172.21.15.45:0/2643834347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.506+0000 7f3761b5a700 1 --2- 172.21.15.45:0/2643834347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c156ce0 0x7f375c1570b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.506+0000 7f3761b5a700 1 -- 172.21.15.45:0/2643834347 >> 172.21.15.45:0/2643834347 conn(0x7f375c06f090 msgr2=0x7f375c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.506+0000 7f3761b5a700 1 -- 172.21.15.45:0/2643834347 shutdown_connections 2023-10-06T05:15:51.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.506+0000 7f3761b5a700 1 -- 172.21.15.45:0/2643834347 wait complete. 2023-10-06T05:15:51.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.507+0000 7f3761b5a700 1 Processor -- start 2023-10-06T05:15:51.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.507+0000 7f3761b5a700 1 -- start start 2023-10-06T05:15:51.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f3761b5a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06bad0 0x7f375c15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f3761b5a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f375c156ce0 0x7f375c1697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f3761b5a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f3761b5a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c16a290 con 0x7f375c06bad0 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f3761b5a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c16a570 con 0x7f375c156ce0 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f3761b5a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c157640 con 0x7f375c15ac10 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06bad0 0x7f375c15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06bad0 0x7f375c15a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37920/0 (socket says 172.21.15.45:37920) 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 -- 172.21.15.45:0/657115021 learned_addr learned my addr 172.21.15.45:0/657115021 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:51.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375bfff700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375affd700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f375c156ce0 0x7f375c1697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 -- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 msgr2=0x7f375c154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 -- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f375c156ce0 msgr2=0x7f375c1697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f375c156ce0 0x7f375c1697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 -- 172.21.15.45:0/657115021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3754009a20 con 0x7f375c06bad0 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.508+0000 7f375b7fe700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06bad0 0x7f375c15a300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f375c06aed0 tx=0x7f374c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.509+0000 7f37537fe700 1 -- 172.21.15.45:0/657115021 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c011b10 con 0x7f375c06bad0 2023-10-06T05:15:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.509+0000 7f37537fe700 1 -- 172.21.15.45:0/657115021 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c0253f0 con 0x7f375c06bad0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.509+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c009420 con 0x7f375c06bad0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.509+0000 7f37537fe700 1 -- 172.21.15.45:0/657115021 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0257b0 con 0x7f375c06bad0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.509+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c157c00 con 0x7f375c06bad0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.510+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c0696a0 con 0x7f375c06bad0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.510+0000 7f37537fe700 1 -- 172.21.15.45:0/657115021 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f374c025ba0 con 0x7f375c06bad0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.511+0000 7f37537fe700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f373c066f30 0x7f373c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.511+0000 7f37537fe700 1 -- 172.21.15.45:0/657115021 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f374c086300 con 0x7f375c06bad0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.511+0000 7f375affd700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f373c066f30 0x7f373c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:51.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.511+0000 7f375affd700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f373c066f30 0x7f373c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f375c0ca810 tx=0x7f3744009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:51.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.515+0000 7f37537fe700 1 -- 172.21.15.45:0/657115021 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f374c0530e0 con 0x7f375c06bad0 2023-10-06T05:15:51.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.799+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) v1 -- 0x7f375c15b940 con 0x7f375c06bad0 2023-10-06T05:15:51.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.899+0000 7f37537fe700 1 -- 172.21.15.45:0/657115021 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v62) v1 ==== 137+0+0 (secure 0 0 0) 0x7f374c056700 con 0x7f375c06bad0 2023-10-06T05:15:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f373c066f30 msgr2=0x7f373c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f373c066f30 0x7f373c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f375c0ca810 tx=0x7f3744009040 comp rx=0 tx=0).stop 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06bad0 msgr2=0x7f375c15a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06bad0 0x7f375c15a300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f375c06aed0 tx=0x7f374c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 shutdown_connections 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f373c066f30 0x7f373c0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f375c15ac10 0x7f375c154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06bad0 0x7f375c15a300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 --2- 172.21.15.45:0/657115021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f375c156ce0 0x7f375c1697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 >> 172.21.15.45:0/657115021 conn(0x7f375c06f090 msgr2=0x7f375c0cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 shutdown_connections 2023-10-06T05:15:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:51.901+0000 7f3761b5a700 1 -- 172.21.15.45:0/657115021 wait complete. 2023-10-06T05:15:51.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:15:52.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.056+0000 7f70d1a1b700 1 Processor -- start 2023-10-06T05:15:52.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.056+0000 7f70d1a1b700 1 -- start start 2023-10-06T05:15:52.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70d1a1b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70cc0d02b0 0x7f70cc0d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70d1a1b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f70cc15c8c0 0x7f70cc0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70d1a1b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70cc0cd740 con 0x7f70cc15d820 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70d1a1b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70cc0ccb30 con 0x7f70cc0d02b0 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70d1a1b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70cc0cce10 con 0x7f70cc15c8c0 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70caffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70cc0d02b0 0x7f70cc0d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70caffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70cc0d02b0 0x7f70cc0d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37922/0 (socket says 172.21.15.45:37922) 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70caffd700 1 -- 172.21.15.45:0/1021251883 learned_addr learned my addr 172.21.15.45:0/1021251883 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70cb7fe700 1 --2- 172.21.15.45:0/1021251883 >> v2:172.21.15.144:3300/0 conn(0x7f70cc15c8c0 0x7f70cc0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70caffd700 1 -- 172.21.15.45:0/1021251883 >> v2:172.21.15.144:3300/0 conn(0x7f70cc15c8c0 msgr2=0x7f70cc0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70caffd700 1 --2- 172.21.15.45:0/1021251883 >> v2:172.21.15.144:3300/0 conn(0x7f70cc15c8c0 0x7f70cc0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70caffd700 1 -- 172.21.15.45:0/1021251883 >> v1:172.21.15.144:6789/0 conn(0x7f70cc15d820 legacy=0x7f70cc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:52.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.057+0000 7f70caffd700 1 -- 172.21.15.45:0/1021251883 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc0cd0f0 con 0x7f70cc0d02b0 2023-10-06T05:15:52.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70caffd700 1 --2- 172.21.15.45:0/1021251883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc0d0680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0b0e00 tx=0x7f70bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91f92f98b58e34a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70c9ffb700 1 -- 172.21.15.45:0/1021251883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00b820 con 0x7f70cc0d02b0 2023-10-06T05:15:52.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70c9ffb700 1 -- 172.21.15.45:0/1021251883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc00b980 con 0x7f70cc0d02b0 2023-10-06T05:15:52.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70c9ffb700 1 -- 172.21.15.45:0/1021251883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00bd20 con 0x7f70cc0d02b0 2023-10-06T05:15:52.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1021251883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 msgr2=0x7f70cc0d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1021251883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc0d0680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0b0e00 tx=0x7f70bc009250 comp rx=0 tx=0).stop 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1021251883 shutdown_connections 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1021251883 >> v2:172.21.15.144:3300/0 conn(0x7f70cc15c8c0 0x7f70cc0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1021251883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc0d0680 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1021251883 >> 172.21.15.45:0/1021251883 conn(0x7f70cc06ed50 msgr2=0x7f70cc06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1021251883 shutdown_connections 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.058+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1021251883 wait complete. 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.060+0000 7f70d1a1b700 1 Processor -- start 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.060+0000 7f70d1a1b700 1 -- start start 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70d1a1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70d1a1b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70cc15c8c0 0x7f70cc169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70d1a1b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70cc15d820 0x7f70cc0c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70d1a1b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0a7930 con 0x7f70cc0d02b0 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70d1a1b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0a6d20 con 0x7f70cc15d820 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70d1a1b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0a7000 con 0x7f70cc15c8c0 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37928/0 (socket says 172.21.15.45:37928) 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 -- 172.21.15.45:0/1208748639 learned_addr learned my addr 172.21.15.45:0/1208748639 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70cb7fe700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70cc15d820 0x7f70cc0c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70ca7fc700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70cc15c8c0 0x7f70cc169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 -- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70cc15c8c0 msgr2=0x7f70cc169870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70cc15c8c0 0x7f70cc169870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 -- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70cc15d820 msgr2=0x7f70cc0c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70cc15d820 0x7f70cc0c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 -- 172.21.15.45:0/1208748639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc0094f0 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70caffd700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc159dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0030c0 tx=0x7f70bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.061+0000 7f70b2ffd700 1 -- 172.21.15.45:0/1208748639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc011e60 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.062+0000 7f70b2ffd700 1 -- 172.21.15.45:0/1208748639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc01b680 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.062+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70cc0a72e0 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.062+0000 7f70b2ffd700 1 -- 172.21.15.45:0/1208748639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc01ba20 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.062+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc0a69b0 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.063+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc088730 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.063+0000 7f70b2ffd700 1 -- 172.21.15.45:0/1208748639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70bc01c450 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.064+0000 7f70b2ffd700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.064+0000 7f70b2ffd700 1 -- 172.21.15.45:0/1208748639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f70bc086720 con 0x7f70cc0d02b0 2023-10-06T05:15:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.064+0000 7f70ca7fc700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.065+0000 7f70ca7fc700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70b8066f30 0x7f70b80693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70c0006fd0 tx=0x7f70c0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.068+0000 7f70b2ffd700 1 -- 172.21.15.45:0/1208748639 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70bc053500 con 0x7f70cc0d02b0 2023-10-06T05:15:52.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.271+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f70cc0cd3d0 con 0x7f70cc0d02b0 2023-10-06T05:15:52.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.275+0000 7f70b2ffd700 1 -- 172.21.15.45:0/1208748639 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70bc056b20 con 0x7f70cc0d02b0 2023-10-06T05:15:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70b8066f30 msgr2=0x7f70b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70b8066f30 0x7f70b80693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70c0006fd0 tx=0x7f70c0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 msgr2=0x7f70cc159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc159dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0030c0 tx=0x7f70bc016000 comp rx=0 tx=0).stop 2023-10-06T05:15:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 shutdown_connections 2023-10-06T05:15:52.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70cc15d820 0x7f70cc0c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70cc0d02b0 0x7f70cc159dc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 --2- 172.21.15.45:0/1208748639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70cc15c8c0 0x7f70cc169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 >> 172.21.15.45:0/1208748639 conn(0x7f70cc06ed50 msgr2=0x7f70cc070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 shutdown_connections 2023-10-06T05:15:52.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.277+0000 7f70d1a1b700 1 -- 172.21.15.45:0/1208748639 wait complete. 2023-10-06T05:15:52.292 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:15:52.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:52.333 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:15:52.381 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:15:52.381 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:15:52.381 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:15:52.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:15:52.429 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:15:52.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:15:52.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:52.429 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:15:52.492 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi144.front.sepia.ceph.com... 2023-10-06T05:15:52.492 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:15:52.492 DEBUG:teuthology.orchestra.run.smithi144:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T05:15:52.533 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:15:52.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:15:52.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:52.534 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:15:52.601 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:15:52.601 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:15:52.601 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-06T05:15:52.648 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:15:52.648 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:15:52.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:52.648 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:15:52.712 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi144.front.sepia.ceph.com... 2023-10-06T05:15:52.712 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:15:52.712 DEBUG:teuthology.orchestra.run.smithi144:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-06T05:15:52.755 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:15:52.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:15:52.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:15:52.756 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:15:52.816 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:15:52.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:52.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.954+0000 7f4589942700 1 Processor -- start 2023-10-06T05:15:52.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.954+0000 7f4589942700 1 -- start start 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f4589942700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4584159170 0x7f4584159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f4589942700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f45840692d0 0x7f45840aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f4589942700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45840ca6d0 con 0x7f458415d040 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f4589942700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458415d550 con 0x7f4584159170 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f4589942700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458415d810 con 0x7f45840692d0 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f4582ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4584159170 0x7f4584159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f45827fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f458415d040 0x7f4584068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49760/0 (socket says 172.21.15.45:49760) 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f4582ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4584159170 0x7f4584159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37938/0 (socket says 172.21.15.45:37938) 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f45827fc700 1 -- 172.21.15.45:0/995107492 learned_addr learned my addr 172.21.15.45:0/995107492 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.955+0000 7f45837fe700 1 --2- 172.21.15.45:0/995107492 >> v2:172.21.15.144:3300/0 conn(0x7f45840692d0 0x7f45840aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4582ffd700 1 -- 172.21.15.45:0/995107492 >> v2:172.21.15.144:3300/0 conn(0x7f45840692d0 msgr2=0x7f45840aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4582ffd700 1 --2- 172.21.15.45:0/995107492 >> v2:172.21.15.144:3300/0 conn(0x7f45840692d0 0x7f45840aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4582ffd700 1 -- 172.21.15.45:0/995107492 >> v1:172.21.15.45:6789/0 conn(0x7f458415d040 legacy=0x7f4584068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4582ffd700 1 -- 172.21.15.45:0/995107492 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458415daf0 con 0x7f4584159170 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4581ffb700 1 -- 172.21.15.45:0/995107492 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352485217 0 0) 0x7f45840ca6d0 con 0x7f458415d040 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4582ffd700 1 --2- 172.21.15.45:0/995107492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4584159170 0x7f4584159540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45840c70c0 tx=0x7f45740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52051731d058a52f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4581ffb700 1 -- 172.21.15.45:0/995107492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574011630 con 0x7f4584159170 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4581ffb700 1 -- 172.21.15.45:0/995107492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4574011790 con 0x7f4584159170 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4581ffb700 1 -- 172.21.15.45:0/995107492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574011b30 con 0x7f4584159170 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4589942700 1 -- 172.21.15.45:0/995107492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4584159170 msgr2=0x7f4584159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.956+0000 7f4589942700 1 --2- 172.21.15.45:0/995107492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4584159170 0x7f4584159540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45840c70c0 tx=0x7f45740091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.957+0000 7f4589942700 1 -- 172.21.15.45:0/995107492 shutdown_connections 2023-10-06T05:15:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.957+0000 7f4589942700 1 --2- 172.21.15.45:0/995107492 >> v2:172.21.15.144:3300/0 conn(0x7f45840692d0 0x7f45840aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.957+0000 7f4589942700 1 --2- 172.21.15.45:0/995107492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4584159170 0x7f4584159540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.957+0000 7f4589942700 1 -- 172.21.15.45:0/995107492 >> 172.21.15.45:0/995107492 conn(0x7f458406f7c0 msgr2=0x7f45840d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.957+0000 7f4589942700 1 -- 172.21.15.45:0/995107492 shutdown_connections 2023-10-06T05:15:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.957+0000 7f4589942700 1 -- 172.21.15.45:0/995107492 wait complete. 2023-10-06T05:15:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.958+0000 7f4589942700 1 Processor -- start 2023-10-06T05:15:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.958+0000 7f4589942700 1 -- start start 2023-10-06T05:15:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.959+0000 7f4589942700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45840692d0 0x7f45840c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.959+0000 7f4589942700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4584159170 0x7f45840a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.959+0000 7f4589942700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f458415d040 0x7f45840cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.959+0000 7f4589942700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584169a90 con 0x7f458415d040 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.959+0000 7f4589942700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584169d70 con 0x7f4584159170 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.959+0000 7f4589942700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458416a050 con 0x7f45840692d0 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f458415d040 0x7f45840cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f458415d040 0x7f45840cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37944/0 (socket says 172.21.15.45:37944) 2023-10-06T05:15:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 -- 172.21.15.45:0/2863293301 learned_addr learned my addr 172.21.15.45:0/2863293301 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45827fc700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4584159170 0x7f45840a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f4582ffd700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45840692d0 0x7f45840c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 -- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45840692d0 msgr2=0x7f45840c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45840692d0 0x7f45840c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 -- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4584159170 msgr2=0x7f45840a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4584159170 0x7f45840a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 -- 172.21.15.45:0/2863293301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4574009420 con 0x7f458415d040 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f45837fe700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f458415d040 0x7f45840cbb30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f457c009e90 tx=0x7f457c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f457affd700 1 -- 172.21.15.45:0/2863293301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c00cb70 con 0x7f458415d040 2023-10-06T05:15:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f457affd700 1 -- 172.21.15.45:0/2863293301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c00f040 con 0x7f458415d040 2023-10-06T05:15:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458416a330 con 0x7f458415d040 2023-10-06T05:15:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f457affd700 1 -- 172.21.15.45:0/2863293301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c014480 con 0x7f458415d040 2023-10-06T05:15:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.960+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4584169740 con 0x7f458415d040 2023-10-06T05:15:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.962+0000 7f457affd700 1 -- 172.21.15.45:0/2863293301 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f457c014830 con 0x7f458415d040 2023-10-06T05:15:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.962+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458416cd20 con 0x7f458415d040 2023-10-06T05:15:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.962+0000 7f457affd700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4564066f30 0x7f45640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.962+0000 7f457affd700 1 -- 172.21.15.45:0/2863293301 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f457c085670 con 0x7f458415d040 2023-10-06T05:15:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.962+0000 7f4582ffd700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4564066f30 0x7f45640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.963+0000 7f4582ffd700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4564066f30 0x7f45640693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45740093f0 tx=0x7f4574017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:52.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:52.967+0000 7f457affd700 1 -- 172.21.15.45:0/2863293301 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f458416cd20 con 0x7f458415d040 2023-10-06T05:15:53.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.214+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f458416cd20 con 0x7f458415d040 2023-10-06T05:15:53.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.215+0000 7f457affd700 1 -- 172.21.15.45:0/2863293301 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f458416cd20 con 0x7f458415d040 2023-10-06T05:15:53.217 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:53.217 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4564066f30 msgr2=0x7f45640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4564066f30 0x7f45640693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45740093f0 tx=0x7f4574017000 comp rx=0 tx=0).stop 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f458415d040 msgr2=0x7f45840cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f458415d040 0x7f45840cbb30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f457c009e90 tx=0x7f457c00b040 comp rx=0 tx=0).stop 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 shutdown_connections 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4564066f30 0x7f45640693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f458415d040 0x7f45840cbb30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45840692d0 0x7f45840c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 --2- 172.21.15.45:0/2863293301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4584159170 0x7f45840a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 >> 172.21.15.45:0/2863293301 conn(0x7f458406f7c0 msgr2=0x7f45840b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.217+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 shutdown_connections 2023-10-06T05:15:53.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.218+0000 7f4589942700 1 -- 172.21.15.45:0/2863293301 wait complete. 2023-10-06T05:15:53.221 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:53.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:53.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.365+0000 7f3c41430700 1 Processor -- start 2023-10-06T05:15:53.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.365+0000 7f3c41430700 1 -- start start 2023-10-06T05:15:53.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.366+0000 7f3c41430700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3c3c15d040 0x7f3c3c159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.366+0000 7f3c41430700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c0ca6d0 con 0x7f3c3c159170 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.366+0000 7f3c41430700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c15d550 con 0x7f3c3c0692d0 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.366+0000 7f3c41430700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c15d810 con 0x7f3c3c15d040 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.366+0000 7f3c3affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c3c159170 0x7f3c3c068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49762/0 (socket says 172.21.15.45:49762) 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.366+0000 7f3c3affd700 1 -- 172.21.15.45:0/3969857314 learned_addr learned my addr 172.21.15.45:0/3969857314 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.366+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/3969857314 >> v2:172.21.15.144:3300/0 conn(0x7f3c3c15d040 0x7f3c3c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c39ffb700 1 -- 172.21.15.45:0/3969857314 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682419869 0 0) 0x7f3c3c0ca6d0 con 0x7f3c3c159170 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c39ffb700 1 -- 172.21.15.45:0/3969857314 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c28003570 con 0x7f3c3c159170 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c3b7fe700 1 -- 172.21.15.45:0/3969857314 >> v1:172.21.15.144:6789/0 conn(0x7f3c3c0692d0 legacy=0x7f3c3c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c3b7fe700 1 -- 172.21.15.45:0/3969857314 >> v1:172.21.15.45:6789/0 conn(0x7f3c3c159170 legacy=0x7f3c3c068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c39ffb700 1 -- 172.21.15.45:0/3969857314 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516897506 0 0) 0x7f3c3c15d550 con 0x7f3c3c0692d0 2023-10-06T05:15:53.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c3b7fe700 1 -- 172.21.15.45:0/3969857314 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c15daf0 con 0x7f3c3c15d040 2023-10-06T05:15:53.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/3969857314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c15d040 0x7f3c3c159540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009e60 tx=0x7f3c34009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354e2c5de94da726 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c39ffb700 1 -- 172.21.15.45:0/3969857314 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400e070 con 0x7f3c3c15d040 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c39ffb700 1 -- 172.21.15.45:0/3969857314 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3400bb60 con 0x7f3c3c15d040 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.367+0000 7f3c39ffb700 1 -- 172.21.15.45:0/3969857314 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34012430 con 0x7f3c3c15d040 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.368+0000 7f3c41430700 1 -- 172.21.15.45:0/3969857314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c15d040 msgr2=0x7f3c3c159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.368+0000 7f3c41430700 1 --2- 172.21.15.45:0/3969857314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c15d040 0x7f3c3c159540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009e60 tx=0x7f3c34009780 comp rx=0 tx=0).stop 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.368+0000 7f3c41430700 1 -- 172.21.15.45:0/3969857314 shutdown_connections 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.368+0000 7f3c41430700 1 --2- 172.21.15.45:0/3969857314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c15d040 0x7f3c3c159540 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.368+0000 7f3c41430700 1 -- 172.21.15.45:0/3969857314 >> 172.21.15.45:0/3969857314 conn(0x7f3c3c06f7c0 msgr2=0x7f3c3c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.368+0000 7f3c41430700 1 -- 172.21.15.45:0/3969857314 shutdown_connections 2023-10-06T05:15:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.368+0000 7f3c41430700 1 -- 172.21.15.45:0/3969857314 wait complete. 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.370+0000 7f3c41430700 1 Processor -- start 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.370+0000 7f3c41430700 1 -- start start 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c41430700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c0692d0 0x7f3c3c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c41430700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c159170 0x7f3c3c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c41430700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c3c15d040 0x7f3c3c0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c41430700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c169530 con 0x7f3c3c159170 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c41430700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0ca570 con 0x7f3c3c0692d0 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c41430700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c1697a0 con 0x7f3c3c15d040 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c159170 0x7f3c3c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c159170 0x7f3c3c0c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37954/0 (socket says 172.21.15.45:37954) 2023-10-06T05:15:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 -- 172.21.15.45:0/3701390022 learned_addr learned my addr 172.21.15.45:0/3701390022 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3affd700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c0692d0 0x7f3c3c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3b7fe700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c3c15d040 0x7f3c3c0cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 -- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c3c15d040 msgr2=0x7f3c3c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c3c15d040 0x7f3c3c0cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 -- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c0692d0 msgr2=0x7f3c3c1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c0692d0 0x7f3c3c1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 -- 172.21.15.45:0/3701390022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34009a20 con 0x7f3c3c159170 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3affd700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c0692d0 0x7f3c3c1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.371+0000 7f3c3a7fc700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c159170 0x7f3c3c0c91f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c7e0 tx=0x7f3c2c007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.372+0000 7f3c32ffd700 1 -- 172.21.15.45:0/3701390022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011590 con 0x7f3c3c159170 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.372+0000 7f3c32ffd700 1 -- 172.21.15.45:0/3701390022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c0116f0 con 0x7f3c3c159170 2023-10-06T05:15:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.372+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c0c9cd0 con 0x7f3c3c159170 2023-10-06T05:15:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.372+0000 7f3c32ffd700 1 -- 172.21.15.45:0/3701390022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011ab0 con 0x7f3c3c159170 2023-10-06T05:15:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.372+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c0ca310 con 0x7f3c3c159170 2023-10-06T05:15:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.373+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c063b50 con 0x7f3c3c159170 2023-10-06T05:15:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.373+0000 7f3c32ffd700 1 -- 172.21.15.45:0/3701390022 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c2c01b3b0 con 0x7f3c3c159170 2023-10-06T05:15:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.374+0000 7f3c32ffd700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c1c066f30 0x7f3c1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.374+0000 7f3c32ffd700 1 -- 172.21.15.45:0/3701390022 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f3c2c083f90 con 0x7f3c3c159170 2023-10-06T05:15:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.374+0000 7f3c3affd700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c1c066f30 0x7f3c1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.375+0000 7f3c3affd700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c1c066f30 0x7f3c1c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c240058d0 tx=0x7f3c24009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.378+0000 7f3c32ffd700 1 -- 172.21.15.45:0/3701390022 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c2c050d70 con 0x7f3c3c159170 2023-10-06T05:15:53.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.587+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c3c1698e0 con 0x7f3c3c159170 2023-10-06T05:15:53.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.588+0000 7f3c32ffd700 1 -- 172.21.15.45:0/3701390022 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18899 (secure 0 0 0) 0x7f3c2c017070 con 0x7f3c3c159170 2023-10-06T05:15:53.590 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:53.591 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:47.878034+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.590+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c1c066f30 msgr2=0x7f3c1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.590+0000 7f3c41430700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c1c066f30 0x7f3c1c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c240058d0 tx=0x7f3c24009040 comp rx=0 tx=0).stop 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.590+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c159170 msgr2=0x7f3c3c0c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.590+0000 7f3c41430700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c159170 0x7f3c3c0c91f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c7e0 tx=0x7f3c2c007dc0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 shutdown_connections 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c1c066f30 0x7f3c1c0693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c3c15d040 0x7f3c3c0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3c0692d0 0x7f3c3c1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 --2- 172.21.15.45:0/3701390022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c159170 0x7f3c3c0c91f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 >> 172.21.15.45:0/3701390022 conn(0x7f3c3c06f7c0 msgr2=0x7f3c3c0aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 shutdown_connections 2023-10-06T05:15:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.591+0000 7f3c41430700 1 -- 172.21.15.45:0/3701390022 wait complete. 2023-10-06T05:15:53.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:14:57.253547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:14:58.338382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T05:15:01.988547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:15:53.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:15:53.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.740+0000 7facc8e20700 1 Processor -- start 2023-10-06T05:15:53.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.740+0000 7facc8e20700 1 -- start start 2023-10-06T05:15:53.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.741+0000 7facc8e20700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7facc40ada80 0x7facc4163e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.741+0000 7facc8e20700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc4067d80 con 0x7facc406aa70 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.741+0000 7facc8e20700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc40c75b0 con 0x7facc406ae40 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.741+0000 7facc8e20700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc40ac620 con 0x7facc40ada80 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.741+0000 7facc259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7facc406aa70 0x7facc40c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49766/0 (socket says 172.21.15.45:49766) 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.741+0000 7facc259c700 1 -- 172.21.15.45:0/2576942262 learned_addr learned my addr 172.21.15.45:0/2576942262 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.741+0000 7facc2d9d700 1 --2- 172.21.15.45:0/2576942262 >> v2:172.21.15.144:3300/0 conn(0x7facc40ada80 0x7facc4163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc159a700 1 -- 172.21.15.45:0/2576942262 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822296061 0 0) 0x7facc4067d80 con 0x7facc406aa70 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc159a700 1 -- 172.21.15.45:0/2576942262 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb0003570 con 0x7facc406aa70 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc2d9d700 1 -- 172.21.15.45:0/2576942262 >> v1:172.21.15.144:6789/0 conn(0x7facc406ae40 legacy=0x7facc415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc159a700 1 -- 172.21.15.45:0/2576942262 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144944793 0 0) 0x7facc40c75b0 con 0x7facc406ae40 2023-10-06T05:15:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc2d9d700 1 -- 172.21.15.45:0/2576942262 >> v1:172.21.15.45:6789/0 conn(0x7facc406aa70 legacy=0x7facc40c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc2d9d700 1 -- 172.21.15.45:0/2576942262 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc40c7890 con 0x7facc40ada80 2023-10-06T05:15:53.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc2d9d700 1 --2- 172.21.15.45:0/2576942262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40ada80 0x7facc4163e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7facb40098e0 tx=0x7facb40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf418829d1b9433 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc159a700 1 -- 172.21.15.45:0/2576942262 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0017f0 con 0x7facc40ada80 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc159a700 1 -- 172.21.15.45:0/2576942262 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb4011420 con 0x7facc40ada80 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.742+0000 7facc159a700 1 -- 172.21.15.45:0/2576942262 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb40117c0 con 0x7facc40ada80 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.743+0000 7facc8e20700 1 -- 172.21.15.45:0/2576942262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40ada80 msgr2=0x7facc4163e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.743+0000 7facc8e20700 1 --2- 172.21.15.45:0/2576942262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40ada80 0x7facc4163e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7facb40098e0 tx=0x7facb40091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.743+0000 7facc8e20700 1 -- 172.21.15.45:0/2576942262 shutdown_connections 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.743+0000 7facc8e20700 1 --2- 172.21.15.45:0/2576942262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40ada80 0x7facc4163e40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.743+0000 7facc8e20700 1 -- 172.21.15.45:0/2576942262 >> 172.21.15.45:0/2576942262 conn(0x7facc40cc570 msgr2=0x7facc40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.743+0000 7facc8e20700 1 -- 172.21.15.45:0/2576942262 shutdown_connections 2023-10-06T05:15:53.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.743+0000 7facc8e20700 1 -- 172.21.15.45:0/2576942262 wait complete. 2023-10-06T05:15:53.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.744+0000 7facc8e20700 1 Processor -- start 2023-10-06T05:15:53.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.744+0000 7facc8e20700 1 -- start start 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc8e20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facc406aa70 0x7facc4156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc8e20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc406ae40 0x7facc40bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc8e20700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc40ada80 0x7facc415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc8e20700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40afd30 con 0x7facc406aa70 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc8e20700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c1310 con 0x7facc406ae40 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc8e20700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c0a70 con 0x7facc40ada80 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facc406aa70 0x7facc4156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facc406aa70 0x7facc4156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37960/0 (socket says 172.21.15.45:37960) 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 -- 172.21.15.45:0/3436949885 learned_addr learned my addr 172.21.15.45:0/3436949885 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc2d9d700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc40ada80 0x7facc415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 -- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc40ada80 msgr2=0x7facc415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc1d9b700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc406ae40 0x7facc40bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc40ada80 0x7facc415aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 -- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc406ae40 msgr2=0x7facc40bffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc406ae40 0x7facc40bffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.745+0000 7facc259c700 1 -- 172.21.15.45:0/3436949885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4009420 con 0x7facc406aa70 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.746+0000 7facc259c700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facc406aa70 0x7facc4156980 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7facbc001560 tx=0x7facbc007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.746+0000 7facba7fc700 1 -- 172.21.15.45:0/3436949885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc004e20 con 0x7facc406aa70 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.746+0000 7facba7fc700 1 -- 172.21.15.45:0/3436949885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00a8d0 con 0x7facc406aa70 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.746+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc40c0d50 con 0x7facc406aa70 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.746+0000 7facba7fc700 1 -- 172.21.15.45:0/3436949885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00ac90 con 0x7facc406aa70 2023-10-06T05:15:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.746+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc40affe0 con 0x7facc406aa70 2023-10-06T05:15:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.747+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc40c63a0 con 0x7facc406aa70 2023-10-06T05:15:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.748+0000 7facba7fc700 1 -- 172.21.15.45:0/3436949885 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7facbc007150 con 0x7facc406aa70 2023-10-06T05:15:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.748+0000 7facba7fc700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faca4066f30 0x7faca40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.748+0000 7facba7fc700 1 -- 172.21.15.45:0/3436949885 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7facbc084840 con 0x7facc406aa70 2023-10-06T05:15:53.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.748+0000 7facc1d9b700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faca4066f30 0x7faca40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:53.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.749+0000 7facc1d9b700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faca4066f30 0x7faca40693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7facac0011e0 tx=0x7facac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:53.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.752+0000 7facba7fc700 1 -- 172.21.15.45:0/3436949885 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facbc055210 con 0x7facc406aa70 2023-10-06T05:15:53.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.953+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facc40c7b70 con 0x7facc406aa70 2023-10-06T05:15:53.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.954+0000 7facba7fc700 1 -- 172.21.15.45:0/3436949885 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7facbc0180a0 con 0x7facc406aa70 2023-10-06T05:15:53.956 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:15:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faca4066f30 msgr2=0x7faca40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faca4066f30 0x7faca40693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7facac0011e0 tx=0x7facac009040 comp rx=0 tx=0).stop 2023-10-06T05:15:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facc406aa70 msgr2=0x7facc4156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facc406aa70 0x7facc4156980 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7facbc001560 tx=0x7facbc007e90 comp rx=0 tx=0).stop 2023-10-06T05:15:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 shutdown_connections 2023-10-06T05:15:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faca4066f30 0x7faca40693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc40ada80 0x7facc415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facc406aa70 0x7facc4156980 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 --2- 172.21.15.45:0/3436949885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc406ae40 0x7facc40bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 >> 172.21.15.45:0/3436949885 conn(0x7facc40cc570 msgr2=0x7facc406e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 shutdown_connections 2023-10-06T05:15:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:53.956+0000 7facc8e20700 1 -- 172.21.15.45:0/3436949885 wait complete. 2023-10-06T05:15:53.971 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:15:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.104+0000 7f840d5ee700 1 Processor -- start 2023-10-06T05:15:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.104+0000 7f840d5ee700 1 -- start start 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.105+0000 7f840d5ee700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 0x7f840815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.105+0000 7f840d5ee700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84080cd740 con 0x7f84080d02b0 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.105+0000 7f840d5ee700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84080ccb30 con 0x7f840815c6f0 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.105+0000 7f840d5ee700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84080d0ad0 con 0x7f84080d0680 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.105+0000 7f8406ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f84080d02b0 0x7f840815d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49774/0 (socket says 172.21.15.45:49774) 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.105+0000 7f8406ffd700 1 -- 172.21.15.45:0/4105233986 learned_addr learned my addr 172.21.15.45:0/4105233986 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f84067fc700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 0x7f840815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831121954 0 0) 0x7f84080cd740 con 0x7f84080d02b0 2023-10-06T05:15:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f4003570 con 0x7f84080d02b0 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370948396 0 0) 0x7f84080ccb30 con 0x7f840815c6f0 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84080cd740 con 0x7f840815c6f0 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1438131127 0 0) 0x7f83f4003570 con 0x7f84080d02b0 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 msgr2=0x7f840815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 0x7f840815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 >> v1:172.21.15.144:6789/0 conn(0x7f840815c6f0 legacy=0x7f84080caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080cce10 con 0x7f84080d02b0 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f83f8001470 con 0x7f84080d02b0 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f84067fc700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 0x7f840815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:15:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 >> v1:172.21.15.45:6789/0 conn(0x7f84080d02b0 legacy=0x7f840815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f4003c20 0x7f83f4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83f4004810 0x7f83f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 0x7f83f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84080d0ad0 con 0x7f83f4003c20 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f400bb10 con 0x7f83f4004810 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f400bcd0 con 0x7f84080d0680 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8406ffd700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f4003c20 0x7f83f4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f84067fc700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83f4004810 0x7f83f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f84077fe700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 0x7f83f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8406ffd700 1 -- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 msgr2=0x7f83f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8406ffd700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84080d0680 0x7f83f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.106+0000 7f8406ffd700 1 -- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83f4004810 msgr2=0x7f83f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f8406ffd700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83f4004810 0x7f83f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f8406ffd700 1 -- 172.21.15.45:0/4105233986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f0009420 con 0x7f83f4003c20 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f8406ffd700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f4003c20 0x7f83f4004010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f83f800c2d0 tx=0x7f83f8001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894ea4286ebe8598 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f800cba0 con 0x7f83f4003c20 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f800cd00 con 0x7f83f4003c20 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f8405ffb700 1 -- 172.21.15.45:0/4105233986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8005b70 con 0x7f83f4003c20 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f840d5ee700 1 -- 172.21.15.45:0/4105233986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f4003c20 msgr2=0x7f83f4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f840d5ee700 1 --2- 172.21.15.45:0/4105233986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f4003c20 0x7f83f4004010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f83f800c2d0 tx=0x7f83f8001e10 comp rx=0 tx=0).stop 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f84077fe700 1 -- 172.21.15.45:0/4105233986 reap_dead start 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f840d5ee700 1 -- 172.21.15.45:0/4105233986 shutdown_connections 2023-10-06T05:15:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f840d5ee700 1 -- 172.21.15.45:0/4105233986 >> 172.21.15.45:0/4105233986 conn(0x7f840806ed50 msgr2=0x7f840806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f840d5ee700 1 -- 172.21.15.45:0/4105233986 shutdown_connections 2023-10-06T05:15:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.107+0000 7f840d5ee700 1 -- 172.21.15.45:0/4105233986 wait complete. 2023-10-06T05:15:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.109+0000 7f840d5ee700 1 Processor -- start 2023-10-06T05:15:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.109+0000 7f840d5ee700 1 -- start start 2023-10-06T05:15:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.109+0000 7f840d5ee700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83f4003c20 0x7f84080cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.109+0000 7f840d5ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408159320 0x7f8408159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f840d5ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84080a7940 0x7f84080a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f840d5ee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84080a8590 con 0x7f8408159320 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f840d5ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840816a190 con 0x7f84080a7940 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f840d5ee700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8408159cd0 con 0x7f83f4003c20 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408159320 0x7f8408159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408159320 0x7f8408159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37986/0 (socket says 172.21.15.45:37986) 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 -- 172.21.15.45:0/882002808 learned_addr learned my addr 172.21.15.45:0/882002808 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84077fe700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84080a7940 0x7f84080a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f8406ffd700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83f4003c20 0x7f84080cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 -- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83f4003c20 msgr2=0x7f84080cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83f4003c20 0x7f84080cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 -- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84080a7940 msgr2=0x7f84080a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84080a7940 0x7f84080a7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 -- 172.21.15.45:0/882002808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080a8300 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84077fe700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84080a7940 0x7f84080a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f84067fc700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408159320 0x7f8408159790 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83f0011d70 tx=0x7f83f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f83feffd700 1 -- 172.21.15.45:0/882002808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f000fcd0 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f83feffd700 1 -- 172.21.15.45:0/882002808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f000fe30 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84081695a0 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f83feffd700 1 -- 172.21.15.45:0/882002808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0014670 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.110+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8408169b40 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.111+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840816a320 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.112+0000 7f83feffd700 1 -- 172.21.15.45:0/882002808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83f0014aa0 con 0x7f8408159320 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.112+0000 7f83feffd700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f83e8066f30 0x7f83e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.112+0000 7f83feffd700 1 -- 172.21.15.45:0/882002808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f83f0018030 con 0x7f8408159320 2023-10-06T05:15:54.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.112+0000 7f8406ffd700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f83e8066f30 0x7f83e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.113+0000 7f8406ffd700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f83e8066f30 0x7f83e80693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84080b5660 tx=0x7f83f8015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.117+0000 7f83feffd700 1 -- 172.21.15.45:0/882002808 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83f00528a0 con 0x7f8408159320 2023-10-06T05:15:54.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.316+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f840804bb40 con 0x7f8408159320 2023-10-06T05:15:54.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.316+0000 7f83feffd700 1 -- 172.21.15.45:0/882002808 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f83f00220a0 con 0x7f8408159320 2023-10-06T05:15:54.318 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f83e8066f30 msgr2=0x7f83e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f83e8066f30 0x7f83e80693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84080b5660 tx=0x7f83f8015000 comp rx=0 tx=0).stop 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408159320 msgr2=0x7f8408159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408159320 0x7f8408159790 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83f0011d70 tx=0x7f83f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 shutdown_connections 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f83e8066f30 0x7f83e80693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84080a7940 0x7f84080a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83f4003c20 0x7f84080cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 --2- 172.21.15.45:0/882002808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8408159320 0x7f8408159790 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 >> 172.21.15.45:0/882002808 conn(0x7f840806ed50 msgr2=0x7f840815b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:54.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 shutdown_connections 2023-10-06T05:15:54.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.318+0000 7f840d5ee700 1 -- 172.21.15.45:0/882002808 wait complete. 2023-10-06T05:15:54.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:15:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.468+0000 7f426d5df700 1 Processor -- start 2023-10-06T05:15:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.468+0000 7f426d5df700 1 -- start start 2023-10-06T05:15:54.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f426d5df700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f42680d02b0 0x7f42680d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f426d5df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f426815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f426d5df700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f426815c6f0 0x7f42680afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f426d5df700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42680cd740 con 0x7f426815c6f0 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f426d5df700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42680ccb30 con 0x7f42680d02b0 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f426d5df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680cce10 con 0x7f426815d820 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f426815c6f0 0x7f42680afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f426815c6f0 0x7f42680afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37994/0 (socket says 172.21.15.45:37994) 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 -- 172.21.15.45:0/3292988861 learned_addr learned my addr 172.21.15.45:0/3292988861 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f4266ffd700 1 --2- 172.21.15.45:0/3292988861 >> v2:172.21.15.144:3300/0 conn(0x7f42680d02b0 0x7f42680d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42667fc700 1 --2- 172.21.15.45:0/3292988861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f426815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 -- 172.21.15.45:0/3292988861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 msgr2=0x7f426815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:54.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 --2- 172.21.15.45:0/3292988861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f426815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 -- 172.21.15.45:0/3292988861 >> v2:172.21.15.144:3300/0 conn(0x7f42680d02b0 msgr2=0x7f42680d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 --2- 172.21.15.45:0/3292988861 >> v2:172.21.15.144:3300/0 conn(0x7f42680d02b0 0x7f42680d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 -- 172.21.15.45:0/3292988861 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680cd0f0 con 0x7f426815c6f0 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42667fc700 1 --2- 172.21.15.45:0/3292988861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f426815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f4266ffd700 1 --2- 172.21.15.45:0/3292988861 >> v2:172.21.15.144:3300/0 conn(0x7f42680d02b0 0x7f42680d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.469+0000 7f42677fe700 1 --2- 172.21.15.45:0/3292988861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426815c6f0 0x7f42680afd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4260009f20 tx=0x7f4260009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2000803181f315a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f4265ffb700 1 -- 172.21.15.45:0/3292988861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000e070 con 0x7f426815c6f0 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f4265ffb700 1 -- 172.21.15.45:0/3292988861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426000ba40 con 0x7f426815c6f0 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f4265ffb700 1 -- 172.21.15.45:0/3292988861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600092e0 con 0x7f426815c6f0 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 -- 172.21.15.45:0/3292988861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426815c6f0 msgr2=0x7f42680afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 --2- 172.21.15.45:0/3292988861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426815c6f0 0x7f42680afd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4260009f20 tx=0x7f4260009840 comp rx=0 tx=0).stop 2023-10-06T05:15:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 -- 172.21.15.45:0/3292988861 shutdown_connections 2023-10-06T05:15:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 --2- 172.21.15.45:0/3292988861 >> v2:172.21.15.144:3300/0 conn(0x7f42680d02b0 0x7f42680d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 --2- 172.21.15.45:0/3292988861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426815c6f0 0x7f42680afd40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 --2- 172.21.15.45:0/3292988861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f426815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 -- 172.21.15.45:0/3292988861 >> 172.21.15.45:0/3292988861 conn(0x7f426806ed50 msgr2=0x7f426806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 -- 172.21.15.45:0/3292988861 shutdown_connections 2023-10-06T05:15:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.470+0000 7f426d5df700 1 -- 172.21.15.45:0/3292988861 wait complete. 2023-10-06T05:15:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 Processor -- start 2023-10-06T05:15:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 -- start start 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680d02b0 0x7f42680a7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f426815c6f0 0x7f426814e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f42680c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680a7bc0 con 0x7f42680d02b0 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426814eb20 con 0x7f426815c6f0 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.472+0000 7f426d5df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680c9070 con 0x7f426815d820 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680d02b0 0x7f42680a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680d02b0 0x7f42680a7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37996/0 (socket says 172.21.15.45:37996) 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 -- 172.21.15.45:0/4271134409 learned_addr learned my addr 172.21.15.45:0/4271134409 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f42667fc700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f426815c6f0 0x7f426814e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f42677fe700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f42680c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 -- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 msgr2=0x7f42680c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f42680c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 -- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f426815c6f0 msgr2=0x7f426814e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f426815c6f0 0x7f426814e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 -- 172.21.15.45:0/4271134409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009ae0 con 0x7f42680d02b0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f4266ffd700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680d02b0 0x7f42680a7680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f425800c880 tx=0x7f4258008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f424effd700 1 -- 172.21.15.45:0/4271134409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258011600 con 0x7f42680d02b0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f424effd700 1 -- 172.21.15.45:0/4271134409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258011760 con 0x7f42680d02b0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42680c9330 con 0x7f42680d02b0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f424effd700 1 -- 172.21.15.45:0/4271134409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258011ae0 con 0x7f42680d02b0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.473+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42680c98f0 con 0x7f42680d02b0 2023-10-06T05:15:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.474+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4268088730 con 0x7f42680d02b0 2023-10-06T05:15:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.475+0000 7f424effd700 1 -- 172.21.15.45:0/4271134409 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4258010350 con 0x7f42680d02b0 2023-10-06T05:15:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.475+0000 7f424effd700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4254066f30 0x7f42540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.475+0000 7f424effd700 1 -- 172.21.15.45:0/4271134409 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f4258084970 con 0x7f42680d02b0 2023-10-06T05:15:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.476+0000 7f42667fc700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4254066f30 0x7f42540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.476+0000 7f42667fc700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4254066f30 0x7f42540693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f426815d7f0 tx=0x7f425c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.480+0000 7f424effd700 1 -- 172.21.15.45:0/4271134409 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42580551e0 con 0x7f42680d02b0 2023-10-06T05:15:54.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.679+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42680cd3d0 con 0x7f42680d02b0 2023-10-06T05:15:54.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.680+0000 7f424effd700 1 -- 172.21.15.45:0/4271134409 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4258054d70 con 0x7f42680d02b0 2023-10-06T05:15:54.682 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4254066f30 msgr2=0x7f42540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4254066f30 0x7f42540693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f426815d7f0 tx=0x7f425c008040 comp rx=0 tx=0).stop 2023-10-06T05:15:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680d02b0 msgr2=0x7f42680a7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680d02b0 0x7f42680a7680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f425800c880 tx=0x7f4258008040 comp rx=0 tx=0).stop 2023-10-06T05:15:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 shutdown_connections 2023-10-06T05:15:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4254066f30 0x7f42540693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f426815d820 0x7f42680c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680d02b0 0x7f42680a7680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 --2- 172.21.15.45:0/4271134409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f426815c6f0 0x7f426814e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 >> 172.21.15.45:0/4271134409 conn(0x7f426806ed50 msgr2=0x7f426815a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 shutdown_connections 2023-10-06T05:15:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.682+0000 7f426d5df700 1 -- 172.21.15.45:0/4271134409 wait complete. 2023-10-06T05:15:54.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:15:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.831+0000 7fb1803e3700 1 Processor -- start 2023-10-06T05:15:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.831+0000 7fb1803e3700 1 -- start start 2023-10-06T05:15:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb1803e3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb17806aa70 0x7fb1780c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb1803e3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb17815c8c0 0x7fb1780aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb1803e3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb178067d80 con 0x7fb1780c7050 2023-10-06T05:15:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb1803e3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1780ac620 con 0x7fb17815c8c0 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb1803e3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1780c75b0 con 0x7fb17806aa70 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17e980700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb17815c8c0 0x7fb1780aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17d97e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb1780c7050 0x7fb17815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49788/0 (socket says 172.21.15.45:49788) 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17d97e700 1 -- 172.21.15.45:0/1463281069 learned_addr learned my addr 172.21.15.45:0/1463281069 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17e17f700 1 --2- 172.21.15.45:0/1463281069 >> v2:172.21.15.144:3300/0 conn(0x7fb17806aa70 0x7fb1780c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17e980700 1 -- 172.21.15.45:0/1463281069 >> v2:172.21.15.144:3300/0 conn(0x7fb17806aa70 msgr2=0x7fb1780c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17e980700 1 --2- 172.21.15.45:0/1463281069 >> v2:172.21.15.144:3300/0 conn(0x7fb17806aa70 0x7fb1780c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17e980700 1 -- 172.21.15.45:0/1463281069 >> v1:172.21.15.45:6789/0 conn(0x7fb1780c7050 legacy=0x7fb17815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.832+0000 7fb17d17d700 1 -- 172.21.15.45:0/1463281069 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995583709 0 0) 0x7fb178067d80 con 0x7fb1780c7050 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.833+0000 7fb17e980700 1 -- 172.21.15.45:0/1463281069 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780c7890 con 0x7fb17815c8c0 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.833+0000 7fb17e980700 1 --2- 172.21.15.45:0/1463281069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb1780aca70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb17806f990 tx=0x7fb170009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6385f4abd0ee2a67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.833+0000 7fb17d17d700 1 -- 172.21.15.45:0/1463281069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000e070 con 0x7fb17815c8c0 2023-10-06T05:15:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.833+0000 7fb17d17d700 1 -- 172.21.15.45:0/1463281069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17000bc30 con 0x7fb17815c8c0 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.833+0000 7fb17d17d700 1 -- 172.21.15.45:0/1463281069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170012430 con 0x7fb17815c8c0 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.833+0000 7fb1803e3700 1 -- 172.21.15.45:0/1463281069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 msgr2=0x7fb1780aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.833+0000 7fb1803e3700 1 --2- 172.21.15.45:0/1463281069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb1780aca70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb17806f990 tx=0x7fb170009780 comp rx=0 tx=0).stop 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.834+0000 7fb1803e3700 1 -- 172.21.15.45:0/1463281069 shutdown_connections 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.834+0000 7fb1803e3700 1 --2- 172.21.15.45:0/1463281069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb1780aca70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.834+0000 7fb1803e3700 1 --2- 172.21.15.45:0/1463281069 >> v2:172.21.15.144:3300/0 conn(0x7fb17806aa70 0x7fb1780c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.834+0000 7fb1803e3700 1 -- 172.21.15.45:0/1463281069 >> 172.21.15.45:0/1463281069 conn(0x7fb1780cc570 msgr2=0x7fb1780cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.834+0000 7fb1803e3700 1 -- 172.21.15.45:0/1463281069 shutdown_connections 2023-10-06T05:15:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.834+0000 7fb1803e3700 1 -- 172.21.15.45:0/1463281069 wait complete. 2023-10-06T05:15:54.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.835+0000 7fb1803e3700 1 Processor -- start 2023-10-06T05:15:54.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.835+0000 7fb1803e3700 1 -- start start 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.835+0000 7fb1803e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb17806aa70 0x7fb178158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.835+0000 7fb1803e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1780c7050 0x7fb1780b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb1803e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb178156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb1803e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b1800 con 0x7fb17815c8c0 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb1803e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780c3d70 con 0x7fb17806aa70 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb1803e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780b1520 con 0x7fb1780c7050 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb178156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb178156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38022/0 (socket says 172.21.15.45:38022) 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 -- 172.21.15.45:0/2774504002 learned_addr learned my addr 172.21.15.45:0/2774504002 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e17f700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb17806aa70 0x7fb178158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17d97e700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1780c7050 0x7fb1780b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 -- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1780c7050 msgr2=0x7fb1780b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1780c7050 0x7fb1780b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 -- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb17806aa70 msgr2=0x7fb178158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb17806aa70 0x7fb178158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 -- 172.21.15.45:0/2774504002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170009a20 con 0x7fb17815c8c0 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e17f700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb17806aa70 0x7fb178158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb17e980700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb178156910 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb17000b6c0 tx=0x7fb170019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb16e7fc700 1 -- 172.21.15.45:0/2774504002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170012e60 con 0x7fb17815c8c0 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb16e7fc700 1 -- 172.21.15.45:0/2774504002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17001d630 con 0x7fb17815c8c0 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1780c3720 con 0x7fb17815c8c0 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb16e7fc700 1 -- 172.21.15.45:0/2774504002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17001d9f0 con 0x7fb17815c8c0 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.836+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1780b1230 con 0x7fb17815c8c0 2023-10-06T05:15:54.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.838+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1780c3580 con 0x7fb17815c8c0 2023-10-06T05:15:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.838+0000 7fb16e7fc700 1 -- 172.21.15.45:0/2774504002 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb170024070 con 0x7fb17815c8c0 2023-10-06T05:15:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.838+0000 7fb16e7fc700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb15c066f30 0x7fb15c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.838+0000 7fb16e7fc700 1 -- 172.21.15.45:0/2774504002 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fb170087290 con 0x7fb17815c8c0 2023-10-06T05:15:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.838+0000 7fb17e17f700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb15c066f30 0x7fb15c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.839+0000 7fb17e17f700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb15c066f30 0x7fb15c0693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb174006fd0 tx=0x7fb174008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:54.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:54.843+0000 7fb16e7fc700 1 -- 172.21.15.45:0/2774504002 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1780c3580 con 0x7fb17815c8c0 2023-10-06T05:15:55.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.048+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1780c3580 con 0x7fb17815c8c0 2023-10-06T05:15:55.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.048+0000 7fb16e7fc700 1 -- 172.21.15.45:0/2774504002 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1780c3580 con 0x7fb17815c8c0 2023-10-06T05:15:55.050 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:15:55.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb15c066f30 msgr2=0x7fb15c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb15c066f30 0x7fb15c0693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb174006fd0 tx=0x7fb174008040 comp rx=0 tx=0).stop 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 msgr2=0x7fb178156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb178156910 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb17000b6c0 tx=0x7fb170019000 comp rx=0 tx=0).stop 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 shutdown_connections 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb15c066f30 0x7fb15c0693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb17815c8c0 0x7fb178156910 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb17806aa70 0x7fb178158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 --2- 172.21.15.45:0/2774504002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1780c7050 0x7fb1780b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 >> 172.21.15.45:0/2774504002 conn(0x7fb1780cc570 msgr2=0x7fb1780ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 shutdown_connections 2023-10-06T05:15:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.050+0000 7fb1803e3700 1 -- 172.21.15.45:0/2774504002 wait complete. 2023-10-06T05:15:55.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T05:15:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.201+0000 7f53d09f7700 1 Processor -- start 2023-10-06T05:15:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.201+0000 7f53d09f7700 1 -- start start 2023-10-06T05:15:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53d09f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 0x7f53cc0ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53d09f7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc0c9940 con 0x7f53cc1406e0 2023-10-06T05:15:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53d09f7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc0ca750 con 0x7f53cc140ab0 2023-10-06T05:15:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53d09f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc06d230 con 0x7f53cc0c9dd0 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53ca59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f53cc1406e0 0x7f53cc06c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49800/0 (socket says 172.21.15.45:49800) 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53ca59c700 1 -- 172.21.15.45:0/1109103809 learned_addr learned my addr 172.21.15.45:0/1109103809 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53cad9d700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 0x7f53cc0ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.202+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494363881 0 0) 0x7f53cc0c9940 con 0x7f53cc1406e0 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b4003570 con 0x7f53cc1406e0 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2774319095 0 0) 0x7f53b4003570 con 0x7f53cc1406e0 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 msgr2=0x7f53cc0ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 0x7f53cc0ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 >> v1:172.21.15.144:6789/0 conn(0x7f53cc140ab0 legacy=0x7f53cc0acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc06d510 con 0x7f53cc1406e0 2023-10-06T05:15:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f53bc001470 con 0x7f53cc1406e0 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 >> v1:172.21.15.45:6789/0 conn(0x7f53cc1406e0 legacy=0x7f53cc06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 0x7f53b4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b4003fd0 0x7f53b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc06d230 con 0x7f53b4004d50 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b400bb40 con 0x7f53b4003fd0 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b400bd00 con 0x7f53cc0c9dd0 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53cad9d700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53ca59c700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 0x7f53b4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53c9d9b700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b4003fd0 0x7f53b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53cad9d700 1 -- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 msgr2=0x7f53b4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53cad9d700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc0c9dd0 0x7f53b4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53cad9d700 1 -- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b4003fd0 msgr2=0x7f53b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53cad9d700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b4003fd0 0x7f53b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.203+0000 7f53cad9d700 1 -- 172.21.15.45:0/1109103809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4003570 con 0x7f53b4004d50 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53cad9d700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53b40051c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e210 tx=0x7f53c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8996d2abf3a0e82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400eac0 con 0x7f53b4004d50 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400f040 con 0x7f53b4004d50 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53c959a700 1 -- 172.21.15.45:0/1109103809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400ee70 con 0x7f53b4004d50 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53d09f7700 1 -- 172.21.15.45:0/1109103809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 msgr2=0x7f53b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53d09f7700 1 --2- 172.21.15.45:0/1109103809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53b40051c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e210 tx=0x7f53c400b040 comp rx=0 tx=0).stop 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53cad9d700 1 -- 172.21.15.45:0/1109103809 reap_dead start 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53d09f7700 1 -- 172.21.15.45:0/1109103809 shutdown_connections 2023-10-06T05:15:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53d09f7700 1 -- 172.21.15.45:0/1109103809 >> 172.21.15.45:0/1109103809 conn(0x7f53cc0691c0 msgr2=0x7f53cc0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53d09f7700 1 -- 172.21.15.45:0/1109103809 shutdown_connections 2023-10-06T05:15:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.204+0000 7f53d09f7700 1 -- 172.21.15.45:0/1109103809 wait complete. 2023-10-06T05:15:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 Processor -- start 2023-10-06T05:15:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 -- start start 2023-10-06T05:15:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53cc14d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc15ce10 0x7f53cc15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53cc0b2140 0x7f53cc0b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc15d7c0 con 0x7f53b4004d50 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0b2af0 con 0x7f53cc0b2140 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.206+0000 7f53d09f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0c3ef0 con 0x7f53cc15ce10 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53cc14d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53cc14d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38040/0 (socket says 172.21.15.45:38040) 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 -- 172.21.15.45:0/2558894857 learned_addr learned my addr 172.21.15.45:0/2558894857 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53cad9d700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53cc0b2140 0x7f53cc0b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53c9d9b700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc15ce10 0x7f53cc15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 -- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc15ce10 msgr2=0x7f53cc15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc15ce10 0x7f53cc15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 -- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53cc0b2140 msgr2=0x7f53cc0b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53cc0b2140 0x7f53cc0b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 -- 172.21.15.45:0/2558894857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0c4140 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ca59c700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53cc14d780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53bc007fd0 tx=0x7f53cc0c98b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ba7fc700 1 -- 172.21.15.45:0/2558894857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc018030 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ba7fc700 1 -- 172.21.15.45:0/2558894857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc0077d0 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc0c4420 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53ba7fc700 1 -- 172.21.15.45:0/2558894857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00faa0 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.207+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc0c49e0 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.208+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc06e2d0 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.209+0000 7f53ba7fc700 1 -- 172.21.15.45:0/2558894857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53bc01b070 con 0x7f53b4004d50 2023-10-06T05:15:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.209+0000 7f53ba7fc700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f53ac066f30 0x7f53ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.209+0000 7f53ba7fc700 1 -- 172.21.15.45:0/2558894857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f53bc019070 con 0x7f53b4004d50 2023-10-06T05:15:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.209+0000 7f53c9d9b700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f53ac066f30 0x7f53ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.210+0000 7f53c9d9b700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f53ac066f30 0x7f53ac0693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53c00010e0 tx=0x7f53c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.213+0000 7f53ba7fc700 1 -- 172.21.15.45:0/2558894857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53bc051680 con 0x7f53b4004d50 2023-10-06T05:15:55.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.414+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f53cc04bb40 con 0x7f53b4004d50 2023-10-06T05:15:55.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.415+0000 7f53ba7fc700 1 -- 172.21.15.45:0/2558894857 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f53bc054ca0 con 0x7f53b4004d50 2023-10-06T05:15:55.417 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f53ac066f30 msgr2=0x7f53ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f53ac066f30 0x7f53ac0693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53c00010e0 tx=0x7f53c0009040 comp rx=0 tx=0).stop 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 msgr2=0x7f53cc14d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53cc14d780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53bc007fd0 tx=0x7f53cc0c98b0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 shutdown_connections 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f53ac066f30 0x7f53ac0693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53cc0b2140 0x7f53cc0b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53b4004d50 0x7f53cc14d780 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 --2- 172.21.15.45:0/2558894857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53cc15ce10 0x7f53cc15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 >> 172.21.15.45:0/2558894857 conn(0x7f53cc0691c0 msgr2=0x7f53cc0aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 shutdown_connections 2023-10-06T05:15:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.417+0000 7f53d09f7700 1 -- 172.21.15.45:0/2558894857 wait complete. 2023-10-06T05:15:55.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.564+0000 7fca60a3a700 1 Processor -- start 2023-10-06T05:15:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.564+0000 7fca60a3a700 1 -- start start 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca60a3a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca5c0c8ff0 0x7fca5c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca60a3a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fca5c0caad0 0x7fca5c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca60a3a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c156af0 0x7fca5c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca60a3a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5c15b000 con 0x7fca5c0c8ff0 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca60a3a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5c15b2e0 con 0x7fca5c0caad0 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca60a3a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c15b5c0 con 0x7fca5c156af0 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca5a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca5c0c8ff0 0x7fca5c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca5a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca5c0c8ff0 0x7fca5c0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38042/0 (socket says 172.21.15.45:38042) 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca5a59c700 1 -- 172.21.15.45:0/236721840 learned_addr learned my addr 172.21.15.45:0/236721840 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca59d9b700 1 --2- 172.21.15.45:0/236721840 >> v2:172.21.15.144:3300/0 conn(0x7fca5c0caad0 0x7fca5c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.565+0000 7fca5ad9d700 1 --2- 172.21.15.45:0/236721840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c156af0 0x7fca5c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5a59c700 1 -- 172.21.15.45:0/236721840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c156af0 msgr2=0x7fca5c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5a59c700 1 --2- 172.21.15.45:0/236721840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c156af0 0x7fca5c156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5a59c700 1 -- 172.21.15.45:0/236721840 >> v2:172.21.15.144:3300/0 conn(0x7fca5c0caad0 msgr2=0x7fca5c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5a59c700 1 --2- 172.21.15.45:0/236721840 >> v2:172.21.15.144:3300/0 conn(0x7fca5c0caad0 0x7fca5c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5a59c700 1 -- 172.21.15.45:0/236721840 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c15b8a0 con 0x7fca5c0c8ff0 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5ad9d700 1 --2- 172.21.15.45:0/236721840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c156af0 0x7fca5c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5a59c700 1 --2- 172.21.15.45:0/236721840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c0c8ff0 0x7fca5c0c93c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca5c06ed80 tx=0x7fca4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1ad2f4edb5c0f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5959a700 1 -- 172.21.15.45:0/236721840 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00b860 con 0x7fca5c0c8ff0 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5959a700 1 -- 172.21.15.45:0/236721840 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c00b9c0 con 0x7fca5c0c8ff0 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca5959a700 1 -- 172.21.15.45:0/236721840 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00bd60 con 0x7fca5c0c8ff0 2023-10-06T05:15:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca60a3a700 1 -- 172.21.15.45:0/236721840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c0c8ff0 msgr2=0x7fca5c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.566+0000 7fca60a3a700 1 --2- 172.21.15.45:0/236721840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c0c8ff0 0x7fca5c0c93c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca5c06ed80 tx=0x7fca4c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.567+0000 7fca60a3a700 1 -- 172.21.15.45:0/236721840 shutdown_connections 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.567+0000 7fca60a3a700 1 --2- 172.21.15.45:0/236721840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c156af0 0x7fca5c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.567+0000 7fca60a3a700 1 --2- 172.21.15.45:0/236721840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c0c8ff0 0x7fca5c0c93c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.567+0000 7fca60a3a700 1 --2- 172.21.15.45:0/236721840 >> v2:172.21.15.144:3300/0 conn(0x7fca5c0caad0 0x7fca5c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.567+0000 7fca60a3a700 1 -- 172.21.15.45:0/236721840 >> 172.21.15.45:0/236721840 conn(0x7fca5c0cdf50 msgr2=0x7fca5c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.567+0000 7fca60a3a700 1 -- 172.21.15.45:0/236721840 shutdown_connections 2023-10-06T05:15:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.567+0000 7fca60a3a700 1 -- 172.21.15.45:0/236721840 wait complete. 2023-10-06T05:15:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.568+0000 7fca60a3a700 1 Processor -- start 2023-10-06T05:15:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.568+0000 7fca60a3a700 1 -- start start 2023-10-06T05:15:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca60a3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca5c0c8ff0 0x7fca5c06c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca60a3a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c0caad0 0x7fca5c0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca60a3a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c156af0 0x7fca5c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca60a3a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0b2740 con 0x7fca5c156af0 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca60a3a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c156f70 con 0x7fca5c0c8ff0 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca60a3a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0b2940 con 0x7fca5c0caad0 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c156af0 0x7fca5c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c156af0 0x7fca5c0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38056/0 (socket says 172.21.15.45:38056) 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 -- 172.21.15.45:0/3591860668 learned_addr learned my addr 172.21.15.45:0/3591860668 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca59d9b700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c0caad0 0x7fca5c0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5a59c700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca5c0c8ff0 0x7fca5c06c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 -- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c0caad0 msgr2=0x7fca5c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c0caad0 0x7fca5c0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 -- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca5c0c8ff0 msgr2=0x7fca5c06c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca5c0c8ff0 0x7fca5c06c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 -- 172.21.15.45:0/3591860668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0094f0 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca5ad9d700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c156af0 0x7fca5c0b2200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca5400cef0 tx=0x7fca54007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.569+0000 7fca4a7fc700 1 -- 172.21.15.45:0/3591860668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5400ad20 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.570+0000 7fca4a7fc700 1 -- 172.21.15.45:0/3591860668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca54016420 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.570+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c0b2c20 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.570+0000 7fca4a7fc700 1 -- 172.21.15.45:0/3591860668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5400aea0 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.570+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c1574a0 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.570+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c0c9ab0 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.571+0000 7fca4a7fc700 1 -- 172.21.15.45:0/3591860668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca54016a70 con 0x7fca5c156af0 2023-10-06T05:15:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.571+0000 7fca4a7fc700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca3c066f30 0x7fca3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.572+0000 7fca4a7fc700 1 -- 172.21.15.45:0/3591860668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fca54012070 con 0x7fca5c156af0 2023-10-06T05:15:55.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.572+0000 7fca5a59c700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca3c066f30 0x7fca3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.572+0000 7fca5a59c700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca3c066f30 0x7fca3c0693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fca5c06ed80 tx=0x7fca4c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.576+0000 7fca4a7fc700 1 -- 172.21.15.45:0/3591860668 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca5404e5a0 con 0x7fca5c156af0 2023-10-06T05:15:55.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.831+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca5c04bb40 con 0x7fca5c156af0 2023-10-06T05:15:55.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.832+0000 7fca4a7fc700 1 -- 172.21.15.45:0/3591860668 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7fca54051fc0 con 0x7fca5c156af0 2023-10-06T05:15:55.834 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:55.834 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:55.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca3c066f30 msgr2=0x7fca3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca3c066f30 0x7fca3c0693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fca5c06ed80 tx=0x7fca4c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:55.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c156af0 msgr2=0x7fca5c0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c156af0 0x7fca5c0b2200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca5400cef0 tx=0x7fca54007860 comp rx=0 tx=0).stop 2023-10-06T05:15:55.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 shutdown_connections 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca3c066f30 0x7fca3c0693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c156af0 0x7fca5c0b2200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca5c0c8ff0 0x7fca5c06c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 --2- 172.21.15.45:0/3591860668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca5c0caad0 0x7fca5c0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 >> 172.21.15.45:0/3591860668 conn(0x7fca5c0cdf50 msgr2=0x7fca5c151960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 shutdown_connections 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.834+0000 7fca60a3a700 1 -- 172.21.15.45:0/3591860668 wait complete. 2023-10-06T05:15:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:55.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:55.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.983+0000 7f31aa56b700 1 Processor -- start 2023-10-06T05:15:55.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.983+0000 7f31aa56b700 1 -- start start 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31aa56b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f31a40cd4d0 0x7f31a415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31aa56b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a40ae8d0 con 0x7f31a4155cd0 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31aa56b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a40aebb0 con 0x7f31a41560a0 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31aa56b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a40ae6b0 con 0x7f31a40cd4d0 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a3fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f31a4155cd0 0x7f31a40adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49810/0 (socket says 172.21.15.45:49810) 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a3fff700 1 -- 172.21.15.45:0/696988197 learned_addr learned my addr 172.21.15.45:0/696988197 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a8b08700 1 --2- 172.21.15.45:0/696988197 >> v2:172.21.15.144:3300/0 conn(0x7f31a40cd4d0 0x7f31a415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834439439 0 0) 0x7f31a40ae8d0 con 0x7f31a4155cd0 2023-10-06T05:15:55.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318c003570 con 0x7f31a4155cd0 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311462931 0 0) 0x7f318c003570 con 0x7f31a4155cd0 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 >> v2:172.21.15.144:3300/0 conn(0x7f31a40cd4d0 msgr2=0x7f31a415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 --2- 172.21.15.45:0/696988197 >> v2:172.21.15.144:3300/0 conn(0x7f31a40cd4d0 0x7f31a415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 >> v1:172.21.15.144:6789/0 conn(0x7f31a41560a0 legacy=0x7f31a40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40aee90 con 0x7f31a4155cd0 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f3194001470 con 0x7f31a4155cd0 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 >> v1:172.21.15.45:6789/0 conn(0x7f31a4155cd0 legacy=0x7f31a40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c003a60 0x7f318c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f318c004810 0x7f318c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c0051c0 0x7f318c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40ae6b0 con 0x7f318c0051c0 2023-10-06T05:15:55.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.984+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c00c160 con 0x7f318c003a60 2023-10-06T05:15:55.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c00c320 con 0x7f318c004810 2023-10-06T05:15:55.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c0051c0 0x7f318c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a3fff700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c003a60 0x7f318c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a37fe700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f318c004810 0x7f318c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 -- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f318c004810 msgr2=0x7f318c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f318c004810 0x7f318c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 -- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c003a60 msgr2=0x7f318c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c003a60 0x7f318c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 -- 172.21.15.45:0/696988197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c003570 con 0x7f318c0051c0 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 -- 172.21.15.45:0/696988197 reap_dead start 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a8b08700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c0051c0 0x7f318c00ba60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f31a40cd8d0 tx=0x7f319c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ede20fb171f12f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c00ea90 con 0x7f318c0051c0 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c00f040 con 0x7f318c0051c0 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.985+0000 7f31a2ffd700 1 -- 172.21.15.45:0/696988197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c00ee60 con 0x7f318c0051c0 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.986+0000 7f31aa56b700 1 -- 172.21.15.45:0/696988197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c0051c0 msgr2=0x7f318c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.986+0000 7f31aa56b700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c0051c0 0x7f318c00ba60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f31a40cd8d0 tx=0x7f319c00b040 comp rx=0 tx=0).stop 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.986+0000 7f31aa56b700 1 -- 172.21.15.45:0/696988197 shutdown_connections 2023-10-06T05:15:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.986+0000 7f31aa56b700 1 --2- 172.21.15.45:0/696988197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318c0051c0 0x7f318c00ba60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.986+0000 7f31aa56b700 1 -- 172.21.15.45:0/696988197 >> 172.21.15.45:0/696988197 conn(0x7f31a406f090 msgr2=0x7f31a406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:55.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.986+0000 7f31aa56b700 1 -- 172.21.15.45:0/696988197 shutdown_connections 2023-10-06T05:15:55.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.986+0000 7f31aa56b700 1 -- 172.21.15.45:0/696988197 wait complete. 2023-10-06T05:15:55.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.987+0000 7f31aa56b700 1 Processor -- start 2023-10-06T05:15:55.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.987+0000 7f31aa56b700 1 -- start start 2023-10-06T05:15:55.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31aa56b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c0051c0 0x7f31a40c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31aa56b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31a41580d0 0x7f31a41565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31aa56b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a4156af0 0x7f31a40cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31aa56b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4157c40 con 0x7f31a4156af0 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31aa56b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40c3ab0 con 0x7f318c0051c0 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31aa56b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4157030 con 0x7f31a41580d0 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31a8b08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a4156af0 0x7f31a40cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31a8b08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a4156af0 0x7f31a40cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38080/0 (socket says 172.21.15.45:38080) 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31a8b08700 1 -- 172.21.15.45:0/1345803010 learned_addr learned my addr 172.21.15.45:0/1345803010 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31a3fff700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c0051c0 0x7f31a40c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.988+0000 7f31a37fe700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31a41580d0 0x7f31a41565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a8b08700 1 -- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31a41580d0 msgr2=0x7f31a41565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a8b08700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31a41580d0 0x7f31a41565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a8b08700 1 -- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c0051c0 msgr2=0x7f31a40c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a8b08700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c0051c0 0x7f31a40c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a8b08700 1 -- 172.21.15.45:0/1345803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4157310 con 0x7f31a4156af0 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a37fe700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31a41580d0 0x7f31a41565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a3fff700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c0051c0 0x7f31a40c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31a8b08700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a4156af0 0x7f31a40cc0b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f319c00afd0 tx=0x7f319c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f318bfff700 1 -- 172.21.15.45:0/1345803010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c024670 con 0x7f31a4156af0 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f318bfff700 1 -- 172.21.15.45:0/1345803010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c0247f0 con 0x7f31a4156af0 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a41575f0 con 0x7f31a4156af0 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f318bfff700 1 -- 172.21.15.45:0/1345803010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c024bf0 con 0x7f31a4156af0 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.989+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a40c5900 con 0x7f31a4156af0 2023-10-06T05:15:55.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.990+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a4157df0 con 0x7f31a4156af0 2023-10-06T05:15:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.991+0000 7f318bfff700 1 -- 172.21.15.45:0/1345803010 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f319c024d50 con 0x7f31a4156af0 2023-10-06T05:15:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.991+0000 7f318bfff700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3190066f30 0x7f31900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.991+0000 7f318bfff700 1 -- 172.21.15.45:0/1345803010 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f319c016070 con 0x7f31a4156af0 2023-10-06T05:15:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.991+0000 7f31a3fff700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3190066f30 0x7f31900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.992+0000 7f31a3fff700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3190066f30 0x7f31900693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31940015a0 tx=0x7f3194009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:55.996+0000 7f318bfff700 1 -- 172.21.15.45:0/1345803010 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31a4157df0 con 0x7f31a4156af0 2023-10-06T05:15:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.199+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31a4157d80 con 0x7f31a4156af0 2023-10-06T05:15:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.200+0000 7f318bfff700 1 -- 172.21.15.45:0/1345803010 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18899 (secure 0 0 0) 0x7f319c055e40 con 0x7f31a4156af0 2023-10-06T05:15:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:56.203 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:47.878034+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3190066f30 msgr2=0x7f31900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3190066f30 0x7f31900693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31940015a0 tx=0x7f3194009040 comp rx=0 tx=0).stop 2023-10-06T05:15:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a4156af0 msgr2=0x7f31a40cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a4156af0 0x7f31a40cc0b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f319c00afd0 tx=0x7f319c01c000 comp rx=0 tx=0).stop 2023-10-06T05:15:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 shutdown_connections 2023-10-06T05:15:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3190066f30 0x7f31900693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a4156af0 0x7f31a40cc0b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f318c0051c0 0x7f31a40c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 --2- 172.21.15.45:0/1345803010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31a41580d0 0x7f31a41565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 >> 172.21.15.45:0/1345803010 conn(0x7f31a406f090 msgr2=0x7f31a40cac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 shutdown_connections 2023-10-06T05:15:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.203+0000 7f31aa56b700 1 -- 172.21.15.45:0/1345803010 wait complete. 2023-10-06T05:15:56.219 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:15:56.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:56.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.348+0000 7f888f689700 1 Processor -- start 2023-10-06T05:15:56.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.348+0000 7f888f689700 1 -- start start 2023-10-06T05:15:56.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.349+0000 7f888f689700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88880cc550 0x7f88880cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.349+0000 7f888f689700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880caa70 0x7f88880caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.349+0000 7f888f689700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8888161f40 con 0x7f888815d0b0 2023-10-06T05:15:56.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.349+0000 7f888f689700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8888161330 con 0x7f88880cc550 2023-10-06T05:15:56.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.349+0000 7f888f689700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8888161610 con 0x7f88880caa70 2023-10-06T05:15:56.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.349+0000 7f888d425700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88880cc550 0x7f88880cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.349+0000 7f888d425700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88880cc550 0x7f88880cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44708/0 (socket says 172.21.15.45:44708) 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f888d425700 1 -- 172.21.15.45:0/3445446416 learned_addr learned my addr 172.21.15.45:0/3445446416 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f888dc26700 1 --2- 172.21.15.45:0/3445446416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880caa70 0x7f88880caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f888d425700 1 -- 172.21.15.45:0/3445446416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880caa70 msgr2=0x7f88880caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f888d425700 1 --2- 172.21.15.45:0/3445446416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880caa70 0x7f88880caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f888d425700 1 -- 172.21.15.45:0/3445446416 >> v1:172.21.15.144:6789/0 conn(0x7f888815d0b0 legacy=0x7f888815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f888d425700 1 -- 172.21.15.45:0/3445446416 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88881618f0 con 0x7f88880cc550 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f888d425700 1 --2- 172.21.15.45:0/3445446416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88880cc550 0x7f88880cc920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f887c0099b0 tx=0x7f887c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6c7b60107f802f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f887bfff700 1 -- 172.21.15.45:0/3445446416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c009ea0 con 0x7f88880cc550 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f887bfff700 1 -- 172.21.15.45:0/3445446416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c00b860 con 0x7f88880cc550 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.350+0000 7f887bfff700 1 -- 172.21.15.45:0/3445446416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c00bc00 con 0x7f88880cc550 2023-10-06T05:15:56.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 -- 172.21.15.45:0/3445446416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88880cc550 msgr2=0x7f88880cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 --2- 172.21.15.45:0/3445446416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88880cc550 0x7f88880cc920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f887c0099b0 tx=0x7f887c009250 comp rx=0 tx=0).stop 2023-10-06T05:15:56.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 -- 172.21.15.45:0/3445446416 shutdown_connections 2023-10-06T05:15:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 --2- 172.21.15.45:0/3445446416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880caa70 0x7f88880caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 --2- 172.21.15.45:0/3445446416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88880cc550 0x7f88880cc920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 -- 172.21.15.45:0/3445446416 >> 172.21.15.45:0/3445446416 conn(0x7f88880cdf50 msgr2=0x7f88880ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 -- 172.21.15.45:0/3445446416 shutdown_connections 2023-10-06T05:15:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.351+0000 7f888f689700 1 -- 172.21.15.45:0/3445446416 wait complete. 2023-10-06T05:15:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.352+0000 7f888f689700 1 Processor -- start 2023-10-06T05:15:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.352+0000 7f888f689700 1 -- start start 2023-10-06T05:15:56.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888f689700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88880caa70 0x7f8888148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888f689700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880cc550 0x7f88880c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888f689700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888815d0b0 0x7f88880c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888f689700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880c6b20 con 0x7f888815d0b0 2023-10-06T05:15:56.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888f689700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880c6e00 con 0x7f88880caa70 2023-10-06T05:15:56.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888f689700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88880c70e0 con 0x7f88880cc550 2023-10-06T05:15:56.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888dc26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888815d0b0 0x7f88880c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888dc26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888815d0b0 0x7f88880c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44724/0 (socket says 172.21.15.45:44724) 2023-10-06T05:15:56.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.353+0000 7f888dc26700 1 -- 172.21.15.45:0/3518534022 learned_addr learned my addr 172.21.15.45:0/3518534022 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:56.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888d425700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88880caa70 0x7f8888148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888cc24700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880cc550 0x7f88880c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888dc26700 1 -- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880cc550 msgr2=0x7f88880c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888dc26700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880cc550 0x7f88880c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888dc26700 1 -- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88880caa70 msgr2=0x7f8888148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888dc26700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88880caa70 0x7f8888148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888dc26700 1 -- 172.21.15.45:0/3518534022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0094f0 con 0x7f888815d0b0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888d425700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88880caa70 0x7f8888148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888cc24700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880cc550 0x7f88880c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888dc26700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888815d0b0 0x7f88880c7bc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8880009e90 tx=0x7f888000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f88797fa700 1 -- 172.21.15.45:0/3518534022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888000cc20 con 0x7f888815d0b0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f88797fa700 1 -- 172.21.15.45:0/3518534022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888000f040 con 0x7f888815d0b0 2023-10-06T05:15:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88880c73c0 con 0x7f888815d0b0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f88797fa700 1 -- 172.21.15.45:0/3518534022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880014480 con 0x7f888815d0b0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.354+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8888148750 con 0x7f888815d0b0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.355+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888806f410 con 0x7f888815d0b0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.355+0000 7f88797fa700 1 -- 172.21.15.45:0/3518534022 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8880014830 con 0x7f888815d0b0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.356+0000 7f88797fa700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f886c066f30 0x7f886c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.356+0000 7f88797fa700 1 -- 172.21.15.45:0/3518534022 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f88800856c0 con 0x7f888815d0b0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.356+0000 7f888d425700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f886c066f30 0x7f886c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.357+0000 7f888d425700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f886c066f30 0x7f886c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f887c0099b0 tx=0x7f887c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.361+0000 7f88797fa700 1 -- 172.21.15.45:0/3518534022 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8880055f30 con 0x7f888815d0b0 2023-10-06T05:15:56.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.608+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8888161bd0 con 0x7f888815d0b0 2023-10-06T05:15:56.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.608+0000 7f88797fa700 1 -- 172.21.15.45:0/3518534022 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7f8880055ac0 con 0x7f888815d0b0 2023-10-06T05:15:56.610 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:56.610 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f886c066f30 msgr2=0x7f886c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f886c066f30 0x7f886c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f887c0099b0 tx=0x7f887c017000 comp rx=0 tx=0).stop 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888815d0b0 msgr2=0x7f88880c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888815d0b0 0x7f88880c7bc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8880009e90 tx=0x7f888000b040 comp rx=0 tx=0).stop 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 shutdown_connections 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f886c066f30 0x7f886c0693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888815d0b0 0x7f88880c7bc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88880caa70 0x7f8888148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 --2- 172.21.15.45:0/3518534022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88880cc550 0x7f88880c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 >> 172.21.15.45:0/3518534022 conn(0x7f88880cdf50 msgr2=0x7f88880c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 shutdown_connections 2023-10-06T05:15:56.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.611+0000 7f888f689700 1 -- 172.21.15.45:0/3518534022 wait complete. 2023-10-06T05:15:56.614 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:56.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:15:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.757+0000 7fc4559df700 1 Processor -- start 2023-10-06T05:15:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.757+0000 7fc4559df700 1 -- start start 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc4559df700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4500692d0 0x7fc450159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc4559df700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc45015d040 0x7fc4500aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc4559df700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4500ca6d0 con 0x7fc450159170 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc4559df700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45015d550 con 0x7fc4500692d0 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc4559df700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45015d810 con 0x7fc45015d040 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc44e7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4500692d0 0x7fc450159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc44e7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4500692d0 0x7fc450159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44734/0 (socket says 172.21.15.45:44734) 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc44e7fc700 1 -- 172.21.15.45:0/1893069334 learned_addr learned my addr 172.21.15.45:0/1893069334 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.758+0000 7fc44f7fe700 1 --2- 172.21.15.45:0/1893069334 >> v2:172.21.15.144:3300/0 conn(0x7fc45015d040 0x7fc4500aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44e7fc700 1 -- 172.21.15.45:0/1893069334 >> v2:172.21.15.144:3300/0 conn(0x7fc45015d040 msgr2=0x7fc4500aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44e7fc700 1 --2- 172.21.15.45:0/1893069334 >> v2:172.21.15.144:3300/0 conn(0x7fc45015d040 0x7fc4500aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44e7fc700 1 -- 172.21.15.45:0/1893069334 >> v1:172.21.15.144:6789/0 conn(0x7fc450159170 legacy=0x7fc450068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44e7fc700 1 -- 172.21.15.45:0/1893069334 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45015daf0 con 0x7fc4500692d0 2023-10-06T05:15:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44e7fc700 1 --2- 172.21.15.45:0/1893069334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4500692d0 0x7fc450159540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c70c0 tx=0x7fc444009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7868b119e0e54345 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44dffb700 1 -- 172.21.15.45:0/1893069334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400b860 con 0x7fc4500692d0 2023-10-06T05:15:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44dffb700 1 -- 172.21.15.45:0/1893069334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44400b9c0 con 0x7fc4500692d0 2023-10-06T05:15:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.759+0000 7fc44dffb700 1 -- 172.21.15.45:0/1893069334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400bd60 con 0x7fc4500692d0 2023-10-06T05:15:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 -- 172.21.15.45:0/1893069334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4500692d0 msgr2=0x7fc450159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 --2- 172.21.15.45:0/1893069334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4500692d0 0x7fc450159540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c70c0 tx=0x7fc444009250 comp rx=0 tx=0).stop 2023-10-06T05:15:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 -- 172.21.15.45:0/1893069334 shutdown_connections 2023-10-06T05:15:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 --2- 172.21.15.45:0/1893069334 >> v2:172.21.15.144:3300/0 conn(0x7fc45015d040 0x7fc4500aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 --2- 172.21.15.45:0/1893069334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4500692d0 0x7fc450159540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 -- 172.21.15.45:0/1893069334 >> 172.21.15.45:0/1893069334 conn(0x7fc45006f7c0 msgr2=0x7fc4500d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 -- 172.21.15.45:0/1893069334 shutdown_connections 2023-10-06T05:15:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.760+0000 7fc4559df700 1 -- 172.21.15.45:0/1893069334 wait complete. 2023-10-06T05:15:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.761+0000 7fc4559df700 1 Processor -- start 2023-10-06T05:15:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.762+0000 7fc4559df700 1 -- start start 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc4559df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4500692d0 0x7fc4500af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc4559df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc450159170 0x7fc4500b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc4559df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45015d040 0x7fc4500b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc4559df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45016a520 con 0x7fc45015d040 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc4559df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc450157640 con 0x7fc450159170 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc4559df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45016a240 con 0x7fc4500692d0 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45015d040 0x7fc4500b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45015d040 0x7fc4500b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44742/0 (socket says 172.21.15.45:44742) 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 -- 172.21.15.45:0/1643618956 learned_addr learned my addr 172.21.15.45:0/1643618956 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:56.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44effd700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4500692d0 0x7fc4500af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44e7fc700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc450159170 0x7fc4500b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 -- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4500692d0 msgr2=0x7fc4500af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4500692d0 0x7fc4500af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 -- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc450159170 msgr2=0x7fc4500b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc450159170 0x7fc4500b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 -- 172.21.15.45:0/1643618956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4440094f0 con 0x7fc45015d040 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc44f7fe700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45015d040 0x7fc4500b4290 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc448007f20 tx=0x7fc448009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc436ffd700 1 -- 172.21.15.45:0/1643618956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800eb90 con 0x7fc45015d040 2023-10-06T05:15:56.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc436ffd700 1 -- 172.21.15.45:0/1643618956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800ed10 con 0x7fc45015d040 2023-10-06T05:15:56.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45016cd00 con 0x7fc45015d040 2023-10-06T05:15:56.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.763+0000 7fc436ffd700 1 -- 172.21.15.45:0/1643618956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448014670 con 0x7fc45015d040 2023-10-06T05:15:56.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.764+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45016d2c0 con 0x7fc45015d040 2023-10-06T05:15:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.765+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc450063b50 con 0x7fc45015d040 2023-10-06T05:15:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.765+0000 7fc436ffd700 1 -- 172.21.15.45:0/1643618956 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4480149e0 con 0x7fc45015d040 2023-10-06T05:15:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.765+0000 7fc436ffd700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.765+0000 7fc436ffd700 1 -- 172.21.15.45:0/1643618956 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fc448084f40 con 0x7fc45015d040 2023-10-06T05:15:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.765+0000 7fc44effd700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.766+0000 7fc44effd700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc440003a80 tx=0x7fc440007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:56.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:56.770+0000 7fc436ffd700 1 -- 172.21.15.45:0/1643618956 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc448051d20 con 0x7fc45015d040 2023-10-06T05:15:57.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.017+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc45015ddd0 con 0x7fc45015d040 2023-10-06T05:15:57.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.017+0000 7fc436ffd700 1 -- 172.21.15.45:0/1643618956 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4593 (secure 0 0 0) 0x7fc448055340 con 0x7fc45015d040 2023-10-06T05:15:57.019 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:57.019 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/775785057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":775785057},{"type":"v1","addr":"172.21.15.144:6836","nonce":775785057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/951239104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":951239104},{"type":"v1","addr":"172.21.15.45:6834","nonce":951239104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4202667972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4202667972},{"type":"v1","addr":"172.21.15.45:6835","nonce":4202667972}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:14:59.369246+0000","modified":"2023-10-06T05:15:18.783356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1292454917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1292454917},{"type":"v1","addr":"172.21.15.144:6837","nonce":1292454917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:15:57.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 msgr2=0x7fc43c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc440003a80 tx=0x7fc440007ec0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45015d040 msgr2=0x7fc4500b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45015d040 0x7fc4500b4290 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc448007f20 tx=0x7fc448009860 comp rx=0 tx=0).stop 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 shutdown_connections 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45015d040 0x7fc4500b4290 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4500692d0 0x7fc4500af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 --2- 172.21.15.45:0/1643618956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc450159170 0x7fc4500b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.019+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 >> 172.21.15.45:0/1643618956 conn(0x7fc45006f7c0 msgr2=0x7fc4500ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.020+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 shutdown_connections 2023-10-06T05:15:57.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.020+0000 7fc4559df700 1 -- 172.21.15.45:0/1643618956 wait complete. 2023-10-06T05:15:57.023 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2023-10-06T05:15:57.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:15:57.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.165+0000 7f72ebc37700 1 Processor -- start 2023-10-06T05:15:57.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.165+0000 7f72ebc37700 1 -- start start 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72ebc37700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e4155c70 0x7f72e40689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72ebc37700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72e406e6e0 0x7f72e406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72ebc37700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e406f050 con 0x7f72e4156040 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72ebc37700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e4068ee0 con 0x7f72e406e6e0 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72ebc37700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4067620 con 0x7f72e4155c70 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72ea1d4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72e406e6e0 0x7f72e406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72e91d2700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f72e4156040 0x7f72e40ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38896/0 (socket says 172.21.15.45:38896) 2023-10-06T05:15:57.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.166+0000 7f72e91d2700 1 -- 172.21.15.45:0/1548855363 learned_addr learned my addr 172.21.15.45:0/1548855363 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72e99d3700 1 --2- 172.21.15.45:0/1548855363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e4155c70 0x7f72e40689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72e89d1700 1 -- 172.21.15.45:0/1548855363 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585383042 0 0) 0x7f72e406f050 con 0x7f72e4156040 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72e89d1700 1 -- 172.21.15.45:0/1548855363 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d0003570 con 0x7f72e4156040 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72ea1d4700 1 -- 172.21.15.45:0/1548855363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e4155c70 msgr2=0x7f72e40689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72ea1d4700 1 --2- 172.21.15.45:0/1548855363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e4155c70 0x7f72e40689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72ea1d4700 1 -- 172.21.15.45:0/1548855363 >> v1:172.21.15.45:6789/0 conn(0x7f72e4156040 legacy=0x7f72e40ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72ea1d4700 1 -- 172.21.15.45:0/1548855363 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4067900 con 0x7f72e406e6e0 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72ea1d4700 1 --2- 172.21.15.45:0/1548855363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e406e6e0 0x7f72e406eb10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009e60 tx=0x7f72dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef7c74afe9ed459 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72e89d1700 1 -- 172.21.15.45:0/1548855363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00e070 con 0x7f72e406e6e0 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72e89d1700 1 -- 172.21.15.45:0/1548855363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00bab0 con 0x7f72e406e6e0 2023-10-06T05:15:57.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.167+0000 7f72e89d1700 1 -- 172.21.15.45:0/1548855363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00be70 con 0x7f72e406e6e0 2023-10-06T05:15:57.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 -- 172.21.15.45:0/1548855363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e406e6e0 msgr2=0x7f72e406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 --2- 172.21.15.45:0/1548855363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e406e6e0 0x7f72e406eb10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009e60 tx=0x7f72dc009780 comp rx=0 tx=0).stop 2023-10-06T05:15:57.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 -- 172.21.15.45:0/1548855363 shutdown_connections 2023-10-06T05:15:57.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 --2- 172.21.15.45:0/1548855363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e406e6e0 0x7f72e406eb10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 --2- 172.21.15.45:0/1548855363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e4155c70 0x7f72e40689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 -- 172.21.15.45:0/1548855363 >> 172.21.15.45:0/1548855363 conn(0x7f72e40cc5a0 msgr2=0x7f72e40cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 -- 172.21.15.45:0/1548855363 shutdown_connections 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.168+0000 7f72ebc37700 1 -- 172.21.15.45:0/1548855363 wait complete. 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.169+0000 7f72ebc37700 1 Processor -- start 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.169+0000 7f72ebc37700 1 -- start start 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ebc37700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e406e6e0 0x7f72e4166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ebc37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72e4155c70 0x7f72e40a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ebc37700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e4156040 0x7f72e40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ebc37700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e416a610 con 0x7f72e4156040 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ebc37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40c72c0 con 0x7f72e4155c70 2023-10-06T05:15:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ebc37700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e41674d0 con 0x7f72e406e6e0 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e4156040 0x7f72e40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e4156040 0x7f72e40b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44754/0 (socket says 172.21.15.45:44754) 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 -- 172.21.15.45:0/604434201 learned_addr learned my addr 172.21.15.45:0/604434201 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72e99d3700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e406e6e0 0x7f72e4166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72e91d2700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72e4155c70 0x7f72e40a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 -- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e406e6e0 msgr2=0x7f72e4166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e406e6e0 0x7f72e4166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 -- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72e4155c70 msgr2=0x7f72e40a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72e4155c70 0x7f72e40a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72ea1d4700 1 -- 172.21.15.45:0/604434201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc009a20 con 0x7f72e4156040 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72e91d2700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72e4155c70 0x7f72e40a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.170+0000 7f72e99d3700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e406e6e0 0x7f72e4166ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:15:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.171+0000 7f72ea1d4700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e4156040 0x7f72e40b1510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b610 tx=0x7f72dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.171+0000 7f72d9ffb700 1 -- 172.21.15.45:0/604434201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc027470 con 0x7f72e4156040 2023-10-06T05:15:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.171+0000 7f72d9ffb700 1 -- 172.21.15.45:0/604434201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc0275f0 con 0x7f72e4156040 2023-10-06T05:15:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.171+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e41677b0 con 0x7f72e4156040 2023-10-06T05:15:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.171+0000 7f72d9ffb700 1 -- 172.21.15.45:0/604434201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0279b0 con 0x7f72e4156040 2023-10-06T05:15:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.171+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e4167d70 con 0x7f72e4156040 2023-10-06T05:15:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.172+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e416a780 con 0x7f72e4156040 2023-10-06T05:15:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.172+0000 7f72d9ffb700 1 -- 172.21.15.45:0/604434201 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72dc024070 con 0x7f72e4156040 2023-10-06T05:15:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.173+0000 7f72d9ffb700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.173+0000 7f72d9ffb700 1 -- 172.21.15.45:0/604434201 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f72dc087410 con 0x7f72e4156040 2023-10-06T05:15:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.173+0000 7f72e99d3700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.174+0000 7f72e99d3700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72e0006f00 tx=0x7f72e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.177+0000 7f72d9ffb700 1 -- 172.21.15.45:0/604434201 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72e416a780 con 0x7f72e4156040 2023-10-06T05:15:57.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.380+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72e416a750 con 0x7f72e4156040 2023-10-06T05:15:57.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.381+0000 7f72d9ffb700 1 -- 172.21.15.45:0/604434201 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18899 (secure 0 0 0) 0x7f72dc0541f0 con 0x7f72e4156040 2023-10-06T05:15:57.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:15:57.384 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:15:47.878034+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:14:57.253547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:14:58.338382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T05:15:01.988547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:15:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72c8066f30 msgr2=0x7f72c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72e0006f00 tx=0x7f72e0008040 comp rx=0 tx=0).stop 2023-10-06T05:15:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e4156040 msgr2=0x7f72e40b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e4156040 0x7f72e40b1510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b610 tx=0x7f72dc019000 comp rx=0 tx=0).stop 2023-10-06T05:15:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 shutdown_connections 2023-10-06T05:15:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72e4156040 0x7f72e40b1510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72e406e6e0 0x7f72e4166ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 --2- 172.21.15.45:0/604434201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72e4155c70 0x7f72e40a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 >> 172.21.15.45:0/604434201 conn(0x7f72e40cc5a0 msgr2=0x7f72e40cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 shutdown_connections 2023-10-06T05:15:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.384+0000 7f72ebc37700 1 -- 172.21.15.45:0/604434201 wait complete. 2023-10-06T05:15:57.400 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:15:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.531+0000 7f406aea0700 1 Processor -- start 2023-10-06T05:15:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.531+0000 7f406aea0700 1 -- start start 2023-10-06T05:15:57.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.532+0000 7f406aea0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 0x7f406406a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.532+0000 7f406aea0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f406406add0 0x7f4064068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.532+0000 7f406aea0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40640676c0 con 0x7f4064153650 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.532+0000 7f406aea0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406406b250 con 0x7f406406add0 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.532+0000 7f406aea0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40640679a0 con 0x7f4064153a20 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.532+0000 7f4068c3c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4064153650 0x7f406415a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38902/0 (socket says 172.21.15.45:38902) 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.532+0000 7f4068c3c700 1 -- 172.21.15.45:0/750682399 learned_addr learned my addr 172.21.15.45:0/750682399 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f4063fff700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 0x7f406406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f406943d700 1 --2- 172.21.15.45:0/750682399 >> v2:172.21.15.144:3300/0 conn(0x7f406406add0 0x7f4064068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042891208 0 0) 0x7f40640676c0 con 0x7f4064153650 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4050003570 con 0x7f4064153650 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3239276161 0 0) 0x7f4050003570 con 0x7f4064153650 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 msgr2=0x7f406406a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 0x7f406406a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 >> v2:172.21.15.144:3300/0 conn(0x7f406406add0 msgr2=0x7f4064068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 --2- 172.21.15.45:0/750682399 >> v2:172.21.15.144:3300/0 conn(0x7f406406add0 0x7f4064068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064067c80 con 0x7f4064153650 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f4054001470 con 0x7f4064153650 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 >> v1:172.21.15.45:6789/0 conn(0x7f4064153650 legacy=0x7f406415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050003a60 0x7f4050003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4050004810 0x7f4050004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 0x7f40500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406406b250 con 0x7f4050003a60 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40640679a0 con 0x7f4050004810 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405000ba60 con 0x7f4064153a20 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f4068c3c700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050003a60 0x7f4050003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f4063fff700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4050004810 0x7f4050004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.533+0000 7f406943d700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 0x7f40500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f4068c3c700 1 -- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 msgr2=0x7f40500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f4068c3c700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4064153a20 0x7f40500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f4068c3c700 1 -- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4050004810 msgr2=0x7f4050004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f4068c3c700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4050004810 0x7f4050004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f4068c3c700 1 -- 172.21.15.45:0/750682399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050003570 con 0x7f4050003a60 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f4068c3c700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050003a60 0x7f4050003e70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f405400c3f0 tx=0x7f4054001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99fb82bc36c3a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400cc80 con 0x7f4050003a60 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405400ce20 con 0x7f4050003a60 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f40637fe700 1 -- 172.21.15.45:0/750682399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054005b70 con 0x7f4050003a60 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f406aea0700 1 -- 172.21.15.45:0/750682399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050003a60 msgr2=0x7f4050003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f406aea0700 1 --2- 172.21.15.45:0/750682399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4050003a60 0x7f4050003e70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f405400c3f0 tx=0x7f4054001e90 comp rx=0 tx=0).stop 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f406943d700 1 -- 172.21.15.45:0/750682399 reap_dead start 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f406aea0700 1 -- 172.21.15.45:0/750682399 shutdown_connections 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.534+0000 7f406aea0700 1 -- 172.21.15.45:0/750682399 >> 172.21.15.45:0/750682399 conn(0x7f40640cc5a0 msgr2=0x7f40640cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.535+0000 7f406aea0700 1 -- 172.21.15.45:0/750682399 shutdown_connections 2023-10-06T05:15:57.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.535+0000 7f406aea0700 1 -- 172.21.15.45:0/750682399 wait complete. 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.536+0000 7f406aea0700 1 Processor -- start 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.536+0000 7f406aea0700 1 -- start start 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f406aea0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4050003a60 0x7f4064154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f406aea0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40641556b0 0x7f4064152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f406aea0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f406406e6e0 0x7f406406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f406aea0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406406f0a0 con 0x7f40641556b0 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f406aea0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4064156cc0 con 0x7f406406e6e0 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f406aea0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4064155c20 con 0x7f4050003a60 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40641556b0 0x7f4064152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40641556b0 0x7f4064152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44774/0 (socket says 172.21.15.45:44774) 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 -- 172.21.15.45:0/3353290067 learned_addr learned my addr 172.21.15.45:0/3353290067 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4068c3c700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4050003a60 0x7f4064154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f406943d700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f406406e6e0 0x7f406406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 -- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4050003a60 msgr2=0x7f4064154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4050003a60 0x7f4064154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 -- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f406406e6e0 msgr2=0x7f406406eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f406406e6e0 0x7f406406eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 -- 172.21.15.45:0/3353290067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064155f00 con 0x7f40641556b0 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.537+0000 7f4063fff700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40641556b0 0x7f4064152900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f404c011b40 tx=0x7f404c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.538+0000 7f4060ff9700 1 -- 172.21.15.45:0/3353290067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00f9f0 con 0x7f40641556b0 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.538+0000 7f4060ff9700 1 -- 172.21.15.45:0/3353290067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c00fb50 con 0x7f40641556b0 2023-10-06T05:15:57.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.538+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40641561e0 con 0x7f40641556b0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.538+0000 7f4060ff9700 1 -- 172.21.15.45:0/3353290067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00fed0 con 0x7f40641556b0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.538+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40641567a0 con 0x7f40641556b0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.539+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40641569c0 con 0x7f40641556b0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.539+0000 7f4060ff9700 1 -- 172.21.15.45:0/3353290067 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f404c014650 con 0x7f40641556b0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.540+0000 7f4060ff9700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4044066f30 0x7f40440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.540+0000 7f4060ff9700 1 -- 172.21.15.45:0/3353290067 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f404c018030 con 0x7f40641556b0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.540+0000 7f4068c3c700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4044066f30 0x7f40440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.540+0000 7f4068c3c700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4044066f30 0x7f40440693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f405400c3f0 tx=0x7f4054015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.544+0000 7f4060ff9700 1 -- 172.21.15.45:0/3353290067 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f404c055cd0 con 0x7f40641556b0 2023-10-06T05:15:57.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.795+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f40640cc0e0 con 0x7f40641556b0 2023-10-06T05:15:57.797 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:15:57.795+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:15:57.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.799+0000 7f4060ff9700 1 -- 172.21.15.45:0/3353290067 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f404c055860 con 0x7f40641556b0 2023-10-06T05:15:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.801+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4044066f30 msgr2=0x7f40440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.801+0000 7f406aea0700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4044066f30 0x7f40440693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f405400c3f0 tx=0x7f4054015000 comp rx=0 tx=0).stop 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.801+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40641556b0 msgr2=0x7f4064152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.801+0000 7f406aea0700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40641556b0 0x7f4064152900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f404c011b40 tx=0x7f404c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 shutdown_connections 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4044066f30 0x7f40440693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f406406e6e0 0x7f406406eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4050003a60 0x7f4064154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 --2- 172.21.15.45:0/3353290067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40641556b0 0x7f4064152900 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 >> 172.21.15.45:0/3353290067 conn(0x7f40640cc5a0 msgr2=0x7f40640cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 shutdown_connections 2023-10-06T05:15:57.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.802+0000 7f406aea0700 1 -- 172.21.15.45:0/3353290067 wait complete. 2023-10-06T05:15:57.805 INFO:tasks.ceph.mds.d.smithi144.stderr: -4311> 2023-10-06T05:15:03.957+0000 7f6148125700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:15:57.805 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:15:57.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:15:57.834 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:15:57.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.952+0000 7fb5722cd700 1 Processor -- start 2023-10-06T05:15:57.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.952+0000 7fb5722cd700 1 -- start start 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb5722cd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb56c1406e0 0x7fb56c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb5722cd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb56c06cd60 0x7fb56c0acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb5722cd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56c0c9940 con 0x7fb56c0ad4e0 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb5722cd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c06d230 con 0x7fb56c06cd60 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb5722cd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c06d510 con 0x7fb56c1406e0 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb56b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb56c06cd60 0x7fb56c0acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb57086a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb56c0ad4e0 0x7fb56c0c9dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38906/0 (socket says 172.21.15.45:38906) 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb57086a700 1 -- 172.21.15.45:0/2467853012 learned_addr learned my addr 172.21.15.45:0/2467853012 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.953+0000 7fb56bfff700 1 --2- 172.21.15.45:0/2467853012 >> v2:172.21.15.144:3300/0 conn(0x7fb56c1406e0 0x7fb56c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56b7fe700 1 -- 172.21.15.45:0/2467853012 >> v2:172.21.15.144:3300/0 conn(0x7fb56c1406e0 msgr2=0x7fb56c06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:57.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56b7fe700 1 --2- 172.21.15.45:0/2467853012 >> v2:172.21.15.144:3300/0 conn(0x7fb56c1406e0 0x7fb56c06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56b7fe700 1 -- 172.21.15.45:0/2467853012 >> v1:172.21.15.45:6789/0 conn(0x7fb56c0ad4e0 legacy=0x7fb56c0c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56b7fe700 1 -- 172.21.15.45:0/2467853012 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c06d7f0 con 0x7fb56c06cd60 2023-10-06T05:15:57.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56affd700 1 -- 172.21.15.45:0/2467853012 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640443444 0 0) 0x7fb56c0c9940 con 0x7fb56c0ad4e0 2023-10-06T05:15:57.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56b7fe700 1 --2- 172.21.15.45:0/2467853012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c06cd60 0x7fb56c0acfa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb56c06ad90 tx=0x7fb5540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad8ea83a2b76e7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56affd700 1 -- 172.21.15.45:0/2467853012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554011630 con 0x7fb56c06cd60 2023-10-06T05:15:57.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56affd700 1 -- 172.21.15.45:0/2467853012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb554011790 con 0x7fb56c06cd60 2023-10-06T05:15:57.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.954+0000 7fb56affd700 1 -- 172.21.15.45:0/2467853012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554011b30 con 0x7fb56c06cd60 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 -- 172.21.15.45:0/2467853012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c06cd60 msgr2=0x7fb56c0acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 --2- 172.21.15.45:0/2467853012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c06cd60 0x7fb56c0acfa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb56c06ad90 tx=0x7fb5540091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 -- 172.21.15.45:0/2467853012 shutdown_connections 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 --2- 172.21.15.45:0/2467853012 >> v2:172.21.15.144:3300/0 conn(0x7fb56c1406e0 0x7fb56c06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 --2- 172.21.15.45:0/2467853012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c06cd60 0x7fb56c0acfa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 -- 172.21.15.45:0/2467853012 >> 172.21.15.45:0/2467853012 conn(0x7fb56c0691c0 msgr2=0x7fb56c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 -- 172.21.15.45:0/2467853012 shutdown_connections 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.955+0000 7fb5722cd700 1 -- 172.21.15.45:0/2467853012 wait complete. 2023-10-06T05:15:57.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.956+0000 7fb5722cd700 1 Processor -- start 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.956+0000 7fb5722cd700 1 -- start start 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb5722cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56c06cd60 0x7fb56c13f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb5722cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56c0ad4e0 0x7fb56c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb5722cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c1406e0 0x7fb56c0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb5722cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c13fdb0 con 0x7fb56c1406e0 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb5722cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c0c6200 con 0x7fb56c06cd60 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb5722cd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c0c5f00 con 0x7fb56c0ad4e0 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c1406e0 0x7fb56c0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c1406e0 0x7fb56c0c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44786/0 (socket says 172.21.15.45:44786) 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 -- 172.21.15.45:0/1993215661 learned_addr learned my addr 172.21.15.45:0/1993215661 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb56bfff700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56c06cd60 0x7fb56c13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb56b7fe700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56c0ad4e0 0x7fb56c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 -- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56c0ad4e0 msgr2=0x7fb56c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56c0ad4e0 0x7fb56c15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 -- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56c06cd60 msgr2=0x7fb56c13f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56c06cd60 0x7fb56c13f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.957+0000 7fb57086a700 1 -- 172.21.15.45:0/1993215661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554009420 con 0x7fb56c1406e0 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.958+0000 7fb57086a700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c1406e0 0x7fb56c0c6690 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ccf0 tx=0x7fb564001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.958+0000 7fb563fff700 1 -- 172.21.15.45:0/1993215661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400aaf0 con 0x7fb56c1406e0 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.958+0000 7fb563fff700 1 -- 172.21.15.45:0/1993215661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56400ac50 con 0x7fb56c1406e0 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.958+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c0c3ef0 con 0x7fb56c1406e0 2023-10-06T05:15:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.958+0000 7fb563fff700 1 -- 172.21.15.45:0/1993215661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640070f0 con 0x7fb56c1406e0 2023-10-06T05:15:57.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.958+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c0c44b0 con 0x7fb56c1406e0 2023-10-06T05:15:57.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.959+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c06e2d0 con 0x7fb56c1406e0 2023-10-06T05:15:57.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.959+0000 7fb563fff700 1 -- 172.21.15.45:0/1993215661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5640074c0 con 0x7fb56c1406e0 2023-10-06T05:15:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.960+0000 7fb563fff700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c066f30 0x7fb54c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.960+0000 7fb563fff700 1 -- 172.21.15.45:0/1993215661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6953+0+0 (secure 0 0 0) 0x7fb564084ea0 con 0x7fb56c1406e0 2023-10-06T05:15:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.960+0000 7fb56bfff700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c066f30 0x7fb54c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.960+0000 7fb56bfff700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c066f30 0x7fb54c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0b1cd0 tx=0x7fb55c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:57.964+0000 7fb563fff700 1 -- 172.21.15.45:0/1993215661 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb564051cb0 con 0x7fb56c1406e0 2023-10-06T05:15:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.217+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb56c04bb40 con 0x7fb56c1406e0 2023-10-06T05:15:58.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.801+0000 7fb563fff700 1 -- 172.21.15.45:0/1993215661 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5640552d0 con 0x7fb56c1406e0 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c066f30 msgr2=0x7fb54c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c066f30 0x7fb54c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0b1cd0 tx=0x7fb55c008040 comp rx=0 tx=0).stop 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c1406e0 msgr2=0x7fb56c0c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c1406e0 0x7fb56c0c6690 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ccf0 tx=0x7fb564001a80 comp rx=0 tx=0).stop 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 shutdown_connections 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c066f30 0x7fb54c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c1406e0 0x7fb56c0c6690 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56c06cd60 0x7fb56c13f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 --2- 172.21.15.45:0/1993215661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56c0ad4e0 0x7fb56c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 >> 172.21.15.45:0/1993215661 conn(0x7fb56c0691c0 msgr2=0x7fb56c0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:58.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 shutdown_connections 2023-10-06T05:15:58.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.804+0000 7fb5722cd700 1 -- 172.21.15.45:0/1993215661 wait complete. 2023-10-06T05:15:58.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:15:58.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.961+0000 7f965cee7700 1 Processor -- start 2023-10-06T05:15:58.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.961+0000 7f965cee7700 1 -- start start 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.962+0000 7f965cee7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96581535e0 0x7f96581539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.962+0000 7f965cee7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9658159f70 0x7f96580ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.962+0000 7f965cee7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96580c9230 con 0x7f9658068730 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.962+0000 7f965cee7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96580cc4b0 con 0x7f96581535e0 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.962+0000 7f965cee7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965815a420 con 0x7f9658159f70 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9658159f70 0x7f96580ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9658159f70 0x7f96580ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:50062/0 (socket says 172.21.15.45:50062) 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 -- 172.21.15.45:0/2202585112 learned_addr learned my addr 172.21.15.45:0/2202585112 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f965659c700 1 --2- 172.21.15.45:0/2202585112 >> v2:172.21.15.144:3300/0 conn(0x7f96581535e0 0x7f96581539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 -- 172.21.15.45:0/2202585112 >> v2:172.21.15.144:3300/0 conn(0x7f96581535e0 msgr2=0x7f96581539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:15:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 --2- 172.21.15.45:0/2202585112 >> v2:172.21.15.144:3300/0 conn(0x7f96581535e0 0x7f96581539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 -- 172.21.15.45:0/2202585112 >> v1:172.21.15.144:6789/0 conn(0x7f9658068730 legacy=0x7f9658159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 -- 172.21.15.45:0/2202585112 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965815a700 con 0x7f9658159f70 2023-10-06T05:15:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f9656d9d700 1 --2- 172.21.15.45:0/2202585112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9658159f70 0x7f96580ca770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9650009f20 tx=0x7f9650009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d8277efab1baa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f965559a700 1 -- 172.21.15.45:0/2202585112 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965000e070 con 0x7f9658159f70 2023-10-06T05:15:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f965559a700 1 -- 172.21.15.45:0/2202585112 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965000b9d0 con 0x7f9658159f70 2023-10-06T05:15:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.963+0000 7f965559a700 1 -- 172.21.15.45:0/2202585112 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500092e0 con 0x7f9658159f70 2023-10-06T05:15:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 -- 172.21.15.45:0/2202585112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9658159f70 msgr2=0x7f96580ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 --2- 172.21.15.45:0/2202585112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9658159f70 0x7f96580ca770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9650009f20 tx=0x7f9650009840 comp rx=0 tx=0).stop 2023-10-06T05:15:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 -- 172.21.15.45:0/2202585112 shutdown_connections 2023-10-06T05:15:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 --2- 172.21.15.45:0/2202585112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9658159f70 0x7f96580ca770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 --2- 172.21.15.45:0/2202585112 >> v2:172.21.15.144:3300/0 conn(0x7f96581535e0 0x7f96581539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 -- 172.21.15.45:0/2202585112 >> 172.21.15.45:0/2202585112 conn(0x7f96580680b0 msgr2=0x7f96580cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:58.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 -- 172.21.15.45:0/2202585112 shutdown_connections 2023-10-06T05:15:58.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.964+0000 7f965cee7700 1 -- 172.21.15.45:0/2202585112 wait complete. 2023-10-06T05:15:58.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.965+0000 7f965cee7700 1 Processor -- start 2023-10-06T05:15:58.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.965+0000 7f965cee7700 1 -- start start 2023-10-06T05:15:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f965cee7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9658068730 0x7f965814b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f965cee7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96581535e0 0x7f965814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f965cee7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9658159f70 0x7f96580b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f965cee7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c66b0 con 0x7f9658159f70 2023-10-06T05:15:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f965cee7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c7d50 con 0x7f9658068730 2023-10-06T05:15:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f965cee7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c5b70 con 0x7f96581535e0 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f9656d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9658159f70 0x7f96580b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f9656d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9658159f70 0x7f96580b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44796/0 (socket says 172.21.15.45:44796) 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.966+0000 7f9656d9d700 1 -- 172.21.15.45:0/3513860579 learned_addr learned my addr 172.21.15.45:0/3513860579 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f965659c700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9658068730 0x7f965814b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9655d9b700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96581535e0 0x7f965814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9656d9d700 1 -- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96581535e0 msgr2=0x7f965814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9656d9d700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96581535e0 0x7f965814ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9656d9d700 1 -- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9658068730 msgr2=0x7f965814b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9656d9d700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9658068730 0x7f965814b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9656d9d700 1 -- 172.21.15.45:0/3513860579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650009ae0 con 0x7f9658159f70 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9655d9b700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96581535e0 0x7f965814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:15:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f9656d9d700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9658159f70 0x7f96580b3030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f965000b610 tx=0x7f9650018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f964e7fc700 1 -- 172.21.15.45:0/3513860579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650007b70 con 0x7f9658159f70 2023-10-06T05:15:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f964e7fc700 1 -- 172.21.15.45:0/3513860579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9650007cd0 con 0x7f9658159f70 2023-10-06T05:15:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96580c5e10 con 0x7f9658159f70 2023-10-06T05:15:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f964e7fc700 1 -- 172.21.15.45:0/3513860579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965001d650 con 0x7f9658159f70 2023-10-06T05:15:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.967+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96580c63d0 con 0x7f9658159f70 2023-10-06T05:15:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.968+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965806a800 con 0x7f9658159f70 2023-10-06T05:15:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.969+0000 7f964e7fc700 1 -- 172.21.15.45:0/3513860579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9650003040 con 0x7f9658159f70 2023-10-06T05:15:58.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.969+0000 7f964e7fc700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9638066f30 0x7f96380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:15:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.969+0000 7f964e7fc700 1 -- 172.21.15.45:0/3513860579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6953+0+0 (secure 0 0 0) 0x7f96500867d0 con 0x7f9658159f70 2023-10-06T05:15:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.969+0000 7f965659c700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9638066f30 0x7f96380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:15:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.970+0000 7f965659c700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9638066f30 0x7f96380693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9640009db0 tx=0x7f96400091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:15:58.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:58.974+0000 7f964e7fc700 1 -- 172.21.15.45:0/3513860579 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9650053560 con 0x7f9658159f70 2023-10-06T05:15:59.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.178+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f965804c380 con 0x7f9658159f70 2023-10-06T05:15:59.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.924+0000 7f964e7fc700 1 -- 172.21.15.45:0/3513860579 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9650056b80 con 0x7f9658159f70 2023-10-06T05:15:59.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9638066f30 msgr2=0x7f96380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9638066f30 0x7f96380693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9640009db0 tx=0x7f96400091a0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9658159f70 msgr2=0x7f96580b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:15:59.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9658159f70 0x7f96580b3030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f965000b610 tx=0x7f9650018000 comp rx=0 tx=0).stop 2023-10-06T05:15:59.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 shutdown_connections 2023-10-06T05:15:59.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9638066f30 0x7f96380693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9658159f70 0x7f96580b3030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9658068730 0x7f965814b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 --2- 172.21.15.45:0/3513860579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96581535e0 0x7f965814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:15:59.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.929+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 >> 172.21.15.45:0/3513860579 conn(0x7f96580680b0 msgr2=0x7f96580cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:15:59.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.931+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 shutdown_connections 2023-10-06T05:15:59.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:15:59.931+0000 7f965cee7700 1 -- 172.21.15.45:0/3513860579 wait complete. 2023-10-06T05:15:59.934 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:15:59.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:16:00.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.087+0000 7f93c51ee700 1 Processor -- start 2023-10-06T05:16:00.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.087+0000 7f93c51ee700 1 -- start start 2023-10-06T05:16:00.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93c51ee700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c01536e0 0x7f93c015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93c51ee700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93c0069380 0x7f93c00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93c51ee700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c00cf8c0 con 0x7f93c00cfd50 2023-10-06T05:16:00.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93c51ee700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c015a710 con 0x7f93c0069380 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93c51ee700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c00cecb0 con 0x7f93c01536e0 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93c0069380 0x7f93c00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93c0069380 0x7f93c00b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44800/0 (socket says 172.21.15.45:44800) 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 -- 172.21.15.45:0/2139104467 learned_addr learned my addr 172.21.15.45:0/2139104467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bed9d700 1 --2- 172.21.15.45:0/2139104467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c01536e0 0x7f93c015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 -- 172.21.15.45:0/2139104467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c01536e0 msgr2=0x7f93c015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 --2- 172.21.15.45:0/2139104467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c01536e0 0x7f93c015a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 -- 172.21.15.45:0/2139104467 >> v1:172.21.15.144:6789/0 conn(0x7f93c00cfd50 legacy=0x7f93c0068c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 -- 172.21.15.45:0/2139104467 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c00cef90 con 0x7f93c0069380 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bf59e700 1 --2- 172.21.15.45:0/2139104467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c00b2f20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009f20 tx=0x7f93b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341d064f6a9f3a3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:00.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.088+0000 7f93bdd9b700 1 -- 172.21.15.45:0/2139104467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800e070 con 0x7f93c0069380 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93bdd9b700 1 -- 172.21.15.45:0/2139104467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b800ba40 con 0x7f93c0069380 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93bdd9b700 1 -- 172.21.15.45:0/2139104467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b80092e0 con 0x7f93c0069380 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 -- 172.21.15.45:0/2139104467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 msgr2=0x7f93c00b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 --2- 172.21.15.45:0/2139104467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c00b2f20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009f20 tx=0x7f93b8009840 comp rx=0 tx=0).stop 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 -- 172.21.15.45:0/2139104467 shutdown_connections 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 --2- 172.21.15.45:0/2139104467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c00b2f20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 --2- 172.21.15.45:0/2139104467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c01536e0 0x7f93c015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 -- 172.21.15.45:0/2139104467 >> 172.21.15.45:0/2139104467 conn(0x7f93c006f820 msgr2=0x7f93c00b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 -- 172.21.15.45:0/2139104467 shutdown_connections 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.089+0000 7f93c51ee700 1 -- 172.21.15.45:0/2139104467 wait complete. 2023-10-06T05:16:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.090+0000 7f93c51ee700 1 Processor -- start 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93c51ee700 1 -- start start 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93c51ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c013aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93c51ee700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c00cfd50 0x7f93c014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93c51ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c01536e0 0x7f93c01540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93c51ee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0155830 con 0x7f93c0069380 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93c51ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c013af40 con 0x7f93c01536e0 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93c51ee700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0158ea0 con 0x7f93c00cfd50 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93bed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c013aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93bed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c013aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44802/0 (socket says 172.21.15.45:44802) 2023-10-06T05:16:00.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93bed9d700 1 -- 172.21.15.45:0/945572710 learned_addr learned my addr 172.21.15.45:0/945572710 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93bf59e700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c01536e0 0x7f93c01540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.091+0000 7f93be59c700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c00cfd50 0x7f93c014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93bed9d700 1 -- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c00cfd50 msgr2=0x7f93c014d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93bed9d700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c00cfd50 0x7f93c014d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93bed9d700 1 -- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c01536e0 msgr2=0x7f93c01540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93bed9d700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c01536e0 0x7f93c01540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93bed9d700 1 -- 172.21.15.45:0/945572710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b8009ae0 con 0x7f93c0069380 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93bed9d700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c013aa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f93b000d890 tx=0x7f93b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93a6ffd700 1 -- 172.21.15.45:0/945572710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00125f0 con 0x7f93c0069380 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93a6ffd700 1 -- 172.21.15.45:0/945572710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0012750 con 0x7f93c0069380 2023-10-06T05:16:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c0154c20 con 0x7f93c0069380 2023-10-06T05:16:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93a6ffd700 1 -- 172.21.15.45:0/945572710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0012b10 con 0x7f93c0069380 2023-10-06T05:16:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.092+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c01551e0 con 0x7f93c0069380 2023-10-06T05:16:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.093+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c004c380 con 0x7f93c0069380 2023-10-06T05:16:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.094+0000 7f93a6ffd700 1 -- 172.21.15.45:0/945572710 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93b001e460 con 0x7f93c0069380 2023-10-06T05:16:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.094+0000 7f93a6ffd700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ac066f30 0x7f93ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.094+0000 7f93a6ffd700 1 -- 172.21.15.45:0/945572710 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6515+0+0 (secure 0 0 0) 0x7f93b0084de0 con 0x7f93c0069380 2023-10-06T05:16:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.094+0000 7f93be59c700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ac066f30 0x7f93ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.095+0000 7f93be59c700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ac066f30 0x7f93ac0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93c014f0f0 tx=0x7f93b4007ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.099+0000 7f93a6ffd700 1 -- 172.21.15.45:0/945572710 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b00557b0 con 0x7f93c0069380 2023-10-06T05:16:00.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.306+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93c00cf270 con 0x7f93c0069380 2023-10-06T05:16:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.938+0000 7f93a6ffd700 1 -- 172.21.15.45:0/945572710 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f93b0055340 con 0x7f93c0069380 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ac066f30 msgr2=0x7f93ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ac066f30 0x7f93ac0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93c014f0f0 tx=0x7f93b4007ea0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 msgr2=0x7f93c013aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c013aa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f93b000d890 tx=0x7f93b0008040 comp rx=0 tx=0).stop 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 shutdown_connections 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ac066f30 0x7f93ac0693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c01536e0 0x7f93c01540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93c0069380 0x7f93c013aa00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 --2- 172.21.15.45:0/945572710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93c00cfd50 0x7f93c014d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 >> 172.21.15.45:0/945572710 conn(0x7f93c006f820 msgr2=0x7f93c014e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 shutdown_connections 2023-10-06T05:16:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:00.941+0000 7f93c51ee700 1 -- 172.21.15.45:0/945572710 wait complete. 2023-10-06T05:16:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-06T05:16:00.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-06T05:16:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.099+0000 7fb2258e0700 1 Processor -- start 2023-10-06T05:16:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.099+0000 7fb2258e0700 1 -- start start 2023-10-06T05:16:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb2258e0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2200b2340 0x7fb2200cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb2258e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200afcd0 0x7fb22015c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb2258e0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2200b0140 con 0x7fb2200b2710 2023-10-06T05:16:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb2258e0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2200b0420 con 0x7fb2200b2340 2023-10-06T05:16:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb2258e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200b0700 con 0x7fb2200afcd0 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200afcd0 0x7fb22015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb21effd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2200b2340 0x7fb2200cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb21f7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb2200b2710 0x7fb22015d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:46226/0 (socket says 172.21.15.45:46226) 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200afcd0 0x7fb22015c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:50092/0 (socket says 172.21.15.45:50092) 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb21effd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2200b2340 0x7fb2200cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50564/0 (socket says 172.21.15.45:50564) 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.100+0000 7fb21f7fe700 1 -- 172.21.15.45:0/3246292956 learned_addr learned my addr 172.21.15.45:0/3246292956 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21effd700 1 -- 172.21.15.45:0/3246292956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200afcd0 msgr2=0x7fb22015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21effd700 1 --2- 172.21.15.45:0/3246292956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200afcd0 0x7fb22015c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21effd700 1 -- 172.21.15.45:0/3246292956 >> v1:172.21.15.144:6789/0 conn(0x7fb2200b2710 legacy=0x7fb22015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21dffb700 1 -- 172.21.15.45:0/3246292956 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638282879 0 0) 0x7fb2200b0140 con 0x7fb2200b2710 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21effd700 1 -- 172.21.15.45:0/3246292956 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2200b09e0 con 0x7fb2200b2340 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21effd700 1 --2- 172.21.15.45:0/3246292956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200b2340 0x7fb2200cc390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb2100098e0 tx=0x7fb2100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc29157be9e6dab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21dffb700 1 -- 172.21.15.45:0/3246292956 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000b6d0 con 0x7fb2200b2340 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21dffb700 1 -- 172.21.15.45:0/3246292956 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21000b830 con 0x7fb2200b2340 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.101+0000 7fb21dffb700 1 -- 172.21.15.45:0/3246292956 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000bbd0 con 0x7fb2200b2340 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 -- 172.21.15.45:0/3246292956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200b2340 msgr2=0x7fb2200cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 --2- 172.21.15.45:0/3246292956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200b2340 0x7fb2200cc390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb2100098e0 tx=0x7fb2100091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 -- 172.21.15.45:0/3246292956 shutdown_connections 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 --2- 172.21.15.45:0/3246292956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200b2340 0x7fb2200cc390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 --2- 172.21.15.45:0/3246292956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200afcd0 0x7fb22015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 -- 172.21.15.45:0/3246292956 >> 172.21.15.45:0/3246292956 conn(0x7fb22006f820 msgr2=0x7fb2200d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 -- 172.21.15.45:0/3246292956 shutdown_connections 2023-10-06T05:16:01.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.102+0000 7fb2258e0700 1 -- 172.21.15.45:0/3246292956 wait complete. 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.103+0000 7fb2258e0700 1 Processor -- start 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.103+0000 7fb2258e0700 1 -- start start 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb2258e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200afcd0 0x7fb2200c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb2258e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2200b2340 0x7fb22015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb2258e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200b2710 0x7fb220158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb2258e0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200c4350 con 0x7fb2200b2340 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb2258e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200c4120 con 0x7fb2200afcd0 2023-10-06T05:16:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb2258e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200c7b90 con 0x7fb2200b2710 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2200b2340 0x7fb22015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2200b2340 0x7fb22015d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44804/0 (socket says 172.21.15.45:44804) 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb21e7fc700 1 -- 172.21.15.45:0/1387678778 learned_addr learned my addr 172.21.15.45:0/1387678778 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb21f7fe700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200b2710 0x7fb220158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb21effd700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200afcd0 0x7fb2200c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.104+0000 7fb21e7fc700 1 -- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200b2710 msgr2=0x7fb220158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb21e7fc700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200b2710 0x7fb220158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb21e7fc700 1 -- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200afcd0 msgr2=0x7fb2200c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb21e7fc700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200afcd0 0x7fb2200c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb21e7fc700 1 -- 172.21.15.45:0/1387678778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210009420 con 0x7fb2200b2340 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb21f7fe700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200b2710 0x7fb220158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:01.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb21effd700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200afcd0 0x7fb2200c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb21e7fc700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2200b2340 0x7fb22015d020 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb21400ecb0 tx=0x7fb2140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb206ffd700 1 -- 172.21.15.45:0/1387678778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400c9d0 con 0x7fb2200b2340 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb206ffd700 1 -- 172.21.15.45:0/1387678778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21400cb30 con 0x7fb2200b2340 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2200c8020 con 0x7fb2200b2340 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb206ffd700 1 -- 172.21.15.45:0/1387678778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400ced0 con 0x7fb2200b2340 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.105+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2200c7540 con 0x7fb2200b2340 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.106+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22004c380 con 0x7fb2200b2340 2023-10-06T05:16:01.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.106+0000 7fb206ffd700 1 -- 172.21.15.45:0/1387678778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb214013680 con 0x7fb2200b2340 2023-10-06T05:16:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.107+0000 7fb206ffd700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb20c066f30 0x7fb20c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.107+0000 7fb206ffd700 1 -- 172.21.15.45:0/1387678778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6043+0+0 (secure 0 0 0) 0x7fb214084a50 con 0x7fb2200b2340 2023-10-06T05:16:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.107+0000 7fb21effd700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb20c066f30 0x7fb20c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:01.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.108+0000 7fb21effd700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb20c066f30 0x7fb20c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb2100091e0 tx=0x7fb210017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:01.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.112+0000 7fb206ffd700 1 -- 172.21.15.45:0/1387678778 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb214051b70 con 0x7fb2200b2340 2023-10-06T05:16:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.314+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2200c7cd0 con 0x7fb2200b2340 2023-10-06T05:16:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.940+0000 7fb206ffd700 1 -- 172.21.15.45:0/1387678778 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb214055190 con 0x7fb2200b2340 2023-10-06T05:16:01.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.943+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb20c066f30 msgr2=0x7fb20c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.943+0000 7fb2258e0700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb20c066f30 0x7fb20c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb2100091e0 tx=0x7fb210017000 comp rx=0 tx=0).stop 2023-10-06T05:16:01.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.943+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2200b2340 msgr2=0x7fb22015d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:01.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.943+0000 7fb2258e0700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2200b2340 0x7fb22015d020 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb21400ecb0 tx=0x7fb2140091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.944+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 shutdown_connections 2023-10-06T05:16:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.944+0000 7fb2258e0700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb20c066f30 0x7fb20c0693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.944+0000 7fb2258e0700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2200b2710 0x7fb220158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.944+0000 7fb2258e0700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2200afcd0 0x7fb2200c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.944+0000 7fb2258e0700 1 --2- 172.21.15.45:0/1387678778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2200b2340 0x7fb22015d020 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.944+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 >> 172.21.15.45:0/1387678778 conn(0x7fb22006f820 msgr2=0x7fb220068ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.945+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 shutdown_connections 2023-10-06T05:16:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:01.945+0000 7fb2258e0700 1 -- 172.21.15.45:0/1387678778 wait complete. 2023-10-06T05:16:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' removed 2023-10-06T05:16:01.964 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:16:01.964 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:16:01.964 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:16:01.964 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:01.964 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:16:01.964 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:16:01.965 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:16:01.965 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:01.965 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:16:01.965 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:16:01.965 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:16:01.965 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:01.965 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:16:01.966 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:16:01.966 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:16:01.966 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:16:01.975 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:16:01.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:16:01.976 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:16:01.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:16:01.978 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:16:01.978 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:16:01.978 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:16:01.979 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:16:02.018 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:16:02.019 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:02.019 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:16:02.020 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:02.021 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:16:02.021 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:02.021 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:16:02.021 DEBUG:teuthology.parallel:result is None 2023-10-06T05:16:02.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:16:02.102 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:16:02.105 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:16:02.109 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:16:02.109 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:16:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.162+0000 7f079edec700 1 Processor -- start 2023-10-06T05:16:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.162+0000 7f079edec700 1 -- start start 2023-10-06T05:16:02.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079edec700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0798153620 0x7f079815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:02.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079edec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07981539f0 0x7f079815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079edec700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07980cc8b0 0x7f07980ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079edec700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079815a7a0 con 0x7f07981539f0 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079edec700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07980ce3f0 con 0x7f07980cc8b0 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079edec700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079815d300 con 0x7f0798153620 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f0797fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07981539f0 0x7f079815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f0797fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07981539f0 0x7f079815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44868/0 (socket says 172.21.15.45:44868) 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f0797fff700 1 -- 172.21.15.45:0/1046896530 learned_addr learned my addr 172.21.15.45:0/1046896530 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079d389700 1 --2- 172.21.15.45:0/1046896530 >> v2:172.21.15.144:3300/0 conn(0x7f07980cc8b0 0x7f07980ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.163+0000 7f079cb88700 1 --2- 172.21.15.45:0/1046896530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0798153620 0x7f079815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f0797fff700 1 -- 172.21.15.45:0/1046896530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0798153620 msgr2=0x7f079815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f0797fff700 1 --2- 172.21.15.45:0/1046896530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0798153620 0x7f079815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f0797fff700 1 -- 172.21.15.45:0/1046896530 >> v2:172.21.15.144:3300/0 conn(0x7f07980cc8b0 msgr2=0x7f07980ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f0797fff700 1 --2- 172.21.15.45:0/1046896530 >> v2:172.21.15.144:3300/0 conn(0x7f07980cc8b0 0x7f07980ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f0797fff700 1 -- 172.21.15.45:0/1046896530 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07980ce690 con 0x7f07981539f0 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f0797fff700 1 --2- 172.21.15.45:0/1046896530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07981539f0 0x7f079815cdc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0798157310 tx=0x7f078c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6beaca2458f5e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f07977fe700 1 -- 172.21.15.45:0/1046896530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00b640 con 0x7f07981539f0 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f07977fe700 1 -- 172.21.15.45:0/1046896530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00b7a0 con 0x7f07981539f0 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f07977fe700 1 -- 172.21.15.45:0/1046896530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00bb60 con 0x7f07981539f0 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f079edec700 1 -- 172.21.15.45:0/1046896530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07981539f0 msgr2=0x7f079815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.164+0000 7f079edec700 1 --2- 172.21.15.45:0/1046896530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07981539f0 0x7f079815cdc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0798157310 tx=0x7f078c009250 comp rx=0 tx=0).stop 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.165+0000 7f079edec700 1 -- 172.21.15.45:0/1046896530 shutdown_connections 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.165+0000 7f079edec700 1 --2- 172.21.15.45:0/1046896530 >> v2:172.21.15.144:3300/0 conn(0x7f07980cc8b0 0x7f07980ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.165+0000 7f079edec700 1 --2- 172.21.15.45:0/1046896530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0798153620 0x7f079815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.165+0000 7f079edec700 1 --2- 172.21.15.45:0/1046896530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07981539f0 0x7f079815cdc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.165+0000 7f079edec700 1 -- 172.21.15.45:0/1046896530 >> 172.21.15.45:0/1046896530 conn(0x7f079806ed50 msgr2=0x7f079806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:02.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.165+0000 7f079edec700 1 -- 172.21.15.45:0/1046896530 shutdown_connections 2023-10-06T05:16:02.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.165+0000 7f079edec700 1 -- 172.21.15.45:0/1046896530 wait complete. 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.166+0000 7f079edec700 1 Processor -- start 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.166+0000 7f079edec700 1 -- start start 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079edec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07980cc8b0 0x7f079815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079edec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0798153620 0x7f079814f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079edec700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f079815bc20 0x7f079814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079edec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07980b4c40 con 0x7f07980cc8b0 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079edec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079814f520 con 0x7f0798153620 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079edec700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079815c180 con 0x7f079815bc20 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07980cc8b0 0x7f079815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07980cc8b0 0x7f079815b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44884/0 (socket says 172.21.15.45:44884) 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 -- 172.21.15.45:0/3038937349 learned_addr learned my addr 172.21.15.45:0/3038937349 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f0797fff700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0798153620 0x7f079814f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079d389700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f079815bc20 0x7f079814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 -- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f079815bc20 msgr2=0x7f079814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f079815bc20 0x7f079814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 -- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0798153620 msgr2=0x7f079814f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0798153620 0x7f079814f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.167+0000 7f079cb88700 1 -- 172.21.15.45:0/3038937349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0094f0 con 0x7f07980cc8b0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.168+0000 7f079cb88700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07980cc8b0 0x7f079815b6e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0790009ef0 tx=0x7f079000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.168+0000 7f0794ff9700 1 -- 172.21.15.45:0/3038937349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000cb70 con 0x7f07980cc8b0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.168+0000 7f0794ff9700 1 -- 172.21.15.45:0/3038937349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000f040 con 0x7f07980cc8b0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.168+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079814eed0 con 0x7f07980cc8b0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.168+0000 7f0794ff9700 1 -- 172.21.15.45:0/3038937349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790014480 con 0x7f07980cc8b0 2023-10-06T05:16:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.168+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0798144530 con 0x7f07980cc8b0 2023-10-06T05:16:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.169+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0798088a60 con 0x7f07980cc8b0 2023-10-06T05:16:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.169+0000 7f0794ff9700 1 -- 172.21.15.45:0/3038937349 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07900147f0 con 0x7f07980cc8b0 2023-10-06T05:16:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.170+0000 7f0794ff9700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0784066f30 0x7f07840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.170+0000 7f0794ff9700 1 -- 172.21.15.45:0/3038937349 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f0790085050 con 0x7f07980cc8b0 2023-10-06T05:16:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.170+0000 7f0797fff700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0784066f30 0x7f07840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.171+0000 7f0797fff700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0784066f30 0x7f07840693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07980b2c90 tx=0x7f078c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:02.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.174+0000 7f0794ff9700 1 -- 172.21.15.45:0/3038937349 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0790052320 con 0x7f07980cc8b0 2023-10-06T05:16:02.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T05:16:02.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.392+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f079804bb40 con 0x7f07980cc8b0 2023-10-06T05:16:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.958+0000 7f0794ff9700 1 -- 172.21.15.45:0/3038937349 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0790055940 con 0x7f07980cc8b0 2023-10-06T05:16:02.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.961+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0784066f30 msgr2=0x7f07840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:02.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.961+0000 7f079edec700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0784066f30 0x7f07840693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07980b2c90 tx=0x7f078c017000 comp rx=0 tx=0).stop 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.961+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07980cc8b0 msgr2=0x7f079815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.961+0000 7f079edec700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07980cc8b0 0x7f079815b6e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0790009ef0 tx=0x7f079000b040 comp rx=0 tx=0).stop 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.962+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 shutdown_connections 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.962+0000 7f079edec700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0784066f30 0x7f07840693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.962+0000 7f079edec700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f079815bc20 0x7f079814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.962+0000 7f079edec700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07980cc8b0 0x7f079815b6e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.962+0000 7f079edec700 1 --2- 172.21.15.45:0/3038937349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0798153620 0x7f079814f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.962+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 >> 172.21.15.45:0/3038937349 conn(0x7f079806ed50 msgr2=0x7f07980b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.962+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 shutdown_connections 2023-10-06T05:16:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:02.963+0000 7f079edec700 1 -- 172.21.15.45:0/3038937349 wait complete. 2023-10-06T05:16:02.965 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-10-06T05:16:02.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:16:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.114+0000 7f38227dd700 1 Processor -- start 2023-10-06T05:16:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.114+0000 7f38227dd700 1 -- start start 2023-10-06T05:16:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f38227dd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c155cd0 0x7f381c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f38227dd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f381c0ccdc0 0x7f381c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f38227dd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381c0cd740 con 0x7f381c1560a0 2023-10-06T05:16:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f38227dd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c0ae6b0 con 0x7f381c0ccdc0 2023-10-06T05:16:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f38227dd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0ae8d0 con 0x7f381c155cd0 2023-10-06T05:16:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f381bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c155cd0 0x7f381c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f3820d7a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f381c0ccdc0 0x7f381c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f3820d7a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f381c0ccdc0 0x7f381c0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50632/0 (socket says 172.21.15.45:50632) 2023-10-06T05:16:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f381b7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f381c1560a0 0x7f381c0adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:46264/0 (socket says 172.21.15.45:46264) 2023-10-06T05:16:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f381bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c155cd0 0x7f381c15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:50144/0 (socket says 172.21.15.45:50144) 2023-10-06T05:16:03.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.115+0000 7f3820d7a700 1 -- 172.21.15.45:0/3044246524 learned_addr learned my addr 172.21.15.45:0/3044246524 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381affd700 1 -- 172.21.15.45:0/3044246524 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711246937 0 0) 0x7f381c0cd740 con 0x7f381c1560a0 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381affd700 1 -- 172.21.15.45:0/3044246524 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3808003570 con 0x7f381c1560a0 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381bfff700 1 -- 172.21.15.45:0/3044246524 >> v2:172.21.15.144:3300/0 conn(0x7f381c0ccdc0 msgr2=0x7f381c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381bfff700 1 --2- 172.21.15.45:0/3044246524 >> v2:172.21.15.144:3300/0 conn(0x7f381c0ccdc0 0x7f381c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381bfff700 1 -- 172.21.15.45:0/3044246524 >> v1:172.21.15.144:6789/0 conn(0x7f381c1560a0 legacy=0x7f381c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381bfff700 1 -- 172.21.15.45:0/3044246524 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0aebb0 con 0x7f381c155cd0 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f3820d7a700 1 --2- 172.21.15.45:0/3044246524 >> v2:172.21.15.144:3300/0 conn(0x7f381c0ccdc0 0x7f381c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381bfff700 1 --2- 172.21.15.45:0/3044246524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c155cd0 0x7f381c15a730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f380c00bb90 tx=0x7f380c00b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6078c664b07071c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381affd700 1 -- 172.21.15.45:0/3044246524 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00da70 con 0x7f381c155cd0 2023-10-06T05:16:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381affd700 1 -- 172.21.15.45:0/3044246524 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c00dbd0 con 0x7f381c155cd0 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.116+0000 7f381affd700 1 -- 172.21.15.45:0/3044246524 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c019480 con 0x7f381c155cd0 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 -- 172.21.15.45:0/3044246524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c155cd0 msgr2=0x7f381c15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 --2- 172.21.15.45:0/3044246524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c155cd0 0x7f381c15a730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f380c00bb90 tx=0x7f380c00b430 comp rx=0 tx=0).stop 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 -- 172.21.15.45:0/3044246524 shutdown_connections 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 --2- 172.21.15.45:0/3044246524 >> v2:172.21.15.144:3300/0 conn(0x7f381c0ccdc0 0x7f381c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 --2- 172.21.15.45:0/3044246524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c155cd0 0x7f381c15a730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 -- 172.21.15.45:0/3044246524 >> 172.21.15.45:0/3044246524 conn(0x7f381c06f090 msgr2=0x7f381c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 -- 172.21.15.45:0/3044246524 shutdown_connections 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.117+0000 7f38227dd700 1 -- 172.21.15.45:0/3044246524 wait complete. 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.118+0000 7f38227dd700 1 Processor -- start 2023-10-06T05:16:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.118+0000 7f38227dd700 1 -- start start 2023-10-06T05:16:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f38227dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c0ccdc0 0x7f381c166440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f38227dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c155cd0 0x7f381c164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f38227dd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c1560a0 0x7f381c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f38227dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c166980 con 0x7f381c155cd0 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f38227dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0aff50 con 0x7f381c0ccdc0 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f38227dd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b0230 con 0x7f381c1560a0 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c155cd0 0x7f381c164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c155cd0 0x7f381c164ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44896/0 (socket says 172.21.15.45:44896) 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 -- 172.21.15.45:0/684816856 learned_addr learned my addr 172.21.15.45:0/684816856 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381bfff700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c0ccdc0 0x7f381c166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f3820d7a700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c1560a0 0x7f381c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 -- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c1560a0 msgr2=0x7f381c0c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c1560a0 0x7f381c0c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 -- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c0ccdc0 msgr2=0x7f381c166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c0ccdc0 0x7f381c166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.119+0000 7f381b7fe700 1 -- 172.21.15.45:0/684816856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3814009ae0 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.120+0000 7f381b7fe700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c155cd0 0x7f381c164ac0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f380400c590 tx=0x7f3804007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.120+0000 7f3813fff700 1 -- 172.21.15.45:0/684816856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804005a70 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.120+0000 7f3813fff700 1 -- 172.21.15.45:0/684816856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804005bf0 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.120+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c00b6d0 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.120+0000 7f3813fff700 1 -- 172.21.15.45:0/684816856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38040070d0 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.120+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c0b07f0 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.121+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c0b09b0 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.121+0000 7f3813fff700 1 -- 172.21.15.45:0/684816856 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38040074c0 con 0x7f381c155cd0 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.122+0000 7f3813fff700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f37fc066f30 0x7f37fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.122+0000 7f3813fff700 1 -- 172.21.15.45:0/684816856 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f3804083970 con 0x7f381c155cd0 2023-10-06T05:16:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.122+0000 7f381bfff700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f37fc066f30 0x7f37fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.123+0000 7f381bfff700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f37fc066f30 0x7f37fc0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b470 tx=0x7f380c01b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.126+0000 7f3813fff700 1 -- 172.21.15.45:0/684816856 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3804050c90 con 0x7f381c155cd0 2023-10-06T05:16:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.408+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f381c0aee90 con 0x7f381c155cd0 2023-10-06T05:16:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.408+0000 7f3813fff700 1 -- 172.21.15.45:0/684816856 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3804017020 con 0x7f381c155cd0 2023-10-06T05:16:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:03.410 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:16:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw==", 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w==", 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-10-06T05:16:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg==", 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw==", 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg==", 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg==", 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w==", 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw==", 2023-10-06T05:16:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA==", 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ==", 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw==", 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA==", 2023-10-06T05:16:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw==", 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:16:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw==", 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g==", 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w==", 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-10-06T05:16:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l/0VtGRAA99fxS1gieVbRro7K6P1/lw==", 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mds", 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQDAlx9lcUg0LxAAXuMR/Qt7BJPKgpCdIqktXA==", 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T05:16:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQDAlx9lhlI0LxAArmNf6/7pSN6FYDvLjBQgIQ==", 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-osd", 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQDAlx9lDl80LxAAEv/Cvs3LkCYhcOCx5tdz5w==", 2023-10-06T05:16:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQDAlx9lq2c0LxAAmlBozPYmS0JfCRqV6L8dLw==", 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQDAlx9ljHI0LxAA1LndJWwv4m7uJ49XP/FaeQ==", 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQDAlx9loXs0LxAAH71M4y2ayKbeU8KBCDbamA==", 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A==", 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg==", 2023-10-06T05:16:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:16:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:16:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:16:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:16:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-06T05:16:03.429 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.410+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f37fc066f30 msgr2=0x7f37fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.410+0000 7f38227dd700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f37fc066f30 0x7f37fc0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b470 tx=0x7f380c01b000 comp rx=0 tx=0).stop 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.410+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c155cd0 msgr2=0x7f381c164ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.410+0000 7f38227dd700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c155cd0 0x7f381c164ac0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f380400c590 tx=0x7f3804007e00 comp rx=0 tx=0).stop 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.410+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 shutdown_connections 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.410+0000 7f38227dd700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f37fc066f30 0x7f37fc0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.410+0000 7f38227dd700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381c1560a0 0x7f381c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.411+0000 7f38227dd700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f381c0ccdc0 0x7f381c166440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.411+0000 7f38227dd700 1 --2- 172.21.15.45:0/684816856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c155cd0 0x7f381c164ac0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.411+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 >> 172.21.15.45:0/684816856 conn(0x7f381c06f090 msgr2=0x7f381c0adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.411+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 shutdown_connections 2023-10-06T05:16:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.411+0000 7f38227dd700 1 -- 172.21.15.45:0/684816856 wait complete. 2023-10-06T05:16:03.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T05:16:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.563+0000 7fb658ac3700 1 Processor -- start 2023-10-06T05:16:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.563+0000 7fb658ac3700 1 -- start start 2023-10-06T05:16:03.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.564+0000 7fb658ac3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb654155cd0 0x7fb65415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.564+0000 7fb658ac3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6541560a0 0x7fb6540adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.564+0000 7fb658ac3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6540ae8d0 con 0x7fb6540ccdc0 2023-10-06T05:16:03.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.564+0000 7fb658ac3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6540aebb0 con 0x7fb654155cd0 2023-10-06T05:16:03.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.564+0000 7fb658ac3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6540aee90 con 0x7fb6541560a0 2023-10-06T05:16:03.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb654155cd0 0x7fb65415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb654155cd0 0x7fb65415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44902/0 (socket says 172.21.15.45:44902) 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 -- 172.21.15.45:0/2269226415 learned_addr learned my addr 172.21.15.45:0/2269226415 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb651d9b700 1 --2- 172.21.15.45:0/2269226415 >> v2:172.21.15.144:3300/0 conn(0x7fb6541560a0 0x7fb6540adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 -- 172.21.15.45:0/2269226415 >> v2:172.21.15.144:3300/0 conn(0x7fb6541560a0 msgr2=0x7fb6540adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 --2- 172.21.15.45:0/2269226415 >> v2:172.21.15.144:3300/0 conn(0x7fb6541560a0 0x7fb6540adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 -- 172.21.15.45:0/2269226415 >> v1:172.21.15.144:6789/0 conn(0x7fb6540ccdc0 legacy=0x7fb6540cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 -- 172.21.15.45:0/2269226415 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540af170 con 0x7fb654155cd0 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65259c700 1 --2- 172.21.15.45:0/2269226415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb654155cd0 0x7fb65415a730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb6540cda20 tx=0x7fb64400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4596806da13a7621 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65159a700 1 -- 172.21.15.45:0/2269226415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400dc50 con 0x7fb654155cd0 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65159a700 1 -- 172.21.15.45:0/2269226415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64400ddb0 con 0x7fb654155cd0 2023-10-06T05:16:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.565+0000 7fb65159a700 1 -- 172.21.15.45:0/2269226415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440196d0 con 0x7fb654155cd0 2023-10-06T05:16:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 -- 172.21.15.45:0/2269226415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb654155cd0 msgr2=0x7fb65415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 --2- 172.21.15.45:0/2269226415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb654155cd0 0x7fb65415a730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb6540cda20 tx=0x7fb64400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 -- 172.21.15.45:0/2269226415 shutdown_connections 2023-10-06T05:16:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 --2- 172.21.15.45:0/2269226415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb654155cd0 0x7fb65415a730 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 --2- 172.21.15.45:0/2269226415 >> v2:172.21.15.144:3300/0 conn(0x7fb6541560a0 0x7fb6540adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 -- 172.21.15.45:0/2269226415 >> 172.21.15.45:0/2269226415 conn(0x7fb65406f090 msgr2=0x7fb65406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 -- 172.21.15.45:0/2269226415 shutdown_connections 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.566+0000 7fb658ac3700 1 -- 172.21.15.45:0/2269226415 wait complete. 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.567+0000 7fb658ac3700 1 Processor -- start 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.567+0000 7fb658ac3700 1 -- start start 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb658ac3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6540ccdc0 0x7fb65416a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb658ac3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb654155cd0 0x7fb6540b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb658ac3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6541560a0 0x7fb6540ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb658ac3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65416b1d0 con 0x7fb6540ccdc0 2023-10-06T05:16:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb658ac3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540c3f70 con 0x7fb6541560a0 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb658ac3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540b29b0 con 0x7fb654155cd0 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb65259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6540ccdc0 0x7fb65416a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb65259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6540ccdc0 0x7fb65416a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44908/0 (socket says 172.21.15.45:44908) 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb65259c700 1 -- 172.21.15.45:0/3996640312 learned_addr learned my addr 172.21.15.45:0/3996640312 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb652d9d700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6541560a0 0x7fb6540ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.568+0000 7fb651d9b700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb654155cd0 0x7fb6540b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb65259c700 1 -- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb654155cd0 msgr2=0x7fb6540b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb65259c700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb654155cd0 0x7fb6540b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb65259c700 1 -- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6541560a0 msgr2=0x7fb6540ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb65259c700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6541560a0 0x7fb6540ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb65259c700 1 -- 172.21.15.45:0/3996640312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64400b790 con 0x7fb6540ccdc0 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb651d9b700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb654155cd0 0x7fb6540b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb652d9d700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6541560a0 0x7fb6540ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb65259c700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6540ccdc0 0x7fb65416a090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb65406aed0 tx=0x7fb64401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb64a7fc700 1 -- 172.21.15.45:0/3996640312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644010680 con 0x7fb6540ccdc0 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb64a7fc700 1 -- 172.21.15.45:0/3996640312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6440107e0 con 0x7fb6540ccdc0 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6540ac690 con 0x7fb6540ccdc0 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb64a7fc700 1 -- 172.21.15.45:0/3996640312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644010ba0 con 0x7fb6540ccdc0 2023-10-06T05:16:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.569+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65416a990 con 0x7fb6540ccdc0 2023-10-06T05:16:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.570+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65406e2f0 con 0x7fb6540ccdc0 2023-10-06T05:16:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.570+0000 7fb64a7fc700 1 -- 172.21.15.45:0/3996640312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6440233f0 con 0x7fb6540ccdc0 2023-10-06T05:16:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.571+0000 7fb64a7fc700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb634066f30 0x7fb6340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.571+0000 7fb64a7fc700 1 -- 172.21.15.45:0/3996640312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fb644014070 con 0x7fb6540ccdc0 2023-10-06T05:16:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.571+0000 7fb651d9b700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb634066f30 0x7fb6340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.572+0000 7fb651d9b700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb634066f30 0x7fb6340693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb654156bd0 tx=0x7fb63c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:03.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.575+0000 7fb64a7fc700 1 -- 172.21.15.45:0/3996640312 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb644053670 con 0x7fb6540ccdc0 2023-10-06T05:16:03.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.869+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fb6540ae4e0 con 0x7fb6540ccdc0 2023-10-06T05:16:03.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.873+0000 7fb64a7fc700 1 -- 172.21.15.45:0/3996640312 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb644056c90 con 0x7fb6540ccdc0 2023-10-06T05:16:03.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb634066f30 msgr2=0x7fb6340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb634066f30 0x7fb6340693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb654156bd0 tx=0x7fb63c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6540ccdc0 msgr2=0x7fb65416a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6540ccdc0 0x7fb65416a090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb65406aed0 tx=0x7fb64401b000 comp rx=0 tx=0).stop 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 shutdown_connections 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb634066f30 0x7fb6340693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6541560a0 0x7fb6540ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6540ccdc0 0x7fb65416a090 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 --2- 172.21.15.45:0/3996640312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb654155cd0 0x7fb6540b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.875+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 >> 172.21.15.45:0/3996640312 conn(0x7fb65406f090 msgr2=0x7fb6540c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.876+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 shutdown_connections 2023-10-06T05:16:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:03.876+0000 7fb658ac3700 1 -- 172.21.15.45:0/3996640312 wait complete. 2023-10-06T05:16:03.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T05:16:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.024+0000 7ff8b8dba700 1 Processor -- start 2023-10-06T05:16:04.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.024+0000 7ff8b8dba700 1 -- start start 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b8dba700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b41595d0 0x7ff8b406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b8dba700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b40c7d00 con 0x7ff8b406b5c0 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b8dba700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b406bb40 con 0x7ff8b4159200 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b8dba700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40c7780 con 0x7ff8b41595d0 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b2d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff8b406b5c0 0x7ff8b40ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38916/0 (socket says 172.21.15.45:38916) 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b2d9d700 1 -- 172.21.15.45:0/1598490930 learned_addr learned my addr 172.21.15.45:0/1598490930 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b1d9b700 1 --2- 172.21.15.45:0/1598490930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b41595d0 0x7ff8b406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209350538 0 0) 0x7ff8b40c7d00 con 0x7ff8b406b5c0 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff89c003570 con 0x7ff8b406b5c0 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244112286 0 0) 0x7ff8b406bb40 con 0x7ff8b4159200 2023-10-06T05:16:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.025+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b40c7d00 con 0x7ff8b4159200 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b1d9b700 1 -- 172.21.15.45:0/1598490930 >> v1:172.21.15.144:6789/0 conn(0x7ff8b4159200 legacy=0x7ff8b40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 536181151 0 0) 0x7ff89c003570 con 0x7ff8b406b5c0 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b1d9b700 1 -- 172.21.15.45:0/1598490930 >> v1:172.21.15.45:6789/0 conn(0x7ff8b406b5c0 legacy=0x7ff8b40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b1d9b700 1 -- 172.21.15.45:0/1598490930 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b406be20 con 0x7ff8b41595d0 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b1d9b700 1 --2- 172.21.15.45:0/1598490930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b41595d0 0x7ff8b406b080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009910 tx=0x7ff8a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dbbd86638096148 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0017f0 con 0x7ff8b41595d0 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a8011420 con 0x7ff8b41595d0 2023-10-06T05:16:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.026+0000 7ff8b159a700 1 -- 172.21.15.45:0/1598490930 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80117c0 con 0x7ff8b41595d0 2023-10-06T05:16:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.027+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1598490930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b41595d0 msgr2=0x7ff8b406b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.027+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1598490930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b41595d0 0x7ff8b406b080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009910 tx=0x7ff8a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.027+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1598490930 shutdown_connections 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.027+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1598490930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b41595d0 0x7ff8b406b080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.027+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1598490930 >> 172.21.15.45:0/1598490930 conn(0x7ff8b406f090 msgr2=0x7ff8b406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.027+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1598490930 shutdown_connections 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.027+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1598490930 wait complete. 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.028+0000 7ff8b8dba700 1 Processor -- start 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.028+0000 7ff8b8dba700 1 -- start start 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b8dba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b406b5c0 0x7ff8b40c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b8dba700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b4159200 0x7ff8b4153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b8dba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b41595d0 0x7ff8b41535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b8dba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40b00f0 con 0x7ff8b406b5c0 2023-10-06T05:16:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b8dba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40ccdf0 con 0x7ff8b41595d0 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b8dba700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40cc830 con 0x7ff8b4159200 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b406b5c0 0x7ff8b40c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b406b5c0 0x7ff8b40c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44914/0 (socket says 172.21.15.45:44914) 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 -- 172.21.15.45:0/1328256785 learned_addr learned my addr 172.21.15.45:0/1328256785 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b1d9b700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b4159200 0x7ff8b4153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b2d9d700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b41595d0 0x7ff8b41535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 -- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b4159200 msgr2=0x7ff8b4153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b4159200 0x7ff8b4153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 -- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b41595d0 msgr2=0x7ff8b41535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b41595d0 0x7ff8b41535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 -- 172.21.15.45:0/1328256785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a8009420 con 0x7ff8b406b5c0 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b2d9d700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b41595d0 0x7ff8b41535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.029+0000 7ff8b259c700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b406b5c0 0x7ff8b40c6950 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c790 tx=0x7ff8a4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.030+0000 7ff8a27fc700 1 -- 172.21.15.45:0/1328256785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4010510 con 0x7ff8b406b5c0 2023-10-06T05:16:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.030+0000 7ff8a27fc700 1 -- 172.21.15.45:0/1328256785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a4010670 con 0x7ff8b406b5c0 2023-10-06T05:16:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.030+0000 7ff8a27fc700 1 -- 172.21.15.45:0/1328256785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4010a10 con 0x7ff8b406b5c0 2023-10-06T05:16:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.030+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b40ccb10 con 0x7ff8b406b5c0 2023-10-06T05:16:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.030+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b40c6e90 con 0x7ff8b406b5c0 2023-10-06T05:16:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.031+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b406e2f0 con 0x7ff8b406b5c0 2023-10-06T05:16:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.031+0000 7ff8a27fc700 1 -- 172.21.15.45:0/1328256785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8a401b400 con 0x7ff8b406b5c0 2023-10-06T05:16:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.031+0000 7ff8a27fc700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff894066f30 0x7ff8940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.032+0000 7ff8a27fc700 1 -- 172.21.15.45:0/1328256785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7ff8a40842a0 con 0x7ff8b406b5c0 2023-10-06T05:16:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.032+0000 7ff8b1d9b700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff894066f30 0x7ff8940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.032+0000 7ff8b1d9b700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff894066f30 0x7ff8940693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80030c0 tx=0x7ff8a8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.036+0000 7ff8a27fc700 1 -- 172.21.15.45:0/1328256785 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8a40515c0 con 0x7ff8b406b5c0 2023-10-06T05:16:04.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.320+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff8b40c7990 con 0x7ff8b406b5c0 2023-10-06T05:16:04.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.323+0000 7ff8a27fc700 1 -- 172.21.15.45:0/1328256785 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff8a4054be0 con 0x7ff8b406b5c0 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.325+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff894066f30 msgr2=0x7ff8940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff894066f30 0x7ff8940693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80030c0 tx=0x7ff8a8016000 comp rx=0 tx=0).stop 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b406b5c0 msgr2=0x7ff8b40c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b406b5c0 0x7ff8b40c6950 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c790 tx=0x7ff8a4007e10 comp rx=0 tx=0).stop 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 shutdown_connections 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff894066f30 0x7ff8940693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b41595d0 0x7ff8b41535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b406b5c0 0x7ff8b40c6950 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 --2- 172.21.15.45:0/1328256785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b4159200 0x7ff8b4153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 >> 172.21.15.45:0/1328256785 conn(0x7ff8b406f090 msgr2=0x7ff8b40686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 shutdown_connections 2023-10-06T05:16:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.326+0000 7ff8b8dba700 1 -- 172.21.15.45:0/1328256785 wait complete. 2023-10-06T05:16:04.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T05:16:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.471+0000 7f35ba3d0700 1 Processor -- start 2023-10-06T05:16:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.471+0000 7f35ba3d0700 1 -- start start 2023-10-06T05:16:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.472+0000 7f35ba3d0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f35b4153650 0x7f35b415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.472+0000 7f35ba3d0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b406afa0 0x7f35b4068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.472+0000 7f35ba3d0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b40676c0 con 0x7f35b4153a20 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.472+0000 7f35ba3d0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b40679a0 con 0x7f35b4153650 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.472+0000 7f35ba3d0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b4067c80 con 0x7f35b406afa0 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f35b4153650 0x7f35b415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b896d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b406afa0 0x7f35b4068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b896d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b406afa0 0x7f35b4068480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:50178/0 (socket says 172.21.15.45:50178) 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b37fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f35b4153a20 0x7f35b406a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:46282/0 (socket says 172.21.15.45:46282) 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f35b4153650 0x7f35b415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50668/0 (socket says 172.21.15.45:50668) 2023-10-06T05:16:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b896d700 1 -- 172.21.15.45:0/1962468869 learned_addr learned my addr 172.21.15.45:0/1962468869 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b3fff700 1 -- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b406afa0 msgr2=0x7f35b4068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b406afa0 0x7f35b4068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b3fff700 1 -- 172.21.15.45:0/1962468869 >> v1:172.21.15.144:6789/0 conn(0x7f35b4153a20 legacy=0x7f35b406a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b2ffd700 1 -- 172.21.15.45:0/1962468869 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522524901 0 0) 0x7f35b40676c0 con 0x7f35b4153a20 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b3fff700 1 -- 172.21.15.45:0/1962468869 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b4067f60 con 0x7f35b4153650 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b896d700 1 --2- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b406afa0 0x7f35b4068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b4153650 0x7f35b415a930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35a40098e0 tx=0x7f35a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e7d8368cabd135 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b2ffd700 1 -- 172.21.15.45:0/1962468869 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400b640 con 0x7f35b4153650 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b2ffd700 1 -- 172.21.15.45:0/1962468869 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a400b7a0 con 0x7f35b4153650 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.473+0000 7f35b2ffd700 1 -- 172.21.15.45:0/1962468869 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400bb40 con 0x7f35b4153650 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b4153650 msgr2=0x7f35b415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b4153650 0x7f35b415a930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35a40098e0 tx=0x7f35a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1962468869 shutdown_connections 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b406afa0 0x7f35b4068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1962468869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b4153650 0x7f35b415a930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1962468869 >> 172.21.15.45:0/1962468869 conn(0x7f35b40cc5a0 msgr2=0x7f35b40cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1962468869 shutdown_connections 2023-10-06T05:16:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.474+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1962468869 wait complete. 2023-10-06T05:16:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.475+0000 7f35ba3d0700 1 Processor -- start 2023-10-06T05:16:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.476+0000 7f35ba3d0700 1 -- start start 2023-10-06T05:16:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.476+0000 7f35ba3d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406afa0 0x7f35b4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.476+0000 7f35ba3d0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b4153650 0x7f35b40c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.476+0000 7f35ba3d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4153a20 0x7f35b40ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.476+0000 7f35ba3d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40b02d0 con 0x7f35b4153a20 2023-10-06T05:16:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.476+0000 7f35ba3d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40af6c0 con 0x7f35b406afa0 2023-10-06T05:16:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.476+0000 7f35ba3d0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40af9a0 con 0x7f35b4153650 2023-10-06T05:16:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4153a20 0x7f35b40ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4153a20 0x7f35b40ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44930/0 (socket says 172.21.15.45:44930) 2023-10-06T05:16:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 -- 172.21.15.45:0/1555610577 learned_addr learned my addr 172.21.15.45:0/1555610577 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406afa0 0x7f35b4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b37fe700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b4153650 0x7f35b40c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 -- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b4153650 msgr2=0x7f35b40c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b4153650 0x7f35b40c07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 -- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406afa0 msgr2=0x7f35b4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406afa0 0x7f35b4151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 -- 172.21.15.45:0/1555610577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac009ae0 con 0x7f35b4153a20 2023-10-06T05:16:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35b896d700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4153a20 0x7f35b40ad580 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009f00 tx=0x7f35ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f359bfff700 1 -- 172.21.15.45:0/1555610577 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0092e0 con 0x7f35b4153a20 2023-10-06T05:16:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f359bfff700 1 -- 172.21.15.45:0/1555610577 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac00f040 con 0x7f35b4153a20 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a4009420 con 0x7f35b4153a20 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f359bfff700 1 -- 172.21.15.45:0/1555610577 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00cca0 con 0x7f35b4153a20 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.477+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b40aff60 con 0x7f35b4153a20 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.478+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b40cc0e0 con 0x7f35b4153a20 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.479+0000 7f359bfff700 1 -- 172.21.15.45:0/1555610577 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35ac003020 con 0x7f35b4153a20 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.479+0000 7f359bfff700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35a0066f30 0x7f35a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.479+0000 7f359bfff700 1 -- 172.21.15.45:0/1555610577 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f35ac0848f0 con 0x7f35b4153a20 2023-10-06T05:16:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.479+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35a0066f30 0x7f35a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.480+0000 7f35b3fff700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35a0066f30 0x7f35a00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35a40093f0 tx=0x7f35a4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.484+0000 7f359bfff700 1 -- 172.21.15.45:0/1555610577 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35ac051c10 con 0x7f35b4153a20 2023-10-06T05:16:04.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.766+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f35b40b0410 con 0x7f35b4153a20 2023-10-06T05:16:04.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.769+0000 7f359bfff700 1 -- 172.21.15.45:0/1555610577 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f35ac055230 con 0x7f35b4153a20 2023-10-06T05:16:04.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35a0066f30 msgr2=0x7f35a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35a0066f30 0x7f35a00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35a40093f0 tx=0x7f35a4017000 comp rx=0 tx=0).stop 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4153a20 msgr2=0x7f35b40ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4153a20 0x7f35b40ad580 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009f00 tx=0x7f35ac00b040 comp rx=0 tx=0).stop 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 shutdown_connections 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35a0066f30 0x7f35a00693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35b4153a20 0x7f35b40ad580 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406afa0 0x7f35b4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 --2- 172.21.15.45:0/1555610577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b4153650 0x7f35b40c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.771+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 >> 172.21.15.45:0/1555610577 conn(0x7f35b40cc5a0 msgr2=0x7f35b415b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.772+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 shutdown_connections 2023-10-06T05:16:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.772+0000 7f35ba3d0700 1 -- 172.21.15.45:0/1555610577 wait complete. 2023-10-06T05:16:04.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T05:16:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.919+0000 7f84d67bd700 1 Processor -- start 2023-10-06T05:16:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.919+0000 7f84d67bd700 1 -- start start 2023-10-06T05:16:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.920+0000 7f84d67bd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84d01560a0 0x7f84d015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.920+0000 7f84d67bd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.920+0000 7f84d67bd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d00cd740 con 0x7f84d0155cd0 2023-10-06T05:16:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.920+0000 7f84d67bd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d00ae6b0 con 0x7f84d01560a0 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.920+0000 7f84d67bd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00ae8d0 con 0x7f84d00ccdc0 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84d4d5a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84cf7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84d01560a0 0x7f84d015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84cffff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f84d0155cd0 0x7f84d00adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:46286/0 (socket says 172.21.15.45:46286) 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84d4d5a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:50190/0 (socket says 172.21.15.45:50190) 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84cf7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84d01560a0 0x7f84d015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50692/0 (socket says 172.21.15.45:50692) 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84cffff700 1 -- 172.21.15.45:0/2328332151 learned_addr learned my addr 172.21.15.45:0/2328332151 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84ceffd700 1 -- 172.21.15.45:0/2328332151 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930053482 0 0) 0x7f84d00cd740 con 0x7f84d0155cd0 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84d4d5a700 1 -- 172.21.15.45:0/2328332151 >> v2:172.21.15.144:3300/0 conn(0x7f84d01560a0 msgr2=0x7f84d015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84d4d5a700 1 --2- 172.21.15.45:0/2328332151 >> v2:172.21.15.144:3300/0 conn(0x7f84d01560a0 0x7f84d015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84d4d5a700 1 -- 172.21.15.45:0/2328332151 >> v1:172.21.15.144:6789/0 conn(0x7f84d0155cd0 legacy=0x7f84d00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84d4d5a700 1 -- 172.21.15.45:0/2328332151 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00aebb0 con 0x7f84d00ccdc0 2023-10-06T05:16:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84d4d5a700 1 --2- 172.21.15.45:0/2328332151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cd200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009e60 tx=0x7f84c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8616257a1b6f94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.921+0000 7f84ceffd700 1 -- 172.21.15.45:0/2328332151 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c800e070 con 0x7f84d00ccdc0 2023-10-06T05:16:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84ceffd700 1 -- 172.21.15.45:0/2328332151 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c800bb60 con 0x7f84d00ccdc0 2023-10-06T05:16:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84ceffd700 1 -- 172.21.15.45:0/2328332151 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8012430 con 0x7f84d00ccdc0 2023-10-06T05:16:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 -- 172.21.15.45:0/2328332151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 msgr2=0x7f84d00cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2328332151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cd200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009e60 tx=0x7f84c8009780 comp rx=0 tx=0).stop 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 -- 172.21.15.45:0/2328332151 shutdown_connections 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2328332151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cd200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2328332151 >> v2:172.21.15.144:3300/0 conn(0x7f84d01560a0 0x7f84d015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 -- 172.21.15.45:0/2328332151 >> 172.21.15.45:0/2328332151 conn(0x7f84d006f090 msgr2=0x7f84d006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 -- 172.21.15.45:0/2328332151 shutdown_connections 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.922+0000 7f84d67bd700 1 -- 172.21.15.45:0/2328332151 wait complete. 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.923+0000 7f84d67bd700 1 Processor -- start 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.924+0000 7f84d67bd700 1 -- start start 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.924+0000 7f84d67bd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.924+0000 7f84d67bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d0155cd0 0x7f84d0144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.924+0000 7f84d67bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84d01560a0 0x7f84d01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.924+0000 7f84d67bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00cc940 con 0x7f84d0155cd0 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.924+0000 7f84d67bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d0159330 con 0x7f84d01560a0 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.924+0000 7f84d67bd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d0159610 con 0x7f84d00ccdc0 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d0155cd0 0x7f84d0144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d0155cd0 0x7f84d0144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44936/0 (socket says 172.21.15.45:44936) 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 -- 172.21.15.45:0/2159861803 learned_addr learned my addr 172.21.15.45:0/2159861803 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cffff700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84d4d5a700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84d01560a0 0x7f84d01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 -- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 msgr2=0x7f84d00cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 -- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84d01560a0 msgr2=0x7f84d01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84d01560a0 0x7f84d01445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:04.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 -- 172.21.15.45:0/2159861803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8009a20 con 0x7f84d0155cd0 2023-10-06T05:16:04.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84cf7fe700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d0155cd0 0x7f84d0144060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ebb0 tx=0x7f84c400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84b7fff700 1 -- 172.21.15.45:0/2159861803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400d9f0 con 0x7f84d0155cd0 2023-10-06T05:16:04.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d00c3fe0 con 0x7f84d0155cd0 2023-10-06T05:16:04.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84b7fff700 1 -- 172.21.15.45:0/2159861803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c400db50 con 0x7f84d0155cd0 2023-10-06T05:16:04.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84b7fff700 1 -- 172.21.15.45:0/2159861803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4010440 con 0x7f84d0155cd0 2023-10-06T05:16:04.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.925+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d00cc100 con 0x7f84d0155cd0 2023-10-06T05:16:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.926+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d00c4480 con 0x7f84d0155cd0 2023-10-06T05:16:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.927+0000 7f84b7fff700 1 -- 172.21.15.45:0/2159861803 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84c40107b0 con 0x7f84d0155cd0 2023-10-06T05:16:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.927+0000 7f84b7fff700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84bc066f30 0x7f84bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.927+0000 7f84b7fff700 1 -- 172.21.15.45:0/2159861803 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f84c4084ae0 con 0x7f84d0155cd0 2023-10-06T05:16:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.928+0000 7f84cffff700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84bc066f30 0x7f84bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.928+0000 7f84cffff700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84bc066f30 0x7f84bc0693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007fd0 tx=0x7f84c0007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:04.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:04.932+0000 7f84b7fff700 1 -- 172.21.15.45:0/2159861803 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84c4051e00 con 0x7f84d0155cd0 2023-10-06T05:16:05.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.222+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f84d00c3e40 con 0x7f84d0155cd0 2023-10-06T05:16:05.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.226+0000 7f84b7fff700 1 -- 172.21.15.45:0/2159861803 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f84d00c3e40 con 0x7f84d0155cd0 2023-10-06T05:16:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84bc066f30 msgr2=0x7f84bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84bc066f30 0x7f84bc0693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007fd0 tx=0x7f84c0007d30 comp rx=0 tx=0).stop 2023-10-06T05:16:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d0155cd0 msgr2=0x7f84d0144060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d0155cd0 0x7f84d0144060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ebb0 tx=0x7f84c400b430 comp rx=0 tx=0).stop 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 shutdown_connections 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84bc066f30 0x7f84bc0693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84d01560a0 0x7f84d01445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84d00ccdc0 0x7f84d00cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 --2- 172.21.15.45:0/2159861803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d0155cd0 0x7f84d0144060 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 >> 172.21.15.45:0/2159861803 conn(0x7f84d006f090 msgr2=0x7f84d01533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 shutdown_connections 2023-10-06T05:16:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.228+0000 7f84d67bd700 1 -- 172.21.15.45:0/2159861803 wait complete. 2023-10-06T05:16:05.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T05:16:05.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.376+0000 7fd073122700 1 Processor -- start 2023-10-06T05:16:05.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.376+0000 7fd073122700 1 -- start start 2023-10-06T05:16:05.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd073122700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd06c0c93c0 0x7fd06c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd073122700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd06c157030 0x7fd06c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd073122700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c15b000 con 0x7fd06c0c8ff0 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd073122700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0c9830 con 0x7fd06c157030 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd073122700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c15b2e0 con 0x7fd06c0c93c0 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd0716bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd06c157030 0x7fd06c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd0716bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd06c157030 0x7fd06c0cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44942/0 (socket says 172.21.15.45:44942) 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd070ebe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd06c0c8ff0 0x7fd06c0caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38922/0 (socket says 172.21.15.45:38922) 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd0716bf700 1 -- 172.21.15.45:0/1779634575 learned_addr learned my addr 172.21.15.45:0/1779634575 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd06bfff700 1 --2- 172.21.15.45:0/1779634575 >> v2:172.21.15.144:3300/0 conn(0x7fd06c0c93c0 0x7fd06c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd0716bf700 1 -- 172.21.15.45:0/1779634575 >> v2:172.21.15.144:3300/0 conn(0x7fd06c0c93c0 msgr2=0x7fd06c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:05.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd0716bf700 1 --2- 172.21.15.45:0/1779634575 >> v2:172.21.15.144:3300/0 conn(0x7fd06c0c93c0 0x7fd06c156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd0716bf700 1 -- 172.21.15.45:0/1779634575 >> v1:172.21.15.45:6789/0 conn(0x7fd06c0c8ff0 legacy=0x7fd06c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd0716bf700 1 -- 172.21.15.45:0/1779634575 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c15b5c0 con 0x7fd06c157030 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.377+0000 7fd06b7fe700 1 -- 172.21.15.45:0/1779634575 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774662429 0 0) 0x7fd06c15b000 con 0x7fd06c0c8ff0 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd06bfff700 1 --2- 172.21.15.45:0/1779634575 >> v2:172.21.15.144:3300/0 conn(0x7fd06c0c93c0 0x7fd06c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd0716bf700 1 --2- 172.21.15.45:0/1779634575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c157030 0x7fd06c0cc330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd06c06ed80 tx=0x7fd064009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416a9395f2557369 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd06b7fe700 1 -- 172.21.15.45:0/1779634575 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400e070 con 0x7fd06c157030 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd06b7fe700 1 -- 172.21.15.45:0/1779634575 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06400ba50 con 0x7fd06c157030 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd06b7fe700 1 -- 172.21.15.45:0/1779634575 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0640092e0 con 0x7fd06c157030 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd073122700 1 -- 172.21.15.45:0/1779634575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c157030 msgr2=0x7fd06c0cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd073122700 1 --2- 172.21.15.45:0/1779634575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c157030 0x7fd06c0cc330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd06c06ed80 tx=0x7fd064009840 comp rx=0 tx=0).stop 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd073122700 1 -- 172.21.15.45:0/1779634575 shutdown_connections 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd073122700 1 --2- 172.21.15.45:0/1779634575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c157030 0x7fd06c0cc330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.378+0000 7fd073122700 1 --2- 172.21.15.45:0/1779634575 >> v2:172.21.15.144:3300/0 conn(0x7fd06c0c93c0 0x7fd06c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.379+0000 7fd073122700 1 -- 172.21.15.45:0/1779634575 >> 172.21.15.45:0/1779634575 conn(0x7fd06c0cdf50 msgr2=0x7fd06c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.379+0000 7fd073122700 1 -- 172.21.15.45:0/1779634575 shutdown_connections 2023-10-06T05:16:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.379+0000 7fd073122700 1 -- 172.21.15.45:0/1779634575 wait complete. 2023-10-06T05:16:05.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.380+0000 7fd073122700 1 Processor -- start 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.381+0000 7fd073122700 1 -- start start 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.381+0000 7fd073122700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd06c0c8ff0 0x7fd06c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.381+0000 7fd073122700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0c93c0 0x7fd06c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.381+0000 7fd073122700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c157030 0x7fd06c0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.381+0000 7fd073122700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c13b690 con 0x7fd06c0c93c0 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.381+0000 7fd073122700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0b3ca0 con 0x7fd06c0c8ff0 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.381+0000 7fd073122700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c5c00 con 0x7fd06c157030 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0c93c0 0x7fd06c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0c93c0 0x7fd06c0b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44956/0 (socket says 172.21.15.45:44956) 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 -- 172.21.15.45:0/2454700025 learned_addr learned my addr 172.21.15.45:0/2454700025 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd070ebe700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd06c0c8ff0 0x7fd06c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd0716bf700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c157030 0x7fd06c0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 -- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c157030 msgr2=0x7fd06c0af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c157030 0x7fd06c0af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 -- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd06c0c8ff0 msgr2=0x7fd06c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd06c0c8ff0 0x7fd06c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 -- 172.21.15.45:0/2454700025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064009ae0 con 0x7fd06c0c93c0 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd06bfff700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0c93c0 0x7fd06c0b3760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c860 tx=0x7fd058008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd068ff9700 1 -- 172.21.15.45:0/2454700025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011600 con 0x7fd06c0c93c0 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd068ff9700 1 -- 172.21.15.45:0/2454700025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058011760 con 0x7fd06c0c93c0 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c0b13d0 con 0x7fd06c0c93c0 2023-10-06T05:16:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd068ff9700 1 -- 172.21.15.45:0/2454700025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011b20 con 0x7fd06c0c93c0 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.382+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c0c55b0 con 0x7fd06c0c93c0 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.383+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c0c5d70 con 0x7fd06c0c93c0 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.384+0000 7fd068ff9700 1 -- 172.21.15.45:0/2454700025 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0580193b0 con 0x7fd06c0c93c0 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.384+0000 7fd068ff9700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd050066f30 0x7fd0500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.384+0000 7fd068ff9700 1 -- 172.21.15.45:0/2454700025 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd058084530 con 0x7fd06c0c93c0 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.384+0000 7fd070ebe700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd050066f30 0x7fd0500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.385+0000 7fd070ebe700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd050066f30 0x7fd0500693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd060007180 tx=0x7fd060007ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.389+0000 7fd068ff9700 1 -- 172.21.15.45:0/2454700025 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0580517b0 con 0x7fd06c0c93c0 2023-10-06T05:16:05.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:05.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:05.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.672+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd06c04bb40 con 0x7fd06c0c93c0 2023-10-06T05:16:05.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.676+0000 7fd068ff9700 1 -- 172.21.15.45:0/2454700025 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd058054dd0 con 0x7fd06c0c93c0 2023-10-06T05:16:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd050066f30 msgr2=0x7fd0500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd050066f30 0x7fd0500693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd060007180 tx=0x7fd060007ef0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0c93c0 msgr2=0x7fd06c0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0c93c0 0x7fd06c0b3760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c860 tx=0x7fd058008040 comp rx=0 tx=0).stop 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 shutdown_connections 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd050066f30 0x7fd0500693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c157030 0x7fd06c0af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd06c0c8ff0 0x7fd06c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 --2- 172.21.15.45:0/2454700025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0c93c0 0x7fd06c0b3760 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 >> 172.21.15.45:0/2454700025 conn(0x7fd06c0cdf50 msgr2=0x7fd06c0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 shutdown_connections 2023-10-06T05:16:05.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.678+0000 7fd073122700 1 -- 172.21.15.45:0/2454700025 wait complete. 2023-10-06T05:16:05.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T05:16:05.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.870+0000 7fc87caef700 1 Processor -- start 2023-10-06T05:16:05.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.870+0000 7fc87caef700 1 -- start start 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc87caef700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc878155cd0 0x7fc87815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc87caef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc87caef700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8780ccdc0 0x7fc8780cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc87caef700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780cd740 con 0x7fc8780ccdc0 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc87caef700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780ae8d0 con 0x7fc878155cd0 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc87caef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780aebb0 con 0x7fc8781560a0 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8780ccdc0 0x7fc8780cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8780ccdc0 0x7fc8780cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44958/0 (socket says 172.21.15.45:44958) 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 -- 172.21.15.45:0/2579694063 learned_addr learned my addr 172.21.15.45:0/2579694063 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc875d9b700 1 --2- 172.21.15.45:0/2579694063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc87659c700 1 --2- 172.21.15.45:0/2579694063 >> v2:172.21.15.144:3300/0 conn(0x7fc878155cd0 0x7fc87815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 -- 172.21.15.45:0/2579694063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 msgr2=0x7fc8780adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 --2- 172.21.15.45:0/2579694063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 -- 172.21.15.45:0/2579694063 >> v2:172.21.15.144:3300/0 conn(0x7fc878155cd0 msgr2=0x7fc87815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 --2- 172.21.15.45:0/2579694063 >> v2:172.21.15.144:3300/0 conn(0x7fc878155cd0 0x7fc87815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.871+0000 7fc876d9d700 1 -- 172.21.15.45:0/2579694063 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780aee90 con 0x7fc8780ccdc0 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.872+0000 7fc875d9b700 1 --2- 172.21.15.45:0/2579694063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.872+0000 7fc876d9d700 1 --2- 172.21.15.45:0/2579694063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8780ccdc0 0x7fc8780cd200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc87806ae50 tx=0x7fc870009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e810b51da3cc52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.872+0000 7fc87559a700 1 -- 172.21.15.45:0/2579694063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000e070 con 0x7fc8780ccdc0 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.872+0000 7fc87559a700 1 -- 172.21.15.45:0/2579694063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000ba50 con 0x7fc8780ccdc0 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.872+0000 7fc87559a700 1 -- 172.21.15.45:0/2579694063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8700092e0 con 0x7fc8780ccdc0 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.872+0000 7fc87caef700 1 -- 172.21.15.45:0/2579694063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8780ccdc0 msgr2=0x7fc8780cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.872+0000 7fc87caef700 1 --2- 172.21.15.45:0/2579694063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8780ccdc0 0x7fc8780cd200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc87806ae50 tx=0x7fc870009840 comp rx=0 tx=0).stop 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.873+0000 7fc87caef700 1 -- 172.21.15.45:0/2579694063 shutdown_connections 2023-10-06T05:16:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.873+0000 7fc87caef700 1 --2- 172.21.15.45:0/2579694063 >> v2:172.21.15.144:3300/0 conn(0x7fc878155cd0 0x7fc87815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.873+0000 7fc87caef700 1 --2- 172.21.15.45:0/2579694063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8780ccdc0 0x7fc8780cd200 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.873+0000 7fc87caef700 1 --2- 172.21.15.45:0/2579694063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.873+0000 7fc87caef700 1 -- 172.21.15.45:0/2579694063 >> 172.21.15.45:0/2579694063 conn(0x7fc87806f090 msgr2=0x7fc87806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.873+0000 7fc87caef700 1 -- 172.21.15.45:0/2579694063 shutdown_connections 2023-10-06T05:16:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.873+0000 7fc87caef700 1 -- 172.21.15.45:0/2579694063 wait complete. 2023-10-06T05:16:05.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.874+0000 7fc87caef700 1 Processor -- start 2023-10-06T05:16:05.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.874+0000 7fc87caef700 1 -- start start 2023-10-06T05:16:05.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87caef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8780ccdc0 0x7fc878154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87caef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878155cd0 0x7fc878167440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87caef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87caef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc878157cd0 con 0x7fc878155cd0 2023-10-06T05:16:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87caef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc878158850 con 0x7fc8780ccdc0 2023-10-06T05:16:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87caef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc878154f80 con 0x7fc8781560a0 2023-10-06T05:16:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878155cd0 0x7fc878167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878155cd0 0x7fc878167440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44960/0 (socket says 172.21.15.45:44960) 2023-10-06T05:16:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 -- 172.21.15.45:0/271696852 learned_addr learned my addr 172.21.15.45:0/271696852 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc876d9d700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87659c700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8780ccdc0 0x7fc878154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 -- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 msgr2=0x7fc8780c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 -- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8780ccdc0 msgr2=0x7fc878154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8780ccdc0 0x7fc878154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 -- 172.21.15.45:0/271696852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc870009ae0 con 0x7fc878155cd0 2023-10-06T05:16:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc875d9b700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878155cd0 0x7fc878167440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ee10 tx=0x7fc86c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc8667fc700 1 -- 172.21.15.45:0/271696852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00ce00 con 0x7fc878155cd0 2023-10-06T05:16:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc8667fc700 1 -- 172.21.15.45:0/271696852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c007a80 con 0x7fc878155cd0 2023-10-06T05:16:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc878157f20 con 0x7fc878155cd0 2023-10-06T05:16:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.875+0000 7fc8667fc700 1 -- 172.21.15.45:0/271696852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c007e20 con 0x7fc878155cd0 2023-10-06T05:16:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.876+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8781584e0 con 0x7fc878155cd0 2023-10-06T05:16:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.877+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8781589e0 con 0x7fc878155cd0 2023-10-06T05:16:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.877+0000 7fc8667fc700 1 -- 172.21.15.45:0/271696852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc86c010650 con 0x7fc878155cd0 2023-10-06T05:16:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.877+0000 7fc8667fc700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.877+0000 7fc8667fc700 1 -- 172.21.15.45:0/271696852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc86c014070 con 0x7fc878155cd0 2023-10-06T05:16:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.877+0000 7fc87659c700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.878+0000 7fc87659c700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc858066f30 0x7fc8580693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc86800acb0 tx=0x7fc868008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:05.882+0000 7fc8667fc700 1 -- 172.21.15.45:0/271696852 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc86c057ec0 con 0x7fc878155cd0 2023-10-06T05:16:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.173+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fc8780af170 con 0x7fc878155cd0 2023-10-06T05:16:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.177+0000 7fc8667fc700 1 -- 172.21.15.45:0/271696852 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc86c057a50 con 0x7fc878155cd0 2023-10-06T05:16:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc858066f30 msgr2=0x7fc8580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc858066f30 0x7fc8580693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc86800acb0 tx=0x7fc868008040 comp rx=0 tx=0).stop 2023-10-06T05:16:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878155cd0 msgr2=0x7fc878167440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878155cd0 0x7fc878167440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ee10 tx=0x7fc86c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 shutdown_connections 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8781560a0 0x7fc8780c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8780ccdc0 0x7fc878154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 --2- 172.21.15.45:0/271696852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc878155cd0 0x7fc878167440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 >> 172.21.15.45:0/271696852 conn(0x7fc87806f090 msgr2=0x7fc8780ca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 shutdown_connections 2023-10-06T05:16:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.179+0000 7fc87caef700 1 -- 172.21.15.45:0/271696852 wait complete. 2023-10-06T05:16:06.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.324+0000 7f3cba149700 1 Processor -- start 2023-10-06T05:16:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.324+0000 7f3cba149700 1 -- start start 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.324+0000 7f3cba149700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3cb4156ce0 0x7f3cb41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.324+0000 7f3cba149700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb406bf70 con 0x7f3cb406bad0 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.324+0000 7f3cba149700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb415b380 con 0x7f3cb415ac10 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.324+0000 7f3cba149700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb40b02f0 con 0x7f3cb4156ce0 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3cb4156ce0 0x7f3cb41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb2ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3cb406bad0 0x7f3cb40afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38926/0 (socket says 172.21.15.45:38926) 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb2ffd700 1 -- 172.21.15.45:0/1003771111 learned_addr learned my addr 172.21.15.45:0/1003771111 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb37fe700 1 -- 172.21.15.45:0/1003771111 >> v1:172.21.15.144:6789/0 conn(0x7f3cb415ac10 legacy=0x7f3cb40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb37fe700 1 -- 172.21.15.45:0/1003771111 >> v1:172.21.15.45:6789/0 conn(0x7f3cb406bad0 legacy=0x7f3cb40afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb37fe700 1 -- 172.21.15.45:0/1003771111 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb415b660 con 0x7f3cb4156ce0 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb37fe700 1 --2- 172.21.15.45:0/1003771111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb4156ce0 0x7f3cb41570b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40098e0 tx=0x7f3ca40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ef5e3632677a7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb27fc700 1 -- 172.21.15.45:0/1003771111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4009e80 con 0x7f3cb4156ce0 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb27fc700 1 -- 172.21.15.45:0/1003771111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca400b860 con 0x7f3cb4156ce0 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.325+0000 7f3cb27fc700 1 -- 172.21.15.45:0/1003771111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400bbe0 con 0x7f3cb4156ce0 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.326+0000 7f3cba149700 1 -- 172.21.15.45:0/1003771111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb4156ce0 msgr2=0x7f3cb41570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.326+0000 7f3cba149700 1 --2- 172.21.15.45:0/1003771111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb4156ce0 0x7f3cb41570b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40098e0 tx=0x7f3ca40091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.326+0000 7f3cba149700 1 -- 172.21.15.45:0/1003771111 shutdown_connections 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.326+0000 7f3cba149700 1 --2- 172.21.15.45:0/1003771111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb4156ce0 0x7f3cb41570b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.326+0000 7f3cba149700 1 -- 172.21.15.45:0/1003771111 >> 172.21.15.45:0/1003771111 conn(0x7f3cb406f090 msgr2=0x7f3cb406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.326+0000 7f3cba149700 1 -- 172.21.15.45:0/1003771111 shutdown_connections 2023-10-06T05:16:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.326+0000 7f3cba149700 1 -- 172.21.15.45:0/1003771111 wait complete. 2023-10-06T05:16:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.327+0000 7f3cba149700 1 Processor -- start 2023-10-06T05:16:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.327+0000 7f3cba149700 1 -- start start 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cba149700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb406bad0 0x7f3cb4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cba149700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb4156ce0 0x7f3cb415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cba149700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb415ac10 0x7f3cb40c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cba149700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb415a150 con 0x7f3cb415ac10 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cba149700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c3ac0 con 0x7f3cb406bad0 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cba149700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4159540 con 0x7f3cb4156ce0 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb415ac10 0x7f3cb40c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb415ac10 0x7f3cb40c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44968/0 (socket says 172.21.15.45:44968) 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cb3fff700 1 -- 172.21.15.45:0/1880422117 learned_addr learned my addr 172.21.15.45:0/1880422117 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.328+0000 7f3cb37fe700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb406bad0 0x7f3cb4158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cb2ffd700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb4156ce0 0x7f3cb415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cb3fff700 1 -- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb4156ce0 msgr2=0x7f3cb415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cb3fff700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb4156ce0 0x7f3cb415a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cb3fff700 1 -- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb406bad0 msgr2=0x7f3cb4158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cb3fff700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb406bad0 0x7f3cb4158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cb3fff700 1 -- 172.21.15.45:0/1880422117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4009420 con 0x7f3cb415ac10 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cb3fff700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb415ac10 0x7f3cb40c3580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c910 tx=0x7f3cac0016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3c9b7fe700 1 -- 172.21.15.45:0/1880422117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac012630 con 0x7f3cb415ac10 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3c9b7fe700 1 -- 172.21.15.45:0/1880422117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac012790 con 0x7f3cb415ac10 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb4159820 con 0x7f3cb415ac10 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3c9b7fe700 1 -- 172.21.15.45:0/1880422117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac012b50 con 0x7f3cb415ac10 2023-10-06T05:16:06.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.329+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb4159de0 con 0x7f3cb415ac10 2023-10-06T05:16:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.330+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb415a300 con 0x7f3cb415ac10 2023-10-06T05:16:06.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.331+0000 7f3c9b7fe700 1 -- 172.21.15.45:0/1880422117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cac01a3a0 con 0x7f3cb415ac10 2023-10-06T05:16:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.331+0000 7f3c9b7fe700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.331+0000 7f3c9b7fe700 1 -- 172.21.15.45:0/1880422117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f3cac084690 con 0x7f3cb415ac10 2023-10-06T05:16:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.331+0000 7f3cb37fe700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.332+0000 7f3cb37fe700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca0066f30 0x7f3ca00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40093f0 tx=0x7f3ca4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.335+0000 7f3c9b7fe700 1 -- 172.21.15.45:0/1880422117 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb415a300 con 0x7f3cb415ac10 2023-10-06T05:16:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.537+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cb415a290 con 0x7f3cb415ac10 2023-10-06T05:16:06.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.538+0000 7f3c9b7fe700 1 -- 172.21.15.45:0/1880422117 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13951 (secure 0 0 0) 0x7f3cac0519b0 con 0x7f3cb415ac10 2023-10-06T05:16:06.540 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:06.540 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:16:02.963742+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca0066f30 msgr2=0x7f3ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca0066f30 0x7f3ca00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40093f0 tx=0x7f3ca4017000 comp rx=0 tx=0).stop 2023-10-06T05:16:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb415ac10 msgr2=0x7f3cb40c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb415ac10 0x7f3cb40c3580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c910 tx=0x7f3cac0016b0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 shutdown_connections 2023-10-06T05:16:06.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca0066f30 0x7f3ca00693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb415ac10 0x7f3cb40c3580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb406bad0 0x7f3cb4158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 --2- 172.21.15.45:0/1880422117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb4156ce0 0x7f3cb415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 >> 172.21.15.45:0/1880422117 conn(0x7f3cb406f090 msgr2=0x7f3cb40687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 shutdown_connections 2023-10-06T05:16:06.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.540+0000 7f3cba149700 1 -- 172.21.15.45:0/1880422117 wait complete. 2023-10-06T05:16:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:16:06.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:16:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.689+0000 7f8699cd6700 1 Processor -- start 2023-10-06T05:16:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.690+0000 7f8699cd6700 1 -- start start 2023-10-06T05:16:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.690+0000 7f8699cd6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f86940d0680 0x7f869415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.690+0000 7f8699cd6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940cd740 con 0x7f86940d02b0 2023-10-06T05:16:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.690+0000 7f8699cd6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940ccb30 con 0x7f869415c6f0 2023-10-06T05:16:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.690+0000 7f8699cd6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940d0ad0 con 0x7f86940d0680 2023-10-06T05:16:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f8692ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f86940d0680 0x7f869415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f86937fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f86940d02b0 0x7f869415d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52286/0 (socket says 172.21.15.45:52286) 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f86937fe700 1 -- 172.21.15.45:0/3604877075 learned_addr learned my addr 172.21.15.45:0/3604877075 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f8692ffd700 1 -- 172.21.15.45:0/3604877075 >> v1:172.21.15.144:6789/0 conn(0x7f869415c6f0 legacy=0x7f86940caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f8692ffd700 1 -- 172.21.15.45:0/3604877075 >> v1:172.21.15.45:6789/0 conn(0x7f86940d02b0 legacy=0x7f869415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f86927fc700 1 -- 172.21.15.45:0/3604877075 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817723589 0 0) 0x7f86940cd740 con 0x7f86940d02b0 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f8692ffd700 1 -- 172.21.15.45:0/3604877075 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940cce10 con 0x7f86940d0680 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f8692ffd700 1 --2- 172.21.15.45:0/3604877075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d0680 0x7f869415c1b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f867c009910 tx=0x7f867c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245636871ed38098 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f86927fc700 1 -- 172.21.15.45:0/3604877075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00b820 con 0x7f86940d0680 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f86927fc700 1 -- 172.21.15.45:0/3604877075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00b980 con 0x7f86940d0680 2023-10-06T05:16:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.691+0000 7f86927fc700 1 -- 172.21.15.45:0/3604877075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00bd20 con 0x7f86940d0680 2023-10-06T05:16:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.692+0000 7f8699cd6700 1 -- 172.21.15.45:0/3604877075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d0680 msgr2=0x7f869415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.692+0000 7f8699cd6700 1 --2- 172.21.15.45:0/3604877075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d0680 0x7f869415c1b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f867c009910 tx=0x7f867c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.692+0000 7f8699cd6700 1 -- 172.21.15.45:0/3604877075 shutdown_connections 2023-10-06T05:16:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.692+0000 7f8699cd6700 1 --2- 172.21.15.45:0/3604877075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d0680 0x7f869415c1b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.692+0000 7f8699cd6700 1 -- 172.21.15.45:0/3604877075 >> 172.21.15.45:0/3604877075 conn(0x7f869406ed50 msgr2=0x7f869406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.692+0000 7f8699cd6700 1 -- 172.21.15.45:0/3604877075 shutdown_connections 2023-10-06T05:16:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.692+0000 7f8699cd6700 1 -- 172.21.15.45:0/3604877075 wait complete. 2023-10-06T05:16:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.693+0000 7f8699cd6700 1 Processor -- start 2023-10-06T05:16:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f8699cd6700 1 -- start start 2023-10-06T05:16:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f8699cd6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d02b0 0x7f869414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f8699cd6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86940d0680 0x7f86940cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f8699cd6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869415c6f0 0x7f86940cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f8699cd6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c5930 con 0x7f86940d02b0 2023-10-06T05:16:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f8699cd6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940c4d90 con 0x7f869415c6f0 2023-10-06T05:16:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f8699cd6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869414baf0 con 0x7f86940d0680 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f86937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d02b0 0x7f869414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f86937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d02b0 0x7f869414b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32940/0 (socket says 172.21.15.45:32940) 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.694+0000 7f86937fe700 1 -- 172.21.15.45:0/944091167 learned_addr learned my addr 172.21.15.45:0/944091167 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f8693fff700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869415c6f0 0x7f86940cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f8692ffd700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86940d0680 0x7f86940cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f86937fe700 1 -- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86940d0680 msgr2=0x7f86940cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f86937fe700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86940d0680 0x7f86940cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f86937fe700 1 -- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869415c6f0 msgr2=0x7f86940cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f86937fe700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869415c6f0 0x7f86940cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f86937fe700 1 -- 172.21.15.45:0/944091167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009420 con 0x7f86940d02b0 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f8693fff700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869415c6f0 0x7f86940cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f86937fe700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d02b0 0x7f869414b5b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f868400c840 tx=0x7f8684007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f868b7fe700 1 -- 172.21.15.45:0/944091167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684005c00 con 0x7f86940d02b0 2023-10-06T05:16:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f868b7fe700 1 -- 172.21.15.45:0/944091167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684005d80 con 0x7f86940d02b0 2023-10-06T05:16:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86940c5000 con 0x7f86940d02b0 2023-10-06T05:16:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f868b7fe700 1 -- 172.21.15.45:0/944091167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684007320 con 0x7f86940d02b0 2023-10-06T05:16:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.695+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86940c55c0 con 0x7f86940d02b0 2023-10-06T05:16:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.696+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8694088730 con 0x7f86940d02b0 2023-10-06T05:16:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.697+0000 7f868b7fe700 1 -- 172.21.15.45:0/944091167 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f868401b3b0 con 0x7f86940d02b0 2023-10-06T05:16:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.697+0000 7f868b7fe700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8674066f30 0x7f86740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.697+0000 7f868b7fe700 1 -- 172.21.15.45:0/944091167 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f8684083a70 con 0x7f86940d02b0 2023-10-06T05:16:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.697+0000 7f8692ffd700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8674066f30 0x7f86740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.698+0000 7f8692ffd700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8674066f30 0x7f86740693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f867c0093f0 tx=0x7f867c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:06.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.701+0000 7f868b7fe700 1 -- 172.21.15.45:0/944091167 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8684050d90 con 0x7f86940d02b0 2023-10-06T05:16:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.908+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86940cd0f0 con 0x7f86940d02b0 2023-10-06T05:16:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.909+0000 7f868b7fe700 1 -- 172.21.15.45:0/944091167 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8684017020 con 0x7f86940d02b0 2023-10-06T05:16:06.911 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:16:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8674066f30 msgr2=0x7f86740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8674066f30 0x7f86740693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f867c0093f0 tx=0x7f867c017000 comp rx=0 tx=0).stop 2023-10-06T05:16:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d02b0 msgr2=0x7f869414b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d02b0 0x7f869414b5b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f868400c840 tx=0x7f8684007d90 comp rx=0 tx=0).stop 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 shutdown_connections 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8674066f30 0x7f86740693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869415c6f0 0x7f86940cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940d02b0 0x7f869414b5b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 --2- 172.21.15.45:0/944091167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86940d0680 0x7f86940cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 >> 172.21.15.45:0/944091167 conn(0x7f869406ed50 msgr2=0x7f869415a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 shutdown_connections 2023-10-06T05:16:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:06.911+0000 7f8699cd6700 1 -- 172.21.15.45:0/944091167 wait complete. 2023-10-06T05:16:06.926 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:16:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.061+0000 7f9a0fb69700 1 Processor -- start 2023-10-06T05:16:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.061+0000 7f9a0fb69700 1 -- start start 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0fb69700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a08159150 0x7f9a080ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0fb69700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a08159520 0x7f9a0815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0fb69700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a0815d3d0 0x7f9a08067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0fb69700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080c9c90 con 0x7f9a0815d3d0 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0fb69700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0815d8e0 con 0x7f9a08159150 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0fb69700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0815dbc0 con 0x7f9a08159520 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a0815d3d0 0x7f9a08067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a0815d3d0 0x7f9a08067620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32948/0 (socket says 172.21.15.45:32948) 2023-10-06T05:16:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 -- 172.21.15.45:0/31830645 learned_addr learned my addr 172.21.15.45:0/31830645 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0d905700 1 --2- 172.21.15.45:0/31830645 >> v2:172.21.15.144:3300/0 conn(0x7f9a08159150 0x7f9a080ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0d104700 1 --2- 172.21.15.45:0/31830645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a08159520 0x7f9a0815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 -- 172.21.15.45:0/31830645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a08159520 msgr2=0x7f9a0815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 --2- 172.21.15.45:0/31830645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a08159520 0x7f9a0815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 -- 172.21.15.45:0/31830645 >> v2:172.21.15.144:3300/0 conn(0x7f9a08159150 msgr2=0x7f9a080ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 --2- 172.21.15.45:0/31830645 >> v2:172.21.15.144:3300/0 conn(0x7f9a08159150 0x7f9a080ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0e106700 1 -- 172.21.15.45:0/31830645 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080c9920 con 0x7f9a0815d3d0 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.062+0000 7f9a0d104700 1 --2- 172.21.15.45:0/31830645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a08159520 0x7f9a0815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0e106700 1 --2- 172.21.15.45:0/31830645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a08067620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9a080c3d80 tx=0x7f9a00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afc6f849a09b931 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0c903700 1 -- 172.21.15.45:0/31830645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000e070 con 0x7f9a0815d3d0 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0c903700 1 -- 172.21.15.45:0/31830645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0000ba20 con 0x7f9a0815d3d0 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0c903700 1 -- 172.21.15.45:0/31830645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000092e0 con 0x7f9a0815d3d0 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 -- 172.21.15.45:0/31830645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 msgr2=0x7f9a08067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/31830645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a08067620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9a080c3d80 tx=0x7f9a00009840 comp rx=0 tx=0).stop 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 -- 172.21.15.45:0/31830645 shutdown_connections 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/31830645 >> v2:172.21.15.144:3300/0 conn(0x7f9a08159150 0x7f9a080ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/31830645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a08067620 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/31830645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a08159520 0x7f9a0815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 -- 172.21.15.45:0/31830645 >> 172.21.15.45:0/31830645 conn(0x7f9a080691c0 msgr2=0x7f9a080cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 -- 172.21.15.45:0/31830645 shutdown_connections 2023-10-06T05:16:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.063+0000 7f9a0fb69700 1 -- 172.21.15.45:0/31830645 wait complete. 2023-10-06T05:16:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 Processor -- start 2023-10-06T05:16:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 -- start start 2023-10-06T05:16:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a08159150 0x7f9a08150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a0815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0815e7d0 0x7f9a08155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08150f80 con 0x7f9a0815d3d0 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0815ec50 con 0x7f9a08159150 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0fb69700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0816aac0 con 0x7f9a0815e7d0 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.065+0000 7f9a0d104700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a0815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a0815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32954/0 (socket says 172.21.15.45:32954) 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 -- 172.21.15.45:0/4279664578 learned_addr learned my addr 172.21.15.45:0/4279664578 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d905700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a08159150 0x7f9a08150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0e106700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0815e7d0 0x7f9a08155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 -- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0815e7d0 msgr2=0x7f9a08155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0815e7d0 0x7f9a08155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 -- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a08159150 msgr2=0x7f9a08150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a08159150 0x7f9a08150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 -- 172.21.15.45:0/4279664578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00009ae0 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d905700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a08159150 0x7f9a08150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0d104700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a0815e290 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400e900 tx=0x7f9a04008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f99fdffb700 1 -- 172.21.15.45:0/4279664578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04013410 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f99fdffb700 1 -- 172.21.15.45:0/4279664578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04013570 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a08169eb0 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f99fdffb700 1 -- 172.21.15.45:0/4279664578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04013910 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.066+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0816a470 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.067+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0806e2d0 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.068+0000 7f99fdffb700 1 -- 172.21.15.45:0/4279664578 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a04013d00 con 0x7f9a0815d3d0 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.068+0000 7f99fdffb700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f99ec066f30 0x7f99ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.069+0000 7f99fdffb700 1 -- 172.21.15.45:0/4279664578 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f9a040842e0 con 0x7f9a0815d3d0 2023-10-06T05:16:07.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.069+0000 7f9a0d905700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f99ec066f30 0x7f99ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.069+0000 7f9a0d905700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f99ec066f30 0x7f99ec0693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009d80 tx=0x7f99f8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.073+0000 7f99fdffb700 1 -- 172.21.15.45:0/4279664578 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a04051560 con 0x7f9a0815d3d0 2023-10-06T05:16:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.275+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a0816ac00 con 0x7f9a0815d3d0 2023-10-06T05:16:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.275+0000 7f99fdffb700 1 -- 172.21.15.45:0/4279664578 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a04054b80 con 0x7f9a0815d3d0 2023-10-06T05:16:07.277 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f99ec066f30 msgr2=0x7f99ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f99ec066f30 0x7f99ec0693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009d80 tx=0x7f99f8008040 comp rx=0 tx=0).stop 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 msgr2=0x7f9a0815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a0815e290 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400e900 tx=0x7f9a04008040 comp rx=0 tx=0).stop 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 shutdown_connections 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f99ec066f30 0x7f99ec0693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0815e7d0 0x7f9a08155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a08159150 0x7f9a08150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 --2- 172.21.15.45:0/4279664578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0815d3d0 0x7f9a0815e290 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 >> 172.21.15.45:0/4279664578 conn(0x7f9a080691c0 msgr2=0x7f9a080b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 shutdown_connections 2023-10-06T05:16:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.278+0000 7f9a0fb69700 1 -- 172.21.15.45:0/4279664578 wait complete. 2023-10-06T05:16:07.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:16:07.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.422+0000 7fbea679d700 1 Processor -- start 2023-10-06T05:16:07.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.422+0000 7fbea679d700 1 -- start start 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea679d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c93c0 0x7fbea0156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea679d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbea0157030 0x7fbea00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea679d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea015b000 con 0x7fbea00c8ff0 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea679d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea00c9830 con 0x7fbea0157030 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea679d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea015b2e0 con 0x7fbea00c93c0 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea4d3a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbea0157030 0x7fbea00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea4d3a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbea0157030 0x7fbea00cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32964/0 (socket says 172.21.15.45:32964) 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbea4d3a700 1 -- 172.21.15.45:0/2660568951 learned_addr learned my addr 172.21.15.45:0/2660568951 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.423+0000 7fbe9f7fe700 1 --2- 172.21.15.45:0/2660568951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c93c0 0x7fbea0156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbea4d3a700 1 -- 172.21.15.45:0/2660568951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c93c0 msgr2=0x7fbea0156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbea4d3a700 1 --2- 172.21.15.45:0/2660568951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c93c0 0x7fbea0156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbea4d3a700 1 -- 172.21.15.45:0/2660568951 >> v1:172.21.15.144:6789/0 conn(0x7fbea00c8ff0 legacy=0x7fbea00caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbea4d3a700 1 -- 172.21.15.45:0/2660568951 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea015b5c0 con 0x7fbea0157030 2023-10-06T05:16:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbe9f7fe700 1 --2- 172.21.15.45:0/2660568951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c93c0 0x7fbea0156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbea4d3a700 1 --2- 172.21.15.45:0/2660568951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea0157030 0x7fbea00cc330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009f20 tx=0x7fbe98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6da6fd7e2753cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbe9effd700 1 -- 172.21.15.45:0/2660568951 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800e070 con 0x7fbea0157030 2023-10-06T05:16:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbe9effd700 1 -- 172.21.15.45:0/2660568951 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9800ba80 con 0x7fbea0157030 2023-10-06T05:16:07.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbe9effd700 1 -- 172.21.15.45:0/2660568951 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe980092e0 con 0x7fbea0157030 2023-10-06T05:16:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbea679d700 1 -- 172.21.15.45:0/2660568951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea0157030 msgr2=0x7fbea00cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.424+0000 7fbea679d700 1 --2- 172.21.15.45:0/2660568951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea0157030 0x7fbea00cc330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009f20 tx=0x7fbe98009840 comp rx=0 tx=0).stop 2023-10-06T05:16:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.425+0000 7fbea679d700 1 -- 172.21.15.45:0/2660568951 shutdown_connections 2023-10-06T05:16:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.425+0000 7fbea679d700 1 --2- 172.21.15.45:0/2660568951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea0157030 0x7fbea00cc330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.425+0000 7fbea679d700 1 --2- 172.21.15.45:0/2660568951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c93c0 0x7fbea0156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.425+0000 7fbea679d700 1 -- 172.21.15.45:0/2660568951 >> 172.21.15.45:0/2660568951 conn(0x7fbea00cdf50 msgr2=0x7fbea00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.425+0000 7fbea679d700 1 -- 172.21.15.45:0/2660568951 shutdown_connections 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.425+0000 7fbea679d700 1 -- 172.21.15.45:0/2660568951 wait complete. 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.426+0000 7fbea679d700 1 Processor -- start 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.426+0000 7fbea679d700 1 -- start start 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbea679d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c8ff0 0x7fbea00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbea679d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00c93c0 0x7fbea01475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbea679d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbea0157030 0x7fbea0158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbea679d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea01582e0 con 0x7fbea00c93c0 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbea679d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00c6b80 con 0x7fbea0157030 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbea679d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0157430 con 0x7fbea00c8ff0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00c93c0 0x7fbea01475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00c93c0 0x7fbea01475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32976/0 (socket says 172.21.15.45:32976) 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 -- 172.21.15.45:0/4246336376 learned_addr learned my addr 172.21.15.45:0/4246336376 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbea4d3a700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbea0157030 0x7fbea0158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9ffff700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c8ff0 0x7fbea00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 -- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c8ff0 msgr2=0x7fbea00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c8ff0 0x7fbea00c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 -- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbea0157030 msgr2=0x7fbea0158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbea0157030 0x7fbea0158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 -- 172.21.15.45:0/4246336376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98009ae0 con 0x7fbea00c93c0 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.427+0000 7fbe9f7fe700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00c93c0 0x7fbea01475c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e900 tx=0x7fbe94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.428+0000 7fbe87fff700 1 -- 172.21.15.45:0/4246336376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94013410 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.428+0000 7fbe87fff700 1 -- 172.21.15.45:0/4246336376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe94013570 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.428+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea0147b00 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.428+0000 7fbe87fff700 1 -- 172.21.15.45:0/4246336376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94013910 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.428+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea01579b0 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.429+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea0158490 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.429+0000 7fbe87fff700 1 -- 172.21.15.45:0/4246336376 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe94013cc0 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.430+0000 7fbe87fff700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe8c066f30 0x7fbe8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.430+0000 7fbe87fff700 1 -- 172.21.15.45:0/4246336376 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fbe94084120 con 0x7fbea00c93c0 2023-10-06T05:16:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.430+0000 7fbe9ffff700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe8c066f30 0x7fbe8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.430+0000 7fbe9ffff700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe8c066f30 0x7fbe8c0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe900039e0 tx=0x7fbe90007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.434+0000 7fbe87fff700 1 -- 172.21.15.45:0/4246336376 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbea0158490 con 0x7fbea00c93c0 2023-10-06T05:16:07.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.639+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbea0158490 con 0x7fbea00c93c0 2023-10-06T05:16:07.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.640+0000 7fbe87fff700 1 -- 172.21.15.45:0/4246336376 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbea0158490 con 0x7fbea00c93c0 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-10-06T05:16:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-10-06T05:16:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:07.643 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:16:07.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe8c066f30 msgr2=0x7fbe8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe8c066f30 0x7fbe8c0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe900039e0 tx=0x7fbe90007ec0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00c93c0 msgr2=0x7fbea01475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00c93c0 0x7fbea01475c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e900 tx=0x7fbe94008040 comp rx=0 tx=0).stop 2023-10-06T05:16:07.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 shutdown_connections 2023-10-06T05:16:07.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe8c066f30 0x7fbe8c0693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbea0157030 0x7fbea0158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbea00c8ff0 0x7fbea00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 --2- 172.21.15.45:0/4246336376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00c93c0 0x7fbea01475c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 >> 172.21.15.45:0/4246336376 conn(0x7fbea00cdf50 msgr2=0x7fbea0151590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 shutdown_connections 2023-10-06T05:16:07.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.642+0000 7fbea679d700 1 -- 172.21.15.45:0/4246336376 wait complete. 2023-10-06T05:16:07.657 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:16:07.657 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:16:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.791+0000 7f144d367700 1 Processor -- start 2023-10-06T05:16:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.792+0000 7f144d367700 1 -- start start 2023-10-06T05:16:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.792+0000 7f144d367700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f580 0x7f144806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.792+0000 7f144d367700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14480aeaa0 0x7f14480ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.792+0000 7f144d367700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14480ca800 con 0x7f144813f950 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.792+0000 7f144d367700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14480c6d60 con 0x7f14480aeaa0 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.792+0000 7f144d367700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480caa30 con 0x7f144813f580 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f580 0x7f144806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f14467fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14480aeaa0 0x7f14480ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f14467fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14480aeaa0 0x7f14480ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:38510/0 (socket says 172.21.15.45:38510) 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f580 0x7f144806b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:49234/0 (socket says 172.21.15.45:49234) 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f14467fc700 1 -- 172.21.15.45:0/2372537505 learned_addr learned my addr 172.21.15.45:0/2372537505 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f14477fe700 1 --1- 172.21.15.45:0/2372537505 >> v1:172.21.15.144:6789/0 conn(0x7f144813f950 0x7f144806a020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:48202/0 (socket says 172.21.15.45:48202) 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f1446ffd700 1 -- 172.21.15.45:0/2372537505 >> v2:172.21.15.144:3300/0 conn(0x7f14480aeaa0 msgr2=0x7f14480ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f1446ffd700 1 --2- 172.21.15.45:0/2372537505 >> v2:172.21.15.144:3300/0 conn(0x7f14480aeaa0 0x7f14480ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f1446ffd700 1 -- 172.21.15.45:0/2372537505 >> v1:172.21.15.144:6789/0 conn(0x7f144813f950 legacy=0x7f144806a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f1446ffd700 1 -- 172.21.15.45:0/2372537505 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480cbab0 con 0x7f144813f580 2023-10-06T05:16:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f14467fc700 1 --2- 172.21.15.45:0/2372537505 >> v2:172.21.15.144:3300/0 conn(0x7f14480aeaa0 0x7f14480ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.793+0000 7f1446ffd700 1 --2- 172.21.15.45:0/2372537505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f580 0x7f144806b250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14380098e0 tx=0x7f14380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f111d0cecd041240 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f1445ffb700 1 -- 172.21.15.45:0/2372537505 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800b640 con 0x7f144813f580 2023-10-06T05:16:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f1445ffb700 1 -- 172.21.15.45:0/2372537505 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800b7a0 con 0x7f144813f580 2023-10-06T05:16:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f1445ffb700 1 -- 172.21.15.45:0/2372537505 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800bb40 con 0x7f144813f580 2023-10-06T05:16:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 -- 172.21.15.45:0/2372537505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f580 msgr2=0x7f144806b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 --2- 172.21.15.45:0/2372537505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f580 0x7f144806b250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14380098e0 tx=0x7f14380091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 -- 172.21.15.45:0/2372537505 shutdown_connections 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 --2- 172.21.15.45:0/2372537505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f580 0x7f144806b250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 --2- 172.21.15.45:0/2372537505 >> v2:172.21.15.144:3300/0 conn(0x7f14480aeaa0 0x7f14480ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 -- 172.21.15.45:0/2372537505 >> 172.21.15.45:0/2372537505 conn(0x7f1448067620 msgr2=0x7f1448067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 -- 172.21.15.45:0/2372537505 shutdown_connections 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.794+0000 7f144d367700 1 -- 172.21.15.45:0/2372537505 wait complete. 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 Processor -- start 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 -- start start 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14480aeaa0 0x7f144813dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144813f580 0x7f14480c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f950 0x7f14480c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c48b0 con 0x7f14480aeaa0 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c4d40 con 0x7f144813f580 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.796+0000 7f144d367700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c3f80 con 0x7f144813f950 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14480aeaa0 0x7f144813dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14480aeaa0 0x7f144813dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32990/0 (socket says 172.21.15.45:32990) 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 -- 172.21.15.45:0/2806440619 learned_addr learned my addr 172.21.15.45:0/2806440619 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f14477fe700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f950 0x7f14480c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f14467fc700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144813f580 0x7f14480c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 -- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f950 msgr2=0x7f14480c0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f950 0x7f14480c0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 -- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144813f580 msgr2=0x7f14480c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144813f580 0x7f14480c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 -- 172.21.15.45:0/2806440619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300094f0 con 0x7f14480aeaa0 2023-10-06T05:16:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f1446ffd700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14480aeaa0 0x7f144813dfb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14480c9240 tx=0x7f1438017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f143effd700 1 -- 172.21.15.45:0/2806440619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143801bc40 con 0x7f14480aeaa0 2023-10-06T05:16:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f143effd700 1 -- 172.21.15.45:0/2806440619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143801bdc0 con 0x7f14480aeaa0 2023-10-06T05:16:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f144d367700 1 -- 172.21.15.45:0/2806440619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1438009420 con 0x7f14480aeaa0 2023-10-06T05:16:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.797+0000 7f143effd700 1 -- 172.21.15.45:0/2806440619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438011670 con 0x7f14480aeaa0 2023-10-06T05:16:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.798+0000 7f144d367700 1 -- 172.21.15.45:0/2806440619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14480c4540 con 0x7f14480aeaa0 2023-10-06T05:16:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.798+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14200052b0 con 0x7f14480aeaa0 2023-10-06T05:16:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.799+0000 7f143effd700 1 -- 172.21.15.45:0/2806440619 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1438011a20 con 0x7f14480aeaa0 2023-10-06T05:16:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.799+0000 7f143effd700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1428066f30 0x7f14280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.799+0000 7f143effd700 1 -- 172.21.15.45:0/2806440619 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f14380859b0 con 0x7f14480aeaa0 2023-10-06T05:16:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.799+0000 7f14467fc700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1428066f30 0x7f14280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:07.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.800+0000 7f14467fc700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1428066f30 0x7f14280693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f144815ba40 tx=0x7f1430009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:07.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:07.804+0000 7f143effd700 1 -- 172.21.15.45:0/2806440619 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1438052cd0 con 0x7f14480aeaa0 2023-10-06T05:16:08.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.008+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1420005600 con 0x7f14480aeaa0 2023-10-06T05:16:08.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.183+0000 7f143effd700 1 -- 172.21.15.45:0/2806440619 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14380562f0 con 0x7f14480aeaa0 2023-10-06T05:16:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1428066f30 msgr2=0x7f14280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1428066f30 0x7f14280693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f144815ba40 tx=0x7f1430009250 comp rx=0 tx=0).stop 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14480aeaa0 msgr2=0x7f144813dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14480aeaa0 0x7f144813dfb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14480c9240 tx=0x7f1438017000 comp rx=0 tx=0).stop 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 shutdown_connections 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1428066f30 0x7f14280693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f144813f950 0x7f14480c0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14480aeaa0 0x7f144813dfb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 --2- 172.21.15.45:0/2806440619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144813f580 0x7f14480c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.186+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 >> 172.21.15.45:0/2806440619 conn(0x7f1448067620 msgr2=0x7f144815f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:08.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.187+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 shutdown_connections 2023-10-06T05:16:08.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.187+0000 7f143cff9700 1 -- 172.21.15.45:0/2806440619 wait complete. 2023-10-06T05:16:08.191 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-06T05:16:08.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:16:08.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.347+0000 7fe539118700 1 Processor -- start 2023-10-06T05:16:08.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.347+0000 7fe539118700 1 -- start start 2023-10-06T05:16:08.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe539118700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe534159520 0x7fe53415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe539118700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 0x7fe534067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe539118700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5340c9c90 con 0x7fe534159150 2023-10-06T05:16:08.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe539118700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53415d8e0 con 0x7fe534159520 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe539118700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415dbc0 con 0x7fe53415d3d0 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe532d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe534159150 0x7fe5340ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52288/0 (socket says 172.21.15.45:52288) 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe532d9d700 1 -- 172.21.15.45:0/351690944 learned_addr learned my addr 172.21.15.45:0/351690944 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe53359e700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 0x7fe534067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe53259c700 1 --2- 172.21.15.45:0/351690944 >> v2:172.21.15.144:3300/0 conn(0x7fe534159520 0x7fe53415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.348+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886025530 0 0) 0x7fe5340c9c90 con 0x7fe534159150 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe520003570 con 0x7fe534159150 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1726277449 0 0) 0x7fe520003570 con 0x7fe534159150 2023-10-06T05:16:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 msgr2=0x7fe534067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 0x7fe534067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 >> v2:172.21.15.144:3300/0 conn(0x7fe534159520 msgr2=0x7fe53415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 --2- 172.21.15.45:0/351690944 >> v2:172.21.15.144:3300/0 conn(0x7fe534159520 0x7fe53415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340c9920 con 0x7fe534159150 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fe524001470 con 0x7fe534159150 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe53359e700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 0x7fe534067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 >> v1:172.21.15.45:6789/0 conn(0x7fe534159150 legacy=0x7fe5340ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe520003a60 0x7fe520003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe520004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 0x7fe5200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415d8e0 con 0x7fe520004810 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415dbc0 con 0x7fe520003a60 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52000ba90 con 0x7fe53415d3d0 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe53259c700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe520004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe532d9d700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe520003a60 0x7fe520003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe53359e700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 0x7fe5200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe53259c700 1 -- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 msgr2=0x7fe5200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe53259c700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe53415d3d0 0x7fe5200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe53259c700 1 -- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe520003a60 msgr2=0x7fe520003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.349+0000 7fe53259c700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe520003a60 0x7fe520003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe53259c700 1 -- 172.21.15.45:0/351690944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009a20 con 0x7fe520004810 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe53259c700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe520004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009c00 tx=0x7fe51c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7b8dfdab63770c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c00e6b0 con 0x7fe520004810 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c00e810 con 0x7fe520004810 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe531d9b700 1 -- 172.21.15.45:0/351690944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c00ebd0 con 0x7fe520004810 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe539118700 1 -- 172.21.15.45:0/351690944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 msgr2=0x7fe520004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe539118700 1 --2- 172.21.15.45:0/351690944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe520004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009c00 tx=0x7fe51c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe53359e700 1 -- 172.21.15.45:0/351690944 reap_dead start 2023-10-06T05:16:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe539118700 1 -- 172.21.15.45:0/351690944 shutdown_connections 2023-10-06T05:16:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.350+0000 7fe539118700 1 -- 172.21.15.45:0/351690944 >> 172.21.15.45:0/351690944 conn(0x7fe5340691c0 msgr2=0x7fe5340cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.351+0000 7fe539118700 1 -- 172.21.15.45:0/351690944 shutdown_connections 2023-10-06T05:16:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.351+0000 7fe539118700 1 -- 172.21.15.45:0/351690944 wait complete. 2023-10-06T05:16:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.352+0000 7fe539118700 1 Processor -- start 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.352+0000 7fe539118700 1 -- start start 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe539118700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe53415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe539118700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534150a80 0x7fe5341697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe539118700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415c140 0x7fe534169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe539118700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340b0240 con 0x7fe520004810 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe539118700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415c6e0 con 0x7fe53415c140 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe539118700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415c9c0 con 0x7fe534150a80 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe53415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe53415bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33000/0 (socket says 172.21.15.45:33000) 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 -- 172.21.15.45:0/1069752165 learned_addr learned my addr 172.21.15.45:0/1069752165 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe53359e700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415c140 0x7fe534169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe53259c700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534150a80 0x7fe5341697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 -- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534150a80 msgr2=0x7fe5341697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534150a80 0x7fe5341697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 -- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415c140 msgr2=0x7fe534169d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415c140 0x7fe534169d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 -- 172.21.15.45:0/1069752165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340b0490 con 0x7fe520004810 2023-10-06T05:16:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.353+0000 7fe532d9d700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe53415bc00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe524007390 tx=0x7fe524001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.354+0000 7fe52affd700 1 -- 172.21.15.45:0/1069752165 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524018030 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.354+0000 7fe52affd700 1 -- 172.21.15.45:0/1069752165 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe524007770 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.354+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5340b0770 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.354+0000 7fe52affd700 1 -- 172.21.15.45:0/1069752165 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52400fa80 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.354+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5340b0c90 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.355+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5340b0950 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.355+0000 7fe52affd700 1 -- 172.21.15.45:0/1069752165 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe52401b070 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.356+0000 7fe52affd700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe514066f30 0x7fe5140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.356+0000 7fe52affd700 1 -- 172.21.15.45:0/1069752165 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5896+0+0 (secure 0 0 0) 0x7fe524019070 con 0x7fe520004810 2023-10-06T05:16:08.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.356+0000 7fe53259c700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe514066f30 0x7fe5140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.357+0000 7fe53259c700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe514066f30 0x7fe5140693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0091e0 tx=0x7fe51c01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.360+0000 7fe52affd700 1 -- 172.21.15.45:0/1069752165 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe524052080 con 0x7fe520004810 2023-10-06T05:16:08.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:08.561+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe5340cdba0 con 0x7fe520004810 2023-10-06T05:16:08.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T05:16:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.189+0000 7fe52affd700 1 -- 172.21.15.45:0/1069752165 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe524012350 con 0x7fe520004810 2023-10-06T05:16:09.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe514066f30 msgr2=0x7fe5140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe514066f30 0x7fe5140693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0091e0 tx=0x7fe51c01a000 comp rx=0 tx=0).stop 2023-10-06T05:16:09.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 msgr2=0x7fe53415bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe53415bc00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe524007390 tx=0x7fe524001e90 comp rx=0 tx=0).stop 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 shutdown_connections 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe514066f30 0x7fe5140693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415c140 0x7fe534169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe520004810 0x7fe53415bc00 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe524007390 tx=0x7fe524001e90 comp rx=0 tx=0).stop 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 --2- 172.21.15.45:0/1069752165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534150a80 0x7fe5341697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.193+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 >> 172.21.15.45:0/1069752165 conn(0x7fe5340691c0 msgr2=0x7fe5340b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.194+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 shutdown_connections 2023-10-06T05:16:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.194+0000 7fe539118700 1 -- 172.21.15.45:0/1069752165 wait complete. 2023-10-06T05:16:09.214 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-06T05:16:09.226 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:16:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.361+0000 7f20afa4f700 1 Processor -- start 2023-10-06T05:16:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.362+0000 7f20afa4f700 1 -- start start 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20afa4f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a813f900 0x7f20a80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20afa4f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20a80c8de0 0x7f20a80c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20afa4f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80c9630 con 0x7f20a80c9d50 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20afa4f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a80ca150 con 0x7f20a80c8de0 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20afa4f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80cb550 con 0x7f20a813f900 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20a80c8de0 0x7f20a80c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20a80c8de0 0x7f20a80c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33016/0 (socket says 172.21.15.45:33016) 2023-10-06T05:16:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 -- 172.21.15.45:0/925481271 learned_addr learned my addr 172.21.15.45:0/925481271 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20ad7eb700 1 --2- 172.21.15.45:0/925481271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a813f900 0x7f20a80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 -- 172.21.15.45:0/925481271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a813f900 msgr2=0x7f20a80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 --2- 172.21.15.45:0/925481271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a813f900 0x7f20a80cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 -- 172.21.15.45:0/925481271 >> v1:172.21.15.144:6789/0 conn(0x7f20a80c9d50 legacy=0x7f20a806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 -- 172.21.15.45:0/925481271 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80cb830 con 0x7f20a80c8de0 2023-10-06T05:16:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.363+0000 7f20acfea700 1 --2- 172.21.15.45:0/925481271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a80c8de0 0x7f20a80c9810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20980099b0 tx=0x7f2098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a8560d5ab2d6db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f209ffff700 1 -- 172.21.15.45:0/925481271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800b860 con 0x7f20a80c8de0 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f209ffff700 1 -- 172.21.15.45:0/925481271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800b9c0 con 0x7f20a80c8de0 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f209ffff700 1 -- 172.21.15.45:0/925481271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800bd60 con 0x7f20a80c8de0 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f20afa4f700 1 -- 172.21.15.45:0/925481271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a80c8de0 msgr2=0x7f20a80c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f20afa4f700 1 --2- 172.21.15.45:0/925481271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a80c8de0 0x7f20a80c9810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20980099b0 tx=0x7f2098009250 comp rx=0 tx=0).stop 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f20afa4f700 1 -- 172.21.15.45:0/925481271 shutdown_connections 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f20afa4f700 1 --2- 172.21.15.45:0/925481271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a813f900 0x7f20a80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f20afa4f700 1 --2- 172.21.15.45:0/925481271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a80c8de0 0x7f20a80c9810 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.364+0000 7f20afa4f700 1 -- 172.21.15.45:0/925481271 >> 172.21.15.45:0/925481271 conn(0x7f20a80691c0 msgr2=0x7f20a80cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.365+0000 7f20afa4f700 1 -- 172.21.15.45:0/925481271 shutdown_connections 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.365+0000 7f20afa4f700 1 -- 172.21.15.45:0/925481271 wait complete. 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.366+0000 7f20afa4f700 1 Processor -- start 2023-10-06T05:16:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.366+0000 7f20afa4f700 1 -- start start 2023-10-06T05:16:09.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20afa4f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a80c8de0 0x7f20a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20afa4f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20a80c9d50 0x7f20a80ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20afa4f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a813f900 0x7f20a80aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20afa4f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a8166570 con 0x7f20a813f900 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20afa4f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a812f9b0 con 0x7f20a80c9d50 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20afa4f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a812eda0 con 0x7f20a80c8de0 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a813f900 0x7f20a80aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a813f900 0x7f20a80aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33020/0 (socket says 172.21.15.45:33020) 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 -- 172.21.15.45:0/818082502 learned_addr learned my addr 172.21.15.45:0/818082502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20acfea700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20a80c9d50 0x7f20a80ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20ad7eb700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a80c8de0 0x7f20a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 -- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a80c8de0 msgr2=0x7f20a806d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a80c8de0 0x7f20a806d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 -- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20a80c9d50 msgr2=0x7f20a80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20a80c9d50 0x7f20a80ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20adfec700 1 -- 172.21.15.45:0/818082502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980094f0 con 0x7f20a813f900 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.367+0000 7f20ad7eb700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a80c8de0 0x7f20a806d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.368+0000 7f20adfec700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a813f900 0x7f20a80aec20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c8e0 tx=0x7f20a0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.368+0000 7f209d7fa700 1 -- 172.21.15.45:0/818082502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0012630 con 0x7f20a813f900 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.368+0000 7f209d7fa700 1 -- 172.21.15.45:0/818082502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a0012790 con 0x7f20a813f900 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.368+0000 7f209d7fa700 1 -- 172.21.15.45:0/818082502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0012b50 con 0x7f20a813f900 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.368+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a812f080 con 0x7f20a813f900 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.368+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a812f640 con 0x7f20a813f900 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.369+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a806e2d0 con 0x7f20a813f900 2023-10-06T05:16:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.369+0000 7f209d7fa700 1 -- 172.21.15.45:0/818082502 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20a000a3b0 con 0x7f20a813f900 2023-10-06T05:16:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.370+0000 7f209d7fa700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f208c066f30 0x7f208c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.370+0000 7f209d7fa700 1 -- 172.21.15.45:0/818082502 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f20a0084a00 con 0x7f20a813f900 2023-10-06T05:16:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.370+0000 7f20ad7eb700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f208c066f30 0x7f208c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.371+0000 7f20ad7eb700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f208c066f30 0x7f208c0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f20a4006fd0 tx=0x7f20a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.375+0000 7f209d7fa700 1 -- 172.21.15.45:0/818082502 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20a00554c0 con 0x7f20a813f900 2023-10-06T05:16:09.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:09.622+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f20a80cbb10 con 0x7f20a813f900 2023-10-06T05:16:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.192+0000 7f209d7fa700 1 -- 172.21.15.45:0/818082502 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f20a0055050 con 0x7f20a813f900 2023-10-06T05:16:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.224+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f208c066f30 msgr2=0x7f208c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.224+0000 7f20afa4f700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f208c066f30 0x7f208c0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f20a4006fd0 tx=0x7f20a4008040 comp rx=0 tx=0).stop 2023-10-06T05:16:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.224+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a813f900 msgr2=0x7f20a80aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.224+0000 7f20afa4f700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a813f900 0x7f20a80aec20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c8e0 tx=0x7f20a0001710 comp rx=0 tx=0).stop 2023-10-06T05:16:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.227+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 shutdown_connections 2023-10-06T05:16:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.227+0000 7f20afa4f700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f208c066f30 0x7f208c0693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.227+0000 7f20afa4f700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a813f900 0x7f20a80aec20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.227+0000 7f20afa4f700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20a80c8de0 0x7f20a806d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.227+0000 7f20afa4f700 1 --2- 172.21.15.45:0/818082502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20a80c9d50 0x7f20a80ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.227+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 >> 172.21.15.45:0/818082502 conn(0x7f20a80691c0 msgr2=0x7f20a80c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.228+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 shutdown_connections 2023-10-06T05:16:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.228+0000 7f20afa4f700 1 -- 172.21.15.45:0/818082502 wait complete. 2023-10-06T05:16:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 6 and data pool 7 2023-10-06T05:16:10.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:10.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.377+0000 7f3cbfe47700 1 Processor -- start 2023-10-06T05:16:10.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.377+0000 7f3cbfe47700 1 -- start start 2023-10-06T05:16:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.378+0000 7f3cbfe47700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.378+0000 7f3cbfe47700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb80ca120 con 0x7f3cb806de20 2023-10-06T05:16:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.378+0000 7f3cbfe47700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb806e230 con 0x7f3cb81609b0 2023-10-06T05:16:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.378+0000 7f3cbfe47700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80c9510 con 0x7f3cb8160d80 2023-10-06T05:16:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.378+0000 7f3cbe3e4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3cb806de20 0x7f3cb8068730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52300/0 (socket says 172.21.15.45:52300) 2023-10-06T05:16:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.378+0000 7f3cbe3e4700 1 -- 172.21.15.45:0/3402333924 learned_addr learned my addr 172.21.15.45:0/3402333924 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.378+0000 7f3cbd3e2700 1 --2- 172.21.15.45:0/3402333924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276050406 0 0) 0x7f3cb80ca120 con 0x7f3cb806de20 2023-10-06T05:16:10.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ca4003570 con 0x7f3cb806de20 2023-10-06T05:16:10.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652760943 0 0) 0x7f3cb806e230 con 0x7f3cb81609b0 2023-10-06T05:16:10.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb80ca120 con 0x7f3cb81609b0 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387018107 0 0) 0x7f3ca4003570 con 0x7f3cb806de20 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbd3e2700 1 -- 172.21.15.45:0/3402333924 >> v1:172.21.15.144:6789/0 conn(0x7f3cb81609b0 legacy=0x7f3cb80c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbd3e2700 1 -- 172.21.15.45:0/3402333924 >> v1:172.21.15.45:6789/0 conn(0x7f3cb806de20 legacy=0x7f3cb8068730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbd3e2700 1 -- 172.21.15.45:0/3402333924 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb80c97f0 con 0x7f3cb8160d80 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbd3e2700 1 --2- 172.21.15.45:0/3402333924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb806d8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009910 tx=0x7f3ca80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3fb846a9b797bae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00017f0 con 0x7f3cb8160d80 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8011420 con 0x7f3cb8160d80 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.379+0000 7f3cbcbe1700 1 -- 172.21.15.45:0/3402333924 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80117a0 con 0x7f3cb8160d80 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.380+0000 7f3cbfe47700 1 -- 172.21.15.45:0/3402333924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 msgr2=0x7f3cb806d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.380+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/3402333924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb806d8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009910 tx=0x7f3ca80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.380+0000 7f3cbfe47700 1 -- 172.21.15.45:0/3402333924 shutdown_connections 2023-10-06T05:16:10.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.380+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/3402333924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb806d8e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.380+0000 7f3cbfe47700 1 -- 172.21.15.45:0/3402333924 >> 172.21.15.45:0/3402333924 conn(0x7f3cb80ce9b0 msgr2=0x7f3cb806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.380+0000 7f3cbfe47700 1 -- 172.21.15.45:0/3402333924 shutdown_connections 2023-10-06T05:16:10.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.380+0000 7f3cbfe47700 1 -- 172.21.15.45:0/3402333924 wait complete. 2023-10-06T05:16:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.381+0000 7f3cbfe47700 1 Processor -- start 2023-10-06T05:16:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.381+0000 7f3cbfe47700 1 -- start start 2023-10-06T05:16:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbfe47700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb806de20 0x7f3cb80c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbfe47700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb81609b0 0x7f3cb8152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbfe47700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb80c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbfe47700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80c7d20 con 0x7f3cb806de20 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbfe47700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80c65b0 con 0x7f3cb81609b0 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbfe47700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80c70a0 con 0x7f3cb8160d80 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbdbe3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb806de20 0x7f3cb80c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbdbe3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb806de20 0x7f3cb80c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33036/0 (socket says 172.21.15.45:33036) 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbdbe3700 1 -- 172.21.15.45:0/2894595474 learned_addr learned my addr 172.21.15.45:0/2894595474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbe3e4700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb80c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.382+0000 7f3cbd3e2700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb81609b0 0x7f3cb8152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbdbe3700 1 -- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 msgr2=0x7f3cb80c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbdbe3700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb80c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbdbe3700 1 -- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb81609b0 msgr2=0x7f3cb8152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbdbe3700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb81609b0 0x7f3cb8152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbdbe3700 1 -- 172.21.15.45:0/2894595474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8009420 con 0x7f3cb806de20 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbdbe3700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb806de20 0x7f3cb80c6ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400c7c0 tx=0x7f3cb4007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cadffb700 1 -- 172.21.15.45:0/2894595474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4002ea0 con 0x7f3cb806de20 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cadffb700 1 -- 172.21.15.45:0/2894595474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4010510 con 0x7f3cb806de20 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb80c7380 con 0x7f3cb806de20 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cadffb700 1 -- 172.21.15.45:0/2894595474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40108b0 con 0x7f3cb806de20 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.383+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb80c7940 con 0x7f3cb806de20 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.384+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb806f390 con 0x7f3cb806de20 2023-10-06T05:16:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.385+0000 7f3cadffb700 1 -- 172.21.15.45:0/2894595474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cb4010c60 con 0x7f3cb806de20 2023-10-06T05:16:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.385+0000 7f3cadffb700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c9c066f30 0x7f3c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.386+0000 7f3cadffb700 1 -- 172.21.15.45:0/2894595474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3cb4083f10 con 0x7f3cb806de20 2023-10-06T05:16:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.386+0000 7f3cbd3e2700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c9c066f30 0x7f3c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.386+0000 7f3cbd3e2700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c9c066f30 0x7f3c9c0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80091e0 tx=0x7f3ca8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.390+0000 7f3cadffb700 1 -- 172.21.15.45:0/2894595474 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb4050e40 con 0x7f3cb806de20 2023-10-06T05:16:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.595+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cb80c9ad0 con 0x7f3cb806de20 2023-10-06T05:16:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.596+0000 7f3cadffb700 1 -- 172.21.15.45:0/2894595474 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17180 (secure 0 0 0) 0x7f3cb4017070 con 0x7f3cb806de20 2023-10-06T05:16:10.598 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:10.599 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:16:09.624228+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:08.040887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:08.594370+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c9c066f30 msgr2=0x7f3c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c9c066f30 0x7f3c9c0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80091e0 tx=0x7f3ca8016000 comp rx=0 tx=0).stop 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb806de20 msgr2=0x7f3cb80c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb806de20 0x7f3cb80c6ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400c7c0 tx=0x7f3cb4007dc0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 shutdown_connections 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c9c066f30 0x7f3c9c0693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cb8160d80 0x7f3cb80c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb806de20 0x7f3cb80c6ac0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 --2- 172.21.15.45:0/2894595474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb81609b0 0x7f3cb8152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 >> 172.21.15.45:0/2894595474 conn(0x7f3cb80ce9b0 msgr2=0x7f3cb80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 shutdown_connections 2023-10-06T05:16:10.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.599+0000 7f3cbfe47700 1 -- 172.21.15.45:0/2894595474 wait complete. 2023-10-06T05:16:10.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.744+0000 7f84aba84700 1 Processor -- start 2023-10-06T05:16:10.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.744+0000 7f84aba84700 1 -- start start 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84aba84700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4153620 0x7f84a415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84aba84700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f84a41539f0 0x7f84a415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84aba84700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a40ce690 con 0x7f84a40cc8b0 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84aba84700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a415a7a0 con 0x7f84a41539f0 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84aba84700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40ce3f0 con 0x7f84a4153620 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84a901f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f84a41539f0 0x7f84a415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84a901f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f84a41539f0 0x7f84a415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33048/0 (socket says 172.21.15.45:33048) 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84a901f700 1 -- 172.21.15.45:0/472399571 learned_addr learned my addr 172.21.15.45:0/472399571 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84a9820700 1 --2- 172.21.15.45:0/472399571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4153620 0x7f84a415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84a901f700 1 -- 172.21.15.45:0/472399571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4153620 msgr2=0x7f84a415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84a901f700 1 --2- 172.21.15.45:0/472399571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4153620 0x7f84a415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.745+0000 7f84a901f700 1 -- 172.21.15.45:0/472399571 >> v1:172.21.15.144:6789/0 conn(0x7f84a40cc8b0 legacy=0x7f84a40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.746+0000 7f84a901f700 1 -- 172.21.15.45:0/472399571 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a415d510 con 0x7f84a41539f0 2023-10-06T05:16:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.746+0000 7f84a901f700 1 --2- 172.21.15.45:0/472399571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a41539f0 0x7f84a415cdc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84980099b0 tx=0x7f8498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac55b4dbac7b8d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.746+0000 7f84a881e700 1 -- 172.21.15.45:0/472399571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800b860 con 0x7f84a41539f0 2023-10-06T05:16:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.746+0000 7f84a881e700 1 -- 172.21.15.45:0/472399571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849800b9c0 con 0x7f84a41539f0 2023-10-06T05:16:10.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.746+0000 7f84a881e700 1 -- 172.21.15.45:0/472399571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800bd60 con 0x7f84a41539f0 2023-10-06T05:16:10.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.746+0000 7f84aba84700 1 -- 172.21.15.45:0/472399571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a41539f0 msgr2=0x7f84a415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.746+0000 7f84aba84700 1 --2- 172.21.15.45:0/472399571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a41539f0 0x7f84a415cdc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84980099b0 tx=0x7f8498009250 comp rx=0 tx=0).stop 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.747+0000 7f84aba84700 1 -- 172.21.15.45:0/472399571 shutdown_connections 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.747+0000 7f84aba84700 1 --2- 172.21.15.45:0/472399571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4153620 0x7f84a415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.747+0000 7f84aba84700 1 --2- 172.21.15.45:0/472399571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a41539f0 0x7f84a415cdc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.747+0000 7f84aba84700 1 -- 172.21.15.45:0/472399571 >> 172.21.15.45:0/472399571 conn(0x7f84a406ed50 msgr2=0x7f84a406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.747+0000 7f84aba84700 1 -- 172.21.15.45:0/472399571 shutdown_connections 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.747+0000 7f84aba84700 1 -- 172.21.15.45:0/472399571 wait complete. 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.748+0000 7f84aba84700 1 Processor -- start 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.748+0000 7f84aba84700 1 -- start start 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aba84700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cc8b0 0x7f84a415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aba84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a4153620 0x7f84a40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aba84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a4151380 0x7f84a40c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aba84700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40b4530 con 0x7f84a4151380 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aba84700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40c7330 con 0x7f84a4153620 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aba84700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4151820 con 0x7f84a40cc8b0 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a4151380 0x7f84a40c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a4151380 0x7f84a40c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33056/0 (socket says 172.21.15.45:33056) 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 -- 172.21.15.45:0/1010154414 learned_addr learned my addr 172.21.15.45:0/1010154414 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84a901f700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a4153620 0x7f84a40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84a9820700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cc8b0 0x7f84a415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 -- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cc8b0 msgr2=0x7f84a415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cc8b0 0x7f84a415b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 -- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a4153620 msgr2=0x7f84a40b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a4153620 0x7f84a40b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 -- 172.21.15.45:0/1010154414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980094f0 con 0x7f84a4151380 2023-10-06T05:16:10.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aa021700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a4151380 0x7f84a40c8d00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c8e0 tx=0x7f849c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f8495ffb700 1 -- 172.21.15.45:0/1010154414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c012630 con 0x7f84a4151380 2023-10-06T05:16:10.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f8495ffb700 1 -- 172.21.15.45:0/1010154414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849c012790 con 0x7f84a4151380 2023-10-06T05:16:10.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a40c9550 con 0x7f84a4151380 2023-10-06T05:16:10.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.749+0000 7f8495ffb700 1 -- 172.21.15.45:0/1010154414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c012b50 con 0x7f84a4151380 2023-10-06T05:16:10.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.750+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a40c9b10 con 0x7f84a4151380 2023-10-06T05:16:10.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.751+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a40c74c0 con 0x7f84a4151380 2023-10-06T05:16:10.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.751+0000 7f8495ffb700 1 -- 172.21.15.45:0/1010154414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f849c01a3a0 con 0x7f84a4151380 2023-10-06T05:16:10.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.752+0000 7f8495ffb700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8488066f30 0x7f84880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:10.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.752+0000 7f8495ffb700 1 -- 172.21.15.45:0/1010154414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f849c0849b0 con 0x7f84a4151380 2023-10-06T05:16:10.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.752+0000 7f84a9820700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8488066f30 0x7f84880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:10.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.752+0000 7f84a9820700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8488066f30 0x7f84880693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84a0006fd0 tx=0x7f84a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:10.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.756+0000 7f8495ffb700 1 -- 172.21.15.45:0/1010154414 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f849c0518e0 con 0x7f84a4151380 2023-10-06T05:16:10.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.958+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84a404bb40 con 0x7f84a4151380 2023-10-06T05:16:10.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.959+0000 7f8495ffb700 1 -- 172.21.15.45:0/1010154414 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17180 (secure 0 0 0) 0x7f849c054f00 con 0x7f84a4151380 2023-10-06T05:16:10.961 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:10.961 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:16:09.624228+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:08.040887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:08.594370+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:10.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8488066f30 msgr2=0x7f84880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8488066f30 0x7f84880693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84a0006fd0 tx=0x7f84a0008040 comp rx=0 tx=0).stop 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a4151380 msgr2=0x7f84a40c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a4151380 0x7f84a40c8d00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c8e0 tx=0x7f849c001710 comp rx=0 tx=0).stop 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 shutdown_connections 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8488066f30 0x7f84880693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84a4151380 0x7f84a40c8d00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cc8b0 0x7f84a415b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 --2- 172.21.15.45:0/1010154414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a4153620 0x7f84a40b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 >> 172.21.15.45:0/1010154414 conn(0x7f84a406ed50 msgr2=0x7f84a4068b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.961+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 shutdown_connections 2023-10-06T05:16:10.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:10.962+0000 7f84aba84700 1 -- 172.21.15.45:0/1010154414 wait complete. 2023-10-06T05:16:10.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:16:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.110+0000 7f0956427700 1 Processor -- start 2023-10-06T05:16:11.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.110+0000 7f0956427700 1 -- start start 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f0956427700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0950153840 0x7f095015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f0956427700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f095015cd70 0x7f09500ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f0956427700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09500ce770 con 0x7f09500cd060 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f0956427700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0950153c10 con 0x7f095015cd70 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f0956427700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09500cd450 con 0x7f0950153840 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f095015cd70 0x7f09500ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f095015cd70 0x7f09500ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33068/0 (socket says 172.21.15.45:33068) 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094f7fe700 1 -- 172.21.15.45:0/3540255876 learned_addr learned my addr 172.21.15.45:0/3540255876 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094ffff700 1 --2- 172.21.15.45:0/3540255876 >> v2:172.21.15.144:3300/0 conn(0x7f0950153840 0x7f095015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094f7fe700 1 -- 172.21.15.45:0/3540255876 >> v2:172.21.15.144:3300/0 conn(0x7f0950153840 msgr2=0x7f095015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094f7fe700 1 --2- 172.21.15.45:0/3540255876 >> v2:172.21.15.144:3300/0 conn(0x7f0950153840 0x7f095015a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094f7fe700 1 -- 172.21.15.45:0/3540255876 >> v1:172.21.15.144:6789/0 conn(0x7f09500cd060 legacy=0x7f0950068bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.111+0000 7f094f7fe700 1 -- 172.21.15.45:0/3540255876 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095015d650 con 0x7f095015cd70 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f094f7fe700 1 --2- 172.21.15.45:0/3540255876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f095015cd70 0x7f09500ccb20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09500caf40 tx=0x7f0944009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87c5b5476b16e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f094effd700 1 -- 172.21.15.45:0/3540255876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0944009e70 con 0x7f095015cd70 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f094effd700 1 -- 172.21.15.45:0/3540255876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094400b8f0 con 0x7f095015cd70 2023-10-06T05:16:11.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f094effd700 1 -- 172.21.15.45:0/3540255876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094400bc90 con 0x7f095015cd70 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f0956427700 1 -- 172.21.15.45:0/3540255876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f095015cd70 msgr2=0x7f09500ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f0956427700 1 --2- 172.21.15.45:0/3540255876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f095015cd70 0x7f09500ccb20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09500caf40 tx=0x7f0944009250 comp rx=0 tx=0).stop 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f0956427700 1 -- 172.21.15.45:0/3540255876 shutdown_connections 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f0956427700 1 --2- 172.21.15.45:0/3540255876 >> v2:172.21.15.144:3300/0 conn(0x7f0950153840 0x7f095015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f0956427700 1 --2- 172.21.15.45:0/3540255876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f095015cd70 0x7f09500ccb20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.112+0000 7f0956427700 1 -- 172.21.15.45:0/3540255876 >> 172.21.15.45:0/3540255876 conn(0x7f095006ed50 msgr2=0x7f095006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.113+0000 7f0956427700 1 -- 172.21.15.45:0/3540255876 shutdown_connections 2023-10-06T05:16:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.113+0000 7f0956427700 1 -- 172.21.15.45:0/3540255876 wait complete. 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.114+0000 7f0956427700 1 Processor -- start 2023-10-06T05:16:11.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.114+0000 7f0956427700 1 -- start start 2023-10-06T05:16:11.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f0956427700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09500cd060 0x7f09500c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f0956427700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0950153840 0x7f0950146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09500cd060 0x7f09500c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09500cd060 0x7f09500c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33070/0 (socket says 172.21.15.45:33070) 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f0956427700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f095015cd70 0x7f095016a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 -- 172.21.15.45:0/1751097504 learned_addr learned my addr 172.21.15.45:0/1751097504 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c9180 con 0x7f09500cd060 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c9610 con 0x7f095015cd70 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c8570 con 0x7f0950153840 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094f7fe700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0950153840 0x7f0950146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f09549c4700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f095015cd70 0x7f095016a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 -- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0950153840 msgr2=0x7f0950146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0950153840 0x7f0950146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 -- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f095015cd70 msgr2=0x7f095016a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f095015cd70 0x7f095016a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.115+0000 7f094ffff700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440094f0 con 0x7f09500cd060 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.116+0000 7f094ffff700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09500cd060 0x7f09500c7bb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f094000ecb0 tx=0x7f09400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.116+0000 7f0937fff700 1 -- 172.21.15.45:0/1751097504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000c920 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.116+0000 7f0937fff700 1 -- 172.21.15.45:0/1751097504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000ca80 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.116+0000 7f0937fff700 1 -- 172.21.15.45:0/1751097504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000ce20 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.116+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09500c8850 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.116+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09500c8e10 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.117+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950088a60 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.117+0000 7f0937fff700 1 -- 172.21.15.45:0/1751097504 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0940013680 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.118+0000 7f0937fff700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f093c066f30 0x7f093c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.118+0000 7f0937fff700 1 -- 172.21.15.45:0/1751097504 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0940084aa0 con 0x7f09500cd060 2023-10-06T05:16:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.118+0000 7f094f7fe700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f093c066f30 0x7f093c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:11.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.118+0000 7f094f7fe700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f093c066f30 0x7f093c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09500caf40 tx=0x7f0944017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:11.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.122+0000 7f0937fff700 1 -- 172.21.15.45:0/1751097504 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0940051a70 con 0x7f09500cd060 2023-10-06T05:16:11.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:11.373+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f095015d930 con 0x7f09500cd060 2023-10-06T05:16:12.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.214+0000 7f0937fff700 1 -- 172.21.15.45:0/1751097504 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0940055090 con 0x7f09500cd060 2023-10-06T05:16:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f093c066f30 msgr2=0x7f093c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f093c066f30 0x7f093c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09500caf40 tx=0x7f0944017000 comp rx=0 tx=0).stop 2023-10-06T05:16:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09500cd060 msgr2=0x7f09500c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09500cd060 0x7f09500c7bb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f094000ecb0 tx=0x7f09400091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 shutdown_connections 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f093c066f30 0x7f093c0693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f095015cd70 0x7f095016a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09500cd060 0x7f09500c7bb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 --2- 172.21.15.45:0/1751097504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0950153840 0x7f0950146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 >> 172.21.15.45:0/1751097504 conn(0x7f095006ed50 msgr2=0x7f09500cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 shutdown_connections 2023-10-06T05:16:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.217+0000 7f0956427700 1 -- 172.21.15.45:0/1751097504 wait complete. 2023-10-06T05:16:12.713 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:16:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.855+0000 7f80a6111700 1 Processor -- start 2023-10-06T05:16:12.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.855+0000 7f80a6111700 1 -- start start 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f80a6111700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f80a006aa70 0x7f80a00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f80a6111700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a015c8c0 0x7f80a00aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f80a6111700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a0067d80 con 0x7f80a00c7050 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f80a6111700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a00ac620 con 0x7f80a006aa70 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f80a6111700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a00c75b0 con 0x7f80a015c8c0 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f809ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a015c8c0 0x7f80a00aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f809f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f80a006aa70 0x7f80a00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f809effd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f80a00c7050 0x7f80a015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:48230/0 (socket says 172.21.15.45:48230) 2023-10-06T05:16:13.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f809effd700 1 -- 172.21.15.45:0/2736592243 learned_addr learned my addr 172.21.15.45:0/2736592243 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f809f7fe700 1 -- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a015c8c0 msgr2=0x7f80a00aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f809f7fe700 1 --2- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a015c8c0 0x7f80a00aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.856+0000 7f809f7fe700 1 -- 172.21.15.45:0/2736592243 >> v1:172.21.15.144:6789/0 conn(0x7f80a00c7050 legacy=0x7f80a015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f809f7fe700 1 -- 172.21.15.45:0/2736592243 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00c7890 con 0x7f80a006aa70 2023-10-06T05:16:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f809ffff700 1 --2- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a015c8c0 0x7f80a00aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f809f7fe700 1 --2- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a006aa70 0x7f80a00c6b10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f80900098e0 tx=0x7f80900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcc05502fb48898 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f809e7fc700 1 -- 172.21.15.45:0/2736592243 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000b640 con 0x7f80a006aa70 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f809e7fc700 1 -- 172.21.15.45:0/2736592243 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809000b7a0 con 0x7f80a006aa70 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f809e7fc700 1 -- 172.21.15.45:0/2736592243 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000bb40 con 0x7f80a006aa70 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f80a6111700 1 -- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a006aa70 msgr2=0x7f80a00c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f80a6111700 1 --2- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a006aa70 0x7f80a00c6b10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f80900098e0 tx=0x7f80900091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f80a6111700 1 -- 172.21.15.45:0/2736592243 shutdown_connections 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f80a6111700 1 --2- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a015c8c0 0x7f80a00aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f80a6111700 1 --2- 172.21.15.45:0/2736592243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a006aa70 0x7f80a00c6b10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.857+0000 7f80a6111700 1 -- 172.21.15.45:0/2736592243 >> 172.21.15.45:0/2736592243 conn(0x7f80a00cc570 msgr2=0x7f80a00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.858+0000 7f80a6111700 1 -- 172.21.15.45:0/2736592243 shutdown_connections 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.858+0000 7f80a6111700 1 -- 172.21.15.45:0/2736592243 wait complete. 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.859+0000 7f80a6111700 1 Processor -- start 2023-10-06T05:16:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.859+0000 7f80a6111700 1 -- start start 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f80a6111700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a006aa70 0x7f80a014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f80a6111700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a00c7050 0x7f80a00c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f80a6111700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015c8c0 0x7f80a015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f80a6111700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a015b040 con 0x7f80a006aa70 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f80a6111700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a015a460 con 0x7f80a015c8c0 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f80a6111700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a015a710 con 0x7f80a00c7050 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a006aa70 0x7f80a014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a006aa70 0x7f80a014e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33084/0 (socket says 172.21.15.45:33084) 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 -- 172.21.15.45:0/2002293239 learned_addr learned my addr 172.21.15.45:0/2002293239 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809effd700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a00c7050 0x7f80a00c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809ffff700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015c8c0 0x7f80a015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 -- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a00c7050 msgr2=0x7f80a00c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a00c7050 0x7f80a00c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 -- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015c8c0 msgr2=0x7f80a015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015c8c0 0x7f80a015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 -- 172.21.15.45:0/2002293239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8098009a20 con 0x7f80a006aa70 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809effd700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a00c7050 0x7f80a00c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.860+0000 7f809f7fe700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a006aa70 0x7f80a014e9a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f80a006fbf0 tx=0x7f8090017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.861+0000 7f80877fe700 1 -- 172.21.15.45:0/2002293239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809001bda0 con 0x7f80a006aa70 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.861+0000 7f80877fe700 1 -- 172.21.15.45:0/2002293239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8090011420 con 0x7f80a006aa70 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.861+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8090009420 con 0x7f80a006aa70 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.861+0000 7f80877fe700 1 -- 172.21.15.45:0/2002293239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80900117e0 con 0x7f80a006aa70 2023-10-06T05:16:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.861+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a015acd0 con 0x7f80a006aa70 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.862+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a015ae60 con 0x7f80a006aa70 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.862+0000 7f80877fe700 1 -- 172.21.15.45:0/2002293239 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8090011b90 con 0x7f80a006aa70 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.863+0000 7f80877fe700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f808c066f30 0x7f808c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.863+0000 7f80877fe700 1 -- 172.21.15.45:0/2002293239 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8090085f60 con 0x7f80a006aa70 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.863+0000 7f809effd700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f808c066f30 0x7f808c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.864+0000 7f809effd700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f808c066f30 0x7f808c0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f80a00acfe0 tx=0x7f8094009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:12.867+0000 7f80877fe700 1 -- 172.21.15.45:0/2002293239 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8090052f30 con 0x7f80a006aa70 2023-10-06T05:16:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.121+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f80a00c7b70 con 0x7f80a006aa70 2023-10-06T05:16:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.220+0000 7f80877fe700 1 -- 172.21.15.45:0/2002293239 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8090056550 con 0x7f80a006aa70 2023-10-06T05:16:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f808c066f30 msgr2=0x7f808c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f808c066f30 0x7f808c0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f80a00acfe0 tx=0x7f8094009040 comp rx=0 tx=0).stop 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a006aa70 msgr2=0x7f80a014e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a006aa70 0x7f80a014e9a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f80a006fbf0 tx=0x7f8090017000 comp rx=0 tx=0).stop 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 shutdown_connections 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f808c066f30 0x7f808c0693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015c8c0 0x7f80a015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a006aa70 0x7f80a014e9a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 --2- 172.21.15.45:0/2002293239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a00c7050 0x7f80a00c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 >> 172.21.15.45:0/2002293239 conn(0x7f80a00cc570 msgr2=0x7f80a015f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 shutdown_connections 2023-10-06T05:16:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.222+0000 7f80a6111700 1 -- 172.21.15.45:0/2002293239 wait complete. 2023-10-06T05:16:13.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.370+0000 7f1b4b0b9700 1 Processor -- start 2023-10-06T05:16:13.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.370+0000 7f1b4b0b9700 1 -- start start 2023-10-06T05:16:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.371+0000 7f1b4b0b9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1b440cd340 0x7f1b44157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.371+0000 7f1b4b0b9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4415b000 con 0x7f1b440c8ff0 2023-10-06T05:16:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.371+0000 7f1b4b0b9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4415b2e0 con 0x7f1b440c93c0 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.371+0000 7f1b4b0b9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b440c9830 con 0x7f1b440cd340 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b48e55700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1b440c8ff0 0x7f1b440caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52304/0 (socket says 172.21.15.45:52304) 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b48e55700 1 -- 172.21.15.45:0/3096157400 learned_addr learned my addr 172.21.15.45:0/3096157400 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b49656700 1 --2- 172.21.15.45:0/3096157400 >> v2:172.21.15.144:3300/0 conn(0x7f1b440cd340 0x7f1b44157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26731865 0 0) 0x7f1b4415b000 con 0x7f1b440c8ff0 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b34003570 con 0x7f1b440c8ff0 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105259716 0 0) 0x7f1b4415b2e0 con 0x7f1b440c93c0 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b4415b000 con 0x7f1b440c93c0 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481946940 0 0) 0x7f1b34003570 con 0x7f1b440c8ff0 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b49656700 1 -- 172.21.15.45:0/3096157400 >> v1:172.21.15.144:6789/0 conn(0x7f1b440c93c0 legacy=0x7f1b44156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b49656700 1 -- 172.21.15.45:0/3096157400 >> v1:172.21.15.45:6789/0 conn(0x7f1b440c8ff0 legacy=0x7f1b440caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b49656700 1 -- 172.21.15.45:0/3096157400 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4415b5c0 con 0x7f1b440cd340 2023-10-06T05:16:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.372+0000 7f1b49656700 1 --2- 172.21.15.45:0/3096157400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b44157200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009e60 tx=0x7f1b3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563b35edb9b7bfd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00e070 con 0x7f1b440cd340 2023-10-06T05:16:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00b7a0 con 0x7f1b440cd340 2023-10-06T05:16:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b437fe700 1 -- 172.21.15.45:0/3096157400 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38001470 con 0x7f1b440cd340 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3096157400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 msgr2=0x7f1b44157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3096157400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b44157200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009e60 tx=0x7f1b3c009780 comp rx=0 tx=0).stop 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3096157400 shutdown_connections 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3096157400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b44157200 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3096157400 >> 172.21.15.45:0/3096157400 conn(0x7f1b440cdf50 msgr2=0x7f1b440ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3096157400 shutdown_connections 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.373+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3096157400 wait complete. 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.374+0000 7f1b4b0b9700 1 Processor -- start 2023-10-06T05:16:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.375+0000 7f1b4b0b9700 1 -- start start 2023-10-06T05:16:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.375+0000 7f1b4b0b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b440c8ff0 0x7f1b440c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.375+0000 7f1b4b0b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b440c93c0 0x7f1b44145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.375+0000 7f1b4b0b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b440adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.375+0000 7f1b4b0b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440b1ba0 con 0x7f1b440c93c0 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.375+0000 7f1b4b0b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440b0cf0 con 0x7f1b440cd340 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.375+0000 7f1b4b0b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440b0f90 con 0x7f1b440c8ff0 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b440c93c0 0x7f1b44145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b440c93c0 0x7f1b44145b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33086/0 (socket says 172.21.15.45:33086) 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 -- 172.21.15.45:0/3134159049 learned_addr learned my addr 172.21.15.45:0/3134159049 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b48e55700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b440c8ff0 0x7f1b440c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b49656700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b440adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 -- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b440c8ff0 msgr2=0x7f1b440c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b440c8ff0 0x7f1b440c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 -- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 msgr2=0x7f1b440adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b440adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 -- 172.21.15.45:0/3134159049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c009a20 con 0x7f1b440c93c0 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b48e55700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b440c8ff0 0x7f1b440c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b49656700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b440adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b43fff700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b440c93c0 0x7f1b44145b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c7c0 tx=0x7f1b30007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b40ff9700 1 -- 172.21.15.45:0/3134159049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30016070 con 0x7f1b440c93c0 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b40ff9700 1 -- 172.21.15.45:0/3134159049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b30011680 con 0x7f1b440c93c0 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b440b1270 con 0x7f1b440c93c0 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b40ff9700 1 -- 172.21.15.45:0/3134159049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30011a20 con 0x7f1b440c93c0 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.376+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b440b1830 con 0x7f1b440c93c0 2023-10-06T05:16:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.377+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b440b1d10 con 0x7f1b440c93c0 2023-10-06T05:16:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.378+0000 7f1b40ff9700 1 -- 172.21.15.45:0/3134159049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b30012440 con 0x7f1b440c93c0 2023-10-06T05:16:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.378+0000 7f1b40ff9700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b28066f30 0x7f1b280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.378+0000 7f1b40ff9700 1 -- 172.21.15.45:0/3134159049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1b30084620 con 0x7f1b440c93c0 2023-10-06T05:16:13.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.379+0000 7f1b48e55700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b28066f30 0x7f1b280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.379+0000 7f1b48e55700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b28066f30 0x7f1b280693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1b380011e0 tx=0x7f1b38000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.383+0000 7f1b40ff9700 1 -- 172.21.15.45:0/3134159049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b300515f0 con 0x7f1b440c93c0 2023-10-06T05:16:13.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.635+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b440add80 con 0x7f1b440c93c0 2023-10-06T05:16:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.636+0000 7f1b40ff9700 1 -- 172.21.15.45:0/3134159049 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f1b30054c10 con 0x7f1b440c93c0 2023-10-06T05:16:13.638 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:13.638 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:13.214114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6834/2918652845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2918652845},{"type":"v1","addr":"172.21.15.45:6834","nonce":2918652845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:16:13.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b28066f30 msgr2=0x7f1b280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b28066f30 0x7f1b280693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1b380011e0 tx=0x7f1b38000bc0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b440c93c0 msgr2=0x7f1b44145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b440c93c0 0x7f1b44145b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c7c0 tx=0x7f1b30007dd0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 shutdown_connections 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b28066f30 0x7f1b280693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b440cd340 0x7f1b440adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b440c8ff0 0x7f1b440c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 --2- 172.21.15.45:0/3134159049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b440c93c0 0x7f1b44145b90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.638+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 >> 172.21.15.45:0/3134159049 conn(0x7f1b440cdf50 msgr2=0x7f1b440c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.639+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 shutdown_connections 2023-10-06T05:16:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.639+0000 7f1b4b0b9700 1 -- 172.21.15.45:0/3134159049 wait complete. 2023-10-06T05:16:13.642 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 16 2023-10-06T05:16:13.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.784+0000 7f74a3ac4700 1 Processor -- start 2023-10-06T05:16:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.784+0000 7f74a3ac4700 1 -- start start 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a3ac4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c0cd3f0 0x7f749c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a3ac4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c15b380 con 0x7f749c1570b0 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a3ac4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c15b660 con 0x7f749c156ce0 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a3ac4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c06c010 con 0x7f749c0cd3f0 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a105f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f749c1570b0 0x7f749c0ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52308/0 (socket says 172.21.15.45:52308) 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a105f700 1 -- 172.21.15.45:0/508648021 learned_addr learned my addr 172.21.15.45:0/508648021 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a2061700 1 --2- 172.21.15.45:0/508648021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c0cd3f0 0x7f749c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a085e700 1 -- 172.21.15.45:0/508648021 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870256117 0 0) 0x7f749c15b380 con 0x7f749c1570b0 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.785+0000 7f74a085e700 1 -- 172.21.15.45:0/508648021 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7488003570 con 0x7f749c1570b0 2023-10-06T05:16:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a085e700 1 -- 172.21.15.45:0/508648021 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107913973 0 0) 0x7f749c15b660 con 0x7f749c156ce0 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a085e700 1 -- 172.21.15.45:0/508648021 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f749c15b380 con 0x7f749c156ce0 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a2061700 1 -- 172.21.15.45:0/508648021 >> v1:172.21.15.144:6789/0 conn(0x7f749c156ce0 legacy=0x7f749c0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a2061700 1 -- 172.21.15.45:0/508648021 >> v1:172.21.15.45:6789/0 conn(0x7f749c1570b0 legacy=0x7f749c0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a2061700 1 -- 172.21.15.45:0/508648021 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c15b940 con 0x7f749c0cd3f0 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a2061700 1 --2- 172.21.15.45:0/508648021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c0cd3f0 0x7f749c06bad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7494009e90 tx=0x7f7494009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e262bfa7f8a218ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a085e700 1 -- 172.21.15.45:0/508648021 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400e070 con 0x7f749c0cd3f0 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a085e700 1 -- 172.21.15.45:0/508648021 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749400bb80 con 0x7f749c0cd3f0 2023-10-06T05:16:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.786+0000 7f74a085e700 1 -- 172.21.15.45:0/508648021 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494012430 con 0x7f749c0cd3f0 2023-10-06T05:16:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.787+0000 7f74a3ac4700 1 -- 172.21.15.45:0/508648021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c0cd3f0 msgr2=0x7f749c06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.787+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/508648021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c0cd3f0 0x7f749c06bad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7494009e90 tx=0x7f7494009780 comp rx=0 tx=0).stop 2023-10-06T05:16:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.787+0000 7f74a3ac4700 1 -- 172.21.15.45:0/508648021 shutdown_connections 2023-10-06T05:16:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.787+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/508648021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c0cd3f0 0x7f749c06bad0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.787+0000 7f74a3ac4700 1 -- 172.21.15.45:0/508648021 >> 172.21.15.45:0/508648021 conn(0x7f749c06f090 msgr2=0x7f749c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.787+0000 7f74a3ac4700 1 -- 172.21.15.45:0/508648021 shutdown_connections 2023-10-06T05:16:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.787+0000 7f74a3ac4700 1 -- 172.21.15.45:0/508648021 wait complete. 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.788+0000 7f74a3ac4700 1 Processor -- start 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.788+0000 7f74a3ac4700 1 -- start start 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a3ac4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749c0cd3f0 0x7f749c0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a3ac4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c156ce0 0x7f749c0c0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a3ac4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c1570b0 0x7f749c0c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a3ac4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0c0990 con 0x7f749c156ce0 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a3ac4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0c8320 con 0x7f749c0cd3f0 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a3ac4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0c4380 con 0x7f749c1570b0 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c156ce0 0x7f749c0c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c156ce0 0x7f749c0c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33088/0 (socket says 172.21.15.45:33088) 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 -- 172.21.15.45:0/1078753562 learned_addr learned my addr 172.21.15.45:0/1078753562 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a2061700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c1570b0 0x7f749c0c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a1860700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749c0cd3f0 0x7f749c0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 -- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c1570b0 msgr2=0x7f749c0c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c1570b0 0x7f749c0c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 -- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749c0cd3f0 msgr2=0x7f749c0c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749c0cd3f0 0x7f749c0c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 -- 172.21.15.45:0/1078753562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494009a20 con 0x7f749c156ce0 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a2061700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c1570b0 0x7f749c0c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a1860700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749c0cd3f0 0x7f749c0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.789+0000 7f74a105f700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c156ce0 0x7f749c0c0450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c790 tx=0x7f748c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.790+0000 7f7491ffb700 1 -- 172.21.15.45:0/1078753562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c005b30 con 0x7f749c156ce0 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.790+0000 7f7491ffb700 1 -- 172.21.15.45:0/1078753562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c005c90 con 0x7f749c156ce0 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.790+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c0c7fd0 con 0x7f749c156ce0 2023-10-06T05:16:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.790+0000 7f7491ffb700 1 -- 172.21.15.45:0/1078753562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0072e0 con 0x7f749c156ce0 2023-10-06T05:16:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.790+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c0c7870 con 0x7f749c156ce0 2023-10-06T05:16:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.791+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c0c8490 con 0x7f749c156ce0 2023-10-06T05:16:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.791+0000 7f7491ffb700 1 -- 172.21.15.45:0/1078753562 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f748c007690 con 0x7f749c156ce0 2023-10-06T05:16:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.792+0000 7f7491ffb700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.792+0000 7f7491ffb700 1 -- 172.21.15.45:0/1078753562 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f748c083cd0 con 0x7f749c156ce0 2023-10-06T05:16:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.792+0000 7f74a1860700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.793+0000 7f74a1860700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7480066f30 0x7f74800693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7498007fd0 tx=0x7f7498009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:13.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:13.796+0000 7f7491ffb700 1 -- 172.21.15.45:0/1078753562 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f748c04b5a0 con 0x7f749c156ce0 2023-10-06T05:16:13.962 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:16:13.960+0000 7f4f0ee13700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:16:14.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.003+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f749c0c7a60 con 0x7f749c156ce0 2023-10-06T05:16:14.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.004+0000 7f7491ffb700 1 -- 172.21.15.45:0/1078753562 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17167 (secure 0 0 0) 0x7f748c017020 con 0x7f749c156ce0 2023-10-06T05:16:14.006 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:14.006 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":38,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:16:11.201269+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:08.040887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:08.594370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7480066f30 msgr2=0x7f74800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7480066f30 0x7f74800693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7498007fd0 tx=0x7f7498009040 comp rx=0 tx=0).stop 2023-10-06T05:16:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c156ce0 msgr2=0x7f749c0c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c156ce0 0x7f749c0c0450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c790 tx=0x7f748c007ea0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 shutdown_connections 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749c1570b0 0x7f749c0c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749c0cd3f0 0x7f749c0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 --2- 172.21.15.45:0/1078753562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c156ce0 0x7f749c0c0450 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.006+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 >> 172.21.15.45:0/1078753562 conn(0x7f749c06f090 msgr2=0x7f749c070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.007+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 shutdown_connections 2023-10-06T05:16:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.007+0000 7f74a3ac4700 1 -- 172.21.15.45:0/1078753562 wait complete. 2023-10-06T05:16:14.022 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:16:14.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:16:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.153+0000 7f06df979700 1 Processor -- start 2023-10-06T05:16:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.153+0000 7f06df979700 1 -- start start 2023-10-06T05:16:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06df979700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06df979700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80ae8d0 con 0x7f06d81560a0 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06df979700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80aebb0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06df979700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80ae6b0 con 0x7f06d8155cd0 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06dcf14700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06d81560a0 0x7f06d80adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52314/0 (socket says 172.21.15.45:52314) 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06dcf14700 1 -- 172.21.15.45:0/3762461381 learned_addr learned my addr 172.21.15.45:0/3762461381 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06dd715700 1 --2- 172.21.15.45:0/3762461381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079932377 0 0) 0x7f06d80ae8d0 con 0x7f06d81560a0 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.154+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06c4003570 con 0x7f06d81560a0 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233244208 0 0) 0x7f06d80aebb0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06d80ae8d0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06dd715700 1 -- 172.21.15.45:0/3762461381 >> v1:172.21.15.144:6789/0 conn(0x7f06d80ccdc0 legacy=0x7f06d80c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06dd715700 1 -- 172.21.15.45:0/3762461381 >> v1:172.21.15.45:6789/0 conn(0x7f06d81560a0 legacy=0x7f06d80adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 823182852 0 0) 0x7f06c4003570 con 0x7f06d81560a0 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06dd715700 1 -- 172.21.15.45:0/3762461381 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80aee90 con 0x7f06d8155cd0 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06dd715700 1 --2- 172.21.15.45:0/3762461381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d815a730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009910 tx=0x7f06d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7c513fe38712a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8001470 con 0x7f06d8155cd0 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4011420 con 0x7f06d8155cd0 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.155+0000 7f06cffff700 1 -- 172.21.15.45:0/3762461381 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d40117c0 con 0x7f06d8155cd0 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.156+0000 7f06df979700 1 -- 172.21.15.45:0/3762461381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 msgr2=0x7f06d815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.156+0000 7f06df979700 1 --2- 172.21.15.45:0/3762461381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d815a730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009910 tx=0x7f06d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.156+0000 7f06df979700 1 -- 172.21.15.45:0/3762461381 shutdown_connections 2023-10-06T05:16:14.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.156+0000 7f06df979700 1 --2- 172.21.15.45:0/3762461381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d815a730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.156+0000 7f06df979700 1 -- 172.21.15.45:0/3762461381 >> 172.21.15.45:0/3762461381 conn(0x7f06d806f090 msgr2=0x7f06d806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:14.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.156+0000 7f06df979700 1 -- 172.21.15.45:0/3762461381 shutdown_connections 2023-10-06T05:16:14.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.156+0000 7f06df979700 1 -- 172.21.15.45:0/3762461381 wait complete. 2023-10-06T05:16:14.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.157+0000 7f06df979700 1 Processor -- start 2023-10-06T05:16:14.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.157+0000 7f06df979700 1 -- start start 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06df979700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ccdc0 0x7f06d81565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06df979700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d80abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06df979700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d81560a0 0x7f06d8153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06df979700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8156af0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06df979700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80c4e30 con 0x7f06d81560a0 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06df979700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80ac2b0 con 0x7f06d8155cd0 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ccdc0 0x7f06d81565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ccdc0 0x7f06d81565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33090/0 (socket says 172.21.15.45:33090) 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 -- 172.21.15.45:0/2034074002 learned_addr learned my addr 172.21.15.45:0/2034074002 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:14.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dcf14700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d80abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06ddf16700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d81560a0 0x7f06d8153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 -- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 msgr2=0x7f06d80abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d80abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 -- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d81560a0 msgr2=0x7f06d8153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d81560a0 0x7f06d8153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.158+0000 7f06dd715700 1 -- 172.21.15.45:0/2034074002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4009420 con 0x7f06d80ccdc0 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.159+0000 7f06dd715700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ccdc0 0x7f06d81565b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f06d40091e0 tx=0x7f06d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.159+0000 7f06cd7fa700 1 -- 172.21.15.45:0/2034074002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401cc90 con 0x7f06d80ccdc0 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.159+0000 7f06cd7fa700 1 -- 172.21.15.45:0/2034074002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d401cdf0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.159+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d80c4220 con 0x7f06d80ccdc0 2023-10-06T05:16:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.159+0000 7f06cd7fa700 1 -- 172.21.15.45:0/2034074002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401a6a0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.159+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d80c47e0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.160+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d806e2f0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.160+0000 7f06cd7fa700 1 -- 172.21.15.45:0/2034074002 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06d401aad0 con 0x7f06d80ccdc0 2023-10-06T05:16:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.161+0000 7f06cd7fa700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06bc066f30 0x7f06bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.161+0000 7f06cd7fa700 1 -- 172.21.15.45:0/2034074002 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f06d4086730 con 0x7f06d80ccdc0 2023-10-06T05:16:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.161+0000 7f06dcf14700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06bc066f30 0x7f06bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.161+0000 7f06dcf14700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06bc066f30 0x7f06bc0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f06d80ccc60 tx=0x7f06c80015c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.165+0000 7f06cd7fa700 1 -- 172.21.15.45:0/2034074002 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06d4053700 con 0x7f06d80ccdc0 2023-10-06T05:16:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.366+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06d80c4f70 con 0x7f06bc066f30 2023-10-06T05:16:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.375+0000 7f06cd7fa700 1 -- 172.21.15.45:0/2034074002 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307435 (secure 0 0 0) 0x7f06c80048f0 con 0x7f06bc066f30 2023-10-06T05:16:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:14.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06bc066f30 msgr2=0x7f06bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06bc066f30 0x7f06bc0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f06d80ccc60 tx=0x7f06c80015c0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ccdc0 msgr2=0x7f06d81565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ccdc0 0x7f06d81565b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f06d40091e0 tx=0x7f06d4016000 comp rx=0 tx=0).stop 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 shutdown_connections 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06bc066f30 0x7f06bc0693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d81560a0 0x7f06d8153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d80ccdc0 0x7f06d81565b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 --2- 172.21.15.45:0/2034074002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d8155cd0 0x7f06d80abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 >> 172.21.15.45:0/2034074002 conn(0x7f06d806f090 msgr2=0x7f06d80ac740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 shutdown_connections 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.379+0000 7f06df979700 1 -- 172.21.15.45:0/2034074002 wait complete. 2023-10-06T05:16:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:16:14.394 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-10-06T05:16:12.458692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591996,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":188,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382812,"kb_used_data":4508,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591908,"statfs":{"total":773094113280,"available":770654113792,"internally_reserved":0,"allocated":4616192,"data_stored":2689567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1628,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.274211"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214591+0000","last_change":"2023-10-06T05:16:10.205723+0000","last_active":"2023-10-06T05:16:10.214591+0000","last_peered":"2023-10-06T05:16:10.214591+0000","last_clean":"2023-10-06T05:16:10.214591+0000","last_became_active":"2023-10-06T05:16:10.205427+0000","last_became_peered":"2023-10-06T05:16:10.205427+0000","last_unstale":"2023-10-06T05:16:10.214591+0000","last_undegraded":"2023-10-06T05:16:10.214591+0000","last_fullsized":"2023-10-06T05:16:10.214591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:08.939753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.212724+0000","last_change":"2023-10-06T05:16:10.210714+0000","last_active":"2023-10-06T05:16:10.212724+0000","last_peered":"2023-10-06T05:16:10.212724+0000","last_clean":"2023-10-06T05:16:10.212724+0000","last_became_active":"2023-10-06T05:16:10.210422+0000","last_became_peered":"2023-10-06T05:16:10.210422+0000","last_unstale":"2023-10-06T05:16:10.212724+0000","last_undegraded":"2023-10-06T05:16:10.212724+0000","last_fullsized":"2023-10-06T05:16:10.212724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:23.294418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218652+0000","last_change":"2023-10-06T05:16:10.211103+0000","last_active":"2023-10-06T05:16:10.218652+0000","last_peered":"2023-10-06T05:16:10.218652+0000","last_clean":"2023-10-06T05:16:10.218652+0000","last_became_active":"2023-10-06T05:16:10.210541+0000","last_became_peered":"2023-10-06T05:16:10.210541+0000","last_unstale":"2023-10-06T05:16:10.218652+0000","last_undegraded":"2023-10-06T05:16:10.218652+0000","last_fullsized":"2023-10-06T05:16:10.218652+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:26.802163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217818+0000","last_change":"2023-10-06T05:16:10.210944+0000","last_active":"2023-10-06T05:16:10.217818+0000","last_peered":"2023-10-06T05:16:10.217818+0000","last_clean":"2023-10-06T05:16:10.217818+0000","last_became_active":"2023-10-06T05:16:10.210711+0000","last_became_peered":"2023-10-06T05:16:10.210711+0000","last_unstale":"2023-10-06T05:16:10.217818+0000","last_undegraded":"2023-10-06T05:16:10.217818+0000","last_fullsized":"2023-10-06T05:16:10.217818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:37.873768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219193+0000","last_change":"2023-10-06T05:16:10.211181+0000","last_active":"2023-10-06T05:16:10.219193+0000","last_peered":"2023-10-06T05:16:10.219193+0000","last_clean":"2023-10-06T05:16:10.219193+0000","last_became_active":"2023-10-06T05:16:10.210229+0000","last_became_peered":"2023-10-06T05:16:10.210229+0000","last_unstale":"2023-10-06T05:16:10.219193+0000","last_undegraded":"2023-10-06T05:16:10.219193+0000","last_fullsized":"2023-10-06T05:16:10.219193+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:07.100657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.220633+0000","last_change":"2023-10-06T05:16:10.216135+0000","last_active":"2023-10-06T05:16:10.220633+0000","last_peered":"2023-10-06T05:16:10.220633+0000","last_clean":"2023-10-06T05:16:10.220633+0000","last_became_active":"2023-10-06T05:16:10.215870+0000","last_became_peered":"2023-10-06T05:16:10.215870+0000","last_unstale":"2023-10-06T05:16:10.220633+0000","last_undegraded":"2023-10-06T05:16:10.220633+0000","last_fullsized":"2023-10-06T05:16:10.220633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:49.253876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221180+0000","last_change":"2023-10-06T05:16:10.211857+0000","last_active":"2023-10-06T05:16:10.221180+0000","last_peered":"2023-10-06T05:16:10.221180+0000","last_clean":"2023-10-06T05:16:10.221180+0000","last_became_active":"2023-10-06T05:16:10.211214+0000","last_became_peered":"2023-10-06T05:16:10.211214+0000","last_unstale":"2023-10-06T05:16:10.221180+0000","last_undegraded":"2023-10-06T05:16:10.221180+0000","last_fullsized":"2023-10-06T05:16:10.221180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:08.939753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229694+0000","last_change":"2023-10-06T05:16:10.228978+0000","last_active":"2023-10-06T05:16:10.229694+0000","last_peered":"2023-10-06T05:16:10.229694+0000","last_clean":"2023-10-06T05:16:10.229694+0000","last_became_active":"2023-10-06T05:16:10.228644+0000","last_became_peered":"2023-10-06T05:16:10.228644+0000","last_unstale":"2023-10-06T05:16:10.229694+0000","last_undegraded":"2023-10-06T05:16:10.229694+0000","last_fullsized":"2023-10-06T05:16:10.229694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:10.767004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226664+0000","last_change":"2023-10-06T05:16:10.223067+0000","last_active":"2023-10-06T05:16:10.226664+0000","last_peered":"2023-10-06T05:16:10.226664+0000","last_clean":"2023-10-06T05:16:10.226664+0000","last_became_active":"2023-10-06T05:16:10.222825+0000","last_became_peered":"2023-10-06T05:16:10.222825+0000","last_unstale":"2023-10-06T05:16:10.226664+0000","last_undegraded":"2023-10-06T05:16:10.226664+0000","last_fullsized":"2023-10-06T05:16:10.226664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:07.636440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.230450+0000","last_change":"2023-10-06T05:16:10.229864+0000","last_active":"2023-10-06T05:16:10.230450+0000","last_peered":"2023-10-06T05:16:10.230450+0000","last_clean":"2023-10-06T05:16:10.230450+0000","last_became_active":"2023-10-06T05:16:10.229548+0000","last_became_peered":"2023-10-06T05:16:10.229548+0000","last_unstale":"2023-10-06T05:16:10.230450+0000","last_undegraded":"2023-10-06T05:16:10.230450+0000","last_fullsized":"2023-10-06T05:16:10.230450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.508964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221099+0000","last_change":"2023-10-06T05:16:10.215942+0000","last_active":"2023-10-06T05:16:10.221099+0000","last_peered":"2023-10-06T05:16:10.221099+0000","last_clean":"2023-10-06T05:16:10.221099+0000","last_became_active":"2023-10-06T05:16:10.215609+0000","last_became_peered":"2023-10-06T05:16:10.215609+0000","last_unstale":"2023-10-06T05:16:10.221099+0000","last_undegraded":"2023-10-06T05:16:10.221099+0000","last_fullsized":"2023-10-06T05:16:10.221099+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:32.849654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214896+0000","last_change":"2023-10-06T05:16:10.207619+0000","last_active":"2023-10-06T05:16:10.214896+0000","last_peered":"2023-10-06T05:16:10.214896+0000","last_clean":"2023-10-06T05:16:10.214896+0000","last_became_active":"2023-10-06T05:16:10.207336+0000","last_became_peered":"2023-10-06T05:16:10.207336+0000","last_unstale":"2023-10-06T05:16:10.214896+0000","last_undegraded":"2023-10-06T05:16:10.214896+0000","last_fullsized":"2023-10-06T05:16:10.214896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:40.277272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210832+0000","last_change":"2023-10-06T05:16:10.203334+0000","last_active":"2023-10-06T05:16:10.210832+0000","last_peered":"2023-10-06T05:16:10.210832+0000","last_clean":"2023-10-06T05:16:10.210832+0000","last_became_active":"2023-10-06T05:16:10.203058+0000","last_became_peered":"2023-10-06T05:16:10.203058+0000","last_unstale":"2023-10-06T05:16:10.210832+0000","last_undegraded":"2023-10-06T05:16:10.210832+0000","last_fullsized":"2023-10-06T05:16:10.210832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:10.767004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217106+0000","last_change":"2023-10-06T05:16:10.211236+0000","last_active":"2023-10-06T05:16:10.217106+0000","last_peered":"2023-10-06T05:16:10.217106+0000","last_clean":"2023-10-06T05:16:10.217106+0000","last_became_active":"2023-10-06T05:16:10.210983+0000","last_became_peered":"2023-10-06T05:16:10.210983+0000","last_unstale":"2023-10-06T05:16:10.217106+0000","last_undegraded":"2023-10-06T05:16:10.217106+0000","last_fullsized":"2023-10-06T05:16:10.217106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:24.430630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217702+0000","last_change":"2023-10-06T05:16:10.209619+0000","last_active":"2023-10-06T05:16:10.217702+0000","last_peered":"2023-10-06T05:16:10.217702+0000","last_clean":"2023-10-06T05:16:10.217702+0000","last_became_active":"2023-10-06T05:16:10.209359+0000","last_became_peered":"2023-10-06T05:16:10.209359+0000","last_unstale":"2023-10-06T05:16:10.217702+0000","last_undegraded":"2023-10-06T05:16:10.217702+0000","last_fullsized":"2023-10-06T05:16:10.217702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:57.452379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.225675+0000","last_change":"2023-10-06T05:16:10.223579+0000","last_active":"2023-10-06T05:16:10.225675+0000","last_peered":"2023-10-06T05:16:10.225675+0000","last_clean":"2023-10-06T05:16:10.225675+0000","last_became_active":"2023-10-06T05:16:10.223087+0000","last_became_peered":"2023-10-06T05:16:10.223087+0000","last_unstale":"2023-10-06T05:16:10.225675+0000","last_undegraded":"2023-10-06T05:16:10.225675+0000","last_fullsized":"2023-10-06T05:16:10.225675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:40.277272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.212461+0000","last_change":"2023-10-06T05:16:10.209348+0000","last_active":"2023-10-06T05:16:10.212461+0000","last_peered":"2023-10-06T05:16:10.212461+0000","last_clean":"2023-10-06T05:16:10.212461+0000","last_became_active":"2023-10-06T05:16:10.208951+0000","last_became_peered":"2023-10-06T05:16:10.208951+0000","last_unstale":"2023-10-06T05:16:10.212461+0000","last_undegraded":"2023-10-06T05:16:10.212461+0000","last_fullsized":"2023-10-06T05:16:10.212461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:35.224737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218019+0000","last_change":"2023-10-06T05:16:10.212964+0000","last_active":"2023-10-06T05:16:10.218019+0000","last_peered":"2023-10-06T05:16:10.218019+0000","last_clean":"2023-10-06T05:16:10.218019+0000","last_became_active":"2023-10-06T05:16:10.212734+0000","last_became_peered":"2023-10-06T05:16:10.212734+0000","last_unstale":"2023-10-06T05:16:10.218019+0000","last_undegraded":"2023-10-06T05:16:10.218019+0000","last_fullsized":"2023-10-06T05:16:10.218019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:37.104161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217950+0000","last_change":"2023-10-06T05:16:10.213013+0000","last_active":"2023-10-06T05:16:10.217950+0000","last_peered":"2023-10-06T05:16:10.217950+0000","last_clean":"2023-10-06T05:16:10.217950+0000","last_became_active":"2023-10-06T05:16:10.212728+0000","last_became_peered":"2023-10-06T05:16:10.212728+0000","last_unstale":"2023-10-06T05:16:10.217950+0000","last_undegraded":"2023-10-06T05:16:10.217950+0000","last_fullsized":"2023-10-06T05:16:10.217950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:24.751361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.223684+0000","last_change":"2023-10-06T05:16:10.220394+0000","last_active":"2023-10-06T05:16:10.223684+0000","last_peered":"2023-10-06T05:16:10.223684+0000","last_clean":"2023-10-06T05:16:10.223684+0000","last_became_active":"2023-10-06T05:16:10.220151+0000","last_became_peered":"2023-10-06T05:16:10.220151+0000","last_unstale":"2023-10-06T05:16:10.223684+0000","last_undegraded":"2023-10-06T05:16:10.223684+0000","last_fullsized":"2023-10-06T05:16:10.223684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:34.955272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229100+0000","last_change":"2023-10-06T05:16:10.226657+0000","last_active":"2023-10-06T05:16:10.229100+0000","last_peered":"2023-10-06T05:16:10.229100+0000","last_clean":"2023-10-06T05:16:10.229100+0000","last_became_active":"2023-10-06T05:16:10.226355+0000","last_became_peered":"2023-10-06T05:16:10.226355+0000","last_unstale":"2023-10-06T05:16:10.229100+0000","last_undegraded":"2023-10-06T05:16:10.229100+0000","last_fullsized":"2023-10-06T05:16:10.229100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:43.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210157+0000","last_change":"2023-10-06T05:16:10.203310+0000","last_active":"2023-10-06T05:16:10.210157+0000","last_peered":"2023-10-06T05:16:10.210157+0000","last_clean":"2023-10-06T05:16:10.210157+0000","last_became_active":"2023-10-06T05:16:10.203011+0000","last_became_peered":"2023-10-06T05:16:10.203011+0000","last_unstale":"2023-10-06T05:16:10.210157+0000","last_undegraded":"2023-10-06T05:16:10.210157+0000","last_fullsized":"2023-10-06T05:16:10.210157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:45.385805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226639+0000","last_change":"2023-10-06T05:16:10.222909+0000","last_active":"2023-10-06T05:16:10.226639+0000","last_peered":"2023-10-06T05:16:10.226639+0000","last_clean":"2023-10-06T05:16:10.226639+0000","last_became_active":"2023-10-06T05:16:10.222662+0000","last_became_peered":"2023-10-06T05:16:10.222662+0000","last_unstale":"2023-10-06T05:16:10.226639+0000","last_undegraded":"2023-10-06T05:16:10.226639+0000","last_fullsized":"2023-10-06T05:16:10.226639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:26.802163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229949+0000","last_change":"2023-10-06T05:16:10.226872+0000","last_active":"2023-10-06T05:16:10.229949+0000","last_peered":"2023-10-06T05:16:10.229949+0000","last_clean":"2023-10-06T05:16:10.229949+0000","last_became_active":"2023-10-06T05:16:10.226600+0000","last_became_peered":"2023-10-06T05:16:10.226600+0000","last_unstale":"2023-10-06T05:16:10.229949+0000","last_undegraded":"2023-10-06T05:16:10.229949+0000","last_fullsized":"2023-10-06T05:16:10.229949+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:38.594765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211415+0000","last_change":"2023-10-06T05:16:10.205229+0000","last_active":"2023-10-06T05:16:10.211415+0000","last_peered":"2023-10-06T05:16:10.211415+0000","last_clean":"2023-10-06T05:16:10.211415+0000","last_became_active":"2023-10-06T05:16:10.204948+0000","last_became_peered":"2023-10-06T05:16:10.204948+0000","last_unstale":"2023-10-06T05:16:10.211415+0000","last_undegraded":"2023-10-06T05:16:10.211415+0000","last_fullsized":"2023-10-06T05:16:10.211415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:19.783131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229145+0000","last_change":"2023-10-06T05:16:10.226881+0000","last_active":"2023-10-06T05:16:10.229145+0000","last_peered":"2023-10-06T05:16:10.229145+0000","last_clean":"2023-10-06T05:16:10.229145+0000","last_became_active":"2023-10-06T05:16:10.226612+0000","last_became_peered":"2023-10-06T05:16:10.226612+0000","last_unstale":"2023-10-06T05:16:10.229145+0000","last_undegraded":"2023-10-06T05:16:10.229145+0000","last_fullsized":"2023-10-06T05:16:10.229145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:39.377431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229186+0000","last_change":"2023-10-06T05:16:10.226299+0000","last_active":"2023-10-06T05:16:10.229186+0000","last_peered":"2023-10-06T05:16:10.229186+0000","last_clean":"2023-10-06T05:16:10.229186+0000","last_became_active":"2023-10-06T05:16:10.221517+0000","last_became_peered":"2023-10-06T05:16:10.221517+0000","last_unstale":"2023-10-06T05:16:10.229186+0000","last_undegraded":"2023-10-06T05:16:10.229186+0000","last_fullsized":"2023-10-06T05:16:10.229186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:28.570142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217894+0000","last_change":"2023-10-06T05:16:10.212385+0000","last_active":"2023-10-06T05:16:10.217894+0000","last_peered":"2023-10-06T05:16:10.217894+0000","last_clean":"2023-10-06T05:16:10.217894+0000","last_became_active":"2023-10-06T05:16:10.212100+0000","last_became_peered":"2023-10-06T05:16:10.212100+0000","last_unstale":"2023-10-06T05:16:10.217894+0000","last_undegraded":"2023-10-06T05:16:10.217894+0000","last_fullsized":"2023-10-06T05:16:10.217894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:43.801764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210468+0000","last_change":"2023-10-06T05:16:09.210776+0000","last_active":"2023-10-06T05:16:10.210468+0000","last_peered":"2023-10-06T05:16:10.210468+0000","last_clean":"2023-10-06T05:16:10.210468+0000","last_became_active":"2023-10-06T05:16:09.210484+0000","last_became_peered":"2023-10-06T05:16:09.210484+0000","last_unstale":"2023-10-06T05:16:10.210468+0000","last_undegraded":"2023-10-06T05:16:10.210468+0000","last_fullsized":"2023-10-06T05:16:10.210468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:31.844638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.223699+0000","last_change":"2023-10-06T05:16:10.219330+0000","last_active":"2023-10-06T05:16:10.223699+0000","last_peered":"2023-10-06T05:16:10.223699+0000","last_clean":"2023-10-06T05:16:10.223699+0000","last_became_active":"2023-10-06T05:16:10.219092+0000","last_became_peered":"2023-10-06T05:16:10.219092+0000","last_unstale":"2023-10-06T05:16:10.223699+0000","last_undegraded":"2023-10-06T05:16:10.223699+0000","last_fullsized":"2023-10-06T05:16:10.223699+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:13.881962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229325+0000","last_change":"2023-10-06T05:16:09.211344+0000","last_active":"2023-10-06T05:16:10.229325+0000","last_peered":"2023-10-06T05:16:10.229325+0000","last_clean":"2023-10-06T05:16:10.229325+0000","last_became_active":"2023-10-06T05:16:09.211052+0000","last_became_peered":"2023-10-06T05:16:09.211052+0000","last_unstale":"2023-10-06T05:16:10.229325+0000","last_undegraded":"2023-10-06T05:16:10.229325+0000","last_fullsized":"2023-10-06T05:16:10.229325+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:43.354681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229379+0000","last_change":"2023-10-06T05:16:10.226547+0000","last_active":"2023-10-06T05:16:10.229379+0000","last_peered":"2023-10-06T05:16:10.229379+0000","last_clean":"2023-10-06T05:16:10.229379+0000","last_became_active":"2023-10-06T05:16:10.220212+0000","last_became_peered":"2023-10-06T05:16:10.220212+0000","last_unstale":"2023-10-06T05:16:10.229379+0000","last_undegraded":"2023-10-06T05:16:10.229379+0000","last_fullsized":"2023-10-06T05:16:10.229379+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.508964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211434+0000","last_change":"2023-10-06T05:16:10.209702+0000","last_active":"2023-10-06T05:16:10.211434+0000","last_peered":"2023-10-06T05:16:10.211434+0000","last_clean":"2023-10-06T05:16:10.211434+0000","last_became_active":"2023-10-06T05:16:10.209390+0000","last_became_peered":"2023-10-06T05:16:10.209390+0000","last_unstale":"2023-10-06T05:16:10.211434+0000","last_undegraded":"2023-10-06T05:16:10.211434+0000","last_fullsized":"2023-10-06T05:16:10.211434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:42.845785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219726+0000","last_change":"2023-10-06T05:16:10.212070+0000","last_active":"2023-10-06T05:16:10.219726+0000","last_peered":"2023-10-06T05:16:10.219726+0000","last_clean":"2023-10-06T05:16:10.219726+0000","last_became_active":"2023-10-06T05:16:10.211782+0000","last_became_peered":"2023-10-06T05:16:10.211782+0000","last_unstale":"2023-10-06T05:16:10.219726+0000","last_undegraded":"2023-10-06T05:16:10.219726+0000","last_fullsized":"2023-10-06T05:16:10.219726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:35.024962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210095+0000","last_change":"2023-10-06T05:16:09.211131+0000","last_active":"2023-10-06T05:16:10.210095+0000","last_peered":"2023-10-06T05:16:10.210095+0000","last_clean":"2023-10-06T05:16:10.210095+0000","last_became_active":"2023-10-06T05:16:09.210879+0000","last_became_peered":"2023-10-06T05:16:09.210879+0000","last_unstale":"2023-10-06T05:16:10.210095+0000","last_undegraded":"2023-10-06T05:16:10.210095+0000","last_fullsized":"2023-10-06T05:16:10.210095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:06.631425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.207977+0000","last_change":"2023-10-06T05:16:10.205445+0000","last_active":"2023-10-06T05:16:10.207977+0000","last_peered":"2023-10-06T05:16:10.207977+0000","last_clean":"2023-10-06T05:16:10.207977+0000","last_became_active":"2023-10-06T05:16:10.205159+0000","last_became_peered":"2023-10-06T05:16:10.205159+0000","last_unstale":"2023-10-06T05:16:10.207977+0000","last_undegraded":"2023-10-06T05:16:10.207977+0000","last_fullsized":"2023-10-06T05:16:10.207977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:23.294418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193687+0000","last_change":"2023-10-06T05:16:09.204336+0000","last_active":"2023-10-06T05:16:10.193687+0000","last_peered":"2023-10-06T05:16:10.193687+0000","last_clean":"2023-10-06T05:16:10.193687+0000","last_became_active":"2023-10-06T05:16:09.204059+0000","last_became_peered":"2023-10-06T05:16:09.204059+0000","last_unstale":"2023-10-06T05:16:10.193687+0000","last_undegraded":"2023-10-06T05:16:10.193687+0000","last_fullsized":"2023-10-06T05:16:10.193687+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:42.471127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218002+0000","last_change":"2023-10-06T05:16:10.210141+0000","last_active":"2023-10-06T05:16:10.218002+0000","last_peered":"2023-10-06T05:16:10.218002+0000","last_clean":"2023-10-06T05:16:10.218002+0000","last_became_active":"2023-10-06T05:16:10.208711+0000","last_became_peered":"2023-10-06T05:16:10.208711+0000","last_unstale":"2023-10-06T05:16:10.218002+0000","last_undegraded":"2023-10-06T05:16:10.218002+0000","last_fullsized":"2023-10-06T05:16:10.218002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:21.353918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.198161+0000","last_change":"2023-10-06T05:16:09.212018+0000","last_active":"2023-10-06T05:16:10.198161+0000","last_peered":"2023-10-06T05:16:10.198161+0000","last_clean":"2023-10-06T05:16:10.198161+0000","last_became_active":"2023-10-06T05:16:09.211333+0000","last_became_peered":"2023-10-06T05:16:09.211333+0000","last_unstale":"2023-10-06T05:16:10.198161+0000","last_undegraded":"2023-10-06T05:16:10.198161+0000","last_fullsized":"2023-10-06T05:16:10.198161+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:38.372416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217287+0000","last_change":"2023-10-06T05:16:10.210825+0000","last_active":"2023-10-06T05:16:10.217287+0000","last_peered":"2023-10-06T05:16:10.217287+0000","last_clean":"2023-10-06T05:16:10.217287+0000","last_became_active":"2023-10-06T05:16:10.210544+0000","last_became_peered":"2023-10-06T05:16:10.210544+0000","last_unstale":"2023-10-06T05:16:10.217287+0000","last_undegraded":"2023-10-06T05:16:10.217287+0000","last_fullsized":"2023-10-06T05:16:10.217287+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:35.024962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.194295+0000","last_change":"2023-10-06T05:16:09.207532+0000","last_active":"2023-10-06T05:16:10.194295+0000","last_peered":"2023-10-06T05:16:10.194295+0000","last_clean":"2023-10-06T05:16:10.194295+0000","last_became_active":"2023-10-06T05:16:09.207231+0000","last_became_peered":"2023-10-06T05:16:09.207231+0000","last_unstale":"2023-10-06T05:16:10.194295+0000","last_undegraded":"2023-10-06T05:16:10.194295+0000","last_fullsized":"2023-10-06T05:16:10.194295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:34.219722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.201998+0000","last_change":"2023-10-06T05:16:09.208953+0000","last_active":"2023-10-06T05:16:10.201998+0000","last_peered":"2023-10-06T05:16:10.201998+0000","last_clean":"2023-10-06T05:16:10.201998+0000","last_became_active":"2023-10-06T05:16:09.208620+0000","last_became_peered":"2023-10-06T05:16:09.208620+0000","last_unstale":"2023-10-06T05:16:10.201998+0000","last_undegraded":"2023-10-06T05:16:10.201998+0000","last_fullsized":"2023-10-06T05:16:10.201998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:43.354681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.230626+0000","last_change":"2023-10-06T05:16:09.206717+0000","last_active":"2023-10-06T05:16:10.230626+0000","last_peered":"2023-10-06T05:16:10.230626+0000","last_clean":"2023-10-06T05:16:10.230626+0000","last_became_active":"2023-10-06T05:16:09.206406+0000","last_became_peered":"2023-10-06T05:16:09.206406+0000","last_unstale":"2023-10-06T05:16:10.230626+0000","last_undegraded":"2023-10-06T05:16:10.230626+0000","last_fullsized":"2023-10-06T05:16:10.230626+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:48.306767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214816+0000","last_change":"2023-10-06T05:16:10.207594+0000","last_active":"2023-10-06T05:16:10.214816+0000","last_peered":"2023-10-06T05:16:10.214816+0000","last_clean":"2023-10-06T05:16:10.214816+0000","last_became_active":"2023-10-06T05:16:10.207287+0000","last_became_peered":"2023-10-06T05:16:10.207287+0000","last_unstale":"2023-10-06T05:16:10.214816+0000","last_undegraded":"2023-10-06T05:16:10.214816+0000","last_fullsized":"2023-10-06T05:16:10.214816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:56.963759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219750+0000","last_change":"2023-10-06T05:16:10.213380+0000","last_active":"2023-10-06T05:16:10.219750+0000","last_peered":"2023-10-06T05:16:10.219750+0000","last_clean":"2023-10-06T05:16:10.219750+0000","last_became_active":"2023-10-06T05:16:10.212923+0000","last_became_peered":"2023-10-06T05:16:10.212923+0000","last_unstale":"2023-10-06T05:16:10.219750+0000","last_undegraded":"2023-10-06T05:16:10.219750+0000","last_fullsized":"2023-10-06T05:16:10.219750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:19.783131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219278+0000","last_change":"2023-10-06T05:16:09.213428+0000","last_active":"2023-10-06T05:16:10.219278+0000","last_peered":"2023-10-06T05:16:10.219278+0000","last_clean":"2023-10-06T05:16:10.219278+0000","last_became_active":"2023-10-06T05:16:09.213153+0000","last_became_peered":"2023-10-06T05:16:09.213153+0000","last_unstale":"2023-10-06T05:16:10.219278+0000","last_undegraded":"2023-10-06T05:16:10.219278+0000","last_fullsized":"2023-10-06T05:16:10.219278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:39.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221604+0000","last_change":"2023-10-06T05:16:10.218301+0000","last_active":"2023-10-06T05:16:10.221604+0000","last_peered":"2023-10-06T05:16:10.221604+0000","last_clean":"2023-10-06T05:16:10.221604+0000","last_became_active":"2023-10-06T05:16:10.218018+0000","last_became_peered":"2023-10-06T05:16:10.218018+0000","last_unstale":"2023-10-06T05:16:10.221604+0000","last_undegraded":"2023-10-06T05:16:10.221604+0000","last_fullsized":"2023-10-06T05:16:10.221604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:30.897795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226641+0000","last_change":"2023-10-06T05:16:09.208223+0000","last_active":"2023-10-06T05:16:10.226641+0000","last_peered":"2023-10-06T05:16:10.226641+0000","last_clean":"2023-10-06T05:16:10.226641+0000","last_became_active":"2023-10-06T05:16:09.207913+0000","last_became_peered":"2023-10-06T05:16:09.207913+0000","last_unstale":"2023-10-06T05:16:10.226641+0000","last_undegraded":"2023-10-06T05:16:10.226641+0000","last_fullsized":"2023-10-06T05:16:10.226641+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:23.746345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221153+0000","last_change":"2023-10-06T05:16:10.220179+0000","last_active":"2023-10-06T05:16:10.221153+0000","last_peered":"2023-10-06T05:16:10.221153+0000","last_clean":"2023-10-06T05:16:10.221153+0000","last_became_active":"2023-10-06T05:16:10.219906+0000","last_became_peered":"2023-10-06T05:16:10.219906+0000","last_unstale":"2023-10-06T05:16:10.221153+0000","last_undegraded":"2023-10-06T05:16:10.221153+0000","last_fullsized":"2023-10-06T05:16:10.221153+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:05.613161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210393+0000","last_change":"2023-10-06T05:16:09.207565+0000","last_active":"2023-10-06T05:16:10.210393+0000","last_peered":"2023-10-06T05:16:10.210393+0000","last_clean":"2023-10-06T05:16:10.210393+0000","last_became_active":"2023-10-06T05:16:09.207265+0000","last_became_peered":"2023-10-06T05:16:09.207265+0000","last_unstale":"2023-10-06T05:16:10.210393+0000","last_undegraded":"2023-10-06T05:16:10.210393+0000","last_fullsized":"2023-10-06T05:16:10.210393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:07.934738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226672+0000","last_change":"2023-10-06T05:16:10.221570+0000","last_active":"2023-10-06T05:16:10.226672+0000","last_peered":"2023-10-06T05:16:10.226672+0000","last_clean":"2023-10-06T05:16:10.226672+0000","last_became_active":"2023-10-06T05:16:10.221389+0000","last_became_peered":"2023-10-06T05:16:10.221389+0000","last_unstale":"2023-10-06T05:16:10.226672+0000","last_undegraded":"2023-10-06T05:16:10.226672+0000","last_fullsized":"2023-10-06T05:16:10.226672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:30.897795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211334+0000","last_change":"2023-10-06T05:16:09.204770+0000","last_active":"2023-10-06T05:16:10.211334+0000","last_peered":"2023-10-06T05:16:10.211334+0000","last_clean":"2023-10-06T05:16:10.211334+0000","last_became_active":"2023-10-06T05:16:09.204549+0000","last_became_peered":"2023-10-06T05:16:09.204549+0000","last_unstale":"2023-10-06T05:16:10.211334+0000","last_undegraded":"2023-10-06T05:16:10.211334+0000","last_fullsized":"2023-10-06T05:16:10.211334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:44.503949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.232760+0000","last_change":"2023-10-06T05:16:10.227849+0000","last_active":"2023-10-06T05:16:10.232760+0000","last_peered":"2023-10-06T05:16:10.232760+0000","last_clean":"2023-10-06T05:16:10.232760+0000","last_became_active":"2023-10-06T05:16:10.227571+0000","last_became_peered":"2023-10-06T05:16:10.227571+0000","last_unstale":"2023-10-06T05:16:10.232760+0000","last_undegraded":"2023-10-06T05:16:10.232760+0000","last_fullsized":"2023-10-06T05:16:10.232760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:38.653708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.227973+0000","last_change":"2023-10-06T05:16:10.220468+0000","last_active":"2023-10-06T05:16:10.227973+0000","last_peered":"2023-10-06T05:16:10.227973+0000","last_clean":"2023-10-06T05:16:10.227973+0000","last_became_active":"2023-10-06T05:16:10.220190+0000","last_became_peered":"2023-10-06T05:16:10.220190+0000","last_unstale":"2023-10-06T05:16:10.227973+0000","last_undegraded":"2023-10-06T05:16:10.227973+0000","last_fullsized":"2023-10-06T05:16:10.227973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:43.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193811+0000","last_change":"2023-10-06T05:16:09.207873+0000","last_active":"2023-10-06T05:16:10.193811+0000","last_peered":"2023-10-06T05:16:10.193811+0000","last_clean":"2023-10-06T05:16:10.193811+0000","last_became_active":"2023-10-06T05:16:09.207577+0000","last_became_peered":"2023-10-06T05:16:09.207577+0000","last_unstale":"2023-10-06T05:16:10.193811+0000","last_undegraded":"2023-10-06T05:16:10.193811+0000","last_fullsized":"2023-10-06T05:16:10.193811+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:39.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.220486+0000","last_change":"2023-10-06T05:16:10.214435+0000","last_active":"2023-10-06T05:16:10.220486+0000","last_peered":"2023-10-06T05:16:10.220486+0000","last_clean":"2023-10-06T05:16:10.220486+0000","last_became_active":"2023-10-06T05:16:10.214248+0000","last_became_peered":"2023-10-06T05:16:10.214248+0000","last_unstale":"2023-10-06T05:16:10.220486+0000","last_undegraded":"2023-10-06T05:16:10.220486+0000","last_fullsized":"2023-10-06T05:16:10.220486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:57.452379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.201737+0000","last_change":"2023-10-06T05:16:09.205408+0000","last_active":"2023-10-06T05:16:10.201737+0000","last_peered":"2023-10-06T05:16:10.201737+0000","last_clean":"2023-10-06T05:16:10.201737+0000","last_became_active":"2023-10-06T05:16:09.205130+0000","last_became_peered":"2023-10-06T05:16:09.205130+0000","last_unstale":"2023-10-06T05:16:10.201737+0000","last_undegraded":"2023-10-06T05:16:10.201737+0000","last_fullsized":"2023-10-06T05:16:10.201737+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:37.589749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221123+0000","last_change":"2023-10-06T05:16:10.217361+0000","last_active":"2023-10-06T05:16:10.221123+0000","last_peered":"2023-10-06T05:16:10.221123+0000","last_clean":"2023-10-06T05:16:10.221123+0000","last_became_active":"2023-10-06T05:16:10.217045+0000","last_became_peered":"2023-10-06T05:16:10.217045+0000","last_unstale":"2023-10-06T05:16:10.221123+0000","last_undegraded":"2023-10-06T05:16:10.221123+0000","last_fullsized":"2023-10-06T05:16:10.221123+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:00.820889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226750+0000","last_change":"2023-10-06T05:16:10.222136+0000","last_active":"2023-10-06T05:16:10.226750+0000","last_peered":"2023-10-06T05:16:10.226750+0000","last_clean":"2023-10-06T05:16:10.226750+0000","last_became_active":"2023-10-06T05:16:10.221565+0000","last_became_peered":"2023-10-06T05:16:10.221565+0000","last_unstale":"2023-10-06T05:16:10.226750+0000","last_undegraded":"2023-10-06T05:16:10.226750+0000","last_fullsized":"2023-10-06T05:16:10.226750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:07.100657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.198391+0000","last_change":"2023-10-06T05:16:09.212218+0000","last_active":"2023-10-06T05:16:10.198391+0000","last_peered":"2023-10-06T05:16:10.198391+0000","last_clean":"2023-10-06T05:16:10.198391+0000","last_became_active":"2023-10-06T05:16:09.211750+0000","last_became_peered":"2023-10-06T05:16:09.211750+0000","last_unstale":"2023-10-06T05:16:10.198391+0000","last_undegraded":"2023-10-06T05:16:10.198391+0000","last_fullsized":"2023-10-06T05:16:10.198391+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:29.892780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.215807+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:16:10.215807+0000","last_peered":"2023-10-06T05:16:10.215807+0000","last_clean":"2023-10-06T05:16:10.215807+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:16:10.215807+0000","last_undegraded":"2023-10-06T05:16:10.215807+0000","last_fullsized":"2023-10-06T05:16:10.215807+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214837+0000","last_change":"2023-10-06T05:16:10.208514+0000","last_active":"2023-10-06T05:16:10.214837+0000","last_peered":"2023-10-06T05:16:10.214837+0000","last_clean":"2023-10-06T05:16:10.214837+0000","last_became_active":"2023-10-06T05:16:10.208212+0000","last_became_peered":"2023-10-06T05:16:10.208212+0000","last_unstale":"2023-10-06T05:16:10.214837+0000","last_undegraded":"2023-10-06T05:16:10.214837+0000","last_fullsized":"2023-10-06T05:16:10.214837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:44.359697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193993+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:16:10.193993+0000","last_peered":"2023-10-06T05:16:10.193993+0000","last_clean":"2023-10-06T05:16:10.193993+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:16:10.193993+0000","last_undegraded":"2023-10-06T05:16:10.193993+0000","last_fullsized":"2023-10-06T05:16:10.193993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219222+0000","last_change":"2023-10-06T05:16:10.211591+0000","last_active":"2023-10-06T05:16:10.219222+0000","last_peered":"2023-10-06T05:16:10.219222+0000","last_clean":"2023-10-06T05:16:10.219222+0000","last_became_active":"2023-10-06T05:16:10.211351+0000","last_became_peered":"2023-10-06T05:16:10.211351+0000","last_unstale":"2023-10-06T05:16:10.219222+0000","last_undegraded":"2023-10-06T05:16:10.219222+0000","last_fullsized":"2023-10-06T05:16:10.219222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:37.873768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211290+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:16:10.211290+0000","last_peered":"2023-10-06T05:16:10.211290+0000","last_clean":"2023-10-06T05:16:10.211290+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:16:10.211290+0000","last_undegraded":"2023-10-06T05:16:10.211290+0000","last_fullsized":"2023-10-06T05:16:10.211290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221191+0000","last_change":"2023-10-06T05:16:10.213644+0000","last_active":"2023-10-06T05:16:10.221191+0000","last_peered":"2023-10-06T05:16:10.221191+0000","last_clean":"2023-10-06T05:16:10.221191+0000","last_became_active":"2023-10-06T05:16:10.212920+0000","last_became_peered":"2023-10-06T05:16:10.212920+0000","last_unstale":"2023-10-06T05:16:10.221191+0000","last_undegraded":"2023-10-06T05:16:10.221191+0000","last_fullsized":"2023-10-06T05:16:10.221191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:13.881962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.224213+0000","last_change":"2023-10-06T05:16:09.207900+0000","last_active":"2023-10-06T05:16:10.224213+0000","last_peered":"2023-10-06T05:16:10.224213+0000","last_clean":"2023-10-06T05:16:10.224213+0000","last_became_active":"2023-10-06T05:16:09.207634+0000","last_became_peered":"2023-10-06T05:16:09.207634+0000","last_unstale":"2023-10-06T05:16:10.224213+0000","last_undegraded":"2023-10-06T05:16:10.224213+0000","last_fullsized":"2023-10-06T05:16:10.224213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:38.372416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193680+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:16:10.193680+0000","last_peered":"2023-10-06T05:16:10.193680+0000","last_clean":"2023-10-06T05:16:10.193680+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:16:10.193680+0000","last_undegraded":"2023-10-06T05:16:10.193680+0000","last_fullsized":"2023-10-06T05:16:10.193680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":174,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T05:16:02.924468+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:16:02.924468+0000","last_peered":"2023-10-06T05:16:02.924468+0000","last_clean":"2023-10-06T05:16:02.924468+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:16:02.924468+0000","last_undegraded":"2023-10-06T05:16:02.924468+0000","last_fullsized":"2023-10-06T05:16:02.924468+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.215707+0000","last_change":"2023-10-06T05:16:09.212220+0000","last_active":"2023-10-06T05:16:10.215707+0000","last_peered":"2023-10-06T05:16:10.215707+0000","last_clean":"2023-10-06T05:16:10.215707+0000","last_became_active":"2023-10-06T05:16:09.211960+0000","last_became_peered":"2023-10-06T05:16:09.211960+0000","last_unstale":"2023-10-06T05:16:10.215707+0000","last_undegraded":"2023-10-06T05:16:10.215707+0000","last_fullsized":"2023-10-06T05:16:10.215707+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:42.796749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T05:16:02.924893+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:16:02.924893+0000","last_peered":"2023-10-06T05:16:02.924893+0000","last_clean":"2023-10-06T05:16:02.924893+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:16:02.924893+0000","last_undegraded":"2023-10-06T05:16:02.924893+0000","last_fullsized":"2023-10-06T05:16:02.924893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218145+0000","last_change":"2023-10-06T05:16:10.215983+0000","last_active":"2023-10-06T05:16:10.218145+0000","last_peered":"2023-10-06T05:16:10.218145+0000","last_clean":"2023-10-06T05:16:10.218145+0000","last_became_active":"2023-10-06T05:16:10.215700+0000","last_became_peered":"2023-10-06T05:16:10.215700+0000","last_unstale":"2023-10-06T05:16:10.218145+0000","last_undegraded":"2023-10-06T05:16:10.218145+0000","last_fullsized":"2023-10-06T05:16:10.218145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:30.610440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.215771+0000","last_change":"2023-10-06T05:16:09.208026+0000","last_active":"2023-10-06T05:16:10.215771+0000","last_peered":"2023-10-06T05:16:10.215771+0000","last_clean":"2023-10-06T05:16:10.215771+0000","last_became_active":"2023-10-06T05:16:09.207756+0000","last_became_peered":"2023-10-06T05:16:09.207756+0000","last_unstale":"2023-10-06T05:16:10.215771+0000","last_undegraded":"2023-10-06T05:16:10.215771+0000","last_fullsized":"2023-10-06T05:16:10.215771+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:37.589749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T05:16:02.925278+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:16:02.925278+0000","last_peered":"2023-10-06T05:16:02.925278+0000","last_clean":"2023-10-06T05:16:02.925278+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:16:02.925278+0000","last_undegraded":"2023-10-06T05:16:02.925278+0000","last_fullsized":"2023-10-06T05:16:02.925278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.194766+0000","last_change":"2023-10-06T05:16:09.212202+0000","last_active":"2023-10-06T05:16:10.194766+0000","last_peered":"2023-10-06T05:16:10.194766+0000","last_clean":"2023-10-06T05:16:10.194766+0000","last_became_active":"2023-10-06T05:16:09.211949+0000","last_became_peered":"2023-10-06T05:16:09.211949+0000","last_unstale":"2023-10-06T05:16:10.194766+0000","last_undegraded":"2023-10-06T05:16:10.194766+0000","last_fullsized":"2023-10-06T05:16:10.194766+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:08.250446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219090+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:16:10.219090+0000","last_peered":"2023-10-06T05:16:10.219090+0000","last_clean":"2023-10-06T05:16:10.219090+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:16:10.219090+0000","last_undegraded":"2023-10-06T05:16:10.219090+0000","last_fullsized":"2023-10-06T05:16:10.219090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:09.182943+0000","last_change":"2023-10-06T05:16:09.182943+0000","last_active":"2023-10-06T05:16:09.182943+0000","last_peered":"2023-10-06T05:16:09.182943+0000","last_clean":"2023-10-06T05:16:09.182943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:09.182943+0000","last_undegraded":"2023-10-06T05:16:09.182943+0000","last_fullsized":"2023-10-06T05:16:09.182943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:16:08.177928+0000","last_change":"2023-10-06T05:16:08.177928+0000","last_active":"2023-10-06T05:16:08.177928+0000","last_peered":"2023-10-06T05:16:08.177928+0000","last_clean":"2023-10-06T05:16:08.177928+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:16:08.177928+0000","last_undegraded":"2023-10-06T05:16:08.177928+0000","last_fullsized":"2023-10-06T05:16:08.177928+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.208812+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:16:10.208812+0000","last_peered":"2023-10-06T05:16:10.208812+0000","last_clean":"2023-10-06T05:16:10.208812+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:16:10.208812+0000","last_undegraded":"2023-10-06T05:16:10.208812+0000","last_fullsized":"2023-10-06T05:16:10.208812+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":98,"acting":98,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1628,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":38,"acting":38,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":983040,"data_stored":759540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.504}]},{"osd":2,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.46999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":197619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":9.6300000000000008,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.595000000000001,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.825,"5min":1.825,"15min":1.825},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":23.146000000000001,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.80700000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":9.9969999999999999,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.622},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":9.5850000000000009,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":9.8420000000000005,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":9.718,"5min":9.718,"15min":9.718},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":9.8930000000000007,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.85299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":430080,"data_stored":190084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.438}]},{"osd":3,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298300,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073540,"statfs":{"total":96636764160,"available":96331304960,"internally_reserved":0,"allocated":1036288,"data_stored":787377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.379}]},{"osd":2,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.33100000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":191572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.42499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.629}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":167819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.35099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.443}]},{"osd":4,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":450560,"data_stored":198511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.622},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":197045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.79900000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.83899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:16:14.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:16:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.567+0000 7f24857b2700 1 Processor -- start 2023-10-06T05:16:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.568+0000 7f24857b2700 1 -- start start 2023-10-06T05:16:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.568+0000 7f24857b2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2480158fa0 0x7f248015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.568+0000 7f24857b2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248015d7b0 con 0x7f248015d310 2023-10-06T05:16:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.568+0000 7f24857b2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248015da90 con 0x7f2480159370 2023-10-06T05:16:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f24857b2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248015dd70 con 0x7f2480158fa0 2023-10-06T05:16:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2480158fa0 0x7f248015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247e7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f248015d310 0x7f24800ace30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52322/0 (socket says 172.21.15.45:52322) 2023-10-06T05:16:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247e7fc700 1 -- 172.21.15.45:0/944554017 learned_addr learned my addr 172.21.15.45:0/944554017 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247effd700 1 -- 172.21.15.45:0/944554017 >> v1:172.21.15.144:6789/0 conn(0x7f2480159370 legacy=0x7f24800ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247effd700 1 -- 172.21.15.45:0/944554017 >> v1:172.21.15.45:6789/0 conn(0x7f248015d310 legacy=0x7f24800ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247effd700 1 -- 172.21.15.45:0/944554017 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800c76d0 con 0x7f2480158fa0 2023-10-06T05:16:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247effd700 1 --2- 172.21.15.45:0/944554017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2480158fa0 0x7f248015cdd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f24700098e0 tx=0x7f24700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe86425546c4c46e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247dffb700 1 -- 172.21.15.45:0/944554017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470009e80 con 0x7f2480158fa0 2023-10-06T05:16:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247dffb700 1 -- 172.21.15.45:0/944554017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247000b860 con 0x7f2480158fa0 2023-10-06T05:16:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.569+0000 7f247dffb700 1 -- 172.21.15.45:0/944554017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000bbe0 con 0x7f2480158fa0 2023-10-06T05:16:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.570+0000 7f24857b2700 1 -- 172.21.15.45:0/944554017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2480158fa0 msgr2=0x7f248015cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.570+0000 7f24857b2700 1 --2- 172.21.15.45:0/944554017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2480158fa0 0x7f248015cdd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f24700098e0 tx=0x7f24700091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.570+0000 7f24857b2700 1 -- 172.21.15.45:0/944554017 shutdown_connections 2023-10-06T05:16:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.570+0000 7f24857b2700 1 --2- 172.21.15.45:0/944554017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2480158fa0 0x7f248015cdd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.570+0000 7f24857b2700 1 -- 172.21.15.45:0/944554017 >> 172.21.15.45:0/944554017 conn(0x7f248006f070 msgr2=0x7f248006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.570+0000 7f24857b2700 1 -- 172.21.15.45:0/944554017 shutdown_connections 2023-10-06T05:16:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.570+0000 7f24857b2700 1 -- 172.21.15.45:0/944554017 wait complete. 2023-10-06T05:16:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.571+0000 7f24857b2700 1 Processor -- start 2023-10-06T05:16:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.571+0000 7f24857b2700 1 -- start start 2023-10-06T05:16:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f24857b2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2480158fa0 0x7f248014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f24857b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2480159370 0x7f248006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f24857b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248015d310 0x7f248006b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f24857b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800b08b0 con 0x7f248015d310 2023-10-06T05:16:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f24857b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800b0b90 con 0x7f2480159370 2023-10-06T05:16:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f24857b2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24800b0ec0 con 0x7f2480158fa0 2023-10-06T05:16:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f247f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248015d310 0x7f248006b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f247f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248015d310 0x7f248006b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33122/0 (socket says 172.21.15.45:33122) 2023-10-06T05:16:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f247f7fe700 1 -- 172.21.15.45:0/1123195590 learned_addr learned my addr 172.21.15.45:0/1123195590 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f247effd700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2480158fa0 0x7f248014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.572+0000 7f247e7fc700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2480159370 0x7f248006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f247f7fe700 1 -- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2480158fa0 msgr2=0x7f248014dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f247f7fe700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2480158fa0 0x7f248014dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f247f7fe700 1 -- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2480159370 msgr2=0x7f248006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f247f7fe700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2480159370 0x7f248006b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f247f7fe700 1 -- 172.21.15.45:0/1123195590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470009420 con 0x7f248015d310 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f247e7fc700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2480159370 0x7f248006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f247f7fe700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248015d310 0x7f248006b6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f247800c930 tx=0x7f24780026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f2466ffd700 1 -- 172.21.15.45:0/1123195590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478012630 con 0x7f248015d310 2023-10-06T05:16:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f2466ffd700 1 -- 172.21.15.45:0/1123195590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24780127d0 con 0x7f248015d310 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24800b11a0 con 0x7f248015d310 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f2466ffd700 1 -- 172.21.15.45:0/1123195590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478012b90 con 0x7f248015d310 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.573+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248014e150 con 0x7f248015d310 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.574+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248006e2d0 con 0x7f248015d310 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.574+0000 7f2466ffd700 1 -- 172.21.15.45:0/1123195590 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f247800a3b0 con 0x7f248015d310 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.575+0000 7f2466ffd700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f246c066f30 0x7f246c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.575+0000 7f2466ffd700 1 -- 172.21.15.45:0/1123195590 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f24780844e0 con 0x7f248015d310 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.575+0000 7f247effd700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f246c066f30 0x7f246c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.576+0000 7f247effd700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f246c066f30 0x7f246c0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f24700030c0 tx=0x7f2470016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:14.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.579+0000 7f2466ffd700 1 -- 172.21.15.45:0/1123195590 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24780514b0 con 0x7f248015d310 2023-10-06T05:16:14.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.777+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24800c79b0 con 0x7f246c066f30 2023-10-06T05:16:14.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.786+0000 7f2466ffd700 1 -- 172.21.15.45:0/1123195590 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311562 (secure 0 0 0) 0x7f24800c79b0 con 0x7f246c066f30 2023-10-06T05:16:14.792 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:14.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f246c066f30 msgr2=0x7f246c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f246c066f30 0x7f246c0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f24700030c0 tx=0x7f2470016000 comp rx=0 tx=0).stop 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248015d310 msgr2=0x7f248006b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248015d310 0x7f248006b6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f247800c930 tx=0x7f24780026e0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 shutdown_connections 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f246c066f30 0x7f246c0693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248015d310 0x7f248006b6f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2480158fa0 0x7f248014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 --2- 172.21.15.45:0/1123195590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2480159370 0x7f248006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 >> 172.21.15.45:0/1123195590 conn(0x7f248006f070 msgr2=0x7f24800cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 shutdown_connections 2023-10-06T05:16:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:14.790+0000 7f24857b2700 1 -- 172.21.15.45:0/1123195590 wait complete. 2023-10-06T05:16:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:16:14.805 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-10-06T05:16:14.459139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382924,"kb_used_data":4620,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591796,"statfs":{"total":773094113280,"available":770653999104,"internally_reserved":0,"allocated":4730880,"data_stored":2745873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001544"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214593+0000","last_change":"2023-10-06T05:16:10.211576+0000","last_active":"2023-10-06T05:16:10.214593+0000","last_peered":"2023-10-06T05:16:10.214593+0000","last_clean":"2023-10-06T05:16:10.214593+0000","last_became_active":"2023-10-06T05:16:10.211322+0000","last_became_peered":"2023-10-06T05:16:10.211322+0000","last_unstale":"2023-10-06T05:16:10.214593+0000","last_undegraded":"2023-10-06T05:16:10.214593+0000","last_fullsized":"2023-10-06T05:16:10.214593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:42.845785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214591+0000","last_change":"2023-10-06T05:16:10.205723+0000","last_active":"2023-10-06T05:16:10.214591+0000","last_peered":"2023-10-06T05:16:10.214591+0000","last_clean":"2023-10-06T05:16:10.214591+0000","last_became_active":"2023-10-06T05:16:10.205427+0000","last_became_peered":"2023-10-06T05:16:10.205427+0000","last_unstale":"2023-10-06T05:16:10.214591+0000","last_undegraded":"2023-10-06T05:16:10.214591+0000","last_fullsized":"2023-10-06T05:16:10.214591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:08.939753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.212724+0000","last_change":"2023-10-06T05:16:10.210714+0000","last_active":"2023-10-06T05:16:10.212724+0000","last_peered":"2023-10-06T05:16:10.212724+0000","last_clean":"2023-10-06T05:16:10.212724+0000","last_became_active":"2023-10-06T05:16:10.210422+0000","last_became_peered":"2023-10-06T05:16:10.210422+0000","last_unstale":"2023-10-06T05:16:10.212724+0000","last_undegraded":"2023-10-06T05:16:10.212724+0000","last_fullsized":"2023-10-06T05:16:10.212724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:23.294418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218652+0000","last_change":"2023-10-06T05:16:10.211103+0000","last_active":"2023-10-06T05:16:10.218652+0000","last_peered":"2023-10-06T05:16:10.218652+0000","last_clean":"2023-10-06T05:16:10.218652+0000","last_became_active":"2023-10-06T05:16:10.210541+0000","last_became_peered":"2023-10-06T05:16:10.210541+0000","last_unstale":"2023-10-06T05:16:10.218652+0000","last_undegraded":"2023-10-06T05:16:10.218652+0000","last_fullsized":"2023-10-06T05:16:10.218652+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:26.802163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217818+0000","last_change":"2023-10-06T05:16:10.210944+0000","last_active":"2023-10-06T05:16:10.217818+0000","last_peered":"2023-10-06T05:16:10.217818+0000","last_clean":"2023-10-06T05:16:10.217818+0000","last_became_active":"2023-10-06T05:16:10.210711+0000","last_became_peered":"2023-10-06T05:16:10.210711+0000","last_unstale":"2023-10-06T05:16:10.217818+0000","last_undegraded":"2023-10-06T05:16:10.217818+0000","last_fullsized":"2023-10-06T05:16:10.217818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:37.873768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.223455+0000","last_change":"2023-10-06T05:16:10.218500+0000","last_active":"2023-10-06T05:16:10.223455+0000","last_peered":"2023-10-06T05:16:10.223455+0000","last_clean":"2023-10-06T05:16:10.223455+0000","last_became_active":"2023-10-06T05:16:10.217901+0000","last_became_peered":"2023-10-06T05:16:10.217901+0000","last_unstale":"2023-10-06T05:16:10.223455+0000","last_undegraded":"2023-10-06T05:16:10.223455+0000","last_fullsized":"2023-10-06T05:16:10.223455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:10.767004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226816+0000","last_change":"2023-10-06T05:16:10.221609+0000","last_active":"2023-10-06T05:16:10.226816+0000","last_peered":"2023-10-06T05:16:10.226816+0000","last_clean":"2023-10-06T05:16:10.226816+0000","last_became_active":"2023-10-06T05:16:10.221341+0000","last_became_peered":"2023-10-06T05:16:10.221341+0000","last_unstale":"2023-10-06T05:16:10.226816+0000","last_undegraded":"2023-10-06T05:16:10.226816+0000","last_fullsized":"2023-10-06T05:16:10.226816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:53.442186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219193+0000","last_change":"2023-10-06T05:16:10.211181+0000","last_active":"2023-10-06T05:16:10.219193+0000","last_peered":"2023-10-06T05:16:10.219193+0000","last_clean":"2023-10-06T05:16:10.219193+0000","last_became_active":"2023-10-06T05:16:10.210229+0000","last_became_peered":"2023-10-06T05:16:10.210229+0000","last_unstale":"2023-10-06T05:16:10.219193+0000","last_undegraded":"2023-10-06T05:16:10.219193+0000","last_fullsized":"2023-10-06T05:16:10.219193+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:07.100657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.220633+0000","last_change":"2023-10-06T05:16:10.216135+0000","last_active":"2023-10-06T05:16:10.220633+0000","last_peered":"2023-10-06T05:16:10.220633+0000","last_clean":"2023-10-06T05:16:10.220633+0000","last_became_active":"2023-10-06T05:16:10.215870+0000","last_became_peered":"2023-10-06T05:16:10.215870+0000","last_unstale":"2023-10-06T05:16:10.220633+0000","last_undegraded":"2023-10-06T05:16:10.220633+0000","last_fullsized":"2023-10-06T05:16:10.220633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:49.253876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221180+0000","last_change":"2023-10-06T05:16:10.211857+0000","last_active":"2023-10-06T05:16:10.221180+0000","last_peered":"2023-10-06T05:16:10.221180+0000","last_clean":"2023-10-06T05:16:10.221180+0000","last_became_active":"2023-10-06T05:16:10.211214+0000","last_became_peered":"2023-10-06T05:16:10.211214+0000","last_unstale":"2023-10-06T05:16:10.221180+0000","last_undegraded":"2023-10-06T05:16:10.221180+0000","last_fullsized":"2023-10-06T05:16:10.221180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:08.939753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229694+0000","last_change":"2023-10-06T05:16:10.228978+0000","last_active":"2023-10-06T05:16:10.229694+0000","last_peered":"2023-10-06T05:16:10.229694+0000","last_clean":"2023-10-06T05:16:10.229694+0000","last_became_active":"2023-10-06T05:16:10.228644+0000","last_became_peered":"2023-10-06T05:16:10.228644+0000","last_unstale":"2023-10-06T05:16:10.229694+0000","last_undegraded":"2023-10-06T05:16:10.229694+0000","last_fullsized":"2023-10-06T05:16:10.229694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:10.767004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226664+0000","last_change":"2023-10-06T05:16:10.223067+0000","last_active":"2023-10-06T05:16:10.226664+0000","last_peered":"2023-10-06T05:16:10.226664+0000","last_clean":"2023-10-06T05:16:10.226664+0000","last_became_active":"2023-10-06T05:16:10.222825+0000","last_became_peered":"2023-10-06T05:16:10.222825+0000","last_unstale":"2023-10-06T05:16:10.226664+0000","last_undegraded":"2023-10-06T05:16:10.226664+0000","last_fullsized":"2023-10-06T05:16:10.226664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:07.636440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.230450+0000","last_change":"2023-10-06T05:16:10.229864+0000","last_active":"2023-10-06T05:16:10.230450+0000","last_peered":"2023-10-06T05:16:10.230450+0000","last_clean":"2023-10-06T05:16:10.230450+0000","last_became_active":"2023-10-06T05:16:10.229548+0000","last_became_peered":"2023-10-06T05:16:10.229548+0000","last_unstale":"2023-10-06T05:16:10.230450+0000","last_undegraded":"2023-10-06T05:16:10.230450+0000","last_fullsized":"2023-10-06T05:16:10.230450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.508964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221099+0000","last_change":"2023-10-06T05:16:10.215942+0000","last_active":"2023-10-06T05:16:10.221099+0000","last_peered":"2023-10-06T05:16:10.221099+0000","last_clean":"2023-10-06T05:16:10.221099+0000","last_became_active":"2023-10-06T05:16:10.215609+0000","last_became_peered":"2023-10-06T05:16:10.215609+0000","last_unstale":"2023-10-06T05:16:10.221099+0000","last_undegraded":"2023-10-06T05:16:10.221099+0000","last_fullsized":"2023-10-06T05:16:10.221099+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:32.849654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.223969+0000","last_change":"2023-10-06T05:16:10.218189+0000","last_active":"2023-10-06T05:16:10.223969+0000","last_peered":"2023-10-06T05:16:10.223969+0000","last_clean":"2023-10-06T05:16:10.223969+0000","last_became_active":"2023-10-06T05:16:10.217917+0000","last_became_peered":"2023-10-06T05:16:10.217917+0000","last_unstale":"2023-10-06T05:16:10.223969+0000","last_undegraded":"2023-10-06T05:16:10.223969+0000","last_fullsized":"2023-10-06T05:16:10.223969+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:49.253876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214896+0000","last_change":"2023-10-06T05:16:10.207619+0000","last_active":"2023-10-06T05:16:10.214896+0000","last_peered":"2023-10-06T05:16:10.214896+0000","last_clean":"2023-10-06T05:16:10.214896+0000","last_became_active":"2023-10-06T05:16:10.207336+0000","last_became_peered":"2023-10-06T05:16:10.207336+0000","last_unstale":"2023-10-06T05:16:10.214896+0000","last_undegraded":"2023-10-06T05:16:10.214896+0000","last_fullsized":"2023-10-06T05:16:10.214896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:40.277272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210832+0000","last_change":"2023-10-06T05:16:10.203334+0000","last_active":"2023-10-06T05:16:10.210832+0000","last_peered":"2023-10-06T05:16:10.210832+0000","last_clean":"2023-10-06T05:16:10.210832+0000","last_became_active":"2023-10-06T05:16:10.203058+0000","last_became_peered":"2023-10-06T05:16:10.203058+0000","last_unstale":"2023-10-06T05:16:10.210832+0000","last_undegraded":"2023-10-06T05:16:10.210832+0000","last_fullsized":"2023-10-06T05:16:10.210832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:10.767004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217106+0000","last_change":"2023-10-06T05:16:10.211236+0000","last_active":"2023-10-06T05:16:10.217106+0000","last_peered":"2023-10-06T05:16:10.217106+0000","last_clean":"2023-10-06T05:16:10.217106+0000","last_became_active":"2023-10-06T05:16:10.210983+0000","last_became_peered":"2023-10-06T05:16:10.210983+0000","last_unstale":"2023-10-06T05:16:10.217106+0000","last_undegraded":"2023-10-06T05:16:10.217106+0000","last_fullsized":"2023-10-06T05:16:10.217106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:24.430630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217702+0000","last_change":"2023-10-06T05:16:10.209619+0000","last_active":"2023-10-06T05:16:10.217702+0000","last_peered":"2023-10-06T05:16:10.217702+0000","last_clean":"2023-10-06T05:16:10.217702+0000","last_became_active":"2023-10-06T05:16:10.209359+0000","last_became_peered":"2023-10-06T05:16:10.209359+0000","last_unstale":"2023-10-06T05:16:10.217702+0000","last_undegraded":"2023-10-06T05:16:10.217702+0000","last_fullsized":"2023-10-06T05:16:10.217702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:57.452379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.225675+0000","last_change":"2023-10-06T05:16:10.223579+0000","last_active":"2023-10-06T05:16:10.225675+0000","last_peered":"2023-10-06T05:16:10.225675+0000","last_clean":"2023-10-06T05:16:10.225675+0000","last_became_active":"2023-10-06T05:16:10.223087+0000","last_became_peered":"2023-10-06T05:16:10.223087+0000","last_unstale":"2023-10-06T05:16:10.225675+0000","last_undegraded":"2023-10-06T05:16:10.225675+0000","last_fullsized":"2023-10-06T05:16:10.225675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:40.277272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229836+0000","last_change":"2023-10-06T05:16:10.227241+0000","last_active":"2023-10-06T05:16:10.229836+0000","last_peered":"2023-10-06T05:16:10.229836+0000","last_clean":"2023-10-06T05:16:10.229836+0000","last_became_active":"2023-10-06T05:16:10.226920+0000","last_became_peered":"2023-10-06T05:16:10.226920+0000","last_unstale":"2023-10-06T05:16:10.229836+0000","last_undegraded":"2023-10-06T05:16:10.229836+0000","last_fullsized":"2023-10-06T05:16:10.229836+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:05.613161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.212461+0000","last_change":"2023-10-06T05:16:10.209348+0000","last_active":"2023-10-06T05:16:10.212461+0000","last_peered":"2023-10-06T05:16:10.212461+0000","last_clean":"2023-10-06T05:16:10.212461+0000","last_became_active":"2023-10-06T05:16:10.208951+0000","last_became_peered":"2023-10-06T05:16:10.208951+0000","last_unstale":"2023-10-06T05:16:10.212461+0000","last_undegraded":"2023-10-06T05:16:10.212461+0000","last_fullsized":"2023-10-06T05:16:10.212461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:35.224737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218019+0000","last_change":"2023-10-06T05:16:10.212964+0000","last_active":"2023-10-06T05:16:10.218019+0000","last_peered":"2023-10-06T05:16:10.218019+0000","last_clean":"2023-10-06T05:16:10.218019+0000","last_became_active":"2023-10-06T05:16:10.212734+0000","last_became_peered":"2023-10-06T05:16:10.212734+0000","last_unstale":"2023-10-06T05:16:10.218019+0000","last_undegraded":"2023-10-06T05:16:10.218019+0000","last_fullsized":"2023-10-06T05:16:10.218019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:37.104161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217950+0000","last_change":"2023-10-06T05:16:10.213013+0000","last_active":"2023-10-06T05:16:10.217950+0000","last_peered":"2023-10-06T05:16:10.217950+0000","last_clean":"2023-10-06T05:16:10.217950+0000","last_became_active":"2023-10-06T05:16:10.212728+0000","last_became_peered":"2023-10-06T05:16:10.212728+0000","last_unstale":"2023-10-06T05:16:10.217950+0000","last_undegraded":"2023-10-06T05:16:10.217950+0000","last_fullsized":"2023-10-06T05:16:10.217950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:24.751361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.223684+0000","last_change":"2023-10-06T05:16:10.220394+0000","last_active":"2023-10-06T05:16:10.223684+0000","last_peered":"2023-10-06T05:16:10.223684+0000","last_clean":"2023-10-06T05:16:10.223684+0000","last_became_active":"2023-10-06T05:16:10.220151+0000","last_became_peered":"2023-10-06T05:16:10.220151+0000","last_unstale":"2023-10-06T05:16:10.223684+0000","last_undegraded":"2023-10-06T05:16:10.223684+0000","last_fullsized":"2023-10-06T05:16:10.223684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:34.955272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229100+0000","last_change":"2023-10-06T05:16:10.226657+0000","last_active":"2023-10-06T05:16:10.229100+0000","last_peered":"2023-10-06T05:16:10.229100+0000","last_clean":"2023-10-06T05:16:10.229100+0000","last_became_active":"2023-10-06T05:16:10.226355+0000","last_became_peered":"2023-10-06T05:16:10.226355+0000","last_unstale":"2023-10-06T05:16:10.229100+0000","last_undegraded":"2023-10-06T05:16:10.229100+0000","last_fullsized":"2023-10-06T05:16:10.229100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:43.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210157+0000","last_change":"2023-10-06T05:16:10.203310+0000","last_active":"2023-10-06T05:16:10.210157+0000","last_peered":"2023-10-06T05:16:10.210157+0000","last_clean":"2023-10-06T05:16:10.210157+0000","last_became_active":"2023-10-06T05:16:10.203011+0000","last_became_peered":"2023-10-06T05:16:10.203011+0000","last_unstale":"2023-10-06T05:16:10.210157+0000","last_undegraded":"2023-10-06T05:16:10.210157+0000","last_fullsized":"2023-10-06T05:16:10.210157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:45.385805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226639+0000","last_change":"2023-10-06T05:16:10.222909+0000","last_active":"2023-10-06T05:16:10.226639+0000","last_peered":"2023-10-06T05:16:10.226639+0000","last_clean":"2023-10-06T05:16:10.226639+0000","last_became_active":"2023-10-06T05:16:10.222662+0000","last_became_peered":"2023-10-06T05:16:10.222662+0000","last_unstale":"2023-10-06T05:16:10.226639+0000","last_undegraded":"2023-10-06T05:16:10.226639+0000","last_fullsized":"2023-10-06T05:16:10.226639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:26.802163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229949+0000","last_change":"2023-10-06T05:16:10.226872+0000","last_active":"2023-10-06T05:16:10.229949+0000","last_peered":"2023-10-06T05:16:10.229949+0000","last_clean":"2023-10-06T05:16:10.229949+0000","last_became_active":"2023-10-06T05:16:10.226600+0000","last_became_peered":"2023-10-06T05:16:10.226600+0000","last_unstale":"2023-10-06T05:16:10.229949+0000","last_undegraded":"2023-10-06T05:16:10.229949+0000","last_fullsized":"2023-10-06T05:16:10.229949+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:38.594765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211415+0000","last_change":"2023-10-06T05:16:10.205229+0000","last_active":"2023-10-06T05:16:10.211415+0000","last_peered":"2023-10-06T05:16:10.211415+0000","last_clean":"2023-10-06T05:16:10.211415+0000","last_became_active":"2023-10-06T05:16:10.204948+0000","last_became_peered":"2023-10-06T05:16:10.204948+0000","last_unstale":"2023-10-06T05:16:10.211415+0000","last_undegraded":"2023-10-06T05:16:10.211415+0000","last_fullsized":"2023-10-06T05:16:10.211415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:19.783131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226176+0000","last_change":"2023-10-06T05:16:10.223615+0000","last_active":"2023-10-06T05:16:10.226176+0000","last_peered":"2023-10-06T05:16:10.226176+0000","last_clean":"2023-10-06T05:16:10.226176+0000","last_became_active":"2023-10-06T05:16:10.220914+0000","last_became_peered":"2023-10-06T05:16:10.220914+0000","last_unstale":"2023-10-06T05:16:10.226176+0000","last_undegraded":"2023-10-06T05:16:10.226176+0000","last_fullsized":"2023-10-06T05:16:10.226176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:53.442186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229145+0000","last_change":"2023-10-06T05:16:10.226881+0000","last_active":"2023-10-06T05:16:10.229145+0000","last_peered":"2023-10-06T05:16:10.229145+0000","last_clean":"2023-10-06T05:16:10.229145+0000","last_became_active":"2023-10-06T05:16:10.226612+0000","last_became_peered":"2023-10-06T05:16:10.226612+0000","last_unstale":"2023-10-06T05:16:10.229145+0000","last_undegraded":"2023-10-06T05:16:10.229145+0000","last_fullsized":"2023-10-06T05:16:10.229145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:39.377431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229186+0000","last_change":"2023-10-06T05:16:10.226299+0000","last_active":"2023-10-06T05:16:10.229186+0000","last_peered":"2023-10-06T05:16:10.229186+0000","last_clean":"2023-10-06T05:16:10.229186+0000","last_became_active":"2023-10-06T05:16:10.221517+0000","last_became_peered":"2023-10-06T05:16:10.221517+0000","last_unstale":"2023-10-06T05:16:10.229186+0000","last_undegraded":"2023-10-06T05:16:10.229186+0000","last_fullsized":"2023-10-06T05:16:10.229186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:28.570142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.224001+0000","last_change":"2023-10-06T05:16:09.212356+0000","last_active":"2023-10-06T05:16:10.224001+0000","last_peered":"2023-10-06T05:16:10.224001+0000","last_clean":"2023-10-06T05:16:10.224001+0000","last_became_active":"2023-10-06T05:16:09.211824+0000","last_became_peered":"2023-10-06T05:16:09.211824+0000","last_unstale":"2023-10-06T05:16:10.224001+0000","last_undegraded":"2023-10-06T05:16:10.224001+0000","last_fullsized":"2023-10-06T05:16:10.224001+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:42.471127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217894+0000","last_change":"2023-10-06T05:16:10.212385+0000","last_active":"2023-10-06T05:16:10.217894+0000","last_peered":"2023-10-06T05:16:10.217894+0000","last_clean":"2023-10-06T05:16:10.217894+0000","last_became_active":"2023-10-06T05:16:10.212100+0000","last_became_peered":"2023-10-06T05:16:10.212100+0000","last_unstale":"2023-10-06T05:16:10.217894+0000","last_undegraded":"2023-10-06T05:16:10.217894+0000","last_fullsized":"2023-10-06T05:16:10.217894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:43.801764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210468+0000","last_change":"2023-10-06T05:16:09.210776+0000","last_active":"2023-10-06T05:16:10.210468+0000","last_peered":"2023-10-06T05:16:10.210468+0000","last_clean":"2023-10-06T05:16:10.210468+0000","last_became_active":"2023-10-06T05:16:09.210484+0000","last_became_peered":"2023-10-06T05:16:09.210484+0000","last_unstale":"2023-10-06T05:16:10.210468+0000","last_undegraded":"2023-10-06T05:16:10.210468+0000","last_fullsized":"2023-10-06T05:16:10.210468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:31.844638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.223699+0000","last_change":"2023-10-06T05:16:10.219330+0000","last_active":"2023-10-06T05:16:10.223699+0000","last_peered":"2023-10-06T05:16:10.223699+0000","last_clean":"2023-10-06T05:16:10.223699+0000","last_became_active":"2023-10-06T05:16:10.219092+0000","last_became_peered":"2023-10-06T05:16:10.219092+0000","last_unstale":"2023-10-06T05:16:10.223699+0000","last_undegraded":"2023-10-06T05:16:10.223699+0000","last_fullsized":"2023-10-06T05:16:10.223699+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:13.881962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229325+0000","last_change":"2023-10-06T05:16:09.211344+0000","last_active":"2023-10-06T05:16:10.229325+0000","last_peered":"2023-10-06T05:16:10.229325+0000","last_clean":"2023-10-06T05:16:10.229325+0000","last_became_active":"2023-10-06T05:16:09.211052+0000","last_became_peered":"2023-10-06T05:16:09.211052+0000","last_unstale":"2023-10-06T05:16:10.229325+0000","last_undegraded":"2023-10-06T05:16:10.229325+0000","last_fullsized":"2023-10-06T05:16:10.229325+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:43.354681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229379+0000","last_change":"2023-10-06T05:16:10.226547+0000","last_active":"2023-10-06T05:16:10.229379+0000","last_peered":"2023-10-06T05:16:10.229379+0000","last_clean":"2023-10-06T05:16:10.229379+0000","last_became_active":"2023-10-06T05:16:10.220212+0000","last_became_peered":"2023-10-06T05:16:10.220212+0000","last_unstale":"2023-10-06T05:16:10.229379+0000","last_undegraded":"2023-10-06T05:16:10.229379+0000","last_fullsized":"2023-10-06T05:16:10.229379+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.508964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217935+0000","last_change":"2023-10-06T05:16:09.211932+0000","last_active":"2023-10-06T05:16:10.217935+0000","last_peered":"2023-10-06T05:16:10.217935+0000","last_clean":"2023-10-06T05:16:10.217935+0000","last_became_active":"2023-10-06T05:16:09.211687+0000","last_became_peered":"2023-10-06T05:16:09.211687+0000","last_unstale":"2023-10-06T05:16:10.217935+0000","last_undegraded":"2023-10-06T05:16:10.217935+0000","last_fullsized":"2023-10-06T05:16:10.217935+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:11.549913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211434+0000","last_change":"2023-10-06T05:16:10.209702+0000","last_active":"2023-10-06T05:16:10.211434+0000","last_peered":"2023-10-06T05:16:10.211434+0000","last_clean":"2023-10-06T05:16:10.211434+0000","last_became_active":"2023-10-06T05:16:10.209390+0000","last_became_peered":"2023-10-06T05:16:10.209390+0000","last_unstale":"2023-10-06T05:16:10.211434+0000","last_undegraded":"2023-10-06T05:16:10.211434+0000","last_fullsized":"2023-10-06T05:16:10.211434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:42.845785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226611+0000","last_change":"2023-10-06T05:16:09.215808+0000","last_active":"2023-10-06T05:16:10.226611+0000","last_peered":"2023-10-06T05:16:10.226611+0000","last_clean":"2023-10-06T05:16:10.226611+0000","last_became_active":"2023-10-06T05:16:09.213426+0000","last_became_peered":"2023-10-06T05:16:09.213426+0000","last_unstale":"2023-10-06T05:16:10.226611+0000","last_undegraded":"2023-10-06T05:16:10.226611+0000","last_fullsized":"2023-10-06T05:16:10.226611+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:28:11.549913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219726+0000","last_change":"2023-10-06T05:16:10.212070+0000","last_active":"2023-10-06T05:16:10.219726+0000","last_peered":"2023-10-06T05:16:10.219726+0000","last_clean":"2023-10-06T05:16:10.219726+0000","last_became_active":"2023-10-06T05:16:10.211782+0000","last_became_peered":"2023-10-06T05:16:10.211782+0000","last_unstale":"2023-10-06T05:16:10.219726+0000","last_undegraded":"2023-10-06T05:16:10.219726+0000","last_fullsized":"2023-10-06T05:16:10.219726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:35.024962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210095+0000","last_change":"2023-10-06T05:16:09.211131+0000","last_active":"2023-10-06T05:16:10.210095+0000","last_peered":"2023-10-06T05:16:10.210095+0000","last_clean":"2023-10-06T05:16:10.210095+0000","last_became_active":"2023-10-06T05:16:09.210879+0000","last_became_peered":"2023-10-06T05:16:09.210879+0000","last_unstale":"2023-10-06T05:16:10.210095+0000","last_undegraded":"2023-10-06T05:16:10.210095+0000","last_fullsized":"2023-10-06T05:16:10.210095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:06.631425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.207977+0000","last_change":"2023-10-06T05:16:10.205445+0000","last_active":"2023-10-06T05:16:10.207977+0000","last_peered":"2023-10-06T05:16:10.207977+0000","last_clean":"2023-10-06T05:16:10.207977+0000","last_became_active":"2023-10-06T05:16:10.205159+0000","last_became_peered":"2023-10-06T05:16:10.205159+0000","last_unstale":"2023-10-06T05:16:10.207977+0000","last_undegraded":"2023-10-06T05:16:10.207977+0000","last_fullsized":"2023-10-06T05:16:10.207977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:23.294418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193687+0000","last_change":"2023-10-06T05:16:09.204336+0000","last_active":"2023-10-06T05:16:10.193687+0000","last_peered":"2023-10-06T05:16:10.193687+0000","last_clean":"2023-10-06T05:16:10.193687+0000","last_became_active":"2023-10-06T05:16:09.204059+0000","last_became_peered":"2023-10-06T05:16:09.204059+0000","last_unstale":"2023-10-06T05:16:10.193687+0000","last_undegraded":"2023-10-06T05:16:10.193687+0000","last_fullsized":"2023-10-06T05:16:10.193687+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:42.471127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218002+0000","last_change":"2023-10-06T05:16:10.210141+0000","last_active":"2023-10-06T05:16:10.218002+0000","last_peered":"2023-10-06T05:16:10.218002+0000","last_clean":"2023-10-06T05:16:10.218002+0000","last_became_active":"2023-10-06T05:16:10.208711+0000","last_became_peered":"2023-10-06T05:16:10.208711+0000","last_unstale":"2023-10-06T05:16:10.218002+0000","last_undegraded":"2023-10-06T05:16:10.218002+0000","last_fullsized":"2023-10-06T05:16:10.218002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:21.353918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.198161+0000","last_change":"2023-10-06T05:16:09.212018+0000","last_active":"2023-10-06T05:16:10.198161+0000","last_peered":"2023-10-06T05:16:10.198161+0000","last_clean":"2023-10-06T05:16:10.198161+0000","last_became_active":"2023-10-06T05:16:09.211333+0000","last_became_peered":"2023-10-06T05:16:09.211333+0000","last_unstale":"2023-10-06T05:16:10.198161+0000","last_undegraded":"2023-10-06T05:16:10.198161+0000","last_fullsized":"2023-10-06T05:16:10.198161+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:38.372416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.217287+0000","last_change":"2023-10-06T05:16:10.210825+0000","last_active":"2023-10-06T05:16:10.217287+0000","last_peered":"2023-10-06T05:16:10.217287+0000","last_clean":"2023-10-06T05:16:10.217287+0000","last_became_active":"2023-10-06T05:16:10.210544+0000","last_became_peered":"2023-10-06T05:16:10.210544+0000","last_unstale":"2023-10-06T05:16:10.217287+0000","last_undegraded":"2023-10-06T05:16:10.217287+0000","last_fullsized":"2023-10-06T05:16:10.217287+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:35.024962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.194295+0000","last_change":"2023-10-06T05:16:09.207532+0000","last_active":"2023-10-06T05:16:10.194295+0000","last_peered":"2023-10-06T05:16:10.194295+0000","last_clean":"2023-10-06T05:16:10.194295+0000","last_became_active":"2023-10-06T05:16:09.207231+0000","last_became_peered":"2023-10-06T05:16:09.207231+0000","last_unstale":"2023-10-06T05:16:10.194295+0000","last_undegraded":"2023-10-06T05:16:10.194295+0000","last_fullsized":"2023-10-06T05:16:10.194295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:34.219722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218856+0000","last_change":"2023-10-06T05:16:10.216209+0000","last_active":"2023-10-06T05:16:10.218856+0000","last_peered":"2023-10-06T05:16:10.218856+0000","last_clean":"2023-10-06T05:16:10.218856+0000","last_became_active":"2023-10-06T05:16:10.215963+0000","last_became_peered":"2023-10-06T05:16:10.215963+0000","last_unstale":"2023-10-06T05:16:10.218856+0000","last_undegraded":"2023-10-06T05:16:10.218856+0000","last_fullsized":"2023-10-06T05:16:10.218856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:55.064969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.201998+0000","last_change":"2023-10-06T05:16:09.208953+0000","last_active":"2023-10-06T05:16:10.201998+0000","last_peered":"2023-10-06T05:16:10.201998+0000","last_clean":"2023-10-06T05:16:10.201998+0000","last_became_active":"2023-10-06T05:16:09.208620+0000","last_became_peered":"2023-10-06T05:16:09.208620+0000","last_unstale":"2023-10-06T05:16:10.201998+0000","last_undegraded":"2023-10-06T05:16:10.201998+0000","last_fullsized":"2023-10-06T05:16:10.201998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:43.354681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229969+0000","last_change":"2023-10-06T05:16:10.227628+0000","last_active":"2023-10-06T05:16:10.229969+0000","last_peered":"2023-10-06T05:16:10.229969+0000","last_clean":"2023-10-06T05:16:10.229969+0000","last_became_active":"2023-10-06T05:16:10.227345+0000","last_became_peered":"2023-10-06T05:16:10.227345+0000","last_unstale":"2023-10-06T05:16:10.229969+0000","last_undegraded":"2023-10-06T05:16:10.229969+0000","last_fullsized":"2023-10-06T05:16:10.229969+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:45.385805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218333+0000","last_change":"2023-10-06T05:16:09.208431+0000","last_active":"2023-10-06T05:16:10.218333+0000","last_peered":"2023-10-06T05:16:10.218333+0000","last_clean":"2023-10-06T05:16:10.218333+0000","last_became_active":"2023-10-06T05:16:09.208186+0000","last_became_peered":"2023-10-06T05:16:09.208186+0000","last_unstale":"2023-10-06T05:16:10.218333+0000","last_undegraded":"2023-10-06T05:16:10.218333+0000","last_fullsized":"2023-10-06T05:16:10.218333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:34.219722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.230146+0000","last_change":"2023-10-06T05:16:10.227538+0000","last_active":"2023-10-06T05:16:10.230146+0000","last_peered":"2023-10-06T05:16:10.230146+0000","last_clean":"2023-10-06T05:16:10.230146+0000","last_became_active":"2023-10-06T05:16:10.227298+0000","last_became_peered":"2023-10-06T05:16:10.227298+0000","last_unstale":"2023-10-06T05:16:10.230146+0000","last_undegraded":"2023-10-06T05:16:10.230146+0000","last_fullsized":"2023-10-06T05:16:10.230146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:54.010782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.230626+0000","last_change":"2023-10-06T05:16:09.206717+0000","last_active":"2023-10-06T05:16:10.230626+0000","last_peered":"2023-10-06T05:16:10.230626+0000","last_clean":"2023-10-06T05:16:10.230626+0000","last_became_active":"2023-10-06T05:16:09.206406+0000","last_became_peered":"2023-10-06T05:16:09.206406+0000","last_unstale":"2023-10-06T05:16:10.230626+0000","last_undegraded":"2023-10-06T05:16:10.230626+0000","last_fullsized":"2023-10-06T05:16:10.230626+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:48.306767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214816+0000","last_change":"2023-10-06T05:16:10.207594+0000","last_active":"2023-10-06T05:16:10.214816+0000","last_peered":"2023-10-06T05:16:10.214816+0000","last_clean":"2023-10-06T05:16:10.214816+0000","last_became_active":"2023-10-06T05:16:10.207287+0000","last_became_peered":"2023-10-06T05:16:10.207287+0000","last_unstale":"2023-10-06T05:16:10.214816+0000","last_undegraded":"2023-10-06T05:16:10.214816+0000","last_fullsized":"2023-10-06T05:16:10.214816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:56.963759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.229905+0000","last_change":"2023-10-06T05:16:09.214314+0000","last_active":"2023-10-06T05:16:10.229905+0000","last_peered":"2023-10-06T05:16:10.229905+0000","last_clean":"2023-10-06T05:16:10.229905+0000","last_became_active":"2023-10-06T05:16:09.213879+0000","last_became_peered":"2023-10-06T05:16:09.213879+0000","last_unstale":"2023-10-06T05:16:10.229905+0000","last_undegraded":"2023-10-06T05:16:10.229905+0000","last_fullsized":"2023-10-06T05:16:10.229905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:59.815874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219750+0000","last_change":"2023-10-06T05:16:10.213380+0000","last_active":"2023-10-06T05:16:10.219750+0000","last_peered":"2023-10-06T05:16:10.219750+0000","last_clean":"2023-10-06T05:16:10.219750+0000","last_became_active":"2023-10-06T05:16:10.212923+0000","last_became_peered":"2023-10-06T05:16:10.212923+0000","last_unstale":"2023-10-06T05:16:10.219750+0000","last_undegraded":"2023-10-06T05:16:10.219750+0000","last_fullsized":"2023-10-06T05:16:10.219750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:19.783131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.201703+0000","last_change":"2023-10-06T05:16:09.204850+0000","last_active":"2023-10-06T05:16:10.201703+0000","last_peered":"2023-10-06T05:16:10.201703+0000","last_clean":"2023-10-06T05:16:10.201703+0000","last_became_active":"2023-10-06T05:16:09.204553+0000","last_became_peered":"2023-10-06T05:16:09.204553+0000","last_unstale":"2023-10-06T05:16:10.201703+0000","last_undegraded":"2023-10-06T05:16:10.201703+0000","last_fullsized":"2023-10-06T05:16:10.201703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:27.565127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.222595+0000","last_change":"2023-10-06T05:16:10.216415+0000","last_active":"2023-10-06T05:16:10.222595+0000","last_peered":"2023-10-06T05:16:10.222595+0000","last_clean":"2023-10-06T05:16:10.222595+0000","last_became_active":"2023-10-06T05:16:10.216095+0000","last_became_peered":"2023-10-06T05:16:10.216095+0000","last_unstale":"2023-10-06T05:16:10.222595+0000","last_undegraded":"2023-10-06T05:16:10.222595+0000","last_fullsized":"2023-10-06T05:16:10.222595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:23.294418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.203441+0000","last_change":"2023-10-06T05:16:09.210209+0000","last_active":"2023-10-06T05:16:10.203441+0000","last_peered":"2023-10-06T05:16:10.203441+0000","last_clean":"2023-10-06T05:16:10.203441+0000","last_became_active":"2023-10-06T05:16:09.209922+0000","last_became_peered":"2023-10-06T05:16:09.209922+0000","last_unstale":"2023-10-06T05:16:10.203441+0000","last_undegraded":"2023-10-06T05:16:10.203441+0000","last_fullsized":"2023-10-06T05:16:10.203441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:31.844638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218851+0000","last_change":"2023-10-06T05:16:10.217668+0000","last_active":"2023-10-06T05:16:10.218851+0000","last_peered":"2023-10-06T05:16:10.218851+0000","last_clean":"2023-10-06T05:16:10.218851+0000","last_became_active":"2023-10-06T05:16:10.217409+0000","last_became_peered":"2023-10-06T05:16:10.217409+0000","last_unstale":"2023-10-06T05:16:10.218851+0000","last_undegraded":"2023-10-06T05:16:10.218851+0000","last_fullsized":"2023-10-06T05:16:10.218851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:37.873768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219278+0000","last_change":"2023-10-06T05:16:09.213428+0000","last_active":"2023-10-06T05:16:10.219278+0000","last_peered":"2023-10-06T05:16:10.219278+0000","last_clean":"2023-10-06T05:16:10.219278+0000","last_became_active":"2023-10-06T05:16:09.213153+0000","last_became_peered":"2023-10-06T05:16:09.213153+0000","last_unstale":"2023-10-06T05:16:10.219278+0000","last_undegraded":"2023-10-06T05:16:10.219278+0000","last_fullsized":"2023-10-06T05:16:10.219278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:39.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221604+0000","last_change":"2023-10-06T05:16:10.218301+0000","last_active":"2023-10-06T05:16:10.221604+0000","last_peered":"2023-10-06T05:16:10.221604+0000","last_clean":"2023-10-06T05:16:10.221604+0000","last_became_active":"2023-10-06T05:16:10.218018+0000","last_became_peered":"2023-10-06T05:16:10.218018+0000","last_unstale":"2023-10-06T05:16:10.221604+0000","last_undegraded":"2023-10-06T05:16:10.221604+0000","last_fullsized":"2023-10-06T05:16:10.221604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:30.897795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226641+0000","last_change":"2023-10-06T05:16:09.208223+0000","last_active":"2023-10-06T05:16:10.226641+0000","last_peered":"2023-10-06T05:16:10.226641+0000","last_clean":"2023-10-06T05:16:10.226641+0000","last_became_active":"2023-10-06T05:16:09.207913+0000","last_became_peered":"2023-10-06T05:16:09.207913+0000","last_unstale":"2023-10-06T05:16:10.226641+0000","last_undegraded":"2023-10-06T05:16:10.226641+0000","last_fullsized":"2023-10-06T05:16:10.226641+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:23.746345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221153+0000","last_change":"2023-10-06T05:16:10.220179+0000","last_active":"2023-10-06T05:16:10.221153+0000","last_peered":"2023-10-06T05:16:10.221153+0000","last_clean":"2023-10-06T05:16:10.221153+0000","last_became_active":"2023-10-06T05:16:10.219906+0000","last_became_peered":"2023-10-06T05:16:10.219906+0000","last_unstale":"2023-10-06T05:16:10.221153+0000","last_undegraded":"2023-10-06T05:16:10.221153+0000","last_fullsized":"2023-10-06T05:16:10.221153+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:05.613161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.210393+0000","last_change":"2023-10-06T05:16:09.207565+0000","last_active":"2023-10-06T05:16:10.210393+0000","last_peered":"2023-10-06T05:16:10.210393+0000","last_clean":"2023-10-06T05:16:10.210393+0000","last_became_active":"2023-10-06T05:16:09.207265+0000","last_became_peered":"2023-10-06T05:16:09.207265+0000","last_unstale":"2023-10-06T05:16:10.210393+0000","last_undegraded":"2023-10-06T05:16:10.210393+0000","last_fullsized":"2023-10-06T05:16:10.210393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:07.934738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226672+0000","last_change":"2023-10-06T05:16:10.221570+0000","last_active":"2023-10-06T05:16:10.226672+0000","last_peered":"2023-10-06T05:16:10.226672+0000","last_clean":"2023-10-06T05:16:10.226672+0000","last_became_active":"2023-10-06T05:16:10.221389+0000","last_became_peered":"2023-10-06T05:16:10.221389+0000","last_unstale":"2023-10-06T05:16:10.226672+0000","last_undegraded":"2023-10-06T05:16:10.226672+0000","last_fullsized":"2023-10-06T05:16:10.226672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:30.897795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211334+0000","last_change":"2023-10-06T05:16:09.204770+0000","last_active":"2023-10-06T05:16:10.211334+0000","last_peered":"2023-10-06T05:16:10.211334+0000","last_clean":"2023-10-06T05:16:10.211334+0000","last_became_active":"2023-10-06T05:16:09.204549+0000","last_became_peered":"2023-10-06T05:16:09.204549+0000","last_unstale":"2023-10-06T05:16:10.211334+0000","last_undegraded":"2023-10-06T05:16:10.211334+0000","last_fullsized":"2023-10-06T05:16:10.211334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:44.503949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.232760+0000","last_change":"2023-10-06T05:16:10.227849+0000","last_active":"2023-10-06T05:16:10.232760+0000","last_peered":"2023-10-06T05:16:10.232760+0000","last_clean":"2023-10-06T05:16:10.232760+0000","last_became_active":"2023-10-06T05:16:10.227571+0000","last_became_peered":"2023-10-06T05:16:10.227571+0000","last_unstale":"2023-10-06T05:16:10.232760+0000","last_undegraded":"2023-10-06T05:16:10.232760+0000","last_fullsized":"2023-10-06T05:16:10.232760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:38.653708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.199505+0000","last_change":"2023-10-06T05:16:09.211565+0000","last_active":"2023-10-06T05:16:10.199505+0000","last_peered":"2023-10-06T05:16:10.199505+0000","last_clean":"2023-10-06T05:16:10.199505+0000","last_became_active":"2023-10-06T05:16:09.211314+0000","last_became_peered":"2023-10-06T05:16:09.211314+0000","last_unstale":"2023-10-06T05:16:10.199505+0000","last_undegraded":"2023-10-06T05:16:10.199505+0000","last_fullsized":"2023-10-06T05:16:10.199505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:31.604204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.223475+0000","last_change":"2023-10-06T05:16:10.218545+0000","last_active":"2023-10-06T05:16:10.223475+0000","last_peered":"2023-10-06T05:16:10.223475+0000","last_clean":"2023-10-06T05:16:10.223475+0000","last_became_active":"2023-10-06T05:16:10.218226+0000","last_became_peered":"2023-10-06T05:16:10.218226+0000","last_unstale":"2023-10-06T05:16:10.223475+0000","last_undegraded":"2023-10-06T05:16:10.223475+0000","last_fullsized":"2023-10-06T05:16:10.223475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:07.100657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226675+0000","last_change":"2023-10-06T05:16:09.214146+0000","last_active":"2023-10-06T05:16:10.226675+0000","last_peered":"2023-10-06T05:16:10.226675+0000","last_clean":"2023-10-06T05:16:10.226675+0000","last_became_active":"2023-10-06T05:16:09.213482+0000","last_became_peered":"2023-10-06T05:16:09.213482+0000","last_unstale":"2023-10-06T05:16:10.226675+0000","last_undegraded":"2023-10-06T05:16:10.226675+0000","last_fullsized":"2023-10-06T05:16:10.226675+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:08.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.227973+0000","last_change":"2023-10-06T05:16:10.220468+0000","last_active":"2023-10-06T05:16:10.227973+0000","last_peered":"2023-10-06T05:16:10.227973+0000","last_clean":"2023-10-06T05:16:10.227973+0000","last_became_active":"2023-10-06T05:16:10.220190+0000","last_became_peered":"2023-10-06T05:16:10.220190+0000","last_unstale":"2023-10-06T05:16:10.227973+0000","last_undegraded":"2023-10-06T05:16:10.227973+0000","last_fullsized":"2023-10-06T05:16:10.227973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:43.476142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193811+0000","last_change":"2023-10-06T05:16:09.207873+0000","last_active":"2023-10-06T05:16:10.193811+0000","last_peered":"2023-10-06T05:16:10.193811+0000","last_clean":"2023-10-06T05:16:10.193811+0000","last_became_active":"2023-10-06T05:16:09.207577+0000","last_became_peered":"2023-10-06T05:16:09.207577+0000","last_unstale":"2023-10-06T05:16:10.193811+0000","last_undegraded":"2023-10-06T05:16:10.193811+0000","last_fullsized":"2023-10-06T05:16:10.193811+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:39.272257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.220486+0000","last_change":"2023-10-06T05:16:10.214435+0000","last_active":"2023-10-06T05:16:10.220486+0000","last_peered":"2023-10-06T05:16:10.220486+0000","last_clean":"2023-10-06T05:16:10.220486+0000","last_became_active":"2023-10-06T05:16:10.214248+0000","last_became_peered":"2023-10-06T05:16:10.214248+0000","last_unstale":"2023-10-06T05:16:10.220486+0000","last_undegraded":"2023-10-06T05:16:10.220486+0000","last_fullsized":"2023-10-06T05:16:10.220486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:57.452379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.201737+0000","last_change":"2023-10-06T05:16:09.205408+0000","last_active":"2023-10-06T05:16:10.201737+0000","last_peered":"2023-10-06T05:16:10.201737+0000","last_clean":"2023-10-06T05:16:10.201737+0000","last_became_active":"2023-10-06T05:16:09.205130+0000","last_became_peered":"2023-10-06T05:16:09.205130+0000","last_unstale":"2023-10-06T05:16:10.201737+0000","last_undegraded":"2023-10-06T05:16:10.201737+0000","last_fullsized":"2023-10-06T05:16:10.201737+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:37.589749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221123+0000","last_change":"2023-10-06T05:16:10.217361+0000","last_active":"2023-10-06T05:16:10.221123+0000","last_peered":"2023-10-06T05:16:10.221123+0000","last_clean":"2023-10-06T05:16:10.221123+0000","last_became_active":"2023-10-06T05:16:10.217045+0000","last_became_peered":"2023-10-06T05:16:10.217045+0000","last_unstale":"2023-10-06T05:16:10.221123+0000","last_undegraded":"2023-10-06T05:16:10.221123+0000","last_fullsized":"2023-10-06T05:16:10.221123+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:00.820889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.199441+0000","last_change":"2023-10-06T05:16:09.204352+0000","last_active":"2023-10-06T05:16:10.199441+0000","last_peered":"2023-10-06T05:16:10.199441+0000","last_clean":"2023-10-06T05:16:10.199441+0000","last_became_active":"2023-10-06T05:16:09.204037+0000","last_became_peered":"2023-10-06T05:16:09.204037+0000","last_unstale":"2023-10-06T05:16:10.199441+0000","last_undegraded":"2023-10-06T05:16:10.199441+0000","last_fullsized":"2023-10-06T05:16:10.199441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:42.471127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226750+0000","last_change":"2023-10-06T05:16:10.222136+0000","last_active":"2023-10-06T05:16:10.226750+0000","last_peered":"2023-10-06T05:16:10.226750+0000","last_clean":"2023-10-06T05:16:10.226750+0000","last_became_active":"2023-10-06T05:16:10.221565+0000","last_became_peered":"2023-10-06T05:16:10.221565+0000","last_unstale":"2023-10-06T05:16:10.226750+0000","last_undegraded":"2023-10-06T05:16:10.226750+0000","last_fullsized":"2023-10-06T05:16:10.226750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:07.100657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.198391+0000","last_change":"2023-10-06T05:16:09.212218+0000","last_active":"2023-10-06T05:16:10.198391+0000","last_peered":"2023-10-06T05:16:10.198391+0000","last_clean":"2023-10-06T05:16:10.198391+0000","last_became_active":"2023-10-06T05:16:09.211750+0000","last_became_peered":"2023-10-06T05:16:09.211750+0000","last_unstale":"2023-10-06T05:16:10.198391+0000","last_undegraded":"2023-10-06T05:16:10.198391+0000","last_fullsized":"2023-10-06T05:16:10.198391+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:29.892780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.215807+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:16:10.215807+0000","last_peered":"2023-10-06T05:16:10.215807+0000","last_clean":"2023-10-06T05:16:10.215807+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:16:10.215807+0000","last_undegraded":"2023-10-06T05:16:10.215807+0000","last_fullsized":"2023-10-06T05:16:10.215807+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.214837+0000","last_change":"2023-10-06T05:16:10.208514+0000","last_active":"2023-10-06T05:16:10.214837+0000","last_peered":"2023-10-06T05:16:10.214837+0000","last_clean":"2023-10-06T05:16:10.214837+0000","last_became_active":"2023-10-06T05:16:10.208212+0000","last_became_peered":"2023-10-06T05:16:10.208212+0000","last_unstale":"2023-10-06T05:16:10.214837+0000","last_undegraded":"2023-10-06T05:16:10.214837+0000","last_fullsized":"2023-10-06T05:16:10.214837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:44.359697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T05:16:11.759952+0000","last_change":"2023-10-06T05:16:09.212135+0000","last_active":"2023-10-06T05:16:11.759952+0000","last_peered":"2023-10-06T05:16:11.759952+0000","last_clean":"2023-10-06T05:16:11.759952+0000","last_became_active":"2023-10-06T05:16:09.211799+0000","last_became_peered":"2023-10-06T05:16:09.211799+0000","last_unstale":"2023-10-06T05:16:11.759952+0000","last_undegraded":"2023-10-06T05:16:11.759952+0000","last_fullsized":"2023-10-06T05:16:11.759952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:34.219722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193993+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:16:10.193993+0000","last_peered":"2023-10-06T05:16:10.193993+0000","last_clean":"2023-10-06T05:16:10.193993+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:16:10.193993+0000","last_undegraded":"2023-10-06T05:16:10.193993+0000","last_fullsized":"2023-10-06T05:16:10.193993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219222+0000","last_change":"2023-10-06T05:16:10.211591+0000","last_active":"2023-10-06T05:16:10.219222+0000","last_peered":"2023-10-06T05:16:10.219222+0000","last_clean":"2023-10-06T05:16:10.219222+0000","last_became_active":"2023-10-06T05:16:10.211351+0000","last_became_peered":"2023-10-06T05:16:10.211351+0000","last_unstale":"2023-10-06T05:16:10.219222+0000","last_undegraded":"2023-10-06T05:16:10.219222+0000","last_fullsized":"2023-10-06T05:16:10.219222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:37.873768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.202947+0000","last_change":"2023-10-06T05:16:09.208527+0000","last_active":"2023-10-06T05:16:10.202947+0000","last_peered":"2023-10-06T05:16:10.202947+0000","last_clean":"2023-10-06T05:16:10.202947+0000","last_became_active":"2023-10-06T05:16:09.208245+0000","last_became_peered":"2023-10-06T05:16:09.208245+0000","last_unstale":"2023-10-06T05:16:10.202947+0000","last_undegraded":"2023-10-06T05:16:10.202947+0000","last_fullsized":"2023-10-06T05:16:10.202947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:08.704326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.211290+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:16:10.211290+0000","last_peered":"2023-10-06T05:16:10.211290+0000","last_clean":"2023-10-06T05:16:10.211290+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:16:10.211290+0000","last_undegraded":"2023-10-06T05:16:10.211290+0000","last_fullsized":"2023-10-06T05:16:10.211290+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.221191+0000","last_change":"2023-10-06T05:16:10.213644+0000","last_active":"2023-10-06T05:16:10.221191+0000","last_peered":"2023-10-06T05:16:10.221191+0000","last_clean":"2023-10-06T05:16:10.221191+0000","last_became_active":"2023-10-06T05:16:10.212920+0000","last_became_peered":"2023-10-06T05:16:10.212920+0000","last_unstale":"2023-10-06T05:16:10.221191+0000","last_undegraded":"2023-10-06T05:16:10.221191+0000","last_fullsized":"2023-10-06T05:16:10.221191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:13.881962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.224213+0000","last_change":"2023-10-06T05:16:09.207900+0000","last_active":"2023-10-06T05:16:10.224213+0000","last_peered":"2023-10-06T05:16:10.224213+0000","last_clean":"2023-10-06T05:16:10.224213+0000","last_became_active":"2023-10-06T05:16:09.207634+0000","last_became_peered":"2023-10-06T05:16:09.207634+0000","last_unstale":"2023-10-06T05:16:10.224213+0000","last_undegraded":"2023-10-06T05:16:10.224213+0000","last_fullsized":"2023-10-06T05:16:10.224213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:38.372416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.193680+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:16:10.193680+0000","last_peered":"2023-10-06T05:16:10.193680+0000","last_clean":"2023-10-06T05:16:10.193680+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:16:10.193680+0000","last_undegraded":"2023-10-06T05:16:10.193680+0000","last_fullsized":"2023-10-06T05:16:10.193680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.199541+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:16:10.199541+0000","last_peered":"2023-10-06T05:16:10.199541+0000","last_clean":"2023-10-06T05:16:10.199541+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:16:10.199541+0000","last_undegraded":"2023-10-06T05:16:10.199541+0000","last_fullsized":"2023-10-06T05:16:10.199541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218249+0000","last_change":"2023-10-06T05:16:10.212234+0000","last_active":"2023-10-06T05:16:10.218249+0000","last_peered":"2023-10-06T05:16:10.218249+0000","last_clean":"2023-10-06T05:16:10.218249+0000","last_became_active":"2023-10-06T05:16:10.211947+0000","last_became_peered":"2023-10-06T05:16:10.211947+0000","last_unstale":"2023-10-06T05:16:10.218249+0000","last_undegraded":"2023-10-06T05:16:10.218249+0000","last_fullsized":"2023-10-06T05:16:10.218249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:13.881962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.215707+0000","last_change":"2023-10-06T05:16:09.212220+0000","last_active":"2023-10-06T05:16:10.215707+0000","last_peered":"2023-10-06T05:16:10.215707+0000","last_clean":"2023-10-06T05:16:10.215707+0000","last_became_active":"2023-10-06T05:16:09.211960+0000","last_became_peered":"2023-10-06T05:16:09.211960+0000","last_unstale":"2023-10-06T05:16:10.215707+0000","last_undegraded":"2023-10-06T05:16:10.215707+0000","last_fullsized":"2023-10-06T05:16:10.215707+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:42.796749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.199625+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:16:10.199625+0000","last_peered":"2023-10-06T05:16:10.199625+0000","last_clean":"2023-10-06T05:16:10.199625+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:16:10.199625+0000","last_undegraded":"2023-10-06T05:16:10.199625+0000","last_fullsized":"2023-10-06T05:16:10.199625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.218145+0000","last_change":"2023-10-06T05:16:10.215983+0000","last_active":"2023-10-06T05:16:10.218145+0000","last_peered":"2023-10-06T05:16:10.218145+0000","last_clean":"2023-10-06T05:16:10.218145+0000","last_became_active":"2023-10-06T05:16:10.215700+0000","last_became_peered":"2023-10-06T05:16:10.215700+0000","last_unstale":"2023-10-06T05:16:10.218145+0000","last_undegraded":"2023-10-06T05:16:10.218145+0000","last_fullsized":"2023-10-06T05:16:10.218145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:30.610440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.215771+0000","last_change":"2023-10-06T05:16:09.208026+0000","last_active":"2023-10-06T05:16:10.215771+0000","last_peered":"2023-10-06T05:16:10.215771+0000","last_clean":"2023-10-06T05:16:10.215771+0000","last_became_active":"2023-10-06T05:16:09.207756+0000","last_became_peered":"2023-10-06T05:16:09.207756+0000","last_unstale":"2023-10-06T05:16:10.215771+0000","last_undegraded":"2023-10-06T05:16:10.215771+0000","last_fullsized":"2023-10-06T05:16:10.215771+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:37.589749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.203034+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:16:10.203034+0000","last_peered":"2023-10-06T05:16:10.203034+0000","last_clean":"2023-10-06T05:16:10.203034+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:16:10.203034+0000","last_undegraded":"2023-10-06T05:16:10.203034+0000","last_fullsized":"2023-10-06T05:16:10.203034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.226735+0000","last_change":"2023-10-06T05:16:10.216632+0000","last_active":"2023-10-06T05:16:10.226735+0000","last_peered":"2023-10-06T05:16:10.226735+0000","last_clean":"2023-10-06T05:16:10.226735+0000","last_became_active":"2023-10-06T05:16:10.216319+0000","last_became_peered":"2023-10-06T05:16:10.216319+0000","last_unstale":"2023-10-06T05:16:10.226735+0000","last_undegraded":"2023-10-06T05:16:10.226735+0000","last_fullsized":"2023-10-06T05:16:10.226735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:24.430630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.194766+0000","last_change":"2023-10-06T05:16:09.212202+0000","last_active":"2023-10-06T05:16:10.194766+0000","last_peered":"2023-10-06T05:16:10.194766+0000","last_clean":"2023-10-06T05:16:10.194766+0000","last_became_active":"2023-10-06T05:16:09.211949+0000","last_became_peered":"2023-10-06T05:16:09.211949+0000","last_unstale":"2023-10-06T05:16:10.194766+0000","last_undegraded":"2023-10-06T05:16:10.194766+0000","last_fullsized":"2023-10-06T05:16:10.194766+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:08.250446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.219090+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:16:10.219090+0000","last_peered":"2023-10-06T05:16:10.219090+0000","last_clean":"2023-10-06T05:16:10.219090+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:16:10.219090+0000","last_undegraded":"2023-10-06T05:16:10.219090+0000","last_fullsized":"2023-10-06T05:16:10.219090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.224679+0000","last_change":"2023-10-06T05:16:10.223561+0000","last_active":"2023-10-06T05:16:10.224679+0000","last_peered":"2023-10-06T05:16:10.224679+0000","last_clean":"2023-10-06T05:16:10.224679+0000","last_became_active":"2023-10-06T05:16:10.223267+0000","last_became_peered":"2023-10-06T05:16:10.223267+0000","last_unstale":"2023-10-06T05:16:10.224679+0000","last_undegraded":"2023-10-06T05:16:10.224679+0000","last_fullsized":"2023-10-06T05:16:10.224679+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:09.182943+0000","last_clean_scrub_stamp":"2023-10-06T05:16:09.182943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:35.024962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.201582+0000","last_change":"2023-10-06T05:16:09.208354+0000","last_active":"2023-10-06T05:16:10.201582+0000","last_peered":"2023-10-06T05:16:10.201582+0000","last_clean":"2023-10-06T05:16:10.201582+0000","last_became_active":"2023-10-06T05:16:09.208075+0000","last_became_peered":"2023-10-06T05:16:09.208075+0000","last_unstale":"2023-10-06T05:16:10.201582+0000","last_undegraded":"2023-10-06T05:16:10.201582+0000","last_fullsized":"2023-10-06T05:16:10.201582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:08.177928+0000","last_clean_scrub_stamp":"2023-10-06T05:16:08.177928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:31.604204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T05:16:10.208812+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:16:10.208812+0000","last_peered":"2023-10-06T05:16:10.208812+0000","last_clean":"2023-10-06T05:16:10.208812+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:16:10.208812+0000","last_undegraded":"2023-10-06T05:16:10.208812+0000","last_fullsized":"2023-10-06T05:16:10.208812+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298300,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073540,"statfs":{"total":96636764160,"available":96331304960,"internally_reserved":0,"allocated":1036288,"data_stored":787433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.496}]},{"osd":3,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.40600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.435},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":197619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":9.6300000000000008,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.595000000000001,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.825,"5min":1.825,"15min":1.825},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":23.146000000000001,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.80700000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":9.9969999999999999,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.622},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":9.5850000000000009,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":9.8420000000000005,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:15:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":9.718,"5min":9.718,"15min":9.718},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":9.8930000000000007,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.85299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297708,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074132,"statfs":{"total":96636764160,"available":96331911168,"internally_reserved":0,"allocated":430080,"data_stored":190084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.438}]},{"osd":3,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298300,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073540,"statfs":{"total":96636764160,"available":96331304960,"internally_reserved":0,"allocated":1036288,"data_stored":787377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.379}]},{"osd":2,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.33100000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":191572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.42499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.629}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":196232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.27600000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.371}]},{"osd":4,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":450560,"data_stored":198511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.622},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":442368,"data_stored":197045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.79900000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.83899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:16:14.959 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:16:14.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:16:15.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.104+0000 7f32f8d7d700 1 Processor -- start 2023-10-06T05:16:15.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.104+0000 7f32f8d7d700 1 -- start start 2023-10-06T05:16:15.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f8d7d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32f4140ab0 0x7f32f40acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f8d7d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f40c9940 con 0x7f32f41406e0 2023-10-06T05:16:15.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f8d7d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f406d230 con 0x7f32f40ad4e0 2023-10-06T05:16:15.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f8d7d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f406d510 con 0x7f32f4140ab0 2023-10-06T05:16:15.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32f41406e0 0x7f32f406c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52336/0 (socket says 172.21.15.45:52336) 2023-10-06T05:16:15.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f259c700 1 -- 172.21.15.45:0/1770544049 learned_addr learned my addr 172.21.15.45:0/1770544049 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:15.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f1d9b700 1 --2- 172.21.15.45:0/1770544049 >> v2:172.21.15.144:3300/0 conn(0x7f32f4140ab0 0x7f32f40acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f159a700 1 -- 172.21.15.45:0/1770544049 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569179883 0 0) 0x7f32f40c9940 con 0x7f32f41406e0 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f159a700 1 -- 172.21.15.45:0/1770544049 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32dc003570 con 0x7f32f41406e0 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f159a700 1 -- 172.21.15.45:0/1770544049 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531554451 0 0) 0x7f32f406d230 con 0x7f32f40ad4e0 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.105+0000 7f32f159a700 1 -- 172.21.15.45:0/1770544049 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f40c9940 con 0x7f32f40ad4e0 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f1d9b700 1 -- 172.21.15.45:0/1770544049 >> v1:172.21.15.144:6789/0 conn(0x7f32f40ad4e0 legacy=0x7f32f40c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f1d9b700 1 -- 172.21.15.45:0/1770544049 >> v1:172.21.15.45:6789/0 conn(0x7f32f41406e0 legacy=0x7f32f406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f1d9b700 1 -- 172.21.15.45:0/1770544049 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f406d7f0 con 0x7f32f4140ab0 2023-10-06T05:16:15.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f1d9b700 1 --2- 172.21.15.45:0/1770544049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f4140ab0 0x7f32f40acfa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32e80098e0 tx=0x7f32e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7f7c9a779d3cad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f159a700 1 -- 172.21.15.45:0/1770544049 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8011630 con 0x7f32f4140ab0 2023-10-06T05:16:15.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f159a700 1 -- 172.21.15.45:0/1770544049 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8011790 con 0x7f32f4140ab0 2023-10-06T05:16:15.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f159a700 1 -- 172.21.15.45:0/1770544049 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8011b10 con 0x7f32f4140ab0 2023-10-06T05:16:15.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f8d7d700 1 -- 172.21.15.45:0/1770544049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f4140ab0 msgr2=0x7f32f40acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.106+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/1770544049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f4140ab0 0x7f32f40acfa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32e80098e0 tx=0x7f32e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.107+0000 7f32f8d7d700 1 -- 172.21.15.45:0/1770544049 shutdown_connections 2023-10-06T05:16:15.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.107+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/1770544049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f4140ab0 0x7f32f40acfa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.107+0000 7f32f8d7d700 1 -- 172.21.15.45:0/1770544049 >> 172.21.15.45:0/1770544049 conn(0x7f32f40691c0 msgr2=0x7f32f40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:15.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.107+0000 7f32f8d7d700 1 -- 172.21.15.45:0/1770544049 shutdown_connections 2023-10-06T05:16:15.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.107+0000 7f32f8d7d700 1 -- 172.21.15.45:0/1770544049 wait complete. 2023-10-06T05:16:15.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.108+0000 7f32f8d7d700 1 Processor -- start 2023-10-06T05:16:15.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.108+0000 7f32f8d7d700 1 -- start start 2023-10-06T05:16:15.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f40ad4e0 0x7f32f41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41406e0 0x7f32f415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32f4140ab0 0x7f32f41595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f415c0c0 con 0x7f32f4140ab0 2023-10-06T05:16:15.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4158e20 con 0x7f32f41406e0 2023-10-06T05:16:15.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f415b790 con 0x7f32f40ad4e0 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32f4140ab0 0x7f32f41595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32f4140ab0 0x7f32f41595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33132/0 (socket says 172.21.15.45:33132) 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 -- 172.21.15.45:0/2252904647 learned_addr learned my addr 172.21.15.45:0/2252904647 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f259c700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f40ad4e0 0x7f32f41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f1d9b700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41406e0 0x7f32f415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 -- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f40ad4e0 msgr2=0x7f32f41588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f40ad4e0 0x7f32f41588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 -- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41406e0 msgr2=0x7f32f415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41406e0 0x7f32f415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 -- 172.21.15.45:0/2252904647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8009420 con 0x7f32f4140ab0 2023-10-06T05:16:15.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f2d9d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32f4140ab0 0x7f32f41595a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c910 tx=0x7f32ec0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32e27fc700 1 -- 172.21.15.45:0/2252904647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec012630 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32e27fc700 1 -- 172.21.15.45:0/2252904647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec0127b0 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32e27fc700 1 -- 172.21.15.45:0/2252904647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec012b70 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f415ba70 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.109+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f415ca90 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.110+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f406e2d0 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.111+0000 7f32e27fc700 1 -- 172.21.15.45:0/2252904647 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32ec01b400 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.111+0000 7f32e27fc700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32d4066f30 0x7f32d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.111+0000 7f32e27fc700 1 -- 172.21.15.45:0/2252904647 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f32ec084b80 con 0x7f32f4140ab0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.112+0000 7f32f259c700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32d4066f30 0x7f32d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.112+0000 7f32f259c700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32d4066f30 0x7f32d40693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f32e4007fd0 tx=0x7f32e4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.116+0000 7f32e27fc700 1 -- 172.21.15.45:0/2252904647 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32ec051ab0 con 0x7f32f4140ab0 2023-10-06T05:16:15.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.399+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f32f404bb40 con 0x7f32f4140ab0 2023-10-06T05:16:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.402+0000 7f32e27fc700 1 -- 172.21.15.45:0/2252904647 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f32ec0550d0 con 0x7f32f4140ab0 2023-10-06T05:16:15.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32d4066f30 msgr2=0x7f32d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32d4066f30 0x7f32d40693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f32e4007fd0 tx=0x7f32e4009040 comp rx=0 tx=0).stop 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32f4140ab0 msgr2=0x7f32f41595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32f4140ab0 0x7f32f41595a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c910 tx=0x7f32ec0027a0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 shutdown_connections 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32d4066f30 0x7f32d40693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32f4140ab0 0x7f32f41595a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f40ad4e0 0x7f32f41588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 --2- 172.21.15.45:0/2252904647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41406e0 0x7f32f415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 >> 172.21.15.45:0/2252904647 conn(0x7f32f40691c0 msgr2=0x7f32f413f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:15.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 shutdown_connections 2023-10-06T05:16:15.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.405+0000 7f32f8d7d700 1 -- 172.21.15.45:0/2252904647 wait complete. 2023-10-06T05:16:15.409 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-10-06T05:16:15.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.550+0000 7f69bdac0700 1 Processor -- start 2023-10-06T05:16:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.550+0000 7f69bdac0700 1 -- start start 2023-10-06T05:16:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69bdac0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69b806afa0 0x7f69b8068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69bdac0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b80676c0 con 0x7f69b8153a20 2023-10-06T05:16:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69bdac0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b80679a0 con 0x7f69b8153650 2023-10-06T05:16:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69bdac0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b8067c80 con 0x7f69b806afa0 2023-10-06T05:16:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69b6ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f69b8153a20 0x7f69b806a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52352/0 (socket says 172.21.15.45:52352) 2023-10-06T05:16:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69b6ffd700 1 -- 172.21.15.45:0/2104626718 learned_addr learned my addr 172.21.15.45:0/2104626718 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179958733 0 0) 0x7f69b80676c0 con 0x7f69b8153a20 2023-10-06T05:16:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69b7fff700 1 --2- 172.21.15.45:0/2104626718 >> v2:172.21.15.144:3300/0 conn(0x7f69b806afa0 0x7f69b8068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.551+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69a4003570 con 0x7f69b8153a20 2023-10-06T05:16:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591457210 0 0) 0x7f69a4003570 con 0x7f69b8153a20 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 >> v2:172.21.15.144:3300/0 conn(0x7f69b806afa0 msgr2=0x7f69b8068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 --2- 172.21.15.45:0/2104626718 >> v2:172.21.15.144:3300/0 conn(0x7f69b806afa0 0x7f69b8068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 >> v1:172.21.15.144:6789/0 conn(0x7f69b8153650 legacy=0x7f69b815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b8067f60 con 0x7f69b8153a20 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f69a0001470 con 0x7f69b8153a20 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 >> v1:172.21.15.45:6789/0 conn(0x7f69b8153a20 legacy=0x7f69b806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4003ab0 0x7f69a4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a4004810 0x7f69a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a4005210 0x7f69a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b8067c80 con 0x7f69a4004810 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a400c1b0 con 0x7f69a4003ab0 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a400c370 con 0x7f69a4005210 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b6ffd700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a4004810 0x7f69a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b77fe700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4003ab0 0x7f69a4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b7fff700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a4005210 0x7f69a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b6ffd700 1 -- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a4005210 msgr2=0x7f69a400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b6ffd700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a4005210 0x7f69a400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b6ffd700 1 -- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4003ab0 msgr2=0x7f69a4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b6ffd700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4003ab0 0x7f69a4003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b6ffd700 1 -- 172.21.15.45:0/2104626718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a4003570 con 0x7f69a4004810 2023-10-06T05:16:15.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b7fff700 1 -- 172.21.15.45:0/2104626718 reap_dead start 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.552+0000 7f69b6ffd700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a4004810 0x7f69a4004c80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f69b8157350 tx=0x7f69b80c8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d80e50bbc781bdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a000cb40 con 0x7f69a4004810 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a000cca0 con 0x7f69a4004810 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69b67fc700 1 -- 172.21.15.45:0/2104626718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0005b70 con 0x7f69a4004810 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69bdac0700 1 -- 172.21.15.45:0/2104626718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a4004810 msgr2=0x7f69a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69bdac0700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a4004810 0x7f69a4004c80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f69b8157350 tx=0x7f69b80c8570 comp rx=0 tx=0).stop 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69bdac0700 1 -- 172.21.15.45:0/2104626718 shutdown_connections 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69bdac0700 1 --2- 172.21.15.45:0/2104626718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a4004810 0x7f69a4004c80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69bdac0700 1 -- 172.21.15.45:0/2104626718 >> 172.21.15.45:0/2104626718 conn(0x7f69b80cc5a0 msgr2=0x7f69b80cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69bdac0700 1 -- 172.21.15.45:0/2104626718 shutdown_connections 2023-10-06T05:16:15.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.553+0000 7f69bdac0700 1 -- 172.21.15.45:0/2104626718 wait complete. 2023-10-06T05:16:15.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 Processor -- start 2023-10-06T05:16:15.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 -- start start 2023-10-06T05:16:15.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4004810 0x7f69b8151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b816d840 0x7f69b816cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b816d160 0x7f69b80c0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b816d5d0 con 0x7f69b816d160 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b80add30 con 0x7f69a4004810 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.555+0000 7f69bdac0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b80ad120 con 0x7f69b816d840 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b816d160 0x7f69b80c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b816d160 0x7f69b80c0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33148/0 (socket says 172.21.15.45:33148) 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 -- 172.21.15.45:0/92356398 learned_addr learned my addr 172.21.15.45:0/92356398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b6ffd700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b816d840 0x7f69b816cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b77fe700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4004810 0x7f69b8151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 -- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b816d840 msgr2=0x7f69b816cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b816d840 0x7f69b816cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 -- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4004810 msgr2=0x7f69b8151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4004810 0x7f69b8151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 -- 172.21.15.45:0/92356398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b80ad400 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69b7fff700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b816d160 0x7f69b80c0420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006f00 tx=0x7f69a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69af7fe700 1 -- 172.21.15.45:0/92356398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8013680 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69af7fe700 1 -- 172.21.15.45:0/92356398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a80137e0 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69af7fe700 1 -- 172.21.15.45:0/92356398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8013b60 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b80ad6e0 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.556+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b80c0960 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.557+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b80adb80 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.558+0000 7f69af7fe700 1 -- 172.21.15.45:0/92356398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69a8014440 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.558+0000 7f69af7fe700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6998066f30 0x7f69980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.558+0000 7f69af7fe700 1 -- 172.21.15.45:0/92356398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f69a8084ba0 con 0x7f69b816d160 2023-10-06T05:16:15.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.558+0000 7f69b77fe700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6998066f30 0x7f69980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:15.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.559+0000 7f69b77fe700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6998066f30 0x7f69980693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69b0002fa0 tx=0x7f69b0001640 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:15.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.563+0000 7f69af7fe700 1 -- 172.21.15.45:0/92356398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69a8051bf0 con 0x7f69b816d160 2023-10-06T05:16:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.809+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69b80cc0e0 con 0x7f69b816d160 2023-10-06T05:16:15.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.809+0000 7f69af7fe700 1 -- 172.21.15.45:0/92356398 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f69a8022090 con 0x7f69b816d160 2023-10-06T05:16:15.811 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:15.812 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:13.214114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6834/2918652845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2918652845},{"type":"v1","addr":"172.21.15.45:6834","nonce":2918652845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:16:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6998066f30 msgr2=0x7f69980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6998066f30 0x7f69980693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69b0002fa0 tx=0x7f69b0001640 comp rx=0 tx=0).stop 2023-10-06T05:16:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b816d160 msgr2=0x7f69b80c0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b816d160 0x7f69b80c0420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006f00 tx=0x7f69a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 shutdown_connections 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6998066f30 0x7f69980693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b816d160 0x7f69b80c0420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a4004810 0x7f69b8151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 --2- 172.21.15.45:0/92356398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b816d840 0x7f69b816cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 >> 172.21.15.45:0/92356398 conn(0x7f69b80cc5a0 msgr2=0x7f69b8158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 shutdown_connections 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:15.812+0000 7f69bdac0700 1 -- 172.21.15.45:0/92356398 wait complete. 2023-10-06T05:16:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 16 2023-10-06T05:16:16.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:16:09.624552+0000', 'modified': '2023-10-06T05:16:13.214114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5133': {'gid': 5133, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6834/2918652845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 2918652845}, {'type': 'v1', 'addr': '172.21.15.45:6834', 'nonce': 2918652845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:16:16.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:16:16.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:16:16.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.369+0000 7f5a5009c700 1 Processor -- start 2023-10-06T05:16:16.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.369+0000 7f5a5009c700 1 -- start start 2023-10-06T05:16:16.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a5009c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5a480c0170 0x7f5a480ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a5009c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a48161c20 0x7f5a480acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a5009c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a48162060 con 0x7f5a480cd300 2023-10-06T05:16:16.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a5009c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a480cd7a0 con 0x7f5a480c0170 2023-10-06T05:16:16.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a5009c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4806b080 con 0x7f5a48161c20 2023-10-06T05:16:16.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a4e639700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a48161c20 0x7f5a480acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a4de38700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5a480c0170 0x7f5a480ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a4d637700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5a480cd300 0x7f5a48161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:59124/0 (socket says 172.21.15.45:59124) 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a4de38700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5a480c0170 0x7f5a480ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:49382/0 (socket says 172.21.15.45:49382) 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a4e639700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a48161c20 0x7f5a480acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:51214/0 (socket says 172.21.15.45:51214) 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.370+0000 7f5a4d637700 1 -- 172.21.15.45:0/650016527 learned_addr learned my addr 172.21.15.45:0/650016527 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4de38700 1 -- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a48161c20 msgr2=0x7f5a480acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4de38700 1 --2- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a48161c20 0x7f5a480acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4de38700 1 -- 172.21.15.45:0/650016527 >> v1:172.21.15.144:6789/0 conn(0x7f5a480cd300 legacy=0x7f5a48161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4de38700 1 -- 172.21.15.45:0/650016527 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4806b2d0 con 0x7f5a480c0170 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4e639700 1 --2- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a48161c20 0x7f5a480acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:16.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4ce36700 1 -- 172.21.15.45:0/650016527 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863987335 0 0) 0x7f5a38001110 con 0x7f5a480cd300 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4de38700 1 --2- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a480c0170 0x7f5a480ccdc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a440098e0 tx=0x7f5a440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a64b30608b924a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4ce36700 1 -- 172.21.15.45:0/650016527 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400b640 con 0x7f5a480c0170 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4ce36700 1 -- 172.21.15.45:0/650016527 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4400b7a0 con 0x7f5a480c0170 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.371+0000 7f5a4ce36700 1 -- 172.21.15.45:0/650016527 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400bb40 con 0x7f5a480c0170 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 -- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a480c0170 msgr2=0x7f5a480ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 --2- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a480c0170 0x7f5a480ccdc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a440098e0 tx=0x7f5a440091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 -- 172.21.15.45:0/650016527 shutdown_connections 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 --2- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a48161c20 0x7f5a480acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 --2- 172.21.15.45:0/650016527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a480c0170 0x7f5a480ccdc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 -- 172.21.15.45:0/650016527 >> 172.21.15.45:0/650016527 conn(0x7f5a4806f090 msgr2=0x7f5a4806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 -- 172.21.15.45:0/650016527 shutdown_connections 2023-10-06T05:16:16.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.372+0000 7f5a5009c700 1 -- 172.21.15.45:0/650016527 wait complete. 2023-10-06T05:16:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.373+0000 7f5a5009c700 1 Processor -- start 2023-10-06T05:16:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.373+0000 7f5a5009c700 1 -- start start 2023-10-06T05:16:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a5009c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a480c0170 0x7f5a48151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a5009c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a480cd300 0x7f5a48068580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a5009c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a48161c20 0x7f5a48068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a5009c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48172110 con 0x7f5a480c0170 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a5009c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480c1c40 con 0x7f5a48161c20 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a5009c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480c1f20 con 0x7f5a480cd300 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a480c0170 0x7f5a48151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a480c0170 0x7f5a48151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34162/0 (socket says 172.21.15.45:34162) 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 -- 172.21.15.45:0/1631826217 learned_addr learned my addr 172.21.15.45:0/1631826217 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4e639700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a48161c20 0x7f5a48068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a480cd300 0x7f5a48068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 -- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a480cd300 msgr2=0x7f5a48068580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a480cd300 0x7f5a48068580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 -- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a48161c20 msgr2=0x7f5a48068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a48161c20 0x7f5a48068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4de38700 1 -- 172.21.15.45:0/1631826217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40009a20 con 0x7f5a480c0170 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.374+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a480cd300 0x7f5a48068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.375+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a480c0170 0x7f5a48151410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5a4806aed0 tx=0x7f5a44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.375+0000 7f5a3dffb700 1 -- 172.21.15.45:0/1631826217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400bb20 con 0x7f5a480c0170 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.375+0000 7f5a3dffb700 1 -- 172.21.15.45:0/1631826217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a44011420 con 0x7f5a480c0170 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.375+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a44009420 con 0x7f5a480c0170 2023-10-06T05:16:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.375+0000 7f5a3dffb700 1 -- 172.21.15.45:0/1631826217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440117a0 con 0x7f5a480c0170 2023-10-06T05:16:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.375+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a48151950 con 0x7f5a480c0170 2023-10-06T05:16:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.377+0000 7f5a3dffb700 1 -- 172.21.15.45:0/1631826217 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a44011b10 con 0x7f5a480c0170 2023-10-06T05:16:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.377+0000 7f5a3dffb700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a2c066f30 0x7f5a2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.377+0000 7f5a3dffb700 1 -- 172.21.15.45:0/1631826217 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5a44085c00 con 0x7f5a480c0170 2023-10-06T05:16:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.377+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a2c066f30 0x7f5a2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.378+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a2c066f30 0x7f5a2c0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a480c3980 tx=0x7f5a38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.379+0000 7f5a593b0b80 1 Processor -- start 2023-10-06T05:16:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.379+0000 7f5a593b0b80 1 -- start start 2023-10-06T05:16:16.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.379+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x5583861d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.379+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861d9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x5583861da940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583860a4310 con 0x5583861b6f40 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583860e1130 con 0x5583861da4d0 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583860deaf0 con 0x5583861d1450 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861d9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861d9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34172/0 (socket says 172.21.15.45:34172) 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 -- 172.21.15.45:0/2776196424 learned_addr learned my addr 172.21.15.45:0/2776196424 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4e639700 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x5583861da940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4de38700 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x5583861d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 -- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 msgr2=0x5583861d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x5583861d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 -- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 msgr2=0x5583861da940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x5583861da940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 -- 172.21.15.45:0/2776196424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558385ee0790 con 0x5583861b6f40 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a4d637700 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861d9f90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005c40 tx=0x7f5a38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53b099018f83ab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a32ffd700 1 -- 172.21.15.45:0/2776196424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800e660 con 0x5583861b6f40 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a32ffd700 1 -- 172.21.15.45:0/2776196424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800e7c0 con 0x5583861b6f40 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.380+0000 7f5a32ffd700 1 -- 172.21.15.45:0/2776196424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800eb80 con 0x5583861b6f40 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 -- 172.21.15.45:0/2776196424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 msgr2=0x5583861d9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861d9f90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005c40 tx=0x7f5a38013000 comp rx=0 tx=0).stop 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 -- 172.21.15.45:0/2776196424 shutdown_connections 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x5583861da940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x5583861d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/2776196424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861d9f90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 -- 172.21.15.45:0/2776196424 >> 172.21.15.45:0/2776196424 conn(0x5583861d4880 msgr2=0x5583861d4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 -- 172.21.15.45:0/2776196424 shutdown_connections 2023-10-06T05:16:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.381+0000 7f5a593b0b80 1 -- 172.21.15.45:0/2776196424 wait complete. 2023-10-06T05:16:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.382+0000 7f5a593b0b80 1 -- start start 2023-10-06T05:16:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.382+0000 7f5a593b0b80 1 Processor -- start 2023-10-06T05:16:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861b6f40 0x55838618bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861da4d0 0x5583861ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583860da8e0 con 0x5583861da4d0 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558386086980 con 0x5583861b6f40 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558386086620 con 0x5583861d1450 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a593b0b80 10 client.0 _ll_register_callbacks cb 0x558385ee9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861da4d0 0x5583861ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4d637700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861da4d0 0x5583861ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34174/0 (socket says 172.21.15.45:34174) 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 -- 172.21.15.45:0/4079332340 learned_addr learned my addr 172.21.15.45:0/4079332340 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4de38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861b6f40 0x55838618bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 -- 172.21.15.45:0/4079332340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 msgr2=0x55838618c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 -- 172.21.15.45:0/4079332340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861b6f40 msgr2=0x55838618bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861b6f40 0x55838618bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 -- 172.21.15.45:0/4079332340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3800f040 con 0x5583861da4d0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a4e639700 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861da4d0 0x5583861ac520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5a40014380 tx=0x7f5a4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f39dae8a1e38fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.383+0000 7f5a27fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a27fff700 1 -- 172.21.15.45:0/4079332340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000e040 con 0x5583861da4d0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a27fff700 1 -- 172.21.15.45:0/4079332340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5a4001a070 con 0x5583861da4d0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55838617ed60 con 0x5583861da4d0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a27fff700 1 -- 172.21.15.45:0/4079332340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a40014ec0 con 0x5583861da4d0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 10 client.5292 fetch_fsmap learned FSMap version 16 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558385fcd0a0 con 0x5583861da4d0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a27fff700 1 -- 172.21.15.45:0/4079332340 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55838617ed60 con 0x5583861da4d0 2023-10-06T05:16:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a27fff700 1 -- 172.21.15.45:0/4079332340 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5a4001e630 con 0x5583861da4d0 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 10 client.5292 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 10 client.5292 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 20 client.5292 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861aca60 0x5583861b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 4 client.5292 mds_command: new command op to 5133 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 --> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55838615bb30 con 0x5583861aca60 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.384+0000 7f5a4de38700 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861aca60 0x5583861b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a27fff700 10 client.5292 ms_handle_connect on v2:172.21.15.45:6832/2918652845 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a4de38700 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861aca60 0x5583861b0de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc004f6580bab1df server_cookie=b5db9711efcaa8ef in_seq=0 out_seq=0 2023-10-06T05:16:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a27fff700 1 -- 172.21.15.45:0/4079332340 <== mds.0 v2:172.21.15.45:6832/2918652845 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55838615bb30 con 0x5583861aca60 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a27fff700 10 client.5292 handle_command_reply: tid=0 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 client.5292 shutdown 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861da4d0 msgr2=0x5583861ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861da4d0 0x5583861ac520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5a40014380 tx=0x7f5a4000b040 comp rx=0 tx=0).stop 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 shutdown_connections 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861aca60 0x5583861b0de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861da4d0 0x5583861ac520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861b6f40 0x55838618bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a27fff700 0 client.5292 ms_handle_reset on v2:172.21.15.45:6832/2918652845 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/4079332340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.385+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 >> 172.21.15.45:0/4079332340 conn(0x5583861d4880 msgr2=0x55838618d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.386+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 shutdown_connections 2023-10-06T05:16:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.386+0000 7f5a593b0b80 1 -- 172.21.15.45:0/4079332340 wait complete. 2023-10-06T05:16:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.386+0000 7f5a593b0b80 20 client.5292 trim_cache size 0 max 16384 2023-10-06T05:16:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.403+0000 7f5a593b0b80 1 Processor -- start 2023-10-06T05:16:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.403+0000 7f5a593b0b80 1 -- start start 2023-10-06T05:16:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x558386180870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x558386180db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558385f721e0 con 0x5583861b6f40 2023-10-06T05:16:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583861b46a0 con 0x5583861da4d0 2023-10-06T05:16:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583861ea970 con 0x5583861d1450 2023-10-06T05:16:16.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a4de38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a4de38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861ea430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34178/0 (socket says 172.21.15.45:34178) 2023-10-06T05:16:16.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a4de38700 1 -- 172.21.15.45:0/1658370622 learned_addr learned my addr 172.21.15.45:0/1658370622 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:16.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x558386180870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a4e639700 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x558386180db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.404+0000 7f5a4de38700 1 -- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 msgr2=0x558386180870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x558386180870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a4de38700 1 -- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 msgr2=0x558386180db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x558386180db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a4de38700 1 -- 172.21.15.45:0/1658370622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583860a93c0 con 0x5583861b6f40 2023-10-06T05:16:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861ea430 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b770 tx=0x7f5a4402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73810421e86a568d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a337fe700 1 -- 172.21.15.45:0/1658370622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44059b80 con 0x5583861b6f40 2023-10-06T05:16:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a337fe700 1 -- 172.21.15.45:0/1658370622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4405d8d0 con 0x5583861b6f40 2023-10-06T05:16:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a337fe700 1 -- 172.21.15.45:0/1658370622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44083ef0 con 0x5583861b6f40 2023-10-06T05:16:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1658370622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 msgr2=0x5583861ea430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861ea430 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b770 tx=0x7f5a4402e040 comp rx=0 tx=0).stop 2023-10-06T05:16:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1658370622 shutdown_connections 2023-10-06T05:16:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x558386180db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861ea430 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1658370622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x558386180870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.405+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1658370622 >> 172.21.15.45:0/1658370622 conn(0x5583861d4880 msgr2=0x5583861d4170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.406+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1658370622 shutdown_connections 2023-10-06T05:16:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.406+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1658370622 wait complete. 2023-10-06T05:16:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.406+0000 7f5a593b0b80 1 -- start start 2023-10-06T05:16:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.406+0000 7f5a593b0b80 1 Processor -- start 2023-10-06T05:16:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a593b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x55838618c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558386236850 con 0x5583861b6f40 2023-10-06T05:16:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558386230d50 con 0x5583861da4d0 2023-10-06T05:16:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a593b0b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558386237fa0 con 0x5583861d1450 2023-10-06T05:16:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a593b0b80 10 client.0 _ll_register_callbacks cb 0x558385ee9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a4de38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a4de38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34180/0 (socket says 172.21.15.45:34180) 2023-10-06T05:16:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a4de38700 1 -- 172.21.15.45:0/1382715368 learned_addr learned my addr 172.21.15.45:0/1382715368 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a4e639700 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x55838618c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.407+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a4de38700 1 -- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 msgr2=0x55838618bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a4de38700 1 -- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 msgr2=0x55838618c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x55838618c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a4de38700 1 -- 172.21.15.45:0/1382715368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a48151950 con 0x5583861b6f40 2023-10-06T05:16:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a4de38700 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861b1e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a44052e30 tx=0x7f5a44008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a77b9e04abbb453 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a27fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a27fff700 1 -- 172.21.15.45:0/1382715368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44083ef0 con 0x5583861b6f40 2023-10-06T05:16:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a3800e7c0 con 0x5583861b6f40 2023-10-06T05:16:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a27fff700 1 -- 172.21.15.45:0/1382715368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5a44055e20 con 0x5583861b6f40 2023-10-06T05:16:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a27fff700 1 -- 172.21.15.45:0/1382715368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a4405d440 con 0x5583861b6f40 2023-10-06T05:16:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 10 client.5298 fetch_fsmap learned FSMap version 16 2023-10-06T05:16:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558385f411d0 con 0x5583861b6f40 2023-10-06T05:16:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a27fff700 1 -- 172.21.15.45:0/1382715368 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800e7c0 con 0x5583861b6f40 2023-10-06T05:16:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a27fff700 1 -- 172.21.15.45:0/1382715368 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5a4405db20 con 0x5583861b6f40 2023-10-06T05:16:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 10 client.5298 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T05:16:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 10 client.5298 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 20 client.5298 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861af620 0x5583861af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 4 client.5298 mds_command: new command op to 5133 tid=0[{"prefix": "status"}] 2023-10-06T05:16:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.408+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 --> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55838607d8d0 con 0x5583861af620 2023-10-06T05:16:16.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861af620 0x5583861af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a4d637700 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861af620 0x5583861af9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cab35af05f7c94c server_cookie=369cbe377ea61ddc in_seq=0 out_seq=0 2023-10-06T05:16:16.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a27fff700 10 client.5298 ms_handle_connect on v2:172.21.15.45:6832/2918652845 2023-10-06T05:16:16.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a27fff700 1 -- 172.21.15.45:0/1382715368 <== mds.0 v2:172.21.15.45:6832/2918652845 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55838607d8d0 con 0x5583861af620 2023-10-06T05:16:16.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a27fff700 10 client.5298 handle_command_reply: tid=0 2023-10-06T05:16:16.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 client.5298 shutdown 2023-10-06T05:16:16.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 msgr2=0x5583861b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861b1e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a44052e30 tx=0x7f5a44008030 comp rx=0 tx=0).stop 2023-10-06T05:16:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 shutdown_connections 2023-10-06T05:16:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:6832/2918652845,v1:172.21.15.45:6834/2918652845] conn(0x5583861af620 0x5583861af9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583861da4d0 0x55838618c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583861b6f40 0x5583861b1e60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 --2- 172.21.15.45:0/1382715368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5583861d1450 0x55838618bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a27fff700 0 client.5298 ms_handle_reset on v2:172.21.15.45:6832/2918652845 2023-10-06T05:16:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 >> 172.21.15.45:0/1382715368 conn(0x5583861d4880 msgr2=0x55838618d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 shutdown_connections 2023-10-06T05:16:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.409+0000 7f5a593b0b80 1 -- 172.21.15.45:0/1382715368 wait complete. 2023-10-06T05:16:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a593b0b80 20 client.5298 trim_cache size 0 max 16384 2023-10-06T05:16:16.432 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:16:16.433 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:16:16.433 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:16:16.433 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5133, 2023-10-06T05:16:16.433 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:16:16.434 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:16:16.434 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:16:16.434 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.2203094270000001, 2023-10-06T05:16:16.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 16, 2023-10-06T05:16:16.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 38, 2023-10-06T05:16:16.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 38, 2023-10-06T05:16:16.436 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 14.299842879 2023-10-06T05:16:16.436 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a2c066f30 msgr2=0x7f5a2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a2c066f30 0x7f5a2c0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a480c3980 tx=0x7f5a38009040 comp rx=0 tx=0).stop 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a480c0170 msgr2=0x7f5a48151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a480c0170 0x7f5a48151410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5a4806aed0 tx=0x7f5a44017000 comp rx=0 tx=0).stop 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 shutdown_connections 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a2c066f30 0x7f5a2c0693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a48161c20 0x7f5a48068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a480c0170 0x7f5a48151410 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 --2- 172.21.15.45:0/1631826217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a480cd300 0x7f5a48068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 >> 172.21.15.45:0/1631826217 conn(0x7f5a4806f090 msgr2=0x7f5a480caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 shutdown_connections 2023-10-06T05:16:16.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.410+0000 7f5a5009c700 1 -- 172.21.15.45:0/1631826217 wait complete. 2023-10-06T05:16:16.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:16:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.570+0000 7ff5a0b3d700 1 Processor -- start 2023-10-06T05:16:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.570+0000 7ff5a0b3d700 1 -- start start 2023-10-06T05:16:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.571+0000 7ff5a0b3d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.571+0000 7ff5a0b3d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff59c157030 0x7ff59c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.571+0000 7ff5a0b3d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59c15b000 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.571+0000 7ff5a0b3d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59c0c9830 con 0x7ff59c157030 2023-10-06T05:16:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.571+0000 7ff5a0b3d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c15b2e0 con 0x7ff59c0c93c0 2023-10-06T05:16:16.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff59c157030 0x7ff59c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:51276/0 (socket says 172.21.15.45:51276) 2023-10-06T05:16:16.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff59c157030 0x7ff59c0cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:49454/0 (socket says 172.21.15.45:49454) 2023-10-06T05:16:16.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59a59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff59c0c8ff0 0x7ff59c0caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:59138/0 (socket says 172.21.15.45:59138) 2023-10-06T05:16:16.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 -- 172.21.15.45:0/959716554 learned_addr learned my addr 172.21.15.45:0/959716554 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:16.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 -- 172.21.15.45:0/959716554 >> v2:172.21.15.144:3300/0 conn(0x7ff59c157030 msgr2=0x7ff59c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 --2- 172.21.15.45:0/959716554 >> v2:172.21.15.144:3300/0 conn(0x7ff59c157030 0x7ff59c0cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 -- 172.21.15.45:0/959716554 >> v1:172.21.15.144:6789/0 conn(0x7ff59c0c8ff0 legacy=0x7ff59c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59959a700 1 -- 172.21.15.45:0/959716554 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263013201 0 0) 0x7ff59c15b000 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 -- 172.21.15.45:0/959716554 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c15b5c0 con 0x7ff59c0c93c0 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59ad9d700 1 --2- 172.21.15.45:0/959716554 >> v2:172.21.15.144:3300/0 conn(0x7ff59c157030 0x7ff59c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff599d9b700 1 --2- 172.21.15.45:0/959716554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c156af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5840099b0 tx=0x7ff584009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a15340903c1d0c4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59959a700 1 -- 172.21.15.45:0/959716554 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584009ea0 con 0x7ff59c0c93c0 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59959a700 1 -- 172.21.15.45:0/959716554 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58400b860 con 0x7ff59c0c93c0 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.572+0000 7ff59959a700 1 -- 172.21.15.45:0/959716554 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400bc00 con 0x7ff59c0c93c0 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/959716554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 msgr2=0x7ff59c156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/959716554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c156af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5840099b0 tx=0x7ff584009250 comp rx=0 tx=0).stop 2023-10-06T05:16:16.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/959716554 shutdown_connections 2023-10-06T05:16:16.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/959716554 >> v2:172.21.15.144:3300/0 conn(0x7ff59c157030 0x7ff59c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/959716554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c156af0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/959716554 >> 172.21.15.45:0/959716554 conn(0x7ff59c0cdf50 msgr2=0x7ff59c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/959716554 shutdown_connections 2023-10-06T05:16:16.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.573+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/959716554 wait complete. 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.575+0000 7ff5a0b3d700 1 Processor -- start 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.575+0000 7ff5a0b3d700 1 -- start start 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff5a0b3d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff59c0c8ff0 0x7ff59c149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff5a0b3d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff5a0b3d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c157030 0x7ff59c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff5a0b3d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c14ede0 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff5a0b3d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c14f010 con 0x7ff59c157030 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff5a0b3d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c14f2f0 con 0x7ff59c0c93c0 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff59c0c8ff0 0x7ff59c149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff59c0c8ff0 0x7ff59c149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34194/0 (socket says 172.21.15.45:34194) 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 -- 172.21.15.45:0/2626480723 learned_addr learned my addr 172.21.15.45:0/2626480723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59ad9d700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c157030 0x7ff59c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff599d9b700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 -- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 msgr2=0x7ff59c14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 -- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c157030 msgr2=0x7ff59c159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c157030 0x7ff59c159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.576+0000 7ff59a59c700 1 -- 172.21.15.45:0/2626480723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594009a20 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.577+0000 7ff59a59c700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff59c0c8ff0 0x7ff59c149840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c5b0 tx=0x7ff58c001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.577+0000 7ff5927fc700 1 -- 172.21.15.45:0/2626480723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c005a70 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.577+0000 7ff5927fc700 1 -- 172.21.15.45:0/2626480723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c005bd0 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.577+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5840094f0 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.577+0000 7ff5927fc700 1 -- 172.21.15.45:0/2626480723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0070d0 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.577+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c14f930 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.578+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c06f410 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.578+0000 7ff5927fc700 1 -- 172.21.15.45:0/2626480723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff58c007480 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.579+0000 7ff5927fc700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff57c066f30 0x7ff57c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.579+0000 7ff5927fc700 1 -- 172.21.15.45:0/2626480723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff58c083bf0 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.579+0000 7ff599d9b700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff57c066f30 0x7ff57c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.580+0000 7ff599d9b700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff57c066f30 0x7ff57c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5840099b0 tx=0x7ff584017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:16.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.584+0000 7ff5927fc700 1 -- 172.21.15.45:0/2626480723 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff58c050c80 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.832+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff59c04bb40 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.833+0000 7ff5927fc700 1 -- 172.21.15.45:0/2626480723 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff58c017070 con 0x7ff59c0c8ff0 2023-10-06T05:16:16.835 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:16:16.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff57c066f30 msgr2=0x7ff57c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff57c066f30 0x7ff57c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5840099b0 tx=0x7ff584017000 comp rx=0 tx=0).stop 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff59c0c8ff0 msgr2=0x7ff59c149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff59c0c8ff0 0x7ff59c149840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c5b0 tx=0x7ff58c001040 comp rx=0 tx=0).stop 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 shutdown_connections 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff57c066f30 0x7ff57c0693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c157030 0x7ff59c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff59c0c8ff0 0x7ff59c149840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 --2- 172.21.15.45:0/2626480723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff59c0c93c0 0x7ff59c14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 >> 172.21.15.45:0/2626480723 conn(0x7ff59c0cdf50 msgr2=0x7ff59c0cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 shutdown_connections 2023-10-06T05:16:16.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:16.835+0000 7ff5a0b3d700 1 -- 172.21.15.45:0/2626480723 wait complete. 2023-10-06T05:16:16.850 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:16:16.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:16:16.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:16:16.851 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:16:16.851 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:16.853 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:16:16.853 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:16:16.853 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:16:16.853 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:16.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:16.854 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:16:16.876 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:16:16.876 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:16:16.876 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:16:16.876 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:16.876 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:16:16.876 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:16:16.877 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:16.878 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:16:16.878 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:16:16.878 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42364sec preferred_lft 42364sec 2023-10-06T05:16:16.878 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:16:16.878 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:16.878 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:16.878 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:16.879 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:16.880 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:16:16.880 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:16:16.880 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:16.880 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:16.880 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:16:16.880 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:16:16.881 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:16.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:16.881 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:16:16.947 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:16:16.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:16.948 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:16:17.015 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:16:17.015 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:16:17.015 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:16:17.015 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:16:17.015 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:16:17.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:17.016 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:16:17.016 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:16:17.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:16:17.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:16:17.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:16:17.016 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:16:17.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:17.141 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:16:17.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:16:17.162 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:16:17.162 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:17.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:17.163 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:16:17.240 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:16:17.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:17.240 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:17.302 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:17.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:17.303 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:17.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:16:17.442 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:16:17.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:16:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:16:17.518 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:16:17.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:17.519 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:16:17.583 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:16:17.583 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:16:17.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:16:17.661 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:16:17.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:16:17.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:16:17.727+0000 7f85e7739540 -1 init, newargv = 0x55c93730ea80 newargc=15 2023-10-06T05:16:17.729 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[136185]: starting ceph client 2023-10-06T05:16:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:16:17.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[136185]: starting fuse 2023-10-06T05:16:17.736 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:16:17.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:17.736 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:16:17.797 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:16:17.798 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:16:17.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:16:17.798 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:16:17.798 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:16:17.798 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:16:17.798 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:16:17.799 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:16:17.800 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:16:17.936 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.136185.asok 2023-10-06T05:16:18.117 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.136185.asok 2023-10-06T05:16:18.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:18.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.136185.asok status 2023-10-06T05:16:18.295 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:16:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:16:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:16:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:16:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:16:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:16:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "136185", 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5304, 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 5304 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3147982297 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3147982297 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.5304 192.168.0.1:0/3147982297", 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/3147982297", 2023-10-06T05:16:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:16:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 16, 2023-10-06T05:16:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 38, 2023-10-06T05:16:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:16:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:16:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:16:18.298 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:16:18.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:18.306 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:16:18.306 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:18.327 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:16:18.327 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:18.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:18.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:18.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:18.400 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2023-10-06T05:16:18.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:18.472 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2023-10-06T05:16:18.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:18.540 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2023-10-06T05:16:18.605 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:16:18.605 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:16:18.606 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:16:18.606 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:18.606 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2023-10-06T05:16:18.606 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:16:18.728 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2023-10-06T05:16:18.915 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:16:18.916 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:16:18.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:18.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[136185]: fuse finished with error 0 and tester_r 0 2023-10-06T05:16:18.998 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:16:25.100 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:16:25.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:25.100 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:16:25.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:25.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.304+0000 7f06b4722700 1 Processor -- start 2023-10-06T05:16:25.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.304+0000 7f06b4722700 1 -- start start 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b4722700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06ac158f60 0x7f06ac159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b4722700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06ac06a060 0x7f06ac0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b4722700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac0cc390 0x7f06ac0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b4722700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac06a500 con 0x7f06ac158f60 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b4722700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac0ccd50 con 0x7f06ac06a060 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b4722700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac069bd0 con 0x7f06ac0cc390 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b24be700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06ac158f60 0x7f06ac159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b24be700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06ac158f60 0x7f06ac159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34198/0 (socket says 172.21.15.45:34198) 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.305+0000 7f06b24be700 1 -- 172.21.15.45:0/1653906200 learned_addr learned my addr 172.21.15.45:0/1653906200 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b1cbd700 1 --2- 172.21.15.45:0/1653906200 >> v2:172.21.15.144:3300/0 conn(0x7f06ac06a060 0x7f06ac0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b2cbf700 1 --2- 172.21.15.45:0/1653906200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac0cc390 0x7f06ac0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b24be700 1 -- 172.21.15.45:0/1653906200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac0cc390 msgr2=0x7f06ac0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b24be700 1 --2- 172.21.15.45:0/1653906200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac0cc390 0x7f06ac0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b24be700 1 -- 172.21.15.45:0/1653906200 >> v2:172.21.15.144:3300/0 conn(0x7f06ac06a060 msgr2=0x7f06ac0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b24be700 1 --2- 172.21.15.45:0/1653906200 >> v2:172.21.15.144:3300/0 conn(0x7f06ac06a060 0x7f06ac0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b24be700 1 -- 172.21.15.45:0/1653906200 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac068fc0 con 0x7f06ac158f60 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b24be700 1 --2- 172.21.15.45:0/1653906200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac159330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0872e0 tx=0x7f06a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dd87df3a9318bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b14bc700 1 -- 172.21.15.45:0/1653906200 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800b640 con 0x7f06ac158f60 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b14bc700 1 -- 172.21.15.45:0/1653906200 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800b7a0 con 0x7f06ac158f60 2023-10-06T05:16:25.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.306+0000 7f06b14bc700 1 -- 172.21.15.45:0/1653906200 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800bb40 con 0x7f06ac158f60 2023-10-06T05:16:25.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 -- 172.21.15.45:0/1653906200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 msgr2=0x7f06ac159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 --2- 172.21.15.45:0/1653906200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac159330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0872e0 tx=0x7f06a8009250 comp rx=0 tx=0).stop 2023-10-06T05:16:25.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 -- 172.21.15.45:0/1653906200 shutdown_connections 2023-10-06T05:16:25.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 --2- 172.21.15.45:0/1653906200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac0cc390 0x7f06ac0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 --2- 172.21.15.45:0/1653906200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac159330 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 --2- 172.21.15.45:0/1653906200 >> v2:172.21.15.144:3300/0 conn(0x7f06ac06a060 0x7f06ac0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 -- 172.21.15.45:0/1653906200 >> 172.21.15.45:0/1653906200 conn(0x7f06ac0cdf50 msgr2=0x7f06ac0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 -- 172.21.15.45:0/1653906200 shutdown_connections 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.307+0000 7f06b4722700 1 -- 172.21.15.45:0/1653906200 wait complete. 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.308+0000 7f06b4722700 1 Processor -- start 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.308+0000 7f06b4722700 1 -- start start 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b4722700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac06a060 0x7f06ac0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b4722700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac0cc390 0x7f06ac14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b4722700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b4722700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0aa4a0 con 0x7f06ac158f60 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b4722700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0a9890 con 0x7f06ac0cc390 2023-10-06T05:16:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b4722700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0a9b70 con 0x7f06ac06a060 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac14de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34212/0 (socket says 172.21.15.45:34212) 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 -- 172.21.15.45:0/1283617839 learned_addr learned my addr 172.21.15.45:0/1283617839 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b24be700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac06a060 0x7f06ac0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b1cbd700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac0cc390 0x7f06ac14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 -- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac06a060 msgr2=0x7f06ac0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac06a060 0x7f06ac0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 -- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac0cc390 msgr2=0x7f06ac14d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac0cc390 0x7f06ac14d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.309+0000 7f06b2cbf700 1 -- 172.21.15.45:0/1283617839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80094f0 con 0x7f06ac158f60 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.310+0000 7f06b2cbf700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac14de60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009e90 tx=0x7f06a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.310+0000 7f069e7fc700 1 -- 172.21.15.45:0/1283617839 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a400cb70 con 0x7f06ac158f60 2023-10-06T05:16:25.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.310+0000 7f069e7fc700 1 -- 172.21.15.45:0/1283617839 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a400f040 con 0x7f06ac158f60 2023-10-06T05:16:25.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.310+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac0a9e50 con 0x7f06ac158f60 2023-10-06T05:16:25.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.310+0000 7f069e7fc700 1 -- 172.21.15.45:0/1283617839 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4014480 con 0x7f06ac158f60 2023-10-06T05:16:25.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.310+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac0addd0 con 0x7f06ac158f60 2023-10-06T05:16:25.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.311+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac06f410 con 0x7f06ac158f60 2023-10-06T05:16:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.311+0000 7f069e7fc700 1 -- 172.21.15.45:0/1283617839 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06a4014830 con 0x7f06ac158f60 2023-10-06T05:16:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.312+0000 7f069e7fc700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0690066f30 0x7f06900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.312+0000 7f069e7fc700 1 -- 172.21.15.45:0/1283617839 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f06a40851a0 con 0x7f06ac158f60 2023-10-06T05:16:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.312+0000 7f06b24be700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0690066f30 0x7f06900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.313+0000 7f06b24be700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0690066f30 0x7f06900693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0872e0 tx=0x7f06a8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.316+0000 7f069e7fc700 1 -- 172.21.15.45:0/1283617839 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06a4052170 con 0x7f06ac158f60 2023-10-06T05:16:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.567+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06ac0692a0 con 0x7f06ac158f60 2023-10-06T05:16:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.568+0000 7f069e7fc700 1 -- 172.21.15.45:0/1283617839 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f06a4055790 con 0x7f06ac158f60 2023-10-06T05:16:25.569 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:25.570 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:13.214114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5133":{"gid":5133,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6834/2918652845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2918652845},{"type":"v1","addr":"172.21.15.45:6834","nonce":2918652845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:16:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0690066f30 msgr2=0x7f06900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0690066f30 0x7f06900693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0872e0 tx=0x7f06a8017000 comp rx=0 tx=0).stop 2023-10-06T05:16:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 msgr2=0x7f06ac14de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac14de60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009e90 tx=0x7f06a400b040 comp rx=0 tx=0).stop 2023-10-06T05:16:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 shutdown_connections 2023-10-06T05:16:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0690066f30 0x7f06900693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06ac158f60 0x7f06ac14de60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06ac06a060 0x7f06ac0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 --2- 172.21.15.45:0/1283617839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06ac0cc390 0x7f06ac14d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 >> 172.21.15.45:0/1283617839 conn(0x7f06ac0cdf50 msgr2=0x7f06ac0ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 shutdown_connections 2023-10-06T05:16:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.570+0000 7f06b4722700 1 -- 172.21.15.45:0/1283617839 wait complete. 2023-10-06T05:16:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 16 2023-10-06T05:16:25.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-10-06T05:16:25.755 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2023-10-06T05:16:25.755 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-06T05:16:25.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:16:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.888+0000 7f9ae7f92700 1 Processor -- start 2023-10-06T05:16:25.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.888+0000 7f9ae7f92700 1 -- start start 2023-10-06T05:16:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.889+0000 7f9ae7f92700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ae0159330 0x7f9ae006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.889+0000 7f9ae7f92700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 0x7f9ae00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.889+0000 7f9ae7f92700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae00cb530 con 0x7f9ae0158f60 2023-10-06T05:16:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.889+0000 7f9ae7f92700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae00ccd50 con 0x7f9ae0159330 2023-10-06T05:16:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.889+0000 7f9ae7f92700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0069bd0 con 0x7f9ae00cc390 2023-10-06T05:16:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.889+0000 7f9ae5d2e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9ae0158f60 0x7f9ae00cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52054/0 (socket says 172.21.15.45:52054) 2023-10-06T05:16:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.889+0000 7f9ae5d2e700 1 -- 172.21.15.45:0/2307211694 learned_addr learned my addr 172.21.15.45:0/2307211694 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae552d700 1 --2- 172.21.15.45:0/2307211694 >> v2:172.21.15.144:3300/0 conn(0x7f9ae0159330 0x7f9ae006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae652f700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 0x7f9ae00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682505390 0 0) 0x7f9ad0001110 con 0x7f9ae0158f60 2023-10-06T05:16:25.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc003570 con 0x7f9ae0158f60 2023-10-06T05:16:25.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 413391963 0 0) 0x7f9acc003570 con 0x7f9ae0158f60 2023-10-06T05:16:25.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 msgr2=0x7f9ae00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 0x7f9ae00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 >> v2:172.21.15.144:3300/0 conn(0x7f9ae0159330 msgr2=0x7f9ae006a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 --2- 172.21.15.45:0/2307211694 >> v2:172.21.15.144:3300/0 conn(0x7f9ae0159330 0x7f9ae006a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0068fc0 con 0x7f9ae0158f60 2023-10-06T05:16:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f9ad0001600 con 0x7f9ae0158f60 2023-10-06T05:16:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 >> v1:172.21.15.45:6789/0 conn(0x7f9ae0158f60 legacy=0x7f9ae00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc003a60 0x7f9acc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 0x7f9acc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc004d50 0x7f9acc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00ccd50 con 0x7f9acc004d50 2023-10-06T05:16:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0069bd0 con 0x7f9acc003a60 2023-10-06T05:16:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc00ba60 con 0x7f9ae00cc390 2023-10-06T05:16:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae652f700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc004d50 0x7f9acc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae5d2e700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc003a60 0x7f9acc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.890+0000 7f9ae552d700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 0x7f9acc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae652f700 1 -- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 msgr2=0x7f9acc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae652f700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae00cc390 0x7f9acc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae652f700 1 -- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc003a60 msgr2=0x7f9acc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae652f700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc003a60 0x7f9acc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae652f700 1 -- 172.21.15.45:0/2307211694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc003570 con 0x7f9acc004d50 2023-10-06T05:16:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae652f700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc004d50 0x7f9acc0051c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800e210 tx=0x7f9ad800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9909946ca3bb1708 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800ea60 con 0x7f9acc004d50 2023-10-06T05:16:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad800f040 con 0x7f9acc004d50 2023-10-06T05:16:25.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.891+0000 7f9ae4d2c700 1 -- 172.21.15.45:0/2307211694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800ee10 con 0x7f9acc004d50 2023-10-06T05:16:25.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.892+0000 7f9ae7f92700 1 -- 172.21.15.45:0/2307211694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc004d50 msgr2=0x7f9acc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.892+0000 7f9ae7f92700 1 --2- 172.21.15.45:0/2307211694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc004d50 0x7f9acc0051c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800e210 tx=0x7f9ad800b040 comp rx=0 tx=0).stop 2023-10-06T05:16:25.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.892+0000 7f9ae652f700 1 -- 172.21.15.45:0/2307211694 reap_dead start 2023-10-06T05:16:25.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.892+0000 7f9ae7f92700 1 -- 172.21.15.45:0/2307211694 shutdown_connections 2023-10-06T05:16:25.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.892+0000 7f9ae7f92700 1 -- 172.21.15.45:0/2307211694 >> 172.21.15.45:0/2307211694 conn(0x7f9ae00cdf50 msgr2=0x7f9ae00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.892+0000 7f9ae7f92700 1 -- 172.21.15.45:0/2307211694 shutdown_connections 2023-10-06T05:16:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.892+0000 7f9ae7f92700 1 -- 172.21.15.45:0/2307211694 wait complete. 2023-10-06T05:16:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.893+0000 7f9ae7f92700 1 Processor -- start 2023-10-06T05:16:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.893+0000 7f9ae7f92700 1 -- start start 2023-10-06T05:16:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.894+0000 7f9ae7f92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae0158190 0x7f9ae015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.894+0000 7f9ae7f92700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae0158560 0x7f9ae015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.894+0000 7f9ae7f92700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae015e6b0 0x7f9ae0169400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.894+0000 7f9ae7f92700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015eb30 con 0x7f9ae0158560 2023-10-06T05:16:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.894+0000 7f9ae7f92700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015cc10 con 0x7f9ae0158190 2023-10-06T05:16:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.894+0000 7f9ae7f92700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015d5e0 con 0x7f9ae015e6b0 2023-10-06T05:16:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae0158560 0x7f9ae015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae0158560 0x7f9ae015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34228/0 (socket says 172.21.15.45:34228) 2023-10-06T05:16:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 -- 172.21.15.45:0/1848908676 learned_addr learned my addr 172.21.15.45:0/1848908676 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae5d2e700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae0158190 0x7f9ae015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae652f700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae015e6b0 0x7f9ae0169400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 -- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae015e6b0 msgr2=0x7f9ae0169400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae015e6b0 0x7f9ae0169400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 -- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae0158190 msgr2=0x7f9ae015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae0158190 0x7f9ae015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 -- 172.21.15.45:0/1848908676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae015c000 con 0x7f9ae0158560 2023-10-06T05:16:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae552d700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae0158560 0x7f9ae015e170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006ac0 tx=0x7f9adc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/1848908676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc011ec0 con 0x7f9ae0158560 2023-10-06T05:16:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/1848908676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc00f940 con 0x7f9ae0158560 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae015c2e0 con 0x7f9ae0158560 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.895+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/1848908676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00fcc0 con 0x7f9ae0158560 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.896+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae015c8a0 con 0x7f9ae0158560 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.897+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae015cd80 con 0x7f9ae0158560 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.897+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/1848908676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9adc014650 con 0x7f9ae0158560 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.897+0000 7f9ad5ffb700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac4066f30 0x7f9ac40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.898+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/1848908676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9adc018030 con 0x7f9ae0158560 2023-10-06T05:16:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.898+0000 7f9ae5d2e700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac4066f30 0x7f9ac40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.898+0000 7f9ae5d2e700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac4066f30 0x7f9ac40693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0001730 tx=0x7f9ad0007a80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:25.902+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/1848908676 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9adc055ad0 con 0x7f9ae0158560 2023-10-06T05:16:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.149+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9ae004bb40 con 0x7f9ae0158560 2023-10-06T05:16:26.151 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:16:26.149+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:16:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.153+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/1848908676 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9adc022090 con 0x7f9ae0158560 2023-10-06T05:16:26.157 INFO:tasks.ceph.mds.c.smithi045.stderr: -3192> 2023-10-06T05:16:13.960+0000 7f4f0ee13700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac4066f30 msgr2=0x7f9ac40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac4066f30 0x7f9ac40693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0001730 tx=0x7f9ad0007a80 comp rx=0 tx=0).stop 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae0158560 msgr2=0x7f9ae015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae0158560 0x7f9ae015e170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006ac0 tx=0x7f9adc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 shutdown_connections 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac4066f30 0x7f9ac40693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae015e6b0 0x7f9ae0169400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae0158190 0x7f9ae015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 --2- 172.21.15.45:0/1848908676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae0158560 0x7f9ae015e170 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 >> 172.21.15.45:0/1848908676 conn(0x7f9ae00cdf50 msgr2=0x7f9ae00c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 shutdown_connections 2023-10-06T05:16:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.155+0000 7f9ae7f92700 1 -- 172.21.15.45:0/1848908676 wait complete. 2023-10-06T05:16:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:16:26.171 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:16:26.184 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:16:26.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.218+0000 7f0ac0569040 1 Processor -- start 2023-10-06T05:16:26.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.218+0000 7f0ac0569040 1 -- start start 2023-10-06T05:16:26.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.219+0000 7f0ac0569040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683072750 0x56168311b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ac0569040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683101930 0x56168311ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ac0569040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x56168306eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ac0569040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561682f8f980 con 0x5616831183c0 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ac0569040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561682f8f620 con 0x561683072750 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ac0569040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561682f86400 con 0x561683101930 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x56168306eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab8f86700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683101930 0x56168311ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9787700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683072750 0x56168311b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x56168306eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34256/0 (socket says 172.21.15.45:34256) 2023-10-06T05:16:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab8f86700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683101930 0x56168311ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:51380/0 (socket says 172.21.15.45:51380) 2023-10-06T05:16:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9787700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683072750 0x56168311b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:49522/0 (socket says 172.21.15.45:49522) 2023-10-06T05:16:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 -- 172.21.15.45:0/4023171476 learned_addr learned my addr 172.21.15.45:0/4023171476 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 -- 172.21.15.45:0/4023171476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683101930 msgr2=0x56168311ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 --2- 172.21.15.45:0/4023171476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683101930 0x56168311ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 -- 172.21.15.45:0/4023171476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683072750 msgr2=0x56168311b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:26.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 --2- 172.21.15.45:0/4023171476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683072750 0x56168311b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 -- 172.21.15.45:0/4023171476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac005db0 con 0x5616831183c0 2023-10-06T05:16:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.220+0000 7f0ab9f88700 1 --2- 172.21.15.45:0/4023171476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x56168306eff0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b7e0 tx=0x7f0ab4005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c384691f46af7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0aab7fe700 1 -- 172.21.15.45:0/4023171476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4012680 con 0x5616831183c0 2023-10-06T05:16:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0aab7fe700 1 -- 172.21.15.45:0/4023171476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40127e0 con 0x5616831183c0 2023-10-06T05:16:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0aab7fe700 1 -- 172.21.15.45:0/4023171476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4012ba0 con 0x5616831183c0 2023-10-06T05:16:26.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 -- 172.21.15.45:0/4023171476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 msgr2=0x56168306eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 --2- 172.21.15.45:0/4023171476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x56168306eff0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b7e0 tx=0x7f0ab4005d40 comp rx=0 tx=0).stop 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 -- 172.21.15.45:0/4023171476 shutdown_connections 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 --2- 172.21.15.45:0/4023171476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x56168306eff0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 --2- 172.21.15.45:0/4023171476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683072750 0x56168311b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 --2- 172.21.15.45:0/4023171476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683101930 0x56168311ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 -- 172.21.15.45:0/4023171476 >> 172.21.15.45:0/4023171476 conn(0x561683044870 msgr2=0x561683100670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 -- 172.21.15.45:0/4023171476 shutdown_connections 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.221+0000 7f0ac0569040 1 -- 172.21.15.45:0/4023171476 wait complete. 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.222+0000 7f0ac0569040 1 Processor -- start 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.222+0000 7f0ac0569040 1 -- start start 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ac0569040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683072750 0x561683100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ac0569040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683101930 0x5616830fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ac0569040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x5616830faf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ac0569040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616830fa5b0 con 0x5616831183c0 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ac0569040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561683115110 con 0x561683101930 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ac0569040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561683100d50 con 0x561683072750 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x5616830faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x5616830faf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34262/0 (socket says 172.21.15.45:34262) 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 -- 172.21.15.45:0/3807252968 learned_addr learned my addr 172.21.15.45:0/3807252968 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9787700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683072750 0x561683100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab8f86700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683101930 0x5616830fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 -- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683072750 msgr2=0x561683100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561683072750 0x561683100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 -- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683101930 msgr2=0x5616830fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561683101930 0x5616830fcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.223+0000 7f0ab9f88700 1 -- 172.21.15.45:0/3807252968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab400b430 con 0x5616831183c0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.224+0000 7f0ab9f88700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x5616830faf20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b6d0 tx=0x7f0ab4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.224+0000 7f0aa9ffb700 1 -- 172.21.15.45:0/3807252968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4012680 con 0x5616831183c0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.224+0000 7f0aa9ffb700 1 -- 172.21.15.45:0/3807252968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4012b60 con 0x5616831183c0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.224+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561682f9e3d0 con 0x5616831183c0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.224+0000 7f0aa9ffb700 1 -- 172.21.15.45:0/3807252968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab401b6a0 con 0x5616831183c0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.224+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561682f91ae0 con 0x5616831183c0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.225+0000 7f0aa9ffb700 1 -- 172.21.15.45:0/3807252968 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ab401ba50 con 0x5616831183c0 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.226+0000 7f0aa9ffb700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a9c065890 0x7f0a9c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.226+0000 7f0aa9ffb700 1 -- 172.21.15.45:0/3807252968 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0ab40858f0 con 0x5616831183c0 2023-10-06T05:16:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.226+0000 7f0ab9787700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a9c065890 0x7f0a9c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.226+0000 7f0ac0569040 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56168308bea0 0x561683113c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.226+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39) v8 -- 0x5616831141b0 con 0x56168308bea0 2023-10-06T05:16:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.226+0000 7f0ab8f86700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56168308bea0 0x561683113c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.227+0000 7f0ab9787700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a9c065890 0x7f0a9c067d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00a5a0 tx=0x7f0aac013000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.227+0000 7f0ab8f86700 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56168308bea0 0x561683113c70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ab8f86700 1 -- 172.21.15.45:0/3807252968 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v39'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0ab000f770 con 0x56168308bea0 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56168308bea0 msgr2=0x561683113c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ac0569040 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56168308bea0 0x561683113c70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a9c065890 msgr2=0x7f0a9c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ac0569040 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a9c065890 0x7f0a9c067d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00a5a0 tx=0x7f0aac013000 comp rx=0 tx=0).stop 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 msgr2=0x5616830faf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ac0569040 1 --2- 172.21.15.45:0/3807252968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616831183c0 0x5616830faf20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b6d0 tx=0x7f0ab4018000 comp rx=0 tx=0).stop 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.230+0000 7f0ab9f88700 1 -- 172.21.15.45:0/3807252968 reap_dead start 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.231+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 shutdown_connections 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.231+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 >> 172.21.15.45:0/3807252968 conn(0x561683044870 msgr2=0x561683070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.231+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 shutdown_connections 2023-10-06T05:16:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.231+0000 7f0ac0569040 1 -- 172.21.15.45:0/3807252968 wait complete. 2023-10-06T05:16:26.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:16:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.412+0000 7f4238151700 1 Processor -- start 2023-10-06T05:16:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.412+0000 7f4238151700 1 -- start start 2023-10-06T05:16:26.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f4238151700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42300d02b0 0x7f42300d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f4238151700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f423015d820 0x7f423015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f4238151700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f423015c6f0 0x7f42300afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f4238151700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42300cd740 con 0x7f423015d820 2023-10-06T05:16:26.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f4238151700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42300ccb30 con 0x7f423015c6f0 2023-10-06T05:16:26.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f4238151700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42300cce10 con 0x7f42300d02b0 2023-10-06T05:16:26.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f42356ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f423015d820 0x7f423015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f42356ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f423015d820 0x7f423015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38078/0 (socket says 172.21.15.45:38078) 2023-10-06T05:16:26.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f42356ec700 1 -- 172.21.15.45:0/960081240 learned_addr learned my addr 172.21.15.45:0/960081240 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:26.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f4235eed700 1 --2- 172.21.15.45:0/960081240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42300d02b0 0x7f42300d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.413+0000 7f42366ee700 1 --2- 172.21.15.45:0/960081240 >> v2:172.21.15.144:3300/0 conn(0x7f423015c6f0 0x7f42300afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f42356ec700 1 -- 172.21.15.45:0/960081240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42300d02b0 msgr2=0x7f42300d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:26.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f42356ec700 1 --2- 172.21.15.45:0/960081240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42300d02b0 0x7f42300d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f42356ec700 1 -- 172.21.15.45:0/960081240 >> v2:172.21.15.144:3300/0 conn(0x7f423015c6f0 msgr2=0x7f42300afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:26.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f42356ec700 1 --2- 172.21.15.45:0/960081240 >> v2:172.21.15.144:3300/0 conn(0x7f423015c6f0 0x7f42300afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f42356ec700 1 -- 172.21.15.45:0/960081240 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42300cd0f0 con 0x7f423015d820 2023-10-06T05:16:26.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f42356ec700 1 --2- 172.21.15.45:0/960081240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f423015c1b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f422c0099b0 tx=0x7f422c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140f5c3e28a4ba26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f4234eeb700 1 -- 172.21.15.45:0/960081240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c00b640 con 0x7f423015d820 2023-10-06T05:16:26.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f4234eeb700 1 -- 172.21.15.45:0/960081240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c00b7a0 con 0x7f423015d820 2023-10-06T05:16:26.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.414+0000 7f4234eeb700 1 -- 172.21.15.45:0/960081240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c00bb40 con 0x7f423015d820 2023-10-06T05:16:26.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 -- 172.21.15.45:0/960081240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 msgr2=0x7f423015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 --2- 172.21.15.45:0/960081240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f423015c1b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f422c0099b0 tx=0x7f422c009250 comp rx=0 tx=0).stop 2023-10-06T05:16:26.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 -- 172.21.15.45:0/960081240 shutdown_connections 2023-10-06T05:16:26.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 --2- 172.21.15.45:0/960081240 >> v2:172.21.15.144:3300/0 conn(0x7f423015c6f0 0x7f42300afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 --2- 172.21.15.45:0/960081240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42300d02b0 0x7f42300d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 --2- 172.21.15.45:0/960081240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f423015c1b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 -- 172.21.15.45:0/960081240 >> 172.21.15.45:0/960081240 conn(0x7f423006ed50 msgr2=0x7f423006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 -- 172.21.15.45:0/960081240 shutdown_connections 2023-10-06T05:16:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.415+0000 7f4238151700 1 -- 172.21.15.45:0/960081240 wait complete. 2023-10-06T05:16:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.416+0000 7f4238151700 1 Processor -- start 2023-10-06T05:16:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.416+0000 7f4238151700 1 -- start start 2023-10-06T05:16:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4238151700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42300d02b0 0x7f42300c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4238151700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f423015c6f0 0x7f42300ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4238151700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f42300a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4238151700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42300ca5f0 con 0x7f423015d820 2023-10-06T05:16:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4238151700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42300a70f0 con 0x7f42300d02b0 2023-10-06T05:16:26.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4238151700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42300c9010 con 0x7f423015c6f0 2023-10-06T05:16:26.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f42300a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f42300a6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38086/0 (socket says 172.21.15.45:38086) 2023-10-06T05:16:26.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 -- 172.21.15.45:0/2517702065 learned_addr learned my addr 172.21.15.45:0/2517702065 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:26.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42356ec700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f423015c6f0 0x7f42300ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4235eed700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42300d02b0 0x7f42300c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 -- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f423015c6f0 msgr2=0x7f42300ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f423015c6f0 0x7f42300ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 -- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42300d02b0 msgr2=0x7f42300c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:26.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42300d02b0 0x7f42300c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:26.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 -- 172.21.15.45:0/2517702065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0094f0 con 0x7f423015d820 2023-10-06T05:16:26.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42356ec700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f423015c6f0 0x7f42300ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:26.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f4235eed700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42300d02b0 0x7f42300c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:26.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.417+0000 7f42366ee700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f42300a6bb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4228009e90 tx=0x7f422800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.418+0000 7f4221ffb700 1 -- 172.21.15.45:0/2517702065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422800cc20 con 0x7f423015d820 2023-10-06T05:16:26.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.418+0000 7f4221ffb700 1 -- 172.21.15.45:0/2517702065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422800f040 con 0x7f423015d820 2023-10-06T05:16:26.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.418+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42300c92f0 con 0x7f423015d820 2023-10-06T05:16:26.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.418+0000 7f4221ffb700 1 -- 172.21.15.45:0/2517702065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228014480 con 0x7f423015d820 2023-10-06T05:16:26.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.418+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42300c98b0 con 0x7f423015d820 2023-10-06T05:16:26.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.419+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4230088730 con 0x7f423015d820 2023-10-06T05:16:26.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.420+0000 7f4221ffb700 1 -- 172.21.15.45:0/2517702065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4228014830 con 0x7f423015d820 2023-10-06T05:16:26.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.420+0000 7f4221ffb700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4214066f30 0x7f42140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:26.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.420+0000 7f4221ffb700 1 -- 172.21.15.45:0/2517702065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f42280851f0 con 0x7f423015d820 2023-10-06T05:16:26.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.420+0000 7f4235eed700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4214066f30 0x7f42140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:26.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.421+0000 7f4235eed700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4214066f30 0x7f42140693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4224006f00 tx=0x7f4224008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:26.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.425+0000 7f4221ffb700 1 -- 172.21.15.45:0/2517702065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4228055c00 con 0x7f423015d820 2023-10-06T05:16:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:26.677+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f42300cd400 con 0x7f423015d820 2023-10-06T05:16:27.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.159+0000 7f4221ffb700 1 -- 172.21.15.45:0/2517702065 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v18) v1 ==== 162+0+0 (secure 0 0 0) 0x7f4228055790 con 0x7f423015d820 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4214066f30 msgr2=0x7f42140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4214066f30 0x7f42140693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4224006f00 tx=0x7f4224008040 comp rx=0 tx=0).stop 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 msgr2=0x7f42300a6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f42300a6bb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4228009e90 tx=0x7f422800b040 comp rx=0 tx=0).stop 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 shutdown_connections 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4214066f30 0x7f42140693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f423015d820 0x7f42300a6bb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42300d02b0 0x7f42300c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 --2- 172.21.15.45:0/2517702065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f423015c6f0 0x7f42300ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 >> 172.21.15.45:0/2517702065 conn(0x7f423006ed50 msgr2=0x7f42300cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 shutdown_connections 2023-10-06T05:16:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.163+0000 7f4238151700 1 -- 172.21.15.45:0/2517702065 wait complete. 2023-10-06T05:16:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:16:27.182 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:27.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.315+0000 7f2cb0613700 1 Processor -- start 2023-10-06T05:16:27.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.315+0000 7f2cb0613700 1 -- start start 2023-10-06T05:16:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cb0613700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca80c93c0 0x7f2ca8156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cb0613700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ca8157030 0x7f2ca80cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cb0613700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca815b000 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cb0613700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80c9830 con 0x7f2ca8157030 2023-10-06T05:16:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cb0613700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca815b2e0 con 0x7f2ca80c93c0 2023-10-06T05:16:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2caebb0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ca8157030 0x7f2ca80cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cae3af700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2ca80c8ff0 0x7f2ca80caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33994/0 (socket says 172.21.15.45:33994) 2023-10-06T05:16:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cae3af700 1 -- 172.21.15.45:0/3959923411 learned_addr learned my addr 172.21.15.45:0/3959923411 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cadbae700 1 --2- 172.21.15.45:0/3959923411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca80c93c0 0x7f2ca8156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2caebb0700 1 -- 172.21.15.45:0/3959923411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca80c93c0 msgr2=0x7f2ca8156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2caebb0700 1 --2- 172.21.15.45:0/3959923411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca80c93c0 0x7f2ca8156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2caebb0700 1 -- 172.21.15.45:0/3959923411 >> v1:172.21.15.45:6789/0 conn(0x7f2ca80c8ff0 legacy=0x7f2ca80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2caebb0700 1 -- 172.21.15.45:0/3959923411 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca815b5c0 con 0x7f2ca8157030 2023-10-06T05:16:27.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.316+0000 7f2cad3ad700 1 -- 172.21.15.45:0/3959923411 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603269093 0 0) 0x7f2ca815b000 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.317+0000 7f2caebb0700 1 --2- 172.21.15.45:0/3959923411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8157030 0x7f2ca80cc330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009e60 tx=0x7f2ca0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac75b04a135f6b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.317+0000 7f2cad3ad700 1 -- 172.21.15.45:0/3959923411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000e070 con 0x7f2ca8157030 2023-10-06T05:16:27.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.317+0000 7f2cad3ad700 1 -- 172.21.15.45:0/3959923411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca000bb60 con 0x7f2ca8157030 2023-10-06T05:16:27.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.317+0000 7f2cad3ad700 1 -- 172.21.15.45:0/3959923411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0012430 con 0x7f2ca8157030 2023-10-06T05:16:27.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.317+0000 7f2cb0613700 1 -- 172.21.15.45:0/3959923411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8157030 msgr2=0x7f2ca80cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.317+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3959923411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8157030 0x7f2ca80cc330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009e60 tx=0x7f2ca0009780 comp rx=0 tx=0).stop 2023-10-06T05:16:27.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.318+0000 7f2cb0613700 1 -- 172.21.15.45:0/3959923411 shutdown_connections 2023-10-06T05:16:27.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.318+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3959923411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8157030 0x7f2ca80cc330 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.318+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3959923411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca80c93c0 0x7f2ca8156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.318+0000 7f2cb0613700 1 -- 172.21.15.45:0/3959923411 >> 172.21.15.45:0/3959923411 conn(0x7f2ca80cdf50 msgr2=0x7f2ca80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:27.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.318+0000 7f2cb0613700 1 -- 172.21.15.45:0/3959923411 shutdown_connections 2023-10-06T05:16:27.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.318+0000 7f2cb0613700 1 -- 172.21.15.45:0/3959923411 wait complete. 2023-10-06T05:16:27.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.319+0000 7f2cb0613700 1 Processor -- start 2023-10-06T05:16:27.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.319+0000 7f2cb0613700 1 -- start start 2023-10-06T05:16:27.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cb0613700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca80c8ff0 0x7f2ca80b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cb0613700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ca80c93c0 0x7f2ca80c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cb0613700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca8157030 0x7f2ca814ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cb0613700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8158ed0 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cb0613700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81588f0 con 0x7f2ca80c93c0 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cb0613700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80b0f80 con 0x7f2ca8157030 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca80c8ff0 0x7f2ca80b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca80c8ff0 0x7f2ca80b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38096/0 (socket says 172.21.15.45:38096) 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 -- 172.21.15.45:0/3226014691 learned_addr learned my addr 172.21.15.45:0/3226014691 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2caebb0700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca8157030 0x7f2ca814ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cadbae700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ca80c93c0 0x7f2ca80c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 -- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca8157030 msgr2=0x7f2ca814ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca8157030 0x7f2ca814ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 -- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ca80c93c0 msgr2=0x7f2ca80c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ca80c93c0 0x7f2ca80c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 -- 172.21.15.45:0/3226014691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009a20 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cadbae700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ca80c93c0 0x7f2ca80c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:27.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2cae3af700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca80c8ff0 0x7f2ca80b31d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c5e0 tx=0x7f2ca4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.320+0000 7f2c9e7fc700 1 -- 172.21.15.45:0/3226014691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4005b30 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.321+0000 7f2c9e7fc700 1 -- 172.21.15.45:0/3226014691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca4005c90 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.321+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca80b3890 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.321+0000 7f2c9e7fc700 1 -- 172.21.15.45:0/3226014691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40072e0 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.321+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca80b3b70 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.322+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8158750 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.322+0000 7f2c9e7fc700 1 -- 172.21.15.45:0/3226014691 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ca401b3b0 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.323+0000 7f2c9e7fc700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2c8c066f30 0x7f2c8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.323+0000 7f2c9e7fc700 1 -- 172.21.15.45:0/3226014691 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f2ca4083f20 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.323+0000 7f2cadbae700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2c8c066f30 0x7f2c8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.324+0000 7f2cadbae700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2c8c066f30 0x7f2c8c0693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2c98006f00 tx=0x7f2c98008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:27.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.327+0000 7f2c9e7fc700 1 -- 172.21.15.45:0/3226014691 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ca4050fd0 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.574+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ca8158750 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.575+0000 7f2c9e7fc700 1 -- 172.21.15.45:0/3226014691 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f2ca8158750 con 0x7f2ca80c8ff0 2023-10-06T05:16:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:27.577 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:27.160790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:16:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2c8c066f30 msgr2=0x7f2c8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2c8c066f30 0x7f2c8c0693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2c98006f00 tx=0x7f2c98008040 comp rx=0 tx=0).stop 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca80c8ff0 msgr2=0x7f2ca80b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca80c8ff0 0x7f2ca80b31d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c5e0 tx=0x7f2ca4001070 comp rx=0 tx=0).stop 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 shutdown_connections 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2c8c066f30 0x7f2c8c0693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ca8157030 0x7f2ca814ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca80c8ff0 0x7f2ca80b31d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 --2- 172.21.15.45:0/3226014691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ca80c93c0 0x7f2ca80c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 >> 172.21.15.45:0/3226014691 conn(0x7f2ca80cdf50 msgr2=0x7f2ca8150060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 shutdown_connections 2023-10-06T05:16:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:27.577+0000 7f2cb0613700 1 -- 172.21.15.45:0/3226014691 wait complete. 2023-10-06T05:16:27.581 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2023-10-06T05:16:27.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:16:09.624552+0000', 'modified': '2023-10-06T05:16:27.160790+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5322}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5322': {'gid': 5322, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6834/1705605429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 1705605429}, {'type': 'v1', 'addr': '172.21.15.45:6834', 'nonce': 1705605429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:16:27.996 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5322:up:replay 2023-10-06T05:16:28.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.136+0000 7f8bddfe5700 1 Processor -- start 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.136+0000 7f8bddfe5700 1 -- start start 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bddfe5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd8067830 0x7f8bd806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bddfe5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd80b2120 con 0x7f8bd815a490 2023-10-06T05:16:29.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bddfe5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd80ab7d0 con 0x7f8bd806add0 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bddfe5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80ae020 con 0x7f8bd8067830 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8bd815a490 0x7f8bd815b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34002/0 (socket says 172.21.15.45:34002) 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd77fe700 1 -- 172.21.15.45:0/662484428 learned_addr learned my addr 172.21.15.45:0/662484428 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd6ffd700 1 --2- 172.21.15.45:0/662484428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd8067830 0x7f8bd806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439200485 0 0) 0x7f8bc8001110 con 0x7f8bd815a490 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bc0003570 con 0x7f8bd815a490 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714910567 0 0) 0x7f8bd80ab7d0 con 0x7f8bd806add0 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bc8001110 con 0x7f8bd806add0 2023-10-06T05:16:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 989887762 0 0) 0x7f8bc0003570 con 0x7f8bd815a490 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd6ffd700 1 -- 172.21.15.45:0/662484428 >> v1:172.21.15.144:6789/0 conn(0x7f8bd806add0 legacy=0x7f8bd80abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd6ffd700 1 -- 172.21.15.45:0/662484428 >> v1:172.21.15.45:6789/0 conn(0x7f8bd815a490 legacy=0x7f8bd815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.137+0000 7f8bd6ffd700 1 -- 172.21.15.45:0/662484428 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd8067e80 con 0x7f8bd8067830 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bd6ffd700 1 --2- 172.21.15.45:0/662484428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd8067830 0x7f8bd806a890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009910 tx=0x7f8bcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42f54354dc87dfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8001600 con 0x7f8bd8067830 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc011420 con 0x7f8bd8067830 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bd67fc700 1 -- 172.21.15.45:0/662484428 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0117c0 con 0x7f8bd8067830 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bddfe5700 1 -- 172.21.15.45:0/662484428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd8067830 msgr2=0x7f8bd806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/662484428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd8067830 0x7f8bd806a890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009910 tx=0x7f8bcc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bddfe5700 1 -- 172.21.15.45:0/662484428 shutdown_connections 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/662484428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd8067830 0x7f8bd806a890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bddfe5700 1 -- 172.21.15.45:0/662484428 >> 172.21.15.45:0/662484428 conn(0x7f8bd80cc5a0 msgr2=0x7f8bd80cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.138+0000 7f8bddfe5700 1 -- 172.21.15.45:0/662484428 shutdown_connections 2023-10-06T05:16:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.139+0000 7f8bddfe5700 1 -- 172.21.15.45:0/662484428 wait complete. 2023-10-06T05:16:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.140+0000 7f8bddfe5700 1 Processor -- start 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.140+0000 7f8bddfe5700 1 -- start start 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.141+0000 7f8bddfe5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd8067830 0x7f8bd8168200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.141+0000 7f8bddfe5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bd806add0 0x7f8bd80c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.141+0000 7f8bddfe5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd815a490 0x7f8bd815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.141+0000 7f8bddfe5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8067ca0 con 0x7f8bd806add0 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.141+0000 7f8bddfe5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd815b850 con 0x7f8bd8067830 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.141+0000 7f8bddfe5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8168740 con 0x7f8bd815a490 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.141+0000 7f8bd6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bd806add0 0x7f8bd80c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bd806add0 0x7f8bd80c1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38104/0 (socket says 172.21.15.45:38104) 2023-10-06T05:16:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 -- 172.21.15.45:0/1911487408 learned_addr learned my addr 172.21.15.45:0/1911487408 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd77fe700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd8067830 0x7f8bd8168200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd7fff700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd815a490 0x7f8bd815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 -- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd815a490 msgr2=0x7f8bd815c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd815a490 0x7f8bd815c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 -- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd8067830 msgr2=0x7f8bd8168200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd8067830 0x7f8bd8168200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 -- 172.21.15.45:0/1911487408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009420 con 0x7f8bd806add0 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bd6ffd700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bd806add0 0x7f8bd80c1730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0030c0 tx=0x7f8bcc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bbf7fe700 1 -- 172.21.15.45:0/1911487408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc01caf0 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bbf7fe700 1 -- 172.21.15.45:0/1911487408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc01cc50 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bbf7fe700 1 -- 172.21.15.45:0/1911487408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc01a450 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd815ac40 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.142+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd815b200 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.143+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd815b9e0 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.144+0000 7f8bbf7fe700 1 -- 172.21.15.45:0/1911487408 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bcc01a840 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.144+0000 7f8bbf7fe700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bc4066f30 0x7f8bc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.144+0000 7f8bbf7fe700 1 -- 172.21.15.45:0/1911487408 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8bcc0864a0 con 0x7f8bd806add0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.145+0000 7f8bd77fe700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bc4066f30 0x7f8bc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.145+0000 7f8bd77fe700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bc4066f30 0x7f8bc40693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80ac880 tx=0x7f8bc8001330 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.149+0000 7f8bbf7fe700 1 -- 172.21.15.45:0/1911487408 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bcc0533a0 con 0x7f8bd806add0 2023-10-06T05:16:29.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.402+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bd80cc0e0 con 0x7f8bd806add0 2023-10-06T05:16:29.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.402+0000 7f8bbf7fe700 1 -- 172.21.15.45:0/1911487408 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f8bcc0569c0 con 0x7f8bd806add0 2023-10-06T05:16:29.404 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:29.404 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:29.164439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:16:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bc4066f30 msgr2=0x7f8bc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bc4066f30 0x7f8bc40693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80ac880 tx=0x7f8bc8001330 comp rx=0 tx=0).stop 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bd806add0 msgr2=0x7f8bd80c1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bd806add0 0x7f8bd80c1730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0030c0 tx=0x7f8bcc016000 comp rx=0 tx=0).stop 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 shutdown_connections 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bc4066f30 0x7f8bc40693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd815a490 0x7f8bd815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd8067830 0x7f8bd8168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 --2- 172.21.15.45:0/1911487408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bd806add0 0x7f8bd80c1730 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.404+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 >> 172.21.15.45:0/1911487408 conn(0x7f8bd80cc5a0 msgr2=0x7f8bd80af350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.405+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 shutdown_connections 2023-10-06T05:16:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:29.405+0000 7f8bddfe5700 1 -- 172.21.15.45:0/1911487408 wait complete. 2023-10-06T05:16:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2023-10-06T05:16:29.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:16:09.624552+0000', 'modified': '2023-10-06T05:16:29.164439+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5322}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5322': {'gid': 5322, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.45:6834/1705605429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 1705605429}, {'type': 'v1', 'addr': '172.21.15.45:6834', 'nonce': 1705605429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:16:29.822 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5322:up:rejoin 2023-10-06T05:16:30.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:30.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.963+0000 7fc14d2b3700 1 Processor -- start 2023-10-06T05:16:30.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.963+0000 7fc14d2b3700 1 -- start start 2023-10-06T05:16:30.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc14d2b3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc14d2b3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc148156b70 0x7fc148156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc14d2b3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1480b0790 con 0x7fc14806c3d0 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc14d2b3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1480b0570 con 0x7fc148156b70 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc14d2b3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480c8e80 con 0x7fc14815b890 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1477fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc148156b70 0x7fc148156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1477fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc148156b70 0x7fc148156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38110/0 (socket says 172.21.15.45:38110) 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1477fe700 1 -- 172.21.15.45:0/187632324 learned_addr learned my addr 172.21.15.45:0/187632324 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1467fc700 1 --2- 172.21.15.45:0/187632324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1477fe700 1 -- 172.21.15.45:0/187632324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815b890 msgr2=0x7fc14806c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1477fe700 1 --2- 172.21.15.45:0/187632324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1477fe700 1 -- 172.21.15.45:0/187632324 >> v1:172.21.15.144:6789/0 conn(0x7fc14806c3d0 legacy=0x7fc1480afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.964+0000 7fc1477fe700 1 -- 172.21.15.45:0/187632324 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1480c9160 con 0x7fc148156b70 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc1477fe700 1 --2- 172.21.15.45:0/187632324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc148156ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc140009f20 tx=0x7fc140009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb8e7ec6223dd9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc145ffb700 1 -- 172.21.15.45:0/187632324 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14000e070 con 0x7fc148156b70 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc145ffb700 1 -- 172.21.15.45:0/187632324 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14000ba80 con 0x7fc148156b70 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc145ffb700 1 -- 172.21.15.45:0/187632324 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1400092e0 con 0x7fc148156b70 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 -- 172.21.15.45:0/187632324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 msgr2=0x7fc148156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/187632324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc148156ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc140009f20 tx=0x7fc140009840 comp rx=0 tx=0).stop 2023-10-06T05:16:30.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 -- 172.21.15.45:0/187632324 shutdown_connections 2023-10-06T05:16:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/187632324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc148156ff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/187632324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 -- 172.21.15.45:0/187632324 >> 172.21.15.45:0/187632324 conn(0x7fc148067620 msgr2=0x7fc148067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 -- 172.21.15.45:0/187632324 shutdown_connections 2023-10-06T05:16:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.965+0000 7fc14d2b3700 1 -- 172.21.15.45:0/187632324 wait complete. 2023-10-06T05:16:30.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.967+0000 7fc14d2b3700 1 Processor -- start 2023-10-06T05:16:30.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.967+0000 7fc14d2b3700 1 -- start start 2023-10-06T05:16:30.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc14d2b3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14806c3d0 0x7fc1480c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc14d2b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc1481599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc14d2b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc14d2b3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14816b490 con 0x7fc148156b70 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc14d2b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14815a3d0 con 0x7fc14815b890 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc14d2b3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14816ae40 con 0x7fc14806c3d0 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc1481599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc1481599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38114/0 (socket says 172.21.15.45:38114) 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 -- 172.21.15.45:0/2258612400 learned_addr learned my addr 172.21.15.45:0/2258612400 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc146ffd700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14806c3d0 0x7fc1480c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1477fe700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 -- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14806c3d0 msgr2=0x7fc1480c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14806c3d0 0x7fc1480c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 -- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc14815b890 msgr2=0x7fc14806c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 -- 172.21.15.45:0/2258612400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc140009ae0 con 0x7fc148156b70 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc146ffd700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14806c3d0 0x7fc1480c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc1467fc700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc1481599f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00d860 tx=0x7fc13c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc12effd700 1 -- 172.21.15.45:0/2258612400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0125f0 con 0x7fc148156b70 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc12effd700 1 -- 172.21.15.45:0/2258612400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c012750 con 0x7fc148156b70 2023-10-06T05:16:30.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc12effd700 1 -- 172.21.15.45:0/2258612400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c012b10 con 0x7fc148156b70 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.968+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14816b120 con 0x7fc148156b70 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.969+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14815a6b0 con 0x7fc148156b70 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.970+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14816b600 con 0x7fc148156b70 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.970+0000 7fc12effd700 1 -- 172.21.15.45:0/2258612400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc13c01e460 con 0x7fc148156b70 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.970+0000 7fc12effd700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc134066f60 0x7fc134069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.971+0000 7fc12effd700 1 -- 172.21.15.45:0/2258612400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc13c084dd0 con 0x7fc148156b70 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.971+0000 7fc146ffd700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc134066f60 0x7fc134069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:30.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.971+0000 7fc146ffd700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc134066f60 0x7fc134069410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc138001840 tx=0x7fc138009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:30.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:30.975+0000 7fc12effd700 1 -- 172.21.15.45:0/2258612400 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc13c051d50 con 0x7fc148156b70 2023-10-06T05:16:31.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.223+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1480c9440 con 0x7fc148156b70 2023-10-06T05:16:31.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.224+0000 7fc12effd700 1 -- 172.21.15.45:0/2258612400 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fc13c086020 con 0x7fc148156b70 2023-10-06T05:16:31.226 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:31.226 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:30.171347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:16:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc134066f60 msgr2=0x7fc134069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc134066f60 0x7fc134069410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc138001840 tx=0x7fc138009040 comp rx=0 tx=0).stop 2023-10-06T05:16:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 msgr2=0x7fc1481599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc1481599f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00d860 tx=0x7fc13c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 shutdown_connections 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc134066f60 0x7fc134069410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc14815b890 0x7fc14806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14806c3d0 0x7fc1480c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 --2- 172.21.15.45:0/2258612400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc148156b70 0x7fc1481599f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 >> 172.21.15.45:0/2258612400 conn(0x7fc148067620 msgr2=0x7fc1481563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 shutdown_connections 2023-10-06T05:16:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.226+0000 7fc14d2b3700 1 -- 172.21.15.45:0/2258612400 wait complete. 2023-10-06T05:16:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-10-06T05:16:31.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:16:09.624552+0000', 'modified': '2023-10-06T05:16:30.171347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5322}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5322': {'gid': 5322, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6834/1705605429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 1705605429}, {'type': 'v1', 'addr': '172.21.15.45:6834', 'nonce': 1705605429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:16:31.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:16:31.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:16:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.785+0000 7f8ebde84700 1 Processor -- start 2023-10-06T05:16:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.785+0000 7f8ebde84700 1 -- start start 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.785+0000 7f8ebde84700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb8159200 0x7f8eb80c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8ebde84700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb81595d0 0x7f8eb806b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8ebde84700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80c7d00 con 0x7f8eb806b5c0 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8ebde84700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80c7780 con 0x7f8eb8159200 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8ebde84700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c86d0 con 0x7f8eb81595d0 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb8159200 0x7f8eb80c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb8159200 0x7f8eb80c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38128/0 (socket says 172.21.15.45:38128) 2023-10-06T05:16:31.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 -- 172.21.15.45:0/448562586 learned_addr learned my addr 172.21.15.45:0/448562586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/448562586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb81595d0 0x7f8eb806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 -- 172.21.15.45:0/448562586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb81595d0 msgr2=0x7f8eb806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/448562586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb81595d0 0x7f8eb806b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 -- 172.21.15.45:0/448562586 >> v1:172.21.15.144:6789/0 conn(0x7f8eb806b5c0 legacy=0x7f8eb80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 -- 172.21.15.45:0/448562586 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb806bb40 con 0x7f8eb8159200 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/448562586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8159200 0x7f8eb80c8190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009f20 tx=0x7f8ea8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cea8a65e09e92cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb67fc700 1 -- 172.21.15.45:0/448562586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800e070 con 0x7f8eb8159200 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.786+0000 7f8eb67fc700 1 -- 172.21.15.45:0/448562586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea800ba80 con 0x7f8eb8159200 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8eb67fc700 1 -- 172.21.15.45:0/448562586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80092e0 con 0x7f8eb8159200 2023-10-06T05:16:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 -- 172.21.15.45:0/448562586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8159200 msgr2=0x7f8eb80c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 --2- 172.21.15.45:0/448562586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8159200 0x7f8eb80c8190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009f20 tx=0x7f8ea8009840 comp rx=0 tx=0).stop 2023-10-06T05:16:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 -- 172.21.15.45:0/448562586 shutdown_connections 2023-10-06T05:16:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 --2- 172.21.15.45:0/448562586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8159200 0x7f8eb80c8190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 --2- 172.21.15.45:0/448562586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb81595d0 0x7f8eb806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 -- 172.21.15.45:0/448562586 >> 172.21.15.45:0/448562586 conn(0x7f8eb806f090 msgr2=0x7f8eb806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 -- 172.21.15.45:0/448562586 shutdown_connections 2023-10-06T05:16:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.787+0000 7f8ebde84700 1 -- 172.21.15.45:0/448562586 wait complete. 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.788+0000 7f8ebde84700 1 Processor -- start 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.788+0000 7f8ebde84700 1 -- start start 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8ebde84700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb806b5c0 0x7f8eb80c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8ebde84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb8159200 0x7f8eb80c0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8ebde84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb81595d0 0x7f8eb80c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8ebde84700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80afbe0 con 0x7f8eb81595d0 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8ebde84700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80b0480 con 0x7f8eb8159200 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8ebde84700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c1400 con 0x7f8eb806b5c0 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb81595d0 0x7f8eb80c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8eb77fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb806b5c0 0x7f8eb80c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb81595d0 0x7f8eb80c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38144/0 (socket says 172.21.15.45:38144) 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3882557362 learned_addr learned my addr 172.21.15.45:0/3882557362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.789+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb8159200 0x7f8eb80c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb806b5c0 msgr2=0x7f8eb80c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb806b5c0 0x7f8eb80c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb8159200 msgr2=0x7f8eb80c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb8159200 0x7f8eb80c0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3882557362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8009ae0 con 0x7f8eb81595d0 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb81595d0 0x7f8eb80c0dd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000c590 tx=0x7f8eb0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8e9f7fe700 1 -- 172.21.15.45:0/3882557362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0011590 con 0x7f8eb81595d0 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8e9f7fe700 1 -- 172.21.15.45:0/3882557362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb00116f0 con 0x7f8eb81595d0 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8e9f7fe700 1 -- 172.21.15.45:0/3882557362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0011ab0 con 0x7f8eb81595d0 2023-10-06T05:16:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb80afe30 con 0x7f8eb81595d0 2023-10-06T05:16:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.790+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80b2920 con 0x7f8eb81595d0 2023-10-06T05:16:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.792+0000 7f8e9f7fe700 1 -- 172.21.15.45:0/3882557362 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8eb001b3b0 con 0x7f8eb81595d0 2023-10-06T05:16:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.792+0000 7f8e9f7fe700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ea4066f30 0x7f8ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.792+0000 7f8e9f7fe700 1 -- 172.21.15.45:0/3882557362 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8eb0083be0 con 0x7f8eb81595d0 2023-10-06T05:16:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.792+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ea4066f30 0x7f8ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.793+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ea4066f30 0x7f8ea40693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009880 tx=0x7f8ea8018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.794+0000 7f8ec7198b80 1 Processor -- start 2023-10-06T05:16:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.794+0000 7f8ec7198b80 1 -- start start 2023-10-06T05:16:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.795+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ae2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.795+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798ac7f40 0x556798aeaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.795+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798aeb4d0 0x556798aeb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.795+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567989b5310 con 0x556798ac7f40 2023-10-06T05:16:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.795+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567989f2130 con 0x556798aeb4d0 2023-10-06T05:16:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.795+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567989efaf0 con 0x556798ae2450 2023-10-06T05:16:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.795+0000 7f8eb6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798ac7f40 0x556798aeaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798ac7f40 0x556798aeaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38156/0 (socket says 172.21.15.45:38156) 2023-10-06T05:16:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 -- 172.21.15.45:0/68219526 learned_addr learned my addr 172.21.15.45:0/68219526 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ae2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798aeb4d0 0x556798aeb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 -- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 msgr2=0x556798ae2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ae2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 -- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798aeb4d0 msgr2=0x556798aeb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798aeb4d0 0x556798aeb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 -- 172.21.15.45:0/68219526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567987f1790 con 0x556798ac7f40 2023-10-06T05:16:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798ac7f40 0x556798aeaf90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ba40 tx=0x7f8eac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe8da95e136a2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8e9cff9700 1 -- 172.21.15.45:0/68219526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0108f0 con 0x556798ac7f40 2023-10-06T05:16:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8e9cff9700 1 -- 172.21.15.45:0/68219526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac010a50 con 0x556798ac7f40 2023-10-06T05:16:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.796+0000 7f8e9cff9700 1 -- 172.21.15.45:0/68219526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac010df0 con 0x556798ac7f40 2023-10-06T05:16:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 -- 172.21.15.45:0/68219526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798ac7f40 msgr2=0x556798aeaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798ac7f40 0x556798aeaf90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ba40 tx=0x7f8eac008040 comp rx=0 tx=0).stop 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 -- 172.21.15.45:0/68219526 shutdown_connections 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798aeb4d0 0x556798aeb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ae2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/68219526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798ac7f40 0x556798aeaf90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 -- 172.21.15.45:0/68219526 >> 172.21.15.45:0/68219526 conn(0x556798ae5880 msgr2=0x556798ae5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 -- 172.21.15.45:0/68219526 shutdown_connections 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.797+0000 7f8ec7198b80 1 -- 172.21.15.45:0/68219526 wait complete. 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.798+0000 7f8ec7198b80 1 -- start start 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.798+0000 7f8ec7198b80 1 Processor -- start 2023-10-06T05:16:31.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ac7f40 0x556798ae51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ac1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567989eb8e0 con 0x556798aeb4d0 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556798997980 con 0x556798ac7f40 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556798997620 con 0x556798ae2450 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 10 client.0 _ll_register_callbacks cb 0x5567987faa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38168/0 (socket says 172.21.15.45:38168) 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 -- 172.21.15.45:0/165174312 learned_addr learned my addr 172.21.15.45:0/165174312 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ac1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ac7f40 0x556798ae51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 -- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 msgr2=0x556798ac1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ac1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 -- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ac7f40 msgr2=0x556798ae51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ac7f40 0x556798ae51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 -- 172.21.15.45:0/165174312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac00e040 con 0x556798aeb4d0 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9c7e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0054760 tx=0x7f8eb001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcff33d5397233c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8e95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8e95ffb700 1 -- 172.21.15.45:0/165174312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0054360 con 0x556798aeb4d0 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8e95ffb700 1 -- 172.21.15.45:0/165174312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8eb006ae40 con 0x556798aeb4d0 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.799+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556798a8fd60 con 0x556798aeb4d0 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8e95ffb700 1 -- 172.21.15.45:0/165174312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb0070250 con 0x556798aeb4d0 2023-10-06T05:16:31.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 10 client.5373 fetch_fsmap learned FSMap version 22 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5567988de0a0 con 0x556798aeb4d0 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8e95ffb700 1 -- 172.21.15.45:0/165174312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556798a8fd60 con 0x556798aeb4d0 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8e95ffb700 1 -- 172.21.15.45:0/165174312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8eb0053d10 con 0x556798aeb4d0 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 10 client.5373 fetch_fsmap finished waiting for FSMap version 22 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 10 client.5373 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 20 client.5373 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798ae3ae0 0x556798ae3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 4 client.5373 mds_command: new command op to 5322 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556798a6cb30 con 0x556798ae3ae0 2023-10-06T05:16:31.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.800+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798ae3ae0 0x556798ae3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8e95ffb700 10 client.5373 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798ae3ae0 0x556798ae3eb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd4ec9e40a241fd server_cookie=d640f4ad2bfeeeea in_seq=0 out_seq=0 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8e95ffb700 1 -- 172.21.15.45:0/165174312 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556798a6cb30 con 0x556798ae3ae0 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8e95ffb700 10 client.5373 handle_command_reply: tid=0 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 client.5373 shutdown 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 msgr2=0x556798a9c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9c7e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0054760 tx=0x7f8eb001e040 comp rx=0 tx=0).stop 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 shutdown_connections 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798ae3ae0 0x556798ae3eb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9c7e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ac7f40 0x556798ae51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8e95ffb700 0 client.5373 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:31.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/165174312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ae2450 0x556798ac1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 >> 172.21.15.45:0/165174312 conn(0x556798ae5880 msgr2=0x556798a97c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 shutdown_connections 2023-10-06T05:16:31.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.801+0000 7f8ec7198b80 1 -- 172.21.15.45:0/165174312 wait complete. 2023-10-06T05:16:31.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.802+0000 7f8ec7198b80 20 client.5373 trim_cache size 0 max 16384 2023-10-06T05:16:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 Processor -- start 2023-10-06T05:16:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 -- start start 2023-10-06T05:16:31.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798a91d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798a9da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567988831e0 con 0x556798aeb4d0 2023-10-06T05:16:31.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556798a93950 con 0x556798ae2450 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.820+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556798a98a10 con 0x556798ac7f40 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38178/0 (socket says 172.21.15.45:38178) 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 -- 172.21.15.45:0/2082461334 learned_addr learned my addr 172.21.15.45:0/2082461334 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798a91d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798a9da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 -- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 msgr2=0x556798a91d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798a91d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 -- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 msgr2=0x556798a9da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798a9da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 -- 172.21.15.45:0/2082461334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567989ba3c0 con 0x556798aeb4d0 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9dfb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0083ec0 tx=0x7f8eb0003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ef87dce9c5c239 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8e9d7fa700 1 -- 172.21.15.45:0/2082461334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb005b820 con 0x556798aeb4d0 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8e9d7fa700 1 -- 172.21.15.45:0/2082461334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0054360 con 0x556798aeb4d0 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.821+0000 7f8e9d7fa700 1 -- 172.21.15.45:0/2082461334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0070250 con 0x556798aeb4d0 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 -- 172.21.15.45:0/2082461334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 msgr2=0x556798a9dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9dfb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0083ec0 tx=0x7f8eb0003000 comp rx=0 tx=0).stop 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 -- 172.21.15.45:0/2082461334 shutdown_connections 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798a9dfb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798a91d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/2082461334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798a9da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 -- 172.21.15.45:0/2082461334 >> 172.21.15.45:0/2082461334 conn(0x556798ae5880 msgr2=0x556798abe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 -- 172.21.15.45:0/2082461334 shutdown_connections 2023-10-06T05:16:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 -- 172.21.15.45:0/2082461334 wait complete. 2023-10-06T05:16:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 -- start start 2023-10-06T05:16:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.822+0000 7f8ec7198b80 1 Processor -- start 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798aca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798ae5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8ec7198b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798ae6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556798a92320 con 0x556798aeb4d0 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556798ac38a0 con 0x556798ae2450 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8ec7198b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556798a9ddb0 con 0x556798ac7f40 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8ec7198b80 10 client.0 _ll_register_callbacks cb 0x5567987faa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798ae6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798ae6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38180/0 (socket says 172.21.15.45:38180) 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3703460244 learned_addr learned my addr 172.21.15.45:0/3703460244 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798aca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb6ffd700 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798ae5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 msgr2=0x556798aca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798aca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 msgr2=0x556798ae5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798ae5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 -- 172.21.15.45:0/3703460244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567988de0a0 con 0x556798aeb4d0 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8eb7fff700 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798ae6190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0054520 tx=0x7f8eb0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aeec19a3617cc81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.824+0000 7f8e95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8e95ffb700 1 -- 172.21.15.45:0/3703460244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0052960 con 0x556798aeb4d0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8e95ffb700 1 -- 172.21.15.45:0/3703460244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8eb006ae40 con 0x556798aeb4d0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eac010a50 con 0x556798aeb4d0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8e95ffb700 1 -- 172.21.15.45:0/3703460244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb0050610 con 0x556798aeb4d0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 10 client.5379 fetch_fsmap learned FSMap version 22 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5567988521d0 con 0x556798aeb4d0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8e95ffb700 1 -- 172.21.15.45:0/3703460244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0064020 con 0x556798aeb4d0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8e95ffb700 1 -- 172.21.15.45:0/3703460244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8eb0050770 con 0x556798aeb4d0 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 10 client.5379 fetch_fsmap finished waiting for FSMap version 22 2023-10-06T05:16:31.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 10 client.5379 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 20 client.5379 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798b47c40 0x556798a9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 4 client.5379 mds_command: new command op to 5322 tid=0[{"prefix": "status"}] 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55679898e8d0 con 0x556798b47c40 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.825+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798b47c40 0x556798a9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8e95ffb700 10 client.5379 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8eb77fe700 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798b47c40 0x556798a9afd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b1c9dc07b2a8d9 server_cookie=2e38162b8f8f5dc8 in_seq=0 out_seq=0 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8e95ffb700 1 -- 172.21.15.45:0/3703460244 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55679898e8d0 con 0x556798b47c40 2023-10-06T05:16:31.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8e95ffb700 10 client.5379 handle_command_reply: tid=0 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 client.5379 shutdown 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 msgr2=0x556798ae6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798ae6190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0054520 tx=0x7f8eb0014070 comp rx=0 tx=0).stop 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 shutdown_connections 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x556798b47c40 0x556798a9afd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556798aeb4d0 0x556798ae6190 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556798ac7f40 0x556798aca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 --2- 172.21.15.45:0/3703460244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556798ae2450 0x556798ae5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8e95ffb700 0 client.5379 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 >> 172.21.15.45:0/3703460244 conn(0x556798ae5880 msgr2=0x556798abe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 shutdown_connections 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 1 -- 172.21.15.45:0/3703460244 wait complete. 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.826+0000 7f8ec7198b80 20 client.5379 trim_cache size 0 max 16384 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5322, 2023-10-06T05:16:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.6619920029999999, 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 22, 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 39, 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 39, 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 5.6416076650000004 2023-10-06T05:16:31.837 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:16:31.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ea4066f30 msgr2=0x7f8ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ea4066f30 0x7f8ea40693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009880 tx=0x7f8ea8018000 comp rx=0 tx=0).stop 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb81595d0 msgr2=0x7f8eb80c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb81595d0 0x7f8eb80c0dd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000c590 tx=0x7f8eb0007ef0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 shutdown_connections 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ea4066f30 0x7f8ea40693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb81595d0 0x7f8eb80c0dd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8eb806b5c0 0x7f8eb80c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 --2- 172.21.15.45:0/3882557362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8eb8159200 0x7f8eb80c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 >> 172.21.15.45:0/3882557362 conn(0x7f8eb806f090 msgr2=0x7f8eb80cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 shutdown_connections 2023-10-06T05:16:31.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.827+0000 7f8ebde84700 1 -- 172.21.15.45:0/3882557362 wait complete. 2023-10-06T05:16:31.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:16:31.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.972+0000 7f12f6e32700 1 Processor -- start 2023-10-06T05:16:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.972+0000 7f12f6e32700 1 -- start start 2023-10-06T05:16:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f6e32700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12f00c8ff0 0x7f12f00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f6e32700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f015b000 con 0x7f12f0157200 2023-10-06T05:16:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f6e32700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f015b2e0 con 0x7f12f00caad0 2023-10-06T05:16:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f6e32700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f015b5c0 con 0x7f12f00c8ff0 2023-10-06T05:16:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f4bce700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12f00c8ff0 0x7f12f00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f4bce700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12f00c8ff0 0x7f12f00c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38188/0 (socket says 172.21.15.45:38188) 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f53cf700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f12f0157200 0x7f12f00cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34018/0 (socket says 172.21.15.45:34018) 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f4bce700 1 -- 172.21.15.45:0/1157532714 learned_addr learned my addr 172.21.15.45:0/1157532714 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f4bce700 1 -- 172.21.15.45:0/1157532714 >> v1:172.21.15.144:6789/0 conn(0x7f12f00caad0 legacy=0x7f12f0156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f4bce700 1 -- 172.21.15.45:0/1157532714 >> v1:172.21.15.45:6789/0 conn(0x7f12f0157200 legacy=0x7f12f00cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12ef7fe700 1 -- 172.21.15.45:0/1157532714 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649191934 0 0) 0x7f12f015b000 con 0x7f12f0157200 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f4bce700 1 -- 172.21.15.45:0/1157532714 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f015b8a0 con 0x7f12f00c8ff0 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.973+0000 7f12f4bce700 1 --2- 172.21.15.45:0/1157532714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00c8ff0 0x7f12f00c93c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f12e40099b0 tx=0x7f12e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90350c34d49705c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12ef7fe700 1 -- 172.21.15.45:0/1157532714 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400b820 con 0x7f12f00c8ff0 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12ef7fe700 1 -- 172.21.15.45:0/1157532714 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400b980 con 0x7f12f00c8ff0 2023-10-06T05:16:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12ef7fe700 1 -- 172.21.15.45:0/1157532714 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400bd20 con 0x7f12f00c8ff0 2023-10-06T05:16:31.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12f6e32700 1 -- 172.21.15.45:0/1157532714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00c8ff0 msgr2=0x7f12f00c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12f6e32700 1 --2- 172.21.15.45:0/1157532714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00c8ff0 0x7f12f00c93c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f12e40099b0 tx=0x7f12e4009250 comp rx=0 tx=0).stop 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12f6e32700 1 -- 172.21.15.45:0/1157532714 shutdown_connections 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12f6e32700 1 --2- 172.21.15.45:0/1157532714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00c8ff0 0x7f12f00c93c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12f6e32700 1 -- 172.21.15.45:0/1157532714 >> 172.21.15.45:0/1157532714 conn(0x7f12f00cdf50 msgr2=0x7f12f00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12f6e32700 1 -- 172.21.15.45:0/1157532714 shutdown_connections 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.974+0000 7f12f6e32700 1 -- 172.21.15.45:0/1157532714 wait complete. 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.975+0000 7f12f6e32700 1 Processor -- start 2023-10-06T05:16:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12f6e32700 1 -- start start 2023-10-06T05:16:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12f6e32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12f00c8ff0 0x7f12f00b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12f6e32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00caad0 0x7f12f0068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12f6e32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12f0157200 0x7f12f0069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12f6e32700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f00c5fd0 con 0x7f12f00caad0 2023-10-06T05:16:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12f6e32700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0069a60 con 0x7f12f0157200 2023-10-06T05:16:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12f6e32700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0069d10 con 0x7f12f00c8ff0 2023-10-06T05:16:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.976+0000 7f12effff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00caad0 0x7f12f0068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00caad0 0x7f12f0068fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38198/0 (socket says 172.21.15.45:38198) 2023-10-06T05:16:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 -- 172.21.15.45:0/2150925444 learned_addr learned my addr 172.21.15.45:0/2150925444 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12f4bce700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12f00c8ff0 0x7f12f00b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12f53cf700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12f0157200 0x7f12f0069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 -- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12f00c8ff0 msgr2=0x7f12f00b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12f00c8ff0 0x7f12f00b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 -- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12f0157200 msgr2=0x7f12f0069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12f0157200 0x7f12f0069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 -- 172.21.15.45:0/2150925444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e40094f0 con 0x7f12f00caad0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12effff700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00caad0 0x7f12f0068fe0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00c5d0 tx=0x7f12dc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12ecff9700 1 -- 172.21.15.45:0/2150925444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc011590 con 0x7f12f00caad0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12ecff9700 1 -- 172.21.15.45:0/2150925444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc011710 con 0x7f12f00caad0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12ecff9700 1 -- 172.21.15.45:0/2150925444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc011ad0 con 0x7f12f00caad0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f0069ff0 con 0x7f12f00caad0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.977+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f006a5b0 con 0x7f12f00caad0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.978+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f00c6180 con 0x7f12f00caad0 2023-10-06T05:16:31.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.979+0000 7f12ecff9700 1 -- 172.21.15.45:0/2150925444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12dc01c460 con 0x7f12f00caad0 2023-10-06T05:16:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.979+0000 7f12ecff9700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12d4066f60 0x7f12d4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.979+0000 7f12ecff9700 1 -- 172.21.15.45:0/2150925444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f12dc0844f0 con 0x7f12f00caad0 2023-10-06T05:16:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.980+0000 7f12f4bce700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12d4066f60 0x7f12d4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.980+0000 7f12f4bce700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12d4066f60 0x7f12d4069410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f12e40099b0 tx=0x7f12e4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:31.984+0000 7f12ecff9700 1 -- 172.21.15.45:0/2150925444 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12f00c6180 con 0x7f12f00caad0 2023-10-06T05:16:32.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.234+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12f00c6110 con 0x7f12f00caad0 2023-10-06T05:16:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.234+0000 7f12ecff9700 1 -- 172.21.15.45:0/2150925444 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12dc0513f0 con 0x7f12f00caad0 2023-10-06T05:16:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12d4066f60 msgr2=0x7f12d4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12d4066f60 0x7f12d4069410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f12e40099b0 tx=0x7f12e4016000 comp rx=0 tx=0).stop 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00caad0 msgr2=0x7f12f0068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00caad0 0x7f12f0068fe0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00c5d0 tx=0x7f12dc007f60 comp rx=0 tx=0).stop 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 shutdown_connections 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12d4066f60 0x7f12d4069410 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12f0157200 0x7f12f0069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12f00c8ff0 0x7f12f00b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 --2- 172.21.15.45:0/2150925444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12f00caad0 0x7f12f0068fe0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 >> 172.21.15.45:0/2150925444 conn(0x7f12f00cdf50 msgr2=0x7f12f0153bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 shutdown_connections 2023-10-06T05:16:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:32.236+0000 7f12f6e32700 1 -- 172.21.15.45:0/2150925444 wait complete. 2023-10-06T05:16:32.251 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:16:32.251 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:16:32.251 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:16:32.251 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:16:32.252 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:32.252 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:16:32.252 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:16:32.252 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:16:32.252 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:32.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:32.252 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:16:32.273 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:16:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:16:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:16:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:16:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:16:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42349sec preferred_lft 42349sec 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:16:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:16:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:16:32.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:32.277 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:16:32.344 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:16:32.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:32.346 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:16:32.411 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:16:32.412 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:16:32.412 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:16:32.412 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:16:32.412 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:16:32.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:32.412 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:16:32.412 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:16:32.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:16:32.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:16:32.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:16:32.413 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:16:32.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:32.536 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:16:32.557 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:16:32.557 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:16:32.558 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:32.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:32.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:16:32.634 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:16:32.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:32.634 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:32.698 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:16:32.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:16:32.699 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:32.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:16:32.837 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:16:32.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:16:32.912 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:16:32.913 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:16:32.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:32.913 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:16:32.977 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:16:32.977 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:16:33.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:16:33.055 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:16:33.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:16:33.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:16:33.119+0000 7f90d997b540 -1 init, newargv = 0x563d0d76f960 newargc=15 2023-10-06T05:16:33.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[137350]: starting ceph client 2023-10-06T05:16:33.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[137350]: starting fuse 2023-10-06T05:16:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:16:33.130 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:16:33.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:33.131 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:16:33.166 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:16:33.164+0000 7f812670b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:16:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:16:33.186 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:16:33.187 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:16:33.188 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:16:33.189 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:16:33.189 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:33.189 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:16:33.189 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:16:33.325 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.137350.asok 2023-10-06T05:16:33.506 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.137350.asok 2023-10-06T05:16:33.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:33.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.137350.asok status 2023-10-06T05:16:33.682 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:16:33.682 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:16:33.682 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "137350", 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5394, 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:16:33.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 5394 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4068987804 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:16:33.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4068987804 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.5394 192.168.0.1:0/4068987804", 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/4068987804", 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 22, 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 39, 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:16:33.685 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:16:33.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:33.693 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:16:33.693 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:33.714 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:16:33.714 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:33.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:33.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:33.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:33.788 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2023-10-06T05:16:33.857 INFO:teuthology.orchestra.run.smithi045.stdout:file_undamaged 2023-10-06T05:16:33.858 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:16:33.858 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:16:33.858 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:16:33.858 DEBUG:teuthology.orchestra.run.smithi045:> import json 2023-10-06T05:16:33.858 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> try: 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2023-10-06T05:16:33.859 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:16:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:daemon-helper: command failed with exit status 2 2023-10-06T05:16:34.168 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:16:34.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:34.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.349+0000 7f0a77a39700 1 Processor -- start 2023-10-06T05:16:34.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.349+0000 7f0a77a39700 1 -- start start 2023-10-06T05:16:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.350+0000 7f0a77a39700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a701570b0 0x7f0a7006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.350+0000 7f0a77a39700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a700ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a77a39700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7015b380 con 0x7f0a70156ce0 2023-10-06T05:16:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a77a39700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7006c010 con 0x7f0a701570b0 2023-10-06T05:16:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a77a39700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700b02f0 con 0x7f0a7015ac10 2023-10-06T05:16:34.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a74fd4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a701570b0 0x7f0a7006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a757d5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0a70156ce0 0x7f0a700afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34030/0 (socket says 172.21.15.45:34030) 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a757d5700 1 -- 172.21.15.45:0/1239353120 learned_addr learned my addr 172.21.15.45:0/1239353120 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a75fd6700 1 --2- 172.21.15.45:0/1239353120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a700ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a74fd4700 1 -- 172.21.15.45:0/1239353120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 msgr2=0x7f0a700ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a74fd4700 1 --2- 172.21.15.45:0/1239353120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a700ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a74fd4700 1 -- 172.21.15.45:0/1239353120 >> v1:172.21.15.45:6789/0 conn(0x7f0a70156ce0 legacy=0x7f0a700afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a74fd4700 1 -- 172.21.15.45:0/1239353120 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7015b660 con 0x7f0a701570b0 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a67fff700 1 -- 172.21.15.45:0/1239353120 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967408294 0 0) 0x7f0a7015b380 con 0x7f0a70156ce0 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a74fd4700 1 --2- 172.21.15.45:0/1239353120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a701570b0 0x7f0a7006bad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009910 tx=0x7f0a600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5623a37122f1709e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a67fff700 1 -- 172.21.15.45:0/1239353120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000b820 con 0x7f0a701570b0 2023-10-06T05:16:34.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a67fff700 1 -- 172.21.15.45:0/1239353120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6000b980 con 0x7f0a701570b0 2023-10-06T05:16:34.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.351+0000 7f0a67fff700 1 -- 172.21.15.45:0/1239353120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000bd20 con 0x7f0a701570b0 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 -- 172.21.15.45:0/1239353120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a701570b0 msgr2=0x7f0a7006bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1239353120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a701570b0 0x7f0a7006bad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009910 tx=0x7f0a600091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 -- 172.21.15.45:0/1239353120 shutdown_connections 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1239353120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a700ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1239353120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a701570b0 0x7f0a7006bad0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 -- 172.21.15.45:0/1239353120 >> 172.21.15.45:0/1239353120 conn(0x7f0a7006f090 msgr2=0x7f0a7006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 -- 172.21.15.45:0/1239353120 shutdown_connections 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.352+0000 7f0a77a39700 1 -- 172.21.15.45:0/1239353120 wait complete. 2023-10-06T05:16:34.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.354+0000 7f0a77a39700 1 Processor -- start 2023-10-06T05:16:34.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.354+0000 7f0a77a39700 1 -- start start 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a77a39700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a70156ce0 0x7f0a700abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a77a39700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a701570b0 0x7f0a70153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a77a39700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a70067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a77a39700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701548c0 con 0x7f0a70156ce0 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a77a39700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700c3480 con 0x7f0a701570b0 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a77a39700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700ac350 con 0x7f0a7015ac10 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a757d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a70156ce0 0x7f0a700abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a757d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a70156ce0 0x7f0a700abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38214/0 (socket says 172.21.15.45:38214) 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a757d5700 1 -- 172.21.15.45:0/1273200699 learned_addr learned my addr 172.21.15.45:0/1273200699 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:34.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a75fd6700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a70067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.355+0000 7f0a74fd4700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a701570b0 0x7f0a70153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a757d5700 1 -- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 msgr2=0x7f0a70067f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a757d5700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a70067f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a757d5700 1 -- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a701570b0 msgr2=0x7f0a70153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a757d5700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a701570b0 0x7f0a70153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a757d5700 1 -- 172.21.15.45:0/1273200699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a60009420 con 0x7f0a70156ce0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a75fd6700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a70067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a757d5700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a70156ce0 0x7f0a700abe10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c5e0 tx=0x7f0a6c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a657fa700 1 -- 172.21.15.45:0/1273200699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c005b30 con 0x7f0a70156ce0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a657fa700 1 -- 172.21.15.45:0/1273200699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c005c90 con 0x7f0a70156ce0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a700c0d60 con 0x7f0a70156ce0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a657fa700 1 -- 172.21.15.45:0/1273200699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c007300 con 0x7f0a70156ce0 2023-10-06T05:16:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.356+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a700c1000 con 0x7f0a70156ce0 2023-10-06T05:16:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.357+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7006e2f0 con 0x7f0a70156ce0 2023-10-06T05:16:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.358+0000 7f0a657fa700 1 -- 172.21.15.45:0/1273200699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a6c01b3b0 con 0x7f0a70156ce0 2023-10-06T05:16:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.358+0000 7f0a657fa700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a54066f30 0x7f0a540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.358+0000 7f0a657fa700 1 -- 172.21.15.45:0/1273200699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0a6c083d00 con 0x7f0a70156ce0 2023-10-06T05:16:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.358+0000 7f0a74fd4700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a54066f30 0x7f0a540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.359+0000 7f0a74fd4700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a54066f30 0x7f0a540693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009860 tx=0x7f0a60017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.363+0000 7f0a657fa700 1 -- 172.21.15.45:0/1273200699 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a6c050c80 con 0x7f0a70156ce0 2023-10-06T05:16:34.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.612+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a700ac740 con 0x7f0a70156ce0 2023-10-06T05:16:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.613+0000 7f0a657fa700 1 -- 172.21.15.45:0/1273200699 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f0a6c017020 con 0x7f0a70156ce0 2023-10-06T05:16:34.615 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:34.615 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:30.171347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:16:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a54066f30 msgr2=0x7f0a540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a54066f30 0x7f0a540693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009860 tx=0x7f0a60017000 comp rx=0 tx=0).stop 2023-10-06T05:16:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a70156ce0 msgr2=0x7f0a700abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a70156ce0 0x7f0a700abe10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c5e0 tx=0x7f0a6c001070 comp rx=0 tx=0).stop 2023-10-06T05:16:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 shutdown_connections 2023-10-06T05:16:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a54066f30 0x7f0a540693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7015ac10 0x7f0a70067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a70156ce0 0x7f0a700abe10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 --2- 172.21.15.45:0/1273200699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a701570b0 0x7f0a70153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 >> 172.21.15.45:0/1273200699 conn(0x7f0a7006f090 msgr2=0x7f0a70068cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 shutdown_connections 2023-10-06T05:16:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.615+0000 7f0a77a39700 1 -- 172.21.15.45:0/1273200699 wait complete. 2023-10-06T05:16:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 22 2023-10-06T05:16:34.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2023-10-06T05:16:34.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.762+0000 7fed68b59700 1 Processor -- start 2023-10-06T05:16:34.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.762+0000 7fed68b59700 1 -- start start 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed68b59700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 0x7fed6406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed68b59700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed640c9940 con 0x7fed6406cd60 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed68b59700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6406d230 con 0x7fed64140ab0 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed68b59700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6406d510 con 0x7fed641406e0 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed61d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fed6406cd60 0x7fed640acfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34036/0 (socket says 172.21.15.45:34036) 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed61d9b700 1 -- 172.21.15.45:0/2379500836 learned_addr learned my addr 172.21.15.45:0/2379500836 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6259c700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 0x7fed6406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725746826 0 0) 0x7fed640c9940 con 0x7fed6406cd60 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4c003570 con 0x7fed6406cd60 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076331619 0 0) 0x7fed6406d230 con 0x7fed64140ab0 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed640c9940 con 0x7fed64140ab0 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3092669904 0 0) 0x7fed4c003570 con 0x7fed6406cd60 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 msgr2=0x7fed6406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 0x7fed6406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.763+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 >> v1:172.21.15.144:6789/0 conn(0x7fed64140ab0 legacy=0x7fed640c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6406d7f0 con 0x7fed6406cd60 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fed58001470 con 0x7fed6406cd60 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 >> v1:172.21.15.45:6789/0 conn(0x7fed6406cd60 legacy=0x7fed640acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 0x7fed4c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed4c004180 0x7fed4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6406d510 con 0x7fed4c004d50 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00bba0 con 0x7fed4c004180 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00bd60 con 0x7fed641406e0 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed61d9b700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed4c004180 0x7fed4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed62d9d700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6259c700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 0x7fed4c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed62d9d700 1 -- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 msgr2=0x7fed4c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed62d9d700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed641406e0 0x7fed4c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed62d9d700 1 -- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed4c004180 msgr2=0x7fed4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed62d9d700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed4c004180 0x7fed4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed62d9d700 1 -- 172.21.15.45:0/2379500836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c003570 con 0x7fed4c004d50 2023-10-06T05:16:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed62d9d700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed4c0051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fed5c002160 tx=0x7fed5c0017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20298e8e8ff6c4fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c001af0 con 0x7fed4c004d50 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00c980 con 0x7fed4c004d50 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.764+0000 7fed6159a700 1 -- 172.21.15.45:0/2379500836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00cd20 con 0x7fed4c004d50 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.765+0000 7fed68b59700 1 -- 172.21.15.45:0/2379500836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 msgr2=0x7fed4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.765+0000 7fed68b59700 1 --2- 172.21.15.45:0/2379500836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed4c0051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fed5c002160 tx=0x7fed5c0017e0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.765+0000 7fed62d9d700 1 -- 172.21.15.45:0/2379500836 reap_dead start 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.765+0000 7fed68b59700 1 -- 172.21.15.45:0/2379500836 shutdown_connections 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.765+0000 7fed68b59700 1 -- 172.21.15.45:0/2379500836 >> 172.21.15.45:0/2379500836 conn(0x7fed640691c0 msgr2=0x7fed640cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.765+0000 7fed68b59700 1 -- 172.21.15.45:0/2379500836 shutdown_connections 2023-10-06T05:16:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.765+0000 7fed68b59700 1 -- 172.21.15.45:0/2379500836 wait complete. 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.766+0000 7fed68b59700 1 Processor -- start 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.766+0000 7fed68b59700 1 -- start start 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed68b59700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed64156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed68b59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed640ada90 0x7fed640adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed68b59700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed6415a3b0 0x7fed6415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed68b59700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed640ae440 con 0x7fed4c004d50 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed68b59700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6415bd00 con 0x7fed640ada90 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed68b59700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed641571c0 con 0x7fed6415a3b0 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed6259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed64156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed6259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed64156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38224/0 (socket says 172.21.15.45:38224) 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed6259c700 1 -- 172.21.15.45:0/2728713861 learned_addr learned my addr 172.21.15.45:0/2728713861 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed61d9b700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed640ada90 0x7fed640adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.767+0000 7fed62d9d700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed6415a3b0 0x7fed6415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed6259c700 1 -- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed6415a3b0 msgr2=0x7fed6415a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed6259c700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed6415a3b0 0x7fed6415a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed6259c700 1 -- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed640ada90 msgr2=0x7fed640adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed6259c700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed640ada90 0x7fed640adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed6259c700 1 -- 172.21.15.45:0/2728713861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6415b0f0 con 0x7fed4c004d50 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed6259c700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed64156c80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fed54006ac0 tx=0x7fed540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed527fc700 1 -- 172.21.15.45:0/2728713861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5400f940 con 0x7fed4c004d50 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed527fc700 1 -- 172.21.15.45:0/2728713861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5400faa0 con 0x7fed4c004d50 2023-10-06T05:16:34.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed527fc700 1 -- 172.21.15.45:0/2728713861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5400fe20 con 0x7fed4c004d50 2023-10-06T05:16:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6415b3d0 con 0x7fed4c004d50 2023-10-06T05:16:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.768+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6415b990 con 0x7fed4c004d50 2023-10-06T05:16:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.770+0000 7fed527fc700 1 -- 172.21.15.45:0/2728713861 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed54014650 con 0x7fed4c004d50 2023-10-06T05:16:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.770+0000 7fed527fc700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed44066f30 0x7fed440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.770+0000 7fed527fc700 1 -- 172.21.15.45:0/2728713861 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fed54018030 con 0x7fed4c004d50 2023-10-06T05:16:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.770+0000 7fed61d9b700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed44066f30 0x7fed440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.771+0000 7fed61d9b700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed44066f30 0x7fed440693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed6415c9f0 tx=0x7fed58001e10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.772+0000 7fed71e6db80 1 Processor -- start 2023-10-06T05:16:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.772+0000 7fed71e6db80 1 -- start start 2023-10-06T05:16:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbeef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbef8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbef89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed71e6db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbdc23b0 con 0x564dbbef8570 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbdff1d0 con 0x564dbbeef4f0 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dbbdfcb90 con 0x564dbbed4fe0 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed62d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbef89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed6259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbeef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed61d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbef8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed62d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbef89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38232/0 (socket says 172.21.15.45:38232) 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed6259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbeef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:41262/0 (socket says 172.21.15.45:41262) 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed61d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbef8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:54500/0 (socket says 172.21.15.45:54500) 2023-10-06T05:16:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.773+0000 7fed62d9d700 1 -- 172.21.15.45:0/3472300332 learned_addr learned my addr 172.21.15.45:0/3472300332 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed62d9d700 1 -- 172.21.15.45:0/3472300332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 msgr2=0x564dbbef8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed62d9d700 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbef8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed62d9d700 1 -- 172.21.15.45:0/3472300332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 msgr2=0x564dbbeef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed62d9d700 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbeef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed62d9d700 1 -- 172.21.15.45:0/3472300332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dbbbfe830 con 0x564dbbef8570 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed6259c700 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbeef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed62d9d700 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbef89e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fed5c003000 tx=0x7fed5c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91c2e94cdff35ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed437fe700 1 -- 172.21.15.45:0/3472300332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00c6d0 con 0x564dbbef8570 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed437fe700 1 -- 172.21.15.45:0/3472300332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c000fb0 con 0x564dbbef8570 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.774+0000 7fed437fe700 1 -- 172.21.15.45:0/3472300332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00cd00 con 0x564dbbef8570 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 -- 172.21.15.45:0/3472300332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 msgr2=0x564dbbef89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbef89e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fed5c003000 tx=0x7fed5c016000 comp rx=0 tx=0).stop 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 -- 172.21.15.45:0/3472300332 shutdown_connections 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbef89e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbeef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3472300332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbef8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 -- 172.21.15.45:0/3472300332 >> 172.21.15.45:0/3472300332 conn(0x564dbbef2920 msgr2=0x564dbbef2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 -- 172.21.15.45:0/3472300332 shutdown_connections 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.775+0000 7fed71e6db80 1 -- 172.21.15.45:0/3472300332 wait complete. 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.776+0000 7fed71e6db80 1 -- start start 2023-10-06T05:16:34.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.776+0000 7fed71e6db80 1 Processor -- start 2023-10-06T05:16:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbec6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbf027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbece2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed71e6db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbdf8980 con 0x564dbbef8570 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbda4a20 con 0x564dbbeef4f0 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dbbda46c0 con 0x564dbbed4fe0 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed71e6db80 10 client.0 _ll_register_callbacks cb 0x564dbbc07aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbece2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed6259c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbec6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbece2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38244/0 (socket says 172.21.15.45:38244) 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 -- 172.21.15.45:0/1829580647 learned_addr learned my addr 172.21.15.45:0/1829580647 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed61d9b700 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbf027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 -- 172.21.15.45:0/1829580647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 msgr2=0x564dbbec6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbec6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 -- 172.21.15.45:0/1829580647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 msgr2=0x564dbbf027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbf027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 -- 172.21.15.45:0/1829580647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54056e70 con 0x564dbbef8570 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.777+0000 7fed62d9d700 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbece2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fed5c001b10 tx=0x7fed5c022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405a324241cb95af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed40ff9700 1 -- 172.21.15.45:0/1829580647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c006db0 con 0x564dbbef8570 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dbbe9ce00 con 0x564dbbef8570 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed40ff9700 1 -- 172.21.15.45:0/1829580647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fed5c00caf0 con 0x564dbbef8570 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed40ff9700 1 -- 172.21.15.45:0/1829580647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed5c00cec0 con 0x564dbbef8570 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 10 client.5415 fetch_fsmap learned FSMap version 22 2023-10-06T05:16:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x7fed4c003570 con 0x564dbbef8570 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed40ff9700 1 -- 172.21.15.45:0/1829580647 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dbbe9ce00 con 0x564dbbef8570 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed40ff9700 1 -- 172.21.15.45:0/1829580647 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fed5c00c0e0 con 0x564dbbef8570 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 10 client.5415 fetch_fsmap finished waiting for FSMap version 22 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 10 client.5415 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 20 client.5415 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbeaa160 0x564dbbeaa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 4 client.5415 mds_command: new command op to 5322 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dbbe79bd0 con 0x564dbbeaa160 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.778+0000 7fed6259c700 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbeaa160 0x564dbbeaa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed40ff9700 10 client.5415 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed6259c700 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbeaa160 0x564dbbeaa530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7cac5f7c3ed9958 server_cookie=e7abc043db0bddc8 in_seq=0 out_seq=0 2023-10-06T05:16:34.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed40ff9700 1 -- 172.21.15.45:0/1829580647 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564dbbe79bd0 con 0x564dbbeaa160 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed40ff9700 10 client.5415 handle_command_reply: tid=0 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 client.5415 shutdown 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 msgr2=0x564dbbece2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbece2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fed5c001b10 tx=0x7fed5c022040 comp rx=0 tx=0).stop 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 shutdown_connections 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbeaa160 0x564dbbeaa530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbece2f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbec6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed40ff9700 0 client.5415 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.779+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1829580647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbf027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.780+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 >> 172.21.15.45:0/1829580647 conn(0x564dbbef2920 msgr2=0x564dbbea4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.780+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 shutdown_connections 2023-10-06T05:16:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.780+0000 7fed71e6db80 1 -- 172.21.15.45:0/1829580647 wait complete. 2023-10-06T05:16:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.780+0000 7fed71e6db80 20 client.5415 trim_cache size 0 max 16384 2023-10-06T05:16:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.793+0000 7fed71e6db80 1 Processor -- start 2023-10-06T05:16:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.793+0000 7fed71e6db80 1 -- start start 2023-10-06T05:16:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbea94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbef2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbec9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed71e6db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbc90280 con 0x564dbbef8570 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbf113e0 con 0x564dbbeef4f0 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dbbed88e0 con 0x564dbbed4fe0 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbec9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbec9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38250/0 (socket says 172.21.15.45:38250) 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 -- 172.21.15.45:0/3703551212 learned_addr learned my addr 172.21.15.45:0/3703551212 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed6259c700 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbea94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed61d9b700 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbef2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 -- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 msgr2=0x564dbbea94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbea94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 -- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 msgr2=0x564dbbef2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbef2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 -- 172.21.15.45:0/3703551212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dbbceb140 con 0x564dbbef8570 2023-10-06T05:16:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed61d9b700 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbef2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed62d9d700 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbec9b10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fed5c006e90 tx=0x7fed5c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc6039a4318f2a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.794+0000 7fed43fff700 1 -- 172.21.15.45:0/3703551212 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00caf0 con 0x564dbbef8570 2023-10-06T05:16:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed43fff700 1 -- 172.21.15.45:0/3703551212 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00c150 con 0x564dbbef8570 2023-10-06T05:16:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed43fff700 1 -- 172.21.15.45:0/3703551212 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c001330 con 0x564dbbef8570 2023-10-06T05:16:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 -- 172.21.15.45:0/3703551212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 msgr2=0x564dbbec9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbec9b10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fed5c006e90 tx=0x7fed5c02e040 comp rx=0 tx=0).stop 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 -- 172.21.15.45:0/3703551212 shutdown_connections 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbef8570 0x564dbbec9b10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbed4fe0 0x564dbbea94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 --2- 172.21.15.45:0/3703551212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbeef4f0 0x564dbbef2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 -- 172.21.15.45:0/3703551212 >> 172.21.15.45:0/3703551212 conn(0x564dbbef2920 msgr2=0x564dbbe9e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 -- 172.21.15.45:0/3703551212 shutdown_connections 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.795+0000 7fed71e6db80 1 -- 172.21.15.45:0/3703551212 wait complete. 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.796+0000 7fed71e6db80 1 -- start start 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.796+0000 7fed71e6db80 1 Processor -- start 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbed4fe0 0x564dbbec9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbeef4f0 0x564dbbeca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed71e6db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbef8570 0x564dbbea7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed71e6db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbef4d00 con 0x564dbbed4fe0 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbbea4e80 con 0x564dbbef8570 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed71e6db80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dbbed0f30 con 0x564dbbeef4f0 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed71e6db80 10 client.0 _ll_register_callbacks cb 0x564dbbc07aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbed4fe0 0x564dbbec9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbed4fe0 0x564dbbec9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38254/0 (socket says 172.21.15.45:38254) 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 -- 172.21.15.45:0/1804364088 learned_addr learned my addr 172.21.15.45:0/1804364088 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed62d9d700 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbef8570 0x564dbbea7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed61d9b700 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbeef4f0 0x564dbbeca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 -- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbeef4f0 msgr2=0x564dbbeca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbeef4f0 0x564dbbeca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 -- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbef8570 msgr2=0x564dbbea7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbef8570 0x564dbbea7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.797+0000 7fed6259c700 1 -- 172.21.15.45:0/1804364088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4c003570 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed6259c700 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbed4fe0 0x564dbbec9b10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fed54052310 tx=0x7fed5401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650841eaee1ffecf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed40ff9700 1 -- 172.21.15.45:0/1804364088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5405ce00 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed40ff9700 1 -- 172.21.15.45:0/1804364088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fed54060070 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5c000fb0 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed40ff9700 1 -- 172.21.15.45:0/1804364088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed54054200 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 10 client.5421 fetch_fsmap learned FSMap version 22 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x564dbbdc7460 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed40ff9700 1 -- 172.21.15.45:0/1804364088 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54069070 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed40ff9700 1 -- 172.21.15.45:0/1804364088 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 22) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fed54055350 con 0x564dbbed4fe0 2023-10-06T05:16:34.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 10 client.5421 fetch_fsmap finished waiting for FSMap version 22 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 10 client.5421 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 20 client.5421 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbea8470 0x564dbbea8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 4 client.5421 mds_command: new command op to 5322 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.798+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x564dbbd9b970 con 0x564dbbea8470 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed61d9b700 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbea8470 0x564dbbea8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed40ff9700 10 client.5421 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed61d9b700 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbea8470 0x564dbbea8840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3332703c1c03dd28 server_cookie=c9034a774a11ca13 in_seq=0 out_seq=0 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed40ff9700 1 -- 172.21.15.45:0/1804364088 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x564dbbd9b970 con 0x564dbbea8470 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed40ff9700 10 client.5421 handle_command_reply: tid=0 2023-10-06T05:16:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 client.5421 shutdown 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbed4fe0 msgr2=0x564dbbec9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbed4fe0 0x564dbbec9b10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fed54052310 tx=0x7fed5401f040 comp rx=0 tx=0).stop 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 shutdown_connections 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x564dbbea8470 0x564dbbea8840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dbbef8570 0x564dbbea7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564dbbed4fe0 0x564dbbec9b10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed40ff9700 0 client.5421 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 --2- 172.21.15.45:0/1804364088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564dbbeef4f0 0x564dbbeca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 >> 172.21.15.45:0/1804364088 conn(0x564dbbef2920 msgr2=0x564dbbecaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 shutdown_connections 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.799+0000 7fed71e6db80 1 -- 172.21.15.45:0/1804364088 wait complete. 2023-10-06T05:16:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed71e6db80 20 client.5421 trim_cache size 0 max 16384 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "dentry", 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4114013756, 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627776, 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: "frag": "*", 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "file_to_be_damaged", 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_id": "head", 2023-10-06T05:16:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/subdir/file_to_be_damaged" 2023-10-06T05:16:34.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:34.809 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:16:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed44066f30 msgr2=0x7fed440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed44066f30 0x7fed440693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed6415c9f0 tx=0x7fed58001e10 comp rx=0 tx=0).stop 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 msgr2=0x7fed64156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed64156c80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fed54006ac0 tx=0x7fed540091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 shutdown_connections 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed44066f30 0x7fed440693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed6415a3b0 0x7fed6415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed4c004d50 0x7fed64156c80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 --2- 172.21.15.45:0/2728713861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed640ada90 0x7fed640adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 >> 172.21.15.45:0/2728713861 conn(0x7fed640691c0 msgr2=0x7fed640b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 shutdown_connections 2023-10-06T05:16:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:34.800+0000 7fed68b59700 1 -- 172.21.15.45:0/2728713861 wait complete. 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> import errno 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> try: 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> f.close() 2023-10-06T05:16:34.815 DEBUG:teuthology.orchestra.run.smithi045:> except IOError as e: 2023-10-06T05:16:34.816 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(errno.EIO) 2023-10-06T05:16:34.816 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:16:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:daemon-helper: command failed with exit status 5 2023-10-06T05:16:35.078 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-10-06T05:16:35.078 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> import json 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> try: 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T05:16:35.079 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2023-10-06T05:16:35.080 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T05:16:35.080 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2023-10-06T05:16:35.080 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:16:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:daemon-helper: command failed with exit status 5 2023-10-06T05:16:35.388 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-10-06T05:16:35.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:16:35.389 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2023-10-06T05:16:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:2 2023-10-06T05:16:35.456 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:16:35.456 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:16:35.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:16:35.537 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[137350]: fuse finished with error 0 and tester_r 0 2023-10-06T05:16:35.539 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:16:35.771 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:35.774 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:37.789 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:16:37.788+0000 7fdb6559f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T05:16:41.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:16:41.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:16:41.641 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:16:41.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2023-10-06T05:16:41.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.848+0000 7f20fc26e700 1 Processor -- start 2023-10-06T05:16:41.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.848+0000 7f20fc26e700 1 -- start start 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fc26e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20f4159160 0x7f20f40cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fc26e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4067620 0x7f20f4067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fc26e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f40c8140 con 0x7f20f4159530 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fc26e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f4067fe0 con 0x7f20f4159160 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fc26e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f40ca1e0 con 0x7f20f4067620 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20f4159160 0x7f20f40cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20f4159160 0x7f20f40cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48670/0 (socket says 172.21.15.45:48670) 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 -- 172.21.15.45:0/2641182761 learned_addr learned my addr 172.21.15.45:0/2641182761 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:41.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2641182761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4067620 0x7f20f4067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 -- 172.21.15.45:0/2641182761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4067620 msgr2=0x7f20f4067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2641182761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4067620 0x7f20f4067aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 -- 172.21.15.45:0/2641182761 >> v1:172.21.15.144:6789/0 conn(0x7f20f4159530 legacy=0x7f20f40c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 -- 172.21.15.45:0/2641182761 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40ca4c0 con 0x7f20f4159160 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.849+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2641182761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4159160 0x7f20f40cadf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f20e80099b0 tx=0x7f20e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad531f7c23264cd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20f9008700 1 -- 172.21.15.45:0/2641182761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8009ea0 con 0x7f20f4159160 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20f9008700 1 -- 172.21.15.45:0/2641182761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e800b860 con 0x7f20f4159160 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20f9008700 1 -- 172.21.15.45:0/2641182761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e800bc00 con 0x7f20f4159160 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 -- 172.21.15.45:0/2641182761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4159160 msgr2=0x7f20f40cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 --2- 172.21.15.45:0/2641182761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4159160 0x7f20f40cadf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f20e80099b0 tx=0x7f20e8009250 comp rx=0 tx=0).stop 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 -- 172.21.15.45:0/2641182761 shutdown_connections 2023-10-06T05:16:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 --2- 172.21.15.45:0/2641182761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4067620 0x7f20f4067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 --2- 172.21.15.45:0/2641182761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4159160 0x7f20f40cadf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 -- 172.21.15.45:0/2641182761 >> 172.21.15.45:0/2641182761 conn(0x7f20f4068730 msgr2=0x7f20f4068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 -- 172.21.15.45:0/2641182761 shutdown_connections 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.850+0000 7f20fc26e700 1 -- 172.21.15.45:0/2641182761 wait complete. 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.851+0000 7f20fc26e700 1 Processor -- start 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fc26e700 1 -- start start 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fc26e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4067620 0x7f20f41448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fc26e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4159160 0x7f20f41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fc26e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f4159530 0x7f20f41589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fc26e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f41559c0 con 0x7f20f4067620 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fc26e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f41567e0 con 0x7f20f4159530 2023-10-06T05:16:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fc26e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f4144e30 con 0x7f20f4159160 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fa00a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4067620 0x7f20f41448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fa00a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4067620 0x7f20f41448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48672/0 (socket says 172.21.15.45:48672) 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.852+0000 7f20fa00a700 1 -- 172.21.15.45:0/1267362818 learned_addr learned my addr 172.21.15.45:0/1267362818 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fa80b700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f4159530 0x7f20f41589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20f9809700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4159160 0x7f20f41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fa00a700 1 -- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4159160 msgr2=0x7f20f41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fa00a700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4159160 0x7f20f41584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fa00a700 1 -- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f4159530 msgr2=0x7f20f41589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fa00a700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f4159530 0x7f20f41589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fa00a700 1 -- 172.21.15.45:0/1267362818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80094f0 con 0x7f20f4067620 2023-10-06T05:16:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fa00a700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4067620 0x7f20f41448f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f20f40c7d90 tx=0x7f20e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20e67fc700 1 -- 172.21.15.45:0/1267362818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8000d10 con 0x7f20f4067620 2023-10-06T05:16:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20e67fc700 1 -- 172.21.15.45:0/1267362818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e800bbe0 con 0x7f20f4067620 2023-10-06T05:16:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20e67fc700 1 -- 172.21.15.45:0/1267362818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011670 con 0x7f20f4067620 2023-10-06T05:16:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f4155bd0 con 0x7f20f4067620 2023-10-06T05:16:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.853+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f4156190 con 0x7f20f4067620 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.855+0000 7f20e67fc700 1 -- 172.21.15.45:0/1267362818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20e8011aa0 con 0x7f20f4067620 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.855+0000 7f20e67fc700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f20d8066f30 0x7f20d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.855+0000 7f20e67fc700 1 -- 172.21.15.45:0/1267362818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f20e80860b0 con 0x7f20f4067620 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.855+0000 7f20f9809700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f20d8066f30 0x7f20d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.856+0000 7f20f9809700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f20d8066f30 0x7f20d80693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f20f40c1ed0 tx=0x7f20f0007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.857+0000 7f2105582b80 1 Processor -- start 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f2105582b80 1 -- start start 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23f228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f2b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f2105582b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23f2ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f2105582b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23df53d0 con 0x559e23f2b590 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23e321f0 con 0x559e23f22510 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e23e2fbb0 con 0x559e23f08000 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23f2ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23f2ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48678/0 (socket says 172.21.15.45:48678) 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.858+0000 7f20fa80b700 1 -- 172.21.15.45:0/2598404198 learned_addr learned my addr 172.21.15.45:0/2598404198 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23f228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20f9809700 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f2b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20fa80b700 1 -- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 msgr2=0x559e23f2b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f2b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20fa80b700 1 -- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 msgr2=0x559e23f228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23f228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20fa80b700 1 -- 172.21.15.45:0/2598404198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e23c31850 con 0x559e23f2b590 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23f2ba00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0114d0 tx=0x7f20ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cea1e026908b601 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20d77fe700 1 -- 172.21.15.45:0/2598404198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00e040 con 0x559e23f2b590 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20d77fe700 1 -- 172.21.15.45:0/2598404198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec00f040 con 0x559e23f2b590 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.859+0000 7f20d77fe700 1 -- 172.21.15.45:0/2598404198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec015640 con 0x559e23f2b590 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 -- 172.21.15.45:0/2598404198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 msgr2=0x559e23f2ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23f2ba00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0114d0 tx=0x7f20ec00b040 comp rx=0 tx=0).stop 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 -- 172.21.15.45:0/2598404198 shutdown_connections 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23f2ba00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23f228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 --2- 172.21.15.45:0/2598404198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f2b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 -- 172.21.15.45:0/2598404198 >> 172.21.15.45:0/2598404198 conn(0x559e23f25940 msgr2=0x559e23f25d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 -- 172.21.15.45:0/2598404198 shutdown_connections 2023-10-06T05:16:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.860+0000 7f2105582b80 1 -- 172.21.15.45:0/2598404198 wait complete. 2023-10-06T05:16:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.861+0000 7f2105582b80 1 -- start start 2023-10-06T05:16:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.861+0000 7f2105582b80 1 Processor -- start 2023-10-06T05:16:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23edd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f2105582b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23edd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f2105582b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23e2b9a0 con 0x559e23f2b590 2023-10-06T05:16:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23dd7a40 con 0x559e23f22510 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e23dd76e0 con 0x559e23f08000 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f2105582b80 10 client.0 _ll_register_callbacks cb 0x559e23c3aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23edd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23edd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48686/0 (socket says 172.21.15.45:48686) 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 -- 172.21.15.45:0/2052781659 learned_addr learned my addr 172.21.15.45:0/2052781659 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20f9809700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23edd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 -- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 msgr2=0x559e23f0adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f0adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 -- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 msgr2=0x559e23edd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23edd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 -- 172.21.15.45:0/2052781659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec011230 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f0adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23edd5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f20ec012010 tx=0x7f20ec01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad3b2b795a94224 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.862+0000 7f20d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f20d4ff9700 1 -- 172.21.15.45:0/2052781659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00e050 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f20d4ff9700 1 -- 172.21.15.45:0/2052781659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f20ec015e30 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e23ecfe20 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f20d4ff9700 1 -- 172.21.15.45:0/2052781659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ec01d400 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 10 client.5433 fetch_fsmap learned FSMap version 23 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x559e23d1e160 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f20d4ff9700 1 -- 172.21.15.45:0/2052781659 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e23ecfe20 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f20d4ff9700 1 -- 172.21.15.45:0/2052781659 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f20ec01b020 con 0x559e23f2b590 2023-10-06T05:16:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 10 client.5433 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 10 client.5433 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 20 client.5433 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23efc720 0x559e23f23d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 4 client.5433 mds_command: new command op to 5322 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e23eacbf0 con 0x559e23efc720 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.863+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23efc720 0x559e23f23d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23efc720 0x559e23f23d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b71307aba9b429 server_cookie=7c9c737c94542133 in_seq=0 out_seq=0 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f20d4ff9700 10 client.5433 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f20d4ff9700 1 -- 172.21.15.45:0/2052781659 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559e23eacbf0 con 0x559e23efc720 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f20d4ff9700 10 client.5433 handle_command_reply: tid=0 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 client.5433 shutdown 2023-10-06T05:16:41.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 msgr2=0x559e23edd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23edd5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f20ec012010 tx=0x7f20ec01d000 comp rx=0 tx=0).stop 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 shutdown_connections 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23efc720 0x559e23f23d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f2b590 0x559e23edd5c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f0adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 --2- 172.21.15.45:0/2052781659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f22510 0x559e23edd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f20d4ff9700 0 client.5433 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 >> 172.21.15.45:0/2052781659 conn(0x559e23f25940 msgr2=0x559e23f228e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 shutdown_connections 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.864+0000 7f2105582b80 1 -- 172.21.15.45:0/2052781659 wait complete. 2023-10-06T05:16:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.865+0000 7f2105582b80 20 client.5433 trim_cache size 0 max 16384 2023-10-06T05:16:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.881+0000 7f2105582b80 1 Processor -- start 2023-10-06T05:16:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.881+0000 7f2105582b80 1 -- start start 2023-10-06T05:16:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f25940 0x559e23edd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f22ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f2105582b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f22510 0x559e23efcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f2105582b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23cc32a0 con 0x559e23f22510 2023-10-06T05:16:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23ede570 con 0x559e23f25940 2023-10-06T05:16:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e23f01010 con 0x559e23f08000 2023-10-06T05:16:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f22510 0x559e23efcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f22510 0x559e23efcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48700/0 (socket says 172.21.15.45:48700) 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f20fa80b700 1 -- 172.21.15.45:0/2767454905 learned_addr learned my addr 172.21.15.45:0/2767454905 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f20fa00a700 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f25940 0x559e23edd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.882+0000 7f20f9809700 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f22ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20fa80b700 1 -- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 msgr2=0x559e23f22ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f22ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20fa80b700 1 -- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f25940 msgr2=0x559e23edd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f25940 0x559e23edd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20fa80b700 1 -- 172.21.15.45:0/2767454905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e23dfa480 con 0x559e23f22510 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20fa80b700 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f22510 0x559e23efcdb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20ec011c20 tx=0x7f20ec023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f09cb1ebbdb1d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20d7fff700 1 -- 172.21.15.45:0/2767454905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00e070 con 0x559e23f22510 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20d7fff700 1 -- 172.21.15.45:0/2767454905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec015aa0 con 0x559e23f22510 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.883+0000 7f20d7fff700 1 -- 172.21.15.45:0/2767454905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec015e30 con 0x559e23f22510 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 -- 172.21.15.45:0/2767454905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f22510 msgr2=0x559e23efcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f22510 0x559e23efcdb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20ec011c20 tx=0x7f20ec023000 comp rx=0 tx=0).stop 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 -- 172.21.15.45:0/2767454905 shutdown_connections 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f22510 0x559e23efcdb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f25940 0x559e23edd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 --2- 172.21.15.45:0/2767454905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f08000 0x559e23f22ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 -- 172.21.15.45:0/2767454905 >> 172.21.15.45:0/2767454905 conn(0x559e23edc850 msgr2=0x559e23ed1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 -- 172.21.15.45:0/2767454905 shutdown_connections 2023-10-06T05:16:41.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 -- 172.21.15.45:0/2767454905 wait complete. 2023-10-06T05:16:41.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 -- start start 2023-10-06T05:16:41.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.884+0000 7f2105582b80 1 Processor -- start 2023-10-06T05:16:41.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.885+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f08000 0x559e23f28520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.885+0000 7f2105582b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f22510 0x559e23f03840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.885+0000 7f2105582b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f25940 0x559e23f341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.885+0000 7f2105582b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23f28f30 con 0x559e23f25940 2023-10-06T05:16:41.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.885+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e23f33d10 con 0x559e23f08000 2023-10-06T05:16:41.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.885+0000 7f2105582b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e23f28a60 con 0x559e23f22510 2023-10-06T05:16:41.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f2105582b80 10 client.0 _ll_register_callbacks cb 0x559e23c3aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f25940 0x559e23f341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f25940 0x559e23f341a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48710/0 (socket says 172.21.15.45:48710) 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 -- 172.21.15.45:0/1233247384 learned_addr learned my addr 172.21.15.45:0/1233247384 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20f9809700 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f22510 0x559e23f03840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa00a700 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f08000 0x559e23f28520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 -- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f22510 msgr2=0x559e23f03840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f22510 0x559e23f03840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 -- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f08000 msgr2=0x559e23f28520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f08000 0x559e23f28520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 -- 172.21.15.45:0/1233247384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e23d1e160 con 0x559e23f25940 2023-10-06T05:16:41.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20fa80b700 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f25940 0x559e23f341a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0070b0 tx=0x7f20ec01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85fd27fd46e29e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20d4ff9700 1 -- 172.21.15.45:0/1233247384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec021040 con 0x559e23f25940 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e23dce990 con 0x559e23f25940 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20d4ff9700 1 -- 172.21.15.45:0/1233247384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f20ec015d80 con 0x559e23f25940 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.886+0000 7f20d4ff9700 1 -- 172.21.15.45:0/1233247384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ec00e040 con 0x559e23f25940 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 10 client.5439 fetch_fsmap learned FSMap version 23 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x559e23c92290 con 0x559e23f25940 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f20d4ff9700 1 -- 172.21.15.45:0/1233247384 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e23dce990 con 0x559e23f25940 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f20d4ff9700 1 -- 172.21.15.45:0/1233247384 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f20ec038070 con 0x559e23f25940 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 10 client.5439 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 10 client.5439 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-06T05:16:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 20 client.5439 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23f2b590 0x559e23f8b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 4 client.5439 mds_command: new command op to 5322 tid=0[{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) v1 -- 0x559e23c8cfb0 con 0x559e23f2b590 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f20fa00a700 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23f2b590 0x559e23f8b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f20fa00a700 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23f2b590 0x559e23f8b9c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af524fd74ca91bd2 server_cookie=7c8437a2a6b24616 in_seq=0 out_seq=0 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.887+0000 7f20d4ff9700 10 client.5439 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f20d4ff9700 1 -- 172.21.15.45:0/1233247384 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559e23c8cfb0 con 0x559e23f2b590 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f20d4ff9700 10 client.5439 handle_command_reply: tid=0 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 client.5439 shutdown 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f25940 msgr2=0x559e23f341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f25940 0x559e23f341a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0070b0 tx=0x7f20ec01b000 comp rx=0 tx=0).stop 2023-10-06T05:16:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 shutdown_connections 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x559e23f2b590 0x559e23f8b9c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e23f25940 0x559e23f341a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e23f08000 0x559e23f28520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 --2- 172.21.15.45:0/1233247384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559e23f22510 0x559e23f03840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f20d4ff9700 0 client.5439 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 >> 172.21.15.45:0/1233247384 conn(0x559e23edc850 msgr2=0x559e23ed1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 shutdown_connections 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 1 -- 172.21.15.45:0/1233247384 wait complete. 2023-10-06T05:16:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.888+0000 7f2105582b80 20 client.5439 trim_cache size 0 max 16384 2023-10-06T05:16:41.899 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:16:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2023-10-06T05:16:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "c50778db-a5e2-4f85-a4aa-d5ae129fe82a", 2023-10-06T05:16:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2023-10-06T05:16:41.899 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f20d8066f30 msgr2=0x7f20d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f20d8066f30 0x7f20d80693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f20f40c1ed0 tx=0x7f20f0007ec0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4067620 msgr2=0x7f20f41448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4067620 0x7f20f41448f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f20f40c7d90 tx=0x7f20e8017000 comp rx=0 tx=0).stop 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 shutdown_connections 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f20d8066f30 0x7f20d80693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f4159530 0x7f20f41589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20f4067620 0x7f20f41448f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 --2- 172.21.15.45:0/1267362818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f4159160 0x7f20f41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 >> 172.21.15.45:0/1267362818 conn(0x7f20f4068730 msgr2=0x7f20f4156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 shutdown_connections 2023-10-06T05:16:41.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:41.889+0000 7f20fc26e700 1 -- 172.21.15.45:0/1267362818 wait complete. 2023-10-06T05:16:41.905 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "c50778db-a5e2-4f85-a4aa-d5ae129fe82a", "mode": "asynchronous" } 2023-10-06T05:16:41.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-10-06T05:16:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.037+0000 7f8d043d1700 1 Processor -- start 2023-10-06T05:16:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.037+0000 7f8d043d1700 1 -- start start 2023-10-06T05:16:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d043d1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cfc157330 0x7f8cfc0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d043d1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d043d1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0c3b50 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d043d1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc157700 con 0x7f8cfc157330 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d043d1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c98e0 con 0x7f8cfc0aaae0 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d0216d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cfc157330 0x7f8cfc0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d0296e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8cfc0c90b0 0x7f8cfc0c3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60326/0 (socket says 172.21.15.45:60326) 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d0296e700 1 -- 172.21.15.45:0/954040069 learned_addr learned my addr 172.21.15.45:0/954040069 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.038+0000 7f8d0196c700 1 --2- 172.21.15.45:0/954040069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0216d700 1 -- 172.21.15.45:0/954040069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 msgr2=0x7f8cfc0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0216d700 1 --2- 172.21.15.45:0/954040069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0216d700 1 -- 172.21.15.45:0/954040069 >> v1:172.21.15.45:6789/0 conn(0x7f8cfc0c90b0 legacy=0x7f8cfc0c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0216d700 1 -- 172.21.15.45:0/954040069 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0c9bc0 con 0x7f8cfc157330 2023-10-06T05:16:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0116b700 1 -- 172.21.15.45:0/954040069 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983624856 0 0) 0x7f8cfc0c3b50 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0216d700 1 --2- 172.21.15.45:0/954040069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc157330 0x7f8cfc0ce8b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80098d0 tx=0x7f8cf8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e258a2a6cb2bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0116b700 1 -- 172.21.15.45:0/954040069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8009e70 con 0x7f8cfc157330 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0116b700 1 -- 172.21.15.45:0/954040069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf8005c10 con 0x7f8cfc157330 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d0116b700 1 -- 172.21.15.45:0/954040069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800b650 con 0x7f8cfc157330 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.039+0000 7f8d043d1700 1 -- 172.21.15.45:0/954040069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc157330 msgr2=0x7f8cfc0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.040+0000 7f8d043d1700 1 --2- 172.21.15.45:0/954040069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc157330 0x7f8cfc0ce8b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80098d0 tx=0x7f8cf8009120 comp rx=0 tx=0).stop 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.040+0000 7f8d043d1700 1 -- 172.21.15.45:0/954040069 shutdown_connections 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.040+0000 7f8d043d1700 1 --2- 172.21.15.45:0/954040069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc157330 0x7f8cfc0ce8b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.040+0000 7f8d043d1700 1 --2- 172.21.15.45:0/954040069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.040+0000 7f8d043d1700 1 -- 172.21.15.45:0/954040069 >> 172.21.15.45:0/954040069 conn(0x7f8cfc067e00 msgr2=0x7f8cfc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.040+0000 7f8d043d1700 1 -- 172.21.15.45:0/954040069 shutdown_connections 2023-10-06T05:16:42.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.040+0000 7f8d043d1700 1 -- 172.21.15.45:0/954040069 wait complete. 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.041+0000 7f8d043d1700 1 Processor -- start 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.041+0000 7f8d043d1700 1 -- start start 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d043d1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d043d1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc0c90b0 0x7f8cfc1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d043d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc157330 0x7f8cfc155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d043d1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0ac840 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d043d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0ab470 con 0x7f8cfc157330 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d043d1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0ab750 con 0x7f8cfc0aaae0 2023-10-06T05:16:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc0c90b0 0x7f8cfc1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc0c90b0 0x7f8cfc1525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48724/0 (socket says 172.21.15.45:48724) 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 -- 172.21.15.45:0/884888230 learned_addr learned my addr 172.21.15.45:0/884888230 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0216d700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0296e700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc157330 0x7f8cfc155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 -- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 msgr2=0x7f8cfc0ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 -- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc157330 msgr2=0x7f8cfc155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc157330 0x7f8cfc155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 -- 172.21.15.45:0/884888230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf80093c0 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.042+0000 7f8d0196c700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc0c90b0 0x7f8cfc1525a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000d890 tx=0x7f8cf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.043+0000 7f8cee7fc700 1 -- 172.21.15.45:0/884888230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00125f0 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.043+0000 7f8cee7fc700 1 -- 172.21.15.45:0/884888230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf0012750 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.043+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc0aba30 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.043+0000 7f8cee7fc700 1 -- 172.21.15.45:0/884888230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0012af0 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.043+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc152ae0 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.044+0000 7f8cee7fc700 1 -- 172.21.15.45:0/884888230 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8cf001e460 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.045+0000 7f8cee7fc700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ce0066f30 0x7f8ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.045+0000 7f8cee7fc700 1 -- 172.21.15.45:0/884888230 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8cf0084d90 con 0x7f8cfc0c90b0 2023-10-06T05:16:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.045+0000 7f8d0216d700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ce0066f30 0x7f8ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.045+0000 7f8d0216d700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ce0066f30 0x7f8ce00693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009160 tx=0x7f8cf8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.047+0000 7f8d0d6e5b80 1 Processor -- start 2023-10-06T05:16:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.047+0000 7f8d0d6e5b80 1 -- start start 2023-10-06T05:16:42.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa15f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa1600990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa14ca360 con 0x55aaa1600520 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa1507180 con 0x55aaa15dcf90 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aaa1504b40 con 0x55aaa15f74a0 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa1600990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa1600990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48730/0 (socket says 172.21.15.45:48730) 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 -- 172.21.15.45:0/1445947753 learned_addr learned my addr 172.21.15.45:0/1445947753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0196c700 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0216d700 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa15f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 -- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 msgr2=0x55aaa15f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa15f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 -- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 msgr2=0x55aaa15fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.048+0000 7f8d0296e700 1 -- 172.21.15.45:0/1445947753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa13067e0 con 0x55aaa1600520 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0296e700 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa1600990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4002020 tx=0x7f8cf40019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546aadebfdd76768 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8cdf7fe700 1 -- 172.21.15.45:0/1445947753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4004de0 con 0x55aaa1600520 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8cdf7fe700 1 -- 172.21.15.45:0/1445947753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4007a90 con 0x55aaa1600520 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8cdf7fe700 1 -- 172.21.15.45:0/1445947753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4013640 con 0x55aaa1600520 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1445947753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 msgr2=0x55aaa1600990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa1600990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4002020 tx=0x7f8cf40019a0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1445947753 shutdown_connections 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa1600990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa15f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1445947753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.049+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1445947753 >> 172.21.15.45:0/1445947753 conn(0x55aaa15fa8d0 msgr2=0x55aaa15faca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.050+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1445947753 shutdown_connections 2023-10-06T05:16:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.050+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1445947753 wait complete. 2023-10-06T05:16:42.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.050+0000 7f8d0d6e5b80 1 -- start start 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.050+0000 7f8d0d6e5b80 1 Processor -- start 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15f74a0 0x55aaa15b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa1600520 0x55aaa15b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa1500930 con 0x55aaa15f74a0 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa14ac9d0 con 0x55aaa15dcf90 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aaa14ac670 con 0x55aaa1600520 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0d6e5b80 10 client.0 _ll_register_callbacks cb 0x55aaa130fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0196c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15f74a0 0x55aaa15b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0196c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15f74a0 0x55aaa15b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48746/0 (socket says 172.21.15.45:48746) 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.051+0000 7f8d0196c700 1 -- 172.21.15.45:0/2738849927 learned_addr learned my addr 172.21.15.45:0/2738849927 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0216d700 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0296e700 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa1600520 0x55aaa15b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0196c700 1 -- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa1600520 msgr2=0x55aaa15b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0196c700 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa1600520 0x55aaa15b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0196c700 1 -- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 msgr2=0x55aaa15cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0196c700 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0196c700 1 -- 172.21.15.45:0/2738849927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf4001c80 con 0x55aaa15f74a0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0196c700 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15f74a0 0x55aaa15b1830 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0082f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435dedc789d51dde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8cdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8cdcff9700 1 -- 172.21.15.45:0/2738849927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0053d70 con 0x55aaa15f74a0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8cdcff9700 1 -- 172.21.15.45:0/2738849927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8cf0071060 con 0x55aaa15f74a0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaa15a4db0 con 0x55aaa15f74a0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8cdcff9700 1 -- 172.21.15.45:0/2738849927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cf0055e40 con 0x55aaa15f74a0 2023-10-06T05:16:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0d6e5b80 10 client.5454 fetch_fsmap learned FSMap version 23 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55aaa13f30f0 con 0x55aaa15f74a0 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.052+0000 7f8cdcff9700 1 -- 172.21.15.45:0/2738849927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf0060020 con 0x55aaa15f74a0 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8cdcff9700 1 -- 172.21.15.45:0/2738849927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8cf0054ec0 con 0x55aaa15f74a0 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0d6e5b80 10 client.5454 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0d6e5b80 10 client.5454 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0d6e5b80 20 client.5454 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa15b2bd0 0x55aaa15cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0d6e5b80 4 client.5454 mds_command: new command op to 5322 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaa1581b80 con 0x55aaa15b2bd0 2023-10-06T05:16:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0216d700 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa15b2bd0 0x55aaa15cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8cdcff9700 10 client.5454 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.053+0000 7f8d0216d700 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa15b2bd0 0x55aaa15cff00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350827df00c35d28 server_cookie=96dab42e056e0594 in_seq=0 out_seq=0 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8cdcff9700 1 -- 172.21.15.45:0/2738849927 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55aaa1581b80 con 0x55aaa15b2bd0 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8cdcff9700 10 client.5454 handle_command_reply: tid=0 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 client.5454 shutdown 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15f74a0 msgr2=0x55aaa15b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15f74a0 0x55aaa15b1830 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0082f20 comp rx=0 tx=0).stop 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 shutdown_connections 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa15b2bd0 0x55aaa15cff00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa1600520 0x55aaa15b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15dcf90 0x55aaa15cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/2738849927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15f74a0 0x55aaa15b1830 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8cdcff9700 0 client.5454 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 >> 172.21.15.45:0/2738849927 conn(0x55aaa15fa8d0 msgr2=0x55aaa15d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 shutdown_connections 2023-10-06T05:16:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/2738849927 wait complete. 2023-10-06T05:16:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.054+0000 7f8d0d6e5b80 20 client.5454 trim_cache size 0 max 16384 2023-10-06T05:16:42.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.071+0000 7f8d0d6e5b80 1 Processor -- start 2023-10-06T05:16:42.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.071+0000 7f8d0d6e5b80 1 -- start start 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15dcf90 0x55aaa15d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15f74a0 0x55aaa15b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa15b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa1398230 con 0x55aaa1600520 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa15d2b60 con 0x55aaa15f74a0 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aaa16180f0 con 0x55aaa15dcf90 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0296e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa15b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0296e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa15b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48754/0 (socket says 172.21.15.45:48754) 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0296e700 1 -- 172.21.15.45:0/1494669305 learned_addr learned my addr 172.21.15.45:0/1494669305 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.072+0000 7f8d0196c700 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15f74a0 0x55aaa15b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8d0216d700 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15dcf90 0x55aaa15d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8d0296e700 1 -- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15dcf90 msgr2=0x55aaa15d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8d0296e700 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15dcf90 0x55aaa15d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8d0296e700 1 -- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15f74a0 msgr2=0x55aaa15b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8d0296e700 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15f74a0 0x55aaa15b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8d0296e700 1 -- 172.21.15.45:0/1494669305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa14cf410 con 0x55aaa1600520 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8d0296e700 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa15b1c30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4014000 tx=0x7f8cf4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852aaa6e538de1ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8cdffff700 1 -- 172.21.15.45:0/1494669305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4024070 con 0x55aaa1600520 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8cdffff700 1 -- 172.21.15.45:0/1494669305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4004bb0 con 0x55aaa1600520 2023-10-06T05:16:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.073+0000 7f8cdffff700 1 -- 172.21.15.45:0/1494669305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4004de0 con 0x55aaa1600520 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1494669305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 msgr2=0x55aaa15b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa15b1c30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4014000 tx=0x7f8cf4018000 comp rx=0 tx=0).stop 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1494669305 shutdown_connections 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa1600520 0x55aaa15b1c30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15dcf90 0x55aaa15d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/1494669305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa15f74a0 0x55aaa15b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1494669305 >> 172.21.15.45:0/1494669305 conn(0x55aaa15fa8d0 msgr2=0x55aaa15ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1494669305 shutdown_connections 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/1494669305 wait complete. 2023-10-06T05:16:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.074+0000 7f8d0d6e5b80 1 -- start start 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.075+0000 7f8d0d6e5b80 1 Processor -- start 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.075+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15dcf90 0x55aaa165a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa1660e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0d6e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa1600520 0x55aaa16613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa161ccd0 con 0x55aaa15dcf90 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa15b3690 con 0x55aaa1600520 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0d6e5b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aaa1660a00 con 0x55aaa15f74a0 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0d6e5b80 10 client.0 _ll_register_callbacks cb 0x55aaa130fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15dcf90 0x55aaa165a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15dcf90 0x55aaa165a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48770/0 (socket says 172.21.15.45:48770) 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 -- 172.21.15.45:0/431809319 learned_addr learned my addr 172.21.15.45:0/431809319 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:16:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0296e700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa1600520 0x55aaa16613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0196c700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa1660e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 -- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 msgr2=0x55aaa1660e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa1660e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 -- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa1600520 msgr2=0x55aaa16613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa1600520 0x55aaa16613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 -- 172.21.15.45:0/431809319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaa13067e0 con 0x55aaa15dcf90 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0196c700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa1660e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0296e700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa1600520 0x55aaa16613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0216d700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15dcf90 0x55aaa165a8d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc06bac0 tx=0x7f8cf802d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61b28dc09c6322c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8cdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8cdcff9700 1 -- 172.21.15.45:0/431809319 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8023070 con 0x55aaa15dcf90 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8cdcff9700 1 -- 172.21.15.45:0/431809319 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8cf8036430 con 0x55aaa15dcf90 2023-10-06T05:16:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.076+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaa14a3920 con 0x55aaa15dcf90 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8cdcff9700 1 -- 172.21.15.45:0/431809319 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cf800bee0 con 0x55aaa15dcf90 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 10 client.5460 fetch_fsmap learned FSMap version 23 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55aaa1367220 con 0x55aaa15dcf90 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8cdcff9700 1 -- 172.21.15.45:0/431809319 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aaa14a3920 con 0x55aaa15dcf90 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8cdcff9700 1 -- 172.21.15.45:0/431809319 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8cf8036800 con 0x55aaa15dcf90 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 10 client.5460 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 10 client.5460 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 20 client.5460 populate_metadata read hostname 'smithi045' 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa161c4f0 0x55aaa15d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 4 client.5460 mds_command: new command op to 5322 tid=0[{"prefix": "scrub status"}] 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aaa1361f40 con 0x55aaa161c4f0 2023-10-06T05:16:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0196c700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa161c4f0 0x55aaa15d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8cdcff9700 10 client.5460 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.077+0000 7f8d0196c700 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa161c4f0 0x55aaa15d0ee0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95883fbff206615 server_cookie=a1d2ff79bfbbb8cd in_seq=0 out_seq=0 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8cdcff9700 1 -- 172.21.15.45:0/431809319 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x55aaa1361f40 con 0x55aaa161c4f0 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8cdcff9700 10 client.5460 handle_command_reply: tid=0 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 client.5460 shutdown 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15dcf90 msgr2=0x55aaa165a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15dcf90 0x55aaa165a8d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc06bac0 tx=0x7f8cf802d040 comp rx=0 tx=0).stop 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 shutdown_connections 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55aaa161c4f0 0x55aaa15d0ee0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaa1600520 0x55aaa16613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8cdcff9700 0 client.5460 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaa15dcf90 0x55aaa165a8d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 --2- 172.21.15.45:0/431809319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaa15f74a0 0x55aaa1660e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 >> 172.21.15.45:0/431809319 conn(0x55aaa15fa8d0 msgr2=0x55aaa15b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 shutdown_connections 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 1 -- 172.21.15.45:0/431809319 wait complete. 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.078+0000 7f8d0d6e5b80 20 client.5460 trim_cache size 0 max 16384 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "c50778db-a5e2-4f85-a4aa-d5ae129fe82a": { 2023-10-06T05:16:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/subdir", 2023-10-06T05:16:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "c50778db-a5e2-4f85-a4aa-d5ae129fe82a", 2023-10-06T05:16:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "repair" 2023-10-06T05:16:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:16:42.088 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ce0066f30 msgr2=0x7f8ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ce0066f30 0x7f8ce00693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009160 tx=0x7f8cf8016000 comp rx=0 tx=0).stop 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc0c90b0 msgr2=0x7f8cfc1525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc0c90b0 0x7f8cfc1525a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000d890 tx=0x7f8cf0008040 comp rx=0 tx=0).stop 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 shutdown_connections 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ce0066f30 0x7f8ce00693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc157330 0x7f8cfc155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc0aaae0 0x7f8cfc0ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 --2- 172.21.15.45:0/884888230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cfc0c90b0 0x7f8cfc1525a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 >> 172.21.15.45:0/884888230 conn(0x7f8cfc067e00 msgr2=0x7f8cfc06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 shutdown_connections 2023-10-06T05:16:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:16:42.079+0000 7f8d043d1700 1 -- 172.21.15.45:0/884888230 wait complete. 2023-10-06T05:16:42.095 INFO:tasks.cephfs.filesystem:scrub status for tag:c50778db-a5e2-4f85-a4aa-d5ae129fe82a - {'path': '/subdir', 'tag': 'c50778db-a5e2-4f85-a4aa-d5ae129fe82a', 'options': 'repair'} 2023-10-06T05:17:05.834 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:05.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:12.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-10-06T05:17:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.239+0000 7fca6e7fe700 1 Processor -- start 2023-10-06T05:17:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.239+0000 7fca6e7fe700 1 -- start start 2023-10-06T05:17:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca6e7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fca68157140 0x7fca680ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca6e7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca6806dff0 0x7fca6806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca6e7fe700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca680ac230 con 0x7fca6806a070 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca6e7fe700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca680aba00 con 0x7fca6806dff0 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca6e7fe700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6806ab80 con 0x7fca68157140 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca6806dff0 0x7fca6806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca6cd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fca6806a070 0x7fca6806a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46672/0 (socket says 172.21.15.45:46672) 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1107729358 learned_addr learned my addr 172.21.15.45:0/1107729358 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca6806dff0 0x7fca6806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43116/0 (socket says 172.21.15.45:43116) 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca67fff700 1 --2- 172.21.15.45:0/1107729358 >> v2:172.21.15.144:3300/0 conn(0x7fca68157140 0x7fca680ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca677fe700 1 -- 172.21.15.45:0/1107729358 >> v2:172.21.15.144:3300/0 conn(0x7fca68157140 msgr2=0x7fca680ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca677fe700 1 --2- 172.21.15.45:0/1107729358 >> v2:172.21.15.144:3300/0 conn(0x7fca68157140 0x7fca680ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca677fe700 1 -- 172.21.15.45:0/1107729358 >> v1:172.21.15.45:6789/0 conn(0x7fca6806a070 legacy=0x7fca6806a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca66ffd700 1 -- 172.21.15.45:0/1107729358 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856157722 0 0) 0x7fca680ac230 con 0x7fca6806a070 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.240+0000 7fca677fe700 1 -- 172.21.15.45:0/1107729358 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6806e980 con 0x7fca6806dff0 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca677fe700 1 --2- 172.21.15.45:0/1107729358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6806dff0 0x7fca6806e440 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fca6806ddf0 tx=0x7fca500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24dd2f5e572d7448 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca66ffd700 1 -- 172.21.15.45:0/1107729358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000b6b0 con 0x7fca6806dff0 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca66ffd700 1 -- 172.21.15.45:0/1107729358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5000b810 con 0x7fca6806dff0 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca66ffd700 1 -- 172.21.15.45:0/1107729358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000bbb0 con 0x7fca6806dff0 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1107729358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6806dff0 msgr2=0x7fca6806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1107729358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6806dff0 0x7fca6806e440 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fca6806ddf0 tx=0x7fca500091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1107729358 shutdown_connections 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1107729358 >> v2:172.21.15.144:3300/0 conn(0x7fca68157140 0x7fca680ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1107729358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6806dff0 0x7fca6806e440 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.241+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1107729358 >> 172.21.15.45:0/1107729358 conn(0x7fca680680f0 msgr2=0x7fca680cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.242+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1107729358 shutdown_connections 2023-10-06T05:17:12.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.242+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1107729358 wait complete. 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.243+0000 7fca6e7fe700 1 Processor -- start 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.243+0000 7fca6e7fe700 1 -- start start 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6e7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca6806a070 0x7fca680b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6e7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6806dff0 0x7fca6814e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6e7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca68157140 0x7fca680c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6e7fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca680b1590 con 0x7fca68157140 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6e7fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca680b0d70 con 0x7fca6806dff0 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6e7fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca680b05c0 con 0x7fca6806a070 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca68157140 0x7fca680c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca68157140 0x7fca680c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43126/0 (socket says 172.21.15.45:43126) 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1408833906 learned_addr learned my addr 172.21.15.45:0/1408833906 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca67fff700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca6806a070 0x7fca680b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca677fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6806dff0 0x7fca6814e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca6806a070 msgr2=0x7fca680b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca6806a070 0x7fca680b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6806dff0 msgr2=0x7fca6814e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6806dff0 0x7fca6814e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1408833906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca50009420 con 0x7fca68157140 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca677fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6806dff0 0x7fca6814e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.244+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca68157140 0x7fca680c3750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca6000cc90 tx=0x7fca60001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.245+0000 7fca5ffff700 1 -- 172.21.15.45:0/1408833906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6000a960 con 0x7fca68157140 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.245+0000 7fca5ffff700 1 -- 172.21.15.45:0/1408833906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6000aac0 con 0x7fca68157140 2023-10-06T05:17:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.245+0000 7fca5ffff700 1 -- 172.21.15.45:0/1408833906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6000ae80 con 0x7fca68157140 2023-10-06T05:17:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.245+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca680b0810 con 0x7fca68157140 2023-10-06T05:17:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.245+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca680c3250 con 0x7fca68157140 2023-10-06T05:17:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.246+0000 7fca5ffff700 1 -- 172.21.15.45:0/1408833906 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca60007300 con 0x7fca68157140 2023-10-06T05:17:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.247+0000 7fca5ffff700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca48066f30 0x7fca480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.247+0000 7fca5ffff700 1 -- 172.21.15.45:0/1408833906 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fca60084ce0 con 0x7fca68157140 2023-10-06T05:17:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.247+0000 7fca67fff700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca48066f30 0x7fca480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.247+0000 7fca67fff700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca48066f30 0x7fca480693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca58006fd0 tx=0x7fca58008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.249+0000 7fca77b12b80 1 Processor -- start 2023-10-06T05:17:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.249+0000 7fca77b12b80 1 -- start start 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa9900fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa9901990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca77b12b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa97cb360 con 0x561fa98f84a0 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa9808180 con 0x561fa9901520 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fa9805b40 con 0x561fa98ddf90 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98f8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43134/0 (socket says 172.21.15.45:43134) 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 -- 172.21.15.45:0/1366469273 learned_addr learned my addr 172.21.15.45:0/1366469273 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca677fe700 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa9900fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa9901990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 -- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 msgr2=0x561fa9900fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa9900fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 -- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 msgr2=0x561fa9901990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa9901990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 -- 172.21.15.45:0/1366469273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fa96077e0 con 0x561fa98f84a0 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.250+0000 7fca67fff700 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98f8870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca5800b4e0 tx=0x7fca58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa94d417779887e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca5d7fa700 1 -- 172.21.15.45:0/1366469273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800bea0 con 0x561fa98f84a0 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca5d7fa700 1 -- 172.21.15.45:0/1366469273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5801a070 con 0x561fa98f84a0 2023-10-06T05:17:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca5d7fa700 1 -- 172.21.15.45:0/1366469273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58016d50 con 0x561fa98f84a0 2023-10-06T05:17:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca77b12b80 1 -- 172.21.15.45:0/1366469273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 msgr2=0x561fa98f8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98f8870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca5800b4e0 tx=0x7fca58017000 comp rx=0 tx=0).stop 2023-10-06T05:17:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca77b12b80 1 -- 172.21.15.45:0/1366469273 shutdown_connections 2023-10-06T05:17:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa9901990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98f8870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1366469273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa9900fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.251+0000 7fca77b12b80 1 -- 172.21.15.45:0/1366469273 >> 172.21.15.45:0/1366469273 conn(0x561fa98fb8d0 msgr2=0x561fa98fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.252+0000 7fca77b12b80 1 -- 172.21.15.45:0/1366469273 shutdown_connections 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.252+0000 7fca77b12b80 1 -- 172.21.15.45:0/1366469273 wait complete. 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.252+0000 7fca77b12b80 1 -- start start 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.252+0000 7fca77b12b80 1 Processor -- start 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98ddf90 0x561fa98d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa98fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa9901520 0x561fa98b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca77b12b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa9801930 con 0x561fa98ddf90 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa97ad9d0 con 0x561fa98f84a0 2023-10-06T05:17:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fa97ad670 con 0x561fa9901520 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca77b12b80 10 client.0 _ll_register_callbacks cb 0x561fa9610a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98ddf90 0x561fa98d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98ddf90 0x561fa98d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43150/0 (socket says 172.21.15.45:43150) 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca67fff700 1 -- 172.21.15.45:0/3305347768 learned_addr learned my addr 172.21.15.45:0/3305347768 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca677fe700 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa98fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.253+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa9901520 0x561fa98b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca67fff700 1 -- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa9901520 msgr2=0x561fa98b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca67fff700 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa9901520 0x561fa98b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca67fff700 1 -- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 msgr2=0x561fa98fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca67fff700 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa98fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca67fff700 1 -- 172.21.15.45:0/3305347768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5800e040 con 0x561fa98ddf90 2023-10-06T05:17:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca67fff700 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98ddf90 0x561fa98d2050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fca58016760 tx=0x7fca5801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b572de9840951 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca467fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca467fc700 1 -- 172.21.15.45:0/3305347768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58021e80 con 0x561fa98ddf90 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca467fc700 1 -- 172.21.15.45:0/3305347768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fca58034e00 con 0x561fa98ddf90 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fa98a5db0 con 0x561fa98ddf90 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca467fc700 1 -- 172.21.15.45:0/3305347768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca5802a680 con 0x561fa98ddf90 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 10 client.5475 fetch_fsmap learned FSMap version 23 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561fa96f40f0 con 0x561fa98ddf90 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca467fc700 1 -- 172.21.15.45:0/3305347768 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561fa98a5db0 con 0x561fa98ddf90 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca467fc700 1 -- 172.21.15.45:0/3305347768 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca5802a7e0 con 0x561fa98ddf90 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 10 client.5475 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 10 client.5475 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-06T05:17:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 20 client.5475 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa98d2bf0 0x561fa98d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 4 client.5475 mds_command: new command op to 5322 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.254+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fa9882b80 con 0x561fa98d2bf0 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.255+0000 7fca677fe700 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa98d2bf0 0x561fa98d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.255+0000 7fca467fc700 10 client.5475 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.255+0000 7fca677fe700 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa98d2bf0 0x561fa98d2fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1412b50760327b8 server_cookie=9c7fd92aa9183381 in_seq=0 out_seq=0 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.255+0000 7fca467fc700 1 -- 172.21.15.45:0/3305347768 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561fa9882b80 con 0x561fa98d2bf0 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.255+0000 7fca467fc700 10 client.5475 handle_command_reply: tid=0 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.255+0000 7fca77b12b80 1 client.5475 shutdown 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98ddf90 msgr2=0x561fa98d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98ddf90 0x561fa98d2050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fca58016760 tx=0x7fca5801a000 comp rx=0 tx=0).stop 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 shutdown_connections 2023-10-06T05:17:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa98d2bf0 0x561fa98d2fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa9901520 0x561fa98b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98ddf90 0x561fa98d2050 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca467fc700 0 client.5475 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 --2- 172.21.15.45:0/3305347768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa98fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 >> 172.21.15.45:0/3305347768 conn(0x561fa98fb8d0 msgr2=0x561fa98adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 shutdown_connections 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 1 -- 172.21.15.45:0/3305347768 wait complete. 2023-10-06T05:17:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.256+0000 7fca77b12b80 20 client.5475 trim_cache size 0 max 16384 2023-10-06T05:17:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.273+0000 7fca77b12b80 1 Processor -- start 2023-10-06T05:17:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.273+0000 7fca77b12b80 1 -- start start 2023-10-06T05:17:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa98b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca77b12b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa9699230 con 0x561fa98f84a0 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa98b4b00 con 0x561fa9901520 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fa98dac40 con 0x561fa98ddf90 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43156/0 (socket says 172.21.15.45:43156) 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 -- 172.21.15.45:0/4164954294 learned_addr learned my addr 172.21.15.45:0/4164954294 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca67fff700 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa98b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 -- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 msgr2=0x561fa98ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 -- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 msgr2=0x561fa98b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa98b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 -- 172.21.15.45:0/4164954294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fa97d0410 con 0x561fa98f84a0 2023-10-06T05:17:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca677fe700 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98d2fc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca50000e50 tx=0x7fca50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11645c0d62553c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.274+0000 7fca5dffb700 1 -- 172.21.15.45:0/4164954294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50020040 con 0x561fa98f84a0 2023-10-06T05:17:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca5dffb700 1 -- 172.21.15.45:0/4164954294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca50021040 con 0x561fa98f84a0 2023-10-06T05:17:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca5dffb700 1 -- 172.21.15.45:0/4164954294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000bb30 con 0x561fa98f84a0 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 -- 172.21.15.45:0/4164954294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 msgr2=0x561fa98d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98d2fc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca50000e50 tx=0x7fca50014000 comp rx=0 tx=0).stop 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 -- 172.21.15.45:0/4164954294 shutdown_connections 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa9901520 0x561fa98b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 --2- 172.21.15.45:0/4164954294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa98f84a0 0x561fa98d2fc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 -- 172.21.15.45:0/4164954294 >> 172.21.15.45:0/4164954294 conn(0x561fa98fb8d0 msgr2=0x561fa98ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 -- 172.21.15.45:0/4164954294 shutdown_connections 2023-10-06T05:17:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.275+0000 7fca77b12b80 1 -- 172.21.15.45:0/4164954294 wait complete. 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.276+0000 7fca77b12b80 1 -- start start 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.276+0000 7fca77b12b80 1 Processor -- start 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98d6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa9953960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca77b12b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa9901520 0x561fa9953ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca77b12b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa98b2ba0 con 0x561fa9901520 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa98d75d0 con 0x561fa98f84a0 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca77b12b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fa99543e0 con 0x561fa98ddf90 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca77b12b80 10 client.0 _ll_register_callbacks cb 0x561fa9610a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa9901520 0x561fa9953ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca67fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa9901520 0x561fa9953ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43162/0 (socket says 172.21.15.45:43162) 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1370214506 learned_addr learned my addr 172.21.15.45:0/1370214506 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca677fe700 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa9953960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1370214506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 msgr2=0x561fa98d6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98d6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1370214506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 msgr2=0x561fa9953960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa9953960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 -- 172.21.15.45:0/1370214506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6806e980 con 0x561fa9901520 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca6cd9b700 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa9901520 0x561fa9953ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fca6000a3f0 tx=0x7fca60058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5626e9b5d0d90f94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca467fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.277+0000 7fca467fc700 1 -- 172.21.15.45:0/1370214506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60053ce0 con 0x561fa9901520 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca467fc700 1 -- 172.21.15.45:0/1370214506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fca6006b040 con 0x561fa9901520 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca5801a070 con 0x561fa9901520 2023-10-06T05:17:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca467fc700 1 -- 172.21.15.45:0/1370214506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca6005c3d0 con 0x561fa9901520 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 10 client.5481 fetch_fsmap learned FSMap version 23 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561fa9668220 con 0x561fa9901520 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca467fc700 1 -- 172.21.15.45:0/1370214506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5801a070 con 0x561fa9901520 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca467fc700 1 -- 172.21.15.45:0/1370214506 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca6001ac10 con 0x561fa9901520 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 10 client.5481 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 10 client.5481 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 20 client.5481 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa9949e10 0x561fa9960a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 4 client.5481 mds_command: new command op to 5322 tid=0[{"prefix": "scrub status"}] 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561fa97a4920 con 0x561fa9949e10 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca67fff700 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa9949e10 0x561fa9960a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca467fc700 10 client.5481 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.278+0000 7fca67fff700 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa9949e10 0x561fa9960a20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285f6ff381e0a399 server_cookie=74c84b68407e05be in_seq=0 out_seq=0 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca467fc700 1 -- 172.21.15.45:0/1370214506 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561fa97a4920 con 0x561fa9949e10 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca467fc700 10 client.5481 handle_command_reply: tid=0 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 client.5481 shutdown 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa9901520 msgr2=0x561fa9953ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa9901520 0x561fa9953ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fca6000a3f0 tx=0x7fca60058040 comp rx=0 tx=0).stop 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 shutdown_connections 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x561fa9949e10 0x561fa9960a20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fa9901520 0x561fa9953ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561fa98ddf90 0x561fa98d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 --2- 172.21.15.45:0/1370214506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fa98f84a0 0x561fa9953960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca467fc700 0 client.5481 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 >> 172.21.15.45:0/1370214506 conn(0x561fa98fb8d0 msgr2=0x561fa98d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 shutdown_connections 2023-10-06T05:17:12.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 1 -- 172.21.15.45:0/1370214506 wait complete. 2023-10-06T05:17:12.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca77b12b80 20 client.5481 trim_cache size 0 max 16384 2023-10-06T05:17:12.287 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:17:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2023-10-06T05:17:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2023-10-06T05:17:12.287 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:17:12.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca48066f30 msgr2=0x7fca480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca48066f30 0x7fca480693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca58006fd0 tx=0x7fca58008040 comp rx=0 tx=0).stop 2023-10-06T05:17:12.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca68157140 msgr2=0x7fca680c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca68157140 0x7fca680c3750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca6000cc90 tx=0x7fca60001a80 comp rx=0 tx=0).stop 2023-10-06T05:17:12.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.279+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 shutdown_connections 2023-10-06T05:17:12.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.280+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fca48066f30 0x7fca480693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.280+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca68157140 0x7fca680c3750 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.280+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca6806a070 0x7fca680b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.280+0000 7fca6e7fe700 1 --2- 172.21.15.45:0/1408833906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca6806dff0 0x7fca6814e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.280+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 >> 172.21.15.45:0/1408833906 conn(0x7fca680680f0 msgr2=0x7fca680cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.280+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 shutdown_connections 2023-10-06T05:17:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.280+0000 7fca6e7fe700 1 -- 172.21.15.45:0/1408833906 wait complete. 2023-10-06T05:17:12.294 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T05:17:12.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:17:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.424+0000 7f3ccae86700 1 Processor -- start 2023-10-06T05:17:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.424+0000 7f3ccae86700 1 -- start start 2023-10-06T05:17:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3ccae86700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3ccae86700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3cc40ccdc0 0x7f3cc40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3ccae86700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40cd740 con 0x7f3cc4155cd0 2023-10-06T05:17:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3ccae86700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40ae6b0 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3ccae86700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40ae8d0 con 0x7f3cc41560a0 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3cc8c22700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3cc4155cd0 0x7f3cc40adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46678/0 (socket says 172.21.15.45:46678) 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3cc8c22700 1 -- 172.21.15.45:0/3944097896 learned_addr learned my addr 172.21.15.45:0/3944097896 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.425+0000 7f3cc3fff700 1 --2- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc9423700 1 --2- 172.21.15.45:0/3944097896 >> v2:172.21.15.144:3300/0 conn(0x7f3cc40ccdc0 0x7f3cc40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc37fe700 1 -- 172.21.15.45:0/3944097896 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100841689 0 0) 0x7f3cc40cd740 con 0x7f3cc4155cd0 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc37fe700 1 -- 172.21.15.45:0/3944097896 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb0003570 con 0x7f3cc4155cd0 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc9423700 1 -- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 msgr2=0x7f3cc415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc9423700 1 --2- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc37fe700 1 -- 172.21.15.45:0/3944097896 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435847313 0 0) 0x7f3cb0003570 con 0x7f3cc4155cd0 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc9423700 1 -- 172.21.15.45:0/3944097896 >> v1:172.21.15.45:6789/0 conn(0x7f3cc4155cd0 legacy=0x7f3cc40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc9423700 1 -- 172.21.15.45:0/3944097896 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40aebb0 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc3fff700 1 --2- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc9423700 1 --2- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40cd200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009e60 tx=0x7f3cbc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17aac5693a8d7be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc37fe700 1 -- 172.21.15.45:0/3944097896 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00e070 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc37fe700 1 -- 172.21.15.45:0/3944097896 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00b7a0 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.426+0000 7f3cc37fe700 1 -- 172.21.15.45:0/3944097896 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4001470 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 -- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40ccdc0 msgr2=0x7f3cc40cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40cd200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009e60 tx=0x7f3cbc009780 comp rx=0 tx=0).stop 2023-10-06T05:17:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 -- 172.21.15.45:0/3944097896 shutdown_connections 2023-10-06T05:17:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40cd200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3944097896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 -- 172.21.15.45:0/3944097896 >> 172.21.15.45:0/3944097896 conn(0x7f3cc406f090 msgr2=0x7f3cc406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 -- 172.21.15.45:0/3944097896 shutdown_connections 2023-10-06T05:17:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.427+0000 7f3ccae86700 1 -- 172.21.15.45:0/3944097896 wait complete. 2023-10-06T05:17:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.429+0000 7f3ccae86700 1 Processor -- start 2023-10-06T05:17:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.429+0000 7f3ccae86700 1 -- start start 2023-10-06T05:17:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3ccae86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3ccae86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc4155cd0 0x7f3cc41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3ccae86700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3ccae86700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c43b0 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3ccae86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c4690 con 0x7f3cc4155cd0 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3ccae86700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c4970 con 0x7f3cc41560a0 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43172/0 (socket says 172.21.15.45:43172) 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 -- 172.21.15.45:0/3323047374 learned_addr learned my addr 172.21.15.45:0/3323047374 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc9423700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc3fff700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc4155cd0 0x7f3cc41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 -- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 msgr2=0x7f3cc414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc414ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 -- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc4155cd0 msgr2=0x7f3cc41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc4155cd0 0x7f3cc41584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.430+0000 7f3cc8c22700 1 -- 172.21.15.45:0/3323047374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac009420 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.431+0000 7f3cc8c22700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40c5450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400cb90 tx=0x7f3cb4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.431+0000 7f3cc0ff9700 1 -- 172.21.15.45:0/3323047374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4007e90 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.431+0000 7f3cc0ff9700 1 -- 172.21.15.45:0/3323047374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4007360 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.431+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc009a20 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.431+0000 7f3cc0ff9700 1 -- 172.21.15.45:0/3323047374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4007700 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.431+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc40c4f30 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.432+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc406e2f0 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.432+0000 7f3cc0ff9700 1 -- 172.21.15.45:0/3323047374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cb4004a40 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.433+0000 7f3cc0ff9700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.433+0000 7f3cc0ff9700 1 -- 172.21.15.45:0/3323047374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f3cb4084060 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.433+0000 7f3cc3fff700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.434+0000 7f3cc3fff700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40ca200 tx=0x7f3cac0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.438+0000 7f3cc0ff9700 1 -- 172.21.15.45:0/3323047374 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb4051020 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.685+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3cc40aee90 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.685+0000 7f3cc0ff9700 1 -- 172.21.15.45:0/3323047374 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3cb4018070 con 0x7f3cc40ccdc0 2023-10-06T05:17:12.687 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 msgr2=0x7f3ca40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40ca200 tx=0x7f3cac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc40ccdc0 msgr2=0x7f3cc40c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40c5450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400cb90 tx=0x7f3cb4001b50 comp rx=0 tx=0).stop 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 shutdown_connections 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc41560a0 0x7f3cc414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc40ccdc0 0x7f3cc40c5450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 --2- 172.21.15.45:0/3323047374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc4155cd0 0x7f3cc41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 >> 172.21.15.45:0/3323047374 conn(0x7f3cc406f090 msgr2=0x7f3cc4067620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 shutdown_connections 2023-10-06T05:17:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:12.688+0000 7f3ccae86700 1 -- 172.21.15.45:0/3323047374 wait complete. 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:17:12.703 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:12.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:12.704 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:17:12.726 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:12.747 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:12.747 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42309sec preferred_lft 42309sec 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:12.748 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:17:12.749 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:17:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:12.750 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:17:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:17:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:17:12.750 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:12.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:12.750 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:17:12.798 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:17:12.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:12.800 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:17:12.868 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:17:12.868 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:17:12.868 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:17:12.869 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:17:12.869 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:17:12.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:12.869 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:17:12.869 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:17:12.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:17:12.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:17:12.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:17:12.870 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:17:12.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:12.995 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:17:13.018 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:17:13.018 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:17:13.019 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:13.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:13.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:17:13.099 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:17:13.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:13.099 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:13.163 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:13.164 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:13.164 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:13.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:17:13.304 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:17:13.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:17:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:17:13.380 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:17:13.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:13.381 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:17:13.445 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:17:13.445 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:17:13.488 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:17:13.523 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:17:13.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:17:13.589 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:17:13.587+0000 7fe26879b540 -1 init, newargv = 0x55e0704dc550 newargc=15 2023-10-06T05:17:13.589 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[138474]: starting ceph client 2023-10-06T05:17:13.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[138474]: starting fuse 2023-10-06T05:17:13.596 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:17:13.597 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:17:13.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:13.598 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:17:13.659 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:17:13.660 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:17:13.660 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:17:13.661 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:17:13.662 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:17:13.662 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:17:13.662 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:17:13.662 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:13.662 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:17:13.662 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:17:13.797 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.138474.asok 2023-10-06T05:17:13.979 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.138474.asok 2023-10-06T05:17:13.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:13.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.138474.asok status 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "138474", 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5496, 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 5496 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 14810968 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 14810968 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.5496 192.168.0.1:0/14810968", 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/14810968", 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 23, 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 41, 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:17:14.159 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:17:14.159 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:17:14.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:14.166 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:17:14.166 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:14.187 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:17:14.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:14.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:14.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:14.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:14.261 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2023-10-06T05:17:14.327 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:17:14.327 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:17:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.418+0000 7fe5b46d5040 1 Processor -- start 2023-10-06T05:17:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.418+0000 7fe5b46d5040 1 -- start start 2023-10-06T05:17:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5b46d5040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd5a250 0x5622ffe05e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5b46d5040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd76d40 0x5622ffe08ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5b46d5040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffd58360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5b46d5040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffc79980 con 0x5622ffe084a0 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5b46d5040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffc79620 con 0x5622ffd76d40 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5b46d5040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622ffc70400 con 0x5622ffd5a250 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffd58360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffd58360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43176/0 (socket says 172.21.15.45:43176) 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/3412104566 learned_addr learned my addr 172.21.15.45:0/3412104566 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ad0f2700 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd76d40 0x5622ffe08ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ad8f3700 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd5a250 0x5622ffe05e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd5a250 msgr2=0x5622ffe05e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd5a250 0x5622ffe05e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd76d40 msgr2=0x5622ffe08ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd76d40 0x5622ffe08ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/3412104566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a800a0e0 con 0x5622ffe084a0 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.419+0000 7fe5ad8f3700 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd5a250 0x5622ffe05e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5ae0f4700 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffd58360 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8005930 tx=0x7fe5a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4b8c4e8aa47392 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5a7fff700 1 -- 172.21.15.45:0/3412104566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800f860 con 0x5622ffe084a0 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5a7fff700 1 -- 172.21.15.45:0/3412104566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a800f9e0 con 0x5622ffe084a0 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5a7fff700 1 -- 172.21.15.45:0/3412104566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80246c0 con 0x5622ffe084a0 2023-10-06T05:17:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 -- 172.21.15.45:0/3412104566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 msgr2=0x5622ffd58360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffd58360 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8005930 tx=0x7fe5a8015000 comp rx=0 tx=0).stop 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 -- 172.21.15.45:0/3412104566 shutdown_connections 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffd58360 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd5a250 0x5622ffe05e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/3412104566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd76d40 0x5622ffe08ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 -- 172.21.15.45:0/3412104566 >> 172.21.15.45:0/3412104566 conn(0x5622ffd2dff0 msgr2=0x5622ffe075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 -- 172.21.15.45:0/3412104566 shutdown_connections 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.420+0000 7fe5b46d5040 1 -- 172.21.15.45:0/3412104566 wait complete. 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.421+0000 7fe5b46d5040 1 Processor -- start 2023-10-06T05:17:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.421+0000 7fe5b46d5040 1 -- start start 2023-10-06T05:17:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5b46d5040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd5a250 0x5622ffd51dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5b46d5040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd76d40 0x5622ffdfa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5b46d5040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffe16580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5b46d5040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffd508c0 con 0x5622ffe084a0 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5b46d5040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffdfa800 con 0x5622ffd5a250 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5b46d5040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622ffd52470 con 0x5622ffd76d40 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffe16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffe16580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43188/0 (socket says 172.21.15.45:43188) 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/2279760592 learned_addr learned my addr 172.21.15.45:0/2279760592 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ad0f2700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd76d40 0x5622ffdfa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ad8f3700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd5a250 0x5622ffd51dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd76d40 msgr2=0x5622ffdfa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622ffd76d40 0x5622ffdfa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd5a250 msgr2=0x5622ffd51dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622ffd5a250 0x5622ffd51dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.422+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/2279760592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622ffca21c0 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.423+0000 7fe5ae0f4700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffe16580 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800f640 tx=0x7fe5a801d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.423+0000 7fe5a67fc700 1 -- 172.21.15.45:0/2279760592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800fe00 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.423+0000 7fe5a67fc700 1 -- 172.21.15.45:0/2279760592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a8024c40 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.423+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5622ffc883d0 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.423+0000 7fe5a67fc700 1 -- 172.21.15.45:0/2279760592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a802f420 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.423+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622ffc7bae0 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.424+0000 7fe5a67fc700 1 -- 172.21.15.45:0/2279760592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5a8021070 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.425+0000 7fe5a67fc700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590065890 0x7fe590067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.425+0000 7fe5a67fc700 1 -- 172.21.15.45:0/2279760592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fe5a802b070 con 0x5622ffe084a0 2023-10-06T05:17:14.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.425+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5622ffe16ac0 0x5622ffd511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.425+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) v8 -- 0x5622ffe05710 con 0x5622ffe16ac0 2023-10-06T05:17:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.425+0000 7fe5ad8f3700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590065890 0x7fe590067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.425+0000 7fe5ad0f2700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5622ffe16ac0 0x5622ffd511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.426+0000 7fe5ad0f2700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5622ffe16ac0 0x5622ffd511e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.426+0000 7fe5ad8f3700 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590065890 0x7fe590067d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe598001ed0 tx=0x7fe598000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5ad0f2700 1 -- 172.21.15.45:0/2279760592 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v41'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe5a0009be0 con 0x5622ffe16ac0 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5622ffe16ac0 msgr2=0x5622ffd511e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5622ffe16ac0 0x5622ffd511e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590065890 msgr2=0x7fe590067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590065890 0x7fe590067d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe598001ed0 tx=0x7fe598000bc0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 msgr2=0x5622ffe16580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 --2- 172.21.15.45:0/2279760592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622ffe084a0 0x5622ffe16580 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800f640 tx=0x7fe5a801d070 comp rx=0 tx=0).stop 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5ae0f4700 1 -- 172.21.15.45:0/2279760592 reap_dead start 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 shutdown_connections 2023-10-06T05:17:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 >> 172.21.15.45:0/2279760592 conn(0x5622ffd2dff0 msgr2=0x5622ffd58320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 shutdown_connections 2023-10-06T05:17:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.427+0000 7fe5b46d5040 1 -- 172.21.15.45:0/2279760592 wait complete. 2023-10-06T05:17:14.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.567+0000 7f852ee60700 1 Processor -- start 2023-10-06T05:17:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.567+0000 7f852ee60700 1 -- start start 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852ee60700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85280ccfc0 0x7f852815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852ee60700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85280ce690 con 0x7f8528153620 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852ee60700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852815d510 con 0x7f85281539f0 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852ee60700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852815a7a0 con 0x7f85280ccfc0 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852d3fd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85280ccfc0 0x7f852815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852d3fd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85280ccfc0 0x7f852815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43204/0 (socket says 172.21.15.45:43204) 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852cbfc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8528153620 0x7f85280cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46698/0 (socket says 172.21.15.45:46698) 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.568+0000 7f852d3fd700 1 -- 172.21.15.45:0/3525248137 learned_addr learned my addr 172.21.15.45:0/3525248137 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f852d3fd700 1 -- 172.21.15.45:0/3525248137 >> v1:172.21.15.144:6789/0 conn(0x7f85281539f0 legacy=0x7f85280ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f852d3fd700 1 -- 172.21.15.45:0/3525248137 >> v1:172.21.15.45:6789/0 conn(0x7f8528153620 legacy=0x7f85280cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f852d3fd700 1 -- 172.21.15.45:0/3525248137 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852815d7f0 con 0x7f85280ccfc0 2023-10-06T05:17:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f851f7fe700 1 -- 172.21.15.45:0/3525248137 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158614110 0 0) 0x7f85280ce690 con 0x7f8528153620 2023-10-06T05:17:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f852d3fd700 1 --2- 172.21.15.45:0/3525248137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280ccfc0 0x7f852815a260 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8520009f20 tx=0x7f8520009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2965b757043931e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f851f7fe700 1 -- 172.21.15.45:0/3525248137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000e070 con 0x7f85280ccfc0 2023-10-06T05:17:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f851f7fe700 1 -- 172.21.15.45:0/3525248137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852000ba80 con 0x7f85280ccfc0 2023-10-06T05:17:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.569+0000 7f851f7fe700 1 -- 172.21.15.45:0/3525248137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200092e0 con 0x7f85280ccfc0 2023-10-06T05:17:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.570+0000 7f852ee60700 1 -- 172.21.15.45:0/3525248137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280ccfc0 msgr2=0x7f852815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.570+0000 7f852ee60700 1 --2- 172.21.15.45:0/3525248137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280ccfc0 0x7f852815a260 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8520009f20 tx=0x7f8520009840 comp rx=0 tx=0).stop 2023-10-06T05:17:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.570+0000 7f852ee60700 1 -- 172.21.15.45:0/3525248137 shutdown_connections 2023-10-06T05:17:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.570+0000 7f852ee60700 1 --2- 172.21.15.45:0/3525248137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280ccfc0 0x7f852815a260 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.570+0000 7f852ee60700 1 -- 172.21.15.45:0/3525248137 >> 172.21.15.45:0/3525248137 conn(0x7f852806ed50 msgr2=0x7f852806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.570+0000 7f852ee60700 1 -- 172.21.15.45:0/3525248137 shutdown_connections 2023-10-06T05:17:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.570+0000 7f852ee60700 1 -- 172.21.15.45:0/3525248137 wait complete. 2023-10-06T05:17:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.571+0000 7f852ee60700 1 Processor -- start 2023-10-06T05:17:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.571+0000 7f852ee60700 1 -- start start 2023-10-06T05:17:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852ee60700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85280ccfc0 0x7f85280afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852ee60700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8528153620 0x7f85280caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852ee60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852814f480 0x7f85280b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852ee60700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528158760 con 0x7f852814f480 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852ee60700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85280b1060 con 0x7f85280ccfc0 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852ee60700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85280b0450 con 0x7f8528153620 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852d3fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852814f480 0x7f85280b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852d3fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852814f480 0x7f85280b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43210/0 (socket says 172.21.15.45:43210) 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852d3fd700 1 -- 172.21.15.45:0/2236058343 learned_addr learned my addr 172.21.15.45:0/2236058343 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f852cbfc700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85280ccfc0 0x7f85280afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.572+0000 7f851ffff700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8528153620 0x7f85280caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852d3fd700 1 -- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8528153620 msgr2=0x7f85280caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852d3fd700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8528153620 0x7f85280caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852d3fd700 1 -- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85280ccfc0 msgr2=0x7f85280afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852d3fd700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85280ccfc0 0x7f85280afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852d3fd700 1 -- 172.21.15.45:0/2236058343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8520009ae0 con 0x7f852814f480 2023-10-06T05:17:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852cbfc700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85280ccfc0 0x7f85280afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852d3fd700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852814f480 0x7f85280b14f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f852806f900 tx=0x7f8520018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f851cff9700 1 -- 172.21.15.45:0/2236058343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520007e70 con 0x7f852814f480 2023-10-06T05:17:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f851cff9700 1 -- 172.21.15.45:0/2236058343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8520026460 con 0x7f852814f480 2023-10-06T05:17:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85280b0730 con 0x7f852814f480 2023-10-06T05:17:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f851cff9700 1 -- 172.21.15.45:0/2236058343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520026840 con 0x7f852814f480 2023-10-06T05:17:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.573+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85280b0cf0 con 0x7f852814f480 2023-10-06T05:17:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.574+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85280b1210 con 0x7f852814f480 2023-10-06T05:17:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.575+0000 7f851cff9700 1 -- 172.21.15.45:0/2236058343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f852001d420 con 0x7f852814f480 2023-10-06T05:17:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.575+0000 7f851cff9700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8514066f30 0x7f85140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.575+0000 7f851cff9700 1 -- 172.21.15.45:0/2236058343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f85200869c0 con 0x7f852814f480 2023-10-06T05:17:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.575+0000 7f852cbfc700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8514066f30 0x7f85140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.576+0000 7f852cbfc700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8514066f30 0x7f85140693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85240011e0 tx=0x7f8524007c70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.579+0000 7f851cff9700 1 -- 172.21.15.45:0/2236058343 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85280b1210 con 0x7f852814f480 2023-10-06T05:17:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.827+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85280b11d0 con 0x7f852814f480 2023-10-06T05:17:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.827+0000 7f851cff9700 1 -- 172.21.15.45:0/2236058343 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f85200539b0 con 0x7f852814f480 2023-10-06T05:17:14.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:14.829 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:37.788147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:17:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.829+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8514066f30 msgr2=0x7f85140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8514066f30 0x7f85140693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85240011e0 tx=0x7f8524007c70 comp rx=0 tx=0).stop 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852814f480 msgr2=0x7f85280b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852814f480 0x7f85280b14f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f852806f900 tx=0x7f8520018000 comp rx=0 tx=0).stop 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 shutdown_connections 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8514066f30 0x7f85140693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852814f480 0x7f85280b14f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85280ccfc0 0x7f85280afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 --2- 172.21.15.45:0/2236058343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8528153620 0x7f85280caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 >> 172.21.15.45:0/2236058343 conn(0x7f852806ed50 msgr2=0x7f85280c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 shutdown_connections 2023-10-06T05:17:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.830+0000 7f852ee60700 1 -- 172.21.15.45:0/2236058343 wait complete. 2023-10-06T05:17:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 23 2023-10-06T05:17:14.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 4114013756 2023-10-06T05:17:14.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.977+0000 7f2ab2e00700 1 Processor -- start 2023-10-06T05:17:14.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.977+0000 7f2ab2e00700 1 -- start start 2023-10-06T05:17:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2ab2e00700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2aac157330 0x7f2aac0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2ab2e00700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2aac0aaae0 0x7f2aac0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2ab2e00700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac0c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2ab2e00700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac0c3b50 con 0x7f2aac0aaae0 2023-10-06T05:17:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2ab2e00700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac157700 con 0x7f2aac157330 2023-10-06T05:17:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2ab2e00700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c98e0 con 0x7f2aac0c90b0 2023-10-06T05:17:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2aabfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2aac0aaae0 0x7f2aac0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2aabfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2aac0aaae0 0x7f2aac0c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43218/0 (socket says 172.21.15.45:43218) 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2aabfff700 1 -- 172.21.15.45:0/342931444 learned_addr learned my addr 172.21.15.45:0/342931444 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.978+0000 7f2ab139d700 1 --2- 172.21.15.45:0/342931444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac0c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/342931444 >> v2:172.21.15.144:3300/0 conn(0x7f2aac157330 0x7f2aac0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aabfff700 1 -- 172.21.15.45:0/342931444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 msgr2=0x7f2aac0c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aabfff700 1 --2- 172.21.15.45:0/342931444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac0c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aabfff700 1 -- 172.21.15.45:0/342931444 >> v2:172.21.15.144:3300/0 conn(0x7f2aac157330 msgr2=0x7f2aac0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aabfff700 1 --2- 172.21.15.45:0/342931444 >> v2:172.21.15.144:3300/0 conn(0x7f2aac157330 0x7f2aac0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aabfff700 1 -- 172.21.15.45:0/342931444 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0c9bc0 con 0x7f2aac0aaae0 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/342931444 >> v2:172.21.15.144:3300/0 conn(0x7f2aac157330 0x7f2aac0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aabfff700 1 --2- 172.21.15.45:0/342931444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac0c8b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0cd1d0 tx=0x7f2aa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d770f1431229bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aab7fe700 1 -- 172.21.15.45:0/342931444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0009e70 con 0x7f2aac0aaae0 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aab7fe700 1 -- 172.21.15.45:0/342931444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa000b860 con 0x7f2aac0aaae0 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.979+0000 7f2aab7fe700 1 -- 172.21.15.45:0/342931444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000bc00 con 0x7f2aac0aaae0 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 -- 172.21.15.45:0/342931444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 msgr2=0x7f2aac0c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/342931444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac0c8b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0cd1d0 tx=0x7f2aa0009250 comp rx=0 tx=0).stop 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 -- 172.21.15.45:0/342931444 shutdown_connections 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/342931444 >> v2:172.21.15.144:3300/0 conn(0x7f2aac157330 0x7f2aac0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/342931444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac0c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/342931444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac0c8b70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 -- 172.21.15.45:0/342931444 >> 172.21.15.45:0/342931444 conn(0x7f2aac067e00 msgr2=0x7f2aac0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 -- 172.21.15.45:0/342931444 shutdown_connections 2023-10-06T05:17:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.980+0000 7f2ab2e00700 1 -- 172.21.15.45:0/342931444 wait complete. 2023-10-06T05:17:14.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.981+0000 7f2ab2e00700 1 Processor -- start 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.981+0000 7f2ab2e00700 1 -- start start 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab2e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab2e00700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab2e00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac157330 0x7f2aac0abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab2e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac1595d0 con 0x7f2aac0aaae0 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab2e00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c6080 con 0x7f2aac157330 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab2e00700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac157de0 con 0x7f2aac0c90b0 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43220/0 (socket says 172.21.15.45:43220) 2023-10-06T05:17:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/931965170 learned_addr learned my addr 172.21.15.45:0/931965170 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2aabfff700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 msgr2=0x7f2aac156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab139d700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac157330 0x7f2aac0abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac157330 msgr2=0x7f2aac0abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac157330 0x7f2aac0abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/931965170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00094f0 con 0x7f2aac0aaae0 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2ab139d700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac157330 0x7f2aac0abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.982+0000 7f2aabfff700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.983+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac154070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e920 tx=0x7f2a9c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.983+0000 7f2aa8ff9700 1 -- 172.21.15.45:0/931965170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c005a00 con 0x7f2aac0aaae0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.983+0000 7f2aa8ff9700 1 -- 172.21.15.45:0/931965170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c005b60 con 0x7f2aac0aaae0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.983+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac1580c0 con 0x7f2aac0aaae0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.983+0000 7f2aa8ff9700 1 -- 172.21.15.45:0/931965170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c006020 con 0x7f2aac0aaae0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.983+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac158680 con 0x7f2aac0aaae0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.984+0000 7f2aa8ff9700 1 -- 172.21.15.45:0/931965170 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a9c006410 con 0x7f2aac0aaae0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.985+0000 7f2aa8ff9700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a98066f30 0x7f2a980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.985+0000 7f2aa8ff9700 1 -- 172.21.15.45:0/931965170 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f2a9c084a30 con 0x7f2aac0aaae0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.985+0000 7f2aabfff700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a98066f30 0x7f2a980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.986+0000 7f2aabfff700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a98066f30 0x7f2a980693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0cd1d0 tx=0x7f2aa0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 Processor -- start 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 -- start start 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c6647818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66478a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66478a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c664654390 con 0x55c664766fc0 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6646911b0 con 0x55c66478a550 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.987+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66468eb70 con 0x55c6647814d0 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66478a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66478a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43234/0 (socket says 172.21.15.45:43234) 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 -- 172.21.15.45:0/718453621 learned_addr learned my addr 172.21.15.45:0/718453621 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c6647818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2ab139d700 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66478a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 -- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 msgr2=0x55c6647818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c6647818a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 -- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 msgr2=0x55c66478a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66478a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 -- 172.21.15.45:0/718453621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c664490810 con 0x55c664766fc0 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.988+0000 7f2aabfff700 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66478a010 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00030a0 tx=0x7f2aa001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d26dbb44fb1740 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2a91ffb700 1 -- 172.21.15.45:0/718453621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000bad0 con 0x55c664766fc0 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2a91ffb700 1 -- 172.21.15.45:0/718453621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa002b070 con 0x55c664766fc0 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2a91ffb700 1 -- 172.21.15.45:0/718453621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa001baf0 con 0x55c664766fc0 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 -- 172.21.15.45:0/718453621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 msgr2=0x55c66478a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66478a010 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00030a0 tx=0x7f2aa001d040 comp rx=0 tx=0).stop 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 -- 172.21.15.45:0/718453621 shutdown_connections 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66478a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c6647818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 --2- 172.21.15.45:0/718453621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66478a010 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 -- 172.21.15.45:0/718453621 >> 172.21.15.45:0/718453621 conn(0x55c664784900 msgr2=0x55c664784cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 -- 172.21.15.45:0/718453621 shutdown_connections 2023-10-06T05:17:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.989+0000 7f2abc114b80 1 -- 172.21.15.45:0/718453621 wait complete. 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.990+0000 7f2abc114b80 1 -- start start 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.990+0000 7f2abc114b80 1 Processor -- start 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66475c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c664780870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66473b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2abc114b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66468a960 con 0x55c664766fc0 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c664636a00 con 0x55c66478a550 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6646366a0 con 0x55c6647814d0 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2abc114b80 10 client.0 _ll_register_callbacks cb 0x55c664499a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2ab0b9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66475c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2ab0b9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66475c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43238/0 (socket says 172.21.15.45:43238) 2023-10-06T05:17:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/4246531642 learned_addr learned my addr 172.21.15.45:0/4246531642 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2aabfff700 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c664780870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.991+0000 7f2ab139d700 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66473b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 msgr2=0x55c664780870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c664780870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 msgr2=0x55c66473b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66473b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2ab0b9c700 1 -- 172.21.15.45:0/4246531642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa0021050 con 0x55c664766fc0 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66475c440 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e8f0 tx=0x7f2a9c058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e4819b80626169 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2a8affd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2a8affd700 1 -- 172.21.15.45:0/4246531642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c053930 con 0x55c664766fc0 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2a8affd700 1 -- 172.21.15.45:0/4246531642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f2a9c0557f0 con 0x55c664766fc0 2023-10-06T05:17:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c66472ede0 con 0x55c664766fc0 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2a8affd700 1 -- 172.21.15.45:0/4246531642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a9c054a80 con 0x55c664766fc0 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 10 client.5526 fetch_fsmap learned FSMap version 23 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55c66457d120 con 0x55c664766fc0 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2a8affd700 1 -- 172.21.15.45:0/4246531642 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c66472ede0 con 0x55c664766fc0 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2a8affd700 1 -- 172.21.15.45:0/4246531642 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2a9c05c0a0 con 0x55c664766fc0 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 10 client.5526 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 10 client.5526 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 20 client.5526 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664760a10 0x55c664760de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 4 client.5526 mds_command: new command op to 5322 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.992+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c66470bbb0 con 0x55c664760a10 2023-10-06T05:17:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.993+0000 7f2aabfff700 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664760a10 0x55c664760de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.993+0000 7f2aabfff700 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664760a10 0x55c664760de0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96818337c7cb878 server_cookie=705c5d1cbe5a3b3c in_seq=0 out_seq=0 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.993+0000 7f2a8affd700 10 client.5526 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.993+0000 7f2a8affd700 1 -- 172.21.15.45:0/4246531642 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c66470bbb0 con 0x55c664760a10 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.993+0000 7f2a8affd700 10 client.5526 handle_command_reply: tid=0 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.993+0000 7f2abc114b80 1 client.5526 shutdown 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 msgr2=0x55c66475c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66475c440 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e8f0 tx=0x7f2a9c058040 comp rx=0 tx=0).stop 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 shutdown_connections 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664760a10 0x55c664760de0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c66473b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2a8affd700 0 client.5526 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c664766fc0 0x55c66475c440 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 --2- 172.21.15.45:0/4246531642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6647814d0 0x55c664780870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 >> 172.21.15.45:0/4246531642 conn(0x55c664784900 msgr2=0x55c664736ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 shutdown_connections 2023-10-06T05:17:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 1 -- 172.21.15.45:0/4246531642 wait complete. 2023-10-06T05:17:15.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:14.994+0000 7f2abc114b80 20 client.5526 trim_cache size 0 max 16384 2023-10-06T05:17:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.007+0000 7f2abc114b80 1 Processor -- start 2023-10-06T05:17:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.007+0000 7f2abc114b80 1 -- start start 2023-10-06T05:17:15.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c664766fc0 0x55c6647308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c66473c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c66478a550 0x55c664736290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2abc114b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c664522260 con 0x55c6647814d0 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c664782870 con 0x55c664766fc0 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c664737a80 con 0x55c66478a550 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c66473c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c66473c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43244/0 (socket says 172.21.15.45:43244) 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 -- 172.21.15.45:0/2787428467 learned_addr learned my addr 172.21.15.45:0/2787428467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2ab139d700 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c66478a550 0x55c664736290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c664766fc0 0x55c6647308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 -- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c66478a550 msgr2=0x55c664736290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c66478a550 0x55c664736290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 -- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c664766fc0 msgr2=0x55c6647308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c664766fc0 0x55c6647308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 -- 172.21.15.45:0/2787428467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c66473c5c0 con 0x55c6647814d0 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c664766fc0 0x55c6647308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.008+0000 7f2aabfff700 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c66473c780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00030c0 tx=0x7f2aa0032040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295f979060316f02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2a927fc700 1 -- 172.21.15.45:0/2787428467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa001bd70 con 0x55c6647814d0 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2a927fc700 1 -- 172.21.15.45:0/2787428467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa000b9d0 con 0x55c6647814d0 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2a927fc700 1 -- 172.21.15.45:0/2787428467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000bb60 con 0x55c6647814d0 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 -- 172.21.15.45:0/2787428467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 msgr2=0x55c66473c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c66473c780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00030c0 tx=0x7f2aa0032040 comp rx=0 tx=0).stop 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 -- 172.21.15.45:0/2787428467 shutdown_connections 2023-10-06T05:17:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c66478a550 0x55c664736290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c664766fc0 0x55c6647308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2787428467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c66473c780 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 -- 172.21.15.45:0/2787428467 >> 172.21.15.45:0/2787428467 conn(0x55c664784900 msgr2=0x55c66475c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 -- 172.21.15.45:0/2787428467 shutdown_connections 2023-10-06T05:17:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.009+0000 7f2abc114b80 1 -- 172.21.15.45:0/2787428467 wait complete. 2023-10-06T05:17:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.010+0000 7f2abc114b80 1 -- start start 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.010+0000 7f2abc114b80 1 Processor -- start 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c664766fc0 0x55c6647cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c664784e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2abc114b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c6647308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2abc114b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6647cf540 con 0x55c6647814d0 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6647e7170 con 0x55c66478a550 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2abc114b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66473a2a0 con 0x55c664766fc0 2023-10-06T05:17:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2abc114b80 10 client.0 _ll_register_callbacks cb 0x55c664499a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c664784e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c664784e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43256/0 (socket says 172.21.15.45:43256) 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 -- 172.21.15.45:0/2989151003 learned_addr learned my addr 172.21.15.45:0/2989151003 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c664766fc0 0x55c6647cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2ab139d700 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c6647308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 -- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c664766fc0 msgr2=0x55c6647cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c664766fc0 0x55c6647cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 -- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 msgr2=0x55c6647308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c6647308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.011+0000 7f2aabfff700 1 -- 172.21.15.45:0/2989151003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa0034040 con 0x55c6647814d0 2023-10-06T05:17:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2aabfff700 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c664784e90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00090b0 tx=0x7f2aa000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a387102b626f97b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2a8affd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2a8affd700 1 -- 172.21.15.45:0/2989151003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000beb0 con 0x55c6647814d0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2a8affd700 1 -- 172.21.15.45:0/2989151003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f2aa0039660 con 0x55c6647814d0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aa002b070 con 0x55c6647814d0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2a8affd700 1 -- 172.21.15.45:0/2989151003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa0039a30 con 0x55c6647814d0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 10 client.5532 fetch_fsmap learned FSMap version 23 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55c664659440 con 0x55c6647814d0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2a8affd700 1 -- 172.21.15.45:0/2989151003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa002b070 con 0x55c6647814d0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2a8affd700 1 -- 172.21.15.45:0/2989151003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 23) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2aa0039b90 con 0x55c6647814d0 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 10 client.5532 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 10 client.5532 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 20 client.5532 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664762440 0x55c664762810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 4 client.5532 mds_command: new command op to 5322 tid=0[{"prefix": "damage rm", "damage_id": 4114013756}] 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 --> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] -- command(tid 0: {"prefix": "damage rm", "damage_id": 4114013756}) v1 -- 0x55c66462d950 con 0x55c664762440 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.012+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664762440 0x55c664762810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2a8affd700 10 client.5532 ms_handle_connect on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2ab0b9c700 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664762440 0x55c664762810 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d499fb3d401682d8 server_cookie=12eb44e5dc43f18b in_seq=0 out_seq=0 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2a8affd700 1 -- 172.21.15.45:0/2989151003 <== mds.0 v2:172.21.15.45:6832/1705605429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55c66462d950 con 0x55c664762440 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2a8affd700 10 client.5532 handle_command_reply: tid=0 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 client.5532 shutdown 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 msgr2=0x55c664784e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c664784e90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00090b0 tx=0x7f2aa000b050 comp rx=0 tx=0).stop 2023-10-06T05:17:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 shutdown_connections 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:6832/1705605429,v1:172.21.15.45:6834/1705605429] conn(0x55c664762440 0x55c664762810 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c66478a550 0x55c6647308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c664766fc0 0x55c6647cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2a8affd700 0 client.5532 ms_handle_reset on v2:172.21.15.45:6832/1705605429 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 --2- 172.21.15.45:0/2989151003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6647814d0 0x55c664784e90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 >> 172.21.15.45:0/2989151003 conn(0x55c664784900 msgr2=0x55c66475c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 shutdown_connections 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 1 -- 172.21.15.45:0/2989151003 wait complete. 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.013+0000 7f2abc114b80 20 client.5532 trim_cache size 0 max 16384 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a98066f30 msgr2=0x7f2a980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a98066f30 0x7f2a980693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0cd1d0 tx=0x7f2aa0017000 comp rx=0 tx=0).stop 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 msgr2=0x7f2aac154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac154070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e920 tx=0x7f2a9c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 shutdown_connections 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a98066f30 0x7f2a980693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac157330 0x7f2aac0abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aac0aaae0 0x7f2aac154070 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 --2- 172.21.15.45:0/931965170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac0c90b0 0x7f2aac156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 >> 172.21.15.45:0/931965170 conn(0x7f2aac067e00 msgr2=0x7f2aac0cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 shutdown_connections 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.014+0000 7f2ab2e00700 1 -- 172.21.15.45:0/931965170 wait complete. 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> import errno 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> try: 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> f.close() 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> except IOError as e: 2023-10-06T05:17:15.029 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(errno.EIO) 2023-10-06T05:17:15.030 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:17:15.293 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:17:15.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:15.383 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:15.434 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:17:15.434 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:17:15.434 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:17:15.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:15.475 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[138474]: fuse finished with error 0 and tester_r 0 2023-10-06T05:17:15.477 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:17:15.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:15.477 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:17:15.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:17:15.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:15.585 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:17:15.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T05:17:15.606 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:17:15.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:15.607 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:17:15.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:17:15.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:17:15.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:15.631 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:17:15.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:15.666 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:17:15.666 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.808+0000 7efc3cc41700 1 Processor -- start 2023-10-06T05:17:15.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.808+0000 7efc3cc41700 1 -- start start 2023-10-06T05:17:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.809+0000 7efc3cc41700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc380c9640 0x7efc380b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.809+0000 7efc3cc41700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc380c84c0 con 0x7efc380c91f0 2023-10-06T05:17:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.809+0000 7efc3cc41700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc380c8d60 con 0x7efc380ca810 2023-10-06T05:17:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.809+0000 7efc3cc41700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc38069d00 con 0x7efc380c9640 2023-10-06T05:17:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc35d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efc380c91f0 0x7efc380afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46712/0 (socket says 172.21.15.45:46712) 2023-10-06T05:17:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc35d9b700 1 -- 172.21.15.45:0/2759175709 learned_addr learned my addr 172.21.15.45:0/2759175709 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc36d9d700 1 --2- 172.21.15.45:0/2759175709 >> v2:172.21.15.144:3300/0 conn(0x7efc380c9640 0x7efc380b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc3559a700 1 -- 172.21.15.45:0/2759175709 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020375245 0 0) 0x7efc380c84c0 con 0x7efc380c91f0 2023-10-06T05:17:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc3559a700 1 -- 172.21.15.45:0/2759175709 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003570 con 0x7efc380c91f0 2023-10-06T05:17:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc3559a700 1 -- 172.21.15.45:0/2759175709 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286609655 0 0) 0x7efc380c8d60 con 0x7efc380ca810 2023-10-06T05:17:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc3559a700 1 -- 172.21.15.45:0/2759175709 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc380c84c0 con 0x7efc380ca810 2023-10-06T05:17:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc36d9d700 1 -- 172.21.15.45:0/2759175709 >> v1:172.21.15.144:6789/0 conn(0x7efc380ca810 legacy=0x7efc380cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc36d9d700 1 -- 172.21.15.45:0/2759175709 >> v1:172.21.15.45:6789/0 conn(0x7efc380c91f0 legacy=0x7efc380afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc36d9d700 1 -- 172.21.15.45:0/2759175709 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380c8710 con 0x7efc380c9640 2023-10-06T05:17:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.810+0000 7efc36d9d700 1 --2- 172.21.15.45:0/2759175709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380c9640 0x7efc380b0190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc30009910 tx=0x7efc300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=957992f06b59fd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3559a700 1 -- 172.21.15.45:0/2759175709 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30011630 con 0x7efc380c9640 2023-10-06T05:17:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3559a700 1 -- 172.21.15.45:0/2759175709 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc30011790 con 0x7efc380c9640 2023-10-06T05:17:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3559a700 1 -- 172.21.15.45:0/2759175709 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30011b30 con 0x7efc380c9640 2023-10-06T05:17:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3cc41700 1 -- 172.21.15.45:0/2759175709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380c9640 msgr2=0x7efc380b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3cc41700 1 --2- 172.21.15.45:0/2759175709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380c9640 0x7efc380b0190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc30009910 tx=0x7efc300091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3cc41700 1 -- 172.21.15.45:0/2759175709 shutdown_connections 2023-10-06T05:17:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3cc41700 1 --2- 172.21.15.45:0/2759175709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380c9640 0x7efc380b0190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3cc41700 1 -- 172.21.15.45:0/2759175709 >> 172.21.15.45:0/2759175709 conn(0x7efc38067620 msgr2=0x7efc38067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3cc41700 1 -- 172.21.15.45:0/2759175709 shutdown_connections 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.811+0000 7efc3cc41700 1 -- 172.21.15.45:0/2759175709 wait complete. 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 Processor -- start 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 -- start start 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c91f0 0x7efc38158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc380c9640 0x7efc3815df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380ca810 0x7efc3815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380c2990 con 0x7efc380c91f0 2023-10-06T05:17:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380c26b0 con 0x7efc380ca810 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3cc41700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380b0d70 con 0x7efc380c9640 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c91f0 0x7efc38158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c91f0 0x7efc38158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43268/0 (socket says 172.21.15.45:43268) 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.813+0000 7efc3659c700 1 -- 172.21.15.45:0/3369902114 learned_addr learned my addr 172.21.15.45:0/3369902114 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc36d9d700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380ca810 0x7efc3815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc35d9b700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc380c9640 0x7efc3815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3659c700 1 -- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc380c9640 msgr2=0x7efc3815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3659c700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc380c9640 0x7efc3815df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3659c700 1 -- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380ca810 msgr2=0x7efc3815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3659c700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380ca810 0x7efc3815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3659c700 1 -- 172.21.15.45:0/3369902114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30009420 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc36d9d700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380ca810 0x7efc3815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3659c700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c91f0 0x7efc38158620 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c900 tx=0x7efc28002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc267fc700 1 -- 172.21.15.45:0/3369902114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28012630 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc267fc700 1 -- 172.21.15.45:0/3369902114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28012790 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc380b1050 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc267fc700 1 -- 172.21.15.45:0/3369902114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28012b50 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.814+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc38158b60 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.815+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc380b14f0 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.816+0000 7efc267fc700 1 -- 172.21.15.45:0/3369902114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc2801b400 con 0x7efc380c91f0 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.816+0000 7efc267fc700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc18066f30 0x7efc180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.816+0000 7efc267fc700 1 -- 172.21.15.45:0/3369902114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7efc28084ba0 con 0x7efc380c91f0 2023-10-06T05:17:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.816+0000 7efc35d9b700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc18066f30 0x7efc180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.817+0000 7efc35d9b700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc18066f30 0x7efc180693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc380cc180 tx=0x7efc2c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:15.821+0000 7efc267fc700 1 -- 172.21.15.45:0/3369902114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc28051ae0 con 0x7efc380c91f0 2023-10-06T05:17:16.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.067+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc380c89f0 con 0x7efc380c91f0 2023-10-06T05:17:16.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.068+0000 7efc267fc700 1 -- 172.21.15.45:0/3369902114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4604 (secure 0 0 0) 0x7efc28055100 con 0x7efc380c91f0 2023-10-06T05:17:16.070 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:16.070 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:16:37.788147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:17:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc18066f30 msgr2=0x7efc180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc18066f30 0x7efc180693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc380cc180 tx=0x7efc2c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c91f0 msgr2=0x7efc38158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c91f0 0x7efc38158620 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c900 tx=0x7efc28002780 comp rx=0 tx=0).stop 2023-10-06T05:17:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 shutdown_connections 2023-10-06T05:17:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc18066f30 0x7efc180693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc380ca810 0x7efc3815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c91f0 0x7efc38158620 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 --2- 172.21.15.45:0/3369902114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc380c9640 0x7efc3815df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 >> 172.21.15.45:0/3369902114 conn(0x7efc38067620 msgr2=0x7efc3815a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 shutdown_connections 2023-10-06T05:17:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.070+0000 7efc3cc41700 1 -- 172.21.15.45:0/3369902114 wait complete. 2023-10-06T05:17:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 23 2023-10-06T05:17:16.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:16.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.216+0000 7fb530f88700 1 Processor -- start 2023-10-06T05:17:16.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.216+0000 7fb530f88700 1 -- start start 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb530f88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c159330 0x7fb52c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb530f88700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0cb530 con 0x7fb52c0cc390 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb530f88700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c069bd0 con 0x7fb52c158f60 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb530f88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1597a0 con 0x7fb52c159330 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb52ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb52c0cc390 0x7fb52c0c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46726/0 (socket says 172.21.15.45:46726) 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb52ad9d700 1 -- 172.21.15.45:0/3387201071 learned_addr learned my addr 172.21.15.45:0/3387201071 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb529d9b700 1 --2- 172.21.15.45:0/3387201071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c159330 0x7fb52c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.217+0000 7fb52959a700 1 -- 172.21.15.45:0/3387201071 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885520215 0 0) 0x7fb524001490 con 0x7fb52c0cc390 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb52959a700 1 -- 172.21.15.45:0/3387201071 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb514003570 con 0x7fb52c0cc390 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb52959a700 1 -- 172.21.15.45:0/3387201071 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518059701 0 0) 0x7fb52c069bd0 con 0x7fb52c158f60 2023-10-06T05:17:16.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb52959a700 1 -- 172.21.15.45:0/3387201071 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb524001490 con 0x7fb52c158f60 2023-10-06T05:17:16.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb529d9b700 1 -- 172.21.15.45:0/3387201071 >> v1:172.21.15.144:6789/0 conn(0x7fb52c158f60 legacy=0x7fb52c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb529d9b700 1 -- 172.21.15.45:0/3387201071 >> v1:172.21.15.45:6789/0 conn(0x7fb52c0cc390 legacy=0x7fb52c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb529d9b700 1 -- 172.21.15.45:0/3387201071 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c068fc0 con 0x7fb52c159330 2023-10-06T05:17:16.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb529d9b700 1 --2- 172.21.15.45:0/3387201071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c159330 0x7fb52c06a060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009910 tx=0x7fb51c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6230a137c48fad81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb52959a700 1 -- 172.21.15.45:0/3387201071 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011630 con 0x7fb52c159330 2023-10-06T05:17:16.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb52959a700 1 -- 172.21.15.45:0/3387201071 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c011790 con 0x7fb52c159330 2023-10-06T05:17:16.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.218+0000 7fb52959a700 1 -- 172.21.15.45:0/3387201071 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011b30 con 0x7fb52c159330 2023-10-06T05:17:16.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.219+0000 7fb530f88700 1 -- 172.21.15.45:0/3387201071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c159330 msgr2=0x7fb52c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.219+0000 7fb530f88700 1 --2- 172.21.15.45:0/3387201071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c159330 0x7fb52c06a060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009910 tx=0x7fb51c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.219+0000 7fb530f88700 1 -- 172.21.15.45:0/3387201071 shutdown_connections 2023-10-06T05:17:16.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.219+0000 7fb530f88700 1 --2- 172.21.15.45:0/3387201071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c159330 0x7fb52c06a060 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.219+0000 7fb530f88700 1 -- 172.21.15.45:0/3387201071 >> 172.21.15.45:0/3387201071 conn(0x7fb52c0cdf50 msgr2=0x7fb52c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.219+0000 7fb530f88700 1 -- 172.21.15.45:0/3387201071 shutdown_connections 2023-10-06T05:17:16.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.219+0000 7fb530f88700 1 -- 172.21.15.45:0/3387201071 wait complete. 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.220+0000 7fb530f88700 1 Processor -- start 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.220+0000 7fb530f88700 1 -- start start 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb530f88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb52c0cc390 0x7fb52c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb530f88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c158f60 0x7fb52c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb530f88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb52c159330 0x7fb52c0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb530f88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0c6de0 con 0x7fb52c0cc390 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb530f88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b2900 con 0x7fb52c159330 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb530f88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b2010 con 0x7fb52c158f60 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb52c0cc390 0x7fb52c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb52c0cc390 0x7fb52c154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43282/0 (socket says 172.21.15.45:43282) 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 -- 172.21.15.45:0/3998520027 learned_addr learned my addr 172.21.15.45:0/3998520027 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb529d9b700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c158f60 0x7fb52c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52ad9d700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb52c159330 0x7fb52c0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 -- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c158f60 msgr2=0x7fb52c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c158f60 0x7fb52c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 -- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb52c159330 msgr2=0x7fb52c0c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb52c159330 0x7fb52c0c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52a59c700 1 -- 172.21.15.45:0/3998520027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c009420 con 0x7fb52c0cc390 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.221+0000 7fb52ad9d700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb52c159330 0x7fb52c0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.222+0000 7fb52a59c700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb52c0cc390 0x7fb52c154f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c790 tx=0x7fb520007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.222+0000 7fb51a7fc700 1 -- 172.21.15.45:0/3998520027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520010510 con 0x7fb52c0cc390 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.222+0000 7fb51a7fc700 1 -- 172.21.15.45:0/3998520027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb520010690 con 0x7fb52c0cc390 2023-10-06T05:17:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.222+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c0b22b0 con 0x7fb52c0cc390 2023-10-06T05:17:16.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.222+0000 7fb51a7fc700 1 -- 172.21.15.45:0/3998520027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520010a50 con 0x7fb52c0cc390 2023-10-06T05:17:16.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.222+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c155460 con 0x7fb52c0cc390 2023-10-06T05:17:16.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.223+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0cbb60 con 0x7fb52c0cc390 2023-10-06T05:17:16.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.223+0000 7fb51a7fc700 1 -- 172.21.15.45:0/3998520027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb52001b400 con 0x7fb52c0cc390 2023-10-06T05:17:16.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.224+0000 7fb51a7fc700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb50c066f30 0x7fb50c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.224+0000 7fb51a7fc700 1 -- 172.21.15.45:0/3998520027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fb520084630 con 0x7fb52c0cc390 2023-10-06T05:17:16.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.224+0000 7fb529d9b700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb50c066f30 0x7fb50c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.225+0000 7fb529d9b700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb50c066f30 0x7fb50c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009860 tx=0x7fb51c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.228+0000 7fb51a7fc700 1 -- 172.21.15.45:0/3998520027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5200514d0 con 0x7fb52c0cc390 2023-10-06T05:17:16.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.431+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb52c0692a0 con 0x7fb52c0cc390 2023-10-06T05:17:16.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.432+0000 7fb51a7fc700 1 -- 172.21.15.45:0/3998520027 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17391 (secure 0 0 0) 0x7fb520054af0 con 0x7fb52c0cc390 2023-10-06T05:17:16.435 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:16.435 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:16:48.823654+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:08.040887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:08.594370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/2918652845":"2023-10-07T05:16:26.150462+0000","172.21.15.45:6832/2918652845":"2023-10-07T05:16:26.150462+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb50c066f30 msgr2=0x7fb50c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb50c066f30 0x7fb50c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009860 tx=0x7fb51c017000 comp rx=0 tx=0).stop 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb52c0cc390 msgr2=0x7fb52c154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb52c0cc390 0x7fb52c154f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c790 tx=0x7fb520007e10 comp rx=0 tx=0).stop 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 shutdown_connections 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb50c066f30 0x7fb50c0693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb52c159330 0x7fb52c0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb52c0cc390 0x7fb52c154f20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 --2- 172.21.15.45:0/3998520027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb52c158f60 0x7fb52c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 >> 172.21.15.45:0/3998520027 conn(0x7fb52c0cdf50 msgr2=0x7fb52c157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 shutdown_connections 2023-10-06T05:17:16.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.435+0000 7fb530f88700 1 -- 172.21.15.45:0/3998520027 wait complete. 2023-10-06T05:17:16.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:08.040887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:16:08.594370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:16.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:16.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.584+0000 7fb6d6091700 1 Processor -- start 2023-10-06T05:17:16.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.584+0000 7fb6d6091700 1 -- start start 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6d6091700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6d013f900 0x7fb6d00cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6d6091700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d00c9f20 0x7fb6d00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6d6091700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d00c9630 con 0x7fb6d00c8de0 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6d6091700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d00cb550 con 0x7fb6d013f900 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6d6091700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00cb830 con 0x7fb6d00c9f20 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6cf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6d013f900 0x7fb6d00cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6ceffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb6d00c8de0 0x7fb6d00c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41672/0 (socket says 172.21.15.45:41672) 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6ceffd700 1 -- 172.21.15.45:0/3371216440 learned_addr learned my addr 172.21.15.45:0/3371216440 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6cffff700 1 --2- 172.21.15.45:0/3371216440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d00c9f20 0x7fb6d00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/3371216440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d00c9f20 msgr2=0x7fb6d00affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6cf7fe700 1 --2- 172.21.15.45:0/3371216440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d00c9f20 0x7fb6d00affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/3371216440 >> v1:172.21.15.45:6789/0 conn(0x7fb6d00c8de0 legacy=0x7fb6d00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/3371216440 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d00cbb10 con 0x7fb6d013f900 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6ce7fc700 1 -- 172.21.15.45:0/3371216440 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956436380 0 0) 0x7fb6d00c9630 con 0x7fb6d00c8de0 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.585+0000 7fb6cffff700 1 --2- 172.21.15.45:0/3371216440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d00c9f20 0x7fb6d00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6cf7fe700 1 --2- 172.21.15.45:0/3371216440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d013f900 0x7fb6d00cc840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098e0 tx=0x7fb6c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9208ef4bc90b714f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6ce7fc700 1 -- 172.21.15.45:0/3371216440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0009e80 con 0x7fb6d013f900 2023-10-06T05:17:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6ce7fc700 1 -- 172.21.15.45:0/3371216440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000b860 con 0x7fb6d013f900 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6ce7fc700 1 -- 172.21.15.45:0/3371216440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000bbe0 con 0x7fb6d013f900 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 -- 172.21.15.45:0/3371216440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d013f900 msgr2=0x7fb6d00cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 --2- 172.21.15.45:0/3371216440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d013f900 0x7fb6d00cc840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098e0 tx=0x7fb6c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 -- 172.21.15.45:0/3371216440 shutdown_connections 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 --2- 172.21.15.45:0/3371216440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d00c9f20 0x7fb6d00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 --2- 172.21.15.45:0/3371216440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d013f900 0x7fb6d00cc840 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 -- 172.21.15.45:0/3371216440 >> 172.21.15.45:0/3371216440 conn(0x7fb6d00691c0 msgr2=0x7fb6d00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 -- 172.21.15.45:0/3371216440 shutdown_connections 2023-10-06T05:17:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.586+0000 7fb6d6091700 1 -- 172.21.15.45:0/3371216440 wait complete. 2023-10-06T05:17:16.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.588+0000 7fb6d6091700 1 Processor -- start 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.588+0000 7fb6d6091700 1 -- start start 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d00c8de0 0x7fb6d00b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d00c9f20 0x7fb6d00c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d013f900 0x7fb6d01557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d0155090 con 0x7fb6d00c9f20 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d0155330 con 0x7fb6d00c8de0 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d013f210 con 0x7fb6d013f900 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d00c9f20 0x7fb6d00c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d00c9f20 0x7fb6d00c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52240/0 (socket says 172.21.15.45:52240) 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 -- 172.21.15.45:0/672101435 learned_addr learned my addr 172.21.15.45:0/672101435 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6cffff700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d013f900 0x7fb6d01557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6cf7fe700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d00c8de0 0x7fb6d00b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 -- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d013f900 msgr2=0x7fb6d01557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d013f900 0x7fb6d01557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 -- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d00c8de0 msgr2=0x7fb6d00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d00c8de0 0x7fb6d00b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 -- 172.21.15.45:0/672101435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0009420 con 0x7fb6d00c9f20 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6cffff700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d013f900 0x7fb6d01557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6cf7fe700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d00c8de0 0x7fb6d00b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6ceffd700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d00c9f20 0x7fb6d00c5f30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c610 tx=0x7fb6b8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6c77fe700 1 -- 172.21.15.45:0/672101435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8005b30 con 0x7fb6d00c9f20 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6c77fe700 1 -- 172.21.15.45:0/672101435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b8005c90 con 0x7fb6d00c9f20 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d013ec50 con 0x7fb6d00c9f20 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6c77fe700 1 -- 172.21.15.45:0/672101435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8007300 con 0x7fb6d00c9f20 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.589+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d00b1c00 con 0x7fb6d00c9f20 2023-10-06T05:17:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.590+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d00b1e20 con 0x7fb6d00c9f20 2023-10-06T05:17:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.591+0000 7fb6c77fe700 1 -- 172.21.15.45:0/672101435 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6b801b3b0 con 0x7fb6d00c9f20 2023-10-06T05:17:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.591+0000 7fb6c77fe700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b0066f30 0x7fb6b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.591+0000 7fb6c77fe700 1 -- 172.21.15.45:0/672101435 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fb6b8083de0 con 0x7fb6d00c9f20 2023-10-06T05:17:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.592+0000 7fb6cf7fe700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b0066f30 0x7fb6b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.592+0000 7fb6cf7fe700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b0066f30 0x7fb6b00693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0003080 tx=0x7fb6c0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.596+0000 7fb6c77fe700 1 -- 172.21.15.45:0/672101435 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6b8050d20 con 0x7fb6d00c9f20 2023-10-06T05:17:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.796+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6d0155470 con 0x7fb6d00c9f20 2023-10-06T05:17:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.797+0000 7fb6c77fe700 1 -- 172.21.15.45:0/672101435 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6b8017020 con 0x7fb6d00c9f20 2023-10-06T05:17:16.799 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b0066f30 msgr2=0x7fb6b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b0066f30 0x7fb6b00693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0003080 tx=0x7fb6c0017000 comp rx=0 tx=0).stop 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d00c9f20 msgr2=0x7fb6d00c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d00c9f20 0x7fb6d00c5f30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c610 tx=0x7fb6b8001070 comp rx=0 tx=0).stop 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 shutdown_connections 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b0066f30 0x7fb6b00693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d013f900 0x7fb6d01557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d00c8de0 0x7fb6d00b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 --2- 172.21.15.45:0/672101435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d00c9f20 0x7fb6d00c5f30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 >> 172.21.15.45:0/672101435 conn(0x7fb6d00691c0 msgr2=0x7fb6d00c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 shutdown_connections 2023-10-06T05:17:16.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.799+0000 7fb6d6091700 1 -- 172.21.15.45:0/672101435 wait complete. 2023-10-06T05:17:16.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:16.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.947+0000 7f27a5d74700 1 Processor -- start 2023-10-06T05:17:16.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.947+0000 7f27a5d74700 1 -- start start 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f27a5d74700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27a0158a60 0x7f27a006a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f27a5d74700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f27a0159ea0 0x7f27a006ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f27a5d74700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a0159a10 con 0x7f27a00631f0 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f27a5d74700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a00ac2e0 con 0x7f27a0158a60 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f27a5d74700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a006f410 con 0x7f27a0159ea0 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f279f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27a0158a60 0x7f27a006a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f279f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27a0158a60 0x7f27a006a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52250/0 (socket says 172.21.15.45:52250) 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.948+0000 7f279f7fe700 1 -- 172.21.15.45:0/3757772134 learned_addr learned my addr 172.21.15.45:0/3757772134 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279effd700 1 --2- 172.21.15.45:0/3757772134 >> v2:172.21.15.144:3300/0 conn(0x7f27a0159ea0 0x7f27a006ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279f7fe700 1 -- 172.21.15.45:0/3757772134 >> v2:172.21.15.144:3300/0 conn(0x7f27a0159ea0 msgr2=0x7f27a006ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279f7fe700 1 --2- 172.21.15.45:0/3757772134 >> v2:172.21.15.144:3300/0 conn(0x7f27a0159ea0 0x7f27a006ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279f7fe700 1 -- 172.21.15.45:0/3757772134 >> v1:172.21.15.144:6789/0 conn(0x7f27a00631f0 legacy=0x7f27a006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279f7fe700 1 -- 172.21.15.45:0/3757772134 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a01593c0 con 0x7f27a0158a60 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279f7fe700 1 --2- 172.21.15.45:0/3757772134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a006a820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f27a006f870 tx=0x7f2790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9907ab2728f8a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279e7fc700 1 -- 172.21.15.45:0/3757772134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000b860 con 0x7f27a0158a60 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279e7fc700 1 -- 172.21.15.45:0/3757772134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000b9c0 con 0x7f27a0158a60 2023-10-06T05:17:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.949+0000 7f279e7fc700 1 -- 172.21.15.45:0/3757772134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000bd60 con 0x7f27a0158a60 2023-10-06T05:17:16.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 -- 172.21.15.45:0/3757772134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 msgr2=0x7f27a006a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 --2- 172.21.15.45:0/3757772134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a006a820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f27a006f870 tx=0x7f2790009250 comp rx=0 tx=0).stop 2023-10-06T05:17:16.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 -- 172.21.15.45:0/3757772134 shutdown_connections 2023-10-06T05:17:16.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 --2- 172.21.15.45:0/3757772134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a006a820 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 --2- 172.21.15.45:0/3757772134 >> v2:172.21.15.144:3300/0 conn(0x7f27a0159ea0 0x7f27a006ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 -- 172.21.15.45:0/3757772134 >> 172.21.15.45:0/3757772134 conn(0x7f27a00cc570 msgr2=0x7f27a00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 -- 172.21.15.45:0/3757772134 shutdown_connections 2023-10-06T05:17:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.950+0000 7f27a5d74700 1 -- 172.21.15.45:0/3757772134 wait complete. 2023-10-06T05:17:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.951+0000 7f27a5d74700 1 Processor -- start 2023-10-06T05:17:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.951+0000 7f27a5d74700 1 -- start start 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f27a5d74700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a00631f0 0x7f27a016b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f27a5d74700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a01543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f27a5d74700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0159ea0 0x7f27a00c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f27a5d74700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00aa2f0 con 0x7f27a0158a60 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f27a5d74700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00c1840 con 0x7f27a0159ea0 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f27a5d74700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00a9ca0 con 0x7f27a00631f0 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a01543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a01543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52266/0 (socket says 172.21.15.45:52266) 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 -- 172.21.15.45:0/1674546779 learned_addr learned my addr 172.21.15.45:0/1674546779 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279ffff700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0159ea0 0x7f27a00c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279f7fe700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a00631f0 0x7f27a016b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 -- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a00631f0 msgr2=0x7f27a016b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a00631f0 0x7f27a016b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 -- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0159ea0 msgr2=0x7f27a00c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0159ea0 0x7f27a00c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.952+0000 7f279effd700 1 -- 172.21.15.45:0/1674546779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27900094f0 con 0x7f27a0158a60 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.953+0000 7f279effd700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a01543e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f278800d860 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.953+0000 7f27977fe700 1 -- 172.21.15.45:0/1674546779 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788009610 con 0x7f27a0158a60 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.953+0000 7f27977fe700 1 -- 172.21.15.45:0/1674546779 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788009770 con 0x7f27a0158a60 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.953+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a00a9f80 con 0x7f27a0158a60 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.953+0000 7f27977fe700 1 -- 172.21.15.45:0/1674546779 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788009b30 con 0x7f27a0158a60 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.953+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a0154920 con 0x7f27a0158a60 2023-10-06T05:17:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.954+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a00a9b00 con 0x7f27a0158a60 2023-10-06T05:17:16.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.954+0000 7f27977fe700 1 -- 172.21.15.45:0/1674546779 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f278801e460 con 0x7f27a0158a60 2023-10-06T05:17:16.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.955+0000 7f27977fe700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2780066f30 0x7f27800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.955+0000 7f27977fe700 1 -- 172.21.15.45:0/1674546779 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f2788084e80 con 0x7f27a0158a60 2023-10-06T05:17:16.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.955+0000 7f279f7fe700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2780066f30 0x7f27800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.955+0000 7f279f7fe700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2780066f30 0x7f27800693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f27a006f870 tx=0x7f2790017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:16.959+0000 7f27977fe700 1 -- 172.21.15.45:0/1674546779 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27a00a9b00 con 0x7f27a0158a60 2023-10-06T05:17:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.159+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f27a00a9b00 con 0x7f27a0158a60 2023-10-06T05:17:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.160+0000 7f27977fe700 1 -- 172.21.15.45:0/1674546779 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f27a00a9b00 con 0x7f27a0158a60 2023-10-06T05:17:17.162 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:17:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2780066f30 msgr2=0x7f27800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2780066f30 0x7f27800693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f27a006f870 tx=0x7f2790017000 comp rx=0 tx=0).stop 2023-10-06T05:17:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 msgr2=0x7f27a01543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a01543e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f278800d860 tx=0x7f2788008040 comp rx=0 tx=0).stop 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 shutdown_connections 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2780066f30 0x7f27800693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0159ea0 0x7f27a00c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a00631f0 0x7f27a016b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 --2- 172.21.15.45:0/1674546779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27a0158a60 0x7f27a01543e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 >> 172.21.15.45:0/1674546779 conn(0x7f27a00cc570 msgr2=0x7f27a00ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 shutdown_connections 2023-10-06T05:17:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.162+0000 7f27a5d74700 1 -- 172.21.15.45:0/1674546779 wait complete. 2023-10-06T05:17:17.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:17.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.310+0000 7fde970de700 1 Processor -- start 2023-10-06T05:17:17.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.310+0000 7fde970de700 1 -- start start 2023-10-06T05:17:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.311+0000 7fde970de700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900b0610 0x7fde900c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.311+0000 7fde970de700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde900c3db0 0x7fde900697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.311+0000 7fde970de700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fde90069d00 0x7fde900c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.311+0000 7fde970de700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900690f0 con 0x7fde900c3db0 2023-10-06T05:17:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.311+0000 7fde970de700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900ceeb0 con 0x7fde90069d00 2023-10-06T05:17:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.311+0000 7fde970de700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900ce6a0 con 0x7fde900b0610 2023-10-06T05:17:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde900c3db0 0x7fde900697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde900c3db0 0x7fde900697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52270/0 (socket says 172.21.15.45:52270) 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 -- 172.21.15.45:0/3277075206 learned_addr learned my addr 172.21.15.45:0/3277075206 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde9567b700 1 --2- 172.21.15.45:0/3277075206 >> v2:172.21.15.144:3300/0 conn(0x7fde90069d00 0x7fde900c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde94e7a700 1 --2- 172.21.15.45:0/3277075206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900b0610 0x7fde900c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 -- 172.21.15.45:0/3277075206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900b0610 msgr2=0x7fde900c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 --2- 172.21.15.45:0/3277075206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900b0610 0x7fde900c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 -- 172.21.15.45:0/3277075206 >> v2:172.21.15.144:3300/0 conn(0x7fde90069d00 msgr2=0x7fde900c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 --2- 172.21.15.45:0/3277075206 >> v2:172.21.15.144:3300/0 conn(0x7fde90069d00 0x7fde900c2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 -- 172.21.15.45:0/3277075206 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90159880 con 0x7fde900c3db0 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8ffff700 1 --2- 172.21.15.45:0/3277075206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde900c3db0 0x7fde900697c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fde9006fce0 tx=0x7fde80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d07399af6935402 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8f7fe700 1 -- 172.21.15.45:0/3277075206 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000b640 con 0x7fde900c3db0 2023-10-06T05:17:17.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8f7fe700 1 -- 172.21.15.45:0/3277075206 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000b7a0 con 0x7fde900c3db0 2023-10-06T05:17:17.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.312+0000 7fde8f7fe700 1 -- 172.21.15.45:0/3277075206 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000bb40 con 0x7fde900c3db0 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 -- 172.21.15.45:0/3277075206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde900c3db0 msgr2=0x7fde900697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 --2- 172.21.15.45:0/3277075206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde900c3db0 0x7fde900697c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fde9006fce0 tx=0x7fde80009250 comp rx=0 tx=0).stop 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 -- 172.21.15.45:0/3277075206 shutdown_connections 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 --2- 172.21.15.45:0/3277075206 >> v2:172.21.15.144:3300/0 conn(0x7fde90069d00 0x7fde900c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 --2- 172.21.15.45:0/3277075206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900b0610 0x7fde900c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 --2- 172.21.15.45:0/3277075206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde900c3db0 0x7fde900697c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 -- 172.21.15.45:0/3277075206 >> 172.21.15.45:0/3277075206 conn(0x7fde90067e00 msgr2=0x7fde900cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 -- 172.21.15.45:0/3277075206 shutdown_connections 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.313+0000 7fde970de700 1 -- 172.21.15.45:0/3277075206 wait complete. 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.314+0000 7fde970de700 1 Processor -- start 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.314+0000 7fde970de700 1 -- start start 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde970de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde90069d00 0x7fde900c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde970de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde900b0610 0x7fde9015d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde970de700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900c3db0 0x7fde90157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde970de700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9015d990 con 0x7fde90069d00 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde970de700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c8270 con 0x7fde900b0610 2023-10-06T05:17:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde970de700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c8550 con 0x7fde900c3db0 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde94e7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde90069d00 0x7fde900c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde94e7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde90069d00 0x7fde900c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52282/0 (socket says 172.21.15.45:52282) 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde94e7a700 1 -- 172.21.15.45:0/662520986 learned_addr learned my addr 172.21.15.45:0/662520986 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.315+0000 7fde8ffff700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde900b0610 0x7fde9015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde94e7a700 1 -- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900c3db0 msgr2=0x7fde90157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde9567b700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900c3db0 0x7fde90157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde94e7a700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900c3db0 0x7fde90157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde94e7a700 1 -- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde900b0610 msgr2=0x7fde9015d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde94e7a700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde900b0610 0x7fde9015d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde94e7a700 1 -- 172.21.15.45:0/662520986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde800094f0 con 0x7fde90069d00 2023-10-06T05:17:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde8ffff700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde900b0610 0x7fde9015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde9567b700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900c3db0 0x7fde90157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde94e7a700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde90069d00 0x7fde900c7c50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fde7800ec80 tx=0x7fde78009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde8cff9700 1 -- 172.21.15.45:0/662520986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800c9d0 con 0x7fde90069d00 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde8cff9700 1 -- 172.21.15.45:0/662520986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7800cb30 con 0x7fde90069d00 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde8cff9700 1 -- 172.21.15.45:0/662520986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78013470 con 0x7fde90069d00 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900c8830 con 0x7fde90069d00 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.316+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde900c53c0 con 0x7fde90069d00 2023-10-06T05:17:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.317+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9006e350 con 0x7fde90069d00 2023-10-06T05:17:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.318+0000 7fde8cff9700 1 -- 172.21.15.45:0/662520986 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde78013860 con 0x7fde90069d00 2023-10-06T05:17:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.318+0000 7fde8cff9700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde70066f30 0x7fde700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.318+0000 7fde8cff9700 1 -- 172.21.15.45:0/662520986 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fde78084fa0 con 0x7fde90069d00 2023-10-06T05:17:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.318+0000 7fde8ffff700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde70066f30 0x7fde700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.319+0000 7fde8ffff700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde70066f30 0x7fde700693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde9006fce0 tx=0x7fde80017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.323+0000 7fde8cff9700 1 -- 172.21.15.45:0/662520986 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde78051ee0 con 0x7fde90069d00 2023-10-06T05:17:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.523+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde900c9b50 con 0x7fde90069d00 2023-10-06T05:17:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.523+0000 7fde8cff9700 1 -- 172.21.15.45:0/662520986 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde78021020 con 0x7fde90069d00 2023-10-06T05:17:17.525 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:17:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde70066f30 msgr2=0x7fde700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde70066f30 0x7fde700693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde9006fce0 tx=0x7fde80017000 comp rx=0 tx=0).stop 2023-10-06T05:17:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde90069d00 msgr2=0x7fde900c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde90069d00 0x7fde900c7c50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fde7800ec80 tx=0x7fde78009120 comp rx=0 tx=0).stop 2023-10-06T05:17:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 shutdown_connections 2023-10-06T05:17:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde70066f30 0x7fde700693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde900c3db0 0x7fde90157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde90069d00 0x7fde900c7c50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 --2- 172.21.15.45:0/662520986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde900b0610 0x7fde9015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 >> 172.21.15.45:0/662520986 conn(0x7fde90067e00 msgr2=0x7fde900cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 shutdown_connections 2023-10-06T05:17:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.526+0000 7fde970de700 1 -- 172.21.15.45:0/662520986 wait complete. 2023-10-06T05:17:17.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.673+0000 7f229dd87700 1 Processor -- start 2023-10-06T05:17:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.674+0000 7f229dd87700 1 -- start start 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.674+0000 7f229dd87700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22980b0610 0x7f22980c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.674+0000 7f229dd87700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f22980697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.674+0000 7f229dd87700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2298069d00 0x7f22980c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.674+0000 7f229dd87700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22980690f0 con 0x7f2298069d00 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f229dd87700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22980ceeb0 con 0x7f22980b0610 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f229dd87700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980ce6a0 con 0x7f22980c3db0 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2298069d00 0x7f22980c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2298069d00 0x7f22980c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52284/0 (socket says 172.21.15.45:52284) 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 -- 172.21.15.45:0/1342031334 learned_addr learned my addr 172.21.15.45:0/1342031334 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2296ffd700 1 --2- 172.21.15.45:0/1342031334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f22980697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f22977fe700 1 --2- 172.21.15.45:0/1342031334 >> v2:172.21.15.144:3300/0 conn(0x7f22980b0610 0x7f22980c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 -- 172.21.15.45:0/1342031334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 msgr2=0x7f22980697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 --2- 172.21.15.45:0/1342031334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f22980697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 -- 172.21.15.45:0/1342031334 >> v2:172.21.15.144:3300/0 conn(0x7f22980b0610 msgr2=0x7f22980c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 --2- 172.21.15.45:0/1342031334 >> v2:172.21.15.144:3300/0 conn(0x7f22980b0610 0x7f22980c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 -- 172.21.15.45:0/1342031334 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298159880 con 0x7f2298069d00 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f2297fff700 1 --2- 172.21.15.45:0/1342031334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2298069d00 0x7f22980c2d30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f229806f080 tx=0x7f2290009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea8d3bf6129ed9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.675+0000 7f22967fc700 1 -- 172.21.15.45:0/1342031334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000e070 con 0x7f2298069d00 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f22967fc700 1 -- 172.21.15.45:0/1342031334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229000b930 con 0x7f2298069d00 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f22967fc700 1 -- 172.21.15.45:0/1342031334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22900092e0 con 0x7f2298069d00 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 -- 172.21.15.45:0/1342031334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2298069d00 msgr2=0x7f22980c2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 --2- 172.21.15.45:0/1342031334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2298069d00 0x7f22980c2d30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f229806f080 tx=0x7f2290009840 comp rx=0 tx=0).stop 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 -- 172.21.15.45:0/1342031334 shutdown_connections 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 --2- 172.21.15.45:0/1342031334 >> v2:172.21.15.144:3300/0 conn(0x7f22980b0610 0x7f22980c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 --2- 172.21.15.45:0/1342031334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2298069d00 0x7f22980c2d30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 --2- 172.21.15.45:0/1342031334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f22980697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 -- 172.21.15.45:0/1342031334 >> 172.21.15.45:0/1342031334 conn(0x7f2298067e00 msgr2=0x7f22980cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 -- 172.21.15.45:0/1342031334 shutdown_connections 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.676+0000 7f229dd87700 1 -- 172.21.15.45:0/1342031334 wait complete. 2023-10-06T05:17:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.677+0000 7f229dd87700 1 Processor -- start 2023-10-06T05:17:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f229dd87700 1 -- start start 2023-10-06T05:17:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f229dd87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298069d00 0x7f22980c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f229dd87700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980b0610 0x7f2298169100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f229dd87700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f2298169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f229dd87700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229815dec0 con 0x7f22980b0610 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f229dd87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22980c7d10 con 0x7f2298069d00 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f229dd87700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980c7ff0 con 0x7f22980c3db0 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f2296ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980b0610 0x7f2298169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f2296ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980b0610 0x7f2298169100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52288/0 (socket says 172.21.15.45:52288) 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.678+0000 7f2296ffd700 1 -- 172.21.15.45:0/2124437548 learned_addr learned my addr 172.21.15.45:0/2124437548 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f2297fff700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f2298169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f22977fe700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298069d00 0x7f22980c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f2296ffd700 1 -- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 msgr2=0x7f2298169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f2296ffd700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f2298169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f2296ffd700 1 -- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298069d00 msgr2=0x7f22980c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f2296ffd700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298069d00 0x7f22980c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f2296ffd700 1 -- 172.21.15.45:0/2124437548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290009ae0 con 0x7f22980b0610 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f2296ffd700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980b0610 0x7f2298169100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f228c00e950 tx=0x7f228c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f227f7fe700 1 -- 172.21.15.45:0/2124437548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c013680 con 0x7f22980b0610 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f227f7fe700 1 -- 172.21.15.45:0/2124437548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c0137e0 con 0x7f22980b0610 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22980c82d0 con 0x7f22980b0610 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f227f7fe700 1 -- 172.21.15.45:0/2124437548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c013ba0 con 0x7f22980b0610 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.679+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22980c79b0 con 0x7f22980b0610 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.680+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229806e350 con 0x7f22980b0610 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.681+0000 7f227f7fe700 1 -- 172.21.15.45:0/2124437548 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f228c0123d0 con 0x7f22980b0610 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.681+0000 7f227f7fe700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.681+0000 7f227f7fe700 1 -- 172.21.15.45:0/2124437548 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f228c0849d0 con 0x7f22980b0610 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.681+0000 7f22977fe700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.682+0000 7f22977fe700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2288009da0 tx=0x7f2288009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.686+0000 7f227f7fe700 1 -- 172.21.15.45:0/2124437548 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f228c051870 con 0x7f22980b0610 2023-10-06T05:17:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.890+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f229804bb40 con 0x7f22980b0610 2023-10-06T05:17:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.890+0000 7f227f7fe700 1 -- 172.21.15.45:0/2124437548 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f228c054e90 con 0x7f22980b0610 2023-10-06T05:17:17.892 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 msgr2=0x7f22840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2288009da0 tx=0x7f2288009120 comp rx=0 tx=0).stop 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980b0610 msgr2=0x7f2298169100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980b0610 0x7f2298169100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f228c00e950 tx=0x7f228c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 shutdown_connections 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980c3db0 0x7f2298169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298069d00 0x7f22980c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 --2- 172.21.15.45:0/2124437548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980b0610 0x7f2298169100 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 >> 172.21.15.45:0/2124437548 conn(0x7f2298067e00 msgr2=0x7f22980ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 shutdown_connections 2023-10-06T05:17:17.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:17.893+0000 7f229dd87700 1 -- 172.21.15.45:0/2124437548 wait complete. 2023-10-06T05:17:17.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.039+0000 7fa079964700 1 Processor -- start 2023-10-06T05:17:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.039+0000 7fa079964700 1 -- start start 2023-10-06T05:17:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa079964700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074159170 0x7fa074159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa079964700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa07415d040 0x7fa074068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa079964700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740ca6d0 con 0x7fa074069b90 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa079964700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07415d550 con 0x7fa07415d040 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa079964700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07415d810 con 0x7fa074159170 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa0727fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa07415d040 0x7fa074068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa0737fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa074069b90 0x7fa0740aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41678/0 (socket says 172.21.15.45:41678) 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa0737fe700 1 -- 172.21.15.45:0/2394718105 learned_addr learned my addr 172.21.15.45:0/2394718105 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa072ffd700 1 --2- 172.21.15.45:0/2394718105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074159170 0x7fa074159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa0727fc700 1 -- 172.21.15.45:0/2394718105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074159170 msgr2=0x7fa074159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa0727fc700 1 --2- 172.21.15.45:0/2394718105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074159170 0x7fa074159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa0727fc700 1 -- 172.21.15.45:0/2394718105 >> v1:172.21.15.45:6789/0 conn(0x7fa074069b90 legacy=0x7fa0740aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa0727fc700 1 -- 172.21.15.45:0/2394718105 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07415daf0 con 0x7fa07415d040 2023-10-06T05:17:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa071ffb700 1 -- 172.21.15.45:0/2394718105 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080575183 0 0) 0x7fa0740ca6d0 con 0x7fa074069b90 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.040+0000 7fa072ffd700 1 --2- 172.21.15.45:0/2394718105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074159170 0x7fa074159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa0727fc700 1 --2- 172.21.15.45:0/2394718105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa074068bc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0098e0 tx=0x7fa05c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ac6b8dcae1cb6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa071ffb700 1 -- 172.21.15.45:0/2394718105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c009e80 con 0x7fa07415d040 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa071ffb700 1 -- 172.21.15.45:0/2394718105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c00b860 con 0x7fa07415d040 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa071ffb700 1 -- 172.21.15.45:0/2394718105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00bbe0 con 0x7fa07415d040 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa079964700 1 -- 172.21.15.45:0/2394718105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 msgr2=0x7fa074068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa079964700 1 --2- 172.21.15.45:0/2394718105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa074068bc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0098e0 tx=0x7fa05c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa079964700 1 -- 172.21.15.45:0/2394718105 shutdown_connections 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa079964700 1 --2- 172.21.15.45:0/2394718105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074159170 0x7fa074159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa079964700 1 --2- 172.21.15.45:0/2394718105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa074068bc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.041+0000 7fa079964700 1 -- 172.21.15.45:0/2394718105 >> 172.21.15.45:0/2394718105 conn(0x7fa07406f7c0 msgr2=0x7fa0740d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.042+0000 7fa079964700 1 -- 172.21.15.45:0/2394718105 shutdown_connections 2023-10-06T05:17:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.042+0000 7fa079964700 1 -- 172.21.15.45:0/2394718105 wait complete. 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.043+0000 7fa079964700 1 Processor -- start 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.043+0000 7fa079964700 1 -- start start 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa079964700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074069b90 0x7fa07414a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa079964700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa074159170 0x7fa0740cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa079964700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa0740cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa079964700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740c8a80 con 0x7fa07415d040 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa079964700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07415c4c0 con 0x7fa074159170 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa079964700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07414aab0 con 0x7fa074069b90 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa0740cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa0740cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52302/0 (socket says 172.21.15.45:52302) 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 -- 172.21.15.45:0/4210623973 learned_addr learned my addr 172.21.15.45:0/4210623973 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa072ffd700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074069b90 0x7fa07414a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0727fc700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa074159170 0x7fa0740cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 -- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074069b90 msgr2=0x7fa07414a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074069b90 0x7fa07414a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 -- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa074159170 msgr2=0x7fa0740cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa074159170 0x7fa0740cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 -- 172.21.15.45:0/4210623973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c009420 con 0x7fa07415d040 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa0737fe700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa0740cf490 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00cc90 tx=0x7fa06c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa06affd700 1 -- 172.21.15.45:0/4210623973 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00a8b0 con 0x7fa07415d040 2023-10-06T05:17:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.044+0000 7fa06affd700 1 -- 172.21.15.45:0/4210623973 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c00aa10 con 0x7fa07415d040 2023-10-06T05:17:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.045+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07415c950 con 0x7fa07415d040 2023-10-06T05:17:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.045+0000 7fa06affd700 1 -- 172.21.15.45:0/4210623973 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00add0 con 0x7fa07415d040 2023-10-06T05:17:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.045+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07415be70 con 0x7fa07415d040 2023-10-06T05:17:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.046+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07415c630 con 0x7fa07415d040 2023-10-06T05:17:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.046+0000 7fa06affd700 1 -- 172.21.15.45:0/4210623973 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa06c007300 con 0x7fa07415d040 2023-10-06T05:17:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.046+0000 7fa06affd700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa054066f30 0x7fa0540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.047+0000 7fa06affd700 1 -- 172.21.15.45:0/4210623973 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fa06c084b80 con 0x7fa07415d040 2023-10-06T05:17:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.047+0000 7fa072ffd700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa054066f30 0x7fa0540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.047+0000 7fa072ffd700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa054066f30 0x7fa0540693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa074156770 tx=0x7fa064008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.051+0000 7fa06affd700 1 -- 172.21.15.45:0/4210623973 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa06c051b40 con 0x7fa07415d040 2023-10-06T05:17:18.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.301+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa074063b50 con 0x7fa07415d040 2023-10-06T05:17:18.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.302+0000 7fa06affd700 1 -- 172.21.15.45:0/4210623973 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fa06c055160 con 0x7fa07415d040 2023-10-06T05:17:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:18.304 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:17:17.873982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:17:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa054066f30 msgr2=0x7fa0540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa054066f30 0x7fa0540693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa074156770 tx=0x7fa064008040 comp rx=0 tx=0).stop 2023-10-06T05:17:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 msgr2=0x7fa0740cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa0740cf490 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00cc90 tx=0x7fa06c001a80 comp rx=0 tx=0).stop 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 shutdown_connections 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa054066f30 0x7fa0540693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa07415d040 0x7fa0740cf490 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa074069b90 0x7fa07414a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 --2- 172.21.15.45:0/4210623973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa074159170 0x7fa0740cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 >> 172.21.15.45:0/4210623973 conn(0x7fa07406f7c0 msgr2=0x7fa0740cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 shutdown_connections 2023-10-06T05:17:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.304+0000 7fa079964700 1 -- 172.21.15.45:0/4210623973 wait complete. 2023-10-06T05:17:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2023-10-06T05:17:18.319 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.449+0000 7f5bca76a700 1 Processor -- start 2023-10-06T05:17:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.449+0000 7f5bca76a700 1 -- start start 2023-10-06T05:17:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bca76a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bca76a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bc41595d0 0x7f5bc406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bca76a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc40c7d00 con 0x7f5bc406b5c0 2023-10-06T05:17:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bca76a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc40c7780 con 0x7f5bc41595d0 2023-10-06T05:17:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bca76a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40c86d0 con 0x7f5bc4159200 2023-10-06T05:17:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bc41595d0 0x7f5bc406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc8d07700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5bc406b5c0 0x7f5bc40ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41680/0 (socket says 172.21.15.45:41680) 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc8d07700 1 -- 172.21.15.45:0/712098290 learned_addr learned my addr 172.21.15.45:0/712098290 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc3fff700 1 --2- 172.21.15.45:0/712098290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc37fe700 1 -- 172.21.15.45:0/712098290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 msgr2=0x7f5bc40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc37fe700 1 --2- 172.21.15.45:0/712098290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc40c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc37fe700 1 -- 172.21.15.45:0/712098290 >> v1:172.21.15.45:6789/0 conn(0x7f5bc406b5c0 legacy=0x7f5bc40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc37fe700 1 -- 172.21.15.45:0/712098290 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc406bb40 con 0x7f5bc41595d0 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.450+0000 7f5bc2ffd700 1 -- 172.21.15.45:0/712098290 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210352258 0 0) 0x7f5bc40c7d00 con 0x7f5bc406b5c0 2023-10-06T05:17:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bc37fe700 1 --2- 172.21.15.45:0/712098290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc41595d0 0x7f5bc406b080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0098e0 tx=0x7f5bac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f5edbaddc1356f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bc2ffd700 1 -- 172.21.15.45:0/712098290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00b640 con 0x7f5bc41595d0 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bc2ffd700 1 -- 172.21.15.45:0/712098290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac00b7a0 con 0x7f5bc41595d0 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bc2ffd700 1 -- 172.21.15.45:0/712098290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00bb40 con 0x7f5bc41595d0 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 -- 172.21.15.45:0/712098290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc41595d0 msgr2=0x7f5bc406b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 --2- 172.21.15.45:0/712098290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc41595d0 0x7f5bc406b080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0098e0 tx=0x7f5bac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 -- 172.21.15.45:0/712098290 shutdown_connections 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 --2- 172.21.15.45:0/712098290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 --2- 172.21.15.45:0/712098290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc41595d0 0x7f5bc406b080 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 -- 172.21.15.45:0/712098290 >> 172.21.15.45:0/712098290 conn(0x7f5bc406f090 msgr2=0x7f5bc406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 -- 172.21.15.45:0/712098290 shutdown_connections 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.451+0000 7f5bca76a700 1 -- 172.21.15.45:0/712098290 wait complete. 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.453+0000 7f5bca76a700 1 Processor -- start 2023-10-06T05:17:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.453+0000 7f5bca76a700 1 -- start start 2023-10-06T05:17:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bca76a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc406b5c0 0x7f5bc40cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bca76a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc41490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bca76a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc41595d0 0x7f5bc41495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bca76a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc4158860 con 0x7f5bc406b5c0 2023-10-06T05:17:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bca76a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40cc020 con 0x7f5bc41595d0 2023-10-06T05:17:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bca76a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40cc2a0 con 0x7f5bc4159200 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc406b5c0 0x7f5bc40cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc406b5c0 0x7f5bc40cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52326/0 (socket says 172.21.15.45:52326) 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 -- 172.21.15.45:0/1107891920 learned_addr learned my addr 172.21.15.45:0/1107891920 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc37fe700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc41490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc8d07700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc41595d0 0x7f5bc41495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 -- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 msgr2=0x7f5bc41490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc41490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 -- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc41595d0 msgr2=0x7f5bc41495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc41595d0 0x7f5bc41495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 -- 172.21.15.45:0/1107891920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac009420 con 0x7f5bc406b5c0 2023-10-06T05:17:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc8d07700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc41595d0 0x7f5bc41495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bc3fff700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc406b5c0 0x7f5bc40cbae0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400ec80 tx=0x7f5bb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bbbfff700 1 -- 172.21.15.45:0/1107891920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400c850 con 0x7f5bc406b5c0 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bbbfff700 1 -- 172.21.15.45:0/1107891920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb400c9d0 con 0x7f5bc406b5c0 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc40cc580 con 0x7f5bc406b5c0 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.454+0000 7f5bbbfff700 1 -- 172.21.15.45:0/1107891920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400cd90 con 0x7f5bc406b5c0 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.455+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc40ccb40 con 0x7f5bc406b5c0 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.455+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc406e2f0 con 0x7f5bc406b5c0 2023-10-06T05:17:18.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.456+0000 7f5bbbfff700 1 -- 172.21.15.45:0/1107891920 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bb4013680 con 0x7f5bc406b5c0 2023-10-06T05:17:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.456+0000 7f5bbbfff700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ba4066f30 0x7f5ba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.456+0000 7f5bbbfff700 1 -- 172.21.15.45:0/1107891920 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f5bb4084c00 con 0x7f5bc406b5c0 2023-10-06T05:17:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.457+0000 7f5bc37fe700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ba4066f30 0x7f5ba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.457+0000 7f5bc37fe700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ba4066f30 0x7f5ba40693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0093f0 tx=0x7f5bac017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.461+0000 7f5bbbfff700 1 -- 172.21.15.45:0/1107891920 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bb4051aa0 con 0x7f5bc406b5c0 2023-10-06T05:17:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.661+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bc404bb40 con 0x7f5bc406b5c0 2023-10-06T05:17:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.662+0000 7f5bbbfff700 1 -- 172.21.15.45:0/1107891920 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17391 (secure 0 0 0) 0x7f5bb4084020 con 0x7f5bc406b5c0 2023-10-06T05:17:18.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:18.664 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:16:48.823654+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:08.040887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:08.594370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/2918652845":"2023-10-07T05:16:26.150462+0000","172.21.15.45:6832/2918652845":"2023-10-07T05:16:26.150462+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ba4066f30 msgr2=0x7f5ba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ba4066f30 0x7f5ba40693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0093f0 tx=0x7f5bac017000 comp rx=0 tx=0).stop 2023-10-06T05:17:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc406b5c0 msgr2=0x7f5bc40cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc406b5c0 0x7f5bc40cbae0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400ec80 tx=0x7f5bb40091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 shutdown_connections 2023-10-06T05:17:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ba4066f30 0x7f5ba40693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc41595d0 0x7f5bc41495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bc406b5c0 0x7f5bc40cbae0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 --2- 172.21.15.45:0/1107891920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc4159200 0x7f5bc41490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 >> 172.21.15.45:0/1107891920 conn(0x7f5bc406f090 msgr2=0x7f5bc415ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 shutdown_connections 2023-10-06T05:17:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.664+0000 7f5bca76a700 1 -- 172.21.15.45:0/1107891920 wait complete. 2023-10-06T05:17:18.679 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:17:18.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:18.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.810+0000 7f6733a52700 1 Processor -- start 2023-10-06T05:17:18.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.810+0000 7f6733a52700 1 -- start start 2023-10-06T05:17:18.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.811+0000 7f6733a52700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.811+0000 7f6733a52700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f672c1560a0 0x7f672c0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.811+0000 7f6733a52700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f672c0ccdc0 0x7f672c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.811+0000 7f6733a52700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c0cd740 con 0x7f672c0ccdc0 2023-10-06T05:17:18.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.811+0000 7f6733a52700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c0ae8d0 con 0x7f672c1560a0 2023-10-06T05:17:18.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.811+0000 7f6733a52700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0aebb0 con 0x7f672c155cd0 2023-10-06T05:17:18.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.811+0000 7f6731fef700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f672c0ccdc0 0x7f672c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f672c0ccdc0 0x7f672c0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52338/0 (socket says 172.21.15.45:52338) 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 -- 172.21.15.45:0/3105140600 learned_addr learned my addr 172.21.15.45:0/3105140600 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f67317ee700 1 --2- 172.21.15.45:0/3105140600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6730fed700 1 --2- 172.21.15.45:0/3105140600 >> v2:172.21.15.144:3300/0 conn(0x7f672c1560a0 0x7f672c0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 -- 172.21.15.45:0/3105140600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 msgr2=0x7f672c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 --2- 172.21.15.45:0/3105140600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 -- 172.21.15.45:0/3105140600 >> v2:172.21.15.144:3300/0 conn(0x7f672c1560a0 msgr2=0x7f672c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 --2- 172.21.15.45:0/3105140600 >> v2:172.21.15.144:3300/0 conn(0x7f672c1560a0 0x7f672c0adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 -- 172.21.15.45:0/3105140600 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c0aee90 con 0x7f672c0ccdc0 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6731fef700 1 --2- 172.21.15.45:0/3105140600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c0ccdc0 0x7f672c0cd200 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f672c06ac20 tx=0x7f6724009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c1f259e115767a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6723fff700 1 -- 172.21.15.45:0/3105140600 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400e070 con 0x7f672c0ccdc0 2023-10-06T05:17:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6723fff700 1 -- 172.21.15.45:0/3105140600 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672400b970 con 0x7f672c0ccdc0 2023-10-06T05:17:18.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.812+0000 7f6723fff700 1 -- 172.21.15.45:0/3105140600 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67240092e0 con 0x7f672c0ccdc0 2023-10-06T05:17:18.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 -- 172.21.15.45:0/3105140600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c0ccdc0 msgr2=0x7f672c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 --2- 172.21.15.45:0/3105140600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c0ccdc0 0x7f672c0cd200 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f672c06ac20 tx=0x7f6724009840 comp rx=0 tx=0).stop 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 -- 172.21.15.45:0/3105140600 shutdown_connections 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 --2- 172.21.15.45:0/3105140600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c0ccdc0 0x7f672c0cd200 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 --2- 172.21.15.45:0/3105140600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 --2- 172.21.15.45:0/3105140600 >> v2:172.21.15.144:3300/0 conn(0x7f672c1560a0 0x7f672c0adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 -- 172.21.15.45:0/3105140600 >> 172.21.15.45:0/3105140600 conn(0x7f672c06f090 msgr2=0x7f672c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 -- 172.21.15.45:0/3105140600 shutdown_connections 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.813+0000 7f6733a52700 1 -- 172.21.15.45:0/3105140600 wait complete. 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.815+0000 7f6733a52700 1 Processor -- start 2023-10-06T05:17:18.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.815+0000 7f6733a52700 1 -- start start 2023-10-06T05:17:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6733a52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f672c0ccdc0 0x7f672c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6733a52700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c16a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6733a52700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c1560a0 0x7f672c146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6733a52700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c16ac90 con 0x7f672c1560a0 2023-10-06T05:17:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6733a52700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1574e0 con 0x7f672c0ccdc0 2023-10-06T05:17:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6733a52700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1584b0 con 0x7f672c155cd0 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6731fef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c1560a0 0x7f672c146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6731fef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c1560a0 0x7f672c146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52352/0 (socket says 172.21.15.45:52352) 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6731fef700 1 -- 172.21.15.45:0/3079907390 learned_addr learned my addr 172.21.15.45:0/3079907390 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f67317ee700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f672c0ccdc0 0x7f672c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6730fed700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c16a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6731fef700 1 -- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 msgr2=0x7f672c16a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.816+0000 7f6731fef700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c16a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f6731fef700 1 -- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f672c0ccdc0 msgr2=0x7f672c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f6731fef700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f672c0ccdc0 0x7f672c156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f6731fef700 1 -- 172.21.15.45:0/3079907390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724009ae0 con 0x7f672c1560a0 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f6730fed700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c16a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f6731fef700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c1560a0 0x7f672c146da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f672400b610 tx=0x7f6724018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f67217fa700 1 -- 172.21.15.45:0/3079907390 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724008b00 con 0x7f672c1560a0 2023-10-06T05:17:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f67217fa700 1 -- 172.21.15.45:0/3079907390 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6724008c60 con 0x7f672c1560a0 2023-10-06T05:17:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f6733a52700 1 -- 172.21.15.45:0/3079907390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c1472e0 con 0x7f672c1560a0 2023-10-06T05:17:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f67217fa700 1 -- 172.21.15.45:0/3079907390 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724026460 con 0x7f672c1560a0 2023-10-06T05:17:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.817+0000 7f6733a52700 1 -- 172.21.15.45:0/3079907390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c156f20 con 0x7f672c1560a0 2023-10-06T05:17:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.818+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67040052b0 con 0x7f672c1560a0 2023-10-06T05:17:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.818+0000 7f67217fa700 1 -- 172.21.15.45:0/3079907390 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6724003040 con 0x7f672c1560a0 2023-10-06T05:17:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.819+0000 7f67217fa700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6710066f30 0x7f67100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.819+0000 7f67217fa700 1 -- 172.21.15.45:0/3079907390 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f67240867b0 con 0x7f672c1560a0 2023-10-06T05:17:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.819+0000 7f67317ee700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6710066f30 0x7f67100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.820+0000 7f67317ee700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6710066f30 0x7f67100693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6728009db0 tx=0x7f67280091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:18.823+0000 7f67217fa700 1 -- 172.21.15.45:0/3079907390 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67240537a0 con 0x7f672c1560a0 2023-10-06T05:17:19.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.072+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6704005560 con 0x7f672c1560a0 2023-10-06T05:17:19.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.072+0000 7f67217fa700 1 -- 172.21.15.45:0/3079907390 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6724056dc0 con 0x7f672c1560a0 2023-10-06T05:17:19.075 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:19.075 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:17:17.873982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:17:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6710066f30 msgr2=0x7f67100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6710066f30 0x7f67100693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6728009db0 tx=0x7f67280091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c1560a0 msgr2=0x7f672c146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c1560a0 0x7f672c146da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f672400b610 tx=0x7f6724018000 comp rx=0 tx=0).stop 2023-10-06T05:17:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 shutdown_connections 2023-10-06T05:17:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6710066f30 0x7f67100693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f672c1560a0 0x7f672c146da0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f672c0ccdc0 0x7f672c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 --2- 172.21.15.45:0/3079907390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f672c155cd0 0x7f672c16a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 >> 172.21.15.45:0/3079907390 conn(0x7f672c06f090 msgr2=0x7f672c0ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 shutdown_connections 2023-10-06T05:17:19.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.075+0000 7f6716ffd700 1 -- 172.21.15.45:0/3079907390 wait complete. 2023-10-06T05:17:19.078 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2023-10-06T05:17:19.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:19.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.220+0000 7ff346b07700 1 Processor -- start 2023-10-06T05:17:19.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.220+0000 7ff346b07700 1 -- start start 2023-10-06T05:17:19.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.221+0000 7ff346b07700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff340153620 0x7ff34015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.221+0000 7ff346b07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3401539f0 0x7ff34015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff346b07700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3400ce690 con 0x7ff3400cc8b0 2023-10-06T05:17:19.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff346b07700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34015a7a0 con 0x7ff340153620 2023-10-06T05:17:19.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff346b07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400ce3f0 con 0x7ff3401539f0 2023-10-06T05:17:19.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff3450a4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff3400cc8b0 0x7ff3400ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41696/0 (socket says 172.21.15.45:41696) 2023-10-06T05:17:19.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff3450a4700 1 -- 172.21.15.45:0/4052092437 learned_addr learned my addr 172.21.15.45:0/4052092437 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:19.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33ffff700 1 --2- 172.21.15.45:0/4052092437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3401539f0 0x7ff34015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff3448a3700 1 --2- 172.21.15.45:0/4052092437 >> v2:172.21.15.144:3300/0 conn(0x7ff340153620 0x7ff34015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33f7fe700 1 -- 172.21.15.45:0/4052092437 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496878293 0 0) 0x7ff3400ce690 con 0x7ff3400cc8b0 2023-10-06T05:17:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33f7fe700 1 -- 172.21.15.45:0/4052092437 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff32c003570 con 0x7ff3400cc8b0 2023-10-06T05:17:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33ffff700 1 -- 172.21.15.45:0/4052092437 >> v2:172.21.15.144:3300/0 conn(0x7ff340153620 msgr2=0x7ff34015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33f7fe700 1 -- 172.21.15.45:0/4052092437 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619693566 0 0) 0x7ff32c003570 con 0x7ff3400cc8b0 2023-10-06T05:17:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33ffff700 1 --2- 172.21.15.45:0/4052092437 >> v2:172.21.15.144:3300/0 conn(0x7ff340153620 0x7ff34015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33ffff700 1 -- 172.21.15.45:0/4052092437 >> v1:172.21.15.45:6789/0 conn(0x7ff3400cc8b0 legacy=0x7ff3400ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff33ffff700 1 -- 172.21.15.45:0/4052092437 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34015d510 con 0x7ff3401539f0 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.222+0000 7ff3448a3700 1 --2- 172.21.15.45:0/4052092437 >> v2:172.21.15.144:3300/0 conn(0x7ff340153620 0x7ff34015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff33ffff700 1 --2- 172.21.15.45:0/4052092437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3401539f0 0x7ff34015cdc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3280098e0 tx=0x7ff3280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54035f2c82aea523 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff33f7fe700 1 -- 172.21.15.45:0/4052092437 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380017f0 con 0x7ff3401539f0 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff33f7fe700 1 -- 172.21.15.45:0/4052092437 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff328011420 con 0x7ff3401539f0 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff33f7fe700 1 -- 172.21.15.45:0/4052092437 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280117c0 con 0x7ff3401539f0 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 -- 172.21.15.45:0/4052092437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3401539f0 msgr2=0x7ff34015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 --2- 172.21.15.45:0/4052092437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3401539f0 0x7ff34015cdc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3280098e0 tx=0x7ff3280091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 -- 172.21.15.45:0/4052092437 shutdown_connections 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 --2- 172.21.15.45:0/4052092437 >> v2:172.21.15.144:3300/0 conn(0x7ff340153620 0x7ff34015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 --2- 172.21.15.45:0/4052092437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3401539f0 0x7ff34015cdc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 -- 172.21.15.45:0/4052092437 >> 172.21.15.45:0/4052092437 conn(0x7ff34006ed50 msgr2=0x7ff34006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 -- 172.21.15.45:0/4052092437 shutdown_connections 2023-10-06T05:17:19.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.223+0000 7ff346b07700 1 -- 172.21.15.45:0/4052092437 wait complete. 2023-10-06T05:17:19.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.225+0000 7ff346b07700 1 Processor -- start 2023-10-06T05:17:19.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.225+0000 7ff346b07700 1 -- start start 2023-10-06T05:17:19.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff346b07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3400cc8b0 0x7ff340146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff346b07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340153620 0x7ff340068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff346b07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff340150b10 0x7ff3400690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff346b07700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340147220 con 0x7ff340153620 2023-10-06T05:17:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff346b07700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34014fa70 con 0x7ff340150b10 2023-10-06T05:17:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff346b07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34014fd50 con 0x7ff3400cc8b0 2023-10-06T05:17:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff33ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340153620 0x7ff340068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff33ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340153620 0x7ff340068b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52364/0 (socket says 172.21.15.45:52364) 2023-10-06T05:17:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.226+0000 7ff33ffff700 1 -- 172.21.15.45:0/2723364467 learned_addr learned my addr 172.21.15.45:0/2723364467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff3450a4700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff340150b10 0x7ff3400690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff3448a3700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3400cc8b0 0x7ff340146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33ffff700 1 -- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3400cc8b0 msgr2=0x7ff340146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33ffff700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3400cc8b0 0x7ff340146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33ffff700 1 -- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff340150b10 msgr2=0x7ff3400690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33ffff700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff340150b10 0x7ff3400690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33ffff700 1 -- 172.21.15.45:0/2723364467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330009420 con 0x7ff340153620 2023-10-06T05:17:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff3450a4700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff340150b10 0x7ff3400690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33ffff700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340153620 0x7ff340068b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff3280030c0 tx=0x7ff328016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33cff9700 1 -- 172.21.15.45:0/2723364467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32801cbe0 con 0x7ff340153620 2023-10-06T05:17:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33cff9700 1 -- 172.21.15.45:0/2723364467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32801cd40 con 0x7ff340153620 2023-10-06T05:17:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff328009420 con 0x7ff340153620 2023-10-06T05:17:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff33cff9700 1 -- 172.21.15.45:0/2723364467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32801a6a0 con 0x7ff340153620 2023-10-06T05:17:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.227+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff340150310 con 0x7ff340153620 2023-10-06T05:17:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.228+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3400699e0 con 0x7ff340153620 2023-10-06T05:17:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.229+0000 7ff33cff9700 1 -- 172.21.15.45:0/2723364467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff32801aad0 con 0x7ff340153620 2023-10-06T05:17:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.229+0000 7ff33cff9700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff320066f30 0x7ff3200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.229+0000 7ff33cff9700 1 -- 172.21.15.45:0/2723364467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7ff328085de0 con 0x7ff340153620 2023-10-06T05:17:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.229+0000 7ff3448a3700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff320066f30 0x7ff3200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.230+0000 7ff3448a3700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff320066f30 0x7ff3200693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff330009f40 tx=0x7ff3300091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.234+0000 7ff33cff9700 1 -- 172.21.15.45:0/2723364467 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff328052da0 con 0x7ff340153620 2023-10-06T05:17:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.484+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff34004bb40 con 0x7ff340153620 2023-10-06T05:17:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.485+0000 7ff33cff9700 1 -- 172.21.15.45:0/2723364467 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4605 (secure 0 0 0) 0x7ff328029070 con 0x7ff340153620 2023-10-06T05:17:19.487 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:19.487 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:16:09.624552+0000","modified":"2023-10-06T05:17:17.873982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.45:6834/1705605429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1705605429},{"type":"v1","addr":"172.21.15.45:6834","nonce":1705605429}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff320066f30 msgr2=0x7ff3200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff320066f30 0x7ff3200693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff330009f40 tx=0x7ff3300091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340153620 msgr2=0x7ff340068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340153620 0x7ff340068b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff3280030c0 tx=0x7ff328016000 comp rx=0 tx=0).stop 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 shutdown_connections 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff320066f30 0x7ff3200693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff340150b10 0x7ff3400690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3400cc8b0 0x7ff340146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 --2- 172.21.15.45:0/2723364467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff340153620 0x7ff340068b90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 >> 172.21.15.45:0/2723364467 conn(0x7ff34006ed50 msgr2=0x7ff3400c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 shutdown_connections 2023-10-06T05:17:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.487+0000 7ff346b07700 1 -- 172.21.15.45:0/2723364467 wait complete. 2023-10-06T05:17:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2023-10-06T05:17:19.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:19.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.633+0000 7efea7d9d700 1 Processor -- start 2023-10-06T05:17:19.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.633+0000 7efea7d9d700 1 -- start start 2023-10-06T05:17:19.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea7d9d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea0156ce0 0x7efea01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea7d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efea006bad0 0x7efea015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea7d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efea00afbe0 0x7efea00ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea7d9d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea015b150 con 0x7efea00afbe0 2023-10-06T05:17:19.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea7d9d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea006bf40 con 0x7efea006bad0 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea7d9d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea015b380 con 0x7efea0156ce0 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efea00afbe0 0x7efea00ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efea00afbe0 0x7efea00ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52374/0 (socket says 172.21.15.45:52374) 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 -- 172.21.15.45:0/4172439433 learned_addr learned my addr 172.21.15.45:0/4172439433 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea5b39700 1 --2- 172.21.15.45:0/4172439433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea0156ce0 0x7efea01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea5338700 1 --2- 172.21.15.45:0/4172439433 >> v2:172.21.15.144:3300/0 conn(0x7efea006bad0 0x7efea015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 -- 172.21.15.45:0/4172439433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea0156ce0 msgr2=0x7efea01570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 --2- 172.21.15.45:0/4172439433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea0156ce0 0x7efea01570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 -- 172.21.15.45:0/4172439433 >> v2:172.21.15.144:3300/0 conn(0x7efea006bad0 msgr2=0x7efea015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 --2- 172.21.15.45:0/4172439433 >> v2:172.21.15.144:3300/0 conn(0x7efea006bad0 0x7efea015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.634+0000 7efea633a700 1 -- 172.21.15.45:0/4172439433 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea015b660 con 0x7efea00afbe0 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea633a700 1 --2- 172.21.15.45:0/4172439433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea00ccce0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efea006ac20 tx=0x7efe98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ead7e8045e40c6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea4b37700 1 -- 172.21.15.45:0/4172439433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800e070 con 0x7efea00afbe0 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea4b37700 1 -- 172.21.15.45:0/4172439433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9800b970 con 0x7efea00afbe0 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea4b37700 1 -- 172.21.15.45:0/4172439433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980092e0 con 0x7efea00afbe0 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 -- 172.21.15.45:0/4172439433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 msgr2=0x7efea00ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 --2- 172.21.15.45:0/4172439433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea00ccce0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efea006ac20 tx=0x7efe98009840 comp rx=0 tx=0).stop 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 -- 172.21.15.45:0/4172439433 shutdown_connections 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 --2- 172.21.15.45:0/4172439433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea00ccce0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 --2- 172.21.15.45:0/4172439433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea0156ce0 0x7efea01570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 --2- 172.21.15.45:0/4172439433 >> v2:172.21.15.144:3300/0 conn(0x7efea006bad0 0x7efea015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 -- 172.21.15.45:0/4172439433 >> 172.21.15.45:0/4172439433 conn(0x7efea006f090 msgr2=0x7efea006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 -- 172.21.15.45:0/4172439433 shutdown_connections 2023-10-06T05:17:19.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.635+0000 7efea7d9d700 1 -- 172.21.15.45:0/4172439433 wait complete. 2023-10-06T05:17:19.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.637+0000 7efea7d9d700 1 Processor -- start 2023-10-06T05:17:19.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.637+0000 7efea7d9d700 1 -- start start 2023-10-06T05:17:19.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.637+0000 7efea7d9d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea006bad0 0x7efea00c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.637+0000 7efea7d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea0153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.637+0000 7efea7d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efea0156ce0 0x7efea0154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea7d9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c6920 con 0x7efea00afbe0 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea7d9d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c6c00 con 0x7efea0156ce0 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea7d9d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c3150 con 0x7efea006bad0 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea0153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea0153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52382/0 (socket says 172.21.15.45:52382) 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 -- 172.21.15.45:0/142558942 learned_addr learned my addr 172.21.15.45:0/142558942 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea633a700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efea0156ce0 0x7efea0154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5b39700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea006bad0 0x7efea00c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 -- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea006bad0 msgr2=0x7efea00c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea006bad0 0x7efea00c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 -- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efea0156ce0 msgr2=0x7efea0154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efea0156ce0 0x7efea0154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 -- 172.21.15.45:0/142558942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe98009ae0 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea633a700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efea0156ce0 0x7efea0154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea5338700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea0153380 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efe9000ec80 tx=0x7efe900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efe95ffb700 1 -- 172.21.15.45:0/142558942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9000c9d0 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efe95ffb700 1 -- 172.21.15.45:0/142558942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9000cb30 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efe95ffb700 1 -- 172.21.15.45:0/142558942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90013470 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.638+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea00c3430 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.639+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea00c2d70 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.639+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea006e2f0 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.640+0000 7efe95ffb700 1 -- 172.21.15.45:0/142558942 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe90013820 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.640+0000 7efe95ffb700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe84066f00 0x7efe840693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.641+0000 7efe95ffb700 1 -- 172.21.15.45:0/142558942 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7efe90084bf0 con 0x7efea00afbe0 2023-10-06T05:17:19.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.641+0000 7efea5b39700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe84066f00 0x7efe840693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.641+0000 7efea5b39700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe84066f00 0x7efe840693b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009ca0 tx=0x7efe9c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.645+0000 7efe95ffb700 1 -- 172.21.15.45:0/142558942 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe90051b30 con 0x7efea00afbe0 2023-10-06T05:17:19.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.846+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efea015b940 con 0x7efea00afbe0 2023-10-06T05:17:19.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.847+0000 7efe95ffb700 1 -- 172.21.15.45:0/142558942 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17391 (secure 0 0 0) 0x7efe90055150 con 0x7efea00afbe0 2023-10-06T05:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:19.849 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:16:48.823654+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:08.040887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T05:16:08.594370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/2918652845":"2023-10-07T05:16:26.150462+0000","172.21.15.45:6832/2918652845":"2023-10-07T05:16:26.150462+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe84066f00 msgr2=0x7efe840693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe84066f00 0x7efe840693b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009ca0 tx=0x7efe9c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 msgr2=0x7efea0153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea0153380 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efe9000ec80 tx=0x7efe900091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 shutdown_connections 2023-10-06T05:17:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe84066f00 0x7efe840693b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efea0156ce0 0x7efea0154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efea006bad0 0x7efea00c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 --2- 172.21.15.45:0/142558942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efea00afbe0 0x7efea0153380 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 >> 172.21.15.45:0/142558942 conn(0x7efea006f090 msgr2=0x7efea00b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 shutdown_connections 2023-10-06T05:17:19.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.849+0000 7efea7d9d700 1 -- 172.21.15.45:0/142558942 wait complete. 2023-10-06T05:17:19.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.997+0000 7faa4ddac700 1 Processor -- start 2023-10-06T05:17:19.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.997+0000 7faa4ddac700 1 -- start start 2023-10-06T05:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa4ddac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa48155440 0x7faa48155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa4ddac700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa4806a890 0x7faa4806acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa4ddac700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4806b230 con 0x7faa4806e6e0 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa4ddac700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa48068cd0 con 0x7faa48155440 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa4ddac700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa480c8a30 con 0x7faa4806a890 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa477fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa48155440 0x7faa48155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa46ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faa4806e6e0 0x7faa480c8320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41706/0 (socket says 172.21.15.45:41706) 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa477fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa48155440 0x7faa48155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52398/0 (socket says 172.21.15.45:52398) 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa46ffd700 1 -- 172.21.15.45:0/414652022 learned_addr learned my addr 172.21.15.45:0/414652022 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa47fff700 1 --2- 172.21.15.45:0/414652022 >> v2:172.21.15.144:3300/0 conn(0x7faa4806a890 0x7faa4806acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa477fe700 1 -- 172.21.15.45:0/414652022 >> v2:172.21.15.144:3300/0 conn(0x7faa4806a890 msgr2=0x7faa4806acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa477fe700 1 --2- 172.21.15.45:0/414652022 >> v2:172.21.15.144:3300/0 conn(0x7faa4806a890 0x7faa4806acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa477fe700 1 -- 172.21.15.45:0/414652022 >> v1:172.21.15.45:6789/0 conn(0x7faa4806e6e0 legacy=0x7faa480c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.998+0000 7faa477fe700 1 -- 172.21.15.45:0/414652022 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480c8080 con 0x7faa48155440 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa477fe700 1 --2- 172.21.15.45:0/414652022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48155440 0x7faa48155810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faa48156bd0 tx=0x7faa3800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2179c7107e11a1e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa467fc700 1 -- 172.21.15.45:0/414652022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800db70 con 0x7faa48155440 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa467fc700 1 -- 172.21.15.45:0/414652022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3800dcd0 con 0x7faa48155440 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa467fc700 1 -- 172.21.15.45:0/414652022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380196b0 con 0x7faa48155440 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa4ddac700 1 -- 172.21.15.45:0/414652022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48155440 msgr2=0x7faa48155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa4ddac700 1 --2- 172.21.15.45:0/414652022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48155440 0x7faa48155810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faa48156bd0 tx=0x7faa3800b430 comp rx=0 tx=0).stop 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa4ddac700 1 -- 172.21.15.45:0/414652022 shutdown_connections 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa4ddac700 1 --2- 172.21.15.45:0/414652022 >> v2:172.21.15.144:3300/0 conn(0x7faa4806a890 0x7faa4806acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa4ddac700 1 --2- 172.21.15.45:0/414652022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48155440 0x7faa48155810 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:19.999+0000 7faa4ddac700 1 -- 172.21.15.45:0/414652022 >> 172.21.15.45:0/414652022 conn(0x7faa480cc5a0 msgr2=0x7faa480cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.000+0000 7faa4ddac700 1 -- 172.21.15.45:0/414652022 shutdown_connections 2023-10-06T05:17:20.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.000+0000 7faa4ddac700 1 -- 172.21.15.45:0/414652022 wait complete. 2023-10-06T05:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.001+0000 7faa4ddac700 1 Processor -- start 2023-10-06T05:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.001+0000 7faa4ddac700 1 -- start start 2023-10-06T05:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa4ddac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4806a890 0x7faa4816da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa4ddac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa4806e6e0 0x7faa480afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa4ddac700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa48155440 0x7faa48157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa4ddac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480a9f60 con 0x7faa4806e6e0 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa4ddac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480b03f0 con 0x7faa4806a890 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa4ddac700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa480c35b0 con 0x7faa48155440 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa4806e6e0 0x7faa480afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa4806e6e0 0x7faa480afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52408/0 (socket says 172.21.15.45:52408) 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 -- 172.21.15.45:0/2977766468 learned_addr learned my addr 172.21.15.45:0/2977766468 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa477fe700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4806a890 0x7faa4816da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa47fff700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa48155440 0x7faa48157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 -- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa48155440 msgr2=0x7faa48157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa48155440 0x7faa48157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 -- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4806a890 msgr2=0x7faa4816da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4806a890 0x7faa4816da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 -- 172.21.15.45:0/2977766468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3800b6d0 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa47fff700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa48155440 0x7faa48157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.002+0000 7faa46ffd700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa4806e6e0 0x7faa480afeb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faa3000c7c0 tx=0x7faa30007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.003+0000 7faa3f7fe700 1 -- 172.21.15.45:0/2977766468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30005b30 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.003+0000 7faa3f7fe700 1 -- 172.21.15.45:0/2977766468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30005cb0 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.003+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa480b0670 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.003+0000 7faa3f7fe700 1 -- 172.21.15.45:0/2977766468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30004a80 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.003+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa480a9910 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.004+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa480cc0e0 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.004+0000 7faa3f7fe700 1 -- 172.21.15.45:0/2977766468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa300070f0 con 0x7faa4806e6e0 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.004+0000 7faa3f7fe700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa28066f30 0x7faa280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.005+0000 7faa3f7fe700 1 -- 172.21.15.45:0/2977766468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6522+0+0 (secure 0 0 0) 0x7faa30083b00 con 0x7faa4806e6e0 2023-10-06T05:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.005+0000 7faa477fe700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa28066f30 0x7faa280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.005+0000 7faa477fe700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa28066f30 0x7faa280693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faa3800b6a0 tx=0x7faa3801b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.009+0000 7faa3f7fe700 1 -- 172.21.15.45:0/2977766468 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa30050ac0 con 0x7faa4806e6e0 2023-10-06T05:17:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.255+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faa4806b230 con 0x7faa4806e6e0 2023-10-06T05:17:20.257 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:17:20.256+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:17:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.260+0000 7faa3f7fe700 1 -- 172.21.15.45:0/2977766468 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7faa4806b230 con 0x7faa4806e6e0 2023-10-06T05:17:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa28066f30 msgr2=0x7faa280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa28066f30 0x7faa280693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faa3800b6a0 tx=0x7faa3801b000 comp rx=0 tx=0).stop 2023-10-06T05:17:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa4806e6e0 msgr2=0x7faa480afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa4806e6e0 0x7faa480afeb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faa3000c7c0 tx=0x7faa30007f70 comp rx=0 tx=0).stop 2023-10-06T05:17:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 shutdown_connections 2023-10-06T05:17:20.265 INFO:tasks.ceph.mds.c.smithi045.stderr: -3271> 2023-10-06T05:16:33.164+0000 7f812670b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa28066f30 0x7faa280693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa48155440 0x7faa48157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4806a890 0x7faa4816da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 --2- 172.21.15.45:0/2977766468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa4806e6e0 0x7faa480afeb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 >> 172.21.15.45:0/2977766468 conn(0x7faa480cc5a0 msgr2=0x7faa480cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 shutdown_connections 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.263+0000 7faa4ddac700 1 -- 172.21.15.45:0/2977766468 wait complete. 2023-10-06T05:17:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:17:20.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:17:20.293 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:17:20.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.414+0000 7f72003a4700 1 Processor -- start 2023-10-06T05:17:20.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.414+0000 7f72003a4700 1 -- start start 2023-10-06T05:17:20.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f72003a4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71f80b18e0 0x7f71f815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f72003a4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71f80b1cb0 0x7f71f806a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f72003a4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f806b1c0 con 0x7f71f806ad60 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f72003a4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f8068750 con 0x7f71f80b1cb0 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f72003a4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f8067860 con 0x7f71f80b18e0 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71f80b1cb0 0x7f71f806a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71f80b1cb0 0x7f71f806a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52430/0 (socket says 172.21.15.45:52430) 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 -- 172.21.15.45:0/4085235149 learned_addr learned my addr 172.21.15.45:0/4085235149 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fe140700 1 --2- 172.21.15.45:0/4085235149 >> v2:172.21.15.144:3300/0 conn(0x7f71f80b18e0 0x7f71f815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 -- 172.21.15.45:0/4085235149 >> v2:172.21.15.144:3300/0 conn(0x7f71f80b18e0 msgr2=0x7f71f815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 --2- 172.21.15.45:0/4085235149 >> v2:172.21.15.144:3300/0 conn(0x7f71f80b18e0 0x7f71f815b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 -- 172.21.15.45:0/4085235149 >> v1:172.21.15.144:6789/0 conn(0x7f71f806ad60 legacy=0x7f71f806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 -- 172.21.15.45:0/4085235149 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8067b40 con 0x7f71f80b1cb0 2023-10-06T05:17:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fe140700 1 --2- 172.21.15.45:0/4085235149 >> v2:172.21.15.144:3300/0 conn(0x7f71f80b18e0 0x7f71f815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:20.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.415+0000 7f71fd93f700 1 --2- 172.21.15.45:0/4085235149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f80b1cb0 0x7f71f806a820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f71f806e230 tx=0x7f71e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550c02033a81bd59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f71fd13e700 1 -- 172.21.15.45:0/4085235149 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e800b820 con 0x7f71f80b1cb0 2023-10-06T05:17:20.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f71fd13e700 1 -- 172.21.15.45:0/4085235149 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e800b980 con 0x7f71f80b1cb0 2023-10-06T05:17:20.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f71fd13e700 1 -- 172.21.15.45:0/4085235149 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e800bd20 con 0x7f71f80b1cb0 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 -- 172.21.15.45:0/4085235149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f80b1cb0 msgr2=0x7f71f806a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 --2- 172.21.15.45:0/4085235149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f80b1cb0 0x7f71f806a820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f71f806e230 tx=0x7f71e8009250 comp rx=0 tx=0).stop 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 -- 172.21.15.45:0/4085235149 shutdown_connections 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 --2- 172.21.15.45:0/4085235149 >> v2:172.21.15.144:3300/0 conn(0x7f71f80b18e0 0x7f71f815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 --2- 172.21.15.45:0/4085235149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f80b1cb0 0x7f71f806a820 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 -- 172.21.15.45:0/4085235149 >> 172.21.15.45:0/4085235149 conn(0x7f71f80cc570 msgr2=0x7f71f80cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 -- 172.21.15.45:0/4085235149 shutdown_connections 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.416+0000 7f72003a4700 1 -- 172.21.15.45:0/4085235149 wait complete. 2023-10-06T05:17:20.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.417+0000 7f72003a4700 1 Processor -- start 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f72003a4700 1 -- start start 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f72003a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f806ad60 0x7f71f80c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f72003a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71f80b18e0 0x7f71f8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f72003a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71f80b1cb0 0x7f71f80c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f72003a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8169610 con 0x7f71f806ad60 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f72003a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f816d010 con 0x7f71f80b1cb0 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f72003a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f816d2f0 con 0x7f71f80b18e0 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f71fe140700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f806ad60 0x7f71f80c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f71fe140700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f806ad60 0x7f71f80c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52442/0 (socket says 172.21.15.45:52442) 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.418+0000 7f71fe140700 1 -- 172.21.15.45:0/3631311172 learned_addr learned my addr 172.21.15.45:0/3631311172 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe941700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71f80b1cb0 0x7f71f80c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fd93f700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71f80b18e0 0x7f71f8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe140700 1 -- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71f80b18e0 msgr2=0x7f71f8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe140700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71f80b18e0 0x7f71f8154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe140700 1 -- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71f80b1cb0 msgr2=0x7f71f80c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe140700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71f80b1cb0 0x7f71f80c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe140700 1 -- 172.21.15.45:0/3631311172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80094f0 con 0x7f71f806ad60 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe941700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71f80b1cb0 0x7f71f80c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71fe140700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f806ad60 0x7f71f80c7ea0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c830 tx=0x7f71f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71ee7fc700 1 -- 172.21.15.45:0/3631311172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4011600 con 0x7f71f806ad60 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71ee7fc700 1 -- 172.21.15.45:0/3631311172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4011760 con 0x7f71f806ad60 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f816d5d0 con 0x7f71f806ad60 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f71ee7fc700 1 -- 172.21.15.45:0/3631311172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4011b20 con 0x7f71f806ad60 2023-10-06T05:17:20.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.419+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f81550f0 con 0x7f71f806ad60 2023-10-06T05:17:20.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.420+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f81575b0 con 0x7f71f806ad60 2023-10-06T05:17:20.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.421+0000 7f71ee7fc700 1 -- 172.21.15.45:0/3631311172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71f400a3e0 con 0x7f71f806ad60 2023-10-06T05:17:20.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.421+0000 7f71ee7fc700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71dc066f30 0x7f71dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.421+0000 7f71ee7fc700 1 -- 172.21.15.45:0/3631311172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f71f4084d70 con 0x7f71f806ad60 2023-10-06T05:17:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.422+0000 7f71fd93f700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71dc066f30 0x7f71dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.422+0000 7f71fd93f700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71dc066f30 0x7f71dc0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71f806e230 tx=0x7f71e8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.426+0000 7f71ee7fc700 1 -- 172.21.15.45:0/3631311172 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71f4051c60 con 0x7f71f806ad60 2023-10-06T05:17:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:20.676+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71f80aa070 con 0x7f71f806ad60 2023-10-06T05:17:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.263+0000 7f71ee7fc700 1 -- 172.21.15.45:0/3631311172 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71f4055280 con 0x7f71f806ad60 2023-10-06T05:17:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71dc066f30 msgr2=0x7f71dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71dc066f30 0x7f71dc0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71f806e230 tx=0x7f71e8016000 comp rx=0 tx=0).stop 2023-10-06T05:17:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f806ad60 msgr2=0x7f71f80c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f806ad60 0x7f71f80c7ea0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c830 tx=0x7f71f4008040 comp rx=0 tx=0).stop 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 shutdown_connections 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f71dc066f30 0x7f71dc0693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71f80b1cb0 0x7f71f80c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f806ad60 0x7f71f80c7ea0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 --2- 172.21.15.45:0/3631311172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71f80b18e0 0x7f71f8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 >> 172.21.15.45:0/3631311172 conn(0x7f71f80cc570 msgr2=0x7f71f8156100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 shutdown_connections 2023-10-06T05:17:21.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.267+0000 7f72003a4700 1 -- 172.21.15.45:0/3631311172 wait complete. 2023-10-06T05:17:21.284 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:17:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.467+0000 7f8986892700 1 Processor -- start 2023-10-06T05:17:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.468+0000 7f8986892700 1 -- start start 2023-10-06T05:17:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.468+0000 7f8986892700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8980154c80 0x7f8980157640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.468+0000 7f8986892700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8980157b80 0x7f89800ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.468+0000 7f8986892700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89800cbfc0 con 0x7f89800ce740 2023-10-06T05:17:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.468+0000 7f8986892700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8980155050 con 0x7f8980157b80 2023-10-06T05:17:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.468+0000 7f8986892700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89801585e0 con 0x7f8980154c80 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f8984e2f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f89800ce740 0x7f89800cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41720/0 (socket says 172.21.15.45:41720) 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f8984e2f700 1 -- 172.21.15.45:0/1298823239 learned_addr learned my addr 172.21.15.45:0/1298823239 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897ffff700 1 --2- 172.21.15.45:0/1298823239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8980154c80 0x7f8980157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897f7fe700 1 --2- 172.21.15.45:0/1298823239 >> v2:172.21.15.144:3300/0 conn(0x7f8980157b80 0x7f89800ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897effd700 1 -- 172.21.15.45:0/1298823239 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390698412 0 0) 0x7f89800cbfc0 con 0x7f89800ce740 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897effd700 1 -- 172.21.15.45:0/1298823239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8968003570 con 0x7f89800ce740 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897ffff700 1 -- 172.21.15.45:0/1298823239 >> v2:172.21.15.144:3300/0 conn(0x7f8980157b80 msgr2=0x7f89800ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897ffff700 1 --2- 172.21.15.45:0/1298823239 >> v2:172.21.15.144:3300/0 conn(0x7f8980157b80 0x7f89800ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897ffff700 1 -- 172.21.15.45:0/1298823239 >> v1:172.21.15.45:6789/0 conn(0x7f89800ce740 legacy=0x7f89800cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897effd700 1 -- 172.21.15.45:0/1298823239 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946757249 0 0) 0x7f8968003570 con 0x7f89800ce740 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897ffff700 1 -- 172.21.15.45:0/1298823239 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980158020 con 0x7f8980154c80 2023-10-06T05:17:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897f7fe700 1 --2- 172.21.15.45:0/1298823239 >> v2:172.21.15.144:3300/0 conn(0x7f8980157b80 0x7f89800ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.469+0000 7f897ffff700 1 --2- 172.21.15.45:0/1298823239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8980154c80 0x7f8980157640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89700098e0 tx=0x7f89700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ea8df2b0711de6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f897effd700 1 -- 172.21.15.45:0/1298823239 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89780017f0 con 0x7f8980154c80 2023-10-06T05:17:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f897effd700 1 -- 172.21.15.45:0/1298823239 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8970011420 con 0x7f8980154c80 2023-10-06T05:17:21.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f897effd700 1 -- 172.21.15.45:0/1298823239 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700117a0 con 0x7f8980154c80 2023-10-06T05:17:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 -- 172.21.15.45:0/1298823239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8980154c80 msgr2=0x7f8980157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 --2- 172.21.15.45:0/1298823239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8980154c80 0x7f8980157640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89700098e0 tx=0x7f89700091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 -- 172.21.15.45:0/1298823239 shutdown_connections 2023-10-06T05:17:21.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 --2- 172.21.15.45:0/1298823239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8980154c80 0x7f8980157640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 --2- 172.21.15.45:0/1298823239 >> v2:172.21.15.144:3300/0 conn(0x7f8980157b80 0x7f89800ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 -- 172.21.15.45:0/1298823239 >> 172.21.15.45:0/1298823239 conn(0x7f89800680b0 msgr2=0x7f89800cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 -- 172.21.15.45:0/1298823239 shutdown_connections 2023-10-06T05:17:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.470+0000 7f8986892700 1 -- 172.21.15.45:0/1298823239 wait complete. 2023-10-06T05:17:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.472+0000 7f8986892700 1 Processor -- start 2023-10-06T05:17:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.472+0000 7f8986892700 1 -- start start 2023-10-06T05:17:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f8986892700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89800ce740 0x7f89801510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f8986892700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8980154c80 0x7f89800c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f8986892700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8980157b80 0x7f898014fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f8986892700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800b0cf0 con 0x7f8980154c80 2023-10-06T05:17:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f8986892700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800b0f60 con 0x7f8980157b80 2023-10-06T05:17:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f8986892700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89800c7830 con 0x7f89800ce740 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f897f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8980154c80 0x7f89800c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f897f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8980154c80 0x7f89800c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52446/0 (socket says 172.21.15.45:52446) 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f897f7fe700 1 -- 172.21.15.45:0/908253359 learned_addr learned my addr 172.21.15.45:0/908253359 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f897ffff700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89800ce740 0x7f89801510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.473+0000 7f8984e2f700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8980157b80 0x7f898014fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f897f7fe700 1 -- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89800ce740 msgr2=0x7f89801510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f897f7fe700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89800ce740 0x7f89801510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f897f7fe700 1 -- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8980157b80 msgr2=0x7f898014fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f897f7fe700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8980157b80 0x7f898014fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f897f7fe700 1 -- 172.21.15.45:0/908253359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8974009420 con 0x7f8980154c80 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f897ffff700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89800ce740 0x7f89801510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f8984e2f700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8980157b80 0x7f898014fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f897f7fe700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8980154c80 0x7f89800c72f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f897400ec80 tx=0x7f89740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f8967fff700 1 -- 172.21.15.45:0/908253359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400c850 con 0x7f8980154c80 2023-10-06T05:17:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8970009420 con 0x7f8980154c80 2023-10-06T05:17:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f8967fff700 1 -- 172.21.15.45:0/908253359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897400c9b0 con 0x7f8980154c80 2023-10-06T05:17:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f8967fff700 1 -- 172.21.15.45:0/908253359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400cd50 con 0x7f8980154c80 2023-10-06T05:17:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.474+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89800c7dc0 con 0x7f8980154c80 2023-10-06T05:17:21.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.475+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898004c380 con 0x7f8980154c80 2023-10-06T05:17:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.476+0000 7f8967fff700 1 -- 172.21.15.45:0/908253359 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8974013680 con 0x7f8980154c80 2023-10-06T05:17:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.476+0000 7f8967fff700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f896c066f30 0x7f896c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.476+0000 7f8967fff700 1 -- 172.21.15.45:0/908253359 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f8974084a10 con 0x7f8980154c80 2023-10-06T05:17:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.476+0000 7f897ffff700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f896c066f30 0x7f896c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:21.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.477+0000 7f897ffff700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f896c066f30 0x7f896c0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f89700091e0 tx=0x7f8970016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.481+0000 7f8967fff700 1 -- 172.21.15.45:0/908253359 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8974051880 con 0x7f8980154c80 2023-10-06T05:17:21.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:21.683+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89800b10a0 con 0x7f8980154c80 2023-10-06T05:17:22.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.265+0000 7f8967fff700 1 -- 172.21.15.45:0/908253359 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8974054ea0 con 0x7f8980154c80 2023-10-06T05:17:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f896c066f30 msgr2=0x7f896c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f896c066f30 0x7f896c0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f89700091e0 tx=0x7f8970016000 comp rx=0 tx=0).stop 2023-10-06T05:17:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8980154c80 msgr2=0x7f89800c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8980154c80 0x7f89800c72f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f897400ec80 tx=0x7f89740091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 shutdown_connections 2023-10-06T05:17:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f896c066f30 0x7f896c0693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8980157b80 0x7f898014fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89800ce740 0x7f89801510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 --2- 172.21.15.45:0/908253359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8980154c80 0x7f89800c72f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 >> 172.21.15.45:0/908253359 conn(0x7f89800680b0 msgr2=0x7f8980156110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 shutdown_connections 2023-10-06T05:17:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.270+0000 7f8986892700 1 -- 172.21.15.45:0/908253359 wait complete. 2023-10-06T05:17:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:17:22.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:17:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.425+0000 7ffb86016700 1 Processor -- start 2023-10-06T05:17:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.425+0000 7ffb86016700 1 -- start start 2023-10-06T05:17:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.426+0000 7ffb86016700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800677e0 0x7ffb800c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.426+0000 7ffb86016700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb800ad1f0 0x7ffb801404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.426+0000 7ffb86016700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb800ca7f0 con 0x7ffb80067bb0 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.426+0000 7ffb86016700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb80068000 con 0x7ffb800ad1f0 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.426+0000 7ffb86016700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800ccdf0 con 0x7ffb800677e0 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb800ad1f0 0x7ffb801404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7f7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800677e0 0x7ffb800c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb800ad1f0 0x7ffb801404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:49688/0 (socket says 172.21.15.45:49688) 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7f7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800677e0 0x7ffb800c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:40700/0 (socket says 172.21.15.45:40700) 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7ffff700 1 -- 172.21.15.45:0/1268906538 learned_addr learned my addr 172.21.15.45:0/1268906538 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7effd700 1 --1- 172.21.15.45:0/1268906538 >> v1:172.21.15.144:6789/0 conn(0x7ffb80067bb0 0x7ffb800c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:35382/0 (socket says 172.21.15.45:35382) 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7f7fe700 1 -- 172.21.15.45:0/1268906538 >> v2:172.21.15.144:3300/0 conn(0x7ffb800ad1f0 msgr2=0x7ffb801404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:22.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7f7fe700 1 --2- 172.21.15.45:0/1268906538 >> v2:172.21.15.144:3300/0 conn(0x7ffb800ad1f0 0x7ffb801404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7f7fe700 1 -- 172.21.15.45:0/1268906538 >> v1:172.21.15.144:6789/0 conn(0x7ffb80067bb0 legacy=0x7ffb800c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7f7fe700 1 -- 172.21.15.45:0/1268906538 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb800c9f50 con 0x7ffb800677e0 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7e7fc700 1 -- 172.21.15.45:0/1268906538 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219742531 0 0) 0x7ffb800ca7f0 con 0x7ffb80067bb0 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7ffff700 1 --2- 172.21.15.45:0/1268906538 >> v2:172.21.15.144:3300/0 conn(0x7ffb800ad1f0 0x7ffb801404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.427+0000 7ffb7f7fe700 1 --2- 172.21.15.45:0/1268906538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800677e0 0x7ffb800c7500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb700098e0 tx=0x7ffb700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a0fe9f50d7a8ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb7e7fc700 1 -- 172.21.15.45:0/1268906538 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000b640 con 0x7ffb800677e0 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb7e7fc700 1 -- 172.21.15.45:0/1268906538 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7000b7a0 con 0x7ffb800677e0 2023-10-06T05:17:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb7e7fc700 1 -- 172.21.15.45:0/1268906538 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000bb40 con 0x7ffb800677e0 2023-10-06T05:17:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 -- 172.21.15.45:0/1268906538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800677e0 msgr2=0x7ffb800c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 --2- 172.21.15.45:0/1268906538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800677e0 0x7ffb800c7500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb700098e0 tx=0x7ffb700091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 -- 172.21.15.45:0/1268906538 shutdown_connections 2023-10-06T05:17:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 --2- 172.21.15.45:0/1268906538 >> v2:172.21.15.144:3300/0 conn(0x7ffb800ad1f0 0x7ffb801404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 --2- 172.21.15.45:0/1268906538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800677e0 0x7ffb800c7500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 -- 172.21.15.45:0/1268906538 >> 172.21.15.45:0/1268906538 conn(0x7ffb80068730 msgr2=0x7ffb80068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 -- 172.21.15.45:0/1268906538 shutdown_connections 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.428+0000 7ffb86016700 1 -- 172.21.15.45:0/1268906538 wait complete. 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 Processor -- start 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 -- start start 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb800677e0 0x7ffb80157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb80067bb0 0x7ffb8016c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800ad1f0 0x7ffb8016d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800cc1a0 con 0x7ffb80067bb0 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb80157d70 con 0x7ffb800677e0 2023-10-06T05:17:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.430+0000 7ffb86016700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800cb590 con 0x7ffb800ad1f0 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb80067bb0 0x7ffb8016c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb80067bb0 0x7ffb8016c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52456/0 (socket says 172.21.15.45:52456) 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 -- 172.21.15.45:0/2607170147 learned_addr learned my addr 172.21.15.45:0/2607170147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7f7fe700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb800677e0 0x7ffb80157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7ffff700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800ad1f0 0x7ffb8016d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 -- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800ad1f0 msgr2=0x7ffb8016d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800ad1f0 0x7ffb8016d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 -- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb800677e0 msgr2=0x7ffb80157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb800677e0 0x7ffb80157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 -- 172.21.15.45:0/2607170147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009ae0 con 0x7ffb80067bb0 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7ffff700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800ad1f0 0x7ffb8016d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb7effd700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb80067bb0 0x7ffb8016c900 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb740011e0 tx=0x7ffb74007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb677fe700 1 -- 172.21.15.45:0/2607170147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74005a70 con 0x7ffb80067bb0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb677fe700 1 -- 172.21.15.45:0/2607170147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb74005bd0 con 0x7ffb80067bb0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb70009420 con 0x7ffb80067bb0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb677fe700 1 -- 172.21.15.45:0/2607170147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb740070d0 con 0x7ffb80067bb0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.431+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb800cbb50 con 0x7ffb80067bb0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.432+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb800cc310 con 0x7ffb80067bb0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.433+0000 7ffb677fe700 1 -- 172.21.15.45:0/2607170147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb74007480 con 0x7ffb80067bb0 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.433+0000 7ffb677fe700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb6c066f30 0x7ffb6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.434+0000 7ffb677fe700 1 -- 172.21.15.45:0/2607170147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6170+0+0 (secure 0 0 0) 0x7ffb74083c10 con 0x7ffb80067bb0 2023-10-06T05:17:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.434+0000 7ffb7f7fe700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb6c066f30 0x7ffb6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.434+0000 7ffb7f7fe700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb6c066f30 0x7ffb6c0693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb700093f0 tx=0x7ffb70017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.438+0000 7ffb677fe700 1 -- 172.21.15.45:0/2607170147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb7404b5a0 con 0x7ffb80067bb0 2023-10-06T05:17:22.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:22.647+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb8004c380 con 0x7ffb80067bb0 2023-10-06T05:17:23.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.270+0000 7ffb677fe700 1 -- 172.21.15.45:0/2607170147 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffb74017070 con 0x7ffb80067bb0 2023-10-06T05:17:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb6c066f30 msgr2=0x7ffb6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb6c066f30 0x7ffb6c0693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb700093f0 tx=0x7ffb70017000 comp rx=0 tx=0).stop 2023-10-06T05:17:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb80067bb0 msgr2=0x7ffb8016c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb80067bb0 0x7ffb8016c900 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb740011e0 tx=0x7ffb74007e10 comp rx=0 tx=0).stop 2023-10-06T05:17:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 shutdown_connections 2023-10-06T05:17:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffb6c066f30 0x7ffb6c0693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb800ad1f0 0x7ffb8016d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb800677e0 0x7ffb80157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 --2- 172.21.15.45:0/2607170147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb80067bb0 0x7ffb8016c900 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.274+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 >> 172.21.15.45:0/2607170147 conn(0x7ffb80068730 msgr2=0x7ffb800c2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.275+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 shutdown_connections 2023-10-06T05:17:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.275+0000 7ffb86016700 1 -- 172.21.15.45:0/2607170147 wait complete. 2023-10-06T05:17:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-06T05:17:23.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:17:23.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.429+0000 7f1e139e0700 1 Processor -- start 2023-10-06T05:17:23.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.429+0000 7f1e139e0700 1 -- start start 2023-10-06T05:17:23.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e139e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c146e40 0x7f1e0c147210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e139e0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1e0c067d30 0x7f1e0c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e139e0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0c8770 con 0x7f1e0c0cb650 2023-10-06T05:17:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e139e0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cc030 con 0x7f1e0c067d30 2023-10-06T05:17:23.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e139e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0c8120 con 0x7f1e0c146e40 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e1177c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c146e40 0x7f1e0c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e11f7d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1e0c067d30 0x7f1e0c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e11f7d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1e0c067d30 0x7f1e0c0c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:49704/0 (socket says 172.21.15.45:49704) 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e1177c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c146e40 0x7f1e0c147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:40724/0 (socket says 172.21.15.45:40724) 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e10f7b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1e0c0cb650 0x7f1e0c067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:35388/0 (socket says 172.21.15.45:35388) 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.430+0000 7f1e11f7d700 1 -- 172.21.15.45:0/1984185770 learned_addr learned my addr 172.21.15.45:0/1984185770 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e03fff700 1 -- 172.21.15.45:0/1984185770 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406828893 0 0) 0x7f1e0c0c8770 con 0x7f1e0c0cb650 2023-10-06T05:17:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e03fff700 1 -- 172.21.15.45:0/1984185770 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df8003570 con 0x7f1e0c0cb650 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e1177c700 1 -- 172.21.15.45:0/1984185770 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c067d30 msgr2=0x7f1e0c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e1177c700 1 --2- 172.21.15.45:0/1984185770 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c067d30 0x7f1e0c0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e1177c700 1 -- 172.21.15.45:0/1984185770 >> v1:172.21.15.144:6789/0 conn(0x7f1e0c0cb650 legacy=0x7f1e0c067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e1177c700 1 -- 172.21.15.45:0/1984185770 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0c8400 con 0x7f1e0c146e40 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e11f7d700 1 --2- 172.21.15.45:0/1984185770 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c067d30 0x7f1e0c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e1177c700 1 --2- 172.21.15.45:0/1984185770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c146e40 0x7f1e0c147210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098e0 tx=0x7f1e080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9964f94282bf7d9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e03fff700 1 -- 172.21.15.45:0/1984185770 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011630 con 0x7f1e0c146e40 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e03fff700 1 -- 172.21.15.45:0/1984185770 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08011790 con 0x7f1e0c146e40 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.431+0000 7f1e03fff700 1 -- 172.21.15.45:0/1984185770 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011b10 con 0x7f1e0c146e40 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 -- 172.21.15.45:0/1984185770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c146e40 msgr2=0x7f1e0c147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1984185770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c146e40 0x7f1e0c147210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098e0 tx=0x7f1e080091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 -- 172.21.15.45:0/1984185770 shutdown_connections 2023-10-06T05:17:23.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1984185770 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c067d30 0x7f1e0c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1984185770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c146e40 0x7f1e0c147210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 -- 172.21.15.45:0/1984185770 >> 172.21.15.45:0/1984185770 conn(0x7f1e0c0691c0 msgr2=0x7f1e0c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:23.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 -- 172.21.15.45:0/1984185770 shutdown_connections 2023-10-06T05:17:23.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.432+0000 7f1e139e0700 1 -- 172.21.15.45:0/1984185770 wait complete. 2023-10-06T05:17:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.433+0000 7f1e139e0700 1 Processor -- start 2023-10-06T05:17:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.433+0000 7f1e139e0700 1 -- start start 2023-10-06T05:17:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e139e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e0c067d30 0x7f1e0c15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e139e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0cb650 0x7f1e0c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e139e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c146e40 0x7f1e0c0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e139e0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15db30 con 0x7f1e0c067d30 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e139e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c16e510 con 0x7f1e0c146e40 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e139e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c16e7f0 con 0x7f1e0c0cb650 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e1177c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e0c067d30 0x7f1e0c15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e1177c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e0c067d30 0x7f1e0c15c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52472/0 (socket says 172.21.15.45:52472) 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e1177c700 1 -- 172.21.15.45:0/1602909378 learned_addr learned my addr 172.21.15.45:0/1602909378 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e11f7d700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c146e40 0x7f1e0c0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.434+0000 7f1e10f7b700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0cb650 0x7f1e0c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e1177c700 1 -- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0cb650 msgr2=0x7f1e0c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e1177c700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0cb650 0x7f1e0c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e1177c700 1 -- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c146e40 msgr2=0x7f1e0c0c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e1177c700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c146e40 0x7f1e0c0c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e1177c700 1 -- 172.21.15.45:0/1602909378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009ae0 con 0x7f1e0c067d30 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e1177c700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e0c067d30 0x7f1e0c15c460 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0c8260 tx=0x7f1e08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e017fa700 1 -- 172.21.15.45:0/1602909378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011900 con 0x7f1e0c067d30 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e017fa700 1 -- 172.21.15.45:0/1602909378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08011a60 con 0x7f1e0c067d30 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e08009420 con 0x7f1e0c067d30 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e017fa700 1 -- 172.21.15.45:0/1602909378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0801a680 con 0x7f1e0c067d30 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.435+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c15e060 con 0x7f1e0c067d30 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.436+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c06e2d0 con 0x7f1e0c067d30 2023-10-06T05:17:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.437+0000 7f1e017fa700 1 -- 172.21.15.45:0/1602909378 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e0801aa70 con 0x7f1e0c067d30 2023-10-06T05:17:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.437+0000 7f1e017fa700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.437+0000 7f1e017fa700 1 -- 172.21.15.45:0/1602909378 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f1e08086010 con 0x7f1e0c067d30 2023-10-06T05:17:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.437+0000 7f1e10f7b700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.438+0000 7f1e10f7b700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1df0066f30 0x7f1df00693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0c93e0 tx=0x7f1dfc000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:23.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.442+0000 7f1e017fa700 1 -- 172.21.15.45:0/1602909378 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e08053280 con 0x7f1e0c067d30 2023-10-06T05:17:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.640+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1e0c04bb40 con 0x7f1e0c067d30 2023-10-06T05:17:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.644+0000 7f1e017fa700 1 -- 172.21.15.45:0/1602909378 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e080568a0 con 0x7f1e0c067d30 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1df0066f30 msgr2=0x7f1df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1df0066f30 0x7f1df00693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0c93e0 tx=0x7f1dfc000bc0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e0c067d30 msgr2=0x7f1e0c15c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e0c067d30 0x7f1e0c15c460 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0c8260 tx=0x7f1e08017000 comp rx=0 tx=0).stop 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 shutdown_connections 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c146e40 0x7f1e0c0c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e0c067d30 0x7f1e0c15c460 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 --2- 172.21.15.45:0/1602909378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0cb650 0x7f1e0c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 >> 172.21.15.45:0/1602909378 conn(0x7f1e0c0691c0 msgr2=0x7f1e0c0c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 shutdown_connections 2023-10-06T05:17:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.648+0000 7f1e139e0700 1 -- 172.21.15.45:0/1602909378 wait complete. 2023-10-06T05:17:23.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2023-10-06T05:17:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.800+0000 7fdd3af57700 1 Processor -- start 2023-10-06T05:17:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.800+0000 7fdd3af57700 1 -- start start 2023-10-06T05:17:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd3af57700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 0x7fdd340c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd3af57700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdd340caad0 0x7fdd340caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd3af57700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3415b000 con 0x7fdd34156af0 2023-10-06T05:17:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd3af57700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3415b2e0 con 0x7fdd340caad0 2023-10-06T05:17:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd3af57700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3415b5c0 con 0x7fdd340c8ff0 2023-10-06T05:17:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd394f4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdd34156af0 0x7fdd34156f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41732/0 (socket says 172.21.15.45:41732) 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd394f4700 1 -- 172.21.15.45:0/1027127910 learned_addr learned my addr 172.21.15.45:0/1027127910 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd33fff700 1 --2- 172.21.15.45:0/1027127910 >> v2:172.21.15.144:3300/0 conn(0x7fdd340caad0 0x7fdd340caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd38cf3700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 0x7fdd340c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188424499 0 0) 0x7fdd3415b000 con 0x7fdd34156af0 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd24003570 con 0x7fdd34156af0 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1685151754 0 0) 0x7fdd24003570 con 0x7fdd34156af0 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.801+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 msgr2=0x7fdd340c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 0x7fdd340c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 >> v2:172.21.15.144:3300/0 conn(0x7fdd340caad0 msgr2=0x7fdd340caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 --2- 172.21.15.45:0/1027127910 >> v2:172.21.15.144:3300/0 conn(0x7fdd340caad0 0x7fdd340caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3415b8a0 con 0x7fdd34156af0 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fdd2c0017f0 con 0x7fdd34156af0 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 >> v1:172.21.15.45:6789/0 conn(0x7fdd34156af0 legacy=0x7fdd34156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 0x7fdd24003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24003fe0 0x7fdd24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24004d50 0x7fdd240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3415b2e0 con 0x7fdd24003fe0 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3415b5c0 con 0x7fdd24004d50 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2400ba90 con 0x7fdd340c8ff0 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd33fff700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24003fe0 0x7fdd24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd38cf3700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 0x7fdd24003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd394f4700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24004d50 0x7fdd240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd33fff700 1 -- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 msgr2=0x7fdd24003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd33fff700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd340c8ff0 0x7fdd24003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd33fff700 1 -- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24004d50 msgr2=0x7fdd240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd33fff700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24004d50 0x7fdd240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd33fff700 1 -- 172.21.15.45:0/1027127910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24003570 con 0x7fdd24003fe0 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd33fff700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24003fe0 0x7fdd24004810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009c00 tx=0x7fdd200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc06f0d7e742f47f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000e6b0 con 0x7fdd24003fe0 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2000e810 con 0x7fdd24003fe0 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.802+0000 7fdd337fe700 1 -- 172.21.15.45:0/1027127910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000ebd0 con 0x7fdd24003fe0 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.803+0000 7fdd3af57700 1 -- 172.21.15.45:0/1027127910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24003fe0 msgr2=0x7fdd24004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.803+0000 7fdd3af57700 1 --2- 172.21.15.45:0/1027127910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd24003fe0 0x7fdd24004810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009c00 tx=0x7fdd200091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.803+0000 7fdd394f4700 1 -- 172.21.15.45:0/1027127910 reap_dead start 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.803+0000 7fdd3af57700 1 -- 172.21.15.45:0/1027127910 shutdown_connections 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.803+0000 7fdd3af57700 1 -- 172.21.15.45:0/1027127910 >> 172.21.15.45:0/1027127910 conn(0x7fdd340cdf50 msgr2=0x7fdd340ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.803+0000 7fdd3af57700 1 -- 172.21.15.45:0/1027127910 shutdown_connections 2023-10-06T05:17:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.803+0000 7fdd3af57700 1 -- 172.21.15.45:0/1027127910 wait complete. 2023-10-06T05:17:23.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.804+0000 7fdd3af57700 1 Processor -- start 2023-10-06T05:17:23.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd3af57700 1 -- start start 2023-10-06T05:17:23.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd3af57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24003fe0 0x7fdd34155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd3af57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd340b33f0 0x7fdd3414f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd3af57700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd3416b470 0x7fdd3416b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd3af57700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3414fea0 con 0x7fdd340b33f0 2023-10-06T05:17:23.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd3af57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34150210 con 0x7fdd24003fe0 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd3af57700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340b2e30 con 0x7fdd3416b470 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd33fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd340b33f0 0x7fdd3414f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd33fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd340b33f0 0x7fdd3414f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52476/0 (socket says 172.21.15.45:52476) 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd33fff700 1 -- 172.21.15.45:0/2177764696 learned_addr learned my addr 172.21.15.45:0/2177764696 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.805+0000 7fdd38cf3700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24003fe0 0x7fdd34155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd394f4700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd3416b470 0x7fdd3416b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd33fff700 1 -- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd3416b470 msgr2=0x7fdd3416b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd33fff700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd3416b470 0x7fdd3416b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd33fff700 1 -- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24003fe0 msgr2=0x7fdd34155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd33fff700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24003fe0 0x7fdd34155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd33fff700 1 -- 172.21.15.45:0/2177764696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340b3110 con 0x7fdd340b33f0 2023-10-06T05:17:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd33fff700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd340b33f0 0x7fdd3414f960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdd24004730 tx=0x7fdd2001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd30ff9700 1 -- 172.21.15.45:0/2177764696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20022070 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd30ff9700 1 -- 172.21.15.45:0/2177764696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2000e980 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3416af50 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd30ff9700 1 -- 172.21.15.45:0/2177764696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20027470 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.806+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd34167800 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.807+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3414fea0 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.808+0000 7fdd30ff9700 1 -- 172.21.15.45:0/2177764696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd2001c450 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.808+0000 7fdd30ff9700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.808+0000 7fdd30ff9700 1 -- 172.21.15.45:0/2177764696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fdd20086050 con 0x7fdd340b33f0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.808+0000 7fdd38cf3700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.809+0000 7fdd38cf3700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd18066f30 0x7fdd180693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd34158600 tx=0x7fdd280091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:23.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:23.813+0000 7fdd30ff9700 1 -- 172.21.15.45:0/2177764696 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd3414fea0 con 0x7fdd340b33f0 2023-10-06T05:17:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.103+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) v1 -- 0x7fdd3414fea0 con 0x7fdd340b33f0 2023-10-06T05:17:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.272+0000 7fdd30ff9700 1 -- 172.21.15.45:0/2177764696 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v133) v1 ==== 134+0+0 (secure 0 0 0) 0x7fdd3414fea0 con 0x7fdd340b33f0 2023-10-06T05:17:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd18066f30 msgr2=0x7fdd180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd18066f30 0x7fdd180693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd34158600 tx=0x7fdd280091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd340b33f0 msgr2=0x7fdd3414f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd340b33f0 0x7fdd3414f960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdd24004730 tx=0x7fdd2001a000 comp rx=0 tx=0).stop 2023-10-06T05:17:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 shutdown_connections 2023-10-06T05:17:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd3416b470 0x7fdd3416b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd24003fe0 0x7fdd34155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 --2- 172.21.15.45:0/2177764696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd340b33f0 0x7fdd3414f960 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 >> 172.21.15.45:0/2177764696 conn(0x7fdd340cdf50 msgr2=0x7fdd340afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 shutdown_connections 2023-10-06T05:17:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.275+0000 7fdd3af57700 1 -- 172.21.15.45:0/2177764696 wait complete. 2023-10-06T05:17:24.292 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2023-10-06T05:17:24.292 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2023-10-06T05:17:24.292 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2023-10-06T05:17:24.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2023-10-06T05:17:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.427+0000 7f9e40f69700 1 Processor -- start 2023-10-06T05:17:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.427+0000 7f9e40f69700 1 -- start start 2023-10-06T05:17:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e40f69700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e3c0c7410 0x7f9e3c0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e40f69700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0c9df0 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e40f69700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0c9810 con 0x7f9e3c13f890 2023-10-06T05:17:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e40f69700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0ca280 con 0x7f9e3c0c7410 2023-10-06T05:17:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e39d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e3c0c7410 0x7f9e3c0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e3ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9e3c0c3e80 0x7f9e3c0c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41748/0 (socket says 172.21.15.45:41748) 2023-10-06T05:17:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e3ad9d700 1 -- 172.21.15.45:0/230066683 learned_addr learned my addr 172.21.15.45:0/230066683 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e39d9b700 1 -- 172.21.15.45:0/230066683 >> v1:172.21.15.144:6789/0 conn(0x7f9e3c13f890 legacy=0x7f9e3c0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e39d9b700 1 -- 172.21.15.45:0/230066683 >> v1:172.21.15.45:6789/0 conn(0x7f9e3c0c3e80 legacy=0x7f9e3c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.429+0000 7f9e39d9b700 1 -- 172.21.15.45:0/230066683 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0cc720 con 0x7f9e3c0c7410 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.428+0000 7f9e3959a700 1 -- 172.21.15.45:0/230066683 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856468992 0 0) 0x7f9e3c0c9df0 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.429+0000 7f9e39d9b700 1 --2- 172.21.15.45:0/230066683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c7410 0x7f9e3c0cb0b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9e300098e0 tx=0x7f9e300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6535b27dbd979b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.429+0000 7f9e3959a700 1 -- 172.21.15.45:0/230066683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30009e80 con 0x7f9e3c0c7410 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.429+0000 7f9e3959a700 1 -- 172.21.15.45:0/230066683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3000b8b0 con 0x7f9e3c0c7410 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.429+0000 7f9e3959a700 1 -- 172.21.15.45:0/230066683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000bc30 con 0x7f9e3c0c7410 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.429+0000 7f9e40f69700 1 -- 172.21.15.45:0/230066683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c7410 msgr2=0x7f9e3c0cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.429+0000 7f9e40f69700 1 --2- 172.21.15.45:0/230066683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c7410 0x7f9e3c0cb0b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9e300098e0 tx=0x7f9e300091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.430+0000 7f9e40f69700 1 -- 172.21.15.45:0/230066683 shutdown_connections 2023-10-06T05:17:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.430+0000 7f9e40f69700 1 --2- 172.21.15.45:0/230066683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c7410 0x7f9e3c0cb0b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.430+0000 7f9e40f69700 1 -- 172.21.15.45:0/230066683 >> 172.21.15.45:0/230066683 conn(0x7f9e3c069200 msgr2=0x7f9e3c0ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.430+0000 7f9e40f69700 1 -- 172.21.15.45:0/230066683 shutdown_connections 2023-10-06T05:17:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.430+0000 7f9e40f69700 1 -- 172.21.15.45:0/230066683 wait complete. 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.431+0000 7f9e40f69700 1 Processor -- start 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.431+0000 7f9e40f69700 1 -- start start 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c3e80 0x7f9e3c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3c0c7410 0x7f9e3c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3c13f890 0x7f9e3c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c159450 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c44e0 con 0x7f9e3c0c7410 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c47c0 con 0x7f9e3c13f890 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c3e80 0x7f9e3c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c3e80 0x7f9e3c1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52496/0 (socket says 172.21.15.45:52496) 2023-10-06T05:17:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 -- 172.21.15.45:0/2816829367 learned_addr learned my addr 172.21.15.45:0/2816829367 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e39d9b700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3c0c7410 0x7f9e3c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3ad9d700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3c13f890 0x7f9e3c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 -- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3c13f890 msgr2=0x7f9e3c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3c13f890 0x7f9e3c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 -- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3c0c7410 msgr2=0x7f9e3c158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3c0c7410 0x7f9e3c158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 -- 172.21.15.45:0/2816829367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009420 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3ad9d700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3c13f890 0x7f9e3c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e3a59c700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c3e80 0x7f9e3c1558c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00b5e0 tx=0x7f9e2c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e2a7fc700 1 -- 172.21.15.45:0/2816829367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00f510 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e2a7fc700 1 -- 172.21.15.45:0/2816829367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c00f670 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c0c4aa0 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e2a7fc700 1 -- 172.21.15.45:0/2816829367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00fa30 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.432+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c0af2b0 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.433+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c06df80 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.434+0000 7f9e2a7fc700 1 -- 172.21.15.45:0/2816829367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e2c01a400 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.434+0000 7f9e2a7fc700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e1c066f30 0x7f9e1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.435+0000 7f9e2a7fc700 1 -- 172.21.15.45:0/2816829367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f9e2c0841a0 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.435+0000 7f9e39d9b700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e1c066f30 0x7f9e1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.435+0000 7f9e39d9b700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e1c066f30 0x7f9e1c0693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e300093f0 tx=0x7f9e30016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:24.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.439+0000 7f9e2a7fc700 1 -- 172.21.15.45:0/2816829367 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e2c0513f0 con 0x7f9e3c0c3e80 2023-10-06T05:17:24.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:24.724+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) v1 -- 0x7f9e3c0c6af0 con 0x7f9e3c0c3e80 2023-10-06T05:17:25.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.276+0000 7f9e2a7fc700 1 -- 172.21.15.45:0/2816829367 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v134) v1 ==== 144+0+0 (secure 0 0 0) 0x7f9e2c054a10 con 0x7f9e3c0c3e80 2023-10-06T05:17:25.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e1c066f30 msgr2=0x7f9e1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e1c066f30 0x7f9e1c0693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e300093f0 tx=0x7f9e30016000 comp rx=0 tx=0).stop 2023-10-06T05:17:25.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c3e80 msgr2=0x7f9e3c1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c3e80 0x7f9e3c1558c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00b5e0 tx=0x7f9e2c009120 comp rx=0 tx=0).stop 2023-10-06T05:17:25.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 shutdown_connections 2023-10-06T05:17:25.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9e1c066f30 0x7f9e1c0693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3c13f890 0x7f9e3c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0c3e80 0x7f9e3c1558c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 --2- 172.21.15.45:0/2816829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3c0c7410 0x7f9e3c158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 >> 172.21.15.45:0/2816829367 conn(0x7f9e3c069200 msgr2=0x7f9e3c0c1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 shutdown_connections 2023-10-06T05:17:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.279+0000 7f9e40f69700 1 -- 172.21.15.45:0/2816829367 wait complete. 2023-10-06T05:17:25.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:17:25.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.433+0000 7fd407df3700 1 Processor -- start 2023-10-06T05:17:25.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.433+0000 7fd407df3700 1 -- start start 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd407df3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4000b0610 0x7fd4000c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd407df3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4000c3db0 0x7fd4000697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd407df3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4000ce6a0 con 0x7fd400069d00 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd407df3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4000690f0 con 0x7fd4000b0610 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd407df3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000ceeb0 con 0x7fd4000c3db0 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd405b8f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4000b0610 0x7fd4000c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd405b8f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4000b0610 0x7fd4000c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52498/0 (socket says 172.21.15.45:52498) 2023-10-06T05:17:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd405b8f700 1 -- 172.21.15.45:0/4128188273 learned_addr learned my addr 172.21.15.45:0/4128188273 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd40538e700 1 --2- 172.21.15.45:0/4128188273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4000c3db0 0x7fd4000697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd405b8f700 1 -- 172.21.15.45:0/4128188273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4000c3db0 msgr2=0x7fd4000697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd405b8f700 1 --2- 172.21.15.45:0/4128188273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4000c3db0 0x7fd4000697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd405b8f700 1 -- 172.21.15.45:0/4128188273 >> v1:172.21.15.144:6789/0 conn(0x7fd400069d00 legacy=0x7fd4000c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd405b8f700 1 -- 172.21.15.45:0/4128188273 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd400159880 con 0x7fd4000b0610 2023-10-06T05:17:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.434+0000 7fd40538e700 1 --2- 172.21.15.45:0/4128188273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4000c3db0 0x7fd4000697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd405b8f700 1 --2- 172.21.15.45:0/4128188273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000b0610 0x7fd4000c85d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00099c0 tx=0x7fd3f0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60299ba396ad52a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd404b8d700 1 -- 172.21.15.45:0/4128188273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0005c10 con 0x7fd4000b0610 2023-10-06T05:17:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd404b8d700 1 -- 172.21.15.45:0/4128188273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f0005d70 con 0x7fd4000b0610 2023-10-06T05:17:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd404b8d700 1 -- 172.21.15.45:0/4128188273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000b840 con 0x7fd4000b0610 2023-10-06T05:17:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd407df3700 1 -- 172.21.15.45:0/4128188273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000b0610 msgr2=0x7fd4000c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd407df3700 1 --2- 172.21.15.45:0/4128188273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000b0610 0x7fd4000c85d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00099c0 tx=0x7fd3f0009220 comp rx=0 tx=0).stop 2023-10-06T05:17:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd407df3700 1 -- 172.21.15.45:0/4128188273 shutdown_connections 2023-10-06T05:17:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd407df3700 1 --2- 172.21.15.45:0/4128188273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000b0610 0x7fd4000c85d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd407df3700 1 --2- 172.21.15.45:0/4128188273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4000c3db0 0x7fd4000697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.435+0000 7fd407df3700 1 -- 172.21.15.45:0/4128188273 >> 172.21.15.45:0/4128188273 conn(0x7fd400067e00 msgr2=0x7fd4000cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.436+0000 7fd407df3700 1 -- 172.21.15.45:0/4128188273 shutdown_connections 2023-10-06T05:17:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.436+0000 7fd407df3700 1 -- 172.21.15.45:0/4128188273 wait complete. 2023-10-06T05:17:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.437+0000 7fd407df3700 1 Processor -- start 2023-10-06T05:17:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.437+0000 7fd407df3700 1 -- start start 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.437+0000 7fd407df3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd400069d00 0x7fd40016a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.437+0000 7fd407df3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4000b0610 0x7fd400157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd407df3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000c3db0 0x7fd4000b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd407df3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000c5620 con 0x7fd4000c3db0 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd407df3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40016abb0 con 0x7fd4000b0610 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd407df3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4001583c0 con 0x7fd400069d00 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000c3db0 0x7fd4000b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000c3db0 0x7fd4000b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52508/0 (socket says 172.21.15.45:52508) 2023-10-06T05:17:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 -- 172.21.15.45:0/74429476 learned_addr learned my addr 172.21.15.45:0/74429476 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd405b8f700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd400069d00 0x7fd40016a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd40538e700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4000b0610 0x7fd400157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 -- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd400069d00 msgr2=0x7fd40016a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd400069d00 0x7fd40016a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 -- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4000b0610 msgr2=0x7fd400157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4000b0610 0x7fd400157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 -- 172.21.15.45:0/74429476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0009480 con 0x7fd4000c3db0 2023-10-06T05:17:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd405b8f700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd400069d00 0x7fd40016a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd40538e700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4000b0610 0x7fd400157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd406390700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000c3db0 0x7fd4000b2e80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c8e0 tx=0x7fd3f8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd3f5ffb700 1 -- 172.21.15.45:0/74429476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800ce80 con 0x7fd4000c3db0 2023-10-06T05:17:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd3f5ffb700 1 -- 172.21.15.45:0/74429476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f800a7e0 con 0x7fd4000c3db0 2023-10-06T05:17:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4000c8170 con 0x7fd4000c3db0 2023-10-06T05:17:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.438+0000 7fd3f5ffb700 1 -- 172.21.15.45:0/74429476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800ab80 con 0x7fd4000c3db0 2023-10-06T05:17:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.439+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40016d9c0 con 0x7fd4000c3db0 2023-10-06T05:17:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.440+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4000c5790 con 0x7fd4000c3db0 2023-10-06T05:17:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.440+0000 7fd3f5ffb700 1 -- 172.21.15.45:0/74429476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3f8012420 con 0x7fd4000c3db0 2023-10-06T05:17:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.440+0000 7fd3f5ffb700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e4066f30 0x7fd3e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.441+0000 7fd3f5ffb700 1 -- 172.21.15.45:0/74429476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fd3f8084820 con 0x7fd4000c3db0 2023-10-06T05:17:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.441+0000 7fd405b8f700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e4066f30 0x7fd3e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.441+0000 7fd405b8f700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e4066f30 0x7fd3e40693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009260 tx=0x7fd3f0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.445+0000 7fd3f5ffb700 1 -- 172.21.15.45:0/74429476 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4000c5790 con 0x7fd4000c3db0 2023-10-06T05:17:25.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.653+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd4000c5790 con 0x7fd4000c3db0 2023-10-06T05:17:25.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.657+0000 7fd3f5ffb700 1 -- 172.21.15.45:0/74429476 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4000c5790 con 0x7fd4000c3db0 2023-10-06T05:17:25.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e4066f30 msgr2=0x7fd3e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e4066f30 0x7fd3e40693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009260 tx=0x7fd3f0016000 comp rx=0 tx=0).stop 2023-10-06T05:17:25.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000c3db0 msgr2=0x7fd4000b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000c3db0 0x7fd4000b2e80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c8e0 tx=0x7fd3f8001710 comp rx=0 tx=0).stop 2023-10-06T05:17:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 shutdown_connections 2023-10-06T05:17:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e4066f30 0x7fd3e40693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4000c3db0 0x7fd4000b2e80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd400069d00 0x7fd40016a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 --2- 172.21.15.45:0/74429476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4000b0610 0x7fd400157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 >> 172.21.15.45:0/74429476 conn(0x7fd400067e00 msgr2=0x7fd40006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:25.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 shutdown_connections 2023-10-06T05:17:25.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.659+0000 7fd407df3700 1 -- 172.21.15.45:0/74429476 wait complete. 2023-10-06T05:17:25.675 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:17:25.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:25.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:25.763 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:17:25.764 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:17:25.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.899+0000 7f12e449b700 1 Processor -- start 2023-10-06T05:17:25.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.899+0000 7f12e449b700 1 -- start start 2023-10-06T05:17:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.900+0000 7f12e449b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc1570b0 0x7f12dc06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.900+0000 7f12e449b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f12dc15ac10 0x7f12dc0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.900+0000 7f12e449b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12dc15b380 con 0x7f12dc156ce0 2023-10-06T05:17:25.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.900+0000 7f12e449b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc157520 con 0x7f12dc15ac10 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.900+0000 7f12e449b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc06c010 con 0x7f12dc1570b0 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.900+0000 7f12e2237700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f12dc156ce0 0x7f12dc0afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41764/0 (socket says 172.21.15.45:41764) 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.900+0000 7f12e2237700 1 -- 172.21.15.45:0/383315298 learned_addr learned my addr 172.21.15.45:0/383315298 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1a36700 1 --2- 172.21.15.45:0/383315298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc1570b0 0x7f12dc06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e2a38700 1 --2- 172.21.15.45:0/383315298 >> v2:172.21.15.144:3300/0 conn(0x7f12dc15ac10 0x7f12dc0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1235700 1 -- 172.21.15.45:0/383315298 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272704658 0 0) 0x7f12dc15b380 con 0x7f12dc156ce0 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1235700 1 -- 172.21.15.45:0/383315298 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c8003570 con 0x7f12dc156ce0 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1a36700 1 -- 172.21.15.45:0/383315298 >> v2:172.21.15.144:3300/0 conn(0x7f12dc15ac10 msgr2=0x7f12dc0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1a36700 1 --2- 172.21.15.45:0/383315298 >> v2:172.21.15.144:3300/0 conn(0x7f12dc15ac10 0x7f12dc0ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1a36700 1 -- 172.21.15.45:0/383315298 >> v1:172.21.15.45:6789/0 conn(0x7f12dc156ce0 legacy=0x7f12dc0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1235700 1 -- 172.21.15.45:0/383315298 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89348436 0 0) 0x7f12c8003570 con 0x7f12dc156ce0 2023-10-06T05:17:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1a36700 1 -- 172.21.15.45:0/383315298 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc15b660 con 0x7f12dc1570b0 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e2a38700 1 --2- 172.21.15.45:0/383315298 >> v2:172.21.15.144:3300/0 conn(0x7f12dc15ac10 0x7f12dc0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.901+0000 7f12e1a36700 1 --2- 172.21.15.45:0/383315298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc1570b0 0x7f12dc06bad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0098e0 tx=0x7f12cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc73c870b90c9229 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e1235700 1 -- 172.21.15.45:0/383315298 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8001470 con 0x7f12dc1570b0 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e1235700 1 -- 172.21.15.45:0/383315298 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc011420 con 0x7f12dc1570b0 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e1235700 1 -- 172.21.15.45:0/383315298 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0117a0 con 0x7f12dc1570b0 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 -- 172.21.15.45:0/383315298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc1570b0 msgr2=0x7f12dc06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 --2- 172.21.15.45:0/383315298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc1570b0 0x7f12dc06bad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0098e0 tx=0x7f12cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 -- 172.21.15.45:0/383315298 shutdown_connections 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 --2- 172.21.15.45:0/383315298 >> v2:172.21.15.144:3300/0 conn(0x7f12dc15ac10 0x7f12dc0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 --2- 172.21.15.45:0/383315298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc1570b0 0x7f12dc06bad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 -- 172.21.15.45:0/383315298 >> 172.21.15.45:0/383315298 conn(0x7f12dc06f090 msgr2=0x7f12dc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 -- 172.21.15.45:0/383315298 shutdown_connections 2023-10-06T05:17:25.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.902+0000 7f12e449b700 1 -- 172.21.15.45:0/383315298 wait complete. 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.903+0000 7f12e449b700 1 Processor -- start 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.903+0000 7f12e449b700 1 -- start start 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e449b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12dc156ce0 0x7f12dc169480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e449b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12dc1570b0 0x7f12dc0c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e449b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc15ac10 0x7f12dc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e449b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0c7590 con 0x7f12dc156ce0 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e449b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc16a420 con 0x7f12dc1570b0 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e449b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc169b80 con 0x7f12dc15ac10 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e2237700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12dc156ce0 0x7f12dc169480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e2237700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12dc156ce0 0x7f12dc169480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52514/0 (socket says 172.21.15.45:52514) 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e2237700 1 -- 172.21.15.45:0/912049175 learned_addr learned my addr 172.21.15.45:0/912049175 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e1a36700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12dc1570b0 0x7f12dc0c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.904+0000 7f12e2a38700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc15ac10 0x7f12dc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e2237700 1 -- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc15ac10 msgr2=0x7f12dc155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e2237700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc15ac10 0x7f12dc155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e2237700 1 -- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12dc1570b0 msgr2=0x7f12dc0c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e2237700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12dc1570b0 0x7f12dc0c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e2237700 1 -- 172.21.15.45:0/912049175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4009a20 con 0x7f12dc156ce0 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e2237700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12dc156ce0 0x7f12dc169480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f12d800cb90 tx=0x7f12d8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12d27fc700 1 -- 172.21.15.45:0/912049175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8007150 con 0x7f12dc156ce0 2023-10-06T05:17:25.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12d27fc700 1 -- 172.21.15.45:0/912049175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80072d0 con 0x7f12dc156ce0 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc009420 con 0x7f12dc156ce0 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12d27fc700 1 -- 172.21.15.45:0/912049175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8007690 con 0x7f12dc156ce0 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.905+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc16a140 con 0x7f12dc156ce0 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.906+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc06e2f0 con 0x7f12dc156ce0 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.907+0000 7f12d27fc700 1 -- 172.21.15.45:0/912049175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12d8004a40 con 0x7f12dc156ce0 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.907+0000 7f12d27fc700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c0066f30 0x7f12c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.907+0000 7f12d27fc700 1 -- 172.21.15.45:0/912049175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f12d8083c30 con 0x7f12dc156ce0 2023-10-06T05:17:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.907+0000 7f12e1a36700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c0066f30 0x7f12c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.908+0000 7f12e1a36700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c0066f30 0x7f12c00693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12cc003080 tx=0x7f12cc016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:25.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:25.912+0000 7f12d27fc700 1 -- 172.21.15.45:0/912049175 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12d8050fe0 con 0x7f12dc156ce0 2023-10-06T05:17:26.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.166+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12dc04bb40 con 0x7f12dc156ce0 2023-10-06T05:17:26.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.167+0000 7f12d27fc700 1 -- 172.21.15.45:0/912049175 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f12d8018020 con 0x7f12dc156ce0 2023-10-06T05:17:26.168 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:26.169 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3799725793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3799725793},{"type":"v1","addr":"172.21.15.144:6836","nonce":3799725793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3807770630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3807770630},{"type":"v1","addr":"172.21.15.144:6837","nonce":3807770630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5089,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1995976641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1995976641},{"type":"v1","addr":"172.21.15.45:6835","nonce":1995976641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5622,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3505357466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3505357466},{"type":"v1","addr":"172.21.15.45:6834","nonce":3505357466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-10-06T05:17:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c0066f30 msgr2=0x7f12c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c0066f30 0x7f12c00693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12cc003080 tx=0x7f12cc016000 comp rx=0 tx=0).stop 2023-10-06T05:17:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12dc156ce0 msgr2=0x7f12dc169480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12dc156ce0 0x7f12dc169480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f12d800cb90 tx=0x7f12d8001a30 comp rx=0 tx=0).stop 2023-10-06T05:17:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 shutdown_connections 2023-10-06T05:17:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c0066f30 0x7f12c00693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12dc15ac10 0x7f12dc155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12dc156ce0 0x7f12dc169480 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 --2- 172.21.15.45:0/912049175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12dc1570b0 0x7f12dc0c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 >> 172.21.15.45:0/912049175 conn(0x7f12dc06f090 msgr2=0x7f12dc0abde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:26.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 shutdown_connections 2023-10-06T05:17:26.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.169+0000 7f12e449b700 1 -- 172.21.15.45:0/912049175 wait complete. 2023-10-06T05:17:26.173 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-10-06T05:17:26.184 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:17:26.186 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:17:26.186 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:17:26.186 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:26.186 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:17:26.187 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:17:26.187 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:17:26.187 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:26.187 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:17:26.187 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:17:26.187 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:17:26.187 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:26.188 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:17:26.188 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:17:26.188 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:17:26.188 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:26.196 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:17:26.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:17:26.197 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:17:26.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:17:26.199 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:17:26.199 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:17:26.200 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:17:26.200 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:17:26.239 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:17:26.239 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:26.240 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:17:26.240 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:26.242 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:17:26.242 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:26.243 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:17:26.243 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:26.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:17:26.326 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:17:26.327 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:17:26.330 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:17:26.330 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.382+0000 7f0c6cf7d700 1 Processor -- start 2023-10-06T05:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.382+0000 7f0c6cf7d700 1 -- start start 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6cf7d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c68159170 0x7f0c68159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6cf7d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c6815d040 0x7f0c68068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6cf7d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c680ca6d0 con 0x7f0c68069b90 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6cf7d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6815d550 con 0x7f0c68159170 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6cf7d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6815d810 con 0x7f0c6815d040 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c68159170 0x7f0c68159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c68159170 0x7f0c68159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49746/0 (socket says 172.21.15.45:49746) 2023-10-06T05:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6659c700 1 -- 172.21.15.45:0/3724961972 learned_addr learned my addr 172.21.15.45:0/3724961972 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c65d9b700 1 --2- 172.21.15.45:0/3724961972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c6815d040 0x7f0c68068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6659c700 1 -- 172.21.15.45:0/3724961972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c6815d040 msgr2=0x7f0c68068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6659c700 1 --2- 172.21.15.45:0/3724961972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c6815d040 0x7f0c68068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.383+0000 7f0c6659c700 1 -- 172.21.15.45:0/3724961972 >> v1:172.21.15.144:6789/0 conn(0x7f0c68069b90 legacy=0x7f0c680aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.384+0000 7f0c6659c700 1 -- 172.21.15.45:0/3724961972 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6815daf0 con 0x7f0c68159170 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.384+0000 7f0c6659c700 1 --2- 172.21.15.45:0/3724961972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c68159540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0c580099b0 tx=0x7f0c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5295dc4a0f0ce6b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.384+0000 7f0c6559a700 1 -- 172.21.15.45:0/3724961972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58009ea0 con 0x7f0c68159170 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.384+0000 7f0c6559a700 1 -- 172.21.15.45:0/3724961972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5800b860 con 0x7f0c68159170 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.384+0000 7f0c6559a700 1 -- 172.21.15.45:0/3724961972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800bc00 con 0x7f0c68159170 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.384+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/3724961972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 msgr2=0x7f0c68159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.384+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/3724961972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c68159540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0c580099b0 tx=0x7f0c58009250 comp rx=0 tx=0).stop 2023-10-06T05:17:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.385+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/3724961972 shutdown_connections 2023-10-06T05:17:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.385+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/3724961972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c68159540 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.385+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/3724961972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c6815d040 0x7f0c68068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.385+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/3724961972 >> 172.21.15.45:0/3724961972 conn(0x7f0c6806f7c0 msgr2=0x7f0c680d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.385+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/3724961972 shutdown_connections 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.385+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/3724961972 wait complete. 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.386+0000 7f0c6cf7d700 1 Processor -- start 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.386+0000 7f0c6cf7d700 1 -- start start 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6cf7d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c68069b90 0x7f0c6814c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6cf7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c680b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6cf7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c6815d040 0x7f0c680b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6cf7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c680c93b0 con 0x7f0c68159170 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6cf7d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c680c8f20 con 0x7f0c6815d040 2023-10-06T05:17:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6cf7d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c680c8310 con 0x7f0c68069b90 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c680b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c680b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49750/0 (socket says 172.21.15.45:49750) 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 -- 172.21.15.45:0/2624851155 learned_addr learned my addr 172.21.15.45:0/2624851155 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6659c700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c68069b90 0x7f0c6814c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c66d9d700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c6815d040 0x7f0c680b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 -- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c68069b90 msgr2=0x7f0c6814c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c68069b90 0x7f0c6814c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 -- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c6815d040 msgr2=0x7f0c680b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c6815d040 0x7f0c680b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 -- 172.21.15.45:0/2624851155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580094f0 con 0x7f0c68159170 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c65d9b700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c680b3880 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000e930 tx=0x7f0c500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c5e7fc700 1 -- 172.21.15.45:0/2624851155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000eed0 con 0x7f0c68159170 2023-10-06T05:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c5e7fc700 1 -- 172.21.15.45:0/2624851155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5000c850 con 0x7f0c68159170 2023-10-06T05:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.387+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c680c85f0 con 0x7f0c68159170 2023-10-06T05:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.388+0000 7f0c5e7fc700 1 -- 172.21.15.45:0/2624851155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000cbf0 con 0x7f0c68159170 2023-10-06T05:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.388+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c680c8bb0 con 0x7f0c68159170 2023-10-06T05:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.389+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c68063b50 con 0x7f0c68159170 2023-10-06T05:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.389+0000 7f0c5e7fc700 1 -- 172.21.15.45:0/2624851155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c50013470 con 0x7f0c68159170 2023-10-06T05:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.389+0000 7f0c5e7fc700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.389+0000 7f0c5e7fc700 1 -- 172.21.15.45:0/2624851155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f0c50084790 con 0x7f0c68159170 2023-10-06T05:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.390+0000 7f0c6659c700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.390+0000 7f0c6659c700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c48066f30 0x7f0c480693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c580099b0 tx=0x7f0c58017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.394+0000 7f0c5e7fc700 1 -- 172.21.15.45:0/2624851155 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c50051960 con 0x7f0c68159170 2023-10-06T05:17:26.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:26.609+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0c6815ddd0 con 0x7f0c68159170 2023-10-06T05:17:27.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.292+0000 7f0c5e7fc700 1 -- 172.21.15.45:0/2624851155 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0c50054f80 con 0x7f0c68159170 2023-10-06T05:17:27.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c48066f30 msgr2=0x7f0c480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c48066f30 0x7f0c480693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c580099b0 tx=0x7f0c58017000 comp rx=0 tx=0).stop 2023-10-06T05:17:27.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 msgr2=0x7f0c680b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c680b3880 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000e930 tx=0x7f0c500091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 shutdown_connections 2023-10-06T05:17:27.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0c48066f30 0x7f0c480693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c6815d040 0x7f0c680b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c68069b90 0x7f0c6814c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 --2- 172.21.15.45:0/2624851155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c68159170 0x7f0c680b3880 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 >> 172.21.15.45:0/2624851155 conn(0x7f0c6806f7c0 msgr2=0x7f0c680cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:27.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 shutdown_connections 2023-10-06T05:17:27.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.296+0000 7f0c6cf7d700 1 -- 172.21.15.45:0/2624851155 wait complete. 2023-10-06T05:17:27.299 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-10-06T05:17:27.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.448+0000 7f08c4052700 1 Processor -- start 2023-10-06T05:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.448+0000 7f08c4052700 1 -- start start 2023-10-06T05:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c4052700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08bc0cd3f0 0x7f08bc06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c4052700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc15b380 con 0x7f08bc156ce0 2023-10-06T05:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c4052700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc15b660 con 0x7f08bc1570b0 2023-10-06T05:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c4052700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc06c010 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c25ef700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08bc0cd3f0 0x7f08bc06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c1dee700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f08bc156ce0 0x7f08bc0afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45734/0 (socket says 172.21.15.45:45734) 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c1dee700 1 -- 172.21.15.45:0/1309297605 learned_addr learned my addr 172.21.15.45:0/1309297605 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c25ef700 1 -- 172.21.15.45:0/1309297605 >> v1:172.21.15.144:6789/0 conn(0x7f08bc1570b0 legacy=0x7f08bc0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c25ef700 1 -- 172.21.15.45:0/1309297605 >> v1:172.21.15.45:6789/0 conn(0x7f08bc156ce0 legacy=0x7f08bc0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c25ef700 1 -- 172.21.15.45:0/1309297605 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc15b940 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.449+0000 7f08c25ef700 1 --2- 172.21.15.45:0/1309297605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc06bad0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e60 tx=0x7f08b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e7e3df4aa654e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c0dec700 1 -- 172.21.15.45:0/1309297605 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b400e070 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c0dec700 1 -- 172.21.15.45:0/1309297605 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b400bc30 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c0dec700 1 -- 172.21.15.45:0/1309297605 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4012430 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c4052700 1 -- 172.21.15.45:0/1309297605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 msgr2=0x7f08bc06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c4052700 1 --2- 172.21.15.45:0/1309297605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc06bad0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e60 tx=0x7f08b4009780 comp rx=0 tx=0).stop 2023-10-06T05:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c4052700 1 -- 172.21.15.45:0/1309297605 shutdown_connections 2023-10-06T05:17:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c4052700 1 --2- 172.21.15.45:0/1309297605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc06bad0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c4052700 1 -- 172.21.15.45:0/1309297605 >> 172.21.15.45:0/1309297605 conn(0x7f08bc06f090 msgr2=0x7f08bc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c4052700 1 -- 172.21.15.45:0/1309297605 shutdown_connections 2023-10-06T05:17:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.450+0000 7f08c4052700 1 -- 172.21.15.45:0/1309297605 wait complete. 2023-10-06T05:17:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 Processor -- start 2023-10-06T05:17:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 -- start start 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc168620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08bc156ce0 0x7f08bc142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08bc1570b0 0x7f08bc167450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0c6300 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0c4100 con 0x7f08bc156ce0 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.452+0000 7f08c4052700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc168030 con 0x7f08bc1570b0 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc168620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc168620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49782/0 (socket says 172.21.15.45:49782) 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 -- 172.21.15.45:0/1878538064 learned_addr learned my addr 172.21.15.45:0/1878538064 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c25ef700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08bc1570b0 0x7f08bc167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c15ed700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08bc156ce0 0x7f08bc142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 -- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08bc1570b0 msgr2=0x7f08bc167450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08bc1570b0 0x7f08bc167450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 -- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08bc156ce0 msgr2=0x7f08bc142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08bc156ce0 0x7f08bc142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 -- 172.21.15.45:0/1878538064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009a20 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c25ef700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08bc1570b0 0x7f08bc167450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c15ed700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08bc156ce0 0x7f08bc142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c1dee700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc168620 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f08b800c7f0 tx=0x7f08b8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08b1ffb700 1 -- 172.21.15.45:0/1878538064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8005b30 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08b1ffb700 1 -- 172.21.15.45:0/1878538064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b8005cb0 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc168370 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.453+0000 7f08b1ffb700 1 -- 172.21.15.45:0/1878538064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004a80 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.454+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc168b60 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.455+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc0c6130 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.455+0000 7f08b1ffb700 1 -- 172.21.15.45:0/1878538064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08b80070f0 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.455+0000 7f08b1ffb700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08a0066f30 0x7f08a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.455+0000 7f08b1ffb700 1 -- 172.21.15.45:0/1878538064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f08b8083900 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.456+0000 7f08c15ed700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08a0066f30 0x7f08a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.456+0000 7f08c15ed700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08a0066f30 0x7f08a00693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0c33a0 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:27.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.460+0000 7f08b1ffb700 1 -- 172.21.15.45:0/1878538064 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08b8050ca0 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.743+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f08bc04bb40 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.743+0000 7f08b1ffb700 1 -- 172.21.15.45:0/1878538064 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f08b8018030 con 0x7f08bc0cd3f0 2023-10-06T05:17:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw==", 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w==", 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg==", 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw==", 2023-10-06T05:17:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg==", 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg==", 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w==", 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw==", 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA==", 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ==", 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw==", 2023-10-06T05:17:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA==", 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:17:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw==", 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-10-06T05:17:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw==", 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g==", 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w==", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:17:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l/0VtGRAA99fxS1gieVbRro7K6P1/lw==", 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A==", 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg==", 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:17:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:17:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:17:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-06T05:17:27.760 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:17:27.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08a0066f30 msgr2=0x7f08a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08a0066f30 0x7f08a00693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0c33a0 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 msgr2=0x7f08bc168620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc168620 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f08b800c7f0 tx=0x7f08b8007f70 comp rx=0 tx=0).stop 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 shutdown_connections 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08a0066f30 0x7f08a00693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08bc1570b0 0x7f08bc167450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08bc0cd3f0 0x7f08bc168620 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 --2- 172.21.15.45:0/1878538064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08bc156ce0 0x7f08bc142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 >> 172.21.15.45:0/1878538064 conn(0x7f08bc06f090 msgr2=0x7f08bc0c3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 shutdown_connections 2023-10-06T05:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.746+0000 7f08c4052700 1 -- 172.21.15.45:0/1878538064 wait complete. 2023-10-06T05:17:27.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.895+0000 7f2a7876f700 1 Processor -- start 2023-10-06T05:17:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.895+0000 7f2a7876f700 1 -- start start 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.896+0000 7f2a7876f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2a700c8ff0 0x7f2a700c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.896+0000 7f2a7876f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a700caad0 0x7f2a700caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.896+0000 7f2a7876f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7015b000 con 0x7f2a70156af0 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.896+0000 7f2a7876f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a70157630 con 0x7f2a700caad0 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.896+0000 7f2a7876f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7015b2e0 con 0x7f2a700c8ff0 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a700caad0 0x7f2a700caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a700caad0 0x7f2a700caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49796/0 (socket says 172.21.15.45:49796) 2023-10-06T05:17:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 -- 172.21.15.45:0/2877711683 learned_addr learned my addr 172.21.15.45:0/2877711683 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a7650b700 1 --2- 172.21.15.45:0/2877711683 >> v2:172.21.15.144:3300/0 conn(0x7f2a700c8ff0 0x7f2a700c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 -- 172.21.15.45:0/2877711683 >> v2:172.21.15.144:3300/0 conn(0x7f2a700c8ff0 msgr2=0x7f2a700c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 --2- 172.21.15.45:0/2877711683 >> v2:172.21.15.144:3300/0 conn(0x7f2a700c8ff0 0x7f2a700c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 -- 172.21.15.45:0/2877711683 >> v1:172.21.15.144:6789/0 conn(0x7f2a70156af0 legacy=0x7f2a70156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 -- 172.21.15.45:0/2877711683 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7015b5c0 con 0x7f2a700caad0 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a7650b700 1 --2- 172.21.15.45:0/2877711683 >> v2:172.21.15.144:3300/0 conn(0x7f2a700c8ff0 0x7f2a700c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75d0a700 1 --2- 172.21.15.45:0/2877711683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a700caf40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2a700cefc0 tx=0x7f2a60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca3cd09f948c54a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75509700 1 -- 172.21.15.45:0/2877711683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000b860 con 0x7f2a700caad0 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75509700 1 -- 172.21.15.45:0/2877711683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6000b9c0 con 0x7f2a700caad0 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.897+0000 7f2a75509700 1 -- 172.21.15.45:0/2877711683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000bd60 con 0x7f2a700caad0 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 -- 172.21.15.45:0/2877711683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 msgr2=0x7f2a700caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 --2- 172.21.15.45:0/2877711683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a700caf40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2a700cefc0 tx=0x7f2a60009250 comp rx=0 tx=0).stop 2023-10-06T05:17:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 -- 172.21.15.45:0/2877711683 shutdown_connections 2023-10-06T05:17:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 --2- 172.21.15.45:0/2877711683 >> v2:172.21.15.144:3300/0 conn(0x7f2a700c8ff0 0x7f2a700c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 --2- 172.21.15.45:0/2877711683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a700caf40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 -- 172.21.15.45:0/2877711683 >> 172.21.15.45:0/2877711683 conn(0x7f2a700cdf50 msgr2=0x7f2a700ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 -- 172.21.15.45:0/2877711683 shutdown_connections 2023-10-06T05:17:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.898+0000 7f2a7876f700 1 -- 172.21.15.45:0/2877711683 wait complete. 2023-10-06T05:17:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.899+0000 7f2a7876f700 1 Processor -- start 2023-10-06T05:17:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.899+0000 7f2a7876f700 1 -- start start 2023-10-06T05:17:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a7876f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a700c8ff0 0x7f2a70142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a7876f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a70157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a7876f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a70156af0 0x7f2a70159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a7876f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70158e50 con 0x7f2a700caad0 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a7876f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70158240 con 0x7f2a700c8ff0 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a7876f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70158520 con 0x7f2a70156af0 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a70157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a70157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49800/0 (socket says 172.21.15.45:49800) 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 -- 172.21.15.45:0/848696353 learned_addr learned my addr 172.21.15.45:0/848696353 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a7650b700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a700c8ff0 0x7f2a70142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a76d0c700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a70156af0 0x7f2a70159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 -- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a70156af0 msgr2=0x7f2a70159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a70156af0 0x7f2a70159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 -- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a700c8ff0 msgr2=0x7f2a70142930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a700c8ff0 0x7f2a70142930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.900+0000 7f2a75d0a700 1 -- 172.21.15.45:0/848696353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600094f0 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.901+0000 7f2a75d0a700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a70157780 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009290 tx=0x7f2a60017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.901+0000 7f2a667fc700 1 -- 172.21.15.45:0/848696353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60011e60 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.901+0000 7f2a667fc700 1 -- 172.21.15.45:0/848696353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6001f070 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.901+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a70158800 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.901+0000 7f2a667fc700 1 -- 172.21.15.45:0/848696353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6001a870 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.901+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a70142e70 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.902+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a700b0cf0 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.902+0000 7f2a667fc700 1 -- 172.21.15.45:0/848696353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a6001abe0 con 0x7f2a700caad0 2023-10-06T05:17:27.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.903+0000 7f2a667fc700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a54066f30 0x7f2a540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:27.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.903+0000 7f2a667fc700 1 -- 172.21.15.45:0/848696353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f2a60021070 con 0x7f2a700caad0 2023-10-06T05:17:27.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.903+0000 7f2a7650b700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a54066f30 0x7f2a540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:27.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.903+0000 7f2a7650b700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a54066f30 0x7f2a540693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c006f00 tx=0x7f2a6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:27.907+0000 7f2a667fc700 1 -- 172.21.15.45:0/848696353 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a700b0cf0 con 0x7f2a700caad0 2023-10-06T05:17:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.112+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a700b0cf0 con 0x7f2a700caad0 2023-10-06T05:17:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.113+0000 7f2a667fc700 1 -- 172.21.15.45:0/848696353 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13951 (secure 0 0 0) 0x7f2a700b0cf0 con 0x7f2a700caad0 2023-10-06T05:17:28.115 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:28.115 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":48,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:17:27.303780+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.115+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a54066f30 msgr2=0x7f2a540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.115+0000 7f2a7876f700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a54066f30 0x7f2a540693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c006f00 tx=0x7f2a6c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.115+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 msgr2=0x7f2a70157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.115+0000 7f2a7876f700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a70157780 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009290 tx=0x7f2a60017000 comp rx=0 tx=0).stop 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 shutdown_connections 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a54066f30 0x7f2a540693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a70156af0 0x7f2a70159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a700c8ff0 0x7f2a70142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 --2- 172.21.15.45:0/848696353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a700caad0 0x7f2a70157780 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 >> 172.21.15.45:0/848696353 conn(0x7f2a700cdf50 msgr2=0x7f2a700c4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 shutdown_connections 2023-10-06T05:17:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.116+0000 7f2a7876f700 1 -- 172.21.15.45:0/848696353 wait complete. 2023-10-06T05:17:28.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:17:28.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.264+0000 7fe53e4cf700 1 Processor -- start 2023-10-06T05:17:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.264+0000 7fe53e4cf700 1 -- start start 2023-10-06T05:17:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.265+0000 7fe53e4cf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe538155e60 0x7fe538068c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.265+0000 7fe53e4cf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe538067be0 con 0x7fe53806e720 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.265+0000 7fe53e4cf700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe538067ec0 con 0x7fe538067620 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.265+0000 7fe53e4cf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5380681a0 con 0x7fe538155e60 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.265+0000 7fe53ca6c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe53806e720 0x7fe5380c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45738/0 (socket says 172.21.15.45:45738) 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.265+0000 7fe53ca6c700 1 -- 172.21.15.45:0/4071099747 learned_addr learned my addr 172.21.15.45:0/4071099747 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe537fff700 1 --2- 172.21.15.45:0/4071099747 >> v2:172.21.15.144:3300/0 conn(0x7fe538155e60 0x7fe538068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870838450 0 0) 0x7fe538067be0 con 0x7fe53806e720 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe524003570 con 0x7fe53806e720 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401389783 0 0) 0x7fe538067ec0 con 0x7fe538067620 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe538067be0 con 0x7fe538067620 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2053316729 0 0) 0x7fe524003570 con 0x7fe53806e720 2023-10-06T05:17:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 >> v2:172.21.15.144:3300/0 conn(0x7fe538155e60 msgr2=0x7fe538068c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 --2- 172.21.15.45:0/4071099747 >> v2:172.21.15.144:3300/0 conn(0x7fe538155e60 0x7fe538068c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 >> v1:172.21.15.144:6789/0 conn(0x7fe538067620 legacy=0x7fe5380ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe538068480 con 0x7fe53806e720 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fe5300017f0 con 0x7fe53806e720 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 >> v1:172.21.15.45:6789/0 conn(0x7fe53806e720 legacy=0x7fe5380c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524003c40 0x7fe524004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524004810 0x7fe524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe524005210 0x7fe52400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380681a0 con 0x7fe524004810 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52400c130 con 0x7fe524005210 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52400c2c0 con 0x7fe524003c40 2023-10-06T05:17:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe5377fe700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524004810 0x7fe524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe537fff700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524003c40 0x7fe524004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe53ca6c700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe524005210 0x7fe52400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe5377fe700 1 -- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524003c40 msgr2=0x7fe524004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe5377fe700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524003c40 0x7fe524004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe5377fe700 1 -- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe524005210 msgr2=0x7fe52400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe5377fe700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe524005210 0x7fe52400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe5377fe700 1 -- 172.21.15.45:0/4071099747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524003570 con 0x7fe524004810 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe53ca6c700 1 -- 172.21.15.45:0/4071099747 reap_dead start 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.266+0000 7fe537fff700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524003c40 0x7fe524004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe5377fe700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524004810 0x7fe524004c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe520001a40 tx=0x7fe520007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b061b044f982eb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000c590 con 0x7fe524004810 2023-10-06T05:17:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52000c6f0 con 0x7fe524004810 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe536ffd700 1 -- 172.21.15.45:0/4071099747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000ca90 con 0x7fe524004810 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe53e4cf700 1 -- 172.21.15.45:0/4071099747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524004810 msgr2=0x7fe524004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524004810 0x7fe524004c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe520001a40 tx=0x7fe520007d30 comp rx=0 tx=0).stop 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe53e4cf700 1 -- 172.21.15.45:0/4071099747 shutdown_connections 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/4071099747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524004810 0x7fe524004c80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe53e4cf700 1 -- 172.21.15.45:0/4071099747 >> 172.21.15.45:0/4071099747 conn(0x7fe5380cc570 msgr2=0x7fe5380cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.267+0000 7fe53e4cf700 1 -- 172.21.15.45:0/4071099747 shutdown_connections 2023-10-06T05:17:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.268+0000 7fe53e4cf700 1 -- 172.21.15.45:0/4071099747 wait complete. 2023-10-06T05:17:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.269+0000 7fe53e4cf700 1 Processor -- start 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.269+0000 7fe53e4cf700 1 -- start start 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53e4cf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524004810 0x7fe5381519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53e4cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5380c1a90 0x7fe5380c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53e4cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5380c63b0 0x7fe5380c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53e4cf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380bfba0 con 0x7fe5380c63b0 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53e4cf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380c0720 con 0x7fe5380c1a90 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53e4cf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380bfdf0 con 0x7fe524004810 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5380c63b0 0x7fe5380c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5380c63b0 0x7fe5380c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49808/0 (socket says 172.21.15.45:49808) 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 -- 172.21.15.45:0/2385636740 learned_addr learned my addr 172.21.15.45:0/2385636740 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe537fff700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524004810 0x7fe5381519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe5377fe700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5380c1a90 0x7fe5380c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 -- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524004810 msgr2=0x7fe5381519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524004810 0x7fe5381519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 -- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5380c1a90 msgr2=0x7fe5380c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5380c1a90 0x7fe5380c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 -- 172.21.15.45:0/2385636740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5380c00d0 con 0x7fe5380c63b0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe53ca6c700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5380c63b0 0x7fe5380c6830 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe53806f550 tx=0x7fe5300013d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.270+0000 7fe52ffff700 1 -- 172.21.15.45:0/2385636740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530017070 con 0x7fe5380c63b0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.271+0000 7fe52ffff700 1 -- 172.21.15.45:0/2385636740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53000e930 con 0x7fe5380c63b0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.271+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5380c03b0 con 0x7fe5380c63b0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.271+0000 7fe52ffff700 1 -- 172.21.15.45:0/2385636740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530001e40 con 0x7fe5380c63b0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.271+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5380c2e20 con 0x7fe5380c63b0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.272+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5380c3010 con 0x7fe5380c63b0 2023-10-06T05:17:28.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.272+0000 7fe52ffff700 1 -- 172.21.15.45:0/2385636740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe530013420 con 0x7fe5380c63b0 2023-10-06T05:17:28.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.272+0000 7fe52ffff700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe518066f20 0x7fe5180693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.273+0000 7fe52ffff700 1 -- 172.21.15.45:0/2385636740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fe530084150 con 0x7fe5380c63b0 2023-10-06T05:17:28.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.273+0000 7fe537fff700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe518066f20 0x7fe5180693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.273+0000 7fe537fff700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe518066f20 0x7fe5180693d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe53804d290 tx=0x7fe528008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:28.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.277+0000 7fe52ffff700 1 -- 172.21.15.45:0/2385636740 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe530051470 con 0x7fe5380c63b0 2023-10-06T05:17:28.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.481+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5380c08b0 con 0x7fe5380c63b0 2023-10-06T05:17:28.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.482+0000 7fe52ffff700 1 -- 172.21.15.45:0/2385636740 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe530054a90 con 0x7fe5380c63b0 2023-10-06T05:17:28.484 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe518066f20 msgr2=0x7fe5180693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe518066f20 0x7fe5180693d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe53804d290 tx=0x7fe528008040 comp rx=0 tx=0).stop 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5380c63b0 msgr2=0x7fe5380c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5380c63b0 0x7fe5380c6830 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe53806f550 tx=0x7fe5300013d0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 shutdown_connections 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe518066f20 0x7fe5180693d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5380c63b0 0x7fe5380c6830 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe524004810 0x7fe5381519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 --2- 172.21.15.45:0/2385636740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5380c1a90 0x7fe5380c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 >> 172.21.15.45:0/2385636740 conn(0x7fe5380cc570 msgr2=0x7fe538158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 shutdown_connections 2023-10-06T05:17:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.484+0000 7fe53e4cf700 1 -- 172.21.15.45:0/2385636740 wait complete. 2023-10-06T05:17:28.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:28.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.633+0000 7f60434b4700 1 Processor -- start 2023-10-06T05:17:28.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.633+0000 7f60434b4700 1 -- start start 2023-10-06T05:17:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f60434b4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f603c0c8b70 0x7f603c0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f60434b4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c0c3b50 con 0x7f603c0ccb70 2023-10-06T05:17:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f60434b4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c157700 con 0x7f603c157330 2023-10-06T05:17:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f60434b4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c0c98e0 con 0x7f603c0c8b70 2023-10-06T05:17:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f6041a51700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f603c0c8b70 0x7f603c0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f6040a4f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f603c0ccb70 0x7f603c0aaae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45744/0 (socket says 172.21.15.45:45744) 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f6040a4f700 1 -- 172.21.15.45:0/3865294665 learned_addr learned my addr 172.21.15.45:0/3865294665 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f6041a51700 1 -- 172.21.15.45:0/3865294665 >> v1:172.21.15.144:6789/0 conn(0x7f603c157330 legacy=0x7f603c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f6041a51700 1 -- 172.21.15.45:0/3865294665 >> v1:172.21.15.45:6789/0 conn(0x7f603c0ccb70 legacy=0x7f603c0aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f6041a51700 1 -- 172.21.15.45:0/3865294665 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0c9bc0 con 0x7f603c0c8b70 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.634+0000 7f6033fff700 1 -- 172.21.15.45:0/3865294665 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113301919 0 0) 0x7f603c0c3b50 con 0x7f603c0ccb70 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f6041a51700 1 --2- 172.21.15.45:0/3865294665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c0c8b70 0x7f603c0c8f90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6034009e60 tx=0x7f6034009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9058667675fea15c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f6033fff700 1 -- 172.21.15.45:0/3865294665 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400e070 con 0x7f603c0c8b70 2023-10-06T05:17:28.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f6033fff700 1 -- 172.21.15.45:0/3865294665 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603400bbf0 con 0x7f603c0c8b70 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f6033fff700 1 -- 172.21.15.45:0/3865294665 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034012430 con 0x7f603c0c8b70 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f60434b4700 1 -- 172.21.15.45:0/3865294665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c0c8b70 msgr2=0x7f603c0c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f60434b4700 1 --2- 172.21.15.45:0/3865294665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c0c8b70 0x7f603c0c8f90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6034009e60 tx=0x7f6034009780 comp rx=0 tx=0).stop 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f60434b4700 1 -- 172.21.15.45:0/3865294665 shutdown_connections 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f60434b4700 1 --2- 172.21.15.45:0/3865294665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c0c8b70 0x7f603c0c8f90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f60434b4700 1 -- 172.21.15.45:0/3865294665 >> 172.21.15.45:0/3865294665 conn(0x7f603c067e00 msgr2=0x7f603c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f60434b4700 1 -- 172.21.15.45:0/3865294665 shutdown_connections 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.635+0000 7f60434b4700 1 -- 172.21.15.45:0/3865294665 wait complete. 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.637+0000 7f60434b4700 1 Processor -- start 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.637+0000 7f60434b4700 1 -- start start 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f60434b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0c8b70 0x7f603c152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f60434b4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c0ccb70 0x7f603c157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f60434b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c157330 0x7f603c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f60434b4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0ada40 con 0x7f603c157330 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f60434b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c158b30 con 0x7f603c0c8b70 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f60434b4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0add00 con 0x7f603c0ccb70 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c157330 0x7f603c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c157330 0x7f603c0ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49814/0 (socket says 172.21.15.45:49814) 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 -- 172.21.15.45:0/2141695692 learned_addr learned my addr 172.21.15.45:0/2141695692 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041250700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0c8b70 0x7f603c152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6040a4f700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c0ccb70 0x7f603c157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 -- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c0ccb70 msgr2=0x7f603c157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c0ccb70 0x7f603c157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 -- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0c8b70 msgr2=0x7f603c152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0c8b70 0x7f603c152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 -- 172.21.15.45:0/2141695692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034009a20 con 0x7f603c157330 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041250700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0c8b70 0x7f603c152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f6041a51700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c157330 0x7f603c0ce330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f603400b6c0 tx=0x7f6034018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.638+0000 7f60317fa700 1 -- 172.21.15.45:0/2141695692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603401e710 con 0x7f603c157330 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.639+0000 7f60317fa700 1 -- 172.21.15.45:0/2141695692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603401e870 con 0x7f603c157330 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.639+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c152c70 con 0x7f603c157330 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.639+0000 7f60317fa700 1 -- 172.21.15.45:0/2141695692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603401ec30 con 0x7f603c157330 2023-10-06T05:17:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.639+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c1585f0 con 0x7f603c157330 2023-10-06T05:17:28.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.640+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c06e350 con 0x7f603c157330 2023-10-06T05:17:28.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.640+0000 7f60317fa700 1 -- 172.21.15.45:0/2141695692 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6034024070 con 0x7f603c157330 2023-10-06T05:17:28.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.640+0000 7f60317fa700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6020066f30 0x7f60200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.641+0000 7f60317fa700 1 -- 172.21.15.45:0/2141695692 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f6034086f10 con 0x7f603c157330 2023-10-06T05:17:28.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.641+0000 7f6041250700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6020066f30 0x7f60200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.641+0000 7f6041250700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6020066f30 0x7f60200693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6038002930 tx=0x7f6038009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:28.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.645+0000 7f60317fa700 1 -- 172.21.15.45:0/2141695692 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6034054230 con 0x7f603c157330 2023-10-06T05:17:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.845+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f603c0b3070 con 0x7f603c157330 2023-10-06T05:17:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.845+0000 7f60317fa700 1 -- 172.21.15.45:0/2141695692 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6034057850 con 0x7f603c157330 2023-10-06T05:17:28.848 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:17:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6020066f30 msgr2=0x7f60200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6020066f30 0x7f60200693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6038002930 tx=0x7f6038009040 comp rx=0 tx=0).stop 2023-10-06T05:17:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c157330 msgr2=0x7f603c0ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c157330 0x7f603c0ce330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f603400b6c0 tx=0x7f6034018000 comp rx=0 tx=0).stop 2023-10-06T05:17:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 shutdown_connections 2023-10-06T05:17:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6020066f30 0x7f60200693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f603c157330 0x7f603c0ce330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0c8b70 0x7f603c152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 --2- 172.21.15.45:0/2141695692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c0ccb70 0x7f603c157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 >> 172.21.15.45:0/2141695692 conn(0x7f603c067e00 msgr2=0x7f603c0c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 shutdown_connections 2023-10-06T05:17:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.848+0000 7f60434b4700 1 -- 172.21.15.45:0/2141695692 wait complete. 2023-10-06T05:17:28.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:17:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.993+0000 7fcc9b642700 1 Processor -- start 2023-10-06T05:17:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.993+0000 7fcc9b642700 1 -- start start 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.994+0000 7fcc9b642700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc940c93c0 0x7fcc94156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.994+0000 7fcc9b642700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9415b000 con 0x7fcc94157030 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.994+0000 7fcc9b642700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9415b2e0 con 0x7fcc940c8ff0 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.994+0000 7fcc9b642700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc940c9830 con 0x7fcc940c93c0 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.994+0000 7fcc99bdf700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcc94157030 0x7fcc940cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45758/0 (socket says 172.21.15.45:45758) 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.994+0000 7fcc99bdf700 1 -- 172.21.15.45:0/2983895307 learned_addr learned my addr 172.21.15.45:0/2983895307 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.994+0000 7fcc98bdd700 1 --2- 172.21.15.45:0/2983895307 >> v2:172.21.15.144:3300/0 conn(0x7fcc940c93c0 0x7fcc94156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc8bfff700 1 -- 172.21.15.45:0/2983895307 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621420147 0 0) 0x7fcc9415b000 con 0x7fcc94157030 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc8bfff700 1 -- 172.21.15.45:0/2983895307 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc80003570 con 0x7fcc94157030 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc98bdd700 1 -- 172.21.15.45:0/2983895307 >> v1:172.21.15.144:6789/0 conn(0x7fcc940c8ff0 legacy=0x7fcc940caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc8bfff700 1 -- 172.21.15.45:0/2983895307 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1458811811 0 0) 0x7fcc80003570 con 0x7fcc94157030 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc98bdd700 1 -- 172.21.15.45:0/2983895307 >> v1:172.21.15.45:6789/0 conn(0x7fcc94157030 legacy=0x7fcc940cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc98bdd700 1 -- 172.21.15.45:0/2983895307 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9415b5c0 con 0x7fcc940c93c0 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc98bdd700 1 --2- 172.21.15.45:0/2983895307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940c93c0 0x7fcc94156af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc840098e0 tx=0x7fcc840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b3c91b8fe147a20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc8bfff700 1 -- 172.21.15.45:0/2983895307 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0017f0 con 0x7fcc940c93c0 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc8bfff700 1 -- 172.21.15.45:0/2983895307 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84011420 con 0x7fcc940c93c0 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.995+0000 7fcc8bfff700 1 -- 172.21.15.45:0/2983895307 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc840117c0 con 0x7fcc940c93c0 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.996+0000 7fcc9b642700 1 -- 172.21.15.45:0/2983895307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940c93c0 msgr2=0x7fcc94156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.996+0000 7fcc9b642700 1 --2- 172.21.15.45:0/2983895307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940c93c0 0x7fcc94156af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc840098e0 tx=0x7fcc840091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.996+0000 7fcc9b642700 1 -- 172.21.15.45:0/2983895307 shutdown_connections 2023-10-06T05:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.996+0000 7fcc9b642700 1 --2- 172.21.15.45:0/2983895307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940c93c0 0x7fcc94156af0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.996+0000 7fcc9b642700 1 -- 172.21.15.45:0/2983895307 >> 172.21.15.45:0/2983895307 conn(0x7fcc940cdf50 msgr2=0x7fcc940ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.996+0000 7fcc9b642700 1 -- 172.21.15.45:0/2983895307 shutdown_connections 2023-10-06T05:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.996+0000 7fcc9b642700 1 -- 172.21.15.45:0/2983895307 wait complete. 2023-10-06T05:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.997+0000 7fcc9b642700 1 Processor -- start 2023-10-06T05:17:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.997+0000 7fcc9b642700 1 -- start start 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc9b642700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc940c8ff0 0x7fcc940c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc9b642700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc940c93c0 0x7fcc940c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc9b642700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc94157030 0x7fcc9416d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc9b642700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94144dd0 con 0x7fcc940c93c0 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc9b642700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940ae570 con 0x7fcc94157030 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc9b642700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94144890 con 0x7fcc940c8ff0 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc98bdd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc940c93c0 0x7fcc940c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc98bdd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc940c93c0 0x7fcc940c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49816/0 (socket says 172.21.15.45:49816) 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.998+0000 7fcc98bdd700 1 -- 172.21.15.45:0/3859328792 learned_addr learned my addr 172.21.15.45:0/3859328792 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc993de700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc940c8ff0 0x7fcc940c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc99bdf700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc94157030 0x7fcc9416d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc98bdd700 1 -- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc940c8ff0 msgr2=0x7fcc940c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc98bdd700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc940c8ff0 0x7fcc940c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc98bdd700 1 -- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc94157030 msgr2=0x7fcc9416d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc98bdd700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc94157030 0x7fcc9416d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc98bdd700 1 -- 172.21.15.45:0/3859328792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84009420 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc98bdd700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc940c93c0 0x7fcc940c7700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcc940c4f80 tx=0x7fcc84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc897fa700 1 -- 172.21.15.45:0/3859328792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401cc90 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc897fa700 1 -- 172.21.15.45:0/3859328792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8401ce10 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc94146b50 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc897fa700 1 -- 172.21.15.45:0/3859328792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401a6a0 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:28.999+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc940ae200 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.000+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc940add80 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.001+0000 7fcc897fa700 1 -- 172.21.15.45:0/3859328792 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc8401aa10 con 0x7fcc940c93c0 2023-10-06T05:17:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.001+0000 7fcc897fa700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc78066f30 0x7fcc780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.001+0000 7fcc897fa700 1 -- 172.21.15.45:0/3859328792 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fcc84085f80 con 0x7fcc940c93c0 2023-10-06T05:17:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.001+0000 7fcc993de700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc78066f30 0x7fcc780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.002+0000 7fcc993de700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc78066f30 0x7fcc780693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcc900037c0 tx=0x7fcc90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.005+0000 7fcc897fa700 1 -- 172.21.15.45:0/3859328792 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc840b75a0 con 0x7fcc940c93c0 2023-10-06T05:17:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.212+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcc940ae6e0 con 0x7fcc940c93c0 2023-10-06T05:17:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.212+0000 7fcc897fa700 1 -- 172.21.15.45:0/3859328792 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+126 (secure 0 0 0) 0x7fcc84029090 con 0x7fcc940c93c0 2023-10-06T05:17:29.214 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:29.215 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:17:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.214+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc78066f30 msgr2=0x7fcc780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.214+0000 7fcc9b642700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc78066f30 0x7fcc780693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcc900037c0 tx=0x7fcc90009040 comp rx=0 tx=0).stop 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.214+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc940c93c0 msgr2=0x7fcc940c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.214+0000 7fcc9b642700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc940c93c0 0x7fcc940c7700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcc940c4f80 tx=0x7fcc84016000 comp rx=0 tx=0).stop 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 shutdown_connections 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc78066f30 0x7fcc780693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc94157030 0x7fcc9416d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc940c8ff0 0x7fcc940c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 --2- 172.21.15.45:0/3859328792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc940c93c0 0x7fcc940c7700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 >> 172.21.15.45:0/3859328792 conn(0x7fcc940cdf50 msgr2=0x7fcc94147c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 shutdown_connections 2023-10-06T05:17:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.215+0000 7fcc9b642700 1 -- 172.21.15.45:0/3859328792 wait complete. 2023-10-06T05:17:29.230 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:17:29.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:17:29.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.366+0000 7f7b704e4700 1 Processor -- start 2023-10-06T05:17:29.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.366+0000 7f7b704e4700 1 -- start start 2023-10-06T05:17:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.367+0000 7f7b704e4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680cd590 0x7f7b680ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.367+0000 7f7b704e4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b680caa20 con 0x7f7b6813fa30 2023-10-06T05:17:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.367+0000 7f7b704e4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b680c93d0 con 0x7f7b680ccb50 2023-10-06T05:17:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.367+0000 7f7b704e4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680c96b0 con 0x7f7b680cd590 2023-10-06T05:17:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.367+0000 7f7b6ea81700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7b6813fa30 0x7f7b680caeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45762/0 (socket says 172.21.15.45:45762) 2023-10-06T05:17:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.367+0000 7f7b6ea81700 1 -- 172.21.15.45:0/236546243 learned_addr learned my addr 172.21.15.45:0/236546243 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6da7f700 1 --2- 172.21.15.45:0/236546243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680cd590 0x7f7b680ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6d27e700 1 -- 172.21.15.45:0/236546243 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855215035 0 0) 0x7f7b680caa20 con 0x7f7b6813fa30 2023-10-06T05:17:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6d27e700 1 -- 172.21.15.45:0/236546243 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b54003570 con 0x7f7b6813fa30 2023-10-06T05:17:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6d27e700 1 -- 172.21.15.45:0/236546243 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038116982 0 0) 0x7f7b680c93d0 con 0x7f7b680ccb50 2023-10-06T05:17:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6d27e700 1 -- 172.21.15.45:0/236546243 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b680caa20 con 0x7f7b680ccb50 2023-10-06T05:17:29.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6da7f700 1 -- 172.21.15.45:0/236546243 >> v1:172.21.15.144:6789/0 conn(0x7f7b680ccb50 legacy=0x7f7b6813f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6da7f700 1 -- 172.21.15.45:0/236546243 >> v1:172.21.15.45:6789/0 conn(0x7f7b6813fa30 legacy=0x7f7b680caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6da7f700 1 -- 172.21.15.45:0/236546243 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680c7630 con 0x7f7b680cd590 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6da7f700 1 --2- 172.21.15.45:0/236546243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680cd590 0x7f7b680ccf20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009910 tx=0x7f7b640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c30e086e906bd10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6d27e700 1 -- 172.21.15.45:0/236546243 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400b690 con 0x7f7b680cd590 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6d27e700 1 -- 172.21.15.45:0/236546243 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6400b7f0 con 0x7f7b680cd590 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.368+0000 7f7b6d27e700 1 -- 172.21.15.45:0/236546243 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400bb90 con 0x7f7b680cd590 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.369+0000 7f7b704e4700 1 -- 172.21.15.45:0/236546243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680cd590 msgr2=0x7f7b680ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.369+0000 7f7b704e4700 1 --2- 172.21.15.45:0/236546243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680cd590 0x7f7b680ccf20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009910 tx=0x7f7b640091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.369+0000 7f7b704e4700 1 -- 172.21.15.45:0/236546243 shutdown_connections 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.369+0000 7f7b704e4700 1 --2- 172.21.15.45:0/236546243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680cd590 0x7f7b680ccf20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.369+0000 7f7b704e4700 1 -- 172.21.15.45:0/236546243 >> 172.21.15.45:0/236546243 conn(0x7f7b68068730 msgr2=0x7f7b68068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.369+0000 7f7b704e4700 1 -- 172.21.15.45:0/236546243 shutdown_connections 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.369+0000 7f7b704e4700 1 -- 172.21.15.45:0/236546243 wait complete. 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.370+0000 7f7b704e4700 1 Processor -- start 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.370+0000 7f7b704e4700 1 -- start start 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b704e4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680ccb50 0x7f7b680c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b704e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b680cd590 0x7f7b68169730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b704e4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b6813fa30 0x7f7b6816d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b704e4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68169f80 con 0x7f7b6813fa30 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b704e4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6816a260 con 0x7f7b680cd590 2023-10-06T05:17:29.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b704e4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6816a540 con 0x7f7b680ccb50 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b6ea81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b6813fa30 0x7f7b6816d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b6ea81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b6813fa30 0x7f7b6816d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49818/0 (socket says 172.21.15.45:49818) 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b6ea81700 1 -- 172.21.15.45:0/1240311649 learned_addr learned my addr 172.21.15.45:0/1240311649 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b6da7f700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b680cd590 0x7f7b68169730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.371+0000 7f7b6e280700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680ccb50 0x7f7b680c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b6ea81700 1 -- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680ccb50 msgr2=0x7f7b680c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b6ea81700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680ccb50 0x7f7b680c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b6ea81700 1 -- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b680cd590 msgr2=0x7f7b68169730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b6ea81700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b680cd590 0x7f7b68169730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b6ea81700 1 -- 172.21.15.45:0/1240311649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009420 con 0x7f7b6813fa30 2023-10-06T05:17:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b6da7f700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b680cd590 0x7f7b68169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b6ea81700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b6813fa30 0x7f7b6816d510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000cc20 tx=0x7f7b60001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b5e7fc700 1 -- 172.21.15.45:0/1240311649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000a960 con 0x7f7b6813fa30 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b5e7fc700 1 -- 172.21.15.45:0/1240311649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000aac0 con 0x7f7b6813fa30 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6816cd10 con 0x7f7b6813fa30 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b5e7fc700 1 -- 172.21.15.45:0/1240311649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000ae80 con 0x7f7b6813fa30 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.372+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b68169c70 con 0x7f7b6813fa30 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.373+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6804c380 con 0x7f7b6813fa30 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.373+0000 7f7b5e7fc700 1 -- 172.21.15.45:0/1240311649 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b60007300 con 0x7f7b6813fa30 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.374+0000 7f7b5e7fc700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b4c066f30 0x7f7b4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.374+0000 7f7b5e7fc700 1 -- 172.21.15.45:0/1240311649 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f7b60084850 con 0x7f7b6813fa30 2023-10-06T05:17:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.374+0000 7f7b6e280700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b4c066f30 0x7f7b4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.375+0000 7f7b6e280700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b4c066f30 0x7f7b4c0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b58003870 tx=0x7f7b58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.379+0000 7f7b5e7fc700 1 -- 172.21.15.45:0/1240311649 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b60055680 con 0x7f7b6813fa30 2023-10-06T05:17:29.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:29.584+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7b680c7910 con 0x7f7b6813fa30 2023-10-06T05:17:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.301+0000 7f7b5e7fc700 1 -- 172.21.15.45:0/1240311649 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7b60055210 con 0x7f7b6813fa30 2023-10-06T05:17:30.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.305+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b4c066f30 msgr2=0x7f7b4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:30.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.305+0000 7f7b704e4700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b4c066f30 0x7f7b4c0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b58003870 tx=0x7f7b58009040 comp rx=0 tx=0).stop 2023-10-06T05:17:30.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.305+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b6813fa30 msgr2=0x7f7b6816d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.305+0000 7f7b704e4700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b6813fa30 0x7f7b6816d510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000cc20 tx=0x7f7b60001a40 comp rx=0 tx=0).stop 2023-10-06T05:17:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.306+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 shutdown_connections 2023-10-06T05:17:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.306+0000 7f7b704e4700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b4c066f30 0x7f7b4c0693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.306+0000 7f7b704e4700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b6813fa30 0x7f7b6816d510 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.306+0000 7f7b704e4700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b680ccb50 0x7f7b680c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.306+0000 7f7b704e4700 1 --2- 172.21.15.45:0/1240311649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b680cd590 0x7f7b68169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.306+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 >> 172.21.15.45:0/1240311649 conn(0x7f7b68068730 msgr2=0x7f7b680c2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:30.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.307+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 shutdown_connections 2023-10-06T05:17:30.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.307+0000 7f7b704e4700 1 -- 172.21.15.45:0/1240311649 wait complete. 2023-10-06T05:17:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-06T05:17:30.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:17:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.470+0000 7f34ff6c4700 1 Processor -- start 2023-10-06T05:17:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.470+0000 7f34ff6c4700 1 -- start start 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.471+0000 7f34ff6c4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34f8159520 0x7f34f815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.471+0000 7f34ff6c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f815d3d0 0x7f34f8067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.471+0000 7f34ff6c4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f80c9c90 con 0x7f34f8159150 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.471+0000 7f34ff6c4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f815d8e0 con 0x7f34f8159520 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.471+0000 7f34ff6c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815dbc0 con 0x7f34f815d3d0 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.471+0000 7f34fcc5f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34f8159520 0x7f34f815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fcc5f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34f8159520 0x7f34f815ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49824/0 (socket says 172.21.15.45:49824) 2023-10-06T05:17:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fcc5f700 1 -- 172.21.15.45:0/3054016427 learned_addr learned my addr 172.21.15.45:0/3054016427 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fdc61700 1 --2- 172.21.15.45:0/3054016427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f815d3d0 0x7f34f8067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fcc5f700 1 -- 172.21.15.45:0/3054016427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f815d3d0 msgr2=0x7f34f8067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fcc5f700 1 --2- 172.21.15.45:0/3054016427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f815d3d0 0x7f34f8067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fcc5f700 1 -- 172.21.15.45:0/3054016427 >> v1:172.21.15.144:6789/0 conn(0x7f34f8159150 legacy=0x7f34f80ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fcc5f700 1 -- 172.21.15.45:0/3054016427 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f80c9920 con 0x7f34f8159520 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fdc61700 1 --2- 172.21.15.45:0/3054016427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f815d3d0 0x7f34f8067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34fcc5f700 1 --2- 172.21.15.45:0/3054016427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159520 0x7f34f815ce90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f34e80099b0 tx=0x7f34e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9114ef66a691077d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34effff700 1 -- 172.21.15.45:0/3054016427 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800b860 con 0x7f34f8159520 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34effff700 1 -- 172.21.15.45:0/3054016427 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e800b9c0 con 0x7f34f8159520 2023-10-06T05:17:30.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.472+0000 7f34effff700 1 -- 172.21.15.45:0/3054016427 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800bd80 con 0x7f34f8159520 2023-10-06T05:17:30.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 -- 172.21.15.45:0/3054016427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159520 msgr2=0x7f34f815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:30.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/3054016427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159520 0x7f34f815ce90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f34e80099b0 tx=0x7f34e8009250 comp rx=0 tx=0).stop 2023-10-06T05:17:30.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 -- 172.21.15.45:0/3054016427 shutdown_connections 2023-10-06T05:17:30.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/3054016427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f815d3d0 0x7f34f8067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/3054016427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159520 0x7f34f815ce90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 -- 172.21.15.45:0/3054016427 >> 172.21.15.45:0/3054016427 conn(0x7f34f80691c0 msgr2=0x7f34f80cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:30.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 -- 172.21.15.45:0/3054016427 shutdown_connections 2023-10-06T05:17:30.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.473+0000 7f34ff6c4700 1 -- 172.21.15.45:0/3054016427 wait complete. 2023-10-06T05:17:30.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.474+0000 7f34ff6c4700 1 Processor -- start 2023-10-06T05:17:30.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.474+0000 7f34ff6c4700 1 -- start start 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34ff6c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159150 0x7f34f81509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34ff6c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f815d3d0 0x7f34f80c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34ff6c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f80c6cd0 0x7f34f8158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34ff6c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80c7150 con 0x7f34f8159150 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34ff6c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f816cc20 con 0x7f34f815d3d0 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34ff6c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8150ee0 con 0x7f34f80c6cd0 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34fd460700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159150 0x7f34f81509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34fd460700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159150 0x7f34f81509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49830/0 (socket says 172.21.15.45:49830) 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34fd460700 1 -- 172.21.15.45:0/2245072198 learned_addr learned my addr 172.21.15.45:0/2245072198 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34fdc61700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f80c6cd0 0x7f34f8158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.475+0000 7f34fcc5f700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f815d3d0 0x7f34f80c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34fd460700 1 -- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f80c6cd0 msgr2=0x7f34f8158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34fd460700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f80c6cd0 0x7f34f8158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34fd460700 1 -- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f815d3d0 msgr2=0x7f34f80c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34fd460700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f815d3d0 0x7f34f80c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34fd460700 1 -- 172.21.15.45:0/2245072198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e80094f0 con 0x7f34f8159150 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34fd460700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159150 0x7f34f81509a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c590 tx=0x7f34f400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34ed7fa700 1 -- 172.21.15.45:0/2245072198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4011590 con 0x7f34f8159150 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34ed7fa700 1 -- 172.21.15.45:0/2245072198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f40116f0 con 0x7f34f8159150 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f8159900 con 0x7f34f8159150 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34ed7fa700 1 -- 172.21.15.45:0/2245072198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4011ab0 con 0x7f34f8159150 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.476+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f816cf20 con 0x7f34f8159150 2023-10-06T05:17:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.477+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f806e2d0 con 0x7f34f8159150 2023-10-06T05:17:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.478+0000 7f34ed7fa700 1 -- 172.21.15.45:0/2245072198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34f401b3b0 con 0x7f34f8159150 2023-10-06T05:17:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.478+0000 7f34ed7fa700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f34dc066f30 0x7f34dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.478+0000 7f34ed7fa700 1 -- 172.21.15.45:0/2245072198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f34f4083ba0 con 0x7f34f8159150 2023-10-06T05:17:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.478+0000 7f34fcc5f700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f34dc066f30 0x7f34dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.479+0000 7f34fcc5f700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f34dc066f30 0x7f34dc0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f34e80099b0 tx=0x7f34e8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.483+0000 7f34ed7fa700 1 -- 172.21.15.45:0/2245072198 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34f4050d50 con 0x7f34f8159150 2023-10-06T05:17:30.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:30.685+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f34f80cdba0 con 0x7f34f8159150 2023-10-06T05:17:31.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.307+0000 7f34ed7fa700 1 -- 172.21.15.45:0/2245072198 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) v1 ==== 170+0+0 (secure 0 0 0) 0x7f34f4017070 con 0x7f34f8159150 2023-10-06T05:17:31.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.314+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f34dc066f30 msgr2=0x7f34dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.314+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f34dc066f30 0x7f34dc0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f34e80099b0 tx=0x7f34e8017000 comp rx=0 tx=0).stop 2023-10-06T05:17:31.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.314+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159150 msgr2=0x7f34f81509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.314+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159150 0x7f34f81509a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c590 tx=0x7f34f400c950 comp rx=0 tx=0).stop 2023-10-06T05:17:31.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.315+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 shutdown_connections 2023-10-06T05:17:31.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.315+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f34dc066f30 0x7f34dc0693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.316+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34f80c6cd0 0x7f34f8158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.316+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34f8159150 0x7f34f81509a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.316+0000 7f34ff6c4700 1 --2- 172.21.15.45:0/2245072198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34f815d3d0 0x7f34f80c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.316+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 >> 172.21.15.45:0/2245072198 conn(0x7f34f80691c0 msgr2=0x7f34f81577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.320+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 shutdown_connections 2023-10-06T05:17:31.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.321+0000 7f34ff6c4700 1 -- 172.21.15.45:0/2245072198 wait complete. 2023-10-06T05:17:31.330 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-06T05:17:31.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:17:31.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.479+0000 7fb453965700 1 Processor -- start 2023-10-06T05:17:31.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.479+0000 7fb453965700 1 -- start start 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb453965700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb44c159150 0x7fb44c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb453965700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c159520 0x7fb44c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb453965700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c0c9c90 con 0x7fb44c15d3d0 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb453965700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c15d8e0 con 0x7fb44c159150 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb453965700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15dbc0 con 0x7fb44c159520 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb450f00700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c159520 0x7fb44c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb451701700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb44c159150 0x7fb44c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb450f00700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c159520 0x7fb44c15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:56052/0 (socket says 172.21.15.45:56052) 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb450f00700 1 -- 172.21.15.45:0/3147437260 learned_addr learned my addr 172.21.15.45:0/3147437260 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.480+0000 7fb451f02700 1 --1- 172.21.15.45:0/3147437260 >> v1:172.21.15.144:6789/0 conn(0x7fb44c15d3d0 0x7fb44c067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:44026/0 (socket says 172.21.15.45:44026) 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb450f00700 1 -- 172.21.15.45:0/3147437260 >> v2:172.21.15.144:3300/0 conn(0x7fb44c159150 msgr2=0x7fb44c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb450f00700 1 --2- 172.21.15.45:0/3147437260 >> v2:172.21.15.144:3300/0 conn(0x7fb44c159150 0x7fb44c0ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb450f00700 1 -- 172.21.15.45:0/3147437260 >> v1:172.21.15.144:6789/0 conn(0x7fb44c15d3d0 legacy=0x7fb44c067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb450f00700 1 -- 172.21.15.45:0/3147437260 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0c9920 con 0x7fb44c159520 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb451701700 1 --2- 172.21.15.45:0/3147437260 >> v2:172.21.15.144:3300/0 conn(0x7fb44c159150 0x7fb44c0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb450f00700 1 --2- 172.21.15.45:0/3147437260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c159520 0x7fb44c15ce90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4480099b0 tx=0x7fb448009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c8577224aaa1b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb43ffff700 1 -- 172.21.15.45:0/3147437260 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448009ea0 con 0x7fb44c159520 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb43ffff700 1 -- 172.21.15.45:0/3147437260 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800b760 con 0x7fb44c159520 2023-10-06T05:17:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.481+0000 7fb43ffff700 1 -- 172.21.15.45:0/3147437260 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800bb20 con 0x7fb44c159520 2023-10-06T05:17:31.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 -- 172.21.15.45:0/3147437260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c159520 msgr2=0x7fb44c15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 --2- 172.21.15.45:0/3147437260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c159520 0x7fb44c15ce90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4480099b0 tx=0x7fb448009250 comp rx=0 tx=0).stop 2023-10-06T05:17:31.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 -- 172.21.15.45:0/3147437260 shutdown_connections 2023-10-06T05:17:31.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 --2- 172.21.15.45:0/3147437260 >> v2:172.21.15.144:3300/0 conn(0x7fb44c159150 0x7fb44c0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 --2- 172.21.15.45:0/3147437260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c159520 0x7fb44c15ce90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 -- 172.21.15.45:0/3147437260 >> 172.21.15.45:0/3147437260 conn(0x7fb44c0691c0 msgr2=0x7fb44c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:31.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 -- 172.21.15.45:0/3147437260 shutdown_connections 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.482+0000 7fb453965700 1 -- 172.21.15.45:0/3147437260 wait complete. 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.483+0000 7fb453965700 1 Processor -- start 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.483+0000 7fb453965700 1 -- start start 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb453965700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb44c159150 0x7fb44c15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb453965700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb44c15d3d0 0x7fb44c147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb453965700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c16a760 0x7fb44c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb453965700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15ab20 con 0x7fb44c15d3d0 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb453965700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c147f50 con 0x7fb44c159150 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb453965700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15c0e0 con 0x7fb44c16a760 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb450f00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb44c15d3d0 0x7fb44c147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb450f00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb44c15d3d0 0x7fb44c147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49840/0 (socket says 172.21.15.45:49840) 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb450f00700 1 -- 172.21.15.45:0/4050390368 learned_addr learned my addr 172.21.15.45:0/4050390368 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:31.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.484+0000 7fb451701700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb44c159150 0x7fb44c15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb451f02700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c16a760 0x7fb44c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb450f00700 1 -- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c16a760 msgr2=0x7fb44c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb450f00700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c16a760 0x7fb44c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb450f00700 1 -- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb44c159150 msgr2=0x7fb44c15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb450f00700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb44c159150 0x7fb44c15c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb450f00700 1 -- 172.21.15.45:0/4050390368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb440009420 con 0x7fb44c15d3d0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb450f00700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb44c15d3d0 0x7fb44c147a10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb448006fd0 tx=0x7fb448016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb43d7fa700 1 -- 172.21.15.45:0/4050390368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448011b20 con 0x7fb44c15d3d0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb43d7fa700 1 -- 172.21.15.45:0/4050390368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448011c80 con 0x7fb44c15d3d0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4480094f0 con 0x7fb44c15d3d0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb43d7fa700 1 -- 172.21.15.45:0/4050390368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44801c460 con 0x7fb44c15d3d0 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.485+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c15ba90 con 0x7fb44c15d3d0 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.486+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c15c290 con 0x7fb44c15d3d0 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.487+0000 7fb43d7fa700 1 -- 172.21.15.45:0/4050390368 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb44801c870 con 0x7fb44c15d3d0 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.487+0000 7fb43d7fa700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb430066f30 0x7fb4300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.487+0000 7fb43d7fa700 1 -- 172.21.15.45:0/4050390368 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6278+0+0 (secure 0 0 0) 0x7fb4480868f0 con 0x7fb44c15d3d0 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.487+0000 7fb451701700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb430066f30 0x7fb4300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.488+0000 7fb451701700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb430066f30 0x7fb4300693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0c70f0 tx=0x7fb4400091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.492+0000 7fb43d7fa700 1 -- 172.21.15.45:0/4050390368 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb44c15c290 con 0x7fb44c15d3d0 2023-10-06T05:17:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:31.742+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb44c15c250 con 0x7fb44c15d3d0 2023-10-06T05:17:32.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.310+0000 7fb43d7fa700 1 -- 172.21.15.45:0/4050390368 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb4480573b0 con 0x7fb44c15d3d0 2023-10-06T05:17:32.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.321+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb430066f30 msgr2=0x7fb4300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.321+0000 7fb453965700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb430066f30 0x7fb4300693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0c70f0 tx=0x7fb4400091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.321+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb44c15d3d0 msgr2=0x7fb44c147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.321+0000 7fb453965700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb44c15d3d0 0x7fb44c147a10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb448006fd0 tx=0x7fb448016000 comp rx=0 tx=0).stop 2023-10-06T05:17:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.322+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 shutdown_connections 2023-10-06T05:17:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.322+0000 7fb453965700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb430066f30 0x7fb4300693e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.322+0000 7fb453965700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44c16a760 0x7fb44c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.322+0000 7fb453965700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb44c159150 0x7fb44c15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.322+0000 7fb453965700 1 --2- 172.21.15.45:0/4050390368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb44c15d3d0 0x7fb44c147a10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.322+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 >> 172.21.15.45:0/4050390368 conn(0x7fb44c0691c0 msgr2=0x7fb44c0c1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.324+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 shutdown_connections 2023-10-06T05:17:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.325+0000 7fb453965700 1 -- 172.21.15.45:0/4050390368 wait complete. 2023-10-06T05:17:32.332 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 8 and data pool 9 2023-10-06T05:17:32.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.487+0000 7f4148b34700 1 Processor -- start 2023-10-06T05:17:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.487+0000 7f4148b34700 1 -- start start 2023-10-06T05:17:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.488+0000 7f4148b34700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4144159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.488+0000 7f4148b34700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41440ca6d0 con 0x7f41440692d0 2023-10-06T05:17:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.488+0000 7f4148b34700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414415d550 con 0x7f414415d040 2023-10-06T05:17:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.488+0000 7f4148b34700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414415d810 con 0x7f4144159170 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.488+0000 7f4142d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f41440692d0 0x7f41440aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45766/0 (socket says 172.21.15.45:45766) 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.488+0000 7f4142d9d700 1 -- 172.21.15.45:0/2991273628 learned_addr learned my addr 172.21.15.45:0/2991273628 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414259c700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4144159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865911340 0 0) 0x7f41440ca6d0 con 0x7f41440692d0 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4130003570 con 0x7f41440692d0 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688643930 0 0) 0x7f414415d550 con 0x7f414415d040 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41440ca6d0 con 0x7f414415d040 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520387452 0 0) 0x7f4130003570 con 0x7f41440692d0 2023-10-06T05:17:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 msgr2=0x7f4144159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4144159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 >> v1:172.21.15.144:6789/0 conn(0x7f414415d040 legacy=0x7f4144068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414415daf0 con 0x7f41440692d0 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f413c0017f0 con 0x7f41440692d0 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414259c700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4144159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 >> v1:172.21.15.45:6789/0 conn(0x7f41440692d0 legacy=0x7f41440aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4130003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130004160 0x7f4130004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4130004d50 0x7f41300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414415d810 con 0x7f4130004160 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413000bba0 con 0x7f4130004d50 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413000bd60 con 0x7f4144159170 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f4141d9b700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130004160 0x7f4130004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f414259c700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4130003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.489+0000 7f4142d9d700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4130004d50 0x7f41300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f4141d9b700 1 -- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 msgr2=0x7f4130003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f4141d9b700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4130003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f4141d9b700 1 -- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4130004d50 msgr2=0x7f41300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:32.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f4141d9b700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4130004d50 0x7f41300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f4141d9b700 1 -- 172.21.15.45:0/2991273628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134009420 con 0x7f4130004160 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f414259c700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4144159170 0x7f4130003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f4141d9b700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130004160 0x7f4130004810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f412c001a10 tx=0x7f412c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bde82dceb601be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c00c5d0 con 0x7f4130004160 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c00c730 con 0x7f4130004160 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.490+0000 7f414159a700 1 -- 172.21.15.45:0/2991273628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c00cad0 con 0x7f4130004160 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.491+0000 7f4148b34700 1 -- 172.21.15.45:0/2991273628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130004160 msgr2=0x7f4130004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.491+0000 7f4148b34700 1 --2- 172.21.15.45:0/2991273628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130004160 0x7f4130004810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f412c001a10 tx=0x7f412c007d30 comp rx=0 tx=0).stop 2023-10-06T05:17:32.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.491+0000 7f4142d9d700 1 -- 172.21.15.45:0/2991273628 reap_dead start 2023-10-06T05:17:32.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.491+0000 7f4148b34700 1 -- 172.21.15.45:0/2991273628 shutdown_connections 2023-10-06T05:17:32.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.491+0000 7f4148b34700 1 -- 172.21.15.45:0/2991273628 >> 172.21.15.45:0/2991273628 conn(0x7f414406f7c0 msgr2=0x7f41440d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.491+0000 7f4148b34700 1 -- 172.21.15.45:0/2991273628 shutdown_connections 2023-10-06T05:17:32.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.491+0000 7f4148b34700 1 -- 172.21.15.45:0/2991273628 wait complete. 2023-10-06T05:17:32.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.492+0000 7f4148b34700 1 Processor -- start 2023-10-06T05:17:32.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.492+0000 7f4148b34700 1 -- start start 2023-10-06T05:17:32.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4148b34700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4130004160 0x7f4144168940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4148b34700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440c7b20 0x7f41440c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4148b34700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41440c9460 0x7f41440c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4148b34700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41440c8bd0 con 0x7f41440c9460 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4148b34700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41440c9e40 con 0x7f41440c7b20 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4148b34700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4144168e80 con 0x7f4130004160 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41440c9460 0x7f41440c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41440c9460 0x7f41440c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49866/0 (socket says 172.21.15.45:49866) 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 -- 172.21.15.45:0/315223595 learned_addr learned my addr 172.21.15.45:0/315223595 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f414259c700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4130004160 0x7f4144168940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4141d9b700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440c7b20 0x7f41440c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 -- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4130004160 msgr2=0x7f4144168940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4130004160 0x7f4144168940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 -- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440c7b20 msgr2=0x7f41440c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440c7b20 0x7f41440c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 -- 172.21.15.45:0/315223595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4144159a50 con 0x7f41440c9460 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4141d9b700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440c7b20 0x7f41440c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.493+0000 7f4142d9d700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41440c9460 0x7f41440c9900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f414406a680 tx=0x7f413c002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.494+0000 7f413a7fc700 1 -- 172.21.15.45:0/315223595 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c00ad40 con 0x7f41440c9460 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.494+0000 7f413a7fc700 1 -- 172.21.15.45:0/315223595 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c0133f0 con 0x7f41440c9460 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.494+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41440c8610 con 0x7f41440c9460 2023-10-06T05:17:32.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.494+0000 7f413a7fc700 1 -- 172.21.15.45:0/315223595 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c00aea0 con 0x7f41440c9460 2023-10-06T05:17:32.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.494+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41441596e0 con 0x7f41440c9460 2023-10-06T05:17:32.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.495+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4144159540 con 0x7f41440c9460 2023-10-06T05:17:32.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.495+0000 7f413a7fc700 1 -- 172.21.15.45:0/315223595 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f413c013a40 con 0x7f41440c9460 2023-10-06T05:17:32.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.495+0000 7f413a7fc700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4124066f30 0x7f41240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.496+0000 7f413a7fc700 1 -- 172.21.15.45:0/315223595 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f413c085170 con 0x7f41440c9460 2023-10-06T05:17:32.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.496+0000 7f414259c700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4124066f30 0x7f41240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.496+0000 7f414259c700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4124066f30 0x7f41240693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4134006dc0 tx=0x7f4134008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.500+0000 7f413a7fc700 1 -- 172.21.15.45:0/315223595 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4144159540 con 0x7f41440c9460 2023-10-06T05:17:32.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.707+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4144159540 con 0x7f41440c9460 2023-10-06T05:17:32.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.708+0000 7f413a7fc700 1 -- 172.21.15.45:0/315223595 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17180 (secure 0 0 0) 0x7f4144159540 con 0x7f41440c9460 2023-10-06T05:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":51,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:17:31.744063+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:29.616613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:30.718338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:32.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.710+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4124066f30 msgr2=0x7f41240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.710+0000 7f4148b34700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4124066f30 0x7f41240693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4134006dc0 tx=0x7f4134008040 comp rx=0 tx=0).stop 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.710+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41440c9460 msgr2=0x7f41440c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.710+0000 7f4148b34700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41440c9460 0x7f41440c9900 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f414406a680 tx=0x7f413c002170 comp rx=0 tx=0).stop 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 shutdown_connections 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4124066f30 0x7f41240693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41440c9460 0x7f41440c9900 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4130004160 0x7f4144168940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 --2- 172.21.15.45:0/315223595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440c7b20 0x7f41440c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 >> 172.21.15.45:0/315223595 conn(0x7f414406f7c0 msgr2=0x7f41440ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 shutdown_connections 2023-10-06T05:17:32.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.711+0000 7f4148b34700 1 -- 172.21.15.45:0/315223595 wait complete. 2023-10-06T05:17:32.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.857+0000 7f80c4e97700 1 Processor -- start 2023-10-06T05:17:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.857+0000 7f80c4e97700 1 -- start start 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80c4e97700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006f070 0x7f80c006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80c4e97700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f80c00c72c0 0x7f80c00acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80c4e97700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c006b240 con 0x7f80c006add0 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80c4e97700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c00ac5a0 con 0x7f80c00c72c0 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80c4e97700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c0159690 con 0x7f80c006f070 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80bdd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f80c006add0 0x7f80c00c6bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45768/0 (socket says 172.21.15.45:45768) 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80bdd9b700 1 -- 172.21.15.45:0/4059509559 learned_addr learned my addr 172.21.15.45:0/4059509559 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80be59c700 1 --2- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006f070 0x7f80c006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80bed9d700 1 --2- 172.21.15.45:0/4059509559 >> v2:172.21.15.144:3300/0 conn(0x7f80c00c72c0 0x7f80c00acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80bd59a700 1 -- 172.21.15.45:0/4059509559 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347783716 0 0) 0x7f80c006b240 con 0x7f80c006add0 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.858+0000 7f80bd59a700 1 -- 172.21.15.45:0/4059509559 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a8003570 con 0x7f80c006add0 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bed9d700 1 -- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006f070 msgr2=0x7f80c006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bed9d700 1 --2- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006f070 0x7f80c006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bed9d700 1 -- 172.21.15.45:0/4059509559 >> v1:172.21.15.45:6789/0 conn(0x7f80c006add0 legacy=0x7f80c00c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bed9d700 1 -- 172.21.15.45:0/4059509559 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c00c78e0 con 0x7f80c00c72c0 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80be59c700 1 --2- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006f070 0x7f80c006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bed9d700 1 --2- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00acd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009e60 tx=0x7f80b8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3320fa1da7861cc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bd59a700 1 -- 172.21.15.45:0/4059509559 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b800e070 con 0x7f80c00c72c0 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bd59a700 1 -- 172.21.15.45:0/4059509559 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b800bab0 con 0x7f80c00c72c0 2023-10-06T05:17:32.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.859+0000 7f80bd59a700 1 -- 172.21.15.45:0/4059509559 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b800be30 con 0x7f80c00c72c0 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 -- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 msgr2=0x7f80c00acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 --2- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00acd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009e60 tx=0x7f80b8009780 comp rx=0 tx=0).stop 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 -- 172.21.15.45:0/4059509559 shutdown_connections 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 --2- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00acd60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 --2- 172.21.15.45:0/4059509559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006f070 0x7f80c006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 -- 172.21.15.45:0/4059509559 >> 172.21.15.45:0/4059509559 conn(0x7f80c00cc5a0 msgr2=0x7f80c00cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 -- 172.21.15.45:0/4059509559 shutdown_connections 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.860+0000 7f80c4e97700 1 -- 172.21.15.45:0/4059509559 wait complete. 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.861+0000 7f80c4e97700 1 Processor -- start 2023-10-06T05:17:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.861+0000 7f80c4e97700 1 -- start start 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80c4e97700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006add0 0x7f80c016d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80c4e97700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c006f070 0x7f80c01503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80c4e97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80c4e97700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c016d740 con 0x7f80c006f070 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80c4e97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c0150930 con 0x7f80c00c72c0 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80c4e97700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00b0210 con 0x7f80c006add0 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c006f070 0x7f80c01503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c006f070 0x7f80c01503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49882/0 (socket says 172.21.15.45:49882) 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 -- 172.21.15.45:0/574135590 learned_addr learned my addr 172.21.15.45:0/574135590 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80be59c700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006add0 0x7f80c016d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bed9d700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 -- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006add0 msgr2=0x7f80c016d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006add0 0x7f80c016d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 -- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 msgr2=0x7f80c00af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 -- 172.21.15.45:0/574135590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0009420 con 0x7f80c006f070 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bed9d700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.862+0000 7f80bdd9b700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c006f070 0x7f80c01503f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f80b40011e0 tx=0x7f80b4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.863+0000 7f80ae7fc700 1 -- 172.21.15.45:0/574135590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4005b30 con 0x7f80c006f070 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.863+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b8009a20 con 0x7f80c006f070 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.863+0000 7f80ae7fc700 1 -- 172.21.15.45:0/574135590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b4005c90 con 0x7f80c006f070 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.863+0000 7f80ae7fc700 1 -- 172.21.15.45:0/574135590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b40072e0 con 0x7f80c006f070 2023-10-06T05:17:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.863+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c00aff30 con 0x7f80c006f070 2023-10-06T05:17:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.864+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c006ac10 con 0x7f80c006f070 2023-10-06T05:17:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.864+0000 7f80ae7fc700 1 -- 172.21.15.45:0/574135590 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80b4007650 con 0x7f80c006f070 2023-10-06T05:17:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.865+0000 7f80ae7fc700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.865+0000 7f80ae7fc700 1 -- 172.21.15.45:0/574135590 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f80b4083d00 con 0x7f80c006f070 2023-10-06T05:17:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.865+0000 7f80be59c700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.865+0000 7f80be59c700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80a0066f30 0x7f80a00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f80b00093f0 tx=0x7f80b0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:32.869+0000 7f80ae7fc700 1 -- 172.21.15.45:0/574135590 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80b4054770 con 0x7f80c006f070 2023-10-06T05:17:33.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.071+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80c00cc0e0 con 0x7f80c006f070 2023-10-06T05:17:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.072+0000 7f80ae7fc700 1 -- 172.21.15.45:0/574135590 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17180 (secure 0 0 0) 0x7f80b4017020 con 0x7f80c006f070 2023-10-06T05:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":51,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:17:31.744063+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:29.616613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:30.718338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80a0066f30 msgr2=0x7f80a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80a0066f30 0x7f80a00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f80b00093f0 tx=0x7f80b0008040 comp rx=0 tx=0).stop 2023-10-06T05:17:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c006f070 msgr2=0x7f80c01503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c006f070 0x7f80c01503f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f80b40011e0 tx=0x7f80b4007df0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 shutdown_connections 2023-10-06T05:17:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80c00c72c0 0x7f80c00af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80c006add0 0x7f80c016d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 --2- 172.21.15.45:0/574135590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c006f070 0x7f80c01503f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 >> 172.21.15.45:0/574135590 conn(0x7f80c00cc5a0 msgr2=0x7f80c00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 shutdown_connections 2023-10-06T05:17:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.074+0000 7f80c4e97700 1 -- 172.21.15.45:0/574135590 wait complete. 2023-10-06T05:17:33.089 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:17:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.222+0000 7f6e88240700 1 Processor -- start 2023-10-06T05:17:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.223+0000 7f6e88240700 1 -- start start 2023-10-06T05:17:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.223+0000 7f6e88240700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800b0610 0x7f6e800c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.223+0000 7f6e88240700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e80069ed0 0x7f6e800cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.223+0000 7f6e88240700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800ce6a0 con 0x7f6e800c3db0 2023-10-06T05:17:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.223+0000 7f6e88240700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e800690f0 con 0x7f6e80069ed0 2023-10-06T05:17:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.223+0000 7f6e88240700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800ceeb0 con 0x7f6e800b0610 2023-10-06T05:17:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e867dd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e80069ed0 0x7f6e800cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e857db700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6e800c3db0 0x7f6e800697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45772/0 (socket says 172.21.15.45:45772) 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e857db700 1 -- 172.21.15.45:0/2755314118 learned_addr learned my addr 172.21.15.45:0/2755314118 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e867dd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e80069ed0 0x7f6e800cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49884/0 (socket says 172.21.15.45:49884) 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e85fdc700 1 --2- 172.21.15.45:0/2755314118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800b0610 0x7f6e800c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e867dd700 1 -- 172.21.15.45:0/2755314118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800b0610 msgr2=0x7f6e800c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e867dd700 1 --2- 172.21.15.45:0/2755314118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800b0610 0x7f6e800c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e867dd700 1 -- 172.21.15.45:0/2755314118 >> v1:172.21.15.45:6789/0 conn(0x7f6e800c3db0 legacy=0x7f6e800697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e867dd700 1 -- 172.21.15.45:0/2755314118 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e80159880 con 0x7f6e80069ed0 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e84fda700 1 -- 172.21.15.45:0/2755314118 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434449148 0 0) 0x7f6e800ce6a0 con 0x7f6e800c3db0 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e867dd700 1 --2- 172.21.15.45:0/2755314118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e80069ed0 0x7f6e800cc350 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009e90 tx=0x7f6e78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ad85511962be64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e84fda700 1 -- 172.21.15.45:0/2755314118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7800e070 con 0x7f6e80069ed0 2023-10-06T05:17:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e84fda700 1 -- 172.21.15.45:0/2755314118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7800bbf0 con 0x7f6e80069ed0 2023-10-06T05:17:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.224+0000 7f6e84fda700 1 -- 172.21.15.45:0/2755314118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78012430 con 0x7f6e80069ed0 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 -- 172.21.15.45:0/2755314118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e80069ed0 msgr2=0x7f6e800cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 --2- 172.21.15.45:0/2755314118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e80069ed0 0x7f6e800cc350 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009e90 tx=0x7f6e78009780 comp rx=0 tx=0).stop 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 -- 172.21.15.45:0/2755314118 shutdown_connections 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 --2- 172.21.15.45:0/2755314118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e80069ed0 0x7f6e800cc350 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 --2- 172.21.15.45:0/2755314118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800b0610 0x7f6e800c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 -- 172.21.15.45:0/2755314118 >> 172.21.15.45:0/2755314118 conn(0x7f6e80067e00 msgr2=0x7f6e800cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 -- 172.21.15.45:0/2755314118 shutdown_connections 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.225+0000 7f6e88240700 1 -- 172.21.15.45:0/2755314118 wait complete. 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.226+0000 7f6e88240700 1 Processor -- start 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e88240700 1 -- start start 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e88240700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e80069ed0 0x7f6e80158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e88240700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e800b0610 0x7f6e800c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e88240700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800c3db0 0x7f6e800c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e88240700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800b3700 con 0x7f6e800b0610 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e88240700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800b2af0 con 0x7f6e80069ed0 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e88240700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800b2dd0 con 0x7f6e800c3db0 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e857db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e800b0610 0x7f6e800c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e857db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e800b0610 0x7f6e800c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49894/0 (socket says 172.21.15.45:49894) 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.227+0000 7f6e857db700 1 -- 172.21.15.45:0/695485310 learned_addr learned my addr 172.21.15.45:0/695485310 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e867dd700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800c3db0 0x7f6e800c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e85fdc700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e80069ed0 0x7f6e80158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e857db700 1 -- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800c3db0 msgr2=0x7f6e800c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e857db700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800c3db0 0x7f6e800c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e857db700 1 -- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e80069ed0 msgr2=0x7f6e80158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e857db700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e80069ed0 0x7f6e80158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e857db700 1 -- 172.21.15.45:0/695485310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78009a20 con 0x7f6e800b0610 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e857db700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e800b0610 0x7f6e800c5510 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c5b0 tx=0x7f6e70001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e75ffb700 1 -- 172.21.15.45:0/695485310 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70005b30 con 0x7f6e800b0610 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e75ffb700 1 -- 172.21.15.45:0/695485310 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70005c90 con 0x7f6e800b0610 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e800b30b0 con 0x7f6e800b0610 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e75ffb700 1 -- 172.21.15.45:0/695485310 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70007300 con 0x7f6e800b0610 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.228+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e801591a0 con 0x7f6e800b0610 2023-10-06T05:17:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.230+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8006e350 con 0x7f6e800b0610 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.230+0000 7f6e75ffb700 1 -- 172.21.15.45:0/695485310 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e7001b3b0 con 0x7f6e800b0610 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.230+0000 7f6e75ffb700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6e64066f30 0x7f6e640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.230+0000 7f6e75ffb700 1 -- 172.21.15.45:0/695485310 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6e70083d10 con 0x7f6e800b0610 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.231+0000 7f6e85fdc700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6e64066f30 0x7f6e640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.231+0000 7f6e85fdc700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6e64066f30 0x7f6e640693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c006aa0 tx=0x7f6e7c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.235+0000 7f6e75ffb700 1 -- 172.21.15.45:0/695485310 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e70050ce0 con 0x7f6e800b0610 2023-10-06T05:17:33.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:33.486+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6e8004bb40 con 0x7f6e800b0610 2023-10-06T05:17:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.325+0000 7f6e75ffb700 1 -- 172.21.15.45:0/695485310 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6e700170b0 con 0x7f6e800b0610 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.328+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6e64066f30 msgr2=0x7f6e640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.328+0000 7f6e88240700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6e64066f30 0x7f6e640693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c006aa0 tx=0x7f6e7c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.328+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e800b0610 msgr2=0x7f6e800c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.328+0000 7f6e88240700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e800b0610 0x7f6e800c5510 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c5b0 tx=0x7f6e70001070 comp rx=0 tx=0).stop 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 shutdown_connections 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6e64066f30 0x7f6e640693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e800c3db0 0x7f6e800c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e80069ed0 0x7f6e80158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 --2- 172.21.15.45:0/695485310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e800b0610 0x7f6e800c5510 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 >> 172.21.15.45:0/695485310 conn(0x7f6e80067e00 msgr2=0x7f6e800696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 shutdown_connections 2023-10-06T05:17:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.329+0000 7f6e88240700 1 -- 172.21.15.45:0/695485310 wait complete. 2023-10-06T05:17:34.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:17:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.483+0000 7fa8b4fb5700 1 Processor -- start 2023-10-06T05:17:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.483+0000 7fa8b4fb5700 1 -- start start 2023-10-06T05:17:34.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8b4fb5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8b0159120 0x7fa8b00b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8b4fb5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b006d8e0 0x7fa8b006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8b4fb5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b006e280 con 0x7fa8b00b3810 2023-10-06T05:17:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8b4fb5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b00b3c80 con 0x7fa8b0159120 2023-10-06T05:17:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8b4fb5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0069b50 con 0x7fa8b006d8e0 2023-10-06T05:17:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8ae59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8b0159120 0x7fa8b00b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8ae59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8b0159120 0x7fa8b00b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49904/0 (socket says 172.21.15.45:49904) 2023-10-06T05:17:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8ae59c700 1 -- 172.21.15.45:0/3445377132 learned_addr learned my addr 172.21.15.45:0/3445377132 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:34.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8aed9d700 1 --2- 172.21.15.45:0/3445377132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b006d8e0 0x7fa8b006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8ae59c700 1 -- 172.21.15.45:0/3445377132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b006d8e0 msgr2=0x7fa8b006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8ae59c700 1 --2- 172.21.15.45:0/3445377132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b006d8e0 0x7fa8b006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8ae59c700 1 -- 172.21.15.45:0/3445377132 >> v1:172.21.15.144:6789/0 conn(0x7fa8b00b3810 legacy=0x7fa8b0069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.484+0000 7fa8ae59c700 1 -- 172.21.15.45:0/3445377132 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b006a6f0 con 0x7fa8b0159120 2023-10-06T05:17:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8ae59c700 1 --2- 172.21.15.45:0/3445377132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b00b1620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa8980099b0 tx=0x7fa898009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe7eff0e0d772c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8ad59a700 1 -- 172.21.15.45:0/3445377132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898009ea0 con 0x7fa8b0159120 2023-10-06T05:17:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8ad59a700 1 -- 172.21.15.45:0/3445377132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89800b860 con 0x7fa8b0159120 2023-10-06T05:17:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8ad59a700 1 -- 172.21.15.45:0/3445377132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89800bc00 con 0x7fa8b0159120 2023-10-06T05:17:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/3445377132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 msgr2=0x7fa8b00b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/3445377132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b00b1620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa8980099b0 tx=0x7fa898009250 comp rx=0 tx=0).stop 2023-10-06T05:17:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/3445377132 shutdown_connections 2023-10-06T05:17:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/3445377132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b006d8e0 0x7fa8b006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/3445377132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b00b1620 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.485+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/3445377132 >> 172.21.15.45:0/3445377132 conn(0x7fa8b00ce9b0 msgr2=0x7fa8b006c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.486+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/3445377132 shutdown_connections 2023-10-06T05:17:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.486+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/3445377132 wait complete. 2023-10-06T05:17:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.487+0000 7fa8b4fb5700 1 Processor -- start 2023-10-06T05:17:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.487+0000 7fa8b4fb5700 1 -- start start 2023-10-06T05:17:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8b4fb5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8b006d8e0 0x7fa8b01698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8b4fb5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b00b3810 0x7fa8b016ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8b4fb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b015bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8b4fb5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0156170 con 0x7fa8b0159120 2023-10-06T05:17:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8b4fb5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b00a47b0 con 0x7fa8b006d8e0 2023-10-06T05:17:34.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8b4fb5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b016d3f0 con 0x7fa8b00b3810 2023-10-06T05:17:34.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b015bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b015bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49906/0 (socket says 172.21.15.45:49906) 2023-10-06T05:17:34.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 -- 172.21.15.45:0/4001657433 learned_addr learned my addr 172.21.15.45:0/4001657433 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:34.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8add9b700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b00b3810 0x7fa8b016ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8ae59c700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8b006d8e0 0x7fa8b01698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 -- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b00b3810 msgr2=0x7fa8b016ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b00b3810 0x7fa8b016ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 -- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8b006d8e0 msgr2=0x7fa8b01698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8b006d8e0 0x7fa8b01698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 -- 172.21.15.45:0/4001657433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980094f0 con 0x7fa8b0159120 2023-10-06T05:17:34.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8aed9d700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b015bc80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009e90 tx=0x7fa8a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8a67fc700 1 -- 172.21.15.45:0/4001657433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a800cb70 con 0x7fa8b0159120 2023-10-06T05:17:34.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8a67fc700 1 -- 172.21.15.45:0/4001657433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a800f040 con 0x7fa8b0159120 2023-10-06T05:17:34.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b016d620 con 0x7fa8b0159120 2023-10-06T05:17:34.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.488+0000 7fa8a67fc700 1 -- 172.21.15.45:0/4001657433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8014480 con 0x7fa8b0159120 2023-10-06T05:17:34.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.489+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b016dbe0 con 0x7fa8b0159120 2023-10-06T05:17:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.490+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b006edd0 con 0x7fa8b0159120 2023-10-06T05:17:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.490+0000 7fa8a67fc700 1 -- 172.21.15.45:0/4001657433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8a8014830 con 0x7fa8b0159120 2023-10-06T05:17:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.490+0000 7fa8a67fc700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa890066f30 0x7fa8900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.491+0000 7fa8a67fc700 1 -- 172.21.15.45:0/4001657433 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa8a8085480 con 0x7fa8b0159120 2023-10-06T05:17:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.491+0000 7fa8ae59c700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa890066f30 0x7fa8900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.491+0000 7fa8ae59c700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa890066f30 0x7fa8900693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8980099b0 tx=0x7fa898017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.495+0000 7fa8a67fc700 1 -- 172.21.15.45:0/4001657433 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8a8052450 con 0x7fa8b0159120 2023-10-06T05:17:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:34.742+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa8b006c340 con 0x7fa8b0159120 2023-10-06T05:17:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.330+0000 7fa8a67fc700 1 -- 172.21.15.45:0/4001657433 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa8a8055a70 con 0x7fa8b0159120 2023-10-06T05:17:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa890066f30 msgr2=0x7fa8900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa890066f30 0x7fa8900693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8980099b0 tx=0x7fa898017000 comp rx=0 tx=0).stop 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 msgr2=0x7fa8b015bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b015bc80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009e90 tx=0x7fa8a800b040 comp rx=0 tx=0).stop 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 shutdown_connections 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa890066f30 0x7fa8900693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b0159120 0x7fa8b015bc80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8b006d8e0 0x7fa8b01698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 --2- 172.21.15.45:0/4001657433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8b00b3810 0x7fa8b016ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 >> 172.21.15.45:0/4001657433 conn(0x7fa8b00ce9b0 msgr2=0x7fa8b00cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 shutdown_connections 2023-10-06T05:17:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.333+0000 7fa8b4fb5700 1 -- 172.21.15.45:0/4001657433 wait complete. 2023-10-06T05:17:35.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:35.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.485+0000 7f9cc140e700 1 Processor -- start 2023-10-06T05:17:35.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.485+0000 7f9cc140e700 1 -- start start 2023-10-06T05:17:35.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.485+0000 7f9cc140e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cbc1595d0 0x7f9cbc06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.485+0000 7f9cc140e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9cbc06b5c0 0x7f9cbc0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cc140e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cbc0c7d00 con 0x7f9cbc159200 2023-10-06T05:17:35.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cc140e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cbc0c7780 con 0x7f9cbc1595d0 2023-10-06T05:17:35.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cc140e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cbc06bb40 con 0x7f9cbc06b5c0 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cbc1595d0 0x7f9cbc06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cbc1595d0 0x7f9cbc06b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49922/0 (socket says 172.21.15.45:49922) 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 -- 172.21.15.45:0/8021560 learned_addr learned my addr 172.21.15.45:0/8021560 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cbb7fe700 1 --2- 172.21.15.45:0/8021560 >> v2:172.21.15.144:3300/0 conn(0x7f9cbc06b5c0 0x7f9cbc0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 -- 172.21.15.45:0/8021560 >> v2:172.21.15.144:3300/0 conn(0x7f9cbc06b5c0 msgr2=0x7f9cbc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 --2- 172.21.15.45:0/8021560 >> v2:172.21.15.144:3300/0 conn(0x7f9cbc06b5c0 0x7f9cbc0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 -- 172.21.15.45:0/8021560 >> v1:172.21.15.144:6789/0 conn(0x7f9cbc159200 legacy=0x7f9cbc0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 -- 172.21.15.45:0/8021560 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc06be20 con 0x7f9cbc1595d0 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cba7fc700 1 --2- 172.21.15.45:0/8021560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc1595d0 0x7f9cbc06b080 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc068680 tx=0x7f9ca4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790dcaba0cbb8c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cb9ffb700 1 -- 172.21.15.45:0/8021560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4009e70 con 0x7f9cbc1595d0 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cb9ffb700 1 -- 172.21.15.45:0/8021560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca400b860 con 0x7f9cbc1595d0 2023-10-06T05:17:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.486+0000 7f9cb9ffb700 1 -- 172.21.15.45:0/8021560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400bc00 con 0x7f9cbc1595d0 2023-10-06T05:17:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 -- 172.21.15.45:0/8021560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc1595d0 msgr2=0x7f9cbc06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 --2- 172.21.15.45:0/8021560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc1595d0 0x7f9cbc06b080 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc068680 tx=0x7f9ca4009250 comp rx=0 tx=0).stop 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 -- 172.21.15.45:0/8021560 shutdown_connections 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 --2- 172.21.15.45:0/8021560 >> v2:172.21.15.144:3300/0 conn(0x7f9cbc06b5c0 0x7f9cbc0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 --2- 172.21.15.45:0/8021560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc1595d0 0x7f9cbc06b080 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 -- 172.21.15.45:0/8021560 >> 172.21.15.45:0/8021560 conn(0x7f9cbc06f090 msgr2=0x7f9cbc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 -- 172.21.15.45:0/8021560 shutdown_connections 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.487+0000 7f9cc140e700 1 -- 172.21.15.45:0/8021560 wait complete. 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.488+0000 7f9cc140e700 1 Processor -- start 2023-10-06T05:17:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.488+0000 7f9cc140e700 1 -- start start 2023-10-06T05:17:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cc140e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cbc06b5c0 0x7f9cbc14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cc140e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc159200 0x7f9cbc0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cc140e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cbc1595d0 0x7f9cbc0c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cc140e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0b05a0 con 0x7f9cbc159200 2023-10-06T05:17:35.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cc140e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0cbe20 con 0x7f9cbc1595d0 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cc140e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0cc3e0 con 0x7f9cbc06b5c0 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cba7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc159200 0x7f9cbc0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cba7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc159200 0x7f9cbc0c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49936/0 (socket says 172.21.15.45:49936) 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.489+0000 7f9cba7fc700 1 -- 172.21.15.45:0/1348316703 learned_addr learned my addr 172.21.15.45:0/1348316703 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cbb7fe700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cbc1595d0 0x7f9cbc0c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cbaffd700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cbc06b5c0 0x7f9cbc14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cba7fc700 1 -- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cbc06b5c0 msgr2=0x7f9cbc14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cba7fc700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cbc06b5c0 0x7f9cbc14e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cba7fc700 1 -- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cbc1595d0 msgr2=0x7f9cbc0c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cba7fc700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cbc1595d0 0x7f9cbc0c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cba7fc700 1 -- 172.21.15.45:0/1348316703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40094f0 con 0x7f9cbc159200 2023-10-06T05:17:35.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cbaffd700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cbc06b5c0 0x7f9cbc14e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cba7fc700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc159200 0x7f9cbc0c29e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc06aed0 tx=0x7f9ca4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cb2ffd700 1 -- 172.21.15.45:0/1348316703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400b9d0 con 0x7f9cbc159200 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cb2ffd700 1 -- 172.21.15.45:0/1348316703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca400bb30 con 0x7f9cbc159200 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc14e7b0 con 0x7f9cbc159200 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cb2ffd700 1 -- 172.21.15.45:0/1348316703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4025670 con 0x7f9cbc159200 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.490+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc0cb8c0 con 0x7f9cbc159200 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.491+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc0cc590 con 0x7f9cbc159200 2023-10-06T05:17:35.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.492+0000 7f9cb2ffd700 1 -- 172.21.15.45:0/1348316703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ca4023070 con 0x7f9cbc159200 2023-10-06T05:17:35.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.492+0000 7f9cb2ffd700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9c9c066f30 0x7f9c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.492+0000 7f9cb2ffd700 1 -- 172.21.15.45:0/1348316703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9ca4085c40 con 0x7f9cbc159200 2023-10-06T05:17:35.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.492+0000 7f9cbaffd700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9c9c066f30 0x7f9c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.493+0000 7f9cbaffd700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9c9c066f30 0x7f9c9c0693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9cac007fd0 tx=0x7f9cac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.497+0000 7f9cb2ffd700 1 -- 172.21.15.45:0/1348316703 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cbc0cc590 con 0x7f9cbc159200 2023-10-06T05:17:35.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.744+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cbc0cc550 con 0x7f9cbc159200 2023-10-06T05:17:35.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.744+0000 7f9cb2ffd700 1 -- 172.21.15.45:0/1348316703 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f9ca4052c10 con 0x7f9cbc159200 2023-10-06T05:17:35.746 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:35.746 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:35.325485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2812290138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2812290138},{"type":"v1","addr":"172.21.15.45:6835","nonce":2812290138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:35.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.746+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9c9c066f30 msgr2=0x7f9c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.746+0000 7f9cc140e700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9c9c066f30 0x7f9c9c0693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9cac007fd0 tx=0x7f9cac009040 comp rx=0 tx=0).stop 2023-10-06T05:17:35.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.746+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc159200 msgr2=0x7f9cbc0c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.746+0000 7f9cc140e700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc159200 0x7f9cbc0c29e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc06aed0 tx=0x7f9ca4017000 comp rx=0 tx=0).stop 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 shutdown_connections 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9c9c066f30 0x7f9c9c0693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cbc1595d0 0x7f9cbc0c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cbc06b5c0 0x7f9cbc14e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 --2- 172.21.15.45:0/1348316703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cbc159200 0x7f9cbc0c29e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 >> 172.21.15.45:0/1348316703 conn(0x7f9cbc06f090 msgr2=0x7f9cbc0c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 shutdown_connections 2023-10-06T05:17:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.747+0000 7f9cc140e700 1 -- 172.21.15.45:0/1348316703 wait complete. 2023-10-06T05:17:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 33 2023-10-06T05:17:35.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.894+0000 7f8785eb5700 1 Processor -- start 2023-10-06T05:17:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.894+0000 7f8785eb5700 1 -- start start 2023-10-06T05:17:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f8785eb5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8780153650 0x7f878015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f8785eb5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878006afa0 0x7f8780068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f8785eb5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87800676c0 con 0x7f8780153a20 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f8785eb5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87800679a0 con 0x7f8780153650 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f8785eb5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8780067c80 con 0x7f878006afa0 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8780153650 0x7f878015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8780153650 0x7f878015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49950/0 (socket says 172.21.15.45:49950) 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877f7fe700 1 -- 172.21.15.45:0/1561765881 learned_addr learned my addr 172.21.15.45:0/1561765881 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877ffff700 1 --2- 172.21.15.45:0/1561765881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878006afa0 0x7f8780068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877f7fe700 1 -- 172.21.15.45:0/1561765881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878006afa0 msgr2=0x7f8780068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877f7fe700 1 --2- 172.21.15.45:0/1561765881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878006afa0 0x7f8780068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877f7fe700 1 -- 172.21.15.45:0/1561765881 >> v1:172.21.15.144:6789/0 conn(0x7f8780153a20 legacy=0x7f878006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877f7fe700 1 -- 172.21.15.45:0/1561765881 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8780067f60 con 0x7f8780153650 2023-10-06T05:17:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.895+0000 7f877ffff700 1 --2- 172.21.15.45:0/1561765881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878006afa0 0x7f8780068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f877f7fe700 1 --2- 172.21.15.45:0/1561765881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f878015a930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f87700099b0 tx=0x7f8770009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8aedf36defc429 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f877e7fc700 1 -- 172.21.15.45:0/1561765881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000b820 con 0x7f8780153650 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f877e7fc700 1 -- 172.21.15.45:0/1561765881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877000b980 con 0x7f8780153650 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f877e7fc700 1 -- 172.21.15.45:0/1561765881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000bd20 con 0x7f8780153650 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f8785eb5700 1 -- 172.21.15.45:0/1561765881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 msgr2=0x7f878015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f8785eb5700 1 --2- 172.21.15.45:0/1561765881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f878015a930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f87700099b0 tx=0x7f8770009250 comp rx=0 tx=0).stop 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f8785eb5700 1 -- 172.21.15.45:0/1561765881 shutdown_connections 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f8785eb5700 1 --2- 172.21.15.45:0/1561765881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878006afa0 0x7f8780068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f8785eb5700 1 --2- 172.21.15.45:0/1561765881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f878015a930 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.896+0000 7f8785eb5700 1 -- 172.21.15.45:0/1561765881 >> 172.21.15.45:0/1561765881 conn(0x7f87800cc5a0 msgr2=0x7f87800cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.897+0000 7f8785eb5700 1 -- 172.21.15.45:0/1561765881 shutdown_connections 2023-10-06T05:17:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.897+0000 7f8785eb5700 1 -- 172.21.15.45:0/1561765881 wait complete. 2023-10-06T05:17:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.898+0000 7f8785eb5700 1 Processor -- start 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.898+0000 7f8785eb5700 1 -- start start 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f8785eb5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878006afa0 0x7f87801523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f8785eb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f87800ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f8785eb5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8780153a20 0x7f87800adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f8785eb5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87800a88c0 con 0x7f8780153650 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f8785eb5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87800af560 con 0x7f878006afa0 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f8785eb5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87800af840 con 0x7f8780153a20 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f87800ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f87800ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49958/0 (socket says 172.21.15.45:49958) 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 -- 172.21.15.45:0/2469953449 learned_addr learned my addr 172.21.15.45:0/2469953449 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877f7fe700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878006afa0 0x7f87801523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877ffff700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8780153a20 0x7f87800adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 -- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8780153a20 msgr2=0x7f87800adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8780153a20 0x7f87800adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 -- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878006afa0 msgr2=0x7f87801523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878006afa0 0x7f87801523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 -- 172.21.15.45:0/2469953449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700094f0 con 0x7f8780153650 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877f7fe700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878006afa0 0x7f87801523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877ffff700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8780153a20 0x7f87800adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.899+0000 7f877effd700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f87800ada90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f876800c590 tx=0x7f8768007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.900+0000 7f87777fe700 1 -- 172.21.15.45:0/2469953449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768011590 con 0x7f8780153650 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.900+0000 7f87777fe700 1 -- 172.21.15.45:0/2469953449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768011730 con 0x7f8780153650 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.900+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87800c26d0 con 0x7f8780153650 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.900+0000 7f87777fe700 1 -- 172.21.15.45:0/2469953449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768011af0 con 0x7f8780153650 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.900+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87800c2c90 con 0x7f8780153650 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.901+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87800cc0e0 con 0x7f8780153650 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.901+0000 7f87777fe700 1 -- 172.21.15.45:0/2469953449 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f876801c460 con 0x7f8780153650 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.902+0000 7f87777fe700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8760066f30 0x7f87600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.902+0000 7f87777fe700 1 -- 172.21.15.45:0/2469953449 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8768084530 con 0x7f8780153650 2023-10-06T05:17:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.902+0000 7f877f7fe700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8760066f30 0x7f87600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.902+0000 7f877f7fe700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8760066f30 0x7f87600693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f87700099b0 tx=0x7f8770016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:35.906+0000 7f87777fe700 1 -- 172.21.15.45:0/2469953449 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8768051500 con 0x7f8780153650 2023-10-06T05:17:35.910 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.115+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f878015ae70 con 0x7f8780153650 2023-10-06T05:17:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.116+0000 7f87777fe700 1 -- 172.21.15.45:0/2469953449 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17167 (secure 0 0 0) 0x7f8768054b20 con 0x7f8780153650 2023-10-06T05:17:36.118 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:36.118 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:17:33.313237+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:29.616613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:30.718338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8760066f30 msgr2=0x7f87600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8760066f30 0x7f87600693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f87700099b0 tx=0x7f8770016000 comp rx=0 tx=0).stop 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 msgr2=0x7f87800ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f87800ada90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f876800c590 tx=0x7f8768007f30 comp rx=0 tx=0).stop 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 shutdown_connections 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8760066f30 0x7f87600693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8780153a20 0x7f87800adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878006afa0 0x7f87801523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 --2- 172.21.15.45:0/2469953449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8780153650 0x7f87800ada90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.118+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 >> 172.21.15.45:0/2469953449 conn(0x7f87800cc5a0 msgr2=0x7f87800c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.119+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 shutdown_connections 2023-10-06T05:17:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.119+0000 7f8785eb5700 1 -- 172.21.15.45:0/2469953449 wait complete. 2023-10-06T05:17:36.134 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:17:36.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:17:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.265+0000 7ffb118c4700 1 Processor -- start 2023-10-06T05:17:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.265+0000 7ffb118c4700 1 -- start start 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.265+0000 7ffb118c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.265+0000 7ffb118c4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb0c15d040 0x7ffb0c0aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.265+0000 7ffb118c4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c0ca6d0 con 0x7ffb0c159170 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.265+0000 7ffb118c4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c15d550 con 0x7ffb0c15d040 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.265+0000 7ffb118c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c15d810 con 0x7ffb0c0692d0 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb0c15d040 0x7ffb0c0aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffb0c159170 0x7ffb0c068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45788/0 (socket says 172.21.15.45:45788) 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0affd700 1 -- 172.21.15.45:0/415593055 learned_addr learned my addr 172.21.15.45:0/415593055 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0a7fc700 1 --2- 172.21.15.45:0/415593055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0b7fe700 1 -- 172.21.15.45:0/415593055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 msgr2=0x7ffb0c159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0b7fe700 1 --2- 172.21.15.45:0/415593055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0b7fe700 1 -- 172.21.15.45:0/415593055 >> v1:172.21.15.45:6789/0 conn(0x7ffb0c159170 legacy=0x7ffb0c068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb09ffb700 1 -- 172.21.15.45:0/415593055 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475388561 0 0) 0x7ffb0c0ca6d0 con 0x7ffb0c159170 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0b7fe700 1 -- 172.21.15.45:0/415593055 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c15daf0 con 0x7ffb0c15d040 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb0b7fe700 1 --2- 172.21.15.45:0/415593055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0aea60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009e60 tx=0x7ffb04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2fb1144168250c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb09ffb700 1 -- 172.21.15.45:0/415593055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e070 con 0x7ffb0c15d040 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb09ffb700 1 -- 172.21.15.45:0/415593055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0400bc30 con 0x7ffb0c15d040 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.266+0000 7ffb09ffb700 1 -- 172.21.15.45:0/415593055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04012430 con 0x7ffb0c15d040 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 -- 172.21.15.45:0/415593055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 msgr2=0x7ffb0c0aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 --2- 172.21.15.45:0/415593055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0aea60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009e60 tx=0x7ffb04009780 comp rx=0 tx=0).stop 2023-10-06T05:17:36.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 -- 172.21.15.45:0/415593055 shutdown_connections 2023-10-06T05:17:36.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 --2- 172.21.15.45:0/415593055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0aea60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 --2- 172.21.15.45:0/415593055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 -- 172.21.15.45:0/415593055 >> 172.21.15.45:0/415593055 conn(0x7ffb0c06f7c0 msgr2=0x7ffb0c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 -- 172.21.15.45:0/415593055 shutdown_connections 2023-10-06T05:17:36.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.267+0000 7ffb118c4700 1 -- 172.21.15.45:0/415593055 wait complete. 2023-10-06T05:17:36.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.268+0000 7ffb118c4700 1 Processor -- start 2023-10-06T05:17:36.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.268+0000 7ffb118c4700 1 -- start start 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb118c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb118c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb0c159170 0x7ffb0c1692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb118c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb118c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0c90a0 con 0x7ffb0c15d040 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb118c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0a6b80 con 0x7ffb0c159170 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb118c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0a6e60 con 0x7ffb0c0692d0 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49978/0 (socket says 172.21.15.45:49978) 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.269+0000 7ffb0b7fe700 1 -- 172.21.15.45:0/1485331194 learned_addr learned my addr 172.21.15.45:0/1485331194 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0affd700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0a7fc700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb0c159170 0x7ffb0c1692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0b7fe700 1 -- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 msgr2=0x7ffb0c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0b7fe700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0b7fe700 1 -- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb0c159170 msgr2=0x7ffb0c1692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0b7fe700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb0c159170 0x7ffb0c1692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0b7fe700 1 -- 172.21.15.45:0/1485331194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb04009a20 con 0x7ffb0c15d040 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb0b7fe700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0c7b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffb04013000 tx=0x7ffb04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffaf2ffd700 1 -- 172.21.15.45:0/1485331194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04012e50 con 0x7ffb0c15d040 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffaf2ffd700 1 -- 172.21.15.45:0/1485331194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0401d630 con 0x7ffb0c15d040 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c0a7140 con 0x7ffb0c15d040 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffaf2ffd700 1 -- 172.21.15.45:0/1485331194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0401da10 con 0x7ffb0c15d040 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.270+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c0a7700 con 0x7ffb0c15d040 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.271+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0a78f0 con 0x7ffb0c15d040 2023-10-06T05:17:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.272+0000 7ffaf2ffd700 1 -- 172.21.15.45:0/1485331194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb04024070 con 0x7ffb0c15d040 2023-10-06T05:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.272+0000 7ffaf2ffd700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffaf8066f30 0x7ffaf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.272+0000 7ffaf2ffd700 1 -- 172.21.15.45:0/1485331194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ffb04087070 con 0x7ffb0c15d040 2023-10-06T05:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.272+0000 7ffb0affd700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffaf8066f30 0x7ffaf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.273+0000 7ffb0affd700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffaf8066f30 0x7ffaf80693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0011e0 tx=0x7ffafc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.277+0000 7ffaf2ffd700 1 -- 172.21.15.45:0/1485331194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb040540f0 con 0x7ffb0c15d040 2023-10-06T05:17:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.476+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb0c063b50 con 0x7ffaf8066f30 2023-10-06T05:17:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.486+0000 7ffaf2ffd700 1 -- 172.21.15.45:0/1485331194 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311743 (secure 0 0 0) 0x7ffb0c063b50 con 0x7ffaf8066f30 2023-10-06T05:17:36.493 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:36.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffaf8066f30 msgr2=0x7ffaf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffaf8066f30 0x7ffaf80693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0011e0 tx=0x7ffafc009040 comp rx=0 tx=0).stop 2023-10-06T05:17:36.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 msgr2=0x7ffb0c0c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0c7b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffb04013000 tx=0x7ffb04019000 comp rx=0 tx=0).stop 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 shutdown_connections 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffaf8066f30 0x7ffaf80693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb0c15d040 0x7ffb0c0c7b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb0c0692d0 0x7ffb0c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 --2- 172.21.15.45:0/1485331194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb0c159170 0x7ffb0c1692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.489+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 >> 172.21.15.45:0/1485331194 conn(0x7ffb0c06f7c0 msgr2=0x7ffb0c0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.490+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 shutdown_connections 2023-10-06T05:17:36.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.490+0000 7ffb118c4700 1 -- 172.21.15.45:0/1485331194 wait complete. 2023-10-06T05:17:36.498 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:17:36.505 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-10-06T05:17:34.479013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384100,"kb_used_data":5796,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590620,"statfs":{"total":773094113280,"available":770652794880,"internally_reserved":0,"allocated":5935104,"data_stored":3370175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.175820"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.334500+0000","last_change":"2023-10-06T05:17:32.334500+0000","last_active":"2023-10-06T05:17:32.334500+0000","last_peered":"2023-10-06T05:17:32.334500+0000","last_clean":"2023-10-06T05:17:32.334500+0000","last_became_active":"2023-10-06T05:17:32.334239+0000","last_became_peered":"2023-10-06T05:17:32.334239+0000","last_unstale":"2023-10-06T05:17:32.334500+0000","last_undegraded":"2023-10-06T05:17:32.334500+0000","last_fullsized":"2023-10-06T05:17:32.334500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:32.572071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.346034+0000","last_change":"2023-10-06T05:17:32.346034+0000","last_active":"2023-10-06T05:17:32.346034+0000","last_peered":"2023-10-06T05:17:32.346034+0000","last_clean":"2023-10-06T05:17:32.346034+0000","last_became_active":"2023-10-06T05:17:32.345723+0000","last_became_peered":"2023-10-06T05:17:32.345723+0000","last_unstale":"2023-10-06T05:17:32.346034+0000","last_undegraded":"2023-10-06T05:17:32.346034+0000","last_fullsized":"2023-10-06T05:17:32.346034+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:17.642543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325291+0000","last_change":"2023-10-06T05:17:32.325291+0000","last_active":"2023-10-06T05:17:32.325291+0000","last_peered":"2023-10-06T05:17:32.325291+0000","last_clean":"2023-10-06T05:17:32.325291+0000","last_became_active":"2023-10-06T05:17:32.325038+0000","last_became_peered":"2023-10-06T05:17:32.325038+0000","last_unstale":"2023-10-06T05:17:32.325291+0000","last_undegraded":"2023-10-06T05:17:32.325291+0000","last_fullsized":"2023-10-06T05:17:32.325291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:41.772378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.327772+0000","last_change":"2023-10-06T05:17:32.327772+0000","last_active":"2023-10-06T05:17:32.327772+0000","last_peered":"2023-10-06T05:17:32.327772+0000","last_clean":"2023-10-06T05:17:32.327772+0000","last_became_active":"2023-10-06T05:17:32.327473+0000","last_became_peered":"2023-10-06T05:17:32.327473+0000","last_unstale":"2023-10-06T05:17:32.327772+0000","last_undegraded":"2023-10-06T05:17:32.327772+0000","last_fullsized":"2023-10-06T05:17:32.327772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:56.585110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323088+0000","last_change":"2023-10-06T05:17:32.323088+0000","last_active":"2023-10-06T05:17:32.323088+0000","last_peered":"2023-10-06T05:17:32.323088+0000","last_clean":"2023-10-06T05:17:32.323088+0000","last_became_active":"2023-10-06T05:17:32.322807+0000","last_became_peered":"2023-10-06T05:17:32.322807+0000","last_unstale":"2023-10-06T05:17:32.323088+0000","last_undegraded":"2023-10-06T05:17:32.323088+0000","last_fullsized":"2023-10-06T05:17:32.323088+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:18.830245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329826+0000","last_change":"2023-10-06T05:17:32.329826+0000","last_active":"2023-10-06T05:17:32.329826+0000","last_peered":"2023-10-06T05:17:32.329826+0000","last_clean":"2023-10-06T05:17:32.329826+0000","last_became_active":"2023-10-06T05:17:32.329526+0000","last_became_peered":"2023-10-06T05:17:32.329526+0000","last_unstale":"2023-10-06T05:17:32.329826+0000","last_undegraded":"2023-10-06T05:17:32.329826+0000","last_fullsized":"2023-10-06T05:17:32.329826+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:44.325973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323572+0000","last_change":"2023-10-06T05:17:32.323572+0000","last_active":"2023-10-06T05:17:32.323572+0000","last_peered":"2023-10-06T05:17:32.323572+0000","last_clean":"2023-10-06T05:17:32.323572+0000","last_became_active":"2023-10-06T05:17:32.323302+0000","last_became_peered":"2023-10-06T05:17:32.323302+0000","last_unstale":"2023-10-06T05:17:32.323572+0000","last_undegraded":"2023-10-06T05:17:32.323572+0000","last_fullsized":"2023-10-06T05:17:32.323572+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:31.332500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.331770+0000","last_change":"2023-10-06T05:17:32.331770+0000","last_active":"2023-10-06T05:17:32.331770+0000","last_peered":"2023-10-06T05:17:32.331770+0000","last_clean":"2023-10-06T05:17:32.331770+0000","last_became_active":"2023-10-06T05:17:32.327662+0000","last_became_peered":"2023-10-06T05:17:32.327662+0000","last_unstale":"2023-10-06T05:17:32.331770+0000","last_undegraded":"2023-10-06T05:17:32.331770+0000","last_fullsized":"2023-10-06T05:17:32.331770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.280650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324379+0000","last_change":"2023-10-06T05:17:32.324379+0000","last_active":"2023-10-06T05:17:32.324379+0000","last_peered":"2023-10-06T05:17:32.324379+0000","last_clean":"2023-10-06T05:17:32.324379+0000","last_became_active":"2023-10-06T05:17:32.324034+0000","last_became_peered":"2023-10-06T05:17:32.324034+0000","last_unstale":"2023-10-06T05:17:32.324379+0000","last_undegraded":"2023-10-06T05:17:32.324379+0000","last_fullsized":"2023-10-06T05:17:32.324379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:19.977222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.333471+0000","last_change":"2023-10-06T05:17:32.333471+0000","last_active":"2023-10-06T05:17:32.333471+0000","last_peered":"2023-10-06T05:17:32.333471+0000","last_clean":"2023-10-06T05:17:32.333471+0000","last_became_active":"2023-10-06T05:17:32.333154+0000","last_became_peered":"2023-10-06T05:17:32.333154+0000","last_unstale":"2023-10-06T05:17:32.333471+0000","last_undegraded":"2023-10-06T05:17:32.333471+0000","last_fullsized":"2023-10-06T05:17:32.333471+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:56.070562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.344067+0000","last_change":"2023-10-06T05:17:32.344067+0000","last_active":"2023-10-06T05:17:32.344067+0000","last_peered":"2023-10-06T05:17:32.344067+0000","last_clean":"2023-10-06T05:17:32.344067+0000","last_became_active":"2023-10-06T05:17:32.343786+0000","last_became_peered":"2023-10-06T05:17:32.343786+0000","last_unstale":"2023-10-06T05:17:32.344067+0000","last_undegraded":"2023-10-06T05:17:32.344067+0000","last_fullsized":"2023-10-06T05:17:32.344067+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:59.032559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325125+0000","last_change":"2023-10-06T05:17:32.325125+0000","last_active":"2023-10-06T05:17:32.325125+0000","last_peered":"2023-10-06T05:17:32.325125+0000","last_clean":"2023-10-06T05:17:32.325125+0000","last_became_active":"2023-10-06T05:17:32.324509+0000","last_became_peered":"2023-10-06T05:17:32.324509+0000","last_unstale":"2023-10-06T05:17:32.325125+0000","last_undegraded":"2023-10-06T05:17:32.325125+0000","last_fullsized":"2023-10-06T05:17:32.325125+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:05.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.335440+0000","last_change":"2023-10-06T05:17:32.335440+0000","last_active":"2023-10-06T05:17:32.335440+0000","last_peered":"2023-10-06T05:17:32.335440+0000","last_clean":"2023-10-06T05:17:32.335440+0000","last_became_active":"2023-10-06T05:17:32.335034+0000","last_became_peered":"2023-10-06T05:17:32.335034+0000","last_unstale":"2023-10-06T05:17:32.335440+0000","last_undegraded":"2023-10-06T05:17:32.335440+0000","last_fullsized":"2023-10-06T05:17:32.335440+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:54.692661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.314125+0000","last_change":"2023-10-06T05:17:32.314125+0000","last_active":"2023-10-06T05:17:32.314125+0000","last_peered":"2023-10-06T05:17:32.314125+0000","last_clean":"2023-10-06T05:17:32.314125+0000","last_became_active":"2023-10-06T05:17:32.313854+0000","last_became_peered":"2023-10-06T05:17:32.313854+0000","last_unstale":"2023-10-06T05:17:32.314125+0000","last_undegraded":"2023-10-06T05:17:32.314125+0000","last_fullsized":"2023-10-06T05:17:32.314125+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:05.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.331920+0000","last_change":"2023-10-06T05:17:32.331920+0000","last_active":"2023-10-06T05:17:32.331920+0000","last_peered":"2023-10-06T05:17:32.331920+0000","last_clean":"2023-10-06T05:17:32.331920+0000","last_became_active":"2023-10-06T05:17:32.331621+0000","last_became_peered":"2023-10-06T05:17:32.331621+0000","last_unstale":"2023-10-06T05:17:32.331920+0000","last_undegraded":"2023-10-06T05:17:32.331920+0000","last_fullsized":"2023-10-06T05:17:32.331920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:52.892989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332877+0000","last_change":"2023-10-06T05:17:32.332877+0000","last_active":"2023-10-06T05:17:32.332877+0000","last_peered":"2023-10-06T05:17:32.332877+0000","last_clean":"2023-10-06T05:17:32.332877+0000","last_became_active":"2023-10-06T05:17:32.332189+0000","last_became_peered":"2023-10-06T05:17:32.332189+0000","last_unstale":"2023-10-06T05:17:32.332877+0000","last_undegraded":"2023-10-06T05:17:32.332877+0000","last_fullsized":"2023-10-06T05:17:32.332877+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:25.110866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.321581+0000","last_change":"2023-10-06T05:17:32.321581+0000","last_active":"2023-10-06T05:17:32.321581+0000","last_peered":"2023-10-06T05:17:32.321581+0000","last_clean":"2023-10-06T05:17:32.321581+0000","last_became_active":"2023-10-06T05:17:32.320558+0000","last_became_peered":"2023-10-06T05:17:32.320558+0000","last_unstale":"2023-10-06T05:17:32.321581+0000","last_undegraded":"2023-10-06T05:17:32.321581+0000","last_fullsized":"2023-10-06T05:17:32.321581+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:40.423307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.326357+0000","last_change":"2023-10-06T05:17:32.326357+0000","last_active":"2023-10-06T05:17:32.326357+0000","last_peered":"2023-10-06T05:17:32.326357+0000","last_clean":"2023-10-06T05:17:32.326357+0000","last_became_active":"2023-10-06T05:17:32.325958+0000","last_became_peered":"2023-10-06T05:17:32.325958+0000","last_unstale":"2023-10-06T05:17:32.326357+0000","last_undegraded":"2023-10-06T05:17:32.326357+0000","last_fullsized":"2023-10-06T05:17:32.326357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.302073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.328283+0000","last_change":"2023-10-06T05:17:32.328283+0000","last_active":"2023-10-06T05:17:32.328283+0000","last_peered":"2023-10-06T05:17:32.328283+0000","last_clean":"2023-10-06T05:17:32.328283+0000","last_became_active":"2023-10-06T05:17:32.327962+0000","last_became_peered":"2023-10-06T05:17:32.327962+0000","last_unstale":"2023-10-06T05:17:32.328283+0000","last_undegraded":"2023-10-06T05:17:32.328283+0000","last_fullsized":"2023-10-06T05:17:32.328283+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:34.940743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325882+0000","last_change":"2023-10-06T05:17:32.325882+0000","last_active":"2023-10-06T05:17:32.325882+0000","last_peered":"2023-10-06T05:17:32.325882+0000","last_clean":"2023-10-06T05:17:32.325882+0000","last_became_active":"2023-10-06T05:17:32.325599+0000","last_became_peered":"2023-10-06T05:17:32.325599+0000","last_unstale":"2023-10-06T05:17:32.325882+0000","last_undegraded":"2023-10-06T05:17:32.325882+0000","last_fullsized":"2023-10-06T05:17:32.325882+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:35.025622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324070+0000","last_change":"2023-10-06T05:17:32.324070+0000","last_active":"2023-10-06T05:17:32.324070+0000","last_peered":"2023-10-06T05:17:32.324070+0000","last_clean":"2023-10-06T05:17:32.324070+0000","last_became_active":"2023-10-06T05:17:32.323639+0000","last_became_peered":"2023-10-06T05:17:32.323639+0000","last_unstale":"2023-10-06T05:17:32.324070+0000","last_undegraded":"2023-10-06T05:17:32.324070+0000","last_fullsized":"2023-10-06T05:17:32.324070+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:32.942267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.330750+0000","last_change":"2023-10-06T05:17:32.330750+0000","last_active":"2023-10-06T05:17:32.330750+0000","last_peered":"2023-10-06T05:17:32.330750+0000","last_clean":"2023-10-06T05:17:32.330750+0000","last_became_active":"2023-10-06T05:17:32.330145+0000","last_became_peered":"2023-10-06T05:17:32.330145+0000","last_unstale":"2023-10-06T05:17:32.330750+0000","last_undegraded":"2023-10-06T05:17:32.330750+0000","last_fullsized":"2023-10-06T05:17:32.330750+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:38.341979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.336566+0000","last_change":"2023-10-06T05:17:32.336566+0000","last_active":"2023-10-06T05:17:32.336566+0000","last_peered":"2023-10-06T05:17:32.336566+0000","last_clean":"2023-10-06T05:17:32.336566+0000","last_became_active":"2023-10-06T05:17:32.333605+0000","last_became_peered":"2023-10-06T05:17:32.333605+0000","last_unstale":"2023-10-06T05:17:32.336566+0000","last_undegraded":"2023-10-06T05:17:32.336566+0000","last_fullsized":"2023-10-06T05:17:32.336566+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329629+0000","last_change":"2023-10-06T05:17:32.329629+0000","last_active":"2023-10-06T05:17:32.329629+0000","last_peered":"2023-10-06T05:17:32.329629+0000","last_clean":"2023-10-06T05:17:32.329629+0000","last_became_active":"2023-10-06T05:17:32.329327+0000","last_became_peered":"2023-10-06T05:17:32.329327+0000","last_unstale":"2023-10-06T05:17:32.329629+0000","last_undegraded":"2023-10-06T05:17:32.329629+0000","last_fullsized":"2023-10-06T05:17:32.329629+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:56.070562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.333084+0000","last_change":"2023-10-06T05:17:32.333084+0000","last_active":"2023-10-06T05:17:32.333084+0000","last_peered":"2023-10-06T05:17:32.333084+0000","last_clean":"2023-10-06T05:17:32.333084+0000","last_became_active":"2023-10-06T05:17:32.332819+0000","last_became_peered":"2023-10-06T05:17:32.332819+0000","last_unstale":"2023-10-06T05:17:32.333084+0000","last_undegraded":"2023-10-06T05:17:32.333084+0000","last_fullsized":"2023-10-06T05:17:32.333084+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:16.444290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.326239+0000","last_change":"2023-10-06T05:17:32.326239+0000","last_active":"2023-10-06T05:17:32.326239+0000","last_peered":"2023-10-06T05:17:32.326239+0000","last_clean":"2023-10-06T05:17:32.326239+0000","last_became_active":"2023-10-06T05:17:32.325839+0000","last_became_peered":"2023-10-06T05:17:32.325839+0000","last_unstale":"2023-10-06T05:17:32.326239+0000","last_undegraded":"2023-10-06T05:17:32.326239+0000","last_fullsized":"2023-10-06T05:17:32.326239+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.302073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.328966+0000","last_change":"2023-10-06T05:17:32.328966+0000","last_active":"2023-10-06T05:17:32.328966+0000","last_peered":"2023-10-06T05:17:32.328966+0000","last_clean":"2023-10-06T05:17:32.328966+0000","last_became_active":"2023-10-06T05:17:32.328642+0000","last_became_peered":"2023-10-06T05:17:32.328642+0000","last_unstale":"2023-10-06T05:17:32.328966+0000","last_undegraded":"2023-10-06T05:17:32.328966+0000","last_fullsized":"2023-10-06T05:17:32.328966+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:17.642543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323743+0000","last_change":"2023-10-06T05:17:32.323743+0000","last_active":"2023-10-06T05:17:32.323743+0000","last_peered":"2023-10-06T05:17:32.323743+0000","last_clean":"2023-10-06T05:17:32.323743+0000","last_became_active":"2023-10-06T05:17:32.323448+0000","last_became_peered":"2023-10-06T05:17:32.323448+0000","last_unstale":"2023-10-06T05:17:32.323743+0000","last_undegraded":"2023-10-06T05:17:32.323743+0000","last_fullsized":"2023-10-06T05:17:32.323743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:25.465841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.335342+0000","last_change":"2023-10-06T05:17:32.335342+0000","last_active":"2023-10-06T05:17:32.335342+0000","last_peered":"2023-10-06T05:17:32.335342+0000","last_clean":"2023-10-06T05:17:32.335342+0000","last_became_active":"2023-10-06T05:17:32.334962+0000","last_became_peered":"2023-10-06T05:17:32.334962+0000","last_unstale":"2023-10-06T05:17:32.335342+0000","last_undegraded":"2023-10-06T05:17:32.335342+0000","last_fullsized":"2023-10-06T05:17:32.335342+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:15.535704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.326336+0000","last_change":"2023-10-06T05:17:32.326336+0000","last_active":"2023-10-06T05:17:32.326336+0000","last_peered":"2023-10-06T05:17:32.326336+0000","last_clean":"2023-10-06T05:17:32.326336+0000","last_became_active":"2023-10-06T05:17:32.325403+0000","last_became_peered":"2023-10-06T05:17:32.325403+0000","last_unstale":"2023-10-06T05:17:32.326336+0000","last_undegraded":"2023-10-06T05:17:32.326336+0000","last_fullsized":"2023-10-06T05:17:32.326336+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:02.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332269+0000","last_change":"2023-10-06T05:17:32.332269+0000","last_active":"2023-10-06T05:17:32.332269+0000","last_peered":"2023-10-06T05:17:32.332269+0000","last_clean":"2023-10-06T05:17:32.332269+0000","last_became_active":"2023-10-06T05:17:32.331989+0000","last_became_peered":"2023-10-06T05:17:32.331989+0000","last_unstale":"2023-10-06T05:17:32.332269+0000","last_undegraded":"2023-10-06T05:17:32.332269+0000","last_fullsized":"2023-10-06T05:17:32.332269+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:11.282204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.338712+0000","last_change":"2023-10-06T05:17:32.338712+0000","last_active":"2023-10-06T05:17:32.338712+0000","last_peered":"2023-10-06T05:17:32.338712+0000","last_clean":"2023-10-06T05:17:32.338712+0000","last_became_active":"2023-10-06T05:17:32.332599+0000","last_became_peered":"2023-10-06T05:17:32.332599+0000","last_unstale":"2023-10-06T05:17:32.338712+0000","last_undegraded":"2023-10-06T05:17:32.338712+0000","last_fullsized":"2023-10-06T05:17:32.338712+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.860772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324919+0000","last_change":"2023-10-06T05:17:32.324919+0000","last_active":"2023-10-06T05:17:32.324919+0000","last_peered":"2023-10-06T05:17:32.324919+0000","last_clean":"2023-10-06T05:17:32.324919+0000","last_became_active":"2023-10-06T05:17:32.324534+0000","last_became_peered":"2023-10-06T05:17:32.324534+0000","last_unstale":"2023-10-06T05:17:32.324919+0000","last_undegraded":"2023-10-06T05:17:32.324919+0000","last_fullsized":"2023-10-06T05:17:32.324919+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:31.332500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.343701+0000","last_change":"2023-10-06T05:17:31.324008+0000","last_active":"2023-10-06T05:17:32.343701+0000","last_peered":"2023-10-06T05:17:32.343701+0000","last_clean":"2023-10-06T05:17:32.343701+0000","last_became_active":"2023-10-06T05:17:31.322815+0000","last_became_peered":"2023-10-06T05:17:31.322815+0000","last_unstale":"2023-10-06T05:17:32.343701+0000","last_undegraded":"2023-10-06T05:17:32.343701+0000","last_fullsized":"2023-10-06T05:17:32.343701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:48.854489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.343670+0000","last_change":"2023-10-06T05:17:32.343670+0000","last_active":"2023-10-06T05:17:32.343670+0000","last_peered":"2023-10-06T05:17:32.343670+0000","last_clean":"2023-10-06T05:17:32.343670+0000","last_became_active":"2023-10-06T05:17:32.343423+0000","last_became_peered":"2023-10-06T05:17:32.343423+0000","last_unstale":"2023-10-06T05:17:32.343670+0000","last_undegraded":"2023-10-06T05:17:32.343670+0000","last_fullsized":"2023-10-06T05:17:32.343670+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:19.889711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.314480+0000","last_change":"2023-10-06T05:17:31.330260+0000","last_active":"2023-10-06T05:17:32.314480+0000","last_peered":"2023-10-06T05:17:32.314480+0000","last_clean":"2023-10-06T05:17:32.314480+0000","last_became_active":"2023-10-06T05:17:31.329970+0000","last_became_peered":"2023-10-06T05:17:31.329970+0000","last_unstale":"2023-10-06T05:17:32.314480+0000","last_undegraded":"2023-10-06T05:17:32.314480+0000","last_fullsized":"2023-10-06T05:17:32.314480+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:40.767026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.333720+0000","last_change":"2023-10-06T05:17:32.333720+0000","last_active":"2023-10-06T05:17:32.333720+0000","last_peered":"2023-10-06T05:17:32.333720+0000","last_clean":"2023-10-06T05:17:32.333720+0000","last_became_active":"2023-10-06T05:17:32.332907+0000","last_became_peered":"2023-10-06T05:17:32.332907+0000","last_unstale":"2023-10-06T05:17:32.333720+0000","last_undegraded":"2023-10-06T05:17:32.333720+0000","last_fullsized":"2023-10-06T05:17:32.333720+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:53.606888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.346753+0000","last_change":"2023-10-06T05:17:31.322504+0000","last_active":"2023-10-06T05:17:32.346753+0000","last_peered":"2023-10-06T05:17:32.346753+0000","last_clean":"2023-10-06T05:17:32.346753+0000","last_became_active":"2023-10-06T05:17:31.322207+0000","last_became_peered":"2023-10-06T05:17:31.322207+0000","last_unstale":"2023-10-06T05:17:32.346753+0000","last_undegraded":"2023-10-06T05:17:32.346753+0000","last_fullsized":"2023-10-06T05:17:32.346753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:55.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323919+0000","last_change":"2023-10-06T05:17:32.323919+0000","last_active":"2023-10-06T05:17:32.323919+0000","last_peered":"2023-10-06T05:17:32.323919+0000","last_clean":"2023-10-06T05:17:32.323919+0000","last_became_active":"2023-10-06T05:17:32.323695+0000","last_became_peered":"2023-10-06T05:17:32.323695+0000","last_unstale":"2023-10-06T05:17:32.323919+0000","last_undegraded":"2023-10-06T05:17:32.323919+0000","last_fullsized":"2023-10-06T05:17:32.323919+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:20.624998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.337958+0000","last_change":"2023-10-06T05:17:31.321969+0000","last_active":"2023-10-06T05:17:32.337958+0000","last_peered":"2023-10-06T05:17:32.337958+0000","last_clean":"2023-10-06T05:17:32.337958+0000","last_became_active":"2023-10-06T05:17:31.321689+0000","last_became_peered":"2023-10-06T05:17:31.321689+0000","last_unstale":"2023-10-06T05:17:32.337958+0000","last_undegraded":"2023-10-06T05:17:32.337958+0000","last_fullsized":"2023-10-06T05:17:32.337958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:14.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.330086+0000","last_change":"2023-10-06T05:17:32.330086+0000","last_active":"2023-10-06T05:17:32.330086+0000","last_peered":"2023-10-06T05:17:32.330086+0000","last_clean":"2023-10-06T05:17:32.330086+0000","last_became_active":"2023-10-06T05:17:32.327658+0000","last_became_peered":"2023-10-06T05:17:32.327658+0000","last_unstale":"2023-10-06T05:17:32.330086+0000","last_undegraded":"2023-10-06T05:17:32.330086+0000","last_fullsized":"2023-10-06T05:17:32.330086+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:25.110866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.310009+0000","last_change":"2023-10-06T05:17:31.316565+0000","last_active":"2023-10-06T05:17:32.310009+0000","last_peered":"2023-10-06T05:17:32.310009+0000","last_clean":"2023-10-06T05:17:32.310009+0000","last_became_active":"2023-10-06T05:17:31.316281+0000","last_became_peered":"2023-10-06T05:17:31.316281+0000","last_unstale":"2023-10-06T05:17:32.310009+0000","last_undegraded":"2023-10-06T05:17:32.310009+0000","last_fullsized":"2023-10-06T05:17:32.310009+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:24.897513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.331320+0000","last_change":"2023-10-06T05:17:32.331320+0000","last_active":"2023-10-06T05:17:32.331320+0000","last_peered":"2023-10-06T05:17:32.331320+0000","last_clean":"2023-10-06T05:17:32.331320+0000","last_became_active":"2023-10-06T05:17:32.331054+0000","last_became_peered":"2023-10-06T05:17:32.331054+0000","last_unstale":"2023-10-06T05:17:32.331320+0000","last_undegraded":"2023-10-06T05:17:32.331320+0000","last_fullsized":"2023-10-06T05:17:32.331320+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:11.707587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.315754+0000","last_change":"2023-10-06T05:17:31.326372+0000","last_active":"2023-10-06T05:17:32.315754+0000","last_peered":"2023-10-06T05:17:32.315754+0000","last_clean":"2023-10-06T05:17:32.315754+0000","last_became_active":"2023-10-06T05:17:31.325435+0000","last_became_peered":"2023-10-06T05:17:31.325435+0000","last_unstale":"2023-10-06T05:17:32.315754+0000","last_undegraded":"2023-10-06T05:17:32.315754+0000","last_fullsized":"2023-10-06T05:17:32.315754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:39.560479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324268+0000","last_change":"2023-10-06T05:17:32.324268+0000","last_active":"2023-10-06T05:17:32.324268+0000","last_peered":"2023-10-06T05:17:32.324268+0000","last_clean":"2023-10-06T05:17:32.324268+0000","last_became_active":"2023-10-06T05:17:32.323996+0000","last_became_peered":"2023-10-06T05:17:32.323996+0000","last_unstale":"2023-10-06T05:17:32.324268+0000","last_undegraded":"2023-10-06T05:17:32.324268+0000","last_fullsized":"2023-10-06T05:17:32.324268+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:32.572071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.325989+0000","last_change":"2023-10-06T05:17:31.322930+0000","last_active":"2023-10-06T05:17:31.325989+0000","last_peered":"2023-10-06T05:17:31.325989+0000","last_clean":"2023-10-06T05:17:31.325989+0000","last_became_active":"2023-10-06T05:17:31.322266+0000","last_became_peered":"2023-10-06T05:17:31.322266+0000","last_unstale":"2023-10-06T05:17:31.325989+0000","last_undegraded":"2023-10-06T05:17:31.325989+0000","last_fullsized":"2023-10-06T05:17:31.325989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:01.814012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.321836+0000","last_change":"2023-10-06T05:17:32.321836+0000","last_active":"2023-10-06T05:17:32.321836+0000","last_peered":"2023-10-06T05:17:32.321836+0000","last_clean":"2023-10-06T05:17:32.321836+0000","last_became_active":"2023-10-06T05:17:32.321344+0000","last_became_peered":"2023-10-06T05:17:32.321344+0000","last_unstale":"2023-10-06T05:17:32.321836+0000","last_undegraded":"2023-10-06T05:17:32.321836+0000","last_fullsized":"2023-10-06T05:17:32.321836+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:13.083592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.332323+0000","last_change":"2023-10-06T05:17:31.331270+0000","last_active":"2023-10-06T05:17:31.332323+0000","last_peered":"2023-10-06T05:17:31.332323+0000","last_clean":"2023-10-06T05:17:31.332323+0000","last_became_active":"2023-10-06T05:17:31.331015+0000","last_became_peered":"2023-10-06T05:17:31.331015+0000","last_unstale":"2023-10-06T05:17:31.332323+0000","last_undegraded":"2023-10-06T05:17:31.332323+0000","last_fullsized":"2023-10-06T05:17:31.332323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:38.572266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.346173+0000","last_change":"2023-10-06T05:17:32.346173+0000","last_active":"2023-10-06T05:17:32.346173+0000","last_peered":"2023-10-06T05:17:32.346173+0000","last_clean":"2023-10-06T05:17:32.346173+0000","last_became_active":"2023-10-06T05:17:32.345913+0000","last_became_peered":"2023-10-06T05:17:32.345913+0000","last_unstale":"2023-10-06T05:17:32.346173+0000","last_undegraded":"2023-10-06T05:17:32.346173+0000","last_fullsized":"2023-10-06T05:17:32.346173+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:46.301441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.328628+0000","last_change":"2023-10-06T05:17:31.328228+0000","last_active":"2023-10-06T05:17:31.328628+0000","last_peered":"2023-10-06T05:17:31.328628+0000","last_clean":"2023-10-06T05:17:31.328628+0000","last_became_active":"2023-10-06T05:17:31.327992+0000","last_became_peered":"2023-10-06T05:17:31.327992+0000","last_unstale":"2023-10-06T05:17:31.328628+0000","last_undegraded":"2023-10-06T05:17:31.328628+0000","last_fullsized":"2023-10-06T05:17:31.328628+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:52.569607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332340+0000","last_change":"2023-10-06T05:17:32.332340+0000","last_active":"2023-10-06T05:17:32.332340+0000","last_peered":"2023-10-06T05:17:32.332340+0000","last_clean":"2023-10-06T05:17:32.332340+0000","last_became_active":"2023-10-06T05:17:32.332074+0000","last_became_peered":"2023-10-06T05:17:32.332074+0000","last_unstale":"2023-10-06T05:17:32.332340+0000","last_undegraded":"2023-10-06T05:17:32.332340+0000","last_fullsized":"2023-10-06T05:17:32.332340+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:39.577618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.332449+0000","last_change":"2023-10-06T05:17:31.325650+0000","last_active":"2023-10-06T05:17:31.332449+0000","last_peered":"2023-10-06T05:17:31.332449+0000","last_clean":"2023-10-06T05:17:31.332449+0000","last_became_active":"2023-10-06T05:17:31.325353+0000","last_became_peered":"2023-10-06T05:17:31.325353+0000","last_unstale":"2023-10-06T05:17:31.332449+0000","last_undegraded":"2023-10-06T05:17:31.332449+0000","last_fullsized":"2023-10-06T05:17:31.332449+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:38.572266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332870+0000","last_change":"2023-10-06T05:17:32.332870+0000","last_active":"2023-10-06T05:17:32.332870+0000","last_peered":"2023-10-06T05:17:32.332870+0000","last_clean":"2023-10-06T05:17:32.332870+0000","last_became_active":"2023-10-06T05:17:32.332435+0000","last_became_peered":"2023-10-06T05:17:32.332435+0000","last_unstale":"2023-10-06T05:17:32.332870+0000","last_undegraded":"2023-10-06T05:17:32.332870+0000","last_fullsized":"2023-10-06T05:17:32.332870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:53.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.345896+0000","last_change":"2023-10-06T05:17:31.325306+0000","last_active":"2023-10-06T05:17:32.345896+0000","last_peered":"2023-10-06T05:17:32.345896+0000","last_clean":"2023-10-06T05:17:32.345896+0000","last_became_active":"2023-10-06T05:17:31.324963+0000","last_became_peered":"2023-10-06T05:17:31.324963+0000","last_unstale":"2023-10-06T05:17:32.345896+0000","last_undegraded":"2023-10-06T05:17:32.345896+0000","last_fullsized":"2023-10-06T05:17:32.345896+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:58.027207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332761+0000","last_change":"2023-10-06T05:17:32.332761+0000","last_active":"2023-10-06T05:17:32.332761+0000","last_peered":"2023-10-06T05:17:32.332761+0000","last_clean":"2023-10-06T05:17:32.332761+0000","last_became_active":"2023-10-06T05:17:32.332306+0000","last_became_peered":"2023-10-06T05:17:32.332306+0000","last_unstale":"2023-10-06T05:17:32.332761+0000","last_undegraded":"2023-10-06T05:17:32.332761+0000","last_fullsized":"2023-10-06T05:17:32.332761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:22.438165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.344001+0000","last_change":"2023-10-06T05:17:31.325799+0000","last_active":"2023-10-06T05:17:32.344001+0000","last_peered":"2023-10-06T05:17:32.344001+0000","last_clean":"2023-10-06T05:17:32.344001+0000","last_became_active":"2023-10-06T05:17:31.325559+0000","last_became_peered":"2023-10-06T05:17:31.325559+0000","last_unstale":"2023-10-06T05:17:32.344001+0000","last_undegraded":"2023-10-06T05:17:32.344001+0000","last_fullsized":"2023-10-06T05:17:32.344001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:39.560479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332491+0000","last_change":"2023-10-06T05:17:32.332491+0000","last_active":"2023-10-06T05:17:32.332491+0000","last_peered":"2023-10-06T05:17:32.332491+0000","last_clean":"2023-10-06T05:17:32.332491+0000","last_became_active":"2023-10-06T05:17:32.332221+0000","last_became_peered":"2023-10-06T05:17:32.332221+0000","last_unstale":"2023-10-06T05:17:32.332491+0000","last_undegraded":"2023-10-06T05:17:32.332491+0000","last_fullsized":"2023-10-06T05:17:32.332491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:19.977222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.341142+0000","last_change":"2023-10-06T05:17:31.329588+0000","last_active":"2023-10-06T05:17:32.341142+0000","last_peered":"2023-10-06T05:17:32.341142+0000","last_clean":"2023-10-06T05:17:32.341142+0000","last_became_active":"2023-10-06T05:17:31.329290+0000","last_became_peered":"2023-10-06T05:17:31.329290+0000","last_unstale":"2023-10-06T05:17:32.341142+0000","last_undegraded":"2023-10-06T05:17:32.341142+0000","last_fullsized":"2023-10-06T05:17:32.341142+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:39.560479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323899+0000","last_change":"2023-10-06T05:17:32.323899+0000","last_active":"2023-10-06T05:17:32.323899+0000","last_peered":"2023-10-06T05:17:32.323899+0000","last_clean":"2023-10-06T05:17:32.323899+0000","last_became_active":"2023-10-06T05:17:32.323309+0000","last_became_peered":"2023-10-06T05:17:32.323309+0000","last_unstale":"2023-10-06T05:17:32.323899+0000","last_undegraded":"2023-10-06T05:17:32.323899+0000","last_fullsized":"2023-10-06T05:17:32.323899+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:56.585110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.323946+0000","last_change":"2023-10-06T05:17:31.321935+0000","last_active":"2023-10-06T05:17:31.323946+0000","last_peered":"2023-10-06T05:17:31.323946+0000","last_clean":"2023-10-06T05:17:31.323946+0000","last_became_active":"2023-10-06T05:17:31.321671+0000","last_became_peered":"2023-10-06T05:17:31.321671+0000","last_unstale":"2023-10-06T05:17:31.323946+0000","last_undegraded":"2023-10-06T05:17:31.323946+0000","last_fullsized":"2023-10-06T05:17:31.323946+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:10.276853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.320953+0000","last_change":"2023-10-06T05:17:32.320953+0000","last_active":"2023-10-06T05:17:32.320953+0000","last_peered":"2023-10-06T05:17:32.320953+0000","last_clean":"2023-10-06T05:17:32.320953+0000","last_became_active":"2023-10-06T05:17:32.320298+0000","last_became_peered":"2023-10-06T05:17:32.320298+0000","last_unstale":"2023-10-06T05:17:32.320953+0000","last_undegraded":"2023-10-06T05:17:32.320953+0000","last_fullsized":"2023-10-06T05:17:32.320953+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.326866+0000","last_change":"2023-10-06T05:17:31.324539+0000","last_active":"2023-10-06T05:17:31.326866+0000","last_peered":"2023-10-06T05:17:31.326866+0000","last_clean":"2023-10-06T05:17:31.326866+0000","last_became_active":"2023-10-06T05:17:31.324225+0000","last_became_peered":"2023-10-06T05:17:31.324225+0000","last_unstale":"2023-10-06T05:17:31.326866+0000","last_undegraded":"2023-10-06T05:17:31.326866+0000","last_fullsized":"2023-10-06T05:17:31.326866+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:01.504929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.336350+0000","last_change":"2023-10-06T05:17:32.336350+0000","last_active":"2023-10-06T05:17:32.336350+0000","last_peered":"2023-10-06T05:17:32.336350+0000","last_clean":"2023-10-06T05:17:32.336350+0000","last_became_active":"2023-10-06T05:17:32.333629+0000","last_became_peered":"2023-10-06T05:17:32.333629+0000","last_unstale":"2023-10-06T05:17:32.336350+0000","last_undegraded":"2023-10-06T05:17:32.336350+0000","last_fullsized":"2023-10-06T05:17:32.336350+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:19.977222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.338458+0000","last_change":"2023-10-06T05:17:31.324029+0000","last_active":"2023-10-06T05:17:32.338458+0000","last_peered":"2023-10-06T05:17:32.338458+0000","last_clean":"2023-10-06T05:17:32.338458+0000","last_became_active":"2023-10-06T05:17:31.323760+0000","last_became_peered":"2023-10-06T05:17:31.323760+0000","last_unstale":"2023-10-06T05:17:32.338458+0000","last_undegraded":"2023-10-06T05:17:32.338458+0000","last_fullsized":"2023-10-06T05:17:32.338458+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:09.104643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325174+0000","last_change":"2023-10-06T05:17:32.325174+0000","last_active":"2023-10-06T05:17:32.325174+0000","last_peered":"2023-10-06T05:17:32.325174+0000","last_clean":"2023-10-06T05:17:32.325174+0000","last_became_active":"2023-10-06T05:17:32.324880+0000","last_became_peered":"2023-10-06T05:17:32.324880+0000","last_unstale":"2023-10-06T05:17:32.325174+0000","last_undegraded":"2023-10-06T05:17:32.325174+0000","last_fullsized":"2023-10-06T05:17:32.325174+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:04.156249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.345437+0000","last_change":"2023-10-06T05:17:31.327171+0000","last_active":"2023-10-06T05:17:32.345437+0000","last_peered":"2023-10-06T05:17:32.345437+0000","last_clean":"2023-10-06T05:17:32.345437+0000","last_became_active":"2023-10-06T05:17:31.326864+0000","last_became_peered":"2023-10-06T05:17:31.326864+0000","last_unstale":"2023-10-06T05:17:32.345437+0000","last_undegraded":"2023-10-06T05:17:32.345437+0000","last_fullsized":"2023-10-06T05:17:32.345437+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:32.489979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325287+0000","last_change":"2023-10-06T05:17:32.325287+0000","last_active":"2023-10-06T05:17:32.325287+0000","last_peered":"2023-10-06T05:17:32.325287+0000","last_clean":"2023-10-06T05:17:32.325287+0000","last_became_active":"2023-10-06T05:17:32.324985+0000","last_became_peered":"2023-10-06T05:17:32.324985+0000","last_unstale":"2023-10-06T05:17:32.325287+0000","last_undegraded":"2023-10-06T05:17:32.325287+0000","last_fullsized":"2023-10-06T05:17:32.325287+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:32.942267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.330365+0000","last_change":"2023-10-06T05:17:31.325945+0000","last_active":"2023-10-06T05:17:31.330365+0000","last_peered":"2023-10-06T05:17:31.330365+0000","last_clean":"2023-10-06T05:17:31.330365+0000","last_became_active":"2023-10-06T05:17:31.325626+0000","last_became_peered":"2023-10-06T05:17:31.325626+0000","last_unstale":"2023-10-06T05:17:31.330365+0000","last_undegraded":"2023-10-06T05:17:31.330365+0000","last_fullsized":"2023-10-06T05:17:31.330365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:38.572266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332688+0000","last_change":"2023-10-06T05:17:32.332688+0000","last_active":"2023-10-06T05:17:32.332688+0000","last_peered":"2023-10-06T05:17:32.332688+0000","last_clean":"2023-10-06T05:17:32.332688+0000","last_became_active":"2023-10-06T05:17:32.332092+0000","last_became_peered":"2023-10-06T05:17:32.332092+0000","last_unstale":"2023-10-06T05:17:32.332688+0000","last_undegraded":"2023-10-06T05:17:32.332688+0000","last_fullsized":"2023-10-06T05:17:32.332688+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:18.765768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.322088+0000","last_change":"2023-10-06T05:17:31.319420+0000","last_active":"2023-10-06T05:17:31.322088+0000","last_peered":"2023-10-06T05:17:31.322088+0000","last_clean":"2023-10-06T05:17:31.322088+0000","last_became_active":"2023-10-06T05:17:31.319157+0000","last_became_peered":"2023-10-06T05:17:31.319157+0000","last_unstale":"2023-10-06T05:17:31.322088+0000","last_undegraded":"2023-10-06T05:17:31.322088+0000","last_fullsized":"2023-10-06T05:17:31.322088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:08.741877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.328163+0000","last_change":"2023-10-06T05:17:32.328163+0000","last_active":"2023-10-06T05:17:32.328163+0000","last_peered":"2023-10-06T05:17:32.328163+0000","last_clean":"2023-10-06T05:17:32.328163+0000","last_became_active":"2023-10-06T05:17:32.327869+0000","last_became_peered":"2023-10-06T05:17:32.327869+0000","last_unstale":"2023-10-06T05:17:32.328163+0000","last_undegraded":"2023-10-06T05:17:32.328163+0000","last_fullsized":"2023-10-06T05:17:32.328163+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:46.301441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.321561+0000","last_change":"2023-10-06T05:17:31.319512+0000","last_active":"2023-10-06T05:17:31.321561+0000","last_peered":"2023-10-06T05:17:31.321561+0000","last_clean":"2023-10-06T05:17:31.321561+0000","last_became_active":"2023-10-06T05:17:31.319272+0000","last_became_peered":"2023-10-06T05:17:31.319272+0000","last_unstale":"2023-10-06T05:17:31.321561+0000","last_undegraded":"2023-10-06T05:17:31.321561+0000","last_fullsized":"2023-10-06T05:17:31.321561+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:14.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.334145+0000","last_change":"2023-10-06T05:17:32.334145+0000","last_active":"2023-10-06T05:17:32.334145+0000","last_peered":"2023-10-06T05:17:32.334145+0000","last_clean":"2023-10-06T05:17:32.334145+0000","last_became_active":"2023-10-06T05:17:32.333884+0000","last_became_peered":"2023-10-06T05:17:32.333884+0000","last_unstale":"2023-10-06T05:17:32.334145+0000","last_undegraded":"2023-10-06T05:17:32.334145+0000","last_fullsized":"2023-10-06T05:17:32.334145+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:46.301441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.322808+0000","last_change":"2023-10-06T05:17:31.322243+0000","last_active":"2023-10-06T05:17:31.322808+0000","last_peered":"2023-10-06T05:17:31.322808+0000","last_clean":"2023-10-06T05:17:31.322808+0000","last_became_active":"2023-10-06T05:17:31.321956+0000","last_became_peered":"2023-10-06T05:17:31.321956+0000","last_unstale":"2023-10-06T05:17:31.322808+0000","last_undegraded":"2023-10-06T05:17:31.322808+0000","last_fullsized":"2023-10-06T05:17:31.322808+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:25.133521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.344083+0000","last_change":"2023-10-06T05:17:32.344083+0000","last_active":"2023-10-06T05:17:32.344083+0000","last_peered":"2023-10-06T05:17:32.344083+0000","last_clean":"2023-10-06T05:17:32.344083+0000","last_became_active":"2023-10-06T05:17:32.343817+0000","last_became_peered":"2023-10-06T05:17:32.343817+0000","last_unstale":"2023-10-06T05:17:32.344083+0000","last_undegraded":"2023-10-06T05:17:32.344083+0000","last_fullsized":"2023-10-06T05:17:32.344083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:05.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.324469+0000","last_change":"2023-10-06T05:17:31.323537+0000","last_active":"2023-10-06T05:17:31.324469+0000","last_peered":"2023-10-06T05:17:31.324469+0000","last_clean":"2023-10-06T05:17:31.324469+0000","last_became_active":"2023-10-06T05:17:31.323260+0000","last_became_peered":"2023-10-06T05:17:31.323260+0000","last_unstale":"2023-10-06T05:17:31.324469+0000","last_undegraded":"2023-10-06T05:17:31.324469+0000","last_fullsized":"2023-10-06T05:17:31.324469+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.432813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329631+0000","last_change":"2023-10-06T05:17:32.329631+0000","last_active":"2023-10-06T05:17:32.329631+0000","last_peered":"2023-10-06T05:17:32.329631+0000","last_clean":"2023-10-06T05:17:32.329631+0000","last_became_active":"2023-10-06T05:17:32.329392+0000","last_became_peered":"2023-10-06T05:17:32.329392+0000","last_unstale":"2023-10-06T05:17:32.329631+0000","last_undegraded":"2023-10-06T05:17:32.329631+0000","last_fullsized":"2023-10-06T05:17:32.329631+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:42.655847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.324402+0000","last_change":"2023-10-06T05:17:31.323405+0000","last_active":"2023-10-06T05:17:31.324402+0000","last_peered":"2023-10-06T05:17:31.324402+0000","last_clean":"2023-10-06T05:17:31.324402+0000","last_became_active":"2023-10-06T05:17:31.323111+0000","last_became_peered":"2023-10-06T05:17:31.323111+0000","last_unstale":"2023-10-06T05:17:31.324402+0000","last_undegraded":"2023-10-06T05:17:31.324402+0000","last_fullsized":"2023-10-06T05:17:31.324402+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:20.012975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.321667+0000","last_change":"2023-10-06T05:17:32.321667+0000","last_active":"2023-10-06T05:17:32.321667+0000","last_peered":"2023-10-06T05:17:32.321667+0000","last_clean":"2023-10-06T05:17:32.321667+0000","last_became_active":"2023-10-06T05:17:32.321053+0000","last_became_peered":"2023-10-06T05:17:32.321053+0000","last_unstale":"2023-10-06T05:17:32.321667+0000","last_undegraded":"2023-10-06T05:17:32.321667+0000","last_fullsized":"2023-10-06T05:17:32.321667+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:54.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.333184+0000","last_change":"2023-10-06T05:17:31.330012+0000","last_active":"2023-10-06T05:17:31.333184+0000","last_peered":"2023-10-06T05:17:31.333184+0000","last_clean":"2023-10-06T05:17:31.333184+0000","last_became_active":"2023-10-06T05:17:31.329826+0000","last_became_peered":"2023-10-06T05:17:31.329826+0000","last_unstale":"2023-10-06T05:17:31.333184+0000","last_undegraded":"2023-10-06T05:17:31.333184+0000","last_fullsized":"2023-10-06T05:17:31.333184+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:48.854489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332061+0000","last_change":"2023-10-06T05:17:32.332061+0000","last_active":"2023-10-06T05:17:32.332061+0000","last_peered":"2023-10-06T05:17:32.332061+0000","last_clean":"2023-10-06T05:17:32.332061+0000","last_became_active":"2023-10-06T05:17:32.331672+0000","last_became_peered":"2023-10-06T05:17:32.331672+0000","last_unstale":"2023-10-06T05:17:32.332061+0000","last_undegraded":"2023-10-06T05:17:32.332061+0000","last_fullsized":"2023-10-06T05:17:32.332061+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.343083+0000","last_change":"2023-10-06T05:17:31.326011+0000","last_active":"2023-10-06T05:17:32.343083+0000","last_peered":"2023-10-06T05:17:32.343083+0000","last_clean":"2023-10-06T05:17:32.343083+0000","last_became_active":"2023-10-06T05:17:31.325708+0000","last_became_peered":"2023-10-06T05:17:31.325708+0000","last_unstale":"2023-10-06T05:17:32.343083+0000","last_undegraded":"2023-10-06T05:17:32.343083+0000","last_fullsized":"2023-10-06T05:17:32.343083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:44.722155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.315343+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:17:31.315343+0000","last_peered":"2023-10-06T05:17:31.315343+0000","last_clean":"2023-10-06T05:17:31.315343+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:17:31.315343+0000","last_undegraded":"2023-10-06T05:17:31.315343+0000","last_fullsized":"2023-10-06T05:17:31.315343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.327832+0000","last_change":"2023-10-06T05:17:32.327832+0000","last_active":"2023-10-06T05:17:32.327832+0000","last_peered":"2023-10-06T05:17:32.327832+0000","last_clean":"2023-10-06T05:17:32.327832+0000","last_became_active":"2023-10-06T05:17:32.327560+0000","last_became_peered":"2023-10-06T05:17:32.327560+0000","last_unstale":"2023-10-06T05:17:32.327832+0000","last_undegraded":"2023-10-06T05:17:32.327832+0000","last_fullsized":"2023-10-06T05:17:32.327832+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:52.892989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.330614+0000","last_change":"2023-10-06T05:17:31.330213+0000","last_active":"2023-10-06T05:17:31.330614+0000","last_peered":"2023-10-06T05:17:31.330614+0000","last_clean":"2023-10-06T05:17:31.330614+0000","last_became_active":"2023-10-06T05:17:31.330002+0000","last_became_peered":"2023-10-06T05:17:31.330002+0000","last_unstale":"2023-10-06T05:17:31.330614+0000","last_undegraded":"2023-10-06T05:17:31.330614+0000","last_fullsized":"2023-10-06T05:17:31.330614+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:49.510325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.313524+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:17:31.313524+0000","last_peered":"2023-10-06T05:17:31.313524+0000","last_clean":"2023-10-06T05:17:31.313524+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:17:31.313524+0000","last_undegraded":"2023-10-06T05:17:31.313524+0000","last_fullsized":"2023-10-06T05:17:31.313524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332341+0000","last_change":"2023-10-06T05:17:32.332341+0000","last_active":"2023-10-06T05:17:32.332341+0000","last_peered":"2023-10-06T05:17:32.332341+0000","last_clean":"2023-10-06T05:17:32.332341+0000","last_became_active":"2023-10-06T05:17:32.331820+0000","last_became_peered":"2023-10-06T05:17:32.331820+0000","last_unstale":"2023-10-06T05:17:32.332341+0000","last_undegraded":"2023-10-06T05:17:32.332341+0000","last_fullsized":"2023-10-06T05:17:32.332341+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:02.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.336649+0000","last_change":"2023-10-06T05:17:31.329363+0000","last_active":"2023-10-06T05:17:32.336649+0000","last_peered":"2023-10-06T05:17:32.336649+0000","last_clean":"2023-10-06T05:17:32.336649+0000","last_became_active":"2023-10-06T05:17:31.329195+0000","last_became_peered":"2023-10-06T05:17:31.329195+0000","last_unstale":"2023-10-06T05:17:32.336649+0000","last_undegraded":"2023-10-06T05:17:32.336649+0000","last_fullsized":"2023-10-06T05:17:32.336649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.432813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.310205+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:17:31.310205+0000","last_peered":"2023-10-06T05:17:31.310205+0000","last_clean":"2023-10-06T05:17:31.310205+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:17:31.310205+0000","last_undegraded":"2023-10-06T05:17:31.310205+0000","last_fullsized":"2023-10-06T05:17:31.310205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.339701+0000","last_change":"2023-10-06T05:17:32.339701+0000","last_active":"2023-10-06T05:17:32.339701+0000","last_peered":"2023-10-06T05:17:32.339701+0000","last_clean":"2023-10-06T05:17:32.339701+0000","last_became_active":"2023-10-06T05:17:32.339397+0000","last_became_peered":"2023-10-06T05:17:32.339397+0000","last_unstale":"2023-10-06T05:17:32.339701+0000","last_undegraded":"2023-10-06T05:17:32.339701+0000","last_fullsized":"2023-10-06T05:17:32.339701+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:21.360640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.322095+0000","last_change":"2023-10-06T05:17:31.319144+0000","last_active":"2023-10-06T05:17:31.322095+0000","last_peered":"2023-10-06T05:17:31.322095+0000","last_clean":"2023-10-06T05:17:31.322095+0000","last_became_active":"2023-10-06T05:17:31.318852+0000","last_became_peered":"2023-10-06T05:17:31.318852+0000","last_unstale":"2023-10-06T05:17:31.322095+0000","last_undegraded":"2023-10-06T05:17:31.322095+0000","last_fullsized":"2023-10-06T05:17:31.322095+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:26.830478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.315620+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:17:31.315620+0000","last_peered":"2023-10-06T05:17:31.315620+0000","last_clean":"2023-10-06T05:17:31.315620+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:17:31.315620+0000","last_undegraded":"2023-10-06T05:17:31.315620+0000","last_fullsized":"2023-10-06T05:17:31.315620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":189,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.320622+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:17:31.320622+0000","last_peered":"2023-10-06T05:17:31.320622+0000","last_clean":"2023-10-06T05:17:31.320622+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:17:31.320622+0000","last_undegraded":"2023-10-06T05:17:31.320622+0000","last_fullsized":"2023-10-06T05:17:31.320622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329854+0000","last_change":"2023-10-06T05:17:32.329854+0000","last_active":"2023-10-06T05:17:32.329854+0000","last_peered":"2023-10-06T05:17:32.329854+0000","last_clean":"2023-10-06T05:17:32.329854+0000","last_became_active":"2023-10-06T05:17:32.328366+0000","last_became_peered":"2023-10-06T05:17:32.328366+0000","last_unstale":"2023-10-06T05:17:32.329854+0000","last_undegraded":"2023-10-06T05:17:32.329854+0000","last_fullsized":"2023-10-06T05:17:32.329854+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:38.341979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.319320+0000","last_change":"2023-10-06T05:17:31.318047+0000","last_active":"2023-10-06T05:17:31.319320+0000","last_peered":"2023-10-06T05:17:31.319320+0000","last_clean":"2023-10-06T05:17:31.319320+0000","last_became_active":"2023-10-06T05:17:31.317704+0000","last_became_peered":"2023-10-06T05:17:31.317704+0000","last_unstale":"2023-10-06T05:17:31.319320+0000","last_undegraded":"2023-10-06T05:17:31.319320+0000","last_fullsized":"2023-10-06T05:17:31.319320+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:26.830478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.320711+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:17:31.320711+0000","last_peered":"2023-10-06T05:17:31.320711+0000","last_clean":"2023-10-06T05:17:31.320711+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:17:31.320711+0000","last_undegraded":"2023-10-06T05:17:31.320711+0000","last_fullsized":"2023-10-06T05:17:31.320711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332735+0000","last_change":"2023-10-06T05:17:32.332735+0000","last_active":"2023-10-06T05:17:32.332735+0000","last_peered":"2023-10-06T05:17:32.332735+0000","last_clean":"2023-10-06T05:17:32.332735+0000","last_became_active":"2023-10-06T05:17:32.332480+0000","last_became_peered":"2023-10-06T05:17:32.332480+0000","last_unstale":"2023-10-06T05:17:32.332735+0000","last_undegraded":"2023-10-06T05:17:32.332735+0000","last_fullsized":"2023-10-06T05:17:32.332735+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:19.012509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.318043+0000","last_change":"2023-10-06T05:17:31.329439+0000","last_active":"2023-10-06T05:17:32.318043+0000","last_peered":"2023-10-06T05:17:32.318043+0000","last_clean":"2023-10-06T05:17:32.318043+0000","last_became_active":"2023-10-06T05:17:31.325911+0000","last_became_peered":"2023-10-06T05:17:31.325911+0000","last_unstale":"2023-10-06T05:17:32.318043+0000","last_undegraded":"2023-10-06T05:17:32.318043+0000","last_fullsized":"2023-10-06T05:17:32.318043+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:24.897513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.319912+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:17:31.319912+0000","last_peered":"2023-10-06T05:17:31.319912+0000","last_clean":"2023-10-06T05:17:31.319912+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:17:31.319912+0000","last_undegraded":"2023-10-06T05:17:31.319912+0000","last_fullsized":"2023-10-06T05:17:31.319912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332592+0000","last_change":"2023-10-06T05:17:32.332592+0000","last_active":"2023-10-06T05:17:32.332592+0000","last_peered":"2023-10-06T05:17:32.332592+0000","last_clean":"2023-10-06T05:17:32.332592+0000","last_became_active":"2023-10-06T05:17:32.331981+0000","last_became_peered":"2023-10-06T05:17:32.331981+0000","last_unstale":"2023-10-06T05:17:32.332592+0000","last_undegraded":"2023-10-06T05:17:32.332592+0000","last_fullsized":"2023-10-06T05:17:32.332592+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:40.565830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.333489+0000","last_change":"2023-10-06T05:17:31.332822+0000","last_active":"2023-10-06T05:17:31.333489+0000","last_peered":"2023-10-06T05:17:31.333489+0000","last_clean":"2023-10-06T05:17:31.333489+0000","last_became_active":"2023-10-06T05:17:31.332529+0000","last_became_peered":"2023-10-06T05:17:31.332529+0000","last_unstale":"2023-10-06T05:17:31.333489+0000","last_undegraded":"2023-10-06T05:17:31.333489+0000","last_fullsized":"2023-10-06T05:17:31.333489+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:24.897513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.315457+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:17:31.315457+0000","last_peered":"2023-10-06T05:17:31.315457+0000","last_clean":"2023-10-06T05:17:31.315457+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:17:31.315457+0000","last_undegraded":"2023-10-06T05:17:31.315457+0000","last_fullsized":"2023-10-06T05:17:31.315457+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.330487+0000","last_change":"2023-10-06T05:17:32.330487+0000","last_active":"2023-10-06T05:17:32.330487+0000","last_peered":"2023-10-06T05:17:32.330487+0000","last_clean":"2023-10-06T05:17:32.330487+0000","last_became_active":"2023-10-06T05:17:32.330172+0000","last_became_peered":"2023-10-06T05:17:32.330172+0000","last_unstale":"2023-10-06T05:17:32.330487+0000","last_undegraded":"2023-10-06T05:17:32.330487+0000","last_fullsized":"2023-10-06T05:17:32.330487+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.314467+0000","last_change":"2023-10-06T05:17:31.321981+0000","last_active":"2023-10-06T05:17:32.314467+0000","last_peered":"2023-10-06T05:17:32.314467+0000","last_clean":"2023-10-06T05:17:32.314467+0000","last_became_active":"2023-10-06T05:17:31.321791+0000","last_became_peered":"2023-10-06T05:17:31.321791+0000","last_unstale":"2023-10-06T05:17:32.314467+0000","last_undegraded":"2023-10-06T05:17:32.314467+0000","last_fullsized":"2023-10-06T05:17:32.314467+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:25.651129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.312116+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:17:31.312116+0000","last_peered":"2023-10-06T05:17:31.312116+0000","last_clean":"2023-10-06T05:17:31.312116+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:17:31.312116+0000","last_undegraded":"2023-10-06T05:17:31.312116+0000","last_fullsized":"2023-10-06T05:17:31.312116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298448,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073392,"statfs":{"total":96636764160,"available":96331153408,"internally_reserved":0,"allocated":1187840,"data_stored":864217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73199999999999998,"5min":0.876,"15min":0.876},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.753,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":1.476,"15min":1.476},"last":0.371},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65500000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.627,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.67800000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.498},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49299999999999999,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64300000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72699999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.746,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76700000000000002,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.97999999999999998,"15min":1.046},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77700000000000002,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82599999999999996,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.04,"15min":1.1120000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77800000000000002,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.81999999999999995,"15min":0.85199999999999998},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":1.5880000000000001,"15min":1.746},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.77800000000000002,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.98899999999999999,"15min":1.0449999999999999},"min":{"1min":0.53100000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.80700000000000005,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.90200000000000002,"15min":0.96899999999999997},"min":{"1min":0.33100000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64700000000000002,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.93300000000000005,"15min":0.997},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80800000000000005,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.34999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70999999999999996,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.92700000000000005,"15min":0.98799999999999999},"min":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77900000000000003,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.96499999999999997,"15min":1.0289999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82899999999999996,"5min":9.718,"15min":9.718},"last":0.497},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.99199999999999999,"15min":1.056},"min":{"1min":0.42999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85299999999999998,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.42899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.39300000000000002,"15min":0.39500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55500000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.442,"15min":0.443},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.48699999999999999,"15min":0.497},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":1.673,"15min":1.673},"last":0.34499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.58899999999999997,"15min":0.60999999999999999},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68000000000000005,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46400000000000002,"15min":0.46999999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.57899999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85799999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91400000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91200000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91700000000000004,"5min":1.026,"15min":1.026},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.878,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298448,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073392,"statfs":{"total":96636764160,"available":96331153408,"internally_reserved":0,"allocated":1187840,"data_stored":864217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.27900000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.47999999999999998,"5min":1.038,"15min":1.038},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.41699999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58299999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91200000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.314,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55200000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.35599999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63500000000000001,"5min":1.016,"15min":1.016},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.625},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.77800000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.81599999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93500000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.746,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70399999999999996,"15min":0.70999999999999996},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.872,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":0.97399999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80800000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.52900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.435,"15min":0.44400000000000001},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.57399999999999995,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.47399999999999998,"15min":0.48299999999999998},"min":{"1min":0.32100000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.57299999999999995,"5min":1.899,"15min":1.899},"last":0.32100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64300000000000002,"5min":2.097,"15min":2.097},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.625,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.314,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.55100000000000005,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.35499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.61399999999999999,"5min":2.121,"15min":2.121},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.52300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90500000000000003,"5min":1.643,"15min":1.643},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.55500000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":1.851,"15min":1.851},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.65400000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83399999999999996,"5min":1.571,"15min":1.571},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.53300000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98599999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86499999999999999,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95799999999999996,"5min":1.613,"15min":1.613},"last":0.5}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297864,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":589824,"data_stored":272496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.629},"min":{"1min":0.52200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.48899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.52700000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63700000000000001,"15min":0.63200000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.51400000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.499},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.48899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75900000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.44800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.372,"15min":0.372},"max":{"1min":0.82099999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.46899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.78700000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.70199999999999996,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.53200000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.623}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.46100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.46300000000000002},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:17:36.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.679+0000 7f10efb20700 1 Processor -- start 2023-10-06T05:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.679+0000 7f10efb20700 1 -- start start 2023-10-06T05:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10efb20700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10e80cc550 0x7f10e80cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10efb20700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f10e815d0b0 0x7f10e815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10efb20700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e8161f40 con 0x7f10e80caa70 2023-10-06T05:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10efb20700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e80cb5d0 con 0x7f10e80cc550 2023-10-06T05:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10efb20700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e8161330 con 0x7f10e815d0b0 2023-10-06T05:17:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10ed8bc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10e80cc550 0x7f10e80cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10ed8bc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10e80cc550 0x7f10e80cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43894/0 (socket says 172.21.15.45:43894) 2023-10-06T05:17:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10ed8bc700 1 -- 172.21.15.45:0/1204081816 learned_addr learned my addr 172.21.15.45:0/1204081816 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.680+0000 7f10ed0bb700 1 --2- 172.21.15.45:0/1204081816 >> v2:172.21.15.144:3300/0 conn(0x7f10e815d0b0 0x7f10e815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ed8bc700 1 -- 172.21.15.45:0/1204081816 >> v2:172.21.15.144:3300/0 conn(0x7f10e815d0b0 msgr2=0x7f10e815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ed8bc700 1 --2- 172.21.15.45:0/1204081816 >> v2:172.21.15.144:3300/0 conn(0x7f10e815d0b0 0x7f10e815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ed8bc700 1 -- 172.21.15.45:0/1204081816 >> v1:172.21.15.144:6789/0 conn(0x7f10e80caa70 legacy=0x7f10e80caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ed8bc700 1 -- 172.21.15.45:0/1204081816 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8161610 con 0x7f10e80cc550 2023-10-06T05:17:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ed8bc700 1 --2- 172.21.15.45:0/1204081816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80cc920 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f10e80c4fb0 tx=0x7f10e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5985323e0937a258 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ec8ba700 1 -- 172.21.15.45:0/1204081816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e400b860 con 0x7f10e80cc550 2023-10-06T05:17:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ec8ba700 1 -- 172.21.15.45:0/1204081816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e400b9c0 con 0x7f10e80cc550 2023-10-06T05:17:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10ec8ba700 1 -- 172.21.15.45:0/1204081816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e400bd60 con 0x7f10e80cc550 2023-10-06T05:17:36.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10efb20700 1 -- 172.21.15.45:0/1204081816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 msgr2=0x7f10e80cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.681+0000 7f10efb20700 1 --2- 172.21.15.45:0/1204081816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80cc920 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f10e80c4fb0 tx=0x7f10e4009250 comp rx=0 tx=0).stop 2023-10-06T05:17:36.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.682+0000 7f10efb20700 1 -- 172.21.15.45:0/1204081816 shutdown_connections 2023-10-06T05:17:36.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.682+0000 7f10efb20700 1 --2- 172.21.15.45:0/1204081816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80cc920 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.682+0000 7f10efb20700 1 --2- 172.21.15.45:0/1204081816 >> v2:172.21.15.144:3300/0 conn(0x7f10e815d0b0 0x7f10e815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.682+0000 7f10efb20700 1 -- 172.21.15.45:0/1204081816 >> 172.21.15.45:0/1204081816 conn(0x7f10e80cdf50 msgr2=0x7f10e80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.682+0000 7f10efb20700 1 -- 172.21.15.45:0/1204081816 shutdown_connections 2023-10-06T05:17:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.682+0000 7f10efb20700 1 -- 172.21.15.45:0/1204081816 wait complete. 2023-10-06T05:17:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.683+0000 7f10efb20700 1 Processor -- start 2023-10-06T05:17:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.683+0000 7f10efb20700 1 -- start start 2023-10-06T05:17:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10efb20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e80caa70 0x7f10e8148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10efb20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10efb20700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10e815d0b0 0x7f10e80c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10efb20700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80b3980 con 0x7f10e80cc550 2023-10-06T05:17:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10efb20700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80b2d00 con 0x7f10e80caa70 2023-10-06T05:17:36.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10efb20700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80b2fe0 con 0x7f10e815d0b0 2023-10-06T05:17:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43900/0 (socket says 172.21.15.45:43900) 2023-10-06T05:17:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 -- 172.21.15.45:0/2726888772 learned_addr learned my addr 172.21.15.45:0/2726888772 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed8bc700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e80caa70 0x7f10e8148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 -- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10e815d0b0 msgr2=0x7f10e80c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ee0bd700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10e815d0b0 0x7f10e80c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10e815d0b0 0x7f10e80c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 -- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e80caa70 msgr2=0x7f10e8148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e80caa70 0x7f10e8148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 -- 172.21.15.45:0/2726888772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40094f0 con 0x7f10e80cc550 2023-10-06T05:17:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed8bc700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e80caa70 0x7f10e8148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.684+0000 7f10ed0bb700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80b2300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d860 tx=0x7f10d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.685+0000 7f10ddffb700 1 -- 172.21.15.45:0/2726888772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d8009610 con 0x7f10e80cc550 2023-10-06T05:17:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.685+0000 7f10ddffb700 1 -- 172.21.15.45:0/2726888772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d8009770 con 0x7f10e80cc550 2023-10-06T05:17:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.685+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e80b32c0 con 0x7f10e80cc550 2023-10-06T05:17:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.685+0000 7f10ddffb700 1 -- 172.21.15.45:0/2726888772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d8009b30 con 0x7f10e80cc550 2023-10-06T05:17:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.685+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e8148750 con 0x7f10e80cc550 2023-10-06T05:17:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.686+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e806f410 con 0x7f10e80cc550 2023-10-06T05:17:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.686+0000 7f10ddffb700 1 -- 172.21.15.45:0/2726888772 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10d801e460 con 0x7f10e80cc550 2023-10-06T05:17:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.687+0000 7f10ddffb700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f10cc066f30 0x7f10cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.687+0000 7f10ddffb700 1 -- 172.21.15.45:0/2726888772 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f10d8084ee0 con 0x7f10e80cc550 2023-10-06T05:17:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.687+0000 7f10ed8bc700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f10cc066f30 0x7f10cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.688+0000 7f10ed8bc700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f10cc066f30 0x7f10cc0693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f10e80c4fb0 tx=0x7f10e4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.691+0000 7f10ddffb700 1 -- 172.21.15.45:0/2726888772 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10d8051e10 con 0x7f10e80cc550 2023-10-06T05:17:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.890+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10e81618f0 con 0x7f10cc066f30 2023-10-06T05:17:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.899+0000 7f10ddffb700 1 -- 172.21.15.45:0/2726888772 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311743 (secure 0 0 0) 0x7f10e81618f0 con 0x7f10cc066f30 2023-10-06T05:17:36.905 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f10cc066f30 msgr2=0x7f10cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f10cc066f30 0x7f10cc0693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f10e80c4fb0 tx=0x7f10e4017000 comp rx=0 tx=0).stop 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 msgr2=0x7f10e80b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80b2300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d860 tx=0x7f10d8008040 comp rx=0 tx=0).stop 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 shutdown_connections 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f10cc066f30 0x7f10cc0693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10e815d0b0 0x7f10e80c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10e80caa70 0x7f10e8148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 --2- 172.21.15.45:0/2726888772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10e80cc550 0x7f10e80b2300 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 >> 172.21.15.45:0/2726888772 conn(0x7f10e80cdf50 msgr2=0x7f10e81523d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 shutdown_connections 2023-10-06T05:17:36.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:36.903+0000 7f10efb20700 1 -- 172.21.15.45:0/2726888772 wait complete. 2023-10-06T05:17:36.909 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:17:36.918 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-10-06T05:17:36.486405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384100,"kb_used_data":5796,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590620,"statfs":{"total":773094113280,"available":770652794880,"internally_reserved":0,"allocated":5935104,"data_stored":3370175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.008264"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.334500+0000","last_change":"2023-10-06T05:17:32.334500+0000","last_active":"2023-10-06T05:17:32.334500+0000","last_peered":"2023-10-06T05:17:32.334500+0000","last_clean":"2023-10-06T05:17:32.334500+0000","last_became_active":"2023-10-06T05:17:32.334239+0000","last_became_peered":"2023-10-06T05:17:32.334239+0000","last_unstale":"2023-10-06T05:17:32.334500+0000","last_undegraded":"2023-10-06T05:17:32.334500+0000","last_fullsized":"2023-10-06T05:17:32.334500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:32.572071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.346034+0000","last_change":"2023-10-06T05:17:32.346034+0000","last_active":"2023-10-06T05:17:32.346034+0000","last_peered":"2023-10-06T05:17:32.346034+0000","last_clean":"2023-10-06T05:17:32.346034+0000","last_became_active":"2023-10-06T05:17:32.345723+0000","last_became_peered":"2023-10-06T05:17:32.345723+0000","last_unstale":"2023-10-06T05:17:32.346034+0000","last_undegraded":"2023-10-06T05:17:32.346034+0000","last_fullsized":"2023-10-06T05:17:32.346034+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:17.642543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325291+0000","last_change":"2023-10-06T05:17:32.325291+0000","last_active":"2023-10-06T05:17:32.325291+0000","last_peered":"2023-10-06T05:17:32.325291+0000","last_clean":"2023-10-06T05:17:32.325291+0000","last_became_active":"2023-10-06T05:17:32.325038+0000","last_became_peered":"2023-10-06T05:17:32.325038+0000","last_unstale":"2023-10-06T05:17:32.325291+0000","last_undegraded":"2023-10-06T05:17:32.325291+0000","last_fullsized":"2023-10-06T05:17:32.325291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:41.772378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.327772+0000","last_change":"2023-10-06T05:17:32.327772+0000","last_active":"2023-10-06T05:17:32.327772+0000","last_peered":"2023-10-06T05:17:32.327772+0000","last_clean":"2023-10-06T05:17:32.327772+0000","last_became_active":"2023-10-06T05:17:32.327473+0000","last_became_peered":"2023-10-06T05:17:32.327473+0000","last_unstale":"2023-10-06T05:17:32.327772+0000","last_undegraded":"2023-10-06T05:17:32.327772+0000","last_fullsized":"2023-10-06T05:17:32.327772+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:56.585110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323088+0000","last_change":"2023-10-06T05:17:32.323088+0000","last_active":"2023-10-06T05:17:32.323088+0000","last_peered":"2023-10-06T05:17:32.323088+0000","last_clean":"2023-10-06T05:17:32.323088+0000","last_became_active":"2023-10-06T05:17:32.322807+0000","last_became_peered":"2023-10-06T05:17:32.322807+0000","last_unstale":"2023-10-06T05:17:32.323088+0000","last_undegraded":"2023-10-06T05:17:32.323088+0000","last_fullsized":"2023-10-06T05:17:32.323088+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:18.830245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329826+0000","last_change":"2023-10-06T05:17:32.329826+0000","last_active":"2023-10-06T05:17:32.329826+0000","last_peered":"2023-10-06T05:17:32.329826+0000","last_clean":"2023-10-06T05:17:32.329826+0000","last_became_active":"2023-10-06T05:17:32.329526+0000","last_became_peered":"2023-10-06T05:17:32.329526+0000","last_unstale":"2023-10-06T05:17:32.329826+0000","last_undegraded":"2023-10-06T05:17:32.329826+0000","last_fullsized":"2023-10-06T05:17:32.329826+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:44.325973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323572+0000","last_change":"2023-10-06T05:17:32.323572+0000","last_active":"2023-10-06T05:17:32.323572+0000","last_peered":"2023-10-06T05:17:32.323572+0000","last_clean":"2023-10-06T05:17:32.323572+0000","last_became_active":"2023-10-06T05:17:32.323302+0000","last_became_peered":"2023-10-06T05:17:32.323302+0000","last_unstale":"2023-10-06T05:17:32.323572+0000","last_undegraded":"2023-10-06T05:17:32.323572+0000","last_fullsized":"2023-10-06T05:17:32.323572+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:31.332500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.331770+0000","last_change":"2023-10-06T05:17:32.331770+0000","last_active":"2023-10-06T05:17:32.331770+0000","last_peered":"2023-10-06T05:17:32.331770+0000","last_clean":"2023-10-06T05:17:32.331770+0000","last_became_active":"2023-10-06T05:17:32.327662+0000","last_became_peered":"2023-10-06T05:17:32.327662+0000","last_unstale":"2023-10-06T05:17:32.331770+0000","last_undegraded":"2023-10-06T05:17:32.331770+0000","last_fullsized":"2023-10-06T05:17:32.331770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.280650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324379+0000","last_change":"2023-10-06T05:17:32.324379+0000","last_active":"2023-10-06T05:17:32.324379+0000","last_peered":"2023-10-06T05:17:32.324379+0000","last_clean":"2023-10-06T05:17:32.324379+0000","last_became_active":"2023-10-06T05:17:32.324034+0000","last_became_peered":"2023-10-06T05:17:32.324034+0000","last_unstale":"2023-10-06T05:17:32.324379+0000","last_undegraded":"2023-10-06T05:17:32.324379+0000","last_fullsized":"2023-10-06T05:17:32.324379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:19.977222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.333471+0000","last_change":"2023-10-06T05:17:32.333471+0000","last_active":"2023-10-06T05:17:32.333471+0000","last_peered":"2023-10-06T05:17:32.333471+0000","last_clean":"2023-10-06T05:17:32.333471+0000","last_became_active":"2023-10-06T05:17:32.333154+0000","last_became_peered":"2023-10-06T05:17:32.333154+0000","last_unstale":"2023-10-06T05:17:32.333471+0000","last_undegraded":"2023-10-06T05:17:32.333471+0000","last_fullsized":"2023-10-06T05:17:32.333471+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:56.070562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.344067+0000","last_change":"2023-10-06T05:17:32.344067+0000","last_active":"2023-10-06T05:17:32.344067+0000","last_peered":"2023-10-06T05:17:32.344067+0000","last_clean":"2023-10-06T05:17:32.344067+0000","last_became_active":"2023-10-06T05:17:32.343786+0000","last_became_peered":"2023-10-06T05:17:32.343786+0000","last_unstale":"2023-10-06T05:17:32.344067+0000","last_undegraded":"2023-10-06T05:17:32.344067+0000","last_fullsized":"2023-10-06T05:17:32.344067+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:59.032559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325125+0000","last_change":"2023-10-06T05:17:32.325125+0000","last_active":"2023-10-06T05:17:32.325125+0000","last_peered":"2023-10-06T05:17:32.325125+0000","last_clean":"2023-10-06T05:17:32.325125+0000","last_became_active":"2023-10-06T05:17:32.324509+0000","last_became_peered":"2023-10-06T05:17:32.324509+0000","last_unstale":"2023-10-06T05:17:32.325125+0000","last_undegraded":"2023-10-06T05:17:32.325125+0000","last_fullsized":"2023-10-06T05:17:32.325125+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:05.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.335440+0000","last_change":"2023-10-06T05:17:32.335440+0000","last_active":"2023-10-06T05:17:32.335440+0000","last_peered":"2023-10-06T05:17:32.335440+0000","last_clean":"2023-10-06T05:17:32.335440+0000","last_became_active":"2023-10-06T05:17:32.335034+0000","last_became_peered":"2023-10-06T05:17:32.335034+0000","last_unstale":"2023-10-06T05:17:32.335440+0000","last_undegraded":"2023-10-06T05:17:32.335440+0000","last_fullsized":"2023-10-06T05:17:32.335440+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:54.692661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.314125+0000","last_change":"2023-10-06T05:17:32.314125+0000","last_active":"2023-10-06T05:17:32.314125+0000","last_peered":"2023-10-06T05:17:32.314125+0000","last_clean":"2023-10-06T05:17:32.314125+0000","last_became_active":"2023-10-06T05:17:32.313854+0000","last_became_peered":"2023-10-06T05:17:32.313854+0000","last_unstale":"2023-10-06T05:17:32.314125+0000","last_undegraded":"2023-10-06T05:17:32.314125+0000","last_fullsized":"2023-10-06T05:17:32.314125+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:05.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.331920+0000","last_change":"2023-10-06T05:17:32.331920+0000","last_active":"2023-10-06T05:17:32.331920+0000","last_peered":"2023-10-06T05:17:32.331920+0000","last_clean":"2023-10-06T05:17:32.331920+0000","last_became_active":"2023-10-06T05:17:32.331621+0000","last_became_peered":"2023-10-06T05:17:32.331621+0000","last_unstale":"2023-10-06T05:17:32.331920+0000","last_undegraded":"2023-10-06T05:17:32.331920+0000","last_fullsized":"2023-10-06T05:17:32.331920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:52.892989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332877+0000","last_change":"2023-10-06T05:17:32.332877+0000","last_active":"2023-10-06T05:17:32.332877+0000","last_peered":"2023-10-06T05:17:32.332877+0000","last_clean":"2023-10-06T05:17:32.332877+0000","last_became_active":"2023-10-06T05:17:32.332189+0000","last_became_peered":"2023-10-06T05:17:32.332189+0000","last_unstale":"2023-10-06T05:17:32.332877+0000","last_undegraded":"2023-10-06T05:17:32.332877+0000","last_fullsized":"2023-10-06T05:17:32.332877+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:25.110866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.321581+0000","last_change":"2023-10-06T05:17:32.321581+0000","last_active":"2023-10-06T05:17:32.321581+0000","last_peered":"2023-10-06T05:17:32.321581+0000","last_clean":"2023-10-06T05:17:32.321581+0000","last_became_active":"2023-10-06T05:17:32.320558+0000","last_became_peered":"2023-10-06T05:17:32.320558+0000","last_unstale":"2023-10-06T05:17:32.321581+0000","last_undegraded":"2023-10-06T05:17:32.321581+0000","last_fullsized":"2023-10-06T05:17:32.321581+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:40.423307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.326357+0000","last_change":"2023-10-06T05:17:32.326357+0000","last_active":"2023-10-06T05:17:32.326357+0000","last_peered":"2023-10-06T05:17:32.326357+0000","last_clean":"2023-10-06T05:17:32.326357+0000","last_became_active":"2023-10-06T05:17:32.325958+0000","last_became_peered":"2023-10-06T05:17:32.325958+0000","last_unstale":"2023-10-06T05:17:32.326357+0000","last_undegraded":"2023-10-06T05:17:32.326357+0000","last_fullsized":"2023-10-06T05:17:32.326357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.302073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.328283+0000","last_change":"2023-10-06T05:17:32.328283+0000","last_active":"2023-10-06T05:17:32.328283+0000","last_peered":"2023-10-06T05:17:32.328283+0000","last_clean":"2023-10-06T05:17:32.328283+0000","last_became_active":"2023-10-06T05:17:32.327962+0000","last_became_peered":"2023-10-06T05:17:32.327962+0000","last_unstale":"2023-10-06T05:17:32.328283+0000","last_undegraded":"2023-10-06T05:17:32.328283+0000","last_fullsized":"2023-10-06T05:17:32.328283+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:34.940743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325882+0000","last_change":"2023-10-06T05:17:32.325882+0000","last_active":"2023-10-06T05:17:32.325882+0000","last_peered":"2023-10-06T05:17:32.325882+0000","last_clean":"2023-10-06T05:17:32.325882+0000","last_became_active":"2023-10-06T05:17:32.325599+0000","last_became_peered":"2023-10-06T05:17:32.325599+0000","last_unstale":"2023-10-06T05:17:32.325882+0000","last_undegraded":"2023-10-06T05:17:32.325882+0000","last_fullsized":"2023-10-06T05:17:32.325882+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:35.025622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324070+0000","last_change":"2023-10-06T05:17:32.324070+0000","last_active":"2023-10-06T05:17:32.324070+0000","last_peered":"2023-10-06T05:17:32.324070+0000","last_clean":"2023-10-06T05:17:32.324070+0000","last_became_active":"2023-10-06T05:17:32.323639+0000","last_became_peered":"2023-10-06T05:17:32.323639+0000","last_unstale":"2023-10-06T05:17:32.324070+0000","last_undegraded":"2023-10-06T05:17:32.324070+0000","last_fullsized":"2023-10-06T05:17:32.324070+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:32.942267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.330750+0000","last_change":"2023-10-06T05:17:32.330750+0000","last_active":"2023-10-06T05:17:32.330750+0000","last_peered":"2023-10-06T05:17:32.330750+0000","last_clean":"2023-10-06T05:17:32.330750+0000","last_became_active":"2023-10-06T05:17:32.330145+0000","last_became_peered":"2023-10-06T05:17:32.330145+0000","last_unstale":"2023-10-06T05:17:32.330750+0000","last_undegraded":"2023-10-06T05:17:32.330750+0000","last_fullsized":"2023-10-06T05:17:32.330750+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:38.341979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.336566+0000","last_change":"2023-10-06T05:17:32.336566+0000","last_active":"2023-10-06T05:17:32.336566+0000","last_peered":"2023-10-06T05:17:32.336566+0000","last_clean":"2023-10-06T05:17:32.336566+0000","last_became_active":"2023-10-06T05:17:32.333605+0000","last_became_peered":"2023-10-06T05:17:32.333605+0000","last_unstale":"2023-10-06T05:17:32.336566+0000","last_undegraded":"2023-10-06T05:17:32.336566+0000","last_fullsized":"2023-10-06T05:17:32.336566+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329629+0000","last_change":"2023-10-06T05:17:32.329629+0000","last_active":"2023-10-06T05:17:32.329629+0000","last_peered":"2023-10-06T05:17:32.329629+0000","last_clean":"2023-10-06T05:17:32.329629+0000","last_became_active":"2023-10-06T05:17:32.329327+0000","last_became_peered":"2023-10-06T05:17:32.329327+0000","last_unstale":"2023-10-06T05:17:32.329629+0000","last_undegraded":"2023-10-06T05:17:32.329629+0000","last_fullsized":"2023-10-06T05:17:32.329629+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:56.070562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.333084+0000","last_change":"2023-10-06T05:17:32.333084+0000","last_active":"2023-10-06T05:17:32.333084+0000","last_peered":"2023-10-06T05:17:32.333084+0000","last_clean":"2023-10-06T05:17:32.333084+0000","last_became_active":"2023-10-06T05:17:32.332819+0000","last_became_peered":"2023-10-06T05:17:32.332819+0000","last_unstale":"2023-10-06T05:17:32.333084+0000","last_undegraded":"2023-10-06T05:17:32.333084+0000","last_fullsized":"2023-10-06T05:17:32.333084+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:16.444290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.326239+0000","last_change":"2023-10-06T05:17:32.326239+0000","last_active":"2023-10-06T05:17:32.326239+0000","last_peered":"2023-10-06T05:17:32.326239+0000","last_clean":"2023-10-06T05:17:32.326239+0000","last_became_active":"2023-10-06T05:17:32.325839+0000","last_became_peered":"2023-10-06T05:17:32.325839+0000","last_unstale":"2023-10-06T05:17:32.326239+0000","last_undegraded":"2023-10-06T05:17:32.326239+0000","last_fullsized":"2023-10-06T05:17:32.326239+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:29.302073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.328966+0000","last_change":"2023-10-06T05:17:32.328966+0000","last_active":"2023-10-06T05:17:32.328966+0000","last_peered":"2023-10-06T05:17:32.328966+0000","last_clean":"2023-10-06T05:17:32.328966+0000","last_became_active":"2023-10-06T05:17:32.328642+0000","last_became_peered":"2023-10-06T05:17:32.328642+0000","last_unstale":"2023-10-06T05:17:32.328966+0000","last_undegraded":"2023-10-06T05:17:32.328966+0000","last_fullsized":"2023-10-06T05:17:32.328966+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:17.642543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323743+0000","last_change":"2023-10-06T05:17:32.323743+0000","last_active":"2023-10-06T05:17:32.323743+0000","last_peered":"2023-10-06T05:17:32.323743+0000","last_clean":"2023-10-06T05:17:32.323743+0000","last_became_active":"2023-10-06T05:17:32.323448+0000","last_became_peered":"2023-10-06T05:17:32.323448+0000","last_unstale":"2023-10-06T05:17:32.323743+0000","last_undegraded":"2023-10-06T05:17:32.323743+0000","last_fullsized":"2023-10-06T05:17:32.323743+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:25.465841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.335342+0000","last_change":"2023-10-06T05:17:32.335342+0000","last_active":"2023-10-06T05:17:32.335342+0000","last_peered":"2023-10-06T05:17:32.335342+0000","last_clean":"2023-10-06T05:17:32.335342+0000","last_became_active":"2023-10-06T05:17:32.334962+0000","last_became_peered":"2023-10-06T05:17:32.334962+0000","last_unstale":"2023-10-06T05:17:32.335342+0000","last_undegraded":"2023-10-06T05:17:32.335342+0000","last_fullsized":"2023-10-06T05:17:32.335342+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:15.535704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.326336+0000","last_change":"2023-10-06T05:17:32.326336+0000","last_active":"2023-10-06T05:17:32.326336+0000","last_peered":"2023-10-06T05:17:32.326336+0000","last_clean":"2023-10-06T05:17:32.326336+0000","last_became_active":"2023-10-06T05:17:32.325403+0000","last_became_peered":"2023-10-06T05:17:32.325403+0000","last_unstale":"2023-10-06T05:17:32.326336+0000","last_undegraded":"2023-10-06T05:17:32.326336+0000","last_fullsized":"2023-10-06T05:17:32.326336+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:02.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332269+0000","last_change":"2023-10-06T05:17:32.332269+0000","last_active":"2023-10-06T05:17:32.332269+0000","last_peered":"2023-10-06T05:17:32.332269+0000","last_clean":"2023-10-06T05:17:32.332269+0000","last_became_active":"2023-10-06T05:17:32.331989+0000","last_became_peered":"2023-10-06T05:17:32.331989+0000","last_unstale":"2023-10-06T05:17:32.332269+0000","last_undegraded":"2023-10-06T05:17:32.332269+0000","last_fullsized":"2023-10-06T05:17:32.332269+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:11.282204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.338712+0000","last_change":"2023-10-06T05:17:32.338712+0000","last_active":"2023-10-06T05:17:32.338712+0000","last_peered":"2023-10-06T05:17:32.338712+0000","last_clean":"2023-10-06T05:17:32.338712+0000","last_became_active":"2023-10-06T05:17:32.332599+0000","last_became_peered":"2023-10-06T05:17:32.332599+0000","last_unstale":"2023-10-06T05:17:32.338712+0000","last_undegraded":"2023-10-06T05:17:32.338712+0000","last_fullsized":"2023-10-06T05:17:32.338712+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:09.860772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324919+0000","last_change":"2023-10-06T05:17:32.324919+0000","last_active":"2023-10-06T05:17:32.324919+0000","last_peered":"2023-10-06T05:17:32.324919+0000","last_clean":"2023-10-06T05:17:32.324919+0000","last_became_active":"2023-10-06T05:17:32.324534+0000","last_became_peered":"2023-10-06T05:17:32.324534+0000","last_unstale":"2023-10-06T05:17:32.324919+0000","last_undegraded":"2023-10-06T05:17:32.324919+0000","last_fullsized":"2023-10-06T05:17:32.324919+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:31.332500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.343701+0000","last_change":"2023-10-06T05:17:31.324008+0000","last_active":"2023-10-06T05:17:32.343701+0000","last_peered":"2023-10-06T05:17:32.343701+0000","last_clean":"2023-10-06T05:17:32.343701+0000","last_became_active":"2023-10-06T05:17:31.322815+0000","last_became_peered":"2023-10-06T05:17:31.322815+0000","last_unstale":"2023-10-06T05:17:32.343701+0000","last_undegraded":"2023-10-06T05:17:32.343701+0000","last_fullsized":"2023-10-06T05:17:32.343701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:48.854489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.343670+0000","last_change":"2023-10-06T05:17:32.343670+0000","last_active":"2023-10-06T05:17:32.343670+0000","last_peered":"2023-10-06T05:17:32.343670+0000","last_clean":"2023-10-06T05:17:32.343670+0000","last_became_active":"2023-10-06T05:17:32.343423+0000","last_became_peered":"2023-10-06T05:17:32.343423+0000","last_unstale":"2023-10-06T05:17:32.343670+0000","last_undegraded":"2023-10-06T05:17:32.343670+0000","last_fullsized":"2023-10-06T05:17:32.343670+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:19.889711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.314480+0000","last_change":"2023-10-06T05:17:31.330260+0000","last_active":"2023-10-06T05:17:32.314480+0000","last_peered":"2023-10-06T05:17:32.314480+0000","last_clean":"2023-10-06T05:17:32.314480+0000","last_became_active":"2023-10-06T05:17:31.329970+0000","last_became_peered":"2023-10-06T05:17:31.329970+0000","last_unstale":"2023-10-06T05:17:32.314480+0000","last_undegraded":"2023-10-06T05:17:32.314480+0000","last_fullsized":"2023-10-06T05:17:32.314480+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:40.767026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.333720+0000","last_change":"2023-10-06T05:17:32.333720+0000","last_active":"2023-10-06T05:17:32.333720+0000","last_peered":"2023-10-06T05:17:32.333720+0000","last_clean":"2023-10-06T05:17:32.333720+0000","last_became_active":"2023-10-06T05:17:32.332907+0000","last_became_peered":"2023-10-06T05:17:32.332907+0000","last_unstale":"2023-10-06T05:17:32.333720+0000","last_undegraded":"2023-10-06T05:17:32.333720+0000","last_fullsized":"2023-10-06T05:17:32.333720+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:53.606888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.346753+0000","last_change":"2023-10-06T05:17:31.322504+0000","last_active":"2023-10-06T05:17:32.346753+0000","last_peered":"2023-10-06T05:17:32.346753+0000","last_clean":"2023-10-06T05:17:32.346753+0000","last_became_active":"2023-10-06T05:17:31.322207+0000","last_became_peered":"2023-10-06T05:17:31.322207+0000","last_unstale":"2023-10-06T05:17:32.346753+0000","last_undegraded":"2023-10-06T05:17:32.346753+0000","last_fullsized":"2023-10-06T05:17:32.346753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:55.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323919+0000","last_change":"2023-10-06T05:17:32.323919+0000","last_active":"2023-10-06T05:17:32.323919+0000","last_peered":"2023-10-06T05:17:32.323919+0000","last_clean":"2023-10-06T05:17:32.323919+0000","last_became_active":"2023-10-06T05:17:32.323695+0000","last_became_peered":"2023-10-06T05:17:32.323695+0000","last_unstale":"2023-10-06T05:17:32.323919+0000","last_undegraded":"2023-10-06T05:17:32.323919+0000","last_fullsized":"2023-10-06T05:17:32.323919+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:20.624998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.337958+0000","last_change":"2023-10-06T05:17:31.321969+0000","last_active":"2023-10-06T05:17:32.337958+0000","last_peered":"2023-10-06T05:17:32.337958+0000","last_clean":"2023-10-06T05:17:32.337958+0000","last_became_active":"2023-10-06T05:17:31.321689+0000","last_became_peered":"2023-10-06T05:17:31.321689+0000","last_unstale":"2023-10-06T05:17:32.337958+0000","last_undegraded":"2023-10-06T05:17:32.337958+0000","last_fullsized":"2023-10-06T05:17:32.337958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:14.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.330086+0000","last_change":"2023-10-06T05:17:32.330086+0000","last_active":"2023-10-06T05:17:32.330086+0000","last_peered":"2023-10-06T05:17:32.330086+0000","last_clean":"2023-10-06T05:17:32.330086+0000","last_became_active":"2023-10-06T05:17:32.327658+0000","last_became_peered":"2023-10-06T05:17:32.327658+0000","last_unstale":"2023-10-06T05:17:32.330086+0000","last_undegraded":"2023-10-06T05:17:32.330086+0000","last_fullsized":"2023-10-06T05:17:32.330086+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:25.110866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.310009+0000","last_change":"2023-10-06T05:17:31.316565+0000","last_active":"2023-10-06T05:17:32.310009+0000","last_peered":"2023-10-06T05:17:32.310009+0000","last_clean":"2023-10-06T05:17:32.310009+0000","last_became_active":"2023-10-06T05:17:31.316281+0000","last_became_peered":"2023-10-06T05:17:31.316281+0000","last_unstale":"2023-10-06T05:17:32.310009+0000","last_undegraded":"2023-10-06T05:17:32.310009+0000","last_fullsized":"2023-10-06T05:17:32.310009+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:24.897513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.331320+0000","last_change":"2023-10-06T05:17:32.331320+0000","last_active":"2023-10-06T05:17:32.331320+0000","last_peered":"2023-10-06T05:17:32.331320+0000","last_clean":"2023-10-06T05:17:32.331320+0000","last_became_active":"2023-10-06T05:17:32.331054+0000","last_became_peered":"2023-10-06T05:17:32.331054+0000","last_unstale":"2023-10-06T05:17:32.331320+0000","last_undegraded":"2023-10-06T05:17:32.331320+0000","last_fullsized":"2023-10-06T05:17:32.331320+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:11.707587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.315754+0000","last_change":"2023-10-06T05:17:31.326372+0000","last_active":"2023-10-06T05:17:32.315754+0000","last_peered":"2023-10-06T05:17:32.315754+0000","last_clean":"2023-10-06T05:17:32.315754+0000","last_became_active":"2023-10-06T05:17:31.325435+0000","last_became_peered":"2023-10-06T05:17:31.325435+0000","last_unstale":"2023-10-06T05:17:32.315754+0000","last_undegraded":"2023-10-06T05:17:32.315754+0000","last_fullsized":"2023-10-06T05:17:32.315754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:39.560479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.324268+0000","last_change":"2023-10-06T05:17:32.324268+0000","last_active":"2023-10-06T05:17:32.324268+0000","last_peered":"2023-10-06T05:17:32.324268+0000","last_clean":"2023-10-06T05:17:32.324268+0000","last_became_active":"2023-10-06T05:17:32.323996+0000","last_became_peered":"2023-10-06T05:17:32.323996+0000","last_unstale":"2023-10-06T05:17:32.324268+0000","last_undegraded":"2023-10-06T05:17:32.324268+0000","last_fullsized":"2023-10-06T05:17:32.324268+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:32.572071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.325989+0000","last_change":"2023-10-06T05:17:31.322930+0000","last_active":"2023-10-06T05:17:31.325989+0000","last_peered":"2023-10-06T05:17:31.325989+0000","last_clean":"2023-10-06T05:17:31.325989+0000","last_became_active":"2023-10-06T05:17:31.322266+0000","last_became_peered":"2023-10-06T05:17:31.322266+0000","last_unstale":"2023-10-06T05:17:31.325989+0000","last_undegraded":"2023-10-06T05:17:31.325989+0000","last_fullsized":"2023-10-06T05:17:31.325989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:01.814012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.321836+0000","last_change":"2023-10-06T05:17:32.321836+0000","last_active":"2023-10-06T05:17:32.321836+0000","last_peered":"2023-10-06T05:17:32.321836+0000","last_clean":"2023-10-06T05:17:32.321836+0000","last_became_active":"2023-10-06T05:17:32.321344+0000","last_became_peered":"2023-10-06T05:17:32.321344+0000","last_unstale":"2023-10-06T05:17:32.321836+0000","last_undegraded":"2023-10-06T05:17:32.321836+0000","last_fullsized":"2023-10-06T05:17:32.321836+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:13.083592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.332323+0000","last_change":"2023-10-06T05:17:31.331270+0000","last_active":"2023-10-06T05:17:31.332323+0000","last_peered":"2023-10-06T05:17:31.332323+0000","last_clean":"2023-10-06T05:17:31.332323+0000","last_became_active":"2023-10-06T05:17:31.331015+0000","last_became_peered":"2023-10-06T05:17:31.331015+0000","last_unstale":"2023-10-06T05:17:31.332323+0000","last_undegraded":"2023-10-06T05:17:31.332323+0000","last_fullsized":"2023-10-06T05:17:31.332323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:38.572266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.346173+0000","last_change":"2023-10-06T05:17:32.346173+0000","last_active":"2023-10-06T05:17:32.346173+0000","last_peered":"2023-10-06T05:17:32.346173+0000","last_clean":"2023-10-06T05:17:32.346173+0000","last_became_active":"2023-10-06T05:17:32.345913+0000","last_became_peered":"2023-10-06T05:17:32.345913+0000","last_unstale":"2023-10-06T05:17:32.346173+0000","last_undegraded":"2023-10-06T05:17:32.346173+0000","last_fullsized":"2023-10-06T05:17:32.346173+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:46.301441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.328628+0000","last_change":"2023-10-06T05:17:31.328228+0000","last_active":"2023-10-06T05:17:31.328628+0000","last_peered":"2023-10-06T05:17:31.328628+0000","last_clean":"2023-10-06T05:17:31.328628+0000","last_became_active":"2023-10-06T05:17:31.327992+0000","last_became_peered":"2023-10-06T05:17:31.327992+0000","last_unstale":"2023-10-06T05:17:31.328628+0000","last_undegraded":"2023-10-06T05:17:31.328628+0000","last_fullsized":"2023-10-06T05:17:31.328628+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:52.569607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332340+0000","last_change":"2023-10-06T05:17:32.332340+0000","last_active":"2023-10-06T05:17:32.332340+0000","last_peered":"2023-10-06T05:17:32.332340+0000","last_clean":"2023-10-06T05:17:32.332340+0000","last_became_active":"2023-10-06T05:17:32.332074+0000","last_became_peered":"2023-10-06T05:17:32.332074+0000","last_unstale":"2023-10-06T05:17:32.332340+0000","last_undegraded":"2023-10-06T05:17:32.332340+0000","last_fullsized":"2023-10-06T05:17:32.332340+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:39.577618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.332449+0000","last_change":"2023-10-06T05:17:31.325650+0000","last_active":"2023-10-06T05:17:31.332449+0000","last_peered":"2023-10-06T05:17:31.332449+0000","last_clean":"2023-10-06T05:17:31.332449+0000","last_became_active":"2023-10-06T05:17:31.325353+0000","last_became_peered":"2023-10-06T05:17:31.325353+0000","last_unstale":"2023-10-06T05:17:31.332449+0000","last_undegraded":"2023-10-06T05:17:31.332449+0000","last_fullsized":"2023-10-06T05:17:31.332449+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:38.572266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332870+0000","last_change":"2023-10-06T05:17:32.332870+0000","last_active":"2023-10-06T05:17:32.332870+0000","last_peered":"2023-10-06T05:17:32.332870+0000","last_clean":"2023-10-06T05:17:32.332870+0000","last_became_active":"2023-10-06T05:17:32.332435+0000","last_became_peered":"2023-10-06T05:17:32.332435+0000","last_unstale":"2023-10-06T05:17:32.332870+0000","last_undegraded":"2023-10-06T05:17:32.332870+0000","last_fullsized":"2023-10-06T05:17:32.332870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:53.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.345896+0000","last_change":"2023-10-06T05:17:31.325306+0000","last_active":"2023-10-06T05:17:32.345896+0000","last_peered":"2023-10-06T05:17:32.345896+0000","last_clean":"2023-10-06T05:17:32.345896+0000","last_became_active":"2023-10-06T05:17:31.324963+0000","last_became_peered":"2023-10-06T05:17:31.324963+0000","last_unstale":"2023-10-06T05:17:32.345896+0000","last_undegraded":"2023-10-06T05:17:32.345896+0000","last_fullsized":"2023-10-06T05:17:32.345896+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:58.027207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332761+0000","last_change":"2023-10-06T05:17:32.332761+0000","last_active":"2023-10-06T05:17:32.332761+0000","last_peered":"2023-10-06T05:17:32.332761+0000","last_clean":"2023-10-06T05:17:32.332761+0000","last_became_active":"2023-10-06T05:17:32.332306+0000","last_became_peered":"2023-10-06T05:17:32.332306+0000","last_unstale":"2023-10-06T05:17:32.332761+0000","last_undegraded":"2023-10-06T05:17:32.332761+0000","last_fullsized":"2023-10-06T05:17:32.332761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:22.438165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.344001+0000","last_change":"2023-10-06T05:17:31.325799+0000","last_active":"2023-10-06T05:17:32.344001+0000","last_peered":"2023-10-06T05:17:32.344001+0000","last_clean":"2023-10-06T05:17:32.344001+0000","last_became_active":"2023-10-06T05:17:31.325559+0000","last_became_peered":"2023-10-06T05:17:31.325559+0000","last_unstale":"2023-10-06T05:17:32.344001+0000","last_undegraded":"2023-10-06T05:17:32.344001+0000","last_fullsized":"2023-10-06T05:17:32.344001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:39.560479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332491+0000","last_change":"2023-10-06T05:17:32.332491+0000","last_active":"2023-10-06T05:17:32.332491+0000","last_peered":"2023-10-06T05:17:32.332491+0000","last_clean":"2023-10-06T05:17:32.332491+0000","last_became_active":"2023-10-06T05:17:32.332221+0000","last_became_peered":"2023-10-06T05:17:32.332221+0000","last_unstale":"2023-10-06T05:17:32.332491+0000","last_undegraded":"2023-10-06T05:17:32.332491+0000","last_fullsized":"2023-10-06T05:17:32.332491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:19.977222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.341142+0000","last_change":"2023-10-06T05:17:31.329588+0000","last_active":"2023-10-06T05:17:32.341142+0000","last_peered":"2023-10-06T05:17:32.341142+0000","last_clean":"2023-10-06T05:17:32.341142+0000","last_became_active":"2023-10-06T05:17:31.329290+0000","last_became_peered":"2023-10-06T05:17:31.329290+0000","last_unstale":"2023-10-06T05:17:32.341142+0000","last_undegraded":"2023-10-06T05:17:32.341142+0000","last_fullsized":"2023-10-06T05:17:32.341142+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:39.560479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.323899+0000","last_change":"2023-10-06T05:17:32.323899+0000","last_active":"2023-10-06T05:17:32.323899+0000","last_peered":"2023-10-06T05:17:32.323899+0000","last_clean":"2023-10-06T05:17:32.323899+0000","last_became_active":"2023-10-06T05:17:32.323309+0000","last_became_peered":"2023-10-06T05:17:32.323309+0000","last_unstale":"2023-10-06T05:17:32.323899+0000","last_undegraded":"2023-10-06T05:17:32.323899+0000","last_fullsized":"2023-10-06T05:17:32.323899+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:56.585110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.323946+0000","last_change":"2023-10-06T05:17:31.321935+0000","last_active":"2023-10-06T05:17:31.323946+0000","last_peered":"2023-10-06T05:17:31.323946+0000","last_clean":"2023-10-06T05:17:31.323946+0000","last_became_active":"2023-10-06T05:17:31.321671+0000","last_became_peered":"2023-10-06T05:17:31.321671+0000","last_unstale":"2023-10-06T05:17:31.323946+0000","last_undegraded":"2023-10-06T05:17:31.323946+0000","last_fullsized":"2023-10-06T05:17:31.323946+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:10.276853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.320953+0000","last_change":"2023-10-06T05:17:32.320953+0000","last_active":"2023-10-06T05:17:32.320953+0000","last_peered":"2023-10-06T05:17:32.320953+0000","last_clean":"2023-10-06T05:17:32.320953+0000","last_became_active":"2023-10-06T05:17:32.320298+0000","last_became_peered":"2023-10-06T05:17:32.320298+0000","last_unstale":"2023-10-06T05:17:32.320953+0000","last_undegraded":"2023-10-06T05:17:32.320953+0000","last_fullsized":"2023-10-06T05:17:32.320953+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.326866+0000","last_change":"2023-10-06T05:17:31.324539+0000","last_active":"2023-10-06T05:17:31.326866+0000","last_peered":"2023-10-06T05:17:31.326866+0000","last_clean":"2023-10-06T05:17:31.326866+0000","last_became_active":"2023-10-06T05:17:31.324225+0000","last_became_peered":"2023-10-06T05:17:31.324225+0000","last_unstale":"2023-10-06T05:17:31.326866+0000","last_undegraded":"2023-10-06T05:17:31.326866+0000","last_fullsized":"2023-10-06T05:17:31.326866+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:01.504929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.336350+0000","last_change":"2023-10-06T05:17:32.336350+0000","last_active":"2023-10-06T05:17:32.336350+0000","last_peered":"2023-10-06T05:17:32.336350+0000","last_clean":"2023-10-06T05:17:32.336350+0000","last_became_active":"2023-10-06T05:17:32.333629+0000","last_became_peered":"2023-10-06T05:17:32.333629+0000","last_unstale":"2023-10-06T05:17:32.336350+0000","last_undegraded":"2023-10-06T05:17:32.336350+0000","last_fullsized":"2023-10-06T05:17:32.336350+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:19.977222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.338458+0000","last_change":"2023-10-06T05:17:31.324029+0000","last_active":"2023-10-06T05:17:32.338458+0000","last_peered":"2023-10-06T05:17:32.338458+0000","last_clean":"2023-10-06T05:17:32.338458+0000","last_became_active":"2023-10-06T05:17:31.323760+0000","last_became_peered":"2023-10-06T05:17:31.323760+0000","last_unstale":"2023-10-06T05:17:32.338458+0000","last_undegraded":"2023-10-06T05:17:32.338458+0000","last_fullsized":"2023-10-06T05:17:32.338458+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:09.104643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325174+0000","last_change":"2023-10-06T05:17:32.325174+0000","last_active":"2023-10-06T05:17:32.325174+0000","last_peered":"2023-10-06T05:17:32.325174+0000","last_clean":"2023-10-06T05:17:32.325174+0000","last_became_active":"2023-10-06T05:17:32.324880+0000","last_became_peered":"2023-10-06T05:17:32.324880+0000","last_unstale":"2023-10-06T05:17:32.325174+0000","last_undegraded":"2023-10-06T05:17:32.325174+0000","last_fullsized":"2023-10-06T05:17:32.325174+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:04.156249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.345437+0000","last_change":"2023-10-06T05:17:31.327171+0000","last_active":"2023-10-06T05:17:32.345437+0000","last_peered":"2023-10-06T05:17:32.345437+0000","last_clean":"2023-10-06T05:17:32.345437+0000","last_became_active":"2023-10-06T05:17:31.326864+0000","last_became_peered":"2023-10-06T05:17:31.326864+0000","last_unstale":"2023-10-06T05:17:32.345437+0000","last_undegraded":"2023-10-06T05:17:32.345437+0000","last_fullsized":"2023-10-06T05:17:32.345437+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:32.489979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.325287+0000","last_change":"2023-10-06T05:17:32.325287+0000","last_active":"2023-10-06T05:17:32.325287+0000","last_peered":"2023-10-06T05:17:32.325287+0000","last_clean":"2023-10-06T05:17:32.325287+0000","last_became_active":"2023-10-06T05:17:32.324985+0000","last_became_peered":"2023-10-06T05:17:32.324985+0000","last_unstale":"2023-10-06T05:17:32.325287+0000","last_undegraded":"2023-10-06T05:17:32.325287+0000","last_fullsized":"2023-10-06T05:17:32.325287+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:32.942267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.330365+0000","last_change":"2023-10-06T05:17:31.325945+0000","last_active":"2023-10-06T05:17:31.330365+0000","last_peered":"2023-10-06T05:17:31.330365+0000","last_clean":"2023-10-06T05:17:31.330365+0000","last_became_active":"2023-10-06T05:17:31.325626+0000","last_became_peered":"2023-10-06T05:17:31.325626+0000","last_unstale":"2023-10-06T05:17:31.330365+0000","last_undegraded":"2023-10-06T05:17:31.330365+0000","last_fullsized":"2023-10-06T05:17:31.330365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:38.572266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332688+0000","last_change":"2023-10-06T05:17:32.332688+0000","last_active":"2023-10-06T05:17:32.332688+0000","last_peered":"2023-10-06T05:17:32.332688+0000","last_clean":"2023-10-06T05:17:32.332688+0000","last_became_active":"2023-10-06T05:17:32.332092+0000","last_became_peered":"2023-10-06T05:17:32.332092+0000","last_unstale":"2023-10-06T05:17:32.332688+0000","last_undegraded":"2023-10-06T05:17:32.332688+0000","last_fullsized":"2023-10-06T05:17:32.332688+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:18.765768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.322088+0000","last_change":"2023-10-06T05:17:31.319420+0000","last_active":"2023-10-06T05:17:31.322088+0000","last_peered":"2023-10-06T05:17:31.322088+0000","last_clean":"2023-10-06T05:17:31.322088+0000","last_became_active":"2023-10-06T05:17:31.319157+0000","last_became_peered":"2023-10-06T05:17:31.319157+0000","last_unstale":"2023-10-06T05:17:31.322088+0000","last_undegraded":"2023-10-06T05:17:31.322088+0000","last_fullsized":"2023-10-06T05:17:31.322088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:08.741877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.328163+0000","last_change":"2023-10-06T05:17:32.328163+0000","last_active":"2023-10-06T05:17:32.328163+0000","last_peered":"2023-10-06T05:17:32.328163+0000","last_clean":"2023-10-06T05:17:32.328163+0000","last_became_active":"2023-10-06T05:17:32.327869+0000","last_became_peered":"2023-10-06T05:17:32.327869+0000","last_unstale":"2023-10-06T05:17:32.328163+0000","last_undegraded":"2023-10-06T05:17:32.328163+0000","last_fullsized":"2023-10-06T05:17:32.328163+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:46.301441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.321561+0000","last_change":"2023-10-06T05:17:31.319512+0000","last_active":"2023-10-06T05:17:31.321561+0000","last_peered":"2023-10-06T05:17:31.321561+0000","last_clean":"2023-10-06T05:17:31.321561+0000","last_became_active":"2023-10-06T05:17:31.319272+0000","last_became_peered":"2023-10-06T05:17:31.319272+0000","last_unstale":"2023-10-06T05:17:31.321561+0000","last_undegraded":"2023-10-06T05:17:31.321561+0000","last_fullsized":"2023-10-06T05:17:31.321561+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:14.577354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.334145+0000","last_change":"2023-10-06T05:17:32.334145+0000","last_active":"2023-10-06T05:17:32.334145+0000","last_peered":"2023-10-06T05:17:32.334145+0000","last_clean":"2023-10-06T05:17:32.334145+0000","last_became_active":"2023-10-06T05:17:32.333884+0000","last_became_peered":"2023-10-06T05:17:32.333884+0000","last_unstale":"2023-10-06T05:17:32.334145+0000","last_undegraded":"2023-10-06T05:17:32.334145+0000","last_fullsized":"2023-10-06T05:17:32.334145+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:46.301441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.322808+0000","last_change":"2023-10-06T05:17:31.322243+0000","last_active":"2023-10-06T05:17:31.322808+0000","last_peered":"2023-10-06T05:17:31.322808+0000","last_clean":"2023-10-06T05:17:31.322808+0000","last_became_active":"2023-10-06T05:17:31.321956+0000","last_became_peered":"2023-10-06T05:17:31.321956+0000","last_unstale":"2023-10-06T05:17:31.322808+0000","last_undegraded":"2023-10-06T05:17:31.322808+0000","last_fullsized":"2023-10-06T05:17:31.322808+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:25.133521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.344083+0000","last_change":"2023-10-06T05:17:32.344083+0000","last_active":"2023-10-06T05:17:32.344083+0000","last_peered":"2023-10-06T05:17:32.344083+0000","last_clean":"2023-10-06T05:17:32.344083+0000","last_became_active":"2023-10-06T05:17:32.343817+0000","last_became_peered":"2023-10-06T05:17:32.343817+0000","last_unstale":"2023-10-06T05:17:32.344083+0000","last_undegraded":"2023-10-06T05:17:32.344083+0000","last_fullsized":"2023-10-06T05:17:32.344083+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:05.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.324469+0000","last_change":"2023-10-06T05:17:31.323537+0000","last_active":"2023-10-06T05:17:31.324469+0000","last_peered":"2023-10-06T05:17:31.324469+0000","last_clean":"2023-10-06T05:17:31.324469+0000","last_became_active":"2023-10-06T05:17:31.323260+0000","last_became_peered":"2023-10-06T05:17:31.323260+0000","last_unstale":"2023-10-06T05:17:31.324469+0000","last_undegraded":"2023-10-06T05:17:31.324469+0000","last_fullsized":"2023-10-06T05:17:31.324469+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.432813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329631+0000","last_change":"2023-10-06T05:17:32.329631+0000","last_active":"2023-10-06T05:17:32.329631+0000","last_peered":"2023-10-06T05:17:32.329631+0000","last_clean":"2023-10-06T05:17:32.329631+0000","last_became_active":"2023-10-06T05:17:32.329392+0000","last_became_peered":"2023-10-06T05:17:32.329392+0000","last_unstale":"2023-10-06T05:17:32.329631+0000","last_undegraded":"2023-10-06T05:17:32.329631+0000","last_fullsized":"2023-10-06T05:17:32.329631+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:42.655847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.324402+0000","last_change":"2023-10-06T05:17:31.323405+0000","last_active":"2023-10-06T05:17:31.324402+0000","last_peered":"2023-10-06T05:17:31.324402+0000","last_clean":"2023-10-06T05:17:31.324402+0000","last_became_active":"2023-10-06T05:17:31.323111+0000","last_became_peered":"2023-10-06T05:17:31.323111+0000","last_unstale":"2023-10-06T05:17:31.324402+0000","last_undegraded":"2023-10-06T05:17:31.324402+0000","last_fullsized":"2023-10-06T05:17:31.324402+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:20.012975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.321667+0000","last_change":"2023-10-06T05:17:32.321667+0000","last_active":"2023-10-06T05:17:32.321667+0000","last_peered":"2023-10-06T05:17:32.321667+0000","last_clean":"2023-10-06T05:17:32.321667+0000","last_became_active":"2023-10-06T05:17:32.321053+0000","last_became_peered":"2023-10-06T05:17:32.321053+0000","last_unstale":"2023-10-06T05:17:32.321667+0000","last_undegraded":"2023-10-06T05:17:32.321667+0000","last_fullsized":"2023-10-06T05:17:32.321667+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:54.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.333184+0000","last_change":"2023-10-06T05:17:31.330012+0000","last_active":"2023-10-06T05:17:31.333184+0000","last_peered":"2023-10-06T05:17:31.333184+0000","last_clean":"2023-10-06T05:17:31.333184+0000","last_became_active":"2023-10-06T05:17:31.329826+0000","last_became_peered":"2023-10-06T05:17:31.329826+0000","last_unstale":"2023-10-06T05:17:31.333184+0000","last_undegraded":"2023-10-06T05:17:31.333184+0000","last_fullsized":"2023-10-06T05:17:31.333184+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:48.854489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332061+0000","last_change":"2023-10-06T05:17:32.332061+0000","last_active":"2023-10-06T05:17:32.332061+0000","last_peered":"2023-10-06T05:17:32.332061+0000","last_clean":"2023-10-06T05:17:32.332061+0000","last_became_active":"2023-10-06T05:17:32.331672+0000","last_became_peered":"2023-10-06T05:17:32.331672+0000","last_unstale":"2023-10-06T05:17:32.332061+0000","last_undegraded":"2023-10-06T05:17:32.332061+0000","last_fullsized":"2023-10-06T05:17:32.332061+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.343083+0000","last_change":"2023-10-06T05:17:31.326011+0000","last_active":"2023-10-06T05:17:32.343083+0000","last_peered":"2023-10-06T05:17:32.343083+0000","last_clean":"2023-10-06T05:17:32.343083+0000","last_became_active":"2023-10-06T05:17:31.325708+0000","last_became_peered":"2023-10-06T05:17:31.325708+0000","last_unstale":"2023-10-06T05:17:32.343083+0000","last_undegraded":"2023-10-06T05:17:32.343083+0000","last_fullsized":"2023-10-06T05:17:32.343083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:44.722155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.315343+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:17:31.315343+0000","last_peered":"2023-10-06T05:17:31.315343+0000","last_clean":"2023-10-06T05:17:31.315343+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:17:31.315343+0000","last_undegraded":"2023-10-06T05:17:31.315343+0000","last_fullsized":"2023-10-06T05:17:31.315343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.327832+0000","last_change":"2023-10-06T05:17:32.327832+0000","last_active":"2023-10-06T05:17:32.327832+0000","last_peered":"2023-10-06T05:17:32.327832+0000","last_clean":"2023-10-06T05:17:32.327832+0000","last_became_active":"2023-10-06T05:17:32.327560+0000","last_became_peered":"2023-10-06T05:17:32.327560+0000","last_unstale":"2023-10-06T05:17:32.327832+0000","last_undegraded":"2023-10-06T05:17:32.327832+0000","last_fullsized":"2023-10-06T05:17:32.327832+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:52.892989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.330614+0000","last_change":"2023-10-06T05:17:31.330213+0000","last_active":"2023-10-06T05:17:31.330614+0000","last_peered":"2023-10-06T05:17:31.330614+0000","last_clean":"2023-10-06T05:17:31.330614+0000","last_became_active":"2023-10-06T05:17:31.330002+0000","last_became_peered":"2023-10-06T05:17:31.330002+0000","last_unstale":"2023-10-06T05:17:31.330614+0000","last_undegraded":"2023-10-06T05:17:31.330614+0000","last_fullsized":"2023-10-06T05:17:31.330614+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:49.510325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.313524+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:17:31.313524+0000","last_peered":"2023-10-06T05:17:31.313524+0000","last_clean":"2023-10-06T05:17:31.313524+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:17:31.313524+0000","last_undegraded":"2023-10-06T05:17:31.313524+0000","last_fullsized":"2023-10-06T05:17:31.313524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332341+0000","last_change":"2023-10-06T05:17:32.332341+0000","last_active":"2023-10-06T05:17:32.332341+0000","last_peered":"2023-10-06T05:17:32.332341+0000","last_clean":"2023-10-06T05:17:32.332341+0000","last_became_active":"2023-10-06T05:17:32.331820+0000","last_became_peered":"2023-10-06T05:17:32.331820+0000","last_unstale":"2023-10-06T05:17:32.332341+0000","last_undegraded":"2023-10-06T05:17:32.332341+0000","last_fullsized":"2023-10-06T05:17:32.332341+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:02.419819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.336649+0000","last_change":"2023-10-06T05:17:31.329363+0000","last_active":"2023-10-06T05:17:32.336649+0000","last_peered":"2023-10-06T05:17:32.336649+0000","last_clean":"2023-10-06T05:17:32.336649+0000","last_became_active":"2023-10-06T05:17:31.329195+0000","last_became_peered":"2023-10-06T05:17:31.329195+0000","last_unstale":"2023-10-06T05:17:32.336649+0000","last_undegraded":"2023-10-06T05:17:32.336649+0000","last_fullsized":"2023-10-06T05:17:32.336649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.432813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.310205+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:17:31.310205+0000","last_peered":"2023-10-06T05:17:31.310205+0000","last_clean":"2023-10-06T05:17:31.310205+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:17:31.310205+0000","last_undegraded":"2023-10-06T05:17:31.310205+0000","last_fullsized":"2023-10-06T05:17:31.310205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.339701+0000","last_change":"2023-10-06T05:17:32.339701+0000","last_active":"2023-10-06T05:17:32.339701+0000","last_peered":"2023-10-06T05:17:32.339701+0000","last_clean":"2023-10-06T05:17:32.339701+0000","last_became_active":"2023-10-06T05:17:32.339397+0000","last_became_peered":"2023-10-06T05:17:32.339397+0000","last_unstale":"2023-10-06T05:17:32.339701+0000","last_undegraded":"2023-10-06T05:17:32.339701+0000","last_fullsized":"2023-10-06T05:17:32.339701+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:21.360640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.322095+0000","last_change":"2023-10-06T05:17:31.319144+0000","last_active":"2023-10-06T05:17:31.322095+0000","last_peered":"2023-10-06T05:17:31.322095+0000","last_clean":"2023-10-06T05:17:31.322095+0000","last_became_active":"2023-10-06T05:17:31.318852+0000","last_became_peered":"2023-10-06T05:17:31.318852+0000","last_unstale":"2023-10-06T05:17:31.322095+0000","last_undegraded":"2023-10-06T05:17:31.322095+0000","last_fullsized":"2023-10-06T05:17:31.322095+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:26.830478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.315620+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:17:31.315620+0000","last_peered":"2023-10-06T05:17:31.315620+0000","last_clean":"2023-10-06T05:17:31.315620+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:17:31.315620+0000","last_undegraded":"2023-10-06T05:17:31.315620+0000","last_fullsized":"2023-10-06T05:17:31.315620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":189,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.320622+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:17:31.320622+0000","last_peered":"2023-10-06T05:17:31.320622+0000","last_clean":"2023-10-06T05:17:31.320622+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:17:31.320622+0000","last_undegraded":"2023-10-06T05:17:31.320622+0000","last_fullsized":"2023-10-06T05:17:31.320622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.329854+0000","last_change":"2023-10-06T05:17:32.329854+0000","last_active":"2023-10-06T05:17:32.329854+0000","last_peered":"2023-10-06T05:17:32.329854+0000","last_clean":"2023-10-06T05:17:32.329854+0000","last_became_active":"2023-10-06T05:17:32.328366+0000","last_became_peered":"2023-10-06T05:17:32.328366+0000","last_unstale":"2023-10-06T05:17:32.329854+0000","last_undegraded":"2023-10-06T05:17:32.329854+0000","last_fullsized":"2023-10-06T05:17:32.329854+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:38.341979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.319320+0000","last_change":"2023-10-06T05:17:31.318047+0000","last_active":"2023-10-06T05:17:31.319320+0000","last_peered":"2023-10-06T05:17:31.319320+0000","last_clean":"2023-10-06T05:17:31.319320+0000","last_became_active":"2023-10-06T05:17:31.317704+0000","last_became_peered":"2023-10-06T05:17:31.317704+0000","last_unstale":"2023-10-06T05:17:31.319320+0000","last_undegraded":"2023-10-06T05:17:31.319320+0000","last_fullsized":"2023-10-06T05:17:31.319320+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:26.830478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.320711+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:17:31.320711+0000","last_peered":"2023-10-06T05:17:31.320711+0000","last_clean":"2023-10-06T05:17:31.320711+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:17:31.320711+0000","last_undegraded":"2023-10-06T05:17:31.320711+0000","last_fullsized":"2023-10-06T05:17:31.320711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332735+0000","last_change":"2023-10-06T05:17:32.332735+0000","last_active":"2023-10-06T05:17:32.332735+0000","last_peered":"2023-10-06T05:17:32.332735+0000","last_clean":"2023-10-06T05:17:32.332735+0000","last_became_active":"2023-10-06T05:17:32.332480+0000","last_became_peered":"2023-10-06T05:17:32.332480+0000","last_unstale":"2023-10-06T05:17:32.332735+0000","last_undegraded":"2023-10-06T05:17:32.332735+0000","last_fullsized":"2023-10-06T05:17:32.332735+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:19.012509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.318043+0000","last_change":"2023-10-06T05:17:31.329439+0000","last_active":"2023-10-06T05:17:32.318043+0000","last_peered":"2023-10-06T05:17:32.318043+0000","last_clean":"2023-10-06T05:17:32.318043+0000","last_became_active":"2023-10-06T05:17:31.325911+0000","last_became_peered":"2023-10-06T05:17:31.325911+0000","last_unstale":"2023-10-06T05:17:32.318043+0000","last_undegraded":"2023-10-06T05:17:32.318043+0000","last_fullsized":"2023-10-06T05:17:32.318043+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:24.897513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.319912+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:17:31.319912+0000","last_peered":"2023-10-06T05:17:31.319912+0000","last_clean":"2023-10-06T05:17:31.319912+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:17:31.319912+0000","last_undegraded":"2023-10-06T05:17:31.319912+0000","last_fullsized":"2023-10-06T05:17:31.319912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.332592+0000","last_change":"2023-10-06T05:17:32.332592+0000","last_active":"2023-10-06T05:17:32.332592+0000","last_peered":"2023-10-06T05:17:32.332592+0000","last_clean":"2023-10-06T05:17:32.332592+0000","last_became_active":"2023-10-06T05:17:32.331981+0000","last_became_peered":"2023-10-06T05:17:32.331981+0000","last_unstale":"2023-10-06T05:17:32.332592+0000","last_undegraded":"2023-10-06T05:17:32.332592+0000","last_fullsized":"2023-10-06T05:17:32.332592+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:40.565830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.333489+0000","last_change":"2023-10-06T05:17:31.332822+0000","last_active":"2023-10-06T05:17:31.333489+0000","last_peered":"2023-10-06T05:17:31.333489+0000","last_clean":"2023-10-06T05:17:31.333489+0000","last_became_active":"2023-10-06T05:17:31.332529+0000","last_became_peered":"2023-10-06T05:17:31.332529+0000","last_unstale":"2023-10-06T05:17:31.333489+0000","last_undegraded":"2023-10-06T05:17:31.333489+0000","last_fullsized":"2023-10-06T05:17:31.333489+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:24.897513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.315457+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:17:31.315457+0000","last_peered":"2023-10-06T05:17:31.315457+0000","last_clean":"2023-10-06T05:17:31.315457+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:17:31.315457+0000","last_undegraded":"2023-10-06T05:17:31.315457+0000","last_fullsized":"2023-10-06T05:17:31.315457+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":8,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.330487+0000","last_change":"2023-10-06T05:17:32.330487+0000","last_active":"2023-10-06T05:17:32.330487+0000","last_peered":"2023-10-06T05:17:32.330487+0000","last_clean":"2023-10-06T05:17:32.330487+0000","last_became_active":"2023-10-06T05:17:32.330172+0000","last_became_peered":"2023-10-06T05:17:32.330172+0000","last_unstale":"2023-10-06T05:17:32.330487+0000","last_undegraded":"2023-10-06T05:17:32.330487+0000","last_fullsized":"2023-10-06T05:17:32.330487+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:31.300550+0000","last_clean_scrub_stamp":"2023-10-06T05:17:31.300550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:57.712279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-06T05:17:32.314467+0000","last_change":"2023-10-06T05:17:31.321981+0000","last_active":"2023-10-06T05:17:32.314467+0000","last_peered":"2023-10-06T05:17:32.314467+0000","last_clean":"2023-10-06T05:17:32.314467+0000","last_became_active":"2023-10-06T05:17:31.321791+0000","last_became_peered":"2023-10-06T05:17:31.321791+0000","last_unstale":"2023-10-06T05:17:32.314467+0000","last_undegraded":"2023-10-06T05:17:32.314467+0000","last_fullsized":"2023-10-06T05:17:32.314467+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:30.295198+0000","last_clean_scrub_stamp":"2023-10-06T05:17:30.295198+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:25.651129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-06T05:17:31.312116+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:17:31.312116+0000","last_peered":"2023-10-06T05:17:31.312116+0000","last_clean":"2023-10-06T05:17:31.312116+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:17:31.312116+0000","last_undegraded":"2023-10-06T05:17:31.312116+0000","last_fullsized":"2023-10-06T05:17:31.312116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298448,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073392,"statfs":{"total":96636764160,"available":96331153408,"internally_reserved":0,"allocated":1187840,"data_stored":864217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73199999999999998,"5min":0.876,"15min":0.876},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.753,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.62,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":1.476,"15min":1.476},"last":0.371},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65500000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.627,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.67800000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.498},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49299999999999999,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64300000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72699999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.746,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76700000000000002,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.97999999999999998,"15min":1.046},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77700000000000002,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82599999999999996,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.04,"15min":1.1120000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77800000000000002,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.81999999999999995,"15min":0.85199999999999998},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":1.5880000000000001,"15min":1.746},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.77800000000000002,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.98899999999999999,"15min":1.0449999999999999},"min":{"1min":0.53100000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.80700000000000005,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.90200000000000002,"15min":0.96899999999999997},"min":{"1min":0.33100000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64700000000000002,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.93300000000000005,"15min":0.997},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80800000000000005,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.34999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70999999999999996,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.92700000000000005,"15min":0.98799999999999999},"min":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77900000000000003,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.96499999999999997,"15min":1.0289999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82899999999999996,"5min":9.718,"15min":9.718},"last":0.497},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.99199999999999999,"15min":1.056},"min":{"1min":0.42999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85299999999999998,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.42899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.39300000000000002,"15min":0.39500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56799999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55500000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.442,"15min":0.443},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.48699999999999999,"15min":0.497},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":1.673,"15min":1.673},"last":0.34499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.58899999999999997,"15min":0.60999999999999999},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68000000000000005,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46400000000000002,"15min":0.46999999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.57899999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85799999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91400000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91200000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.91700000000000004,"5min":1.026,"15min":1.026},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.878,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298448,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073392,"statfs":{"total":96636764160,"available":96331153408,"internally_reserved":0,"allocated":1187840,"data_stored":864217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.27900000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.47999999999999998,"5min":1.038,"15min":1.038},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.41699999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58299999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91200000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.314,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55200000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.35599999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63500000000000001,"5min":1.016,"15min":1.016},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.625},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.77800000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.81599999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93500000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.746,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70399999999999996,"15min":0.70999999999999996},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.872,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":0.97399999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.80800000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.52900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.435,"15min":0.44400000000000001},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.57399999999999995,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.47399999999999998,"15min":0.48299999999999998},"min":{"1min":0.32100000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.57299999999999995,"5min":1.899,"15min":1.899},"last":0.32100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64300000000000002,"5min":2.097,"15min":2.097},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.625,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44700000000000001,"15min":0.45300000000000001},"min":{"1min":0.314,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.55100000000000005,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.35499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.61399999999999999,"5min":2.121,"15min":2.121},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.52300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90500000000000003,"5min":1.643,"15min":1.643},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71799999999999997,"15min":0.71999999999999997},"min":{"1min":0.55500000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":1.851,"15min":1.851},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.65400000000000003},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83399999999999996,"5min":1.571,"15min":1.571},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.53300000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98599999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86499999999999999,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95799999999999996,"5min":1.613,"15min":1.613},"last":0.5}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297864,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":589824,"data_stored":272496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63600000000000001,"15min":0.629},"min":{"1min":0.52200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.48899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.52700000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63700000000000001,"15min":0.63200000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.51400000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.499},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.48899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.51300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75900000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.44800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.372,"15min":0.372},"max":{"1min":0.82099999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.46899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.78700000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.70199999999999996,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.45900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.53200000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.747,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.623}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297868,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073972,"statfs":{"total":96636764160,"available":96331747328,"internally_reserved":0,"allocated":593920,"data_stored":273849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.40100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.46100000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.46300000000000002},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:17:36.927 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:17:36.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:17:37.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.098+0000 7fc7f72b0700 1 Processor -- start 2023-10-06T05:17:37.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.098+0000 7fc7f72b0700 1 -- start start 2023-10-06T05:17:37.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f72b0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7f006aa70 0x7fc7f00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f72b0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc7f015c8c0 0x7fc7f00aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f72b0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f0067d80 con 0x7fc7f00c7050 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f72b0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00ac620 con 0x7fc7f006aa70 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f72b0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00c75b0 con 0x7fc7f015c8c0 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f504c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7f006aa70 0x7fc7f00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f484b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc7f00c7050 0x7fc7f015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52242/0 (socket says 172.21.15.45:52242) 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f484b700 1 -- 172.21.15.45:0/3265429064 learned_addr learned my addr 172.21.15.45:0/3265429064 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.099+0000 7fc7f584d700 1 --2- 172.21.15.45:0/3265429064 >> v2:172.21.15.144:3300/0 conn(0x7fc7f015c8c0 0x7fc7f00aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7f504c700 1 -- 172.21.15.45:0/3265429064 >> v2:172.21.15.144:3300/0 conn(0x7fc7f015c8c0 msgr2=0x7fc7f00aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7f504c700 1 --2- 172.21.15.45:0/3265429064 >> v2:172.21.15.144:3300/0 conn(0x7fc7f015c8c0 0x7fc7f00aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7f504c700 1 -- 172.21.15.45:0/3265429064 >> v1:172.21.15.45:6789/0 conn(0x7fc7f00c7050 legacy=0x7fc7f015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7f504c700 1 -- 172.21.15.45:0/3265429064 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00c7890 con 0x7fc7f006aa70 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7e7fff700 1 -- 172.21.15.45:0/3265429064 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701714515 0 0) 0x7fc7f0067d80 con 0x7fc7f00c7050 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7f504c700 1 --2- 172.21.15.45:0/3265429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f006aa70 0x7fc7f00c6b10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00c8260 tx=0x7fc7ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b20ec84308e54e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7e7fff700 1 -- 172.21.15.45:0/3265429064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00b6b0 con 0x7fc7f006aa70 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7e7fff700 1 -- 172.21.15.45:0/3265429064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec00b810 con 0x7fc7f006aa70 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.100+0000 7fc7e7fff700 1 -- 172.21.15.45:0/3265429064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00bbb0 con 0x7fc7f006aa70 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 -- 172.21.15.45:0/3265429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f006aa70 msgr2=0x7fc7f00c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/3265429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f006aa70 0x7fc7f00c6b10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00c8260 tx=0x7fc7ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 -- 172.21.15.45:0/3265429064 shutdown_connections 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/3265429064 >> v2:172.21.15.144:3300/0 conn(0x7fc7f015c8c0 0x7fc7f00aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/3265429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f006aa70 0x7fc7f00c6b10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 -- 172.21.15.45:0/3265429064 >> 172.21.15.45:0/3265429064 conn(0x7fc7f00cc570 msgr2=0x7fc7f00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 -- 172.21.15.45:0/3265429064 shutdown_connections 2023-10-06T05:17:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.101+0000 7fc7f72b0700 1 -- 172.21.15.45:0/3265429064 wait complete. 2023-10-06T05:17:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.102+0000 7fc7f72b0700 1 Processor -- start 2023-10-06T05:17:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.102+0000 7fc7f72b0700 1 -- start start 2023-10-06T05:17:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f72b0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7f006aa70 0x7fc7f0159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f72b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f00c7050 0x7fc7f0155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f72b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f015c8c0 0x7fc7f0158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f72b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f015a4d0 con 0x7fc7f00c7050 2023-10-06T05:17:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f72b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00c1310 con 0x7fc7f015c8c0 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f72b0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00c15f0 con 0x7fc7f006aa70 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f00c7050 0x7fc7f0155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f00c7050 0x7fc7f0155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43922/0 (socket says 172.21.15.45:43922) 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 -- 172.21.15.45:0/1496983434 learned_addr learned my addr 172.21.15.45:0/1496983434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f504c700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7f006aa70 0x7fc7f0159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f584d700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f015c8c0 0x7fc7f0158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 -- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7f006aa70 msgr2=0x7fc7f0159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7f006aa70 0x7fc7f0159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 -- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f015c8c0 msgr2=0x7fc7f0158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f015c8c0 0x7fc7f0158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.103+0000 7fc7f484b700 1 -- 172.21.15.45:0/1496983434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009420 con 0x7fc7f00c7050 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.104+0000 7fc7f484b700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f00c7050 0x7fc7f0155a10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000c7e0 tx=0x7fc7e0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.104+0000 7fc7e57fa700 1 -- 172.21.15.45:0/1496983434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0005b30 con 0x7fc7f00c7050 2023-10-06T05:17:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.104+0000 7fc7e57fa700 1 -- 172.21.15.45:0/1496983434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0005c90 con 0x7fc7f00c7050 2023-10-06T05:17:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.104+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f00c18d0 con 0x7fc7f00c7050 2023-10-06T05:17:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.104+0000 7fc7e57fa700 1 -- 172.21.15.45:0/1496983434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0007300 con 0x7fc7f00c7050 2023-10-06T05:17:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.104+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f00c1e90 con 0x7fc7f00c7050 2023-10-06T05:17:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.105+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f015b180 con 0x7fc7f00c7050 2023-10-06T05:17:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.105+0000 7fc7e57fa700 1 -- 172.21.15.45:0/1496983434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7e00076b0 con 0x7fc7f00c7050 2023-10-06T05:17:37.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.106+0000 7fc7e57fa700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7d4066f30 0x7fc7d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.106+0000 7fc7e57fa700 1 -- 172.21.15.45:0/1496983434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc7e00842c0 con 0x7fc7f00c7050 2023-10-06T05:17:37.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.106+0000 7fc7f504c700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7d4066f30 0x7fc7d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.107+0000 7fc7f504c700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7d4066f30 0x7fc7d40693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0093f0 tx=0x7fc7ec017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.110+0000 7fc7e57fa700 1 -- 172.21.15.45:0/1496983434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7e0051310 con 0x7fc7f00c7050 2023-10-06T05:17:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.396+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc7f00c20b0 con 0x7fc7f00c7050 2023-10-06T05:17:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.399+0000 7fc7e57fa700 1 -- 172.21.15.45:0/1496983434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc7e0054930 con 0x7fc7f00c7050 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7d4066f30 msgr2=0x7fc7d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7d4066f30 0x7fc7d40693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0093f0 tx=0x7fc7ec017000 comp rx=0 tx=0).stop 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f00c7050 msgr2=0x7fc7f0155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f00c7050 0x7fc7f0155a10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000c7e0 tx=0x7fc7e0007df0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 shutdown_connections 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7d4066f30 0x7fc7d40693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f015c8c0 0x7fc7f0158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7f006aa70 0x7fc7f0159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 --2- 172.21.15.45:0/1496983434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f00c7050 0x7fc7f0155a10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 >> 172.21.15.45:0/1496983434 conn(0x7fc7f00cc570 msgr2=0x7fc7f00acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 shutdown_connections 2023-10-06T05:17:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.402+0000 7fc7f72b0700 1 -- 172.21.15.45:0/1496983434 wait complete. 2023-10-06T05:17:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-10-06T05:17:37.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:37.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.547+0000 7f73ff77a700 1 Processor -- start 2023-10-06T05:17:37.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.548+0000 7f73ff77a700 1 -- start start 2023-10-06T05:17:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.548+0000 7f73ff77a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73f8153620 0x7f73f815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73ff77a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f73f80ccfc0 0x7f73f815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73ff77a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f80ce690 con 0x7f73f81539f0 2023-10-06T05:17:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73ff77a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f815a7a0 con 0x7f73f8153620 2023-10-06T05:17:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73ff77a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f80ce3f0 con 0x7f73f80ccfc0 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73f8153620 0x7f73f815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73f8153620 0x7f73f815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43932/0 (socket says 172.21.15.45:43932) 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 -- 172.21.15.45:0/922013200 learned_addr learned my addr 172.21.15.45:0/922013200 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 -- 172.21.15.45:0/922013200 >> v2:172.21.15.144:3300/0 conn(0x7f73f80ccfc0 msgr2=0x7f73f815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fdd17700 1 --2- 172.21.15.45:0/922013200 >> v2:172.21.15.144:3300/0 conn(0x7f73f80ccfc0 0x7f73f815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 --2- 172.21.15.45:0/922013200 >> v2:172.21.15.144:3300/0 conn(0x7f73f80ccfc0 0x7f73f815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 -- 172.21.15.45:0/922013200 >> v1:172.21.15.144:6789/0 conn(0x7f73f81539f0 legacy=0x7f73f80cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 -- 172.21.15.45:0/922013200 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f815d510 con 0x7f73f8153620 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73fd516700 1 --2- 172.21.15.45:0/922013200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f815a260 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ce600 tx=0x7f73f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0925b7c6f5d684 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73effff700 1 -- 172.21.15.45:0/922013200 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400b820 con 0x7f73f8153620 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73effff700 1 -- 172.21.15.45:0/922013200 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f400b980 con 0x7f73f8153620 2023-10-06T05:17:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.549+0000 7f73effff700 1 -- 172.21.15.45:0/922013200 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400bd20 con 0x7f73f8153620 2023-10-06T05:17:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 -- 172.21.15.45:0/922013200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 msgr2=0x7f73f815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 --2- 172.21.15.45:0/922013200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f815a260 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ce600 tx=0x7f73f4009250 comp rx=0 tx=0).stop 2023-10-06T05:17:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 -- 172.21.15.45:0/922013200 shutdown_connections 2023-10-06T05:17:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 --2- 172.21.15.45:0/922013200 >> v2:172.21.15.144:3300/0 conn(0x7f73f80ccfc0 0x7f73f815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 --2- 172.21.15.45:0/922013200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f815a260 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 -- 172.21.15.45:0/922013200 >> 172.21.15.45:0/922013200 conn(0x7f73f806ed50 msgr2=0x7f73f806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 -- 172.21.15.45:0/922013200 shutdown_connections 2023-10-06T05:17:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.550+0000 7f73ff77a700 1 -- 172.21.15.45:0/922013200 wait complete. 2023-10-06T05:17:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.551+0000 7f73ff77a700 1 Processor -- start 2023-10-06T05:17:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.551+0000 7f73ff77a700 1 -- start start 2023-10-06T05:17:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73ff77a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f80ccfc0 0x7f73f8151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73ff77a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73ff77a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73f80c5740 0x7f73f80b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73ff77a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80b3430 con 0x7f73f8153620 2023-10-06T05:17:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73ff77a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80c7d80 con 0x7f73f80ccfc0 2023-10-06T05:17:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73ff77a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80b3660 con 0x7f73f80c5740 2023-10-06T05:17:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73fcd15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73fcd15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f80b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43940/0 (socket says 172.21.15.45:43940) 2023-10-06T05:17:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.552+0000 7f73fcd15700 1 -- 172.21.15.45:0/764545823 learned_addr learned my addr 172.21.15.45:0/764545823 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fd516700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f80ccfc0 0x7f73f8151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fdd17700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73f80c5740 0x7f73f80b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fcd15700 1 -- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73f80c5740 msgr2=0x7f73f80b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fcd15700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73f80c5740 0x7f73f80b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fcd15700 1 -- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f80ccfc0 msgr2=0x7f73f8151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fcd15700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f80ccfc0 0x7f73f8151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fcd15700 1 -- 172.21.15.45:0/764545823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f40094f0 con 0x7f73f8153620 2023-10-06T05:17:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73fcd15700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f80b2ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b5e0 tx=0x7f73e80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73ed7fa700 1 -- 172.21.15.45:0/764545823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e800f510 con 0x7f73f8153620 2023-10-06T05:17:37.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73ed7fa700 1 -- 172.21.15.45:0/764545823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e800f670 con 0x7f73f8153620 2023-10-06T05:17:37.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f80b3940 con 0x7f73f8153620 2023-10-06T05:17:37.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73ed7fa700 1 -- 172.21.15.45:0/764545823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e800fa10 con 0x7f73f8153620 2023-10-06T05:17:37.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.553+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f80b3f00 con 0x7f73f8153620 2023-10-06T05:17:37.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.554+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f80c7f80 con 0x7f73f8153620 2023-10-06T05:17:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.555+0000 7f73ed7fa700 1 -- 172.21.15.45:0/764545823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73e801a400 con 0x7f73f8153620 2023-10-06T05:17:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.555+0000 7f73ed7fa700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f73dc066f30 0x7f73dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.555+0000 7f73ed7fa700 1 -- 172.21.15.45:0/764545823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f73e8084250 con 0x7f73f8153620 2023-10-06T05:17:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.556+0000 7f73fd516700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f73dc066f30 0x7f73dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.556+0000 7f73fd516700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f73dc066f30 0x7f73dc0693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ce600 tx=0x7f73f4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.560+0000 7f73ed7fa700 1 -- 172.21.15.45:0/764545823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73f80c7f80 con 0x7f73f8153620 2023-10-06T05:17:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.808+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73f804bb40 con 0x7f73f8153620 2023-10-06T05:17:37.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.809+0000 7f73ed7fa700 1 -- 172.21.15.45:0/764545823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f73e80512a0 con 0x7f73f8153620 2023-10-06T05:17:37.814 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:35.325485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2812290138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2812290138},{"type":"v1","addr":"172.21.15.45:6835","nonce":2812290138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:37.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.811+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f73dc066f30 msgr2=0x7f73dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.811+0000 7f73ff77a700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f73dc066f30 0x7f73dc0693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ce600 tx=0x7f73f4016000 comp rx=0 tx=0).stop 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.811+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 msgr2=0x7f73f80b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.811+0000 7f73ff77a700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f80b2ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b5e0 tx=0x7f73e80090f0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 shutdown_connections 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f73dc066f30 0x7f73dc0693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73f80c5740 0x7f73f80b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73f80ccfc0 0x7f73f8151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 --2- 172.21.15.45:0/764545823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73f8153620 0x7f73f80b2ef0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 >> 172.21.15.45:0/764545823 conn(0x7f73f806ed50 msgr2=0x7f73f80c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 shutdown_connections 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:37.812+0000 7f73ff77a700 1 -- 172.21.15.45:0/764545823 wait complete. 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 33 2023-10-06T05:17:38.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:17:31.744407+0000', 'modified': '2023-10-06T05:17:35.325485+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5682': {'gid': 5682, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/2812290138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 2812290138}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2812290138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:17:38.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:17:38.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:17:38.297 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:17:38.296+0000 7f48a1c34700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:17:38.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.372+0000 7f386702a700 1 Processor -- start 2023-10-06T05:17:38.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.372+0000 7f386702a700 1 -- start start 2023-10-06T05:17:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.373+0000 7f386702a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f386006f070 0x7f386006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.373+0000 7f386702a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f386006add0 0x7f38600c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f386702a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600ac5a0 con 0x7f38600c70f0 2023-10-06T05:17:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f386702a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386006b270 con 0x7f386006add0 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f386702a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3860159690 con 0x7f386006f070 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f386006add0 0x7f38600c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f38655c7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f38600c70f0 0x7f38600676b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52246/0 (socket says 172.21.15.45:52246) 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f38655c7700 1 -- 172.21.15.45:0/894960357 learned_addr learned my addr 172.21.15.45:0/894960357 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f386006add0 0x7f38600c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43942/0 (socket says 172.21.15.45:43942) 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f3864dc6700 1 --2- 172.21.15.45:0/894960357 >> v2:172.21.15.144:3300/0 conn(0x7f386006f070 0x7f386006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385ffff700 1 -- 172.21.15.45:0/894960357 >> v2:172.21.15.144:3300/0 conn(0x7f386006f070 msgr2=0x7f386006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385ffff700 1 --2- 172.21.15.45:0/894960357 >> v2:172.21.15.144:3300/0 conn(0x7f386006f070 0x7f386006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385ffff700 1 -- 172.21.15.45:0/894960357 >> v1:172.21.15.45:6789/0 conn(0x7f38600c70f0 legacy=0x7f38600676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385f7fe700 1 -- 172.21.15.45:0/894960357 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090273431 0 0) 0x7f38600ac5a0 con 0x7f38600c70f0 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385ffff700 1 -- 172.21.15.45:0/894960357 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600c78e0 con 0x7f386006add0 2023-10-06T05:17:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385ffff700 1 --2- 172.21.15.45:0/894960357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600c6bb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f386006b9e0 tx=0x7f38540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ace44f26d01d66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.374+0000 7f385f7fe700 1 -- 172.21.15.45:0/894960357 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854011630 con 0x7f386006add0 2023-10-06T05:17:38.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f385f7fe700 1 -- 172.21.15.45:0/894960357 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854011790 con 0x7f386006add0 2023-10-06T05:17:38.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f385f7fe700 1 -- 172.21.15.45:0/894960357 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854011b30 con 0x7f386006add0 2023-10-06T05:17:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 -- 172.21.15.45:0/894960357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 msgr2=0x7f38600c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 --2- 172.21.15.45:0/894960357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600c6bb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f386006b9e0 tx=0x7f38540091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 -- 172.21.15.45:0/894960357 shutdown_connections 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 --2- 172.21.15.45:0/894960357 >> v2:172.21.15.144:3300/0 conn(0x7f386006f070 0x7f386006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 --2- 172.21.15.45:0/894960357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600c6bb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 -- 172.21.15.45:0/894960357 >> 172.21.15.45:0/894960357 conn(0x7f38600cc5a0 msgr2=0x7f38600cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 -- 172.21.15.45:0/894960357 shutdown_connections 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.375+0000 7f386702a700 1 -- 172.21.15.45:0/894960357 wait complete. 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.376+0000 7f386702a700 1 Processor -- start 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.377+0000 7f386702a700 1 -- start start 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.377+0000 7f386702a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.377+0000 7f386702a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f386006f070 0x7f38600c0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.377+0000 7f386702a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38600c70f0 0x7f386014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.377+0000 7f386702a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600a98e0 con 0x7f386006add0 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f386702a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600a8f10 con 0x7f386006f070 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f386702a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38600c0e90 con 0x7f38600c70f0 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43948/0 (socket says 172.21.15.45:43948) 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 -- 172.21.15.45:0/2843896387 learned_addr learned my addr 172.21.15.45:0/2843896387 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f38655c7700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38600c70f0 0x7f386014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f385ffff700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f386006f070 0x7f38600c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 -- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38600c70f0 msgr2=0x7f386014d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38600c70f0 0x7f386014d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 -- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f386006f070 msgr2=0x7f38600c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f386006f070 0x7f38600c0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 -- 172.21.15.45:0/2843896387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854009420 con 0x7f386006add0 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f38655c7700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38600c70f0 0x7f386014d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f385ffff700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f386006f070 0x7f38600c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f3864dc6700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600a93a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f385000ecb0 tx=0x7f38500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f385cff9700 1 -- 172.21.15.45:0/2843896387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385000c9d0 con 0x7f386006add0 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f385cff9700 1 -- 172.21.15.45:0/2843896387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385000cb50 con 0x7f386006add0 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38600c1170 con 0x7f386006add0 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f385cff9700 1 -- 172.21.15.45:0/2843896387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850013470 con 0x7f386006add0 2023-10-06T05:17:38.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.378+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38600a8ba0 con 0x7f386006add0 2023-10-06T05:17:38.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.381+0000 7f385cff9700 1 -- 172.21.15.45:0/2843896387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3850013860 con 0x7f386006add0 2023-10-06T05:17:38.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.381+0000 7f385cff9700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f384c066f30 0x7f384c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.381+0000 7f385cff9700 1 -- 172.21.15.45:0/2843896387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3850084b60 con 0x7f386006add0 2023-10-06T05:17:38.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.381+0000 7f385ffff700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f384c066f30 0x7f384c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.382+0000 7f385ffff700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f384c066f30 0x7f384c0693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f38540093f0 tx=0x7f3854017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.383+0000 7f387033eb80 1 Processor -- start 2023-10-06T05:17:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.383+0000 7f387033eb80 1 -- start start 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c6b450 0x5571b7c6b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f387033eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c73f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c744d0 0x5571b7c74940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f387033eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7b3e310 con 0x5571b7c50f40 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7b7b130 con 0x5571b7c6b450 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571b7b78af0 con 0x5571b7c744d0 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c73f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c73f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43960/0 (socket says 172.21.15.45:43960) 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 -- 172.21.15.45:0/59727160 learned_addr learned my addr 172.21.15.45:0/59727160 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:38.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f3864dc6700 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c6b450 0x5571b7c6b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f38655c7700 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c744d0 0x5571b7c74940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 -- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c744d0 msgr2=0x5571b7c74940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c744d0 0x5571b7c74940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 -- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c6b450 msgr2=0x5571b7c6b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c6b450 0x5571b7c6b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.384+0000 7f385ffff700 1 -- 172.21.15.45:0/59727160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571b797a790 con 0x5571b7c50f40 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.385+0000 7f385ffff700 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c73f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f386006f7d0 tx=0x7f3854020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be41837285372bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.385+0000 7f3845ffb700 1 -- 172.21.15.45:0/59727160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854020070 con 0x5571b7c50f40 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.385+0000 7f3845ffb700 1 -- 172.21.15.45:0/59727160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854029b70 con 0x5571b7c50f40 2023-10-06T05:17:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.385+0000 7f3845ffb700 1 -- 172.21.15.45:0/59727160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854029ce0 con 0x5571b7c50f40 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.385+0000 7f387033eb80 1 -- 172.21.15.45:0/59727160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 msgr2=0x5571b7c73f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.385+0000 7f387033eb80 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c73f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f386006f7d0 tx=0x7f3854020000 comp rx=0 tx=0).stop 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 -- 172.21.15.45:0/59727160 shutdown_connections 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c744d0 0x5571b7c74940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c6b450 0x5571b7c6b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 --2- 172.21.15.45:0/59727160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c73f90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 -- 172.21.15.45:0/59727160 >> 172.21.15.45:0/59727160 conn(0x5571b7c6e880 msgr2=0x5571b7c6ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 -- 172.21.15.45:0/59727160 shutdown_connections 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 -- 172.21.15.45:0/59727160 wait complete. 2023-10-06T05:17:38.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.386+0000 7f387033eb80 1 -- start start 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.387+0000 7f387033eb80 1 Processor -- start 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f387033eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c46400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 0x5571b7c25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f387033eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7b748e0 con 0x5571b7c50f40 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7b20980 con 0x5571b7c744d0 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571b7b20620 con 0x5571b7c6b450 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f387033eb80 10 client.0 _ll_register_callbacks cb 0x5571b7983a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43974/0 (socket says 172.21.15.45:43974) 2023-10-06T05:17:38.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 -- 172.21.15.45:0/376124037 learned_addr learned my addr 172.21.15.45:0/376124037 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 -- 172.21.15.45:0/376124037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 msgr2=0x5571b7c46400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c46400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 -- 172.21.15.45:0/376124037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 msgr2=0x5571b7c25e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 0x5571b7c25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 -- 172.21.15.45:0/376124037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38600c78e0 con 0x5571b7c50f40 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.388+0000 7f3864dc6700 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6aa20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3850051cf0 tx=0x7f3850082d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ae3639d61dbdcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f383effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f383effd700 1 -- 172.21.15.45:0/376124037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850058a40 con 0x5571b7c50f40 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f383effd700 1 -- 172.21.15.45:0/376124037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f385005c790 con 0x5571b7c50f40 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571b7c18d60 con 0x5571b7c50f40 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f383effd700 1 -- 172.21.15.45:0/376124037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3850054ef0 con 0x5571b7c50f40 2023-10-06T05:17:38.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 10 client.5829 fetch_fsmap learned FSMap version 33 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5571b7a670a0 con 0x5571b7c50f40 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f383effd700 1 -- 172.21.15.45:0/376124037 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5571b7c18d60 con 0x5571b7c50f40 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f383effd700 1 -- 172.21.15.45:0/376124037 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3850071250 con 0x5571b7c50f40 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 10 client.5829 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 10 client.5829 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 20 client.5829 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7c4b750 0x5571b7c4bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 4 client.5829 mds_command: new command op to 5682 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 --> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571b7bf5b30 con 0x5571b7c4b750 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.389+0000 7f385ffff700 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7c4b750 0x5571b7c4bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.390+0000 7f383effd700 10 client.5829 ms_handle_connect on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:38.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.390+0000 7f385ffff700 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7c4b750 0x5571b7c4bb20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d46735ac366b20d server_cookie=36a7f5d03f577b22 in_seq=0 out_seq=0 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.390+0000 7f383effd700 1 -- 172.21.15.45:0/376124037 <== mds.0 v2:172.21.15.45:6833/2812290138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5571b7bf5b30 con 0x5571b7c4b750 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.390+0000 7f383effd700 10 client.5829 handle_command_reply: tid=0 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.390+0000 7f387033eb80 1 client.5829 shutdown 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.390+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 msgr2=0x5571b7c6aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.390+0000 7f387033eb80 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6aa20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3850051cf0 tx=0x7f3850082d40 comp rx=0 tx=0).stop 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 shutdown_connections 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7c4b750 0x5571b7c4bb20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 0x5571b7c25e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6aa20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f383effd700 0 client.5829 ms_handle_reset on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 --2- 172.21.15.45:0/376124037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c46400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 >> 172.21.15.45:0/376124037 conn(0x5571b7c6e880 msgr2=0x5571b7c1fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 shutdown_connections 2023-10-06T05:17:38.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 1 -- 172.21.15.45:0/376124037 wait complete. 2023-10-06T05:17:38.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.391+0000 7f387033eb80 20 client.5829 trim_cache size 0 max 16384 2023-10-06T05:17:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 Processor -- start 2023-10-06T05:17:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 -- start start 2023-10-06T05:17:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c50f40 0x5571b7c45de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c46320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c744d0 0x5571b7c83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7a0c1e0 con 0x5571b7c744d0 2023-10-06T05:17:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7c6abc0 con 0x5571b7c50f40 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.409+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571b7c21470 con 0x5571b7c6b450 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c744d0 0x5571b7c83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c744d0 0x5571b7c83d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43976/0 (socket says 172.21.15.45:43976) 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 -- 172.21.15.45:0/2973523946 learned_addr learned my addr 172.21.15.45:0/2973523946 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f3864dc6700 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c50f40 0x5571b7c45de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f385ffff700 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c46320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 -- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 msgr2=0x5571b7c46320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c46320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 -- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c50f40 msgr2=0x5571b7c45de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c50f40 0x5571b7c45de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 -- 172.21.15.45:0/2973523946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571b7b433c0 con 0x5571b7c744d0 2023-10-06T05:17:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38655c7700 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c744d0 0x5571b7c83d30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3858001fd0 tx=0x7f3858009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecdd1a659c88788 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38467fc700 1 -- 172.21.15.45:0/2973523946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858011c30 con 0x5571b7c744d0 2023-10-06T05:17:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38467fc700 1 -- 172.21.15.45:0/2973523946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3858011d90 con 0x5571b7c744d0 2023-10-06T05:17:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.410+0000 7f38467fc700 1 -- 172.21.15.45:0/2973523946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38580156a0 con 0x5571b7c744d0 2023-10-06T05:17:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 -- 172.21.15.45:0/2973523946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c744d0 msgr2=0x5571b7c83d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c744d0 0x5571b7c83d30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3858001fd0 tx=0x7f3858009040 comp rx=0 tx=0).stop 2023-10-06T05:17:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 -- 172.21.15.45:0/2973523946 shutdown_connections 2023-10-06T05:17:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c744d0 0x5571b7c83d30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c50f40 0x5571b7c45de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 --2- 172.21.15.45:0/2973523946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c46320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 -- 172.21.15.45:0/2973523946 >> 172.21.15.45:0/2973523946 conn(0x5571b7c6e880 msgr2=0x5571b7c20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 -- 172.21.15.45:0/2973523946 shutdown_connections 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.411+0000 7f387033eb80 1 -- 172.21.15.45:0/2973523946 wait complete. 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.412+0000 7f387033eb80 1 -- start start 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.412+0000 7f387033eb80 1 Processor -- start 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f387033eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c84660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f387033eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 0x5571b7cd3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f387033eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7c6a440 con 0x5571b7c50f40 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b7c243c0 con 0x5571b7c744d0 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f387033eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571b7cd4ab0 con 0x5571b7c6b450 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f387033eb80 10 client.0 _ll_register_callbacks cb 0x5571b7983a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f3864dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f3864dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43978/0 (socket says 172.21.15.45:43978) 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.413+0000 7f3864dc6700 1 -- 172.21.15.45:0/2158565990 learned_addr learned my addr 172.21.15.45:0/2158565990 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f385ffff700 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c84660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f38655c7700 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 0x5571b7cd3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f3864dc6700 1 -- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 msgr2=0x5571b7c84660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f3864dc6700 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c84660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f3864dc6700 1 -- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 msgr2=0x5571b7cd3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f3864dc6700 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 0x5571b7cd3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f3864dc6700 1 -- 172.21.15.45:0/2158565990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385800f040 con 0x5571b7c50f40 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f3864dc6700 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6da40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3850009870 tx=0x7f3850069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7107b46f9d9406c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f383effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:38.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f383effd700 1 -- 172.21.15.45:0/2158565990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385005c790 con 0x5571b7c50f40 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f383effd700 1 -- 172.21.15.45:0/2158565990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3850058a40 con 0x5571b7c50f40 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3854029b70 con 0x5571b7c50f40 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f383effd700 1 -- 172.21.15.45:0/2158565990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f385005c300 con 0x5571b7c50f40 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f387033eb80 10 client.5835 fetch_fsmap learned FSMap version 33 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5571b79db1d0 con 0x5571b7c50f40 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f383effd700 1 -- 172.21.15.45:0/2158565990 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3850064080 con 0x5571b7c50f40 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f383effd700 1 -- 172.21.15.45:0/2158565990 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3850013bb0 con 0x5571b7c50f40 2023-10-06T05:17:38.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f387033eb80 10 client.5835 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f387033eb80 10 client.5835 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.414+0000 7f387033eb80 20 client.5835 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f387033eb80 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7cd4180 0x5571b7ccfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f387033eb80 4 client.5835 mds_command: new command op to 5682 tid=0[{"prefix": "status"}] 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 --> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5571b7b178d0 con 0x5571b7cd4180 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f385ffff700 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7cd4180 0x5571b7ccfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f383effd700 10 client.5835 ms_handle_connect on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f385ffff700 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7cd4180 0x5571b7ccfa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0e4ebec26f432a server_cookie=9f9706ddea2e056a in_seq=0 out_seq=0 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f383effd700 1 -- 172.21.15.45:0/2158565990 <== mds.0 v2:172.21.15.45:6833/2812290138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5571b7b178d0 con 0x5571b7cd4180 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f383effd700 10 client.5835 handle_command_reply: tid=0 2023-10-06T05:17:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f387033eb80 1 client.5835 shutdown 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 msgr2=0x5571b7c6da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.415+0000 7f387033eb80 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6da40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3850009870 tx=0x7f3850069000 comp rx=0 tx=0).stop 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 shutdown_connections 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x5571b7cd4180 0x5571b7ccfa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571b7c744d0 0x5571b7cd3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571b7c50f40 0x5571b7c6da40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f383effd700 0 client.5835 ms_handle_reset on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 --2- 172.21.15.45:0/2158565990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5571b7c6b450 0x5571b7c84660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 >> 172.21.15.45:0/2158565990 conn(0x5571b7c6e880 msgr2=0x5571b7c20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 shutdown_connections 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 1 -- 172.21.15.45:0/2158565990 wait complete. 2023-10-06T05:17:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f387033eb80 20 client.5835 trim_cache size 0 max 16384 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5682, 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.1094512710000002, 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 33, 2023-10-06T05:17:38.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 52, 2023-10-06T05:17:38.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 52, 2023-10-06T05:17:38.425 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 12.087272456999999 2023-10-06T05:17:38.425 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:17:38.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f384c066f30 msgr2=0x7f384c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f384c066f30 0x7f384c0693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f38540093f0 tx=0x7f3854017000 comp rx=0 tx=0).stop 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 msgr2=0x7f38600a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600a93a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f385000ecb0 tx=0x7f38500091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 shutdown_connections 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f384c066f30 0x7f384c0693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38600c70f0 0x7f386014d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386006add0 0x7f38600a93a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 --2- 172.21.15.45:0/2843896387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f386006f070 0x7f38600c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.416+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 >> 172.21.15.45:0/2843896387 conn(0x7f38600cc5a0 msgr2=0x7f38600acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.417+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 shutdown_connections 2023-10-06T05:17:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.417+0000 7f386702a700 1 -- 172.21.15.45:0/2843896387 wait complete. 2023-10-06T05:17:38.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:17:38.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.562+0000 7fef885db700 1 Processor -- start 2023-10-06T05:17:38.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.562+0000 7fef885db700 1 -- start start 2023-10-06T05:17:38.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef885db700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef8006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef885db700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef8006b5c0 0x7fef800ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef885db700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef800c7d00 con 0x7fef80159200 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef885db700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800c7780 con 0x7fef8006b5c0 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef885db700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8006bb40 con 0x7fef801595d0 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86b78700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef8006b5c0 0x7fef800ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86377700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef80159200 0x7fef800c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52248/0 (socket says 172.21.15.45:52248) 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86b78700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef8006b5c0 0x7fef800ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43984/0 (socket says 172.21.15.45:43984) 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86377700 1 -- 172.21.15.45:0/3885650003 learned_addr learned my addr 172.21.15.45:0/3885650003 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef85b76700 1 --2- 172.21.15.45:0/3885650003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef8006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86b78700 1 -- 172.21.15.45:0/3885650003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 msgr2=0x7fef8006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86b78700 1 --2- 172.21.15.45:0/3885650003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef8006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86b78700 1 -- 172.21.15.45:0/3885650003 >> v1:172.21.15.45:6789/0 conn(0x7fef80159200 legacy=0x7fef800c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86b78700 1 -- 172.21.15.45:0/3885650003 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8006be20 con 0x7fef8006b5c0 2023-10-06T05:17:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef85375700 1 -- 172.21.15.45:0/3885650003 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169647908 0 0) 0x7fef800c7d00 con 0x7fef80159200 2023-10-06T05:17:38.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.563+0000 7fef86b78700 1 --2- 172.21.15.45:0/3885650003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8006b5c0 0x7fef800ccdc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fef78009e60 tx=0x7fef78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f846fc40979dbcf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef85375700 1 -- 172.21.15.45:0/3885650003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800e070 con 0x7fef8006b5c0 2023-10-06T05:17:38.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef85375700 1 -- 172.21.15.45:0/3885650003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7800bb60 con 0x7fef8006b5c0 2023-10-06T05:17:38.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef85375700 1 -- 172.21.15.45:0/3885650003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78012430 con 0x7fef8006b5c0 2023-10-06T05:17:38.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 -- 172.21.15.45:0/3885650003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8006b5c0 msgr2=0x7fef800ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 --2- 172.21.15.45:0/3885650003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8006b5c0 0x7fef800ccdc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fef78009e60 tx=0x7fef78009780 comp rx=0 tx=0).stop 2023-10-06T05:17:38.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 -- 172.21.15.45:0/3885650003 shutdown_connections 2023-10-06T05:17:38.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 --2- 172.21.15.45:0/3885650003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8006b5c0 0x7fef800ccdc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 --2- 172.21.15.45:0/3885650003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef8006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 -- 172.21.15.45:0/3885650003 >> 172.21.15.45:0/3885650003 conn(0x7fef8006f090 msgr2=0x7fef8006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 -- 172.21.15.45:0/3885650003 shutdown_connections 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.564+0000 7fef885db700 1 -- 172.21.15.45:0/3885650003 wait complete. 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.565+0000 7fef885db700 1 Processor -- start 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef885db700 1 -- start start 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef885db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8006b5c0 0x7fef800cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef885db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef80159200 0x7fef800ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef885db700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef800cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef885db700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800cafb0 con 0x7fef80159200 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef885db700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800ccc30 con 0x7fef8006b5c0 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef885db700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800ccf10 con 0x7fef801595d0 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef85b76700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef80159200 0x7fef800ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef85b76700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef80159200 0x7fef800ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43998/0 (socket says 172.21.15.45:43998) 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.566+0000 7fef85b76700 1 -- 172.21.15.45:0/2453045408 learned_addr learned my addr 172.21.15.45:0/2453045408 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef86b78700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef800cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef86377700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8006b5c0 0x7fef800cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef85b76700 1 -- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 msgr2=0x7fef800cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef85b76700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef800cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef85b76700 1 -- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8006b5c0 msgr2=0x7fef800cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef85b76700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8006b5c0 0x7fef800cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef85b76700 1 -- 172.21.15.45:0/2453045408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78009a20 con 0x7fef80159200 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef85b76700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef80159200 0x7fef800ada60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fef7000e930 tx=0x7fef700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef767fc700 1 -- 172.21.15.45:0/2453045408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7000eed0 con 0x7fef80159200 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef767fc700 1 -- 172.21.15.45:0/2453045408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7000c850 con 0x7fef80159200 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef767fc700 1 -- 172.21.15.45:0/2453045408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7000cbf0 con 0x7fef80159200 2023-10-06T05:17:38.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef800ca960 con 0x7fef80159200 2023-10-06T05:17:38.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.567+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef800cb980 con 0x7fef80159200 2023-10-06T05:17:38.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.568+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8006e2f0 con 0x7fef80159200 2023-10-06T05:17:38.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.569+0000 7fef767fc700 1 -- 172.21.15.45:0/2453045408 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef70013470 con 0x7fef80159200 2023-10-06T05:17:38.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.569+0000 7fef767fc700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef64066f30 0x7fef640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.569+0000 7fef767fc700 1 -- 172.21.15.45:0/2453045408 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fef700849c0 con 0x7fef80159200 2023-10-06T05:17:38.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.569+0000 7fef86377700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef64066f30 0x7fef640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.570+0000 7fef86377700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef64066f30 0x7fef640693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0011e0 tx=0x7fef7c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.574+0000 7fef767fc700 1 -- 172.21.15.45:0/2453045408 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef70051990 con 0x7fef80159200 2023-10-06T05:17:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.819+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fef800c7990 con 0x7fef80159200 2023-10-06T05:17:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.819+0000 7fef767fc700 1 -- 172.21.15.45:0/2453045408 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7fef70054fb0 con 0x7fef80159200 2023-10-06T05:17:38.821 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:17:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef64066f30 msgr2=0x7fef640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef64066f30 0x7fef640693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0011e0 tx=0x7fef7c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef80159200 msgr2=0x7fef800ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef80159200 0x7fef800ada60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fef7000e930 tx=0x7fef700091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 shutdown_connections 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef64066f30 0x7fef640693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef801595d0 0x7fef800cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8006b5c0 0x7fef800cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 --2- 172.21.15.45:0/2453045408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef80159200 0x7fef800ada60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 >> 172.21.15.45:0/2453045408 conn(0x7fef8006f090 msgr2=0x7fef8015ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.821+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 shutdown_connections 2023-10-06T05:17:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:38.822+0000 7fef885db700 1 -- 172.21.15.45:0/2453045408 wait complete. 2023-10-06T05:17:38.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:17:38.837 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:38.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:38.838 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:17:38.858 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:38.858 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:38.858 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:38.858 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:17:38.859 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42282sec preferred_lft 42282sec 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:17:38.860 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:38.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:38.862 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:17:38.930 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:17:38.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:38.931 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:17:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:17:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:17:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:17:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:17:38.997 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:17:38.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:38.998 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:17:38.998 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:17:38.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:17:38.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:17:38.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:17:38.998 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:17:39.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:39.121 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:17:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:17:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:17:39.143 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:39.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:39.143 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:17:39.219 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:17:39.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:39.219 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:39.283 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:39.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:39.283 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:39.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:17:39.423 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:17:39.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:17:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:17:39.499 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:17:39.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:39.500 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:17:39.564 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:17:39.564 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:17:39.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:17:39.642 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:17:39.643 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:17:39.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:17:39.706+0000 7fa8ac1df540 -1 init, newargv = 0x55a8892047d0 newargc=15 2023-10-06T05:17:39.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[141499]: starting ceph client 2023-10-06T05:17:39.714 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[141499]: starting fuse 2023-10-06T05:17:39.714 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:17:39.715 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:17:39.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:39.716 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:17:39.777 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:17:39.778 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:17:39.778 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:17:39.779 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:17:39.780 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:17:39.780 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:17:39.780 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:17:39.780 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:17:39.780 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:17:39.780 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:17:39.780 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:17:39.917 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.141499.asok 2023-10-06T05:17:40.098 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.141499.asok 2023-10-06T05:17:40.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:40.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.141499.asok status 2023-10-06T05:17:40.274 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:17:40.274 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:17:40.274 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:17:40.274 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:17:40.275 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:17:40.275 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:17:40.275 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:17:40.275 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "141499", 2023-10-06T05:17:40.275 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:17:40.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:40.275 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:17:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:17:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5847, 2023-10-06T05:17:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:17:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:17:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:17:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 5847 2023-10-06T05:17:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:17:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:17:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:17:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2906220994 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2906220994 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.5847 192.168.0.1:0/2906220994", 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2906220994", 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 33, 2023-10-06T05:17:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 52, 2023-10-06T05:17:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:17:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:17:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:17:40.279 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:17:40.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:40.284 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:17:40.284 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:40.305 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:17:40.305 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:40.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:40.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:40.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:40.380 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p a/b') 2023-10-06T05:17:40.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2023-10-06T05:17:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.622+0000 7f7b20d7a700 1 Processor -- start 2023-10-06T05:17:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.622+0000 7f7b20d7a700 1 -- start start 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b20d7a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7b1c0b0610 0x7f7b1c0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b20d7a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7b1c0c3db0 0x7f7b1c0697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b20d7a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0ce6a0 con 0x7f7b1c069d00 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b20d7a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c0690f0 con 0x7f7b1c0b0610 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b20d7a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c0ceeb0 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b1a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7b1c0b0610 0x7f7b1c0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b1ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7b1c069d00 0x7f7b1c0c2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52254/0 (socket says 172.21.15.45:52254) 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.623+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/2269404591 learned_addr learned my addr 172.21.15.45:0/2269404591 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/2269404591 >> v2:172.21.15.144:3300/0 conn(0x7f7b1c0c3db0 0x7f7b1c0697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1a59c700 1 -- 172.21.15.45:0/2269404591 >> v2:172.21.15.144:3300/0 conn(0x7f7b1c0c3db0 msgr2=0x7f7b1c0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/2269404591 >> v2:172.21.15.144:3300/0 conn(0x7f7b1c0c3db0 0x7f7b1c0697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1a59c700 1 -- 172.21.15.45:0/2269404591 >> v1:172.21.15.45:6789/0 conn(0x7f7b1c069d00 legacy=0x7f7b1c0c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1a59c700 1 -- 172.21.15.45:0/2269404591 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c159880 con 0x7f7b1c0b0610 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1959a700 1 -- 172.21.15.45:0/2269404591 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213707167 0 0) 0x7f7b1c0ce6a0 con 0x7f7b1c069d00 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/2269404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0b0610 0x7f7b1c0c85d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c064110 tx=0x7f7b0c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a3263287b74828 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1959a700 1 -- 172.21.15.45:0/2269404591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c009e70 con 0x7f7b1c0b0610 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1959a700 1 -- 172.21.15.45:0/2269404591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c005c10 con 0x7f7b1c0b0610 2023-10-06T05:17:40.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.624+0000 7f7b1959a700 1 -- 172.21.15.45:0/2269404591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c00b610 con 0x7f7b1c0b0610 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 -- 172.21.15.45:0/2269404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0b0610 msgr2=0x7f7b1c0c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/2269404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0b0610 0x7f7b1c0c85d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c064110 tx=0x7f7b0c009120 comp rx=0 tx=0).stop 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 -- 172.21.15.45:0/2269404591 shutdown_connections 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/2269404591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0b0610 0x7f7b1c0c85d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/2269404591 >> v2:172.21.15.144:3300/0 conn(0x7f7b1c0c3db0 0x7f7b1c0697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 -- 172.21.15.45:0/2269404591 >> 172.21.15.45:0/2269404591 conn(0x7f7b1c067e00 msgr2=0x7f7b1c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 -- 172.21.15.45:0/2269404591 shutdown_connections 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.625+0000 7f7b20d7a700 1 -- 172.21.15.45:0/2269404591 wait complete. 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.626+0000 7f7b20d7a700 1 Processor -- start 2023-10-06T05:17:40.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.626+0000 7f7b20d7a700 1 -- start start 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b20d7a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b1c069d00 0x7f7b1c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b20d7a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b1c0b0610 0x7f7b1c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b20d7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0c3db0 0x7f7b1c16ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b20d7a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c16dc20 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b20d7a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c15aa90 con 0x7f7b1c069d00 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b20d7a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c15c430 con 0x7f7b1c0b0610 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0c3db0 0x7f7b1c16ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0c3db0 0x7f7b1c16ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44010/0 (socket says 172.21.15.45:44010) 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/3425167899 learned_addr learned my addr 172.21.15.45:0/3425167899 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b1c069d00 0x7f7b1c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b1c0b0610 0x7f7b1c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b1c0b0610 msgr2=0x7f7b1c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b1c0b0610 0x7f7b1c0c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b1c069d00 msgr2=0x7f7b1c15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b1c069d00 0x7f7b1c15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/3425167899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0093c0 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b1c0b0610 0x7f7b1c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0c3db0 0x7f7b1c16ce30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400cc90 tx=0x7f7b14001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b127fc700 1 -- 172.21.15.45:0/3425167899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400cea0 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b127fc700 1 -- 172.21.15.45:0/3425167899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1400ab30 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.627+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c15c710 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.628+0000 7f7b127fc700 1 -- 172.21.15.45:0/3425167899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b140070f0 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.628+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c0c8a30 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.629+0000 7f7b127fc700 1 -- 172.21.15.45:0/3425167899 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b140074a0 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.630+0000 7f7b127fc700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7afc066f30 0x7f7afc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.630+0000 7f7b127fc700 1 -- 172.21.15.45:0/3425167899 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7b14084d30 con 0x7f7b1c0c3db0 2023-10-06T05:17:40.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.630+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7afc066f30 0x7f7afc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.630+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7afc066f30 0x7f7afc0693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009160 tx=0x7f7b0c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.632+0000 7f7b2a08eb80 1 Processor -- start 2023-10-06T05:17:40.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.632+0000 7f7b2a08eb80 1 -- start start 2023-10-06T05:17:40.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb2934a0 0x55ebcb293870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb29bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb29c520 0x55ebcb29c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb166360 con 0x55ebcb29c520 2023-10-06T05:17:40.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb1a3180 con 0x55ebcb2934a0 2023-10-06T05:17:40.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb1a0b40 con 0x55ebcb278f90 2023-10-06T05:17:40.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb29c520 0x55ebcb29c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb29c520 0x55ebcb29c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44018/0 (socket says 172.21.15.45:44018) 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/4185805290 learned_addr learned my addr 172.21.15.45:0/4185805290 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb2934a0 0x55ebcb293870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb29bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 msgr2=0x55ebcb29bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb29bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb2934a0 msgr2=0x55ebcb293870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb2934a0 0x55ebcb293870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 -- 172.21.15.45:0/4185805290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebcafa27e0 con 0x55ebcb29c520 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb29c520 0x55ebcb29c990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b14051e20 tx=0x7f7b14082e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f461a19299ea78a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7afb7fe700 1 -- 172.21.15.45:0/4185805290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b14055920 con 0x55ebcb29c520 2023-10-06T05:17:40.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7afb7fe700 1 -- 172.21.15.45:0/4185805290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b14058b70 con 0x55ebcb29c520 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.633+0000 7f7afb7fe700 1 -- 172.21.15.45:0/4185805290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1405c8c0 con 0x55ebcb29c520 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4185805290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb29c520 msgr2=0x55ebcb29c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb29c520 0x55ebcb29c990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7b14051e20 tx=0x7f7b14082e70 comp rx=0 tx=0).stop 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4185805290 shutdown_connections 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb29c520 0x55ebcb29c990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb2934a0 0x55ebcb293870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4185805290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb29bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4185805290 >> 172.21.15.45:0/4185805290 conn(0x55ebcb2968d0 msgr2=0x55ebcb296ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4185805290 shutdown_connections 2023-10-06T05:17:40.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.634+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4185805290 wait complete. 2023-10-06T05:17:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.635+0000 7f7b2a08eb80 1 -- start start 2023-10-06T05:17:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.635+0000 7f7b2a08eb80 1 Processor -- start 2023-10-06T05:17:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb26e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb271630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb271b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb19c930 con 0x55ebcb278f90 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb1489d0 con 0x55ebcb29c520 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb148670 con 0x55ebcb2934a0 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b2a08eb80 10 client.0 _ll_register_callbacks cb 0x55ebcafaba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb26e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb26e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44022/0 (socket says 172.21.15.45:44022) 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 -- 172.21.15.45:0/3712699938 learned_addr learned my addr 172.21.15.45:0/3712699938 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb271630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb271b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 -- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 msgr2=0x55ebcb271630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb271630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 -- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 msgr2=0x55ebcb271b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb271b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.636+0000 7f7b1a59c700 1 -- 172.21.15.45:0/3712699938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c0c8a30 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb26e860 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c012000 tx=0x7f7b0c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04cb92138ed2f29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7af8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7af8ff9700 1 -- 172.21.15.45:0/3712699938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c031070 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7af8ff9700 1 -- 172.21.15.45:0/3712699938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7b0c006670 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebcb240db0 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7af8ff9700 1 -- 172.21.15.45:0/3712699938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b0c022070 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 10 client.5862 fetch_fsmap learned FSMap version 33 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ebcb08f0f0 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7af8ff9700 1 -- 172.21.15.45:0/3712699938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ebcb240db0 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7af8ff9700 1 -- 172.21.15.45:0/3712699938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7b0c028430 con 0x55ebcb278f90 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 10 client.5862 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T05:17:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 10 client.5862 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 20 client.5862 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb2a2ab0 0x55ebcb273690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 4 client.5862 mds_command: new command op to 5682 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.637+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 --> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebcb21db80 con 0x55ebcb2a2ab0 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.638+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb2a2ab0 0x55ebcb273690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.638+0000 7f7af8ff9700 10 client.5862 ms_handle_connect on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.638+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb2a2ab0 0x55ebcb273690 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1e8e01af8382ce server_cookie=f40220371f1d4153 in_seq=0 out_seq=0 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.638+0000 7f7af8ff9700 1 -- 172.21.15.45:0/3712699938 <== mds.0 v2:172.21.15.45:6833/2812290138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ebcb21db80 con 0x55ebcb2a2ab0 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.638+0000 7f7af8ff9700 10 client.5862 handle_command_reply: tid=0 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 client.5862 shutdown 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 msgr2=0x55ebcb26e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb26e860 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c012000 tx=0x7f7b0c022000 comp rx=0 tx=0).stop 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 shutdown_connections 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb2a2ab0 0x55ebcb273690 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb271b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7af8ff9700 0 client.5862 ms_handle_reset on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb26e860 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/3712699938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb271630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 >> 172.21.15.45:0/3712699938 conn(0x55ebcb2968d0 msgr2=0x55ebcb248cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 shutdown_connections 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/3712699938 wait complete. 2023-10-06T05:17:40.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.639+0000 7f7b2a08eb80 20 client.5862 trim_cache size 0 max 16384 2023-10-06T05:17:40.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.653+0000 7f7b2a08eb80 1 Processor -- start 2023-10-06T05:17:40.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.653+0000 7f7b2a08eb80 1 -- start start 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb24a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb26d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb24d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb034230 con 0x55ebcb278f90 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb24ab50 con 0x55ebcb29c520 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb275720 con 0x55ebcb2934a0 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b1a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb24a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b1a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb24a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44030/0 (socket says 172.21.15.45:44030) 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b1a59c700 1 -- 172.21.15.45:0/4113258177 learned_addr learned my addr 172.21.15.45:0/4113258177 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb24d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.654+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb26d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b1a59c700 1 -- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 msgr2=0x55ebcb26d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb26d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b1a59c700 1 -- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 msgr2=0x55ebcb24d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb24d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b1a59c700 1 -- 172.21.15.45:0/4113258177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebcb16b410 con 0x55ebcb278f90 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb24d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb26d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb24a020 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0091f0 tx=0x7f7b0c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132015e6f27bdf0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7afbfff700 1 -- 172.21.15.45:0/4113258177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c022040 con 0x55ebcb278f90 2023-10-06T05:17:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7afbfff700 1 -- 172.21.15.45:0/4113258177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c028940 con 0x55ebcb278f90 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.655+0000 7f7afbfff700 1 -- 172.21.15.45:0/4113258177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c006670 con 0x55ebcb278f90 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4113258177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 msgr2=0x55ebcb24a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb24a020 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0091f0 tx=0x7f7b0c031000 comp rx=0 tx=0).stop 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4113258177 shutdown_connections 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb24d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb278f90 0x55ebcb24a020 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/4113258177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb2934a0 0x55ebcb26d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4113258177 >> 172.21.15.45:0/4113258177 conn(0x55ebcb2968d0 msgr2=0x55ebcb26e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4113258177 shutdown_connections 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/4113258177 wait complete. 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 -- start start 2023-10-06T05:17:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.656+0000 7f7b2a08eb80 1 Processor -- start 2023-10-06T05:17:40.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb2716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb2934a0 0x55ebcb271c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b2a08eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb2fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb248d70 con 0x55ebcb2934a0 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb2f8a90 con 0x55ebcb29c520 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b2a08eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ebcb249db0 con 0x55ebcb278f90 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b2a08eb80 10 client.0 _ll_register_callbacks cb 0x55ebcafaba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.657+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb2934a0 0x55ebcb271c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb2934a0 0x55ebcb271c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44044/0 (socket says 172.21.15.45:44044) 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 -- 172.21.15.45:0/2321470030 learned_addr learned my addr 172.21.15.45:0/2321470030 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb2716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb2fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 -- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 msgr2=0x55ebcb2716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb2716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 -- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 msgr2=0x55ebcb2fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb2fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 -- 172.21.15.45:0/2321470030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebcb08f0f0 con 0x55ebcb2934a0 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b1ad9d700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb2fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b19d9b700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb2934a0 0x55ebcb271c00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0cc550 tx=0x7f7b0401c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bd48c41af61b3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7af8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7af8ff9700 1 -- 172.21.15.45:0/2321470030 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04016d80 con 0x55ebcb2934a0 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7af8ff9700 1 -- 172.21.15.45:0/2321470030 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7b0401f440 con 0x55ebcb2934a0 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b14058b70 con 0x55ebcb2934a0 2023-10-06T05:17:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7af8ff9700 1 -- 172.21.15.45:0/2321470030 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b0401f810 con 0x55ebcb2934a0 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b2a08eb80 10 client.5868 fetch_fsmap learned FSMap version 33 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.658+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ebcb003220 con 0x55ebcb2934a0 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7af8ff9700 1 -- 172.21.15.45:0/2321470030 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b14058b70 con 0x55ebcb2934a0 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7af8ff9700 1 -- 172.21.15.45:0/2321470030 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7b04026060 con 0x55ebcb2934a0 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b2a08eb80 10 client.5868 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b2a08eb80 10 client.5868 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b2a08eb80 20 client.5868 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb242f90 0x55ebcb2fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b2a08eb80 4 client.5868 mds_command: new command op to 5682 tid=0[{"prefix": "flush journal"}] 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 --> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ebcb13f920 con 0x55ebcb242f90 2023-10-06T05:17:40.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb242f90 0x55ebcb2fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7b1a59c700 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb242f90 0x55ebcb2fc2f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d554ad31ff44891 server_cookie=3715360046c49972 in_seq=0 out_seq=0 2023-10-06T05:17:40.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.659+0000 7f7af8ff9700 10 client.5868 ms_handle_connect on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7af8ff9700 1 -- 172.21.15.45:0/2321470030 <== mds.0 v2:172.21.15.45:6833/2812290138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ebcb13f920 con 0x55ebcb242f90 2023-10-06T05:17:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7af8ff9700 10 client.5868 handle_command_reply: tid=0 2023-10-06T05:17:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 client.5868 shutdown 2023-10-06T05:17:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb2934a0 msgr2=0x55ebcb271c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb2934a0 0x55ebcb271c00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0cc550 tx=0x7f7b0401c250 comp rx=0 tx=0).stop 2023-10-06T05:17:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 shutdown_connections 2023-10-06T05:17:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:6833/2812290138,v1:172.21.15.45:6835/2812290138] conn(0x55ebcb242f90 0x55ebcb2fc2f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebcb29c520 0x55ebcb2fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ebcb278f90 0x55ebcb2716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 --2- 172.21.15.45:0/2321470030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebcb2934a0 0x55ebcb271c00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 >> 172.21.15.45:0/2321470030 conn(0x55ebcb2968d0 msgr2=0x55ebcb24a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.670+0000 7f7af8ff9700 0 client.5868 ms_handle_reset on v2:172.21.15.45:6833/2812290138 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 shutdown_connections 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b2a08eb80 1 -- 172.21.15.45:0/2321470030 wait complete. 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b2a08eb80 20 client.5868 trim_cache size 0 max 16384 2023-10-06T05:17:40.675 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:17:40.676 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2023-10-06T05:17:40.676 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2023-10-06T05:17:40.676 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:17:40.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7afc066f30 msgr2=0x7f7afc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7afc066f30 0x7f7afc0693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009160 tx=0x7f7b0c016000 comp rx=0 tx=0).stop 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0c3db0 msgr2=0x7f7b1c16ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0c3db0 0x7f7b1c16ce30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400cc90 tx=0x7f7b14001a20 comp rx=0 tx=0).stop 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 shutdown_connections 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7afc066f30 0x7f7afc0693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b1c0c3db0 0x7f7b1c16ce30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b1c069d00 0x7f7b1c15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 --2- 172.21.15.45:0/3425167899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b1c0b0610 0x7f7b1c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 >> 172.21.15.45:0/3425167899 conn(0x7f7b1c067e00 msgr2=0x7f7b1c158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 shutdown_connections 2023-10-06T05:17:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.671+0000 7f7b20d7a700 1 -- 172.21.15.45:0/3425167899 wait complete. 2023-10-06T05:17:40.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2023-10-06T05:17:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.821+0000 7f8d21000700 1 Processor -- start 2023-10-06T05:17:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.821+0000 7f8d21000700 1 -- start start 2023-10-06T05:17:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d21000700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c0c3db0 0x7f8d1c0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d21000700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8d1c069ed0 0x7f8d1c0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d21000700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c0ce6a0 con 0x7f8d1c0b0610 2023-10-06T05:17:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d21000700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c0690f0 con 0x7f8d1c069ed0 2023-10-06T05:17:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d21000700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0ceeb0 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d19d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c0c3db0 0x7f8d1c0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d1ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8d1c069ed0 0x7f8d1c0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d1a59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8d1c0b0610 0x7f8d1c0697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:35124/0 (socket says 172.21.15.45:35124) 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d19d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c0c3db0 0x7f8d1c0c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:37232/0 (socket says 172.21.15.45:37232) 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d1ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8d1c069ed0 0x7f8d1c0cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:44310/0 (socket says 172.21.15.45:44310) 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.822+0000 7f8d1a59c700 1 -- 172.21.15.45:0/1591244599 learned_addr learned my addr 172.21.15.45:0/1591244599 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1ad9d700 1 -- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c0c3db0 msgr2=0x7f8d1c0c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1ad9d700 1 --2- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c0c3db0 0x7f8d1c0c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1ad9d700 1 -- 172.21.15.45:0/1591244599 >> v1:172.21.15.144:6789/0 conn(0x7f8d1c0b0610 legacy=0x7f8d1c0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1ad9d700 1 -- 172.21.15.45:0/1591244599 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c159880 con 0x7f8d1c069ed0 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d19d9b700 1 --2- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c0c3db0 0x7f8d1c0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1959a700 1 -- 172.21.15.45:0/1591244599 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839984748 0 0) 0x7f8d1c0ce6a0 con 0x7f8d1c0b0610 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1ad9d700 1 --2- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c069ed0 0x7f8d1c0cc350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009e90 tx=0x7f8d14009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87e6c76de0c546be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1959a700 1 -- 172.21.15.45:0/1591244599 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400e070 con 0x7f8d1c069ed0 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1959a700 1 -- 172.21.15.45:0/1591244599 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400bbf0 con 0x7f8d1c069ed0 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.823+0000 7f8d1959a700 1 -- 172.21.15.45:0/1591244599 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14012430 con 0x7f8d1c069ed0 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 -- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c069ed0 msgr2=0x7f8d1c0cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 --2- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c069ed0 0x7f8d1c0cc350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009e90 tx=0x7f8d14009780 comp rx=0 tx=0).stop 2023-10-06T05:17:40.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 -- 172.21.15.45:0/1591244599 shutdown_connections 2023-10-06T05:17:40.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 --2- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c069ed0 0x7f8d1c0cc350 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 --2- 172.21.15.45:0/1591244599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c0c3db0 0x7f8d1c0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 -- 172.21.15.45:0/1591244599 >> 172.21.15.45:0/1591244599 conn(0x7f8d1c067e00 msgr2=0x7f8d1c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 -- 172.21.15.45:0/1591244599 shutdown_connections 2023-10-06T05:17:40.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.824+0000 7f8d21000700 1 -- 172.21.15.45:0/1591244599 wait complete. 2023-10-06T05:17:40.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.825+0000 7f8d21000700 1 Processor -- start 2023-10-06T05:17:40.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.825+0000 7f8d21000700 1 -- start start 2023-10-06T05:17:40.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d21000700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c069ed0 0x7f8d1c14ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d21000700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c0b0610 0x7f8d1c0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d21000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c0c3db0 0x7f8d1c15f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d21000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c15d800 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d21000700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0c6360 con 0x7f8d1c0b0610 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d21000700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c15cef0 con 0x7f8d1c069ed0 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c0c3db0 0x7f8d1c15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c0c3db0 0x7f8d1c15f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44052/0 (socket says 172.21.15.45:44052) 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 -- 172.21.15.45:0/3546967574 learned_addr learned my addr 172.21.15.45:0/3546967574 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1a59c700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c069ed0 0x7f8d1c14ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d19d9b700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c0b0610 0x7f8d1c0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 -- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c069ed0 msgr2=0x7f8d1c14ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c069ed0 0x7f8d1c14ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 -- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c0b0610 msgr2=0x7f8d1c0b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c0b0610 0x7f8d1c0b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.826+0000 7f8d1ad9d700 1 -- 172.21.15.45:0/3546967574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c009420 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.827+0000 7f8d1ad9d700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c0c3db0 0x7f8d1c15f620 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400b6c0 tx=0x7f8d14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.827+0000 7f8d0a7fc700 1 -- 172.21.15.45:0/3546967574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1401d630 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.827+0000 7f8d0a7fc700 1 -- 172.21.15.45:0/3546967574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1401d790 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.827+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d14009a20 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.827+0000 7f8d0a7fc700 1 -- 172.21.15.45:0/3546967574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1401db50 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.827+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c15d1b0 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.828+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c15d9b0 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.829+0000 7f8d0a7fc700 1 -- 172.21.15.45:0/3546967574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d14024070 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.829+0000 7f8d0a7fc700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8cfc066f30 0x7f8cfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.830+0000 7f8d0a7fc700 1 -- 172.21.15.45:0/3546967574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8d14087280 con 0x7f8d1c0c3db0 2023-10-06T05:17:40.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.830+0000 7f8d1a59c700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8cfc066f30 0x7f8cfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.830+0000 7f8d1a59c700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8cfc066f30 0x7f8cfc0693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d10001230 tx=0x7f8d10009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:40.834+0000 7f8d0a7fc700 1 -- 172.21.15.45:0/3546967574 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d140542d0 con 0x7f8d1c0c3db0 2023-10-06T05:17:41.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.035+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) v1 -- 0x7f8d1c06e3c0 con 0x7f8d1c0c3db0 2023-10-06T05:17:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.039+0000 7f8d0a7fc700 1 -- 172.21.15.45:0/3546967574 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v13)=0 v13) v1 ==== 135+0+0 (secure 0 0 0) 0x7f8d1c06e3c0 con 0x7f8d1c0c3db0 2023-10-06T05:17:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8cfc066f30 msgr2=0x7f8cfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8cfc066f30 0x7f8cfc0693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d10001230 tx=0x7f8d10009040 comp rx=0 tx=0).stop 2023-10-06T05:17:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c0c3db0 msgr2=0x7f8d1c15f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c0c3db0 0x7f8d1c15f620 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400b6c0 tx=0x7f8d14019000 comp rx=0 tx=0).stop 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 shutdown_connections 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8cfc066f30 0x7f8cfc0693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c0c3db0 0x7f8d1c15f620 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1c069ed0 0x7f8d1c14ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 --2- 172.21.15.45:0/3546967574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c0b0610 0x7f8d1c0b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 >> 172.21.15.45:0/3546967574 conn(0x7f8d1c067e00 msgr2=0x7f8d1c0ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 shutdown_connections 2023-10-06T05:17:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.041+0000 7f8d21000700 1 -- 172.21.15.45:0/3546967574 wait complete. 2023-10-06T05:17:41.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2023-10-06T05:17:41.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.189+0000 7fc46d94b700 1 Processor -- start 2023-10-06T05:17:41.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.189+0000 7fc46d94b700 1 -- start start 2023-10-06T05:17:41.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc46d94b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc46d94b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc46815c6f0 0x7fc4680caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc46d94b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4680cd740 con 0x7fc4680d02b0 2023-10-06T05:17:41.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc46d94b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4680d0ad0 con 0x7fc46815c6f0 2023-10-06T05:17:41.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc46d94b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4680ccb30 con 0x7fc4680d0680 2023-10-06T05:17:41.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc466ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4680d02b0 0x7fc46815d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52266/0 (socket says 172.21.15.45:52266) 2023-10-06T05:17:41.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc466ffd700 1 -- 172.21.15.45:0/2841121486 learned_addr learned my addr 172.21.15.45:0/2841121486 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc4677fe700 1 --2- 172.21.15.45:0/2841121486 >> v2:172.21.15.144:3300/0 conn(0x7fc46815c6f0 0x7fc4680caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.190+0000 7fc4667fc700 1 --2- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc465ffb700 1 -- 172.21.15.45:0/2841121486 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581732956 0 0) 0x7fc4680cd740 con 0x7fc4680d02b0 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc465ffb700 1 -- 172.21.15.45:0/2841121486 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc454003570 con 0x7fc4680d02b0 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc4677fe700 1 -- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 msgr2=0x7fc46815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc4677fe700 1 --2- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc4677fe700 1 -- 172.21.15.45:0/2841121486 >> v1:172.21.15.45:6789/0 conn(0x7fc4680d02b0 legacy=0x7fc46815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc4677fe700 1 -- 172.21.15.45:0/2841121486 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4680cce10 con 0x7fc46815c6f0 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc4667fc700 1 --2- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc4677fe700 1 --2- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc46815c6f0 0x7fc4680caf10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc460009e90 tx=0x7fc460009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56308e28c2c971dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc465ffb700 1 -- 172.21.15.45:0/2841121486 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000e070 con 0x7fc46815c6f0 2023-10-06T05:17:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc465ffb700 1 -- 172.21.15.45:0/2841121486 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46000bb40 con 0x7fc46815c6f0 2023-10-06T05:17:41.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.191+0000 7fc465ffb700 1 -- 172.21.15.45:0/2841121486 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460012430 con 0x7fc46815c6f0 2023-10-06T05:17:41.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 -- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc46815c6f0 msgr2=0x7fc4680caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 --2- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc46815c6f0 0x7fc4680caf10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc460009e90 tx=0x7fc460009780 comp rx=0 tx=0).stop 2023-10-06T05:17:41.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 -- 172.21.15.45:0/2841121486 shutdown_connections 2023-10-06T05:17:41.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 --2- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc46815c6f0 0x7fc4680caf10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 --2- 172.21.15.45:0/2841121486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 -- 172.21.15.45:0/2841121486 >> 172.21.15.45:0/2841121486 conn(0x7fc46806ed50 msgr2=0x7fc46806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 -- 172.21.15.45:0/2841121486 shutdown_connections 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.192+0000 7fc46d94b700 1 -- 172.21.15.45:0/2841121486 wait complete. 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.193+0000 7fc46d94b700 1 Processor -- start 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.193+0000 7fc46d94b700 1 -- start start 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc46d94b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4680d02b0 0x7fc4680b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc46d94b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc46d94b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc46815c6f0 0x7fc4680c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc46d94b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4680c3a00 con 0x7fc46815c6f0 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc46d94b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46815a8c0 con 0x7fc4680d02b0 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc46d94b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4680c8390 con 0x7fc4680d0680 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc46815c6f0 0x7fc4680c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc46815c6f0 0x7fc4680c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44062/0 (socket says 172.21.15.45:44062) 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.194+0000 7fc4677fe700 1 -- 172.21.15.45:0/1555425022 learned_addr learned my addr 172.21.15.45:0/1555425022 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc466ffd700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4680d02b0 0x7fc4680b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc4667fc700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc4677fe700 1 -- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 msgr2=0x7fc46815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc4677fe700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc4677fe700 1 -- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4680d02b0 msgr2=0x7fc4680b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc4677fe700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4680d02b0 0x7fc4680b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc4677fe700 1 -- 172.21.15.45:0/1555425022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc450009420 con 0x7fc46815c6f0 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc4677fe700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc46815c6f0 0x7fc4680c7b30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc4680890c0 tx=0x7fc460019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc45effd700 1 -- 172.21.15.45:0/1555425022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460027470 con 0x7fc46815c6f0 2023-10-06T05:17:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc45effd700 1 -- 172.21.15.45:0/1555425022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4600275d0 con 0x7fc46815c6f0 2023-10-06T05:17:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc45effd700 1 -- 172.21.15.45:0/1555425022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460027990 con 0x7fc46815c6f0 2023-10-06T05:17:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc460009a20 con 0x7fc46815c6f0 2023-10-06T05:17:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.195+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4680c8950 con 0x7fc46815c6f0 2023-10-06T05:17:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.196+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc468088730 con 0x7fc46815c6f0 2023-10-06T05:17:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.197+0000 7fc45effd700 1 -- 172.21.15.45:0/1555425022 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc460024070 con 0x7fc46815c6f0 2023-10-06T05:17:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.197+0000 7fc45effd700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc448066f30 0x7fc4480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.197+0000 7fc45effd700 1 -- 172.21.15.45:0/1555425022 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc460086f20 con 0x7fc46815c6f0 2023-10-06T05:17:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.197+0000 7fc466ffd700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc448066f30 0x7fc4480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.198+0000 7fc466ffd700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc448066f30 0x7fc4480693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc458007fd0 tx=0x7fc458009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:41.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.202+0000 7fc45effd700 1 -- 172.21.15.45:0/1555425022 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc460053e50 con 0x7fc46815c6f0 2023-10-06T05:17:41.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.405+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) v1 -- 0x7fc4680cd0f0 con 0x7fc46815c6f0 2023-10-06T05:17:41.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.409+0000 7fc45effd700 1 -- 172.21.15.45:0/1555425022 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v14)=0 v14) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc46002c020 con 0x7fc46815c6f0 2023-10-06T05:17:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc448066f30 msgr2=0x7fc4480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc448066f30 0x7fc4480693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc458007fd0 tx=0x7fc458009040 comp rx=0 tx=0).stop 2023-10-06T05:17:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc46815c6f0 msgr2=0x7fc4680c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc46815c6f0 0x7fc4680c7b30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc4680890c0 tx=0x7fc460019000 comp rx=0 tx=0).stop 2023-10-06T05:17:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 shutdown_connections 2023-10-06T05:17:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc448066f30 0x7fc4480693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc46815c6f0 0x7fc4680c7b30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4680d02b0 0x7fc4680b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 --2- 172.21.15.45:0/1555425022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4680d0680 0x7fc46815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 >> 172.21.15.45:0/1555425022 conn(0x7fc46806ed50 msgr2=0x7fc4680cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 shutdown_connections 2023-10-06T05:17:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:41.412+0000 7fc46d94b700 1 -- 172.21.15.45:0/1555425022 wait complete. 2023-10-06T05:17:46.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:46.428 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mv a/b a/c; sync .') 2023-10-06T05:17:46.477 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:17:46.474+0000 7f48a4439700 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x55e89f9ad400) (dversion lock w=1 last_client=5847) pv=4 v=3 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x55e89fa47400] 2023-10-06T05:17:46.480 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:17:46.480 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:17:46.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:46.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[141499]: fuse finished with error 0 and tester_r 0 2023-10-06T05:17:46.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:17:46.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:46.529 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:17:46.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.732+0000 7fd197821700 1 Processor -- start 2023-10-06T05:17:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.732+0000 7fd197821700 1 -- start start 2023-10-06T05:17:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd197821700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd190156ce0 0x7fd1901570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd197821700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd197821700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1900afbe0 0x7fd1900ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd197821700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19015b150 con 0x7fd190156ce0 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd197821700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19006bf40 con 0x7fd1900afbe0 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd197821700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19015b380 con 0x7fd19006bad0 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd190156ce0 0x7fd1901570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd190156ce0 0x7fd1901570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50746/0 (socket says 172.21.15.45:50746) 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 -- 172.21.15.45:0/946155049 learned_addr learned my addr 172.21.15.45:0/946155049 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd195dbe700 1 --2- 172.21.15.45:0/946155049 >> v2:172.21.15.144:3300/0 conn(0x7fd1900afbe0 0x7fd1900ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd194dbc700 1 --2- 172.21.15.45:0/946155049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 -- 172.21.15.45:0/946155049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 msgr2=0x7fd19015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 --2- 172.21.15.45:0/946155049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 -- 172.21.15.45:0/946155049 >> v2:172.21.15.144:3300/0 conn(0x7fd1900afbe0 msgr2=0x7fd1900ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 --2- 172.21.15.45:0/946155049 >> v2:172.21.15.144:3300/0 conn(0x7fd1900afbe0 0x7fd1900ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd1955bd700 1 -- 172.21.15.45:0/946155049 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19015b660 con 0x7fd190156ce0 2023-10-06T05:17:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.733+0000 7fd195dbe700 1 --2- 172.21.15.45:0/946155049 >> v2:172.21.15.144:3300/0 conn(0x7fd1900afbe0 0x7fd1900ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd1955bd700 1 --2- 172.21.15.45:0/946155049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd190156ce0 0x7fd1901570b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd1900aec60 tx=0x7fd18c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3270b72a46005d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd187fff700 1 -- 172.21.15.45:0/946155049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00b860 con 0x7fd190156ce0 2023-10-06T05:17:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd187fff700 1 -- 172.21.15.45:0/946155049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c00b9c0 con 0x7fd190156ce0 2023-10-06T05:17:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd187fff700 1 -- 172.21.15.45:0/946155049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00bd60 con 0x7fd190156ce0 2023-10-06T05:17:46.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 -- 172.21.15.45:0/946155049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd190156ce0 msgr2=0x7fd1901570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 --2- 172.21.15.45:0/946155049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd190156ce0 0x7fd1901570b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd1900aec60 tx=0x7fd18c009250 comp rx=0 tx=0).stop 2023-10-06T05:17:46.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 -- 172.21.15.45:0/946155049 shutdown_connections 2023-10-06T05:17:46.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 --2- 172.21.15.45:0/946155049 >> v2:172.21.15.144:3300/0 conn(0x7fd1900afbe0 0x7fd1900ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 --2- 172.21.15.45:0/946155049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd190156ce0 0x7fd1901570b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 --2- 172.21.15.45:0/946155049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 -- 172.21.15.45:0/946155049 >> 172.21.15.45:0/946155049 conn(0x7fd19006f090 msgr2=0x7fd19006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 -- 172.21.15.45:0/946155049 shutdown_connections 2023-10-06T05:17:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.734+0000 7fd197821700 1 -- 172.21.15.45:0/946155049 wait complete. 2023-10-06T05:17:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.736+0000 7fd197821700 1 Processor -- start 2023-10-06T05:17:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.736+0000 7fd197821700 1 -- start start 2023-10-06T05:17:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd197821700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd197821700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1900afbe0 0x7fd1900c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd197821700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd190156ce0 0x7fd19016b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd197821700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900c2190 con 0x7fd1900afbe0 2023-10-06T05:17:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd197821700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900c0e40 con 0x7fd190156ce0 2023-10-06T05:17:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd197821700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900c1120 con 0x7fd19006bad0 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd194dbc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1900afbe0 0x7fd1900c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd194dbc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1900afbe0 0x7fd1900c3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50750/0 (socket says 172.21.15.45:50750) 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.737+0000 7fd194dbc700 1 -- 172.21.15.45:0/766362178 learned_addr learned my addr 172.21.15.45:0/766362178 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd1955bd700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd195dbe700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd190156ce0 0x7fd19016b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd194dbc700 1 -- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 msgr2=0x7fd19015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd194dbc700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd194dbc700 1 -- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd190156ce0 msgr2=0x7fd19016b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd194dbc700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd190156ce0 0x7fd19016b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd194dbc700 1 -- 172.21.15.45:0/766362178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0094f0 con 0x7fd1900afbe0 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd195dbe700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd190156ce0 0x7fd19016b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd194dbc700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1900afbe0 0x7fd1900c3590 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd18000d890 tx=0x7fd180008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd1857fa700 1 -- 172.21.15.45:0/766362178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800125f0 con 0x7fd1900afbe0 2023-10-06T05:17:46.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd1857fa700 1 -- 172.21.15.45:0/766362178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180012750 con 0x7fd1900afbe0 2023-10-06T05:17:46.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900c1400 con 0x7fd1900afbe0 2023-10-06T05:17:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd1857fa700 1 -- 172.21.15.45:0/766362178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180012ad0 con 0x7fd1900afbe0 2023-10-06T05:17:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.738+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900c26d0 con 0x7fd1900afbe0 2023-10-06T05:17:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.739+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19006e2f0 con 0x7fd1900afbe0 2023-10-06T05:17:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.740+0000 7fd1857fa700 1 -- 172.21.15.45:0/766362178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd18001e460 con 0x7fd1900afbe0 2023-10-06T05:17:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.740+0000 7fd1857fa700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd174066f30 0x7fd1740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.740+0000 7fd1857fa700 1 -- 172.21.15.45:0/766362178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd180084df0 con 0x7fd1900afbe0 2023-10-06T05:17:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.741+0000 7fd1955bd700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd174066f30 0x7fd1740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.741+0000 7fd1955bd700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd174066f30 0x7fd1740693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1900aec60 tx=0x7fd18c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.745+0000 7fd1857fa700 1 -- 172.21.15.45:0/766362178 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd180051dc0 con 0x7fd1900afbe0 2023-10-06T05:17:46.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.994+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd19015b940 con 0x7fd1900afbe0 2023-10-06T05:17:46.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.994+0000 7fd1857fa700 1 -- 172.21.15.45:0/766362178 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fd1800553e0 con 0x7fd1900afbe0 2023-10-06T05:17:46.997 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:46.998 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:35.325485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5682":{"gid":5682,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2812290138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2812290138},{"type":"v1","addr":"172.21.15.45:6835","nonce":2812290138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:46.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.996+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd174066f30 msgr2=0x7fd1740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.996+0000 7fd197821700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd174066f30 0x7fd1740693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1900aec60 tx=0x7fd18c017000 comp rx=0 tx=0).stop 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.996+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1900afbe0 msgr2=0x7fd1900c3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.996+0000 7fd197821700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1900afbe0 0x7fd1900c3590 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd18000d890 tx=0x7fd180008040 comp rx=0 tx=0).stop 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 shutdown_connections 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd174066f30 0x7fd1740693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd190156ce0 0x7fd19016b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19006bad0 0x7fd19015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 --2- 172.21.15.45:0/766362178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1900afbe0 0x7fd1900c3590 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 >> 172.21.15.45:0/766362178 conn(0x7fd19006f090 msgr2=0x7fd190154040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 shutdown_connections 2023-10-06T05:17:47.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:46.997+0000 7fd197821700 1 -- 172.21.15.45:0/766362178 wait complete. 2023-10-06T05:17:47.001 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 33 2023-10-06T05:17:47.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2023-10-06T05:17:47.205 INFO:teuthology.orchestra.run.smithi045.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2023-10-06T05:17:46.475673+0000","btime":"2023-10-06T05:17:40.477885+0000","mode":16893,"uid":1000,"gid":1285,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2023-10-06T05:17:46.475673+0000","atime":"2023-10-06T05:17:40.477885+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2023-10-06T05:17:46.475673+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-10-06T05:17:46.475673+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-10-06T05:17:46.475673+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"waiter":0,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}]}] 2023-10-06T05:17:47.206 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2023-10-06T05:17:46.475673+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2023-10-06T05:17:40.477885+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2023-10-06T05:17:40.477885+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2023-10-06T05:17:46.475673+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2023-10-06T05:17:46.475673+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1285, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2023-10-06T05:17:46.475673+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2023-10-06T05:17:46.475673+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2023-10-06T05:17:47.206 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2023-10-06T05:17:46.475673+0000', 'btime': '2023-10-06T05:17:40.477885+0000', 'mode': 16893, 'uid': 1000, 'gid': 1285, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2023-10-06T05:17:46.475673+0000', 'atime': '2023-10-06T05:17:40.477885+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2023-10-06T05:17:46.475673+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-10-06T05:17:46.475673+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-10-06T05:17:46.475673+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'waiter': 0, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}]}] 2023-10-06T05:17:47.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:17:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.339+0000 7ff767761700 1 Processor -- start 2023-10-06T05:17:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.339+0000 7ff767761700 1 -- start start 2023-10-06T05:17:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.340+0000 7ff767761700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7601560a0 0x7ff76015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.340+0000 7ff767761700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7600ccdc0 0x7ff7600cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.340+0000 7ff767761700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600cd740 con 0x7ff760155cd0 2023-10-06T05:17:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.340+0000 7ff767761700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7600ae6b0 con 0x7ff7601560a0 2023-10-06T05:17:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.340+0000 7ff767761700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600ae8d0 con 0x7ff7600ccdc0 2023-10-06T05:17:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.340+0000 7ff7654fd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff760155cd0 0x7ff7600adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41128/0 (socket says 172.21.15.45:41128) 2023-10-06T05:17:47.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.340+0000 7ff764cfc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7601560a0 0x7ff76015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff7654fd700 1 -- 172.21.15.45:0/1781849947 learned_addr learned my addr 172.21.15.45:0/1781849947 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff765cfe700 1 --2- 172.21.15.45:0/1781849947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7600ccdc0 0x7ff7600cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff764cfc700 1 -- 172.21.15.45:0/1781849947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7600ccdc0 msgr2=0x7ff7600cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff764cfc700 1 --2- 172.21.15.45:0/1781849947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7600ccdc0 0x7ff7600cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff764cfc700 1 -- 172.21.15.45:0/1781849947 >> v1:172.21.15.45:6789/0 conn(0x7ff760155cd0 legacy=0x7ff7600adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff764cfc700 1 -- 172.21.15.45:0/1781849947 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600aebb0 con 0x7ff7601560a0 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff753fff700 1 -- 172.21.15.45:0/1781849947 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771014786 0 0) 0x7ff7600cd740 con 0x7ff760155cd0 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff764cfc700 1 --2- 172.21.15.45:0/1781849947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7601560a0 0x7ff76015a730 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7540098e0 tx=0x7ff7540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a305086cb48ede server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff753fff700 1 -- 172.21.15.45:0/1781849947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754009e80 con 0x7ff7601560a0 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff753fff700 1 -- 172.21.15.45:0/1781849947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400b860 con 0x7ff7601560a0 2023-10-06T05:17:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.341+0000 7ff753fff700 1 -- 172.21.15.45:0/1781849947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400bbe0 con 0x7ff7601560a0 2023-10-06T05:17:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 -- 172.21.15.45:0/1781849947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7601560a0 msgr2=0x7ff76015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 --2- 172.21.15.45:0/1781849947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7601560a0 0x7ff76015a730 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7540098e0 tx=0x7ff7540091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 -- 172.21.15.45:0/1781849947 shutdown_connections 2023-10-06T05:17:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 --2- 172.21.15.45:0/1781849947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7600ccdc0 0x7ff7600cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 --2- 172.21.15.45:0/1781849947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7601560a0 0x7ff76015a730 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 -- 172.21.15.45:0/1781849947 >> 172.21.15.45:0/1781849947 conn(0x7ff76006f090 msgr2=0x7ff76006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 -- 172.21.15.45:0/1781849947 shutdown_connections 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.342+0000 7ff767761700 1 -- 172.21.15.45:0/1781849947 wait complete. 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.343+0000 7ff767761700 1 Processor -- start 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.343+0000 7ff767761700 1 -- start start 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff767761700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7600ccdc0 0x7ff760169260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff767761700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff760155cd0 0x7ff7600a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff767761700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7601560a0 0x7ff7600a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff767761700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601697a0 con 0x7ff760155cd0 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff767761700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760159160 con 0x7ff7600ccdc0 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff767761700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600a5aa0 con 0x7ff7601560a0 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff764cfc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff760155cd0 0x7ff7600a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff764cfc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff760155cd0 0x7ff7600a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50766/0 (socket says 172.21.15.45:50766) 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff764cfc700 1 -- 172.21.15.45:0/1180271442 learned_addr learned my addr 172.21.15.45:0/1180271442 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff765cfe700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7601560a0 0x7ff7600a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.344+0000 7ff7654fd700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7600ccdc0 0x7ff760169260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff764cfc700 1 -- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7601560a0 msgr2=0x7ff7600a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff764cfc700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7601560a0 0x7ff7600a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff764cfc700 1 -- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7600ccdc0 msgr2=0x7ff760169260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff764cfc700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7600ccdc0 0x7ff760169260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff764cfc700 1 -- 172.21.15.45:0/1180271442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754009420 con 0x7ff760155cd0 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff7654fd700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7600ccdc0 0x7ff760169260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff764cfc700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff760155cd0 0x7ff7600a4ec0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff76006aed0 tx=0x7ff754017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff7517fa700 1 -- 172.21.15.45:0/1180271442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400bbe0 con 0x7ff760155cd0 2023-10-06T05:17:47.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff7517fa700 1 -- 172.21.15.45:0/1180271442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754011420 con 0x7ff760155cd0 2023-10-06T05:17:47.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7600a5d80 con 0x7ff760155cd0 2023-10-06T05:17:47.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff7517fa700 1 -- 172.21.15.45:0/1180271442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7540117c0 con 0x7ff760155cd0 2023-10-06T05:17:47.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.345+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7600a6340 con 0x7ff760155cd0 2023-10-06T05:17:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.346+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76006e2f0 con 0x7ff760155cd0 2023-10-06T05:17:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.347+0000 7ff7517fa700 1 -- 172.21.15.45:0/1180271442 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff754011bb0 con 0x7ff760155cd0 2023-10-06T05:17:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.347+0000 7ff7517fa700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.347+0000 7ff7517fa700 1 -- 172.21.15.45:0/1180271442 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff754085cf0 con 0x7ff760155cd0 2023-10-06T05:17:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.347+0000 7ff7654fd700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.348+0000 7ff7654fd700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff744066f30 0x7ff7440693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0011e0 tx=0x7ff75c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.352+0000 7ff7517fa700 1 -- 172.21.15.45:0/1180271442 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff75408a070 con 0x7ff760155cd0 2023-10-06T05:17:47.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.601+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff7600aee90 con 0x7ff760155cd0 2023-10-06T05:17:47.604 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:17:47.601+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:17:47.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.605+0000 7ff7517fa700 1 -- 172.21.15.45:0/1180271442 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff754011e60 con 0x7ff760155cd0 2023-10-06T05:17:47.610 INFO:tasks.ceph.mds.c.smithi045.stderr: -3336> 2023-10-06T05:17:38.296+0000 7f48a1c34700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:17:47.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.607+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff744066f30 msgr2=0x7ff7440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.607+0000 7ff767761700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff744066f30 0x7ff7440693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0011e0 tx=0x7ff75c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.607+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff760155cd0 msgr2=0x7ff7600a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.607+0000 7ff767761700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff760155cd0 0x7ff7600a4ec0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff76006aed0 tx=0x7ff754017000 comp rx=0 tx=0).stop 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 shutdown_connections 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7601560a0 0x7ff7600a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7600ccdc0 0x7ff760169260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 --2- 172.21.15.45:0/1180271442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff760155cd0 0x7ff7600a4ec0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 >> 172.21.15.45:0/1180271442 conn(0x7ff76006f090 msgr2=0x7ff760159720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 shutdown_connections 2023-10-06T05:17:47.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.608+0000 7ff767761700 1 -- 172.21.15.45:0/1180271442 wait complete. 2023-10-06T05:17:47.612 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:17:47.614 INFO:tasks.ceph.mds.c.smithi045.stderr: -256> 2023-10-06T05:17:46.474+0000 7f48a4439700 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x55e89f9ad400) (dversion lock w=1 last_client=5847) pv=4 v=3 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x55e89fa47400] 2023-10-06T05:17:47.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-10-06T05:17:47.639 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:17:47.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.758+0000 7fdfb38b6700 1 Processor -- start 2023-10-06T05:17:47.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.758+0000 7fdfb38b6700 1 -- start start 2023-10-06T05:17:47.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb38b6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac0aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb38b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfac0c8b70 0x7fdfac0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb38b6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac0c3b50 con 0x7fdfac157330 2023-10-06T05:17:47.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb38b6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfac157700 con 0x7fdfac0c8b70 2023-10-06T05:17:47.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb38b6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0c98e0 con 0x7fdfac0ccb70 2023-10-06T05:17:47.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb1e53700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfac0c8b70 0x7fdfac0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb1e53700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfac0c8b70 0x7fdfac0c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50790/0 (socket says 172.21.15.45:50790) 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb1e53700 1 -- 172.21.15.45:0/117711044 learned_addr learned my addr 172.21.15.45:0/117711044 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.759+0000 7fdfb0e51700 1 --2- 172.21.15.45:0/117711044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac0aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdfb1e53700 1 -- 172.21.15.45:0/117711044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 msgr2=0x7fdfac0aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdfb1e53700 1 --2- 172.21.15.45:0/117711044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac0aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdfb1e53700 1 -- 172.21.15.45:0/117711044 >> v1:172.21.15.144:6789/0 conn(0x7fdfac157330 legacy=0x7fdfac0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdfb1e53700 1 -- 172.21.15.45:0/117711044 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac0c9bc0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdfb1e53700 1 --2- 172.21.15.45:0/117711044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac0c8ff0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009f20 tx=0x7fdfa4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e404c7ea5f8c46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdf9ffff700 1 -- 172.21.15.45:0/117711044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400e070 con 0x7fdfac0c8b70 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdf9ffff700 1 -- 172.21.15.45:0/117711044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa400ba80 con 0x7fdfac0c8b70 2023-10-06T05:17:47.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.760+0000 7fdf9ffff700 1 -- 172.21.15.45:0/117711044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40092e0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 -- 172.21.15.45:0/117711044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 msgr2=0x7fdfac0c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/117711044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac0c8ff0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009f20 tx=0x7fdfa4009840 comp rx=0 tx=0).stop 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 -- 172.21.15.45:0/117711044 shutdown_connections 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/117711044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac0c8ff0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/117711044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac0aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 -- 172.21.15.45:0/117711044 >> 172.21.15.45:0/117711044 conn(0x7fdfac067e00 msgr2=0x7fdfac0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 -- 172.21.15.45:0/117711044 shutdown_connections 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.761+0000 7fdfb38b6700 1 -- 172.21.15.45:0/117711044 wait complete. 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.762+0000 7fdfb38b6700 1 Processor -- start 2023-10-06T05:17:47.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.762+0000 7fdfb38b6700 1 -- start start 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb38b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb38b6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb38b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfac157330 0x7fdfac1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb38b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac158ca0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb38b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac159df0 con 0x7fdfac157330 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb38b6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac15a7c0 con 0x7fdfac0ccb70 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb1652700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb1652700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50794/0 (socket says 172.21.15.45:50794) 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.763+0000 7fdfb1652700 1 -- 172.21.15.45:0/3852374795 learned_addr learned my addr 172.21.15.45:0/3852374795 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb1e53700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfac157330 0x7fdfac1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb0e51700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb1652700 1 -- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 msgr2=0x7fdfac152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb1652700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb1652700 1 -- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfac157330 msgr2=0x7fdfac1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb1652700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfac157330 0x7fdfac1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb1652700 1 -- 172.21.15.45:0/3852374795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4009ae0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb1652700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac15a280 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c7c0 tx=0x7fdfa8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdf9d7fa700 1 -- 172.21.15.45:0/3852374795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8011590 con 0x7fdfac0c8b70 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdf9d7fa700 1 -- 172.21.15.45:0/3852374795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa8011710 con 0x7fdfac0c8b70 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac1594c0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdf9d7fa700 1 -- 172.21.15.45:0/3852374795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8011ad0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.764+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac159a80 con 0x7fdfac0c8b70 2023-10-06T05:17:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.765+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac06e350 con 0x7fdfac0c8b70 2023-10-06T05:17:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.766+0000 7fdf9d7fa700 1 -- 172.21.15.45:0/3852374795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfa801b3b0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.767+0000 7fdf9d7fa700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf90066f30 0x7fdf900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.767+0000 7fdf9d7fa700 1 -- 172.21.15.45:0/3852374795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fdfa8084100 con 0x7fdfac0c8b70 2023-10-06T05:17:47.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.767+0000 7fdfb0e51700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf90066f30 0x7fdf900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.768+0000 7fdfb0e51700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf90066f30 0x7fdf900693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0abca0 tx=0x7fdfa0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.771+0000 7fdf9d7fa700 1 -- 172.21.15.45:0/3852374795 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfa80511b0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.975+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7fdfac0c37e0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.979+0000 7fdf9d7fa700 1 -- 172.21.15.45:0/3852374795 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v15) v1 ==== 121+0+0 (secure 0 0 0) 0x7fdfa80170c0 con 0x7fdfac0c8b70 2023-10-06T05:17:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf90066f30 msgr2=0x7fdf900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf90066f30 0x7fdf900693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0abca0 tx=0x7fdfa0008040 comp rx=0 tx=0).stop 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 msgr2=0x7fdfac15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac15a280 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c7c0 tx=0x7fdfa8007f70 comp rx=0 tx=0).stop 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 shutdown_connections 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf90066f30 0x7fdf900693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfac157330 0x7fdfac1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac0c8b70 0x7fdfac15a280 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 --2- 172.21.15.45:0/3852374795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfac0ccb70 0x7fdfac152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 >> 172.21.15.45:0/3852374795 conn(0x7fdfac067e00 msgr2=0x7fdfac06d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 shutdown_connections 2023-10-06T05:17:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:47.981+0000 7fdfb38b6700 1 -- 172.21.15.45:0/3852374795 wait complete. 2023-10-06T05:17:47.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2023-10-06T05:17:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.174+0000 7f0201d94700 1 Processor -- start 2023-10-06T05:17:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.174+0000 7f0201d94700 1 -- start start 2023-10-06T05:17:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f0201d94700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f01fc0d02b0 0x7f01fc0d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f0201d94700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 0x7f01fc0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f0201d94700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc0cd740 con 0x7f01fc15d820 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f0201d94700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01fc0ccb30 con 0x7f01fc0d02b0 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f0201d94700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0cce10 con 0x7f01fc15c8c0 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01faffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01fc15d820 0x7f01fc15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41158/0 (socket says 172.21.15.45:41158) 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01faffd700 1 -- 172.21.15.45:0/2043336522 learned_addr learned my addr 172.21.15.45:0/2043336522 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fb7fe700 1 --2- 172.21.15.45:0/2043336522 >> v2:172.21.15.144:3300/0 conn(0x7f01fc0d02b0 0x7f01fc0d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fbfff700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 0x7f01fc0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722024909 0 0) 0x7f01fc0cd740 con 0x7f01fc15d820 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e8003570 con 0x7f01fc15d820 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157028414 0 0) 0x7f01e8003570 con 0x7f01fc15d820 2023-10-06T05:17:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 msgr2=0x7f01fc0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 0x7f01fc0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 >> v2:172.21.15.144:3300/0 conn(0x7f01fc0d02b0 msgr2=0x7f01fc0d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 --2- 172.21.15.45:0/2043336522 >> v2:172.21.15.144:3300/0 conn(0x7f01fc0d02b0 0x7f01fc0d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.175+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0cd0f0 con 0x7f01fc15d820 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f01ec001470 con 0x7f01fc15d820 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 >> v1:172.21.15.45:6789/0 conn(0x7f01fc15d820 legacy=0x7f01fc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8003a60 0x7f01e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e8004810 0x7f01e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 0x7f01e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0ccb30 con 0x7f01e8004810 2023-10-06T05:17:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0cce10 con 0x7f01e8003a60 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e800ba60 con 0x7f01fc15c8c0 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fb7fe700 1 --2- 172.21.15.45:0/2043336522 >> v2:172.21.15.144:3300/0 conn(0x7f01fc0d02b0 0x7f01fc0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01faffd700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e8004810 0x7f01e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fbfff700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 0x7f01e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fb7fe700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8003a60 0x7f01e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01faffd700 1 -- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 msgr2=0x7f01e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01faffd700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 0x7f01e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01faffd700 1 -- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8003a60 msgr2=0x7f01e8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01faffd700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8003a60 0x7f01e8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01faffd700 1 -- 172.21.15.45:0/2043336522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8003570 con 0x7f01e8004810 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fbfff700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc15c8c0 0x7f01e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01faffd700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e8004810 0x7f01e8004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c3f0 tx=0x7f01ec001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eebc4be35d4cc59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00cc80 con 0x7f01e8004810 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00cde0 con 0x7f01e8004810 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.176+0000 7f01fa7fc700 1 -- 172.21.15.45:0/2043336522 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec005b70 con 0x7f01e8004810 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.177+0000 7f0201d94700 1 -- 172.21.15.45:0/2043336522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e8004810 msgr2=0x7f01e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.177+0000 7f0201d94700 1 --2- 172.21.15.45:0/2043336522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e8004810 0x7f01e8004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c3f0 tx=0x7f01ec001e90 comp rx=0 tx=0).stop 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.177+0000 7f01fbfff700 1 -- 172.21.15.45:0/2043336522 reap_dead start 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.177+0000 7f0201d94700 1 -- 172.21.15.45:0/2043336522 shutdown_connections 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.177+0000 7f0201d94700 1 -- 172.21.15.45:0/2043336522 >> 172.21.15.45:0/2043336522 conn(0x7f01fc06ed50 msgr2=0x7f01fc06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.177+0000 7f0201d94700 1 -- 172.21.15.45:0/2043336522 shutdown_connections 2023-10-06T05:17:48.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.177+0000 7f0201d94700 1 -- 172.21.15.45:0/2043336522 wait complete. 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.178+0000 7f0201d94700 1 Processor -- start 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.178+0000 7f0201d94700 1 -- start start 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f0201d94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8004810 0x7f01fc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f0201d94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01fc0ce720 0x7f01fc0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f0201d94700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc169880 0x7f01fc169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f0201d94700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc16a980 con 0x7f01fc0ce720 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f0201d94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0cf0d0 con 0x7f01e8004810 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f0201d94700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0c85c0 con 0x7f01fc169880 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f01faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01fc0ce720 0x7f01fc0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f01faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01fc0ce720 0x7f01fc0ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50800/0 (socket says 172.21.15.45:50800) 2023-10-06T05:17:48.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f01faffd700 1 -- 172.21.15.45:0/1556085484 learned_addr learned my addr 172.21.15.45:0/1556085484 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f01fb7fe700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8004810 0x7f01fc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.179+0000 7f01fbfff700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc169880 0x7f01fc169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01faffd700 1 -- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc169880 msgr2=0x7f01fc169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01faffd700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc169880 0x7f01fc169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01faffd700 1 -- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8004810 msgr2=0x7f01fc0b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01faffd700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8004810 0x7f01fc0b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01faffd700 1 -- 172.21.15.45:0/1556085484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc16a640 con 0x7f01fc0ce720 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01faffd700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01fc0ce720 0x7f01fc0ceb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f01ec001ed0 tx=0x7f01ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01f37fe700 1 -- 172.21.15.45:0/1556085484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0072e0 con 0x7f01fc0ce720 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01f37fe700 1 -- 172.21.15.45:0/1556085484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec007480 con 0x7f01fc0ce720 2023-10-06T05:17:48.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc0c8880 con 0x7f01fc0ce720 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f01f37fe700 1 -- 172.21.15.45:0/1556085484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec007840 con 0x7f01fc0ce720 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.180+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc0c8e40 con 0x7f01fc0ce720 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.181+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc088730 con 0x7f01fc0ce720 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.182+0000 7f01f37fe700 1 -- 172.21.15.45:0/1556085484 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01ec019030 con 0x7f01fc0ce720 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.182+0000 7f01f37fe700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01dc066f30 0x7f01dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.182+0000 7f01f37fe700 1 -- 172.21.15.45:0/1556085484 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f01ec0859a0 con 0x7f01fc0ce720 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.182+0000 7f01fb7fe700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01dc066f30 0x7f01dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.183+0000 7f01fb7fe700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01dc066f30 0x7f01dc0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f01e40093f0 tx=0x7f01e4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.186+0000 7f01f37fe700 1 -- 172.21.15.45:0/1556085484 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01ec052920 con 0x7f01fc0ce720 2023-10-06T05:17:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.390+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) v1 -- 0x7f01fc16a980 con 0x7f01fc0ce720 2023-10-06T05:17:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.391+0000 7f01f37fe700 1 -- 172.21.15.45:0/1556085484 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v15)=0 v15) v1 ==== 135+0+0 (secure 0 0 0) 0x7f01ec055f40 con 0x7f01fc0ce720 2023-10-06T05:17:48.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01dc066f30 msgr2=0x7f01dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01dc066f30 0x7f01dc0693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f01e40093f0 tx=0x7f01e4008040 comp rx=0 tx=0).stop 2023-10-06T05:17:48.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01fc0ce720 msgr2=0x7f01fc0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01fc0ce720 0x7f01fc0ceb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f01ec001ed0 tx=0x7f01ec015000 comp rx=0 tx=0).stop 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 shutdown_connections 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01dc066f30 0x7f01dc0693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01fc169880 0x7f01fc169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01e8004810 0x7f01fc0b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 --2- 172.21.15.45:0/1556085484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01fc0ce720 0x7f01fc0ceb90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 >> 172.21.15.45:0/1556085484 conn(0x7f01fc06ed50 msgr2=0x7f01fc0cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 shutdown_connections 2023-10-06T05:17:48.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.393+0000 7f0201d94700 1 -- 172.21.15.45:0/1556085484 wait complete. 2023-10-06T05:17:48.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:17:48.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.541+0000 7fdb643ab700 1 Processor -- start 2023-10-06T05:17:48.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.541+0000 7fdb643ab700 1 -- start start 2023-10-06T05:17:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb643ab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb5c140ab0 0x7fdb5c0acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb643ab700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c0c9940 con 0x7fdb5c1406e0 2023-10-06T05:17:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb643ab700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c06d230 con 0x7fdb5c0ad4e0 2023-10-06T05:17:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb643ab700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5c06d510 con 0x7fdb5c140ab0 2023-10-06T05:17:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb62147700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdb5c1406e0 0x7fdb5c06c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41162/0 (socket says 172.21.15.45:41162) 2023-10-06T05:17:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb62147700 1 -- 172.21.15.45:0/408799775 learned_addr learned my addr 172.21.15.45:0/408799775 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb61946700 1 --2- 172.21.15.45:0/408799775 >> v2:172.21.15.144:3300/0 conn(0x7fdb5c140ab0 0x7fdb5c0acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830947883 0 0) 0x7fdb5c0c9940 con 0x7fdb5c1406e0 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.542+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb48003570 con 0x7fdb5c1406e0 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372902174 0 0) 0x7fdb48003570 con 0x7fdb5c1406e0 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 >> v2:172.21.15.144:3300/0 conn(0x7fdb5c140ab0 msgr2=0x7fdb5c0acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 --2- 172.21.15.45:0/408799775 >> v2:172.21.15.144:3300/0 conn(0x7fdb5c140ab0 0x7fdb5c0acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 >> v1:172.21.15.144:6789/0 conn(0x7fdb5c0ad4e0 legacy=0x7fdb5c0c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c06d7f0 con 0x7fdb5c1406e0 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fdb58001470 con 0x7fdb5c1406e0 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 >> v1:172.21.15.45:6789/0 conn(0x7fdb5c1406e0 legacy=0x7fdb5c06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb48003a60 0x7fdb48003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb48004810 0x7fdb48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb4800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c06d510 con 0x7fdb480051c0 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4800c160 con 0x7fdb48004810 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4800c320 con 0x7fdb48003a60 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb61946700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb48004810 0x7fdb48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62147700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb48003a60 0x7fdb48003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62948700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb4800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62948700 1 -- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb48003a60 msgr2=0x7fdb48003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62948700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb48003a60 0x7fdb48003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62948700 1 -- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb48004810 msgr2=0x7fdb48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62948700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb48004810 0x7fdb48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62948700 1 -- 172.21.15.45:0/408799775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48003570 con 0x7fdb480051c0 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.543+0000 7fdb62948700 1 -- 172.21.15.45:0/408799775 reap_dead start 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb62948700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb4800ba60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdb54001fc0 tx=0x7fdb54001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a722bd351a6cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54001950 con 0x7fdb480051c0 2023-10-06T05:17:48.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5400c8b0 con 0x7fdb480051c0 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb61145700 1 -- 172.21.15.45:0/408799775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400cc90 con 0x7fdb480051c0 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb643ab700 1 -- 172.21.15.45:0/408799775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 msgr2=0x7fdb4800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb643ab700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb4800ba60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdb54001fc0 tx=0x7fdb54001640 comp rx=0 tx=0).stop 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb643ab700 1 -- 172.21.15.45:0/408799775 shutdown_connections 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb643ab700 1 --2- 172.21.15.45:0/408799775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb4800ba60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.544+0000 7fdb643ab700 1 -- 172.21.15.45:0/408799775 >> 172.21.15.45:0/408799775 conn(0x7fdb5c0691c0 msgr2=0x7fdb5c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.545+0000 7fdb643ab700 1 -- 172.21.15.45:0/408799775 shutdown_connections 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.545+0000 7fdb643ab700 1 -- 172.21.15.45:0/408799775 wait complete. 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 Processor -- start 2023-10-06T05:17:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 -- start start 2023-10-06T05:17:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb5c13d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb5c0b05f0 0x7fdb5c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5c0c2d20 0x7fdb5c0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c13deb0 con 0x7fdb480051c0 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0c36d0 con 0x7fdb5c0c2d20 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.546+0000 7fdb643ab700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0b0d40 con 0x7fdb5c0b05f0 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb5c13d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb5c13d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50810/0 (socket says 172.21.15.45:50810) 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 -- 172.21.15.45:0/2413036281 learned_addr learned my addr 172.21.15.45:0/2413036281 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb61946700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb5c0b05f0 0x7fdb5c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62948700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5c0c2d20 0x7fdb5c0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 -- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb5c0b05f0 msgr2=0x7fdb5c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb5c0b05f0 0x7fdb5c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 -- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5c0c2d20 msgr2=0x7fdb5c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5c0c2d20 0x7fdb5c0c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 -- 172.21.15.45:0/2413036281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c0cc3d0 con 0x7fdb480051c0 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb61946700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb5c0b05f0 0x7fdb5c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62948700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5c0c2d20 0x7fdb5c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb62147700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb5c13d970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdb580073c0 tx=0x7fdb58001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb4e7fc700 1 -- 172.21.15.45:0/2413036281 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58007830 con 0x7fdb480051c0 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb4e7fc700 1 -- 172.21.15.45:0/2413036281 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5800f8d0 con 0x7fdb480051c0 2023-10-06T05:17:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb4e7fc700 1 -- 172.21.15.45:0/2413036281 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800fc50 con 0x7fdb480051c0 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c0b0a60 con 0x7fdb480051c0 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.547+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c0cc060 con 0x7fdb480051c0 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.548+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c06e2d0 con 0x7fdb480051c0 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.549+0000 7fdb4e7fc700 1 -- 172.21.15.45:0/2413036281 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb5801e3b0 con 0x7fdb480051c0 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.549+0000 7fdb4e7fc700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb40066f30 0x7fdb400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.549+0000 7fdb4e7fc700 1 -- 172.21.15.45:0/2413036281 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fdb580844b0 con 0x7fdb480051c0 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.549+0000 7fdb61946700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb40066f30 0x7fdb400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.550+0000 7fdb61946700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb40066f30 0x7fdb400693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0aeca0 tx=0x7fdb50008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.554+0000 7fdb4e7fc700 1 -- 172.21.15.45:0/2413036281 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb5804e5a0 con 0x7fdb480051c0 2023-10-06T05:17:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:48.807+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fdb5c04bb40 con 0x7fdb480051c0 2023-10-06T05:17:49.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.607+0000 7fdb4e7fc700 1 -- 172.21.15.45:0/2413036281 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v36) v1 ==== 162+0+0 (secure 0 0 0) 0x7fdb58054ef0 con 0x7fdb480051c0 2023-10-06T05:17:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb40066f30 msgr2=0x7fdb400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb40066f30 0x7fdb400693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0aeca0 tx=0x7fdb50008040 comp rx=0 tx=0).stop 2023-10-06T05:17:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 msgr2=0x7fdb5c13d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb5c13d970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdb580073c0 tx=0x7fdb58001e50 comp rx=0 tx=0).stop 2023-10-06T05:17:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 shutdown_connections 2023-10-06T05:17:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb40066f30 0x7fdb400693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5c0c2d20 0x7fdb5c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb480051c0 0x7fdb5c13d970 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 --2- 172.21.15.45:0/2413036281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb5c0b05f0 0x7fdb5c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 >> 172.21.15.45:0/2413036281 conn(0x7fdb5c0691c0 msgr2=0x7fdb5c13f740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.611+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 shutdown_connections 2023-10-06T05:17:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.612+0000 7fdb643ab700 1 -- 172.21.15.45:0/2413036281 wait complete. 2023-10-06T05:17:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:17:49.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:49.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.765+0000 7f44ff2c4700 1 Processor -- start 2023-10-06T05:17:49.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.765+0000 7f44ff2c4700 1 -- start start 2023-10-06T05:17:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.766+0000 7f44ff2c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.766+0000 7f44ff2c4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f44f8157030 0x7f44f80cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.766+0000 7f44ff2c4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f815b000 con 0x7f44f80c8ff0 2023-10-06T05:17:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.766+0000 7f44ff2c4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f80c9830 con 0x7f44f8157030 2023-10-06T05:17:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.766+0000 7f44ff2c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815b2e0 con 0x7f44f80c93c0 2023-10-06T05:17:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.766+0000 7f44fd060700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f44f80c8ff0 0x7f44f80caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41172/0 (socket says 172.21.15.45:41172) 2023-10-06T05:17:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.766+0000 7f44fd060700 1 -- 172.21.15.45:0/986908093 learned_addr learned my addr 172.21.15.45:0/986908093 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44fc85f700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44fd861700 1 --2- 172.21.15.45:0/986908093 >> v2:172.21.15.144:3300/0 conn(0x7f44f8157030 0x7f44f80cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160429404 0 0) 0x7f44f815b000 con 0x7f44f80c8ff0 2023-10-06T05:17:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e4003570 con 0x7f44f80c8ff0 2023-10-06T05:17:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 599775756 0 0) 0x7f44e4003570 con 0x7f44f80c8ff0 2023-10-06T05:17:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 msgr2=0x7f44f8156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 >> v2:172.21.15.144:3300/0 conn(0x7f44f8157030 msgr2=0x7f44f80cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 --2- 172.21.15.45:0/986908093 >> v2:172.21.15.144:3300/0 conn(0x7f44f8157030 0x7f44f80cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f815b5c0 con 0x7f44f80c8ff0 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f44ec001470 con 0x7f44f80c8ff0 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44fd861700 1 --2- 172.21.15.45:0/986908093 >> v2:172.21.15.144:3300/0 conn(0x7f44f8157030 0x7f44f80cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 >> v1:172.21.15.45:6789/0 conn(0x7f44f80c8ff0 legacy=0x7f44f80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e4003a60 0x7f44e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4004d50 0x7f44e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80c9830 con 0x7f44e4004d50 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815b2e0 con 0x7f44e4003a60 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e400ba60 con 0x7f44f80c93c0 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44fd861700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4004d50 0x7f44e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44fc85f700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.767+0000 7f44fd060700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e4003a60 0x7f44e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44fc85f700 1 -- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e4003a60 msgr2=0x7f44e4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44fc85f700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e4003a60 0x7f44e4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44fc85f700 1 -- 172.21.15.45:0/986908093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4004d50 msgr2=0x7f44e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44fc85f700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4004d50 0x7f44e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44fc85f700 1 -- 172.21.15.45:0/986908093 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4003570 con 0x7f44f80c93c0 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44fd861700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4004d50 0x7f44e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44fc85f700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44e4004810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009c00 tx=0x7f44f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0212880a152dc9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400e6b0 con 0x7f44f80c93c0 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f400e810 con 0x7f44f80c93c0 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.768+0000 7f44ebfff700 1 -- 172.21.15.45:0/986908093 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400ebd0 con 0x7f44f80c93c0 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.769+0000 7f44ff2c4700 1 -- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 msgr2=0x7f44e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.769+0000 7f44ff2c4700 1 --2- 172.21.15.45:0/986908093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44e4004810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009c00 tx=0x7f44f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.769+0000 7f44fd861700 1 -- 172.21.15.45:0/986908093 reap_dead start 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.769+0000 7f44ff2c4700 1 -- 172.21.15.45:0/986908093 shutdown_connections 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.769+0000 7f44ff2c4700 1 -- 172.21.15.45:0/986908093 >> 172.21.15.45:0/986908093 conn(0x7f44f80cdf50 msgr2=0x7f44f80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.769+0000 7f44ff2c4700 1 -- 172.21.15.45:0/986908093 shutdown_connections 2023-10-06T05:17:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.769+0000 7f44ff2c4700 1 -- 172.21.15.45:0/986908093 wait complete. 2023-10-06T05:17:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.770+0000 7f44ff2c4700 1 Processor -- start 2023-10-06T05:17:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.770+0000 7f44ff2c4700 1 -- start start 2023-10-06T05:17:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44ff2c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44ff2c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f814a880 0x7f44f80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44ff2c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f814acf0 0x7f44f80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44ff2c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8157b20 con 0x7f44f814acf0 2023-10-06T05:17:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44ff2c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f81599c0 con 0x7f44f814a880 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44ff2c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8159ca0 con 0x7f44f80c93c0 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f814acf0 0x7f44f80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f814acf0 0x7f44f80ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50832/0 (socket says 172.21.15.45:50832) 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 -- 172.21.15.45:0/3204941199 learned_addr learned my addr 172.21.15.45:0/3204941199 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd060700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fc85f700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f814a880 0x7f44f80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 -- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 msgr2=0x7f44f8157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 -- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f814a880 msgr2=0x7f44f80cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f814a880 0x7f44f80cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd861700 1 -- 172.21.15.45:0/3204941199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f81574d0 con 0x7f44f814acf0 2023-10-06T05:17:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fd060700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.771+0000 7f44fc85f700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f814a880 0x7f44f80cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.772+0000 7f44fd861700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f814acf0 0x7f44f80ccbf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f44f80cbe00 tx=0x7f44f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.772+0000 7f44e97fa700 1 -- 172.21.15.45:0/3204941199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f000e040 con 0x7f44f814acf0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.772+0000 7f44e97fa700 1 -- 172.21.15.45:0/3204941199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f000f040 con 0x7f44f814acf0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.772+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f81577b0 con 0x7f44f814acf0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.772+0000 7f44e97fa700 1 -- 172.21.15.45:0/3204941199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0015660 con 0x7f44f814acf0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.772+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f8158cd0 con 0x7f44f814acf0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.773+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f8157cd0 con 0x7f44f814acf0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.773+0000 7f44e97fa700 1 -- 172.21.15.45:0/3204941199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44f0015a10 con 0x7f44f814acf0 2023-10-06T05:17:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.774+0000 7f44e97fa700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44dc066f30 0x7f44dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.774+0000 7f44e97fa700 1 -- 172.21.15.45:0/3204941199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f44f0085230 con 0x7f44f814acf0 2023-10-06T05:17:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.774+0000 7f44fd060700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44dc066f30 0x7f44dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.776+0000 7f44fd060700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44dc066f30 0x7f44dc0693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f44ec000e10 tx=0x7f44ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:49.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:49.779+0000 7f44e97fa700 1 -- 172.21.15.45:0/3204941199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44f8157cd0 con 0x7f44f814acf0 2023-10-06T05:17:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.028+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44f8157940 con 0x7f44f814acf0 2023-10-06T05:17:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.028+0000 7f44e97fa700 1 -- 172.21.15.45:0/3204941199 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f44f00521b0 con 0x7f44f814acf0 2023-10-06T05:17:50.031 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:50.031 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:49.609783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5779},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5779":{"gid":5779,"name":"c","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/3967110338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3967110338},{"type":"v1","addr":"172.21.15.45:6835","nonce":3967110338}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.030+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44dc066f30 msgr2=0x7f44dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.030+0000 7f44ff2c4700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44dc066f30 0x7f44dc0693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f44ec000e10 tx=0x7f44ec009040 comp rx=0 tx=0).stop 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.030+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f814acf0 msgr2=0x7f44f80ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.030+0000 7f44ff2c4700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f814acf0 0x7f44f80ccbf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f44f80cbe00 tx=0x7f44f000b040 comp rx=0 tx=0).stop 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 shutdown_connections 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44dc066f30 0x7f44dc0693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f814acf0 0x7f44f80ccbf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f80c93c0 0x7f44f8157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 --2- 172.21.15.45:0/3204941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f814a880 0x7f44f80cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 >> 172.21.15.45:0/3204941199 conn(0x7f44f80cdf50 msgr2=0x7f44f80afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 shutdown_connections 2023-10-06T05:17:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.031+0000 7f44ff2c4700 1 -- 172.21.15.45:0/3204941199 wait complete. 2023-10-06T05:17:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 37 2023-10-06T05:17:50.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2023-10-06T05:17:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.179+0000 7facaa132700 1 Processor -- start 2023-10-06T05:17:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.179+0000 7facaa132700 1 -- start start 2023-10-06T05:17:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7facaa132700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faca41406e0 0x7faca406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7facaa132700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faca406cd60 0x7faca40acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7facaa132700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca40c9940 con 0x7faca40ad4e0 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7facaa132700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca406d230 con 0x7faca41406e0 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7facaa132700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca406d510 con 0x7faca406cd60 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faca41406e0 0x7faca406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faca41406e0 0x7faca406c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50836/0 (socket says 172.21.15.45:50836) 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca37fe700 1 -- 172.21.15.45:0/280059195 learned_addr learned my addr 172.21.15.45:0/280059195 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca2ffd700 1 --2- 172.21.15.45:0/280059195 >> v2:172.21.15.144:3300/0 conn(0x7faca406cd60 0x7faca40acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca37fe700 1 -- 172.21.15.45:0/280059195 >> v2:172.21.15.144:3300/0 conn(0x7faca406cd60 msgr2=0x7faca40acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca37fe700 1 --2- 172.21.15.45:0/280059195 >> v2:172.21.15.144:3300/0 conn(0x7faca406cd60 0x7faca40acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca37fe700 1 -- 172.21.15.45:0/280059195 >> v1:172.21.15.144:6789/0 conn(0x7faca40ad4e0 legacy=0x7faca40c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.180+0000 7faca37fe700 1 -- 172.21.15.45:0/280059195 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca406d7f0 con 0x7faca41406e0 2023-10-06T05:17:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7faca37fe700 1 --2- 172.21.15.45:0/280059195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca41406e0 0x7faca406c820 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faca406ed80 tx=0x7fac94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2ed5cf7af56b4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7faca27fc700 1 -- 172.21.15.45:0/280059195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400b860 con 0x7faca41406e0 2023-10-06T05:17:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7faca27fc700 1 -- 172.21.15.45:0/280059195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9400b9c0 con 0x7faca41406e0 2023-10-06T05:17:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7faca27fc700 1 -- 172.21.15.45:0/280059195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400bd80 con 0x7faca41406e0 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 -- 172.21.15.45:0/280059195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca41406e0 msgr2=0x7faca406c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 --2- 172.21.15.45:0/280059195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca41406e0 0x7faca406c820 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faca406ed80 tx=0x7fac94009250 comp rx=0 tx=0).stop 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 -- 172.21.15.45:0/280059195 shutdown_connections 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 --2- 172.21.15.45:0/280059195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca41406e0 0x7faca406c820 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 --2- 172.21.15.45:0/280059195 >> v2:172.21.15.144:3300/0 conn(0x7faca406cd60 0x7faca40acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 -- 172.21.15.45:0/280059195 >> 172.21.15.45:0/280059195 conn(0x7faca40691c0 msgr2=0x7faca40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 -- 172.21.15.45:0/280059195 shutdown_connections 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.181+0000 7facaa132700 1 -- 172.21.15.45:0/280059195 wait complete. 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.183+0000 7facaa132700 1 Processor -- start 2023-10-06T05:17:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.183+0000 7facaa132700 1 -- start start 2023-10-06T05:17:50.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faca406cd60 0x7faca4156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40ad4e0 0x7faca4158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca41406e0 0x7faca415c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4158570 con 0x7faca40ad4e0 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4157710 con 0x7faca41406e0 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca4157960 con 0x7faca406cd60 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40ad4e0 0x7faca4158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40ad4e0 0x7faca4158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50840/0 (socket says 172.21.15.45:50840) 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 -- 172.21.15.45:0/3732379639 learned_addr learned my addr 172.21.15.45:0/3732379639 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca3fff700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca41406e0 0x7faca415c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca37fe700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faca406cd60 0x7faca4156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 -- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faca406cd60 msgr2=0x7faca4156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faca406cd60 0x7faca4156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 -- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca41406e0 msgr2=0x7faca415c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca41406e0 0x7faca415c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 -- 172.21.15.45:0/3732379639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac940094f0 con 0x7faca40ad4e0 2023-10-06T05:17:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7faca2ffd700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40ad4e0 0x7faca4158a00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fac9800d860 tx=0x7fac98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7fac8b7fe700 1 -- 172.21.15.45:0/3732379639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980125f0 con 0x7faca40ad4e0 2023-10-06T05:17:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7fac8b7fe700 1 -- 172.21.15.45:0/3732379639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98012750 con 0x7faca40ad4e0 2023-10-06T05:17:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca4157c40 con 0x7faca40ad4e0 2023-10-06T05:17:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7fac8b7fe700 1 -- 172.21.15.45:0/3732379639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98012ad0 con 0x7faca40ad4e0 2023-10-06T05:17:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.184+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4158200 con 0x7faca40ad4e0 2023-10-06T05:17:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.186+0000 7fac8b7fe700 1 -- 172.21.15.45:0/3732379639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac9801e460 con 0x7faca40ad4e0 2023-10-06T05:17:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.186+0000 7fac8b7fe700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac90066f30 0x7fac900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.187+0000 7fac8b7fe700 1 -- 172.21.15.45:0/3732379639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fac98084f60 con 0x7faca40ad4e0 2023-10-06T05:17:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.187+0000 7faca37fe700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac90066f30 0x7fac900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.187+0000 7faca37fe700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac90066f30 0x7fac900693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faca406ed80 tx=0x7fac94017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.189+0000 7facb3446b80 1 Processor -- start 2023-10-06T05:17:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.189+0000 7facb3446b80 1 -- start start 2023-10-06T05:17:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07245870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7facb3446b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0724dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0724e520 0x564f0724e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7facb3446b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f07118360 con 0x564f0722af90 2023-10-06T05:17:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f07155180 con 0x564f0724e520 2023-10-06T05:17:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f07152b40 con 0x564f072454a0 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0724dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0724dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50842/0 (socket says 172.21.15.45:50842) 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 -- 172.21.15.45:0/3322271628 learned_addr learned my addr 172.21.15.45:0/3322271628 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca3fff700 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0724e520 0x564f0724e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca37fe700 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07245870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 -- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 msgr2=0x564f07245870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07245870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 -- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0724e520 msgr2=0x564f0724e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0724e520 0x564f0724e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 -- 172.21.15.45:0/3322271628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f06f547e0 con 0x564f0722af90 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7faca2ffd700 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0724dfe0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac98052020 tx=0x7fac98083070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d23d372e190b070 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7fac88ff9700 1 -- 172.21.15.45:0/3322271628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98055480 con 0x564f0722af90 2023-10-06T05:17:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7fac88ff9700 1 -- 172.21.15.45:0/3322271628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9806d070 con 0x564f0722af90 2023-10-06T05:17:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.190+0000 7fac88ff9700 1 -- 172.21.15.45:0/3322271628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9805cac0 con 0x564f0722af90 2023-10-06T05:17:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 -- 172.21.15.45:0/3322271628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 msgr2=0x564f0724dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0724dfe0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac98052020 tx=0x7fac98083070 comp rx=0 tx=0).stop 2023-10-06T05:17:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 -- 172.21.15.45:0/3322271628 shutdown_connections 2023-10-06T05:17:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0724e520 0x564f0724e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07245870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 --2- 172.21.15.45:0/3322271628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0724dfe0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 -- 172.21.15.45:0/3322271628 >> 172.21.15.45:0/3322271628 conn(0x564f072488d0 msgr2=0x564f07248ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 -- 172.21.15.45:0/3322271628 shutdown_connections 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.191+0000 7facb3446b80 1 -- 172.21.15.45:0/3322271628 wait complete. 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.192+0000 7facb3446b80 1 -- start start 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.192+0000 7facb3446b80 1 Processor -- start 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f072205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07225520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7facb3446b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0724e520 0x564f07225a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7facb3446b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0714e930 con 0x564f0724e520 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f070fa9d0 con 0x564f0722af90 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f070fa670 con 0x564f072454a0 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7facb3446b80 10 client.0 _ll_register_callbacks cb 0x564f06f5da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0724e520 0x564f07225a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0724e520 0x564f07225a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50858/0 (socket says 172.21.15.45:50858) 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 -- 172.21.15.45:0/774652418 learned_addr learned my addr 172.21.15.45:0/774652418 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca37fe700 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f072205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca2ffd700 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07225520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 -- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 msgr2=0x564f07225520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07225520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 -- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 msgr2=0x564f072205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f072205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.193+0000 7faca3fff700 1 -- 172.21.15.45:0/774652418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4158200 con 0x564f0724e520 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7faca3fff700 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0724e520 0x564f07225a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fac9c011730 tx=0x7fac9c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f311d5086340299 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7fac81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7fac81ffb700 1 -- 172.21.15.45:0/774652418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0032e0 con 0x564f0724e520 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7fac81ffb700 1 -- 172.21.15.45:0/774652418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fac9c0014a0 con 0x564f0724e520 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f071f2db0 con 0x564f0724e520 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7fac81ffb700 1 -- 172.21.15.45:0/774652418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac9c001870 con 0x564f0724e520 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 10 client.5934 fetch_fsmap learned FSMap version 37 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x564f070410f0 con 0x564f0724e520 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7fac81ffb700 1 -- 172.21.15.45:0/774652418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f071f2db0 con 0x564f0724e520 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7fac81ffb700 1 -- 172.21.15.45:0/774652418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fac9c0019d0 con 0x564f0724e520 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 10 client.5934 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 10 client.5934 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 20 client.5934 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0722b3a0 0x564f07247910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 4 client.5934 mds_command: new command op to 5779 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 --> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f071cfb80 con 0x564f0722b3a0 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.194+0000 7faca37fe700 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0722b3a0 0x564f07247910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.195+0000 7fac81ffb700 10 client.5934 ms_handle_connect on v2:172.21.15.45:6833/3967110338 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.195+0000 7faca37fe700 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0722b3a0 0x564f07247910 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f244f3f3511f9e server_cookie=ebd6f553abb068af in_seq=0 out_seq=0 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.195+0000 7fac81ffb700 1 -- 172.21.15.45:0/774652418 <== mds.0 v2:172.21.15.45:6833/3967110338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564f071cfb80 con 0x564f0722b3a0 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.195+0000 7fac81ffb700 10 client.5934 handle_command_reply: tid=0 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.195+0000 7facb3446b80 1 client.5934 shutdown 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0724e520 msgr2=0x564f07225a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0724e520 0x564f07225a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fac9c011730 tx=0x7fac9c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 shutdown_connections 2023-10-06T05:17:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0722b3a0 0x564f07247910 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0724e520 0x564f07225a60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7fac81ffb700 0 client.5934 ms_handle_reset on v2:172.21.15.45:6833/3967110338 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f072205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 --2- 172.21.15.45:0/774652418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f07225520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 >> 172.21.15.45:0/774652418 conn(0x564f072488d0 msgr2=0x564f07201470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 shutdown_connections 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 1 -- 172.21.15.45:0/774652418 wait complete. 2023-10-06T05:17:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.196+0000 7facb3446b80 20 client.5934 trim_cache size 0 max 16384 2023-10-06T05:17:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.211+0000 7facb3446b80 1 Processor -- start 2023-10-06T05:17:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.211+0000 7facb3446b80 1 -- start start 2023-10-06T05:17:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.211+0000 7facb3446b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f072488d0 0x564f07201620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f071f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f071ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7facb3446b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f06fe6230 con 0x564f072488d0 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f071f6c70 con 0x564f0722af90 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f07200f00 con 0x564f072454a0 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f072488d0 0x564f07201620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f072488d0 0x564f07201620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50866/0 (socket says 172.21.15.45:50866) 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 -- 172.21.15.45:0/244866716 learned_addr learned my addr 172.21.15.45:0/244866716 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca2ffd700 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f071f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca3fff700 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f071ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 -- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 msgr2=0x564f071ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f071ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 -- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 msgr2=0x564f071f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f071f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 -- 172.21.15.45:0/244866716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f071ff350 con 0x564f072488d0 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca2ffd700 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f071f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7faca37fe700 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f072488d0 0x564f07201620 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fac940030c0 tx=0x7fac9402a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ca12ae2bfa1cd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7fac897fa700 1 -- 172.21.15.45:0/244866716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9402d040 con 0x564f072488d0 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7fac897fa700 1 -- 172.21.15.45:0/244866716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9402e040 con 0x564f072488d0 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.212+0000 7fac897fa700 1 -- 172.21.15.45:0/244866716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94021070 con 0x564f072488d0 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 -- 172.21.15.45:0/244866716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f072488d0 msgr2=0x564f07201620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f072488d0 0x564f07201620 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fac940030c0 tx=0x7fac9402a040 comp rx=0 tx=0).stop 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 -- 172.21.15.45:0/244866716 shutdown_connections 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f071ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f072488d0 0x564f07201620 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 --2- 172.21.15.45:0/244866716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0722af90 0x564f071f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 -- 172.21.15.45:0/244866716 >> 172.21.15.45:0/244866716 conn(0x564f07246230 msgr2=0x564f07222070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 -- 172.21.15.45:0/244866716 shutdown_connections 2023-10-06T05:17:50.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.213+0000 7facb3446b80 1 -- 172.21.15.45:0/244866716 wait complete. 2023-10-06T05:17:50.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 -- start start 2023-10-06T05:17:50.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 Processor -- start 2023-10-06T05:17:50.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0729d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f0722e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f072488d0 0x564f071fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f072a2020 con 0x564f0722af90 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f071fae30 con 0x564f072488d0 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f07224900 con 0x564f072454a0 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.214+0000 7facb3446b80 10 client.0 _ll_register_callbacks cb 0x564f06f5da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0729d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0729d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50878/0 (socket says 172.21.15.45:50878) 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 -- 172.21.15.45:0/2228168471 learned_addr learned my addr 172.21.15.45:0/2228168471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca2ffd700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f0722e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca3fff700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f072488d0 0x564f071fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 -- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 msgr2=0x564f0722e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f0722e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 -- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f072488d0 msgr2=0x564f071fa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f072488d0 0x564f071fa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 -- 172.21.15.45:0/2228168471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca406d7f0 con 0x564f0722af90 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca2ffd700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f0722e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca3fff700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f072488d0 0x564f071fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7faca37fe700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0729d2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac94000d60 tx=0x7fac9401c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d555798067e6944 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7fac81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7fac81ffb700 1 -- 172.21.15.45:0/2228168471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9402d070 con 0x564f0722af90 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7fac81ffb700 1 -- 172.21.15.45:0/2228168471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fac94036730 con 0x564f0722af90 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac9806d070 con 0x564f0722af90 2023-10-06T05:17:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7fac81ffb700 1 -- 172.21.15.45:0/2228168471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac94021040 con 0x564f0722af90 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7facb3446b80 10 client.5940 fetch_fsmap learned FSMap version 37 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.215+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x7faca40c70c0 con 0x564f0722af90 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7fac81ffb700 1 -- 172.21.15.45:0/2228168471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9806d070 con 0x564f0722af90 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7fac81ffb700 1 -- 172.21.15.45:0/2228168471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fac9403d020 con 0x564f0722af90 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7facb3446b80 10 client.5940 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7facb3446b80 10 client.5940 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7facb3446b80 20 client.5940 populate_metadata read hostname 'smithi045' 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7facb3446b80 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0724e520 0x564f07224ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7facb3446b80 4 client.5940 mds_command: new command op to 5779 tid=0[{"prefix": "flush journal"}] 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 --> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f070f1920 con 0x564f0724e520 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7faca2ffd700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0724e520 0x564f07224ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7faca2ffd700 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0724e520 0x564f07224ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb5f140070f4171 server_cookie=ae0683ffa372f4f in_seq=0 out_seq=0 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7fac81ffb700 10 client.5940 ms_handle_connect on v2:172.21.15.45:6833/3967110338 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7fac81ffb700 1 -- 172.21.15.45:0/2228168471 <== mds.0 v2:172.21.15.45:6833/3967110338 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f070f1920 con 0x564f0724e520 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7fac81ffb700 10 client.5940 handle_command_reply: tid=0 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.216+0000 7facb3446b80 1 client.5940 shutdown 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 msgr2=0x564f0729d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0729d2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac94000d60 tx=0x7fac9401c070 comp rx=0 tx=0).stop 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 shutdown_connections 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f072488d0 0x564f071fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:6833/3967110338,v1:172.21.15.45:6835/3967110338] conn(0x564f0724e520 0x564f07224ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564f0722af90 0x564f0729d2f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 --2- 172.21.15.45:0/2228168471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f072454a0 0x564f0722e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7fac81ffb700 0 client.5940 ms_handle_reset on v2:172.21.15.45:6833/3967110338 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 >> 172.21.15.45:0/2228168471 conn(0x564f07246230 msgr2=0x564f071f4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 shutdown_connections 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 1 -- 172.21.15.45:0/2228168471 wait complete. 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facb3446b80 20 client.5940 trim_cache size 0 max 16384 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2023-10-06T05:17:50.227 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:17:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac90066f30 msgr2=0x7fac900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac90066f30 0x7fac900693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faca406ed80 tx=0x7fac94017000 comp rx=0 tx=0).stop 2023-10-06T05:17:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40ad4e0 msgr2=0x7faca4158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40ad4e0 0x7faca4158a00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fac9800d860 tx=0x7fac98008040 comp rx=0 tx=0).stop 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 shutdown_connections 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac90066f30 0x7fac900693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faca41406e0 0x7faca415c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faca406cd60 0x7faca4156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 --2- 172.21.15.45:0/3732379639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faca40ad4e0 0x7faca4158a00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 >> 172.21.15.45:0/3732379639 conn(0x7faca40691c0 msgr2=0x7faca413f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 shutdown_connections 2023-10-06T05:17:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.217+0000 7facaa132700 1 -- 172.21.15.45:0/3732379639 wait complete. 2023-10-06T05:17:50.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:50.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f340dd17700 1 Processor -- start 2023-10-06T05:17:50.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f340dd17700 1 -- start start 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f340dd17700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f340815a970 0x7f340806a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f340dd17700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34080aed10 con 0x7f34080cc3c0 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f340dd17700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34080a8ac0 con 0x7f3408153720 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f340dd17700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340815ad80 con 0x7f340815a970 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f3407fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f340815a970 0x7f340806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f3407fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f340815a970 0x7f340806a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50882/0 (socket says 172.21.15.45:50882) 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f3407fff700 1 -- 172.21.15.45:0/2276093718 learned_addr learned my addr 172.21.15.45:0/2276093718 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.364+0000 7f3406ffd700 1 --1- 172.21.15.45:0/2276093718 >> v1:172.21.15.45:6789/0 conn(0x7f34080cc3c0 0x7f340806a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41188/0 (socket says 172.21.15.45:41188) 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f3407fff700 1 -- 172.21.15.45:0/2276093718 >> v1:172.21.15.144:6789/0 conn(0x7f3408153720 legacy=0x7f34080cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f3407fff700 1 -- 172.21.15.45:0/2276093718 >> v1:172.21.15.45:6789/0 conn(0x7f34080cc3c0 legacy=0x7f340806a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f3407fff700 1 -- 172.21.15.45:0/2276093718 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340806af80 con 0x7f340815a970 2023-10-06T05:17:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f34067fc700 1 -- 172.21.15.45:0/2276093718 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981055989 0 0) 0x7f34080aed10 con 0x7f34080cc3c0 2023-10-06T05:17:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f3407fff700 1 --2- 172.21.15.45:0/2276093718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f340815a970 0x7f340806a920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3400009f20 tx=0x7f3400009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d84211ca524cfe8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f34067fc700 1 -- 172.21.15.45:0/2276093718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340000e070 con 0x7f340815a970 2023-10-06T05:17:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f34067fc700 1 -- 172.21.15.45:0/2276093718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340000bb10 con 0x7f340815a970 2023-10-06T05:17:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.365+0000 7f34067fc700 1 -- 172.21.15.45:0/2276093718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000092e0 con 0x7f340815a970 2023-10-06T05:17:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.366+0000 7f340dd17700 1 -- 172.21.15.45:0/2276093718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f340815a970 msgr2=0x7f340806a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.366+0000 7f340dd17700 1 --2- 172.21.15.45:0/2276093718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f340815a970 0x7f340806a920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3400009f20 tx=0x7f3400009840 comp rx=0 tx=0).stop 2023-10-06T05:17:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.366+0000 7f340dd17700 1 -- 172.21.15.45:0/2276093718 shutdown_connections 2023-10-06T05:17:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.366+0000 7f340dd17700 1 --2- 172.21.15.45:0/2276093718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f340815a970 0x7f340806a920 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.366+0000 7f340dd17700 1 -- 172.21.15.45:0/2276093718 >> 172.21.15.45:0/2276093718 conn(0x7f3408067620 msgr2=0x7f3408067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.366+0000 7f340dd17700 1 -- 172.21.15.45:0/2276093718 shutdown_connections 2023-10-06T05:17:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.366+0000 7f340dd17700 1 -- 172.21.15.45:0/2276093718 wait complete. 2023-10-06T05:17:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.367+0000 7f340dd17700 1 Processor -- start 2023-10-06T05:17:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.367+0000 7f340dd17700 1 -- start start 2023-10-06T05:17:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f340dd17700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34080cc3c0 0x7f340814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f340dd17700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3408153720 0x7f34080ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f340dd17700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f340815a970 0x7f34080ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f340dd17700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34080ac4d0 con 0x7f34080cc3c0 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f340dd17700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34081500f0 con 0x7f340815a970 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f340dd17700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34080cae10 con 0x7f3408153720 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34080cc3c0 0x7f340814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34080cc3c0 0x7f340814fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50894/0 (socket says 172.21.15.45:50894) 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 -- 172.21.15.45:0/1292461244 learned_addr learned my addr 172.21.15.45:0/1292461244 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f3406ffd700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3408153720 0x7f34080ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f3407fff700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f340815a970 0x7f34080ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 -- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3408153720 msgr2=0x7f34080ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3408153720 0x7f34080ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 -- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f340815a970 msgr2=0x7f34080ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f340815a970 0x7f34080ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 -- 172.21.15.45:0/1292461244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400009ae0 con 0x7f34080cc3c0 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.368+0000 7f34077fe700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34080cc3c0 0x7f340814fbb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c590 tx=0x7f33f800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.369+0000 7f33ff7fe700 1 -- 172.21.15.45:0/1292461244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8011590 con 0x7f34080cc3c0 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.369+0000 7f33ff7fe700 1 -- 172.21.15.45:0/1292461244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f80116f0 con 0x7f34080cc3c0 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.369+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34080ab8c0 con 0x7f34080cc3c0 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.369+0000 7f33ff7fe700 1 -- 172.21.15.45:0/1292461244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8011ab0 con 0x7f34080cc3c0 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.369+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34080abe80 con 0x7f34080cc3c0 2023-10-06T05:17:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.370+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340806cec0 con 0x7f34080cc3c0 2023-10-06T05:17:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.371+0000 7f33ff7fe700 1 -- 172.21.15.45:0/1292461244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33f801b3b0 con 0x7f34080cc3c0 2023-10-06T05:17:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.371+0000 7f33ff7fe700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f33e8066f30 0x7f33e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.371+0000 7f33ff7fe700 1 -- 172.21.15.45:0/1292461244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f33f80841f0 con 0x7f34080cc3c0 2023-10-06T05:17:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.371+0000 7f3406ffd700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f33e8066f30 0x7f33e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.372+0000 7f3406ffd700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f33e8066f30 0x7f33e80693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f33f0007fd0 tx=0x7f33f0007db0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.376+0000 7f33ff7fe700 1 -- 172.21.15.45:0/1292461244 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33f80511f0 con 0x7f34080cc3c0 2023-10-06T05:17:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.629+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f340804bb40 con 0x7f34080cc3c0 2023-10-06T05:17:50.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.630+0000 7f33ff7fe700 1 -- 172.21.15.45:0/1292461244 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f33f8017090 con 0x7f34080cc3c0 2023-10-06T05:17:50.632 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:50.632 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:50.611056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5779},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5779":{"gid":5779,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.45:6835/3967110338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3967110338},{"type":"v1","addr":"172.21.15.45:6835","nonce":3967110338}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:50.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f33e8066f30 msgr2=0x7f33e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f33e8066f30 0x7f33e80693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f33f0007fd0 tx=0x7f33f0007db0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34080cc3c0 msgr2=0x7f340814fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34080cc3c0 0x7f340814fbb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c590 tx=0x7f33f800c950 comp rx=0 tx=0).stop 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 shutdown_connections 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f33e8066f30 0x7f33e80693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f340815a970 0x7f34080ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34080cc3c0 0x7f340814fbb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 --2- 172.21.15.45:0/1292461244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3408153720 0x7f34080ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 >> 172.21.15.45:0/1292461244 conn(0x7f3408067620 msgr2=0x7f34081526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 shutdown_connections 2023-10-06T05:17:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.632+0000 7f340dd17700 1 -- 172.21.15.45:0/1292461244 wait complete. 2023-10-06T05:17:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 38 2023-10-06T05:17:50.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:50.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.777+0000 7f6f3bd07700 1 Processor -- start 2023-10-06T05:17:50.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.778+0000 7f6f3bd07700 1 -- start start 2023-10-06T05:17:50.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.778+0000 7f6f3bd07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f34155cd0 0x7f6f3415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.778+0000 7f6f3bd07700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f340ccdc0 0x7f6f340cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.778+0000 7f6f3bd07700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f340cd740 con 0x7f6f341560a0 2023-10-06T05:17:50.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.778+0000 7f6f3bd07700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f340ae6b0 con 0x7f6f340ccdc0 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.778+0000 7f6f3bd07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340ae8d0 con 0x7f6f34155cd0 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f340ccdc0 0x7f6f340cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f340ccdc0 0x7f6f340cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50902/0 (socket says 172.21.15.45:50902) 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f392a2700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6f341560a0 0x7f6f340adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41190/0 (socket says 172.21.15.45:41190) 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 -- 172.21.15.45:0/1049710315 learned_addr learned my addr 172.21.15.45:0/1049710315 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f39aa3700 1 --2- 172.21.15.45:0/1049710315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f34155cd0 0x7f6f3415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 -- 172.21.15.45:0/1049710315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f34155cd0 msgr2=0x7f6f3415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 --2- 172.21.15.45:0/1049710315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f34155cd0 0x7f6f3415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 -- 172.21.15.45:0/1049710315 >> v1:172.21.15.45:6789/0 conn(0x7f6f341560a0 legacy=0x7f6f340adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 -- 172.21.15.45:0/1049710315 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f340aebb0 con 0x7f6f340ccdc0 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f38aa1700 1 -- 172.21.15.45:0/1049710315 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178859413 0 0) 0x7f6f340cd740 con 0x7f6f341560a0 2023-10-06T05:17:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f3a2a4700 1 --2- 172.21.15.45:0/1049710315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f340ccdc0 0x7f6f340cd200 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00d1a0 tx=0x7f6f2c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335bc852500ad754 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f38aa1700 1 -- 172.21.15.45:0/1049710315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c010070 con 0x7f6f340ccdc0 2023-10-06T05:17:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f38aa1700 1 -- 172.21.15.45:0/1049710315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c00b0d0 con 0x7f6f340ccdc0 2023-10-06T05:17:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.779+0000 7f6f38aa1700 1 -- 172.21.15.45:0/1049710315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00b490 con 0x7f6f340ccdc0 2023-10-06T05:17:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 -- 172.21.15.45:0/1049710315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f340ccdc0 msgr2=0x7f6f340cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/1049710315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f340ccdc0 0x7f6f340cd200 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00d1a0 tx=0x7f6f2c00ba50 comp rx=0 tx=0).stop 2023-10-06T05:17:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 -- 172.21.15.45:0/1049710315 shutdown_connections 2023-10-06T05:17:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/1049710315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f340ccdc0 0x7f6f340cd200 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/1049710315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f34155cd0 0x7f6f3415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 -- 172.21.15.45:0/1049710315 >> 172.21.15.45:0/1049710315 conn(0x7f6f3406f090 msgr2=0x7f6f3406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 -- 172.21.15.45:0/1049710315 shutdown_connections 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.780+0000 7f6f3bd07700 1 -- 172.21.15.45:0/1049710315 wait complete. 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.781+0000 7f6f3bd07700 1 Processor -- start 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.781+0000 7f6f3bd07700 1 -- start start 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f3bd07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f340ccdc0 0x7f6f34149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f3bd07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f34155cd0 0x7f6f340b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f3bd07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f341560a0 0x7f6f340685f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f3bd07700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f341498d0 con 0x7f6f34155cd0 2023-10-06T05:17:50.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f3bd07700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340c3e40 con 0x7f6f340ccdc0 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f3bd07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340c4400 con 0x7f6f341560a0 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f392a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f34155cd0 0x7f6f340b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f392a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f34155cd0 0x7f6f340b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50916/0 (socket says 172.21.15.45:50916) 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f392a2700 1 -- 172.21.15.45:0/2670165902 learned_addr learned my addr 172.21.15.45:0/2670165902 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f3a2a4700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f341560a0 0x7f6f340685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.782+0000 7f6f39aa3700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f340ccdc0 0x7f6f34149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f392a2700 1 -- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f341560a0 msgr2=0x7f6f340685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f392a2700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f341560a0 0x7f6f340685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f392a2700 1 -- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f340ccdc0 msgr2=0x7f6f34149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f392a2700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f340ccdc0 0x7f6f34149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f392a2700 1 -- 172.21.15.45:0/2670165902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c00bcf0 con 0x7f6f34155cd0 2023-10-06T05:17:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f392a2700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f34155cd0 0x7f6f340b1b10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c5e0 tx=0x7f6f24001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f29ffb700 1 -- 172.21.15.45:0/2670165902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24005a70 con 0x7f6f34155cd0 2023-10-06T05:17:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f29ffb700 1 -- 172.21.15.45:0/2670165902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f24005bd0 con 0x7f6f34155cd0 2023-10-06T05:17:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f340b2560 con 0x7f6f34155cd0 2023-10-06T05:17:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f29ffb700 1 -- 172.21.15.45:0/2670165902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f240070d0 con 0x7f6f34155cd0 2023-10-06T05:17:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.783+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f340c4090 con 0x7f6f34155cd0 2023-10-06T05:17:50.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.784+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f340c4590 con 0x7f6f34155cd0 2023-10-06T05:17:50.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.785+0000 7f6f29ffb700 1 -- 172.21.15.45:0/2670165902 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f24007480 con 0x7f6f34155cd0 2023-10-06T05:17:50.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.785+0000 7f6f29ffb700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f18066fb0 0x7f6f18069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.785+0000 7f6f29ffb700 1 -- 172.21.15.45:0/2670165902 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f6f24083cf0 con 0x7f6f34155cd0 2023-10-06T05:17:50.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.785+0000 7f6f39aa3700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f18066fb0 0x7f6f18069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.786+0000 7f6f39aa3700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f18066fb0 0x7f6f18069460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006fd0 tx=0x7f6f30008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:50.790+0000 7f6f29ffb700 1 -- 172.21.15.45:0/2670165902 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f24050bf0 con 0x7f6f34155cd0 2023-10-06T05:17:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.043+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f340aee90 con 0x7f6f34155cd0 2023-10-06T05:17:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.044+0000 7f6f29ffb700 1 -- 172.21.15.45:0/2670165902 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f6f24017070 con 0x7f6f34155cd0 2023-10-06T05:17:51.046 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:51.046 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:50.611056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5779},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5779":{"gid":5779,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.45:6835/3967110338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3967110338},{"type":"v1","addr":"172.21.15.45:6835","nonce":3967110338}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f18066fb0 msgr2=0x7f6f18069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f18066fb0 0x7f6f18069460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006fd0 tx=0x7f6f30008040 comp rx=0 tx=0).stop 2023-10-06T05:17:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f34155cd0 msgr2=0x7f6f340b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f34155cd0 0x7f6f340b1b10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c5e0 tx=0x7f6f24001040 comp rx=0 tx=0).stop 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 shutdown_connections 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f18066fb0 0x7f6f18069460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f341560a0 0x7f6f340685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f340ccdc0 0x7f6f34149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 --2- 172.21.15.45:0/2670165902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f34155cd0 0x7f6f340b1b10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 >> 172.21.15.45:0/2670165902 conn(0x7f6f3406f090 msgr2=0x7f6f340caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 shutdown_connections 2023-10-06T05:17:51.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.046+0000 7f6f3bd07700 1 -- 172.21.15.45:0/2670165902 wait complete. 2023-10-06T05:17:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 38 2023-10-06T05:17:51.061 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /a 0 2023-10-06T05:17:51.211 INFO:teuthology.orchestra.run.smithi045.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2023-10-06T05:17:46.475673+0000","btime":"2023-10-06T05:17:40.477885+0000","mode":16893,"uid":1000,"gid":1285,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2023-10-06T05:17:46.475673+0000","atime":"2023-10-06T05:17:40.477885+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2023-10-06T05:17:46.475673+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-10-06T05:17:46.475673+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2023-10-06T05:17:46.475673+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirfrag":1,"dirty":1},"nref":2,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}]}] 2023-10-06T05:17:51.211 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2023-10-06T05:17:46.475673+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2023-10-06T05:17:40.477885+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2023-10-06T05:17:40.477885+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2023-10-06T05:17:46.475673+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2023-10-06T05:17:46.475673+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1285, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2023-10-06T05:17:46.475673+0000", "nestlock": {}, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2023-10-06T05:17:46.475673+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2023-10-06T05:17:51.212 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2023-10-06T05:17:46.475673+0000', 'btime': '2023-10-06T05:17:40.477885+0000', 'mode': 16893, 'uid': 1000, 'gid': 1285, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2023-10-06T05:17:46.475673+0000', 'atime': '2023-10-06T05:17:40.477885+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2023-10-06T05:17:46.475673+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-10-06T05:17:46.475673+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2023-10-06T05:17:46.475673+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirfrag': 1, 'dirty': 1}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {}, 'flocklock': {}, 'policylock': {}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}]}] 2023-10-06T05:17:51.212 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:17:51.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:51.259 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:51.305 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:17:51.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:17:51.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:51.305 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:17:51.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:51.327 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:17:51.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:17:51.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:51.328 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:17:51.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:51.349 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T05:17:51.349 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:17:51.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:51.349 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:17:51.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:51.390 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:17:51.391 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:17:51.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:51.391 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:17:51.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:51.411 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:17:51.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:51.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.570+0000 7fa9fe0aa700 1 Processor -- start 2023-10-06T05:17:51.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.570+0000 7fa9fe0aa700 1 -- start start 2023-10-06T05:17:51.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9fe0aa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9f8159200 0x7fa9f80c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9fe0aa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f80ccdc0 0x7fa9f80cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9fe0aa700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80c7d00 con 0x7fa9f81595d0 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9fe0aa700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f80c7780 con 0x7fa9f8159200 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9fe0aa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80cd720 con 0x7fa9f80ccdc0 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9f8159200 0x7fa9f80c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f6ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa9f81595d0 0x7fa9f806b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41194/0 (socket says 172.21.15.45:41194) 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f6ffd700 1 -- 172.21.15.45:0/1089623332 learned_addr learned my addr 172.21.15.45:0/1089623332 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f77fe700 1 --2- 172.21.15.45:0/1089623332 >> v2:172.21.15.45:3300/0 conn(0x7fa9f8159200 0x7fa9f80c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50918/0 (socket says 172.21.15.45:50918) 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f7fff700 1 --2- 172.21.15.45:0/1089623332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f80ccdc0 0x7fa9f80cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f77fe700 1 -- 172.21.15.45:0/1089623332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f80ccdc0 msgr2=0x7fa9f80cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f77fe700 1 --2- 172.21.15.45:0/1089623332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f80ccdc0 0x7fa9f80cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f77fe700 1 -- 172.21.15.45:0/1089623332 >> v1:172.21.15.45:6789/0 conn(0x7fa9f81595d0 legacy=0x7fa9f806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f77fe700 1 -- 172.21.15.45:0/1089623332 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80c86d0 con 0x7fa9f8159200 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f67fc700 1 -- 172.21.15.45:0/1089623332 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568519913 0 0) 0x7fa9f80c7d00 con 0x7fa9f81595d0 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.571+0000 7fa9f7fff700 1 --2- 172.21.15.45:0/1089623332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f80ccdc0 0x7fa9f80cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9f77fe700 1 --2- 172.21.15.45:0/1089623332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f8159200 0x7fa9f80c8190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80098e0 tx=0x7fa9e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba46c96c918b8b20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9f67fc700 1 -- 172.21.15.45:0/1089623332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8009e80 con 0x7fa9f8159200 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9f67fc700 1 -- 172.21.15.45:0/1089623332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e800b860 con 0x7fa9f8159200 2023-10-06T05:17:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9f67fc700 1 -- 172.21.15.45:0/1089623332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800bbe0 con 0x7fa9f8159200 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/1089623332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f8159200 msgr2=0x7fa9f80c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/1089623332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f8159200 0x7fa9f80c8190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80098e0 tx=0x7fa9e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/1089623332 shutdown_connections 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/1089623332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f80ccdc0 0x7fa9f80cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/1089623332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f8159200 0x7fa9f80c8190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/1089623332 >> 172.21.15.45:0/1089623332 conn(0x7fa9f806f090 msgr2=0x7fa9f806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/1089623332 shutdown_connections 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.572+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/1089623332 wait complete. 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.574+0000 7fa9fe0aa700 1 Processor -- start 2023-10-06T05:17:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.574+0000 7fa9fe0aa700 1 -- start start 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9fe0aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f80ccdc0 0x7fa9f81691b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9fe0aa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f8159200 0x7fa9f80c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9fe0aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f81595d0 0x7fa9f80c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9fe0aa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f81696f0 con 0x7fa9f81595d0 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9fe0aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80c5420 con 0x7fa9f80ccdc0 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9fe0aa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80c4750 con 0x7fa9f8159200 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f81595d0 0x7fa9f80c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f81595d0 0x7fa9f80c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50930/0 (socket says 172.21.15.45:50930) 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 -- 172.21.15.45:0/2868769755 learned_addr learned my addr 172.21.15.45:0/2868769755 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f6ffd700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f8159200 0x7fa9f80c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f77fe700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f80ccdc0 0x7fa9f81691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 -- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f8159200 msgr2=0x7fa9f80c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f8159200 0x7fa9f80c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 -- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f80ccdc0 msgr2=0x7fa9f81691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f80ccdc0 0x7fa9f81691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 -- 172.21.15.45:0/2868769755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e8009420 con 0x7fa9f81595d0 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9f7fff700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f81595d0 0x7fa9f80c2910 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000a380 tx=0x7fa9f0007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.575+0000 7fa9df7fe700 1 -- 172.21.15.45:0/2868769755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000ae10 con 0x7fa9f81595d0 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.576+0000 7fa9df7fe700 1 -- 172.21.15.45:0/2868769755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f000d650 con 0x7fa9f81595d0 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.576+0000 7fa9df7fe700 1 -- 172.21.15.45:0/2868769755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000da10 con 0x7fa9f81595d0 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.576+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f80c2f90 con 0x7fa9f81595d0 2023-10-06T05:17:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.576+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f80c50b0 con 0x7fa9f81595d0 2023-10-06T05:17:51.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.577+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f806e2f0 con 0x7fa9f81595d0 2023-10-06T05:17:51.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.577+0000 7fa9df7fe700 1 -- 172.21.15.45:0/2868769755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9f00143a0 con 0x7fa9f81595d0 2023-10-06T05:17:51.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.577+0000 7fa9df7fe700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9e4066f30 0x7fa9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.578+0000 7fa9df7fe700 1 -- 172.21.15.45:0/2868769755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fa9f0085220 con 0x7fa9f81595d0 2023-10-06T05:17:51.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.578+0000 7fa9f77fe700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9e4066f30 0x7fa9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.578+0000 7fa9f77fe700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9e4066f30 0x7fa9e40693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80091e0 tx=0x7fa9e8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.582+0000 7fa9df7fe700 1 -- 172.21.15.45:0/2868769755 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9f00521a0 con 0x7fa9f81595d0 2023-10-06T05:17:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.832+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9f80c5560 con 0x7fa9f81595d0 2023-10-06T05:17:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.833+0000 7fa9df7fe700 1 -- 172.21.15.45:0/2868769755 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4598 (secure 0 0 0) 0x7fa9f00557c0 con 0x7fa9f81595d0 2023-10-06T05:17:51.835 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:51.835 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:51.612236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5779},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5779":{"gid":5779,"name":"c","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.45:6835/3967110338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3967110338},{"type":"v1","addr":"172.21.15.45:6835","nonce":3967110338}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9e4066f30 msgr2=0x7fa9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9e4066f30 0x7fa9e40693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80091e0 tx=0x7fa9e8017000 comp rx=0 tx=0).stop 2023-10-06T05:17:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f81595d0 msgr2=0x7fa9f80c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f81595d0 0x7fa9f80c2910 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000a380 tx=0x7fa9f0007900 comp rx=0 tx=0).stop 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 shutdown_connections 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9e4066f30 0x7fa9e40693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9f81595d0 0x7fa9f80c2910 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f80ccdc0 0x7fa9f81691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 --2- 172.21.15.45:0/2868769755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9f8159200 0x7fa9f80c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 >> 172.21.15.45:0/2868769755 conn(0x7fa9f806f090 msgr2=0x7fa9f8070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 shutdown_connections 2023-10-06T05:17:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.835+0000 7fa9fe0aa700 1 -- 172.21.15.45:0/2868769755 wait complete. 2023-10-06T05:17:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 39 2023-10-06T05:17:51.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.981+0000 7f506bf85700 1 Processor -- start 2023-10-06T05:17:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.982+0000 7f506bf85700 1 -- start start 2023-10-06T05:17:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.982+0000 7f506bf85700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5064159200 0x7f50640c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.982+0000 7f506bf85700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 0x7f50640cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.982+0000 7f506bf85700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50640c7d00 con 0x7f50641595d0 2023-10-06T05:17:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.982+0000 7f506bf85700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50640c7780 con 0x7f5064159200 2023-10-06T05:17:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.982+0000 7f506bf85700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50640cd720 con 0x7f50640ccdc0 2023-10-06T05:17:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5069520700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f50641595d0 0x7f506406b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41204/0 (socket says 172.21.15.45:41204) 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5069520700 1 -- 172.21.15.45:0/2280012875 learned_addr learned my addr 172.21.15.45:0/2280012875 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f506a522700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 0x7f50640cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5069d21700 1 --2- 172.21.15.45:0/2280012875 >> v2:172.21.15.144:3300/0 conn(0x7f5064159200 0x7f50640c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334484681 0 0) 0x7f50640c7d00 con 0x7f50641595d0 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5050003570 con 0x7f50641595d0 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3364054977 0 0) 0x7f5050003570 con 0x7f50641595d0 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 msgr2=0x7f50640cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 0x7f50640cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 >> v2:172.21.15.144:3300/0 conn(0x7f5064159200 msgr2=0x7f50640c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 --2- 172.21.15.45:0/2280012875 >> v2:172.21.15.144:3300/0 conn(0x7f5064159200 0x7f50640c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640c86d0 con 0x7f50641595d0 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f5054001470 con 0x7f50641595d0 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 >> v1:172.21.15.45:6789/0 conn(0x7f50641595d0 legacy=0x7f506406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f5050003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 0x7f5050004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5050004d50 0x7f50500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640c7780 con 0x7f5050003a60 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640cd720 con 0x7f5050004d50 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505000ba60 con 0x7f50640ccdc0 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.983+0000 7f5069d21700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f5050003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f506a522700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5050004d50 0x7f50500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5069520700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 0x7f5050004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5069d21700 1 -- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 msgr2=0x7f5050004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5069d21700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640ccdc0 0x7f5050004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5069d21700 1 -- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5050004d50 msgr2=0x7f50500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5069d21700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5050004d50 0x7f50500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5069d21700 1 -- 172.21.15.45:0/2280012875 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5050003570 con 0x7f5050003a60 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5069d21700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f5050003e70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5060009c00 tx=0x7f50600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6330b3ea984f33c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506000e6b0 con 0x7f5050003a60 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506000e810 con 0x7f5050003a60 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.984+0000 7f5068d1f700 1 -- 172.21.15.45:0/2280012875 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506000ebd0 con 0x7f5050003a60 2023-10-06T05:17:51.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.985+0000 7f506bf85700 1 -- 172.21.15.45:0/2280012875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 msgr2=0x7f5050003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.985+0000 7f506bf85700 1 --2- 172.21.15.45:0/2280012875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f5050003e70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5060009c00 tx=0x7f50600091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.985+0000 7f506a522700 1 -- 172.21.15.45:0/2280012875 reap_dead start 2023-10-06T05:17:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.985+0000 7f506bf85700 1 -- 172.21.15.45:0/2280012875 shutdown_connections 2023-10-06T05:17:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.985+0000 7f506bf85700 1 -- 172.21.15.45:0/2280012875 >> 172.21.15.45:0/2280012875 conn(0x7f506406f090 msgr2=0x7f506406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.985+0000 7f506bf85700 1 -- 172.21.15.45:0/2280012875 shutdown_connections 2023-10-06T05:17:51.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.985+0000 7f506bf85700 1 -- 172.21.15.45:0/2280012875 wait complete. 2023-10-06T05:17:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.987+0000 7f506bf85700 1 Processor -- start 2023-10-06T05:17:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.987+0000 7f506bf85700 1 -- start start 2023-10-06T05:17:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f506bf85700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f506414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f506bf85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50640b1930 0x7f50640c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f506bf85700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640c69d0 0x7f506415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f506bf85700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640b14a0 con 0x7f5050003a60 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f506bf85700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506414eac0 con 0x7f50640b1930 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f506bf85700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50640b05f0 con 0x7f50640c69d0 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f506414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f506414e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50952/0 (socket says 172.21.15.45:50952) 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 -- 172.21.15.45:0/1576170895 learned_addr learned my addr 172.21.15.45:0/1576170895 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f506a522700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640c69d0 0x7f506415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069520700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50640b1930 0x7f50640c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 -- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640c69d0 msgr2=0x7f506415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640c69d0 0x7f506415b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 -- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50640b1930 msgr2=0x7f50640c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50640b1930 0x7f50640c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 -- 172.21.15.45:0/1576170895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640b0890 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069520700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50640b1930 0x7f50640c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5069d21700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f506414e580 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5050004730 tx=0x7f506001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.988+0000 7f5059ffb700 1 -- 172.21.15.45:0/1576170895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060022070 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.989+0000 7f5059ffb700 1 -- 172.21.15.45:0/1576170895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506000e980 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.989+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50640b14a0 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.989+0000 7f5059ffb700 1 -- 172.21.15.45:0/1576170895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060027470 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.989+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50640b0e50 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.990+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50640b1780 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.990+0000 7f5059ffb700 1 -- 172.21.15.45:0/1576170895 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50600278a0 con 0x7f5050003a60 2023-10-06T05:17:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.991+0000 7f5059ffb700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5048066f30 0x7f50480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.991+0000 7f5059ffb700 1 -- 172.21.15.45:0/1576170895 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f50600861f0 con 0x7f5050003a60 2023-10-06T05:17:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.991+0000 7f5069520700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5048066f30 0x7f50480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.991+0000 7f5069520700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5048066f30 0x7f50480693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50540010e0 tx=0x7f5054009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:51.995+0000 7f5059ffb700 1 -- 172.21.15.45:0/1576170895 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5060053220 con 0x7f5050003a60 2023-10-06T05:17:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.196+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f506406bb40 con 0x7f5050003a60 2023-10-06T05:17:52.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.197+0000 7f5059ffb700 1 -- 172.21.15.45:0/1576170895 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17296 (secure 0 0 0) 0x7f5060056840 con 0x7f5050003a60 2023-10-06T05:17:52.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:52.199 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":53,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:17:47.603415+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:29.616613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:30.718338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2812290138":"2023-10-07T05:17:47.603375+0000","172.21.15.45:6833/2812290138":"2023-10-07T05:17:47.603375+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5048066f30 msgr2=0x7f50480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5048066f30 0x7f50480693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50540010e0 tx=0x7f5054009040 comp rx=0 tx=0).stop 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 msgr2=0x7f506414e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f506414e580 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5050004730 tx=0x7f506001a000 comp rx=0 tx=0).stop 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 shutdown_connections 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5048066f30 0x7f50480693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50640c69d0 0x7f506415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5050003a60 0x7f506414e580 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 --2- 172.21.15.45:0/1576170895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50640b1930 0x7f50640c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 >> 172.21.15.45:0/1576170895 conn(0x7f506406f090 msgr2=0x7f50640cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 shutdown_connections 2023-10-06T05:17:52.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.199+0000 7f506bf85700 1 -- 172.21.15.45:0/1576170895 wait complete. 2023-10-06T05:17:52.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:17:29.616613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:30.718338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:52.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.349+0000 7fbc1cd0b700 1 Processor -- start 2023-10-06T05:17:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.349+0000 7fbc1cd0b700 1 -- start start 2023-10-06T05:17:52.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc1cd0b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc1806aa70 0x7fbc180c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc1cd0b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc180c7050 0x7fbc1815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc1cd0b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc18067d80 con 0x7fbc1806ae40 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc1cd0b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc180ac620 con 0x7fbc180c7050 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc1cd0b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc180c75b0 con 0x7fbc1806aa70 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc15d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc180c7050 0x7fbc1815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc15d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc180c7050 0x7fbc1815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50968/0 (socket says 172.21.15.45:50968) 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc15d9b700 1 -- 172.21.15.45:0/4235084701 learned_addr learned my addr 172.21.15.45:0/4235084701 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc1659c700 1 --2- 172.21.15.45:0/4235084701 >> v2:172.21.15.144:3300/0 conn(0x7fbc1806aa70 0x7fbc180c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc15d9b700 1 -- 172.21.15.45:0/4235084701 >> v2:172.21.15.144:3300/0 conn(0x7fbc1806aa70 msgr2=0x7fbc180c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc15d9b700 1 --2- 172.21.15.45:0/4235084701 >> v2:172.21.15.144:3300/0 conn(0x7fbc1806aa70 0x7fbc180c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc15d9b700 1 -- 172.21.15.45:0/4235084701 >> v1:172.21.15.144:6789/0 conn(0x7fbc1806ae40 legacy=0x7fbc1806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.350+0000 7fbc15d9b700 1 -- 172.21.15.45:0/4235084701 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc180c7890 con 0x7fbc180c7050 2023-10-06T05:17:52.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.351+0000 7fbc15d9b700 1 --2- 172.21.15.45:0/4235084701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc1815c1b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbc1806f990 tx=0x7fbc0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bf256b4f6937b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.351+0000 7fbc1559a700 1 -- 172.21.15.45:0/4235084701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00b860 con 0x7fbc180c7050 2023-10-06T05:17:52.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.351+0000 7fbc1559a700 1 -- 172.21.15.45:0/4235084701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c00b9c0 con 0x7fbc180c7050 2023-10-06T05:17:52.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.351+0000 7fbc1559a700 1 -- 172.21.15.45:0/4235084701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00bd60 con 0x7fbc180c7050 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.351+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/4235084701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 msgr2=0x7fbc1815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.351+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/4235084701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc1815c1b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbc1806f990 tx=0x7fbc0c009250 comp rx=0 tx=0).stop 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.352+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/4235084701 shutdown_connections 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.352+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/4235084701 >> v2:172.21.15.144:3300/0 conn(0x7fbc1806aa70 0x7fbc180c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.352+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/4235084701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc1815c1b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.352+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/4235084701 >> 172.21.15.45:0/4235084701 conn(0x7fbc180cc570 msgr2=0x7fbc180cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.352+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/4235084701 shutdown_connections 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.352+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/4235084701 wait complete. 2023-10-06T05:17:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.353+0000 7fbc1cd0b700 1 Processor -- start 2023-10-06T05:17:52.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.353+0000 7fbc1cd0b700 1 -- start start 2023-10-06T05:17:52.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1cd0b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1806aa70 0x7fbc180c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1cd0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1806ae40 0x7fbc180c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1cd0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc180c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1cd0b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180c4770 con 0x7fbc180c7050 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1cd0b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180c2d80 con 0x7fbc1806ae40 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1cd0b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180c3b60 con 0x7fbc1806aa70 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc180c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc180c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50972/0 (socket says 172.21.15.45:50972) 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 -- 172.21.15.45:0/2635485886 learned_addr learned my addr 172.21.15.45:0/2635485886 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc15d9b700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1806ae40 0x7fbc180c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1659c700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1806aa70 0x7fbc180c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 -- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1806aa70 msgr2=0x7fbc180c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1806aa70 0x7fbc180c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 -- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1806ae40 msgr2=0x7fbc180c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1806ae40 0x7fbc180c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 -- 172.21.15.45:0/2635485886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0094f0 con 0x7fbc180c7050 2023-10-06T05:17:52.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc16d9d700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc180c2840 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c8e0 tx=0x7fbc10001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc067fc700 1 -- 172.21.15.45:0/2635485886 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000ce80 con 0x7fbc180c7050 2023-10-06T05:17:52.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc067fc700 1 -- 172.21.15.45:0/2635485886 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1000a7e0 con 0x7fbc180c7050 2023-10-06T05:17:52.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.354+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc180c3e40 con 0x7fbc180c7050 2023-10-06T05:17:52.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.355+0000 7fbc067fc700 1 -- 172.21.15.45:0/2635485886 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000ab80 con 0x7fbc180c7050 2023-10-06T05:17:52.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.355+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc180c4400 con 0x7fbc180c7050 2023-10-06T05:17:52.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.356+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc180c48e0 con 0x7fbc180c7050 2023-10-06T05:17:52.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.356+0000 7fbc067fc700 1 -- 172.21.15.45:0/2635485886 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc10012420 con 0x7fbc180c7050 2023-10-06T05:17:52.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.356+0000 7fbc067fc700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbbf8066f30 0x7fbbf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.357+0000 7fbc067fc700 1 -- 172.21.15.45:0/2635485886 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fbc10084bf0 con 0x7fbc180c7050 2023-10-06T05:17:52.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.357+0000 7fbc1659c700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbbf8066f30 0x7fbbf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.357+0000 7fbc1659c700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbbf8066f30 0x7fbbf80693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006fd0 tx=0x7fbc08008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.361+0000 7fbc067fc700 1 -- 172.21.15.45:0/2635485886 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc10051af0 con 0x7fbc180c7050 2023-10-06T05:17:52.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.565+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc180c7b70 con 0x7fbc180c7050 2023-10-06T05:17:52.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.566+0000 7fbc067fc700 1 -- 172.21.15.45:0/2635485886 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc10055110 con 0x7fbc180c7050 2023-10-06T05:17:52.568 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:17:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbbf8066f30 msgr2=0x7fbbf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbbf8066f30 0x7fbbf80693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006fd0 tx=0x7fbc08008040 comp rx=0 tx=0).stop 2023-10-06T05:17:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 msgr2=0x7fbc180c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc180c2840 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c8e0 tx=0x7fbc10001710 comp rx=0 tx=0).stop 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 shutdown_connections 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbbf8066f30 0x7fbbf80693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc180c7050 0x7fbc180c2840 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1806aa70 0x7fbc180c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 --2- 172.21.15.45:0/2635485886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1806ae40 0x7fbc180c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 >> 172.21.15.45:0/2635485886 conn(0x7fbc180cc570 msgr2=0x7fbc180ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 shutdown_connections 2023-10-06T05:17:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.568+0000 7fbc1cd0b700 1 -- 172.21.15.45:0/2635485886 wait complete. 2023-10-06T05:17:52.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:52.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.717+0000 7fee28e34700 1 Processor -- start 2023-10-06T05:17:52.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.717+0000 7fee28e34700 1 -- start start 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee28e34700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee240d02b0 0x7fee240d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee28e34700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee2415c8c0 0x7fee240caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee28e34700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee240cd740 con 0x7fee2415d820 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee28e34700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee240ccb30 con 0x7fee240d02b0 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee28e34700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee240cce10 con 0x7fee2415c8c0 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee2259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee240d02b0 0x7fee240d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee2259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee240d02b0 0x7fee240d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50974/0 (socket says 172.21.15.45:50974) 2023-10-06T05:17:52.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee2259c700 1 -- 172.21.15.45:0/76337413 learned_addr learned my addr 172.21.15.45:0/76337413 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee22d9d700 1 --2- 172.21.15.45:0/76337413 >> v2:172.21.15.144:3300/0 conn(0x7fee2415c8c0 0x7fee240caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee2259c700 1 -- 172.21.15.45:0/76337413 >> v2:172.21.15.144:3300/0 conn(0x7fee2415c8c0 msgr2=0x7fee240caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee2259c700 1 --2- 172.21.15.45:0/76337413 >> v2:172.21.15.144:3300/0 conn(0x7fee2415c8c0 0x7fee240caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee2259c700 1 -- 172.21.15.45:0/76337413 >> v1:172.21.15.144:6789/0 conn(0x7fee2415d820 legacy=0x7fee2415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee2259c700 1 -- 172.21.15.45:0/76337413 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240cd0f0 con 0x7fee240d02b0 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.718+0000 7fee22d9d700 1 --2- 172.21.15.45:0/76337413 >> v2:172.21.15.144:3300/0 conn(0x7fee2415c8c0 0x7fee240caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.719+0000 7fee2259c700 1 --2- 172.21.15.45:0/76337413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee240d02b0 0x7fee240d0680 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fee240b0e00 tx=0x7fee14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096cc925647210d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.719+0000 7fee2159a700 1 -- 172.21.15.45:0/76337413 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400b860 con 0x7fee240d02b0 2023-10-06T05:17:52.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.719+0000 7fee2159a700 1 -- 172.21.15.45:0/76337413 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1400b9c0 con 0x7fee240d02b0 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.719+0000 7fee2159a700 1 -- 172.21.15.45:0/76337413 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400bd60 con 0x7fee240d02b0 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.719+0000 7fee28e34700 1 -- 172.21.15.45:0/76337413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee240d02b0 msgr2=0x7fee240d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.719+0000 7fee28e34700 1 --2- 172.21.15.45:0/76337413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee240d02b0 0x7fee240d0680 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fee240b0e00 tx=0x7fee14009250 comp rx=0 tx=0).stop 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.720+0000 7fee28e34700 1 -- 172.21.15.45:0/76337413 shutdown_connections 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.720+0000 7fee28e34700 1 --2- 172.21.15.45:0/76337413 >> v2:172.21.15.144:3300/0 conn(0x7fee2415c8c0 0x7fee240caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.720+0000 7fee28e34700 1 --2- 172.21.15.45:0/76337413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee240d02b0 0x7fee240d0680 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.720+0000 7fee28e34700 1 -- 172.21.15.45:0/76337413 >> 172.21.15.45:0/76337413 conn(0x7fee2406ed50 msgr2=0x7fee2406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.720+0000 7fee28e34700 1 -- 172.21.15.45:0/76337413 shutdown_connections 2023-10-06T05:17:52.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.720+0000 7fee28e34700 1 -- 172.21.15.45:0/76337413 wait complete. 2023-10-06T05:17:52.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.721+0000 7fee28e34700 1 Processor -- start 2023-10-06T05:17:52.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.721+0000 7fee28e34700 1 -- start start 2023-10-06T05:17:52.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee28e34700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee240d02b0 0x7fee2415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee28e34700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2415c8c0 0x7fee240b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee28e34700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee2415d820 0x7fee240b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee28e34700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee240a8990 con 0x7fee2415c8c0 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee28e34700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee24169bd0 con 0x7fee240d02b0 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee28e34700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee24169880 con 0x7fee2415d820 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2415c8c0 0x7fee240b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2415c8c0 0x7fee240b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50976/0 (socket says 172.21.15.45:50976) 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 -- 172.21.15.45:0/1551623786 learned_addr learned my addr 172.21.15.45:0/1551623786 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee2259c700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee240d02b0 0x7fee2415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee22d9d700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee2415d820 0x7fee240b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 -- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee2415d820 msgr2=0x7fee240b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee2415d820 0x7fee240b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 -- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee240d02b0 msgr2=0x7fee2415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee240d02b0 0x7fee2415a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 -- 172.21.15.45:0/1551623786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee140094f0 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee21d9b700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2415c8c0 0x7fee240b4530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00c4e0 tx=0x7fee0c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.722+0000 7fee1a7fc700 1 -- 172.21.15.45:0/1551623786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c011590 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.723+0000 7fee1a7fc700 1 -- 172.21.15.45:0/1551623786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c011710 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.723+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee240a80f0 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.723+0000 7fee1a7fc700 1 -- 172.21.15.45:0/1551623786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c011ad0 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.723+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee240a86b0 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.724+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee24169d40 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.724+0000 7fee1a7fc700 1 -- 172.21.15.45:0/1551623786 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee0c01b3b0 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.724+0000 7fee1a7fc700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fee04066f30 0x7fee040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.725+0000 7fee1a7fc700 1 -- 172.21.15.45:0/1551623786 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fee0c083c60 con 0x7fee2415c8c0 2023-10-06T05:17:52.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.725+0000 7fee2259c700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fee04066f30 0x7fee040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.725+0000 7fee2259c700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fee04066f30 0x7fee040693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee240b0e00 tx=0x7fee14017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.729+0000 7fee1a7fc700 1 -- 172.21.15.45:0/1551623786 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee0c050b60 con 0x7fee2415c8c0 2023-10-06T05:17:52.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.933+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fee240cd3d0 con 0x7fee2415c8c0 2023-10-06T05:17:52.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.933+0000 7fee1a7fc700 1 -- 172.21.15.45:0/1551623786 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7fee0c017070 con 0x7fee2415c8c0 2023-10-06T05:17:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.935+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fee04066f30 msgr2=0x7fee040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.935+0000 7fee28e34700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fee04066f30 0x7fee040693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee240b0e00 tx=0x7fee14017000 comp rx=0 tx=0).stop 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.935+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2415c8c0 msgr2=0x7fee240b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.935+0000 7fee28e34700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2415c8c0 0x7fee240b4530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00c4e0 tx=0x7fee0c00c7f0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 shutdown_connections 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fee04066f30 0x7fee040693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee2415d820 0x7fee240b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee240d02b0 0x7fee2415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 --2- 172.21.15.45:0/1551623786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee2415c8c0 0x7fee240b4530 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 >> 172.21.15.45:0/1551623786 conn(0x7fee2406ed50 msgr2=0x7fee24070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 shutdown_connections 2023-10-06T05:17:52.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:52.936+0000 7fee28e34700 1 -- 172.21.15.45:0/1551623786 wait complete. 2023-10-06T05:17:52.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:53.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.084+0000 7fb338441700 1 Processor -- start 2023-10-06T05:17:53.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.084+0000 7fb338441700 1 -- start start 2023-10-06T05:17:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.085+0000 7fb338441700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3300c9dd0 0x7fb3300ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.085+0000 7fb338441700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300c9940 con 0x7fb330140ab0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.085+0000 7fb338441700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300ca750 con 0x7fb3301406e0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.085+0000 7fb338441700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33006d230 con 0x7fb3300c9dd0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3359dc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb330140ab0 0x7fb3300acfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41220/0 (socket says 172.21.15.45:41220) 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3359dc700 1 -- 172.21.15.45:0/799711812 learned_addr learned my addr 172.21.15.45:0/799711812 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3369de700 1 --2- 172.21.15.45:0/799711812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3300c9dd0 0x7fb3300ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3351db700 1 -- 172.21.15.45:0/799711812 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008534230 0 0) 0x7fb3300c9940 con 0x7fb330140ab0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3351db700 1 -- 172.21.15.45:0/799711812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb31c003570 con 0x7fb330140ab0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3351db700 1 -- 172.21.15.45:0/799711812 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037205366 0 0) 0x7fb3300ca750 con 0x7fb3301406e0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3351db700 1 -- 172.21.15.45:0/799711812 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3300c9940 con 0x7fb3301406e0 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3369de700 1 -- 172.21.15.45:0/799711812 >> v1:172.21.15.144:6789/0 conn(0x7fb3301406e0 legacy=0x7fb33006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3369de700 1 -- 172.21.15.45:0/799711812 >> v1:172.21.15.45:6789/0 conn(0x7fb330140ab0 legacy=0x7fb3300acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3369de700 1 -- 172.21.15.45:0/799711812 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33006d510 con 0x7fb3300c9dd0 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3369de700 1 --2- 172.21.15.45:0/799711812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3300c9dd0 0x7fb3300ca210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb328009e90 tx=0x7fb328009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d03d73cba4205f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.086+0000 7fb3351db700 1 -- 172.21.15.45:0/799711812 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800e070 con 0x7fb3300c9dd0 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb3351db700 1 -- 172.21.15.45:0/799711812 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32800bb80 con 0x7fb3300c9dd0 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb3351db700 1 -- 172.21.15.45:0/799711812 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328012430 con 0x7fb3300c9dd0 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb338441700 1 -- 172.21.15.45:0/799711812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3300c9dd0 msgr2=0x7fb3300ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb338441700 1 --2- 172.21.15.45:0/799711812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3300c9dd0 0x7fb3300ca210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb328009e90 tx=0x7fb328009780 comp rx=0 tx=0).stop 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb338441700 1 -- 172.21.15.45:0/799711812 shutdown_connections 2023-10-06T05:17:53.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb338441700 1 --2- 172.21.15.45:0/799711812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3300c9dd0 0x7fb3300ca210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb338441700 1 -- 172.21.15.45:0/799711812 >> 172.21.15.45:0/799711812 conn(0x7fb3300691c0 msgr2=0x7fb3300cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb338441700 1 -- 172.21.15.45:0/799711812 shutdown_connections 2023-10-06T05:17:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.087+0000 7fb338441700 1 -- 172.21.15.45:0/799711812 wait complete. 2023-10-06T05:17:53.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.088+0000 7fb338441700 1 Processor -- start 2023-10-06T05:17:53.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb338441700 1 -- start start 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb338441700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3300c9dd0 0x7fb3300c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb338441700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3301406e0 0x7fb33015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb338441700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb330140ab0 0x7fb3300b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb338441700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33015c250 con 0x7fb3301406e0 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb338441700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33013cdc0 con 0x7fb3300c9dd0 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb338441700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33015b640 con 0x7fb330140ab0 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb3359dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3301406e0 0x7fb33015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb3359dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3301406e0 0x7fb33015ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50988/0 (socket says 172.21.15.45:50988) 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb3359dc700 1 -- 172.21.15.45:0/1725976504 learned_addr learned my addr 172.21.15.45:0/1725976504 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.089+0000 7fb3361dd700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3300c9dd0 0x7fb3300c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3369de700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb330140ab0 0x7fb3300b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3359dc700 1 -- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb330140ab0 msgr2=0x7fb3300b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3359dc700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb330140ab0 0x7fb3300b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3359dc700 1 -- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3300c9dd0 msgr2=0x7fb3300c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3359dc700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3300c9dd0 0x7fb3300c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3359dc700 1 -- 172.21.15.45:0/1725976504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328009a20 con 0x7fb3301406e0 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3359dc700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3301406e0 0x7fb33015ad60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c790 tx=0x7fb32c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3267fc700 1 -- 172.21.15.45:0/1725976504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c005b30 con 0x7fb3301406e0 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3267fc700 1 -- 172.21.15.45:0/1725976504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c005cd0 con 0x7fb3301406e0 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb3267fc700 1 -- 172.21.15.45:0/1725976504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c007320 con 0x7fb3301406e0 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33015b920 con 0x7fb3301406e0 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.090+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33015bee0 con 0x7fb3301406e0 2023-10-06T05:17:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.091+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33015c3c0 con 0x7fb3301406e0 2023-10-06T05:17:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.092+0000 7fb3267fc700 1 -- 172.21.15.45:0/1725976504 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb32c012400 con 0x7fb3301406e0 2023-10-06T05:17:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.092+0000 7fb3267fc700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb314066f30 0x7fb3140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.092+0000 7fb3267fc700 1 -- 172.21.15.45:0/1725976504 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fb32c083b60 con 0x7fb3301406e0 2023-10-06T05:17:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.092+0000 7fb3361dd700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb314066f30 0x7fb3140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.093+0000 7fb3361dd700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb314066f30 0x7fb3140693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb320007fd0 tx=0x7fb320007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.097+0000 7fb3267fc700 1 -- 172.21.15.45:0/1725976504 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb32c04b5a0 con 0x7fb3301406e0 2023-10-06T05:17:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.300+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb33004bb40 con 0x7fb3301406e0 2023-10-06T05:17:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.300+0000 7fb3267fc700 1 -- 172.21.15.45:0/1725976504 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb32c0170a0 con 0x7fb3301406e0 2023-10-06T05:17:53.302 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:17:53.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb314066f30 msgr2=0x7fb3140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb314066f30 0x7fb3140693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb320007fd0 tx=0x7fb320007d30 comp rx=0 tx=0).stop 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3301406e0 msgr2=0x7fb33015ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3301406e0 0x7fb33015ad60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c790 tx=0x7fb32c007e80 comp rx=0 tx=0).stop 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 shutdown_connections 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb314066f30 0x7fb3140693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb330140ab0 0x7fb3300b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3300c9dd0 0x7fb3300c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 --2- 172.21.15.45:0/1725976504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3301406e0 0x7fb33015ad60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.302+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 >> 172.21.15.45:0/1725976504 conn(0x7fb3300691c0 msgr2=0x7fb33013f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.303+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 shutdown_connections 2023-10-06T05:17:53.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.303+0000 7fb338441700 1 -- 172.21.15.45:0/1725976504 wait complete. 2023-10-06T05:17:53.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.453+0000 7fcad199d700 1 Processor -- start 2023-10-06T05:17:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.453+0000 7fcad199d700 1 -- start start 2023-10-06T05:17:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad199d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcacc0cc370 0x7fcacc0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad199d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0c97d0 con 0x7fcacc1572a0 2023-10-06T05:17:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad199d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0c8c30 con 0x7fcacc0c3b90 2023-10-06T05:17:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad199d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc0c4570 con 0x7fcacc0cc370 2023-10-06T05:17:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad119c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcacc0cc370 0x7fcacc0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad099b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcacc1572a0 0x7fcacc0aa930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41222/0 (socket says 172.21.15.45:41222) 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad099b700 1 -- 172.21.15.45:0/1537991767 learned_addr learned my addr 172.21.15.45:0/1537991767 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad119c700 1 -- 172.21.15.45:0/1537991767 >> v1:172.21.15.144:6789/0 conn(0x7fcacc0c3b90 legacy=0x7fcacc0cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad119c700 1 -- 172.21.15.45:0/1537991767 >> v1:172.21.15.45:6789/0 conn(0x7fcacc1572a0 legacy=0x7fcacc0aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcacb7fe700 1 -- 172.21.15.45:0/1537991767 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705740237 0 0) 0x7fcacc0c97d0 con 0x7fcacc1572a0 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad119c700 1 -- 172.21.15.45:0/1537991767 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc0ab040 con 0x7fcacc0cc370 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcad119c700 1 --2- 172.21.15.45:0/1537991767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c4030 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009e60 tx=0x7fcac4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c37e1598079b808 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcacb7fe700 1 -- 172.21.15.45:0/1537991767 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400e070 con 0x7fcacc0cc370 2023-10-06T05:17:53.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcacb7fe700 1 -- 172.21.15.45:0/1537991767 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac400bc30 con 0x7fcacc0cc370 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.454+0000 7fcacb7fe700 1 -- 172.21.15.45:0/1537991767 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4012430 con 0x7fcacc0cc370 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.455+0000 7fcad199d700 1 -- 172.21.15.45:0/1537991767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 msgr2=0x7fcacc0c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.455+0000 7fcad199d700 1 --2- 172.21.15.45:0/1537991767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c4030 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009e60 tx=0x7fcac4009780 comp rx=0 tx=0).stop 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.455+0000 7fcad199d700 1 -- 172.21.15.45:0/1537991767 shutdown_connections 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.455+0000 7fcad199d700 1 --2- 172.21.15.45:0/1537991767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c4030 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.455+0000 7fcad199d700 1 -- 172.21.15.45:0/1537991767 >> 172.21.15.45:0/1537991767 conn(0x7fcacc067e00 msgr2=0x7fcacc0cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.455+0000 7fcad199d700 1 -- 172.21.15.45:0/1537991767 shutdown_connections 2023-10-06T05:17:53.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.455+0000 7fcad199d700 1 -- 172.21.15.45:0/1537991767 wait complete. 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.456+0000 7fcad199d700 1 Processor -- start 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcad199d700 1 -- start start 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcad199d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc0c3b90 0x7fcacc149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcad199d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcad199d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1572a0 0x7fcacc0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcad199d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0c6850 con 0x7fcacc0cc370 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcad199d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0c5e80 con 0x7fcacc1572a0 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcad199d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0c5270 con 0x7fcacc0c3b90 2023-10-06T05:17:53.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.457+0000 7fcacbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51010/0 (socket says 172.21.15.45:51010) 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 -- 172.21.15.45:0/3587687602 learned_addr learned my addr 172.21.15.45:0/3587687602 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcad119c700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1572a0 0x7fcacc0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcad099b700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc0c3b90 0x7fcacc149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 -- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc0c3b90 msgr2=0x7fcacc149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc0c3b90 0x7fcacc149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 -- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1572a0 msgr2=0x7fcacc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1572a0 0x7fcacc0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 -- 172.21.15.45:0/3587687602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac4009a20 con 0x7fcacc0cc370 2023-10-06T05:17:53.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcacbfff700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c6310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c790 tx=0x7fcab4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcac8ff9700 1 -- 172.21.15.45:0/3587687602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011590 con 0x7fcacc0cc370 2023-10-06T05:17:53.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcac8ff9700 1 -- 172.21.15.45:0/3587687602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab4011710 con 0x7fcacc0cc370 2023-10-06T05:17:53.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcac8ff9700 1 -- 172.21.15.45:0/3587687602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011ad0 con 0x7fcacc0cc370 2023-10-06T05:17:53.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc0c5550 con 0x7fcacc0cc370 2023-10-06T05:17:53.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.458+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc0c5b10 con 0x7fcacc0cc370 2023-10-06T05:17:53.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.459+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc0c5ff0 con 0x7fcacc0cc370 2023-10-06T05:17:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.460+0000 7fcac8ff9700 1 -- 172.21.15.45:0/3587687602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcab401c440 con 0x7fcacc0cc370 2023-10-06T05:17:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.460+0000 7fcac8ff9700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcaac066f00 0x7fcaac0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.460+0000 7fcac8ff9700 1 -- 172.21.15.45:0/3587687602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fcab4083b70 con 0x7fcacc0cc370 2023-10-06T05:17:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.460+0000 7fcad099b700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcaac066f00 0x7fcaac0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.461+0000 7fcad099b700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcaac066f00 0x7fcaac0693b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcabc002970 tx=0x7fcabc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.465+0000 7fcac8ff9700 1 -- 172.21.15.45:0/3587687602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcab4050af0 con 0x7fcacc0cc370 2023-10-06T05:17:53.613 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:17:53.612+0000 7f71d24f3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:17:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.667+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcacc064a60 con 0x7fcacc0cc370 2023-10-06T05:17:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.667+0000 7fcac8ff9700 1 -- 172.21.15.45:0/3587687602 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcab4054110 con 0x7fcacc0cc370 2023-10-06T05:17:53.669 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:17:53.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.669+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcaac066f00 msgr2=0x7fcaac0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.669+0000 7fcad199d700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcaac066f00 0x7fcaac0693b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcabc002970 tx=0x7fcabc009040 comp rx=0 tx=0).stop 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.669+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 msgr2=0x7fcacc0c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.669+0000 7fcad199d700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c6310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c790 tx=0x7fcab4007f40 comp rx=0 tx=0).stop 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 shutdown_connections 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcaac066f00 0x7fcaac0693b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1572a0 0x7fcacc0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc0c3b90 0x7fcacc149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 --2- 172.21.15.45:0/3587687602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cc370 0x7fcacc0c6310 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 >> 172.21.15.45:0/3587687602 conn(0x7fcacc067e00 msgr2=0x7fcacc154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 shutdown_connections 2023-10-06T05:17:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.670+0000 7fcad199d700 1 -- 172.21.15.45:0/3587687602 wait complete. 2023-10-06T05:17:53.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.816+0000 7fcecb1fe700 1 Processor -- start 2023-10-06T05:17:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.816+0000 7fcecb1fe700 1 -- start start 2023-10-06T05:17:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcecb1fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcec41560a0 0x7fcec415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcecb1fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec40ccdc0 0x7fcec40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcecb1fe700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec40cd740 con 0x7fcec4155cd0 2023-10-06T05:17:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcecb1fe700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec40ae6b0 con 0x7fcec41560a0 2023-10-06T05:17:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcecb1fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40ae8d0 con 0x7fcec40ccdc0 2023-10-06T05:17:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcec41560a0 0x7fcec415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec8f9a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcec4155cd0 0x7fcec40adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41234/0 (socket says 172.21.15.45:41234) 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec8f9a700 1 -- 172.21.15.45:0/1799130839 learned_addr learned my addr 172.21.15.45:0/1799130839 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec3fff700 1 --2- 172.21.15.45:0/1799130839 >> v2:172.21.15.45:3300/0 conn(0x7fcec41560a0 0x7fcec415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51016/0 (socket says 172.21.15.45:51016) 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec979b700 1 --2- 172.21.15.45:0/1799130839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec40ccdc0 0x7fcec40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec3fff700 1 -- 172.21.15.45:0/1799130839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec40ccdc0 msgr2=0x7fcec40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec3fff700 1 --2- 172.21.15.45:0/1799130839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec40ccdc0 0x7fcec40cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec3fff700 1 -- 172.21.15.45:0/1799130839 >> v1:172.21.15.45:6789/0 conn(0x7fcec4155cd0 legacy=0x7fcec40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec3fff700 1 -- 172.21.15.45:0/1799130839 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec40aebb0 con 0x7fcec41560a0 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec37fe700 1 -- 172.21.15.45:0/1799130839 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712034813 0 0) 0x7fcec40cd740 con 0x7fcec4155cd0 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.817+0000 7fcec3fff700 1 --2- 172.21.15.45:0/1799130839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec415a730 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fceac00bb90 tx=0x7fceac00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e5bc4fa1c87c0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcec37fe700 1 -- 172.21.15.45:0/1799130839 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac00dba0 con 0x7fcec41560a0 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcec37fe700 1 -- 172.21.15.45:0/1799130839 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac00dd00 con 0x7fcec41560a0 2023-10-06T05:17:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcec37fe700 1 -- 172.21.15.45:0/1799130839 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac0196b0 con 0x7fcec41560a0 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 -- 172.21.15.45:0/1799130839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 msgr2=0x7fcec415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/1799130839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec415a730 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fceac00bb90 tx=0x7fceac00b430 comp rx=0 tx=0).stop 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 -- 172.21.15.45:0/1799130839 shutdown_connections 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/1799130839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec40ccdc0 0x7fcec40cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/1799130839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec415a730 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 -- 172.21.15.45:0/1799130839 >> 172.21.15.45:0/1799130839 conn(0x7fcec406f090 msgr2=0x7fcec406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 -- 172.21.15.45:0/1799130839 shutdown_connections 2023-10-06T05:17:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.818+0000 7fcecb1fe700 1 -- 172.21.15.45:0/1799130839 wait complete. 2023-10-06T05:17:53.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.819+0000 7fcecb1fe700 1 Processor -- start 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.820+0000 7fcecb1fe700 1 -- start start 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.820+0000 7fcecb1fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec40ccdc0 0x7fcec41699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.820+0000 7fcecb1fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec4155cd0 0x7fcec416c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.820+0000 7fcecb1fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.820+0000 7fcecb1fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40c5840 con 0x7fcec41560a0 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.820+0000 7fcecb1fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec416cd10 con 0x7fcec40ccdc0 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.820+0000 7fcecb1fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40c4c30 con 0x7fcec4155cd0 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec4156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51030/0 (socket says 172.21.15.45:51030) 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec8f9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec40ccdc0 0x7fcec41699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec3fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec4155cd0 0x7fcec416c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 -- 172.21.15.45:0/2378762349 learned_addr learned my addr 172.21.15.45:0/2378762349 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 -- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec4155cd0 msgr2=0x7fcec416c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec4155cd0 0x7fcec416c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 -- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec40ccdc0 msgr2=0x7fcec41699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec40ccdc0 0x7fcec41699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 -- 172.21.15.45:0/2378762349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac00b6d0 con 0x7fcec41560a0 2023-10-06T05:17:53.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.821+0000 7fcec979b700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec4156e80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009fa0 tx=0x7fcebc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.823+0000 7fcec0ff9700 1 -- 172.21.15.45:0/2378762349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00ebc0 con 0x7fcec41560a0 2023-10-06T05:17:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.823+0000 7fcec0ff9700 1 -- 172.21.15.45:0/2378762349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc00ed20 con 0x7fcec41560a0 2023-10-06T05:17:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.823+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec40c4f10 con 0x7fcec41560a0 2023-10-06T05:17:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.823+0000 7fcec0ff9700 1 -- 172.21.15.45:0/2378762349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc014670 con 0x7fcec41560a0 2023-10-06T05:17:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.823+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec40c54d0 con 0x7fcec41560a0 2023-10-06T05:17:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.824+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec40c59b0 con 0x7fcec41560a0 2023-10-06T05:17:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.825+0000 7fcec0ff9700 1 -- 172.21.15.45:0/2378762349 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcebc0149e0 con 0x7fcec41560a0 2023-10-06T05:17:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.825+0000 7fcec0ff9700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcea4066f30 0x7fcea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.825+0000 7fcec0ff9700 1 -- 172.21.15.45:0/2378762349 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fcebc084f90 con 0x7fcec41560a0 2023-10-06T05:17:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.825+0000 7fcec8f9a700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcea4066f30 0x7fcea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.826+0000 7fcec8f9a700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcea4066f30 0x7fcea40693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fceb4007fd0 tx=0x7fceb4007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:53.830+0000 7fcec0ff9700 1 -- 172.21.15.45:0/2378762349 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcebc051e90 con 0x7fcec41560a0 2023-10-06T05:17:54.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.078+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcec40aee90 con 0x7fcec41560a0 2023-10-06T05:17:54.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.078+0000 7fcec0ff9700 1 -- 172.21.15.45:0/2378762349 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4598 (secure 0 0 0) 0x7fcebc0554b0 con 0x7fcec41560a0 2023-10-06T05:17:54.080 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:54.080 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:52.613443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5779},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5779":{"gid":5779,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/3967110338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3967110338},{"type":"v1","addr":"172.21.15.45:6835","nonce":3967110338}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcea4066f30 msgr2=0x7fcea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcea4066f30 0x7fcea40693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fceb4007fd0 tx=0x7fceb4007d30 comp rx=0 tx=0).stop 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 msgr2=0x7fcec4156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec4156e80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009fa0 tx=0x7fcebc009900 comp rx=0 tx=0).stop 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 shutdown_connections 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcea4066f30 0x7fcea40693e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec41560a0 0x7fcec4156e80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec40ccdc0 0x7fcec41699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 --2- 172.21.15.45:0/2378762349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec4155cd0 0x7fcec416c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 >> 172.21.15.45:0/2378762349 conn(0x7fcec406f090 msgr2=0x7fcec40cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 shutdown_connections 2023-10-06T05:17:54.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.081+0000 7fcecb1fe700 1 -- 172.21.15.45:0/2378762349 wait complete. 2023-10-06T05:17:54.084 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 40 2023-10-06T05:17:54.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:54.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.226+0000 7fe45ccad700 1 Processor -- start 2023-10-06T05:17:54.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.226+0000 7fe45ccad700 1 -- start start 2023-10-06T05:17:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.227+0000 7fe45ccad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 0x7fe458159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.227+0000 7fe45ccad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe4580cc390 0x7fe4580cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.227+0000 7fe45ccad700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580cb530 con 0x7fe45806a060 2023-10-06T05:17:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.227+0000 7fe45ccad700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580ccd50 con 0x7fe4580cc390 2023-10-06T05:17:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.227+0000 7fe45ccad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe458069bd0 con 0x7fe458158f60 2023-10-06T05:17:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe455d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe45806a060 0x7fe4580cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41242/0 (socket says 172.21.15.45:41242) 2023-10-06T05:17:54.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe455d9b700 1 -- 172.21.15.45:0/2315123478 learned_addr learned my addr 172.21.15.45:0/2315123478 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe456d9d700 1 --2- 172.21.15.45:0/2315123478 >> v2:172.21.15.144:3300/0 conn(0x7fe4580cc390 0x7fe4580cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45659c700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 0x7fe458159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883904469 0 0) 0x7fe440001110 con 0x7fe45806a060 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe444003570 con 0x7fe45806a060 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2664580313 0 0) 0x7fe444003570 con 0x7fe45806a060 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 msgr2=0x7fe458159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 0x7fe458159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 >> v2:172.21.15.144:3300/0 conn(0x7fe4580cc390 msgr2=0x7fe4580cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:54.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 --2- 172.21.15.45:0/2315123478 >> v2:172.21.15.144:3300/0 conn(0x7fe4580cc390 0x7fe4580cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45806a4d0 con 0x7fe45806a060 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fe440001600 con 0x7fe45806a060 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 >> v1:172.21.15.45:6789/0 conn(0x7fe45806a060 legacy=0x7fe4580cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe444003a60 0x7fe444003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe444004810 0x7fe444004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 0x7fe4440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580ccd50 con 0x7fe444004810 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458069bd0 con 0x7fe444003a60 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44400ba60 con 0x7fe458158f60 2023-10-06T05:17:54.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe455d9b700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe444004810 0x7fe444004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.228+0000 7fe45659c700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe444003a60 0x7fe444003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe456d9d700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 0x7fe4440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe455d9b700 1 -- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 msgr2=0x7fe4440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe455d9b700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe458158f60 0x7fe4440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe455d9b700 1 -- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe444003a60 msgr2=0x7fe444003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe455d9b700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe444003a60 0x7fe444003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe455d9b700 1 -- 172.21.15.45:0/2315123478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe444003570 con 0x7fe444004810 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe455d9b700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe444004810 0x7fe444004c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe44000c620 tx=0x7fe440007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47eab400e3f5dba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000cde0 con 0x7fe444004810 2023-10-06T05:17:54.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440005920 con 0x7fe444004810 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe45559a700 1 -- 172.21.15.45:0/2315123478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440005ce0 con 0x7fe444004810 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.229+0000 7fe45ccad700 1 -- 172.21.15.45:0/2315123478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe444004810 msgr2=0x7fe444004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.230+0000 7fe45ccad700 1 --2- 172.21.15.45:0/2315123478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe444004810 0x7fe444004c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe44000c620 tx=0x7fe440007a80 comp rx=0 tx=0).stop 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.230+0000 7fe456d9d700 1 -- 172.21.15.45:0/2315123478 reap_dead start 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.230+0000 7fe45ccad700 1 -- 172.21.15.45:0/2315123478 shutdown_connections 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.230+0000 7fe45ccad700 1 -- 172.21.15.45:0/2315123478 >> 172.21.15.45:0/2315123478 conn(0x7fe4580cdf50 msgr2=0x7fe4580ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.230+0000 7fe45ccad700 1 -- 172.21.15.45:0/2315123478 shutdown_connections 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.230+0000 7fe45ccad700 1 -- 172.21.15.45:0/2315123478 wait complete. 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.231+0000 7fe45ccad700 1 Processor -- start 2023-10-06T05:17:54.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.231+0000 7fe45ccad700 1 -- start start 2023-10-06T05:17:54.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe45ccad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe444004810 0x7fe458156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe45ccad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45815d140 0x7fe458156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe45ccad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4580b1f00 0x7fe4580b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe45ccad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580b28c0 con 0x7fe4580b1f00 2023-10-06T05:17:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe45ccad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4581569d0 con 0x7fe45815d140 2023-10-06T05:17:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe45ccad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580b0730 con 0x7fe444004810 2023-10-06T05:17:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4580b1f00 0x7fe4580b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4580b1f00 0x7fe4580b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51040/0 (socket says 172.21.15.45:51040) 2023-10-06T05:17:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 -- 172.21.15.45:0/4207828211 learned_addr learned my addr 172.21.15.45:0/4207828211 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe45659c700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe444004810 0x7fe458156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe455d9b700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45815d140 0x7fe458156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 -- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe444004810 msgr2=0x7fe458156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe444004810 0x7fe458156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 -- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45815d140 msgr2=0x7fe458156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45815d140 0x7fe458156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 -- 172.21.15.45:0/4207828211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45815c0a0 con 0x7fe4580b1f00 2023-10-06T05:17:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe456d9d700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4580b1f00 0x7fe4580b2380 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe45806ed80 tx=0x7fe45000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.232+0000 7fe44e7fc700 1 -- 172.21.15.45:0/4207828211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000e070 con 0x7fe4580b1f00 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.233+0000 7fe44e7fc700 1 -- 172.21.15.45:0/4207828211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450011d00 con 0x7fe4580b1f00 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.233+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45815c380 con 0x7fe4580b1f00 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.233+0000 7fe44e7fc700 1 -- 172.21.15.45:0/4207828211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4500166c0 con 0x7fe4580b1f00 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.233+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45815c940 con 0x7fe4580b1f00 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.234+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45806f410 con 0x7fe4580b1f00 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.234+0000 7fe44e7fc700 1 -- 172.21.15.45:0/4207828211 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe450016a70 con 0x7fe4580b1f00 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.234+0000 7fe44e7fc700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe438066f30 0x7fe4380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.234+0000 7fe44e7fc700 1 -- 172.21.15.45:0/4207828211 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fe4500853c0 con 0x7fe4580b1f00 2023-10-06T05:17:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.235+0000 7fe45659c700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe438066f30 0x7fe4380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.235+0000 7fe45659c700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe438066f30 0x7fe4380693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe448006dc0 tx=0x7fe448008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.239+0000 7fe44e7fc700 1 -- 172.21.15.45:0/4207828211 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe450055dd0 con 0x7fe4580b1f00 2023-10-06T05:17:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.442+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe458068fc0 con 0x7fe4580b1f00 2023-10-06T05:17:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.443+0000 7fe44e7fc700 1 -- 172.21.15.45:0/4207828211 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17296 (secure 0 0 0) 0x7fe450055960 con 0x7fe4580b1f00 2023-10-06T05:17:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:54.446 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":53,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:17:47.603415+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:29.616613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:30.718338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2812290138":"2023-10-07T05:17:47.603375+0000","172.21.15.45:6833/2812290138":"2023-10-07T05:17:47.603375+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe438066f30 msgr2=0x7fe4380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe438066f30 0x7fe4380693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe448006dc0 tx=0x7fe448008040 comp rx=0 tx=0).stop 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4580b1f00 msgr2=0x7fe4580b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4580b1f00 0x7fe4580b2380 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe45806ed80 tx=0x7fe45000b040 comp rx=0 tx=0).stop 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 shutdown_connections 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe438066f30 0x7fe4380693e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4580b1f00 0x7fe4580b2380 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe444004810 0x7fe458156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 --2- 172.21.15.45:0/4207828211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe45815d140 0x7fe458156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 >> 172.21.15.45:0/4207828211 conn(0x7fe4580cdf50 msgr2=0x7fe458158020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 shutdown_connections 2023-10-06T05:17:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.446+0000 7fe45ccad700 1 -- 172.21.15.45:0/4207828211 wait complete. 2023-10-06T05:17:54.461 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:17:54.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.591+0000 7f3093796700 1 Processor -- start 2023-10-06T05:17:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.592+0000 7f3093796700 1 -- start start 2023-10-06T05:17:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.592+0000 7f3093796700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f308c0ccaa0 0x7f308c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.592+0000 7f3093796700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c0cb530 con 0x7f308c158f60 2023-10-06T05:17:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3093796700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c069bd0 con 0x7f308c159330 2023-10-06T05:17:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3093796700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c068fc0 con 0x7f308c0ccaa0 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3091532700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f308c158f60 0x7f308c0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41254/0 (socket says 172.21.15.45:41254) 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3091532700 1 -- 172.21.15.45:0/3333930026 learned_addr learned my addr 172.21.15.45:0/3333930026 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3091d33700 1 --2- 172.21.15.45:0/3333930026 >> v2:172.21.15.144:3300/0 conn(0x7f308c0ccaa0 0x7f308c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3083fff700 1 -- 172.21.15.45:0/3333930026 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492381633 0 0) 0x7f307c001110 con 0x7f308c158f60 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3083fff700 1 -- 172.21.15.45:0/3333930026 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3078003570 con 0x7f308c158f60 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3083fff700 1 -- 172.21.15.45:0/3333930026 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819761554 0 0) 0x7f308c069bd0 con 0x7f308c159330 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3083fff700 1 -- 172.21.15.45:0/3333930026 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f307c001110 con 0x7f308c159330 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3091d33700 1 -- 172.21.15.45:0/3333930026 >> v1:172.21.15.144:6789/0 conn(0x7f308c159330 legacy=0x7f308c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3091d33700 1 -- 172.21.15.45:0/3333930026 >> v1:172.21.15.45:6789/0 conn(0x7f308c158f60 legacy=0x7f308c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.593+0000 7f3091d33700 1 -- 172.21.15.45:0/3333930026 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c0692a0 con 0x7f308c0ccaa0 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3091d33700 1 --2- 172.21.15.45:0/3333930026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c0ccaa0 0x7f308c06a060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3084009e60 tx=0x7f3084009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ba6a167a2de9e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3083fff700 1 -- 172.21.15.45:0/3333930026 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308400e070 con 0x7f308c0ccaa0 2023-10-06T05:17:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3083fff700 1 -- 172.21.15.45:0/3333930026 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308400bab0 con 0x7f308c0ccaa0 2023-10-06T05:17:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3083fff700 1 -- 172.21.15.45:0/3333930026 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308400be70 con 0x7f308c0ccaa0 2023-10-06T05:17:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3093796700 1 -- 172.21.15.45:0/3333930026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c0ccaa0 msgr2=0x7f308c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3093796700 1 --2- 172.21.15.45:0/3333930026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c0ccaa0 0x7f308c06a060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3084009e60 tx=0x7f3084009780 comp rx=0 tx=0).stop 2023-10-06T05:17:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3093796700 1 -- 172.21.15.45:0/3333930026 shutdown_connections 2023-10-06T05:17:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3093796700 1 --2- 172.21.15.45:0/3333930026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c0ccaa0 0x7f308c06a060 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3093796700 1 -- 172.21.15.45:0/3333930026 >> 172.21.15.45:0/3333930026 conn(0x7f308c0cdf50 msgr2=0x7f308c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3093796700 1 -- 172.21.15.45:0/3333930026 shutdown_connections 2023-10-06T05:17:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.594+0000 7f3093796700 1 -- 172.21.15.45:0/3333930026 wait complete. 2023-10-06T05:17:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.596+0000 7f3093796700 1 Processor -- start 2023-10-06T05:17:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.596+0000 7f3093796700 1 -- start start 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3093796700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308c0ccaa0 0x7f308c154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3093796700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f308c158f60 0x7f308c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3093796700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c159330 0x7f308c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3093796700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c156680 con 0x7f308c158f60 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3093796700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0c5a10 con 0x7f308c159330 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3093796700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0c5c60 con 0x7f308c0ccaa0 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f308c158f60 0x7f308c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f308c158f60 0x7f308c0c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51054/0 (socket says 172.21.15.45:51054) 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 -- 172.21.15.45:0/4209529372 learned_addr learned my addr 172.21.15.45:0/4209529372 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3091d33700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c159330 0x7f308c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3091532700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308c0ccaa0 0x7f308c154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 -- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308c0ccaa0 msgr2=0x7f308c154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308c0ccaa0 0x7f308c154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 -- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c159330 msgr2=0x7f308c0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c159330 0x7f308c0c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 -- 172.21.15.45:0/4209529372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3084009a20 con 0x7f308c158f60 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3091d33700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c159330 0x7f308c0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.597+0000 7f3090d31700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f308c158f60 0x7f308c0c74a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f308800c590 tx=0x7f3088007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.598+0000 7f30817fa700 1 -- 172.21.15.45:0/4209529372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088010510 con 0x7f308c158f60 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.598+0000 7f30817fa700 1 -- 172.21.15.45:0/4209529372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3088010690 con 0x7f308c158f60 2023-10-06T05:17:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.598+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c0c5f40 con 0x7f308c158f60 2023-10-06T05:17:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.598+0000 7f30817fa700 1 -- 172.21.15.45:0/4209529372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088010a50 con 0x7f308c158f60 2023-10-06T05:17:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.598+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c0c6500 con 0x7f308c158f60 2023-10-06T05:17:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.599+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c0c6720 con 0x7f308c158f60 2023-10-06T05:17:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.599+0000 7f30817fa700 1 -- 172.21.15.45:0/4209529372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f308801b400 con 0x7f308c158f60 2023-10-06T05:17:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.599+0000 7f30817fa700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3070066f30 0x7f30700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.600+0000 7f30817fa700 1 -- 172.21.15.45:0/4209529372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f30880845c0 con 0x7f308c158f60 2023-10-06T05:17:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.600+0000 7f3091532700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3070066f30 0x7f30700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.600+0000 7f3091532700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3070066f30 0x7f30700693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f307c0058d0 tx=0x7f307c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.604+0000 7f30817fa700 1 -- 172.21.15.45:0/4209529372 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3088051540 con 0x7f308c158f60 2023-10-06T05:17:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.856+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f308c04bb40 con 0x7f308c158f60 2023-10-06T05:17:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.857+0000 7f30817fa700 1 -- 172.21.15.45:0/4209529372 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f3088054b60 con 0x7f308c158f60 2023-10-06T05:17:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:54.859 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:52.613443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5779},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5779":{"gid":5779,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/3967110338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3967110338},{"type":"v1","addr":"172.21.15.45:6835","nonce":3967110338}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3070066f30 msgr2=0x7f30700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3070066f30 0x7f30700693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f307c0058d0 tx=0x7f307c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f308c158f60 msgr2=0x7f308c0c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f308c158f60 0x7f308c0c74a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f308800c590 tx=0x7f3088007dd0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 shutdown_connections 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3070066f30 0x7f30700693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308c159330 0x7f308c0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308c0ccaa0 0x7f308c154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 --2- 172.21.15.45:0/4209529372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f308c158f60 0x7f308c0c74a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 >> 172.21.15.45:0/4209529372 conn(0x7f308c0cdf50 msgr2=0x7f308c06d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 shutdown_connections 2023-10-06T05:17:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:54.859+0000 7f3093796700 1 -- 172.21.15.45:0/4209529372 wait complete. 2023-10-06T05:17:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 40 2023-10-06T05:17:54.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.004+0000 7f3aa7eff700 1 Processor -- start 2023-10-06T05:17:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.004+0000 7f3aa7eff700 1 -- start start 2023-10-06T05:17:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa7eff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3aa00c93c0 0x7f3aa0156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa7eff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa0157030 0x7f3aa00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa7eff700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa015b000 con 0x7f3aa00c8ff0 2023-10-06T05:17:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa7eff700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa00c9830 con 0x7f3aa00c93c0 2023-10-06T05:17:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa7eff700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa015b2e0 con 0x7f3aa0157030 2023-10-06T05:17:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa649c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa0157030 0x7f3aa00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa649c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa0157030 0x7f3aa00cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:45008/0 (socket says 172.21.15.45:45008) 2023-10-06T05:17:55.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa549a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3aa00c93c0 0x7f3aa0156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.005+0000 7f3aa649c700 1 -- 172.21.15.45:0/2633095044 learned_addr learned my addr 172.21.15.45:0/2633095044 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:55.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa4c99700 1 -- 172.21.15.45:0/2633095044 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207951253 0 0) 0x7f3aa015b000 con 0x7f3aa00c8ff0 2023-10-06T05:17:55.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa649c700 1 -- 172.21.15.45:0/2633095044 >> v2:172.21.15.144:3300/0 conn(0x7f3aa00c93c0 msgr2=0x7f3aa0156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa649c700 1 --2- 172.21.15.45:0/2633095044 >> v2:172.21.15.144:3300/0 conn(0x7f3aa00c93c0 0x7f3aa0156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa649c700 1 -- 172.21.15.45:0/2633095044 >> v1:172.21.15.144:6789/0 conn(0x7f3aa00c8ff0 legacy=0x7f3aa00caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa649c700 1 -- 172.21.15.45:0/2633095044 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa015b5c0 con 0x7f3aa0157030 2023-10-06T05:17:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa649c700 1 --2- 172.21.15.45:0/2633095044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa0157030 0x7f3aa00cc330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009f20 tx=0x7f3a98009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ece36874d8577b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa4c99700 1 -- 172.21.15.45:0/2633095044 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800e070 con 0x7f3aa0157030 2023-10-06T05:17:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa4c99700 1 -- 172.21.15.45:0/2633095044 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9800b9d0 con 0x7f3aa0157030 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.006+0000 7f3aa4c99700 1 -- 172.21.15.45:0/2633095044 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980092e0 con 0x7f3aa0157030 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 -- 172.21.15.45:0/2633095044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa0157030 msgr2=0x7f3aa00cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/2633095044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa0157030 0x7f3aa00cc330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009f20 tx=0x7f3a98009840 comp rx=0 tx=0).stop 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 -- 172.21.15.45:0/2633095044 shutdown_connections 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/2633095044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa0157030 0x7f3aa00cc330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/2633095044 >> v2:172.21.15.144:3300/0 conn(0x7f3aa00c93c0 0x7f3aa0156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 -- 172.21.15.45:0/2633095044 >> 172.21.15.45:0/2633095044 conn(0x7f3aa00cdf50 msgr2=0x7f3aa00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 -- 172.21.15.45:0/2633095044 shutdown_connections 2023-10-06T05:17:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.007+0000 7f3aa7eff700 1 -- 172.21.15.45:0/2633095044 wait complete. 2023-10-06T05:17:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.008+0000 7f3aa7eff700 1 Processor -- start 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.008+0000 7f3aa7eff700 1 -- start start 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa7eff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa00c8ff0 0x7f3aa014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa7eff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa00c93c0 0x7f3aa0068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa7eff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa0157030 0x7f3aa0068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa7eff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa0069eb0 con 0x7f3aa00c93c0 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa7eff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa016dbb0 con 0x7f3aa0157030 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa7eff700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa00692a0 con 0x7f3aa00c8ff0 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa549a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa00c93c0 0x7f3aa0068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa549a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa00c93c0 0x7f3aa0068730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51056/0 (socket says 172.21.15.45:51056) 2023-10-06T05:17:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa549a700 1 -- 172.21.15.45:0/1246731400 learned_addr learned my addr 172.21.15.45:0/1246731400 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa649c700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa0157030 0x7f3aa0068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.009+0000 7f3aa5c9b700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa00c8ff0 0x7f3aa014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa549a700 1 -- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa00c8ff0 msgr2=0x7f3aa014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa549a700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa00c8ff0 0x7f3aa014f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa549a700 1 -- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa0157030 msgr2=0x7f3aa0068c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa549a700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa0157030 0x7f3aa0068c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa549a700 1 -- 172.21.15.45:0/1246731400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98009ae0 con 0x7f3aa00c93c0 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa649c700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa0157030 0x7f3aa0068c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa549a700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa00c93c0 0x7f3aa0068730 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ecb0 tx=0x7f3a900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3a95ffb700 1 -- 172.21.15.45:0/1246731400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000c850 con 0x7f3aa00c93c0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3a95ffb700 1 -- 172.21.15.45:0/1246731400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9000c9b0 con 0x7f3aa00c93c0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa0069580 con 0x7f3aa00c93c0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3a95ffb700 1 -- 172.21.15.45:0/1246731400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000cd50 con 0x7f3aa00c93c0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.010+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa0069b40 con 0x7f3aa00c93c0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.011+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa006a060 con 0x7f3aa00c93c0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.012+0000 7f3a95ffb700 1 -- 172.21.15.45:0/1246731400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a90013680 con 0x7f3aa00c93c0 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.013+0000 7f3a95ffb700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a84066f30 0x7f3a840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.013+0000 7f3a95ffb700 1 -- 172.21.15.45:0/1246731400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f3a90085080 con 0x7f3aa00c93c0 2023-10-06T05:17:55.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.013+0000 7f3aa5c9b700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a84066f30 0x7f3a840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.014+0000 7f3aa5c9b700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a84066f30 0x7f3a840693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c003a40 tx=0x7f3a9c007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.016+0000 7f3a95ffb700 1 -- 172.21.15.45:0/1246731400 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a90052000 con 0x7f3aa00c93c0 2023-10-06T05:17:55.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.266+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3aa006a060 con 0x7f3aa00c93c0 2023-10-06T05:17:55.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.267+0000 7f3a95ffb700 1 -- 172.21.15.45:0/1246731400 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f3aa006a060 con 0x7f3aa00c93c0 2023-10-06T05:17:55.269 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:55.269 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:31.744407+0000","modified":"2023-10-06T05:17:52.613443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5779},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5779":{"gid":5779,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/3967110338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3967110338},{"type":"v1","addr":"172.21.15.45:6835","nonce":3967110338}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:17:55.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.269+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a84066f30 msgr2=0x7f3a840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.269+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a84066f30 0x7f3a840693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c003a40 tx=0x7f3a9c007ec0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.269+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa00c93c0 msgr2=0x7f3aa0068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.269+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa00c93c0 0x7f3aa0068730 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ecb0 tx=0x7f3a900091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 shutdown_connections 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a84066f30 0x7f3a840693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa0157030 0x7f3aa0068c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa00c8ff0 0x7f3aa014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 --2- 172.21.15.45:0/1246731400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa00c93c0 0x7f3aa0068730 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 >> 172.21.15.45:0/1246731400 conn(0x7f3aa00cdf50 msgr2=0x7f3aa00c3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 shutdown_connections 2023-10-06T05:17:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.270+0000 7f3aa7eff700 1 -- 172.21.15.45:0/1246731400 wait complete. 2023-10-06T05:17:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 40 2023-10-06T05:17:55.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.415+0000 7fa599c40700 1 Processor -- start 2023-10-06T05:17:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.415+0000 7fa599c40700 1 -- start start 2023-10-06T05:17:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.416+0000 7fa599c40700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940c8ff0 0x7fa5940c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.416+0000 7fa599c40700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59415b000 con 0x7fa5940caad0 2023-10-06T05:17:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.416+0000 7fa599c40700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59415b2e0 con 0x7fa594157200 2023-10-06T05:17:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.416+0000 7fa599c40700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59415b5c0 con 0x7fa5940c8ff0 2023-10-06T05:17:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.416+0000 7fa592ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa5940caad0 0x7fa594156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41268/0 (socket says 172.21.15.45:41268) 2023-10-06T05:17:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.416+0000 7fa592ffd700 1 -- 172.21.15.45:0/3990217283 learned_addr learned my addr 172.21.15.45:0/3990217283 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3990217283 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940c8ff0 0x7fa5940c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5927fc700 1 -- 172.21.15.45:0/3990217283 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310370438 0 0) 0x7fa59415b000 con 0x7fa5940caad0 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5927fc700 1 -- 172.21.15.45:0/3990217283 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c003570 con 0x7fa5940caad0 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5927fc700 1 -- 172.21.15.45:0/3990217283 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096693574 0 0) 0x7fa59415b2e0 con 0x7fa594157200 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5927fc700 1 -- 172.21.15.45:0/3990217283 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa59415b000 con 0x7fa594157200 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5937fe700 1 -- 172.21.15.45:0/3990217283 >> v1:172.21.15.144:6789/0 conn(0x7fa594157200 legacy=0x7fa5940cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5937fe700 1 -- 172.21.15.45:0/3990217283 >> v1:172.21.15.45:6789/0 conn(0x7fa5940caad0 legacy=0x7fa594156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5937fe700 1 -- 172.21.15.45:0/3990217283 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59415b8a0 con 0x7fa5940c8ff0 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3990217283 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940c8ff0 0x7fa5940c93c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa584009910 tx=0x7fa5840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8605b7e12ac19f0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5927fc700 1 -- 172.21.15.45:0/3990217283 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584011630 con 0x7fa5940c8ff0 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5927fc700 1 -- 172.21.15.45:0/3990217283 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584011790 con 0x7fa5940c8ff0 2023-10-06T05:17:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.417+0000 7fa5927fc700 1 -- 172.21.15.45:0/3990217283 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584011b30 con 0x7fa5940c8ff0 2023-10-06T05:17:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.418+0000 7fa599c40700 1 -- 172.21.15.45:0/3990217283 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940c8ff0 msgr2=0x7fa5940c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.418+0000 7fa599c40700 1 --2- 172.21.15.45:0/3990217283 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940c8ff0 0x7fa5940c93c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa584009910 tx=0x7fa5840091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.418+0000 7fa599c40700 1 -- 172.21.15.45:0/3990217283 shutdown_connections 2023-10-06T05:17:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.418+0000 7fa599c40700 1 --2- 172.21.15.45:0/3990217283 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940c8ff0 0x7fa5940c93c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.418+0000 7fa599c40700 1 -- 172.21.15.45:0/3990217283 >> 172.21.15.45:0/3990217283 conn(0x7fa5940cdf50 msgr2=0x7fa5940ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.418+0000 7fa599c40700 1 -- 172.21.15.45:0/3990217283 shutdown_connections 2023-10-06T05:17:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.418+0000 7fa599c40700 1 -- 172.21.15.45:0/3990217283 wait complete. 2023-10-06T05:17:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.419+0000 7fa599c40700 1 Processor -- start 2023-10-06T05:17:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa599c40700 1 -- start start 2023-10-06T05:17:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa599c40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5940c8ff0 0x7fa59414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa599c40700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940caad0 0x7fa594141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa599c40700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa594157200 0x7fa5940c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa599c40700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c6b00 con 0x7fa5940c8ff0 2023-10-06T05:17:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa599c40700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c7e20 con 0x7fa594157200 2023-10-06T05:17:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa599c40700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c6820 con 0x7fa5940caad0 2023-10-06T05:17:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.420+0000 7fa5937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5940c8ff0 0x7fa59414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5940c8ff0 0x7fa59414e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51062/0 (socket says 172.21.15.45:51062) 2023-10-06T05:17:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 -- 172.21.15.45:0/3708669462 learned_addr learned my addr 172.21.15.45:0/3708669462 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940caad0 0x7fa594141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa593fff700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa594157200 0x7fa5940c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 -- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940caad0 msgr2=0x7fa594141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940caad0 0x7fa594141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 -- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa594157200 msgr2=0x7fa5940c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa594157200 0x7fa5940c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 -- 172.21.15.45:0/3708669462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009420 con 0x7fa5940c8ff0 2023-10-06T05:17:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa5937fe700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5940c8ff0 0x7fa59414e9b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa584009f00 tx=0x7fa584017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa57b7fe700 1 -- 172.21.15.45:0/3708669462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58401f070 con 0x7fa5940c8ff0 2023-10-06T05:17:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa57b7fe700 1 -- 172.21.15.45:0/3708669462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584002f70 con 0x7fa5940c8ff0 2023-10-06T05:17:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa57b7fe700 1 -- 172.21.15.45:0/3708669462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584011900 con 0x7fa5940c8ff0 2023-10-06T05:17:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5940c6e20 con 0x7fa5940c8ff0 2023-10-06T05:17:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.421+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5940c73e0 con 0x7fa5940c8ff0 2023-10-06T05:17:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.422+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5940c7750 con 0x7fa5940c8ff0 2023-10-06T05:17:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.423+0000 7fa57b7fe700 1 -- 172.21.15.45:0/3708669462 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa584023070 con 0x7fa5940c8ff0 2023-10-06T05:17:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.423+0000 7fa57b7fe700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.423+0000 7fa57b7fe700 1 -- 172.21.15.45:0/3708669462 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fa584085b60 con 0x7fa5940c8ff0 2023-10-06T05:17:55.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.423+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.424+0000 7fa592ffd700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa5940c8970 tx=0x7fa588009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.428+0000 7fa57b7fe700 1 -- 172.21.15.45:0/3708669462 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa584052a60 con 0x7fa5940c8ff0 2023-10-06T05:17:55.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.626+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5940c6c80 con 0x7fa5940c8ff0 2023-10-06T05:17:55.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.627+0000 7fa57b7fe700 1 -- 172.21.15.45:0/3708669462 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17296 (secure 0 0 0) 0x7fa5940c6c80 con 0x7fa5940c8ff0 2023-10-06T05:17:55.629 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:17:55.629 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":53,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:17:47.603415+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:29.616613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:30.718338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2812290138":"2023-10-07T05:17:47.603375+0000","172.21.15.45:6833/2812290138":"2023-10-07T05:17:47.603375+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:55.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 msgr2=0x7fa5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa5940c8970 tx=0x7fa588009040 comp rx=0 tx=0).stop 2023-10-06T05:17:55.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5940c8ff0 msgr2=0x7fa59414e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5940c8ff0 0x7fa59414e9b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa584009f00 tx=0x7fa584017000 comp rx=0 tx=0).stop 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 shutdown_connections 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa594157200 0x7fa5940c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5940c8ff0 0x7fa59414e9b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 --2- 172.21.15.45:0/3708669462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5940caad0 0x7fa594141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 >> 172.21.15.45:0/3708669462 conn(0x7fa5940cdf50 msgr2=0x7fa5940c4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 shutdown_connections 2023-10-06T05:17:55.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.629+0000 7fa599c40700 1 -- 172.21.15.45:0/3708669462 wait complete. 2023-10-06T05:17:55.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:17:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.776+0000 7ffa16341700 1 Processor -- start 2023-10-06T05:17:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.776+0000 7ffa16341700 1 -- start start 2023-10-06T05:17:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.777+0000 7ffa16341700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa10155cd0 0x7ffa1015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.777+0000 7ffa16341700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffa100ccdc0 0x7ffa100cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.777+0000 7ffa16341700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa100cd740 con 0x7ffa101560a0 2023-10-06T05:17:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.777+0000 7ffa16341700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa100ae6b0 con 0x7ffa100ccdc0 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.777+0000 7ffa16341700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100ae8d0 con 0x7ffa10155cd0 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.777+0000 7ffa148de700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffa100ccdc0 0x7ffa100cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa0f7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ffa101560a0 0x7ffa100adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:51576/0 (socket says 172.21.15.45:51576) 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.777+0000 7ffa0ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa10155cd0 0x7ffa1015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa0f7fe700 1 -- 172.21.15.45:0/2870642236 learned_addr learned my addr 172.21.15.45:0/2870642236 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa148de700 1 -- 172.21.15.45:0/2870642236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa10155cd0 msgr2=0x7ffa1015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa148de700 1 --2- 172.21.15.45:0/2870642236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa10155cd0 0x7ffa1015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa148de700 1 -- 172.21.15.45:0/2870642236 >> v1:172.21.15.144:6789/0 conn(0x7ffa101560a0 legacy=0x7ffa100adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa148de700 1 -- 172.21.15.45:0/2870642236 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa100aebb0 con 0x7ffa100ccdc0 2023-10-06T05:17:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa0effd700 1 -- 172.21.15.45:0/2870642236 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500949985 0 0) 0x7ffa100cd740 con 0x7ffa101560a0 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa148de700 1 --2- 172.21.15.45:0/2870642236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa100ccdc0 0x7ffa100cd200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e60 tx=0x7ffa08009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6d6c0bd5ee1d87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa0effd700 1 -- 172.21.15.45:0/2870642236 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800e070 con 0x7ffa100ccdc0 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa0effd700 1 -- 172.21.15.45:0/2870642236 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0800bb60 con 0x7ffa100ccdc0 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.778+0000 7ffa0effd700 1 -- 172.21.15.45:0/2870642236 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08012430 con 0x7ffa100ccdc0 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 -- 172.21.15.45:0/2870642236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa100ccdc0 msgr2=0x7ffa100cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 --2- 172.21.15.45:0/2870642236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa100ccdc0 0x7ffa100cd200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e60 tx=0x7ffa08009780 comp rx=0 tx=0).stop 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 -- 172.21.15.45:0/2870642236 shutdown_connections 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 --2- 172.21.15.45:0/2870642236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa100ccdc0 0x7ffa100cd200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 --2- 172.21.15.45:0/2870642236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa10155cd0 0x7ffa1015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 -- 172.21.15.45:0/2870642236 >> 172.21.15.45:0/2870642236 conn(0x7ffa1006f090 msgr2=0x7ffa1006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 -- 172.21.15.45:0/2870642236 shutdown_connections 2023-10-06T05:17:55.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.779+0000 7ffa16341700 1 -- 172.21.15.45:0/2870642236 wait complete. 2023-10-06T05:17:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.781+0000 7ffa16341700 1 Processor -- start 2023-10-06T05:17:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.781+0000 7ffa16341700 1 -- start start 2023-10-06T05:17:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa16341700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa100ccdc0 0x7ffa10155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa16341700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa10155cd0 0x7ffa100c3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa16341700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa101560a0 0x7ffa100a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa16341700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10155560 con 0x7ffa101560a0 2023-10-06T05:17:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa16341700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10166020 con 0x7ffa10155cd0 2023-10-06T05:17:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa16341700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10166270 con 0x7ffa100ccdc0 2023-10-06T05:17:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa101560a0 0x7ffa100a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa101560a0 0x7ffa100a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51072/0 (socket says 172.21.15.45:51072) 2023-10-06T05:17:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 -- 172.21.15.45:0/2803262403 learned_addr learned my addr 172.21.15.45:0/2803262403 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa0f7fe700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa10155cd0 0x7ffa100c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa0ffff700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa100ccdc0 0x7ffa10155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 -- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa100ccdc0 msgr2=0x7ffa10155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa100ccdc0 0x7ffa10155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 -- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa10155cd0 msgr2=0x7ffa100c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa10155cd0 0x7ffa100c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.782+0000 7ffa148de700 1 -- 172.21.15.45:0/2803262403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa08009a20 con 0x7ffa101560a0 2023-10-06T05:17:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.783+0000 7ffa148de700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa101560a0 0x7ffa100a5e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b6c0 tx=0x7ffa08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.783+0000 7ffa07fff700 1 -- 172.21.15.45:0/2803262403 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08020630 con 0x7ffa101560a0 2023-10-06T05:17:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.783+0000 7ffa07fff700 1 -- 172.21.15.45:0/2803262403 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa08020790 con 0x7ffa101560a0 2023-10-06T05:17:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.783+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa10166550 con 0x7ffa101560a0 2023-10-06T05:17:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.783+0000 7ffa07fff700 1 -- 172.21.15.45:0/2803262403 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08020b50 con 0x7ffa101560a0 2023-10-06T05:17:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.783+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa10166b10 con 0x7ffa101560a0 2023-10-06T05:17:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.784+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1006e2f0 con 0x7ffa101560a0 2023-10-06T05:17:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.784+0000 7ffa07fff700 1 -- 172.21.15.45:0/2803262403 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa08026070 con 0x7ffa101560a0 2023-10-06T05:17:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.785+0000 7ffa07fff700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff9f0066f30 0x7ff9f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.785+0000 7ffa07fff700 1 -- 172.21.15.45:0/2803262403 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ffa08089110 con 0x7ffa101560a0 2023-10-06T05:17:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.785+0000 7ffa0ffff700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff9f0066f30 0x7ff9f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.786+0000 7ffa0ffff700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff9f0066f30 0x7ff9f00693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa100b1aa0 tx=0x7ffa000091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:55.789+0000 7ffa07fff700 1 -- 172.21.15.45:0/2803262403 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa08056090 con 0x7ffa101560a0 2023-10-06T05:17:56.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.040+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ffa100aee90 con 0x7ffa101560a0 2023-10-06T05:17:56.042 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:17:56.041+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:17:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.044+0000 7ffa07fff700 1 -- 172.21.15.45:0/2803262403 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7ffa080596b0 con 0x7ffa101560a0 2023-10-06T05:17:56.049 INFO:tasks.ceph.mds.c.smithi045.stderr: -59> 2023-10-06T05:17:53.612+0000 7f71d24f3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:17:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff9f0066f30 msgr2=0x7ff9f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff9f0066f30 0x7ff9f00693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa100b1aa0 tx=0x7ffa000091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa101560a0 msgr2=0x7ffa100a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa101560a0 0x7ffa100a5e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b6c0 tx=0x7ffa08019000 comp rx=0 tx=0).stop 2023-10-06T05:17:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 shutdown_connections 2023-10-06T05:17:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff9f0066f30 0x7ff9f00693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa101560a0 0x7ffa100a5e20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa100ccdc0 0x7ffa10155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 --2- 172.21.15.45:0/2803262403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa10155cd0 0x7ffa100c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 >> 172.21.15.45:0/2803262403 conn(0x7ffa1006f090 msgr2=0x7ffa10070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 shutdown_connections 2023-10-06T05:17:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.048+0000 7ffa16341700 1 -- 172.21.15.45:0/2803262403 wait complete. 2023-10-06T05:17:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:17:56.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:17:56.070 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:17:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.196+0000 7f4717ef1700 1 Processor -- start 2023-10-06T05:17:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.197+0000 7f4717ef1700 1 -- start start 2023-10-06T05:17:56.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.197+0000 7f4717ef1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159520 0x7f471015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.197+0000 7f4717ef1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f471015d3d0 0x7f4710067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.197+0000 7f4717ef1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47100c9c90 con 0x7f4710159150 2023-10-06T05:17:56.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.197+0000 7f4717ef1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471015d8e0 con 0x7f471015d3d0 2023-10-06T05:17:56.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.197+0000 7f4717ef1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471015dbc0 con 0x7f4710159520 2023-10-06T05:17:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f471648e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f471015d3d0 0x7f4710067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f4715c8d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4710159150 0x7f47100ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:51600/0 (socket says 172.21.15.45:51600) 2023-10-06T05:17:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f4715c8d700 1 -- 172.21.15.45:0/3319359110 learned_addr learned my addr 172.21.15.45:0/3319359110 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f471548c700 1 --2- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159520 0x7f471015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f471648e700 1 -- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159520 msgr2=0x7f471015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f471648e700 1 --2- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159520 0x7f471015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f471648e700 1 -- 172.21.15.45:0/3319359110 >> v1:172.21.15.144:6789/0 conn(0x7f4710159150 legacy=0x7f47100ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f471648e700 1 -- 172.21.15.45:0/3319359110 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100c9920 con 0x7f471015d3d0 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.198+0000 7f471648e700 1 --2- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471015d3d0 0x7f4710067620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4708009e60 tx=0x7f4708009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef57b2628fa5bf9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4714c8b700 1 -- 172.21.15.45:0/3319359110 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800e070 con 0x7f471015d3d0 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4714c8b700 1 -- 172.21.15.45:0/3319359110 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800bb60 con 0x7f471015d3d0 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4714c8b700 1 -- 172.21.15.45:0/3319359110 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708012430 con 0x7f471015d3d0 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 -- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471015d3d0 msgr2=0x7f4710067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 --2- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471015d3d0 0x7f4710067620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4708009e60 tx=0x7f4708009780 comp rx=0 tx=0).stop 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 -- 172.21.15.45:0/3319359110 shutdown_connections 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 --2- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471015d3d0 0x7f4710067620 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 --2- 172.21.15.45:0/3319359110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159520 0x7f471015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 -- 172.21.15.45:0/3319359110 >> 172.21.15.45:0/3319359110 conn(0x7f47100691c0 msgr2=0x7f47100cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:56.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 -- 172.21.15.45:0/3319359110 shutdown_connections 2023-10-06T05:17:56.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.199+0000 7f4717ef1700 1 -- 172.21.15.45:0/3319359110 wait complete. 2023-10-06T05:17:56.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.200+0000 7f4717ef1700 1 Processor -- start 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.200+0000 7f4717ef1700 1 -- start start 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f4717ef1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159150 0x7f47100adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f4717ef1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471015d3d0 0x7f471016a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f4717ef1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47100c46e0 0x7f47100c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f4717ef1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710169980 con 0x7f471015d3d0 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f4717ef1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710169c60 con 0x7f47100c46e0 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f4717ef1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4710169f40 con 0x7f4710159150 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f471548c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471015d3d0 0x7f471016a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f471548c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471015d3d0 0x7f471016a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51100/0 (socket says 172.21.15.45:51100) 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f471548c700 1 -- 172.21.15.45:0/507913215 learned_addr learned my addr 172.21.15.45:0/507913215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f4715c8d700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159150 0x7f47100adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.201+0000 7f471648e700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47100c46e0 0x7f47100c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f471548c700 1 -- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159150 msgr2=0x7f47100adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f471548c700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159150 0x7f47100adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f471548c700 1 -- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47100c46e0 msgr2=0x7f47100c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f471548c700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47100c46e0 0x7f47100c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f471548c700 1 -- 172.21.15.45:0/507913215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4708009a20 con 0x7f471015d3d0 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f471548c700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471015d3d0 0x7f471016a590 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f470c00ec80 tx=0x7f470c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f4705ffb700 1 -- 172.21.15.45:0/507913215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00c850 con 0x7f471015d3d0 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f4705ffb700 1 -- 172.21.15.45:0/507913215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00c9b0 con 0x7f471015d3d0 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471016a220 con 0x7f471015d3d0 2023-10-06T05:17:56.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f4705ffb700 1 -- 172.21.15.45:0/507913215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00cd50 con 0x7f471015d3d0 2023-10-06T05:17:56.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.202+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4710169580 con 0x7f471015d3d0 2023-10-06T05:17:56.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.203+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471016cc30 con 0x7f471015d3d0 2023-10-06T05:17:56.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.204+0000 7f4705ffb700 1 -- 172.21.15.45:0/507913215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f470c013680 con 0x7f471015d3d0 2023-10-06T05:17:56.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.204+0000 7f4705ffb700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46f4066fb0 0x7f46f4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.204+0000 7f4705ffb700 1 -- 172.21.15.45:0/507913215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f470c084b40 con 0x7f471015d3d0 2023-10-06T05:17:56.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.204+0000 7f4715c8d700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46f4066fb0 0x7f46f4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.205+0000 7f4715c8d700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46f4066fb0 0x7f46f4069460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47100c8fe0 tx=0x7f4700009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.209+0000 7f4705ffb700 1 -- 172.21.15.45:0/507913215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f471016cc30 con 0x7f471015d3d0 2023-10-06T05:17:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:56.460+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f471016cbf0 con 0x7f471015d3d0 2023-10-06T05:17:57.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.048+0000 7f4705ffb700 1 -- 172.21.15.45:0/507913215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f470c0519c0 con 0x7f471015d3d0 2023-10-06T05:17:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46f4066fb0 msgr2=0x7f46f4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46f4066fb0 0x7f46f4069460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47100c8fe0 tx=0x7f4700009040 comp rx=0 tx=0).stop 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471015d3d0 msgr2=0x7f471016a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471015d3d0 0x7f471016a590 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f470c00ec80 tx=0x7f470c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 shutdown_connections 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46f4066fb0 0x7f46f4069460 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47100c46e0 0x7f47100c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4710159150 0x7f47100adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 --2- 172.21.15.45:0/507913215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471015d3d0 0x7f471016a590 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.051+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 >> 172.21.15.45:0/507913215 conn(0x7f47100691c0 msgr2=0x7f4710156460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.052+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 shutdown_connections 2023-10-06T05:17:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.052+0000 7f4717ef1700 1 -- 172.21.15.45:0/507913215 wait complete. 2023-10-06T05:17:57.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:17:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.252+0000 7fd9d5adb700 1 Processor -- start 2023-10-06T05:17:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.252+0000 7fd9d5adb700 1 -- start start 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9d5adb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9d5adb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9d015d3d0 0x7fd9d00c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9d5adb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d0068040 con 0x7fd9d01590c0 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9d5adb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d015d850 con 0x7fd9d015d3d0 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9d5adb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d015dcf0 con 0x7fd9d0159490 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9cffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9d015d3d0 0x7fd9d00c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9cffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9d015d3d0 0x7fd9d00c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36072/0 (socket says 172.21.15.45:36072) 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9cffff700 1 -- 172.21.15.45:0/2719065884 learned_addr learned my addr 172.21.15.45:0/2719065884 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.253+0000 7fd9ceffd700 1 --2- 172.21.15.45:0/2719065884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9cffff700 1 -- 172.21.15.45:0/2719065884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 msgr2=0x7fd9d015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9cffff700 1 --2- 172.21.15.45:0/2719065884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9cffff700 1 -- 172.21.15.45:0/2719065884 >> v1:172.21.15.144:6789/0 conn(0x7fd9d01590c0 legacy=0x7fd9d00ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9cffff700 1 -- 172.21.15.45:0/2719065884 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0067750 con 0x7fd9d015d3d0 2023-10-06T05:17:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9cffff700 1 --2- 172.21.15.45:0/2719065884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00c74f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009f20 tx=0x7fd9c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76b604f77554ed0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9ce7fc700 1 -- 172.21.15.45:0/2719065884 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800e070 con 0x7fd9d015d3d0 2023-10-06T05:17:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9ce7fc700 1 -- 172.21.15.45:0/2719065884 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c800ba80 con 0x7fd9d015d3d0 2023-10-06T05:17:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9ce7fc700 1 -- 172.21.15.45:0/2719065884 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80092e0 con 0x7fd9d015d3d0 2023-10-06T05:17:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9d5adb700 1 -- 172.21.15.45:0/2719065884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 msgr2=0x7fd9d00c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.254+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/2719065884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00c74f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009f20 tx=0x7fd9c8009840 comp rx=0 tx=0).stop 2023-10-06T05:17:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.255+0000 7fd9d5adb700 1 -- 172.21.15.45:0/2719065884 shutdown_connections 2023-10-06T05:17:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.255+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/2719065884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00c74f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.255+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/2719065884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.255+0000 7fd9d5adb700 1 -- 172.21.15.45:0/2719065884 >> 172.21.15.45:0/2719065884 conn(0x7fd9d00cd060 msgr2=0x7fd9d00c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:57.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.255+0000 7fd9d5adb700 1 -- 172.21.15.45:0/2719065884 shutdown_connections 2023-10-06T05:17:57.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.255+0000 7fd9d5adb700 1 -- 172.21.15.45:0/2719065884 wait complete. 2023-10-06T05:17:57.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.256+0000 7fd9d5adb700 1 Processor -- start 2023-10-06T05:17:57.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.256+0000 7fd9d5adb700 1 -- start start 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9d5adb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9d01590c0 0x7fd9d016d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9d5adb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9d5adb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9d5adb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d016d710 con 0x7fd9d015d3d0 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9d5adb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d015c800 con 0x7fd9d01590c0 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9d5adb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d015be30 con 0x7fd9d0159490 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36082/0 (socket says 172.21.15.45:36082) 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 -- 172.21.15.45:0/3947358851 learned_addr learned my addr 172.21.15.45:0/3947358851 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cf7fe700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9d01590c0 0x7fd9d016d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9ceffd700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 -- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 msgr2=0x7fd9d015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 -- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9d01590c0 msgr2=0x7fd9d016d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9d01590c0 0x7fd9d016d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 -- 172.21.15.45:0/3947358851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8009ae0 con 0x7fd9d015d3d0 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.257+0000 7fd9cffff700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00b1c50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8008950 tx=0x7fd9c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.258+0000 7fd9c77fe700 1 -- 172.21.15.45:0/3947358851 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80092e0 con 0x7fd9d015d3d0 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.258+0000 7fd9c77fe700 1 -- 172.21.15.45:0/3947358851 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c8008df0 con 0x7fd9d015d3d0 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.258+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d00b2190 con 0x7fd9d015d3d0 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.258+0000 7fd9c77fe700 1 -- 172.21.15.45:0/3947358851 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8026690 con 0x7fd9d015d3d0 2023-10-06T05:17:57.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.258+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d0159e40 con 0x7fd9d015d3d0 2023-10-06T05:17:57.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.259+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d015bfa0 con 0x7fd9d015d3d0 2023-10-06T05:17:57.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.259+0000 7fd9c77fe700 1 -- 172.21.15.45:0/3947358851 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9c8003040 con 0x7fd9d015d3d0 2023-10-06T05:17:57.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.259+0000 7fd9c77fe700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9b0066f30 0x7fd9b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.260+0000 7fd9c77fe700 1 -- 172.21.15.45:0/3947358851 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fd9c8086970 con 0x7fd9d015d3d0 2023-10-06T05:17:57.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.260+0000 7fd9cf7fe700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9b0066f30 0x7fd9b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.260+0000 7fd9cf7fe700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9b0066f30 0x7fd9b00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00039e0 tx=0x7fd9c0007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:57.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.264+0000 7fd9c77fe700 1 -- 172.21.15.45:0/3947358851 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9c8053890 con 0x7fd9d015d3d0 2023-10-06T05:17:57.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:57.471+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9d004c380 con 0x7fd9d015d3d0 2023-10-06T05:17:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.050+0000 7fd9c77fe700 1 -- 172.21.15.45:0/3947358851 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd9c8056eb0 con 0x7fd9d015d3d0 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9b0066f30 msgr2=0x7fd9b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9b0066f30 0x7fd9b00693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00039e0 tx=0x7fd9c0007ec0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 msgr2=0x7fd9d00b1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00b1c50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8008950 tx=0x7fd9c8018000 comp rx=0 tx=0).stop 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 shutdown_connections 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9b0066f30 0x7fd9b00693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d015d3d0 0x7fd9d00b1c50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9d01590c0 0x7fd9d016d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 --2- 172.21.15.45:0/3947358851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9d0159490 0x7fd9d015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 >> 172.21.15.45:0/3947358851 conn(0x7fd9d00cd060 msgr2=0x7fd9d00ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 shutdown_connections 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.053+0000 7fd9d5adb700 1 -- 172.21.15.45:0/3947358851 wait complete. 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:17:58.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:17:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.332+0000 7fe259595700 1 Processor -- start 2023-10-06T05:17:58.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.332+0000 7fe259595700 1 -- start start 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.333+0000 7fe259595700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2540c8460 0x7fe25406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.333+0000 7fe259595700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2540ace00 0x7fe25406ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.333+0000 7fe259595700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2540680f0 con 0x7fe254155660 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.333+0000 7fe259595700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25406f220 con 0x7fe2540c8460 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.333+0000 7fe259595700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540677c0 con 0x7fe2540ace00 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe2537fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2540ace00 0x7fe25406ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe2527fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2540c8460 0x7fe25406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe252ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe254155660 0x7fe2540c7d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:44422/0 (socket says 172.21.15.45:44422) 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe252ffd700 1 -- 172.21.15.45:0/3308235639 learned_addr learned my addr 172.21.15.45:0/3308235639 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:58.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe2537fe700 1 -- 172.21.15.45:0/3308235639 >> v2:172.21.15.144:3300/0 conn(0x7fe2540c8460 msgr2=0x7fe25406e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe2537fe700 1 --2- 172.21.15.45:0/3308235639 >> v2:172.21.15.144:3300/0 conn(0x7fe2540c8460 0x7fe25406e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe2537fe700 1 -- 172.21.15.45:0/3308235639 >> v1:172.21.15.144:6789/0 conn(0x7fe254155660 legacy=0x7fe2540c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe251ffb700 1 -- 172.21.15.45:0/3308235639 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122734140 0 0) 0x7fe2540680f0 con 0x7fe254155660 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe2537fe700 1 -- 172.21.15.45:0/3308235639 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254067aa0 con 0x7fe2540ace00 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.334+0000 7fe2527fc700 1 --2- 172.21.15.45:0/3308235639 >> v2:172.21.15.144:3300/0 conn(0x7fe2540c8460 0x7fe25406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe2537fe700 1 --2- 172.21.15.45:0/3308235639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2540ace00 0x7fe25406ece0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009e60 tx=0x7fe24c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc2bd342d1cecce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe251ffb700 1 -- 172.21.15.45:0/3308235639 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00bb60 con 0x7fe2540ace00 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe251ffb700 1 -- 172.21.15.45:0/3308235639 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c00bcc0 con 0x7fe2540ace00 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe251ffb700 1 -- 172.21.15.45:0/3308235639 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0196d0 con 0x7fe2540ace00 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe259595700 1 -- 172.21.15.45:0/3308235639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2540ace00 msgr2=0x7fe25406ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe259595700 1 --2- 172.21.15.45:0/3308235639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2540ace00 0x7fe25406ece0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009e60 tx=0x7fe24c009780 comp rx=0 tx=0).stop 2023-10-06T05:17:58.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe259595700 1 -- 172.21.15.45:0/3308235639 shutdown_connections 2023-10-06T05:17:58.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe259595700 1 --2- 172.21.15.45:0/3308235639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2540ace00 0x7fe25406ece0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe259595700 1 --2- 172.21.15.45:0/3308235639 >> v2:172.21.15.144:3300/0 conn(0x7fe2540c8460 0x7fe25406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe259595700 1 -- 172.21.15.45:0/3308235639 >> 172.21.15.45:0/3308235639 conn(0x7fe2540cd060 msgr2=0x7fe2540c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:58.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.335+0000 7fe259595700 1 -- 172.21.15.45:0/3308235639 shutdown_connections 2023-10-06T05:17:58.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.336+0000 7fe259595700 1 -- 172.21.15.45:0/3308235639 wait complete. 2023-10-06T05:17:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.337+0000 7fe259595700 1 Processor -- start 2023-10-06T05:17:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.337+0000 7fe259595700 1 -- start start 2023-10-06T05:17:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe259595700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2540ace00 0x7fe2541579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe259595700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2540c8460 0x7fe254155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe259595700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe254155660 0x7fe254155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe259595700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254158330 con 0x7fe2540c8460 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe259595700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254158610 con 0x7fe2540ace00 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe259595700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2541588f0 con 0x7fe254155660 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2540c8460 0x7fe254155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2540c8460 0x7fe254155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36092/0 (socket says 172.21.15.45:36092) 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 -- 172.21.15.45:0/1367669339 learned_addr learned my addr 172.21.15.45:0/1367669339 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2537fe700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe254155660 0x7fe254155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe252ffd700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2540ace00 0x7fe2541579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 -- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe254155660 msgr2=0x7fe254155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe254155660 0x7fe254155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 -- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2540ace00 msgr2=0x7fe2541579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2540ace00 0x7fe2541579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 -- 172.21.15.45:0/1367669339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c009420 con 0x7fe2540c8460 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.338+0000 7fe2527fc700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2540c8460 0x7fe254155a30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00ec80 tx=0x7fe23c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.339+0000 7fe24affd700 1 -- 172.21.15.45:0/1367669339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00c9d0 con 0x7fe2540c8460 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.339+0000 7fe24affd700 1 -- 172.21.15.45:0/1367669339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c00cb30 con 0x7fe2540c8460 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.339+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c009a20 con 0x7fe2540c8460 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.339+0000 7fe24affd700 1 -- 172.21.15.45:0/1367669339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00ceb0 con 0x7fe2540c8460 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.339+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2540b0310 con 0x7fe2540c8460 2023-10-06T05:17:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.340+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2540b0680 con 0x7fe2540c8460 2023-10-06T05:17:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.340+0000 7fe24affd700 1 -- 172.21.15.45:0/1367669339 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe23c013680 con 0x7fe2540c8460 2023-10-06T05:17:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.341+0000 7fe24affd700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe234066f30 0x7fe2340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.341+0000 7fe24affd700 1 -- 172.21.15.45:0/1367669339 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fe23c084b90 con 0x7fe2540c8460 2023-10-06T05:17:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.341+0000 7fe252ffd700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe234066f30 0x7fe2340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.341+0000 7fe252ffd700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe234066f30 0x7fe2340693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe24400c8f0 tx=0x7fe244009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.345+0000 7fe24affd700 1 -- 172.21.15.45:0/1367669339 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe23c051c70 con 0x7fe2540c8460 2023-10-06T05:17:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:58.545+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe254158470 con 0x7fe2540c8460 2023-10-06T05:17:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.055+0000 7fe24affd700 1 -- 172.21.15.45:0/1367669339 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe254158470 con 0x7fe2540c8460 2023-10-06T05:17:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe234066f30 msgr2=0x7fe2340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe234066f30 0x7fe2340693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe24400c8f0 tx=0x7fe244009040 comp rx=0 tx=0).stop 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2540c8460 msgr2=0x7fe254155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2540c8460 0x7fe254155a30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00ec80 tx=0x7fe23c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 shutdown_connections 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe234066f30 0x7fe2340693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe254155660 0x7fe254155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2540ace00 0x7fe2541579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 --2- 172.21.15.45:0/1367669339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2540c8460 0x7fe254155a30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 >> 172.21.15.45:0/1367669339 conn(0x7fe2540cd060 msgr2=0x7fe2540632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 shutdown_connections 2023-10-06T05:17:59.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.058+0000 7fe259595700 1 -- 172.21.15.45:0/1367669339 wait complete. 2023-10-06T05:17:59.212 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-06T05:17:59.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:17:59.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.359+0000 7f3ad0379700 1 Processor -- start 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.359+0000 7f3ad0379700 1 -- start start 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.361+0000 7f3ad0379700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ac80c6a70 0x7f3ac806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.361+0000 7f3ad0379700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80acb70 con 0x7f3ac8155260 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.361+0000 7f3ad0379700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac806eef0 con 0x7f3ac8155630 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.361+0000 7f3ad0379700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac806f1d0 con 0x7f3ac80c6a70 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.361+0000 7f3ace916700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ac80c6a70 0x7f3ac806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.361+0000 7f3ace916700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ac80c6a70 0x7f3ac806ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36108/0 (socket says 172.21.15.45:36108) 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.361+0000 7f3ace916700 1 -- 172.21.15.45:0/340445463 learned_addr learned my addr 172.21.15.45:0/340445463 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:59.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.362+0000 7f3ace916700 1 -- 172.21.15.45:0/340445463 >> v1:172.21.15.144:6789/0 conn(0x7f3ac8155630 legacy=0x7f3ac80c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.362+0000 7f3ace916700 1 -- 172.21.15.45:0/340445463 >> v1:172.21.15.45:6789/0 conn(0x7f3ac8155260 legacy=0x7f3ac80ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.362+0000 7f3ace916700 1 -- 172.21.15.45:0/340445463 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80ac800 con 0x7f3ac80c6a70 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.362+0000 7f3ace916700 1 --2- 172.21.15.45:0/340445463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac806ba80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40099b0 tx=0x7f3ac4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6c2c8b30120dc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.363+0000 7f3acd113700 1 -- 172.21.15.45:0/340445463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400b860 con 0x7f3ac80c6a70 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.363+0000 7f3acd113700 1 -- 172.21.15.45:0/340445463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac400b9c0 con 0x7f3ac80c6a70 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.363+0000 7f3acd113700 1 -- 172.21.15.45:0/340445463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400bd40 con 0x7f3ac80c6a70 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.364+0000 7f3ad0379700 1 -- 172.21.15.45:0/340445463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 msgr2=0x7f3ac806ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.364+0000 7f3ad0379700 1 --2- 172.21.15.45:0/340445463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac806ba80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40099b0 tx=0x7f3ac4009250 comp rx=0 tx=0).stop 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.364+0000 7f3ad0379700 1 -- 172.21.15.45:0/340445463 shutdown_connections 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.364+0000 7f3ad0379700 1 --2- 172.21.15.45:0/340445463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac806ba80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.364+0000 7f3ad0379700 1 -- 172.21.15.45:0/340445463 >> 172.21.15.45:0/340445463 conn(0x7f3ac80cbd20 msgr2=0x7f3ac80cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.364+0000 7f3ad0379700 1 -- 172.21.15.45:0/340445463 shutdown_connections 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.364+0000 7f3ad0379700 1 -- 172.21.15.45:0/340445463 wait complete. 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.366+0000 7f3ad0379700 1 Processor -- start 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.366+0000 7f3ad0379700 1 -- start start 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ad0379700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac80af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ad0379700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ac8155260 0x7f3ac80c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ad0379700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ac8155630 0x7f3ac80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ad0379700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80b1be0 con 0x7f3ac80c6a70 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ad0379700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80c5b30 con 0x7f3ac8155260 2023-10-06T05:17:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ad0379700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80c4f20 con 0x7f3ac8155630 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac80af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac80af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36116/0 (socket says 172.21.15.45:36116) 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 -- 172.21.15.45:0/4050246044 learned_addr learned my addr 172.21.15.45:0/4050246044 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3acd914700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ac8155260 0x7f3ac80c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace916700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ac8155630 0x7f3ac80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 -- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ac8155630 msgr2=0x7f3ac80c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ac8155630 0x7f3ac80c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 -- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ac8155260 msgr2=0x7f3ac80c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ac8155260 0x7f3ac80c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 -- 172.21.15.45:0/4050246044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac40094f0 con 0x7f3ac80c6a70 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.368+0000 7f3ace115700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac80af380 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c790 tx=0x7f3ab8007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.369+0000 7f3abe7fc700 1 -- 172.21.15.45:0/4050246044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8011590 con 0x7f3ac80c6a70 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.369+0000 7f3abe7fc700 1 -- 172.21.15.45:0/4050246044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8011710 con 0x7f3ac80c6a70 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.369+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac80c5200 con 0x7f3ac80c6a70 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.369+0000 7f3abe7fc700 1 -- 172.21.15.45:0/4050246044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8011ad0 con 0x7f3ac80c6a70 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.369+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac80c57c0 con 0x7f3ac80c6a70 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.370+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac80c5e10 con 0x7f3ac80c6a70 2023-10-06T05:17:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.370+0000 7f3abe7fc700 1 -- 172.21.15.45:0/4050246044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ab801b3b0 con 0x7f3ac80c6a70 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.371+0000 7f3abe7fc700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3aac066f30 0x7f3aac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.371+0000 7f3abe7fc700 1 -- 172.21.15.45:0/4050246044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f3ab8083920 con 0x7f3ac80c6a70 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.371+0000 7f3acd914700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3aac066f30 0x7f3aac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.372+0000 7f3acd914700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3aac066f30 0x7f3aac0693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8154290 tx=0x7f3ac0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.375+0000 7f3abe7fc700 1 -- 172.21.15.45:0/4050246044 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ab8050b90 con 0x7f3ac80c6a70 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.578+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3ac80ad710 con 0x7f3ac80c6a70 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.582+0000 7f3abe7fc700 1 -- 172.21.15.45:0/4050246044 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ab8017020 con 0x7f3ac80c6a70 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3aac066f30 msgr2=0x7f3aac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3aac066f30 0x7f3aac0693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8154290 tx=0x7f3ac0009040 comp rx=0 tx=0).stop 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 msgr2=0x7f3ac80af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac80af380 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c790 tx=0x7f3ab8007f40 comp rx=0 tx=0).stop 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 shutdown_connections 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3aac066f30 0x7f3aac0693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ac8155630 0x7f3ac80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac80c6a70 0x7f3ac80af380 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 --2- 172.21.15.45:0/4050246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ac8155260 0x7f3ac80c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 >> 172.21.15.45:0/4050246044 conn(0x7f3ac80cbd20 msgr2=0x7f3ac8157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 shutdown_connections 2023-10-06T05:17:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.584+0000 7f3ad0379700 1 -- 172.21.15.45:0/4050246044 wait complete. 2023-10-06T05:17:59.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-10-06T05:17:59.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.766+0000 7f1139158700 1 Processor -- start 2023-10-06T05:17:59.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.766+0000 7f1139158700 1 -- start start 2023-10-06T05:17:59.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f1139158700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f113406ae40 0x7f113415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f1139158700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1134067d80 con 0x7f113406aa70 2023-10-06T05:17:59.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f1139158700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113415cb70 con 0x7f113415c6f0 2023-10-06T05:17:59.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f1139158700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11340ac620 con 0x7f113406ae40 2023-10-06T05:17:59.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f113259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f113406ae40 0x7f113415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f1132d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f113406aa70 0x7f11340c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35424/0 (socket says 172.21.15.45:35424) 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f1132d9d700 1 -- 172.21.15.45:0/1280067133 learned_addr learned my addr 172.21.15.45:0/1280067133 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f113259c700 1 -- 172.21.15.45:0/1280067133 >> v1:172.21.15.144:6789/0 conn(0x7f113415c6f0 legacy=0x7f11340aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f113259c700 1 -- 172.21.15.45:0/1280067133 >> v1:172.21.15.45:6789/0 conn(0x7f113406aa70 legacy=0x7f11340c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f113259c700 1 -- 172.21.15.45:0/1280067133 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340c75b0 con 0x7f113406ae40 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.767+0000 7f113259c700 1 --2- 172.21.15.45:0/1280067133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f113406ae40 0x7f113415c1b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11280098e0 tx=0x7f11280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5885da6e957910d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1131d9b700 1 -- 172.21.15.45:0/1280067133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128009e80 con 0x7f113406ae40 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1131d9b700 1 -- 172.21.15.45:0/1280067133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112800b860 con 0x7f113406ae40 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1131d9b700 1 -- 172.21.15.45:0/1280067133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800bbe0 con 0x7f113406ae40 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1139158700 1 -- 172.21.15.45:0/1280067133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f113406ae40 msgr2=0x7f113415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1139158700 1 --2- 172.21.15.45:0/1280067133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f113406ae40 0x7f113415c1b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11280098e0 tx=0x7f11280091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1139158700 1 -- 172.21.15.45:0/1280067133 shutdown_connections 2023-10-06T05:17:59.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1139158700 1 --2- 172.21.15.45:0/1280067133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f113406ae40 0x7f113415c1b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1139158700 1 -- 172.21.15.45:0/1280067133 >> 172.21.15.45:0/1280067133 conn(0x7f11340cc570 msgr2=0x7f11340cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:59.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1139158700 1 -- 172.21.15.45:0/1280067133 shutdown_connections 2023-10-06T05:17:59.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.768+0000 7f1139158700 1 -- 172.21.15.45:0/1280067133 wait complete. 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.769+0000 7f1139158700 1 Processor -- start 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f1139158700 1 -- start start 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f1139158700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f113406aa70 0x7f11340b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f1139158700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f113415c6f0 0x7f11340c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f1139158700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11340c3580 0x7f113416ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f1139158700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134156630 con 0x7f11340c3580 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f1139158700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134169c50 con 0x7f113415c6f0 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f1139158700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340c3af0 con 0x7f113406aa70 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f113359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11340c3580 0x7f113416ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f113359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11340c3580 0x7f113416ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36120/0 (socket says 172.21.15.45:36120) 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.770+0000 7f113359e700 1 -- 172.21.15.45:0/4195577368 learned_addr learned my addr 172.21.15.45:0/4195577368 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f113259c700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f113415c6f0 0x7f11340c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f1132d9d700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f113406aa70 0x7f11340b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f113359e700 1 -- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f113406aa70 msgr2=0x7f11340b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f113359e700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f113406aa70 0x7f11340b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f113359e700 1 -- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f113415c6f0 msgr2=0x7f11340c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f113359e700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f113415c6f0 0x7f11340c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f113359e700 1 -- 172.21.15.45:0/4195577368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128009420 con 0x7f11340c3580 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f113359e700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11340c3580 0x7f113416ac80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c8e0 tx=0x7f112c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f111affd700 1 -- 172.21.15.45:0/4195577368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c012630 con 0x7f11340c3580 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f111affd700 1 -- 172.21.15.45:0/4195577368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c012790 con 0x7f11340c3580 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f111affd700 1 -- 172.21.15.45:0/4195577368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c012b50 con 0x7f11340c3580 2023-10-06T05:17:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11340c3dd0 con 0x7f11340c3580 2023-10-06T05:17:59.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.771+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11340c4390 con 0x7f11340c3580 2023-10-06T05:17:59.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.772+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134169de0 con 0x7f11340c3580 2023-10-06T05:17:59.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.773+0000 7f111affd700 1 -- 172.21.15.45:0/4195577368 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f112c01a3a0 con 0x7f11340c3580 2023-10-06T05:17:59.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.773+0000 7f111affd700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.773+0000 7f111affd700 1 -- 172.21.15.45:0/4195577368 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f112c084670 con 0x7f11340c3580 2023-10-06T05:17:59.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.773+0000 7f1132d9d700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.774+0000 7f1132d9d700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1120066f30 0x7f11200693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f11340ca730 tx=0x7f1124007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.778+0000 7f111affd700 1 -- 172.21.15.45:0/4195577368 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f112c055370 con 0x7f11340c3580 2023-10-06T05:17:59.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.980+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7f11340c7890 con 0x7f11340c3580 2023-10-06T05:18:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.980+0000 7f111affd700 1 -- 172.21.15.45:0/4195577368 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v16) v1 ==== 121+0+0 (secure 0 0 0) 0x7f112c054f00 con 0x7f11340c3580 2023-10-06T05:18:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1120066f30 msgr2=0x7f11200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1120066f30 0x7f11200693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f11340ca730 tx=0x7f1124007eb0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11340c3580 msgr2=0x7f113416ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11340c3580 0x7f113416ac80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c8e0 tx=0x7f112c001710 comp rx=0 tx=0).stop 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 shutdown_connections 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1120066f30 0x7f11200693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11340c3580 0x7f113416ac80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f113406aa70 0x7f11340b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 --2- 172.21.15.45:0/4195577368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f113415c6f0 0x7f11340c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 >> 172.21.15.45:0/4195577368 conn(0x7f11340cc570 msgr2=0x7f11340caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 shutdown_connections 2023-10-06T05:18:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:17:59.983+0000 7f1139158700 1 -- 172.21.15.45:0/4195577368 wait complete. 2023-10-06T05:18:00.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_abort_on_newly_corrupt_dentry 2023-10-06T05:18:00.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.167+0000 7f152a133700 1 Processor -- start 2023-10-06T05:18:00.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.167+0000 7f152a133700 1 -- start start 2023-10-06T05:18:00.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f152a133700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1524140ab0 0x7f15240acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f152a133700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f152a133700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15240c9940 con 0x7f15241406e0 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f152a133700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152406d230 con 0x7f1524140ab0 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f152a133700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152406d510 con 0x7f15240ad4e0 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f1522ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1524140ab0 0x7f15240acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f15237fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f15241406e0 0x7f152406c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35434/0 (socket says 172.21.15.45:35434) 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f15237fe700 1 -- 172.21.15.45:0/1182112588 learned_addr learned my addr 172.21.15.45:0/1182112588 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f1523fff700 1 --2- 172.21.15.45:0/1182112588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f1522ffd700 1 -- 172.21.15.45:0/1182112588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 msgr2=0x7f15240c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f1522ffd700 1 --2- 172.21.15.45:0/1182112588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f1522ffd700 1 -- 172.21.15.45:0/1182112588 >> v1:172.21.15.45:6789/0 conn(0x7f15241406e0 legacy=0x7f152406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f1522ffd700 1 -- 172.21.15.45:0/1182112588 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152406d7f0 con 0x7f1524140ab0 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f15227fc700 1 -- 172.21.15.45:0/1182112588 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388378905 0 0) 0x7f15240c9940 con 0x7f15241406e0 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.168+0000 7f1522ffd700 1 --2- 172.21.15.45:0/1182112588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1524140ab0 0x7f15240acfa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f15180098e0 tx=0x7f15180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632a82636b054fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f15227fc700 1 -- 172.21.15.45:0/1182112588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518009e80 con 0x7f1524140ab0 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f15227fc700 1 -- 172.21.15.45:0/1182112588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800b860 con 0x7f1524140ab0 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f15227fc700 1 -- 172.21.15.45:0/1182112588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800bbe0 con 0x7f1524140ab0 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 -- 172.21.15.45:0/1182112588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1524140ab0 msgr2=0x7f15240acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 --2- 172.21.15.45:0/1182112588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1524140ab0 0x7f15240acfa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f15180098e0 tx=0x7f15180091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 -- 172.21.15.45:0/1182112588 shutdown_connections 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 --2- 172.21.15.45:0/1182112588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 --2- 172.21.15.45:0/1182112588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1524140ab0 0x7f15240acfa0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 -- 172.21.15.45:0/1182112588 >> 172.21.15.45:0/1182112588 conn(0x7f15240691c0 msgr2=0x7f15240cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 -- 172.21.15.45:0/1182112588 shutdown_connections 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.169+0000 7f152a133700 1 -- 172.21.15.45:0/1182112588 wait complete. 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 Processor -- start 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 -- start start 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241406e0 0x7f152413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1524140ab0 0x7f152415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240c5430 con 0x7f15241406e0 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240b18f0 con 0x7f1524140ab0 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.171+0000 7f152a133700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15240c56f0 con 0x7f15240ad4e0 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241406e0 0x7f152413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241406e0 0x7f152413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36150/0 (socket says 172.21.15.45:36150) 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 -- 172.21.15.45:0/222641190 learned_addr learned my addr 172.21.15.45:0/222641190 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f15237fe700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1523fff700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1524140ab0 0x7f152415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 -- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 msgr2=0x7f15240b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 -- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1524140ab0 msgr2=0x7f152415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1524140ab0 0x7f152415bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 -- 172.21.15.45:0/222641190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009420 con 0x7f15241406e0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f1522ffd700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241406e0 0x7f152413d8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f151800b3c0 tx=0x7f1518017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f150b7fe700 1 -- 172.21.15.45:0/222641190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800b9d0 con 0x7f15241406e0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f150b7fe700 1 -- 172.21.15.45:0/222641190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800bb50 con 0x7f15241406e0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15240b1050 con 0x7f15241406e0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f150b7fe700 1 -- 172.21.15.45:0/222641190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518011670 con 0x7f15241406e0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.172+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15240b1610 con 0x7f15241406e0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.173+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15240c5880 con 0x7f15241406e0 2023-10-06T05:18:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.174+0000 7f150b7fe700 1 -- 172.21.15.45:0/222641190 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1518011a20 con 0x7f15241406e0 2023-10-06T05:18:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.174+0000 7f150b7fe700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1510066f30 0x7f15100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.175+0000 7f150b7fe700 1 -- 172.21.15.45:0/222641190 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f1518085970 con 0x7f15241406e0 2023-10-06T05:18:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.175+0000 7f15237fe700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1510066f30 0x7f15100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.175+0000 7f15237fe700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1510066f30 0x7f15100693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1514007fd0 tx=0x7f1514007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.179+0000 7f150b7fe700 1 -- 172.21.15.45:0/222641190 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1518052c60 con 0x7f15241406e0 2023-10-06T05:18:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.379+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"} v 0) v1 -- 0x7f152404bb40 con 0x7f15241406e0 2023-10-06T05:18:00.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.383+0000 7f150b7fe700 1 -- 172.21.15.45:0/222641190 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"}]=0 v17) v1 ==== 116+0+0 (secure 0 0 0) 0x7f1518056280 con 0x7f15241406e0 2023-10-06T05:18:00.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1510066f30 msgr2=0x7f15100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1510066f30 0x7f15100693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1514007fd0 tx=0x7f1514007d30 comp rx=0 tx=0).stop 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241406e0 msgr2=0x7f152413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241406e0 0x7f152413d8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f151800b3c0 tx=0x7f1518017000 comp rx=0 tx=0).stop 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 shutdown_connections 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1510066f30 0x7f15100693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1524140ab0 0x7f152415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15240ad4e0 0x7f15240b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 --2- 172.21.15.45:0/222641190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241406e0 0x7f152413d8f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 >> 172.21.15.45:0/222641190 conn(0x7f15240691c0 msgr2=0x7f152413f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:00.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 shutdown_connections 2023-10-06T05:18:00.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.386+0000 7f152a133700 1 -- 172.21.15.45:0/222641190 wait complete. 2023-10-06T05:18:00.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2023-10-06T05:18:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.575+0000 7fda69cd9700 1 Processor -- start 2023-10-06T05:18:00.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.575+0000 7fda69cd9700 1 -- start start 2023-10-06T05:18:00.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.576+0000 7fda69cd9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda641570b0 0x7fda6406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.576+0000 7fda69cd9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda6415ac10 0x7fda640ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.576+0000 7fda69cd9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6415b380 con 0x7fda64156ce0 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.576+0000 7fda69cd9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda64157520 con 0x7fda6415ac10 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.576+0000 7fda69cd9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6406c010 con 0x7fda641570b0 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda63fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda6415ac10 0x7fda640ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda62ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda641570b0 0x7fda6406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda637fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fda64156ce0 0x7fda640afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:44446/0 (socket says 172.21.15.45:44446) 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda637fe700 1 -- 172.21.15.45:0/732562662 learned_addr learned my addr 172.21.15.45:0/732562662 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda63fff700 1 -- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda641570b0 msgr2=0x7fda6406bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:00.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda63fff700 1 --2- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda641570b0 0x7fda6406bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda63fff700 1 -- 172.21.15.45:0/732562662 >> v1:172.21.15.144:6789/0 conn(0x7fda64156ce0 legacy=0x7fda640afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda63fff700 1 -- 172.21.15.45:0/732562662 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6415b660 con 0x7fda6415ac10 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda627fc700 1 -- 172.21.15.45:0/732562662 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045550344 0 0) 0x7fda6415b380 con 0x7fda64156ce0 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda62ffd700 1 --2- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda641570b0 0x7fda6406bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.577+0000 7fda63fff700 1 --2- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6415ac10 0x7fda640ccce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009e90 tx=0x7fda5c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6c35d040d25423 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda627fc700 1 -- 172.21.15.45:0/732562662 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00e070 con 0x7fda6415ac10 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda627fc700 1 -- 172.21.15.45:0/732562662 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c00bbf0 con 0x7fda6415ac10 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda627fc700 1 -- 172.21.15.45:0/732562662 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c012430 con 0x7fda6415ac10 2023-10-06T05:18:00.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 -- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6415ac10 msgr2=0x7fda640ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 --2- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6415ac10 0x7fda640ccce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009e90 tx=0x7fda5c009780 comp rx=0 tx=0).stop 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 -- 172.21.15.45:0/732562662 shutdown_connections 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 --2- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6415ac10 0x7fda640ccce0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 --2- 172.21.15.45:0/732562662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda641570b0 0x7fda6406bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 -- 172.21.15.45:0/732562662 >> 172.21.15.45:0/732562662 conn(0x7fda6406f090 msgr2=0x7fda6406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 -- 172.21.15.45:0/732562662 shutdown_connections 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.578+0000 7fda69cd9700 1 -- 172.21.15.45:0/732562662 wait complete. 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.580+0000 7fda69cd9700 1 Processor -- start 2023-10-06T05:18:00.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.580+0000 7fda69cd9700 1 -- start start 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda69cd9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda64156ce0 0x7fda640c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda69cd9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda641570b0 0x7fda6414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda69cd9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda6415ac10 0x7fda640c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda69cd9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640c2600 con 0x7fda641570b0 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda69cd9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640c51f0 con 0x7fda64156ce0 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda69cd9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda640c1cd0 con 0x7fda6415ac10 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda62ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda641570b0 0x7fda6414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda62ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda641570b0 0x7fda6414a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36162/0 (socket says 172.21.15.45:36162) 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda62ffd700 1 -- 172.21.15.45:0/1868610443 learned_addr learned my addr 172.21.15.45:0/1868610443 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:00.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda637fe700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda64156ce0 0x7fda640c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.581+0000 7fda63fff700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda6415ac10 0x7fda640c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda62ffd700 1 -- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda6415ac10 msgr2=0x7fda640c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda62ffd700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda6415ac10 0x7fda640c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda62ffd700 1 -- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda64156ce0 msgr2=0x7fda640c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda62ffd700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda64156ce0 0x7fda640c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda62ffd700 1 -- 172.21.15.45:0/1868610443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c009420 con 0x7fda641570b0 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda62ffd700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda641570b0 0x7fda6414a0e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00ec80 tx=0x7fda4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda5b7fe700 1 -- 172.21.15.45:0/1868610443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00c920 con 0x7fda641570b0 2023-10-06T05:18:00.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda5b7fe700 1 -- 172.21.15.45:0/1868610443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00ca80 con 0x7fda641570b0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c009a20 con 0x7fda641570b0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda5b7fe700 1 -- 172.21.15.45:0/1868610443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00ce00 con 0x7fda641570b0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.582+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda640c2290 con 0x7fda641570b0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.583+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda640c2770 con 0x7fda641570b0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.584+0000 7fda5b7fe700 1 -- 172.21.15.45:0/1868610443 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda4c013680 con 0x7fda641570b0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.584+0000 7fda5b7fe700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda44066f30 0x7fda440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.584+0000 7fda5b7fe700 1 -- 172.21.15.45:0/1868610443 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fda4c084910 con 0x7fda641570b0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.584+0000 7fda637fe700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda44066f30 0x7fda440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.585+0000 7fda637fe700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda44066f30 0x7fda440693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fda54007160 tx=0x7fda54007f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.589+0000 7fda5b7fe700 1 -- 172.21.15.45:0/1868610443 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda4c051b80 con 0x7fda641570b0 2023-10-06T05:18:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:00.881+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) v1 -- 0x7fda6404bb40 con 0x7fda641570b0 2023-10-06T05:18:01.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.058+0000 7fda5b7fe700 1 -- 172.21.15.45:0/1868610443 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v165) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda4c0551a0 con 0x7fda641570b0 2023-10-06T05:18:01.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda44066f30 msgr2=0x7fda440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda44066f30 0x7fda440693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fda54007160 tx=0x7fda54007f50 comp rx=0 tx=0).stop 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda641570b0 msgr2=0x7fda6414a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda641570b0 0x7fda6414a0e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00ec80 tx=0x7fda4c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 shutdown_connections 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda44066f30 0x7fda440693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda6415ac10 0x7fda640c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda64156ce0 0x7fda640c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 --2- 172.21.15.45:0/1868610443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda641570b0 0x7fda6414a0e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 >> 172.21.15.45:0/1868610443 conn(0x7fda6406f090 msgr2=0x7fda64156120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:01.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 shutdown_connections 2023-10-06T05:18:01.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.061+0000 7fda69cd9700 1 -- 172.21.15.45:0/1868610443 wait complete. 2023-10-06T05:18:01.079 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2023-10-06T05:18:01.079 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2023-10-06T05:18:01.079 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2023-10-06T05:18:01.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2023-10-06T05:18:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.214+0000 7f5123135700 1 Processor -- start 2023-10-06T05:18:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.214+0000 7f5123135700 1 -- start start 2023-10-06T05:18:01.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.215+0000 7f5123135700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f511c0cac50 0x7f511c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.215+0000 7f5123135700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0b0790 con 0x7f511c15b890 2023-10-06T05:18:01.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.215+0000 7f5123135700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0c8e80 con 0x7f511c06c3d0 2023-10-06T05:18:01.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.215+0000 7f5123135700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511c0b0570 con 0x7f511c0cac50 2023-10-06T05:18:01.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.215+0000 7f511bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f511c15b890 0x7f511c156b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35444/0 (socket says 172.21.15.45:35444) 2023-10-06T05:18:01.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.215+0000 7f511bfff700 1 -- 172.21.15.45:0/2729641736 learned_addr learned my addr 172.21.15.45:0/2729641736 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:01.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f51216d2700 1 --2- 172.21.15.45:0/2729641736 >> v2:172.21.15.144:3300/0 conn(0x7f511c0cac50 0x7f511c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049168169 0 0) 0x7f511c0b0790 con 0x7f511c15b890 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5108003570 con 0x7f511c15b890 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819901717 0 0) 0x7f511c0c8e80 con 0x7f511c06c3d0 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f511c0b0790 con 0x7f511c06c3d0 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2966364053 0 0) 0x7f5108003570 con 0x7f511c15b890 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f51216d2700 1 -- 172.21.15.45:0/2729641736 >> v1:172.21.15.144:6789/0 conn(0x7f511c06c3d0 legacy=0x7f511c0afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f51216d2700 1 -- 172.21.15.45:0/2729641736 >> v1:172.21.15.45:6789/0 conn(0x7f511c15b890 legacy=0x7f511c156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f51216d2700 1 -- 172.21.15.45:0/2729641736 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0c9160 con 0x7f511c0cac50 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f51216d2700 1 --2- 172.21.15.45:0/2729641736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c0cac50 0x7f511c06c7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5114009e60 tx=0x7f5114009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42bf53c133663f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400e070 con 0x7f511c0cac50 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511400b7a0 con 0x7f511c0cac50 2023-10-06T05:18:01.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.216+0000 7f511b7fe700 1 -- 172.21.15.45:0/2729641736 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104001470 con 0x7f511c0cac50 2023-10-06T05:18:01.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.217+0000 7f5123135700 1 -- 172.21.15.45:0/2729641736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c0cac50 msgr2=0x7f511c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.217+0000 7f5123135700 1 --2- 172.21.15.45:0/2729641736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c0cac50 0x7f511c06c7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5114009e60 tx=0x7f5114009780 comp rx=0 tx=0).stop 2023-10-06T05:18:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.217+0000 7f5123135700 1 -- 172.21.15.45:0/2729641736 shutdown_connections 2023-10-06T05:18:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.217+0000 7f5123135700 1 --2- 172.21.15.45:0/2729641736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c0cac50 0x7f511c06c7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.217+0000 7f5123135700 1 -- 172.21.15.45:0/2729641736 >> 172.21.15.45:0/2729641736 conn(0x7f511c067620 msgr2=0x7f511c067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.217+0000 7f5123135700 1 -- 172.21.15.45:0/2729641736 shutdown_connections 2023-10-06T05:18:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.217+0000 7f5123135700 1 -- 172.21.15.45:0/2729641736 wait complete. 2023-10-06T05:18:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.218+0000 7f5123135700 1 Processor -- start 2023-10-06T05:18:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.218+0000 7f5123135700 1 -- start start 2023-10-06T05:18:01.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f5123135700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f511c06c3d0 0x7f511c16a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f5123135700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511c0cac50 0x7f511c0c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f5123135700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c15b890 0x7f511c0c3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f5123135700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0cee90 con 0x7f511c0cac50 2023-10-06T05:18:01.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f5123135700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0ce280 con 0x7f511c15b890 2023-10-06T05:18:01.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f5123135700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0ce560 con 0x7f511c06c3d0 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f511bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511c0cac50 0x7f511c0c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f511bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511c0cac50 0x7f511c0c3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36170/0 (socket says 172.21.15.45:36170) 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.219+0000 7f511bfff700 1 -- 172.21.15.45:0/253515973 learned_addr learned my addr 172.21.15.45:0/253515973 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f5120ed1700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f511c06c3d0 0x7f511c16a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f51216d2700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c15b890 0x7f511c0c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f511bfff700 1 -- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f511c06c3d0 msgr2=0x7f511c16a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f511bfff700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f511c06c3d0 0x7f511c16a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f511bfff700 1 -- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c15b890 msgr2=0x7f511c0c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f511bfff700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c15b890 0x7f511c0c3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f511bfff700 1 -- 172.21.15.45:0/253515973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114009a20 con 0x7f511c0cac50 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f51216d2700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c15b890 0x7f511c0c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f511bfff700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511c0cac50 0x7f511c0c3330 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f510400cb70 tx=0x7f5104001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f5118ff9700 1 -- 172.21.15.45:0/253515973 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104007e90 con 0x7f511c0cac50 2023-10-06T05:18:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c0ce840 con 0x7f511c0cac50 2023-10-06T05:18:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f5118ff9700 1 -- 172.21.15.45:0/253515973 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5104007360 con 0x7f511c0cac50 2023-10-06T05:18:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f5118ff9700 1 -- 172.21.15.45:0/253515973 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104004830 con 0x7f511c0cac50 2023-10-06T05:18:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.220+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c16a890 con 0x7f511c0cac50 2023-10-06T05:18:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.221+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c06f160 con 0x7f511c0cac50 2023-10-06T05:18:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.222+0000 7f5118ff9700 1 -- 172.21.15.45:0/253515973 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5104004ba0 con 0x7f511c0cac50 2023-10-06T05:18:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.222+0000 7f5118ff9700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50fc066f30 0x7f50fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.222+0000 7f5118ff9700 1 -- 172.21.15.45:0/253515973 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f5104083a00 con 0x7f511c0cac50 2023-10-06T05:18:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.222+0000 7f5120ed1700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50fc066f30 0x7f50fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.223+0000 7f5120ed1700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50fc066f30 0x7f50fc0693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f510c007fd0 tx=0x7f510c007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.227+0000 7f5118ff9700 1 -- 172.21.15.45:0/253515973 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5104050c70 con 0x7f511c0cac50 2023-10-06T05:18:01.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:01.510+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) v1 -- 0x7f511c0c9440 con 0x7f511c0cac50 2023-10-06T05:18:02.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.060+0000 7f5118ff9700 1 -- 172.21.15.45:0/253515973 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v166) v1 ==== 145+0+0 (secure 0 0 0) 0x7f5104018020 con 0x7f511c0cac50 2023-10-06T05:18:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50fc066f30 msgr2=0x7f50fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50fc066f30 0x7f50fc0693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f510c007fd0 tx=0x7f510c007d50 comp rx=0 tx=0).stop 2023-10-06T05:18:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511c0cac50 msgr2=0x7f511c0c3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511c0cac50 0x7f511c0c3330 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f510400cb70 tx=0x7f5104001b50 comp rx=0 tx=0).stop 2023-10-06T05:18:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 shutdown_connections 2023-10-06T05:18:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50fc066f30 0x7f50fc0693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c15b890 0x7f511c0c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f511c06c3d0 0x7f511c16a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 --2- 172.21.15.45:0/253515973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511c0cac50 0x7f511c0c3330 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.062+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 >> 172.21.15.45:0/253515973 conn(0x7f511c067620 msgr2=0x7f511c15a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.063+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 shutdown_connections 2023-10-06T05:18:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.063+0000 7f5123135700 1 -- 172.21.15.45:0/253515973 wait complete. 2023-10-06T05:18:02.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:18:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.277+0000 7f3391a73700 1 Processor -- start 2023-10-06T05:18:02.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.277+0000 7f3391a73700 1 -- start start 2023-10-06T05:18:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.278+0000 7f3391a73700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f338c159520 0x7f338c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.278+0000 7f3391a73700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f338c15d3d0 0x7f338c067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.278+0000 7f3391a73700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338c0c9c90 con 0x7f338c159150 2023-10-06T05:18:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.278+0000 7f3391a73700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c15d8e0 con 0x7f338c15d3d0 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.278+0000 7f3391a73700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c15dbc0 con 0x7f338c159520 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.278+0000 7f338b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f338c159150 0x7f338c0ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35456/0 (socket says 172.21.15.45:35456) 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.278+0000 7f338b7fe700 1 -- 172.21.15.45:0/1707964767 learned_addr learned my addr 172.21.15.45:0/1707964767 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338bfff700 1 --2- 172.21.15.45:0/1707964767 >> v2:172.21.15.144:3300/0 conn(0x7f338c15d3d0 0x7f338c067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338affd700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f338c159520 0x7f338c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201292692 0 0) 0x7f338c0c9c90 con 0x7f338c159150 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3378003570 con 0x7f338c159150 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4147901382 0 0) 0x7f3378003570 con 0x7f338c159150 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f338c159520 msgr2=0x7f338c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f338c159520 0x7f338c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 >> v2:172.21.15.144:3300/0 conn(0x7f338c15d3d0 msgr2=0x7f338c067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 --2- 172.21.15.45:0/1707964767 >> v2:172.21.15.144:3300/0 conn(0x7f338c15d3d0 0x7f338c067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c0c9920 con 0x7f338c159150 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f337c001470 con 0x7f338c159150 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 >> v1:172.21.15.45:6789/0 conn(0x7f338c159150 legacy=0x7f338c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3378003a60 0x7f3378003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3378004810 0x7f3378004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33780051c0 0x7f337800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c15d8e0 con 0x7f3378004810 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c15dbc0 con 0x7f3378003a60 2023-10-06T05:18:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337800bfa0 con 0x7f33780051c0 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338affd700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3378004810 0x7f3378004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338b7fe700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3378003a60 0x7f3378003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338bfff700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33780051c0 0x7f337800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338affd700 1 -- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33780051c0 msgr2=0x7f337800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338affd700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33780051c0 0x7f337800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338affd700 1 -- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3378003a60 msgr2=0x7f3378003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338affd700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3378003a60 0x7f3378003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.279+0000 7f338affd700 1 -- 172.21.15.45:0/1707964767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378003570 con 0x7f3378004810 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f338affd700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3378004810 0x7f3378004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3374009c00 tx=0x7f33740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817cfebde47eeba4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337400e6b0 con 0x7f3378004810 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337400e810 con 0x7f3378004810 2023-10-06T05:18:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f338a7fc700 1 -- 172.21.15.45:0/1707964767 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337400ebd0 con 0x7f3378004810 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f3391a73700 1 -- 172.21.15.45:0/1707964767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3378004810 msgr2=0x7f3378004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f3391a73700 1 --2- 172.21.15.45:0/1707964767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3378004810 0x7f3378004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3374009c00 tx=0x7f33740091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f338bfff700 1 -- 172.21.15.45:0/1707964767 reap_dead start 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f3391a73700 1 -- 172.21.15.45:0/1707964767 shutdown_connections 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f3391a73700 1 -- 172.21.15.45:0/1707964767 >> 172.21.15.45:0/1707964767 conn(0x7f338c0691c0 msgr2=0x7f338c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.280+0000 7f3391a73700 1 -- 172.21.15.45:0/1707964767 shutdown_connections 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.281+0000 7f3391a73700 1 -- 172.21.15.45:0/1707964767 wait complete. 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.282+0000 7f3391a73700 1 Processor -- start 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.282+0000 7f3391a73700 1 -- start start 2023-10-06T05:18:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f3391a73700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3378004810 0x7f338c16a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f3391a73700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c0b0b40 0x7f338c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f3391a73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f338c154260 0x7f338c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f3391a73700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0b20e0 con 0x7f338c0b0b40 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f3391a73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0afd80 con 0x7f338c154260 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f3391a73700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0b0060 con 0x7f3378004810 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c0b0b40 0x7f338c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c0b0b40 0x7f338c0b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36184/0 (socket says 172.21.15.45:36184) 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 -- 172.21.15.45:0/3865435967 learned_addr learned my addr 172.21.15.45:0/3865435967 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338b7fe700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3378004810 0x7f338c16a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338bfff700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f338c154260 0x7f338c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 -- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3378004810 msgr2=0x7f338c16a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3378004810 0x7f338c16a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 -- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f338c154260 msgr2=0x7f338c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f338c154260 0x7f338c156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 -- 172.21.15.45:0/3865435967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c0b0340 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338bfff700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f338c154260 0x7f338c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f338affd700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c0b0b40 0x7f338c0b18b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f33780056c0 tx=0x7f337401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.283+0000 7f33837fe700 1 -- 172.21.15.45:0/3865435967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374023070 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.284+0000 7f33837fe700 1 -- 172.21.15.45:0/3865435967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337400e980 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.284+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c0b0620 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.284+0000 7f33837fe700 1 -- 172.21.15.45:0/3865435967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337401e030 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.284+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c1546e0 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.285+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c06e2d0 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.285+0000 7f33837fe700 1 -- 172.21.15.45:0/3865435967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3374013470 con 0x7f338c0b0b40 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.286+0000 7f33837fe700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f336c066f30 0x7f336c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.286+0000 7f33837fe700 1 -- 172.21.15.45:0/3865435967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f3374085c40 con 0x7f338c0b0b40 2023-10-06T05:18:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.286+0000 7f338b7fe700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f336c066f30 0x7f336c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.286+0000 7f338b7fe700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f336c066f30 0x7f336c0693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f337c001620 tx=0x7f337c001e90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.290+0000 7f33837fe700 1 -- 172.21.15.45:0/3865435967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3374052eb0 con 0x7f338c0b0b40 2023-10-06T05:18:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.492+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f338c0afec0 con 0x7f338c0b0b40 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.496+0000 7f33837fe700 1 -- 172.21.15.45:0/3865435967 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f338c0afec0 con 0x7f338c0b0b40 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f336c066f30 msgr2=0x7f336c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f336c066f30 0x7f336c0693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f337c001620 tx=0x7f337c001e90 comp rx=0 tx=0).stop 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c0b0b40 msgr2=0x7f338c0b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c0b0b40 0x7f338c0b18b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f33780056c0 tx=0x7f337401a000 comp rx=0 tx=0).stop 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 shutdown_connections 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f336c066f30 0x7f336c0693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f338c154260 0x7f338c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3378004810 0x7f338c16a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 --2- 172.21.15.45:0/3865435967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c0b0b40 0x7f338c0b18b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 >> 172.21.15.45:0/3865435967 conn(0x7f338c0691c0 msgr2=0x7f338c0c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:02.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 shutdown_connections 2023-10-06T05:18:02.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.498+0000 7f3391a73700 1 -- 172.21.15.45:0/3865435967 wait complete. 2023-10-06T05:18:02.513 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:18:02.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:18:02.672 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:18:02.810 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:18:02.810 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:18:02.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.950+0000 7f8d47f4d700 1 Processor -- start 2023-10-06T05:18:02.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.950+0000 7f8d47f4d700 1 -- start start 2023-10-06T05:18:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d47f4d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d400c8ff0 0x7f8d400c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d47f4d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4015b000 con 0x7f8d400caad0 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d47f4d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4015b2e0 con 0x7f8d40157200 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d47f4d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4015b5c0 con 0x7f8d400c8ff0 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d45ce9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d400c8ff0 0x7f8d400c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d454e8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8d400caad0 0x7f8d40156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35462/0 (socket says 172.21.15.45:35462) 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d45ce9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d400c8ff0 0x7f8d400c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36196/0 (socket says 172.21.15.45:36196) 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.951+0000 7f8d454e8700 1 -- 172.21.15.45:0/53528384 learned_addr learned my addr 172.21.15.45:0/53528384 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d45ce9700 1 -- 172.21.15.45:0/53528384 >> v1:172.21.15.144:6789/0 conn(0x7f8d40157200 legacy=0x7f8d400cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d45ce9700 1 -- 172.21.15.45:0/53528384 >> v1:172.21.15.45:6789/0 conn(0x7f8d400caad0 legacy=0x7f8d40156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d45ce9700 1 -- 172.21.15.45:0/53528384 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4015b8a0 con 0x7f8d400c8ff0 2023-10-06T05:18:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d44ce7700 1 -- 172.21.15.45:0/53528384 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099696925 0 0) 0x7f8d4015b000 con 0x7f8d400caad0 2023-10-06T05:18:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d45ce9700 1 --2- 172.21.15.45:0/53528384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400c8ff0 0x7f8d400c93c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0098e0 tx=0x7f8d3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea726dd4537b758 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d44ce7700 1 -- 172.21.15.45:0/53528384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c009e80 con 0x7f8d400c8ff0 2023-10-06T05:18:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d44ce7700 1 -- 172.21.15.45:0/53528384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c00b820 con 0x7f8d400c8ff0 2023-10-06T05:18:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d44ce7700 1 -- 172.21.15.45:0/53528384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00bba0 con 0x7f8d400c8ff0 2023-10-06T05:18:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d47f4d700 1 -- 172.21.15.45:0/53528384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400c8ff0 msgr2=0x7f8d400c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.952+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/53528384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400c8ff0 0x7f8d400c93c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0098e0 tx=0x7f8d3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.953+0000 7f8d47f4d700 1 -- 172.21.15.45:0/53528384 shutdown_connections 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.953+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/53528384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400c8ff0 0x7f8d400c93c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.953+0000 7f8d47f4d700 1 -- 172.21.15.45:0/53528384 >> 172.21.15.45:0/53528384 conn(0x7f8d400cdf50 msgr2=0x7f8d400ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.953+0000 7f8d47f4d700 1 -- 172.21.15.45:0/53528384 shutdown_connections 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.953+0000 7f8d47f4d700 1 -- 172.21.15.45:0/53528384 wait complete. 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.954+0000 7f8d47f4d700 1 Processor -- start 2023-10-06T05:18:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.954+0000 7f8d47f4d700 1 -- start start 2023-10-06T05:18:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.955+0000 7f8d47f4d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d400c8ff0 0x7f8d40159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.955+0000 7f8d47f4d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400caad0 0x7f8d40159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.955+0000 7f8d47f4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d40157200 0x7f8d40159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.955+0000 7f8d47f4d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400c79c0 con 0x7f8d400caad0 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.955+0000 7f8d47f4d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401676f0 con 0x7f8d40157200 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.955+0000 7f8d47f4d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40144eb0 con 0x7f8d400c8ff0 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.955+0000 7f8d454e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400caad0 0x7f8d40159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400caad0 0x7f8d40159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36206/0 (socket says 172.21.15.45:36206) 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 -- 172.21.15.45:0/82574101 learned_addr learned my addr 172.21.15.45:0/82574101 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d45ce9700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d400c8ff0 0x7f8d40159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d464ea700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d40157200 0x7f8d40159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 -- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d400c8ff0 msgr2=0x7f8d40159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d400c8ff0 0x7f8d40159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 -- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d40157200 msgr2=0x7f8d40159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d40157200 0x7f8d40159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 -- 172.21.15.45:0/82574101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c009420 con 0x7f8d400caad0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d464ea700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d40157200 0x7f8d40159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d454e8700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400caad0 0x7f8d40159850 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000c7e0 tx=0x7f8d30007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d35ffb700 1 -- 172.21.15.45:0/82574101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30005b30 con 0x7f8d400caad0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d35ffb700 1 -- 172.21.15.45:0/82574101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d30005c90 con 0x7f8d400caad0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d40158550 con 0x7f8d400caad0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.956+0000 7f8d35ffb700 1 -- 172.21.15.45:0/82574101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30007300 con 0x7f8d400caad0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.957+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d40158b10 con 0x7f8d400caad0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.957+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4006f410 con 0x7f8d400caad0 2023-10-06T05:18:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.958+0000 7f8d35ffb700 1 -- 172.21.15.45:0/82574101 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d300076b0 con 0x7f8d400caad0 2023-10-06T05:18:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.958+0000 7f8d35ffb700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8d24066f30 0x7f8d240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.958+0000 7f8d35ffb700 1 -- 172.21.15.45:0/82574101 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f8d30083aa0 con 0x7f8d400caad0 2023-10-06T05:18:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.959+0000 7f8d45ce9700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8d24066f30 0x7f8d240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.959+0000 7f8d45ce9700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8d24066f30 0x7f8d240693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0030c0 tx=0x7f8d3c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:02.963+0000 7f8d35ffb700 1 -- 172.21.15.45:0/82574101 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d30050d10 con 0x7f8d400caad0 2023-10-06T05:18:03.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.213+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d4004bb40 con 0x7f8d400caad0 2023-10-06T05:18:03.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.213+0000 7f8d35ffb700 1 -- 172.21.15.45:0/82574101 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f8d30017090 con 0x7f8d400caad0 2023-10-06T05:18:03.215 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:03.215 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1795279212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1795279212},{"type":"v1","addr":"172.21.15.144:6836","nonce":1795279212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/53989979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":53989979},{"type":"v1","addr":"172.21.15.144:6837","nonce":53989979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2584960060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2584960060},{"type":"v1","addr":"172.21.15.45:6834","nonce":2584960060}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3384844756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3384844756},{"type":"v1","addr":"172.21.15.45:6835","nonce":3384844756}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[]} 2023-10-06T05:18:03.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8d24066f30 msgr2=0x7f8d240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8d24066f30 0x7f8d240693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0030c0 tx=0x7f8d3c016000 comp rx=0 tx=0).stop 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400caad0 msgr2=0x7f8d40159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400caad0 0x7f8d40159850 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000c7e0 tx=0x7f8d30007df0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 shutdown_connections 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8d24066f30 0x7f8d240693e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d40157200 0x7f8d40159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d400c8ff0 0x7f8d40159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 --2- 172.21.15.45:0/82574101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d400caad0 0x7f8d40159850 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.215+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 >> 172.21.15.45:0/82574101 conn(0x7f8d400cdf50 msgr2=0x7f8d40150470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.216+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 shutdown_connections 2023-10-06T05:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.216+0000 7f8d47f4d700 1 -- 172.21.15.45:0/82574101 wait complete. 2023-10-06T05:18:03.241 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 42 2023-10-06T05:18:03.242 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:18:03.242 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:18:03.242 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:18:03.242 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:03.243 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:18:03.243 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:18:03.243 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:18:03.243 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:03.243 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:18:03.243 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:18:03.244 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:18:03.244 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:03.244 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:18:03.244 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:18:03.244 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:18:03.244 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:03.252 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:18:03.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:18:03.253 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:18:03.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:18:03.255 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:18:03.255 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:18:03.258 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:18:03.259 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:18:03.295 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:18:03.346 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:03.347 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:18:03.347 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:03.348 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:18:03.348 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:03.349 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:18:03.349 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:03.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:18:03.382 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:18:03.412 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:18:03.435 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:18:03.435 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:18:03.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.485+0000 7fe5b3bc1700 1 Processor -- start 2023-10-06T05:18:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.485+0000 7fe5b3bc1700 1 -- start start 2023-10-06T05:18:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b3bc1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5ac1595d0 0x7fe5ac06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b3bc1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5ac06b5c0 0x7fe5ac0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b3bc1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ac0c7d00 con 0x7fe5ac159200 2023-10-06T05:18:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b3bc1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c7780 con 0x7fe5ac06b5c0 2023-10-06T05:18:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b3bc1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac06bb40 con 0x7fe5ac1595d0 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b195d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe5ac159200 0x7fe5ac0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35478/0 (socket says 172.21.15.45:35478) 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b215e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5ac06b5c0 0x7fe5ac0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b195d700 1 -- 172.21.15.45:0/1967140494 learned_addr learned my addr 172.21.15.45:0/1967140494 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b115c700 1 --2- 172.21.15.45:0/1967140494 >> v2:172.21.15.144:3300/0 conn(0x7fe5ac1595d0 0x7fe5ac06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b095b700 1 -- 172.21.15.45:0/1967140494 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811475826 0 0) 0x7fe5ac0c7d00 con 0x7fe5ac159200 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b095b700 1 -- 172.21.15.45:0/1967140494 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe598003570 con 0x7fe5ac159200 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b215e700 1 -- 172.21.15.45:0/1967140494 >> v2:172.21.15.144:3300/0 conn(0x7fe5ac1595d0 msgr2=0x7fe5ac06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b215e700 1 --2- 172.21.15.45:0/1967140494 >> v2:172.21.15.144:3300/0 conn(0x7fe5ac1595d0 0x7fe5ac06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b215e700 1 -- 172.21.15.45:0/1967140494 >> v1:172.21.15.45:6789/0 conn(0x7fe5ac159200 legacy=0x7fe5ac0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.486+0000 7fe5b215e700 1 -- 172.21.15.45:0/1967140494 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac06be20 con 0x7fe5ac06b5c0 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b215e700 1 --2- 172.21.15.45:0/1967140494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac06b5c0 0x7fe5ac0ccdc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06e790 tx=0x7fe5a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ae09975b3aaba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b095b700 1 -- 172.21.15.45:0/1967140494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400e070 con 0x7fe5ac06b5c0 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b095b700 1 -- 172.21.15.45:0/1967140494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a400bab0 con 0x7fe5ac06b5c0 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b095b700 1 -- 172.21.15.45:0/1967140494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400be70 con 0x7fe5ac06b5c0 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/1967140494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac06b5c0 msgr2=0x7fe5ac0ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/1967140494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac06b5c0 0x7fe5ac0ccdc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06e790 tx=0x7fe5a4009780 comp rx=0 tx=0).stop 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/1967140494 shutdown_connections 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/1967140494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac06b5c0 0x7fe5ac0ccdc0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/1967140494 >> v2:172.21.15.144:3300/0 conn(0x7fe5ac1595d0 0x7fe5ac06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.487+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/1967140494 >> 172.21.15.45:0/1967140494 conn(0x7fe5ac06f090 msgr2=0x7fe5ac06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.488+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/1967140494 shutdown_connections 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.488+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/1967140494 wait complete. 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.489+0000 7fe5b3bc1700 1 Processor -- start 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.489+0000 7fe5b3bc1700 1 -- start start 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.489+0000 7fe5b3bc1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5ac06b5c0 0x7fe5ac158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.489+0000 7fe5b3bc1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5ac159200 0x7fe5ac0b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.489+0000 7fe5b3bc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac1595d0 0x7fe5ac0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.489+0000 7fe5b3bc1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c59d0 con 0x7fe5ac1595d0 2023-10-06T05:18:03.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b3bc1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c5c20 con 0x7fe5ac159200 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b3bc1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c5f00 con 0x7fe5ac06b5c0 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac1595d0 0x7fe5ac0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac1595d0 0x7fe5ac0b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36268/0 (socket says 172.21.15.45:36268) 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 -- 172.21.15.45:0/3183708027 learned_addr learned my addr 172.21.15.45:0/3183708027 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b115c700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5ac159200 0x7fe5ac0b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b195d700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5ac06b5c0 0x7fe5ac158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 -- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5ac06b5c0 msgr2=0x7fe5ac158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5ac06b5c0 0x7fe5ac158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 -- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5ac159200 msgr2=0x7fe5ac0b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5ac159200 0x7fe5ac0b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 -- 172.21.15.45:0/3183708027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a4009a20 con 0x7fe5ac1595d0 2023-10-06T05:18:03.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b195d700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5ac06b5c0 0x7fe5ac158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b215e700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac1595d0 0x7fe5ac0b27a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06aed0 tx=0x7fe5a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe59dffb700 1 -- 172.21.15.45:0/3183708027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4027470 con 0x7fe5ac1595d0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe59dffb700 1 -- 172.21.15.45:0/3183708027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a40275d0 con 0x7fe5ac1595d0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac0c61e0 con 0x7fe5ac1595d0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.490+0000 7fe59dffb700 1 -- 172.21.15.45:0/3183708027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4027990 con 0x7fe5ac1595d0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.491+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac0c67a0 con 0x7fe5ac1595d0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.491+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac0c6990 con 0x7fe5ac1595d0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.492+0000 7fe59dffb700 1 -- 172.21.15.45:0/3183708027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5a4024070 con 0x7fe5ac1595d0 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.493+0000 7fe59dffb700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590066f30 0x7fe5900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.493+0000 7fe59dffb700 1 -- 172.21.15.45:0/3183708027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fe5a4086ff0 con 0x7fe5ac1595d0 2023-10-06T05:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.493+0000 7fe5b195d700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590066f30 0x7fe5900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.493+0000 7fe5b195d700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590066f30 0x7fe5900693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8007fd0 tx=0x7fe5a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.497+0000 7fe59dffb700 1 -- 172.21.15.45:0/3183708027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5a40b85a0 con 0x7fe5ac1595d0 2023-10-06T05:18:03.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:03.719+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe5ac0c7990 con 0x7fe5ac1595d0 2023-10-06T05:18:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.073+0000 7fe59dffb700 1 -- 172.21.15.45:0/3183708027 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe5a40541e0 con 0x7fe5ac1595d0 2023-10-06T05:18:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.076+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590066f30 msgr2=0x7fe5900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.076+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590066f30 0x7fe5900693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8007fd0 tx=0x7fe5a8009040 comp rx=0 tx=0).stop 2023-10-06T05:18:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.076+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac1595d0 msgr2=0x7fe5ac0b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.076+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac1595d0 0x7fe5ac0b27a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06aed0 tx=0x7fe5a4019000 comp rx=0 tx=0).stop 2023-10-06T05:18:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.076+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 shutdown_connections 2023-10-06T05:18:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.076+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe590066f30 0x7fe5900693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.077+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5ac1595d0 0x7fe5ac0b27a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.077+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5ac06b5c0 0x7fe5ac158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.077+0000 7fe5b3bc1700 1 --2- 172.21.15.45:0/3183708027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5ac159200 0x7fe5ac0b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.077+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 >> 172.21.15.45:0/3183708027 conn(0x7fe5ac06f090 msgr2=0x7fe5ac0c9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.077+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 shutdown_connections 2023-10-06T05:18:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.077+0000 7fe5b3bc1700 1 -- 172.21.15.45:0/3183708027 wait complete. 2023-10-06T05:18:04.158 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-10-06T05:18:04.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:18:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.336+0000 7fce586a4700 1 Processor -- start 2023-10-06T05:18:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.336+0000 7fce586a4700 1 -- start start 2023-10-06T05:18:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.337+0000 7fce586a4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce500cd340 0x7fce50157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.337+0000 7fce586a4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5015b000 con 0x7fce500c93c0 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.337+0000 7fce586a4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5015b2e0 con 0x7fce500c8ff0 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.337+0000 7fce586a4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce500c9830 con 0x7fce500cd340 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.337+0000 7fce56c41700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce500cd340 0x7fce50157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.337+0000 7fce55c3f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fce500c93c0 0x7fce50156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35492/0 (socket says 172.21.15.45:35492) 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.337+0000 7fce55c3f700 1 -- 172.21.15.45:0/3389839305 learned_addr learned my addr 172.21.15.45:0/3389839305 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce56c41700 1 -- 172.21.15.45:0/3389839305 >> v1:172.21.15.144:6789/0 conn(0x7fce500c8ff0 legacy=0x7fce500caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce56c41700 1 -- 172.21.15.45:0/3389839305 >> v1:172.21.15.45:6789/0 conn(0x7fce500c93c0 legacy=0x7fce50156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce56c41700 1 -- 172.21.15.45:0/3389839305 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5015b5c0 con 0x7fce500cd340 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce5543e700 1 -- 172.21.15.45:0/3389839305 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168897691 0 0) 0x7fce5015b000 con 0x7fce500c93c0 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce56c41700 1 --2- 172.21.15.45:0/3389839305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500cd340 0x7fce50157200 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fce48009e60 tx=0x7fce48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4429f4612c69882 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce5543e700 1 -- 172.21.15.45:0/3389839305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800e070 con 0x7fce500cd340 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce5543e700 1 -- 172.21.15.45:0/3389839305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800bc30 con 0x7fce500cd340 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.338+0000 7fce5543e700 1 -- 172.21.15.45:0/3389839305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48012430 con 0x7fce500cd340 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.339+0000 7fce586a4700 1 -- 172.21.15.45:0/3389839305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500cd340 msgr2=0x7fce50157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.339+0000 7fce586a4700 1 --2- 172.21.15.45:0/3389839305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500cd340 0x7fce50157200 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fce48009e60 tx=0x7fce48009780 comp rx=0 tx=0).stop 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.339+0000 7fce586a4700 1 -- 172.21.15.45:0/3389839305 shutdown_connections 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.339+0000 7fce586a4700 1 --2- 172.21.15.45:0/3389839305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500cd340 0x7fce50157200 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.339+0000 7fce586a4700 1 -- 172.21.15.45:0/3389839305 >> 172.21.15.45:0/3389839305 conn(0x7fce500cdf50 msgr2=0x7fce500ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.339+0000 7fce586a4700 1 -- 172.21.15.45:0/3389839305 shutdown_connections 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.339+0000 7fce586a4700 1 -- 172.21.15.45:0/3389839305 wait complete. 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.340+0000 7fce586a4700 1 Processor -- start 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.340+0000 7fce586a4700 1 -- start start 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce586a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500c8ff0 0x7fce500b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce586a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce500c93c0 0x7fce500c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce586a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce500cd340 0x7fce500c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce586a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b2960 con 0x7fce500c8ff0 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce586a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b2c40 con 0x7fce500cd340 2023-10-06T05:18:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce586a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b2f20 con 0x7fce500c93c0 2023-10-06T05:18:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500c8ff0 0x7fce500b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500c8ff0 0x7fce500b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36292/0 (socket says 172.21.15.45:36292) 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 -- 172.21.15.45:0/2966613086 learned_addr learned my addr 172.21.15.45:0/2966613086 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56c41700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce500cd340 0x7fce500c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce55c3f700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce500c93c0 0x7fce500c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 -- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce500c93c0 msgr2=0x7fce500c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce500c93c0 0x7fce500c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 -- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce500cd340 msgr2=0x7fce500c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce500cd340 0x7fce500c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 -- 172.21.15.45:0/2966613086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce48009a20 con 0x7fce500c8ff0 2023-10-06T05:18:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56c41700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce500cd340 0x7fce500c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce55c3f700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce500c93c0 0x7fce500c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.341+0000 7fce56440700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500c8ff0 0x7fce500b2080 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b5e0 tx=0x7fce44009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.342+0000 7fce427fc700 1 -- 172.21.15.45:0/2966613086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400f510 con 0x7fce500c8ff0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.342+0000 7fce427fc700 1 -- 172.21.15.45:0/2966613086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4400f670 con 0x7fce500c8ff0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.342+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500b3200 con 0x7fce500c8ff0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.342+0000 7fce427fc700 1 -- 172.21.15.45:0/2966613086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400fa30 con 0x7fce500c8ff0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.342+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce50068bb0 con 0x7fce500c8ff0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.343+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5006f410 con 0x7fce500c8ff0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.343+0000 7fce427fc700 1 -- 172.21.15.45:0/2966613086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce4401a400 con 0x7fce500c8ff0 2023-10-06T05:18:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.344+0000 7fce427fc700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce34066f30 0x7fce340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.344+0000 7fce427fc700 1 -- 172.21.15.45:0/2966613086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fce44083fb0 con 0x7fce500c8ff0 2023-10-06T05:18:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.344+0000 7fce55c3f700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce34066f30 0x7fce340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.345+0000 7fce55c3f700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce34066f30 0x7fce340693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce500b3800 tx=0x7fce4c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.348+0000 7fce427fc700 1 -- 172.21.15.45:0/2966613086 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce44051350 con 0x7fce500c8ff0 2023-10-06T05:18:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.633+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fce500c5000 con 0x7fce500c8ff0 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.633+0000 7fce427fc700 1 -- 172.21.15.45:0/2966613086 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fce44054970 con 0x7fce500c8ff0 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw==", 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w==", 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg==", 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw==", 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg==", 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg==", 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.768 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w==", 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.769 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw==", 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA==", 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ==", 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-10-06T05:18:04.772 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw==", 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA==", 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw==", 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw==", 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g==", 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-10-06T05:18:04.776 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w==", 2023-10-06T05:18:04.777 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:04.777 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:18:04.777 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:18:04.777 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:18:04.777 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:18:04.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:05.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l/0VtGRAA99fxS1gieVbRro7K6P1/lw==", 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:05.095 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A==", 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:05.096 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-10-06T05:18:05.097 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg==", 2023-10-06T05:18:05.097 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:18:05.133 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:18:05.133 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:18:05.133 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:18:05.133 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:05.134 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:05.134 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-06T05:18:05.134 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:18:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce34066f30 msgr2=0x7fce340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce34066f30 0x7fce340693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce500b3800 tx=0x7fce4c009040 comp rx=0 tx=0).stop 2023-10-06T05:18:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500c8ff0 msgr2=0x7fce500b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500c8ff0 0x7fce500b2080 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b5e0 tx=0x7fce44009120 comp rx=0 tx=0).stop 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 shutdown_connections 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce34066f30 0x7fce340693e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce500cd340 0x7fce500c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce500c8ff0 0x7fce500b2080 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 --2- 172.21.15.45:0/2966613086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce500c93c0 0x7fce500c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 >> 172.21.15.45:0/2966613086 conn(0x7fce500cdf50 msgr2=0x7fce501511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 shutdown_connections 2023-10-06T05:18:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:04.636+0000 7fce586a4700 1 -- 172.21.15.45:0/2966613086 wait complete. 2023-10-06T05:18:05.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:05.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.275+0000 7fbdda55f700 1 Processor -- start 2023-10-06T05:18:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.275+0000 7fbdda55f700 1 -- start start 2023-10-06T05:18:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.276+0000 7fbdda55f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd4159200 0x7fbdd40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.276+0000 7fbdda55f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdd40ccdc0 0x7fbdd40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.276+0000 7fbdda55f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd40c7d00 con 0x7fbdd41595d0 2023-10-06T05:18:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.276+0000 7fbdda55f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd40c7780 con 0x7fbdd40ccdc0 2023-10-06T05:18:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.276+0000 7fbdda55f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40cd740 con 0x7fbdd4159200 2023-10-06T05:18:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.276+0000 7fbdd8afc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdd40ccdc0 0x7fbdd40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd37fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbdd41595d0 0x7fbdd406b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35508/0 (socket says 172.21.15.45:35508) 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd37fe700 1 -- 172.21.15.45:0/2983634585 learned_addr learned my addr 172.21.15.45:0/2983634585 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd3fff700 1 --2- 172.21.15.45:0/2983634585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd4159200 0x7fbdd40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd8afc700 1 -- 172.21.15.45:0/2983634585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd4159200 msgr2=0x7fbdd40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd8afc700 1 --2- 172.21.15.45:0/2983634585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd4159200 0x7fbdd40c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd8afc700 1 -- 172.21.15.45:0/2983634585 >> v1:172.21.15.45:6789/0 conn(0x7fbdd41595d0 legacy=0x7fbdd406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd8afc700 1 -- 172.21.15.45:0/2983634585 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40c86d0 con 0x7fbdd40ccdc0 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd3fff700 1 --2- 172.21.15.45:0/2983634585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd4159200 0x7fbdd40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd2ffd700 1 -- 172.21.15.45:0/2983634585 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540257016 0 0) 0x7fbdd40c7d00 con 0x7fbdd41595d0 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd8afc700 1 --2- 172.21.15.45:0/2983634585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd40ccdc0 0x7fbdd40cd200 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009e60 tx=0x7fbdcc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e597a62377ddee67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd2ffd700 1 -- 172.21.15.45:0/2983634585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00e070 con 0x7fbdd40ccdc0 2023-10-06T05:18:05.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd2ffd700 1 -- 172.21.15.45:0/2983634585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc00bc30 con 0x7fbdd40ccdc0 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.277+0000 7fbdd2ffd700 1 -- 172.21.15.45:0/2983634585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc012430 con 0x7fbdd40ccdc0 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 -- 172.21.15.45:0/2983634585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd40ccdc0 msgr2=0x7fbdd40cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 --2- 172.21.15.45:0/2983634585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd40ccdc0 0x7fbdd40cd200 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009e60 tx=0x7fbdcc009780 comp rx=0 tx=0).stop 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 -- 172.21.15.45:0/2983634585 shutdown_connections 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 --2- 172.21.15.45:0/2983634585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd40ccdc0 0x7fbdd40cd200 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 --2- 172.21.15.45:0/2983634585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd4159200 0x7fbdd40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 -- 172.21.15.45:0/2983634585 >> 172.21.15.45:0/2983634585 conn(0x7fbdd406f090 msgr2=0x7fbdd406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 -- 172.21.15.45:0/2983634585 shutdown_connections 2023-10-06T05:18:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.278+0000 7fbdda55f700 1 -- 172.21.15.45:0/2983634585 wait complete. 2023-10-06T05:18:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.279+0000 7fbdda55f700 1 Processor -- start 2023-10-06T05:18:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.280+0000 7fbdda55f700 1 -- start start 2023-10-06T05:18:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdda55f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd40ccdc0 0x7fbdd40c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdda55f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd4159200 0x7fbdd4153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdda55f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdd41595d0 0x7fbdd41699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdda55f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40c4470 con 0x7fbdd4159200 2023-10-06T05:18:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdda55f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40c4e40 con 0x7fbdd41595d0 2023-10-06T05:18:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdda55f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40c3e40 con 0x7fbdd40ccdc0 2023-10-06T05:18:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd4159200 0x7fbdd4153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd4159200 0x7fbdd4153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36314/0 (socket says 172.21.15.45:36314) 2023-10-06T05:18:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 -- 172.21.15.45:0/3911607315 learned_addr learned my addr 172.21.15.45:0/3911607315 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd3fff700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd40ccdc0 0x7fbdd40c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd8afc700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdd41595d0 0x7fbdd41699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 -- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd40ccdc0 msgr2=0x7fbdd40c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd40ccdc0 0x7fbdd40c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 -- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdd41595d0 msgr2=0x7fbdd41699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdd41595d0 0x7fbdd41699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 -- 172.21.15.45:0/3911607315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc009a20 con 0x7fbdd4159200 2023-10-06T05:18:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd8afc700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdd41595d0 0x7fbdd41699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.281+0000 7fbdd37fe700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd4159200 0x7fbdd4153800 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c5b0 tx=0x7fbdc8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.282+0000 7fbdbbfff700 1 -- 172.21.15.45:0/3911607315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8005b30 con 0x7fbdd4159200 2023-10-06T05:18:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.282+0000 7fbdbbfff700 1 -- 172.21.15.45:0/3911607315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8005c90 con 0x7fbdd4159200 2023-10-06T05:18:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.282+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd416a520 con 0x7fbdd4159200 2023-10-06T05:18:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.282+0000 7fbdbbfff700 1 -- 172.21.15.45:0/3911607315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8007300 con 0x7fbdd4159200 2023-10-06T05:18:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.282+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd40c4100 con 0x7fbdd4159200 2023-10-06T05:18:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.283+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd40c45e0 con 0x7fbdd4159200 2023-10-06T05:18:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.284+0000 7fbdbbfff700 1 -- 172.21.15.45:0/3911607315 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdc801b3b0 con 0x7fbdd4159200 2023-10-06T05:18:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.284+0000 7fbdbbfff700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.284+0000 7fbdbbfff700 1 -- 172.21.15.45:0/3911607315 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fbdc8083d90 con 0x7fbdd4159200 2023-10-06T05:18:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.284+0000 7fbdd3fff700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.285+0000 7fbdd3fff700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbdc0066f30 0x7fbdc00693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40c05f0 tx=0x7fbdc4006f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.288+0000 7fbdbbfff700 1 -- 172.21.15.45:0/3911607315 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdc8051130 con 0x7fbdd4159200 2023-10-06T05:18:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.491+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdd406bb40 con 0x7fbdd4159200 2023-10-06T05:18:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.492+0000 7fbdbbfff700 1 -- 172.21.15.45:0/3911607315 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13951 (secure 0 0 0) 0x7fbdc8017090 con 0x7fbdd4159200 2023-10-06T05:18:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:05.556 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":60,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:18:04.080609+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbdc0066f30 msgr2=0x7fbdc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbdc0066f30 0x7fbdc00693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40c05f0 tx=0x7fbdc4006f90 comp rx=0 tx=0).stop 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd4159200 msgr2=0x7fbdd4153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd4159200 0x7fbdd4153800 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c5b0 tx=0x7fbdc8001070 comp rx=0 tx=0).stop 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 shutdown_connections 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdd41595d0 0x7fbdd41699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdd40ccdc0 0x7fbdd40c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 --2- 172.21.15.45:0/3911607315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd4159200 0x7fbdd4153800 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 >> 172.21.15.45:0/3911607315 conn(0x7fbdd406f090 msgr2=0x7fbdd415e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 shutdown_connections 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.494+0000 7fbdda55f700 1 -- 172.21.15.45:0/3911607315 wait complete. 2023-10-06T05:18:05.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:18:05.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:18:05.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.718+0000 7ff0aa2a5700 1 Processor -- start 2023-10-06T05:18:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.718+0000 7ff0aa2a5700 1 -- start start 2023-10-06T05:18:06.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:06.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.719+0000 7ff0aa2a5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a41406e0 0x7ff0a406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.719+0000 7ff0aa2a5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff0a406cd60 0x7ff0a40acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.719+0000 7ff0aa2a5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a40c9940 con 0x7ff0a40ad4e0 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.719+0000 7ff0aa2a5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a406d230 con 0x7ff0a406cd60 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.719+0000 7ff0aa2a5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a406d510 con 0x7ff0a41406e0 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a41406e0 0x7ff0a406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a37fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff0a406cd60 0x7ff0a40acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a41406e0 0x7ff0a406c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:53288/0 (socket says 172.21.15.45:53288) 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a37fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff0a406cd60 0x7ff0a40acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:59220/0 (socket says 172.21.15.45:59220) 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 -- 172.21.15.45:0/3997076925 learned_addr learned my addr 172.21.15.45:0/3997076925 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a8842700 1 --1- 172.21.15.45:0/3997076925 >> v1:172.21.15.144:6789/0 conn(0x7ff0a40ad4e0 0x7ff0a40c9dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:44488/0 (socket says 172.21.15.45:44488) 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 -- 172.21.15.45:0/3997076925 >> v2:172.21.15.144:3300/0 conn(0x7ff0a406cd60 msgr2=0x7ff0a40acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 --2- 172.21.15.45:0/3997076925 >> v2:172.21.15.144:3300/0 conn(0x7ff0a406cd60 0x7ff0a40acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 -- 172.21.15.45:0/3997076925 >> v1:172.21.15.144:6789/0 conn(0x7ff0a40ad4e0 legacy=0x7ff0a40c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 -- 172.21.15.45:0/3997076925 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a406d7f0 con 0x7ff0a41406e0 2023-10-06T05:18:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.720+0000 7ff0a3fff700 1 --2- 172.21.15.45:0/3997076925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a41406e0 0x7ff0a406c820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff0940099b0 tx=0x7ff094009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=581deb0847ecaaf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:06.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0a2ffd700 1 -- 172.21.15.45:0/3997076925 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094009ea0 con 0x7ff0a41406e0 2023-10-06T05:18:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0a2ffd700 1 -- 172.21.15.45:0/3997076925 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09400b860 con 0x7ff0a41406e0 2023-10-06T05:18:06.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0a2ffd700 1 -- 172.21.15.45:0/3997076925 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09400bc20 con 0x7ff0a41406e0 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/3997076925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a41406e0 msgr2=0x7ff0a406c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/3997076925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a41406e0 0x7ff0a406c820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff0940099b0 tx=0x7ff094009250 comp rx=0 tx=0).stop 2023-10-06T05:18:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/3997076925 shutdown_connections 2023-10-06T05:18:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/3997076925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a41406e0 0x7ff0a406c820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/3997076925 >> v2:172.21.15.144:3300/0 conn(0x7ff0a406cd60 0x7ff0a40acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/3997076925 >> 172.21.15.45:0/3997076925 conn(0x7ff0a40691c0 msgr2=0x7ff0a40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:06.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/3997076925 shutdown_connections 2023-10-06T05:18:06.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.721+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/3997076925 wait complete. 2023-10-06T05:18:06.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.723+0000 7ff0aa2a5700 1 Processor -- start 2023-10-06T05:18:06.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.723+0000 7ff0aa2a5700 1 -- start start 2023-10-06T05:18:06.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0aa2a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0a406cd60 0x7ff0a40a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0aa2a5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a40ad4e0 0x7ff0a413d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0aa2a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a41406e0 0x7ff0a4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0aa2a5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40b1f30 con 0x7ff0a41406e0 2023-10-06T05:18:06.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0aa2a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40b2210 con 0x7ff0a406cd60 2023-10-06T05:18:06.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0aa2a5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40b24f0 con 0x7ff0a40ad4e0 2023-10-06T05:18:06.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0a8842700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a41406e0 0x7ff0a4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0a8842700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a41406e0 0x7ff0a4158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36318/0 (socket says 172.21.15.45:36318) 2023-10-06T05:18:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0a8842700 1 -- 172.21.15.45:0/947682645 learned_addr learned my addr 172.21.15.45:0/947682645 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.724+0000 7ff0a3fff700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0a406cd60 0x7ff0a40a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a37fe700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a40ad4e0 0x7ff0a413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a8842700 1 -- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a40ad4e0 msgr2=0x7ff0a413d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a8842700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a40ad4e0 0x7ff0a413d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a8842700 1 -- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0a406cd60 msgr2=0x7ff0a40a88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a8842700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0a406cd60 0x7ff0a40a88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a8842700 1 -- 172.21.15.45:0/947682645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940094f0 con 0x7ff0a41406e0 2023-10-06T05:18:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a37fe700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a40ad4e0 0x7ff0a413d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0a8842700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a41406e0 0x7ff0a4158720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00cd00 tx=0x7ff09c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff08bfff700 1 -- 172.21.15.45:0/947682645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00ce60 con 0x7ff0a41406e0 2023-10-06T05:18:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff08bfff700 1 -- 172.21.15.45:0/947682645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c005df0 con 0x7ff0a41406e0 2023-10-06T05:18:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a40b27d0 con 0x7ff0a41406e0 2023-10-06T05:18:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff08bfff700 1 -- 172.21.15.45:0/947682645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00ab90 con 0x7ff0a41406e0 2023-10-06T05:18:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.725+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a413deb0 con 0x7ff0a41406e0 2023-10-06T05:18:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.726+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a406e2d0 con 0x7ff0a41406e0 2023-10-06T05:18:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.727+0000 7ff08bfff700 1 -- 172.21.15.45:0/947682645 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff09c0070f0 con 0x7ff0a41406e0 2023-10-06T05:18:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.727+0000 7ff08bfff700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff090066f30 0x7ff0900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.727+0000 7ff08bfff700 1 -- 172.21.15.45:0/947682645 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7ff09c084510 con 0x7ff0a41406e0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.727+0000 7ff0a3fff700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff090066f30 0x7ff0900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.728+0000 7ff0a3fff700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff090066f30 0x7ff0900693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff0940099b0 tx=0x7ff094017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.731+0000 7ff08bfff700 1 -- 172.21.15.45:0/947682645 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff09c051830 con 0x7ff0a41406e0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.933+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0a404bb40 con 0x7ff0a41406e0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.934+0000 7ff08bfff700 1 -- 172.21.15.45:0/947682645 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff09c054e50 con 0x7ff0a41406e0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff090066f30 msgr2=0x7ff0900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff090066f30 0x7ff0900693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff0940099b0 tx=0x7ff094017000 comp rx=0 tx=0).stop 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a41406e0 msgr2=0x7ff0a4158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a41406e0 0x7ff0a4158720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00cd00 tx=0x7ff09c001b10 comp rx=0 tx=0).stop 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 shutdown_connections 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff090066f30 0x7ff0900693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a41406e0 0x7ff0a4158720 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0a406cd60 0x7ff0a40a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 --2- 172.21.15.45:0/947682645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0a40ad4e0 0x7ff0a413d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 >> 172.21.15.45:0/947682645 conn(0x7ff0a40691c0 msgr2=0x7ff0a40c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 shutdown_connections 2023-10-06T05:18:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:05.936+0000 7ff0aa2a5700 1 -- 172.21.15.45:0/947682645 wait complete. 2023-10-06T05:18:06.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:18:06.519 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.969+0000 7f0f5e0fe700 1 Processor -- start 2023-10-06T05:18:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.969+0000 7f0f5e0fe700 1 -- start start 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f5e0fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0f58159150 0x7f0f580ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f5e0fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f58159520 0x7f0f5815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f5e0fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5815d3d0 0x7f0f58067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f5e0fe700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580c9c90 con 0x7f0f58159520 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f5e0fe700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5815d8e0 con 0x7f0f58159150 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f5e0fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815dbc0 con 0x7f0f5815d3d0 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f56ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f58159520 0x7f0f5815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f56ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f58159520 0x7f0f5815ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41454/0 (socket says 172.21.15.45:41454) 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f56ffd700 1 -- 172.21.15.45:0/2734694207 learned_addr learned my addr 172.21.15.45:0/2734694207 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f577fe700 1 --2- 172.21.15.45:0/2734694207 >> v2:172.21.15.144:3300/0 conn(0x7f0f58159150 0x7f0f580ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.970+0000 7f0f57fff700 1 --2- 172.21.15.45:0/2734694207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5815d3d0 0x7f0f58067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f56ffd700 1 -- 172.21.15.45:0/2734694207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5815d3d0 msgr2=0x7f0f58067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f56ffd700 1 --2- 172.21.15.45:0/2734694207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5815d3d0 0x7f0f58067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f56ffd700 1 -- 172.21.15.45:0/2734694207 >> v2:172.21.15.144:3300/0 conn(0x7f0f58159150 msgr2=0x7f0f580ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:07.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f56ffd700 1 --2- 172.21.15.45:0/2734694207 >> v2:172.21.15.144:3300/0 conn(0x7f0f58159150 0x7f0f580ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f56ffd700 1 -- 172.21.15.45:0/2734694207 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580c9920 con 0x7f0f58159520 2023-10-06T05:18:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f56ffd700 1 --2- 172.21.15.45:0/2734694207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159520 0x7f0f5815ce90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f580c3d80 tx=0x7f0f40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d29daf7592aadb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f567fc700 1 -- 172.21.15.45:0/2734694207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000b860 con 0x7f0f58159520 2023-10-06T05:18:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f567fc700 1 -- 172.21.15.45:0/2734694207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4000b9c0 con 0x7f0f58159520 2023-10-06T05:18:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f567fc700 1 -- 172.21.15.45:0/2734694207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000bd60 con 0x7f0f58159520 2023-10-06T05:18:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2734694207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159520 msgr2=0x7f0f5815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.971+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2734694207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159520 0x7f0f5815ce90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f580c3d80 tx=0x7f0f40009250 comp rx=0 tx=0).stop 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.972+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2734694207 shutdown_connections 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.972+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2734694207 >> v2:172.21.15.144:3300/0 conn(0x7f0f58159150 0x7f0f580ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.972+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2734694207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5815d3d0 0x7f0f58067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.972+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2734694207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159520 0x7f0f5815ce90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.972+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2734694207 >> 172.21.15.45:0/2734694207 conn(0x7f0f580691c0 msgr2=0x7f0f580cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.972+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2734694207 shutdown_connections 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.972+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2734694207 wait complete. 2023-10-06T05:18:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.973+0000 7f0f5e0fe700 1 Processor -- start 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.974+0000 7f0f5e0fe700 1 -- start start 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.974+0000 7f0f5e0fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f58159150 0x7f0f5814a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.974+0000 7f0f5e0fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5815d3d0 0x7f0f58159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.974+0000 7f0f5e0fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159e40 0x7f0f58158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.974+0000 7f0f5e0fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815af60 con 0x7f0f58159e40 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.974+0000 7f0f5e0fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815a350 con 0x7f0f5815d3d0 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.974+0000 7f0f5e0fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815a630 con 0x7f0f58159150 2023-10-06T05:18:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159e40 0x7f0f58158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159e40 0x7f0f58158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41460/0 (socket says 172.21.15.45:41460) 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 -- 172.21.15.45:0/2305253574 learned_addr learned my addr 172.21.15.45:0/2305253574 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f577fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f58159150 0x7f0f5814a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f56ffd700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5815d3d0 0x7f0f58159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 -- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f58159150 msgr2=0x7f0f5814a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f58159150 0x7f0f5814a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 -- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5815d3d0 msgr2=0x7f0f58159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5815d3d0 0x7f0f58159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 -- 172.21.15.45:0/2305253574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f400094f0 con 0x7f0f58159e40 2023-10-06T05:18:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f57fff700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159e40 0x7f0f58158420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0f50007ef0 tx=0x7f0f50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f4f7fe700 1 -- 172.21.15.45:0/2305253574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000eb30 con 0x7f0f58159e40 2023-10-06T05:18:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f4f7fe700 1 -- 172.21.15.45:0/2305253574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5000ec90 con 0x7f0f58159e40 2023-10-06T05:18:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f4f7fe700 1 -- 172.21.15.45:0/2305253574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50014670 con 0x7f0f58159e40 2023-10-06T05:18:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5815a910 con 0x7f0f58159e40 2023-10-06T05:18:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.975+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5814ae80 con 0x7f0f58159e40 2023-10-06T05:18:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.976+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f580635b0 con 0x7f0f58159e40 2023-10-06T05:18:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.977+0000 7f0f4f7fe700 1 -- 172.21.15.45:0/2305253574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f50014a60 con 0x7f0f58159e40 2023-10-06T05:18:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.977+0000 7f0f4f7fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f38066f30 0x7f0f380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.977+0000 7f0f4f7fe700 1 -- 172.21.15.45:0/2305253574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f0f50084a20 con 0x7f0f58159e40 2023-10-06T05:18:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.977+0000 7f0f577fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f38066f30 0x7f0f380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.978+0000 7f0f577fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f38066f30 0x7f0f380693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0f48006fd0 tx=0x7f0f48008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:06.982+0000 7f0f4f7fe700 1 -- 172.21.15.45:0/2305253574 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f50051d40 con 0x7f0f58159e40 2023-10-06T05:18:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.185+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0f580ca660 con 0x7f0f58159e40 2023-10-06T05:18:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.185+0000 7f0f4f7fe700 1 -- 172.21.15.45:0/2305253574 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v60) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0f50055360 con 0x7f0f58159e40 2023-10-06T05:18:07.275 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.187+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f38066f30 msgr2=0x7f0f380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.187+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f38066f30 0x7f0f380693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0f48006fd0 tx=0x7f0f48008040 comp rx=0 tx=0).stop 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.187+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159e40 msgr2=0x7f0f58158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.187+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159e40 0x7f0f58158420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0f50007ef0 tx=0x7f0f50009840 comp rx=0 tx=0).stop 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 shutdown_connections 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f38066f30 0x7f0f380693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58159e40 0x7f0f58158420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f58159150 0x7f0f5814a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 --2- 172.21.15.45:0/2305253574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5815d3d0 0x7f0f58159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 >> 172.21.15.45:0/2305253574 conn(0x7f0f580691c0 msgr2=0x7f0f580c3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 shutdown_connections 2023-10-06T05:18:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.188+0000 7f0f5e0fe700 1 -- 172.21.15.45:0/2305253574 wait complete. 2023-10-06T05:18:07.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:18:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.411+0000 7f3603eb1700 1 Processor -- start 2023-10-06T05:18:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.411+0000 7f3603eb1700 1 -- start start 2023-10-06T05:18:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3603eb1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35fc159200 0x7f35fc0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3603eb1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f35fc1595d0 0x7f35fc06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3603eb1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc0c7d00 con 0x7f35fc06b5c0 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3603eb1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0c7780 con 0x7f35fc159200 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3603eb1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0c86d0 con 0x7f35fc1595d0 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3601c4d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35fc159200 0x7f35fc0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3601c4d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35fc159200 0x7f35fc0c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41472/0 (socket says 172.21.15.45:41472) 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f3601c4d700 1 -- 172.21.15.45:0/1876615508 learned_addr learned my addr 172.21.15.45:0/1876615508 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.412+0000 7f360144c700 1 --2- 172.21.15.45:0/1876615508 >> v2:172.21.15.144:3300/0 conn(0x7f35fc1595d0 0x7f35fc06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3601c4d700 1 -- 172.21.15.45:0/1876615508 >> v2:172.21.15.144:3300/0 conn(0x7f35fc1595d0 msgr2=0x7f35fc06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3601c4d700 1 --2- 172.21.15.45:0/1876615508 >> v2:172.21.15.144:3300/0 conn(0x7f35fc1595d0 0x7f35fc06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3601c4d700 1 -- 172.21.15.45:0/1876615508 >> v1:172.21.15.144:6789/0 conn(0x7f35fc06b5c0 legacy=0x7f35fc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3601c4d700 1 -- 172.21.15.45:0/1876615508 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc06bb40 con 0x7f35fc159200 2023-10-06T05:18:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3601c4d700 1 --2- 172.21.15.45:0/1876615508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc0c8190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06e790 tx=0x7f35f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10abc7edb8f6a5e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3600c4b700 1 -- 172.21.15.45:0/1876615508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f800b860 con 0x7f35fc159200 2023-10-06T05:18:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3600c4b700 1 -- 172.21.15.45:0/1876615508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f800b9c0 con 0x7f35fc159200 2023-10-06T05:18:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3600c4b700 1 -- 172.21.15.45:0/1876615508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f800bd60 con 0x7f35fc159200 2023-10-06T05:18:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3603eb1700 1 -- 172.21.15.45:0/1876615508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 msgr2=0x7f35fc0c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.413+0000 7f3603eb1700 1 --2- 172.21.15.45:0/1876615508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc0c8190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06e790 tx=0x7f35f8009250 comp rx=0 tx=0).stop 2023-10-06T05:18:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.414+0000 7f3603eb1700 1 -- 172.21.15.45:0/1876615508 shutdown_connections 2023-10-06T05:18:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.414+0000 7f3603eb1700 1 --2- 172.21.15.45:0/1876615508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc0c8190 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.414+0000 7f3603eb1700 1 --2- 172.21.15.45:0/1876615508 >> v2:172.21.15.144:3300/0 conn(0x7f35fc1595d0 0x7f35fc06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.414+0000 7f3603eb1700 1 -- 172.21.15.45:0/1876615508 >> 172.21.15.45:0/1876615508 conn(0x7f35fc06f090 msgr2=0x7f35fc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.414+0000 7f3603eb1700 1 -- 172.21.15.45:0/1876615508 shutdown_connections 2023-10-06T05:18:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.414+0000 7f3603eb1700 1 -- 172.21.15.45:0/1876615508 wait complete. 2023-10-06T05:18:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.415+0000 7f3603eb1700 1 Processor -- start 2023-10-06T05:18:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.415+0000 7f3603eb1700 1 -- start start 2023-10-06T05:18:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3603eb1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc06b5c0 0x7f35fc0b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3603eb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc16d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3603eb1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35fc1595d0 0x7f35fc15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3603eb1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0b2b20 con 0x7f35fc159200 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3603eb1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc16dc00 con 0x7f35fc06b5c0 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3603eb1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c4da0 con 0x7f35fc1595d0 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc16d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc16d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41486/0 (socket says 172.21.15.45:41486) 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 -- 172.21.15.45:0/626463914 learned_addr learned my addr 172.21.15.45:0/626463914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3601c4d700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc06b5c0 0x7f35fc0b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360244e700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35fc1595d0 0x7f35fc15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 -- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35fc1595d0 msgr2=0x7f35fc15b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35fc1595d0 0x7f35fc15b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 -- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc06b5c0 msgr2=0x7f35fc0b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc06b5c0 0x7f35fc0b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 -- 172.21.15.45:0/626463914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80094f0 con 0x7f35fc159200 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360244e700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35fc1595d0 0x7f35fc15b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f3601c4d700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc06b5c0 0x7f35fc0b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f360144c700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc16d6c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d860 tx=0x7f35ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.416+0000 7f35f1ffb700 1 -- 172.21.15.45:0/626463914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec009610 con 0x7f35fc159200 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.417+0000 7f35f1ffb700 1 -- 172.21.15.45:0/626463914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec009770 con 0x7f35fc159200 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.417+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc0c4220 con 0x7f35fc159200 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.417+0000 7f35f1ffb700 1 -- 172.21.15.45:0/626463914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec009b30 con 0x7f35fc159200 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.417+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc0c47e0 con 0x7f35fc159200 2023-10-06T05:18:07.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.418+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc06e2f0 con 0x7f35fc159200 2023-10-06T05:18:07.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.418+0000 7f35f1ffb700 1 -- 172.21.15.45:0/626463914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35ec01e460 con 0x7f35fc159200 2023-10-06T05:18:07.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.419+0000 7f35f1ffb700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35e0066f30 0x7f35e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.419+0000 7f35f1ffb700 1 -- 172.21.15.45:0/626463914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f35ec084af0 con 0x7f35fc159200 2023-10-06T05:18:07.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.419+0000 7f3601c4d700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35e0066f30 0x7f35e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.420+0000 7f3601c4d700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35e0066f30 0x7f35e00693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06e790 tx=0x7f35f8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.423+0000 7f35f1ffb700 1 -- 172.21.15.45:0/626463914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35ec051e10 con 0x7f35fc159200 2023-10-06T05:18:07.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.628+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f35fc06be20 con 0x7f35fc159200 2023-10-06T05:18:07.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.629+0000 7f35f1ffb700 1 -- 172.21.15.45:0/626463914 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+126 (secure 0 0 0) 0x7f35ec055430 con 0x7f35fc159200 2023-10-06T05:18:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:07.657 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:18:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-06T05:18:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-06T05:18:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:18:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-10-06T05:18:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-10-06T05:18:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:07.658 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:18:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35e0066f30 msgr2=0x7f35e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35e0066f30 0x7f35e00693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06e790 tx=0x7f35f8017000 comp rx=0 tx=0).stop 2023-10-06T05:18:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 msgr2=0x7f35fc16d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc16d6c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d860 tx=0x7f35ec008040 comp rx=0 tx=0).stop 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 shutdown_connections 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f35e0066f30 0x7f35e00693e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35fc1595d0 0x7f35fc15b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc06b5c0 0x7f35fc0b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 --2- 172.21.15.45:0/626463914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35fc159200 0x7f35fc16d6c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 >> 172.21.15.45:0/626463914 conn(0x7f35fc06f090 msgr2=0x7f35fc15c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 shutdown_connections 2023-10-06T05:18:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.631+0000 7f3603eb1700 1 -- 172.21.15.45:0/626463914 wait complete. 2023-10-06T05:18:07.661 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:18:07.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:18:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.796+0000 7f0a0ea41700 1 Processor -- start 2023-10-06T05:18:08.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.796+0000 7f0a0ea41700 1 -- start start 2023-10-06T05:18:08.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a0ea41700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a08159490 0x7f0a0815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a0ea41700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a0815d3d0 0x7f0a080c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a0ea41700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a08068040 con 0x7f0a081590c0 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a0ea41700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0815d850 con 0x7f0a08159490 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a0ea41700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815dcf0 con 0x7f0a0815d3d0 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a077fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a08159490 0x7f0a0815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a077fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a08159490 0x7f0a0815ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41500/0 (socket says 172.21.15.45:41500) 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a077fe700 1 -- 172.21.15.45:0/3186732969 learned_addr learned my addr 172.21.15.45:0/3186732969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.797+0000 7f0a0cfde700 1 --2- 172.21.15.45:0/3186732969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a0815d3d0 0x7f0a080c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a077fe700 1 -- 172.21.15.45:0/3186732969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a0815d3d0 msgr2=0x7f0a080c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a077fe700 1 --2- 172.21.15.45:0/3186732969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a0815d3d0 0x7f0a080c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a077fe700 1 -- 172.21.15.45:0/3186732969 >> v1:172.21.15.144:6789/0 conn(0x7f0a081590c0 legacy=0x7f0a080ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a077fe700 1 -- 172.21.15.45:0/3186732969 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08067750 con 0x7f0a08159490 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a077fe700 1 --2- 172.21.15.45:0/3186732969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a0815ce90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0099b0 tx=0x7f09fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c748402413b8534f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a06ffd700 1 -- 172.21.15.45:0/3186732969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc009ea0 con 0x7f0a08159490 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a06ffd700 1 -- 172.21.15.45:0/3186732969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc00b860 con 0x7f0a08159490 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.798+0000 7f0a06ffd700 1 -- 172.21.15.45:0/3186732969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00bc00 con 0x7f0a08159490 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 -- 172.21.15.45:0/3186732969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 msgr2=0x7f0a0815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/3186732969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a0815ce90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0099b0 tx=0x7f09fc009250 comp rx=0 tx=0).stop 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 -- 172.21.15.45:0/3186732969 shutdown_connections 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/3186732969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a0815d3d0 0x7f0a080c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/3186732969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a0815ce90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 -- 172.21.15.45:0/3186732969 >> 172.21.15.45:0/3186732969 conn(0x7f0a080cd060 msgr2=0x7f0a080c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 -- 172.21.15.45:0/3186732969 shutdown_connections 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.799+0000 7f0a0ea41700 1 -- 172.21.15.45:0/3186732969 wait complete. 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.800+0000 7f0a0ea41700 1 Processor -- start 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.800+0000 7f0a0ea41700 1 -- start start 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a0ea41700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a081590c0 0x7f0a081451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a0ea41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a080a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a0ea41700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a0815d3d0 0x7f0a0816cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a0ea41700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080b1b80 con 0x7f0a08159490 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a0ea41700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080aa2c0 con 0x7f0a0815d3d0 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a0ea41700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080aa5a0 con 0x7f0a081590c0 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a080a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a080a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41506/0 (socket says 172.21.15.45:41506) 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a077fe700 1 -- 172.21.15.45:0/1077054494 learned_addr learned my addr 172.21.15.45:0/1077054494 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a0cfde700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a0815d3d0 0x7f0a0816cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.801+0000 7f0a07fff700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a081590c0 0x7f0a081451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a077fe700 1 -- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a081590c0 msgr2=0x7f0a081451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a077fe700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a081590c0 0x7f0a081451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a077fe700 1 -- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a0815d3d0 msgr2=0x7f0a0816cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a077fe700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a0815d3d0 0x7f0a0816cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a077fe700 1 -- 172.21.15.45:0/1077054494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0094f0 con 0x7f0a08159490 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a077fe700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a080a98c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a0806e2b0 tx=0x7f09fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f09effff700 1 -- 172.21.15.45:0/1077054494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc000d10 con 0x7f0a08159490 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f09effff700 1 -- 172.21.15.45:0/1077054494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc00bbe0 con 0x7f0a08159490 2023-10-06T05:18:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a080aa880 con 0x7f0a08159490 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f09effff700 1 -- 172.21.15.45:0/1077054494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc011670 con 0x7f0a08159490 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.802+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a080aae40 con 0x7f0a08159490 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.803+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a080ab030 con 0x7f0a08159490 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.803+0000 7f09effff700 1 -- 172.21.15.45:0/1077054494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09fc011aa0 con 0x7f0a08159490 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.804+0000 7f09effff700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09f4066f30 0x7f09f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.804+0000 7f09effff700 1 -- 172.21.15.45:0/1077054494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f09fc085c90 con 0x7f0a08159490 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.804+0000 7f0a07fff700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09f4066f30 0x7f09f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.805+0000 7f0a07fff700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09f4066f30 0x7f09f40693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f09f8003a80 tx=0x7f09f8007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:07.808+0000 7f09effff700 1 -- 172.21.15.45:0/1077054494 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09fc052fb0 con 0x7f0a08159490 2023-10-06T05:18:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.010+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0a0804c380 con 0x7f0a08159490 2023-10-06T05:18:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.083+0000 7f09effff700 1 -- 172.21.15.45:0/1077054494 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f09fc0565d0 con 0x7f0a08159490 2023-10-06T05:18:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09f4066f30 msgr2=0x7f09f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09f4066f30 0x7f09f40693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f09f8003a80 tx=0x7f09f8007ec0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 msgr2=0x7f0a080a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a080a98c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a0806e2b0 tx=0x7f09fc017000 comp rx=0 tx=0).stop 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 shutdown_connections 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09f4066f30 0x7f09f40693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a0815d3d0 0x7f0a0816cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a081590c0 0x7f0a081451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 --2- 172.21.15.45:0/1077054494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a08159490 0x7f0a080a98c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.085+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 >> 172.21.15.45:0/1077054494 conn(0x7f0a080cd060 msgr2=0x7f0a0815b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.086+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 shutdown_connections 2023-10-06T05:18:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.086+0000 7f0a0ea41700 1 -- 172.21.15.45:0/1077054494 wait complete. 2023-10-06T05:18:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-06T05:18:08.104 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:18:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.242+0000 7fd254fb5700 1 Processor -- start 2023-10-06T05:18:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.242+0000 7fd254fb5700 1 -- start start 2023-10-06T05:18:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.243+0000 7fd254fb5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd2500c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.243+0000 7fd254fb5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2500c9630 con 0x7fd2500c9d50 2023-10-06T05:18:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.243+0000 7fd254fb5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2500ccc70 con 0x7fd25013f900 2023-10-06T05:18:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.243+0000 7fd254fb5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2500ca150 con 0x7fd2500cc840 2023-10-06T05:18:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.243+0000 7fd24ed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd2500c9d50 0x7fd2500affa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36988/0 (socket says 172.21.15.45:36988) 2023-10-06T05:18:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.243+0000 7fd24ed9d700 1 -- 172.21.15.45:0/1407538073 learned_addr learned my addr 172.21.15.45:0/1407538073 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.243+0000 7fd24dd9b700 1 --2- 172.21.15.45:0/1407538073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd2500c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24d59a700 1 -- 172.21.15.45:0/1407538073 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411786743 0 0) 0x7fd2500c9630 con 0x7fd2500c9d50 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24d59a700 1 -- 172.21.15.45:0/1407538073 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd238003570 con 0x7fd2500c9d50 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24d59a700 1 -- 172.21.15.45:0/1407538073 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733442294 0 0) 0x7fd2500ccc70 con 0x7fd25013f900 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24d59a700 1 -- 172.21.15.45:0/1407538073 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2500c9630 con 0x7fd25013f900 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24dd9b700 1 -- 172.21.15.45:0/1407538073 >> v1:172.21.15.144:6789/0 conn(0x7fd25013f900 legacy=0x7fd2500c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24dd9b700 1 -- 172.21.15.45:0/1407538073 >> v1:172.21.15.45:6789/0 conn(0x7fd2500c9d50 legacy=0x7fd2500affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24dd9b700 1 -- 172.21.15.45:0/1407538073 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2500cb550 con 0x7fd2500cc840 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24dd9b700 1 --2- 172.21.15.45:0/1407538073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd2500c9810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd244009910 tx=0x7fd2440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4491d06e70073ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24d59a700 1 -- 172.21.15.45:0/1407538073 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244011630 con 0x7fd2500cc840 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24d59a700 1 -- 172.21.15.45:0/1407538073 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd244011790 con 0x7fd2500cc840 2023-10-06T05:18:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.244+0000 7fd24d59a700 1 -- 172.21.15.45:0/1407538073 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244011b30 con 0x7fd2500cc840 2023-10-06T05:18:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.245+0000 7fd254fb5700 1 -- 172.21.15.45:0/1407538073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 msgr2=0x7fd2500c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.245+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1407538073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd2500c9810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd244009910 tx=0x7fd2440091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.245+0000 7fd254fb5700 1 -- 172.21.15.45:0/1407538073 shutdown_connections 2023-10-06T05:18:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.245+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1407538073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd2500c9810 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.245+0000 7fd254fb5700 1 -- 172.21.15.45:0/1407538073 >> 172.21.15.45:0/1407538073 conn(0x7fd2500691c0 msgr2=0x7fd2500cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.245+0000 7fd254fb5700 1 -- 172.21.15.45:0/1407538073 shutdown_connections 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.245+0000 7fd254fb5700 1 -- 172.21.15.45:0/1407538073 wait complete. 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.246+0000 7fd254fb5700 1 Processor -- start 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.246+0000 7fd254fb5700 1 -- start start 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd254fb5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2500c9d50 0x7fd250159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd254fb5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd25015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd254fb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd25013f900 0x7fd25015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd254fb5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250167180 con 0x7fd25013f900 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd254fb5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25016a2b0 con 0x7fd2500c9d50 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd254fb5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd250169980 con 0x7fd2500cc840 2023-10-06T05:18:08.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd24ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd25013f900 0x7fd25015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd24ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd25013f900 0x7fd25015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41514/0 (socket says 172.21.15.45:41514) 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd24ed9d700 1 -- 172.21.15.45:0/1802345779 learned_addr learned my addr 172.21.15.45:0/1802345779 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd24e59c700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2500c9d50 0x7fd250159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.247+0000 7fd24dd9b700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd25015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24ed9d700 1 -- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 msgr2=0x7fd25015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24ed9d700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd25015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24ed9d700 1 -- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2500c9d50 msgr2=0x7fd250159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24ed9d700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2500c9d50 0x7fd250159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24ed9d700 1 -- 172.21.15.45:0/1802345779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244009420 con 0x7fd25013f900 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24dd9b700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd25015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24e59c700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2500c9d50 0x7fd250159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd24ed9d700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd25013f900 0x7fd25015a650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd24000cd00 tx=0x7fd240001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd23e7fc700 1 -- 172.21.15.45:0/1802345779 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24000ce00 con 0x7fd25013f900 2023-10-06T05:18:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd23e7fc700 1 -- 172.21.15.45:0/1802345779 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24000aac0 con 0x7fd25013f900 2023-10-06T05:18:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd250169c60 con 0x7fd25013f900 2023-10-06T05:18:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd23e7fc700 1 -- 172.21.15.45:0/1802345779 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24000ae80 con 0x7fd25013f900 2023-10-06T05:18:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.248+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd250159840 con 0x7fd25013f900 2023-10-06T05:18:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.249+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25016a440 con 0x7fd25013f900 2023-10-06T05:18:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.250+0000 7fd23e7fc700 1 -- 172.21.15.45:0/1802345779 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd240007300 con 0x7fd25013f900 2023-10-06T05:18:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.250+0000 7fd23e7fc700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd230066f30 0x7fd2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.250+0000 7fd23e7fc700 1 -- 172.21.15.45:0/1802345779 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5896+0+0 (secure 0 0 0) 0x7fd240084920 con 0x7fd25013f900 2023-10-06T05:18:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.250+0000 7fd24e59c700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd230066f30 0x7fd2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.251+0000 7fd24e59c700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd230066f30 0x7fd2300693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd248007fd0 tx=0x7fd248007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.255+0000 7fd23e7fc700 1 -- 172.21.15.45:0/1802345779 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd240051b50 con 0x7fd25013f900 2023-10-06T05:18:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:08.455+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd2500cb830 con 0x7fd25013f900 2023-10-06T05:18:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.092+0000 7fd23e7fc700 1 -- 172.21.15.45:0/1802345779 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd240055170 con 0x7fd25013f900 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd230066f30 msgr2=0x7fd2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd230066f30 0x7fd2300693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd248007fd0 tx=0x7fd248007d50 comp rx=0 tx=0).stop 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd25013f900 msgr2=0x7fd25015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd25013f900 0x7fd25015a650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd24000cd00 tx=0x7fd240001ac0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 shutdown_connections 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd230066f30 0x7fd2300693e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd25013f900 0x7fd25015a650 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2500c9d50 0x7fd250159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 --2- 172.21.15.45:0/1802345779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2500cc840 0x7fd25015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 >> 172.21.15.45:0/1802345779 conn(0x7fd2500691c0 msgr2=0x7fd25013e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 shutdown_connections 2023-10-06T05:18:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.095+0000 7fd254fb5700 1 -- 172.21.15.45:0/1802345779 wait complete. 2023-10-06T05:18:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-06T05:18:09.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:18:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.260+0000 7f3590319700 1 Processor -- start 2023-10-06T05:18:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.260+0000 7f3590319700 1 -- start start 2023-10-06T05:18:09.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f3590319700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3588159150 0x7f35880ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:09.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f3590319700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3588067620 0x7f3588067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:09.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f3590319700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35880c9c90 con 0x7f3588159520 2023-10-06T05:18:09.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f3590319700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3588067f80 con 0x7f3588067620 2023-10-06T05:18:09.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f3590319700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358815d8e0 con 0x7f3588159150 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f358e8b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3588067620 0x7f3588067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f358e8b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3588067620 0x7f3588067a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41516/0 (socket says 172.21.15.45:41516) 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f358e8b6700 1 -- 172.21.15.45:0/4186465099 learned_addr learned my addr 172.21.15.45:0/4186465099 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.261+0000 7f358d8b4700 1 --1- 172.21.15.45:0/4186465099 >> v1:172.21.15.45:6789/0 conn(0x7f3588159520 0x7f358815ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36996/0 (socket says 172.21.15.45:36996) 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358e0b5700 1 --2- 172.21.15.45:0/4186465099 >> v2:172.21.15.144:3300/0 conn(0x7f3588159150 0x7f35880ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358e8b6700 1 -- 172.21.15.45:0/4186465099 >> v2:172.21.15.144:3300/0 conn(0x7f3588159150 msgr2=0x7f35880ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358e8b6700 1 --2- 172.21.15.45:0/4186465099 >> v2:172.21.15.144:3300/0 conn(0x7f3588159150 0x7f35880ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358e8b6700 1 -- 172.21.15.45:0/4186465099 >> v1:172.21.15.45:6789/0 conn(0x7f3588159520 legacy=0x7f358815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358e8b6700 1 -- 172.21.15.45:0/4186465099 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358815dbc0 con 0x7f3588067620 2023-10-06T05:18:09.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358e8b6700 1 --2- 172.21.15.45:0/4186465099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3588067620 0x7f3588067a40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f35880c3d80 tx=0x7f3580009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625f82674f5813ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358d0b3700 1 -- 172.21.15.45:0/4186465099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000e070 con 0x7f3588067620 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358d0b3700 1 -- 172.21.15.45:0/4186465099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000ba50 con 0x7f3588067620 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.262+0000 7f358d0b3700 1 -- 172.21.15.45:0/4186465099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35800092e0 con 0x7f3588067620 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 -- 172.21.15.45:0/4186465099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3588067620 msgr2=0x7f3588067a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 --2- 172.21.15.45:0/4186465099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3588067620 0x7f3588067a40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f35880c3d80 tx=0x7f3580009840 comp rx=0 tx=0).stop 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 -- 172.21.15.45:0/4186465099 shutdown_connections 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 --2- 172.21.15.45:0/4186465099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3588067620 0x7f3588067a40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 --2- 172.21.15.45:0/4186465099 >> v2:172.21.15.144:3300/0 conn(0x7f3588159150 0x7f35880ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 -- 172.21.15.45:0/4186465099 >> 172.21.15.45:0/4186465099 conn(0x7f35880691c0 msgr2=0x7f35880cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 -- 172.21.15.45:0/4186465099 shutdown_connections 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.263+0000 7f3590319700 1 -- 172.21.15.45:0/4186465099 wait complete. 2023-10-06T05:18:09.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.264+0000 7f3590319700 1 Processor -- start 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.264+0000 7f3590319700 1 -- start start 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f3590319700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3588067620 0x7f3588159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f3590319700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3588159150 0x7f358815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f3590319700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f358815b850 0x7f358815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f3590319700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b21d0 con 0x7f358815b850 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f3590319700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b1930 con 0x7f3588159150 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f3590319700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b1c10 con 0x7f3588067620 2023-10-06T05:18:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f358815b850 0x7f358815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f358815b850 0x7f358815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41522/0 (socket says 172.21.15.45:41522) 2023-10-06T05:18:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 -- 172.21.15.45:0/3696551033 learned_addr learned my addr 172.21.15.45:0/3696551033 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358d8b4700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3588159150 0x7f358815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e0b5700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3588067620 0x7f3588159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 -- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3588067620 msgr2=0x7f3588159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3588067620 0x7f3588159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 -- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3588159150 msgr2=0x7f358815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:09.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3588159150 0x7f358815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:09.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 -- 172.21.15.45:0/3696551033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3580009ae0 con 0x7f358815b850 2023-10-06T05:18:09.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e0b5700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3588067620 0x7f3588159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:09.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.265+0000 7f358e8b6700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f358815b850 0x7f358815bcd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f358806c4b0 tx=0x7f3580018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:09.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.266+0000 7f357e7fc700 1 -- 172.21.15.45:0/3696551033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000be60 con 0x7f358815b850 2023-10-06T05:18:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.266+0000 7f357e7fc700 1 -- 172.21.15.45:0/3696551033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3580008dd0 con 0x7f358815b850 2023-10-06T05:18:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.266+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35880b1ef0 con 0x7f358815b850 2023-10-06T05:18:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.266+0000 7f357e7fc700 1 -- 172.21.15.45:0/3696551033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35800266b0 con 0x7f358815b850 2023-10-06T05:18:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.266+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358815c210 con 0x7f358815b850 2023-10-06T05:18:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.267+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358806e2d0 con 0x7f358815b850 2023-10-06T05:18:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.267+0000 7f357e7fc700 1 -- 172.21.15.45:0/3696551033 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3580003040 con 0x7f358815b850 2023-10-06T05:18:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.268+0000 7f357e7fc700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.268+0000 7f357e7fc700 1 -- 172.21.15.45:0/3696551033 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f3580086770 con 0x7f358815b850 2023-10-06T05:18:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.268+0000 7f358e0b5700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.268+0000 7f358e0b5700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f356c066f30 0x7f356c0693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3584006fd0 tx=0x7f3584008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.272+0000 7f357e7fc700 1 -- 172.21.15.45:0/3696551033 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3580053820 con 0x7f358815b850 2023-10-06T05:18:09.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:09.522+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f35880c9920 con 0x7f358815b850 2023-10-06T05:18:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.095+0000 7f357e7fc700 1 -- 172.21.15.45:0/3696551033 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3580009940 con 0x7f358815b850 2023-10-06T05:18:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.100+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f356c066f30 msgr2=0x7f356c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.100+0000 7f3590319700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f356c066f30 0x7f356c0693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3584006fd0 tx=0x7f3584008040 comp rx=0 tx=0).stop 2023-10-06T05:18:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.100+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f358815b850 msgr2=0x7f358815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.100+0000 7f3590319700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f358815b850 0x7f358815bcd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f358806c4b0 tx=0x7f3580018000 comp rx=0 tx=0).stop 2023-10-06T05:18:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 shutdown_connections 2023-10-06T05:18:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f358815b850 0x7f358815bcd0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3588067620 0x7f3588159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 --2- 172.21.15.45:0/3696551033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3588159150 0x7f358815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 >> 172.21.15.45:0/3696551033 conn(0x7f35880691c0 msgr2=0x7f35880c1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 shutdown_connections 2023-10-06T05:18:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.101+0000 7f3590319700 1 -- 172.21.15.45:0/3696551033 wait complete. 2023-10-06T05:18:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 10 and data pool 11 2023-10-06T05:18:10.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.267+0000 7f8584773700 1 Processor -- start 2023-10-06T05:18:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.267+0000 7f8584773700 1 -- start start 2023-10-06T05:18:10.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8584773700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f857c159200 0x7f857c0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8584773700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f857c1595d0 0x7f857c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8584773700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c0c7d00 con 0x7f857c06b5c0 2023-10-06T05:18:10.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8584773700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c0c7780 con 0x7f857c1595d0 2023-10-06T05:18:10.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8584773700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c0c86d0 con 0x7f857c159200 2023-10-06T05:18:10.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8581d0e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f857c1595d0 0x7f857c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8581d0e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f857c1595d0 0x7f857c06b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41526/0 (socket says 172.21.15.45:41526) 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.268+0000 7f8581d0e700 1 -- 172.21.15.45:0/1239097199 learned_addr learned my addr 172.21.15.45:0/1239097199 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f858250f700 1 --2- 172.21.15.45:0/1239097199 >> v2:172.21.15.144:3300/0 conn(0x7f857c159200 0x7f857c0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f8581d0e700 1 -- 172.21.15.45:0/1239097199 >> v2:172.21.15.144:3300/0 conn(0x7f857c159200 msgr2=0x7f857c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f8581d0e700 1 --2- 172.21.15.45:0/1239097199 >> v2:172.21.15.144:3300/0 conn(0x7f857c159200 0x7f857c0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f8581d0e700 1 -- 172.21.15.45:0/1239097199 >> v1:172.21.15.45:6789/0 conn(0x7f857c06b5c0 legacy=0x7f857c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f8581d0e700 1 -- 172.21.15.45:0/1239097199 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c06bb40 con 0x7f857c1595d0 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f858150d700 1 -- 172.21.15.45:0/1239097199 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104200475 0 0) 0x7f857c0c7d00 con 0x7f857c06b5c0 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f8581d0e700 1 --2- 172.21.15.45:0/1239097199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c1595d0 0x7f857c06b080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f857c06ae50 tx=0x7f8570009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ff789a2a914d06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f858150d700 1 -- 172.21.15.45:0/1239097199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570009e70 con 0x7f857c1595d0 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f858150d700 1 -- 172.21.15.45:0/1239097199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857000b860 con 0x7f857c1595d0 2023-10-06T05:18:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.269+0000 7f858150d700 1 -- 172.21.15.45:0/1239097199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857000bc00 con 0x7f857c1595d0 2023-10-06T05:18:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 -- 172.21.15.45:0/1239097199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c1595d0 msgr2=0x7f857c06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 --2- 172.21.15.45:0/1239097199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c1595d0 0x7f857c06b080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f857c06ae50 tx=0x7f8570009250 comp rx=0 tx=0).stop 2023-10-06T05:18:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 -- 172.21.15.45:0/1239097199 shutdown_connections 2023-10-06T05:18:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 --2- 172.21.15.45:0/1239097199 >> v2:172.21.15.144:3300/0 conn(0x7f857c159200 0x7f857c0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 --2- 172.21.15.45:0/1239097199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c1595d0 0x7f857c06b080 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 -- 172.21.15.45:0/1239097199 >> 172.21.15.45:0/1239097199 conn(0x7f857c06f090 msgr2=0x7f857c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 -- 172.21.15.45:0/1239097199 shutdown_connections 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.270+0000 7f8584773700 1 -- 172.21.15.45:0/1239097199 wait complete. 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.271+0000 7f8584773700 1 Processor -- start 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.271+0000 7f8584773700 1 -- start start 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f8584773700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c06b5c0 0x7f857c0c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f8584773700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f857c159200 0x7f857c0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f8584773700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f857c1595d0 0x7f857c157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f8584773700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c0c2f40 con 0x7f857c06b5c0 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f8584773700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c0c2330 con 0x7f857c159200 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f8584773700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857c0c2610 con 0x7f857c1595d0 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c06b5c0 0x7f857c0c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c06b5c0 0x7f857c0c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41534/0 (socket says 172.21.15.45:41534) 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 -- 172.21.15.45:0/2178712878 learned_addr learned my addr 172.21.15.45:0/2178712878 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 -- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f857c1595d0 msgr2=0x7f857c157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f857c1595d0 0x7f857c157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 -- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f857c159200 msgr2=0x7f857c0b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f857c159200 0x7f857c0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 -- 172.21.15.45:0/2178712878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700094f0 con 0x7f857c06b5c0 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.272+0000 7f858250f700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c06b5c0 0x7f857c0c1ae0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f857800ec80 tx=0x7f85780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.273+0000 7f856e7fc700 1 -- 172.21.15.45:0/2178712878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857800c9d0 con 0x7f857c06b5c0 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.273+0000 7f856e7fc700 1 -- 172.21.15.45:0/2178712878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857800cb50 con 0x7f857c06b5c0 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.273+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c0c28f0 con 0x7f857c06b5c0 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.273+0000 7f856e7fc700 1 -- 172.21.15.45:0/2178712878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578013470 con 0x7f857c06b5c0 2023-10-06T05:18:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.273+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c0c2020 con 0x7f857c06b5c0 2023-10-06T05:18:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.274+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857c06e2f0 con 0x7f857c06b5c0 2023-10-06T05:18:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.274+0000 7f856e7fc700 1 -- 172.21.15.45:0/2178712878 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8578013860 con 0x7f857c06b5c0 2023-10-06T05:18:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.275+0000 7f856e7fc700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8560066f30 0x7f85600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.275+0000 7f856e7fc700 1 -- 172.21.15.45:0/2178712878 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8578084ba0 con 0x7f857c06b5c0 2023-10-06T05:18:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.275+0000 7f8581d0e700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8560066f30 0x7f85600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.276+0000 7f8581d0e700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8560066f30 0x7f85600693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f857c06ae50 tx=0x7f8570017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.279+0000 7f856e7fc700 1 -- 172.21.15.45:0/2178712878 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8578051b70 con 0x7f857c06b5c0 2023-10-06T05:18:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.483+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f857c06be20 con 0x7f857c06b5c0 2023-10-06T05:18:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.484+0000 7f856e7fc700 1 -- 172.21.15.45:0/2178712878 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17183 (secure 0 0 0) 0x7f8578055190 con 0x7f857c06b5c0 2023-10-06T05:18:10.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:10.486 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":63,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:18:09.524254+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:08.043401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:08.487972+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:10.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8560066f30 msgr2=0x7f85600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8560066f30 0x7f85600693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f857c06ae50 tx=0x7f8570017000 comp rx=0 tx=0).stop 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c06b5c0 msgr2=0x7f857c0c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c06b5c0 0x7f857c0c1ae0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f857800ec80 tx=0x7f85780091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 shutdown_connections 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8560066f30 0x7f85600693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f857c1595d0 0x7f857c157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857c06b5c0 0x7f857c0c1ae0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 --2- 172.21.15.45:0/2178712878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f857c159200 0x7f857c0b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.486+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 >> 172.21.15.45:0/2178712878 conn(0x7f857c06f090 msgr2=0x7f857c15afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.487+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 shutdown_connections 2023-10-06T05:18:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.487+0000 7f8584773700 1 -- 172.21.15.45:0/2178712878 wait complete. 2023-10-06T05:18:10.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.631+0000 7fa3da2cc700 1 Processor -- start 2023-10-06T05:18:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.631+0000 7fa3da2cc700 1 -- start start 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3da2cc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3d4153650 0x7fa3d415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3da2cc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d406a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3da2cc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa3d406add0 0x7fa3d4068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3da2cc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d406b250 con 0x7fa3d4153650 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3da2cc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d40676c0 con 0x7fa3d406add0 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3da2cc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40679a0 con 0x7fa3d4153a20 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3d3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3d4153650 0x7fa3d415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3d3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3d4153650 0x7fa3d415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41548/0 (socket says 172.21.15.45:41548) 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3d3fff700 1 -- 172.21.15.45:0/98386991 learned_addr learned my addr 172.21.15.45:0/98386991 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.632+0000 7fa3d37fe700 1 --2- 172.21.15.45:0/98386991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d8869700 1 --2- 172.21.15.45:0/98386991 >> v2:172.21.15.144:3300/0 conn(0x7fa3d406add0 0x7fa3d4068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d3fff700 1 -- 172.21.15.45:0/98386991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 msgr2=0x7fa3d406a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d3fff700 1 --2- 172.21.15.45:0/98386991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d406a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d3fff700 1 -- 172.21.15.45:0/98386991 >> v2:172.21.15.144:3300/0 conn(0x7fa3d406add0 msgr2=0x7fa3d4068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d3fff700 1 --2- 172.21.15.45:0/98386991 >> v2:172.21.15.144:3300/0 conn(0x7fa3d406add0 0x7fa3d4068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d3fff700 1 -- 172.21.15.45:0/98386991 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4067c80 con 0x7fa3d4153650 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d3fff700 1 --2- 172.21.15.45:0/98386991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d415a930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40c8e10 tx=0x7fa3c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa474752d28022 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d2ffd700 1 -- 172.21.15.45:0/98386991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400b640 con 0x7fa3d4153650 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d2ffd700 1 -- 172.21.15.45:0/98386991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c400b7a0 con 0x7fa3d4153650 2023-10-06T05:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.633+0000 7fa3d2ffd700 1 -- 172.21.15.45:0/98386991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400bb60 con 0x7fa3d4153650 2023-10-06T05:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 -- 172.21.15.45:0/98386991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 msgr2=0x7fa3d415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/98386991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d415a930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40c8e10 tx=0x7fa3c4009250 comp rx=0 tx=0).stop 2023-10-06T05:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 -- 172.21.15.45:0/98386991 shutdown_connections 2023-10-06T05:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/98386991 >> v2:172.21.15.144:3300/0 conn(0x7fa3d406add0 0x7fa3d4068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/98386991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d415a930 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/98386991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d406a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 -- 172.21.15.45:0/98386991 >> 172.21.15.45:0/98386991 conn(0x7fa3d40cc5a0 msgr2=0x7fa3d40cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 -- 172.21.15.45:0/98386991 shutdown_connections 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.634+0000 7fa3da2cc700 1 -- 172.21.15.45:0/98386991 wait complete. 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.635+0000 7fa3da2cc700 1 Processor -- start 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.635+0000 7fa3da2cc700 1 -- start start 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3da2cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d406add0 0x7fa3d40b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3da2cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d4156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3da2cc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d416d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3da2cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40ae110 con 0x7fa3d4153650 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3da2cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40b0b30 con 0x7fa3d406add0 2023-10-06T05:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3da2cc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d4156ca0 con 0x7fa3d4153a20 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3d37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d4156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3d37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d4156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41564/0 (socket says 172.21.15.45:41564) 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3d37fe700 1 -- 172.21.15.45:0/1075294249 learned_addr learned my addr 172.21.15.45:0/1075294249 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3d3fff700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d406add0 0x7fa3d40b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.636+0000 7fa3d8869700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d416d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3d37fe700 1 -- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 msgr2=0x7fa3d416d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3d37fe700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d416d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3d37fe700 1 -- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d406add0 msgr2=0x7fa3d40b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3d37fe700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d406add0 0x7fa3d40b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3d37fe700 1 -- 172.21.15.45:0/1075294249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40094f0 con 0x7fa3d4153650 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3d37fe700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d4156370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800e900 tx=0x7fa3c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3bbfff700 1 -- 172.21.15.45:0/1075294249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800eea0 con 0x7fa3d4153650 2023-10-06T05:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3bbfff700 1 -- 172.21.15.45:0/1075294249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c800c850 con 0x7fa3d4153650 2023-10-06T05:18:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d40b0d60 con 0x7fa3d4153650 2023-10-06T05:18:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3bbfff700 1 -- 172.21.15.45:0/1075294249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800cc10 con 0x7fa3d4153650 2023-10-06T05:18:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.637+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d40b1320 con 0x7fa3d4153650 2023-10-06T05:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.638+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d40cc0e0 con 0x7fa3d4153650 2023-10-06T05:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.639+0000 7fa3bbfff700 1 -- 172.21.15.45:0/1075294249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3c8013470 con 0x7fa3d4153650 2023-10-06T05:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.639+0000 7fa3bbfff700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c0066f30 0x7fa3c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.639+0000 7fa3bbfff700 1 -- 172.21.15.45:0/1075294249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa3c80849b0 con 0x7fa3d4153650 2023-10-06T05:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.639+0000 7fa3d3fff700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c0066f30 0x7fa3c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.640+0000 7fa3d3fff700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c0066f30 0x7fa3c00693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40c8e10 tx=0x7fa3c4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.644+0000 7fa3bbfff700 1 -- 172.21.15.45:0/1075294249 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3c8051980 con 0x7fa3d4153650 2023-10-06T05:18:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.848+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3d4067f60 con 0x7fa3d4153650 2023-10-06T05:18:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.849+0000 7fa3bbfff700 1 -- 172.21.15.45:0/1075294249 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17183 (secure 0 0 0) 0x7fa3c8054fa0 con 0x7fa3d4153650 2023-10-06T05:18:10.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:10.851 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":63,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:18:09.524254+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:08.043401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:08.487972+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:10.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c0066f30 msgr2=0x7fa3c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c0066f30 0x7fa3c00693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40c8e10 tx=0x7fa3c4017000 comp rx=0 tx=0).stop 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 msgr2=0x7fa3d4156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d4156370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800e900 tx=0x7fa3c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 shutdown_connections 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c0066f30 0x7fa3c00693e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d4153a20 0x7fa3d416d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d406add0 0x7fa3d40b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 --2- 172.21.15.45:0/1075294249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3d4153650 0x7fa3d4156370 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.851+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 >> 172.21.15.45:0/1075294249 conn(0x7fa3d40cc5a0 msgr2=0x7fa3d415e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.852+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 shutdown_connections 2023-10-06T05:18:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:10.852+0000 7fa3da2cc700 1 -- 172.21.15.45:0/1075294249 wait complete. 2023-10-06T05:18:10.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:18:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.001+0000 7f82ed019700 1 Processor -- start 2023-10-06T05:18:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.001+0000 7f82ed019700 1 -- start start 2023-10-06T05:18:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82ed019700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82e80b0610 0x7f82e80c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82ed019700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f82e80c3db0 0x7f82e80697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82ed019700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e80ce6a0 con 0x7f82e8069d00 2023-10-06T05:18:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82ed019700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80690f0 con 0x7f82e80b0610 2023-10-06T05:18:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82ed019700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80ceeb0 con 0x7f82e80c3db0 2023-10-06T05:18:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e6d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82e80b0610 0x7f82e80c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e759e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f82e8069d00 0x7f82e80c2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37014/0 (socket says 172.21.15.45:37014) 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e6d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82e80b0610 0x7f82e80c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41566/0 (socket says 172.21.15.45:41566) 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e759e700 1 -- 172.21.15.45:0/1977526304 learned_addr learned my addr 172.21.15.45:0/1977526304 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e659c700 1 --2- 172.21.15.45:0/1977526304 >> v2:172.21.15.144:3300/0 conn(0x7f82e80c3db0 0x7f82e80697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e6d9d700 1 -- 172.21.15.45:0/1977526304 >> v2:172.21.15.144:3300/0 conn(0x7f82e80c3db0 msgr2=0x7f82e80697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e6d9d700 1 --2- 172.21.15.45:0/1977526304 >> v2:172.21.15.144:3300/0 conn(0x7f82e80c3db0 0x7f82e80697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e6d9d700 1 -- 172.21.15.45:0/1977526304 >> v1:172.21.15.45:6789/0 conn(0x7f82e8069d00 legacy=0x7f82e80c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e6d9d700 1 -- 172.21.15.45:0/1977526304 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8159880 con 0x7f82e80b0610 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.002+0000 7f82e5d9b700 1 -- 172.21.15.45:0/1977526304 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172448388 0 0) 0x7f82e80ce6a0 con 0x7f82e8069d00 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82e6d9d700 1 --2- 172.21.15.45:0/1977526304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80b0610 0x7f82e80c85d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f82e8064110 tx=0x7f82d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0507bb6a6c51f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82e5d9b700 1 -- 172.21.15.45:0/1977526304 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0009e70 con 0x7f82e80b0610 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82e5d9b700 1 -- 172.21.15.45:0/1977526304 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0005c10 con 0x7f82e80b0610 2023-10-06T05:18:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82e5d9b700 1 -- 172.21.15.45:0/1977526304 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000b610 con 0x7f82e80b0610 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 -- 172.21.15.45:0/1977526304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80b0610 msgr2=0x7f82e80c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 --2- 172.21.15.45:0/1977526304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80b0610 0x7f82e80c85d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f82e8064110 tx=0x7f82d0009120 comp rx=0 tx=0).stop 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 -- 172.21.15.45:0/1977526304 shutdown_connections 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 --2- 172.21.15.45:0/1977526304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80b0610 0x7f82e80c85d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 --2- 172.21.15.45:0/1977526304 >> v2:172.21.15.144:3300/0 conn(0x7f82e80c3db0 0x7f82e80697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 -- 172.21.15.45:0/1977526304 >> 172.21.15.45:0/1977526304 conn(0x7f82e8067e00 msgr2=0x7f82e80cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 -- 172.21.15.45:0/1977526304 shutdown_connections 2023-10-06T05:18:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.003+0000 7f82ed019700 1 -- 172.21.15.45:0/1977526304 wait complete. 2023-10-06T05:18:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.005+0000 7f82ed019700 1 Processor -- start 2023-10-06T05:18:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.005+0000 7f82ed019700 1 -- start start 2023-10-06T05:18:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82ed019700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82e8069d00 0x7f82e80b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82ed019700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82e80b0610 0x7f82e80b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82ed019700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80c3db0 0x7f82e80c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82ed019700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e815a0a0 con 0x7f82e80c3db0 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82ed019700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e81574d0 con 0x7f82e80b0610 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82ed019700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b23e0 con 0x7f82e8069d00 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82e759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80c3db0 0x7f82e80c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82e759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80c3db0 0x7f82e80c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41572/0 (socket says 172.21.15.45:41572) 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82e759e700 1 -- 172.21.15.45:0/1899658395 learned_addr learned my addr 172.21.15.45:0/1899658395 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82e659c700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82e80b0610 0x7f82e80b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.006+0000 7f82e6d9d700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82e8069d00 0x7f82e80b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82e759e700 1 -- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82e8069d00 msgr2=0x7f82e80b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82e759e700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82e8069d00 0x7f82e80b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82e759e700 1 -- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82e80b0610 msgr2=0x7f82e80b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82e759e700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82e80b0610 0x7f82e80b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82e759e700 1 -- 172.21.15.45:0/1899658395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d00093c0 con 0x7f82e80c3db0 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82e759e700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80c3db0 0x7f82e80c67b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f82e000cc90 tx=0x7f82e0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82deffd700 1 -- 172.21.15.45:0/1899658395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000a8b0 con 0x7f82e80c3db0 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82deffd700 1 -- 172.21.15.45:0/1899658395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e000aa10 con 0x7f82e80c3db0 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e80b2680 con 0x7f82e80c3db0 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82deffd700 1 -- 172.21.15.45:0/1899658395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000add0 con 0x7f82e80c3db0 2023-10-06T05:18:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.007+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e8159a50 con 0x7f82e80c3db0 2023-10-06T05:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.008+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e815a250 con 0x7f82e80c3db0 2023-10-06T05:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.009+0000 7f82deffd700 1 -- 172.21.15.45:0/1899658395 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82e0007300 con 0x7f82e80c3db0 2023-10-06T05:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.009+0000 7f82deffd700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82c8066f30 0x7f82c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.009+0000 7f82deffd700 1 -- 172.21.15.45:0/1899658395 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f82e0084ab0 con 0x7f82e80c3db0 2023-10-06T05:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.009+0000 7f82e6d9d700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82c8066f30 0x7f82c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.010+0000 7f82e6d9d700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82c8066f30 0x7f82c80693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009160 tx=0x7f82d0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.014+0000 7f82deffd700 1 -- 172.21.15.45:0/1899658395 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82e815a250 con 0x7f82e80c3db0 2023-10-06T05:18:11.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:11.268+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f82e806e350 con 0x7f82e80c3db0 2023-10-06T05:18:12.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.113+0000 7f82deffd700 1 -- 172.21.15.45:0/1899658395 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f82e0051b00 con 0x7f82e80c3db0 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.116+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82c8066f30 msgr2=0x7f82c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.116+0000 7f82ed019700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82c8066f30 0x7f82c80693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009160 tx=0x7f82d0016000 comp rx=0 tx=0).stop 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.116+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80c3db0 msgr2=0x7f82e80c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.116+0000 7f82ed019700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80c3db0 0x7f82e80c67b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f82e000cc90 tx=0x7f82e0001a80 comp rx=0 tx=0).stop 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 shutdown_connections 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82c8066f30 0x7f82c80693e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e80c3db0 0x7f82e80c67b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82e8069d00 0x7f82e80b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 --2- 172.21.15.45:0/1899658395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82e80b0610 0x7f82e80b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 >> 172.21.15.45:0/1899658395 conn(0x7f82e8067e00 msgr2=0x7f82e8158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 shutdown_connections 2023-10-06T05:18:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.117+0000 7f82ed019700 1 -- 172.21.15.45:0/1899658395 wait complete. 2023-10-06T05:18:12.133 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:18:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.271+0000 7f7a0acfa700 1 Processor -- start 2023-10-06T05:18:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.271+0000 7f7a0acfa700 1 -- start start 2023-10-06T05:18:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.271+0000 7f7a0acfa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a04157330 0x7f7a040ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.271+0000 7f7a0acfa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a040ccb70 0x7f7a040c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.271+0000 7f7a0acfa700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a040c3b50 con 0x7f7a040aaae0 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.271+0000 7f7a0acfa700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a04157700 con 0x7f7a04157330 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a0acfa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040c98e0 con 0x7f7a040ccb70 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a04157330 0x7f7a040ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a04157330 0x7f7a040ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41576/0 (socket says 172.21.15.45:41576) 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 -- 172.21.15.45:0/2645714846 learned_addr learned my addr 172.21.15.45:0/2645714846 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a09297700 1 --2- 172.21.15.45:0/2645714846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a040ccb70 0x7f7a040c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 -- 172.21.15.45:0/2645714846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a040ccb70 msgr2=0x7f7a040c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 --2- 172.21.15.45:0/2645714846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a040ccb70 0x7f7a040c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 -- 172.21.15.45:0/2645714846 >> v1:172.21.15.144:6789/0 conn(0x7f7a040aaae0 legacy=0x7f7a040c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 -- 172.21.15.45:0/2645714846 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a040c9bc0 con 0x7f7a04157330 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a08a96700 1 --2- 172.21.15.45:0/2645714846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a04157330 0x7f7a040ce8b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009990 tx=0x7f79f4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f4334bbcd9ccbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a037fe700 1 -- 172.21.15.45:0/2645714846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4005c10 con 0x7f7a04157330 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a037fe700 1 -- 172.21.15.45:0/2645714846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f4005d70 con 0x7f7a04157330 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.272+0000 7f7a037fe700 1 -- 172.21.15.45:0/2645714846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400b880 con 0x7f7a04157330 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 -- 172.21.15.45:0/2645714846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a04157330 msgr2=0x7f7a040ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/2645714846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a04157330 0x7f7a040ce8b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009990 tx=0x7f79f4009220 comp rx=0 tx=0).stop 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 -- 172.21.15.45:0/2645714846 shutdown_connections 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/2645714846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a040ccb70 0x7f7a040c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/2645714846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a04157330 0x7f7a040ce8b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 -- 172.21.15.45:0/2645714846 >> 172.21.15.45:0/2645714846 conn(0x7f7a04067e00 msgr2=0x7f7a040cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 -- 172.21.15.45:0/2645714846 shutdown_connections 2023-10-06T05:18:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.273+0000 7f7a0acfa700 1 -- 172.21.15.45:0/2645714846 wait complete. 2023-10-06T05:18:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.274+0000 7f7a0acfa700 1 Processor -- start 2023-10-06T05:18:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a0acfa700 1 -- start start 2023-10-06T05:18:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a0acfa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a040aaae0 0x7f7a04154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a0acfa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a040ccb70 0x7f7a0416b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a0acfa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a04157330 0x7f7a0416b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a0acfa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a04156350 con 0x7f7a040ccb70 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a0acfa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040b2d30 con 0x7f7a040aaae0 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a0acfa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a04155ec0 con 0x7f7a04157330 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a040ccb70 0x7f7a0416b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a08a96700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a040aaae0 0x7f7a04154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.275+0000 7f7a03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a040ccb70 0x7f7a0416b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41580/0 (socket says 172.21.15.45:41580) 2023-10-06T05:18:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a03fff700 1 -- 172.21.15.45:0/245630868 learned_addr learned my addr 172.21.15.45:0/245630868 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a09297700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a04157330 0x7f7a0416b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a03fff700 1 -- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a04157330 msgr2=0x7f7a0416b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a03fff700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a04157330 0x7f7a0416b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a03fff700 1 -- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a040aaae0 msgr2=0x7f7a04154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a03fff700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a040aaae0 0x7f7a04154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a03fff700 1 -- 172.21.15.45:0/245630868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4009480 con 0x7f7a040ccb70 2023-10-06T05:18:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a03fff700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a040ccb70 0x7f7a0416b050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79f800c4e0 tx=0x7f79f800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a00ff9700 1 -- 172.21.15.45:0/245630868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f800cea0 con 0x7f7a040ccb70 2023-10-06T05:18:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a00ff9700 1 -- 172.21.15.45:0/245630868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8011590 con 0x7f7a040ccb70 2023-10-06T05:18:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a041552b0 con 0x7f7a040ccb70 2023-10-06T05:18:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a00ff9700 1 -- 172.21.15.45:0/245630868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8011930 con 0x7f7a040ccb70 2023-10-06T05:18:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.276+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a04155870 con 0x7f7a040ccb70 2023-10-06T05:18:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.277+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0406e350 con 0x7f7a040ccb70 2023-10-06T05:18:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.278+0000 7f7a00ff9700 1 -- 172.21.15.45:0/245630868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f79f8011d20 con 0x7f7a040ccb70 2023-10-06T05:18:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.278+0000 7f7a00ff9700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f79f0066f30 0x7f79f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.278+0000 7f7a00ff9700 1 -- 172.21.15.45:0/245630868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f79f8083b90 con 0x7f7a040ccb70 2023-10-06T05:18:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.279+0000 7f7a08a96700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f79f0066f30 0x7f79f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.279+0000 7f7a08a96700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f79f0066f30 0x7f79f00693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7a040c3f00 tx=0x7f79f4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:12.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.283+0000 7f7a00ff9700 1 -- 172.21.15.45:0/245630868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79f8050ac0 con 0x7f7a040ccb70 2023-10-06T05:18:12.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:12.529+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a04156000 con 0x7f7a040ccb70 2023-10-06T05:18:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.118+0000 7f7a00ff9700 1 -- 172.21.15.45:0/245630868 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f79f8017070 con 0x7f7a040ccb70 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f79f0066f30 msgr2=0x7f79f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f79f0066f30 0x7f79f00693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7a040c3f00 tx=0x7f79f4016000 comp rx=0 tx=0).stop 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a040ccb70 msgr2=0x7f7a0416b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a040ccb70 0x7f7a0416b050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79f800c4e0 tx=0x7f79f800c7f0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 shutdown_connections 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f79f0066f30 0x7f79f00693e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a04157330 0x7f7a0416b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a040aaae0 0x7f7a04154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 --2- 172.21.15.45:0/245630868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a040ccb70 0x7f7a0416b050 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.121+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 >> 172.21.15.45:0/245630868 conn(0x7f7a04067e00 msgr2=0x7f7a0406cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.122+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 shutdown_connections 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.122+0000 7f7a0acfa700 1 -- 172.21.15.45:0/245630868 wait complete. 2023-10-06T05:18:13.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:13.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.273+0000 7f6da6ba2700 1 Processor -- start 2023-10-06T05:18:13.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.273+0000 7f6da6ba2700 1 -- start start 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da6ba2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 0x7f6da0159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da6ba2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6da015d040 0x7f6da0068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da6ba2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da00ca6d0 con 0x7f6da0069b90 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da6ba2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da015d550 con 0x7f6da015d040 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da6ba2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da015d810 con 0x7f6da0159170 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da513f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6da0069b90 0x7f6da00aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37026/0 (socket says 172.21.15.45:37026) 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da513f700 1 -- 172.21.15.45:0/2657266425 learned_addr learned my addr 172.21.15.45:0/2657266425 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:13.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6da493e700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 0x7f6da0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/2657266425 >> v2:172.21.15.144:3300/0 conn(0x7f6da015d040 0x7f6da0068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.274+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468102504 0 0) 0x7f6da00ca6d0 con 0x7f6da0069b90 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d8c003570 con 0x7f6da0069b90 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 216100388 0 0) 0x7f6d8c003570 con 0x7f6da0069b90 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 msgr2=0x7f6da0159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 0x7f6da0159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 >> v2:172.21.15.144:3300/0 conn(0x7f6da015d040 msgr2=0x7f6da0068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 --2- 172.21.15.45:0/2657266425 >> v2:172.21.15.144:3300/0 conn(0x7f6da015d040 0x7f6da0068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da015daf0 con 0x7f6da0069b90 2023-10-06T05:18:13.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f6d980017f0 con 0x7f6da0069b90 2023-10-06T05:18:13.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 >> v1:172.21.15.45:6789/0 conn(0x7f6da0069b90 legacy=0x7f6da00aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 0x7f6d8c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6d8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c004d50 0x7f6d8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da015d550 con 0x7f6d8c003fe0 2023-10-06T05:18:13.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da015d810 con 0x7f6d8c004d50 2023-10-06T05:18:13.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c00ba90 con 0x7f6da0159170 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6d8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6da493e700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 0x7f6d8c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6da513f700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c004d50 0x7f6d8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9ffff700 1 -- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 msgr2=0x7f6d8c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da0159170 0x7f6d8c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9ffff700 1 -- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c004d50 msgr2=0x7f6d8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d8c004d50 0x7f6d8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.275+0000 7f6d9ffff700 1 -- 172.21.15.45:0/2657266425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c003570 con 0x7f6d8c003fe0 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6d8c004810 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009c00 tx=0x7f6d880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331e05b6a39ac1fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800e6b0 con 0x7f6d8c003fe0 2023-10-06T05:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800e810 con 0x7f6d8c003fe0 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6d9f7fe700 1 -- 172.21.15.45:0/2657266425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800ebd0 con 0x7f6d8c003fe0 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6da6ba2700 1 -- 172.21.15.45:0/2657266425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 msgr2=0x7f6d8c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6da6ba2700 1 --2- 172.21.15.45:0/2657266425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6d8c004810 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009c00 tx=0x7f6d880091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6da513f700 1 -- 172.21.15.45:0/2657266425 reap_dead start 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6da6ba2700 1 -- 172.21.15.45:0/2657266425 shutdown_connections 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6da6ba2700 1 -- 172.21.15.45:0/2657266425 >> 172.21.15.45:0/2657266425 conn(0x7f6da006f7c0 msgr2=0x7f6da00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6da6ba2700 1 -- 172.21.15.45:0/2657266425 shutdown_connections 2023-10-06T05:18:13.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.276+0000 7f6da6ba2700 1 -- 172.21.15.45:0/2657266425 wait complete. 2023-10-06T05:18:13.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.278+0000 7f6da6ba2700 1 Processor -- start 2023-10-06T05:18:13.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.278+0000 7f6da6ba2700 1 -- start start 2023-10-06T05:18:13.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da6ba2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6da00c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da6ba2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da01588f0 0x7f6da015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da6ba2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da00c8cc0 0x7f6da015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da6ba2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00c82f0 con 0x7f6d8c003fe0 2023-10-06T05:18:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da6ba2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da0159890 con 0x7f6da01588f0 2023-10-06T05:18:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da6ba2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00c7700 con 0x7f6da00c8cc0 2023-10-06T05:18:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6da00c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6da00c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41598/0 (socket says 172.21.15.45:41598) 2023-10-06T05:18:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 -- 172.21.15.45:0/3573661639 learned_addr learned my addr 172.21.15.45:0/3573661639 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da513f700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da00c8cc0 0x7f6da015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da01588f0 0x7f6da015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 -- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da00c8cc0 msgr2=0x7f6da015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da00c8cc0 0x7f6da015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 -- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da01588f0 msgr2=0x7f6da015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da01588f0 0x7f6da015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 -- 172.21.15.45:0/3573661639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00c79c0 con 0x7f6d8c003fe0 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da513f700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da00c8cc0 0x7f6da015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da493e700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6da00c8780 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6d90011d40 tx=0x7f6d900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6d9cff9700 1 -- 172.21.15.45:0/3573661639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000fac0 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6d9cff9700 1 -- 172.21.15.45:0/3573661639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9000fc20 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6d9cff9700 1 -- 172.21.15.45:0/3573661639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90014440 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.279+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da00c7ca0 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.280+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da0169730 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.280+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da00c8140 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.281+0000 7f6d9cff9700 1 -- 172.21.15.45:0/3573661639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d90014a80 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.281+0000 7f6d9cff9700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6d80066f30 0x7f6d800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.282+0000 7f6d9cff9700 1 -- 172.21.15.45:0/3573661639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6d90018030 con 0x7f6d8c003fe0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.282+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6d80066f30 0x7f6d800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.282+0000 7f6d9ffff700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6d80066f30 0x7f6d800693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6d880091e0 tx=0x7f6d8801a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.286+0000 7f6d9cff9700 1 -- 172.21.15.45:0/3573661639 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d90052460 con 0x7f6d8c003fe0 2023-10-06T05:18:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.533+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6da0063b50 con 0x7f6d8c003fe0 2023-10-06T05:18:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.534+0000 7f6d9cff9700 1 -- 172.21.15.45:0/3573661639 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f6d90055a80 con 0x7f6d8c003fe0 2023-10-06T05:18:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:13.536 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":48,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:13.113636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6114":{"gid":6114,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/896514784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":896514784},{"type":"v1","addr":"172.21.15.45:6833","nonce":896514784}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6d80066f30 msgr2=0x7f6d800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6d80066f30 0x7f6d800693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6d880091e0 tx=0x7f6d8801a000 comp rx=0 tx=0).stop 2023-10-06T05:18:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 msgr2=0x7f6da00c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6da00c8780 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6d90011d40 tx=0x7f6d900091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 shutdown_connections 2023-10-06T05:18:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6d80066f30 0x7f6d800693e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da00c8cc0 0x7f6da015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d8c003fe0 0x7f6da00c8780 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 --2- 172.21.15.45:0/3573661639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da01588f0 0x7f6da015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 >> 172.21.15.45:0/3573661639 conn(0x7f6da006f7c0 msgr2=0x7f6da00cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 shutdown_connections 2023-10-06T05:18:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.536+0000 7f6da6ba2700 1 -- 172.21.15.45:0/3573661639 wait complete. 2023-10-06T05:18:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 48 2023-10-06T05:18:13.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:13.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.682+0000 7f0f602a0700 1 Processor -- start 2023-10-06T05:18:13.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.682+0000 7f0f602a0700 1 -- start start 2023-10-06T05:18:13.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f602a0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f58155cd0 0x7f0f5815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f602a0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f580ccdc0 0x7f0f580cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f602a0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580cd740 con 0x7f0f581560a0 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f602a0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580ae6b0 con 0x7f0f58155cd0 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f602a0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580ae8d0 con 0x7f0f580ccdc0 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f58155cd0 0x7f0f5815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f58155cd0 0x7f0f5815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41610/0 (socket says 172.21.15.45:41610) 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 -- 172.21.15.45:0/1580972732 learned_addr learned my addr 172.21.15.45:0/1580972732 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e83d700 1 --2- 172.21.15.45:0/1580972732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f580ccdc0 0x7f0f580cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 -- 172.21.15.45:0/1580972732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f580ccdc0 msgr2=0x7f0f580cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 --2- 172.21.15.45:0/1580972732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f580ccdc0 0x7f0f580cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 -- 172.21.15.45:0/1580972732 >> v1:172.21.15.144:6789/0 conn(0x7f0f581560a0 legacy=0x7f0f580adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 -- 172.21.15.45:0/1580972732 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580aebb0 con 0x7f0f58155cd0 2023-10-06T05:18:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.683+0000 7f0f5e03c700 1 --2- 172.21.15.45:0/1580972732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f5815a730 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400bc50 tx=0x7f0f5400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36ca5b3b6366b64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f5d03a700 1 -- 172.21.15.45:0/1580972732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400be60 con 0x7f0f58155cd0 2023-10-06T05:18:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f5d03a700 1 -- 172.21.15.45:0/1580972732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5400de00 con 0x7f0f58155cd0 2023-10-06T05:18:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f5d03a700 1 -- 172.21.15.45:0/1580972732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540196b0 con 0x7f0f58155cd0 2023-10-06T05:18:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 -- 172.21.15.45:0/1580972732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 msgr2=0x7f0f5815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 --2- 172.21.15.45:0/1580972732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f5815a730 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400bc50 tx=0x7f0f5400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 -- 172.21.15.45:0/1580972732 shutdown_connections 2023-10-06T05:18:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 --2- 172.21.15.45:0/1580972732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f580ccdc0 0x7f0f580cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 --2- 172.21.15.45:0/1580972732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f5815a730 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 -- 172.21.15.45:0/1580972732 >> 172.21.15.45:0/1580972732 conn(0x7f0f5806f090 msgr2=0x7f0f5806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 -- 172.21.15.45:0/1580972732 shutdown_connections 2023-10-06T05:18:13.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.684+0000 7f0f602a0700 1 -- 172.21.15.45:0/1580972732 wait complete. 2023-10-06T05:18:13.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.686+0000 7f0f602a0700 1 Processor -- start 2023-10-06T05:18:13.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.686+0000 7f0f602a0700 1 -- start start 2023-10-06T05:18:13.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f602a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f580ccdc0 0x7f0f58151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f602a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f580b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f602a0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f581560a0 0x7f0f580b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f602a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58151a10 con 0x7f0f58155cd0 2023-10-06T05:18:13.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f602a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5814f6c0 con 0x7f0f580ccdc0 2023-10-06T05:18:13.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f602a0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58150e00 con 0x7f0f581560a0 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f5d83b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f580b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f5d83b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f580b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41620/0 (socket says 172.21.15.45:41620) 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f5d83b700 1 -- 172.21.15.45:0/2170654969 learned_addr learned my addr 172.21.15.45:0/2170654969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f5e03c700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f580ccdc0 0x7f0f58151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.687+0000 7f0f5e83d700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f581560a0 0x7f0f580b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f5d83b700 1 -- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f581560a0 msgr2=0x7f0f580b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f5d83b700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f581560a0 0x7f0f580b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f5d83b700 1 -- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f580ccdc0 msgr2=0x7f0f58151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f5d83b700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f580ccdc0 0x7f0f58151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f5d83b700 1 -- 172.21.15.45:0/2170654969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5400b790 con 0x7f0f58155cd0 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f5d83b700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f580b2050 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c4e0 tx=0x7f0f4800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f4e7fc700 1 -- 172.21.15.45:0/2170654969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48011590 con 0x7f0f58155cd0 2023-10-06T05:18:13.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f4e7fc700 1 -- 172.21.15.45:0/2170654969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48011710 con 0x7f0f58155cd0 2023-10-06T05:18:13.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581510e0 con 0x7f0f58155cd0 2023-10-06T05:18:13.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f4e7fc700 1 -- 172.21.15.45:0/2170654969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48011ad0 con 0x7f0f58155cd0 2023-10-06T05:18:13.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.688+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581516a0 con 0x7f0f58155cd0 2023-10-06T05:18:13.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.689+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5806e2f0 con 0x7f0f58155cd0 2023-10-06T05:18:13.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.689+0000 7f0f4e7fc700 1 -- 172.21.15.45:0/2170654969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f4801b3b0 con 0x7f0f58155cd0 2023-10-06T05:18:13.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.690+0000 7f0f4e7fc700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f3c066f30 0x7f0f3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.690+0000 7f0f4e7fc700 1 -- 172.21.15.45:0/2170654969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0f48083bd0 con 0x7f0f58155cd0 2023-10-06T05:18:13.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.690+0000 7f0f5e03c700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f3c066f30 0x7f0f3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.691+0000 7f0f5e03c700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f3c066f30 0x7f0f3c0693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400bc50 tx=0x7f0f5401b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.694+0000 7f0f4e7fc700 1 -- 172.21.15.45:0/2170654969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f48050ba0 con 0x7f0f58155cd0 2023-10-06T05:18:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.898+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f58151b50 con 0x7f0f58155cd0 2023-10-06T05:18:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.898+0000 7f0f4e7fc700 1 -- 172.21.15.45:0/2170654969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17170 (secure 0 0 0) 0x7f0f48017070 con 0x7f0f58155cd0 2023-10-06T05:18:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:13.901 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":64,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:18:11.103758+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:08.043401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:08.487972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f3c066f30 msgr2=0x7f0f3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f3c066f30 0x7f0f3c0693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400bc50 tx=0x7f0f5401b000 comp rx=0 tx=0).stop 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 msgr2=0x7f0f580b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f580b2050 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c4e0 tx=0x7f0f4800c7f0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 shutdown_connections 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f3c066f30 0x7f0f3c0693e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f581560a0 0x7f0f580b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f580ccdc0 0x7f0f58151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 --2- 172.21.15.45:0/2170654969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f58155cd0 0x7f0f580b2050 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 >> 172.21.15.45:0/2170654969 conn(0x7f0f5806f090 msgr2=0x7f0f580cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 shutdown_connections 2023-10-06T05:18:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:13.901+0000 7f0f602a0700 1 -- 172.21.15.45:0/2170654969 wait complete. 2023-10-06T05:18:13.916 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:18:13.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.046+0000 7f9575314700 1 Processor -- start 2023-10-06T05:18:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.046+0000 7f9575314700 1 -- start start 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.047+0000 7f9575314700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9570159170 0x7f9570159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.047+0000 7f9575314700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95700ca6d0 con 0x7f957015d040 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.047+0000 7f9575314700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957015d550 con 0x7f95700692d0 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.047+0000 7f9575314700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957015d810 con 0x7f9570159170 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.047+0000 7f956effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9570159170 0x7f9570159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.047+0000 7f956e7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f957015d040 0x7f9570068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37032/0 (socket says 172.21.15.45:37032) 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.047+0000 7f956e7fc700 1 -- 172.21.15.45:0/35173879 learned_addr learned my addr 172.21.15.45:0/35173879 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956effd700 1 -- 172.21.15.45:0/35173879 >> v1:172.21.15.144:6789/0 conn(0x7f95700692d0 legacy=0x7f95700aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956effd700 1 -- 172.21.15.45:0/35173879 >> v1:172.21.15.45:6789/0 conn(0x7f957015d040 legacy=0x7f9570068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956effd700 1 -- 172.21.15.45:0/35173879 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957015daf0 con 0x7f9570159170 2023-10-06T05:18:14.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956dffb700 1 -- 172.21.15.45:0/35173879 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643898828 0 0) 0x7f95700ca6d0 con 0x7f957015d040 2023-10-06T05:18:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956effd700 1 --2- 172.21.15.45:0/35173879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f9570159540 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95600098e0 tx=0x7f95600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253e6969b3fc1e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956dffb700 1 -- 172.21.15.45:0/35173879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560009e80 con 0x7f9570159170 2023-10-06T05:18:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956dffb700 1 -- 172.21.15.45:0/35173879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000b860 con 0x7f9570159170 2023-10-06T05:18:14.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.048+0000 7f956dffb700 1 -- 172.21.15.45:0/35173879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000bbe0 con 0x7f9570159170 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.049+0000 7f9575314700 1 -- 172.21.15.45:0/35173879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 msgr2=0x7f9570159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.049+0000 7f9575314700 1 --2- 172.21.15.45:0/35173879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f9570159540 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95600098e0 tx=0x7f95600091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.049+0000 7f9575314700 1 -- 172.21.15.45:0/35173879 shutdown_connections 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.049+0000 7f9575314700 1 --2- 172.21.15.45:0/35173879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f9570159540 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.049+0000 7f9575314700 1 -- 172.21.15.45:0/35173879 >> 172.21.15.45:0/35173879 conn(0x7f957006f7c0 msgr2=0x7f95700d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.049+0000 7f9575314700 1 -- 172.21.15.45:0/35173879 shutdown_connections 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.049+0000 7f9575314700 1 -- 172.21.15.45:0/35173879 wait complete. 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.050+0000 7f9575314700 1 Processor -- start 2023-10-06T05:18:14.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.050+0000 7f9575314700 1 -- start start 2023-10-06T05:18:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f9575314700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95700692d0 0x7f95701585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f9575314700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f95700c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f9575314700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957015d040 0x7f957015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f9575314700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700a7dc0 con 0x7f9570159170 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f9575314700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700c8e30 con 0x7f957015d040 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f9575314700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95700c9080 con 0x7f95700692d0 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f95700c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f95700c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41636/0 (socket says 172.21.15.45:41636) 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 -- 172.21.15.45:0/3047362606 learned_addr learned my addr 172.21.15.45:0/3047362606 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956effd700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95700692d0 0x7f95701585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956f7fe700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957015d040 0x7f957015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 -- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95700692d0 msgr2=0x7f95701585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95700692d0 0x7f95701585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 -- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957015d040 msgr2=0x7f957015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957015d040 0x7f957015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 -- 172.21.15.45:0/3047362606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9560009420 con 0x7f9570159170 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.051+0000 7f956e7fc700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f95700c7b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f955800c530 tx=0x7f9558007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.052+0000 7f9566ffd700 1 -- 172.21.15.45:0/3047362606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558005b30 con 0x7f9570159170 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.052+0000 7f9566ffd700 1 -- 172.21.15.45:0/3047362606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558005c90 con 0x7f9570159170 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.052+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95700c9360 con 0x7f9570159170 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.052+0000 7f9566ffd700 1 -- 172.21.15.45:0/3047362606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558007320 con 0x7f9570159170 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.052+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95700c96f0 con 0x7f9570159170 2023-10-06T05:18:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.053+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95700cd8c0 con 0x7f9570159170 2023-10-06T05:18:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.053+0000 7f9566ffd700 1 -- 172.21.15.45:0/3047362606 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f955801b3b0 con 0x7f9570159170 2023-10-06T05:18:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.054+0000 7f9566ffd700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9550066f30 0x7f95500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.054+0000 7f9566ffd700 1 -- 172.21.15.45:0/3047362606 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9558083b50 con 0x7f9570159170 2023-10-06T05:18:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.054+0000 7f956effd700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9550066f30 0x7f95500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.054+0000 7f956effd700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9550066f30 0x7f95500693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f95600093f0 tx=0x7f9560017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.058+0000 7f9566ffd700 1 -- 172.21.15.45:0/3047362606 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9558050b20 con 0x7f9570159170 2023-10-06T05:18:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.255+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9570063b50 con 0x7f9550066f30 2023-10-06T05:18:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.264+0000 7f9566ffd700 1 -- 172.21.15.45:0/3047362606 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312049 (secure 0 0 0) 0x7f9570063b50 con 0x7f9550066f30 2023-10-06T05:18:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9550066f30 msgr2=0x7f95500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9550066f30 0x7f95500693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f95600093f0 tx=0x7f9560017000 comp rx=0 tx=0).stop 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 msgr2=0x7f95700c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f95700c7b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f955800c530 tx=0x7f9558007f90 comp rx=0 tx=0).stop 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 shutdown_connections 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9550066f30 0x7f95500693e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957015d040 0x7f957015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95700692d0 0x7f95701585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 --2- 172.21.15.45:0/3047362606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9570159170 0x7f95700c7b50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 >> 172.21.15.45:0/3047362606 conn(0x7f957006f7c0 msgr2=0x7f95700af160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 shutdown_connections 2023-10-06T05:18:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.268+0000 7f9575314700 1 -- 172.21.15.45:0/3047362606 wait complete. 2023-10-06T05:18:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:18:14.283 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-10-06T05:18:12.494201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385536,"kb_used_data":7232,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589184,"statfs":{"total":773094113280,"available":770651324416,"internally_reserved":0,"allocated":7405568,"data_stored":4090096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.407915"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123670+0000","last_change":"2023-10-06T05:18:10.119657+0000","last_active":"2023-10-06T05:18:10.123670+0000","last_peered":"2023-10-06T05:18:10.123670+0000","last_clean":"2023-10-06T05:18:10.123670+0000","last_became_active":"2023-10-06T05:18:10.119339+0000","last_became_peered":"2023-10-06T05:18:10.119339+0000","last_unstale":"2023-10-06T05:18:10.123670+0000","last_undegraded":"2023-10-06T05:18:10.123670+0000","last_fullsized":"2023-10-06T05:18:10.123670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:48.931102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127791+0000","last_change":"2023-10-06T05:18:10.119792+0000","last_active":"2023-10-06T05:18:10.127791+0000","last_peered":"2023-10-06T05:18:10.127791+0000","last_clean":"2023-10-06T05:18:10.127791+0000","last_became_active":"2023-10-06T05:18:10.119147+0000","last_became_peered":"2023-10-06T05:18:10.119147+0000","last_unstale":"2023-10-06T05:18:10.127791+0000","last_undegraded":"2023-10-06T05:18:10.127791+0000","last_fullsized":"2023-10-06T05:18:10.127791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:56.381468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132678+0000","last_change":"2023-10-06T05:18:10.127793+0000","last_active":"2023-10-06T05:18:10.132678+0000","last_peered":"2023-10-06T05:18:10.132678+0000","last_clean":"2023-10-06T05:18:10.132678+0000","last_became_active":"2023-10-06T05:18:10.123600+0000","last_became_peered":"2023-10-06T05:18:10.123600+0000","last_unstale":"2023-10-06T05:18:10.132678+0000","last_undegraded":"2023-10-06T05:18:10.132678+0000","last_fullsized":"2023-10-06T05:18:10.132678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:02.940820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.129364+0000","last_change":"2023-10-06T05:18:10.126915+0000","last_active":"2023-10-06T05:18:10.129364+0000","last_peered":"2023-10-06T05:18:10.129364+0000","last_clean":"2023-10-06T05:18:10.129364+0000","last_became_active":"2023-10-06T05:18:10.126352+0000","last_became_peered":"2023-10-06T05:18:10.126352+0000","last_unstale":"2023-10-06T05:18:10.129364+0000","last_undegraded":"2023-10-06T05:18:10.129364+0000","last_fullsized":"2023-10-06T05:18:10.129364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:28.330696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.119530+0000","last_change":"2023-10-06T05:18:10.116249+0000","last_active":"2023-10-06T05:18:10.119530+0000","last_peered":"2023-10-06T05:18:10.119530+0000","last_clean":"2023-10-06T05:18:10.119530+0000","last_became_active":"2023-10-06T05:18:10.112733+0000","last_became_peered":"2023-10-06T05:18:10.112733+0000","last_unstale":"2023-10-06T05:18:10.119530+0000","last_undegraded":"2023-10-06T05:18:10.119530+0000","last_fullsized":"2023-10-06T05:18:10.119530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:11.364552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127874+0000","last_change":"2023-10-06T05:18:10.121217+0000","last_active":"2023-10-06T05:18:10.127874+0000","last_peered":"2023-10-06T05:18:10.127874+0000","last_clean":"2023-10-06T05:18:10.127874+0000","last_became_active":"2023-10-06T05:18:10.120953+0000","last_became_peered":"2023-10-06T05:18:10.120953+0000","last_unstale":"2023-10-06T05:18:10.127874+0000","last_undegraded":"2023-10-06T05:18:10.127874+0000","last_fullsized":"2023-10-06T05:18:10.127874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:44.276220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123424+0000","last_change":"2023-10-06T05:18:10.116190+0000","last_active":"2023-10-06T05:18:10.123424+0000","last_peered":"2023-10-06T05:18:10.123424+0000","last_clean":"2023-10-06T05:18:10.123424+0000","last_became_active":"2023-10-06T05:18:10.115940+0000","last_became_peered":"2023-10-06T05:18:10.115940+0000","last_unstale":"2023-10-06T05:18:10.123424+0000","last_undegraded":"2023-10-06T05:18:10.123424+0000","last_fullsized":"2023-10-06T05:18:10.123424+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:28.900345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125752+0000","last_change":"2023-10-06T05:18:10.119970+0000","last_active":"2023-10-06T05:18:10.125752+0000","last_peered":"2023-10-06T05:18:10.125752+0000","last_clean":"2023-10-06T05:18:10.125752+0000","last_became_active":"2023-10-06T05:18:10.119705+0000","last_became_peered":"2023-10-06T05:18:10.119705+0000","last_unstale":"2023-10-06T05:18:10.125752+0000","last_undegraded":"2023-10-06T05:18:10.125752+0000","last_fullsized":"2023-10-06T05:18:10.125752+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:12.014329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132336+0000","last_change":"2023-10-06T05:18:10.130811+0000","last_active":"2023-10-06T05:18:10.132336+0000","last_peered":"2023-10-06T05:18:10.132336+0000","last_clean":"2023-10-06T05:18:10.132336+0000","last_became_active":"2023-10-06T05:18:10.130250+0000","last_became_peered":"2023-10-06T05:18:10.130250+0000","last_unstale":"2023-10-06T05:18:10.132336+0000","last_undegraded":"2023-10-06T05:18:10.132336+0000","last_fullsized":"2023-10-06T05:18:10.132336+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:44.481191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127916+0000","last_change":"2023-10-06T05:18:10.121996+0000","last_active":"2023-10-06T05:18:10.127916+0000","last_peered":"2023-10-06T05:18:10.127916+0000","last_clean":"2023-10-06T05:18:10.127916+0000","last_became_active":"2023-10-06T05:18:10.121712+0000","last_became_peered":"2023-10-06T05:18:10.121712+0000","last_unstale":"2023-10-06T05:18:10.127916+0000","last_undegraded":"2023-10-06T05:18:10.127916+0000","last_fullsized":"2023-10-06T05:18:10.127916+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:47.633801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.133448+0000","last_change":"2023-10-06T05:18:10.128141+0000","last_active":"2023-10-06T05:18:10.133448+0000","last_peered":"2023-10-06T05:18:10.133448+0000","last_clean":"2023-10-06T05:18:10.133448+0000","last_became_active":"2023-10-06T05:18:10.127695+0000","last_became_peered":"2023-10-06T05:18:10.127695+0000","last_unstale":"2023-10-06T05:18:10.133448+0000","last_undegraded":"2023-10-06T05:18:10.133448+0000","last_fullsized":"2023-10-06T05:18:10.133448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:59.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127044+0000","last_change":"2023-10-06T05:18:10.123556+0000","last_active":"2023-10-06T05:18:10.127044+0000","last_peered":"2023-10-06T05:18:10.127044+0000","last_clean":"2023-10-06T05:18:10.127044+0000","last_became_active":"2023-10-06T05:18:10.123275+0000","last_became_peered":"2023-10-06T05:18:10.123275+0000","last_unstale":"2023-10-06T05:18:10.127044+0000","last_undegraded":"2023-10-06T05:18:10.127044+0000","last_fullsized":"2023-10-06T05:18:10.127044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:54.310797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120232+0000","last_change":"2023-10-06T05:18:10.119339+0000","last_active":"2023-10-06T05:18:10.120232+0000","last_peered":"2023-10-06T05:18:10.120232+0000","last_clean":"2023-10-06T05:18:10.120232+0000","last_became_active":"2023-10-06T05:18:10.119069+0000","last_became_peered":"2023-10-06T05:18:10.119069+0000","last_unstale":"2023-10-06T05:18:10.120232+0000","last_undegraded":"2023-10-06T05:18:10.120232+0000","last_fullsized":"2023-10-06T05:18:10.120232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.119595+0000","last_change":"2023-10-06T05:18:10.113625+0000","last_active":"2023-10-06T05:18:10.119595+0000","last_peered":"2023-10-06T05:18:10.119595+0000","last_clean":"2023-10-06T05:18:10.119595+0000","last_became_active":"2023-10-06T05:18:10.113301+0000","last_became_peered":"2023-10-06T05:18:10.113301+0000","last_unstale":"2023-10-06T05:18:10.119595+0000","last_undegraded":"2023-10-06T05:18:10.119595+0000","last_fullsized":"2023-10-06T05:18:10.119595+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:49.968753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.122960+0000","last_change":"2023-10-06T05:18:10.120378+0000","last_active":"2023-10-06T05:18:10.122960+0000","last_peered":"2023-10-06T05:18:10.122960+0000","last_clean":"2023-10-06T05:18:10.122960+0000","last_became_active":"2023-10-06T05:18:10.120111+0000","last_became_peered":"2023-10-06T05:18:10.120111+0000","last_unstale":"2023-10-06T05:18:10.122960+0000","last_undegraded":"2023-10-06T05:18:10.122960+0000","last_fullsized":"2023-10-06T05:18:10.122960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:44.945540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128212+0000","last_change":"2023-10-06T05:18:10.122522+0000","last_active":"2023-10-06T05:18:10.128212+0000","last_peered":"2023-10-06T05:18:10.128212+0000","last_clean":"2023-10-06T05:18:10.128212+0000","last_became_active":"2023-10-06T05:18:10.121501+0000","last_became_peered":"2023-10-06T05:18:10.121501+0000","last_unstale":"2023-10-06T05:18:10.128212+0000","last_undegraded":"2023-10-06T05:18:10.128212+0000","last_fullsized":"2023-10-06T05:18:10.128212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:48.931102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132320+0000","last_change":"2023-10-06T05:18:10.130655+0000","last_active":"2023-10-06T05:18:10.132320+0000","last_peered":"2023-10-06T05:18:10.132320+0000","last_clean":"2023-10-06T05:18:10.132320+0000","last_became_active":"2023-10-06T05:18:10.129891+0000","last_became_peered":"2023-10-06T05:18:10.129891+0000","last_unstale":"2023-10-06T05:18:10.132320+0000","last_undegraded":"2023-10-06T05:18:10.132320+0000","last_fullsized":"2023-10-06T05:18:10.132320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:11.408510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126219+0000","last_change":"2023-10-06T05:18:10.122353+0000","last_active":"2023-10-06T05:18:10.126219+0000","last_peered":"2023-10-06T05:18:10.126219+0000","last_clean":"2023-10-06T05:18:10.126219+0000","last_became_active":"2023-10-06T05:18:10.122084+0000","last_became_peered":"2023-10-06T05:18:10.122084+0000","last_unstale":"2023-10-06T05:18:10.126219+0000","last_undegraded":"2023-10-06T05:18:10.126219+0000","last_fullsized":"2023-10-06T05:18:10.126219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:41.741749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125304+0000","last_change":"2023-10-06T05:18:10.118811+0000","last_active":"2023-10-06T05:18:10.125304+0000","last_peered":"2023-10-06T05:18:10.125304+0000","last_clean":"2023-10-06T05:18:10.125304+0000","last_became_active":"2023-10-06T05:18:10.118466+0000","last_became_peered":"2023-10-06T05:18:10.118466+0000","last_unstale":"2023-10-06T05:18:10.125304+0000","last_undegraded":"2023-10-06T05:18:10.125304+0000","last_fullsized":"2023-10-06T05:18:10.125304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:45.567578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114082+0000","last_change":"2023-10-06T05:18:10.112010+0000","last_active":"2023-10-06T05:18:10.114082+0000","last_peered":"2023-10-06T05:18:10.114082+0000","last_clean":"2023-10-06T05:18:10.114082+0000","last_became_active":"2023-10-06T05:18:10.111731+0000","last_became_peered":"2023-10-06T05:18:10.111731+0000","last_unstale":"2023-10-06T05:18:10.114082+0000","last_undegraded":"2023-10-06T05:18:10.114082+0000","last_fullsized":"2023-10-06T05:18:10.114082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:48.931102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120347+0000","last_change":"2023-10-06T05:18:10.117739+0000","last_active":"2023-10-06T05:18:10.120347+0000","last_peered":"2023-10-06T05:18:10.120347+0000","last_clean":"2023-10-06T05:18:10.120347+0000","last_became_active":"2023-10-06T05:18:10.116864+0000","last_became_peered":"2023-10-06T05:18:10.116864+0000","last_unstale":"2023-10-06T05:18:10.120347+0000","last_undegraded":"2023-10-06T05:18:10.120347+0000","last_fullsized":"2023-10-06T05:18:10.120347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:21.999081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125958+0000","last_change":"2023-10-06T05:18:10.121857+0000","last_active":"2023-10-06T05:18:10.125958+0000","last_peered":"2023-10-06T05:18:10.125958+0000","last_clean":"2023-10-06T05:18:10.125958+0000","last_became_active":"2023-10-06T05:18:10.121556+0000","last_became_peered":"2023-10-06T05:18:10.121556+0000","last_unstale":"2023-10-06T05:18:10.125958+0000","last_undegraded":"2023-10-06T05:18:10.125958+0000","last_fullsized":"2023-10-06T05:18:10.125958+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:30.837974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.107882+0000","last_change":"2023-10-06T05:18:10.100883+0000","last_active":"2023-10-06T05:18:10.107882+0000","last_peered":"2023-10-06T05:18:10.107882+0000","last_clean":"2023-10-06T05:18:10.107882+0000","last_became_active":"2023-10-06T05:18:10.100543+0000","last_became_peered":"2023-10-06T05:18:10.100543+0000","last_unstale":"2023-10-06T05:18:10.107882+0000","last_undegraded":"2023-10-06T05:18:10.107882+0000","last_fullsized":"2023-10-06T05:18:10.107882+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120865+0000","last_change":"2023-10-06T05:18:10.116249+0000","last_active":"2023-10-06T05:18:10.120865+0000","last_peered":"2023-10-06T05:18:10.120865+0000","last_clean":"2023-10-06T05:18:10.120865+0000","last_became_active":"2023-10-06T05:18:10.115564+0000","last_became_peered":"2023-10-06T05:18:10.115564+0000","last_unstale":"2023-10-06T05:18:10.120865+0000","last_undegraded":"2023-10-06T05:18:10.120865+0000","last_fullsized":"2023-10-06T05:18:10.120865+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:16.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.119867+0000","last_change":"2023-10-06T05:18:10.116687+0000","last_active":"2023-10-06T05:18:10.119867+0000","last_peered":"2023-10-06T05:18:10.119867+0000","last_clean":"2023-10-06T05:18:10.119867+0000","last_became_active":"2023-10-06T05:18:10.116236+0000","last_became_peered":"2023-10-06T05:18:10.116236+0000","last_unstale":"2023-10-06T05:18:10.119867+0000","last_undegraded":"2023-10-06T05:18:10.119867+0000","last_fullsized":"2023-10-06T05:18:10.119867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125268+0000","last_change":"2023-10-06T05:18:10.122192+0000","last_active":"2023-10-06T05:18:10.125268+0000","last_peered":"2023-10-06T05:18:10.125268+0000","last_clean":"2023-10-06T05:18:10.125268+0000","last_became_active":"2023-10-06T05:18:10.121893+0000","last_became_peered":"2023-10-06T05:18:10.121893+0000","last_unstale":"2023-10-06T05:18:10.125268+0000","last_undegraded":"2023-10-06T05:18:10.125268+0000","last_fullsized":"2023-10-06T05:18:10.125268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:16.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132656+0000","last_change":"2023-10-06T05:18:10.127021+0000","last_active":"2023-10-06T05:18:10.132656+0000","last_peered":"2023-10-06T05:18:10.132656+0000","last_clean":"2023-10-06T05:18:10.132656+0000","last_became_active":"2023-10-06T05:18:10.118560+0000","last_became_peered":"2023-10-06T05:18:10.118560+0000","last_unstale":"2023-10-06T05:18:10.132656+0000","last_undegraded":"2023-10-06T05:18:10.132656+0000","last_fullsized":"2023-10-06T05:18:10.132656+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:47.633801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125837+0000","last_change":"2023-10-06T05:18:10.122016+0000","last_active":"2023-10-06T05:18:10.125837+0000","last_peered":"2023-10-06T05:18:10.125837+0000","last_clean":"2023-10-06T05:18:10.125837+0000","last_became_active":"2023-10-06T05:18:10.121746+0000","last_became_peered":"2023-10-06T05:18:10.121746+0000","last_unstale":"2023-10-06T05:18:10.125837+0000","last_undegraded":"2023-10-06T05:18:10.125837+0000","last_fullsized":"2023-10-06T05:18:10.125837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:30.712493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126551+0000","last_change":"2023-10-06T05:18:10.118058+0000","last_active":"2023-10-06T05:18:10.126551+0000","last_peered":"2023-10-06T05:18:10.126551+0000","last_clean":"2023-10-06T05:18:10.126551+0000","last_became_active":"2023-10-06T05:18:10.117774+0000","last_became_peered":"2023-10-06T05:18:10.117774+0000","last_unstale":"2023-10-06T05:18:10.126551+0000","last_undegraded":"2023-10-06T05:18:10.126551+0000","last_fullsized":"2023-10-06T05:18:10.126551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:56.588067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125259+0000","last_change":"2023-10-06T05:18:10.122261+0000","last_active":"2023-10-06T05:18:10.125259+0000","last_peered":"2023-10-06T05:18:10.125259+0000","last_clean":"2023-10-06T05:18:10.125259+0000","last_became_active":"2023-10-06T05:18:10.121993+0000","last_became_peered":"2023-10-06T05:18:10.121993+0000","last_unstale":"2023-10-06T05:18:10.125259+0000","last_undegraded":"2023-10-06T05:18:10.125259+0000","last_fullsized":"2023-10-06T05:18:10.125259+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120526+0000","last_change":"2023-10-06T05:18:10.113305+0000","last_active":"2023-10-06T05:18:10.120526+0000","last_peered":"2023-10-06T05:18:10.120526+0000","last_clean":"2023-10-06T05:18:10.120526+0000","last_became_active":"2023-10-06T05:18:10.113112+0000","last_became_peered":"2023-10-06T05:18:10.113112+0000","last_unstale":"2023-10-06T05:18:10.120526+0000","last_undegraded":"2023-10-06T05:18:10.120526+0000","last_fullsized":"2023-10-06T05:18:10.120526+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:57.625054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130602+0000","last_change":"2023-10-06T05:18:10.124955+0000","last_active":"2023-10-06T05:18:10.130602+0000","last_peered":"2023-10-06T05:18:10.130602+0000","last_clean":"2023-10-06T05:18:10.130602+0000","last_became_active":"2023-10-06T05:18:10.124689+0000","last_became_peered":"2023-10-06T05:18:10.124689+0000","last_unstale":"2023-10-06T05:18:10.130602+0000","last_undegraded":"2023-10-06T05:18:10.130602+0000","last_fullsized":"2023-10-06T05:18:10.130602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:44.101041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130473+0000","last_change":"2023-10-06T05:18:10.123556+0000","last_active":"2023-10-06T05:18:10.130473+0000","last_peered":"2023-10-06T05:18:10.130473+0000","last_clean":"2023-10-06T05:18:10.130473+0000","last_became_active":"2023-10-06T05:18:10.123293+0000","last_became_peered":"2023-10-06T05:18:10.123293+0000","last_unstale":"2023-10-06T05:18:10.130473+0000","last_undegraded":"2023-10-06T05:18:10.130473+0000","last_fullsized":"2023-10-06T05:18:10.130473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:27.848967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"63'4","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.112989+0000","last_change":"2023-10-06T05:18:09.113465+0000","last_active":"2023-10-06T05:18:10.112989+0000","last_peered":"2023-10-06T05:18:10.112989+0000","last_clean":"2023-10-06T05:18:10.112989+0000","last_became_active":"2023-10-06T05:18:09.113182+0000","last_became_peered":"2023-10-06T05:18:09.113182+0000","last_unstale":"2023-10-06T05:18:10.112989+0000","last_undegraded":"2023-10-06T05:18:10.112989+0000","last_fullsized":"2023-10-06T05:18:10.112989+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:48.446547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132303+0000","last_change":"2023-10-06T05:18:10.130498+0000","last_active":"2023-10-06T05:18:10.132303+0000","last_peered":"2023-10-06T05:18:10.132303+0000","last_clean":"2023-10-06T05:18:10.132303+0000","last_became_active":"2023-10-06T05:18:10.129602+0000","last_became_peered":"2023-10-06T05:18:10.129602+0000","last_unstale":"2023-10-06T05:18:10.132303+0000","last_undegraded":"2023-10-06T05:18:10.132303+0000","last_fullsized":"2023-10-06T05:18:10.132303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:35.670004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121472+0000","last_change":"2023-10-06T05:18:09.120519+0000","last_active":"2023-10-06T05:18:10.121472+0000","last_peered":"2023-10-06T05:18:10.121472+0000","last_clean":"2023-10-06T05:18:10.121472+0000","last_became_active":"2023-10-06T05:18:09.120179+0000","last_became_peered":"2023-10-06T05:18:09.120179+0000","last_unstale":"2023-10-06T05:18:10.121472+0000","last_undegraded":"2023-10-06T05:18:10.121472+0000","last_fullsized":"2023-10-06T05:18:10.121472+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.857433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128037+0000","last_change":"2023-10-06T05:18:10.122916+0000","last_active":"2023-10-06T05:18:10.128037+0000","last_peered":"2023-10-06T05:18:10.128037+0000","last_clean":"2023-10-06T05:18:10.128037+0000","last_became_active":"2023-10-06T05:18:10.122417+0000","last_became_peered":"2023-10-06T05:18:10.122417+0000","last_unstale":"2023-10-06T05:18:10.128037+0000","last_undegraded":"2023-10-06T05:18:10.128037+0000","last_fullsized":"2023-10-06T05:18:10.128037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.434161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125020+0000","last_change":"2023-10-06T05:18:09.117418+0000","last_active":"2023-10-06T05:18:10.125020+0000","last_peered":"2023-10-06T05:18:10.125020+0000","last_clean":"2023-10-06T05:18:10.125020+0000","last_became_active":"2023-10-06T05:18:09.117138+0000","last_became_peered":"2023-10-06T05:18:09.117138+0000","last_unstale":"2023-10-06T05:18:10.125020+0000","last_undegraded":"2023-10-06T05:18:10.125020+0000","last_fullsized":"2023-10-06T05:18:10.125020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:55.375791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114201+0000","last_change":"2023-10-06T05:18:10.109803+0000","last_active":"2023-10-06T05:18:10.114201+0000","last_peered":"2023-10-06T05:18:10.114201+0000","last_clean":"2023-10-06T05:18:10.114201+0000","last_became_active":"2023-10-06T05:18:10.109498+0000","last_became_peered":"2023-10-06T05:18:10.109498+0000","last_unstale":"2023-10-06T05:18:10.114201+0000","last_undegraded":"2023-10-06T05:18:10.114201+0000","last_fullsized":"2023-10-06T05:18:10.114201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:57.625054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125249+0000","last_change":"2023-10-06T05:18:09.112546+0000","last_active":"2023-10-06T05:18:10.125249+0000","last_peered":"2023-10-06T05:18:10.125249+0000","last_clean":"2023-10-06T05:18:10.125249+0000","last_became_active":"2023-10-06T05:18:09.112283+0000","last_became_peered":"2023-10-06T05:18:09.112283+0000","last_unstale":"2023-10-06T05:18:10.125249+0000","last_undegraded":"2023-10-06T05:18:10.125249+0000","last_fullsized":"2023-10-06T05:18:10.125249+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:56.700869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123601+0000","last_change":"2023-10-06T05:18:10.118448+0000","last_active":"2023-10-06T05:18:10.123601+0000","last_peered":"2023-10-06T05:18:10.123601+0000","last_clean":"2023-10-06T05:18:10.123601+0000","last_became_active":"2023-10-06T05:18:10.118224+0000","last_became_peered":"2023-10-06T05:18:10.118224+0000","last_unstale":"2023-10-06T05:18:10.123601+0000","last_undegraded":"2023-10-06T05:18:10.123601+0000","last_fullsized":"2023-10-06T05:18:10.123601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:27.848967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121951+0000","last_change":"2023-10-06T05:18:09.118792+0000","last_active":"2023-10-06T05:18:10.121951+0000","last_peered":"2023-10-06T05:18:10.121951+0000","last_clean":"2023-10-06T05:18:10.121951+0000","last_became_active":"2023-10-06T05:18:09.118501+0000","last_became_peered":"2023-10-06T05:18:09.118501+0000","last_unstale":"2023-10-06T05:18:10.121951+0000","last_undegraded":"2023-10-06T05:18:10.121951+0000","last_fullsized":"2023-10-06T05:18:10.121951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:16.518119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120329+0000","last_change":"2023-10-06T05:18:10.117359+0000","last_active":"2023-10-06T05:18:10.120329+0000","last_peered":"2023-10-06T05:18:10.120329+0000","last_clean":"2023-10-06T05:18:10.120329+0000","last_became_active":"2023-10-06T05:18:10.116796+0000","last_became_peered":"2023-10-06T05:18:10.116796+0000","last_unstale":"2023-10-06T05:18:10.120329+0000","last_undegraded":"2023-10-06T05:18:10.120329+0000","last_fullsized":"2023-10-06T05:18:10.120329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125778+0000","last_change":"2023-10-06T05:18:09.110260+0000","last_active":"2023-10-06T05:18:10.125778+0000","last_peered":"2023-10-06T05:18:10.125778+0000","last_clean":"2023-10-06T05:18:10.125778+0000","last_became_active":"2023-10-06T05:18:09.110007+0000","last_became_peered":"2023-10-06T05:18:09.110007+0000","last_unstale":"2023-10-06T05:18:10.125778+0000","last_undegraded":"2023-10-06T05:18:10.125778+0000","last_fullsized":"2023-10-06T05:18:10.125778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:03.361424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.131654+0000","last_change":"2023-10-06T05:18:10.121758+0000","last_active":"2023-10-06T05:18:10.131654+0000","last_peered":"2023-10-06T05:18:10.131654+0000","last_clean":"2023-10-06T05:18:10.131654+0000","last_became_active":"2023-10-06T05:18:10.121255+0000","last_became_peered":"2023-10-06T05:18:10.121255+0000","last_unstale":"2023-10-06T05:18:10.131654+0000","last_undegraded":"2023-10-06T05:18:10.131654+0000","last_fullsized":"2023-10-06T05:18:10.131654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:13.192505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.105852+0000","last_change":"2023-10-06T05:18:09.115267+0000","last_active":"2023-10-06T05:18:10.105852+0000","last_peered":"2023-10-06T05:18:10.105852+0000","last_clean":"2023-10-06T05:18:10.105852+0000","last_became_active":"2023-10-06T05:18:09.114960+0000","last_became_peered":"2023-10-06T05:18:09.114960+0000","last_unstale":"2023-10-06T05:18:10.105852+0000","last_undegraded":"2023-10-06T05:18:10.105852+0000","last_fullsized":"2023-10-06T05:18:10.105852+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.052516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114104+0000","last_change":"2023-10-06T05:18:10.112648+0000","last_active":"2023-10-06T05:18:10.114104+0000","last_peered":"2023-10-06T05:18:10.114104+0000","last_clean":"2023-10-06T05:18:10.114104+0000","last_became_active":"2023-10-06T05:18:10.112351+0000","last_became_peered":"2023-10-06T05:18:10.112351+0000","last_unstale":"2023-10-06T05:18:10.114104+0000","last_undegraded":"2023-10-06T05:18:10.114104+0000","last_fullsized":"2023-10-06T05:18:10.114104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:17.523797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.100482+0000","last_change":"2023-10-06T05:18:09.114329+0000","last_active":"2023-10-06T05:18:10.100482+0000","last_peered":"2023-10-06T05:18:10.100482+0000","last_clean":"2023-10-06T05:18:10.100482+0000","last_became_active":"2023-10-06T05:18:09.114007+0000","last_became_peered":"2023-10-06T05:18:09.114007+0000","last_unstale":"2023-10-06T05:18:10.100482+0000","last_undegraded":"2023-10-06T05:18:10.100482+0000","last_fullsized":"2023-10-06T05:18:10.100482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:08.609356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.112415+0000","last_change":"2023-10-06T05:18:10.109090+0000","last_active":"2023-10-06T05:18:10.112415+0000","last_peered":"2023-10-06T05:18:10.112415+0000","last_clean":"2023-10-06T05:18:10.112415+0000","last_became_active":"2023-10-06T05:18:10.108820+0000","last_became_peered":"2023-10-06T05:18:10.108820+0000","last_unstale":"2023-10-06T05:18:10.112415+0000","last_undegraded":"2023-10-06T05:18:10.112415+0000","last_fullsized":"2023-10-06T05:18:10.112415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:17.523797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.108837+0000","last_change":"2023-10-06T05:18:09.114453+0000","last_active":"2023-10-06T05:18:10.108837+0000","last_peered":"2023-10-06T05:18:10.108837+0000","last_clean":"2023-10-06T05:18:10.108837+0000","last_became_active":"2023-10-06T05:18:09.114187+0000","last_became_peered":"2023-10-06T05:18:09.114187+0000","last_unstale":"2023-10-06T05:18:10.108837+0000","last_undegraded":"2023-10-06T05:18:10.108837+0000","last_fullsized":"2023-10-06T05:18:10.108837+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:43.270543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130449+0000","last_change":"2023-10-06T05:18:10.123178+0000","last_active":"2023-10-06T05:18:10.130449+0000","last_peered":"2023-10-06T05:18:10.130449+0000","last_clean":"2023-10-06T05:18:10.130449+0000","last_became_active":"2023-10-06T05:18:10.122914+0000","last_became_peered":"2023-10-06T05:18:10.122914+0000","last_unstale":"2023-10-06T05:18:10.130449+0000","last_undegraded":"2023-10-06T05:18:10.130449+0000","last_fullsized":"2023-10-06T05:18:10.130449+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:17.523797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.115446+0000","last_change":"2023-10-06T05:18:09.114367+0000","last_active":"2023-10-06T05:18:10.115446+0000","last_peered":"2023-10-06T05:18:10.115446+0000","last_clean":"2023-10-06T05:18:10.115446+0000","last_became_active":"2023-10-06T05:18:09.114095+0000","last_became_peered":"2023-10-06T05:18:09.114095+0000","last_unstale":"2023-10-06T05:18:10.115446+0000","last_undegraded":"2023-10-06T05:18:10.115446+0000","last_fullsized":"2023-10-06T05:18:10.115446+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:48.963076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128016+0000","last_change":"2023-10-06T05:18:10.122407+0000","last_active":"2023-10-06T05:18:10.128016+0000","last_peered":"2023-10-06T05:18:10.128016+0000","last_clean":"2023-10-06T05:18:10.128016+0000","last_became_active":"2023-10-06T05:18:10.122114+0000","last_became_peered":"2023-10-06T05:18:10.122114+0000","last_unstale":"2023-10-06T05:18:10.128016+0000","last_undegraded":"2023-10-06T05:18:10.128016+0000","last_fullsized":"2023-10-06T05:18:10.128016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:06.700658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127930+0000","last_change":"2023-10-06T05:18:09.109140+0000","last_active":"2023-10-06T05:18:10.127930+0000","last_peered":"2023-10-06T05:18:10.127930+0000","last_clean":"2023-10-06T05:18:10.127930+0000","last_became_active":"2023-10-06T05:18:09.108726+0000","last_became_peered":"2023-10-06T05:18:09.108726+0000","last_unstale":"2023-10-06T05:18:10.127930+0000","last_undegraded":"2023-10-06T05:18:10.127930+0000","last_fullsized":"2023-10-06T05:18:10.127930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:58.944793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130014+0000","last_change":"2023-10-06T05:18:10.119334+0000","last_active":"2023-10-06T05:18:10.130014+0000","last_peered":"2023-10-06T05:18:10.130014+0000","last_clean":"2023-10-06T05:18:10.130014+0000","last_became_active":"2023-10-06T05:18:10.119050+0000","last_became_peered":"2023-10-06T05:18:10.119050+0000","last_unstale":"2023-10-06T05:18:10.130014+0000","last_undegraded":"2023-10-06T05:18:10.130014+0000","last_fullsized":"2023-10-06T05:18:10.130014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:28.330696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.133752+0000","last_change":"2023-10-06T05:18:09.113355+0000","last_active":"2023-10-06T05:18:10.133752+0000","last_peered":"2023-10-06T05:18:10.133752+0000","last_clean":"2023-10-06T05:18:10.133752+0000","last_became_active":"2023-10-06T05:18:09.113052+0000","last_became_peered":"2023-10-06T05:18:09.113052+0000","last_unstale":"2023-10-06T05:18:10.133752+0000","last_undegraded":"2023-10-06T05:18:10.133752+0000","last_fullsized":"2023-10-06T05:18:10.133752+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:48.020566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120867+0000","last_change":"2023-10-06T05:18:10.114204+0000","last_active":"2023-10-06T05:18:10.120867+0000","last_peered":"2023-10-06T05:18:10.120867+0000","last_clean":"2023-10-06T05:18:10.120867+0000","last_became_active":"2023-10-06T05:18:10.113888+0000","last_became_peered":"2023-10-06T05:18:10.113888+0000","last_unstale":"2023-10-06T05:18:10.120867+0000","last_undegraded":"2023-10-06T05:18:10.120867+0000","last_fullsized":"2023-10-06T05:18:10.120867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:21.999081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121439+0000","last_change":"2023-10-06T05:18:09.113183+0000","last_active":"2023-10-06T05:18:10.121439+0000","last_peered":"2023-10-06T05:18:10.121439+0000","last_clean":"2023-10-06T05:18:10.121439+0000","last_became_active":"2023-10-06T05:18:09.112945+0000","last_became_peered":"2023-10-06T05:18:09.112945+0000","last_unstale":"2023-10-06T05:18:10.121439+0000","last_undegraded":"2023-10-06T05:18:10.121439+0000","last_fullsized":"2023-10-06T05:18:10.121439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:14.036995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.115155+0000","last_change":"2023-10-06T05:18:10.111828+0000","last_active":"2023-10-06T05:18:10.115155+0000","last_peered":"2023-10-06T05:18:10.115155+0000","last_clean":"2023-10-06T05:18:10.115155+0000","last_became_active":"2023-10-06T05:18:10.111600+0000","last_became_peered":"2023-10-06T05:18:10.111600+0000","last_unstale":"2023-10-06T05:18:10.115155+0000","last_undegraded":"2023-10-06T05:18:10.115155+0000","last_fullsized":"2023-10-06T05:18:10.115155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120065+0000","last_change":"2023-10-06T05:18:09.120060+0000","last_active":"2023-10-06T05:18:10.120065+0000","last_peered":"2023-10-06T05:18:10.120065+0000","last_clean":"2023-10-06T05:18:10.120065+0000","last_became_active":"2023-10-06T05:18:09.119815+0000","last_became_peered":"2023-10-06T05:18:09.119815+0000","last_unstale":"2023-10-06T05:18:10.120065+0000","last_undegraded":"2023-10-06T05:18:10.120065+0000","last_fullsized":"2023-10-06T05:18:10.120065+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:48.020566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128297+0000","last_change":"2023-10-06T05:18:10.125863+0000","last_active":"2023-10-06T05:18:10.128297+0000","last_peered":"2023-10-06T05:18:10.128297+0000","last_clean":"2023-10-06T05:18:10.128297+0000","last_became_active":"2023-10-06T05:18:10.125549+0000","last_became_peered":"2023-10-06T05:18:10.125549+0000","last_unstale":"2023-10-06T05:18:10.128297+0000","last_undegraded":"2023-10-06T05:18:10.128297+0000","last_fullsized":"2023-10-06T05:18:10.128297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099518+0000","last_change":"2023-10-06T05:18:09.114978+0000","last_active":"2023-10-06T05:18:10.099518+0000","last_peered":"2023-10-06T05:18:10.099518+0000","last_clean":"2023-10-06T05:18:10.099518+0000","last_became_active":"2023-10-06T05:18:09.114722+0000","last_became_peered":"2023-10-06T05:18:09.114722+0000","last_unstale":"2023-10-06T05:18:10.099518+0000","last_undegraded":"2023-10-06T05:18:10.099518+0000","last_fullsized":"2023-10-06T05:18:10.099518+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:14.428483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132274+0000","last_change":"2023-10-06T05:18:10.131329+0000","last_active":"2023-10-06T05:18:10.132274+0000","last_peered":"2023-10-06T05:18:10.132274+0000","last_clean":"2023-10-06T05:18:10.132274+0000","last_became_active":"2023-10-06T05:18:10.131053+0000","last_became_peered":"2023-10-06T05:18:10.131053+0000","last_unstale":"2023-10-06T05:18:10.132274+0000","last_undegraded":"2023-10-06T05:18:10.132274+0000","last_fullsized":"2023-10-06T05:18:10.132274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:04.367101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123024+0000","last_change":"2023-10-06T05:18:09.113573+0000","last_active":"2023-10-06T05:18:10.123024+0000","last_peered":"2023-10-06T05:18:10.123024+0000","last_clean":"2023-10-06T05:18:10.123024+0000","last_became_active":"2023-10-06T05:18:09.113319+0000","last_became_peered":"2023-10-06T05:18:09.113319+0000","last_unstale":"2023-10-06T05:18:10.123024+0000","last_undegraded":"2023-10-06T05:18:10.123024+0000","last_fullsized":"2023-10-06T05:18:10.123024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:45.638130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120894+0000","last_change":"2023-10-06T05:18:10.119815+0000","last_active":"2023-10-06T05:18:10.120894+0000","last_peered":"2023-10-06T05:18:10.120894+0000","last_clean":"2023-10-06T05:18:10.120894+0000","last_became_active":"2023-10-06T05:18:10.119512+0000","last_became_peered":"2023-10-06T05:18:10.119512+0000","last_unstale":"2023-10-06T05:18:10.120894+0000","last_undegraded":"2023-10-06T05:18:10.120894+0000","last_fullsized":"2023-10-06T05:18:10.120894+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:21.999081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128529+0000","last_change":"2023-10-06T05:18:09.109678+0000","last_active":"2023-10-06T05:18:10.128529+0000","last_peered":"2023-10-06T05:18:10.128529+0000","last_clean":"2023-10-06T05:18:10.128529+0000","last_became_active":"2023-10-06T05:18:09.109397+0000","last_became_peered":"2023-10-06T05:18:09.109397+0000","last_unstale":"2023-10-06T05:18:10.128529+0000","last_undegraded":"2023-10-06T05:18:10.128529+0000","last_fullsized":"2023-10-06T05:18:10.128529+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:56.500917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123386+0000","last_change":"2023-10-06T05:18:10.116625+0000","last_active":"2023-10-06T05:18:10.123386+0000","last_peered":"2023-10-06T05:18:10.123386+0000","last_clean":"2023-10-06T05:18:10.123386+0000","last_became_active":"2023-10-06T05:18:10.116421+0000","last_became_peered":"2023-10-06T05:18:10.116421+0000","last_unstale":"2023-10-06T05:18:10.123386+0000","last_undegraded":"2023-10-06T05:18:10.123386+0000","last_fullsized":"2023-10-06T05:18:10.123386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:59.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099854+0000","last_change":"2023-10-06T05:18:09.113303+0000","last_active":"2023-10-06T05:18:10.099854+0000","last_peered":"2023-10-06T05:18:10.099854+0000","last_clean":"2023-10-06T05:18:10.099854+0000","last_became_active":"2023-10-06T05:18:09.113046+0000","last_became_peered":"2023-10-06T05:18:09.113046+0000","last_unstale":"2023-10-06T05:18:10.099854+0000","last_undegraded":"2023-10-06T05:18:10.099854+0000","last_fullsized":"2023-10-06T05:18:10.099854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:43.095363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121094+0000","last_change":"2023-10-06T05:18:10.115097+0000","last_active":"2023-10-06T05:18:10.121094+0000","last_peered":"2023-10-06T05:18:10.121094+0000","last_clean":"2023-10-06T05:18:10.121094+0000","last_became_active":"2023-10-06T05:18:10.114856+0000","last_became_peered":"2023-10-06T05:18:10.114856+0000","last_unstale":"2023-10-06T05:18:10.121094+0000","last_undegraded":"2023-10-06T05:18:10.121094+0000","last_fullsized":"2023-10-06T05:18:10.121094+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:28.900345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.101678+0000","last_change":"2023-10-06T05:18:09.115555+0000","last_active":"2023-10-06T05:18:10.101678+0000","last_peered":"2023-10-06T05:18:10.101678+0000","last_clean":"2023-10-06T05:18:10.101678+0000","last_became_active":"2023-10-06T05:18:09.115243+0000","last_became_peered":"2023-10-06T05:18:09.115243+0000","last_unstale":"2023-10-06T05:18:10.101678+0000","last_undegraded":"2023-10-06T05:18:10.101678+0000","last_fullsized":"2023-10-06T05:18:10.101678+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:34.059353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.111840+0000","last_change":"2023-10-06T05:18:10.106640+0000","last_active":"2023-10-06T05:18:10.111840+0000","last_peered":"2023-10-06T05:18:10.111840+0000","last_clean":"2023-10-06T05:18:10.111840+0000","last_became_active":"2023-10-06T05:18:10.106422+0000","last_became_peered":"2023-10-06T05:18:10.106422+0000","last_unstale":"2023-10-06T05:18:10.111840+0000","last_undegraded":"2023-10-06T05:18:10.111840+0000","last_fullsized":"2023-10-06T05:18:10.111840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.107391+0000","last_change":"2023-10-06T05:18:09.116724+0000","last_active":"2023-10-06T05:18:10.107391+0000","last_peered":"2023-10-06T05:18:10.107391+0000","last_clean":"2023-10-06T05:18:10.107391+0000","last_became_active":"2023-10-06T05:18:09.116436+0000","last_became_peered":"2023-10-06T05:18:09.116436+0000","last_unstale":"2023-10-06T05:18:10.107391+0000","last_undegraded":"2023-10-06T05:18:10.107391+0000","last_fullsized":"2023-10-06T05:18:10.107391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:01.335370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132578+0000","last_change":"2023-10-06T05:18:10.131377+0000","last_active":"2023-10-06T05:18:10.132578+0000","last_peered":"2023-10-06T05:18:10.132578+0000","last_clean":"2023-10-06T05:18:10.132578+0000","last_became_active":"2023-10-06T05:18:10.131121+0000","last_became_peered":"2023-10-06T05:18:10.131121+0000","last_unstale":"2023-10-06T05:18:10.132578+0000","last_undegraded":"2023-10-06T05:18:10.132578+0000","last_fullsized":"2023-10-06T05:18:10.132578+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:41.741749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.100526+0000","last_change":"2023-10-06T05:18:09.118753+0000","last_active":"2023-10-06T05:18:10.100526+0000","last_peered":"2023-10-06T05:18:10.100526+0000","last_clean":"2023-10-06T05:18:10.100526+0000","last_became_active":"2023-10-06T05:18:09.116166+0000","last_became_peered":"2023-10-06T05:18:09.116166+0000","last_unstale":"2023-10-06T05:18:10.100526+0000","last_undegraded":"2023-10-06T05:18:10.100526+0000","last_fullsized":"2023-10-06T05:18:10.100526+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:45.461675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127852+0000","last_change":"2023-10-06T05:18:10.119831+0000","last_active":"2023-10-06T05:18:10.127852+0000","last_peered":"2023-10-06T05:18:10.127852+0000","last_clean":"2023-10-06T05:18:10.127852+0000","last_became_active":"2023-10-06T05:18:10.118875+0000","last_became_peered":"2023-10-06T05:18:10.118875+0000","last_unstale":"2023-10-06T05:18:10.127852+0000","last_undegraded":"2023-10-06T05:18:10.127852+0000","last_fullsized":"2023-10-06T05:18:10.127852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:01.774391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.105629+0000","last_change":"2023-10-06T05:18:09.115262+0000","last_active":"2023-10-06T05:18:10.105629+0000","last_peered":"2023-10-06T05:18:10.105629+0000","last_clean":"2023-10-06T05:18:10.105629+0000","last_became_active":"2023-10-06T05:18:09.114932+0000","last_became_peered":"2023-10-06T05:18:09.114932+0000","last_unstale":"2023-10-06T05:18:10.105629+0000","last_undegraded":"2023-10-06T05:18:10.105629+0000","last_fullsized":"2023-10-06T05:18:10.105629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:34.664326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.129542+0000","last_change":"2023-10-06T05:18:10.128718+0000","last_active":"2023-10-06T05:18:10.129542+0000","last_peered":"2023-10-06T05:18:10.129542+0000","last_clean":"2023-10-06T05:18:10.129542+0000","last_became_active":"2023-10-06T05:18:10.128411+0000","last_became_peered":"2023-10-06T05:18:10.128411+0000","last_unstale":"2023-10-06T05:18:10.129542+0000","last_undegraded":"2023-10-06T05:18:10.129542+0000","last_fullsized":"2023-10-06T05:18:10.129542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:49.968753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.108936+0000","last_change":"2023-10-06T05:18:09.116301+0000","last_active":"2023-10-06T05:18:10.108936+0000","last_peered":"2023-10-06T05:18:10.108936+0000","last_clean":"2023-10-06T05:18:10.108936+0000","last_became_active":"2023-10-06T05:18:09.116057+0000","last_became_peered":"2023-10-06T05:18:09.116057+0000","last_unstale":"2023-10-06T05:18:10.108936+0000","last_undegraded":"2023-10-06T05:18:10.108936+0000","last_fullsized":"2023-10-06T05:18:10.108936+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.857433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130755+0000","last_change":"2023-10-06T05:18:10.127141+0000","last_active":"2023-10-06T05:18:10.130755+0000","last_peered":"2023-10-06T05:18:10.130755+0000","last_clean":"2023-10-06T05:18:10.130755+0000","last_became_active":"2023-10-06T05:18:10.126906+0000","last_became_peered":"2023-10-06T05:18:10.126906+0000","last_unstale":"2023-10-06T05:18:10.130755+0000","last_undegraded":"2023-10-06T05:18:10.130755+0000","last_fullsized":"2023-10-06T05:18:10.130755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:49.452224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114838+0000","last_change":"2023-10-06T05:18:09.110714+0000","last_active":"2023-10-06T05:18:10.114838+0000","last_peered":"2023-10-06T05:18:10.114838+0000","last_clean":"2023-10-06T05:18:10.114838+0000","last_became_active":"2023-10-06T05:18:09.110431+0000","last_became_peered":"2023-10-06T05:18:09.110431+0000","last_unstale":"2023-10-06T05:18:10.114838+0000","last_undegraded":"2023-10-06T05:18:10.114838+0000","last_fullsized":"2023-10-06T05:18:10.114838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:14.036995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130153+0000","last_change":"2023-10-06T05:18:10.121750+0000","last_active":"2023-10-06T05:18:10.130153+0000","last_peered":"2023-10-06T05:18:10.130153+0000","last_clean":"2023-10-06T05:18:10.130153+0000","last_became_active":"2023-10-06T05:18:10.121546+0000","last_became_peered":"2023-10-06T05:18:10.121546+0000","last_unstale":"2023-10-06T05:18:10.130153+0000","last_undegraded":"2023-10-06T05:18:10.130153+0000","last_fullsized":"2023-10-06T05:18:10.130153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:54.310797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.096605+0000","last_change":"2023-10-06T05:18:09.115158+0000","last_active":"2023-10-06T05:18:10.096605+0000","last_peered":"2023-10-06T05:18:10.096605+0000","last_clean":"2023-10-06T05:18:10.096605+0000","last_became_active":"2023-10-06T05:18:09.114899+0000","last_became_peered":"2023-10-06T05:18:09.114899+0000","last_unstale":"2023-10-06T05:18:10.096605+0000","last_undegraded":"2023-10-06T05:18:10.096605+0000","last_fullsized":"2023-10-06T05:18:10.096605+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:44.561901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.113001+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:18:10.113001+0000","last_peered":"2023-10-06T05:18:10.113001+0000","last_clean":"2023-10-06T05:18:10.113001+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:18:10.113001+0000","last_undegraded":"2023-10-06T05:18:10.113001+0000","last_fullsized":"2023-10-06T05:18:10.113001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126571+0000","last_change":"2023-10-06T05:18:10.122106+0000","last_active":"2023-10-06T05:18:10.126571+0000","last_peered":"2023-10-06T05:18:10.126571+0000","last_clean":"2023-10-06T05:18:10.126571+0000","last_became_active":"2023-10-06T05:18:10.121740+0000","last_became_peered":"2023-10-06T05:18:10.121740+0000","last_unstale":"2023-10-06T05:18:10.126571+0000","last_undegraded":"2023-10-06T05:18:10.126571+0000","last_fullsized":"2023-10-06T05:18:10.126571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:55.632117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120173+0000","last_change":"2023-10-06T05:18:09.110282+0000","last_active":"2023-10-06T05:18:10.120173+0000","last_peered":"2023-10-06T05:18:10.120173+0000","last_clean":"2023-10-06T05:18:10.120173+0000","last_became_active":"2023-10-06T05:18:09.110025+0000","last_became_peered":"2023-10-06T05:18:09.110025+0000","last_unstale":"2023-10-06T05:18:10.120173+0000","last_undegraded":"2023-10-06T05:18:10.120173+0000","last_fullsized":"2023-10-06T05:18:10.120173+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:34.664326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099751+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:18:10.099751+0000","last_peered":"2023-10-06T05:18:10.099751+0000","last_clean":"2023-10-06T05:18:10.099751+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:18:10.099751+0000","last_undegraded":"2023-10-06T05:18:10.099751+0000","last_fullsized":"2023-10-06T05:18:10.099751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127985+0000","last_change":"2023-10-06T05:18:10.123435+0000","last_active":"2023-10-06T05:18:10.127985+0000","last_peered":"2023-10-06T05:18:10.127985+0000","last_clean":"2023-10-06T05:18:10.127985+0000","last_became_active":"2023-10-06T05:18:10.123102+0000","last_became_peered":"2023-10-06T05:18:10.123102+0000","last_unstale":"2023-10-06T05:18:10.127985+0000","last_undegraded":"2023-10-06T05:18:10.127985+0000","last_fullsized":"2023-10-06T05:18:10.127985+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:16.218937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099653+0000","last_change":"2023-10-06T05:18:09.111951+0000","last_active":"2023-10-06T05:18:10.099653+0000","last_peered":"2023-10-06T05:18:10.099653+0000","last_clean":"2023-10-06T05:18:10.099653+0000","last_became_active":"2023-10-06T05:18:09.111669+0000","last_became_peered":"2023-10-06T05:18:09.111669+0000","last_unstale":"2023-10-06T05:18:10.099653+0000","last_undegraded":"2023-10-06T05:18:10.099653+0000","last_fullsized":"2023-10-06T05:18:10.099653+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:00.768714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.102484+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:18:10.102484+0000","last_peered":"2023-10-06T05:18:10.102484+0000","last_clean":"2023-10-06T05:18:10.102484+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:18:10.102484+0000","last_undegraded":"2023-10-06T05:18:10.102484+0000","last_fullsized":"2023-10-06T05:18:10.102484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126729+0000","last_change":"2023-10-06T05:18:10.123253+0000","last_active":"2023-10-06T05:18:10.126729+0000","last_peered":"2023-10-06T05:18:10.126729+0000","last_clean":"2023-10-06T05:18:10.126729+0000","last_became_active":"2023-10-06T05:18:10.123028+0000","last_became_peered":"2023-10-06T05:18:10.123028+0000","last_unstale":"2023-10-06T05:18:10.126729+0000","last_undegraded":"2023-10-06T05:18:10.126729+0000","last_fullsized":"2023-10-06T05:18:10.126729+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:43.275002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120907+0000","last_change":"2023-10-06T05:18:09.112557+0000","last_active":"2023-10-06T05:18:10.120907+0000","last_peered":"2023-10-06T05:18:10.120907+0000","last_clean":"2023-10-06T05:18:10.120907+0000","last_became_active":"2023-10-06T05:18:09.112298+0000","last_became_peered":"2023-10-06T05:18:09.112298+0000","last_unstale":"2023-10-06T05:18:10.120907+0000","last_undegraded":"2023-10-06T05:18:10.120907+0000","last_fullsized":"2023-10-06T05:18:10.120907+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:10.402833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099636+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:18:10.099636+0000","last_peered":"2023-10-06T05:18:10.099636+0000","last_clean":"2023-10-06T05:18:10.099636+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:18:10.099636+0000","last_undegraded":"2023-10-06T05:18:10.099636+0000","last_fullsized":"2023-10-06T05:18:10.099636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.096083+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:18:10.096083+0000","last_peered":"2023-10-06T05:18:10.096083+0000","last_clean":"2023-10-06T05:18:10.096083+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:18:10.096083+0000","last_undegraded":"2023-10-06T05:18:10.096083+0000","last_fullsized":"2023-10-06T05:18:10.096083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132550+0000","last_change":"2023-10-06T05:18:10.127170+0000","last_active":"2023-10-06T05:18:10.132550+0000","last_peered":"2023-10-06T05:18:10.132550+0000","last_clean":"2023-10-06T05:18:10.132550+0000","last_became_active":"2023-10-06T05:18:10.126930+0000","last_became_peered":"2023-10-06T05:18:10.126930+0000","last_unstale":"2023-10-06T05:18:10.132550+0000","last_undegraded":"2023-10-06T05:18:10.132550+0000","last_fullsized":"2023-10-06T05:18:10.132550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:06.790519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.106894+0000","last_change":"2023-10-06T05:18:09.115922+0000","last_active":"2023-10-06T05:18:10.106894+0000","last_peered":"2023-10-06T05:18:10.106894+0000","last_clean":"2023-10-06T05:18:10.106894+0000","last_became_active":"2023-10-06T05:18:09.115611+0000","last_became_peered":"2023-10-06T05:18:09.115611+0000","last_unstale":"2023-10-06T05:18:10.106894+0000","last_undegraded":"2023-10-06T05:18:10.106894+0000","last_fullsized":"2023-10-06T05:18:10.106894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:45.638130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.096181+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:18:10.096181+0000","last_peered":"2023-10-06T05:18:10.096181+0000","last_clean":"2023-10-06T05:18:10.096181+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:18:10.096181+0000","last_undegraded":"2023-10-06T05:18:10.096181+0000","last_fullsized":"2023-10-06T05:18:10.096181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127021+0000","last_change":"2023-10-06T05:18:10.123545+0000","last_active":"2023-10-06T05:18:10.127021+0000","last_peered":"2023-10-06T05:18:10.127021+0000","last_clean":"2023-10-06T05:18:10.127021+0000","last_became_active":"2023-10-06T05:18:10.123258+0000","last_became_peered":"2023-10-06T05:18:10.123258+0000","last_unstale":"2023-10-06T05:18:10.127021+0000","last_undegraded":"2023-10-06T05:18:10.127021+0000","last_fullsized":"2023-10-06T05:18:10.127021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:13.192505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126028+0000","last_change":"2023-10-06T05:18:09.113206+0000","last_active":"2023-10-06T05:18:10.126028+0000","last_peered":"2023-10-06T05:18:10.126028+0000","last_clean":"2023-10-06T05:18:10.126028+0000","last_became_active":"2023-10-06T05:18:09.112942+0000","last_became_peered":"2023-10-06T05:18:09.112942+0000","last_unstale":"2023-10-06T05:18:10.126028+0000","last_undegraded":"2023-10-06T05:18:10.126028+0000","last_fullsized":"2023-10-06T05:18:10.126028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:10.358875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.098186+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:18:10.098186+0000","last_peered":"2023-10-06T05:18:10.098186+0000","last_clean":"2023-10-06T05:18:10.098186+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:18:10.098186+0000","last_undegraded":"2023-10-06T05:18:10.098186+0000","last_fullsized":"2023-10-06T05:18:10.098186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120886+0000","last_change":"2023-10-06T05:18:10.114598+0000","last_active":"2023-10-06T05:18:10.120886+0000","last_peered":"2023-10-06T05:18:10.120886+0000","last_clean":"2023-10-06T05:18:10.120886+0000","last_became_active":"2023-10-06T05:18:10.114362+0000","last_became_peered":"2023-10-06T05:18:10.114362+0000","last_unstale":"2023-10-06T05:18:10.120886+0000","last_undegraded":"2023-10-06T05:18:10.120886+0000","last_fullsized":"2023-10-06T05:18:10.120886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:44.945540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.122213+0000","last_change":"2023-10-06T05:18:09.117430+0000","last_active":"2023-10-06T05:18:10.122213+0000","last_peered":"2023-10-06T05:18:10.122213+0000","last_clean":"2023-10-06T05:18:10.122213+0000","last_became_active":"2023-10-06T05:18:09.117132+0000","last_became_peered":"2023-10-06T05:18:09.117132+0000","last_unstale":"2023-10-06T05:18:10.122213+0000","last_undegraded":"2023-10-06T05:18:10.122213+0000","last_fullsized":"2023-10-06T05:18:10.122213+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:05.694981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.104712+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:18:10.104712+0000","last_peered":"2023-10-06T05:18:10.104712+0000","last_clean":"2023-10-06T05:18:10.104712+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:18:10.104712+0000","last_undegraded":"2023-10-06T05:18:10.104712+0000","last_fullsized":"2023-10-06T05:18:10.104712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120083+0000","last_change":"2023-10-06T05:18:10.115551+0000","last_active":"2023-10-06T05:18:10.120083+0000","last_peered":"2023-10-06T05:18:10.120083+0000","last_clean":"2023-10-06T05:18:10.120083+0000","last_became_active":"2023-10-06T05:18:10.115247+0000","last_became_peered":"2023-10-06T05:18:10.115247+0000","last_unstale":"2023-10-06T05:18:10.120083+0000","last_undegraded":"2023-10-06T05:18:10.120083+0000","last_fullsized":"2023-10-06T05:18:10.120083+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:44.945540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128694+0000","last_change":"2023-10-06T05:18:09.109718+0000","last_active":"2023-10-06T05:18:10.128694+0000","last_peered":"2023-10-06T05:18:10.128694+0000","last_clean":"2023-10-06T05:18:10.128694+0000","last_became_active":"2023-10-06T05:18:09.109407+0000","last_became_peered":"2023-10-06T05:18:09.109407+0000","last_unstale":"2023-10-06T05:18:10.128694+0000","last_undegraded":"2023-10-06T05:18:10.128694+0000","last_fullsized":"2023-10-06T05:18:10.128694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:43.475514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.117252+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:18:10.117252+0000","last_peered":"2023-10-06T05:18:10.117252+0000","last_clean":"2023-10-06T05:18:10.117252+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:18:10.117252+0000","last_undegraded":"2023-10-06T05:18:10.117252+0000","last_fullsized":"2023-10-06T05:18:10.117252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073208,"statfs":{"total":96636764160,"available":96330964992,"internally_reserved":0,"allocated":1376256,"data_stored":956829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.625},{"interface":"front","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79500000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92300000000000004,"5min":1.476,"15min":1.476},"last":0.504},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.42699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71599999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86599999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63400000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.67600000000000005},"min":{"1min":0.48599999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.96799999999999997,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.504,"15min":0.5},"min":{"1min":0.39400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.35299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.87,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":359004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.69399999999999995,"15min":0.76200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.871,"15min":1.0089999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80200000000000005,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.72399999999999998,"15min":0.79000000000000004},"min":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77300000000000002,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.92000000000000004,"15min":1.0720000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80500000000000005,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.496},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75900000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.337,"15min":1.6619999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78900000000000003,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.89000000000000001,"15min":1.012},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82199999999999995,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.79700000000000004,"15min":0.93400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.83199999999999996,"15min":0.96299999999999997},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65800000000000003,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.83299999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.86699999999999999,"15min":0.996},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70399999999999996,"5min":9.718,"15min":9.718},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.89100000000000001,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75600000000000001,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.53200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":782336,"data_stored":361036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38700000000000001,"15min":0.39300000000000002},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48199999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.315},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.41799999999999998,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.442},"min":{"1min":0.29599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52100000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.46400000000000002,"15min":0.48899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.52000000000000002,"5min":1.673,"15min":1.673},"last":0.40699999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.55400000000000005,"15min":0.59799999999999998},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.56200000000000006,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45300000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59199999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.79900000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82299999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84199999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.93899999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85299999999999998,"5min":1.026,"15min":1.026},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.499,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298624,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073216,"statfs":{"total":96636764160,"available":96330973184,"internally_reserved":0,"allocated":1368064,"data_stored":956241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.82799999999999996,"5min":1.038,"15min":1.038},"last":0.309},{"interface":"front","average":{"1min":0.434,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.56100000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":1.0609999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.36699999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.66400000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":3.855},{"interface":"front","average":{"1min":0.439,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60999999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.33300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":7.5880000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":1.016,"15min":1.016},"last":7.7670000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":2.4039999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80700000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":2.0870000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83699999999999997,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":2.4940000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.48399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78700000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":5.3600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.626},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.748,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88200000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":0.91900000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.52400000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83499999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.2150000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":367409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58699999999999997,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.45600000000000002,"15min":0.47699999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55700000000000005,"5min":1.899,"15min":1.899},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57199999999999995,"5min":2.097,"15min":2.097},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51400000000000001,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71099999999999997,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44,"15min":0.45000000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.38},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.50700000000000001,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":2.121,"15min":2.121},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91800000000000004,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91100000000000003,"5min":1.643,"15min":1.643},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88600000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93000000000000005,"5min":1.851,"15min":1.851},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.504,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":1.571,"15min":1.571},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81100000000000005,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.51100000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.90500000000000003,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89400000000000002,"5min":1.613,"15min":1.613},"last":0.54300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":364632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.50800000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.5},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.38800000000000001},"min":{"1min":0.29999999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.625}]},{"osd":1,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.38400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.52600000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57499999999999996,"5min":0.755,"15min":0.755},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63900000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79100000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82899999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88100000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.64300000000000002},"min":{"1min":0.46100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.75700000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.872,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.92000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":365943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70399999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69799999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.82599999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.443,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78900000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77400000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60499999999999998,"15min":0.625},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71099999999999997,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72799999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.69099999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48199999999999998,"15min":0.46999999999999997},"min":{"1min":0.375,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68500000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.35599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76000000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.51400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":359002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.504},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72299999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.625}]},{"osd":2,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.48299999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.71199999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.46300000000000002},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64100000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.46800000000000003},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.876,"15min":0.876},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.40899999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79400000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.755,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:14.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.456+0000 7f6a2b4e8700 1 Processor -- start 2023-10-06T05:18:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.456+0000 7f6a2b4e8700 1 -- start start 2023-10-06T05:18:14.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.457+0000 7f6a2b4e8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 0x7f6a2415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.457+0000 7f6a2b4e8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a241539f0 0x7f6a2415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.457+0000 7f6a2b4e8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a240ce690 con 0x7f6a240cc8b0 2023-10-06T05:18:14.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.457+0000 7f6a2b4e8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2415a7a0 con 0x7f6a241539f0 2023-10-06T05:18:14.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.457+0000 7f6a2b4e8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a240ce3f0 con 0x7f6a24153620 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a29a85700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6a240cc8b0 0x7f6a240ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37044/0 (socket says 172.21.15.45:37044) 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a29a85700 1 -- 172.21.15.45:0/3227518699 learned_addr learned my addr 172.21.15.45:0/3227518699 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a28a83700 1 --2- 172.21.15.45:0/3227518699 >> v2:172.21.15.144:3300/0 conn(0x7f6a241539f0 0x7f6a2415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a29284700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 0x7f6a2415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964527652 0 0) 0x7f6a240ce690 con 0x7f6a240cc8b0 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a10003570 con 0x7f6a240cc8b0 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2049929975 0 0) 0x7f6a10003570 con 0x7f6a240cc8b0 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 msgr2=0x7f6a2415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 0x7f6a2415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 >> v2:172.21.15.144:3300/0 conn(0x7f6a241539f0 msgr2=0x7f6a2415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 --2- 172.21.15.45:0/3227518699 >> v2:172.21.15.144:3300/0 conn(0x7f6a241539f0 0x7f6a2415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2415d510 con 0x7f6a240cc8b0 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f6a1c0017f0 con 0x7f6a240cc8b0 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a28a83700 1 --2- 172.21.15.45:0/3227518699 >> v2:172.21.15.144:3300/0 conn(0x7f6a241539f0 0x7f6a2415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 >> v1:172.21.15.45:6789/0 conn(0x7f6a240cc8b0 legacy=0x7f6a240ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a10003a60 0x7f6a10003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a10004810 0x7f6a10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 0x7f6a100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a240ce3f0 con 0x7f6a10003a60 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1000ba60 con 0x7f6a10004810 2023-10-06T05:18:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1000bba0 con 0x7f6a24153620 2023-10-06T05:18:14.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.458+0000 7f6a29284700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a10003a60 0x7f6a10003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a28a83700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a10004810 0x7f6a10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29a85700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 0x7f6a100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29284700 1 -- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 msgr2=0x7f6a100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29284700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 0x7f6a100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29284700 1 -- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a10004810 msgr2=0x7f6a10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29284700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a10004810 0x7f6a10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29284700 1 -- 172.21.15.45:0/3227518699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a10003570 con 0x7f6a10003a60 2023-10-06T05:18:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29a85700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a24153620 0x7f6a100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a29284700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a10003a60 0x7f6a10003e70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009c30 tx=0x7f6a200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc6e90c58c32223 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000e710 con 0x7f6a10003a60 2023-10-06T05:18:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2000e870 con 0x7f6a10003a60 2023-10-06T05:18:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.459+0000 7f6a1bfff700 1 -- 172.21.15.45:0/3227518699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000ec30 con 0x7f6a10003a60 2023-10-06T05:18:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.460+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/3227518699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a10003a60 msgr2=0x7f6a10003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.460+0000 7f6a2b4e8700 1 --2- 172.21.15.45:0/3227518699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a10003a60 0x7f6a10003e70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009c30 tx=0x7f6a200091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.460+0000 7f6a29a85700 1 -- 172.21.15.45:0/3227518699 reap_dead start 2023-10-06T05:18:14.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.460+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/3227518699 shutdown_connections 2023-10-06T05:18:14.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.460+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/3227518699 >> 172.21.15.45:0/3227518699 conn(0x7f6a2406ed50 msgr2=0x7f6a2406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.460+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/3227518699 shutdown_connections 2023-10-06T05:18:14.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.460+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/3227518699 wait complete. 2023-10-06T05:18:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.461+0000 7f6a2b4e8700 1 Processor -- start 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.462+0000 7f6a2b4e8700 1 -- start start 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a2b4e8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a10003a60 0x7f6a240b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a2b4e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2415bff0 0x7f6a240cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a2b4e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a240d03c0 0x7f6a240d0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a2b4e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2414e9a0 con 0x7f6a240d03c0 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a2b4e8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a240b3ea0 con 0x7f6a2415bff0 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a2b4e8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a240b4180 con 0x7f6a10003a60 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a240d03c0 0x7f6a240d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a240d03c0 0x7f6a240d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41664/0 (socket says 172.21.15.45:41664) 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 -- 172.21.15.45:0/293627428 learned_addr learned my addr 172.21.15.45:0/293627428 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a28a83700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2415bff0 0x7f6a240cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29284700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a10003a60 0x7f6a240b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 -- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a10003a60 msgr2=0x7f6a240b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a10003a60 0x7f6a240b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 -- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2415bff0 msgr2=0x7f6a240cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2415bff0 0x7f6a240cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 -- 172.21.15.45:0/293627428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a240b4460 con 0x7f6a240d03c0 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29284700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a10003a60 0x7f6a240b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.463+0000 7f6a29a85700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a240d03c0 0x7f6a240d0860 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6a24067840 tx=0x7f6a1c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.464+0000 7f6a197fa700 1 -- 172.21.15.45:0/293627428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00ebc0 con 0x7f6a240d03c0 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.464+0000 7f6a197fa700 1 -- 172.21.15.45:0/293627428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c00ed60 con 0x7f6a240d03c0 2023-10-06T05:18:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.464+0000 7f6a197fa700 1 -- 172.21.15.45:0/293627428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c013670 con 0x7f6a240d03c0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.464+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a240b4740 con 0x7f6a240d03c0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.464+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2414ee30 con 0x7f6a240d03c0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.465+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a24088a60 con 0x7f6a240d03c0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.465+0000 7f6a197fa700 1 -- 172.21.15.45:0/293627428 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a1c0139e0 con 0x7f6a240d03c0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.466+0000 7f6a197fa700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a08066f30 0x7f6a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.466+0000 7f6a197fa700 1 -- 172.21.15.45:0/293627428 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6a1c084e60 con 0x7f6a240d03c0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.466+0000 7f6a29284700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a08066f30 0x7f6a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.467+0000 7f6a29284700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a08066f30 0x7f6a080693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6a200091e0 tx=0x7f6a2001a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.470+0000 7f6a197fa700 1 -- 172.21.15.45:0/293627428 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a1c051d90 con 0x7f6a240d03c0 2023-10-06T05:18:14.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.667+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a24088a60 con 0x7f6a08066f30 2023-10-06T05:18:14.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.676+0000 7f6a197fa700 1 -- 172.21.15.45:0/293627428 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312049 (secure 0 0 0) 0x7f6a24088a60 con 0x7f6a08066f30 2023-10-06T05:18:14.682 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a08066f30 msgr2=0x7f6a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a08066f30 0x7f6a080693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6a200091e0 tx=0x7f6a2001a000 comp rx=0 tx=0).stop 2023-10-06T05:18:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a240d03c0 msgr2=0x7f6a240d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a240d03c0 0x7f6a240d0860 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6a24067840 tx=0x7f6a1c0017b0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 shutdown_connections 2023-10-06T05:18:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a08066f30 0x7f6a080693e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a240d03c0 0x7f6a240d0860 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a10003a60 0x7f6a240b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 --2- 172.21.15.45:0/293627428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2415bff0 0x7f6a240cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 >> 172.21.15.45:0/293627428 conn(0x7f6a2406ed50 msgr2=0x7f6a240afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 shutdown_connections 2023-10-06T05:18:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.680+0000 7f6a2b4e8700 1 -- 172.21.15.45:0/293627428 wait complete. 2023-10-06T05:18:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:18:14.695 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2023-10-06T05:18:14.494737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385536,"kb_used_data":7232,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589184,"statfs":{"total":773094113280,"available":770651324416,"internally_reserved":0,"allocated":7405568,"data_stored":4090096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001377"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123670+0000","last_change":"2023-10-06T05:18:10.119657+0000","last_active":"2023-10-06T05:18:10.123670+0000","last_peered":"2023-10-06T05:18:10.123670+0000","last_clean":"2023-10-06T05:18:10.123670+0000","last_became_active":"2023-10-06T05:18:10.119339+0000","last_became_peered":"2023-10-06T05:18:10.119339+0000","last_unstale":"2023-10-06T05:18:10.123670+0000","last_undegraded":"2023-10-06T05:18:10.123670+0000","last_fullsized":"2023-10-06T05:18:10.123670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:48.931102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127791+0000","last_change":"2023-10-06T05:18:10.119792+0000","last_active":"2023-10-06T05:18:10.127791+0000","last_peered":"2023-10-06T05:18:10.127791+0000","last_clean":"2023-10-06T05:18:10.127791+0000","last_became_active":"2023-10-06T05:18:10.119147+0000","last_became_peered":"2023-10-06T05:18:10.119147+0000","last_unstale":"2023-10-06T05:18:10.127791+0000","last_undegraded":"2023-10-06T05:18:10.127791+0000","last_fullsized":"2023-10-06T05:18:10.127791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:56.381468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132678+0000","last_change":"2023-10-06T05:18:10.127793+0000","last_active":"2023-10-06T05:18:10.132678+0000","last_peered":"2023-10-06T05:18:10.132678+0000","last_clean":"2023-10-06T05:18:10.132678+0000","last_became_active":"2023-10-06T05:18:10.123600+0000","last_became_peered":"2023-10-06T05:18:10.123600+0000","last_unstale":"2023-10-06T05:18:10.132678+0000","last_undegraded":"2023-10-06T05:18:10.132678+0000","last_fullsized":"2023-10-06T05:18:10.132678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:02.940820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.129364+0000","last_change":"2023-10-06T05:18:10.126915+0000","last_active":"2023-10-06T05:18:10.129364+0000","last_peered":"2023-10-06T05:18:10.129364+0000","last_clean":"2023-10-06T05:18:10.129364+0000","last_became_active":"2023-10-06T05:18:10.126352+0000","last_became_peered":"2023-10-06T05:18:10.126352+0000","last_unstale":"2023-10-06T05:18:10.129364+0000","last_undegraded":"2023-10-06T05:18:10.129364+0000","last_fullsized":"2023-10-06T05:18:10.129364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:28.330696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.119530+0000","last_change":"2023-10-06T05:18:10.116249+0000","last_active":"2023-10-06T05:18:10.119530+0000","last_peered":"2023-10-06T05:18:10.119530+0000","last_clean":"2023-10-06T05:18:10.119530+0000","last_became_active":"2023-10-06T05:18:10.112733+0000","last_became_peered":"2023-10-06T05:18:10.112733+0000","last_unstale":"2023-10-06T05:18:10.119530+0000","last_undegraded":"2023-10-06T05:18:10.119530+0000","last_fullsized":"2023-10-06T05:18:10.119530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:11.364552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127874+0000","last_change":"2023-10-06T05:18:10.121217+0000","last_active":"2023-10-06T05:18:10.127874+0000","last_peered":"2023-10-06T05:18:10.127874+0000","last_clean":"2023-10-06T05:18:10.127874+0000","last_became_active":"2023-10-06T05:18:10.120953+0000","last_became_peered":"2023-10-06T05:18:10.120953+0000","last_unstale":"2023-10-06T05:18:10.127874+0000","last_undegraded":"2023-10-06T05:18:10.127874+0000","last_fullsized":"2023-10-06T05:18:10.127874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:44.276220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123424+0000","last_change":"2023-10-06T05:18:10.116190+0000","last_active":"2023-10-06T05:18:10.123424+0000","last_peered":"2023-10-06T05:18:10.123424+0000","last_clean":"2023-10-06T05:18:10.123424+0000","last_became_active":"2023-10-06T05:18:10.115940+0000","last_became_peered":"2023-10-06T05:18:10.115940+0000","last_unstale":"2023-10-06T05:18:10.123424+0000","last_undegraded":"2023-10-06T05:18:10.123424+0000","last_fullsized":"2023-10-06T05:18:10.123424+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:28.900345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125752+0000","last_change":"2023-10-06T05:18:10.119970+0000","last_active":"2023-10-06T05:18:10.125752+0000","last_peered":"2023-10-06T05:18:10.125752+0000","last_clean":"2023-10-06T05:18:10.125752+0000","last_became_active":"2023-10-06T05:18:10.119705+0000","last_became_peered":"2023-10-06T05:18:10.119705+0000","last_unstale":"2023-10-06T05:18:10.125752+0000","last_undegraded":"2023-10-06T05:18:10.125752+0000","last_fullsized":"2023-10-06T05:18:10.125752+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:12.014329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132336+0000","last_change":"2023-10-06T05:18:10.130811+0000","last_active":"2023-10-06T05:18:10.132336+0000","last_peered":"2023-10-06T05:18:10.132336+0000","last_clean":"2023-10-06T05:18:10.132336+0000","last_became_active":"2023-10-06T05:18:10.130250+0000","last_became_peered":"2023-10-06T05:18:10.130250+0000","last_unstale":"2023-10-06T05:18:10.132336+0000","last_undegraded":"2023-10-06T05:18:10.132336+0000","last_fullsized":"2023-10-06T05:18:10.132336+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:44.481191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127916+0000","last_change":"2023-10-06T05:18:10.121996+0000","last_active":"2023-10-06T05:18:10.127916+0000","last_peered":"2023-10-06T05:18:10.127916+0000","last_clean":"2023-10-06T05:18:10.127916+0000","last_became_active":"2023-10-06T05:18:10.121712+0000","last_became_peered":"2023-10-06T05:18:10.121712+0000","last_unstale":"2023-10-06T05:18:10.127916+0000","last_undegraded":"2023-10-06T05:18:10.127916+0000","last_fullsized":"2023-10-06T05:18:10.127916+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:47.633801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.133448+0000","last_change":"2023-10-06T05:18:10.128141+0000","last_active":"2023-10-06T05:18:10.133448+0000","last_peered":"2023-10-06T05:18:10.133448+0000","last_clean":"2023-10-06T05:18:10.133448+0000","last_became_active":"2023-10-06T05:18:10.127695+0000","last_became_peered":"2023-10-06T05:18:10.127695+0000","last_unstale":"2023-10-06T05:18:10.133448+0000","last_undegraded":"2023-10-06T05:18:10.133448+0000","last_fullsized":"2023-10-06T05:18:10.133448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:59.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127044+0000","last_change":"2023-10-06T05:18:10.123556+0000","last_active":"2023-10-06T05:18:10.127044+0000","last_peered":"2023-10-06T05:18:10.127044+0000","last_clean":"2023-10-06T05:18:10.127044+0000","last_became_active":"2023-10-06T05:18:10.123275+0000","last_became_peered":"2023-10-06T05:18:10.123275+0000","last_unstale":"2023-10-06T05:18:10.127044+0000","last_undegraded":"2023-10-06T05:18:10.127044+0000","last_fullsized":"2023-10-06T05:18:10.127044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:54.310797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120232+0000","last_change":"2023-10-06T05:18:10.119339+0000","last_active":"2023-10-06T05:18:10.120232+0000","last_peered":"2023-10-06T05:18:10.120232+0000","last_clean":"2023-10-06T05:18:10.120232+0000","last_became_active":"2023-10-06T05:18:10.119069+0000","last_became_peered":"2023-10-06T05:18:10.119069+0000","last_unstale":"2023-10-06T05:18:10.120232+0000","last_undegraded":"2023-10-06T05:18:10.120232+0000","last_fullsized":"2023-10-06T05:18:10.120232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.119595+0000","last_change":"2023-10-06T05:18:10.113625+0000","last_active":"2023-10-06T05:18:10.119595+0000","last_peered":"2023-10-06T05:18:10.119595+0000","last_clean":"2023-10-06T05:18:10.119595+0000","last_became_active":"2023-10-06T05:18:10.113301+0000","last_became_peered":"2023-10-06T05:18:10.113301+0000","last_unstale":"2023-10-06T05:18:10.119595+0000","last_undegraded":"2023-10-06T05:18:10.119595+0000","last_fullsized":"2023-10-06T05:18:10.119595+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:49.968753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.122960+0000","last_change":"2023-10-06T05:18:10.120378+0000","last_active":"2023-10-06T05:18:10.122960+0000","last_peered":"2023-10-06T05:18:10.122960+0000","last_clean":"2023-10-06T05:18:10.122960+0000","last_became_active":"2023-10-06T05:18:10.120111+0000","last_became_peered":"2023-10-06T05:18:10.120111+0000","last_unstale":"2023-10-06T05:18:10.122960+0000","last_undegraded":"2023-10-06T05:18:10.122960+0000","last_fullsized":"2023-10-06T05:18:10.122960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:44.945540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128212+0000","last_change":"2023-10-06T05:18:10.122522+0000","last_active":"2023-10-06T05:18:10.128212+0000","last_peered":"2023-10-06T05:18:10.128212+0000","last_clean":"2023-10-06T05:18:10.128212+0000","last_became_active":"2023-10-06T05:18:10.121501+0000","last_became_peered":"2023-10-06T05:18:10.121501+0000","last_unstale":"2023-10-06T05:18:10.128212+0000","last_undegraded":"2023-10-06T05:18:10.128212+0000","last_fullsized":"2023-10-06T05:18:10.128212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:48.931102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132320+0000","last_change":"2023-10-06T05:18:10.130655+0000","last_active":"2023-10-06T05:18:10.132320+0000","last_peered":"2023-10-06T05:18:10.132320+0000","last_clean":"2023-10-06T05:18:10.132320+0000","last_became_active":"2023-10-06T05:18:10.129891+0000","last_became_peered":"2023-10-06T05:18:10.129891+0000","last_unstale":"2023-10-06T05:18:10.132320+0000","last_undegraded":"2023-10-06T05:18:10.132320+0000","last_fullsized":"2023-10-06T05:18:10.132320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:11.408510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126219+0000","last_change":"2023-10-06T05:18:10.122353+0000","last_active":"2023-10-06T05:18:10.126219+0000","last_peered":"2023-10-06T05:18:10.126219+0000","last_clean":"2023-10-06T05:18:10.126219+0000","last_became_active":"2023-10-06T05:18:10.122084+0000","last_became_peered":"2023-10-06T05:18:10.122084+0000","last_unstale":"2023-10-06T05:18:10.126219+0000","last_undegraded":"2023-10-06T05:18:10.126219+0000","last_fullsized":"2023-10-06T05:18:10.126219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:41.741749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125304+0000","last_change":"2023-10-06T05:18:10.118811+0000","last_active":"2023-10-06T05:18:10.125304+0000","last_peered":"2023-10-06T05:18:10.125304+0000","last_clean":"2023-10-06T05:18:10.125304+0000","last_became_active":"2023-10-06T05:18:10.118466+0000","last_became_peered":"2023-10-06T05:18:10.118466+0000","last_unstale":"2023-10-06T05:18:10.125304+0000","last_undegraded":"2023-10-06T05:18:10.125304+0000","last_fullsized":"2023-10-06T05:18:10.125304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:45.567578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114082+0000","last_change":"2023-10-06T05:18:10.112010+0000","last_active":"2023-10-06T05:18:10.114082+0000","last_peered":"2023-10-06T05:18:10.114082+0000","last_clean":"2023-10-06T05:18:10.114082+0000","last_became_active":"2023-10-06T05:18:10.111731+0000","last_became_peered":"2023-10-06T05:18:10.111731+0000","last_unstale":"2023-10-06T05:18:10.114082+0000","last_undegraded":"2023-10-06T05:18:10.114082+0000","last_fullsized":"2023-10-06T05:18:10.114082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:48.931102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120347+0000","last_change":"2023-10-06T05:18:10.117739+0000","last_active":"2023-10-06T05:18:10.120347+0000","last_peered":"2023-10-06T05:18:10.120347+0000","last_clean":"2023-10-06T05:18:10.120347+0000","last_became_active":"2023-10-06T05:18:10.116864+0000","last_became_peered":"2023-10-06T05:18:10.116864+0000","last_unstale":"2023-10-06T05:18:10.120347+0000","last_undegraded":"2023-10-06T05:18:10.120347+0000","last_fullsized":"2023-10-06T05:18:10.120347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:21.999081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125958+0000","last_change":"2023-10-06T05:18:10.121857+0000","last_active":"2023-10-06T05:18:10.125958+0000","last_peered":"2023-10-06T05:18:10.125958+0000","last_clean":"2023-10-06T05:18:10.125958+0000","last_became_active":"2023-10-06T05:18:10.121556+0000","last_became_peered":"2023-10-06T05:18:10.121556+0000","last_unstale":"2023-10-06T05:18:10.125958+0000","last_undegraded":"2023-10-06T05:18:10.125958+0000","last_fullsized":"2023-10-06T05:18:10.125958+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:30.837974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.107882+0000","last_change":"2023-10-06T05:18:10.100883+0000","last_active":"2023-10-06T05:18:10.107882+0000","last_peered":"2023-10-06T05:18:10.107882+0000","last_clean":"2023-10-06T05:18:10.107882+0000","last_became_active":"2023-10-06T05:18:10.100543+0000","last_became_peered":"2023-10-06T05:18:10.100543+0000","last_unstale":"2023-10-06T05:18:10.107882+0000","last_undegraded":"2023-10-06T05:18:10.107882+0000","last_fullsized":"2023-10-06T05:18:10.107882+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120865+0000","last_change":"2023-10-06T05:18:10.116249+0000","last_active":"2023-10-06T05:18:10.120865+0000","last_peered":"2023-10-06T05:18:10.120865+0000","last_clean":"2023-10-06T05:18:10.120865+0000","last_became_active":"2023-10-06T05:18:10.115564+0000","last_became_peered":"2023-10-06T05:18:10.115564+0000","last_unstale":"2023-10-06T05:18:10.120865+0000","last_undegraded":"2023-10-06T05:18:10.120865+0000","last_fullsized":"2023-10-06T05:18:10.120865+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:16.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.119867+0000","last_change":"2023-10-06T05:18:10.116687+0000","last_active":"2023-10-06T05:18:10.119867+0000","last_peered":"2023-10-06T05:18:10.119867+0000","last_clean":"2023-10-06T05:18:10.119867+0000","last_became_active":"2023-10-06T05:18:10.116236+0000","last_became_peered":"2023-10-06T05:18:10.116236+0000","last_unstale":"2023-10-06T05:18:10.119867+0000","last_undegraded":"2023-10-06T05:18:10.119867+0000","last_fullsized":"2023-10-06T05:18:10.119867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125268+0000","last_change":"2023-10-06T05:18:10.122192+0000","last_active":"2023-10-06T05:18:10.125268+0000","last_peered":"2023-10-06T05:18:10.125268+0000","last_clean":"2023-10-06T05:18:10.125268+0000","last_became_active":"2023-10-06T05:18:10.121893+0000","last_became_peered":"2023-10-06T05:18:10.121893+0000","last_unstale":"2023-10-06T05:18:10.125268+0000","last_undegraded":"2023-10-06T05:18:10.125268+0000","last_fullsized":"2023-10-06T05:18:10.125268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:16.272467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132656+0000","last_change":"2023-10-06T05:18:10.127021+0000","last_active":"2023-10-06T05:18:10.132656+0000","last_peered":"2023-10-06T05:18:10.132656+0000","last_clean":"2023-10-06T05:18:10.132656+0000","last_became_active":"2023-10-06T05:18:10.118560+0000","last_became_peered":"2023-10-06T05:18:10.118560+0000","last_unstale":"2023-10-06T05:18:10.132656+0000","last_undegraded":"2023-10-06T05:18:10.132656+0000","last_fullsized":"2023-10-06T05:18:10.132656+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:47.633801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125837+0000","last_change":"2023-10-06T05:18:10.122016+0000","last_active":"2023-10-06T05:18:10.125837+0000","last_peered":"2023-10-06T05:18:10.125837+0000","last_clean":"2023-10-06T05:18:10.125837+0000","last_became_active":"2023-10-06T05:18:10.121746+0000","last_became_peered":"2023-10-06T05:18:10.121746+0000","last_unstale":"2023-10-06T05:18:10.125837+0000","last_undegraded":"2023-10-06T05:18:10.125837+0000","last_fullsized":"2023-10-06T05:18:10.125837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:30.712493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126551+0000","last_change":"2023-10-06T05:18:10.118058+0000","last_active":"2023-10-06T05:18:10.126551+0000","last_peered":"2023-10-06T05:18:10.126551+0000","last_clean":"2023-10-06T05:18:10.126551+0000","last_became_active":"2023-10-06T05:18:10.117774+0000","last_became_peered":"2023-10-06T05:18:10.117774+0000","last_unstale":"2023-10-06T05:18:10.126551+0000","last_undegraded":"2023-10-06T05:18:10.126551+0000","last_fullsized":"2023-10-06T05:18:10.126551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:56.588067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125259+0000","last_change":"2023-10-06T05:18:10.122261+0000","last_active":"2023-10-06T05:18:10.125259+0000","last_peered":"2023-10-06T05:18:10.125259+0000","last_clean":"2023-10-06T05:18:10.125259+0000","last_became_active":"2023-10-06T05:18:10.121993+0000","last_became_peered":"2023-10-06T05:18:10.121993+0000","last_unstale":"2023-10-06T05:18:10.125259+0000","last_undegraded":"2023-10-06T05:18:10.125259+0000","last_fullsized":"2023-10-06T05:18:10.125259+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:40.836109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120526+0000","last_change":"2023-10-06T05:18:10.113305+0000","last_active":"2023-10-06T05:18:10.120526+0000","last_peered":"2023-10-06T05:18:10.120526+0000","last_clean":"2023-10-06T05:18:10.120526+0000","last_became_active":"2023-10-06T05:18:10.113112+0000","last_became_peered":"2023-10-06T05:18:10.113112+0000","last_unstale":"2023-10-06T05:18:10.120526+0000","last_undegraded":"2023-10-06T05:18:10.120526+0000","last_fullsized":"2023-10-06T05:18:10.120526+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:57.625054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130602+0000","last_change":"2023-10-06T05:18:10.124955+0000","last_active":"2023-10-06T05:18:10.130602+0000","last_peered":"2023-10-06T05:18:10.130602+0000","last_clean":"2023-10-06T05:18:10.130602+0000","last_became_active":"2023-10-06T05:18:10.124689+0000","last_became_peered":"2023-10-06T05:18:10.124689+0000","last_unstale":"2023-10-06T05:18:10.130602+0000","last_undegraded":"2023-10-06T05:18:10.130602+0000","last_fullsized":"2023-10-06T05:18:10.130602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:44.101041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130473+0000","last_change":"2023-10-06T05:18:10.123556+0000","last_active":"2023-10-06T05:18:10.130473+0000","last_peered":"2023-10-06T05:18:10.130473+0000","last_clean":"2023-10-06T05:18:10.130473+0000","last_became_active":"2023-10-06T05:18:10.123293+0000","last_became_peered":"2023-10-06T05:18:10.123293+0000","last_unstale":"2023-10-06T05:18:10.130473+0000","last_undegraded":"2023-10-06T05:18:10.130473+0000","last_fullsized":"2023-10-06T05:18:10.130473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:27.848967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"63'4","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.112989+0000","last_change":"2023-10-06T05:18:09.113465+0000","last_active":"2023-10-06T05:18:10.112989+0000","last_peered":"2023-10-06T05:18:10.112989+0000","last_clean":"2023-10-06T05:18:10.112989+0000","last_became_active":"2023-10-06T05:18:09.113182+0000","last_became_peered":"2023-10-06T05:18:09.113182+0000","last_unstale":"2023-10-06T05:18:10.112989+0000","last_undegraded":"2023-10-06T05:18:10.112989+0000","last_fullsized":"2023-10-06T05:18:10.112989+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:48.446547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132303+0000","last_change":"2023-10-06T05:18:10.130498+0000","last_active":"2023-10-06T05:18:10.132303+0000","last_peered":"2023-10-06T05:18:10.132303+0000","last_clean":"2023-10-06T05:18:10.132303+0000","last_became_active":"2023-10-06T05:18:10.129602+0000","last_became_peered":"2023-10-06T05:18:10.129602+0000","last_unstale":"2023-10-06T05:18:10.132303+0000","last_undegraded":"2023-10-06T05:18:10.132303+0000","last_fullsized":"2023-10-06T05:18:10.132303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:35.670004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121472+0000","last_change":"2023-10-06T05:18:09.120519+0000","last_active":"2023-10-06T05:18:10.121472+0000","last_peered":"2023-10-06T05:18:10.121472+0000","last_clean":"2023-10-06T05:18:10.121472+0000","last_became_active":"2023-10-06T05:18:09.120179+0000","last_became_peered":"2023-10-06T05:18:09.120179+0000","last_unstale":"2023-10-06T05:18:10.121472+0000","last_undegraded":"2023-10-06T05:18:10.121472+0000","last_fullsized":"2023-10-06T05:18:10.121472+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.857433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128037+0000","last_change":"2023-10-06T05:18:10.122916+0000","last_active":"2023-10-06T05:18:10.128037+0000","last_peered":"2023-10-06T05:18:10.128037+0000","last_clean":"2023-10-06T05:18:10.128037+0000","last_became_active":"2023-10-06T05:18:10.122417+0000","last_became_peered":"2023-10-06T05:18:10.122417+0000","last_unstale":"2023-10-06T05:18:10.128037+0000","last_undegraded":"2023-10-06T05:18:10.128037+0000","last_fullsized":"2023-10-06T05:18:10.128037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.434161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125020+0000","last_change":"2023-10-06T05:18:09.117418+0000","last_active":"2023-10-06T05:18:10.125020+0000","last_peered":"2023-10-06T05:18:10.125020+0000","last_clean":"2023-10-06T05:18:10.125020+0000","last_became_active":"2023-10-06T05:18:09.117138+0000","last_became_peered":"2023-10-06T05:18:09.117138+0000","last_unstale":"2023-10-06T05:18:10.125020+0000","last_undegraded":"2023-10-06T05:18:10.125020+0000","last_fullsized":"2023-10-06T05:18:10.125020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:55.375791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114201+0000","last_change":"2023-10-06T05:18:10.109803+0000","last_active":"2023-10-06T05:18:10.114201+0000","last_peered":"2023-10-06T05:18:10.114201+0000","last_clean":"2023-10-06T05:18:10.114201+0000","last_became_active":"2023-10-06T05:18:10.109498+0000","last_became_peered":"2023-10-06T05:18:10.109498+0000","last_unstale":"2023-10-06T05:18:10.114201+0000","last_undegraded":"2023-10-06T05:18:10.114201+0000","last_fullsized":"2023-10-06T05:18:10.114201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:57.625054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125249+0000","last_change":"2023-10-06T05:18:09.112546+0000","last_active":"2023-10-06T05:18:10.125249+0000","last_peered":"2023-10-06T05:18:10.125249+0000","last_clean":"2023-10-06T05:18:10.125249+0000","last_became_active":"2023-10-06T05:18:09.112283+0000","last_became_peered":"2023-10-06T05:18:09.112283+0000","last_unstale":"2023-10-06T05:18:10.125249+0000","last_undegraded":"2023-10-06T05:18:10.125249+0000","last_fullsized":"2023-10-06T05:18:10.125249+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:56.700869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123601+0000","last_change":"2023-10-06T05:18:10.118448+0000","last_active":"2023-10-06T05:18:10.123601+0000","last_peered":"2023-10-06T05:18:10.123601+0000","last_clean":"2023-10-06T05:18:10.123601+0000","last_became_active":"2023-10-06T05:18:10.118224+0000","last_became_peered":"2023-10-06T05:18:10.118224+0000","last_unstale":"2023-10-06T05:18:10.123601+0000","last_undegraded":"2023-10-06T05:18:10.123601+0000","last_fullsized":"2023-10-06T05:18:10.123601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:27.848967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121951+0000","last_change":"2023-10-06T05:18:09.118792+0000","last_active":"2023-10-06T05:18:10.121951+0000","last_peered":"2023-10-06T05:18:10.121951+0000","last_clean":"2023-10-06T05:18:10.121951+0000","last_became_active":"2023-10-06T05:18:09.118501+0000","last_became_peered":"2023-10-06T05:18:09.118501+0000","last_unstale":"2023-10-06T05:18:10.121951+0000","last_undegraded":"2023-10-06T05:18:10.121951+0000","last_fullsized":"2023-10-06T05:18:10.121951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:16.518119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120329+0000","last_change":"2023-10-06T05:18:10.117359+0000","last_active":"2023-10-06T05:18:10.120329+0000","last_peered":"2023-10-06T05:18:10.120329+0000","last_clean":"2023-10-06T05:18:10.120329+0000","last_became_active":"2023-10-06T05:18:10.116796+0000","last_became_peered":"2023-10-06T05:18:10.116796+0000","last_unstale":"2023-10-06T05:18:10.120329+0000","last_undegraded":"2023-10-06T05:18:10.120329+0000","last_fullsized":"2023-10-06T05:18:10.120329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.125778+0000","last_change":"2023-10-06T05:18:09.110260+0000","last_active":"2023-10-06T05:18:10.125778+0000","last_peered":"2023-10-06T05:18:10.125778+0000","last_clean":"2023-10-06T05:18:10.125778+0000","last_became_active":"2023-10-06T05:18:09.110007+0000","last_became_peered":"2023-10-06T05:18:09.110007+0000","last_unstale":"2023-10-06T05:18:10.125778+0000","last_undegraded":"2023-10-06T05:18:10.125778+0000","last_fullsized":"2023-10-06T05:18:10.125778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:03.361424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.131654+0000","last_change":"2023-10-06T05:18:10.121758+0000","last_active":"2023-10-06T05:18:10.131654+0000","last_peered":"2023-10-06T05:18:10.131654+0000","last_clean":"2023-10-06T05:18:10.131654+0000","last_became_active":"2023-10-06T05:18:10.121255+0000","last_became_peered":"2023-10-06T05:18:10.121255+0000","last_unstale":"2023-10-06T05:18:10.131654+0000","last_undegraded":"2023-10-06T05:18:10.131654+0000","last_fullsized":"2023-10-06T05:18:10.131654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:13.192505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.105852+0000","last_change":"2023-10-06T05:18:09.115267+0000","last_active":"2023-10-06T05:18:10.105852+0000","last_peered":"2023-10-06T05:18:10.105852+0000","last_clean":"2023-10-06T05:18:10.105852+0000","last_became_active":"2023-10-06T05:18:09.114960+0000","last_became_peered":"2023-10-06T05:18:09.114960+0000","last_unstale":"2023-10-06T05:18:10.105852+0000","last_undegraded":"2023-10-06T05:18:10.105852+0000","last_fullsized":"2023-10-06T05:18:10.105852+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:56:48.052516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114104+0000","last_change":"2023-10-06T05:18:10.112648+0000","last_active":"2023-10-06T05:18:10.114104+0000","last_peered":"2023-10-06T05:18:10.114104+0000","last_clean":"2023-10-06T05:18:10.114104+0000","last_became_active":"2023-10-06T05:18:10.112351+0000","last_became_peered":"2023-10-06T05:18:10.112351+0000","last_unstale":"2023-10-06T05:18:10.114104+0000","last_undegraded":"2023-10-06T05:18:10.114104+0000","last_fullsized":"2023-10-06T05:18:10.114104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:17.523797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.100482+0000","last_change":"2023-10-06T05:18:09.114329+0000","last_active":"2023-10-06T05:18:10.100482+0000","last_peered":"2023-10-06T05:18:10.100482+0000","last_clean":"2023-10-06T05:18:10.100482+0000","last_became_active":"2023-10-06T05:18:09.114007+0000","last_became_peered":"2023-10-06T05:18:09.114007+0000","last_unstale":"2023-10-06T05:18:10.100482+0000","last_undegraded":"2023-10-06T05:18:10.100482+0000","last_fullsized":"2023-10-06T05:18:10.100482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:08.609356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.112415+0000","last_change":"2023-10-06T05:18:10.109090+0000","last_active":"2023-10-06T05:18:10.112415+0000","last_peered":"2023-10-06T05:18:10.112415+0000","last_clean":"2023-10-06T05:18:10.112415+0000","last_became_active":"2023-10-06T05:18:10.108820+0000","last_became_peered":"2023-10-06T05:18:10.108820+0000","last_unstale":"2023-10-06T05:18:10.112415+0000","last_undegraded":"2023-10-06T05:18:10.112415+0000","last_fullsized":"2023-10-06T05:18:10.112415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:17.523797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.108837+0000","last_change":"2023-10-06T05:18:09.114453+0000","last_active":"2023-10-06T05:18:10.108837+0000","last_peered":"2023-10-06T05:18:10.108837+0000","last_clean":"2023-10-06T05:18:10.108837+0000","last_became_active":"2023-10-06T05:18:09.114187+0000","last_became_peered":"2023-10-06T05:18:09.114187+0000","last_unstale":"2023-10-06T05:18:10.108837+0000","last_undegraded":"2023-10-06T05:18:10.108837+0000","last_fullsized":"2023-10-06T05:18:10.108837+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:43.270543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130449+0000","last_change":"2023-10-06T05:18:10.123178+0000","last_active":"2023-10-06T05:18:10.130449+0000","last_peered":"2023-10-06T05:18:10.130449+0000","last_clean":"2023-10-06T05:18:10.130449+0000","last_became_active":"2023-10-06T05:18:10.122914+0000","last_became_peered":"2023-10-06T05:18:10.122914+0000","last_unstale":"2023-10-06T05:18:10.130449+0000","last_undegraded":"2023-10-06T05:18:10.130449+0000","last_fullsized":"2023-10-06T05:18:10.130449+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:17.523797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.115446+0000","last_change":"2023-10-06T05:18:09.114367+0000","last_active":"2023-10-06T05:18:10.115446+0000","last_peered":"2023-10-06T05:18:10.115446+0000","last_clean":"2023-10-06T05:18:10.115446+0000","last_became_active":"2023-10-06T05:18:09.114095+0000","last_became_peered":"2023-10-06T05:18:09.114095+0000","last_unstale":"2023-10-06T05:18:10.115446+0000","last_undegraded":"2023-10-06T05:18:10.115446+0000","last_fullsized":"2023-10-06T05:18:10.115446+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:48.963076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128016+0000","last_change":"2023-10-06T05:18:10.122407+0000","last_active":"2023-10-06T05:18:10.128016+0000","last_peered":"2023-10-06T05:18:10.128016+0000","last_clean":"2023-10-06T05:18:10.128016+0000","last_became_active":"2023-10-06T05:18:10.122114+0000","last_became_peered":"2023-10-06T05:18:10.122114+0000","last_unstale":"2023-10-06T05:18:10.128016+0000","last_undegraded":"2023-10-06T05:18:10.128016+0000","last_fullsized":"2023-10-06T05:18:10.128016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:06.700658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127930+0000","last_change":"2023-10-06T05:18:09.109140+0000","last_active":"2023-10-06T05:18:10.127930+0000","last_peered":"2023-10-06T05:18:10.127930+0000","last_clean":"2023-10-06T05:18:10.127930+0000","last_became_active":"2023-10-06T05:18:09.108726+0000","last_became_peered":"2023-10-06T05:18:09.108726+0000","last_unstale":"2023-10-06T05:18:10.127930+0000","last_undegraded":"2023-10-06T05:18:10.127930+0000","last_fullsized":"2023-10-06T05:18:10.127930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:58.944793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130014+0000","last_change":"2023-10-06T05:18:10.119334+0000","last_active":"2023-10-06T05:18:10.130014+0000","last_peered":"2023-10-06T05:18:10.130014+0000","last_clean":"2023-10-06T05:18:10.130014+0000","last_became_active":"2023-10-06T05:18:10.119050+0000","last_became_peered":"2023-10-06T05:18:10.119050+0000","last_unstale":"2023-10-06T05:18:10.130014+0000","last_undegraded":"2023-10-06T05:18:10.130014+0000","last_fullsized":"2023-10-06T05:18:10.130014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:28.330696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.133752+0000","last_change":"2023-10-06T05:18:09.113355+0000","last_active":"2023-10-06T05:18:10.133752+0000","last_peered":"2023-10-06T05:18:10.133752+0000","last_clean":"2023-10-06T05:18:10.133752+0000","last_became_active":"2023-10-06T05:18:09.113052+0000","last_became_peered":"2023-10-06T05:18:09.113052+0000","last_unstale":"2023-10-06T05:18:10.133752+0000","last_undegraded":"2023-10-06T05:18:10.133752+0000","last_fullsized":"2023-10-06T05:18:10.133752+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:48.020566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120867+0000","last_change":"2023-10-06T05:18:10.114204+0000","last_active":"2023-10-06T05:18:10.120867+0000","last_peered":"2023-10-06T05:18:10.120867+0000","last_clean":"2023-10-06T05:18:10.120867+0000","last_became_active":"2023-10-06T05:18:10.113888+0000","last_became_peered":"2023-10-06T05:18:10.113888+0000","last_unstale":"2023-10-06T05:18:10.120867+0000","last_undegraded":"2023-10-06T05:18:10.120867+0000","last_fullsized":"2023-10-06T05:18:10.120867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:21.999081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121439+0000","last_change":"2023-10-06T05:18:09.113183+0000","last_active":"2023-10-06T05:18:10.121439+0000","last_peered":"2023-10-06T05:18:10.121439+0000","last_clean":"2023-10-06T05:18:10.121439+0000","last_became_active":"2023-10-06T05:18:09.112945+0000","last_became_peered":"2023-10-06T05:18:09.112945+0000","last_unstale":"2023-10-06T05:18:10.121439+0000","last_undegraded":"2023-10-06T05:18:10.121439+0000","last_fullsized":"2023-10-06T05:18:10.121439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:14.036995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.115155+0000","last_change":"2023-10-06T05:18:10.111828+0000","last_active":"2023-10-06T05:18:10.115155+0000","last_peered":"2023-10-06T05:18:10.115155+0000","last_clean":"2023-10-06T05:18:10.115155+0000","last_became_active":"2023-10-06T05:18:10.111600+0000","last_became_peered":"2023-10-06T05:18:10.111600+0000","last_unstale":"2023-10-06T05:18:10.115155+0000","last_undegraded":"2023-10-06T05:18:10.115155+0000","last_fullsized":"2023-10-06T05:18:10.115155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120065+0000","last_change":"2023-10-06T05:18:09.120060+0000","last_active":"2023-10-06T05:18:10.120065+0000","last_peered":"2023-10-06T05:18:10.120065+0000","last_clean":"2023-10-06T05:18:10.120065+0000","last_became_active":"2023-10-06T05:18:09.119815+0000","last_became_peered":"2023-10-06T05:18:09.119815+0000","last_unstale":"2023-10-06T05:18:10.120065+0000","last_undegraded":"2023-10-06T05:18:10.120065+0000","last_fullsized":"2023-10-06T05:18:10.120065+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:48.020566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128297+0000","last_change":"2023-10-06T05:18:10.125863+0000","last_active":"2023-10-06T05:18:10.128297+0000","last_peered":"2023-10-06T05:18:10.128297+0000","last_clean":"2023-10-06T05:18:10.128297+0000","last_became_active":"2023-10-06T05:18:10.125549+0000","last_became_peered":"2023-10-06T05:18:10.125549+0000","last_unstale":"2023-10-06T05:18:10.128297+0000","last_undegraded":"2023-10-06T05:18:10.128297+0000","last_fullsized":"2023-10-06T05:18:10.128297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099518+0000","last_change":"2023-10-06T05:18:09.114978+0000","last_active":"2023-10-06T05:18:10.099518+0000","last_peered":"2023-10-06T05:18:10.099518+0000","last_clean":"2023-10-06T05:18:10.099518+0000","last_became_active":"2023-10-06T05:18:09.114722+0000","last_became_peered":"2023-10-06T05:18:09.114722+0000","last_unstale":"2023-10-06T05:18:10.099518+0000","last_undegraded":"2023-10-06T05:18:10.099518+0000","last_fullsized":"2023-10-06T05:18:10.099518+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:14.428483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132274+0000","last_change":"2023-10-06T05:18:10.131329+0000","last_active":"2023-10-06T05:18:10.132274+0000","last_peered":"2023-10-06T05:18:10.132274+0000","last_clean":"2023-10-06T05:18:10.132274+0000","last_became_active":"2023-10-06T05:18:10.131053+0000","last_became_peered":"2023-10-06T05:18:10.131053+0000","last_unstale":"2023-10-06T05:18:10.132274+0000","last_undegraded":"2023-10-06T05:18:10.132274+0000","last_fullsized":"2023-10-06T05:18:10.132274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:04.367101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123024+0000","last_change":"2023-10-06T05:18:09.113573+0000","last_active":"2023-10-06T05:18:10.123024+0000","last_peered":"2023-10-06T05:18:10.123024+0000","last_clean":"2023-10-06T05:18:10.123024+0000","last_became_active":"2023-10-06T05:18:09.113319+0000","last_became_peered":"2023-10-06T05:18:09.113319+0000","last_unstale":"2023-10-06T05:18:10.123024+0000","last_undegraded":"2023-10-06T05:18:10.123024+0000","last_fullsized":"2023-10-06T05:18:10.123024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:45.638130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120894+0000","last_change":"2023-10-06T05:18:10.119815+0000","last_active":"2023-10-06T05:18:10.120894+0000","last_peered":"2023-10-06T05:18:10.120894+0000","last_clean":"2023-10-06T05:18:10.120894+0000","last_became_active":"2023-10-06T05:18:10.119512+0000","last_became_peered":"2023-10-06T05:18:10.119512+0000","last_unstale":"2023-10-06T05:18:10.120894+0000","last_undegraded":"2023-10-06T05:18:10.120894+0000","last_fullsized":"2023-10-06T05:18:10.120894+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:21.999081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128529+0000","last_change":"2023-10-06T05:18:09.109678+0000","last_active":"2023-10-06T05:18:10.128529+0000","last_peered":"2023-10-06T05:18:10.128529+0000","last_clean":"2023-10-06T05:18:10.128529+0000","last_became_active":"2023-10-06T05:18:09.109397+0000","last_became_peered":"2023-10-06T05:18:09.109397+0000","last_unstale":"2023-10-06T05:18:10.128529+0000","last_undegraded":"2023-10-06T05:18:10.128529+0000","last_fullsized":"2023-10-06T05:18:10.128529+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:56.500917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.123386+0000","last_change":"2023-10-06T05:18:10.116625+0000","last_active":"2023-10-06T05:18:10.123386+0000","last_peered":"2023-10-06T05:18:10.123386+0000","last_clean":"2023-10-06T05:18:10.123386+0000","last_became_active":"2023-10-06T05:18:10.116421+0000","last_became_peered":"2023-10-06T05:18:10.116421+0000","last_unstale":"2023-10-06T05:18:10.123386+0000","last_undegraded":"2023-10-06T05:18:10.123386+0000","last_fullsized":"2023-10-06T05:18:10.123386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:59.950470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099854+0000","last_change":"2023-10-06T05:18:09.113303+0000","last_active":"2023-10-06T05:18:10.099854+0000","last_peered":"2023-10-06T05:18:10.099854+0000","last_clean":"2023-10-06T05:18:10.099854+0000","last_became_active":"2023-10-06T05:18:09.113046+0000","last_became_peered":"2023-10-06T05:18:09.113046+0000","last_unstale":"2023-10-06T05:18:10.099854+0000","last_undegraded":"2023-10-06T05:18:10.099854+0000","last_fullsized":"2023-10-06T05:18:10.099854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:43.095363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.121094+0000","last_change":"2023-10-06T05:18:10.115097+0000","last_active":"2023-10-06T05:18:10.121094+0000","last_peered":"2023-10-06T05:18:10.121094+0000","last_clean":"2023-10-06T05:18:10.121094+0000","last_became_active":"2023-10-06T05:18:10.114856+0000","last_became_peered":"2023-10-06T05:18:10.114856+0000","last_unstale":"2023-10-06T05:18:10.121094+0000","last_undegraded":"2023-10-06T05:18:10.121094+0000","last_fullsized":"2023-10-06T05:18:10.121094+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:28.900345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.101678+0000","last_change":"2023-10-06T05:18:09.115555+0000","last_active":"2023-10-06T05:18:10.101678+0000","last_peered":"2023-10-06T05:18:10.101678+0000","last_clean":"2023-10-06T05:18:10.101678+0000","last_became_active":"2023-10-06T05:18:09.115243+0000","last_became_peered":"2023-10-06T05:18:09.115243+0000","last_unstale":"2023-10-06T05:18:10.101678+0000","last_undegraded":"2023-10-06T05:18:10.101678+0000","last_fullsized":"2023-10-06T05:18:10.101678+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:34.059353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.111840+0000","last_change":"2023-10-06T05:18:10.106640+0000","last_active":"2023-10-06T05:18:10.111840+0000","last_peered":"2023-10-06T05:18:10.111840+0000","last_clean":"2023-10-06T05:18:10.111840+0000","last_became_active":"2023-10-06T05:18:10.106422+0000","last_became_peered":"2023-10-06T05:18:10.106422+0000","last_unstale":"2023-10-06T05:18:10.111840+0000","last_undegraded":"2023-10-06T05:18:10.111840+0000","last_fullsized":"2023-10-06T05:18:10.111840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:36.771399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.107391+0000","last_change":"2023-10-06T05:18:09.116724+0000","last_active":"2023-10-06T05:18:10.107391+0000","last_peered":"2023-10-06T05:18:10.107391+0000","last_clean":"2023-10-06T05:18:10.107391+0000","last_became_active":"2023-10-06T05:18:09.116436+0000","last_became_peered":"2023-10-06T05:18:09.116436+0000","last_unstale":"2023-10-06T05:18:10.107391+0000","last_undegraded":"2023-10-06T05:18:10.107391+0000","last_fullsized":"2023-10-06T05:18:10.107391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:01.335370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132578+0000","last_change":"2023-10-06T05:18:10.131377+0000","last_active":"2023-10-06T05:18:10.132578+0000","last_peered":"2023-10-06T05:18:10.132578+0000","last_clean":"2023-10-06T05:18:10.132578+0000","last_became_active":"2023-10-06T05:18:10.131121+0000","last_became_peered":"2023-10-06T05:18:10.131121+0000","last_unstale":"2023-10-06T05:18:10.132578+0000","last_undegraded":"2023-10-06T05:18:10.132578+0000","last_fullsized":"2023-10-06T05:18:10.132578+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:41.741749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.100526+0000","last_change":"2023-10-06T05:18:09.118753+0000","last_active":"2023-10-06T05:18:10.100526+0000","last_peered":"2023-10-06T05:18:10.100526+0000","last_clean":"2023-10-06T05:18:10.100526+0000","last_became_active":"2023-10-06T05:18:09.116166+0000","last_became_peered":"2023-10-06T05:18:09.116166+0000","last_unstale":"2023-10-06T05:18:10.100526+0000","last_undegraded":"2023-10-06T05:18:10.100526+0000","last_fullsized":"2023-10-06T05:18:10.100526+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:45.461675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127852+0000","last_change":"2023-10-06T05:18:10.119831+0000","last_active":"2023-10-06T05:18:10.127852+0000","last_peered":"2023-10-06T05:18:10.127852+0000","last_clean":"2023-10-06T05:18:10.127852+0000","last_became_active":"2023-10-06T05:18:10.118875+0000","last_became_peered":"2023-10-06T05:18:10.118875+0000","last_unstale":"2023-10-06T05:18:10.127852+0000","last_undegraded":"2023-10-06T05:18:10.127852+0000","last_fullsized":"2023-10-06T05:18:10.127852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:01.774391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.105629+0000","last_change":"2023-10-06T05:18:09.115262+0000","last_active":"2023-10-06T05:18:10.105629+0000","last_peered":"2023-10-06T05:18:10.105629+0000","last_clean":"2023-10-06T05:18:10.105629+0000","last_became_active":"2023-10-06T05:18:09.114932+0000","last_became_peered":"2023-10-06T05:18:09.114932+0000","last_unstale":"2023-10-06T05:18:10.105629+0000","last_undegraded":"2023-10-06T05:18:10.105629+0000","last_fullsized":"2023-10-06T05:18:10.105629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:34.664326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.129542+0000","last_change":"2023-10-06T05:18:10.128718+0000","last_active":"2023-10-06T05:18:10.129542+0000","last_peered":"2023-10-06T05:18:10.129542+0000","last_clean":"2023-10-06T05:18:10.129542+0000","last_became_active":"2023-10-06T05:18:10.128411+0000","last_became_peered":"2023-10-06T05:18:10.128411+0000","last_unstale":"2023-10-06T05:18:10.129542+0000","last_undegraded":"2023-10-06T05:18:10.129542+0000","last_fullsized":"2023-10-06T05:18:10.129542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:49.968753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.108936+0000","last_change":"2023-10-06T05:18:09.116301+0000","last_active":"2023-10-06T05:18:10.108936+0000","last_peered":"2023-10-06T05:18:10.108936+0000","last_clean":"2023-10-06T05:18:10.108936+0000","last_became_active":"2023-10-06T05:18:09.116057+0000","last_became_peered":"2023-10-06T05:18:09.116057+0000","last_unstale":"2023-10-06T05:18:10.108936+0000","last_undegraded":"2023-10-06T05:18:10.108936+0000","last_fullsized":"2023-10-06T05:18:10.108936+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.857433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130755+0000","last_change":"2023-10-06T05:18:10.127141+0000","last_active":"2023-10-06T05:18:10.130755+0000","last_peered":"2023-10-06T05:18:10.130755+0000","last_clean":"2023-10-06T05:18:10.130755+0000","last_became_active":"2023-10-06T05:18:10.126906+0000","last_became_peered":"2023-10-06T05:18:10.126906+0000","last_unstale":"2023-10-06T05:18:10.130755+0000","last_undegraded":"2023-10-06T05:18:10.130755+0000","last_fullsized":"2023-10-06T05:18:10.130755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:49.452224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.114838+0000","last_change":"2023-10-06T05:18:09.110714+0000","last_active":"2023-10-06T05:18:10.114838+0000","last_peered":"2023-10-06T05:18:10.114838+0000","last_clean":"2023-10-06T05:18:10.114838+0000","last_became_active":"2023-10-06T05:18:09.110431+0000","last_became_peered":"2023-10-06T05:18:09.110431+0000","last_unstale":"2023-10-06T05:18:10.114838+0000","last_undegraded":"2023-10-06T05:18:10.114838+0000","last_fullsized":"2023-10-06T05:18:10.114838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:14.036995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.130153+0000","last_change":"2023-10-06T05:18:10.121750+0000","last_active":"2023-10-06T05:18:10.130153+0000","last_peered":"2023-10-06T05:18:10.130153+0000","last_clean":"2023-10-06T05:18:10.130153+0000","last_became_active":"2023-10-06T05:18:10.121546+0000","last_became_peered":"2023-10-06T05:18:10.121546+0000","last_unstale":"2023-10-06T05:18:10.130153+0000","last_undegraded":"2023-10-06T05:18:10.130153+0000","last_fullsized":"2023-10-06T05:18:10.130153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:14:54.310797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.096605+0000","last_change":"2023-10-06T05:18:09.115158+0000","last_active":"2023-10-06T05:18:10.096605+0000","last_peered":"2023-10-06T05:18:10.096605+0000","last_clean":"2023-10-06T05:18:10.096605+0000","last_became_active":"2023-10-06T05:18:09.114899+0000","last_became_peered":"2023-10-06T05:18:09.114899+0000","last_unstale":"2023-10-06T05:18:10.096605+0000","last_undegraded":"2023-10-06T05:18:10.096605+0000","last_fullsized":"2023-10-06T05:18:10.096605+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:44.561901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.113001+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:18:10.113001+0000","last_peered":"2023-10-06T05:18:10.113001+0000","last_clean":"2023-10-06T05:18:10.113001+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:18:10.113001+0000","last_undegraded":"2023-10-06T05:18:10.113001+0000","last_fullsized":"2023-10-06T05:18:10.113001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126571+0000","last_change":"2023-10-06T05:18:10.122106+0000","last_active":"2023-10-06T05:18:10.126571+0000","last_peered":"2023-10-06T05:18:10.126571+0000","last_clean":"2023-10-06T05:18:10.126571+0000","last_became_active":"2023-10-06T05:18:10.121740+0000","last_became_peered":"2023-10-06T05:18:10.121740+0000","last_unstale":"2023-10-06T05:18:10.126571+0000","last_undegraded":"2023-10-06T05:18:10.126571+0000","last_fullsized":"2023-10-06T05:18:10.126571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:55.632117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120173+0000","last_change":"2023-10-06T05:18:09.110282+0000","last_active":"2023-10-06T05:18:10.120173+0000","last_peered":"2023-10-06T05:18:10.120173+0000","last_clean":"2023-10-06T05:18:10.120173+0000","last_became_active":"2023-10-06T05:18:09.110025+0000","last_became_peered":"2023-10-06T05:18:09.110025+0000","last_unstale":"2023-10-06T05:18:10.120173+0000","last_undegraded":"2023-10-06T05:18:10.120173+0000","last_fullsized":"2023-10-06T05:18:10.120173+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:34.664326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099751+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:18:10.099751+0000","last_peered":"2023-10-06T05:18:10.099751+0000","last_clean":"2023-10-06T05:18:10.099751+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:18:10.099751+0000","last_undegraded":"2023-10-06T05:18:10.099751+0000","last_fullsized":"2023-10-06T05:18:10.099751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127985+0000","last_change":"2023-10-06T05:18:10.123435+0000","last_active":"2023-10-06T05:18:10.127985+0000","last_peered":"2023-10-06T05:18:10.127985+0000","last_clean":"2023-10-06T05:18:10.127985+0000","last_became_active":"2023-10-06T05:18:10.123102+0000","last_became_peered":"2023-10-06T05:18:10.123102+0000","last_unstale":"2023-10-06T05:18:10.127985+0000","last_undegraded":"2023-10-06T05:18:10.127985+0000","last_fullsized":"2023-10-06T05:18:10.127985+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:16.218937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099653+0000","last_change":"2023-10-06T05:18:09.111951+0000","last_active":"2023-10-06T05:18:10.099653+0000","last_peered":"2023-10-06T05:18:10.099653+0000","last_clean":"2023-10-06T05:18:10.099653+0000","last_became_active":"2023-10-06T05:18:09.111669+0000","last_became_peered":"2023-10-06T05:18:09.111669+0000","last_unstale":"2023-10-06T05:18:10.099653+0000","last_undegraded":"2023-10-06T05:18:10.099653+0000","last_fullsized":"2023-10-06T05:18:10.099653+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:00.768714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.102484+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:18:10.102484+0000","last_peered":"2023-10-06T05:18:10.102484+0000","last_clean":"2023-10-06T05:18:10.102484+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:18:10.102484+0000","last_undegraded":"2023-10-06T05:18:10.102484+0000","last_fullsized":"2023-10-06T05:18:10.102484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126729+0000","last_change":"2023-10-06T05:18:10.123253+0000","last_active":"2023-10-06T05:18:10.126729+0000","last_peered":"2023-10-06T05:18:10.126729+0000","last_clean":"2023-10-06T05:18:10.126729+0000","last_became_active":"2023-10-06T05:18:10.123028+0000","last_became_peered":"2023-10-06T05:18:10.123028+0000","last_unstale":"2023-10-06T05:18:10.126729+0000","last_undegraded":"2023-10-06T05:18:10.126729+0000","last_fullsized":"2023-10-06T05:18:10.126729+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:43.275002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120907+0000","last_change":"2023-10-06T05:18:09.112557+0000","last_active":"2023-10-06T05:18:10.120907+0000","last_peered":"2023-10-06T05:18:10.120907+0000","last_clean":"2023-10-06T05:18:10.120907+0000","last_became_active":"2023-10-06T05:18:09.112298+0000","last_became_peered":"2023-10-06T05:18:09.112298+0000","last_unstale":"2023-10-06T05:18:10.120907+0000","last_undegraded":"2023-10-06T05:18:10.120907+0000","last_fullsized":"2023-10-06T05:18:10.120907+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:10.402833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.099636+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:18:10.099636+0000","last_peered":"2023-10-06T05:18:10.099636+0000","last_clean":"2023-10-06T05:18:10.099636+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:18:10.099636+0000","last_undegraded":"2023-10-06T05:18:10.099636+0000","last_fullsized":"2023-10-06T05:18:10.099636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.096083+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:18:10.096083+0000","last_peered":"2023-10-06T05:18:10.096083+0000","last_clean":"2023-10-06T05:18:10.096083+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:18:10.096083+0000","last_undegraded":"2023-10-06T05:18:10.096083+0000","last_fullsized":"2023-10-06T05:18:10.096083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.132550+0000","last_change":"2023-10-06T05:18:10.127170+0000","last_active":"2023-10-06T05:18:10.132550+0000","last_peered":"2023-10-06T05:18:10.132550+0000","last_clean":"2023-10-06T05:18:10.132550+0000","last_became_active":"2023-10-06T05:18:10.126930+0000","last_became_peered":"2023-10-06T05:18:10.126930+0000","last_unstale":"2023-10-06T05:18:10.132550+0000","last_undegraded":"2023-10-06T05:18:10.132550+0000","last_fullsized":"2023-10-06T05:18:10.132550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:06.790519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.106894+0000","last_change":"2023-10-06T05:18:09.115922+0000","last_active":"2023-10-06T05:18:10.106894+0000","last_peered":"2023-10-06T05:18:10.106894+0000","last_clean":"2023-10-06T05:18:10.106894+0000","last_became_active":"2023-10-06T05:18:09.115611+0000","last_became_peered":"2023-10-06T05:18:09.115611+0000","last_unstale":"2023-10-06T05:18:10.106894+0000","last_undegraded":"2023-10-06T05:18:10.106894+0000","last_fullsized":"2023-10-06T05:18:10.106894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:45.638130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.096181+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:18:10.096181+0000","last_peered":"2023-10-06T05:18:10.096181+0000","last_clean":"2023-10-06T05:18:10.096181+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:18:10.096181+0000","last_undegraded":"2023-10-06T05:18:10.096181+0000","last_fullsized":"2023-10-06T05:18:10.096181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.127021+0000","last_change":"2023-10-06T05:18:10.123545+0000","last_active":"2023-10-06T05:18:10.127021+0000","last_peered":"2023-10-06T05:18:10.127021+0000","last_clean":"2023-10-06T05:18:10.127021+0000","last_became_active":"2023-10-06T05:18:10.123258+0000","last_became_peered":"2023-10-06T05:18:10.123258+0000","last_unstale":"2023-10-06T05:18:10.127021+0000","last_undegraded":"2023-10-06T05:18:10.127021+0000","last_fullsized":"2023-10-06T05:18:10.127021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:13.192505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.126028+0000","last_change":"2023-10-06T05:18:09.113206+0000","last_active":"2023-10-06T05:18:10.126028+0000","last_peered":"2023-10-06T05:18:10.126028+0000","last_clean":"2023-10-06T05:18:10.126028+0000","last_became_active":"2023-10-06T05:18:09.112942+0000","last_became_peered":"2023-10-06T05:18:09.112942+0000","last_unstale":"2023-10-06T05:18:10.126028+0000","last_undegraded":"2023-10-06T05:18:10.126028+0000","last_fullsized":"2023-10-06T05:18:10.126028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:10.358875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.098186+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:18:10.098186+0000","last_peered":"2023-10-06T05:18:10.098186+0000","last_clean":"2023-10-06T05:18:10.098186+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:18:10.098186+0000","last_undegraded":"2023-10-06T05:18:10.098186+0000","last_fullsized":"2023-10-06T05:18:10.098186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120886+0000","last_change":"2023-10-06T05:18:10.114598+0000","last_active":"2023-10-06T05:18:10.120886+0000","last_peered":"2023-10-06T05:18:10.120886+0000","last_clean":"2023-10-06T05:18:10.120886+0000","last_became_active":"2023-10-06T05:18:10.114362+0000","last_became_peered":"2023-10-06T05:18:10.114362+0000","last_unstale":"2023-10-06T05:18:10.120886+0000","last_undegraded":"2023-10-06T05:18:10.120886+0000","last_fullsized":"2023-10-06T05:18:10.120886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:44.945540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.122213+0000","last_change":"2023-10-06T05:18:09.117430+0000","last_active":"2023-10-06T05:18:10.122213+0000","last_peered":"2023-10-06T05:18:10.122213+0000","last_clean":"2023-10-06T05:18:10.122213+0000","last_became_active":"2023-10-06T05:18:09.117132+0000","last_became_peered":"2023-10-06T05:18:09.117132+0000","last_unstale":"2023-10-06T05:18:10.122213+0000","last_undegraded":"2023-10-06T05:18:10.122213+0000","last_fullsized":"2023-10-06T05:18:10.122213+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:05.694981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.104712+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:18:10.104712+0000","last_peered":"2023-10-06T05:18:10.104712+0000","last_clean":"2023-10-06T05:18:10.104712+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:18:10.104712+0000","last_undegraded":"2023-10-06T05:18:10.104712+0000","last_fullsized":"2023-10-06T05:18:10.104712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.120083+0000","last_change":"2023-10-06T05:18:10.115551+0000","last_active":"2023-10-06T05:18:10.120083+0000","last_peered":"2023-10-06T05:18:10.120083+0000","last_clean":"2023-10-06T05:18:10.120083+0000","last_became_active":"2023-10-06T05:18:10.115247+0000","last_became_peered":"2023-10-06T05:18:10.115247+0000","last_unstale":"2023-10-06T05:18:10.120083+0000","last_undegraded":"2023-10-06T05:18:10.120083+0000","last_fullsized":"2023-10-06T05:18:10.120083+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:09.083094+0000","last_clean_scrub_stamp":"2023-10-06T05:18:09.083094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:44.945540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.128694+0000","last_change":"2023-10-06T05:18:09.109718+0000","last_active":"2023-10-06T05:18:10.128694+0000","last_peered":"2023-10-06T05:18:10.128694+0000","last_clean":"2023-10-06T05:18:10.128694+0000","last_became_active":"2023-10-06T05:18:09.109407+0000","last_became_peered":"2023-10-06T05:18:09.109407+0000","last_unstale":"2023-10-06T05:18:10.128694+0000","last_undegraded":"2023-10-06T05:18:10.128694+0000","last_fullsized":"2023-10-06T05:18:10.128694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:08.077417+0000","last_clean_scrub_stamp":"2023-10-06T05:18:08.077417+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:43.475514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T05:18:10.117252+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:18:10.117252+0000","last_peered":"2023-10-06T05:18:10.117252+0000","last_clean":"2023-10-06T05:18:10.117252+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:18:10.117252+0000","last_undegraded":"2023-10-06T05:18:10.117252+0000","last_fullsized":"2023-10-06T05:18:10.117252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073208,"statfs":{"total":96636764160,"available":96330964992,"internally_reserved":0,"allocated":1376256,"data_stored":956829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.625},{"interface":"front","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79500000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92300000000000004,"5min":1.476,"15min":1.476},"last":0.504},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.42699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71599999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86599999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63400000000000001,"15min":0.64300000000000002},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.67600000000000005},"min":{"1min":0.48599999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.96799999999999997,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.504,"15min":0.5},"min":{"1min":0.39400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.35299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.87,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":359004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.69399999999999995,"15min":0.76200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.871,"15min":1.0089999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80200000000000005,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.72399999999999998,"15min":0.79000000000000004},"min":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77300000000000002,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.92000000000000004,"15min":1.0720000000000001},"min":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80500000000000005,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.496},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75900000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.337,"15min":1.6619999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78900000000000003,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.89000000000000001,"15min":1.012},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82199999999999995,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.79700000000000004,"15min":0.93400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.83199999999999996,"15min":0.96299999999999997},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65800000000000003,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.83299999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69299999999999995,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.86699999999999999,"15min":0.996},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70399999999999996,"5min":9.718,"15min":9.718},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.89100000000000001,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75600000000000001,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.53200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":782336,"data_stored":361036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38700000000000001,"15min":0.39300000000000002},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48199999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.315},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.41799999999999998,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.442},"min":{"1min":0.29599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52100000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.46400000000000002,"15min":0.48899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.52000000000000002,"5min":1.673,"15min":1.673},"last":0.40699999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.55400000000000005,"15min":0.59799999999999998},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.56200000000000006,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45300000000000001,"15min":0.46600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59199999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.79900000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82299999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84199999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.48199999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.93899999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85299999999999998,"5min":1.026,"15min":1.026},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.499,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298624,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073216,"statfs":{"total":96636764160,"available":96330973184,"internally_reserved":0,"allocated":1368064,"data_stored":956241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.82799999999999996,"5min":1.038,"15min":1.038},"last":0.309},{"interface":"front","average":{"1min":0.434,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.56100000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":1.0609999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.36699999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.66400000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":3.855},{"interface":"front","average":{"1min":0.439,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60999999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.33300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":7.5880000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":1.016,"15min":1.016},"last":7.7670000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76500000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":2.4039999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80700000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":2.0870000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83699999999999997,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":2.4940000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.48399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78700000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":5.3600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.626},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.748,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88200000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":0.91900000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.52400000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83499999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.2150000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":367409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.58699999999999997,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.45600000000000002,"15min":0.47699999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55700000000000005,"5min":1.899,"15min":1.899},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.48399999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57199999999999995,"5min":2.097,"15min":2.097},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51400000000000001,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71099999999999997,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44,"15min":0.45000000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66200000000000003,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.38},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.50700000000000001,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":2.121,"15min":2.121},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91800000000000004,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91100000000000003,"5min":1.643,"15min":1.643},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88600000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93000000000000005,"5min":1.851,"15min":1.851},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.504,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":1.571,"15min":1.571},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81100000000000005,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.51100000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.90500000000000003,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89400000000000002,"5min":1.613,"15min":1.613},"last":0.54300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":364632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.50800000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.5},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.38800000000000001},"min":{"1min":0.29999999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.625}]},{"osd":1,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.38400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.52600000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57499999999999996,"5min":0.755,"15min":0.755},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63900000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83399999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79100000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82899999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88100000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.64300000000000002},"min":{"1min":0.46100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.75700000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.872,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.92000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":365943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70399999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.47299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69799999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.82599999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.443,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78900000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77400000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60499999999999998,"15min":0.625},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71099999999999997,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72799999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.69099999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48199999999999998,"15min":0.46999999999999997},"min":{"1min":0.375,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68500000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.35599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76000000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.51400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":359002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.504},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72299999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.625}]},{"osd":2,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77400000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.48299999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.71199999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.46300000000000002},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64100000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.46800000000000003},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.876,"15min":0.876},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.40899999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79400000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.755,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:14.702 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:18:14.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:18:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.876+0000 7fed7d372700 1 Processor -- start 2023-10-06T05:18:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.876+0000 7fed7d372700 1 -- start start 2023-10-06T05:18:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.876+0000 7fed7d372700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fed78159060 0x7fed7815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed7d372700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed78159430 0x7fed780c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed7d372700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed780c9c70 0x7fed7806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed7d372700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7815de20 con 0x7fed78159060 2023-10-06T05:18:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed7d372700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed780ca080 con 0x7fed78159430 2023-10-06T05:18:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed7d372700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed780ac6e0 con 0x7fed780c9c70 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fed78159060 0x7fed7815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fed78159060 0x7fed7815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41678/0 (socket says 172.21.15.45:41678) 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 -- 172.21.15.45:0/531910715 learned_addr learned my addr 172.21.15.45:0/531910715 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed777fe700 1 --2- 172.21.15.45:0/531910715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed780c9c70 0x7fed7806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed767fc700 1 --2- 172.21.15.45:0/531910715 >> v2:172.21.15.144:3300/0 conn(0x7fed78159430 0x7fed780c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 -- 172.21.15.45:0/531910715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed780c9c70 msgr2=0x7fed7806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 --2- 172.21.15.45:0/531910715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed780c9c70 0x7fed7806bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 -- 172.21.15.45:0/531910715 >> v2:172.21.15.144:3300/0 conn(0x7fed78159430 msgr2=0x7fed780c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 --2- 172.21.15.45:0/531910715 >> v2:172.21.15.144:3300/0 conn(0x7fed78159430 0x7fed780c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 -- 172.21.15.45:0/531910715 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed780ac9c0 con 0x7fed78159060 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed76ffd700 1 --2- 172.21.15.45:0/531910715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159060 0x7fed7815d8e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fed780aeff0 tx=0x7fed68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd7eb6331de5794 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed75ffb700 1 -- 172.21.15.45:0/531910715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800b860 con 0x7fed78159060 2023-10-06T05:18:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed75ffb700 1 -- 172.21.15.45:0/531910715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6800b9c0 con 0x7fed78159060 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.877+0000 7fed75ffb700 1 -- 172.21.15.45:0/531910715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800bd60 con 0x7fed78159060 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 -- 172.21.15.45:0/531910715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159060 msgr2=0x7fed7815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 --2- 172.21.15.45:0/531910715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159060 0x7fed7815d8e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fed780aeff0 tx=0x7fed68009250 comp rx=0 tx=0).stop 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 -- 172.21.15.45:0/531910715 shutdown_connections 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 --2- 172.21.15.45:0/531910715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed780c9c70 0x7fed7806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 --2- 172.21.15.45:0/531910715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159060 0x7fed7815d8e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 --2- 172.21.15.45:0/531910715 >> v2:172.21.15.144:3300/0 conn(0x7fed78159430 0x7fed780c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 -- 172.21.15.45:0/531910715 >> 172.21.15.45:0/531910715 conn(0x7fed780cd080 msgr2=0x7fed780c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 -- 172.21.15.45:0/531910715 shutdown_connections 2023-10-06T05:18:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.878+0000 7fed7d372700 1 -- 172.21.15.45:0/531910715 wait complete. 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.879+0000 7fed7d372700 1 Processor -- start 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.880+0000 7fed7d372700 1 -- start start 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.880+0000 7fed7d372700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed780c9c70 0x7fed780aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.880+0000 7fed7d372700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed78159060 0x7fed78157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.880+0000 7fed7d372700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159430 0x7fed7815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.880+0000 7fed7d372700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed780aa950 con 0x7fed78159430 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.880+0000 7fed7d372700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed780a9d40 con 0x7fed780c9c70 2023-10-06T05:18:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.880+0000 7fed7d372700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed780aa020 con 0x7fed78159060 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159430 0x7fed7815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159430 0x7fed7815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41692/0 (socket says 172.21.15.45:41692) 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 -- 172.21.15.45:0/1450313507 learned_addr learned my addr 172.21.15.45:0/1450313507 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed76ffd700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed780c9c70 0x7fed780aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed767fc700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed78159060 0x7fed78157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 -- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed78159060 msgr2=0x7fed78157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed78159060 0x7fed78157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 -- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed780c9c70 msgr2=0x7fed780aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed780c9c70 0x7fed780aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 -- 172.21.15.45:0/1450313507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed680094f0 con 0x7fed78159430 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed777fe700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159430 0x7fed7815a630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fed70009fa0 tx=0x7fed70009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed5effd700 1 -- 172.21.15.45:0/1450313507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000ed20 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed5effd700 1 -- 172.21.15.45:0/1450313507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7000ee80 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed780aa300 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed5effd700 1 -- 172.21.15.45:0/1450313507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70014670 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.881+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7814b5b0 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.882+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed780aab00 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.883+0000 7fed5effd700 1 -- 172.21.15.45:0/1450313507 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed700149e0 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.883+0000 7fed5effd700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed64066f30 0x7fed640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.883+0000 7fed5effd700 1 -- 172.21.15.45:0/1450313507 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fed70084fe0 con 0x7fed78159430 2023-10-06T05:18:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.883+0000 7fed76ffd700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed64066f30 0x7fed640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.884+0000 7fed76ffd700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed64066f30 0x7fed640693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed780aeff0 tx=0x7fed68017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:14.887+0000 7fed5effd700 1 -- 172.21.15.45:0/1450313507 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed780aab00 con 0x7fed78159430 2023-10-06T05:18:15.077 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:18:15.075+0000 7efe4f6f2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:18:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.171+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fed780c95e0 con 0x7fed78159430 2023-10-06T05:18:15.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.174+0000 7fed5effd700 1 -- 172.21.15.45:0/1450313507 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fed70051fb0 con 0x7fed78159430 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed64066f30 msgr2=0x7fed640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed64066f30 0x7fed640693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed780aeff0 tx=0x7fed68017000 comp rx=0 tx=0).stop 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159430 msgr2=0x7fed7815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159430 0x7fed7815a630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fed70009fa0 tx=0x7fed70009900 comp rx=0 tx=0).stop 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 shutdown_connections 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed64066f30 0x7fed640693e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed78159430 0x7fed7815a630 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed780c9c70 0x7fed780aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.176+0000 7fed7d372700 1 --2- 172.21.15.45:0/1450313507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed78159060 0x7fed78157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.177+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 >> 172.21.15.45:0/1450313507 conn(0x7fed780cd080 msgr2=0x7fed7814e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.177+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 shutdown_connections 2023-10-06T05:18:15.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.177+0000 7fed7d372700 1 -- 172.21.15.45:0/1450313507 wait complete. 2023-10-06T05:18:15.180 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-10-06T05:18:15.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.323+0000 7f606e92b700 1 Processor -- start 2023-10-06T05:18:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.323+0000 7f606e92b700 1 -- start start 2023-10-06T05:18:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606e92b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068159330 0x7f606806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606e92b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f60680cc390 0x7f60680cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606e92b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60680cb530 con 0x7f6068158f60 2023-10-06T05:18:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606e92b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60681597a0 con 0x7f60680cc390 2023-10-06T05:18:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606e92b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60680ccd50 con 0x7f6068159330 2023-10-06T05:18:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f60677fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068159330 0x7f606806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606cec8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f60680cc390 0x7f60680cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f6067fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6068158f60 0x7f60680cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:49800/0 (socket says 172.21.15.45:49800) 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f60677fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068159330 0x7f606806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:42102/0 (socket says 172.21.15.45:42102) 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606cec8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f60680cc390 0x7f60680cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50434/0 (socket says 172.21.15.45:50434) 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f6067fff700 1 -- 172.21.15.45:0/324941708 learned_addr learned my addr 172.21.15.45:0/324941708 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f60677fe700 1 -- 172.21.15.45:0/324941708 >> v2:172.21.15.144:3300/0 conn(0x7f60680cc390 msgr2=0x7f60680cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f60677fe700 1 --2- 172.21.15.45:0/324941708 >> v2:172.21.15.144:3300/0 conn(0x7f60680cc390 0x7f60680cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f60677fe700 1 -- 172.21.15.45:0/324941708 >> v1:172.21.15.144:6789/0 conn(0x7f6068158f60 legacy=0x7f60680cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f60677fe700 1 -- 172.21.15.45:0/324941708 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6068069bd0 con 0x7f6068159330 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.324+0000 7f606cec8700 1 --2- 172.21.15.45:0/324941708 >> v2:172.21.15.144:3300/0 conn(0x7f60680cc390 0x7f60680cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f60677fe700 1 --2- 172.21.15.45:0/324941708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068159330 0x7f606806a060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f605c0098e0 tx=0x7f605c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee21fa1cb8d2a823 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f6066ffd700 1 -- 172.21.15.45:0/324941708 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c00b640 con 0x7f6068159330 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f6066ffd700 1 -- 172.21.15.45:0/324941708 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c00b7a0 con 0x7f6068159330 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f6066ffd700 1 -- 172.21.15.45:0/324941708 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c00bb40 con 0x7f6068159330 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f606e92b700 1 -- 172.21.15.45:0/324941708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068159330 msgr2=0x7f606806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f606e92b700 1 --2- 172.21.15.45:0/324941708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068159330 0x7f606806a060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f605c0098e0 tx=0x7f605c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f606e92b700 1 -- 172.21.15.45:0/324941708 shutdown_connections 2023-10-06T05:18:15.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f606e92b700 1 --2- 172.21.15.45:0/324941708 >> v2:172.21.15.144:3300/0 conn(0x7f60680cc390 0x7f60680cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f606e92b700 1 --2- 172.21.15.45:0/324941708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068159330 0x7f606806a060 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.325+0000 7f606e92b700 1 -- 172.21.15.45:0/324941708 >> 172.21.15.45:0/324941708 conn(0x7f60680cdf50 msgr2=0x7f60680ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:15.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.326+0000 7f606e92b700 1 -- 172.21.15.45:0/324941708 shutdown_connections 2023-10-06T05:18:15.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.326+0000 7f606e92b700 1 -- 172.21.15.45:0/324941708 wait complete. 2023-10-06T05:18:15.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.327+0000 7f606e92b700 1 Processor -- start 2023-10-06T05:18:15.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.327+0000 7f606e92b700 1 -- start start 2023-10-06T05:18:15.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606e92b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60680cc390 0x7f6068149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:15.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606e92b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068158f60 0x7f6068147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:15.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606e92b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6068159330 0x7f606814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:15.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606e92b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6068148550 con 0x7f6068159330 2023-10-06T05:18:15.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606e92b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6068147940 con 0x7f60680cc390 2023-10-06T05:18:15.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606e92b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6068147c20 con 0x7f6068158f60 2023-10-06T05:18:15.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6068159330 0x7f606814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:15.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6068159330 0x7f606814c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41708/0 (socket says 172.21.15.45:41708) 2023-10-06T05:18:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 -- 172.21.15.45:0/455994725 learned_addr learned my addr 172.21.15.45:0/455994725 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f60677fe700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068158f60 0x7f6068147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f6067fff700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60680cc390 0x7f6068149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 -- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068158f60 msgr2=0x7f6068147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068158f60 0x7f6068147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 -- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60680cc390 msgr2=0x7f6068149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60680cc390 0x7f6068149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.328+0000 7f606cec8700 1 -- 172.21.15.45:0/455994725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6060009a20 con 0x7f6068159330 2023-10-06T05:18:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.329+0000 7f606cec8700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6068159330 0x7f606814c2a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6060009fc0 tx=0x7f606000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.329+0000 7f604ffff700 1 -- 172.21.15.45:0/455994725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000cbd0 con 0x7f6068159330 2023-10-06T05:18:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.329+0000 7f604ffff700 1 -- 172.21.15.45:0/455994725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606000f040 con 0x7f6068159330 2023-10-06T05:18:15.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.329+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c009420 con 0x7f6068159330 2023-10-06T05:18:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.329+0000 7f604ffff700 1 -- 172.21.15.45:0/455994725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060014480 con 0x7f6068159330 2023-10-06T05:18:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.329+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60681481e0 con 0x7f6068159330 2023-10-06T05:18:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.330+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6068148700 con 0x7f6068159330 2023-10-06T05:18:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.330+0000 7f604ffff700 1 -- 172.21.15.45:0/455994725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6060003020 con 0x7f6068159330 2023-10-06T05:18:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.331+0000 7f604ffff700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6054066f30 0x7f60540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:15.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.331+0000 7f604ffff700 1 -- 172.21.15.45:0/455994725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f60600850c0 con 0x7f6068159330 2023-10-06T05:18:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.331+0000 7f6067fff700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6054066f30 0x7f60540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:15.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.331+0000 7f6067fff700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6054066f30 0x7f60540693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6058007130 tx=0x7f6058007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:15.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.335+0000 7f604ffff700 1 -- 172.21.15.45:0/455994725 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6060052090 con 0x7f6068159330 2023-10-06T05:18:15.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.582+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6068068fc0 con 0x7f6068159330 2023-10-06T05:18:15.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.582+0000 7f604ffff700 1 -- 172.21.15.45:0/455994725 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f60600556b0 con 0x7f6068159330 2023-10-06T05:18:15.584 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:15.584 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":48,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:13.113636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6114},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6114":{"gid":6114,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/896514784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":896514784},{"type":"v1","addr":"172.21.15.45:6833","nonce":896514784}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6054066f30 msgr2=0x7f60540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6054066f30 0x7f60540693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6058007130 tx=0x7f6058007eb0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6068159330 msgr2=0x7f606814c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6068159330 0x7f606814c2a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6060009fc0 tx=0x7f606000b040 comp rx=0 tx=0).stop 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 shutdown_connections 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6054066f30 0x7f60540693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6068159330 0x7f606814c2a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60680cc390 0x7f6068149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 --2- 172.21.15.45:0/455994725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6068158f60 0x7f6068147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 >> 172.21.15.45:0/455994725 conn(0x7f60680cdf50 msgr2=0x7f60680ca110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 shutdown_connections 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:15.584+0000 7f606e92b700 1 -- 172.21.15.45:0/455994725 wait complete. 2023-10-06T05:18:15.587 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 48 2023-10-06T05:18:16.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:09.524580+0000', 'modified': '2023-10-06T05:18:13.113636+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6114}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6114': {'gid': 6114, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6833/896514784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 896514784}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 896514784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:18:16.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:18:16.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:18:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.143+0000 7ff6818ec700 1 Processor -- start 2023-10-06T05:18:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.143+0000 7ff6818ec700 1 -- start start 2023-10-06T05:18:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff6818ec700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0b0290 0x7ff67c0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff6818ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff67c0c33f0 0x7ff67c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff6818ec700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c159a60 con 0x7ff67c0b0660 2023-10-06T05:18:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff6818ec700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c068730 con 0x7ff67c0c33f0 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff6818ec700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0689d0 con 0x7ff67c0b0290 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff67c0c33f0 0x7ff67c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67a7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff67c0b0660 0x7ff67c0c2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37046/0 (socket says 172.21.15.45:37046) 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67a7fc700 1 -- 172.21.15.45:0/2766409661 learned_addr learned my addr 172.21.15.45:0/2766409661 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67affd700 1 --2- 172.21.15.45:0/2766409661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0b0290 0x7ff67c0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67b7fe700 1 -- 172.21.15.45:0/2766409661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0b0290 msgr2=0x7ff67c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/2766409661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0b0290 0x7ff67c0c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67b7fe700 1 -- 172.21.15.45:0/2766409661 >> v1:172.21.15.45:6789/0 conn(0x7ff67c0b0660 legacy=0x7ff67c0c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff67b7fe700 1 -- 172.21.15.45:0/2766409661 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c068cb0 con 0x7ff67c0c33f0 2023-10-06T05:18:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.144+0000 7ff679ffb700 1 -- 172.21.15.45:0/2766409661 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656766345 0 0) 0x7ff67c159a60 con 0x7ff67c0b0660 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/2766409661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0c33f0 0x7ff67c0cd350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff674009e60 tx=0x7ff674009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69cfaba9db1b882 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff679ffb700 1 -- 172.21.15.45:0/2766409661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67400e070 con 0x7ff67c0c33f0 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff679ffb700 1 -- 172.21.15.45:0/2766409661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67400bc30 con 0x7ff67c0c33f0 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff679ffb700 1 -- 172.21.15.45:0/2766409661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674012430 con 0x7ff67c0c33f0 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff6818ec700 1 -- 172.21.15.45:0/2766409661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0c33f0 msgr2=0x7ff67c0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff6818ec700 1 --2- 172.21.15.45:0/2766409661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0c33f0 0x7ff67c0cd350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff674009e60 tx=0x7ff674009780 comp rx=0 tx=0).stop 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff6818ec700 1 -- 172.21.15.45:0/2766409661 shutdown_connections 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff6818ec700 1 --2- 172.21.15.45:0/2766409661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0c33f0 0x7ff67c0cd350 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff6818ec700 1 --2- 172.21.15.45:0/2766409661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0b0290 0x7ff67c0c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff6818ec700 1 -- 172.21.15.45:0/2766409661 >> 172.21.15.45:0/2766409661 conn(0x7ff67c067e00 msgr2=0x7ff67c0cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.145+0000 7ff6818ec700 1 -- 172.21.15.45:0/2766409661 shutdown_connections 2023-10-06T05:18:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.146+0000 7ff6818ec700 1 -- 172.21.15.45:0/2766409661 wait complete. 2023-10-06T05:18:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.147+0000 7ff6818ec700 1 Processor -- start 2023-10-06T05:18:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.147+0000 7ff6818ec700 1 -- start start 2023-10-06T05:18:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff6818ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0b0290 0x7ff67c0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff6818ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff67c0b0660 0x7ff67c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff6818ec700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0c33f0 0x7ff67c0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff6818ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c9650 con 0x7ff67c0b0290 2023-10-06T05:18:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff6818ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c15ea10 con 0x7ff67c0b0660 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff6818ec700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c15e0e0 con 0x7ff67c0c33f0 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0b0290 0x7ff67c0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0b0290 0x7ff67c0cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41740/0 (socket says 172.21.15.45:41740) 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 -- 172.21.15.45:0/3580508991 learned_addr learned my addr 172.21.15.45:0/3580508991 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff67c0b0660 0x7ff67c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0c33f0 0x7ff67c0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 -- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0c33f0 msgr2=0x7ff67c0c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0c33f0 0x7ff67c0c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 -- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff67c0b0660 msgr2=0x7ff67c0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff67c0b0660 0x7ff67c0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 -- 172.21.15.45:0/3580508991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674009a20 con 0x7ff67c0b0290 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.148+0000 7ff67affd700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0b0290 0x7ff67c0cb0d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00d810 tx=0x7ff66c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.149+0000 7ff672ffd700 1 -- 172.21.15.45:0/3580508991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0125f0 con 0x7ff67c0b0290 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.149+0000 7ff672ffd700 1 -- 172.21.15.45:0/3580508991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c012750 con 0x7ff67c0b0290 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.149+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c15e3c0 con 0x7ff67c0b0290 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.149+0000 7ff672ffd700 1 -- 172.21.15.45:0/3580508991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c012b10 con 0x7ff67c0b0290 2023-10-06T05:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.149+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c157250 con 0x7ff67c0b0290 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.151+0000 7ff672ffd700 1 -- 172.21.15.45:0/3580508991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff66c01e460 con 0x7ff67c0b0290 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.151+0000 7ff672ffd700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.151+0000 7ff672ffd700 1 -- 172.21.15.45:0/3580508991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff66c084e60 con 0x7ff67c0b0290 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.151+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.152+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff65c066f30 0x7ff65c0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff67c1600c0 tx=0x7ff664009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.153+0000 7ff68ac00b80 1 Processor -- start 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.153+0000 7ff68ac00b80 1 -- start start 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7dd450 0x55713d7dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7c2f40 0x55713d7e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d6b0310 con 0x55713d7e64d0 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d6ed130 con 0x55713d7dd450 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55713d6eaaf0 con 0x55713d7c2f40 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7e6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41742/0 (socket says 172.21.15.45:41742) 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 -- 172.21.15.45:0/742460576 learned_addr learned my addr 172.21.15.45:0/742460576 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67affd700 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7dd450 0x55713d7dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7c2f40 0x55713d7e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 -- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7c2f40 msgr2=0x55713d7e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7c2f40 0x55713d7e5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 -- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7dd450 msgr2=0x55713d7dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7dd450 0x55713d7dd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.154+0000 7ff67b7fe700 1 -- 172.21.15.45:0/742460576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55713d4ec790 con 0x55713d7e64d0 2023-10-06T05:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.155+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7e6940 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff674013000 tx=0x7ff674019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bc4b0fead89754 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.155+0000 7ff65bfff700 1 -- 172.21.15.45:0/742460576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67400bda0 con 0x55713d7e64d0 2023-10-06T05:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.155+0000 7ff65bfff700 1 -- 172.21.15.45:0/742460576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740127c0 con 0x55713d7e64d0 2023-10-06T05:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.155+0000 7ff65bfff700 1 -- 172.21.15.45:0/742460576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674012b80 con 0x55713d7e64d0 2023-10-06T05:18:16.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.155+0000 7ff68ac00b80 1 -- 172.21.15.45:0/742460576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 msgr2=0x55713d7e6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.155+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7e6940 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff674013000 tx=0x7ff674019000 comp rx=0 tx=0).stop 2023-10-06T05:18:16.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 -- 172.21.15.45:0/742460576 shutdown_connections 2023-10-06T05:18:16.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7e6940 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7dd450 0x55713d7dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/742460576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7c2f40 0x55713d7e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 -- 172.21.15.45:0/742460576 >> 172.21.15.45:0/742460576 conn(0x55713d7e0880 msgr2=0x55713d7e0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 -- 172.21.15.45:0/742460576 shutdown_connections 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 -- 172.21.15.45:0/742460576 wait complete. 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 -- start start 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.156+0000 7ff68ac00b80 1 Processor -- start 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d7bdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d6e68e0 con 0x55713d7e64d0 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d692980 con 0x55713d7c2f40 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55713d692620 con 0x55713d7dd450 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff68ac00b80 10 client.0 _ll_register_callbacks cb 0x55713d4f5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.157+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7bb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41750/0 (socket says 172.21.15.45:41750) 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67a7fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d7bdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 -- 172.21.15.45:0/503325748 learned_addr learned my addr 172.21.15.45:0/503325748 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67affd700 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 -- 172.21.15.45:0/503325748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 msgr2=0x55713d7bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d7bdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 -- 172.21.15.45:0/503325748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 msgr2=0x55713d7b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 -- 172.21.15.45:0/503325748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c068cb0 con 0x55713d7e64d0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7bb670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff67c06f2f0 tx=0x7ff674020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129e416cdd3a0542 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff6597fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff6597fa700 1 -- 172.21.15.45:0/503325748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674032680 con 0x55713d7e64d0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff6597fa700 1 -- 172.21.15.45:0/503325748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff6740327e0 con 0x55713d7e64d0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55713d78ad60 con 0x55713d7e64d0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff6597fa700 1 -- 172.21.15.45:0/503325748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff674032bb0 con 0x55713d7e64d0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff68ac00b80 10 client.6273 fetch_fsmap learned FSMap version 48 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.158+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55713d5d90a0 con 0x55713d7e64d0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff6597fa700 1 -- 172.21.15.45:0/503325748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67403a020 con 0x55713d7e64d0 2023-10-06T05:18:16.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff6597fa700 1 -- 172.21.15.45:0/503325748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff67402e040 con 0x55713d7e64d0 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff68ac00b80 10 client.6273 fetch_fsmap finished waiting for FSMap version 48 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff68ac00b80 10 client.6273 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff68ac00b80 20 client.6273 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d7bc510 0x55713d7bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff68ac00b80 4 client.6273 mds_command: new command op to 6114 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 --> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55713d767b30 con 0x55713d7bc510 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff67affd700 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d7bc510 0x55713d7bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff6597fa700 10 client.6273 ms_handle_connect on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.159+0000 7ff67affd700 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d7bc510 0x55713d7bc8e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a294e27d2fac50 server_cookie=a4bac69a31a0ecad in_seq=0 out_seq=0 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff6597fa700 1 -- 172.21.15.45:0/503325748 <== mds.0 v2:172.21.15.45:6832/896514784 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55713d767b30 con 0x55713d7bc510 2023-10-06T05:18:16.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff6597fa700 10 client.6273 handle_command_reply: tid=0 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 client.6273 shutdown 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 msgr2=0x55713d7bb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7bb670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff67c06f2f0 tx=0x7ff674020040 comp rx=0 tx=0).stop 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 shutdown_connections 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d7bc510 0x55713d7bc8e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7bb670 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/503325748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d7bdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff6597fa700 0 client.6273 ms_handle_reset on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 >> 172.21.15.45:0/503325748 conn(0x55713d7e0880 msgr2=0x55713d792c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 shutdown_connections 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 1 -- 172.21.15.45:0/503325748 wait complete. 2023-10-06T05:18:16.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.160+0000 7ff68ac00b80 20 client.6273 trim_cache size 0 max 16384 2023-10-06T05:18:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.178+0000 7ff68ac00b80 1 Processor -- start 2023-10-06T05:18:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.178+0000 7ff68ac00b80 1 -- start start 2023-10-06T05:18:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7dfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d792da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d57e1e0 con 0x55713d7e64d0 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d7e1620 con 0x55713d7c2f40 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55713d7e1400 con 0x55713d7dd450 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41752/0 (socket says 172.21.15.45:41752) 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 -- 172.21.15.45:0/2499084528 learned_addr learned my addr 172.21.15.45:0/2499084528 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 -- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 msgr2=0x55713d792da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d792da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67affd700 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7dfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d792da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 -- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 msgr2=0x55713d7dfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7dfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 -- 172.21.15.45:0/2499084528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55713d6b53c0 con 0x55713d7e64d0 2023-10-06T05:18:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7b69e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff674032040 tx=0x7ff67402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e8e8aca4653824 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.179+0000 7ff670ff9700 1 -- 172.21.15.45:0/2499084528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740020f0 con 0x55713d7e64d0 2023-10-06T05:18:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff670ff9700 1 -- 172.21.15.45:0/2499084528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67403a070 con 0x55713d7e64d0 2023-10-06T05:18:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff670ff9700 1 -- 172.21.15.45:0/2499084528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674032bb0 con 0x55713d7e64d0 2023-10-06T05:18:16.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2499084528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 msgr2=0x55713d7b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7b69e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff674032040 tx=0x7ff67402f000 comp rx=0 tx=0).stop 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2499084528 shutdown_connections 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d7dfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2499084528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7dd450 0x55713d792da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2499084528 >> 172.21.15.45:0/2499084528 conn(0x55713d7e0880 msgr2=0x55713d7e5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2499084528 shutdown_connections 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.180+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2499084528 wait complete. 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.181+0000 7ff68ac00b80 1 -- start start 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.181+0000 7ff68ac00b80 1 Processor -- start 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d793020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7dd450 0x55713d7b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff68ac00b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d842750 con 0x55713d7dd450 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713d793560 con 0x55713d7c2f40 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff68ac00b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55713d7b8480 con 0x55713d7e64d0 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff68ac00b80 10 client.0 _ll_register_callbacks cb 0x55713d4f5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7dd450 0x55713d7b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7dd450 0x55713d7b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41756/0 (socket says 172.21.15.45:41756) 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 -- 172.21.15.45:0/2992707506 learned_addr learned my addr 172.21.15.45:0/2992707506 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67affd700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d793020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 -- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7e64d0 msgr2=0x55713d7b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 -- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 msgr2=0x55713d793020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d793020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 -- 172.21.15.45:0/2992707506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55713d5d90a0 con 0x55713d7dd450 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67b7fe700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff67a7fc700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7dd450 0x55713d7b8ed0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0c7f90 tx=0x7ff664013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5ae0e3f35a6f1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff6597fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.182+0000 7ff6597fa700 1 -- 172.21.15.45:0/2992707506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66400eca0 con 0x55713d7dd450 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff6597fa700 1 -- 172.21.15.45:0/2992707506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff66400ee00 con 0x55713d7dd450 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6740127c0 con 0x55713d7dd450 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff6597fa700 1 -- 172.21.15.45:0/2992707506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff664004830 con 0x55713d7dd450 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 10 client.6279 fetch_fsmap learned FSMap version 48 2023-10-06T05:18:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55713d54d1d0 con 0x55713d7dd450 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff6597fa700 1 -- 172.21.15.45:0/2992707506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740127c0 con 0x55713d7dd450 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff6597fa700 1 -- 172.21.15.45:0/2992707506 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff664004990 con 0x55713d7dd450 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 10 client.6279 fetch_fsmap finished waiting for FSMap version 48 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 10 client.6279 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 20 client.6279 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d841c30 0x55713d842000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 4 client.6279 mds_command: new command op to 6114 tid=0[{"prefix": "status"}] 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 --> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55713d6898d0 con 0x55713d841c30 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff67affd700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d841c30 0x55713d842000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.183+0000 7ff6597fa700 10 client.6279 ms_handle_connect on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff67affd700 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d841c30 0x55713d842000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e897efd8fbabe57 server_cookie=972efa5abdb355ed in_seq=0 out_seq=0 2023-10-06T05:18:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff6597fa700 1 -- 172.21.15.45:0/2992707506 <== mds.0 v2:172.21.15.45:6832/896514784 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55713d6898d0 con 0x55713d841c30 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff6597fa700 10 client.6279 handle_command_reply: tid=0 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 client.6279 shutdown 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7dd450 msgr2=0x55713d7b8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7dd450 0x55713d7b8ed0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0c7f90 tx=0x7ff664013000 comp rx=0 tx=0).stop 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 shutdown_connections 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55713d7e64d0 0x55713d7b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55713d7c2f40 0x55713d793020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55713d841c30 0x55713d842000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 --2- 172.21.15.45:0/2992707506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55713d7dd450 0x55713d7b8ed0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff6597fa700 0 client.6279 ms_handle_reset on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 >> 172.21.15.45:0/2992707506 conn(0x55713d7e0880 msgr2=0x55713d7e5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 shutdown_connections 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 1 -- 172.21.15.45:0/2992707506 wait complete. 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.184+0000 7ff68ac00b80 20 client.6279 trim_cache size 0 max 16384 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6114, 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:18:16.193 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:18:16.194 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.0919722299999997, 2023-10-06T05:18:16.194 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 48, 2023-10-06T05:18:16.194 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 64, 2023-10-06T05:18:16.194 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 64, 2023-10-06T05:18:16.194 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 12.801783055 2023-10-06T05:18:16.194 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:18:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff65c066f30 msgr2=0x7ff65c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff65c066f30 0x7ff65c0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff67c1600c0 tx=0x7ff664009040 comp rx=0 tx=0).stop 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0b0290 msgr2=0x7ff67c0cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0b0290 0x7ff67c0cb0d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00d810 tx=0x7ff66c008040 comp rx=0 tx=0).stop 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 shutdown_connections 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff67c0c33f0 0x7ff67c0c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff67c0b0290 0x7ff67c0cb0d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 --2- 172.21.15.45:0/3580508991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff67c0b0660 0x7ff67c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 >> 172.21.15.45:0/3580508991 conn(0x7ff67c067e00 msgr2=0x7ff67c06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 shutdown_connections 2023-10-06T05:18:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.185+0000 7ff6818ec700 1 -- 172.21.15.45:0/3580508991 wait complete. 2023-10-06T05:18:16.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:18:16.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.329+0000 7f90b7b1b700 1 Processor -- start 2023-10-06T05:18:16.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.329+0000 7f90b7b1b700 1 -- start start 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.330+0000 7f90b7b1b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f90b0153620 0x7f90b015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.330+0000 7f90b7b1b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b01539f0 0x7f90b015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.330+0000 7f90b7b1b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b00cc8b0 0x7f90b00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.330+0000 7f90b7b1b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b015a7a0 con 0x7f90b01539f0 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.330+0000 7f90b7b1b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b00ce3f0 con 0x7f90b0153620 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.330+0000 7f90b7b1b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b015d300 con 0x7f90b00cc8b0 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.330+0000 7f90b50b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b01539f0 0x7f90b015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b01539f0 0x7f90b015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41770/0 (socket says 172.21.15.45:41770) 2023-10-06T05:18:16.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 -- 172.21.15.45:0/1573907708 learned_addr learned my addr 172.21.15.45:0/1573907708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b60b8700 1 --2- 172.21.15.45:0/1573907708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b00cc8b0 0x7f90b00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b58b7700 1 --2- 172.21.15.45:0/1573907708 >> v2:172.21.15.144:3300/0 conn(0x7f90b0153620 0x7f90b015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 -- 172.21.15.45:0/1573907708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b00cc8b0 msgr2=0x7f90b00ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 --2- 172.21.15.45:0/1573907708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b00cc8b0 0x7f90b00ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 -- 172.21.15.45:0/1573907708 >> v2:172.21.15.144:3300/0 conn(0x7f90b0153620 msgr2=0x7f90b015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 --2- 172.21.15.45:0/1573907708 >> v2:172.21.15.144:3300/0 conn(0x7f90b0153620 0x7f90b015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 -- 172.21.15.45:0/1573907708 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b00ce690 con 0x7f90b01539f0 2023-10-06T05:18:16.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b50b6700 1 --2- 172.21.15.45:0/1573907708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b01539f0 0x7f90b015cdc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f90b00c6f30 tx=0x7f90ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a14b0c42098873 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b48b5700 1 -- 172.21.15.45:0/1573907708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00b640 con 0x7f90b01539f0 2023-10-06T05:18:16.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b48b5700 1 -- 172.21.15.45:0/1573907708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac00b7a0 con 0x7f90b01539f0 2023-10-06T05:18:16.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.331+0000 7f90b48b5700 1 -- 172.21.15.45:0/1573907708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00bb60 con 0x7f90b01539f0 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 -- 172.21.15.45:0/1573907708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b01539f0 msgr2=0x7f90b015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/1573907708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b01539f0 0x7f90b015cdc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f90b00c6f30 tx=0x7f90ac009250 comp rx=0 tx=0).stop 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 -- 172.21.15.45:0/1573907708 shutdown_connections 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/1573907708 >> v2:172.21.15.144:3300/0 conn(0x7f90b0153620 0x7f90b015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/1573907708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b00cc8b0 0x7f90b00ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/1573907708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b01539f0 0x7f90b015cdc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 -- 172.21.15.45:0/1573907708 >> 172.21.15.45:0/1573907708 conn(0x7f90b006ed50 msgr2=0x7f90b006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 -- 172.21.15.45:0/1573907708 shutdown_connections 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.332+0000 7f90b7b1b700 1 -- 172.21.15.45:0/1573907708 wait complete. 2023-10-06T05:18:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.334+0000 7f90b7b1b700 1 Processor -- start 2023-10-06T05:18:16.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.334+0000 7f90b7b1b700 1 -- start start 2023-10-06T05:18:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b7b1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00cc8b0 0x7f90b014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b7b1b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90b0153620 0x7f90b01511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b7b1b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b0151710 0x7f90b00b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b7b1b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0151bb0 con 0x7f90b00cc8b0 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b7b1b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0154570 con 0x7f90b0153620 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b7b1b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0154700 con 0x7f90b0151710 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00cc8b0 0x7f90b014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00cc8b0 0x7f90b014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41772/0 (socket says 172.21.15.45:41772) 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 -- 172.21.15.45:0/2737943613 learned_addr learned my addr 172.21.15.45:0/2737943613 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b60b8700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b0151710 0x7f90b00b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b50b6700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90b0153620 0x7f90b01511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 -- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b0151710 msgr2=0x7f90b00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b0151710 0x7f90b00b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 -- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90b0153620 msgr2=0x7f90b01511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90b0153620 0x7f90b01511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 -- 172.21.15.45:0/2737943613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac0094f0 con 0x7f90b00cc8b0 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b58b7700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00cc8b0 0x7f90b014fea0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90a400ec80 tx=0x7f90a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90a1ffb700 1 -- 172.21.15.45:0/2737943613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a400c9d0 con 0x7f90b00cc8b0 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90a1ffb700 1 -- 172.21.15.45:0/2737943613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a400cb30 con 0x7f90b00cc8b0 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.335+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b01503e0 con 0x7f90b00cc8b0 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.336+0000 7f90a1ffb700 1 -- 172.21.15.45:0/2737943613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4013470 con 0x7f90b00cc8b0 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.336+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b00b4800 con 0x7f90b00cc8b0 2023-10-06T05:18:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.337+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b0088a60 con 0x7f90b00cc8b0 2023-10-06T05:18:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.338+0000 7f90a1ffb700 1 -- 172.21.15.45:0/2737943613 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90a4013820 con 0x7f90b00cc8b0 2023-10-06T05:18:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.338+0000 7f90a1ffb700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9094066f30 0x7f90940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.338+0000 7f90a1ffb700 1 -- 172.21.15.45:0/2737943613 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f90a4085350 con 0x7f90b00cc8b0 2023-10-06T05:18:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.338+0000 7f90b50b6700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9094066f30 0x7f90940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.339+0000 7f90b50b6700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9094066f30 0x7f90940693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f90b00c6f30 tx=0x7f90ac017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.342+0000 7f90a1ffb700 1 -- 172.21.15.45:0/2737943613 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90a40523a0 con 0x7f90b00cc8b0 2023-10-06T05:18:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.592+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f90b004bb40 con 0x7f90b00cc8b0 2023-10-06T05:18:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.593+0000 7f90a1ffb700 1 -- 172.21.15.45:0/2737943613 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f90a4017060 con 0x7f90b00cc8b0 2023-10-06T05:18:16.595 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:18:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9094066f30 msgr2=0x7f90940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9094066f30 0x7f90940693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f90b00c6f30 tx=0x7f90ac017000 comp rx=0 tx=0).stop 2023-10-06T05:18:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00cc8b0 msgr2=0x7f90b014fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00cc8b0 0x7f90b014fea0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90a400ec80 tx=0x7f90a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 shutdown_connections 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9094066f30 0x7f90940693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90b0151710 0x7f90b00b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00cc8b0 0x7f90b014fea0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 --2- 172.21.15.45:0/2737943613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90b0153620 0x7f90b01511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 >> 172.21.15.45:0/2737943613 conn(0x7f90b006ed50 msgr2=0x7f90b0150870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 shutdown_connections 2023-10-06T05:18:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:16.595+0000 7f90b7b1b700 1 -- 172.21.15.45:0/2737943613 wait complete. 2023-10-06T05:18:16.610 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:18:16.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:18:16.610 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:18:16.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:18:16.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:16.611 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:18:16.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:18:16.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:18:16.611 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:16.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:16.611 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:18:16.632 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:18:16.632 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:18:16.632 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:18:16.632 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:16.632 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:18:16.632 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:16.632 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42245sec preferred_lft 42245sec 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:18:16.633 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:18:16.634 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:16.635 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:16.635 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:18:16.635 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:18:16.635 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:16.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:16.635 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:18:16.703 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:18:16.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:16.705 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:18:16.771 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:18:16.771 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:18:16.772 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:18:16.772 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:18:16.772 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:18:16.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:16.772 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:18:16.772 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:18:16.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:18:16.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:18:16.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:18:16.773 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:18:16.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:16.895 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:18:16.917 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:18:16.917 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:18:16.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:16.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:16.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:18:16.993 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:18:16.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:16.993 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:17.056 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:17.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:17.056 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:17.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:18:17.196 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:18:17.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:18:17.271 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:18:17.272 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:18:17.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:17.273 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:18:17.336 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:18:17.336 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:18:17.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:18:17.413 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:18:17.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:18:17.477 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:18:17.476+0000 7f67df684540 -1 init, newargv = 0x55577ca26940 newargc=15 2023-10-06T05:18:17.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[145247]: starting ceph client 2023-10-06T05:18:17.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[145247]: starting fuse 2023-10-06T05:18:17.487 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:18:17.489 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:18:17.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:17.489 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:18:17.550 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:18:17.551 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:18:17.551 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:18:17.552 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:18:17.553 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:18:17.553 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:18:17.553 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:17.553 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:18:17.553 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:18:17.690 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.145247.asok 2023-10-06T05:18:17.871 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.145247.asok 2023-10-06T05:18:17.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:17.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.145247.asok status 2023-10-06T05:18:18.046 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "145247", 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:18:18.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6291, 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 6291 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:18:18.048 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1843364986 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1843364986 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.6291 192.168.0.1:0/1843364986", 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1843364986", 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:18:18.049 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 48, 2023-10-06T05:18:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 64, 2023-10-06T05:18:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:18:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:18:18.050 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:18:18.050 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:18:18.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:18.057 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:18:18.058 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:18.078 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:18:18.078 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:18.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:18.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:18.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:18.153 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p a/{b,c}/d; sync .') 2023-10-06T05:18:18.265 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:18:18.265 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:18:18.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:18.305 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[145247]: fuse finished with error 0 and tester_r 0 2023-10-06T05:18:18.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:18:18.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:18.307 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:18:18.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2023-10-06T05:18:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.554+0000 7fec32a0c700 1 Processor -- start 2023-10-06T05:18:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.554+0000 7fec32a0c700 1 -- start start 2023-10-06T05:18:18.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec32a0c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0c7fd0 0x7fec2c155d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec32a0c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec2c06dff0 0x7fec2c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec32a0c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c1596f0 con 0x7fec2c0ae310 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec32a0c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c15a290 con 0x7fec2c06dff0 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec32a0c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c06e9d0 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec2c06dff0 0x7fec2c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec2c06dff0 0x7fec2c06e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44704/0 (socket says 172.21.15.45:44704) 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 -- 172.21.15.45:0/2977536950 learned_addr learned my addr 172.21.15.45:0/2977536950 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/2977536950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0c7fd0 0x7fec2c155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 -- 172.21.15.45:0/2977536950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0c7fd0 msgr2=0x7fec2c155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 --2- 172.21.15.45:0/2977536950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0c7fd0 0x7fec2c155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 -- 172.21.15.45:0/2977536950 >> v1:172.21.15.144:6789/0 conn(0x7fec2c0ae310 legacy=0x7fec2c159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 -- 172.21.15.45:0/2977536950 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c1566d0 con 0x7fec2c06dff0 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/2977536950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0c7fd0 0x7fec2c155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.555+0000 7fec30fa9700 1 --2- 172.21.15.45:0/2977536950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c06dff0 0x7fec2c06e490 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fec2400d1a0 tx=0x7fec2400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91a1a42dcfeab6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec2affd700 1 -- 172.21.15.45:0/2977536950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24010070 con 0x7fec2c06dff0 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec2affd700 1 -- 172.21.15.45:0/2977536950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2400de80 con 0x7fec2c06dff0 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec2affd700 1 -- 172.21.15.45:0/2977536950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2400b320 con 0x7fec2c06dff0 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 -- 172.21.15.45:0/2977536950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c06dff0 msgr2=0x7fec2c06e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2977536950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c06dff0 0x7fec2c06e490 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fec2400d1a0 tx=0x7fec2400ba50 comp rx=0 tx=0).stop 2023-10-06T05:18:18.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 -- 172.21.15.45:0/2977536950 shutdown_connections 2023-10-06T05:18:18.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2977536950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c06dff0 0x7fec2c06e490 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2977536950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0c7fd0 0x7fec2c155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 -- 172.21.15.45:0/2977536950 >> 172.21.15.45:0/2977536950 conn(0x7fec2c0680f0 msgr2=0x7fec2c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 -- 172.21.15.45:0/2977536950 shutdown_connections 2023-10-06T05:18:18.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.556+0000 7fec32a0c700 1 -- 172.21.15.45:0/2977536950 wait complete. 2023-10-06T05:18:18.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.558+0000 7fec32a0c700 1 Processor -- start 2023-10-06T05:18:18.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.558+0000 7fec32a0c700 1 -- start start 2023-10-06T05:18:18.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec32a0c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c06dff0 0x7fec2c0c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec32a0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ae310 0x7fec2c1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec32a0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c0c7fd0 0x7fec2c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec32a0c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0ab350 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec32a0c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0c2dd0 con 0x7fec2c0ae310 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec32a0c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0c30b0 con 0x7fec2c06dff0 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec30fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c0c7fd0 0x7fec2c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec30fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c0c7fd0 0x7fec2c0cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44708/0 (socket says 172.21.15.45:44708) 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec30fa9700 1 -- 172.21.15.45:0/2401502433 learned_addr learned my addr 172.21.15.45:0/2401502433 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ae310 0x7fec2c1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.559+0000 7fec2bfff700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c06dff0 0x7fec2c0c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec30fa9700 1 -- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c06dff0 msgr2=0x7fec2c0c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec30fa9700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c06dff0 0x7fec2c0c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec30fa9700 1 -- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ae310 msgr2=0x7fec2c1519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec30fa9700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ae310 0x7fec2c1519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec30fa9700 1 -- 172.21.15.45:0/2401502433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2400bcf0 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec30fa9700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c0c7fd0 0x7fec2c0cadf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fec2c06d560 tx=0x7fec24019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec13fff700 1 -- 172.21.15.45:0/2401502433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24009ab0 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec13fff700 1 -- 172.21.15.45:0/2401502433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24009c30 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c0c3390 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec13fff700 1 -- 172.21.15.45:0/2401502433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2401e440 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.560+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c152750 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.562+0000 7fec13fff700 1 -- 172.21.15.45:0/2401502433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec2401e850 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.562+0000 7fec13fff700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.562+0000 7fec13fff700 1 -- 172.21.15.45:0/2401502433 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fec24086100 con 0x7fec2c0c7fd0 2023-10-06T05:18:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.562+0000 7fec2bfff700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.563+0000 7fec2bfff700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fec2c16ccb0 tx=0x7fec1c007ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.564+0000 7fec3bd20b80 1 Processor -- start 2023-10-06T05:18:18.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.564+0000 7fec3bd20b80 1 -- start start 2023-10-06T05:18:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb68a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb692fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb693990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb55d360 con 0x55d5cb68a4a0 2023-10-06T05:18:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb59a180 con 0x55d5cb693520 2023-10-06T05:18:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb597b40 con 0x55d5cb66ff90 2023-10-06T05:18:18.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb68a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb68a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44724/0 (socket says 172.21.15.45:44724) 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 -- 172.21.15.45:0/3011179425 learned_addr learned my addr 172.21.15.45:0/3011179425 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec30fa9700 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb693990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb692fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 -- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 msgr2=0x55d5cb692fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb692fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 -- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 msgr2=0x55d5cb693990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb693990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.565+0000 7fec2bfff700 1 -- 172.21.15.45:0/3011179425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5cb3997e0 con 0x55d5cb68a4a0 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.566+0000 7fec2bfff700 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb68a870 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fec1c002180 tx=0x7fec1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1155b84b60163bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.566+0000 7fec117fa700 1 -- 172.21.15.45:0/3011179425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0146e0 con 0x55d5cb68a4a0 2023-10-06T05:18:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.566+0000 7fec117fa700 1 -- 172.21.15.45:0/3011179425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c019070 con 0x55d5cb68a4a0 2023-10-06T05:18:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.566+0000 7fec117fa700 1 -- 172.21.15.45:0/3011179425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c014a50 con 0x55d5cb68a4a0 2023-10-06T05:18:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 -- 172.21.15.45:0/3011179425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 msgr2=0x55d5cb68a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb68a870 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fec1c002180 tx=0x7fec1c016000 comp rx=0 tx=0).stop 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 -- 172.21.15.45:0/3011179425 shutdown_connections 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb693990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb68a870 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/3011179425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb692fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 -- 172.21.15.45:0/3011179425 >> 172.21.15.45:0/3011179425 conn(0x55d5cb68d8d0 msgr2=0x55d5cb68dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 -- 172.21.15.45:0/3011179425 shutdown_connections 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 -- 172.21.15.45:0/3011179425 wait complete. 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.567+0000 7fec3bd20b80 1 -- start start 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 1 Processor -- start 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb68a4a0 0x55d5cb665520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb644b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb593930 con 0x55d5cb693520 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb53f9d0 con 0x55d5cb66ff90 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb53f670 con 0x55d5cb68a4a0 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.568+0000 7fec3bd20b80 10 client.0 _ll_register_callbacks cb 0x55d5cb3a2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb644b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb644b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44740/0 (socket says 172.21.15.45:44740) 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 -- 172.21.15.45:0/745958352 learned_addr learned my addr 172.21.15.45:0/745958352 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec2bfff700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb68a4a0 0x55d5cb665520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 -- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb68a4a0 msgr2=0x55d5cb665520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb68a4a0 0x55d5cb665520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 -- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb66ff90 msgr2=0x55d5cb6898d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 -- 172.21.15.45:0/745958352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c00f7d0 con 0x55d5cb693520 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb68a4a0 0x55d5cb665520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec30fa9700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb644b60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fec24056a90 tx=0x7fec24086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ca1f1d9a5c7266 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec0a7fc700 1 -- 172.21.15.45:0/745958352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24010030 con 0x55d5cb693520 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec0a7fc700 1 -- 172.21.15.45:0/745958352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fec2406d170 con 0x55d5cb693520 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5cb637db0 con 0x55d5cb693520 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec0a7fc700 1 -- 172.21.15.45:0/745958352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec24056d90 con 0x55d5cb693520 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec3bd20b80 10 client.6303 fetch_fsmap learned FSMap version 48 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55d5cb4860f0 con 0x55d5cb693520 2023-10-06T05:18:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.569+0000 7fec0a7fc700 1 -- 172.21.15.45:0/745958352 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24082020 con 0x55d5cb693520 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec0a7fc700 1 -- 172.21.15.45:0/745958352 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fec2405ad20 con 0x55d5cb693520 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec3bd20b80 10 client.6303 fetch_fsmap finished waiting for FSMap version 48 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec3bd20b80 10 client.6303 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec3bd20b80 20 client.6303 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb667f20 0x55d5cb68b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec3bd20b80 4 client.6303 mds_command: new command op to 6114 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 --> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5cb614b80 con 0x55d5cb667f20 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec2bfff700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb667f20 0x55d5cb68b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec0a7fc700 10 client.6303 ms_handle_connect on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.570+0000 7fec2bfff700 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb667f20 0x55d5cb68b7d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7c6152dfc35598 server_cookie=46bfea1628443b87 in_seq=0 out_seq=0 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec0a7fc700 1 -- 172.21.15.45:0/745958352 <== mds.0 v2:172.21.15.45:6832/896514784 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d5cb614b80 con 0x55d5cb667f20 2023-10-06T05:18:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec0a7fc700 10 client.6303 handle_command_reply: tid=0 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 client.6303 shutdown 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 msgr2=0x55d5cb644b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb644b60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fec24056a90 tx=0x7fec24086040 comp rx=0 tx=0).stop 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 shutdown_connections 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb667f20 0x55d5cb68b7d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb644b60 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec0a7fc700 0 client.6303 ms_handle_reset on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/745958352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb68a4a0 0x55d5cb665520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 >> 172.21.15.45:0/745958352 conn(0x55d5cb68d8d0 msgr2=0x55d5cb63fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 shutdown_connections 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 1 -- 172.21.15.45:0/745958352 wait complete. 2023-10-06T05:18:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.571+0000 7fec3bd20b80 20 client.6303 trim_cache size 0 max 16384 2023-10-06T05:18:18.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.586+0000 7fec3bd20b80 1 Processor -- start 2023-10-06T05:18:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.586+0000 7fec3bd20b80 1 -- start start 2023-10-06T05:18:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb644aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb68cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb42b230 con 0x55d5cb68a4a0 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb68a180 con 0x55d5cb693520 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb66daa0 con 0x55d5cb66ff90 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb644aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb644aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44752/0 (socket says 172.21.15.45:44752) 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 -- 172.21.15.45:0/314837141 learned_addr learned my addr 172.21.15.45:0/314837141 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec30fa9700 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb68cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2bfff700 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 -- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 msgr2=0x55d5cb6898d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 -- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 msgr2=0x55d5cb68cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb68cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.587+0000 7fec2b7fe700 1 -- 172.21.15.45:0/314837141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5cb562410 con 0x55d5cb68a4a0 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb644aa0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fec2000ac40 tx=0x7fec20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839c49c604eff7c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec11ffb700 1 -- 172.21.15.45:0/314837141 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20013ba0 con 0x55d5cb68a4a0 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec11ffb700 1 -- 172.21.15.45:0/314837141 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2000c040 con 0x55d5cb68a4a0 2023-10-06T05:18:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec11ffb700 1 -- 172.21.15.45:0/314837141 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2001e460 con 0x55d5cb68a4a0 2023-10-06T05:18:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 -- 172.21.15.45:0/314837141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 msgr2=0x55d5cb644aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb644aa0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fec2000ac40 tx=0x7fec20008040 comp rx=0 tx=0).stop 2023-10-06T05:18:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 -- 172.21.15.45:0/314837141 shutdown_connections 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb693520 0x55d5cb68cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb6898d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/314837141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb68a4a0 0x55d5cb644aa0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 -- 172.21.15.45:0/314837141 >> 172.21.15.45:0/314837141 conn(0x55d5cb68d8d0 msgr2=0x55d5cb640800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 -- 172.21.15.45:0/314837141 shutdown_connections 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.588+0000 7fec3bd20b80 1 -- 172.21.15.45:0/314837141 wait complete. 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.589+0000 7fec3bd20b80 1 -- start start 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.589+0000 7fec3bd20b80 1 Processor -- start 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb63fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb68a4a0 0x55d5cb6d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec3bd20b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb6d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb6f0060 con 0x55d5cb693520 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb66b4f0 con 0x55d5cb68a4a0 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec3bd20b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5cb6729d0 con 0x55d5cb66ff90 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec3bd20b80 10 client.0 _ll_register_callbacks cb 0x55d5cb3a2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec30fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb6d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec30fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb6d5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44764/0 (socket says 172.21.15.45:44764) 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec30fa9700 1 -- 172.21.15.45:0/1307093385 learned_addr learned my addr 172.21.15.45:0/1307093385 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec2b7fe700 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb68a4a0 0x55d5cb6d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec2bfff700 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb63fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec30fa9700 1 -- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 msgr2=0x55d5cb63fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec30fa9700 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb63fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec30fa9700 1 -- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb68a4a0 msgr2=0x55d5cb6d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.590+0000 7fec30fa9700 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb68a4a0 0x55d5cb6d4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec30fa9700 1 -- 172.21.15.45:0/1307093385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2000a970 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec30fa9700 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb6d5060 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fec2400d8f0 tx=0x7fec24029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac141f0209b7e1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec0a7fc700 1 -- 172.21.15.45:0/1307093385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24010040 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec0a7fc700 1 -- 172.21.15.45:0/1307093385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fec24083080 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec1c019070 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec0a7fc700 1 -- 172.21.15.45:0/1307093385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec2400de80 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 10 client.6309 fetch_fsmap learned FSMap version 48 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55d5cb3fa220 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec0a7fc700 1 -- 172.21.15.45:0/1307093385 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c019070 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec0a7fc700 1 -- 172.21.15.45:0/1307093385 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fec2401d3a0 con 0x55d5cb693520 2023-10-06T05:18:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 10 client.6309 fetch_fsmap finished waiting for FSMap version 48 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 10 client.6309 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 20 client.6309 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb68cac0 0x55d5cb66b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 4 client.6309 mds_command: new command op to 6114 tid=0[{"prefix": "flush journal"}] 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 --> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5cb536920 con 0x55d5cb68cac0 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.591+0000 7fec2bfff700 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb68cac0 0x55d5cb66b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.592+0000 7fec0a7fc700 10 client.6309 ms_handle_connect on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.592+0000 7fec2bfff700 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb68cac0 0x55d5cb66b940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e790775fdcf5415 server_cookie=769303d1817510ee in_seq=0 out_seq=0 2023-10-06T05:18:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec0a7fc700 1 -- 172.21.15.45:0/1307093385 <== mds.0 v2:172.21.15.45:6832/896514784 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5cb536920 con 0x55d5cb68cac0 2023-10-06T05:18:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec0a7fc700 10 client.6309 handle_command_reply: tid=0 2023-10-06T05:18:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 client.6309 shutdown 2023-10-06T05:18:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 msgr2=0x55d5cb6d5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb6d5060 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fec2400d8f0 tx=0x7fec24029000 comp rx=0 tx=0).stop 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 shutdown_connections 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:6832/896514784,v1:172.21.15.45:6833/896514784] conn(0x55d5cb68cac0 0x55d5cb66b940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5cb693520 0x55d5cb6d5060 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5cb66ff90 0x55d5cb63fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 --2- 172.21.15.45:0/1307093385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5cb68a4a0 0x55d5cb6d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec0a7fc700 0 client.6309 ms_handle_reset on v2:172.21.15.45:6832/896514784 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 >> 172.21.15.45:0/1307093385 conn(0x55d5cb68d8d0 msgr2=0x55d5cb640800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 shutdown_connections 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 1 -- 172.21.15.45:0/1307093385 wait complete. 2023-10-06T05:18:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.601+0000 7fec3bd20b80 20 client.6309 trim_cache size 0 max 16384 2023-10-06T05:18:18.605 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:18:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2023-10-06T05:18:18.605 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2023-10-06T05:18:18.605 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 msgr2=0x7fec180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fec2c16ccb0 tx=0x7fec1c007ea0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c0c7fd0 msgr2=0x7fec2c0cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c0c7fd0 0x7fec2c0cadf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fec2c06d560 tx=0x7fec24019000 comp rx=0 tx=0).stop 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 shutdown_connections 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec18066f30 0x7fec180693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec2c0c7fd0 0x7fec2c0cadf0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c06dff0 0x7fec2c0c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 --2- 172.21.15.45:0/2401502433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ae310 0x7fec2c1519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 >> 172.21.15.45:0/2401502433 conn(0x7fec2c0680f0 msgr2=0x7fec2c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 shutdown_connections 2023-10-06T05:18:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.602+0000 7fec32a0c700 1 -- 172.21.15.45:0/2401502433 wait complete. 2023-10-06T05:18:18.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:18:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.747+0000 7f59e4cb7700 1 Processor -- start 2023-10-06T05:18:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.747+0000 7f59e4cb7700 1 -- start start 2023-10-06T05:18:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.748+0000 7f59e4cb7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59e0159200 0x7f59e00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.748+0000 7f59e4cb7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 0x7f59e00cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.748+0000 7f59e4cb7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e00c7d00 con 0x7f59e01595d0 2023-10-06T05:18:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.748+0000 7f59e4cb7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e00c7780 con 0x7f59e0159200 2023-10-06T05:18:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.748+0000 7f59e4cb7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00cd720 con 0x7f59e00ccdc0 2023-10-06T05:18:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59ddd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59e01595d0 0x7f59e006b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40786/0 (socket says 172.21.15.45:40786) 2023-10-06T05:18:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59ddd9b700 1 -- 172.21.15.45:0/3753485246 learned_addr learned my addr 172.21.15.45:0/3753485246 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59de59c700 1 --2- 172.21.15.45:0/3753485246 >> v2:172.21.15.144:3300/0 conn(0x7f59e0159200 0x7f59e00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59ded9d700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 0x7f59e00cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824924166 0 0) 0x7f59e00c7d00 con 0x7f59e01595d0 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59cc003570 con 0x7f59e01595d0 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756514097 0 0) 0x7f59cc003570 con 0x7f59e01595d0 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 msgr2=0x7f59e00cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 0x7f59e00cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 >> v2:172.21.15.144:3300/0 conn(0x7f59e0159200 msgr2=0x7f59e00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 --2- 172.21.15.45:0/3753485246 >> v2:172.21.15.144:3300/0 conn(0x7f59e0159200 0x7f59e00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e00c86d0 con 0x7f59e01595d0 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f59c8001470 con 0x7f59e01595d0 2023-10-06T05:18:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 >> v1:172.21.15.45:6789/0 conn(0x7f59e01595d0 legacy=0x7f59e006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc003a60 0x7f59cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 0x7f59cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc004d50 0x7f59cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00c7780 con 0x7f59cc004d50 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00cd720 con 0x7f59cc003a60 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc00ba60 con 0x7f59e00ccdc0 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.749+0000 7f59ded9d700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc004d50 0x7f59cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59ddd9b700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 0x7f59cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59de59c700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc003a60 0x7f59cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59ded9d700 1 -- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 msgr2=0x7f59cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59ded9d700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00ccdc0 0x7f59cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59ded9d700 1 -- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc003a60 msgr2=0x7f59cc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59ded9d700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc003a60 0x7f59cc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59ded9d700 1 -- 172.21.15.45:0/3753485246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc003570 con 0x7f59cc004d50 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59ded9d700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc004d50 0x7f59cc0051c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e210 tx=0x7f59d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c0072c5930efff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800ea60 con 0x7f59cc004d50 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d800f040 con 0x7f59cc004d50 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.750+0000 7f59dd59a700 1 -- 172.21.15.45:0/3753485246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800ee10 con 0x7f59cc004d50 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.751+0000 7f59e4cb7700 1 -- 172.21.15.45:0/3753485246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc004d50 msgr2=0x7f59cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.751+0000 7f59e4cb7700 1 --2- 172.21.15.45:0/3753485246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc004d50 0x7f59cc0051c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e210 tx=0x7f59d800b040 comp rx=0 tx=0).stop 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.751+0000 7f59ded9d700 1 -- 172.21.15.45:0/3753485246 reap_dead start 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.751+0000 7f59e4cb7700 1 -- 172.21.15.45:0/3753485246 shutdown_connections 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.751+0000 7f59e4cb7700 1 -- 172.21.15.45:0/3753485246 >> 172.21.15.45:0/3753485246 conn(0x7f59e006f090 msgr2=0x7f59e006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.751+0000 7f59e4cb7700 1 -- 172.21.15.45:0/3753485246 shutdown_connections 2023-10-06T05:18:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.751+0000 7f59e4cb7700 1 -- 172.21.15.45:0/3753485246 wait complete. 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.752+0000 7f59e4cb7700 1 Processor -- start 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.752+0000 7f59e4cb7700 1 -- start start 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59e4cb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc004d50 0x7f59e00c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59e4cb7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e015a520 0x7f59e00c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59e4cb7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e015b450 0x7f59e00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59e4cb7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00c4dc0 con 0x7f59e015a520 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59e4cb7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015ad20 con 0x7f59cc004d50 2023-10-06T05:18:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59e4cb7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015afc0 con 0x7f59e015b450 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e015a520 0x7f59e00c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e015a520 0x7f59e00c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44778/0 (socket says 172.21.15.45:44778) 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 -- 172.21.15.45:0/2939682420 learned_addr learned my addr 172.21.15.45:0/2939682420 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59de59c700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc004d50 0x7f59e00c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ded9d700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e015b450 0x7f59e00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 -- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e015b450 msgr2=0x7f59e00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e015b450 0x7f59e00c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 -- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc004d50 msgr2=0x7f59e00c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc004d50 0x7f59e00c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 -- 172.21.15.45:0/2939682420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e015a990 con 0x7f59e015a520 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59de59c700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc004d50 0x7f59e00c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.753+0000 7f59ddd9b700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e015a520 0x7f59e00c6670 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f59c8001d20 tx=0x7f59c8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.754+0000 7f59d67fc700 1 -- 172.21.15.45:0/2939682420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8018030 con 0x7f59e015a520 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.754+0000 7f59d67fc700 1 -- 172.21.15.45:0/2939682420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c800f8a0 con 0x7f59e015a520 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.754+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e00c4520 con 0x7f59e015a520 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.754+0000 7f59d67fc700 1 -- 172.21.15.45:0/2939682420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c800fc60 con 0x7f59e015a520 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.754+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e00c4ae0 con 0x7f59e015a520 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.755+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e006e2f0 con 0x7f59e015a520 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.755+0000 7f59d67fc700 1 -- 172.21.15.45:0/2939682420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59c801b070 con 0x7f59e015a520 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.756+0000 7f59d67fc700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.756+0000 7f59d67fc700 1 -- 172.21.15.45:0/2939682420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f59c8019070 con 0x7f59e015a520 2023-10-06T05:18:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.756+0000 7f59de59c700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.756+0000 7f59de59c700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c0066f30 0x7f59c00693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f59e014ffb0 tx=0x7f59d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:18.760+0000 7f59d67fc700 1 -- 172.21.15.45:0/2939682420 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59c8051740 con 0x7f59e015a520 2023-10-06T05:18:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.010+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59e006bb40 con 0x7f59e015a520 2023-10-06T05:18:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.010+0000 7f59d67fc700 1 -- 172.21.15.45:0/2939682420 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59c8054d60 con 0x7f59e015a520 2023-10-06T05:18:19.012 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:18:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c0066f30 msgr2=0x7f59c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c0066f30 0x7f59c00693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f59e014ffb0 tx=0x7f59d0008040 comp rx=0 tx=0).stop 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e015a520 msgr2=0x7f59e00c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e015a520 0x7f59e00c6670 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f59c8001d20 tx=0x7f59c8001e90 comp rx=0 tx=0).stop 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 shutdown_connections 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c0066f30 0x7f59c00693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e015b450 0x7f59e00c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59cc004d50 0x7f59e00c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 --2- 172.21.15.45:0/2939682420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e015a520 0x7f59e00c6670 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 >> 172.21.15.45:0/2939682420 conn(0x7f59e006f090 msgr2=0x7f59e00ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.012+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 shutdown_connections 2023-10-06T05:18:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:19.013+0000 7f59e4cb7700 1 -- 172.21.15.45:0/2939682420 wait complete. 2023-10-06T05:18:19.027 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:18:19.028 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:19.028 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:18:19.049 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:18:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:18:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:18:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:18:19.049 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42242sec preferred_lft 42242sec 2023-10-06T05:18:19.050 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:19.051 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:18:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:18:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:19.052 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:18:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:18:19.052 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:19.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:19.052 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:18:19.120 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:18:19.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:19.122 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:18:19.187 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:18:19.187 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:18:19.187 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:18:19.187 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:18:19.187 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:18:19.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:19.188 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:18:19.188 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:18:19.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:18:19.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:18:19.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:18:19.188 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:18:19.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:19.311 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:18:19.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:18:19.333 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:18:19.334 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:19.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:19.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:18:19.409 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:18:19.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:19.409 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:19.473 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:19.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:19.473 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:19.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:18:19.612 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:18:19.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:18:19.687 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:18:19.688 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:18:19.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:19.689 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:18:19.752 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:18:19.752 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:18:19.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:18:19.830 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:18:19.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:18:19.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:18:19.894+0000 7fbab3641540 -1 init, newargv = 0x56048edf99d0 newargc=15 2023-10-06T05:18:19.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[145969]: starting ceph client 2023-10-06T05:18:19.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[145969]: starting fuse 2023-10-06T05:18:19.905 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:18:19.906 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:18:19.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:19.907 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:18:19.968 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:18:19.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:18:19.969 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:18:19.970 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:18:19.971 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:18:19.971 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:18:19.971 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:18:19.971 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:18:19.971 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:18:19.971 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:18:19.971 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:18:20.107 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.145969.asok 2023-10-06T05:18:20.288 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.145969.asok 2023-10-06T05:18:20.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:20.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.145969.asok status 2023-10-06T05:18:20.462 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:18:20.462 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "145969", 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6324, 2023-10-06T05:18:20.463 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 6324 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 734621525 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:18:20.464 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 734621525 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.6324 192.168.0.1:0/734621525", 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/734621525", 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 48, 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 64, 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:18:20.465 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:18:20.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:20.473 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:18:20.473 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:20.494 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:18:20.494 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:20.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:20.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:20.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:20.567 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'chmod 711 a/b/d; sync .') 2023-10-06T05:18:20.657 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:18:20.658 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:18:20.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:20.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[145969]: fuse finished with error 0 and tester_r 0 2023-10-06T05:18:20.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:18:20.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:20.708 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:18:20.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:18:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.912+0000 7f6f07077700 1 Processor -- start 2023-10-06T05:18:20.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.912+0000 7f6f07077700 1 -- start start 2023-10-06T05:18:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.913+0000 7f6f07077700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f000c8ff0 0x7f6f000c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.913+0000 7f6f07077700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f07077700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0015b000 con 0x7f6f00156af0 2023-10-06T05:18:20.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f07077700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0015b2e0 con 0x7f6f000c8ff0 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f07077700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0015b5c0 con 0x7f6f000caad0 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f04e13700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f000c8ff0 0x7f6f000c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f05614700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6f00156af0 0x7f6f00156f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40802/0 (socket says 172.21.15.45:40802) 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f05614700 1 -- 172.21.15.45:0/2057698883 learned_addr learned my addr 172.21.15.45:0/2057698883 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f04e13700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f000c8ff0 0x7f6f000c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44790/0 (socket says 172.21.15.45:44790) 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6efffff700 1 --2- 172.21.15.45:0/2057698883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f04e13700 1 -- 172.21.15.45:0/2057698883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 msgr2=0x7f6f000caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f04e13700 1 --2- 172.21.15.45:0/2057698883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f04e13700 1 -- 172.21.15.45:0/2057698883 >> v1:172.21.15.45:6789/0 conn(0x7f6f00156af0 legacy=0x7f6f00156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f04e13700 1 -- 172.21.15.45:0/2057698883 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0015b8a0 con 0x7f6f000c8ff0 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6eff7fe700 1 -- 172.21.15.45:0/2057698883 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509856753 0 0) 0x7f6f0015b000 con 0x7f6f00156af0 2023-10-06T05:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6efffff700 1 --2- 172.21.15.45:0/2057698883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6f04e13700 1 --2- 172.21.15.45:0/2057698883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000c93c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00098e0 tx=0x7f6ef00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d71e69f2c168c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.914+0000 7f6eff7fe700 1 -- 172.21.15.45:0/2057698883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0009e80 con 0x7f6f000c8ff0 2023-10-06T05:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6eff7fe700 1 -- 172.21.15.45:0/2057698883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef000b860 con 0x7f6f000c8ff0 2023-10-06T05:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6eff7fe700 1 -- 172.21.15.45:0/2057698883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000bbe0 con 0x7f6f000c8ff0 2023-10-06T05:18:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 -- 172.21.15.45:0/2057698883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 msgr2=0x7f6f000c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 --2- 172.21.15.45:0/2057698883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000c93c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00098e0 tx=0x7f6ef00091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 -- 172.21.15.45:0/2057698883 shutdown_connections 2023-10-06T05:18:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 --2- 172.21.15.45:0/2057698883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000c93c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 --2- 172.21.15.45:0/2057698883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 -- 172.21.15.45:0/2057698883 >> 172.21.15.45:0/2057698883 conn(0x7f6f000cdf50 msgr2=0x7f6f000ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 -- 172.21.15.45:0/2057698883 shutdown_connections 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.915+0000 7f6f07077700 1 -- 172.21.15.45:0/2057698883 wait complete. 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.916+0000 7f6f07077700 1 Processor -- start 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.917+0000 7f6f07077700 1 -- start start 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.917+0000 7f6f07077700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.917+0000 7f6f07077700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.917+0000 7f6f07077700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f00156af0 0x7f6f00158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.917+0000 7f6f07077700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f000b1be0 con 0x7f6f000c8ff0 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.917+0000 7f6f07077700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f000b0cf0 con 0x7f6f00156af0 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.917+0000 7f6f07077700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f000b0fd0 con 0x7f6f000caad0 2023-10-06T05:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44794/0 (socket says 172.21.15.45:44794) 2023-10-06T05:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 -- 172.21.15.45:0/2534755972 learned_addr learned my addr 172.21.15.45:0/2534755972 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f05614700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f00156af0 0x7f6f00158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6efffff700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 -- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 msgr2=0x7f6f000b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 -- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f00156af0 msgr2=0x7f6f00158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f00156af0 0x7f6f00158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 -- 172.21.15.45:0/2534755972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef0009420 con 0x7f6f000c8ff0 2023-10-06T05:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f04e13700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000b2070 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6f000c4f80 tx=0x7f6ef0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6efcff9700 1 -- 172.21.15.45:0/2534755972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0000c90 con 0x7f6f000c8ff0 2023-10-06T05:18:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6efcff9700 1 -- 172.21.15.45:0/2534755972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef000b9d0 con 0x7f6f000c8ff0 2023-10-06T05:18:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f000b12b0 con 0x7f6f000c8ff0 2023-10-06T05:18:20.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6efcff9700 1 -- 172.21.15.45:0/2534755972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0025670 con 0x7f6f000c8ff0 2023-10-06T05:18:20.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.918+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f000b1870 con 0x7f6f000c8ff0 2023-10-06T05:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.919+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f000b1ec0 con 0x7f6f000c8ff0 2023-10-06T05:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.920+0000 7f6efcff9700 1 -- 172.21.15.45:0/2534755972 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ef0023070 con 0x7f6f000c8ff0 2023-10-06T05:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.920+0000 7f6efcff9700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ee0066f30 0x7f6ee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.920+0000 7f6efcff9700 1 -- 172.21.15.45:0/2534755972 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6ef0085c70 con 0x7f6f000c8ff0 2023-10-06T05:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.921+0000 7f6efffff700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ee0066f30 0x7f6ee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.921+0000 7f6efffff700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ee0066f30 0x7f6ee00693e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f00158b70 tx=0x7f6ee8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:20.925+0000 7f6efcff9700 1 -- 172.21.15.45:0/2534755972 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ef0052c40 con 0x7f6f000c8ff0 2023-10-06T05:18:21.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.172+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6f0004bb40 con 0x7f6f000c8ff0 2023-10-06T05:18:21.175 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:18:21.173+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:18:21.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.178+0000 7f6efcff9700 1 -- 172.21.15.45:0/2534755972 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6ef0056260 con 0x7f6f000c8ff0 2023-10-06T05:18:21.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ee0066f30 msgr2=0x7f6ee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ee0066f30 0x7f6ee00693e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f00158b70 tx=0x7f6ee8008040 comp rx=0 tx=0).stop 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 msgr2=0x7f6f000b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000b2070 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6f000c4f80 tx=0x7f6ef0017000 comp rx=0 tx=0).stop 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 shutdown_connections 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ee0066f30 0x7f6ee00693e0 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f00158b70 tx=0x7f6ee8008040 comp rx=0 tx=0).stop 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f00156af0 0x7f6f00158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f000c8ff0 0x7f6f000b2070 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 --2- 172.21.15.45:0/2534755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f000caad0 0x7f6f000b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 >> 172.21.15.45:0/2534755972 conn(0x7f6f000cdf50 msgr2=0x7f6f000c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.180+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 shutdown_connections 2023-10-06T05:18:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.181+0000 7f6f07077700 1 -- 172.21.15.45:0/2534755972 wait complete. 2023-10-06T05:18:21.184 INFO:tasks.ceph.mds.a.smithi045.stderr: -4596> 2023-10-06T05:18:15.075+0000 7efe4f6f2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:18:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:18:21.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2023-10-06T05:18:21.210 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:18:21.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.343+0000 7f55763fc700 1 Processor -- start 2023-10-06T05:18:21.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.343+0000 7f55763fc700 1 -- start start 2023-10-06T05:18:21.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.344+0000 7f55763fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f55700c9f20 0x7f5570160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.344+0000 7f55763fc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55700ccca0 con 0x7f55700cc840 2023-10-06T05:18:21.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.344+0000 7f55763fc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55700c9630 con 0x7f557013f900 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.344+0000 7f55763fc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55700cb550 con 0x7f55700c9f20 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f556f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f55700cc840 0x7f55700c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40810/0 (socket says 172.21.15.45:40810) 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f556f7fe700 1 -- 172.21.15.45:0/3969152296 learned_addr learned my addr 172.21.15.45:0/3969152296 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f5574999700 1 --2- 172.21.15.45:0/3969152296 >> v2:172.21.15.144:3300/0 conn(0x7f55700c9f20 0x7f5570160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f556effd700 1 -- 172.21.15.45:0/3969152296 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404913457 0 0) 0x7f5564001110 con 0x7f55700cc840 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f556effd700 1 -- 172.21.15.45:0/3969152296 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5558003570 con 0x7f55700cc840 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f5574999700 1 -- 172.21.15.45:0/3969152296 >> v1:172.21.15.144:6789/0 conn(0x7f557013f900 legacy=0x7f55700c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f556effd700 1 -- 172.21.15.45:0/3969152296 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3559306932 0 0) 0x7f5558003570 con 0x7f55700cc840 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f5574999700 1 -- 172.21.15.45:0/3969152296 >> v1:172.21.15.45:6789/0 conn(0x7f55700cc840 legacy=0x7f55700c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f5574999700 1 -- 172.21.15.45:0/3969152296 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55700cb830 con 0x7f55700c9f20 2023-10-06T05:18:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.345+0000 7f5574999700 1 --2- 172.21.15.45:0/3969152296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700c9f20 0x7f5570160c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5568009e60 tx=0x7f5568009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5914cd5d688cfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f556effd700 1 -- 172.21.15.45:0/3969152296 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556800e070 con 0x7f55700c9f20 2023-10-06T05:18:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f556effd700 1 -- 172.21.15.45:0/3969152296 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556800b850 con 0x7f55700c9f20 2023-10-06T05:18:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f556effd700 1 -- 172.21.15.45:0/3969152296 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564001600 con 0x7f55700c9f20 2023-10-06T05:18:21.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f55763fc700 1 -- 172.21.15.45:0/3969152296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700c9f20 msgr2=0x7f5570160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f55763fc700 1 --2- 172.21.15.45:0/3969152296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700c9f20 0x7f5570160c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5568009e60 tx=0x7f5568009780 comp rx=0 tx=0).stop 2023-10-06T05:18:21.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f55763fc700 1 -- 172.21.15.45:0/3969152296 shutdown_connections 2023-10-06T05:18:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f55763fc700 1 --2- 172.21.15.45:0/3969152296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700c9f20 0x7f5570160c90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f55763fc700 1 -- 172.21.15.45:0/3969152296 >> 172.21.15.45:0/3969152296 conn(0x7f55700691c0 msgr2=0x7f55700cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f55763fc700 1 -- 172.21.15.45:0/3969152296 shutdown_connections 2023-10-06T05:18:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.346+0000 7f55763fc700 1 -- 172.21.15.45:0/3969152296 wait complete. 2023-10-06T05:18:21.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.347+0000 7f55763fc700 1 Processor -- start 2023-10-06T05:18:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.348+0000 7f55763fc700 1 -- start start 2023-10-06T05:18:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.348+0000 7f55763fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55700c9f20 0x7f5570130670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.348+0000 7f55763fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700cc840 0x7f55700b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.348+0000 7f55763fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557013f900 0x7f55700b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.348+0000 7f55763fc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55701595f0 con 0x7f557013f900 2023-10-06T05:18:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.348+0000 7f55763fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55701589e0 con 0x7f55700cc840 2023-10-06T05:18:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.348+0000 7f55763fc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5570158cc0 con 0x7f55700c9f20 2023-10-06T05:18:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557013f900 0x7f55700b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557013f900 0x7f55700b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44824/0 (socket says 172.21.15.45:44824) 2023-10-06T05:18:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 -- 172.21.15.45:0/2943235304 learned_addr learned my addr 172.21.15.45:0/2943235304 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f556ffff700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55700c9f20 0x7f5570130670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f556f7fe700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700cc840 0x7f55700b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 -- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55700c9f20 msgr2=0x7f5570130670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55700c9f20 0x7f5570130670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 -- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700cc840 msgr2=0x7f55700b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700cc840 0x7f55700b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 -- 172.21.15.45:0/2943235304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5568009a20 con 0x7f557013f900 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5574999700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557013f900 0x7f55700b0700 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f55680099f0 tx=0x7f5568017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5557fff700 1 -- 172.21.15.45:0/2943235304 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568022070 con 0x7f557013f900 2023-10-06T05:18:21.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5557fff700 1 -- 172.21.15.45:0/2943235304 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5568009420 con 0x7f557013f900 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5570158fa0 con 0x7f557013f900 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f5557fff700 1 -- 172.21.15.45:0/2943235304 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556801d280 con 0x7f557013f900 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.349+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5570159a80 con 0x7f557013f900 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.350+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557006e2d0 con 0x7f557013f900 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.351+0000 7f5557fff700 1 -- 172.21.15.45:0/2943235304 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f556801b6b0 con 0x7f557013f900 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.351+0000 7f5557fff700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f555c066f30 0x7f555c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.351+0000 7f5557fff700 1 -- 172.21.15.45:0/2943235304 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5568087a30 con 0x7f557013f900 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.351+0000 7f556ffff700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f555c066f30 0x7f555c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.352+0000 7f556ffff700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f555c066f30 0x7f555c0693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5560003a80 tx=0x7f5560007e60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.356+0000 7f5557fff700 1 -- 172.21.15.45:0/2943235304 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55680549b0 con 0x7f557013f900 2023-10-06T05:18:21.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.558+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) v1 -- 0x7f55700cbb10 con 0x7f557013f900 2023-10-06T05:18:21.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.562+0000 7f5557fff700 1 -- 172.21.15.45:0/2943235304 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v19)=0 v19) v1 ==== 139+0+0 (secure 0 0 0) 0x7f556802c0d0 con 0x7f557013f900 2023-10-06T05:18:21.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.564+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f555c066f30 msgr2=0x7f555c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.564+0000 7f55763fc700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f555c066f30 0x7f555c0693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5560003a80 tx=0x7f5560007e60 comp rx=0 tx=0).stop 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.564+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557013f900 msgr2=0x7f55700b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.564+0000 7f55763fc700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557013f900 0x7f55700b0700 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f55680099f0 tx=0x7f5568017000 comp rx=0 tx=0).stop 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 shutdown_connections 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f555c066f30 0x7f555c0693e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557013f900 0x7f55700b0700 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55700c9f20 0x7f5570130670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 --2- 172.21.15.45:0/2943235304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55700cc840 0x7f55700b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 >> 172.21.15.45:0/2943235304 conn(0x7f55700691c0 msgr2=0x7f557013ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 shutdown_connections 2023-10-06T05:18:21.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:21.565+0000 7f55763fc700 1 -- 172.21.15.45:0/2943235304 wait complete. 2023-10-06T05:18:26.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:18:26.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.722+0000 7f603370d700 1 Processor -- start 2023-10-06T05:18:26.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.723+0000 7f603370d700 1 -- start start 2023-10-06T05:18:26.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.723+0000 7f603370d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.723+0000 7f603370d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f602c0ccb70 0x7f602c0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.723+0000 7f603370d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c0c3b50 con 0x7f602c0aaae0 2023-10-06T05:18:26.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.723+0000 7f603370d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602c157700 con 0x7f602c0ccb70 2023-10-06T05:18:26.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.723+0000 7f603370d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0c98e0 con 0x7f602c157330 2023-10-06T05:18:26.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f6030ca8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f602c0aaae0 0x7f602c0c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59988/0 (socket says 172.21.15.45:59988) 2023-10-06T05:18:26.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f6030ca8700 1 -- 172.21.15.45:0/1525821274 learned_addr learned my addr 172.21.15.45:0/1525821274 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:26.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f60314a9700 1 --2- 172.21.15.45:0/1525821274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f6031caa700 1 --2- 172.21.15.45:0/1525821274 >> v2:172.21.15.144:3300/0 conn(0x7f602c0ccb70 0x7f602c0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f6023fff700 1 -- 172.21.15.45:0/1525821274 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878786544 0 0) 0x7f602c0c3b50 con 0x7f602c0aaae0 2023-10-06T05:18:26.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f6023fff700 1 -- 172.21.15.45:0/1525821274 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6018003570 con 0x7f602c0aaae0 2023-10-06T05:18:26.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f60314a9700 1 -- 172.21.15.45:0/1525821274 >> v2:172.21.15.144:3300/0 conn(0x7f602c0ccb70 msgr2=0x7f602c0c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f60314a9700 1 --2- 172.21.15.45:0/1525821274 >> v2:172.21.15.144:3300/0 conn(0x7f602c0ccb70 0x7f602c0c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f60314a9700 1 -- 172.21.15.45:0/1525821274 >> v1:172.21.15.45:6789/0 conn(0x7f602c0aaae0 legacy=0x7f602c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f60314a9700 1 -- 172.21.15.45:0/1525821274 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c0c9bc0 con 0x7f602c157330 2023-10-06T05:18:26.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f6031caa700 1 --2- 172.21.15.45:0/1525821274 >> v2:172.21.15.144:3300/0 conn(0x7f602c0ccb70 0x7f602c0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:26.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.724+0000 7f60314a9700 1 --2- 172.21.15.45:0/1525821274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0ce8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f60280098d0 tx=0x7f6028009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac7280dc712322c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f6023fff700 1 -- 172.21.15.45:0/1525821274 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028005c10 con 0x7f602c157330 2023-10-06T05:18:26.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f6023fff700 1 -- 172.21.15.45:0/1525821274 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6028005d70 con 0x7f602c157330 2023-10-06T05:18:26.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f6023fff700 1 -- 172.21.15.45:0/1525821274 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028006260 con 0x7f602c157330 2023-10-06T05:18:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 -- 172.21.15.45:0/1525821274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 msgr2=0x7f602c0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 --2- 172.21.15.45:0/1525821274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0ce8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f60280098d0 tx=0x7f6028009120 comp rx=0 tx=0).stop 2023-10-06T05:18:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 -- 172.21.15.45:0/1525821274 shutdown_connections 2023-10-06T05:18:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 --2- 172.21.15.45:0/1525821274 >> v2:172.21.15.144:3300/0 conn(0x7f602c0ccb70 0x7f602c0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 --2- 172.21.15.45:0/1525821274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0ce8b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 -- 172.21.15.45:0/1525821274 >> 172.21.15.45:0/1525821274 conn(0x7f602c067e00 msgr2=0x7f602c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 -- 172.21.15.45:0/1525821274 shutdown_connections 2023-10-06T05:18:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.725+0000 7f603370d700 1 -- 172.21.15.45:0/1525821274 wait complete. 2023-10-06T05:18:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.726+0000 7f603370d700 1 Processor -- start 2023-10-06T05:18:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.727+0000 7f603370d700 1 -- start start 2023-10-06T05:18:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.727+0000 7f603370d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f602c0aaae0 0x7f602c0c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.727+0000 7f603370d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f602c0ccb70 0x7f602c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.727+0000 7f603370d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f603370d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0b0ed0 con 0x7f602c0ccb70 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f603370d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0ae4c0 con 0x7f602c0aaae0 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f603370d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0ae740 con 0x7f602c157330 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f602c0ccb70 0x7f602c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f602c0ccb70 0x7f602c152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56962/0 (socket says 172.21.15.45:56962) 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 -- 172.21.15.45:0/1482236018 learned_addr learned my addr 172.21.15.45:0/1482236018 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6031caa700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f60314a9700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f602c0aaae0 0x7f602c0c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 -- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 msgr2=0x7f602c0adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 -- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f602c0aaae0 msgr2=0x7f602c0c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f602c0aaae0 0x7f602c0c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 -- 172.21.15.45:0/1482236018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024009a20 con 0x7f602c0ccb70 2023-10-06T05:18:26.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f6030ca8700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f602c0ccb70 0x7f602c152020 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c590 tx=0x7f601c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f60217fa700 1 -- 172.21.15.45:0/1482236018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c005cd0 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f60217fa700 1 -- 172.21.15.45:0/1482236018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c005e50 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60280093c0 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.728+0000 7f60217fa700 1 -- 172.21.15.45:0/1482236018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c007320 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.729+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c0aed00 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.729+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602c06e350 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.730+0000 7f60217fa700 1 -- 172.21.15.45:0/1482236018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f601c01b3b0 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.731+0000 7f60217fa700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6010066f30 0x7f60100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.731+0000 7f60217fa700 1 -- 172.21.15.45:0/1482236018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f601c083ed0 con 0x7f602c0ccb70 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.731+0000 7f60314a9700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6010066f30 0x7f60100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.731+0000 7f60314a9700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6010066f30 0x7f60100693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6028009160 tx=0x7f6028016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.735+0000 7f60217fa700 1 -- 172.21.15.45:0/1482236018 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f601c050e90 con 0x7f602c0ccb70 2023-10-06T05:18:26.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:26.986+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f602c0c37e0 con 0x7f602c0ccb70 2023-10-06T05:18:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.811+0000 7f60217fa700 1 -- 172.21.15.45:0/1482236018 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v51) v1 ==== 162+0+0 (secure 0 0 0) 0x7f601c017070 con 0x7f602c0ccb70 2023-10-06T05:18:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6010066f30 msgr2=0x7f60100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6010066f30 0x7f60100693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6028009160 tx=0x7f6028016000 comp rx=0 tx=0).stop 2023-10-06T05:18:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f602c0ccb70 msgr2=0x7f602c152020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f602c0ccb70 0x7f602c152020 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c590 tx=0x7f601c007e00 comp rx=0 tx=0).stop 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 shutdown_connections 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6010066f30 0x7f60100693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f602c157330 0x7f602c0adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f602c0aaae0 0x7f602c0c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 --2- 172.21.15.45:0/1482236018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f602c0ccb70 0x7f602c152020 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.813+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 >> 172.21.15.45:0/1482236018 conn(0x7f602c067e00 msgr2=0x7f602c153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.814+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 shutdown_connections 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.814+0000 7f603370d700 1 -- 172.21.15.45:0/1482236018 wait complete. 2023-10-06T05:18:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:18:27.834 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.969+0000 7fbdc19a0700 1 Processor -- start 2023-10-06T05:18:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.969+0000 7fbdc19a0700 1 -- start start 2023-10-06T05:18:27.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdc19a0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbdbc159170 0x7fbdbc159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdc19a0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc0ca6d0 con 0x7fbdbc0692d0 2023-10-06T05:18:27.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdc19a0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc15d550 con 0x7fbdbc15d040 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdc19a0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc15d810 con 0x7fbdbc159170 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdbb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbdbc0692d0 0x7fbdbc0aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60004/0 (socket says 172.21.15.45:60004) 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdbb7fe700 1 -- 172.21.15.45:0/1731622164 learned_addr learned my addr 172.21.15.45:0/1731622164 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdbaffd700 1 --2- 172.21.15.45:0/1731622164 >> v2:172.21.15.144:3300/0 conn(0x7fbdbc159170 0x7fbdbc159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.970+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894394007 0 0) 0x7fbdbc0ca6d0 con 0x7fbdbc0692d0 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda4003570 con 0x7fbdbc0692d0 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3279394605 0 0) 0x7fbda4003570 con 0x7fbdbc0692d0 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 >> v2:172.21.15.144:3300/0 conn(0x7fbdbc159170 msgr2=0x7fbdbc159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 --2- 172.21.15.45:0/1731622164 >> v2:172.21.15.144:3300/0 conn(0x7fbdbc159170 0x7fbdbc159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 >> v1:172.21.15.144:6789/0 conn(0x7fbdbc15d040 legacy=0x7fbdbc068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc15daf0 con 0x7fbdbc0692d0 2023-10-06T05:18:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fbdb40017f0 con 0x7fbdbc0692d0 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 >> v1:172.21.15.45:6789/0 conn(0x7fbdbc0692d0 legacy=0x7fbdbc0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda4003a60 0x7fbda4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbda4004810 0x7fbda4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbda400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15d810 con 0x7fbda40051c0 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400c160 con 0x7fbda4003a60 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400c320 con 0x7fbda4004810 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbb7fe700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbda400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbaffd700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda4003a60 0x7fbda4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdba7fc700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbda4004810 0x7fbda4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbb7fe700 1 -- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbda4004810 msgr2=0x7fbda4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbb7fe700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbda4004810 0x7fbda4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbb7fe700 1 -- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda4003a60 msgr2=0x7fbda4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbb7fe700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbda4003a60 0x7fbda4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbb7fe700 1 -- 172.21.15.45:0/1731622164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4003570 con 0x7fbda40051c0 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.971+0000 7fbdbb7fe700 1 -- 172.21.15.45:0/1731622164 reap_dead start 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdbb7fe700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbda400ba60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400a930 tx=0x7fbdb40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c63cf8a684882f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40070d0 con 0x7fbda40051c0 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4007230 con 0x7fbda40051c0 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdb9ffb700 1 -- 172.21.15.45:0/1731622164 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40075d0 con 0x7fbda40051c0 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1731622164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 msgr2=0x7fbda400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbda400ba60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400a930 tx=0x7fbdb40021d0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1731622164 shutdown_connections 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1731622164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbda400ba60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1731622164 >> 172.21.15.45:0/1731622164 conn(0x7fbdbc06f7c0 msgr2=0x7fbdbc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1731622164 shutdown_connections 2023-10-06T05:18:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.972+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1731622164 wait complete. 2023-10-06T05:18:27.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 Processor -- start 2023-10-06T05:18:27.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 -- start start 2023-10-06T05:18:27.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbdbc169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdbc15af10 0x7fbdbc157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdbc15b380 0x7fbdbc157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc157cd0 con 0x7fbda40051c0 2023-10-06T05:18:27.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc16db30 con 0x7fbdbc15b380 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.974+0000 7fbdc19a0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0c7ba0 con 0x7fbdbc15af10 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbdbc169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbdbc169680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56968/0 (socket says 172.21.15.45:56968) 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 -- 172.21.15.45:0/1370244953 learned_addr learned my addr 172.21.15.45:0/1370244953 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdba7fc700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdbc15af10 0x7fbdbc157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbb7fe700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdbc15b380 0x7fbdbc157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 -- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdbc15af10 msgr2=0x7fbdbc157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdbc15af10 0x7fbdbc157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 -- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdbc15b380 msgr2=0x7fbdbc157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdbc15b380 0x7fbdbc157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 -- 172.21.15.45:0/1370244953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc0c7270 con 0x7fbda40051c0 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdbaffd700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbdbc169680 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbdac011b10 tx=0x7fbdac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbda2ffd700 1 -- 172.21.15.45:0/1370244953 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00f940 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbda2ffd700 1 -- 172.21.15.45:0/1370244953 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00faa0 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc0c7550 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbda2ffd700 1 -- 172.21.15.45:0/1370244953 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00fe20 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.975+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc16ce00 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.976+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdbc063b50 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.977+0000 7fbda2ffd700 1 -- 172.21.15.45:0/1370244953 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdac014650 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.977+0000 7fbda2ffd700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbda8066f30 0x7fbda80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.977+0000 7fbda2ffd700 1 -- 172.21.15.45:0/1370244953 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fbdac018030 con 0x7fbda40051c0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.977+0000 7fbdba7fc700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbda8066f30 0x7fbda80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.978+0000 7fbdba7fc700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbda8066f30 0x7fbda80693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c00 tx=0x7fbdb0007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:27.982+0000 7fbda2ffd700 1 -- 172.21.15.45:0/1370244953 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdac052050 con 0x7fbda40051c0 2023-10-06T05:18:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.230+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdbc15ddd0 con 0x7fbda40051c0 2023-10-06T05:18:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.230+0000 7fbda2ffd700 1 -- 172.21.15.45:0/1370244953 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fbdac022090 con 0x7fbda40051c0 2023-10-06T05:18:28.232 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:28.232 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:27.811891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbda8066f30 msgr2=0x7fbda80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbda8066f30 0x7fbda80693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c00 tx=0x7fbdb0007d30 comp rx=0 tx=0).stop 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 msgr2=0x7fbdbc169680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbdbc169680 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbdac011b10 tx=0x7fbdac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 shutdown_connections 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbda8066f30 0x7fbda80693e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdbc15b380 0x7fbdbc157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40051c0 0x7fbdbc169680 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 --2- 172.21.15.45:0/1370244953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdbc15af10 0x7fbdbc157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 >> 172.21.15.45:0/1370244953 conn(0x7fbdbc06f7c0 msgr2=0x7fbdbc157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 shutdown_connections 2023-10-06T05:18:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:28.233+0000 7fbdc19a0700 1 -- 172.21.15.45:0/1370244953 wait complete. 2023-10-06T05:18:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 52 2023-10-06T05:18:28.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:09.524580+0000', 'modified': '2023-10-06T05:18:27.811891+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6342}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6342': {'gid': 6342, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6833/4169661273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 4169661273}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 4169661273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:18:28.652 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6342:up:replay 2023-10-06T05:18:29.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.792+0000 7fc0b30f0700 1 Processor -- start 2023-10-06T05:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.792+0000 7fc0b30f0700 1 -- start start 2023-10-06T05:18:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.792+0000 7fc0b30f0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac159330 0x7fc0ac06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.792+0000 7fc0b30f0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0ac0cc390 0x7fc0ac0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.792+0000 7fc0b30f0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ac0cb530 con 0x7fc0ac158f60 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b30f0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac1597a0 con 0x7fc0ac0cc390 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b30f0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0ccd50 con 0x7fc0ac159330 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b0e8c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc0ac158f60 0x7fc0ac0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60014/0 (socket says 172.21.15.45:60014) 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b0e8c700 1 -- 172.21.15.45:0/707901443 learned_addr learned my addr 172.21.15.45:0/707901443 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b168d700 1 --2- 172.21.15.45:0/707901443 >> v2:172.21.15.144:3300/0 conn(0x7fc0ac0cc390 0x7fc0ac0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0abfff700 1 --2- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac159330 0x7fc0ac06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0ab7fe700 1 -- 172.21.15.45:0/707901443 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71514269 0 0) 0x7fc0a0001110 con 0x7fc0ac158f60 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0ab7fe700 1 -- 172.21.15.45:0/707901443 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc09c003570 con 0x7fc0ac158f60 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b168d700 1 -- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac159330 msgr2=0x7fc0ac06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b168d700 1 --2- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac159330 0x7fc0ac06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b168d700 1 -- 172.21.15.45:0/707901443 >> v1:172.21.15.45:6789/0 conn(0x7fc0ac158f60 legacy=0x7fc0ac0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b168d700 1 -- 172.21.15.45:0/707901443 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac069bd0 con 0x7fc0ac0cc390 2023-10-06T05:18:29.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.793+0000 7fc0b168d700 1 --2- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac0cc390 0x7fc0ac0cc810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009e90 tx=0x7fc0a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17b5731cb49423b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:29.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0ab7fe700 1 -- 172.21.15.45:0/707901443 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a400e070 con 0x7fc0ac0cc390 2023-10-06T05:18:29.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0ab7fe700 1 -- 172.21.15.45:0/707901443 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a400bb40 con 0x7fc0ac0cc390 2023-10-06T05:18:29.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0ab7fe700 1 -- 172.21.15.45:0/707901443 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4012430 con 0x7fc0ac0cc390 2023-10-06T05:18:29.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 -- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac0cc390 msgr2=0x7fc0ac0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac0cc390 0x7fc0ac0cc810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009e90 tx=0x7fc0a4009780 comp rx=0 tx=0).stop 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 -- 172.21.15.45:0/707901443 shutdown_connections 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac0cc390 0x7fc0ac0cc810 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/707901443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac159330 0x7fc0ac06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 -- 172.21.15.45:0/707901443 >> 172.21.15.45:0/707901443 conn(0x7fc0ac0cdf50 msgr2=0x7fc0ac0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 -- 172.21.15.45:0/707901443 shutdown_connections 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.794+0000 7fc0b30f0700 1 -- 172.21.15.45:0/707901443 wait complete. 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.795+0000 7fc0b30f0700 1 Processor -- start 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.796+0000 7fc0b30f0700 1 -- start start 2023-10-06T05:18:29.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.796+0000 7fc0b30f0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac0cc390 0x7fc0ac15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b30f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac158f60 0x7fc0ac154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b30f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ac159330 0x7fc0ac0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b30f0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0b1dc0 con 0x7fc0ac159330 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b30f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac155640 con 0x7fc0ac158f60 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b30f0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac1551c0 con 0x7fc0ac0cc390 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ac159330 0x7fc0ac0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ac159330 0x7fc0ac0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56970/0 (socket says 172.21.15.45:56970) 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 -- 172.21.15.45:0/2881642685 learned_addr learned my addr 172.21.15.45:0/2881642685 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0abfff700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac158f60 0x7fc0ac154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b0e8c700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac0cc390 0x7fc0ac15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 -- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac0cc390 msgr2=0x7fc0ac15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac0cc390 0x7fc0ac15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 -- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac158f60 msgr2=0x7fc0ac154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac158f60 0x7fc0ac154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 -- 172.21.15.45:0/2881642685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a4009a20 con 0x7fc0ac159330 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0abfff700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac158f60 0x7fc0ac154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b168d700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ac159330 0x7fc0ac0b0cf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0c4f80 tx=0x7fc0a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0a8ff9700 1 -- 172.21.15.45:0/2881642685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4012e80 con 0x7fc0ac159330 2023-10-06T05:18:29.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0a8ff9700 1 -- 172.21.15.45:0/2881642685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a401e630 con 0x7fc0ac159330 2023-10-06T05:18:29.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0a8ff9700 1 -- 172.21.15.45:0/2881642685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a401e9d0 con 0x7fc0ac159330 2023-10-06T05:18:29.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac0b1520 con 0x7fc0ac159330 2023-10-06T05:18:29.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.797+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac0b1ae0 con 0x7fc0ac159330 2023-10-06T05:18:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.798+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac155410 con 0x7fc0ac159330 2023-10-06T05:18:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.799+0000 7fc0a8ff9700 1 -- 172.21.15.45:0/2881642685 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0a4024070 con 0x7fc0ac159330 2023-10-06T05:18:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.799+0000 7fc0a8ff9700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc090066f30 0x7fc0900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.799+0000 7fc0a8ff9700 1 -- 172.21.15.45:0/2881642685 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc0a40870b0 con 0x7fc0ac159330 2023-10-06T05:18:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.800+0000 7fc0b0e8c700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc090066f30 0x7fc0900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.800+0000 7fc0b0e8c700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc090066f30 0x7fc0900693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0b0410 tx=0x7fc0a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:29.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:29.804+0000 7fc0a8ff9700 1 -- 172.21.15.45:0/2881642685 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0ac155410 con 0x7fc0ac159330 2023-10-06T05:18:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.057+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0ac155410 con 0x7fc0ac159330 2023-10-06T05:18:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.058+0000 7fc0a8ff9700 1 -- 172.21.15.45:0/2881642685 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fc0ac155410 con 0x7fc0ac159330 2023-10-06T05:18:30.059 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:30.060 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:29.824576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc090066f30 msgr2=0x7fc0900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc090066f30 0x7fc0900693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0b0410 tx=0x7fc0a0009040 comp rx=0 tx=0).stop 2023-10-06T05:18:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ac159330 msgr2=0x7fc0ac0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ac159330 0x7fc0ac0b0cf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0c4f80 tx=0x7fc0a4019000 comp rx=0 tx=0).stop 2023-10-06T05:18:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 shutdown_connections 2023-10-06T05:18:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc090066f30 0x7fc0900693e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ac159330 0x7fc0ac0b0cf0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0ac0cc390 0x7fc0ac15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 --2- 172.21.15.45:0/2881642685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0ac158f60 0x7fc0ac154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 >> 172.21.15.45:0/2881642685 conn(0x7fc0ac0cdf50 msgr2=0x7fc0ac0c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 shutdown_connections 2023-10-06T05:18:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:30.060+0000 7fc0b30f0700 1 -- 172.21.15.45:0/2881642685 wait complete. 2023-10-06T05:18:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 54 2023-10-06T05:18:30.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:09.524580+0000', 'modified': '2023-10-06T05:18:29.824576+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6342}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6342': {'gid': 6342, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.45:6833/4169661273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 4169661273}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 4169661273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:18:30.478 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6342:up:rejoin 2023-10-06T05:18:31.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.617+0000 7f59e7595700 1 Processor -- start 2023-10-06T05:18:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.617+0000 7f59e7595700 1 -- start start 2023-10-06T05:18:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e7595700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59e00c93c0 0x7f59e0156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e7595700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e0157030 0x7f59e00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e7595700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e015b000 con 0x7f59e00c8ff0 2023-10-06T05:18:31.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e7595700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e00c9830 con 0x7f59e00c93c0 2023-10-06T05:18:31.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e7595700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015b2e0 con 0x7f59e0157030 2023-10-06T05:18:31.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e5b32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e0157030 0x7f59e00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e4b30700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59e00c93c0 0x7f59e0156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e4b30700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59e00c93c0 0x7f59e0156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:51704/0 (socket says 172.21.15.45:51704) 2023-10-06T05:18:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e5b32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e0157030 0x7f59e00cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:33858/0 (socket says 172.21.15.45:33858) 2023-10-06T05:18:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e4b30700 1 -- 172.21.15.45:0/2977217560 learned_addr learned my addr 172.21.15.45:0/2977217560 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.618+0000 7f59e5331700 1 --1- 172.21.15.45:0/2977217560 >> v1:172.21.15.144:6789/0 conn(0x7f59e00c8ff0 0x7f59e00caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:60478/0 (socket says 172.21.15.45:56948) 2023-10-06T05:18:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59e4b30700 1 -- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e0157030 msgr2=0x7f59e00cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59e4b30700 1 --2- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e0157030 0x7f59e00cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59e4b30700 1 -- 172.21.15.45:0/2977217560 >> v1:172.21.15.144:6789/0 conn(0x7f59e00c8ff0 legacy=0x7f59e00caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59e4b30700 1 -- 172.21.15.45:0/2977217560 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e015b5c0 con 0x7f59e00c93c0 2023-10-06T05:18:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59e5b32700 1 --2- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e0157030 0x7f59e00cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59d7fff700 1 -- 172.21.15.45:0/2977217560 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975338721 0 0) 0x7f59e015b000 con 0x7f59e00c8ff0 2023-10-06T05:18:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59e4b30700 1 --2- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 0x7f59e0156af0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59d00099b0 tx=0x7f59d0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c80f56b9af0bf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59d7fff700 1 -- 172.21.15.45:0/2977217560 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d0009ea0 con 0x7f59e00c93c0 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59d7fff700 1 -- 172.21.15.45:0/2977217560 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d000b860 con 0x7f59e00c93c0 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.619+0000 7f59d7fff700 1 -- 172.21.15.45:0/2977217560 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d000bc00 con 0x7f59e00c93c0 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 -- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 msgr2=0x7f59e0156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 --2- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 0x7f59e0156af0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59d00099b0 tx=0x7f59d0009250 comp rx=0 tx=0).stop 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 -- 172.21.15.45:0/2977217560 shutdown_connections 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 --2- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e0157030 0x7f59e00cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 --2- 172.21.15.45:0/2977217560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 0x7f59e0156af0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 -- 172.21.15.45:0/2977217560 >> 172.21.15.45:0/2977217560 conn(0x7f59e00cdf50 msgr2=0x7f59e00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 -- 172.21.15.45:0/2977217560 shutdown_connections 2023-10-06T05:18:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.620+0000 7f59e7595700 1 -- 172.21.15.45:0/2977217560 wait complete. 2023-10-06T05:18:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.621+0000 7f59e7595700 1 Processor -- start 2023-10-06T05:18:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.621+0000 7f59e7595700 1 -- start start 2023-10-06T05:18:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e7595700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00c8ff0 0x7f59e016a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e7595700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 0x7f59e0169360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e7595700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0157030 0x7f59e016c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e7595700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0169990 con 0x7f59e0157030 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e7595700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0169c70 con 0x7f59e00c93c0 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e7595700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0169f50 con 0x7f59e00c8ff0 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e5b32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0157030 0x7f59e016c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e5b32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0157030 0x7f59e016c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56984/0 (socket says 172.21.15.45:56984) 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e5b32700 1 -- 172.21.15.45:0/3746317361 learned_addr learned my addr 172.21.15.45:0/3746317361 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e5331700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00c8ff0 0x7f59e016a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.622+0000 7f59e4b30700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 0x7f59e0169360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e5b32700 1 -- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00c8ff0 msgr2=0x7f59e016a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e5b32700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00c8ff0 0x7f59e016a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e5b32700 1 -- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 msgr2=0x7f59e0169360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e5b32700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 0x7f59e0169360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e5b32700 1 -- 172.21.15.45:0/3746317361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8009a20 con 0x7f59e0157030 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e5331700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00c8ff0 0x7f59e016a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e5b32700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0157030 0x7f59e016c130 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009f20 tx=0x7f59d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59d57fa700 1 -- 172.21.15.45:0/3746317361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8009c80 con 0x7f59e0157030 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59d57fa700 1 -- 172.21.15.45:0/3746317361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d800f040 con 0x7f59e0157030 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d00094f0 con 0x7f59e0157030 2023-10-06T05:18:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59d57fa700 1 -- 172.21.15.45:0/3746317361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800ce20 con 0x7f59e0157030 2023-10-06T05:18:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.623+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e00b0d90 con 0x7f59e0157030 2023-10-06T05:18:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.624+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e016c6d0 con 0x7f59e0157030 2023-10-06T05:18:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.625+0000 7f59d57fa700 1 -- 172.21.15.45:0/3746317361 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59d8003020 con 0x7f59e0157030 2023-10-06T05:18:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.625+0000 7f59d57fa700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c4066f30 0x7f59c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.625+0000 7f59d57fa700 1 -- 172.21.15.45:0/3746317361 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f59d8085b90 con 0x7f59e0157030 2023-10-06T05:18:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.626+0000 7f59e5331700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c4066f30 0x7f59c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.626+0000 7f59e5331700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c4066f30 0x7f59c40693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f59e00cc250 tx=0x7f59dc007c20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:31.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.630+0000 7f59d57fa700 1 -- 172.21.15.45:0/3746317361 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59d8052b90 con 0x7f59e0157030 2023-10-06T05:18:31.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.878+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e004bb40 con 0x7f59e0157030 2023-10-06T05:18:31.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.879+0000 7f59d57fa700 1 -- 172.21.15.45:0/3746317361 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f59d8018090 con 0x7f59e0157030 2023-10-06T05:18:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:31.881 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:30.825755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c4066f30 msgr2=0x7f59c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c4066f30 0x7f59c40693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f59e00cc250 tx=0x7f59dc007c20 comp rx=0 tx=0).stop 2023-10-06T05:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0157030 msgr2=0x7f59e016c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0157030 0x7f59e016c130 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009f20 tx=0x7f59d800b040 comp rx=0 tx=0).stop 2023-10-06T05:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 shutdown_connections 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59c4066f30 0x7f59c40693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0157030 0x7f59e016c130 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59e00c8ff0 0x7f59e016a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 --2- 172.21.15.45:0/3746317361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59e00c93c0 0x7f59e0169360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 >> 172.21.15.45:0/3746317361 conn(0x7f59e00cdf50 msgr2=0x7f59e0151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 shutdown_connections 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:31.881+0000 7f59e7595700 1 -- 172.21.15.45:0/3746317361 wait complete. 2023-10-06T05:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 55 2023-10-06T05:18:32.185 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:18:32.184+0000 7f1482e9c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:18:32.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:09.524580+0000', 'modified': '2023-10-06T05:18:30.825755+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6342}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6342': {'gid': 6342, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.45:6833/4169661273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 4169661273}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 4169661273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:18:32.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:18:32.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:18:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.440+0000 7ff7a8231700 1 Processor -- start 2023-10-06T05:18:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.440+0000 7ff7a8231700 1 -- start start 2023-10-06T05:18:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a8231700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7a015d7c0 0x7ff7a00cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a8231700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff7a00caa70 0x7ff7a00caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a8231700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a0161f40 con 0x7ff7a00cc550 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a8231700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a0161330 con 0x7ff7a015d7c0 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a8231700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a0161610 con 0x7ff7a00caa70 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a57cc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7a015d7c0 0x7ff7a00cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a57cc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7a015d7c0 0x7ff7a00cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56988/0 (socket says 172.21.15.45:56988) 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a57cc700 1 -- 172.21.15.45:0/4285743214 learned_addr learned my addr 172.21.15.45:0/4285743214 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.441+0000 7ff7a67ce700 1 --2- 172.21.15.45:0/4285743214 >> v2:172.21.15.144:3300/0 conn(0x7ff7a00caa70 0x7ff7a00caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a57cc700 1 -- 172.21.15.45:0/4285743214 >> v2:172.21.15.144:3300/0 conn(0x7ff7a00caa70 msgr2=0x7ff7a00caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/4285743214 >> v2:172.21.15.144:3300/0 conn(0x7ff7a00caa70 0x7ff7a00caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a57cc700 1 -- 172.21.15.45:0/4285743214 >> v1:172.21.15.144:6789/0 conn(0x7ff7a00cc550 legacy=0x7ff7a015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a57cc700 1 -- 172.21.15.45:0/4285743214 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a01618f0 con 0x7ff7a015d7c0 2023-10-06T05:18:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/4285743214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d7c0 0x7ff7a00cc920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00cef90 tx=0x7ff790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43e6fb50d39b5fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a4fcb700 1 -- 172.21.15.45:0/4285743214 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79000b820 con 0x7ff7a015d7c0 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a4fcb700 1 -- 172.21.15.45:0/4285743214 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79000b980 con 0x7ff7a015d7c0 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.442+0000 7ff7a4fcb700 1 -- 172.21.15.45:0/4285743214 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79000bd20 con 0x7ff7a015d7c0 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 -- 172.21.15.45:0/4285743214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d7c0 msgr2=0x7ff7a00cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 --2- 172.21.15.45:0/4285743214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d7c0 0x7ff7a00cc920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00cef90 tx=0x7ff790009250 comp rx=0 tx=0).stop 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 -- 172.21.15.45:0/4285743214 shutdown_connections 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 --2- 172.21.15.45:0/4285743214 >> v2:172.21.15.144:3300/0 conn(0x7ff7a00caa70 0x7ff7a00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 --2- 172.21.15.45:0/4285743214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d7c0 0x7ff7a00cc920 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 -- 172.21.15.45:0/4285743214 >> 172.21.15.45:0/4285743214 conn(0x7ff7a00cdf50 msgr2=0x7ff7a00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 -- 172.21.15.45:0/4285743214 shutdown_connections 2023-10-06T05:18:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.443+0000 7ff7a8231700 1 -- 172.21.15.45:0/4285743214 wait complete. 2023-10-06T05:18:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.444+0000 7ff7a8231700 1 Processor -- start 2023-10-06T05:18:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.444+0000 7ff7a8231700 1 -- start start 2023-10-06T05:18:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a8231700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00caa70 0x7ff7a0142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a8231700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cc550 0x7ff7a0153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a8231700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7a015d7c0 0x7ff7a015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a8231700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a015bb00 con 0x7ff7a00cc550 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a8231700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0142680 con 0x7ff7a00caa70 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a8231700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a015b1f0 con 0x7ff7a015d7c0 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cc550 0x7ff7a0153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cc550 0x7ff7a0153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57002/0 (socket says 172.21.15.45:57002) 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 -- 172.21.15.45:0/2227425779 learned_addr learned my addr 172.21.15.45:0/2227425779 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 -- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7a015d7c0 msgr2=0x7ff7a015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7a015d7c0 0x7ff7a015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00caa70 0x7ff7a0142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 -- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00caa70 msgr2=0x7ff7a0142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00caa70 0x7ff7a0142140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 -- 172.21.15.45:0/2227425779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900094f0 con 0x7ff7a00cc550 2023-10-06T05:18:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.445+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cc550 0x7ff7a0153de0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff790009290 tx=0x7ff790016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.446+0000 7ff795ffb700 1 -- 172.21.15.45:0/2227425779 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790011e60 con 0x7ff7a00cc550 2023-10-06T05:18:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.446+0000 7ff795ffb700 1 -- 172.21.15.45:0/2227425779 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79001c670 con 0x7ff7a00cc550 2023-10-06T05:18:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.446+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a00c9a30 con 0x7ff7a00cc550 2023-10-06T05:18:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.446+0000 7ff795ffb700 1 -- 172.21.15.45:0/2227425779 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79001ca30 con 0x7ff7a00cc550 2023-10-06T05:18:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.446+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a015b790 con 0x7ff7a00cc550 2023-10-06T05:18:32.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.448+0000 7ff795ffb700 1 -- 172.21.15.45:0/2227425779 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff79001a450 con 0x7ff7a00cc550 2023-10-06T05:18:32.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.448+0000 7ff795ffb700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff784066f30 0x7ff7840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.448+0000 7ff795ffb700 1 -- 172.21.15.45:0/2227425779 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ff790086ec0 con 0x7ff7a00cc550 2023-10-06T05:18:32.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.448+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff784066f30 0x7ff7840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.449+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff784066f30 0x7ff7840693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0027e0 tx=0x7ff79c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.450+0000 7ff7b1545b80 1 Processor -- start 2023-10-06T05:18:32.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.450+0000 7ff7b1545b80 1 -- start start 2023-10-06T05:18:32.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c225bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225a2f40 0x557c225c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c225c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c22490310 con 0x557c225a2f40 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c224cd130 con 0x557c225c64d0 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c224caaf0 con 0x557c225bd450 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225a2f40 0x557c225c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225a2f40 0x557c225c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57012/0 (socket says 172.21.15.45:57012) 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 -- 172.21.15.45:0/65464722 learned_addr learned my addr 172.21.15.45:0/65464722 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c225bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a67ce700 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c225c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 -- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 msgr2=0x557c225bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c225bd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 -- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 msgr2=0x557c225c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c225c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 -- 172.21.15.45:0/65464722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c222cc790 con 0x557c225a2f40 2023-10-06T05:18:32.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.451+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225a2f40 0x557c225c5f90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff790085070 tx=0x7ff790084fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcbf7173c6735fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff78affd700 1 -- 172.21.15.45:0/65464722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900581e0 con 0x557c225a2f40 2023-10-06T05:18:32.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff78affd700 1 -- 172.21.15.45:0/65464722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790057a80 con 0x557c225a2f40 2023-10-06T05:18:32.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff78affd700 1 -- 172.21.15.45:0/65464722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79005acd0 con 0x557c225a2f40 2023-10-06T05:18:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 -- 172.21.15.45:0/65464722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225a2f40 msgr2=0x557c225c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225a2f40 0x557c225c5f90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff790085070 tx=0x7ff790084fd0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 -- 172.21.15.45:0/65464722 shutdown_connections 2023-10-06T05:18:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c225c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c225bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/65464722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225a2f40 0x557c225c5f90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 -- 172.21.15.45:0/65464722 >> 172.21.15.45:0/65464722 conn(0x557c225c0880 msgr2=0x557c225c0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 -- 172.21.15.45:0/65464722 shutdown_connections 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.452+0000 7ff7b1545b80 1 -- 172.21.15.45:0/65464722 wait complete. 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.453+0000 7ff7b1545b80 1 -- start start 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.453+0000 7ff7b1545b80 1 Processor -- start 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c22577f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c2259bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225c64d0 0x557c2259c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c224c68e0 con 0x557c225c64d0 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c22472980 con 0x557c225a2f40 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c22472620 con 0x557c225bd450 2023-10-06T05:18:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7b1545b80 10 client.0 _ll_register_callbacks cb 0x557c222d5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7a67ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225c64d0 0x557c2259c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7a67ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225c64d0 0x557c2259c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57020/0 (socket says 172.21.15.45:57020) 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7a67ce700 1 -- 172.21.15.45:0/3347433905 learned_addr learned my addr 172.21.15.45:0/3347433905 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c2259bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.454+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c22577f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7a67ce700 1 -- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 msgr2=0x557c2259bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7a67ce700 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c2259bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7a67ce700 1 -- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 msgr2=0x557c22577f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7a67ce700 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c22577f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7a67ce700 1 -- 172.21.15.45:0/3347433905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a015b790 con 0x557c225c64d0 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7a67ce700 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225c64d0 0x557c2259c4d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff798007dc0 tx=0x7ff798009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5819af449c4af8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff77ffff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff77ffff700 1 -- 172.21.15.45:0/3347433905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798009d20 con 0x557c225c64d0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff77ffff700 1 -- 172.21.15.45:0/3347433905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ff798002c60 con 0x557c225c64d0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c2256ad60 con 0x557c225c64d0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff77ffff700 1 -- 172.21.15.45:0/3347433905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff798015420 con 0x557c225c64d0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 10 client.6381 fetch_fsmap learned FSMap version 55 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x557c223b90a0 con 0x557c225c64d0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff77ffff700 1 -- 172.21.15.45:0/3347433905 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557c2256ad60 con 0x557c225c64d0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff77ffff700 1 -- 172.21.15.45:0/3347433905 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff79801d350 con 0x557c225c64d0 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 10 client.6381 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 10 client.6381 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 20 client.6381 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c225c0110 0x557c225bec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.455+0000 7ff7b1545b80 4 client.6381 mds_command: new command op to 6342 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.456+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 --> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c22547b30 con 0x557c225c0110 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.456+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c225c0110 0x557c225bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.456+0000 7ff77ffff700 10 client.6381 ms_handle_connect on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.456+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c225c0110 0x557c225bec70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a2c236d46fa85d server_cookie=3a3c9313d4c67ed1 in_seq=0 out_seq=0 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff77ffff700 1 -- 172.21.15.45:0/3347433905 <== mds.0 v2:172.21.15.45:6832/4169661273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x557c22547b30 con 0x557c225c0110 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff77ffff700 10 client.6381 handle_command_reply: tid=0 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 client.6381 shutdown 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225c64d0 msgr2=0x557c2259c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225c64d0 0x557c2259c4d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff798007dc0 tx=0x7ff798009850 comp rx=0 tx=0).stop 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 shutdown_connections 2023-10-06T05:18:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c225c0110 0x557c225bec70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225c64d0 0x557c2259c4d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c22577f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/3347433905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225bd450 0x557c2259bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff77ffff700 0 client.6381 ms_handle_reset on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 >> 172.21.15.45:0/3347433905 conn(0x557c225c0880 msgr2=0x557c22571fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 shutdown_connections 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 1 -- 172.21.15.45:0/3347433905 wait complete. 2023-10-06T05:18:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.457+0000 7ff7b1545b80 20 client.6381 trim_cache size 0 max 16384 2023-10-06T05:18:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.475+0000 7ff7b1545b80 1 Processor -- start 2023-10-06T05:18:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.475+0000 7ff7b1545b80 1 -- start start 2023-10-06T05:18:32.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c2256bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225c64d0 0x557c225976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2235e1e0 con 0x557c225bd450 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2259a250 con 0x557c225a2f40 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c22574b00 con 0x557c225c64d0 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225bee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57026/0 (socket says 172.21.15.45:57026) 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 -- 172.21.15.45:0/626382251 learned_addr learned my addr 172.21.15.45:0/626382251 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c2256bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a67ce700 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225c64d0 0x557c225976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 -- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225c64d0 msgr2=0x557c225976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225c64d0 0x557c225976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 -- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 msgr2=0x557c2256bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c2256bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 -- 172.21.15.45:0/626382251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c224953c0 con 0x557c225bd450 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.476+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225bee40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff7900030c0 tx=0x7ff790071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffb1cf01e689333 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff78b7fe700 1 -- 172.21.15.45:0/626382251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900581e0 con 0x557c225bd450 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff78b7fe700 1 -- 172.21.15.45:0/626382251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790053dc0 con 0x557c225bd450 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff78b7fe700 1 -- 172.21.15.45:0/626382251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79005ea20 con 0x557c225bd450 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 -- 172.21.15.45:0/626382251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 msgr2=0x557c225bee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225bee40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff7900030c0 tx=0x7ff790071000 comp rx=0 tx=0).stop 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 -- 172.21.15.45:0/626382251 shutdown_connections 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225c64d0 0x557c225976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225a2f40 0x557c2256bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/626382251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225bee40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 -- 172.21.15.45:0/626382251 >> 172.21.15.45:0/626382251 conn(0x557c225c0880 msgr2=0x557c22573fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 -- 172.21.15.45:0/626382251 shutdown_connections 2023-10-06T05:18:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.477+0000 7ff7b1545b80 1 -- 172.21.15.45:0/626382251 wait complete. 2023-10-06T05:18:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.478+0000 7ff7b1545b80 1 -- start start 2023-10-06T05:18:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.478+0000 7ff7b1545b80 1 Processor -- start 2023-10-06T05:18:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225a2f40 0x557c22596360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7b1545b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c2259ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c22596de0 con 0x557c225bd450 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2261c8d0 con 0x557c225c64d0 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7b1545b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c22623880 con 0x557c225a2f40 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7b1545b80 10 client.0 _ll_register_callbacks cb 0x557c222d5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7a57cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57042/0 (socket says 172.21.15.45:57042) 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7a57cc700 1 -- 172.21.15.45:0/536925341 learned_addr learned my addr 172.21.15.45:0/536925341 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225a2f40 0x557c22596360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.479+0000 7ff7a67ce700 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c2259ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7a57cc700 1 -- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225a2f40 msgr2=0x557c22596360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225a2f40 0x557c22596360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7a57cc700 1 -- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 msgr2=0x557c2259ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c2259ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7a57cc700 1 -- 172.21.15.45:0/536925341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c222cc790 con 0x557c225bd450 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7a57cc700 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225968a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff790057a10 tx=0x7ff790082050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1f52f85486f808 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff77ffff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff77ffff700 1 -- 172.21.15.45:0/536925341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790058180 con 0x557c225bd450 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff77ffff700 1 -- 172.21.15.45:0/536925341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ff79005acd0 con 0x557c225bd450 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff790057a80 con 0x557c225bd450 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff77ffff700 1 -- 172.21.15.45:0/536925341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff790018560 con 0x557c225bd450 2023-10-06T05:18:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 10 client.6387 fetch_fsmap learned FSMap version 55 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x557c2232d1d0 con 0x557c225bd450 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff77ffff700 1 -- 172.21.15.45:0/536925341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790057a80 con 0x557c225bd450 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff77ffff700 1 -- 172.21.15.45:0/536925341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff790067070 con 0x557c225bd450 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 10 client.6387 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 10 client.6387 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 20 client.6387 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c2259d5c0 0x557c22625a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 4 client.6387 mds_command: new command op to 6342 tid=0[{"prefix": "status"}] 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.480+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 --> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557c224698d0 con 0x557c2259d5c0 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c2259d5c0 0x557c22625a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff77ffff700 10 client.6387 ms_handle_connect on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7a5fcd700 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c2259d5c0 0x557c22625a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b363f63001e48c5 server_cookie=402ec8597e0cbd28 in_seq=0 out_seq=0 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff77ffff700 1 -- 172.21.15.45:0/536925341 <== mds.0 v2:172.21.15.45:6832/4169661273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557c224698d0 con 0x557c2259d5c0 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff77ffff700 10 client.6387 handle_command_reply: tid=0 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 client.6387 shutdown 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 msgr2=0x557c225968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225968a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff790057a10 tx=0x7ff790082050 comp rx=0 tx=0).stop 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 shutdown_connections 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x557c2259d5c0 0x557c22625a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c225c64d0 0x557c2259ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c225a2f40 0x557c22596360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff77ffff700 0 client.6387 ms_handle_reset on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 --2- 172.21.15.45:0/536925341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c225bd450 0x557c225968a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.481+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 >> 172.21.15.45:0/536925341 conn(0x557c225c0880 msgr2=0x557c22577f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 shutdown_connections 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7b1545b80 1 -- 172.21.15.45:0/536925341 wait complete. 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7b1545b80 20 client.6387 trim_cache size 0 max 16384 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:18:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6342, 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.6657900100000003, 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 55, 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 65, 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 65, 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.270987423999999 2023-10-06T05:18:32.492 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:18:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff784066f30 msgr2=0x7ff7840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff784066f30 0x7ff7840693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0027e0 tx=0x7ff79c009040 comp rx=0 tx=0).stop 2023-10-06T05:18:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cc550 msgr2=0x7ff7a0153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cc550 0x7ff7a0153de0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff790009290 tx=0x7ff790016000 comp rx=0 tx=0).stop 2023-10-06T05:18:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 shutdown_connections 2023-10-06T05:18:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff784066f30 0x7ff7840693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7a015d7c0 0x7ff7a015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00caa70 0x7ff7a0142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 --2- 172.21.15.45:0/2227425779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cc550 0x7ff7a0153de0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 >> 172.21.15.45:0/2227425779 conn(0x7ff7a00cdf50 msgr2=0x7ff7a0069450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 shutdown_connections 2023-10-06T05:18:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.482+0000 7ff7a8231700 1 -- 172.21.15.45:0/2227425779 wait complete. 2023-10-06T05:18:32.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.628+0000 7fcde968b700 1 Processor -- start 2023-10-06T05:18:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.628+0000 7fcde968b700 1 -- start start 2023-10-06T05:18:32.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde968b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde4159170 0x7fcde4159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde968b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcde40692d0 0x7fcde40aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde968b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde40ca6d0 con 0x7fcde415d040 2023-10-06T05:18:32.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde968b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde415d550 con 0x7fcde40692d0 2023-10-06T05:18:32.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde968b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415d810 con 0x7fcde4159170 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcde40692d0 0x7fcde40aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcde40692d0 0x7fcde40aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57046/0 (socket says 172.21.15.45:57046) 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde27fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcde415d040 0x7fcde4068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60028/0 (socket says 172.21.15.45:60028) 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde37fe700 1 -- 172.21.15.45:0/3218181436 learned_addr learned my addr 172.21.15.45:0/3218181436 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde2ffd700 1 --2- 172.21.15.45:0/3218181436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde4159170 0x7fcde4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde37fe700 1 -- 172.21.15.45:0/3218181436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde4159170 msgr2=0x7fcde4159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde37fe700 1 --2- 172.21.15.45:0/3218181436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde4159170 0x7fcde4159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde37fe700 1 -- 172.21.15.45:0/3218181436 >> v1:172.21.15.45:6789/0 conn(0x7fcde415d040 legacy=0x7fcde4068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde37fe700 1 -- 172.21.15.45:0/3218181436 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde415daf0 con 0x7fcde40692d0 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.629+0000 7fcde1ffb700 1 -- 172.21.15.45:0/3218181436 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309794943 0 0) 0x7fcde40ca6d0 con 0x7fcde415d040 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde37fe700 1 --2- 172.21.15.45:0/3218181436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde40692d0 0x7fcde40aea60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009f20 tx=0x7fcddc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5daa09a9164d799 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde1ffb700 1 -- 172.21.15.45:0/3218181436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00e070 con 0x7fcde40692d0 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde1ffb700 1 -- 172.21.15.45:0/3218181436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc00ba80 con 0x7fcde40692d0 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde1ffb700 1 -- 172.21.15.45:0/3218181436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0092e0 con 0x7fcde40692d0 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 -- 172.21.15.45:0/3218181436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde40692d0 msgr2=0x7fcde40aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 --2- 172.21.15.45:0/3218181436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde40692d0 0x7fcde40aea60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009f20 tx=0x7fcddc009840 comp rx=0 tx=0).stop 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 -- 172.21.15.45:0/3218181436 shutdown_connections 2023-10-06T05:18:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 --2- 172.21.15.45:0/3218181436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde40692d0 0x7fcde40aea60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 --2- 172.21.15.45:0/3218181436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde4159170 0x7fcde4159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 -- 172.21.15.45:0/3218181436 >> 172.21.15.45:0/3218181436 conn(0x7fcde406f7c0 msgr2=0x7fcde40d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 -- 172.21.15.45:0/3218181436 shutdown_connections 2023-10-06T05:18:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.630+0000 7fcde968b700 1 -- 172.21.15.45:0/3218181436 wait complete. 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.632+0000 7fcde968b700 1 Processor -- start 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.632+0000 7fcde968b700 1 -- start start 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde40692d0 0x7fcde40c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde4159170 0x7fcde415b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde415d040 0x7fcde4169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40c8d10 con 0x7fcde4159170 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415b750 con 0x7fcde40692d0 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415c730 con 0x7fcde415d040 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde4159170 0x7fcde415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde4159170 0x7fcde415b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57058/0 (socket says 172.21.15.45:57058) 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 -- 172.21.15.45:0/3048303657 learned_addr learned my addr 172.21.15.45:0/3048303657 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde2ffd700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde40692d0 0x7fcde40c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde37fe700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde415d040 0x7fcde4169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 -- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde415d040 msgr2=0x7fcde4169d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde415d040 0x7fcde4169d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 -- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde40692d0 msgr2=0x7fcde40c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde40692d0 0x7fcde40c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 -- 172.21.15.45:0/3048303657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009ae0 con 0x7fcde4159170 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde2ffd700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde40692d0 0x7fcde40c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde27fc700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde4159170 0x7fcde415b210 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c5b0 tx=0x7fcdd8001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcdcaffd700 1 -- 172.21.15.45:0/3048303657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8005a70 con 0x7fcde4159170 2023-10-06T05:18:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcdcaffd700 1 -- 172.21.15.45:0/3048303657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8005bd0 con 0x7fcde4159170 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcdcaffd700 1 -- 172.21.15.45:0/3048303657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80070d0 con 0x7fcde4159170 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde40c8400 con 0x7fcde4159170 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.633+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde40c8650 con 0x7fcde4159170 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.634+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde40c8e50 con 0x7fcde4159170 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.635+0000 7fcdcaffd700 1 -- 172.21.15.45:0/3048303657 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdd8007480 con 0x7fcde4159170 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.635+0000 7fcdcaffd700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdd0066f30 0x7fcdd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.635+0000 7fcdcaffd700 1 -- 172.21.15.45:0/3048303657 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fcdd8083dc0 con 0x7fcde4159170 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.635+0000 7fcde2ffd700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdd0066f30 0x7fcdd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.636+0000 7fcde2ffd700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdd0066f30 0x7fcdd00693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4006fd0 tx=0x7fcdd4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.640+0000 7fcdcaffd700 1 -- 172.21.15.45:0/3048303657 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcde40c8e50 con 0x7fcde4159170 2023-10-06T05:18:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.888+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcde40c8e50 con 0x7fcde4159170 2023-10-06T05:18:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.888+0000 7fcdcaffd700 1 -- 172.21.15.45:0/3048303657 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fcde40c8e50 con 0x7fcde4159170 2023-10-06T05:18:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:32.890 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:30.825755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.890+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdd0066f30 msgr2=0x7fcdd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.890+0000 7fcde968b700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdd0066f30 0x7fcdd00693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4006fd0 tx=0x7fcdd4008040 comp rx=0 tx=0).stop 2023-10-06T05:18:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.890+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde4159170 msgr2=0x7fcde415b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.890+0000 7fcde968b700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde4159170 0x7fcde415b210 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c5b0 tx=0x7fcdd8001040 comp rx=0 tx=0).stop 2023-10-06T05:18:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 shutdown_connections 2023-10-06T05:18:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdd0066f30 0x7fcdd00693e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcde415d040 0x7fcde4169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde40692d0 0x7fcde40c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 --2- 172.21.15.45:0/3048303657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcde4159170 0x7fcde415b210 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 >> 172.21.15.45:0/3048303657 conn(0x7fcde406f7c0 msgr2=0x7fcde40caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 shutdown_connections 2023-10-06T05:18:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:32.891+0000 7fcde968b700 1 -- 172.21.15.45:0/3048303657 wait complete. 2023-10-06T05:18:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 55 2023-10-06T05:18:32.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2023-10-06T05:18:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.038+0000 7fabb1d46700 1 Processor -- start 2023-10-06T05:18:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.038+0000 7fabb1d46700 1 -- start start 2023-10-06T05:18:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.039+0000 7fabb1d46700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fabac0ccb70 0x7fabac0aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.039+0000 7fabb1d46700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac0c8b70 0x7fabac0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.039+0000 7fabb1d46700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac0c3b50 con 0x7fabac157330 2023-10-06T05:18:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.039+0000 7fabb1d46700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac157700 con 0x7fabac0ccb70 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.039+0000 7fabb1d46700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0c98e0 con 0x7fabac0c8b70 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fababfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac0c8b70 0x7fabac0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabaaffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fabac0ccb70 0x7fabac0aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabab7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fabac157330 0x7fabac0ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:56954/0 (socket says 172.21.15.45:56954) 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabab7fe700 1 -- 172.21.15.45:0/3863176885 learned_addr learned my addr 172.21.15.45:0/3863176885 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabaaffd700 1 -- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac0c8b70 msgr2=0x7fabac0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac0c8b70 0x7fabac0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabaaffd700 1 -- 172.21.15.45:0/3863176885 >> v1:172.21.15.144:6789/0 conn(0x7fabac157330 legacy=0x7fabac0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabaaffd700 1 -- 172.21.15.45:0/3863176885 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0c9bc0 con 0x7fabac0ccb70 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabaa7fc700 1 -- 172.21.15.45:0/3863176885 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071061028 0 0) 0x7fabac0c3b50 con 0x7fabac157330 2023-10-06T05:18:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fababfff700 1 --2- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac0c8b70 0x7fabac0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.040+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0ccb70 0x7fabac0aaae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faba00098e0 tx=0x7faba00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6181f351af48001 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabaa7fc700 1 -- 172.21.15.45:0/3863176885 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba000b640 con 0x7fabac0ccb70 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabaa7fc700 1 -- 172.21.15.45:0/3863176885 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba000b7a0 con 0x7fabac0ccb70 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabaa7fc700 1 -- 172.21.15.45:0/3863176885 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba000bb40 con 0x7fabac0ccb70 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 -- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0ccb70 msgr2=0x7fabac0aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0ccb70 0x7fabac0aaae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faba00098e0 tx=0x7faba00091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 -- 172.21.15.45:0/3863176885 shutdown_connections 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac0c8b70 0x7fabac0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3863176885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0ccb70 0x7fabac0aaae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 -- 172.21.15.45:0/3863176885 >> 172.21.15.45:0/3863176885 conn(0x7fabac067e00 msgr2=0x7fabac0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 -- 172.21.15.45:0/3863176885 shutdown_connections 2023-10-06T05:18:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.041+0000 7fabb1d46700 1 -- 172.21.15.45:0/3863176885 wait complete. 2023-10-06T05:18:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.043+0000 7fabb1d46700 1 Processor -- start 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.043+0000 7fabb1d46700 1 -- start start 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabb1d46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0c8b70 0x7fabac151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabb1d46700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac0ccb70 0x7fabac0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabb1d46700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac157330 0x7fabac158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabb1d46700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac158d10 con 0x7fabac0ccb70 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabb1d46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0b2d90 con 0x7fabac0c8b70 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabb1d46700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1523c0 con 0x7fabac157330 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac0ccb70 0x7fabac0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac0ccb70 0x7fabac0b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57070/0 (socket says 172.21.15.45:57070) 2023-10-06T05:18:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabaaffd700 1 -- 172.21.15.45:0/3334505230 learned_addr learned my addr 172.21.15.45:0/3334505230 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fabab7fe700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0c8b70 0x7fabac151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.044+0000 7fababfff700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac157330 0x7fabac158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabaaffd700 1 -- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac157330 msgr2=0x7fabac158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac157330 0x7fabac158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabaaffd700 1 -- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0c8b70 msgr2=0x7fabac151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0c8b70 0x7fabac151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabaaffd700 1 -- 172.21.15.45:0/3334505230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4009a20 con 0x7fabac0ccb70 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac0ccb70 0x7fabac0b3220 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fabac06bac0 tx=0x7faba0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fab937fe700 1 -- 172.21.15.45:0/3334505230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0000d10 con 0x7fabac0ccb70 2023-10-06T05:18:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fab937fe700 1 -- 172.21.15.45:0/3334505230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba000bb20 con 0x7fabac0ccb70 2023-10-06T05:18:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba0009420 con 0x7fabac0ccb70 2023-10-06T05:18:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fab937fe700 1 -- 172.21.15.45:0/3334505230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0011650 con 0x7fabac0ccb70 2023-10-06T05:18:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.045+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac0b24e0 con 0x7fabac0ccb70 2023-10-06T05:18:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.046+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac0b2f00 con 0x7fabac0ccb70 2023-10-06T05:18:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.047+0000 7fab937fe700 1 -- 172.21.15.45:0/3334505230 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faba0011a00 con 0x7fabac0ccb70 2023-10-06T05:18:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.048+0000 7fab937fe700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab98066f30 0x7fab980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.048+0000 7fab937fe700 1 -- 172.21.15.45:0/3334505230 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7faba0086410 con 0x7fabac0ccb70 2023-10-06T05:18:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.048+0000 7fabab7fe700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab98066f30 0x7fab980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:33.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.048+0000 7fabab7fe700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab98066f30 0x7fab980693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fabac152610 tx=0x7fab9c007d20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:33.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.052+0000 7fab937fe700 1 -- 172.21.15.45:0/3334505230 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faba0053450 con 0x7fabac0ccb70 2023-10-06T05:18:33.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.341+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) v1 -- 0x7fabac06e350 con 0x7fabac0ccb70 2023-10-06T05:18:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.845+0000 7fab937fe700 1 -- 172.21.15.45:0/3334505230 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.6342 v56) v1 ==== 112+0+0 (secure 0 0 0) 0x7faba0011cb0 con 0x7fabac0ccb70 2023-10-06T05:18:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab98066f30 msgr2=0x7fab980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab98066f30 0x7fab980693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fabac152610 tx=0x7fab9c007d20 comp rx=0 tx=0).stop 2023-10-06T05:18:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac0ccb70 msgr2=0x7fabac0b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac0ccb70 0x7fabac0b3220 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fabac06bac0 tx=0x7faba0017000 comp rx=0 tx=0).stop 2023-10-06T05:18:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 shutdown_connections 2023-10-06T05:18:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab98066f30 0x7fab980693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabac157330 0x7fabac158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabac0c8b70 0x7fabac151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 --2- 172.21.15.45:0/3334505230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac0ccb70 0x7fabac0b3220 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 >> 172.21.15.45:0/3334505230 conn(0x7fabac067e00 msgr2=0x7fabac0c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 shutdown_connections 2023-10-06T05:18:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:33.848+0000 7fabb1d46700 1 -- 172.21.15.45:0/3334505230 wait complete. 2023-10-06T05:18:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:freezing mds.6342 2023-10-06T05:18:33.865 DEBUG:teuthology.orchestra.run.smithi045:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2023-10-06T05:18:33.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-10-06T05:18:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.004+0000 7efdd28ab700 1 Processor -- start 2023-10-06T05:18:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.004+0000 7efdd28ab700 1 -- start start 2023-10-06T05:18:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.004+0000 7efdd28ab700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efdcc153840 0x7efdcc15a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdd28ab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efdcc15cd70 0x7efdcc0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdd28ab700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdcc0ce770 con 0x7efdcc0cd060 2023-10-06T05:18:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdd28ab700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc153c10 con 0x7efdcc153840 2023-10-06T05:18:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdd28ab700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc0cd450 con 0x7efdcc15cd70 2023-10-06T05:18:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efdcc153840 0x7efdcc15a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efdcc153840 0x7efdcc15a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57078/0 (socket says 172.21.15.45:57078) 2023-10-06T05:18:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 -- 172.21.15.45:0/3917417374 learned_addr learned my addr 172.21.15.45:0/3917417374 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdd0e48700 1 --1- 172.21.15.45:0/3917417374 >> v1:172.21.15.45:6789/0 conn(0x7efdcc0cd060 0x7efdcc068bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60040/0 (socket says 172.21.15.45:60040) 2023-10-06T05:18:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/3917417374 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15cd70 0x7efdcc0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 -- 172.21.15.45:0/3917417374 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15cd70 msgr2=0x7efdcc0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 --2- 172.21.15.45:0/3917417374 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15cd70 0x7efdcc0ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 -- 172.21.15.45:0/3917417374 >> v1:172.21.15.45:6789/0 conn(0x7efdcc0cd060 legacy=0x7efdcc068bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 -- 172.21.15.45:0/3917417374 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc15d650 con 0x7efdcc153840 2023-10-06T05:18:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcaffd700 1 -- 172.21.15.45:0/3917417374 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129278706 0 0) 0x7efdcc0ce770 con 0x7efdcc0cd060 2023-10-06T05:18:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.005+0000 7efdcbfff700 1 --2- 172.21.15.45:0/3917417374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 0x7efdcc15a3d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0c70a0 tx=0x7efdbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b75651c02acebe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdcaffd700 1 -- 172.21.15.45:0/3917417374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc009e70 con 0x7efdcc153840 2023-10-06T05:18:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdcaffd700 1 -- 172.21.15.45:0/3917417374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc00b8f0 con 0x7efdcc153840 2023-10-06T05:18:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdcaffd700 1 -- 172.21.15.45:0/3917417374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00bc90 con 0x7efdcc153840 2023-10-06T05:18:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 -- 172.21.15.45:0/3917417374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 msgr2=0x7efdcc15a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 --2- 172.21.15.45:0/3917417374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 0x7efdcc15a3d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0c70a0 tx=0x7efdbc009250 comp rx=0 tx=0).stop 2023-10-06T05:18:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 -- 172.21.15.45:0/3917417374 shutdown_connections 2023-10-06T05:18:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 --2- 172.21.15.45:0/3917417374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 0x7efdcc15a3d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 --2- 172.21.15.45:0/3917417374 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15cd70 0x7efdcc0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 -- 172.21.15.45:0/3917417374 >> 172.21.15.45:0/3917417374 conn(0x7efdcc06ed50 msgr2=0x7efdcc06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 -- 172.21.15.45:0/3917417374 shutdown_connections 2023-10-06T05:18:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.006+0000 7efdd28ab700 1 -- 172.21.15.45:0/3917417374 wait complete. 2023-10-06T05:18:34.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.008+0000 7efdd28ab700 1 Processor -- start 2023-10-06T05:18:34.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.008+0000 7efdd28ab700 1 -- start start 2023-10-06T05:18:34.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd28ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc0cd060 0x7efdcc0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd28ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 0x7efdcc0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd28ab700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15cd70 0x7efdcc0c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd28ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc169f60 con 0x7efdcc153840 2023-10-06T05:18:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd28ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc16a240 con 0x7efdcc0cd060 2023-10-06T05:18:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd28ab700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc16a520 con 0x7efdcc15cd70 2023-10-06T05:18:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 0x7efdcc0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 0x7efdcc0c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57086/0 (socket says 172.21.15.45:57086) 2023-10-06T05:18:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 -- 172.21.15.45:0/2166485454 learned_addr learned my addr 172.21.15.45:0/2166485454 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcbfff700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc0cd060 0x7efdcc0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd0e48700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15cd70 0x7efdcc0c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 -- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15cd70 msgr2=0x7efdcc0c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15cd70 0x7efdcc0c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 -- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc0cd060 msgr2=0x7efdcc0c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc0cd060 0x7efdcc0c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 -- 172.21.15.45:0/2166485454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0094f0 con 0x7efdcc153840 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdcc153840 0x7efdcc0c5eb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efdc000e930 tx=0x7efdc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdb3fff700 1 -- 172.21.15.45:0/2166485454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0013680 con 0x7efdcc153840 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdb3fff700 1 -- 172.21.15.45:0/2166485454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc00137e0 con 0x7efdcc153840 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.009+0000 7efdd28ab700 1 -- 172.21.15.45:0/2166485454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc16a800 con 0x7efdcc153840 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.010+0000 7efdb3fff700 1 -- 172.21.15.45:0/2166485454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0013b80 con 0x7efdcc153840 2023-10-06T05:18:34.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.010+0000 7efdd28ab700 1 -- 172.21.15.45:0/2166485454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc169d10 con 0x7efdcc153840 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.011+0000 7efdb3fff700 1 -- 172.21.15.45:0/2166485454 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdc000c3e0 con 0x7efdcc153840 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.012+0000 7efdb3fff700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdb8066f30 0x7efdb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.012+0000 7efdb3fff700 1 -- 172.21.15.45:0/2166485454 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7efdc0084950 con 0x7efdcc153840 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.012+0000 7efdcbfff700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdb8066f30 0x7efdb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.013+0000 7efdcbfff700 1 --2- 172.21.15.45:0/2166485454 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdb8066f30 0x7efdb80693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0c70a0 tx=0x7efdbc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.015+0000 7efddbbbfb80 1 Processor -- start 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.015+0000 7efddbbbfb80 1 -- start start 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fde8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 0x55ff96fe79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96eb13c0 con 0x55ff96fc3ff0 2023-10-06T05:18:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96eee1e0 con 0x55ff96fde500 2023-10-06T05:18:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff96eebba0 con 0x55ff96fe7580 2023-10-06T05:18:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57100/0 (socket says 172.21.15.45:57100) 2023-10-06T05:18:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 -- 172.21.15.45:0/89355721 learned_addr learned my addr 172.21.15.45:0/89355721 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcbfff700 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fde8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdd0e48700 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 0x55ff96fe79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 -- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 msgr2=0x55ff96fe79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 0x55ff96fe79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 -- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 msgr2=0x55ff96fde8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fde8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.017+0000 7efdcb7fe700 1 -- 172.21.15.45:0/89355721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff96ced840 con 0x55ff96fc3ff0 2023-10-06T05:18:34.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.018+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe7040 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7efdc0082b00 tx=0x7efdc0082a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1c9e457b46c1d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.018+0000 7efdb17fa700 1 -- 172.21.15.45:0/89355721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0055590 con 0x55ff96fc3ff0 2023-10-06T05:18:34.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.018+0000 7efdb17fa700 1 -- 172.21.15.45:0/89355721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc0058760 con 0x55ff96fc3ff0 2023-10-06T05:18:34.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.018+0000 7efdb17fa700 1 -- 172.21.15.45:0/89355721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc005c4b0 con 0x55ff96fc3ff0 2023-10-06T05:18:34.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 -- 172.21.15.45:0/89355721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fc3ff0 msgr2=0x55ff96fe7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe7040 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7efdc0082b00 tx=0x7efdc0082a60 comp rx=0 tx=0).stop 2023-10-06T05:18:34.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 -- 172.21.15.45:0/89355721 shutdown_connections 2023-10-06T05:18:34.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 0x55ff96fe79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fde8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/89355721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe7040 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 -- 172.21.15.45:0/89355721 >> 172.21.15.45:0/89355721 conn(0x55ff96fe1930 msgr2=0x55ff96fe1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 -- 172.21.15.45:0/89355721 shutdown_connections 2023-10-06T05:18:34.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.019+0000 7efddbbbfb80 1 -- 172.21.15.45:0/89355721 wait complete. 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.020+0000 7efddbbbfb80 1 -- start start 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.020+0000 7efddbbbfb80 1 Processor -- start 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.021+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.021+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fb7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.021+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96fb8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.021+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96ee7990 con 0x55ff96fe7580 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.021+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96e93a30 con 0x55ff96fde500 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.021+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff96e936d0 con 0x55ff96fc3ff0 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efddbbbfb80 10 client.0 _ll_register_callbacks cb 0x55ff96cf6ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96fb8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96fb8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57108/0 (socket says 172.21.15.45:57108) 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 -- 172.21.15.45:0/1145264437 learned_addr learned my addr 172.21.15.45:0/1145264437 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fb7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 -- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 msgr2=0x55ff96fb9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 -- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 msgr2=0x55ff96fb7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fb7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 -- 172.21.15.45:0/1145264437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc169d10 con 0x55ff96fe7580 2023-10-06T05:18:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdd0e48700 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96fb8310 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007440 tx=0x7efdc4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc392ba2ea046c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1145264437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc40125a0 con 0x55ff96fe7580 2023-10-06T05:18:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1145264437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7efdc400ca80 con 0x55ff96fe7580 2023-10-06T05:18:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.022+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff96f8be10 con 0x55ff96fe7580 2023-10-06T05:18:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1145264437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdc400ce50 con 0x55ff96fe7580 2023-10-06T05:18:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 10 client.6414 fetch_fsmap learned FSMap version 56 2023-10-06T05:18:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55ff96dda150 con 0x55ff96fe7580 2023-10-06T05:18:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1145264437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff96f8be10 con 0x55ff96fe7580 2023-10-06T05:18:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1145264437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efdc4012700 con 0x55ff96fe7580 2023-10-06T05:18:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 10 client.6414 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T05:18:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-10-06T05:18:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 20 client.6414 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:34.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff96fb8850 0x55ff96fe0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 4 client.6414 mds_command: new command op to 6342 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:18:34.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 --> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff96f68be0 con 0x55ff96fb8850 2023-10-06T05:18:34.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff96fb8850 0x55ff96fe0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.023+0000 7efdaa7fc700 10 client.6414 ms_handle_connect on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff96fb8850 0x55ff96fe0930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247afb967a4b65b9 server_cookie=b097938e1c551360 in_seq=0 out_seq=0 2023-10-06T05:18:34.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1145264437 <== mds.0 v2:172.21.15.45:6832/4169661273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ff96f68be0 con 0x55ff96fb8850 2023-10-06T05:18:34.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efdaa7fc700 10 client.6414 handle_command_reply: tid=0 2023-10-06T05:18:34.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 client.6414 shutdown 2023-10-06T05:18:34.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 msgr2=0x55ff96fb8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96fb8310 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007440 tx=0x7efdc4009040 comp rx=0 tx=0).stop 2023-10-06T05:18:34.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 shutdown_connections 2023-10-06T05:18:34.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff96fb8850 0x55ff96fe0930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96fb8310 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efdaa7fc700 0 client.6414 ms_handle_reset on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/1145264437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fb7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 >> 172.21.15.45:0/1145264437 conn(0x55ff96fe1930 msgr2=0x55ff96f93d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 shutdown_connections 2023-10-06T05:18:34.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.024+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1145264437 wait complete. 2023-10-06T05:18:34.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.025+0000 7efddbbbfb80 20 client.6414 trim_cache size 0 max 16384 2023-10-06T05:18:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.054+0000 7efddbbbfb80 1 Processor -- start 2023-10-06T05:18:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.055+0000 7efddbbbfb80 1 -- start start 2023-10-06T05:18:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fbeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96f8d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96d7f290 con 0x55ff96fe7580 2023-10-06T05:18:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96f99ef0 con 0x55ff96fde500 2023-10-06T05:18:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff96fe6f00 con 0x55ff96fc3ff0 2023-10-06T05:18:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efdd0e48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96f8d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efdd0e48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96f8d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57116/0 (socket says 172.21.15.45:57116) 2023-10-06T05:18:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efdcbfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efdd0e48700 1 -- 172.21.15.45:0/2399706538 learned_addr learned my addr 172.21.15.45:0/2399706538 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.056+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fbeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdd0e48700 1 -- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 msgr2=0x55ff96fb7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdd0e48700 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdd0e48700 1 -- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 msgr2=0x55ff96fbeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdd0e48700 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fbeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdd0e48700 1 -- 172.21.15.45:0/2399706538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff96fde1a0 con 0x55ff96fe7580 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fbeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdd0e48700 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96f8d920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7efdc4012fd0 tx=0x7efdc4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb43f94aca3af27e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdb1ffb700 1 -- 172.21.15.45:0/2399706538 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4004ce0 con 0x55ff96fe7580 2023-10-06T05:18:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdb1ffb700 1 -- 172.21.15.45:0/2399706538 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc4012b70 con 0x55ff96fe7580 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.057+0000 7efdb1ffb700 1 -- 172.21.15.45:0/2399706538 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc40127f0 con 0x55ff96fe7580 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 -- 172.21.15.45:0/2399706538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 msgr2=0x55ff96f8d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96f8d920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7efdc4012fd0 tx=0x7efdc4014000 comp rx=0 tx=0).stop 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 -- 172.21.15.45:0/2399706538 shutdown_connections 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fe7580 0x55ff96f8d920 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fc3ff0 0x55ff96fb7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/2399706538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fde500 0x55ff96fbeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 -- 172.21.15.45:0/2399706538 >> 172.21.15.45:0/2399706538 conn(0x55ff96fe1930 msgr2=0x55ff96f93c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 -- 172.21.15.45:0/2399706538 shutdown_connections 2023-10-06T05:18:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.058+0000 7efddbbbfb80 1 -- 172.21.15.45:0/2399706538 wait complete. 2023-10-06T05:18:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.059+0000 7efddbbbfb80 1 -- start start 2023-10-06T05:18:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.059+0000 7efddbbbfb80 1 Processor -- start 2023-10-06T05:18:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fde500 0x55ff96fe1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efddbbbfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 0x55ff96fb1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96fb2e20 con 0x55ff96fde500 2023-10-06T05:18:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff96fe5c10 con 0x55ff96fc3ff0 2023-10-06T05:18:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efddbbbfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff96fb1650 con 0x55ff96fe7580 2023-10-06T05:18:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efddbbbfb80 10 client.0 _ll_register_callbacks cb 0x55ff96cf6ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fde500 0x55ff96fe1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fde500 0x55ff96fe1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57118/0 (socket says 172.21.15.45:57118) 2023-10-06T05:18:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 -- 172.21.15.45:0/1403546477 learned_addr learned my addr 172.21.15.45:0/1403546477 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdd0e48700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 0x55ff96fb1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 msgr2=0x55ff96fb1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff96fe7580 0x55ff96fb1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fc3ff0 msgr2=0x55ff96fe0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcb7fe700 1 -- 172.21.15.45:0/1403546477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff96ced840 con 0x55ff96fde500 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.061+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff96fc3ff0 0x55ff96fe0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdcb7fe700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff96fde500 0x55ff96fe1120 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7efdc0055dc0 tx=0x7efdc006c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f715babd8e151 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1403546477 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0055510 con 0x55ff96fde500 2023-10-06T05:18:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1403546477 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7efdc0054a00 con 0x55ff96fde500 2023-10-06T05:18:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1403546477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdc0058760 con 0x55ff96fde500 2023-10-06T05:18:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1403546477 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdc005c020 con 0x55ff96fde500 2023-10-06T05:18:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 10 client.6420 fetch_fsmap learned FSMap version 56 2023-10-06T05:18:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1403546477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55ff96dda150 con 0x55ff96fde500 2023-10-06T05:18:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1403546477 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc0058760 con 0x55ff96fde500 2023-10-06T05:18:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdaa7fc700 1 -- 172.21.15.45:0/1403546477 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efdc006bb30 con 0x55ff96fde500 2023-10-06T05:18:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 10 client.6420 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 10 client.6420 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 20 client.6420 populate_metadata read hostname 'smithi045' 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 4 client.6420 mds_command: new command op to 6342 tid=0[{"prefix": "flush journal"}] 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efddbbbfb80 1 -- 172.21.15.45:0/1403546477 --> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff96e8a980 con 0x55ff9702b210 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.062+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.063+0000 7efdaa7fc700 10 client.6420 ms_handle_connect on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.063+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef7191f0549da07 server_cookie=ca6f4495f445f57f in_seq=0 out_seq=0 2023-10-06T05:18:34.076 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:18:34.065+0000 7f147f695700 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x555e3d401400] 2023-10-06T05:18:34.076 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:18:34.065+0000 7f147f695700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x555e3d401400] 2023-10-06T05:18:34.077 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f147f695700 time 2023-10-06T05:18:34.066858+0000 2023-10-06T05:18:34.077 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:18:34.077 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.077 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f148dec0afd] 2023-10-06T05:18:34.077 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.077 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.077 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.078 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: clone() 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:18:34.068+0000 7f147f695700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f147f695700 time 2023-10-06T05:18:34.066858+0000 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:18:34.079 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.080 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.080 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f148dec0afd] 2023-10-06T05:18:34.080 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.080 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.080 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.080 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.080 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.081 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: clone() 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr:*** Caught signal (Aborted) ** 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f147f695700 thread_name:MR_Finisher 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.082 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f148ceb8cf0] 2023-10-06T05:18:34.083 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:18:34.083 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:18:34.083 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f148dec0bbd] 2023-10-06T05:18:34.083 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.083 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.083 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.083 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.084 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.084 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.084 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.084 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.084 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.084 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.084 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 19: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 20: clone() 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:18:34.069+0000 7f147f695700 -1 *** Caught signal (Aborted) ** 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f147f695700 thread_name:MR_Finisher 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f148ceb8cf0] 2023-10-06T05:18:34.085 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f148dec0bbd] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.086 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: 19: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: 20: clone() 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: -197> 2023-10-06T05:18:32.184+0000 7f1482e9c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: -6> 2023-10-06T05:18:34.065+0000 7f147f695700 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x555e3d401400] 2023-10-06T05:18:34.087 INFO:tasks.ceph.mds.a.smithi045.stderr: -4> 2023-10-06T05:18:34.065+0000 7f147f695700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x555e3d401400] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: -1> 2023-10-06T05:18:34.068+0000 7f147f695700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f147f695700 time 2023-10-06T05:18:34.066858+0000 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f148dec0afd] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.088 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: clone() 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 0> 2023-10-06T05:18:34.069+0000 7f147f695700 -1 *** Caught signal (Aborted) ** 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f147f695700 thread_name:MR_Finisher 2023-10-06T05:18:34.089 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f148ceb8cf0] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f148dec0bbd] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.090 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 19: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 20: clone() 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T05:18:34.091 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.092 INFO:teuthology.orchestra.run.smithi045.stdout:{"mds_beacon_grace":"15.000000"} 2023-10-06T05:18:34.092 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-10-06T05:18:34.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:34.093 INFO:tasks.ceph.mds.a.smithi045.stderr: -197> 2023-10-06T05:18:32.184+0000 7f1482e9c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:18:34.093 INFO:tasks.ceph.mds.a.smithi045.stderr: -6> 2023-10-06T05:18:34.065+0000 7f147f695700 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x555e3d401400] 2023-10-06T05:18:34.093 INFO:tasks.ceph.mds.a.smithi045.stderr: -4> 2023-10-06T05:18:34.065+0000 7f147f695700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x555e3d401400] 2023-10-06T05:18:34.093 INFO:tasks.ceph.mds.a.smithi045.stderr: -1> 2023-10-06T05:18:34.068+0000 7f147f695700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f147f695700 time 2023-10-06T05:18:34.066858+0000 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f148dec0afd] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.094 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: clone() 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 0> 2023-10-06T05:18:34.069+0000 7f147f695700 -1 *** Caught signal (Aborted) ** 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f147f695700 thread_name:MR_Finisher 2023-10-06T05:18:34.095 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f148ceb8cf0] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f148dec0bbd] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x555e3c55c402] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: ceph-mds(+0x448282) [0x555e3c574282] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (CDir::_omap_commit(int)+0x2da) [0x555e3c5749ea] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x21e) [0x555e3c575dbe] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xc8f) [0x555e3c6d32df] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDLog::try_expire(LogSegment*, int)+0x6a) [0x555e3c65233a] 2023-10-06T05:18:34.096 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDLog::trim_all()+0x764) [0x555e3c653c54] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (C_Flush_Journal::trim_mdlog()+0x154) [0x555e3c320a64] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (Context::complete(int)+0xd) [0x555e3c2d988d] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (C_IO_Wrapper::finish(int)+0x12) [0x555e3c63a082] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (MDSContext::complete(int)+0x5f) [0x555e3c63919f] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: (MDSIOContextBase::complete(int)+0x534) [0x555e3c639934] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: (Finisher::finisher_thread_entry()+0x18d) [0x7f148df5f82d] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 19: /lib64/libpthread.so.0(+0x81ca) [0x7f148ceae1ca] 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 20: clone() 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T05:18:34.097 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:18:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.094+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:18:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.094+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:18:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.094+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:18:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.094+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2023-10-06T05:18:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.095+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.095+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:18:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.095+0000 7efdaa7fc700 1 client.6420 ms_handle_refused on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.229+0000 7fecb3bf5700 1 Processor -- start 2023-10-06T05:18:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.229+0000 7fecb3bf5700 1 -- start start 2023-10-06T05:18:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb3bf5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fecac0c8ff0 0x7fecac0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb3bf5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fecac0caad0 0x7fecac0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb3bf5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac156af0 0x7fecac156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb3bf5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecac15b000 con 0x7fecac0c8ff0 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb3bf5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecac15b2e0 con 0x7fecac0caad0 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb3bf5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac15b5c0 con 0x7fecac156af0 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb1991700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fecac0c8ff0 0x7fecac0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb1991700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fecac0c8ff0 0x7fecac0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57124/0 (socket says 172.21.15.45:57124) 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.230+0000 7fecb1991700 1 -- 172.21.15.45:0/3804928473 learned_addr learned my addr 172.21.15.45:0/3804928473 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb1190700 1 --2- 172.21.15.45:0/3804928473 >> v2:172.21.15.144:3300/0 conn(0x7fecac0caad0 0x7fecac0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb2192700 1 --2- 172.21.15.45:0/3804928473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac156af0 0x7fecac156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb1991700 1 -- 172.21.15.45:0/3804928473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac156af0 msgr2=0x7fecac156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb1991700 1 --2- 172.21.15.45:0/3804928473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac156af0 0x7fecac156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb1991700 1 -- 172.21.15.45:0/3804928473 >> v2:172.21.15.144:3300/0 conn(0x7fecac0caad0 msgr2=0x7fecac0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb1991700 1 --2- 172.21.15.45:0/3804928473 >> v2:172.21.15.144:3300/0 conn(0x7fecac0caad0 0x7fecac0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb1991700 1 -- 172.21.15.45:0/3804928473 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac15b8a0 con 0x7fecac0c8ff0 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb1991700 1 --2- 172.21.15.45:0/3804928473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac0c8ff0 0x7fecac0c93c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fecac06ed80 tx=0x7feca0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ff1204f7f61869 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb098f700 1 -- 172.21.15.45:0/3804928473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca000b860 con 0x7fecac0c8ff0 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb098f700 1 -- 172.21.15.45:0/3804928473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca000b9c0 con 0x7fecac0c8ff0 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.231+0000 7fecb098f700 1 -- 172.21.15.45:0/3804928473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca000bd60 con 0x7fecac0c8ff0 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 -- 172.21.15.45:0/3804928473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac0c8ff0 msgr2=0x7fecac0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/3804928473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac0c8ff0 0x7fecac0c93c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fecac06ed80 tx=0x7feca0009250 comp rx=0 tx=0).stop 2023-10-06T05:18:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 -- 172.21.15.45:0/3804928473 shutdown_connections 2023-10-06T05:18:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/3804928473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac156af0 0x7fecac156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/3804928473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac0c8ff0 0x7fecac0c93c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/3804928473 >> v2:172.21.15.144:3300/0 conn(0x7fecac0caad0 0x7fecac0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 -- 172.21.15.45:0/3804928473 >> 172.21.15.45:0/3804928473 conn(0x7fecac0cdf50 msgr2=0x7fecac0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 -- 172.21.15.45:0/3804928473 shutdown_connections 2023-10-06T05:18:34.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.232+0000 7fecb3bf5700 1 -- 172.21.15.45:0/3804928473 wait complete. 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.233+0000 7fecb3bf5700 1 Processor -- start 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.233+0000 7fecb3bf5700 1 -- start start 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb3bf5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecac0c8ff0 0x7fecac149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb3bf5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac0caad0 0x7fecac0c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb3bf5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac156af0 0x7fecac0c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb3bf5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac14a3a0 con 0x7fecac156af0 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb3bf5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0b2310 con 0x7fecac0c8ff0 2023-10-06T05:18:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb3bf5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac0b25f0 con 0x7fecac0caad0 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac156af0 0x7fecac0c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac156af0 0x7fecac0c3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57134/0 (socket says 172.21.15.45:57134) 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 -- 172.21.15.45:0/984754920 learned_addr learned my addr 172.21.15.45:0/984754920 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb1991700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecac0c8ff0 0x7fecac149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb1190700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac0caad0 0x7fecac0c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 -- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac0caad0 msgr2=0x7fecac0c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac0caad0 0x7fecac0c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 -- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecac0c8ff0 msgr2=0x7fecac149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecac0c8ff0 0x7fecac149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb2192700 1 -- 172.21.15.45:0/984754920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca00094f0 con 0x7fecac156af0 2023-10-06T05:18:34.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb1991700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecac0c8ff0 0x7fecac149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.234+0000 7fecb1190700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac0caad0 0x7fecac0c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.235+0000 7fecb2192700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac156af0 0x7fecac0c3230 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feca4007f20 tx=0x7feca4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.235+0000 7fec9dffb700 1 -- 172.21.15.45:0/984754920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca400eca0 con 0x7fecac156af0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.235+0000 7fec9dffb700 1 -- 172.21.15.45:0/984754920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca400ee20 con 0x7fecac156af0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.235+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac0b28d0 con 0x7fecac156af0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.235+0000 7fec9dffb700 1 -- 172.21.15.45:0/984754920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4014670 con 0x7fecac156af0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.235+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac0b2e90 con 0x7fecac156af0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.236+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac0b30b0 con 0x7fecac156af0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.236+0000 7fec9dffb700 1 -- 172.21.15.45:0/984754920 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feca40149e0 con 0x7fecac156af0 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.237+0000 7fec9dffb700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec90066f30 0x7fec900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.237+0000 7fec9dffb700 1 -- 172.21.15.45:0/984754920 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7feca4084f30 con 0x7fecac156af0 2023-10-06T05:18:34.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.237+0000 7fecb1991700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec90066f30 0x7fec900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.238+0000 7fecb1991700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec90066f30 0x7fec900693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fecac06ed80 tx=0x7feca0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.241+0000 7fec9dffb700 1 -- 172.21.15.45:0/984754920 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feca4051eb0 con 0x7fecac156af0 2023-10-06T05:18:34.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.296+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.296+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:18:34.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.296+0000 7efdaa7fc700 1 client.6420 ms_handle_refused on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.404 INFO:tasks.ceph.mds.a.smithi045.stderr:daemon-helper: command crashed with signal 6 2023-10-06T05:18:34.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.490+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecac04bb40 con 0x7fecac156af0 2023-10-06T05:18:34.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.490+0000 7fec9dffb700 1 -- 172.21.15.45:0/984754920 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4603 (secure 0 0 0) 0x7feca40554d0 con 0x7fecac156af0 2023-10-06T05:18:34.492 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:34.492 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:33.839677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec90066f30 msgr2=0x7fec900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec90066f30 0x7fec900693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fecac06ed80 tx=0x7feca0017000 comp rx=0 tx=0).stop 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac156af0 msgr2=0x7fecac0c3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac156af0 0x7fecac0c3230 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feca4007f20 tx=0x7feca4009860 comp rx=0 tx=0).stop 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 shutdown_connections 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec90066f30 0x7fec900693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac156af0 0x7fecac0c3230 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecac0c8ff0 0x7fecac149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 --2- 172.21.15.45:0/984754920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecac0caad0 0x7fecac0c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 >> 172.21.15.45:0/984754920 conn(0x7fecac0cdf50 msgr2=0x7fecac14ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 shutdown_connections 2023-10-06T05:18:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.493+0000 7fecb3bf5700 1 -- 172.21.15.45:0/984754920 wait complete. 2023-10-06T05:18:34.496 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2023-10-06T05:18:34.508 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-10-06T05:18:34.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.697+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:34.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.697+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T05:18:34.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:34.697+0000 7efdaa7fc700 1 client.6420 ms_handle_refused on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:35.498+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:35.498+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T05:18:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:35.498+0000 7efdaa7fc700 1 client.6420 ms_handle_refused on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:36.854 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T05:18:36.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:36.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:37.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:37.100+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:37.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:37.100+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=15 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T05:18:37.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:37.100+0000 7efdaa7fc700 1 client.6420 ms_handle_refused on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:39.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.648+0000 7f1cb2736700 1 Processor -- start 2023-10-06T05:18:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.648+0000 7f1cb2736700 1 -- start start 2023-10-06T05:18:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb2736700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1cac1539f0 0x7f1cac15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb2736700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cac0ccfc0 0x7f1cac15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb2736700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac0ce690 con 0x7f1cac153620 2023-10-06T05:18:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb2736700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac15a7a0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb2736700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac0ce3f0 con 0x7f1cac1539f0 2023-10-06T05:18:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb0cd3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cac0ccfc0 0x7f1cac15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb0cd3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cac0ccfc0 0x7f1cac15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46912/0 (socket says 172.21.15.45:46912) 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.649+0000 7f1cb0cd3700 1 -- 172.21.15.45:0/340247340 learned_addr learned my addr 172.21.15.45:0/340247340 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1cab7fe700 1 --2- 172.21.15.45:0/340247340 >> v2:172.21.15.144:3300/0 conn(0x7f1cac1539f0 0x7f1cac15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1cb0cd3700 1 -- 172.21.15.45:0/340247340 >> v2:172.21.15.144:3300/0 conn(0x7f1cac1539f0 msgr2=0x7f1cac15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1cb0cd3700 1 --2- 172.21.15.45:0/340247340 >> v2:172.21.15.144:3300/0 conn(0x7f1cac1539f0 0x7f1cac15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1cb0cd3700 1 -- 172.21.15.45:0/340247340 >> v1:172.21.15.144:6789/0 conn(0x7f1cac153620 legacy=0x7f1cac0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1cb0cd3700 1 -- 172.21.15.45:0/340247340 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac15d510 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1cb0cd3700 1 --2- 172.21.15.45:0/340247340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac15cdc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1cac067840 tx=0x7f1ca4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a0111ea70b4f11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1caaffd700 1 -- 172.21.15.45:0/340247340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400e070 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1caaffd700 1 -- 172.21.15.45:0/340247340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca400ba10 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.650+0000 7f1caaffd700 1 -- 172.21.15.45:0/340247340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40092e0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 -- 172.21.15.45:0/340247340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 msgr2=0x7f1cac15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 --2- 172.21.15.45:0/340247340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac15cdc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1cac067840 tx=0x7f1ca4009840 comp rx=0 tx=0).stop 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 -- 172.21.15.45:0/340247340 shutdown_connections 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 --2- 172.21.15.45:0/340247340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac15cdc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 --2- 172.21.15.45:0/340247340 >> v2:172.21.15.144:3300/0 conn(0x7f1cac1539f0 0x7f1cac15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 -- 172.21.15.45:0/340247340 >> 172.21.15.45:0/340247340 conn(0x7f1cac06ed50 msgr2=0x7f1cac06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 -- 172.21.15.45:0/340247340 shutdown_connections 2023-10-06T05:18:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.651+0000 7f1cb2736700 1 -- 172.21.15.45:0/340247340 wait complete. 2023-10-06T05:18:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.652+0000 7f1cb2736700 1 Processor -- start 2023-10-06T05:18:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.652+0000 7f1cb2736700 1 -- start start 2023-10-06T05:18:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cb2736700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cb2736700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cac153620 0x7f1cac151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cb2736700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cac0c5f90 0x7f1cac0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cb2736700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0c6910 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cb2736700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b3430 con 0x7f1cac0c5f90 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cb2736700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac151580 con 0x7f1cac153620 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cabfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46922/0 (socket says 172.21.15.45:46922) 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cabfff700 1 -- 172.21.15.45:0/2981198026 learned_addr learned my addr 172.21.15.45:0/2981198026 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cab7fe700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cac153620 0x7f1cac151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.653+0000 7f1cb0cd3700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cac0c5f90 0x7f1cac0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cabfff700 1 -- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cac153620 msgr2=0x7f1cac151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cabfff700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cac153620 0x7f1cac151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cabfff700 1 -- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cac0c5f90 msgr2=0x7f1cac0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cabfff700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cac0c5f90 0x7f1cac0c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cabfff700 1 -- 172.21.15.45:0/2981198026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009ae0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cb0cd3700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cac0c5f90 0x7f1cac0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cab7fe700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cac153620 0x7f1cac151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cabfff700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac0b2ef0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c5d0 tx=0x7f1c9c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1c93fff700 1 -- 172.21.15.45:0/2981198026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c011590 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1c93fff700 1 -- 172.21.15.45:0/2981198026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c011710 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac16a6a0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1c93fff700 1 -- 172.21.15.45:0/2981198026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c011ad0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.654+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac1557a0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.655+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac088a60 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.656+0000 7f1c93fff700 1 -- 172.21.15.45:0/2981198026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c9c01c460 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.656+0000 7f1c93fff700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.656+0000 7f1c93fff700 1 -- 172.21.15.45:0/2981198026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f1c9c0846a0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.656+0000 7f1cab7fe700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.657+0000 7f1cab7fe700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c98066f30 0x7f1c980693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0afb50 tx=0x7f1ca0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.661+0000 7f1c93fff700 1 -- 172.21.15.45:0/2981198026 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c9c051620 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.911+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cac1531b0 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.911+0000 7f1c93fff700 1 -- 172.21.15.45:0/2981198026 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f1c9c054c40 con 0x7f1cac0ccfc0 2023-10-06T05:18:39.913 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:39.914 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:33.839677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c98066f30 msgr2=0x7f1c980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c98066f30 0x7f1c980693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0afb50 tx=0x7f1ca0008040 comp rx=0 tx=0).stop 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 msgr2=0x7f1cac0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac0b2ef0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c5d0 tx=0x7f1c9c007f60 comp rx=0 tx=0).stop 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 shutdown_connections 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c98066f30 0x7f1c980693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cac0c5f90 0x7f1cac0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cac0ccfc0 0x7f1cac0b2ef0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 --2- 172.21.15.45:0/2981198026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cac153620 0x7f1cac151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 >> 172.21.15.45:0/2981198026 conn(0x7f1cac06ed50 msgr2=0x7f1cac15cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 shutdown_connections 2023-10-06T05:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:39.914+0000 7f1cb2736700 1 -- 172.21.15.45:0/2981198026 wait complete. 2023-10-06T05:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2023-10-06T05:18:39.929 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-10-06T05:18:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:40.303+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:40.303+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=15 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T05:18:40.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:40.303+0000 7efdaa7fc700 1 client.6420 ms_handle_refused on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:43.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-10-06T05:18:44.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.070+0000 7fcdb8511700 1 Processor -- start 2023-10-06T05:18:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.070+0000 7fcdb8511700 1 -- start start 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.071+0000 7fcdb8511700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcdb01560a0 0x7fcdb015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.071+0000 7fcdb8511700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcdb00ccdc0 0x7fcdb00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.071+0000 7fcdb8511700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb00cd700 con 0x7fcdb0155cd0 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.071+0000 7fcdb8511700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb00ae6b0 con 0x7fcdb01560a0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.071+0000 7fcdb8511700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb00ae8d0 con 0x7fcdb00ccdc0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.071+0000 7fcdb5aac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcdb01560a0 0x7fcdb015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb5aac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcdb01560a0 0x7fcdb015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46928/0 (socket says 172.21.15.45:46928) 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb5aac700 1 -- 172.21.15.45:0/3588605426 learned_addr learned my addr 172.21.15.45:0/3588605426 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb6aae700 1 --2- 172.21.15.45:0/3588605426 >> v2:172.21.15.144:3300/0 conn(0x7fcdb00ccdc0 0x7fcdb00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb5aac700 1 -- 172.21.15.45:0/3588605426 >> v2:172.21.15.144:3300/0 conn(0x7fcdb00ccdc0 msgr2=0x7fcdb00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb5aac700 1 --2- 172.21.15.45:0/3588605426 >> v2:172.21.15.144:3300/0 conn(0x7fcdb00ccdc0 0x7fcdb00cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb5aac700 1 -- 172.21.15.45:0/3588605426 >> v1:172.21.15.144:6789/0 conn(0x7fcdb0155cd0 legacy=0x7fcdb00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb5aac700 1 -- 172.21.15.45:0/3588605426 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb00aebb0 con 0x7fcdb01560a0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb6aae700 1 --2- 172.21.15.45:0/3588605426 >> v2:172.21.15.144:3300/0 conn(0x7fcdb00ccdc0 0x7fcdb00cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb5aac700 1 --2- 172.21.15.45:0/3588605426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb015a730 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00c3dd0 tx=0x7fcda000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a234c089e1eff41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb52ab700 1 -- 172.21.15.45:0/3588605426 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000dc50 con 0x7fcdb01560a0 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb52ab700 1 -- 172.21.15.45:0/3588605426 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda000ddb0 con 0x7fcdb01560a0 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.072+0000 7fcdb52ab700 1 -- 172.21.15.45:0/3588605426 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00196d0 con 0x7fcdb01560a0 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 -- 172.21.15.45:0/3588605426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 msgr2=0x7fcdb015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 --2- 172.21.15.45:0/3588605426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb015a730 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00c3dd0 tx=0x7fcda000b4f0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 -- 172.21.15.45:0/3588605426 shutdown_connections 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 --2- 172.21.15.45:0/3588605426 >> v2:172.21.15.144:3300/0 conn(0x7fcdb00ccdc0 0x7fcdb00cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 --2- 172.21.15.45:0/3588605426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb015a730 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 -- 172.21.15.45:0/3588605426 >> 172.21.15.45:0/3588605426 conn(0x7fcdb006f090 msgr2=0x7fcdb006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 -- 172.21.15.45:0/3588605426 shutdown_connections 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.073+0000 7fcdb8511700 1 -- 172.21.15.45:0/3588605426 wait complete. 2023-10-06T05:18:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.074+0000 7fcdb8511700 1 Processor -- start 2023-10-06T05:18:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.074+0000 7fcdb8511700 1 -- start start 2023-10-06T05:18:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb8511700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdb00ccdc0 0x7fcdb00b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb8511700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdb0155cd0 0x7fcdb014f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb8511700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb00b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb8511700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00afbe0 con 0x7fcdb01560a0 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb8511700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00b0480 con 0x7fcdb0155cd0 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb8511700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00b1fa0 con 0x7fcdb00ccdc0 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb6aae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb00b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb6aae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb00b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46932/0 (socket says 172.21.15.45:46932) 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb6aae700 1 -- 172.21.15.45:0/555508137 learned_addr learned my addr 172.21.15.45:0/555508137 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb5aac700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdb0155cd0 0x7fcdb014f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb62ad700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdb00ccdc0 0x7fcdb00b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb6aae700 1 -- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdb00ccdc0 msgr2=0x7fcdb00b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb6aae700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdb00ccdc0 0x7fcdb00b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb6aae700 1 -- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdb0155cd0 msgr2=0x7fcdb014f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.075+0000 7fcdb6aae700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdb0155cd0 0x7fcdb014f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcdb6aae700 1 -- 172.21.15.45:0/555508137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda000b790 con 0x7fcdb01560a0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcdb62ad700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdb00ccdc0 0x7fcdb00b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcdb5aac700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdb0155cd0 0x7fcdb014f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcdb6aae700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb00b1500 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcda800d180 tx=0x7fcda8009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcda67fc700 1 -- 172.21.15.45:0/555508137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda800de20 con 0x7fcdb01560a0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcda67fc700 1 -- 172.21.15.45:0/555508137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda800f040 con 0x7fcdb01560a0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb00afe30 con 0x7fcdb01560a0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcda67fc700 1 -- 172.21.15.45:0/555508137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8015690 con 0x7fcdb01560a0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.076+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb00abd30 con 0x7fcdb01560a0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.077+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb006e2f0 con 0x7fcdb01560a0 2023-10-06T05:18:45.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.077+0000 7fcda67fc700 1 -- 172.21.15.45:0/555508137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcda801e3b0 con 0x7fcdb01560a0 2023-10-06T05:18:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.078+0000 7fcda67fc700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd94066f30 0x7fcd940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.078+0000 7fcda67fc700 1 -- 172.21.15.45:0/555508137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fcda8084d90 con 0x7fcdb01560a0 2023-10-06T05:18:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.078+0000 7fcdb62ad700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd94066f30 0x7fcd940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.079+0000 7fcdb62ad700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd94066f30 0x7fcd940693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0017a0 tx=0x7fcdac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.083+0000 7fcda67fc700 1 -- 172.21.15.45:0/555508137 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcda8051d10 con 0x7fcdb01560a0 2023-10-06T05:18:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.328+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdb00aee90 con 0x7fcdb01560a0 2023-10-06T05:18:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.329+0000 7fcda67fc700 1 -- 172.21.15.45:0/555508137 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fcda801a090 con 0x7fcdb01560a0 2023-10-06T05:18:45.330 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:45.331 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:33.839677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd94066f30 msgr2=0x7fcd940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd94066f30 0x7fcd940693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0017a0 tx=0x7fcdac009040 comp rx=0 tx=0).stop 2023-10-06T05:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 msgr2=0x7fcdb00b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb00b1500 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcda800d180 tx=0x7fcda8009a50 comp rx=0 tx=0).stop 2023-10-06T05:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 shutdown_connections 2023-10-06T05:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd94066f30 0x7fcd940693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb01560a0 0x7fcdb00b1500 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdb00ccdc0 0x7fcdb00b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 --2- 172.21.15.45:0/555508137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdb0155cd0 0x7fcdb014f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 >> 172.21.15.45:0/555508137 conn(0x7fcdb006f090 msgr2=0x7fcdb0068130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 shutdown_connections 2023-10-06T05:18:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:45.331+0000 7fcdb8511700 1 -- 172.21.15.45:0/555508137 wait complete. 2023-10-06T05:18:45.334 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2023-10-06T05:18:45.346 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-10-06T05:18:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:46.710+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:46.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:46.710+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-10-06T05:18:46.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:46.710+0000 7efdaa7fc700 1 client.6420 ms_handle_refused on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:50.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-10-06T05:18:50.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.487+0000 7f0b277a4700 1 Processor -- start 2023-10-06T05:18:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.487+0000 7f0b277a4700 1 -- start start 2023-10-06T05:18:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b277a4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b20155cd0 0x7f0b2015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b277a4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b200ae8d0 con 0x7f0b201560a0 2023-10-06T05:18:50.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b277a4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b200aebb0 con 0x7f0b200ccdc0 2023-10-06T05:18:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b277a4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200ae6b0 con 0x7f0b20155cd0 2023-10-06T05:18:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b24d3f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0b201560a0 0x7f0b200adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49806/0 (socket says 172.21.15.45:49806) 2023-10-06T05:18:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b24d3f700 1 -- 172.21.15.45:0/459775057 learned_addr learned my addr 172.21.15.45:0/459775057 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b25540700 1 --2- 172.21.15.45:0/459775057 >> v2:172.21.15.144:3300/0 conn(0x7f0b20155cd0 0x7f0b2015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991055782 0 0) 0x7f0b200ae8d0 con 0x7f0b201560a0 2023-10-06T05:18:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.488+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b0c003570 con 0x7f0b201560a0 2023-10-06T05:18:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532949055 0 0) 0x7f0b200aebb0 con 0x7f0b200ccdc0 2023-10-06T05:18:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b200ae8d0 con 0x7f0b200ccdc0 2023-10-06T05:18:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88295700 0 0) 0x7f0b0c003570 con 0x7f0b201560a0 2023-10-06T05:18:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 >> v2:172.21.15.144:3300/0 conn(0x7f0b20155cd0 msgr2=0x7f0b2015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 --2- 172.21.15.45:0/459775057 >> v2:172.21.15.144:3300/0 conn(0x7f0b20155cd0 0x7f0b2015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 >> v1:172.21.15.144:6789/0 conn(0x7f0b200ccdc0 legacy=0x7f0b200cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200aee90 con 0x7f0b201560a0 2023-10-06T05:18:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f0b10001470 con 0x7f0b201560a0 2023-10-06T05:18:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 >> v1:172.21.15.45:6789/0 conn(0x7f0b201560a0 legacy=0x7f0b200adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c003bf0 0x7f0b0c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b0c004810 0x7f0b0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c0051c0 0x7f0b0c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200ae6b0 con 0x7f0b0c0051c0 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c00c160 con 0x7f0b0c004810 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c00c320 con 0x7f0b0c003bf0 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25d41700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c0051c0 0x7f0b0c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b24d3f700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b0c004810 0x7f0b0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25540700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c003bf0 0x7f0b0c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25d41700 1 -- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c003bf0 msgr2=0x7f0b0c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25d41700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c003bf0 0x7f0b0c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25d41700 1 -- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b0c004810 msgr2=0x7f0b0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25d41700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b0c004810 0x7f0b0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25d41700 1 -- 172.21.15.45:0/459775057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c003570 con 0x7f0b0c0051c0 2023-10-06T05:18:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.489+0000 7f0b25d41700 1 -- 172.21.15.45:0/459775057 reap_dead start 2023-10-06T05:18:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b25d41700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c0051c0 0x7f0b0c00ba60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cd8d0 tx=0x7f0b180017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48d1b6b05d50808 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18001ab0 con 0x7f0b0c0051c0 2023-10-06T05:18:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18006ba0 con 0x7f0b0c0051c0 2023-10-06T05:18:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b17fff700 1 -- 172.21.15.45:0/459775057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800c8b0 con 0x7f0b0c0051c0 2023-10-06T05:18:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b277a4700 1 -- 172.21.15.45:0/459775057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c0051c0 msgr2=0x7f0b0c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b277a4700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c0051c0 0x7f0b0c00ba60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cd8d0 tx=0x7f0b180017a0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b277a4700 1 -- 172.21.15.45:0/459775057 shutdown_connections 2023-10-06T05:18:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b277a4700 1 --2- 172.21.15.45:0/459775057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c0051c0 0x7f0b0c00ba60 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b277a4700 1 -- 172.21.15.45:0/459775057 >> 172.21.15.45:0/459775057 conn(0x7f0b2006f090 msgr2=0x7f0b2006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b277a4700 1 -- 172.21.15.45:0/459775057 shutdown_connections 2023-10-06T05:18:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.490+0000 7f0b277a4700 1 -- 172.21.15.45:0/459775057 wait complete. 2023-10-06T05:18:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.492+0000 7f0b277a4700 1 Processor -- start 2023-10-06T05:18:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.492+0000 7f0b277a4700 1 -- start start 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b277a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c0051c0 0x7f0b20159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b277a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b2016cdc0 0x7f0b2016d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b277a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b201535b0 0x7f0b20153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b277a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2016d770 con 0x7f0b2016cdc0 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b277a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2014b620 con 0x7f0b201535b0 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b277a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b201542e0 con 0x7f0b0c0051c0 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b24d3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b2016cdc0 0x7f0b2016d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b24d3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b2016cdc0 0x7f0b2016d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51732/0 (socket says 172.21.15.45:51732) 2023-10-06T05:18:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.493+0000 7f0b24d3f700 1 -- 172.21.15.45:0/3228205353 learned_addr learned my addr 172.21.15.45:0/3228205353 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b25540700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c0051c0 0x7f0b20159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b25d41700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b201535b0 0x7f0b20153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b24d3f700 1 -- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c0051c0 msgr2=0x7f0b20159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b24d3f700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c0051c0 0x7f0b20159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b24d3f700 1 -- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b201535b0 msgr2=0x7f0b20153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b24d3f700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b201535b0 0x7f0b20153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b24d3f700 1 -- 172.21.15.45:0/3228205353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b201545c0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b24d3f700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b2016cdc0 0x7f0b2016d230 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007fd0 tx=0x7f0b20156540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b157fa700 1 -- 172.21.15.45:0/3228205353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10011590 con 0x7f0b2016cdc0 2023-10-06T05:18:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b157fa700 1 -- 172.21.15.45:0/3228205353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100116f0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b201548a0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b157fa700 1 -- 172.21.15.45:0/3228205353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10011ab0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.494+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b20153f40 con 0x7f0b2016cdc0 2023-10-06T05:18:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.495+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2014bab0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.496+0000 7f0b157fa700 1 -- 172.21.15.45:0/3228205353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b1001d460 con 0x7f0b2016cdc0 2023-10-06T05:18:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.496+0000 7f0b157fa700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b04066f30 0x7f0b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.496+0000 7f0b157fa700 1 -- 172.21.15.45:0/3228205353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f0b10084940 con 0x7f0b2016cdc0 2023-10-06T05:18:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.496+0000 7f0b25540700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b04066f30 0x7f0b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.497+0000 7f0b25540700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b04066f30 0x7f0b040693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c006f00 tx=0x7f0b1c0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.501+0000 7f0b157fa700 1 -- 172.21.15.45:0/3228205353 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b2014bab0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.750+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b2014bab0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.751+0000 7f0b157fa700 1 -- 172.21.15.45:0/3228205353 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f0b2014bab0 con 0x7f0b2016cdc0 2023-10-06T05:18:50.752 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:50.753 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:33.839677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b04066f30 msgr2=0x7f0b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b04066f30 0x7f0b040693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c006f00 tx=0x7f0b1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b2016cdc0 msgr2=0x7f0b2016d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b2016cdc0 0x7f0b2016d230 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007fd0 tx=0x7f0b20156540 comp rx=0 tx=0).stop 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 shutdown_connections 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b04066f30 0x7f0b040693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b201535b0 0x7f0b20153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c0051c0 0x7f0b20159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 --2- 172.21.15.45:0/3228205353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b2016cdc0 0x7f0b2016d230 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 >> 172.21.15.45:0/3228205353 conn(0x7f0b2006f090 msgr2=0x7f0b20154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:50.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 shutdown_connections 2023-10-06T05:18:50.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:50.753+0000 7f0b277a4700 1 -- 172.21.15.45:0/3228205353 wait complete. 2023-10-06T05:18:50.756 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2023-10-06T05:18:50.769 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-10-06T05:18:55.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.910+0000 7f4349d7b700 1 Processor -- start 2023-10-06T05:18:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.910+0000 7f4349d7b700 1 -- start start 2023-10-06T05:18:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4349d7b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4344159200 0x7f43440c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4349d7b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43441595d0 0x7f434406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4349d7b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43440c7d00 con 0x7f434406b5c0 2023-10-06T05:18:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4349d7b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43440c7780 con 0x7f43441595d0 2023-10-06T05:18:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4349d7b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43440c86d0 con 0x7f4344159200 2023-10-06T05:18:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4343fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f434406b5c0 0x7f43440ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49818/0 (socket says 172.21.15.45:49818) 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4343fff700 1 -- 172.21.15.45:0/1777863409 learned_addr learned my addr 172.21.15.45:0/1777863409 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f43437fe700 1 --2- 172.21.15.45:0/1777863409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4344159200 0x7f43440c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.911+0000 7f4342ffd700 1 --2- 172.21.15.45:0/1777863409 >> v2:172.21.15.144:3300/0 conn(0x7f43441595d0 0x7f434406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43427fc700 1 -- 172.21.15.45:0/1777863409 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228546893 0 0) 0x7f43440c7d00 con 0x7f434406b5c0 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43427fc700 1 -- 172.21.15.45:0/1777863409 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f432c003570 con 0x7f434406b5c0 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43437fe700 1 -- 172.21.15.45:0/1777863409 >> v2:172.21.15.144:3300/0 conn(0x7f43441595d0 msgr2=0x7f434406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43437fe700 1 --2- 172.21.15.45:0/1777863409 >> v2:172.21.15.144:3300/0 conn(0x7f43441595d0 0x7f434406b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43437fe700 1 -- 172.21.15.45:0/1777863409 >> v1:172.21.15.45:6789/0 conn(0x7f434406b5c0 legacy=0x7f43440ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43437fe700 1 -- 172.21.15.45:0/1777863409 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434406bb40 con 0x7f4344159200 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f4342ffd700 1 --2- 172.21.15.45:0/1777863409 >> v2:172.21.15.144:3300/0 conn(0x7f43441595d0 0x7f434406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43437fe700 1 --2- 172.21.15.45:0/1777863409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4344159200 0x7f43440c8190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f43340098e0 tx=0x7f43340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ce539a986fa4ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43427fc700 1 -- 172.21.15.45:0/1777863409 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334011630 con 0x7f4344159200 2023-10-06T05:18:55.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43427fc700 1 -- 172.21.15.45:0/1777863409 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4334011790 con 0x7f4344159200 2023-10-06T05:18:55.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.912+0000 7f43427fc700 1 -- 172.21.15.45:0/1777863409 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334011b30 con 0x7f4344159200 2023-10-06T05:18:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 -- 172.21.15.45:0/1777863409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4344159200 msgr2=0x7f43440c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 --2- 172.21.15.45:0/1777863409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4344159200 0x7f43440c8190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f43340098e0 tx=0x7f43340091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 -- 172.21.15.45:0/1777863409 shutdown_connections 2023-10-06T05:18:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 --2- 172.21.15.45:0/1777863409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4344159200 0x7f43440c8190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 --2- 172.21.15.45:0/1777863409 >> v2:172.21.15.144:3300/0 conn(0x7f43441595d0 0x7f434406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 -- 172.21.15.45:0/1777863409 >> 172.21.15.45:0/1777863409 conn(0x7f434406f090 msgr2=0x7f434406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 -- 172.21.15.45:0/1777863409 shutdown_connections 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.913+0000 7f4349d7b700 1 -- 172.21.15.45:0/1777863409 wait complete. 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.914+0000 7f4349d7b700 1 Processor -- start 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4349d7b700 1 -- start start 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4349d7b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f434406b5c0 0x7f43440b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4349d7b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4344159200 0x7f43440cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4349d7b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441595d0 0x7f43440c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4349d7b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43440b1040 con 0x7f43441595d0 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4349d7b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43440b1320 con 0x7f4344159200 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4349d7b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43440b1600 con 0x7f434406b5c0 2023-10-06T05:18:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4343fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441595d0 0x7f43440c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f4343fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441595d0 0x7f43440c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51748/0 (socket says 172.21.15.45:51748) 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4342ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4344159200 0x7f43440cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.915+0000 7f43437fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f434406b5c0 0x7f43440b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4343fff700 1 -- 172.21.15.45:0/998568074 learned_addr learned my addr 172.21.15.45:0/998568074 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4343fff700 1 -- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f434406b5c0 msgr2=0x7f43440b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4343fff700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f434406b5c0 0x7f43440b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4343fff700 1 -- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4344159200 msgr2=0x7f43440cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4343fff700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4344159200 0x7f43440cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4343fff700 1 -- 172.21.15.45:0/998568074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4338009420 con 0x7f43441595d0 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4342ffd700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4344159200 0x7f43440cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4343fff700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441595d0 0x7f43440c3e40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f433c001460 tx=0x7f433c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f432b7fe700 1 -- 172.21.15.45:0/998568074 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c00a890 con 0x7f43441595d0 2023-10-06T05:18:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f432b7fe700 1 -- 172.21.15.45:0/998568074 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c00a9f0 con 0x7f43441595d0 2023-10-06T05:18:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4334009420 con 0x7f43441595d0 2023-10-06T05:18:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f432b7fe700 1 -- 172.21.15.45:0/998568074 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c00ad70 con 0x7f43441595d0 2023-10-06T05:18:55.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.916+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43440b1bc0 con 0x7f43441595d0 2023-10-06T05:18:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.917+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43440b1de0 con 0x7f43441595d0 2023-10-06T05:18:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.918+0000 7f432b7fe700 1 -- 172.21.15.45:0/998568074 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f433c007300 con 0x7f43441595d0 2023-10-06T05:18:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.918+0000 7f432b7fe700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.918+0000 7f432b7fe700 1 -- 172.21.15.45:0/998568074 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f433c084c40 con 0x7f43441595d0 2023-10-06T05:18:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.918+0000 7f43437fe700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.919+0000 7f43437fe700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43340091e0 tx=0x7f4334017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:55.923+0000 7f432b7fe700 1 -- 172.21.15.45:0/998568074 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f433c051b60 con 0x7f43441595d0 2023-10-06T05:18:56.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.165+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f434406be20 con 0x7f43441595d0 2023-10-06T05:18:56.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.166+0000 7f432b7fe700 1 -- 172.21.15.45:0/998568074 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f433c055180 con 0x7f43441595d0 2023-10-06T05:18:56.169 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:56.169 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:50.810628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6342},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6342":{"gid":6342,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/4169661273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4169661273},{"type":"v1","addr":"172.21.15.45:6833","nonce":4169661273}]},"join_fscid":-1,"laggy_since":"2023-10-06T05:18:50.810562+0000","export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:56.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 msgr2=0x7f43300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43340091e0 tx=0x7f4334017000 comp rx=0 tx=0).stop 2023-10-06T05:18:56.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441595d0 msgr2=0x7f43440c3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441595d0 0x7f43440c3e40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f433c001460 tx=0x7f433c001ae0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 shutdown_connections 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441595d0 0x7f43440c3e40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f434406b5c0 0x7f43440b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 --2- 172.21.15.45:0/998568074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4344159200 0x7f43440cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 >> 172.21.15.45:0/998568074 conn(0x7f434406f090 msgr2=0x7f43440aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 shutdown_connections 2023-10-06T05:18:56.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.168+0000 7f4349d7b700 1 -- 172.21.15.45:0/998568074 wait complete. 2023-10-06T05:18:56.173 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 57 2023-10-06T05:18:56.186 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-10-06T05:18:56.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2023-10-06T05:18:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdaba395700 1 Processor -- start 2023-10-06T05:18:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdaba395700 1 -- start start 2023-10-06T05:18:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdaba395700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdab406ae40 0x7fdab415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdaba395700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab4067d80 con 0x7fdab406aa70 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdaba395700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab415cb70 con 0x7fdab415c6f0 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdaba395700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab40ac620 con 0x7fdab406ae40 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdab37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdab406ae40 0x7fdab415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdab37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdab406ae40 0x7fdab415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51764/0 (socket says 172.21.15.45:51764) 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdab37fe700 1 -- 172.21.15.45:0/1799765026 learned_addr learned my addr 172.21.15.45:0/1799765026 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdab37fe700 1 -- 172.21.15.45:0/1799765026 >> v1:172.21.15.144:6789/0 conn(0x7fdab415c6f0 legacy=0x7fdab40aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdab37fe700 1 -- 172.21.15.45:0/1799765026 >> v1:172.21.15.45:6789/0 conn(0x7fdab406aa70 legacy=0x7fdab40c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdab37fe700 1 -- 172.21.15.45:0/1799765026 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab40c75b0 con 0x7fdab406ae40 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.319+0000 7fdab2ffd700 1 -- 172.21.15.45:0/1799765026 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497049458 0 0) 0x7fdab415cb70 con 0x7fdab415c6f0 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdab37fe700 1 --2- 172.21.15.45:0/1799765026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab415c1b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0099b0 tx=0x7fda9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c4a5c099ada73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdab2ffd700 1 -- 172.21.15.45:0/1799765026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00b860 con 0x7fdab406ae40 2023-10-06T05:18:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdab2ffd700 1 -- 172.21.15.45:0/1799765026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c00b9c0 con 0x7fdab406ae40 2023-10-06T05:18:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdab2ffd700 1 -- 172.21.15.45:0/1799765026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00bd60 con 0x7fdab406ae40 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdaba395700 1 -- 172.21.15.45:0/1799765026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 msgr2=0x7fdab415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdaba395700 1 --2- 172.21.15.45:0/1799765026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab415c1b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0099b0 tx=0x7fda9c009250 comp rx=0 tx=0).stop 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdaba395700 1 -- 172.21.15.45:0/1799765026 shutdown_connections 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdaba395700 1 --2- 172.21.15.45:0/1799765026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab415c1b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.320+0000 7fdaba395700 1 -- 172.21.15.45:0/1799765026 >> 172.21.15.45:0/1799765026 conn(0x7fdab40cc570 msgr2=0x7fdab40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.321+0000 7fdaba395700 1 -- 172.21.15.45:0/1799765026 shutdown_connections 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.321+0000 7fdaba395700 1 -- 172.21.15.45:0/1799765026 wait complete. 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.322+0000 7fdaba395700 1 Processor -- start 2023-10-06T05:18:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.322+0000 7fdaba395700 1 -- start start 2023-10-06T05:18:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdaba395700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdab406aa70 0x7fdab4154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdaba395700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab40c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdaba395700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdab415c6f0 0x7fdab40acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdaba395700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40c3eb0 con 0x7fdab406ae40 2023-10-06T05:18:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdaba395700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40c1460 con 0x7fdab415c6f0 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdaba395700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40c3580 con 0x7fdab406aa70 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab40c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab40c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51774/0 (socket says 172.21.15.45:51774) 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 -- 172.21.15.45:0/3892207650 learned_addr learned my addr 172.21.15.45:0/3892207650 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab8932700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdab415c6f0 0x7fdab40acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab3fff700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdab406aa70 0x7fdab4154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 -- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdab406aa70 msgr2=0x7fdab4154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdab406aa70 0x7fdab4154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 -- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdab415c6f0 msgr2=0x7fdab40acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdab415c6f0 0x7fdab40acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 -- 172.21.15.45:0/3892207650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0094f0 con 0x7fdab406ae40 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.323+0000 7fdab37fe700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab40c0a10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006f50 tx=0x7fda9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaabfff700 1 -- 172.21.15.45:0/3892207650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c01f070 con 0x7fdab406ae40 2023-10-06T05:18:56.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaabfff700 1 -- 172.21.15.45:0/3892207650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c011d10 con 0x7fdab406ae40 2023-10-06T05:18:56.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab40c3860 con 0x7fdab406ae40 2023-10-06T05:18:56.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaabfff700 1 -- 172.21.15.45:0/3892207650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c025670 con 0x7fdab406ae40 2023-10-06T05:18:56.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab41550d0 con 0x7fdab406ae40 2023-10-06T05:18:56.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab40c4020 con 0x7fdab406ae40 2023-10-06T05:18:56.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaabfff700 1 -- 172.21.15.45:0/3892207650 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda9c023070 con 0x7fdab406ae40 2023-10-06T05:18:56.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.325+0000 7fdaabfff700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda94066f30 0x7fda940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.326+0000 7fdaabfff700 1 -- 172.21.15.45:0/3892207650 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fda9c085ef0 con 0x7fdab406ae40 2023-10-06T05:18:56.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.326+0000 7fdab3fff700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda94066f30 0x7fda940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.326+0000 7fdab3fff700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda94066f30 0x7fda940693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdab40ac0c0 tx=0x7fdaa4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.331+0000 7fdaabfff700 1 -- 172.21.15.45:0/3892207650 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda9c052e30 con 0x7fdab406ae40 2023-10-06T05:18:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.531+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) v1 -- 0x7fdab40c7890 con 0x7fdab406ae40 2023-10-06T05:18:56.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.535+0000 7fdaabfff700 1 -- 172.21.15.45:0/3892207650 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fda9c0150c0 con 0x7fdab406ae40 2023-10-06T05:18:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda94066f30 msgr2=0x7fda940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda94066f30 0x7fda940693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdab40ac0c0 tx=0x7fdaa4009040 comp rx=0 tx=0).stop 2023-10-06T05:18:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 msgr2=0x7fdab40c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab40c0a10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006f50 tx=0x7fda9c017000 comp rx=0 tx=0).stop 2023-10-06T05:18:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 shutdown_connections 2023-10-06T05:18:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda94066f30 0x7fda940693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdab415c6f0 0x7fdab40acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdab406aa70 0x7fdab4154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 --2- 172.21.15.45:0/3892207650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab406ae40 0x7fdab40c0a10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 >> 172.21.15.45:0/3892207650 conn(0x7fdab40cc570 msgr2=0x7fdab40ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 shutdown_connections 2023-10-06T05:18:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.537+0000 7fdaba395700 1 -- 172.21.15.45:0/3892207650 wait complete. 2023-10-06T05:18:56.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2023-10-06T05:18:56.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.686+0000 7f78516bd700 1 Processor -- start 2023-10-06T05:18:56.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.686+0000 7f78516bd700 1 -- start start 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f78516bd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c0d02b0 0x7f784c0d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f78516bd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f784c15c8c0 0x7f784c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f78516bd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784c0cd740 con 0x7f784c15d820 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f78516bd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784c0ccb30 con 0x7f784c15c8c0 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f78516bd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0cce10 con 0x7f784c0d02b0 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f784affd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c0d02b0 0x7f784c0d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f784b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f784c15c8c0 0x7f784c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f784a7fc700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f784c15d820 0x7f784c15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:34514/0 (socket says 172.21.15.45:34514) 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f784b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f784c15c8c0 0x7f784c0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:46208/0 (socket says 172.21.15.45:46208) 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f784a7fc700 1 -- 172.21.15.45:0/985373025 learned_addr learned my addr 172.21.15.45:0/985373025 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:18:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.687+0000 7f784affd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c0d02b0 0x7f784c0d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:51940/0 (socket says 172.21.15.45:51940) 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f784b7fe700 1 -- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c0d02b0 msgr2=0x7f784c0d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f784b7fe700 1 --2- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c0d02b0 0x7f784c0d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f784b7fe700 1 -- 172.21.15.45:0/985373025 >> v1:172.21.15.144:6789/0 conn(0x7f784c15d820 legacy=0x7f784c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f784b7fe700 1 -- 172.21.15.45:0/985373025 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0cd0f0 con 0x7f784c15c8c0 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f7849ffb700 1 -- 172.21.15.45:0/985373025 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319738798 0 0) 0x7f784c0cd740 con 0x7f784c15d820 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f784affd700 1 --2- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c0d02b0 0x7f784c0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f784b7fe700 1 --2- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 0x7f784c0caf10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7844009e90 tx=0x7f7844009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d592349d8d3d39e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f7849ffb700 1 -- 172.21.15.45:0/985373025 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784400e070 con 0x7f784c15c8c0 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f7849ffb700 1 -- 172.21.15.45:0/985373025 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784400bbf0 con 0x7f784c15c8c0 2023-10-06T05:18:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.688+0000 7f7849ffb700 1 -- 172.21.15.45:0/985373025 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844012430 con 0x7f784c15c8c0 2023-10-06T05:18:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 -- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 msgr2=0x7f784c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 --2- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 0x7f784c0caf10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7844009e90 tx=0x7f7844009780 comp rx=0 tx=0).stop 2023-10-06T05:18:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 -- 172.21.15.45:0/985373025 shutdown_connections 2023-10-06T05:18:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 --2- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 0x7f784c0caf10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 --2- 172.21.15.45:0/985373025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c0d02b0 0x7f784c0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 -- 172.21.15.45:0/985373025 >> 172.21.15.45:0/985373025 conn(0x7f784c06ed50 msgr2=0x7f784c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 -- 172.21.15.45:0/985373025 shutdown_connections 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.689+0000 7f78516bd700 1 -- 172.21.15.45:0/985373025 wait complete. 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.690+0000 7f78516bd700 1 Processor -- start 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.690+0000 7f78516bd700 1 -- start start 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f78516bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784c0d02b0 0x7f784c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f78516bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 0x7f784c16a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f78516bd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c15d820 0x7f784c15aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f78516bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0ce050 con 0x7f784c0d02b0 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f78516bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0c8140 con 0x7f784c15c8c0 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f78516bd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c16a6d0 con 0x7f784c15d820 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f784affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784c0d02b0 0x7f784c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f784affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784c0d02b0 0x7f784c0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57800/0 (socket says 172.21.15.45:57800) 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f784affd700 1 -- 172.21.15.45:0/460743357 learned_addr learned my addr 172.21.15.45:0/460743357 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f784b7fe700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c15d820 0x7f784c15aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.691+0000 7f784a7fc700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 0x7f784c16a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f784affd700 1 -- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c15d820 msgr2=0x7f784c15aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f784affd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c15d820 0x7f784c15aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f784affd700 1 -- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 msgr2=0x7f784c16a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f784affd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 0x7f784c16a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f784affd700 1 -- 172.21.15.45:0/460743357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c009420 con 0x7f784c0d02b0 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f784affd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784c0d02b0 0x7f784c0c7c00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f783c00ecb0 tx=0x7f783c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f7832ffd700 1 -- 172.21.15.45:0/460743357 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c00ee60 con 0x7f784c0d02b0 2023-10-06T05:18:56.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f7832ffd700 1 -- 172.21.15.45:0/460743357 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783c00cbe0 con 0x7f784c0d02b0 2023-10-06T05:18:56.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f7832ffd700 1 -- 172.21.15.45:0/460743357 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c013470 con 0x7f784c0d02b0 2023-10-06T05:18:56.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7844009a20 con 0x7f784c0d02b0 2023-10-06T05:18:56.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.692+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c0b0140 con 0x7f784c0d02b0 2023-10-06T05:18:56.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2023-10-06T05:18:56.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.693+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c0ce330 con 0x7f784c0d02b0 2023-10-06T05:18:56.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.694+0000 7f7832ffd700 1 -- 172.21.15.45:0/460743357 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f783c013820 con 0x7f784c0d02b0 2023-10-06T05:18:56.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.694+0000 7f7832ffd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7838066fb0 0x7f7838069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.694+0000 7f7832ffd700 1 -- 172.21.15.45:0/460743357 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f783c084ce0 con 0x7f784c0d02b0 2023-10-06T05:18:56.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.694+0000 7f784a7fc700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7838066fb0 0x7f7838069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.695+0000 7f784a7fc700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7838066fb0 0x7f7838069460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f784c06f7f0 tx=0x7f7840009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.698+0000 7f7832ffd700 1 -- 172.21.15.45:0/460743357 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f783c08b080 con 0x7f784c0d02b0 2023-10-06T05:18:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:56.988+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) v1 -- 0x7f784c0887c0 con 0x7f784c0d02b0 2023-10-06T05:18:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.904+0000 7f7832ffd700 1 -- 172.21.15.45:0/460743357 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.6342 v58) v1 ==== 115+0+0 (secure 0 0 0) 0x7f783c055610 con 0x7f784c0d02b0 2023-10-06T05:18:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7838066fb0 msgr2=0x7f7838069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7838066fb0 0x7f7838069460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f784c06f7f0 tx=0x7f7840009040 comp rx=0 tx=0).stop 2023-10-06T05:18:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784c0d02b0 msgr2=0x7f784c0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784c0d02b0 0x7f784c0c7c00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f783c00ecb0 tx=0x7f783c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 shutdown_connections 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7838066fb0 0x7f7838069460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f784c15d820 0x7f784c15aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784c0d02b0 0x7f784c0c7c00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.907+0000 7f78516bd700 1 --2- 172.21.15.45:0/460743357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f784c15c8c0 0x7f784c16a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.908+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 >> 172.21.15.45:0/460743357 conn(0x7f784c06ed50 msgr2=0x7f784c0b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.908+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 shutdown_connections 2023-10-06T05:18:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:57.908+0000 7f78516bd700 1 -- 172.21.15.45:0/460743357 wait complete. 2023-10-06T05:18:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:unfreezing mds.6342 2023-10-06T05:18:57.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -n kernel.core_pattern 2023-10-06T05:18:57.960 INFO:teuthology.orchestra.run.smithi045.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:18:57.962 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:18:57.962 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-10-06T05:18:58.041 INFO:teuthology.orchestra.run.smithi045.stdout:1696569514.143793.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2023-10-06T05:18:58.043 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1696569514.143793.core'] 2023-10-06T05:18:58.043 INFO:tasks.cephfs.cephfs_test_case:Found core file 1696569514.143793.core, deleting it 2023-10-06T05:18:58.043 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1696569514.143793.core 2023-10-06T05:18:58.121 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:18:58.122 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:18:58.122 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:18:58.122 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:58.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:18:58.122 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi045 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-10-06T05:18:58.123 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:18:58.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:18:58.164 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:18:58.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:58.256 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:18:58.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.304+0000 7f4089110700 1 Processor -- start 2023-10-06T05:18:58.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.304+0000 7f4089110700 1 -- start start 2023-10-06T05:18:58.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4089110700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40840ca810 0x7f40840cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4089110700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840c9640 0x7f40840b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4089110700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840c84c0 con 0x7f40840c91f0 2023-10-06T05:18:58.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4089110700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840c8d60 con 0x7f40840ca810 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4089110700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4084069d00 con 0x7f40840c9640 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4082d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40840ca810 0x7f40840cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f408359e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840c9640 0x7f40840b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4082d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40840ca810 0x7f40840cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:46228/0 (socket says 172.21.15.45:46228) 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f4082d9d700 1 -- 172.21.15.45:0/866412160 learned_addr learned my addr 172.21.15.45:0/866412160 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.305+0000 7f408259c700 1 --1- 172.21.15.45:0/866412160 >> v1:172.21.15.144:6789/0 conn(0x7f40840c91f0 0x7f40840afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:34528/0 (socket says 172.21.15.45:34528) 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4082d9d700 1 -- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840c9640 msgr2=0x7f40840b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4082d9d700 1 --2- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840c9640 0x7f40840b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4082d9d700 1 -- 172.21.15.45:0/866412160 >> v1:172.21.15.144:6789/0 conn(0x7f40840c91f0 legacy=0x7f40840afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4082d9d700 1 -- 172.21.15.45:0/866412160 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840c8710 con 0x7f40840ca810 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4081d9b700 1 -- 172.21.15.45:0/866412160 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215797425 0 0) 0x7f40840c84c0 con 0x7f40840c91f0 2023-10-06T05:18:58.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f408359e700 1 --2- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840c9640 0x7f40840b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4082d9d700 1 --2- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840ca810 0x7f40840cabe0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40740099b0 tx=0x7f4074009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d60620c8808f7f7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4081d9b700 1 -- 172.21.15.45:0/866412160 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074009ea0 con 0x7f40840ca810 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4081d9b700 1 -- 172.21.15.45:0/866412160 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407400b860 con 0x7f40840ca810 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.306+0000 7f4081d9b700 1 -- 172.21.15.45:0/866412160 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407400bc00 con 0x7f40840ca810 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 -- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840ca810 msgr2=0x7f40840cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 --2- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840ca810 0x7f40840cabe0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40740099b0 tx=0x7f4074009250 comp rx=0 tx=0).stop 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 -- 172.21.15.45:0/866412160 shutdown_connections 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 --2- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840c9640 0x7f40840b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 --2- 172.21.15.45:0/866412160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840ca810 0x7f40840cabe0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 -- 172.21.15.45:0/866412160 >> 172.21.15.45:0/866412160 conn(0x7f4084067620 msgr2=0x7f4084067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 -- 172.21.15.45:0/866412160 shutdown_connections 2023-10-06T05:18:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.307+0000 7f4089110700 1 -- 172.21.15.45:0/866412160 wait complete. 2023-10-06T05:18:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.308+0000 7f4089110700 1 Processor -- start 2023-10-06T05:18:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.308+0000 7f4089110700 1 -- start start 2023-10-06T05:18:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f4089110700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840c91f0 0x7f40840c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f4089110700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40840c9640 0x7f40840c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f4089110700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840ca810 0x7f4084169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f4089110700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c6370 con 0x7f40840c9640 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f4089110700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c5760 con 0x7f40840c91f0 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f4089110700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c5a40 con 0x7f40840ca810 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40840c9640 0x7f40840c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40840c9640 0x7f40840c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57826/0 (socket says 172.21.15.45:57826) 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 -- 172.21.15.45:0/3290950120 learned_addr learned my addr 172.21.15.45:0/3290950120 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f4082d9d700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840c91f0 0x7f40840c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408359e700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840ca810 0x7f4084169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 -- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840ca810 msgr2=0x7f4084169aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840ca810 0x7f4084169aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 -- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840c91f0 msgr2=0x7f40840c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840c91f0 0x7f40840c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 -- 172.21.15.45:0/3290950120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c009a20 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408359e700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840ca810 0x7f4084169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.309+0000 7f408259c700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40840c9640 0x7f40840c4f30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f406c0011e0 tx=0x7f406c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.310+0000 7f407affd700 1 -- 172.21.15.45:0/3290950120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c005a70 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.310+0000 7f407affd700 1 -- 172.21.15.45:0/3290950120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c005bd0 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.310+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40740094f0 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.310+0000 7f407affd700 1 -- 172.21.15.45:0/3290950120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c0070d0 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.310+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40840c6000 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.311+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408406f160 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.311+0000 7f407affd700 1 -- 172.21.15.45:0/3290950120 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f406c007480 con 0x7f40840c9640 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.312+0000 7f407affd700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4064066f30 0x7f40640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.312+0000 7f407affd700 1 -- 172.21.15.45:0/3290950120 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f406c083d80 con 0x7f40840c9640 2023-10-06T05:18:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.312+0000 7f4082d9d700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4064066f30 0x7f40640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.312+0000 7f4082d9d700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4064066f30 0x7f40640693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40740099b0 tx=0x7f4074017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:58.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.316+0000 7f407affd700 1 -- 172.21.15.45:0/3290950120 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f406c050d20 con 0x7f40840c9640 2023-10-06T05:18:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.565+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40840c64b0 con 0x7f40840c9640 2023-10-06T05:18:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.566+0000 7f407affd700 1 -- 172.21.15.45:0/3290950120 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3846 (secure 0 0 0) 0x7f40840c64b0 con 0x7f40840c9640 2023-10-06T05:18:58.568 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:18:58.568 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:57.905890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5543":{"gid":5543,"name":"d","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:18:58.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4064066f30 msgr2=0x7f40640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4064066f30 0x7f40640693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40740099b0 tx=0x7f4074017000 comp rx=0 tx=0).stop 2023-10-06T05:18:58.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40840c9640 msgr2=0x7f40840c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40840c9640 0x7f40840c4f30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f406c0011e0 tx=0x7f406c007e10 comp rx=0 tx=0).stop 2023-10-06T05:18:58.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 shutdown_connections 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4064066f30 0x7f40640693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40840ca810 0x7f4084169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40840c91f0 0x7f40840c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 --2- 172.21.15.45:0/3290950120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40840c9640 0x7f40840c4f30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 >> 172.21.15.45:0/3290950120 conn(0x7f4084067620 msgr2=0x7f40840ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 shutdown_connections 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:58.568+0000 7f4089110700 1 -- 172.21.15.45:0/3290950120 wait complete. 2023-10-06T05:18:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 59 2023-10-06T05:18:58.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:09.524580+0000', 'modified': '2023-10-06T05:18:57.905890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5543}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5543': {'gid': 5543, 'name': 'd', 'rank': 0, 'incarnation': 59, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/517622572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 517622572}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 517622572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:18:58.985 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5543:up:replay 2023-10-06T05:18:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.516+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.517+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2023-10-06T05:18:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.517+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2023-10-06T05:18:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.517+0000 7efdaa7fc700 0 client.6420 ms_handle_remote_reset on v2:172.21.15.45:6832/4169661273 2023-10-06T05:18:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.517+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:18:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.517+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:18:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.517+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:18:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:18:59.517+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-10-06T05:18:59.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.126+0000 7f472216b700 1 Processor -- start 2023-10-06T05:19:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.127+0000 7f472216b700 1 -- start start 2023-10-06T05:19:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.127+0000 7f472216b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c0c93c0 0x7f471c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.127+0000 7f472216b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c15b000 con 0x7f471c157030 2023-10-06T05:19:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.127+0000 7f472216b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c15b2e0 con 0x7f471c0c8ff0 2023-10-06T05:19:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.127+0000 7f472216b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0c9830 con 0x7f471c0c93c0 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f471c157030 0x7f471c0cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55064/0 (socket says 172.21.15.45:55064) 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471bfff700 1 -- 172.21.15.45:0/2205823393 learned_addr learned my addr 172.21.15.45:0/2205823393 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471affd700 1 --2- 172.21.15.45:0/2205823393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c0c93c0 0x7f471c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471a7fc700 1 -- 172.21.15.45:0/2205823393 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348254336 0 0) 0x7f471c15b000 con 0x7f471c157030 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471a7fc700 1 -- 172.21.15.45:0/2205823393 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4704003570 con 0x7f471c157030 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471a7fc700 1 -- 172.21.15.45:0/2205823393 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478171777 0 0) 0x7f471c15b2e0 con 0x7f471c0c8ff0 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471a7fc700 1 -- 172.21.15.45:0/2205823393 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f471c15b000 con 0x7f471c0c8ff0 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471affd700 1 -- 172.21.15.45:0/2205823393 >> v1:172.21.15.144:6789/0 conn(0x7f471c0c8ff0 legacy=0x7f471c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471affd700 1 -- 172.21.15.45:0/2205823393 >> v1:172.21.15.45:6789/0 conn(0x7f471c157030 legacy=0x7f471c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471affd700 1 -- 172.21.15.45:0/2205823393 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c15b5c0 con 0x7f471c0c93c0 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471affd700 1 --2- 172.21.15.45:0/2205823393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c0c93c0 0x7f471c156af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4710009910 tx=0x7f47100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6d37b4311489ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.128+0000 7f471a7fc700 1 -- 172.21.15.45:0/2205823393 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710011630 con 0x7f471c0c93c0 2023-10-06T05:19:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f471a7fc700 1 -- 172.21.15.45:0/2205823393 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710011790 con 0x7f471c0c93c0 2023-10-06T05:19:00.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f471a7fc700 1 -- 172.21.15.45:0/2205823393 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710011b30 con 0x7f471c0c93c0 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f472216b700 1 -- 172.21.15.45:0/2205823393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c0c93c0 msgr2=0x7f471c156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f472216b700 1 --2- 172.21.15.45:0/2205823393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c0c93c0 0x7f471c156af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4710009910 tx=0x7f47100091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f472216b700 1 -- 172.21.15.45:0/2205823393 shutdown_connections 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f472216b700 1 --2- 172.21.15.45:0/2205823393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c0c93c0 0x7f471c156af0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f472216b700 1 -- 172.21.15.45:0/2205823393 >> 172.21.15.45:0/2205823393 conn(0x7f471c0cdf50 msgr2=0x7f471c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f472216b700 1 -- 172.21.15.45:0/2205823393 shutdown_connections 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.129+0000 7f472216b700 1 -- 172.21.15.45:0/2205823393 wait complete. 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.131+0000 7f472216b700 1 Processor -- start 2023-10-06T05:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.131+0000 7f472216b700 1 -- start start 2023-10-06T05:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f472216b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471c0c8ff0 0x7f471c14d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f472216b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0c93c0 0x7f471c0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f472216b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c157030 0x7f471c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f472216b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0c6bb0 con 0x7f471c0c93c0 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f472216b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0c6e90 con 0x7f471c0c8ff0 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f472216b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0c7170 con 0x7f471c157030 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f471affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0c93c0 0x7f471c0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f471affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0c93c0 0x7f471c0c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57842/0 (socket says 172.21.15.45:57842) 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f471affd700 1 -- 172.21.15.45:0/657753712 learned_addr learned my addr 172.21.15.45:0/657753712 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f471bfff700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c157030 0x7f471c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.132+0000 7f471b7fe700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471c0c8ff0 0x7f471c14d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f471affd700 1 -- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c157030 msgr2=0x7f471c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f471affd700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c157030 0x7f471c0cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f471affd700 1 -- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471c0c8ff0 msgr2=0x7f471c14d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f471affd700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471c0c8ff0 0x7f471c14d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f471affd700 1 -- 172.21.15.45:0/657753712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710009420 con 0x7f471c0c93c0 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f471affd700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0c93c0 0x7f471c0c6240 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4710009f00 tx=0x7f4710017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f47037fe700 1 -- 172.21.15.45:0/657753712 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471001f070 con 0x7f471c0c93c0 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f47037fe700 1 -- 172.21.15.45:0/657753712 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710000bc0 con 0x7f471c0c93c0 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c0c7450 con 0x7f471c0c93c0 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f47037fe700 1 -- 172.21.15.45:0/657753712 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710025420 con 0x7f471c0c93c0 2023-10-06T05:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.133+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c14d7c0 con 0x7f471c0c93c0 2023-10-06T05:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.134+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c0c7970 con 0x7f471c0c93c0 2023-10-06T05:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.135+0000 7f47037fe700 1 -- 172.21.15.45:0/657753712 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4710023070 con 0x7f471c0c93c0 2023-10-06T05:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.135+0000 7f47037fe700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4708066f30 0x7f47080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.135+0000 7f47037fe700 1 -- 172.21.15.45:0/657753712 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f4710085d00 con 0x7f471c0c93c0 2023-10-06T05:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.135+0000 7f471b7fe700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4708066f30 0x7f47080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.136+0000 7f471b7fe700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4708066f30 0x7f47080693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f470c0037e0 tx=0x7f470c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:00.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.140+0000 7f47037fe700 1 -- 172.21.15.45:0/657753712 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f471c0c7970 con 0x7f471c0c93c0 2023-10-06T05:19:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.382+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f471c0c7970 con 0x7f471c0c93c0 2023-10-06T05:19:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.383+0000 7f47037fe700 1 -- 172.21.15.45:0/657753712 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f471c0c7970 con 0x7f471c0c93c0 2023-10-06T05:19:00.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:19:00.385 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1230704519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1230704519},{"type":"v1","addr":"172.21.15.45:6833","nonce":1230704519}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:18:59.911975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5543":{"gid":5543,"name":"d","rank":0,"incarnation":59,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4708066f30 msgr2=0x7f47080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4708066f30 0x7f47080693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f470c0037e0 tx=0x7f470c009040 comp rx=0 tx=0).stop 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0c93c0 msgr2=0x7f471c0c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0c93c0 0x7f471c0c6240 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4710009f00 tx=0x7f4710017000 comp rx=0 tx=0).stop 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 shutdown_connections 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4708066f30 0x7f47080693e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f471c157030 0x7f471c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f471c0c8ff0 0x7f471c14d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 --2- 172.21.15.45:0/657753712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0c93c0 0x7f471c0c6240 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 >> 172.21.15.45:0/657753712 conn(0x7f471c0cdf50 msgr2=0x7f471c0c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 shutdown_connections 2023-10-06T05:19:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:00.385+0000 7f472216b700 1 -- 172.21.15.45:0/657753712 wait complete. 2023-10-06T05:19:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 61 2023-10-06T05:19:00.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:09.524580+0000', 'modified': '2023-10-06T05:18:59.911975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5543}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5543': {'gid': 5543, 'name': 'd', 'rank': 0, 'incarnation': 59, 'state': 'up:rejoin', 'state_seq': 16, 'addr': '172.21.15.144:6837/517622572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 517622572}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 517622572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:19:00.802 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5543:up:rejoin 2023-10-06T05:19:01.803 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.944+0000 7fe8a3c8f700 1 Processor -- start 2023-10-06T05:19:01.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.944+0000 7fe8a3c8f700 1 -- start start 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.945+0000 7fe8a3c8f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe89c0cd4d0 0x7fe89c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.945+0000 7fe8a3c8f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c0ae8d0 con 0x7fe89c1560a0 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.945+0000 7fe8a3c8f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c0aebb0 con 0x7fe89c155cd0 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.945+0000 7fe8a3c8f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c0ae6b0 con 0x7fe89c0cd4d0 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.945+0000 7fe8a122a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe89c1560a0 0x7fe89c0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55068/0 (socket says 172.21.15.45:55068) 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a122a700 1 -- 172.21.15.45:0/3685679105 learned_addr learned my addr 172.21.15.45:0/3685679105 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a222c700 1 --2- 172.21.15.45:0/3685679105 >> v2:172.21.15.144:3300/0 conn(0x7fe89c0cd4d0 0x7fe89c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562198344 0 0) 0x7fe89c0ae8d0 con 0x7fe89c1560a0 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe888003570 con 0x7fe89c1560a0 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 498586469 0 0) 0x7fe888003570 con 0x7fe89c1560a0 2023-10-06T05:19:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 >> v2:172.21.15.144:3300/0 conn(0x7fe89c0cd4d0 msgr2=0x7fe89c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 --2- 172.21.15.45:0/3685679105 >> v2:172.21.15.144:3300/0 conn(0x7fe89c0cd4d0 0x7fe89c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 >> v1:172.21.15.144:6789/0 conn(0x7fe89c155cd0 legacy=0x7fe89c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0aee90 con 0x7fe89c1560a0 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fe88c001470 con 0x7fe89c1560a0 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a222c700 1 --2- 172.21.15.45:0/3685679105 >> v2:172.21.15.144:3300/0 conn(0x7fe89c0cd4d0 0x7fe89c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 >> v1:172.21.15.45:6789/0 conn(0x7fe89c1560a0 legacy=0x7fe89c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe888003a60 0x7fe888003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe888004810 0x7fe888004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe88800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0ae6b0 con 0x7fe8880051c0 2023-10-06T05:19:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88800c0e0 con 0x7fe888004810 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88800c2a0 con 0x7fe888003a60 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a222c700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe88800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a122a700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe888004810 0x7fe888004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.946+0000 7fe8a1a2b700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe888003a60 0x7fe888003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a222c700 1 -- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe888003a60 msgr2=0x7fe888003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a222c700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe888003a60 0x7fe888003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a222c700 1 -- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe888004810 msgr2=0x7fe888004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a222c700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe888004810 0x7fe888004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a222c700 1 -- 172.21.15.45:0/3685679105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe888003570 con 0x7fe8880051c0 2023-10-06T05:19:01.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a1a2b700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe888003a60 0x7fe888003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a222c700 1 -- 172.21.15.45:0/3685679105 reap_dead start 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a222c700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe88800ba60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0cd8d0 tx=0x7fe89400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ff3cc76de46f04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894009c50 con 0x7fe8880051c0 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89400f040 con 0x7fe8880051c0 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.947+0000 7fe8a0a29700 1 -- 172.21.15.45:0/3685679105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400ee90 con 0x7fe8880051c0 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.948+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/3685679105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 msgr2=0x7fe88800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.948+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe88800ba60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0cd8d0 tx=0x7fe89400b040 comp rx=0 tx=0).stop 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.948+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/3685679105 shutdown_connections 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.948+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/3685679105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe88800ba60 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.948+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/3685679105 >> 172.21.15.45:0/3685679105 conn(0x7fe89c06f090 msgr2=0x7fe89c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.948+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/3685679105 shutdown_connections 2023-10-06T05:19:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.948+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/3685679105 wait complete. 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.949+0000 7fe8a3c8f700 1 Processor -- start 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.949+0000 7fe8a3c8f700 1 -- start start 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a3c8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe89c0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a3c8f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe89c15b0e0 0x7fe89c169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a3c8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe89c1583d0 0x7fe89c169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a3c8f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c15b850 con 0x7fe8880051c0 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a3c8f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0cb320 con 0x7fe89c1583d0 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a3c8f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0cbb60 con 0x7fe89c15b0e0 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe89c0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe89c0cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57870/0 (socket says 172.21.15.45:57870) 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 -- 172.21.15.45:0/803611442 learned_addr learned my addr 172.21.15.45:0/803611442 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a222c700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe89c1583d0 0x7fe89c169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a122a700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe89c15b0e0 0x7fe89c169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 -- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe89c15b0e0 msgr2=0x7fe89c169520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe89c15b0e0 0x7fe89c169520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 -- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe89c1583d0 msgr2=0x7fe89c169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe89c1583d0 0x7fe89c169a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 -- 172.21.15.45:0/803611442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c15b570 con 0x7fe8880051c0 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.950+0000 7fe8a1a2b700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe89c0cbff0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe89800e800 tx=0x7fe898009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.951+0000 7fe891ffb700 1 -- 172.21.15.45:0/803611442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898011460 con 0x7fe8880051c0 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.951+0000 7fe891ffb700 1 -- 172.21.15.45:0/803611442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe898010040 con 0x7fe8880051c0 2023-10-06T05:19:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.951+0000 7fe891ffb700 1 -- 172.21.15.45:0/803611442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980117d0 con 0x7fe8880051c0 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.951+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c0cb510 con 0x7fe8880051c0 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.951+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c155780 con 0x7fe8880051c0 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.952+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c06e2f0 con 0x7fe8880051c0 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.952+0000 7fe891ffb700 1 -- 172.21.15.45:0/803611442 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe898011bc0 con 0x7fe8880051c0 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.953+0000 7fe891ffb700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe880066f30 0x7fe8800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.953+0000 7fe891ffb700 1 -- 172.21.15.45:0/803611442 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fe898083b10 con 0x7fe8880051c0 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.953+0000 7fe8a122a700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe880066f30 0x7fe8800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.953+0000 7fe8a122a700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe880066f30 0x7fe8800693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe89c157070 tx=0x7fe89c156540 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:01.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:01.957+0000 7fe891ffb700 1 -- 172.21.15.45:0/803611442 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe89804d5a0 con 0x7fe8880051c0 2023-10-06T05:19:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.207+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe89c0af170 con 0x7fe8880051c0 2023-10-06T05:19:02.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.208+0000 7fe891ffb700 1 -- 172.21.15.45:0/803611442 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fe898050ab0 con 0x7fe8880051c0 2023-10-06T05:19:02.209 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:19:02.210 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1230704519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1230704519},{"type":"v1","addr":"172.21.15.45:6833","nonce":1230704519}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:19:00.913269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5543":{"gid":5543,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:19:02.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe880066f30 msgr2=0x7fe8800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe880066f30 0x7fe8800693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe89c157070 tx=0x7fe89c156540 comp rx=0 tx=0).stop 2023-10-06T05:19:02.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 msgr2=0x7fe89c0cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe89c0cbff0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe89800e800 tx=0x7fe898009040 comp rx=0 tx=0).stop 2023-10-06T05:19:02.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 shutdown_connections 2023-10-06T05:19:02.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe880066f30 0x7fe8800693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe89c1583d0 0x7fe89c169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8880051c0 0x7fe89c0cbff0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 --2- 172.21.15.45:0/803611442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe89c15b0e0 0x7fe89c169520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 >> 172.21.15.45:0/803611442 conn(0x7fe89c06f090 msgr2=0x7fe89c0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:02.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 shutdown_connections 2023-10-06T05:19:02.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.210+0000 7fe8a3c8f700 1 -- 172.21.15.45:0/803611442 wait complete. 2023-10-06T05:19:02.214 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 62 2023-10-06T05:19:02.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:09.524580+0000', 'modified': '2023-10-06T05:19:00.913269+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5543}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5543': {'gid': 5543, 'name': 'd', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.144:6837/517622572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 517622572}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 517622572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:19:02.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:19:02.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:19:02.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.767+0000 7fe2a671d700 1 Processor -- start 2023-10-06T05:19:02.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.767+0000 7fe2a671d700 1 -- start start 2023-10-06T05:19:02.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe2a671d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2a0159170 0x7fe2a0159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe2a671d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe2a015d040 0x7fe2a0068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe2a671d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a00ca6d0 con 0x7fe2a0069b90 2023-10-06T05:19:02.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe2a671d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a015d550 con 0x7fe2a015d040 2023-10-06T05:19:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe2a671d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a015d810 con 0x7fe2a0159170 2023-10-06T05:19:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe29f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe2a015d040 0x7fe2a0068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe29f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe2a015d040 0x7fe2a0068bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57880/0 (socket says 172.21.15.45:57880) 2023-10-06T05:19:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.768+0000 7fe29f7fe700 1 -- 172.21.15.45:0/2200161188 learned_addr learned my addr 172.21.15.45:0/2200161188 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:02.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2200161188 >> v2:172.21.15.144:3300/0 conn(0x7fe2a0159170 0x7fe2a0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29f7fe700 1 -- 172.21.15.45:0/2200161188 >> v2:172.21.15.144:3300/0 conn(0x7fe2a0159170 msgr2=0x7fe2a0159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/2200161188 >> v2:172.21.15.144:3300/0 conn(0x7fe2a0159170 0x7fe2a0159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29f7fe700 1 -- 172.21.15.45:0/2200161188 >> v1:172.21.15.144:6789/0 conn(0x7fe2a0069b90 legacy=0x7fe2a00aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29f7fe700 1 -- 172.21.15.45:0/2200161188 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a015daf0 con 0x7fe2a015d040 2023-10-06T05:19:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2200161188 >> v2:172.21.15.144:3300/0 conn(0x7fe2a0159170 0x7fe2a0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/2200161188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a015d040 0x7fe2a0068bc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0068a90 tx=0x7fe294009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e2f850b8adf4fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29effd700 1 -- 172.21.15.45:0/2200161188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400b820 con 0x7fe2a015d040 2023-10-06T05:19:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29effd700 1 -- 172.21.15.45:0/2200161188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29400b980 con 0x7fe2a015d040 2023-10-06T05:19:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.769+0000 7fe29effd700 1 -- 172.21.15.45:0/2200161188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400bd20 con 0x7fe2a015d040 2023-10-06T05:19:02.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 -- 172.21.15.45:0/2200161188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a015d040 msgr2=0x7fe2a0068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 --2- 172.21.15.45:0/2200161188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a015d040 0x7fe2a0068bc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0068a90 tx=0x7fe294009250 comp rx=0 tx=0).stop 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 -- 172.21.15.45:0/2200161188 shutdown_connections 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 --2- 172.21.15.45:0/2200161188 >> v2:172.21.15.144:3300/0 conn(0x7fe2a0159170 0x7fe2a0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 --2- 172.21.15.45:0/2200161188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a015d040 0x7fe2a0068bc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 -- 172.21.15.45:0/2200161188 >> 172.21.15.45:0/2200161188 conn(0x7fe2a006f7c0 msgr2=0x7fe2a00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 -- 172.21.15.45:0/2200161188 shutdown_connections 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.770+0000 7fe2a671d700 1 -- 172.21.15.45:0/2200161188 wait complete. 2023-10-06T05:19:02.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.771+0000 7fe2a671d700 1 Processor -- start 2023-10-06T05:19:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.771+0000 7fe2a671d700 1 -- start start 2023-10-06T05:19:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe2a671d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a0069b90 0x7fe2a01583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe2a671d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2a0159170 0x7fe2a00a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe2a671d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2a015d040 0x7fe2a00b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe2a671d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a016b750 con 0x7fe2a0069b90 2023-10-06T05:19:02.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe2a671d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a016ba30 con 0x7fe2a015d040 2023-10-06T05:19:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe2a671d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a016bd10 con 0x7fe2a0159170 2023-10-06T05:19:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe29ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a0069b90 0x7fe2a01583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe29ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a0069b90 0x7fe2a01583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57886/0 (socket says 172.21.15.45:57886) 2023-10-06T05:19:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe29ffff700 1 -- 172.21.15.45:0/1682969730 learned_addr learned my addr 172.21.15.45:0/1682969730 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2a015d040 0x7fe2a00b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.772+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2a0159170 0x7fe2a00a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe29ffff700 1 -- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2a0159170 msgr2=0x7fe2a00a6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe29ffff700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2a0159170 0x7fe2a00a6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe29ffff700 1 -- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2a015d040 msgr2=0x7fe2a00b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe29ffff700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2a015d040 0x7fe2a00b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe29ffff700 1 -- 172.21.15.45:0/1682969730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2940094f0 con 0x7fe2a0069b90 2023-10-06T05:19:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe29ffff700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a0069b90 0x7fe2a01583c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe29000c830 tx=0x7fe290008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe287fff700 1 -- 172.21.15.45:0/1682969730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290011600 con 0x7fe2a0069b90 2023-10-06T05:19:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe287fff700 1 -- 172.21.15.45:0/1682969730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe290011760 con 0x7fe2a0069b90 2023-10-06T05:19:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a016bff0 con 0x7fe2a0069b90 2023-10-06T05:19:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe287fff700 1 -- 172.21.15.45:0/1682969730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290011b20 con 0x7fe2a0069b90 2023-10-06T05:19:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.773+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a0158900 con 0x7fe2a0069b90 2023-10-06T05:19:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.774+0000 7fe287fff700 1 -- 172.21.15.45:0/1682969730 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe290018360 con 0x7fe2a0069b90 2023-10-06T05:19:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.775+0000 7fe287fff700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe28c066f60 0x7fe28c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.775+0000 7fe287fff700 1 -- 172.21.15.45:0/1682969730 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fe290084730 con 0x7fe2a0069b90 2023-10-06T05:19:02.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.775+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe28c066f60 0x7fe28c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.776+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe28c066f60 0x7fe28c069410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0068a90 tx=0x7fe294016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.777+0000 7fe2afa31b80 1 Processor -- start 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.777+0000 7fe2afa31b80 1 -- start start 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f50698310 con 0x555f507ce4d0 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f506d5130 con 0x555f507c5450 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f506d2af0 con 0x555f507aaf40 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57898/0 (socket says 172.21.15.45:57898) 2023-10-06T05:19:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 -- 172.21.15.45:0/3477117443 learned_addr learned my addr 172.21.15.45:0/3477117443 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe29ffff700 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 -- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 msgr2=0x555f507cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 -- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 msgr2=0x555f507c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.778+0000 7fe2a4cba700 1 -- 172.21.15.45:0/3477117443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f504d4790 con 0x555f507ce4d0 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.779+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ce940 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe298004050 tx=0x7fe298002620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f30a96e0ac5fe18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:02.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.779+0000 7fe2857fa700 1 -- 172.21.15.45:0/3477117443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298003ac0 con 0x555f507ce4d0 2023-10-06T05:19:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.779+0000 7fe2857fa700 1 -- 172.21.15.45:0/3477117443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29800fb50 con 0x555f507ce4d0 2023-10-06T05:19:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.779+0000 7fe2857fa700 1 -- 172.21.15.45:0/3477117443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298014470 con 0x555f507ce4d0 2023-10-06T05:19:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.779+0000 7fe2afa31b80 1 -- 172.21.15.45:0/3477117443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 msgr2=0x555f507ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.779+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ce940 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe298004050 tx=0x7fe298002620 comp rx=0 tx=0).stop 2023-10-06T05:19:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 -- 172.21.15.45:0/3477117443 shutdown_connections 2023-10-06T05:19:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ce940 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/3477117443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 -- 172.21.15.45:0/3477117443 >> 172.21.15.45:0/3477117443 conn(0x555f507c8880 msgr2=0x555f507c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 -- 172.21.15.45:0/3477117443 shutdown_connections 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 -- 172.21.15.45:0/3477117443 wait complete. 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 -- start start 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.780+0000 7fe2afa31b80 1 Processor -- start 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507aaf40 0x555f507a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507ce4d0 0x555f5079eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f506ce8e0 con 0x555f507aaf40 2023-10-06T05:19:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5067a980 con 0x555f507c5450 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f5067a620 con 0x555f507ce4d0 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2afa31b80 10 client.0 _ll_register_callbacks cb 0x555f504dda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe29ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507aaf40 0x555f507a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe29ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507aaf40 0x555f507a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57908/0 (socket says 172.21.15.45:57908) 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe29ffff700 1 -- 172.21.15.45:0/2827751583 learned_addr learned my addr 172.21.15.45:0/2827751583 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.781+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507ce4d0 0x555f5079eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe29ffff700 1 -- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507ce4d0 msgr2=0x555f5079eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507ce4d0 0x555f5079eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe29ffff700 1 -- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 msgr2=0x555f507a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe29ffff700 1 -- 172.21.15.45:0/2827751583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe298003cb0 con 0x555f507aaf40 2023-10-06T05:19:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507aaf40 0x555f507a2c50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe29000ce20 tx=0x7fe29000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aff4573ae552f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe27e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:19:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2827751583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2900552e0 con 0x555f507aaf40 2023-10-06T05:19:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2827751583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fe290058530 con 0x555f507aaf40 2023-10-06T05:19:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f50772d60 con 0x555f507aaf40 2023-10-06T05:19:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2827751583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe29005c280 con 0x555f507aaf40 2023-10-06T05:19:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 10 client.6495 fetch_fsmap learned FSMap version 62 2023-10-06T05:19:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x555f505c10a0 con 0x555f507aaf40 2023-10-06T05:19:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2827751583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f50772d60 con 0x555f507aaf40 2023-10-06T05:19:02.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2827751583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 62) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe29005bdf0 con 0x555f507aaf40 2023-10-06T05:19:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 10 client.6495 fetch_fsmap finished waiting for FSMap version 62 2023-10-06T05:19:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 10 client.6495 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:19:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 20 client.6495 populate_metadata read hostname 'smithi045' 2023-10-06T05:19:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f507d7d50 0x555f507d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 4 client.6495 mds_command: new command op to 5543 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:19:02.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.782+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 --> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f5074fb30 con 0x555f507d7d50 2023-10-06T05:19:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.783+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f507d7d50 0x555f507d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.783+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f507d7d50 0x555f507d8120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4500860f396845 server_cookie=cb67083004fc4f01 in_seq=0 out_seq=0 2023-10-06T05:19:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.783+0000 7fe27e7fc700 10 client.6495 ms_handle_connect on v2:172.21.15.144:6835/517622572 2023-10-06T05:19:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2827751583 <== mds.0 v2:172.21.15.144:6835/517622572 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x555f5074fb30 con 0x555f507d7d50 2023-10-06T05:19:02.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe27e7fc700 10 client.6495 handle_command_reply: tid=0 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 client.6495 shutdown 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507aaf40 msgr2=0x555f507a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507aaf40 0x555f507a2c50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe29000ce20 tx=0x7fe29000cf90 comp rx=0 tx=0).stop 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 shutdown_connections 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f507d7d50 0x555f507d8120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507ce4d0 0x555f5079eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507aaf40 0x555f507a2c50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe27e7fc700 0 client.6495 ms_handle_reset on v2:172.21.15.144:6835/517622572 2023-10-06T05:19:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2827751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 >> 172.21.15.45:0/2827751583 conn(0x555f507c8880 msgr2=0x555f5077ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 shutdown_connections 2023-10-06T05:19:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2827751583 wait complete. 2023-10-06T05:19:02.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.784+0000 7fe2afa31b80 20 client.6495 trim_cache size 0 max 16384 2023-10-06T05:19:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.802+0000 7fe2afa31b80 1 Processor -- start 2023-10-06T05:19:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.802+0000 7fe2afa31b80 1 -- start start 2023-10-06T05:19:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f5077a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f50774a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f505661e0 con 0x555f507ce4d0 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f507a1830 con 0x555f507c5450 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f507ae770 con 0x555f507aaf40 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ad320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57910/0 (socket says 172.21.15.45:57910) 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 -- 172.21.15.45:0/1392797127 learned_addr learned my addr 172.21.15.45:0/1392797127 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe29ffff700 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f5077a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f50774a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 -- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 msgr2=0x555f5077a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f5077a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 -- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 msgr2=0x555f50774a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f50774a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 -- 172.21.15.45:0/1392797127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f5069d3c0 con 0x555f507ce4d0 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f50774a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.803+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ad320 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe29800f790 tx=0x7fe29801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f529713212eb1c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe285ffb700 1 -- 172.21.15.45:0/1392797127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298025070 con 0x555f507ce4d0 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe285ffb700 1 -- 172.21.15.45:0/1392797127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe298014470 con 0x555f507ce4d0 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe285ffb700 1 -- 172.21.15.45:0/1392797127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298000f40 con 0x555f507ce4d0 2023-10-06T05:19:02.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 -- 172.21.15.45:0/1392797127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 msgr2=0x555f507ad320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ad320 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe29800f790 tx=0x7fe29801a000 comp rx=0 tx=0).stop 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 -- 172.21.15.45:0/1392797127 shutdown_connections 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507ad320 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f5077a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/1392797127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f50774a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 -- 172.21.15.45:0/1392797127 >> 172.21.15.45:0/1392797127 conn(0x555f507c8880 msgr2=0x555f507a3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 -- 172.21.15.45:0/1392797127 shutdown_connections 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.804+0000 7fe2afa31b80 1 -- 172.21.15.45:0/1392797127 wait complete. 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.805+0000 7fe2afa31b80 1 -- start start 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.805+0000 7fe2afa31b80 1 Processor -- start 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2afa31b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5082b470 con 0x555f507ce4d0 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f507c9090 con 0x555f507c5450 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2afa31b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f507ae3f0 con 0x555f507aaf40 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2afa31b80 10 client.0 _ll_register_callbacks cb 0x555f504dda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57922/0 (socket says 172.21.15.45:57922) 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 -- 172.21.15.45:0/2791135683 learned_addr learned my addr 172.21.15.45:0/2791135683 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe29f7fe700 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 -- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 msgr2=0x555f507ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 -- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 msgr2=0x555f507c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 -- 172.21.15.45:0/2791135683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f504d4790 con 0x555f507ce4d0 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe2a4cba700 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507c8100 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe298004180 tx=0x7fe298025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4e81b45b12c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.806+0000 7fe27e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2791135683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29802f070 con 0x555f507ce4d0 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2791135683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fe298006db0 con 0x555f507ce4d0 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe29800fb50 con 0x555f507ce4d0 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2791135683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe298014a70 con 0x555f507ce4d0 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 10 client.6501 fetch_fsmap learned FSMap version 62 2023-10-06T05:19:02.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x555f505351d0 con 0x555f507ce4d0 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2791135683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29800fb50 con 0x555f507ce4d0 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2791135683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 62) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe298012070 con 0x555f507ce4d0 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 10 client.6501 fetch_fsmap finished waiting for FSMap version 62 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 10 client.6501 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 20 client.6501 populate_metadata read hostname 'smithi045' 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f5082a340 0x555f5082a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 4 client.6501 mds_command: new command op to 5543 tid=0[{"prefix": "status"}] 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 --> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555f506718d0 con 0x555f5082a340 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.807+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f5082a340 0x555f5082a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe27e7fc700 10 client.6501 ms_handle_connect on v2:172.21.15.144:6835/517622572 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe29ffff700 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f5082a340 0x555f5082a710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23ce560aefe7a01 server_cookie=bb6db709526392b1 in_seq=0 out_seq=0 2023-10-06T05:19:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe27e7fc700 1 -- 172.21.15.45:0/2791135683 <== mds.0 v2:172.21.15.144:6835/517622572 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555f506718d0 con 0x555f5082a340 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe27e7fc700 10 client.6501 handle_command_reply: tid=0 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 client.6501 shutdown 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 msgr2=0x555f507c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507c8100 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe298004180 tx=0x7fe298025000 comp rx=0 tx=0).stop 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 shutdown_connections 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x555f5082a340 0x555f5082a710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f507ce4d0 0x555f507c8100 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555f507aaf40 0x555f507ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe27e7fc700 0 client.6501 ms_handle_reset on v2:172.21.15.144:6835/517622572 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 --2- 172.21.15.45:0/2791135683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f507c5450 0x555f507c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 >> 172.21.15.45:0/2791135683 conn(0x555f507c8880 msgr2=0x555f507a3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 shutdown_connections 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 1 -- 172.21.15.45:0/2791135683 wait complete. 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.808+0000 7fe2afa31b80 20 client.6501 trim_cache size 0 max 16384 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5543, 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:19:02.820 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:19:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:19:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.8994940500000004, 2023-10-06T05:19:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 62, 2023-10-06T05:19:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 68, 2023-10-06T05:19:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 68, 2023-10-06T05:19:02.821 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 59.373062152999999 2023-10-06T05:19:02.821 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe28c066f60 msgr2=0x7fe28c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe28c066f60 0x7fe28c069410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0068a90 tx=0x7fe294016000 comp rx=0 tx=0).stop 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a0069b90 msgr2=0x7fe2a01583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a0069b90 0x7fe2a01583c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe29000c830 tx=0x7fe290008040 comp rx=0 tx=0).stop 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 shutdown_connections 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe28c066f60 0x7fe28c069410 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2a015d040 0x7fe2a00b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2a0069b90 0x7fe2a01583c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 --2- 172.21.15.45:0/1682969730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2a0159170 0x7fe2a00a6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:02.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 >> 172.21.15.45:0/1682969730 conn(0x7fe2a006f7c0 msgr2=0x7fe2a00cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:02.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 shutdown_connections 2023-10-06T05:19:02.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:02.809+0000 7fe2a671d700 1 -- 172.21.15.45:0/1682969730 wait complete. 2023-10-06T05:19:03.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T05:19:05.079 INFO:tasks.ceph.mds.d.smithi144.stderr:2023-10-06T05:19:05.078+0000 7f53b2120700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:19:06.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:06.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:14.532+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:14.532+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:19:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:14.532+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:19:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:14.532+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:19:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:14.532+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-10-06T05:19:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:29.548+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:29.548+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:19:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:29.548+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:19:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:29.548+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:19:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:29.548+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-10-06T05:19:37.057 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:37.060 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:44.551+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:44.551+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:19:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:44.551+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:19:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:44.551+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:19:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:44.551+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-10-06T05:19:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:59.566+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:59.566+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:19:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:59.566+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:19:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:59.566+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:19:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:19:59.566+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-10-06T05:20:07.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:07.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:14.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:14.582+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:14.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:14.582+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:20:14.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:14.582+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:20:14.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:14.582+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:20:14.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:14.582+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-10-06T05:20:29.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:29.596+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:29.596+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-10-06T05:20:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:29.596+0000 7efdcbfff700 1 -- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 msgr2=0x55ff96fb2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:20:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:29.596+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:20:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:29.596+0000 7efdcbfff700 1 --2- 172.21.15.45:0/1403546477 >> [v2:172.21.15.45:6832/4169661273,v1:172.21.15.45:6833/4169661273] conn(0x55ff9702b210 0x55ff96fb2020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-10-06T05:20:33.904 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-06T05:20:33.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:20:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.046+0000 7fb104d73700 1 Processor -- start 2023-10-06T05:20:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.046+0000 7fb104d73700 1 -- start start 2023-10-06T05:20:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb104d73700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb10006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb104d73700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb10006b5c0 0x7fb1000ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb104d73700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1000c7d00 con 0x7fb100159200 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb104d73700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1000c7780 con 0x7fb10006b5c0 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb104d73700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10006bb40 con 0x7fb1001595d0 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb10006b5c0 0x7fb1000ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb10006b5c0 0x7fb1000ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44102/0 (socket says 172.21.15.45:44102) 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fed9d700 1 -- 172.21.15.45:0/649971361 learned_addr learned my addr 172.21.15.45:0/649971361 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fdd9b700 1 --2- 172.21.15.45:0/649971361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb10006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fed9d700 1 -- 172.21.15.45:0/649971361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 msgr2=0x7fb10006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fed9d700 1 --2- 172.21.15.45:0/649971361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb10006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fd59a700 1 -- 172.21.15.45:0/649971361 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193794197 0 0) 0x7fb1000c7d00 con 0x7fb100159200 2023-10-06T05:20:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fed9d700 1 -- 172.21.15.45:0/649971361 >> v1:172.21.15.144:6789/0 conn(0x7fb100159200 legacy=0x7fb1000c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.047+0000 7fb0fed9d700 1 -- 172.21.15.45:0/649971361 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10006be20 con 0x7fb10006b5c0 2023-10-06T05:20:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb0fed9d700 1 --2- 172.21.15.45:0/649971361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10006b5c0 0x7fb1000ccdc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009f20 tx=0x7fb0f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703ff88f2eb0c1b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb0fd59a700 1 -- 172.21.15.45:0/649971361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800e070 con 0x7fb10006b5c0 2023-10-06T05:20:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb0fd59a700 1 -- 172.21.15.45:0/649971361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f800b9d0 con 0x7fb10006b5c0 2023-10-06T05:20:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb0fd59a700 1 -- 172.21.15.45:0/649971361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80092e0 con 0x7fb10006b5c0 2023-10-06T05:20:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb104d73700 1 -- 172.21.15.45:0/649971361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10006b5c0 msgr2=0x7fb1000ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb104d73700 1 --2- 172.21.15.45:0/649971361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10006b5c0 0x7fb1000ccdc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009f20 tx=0x7fb0f8009840 comp rx=0 tx=0).stop 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb104d73700 1 -- 172.21.15.45:0/649971361 shutdown_connections 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb104d73700 1 --2- 172.21.15.45:0/649971361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10006b5c0 0x7fb1000ccdc0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb104d73700 1 --2- 172.21.15.45:0/649971361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb10006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb104d73700 1 -- 172.21.15.45:0/649971361 >> 172.21.15.45:0/649971361 conn(0x7fb10006f090 msgr2=0x7fb10006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.048+0000 7fb104d73700 1 -- 172.21.15.45:0/649971361 shutdown_connections 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.049+0000 7fb104d73700 1 -- 172.21.15.45:0/649971361 wait complete. 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 Processor -- start 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 -- start start 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb10006b5c0 0x7fb1000cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100159200 0x7fb1000ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb1000cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000cafb0 con 0x7fb100159200 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000ccc30 con 0x7fb10006b5c0 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.050+0000 7fb104d73700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000ccf10 con 0x7fb1001595d0 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100159200 0x7fb1000ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100159200 0x7fb1000ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44114/0 (socket says 172.21.15.45:44114) 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 -- 172.21.15.45:0/1911999041 learned_addr learned my addr 172.21.15.45:0/1911999041 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fe59c700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb10006b5c0 0x7fb1000cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fed9d700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb1000cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 -- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 msgr2=0x7fb1000cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb1000cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 -- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb10006b5c0 msgr2=0x7fb1000cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb10006b5c0 0x7fb1000cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 -- 172.21.15.45:0/1911999041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f8009ae0 con 0x7fb100159200 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0fdd9b700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100159200 0x7fb1000ada60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400e930 tx=0x7fb0f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0ee7fc700 1 -- 172.21.15.45:0/1911999041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4013680 con 0x7fb100159200 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0ee7fc700 1 -- 172.21.15.45:0/1911999041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f40137e0 con 0x7fb100159200 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb0ee7fc700 1 -- 172.21.15.45:0/1911999041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4013b80 con 0x7fb100159200 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1000ca960 con 0x7fb100159200 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.051+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1000cb980 con 0x7fb100159200 2023-10-06T05:20:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.052+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10006e2f0 con 0x7fb100159200 2023-10-06T05:20:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.053+0000 7fb0ee7fc700 1 -- 172.21.15.45:0/1911999041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0f400c3e0 con 0x7fb100159200 2023-10-06T05:20:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.053+0000 7fb0ee7fc700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0e0066f30 0x7fb0e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.053+0000 7fb0ee7fc700 1 -- 172.21.15.45:0/1911999041 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fb0f4084930 con 0x7fb100159200 2023-10-06T05:20:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.053+0000 7fb0fe59c700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0e0066f30 0x7fb0e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.054+0000 7fb0fe59c700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0e0066f30 0x7fb0e00693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007fd0 tx=0x7fb0f0007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.058+0000 7fb0ee7fc700 1 -- 172.21.15.45:0/1911999041 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0f4051820 con 0x7fb100159200 2023-10-06T05:20:34.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.311+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb1000c7990 con 0x7fb100159200 2023-10-06T05:20:34.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.311+0000 7fb0ee7fc700 1 -- 172.21.15.45:0/1911999041 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0f4054e40 con 0x7fb100159200 2023-10-06T05:20:34.314 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:20:34.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0e0066f30 msgr2=0x7fb0e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0e0066f30 0x7fb0e00693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007fd0 tx=0x7fb0f0007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100159200 msgr2=0x7fb1000ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100159200 0x7fb1000ada60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400e930 tx=0x7fb0f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 shutdown_connections 2023-10-06T05:20:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0e0066f30 0x7fb0e00693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1001595d0 0x7fb1000cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb10006b5c0 0x7fb1000cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 --2- 172.21.15.45:0/1911999041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb100159200 0x7fb1000ada60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 >> 172.21.15.45:0/1911999041 conn(0x7fb10006f090 msgr2=0x7fb10015ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 shutdown_connections 2023-10-06T05:20:34.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:34.313+0000 7fb104d73700 1 -- 172.21.15.45:0/1911999041 wait complete. 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:20:34.329 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:20:34.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:34.330 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:20:34.351 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:20:34.351 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:34.352 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42107sec preferred_lft 42107sec 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:34.353 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:34.354 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:34.354 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:20:34.354 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:20:34.354 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:34.354 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:20:34.355 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:34.355 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:20:34.355 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:34.355 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:34.355 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:20:34.356 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:20:34.356 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:34.356 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:34.356 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:20:34.356 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:20:34.356 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:34.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:34.357 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:20:34.423 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:20:34.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:34.425 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:20:34.490 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:20:34.491 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:20:34.491 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:20:34.491 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:20:34.491 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:20:34.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:34.491 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:20:34.491 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:20:34.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:20:34.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:20:34.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:20:34.492 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:20:34.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:34.614 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:20:34.635 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:20:34.635 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:20:34.635 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:20:34.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:34.636 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:20:34.711 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:20:34.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:34.712 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:34.775 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:20:34.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:34.775 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:34.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:20:34.915 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:20:34.915 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:20:34.989 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:20:34.990 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:20:34.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:34.991 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:20:35.055 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:20:35.055 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:20:35.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:20:35.134 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:20:35.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:20:35.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:20:35.198+0000 7f49b84aa540 -1 init, newargv = 0x556c24ab0bc0 newargc=15 2023-10-06T05:20:35.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[148013]: starting ceph client 2023-10-06T05:20:35.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[148013]: starting fuse 2023-10-06T05:20:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:20:35.210 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:20:35.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:35.211 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:20:35.271 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:20:35.272 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:20:35.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:20:35.272 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:20:35.272 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:20:35.272 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:20:35.272 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:20:35.272 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:20:35.273 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:20:35.274 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:20:35.275 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:20:35.411 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.148013.asok 2023-10-06T05:20:35.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.148013.asok 2023-10-06T05:20:35.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:35.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.148013.asok status 2023-10-06T05:20:35.768 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:20:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:20:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:20:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:20:35.768 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "148013", 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6513, 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:20:35.769 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 6513 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2163910436 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2163910436 2023-10-06T05:20:35.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.6513 192.168.0.1:0/2163910436", 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2163910436", 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 62, 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 68, 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:20:35.771 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:20:35.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:35.779 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:20:35.779 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:35.800 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:20:35.800 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:35.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:35.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:35.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:35.873 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat a/ && find a/') 2023-10-06T05:20:35.960 INFO:teuthology.orchestra.run.smithi045.stdout: File: a/ 2023-10-06T05:20:35.960 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 1 IO Block: 4096 directory 2023-10-06T05:20:35.960 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 2fh/47d Inode: 1099511627776 Links: 4 2023-10-06T05:20:35.960 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1285/ ubuntu) 2023-10-06T05:20:35.960 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:fusefs_t:s0 2023-10-06T05:20:35.961 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-06 05:18:18.251311896 +0000 2023-10-06T05:20:35.961 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-06 05:18:18.258604140 +0000 2023-10-06T05:20:35.961 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-06 05:18:18.258604140 +0000 2023-10-06T05:20:35.961 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-10-06T05:20:35.975 INFO:teuthology.orchestra.run.smithi045.stdout:a/ 2023-10-06T05:20:35.975 INFO:teuthology.orchestra.run.smithi045.stdout:a/c 2023-10-06T05:20:35.975 INFO:teuthology.orchestra.run.smithi045.stdout:a/c/d 2023-10-06T05:20:35.975 INFO:teuthology.orchestra.run.smithi045.stdout:a/b 2023-10-06T05:20:35.975 INFO:teuthology.orchestra.run.smithi045.stdout:a/b/d 2023-10-06T05:20:35.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2023-10-06T05:20:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.114+0000 7f5670d5c700 1 Processor -- start 2023-10-06T05:20:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.114+0000 7f5670d5c700 1 -- start start 2023-10-06T05:20:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f5670d5c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f566c06ae40 0x7f566c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f5670d5c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f566c15c6f0 0x7f566c0aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f5670d5c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566c067d80 con 0x7f566c06aa70 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f5670d5c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c0ac620 con 0x7f566c15c6f0 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f5670d5c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c15cb70 con 0x7f566c06ae40 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f566ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f566c15c6f0 0x7f566c0aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f566a59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f566c06aa70 0x7f566c0c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57116/0 (socket says 172.21.15.45:57116) 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f566a59c700 1 -- 172.21.15.45:0/128823935 learned_addr learned my addr 172.21.15.45:0/128823935 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f5669d9b700 1 --2- 172.21.15.45:0/128823935 >> v2:172.21.15.144:3300/0 conn(0x7f566c06ae40 0x7f566c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f566ad9d700 1 -- 172.21.15.45:0/128823935 >> v2:172.21.15.144:3300/0 conn(0x7f566c06ae40 msgr2=0x7f566c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f566ad9d700 1 --2- 172.21.15.45:0/128823935 >> v2:172.21.15.144:3300/0 conn(0x7f566c06ae40 0x7f566c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.115+0000 7f566ad9d700 1 -- 172.21.15.45:0/128823935 >> v1:172.21.15.45:6789/0 conn(0x7f566c06aa70 legacy=0x7f566c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.116+0000 7f566ad9d700 1 -- 172.21.15.45:0/128823935 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0c75b0 con 0x7f566c15c6f0 2023-10-06T05:20:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.116+0000 7f566ad9d700 1 --2- 172.21.15.45:0/128823935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c0aca70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f566c0cd5e0 tx=0x7f5664009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ef79e6c49d3682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.116+0000 7f566959a700 1 -- 172.21.15.45:0/128823935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566400e070 con 0x7f566c15c6f0 2023-10-06T05:20:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.116+0000 7f566959a700 1 -- 172.21.15.45:0/128823935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566400bc30 con 0x7f566c15c6f0 2023-10-06T05:20:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.116+0000 7f566959a700 1 -- 172.21.15.45:0/128823935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664012430 con 0x7f566c15c6f0 2023-10-06T05:20:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.116+0000 7f5670d5c700 1 -- 172.21.15.45:0/128823935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 msgr2=0x7f566c0aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.116+0000 7f5670d5c700 1 --2- 172.21.15.45:0/128823935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c0aca70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f566c0cd5e0 tx=0x7f5664009780 comp rx=0 tx=0).stop 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.117+0000 7f5670d5c700 1 -- 172.21.15.45:0/128823935 shutdown_connections 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.117+0000 7f5670d5c700 1 --2- 172.21.15.45:0/128823935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c0aca70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.117+0000 7f5670d5c700 1 --2- 172.21.15.45:0/128823935 >> v2:172.21.15.144:3300/0 conn(0x7f566c06ae40 0x7f566c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.117+0000 7f5670d5c700 1 -- 172.21.15.45:0/128823935 >> 172.21.15.45:0/128823935 conn(0x7f566c0cc570 msgr2=0x7f566c0cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.117+0000 7f5670d5c700 1 -- 172.21.15.45:0/128823935 shutdown_connections 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.117+0000 7f5670d5c700 1 -- 172.21.15.45:0/128823935 wait complete. 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.118+0000 7f5670d5c700 1 Processor -- start 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.118+0000 7f5670d5c700 1 -- start start 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.118+0000 7f5670d5c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566c06aa70 0x7f566c15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5670d5c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f566c06ae40 0x7f566c155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5670d5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5670d5c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c1563e0 con 0x7f566c15c6f0 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5670d5c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c159d00 con 0x7f566c06ae40 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5670d5c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c159fe0 con 0x7f566c06aa70 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44142/0 (socket says 172.21.15.45:44142) 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 -- 172.21.15.45:0/922875998 learned_addr learned my addr 172.21.15.45:0/922875998 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566a59c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566c06aa70 0x7f566c15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5669d9b700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f566c06ae40 0x7f566c155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 -- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566c06aa70 msgr2=0x7f566c15ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566c06aa70 0x7f566c15ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 -- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f566c06ae40 msgr2=0x7f566c155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f566c06ae40 0x7f566c155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 -- 172.21.15.45:0/922875998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664009a20 con 0x7f566c15c6f0 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566a59c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566c06aa70 0x7f566c15ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f566ad9d700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c155ea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5664013000 tx=0x7f5664019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f565a7fc700 1 -- 172.21.15.45:0/922875998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664012e40 con 0x7f566c15c6f0 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f565a7fc700 1 -- 172.21.15.45:0/922875998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566401e630 con 0x7f566c15c6f0 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c15a2c0 con 0x7f566c15c6f0 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f565a7fc700 1 -- 172.21.15.45:0/922875998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566401e9f0 con 0x7f566c15c6f0 2023-10-06T05:20:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.119+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c0af720 con 0x7f566c15c6f0 2023-10-06T05:20:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.121+0000 7f565a7fc700 1 -- 172.21.15.45:0/922875998 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5664024070 con 0x7f566c15c6f0 2023-10-06T05:20:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.122+0000 7f565a7fc700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f564c066f30 0x7f564c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.122+0000 7f565a7fc700 1 -- 172.21.15.45:0/922875998 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f5664087070 con 0x7f566c15c6f0 2023-10-06T05:20:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.122+0000 7f566a59c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f564c066f30 0x7f564c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.122+0000 7f566a59c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f564c066f30 0x7f564c0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f565c0018c0 tx=0x7f565c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 Processor -- start 2023-10-06T05:20:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 -- start start 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b3097b4a0 0x559b3097b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30960f90 0x559b30983fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30984990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3084e360 con 0x559b30960f90 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3088b180 con 0x559b3097b4a0 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.124+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b30888b40 con 0x559b30984520 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30960f90 0x559b30983fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30960f90 0x559b30983fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44148/0 (socket says 172.21.15.45:44148) 2023-10-06T05:20:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 -- 172.21.15.45:0/2367965589 learned_addr learned my addr 172.21.15.45:0/2367965589 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f566a59c700 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b3097b4a0 0x559b3097b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f566ad9d700 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30984990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 -- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 msgr2=0x559b30984990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30984990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 -- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b3097b4a0 msgr2=0x559b3097b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b3097b4a0 0x559b3097b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 -- 172.21.15.45:0/2367965589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3068a7e0 con 0x559b30960f90 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f566ad9d700 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30984990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f5669d9b700 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30960f90 0x559b30983fe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5660007a50 tx=0x7f5660008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f6c65a0cd90fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f564b7fe700 1 -- 172.21.15.45:0/2367965589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56600109c0 con 0x559b30960f90 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f564b7fe700 1 -- 172.21.15.45:0/2367965589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660010b20 con 0x559b30960f90 2023-10-06T05:20:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.125+0000 7f564b7fe700 1 -- 172.21.15.45:0/2367965589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660010ea0 con 0x559b30960f90 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 -- 172.21.15.45:0/2367965589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30960f90 msgr2=0x559b30983fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30960f90 0x559b30983fe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5660007a50 tx=0x7f5660008040 comp rx=0 tx=0).stop 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 -- 172.21.15.45:0/2367965589 shutdown_connections 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30984990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b3097b4a0 0x559b3097b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 --2- 172.21.15.45:0/2367965589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30960f90 0x559b30983fe0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 -- 172.21.15.45:0/2367965589 >> 172.21.15.45:0/2367965589 conn(0x559b3097e8d0 msgr2=0x559b3097eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 -- 172.21.15.45:0/2367965589 shutdown_connections 2023-10-06T05:20:36.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.126+0000 7f567a070b80 1 -- 172.21.15.45:0/2367965589 wait complete. 2023-10-06T05:20:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.127+0000 7f567a070b80 1 -- start start 2023-10-06T05:20:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.127+0000 7f567a070b80 1 Processor -- start 2023-10-06T05:20:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f567a070b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b30935e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30956370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f567a070b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b30884930 con 0x559b3097b4a0 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b308309d0 con 0x559b30960f90 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b30830670 con 0x559b30984520 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f567a070b80 10 client.0 _ll_register_callbacks cb 0x559b30693a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b30935e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b30935e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44156/0 (socket says 172.21.15.45:44156) 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 -- 172.21.15.45:0/573471362 learned_addr learned my addr 172.21.15.45:0/573471362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f566a59c700 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f566ad9d700 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30956370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 -- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 msgr2=0x559b30956370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30956370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 -- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 msgr2=0x559b309556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.128+0000 7f5669d9b700 1 -- 172.21.15.45:0/573471362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566000e040 con 0x559b3097b4a0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f5669d9b700 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b30935e70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5660017000 tx=0x7f566001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b045ea28bb0b2b35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f5648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f5648ff9700 1 -- 172.21.15.45:0/573471362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56600149a0 con 0x559b3097b4a0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f5648ff9700 1 -- 172.21.15.45:0/573471362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f5660014b00 con 0x559b3097b4a0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b30928db0 con 0x559b3097b4a0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f5648ff9700 1 -- 172.21.15.45:0/573471362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f566001d450 con 0x559b3097b4a0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 10 client.6528 fetch_fsmap learned FSMap version 62 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x559b307770f0 con 0x559b3097b4a0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f5648ff9700 1 -- 172.21.15.45:0/573471362 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559b30928db0 con 0x559b3097b4a0 2023-10-06T05:20:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f5648ff9700 1 -- 172.21.15.45:0/573471362 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 62) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5660026070 con 0x559b3097b4a0 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 10 client.6528 fetch_fsmap finished waiting for FSMap version 62 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 10 client.6528 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 20 client.6528 populate_metadata read hostname 'smithi045' 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b309363b0 0x559b3095a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 4 client.6528 mds_command: new command op to 5543 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.129+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 --> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b30905b80 con 0x559b309363b0 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.130+0000 7f566a59c700 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b309363b0 0x559b3095a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.130+0000 7f5648ff9700 10 client.6528 ms_handle_connect on v2:172.21.15.144:6835/517622572 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.130+0000 7f566a59c700 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b309363b0 0x559b3095a440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e939bdadfa56d5 server_cookie=50e793a2e37e6fb4 in_seq=0 out_seq=0 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f5648ff9700 1 -- 172.21.15.45:0/573471362 <== mds.0 v2:172.21.15.144:6835/517622572 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559b30905b80 con 0x559b309363b0 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f5648ff9700 10 client.6528 handle_command_reply: tid=0 2023-10-06T05:20:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 client.6528 shutdown 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 msgr2=0x559b30935e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b30935e70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5660017000 tx=0x7f566001d000 comp rx=0 tx=0).stop 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 shutdown_connections 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b309363b0 0x559b3095a440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30956370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f5648ff9700 0 client.6528 ms_handle_reset on v2:172.21.15.144:6835/517622572 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 --2- 172.21.15.45:0/573471362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b30935e70 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 >> 172.21.15.45:0/573471362 conn(0x559b3097e8d0 msgr2=0x559b30930cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 shutdown_connections 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 1 -- 172.21.15.45:0/573471362 wait complete. 2023-10-06T05:20:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.131+0000 7f567a070b80 20 client.6528 trim_cache size 0 max 16384 2023-10-06T05:20:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 Processor -- start 2023-10-06T05:20:36.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 -- start start 2023-10-06T05:20:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b30983f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b3097acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30930110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3071c230 con 0x559b3097b4a0 2023-10-06T05:20:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3092cc80 con 0x559b30960f90 2023-10-06T05:20:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.146+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b30935f90 con 0x559b30984520 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b3097acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b3097acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44158/0 (socket says 172.21.15.45:44158) 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 -- 172.21.15.45:0/925061542 learned_addr learned my addr 172.21.15.45:0/925061542 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f566ad9d700 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30930110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f566a59c700 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b30983f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 -- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 msgr2=0x559b30930110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30930110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 -- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 msgr2=0x559b30983f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b30983f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 -- 172.21.15.45:0/925061542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b30853410 con 0x559b3097b4a0 2023-10-06T05:20:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f5669d9b700 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b3097acb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5660007920 tx=0x7f566001f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac938f4b6517b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f564bfff700 1 -- 172.21.15.45:0/925061542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566001d450 con 0x559b3097b4a0 2023-10-06T05:20:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f564bfff700 1 -- 172.21.15.45:0/925061542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660023040 con 0x559b3097b4a0 2023-10-06T05:20:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.147+0000 7f564bfff700 1 -- 172.21.15.45:0/925061542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660002f30 con 0x559b3097b4a0 2023-10-06T05:20:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 -- 172.21.15.45:0/925061542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 msgr2=0x559b3097acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b3097acb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5660007920 tx=0x7f566001f070 comp rx=0 tx=0).stop 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 -- 172.21.15.45:0/925061542 shutdown_connections 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b30984520 0x559b30930110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b30983f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 --2- 172.21.15.45:0/925061542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3097b4a0 0x559b3097acb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 -- 172.21.15.45:0/925061542 >> 172.21.15.45:0/925061542 conn(0x559b3097e8d0 msgr2=0x559b3095b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 -- 172.21.15.45:0/925061542 shutdown_connections 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 -- 172.21.15.45:0/925061542 wait complete. 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.148+0000 7f567a070b80 1 -- start start 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.149+0000 7f567a070b80 1 Processor -- start 2023-10-06T05:20:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b3097b4a0 0x559b309d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30984520 0x559b3095cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b30934c70 con 0x559b30984520 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b30930bf0 con 0x559b30960f90 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b309348d0 con 0x559b3097b4a0 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 10 client.0 _ll_register_callbacks cb 0x559b30693a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30984520 0x559b3095cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30984520 0x559b3095cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44170/0 (socket says 172.21.15.45:44170) 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 -- 172.21.15.45:0/4208020690 learned_addr learned my addr 172.21.15.45:0/4208020690 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f5669d9b700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b3097b4a0 0x559b309d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566a59c700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 -- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b3097b4a0 msgr2=0x559b309d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b3097b4a0 0x559b309d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 -- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 msgr2=0x559b309942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 -- 172.21.15.45:0/4208020690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b307770f0 con 0x559b30984520 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566a59c700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f566ad9d700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30984520 0x559b3095cad0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f56640099f0 tx=0x7f566405d0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadbb63391c265f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f5648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f5648ff9700 1 -- 172.21.15.45:0/4208020690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566400e040 con 0x559b30984520 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f5648ff9700 1 -- 172.21.15.45:0/4208020690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f566401ed60 con 0x559b30984520 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5660010b20 con 0x559b30984520 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.150+0000 7f5648ff9700 1 -- 172.21.15.45:0/4208020690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f566405e940 con 0x559b30984520 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 10 client.6534 fetch_fsmap learned FSMap version 62 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x559b306eb220 con 0x559b30984520 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f5648ff9700 1 -- 172.21.15.45:0/4208020690 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660010b20 con 0x559b30984520 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f5648ff9700 1 -- 172.21.15.45:0/4208020690 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 62) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f566405bbb0 con 0x559b30984520 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 10 client.6534 fetch_fsmap finished waiting for FSMap version 62 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 10 client.6534 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-10-06T05:20:36.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 20 client.6534 populate_metadata read hostname 'smithi045' 2023-10-06T05:20:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b30930660 0x559b3098c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 4 client.6534 mds_command: new command op to 5543 tid=0[{"prefix": "flush journal"}] 2023-10-06T05:20:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 --> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559b30827920 con 0x559b30930660 2023-10-06T05:20:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f566a59c700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b30930660 0x559b3098c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f566a59c700 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b30930660 0x559b3098c950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96cda784542da008 server_cookie=d393c9e7c90d7e94 in_seq=0 out_seq=0 2023-10-06T05:20:36.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.151+0000 7f5648ff9700 10 client.6534 ms_handle_connect on v2:172.21.15.144:6835/517622572 2023-10-06T05:20:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f5648ff9700 1 -- 172.21.15.45:0/4208020690 <== mds.0 v2:172.21.15.144:6835/517622572 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559b30827920 con 0x559b30930660 2023-10-06T05:20:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f5648ff9700 10 client.6534 handle_command_reply: tid=0 2023-10-06T05:20:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 client.6534 shutdown 2023-10-06T05:20:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30984520 msgr2=0x559b3095cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30984520 0x559b3095cad0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f56640099f0 tx=0x7f566405d0b0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 shutdown_connections 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:6835/517622572,v1:172.21.15.144:6837/517622572] conn(0x559b30930660 0x559b3098c950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b30984520 0x559b3095cad0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b30960f90 0x559b309942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f5648ff9700 0 client.6534 ms_handle_reset on v2:172.21.15.144:6835/517622572 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 --2- 172.21.15.45:0/4208020690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559b3097b4a0 0x559b309d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 >> 172.21.15.45:0/4208020690 conn(0x559b3097e8d0 msgr2=0x559b309310d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 shutdown_connections 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 1 -- 172.21.15.45:0/4208020690 wait complete. 2023-10-06T05:20:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.161+0000 7f567a070b80 20 client.6534 trim_cache size 0 max 16384 2023-10-06T05:20:36.165 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:20:36.165 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2023-10-06T05:20:36.165 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2023-10-06T05:20:36.165 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f564c066f30 msgr2=0x7f564c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f564c066f30 0x7f564c0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f565c0018c0 tx=0x7f565c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 msgr2=0x7f566c155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c155ea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5664013000 tx=0x7f5664019000 comp rx=0 tx=0).stop 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 shutdown_connections 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f564c066f30 0x7f564c0693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c15c6f0 0x7f566c155ea0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566c06aa70 0x7f566c15ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 --2- 172.21.15.45:0/922875998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f566c06ae40 0x7f566c155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 >> 172.21.15.45:0/922875998 conn(0x7f566c0cc570 msgr2=0x7f566c156c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 shutdown_connections 2023-10-06T05:20:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.162+0000 7f5670d5c700 1 -- 172.21.15.45:0/922875998 wait complete. 2023-10-06T05:20:36.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:20:36.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:36.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:36.275 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:20:36.275 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:20:36.275 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:20:36.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:36.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[148013]: fuse finished with error 0 and tester_r 0 2023-10-06T05:20:36.318 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:20:36.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:36.318 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:20:36.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:20:36.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:36.392 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:20:36.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:20:36.415 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T05:20:36.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:20:36.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:36.416 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:20:36.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:20:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:20:36.457 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:20:36.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:36.457 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:20:36.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:20:36.479 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:20:36.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:36.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.635+0000 7f38f73bf700 1 Processor -- start 2023-10-06T05:20:36.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.635+0000 7f38f73bf700 1 -- start start 2023-10-06T05:20:36.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f73bf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f73bf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38f00caad0 0x7f38f00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f73bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38f0156af0 0x7f38f0156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f73bf700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f015b000 con 0x7f38f0156af0 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f73bf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f015b2e0 con 0x7f38f00caad0 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f73bf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f015b5c0 con 0x7f38f00c8ff0 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f595c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38f0156af0 0x7f38f0156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f595c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38f0156af0 0x7f38f0156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35828/0 (socket says 172.21.15.45:35828) 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f595c700 1 -- 172.21.15.45:0/700688962 learned_addr learned my addr 172.21.15.45:0/700688962 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f495a700 1 --2- 172.21.15.45:0/700688962 >> v2:172.21.15.144:3300/0 conn(0x7f38f00caad0 0x7f38f00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.636+0000 7f38f515b700 1 --2- 172.21.15.45:0/700688962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38f595c700 1 -- 172.21.15.45:0/700688962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 msgr2=0x7f38f00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38f595c700 1 --2- 172.21.15.45:0/700688962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38f595c700 1 -- 172.21.15.45:0/700688962 >> v2:172.21.15.144:3300/0 conn(0x7f38f00caad0 msgr2=0x7f38f00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38f595c700 1 --2- 172.21.15.45:0/700688962 >> v2:172.21.15.144:3300/0 conn(0x7f38f00caad0 0x7f38f00caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38f595c700 1 -- 172.21.15.45:0/700688962 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f015b8a0 con 0x7f38f0156af0 2023-10-06T05:20:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38f595c700 1 --2- 172.21.15.45:0/700688962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f0156af0 0x7f38f0156f70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f38f00c4fb0 tx=0x7f38e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec53b6c126c768d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38e3fff700 1 -- 172.21.15.45:0/700688962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e800e070 con 0x7f38f0156af0 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38e3fff700 1 -- 172.21.15.45:0/700688962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e800b970 con 0x7f38f0156af0 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.637+0000 7f38e3fff700 1 -- 172.21.15.45:0/700688962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80092e0 con 0x7f38f0156af0 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 -- 172.21.15.45:0/700688962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f0156af0 msgr2=0x7f38f0156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 --2- 172.21.15.45:0/700688962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f0156af0 0x7f38f0156f70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f38f00c4fb0 tx=0x7f38e8009840 comp rx=0 tx=0).stop 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 -- 172.21.15.45:0/700688962 shutdown_connections 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 --2- 172.21.15.45:0/700688962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f0156af0 0x7f38f0156f70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 --2- 172.21.15.45:0/700688962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 --2- 172.21.15.45:0/700688962 >> v2:172.21.15.144:3300/0 conn(0x7f38f00caad0 0x7f38f00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 -- 172.21.15.45:0/700688962 >> 172.21.15.45:0/700688962 conn(0x7f38f00cdf50 msgr2=0x7f38f00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 -- 172.21.15.45:0/700688962 shutdown_connections 2023-10-06T05:20:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.638+0000 7f38f73bf700 1 -- 172.21.15.45:0/700688962 wait complete. 2023-10-06T05:20:36.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.639+0000 7f38f73bf700 1 Processor -- start 2023-10-06T05:20:36.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.639+0000 7f38f73bf700 1 -- start start 2023-10-06T05:20:36.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f73bf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f0154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f73bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f00caad0 0x7f38f0149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f73bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f0156af0 0x7f38f00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f73bf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0159360 con 0x7f38f00caad0 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f73bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0149fe0 con 0x7f38f0156af0 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f73bf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0167430 con 0x7f38f00c8ff0 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f495a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f00caad0 0x7f38f0149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f495a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f00caad0 0x7f38f0149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35834/0 (socket says 172.21.15.45:35834) 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f495a700 1 -- 172.21.15.45:0/2389047184 learned_addr learned my addr 172.21.15.45:0/2389047184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f515b700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f0154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.640+0000 7f38f595c700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f0156af0 0x7f38f00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f495a700 1 -- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 msgr2=0x7f38f0154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f495a700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f0154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f495a700 1 -- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f0156af0 msgr2=0x7f38f00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f495a700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f0156af0 0x7f38f00b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f495a700 1 -- 172.21.15.45:0/2389047184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8009ae0 con 0x7f38f00caad0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f495a700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f00caad0 0x7f38f0149aa0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00ecb0 tx=0x7f38ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38e17fa700 1 -- 172.21.15.45:0/2389047184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00c850 con 0x7f38f00caad0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38e17fa700 1 -- 172.21.15.45:0/2389047184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec00c9b0 con 0x7f38f00caad0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38e17fa700 1 -- 172.21.15.45:0/2389047184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00cd50 con 0x7f38f00caad0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f01679f0 con 0x7f38f00caad0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.641+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f00b31e0 con 0x7f38f00caad0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.642+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f0071500 con 0x7f38f00caad0 2023-10-06T05:20:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.642+0000 7f38e17fa700 1 -- 172.21.15.45:0/2389047184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38ec013680 con 0x7f38f00caad0 2023-10-06T05:20:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.643+0000 7f38e17fa700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38d4066f30 0x7f38d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.643+0000 7f38e17fa700 1 -- 172.21.15.45:0/2389047184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f38ec084b10 con 0x7f38f00caad0 2023-10-06T05:20:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.643+0000 7f38f515b700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38d4066f30 0x7f38d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.644+0000 7f38f515b700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38d4066f30 0x7f38d40693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38f00cded0 tx=0x7f38e40091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.647+0000 7f38e17fa700 1 -- 172.21.15.45:0/2389047184 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38ec051a00 con 0x7f38f00caad0 2023-10-06T05:20:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.897+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38f004bb40 con 0x7f38f00caad0 2023-10-06T05:20:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.898+0000 7f38e17fa700 1 -- 172.21.15.45:0/2389047184 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f38ec055020 con 0x7f38f00caad0 2023-10-06T05:20:36.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:36.900 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1230704519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1230704519},{"type":"v1","addr":"172.21.15.45:6833","nonce":1230704519}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:19:00.913269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5543":{"gid":5543,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38d4066f30 msgr2=0x7f38d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38d4066f30 0x7f38d40693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38f00cded0 tx=0x7f38e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f00caad0 msgr2=0x7f38f0149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f00caad0 0x7f38f0149aa0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00ecb0 tx=0x7f38ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 shutdown_connections 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38d4066f30 0x7f38d40693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f0156af0 0x7f38f00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f00c8ff0 0x7f38f0154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 --2- 172.21.15.45:0/2389047184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38f00caad0 0x7f38f0149aa0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 >> 172.21.15.45:0/2389047184 conn(0x7f38f00cdf50 msgr2=0x7f38f00adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 shutdown_connections 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:36.900+0000 7f38f73bf700 1 -- 172.21.15.45:0/2389047184 wait complete. 2023-10-06T05:20:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 62 2023-10-06T05:20:36.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.044+0000 7f50ef2bc700 1 Processor -- start 2023-10-06T05:20:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.045+0000 7f50ef2bc700 1 -- start start 2023-10-06T05:20:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.045+0000 7f50ef2bc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e8156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.045+0000 7f50ef2bc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f50e8157030 0x7f50e80cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.045+0000 7f50ef2bc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e815b000 con 0x7f50e80c8ff0 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.045+0000 7f50ef2bc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e80c9830 con 0x7f50e8157030 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.045+0000 7f50ef2bc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e815b2e0 con 0x7f50e80c93c0 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed058700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f50e80c8ff0 0x7f50e80caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33226/0 (socket says 172.21.15.45:33226) 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed058700 1 -- 172.21.15.45:0/1815641674 learned_addr learned my addr 172.21.15.45:0/1815641674 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed859700 1 --2- 172.21.15.45:0/1815641674 >> v2:172.21.15.144:3300/0 conn(0x7f50e8157030 0x7f50e80cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ec857700 1 --2- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e8156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50dbfff700 1 -- 172.21.15.45:0/1815641674 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602265478 0 0) 0x7f50e815b000 con 0x7f50e80c8ff0 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50dbfff700 1 -- 172.21.15.45:0/1815641674 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d4003570 con 0x7f50e80c8ff0 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed859700 1 -- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 msgr2=0x7f50e8156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed859700 1 --2- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e8156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed859700 1 -- 172.21.15.45:0/1815641674 >> v1:172.21.15.45:6789/0 conn(0x7f50e80c8ff0 legacy=0x7f50e80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50dbfff700 1 -- 172.21.15.45:0/1815641674 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1019587631 0 0) 0x7f50d4003570 con 0x7f50e80c8ff0 2023-10-06T05:20:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed859700 1 -- 172.21.15.45:0/1815641674 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e815b5c0 con 0x7f50e8157030 2023-10-06T05:20:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ec857700 1 --2- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e8156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.046+0000 7f50ed859700 1 --2- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e8157030 0x7f50e80cc330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e60 tx=0x7f50e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b6bd9633d840f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50dbfff700 1 -- 172.21.15.45:0/1815641674 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000e070 con 0x7f50e8157030 2023-10-06T05:20:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50dbfff700 1 -- 172.21.15.45:0/1815641674 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e000b7a0 con 0x7f50e8157030 2023-10-06T05:20:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50dbfff700 1 -- 172.21.15.45:0/1815641674 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4001470 con 0x7f50e8157030 2023-10-06T05:20:37.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 -- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e8157030 msgr2=0x7f50e80cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e8157030 0x7f50e80cc330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e60 tx=0x7f50e0009780 comp rx=0 tx=0).stop 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 -- 172.21.15.45:0/1815641674 shutdown_connections 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e8157030 0x7f50e80cc330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/1815641674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e8156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 -- 172.21.15.45:0/1815641674 >> 172.21.15.45:0/1815641674 conn(0x7f50e80cdf50 msgr2=0x7f50e80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 -- 172.21.15.45:0/1815641674 shutdown_connections 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.047+0000 7f50ef2bc700 1 -- 172.21.15.45:0/1815641674 wait complete. 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.048+0000 7f50ef2bc700 1 Processor -- start 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.049+0000 7f50ef2bc700 1 -- start start 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.049+0000 7f50ef2bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e80c8ff0 0x7f50e8069c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.049+0000 7f50ef2bc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e80b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.049+0000 7f50ef2bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e8157030 0x7f50e80ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.049+0000 7f50ef2bc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e806a4d0 con 0x7f50e8157030 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ef2bc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80b0cf0 con 0x7f50e80c8ff0 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ef2bc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e8069910 con 0x7f50e80c93c0 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e8157030 0x7f50e80ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e8157030 0x7f50e80ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35836/0 (socket says 172.21.15.45:35836) 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 -- 172.21.15.45:0/2782006823 learned_addr learned my addr 172.21.15.45:0/2782006823 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed058700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e80c8ff0 0x7f50e8069c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ec857700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e80b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 -- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 msgr2=0x7f50e80b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e80b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 -- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e80c8ff0 msgr2=0x7f50e8069c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e80c8ff0 0x7f50e8069c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 -- 172.21.15.45:0/2782006823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc009420 con 0x7f50e8157030 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ed859700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e8157030 0x7f50e80ad0d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c4f80 tx=0x7f50e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50d97fa700 1 -- 172.21.15.45:0/2782006823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0012bc0 con 0x7f50e8157030 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50d97fa700 1 -- 172.21.15.45:0/2782006823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0012d60 con 0x7f50e8157030 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e0009a20 con 0x7f50e8157030 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50d97fa700 1 -- 172.21.15.45:0/2782006823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e001d680 con 0x7f50e8157030 2023-10-06T05:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.050+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e806a1f0 con 0x7f50e8157030 2023-10-06T05:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.051+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e806f410 con 0x7f50e8157030 2023-10-06T05:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.052+0000 7f50d97fa700 1 -- 172.21.15.45:0/2782006823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50e0024070 con 0x7f50e8157030 2023-10-06T05:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.052+0000 7f50d97fa700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50cc066f30 0x7f50cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.052+0000 7f50d97fa700 1 -- 172.21.15.45:0/2782006823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f50e0086440 con 0x7f50e8157030 2023-10-06T05:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.053+0000 7f50ed058700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50cc066f30 0x7f50cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.053+0000 7f50ed058700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50cc066f30 0x7f50cc0693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f50e40011e0 tx=0x7f50e4001a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.057+0000 7f50d97fa700 1 -- 172.21.15.45:0/2782006823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50e00532b0 con 0x7f50e8157030 2023-10-06T05:20:37.226 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:37.228 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.262+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50e804bb40 con 0x7f50e8157030 2023-10-06T05:20:37.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.263+0000 7f50d97fa700 1 -- 172.21.15.45:0/2782006823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+17524 (secure 0 0 0) 0x7f50e00568d0 con 0x7f50e8157030 2023-10-06T05:20:37.265 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":68,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:18:57.905683+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:08.043401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:08.487972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.17","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/4169661273":"2023-10-07T05:18:57.905623+0000","172.21.15.45:6833/896514784":"2023-10-07T05:18:21.174171+0000","172.21.15.45:6832/4169661273":"2023-10-07T05:18:57.905623+0000","172.21.15.45:6832/896514784":"2023-10-07T05:18:21.174171+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50cc066f30 msgr2=0x7f50cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50cc066f30 0x7f50cc0693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f50e40011e0 tx=0x7f50e4001a00 comp rx=0 tx=0).stop 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e8157030 msgr2=0x7f50e80ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e8157030 0x7f50e80ad0d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f50e80c4f80 tx=0x7f50e0018000 comp rx=0 tx=0).stop 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 shutdown_connections 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50cc066f30 0x7f50cc0693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50e8157030 0x7f50e80ad0d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e80c8ff0 0x7f50e8069c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 --2- 172.21.15.45:0/2782006823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50e80c93c0 0x7f50e80b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 >> 172.21.15.45:0/2782006823 conn(0x7f50e80cdf50 msgr2=0x7f50e80cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 shutdown_connections 2023-10-06T05:20:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.265+0000 7f50ef2bc700 1 -- 172.21.15.45:0/2782006823 wait complete. 2023-10-06T05:20:37.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:18:08.043401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:18:08.487972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:20:37.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:20:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.493+0000 7f6a6a176700 1 Processor -- start 2023-10-06T05:20:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.493+0000 7f6a6a176700 1 -- start start 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a6a176700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a640b0610 0x7f6a640c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a6a176700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640c3db0 0x7f6a640697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a6a176700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a64069d00 0x7f6a640c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a6a176700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a640690f0 con 0x7f6a640b0610 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a6a176700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a640ceeb0 con 0x7f6a64069d00 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a6a176700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640ce6a0 con 0x7f6a640c3db0 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a637fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a640b0610 0x7f6a640c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a637fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a640b0610 0x7f6a640c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35846/0 (socket says 172.21.15.45:35846) 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a637fe700 1 -- 172.21.15.45:0/909642763 learned_addr learned my addr 172.21.15.45:0/909642763 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a63fff700 1 --2- 172.21.15.45:0/909642763 >> v2:172.21.15.144:3300/0 conn(0x7f6a64069d00 0x7f6a640c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a62ffd700 1 --2- 172.21.15.45:0/909642763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640c3db0 0x7f6a640697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a637fe700 1 -- 172.21.15.45:0/909642763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640c3db0 msgr2=0x7f6a640697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a637fe700 1 --2- 172.21.15.45:0/909642763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640c3db0 0x7f6a640697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a637fe700 1 -- 172.21.15.45:0/909642763 >> v2:172.21.15.144:3300/0 conn(0x7f6a64069d00 msgr2=0x7f6a640c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.494+0000 7f6a637fe700 1 --2- 172.21.15.45:0/909642763 >> v2:172.21.15.144:3300/0 conn(0x7f6a64069d00 0x7f6a640c2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a637fe700 1 -- 172.21.15.45:0/909642763 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64159880 con 0x7f6a640b0610 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a637fe700 1 --2- 172.21.15.45:0/909642763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640b0610 0x7f6a640c85d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6a6406fce0 tx=0x7f6a54009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b74a1901e640316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a627fc700 1 -- 172.21.15.45:0/909642763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54005c10 con 0x7f6a640b0610 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a627fc700 1 -- 172.21.15.45:0/909642763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54005d70 con 0x7f6a640b0610 2023-10-06T05:20:37.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a627fc700 1 -- 172.21.15.45:0/909642763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400b840 con 0x7f6a640b0610 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a6a176700 1 -- 172.21.15.45:0/909642763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640b0610 msgr2=0x7f6a640c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a6a176700 1 --2- 172.21.15.45:0/909642763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640b0610 0x7f6a640c85d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6a6406fce0 tx=0x7f6a54009220 comp rx=0 tx=0).stop 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a6a176700 1 -- 172.21.15.45:0/909642763 shutdown_connections 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a6a176700 1 --2- 172.21.15.45:0/909642763 >> v2:172.21.15.144:3300/0 conn(0x7f6a64069d00 0x7f6a640c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a6a176700 1 --2- 172.21.15.45:0/909642763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640b0610 0x7f6a640c85d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a6a176700 1 --2- 172.21.15.45:0/909642763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640c3db0 0x7f6a640697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.495+0000 7f6a6a176700 1 -- 172.21.15.45:0/909642763 >> 172.21.15.45:0/909642763 conn(0x7f6a64067e00 msgr2=0x7f6a640cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.496+0000 7f6a6a176700 1 -- 172.21.15.45:0/909642763 shutdown_connections 2023-10-06T05:20:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.496+0000 7f6a6a176700 1 -- 172.21.15.45:0/909642763 wait complete. 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.497+0000 7f6a6a176700 1 Processor -- start 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.497+0000 7f6a6a176700 1 -- start start 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a6a176700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64069d00 0x7f6a640c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a6a176700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640b0610 0x7f6a640b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a6a176700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640c3db0 0x7f6a64156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a6a176700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640b13b0 con 0x7f6a640c3db0 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a6a176700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640b0dd0 con 0x7f6a64069d00 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a6a176700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640b3c10 con 0x7f6a640b0610 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640c3db0 0x7f6a64156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640c3db0 0x7f6a64156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35852/0 (socket says 172.21.15.45:35852) 2023-10-06T05:20:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 -- 172.21.15.45:0/565625563 learned_addr learned my addr 172.21.15.45:0/565625563 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a637fe700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64069d00 0x7f6a640c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a62ffd700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640b0610 0x7f6a640b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 -- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640b0610 msgr2=0x7f6a640b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640b0610 0x7f6a640b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 -- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64069d00 msgr2=0x7f6a640c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64069d00 0x7f6a640c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.498+0000 7f6a63fff700 1 -- 172.21.15.45:0/565625563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009480 con 0x7f6a640c3db0 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.499+0000 7f6a63fff700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640c3db0 0x7f6a64156a20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009fa0 tx=0x7f6a5c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.499+0000 7f6a5b7fe700 1 -- 172.21.15.45:0/565625563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00ebc0 con 0x7f6a640c3db0 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.499+0000 7f6a5b7fe700 1 -- 172.21.15.45:0/565625563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00ed20 con 0x7f6a640c3db0 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.499+0000 7f6a5b7fe700 1 -- 172.21.15.45:0/565625563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c014670 con 0x7f6a640c3db0 2023-10-06T05:20:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.499+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a64156f60 con 0x7f6a640c3db0 2023-10-06T05:20:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.499+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a640b1040 con 0x7f6a640c3db0 2023-10-06T05:20:37.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.500+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6406e350 con 0x7f6a640c3db0 2023-10-06T05:20:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.501+0000 7f6a5b7fe700 1 -- 172.21.15.45:0/565625563 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a5c0149e0 con 0x7f6a640c3db0 2023-10-06T05:20:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.501+0000 7f6a5b7fe700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a44066f30 0x7f6a440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.501+0000 7f6a5b7fe700 1 -- 172.21.15.45:0/565625563 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f6a5c085e50 con 0x7f6a640c3db0 2023-10-06T05:20:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.501+0000 7f6a637fe700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a44066f30 0x7f6a440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.502+0000 7f6a637fe700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a44066f30 0x7f6a440693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009260 tx=0x7f6a54016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.505+0000 7f6a5b7fe700 1 -- 172.21.15.45:0/565625563 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a5c052dc0 con 0x7f6a640c3db0 2023-10-06T05:20:37.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.705+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a640b14f0 con 0x7f6a640c3db0 2023-10-06T05:20:37.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.705+0000 7f6a5b7fe700 1 -- 172.21.15.45:0/565625563 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a5c0563e0 con 0x7f6a640c3db0 2023-10-06T05:20:37.708 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:20:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a44066f30 msgr2=0x7f6a440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a44066f30 0x7f6a440693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009260 tx=0x7f6a54016000 comp rx=0 tx=0).stop 2023-10-06T05:20:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640c3db0 msgr2=0x7f6a64156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640c3db0 0x7f6a64156a20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009fa0 tx=0x7f6a5c009900 comp rx=0 tx=0).stop 2023-10-06T05:20:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 shutdown_connections 2023-10-06T05:20:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a44066f30 0x7f6a440693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a640c3db0 0x7f6a64156a20 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a64069d00 0x7f6a640c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 --2- 172.21.15.45:0/565625563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a640b0610 0x7f6a640b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 >> 172.21.15.45:0/565625563 conn(0x7f6a64067e00 msgr2=0x7f6a64158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 shutdown_connections 2023-10-06T05:20:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.708+0000 7f6a6a176700 1 -- 172.21.15.45:0/565625563 wait complete. 2023-10-06T05:20:37.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:20:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.856+0000 7f52c4b2c700 1 Processor -- start 2023-10-06T05:20:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.856+0000 7f52c4b2c700 1 -- start start 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.857+0000 7f52c4b2c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52c00ccb70 0x7f52c00aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.857+0000 7f52c4b2c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00c96a0 con 0x7f52c00c8b70 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.857+0000 7f52c4b2c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00c3b50 con 0x7f52c0157330 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.857+0000 7f52c4b2c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0157700 con 0x7f52c00ccb70 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.857+0000 7f52bdd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52c00ccb70 0x7f52c00aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.857+0000 7f52bed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f52c00c8b70 0x7f52c00c8f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33232/0 (socket says 172.21.15.45:33232) 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.857+0000 7f52bed9d700 1 -- 172.21.15.45:0/277341877 learned_addr learned my addr 172.21.15.45:0/277341877 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:37.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bdd9b700 1 -- 172.21.15.45:0/277341877 >> v1:172.21.15.144:6789/0 conn(0x7f52c0157330 legacy=0x7f52c00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bdd9b700 1 -- 172.21.15.45:0/277341877 >> v1:172.21.15.45:6789/0 conn(0x7f52c00c8b70 legacy=0x7f52c00c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bdd9b700 1 -- 172.21.15.45:0/277341877 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00c98e0 con 0x7f52c00ccb70 2023-10-06T05:20:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bd59a700 1 -- 172.21.15.45:0/277341877 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495577324 0 0) 0x7f52b8001490 con 0x7f52c00c8b70 2023-10-06T05:20:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bdd9b700 1 --2- 172.21.15.45:0/277341877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00ccb70 0x7f52c00aaae0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f52a80098e0 tx=0x7f52a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2701ac09218500e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bd59a700 1 -- 172.21.15.45:0/277341877 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8009e80 con 0x7f52c00ccb70 2023-10-06T05:20:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bd59a700 1 -- 172.21.15.45:0/277341877 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a800b860 con 0x7f52c00ccb70 2023-10-06T05:20:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.858+0000 7f52bd59a700 1 -- 172.21.15.45:0/277341877 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a800bbe0 con 0x7f52c00ccb70 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.859+0000 7f52c4b2c700 1 -- 172.21.15.45:0/277341877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00ccb70 msgr2=0x7f52c00aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.859+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/277341877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00ccb70 0x7f52c00aaae0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f52a80098e0 tx=0x7f52a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.859+0000 7f52c4b2c700 1 -- 172.21.15.45:0/277341877 shutdown_connections 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.859+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/277341877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00ccb70 0x7f52c00aaae0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.859+0000 7f52c4b2c700 1 -- 172.21.15.45:0/277341877 >> 172.21.15.45:0/277341877 conn(0x7f52c0067e00 msgr2=0x7f52c00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.859+0000 7f52c4b2c700 1 -- 172.21.15.45:0/277341877 shutdown_connections 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.859+0000 7f52c4b2c700 1 -- 172.21.15.45:0/277341877 wait complete. 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.860+0000 7f52c4b2c700 1 Processor -- start 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.860+0000 7f52c4b2c700 1 -- start start 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52c4b2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00c8b70 0x7f52c0151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52c4b2c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c00ccb70 0x7f52c006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52c4b2c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c0157330 0x7f52c006d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52c4b2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00b2310 con 0x7f52c00c8b70 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52c4b2c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c73b0 con 0x7f52c00ccb70 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52c4b2c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c7690 con 0x7f52c0157330 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00c8b70 0x7f52c0151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00c8b70 0x7f52c0151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35868/0 (socket says 172.21.15.45:35868) 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 -- 172.21.15.45:0/363470877 learned_addr learned my addr 172.21.15.45:0/363470877 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52bdd9b700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c00ccb70 0x7f52c006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52bed9d700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c0157330 0x7f52c006d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 -- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c0157330 msgr2=0x7f52c006d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c0157330 0x7f52c006d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 -- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c00ccb70 msgr2=0x7f52c006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c00ccb70 0x7f52c006d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 -- 172.21.15.45:0/363470877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8009420 con 0x7f52c00c8b70 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52bed9d700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c0157330 0x7f52c006d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.861+0000 7f52be59c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00c8b70 0x7f52c0151bf0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c7c0 tx=0x7f52b0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.862+0000 7f52b67fc700 1 -- 172.21.15.45:0/363470877 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0011590 con 0x7f52c00c8b70 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.862+0000 7f52b67fc700 1 -- 172.21.15.45:0/363470877 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b0011710 con 0x7f52c00c8b70 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.862+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c00c7970 con 0x7f52c00c8b70 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.862+0000 7f52b67fc700 1 -- 172.21.15.45:0/363470877 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0011ad0 con 0x7f52c00c8b70 2023-10-06T05:20:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.862+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c0152130 con 0x7f52c00c8b70 2023-10-06T05:20:37.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.863+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c006e350 con 0x7f52c00c8b70 2023-10-06T05:20:37.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.863+0000 7f52b67fc700 1 -- 172.21.15.45:0/363470877 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52b001b3b0 con 0x7f52c00c8b70 2023-10-06T05:20:37.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.864+0000 7f52b67fc700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f52a0066f30 0x7f52a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.864+0000 7f52b67fc700 1 -- 172.21.15.45:0/363470877 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f52b0083db0 con 0x7f52c00c8b70 2023-10-06T05:20:37.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.864+0000 7f52bdd9b700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f52a0066f30 0x7f52a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.864+0000 7f52bdd9b700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f52a0066f30 0x7f52a00693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f52a80093f0 tx=0x7f52a8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:37.868+0000 7f52b67fc700 1 -- 172.21.15.45:0/363470877 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52b0050d50 con 0x7f52c00c8b70 2023-10-06T05:20:38.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.071+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52c00c9bc0 con 0x7f52c00c8b70 2023-10-06T05:20:38.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.072+0000 7f52b67fc700 1 -- 172.21.15.45:0/363470877 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52b0017070 con 0x7f52c00c8b70 2023-10-06T05:20:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:20:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f52a0066f30 msgr2=0x7f52a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f52a0066f30 0x7f52a00693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f52a80093f0 tx=0x7f52a8017000 comp rx=0 tx=0).stop 2023-10-06T05:20:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00c8b70 msgr2=0x7f52c0151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00c8b70 0x7f52c0151bf0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c7c0 tx=0x7f52b0007f70 comp rx=0 tx=0).stop 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 shutdown_connections 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f52a0066f30 0x7f52a00693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c0157330 0x7f52c006d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52c00c8b70 0x7f52c0151bf0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 --2- 172.21.15.45:0/363470877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c00ccb70 0x7f52c006d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 >> 172.21.15.45:0/363470877 conn(0x7f52c0067e00 msgr2=0x7f52c00cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 shutdown_connections 2023-10-06T05:20:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.074+0000 7f52c4b2c700 1 -- 172.21.15.45:0/363470877 wait complete. 2023-10-06T05:20:38.089 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:20:38.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.222+0000 7f62d347f700 1 Processor -- start 2023-10-06T05:20:38.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.223+0000 7f62d347f700 1 -- start start 2023-10-06T05:20:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.223+0000 7f62d347f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f62cc0ccb70 0x7f62cc0aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.223+0000 7f62d347f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc0c96a0 con 0x7f62cc0c8b70 2023-10-06T05:20:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.223+0000 7f62d347f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc0c3b50 con 0x7f62cc157330 2023-10-06T05:20:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.223+0000 7f62d347f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc157700 con 0x7f62cc0ccb70 2023-10-06T05:20:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62d1a1c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f62cc0c8b70 0x7f62cc0c8f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33238/0 (socket says 172.21.15.45:33238) 2023-10-06T05:20:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62d1a1c700 1 -- 172.21.15.45:0/1257101129 learned_addr learned my addr 172.21.15.45:0/1257101129 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62d0a1a700 1 --2- 172.21.15.45:0/1257101129 >> v2:172.21.15.144:3300/0 conn(0x7f62cc0ccb70 0x7f62cc0aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329199071 0 0) 0x7f62c4001110 con 0x7f62cc0c8b70 2023-10-06T05:20:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b8003570 con 0x7f62cc0c8b70 2023-10-06T05:20:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2183455093 0 0) 0x7f62b8003570 con 0x7f62cc0c8b70 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 >> v2:172.21.15.144:3300/0 conn(0x7f62cc0ccb70 msgr2=0x7f62cc0aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 --2- 172.21.15.45:0/1257101129 >> v2:172.21.15.144:3300/0 conn(0x7f62cc0ccb70 0x7f62cc0aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 >> v1:172.21.15.144:6789/0 conn(0x7f62cc157330 legacy=0x7f62cc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0c98e0 con 0x7f62cc0c8b70 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f62c4001600 con 0x7f62cc0c8b70 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 >> v1:172.21.15.45:6789/0 conn(0x7f62cc0c8b70 legacy=0x7f62cc0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8003a60 0x7f62b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62b8004810 0x7f62b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b80051c0 0x7f62b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc157700 con 0x7f62b8003a60 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b800c0d0 con 0x7f62b8004810 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b800c290 con 0x7f62b80051c0 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.224+0000 7f62d121b700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8003a60 0x7f62b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d0a1a700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62b8004810 0x7f62b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d1a1c700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b80051c0 0x7f62b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d121b700 1 -- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b80051c0 msgr2=0x7f62b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d121b700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b80051c0 0x7f62b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d121b700 1 -- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62b8004810 msgr2=0x7f62b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d121b700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62b8004810 0x7f62b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d121b700 1 -- 172.21.15.45:0/1257101129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b8003570 con 0x7f62b8003a60 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d1a1c700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b80051c0 0x7f62b800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d1a1c700 1 -- 172.21.15.45:0/1257101129 reap_dead start 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62d121b700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8003a60 0x7f62b8003e70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c2d0 tx=0x7f62c8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01862df41a762de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8001d80 con 0x7f62b8003a60 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c800cba0 con 0x7f62b8003a60 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.225+0000 7f62bffff700 1 -- 172.21.15.45:0/1257101129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8012400 con 0x7f62b8003a60 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.226+0000 7f62d347f700 1 -- 172.21.15.45:0/1257101129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8003a60 msgr2=0x7f62b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.226+0000 7f62d347f700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8003a60 0x7f62b8003e70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c2d0 tx=0x7f62c8001a70 comp rx=0 tx=0).stop 2023-10-06T05:20:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.226+0000 7f62d347f700 1 -- 172.21.15.45:0/1257101129 shutdown_connections 2023-10-06T05:20:38.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.226+0000 7f62d347f700 1 --2- 172.21.15.45:0/1257101129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62b8003a60 0x7f62b8003e70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.226+0000 7f62d347f700 1 -- 172.21.15.45:0/1257101129 >> 172.21.15.45:0/1257101129 conn(0x7f62cc067e00 msgr2=0x7f62cc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.226+0000 7f62d347f700 1 -- 172.21.15.45:0/1257101129 shutdown_connections 2023-10-06T05:20:38.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.226+0000 7f62d347f700 1 -- 172.21.15.45:0/1257101129 wait complete. 2023-10-06T05:20:38.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.227+0000 7f62d347f700 1 Processor -- start 2023-10-06T05:20:38.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.227+0000 7f62d347f700 1 -- start start 2023-10-06T05:20:38.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d347f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b8003a60 0x7f62cc150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d347f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62cc0b3030 0x7f62cc0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d347f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc068960 0x7f62cc068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d347f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0aac70 con 0x7f62cc0b3030 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d347f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc069690 con 0x7f62cc068960 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d347f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc069970 con 0x7f62b8003a60 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62cc0b3030 0x7f62cc0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62cc0b3030 0x7f62cc0b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35878/0 (socket says 172.21.15.45:35878) 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 -- 172.21.15.45:0/1403467337 learned_addr learned my addr 172.21.15.45:0/1403467337 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d121b700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b8003a60 0x7f62cc150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d1a1c700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc068960 0x7f62cc068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 -- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b8003a60 msgr2=0x7f62cc150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b8003a60 0x7f62cc150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 -- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc068960 msgr2=0x7f62cc068dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc068960 0x7f62cc068dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d0a1a700 1 -- 172.21.15.45:0/1403467337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc069c50 con 0x7f62cc0b3030 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d121b700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b8003a60 0x7f62cc150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.228+0000 7f62d1a1c700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc068960 0x7f62cc068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.229+0000 7f62d0a1a700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62cc0b3030 0x7f62cc0b34a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f62c0011d40 tx=0x7f62c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.229+0000 7f62bd7fa700 1 -- 172.21.15.45:0/1403467337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c000fac0 con 0x7f62cc0b3030 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.229+0000 7f62bd7fa700 1 -- 172.21.15.45:0/1403467337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c000fc20 con 0x7f62cc0b3030 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.229+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc069f30 con 0x7f62cc0b3030 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.229+0000 7f62bd7fa700 1 -- 172.21.15.45:0/1403467337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0014440 con 0x7f62cc0b3030 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.229+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc151020 con 0x7f62cc0b3030 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.230+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc06e350 con 0x7f62cc0b3030 2023-10-06T05:20:38.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.230+0000 7f62bd7fa700 1 -- 172.21.15.45:0/1403467337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62c0014a80 con 0x7f62cc0b3030 2023-10-06T05:20:38.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.231+0000 7f62bd7fa700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f62b0066f30 0x7f62b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.231+0000 7f62bd7fa700 1 -- 172.21.15.45:0/1403467337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f62c0018030 con 0x7f62cc0b3030 2023-10-06T05:20:38.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.231+0000 7f62d121b700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f62b0066f30 0x7f62b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.232+0000 7f62d121b700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f62b0066f30 0x7f62b00693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62c80013e0 tx=0x7f62c8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.235+0000 7f62bd7fa700 1 -- 172.21.15.45:0/1403467337 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62c0052460 con 0x7f62cc0b3030 2023-10-06T05:20:38.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.438+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62cc0c9bc0 con 0x7f62cc0b3030 2023-10-06T05:20:38.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.439+0000 7f62bd7fa700 1 -- 172.21.15.45:0/1403467337 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62c0055a80 con 0x7f62cc0b3030 2023-10-06T05:20:38.441 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:20:38.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f62b0066f30 msgr2=0x7f62b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f62b0066f30 0x7f62b00693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62c80013e0 tx=0x7f62c8016000 comp rx=0 tx=0).stop 2023-10-06T05:20:38.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62cc0b3030 msgr2=0x7f62cc0b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62cc0b3030 0x7f62cc0b34a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f62c0011d40 tx=0x7f62c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 shutdown_connections 2023-10-06T05:20:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f62b0066f30 0x7f62b00693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc068960 0x7f62cc068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62b8003a60 0x7f62cc150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 --2- 172.21.15.45:0/1403467337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62cc0b3030 0x7f62cc0b34a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 >> 172.21.15.45:0/1403467337 conn(0x7f62cc067e00 msgr2=0x7f62cc1539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 shutdown_connections 2023-10-06T05:20:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.441+0000 7f62d347f700 1 -- 172.21.15.45:0/1403467337 wait complete. 2023-10-06T05:20:38.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:20:38.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.589+0000 7f021faaa700 1 Processor -- start 2023-10-06T05:20:38.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.590+0000 7f021faaa700 1 -- start start 2023-10-06T05:20:38.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.590+0000 7f021faaa700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f02180b0610 0x7f02180c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021faaa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0218069ed0 0x7f02180cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021faaa700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02180ce6a0 con 0x7f02180c3db0 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021faaa700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02180690f0 con 0x7f0218069ed0 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021faaa700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02180ceeb0 con 0x7f02180b0610 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0218069ed0 0x7f02180cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0218069ed0 0x7f02180cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35894/0 (socket says 172.21.15.45:35894) 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 -- 172.21.15.45:0/3786752206 learned_addr learned my addr 172.21.15.45:0/3786752206 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021d045700 1 --1- 172.21.15.45:0/3786752206 >> v1:172.21.15.45:6789/0 conn(0x7f02180c3db0 0x7f02180697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33254/0 (socket says 172.21.15.45:33254) 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021d846700 1 --2- 172.21.15.45:0/3786752206 >> v2:172.21.15.144:3300/0 conn(0x7f02180b0610 0x7f02180c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 -- 172.21.15.45:0/3786752206 >> v2:172.21.15.144:3300/0 conn(0x7f02180b0610 msgr2=0x7f02180c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 --2- 172.21.15.45:0/3786752206 >> v2:172.21.15.144:3300/0 conn(0x7f02180b0610 0x7f02180c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 -- 172.21.15.45:0/3786752206 >> v1:172.21.15.45:6789/0 conn(0x7f02180c3db0 legacy=0x7f02180697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 -- 172.21.15.45:0/3786752206 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218159880 con 0x7f0218069ed0 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021c844700 1 -- 172.21.15.45:0/3786752206 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714312030 0 0) 0x7f02180ce6a0 con 0x7f02180c3db0 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021d846700 1 --2- 172.21.15.45:0/3786752206 >> v2:172.21.15.144:3300/0 conn(0x7f02180b0610 0x7f02180c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021e047700 1 --2- 172.21.15.45:0/3786752206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f02180cc350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f021806f080 tx=0x7f0210009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b55aaa11d3ed62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021c844700 1 -- 172.21.15.45:0/3786752206 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000e070 con 0x7f0218069ed0 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.591+0000 7f021c844700 1 -- 172.21.15.45:0/3786752206 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021000ba40 con 0x7f0218069ed0 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021c844700 1 -- 172.21.15.45:0/3786752206 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02100092e0 con 0x7f0218069ed0 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 -- 172.21.15.45:0/3786752206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 msgr2=0x7f02180cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 --2- 172.21.15.45:0/3786752206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f02180cc350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f021806f080 tx=0x7f0210009840 comp rx=0 tx=0).stop 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 -- 172.21.15.45:0/3786752206 shutdown_connections 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 --2- 172.21.15.45:0/3786752206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f02180cc350 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 --2- 172.21.15.45:0/3786752206 >> v2:172.21.15.144:3300/0 conn(0x7f02180b0610 0x7f02180c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 -- 172.21.15.45:0/3786752206 >> 172.21.15.45:0/3786752206 conn(0x7f0218067e00 msgr2=0x7f02180cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 -- 172.21.15.45:0/3786752206 shutdown_connections 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.592+0000 7f021faaa700 1 -- 172.21.15.45:0/3786752206 wait complete. 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.593+0000 7f021faaa700 1 Processor -- start 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.593+0000 7f021faaa700 1 -- start start 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021faaa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f0218168e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021faaa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02180b0610 0x7f0218159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021faaa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02180c3db0 0x7f021815c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021faaa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02180c5b10 con 0x7f0218069ed0 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021faaa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021815a460 con 0x7f02180c3db0 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021faaa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02180c5d60 con 0x7f02180b0610 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021d846700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f0218168e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021d846700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f0218168e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35904/0 (socket says 172.21.15.45:35904) 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021d846700 1 -- 172.21.15.45:0/758712425 learned_addr learned my addr 172.21.15.45:0/758712425 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021e047700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02180c3db0 0x7f021815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.594+0000 7f021d045700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02180b0610 0x7f0218159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021d846700 1 -- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02180b0610 msgr2=0x7f0218159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021d846700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02180b0610 0x7f0218159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021d846700 1 -- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02180c3db0 msgr2=0x7f021815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021d846700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02180c3db0 0x7f021815c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021d846700 1 -- 172.21.15.45:0/758712425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0210009ae0 con 0x7f0218069ed0 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021e047700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02180c3db0 0x7f021815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021d846700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f0218168e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f021400e920 tx=0x7f0214008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f020dffb700 1 -- 172.21.15.45:0/758712425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214005a00 con 0x7f0218069ed0 2023-10-06T05:20:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f020dffb700 1 -- 172.21.15.45:0/758712425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214005b60 con 0x7f0218069ed0 2023-10-06T05:20:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02180c6040 con 0x7f0218069ed0 2023-10-06T05:20:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f020dffb700 1 -- 172.21.15.45:0/758712425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214006020 con 0x7f0218069ed0 2023-10-06T05:20:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.595+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0218169390 con 0x7f0218069ed0 2023-10-06T05:20:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.596+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02180c64e0 con 0x7f0218069ed0 2023-10-06T05:20:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.597+0000 7f020dffb700 1 -- 172.21.15.45:0/758712425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0214006410 con 0x7f0218069ed0 2023-10-06T05:20:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.597+0000 7f020dffb700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01fc066f30 0x7f01fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.597+0000 7f020dffb700 1 -- 172.21.15.45:0/758712425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f0214084a30 con 0x7f0218069ed0 2023-10-06T05:20:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.597+0000 7f021d045700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01fc066f30 0x7f01fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.598+0000 7f021d045700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01fc066f30 0x7f01fc0693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0208007fd0 tx=0x7f0208007e10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.601+0000 7f020dffb700 1 -- 172.21.15.45:0/758712425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0214051920 con 0x7f0218069ed0 2023-10-06T05:20:38.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.799+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02180c6690 con 0x7f0218069ed0 2023-10-06T05:20:38.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.800+0000 7f020dffb700 1 -- 172.21.15.45:0/758712425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0214054f40 con 0x7f0218069ed0 2023-10-06T05:20:38.802 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01fc066f30 msgr2=0x7f01fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01fc066f30 0x7f01fc0693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0208007fd0 tx=0x7f0208007e10 comp rx=0 tx=0).stop 2023-10-06T05:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 msgr2=0x7f0218168e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f0218168e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f021400e920 tx=0x7f0214008040 comp rx=0 tx=0).stop 2023-10-06T05:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 shutdown_connections 2023-10-06T05:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f01fc066f30 0x7f01fc0693e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02180c3db0 0x7f021815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218069ed0 0x7f0218168e50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 --2- 172.21.15.45:0/758712425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02180b0610 0x7f0218159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 >> 172.21.15.45:0/758712425 conn(0x7f0218067e00 msgr2=0x7f0218069660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 shutdown_connections 2023-10-06T05:20:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.802+0000 7f021faaa700 1 -- 172.21.15.45:0/758712425 wait complete. 2023-10-06T05:20:38.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.950+0000 7fb8f13e3700 1 Processor -- start 2023-10-06T05:20:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.950+0000 7fb8f13e3700 1 -- start start 2023-10-06T05:20:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8f13e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c91f0 0x7fb8ec0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8f13e3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8ec0c9640 0x7fb8ec0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8f13e3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0c84c0 con 0x7fb8ec0ca810 2023-10-06T05:20:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8f13e3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec0c8d60 con 0x7fb8ec0c9640 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8f13e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0b0500 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8ec0c9640 0x7fb8ec0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8ec0c9640 0x7fb8ec0affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35906/0 (socket says 172.21.15.45:35906) 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 -- 172.21.15.45:0/7510650 learned_addr learned my addr 172.21.15.45:0/7510650 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8ea7fc700 1 --2- 172.21.15.45:0/7510650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c91f0 0x7fb8ec0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 -- 172.21.15.45:0/7510650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c91f0 msgr2=0x7fb8ec0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 --2- 172.21.15.45:0/7510650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c91f0 0x7fb8ec0afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 -- 172.21.15.45:0/7510650 >> v1:172.21.15.144:6789/0 conn(0x7fb8ec0ca810 legacy=0x7fb8ec0cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 -- 172.21.15.45:0/7510650 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec069d00 con 0x7fb8ec0c9640 2023-10-06T05:20:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.951+0000 7fb8eb7fe700 1 --2- 172.21.15.45:0/7510650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c9640 0x7fb8ec0affc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009f20 tx=0x7fb8e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda73f1b47a6d4b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8e9ffb700 1 -- 172.21.15.45:0/7510650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400e070 con 0x7fb8ec0c9640 2023-10-06T05:20:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8e9ffb700 1 -- 172.21.15.45:0/7510650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e400b9d0 con 0x7fb8ec0c9640 2023-10-06T05:20:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8e9ffb700 1 -- 172.21.15.45:0/7510650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40092e0 con 0x7fb8ec0c9640 2023-10-06T05:20:38.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 -- 172.21.15.45:0/7510650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c9640 msgr2=0x7fb8ec0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/7510650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c9640 0x7fb8ec0affc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009f20 tx=0x7fb8e4009840 comp rx=0 tx=0).stop 2023-10-06T05:20:38.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 -- 172.21.15.45:0/7510650 shutdown_connections 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/7510650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c9640 0x7fb8ec0affc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/7510650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c91f0 0x7fb8ec0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 -- 172.21.15.45:0/7510650 >> 172.21.15.45:0/7510650 conn(0x7fb8ec067620 msgr2=0x7fb8ec067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 -- 172.21.15.45:0/7510650 shutdown_connections 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.952+0000 7fb8f13e3700 1 -- 172.21.15.45:0/7510650 wait complete. 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.954+0000 7fb8f13e3700 1 Processor -- start 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.954+0000 7fb8f13e3700 1 -- start start 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8f13e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c91f0 0x7fb8ec150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8f13e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c9640 0x7fb8ec16a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8f13e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ec0ca810 0x7fb8ec0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8f13e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0cbaf0 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8f13e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0cbdd0 con 0x7fb8ec0ca810 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8f13e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0cc0b0 con 0x7fb8ec0c9640 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c91f0 0x7fb8ec150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c91f0 0x7fb8ec150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35912/0 (socket says 172.21.15.45:35912) 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 -- 172.21.15.45:0/2818993695 learned_addr learned my addr 172.21.15.45:0/2818993695 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eb7fe700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ec0ca810 0x7fb8ec0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8ea7fc700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c9640 0x7fb8ec16a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 -- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c9640 msgr2=0x7fb8ec16a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c9640 0x7fb8ec16a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 -- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ec0ca810 msgr2=0x7fb8ec0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ec0ca810 0x7fb8ec0cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 -- 172.21.15.45:0/2818993695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e4009ae0 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8ea7fc700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c9640 0x7fb8ec16a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8eaffd700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c91f0 0x7fb8ec150350 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00c4e0 tx=0x7fb8dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8d2ffd700 1 -- 172.21.15.45:0/2818993695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011590 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8d2ffd700 1 -- 172.21.15.45:0/2818993695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc011710 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.955+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec0cc390 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.956+0000 7fb8d2ffd700 1 -- 172.21.15.45:0/2818993695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011ad0 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.956+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec150890 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.957+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec06f160 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.957+0000 7fb8d2ffd700 1 -- 172.21.15.45:0/2818993695 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8dc01b3b0 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.957+0000 7fb8d2ffd700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.957+0000 7fb8d2ffd700 1 -- 172.21.15.45:0/2818993695 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fb8dc083e70 con 0x7fb8ec0c91f0 2023-10-06T05:20:38.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.958+0000 7fb8ea7fc700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.958+0000 7fb8ea7fc700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8d8066f30 0x7fb8d80693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0006f00 tx=0x7fb8e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:38.962+0000 7fb8d2ffd700 1 -- 172.21.15.45:0/2818993695 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8dc050ce0 con 0x7fb8ec0c91f0 2023-10-06T05:20:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.214+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8ec0c8710 con 0x7fb8ec0c91f0 2023-10-06T05:20:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.215+0000 7fb8d2ffd700 1 -- 172.21.15.45:0/2818993695 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fb8dc017070 con 0x7fb8ec0c91f0 2023-10-06T05:20:39.217 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1230704519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1230704519},{"type":"v1","addr":"172.21.15.45:6833","nonce":1230704519}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:19:00.913269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5543":{"gid":5543,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8d8066f30 msgr2=0x7fb8d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8d8066f30 0x7fb8d80693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0006f00 tx=0x7fb8e0008040 comp rx=0 tx=0).stop 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c91f0 msgr2=0x7fb8ec150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c91f0 0x7fb8ec150350 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00c4e0 tx=0x7fb8dc00c8a0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 shutdown_connections 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ec0ca810 0x7fb8ec0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8ec0c91f0 0x7fb8ec150350 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 --2- 172.21.15.45:0/2818993695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ec0c9640 0x7fb8ec16a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 >> 172.21.15.45:0/2818993695 conn(0x7fb8ec067620 msgr2=0x7fb8ec15b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.217+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 shutdown_connections 2023-10-06T05:20:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.218+0000 7fb8f13e3700 1 -- 172.21.15.45:0/2818993695 wait complete. 2023-10-06T05:20:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 62 2023-10-06T05:20:39.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.363+0000 7f4acec50700 1 Processor -- start 2023-10-06T05:20:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.363+0000 7f4acec50700 1 -- start start 2023-10-06T05:20:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.363+0000 7f4acec50700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ac815a490 0x7f4ac8067830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acec50700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80b2120 con 0x7f4ac806afa0 2023-10-06T05:20:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acec50700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80ab7d0 con 0x7f4ac815b5e0 2023-10-06T05:20:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acec50700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80ae020 con 0x7f4ac815a490 2023-10-06T05:20:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acc9ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ac815a490 0x7f4ac8067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acd1ed700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4ac806afa0 0x7f4ac80abc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33264/0 (socket says 172.21.15.45:33264) 2023-10-06T05:20:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acd1ed700 1 -- 172.21.15.45:0/1022396943 learned_addr learned my addr 172.21.15.45:0/1022396943 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acc9ec700 1 -- 172.21.15.45:0/1022396943 >> v1:172.21.15.144:6789/0 conn(0x7f4ac815b5e0 legacy=0x7f4ac806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acc9ec700 1 -- 172.21.15.45:0/1022396943 >> v1:172.21.15.45:6789/0 conn(0x7f4ac806afa0 legacy=0x7f4ac80abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4ac77fe700 1 -- 172.21.15.45:0/1022396943 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444004520 0 0) 0x7f4ac0001490 con 0x7f4ac806afa0 2023-10-06T05:20:39.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acc9ec700 1 -- 172.21.15.45:0/1022396943 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac8067e80 con 0x7f4ac815a490 2023-10-06T05:20:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4acc9ec700 1 --2- 172.21.15.45:0/1022396943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac8067830 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80098e0 tx=0x7f4ab80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfecdcceef7b444 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4ac77fe700 1 -- 172.21.15.45:0/1022396943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8009e80 con 0x7f4ac815a490 2023-10-06T05:20:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4ac77fe700 1 -- 172.21.15.45:0/1022396943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab800b860 con 0x7f4ac815a490 2023-10-06T05:20:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.364+0000 7f4ac77fe700 1 -- 172.21.15.45:0/1022396943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800bbe0 con 0x7f4ac815a490 2023-10-06T05:20:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.365+0000 7f4acec50700 1 -- 172.21.15.45:0/1022396943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 msgr2=0x7f4ac8067830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.365+0000 7f4acec50700 1 --2- 172.21.15.45:0/1022396943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac8067830 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80098e0 tx=0x7f4ab80091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.365+0000 7f4acec50700 1 -- 172.21.15.45:0/1022396943 shutdown_connections 2023-10-06T05:20:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.365+0000 7f4acec50700 1 --2- 172.21.15.45:0/1022396943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac8067830 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.365+0000 7f4acec50700 1 -- 172.21.15.45:0/1022396943 >> 172.21.15.45:0/1022396943 conn(0x7f4ac80cc5a0 msgr2=0x7f4ac80cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.365+0000 7f4acec50700 1 -- 172.21.15.45:0/1022396943 shutdown_connections 2023-10-06T05:20:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.365+0000 7f4acec50700 1 -- 172.21.15.45:0/1022396943 wait complete. 2023-10-06T05:20:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.366+0000 7f4acec50700 1 Processor -- start 2023-10-06T05:20:39.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4acec50700 1 -- start start 2023-10-06T05:20:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4acec50700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ac806afa0 0x7f4ac80c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4acec50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac814aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4acec50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ac815b5e0 0x7f4ac8167f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4acec50700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80c2320 con 0x7f4ac815a490 2023-10-06T05:20:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4acec50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80c1710 con 0x7f4ac815b5e0 2023-10-06T05:20:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4acec50700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80c19f0 con 0x7f4ac806afa0 2023-10-06T05:20:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.367+0000 7f4ac7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac814aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac814aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35930/0 (socket says 172.21.15.45:35930) 2023-10-06T05:20:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 -- 172.21.15.45:0/2180815553 learned_addr learned my addr 172.21.15.45:0/2180815553 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4acd1ed700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ac815b5e0 0x7f4ac8167f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4acc9ec700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ac806afa0 0x7f4ac80c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 -- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ac806afa0 msgr2=0x7f4ac80c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ac806afa0 0x7f4ac80c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 -- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ac815b5e0 msgr2=0x7f4ac8167f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ac815b5e0 0x7f4ac8167f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 -- 172.21.15.45:0/2180815553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab8009420 con 0x7f4ac815a490 2023-10-06T05:20:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac7fff700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac814aa20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00c4e0 tx=0x7f4abc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac4ff9700 1 -- 172.21.15.45:0/2180815553 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc011590 con 0x7f4ac815a490 2023-10-06T05:20:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac4ff9700 1 -- 172.21.15.45:0/2180815553 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc011710 con 0x7f4ac815a490 2023-10-06T05:20:39.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac80c1cd0 con 0x7f4ac815a490 2023-10-06T05:20:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4ac4ff9700 1 -- 172.21.15.45:0/2180815553 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc011ad0 con 0x7f4ac815a490 2023-10-06T05:20:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.368+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac80c67d0 con 0x7f4ac815a490 2023-10-06T05:20:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.369+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac80c24d0 con 0x7f4ac815a490 2023-10-06T05:20:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.370+0000 7f4ac4ff9700 1 -- 172.21.15.45:0/2180815553 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4abc01b3b0 con 0x7f4ac815a490 2023-10-06T05:20:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.370+0000 7f4ac4ff9700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ab4066f30 0x7f4ab40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.370+0000 7f4ac4ff9700 1 -- 172.21.15.45:0/2180815553 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f4abc083de0 con 0x7f4ac815a490 2023-10-06T05:20:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.371+0000 7f4acc9ec700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ab4066f30 0x7f4ab40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.371+0000 7f4acc9ec700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ab4066f30 0x7f4ab40693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80093f0 tx=0x7f4ab8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.375+0000 7f4ac4ff9700 1 -- 172.21.15.45:0/2180815553 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ac80c24d0 con 0x7f4ac815a490 2023-10-06T05:20:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.579+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ac80cc0e0 con 0x7f4ac815a490 2023-10-06T05:20:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.580+0000 7f4ac4ff9700 1 -- 172.21.15.45:0/2180815553 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+17524 (secure 0 0 0) 0x7f4abc017070 con 0x7f4ac815a490 2023-10-06T05:20:39.582 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:39.583 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":68,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:18:57.905683+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:08.043401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:08.487972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.17","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/4169661273":"2023-10-07T05:18:57.905623+0000","172.21.15.45:6833/896514784":"2023-10-07T05:18:21.174171+0000","172.21.15.45:6832/4169661273":"2023-10-07T05:18:57.905623+0000","172.21.15.45:6832/896514784":"2023-10-07T05:18:21.174171+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ab4066f30 msgr2=0x7f4ab40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ab4066f30 0x7f4ab40693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80093f0 tx=0x7f4ab8017000 comp rx=0 tx=0).stop 2023-10-06T05:20:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 msgr2=0x7f4ac814aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac814aa20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00c4e0 tx=0x7f4abc00c7f0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 shutdown_connections 2023-10-06T05:20:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ab4066f30 0x7f4ab40693e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ac815b5e0 0x7f4ac8167f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ac806afa0 0x7f4ac80c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 --2- 172.21.15.45:0/2180815553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ac815a490 0x7f4ac814aa20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 >> 172.21.15.45:0/2180815553 conn(0x7f4ac80cc5a0 msgr2=0x7f4ac80caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.582+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 shutdown_connections 2023-10-06T05:20:39.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.583+0000 7f4acec50700 1 -- 172.21.15.45:0/2180815553 wait complete. 2023-10-06T05:20:39.598 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:20:39.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.728+0000 7fd2e40db700 1 Processor -- start 2023-10-06T05:20:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.728+0000 7fd2e40db700 1 -- start start 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e40db700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2dc155440 0x7fd2dc155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e40db700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06e6e0 0x7fd2dc0c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e40db700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2dc0c8080 con 0x7fd2dc06a890 2023-10-06T05:20:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e40db700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc068cd0 con 0x7fd2dc155440 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e40db700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc156050 con 0x7fd2dc06e6e0 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2dc155440 0x7fd2dc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2dc155440 0x7fd2dc155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35932/0 (socket says 172.21.15.45:35932) 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 -- 172.21.15.45:0/1341721065 learned_addr learned my addr 172.21.15.45:0/1341721065 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1676700 1 --2- 172.21.15.45:0/1341721065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06e6e0 0x7fd2dc0c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 -- 172.21.15.45:0/1341721065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06e6e0 msgr2=0x7fd2dc0c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 --2- 172.21.15.45:0/1341721065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06e6e0 0x7fd2dc0c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 -- 172.21.15.45:0/1341721065 >> v1:172.21.15.144:6789/0 conn(0x7fd2dc06a890 legacy=0x7fd2dc1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 -- 172.21.15.45:0/1341721065 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc156330 con 0x7fd2dc155440 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.729+0000 7fd2e1e77700 1 --2- 172.21.15.45:0/1341721065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc155810 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800bc40 tx=0x7fd2d800b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ae37e508f81b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e0e75700 1 -- 172.21.15.45:0/1341721065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800db40 con 0x7fd2dc155440 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e0e75700 1 -- 172.21.15.45:0/1341721065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d800dca0 con 0x7fd2dc155440 2023-10-06T05:20:39.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e0e75700 1 -- 172.21.15.45:0/1341721065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8017640 con 0x7fd2dc155440 2023-10-06T05:20:39.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 -- 172.21.15.45:0/1341721065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 msgr2=0x7fd2dc155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 --2- 172.21.15.45:0/1341721065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc155810 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800bc40 tx=0x7fd2d800b4e0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 -- 172.21.15.45:0/1341721065 shutdown_connections 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 --2- 172.21.15.45:0/1341721065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc155810 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 --2- 172.21.15.45:0/1341721065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06e6e0 0x7fd2dc0c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 -- 172.21.15.45:0/1341721065 >> 172.21.15.45:0/1341721065 conn(0x7fd2dc0cc5a0 msgr2=0x7fd2dc0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 -- 172.21.15.45:0/1341721065 shutdown_connections 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.730+0000 7fd2e40db700 1 -- 172.21.15.45:0/1341721065 wait complete. 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.731+0000 7fd2e40db700 1 Processor -- start 2023-10-06T05:20:39.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e40db700 1 -- start start 2023-10-06T05:20:39.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e40db700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06a890 0x7fd2dc0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e40db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc06e6e0 0x7fd2dc154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e40db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e40db700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0b1490 con 0x7fd2dc155440 2023-10-06T05:20:39.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e40db700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0b0ed0 con 0x7fd2dc06e6e0 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e40db700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0b11b0 con 0x7fd2dc06a890 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.732+0000 7fd2e2678700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc0b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35942/0 (socket says 172.21.15.45:35942) 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 -- 172.21.15.45:0/2663962662 learned_addr learned my addr 172.21.15.45:0/2663962662 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e1e77700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06a890 0x7fd2dc0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e1676700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc06e6e0 0x7fd2dc154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 -- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06a890 msgr2=0x7fd2dc0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06a890 0x7fd2dc0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 -- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc06e6e0 msgr2=0x7fd2dc154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc06e6e0 0x7fd2dc154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 -- 172.21.15.45:0/2663962662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d800b780 con 0x7fd2dc155440 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e2678700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc0b1880 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c8e0 tx=0x7fd2d4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2d1ffb700 1 -- 172.21.15.45:0/2663962662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4012630 con 0x7fd2dc155440 2023-10-06T05:20:39.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2d1ffb700 1 -- 172.21.15.45:0/2663962662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d40127b0 con 0x7fd2dc155440 2023-10-06T05:20:39.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2d1ffb700 1 -- 172.21.15.45:0/2663962662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4012b70 con 0x7fd2dc155440 2023-10-06T05:20:39.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.733+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc0c46b0 con 0x7fd2dc155440 2023-10-06T05:20:39.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.734+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc0b0be0 con 0x7fd2dc155440 2023-10-06T05:20:39.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.735+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc0c4bd0 con 0x7fd2dc155440 2023-10-06T05:20:39.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.735+0000 7fd2d1ffb700 1 -- 172.21.15.45:0/2663962662 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2d4003030 con 0x7fd2dc155440 2023-10-06T05:20:39.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.735+0000 7fd2d1ffb700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c0066f60 0x7fd2c0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.735+0000 7fd2d1ffb700 1 -- 172.21.15.45:0/2663962662 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fd2d4084700 con 0x7fd2dc155440 2023-10-06T05:20:39.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.736+0000 7fd2e1e77700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c0066f60 0x7fd2c0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.736+0000 7fd2e1e77700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c0066f60 0x7fd2c0069410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800b520 tx=0x7fd2d8019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.740+0000 7fd2d1ffb700 1 -- 172.21.15.45:0/2663962662 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2dc0c4bd0 con 0x7fd2dc155440 2023-10-06T05:20:39.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.991+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2dc0c4bd0 con 0x7fd2dc155440 2023-10-06T05:20:39.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.991+0000 7fd2d1ffb700 1 -- 172.21.15.45:0/2663962662 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fd2dc0c4bd0 con 0x7fd2dc155440 2023-10-06T05:20:39.993 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:39.993 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1230704519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1230704519},{"type":"v1","addr":"172.21.15.45:6833","nonce":1230704519}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:19:00.913269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5543":{"gid":5543,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c0066f60 msgr2=0x7fd2c0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c0066f60 0x7fd2c0069410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800b520 tx=0x7fd2d8019000 comp rx=0 tx=0).stop 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 msgr2=0x7fd2dc0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc0b1880 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c8e0 tx=0x7fd2d4001710 comp rx=0 tx=0).stop 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 shutdown_connections 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c0066f60 0x7fd2c0069410 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc155440 0x7fd2dc0b1880 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2dc06a890 0x7fd2dc0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 --2- 172.21.15.45:0/2663962662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc06e6e0 0x7fd2dc154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 >> 172.21.15.45:0/2663962662 conn(0x7fd2dc0cc5a0 msgr2=0x7fd2dc0c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 shutdown_connections 2023-10-06T05:20:39.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:39.994+0000 7fd2e40db700 1 -- 172.21.15.45:0/2663962662 wait complete. 2023-10-06T05:20:39.997 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 62 2023-10-06T05:20:40.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:40.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.139+0000 7fc024a46700 1 Processor -- start 2023-10-06T05:20:40.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.139+0000 7fc024a46700 1 -- start start 2023-10-06T05:20:40.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc024a46700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0200c8ff0 0x7fc0200c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc024a46700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc020157200 0x7fc0200caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc024a46700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02015b000 con 0x7fc0200caad0 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc024a46700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02015b2e0 con 0x7fc020157200 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc024a46700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02015b5c0 con 0x7fc0200c8ff0 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc01ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc020157200 0x7fc0200caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc01dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc0200caad0 0x7fc020156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33274/0 (socket says 172.21.15.45:33274) 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc01ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc020157200 0x7fc0200caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35946/0 (socket says 172.21.15.45:35946) 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.140+0000 7fc01dd9b700 1 -- 172.21.15.45:0/329557902 learned_addr learned my addr 172.21.15.45:0/329557902 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:40.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01e59c700 1 --2- 172.21.15.45:0/329557902 >> v2:172.21.15.144:3300/0 conn(0x7fc0200c8ff0 0x7fc0200c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01ed9d700 1 -- 172.21.15.45:0/329557902 >> v2:172.21.15.144:3300/0 conn(0x7fc0200c8ff0 msgr2=0x7fc0200c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01ed9d700 1 --2- 172.21.15.45:0/329557902 >> v2:172.21.15.144:3300/0 conn(0x7fc0200c8ff0 0x7fc0200c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01ed9d700 1 -- 172.21.15.45:0/329557902 >> v1:172.21.15.45:6789/0 conn(0x7fc0200caad0 legacy=0x7fc020156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01ed9d700 1 -- 172.21.15.45:0/329557902 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02015b8a0 con 0x7fc020157200 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01d59a700 1 -- 172.21.15.45:0/329557902 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238814452 0 0) 0x7fc02015b000 con 0x7fc0200caad0 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01ed9d700 1 --2- 172.21.15.45:0/329557902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc020157200 0x7fc0200caf40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc0200cefc0 tx=0x7fc018009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0d2c96a3a39be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01d59a700 1 -- 172.21.15.45:0/329557902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01800e070 con 0x7fc020157200 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01d59a700 1 -- 172.21.15.45:0/329557902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01800bb60 con 0x7fc020157200 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.141+0000 7fc01d59a700 1 -- 172.21.15.45:0/329557902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018012430 con 0x7fc020157200 2023-10-06T05:20:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 -- 172.21.15.45:0/329557902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc020157200 msgr2=0x7fc0200caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 --2- 172.21.15.45:0/329557902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc020157200 0x7fc0200caf40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc0200cefc0 tx=0x7fc018009780 comp rx=0 tx=0).stop 2023-10-06T05:20:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 -- 172.21.15.45:0/329557902 shutdown_connections 2023-10-06T05:20:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 --2- 172.21.15.45:0/329557902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc020157200 0x7fc0200caf40 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 --2- 172.21.15.45:0/329557902 >> v2:172.21.15.144:3300/0 conn(0x7fc0200c8ff0 0x7fc0200c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 -- 172.21.15.45:0/329557902 >> 172.21.15.45:0/329557902 conn(0x7fc0200cdf50 msgr2=0x7fc0200ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 -- 172.21.15.45:0/329557902 shutdown_connections 2023-10-06T05:20:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.142+0000 7fc024a46700 1 -- 172.21.15.45:0/329557902 wait complete. 2023-10-06T05:20:40.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.144+0000 7fc024a46700 1 Processor -- start 2023-10-06T05:20:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.144+0000 7fc024a46700 1 -- start start 2023-10-06T05:20:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc024a46700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0200c8ff0 0x7fc020164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc024a46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0200caad0 0x7fc020167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc024a46700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc020157200 0x7fc0200c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc024a46700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c7ef0 con 0x7fc0200c8ff0 2023-10-06T05:20:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc024a46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc020167ff0 con 0x7fc0200caad0 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc024a46700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200af6a0 con 0x7fc020157200 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0200c8ff0 0x7fc020164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0200c8ff0 0x7fc020164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35962/0 (socket says 172.21.15.45:35962) 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 -- 172.21.15.45:0/1932219890 learned_addr learned my addr 172.21.15.45:0/1932219890 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01dd9b700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0200caad0 0x7fc020167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 -- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc020157200 msgr2=0x7fc0200c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc020157200 0x7fc0200c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 -- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0200caad0 msgr2=0x7fc020167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0200caad0 0x7fc020167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 -- 172.21.15.45:0/1932219890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc018009a20 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01dd9b700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0200caad0 0x7fc020167ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc01e59c700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0200c8ff0 0x7fc020164610 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc01000e930 tx=0x7fc0100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc0167fc700 1 -- 172.21.15.45:0/1932219890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000eed0 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc0167fc700 1 -- 172.21.15.45:0/1932219890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01000c850 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0200aea90 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.145+0000 7fc0167fc700 1 -- 172.21.15.45:0/1932219890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000cbf0 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.146+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0200af050 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.147+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02006f410 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.147+0000 7fc0167fc700 1 -- 172.21.15.45:0/1932219890 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc010013470 con 0x7fc0200c8ff0 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.147+0000 7fc0167fc700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc000066f30 0x7fc0000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.147+0000 7fc0167fc700 1 -- 172.21.15.45:0/1932219890 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fc010084ae0 con 0x7fc0200c8ff0 2023-10-06T05:20:40.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.148+0000 7fc01dd9b700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc000066f30 0x7fc0000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.148+0000 7fc01dd9b700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc000066f30 0x7fc0000693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc02015a860 tx=0x7fc008009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.152+0000 7fc0167fc700 1 -- 172.21.15.45:0/1932219890 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0100519d0 con 0x7fc0200c8ff0 2023-10-06T05:20:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.396+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc02004bb40 con 0x7fc0200c8ff0 2023-10-06T05:20:40.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.397+0000 7fc0167fc700 1 -- 172.21.15.45:0/1932219890 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fc010054ff0 con 0x7fc0200c8ff0 2023-10-06T05:20:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:40.399 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1230704519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1230704519},{"type":"v1","addr":"172.21.15.45:6833","nonce":1230704519}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:09.524580+0000","modified":"2023-10-06T05:19:00.913269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5543},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5543":{"gid":5543,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/517622572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":517622572},{"type":"v1","addr":"172.21.15.144:6837","nonce":517622572}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:40.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc000066f30 msgr2=0x7fc0000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc000066f30 0x7fc0000693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc02015a860 tx=0x7fc008009040 comp rx=0 tx=0).stop 2023-10-06T05:20:40.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0200c8ff0 msgr2=0x7fc020164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0200c8ff0 0x7fc020164610 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc01000e930 tx=0x7fc0100091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 shutdown_connections 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc000066f30 0x7fc0000693e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc020157200 0x7fc0200c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0200c8ff0 0x7fc020164610 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 --2- 172.21.15.45:0/1932219890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0200caad0 0x7fc020167ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 >> 172.21.15.45:0/1932219890 conn(0x7fc0200cdf50 msgr2=0x7fc02006d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 shutdown_connections 2023-10-06T05:20:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.399+0000 7fc024a46700 1 -- 172.21.15.45:0/1932219890 wait complete. 2023-10-06T05:20:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 62 2023-10-06T05:20:40.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:40.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.545+0000 7f837fd7c700 1 Processor -- start 2023-10-06T05:20:40.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.545+0000 7f837fd7c700 1 -- start start 2023-10-06T05:20:40.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.546+0000 7f837fd7c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8378155cd0 0x7f837815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.546+0000 7f837fd7c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83781560a0 0x7f83780adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.546+0000 7f837fd7c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83780ae8d0 con 0x7f83780ccdc0 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.546+0000 7f837fd7c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83780aebb0 con 0x7f8378155cd0 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.546+0000 7f837fd7c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83780aee90 con 0x7f83781560a0 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.546+0000 7f837db18700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8378155cd0 0x7f837815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837db18700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8378155cd0 0x7f837815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35976/0 (socket says 172.21.15.45:35976) 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837db18700 1 -- 172.21.15.45:0/3678527787 learned_addr learned my addr 172.21.15.45:0/3678527787 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837d317700 1 --2- 172.21.15.45:0/3678527787 >> v2:172.21.15.144:3300/0 conn(0x7f83781560a0 0x7f83780adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837db18700 1 -- 172.21.15.45:0/3678527787 >> v2:172.21.15.144:3300/0 conn(0x7f83781560a0 msgr2=0x7f83780adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837db18700 1 --2- 172.21.15.45:0/3678527787 >> v2:172.21.15.144:3300/0 conn(0x7f83781560a0 0x7f83780adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837db18700 1 -- 172.21.15.45:0/3678527787 >> v1:172.21.15.144:6789/0 conn(0x7f83780ccdc0 legacy=0x7f83780cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837db18700 1 -- 172.21.15.45:0/3678527787 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780af170 con 0x7f8378155cd0 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837db18700 1 --2- 172.21.15.45:0/3678527787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8378155cd0 0x7f837815a730 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f83780cda20 tx=0x7f8374009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3a0d99d3bf8329 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837cb16700 1 -- 172.21.15.45:0/3678527787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374009e70 con 0x7f8378155cd0 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837cb16700 1 -- 172.21.15.45:0/3678527787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400b860 con 0x7f8378155cd0 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.547+0000 7f837cb16700 1 -- 172.21.15.45:0/3678527787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400bc00 con 0x7f8378155cd0 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 -- 172.21.15.45:0/3678527787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8378155cd0 msgr2=0x7f837815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3678527787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8378155cd0 0x7f837815a730 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f83780cda20 tx=0x7f8374009250 comp rx=0 tx=0).stop 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 -- 172.21.15.45:0/3678527787 shutdown_connections 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3678527787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8378155cd0 0x7f837815a730 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3678527787 >> v2:172.21.15.144:3300/0 conn(0x7f83781560a0 0x7f83780adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 -- 172.21.15.45:0/3678527787 >> 172.21.15.45:0/3678527787 conn(0x7f837806f090 msgr2=0x7f837806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 -- 172.21.15.45:0/3678527787 shutdown_connections 2023-10-06T05:20:40.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.548+0000 7f837fd7c700 1 -- 172.21.15.45:0/3678527787 wait complete. 2023-10-06T05:20:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.549+0000 7f837fd7c700 1 Processor -- start 2023-10-06T05:20:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.549+0000 7f837fd7c700 1 -- start start 2023-10-06T05:20:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837fd7c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83780ccdc0 0x7f83780c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837fd7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8378155cd0 0x7f8378156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837fd7c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83781560a0 0x7f83780cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837fd7c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8378169ec0 con 0x7f83781560a0 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837fd7c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8378156f90 con 0x7f8378155cd0 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837fd7c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8378157270 con 0x7f83780ccdc0 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837e319700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83781560a0 0x7f83780cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837e319700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83781560a0 0x7f83780cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35980/0 (socket says 172.21.15.45:35980) 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837e319700 1 -- 172.21.15.45:0/3028553906 learned_addr learned my addr 172.21.15.45:0/3028553906 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837d317700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8378155cd0 0x7f8378156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837db18700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83780ccdc0 0x7f83780c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837e319700 1 -- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83780ccdc0 msgr2=0x7f83780c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837e319700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83780ccdc0 0x7f83780c2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837e319700 1 -- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8378155cd0 msgr2=0x7f8378156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.550+0000 7f837e319700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8378155cd0 0x7f8378156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.551+0000 7f837e319700 1 -- 172.21.15.45:0/3028553906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740094f0 con 0x7f83781560a0 2023-10-06T05:20:40.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.551+0000 7f837e319700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83781560a0 0x7f83780cbd80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f837000c810 tx=0x7f8370001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.551+0000 7f836dffb700 1 -- 172.21.15.45:0/3028553906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370012630 con 0x7f83781560a0 2023-10-06T05:20:40.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.551+0000 7f836dffb700 1 -- 172.21.15.45:0/3028553906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83700127b0 con 0x7f83781560a0 2023-10-06T05:20:40.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.551+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8378157550 con 0x7f83781560a0 2023-10-06T05:20:40.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.551+0000 7f836dffb700 1 -- 172.21.15.45:0/3028553906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370012b70 con 0x7f83781560a0 2023-10-06T05:20:40.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.551+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83780c3440 con 0x7f83781560a0 2023-10-06T05:20:40.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.552+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837816a070 con 0x7f83781560a0 2023-10-06T05:20:40.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.553+0000 7f836dffb700 1 -- 172.21.15.45:0/3028553906 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8370003030 con 0x7f83781560a0 2023-10-06T05:20:40.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.553+0000 7f836dffb700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f835c066f30 0x7f835c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.554+0000 7f836dffb700 1 -- 172.21.15.45:0/3028553906 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f8370084e70 con 0x7f83781560a0 2023-10-06T05:20:40.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.554+0000 7f837db18700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f835c066f30 0x7f835c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.554+0000 7f837db18700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f835c066f30 0x7f835c0693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f83780cda20 tx=0x7f8374017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.557+0000 7f836dffb700 1 -- 172.21.15.45:0/3028553906 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8370051d60 con 0x7f83781560a0 2023-10-06T05:20:40.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.760+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83780ae4e0 con 0x7f83781560a0 2023-10-06T05:20:40.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.761+0000 7f836dffb700 1 -- 172.21.15.45:0/3028553906 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+17524 (secure 0 0 0) 0x7f8370055380 con 0x7f83781560a0 2023-10-06T05:20:40.763 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:40.764 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":68,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:18:57.905683+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:08.043401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:08.487972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.17","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/4169661273":"2023-10-07T05:18:57.905623+0000","172.21.15.45:6833/896514784":"2023-10-07T05:18:21.174171+0000","172.21.15.45:6832/4169661273":"2023-10-07T05:18:57.905623+0000","172.21.15.45:6832/896514784":"2023-10-07T05:18:21.174171+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:40.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f835c066f30 msgr2=0x7f835c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f835c066f30 0x7f835c0693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f83780cda20 tx=0x7f8374017000 comp rx=0 tx=0).stop 2023-10-06T05:20:40.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83781560a0 msgr2=0x7f83780cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83781560a0 0x7f83780cbd80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f837000c810 tx=0x7f8370001650 comp rx=0 tx=0).stop 2023-10-06T05:20:40.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 shutdown_connections 2023-10-06T05:20:40.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f835c066f30 0x7f835c0693e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83781560a0 0x7f83780cbd80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83780ccdc0 0x7f83780c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 --2- 172.21.15.45:0/3028553906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8378155cd0 0x7f8378156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 >> 172.21.15.45:0/3028553906 conn(0x7f837806f090 msgr2=0x7f8378067620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 shutdown_connections 2023-10-06T05:20:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.764+0000 7f837fd7c700 1 -- 172.21.15.45:0/3028553906 wait complete. 2023-10-06T05:20:40.779 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:20:40.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.911+0000 7fd3bc0ec700 1 Processor -- start 2023-10-06T05:20:40.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.912+0000 7fd3bc0ec700 1 -- start start 2023-10-06T05:20:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3bc0ec700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3b40c93c0 0x7fd3b4156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3bc0ec700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b4157030 0x7fd3b40cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3bc0ec700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b415b000 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3bc0ec700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b40c9830 con 0x7fd3b40c93c0 2023-10-06T05:20:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3bc0ec700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b415b2e0 con 0x7fd3b4157030 2023-10-06T05:20:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3b9687700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3b40c93c0 0x7fd3b4156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3ba689700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b4157030 0x7fd3b40cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3b9e88700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd3b40c8ff0 0x7fd3b40caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:35600/0 (socket says 172.21.15.45:35600) 2023-10-06T05:20:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3b9687700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3b40c93c0 0x7fd3b4156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:55970/0 (socket says 172.21.15.45:55970) 2023-10-06T05:20:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3ba689700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b4157030 0x7fd3b40cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:40020/0 (socket says 172.21.15.45:40020) 2023-10-06T05:20:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3b9e88700 1 -- 172.21.15.45:0/2398419122 learned_addr learned my addr 172.21.15.45:0/2398419122 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3ba689700 1 -- 172.21.15.45:0/2398419122 >> v2:172.21.15.144:3300/0 conn(0x7fd3b40c93c0 msgr2=0x7fd3b4156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3ba689700 1 --2- 172.21.15.45:0/2398419122 >> v2:172.21.15.144:3300/0 conn(0x7fd3b40c93c0 0x7fd3b4156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3ba689700 1 -- 172.21.15.45:0/2398419122 >> v1:172.21.15.144:6789/0 conn(0x7fd3b40c8ff0 legacy=0x7fd3b40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3ba689700 1 -- 172.21.15.45:0/2398419122 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b415b5c0 con 0x7fd3b4157030 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.913+0000 7fd3b9687700 1 --2- 172.21.15.45:0/2398419122 >> v2:172.21.15.144:3300/0 conn(0x7fd3b40c93c0 0x7fd3b4156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3ba689700 1 --2- 172.21.15.45:0/2398419122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b4157030 0x7fd3b40cc330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e60 tx=0x7fd3ac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b09d388b818591e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3b8e86700 1 -- 172.21.15.45:0/2398419122 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00e070 con 0x7fd3b4157030 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3b8e86700 1 -- 172.21.15.45:0/2398419122 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac00bb60 con 0x7fd3b4157030 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3b8e86700 1 -- 172.21.15.45:0/2398419122 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac012430 con 0x7fd3b4157030 2023-10-06T05:20:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/2398419122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b4157030 msgr2=0x7fd3b40cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/2398419122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b4157030 0x7fd3b40cc330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e60 tx=0x7fd3ac009780 comp rx=0 tx=0).stop 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/2398419122 shutdown_connections 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/2398419122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b4157030 0x7fd3b40cc330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/2398419122 >> v2:172.21.15.144:3300/0 conn(0x7fd3b40c93c0 0x7fd3b4156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.914+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/2398419122 >> 172.21.15.45:0/2398419122 conn(0x7fd3b40cdf50 msgr2=0x7fd3b40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.915+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/2398419122 shutdown_connections 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.915+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/2398419122 wait complete. 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.916+0000 7fd3bc0ec700 1 Processor -- start 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.916+0000 7fd3bc0ec700 1 -- start start 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.916+0000 7fd3bc0ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b40c8ff0 0x7fd3b4069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.916+0000 7fd3bc0ec700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b40c93c0 0x7fd3b40c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3bc0ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b4157030 0x7fd3b4149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3bc0ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c6bc0 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3bc0ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b4069720 con 0x7fd3b4157030 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3bc0ec700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c6660 con 0x7fd3b40c93c0 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b40c8ff0 0x7fd3b4069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b40c8ff0 0x7fd3b4069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35994/0 (socket says 172.21.15.45:35994) 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 -- 172.21.15.45:0/4088608675 learned_addr learned my addr 172.21.15.45:0/4088608675 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9687700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b40c93c0 0x7fd3b40c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3ba689700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b4157030 0x7fd3b4149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 -- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b40c93c0 msgr2=0x7fd3b40c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b40c93c0 0x7fd3b40c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 -- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b4157030 msgr2=0x7fd3b4149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b4157030 0x7fd3b4149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 -- 172.21.15.45:0/4088608675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b0009420 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9687700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b40c93c0 0x7fd3b40c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3ba689700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b4157030 0x7fd3b4149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3b9e88700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b40c8ff0 0x7fd3b4069ef0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400c7f0 tx=0x7fd3a4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3a9ffb700 1 -- 172.21.15.45:0/4088608675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4005b30 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.917+0000 7fd3a9ffb700 1 -- 172.21.15.45:0/4088608675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a4005cb0 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.918+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac009a20 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.918+0000 7fd3a9ffb700 1 -- 172.21.15.45:0/4088608675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004a80 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.918+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b40c68e0 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.919+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b406a810 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.919+0000 7fd3a9ffb700 1 -- 172.21.15.45:0/4088608675 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3a40070d0 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.919+0000 7fd3a9ffb700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd398066f30 0x7fd3980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.919+0000 7fd3a9ffb700 1 -- 172.21.15.45:0/4088608675 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fd3a4083ba0 con 0x7fd3b40c8ff0 2023-10-06T05:20:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.920+0000 7fd3b9687700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd398066f30 0x7fd3980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.920+0000 7fd3b9687700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd398066f30 0x7fd3980693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40b3b30 tx=0x7fd3b00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:40.924+0000 7fd3a9ffb700 1 -- 172.21.15.45:0/4088608675 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3a404b5a0 con 0x7fd3b40c8ff0 2023-10-06T05:20:41.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.177+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd3b404bb40 con 0x7fd3b40c8ff0 2023-10-06T05:20:41.180 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:20:41.178+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:20:41.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.182+0000 7fd3a9ffb700 1 -- 172.21.15.45:0/4088608675 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd3a4050a90 con 0x7fd3b40c8ff0 2023-10-06T05:20:41.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd398066f30 msgr2=0x7fd3980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd398066f30 0x7fd3980693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40b3b30 tx=0x7fd3b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b40c8ff0 msgr2=0x7fd3b4069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b40c8ff0 0x7fd3b4069ef0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400c7f0 tx=0x7fd3a4007f70 comp rx=0 tx=0).stop 2023-10-06T05:20:41.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 shutdown_connections 2023-10-06T05:20:41.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd398066f30 0x7fd3980693e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b4157030 0x7fd3b4149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b40c8ff0 0x7fd3b4069ef0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 --2- 172.21.15.45:0/4088608675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b40c93c0 0x7fd3b40c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 >> 172.21.15.45:0/4088608675 conn(0x7fd3b40cdf50 msgr2=0x7fd3b41509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 shutdown_connections 2023-10-06T05:20:41.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.184+0000 7fd3bc0ec700 1 -- 172.21.15.45:0/4088608675 wait complete. 2023-10-06T05:20:41.187 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:20:41.188 INFO:tasks.ceph.mds.d.smithi144.stderr: -3693> 2023-10-06T05:19:05.078+0000 7f53b2120700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:20:41.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:20:41.214 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:20:41.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.331+0000 7f72249dc700 1 Processor -- start 2023-10-06T05:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.331+0000 7f72249dc700 1 -- start start 2023-10-06T05:20:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f72249dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72200ccb70 0x7f72200aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f72249dc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72200c96a0 con 0x7f7220157330 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f72249dc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72200c3b50 con 0x7f72200c8b70 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f72249dc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7220157700 con 0x7f72200ccb70 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721dd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72200ccb70 0x7f72200aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721dd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72200ccb70 0x7f72200aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36010/0 (socket says 172.21.15.45:36010) 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721dd9b700 1 -- 172.21.15.45:0/3439782859 learned_addr learned my addr 172.21.15.45:0/3439782859 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721e59c700 1 --1- 172.21.15.45:0/3439782859 >> v1:172.21.15.45:6789/0 conn(0x7f7220157330 0x7f72200ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33282/0 (socket says 172.21.15.45:33282) 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721dd9b700 1 -- 172.21.15.45:0/3439782859 >> v1:172.21.15.144:6789/0 conn(0x7f72200c8b70 legacy=0x7f72200c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721dd9b700 1 -- 172.21.15.45:0/3439782859 >> v1:172.21.15.45:6789/0 conn(0x7f7220157330 legacy=0x7f72200ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721d59a700 1 -- 172.21.15.45:0/3439782859 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152324230 0 0) 0x7f7210001550 con 0x7f7220157330 2023-10-06T05:20:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721dd9b700 1 -- 172.21.15.45:0/3439782859 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200c98e0 con 0x7f72200ccb70 2023-10-06T05:20:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.332+0000 7f721dd9b700 1 --2- 172.21.15.45:0/3439782859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200aaae0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f72080099b0 tx=0x7f7208009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884c7ac188eed7c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f721d59a700 1 -- 172.21.15.45:0/3439782859 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720800b860 con 0x7f72200ccb70 2023-10-06T05:20:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f721d59a700 1 -- 172.21.15.45:0/3439782859 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720800b9c0 con 0x7f72200ccb70 2023-10-06T05:20:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f721d59a700 1 -- 172.21.15.45:0/3439782859 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720800bd60 con 0x7f72200ccb70 2023-10-06T05:20:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f72249dc700 1 -- 172.21.15.45:0/3439782859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 msgr2=0x7f72200aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f72249dc700 1 --2- 172.21.15.45:0/3439782859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200aaae0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f72080099b0 tx=0x7f7208009250 comp rx=0 tx=0).stop 2023-10-06T05:20:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f72249dc700 1 -- 172.21.15.45:0/3439782859 shutdown_connections 2023-10-06T05:20:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f72249dc700 1 --2- 172.21.15.45:0/3439782859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200aaae0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f72249dc700 1 -- 172.21.15.45:0/3439782859 >> 172.21.15.45:0/3439782859 conn(0x7f7220067e00 msgr2=0x7f72200cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f72249dc700 1 -- 172.21.15.45:0/3439782859 shutdown_connections 2023-10-06T05:20:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.333+0000 7f72249dc700 1 -- 172.21.15.45:0/3439782859 wait complete. 2023-10-06T05:20:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.335+0000 7f72249dc700 1 Processor -- start 2023-10-06T05:20:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.335+0000 7f72249dc700 1 -- start start 2023-10-06T05:20:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f72249dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72200c8b70 0x7f72200ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f72249dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f72249dc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7220157330 0x7f72200b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f72249dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200c60a0 con 0x7f72200ccb70 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f72249dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200c5490 con 0x7f72200c8b70 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f72249dc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72200c5770 con 0x7f7220157330 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f721dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f721dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36022/0 (socket says 172.21.15.45:36022) 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f721dd9b700 1 -- 172.21.15.45:0/2915837285 learned_addr learned my addr 172.21.15.45:0/2915837285 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f721e59c700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72200c8b70 0x7f72200ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f721dd9b700 1 -- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7220157330 msgr2=0x7f72200b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.336+0000 7f721ed9d700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7220157330 0x7f72200b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f721dd9b700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7220157330 0x7f72200b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f721dd9b700 1 -- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72200c8b70 msgr2=0x7f72200ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f721dd9b700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72200c8b70 0x7f72200ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f721dd9b700 1 -- 172.21.15.45:0/2915837285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72080094f0 con 0x7f72200ccb70 2023-10-06T05:20:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f721ed9d700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7220157330 0x7f72200b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f721e59c700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72200c8b70 0x7f72200ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f721dd9b700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200c6530 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f7208006f50 tx=0x7f7208017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f72167fc700 1 -- 172.21.15.45:0/2915837285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720801f070 con 0x7f72200ccb70 2023-10-06T05:20:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f72167fc700 1 -- 172.21.15.45:0/2915837285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208011d10 con 0x7f72200ccb70 2023-10-06T05:20:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72200c5a50 con 0x7f72200ccb70 2023-10-06T05:20:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f72167fc700 1 -- 172.21.15.45:0/2915837285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208025670 con 0x7f72200ccb70 2023-10-06T05:20:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.337+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72200ad770 con 0x7f72200ccb70 2023-10-06T05:20:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.338+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72200c6250 con 0x7f72200ccb70 2023-10-06T05:20:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.338+0000 7f72167fc700 1 -- 172.21.15.45:0/2915837285 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7208023070 con 0x7f72200ccb70 2023-10-06T05:20:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.339+0000 7f72167fc700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.339+0000 7f72167fc700 1 -- 172.21.15.45:0/2915837285 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f7208085ff0 con 0x7f72200ccb70 2023-10-06T05:20:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.339+0000 7f721e59c700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.340+0000 7f721e59c700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7210000f80 tx=0x7f7210009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.343+0000 7f72167fc700 1 -- 172.21.15.45:0/2915837285 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72200c6250 con 0x7f72200ccb70 2023-10-06T05:20:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:41.595+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f72200c6250 con 0x7f72200ccb70 2023-10-06T05:20:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.183+0000 7f72167fc700 1 -- 172.21.15.45:0/2915837285 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f72200c6250 con 0x7f72200ccb70 2023-10-06T05:20:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7200066f30 msgr2=0x7f72000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7210000f80 tx=0x7f7210009040 comp rx=0 tx=0).stop 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 msgr2=0x7f72200c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200c6530 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f7208006f50 tx=0x7f7208017000 comp rx=0 tx=0).stop 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 shutdown_connections 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7220157330 0x7f72200b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72200c8b70 0x7f72200ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 --2- 172.21.15.45:0/2915837285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72200ccb70 0x7f72200c6530 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 >> 172.21.15.45:0/2915837285 conn(0x7f7220067e00 msgr2=0x7f72200c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.186+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 shutdown_connections 2023-10-06T05:20:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.187+0000 7f72249dc700 1 -- 172.21.15.45:0/2915837285 wait complete. 2023-10-06T05:20:42.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:20:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.342+0000 7f06c3bef700 1 Processor -- start 2023-10-06T05:20:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.342+0000 7f06c3bef700 1 -- start start 2023-10-06T05:20:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c3bef700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06bc159160 0x7f06bc0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c3bef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 0x7f06bc067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c3bef700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc0c8140 con 0x7f06bc159530 2023-10-06T05:20:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c3bef700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc067fe0 con 0x7f06bc159160 2023-10-06T05:20:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c3bef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0ca1e0 con 0x7f06bc067620 2023-10-06T05:20:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c118a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06bc159530 0x7f06bc0c9780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33286/0 (socket says 172.21.15.45:33286) 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c118a700 1 -- 172.21.15.45:0/3627917085 learned_addr learned my addr 172.21.15.45:0/3627917085 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c198b700 1 --2- 172.21.15.45:0/3627917085 >> v2:172.21.15.144:3300/0 conn(0x7f06bc159160 0x7f06bc0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c218c700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 0x7f06bc067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439218993 0 0) 0x7f06bc0c8140 con 0x7f06bc159530 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.343+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a8003570 con 0x7f06bc159530 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 541292779 0 0) 0x7f06a8003570 con 0x7f06bc159530 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 msgr2=0x7f06bc067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 0x7f06bc067aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 >> v2:172.21.15.144:3300/0 conn(0x7f06bc159160 msgr2=0x7f06bc0cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 --2- 172.21.15.45:0/3627917085 >> v2:172.21.15.144:3300/0 conn(0x7f06bc159160 0x7f06bc0cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0ca4c0 con 0x7f06bc159530 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f06ac001470 con 0x7f06bc159530 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 >> v1:172.21.15.45:6789/0 conn(0x7f06bc159530 legacy=0x7f06bc0c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a8003a60 0x7f06a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 0x7f06a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8004d50 0x7f06a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc067fe0 con 0x7f06a8003a60 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0ca1e0 con 0x7f06a8004d50 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a800ba60 con 0x7f06bc067620 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c198b700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a8003a60 0x7f06a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c118a700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 0x7f06a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c218c700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8004d50 0x7f06a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c198b700 1 -- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 msgr2=0x7f06a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c198b700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc067620 0x7f06a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c198b700 1 -- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8004d50 msgr2=0x7f06a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c198b700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8004d50 0x7f06a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c198b700 1 -- 172.21.15.45:0/3627917085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8003570 con 0x7f06a8003a60 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.344+0000 7f06c198b700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a8003a60 0x7f06a8003e70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009c00 tx=0x7f06b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868e6781c4629c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800e6b0 con 0x7f06a8003a60 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b800e810 con 0x7f06a8003a60 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c0989700 1 -- 172.21.15.45:0/3627917085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800ebd0 con 0x7f06a8003a60 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c3bef700 1 -- 172.21.15.45:0/3627917085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a8003a60 msgr2=0x7f06a8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c3bef700 1 --2- 172.21.15.45:0/3627917085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a8003a60 0x7f06a8003e70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009c00 tx=0x7f06b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c218c700 1 -- 172.21.15.45:0/3627917085 reap_dead start 2023-10-06T05:20:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c3bef700 1 -- 172.21.15.45:0/3627917085 shutdown_connections 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c3bef700 1 -- 172.21.15.45:0/3627917085 >> 172.21.15.45:0/3627917085 conn(0x7f06bc068730 msgr2=0x7f06bc068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c3bef700 1 -- 172.21.15.45:0/3627917085 shutdown_connections 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.345+0000 7f06c3bef700 1 -- 172.21.15.45:0/3627917085 wait complete. 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.346+0000 7f06c3bef700 1 Processor -- start 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.346+0000 7f06c3bef700 1 -- start start 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c3bef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8003a60 0x7f06bc149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c3bef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc156d60 0x7f06bc15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c3bef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06bc0c5540 0x7f06bc0c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c3bef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0c5f20 con 0x7f06bc0c5540 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c3bef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc169a80 con 0x7f06a8003a60 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c3bef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc169c90 con 0x7f06bc156d60 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c218c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06bc0c5540 0x7f06bc0c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c218c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06bc0c5540 0x7f06bc0c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36046/0 (socket says 172.21.15.45:36046) 2023-10-06T05:20:42.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c218c700 1 -- 172.21.15.45:0/1281257909 learned_addr learned my addr 172.21.15.45:0/1281257909 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:42.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c198b700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8003a60 0x7f06bc149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.347+0000 7f06c118a700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc156d60 0x7f06bc15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c218c700 1 -- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc156d60 msgr2=0x7f06bc15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c218c700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc156d60 0x7f06bc15c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c218c700 1 -- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8003a60 msgr2=0x7f06bc149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c218c700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8003a60 0x7f06bc149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c218c700 1 -- 172.21.15.45:0/1281257909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc149fe0 con 0x7f06bc0c5540 2023-10-06T05:20:42.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c218c700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06bc0c5540 0x7f06bc0c59e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0697f0 tx=0x7f06b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06b1ffb700 1 -- 172.21.15.45:0/1281257909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400e070 con 0x7f06bc0c5540 2023-10-06T05:20:42.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06b1ffb700 1 -- 172.21.15.45:0/1281257909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b4011b80 con 0x7f06bc0c5540 2023-10-06T05:20:42.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc169f30 con 0x7f06bc0c5540 2023-10-06T05:20:42.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06b1ffb700 1 -- 172.21.15.45:0/1281257909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4016470 con 0x7f06bc0c5540 2023-10-06T05:20:42.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.348+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc16a4f0 con 0x7f06bc0c5540 2023-10-06T05:20:42.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.349+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc04c380 con 0x7f06bc0c5540 2023-10-06T05:20:42.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.350+0000 7f06b1ffb700 1 -- 172.21.15.45:0/1281257909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06b4016860 con 0x7f06bc0c5540 2023-10-06T05:20:42.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.350+0000 7f06b1ffb700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06a0066f30 0x7f06a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.350+0000 7f06b1ffb700 1 -- 172.21.15.45:0/1281257909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f06b40855d0 con 0x7f06bc0c5540 2023-10-06T05:20:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.350+0000 7f06c198b700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06a0066f30 0x7f06a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.351+0000 7f06c198b700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06a0066f30 0x7f06a00693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06b80091e0 tx=0x7f06b801a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.355+0000 7f06b1ffb700 1 -- 172.21.15.45:0/1281257909 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06b40523c0 con 0x7f06bc0c5540 2023-10-06T05:20:42.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:42.562+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06bc16abf0 con 0x7f06bc0c5540 2023-10-06T05:20:43.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.190+0000 7f06b1ffb700 1 -- 172.21.15.45:0/1281257909 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) v1 ==== 181+0+0 (secure 0 0 0) 0x7f06b40559e0 con 0x7f06bc0c5540 2023-10-06T05:20:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06a0066f30 msgr2=0x7f06a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06a0066f30 0x7f06a00693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06b80091e0 tx=0x7f06b801a000 comp rx=0 tx=0).stop 2023-10-06T05:20:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06bc0c5540 msgr2=0x7f06bc0c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06bc0c5540 0x7f06bc0c59e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0697f0 tx=0x7f06b400b040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 shutdown_connections 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f06a0066f30 0x7f06a00693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06bc0c5540 0x7f06bc0c59e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a8003a60 0x7f06bc149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 --2- 172.21.15.45:0/1281257909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06bc156d60 0x7f06bc15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 >> 172.21.15.45:0/1281257909 conn(0x7f06bc068730 msgr2=0x7f06bc0c2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 shutdown_connections 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.193+0000 7f06c3bef700 1 -- 172.21.15.45:0/1281257909 wait complete. 2023-10-06T05:20:43.197 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:20:43.213 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:20:43.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.350+0000 7f768b0c2700 1 Processor -- start 2023-10-06T05:20:43.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.350+0000 7f768b0c2700 1 -- start start 2023-10-06T05:20:43.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.351+0000 7f768b0c2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76840cd590 0x7f76840ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.351+0000 7f768b0c2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76840caa20 con 0x7f76840ccb50 2023-10-06T05:20:43.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.351+0000 7f768b0c2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76840c93d0 con 0x7f768413fa30 2023-10-06T05:20:43.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.351+0000 7f768b0c2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76840c96b0 con 0x7f76840cd590 2023-10-06T05:20:43.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f7683fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76840cd590 0x7f76840ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f7688e5e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f76840ccb50 0x7f768413f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33302/0 (socket says 172.21.15.45:33302) 2023-10-06T05:20:43.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f7688e5e700 1 -- 172.21.15.45:0/804126837 learned_addr learned my addr 172.21.15.45:0/804126837 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f7683fff700 1 -- 172.21.15.45:0/804126837 >> v1:172.21.15.144:6789/0 conn(0x7f768413fa30 legacy=0x7f76840caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f7683fff700 1 -- 172.21.15.45:0/804126837 >> v1:172.21.15.45:6789/0 conn(0x7f76840ccb50 legacy=0x7f768413f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f7683fff700 1 -- 172.21.15.45:0/804126837 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840c7630 con 0x7f76840cd590 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f76837fe700 1 -- 172.21.15.45:0/804126837 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749298282 0 0) 0x7f76840caa20 con 0x7f76840ccb50 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f7683fff700 1 --2- 172.21.15.45:0/804126837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840cd590 0x7f76840ccf20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f76700098e0 tx=0x7f76700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f26cd74bc5b58f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f76837fe700 1 -- 172.21.15.45:0/804126837 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670009e80 con 0x7f76840cd590 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f76837fe700 1 -- 172.21.15.45:0/804126837 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767000b8f0 con 0x7f76840cd590 2023-10-06T05:20:43.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.352+0000 7f76837fe700 1 -- 172.21.15.45:0/804126837 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767000bc70 con 0x7f76840cd590 2023-10-06T05:20:43.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.353+0000 7f768b0c2700 1 -- 172.21.15.45:0/804126837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840cd590 msgr2=0x7f76840ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.353+0000 7f768b0c2700 1 --2- 172.21.15.45:0/804126837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840cd590 0x7f76840ccf20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f76700098e0 tx=0x7f76700091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.353+0000 7f768b0c2700 1 -- 172.21.15.45:0/804126837 shutdown_connections 2023-10-06T05:20:43.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.353+0000 7f768b0c2700 1 --2- 172.21.15.45:0/804126837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840cd590 0x7f76840ccf20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.353+0000 7f768b0c2700 1 -- 172.21.15.45:0/804126837 >> 172.21.15.45:0/804126837 conn(0x7f7684068730 msgr2=0x7f7684068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.353+0000 7f768b0c2700 1 -- 172.21.15.45:0/804126837 shutdown_connections 2023-10-06T05:20:43.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.353+0000 7f768b0c2700 1 -- 172.21.15.45:0/804126837 wait complete. 2023-10-06T05:20:43.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.354+0000 7f768b0c2700 1 Processor -- start 2023-10-06T05:20:43.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.354+0000 7f768b0c2700 1 -- start start 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f768b0c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840ccb50 0x7f76841596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f768b0c2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840cd590 0x7f7684153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f768b0c2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768413fa30 0x7f76840c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f768b0c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841694d0 con 0x7f76840ccb50 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f768b0c2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7684159c00 con 0x7f768413fa30 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f768b0c2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7684168900 con 0x7f76840cd590 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f7688e5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840ccb50 0x7f76841596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f7688e5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840ccb50 0x7f76841596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36052/0 (socket says 172.21.15.45:36052) 2023-10-06T05:20:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f7688e5e700 1 -- 172.21.15.45:0/4249937476 learned_addr learned my addr 172.21.15.45:0/4249937476 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.355+0000 7f768965f700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768413fa30 0x7f76840c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7683fff700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840cd590 0x7f7684153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7688e5e700 1 -- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840cd590 msgr2=0x7f7684153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7688e5e700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840cd590 0x7f7684153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7688e5e700 1 -- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768413fa30 msgr2=0x7f76840c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7688e5e700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768413fa30 0x7f76840c1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7688e5e700 1 -- 172.21.15.45:0/4249937476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7670009420 con 0x7f76840ccb50 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7688e5e700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840ccb50 0x7f76841596c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f767800c840 tx=0x7f7678007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7680ff9700 1 -- 172.21.15.45:0/4249937476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678005c00 con 0x7f76840ccb50 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7680ff9700 1 -- 172.21.15.45:0/4249937476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678005d80 con 0x7f76840ccb50 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f7680ff9700 1 -- 172.21.15.45:0/4249937476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678007320 con 0x7f76840ccb50 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7684168ba0 con 0x7f76840ccb50 2023-10-06T05:20:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.356+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7684169160 con 0x7f76840ccb50 2023-10-06T05:20:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.357+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768404c380 con 0x7f76840ccb50 2023-10-06T05:20:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.358+0000 7f7680ff9700 1 -- 172.21.15.45:0/4249937476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f767801b3b0 con 0x7f76840ccb50 2023-10-06T05:20:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.358+0000 7f7680ff9700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7668066f30 0x7f76680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.358+0000 7f7680ff9700 1 -- 172.21.15.45:0/4249937476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6256+0+0 (secure 0 0 0) 0x7f7678083d40 con 0x7f76840ccb50 2023-10-06T05:20:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.358+0000 7f7683fff700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7668066f30 0x7f76680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.359+0000 7f7683fff700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7668066f30 0x7f76680693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f76700093f0 tx=0x7f7670017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.362+0000 7f7680ff9700 1 -- 172.21.15.45:0/4249937476 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7678050d90 con 0x7f76840ccb50 2023-10-06T05:20:43.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:43.564+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f76840c7910 con 0x7f76840ccb50 2023-10-06T05:20:44.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.193+0000 7f7680ff9700 1 -- 172.21.15.45:0/4249937476 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7678017020 con 0x7f76840ccb50 2023-10-06T05:20:44.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.195+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7668066f30 msgr2=0x7f76680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.195+0000 7f768b0c2700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7668066f30 0x7f76680693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f76700093f0 tx=0x7f7670017000 comp rx=0 tx=0).stop 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.195+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840ccb50 msgr2=0x7f76841596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.195+0000 7f768b0c2700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840ccb50 0x7f76841596c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f767800c840 tx=0x7f7678007d90 comp rx=0 tx=0).stop 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 shutdown_connections 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7668066f30 0x7f76680693e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768413fa30 0x7f76840c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76840ccb50 0x7f76841596c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 --2- 172.21.15.45:0/4249937476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840cd590 0x7f7684153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 >> 172.21.15.45:0/4249937476 conn(0x7f7684068730 msgr2=0x7f7684158ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 shutdown_connections 2023-10-06T05:20:44.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.196+0000 7f768b0c2700 1 -- 172.21.15.45:0/4249937476 wait complete. 2023-10-06T05:20:44.199 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-06T05:20:44.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:20:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.360+0000 7fd21a6b3700 1 Processor -- start 2023-10-06T05:20:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.360+0000 7fd21a6b3700 1 -- start start 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd21a6b3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd214157330 0x7fd2140ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd21a6b3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd2140c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd21a6b3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2140c3b50 con 0x7fd2140aaae0 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd21a6b3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd214157700 con 0x7fd214157330 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd21a6b3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140c98e0 con 0x7fd2140ccb70 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd213fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd214157330 0x7fd2140ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd2137fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd2140aaae0 0x7fd2140c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33306/0 (socket says 172.21.15.45:33306) 2023-10-06T05:20:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.361+0000 7fd2137fe700 1 -- 172.21.15.45:0/1916961453 learned_addr learned my addr 172.21.15.45:0/1916961453 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd218c50700 1 --2- 172.21.15.45:0/1916961453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd2140c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd213fff700 1 -- 172.21.15.45:0/1916961453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 msgr2=0x7fd2140c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd213fff700 1 --2- 172.21.15.45:0/1916961453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd2140c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd213fff700 1 -- 172.21.15.45:0/1916961453 >> v1:172.21.15.45:6789/0 conn(0x7fd2140aaae0 legacy=0x7fd2140c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd212ffd700 1 -- 172.21.15.45:0/1916961453 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934617178 0 0) 0x7fd2140c3b50 con 0x7fd2140aaae0 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd213fff700 1 -- 172.21.15.45:0/1916961453 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2140c9bc0 con 0x7fd214157330 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd213fff700 1 --2- 172.21.15.45:0/1916961453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd2140ce8b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098d0 tx=0x7fd204009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5fd42c13468882 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd212ffd700 1 -- 172.21.15.45:0/1916961453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204009e70 con 0x7fd214157330 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd212ffd700 1 -- 172.21.15.45:0/1916961453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204005c10 con 0x7fd214157330 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.362+0000 7fd212ffd700 1 -- 172.21.15.45:0/1916961453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400b650 con 0x7fd214157330 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 -- 172.21.15.45:0/1916961453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 msgr2=0x7fd2140ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/1916961453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd2140ce8b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098d0 tx=0x7fd204009120 comp rx=0 tx=0).stop 2023-10-06T05:20:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 -- 172.21.15.45:0/1916961453 shutdown_connections 2023-10-06T05:20:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/1916961453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd2140c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/1916961453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd2140ce8b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 -- 172.21.15.45:0/1916961453 >> 172.21.15.45:0/1916961453 conn(0x7fd214067e00 msgr2=0x7fd2140cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 -- 172.21.15.45:0/1916961453 shutdown_connections 2023-10-06T05:20:44.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.363+0000 7fd21a6b3700 1 -- 172.21.15.45:0/1916961453 wait complete. 2023-10-06T05:20:44.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.364+0000 7fd21a6b3700 1 Processor -- start 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.364+0000 7fd21a6b3700 1 -- start start 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd21a6b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2140aaae0 0x7fd21415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd21a6b3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd214153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd21a6b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd214153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd21a6b3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21415bcb0 con 0x7fd214157330 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd21a6b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214167ef0 con 0x7fd2140aaae0 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd21a6b3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd214167130 con 0x7fd2140ccb70 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd218c50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd214153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd218c50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd214153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36072/0 (socket says 172.21.15.45:36072) 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd218c50700 1 -- 172.21.15.45:0/3760350243 learned_addr learned my addr 172.21.15.45:0/3760350243 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd2137fe700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd214153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.365+0000 7fd213fff700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2140aaae0 0x7fd21415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd218c50700 1 -- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 msgr2=0x7fd214153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd218c50700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd214153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd218c50700 1 -- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2140aaae0 msgr2=0x7fd21415c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd218c50700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2140aaae0 0x7fd21415c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd218c50700 1 -- 172.21.15.45:0/3760350243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040093c0 con 0x7fd214157330 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd213fff700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2140aaae0 0x7fd21415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd218c50700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd214153670 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009fa0 tx=0x7fd20c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd1fbfff700 1 -- 172.21.15.45:0/3760350243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00ec70 con 0x7fd214157330 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd1fbfff700 1 -- 172.21.15.45:0/3760350243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c00edd0 con 0x7fd214157330 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd214167410 con 0x7fd214157330 2023-10-06T05:20:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd1fbfff700 1 -- 172.21.15.45:0/3760350243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c014670 con 0x7fd214157330 2023-10-06T05:20:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.366+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2141679d0 con 0x7fd214157330 2023-10-06T05:20:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.367+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd214167bf0 con 0x7fd214157330 2023-10-06T05:20:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.368+0000 7fd1fbfff700 1 -- 172.21.15.45:0/3760350243 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd20c0149e0 con 0x7fd214157330 2023-10-06T05:20:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.368+0000 7fd1fbfff700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200066f30 0x7fd2000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.368+0000 7fd1fbfff700 1 -- 172.21.15.45:0/3760350243 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5784+0+0 (secure 0 0 0) 0x7fd20c084ce0 con 0x7fd214157330 2023-10-06T05:20:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.368+0000 7fd213fff700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200066f30 0x7fd2000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.369+0000 7fd213fff700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200066f30 0x7fd2000693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd21415ad10 tx=0x7fd204016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.373+0000 7fd1fbfff700 1 -- 172.21.15.45:0/3760350243 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd20c051f00 con 0x7fd214157330 2023-10-06T05:20:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.574+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd21406e350 con 0x7fd214157330 2023-10-06T05:20:44.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.578+0000 7fd1fbfff700 1 -- 172.21.15.45:0/3760350243 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd20c055520 con 0x7fd214157330 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.580+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200066f30 msgr2=0x7fd2000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.580+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200066f30 0x7fd2000693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd21415ad10 tx=0x7fd204016000 comp rx=0 tx=0).stop 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.580+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 msgr2=0x7fd214153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.580+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd214153670 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009fa0 tx=0x7fd20c009900 comp rx=0 tx=0).stop 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 shutdown_connections 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200066f30 0x7fd2000693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd214157330 0x7fd214153670 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2140aaae0 0x7fd21415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 --2- 172.21.15.45:0/3760350243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2140ccb70 0x7fd214153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 >> 172.21.15.45:0/3760350243 conn(0x7fd214067e00 msgr2=0x7fd2140c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 shutdown_connections 2023-10-06T05:20:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.581+0000 7fd21a6b3700 1 -- 172.21.15.45:0/3760350243 wait complete. 2023-10-06T05:20:44.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2023-10-06T05:20:44.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.728+0000 7f675c47e700 1 Processor -- start 2023-10-06T05:20:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.728+0000 7f675c47e700 1 -- start start 2023-10-06T05:20:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675c47e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f67540b1cb0 0x7f675406a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675c47e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f675406ad60 0x7f67540ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675c47e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675406b1c0 con 0x7f67540b18e0 2023-10-06T05:20:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675c47e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6754068750 con 0x7f675406ad60 2023-10-06T05:20:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675c47e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6754067860 con 0x7f67540b1cb0 2023-10-06T05:20:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675aa1b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f675406ad60 0x7f67540ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675aa1b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f675406ad60 0x7f67540ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36086/0 (socket says 172.21.15.45:36086) 2023-10-06T05:20:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675a21a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f67540b18e0 0x7f675415b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33314/0 (socket says 172.21.15.45:33314) 2023-10-06T05:20:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675aa1b700 1 -- 172.21.15.45:0/3202949845 learned_addr learned my addr 172.21.15.45:0/3202949845 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f6759a19700 1 --2- 172.21.15.45:0/3202949845 >> v2:172.21.15.144:3300/0 conn(0x7f67540b1cb0 0x7f675406a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675aa1b700 1 -- 172.21.15.45:0/3202949845 >> v2:172.21.15.144:3300/0 conn(0x7f67540b1cb0 msgr2=0x7f675406a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675aa1b700 1 --2- 172.21.15.45:0/3202949845 >> v2:172.21.15.144:3300/0 conn(0x7f67540b1cb0 0x7f675406a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675aa1b700 1 -- 172.21.15.45:0/3202949845 >> v1:172.21.15.45:6789/0 conn(0x7f67540b18e0 legacy=0x7f675415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f675aa1b700 1 -- 172.21.15.45:0/3202949845 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754067b40 con 0x7f675406ad60 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.729+0000 7f6759218700 1 -- 172.21.15.45:0/3202949845 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155213959 0 0) 0x7f6750001110 con 0x7f67540b18e0 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675aa1b700 1 --2- 172.21.15.45:0/3202949845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675406ad60 0x7f67540ab9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f67540c8db0 tx=0x7f674c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11da71b4f154d149 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f6759218700 1 -- 172.21.15.45:0/3202949845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00e070 con 0x7f675406ad60 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f6759218700 1 -- 172.21.15.45:0/3202949845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c00ba50 con 0x7f675406ad60 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f6759218700 1 -- 172.21.15.45:0/3202949845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0092e0 con 0x7f675406ad60 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 -- 172.21.15.45:0/3202949845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675406ad60 msgr2=0x7f67540ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 --2- 172.21.15.45:0/3202949845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675406ad60 0x7f67540ab9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f67540c8db0 tx=0x7f674c009840 comp rx=0 tx=0).stop 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 -- 172.21.15.45:0/3202949845 shutdown_connections 2023-10-06T05:20:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 --2- 172.21.15.45:0/3202949845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675406ad60 0x7f67540ab9d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 --2- 172.21.15.45:0/3202949845 >> v2:172.21.15.144:3300/0 conn(0x7f67540b1cb0 0x7f675406a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 -- 172.21.15.45:0/3202949845 >> 172.21.15.45:0/3202949845 conn(0x7f67540cc570 msgr2=0x7f67540cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 -- 172.21.15.45:0/3202949845 shutdown_connections 2023-10-06T05:20:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.730+0000 7f675c47e700 1 -- 172.21.15.45:0/3202949845 wait complete. 2023-10-06T05:20:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.732+0000 7f675c47e700 1 Processor -- start 2023-10-06T05:20:44.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.732+0000 7f675c47e700 1 -- start start 2023-10-06T05:20:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675c47e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f675406ad60 0x7f67540c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675c47e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67540b18e0 0x7f67540c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675c47e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67540b1cb0 0x7f67541565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675c47e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c52f0 con 0x7f67540b18e0 2023-10-06T05:20:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675c47e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c0360 con 0x7f67540b1cb0 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675c47e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c0640 con 0x7f675406ad60 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f6759a19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67540b18e0 0x7f67540c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f6759a19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67540b18e0 0x7f67540c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36096/0 (socket says 172.21.15.45:36096) 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f6759a19700 1 -- 172.21.15.45:0/1915735580 learned_addr learned my addr 172.21.15.45:0/1915735580 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675aa1b700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67540b1cb0 0x7f67541565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.733+0000 7f675a21a700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f675406ad60 0x7f67540c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f6759a19700 1 -- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f675406ad60 msgr2=0x7f67540c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f6759a19700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f675406ad60 0x7f67540c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f6759a19700 1 -- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67540b1cb0 msgr2=0x7f67541565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f6759a19700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67540b1cb0 0x7f67541565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f6759a19700 1 -- 172.21.15.45:0/1915735580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c009ae0 con 0x7f67540b18e0 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f6759a19700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67540b18e0 0x7f67540c5560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f674400d860 tx=0x7f6744008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f674a7fc700 1 -- 172.21.15.45:0/1915735580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67440125f0 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f674a7fc700 1 -- 172.21.15.45:0/1915735580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6744012750 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67540c0920 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f674a7fc700 1 -- 172.21.15.45:0/1915735580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744012af0 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.734+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67540c2d80 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.735+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675406e300 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.736+0000 7f674a7fc700 1 -- 172.21.15.45:0/1915735580 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6744013480 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.736+0000 7f674a7fc700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6738066f30 0x7f67380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.736+0000 7f674a7fc700 1 -- 172.21.15.45:0/1915735580 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f6744084900 con 0x7f67540b18e0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.736+0000 7f675a21a700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6738066f30 0x7f67380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.737+0000 7f675a21a700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6738066f30 0x7f67380693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6750007fd0 tx=0x7f6750007b90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.741+0000 7f674a7fc700 1 -- 172.21.15.45:0/1915735580 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67440555b0 con 0x7f67540b18e0 2023-10-06T05:20:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.942+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) v1 -- 0x7f67540ca5b0 con 0x7f67540b18e0 2023-10-06T05:20:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.942+0000 7f674a7fc700 1 -- 172.21.15.45:0/1915735580 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6744055140 con 0x7f67540b18e0 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6738066f30 msgr2=0x7f67380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6738066f30 0x7f67380693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6750007fd0 tx=0x7f6750007b90 comp rx=0 tx=0).stop 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67540b18e0 msgr2=0x7f67540c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67540b18e0 0x7f67540c5560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f674400d860 tx=0x7f6744008040 comp rx=0 tx=0).stop 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 shutdown_connections 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6738066f30 0x7f67380693e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67540b1cb0 0x7f67541565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f675406ad60 0x7f67540c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 --2- 172.21.15.45:0/1915735580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67540b18e0 0x7f67540c5560 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.944+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 >> 172.21.15.45:0/1915735580 conn(0x7f67540cc570 msgr2=0x7f6754154590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.945+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 shutdown_connections 2023-10-06T05:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:44.945+0000 7f675c47e700 1 -- 172.21.15.45:0/1915735580 wait complete. 2023-10-06T05:20:44.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2023-10-06T05:20:45.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.091+0000 7f72871dd700 1 Processor -- start 2023-10-06T05:20:45.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.091+0000 7f72871dd700 1 -- start start 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.092+0000 7f72871dd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7280159200 0x7f72800c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.092+0000 7f72871dd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f72801595d0 0x7f728006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f72871dd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800c7d00 con 0x7f728006b5c0 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f72871dd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72800c7780 con 0x7f7280159200 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f72871dd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72800c86d0 con 0x7f72801595d0 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7280159200 0x7f72800c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7280159200 0x7f72800c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36100/0 (socket says 172.21.15.45:36100) 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 -- 172.21.15.45:0/402696624 learned_addr learned my addr 172.21.15.45:0/402696624 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:45.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f727ffff700 1 --2- 172.21.15.45:0/402696624 >> v2:172.21.15.144:3300/0 conn(0x7f72801595d0 0x7f728006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 -- 172.21.15.45:0/402696624 >> v2:172.21.15.144:3300/0 conn(0x7f72801595d0 msgr2=0x7f728006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 --2- 172.21.15.45:0/402696624 >> v2:172.21.15.144:3300/0 conn(0x7f72801595d0 0x7f728006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 -- 172.21.15.45:0/402696624 >> v1:172.21.15.144:6789/0 conn(0x7f728006b5c0 legacy=0x7f72800ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 -- 172.21.15.45:0/402696624 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728006bb40 con 0x7f7280159200 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f7284f79700 1 --2- 172.21.15.45:0/402696624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7280159200 0x7f72800c8190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f728006e790 tx=0x7f7270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bce5d07c3426ae6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.093+0000 7f727f7fe700 1 -- 172.21.15.45:0/402696624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270009e70 con 0x7f7280159200 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f727f7fe700 1 -- 172.21.15.45:0/402696624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727000b860 con 0x7f7280159200 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f727f7fe700 1 -- 172.21.15.45:0/402696624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727000bc00 con 0x7f7280159200 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 -- 172.21.15.45:0/402696624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7280159200 msgr2=0x7f72800c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 --2- 172.21.15.45:0/402696624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7280159200 0x7f72800c8190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f728006e790 tx=0x7f7270009250 comp rx=0 tx=0).stop 2023-10-06T05:20:45.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 -- 172.21.15.45:0/402696624 shutdown_connections 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 --2- 172.21.15.45:0/402696624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7280159200 0x7f72800c8190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 --2- 172.21.15.45:0/402696624 >> v2:172.21.15.144:3300/0 conn(0x7f72801595d0 0x7f728006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 -- 172.21.15.45:0/402696624 >> 172.21.15.45:0/402696624 conn(0x7f728006f090 msgr2=0x7f728006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 -- 172.21.15.45:0/402696624 shutdown_connections 2023-10-06T05:20:45.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.094+0000 7f72871dd700 1 -- 172.21.15.45:0/402696624 wait complete. 2023-10-06T05:20:45.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.095+0000 7f72871dd700 1 Processor -- start 2023-10-06T05:20:45.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f72871dd700 1 -- start start 2023-10-06T05:20:45.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f72871dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728006b5c0 0x7f7280143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f72871dd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7280159200 0x7f72800afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f72871dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72801595d0 0x7f72800c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f72871dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800c4d90 con 0x7f72801595d0 2023-10-06T05:20:45.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f72871dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800b11b0 con 0x7f728006b5c0 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f72871dd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72800b05a0 con 0x7f7280159200 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f728577a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72801595d0 0x7f72800c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f7284f79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728006b5c0 0x7f7280143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f7284f79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728006b5c0 0x7f7280143bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:56090/0 (socket says 172.21.15.45:56090) 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.096+0000 7f728577a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72801595d0 0x7f72800c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36114/0 (socket says 172.21.15.45:36114) 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f7284f79700 1 -- 172.21.15.45:0/3797435996 learned_addr learned my addr 172.21.15.45:0/3797435996 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f727ffff700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7280159200 0x7f72800afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f728577a700 1 -- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7280159200 msgr2=0x7f72800afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f728577a700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7280159200 0x7f72800afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f728577a700 1 -- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728006b5c0 msgr2=0x7f7280143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f728577a700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728006b5c0 0x7f7280143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f728577a700 1 -- 172.21.15.45:0/3797435996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72700094f0 con 0x7f72801595d0 2023-10-06T05:20:45.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f728577a700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72801595d0 0x7f72800c29e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f727800c830 tx=0x7f7278001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f727cff9700 1 -- 172.21.15.45:0/3797435996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278012630 con 0x7f72801595d0 2023-10-06T05:20:45.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f727cff9700 1 -- 172.21.15.45:0/3797435996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278012790 con 0x7f72801595d0 2023-10-06T05:20:45.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72800b0880 con 0x7f72801595d0 2023-10-06T05:20:45.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f727cff9700 1 -- 172.21.15.45:0/3797435996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278012b50 con 0x7f72801595d0 2023-10-06T05:20:45.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.097+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72800b0e40 con 0x7f72801595d0 2023-10-06T05:20:45.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.098+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728006e2f0 con 0x7f72801595d0 2023-10-06T05:20:45.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.099+0000 7f727cff9700 1 -- 172.21.15.45:0/3797435996 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f727801a3a0 con 0x7f72801595d0 2023-10-06T05:20:45.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.099+0000 7f727cff9700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7260066f30 0x7f72600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.099+0000 7f727cff9700 1 -- 172.21.15.45:0/3797435996 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f7278084750 con 0x7f72801595d0 2023-10-06T05:20:45.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.100+0000 7f7284f79700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7260066f30 0x7f72600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.100+0000 7f7284f79700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7260066f30 0x7f72600693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f728016d700 tx=0x7f7270017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.104+0000 7f727cff9700 1 -- 172.21.15.45:0/3797435996 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72780518d0 con 0x7f72801595d0 2023-10-06T05:20:45.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:45.385+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) v1 -- 0x7f728006be20 con 0x7f72801595d0 2023-10-06T05:20:46.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.199+0000 7f727cff9700 1 -- 172.21.15.45:0/3797435996 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v272) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7278054ef0 con 0x7f72801595d0 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.201+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7260066f30 msgr2=0x7f72600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.201+0000 7f72871dd700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7260066f30 0x7f72600693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f728016d700 tx=0x7f7270017000 comp rx=0 tx=0).stop 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.201+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72801595d0 msgr2=0x7f72800c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.201+0000 7f72871dd700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72801595d0 0x7f72800c29e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f727800c830 tx=0x7f7278001690 comp rx=0 tx=0).stop 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.201+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 shutdown_connections 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.202+0000 7f72871dd700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7260066f30 0x7f72600693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.202+0000 7f72871dd700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72801595d0 0x7f72800c29e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.202+0000 7f72871dd700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728006b5c0 0x7f7280143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.202+0000 7f72871dd700 1 --2- 172.21.15.45:0/3797435996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7280159200 0x7f72800afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.202+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 >> 172.21.15.45:0/3797435996 conn(0x7f728006f090 msgr2=0x7f72801599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:46.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.202+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 shutdown_connections 2023-10-06T05:20:46.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.202+0000 7f72871dd700 1 -- 172.21.15.45:0/3797435996 wait complete. 2023-10-06T05:20:46.219 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2023-10-06T05:20:46.219 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2023-10-06T05:20:46.219 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2023-10-06T05:20:46.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2023-10-06T05:20:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.354+0000 7f3c5e185700 1 Processor -- start 2023-10-06T05:20:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.354+0000 7f3c5e185700 1 -- start start 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c5e185700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c580692d0 0x7f3c58159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c5e185700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c580ca6d0 con 0x7f3c5815d040 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c5e185700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5815d550 con 0x7f3c58159170 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c5e185700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5815d810 con 0x7f3c580692d0 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c56ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c580692d0 0x7f3c58159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c56ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c580692d0 0x7f3c58159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52956/0 (socket says 172.21.15.45:52956) 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c57fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c5815d040 0x7f3c580aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53750/0 (socket says 172.21.15.45:53750) 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c56ffd700 1 -- 172.21.15.45:0/97959684 learned_addr learned my addr 172.21.15.45:0/97959684 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c56ffd700 1 -- 172.21.15.45:0/97959684 >> v1:172.21.15.144:6789/0 conn(0x7f3c58159170 legacy=0x7f3c58068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c56ffd700 1 -- 172.21.15.45:0/97959684 >> v1:172.21.15.45:6789/0 conn(0x7f3c5815d040 legacy=0x7f3c580aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c56ffd700 1 -- 172.21.15.45:0/97959684 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5815daf0 con 0x7f3c580692d0 2023-10-06T05:20:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c567fc700 1 -- 172.21.15.45:0/97959684 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405640374 0 0) 0x7f3c580ca6d0 con 0x7f3c5815d040 2023-10-06T05:20:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.355+0000 7f3c56ffd700 1 --2- 172.21.15.45:0/97959684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c58159540 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0099b0 tx=0x7f3c4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05886baa68c19ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c567fc700 1 -- 172.21.15.45:0/97959684 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00b860 con 0x7f3c580692d0 2023-10-06T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c567fc700 1 -- 172.21.15.45:0/97959684 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c00b9c0 con 0x7f3c580692d0 2023-10-06T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c567fc700 1 -- 172.21.15.45:0/97959684 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00bd60 con 0x7f3c580692d0 2023-10-06T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c5e185700 1 -- 172.21.15.45:0/97959684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 msgr2=0x7f3c58159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c5e185700 1 --2- 172.21.15.45:0/97959684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c58159540 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0099b0 tx=0x7f3c4c009250 comp rx=0 tx=0).stop 2023-10-06T05:20:46.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c5e185700 1 -- 172.21.15.45:0/97959684 shutdown_connections 2023-10-06T05:20:46.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c5e185700 1 --2- 172.21.15.45:0/97959684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c58159540 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c5e185700 1 -- 172.21.15.45:0/97959684 >> 172.21.15.45:0/97959684 conn(0x7f3c5806f7c0 msgr2=0x7f3c580d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:46.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c5e185700 1 -- 172.21.15.45:0/97959684 shutdown_connections 2023-10-06T05:20:46.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.356+0000 7f3c5e185700 1 -- 172.21.15.45:0/97959684 wait complete. 2023-10-06T05:20:46.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 Processor -- start 2023-10-06T05:20:46.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 -- start start 2023-10-06T05:20:46.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c5815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58159170 0x7f3c581575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5815d040 0x7f3c58159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580a6df0 con 0x7f3c580692d0 2023-10-06T05:20:46.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5815a390 con 0x7f3c5815d040 2023-10-06T05:20:46.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c5e185700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5815a670 con 0x7f3c58159170 2023-10-06T05:20:46.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.358+0000 7f3c577fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c5815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c5815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52958/0 (socket says 172.21.15.45:52958) 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 -- 172.21.15.45:0/2457628897 learned_addr learned my addr 172.21.15.45:0/2457628897 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c57fff700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5815d040 0x7f3c58159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c56ffd700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58159170 0x7f3c581575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 -- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58159170 msgr2=0x7f3c581575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58159170 0x7f3c581575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 -- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5815d040 msgr2=0x7f3c58159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5815d040 0x7f3c58159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 -- 172.21.15.45:0/2457628897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c0094f0 con 0x7f3c580692d0 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c57fff700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5815d040 0x7f3c58159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c56ffd700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58159170 0x7f3c581575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:46.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c577fe700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c5815b090 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800c590 tx=0x7f3c4800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c3f7fe700 1 -- 172.21.15.45:0/2457628897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48011590 con 0x7f3c580692d0 2023-10-06T05:20:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c3f7fe700 1 -- 172.21.15.45:0/2457628897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c480116f0 con 0x7f3c580692d0 2023-10-06T05:20:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5815a950 con 0x7f3c580692d0 2023-10-06T05:20:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c3f7fe700 1 -- 172.21.15.45:0/2457628897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48011ab0 con 0x7f3c580692d0 2023-10-06T05:20:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.359+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c58157b20 con 0x7f3c580692d0 2023-10-06T05:20:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.360+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c58063b50 con 0x7f3c580692d0 2023-10-06T05:20:46.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.361+0000 7f3c3f7fe700 1 -- 172.21.15.45:0/2457628897 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c4801b3b0 con 0x7f3c580692d0 2023-10-06T05:20:46.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.362+0000 7f3c3f7fe700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.362+0000 7f3c3f7fe700 1 -- 172.21.15.45:0/2457628897 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f3c48083f50 con 0x7f3c580692d0 2023-10-06T05:20:46.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.362+0000 7f3c56ffd700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.363+0000 7f3c56ffd700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c44066f30 0x7f3c440693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0099b0 tx=0x7f3c4c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.366+0000 7f3c3f7fe700 1 -- 172.21.15.45:0/2457628897 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c480511f0 con 0x7f3c580692d0 2023-10-06T05:20:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:46.647+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) v1 -- 0x7f3c5815ddd0 con 0x7f3c580692d0 2023-10-06T05:20:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.204+0000 7f3c3f7fe700 1 -- 172.21.15.45:0/2457628897 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v273) v1 ==== 144+0+0 (secure 0 0 0) 0x7f3c48017020 con 0x7f3c580692d0 2023-10-06T05:20:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c44066f30 msgr2=0x7f3c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c44066f30 0x7f3c440693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0099b0 tx=0x7f3c4c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 msgr2=0x7f3c5815b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c5815b090 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800c590 tx=0x7f3c4800c950 comp rx=0 tx=0).stop 2023-10-06T05:20:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 shutdown_connections 2023-10-06T05:20:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5815d040 0x7f3c58159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c580692d0 0x7f3c5815b090 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 --2- 172.21.15.45:0/2457628897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58159170 0x7f3c581575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 >> 172.21.15.45:0/2457628897 conn(0x7f3c5806f7c0 msgr2=0x7f3c580ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 shutdown_connections 2023-10-06T05:20:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.207+0000 7f3c5e185700 1 -- 172.21.15.45:0/2457628897 wait complete. 2023-10-06T05:20:47.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:20:47.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.364+0000 7f042d06c700 1 Processor -- start 2023-10-06T05:20:47.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.364+0000 7f042d06c700 1 -- start start 2023-10-06T05:20:47.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.365+0000 7f042d06c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042806c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.365+0000 7f042d06c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f042806cd60 0x7f04280acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.365+0000 7f042d06c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04280c9940 con 0x7f04280ad4e0 2023-10-06T05:20:47.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.365+0000 7f042d06c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042806d230 con 0x7f042806cd60 2023-10-06T05:20:47.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.365+0000 7f042d06c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042806d510 con 0x7f04281406e0 2023-10-06T05:20:47.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042806c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:45578/0 (socket says 172.21.15.45:45578) 2023-10-06T05:20:47.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f042659c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f042806cd60 0x7f04280acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 -- 172.21.15.45:0/2631000854 learned_addr learned my addr 172.21.15.45:0/2631000854 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:47.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 -- 172.21.15.45:0/2631000854 >> v2:172.21.15.144:3300/0 conn(0x7f042806cd60 msgr2=0x7f04280acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:47.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 --2- 172.21.15.45:0/2631000854 >> v2:172.21.15.144:3300/0 conn(0x7f042806cd60 0x7f04280acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 -- 172.21.15.45:0/2631000854 >> v1:172.21.15.144:6789/0 conn(0x7f04280ad4e0 legacy=0x7f04280c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 -- 172.21.15.45:0/2631000854 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042806d7f0 con 0x7f04281406e0 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0426d9d700 1 --2- 172.21.15.45:0/2631000854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042806c820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f04180099b0 tx=0x7f0418009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22eee163d663b864 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0425d9b700 1 -- 172.21.15.45:0/2631000854 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418009ea0 con 0x7f04281406e0 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.366+0000 7f0425d9b700 1 -- 172.21.15.45:0/2631000854 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041800b860 con 0x7f04281406e0 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f0425d9b700 1 -- 172.21.15.45:0/2631000854 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800bc20 con 0x7f04281406e0 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 -- 172.21.15.45:0/2631000854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 msgr2=0x7f042806c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 --2- 172.21.15.45:0/2631000854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042806c820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f04180099b0 tx=0x7f0418009250 comp rx=0 tx=0).stop 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 -- 172.21.15.45:0/2631000854 shutdown_connections 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 --2- 172.21.15.45:0/2631000854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042806c820 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 --2- 172.21.15.45:0/2631000854 >> v2:172.21.15.144:3300/0 conn(0x7f042806cd60 0x7f04280acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 -- 172.21.15.45:0/2631000854 >> 172.21.15.45:0/2631000854 conn(0x7f04280691c0 msgr2=0x7f04280cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 -- 172.21.15.45:0/2631000854 shutdown_connections 2023-10-06T05:20:47.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.367+0000 7f042d06c700 1 -- 172.21.15.45:0/2631000854 wait complete. 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.368+0000 7f042d06c700 1 Processor -- start 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042d06c700 1 -- start start 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042d06c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f042806cd60 0x7f0428156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042d06c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04280ad4e0 0x7f0428158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042d06c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042d06c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042815c0e0 con 0x7f042806cd60 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042d06c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0428156aa0 con 0x7f04280ad4e0 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042d06c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042815b230 con 0x7f04281406e0 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f0426d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f042806cd60 0x7f0428156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f0426d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f042806cd60 0x7f0428156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52972/0 (socket says 172.21.15.45:52972) 2023-10-06T05:20:47.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f0426d9d700 1 -- 172.21.15.45:0/3539048217 learned_addr learned my addr 172.21.15.45:0/3539048217 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042759e700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.369+0000 7f042659c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04280ad4e0 0x7f0428158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f0426d9d700 1 -- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 msgr2=0x7f042815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f0426d9d700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f0426d9d700 1 -- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04280ad4e0 msgr2=0x7f0428158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f0426d9d700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04280ad4e0 0x7f0428158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f0426d9d700 1 -- 172.21.15.45:0/3539048217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180094f0 con 0x7f042806cd60 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f0426d9d700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f042806cd60 0x7f0428156560 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0418006fd0 tx=0x7f0418017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f041effd700 1 -- 172.21.15.45:0/3539048217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418020070 con 0x7f042806cd60 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f041effd700 1 -- 172.21.15.45:0/3539048217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0418000bc0 con 0x7f042806cd60 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042815b4d0 con 0x7f042806cd60 2023-10-06T05:20:47.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f041effd700 1 -- 172.21.15.45:0/3539048217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800bc20 con 0x7f042806cd60 2023-10-06T05:20:47.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.370+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042815ba90 con 0x7f042806cd60 2023-10-06T05:20:47.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.371+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042806e2d0 con 0x7f042806cd60 2023-10-06T05:20:47.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.372+0000 7f041effd700 1 -- 172.21.15.45:0/3539048217 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0418024070 con 0x7f042806cd60 2023-10-06T05:20:47.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.373+0000 7f041effd700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0408066f30 0x7f04080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.373+0000 7f041effd700 1 -- 172.21.15.45:0/3539048217 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f0418085b80 con 0x7f042806cd60 2023-10-06T05:20:47.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.373+0000 7f042659c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0408066f30 0x7f04080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.374+0000 7f042659c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0408066f30 0x7f04080693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f04280c6080 tx=0x7f04100091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.377+0000 7f041effd700 1 -- 172.21.15.45:0/3539048217 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0418052e00 con 0x7f042806cd60 2023-10-06T05:20:47.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.578+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f042804bb40 con 0x7f042806cd60 2023-10-06T05:20:47.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.583+0000 7f041effd700 1 -- 172.21.15.45:0/3539048217 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f04180150c0 con 0x7f042806cd60 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0408066f30 msgr2=0x7f04080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0408066f30 0x7f04080693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f04280c6080 tx=0x7f04100091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f042806cd60 msgr2=0x7f0428156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f042806cd60 0x7f0428156560 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0418006fd0 tx=0x7f0418017000 comp rx=0 tx=0).stop 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 shutdown_connections 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0408066f30 0x7f04080693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04281406e0 0x7f042815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f042806cd60 0x7f0428156560 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 --2- 172.21.15.45:0/3539048217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04280ad4e0 0x7f0428158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 >> 172.21.15.45:0/3539048217 conn(0x7f04280691c0 msgr2=0x7f042813f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 shutdown_connections 2023-10-06T05:20:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.585+0000 7f042d06c700 1 -- 172.21.15.45:0/3539048217 wait complete. 2023-10-06T05:20:47.600 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:20:47.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:47.646 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:47.698 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:20:47.698 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:20:47.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.833+0000 7f4ca8b3f700 1 Processor -- start 2023-10-06T05:20:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.833+0000 7f4ca8b3f700 1 -- start start 2023-10-06T05:20:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca8b3f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ca4158f60 0x7f4ca4159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca8b3f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca406a060 0x7f4ca40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca8b3f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca4069bd0 con 0x7f4ca40cc390 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca8b3f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca406a4d0 con 0x7f4ca4158f60 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca8b3f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca4068fc0 con 0x7f4ca406a060 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ca4158f60 0x7f4ca4159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca2d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4ca40cc390 0x7f4ca40c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53758/0 (socket says 172.21.15.45:53758) 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca2d9d700 1 -- 172.21.15.45:0/954935068 learned_addr learned my addr 172.21.15.45:0/954935068 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca1d9b700 1 --2- 172.21.15.45:0/954935068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca406a060 0x7f4ca40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.834+0000 7f4ca259c700 1 -- 172.21.15.45:0/954935068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca406a060 msgr2=0x7f4ca40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca259c700 1 --2- 172.21.15.45:0/954935068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca406a060 0x7f4ca40cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca259c700 1 -- 172.21.15.45:0/954935068 >> v1:172.21.15.45:6789/0 conn(0x7f4ca40cc390 legacy=0x7f4ca40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca159a700 1 -- 172.21.15.45:0/954935068 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714224913 0 0) 0x7f4ca4069bd0 con 0x7f4ca40cc390 2023-10-06T05:20:47.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca259c700 1 -- 172.21.15.45:0/954935068 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca40692a0 con 0x7f4ca4158f60 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca259c700 1 --2- 172.21.15.45:0/954935068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca4158f60 0x7f4ca4159330 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4c980098e0 tx=0x7f4c980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62d7baeef7315a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca159a700 1 -- 172.21.15.45:0/954935068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98009e80 con 0x7f4ca4158f60 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca159a700 1 -- 172.21.15.45:0/954935068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9800b860 con 0x7f4ca4158f60 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca159a700 1 -- 172.21.15.45:0/954935068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9800bbe0 con 0x7f4ca4158f60 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/954935068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca4158f60 msgr2=0x7f4ca4159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.835+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/954935068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca4158f60 0x7f4ca4159330 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4c980098e0 tx=0x7f4c980091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.836+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/954935068 shutdown_connections 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.836+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/954935068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca4158f60 0x7f4ca4159330 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.836+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/954935068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca406a060 0x7f4ca40cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.836+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/954935068 >> 172.21.15.45:0/954935068 conn(0x7f4ca40cdf50 msgr2=0x7f4ca40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.836+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/954935068 shutdown_connections 2023-10-06T05:20:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.836+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/954935068 wait complete. 2023-10-06T05:20:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.837+0000 7f4ca8b3f700 1 Processor -- start 2023-10-06T05:20:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.837+0000 7f4ca8b3f700 1 -- start start 2023-10-06T05:20:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca8b3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca406a060 0x7f4ca415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca8b3f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca40cc390 0x7f4ca416a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca8b3f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca4158f60 0x7f4ca41696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca8b3f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca40c6e30 con 0x7f4ca406a060 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca8b3f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca40c5130 con 0x7f4ca40cc390 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca8b3f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca4169d70 con 0x7f4ca4158f60 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca406a060 0x7f4ca415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca406a060 0x7f4ca415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52984/0 (socket says 172.21.15.45:52984) 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 -- 172.21.15.45:0/3000885346 learned_addr learned my addr 172.21.15.45:0/3000885346 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca2d9d700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca4158f60 0x7f4ca41696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 -- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca4158f60 msgr2=0x7f4ca41696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca4158f60 0x7f4ca41696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca1d9b700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca40cc390 0x7f4ca416a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 -- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca40cc390 msgr2=0x7f4ca416a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca40cc390 0x7f4ca416a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 -- 172.21.15.45:0/3000885346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98009420 con 0x7f4ca406a060 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca2d9d700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca4158f60 0x7f4ca41696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca1d9b700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca40cc390 0x7f4ca416a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca259c700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca406a060 0x7f4ca415e590 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40c4f80 tx=0x7f4c98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4c927fc700 1 -- 172.21.15.45:0/3000885346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9800b9d0 con 0x7f4ca406a060 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4c927fc700 1 -- 172.21.15.45:0/3000885346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9800bb30 con 0x7f4ca406a060 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca416a050 con 0x7f4ca406a060 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.838+0000 7f4c927fc700 1 -- 172.21.15.45:0/3000885346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98025670 con 0x7f4ca406a060 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.839+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca416a610 con 0x7f4ca406a060 2023-10-06T05:20:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.839+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca406f410 con 0x7f4ca406a060 2023-10-06T05:20:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.840+0000 7f4c927fc700 1 -- 172.21.15.45:0/3000885346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c98023070 con 0x7f4ca406a060 2023-10-06T05:20:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.840+0000 7f4c927fc700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c84066f30 0x7f4c840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.840+0000 7f4c927fc700 1 -- 172.21.15.45:0/3000885346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f4c98085b10 con 0x7f4ca406a060 2023-10-06T05:20:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.841+0000 7f4ca1d9b700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c84066f30 0x7f4c840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.841+0000 7f4ca1d9b700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c84066f30 0x7f4c840693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40a8fc0 tx=0x7f4c94008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:47.845+0000 7f4c927fc700 1 -- 172.21.15.45:0/3000885346 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c98052d30 con 0x7f4ca406a060 2023-10-06T05:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.091+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ca4069580 con 0x7f4ca406a060 2023-10-06T05:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.092+0000 7f4c927fc700 1 -- 172.21.15.45:0/3000885346 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f4c98015de0 con 0x7f4ca406a060 2023-10-06T05:20:48.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:48.094 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3726926923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3726926923},{"type":"v1","addr":"172.21.15.45:6835","nonce":3726926923}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1844545629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1844545629},{"type":"v1","addr":"172.21.15.144:6836","nonce":1844545629}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/902867469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":902867469},{"type":"v1","addr":"172.21.15.144:6837","nonce":902867469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1230704519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1230704519},{"type":"v1","addr":"172.21.15.45:6833","nonce":1230704519}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[]} 2023-10-06T05:20:48.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c84066f30 msgr2=0x7f4c840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c84066f30 0x7f4c840693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40a8fc0 tx=0x7f4c94008040 comp rx=0 tx=0).stop 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca406a060 msgr2=0x7f4ca415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca406a060 0x7f4ca415e590 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40c4f80 tx=0x7f4c98017000 comp rx=0 tx=0).stop 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 shutdown_connections 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c84066f30 0x7f4c840693e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ca4158f60 0x7f4ca41696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca406a060 0x7f4ca415e590 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 --2- 172.21.15.45:0/3000885346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca40cc390 0x7f4ca416a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 >> 172.21.15.45:0/3000885346 conn(0x7f4ca40cdf50 msgr2=0x7f4ca40b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 shutdown_connections 2023-10-06T05:20:48.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.094+0000 7f4ca8b3f700 1 -- 172.21.15.45:0/3000885346 wait complete. 2023-10-06T05:20:48.098 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 64 2023-10-06T05:20:48.109 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:20:48.110 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:20:48.110 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:20:48.110 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:48.110 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:20:48.110 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:20:48.111 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:20:48.111 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:48.111 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:20:48.111 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:20:48.111 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:20:48.111 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:48.111 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:20:48.111 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:20:48.112 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:20:48.112 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:48.119 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:20:48.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:20:48.120 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:20:48.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:20:48.122 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:20:48.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:20:48.123 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:20:48.123 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:20:48.162 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:20:48.163 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:48.164 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:20:48.165 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:48.166 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:20:48.166 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:48.167 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:20:48.167 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:48.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:20:48.250 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:20:48.250 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:20:48.253 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:20:48.253 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:20:48.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.309+0000 7f4c4ebd9700 1 Processor -- start 2023-10-06T05:20:48.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.309+0000 7f4c4ebd9700 1 -- start start 2023-10-06T05:20:48.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c4ebd9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4c481595d0 0x7f4c4806b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c4ebd9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480c7d00 con 0x7f4c48159200 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c4ebd9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4806bb40 con 0x7f4c4806b5c0 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c4ebd9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c480c7780 con 0x7f4c481595d0 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c4c975700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4c48159200 0x7f4c480c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53782/0 (socket says 172.21.15.45:53782) 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c4c975700 1 -- 172.21.15.45:0/3356043642 learned_addr learned my addr 172.21.15.45:0/3356043642 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c47fff700 1 --2- 172.21.15.45:0/3356043642 >> v2:172.21.15.144:3300/0 conn(0x7f4c481595d0 0x7f4c4806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c477fe700 1 -- 172.21.15.45:0/3356043642 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021141849 0 0) 0x7f4c480c7d00 con 0x7f4c48159200 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c477fe700 1 -- 172.21.15.45:0/3356043642 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c38003570 con 0x7f4c48159200 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c477fe700 1 -- 172.21.15.45:0/3356043642 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722204622 0 0) 0x7f4c4806bb40 con 0x7f4c4806b5c0 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.310+0000 7f4c477fe700 1 -- 172.21.15.45:0/3356043642 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c480c7d00 con 0x7f4c4806b5c0 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.311+0000 7f4c47fff700 1 -- 172.21.15.45:0/3356043642 >> v1:172.21.15.144:6789/0 conn(0x7f4c4806b5c0 legacy=0x7f4c480ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.311+0000 7f4c47fff700 1 -- 172.21.15.45:0/3356043642 >> v1:172.21.15.45:6789/0 conn(0x7f4c48159200 legacy=0x7f4c480c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.311+0000 7f4c47fff700 1 -- 172.21.15.45:0/3356043642 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4806be20 con 0x7f4c481595d0 2023-10-06T05:20:48.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.311+0000 7f4c47fff700 1 --2- 172.21.15.45:0/3356043642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 0x7f4c4806b080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4c340098e0 tx=0x7f4c340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc3e0e94cba5568 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.311+0000 7f4c477fe700 1 -- 172.21.15.45:0/3356043642 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34011630 con 0x7f4c481595d0 2023-10-06T05:20:48.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.311+0000 7f4c477fe700 1 -- 172.21.15.45:0/3356043642 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c34011790 con 0x7f4c481595d0 2023-10-06T05:20:48.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.311+0000 7f4c477fe700 1 -- 172.21.15.45:0/3356043642 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34011b30 con 0x7f4c481595d0 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.312+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/3356043642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 msgr2=0x7f4c4806b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.312+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/3356043642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 0x7f4c4806b080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4c340098e0 tx=0x7f4c340091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.312+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/3356043642 shutdown_connections 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.312+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/3356043642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 0x7f4c4806b080 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.312+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/3356043642 >> 172.21.15.45:0/3356043642 conn(0x7f4c4806f090 msgr2=0x7f4c4806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.312+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/3356043642 shutdown_connections 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.312+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/3356043642 wait complete. 2023-10-06T05:20:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.313+0000 7f4c4ebd9700 1 Processor -- start 2023-10-06T05:20:48.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.313+0000 7f4c4ebd9700 1 -- start start 2023-10-06T05:20:48.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4ebd9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4806b5c0 0x7f4c480c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4ebd9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c48159200 0x7f4c4814df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4ebd9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 0x7f4c4814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4ebd9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48169e80 con 0x7f4c4806b5c0 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4ebd9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4816a120 con 0x7f4c481595d0 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4ebd9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4816a400 con 0x7f4c48159200 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4806b5c0 0x7f4c480c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4806b5c0 0x7f4c480c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53018/0 (socket says 172.21.15.45:53018) 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 -- 172.21.15.45:0/2367266157 learned_addr learned my addr 172.21.15.45:0/2367266157 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4d176700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 0x7f4c4814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c47fff700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c48159200 0x7f4c4814df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 -- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c48159200 msgr2=0x7f4c4814df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c48159200 0x7f4c4814df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 -- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 msgr2=0x7f4c4814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 0x7f4c4814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.314+0000 7f4c4c975700 1 -- 172.21.15.45:0/2367266157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009420 con 0x7f4c4806b5c0 2023-10-06T05:20:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.315+0000 7f4c4c975700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4806b5c0 0x7f4c480c6260 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00c790 tx=0x7f4c3c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.315+0000 7f4c44ff9700 1 -- 172.21.15.45:0/2367266157 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c005b30 con 0x7f4c4806b5c0 2023-10-06T05:20:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.315+0000 7f4c44ff9700 1 -- 172.21.15.45:0/2367266157 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c005c90 con 0x7f4c4806b5c0 2023-10-06T05:20:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.315+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4816a6e0 con 0x7f4c4806b5c0 2023-10-06T05:20:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.315+0000 7f4c44ff9700 1 -- 172.21.15.45:0/2367266157 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c007300 con 0x7f4c4806b5c0 2023-10-06T05:20:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.315+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4816abe0 con 0x7f4c4806b5c0 2023-10-06T05:20:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.316+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4816aa50 con 0x7f4c4806b5c0 2023-10-06T05:20:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.316+0000 7f4c44ff9700 1 -- 172.21.15.45:0/2367266157 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c3c01b3b0 con 0x7f4c4806b5c0 2023-10-06T05:20:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.317+0000 7f4c44ff9700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c2c066f30 0x7f4c2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.317+0000 7f4c44ff9700 1 -- 172.21.15.45:0/2367266157 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5784+0+0 (secure 0 0 0) 0x7f4c3c083a60 con 0x7f4c4806b5c0 2023-10-06T05:20:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.317+0000 7f4c47fff700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c2c066f30 0x7f4c2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.318+0000 7f4c47fff700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c2c066f30 0x7f4c2c0693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4c340093f0 tx=0x7f4c34017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.321+0000 7f4c44ff9700 1 -- 172.21.15.45:0/2367266157 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c3c050c80 con 0x7f4c4806b5c0 2023-10-06T05:20:48.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:48.539+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4c480c7990 con 0x7f4c4806b5c0 2023-10-06T05:20:49.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.217+0000 7f4c44ff9700 1 -- 172.21.15.45:0/2367266157 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4c3c017020 con 0x7f4c4806b5c0 2023-10-06T05:20:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c2c066f30 msgr2=0x7f4c2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c2c066f30 0x7f4c2c0693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4c340093f0 tx=0x7f4c34017000 comp rx=0 tx=0).stop 2023-10-06T05:20:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4806b5c0 msgr2=0x7f4c480c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4806b5c0 0x7f4c480c6260 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00c790 tx=0x7f4c3c007ea0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 shutdown_connections 2023-10-06T05:20:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c2c066f30 0x7f4c2c0693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c481595d0 0x7f4c4814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4806b5c0 0x7f4c480c6260 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 --2- 172.21.15.45:0/2367266157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c48159200 0x7f4c4814df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 >> 172.21.15.45:0/2367266157 conn(0x7f4c4806f090 msgr2=0x7f4c4815a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 shutdown_connections 2023-10-06T05:20:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.220+0000 7f4c4ebd9700 1 -- 172.21.15.45:0/2367266157 wait complete. 2023-10-06T05:20:49.223 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-10-06T05:20:49.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:20:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.375+0000 7fc58f71c700 1 Processor -- start 2023-10-06T05:20:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.375+0000 7fc58f71c700 1 -- start start 2023-10-06T05:20:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.376+0000 7fc58f71c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5881595d0 0x7fc58806b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.376+0000 7fc58f71c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5880c7d00 con 0x7fc58806b5c0 2023-10-06T05:20:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.376+0000 7fc58f71c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58806bb40 con 0x7fc588159200 2023-10-06T05:20:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.376+0000 7fc58f71c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5880c7780 con 0x7fc5881595d0 2023-10-06T05:20:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.376+0000 7fc58dcb9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc58806b5c0 0x7fc5880ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53790/0 (socket says 172.21.15.45:53790) 2023-10-06T05:20:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.376+0000 7fc58dcb9700 1 -- 172.21.15.45:0/3657089842 learned_addr learned my addr 172.21.15.45:0/3657089842 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58ccb7700 1 --2- 172.21.15.45:0/3657089842 >> v2:172.21.15.144:3300/0 conn(0x7fc5881595d0 0x7fc58806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503765984 0 0) 0x7fc5880c7d00 con 0x7fc58806b5c0 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc574003570 con 0x7fc58806b5c0 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2292231345 0 0) 0x7fc574003570 con 0x7fc58806b5c0 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 >> v2:172.21.15.144:3300/0 conn(0x7fc5881595d0 msgr2=0x7fc58806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 --2- 172.21.15.45:0/3657089842 >> v2:172.21.15.144:3300/0 conn(0x7fc5881595d0 0x7fc58806b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 >> v1:172.21.15.144:6789/0 conn(0x7fc588159200 legacy=0x7fc5880c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58806be20 con 0x7fc58806b5c0 2023-10-06T05:20:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fc5800017f0 con 0x7fc58806b5c0 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 >> v1:172.21.15.45:6789/0 conn(0x7fc58806b5c0 legacy=0x7fc5880ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc574003a60 0x7fc574003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574004810 0x7fc574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740051c0 0x7fc57400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880c7780 con 0x7fc574003a60 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57400c0d0 con 0x7fc5740051c0 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57400c290 con 0x7fc574004810 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58d4b8700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc574003a60 0x7fc574003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58ccb7700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574004810 0x7fc574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58dcb9700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740051c0 0x7fc57400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58d4b8700 1 -- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574004810 msgr2=0x7fc574004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58d4b8700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574004810 0x7fc574004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58d4b8700 1 -- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740051c0 msgr2=0x7fc57400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58d4b8700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740051c0 0x7fc57400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58dcb9700 1 -- 172.21.15.45:0/3657089842 reap_dead start 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.377+0000 7fc58d4b8700 1 -- 172.21.15.45:0/3657089842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574003570 con 0x7fc574003a60 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc58d4b8700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc574003a60 0x7fc574003e70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc5840018b0 tx=0x7fc584007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd48fc72304ab203 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584012420 con 0x7fc574003a60 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5840125a0 con 0x7fc574003a60 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc57bfff700 1 -- 172.21.15.45:0/3657089842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584012960 con 0x7fc574003a60 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc58f71c700 1 -- 172.21.15.45:0/3657089842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc574003a60 msgr2=0x7fc574003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc58f71c700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc574003a60 0x7fc574003e70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc5840018b0 tx=0x7fc584007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc58f71c700 1 -- 172.21.15.45:0/3657089842 shutdown_connections 2023-10-06T05:20:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc58f71c700 1 --2- 172.21.15.45:0/3657089842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc574003a60 0x7fc574003e70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.378+0000 7fc58f71c700 1 -- 172.21.15.45:0/3657089842 >> 172.21.15.45:0/3657089842 conn(0x7fc58806f090 msgr2=0x7fc58806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:49.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.379+0000 7fc58f71c700 1 -- 172.21.15.45:0/3657089842 shutdown_connections 2023-10-06T05:20:49.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.379+0000 7fc58f71c700 1 -- 172.21.15.45:0/3657089842 wait complete. 2023-10-06T05:20:49.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.380+0000 7fc58f71c700 1 Processor -- start 2023-10-06T05:20:49.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.380+0000 7fc58f71c700 1 -- start start 2023-10-06T05:20:49.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.381+0000 7fc58f71c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574003a60 0x7fc5880c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.381+0000 7fc58f71c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5880c2370 0x7fc58816cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.381+0000 7fc58f71c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5880c27e0 0x7fc5880b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.381+0000 7fc58f71c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881698e0 con 0x7fc5880c2370 2023-10-06T05:20:49.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.381+0000 7fc58f71c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880c4450 con 0x7fc5880c27e0 2023-10-06T05:20:49.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.381+0000 7fc58f71c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880c3e40 con 0x7fc574003a60 2023-10-06T05:20:49.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5880c2370 0x7fc58816cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5880c2370 0x7fc58816cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53042/0 (socket says 172.21.15.45:53042) 2023-10-06T05:20:49.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 -- 172.21.15.45:0/56936941 learned_addr learned my addr 172.21.15.45:0/56936941 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:49.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58dcb9700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5880c27e0 0x7fc5880b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58d4b8700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574003a60 0x7fc5880c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 -- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574003a60 msgr2=0x7fc5880c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574003a60 0x7fc5880c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 -- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5880c27e0 msgr2=0x7fc5880b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5880c27e0 0x7fc5880b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 -- 172.21.15.45:0/56936941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58816da40 con 0x7fc5880c2370 2023-10-06T05:20:49.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58ccb7700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5880c2370 0x7fc58816cfa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc57c011d70 tx=0x7fc57c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc5797fa700 1 -- 172.21.15.45:0/56936941 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c009e20 con 0x7fc5880c2370 2023-10-06T05:20:49.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc5797fa700 1 -- 172.21.15.45:0/56936941 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c00fcd0 con 0x7fc5880c2370 2023-10-06T05:20:49.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc5797fa700 1 -- 172.21.15.45:0/56936941 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c014650 con 0x7fc5880c2370 2023-10-06T05:20:49.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58816dd20 con 0x7fc5880c2370 2023-10-06T05:20:49.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.382+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5880cd380 con 0x7fc5880c2370 2023-10-06T05:20:49.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.383+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58806e2f0 con 0x7fc5880c2370 2023-10-06T05:20:49.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.384+0000 7fc5797fa700 1 -- 172.21.15.45:0/56936941 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc57c014a80 con 0x7fc5880c2370 2023-10-06T05:20:49.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.384+0000 7fc5797fa700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc56c066f30 0x7fc56c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.384+0000 7fc5797fa700 1 -- 172.21.15.45:0/56936941 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc57c018030 con 0x7fc5880c2370 2023-10-06T05:20:49.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.384+0000 7fc58d4b8700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc56c066f30 0x7fc56c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.385+0000 7fc58d4b8700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc56c066f30 0x7fc56c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc5840018b0 tx=0x7fc584015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.389+0000 7fc5797fa700 1 -- 172.21.15.45:0/56936941 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc57c052240 con 0x7fc5880c2370 2023-10-06T05:20:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.675+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc5880c7990 con 0x7fc5880c2370 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.676+0000 7fc5797fa700 1 -- 172.21.15.45:0/56936941 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc57c055860 con 0x7fc5880c2370 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw==", 2023-10-06T05:20:49.678 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w==", 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.679 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg==", 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:20:49.680 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw==", 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.681 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:20:49.682 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:20:49.682 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.682 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.682 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-10-06T05:20:49.683 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg==", 2023-10-06T05:20:49.683 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.683 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.684 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.684 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.685 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg==", 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-10-06T05:20:49.686 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w==", 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw==", 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.687 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA==", 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ==", 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw==", 2023-10-06T05:20:49.689 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA==", 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:20:49.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw==", 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw==", 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-10-06T05:20:49.692 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g==", 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w==", 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.693 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l/0VtGRAA99fxS1gieVbRro7K6P1/lw==", 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:20:49.694 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A==", 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg==", 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-06T05:20:49.696 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc56c066f30 msgr2=0x7fc56c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc56c066f30 0x7fc56c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc5840018b0 tx=0x7fc584015000 comp rx=0 tx=0).stop 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5880c2370 msgr2=0x7fc58816cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5880c2370 0x7fc58816cfa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc57c011d70 tx=0x7fc57c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 shutdown_connections 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc56c066f30 0x7fc56c0693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5880c27e0 0x7fc5880b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574003a60 0x7fc5880c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 --2- 172.21.15.45:0/56936941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5880c2370 0x7fc58816cfa0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 >> 172.21.15.45:0/56936941 conn(0x7fc58806f090 msgr2=0x7fc588153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:49.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 shutdown_connections 2023-10-06T05:20:49.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.678+0000 7fc58f71c700 1 -- 172.21.15.45:0/56936941 wait complete. 2023-10-06T05:20:49.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:49.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.830+0000 7fda0e456700 1 Processor -- start 2023-10-06T05:20:49.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.830+0000 7fda0e456700 1 -- start start 2023-10-06T05:20:49.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0e456700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda081539f0 0x7fda0815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0e456700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fda080ccfc0 0x7fda0815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0e456700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080ce690 con 0x7fda08153620 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0e456700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0815a7a0 con 0x7fda080ccfc0 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0e456700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda080ce3f0 con 0x7fda081539f0 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0c9f3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fda080ccfc0 0x7fda0815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0c9f3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fda080ccfc0 0x7fda0815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53054/0 (socket says 172.21.15.45:53054) 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0c9f3700 1 -- 172.21.15.45:0/2354147123 learned_addr learned my addr 172.21.15.45:0/2354147123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda077fe700 1 --2- 172.21.15.45:0/2354147123 >> v2:172.21.15.144:3300/0 conn(0x7fda081539f0 0x7fda0815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0c9f3700 1 -- 172.21.15.45:0/2354147123 >> v2:172.21.15.144:3300/0 conn(0x7fda081539f0 msgr2=0x7fda0815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0c9f3700 1 --2- 172.21.15.45:0/2354147123 >> v2:172.21.15.144:3300/0 conn(0x7fda081539f0 0x7fda0815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.831+0000 7fda0c9f3700 1 -- 172.21.15.45:0/2354147123 >> v1:172.21.15.144:6789/0 conn(0x7fda08153620 legacy=0x7fda080cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.832+0000 7fda0c9f3700 1 -- 172.21.15.45:0/2354147123 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0815d510 con 0x7fda080ccfc0 2023-10-06T05:20:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.832+0000 7fda0c9f3700 1 --2- 172.21.15.45:0/2354147123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda0815cdc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fda08067840 tx=0x7fda00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0442ccb16cb346 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.832+0000 7fda06ffd700 1 -- 172.21.15.45:0/2354147123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000e070 con 0x7fda080ccfc0 2023-10-06T05:20:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.832+0000 7fda06ffd700 1 -- 172.21.15.45:0/2354147123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0000ba10 con 0x7fda080ccfc0 2023-10-06T05:20:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.832+0000 7fda06ffd700 1 -- 172.21.15.45:0/2354147123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000092e0 con 0x7fda080ccfc0 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.832+0000 7fda0e456700 1 -- 172.21.15.45:0/2354147123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 msgr2=0x7fda0815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.832+0000 7fda0e456700 1 --2- 172.21.15.45:0/2354147123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda0815cdc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fda08067840 tx=0x7fda00009840 comp rx=0 tx=0).stop 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.833+0000 7fda0e456700 1 -- 172.21.15.45:0/2354147123 shutdown_connections 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.833+0000 7fda0e456700 1 --2- 172.21.15.45:0/2354147123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda0815cdc0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.833+0000 7fda0e456700 1 --2- 172.21.15.45:0/2354147123 >> v2:172.21.15.144:3300/0 conn(0x7fda081539f0 0x7fda0815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.833+0000 7fda0e456700 1 -- 172.21.15.45:0/2354147123 >> 172.21.15.45:0/2354147123 conn(0x7fda0806ed50 msgr2=0x7fda0806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.833+0000 7fda0e456700 1 -- 172.21.15.45:0/2354147123 shutdown_connections 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.833+0000 7fda0e456700 1 -- 172.21.15.45:0/2354147123 wait complete. 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.834+0000 7fda0e456700 1 Processor -- start 2023-10-06T05:20:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.834+0000 7fda0e456700 1 -- start start 2023-10-06T05:20:49.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda0e456700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda080b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda0e456700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda08153620 0x7fda08151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda0e456700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda080c5f90 0x7fda080c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda0e456700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080c6910 con 0x7fda080ccfc0 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda0e456700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080b3430 con 0x7fda080c5f90 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda0e456700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda08151580 con 0x7fda08153620 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda080b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda080b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53064/0 (socket says 172.21.15.45:53064) 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 -- 172.21.15.45:0/74830546 learned_addr learned my addr 172.21.15.45:0/74830546 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda0c9f3700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda080c5f90 0x7fda080c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda077fe700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda08153620 0x7fda08151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 -- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda08153620 msgr2=0x7fda08151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda08153620 0x7fda08151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 -- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda080c5f90 msgr2=0x7fda080c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda080c5f90 0x7fda080c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda07fff700 1 -- 172.21.15.45:0/74830546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda00009ae0 con 0x7fda080ccfc0 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.835+0000 7fda077fe700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda08153620 0x7fda08151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.836+0000 7fda07fff700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda080b2ef0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c5d0 tx=0x7fd9f8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.836+0000 7fd9fffff700 1 -- 172.21.15.45:0/74830546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011590 con 0x7fda080ccfc0 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.836+0000 7fd9fffff700 1 -- 172.21.15.45:0/74830546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8011710 con 0x7fda080ccfc0 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.836+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0816a6a0 con 0x7fda080ccfc0 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.836+0000 7fd9fffff700 1 -- 172.21.15.45:0/74830546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011ad0 con 0x7fda080ccfc0 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.836+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda081557a0 con 0x7fda080ccfc0 2023-10-06T05:20:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.837+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda08088a60 con 0x7fda080ccfc0 2023-10-06T05:20:49.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.838+0000 7fd9fffff700 1 -- 172.21.15.45:0/74830546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9f801c460 con 0x7fda080ccfc0 2023-10-06T05:20:49.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.838+0000 7fd9fffff700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9e8066f30 0x7fd9e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:49.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.838+0000 7fd9fffff700 1 -- 172.21.15.45:0/74830546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd9f8084cf0 con 0x7fda080ccfc0 2023-10-06T05:20:49.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.838+0000 7fda077fe700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9e8066f30 0x7fd9e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:49.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.839+0000 7fda077fe700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9e8066f30 0x7fd9e80693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fda080afb50 tx=0x7fd9f0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:49.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:49.842+0000 7fd9fffff700 1 -- 172.21.15.45:0/74830546 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f8052140 con 0x7fda080ccfc0 2023-10-06T05:20:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.046+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda081531b0 con 0x7fda080ccfc0 2023-10-06T05:20:50.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.047+0000 7fd9fffff700 1 -- 172.21.15.45:0/74830546 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13952 (secure 0 0 0) 0x7fd9f8055760 con 0x7fda080ccfc0 2023-10-06T05:20:50.049 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:50.049 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":75,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:20:49.223370+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:50.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9e8066f30 msgr2=0x7fd9e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9e8066f30 0x7fd9e80693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fda080afb50 tx=0x7fd9f0008040 comp rx=0 tx=0).stop 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 msgr2=0x7fda080b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda080b2ef0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c5d0 tx=0x7fd9f8007f60 comp rx=0 tx=0).stop 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 shutdown_connections 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd9e8066f30 0x7fd9e80693e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda080c5f90 0x7fda080c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda080ccfc0 0x7fda080b2ef0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 --2- 172.21.15.45:0/74830546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda08153620 0x7fda08151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 >> 172.21.15.45:0/74830546 conn(0x7fda0806ed50 msgr2=0x7fda0815cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 shutdown_connections 2023-10-06T05:20:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.050+0000 7fda0e456700 1 -- 172.21.15.45:0/74830546 wait complete. 2023-10-06T05:20:50.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:20:50.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:20:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.199+0000 7f159a6a7700 1 Processor -- start 2023-10-06T05:20:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.199+0000 7f159a6a7700 1 -- start start 2023-10-06T05:20:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.199+0000 7f159a6a7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f15940ada80 0x7f1594163e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.199+0000 7f159a6a7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1594067d80 con 0x7f159406ae40 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f159a6a7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15940c75b0 con 0x7f159406aa70 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f159a6a7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15940ac620 con 0x7f15940ada80 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f15937fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f159406ae40 0x7f159415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53804/0 (socket says 172.21.15.45:53804) 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f15937fe700 1 -- 172.21.15.45:0/2681119999 learned_addr learned my addr 172.21.15.45:0/2681119999 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845992175 0 0) 0x7f1594067d80 con 0x7f159406ae40 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1598c44700 1 --2- 172.21.15.45:0/2681119999 >> v2:172.21.15.144:3300/0 conn(0x7f15940ada80 0x7f1594163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f157c003570 con 0x7f159406ae40 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249144122 0 0) 0x7f157c003570 con 0x7f159406ae40 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 >> v2:172.21.15.144:3300/0 conn(0x7f15940ada80 msgr2=0x7f1594163e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 --2- 172.21.15.45:0/2681119999 >> v2:172.21.15.144:3300/0 conn(0x7f15940ada80 0x7f1594163e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 >> v1:172.21.15.144:6789/0 conn(0x7f159406aa70 legacy=0x7f15940c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940c7890 con 0x7f159406ae40 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f1588001470 con 0x7f159406ae40 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 >> v1:172.21.15.45:6789/0 conn(0x7f159406ae40 legacy=0x7f159415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c003a60 0x7f157c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c004810 0x7f157c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c0051c0 0x7f157c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940ac620 con 0x7f157c003a60 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c00c0d0 con 0x7f157c0051c0 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c00c290 con 0x7f157c004810 2023-10-06T05:20:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.200+0000 7f1593fff700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c003a60 0x7f157c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1598c44700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c0051c0 0x7f157c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f15937fe700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c004810 0x7f157c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1593fff700 1 -- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c004810 msgr2=0x7f157c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1593fff700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c004810 0x7f157c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1593fff700 1 -- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c0051c0 msgr2=0x7f157c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1593fff700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c0051c0 0x7f157c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1593fff700 1 -- 172.21.15.45:0/2681119999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c003570 con 0x7f157c003a60 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1598c44700 1 -- 172.21.15.45:0/2681119999 reap_dead start 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1593fff700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c003a60 0x7f157c003e70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f15840018b0 tx=0x7f1584007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a9b9ebc93647c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584012420 con 0x7f157c003a60 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15840125a0 con 0x7f157c003a60 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.201+0000 7f1592ffd700 1 -- 172.21.15.45:0/2681119999 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584012960 con 0x7f157c003a60 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.202+0000 7f159a6a7700 1 -- 172.21.15.45:0/2681119999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c003a60 msgr2=0x7f157c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.202+0000 7f159a6a7700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c003a60 0x7f157c003e70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f15840018b0 tx=0x7f1584007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.202+0000 7f159a6a7700 1 -- 172.21.15.45:0/2681119999 shutdown_connections 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.202+0000 7f159a6a7700 1 --2- 172.21.15.45:0/2681119999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c003a60 0x7f157c003e70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.202+0000 7f159a6a7700 1 -- 172.21.15.45:0/2681119999 >> 172.21.15.45:0/2681119999 conn(0x7f15940cc570 msgr2=0x7f15940cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.202+0000 7f159a6a7700 1 -- 172.21.15.45:0/2681119999 shutdown_connections 2023-10-06T05:20:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.202+0000 7f159a6a7700 1 -- 172.21.15.45:0/2681119999 wait complete. 2023-10-06T05:20:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.203+0000 7f159a6a7700 1 Processor -- start 2023-10-06T05:20:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.204+0000 7f159a6a7700 1 -- start start 2023-10-06T05:20:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f159a6a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c003a60 0x7f15940aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f159a6a7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1594158a70 0x7f15940adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f159a6a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15940b0440 0x7f15940ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f159a6a7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940a88c0 con 0x7f15940b0440 2023-10-06T05:20:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f159a6a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940b0950 con 0x7f157c003a60 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f159a6a7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15940a8b10 con 0x7f1594158a70 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15940b0440 0x7f15940ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15940b0440 0x7f15940ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53088/0 (socket says 172.21.15.45:53088) 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 -- 172.21.15.45:0/3432230806 learned_addr learned my addr 172.21.15.45:0/3432230806 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f15937fe700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1594158a70 0x7f15940adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1593fff700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c003a60 0x7f15940aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 -- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1594158a70 msgr2=0x7f15940adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1594158a70 0x7f15940adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 -- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c003a60 msgr2=0x7f15940aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c003a60 0x7f15940aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 -- 172.21.15.45:0/3432230806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940a8df0 con 0x7f15940b0440 2023-10-06T05:20:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1593fff700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c003a60 0x7f15940aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.205+0000 7f1598c44700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15940b0440 0x7f15940ae480 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f15940bff80 tx=0x7f158c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.206+0000 7f157bfff700 1 -- 172.21.15.45:0/3432230806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c00e040 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.206+0000 7f157bfff700 1 -- 172.21.15.45:0/3432230806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c00f040 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.206+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15940a90d0 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.206+0000 7f157bfff700 1 -- 172.21.15.45:0/3432230806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c00fe20 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.206+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15940af670 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.207+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15940a92c0 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.207+0000 7f157bfff700 1 -- 172.21.15.45:0/3432230806 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f158c015640 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.208+0000 7f157bfff700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1580066f30 0x7f15800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.208+0000 7f157bfff700 1 -- 172.21.15.45:0/3432230806 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f158c084bc0 con 0x7f15940b0440 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.208+0000 7f1593fff700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1580066f30 0x7f15800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.208+0000 7f1593fff700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1580066f30 0x7f15800693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1584001690 tx=0x7f1584015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.214+0000 7f157bfff700 1 -- 172.21.15.45:0/3432230806 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f158c051ee0 con 0x7f15940b0440 2023-10-06T05:20:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.415+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15940c7b70 con 0x7f15940b0440 2023-10-06T05:20:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.416+0000 7f157bfff700 1 -- 172.21.15.45:0/3432230806 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f158c055500 con 0x7f15940b0440 2023-10-06T05:20:50.418 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:20:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1580066f30 msgr2=0x7f15800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1580066f30 0x7f15800693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1584001690 tx=0x7f1584015000 comp rx=0 tx=0).stop 2023-10-06T05:20:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15940b0440 msgr2=0x7f15940ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15940b0440 0x7f15940ae480 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f15940bff80 tx=0x7f158c00b040 comp rx=0 tx=0).stop 2023-10-06T05:20:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 shutdown_connections 2023-10-06T05:20:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1580066f30 0x7f15800693e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15940b0440 0x7f15940ae480 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c003a60 0x7f15940aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 --2- 172.21.15.45:0/3432230806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1594158a70 0x7f15940adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 >> 172.21.15.45:0/3432230806 conn(0x7f15940cc570 msgr2=0x7f15940cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 shutdown_connections 2023-10-06T05:20:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.418+0000 7f159a6a7700 1 -- 172.21.15.45:0/3432230806 wait complete. 2023-10-06T05:20:50.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:20:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.566+0000 7f29d3daa700 1 Processor -- start 2023-10-06T05:20:50.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.566+0000 7f29d3daa700 1 -- start start 2023-10-06T05:20:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.567+0000 7f29d3daa700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f29cc06af30 0x7f29cc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.567+0000 7f29d3daa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc159ea0 0x7f29cc06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.567+0000 7f29d3daa700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc159a10 con 0x7f29cc158a60 2023-10-06T05:20:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.567+0000 7f29d3daa700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc0ac2e0 con 0x7f29cc06af30 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.567+0000 7f29d3daa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc06f1a0 con 0x7f29cc159ea0 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d2347700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc159ea0 0x7f29cc06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d1345700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f29cc06af30 0x7f29cc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d1345700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f29cc06af30 0x7f29cc06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:47312/0 (socket says 172.21.15.45:47312) 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d1345700 1 -- 172.21.15.45:0/3634689099 learned_addr learned my addr 172.21.15.45:0/3634689099 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d2347700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc159ea0 0x7f29cc06ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:45696/0 (socket says 172.21.15.45:45696) 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d1b46700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f29cc158a60 0x7f29cc06a820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:58854/0 (socket says 172.21.15.45:58854) 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d2347700 1 -- 172.21.15.45:0/3634689099 >> v2:172.21.15.144:3300/0 conn(0x7f29cc06af30 msgr2=0x7f29cc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d2347700 1 --2- 172.21.15.45:0/3634689099 >> v2:172.21.15.144:3300/0 conn(0x7f29cc06af30 0x7f29cc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d2347700 1 -- 172.21.15.45:0/3634689099 >> v1:172.21.15.144:6789/0 conn(0x7f29cc158a60 legacy=0x7f29cc06a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d2347700 1 -- 172.21.15.45:0/3634689099 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc06f410 con 0x7f29cc159ea0 2023-10-06T05:20:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d0b44700 1 -- 172.21.15.45:0/3634689099 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109819937 0 0) 0x7f29cc159a10 con 0x7f29cc158a60 2023-10-06T05:20:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d1345700 1 --2- 172.21.15.45:0/3634689099 >> v2:172.21.15.144:3300/0 conn(0x7f29cc06af30 0x7f29cc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d2347700 1 --2- 172.21.15.45:0/3634689099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc159ea0 0x7f29cc06ec60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009e60 tx=0x7f29c4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319659f5c57f9946 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d0b44700 1 -- 172.21.15.45:0/3634689099 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400e070 con 0x7f29cc159ea0 2023-10-06T05:20:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d0b44700 1 -- 172.21.15.45:0/3634689099 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c400bb60 con 0x7f29cc159ea0 2023-10-06T05:20:50.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.568+0000 7f29d0b44700 1 -- 172.21.15.45:0/3634689099 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4012430 con 0x7f29cc159ea0 2023-10-06T05:20:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 -- 172.21.15.45:0/3634689099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc159ea0 msgr2=0x7f29cc06ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 --2- 172.21.15.45:0/3634689099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc159ea0 0x7f29cc06ec60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009e60 tx=0x7f29c4009780 comp rx=0 tx=0).stop 2023-10-06T05:20:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 -- 172.21.15.45:0/3634689099 shutdown_connections 2023-10-06T05:20:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 --2- 172.21.15.45:0/3634689099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc159ea0 0x7f29cc06ec60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 --2- 172.21.15.45:0/3634689099 >> v2:172.21.15.144:3300/0 conn(0x7f29cc06af30 0x7f29cc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 -- 172.21.15.45:0/3634689099 >> 172.21.15.45:0/3634689099 conn(0x7f29cc0cc570 msgr2=0x7f29cc0cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 -- 172.21.15.45:0/3634689099 shutdown_connections 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.569+0000 7f29d3daa700 1 -- 172.21.15.45:0/3634689099 wait complete. 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.570+0000 7f29d3daa700 1 Processor -- start 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.570+0000 7f29d3daa700 1 -- start start 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d3daa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29cc06af30 0x7f29cc14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d3daa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc158a60 0x7f29cc0a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d3daa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29cc159ea0 0x7f29cc0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d3daa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0a9d30 con 0x7f29cc06af30 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d3daa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0aa2f0 con 0x7f29cc159ea0 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d3daa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0a93c0 con 0x7f29cc158a60 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d1b46700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29cc06af30 0x7f29cc14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d1b46700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29cc06af30 0x7f29cc14eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53098/0 (socket says 172.21.15.45:53098) 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d1b46700 1 -- 172.21.15.45:0/1420358914 learned_addr learned my addr 172.21.15.45:0/1420358914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d1345700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc158a60 0x7f29cc0a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.571+0000 7f29d2347700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29cc159ea0 0x7f29cc0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29d1b46700 1 -- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc158a60 msgr2=0x7f29cc0a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29d1b46700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc158a60 0x7f29cc0a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29d1b46700 1 -- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29cc159ea0 msgr2=0x7f29cc0ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29d1b46700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29cc159ea0 0x7f29cc0ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29d1b46700 1 -- 172.21.15.45:0/1420358914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0094f0 con 0x7f29cc06af30 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29d1b46700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29cc06af30 0x7f29cc14eab0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f29c80011e0 tx=0x7f29c8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29c1ffb700 1 -- 172.21.15.45:0/1420358914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8005a70 con 0x7f29cc06af30 2023-10-06T05:20:50.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29c1ffb700 1 -- 172.21.15.45:0/1420358914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c8005bd0 con 0x7f29cc06af30 2023-10-06T05:20:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c4009a20 con 0x7f29cc06af30 2023-10-06T05:20:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.572+0000 7f29c1ffb700 1 -- 172.21.15.45:0/1420358914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c80070d0 con 0x7f29cc06af30 2023-10-06T05:20:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.573+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc0a9f80 con 0x7f29cc06af30 2023-10-06T05:20:50.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.573+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc06e300 con 0x7f29cc06af30 2023-10-06T05:20:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.574+0000 7f29c1ffb700 1 -- 172.21.15.45:0/1420358914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29c8007480 con 0x7f29cc06af30 2023-10-06T05:20:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.574+0000 7f29c1ffb700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f29b0066f30 0x7f29b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.574+0000 7f29c1ffb700 1 -- 172.21.15.45:0/1420358914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f29c80839e0 con 0x7f29cc06af30 2023-10-06T05:20:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.574+0000 7f29d1345700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f29b0066f30 0x7f29b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.575+0000 7f29d1345700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f29b0066f30 0x7f29b00693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0c6a80 tx=0x7f29bc009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.579+0000 7f29c1ffb700 1 -- 172.21.15.45:0/1420358914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29c8050d00 con 0x7f29cc06af30 2023-10-06T05:20:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.782+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29cc04bb40 con 0x7f29cc06af30 2023-10-06T05:20:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.782+0000 7f29c1ffb700 1 -- 172.21.15.45:0/1420358914 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f29c8017070 con 0x7f29cc06af30 2023-10-06T05:20:50.784 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f29b0066f30 msgr2=0x7f29b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f29b0066f30 0x7f29b00693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0c6a80 tx=0x7f29bc009250 comp rx=0 tx=0).stop 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29cc06af30 msgr2=0x7f29cc14eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29cc06af30 0x7f29cc14eab0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f29c80011e0 tx=0x7f29c8007e10 comp rx=0 tx=0).stop 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 shutdown_connections 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f29b0066f30 0x7f29b00693e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29cc159ea0 0x7f29cc0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29cc06af30 0x7f29cc14eab0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 --2- 172.21.15.45:0/1420358914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29cc158a60 0x7f29cc0a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 >> 172.21.15.45:0/1420358914 conn(0x7f29cc0cc570 msgr2=0x7f29cc0cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 shutdown_connections 2023-10-06T05:20:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.785+0000 7f29d3daa700 1 -- 172.21.15.45:0/1420358914 wait complete. 2023-10-06T05:20:50.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:20:50.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.931+0000 7fef8d955700 1 Processor -- start 2023-10-06T05:20:50.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.931+0000 7fef8d955700 1 -- start start 2023-10-06T05:20:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef8d955700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef88159200 0x7fef880c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef8d955700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fef880ccdc0 0x7fef880cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef8d955700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef880c7d00 con 0x7fef881595d0 2023-10-06T05:20:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef8d955700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef880c7780 con 0x7fef88159200 2023-10-06T05:20:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef8d955700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef880c86d0 con 0x7fef880ccdc0 2023-10-06T05:20:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef86ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef88159200 0x7fef880c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef86ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef88159200 0x7fef880c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53108/0 (socket says 172.21.15.45:53108) 2023-10-06T05:20:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.932+0000 7fef86ffd700 1 -- 172.21.15.45:0/1350837184 learned_addr learned my addr 172.21.15.45:0/1350837184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef877fe700 1 --2- 172.21.15.45:0/1350837184 >> v2:172.21.15.144:3300/0 conn(0x7fef880ccdc0 0x7fef880cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef86ffd700 1 -- 172.21.15.45:0/1350837184 >> v2:172.21.15.144:3300/0 conn(0x7fef880ccdc0 msgr2=0x7fef880cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef86ffd700 1 --2- 172.21.15.45:0/1350837184 >> v2:172.21.15.144:3300/0 conn(0x7fef880ccdc0 0x7fef880cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef86ffd700 1 -- 172.21.15.45:0/1350837184 >> v1:172.21.15.144:6789/0 conn(0x7fef881595d0 legacy=0x7fef8806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef86ffd700 1 -- 172.21.15.45:0/1350837184 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef880cd700 con 0x7fef88159200 2023-10-06T05:20:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef877fe700 1 --2- 172.21.15.45:0/1350837184 >> v2:172.21.15.144:3300/0 conn(0x7fef880ccdc0 0x7fef880cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef86ffd700 1 --2- 172.21.15.45:0/1350837184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef88159200 0x7fef880c8190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fef880c3dd0 tx=0x7fef78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91575e5fb8b9186 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef85ffb700 1 -- 172.21.15.45:0/1350837184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800b820 con 0x7fef88159200 2023-10-06T05:20:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef85ffb700 1 -- 172.21.15.45:0/1350837184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7800b980 con 0x7fef88159200 2023-10-06T05:20:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.933+0000 7fef85ffb700 1 -- 172.21.15.45:0/1350837184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800bd20 con 0x7fef88159200 2023-10-06T05:20:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 -- 172.21.15.45:0/1350837184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef88159200 msgr2=0x7fef880c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 --2- 172.21.15.45:0/1350837184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef88159200 0x7fef880c8190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fef880c3dd0 tx=0x7fef78009250 comp rx=0 tx=0).stop 2023-10-06T05:20:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 -- 172.21.15.45:0/1350837184 shutdown_connections 2023-10-06T05:20:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 --2- 172.21.15.45:0/1350837184 >> v2:172.21.15.144:3300/0 conn(0x7fef880ccdc0 0x7fef880cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 --2- 172.21.15.45:0/1350837184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef88159200 0x7fef880c8190 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 -- 172.21.15.45:0/1350837184 >> 172.21.15.45:0/1350837184 conn(0x7fef8806f090 msgr2=0x7fef8806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:50.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 -- 172.21.15.45:0/1350837184 shutdown_connections 2023-10-06T05:20:50.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.934+0000 7fef8d955700 1 -- 172.21.15.45:0/1350837184 wait complete. 2023-10-06T05:20:50.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.935+0000 7fef8d955700 1 Processor -- start 2023-10-06T05:20:50.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.935+0000 7fef8d955700 1 -- start start 2023-10-06T05:20:50.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef8d955700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef880ccdc0 0x7fef8815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef8d955700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef88159200 0x7fef8815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef8d955700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef881595d0 0x7fef880c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef8d955700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8815a910 con 0x7fef881595d0 2023-10-06T05:20:50.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef8d955700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8815b1c0 con 0x7fef88159200 2023-10-06T05:20:50.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef8d955700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8815b410 con 0x7fef880ccdc0 2023-10-06T05:20:50.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef881595d0 0x7fef880c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef881595d0 0x7fef880c2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53110/0 (socket says 172.21.15.45:53110) 2023-10-06T05:20:50.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef877fe700 1 -- 172.21.15.45:0/3922340649 learned_addr learned my addr 172.21.15.45:0/3922340649 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:50.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef867fc700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef88159200 0x7fef8815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef86ffd700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef880ccdc0 0x7fef8815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef877fe700 1 -- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef880ccdc0 msgr2=0x7fef8815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef877fe700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef880ccdc0 0x7fef8815c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.936+0000 7fef877fe700 1 -- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef88159200 msgr2=0x7fef8815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef877fe700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef88159200 0x7fef8815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef877fe700 1 -- 172.21.15.45:0/3922340649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780094f0 con 0x7fef881595d0 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef877fe700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef881595d0 0x7fef880c2c30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fef8000a380 tx=0x7fef80007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef7effd700 1 -- 172.21.15.45:0/3922340649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000ae10 con 0x7fef881595d0 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef7effd700 1 -- 172.21.15.45:0/3922340649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8000d650 con 0x7fef881595d0 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8815b6f0 con 0x7fef881595d0 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef7effd700 1 -- 172.21.15.45:0/3922340649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000da10 con 0x7fef881595d0 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.937+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8815bd30 con 0x7fef881595d0 2023-10-06T05:20:50.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.938+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef880679c0 con 0x7fef881595d0 2023-10-06T05:20:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.938+0000 7fef7effd700 1 -- 172.21.15.45:0/3922340649 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef8001e3b0 con 0x7fef881595d0 2023-10-06T05:20:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.939+0000 7fef7effd700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef68066f30 0x7fef680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.939+0000 7fef7effd700 1 -- 172.21.15.45:0/3922340649 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fef80084e90 con 0x7fef881595d0 2023-10-06T05:20:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.939+0000 7fef86ffd700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef68066f30 0x7fef680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.940+0000 7fef86ffd700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef68066f30 0x7fef680693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef880c3dd0 tx=0x7fef78016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:50.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:50.943+0000 7fef7effd700 1 -- 172.21.15.45:0/3922340649 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef800521b0 con 0x7fef881595d0 2023-10-06T05:20:51.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.151+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fef8806bb40 con 0x7fef881595d0 2023-10-06T05:20:51.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.151+0000 7fef7effd700 1 -- 172.21.15.45:0/3922340649 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+126 (secure 0 0 0) 0x7fef8001a020 con 0x7fef881595d0 2023-10-06T05:20:51.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:51.153 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:20:51.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:20:51.154 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:20:51.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef68066f30 msgr2=0x7fef680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef68066f30 0x7fef680693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef880c3dd0 tx=0x7fef78016000 comp rx=0 tx=0).stop 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef881595d0 msgr2=0x7fef880c2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef881595d0 0x7fef880c2c30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fef8000a380 tx=0x7fef80007900 comp rx=0 tx=0).stop 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 shutdown_connections 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef68066f30 0x7fef680693e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef881595d0 0x7fef880c2c30 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef880ccdc0 0x7fef8815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 --2- 172.21.15.45:0/3922340649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef88159200 0x7fef8815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.153+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 >> 172.21.15.45:0/3922340649 conn(0x7fef8806f090 msgr2=0x7fef880ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.154+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 shutdown_connections 2023-10-06T05:20:51.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.154+0000 7fef8d955700 1 -- 172.21.15.45:0/3922340649 wait complete. 2023-10-06T05:20:51.169 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:20:51.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:20:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.303+0000 7f38e38f2700 1 Processor -- start 2023-10-06T05:20:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.304+0000 7f38e38f2700 1 -- start start 2023-10-06T05:20:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.304+0000 7f38e38f2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38dc159490 0x7f38dc15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e38f2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38dc15d3d0 0x7f38dc0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e38f2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc068040 con 0x7f38dc1590c0 2023-10-06T05:20:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e38f2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc15dcf0 con 0x7f38dc159490 2023-10-06T05:20:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e38f2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc15d810 con 0x7f38dc15d3d0 2023-10-06T05:20:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e0e8d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38dc159490 0x7f38dc15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e168e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f38dc1590c0 0x7f38dc0ac5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53818/0 (socket says 172.21.15.45:53818) 2023-10-06T05:20:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e168e700 1 -- 172.21.15.45:0/3999309079 learned_addr learned my addr 172.21.15.45:0/3999309079 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e1e8f700 1 --2- 172.21.15.45:0/3999309079 >> v2:172.21.15.144:3300/0 conn(0x7f38dc15d3d0 0x7f38dc0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e0e8d700 1 -- 172.21.15.45:0/3999309079 >> v2:172.21.15.144:3300/0 conn(0x7f38dc15d3d0 msgr2=0x7f38dc0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e0e8d700 1 --2- 172.21.15.45:0/3999309079 >> v2:172.21.15.144:3300/0 conn(0x7f38dc15d3d0 0x7f38dc0c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e0e8d700 1 -- 172.21.15.45:0/3999309079 >> v1:172.21.15.45:6789/0 conn(0x7f38dc1590c0 legacy=0x7f38dc0ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e0e8d700 1 -- 172.21.15.45:0/3999309079 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc067750 con 0x7f38dc159490 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38d3fff700 1 -- 172.21.15.45:0/3999309079 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318732610 0 0) 0x7f38dc068040 con 0x7f38dc1590c0 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38e0e8d700 1 --2- 172.21.15.45:0/3999309079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc15ce90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0b2650 tx=0x7f38cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b7ffd3bb1b804a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38d3fff700 1 -- 172.21.15.45:0/3999309079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00b6b0 con 0x7f38dc159490 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38d3fff700 1 -- 172.21.15.45:0/3999309079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc00b810 con 0x7f38dc159490 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.305+0000 7f38d3fff700 1 -- 172.21.15.45:0/3999309079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00bbb0 con 0x7f38dc159490 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 -- 172.21.15.45:0/3999309079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 msgr2=0x7f38dc15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 --2- 172.21.15.45:0/3999309079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc15ce90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0b2650 tx=0x7f38cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 -- 172.21.15.45:0/3999309079 shutdown_connections 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 --2- 172.21.15.45:0/3999309079 >> v2:172.21.15.144:3300/0 conn(0x7f38dc15d3d0 0x7f38dc0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 --2- 172.21.15.45:0/3999309079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc15ce90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 -- 172.21.15.45:0/3999309079 >> 172.21.15.45:0/3999309079 conn(0x7f38dc0cd060 msgr2=0x7f38dc0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 -- 172.21.15.45:0/3999309079 shutdown_connections 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.306+0000 7f38e38f2700 1 -- 172.21.15.45:0/3999309079 wait complete. 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.307+0000 7f38e38f2700 1 Processor -- start 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.307+0000 7f38e38f2700 1 -- start start 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.308+0000 7f38e38f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1590c0 0x7f38dc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.308+0000 7f38e38f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc0aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.308+0000 7f38e38f2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38dc15d3d0 0x7f38dc152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.308+0000 7f38e38f2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0b1c40 con 0x7f38dc159490 2023-10-06T05:20:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.308+0000 7f38e38f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0aa7f0 con 0x7f38dc1590c0 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.308+0000 7f38e38f2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0b1030 con 0x7f38dc15d3d0 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc0aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc0aa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53118/0 (socket says 172.21.15.45:53118) 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 -- 172.21.15.45:0/2422474798 learned_addr learned my addr 172.21.15.45:0/2422474798 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e168e700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1590c0 0x7f38dc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e1e8f700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38dc15d3d0 0x7f38dc152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 -- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38dc15d3d0 msgr2=0x7f38dc152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38dc15d3d0 0x7f38dc152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 -- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1590c0 msgr2=0x7f38dc0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1590c0 0x7f38dc0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 -- 172.21.15.45:0/2422474798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009420 con 0x7f38dc159490 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e168e700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1590c0 0x7f38dc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e0e8d700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc0aa2b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00b0e0 tx=0x7f38cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38d17fa700 1 -- 172.21.15.45:0/2422474798 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc011420 con 0x7f38dc159490 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38d17fa700 1 -- 172.21.15.45:0/2422474798 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc011580 con 0x7f38dc159490 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc0b1310 con 0x7f38dc159490 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38d17fa700 1 -- 172.21.15.45:0/2422474798 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc011940 con 0x7f38dc159490 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.309+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc0b18d0 con 0x7f38dc159490 2023-10-06T05:20:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.310+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc0b1a60 con 0x7f38dc159490 2023-10-06T05:20:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.311+0000 7f38d17fa700 1 -- 172.21.15.45:0/2422474798 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38cc011d30 con 0x7f38dc159490 2023-10-06T05:20:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.311+0000 7f38d17fa700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38c0066f30 0x7f38c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.312+0000 7f38d17fa700 1 -- 172.21.15.45:0/2422474798 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f38cc086390 con 0x7f38dc159490 2023-10-06T05:20:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.312+0000 7f38e168e700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38c0066f30 0x7f38c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.312+0000 7f38e168e700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38c0066f30 0x7f38c00693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f38d8001890 tx=0x7f38d8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.315+0000 7f38d17fa700 1 -- 172.21.15.45:0/2422474798 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38cc053730 con 0x7f38dc159490 2023-10-06T05:20:51.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:51.521+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f38dc0b1df0 con 0x7f38dc159490 2023-10-06T05:20:52.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.227+0000 7f38d17fa700 1 -- 172.21.15.45:0/2422474798 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f38cc056d50 con 0x7f38dc159490 2023-10-06T05:20:52.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38c0066f30 msgr2=0x7f38c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38c0066f30 0x7f38c00693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f38d8001890 tx=0x7f38d8009040 comp rx=0 tx=0).stop 2023-10-06T05:20:52.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 msgr2=0x7f38dc0aa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc0aa2b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00b0e0 tx=0x7f38cc017000 comp rx=0 tx=0).stop 2023-10-06T05:20:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 shutdown_connections 2023-10-06T05:20:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38c0066f30 0x7f38c00693e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38dc15d3d0 0x7f38dc152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38dc1590c0 0x7f38dc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 --2- 172.21.15.45:0/2422474798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38dc159490 0x7f38dc0aa2b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.234+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 >> 172.21.15.45:0/2422474798 conn(0x7f38dc0cd060 msgr2=0x7f38dc15b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.235+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 shutdown_connections 2023-10-06T05:20:52.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.235+0000 7f38e38f2700 1 -- 172.21.15.45:0/2422474798 wait complete. 2023-10-06T05:20:52.239 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-06T05:20:52.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:20:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.393+0000 7f3d512a4700 1 Processor -- start 2023-10-06T05:20:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.393+0000 7f3d512a4700 1 -- start start 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.394+0000 7f3d512a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.394+0000 7f3d512a4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c0c9210 con 0x7f3d4c0b0170 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.394+0000 7f3d512a4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c06e090 con 0x7f3d4c068730 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.394+0000 7f3d512a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c06e370 con 0x7f3d4c0ccc00 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.394+0000 7f3d4affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3d4c0b0170 0x7f3d4c0cc4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53820/0 (socket says 172.21.15.45:53820) 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.394+0000 7f3d4affd700 1 -- 172.21.15.45:0/2695039633 learned_addr learned my addr 172.21.15.45:0/2695039633 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.394+0000 7f3d4b7fe700 1 --2- 172.21.15.45:0/2695039633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386878903 0 0) 0x7f3d4c0c9210 con 0x7f3d4c0b0170 2023-10-06T05:20:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d38003570 con 0x7f3d4c0b0170 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806829337 0 0) 0x7f3d4c06e090 con 0x7f3d4c068730 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d4c0c9210 con 0x7f3d4c068730 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 21685853 0 0) 0x7f3d38003570 con 0x7f3d4c0b0170 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d4b7fe700 1 -- 172.21.15.45:0/2695039633 >> v1:172.21.15.144:6789/0 conn(0x7f3d4c068730 legacy=0x7f3d4c0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d4b7fe700 1 -- 172.21.15.45:0/2695039633 >> v1:172.21.15.45:6789/0 conn(0x7f3d4c0b0170 legacy=0x7f3d4c0cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d4b7fe700 1 -- 172.21.15.45:0/2695039633 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0c8ea0 con 0x7f3d4c0ccc00 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d4b7fe700 1 --2- 172.21.15.45:0/2695039633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c0b0540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009e90 tx=0x7f3d44009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f69f9b04f9273 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400e070 con 0x7f3d4c0ccc00 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4400b870 con 0x7f3d4c0ccc00 2023-10-06T05:20:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.395+0000 7f3d49ffb700 1 -- 172.21.15.45:0/2695039633 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c001470 con 0x7f3d4c0ccc00 2023-10-06T05:20:52.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.396+0000 7f3d512a4700 1 -- 172.21.15.45:0/2695039633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 msgr2=0x7f3d4c0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.396+0000 7f3d512a4700 1 --2- 172.21.15.45:0/2695039633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c0b0540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009e90 tx=0x7f3d44009780 comp rx=0 tx=0).stop 2023-10-06T05:20:52.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.396+0000 7f3d512a4700 1 -- 172.21.15.45:0/2695039633 shutdown_connections 2023-10-06T05:20:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.396+0000 7f3d512a4700 1 --2- 172.21.15.45:0/2695039633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c0b0540 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.396+0000 7f3d512a4700 1 -- 172.21.15.45:0/2695039633 >> 172.21.15.45:0/2695039633 conn(0x7f3d4c0ce9d0 msgr2=0x7f3d4c06c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.396+0000 7f3d512a4700 1 -- 172.21.15.45:0/2695039633 shutdown_connections 2023-10-06T05:20:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.396+0000 7f3d512a4700 1 -- 172.21.15.45:0/2695039633 wait complete. 2023-10-06T05:20:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.397+0000 7f3d512a4700 1 Processor -- start 2023-10-06T05:20:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.397+0000 7f3d512a4700 1 -- start start 2023-10-06T05:20:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d512a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c068730 0x7f3d4c156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d512a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d4c0b0170 0x7f3d4c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d512a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d512a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0b2770 con 0x7f3d4c0b0170 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d512a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0b29c0 con 0x7f3d4c068730 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d512a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0b2ca0 con 0x7f3d4c0ccc00 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d4a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d4c0b0170 0x7f3d4c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d4a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d4c0b0170 0x7f3d4c0c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53128/0 (socket says 172.21.15.45:53128) 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d4a7fc700 1 -- 172.21.15.45:0/1525010734 learned_addr learned my addr 172.21.15.45:0/1525010734 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d4b7fe700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.398+0000 7f3d4affd700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c068730 0x7f3d4c156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d4a7fc700 1 -- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 msgr2=0x7f3d4c068b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d4a7fc700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c068b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d4a7fc700 1 -- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c068730 msgr2=0x7f3d4c156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d4a7fc700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c068730 0x7f3d4c156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d4a7fc700 1 -- 172.21.15.45:0/1525010734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d44009a20 con 0x7f3d4c0b0170 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d4a7fc700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d4c0b0170 0x7f3d4c0c7b50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c790 tx=0x7f3d34007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d42ffd700 1 -- 172.21.15.45:0/1525010734 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34010510 con 0x7f3d4c0b0170 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d42ffd700 1 -- 172.21.15.45:0/1525010734 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d34010670 con 0x7f3d4c0b0170 2023-10-06T05:20:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c0b2f80 con 0x7f3d4c0b0170 2023-10-06T05:20:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d42ffd700 1 -- 172.21.15.45:0/1525010734 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34010a10 con 0x7f3d4c0b0170 2023-10-06T05:20:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.399+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c0b3540 con 0x7f3d4c0b0170 2023-10-06T05:20:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.400+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c0b3760 con 0x7f3d4c0b0170 2023-10-06T05:20:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.401+0000 7f3d42ffd700 1 -- 172.21.15.45:0/1525010734 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d3401b400 con 0x7f3d4c0b0170 2023-10-06T05:20:52.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.401+0000 7f3d42ffd700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d2c066f30 0x7f3d2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.401+0000 7f3d42ffd700 1 -- 172.21.15.45:0/1525010734 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f3d340843c0 con 0x7f3d4c0b0170 2023-10-06T05:20:52.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.401+0000 7f3d4affd700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d2c066f30 0x7f3d2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.402+0000 7f3d4affd700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d2c066f30 0x7f3d2c0693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0015a0 tx=0x7f3d3c000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.406+0000 7f3d42ffd700 1 -- 172.21.15.45:0/1525010734 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d34051570 con 0x7f3d4c0b0170 2023-10-06T05:20:52.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:52.611+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3d4c04c380 con 0x7f3d4c0b0170 2023-10-06T05:20:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.235+0000 7f3d42ffd700 1 -- 172.21.15.45:0/1525010734 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3d34054b90 con 0x7f3d4c0b0170 2023-10-06T05:20:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.240+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d2c066f30 msgr2=0x7f3d2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.240+0000 7f3d512a4700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d2c066f30 0x7f3d2c0693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0015a0 tx=0x7f3d3c000bc0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.240+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d4c0b0170 msgr2=0x7f3d4c0c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.240+0000 7f3d512a4700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d4c0b0170 0x7f3d4c0c7b50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c790 tx=0x7f3d34007e10 comp rx=0 tx=0).stop 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.247+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 shutdown_connections 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.247+0000 7f3d512a4700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d2c066f30 0x7f3d2c0693e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.247+0000 7f3d512a4700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4c0ccc00 0x7f3d4c068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.247+0000 7f3d512a4700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4c068730 0x7f3d4c156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.247+0000 7f3d512a4700 1 --2- 172.21.15.45:0/1525010734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d4c0b0170 0x7f3d4c0c7b50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.247+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 >> 172.21.15.45:0/1525010734 conn(0x7f3d4c0ce9d0 msgr2=0x7f3d4c1588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.248+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 shutdown_connections 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.249+0000 7f3d512a4700 1 -- 172.21.15.45:0/1525010734 wait complete. 2023-10-06T05:20:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-06T05:20:53.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:20:53.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.500+0000 7efce61f4700 1 Processor -- start 2023-10-06T05:20:53.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.500+0000 7efce61f4700 1 -- start start 2023-10-06T05:20:53.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efce61f4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efce006a040 0x7efce0068730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efce61f4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a410 0x7efce006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efce61f4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00c8ad0 con 0x7efce006de20 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efce61f4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce006e2a0 con 0x7efce006a040 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efce61f4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce015a800 con 0x7efce006a410 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efce006a040 0x7efce0068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efce006de20 0x7efce01597d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53824/0 (socket says 172.21.15.45:53824) 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efce006a040 0x7efce0068730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53140/0 (socket says 172.21.15.45:53140) 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdffff700 1 -- 172.21.15.45:0/2089681748 learned_addr learned my addr 172.21.15.45:0/2089681748 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdeffd700 1 --2- 172.21.15.45:0/2089681748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a410 0x7efce006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdf7fe700 1 -- 172.21.15.45:0/2089681748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a410 msgr2=0x7efce006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdf7fe700 1 --2- 172.21.15.45:0/2089681748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a410 0x7efce006d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdf7fe700 1 -- 172.21.15.45:0/2089681748 >> v1:172.21.15.45:6789/0 conn(0x7efce006de20 legacy=0x7efce01597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.501+0000 7efcdf7fe700 1 -- 172.21.15.45:0/2089681748 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00c9350 con 0x7efce006a040 2023-10-06T05:20:53.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efcdf7fe700 1 --2- 172.21.15.45:0/2089681748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006a040 0x7efce0068730 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efcd00098e0 tx=0x7efcd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5fa887e4e6adec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efcde7fc700 1 -- 172.21.15.45:0/2089681748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0009e80 con 0x7efce006a040 2023-10-06T05:20:53.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efcde7fc700 1 -- 172.21.15.45:0/2089681748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd000b860 con 0x7efce006a040 2023-10-06T05:20:53.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efcde7fc700 1 -- 172.21.15.45:0/2089681748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000bbe0 con 0x7efce006a040 2023-10-06T05:20:53.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 -- 172.21.15.45:0/2089681748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006a040 msgr2=0x7efce0068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 --2- 172.21.15.45:0/2089681748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006a040 0x7efce0068730 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efcd00098e0 tx=0x7efcd00091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 -- 172.21.15.45:0/2089681748 shutdown_connections 2023-10-06T05:20:53.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 --2- 172.21.15.45:0/2089681748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006a040 0x7efce0068730 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 --2- 172.21.15.45:0/2089681748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a410 0x7efce006d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 -- 172.21.15.45:0/2089681748 >> 172.21.15.45:0/2089681748 conn(0x7efce00cea10 msgr2=0x7efce006c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 -- 172.21.15.45:0/2089681748 shutdown_connections 2023-10-06T05:20:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.502+0000 7efce61f4700 1 -- 172.21.15.45:0/2089681748 wait complete. 2023-10-06T05:20:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.504+0000 7efce61f4700 1 Processor -- start 2023-10-06T05:20:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.504+0000 7efce61f4700 1 -- start start 2023-10-06T05:20:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efce61f4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a040 0x7efce015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efce61f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efce006a410 0x7efce00ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efce61f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006de20 0x7efce0156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efce61f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00accd0 con 0x7efce006de20 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efce61f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce015c2d0 con 0x7efce006a410 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efce61f4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce015ba30 con 0x7efce006a040 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006de20 0x7efce0156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006de20 0x7efce0156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53150/0 (socket says 172.21.15.45:53150) 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 -- 172.21.15.45:0/4185420509 learned_addr learned my addr 172.21.15.45:0/4185420509 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdf7fe700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a040 0x7efce015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdeffd700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efce006a410 0x7efce00ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 -- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a040 msgr2=0x7efce015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a040 0x7efce015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 -- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efce006a410 msgr2=0x7efce00ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efce006a410 0x7efce00ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 -- 172.21.15.45:0/4185420509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0009420 con 0x7efce006de20 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.505+0000 7efcdffff700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006de20 0x7efce0156fc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efcd800cc40 tx=0x7efcd8001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.506+0000 7efcd77fe700 1 -- 172.21.15.45:0/4185420509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800a8b0 con 0x7efce006de20 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.506+0000 7efcd77fe700 1 -- 172.21.15.45:0/4185420509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd800aa10 con 0x7efce006de20 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.506+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce015bd10 con 0x7efce006de20 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.506+0000 7efcd77fe700 1 -- 172.21.15.45:0/4185420509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800add0 con 0x7efce006de20 2023-10-06T05:20:53.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.506+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce00c63c0 con 0x7efce006de20 2023-10-06T05:20:53.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.507+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce00c65e0 con 0x7efce006de20 2023-10-06T05:20:53.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.508+0000 7efcd77fe700 1 -- 172.21.15.45:0/4185420509 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcd8007300 con 0x7efce006de20 2023-10-06T05:20:53.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.508+0000 7efcd77fe700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efcc0066f30 0x7efcc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.508+0000 7efcd77fe700 1 -- 172.21.15.45:0/4185420509 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6278+0+0 (secure 0 0 0) 0x7efcd8085520 con 0x7efce006de20 2023-10-06T05:20:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.508+0000 7efcdf7fe700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efcc0066f30 0x7efcc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.509+0000 7efcdf7fe700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efcc0066f30 0x7efcc00693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efcd00091e0 tx=0x7efcd0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.513+0000 7efcd77fe700 1 -- 172.21.15.45:0/4185420509 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcd80525b0 con 0x7efce006de20 2023-10-06T05:20:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:53.762+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efce015a520 con 0x7efce006de20 2023-10-06T05:20:54.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.238+0000 7efcd77fe700 1 -- 172.21.15.45:0/4185420509 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7efcd8055bd0 con 0x7efce006de20 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efcc0066f30 msgr2=0x7efcc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efcc0066f30 0x7efcc00693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efcd00091e0 tx=0x7efcd0017000 comp rx=0 tx=0).stop 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006de20 msgr2=0x7efce0156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006de20 0x7efce0156fc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efcd800cc40 tx=0x7efcd8001a50 comp rx=0 tx=0).stop 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 shutdown_connections 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efcc0066f30 0x7efcc00693e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce006de20 0x7efce0156fc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efce006a040 0x7efce015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 --2- 172.21.15.45:0/4185420509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efce006a410 0x7efce00ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 >> 172.21.15.45:0/4185420509 conn(0x7efce00cea10 msgr2=0x7efce00c31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 shutdown_connections 2023-10-06T05:20:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.243+0000 7efce61f4700 1 -- 172.21.15.45:0/4185420509 wait complete. 2023-10-06T05:20:54.254 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 12 and data pool 13 2023-10-06T05:20:54.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.413+0000 7fef976ff700 1 Processor -- start 2023-10-06T05:20:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.413+0000 7fef976ff700 1 -- start start 2023-10-06T05:20:54.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef976ff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fef9006eab0 0x7fef900631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef976ff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef9006a890 0x7fef9006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef976ff700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900c8080 con 0x7fef9006e6e0 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef976ff700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef90068cd0 con 0x7fef9006a890 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef976ff700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900c8a30 con 0x7fef9006eab0 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef95c9c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef9006a890 0x7fef9006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef95c9c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef9006a890 0x7fef9006ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53164/0 (socket says 172.21.15.45:53164) 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef95c9c700 1 -- 172.21.15.45:0/3770557138 learned_addr learned my addr 172.21.15.45:0/3770557138 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef94c9a700 1 --2- 172.21.15.45:0/3770557138 >> v2:172.21.15.144:3300/0 conn(0x7fef9006eab0 0x7fef900631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef95c9c700 1 -- 172.21.15.45:0/3770557138 >> v2:172.21.15.144:3300/0 conn(0x7fef9006eab0 msgr2=0x7fef900631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef95c9c700 1 --2- 172.21.15.45:0/3770557138 >> v2:172.21.15.144:3300/0 conn(0x7fef9006eab0 0x7fef900631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef95c9c700 1 -- 172.21.15.45:0/3770557138 >> v1:172.21.15.144:6789/0 conn(0x7fef9006e6e0 legacy=0x7fef900c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.414+0000 7fef95c9c700 1 -- 172.21.15.45:0/3770557138 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90156050 con 0x7fef9006a890 2023-10-06T05:20:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef95c9c700 1 --2- 172.21.15.45:0/3770557138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006a890 0x7fef9006ad10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef90156bd0 tx=0x7fef88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf78825603b48ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef87fff700 1 -- 172.21.15.45:0/3770557138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800e070 con 0x7fef9006a890 2023-10-06T05:20:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef87fff700 1 -- 172.21.15.45:0/3770557138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800ba10 con 0x7fef9006a890 2023-10-06T05:20:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef87fff700 1 -- 172.21.15.45:0/3770557138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880092e0 con 0x7fef9006a890 2023-10-06T05:20:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef976ff700 1 -- 172.21.15.45:0/3770557138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006a890 msgr2=0x7fef9006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef976ff700 1 --2- 172.21.15.45:0/3770557138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006a890 0x7fef9006ad10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef90156bd0 tx=0x7fef88009840 comp rx=0 tx=0).stop 2023-10-06T05:20:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef976ff700 1 -- 172.21.15.45:0/3770557138 shutdown_connections 2023-10-06T05:20:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef976ff700 1 --2- 172.21.15.45:0/3770557138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006a890 0x7fef9006ad10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef976ff700 1 --2- 172.21.15.45:0/3770557138 >> v2:172.21.15.144:3300/0 conn(0x7fef9006eab0 0x7fef900631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.415+0000 7fef976ff700 1 -- 172.21.15.45:0/3770557138 >> 172.21.15.45:0/3770557138 conn(0x7fef900cc5a0 msgr2=0x7fef900cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.416+0000 7fef976ff700 1 -- 172.21.15.45:0/3770557138 shutdown_connections 2023-10-06T05:20:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.416+0000 7fef976ff700 1 -- 172.21.15.45:0/3770557138 wait complete. 2023-10-06T05:20:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.417+0000 7fef976ff700 1 Processor -- start 2023-10-06T05:20:54.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.417+0000 7fef976ff700 1 -- start start 2023-10-06T05:20:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef976ff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef9006a890 0x7fef901437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef976ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006e6e0 0x7fef900aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef976ff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9006eab0 0x7fef900cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef976ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900aae30 con 0x7fef9006e6e0 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef976ff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900ab110 con 0x7fef9006eab0 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef976ff700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900ab3f0 con 0x7fef9006a890 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006e6e0 0x7fef900aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006e6e0 0x7fef900aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53168/0 (socket says 172.21.15.45:53168) 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 -- 172.21.15.45:0/3820786866 learned_addr learned my addr 172.21.15.45:0/3820786866 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef95c9c700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9006eab0 0x7fef900cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef9549b700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef9006a890 0x7fef901437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 -- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef9006a890 msgr2=0x7fef901437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef9006a890 0x7fef901437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 -- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9006eab0 msgr2=0x7fef900cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9006eab0 0x7fef900cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 -- 172.21.15.45:0/3820786866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009ae0 con 0x7fef9006e6e0 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef95c9c700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9006eab0 0x7fef900cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef9549b700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef9006a890 0x7fef901437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef94c9a700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006e6e0 0x7fef900aa340 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef8000d860 tx=0x7fef80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef857fa700 1 -- 172.21.15.45:0/3820786866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80018070 con 0x7fef9006e6e0 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef857fa700 1 -- 172.21.15.45:0/3820786866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef800125f0 con 0x7fef9006e6e0 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.418+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef900ab6d0 con 0x7fef9006e6e0 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.419+0000 7fef857fa700 1 -- 172.21.15.45:0/3820786866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef800129b0 con 0x7fef9006e6e0 2023-10-06T05:20:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.419+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef90143d30 con 0x7fef9006e6e0 2023-10-06T05:20:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.419+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef90143f50 con 0x7fef9006e6e0 2023-10-06T05:20:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.420+0000 7fef857fa700 1 -- 172.21.15.45:0/3820786866 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef80014430 con 0x7fef9006e6e0 2023-10-06T05:20:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.420+0000 7fef857fa700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef74066f00 0x7fef740693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.420+0000 7fef857fa700 1 -- 172.21.15.45:0/3820786866 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fef80084ac0 con 0x7fef9006e6e0 2023-10-06T05:20:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.421+0000 7fef9549b700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef74066f00 0x7fef740693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.421+0000 7fef9549b700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef74066f00 0x7fef740693b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef8c002880 tx=0x7fef8c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.425+0000 7fef857fa700 1 -- 172.21.15.45:0/3820786866 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef800519f0 con 0x7fef9006e6e0 2023-10-06T05:20:54.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.629+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef9006b060 con 0x7fef9006e6e0 2023-10-06T05:20:54.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.630+0000 7fef857fa700 1 -- 172.21.15.45:0/3820786866 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17183 (secure 0 0 0) 0x7fef80055010 con 0x7fef9006e6e0 2023-10-06T05:20:54.632 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:54.632 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":78,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:20:53.763627+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:51.554669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:52.645101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.632+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef74066f00 msgr2=0x7fef740693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.632+0000 7fef976ff700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef74066f00 0x7fef740693b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef8c002880 tx=0x7fef8c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.632+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006e6e0 msgr2=0x7fef900aa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.632+0000 7fef976ff700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006e6e0 0x7fef900aa340 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef8000d860 tx=0x7fef80008040 comp rx=0 tx=0).stop 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 shutdown_connections 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef74066f00 0x7fef740693b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef9006eab0 0x7fef900cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef9006a890 0x7fef901437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 --2- 172.21.15.45:0/3820786866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9006e6e0 0x7fef900aa340 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 >> 172.21.15.45:0/3820786866 conn(0x7fef900cc5a0 msgr2=0x7fef900ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 shutdown_connections 2023-10-06T05:20:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.633+0000 7fef976ff700 1 -- 172.21.15.45:0/3820786866 wait complete. 2023-10-06T05:20:54.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.780+0000 7fb5cf0f9700 1 Processor -- start 2023-10-06T05:20:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.780+0000 7fb5cf0f9700 1 -- start start 2023-10-06T05:20:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.781+0000 7fb5cf0f9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80c8ff0 0x7fb5c80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.781+0000 7fb5cf0f9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb5c80caad0 0x7fb5c80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cf0f9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5c8156af0 0x7fb5c8156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cf0f9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c815b000 con 0x7fb5c8156af0 2023-10-06T05:20:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cf0f9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c815b2e0 con 0x7fb5c80caad0 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cf0f9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815b5c0 con 0x7fb5c80c8ff0 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5c8156af0 0x7fb5c8156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5c8156af0 0x7fb5c8156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53176/0 (socket says 172.21.15.45:53176) 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 -- 172.21.15.45:0/3420696066 learned_addr learned my addr 172.21.15.45:0/3420696066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cce95700 1 --2- 172.21.15.45:0/3420696066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80c8ff0 0x7fb5c80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5c7fff700 1 --2- 172.21.15.45:0/3420696066 >> v2:172.21.15.144:3300/0 conn(0x7fb5c80caad0 0x7fb5c80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 -- 172.21.15.45:0/3420696066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80c8ff0 msgr2=0x7fb5c80c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 --2- 172.21.15.45:0/3420696066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80c8ff0 0x7fb5c80c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 -- 172.21.15.45:0/3420696066 >> v2:172.21.15.144:3300/0 conn(0x7fb5c80caad0 msgr2=0x7fb5c80caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 --2- 172.21.15.45:0/3420696066 >> v2:172.21.15.144:3300/0 conn(0x7fb5c80caad0 0x7fb5c80caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 -- 172.21.15.45:0/3420696066 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c815b8a0 con 0x7fb5c8156af0 2023-10-06T05:20:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cce95700 1 --2- 172.21.15.45:0/3420696066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80c8ff0 0x7fb5c80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5c7fff700 1 --2- 172.21.15.45:0/3420696066 >> v2:172.21.15.144:3300/0 conn(0x7fb5c80caad0 0x7fb5c80caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5cd696700 1 --2- 172.21.15.45:0/3420696066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c8156f70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80c4fb0 tx=0x7fb5b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ca1d78b183e17a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5c77fe700 1 -- 172.21.15.45:0/3420696066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800b860 con 0x7fb5c8156af0 2023-10-06T05:20:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5c77fe700 1 -- 172.21.15.45:0/3420696066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b800b9c0 con 0x7fb5c8156af0 2023-10-06T05:20:54.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.782+0000 7fb5c77fe700 1 -- 172.21.15.45:0/3420696066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800bd60 con 0x7fb5c8156af0 2023-10-06T05:20:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/3420696066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 msgr2=0x7fb5c8156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/3420696066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c8156f70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80c4fb0 tx=0x7fb5b8009250 comp rx=0 tx=0).stop 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/3420696066 shutdown_connections 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/3420696066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c8156f70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/3420696066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80c8ff0 0x7fb5c80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/3420696066 >> v2:172.21.15.144:3300/0 conn(0x7fb5c80caad0 0x7fb5c80caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/3420696066 >> 172.21.15.45:0/3420696066 conn(0x7fb5c80cdf50 msgr2=0x7fb5c80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/3420696066 shutdown_connections 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.783+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/3420696066 wait complete. 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.784+0000 7fb5cf0f9700 1 Processor -- start 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.784+0000 7fb5cf0f9700 1 -- start start 2023-10-06T05:20:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cf0f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c80c8ff0 0x7fb5c80b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cf0f9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80caad0 0x7fb5c8154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cf0f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c80b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cf0f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c72d0 con 0x7fb5c8156af0 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cf0f9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80ad780 con 0x7fb5c80c8ff0 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cf0f9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c7760 con 0x7fb5c80caad0 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cd696700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c80b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cd696700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c80b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53178/0 (socket says 172.21.15.45:53178) 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cd696700 1 -- 172.21.15.45:0/4276772728 learned_addr learned my addr 172.21.15.45:0/4276772728 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5c7fff700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80caad0 0x7fb5c8154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.785+0000 7fb5cce95700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c80c8ff0 0x7fb5c80b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cd696700 1 -- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80caad0 msgr2=0x7fb5c8154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cd696700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80caad0 0x7fb5c8154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cd696700 1 -- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c80c8ff0 msgr2=0x7fb5c80b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cd696700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c80c8ff0 0x7fb5c80b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cd696700 1 -- 172.21.15.45:0/4276772728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b80094f0 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cd696700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c80b1590 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80c4f80 tx=0x7fb5b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5c4ff9700 1 -- 172.21.15.45:0/4276772728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8011ea0 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5c4ff9700 1 -- 172.21.15.45:0/4276772728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b801a630 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c80b1180 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5c4ff9700 1 -- 172.21.15.45:0/4276772728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b801aa10 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.786+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c80c6f60 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.787+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c80c7480 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.788+0000 7fb5c4ff9700 1 -- 172.21.15.45:0/4276772728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5b802d440 con 0x7fb5c8156af0 2023-10-06T05:20:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.788+0000 7fb5c4ff9700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5a8066f30 0x7fb5a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.788+0000 7fb5c4ff9700 1 -- 172.21.15.45:0/4276772728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb5b80863a0 con 0x7fb5c8156af0 2023-10-06T05:20:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.788+0000 7fb5cce95700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5a8066f30 0x7fb5a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.789+0000 7fb5cce95700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5a8066f30 0x7fb5a80693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000f600 tx=0x7fb5c0007720 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.793+0000 7fb5c4ff9700 1 -- 172.21.15.45:0/4276772728 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5c80c7480 con 0x7fb5c8156af0 2023-10-06T05:20:54.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.994+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5c80c7410 con 0x7fb5c8156af0 2023-10-06T05:20:54.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.995+0000 7fb5c4ff9700 1 -- 172.21.15.45:0/4276772728 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17183 (secure 0 0 0) 0x7fb5b8053370 con 0x7fb5c8156af0 2023-10-06T05:20:54.997 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:54.997 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":78,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:20:53.763627+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:51.554669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:52.645101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5a8066f30 msgr2=0x7fb5a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5a8066f30 0x7fb5a80693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000f600 tx=0x7fb5c0007720 comp rx=0 tx=0).stop 2023-10-06T05:20:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 msgr2=0x7fb5c80b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c80b1590 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80c4f80 tx=0x7fb5b8017000 comp rx=0 tx=0).stop 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 shutdown_connections 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5a8066f30 0x7fb5a80693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c8156af0 0x7fb5c80b1590 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c80c8ff0 0x7fb5c80b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 --2- 172.21.15.45:0/4276772728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c80caad0 0x7fb5c8154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 >> 172.21.15.45:0/4276772728 conn(0x7fb5c80cdf50 msgr2=0x7fb5c80adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 shutdown_connections 2023-10-06T05:20:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:54.997+0000 7fb5cf0f9700 1 -- 172.21.15.45:0/4276772728 wait complete. 2023-10-06T05:20:55.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:20:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.146+0000 7fdf9b763700 1 Processor -- start 2023-10-06T05:20:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.146+0000 7fdf9b763700 1 -- start start 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf9b763700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdf940cc840 0x7fdf940c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf9b763700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf940c9d50 0x7fdf940affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf9b763700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf940ccca0 con 0x7fdf9413f900 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf9b763700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf940c9630 con 0x7fdf940c9d50 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf9b763700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf940ca150 con 0x7fdf940cc840 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf99d00700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf940c9d50 0x7fdf940affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf994ff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdf9413f900 0x7fdf940c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53832/0 (socket says 172.21.15.45:53832) 2023-10-06T05:20:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.147+0000 7fdf994ff700 1 -- 172.21.15.45:0/859486807 learned_addr learned my addr 172.21.15.45:0/859486807 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf98cfe700 1 --2- 172.21.15.45:0/859486807 >> v2:172.21.15.144:3300/0 conn(0x7fdf940cc840 0x7fdf940c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf99d00700 1 -- 172.21.15.45:0/859486807 >> v2:172.21.15.144:3300/0 conn(0x7fdf940cc840 msgr2=0x7fdf940c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf99d00700 1 --2- 172.21.15.45:0/859486807 >> v2:172.21.15.144:3300/0 conn(0x7fdf940cc840 0x7fdf940c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf99d00700 1 -- 172.21.15.45:0/859486807 >> v1:172.21.15.45:6789/0 conn(0x7fdf9413f900 legacy=0x7fdf940c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf99d00700 1 -- 172.21.15.45:0/859486807 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940cb550 con 0x7fdf940c9d50 2023-10-06T05:20:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf87fff700 1 -- 172.21.15.45:0/859486807 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253842194 0 0) 0x7fdf88001110 con 0x7fdf9413f900 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf99d00700 1 --2- 172.21.15.45:0/859486807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940c9d50 0x7fdf940affa0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdf940cc620 tx=0x7fdf8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b48eb3853739e20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf87fff700 1 -- 172.21.15.45:0/859486807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00e070 con 0x7fdf940c9d50 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf87fff700 1 -- 172.21.15.45:0/859486807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c00bc30 con 0x7fdf940c9d50 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.148+0000 7fdf87fff700 1 -- 172.21.15.45:0/859486807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c012430 con 0x7fdf940c9d50 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 -- 172.21.15.45:0/859486807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940c9d50 msgr2=0x7fdf940affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 --2- 172.21.15.45:0/859486807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940c9d50 0x7fdf940affa0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdf940cc620 tx=0x7fdf8c009780 comp rx=0 tx=0).stop 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 -- 172.21.15.45:0/859486807 shutdown_connections 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 --2- 172.21.15.45:0/859486807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940c9d50 0x7fdf940affa0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 --2- 172.21.15.45:0/859486807 >> v2:172.21.15.144:3300/0 conn(0x7fdf940cc840 0x7fdf940c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 -- 172.21.15.45:0/859486807 >> 172.21.15.45:0/859486807 conn(0x7fdf940691c0 msgr2=0x7fdf940cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 -- 172.21.15.45:0/859486807 shutdown_connections 2023-10-06T05:20:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.149+0000 7fdf9b763700 1 -- 172.21.15.45:0/859486807 wait complete. 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.150+0000 7fdf9b763700 1 Processor -- start 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.150+0000 7fdf9b763700 1 -- start start 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf9b763700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c9d50 0x7fdf94167650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf9b763700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940cc840 0x7fdf940b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf9b763700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9413f900 0x7fdf9416d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf9b763700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9416bd90 con 0x7fdf940cc840 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf9b763700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9416b180 con 0x7fdf9413f900 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf9b763700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9416b460 con 0x7fdf940c9d50 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf98cfe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940cc840 0x7fdf940b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf98cfe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940cc840 0x7fdf940b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53190/0 (socket says 172.21.15.45:53190) 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf98cfe700 1 -- 172.21.15.45:0/364131889 learned_addr learned my addr 172.21.15.45:0/364131889 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf99d00700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9413f900 0x7fdf9416d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.151+0000 7fdf994ff700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c9d50 0x7fdf94167650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf98cfe700 1 -- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c9d50 msgr2=0x7fdf94167650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf98cfe700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c9d50 0x7fdf94167650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf98cfe700 1 -- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9413f900 msgr2=0x7fdf9416d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf98cfe700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9413f900 0x7fdf9416d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf98cfe700 1 -- 172.21.15.45:0/364131889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c009a20 con 0x7fdf940cc840 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf994ff700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c9d50 0x7fdf94167650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf98cfe700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940cc840 0x7fdf940b17f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000d890 tx=0x7fdf90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf857fa700 1 -- 172.21.15.45:0/364131889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000b600 con 0x7fdf940cc840 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf857fa700 1 -- 172.21.15.45:0/364131889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9000b760 con 0x7fdf940cc840 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9416b740 con 0x7fdf940cc840 2023-10-06T05:20:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf857fa700 1 -- 172.21.15.45:0/364131889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000bae0 con 0x7fdf940cc840 2023-10-06T05:20:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.152+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf94167b90 con 0x7fdf940cc840 2023-10-06T05:20:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.153+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9416bf00 con 0x7fdf940cc840 2023-10-06T05:20:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.153+0000 7fdf857fa700 1 -- 172.21.15.45:0/364131889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf9001e460 con 0x7fdf940cc840 2023-10-06T05:20:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.154+0000 7fdf857fa700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf78066f30 0x7fdf780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.154+0000 7fdf857fa700 1 -- 172.21.15.45:0/364131889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdf90084e80 con 0x7fdf940cc840 2023-10-06T05:20:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.154+0000 7fdf994ff700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf78066f30 0x7fdf780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.155+0000 7fdf994ff700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf78066f30 0x7fdf780693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf94067c70 tx=0x7fdf88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.158+0000 7fdf857fa700 1 -- 172.21.15.45:0/364131889 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf90051e50 con 0x7fdf940cc840 2023-10-06T05:20:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:55.418+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdf940cb830 con 0x7fdf940cc840 2023-10-06T05:20:56.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.257+0000 7fdf857fa700 1 -- 172.21.15.45:0/364131889 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdf90055470 con 0x7fdf940cc840 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.260+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf78066f30 msgr2=0x7fdf780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.260+0000 7fdf9b763700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf78066f30 0x7fdf780693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf94067c70 tx=0x7fdf88009040 comp rx=0 tx=0).stop 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.260+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940cc840 msgr2=0x7fdf940b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.260+0000 7fdf9b763700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940cc840 0x7fdf940b17f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000d890 tx=0x7fdf90008040 comp rx=0 tx=0).stop 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.260+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 shutdown_connections 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.261+0000 7fdf9b763700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf78066f30 0x7fdf780693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.261+0000 7fdf9b763700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9413f900 0x7fdf9416d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.261+0000 7fdf9b763700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf940c9d50 0x7fdf94167650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.261+0000 7fdf9b763700 1 --2- 172.21.15.45:0/364131889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf940cc840 0x7fdf940b17f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.261+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 >> 172.21.15.45:0/364131889 conn(0x7fdf940691c0 msgr2=0x7fdf9413e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.261+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 shutdown_connections 2023-10-06T05:20:56.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.261+0000 7fdf9b763700 1 -- 172.21.15.45:0/364131889 wait complete. 2023-10-06T05:20:56.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:20:56.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.498+0000 7f0b5b663700 1 Processor -- start 2023-10-06T05:20:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.499+0000 7f0b5b663700 1 -- start start 2023-10-06T05:20:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.499+0000 7f0b5b663700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b5413f900 0x7f0b540cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.499+0000 7f0b5b663700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c8de0 0x7f0b540c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.499+0000 7f0b5b663700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b540c9d50 0x7f0b5406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.499+0000 7f0b5b663700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540c9630 con 0x7f0b540c9d50 2023-10-06T05:20:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.499+0000 7f0b5b663700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540ca150 con 0x7f0b5413f900 2023-10-06T05:20:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.499+0000 7f0b5b663700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cb550 con 0x7f0b540c8de0 2023-10-06T05:20:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b540c9d50 0x7f0b5406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b540c9d50 0x7f0b5406c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45312/0 (socket says 172.21.15.45:45312) 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 -- 172.21.15.45:0/2147123535 learned_addr learned my addr 172.21.15.45:0/2147123535 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b58bfe700 1 --2- 172.21.15.45:0/2147123535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c8de0 0x7f0b540c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b593ff700 1 --2- 172.21.15.45:0/2147123535 >> v2:172.21.15.144:3300/0 conn(0x7f0b5413f900 0x7f0b540cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 -- 172.21.15.45:0/2147123535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c8de0 msgr2=0x7f0b540c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 --2- 172.21.15.45:0/2147123535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c8de0 0x7f0b540c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 -- 172.21.15.45:0/2147123535 >> v2:172.21.15.144:3300/0 conn(0x7f0b5413f900 msgr2=0x7f0b540cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 --2- 172.21.15.45:0/2147123535 >> v2:172.21.15.144:3300/0 conn(0x7f0b5413f900 0x7f0b540cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 -- 172.21.15.45:0/2147123535 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540cb830 con 0x7f0b540c9d50 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b59c00700 1 --2- 172.21.15.45:0/2147123535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b540c9d50 0x7f0b5406c820 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406aeb0 tx=0x7f0b4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c581e91c948ffe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b47fff700 1 -- 172.21.15.45:0/2147123535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0092e0 con 0x7f0b540c9d50 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b47fff700 1 -- 172.21.15.45:0/2147123535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c00b970 con 0x7f0b540c9d50 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.500+0000 7f0b47fff700 1 -- 172.21.15.45:0/2147123535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00bd30 con 0x7f0b540c9d50 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 -- 172.21.15.45:0/2147123535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b540c9d50 msgr2=0x7f0b5406c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 --2- 172.21.15.45:0/2147123535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b540c9d50 0x7f0b5406c820 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406aeb0 tx=0x7f0b4c009840 comp rx=0 tx=0).stop 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 -- 172.21.15.45:0/2147123535 shutdown_connections 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 --2- 172.21.15.45:0/2147123535 >> v2:172.21.15.144:3300/0 conn(0x7f0b5413f900 0x7f0b540cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 --2- 172.21.15.45:0/2147123535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b540c9d50 0x7f0b5406c820 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 --2- 172.21.15.45:0/2147123535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c8de0 0x7f0b540c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 -- 172.21.15.45:0/2147123535 >> 172.21.15.45:0/2147123535 conn(0x7f0b540691c0 msgr2=0x7f0b540cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 -- 172.21.15.45:0/2147123535 shutdown_connections 2023-10-06T05:20:56.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.501+0000 7f0b5b663700 1 -- 172.21.15.45:0/2147123535 wait complete. 2023-10-06T05:20:56.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.502+0000 7f0b5b663700 1 Processor -- start 2023-10-06T05:20:56.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.503+0000 7f0b5b663700 1 -- start start 2023-10-06T05:20:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.503+0000 7f0b5b663700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b540c8de0 0x7f0b54169250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.503+0000 7f0b5b663700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c9d50 0x7f0b54132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.503+0000 7f0b5b663700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5413f900 0x7f0b5416c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.503+0000 7f0b5b663700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54134150 con 0x7f0b5413f900 2023-10-06T05:20:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b5b663700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54169790 con 0x7f0b540c8de0 2023-10-06T05:20:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b5b663700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54133540 con 0x7f0b540c9d50 2023-10-06T05:20:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5413f900 0x7f0b5416c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5413f900 0x7f0b5416c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45320/0 (socket says 172.21.15.45:45320) 2023-10-06T05:20:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 -- 172.21.15.45:0/310429839 learned_addr learned my addr 172.21.15.45:0/310429839 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b593ff700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b540c8de0 0x7f0b54169250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b58bfe700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c9d50 0x7f0b54132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 -- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c9d50 msgr2=0x7f0b54132ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c9d50 0x7f0b54132ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 -- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b540c8de0 msgr2=0x7f0b54169250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b540c8de0 0x7f0b54169250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 -- 172.21.15.45:0/310429839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c009ae0 con 0x7f0b5413f900 2023-10-06T05:20:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b59c00700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5413f900 0x7f0b5416c810 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b610 tx=0x7f0b4c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b457fa700 1 -- 172.21.15.45:0/310429839 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c011ea0 con 0x7f0b5413f900 2023-10-06T05:20:56.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b457fa700 1 -- 172.21.15.45:0/310429839 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c0032f0 con 0x7f0b5413f900 2023-10-06T05:20:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b457fa700 1 -- 172.21.15.45:0/310429839 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c01a890 con 0x7f0b5413f900 2023-10-06T05:20:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.504+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54133820 con 0x7f0b5413f900 2023-10-06T05:20:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.505+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54133de0 con 0x7f0b5413f900 2023-10-06T05:20:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.505+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5406e2d0 con 0x7f0b5413f900 2023-10-06T05:20:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.506+0000 7f0b457fa700 1 -- 172.21.15.45:0/310429839 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b4c003040 con 0x7f0b5413f900 2023-10-06T05:20:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.506+0000 7f0b457fa700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b38066f30 0x7f0b380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.506+0000 7f0b457fa700 1 -- 172.21.15.45:0/310429839 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0b4c0862e0 con 0x7f0b5413f900 2023-10-06T05:20:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.507+0000 7f0b593ff700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b38066f30 0x7f0b380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.507+0000 7f0b593ff700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b38066f30 0x7f0b380693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009d10 tx=0x7f0b500091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.511+0000 7f0b457fa700 1 -- 172.21.15.45:0/310429839 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b4c0532b0 con 0x7f0b5413f900 2023-10-06T05:20:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:56.760+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0b540cbb10 con 0x7f0b5413f900 2023-10-06T05:20:57.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.262+0000 7f0b457fa700 1 -- 172.21.15.45:0/310429839 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b4c0568d0 con 0x7f0b5413f900 2023-10-06T05:20:57.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b38066f30 msgr2=0x7f0b380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b38066f30 0x7f0b380693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009d10 tx=0x7f0b500091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5413f900 msgr2=0x7f0b5416c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5413f900 0x7f0b5416c810 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b610 tx=0x7f0b4c01a000 comp rx=0 tx=0).stop 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 shutdown_connections 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b38066f30 0x7f0b380693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5413f900 0x7f0b5416c810 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b540c8de0 0x7f0b54169250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 --2- 172.21.15.45:0/310429839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b540c9d50 0x7f0b54132ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 >> 172.21.15.45:0/310429839 conn(0x7f0b540691c0 msgr2=0x7f0b5413eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 shutdown_connections 2023-10-06T05:20:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.265+0000 7f0b5b663700 1 -- 172.21.15.45:0/310429839 wait complete. 2023-10-06T05:20:57.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.416+0000 7f8940332700 1 Processor -- start 2023-10-06T05:20:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.416+0000 7f8940332700 1 -- start start 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f8940332700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f89380c8ff0 0x7f89380c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f8940332700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893815b000 con 0x7f8938157200 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f8940332700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893815b2e0 con 0x7f89380caad0 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f8940332700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893815b5c0 con 0x7f89380c8ff0 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f893e8cf700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8938157200 0x7f89380cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58764/0 (socket says 172.21.15.45:58764) 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f893e8cf700 1 -- 172.21.15.45:0/2784962863 learned_addr learned my addr 172.21.15.45:0/2784962863 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f893e0ce700 1 --2- 172.21.15.45:0/2784962863 >> v2:172.21.15.144:3300/0 conn(0x7f89380c8ff0 0x7f89380c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938633758 0 0) 0x7f893815b000 con 0x7f8938157200 2023-10-06T05:20:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.417+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8924003570 con 0x7f8938157200 2023-10-06T05:20:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825267169 0 0) 0x7f893815b2e0 con 0x7f89380caad0 2023-10-06T05:20:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f893815b000 con 0x7f89380caad0 2023-10-06T05:20:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2332992213 0 0) 0x7f8924003570 con 0x7f8938157200 2023-10-06T05:20:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 >> v2:172.21.15.144:3300/0 conn(0x7f89380c8ff0 msgr2=0x7f89380c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 --2- 172.21.15.45:0/2784962863 >> v2:172.21.15.144:3300/0 conn(0x7f89380c8ff0 0x7f89380c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 >> v1:172.21.15.144:6789/0 conn(0x7f89380caad0 legacy=0x7f8938156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893815b8a0 con 0x7f8938157200 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f89300017f0 con 0x7f8938157200 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 >> v1:172.21.15.45:6789/0 conn(0x7f8938157200 legacy=0x7f89380cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8924003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8924004810 0x7f8924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89240051c0 0x7f892400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893815b5c0 con 0x7f89240051c0 2023-10-06T05:20:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892400c0b0 con 0x7f8924003bf0 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892400c270 con 0x7f8924004810 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e0ce700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8924003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893d8cd700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8924004810 0x7f8924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e8cf700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89240051c0 0x7f892400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e0ce700 1 -- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8924004810 msgr2=0x7f8924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e0ce700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8924004810 0x7f8924004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e0ce700 1 -- 172.21.15.45:0/2784962863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89240051c0 msgr2=0x7f892400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e0ce700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89240051c0 0x7f892400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e0ce700 1 -- 172.21.15.45:0/2784962863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8924003570 con 0x7f8924003bf0 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e8cf700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89240051c0 0x7f892400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.418+0000 7f893e8cf700 1 -- 172.21.15.45:0/2784962863 reap_dead start 2023-10-06T05:20:57.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f893e0ce700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8924003fe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8934009c00 tx=0x7f89340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb7c45c169ce702 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400e710 con 0x7f8924003bf0 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893400e870 con 0x7f8924003bf0 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f893d0cc700 1 -- 172.21.15.45:0/2784962863 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400ec30 con 0x7f8924003bf0 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f8940332700 1 -- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 msgr2=0x7f8924003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f8940332700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8924003fe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8934009c00 tx=0x7f89340091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f8940332700 1 -- 172.21.15.45:0/2784962863 shutdown_connections 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f8940332700 1 --2- 172.21.15.45:0/2784962863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8924003fe0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.419+0000 7f8940332700 1 -- 172.21.15.45:0/2784962863 >> 172.21.15.45:0/2784962863 conn(0x7f89380cdf50 msgr2=0x7f89380ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.420+0000 7f8940332700 1 -- 172.21.15.45:0/2784962863 shutdown_connections 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.420+0000 7f8940332700 1 -- 172.21.15.45:0/2784962863 wait complete. 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 Processor -- start 2023-10-06T05:20:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 -- start start 2023-10-06T05:20:57.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8938159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89380b2690 0x7f8938144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893806d960 0x7f893806de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380b3a10 con 0x7f893806d960 2023-10-06T05:20:57.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893806e340 con 0x7f8924003bf0 2023-10-06T05:20:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.421+0000 7f8940332700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89380c7480 con 0x7f89380b2690 2023-10-06T05:20:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893806d960 0x7f893806de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893806d960 0x7f893806de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45338/0 (socket says 172.21.15.45:45338) 2023-10-06T05:20:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 -- 172.21.15.45:0/137948668 learned_addr learned my addr 172.21.15.45:0/137948668 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893d8cd700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89380b2690 0x7f8938144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e0ce700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8938159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 -- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89380b2690 msgr2=0x7f8938144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89380b2690 0x7f8938144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 -- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 msgr2=0x7f8938159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8938159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 -- 172.21.15.45:0/137948668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380b2b00 con 0x7f893806d960 2023-10-06T05:20:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893d8cd700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89380b2690 0x7f8938144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e0ce700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8938159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f893e8cf700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893806d960 0x7f893806de00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f89380cbe00 tx=0x7f8930002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f892e7fc700 1 -- 172.21.15.45:0/137948668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930007a90 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f892e7fc700 1 -- 172.21.15.45:0/137948668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300133f0 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89380b2d90 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f892e7fc700 1 -- 172.21.15.45:0/137948668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930013790 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.422+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89380b3350 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.423+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89380b3b50 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.424+0000 7f892e7fc700 1 -- 172.21.15.45:0/137948668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8930013b40 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.424+0000 7f892e7fc700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f891c066f30 0x7f891c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.424+0000 7f892e7fc700 1 -- 172.21.15.45:0/137948668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f89300852b0 con 0x7f893806d960 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.424+0000 7f893e0ce700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f891c066f30 0x7f891c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.425+0000 7f893e0ce700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f891c066f30 0x7f891c0693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f89340091e0 tx=0x7f893401a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.429+0000 7f892e7fc700 1 -- 172.21.15.45:0/137948668 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89380b3b50 con 0x7f893806d960 2023-10-06T05:20:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.674+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89380b3b50 con 0x7f893806d960 2023-10-06T05:20:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.675+0000 7f892e7fc700 1 -- 172.21.15.45:0/137948668 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f89380b3b50 con 0x7f893806d960 2023-10-06T05:20:57.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:57.677 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:20:57.257456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1530850582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1530850582},{"type":"v1","addr":"172.21.15.45:6834","nonce":1530850582}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f891c066f30 msgr2=0x7f891c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f891c066f30 0x7f891c0693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f89340091e0 tx=0x7f893401a000 comp rx=0 tx=0).stop 2023-10-06T05:20:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893806d960 msgr2=0x7f893806de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893806d960 0x7f893806de00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f89380cbe00 tx=0x7f8930002170 comp rx=0 tx=0).stop 2023-10-06T05:20:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 shutdown_connections 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f891c066f30 0x7f891c0693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893806d960 0x7f893806de00 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8924003bf0 0x7f8938159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 --2- 172.21.15.45:0/137948668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89380b2690 0x7f8938144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 >> 172.21.15.45:0/137948668 conn(0x7f89380cdf50 msgr2=0x7f8938150500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 shutdown_connections 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.677+0000 7f8940332700 1 -- 172.21.15.45:0/137948668 wait complete. 2023-10-06T05:20:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 71 2023-10-06T05:20:57.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:57.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.823+0000 7fb0d557d700 1 Processor -- start 2023-10-06T05:20:57.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.823+0000 7fb0d557d700 1 -- start start 2023-10-06T05:20:57.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0d557d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb0d00ca810 0x7fb0d00cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0d557d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c9640 0x7fb0d00b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0d557d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d00c84c0 con 0x7fb0d00c91f0 2023-10-06T05:20:57.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0d557d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d00c8d60 con 0x7fb0d00ca810 2023-10-06T05:20:57.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0d557d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0069d00 con 0x7fb0d00c9640 2023-10-06T05:20:57.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0cf7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c9640 0x7fb0d00b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0ceffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb0d00ca810 0x7fb0d00cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0cf7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c9640 0x7fb0d00b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:36738/0 (socket says 172.21.15.45:36738) 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0cf7fe700 1 -- 172.21.15.45:0/1537909851 learned_addr learned my addr 172.21.15.45:0/1537909851 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0ce7fc700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb0d00c91f0 0x7fb0d00afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:41230/0 (socket says 172.21.15.45:41230) 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0ceffd700 1 -- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c9640 msgr2=0x7fb0d00b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0ceffd700 1 --2- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c9640 0x7fb0d00b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0ceffd700 1 -- 172.21.15.45:0/1537909851 >> v1:172.21.15.144:6789/0 conn(0x7fb0d00c91f0 legacy=0x7fb0d00afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0cdffb700 1 -- 172.21.15.45:0/1537909851 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445734416 0 0) 0x7fb0d00c84c0 con 0x7fb0d00c91f0 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0ceffd700 1 -- 172.21.15.45:0/1537909851 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d00c8710 con 0x7fb0d00ca810 2023-10-06T05:20:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.824+0000 7fb0cf7fe700 1 --2- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c9640 0x7fb0d00b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0ceffd700 1 --2- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00ca810 0x7fb0d00cabe0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098e0 tx=0x7fb0c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc66bf6b89aa2604 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0cdffb700 1 -- 172.21.15.45:0/1537909851 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000b640 con 0x7fb0d00ca810 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0cdffb700 1 -- 172.21.15.45:0/1537909851 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c000b7a0 con 0x7fb0d00ca810 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0cdffb700 1 -- 172.21.15.45:0/1537909851 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000bb40 con 0x7fb0d00ca810 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0d557d700 1 -- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00ca810 msgr2=0x7fb0d00cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00ca810 0x7fb0d00cabe0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098e0 tx=0x7fb0c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0d557d700 1 -- 172.21.15.45:0/1537909851 shutdown_connections 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c9640 0x7fb0d00b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1537909851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00ca810 0x7fb0d00cabe0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0d557d700 1 -- 172.21.15.45:0/1537909851 >> 172.21.15.45:0/1537909851 conn(0x7fb0d0067620 msgr2=0x7fb0d0067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.825+0000 7fb0d557d700 1 -- 172.21.15.45:0/1537909851 shutdown_connections 2023-10-06T05:20:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.826+0000 7fb0d557d700 1 -- 172.21.15.45:0/1537909851 wait complete. 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.827+0000 7fb0d557d700 1 Processor -- start 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.827+0000 7fb0d557d700 1 -- start start 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0d557d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c91f0 0x7fb0d00c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0d557d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00c9640 0x7fb0d00c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0d557d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d00ca810 0x7fb0d00b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0d557d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00c3e00 con 0x7fb0d00ca810 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0d557d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00c31f0 con 0x7fb0d00c9640 2023-10-06T05:20:57.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0d557d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00c34d0 con 0x7fb0d00c91f0 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d00ca810 0x7fb0d00b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d00ca810 0x7fb0d00b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45348/0 (socket says 172.21.15.45:45348) 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 -- 172.21.15.45:0/1489803763 learned_addr learned my addr 172.21.15.45:0/1489803763 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0ce7fc700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00c9640 0x7fb0d00c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0ceffd700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c91f0 0x7fb0d00c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 -- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c91f0 msgr2=0x7fb0d00c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c91f0 0x7fb0d00c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 -- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00c9640 msgr2=0x7fb0d00c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00c9640 0x7fb0d00c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 -- 172.21.15.45:0/1489803763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c8009ae0 con 0x7fb0d00ca810 2023-10-06T05:20:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.828+0000 7fb0cf7fe700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d00ca810 0x7fb0d00b0cd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009fd0 tx=0x7fb0c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.829+0000 7fb0b6ffd700 1 -- 172.21.15.45:0/1489803763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8018070 con 0x7fb0d00ca810 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.829+0000 7fb0b6ffd700 1 -- 172.21.15.45:0/1489803763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c800f040 con 0x7fb0d00ca810 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.829+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c0009420 con 0x7fb0d00ca810 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.829+0000 7fb0b6ffd700 1 -- 172.21.15.45:0/1489803763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800cbd0 con 0x7fb0d00ca810 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.829+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d00c3a90 con 0x7fb0d00ca810 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.830+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d00c3fb0 con 0x7fb0d00ca810 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.830+0000 7fb0b6ffd700 1 -- 172.21.15.45:0/1489803763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0c8003020 con 0x7fb0d00ca810 2023-10-06T05:20:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.831+0000 7fb0b6ffd700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0bc066f30 0x7fb0bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.831+0000 7fb0b6ffd700 1 -- 172.21.15.45:0/1489803763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb0c8084bd0 con 0x7fb0d00ca810 2023-10-06T05:20:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.831+0000 7fb0ceffd700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0bc066f30 0x7fb0bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.831+0000 7fb0ceffd700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0bc066f30 0x7fb0bc0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00091e0 tx=0x7fb0c0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:57.835+0000 7fb0b6ffd700 1 -- 172.21.15.45:0/1489803763 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0c8051ba0 con 0x7fb0d00ca810 2023-10-06T05:20:58.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.037+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0d00c89f0 con 0x7fb0d00ca810 2023-10-06T05:20:58.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.038+0000 7fb0b6ffd700 1 -- 172.21.15.45:0/1489803763 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17170 (secure 0 0 0) 0x7fb0c80551c0 con 0x7fb0d00ca810 2023-10-06T05:20:58.040 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:58.040 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:20:55.245569+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:51.554669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:52.645101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0bc066f30 msgr2=0x7fb0bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0bc066f30 0x7fb0bc0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00091e0 tx=0x7fb0c0017000 comp rx=0 tx=0).stop 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d00ca810 msgr2=0x7fb0d00b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d00ca810 0x7fb0d00b0cd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009fd0 tx=0x7fb0c800b040 comp rx=0 tx=0).stop 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 shutdown_connections 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0bc066f30 0x7fb0bc0693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d00ca810 0x7fb0d00b0cd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0d00c91f0 0x7fb0d00c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 --2- 172.21.15.45:0/1489803763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0d00c9640 0x7fb0d00c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.040+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 >> 172.21.15.45:0/1489803763 conn(0x7fb0d0067620 msgr2=0x7fb0d0069050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.041+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 shutdown_connections 2023-10-06T05:20:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.041+0000 7fb0d557d700 1 -- 172.21.15.45:0/1489803763 wait complete. 2023-10-06T05:20:58.056 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:20:58.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:20:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.187+0000 7ff846229700 1 Processor -- start 2023-10-06T05:20:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.187+0000 7ff846229700 1 -- start start 2023-10-06T05:20:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.187+0000 7ff846229700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff840155cd0 0x7ff84015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.187+0000 7ff846229700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff8401560a0 0x7ff8400adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.187+0000 7ff846229700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff8400cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.187+0000 7ff846229700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400cd740 con 0x7ff840155cd0 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.187+0000 7ff846229700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400ae8d0 con 0x7ff8401560a0 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff846229700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400aebb0 con 0x7ff8400ccdc0 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff840155cd0 0x7ff84015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff840155cd0 0x7ff84015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45362/0 (socket says 172.21.15.45:45362) 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 -- 172.21.15.45:0/3127927815 learned_addr learned my addr 172.21.15.45:0/3127927815 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83effd700 1 --2- 172.21.15.45:0/3127927815 >> v2:172.21.15.144:3300/0 conn(0x7ff8401560a0 0x7ff8400adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83ffff700 1 --2- 172.21.15.45:0/3127927815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff8400cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 -- 172.21.15.45:0/3127927815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 msgr2=0x7ff8400cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 --2- 172.21.15.45:0/3127927815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff8400cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 -- 172.21.15.45:0/3127927815 >> v2:172.21.15.144:3300/0 conn(0x7ff8401560a0 msgr2=0x7ff8400adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 --2- 172.21.15.45:0/3127927815 >> v2:172.21.15.144:3300/0 conn(0x7ff8401560a0 0x7ff8400adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 -- 172.21.15.45:0/3127927815 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400aee90 con 0x7ff840155cd0 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83f7fe700 1 --2- 172.21.15.45:0/3127927815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff840155cd0 0x7ff84015a730 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cda20 tx=0x7ff830009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f67a3e97d01c7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83e7fc700 1 -- 172.21.15.45:0/3127927815 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83000b860 con 0x7ff840155cd0 2023-10-06T05:20:58.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.188+0000 7ff83e7fc700 1 -- 172.21.15.45:0/3127927815 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83000b9c0 con 0x7ff840155cd0 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff83e7fc700 1 -- 172.21.15.45:0/3127927815 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83000bd60 con 0x7ff840155cd0 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 -- 172.21.15.45:0/3127927815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff840155cd0 msgr2=0x7ff84015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 --2- 172.21.15.45:0/3127927815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff840155cd0 0x7ff84015a730 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cda20 tx=0x7ff830009250 comp rx=0 tx=0).stop 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 -- 172.21.15.45:0/3127927815 shutdown_connections 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 --2- 172.21.15.45:0/3127927815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff8400cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 --2- 172.21.15.45:0/3127927815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff840155cd0 0x7ff84015a730 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 --2- 172.21.15.45:0/3127927815 >> v2:172.21.15.144:3300/0 conn(0x7ff8401560a0 0x7ff8400adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 -- 172.21.15.45:0/3127927815 >> 172.21.15.45:0/3127927815 conn(0x7ff84006f090 msgr2=0x7ff84006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 -- 172.21.15.45:0/3127927815 shutdown_connections 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.189+0000 7ff846229700 1 -- 172.21.15.45:0/3127927815 wait complete. 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.190+0000 7ff846229700 1 Processor -- start 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff846229700 1 -- start start 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff846229700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff840166f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff846229700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff840155cd0 0x7ff8400a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff846229700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8401560a0 0x7ff8400a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff846229700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff840167b70 con 0x7ff8401560a0 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff846229700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400a5130 con 0x7ff840155cd0 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff846229700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400a5410 con 0x7ff8400ccdc0 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff83ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8401560a0 0x7ff8400a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff83ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8401560a0 0x7ff8400a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45366/0 (socket says 172.21.15.45:45366) 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff83f7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff840166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.191+0000 7ff83ffff700 1 -- 172.21.15.45:0/2628371840 learned_addr learned my addr 172.21.15.45:0/2628371840 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83effd700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff840155cd0 0x7ff8400a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83ffff700 1 -- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 msgr2=0x7ff840166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83ffff700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff840166f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83ffff700 1 -- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff840155cd0 msgr2=0x7ff8400a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83ffff700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff840155cd0 0x7ff8400a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83ffff700 1 -- 172.21.15.45:0/2628371840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8300094f0 con 0x7ff8401560a0 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83f7fe700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff840166f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83effd700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff840155cd0 0x7ff8400a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff83ffff700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8401560a0 0x7ff8400a5df0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff838009fa0 tx=0x7ff838009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff8277fe700 1 -- 172.21.15.45:0/2628371840 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83800ebc0 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff8277fe700 1 -- 172.21.15.45:0/2628371840 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83800ed20 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff840167800 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff8277fe700 1 -- 172.21.15.45:0/2628371840 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838014670 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.192+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8400a44b0 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.193+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84006e2f0 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.194+0000 7ff8277fe700 1 -- 172.21.15.45:0/2628371840 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8380149e0 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.194+0000 7ff8277fe700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff82c066f30 0x7ff82c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.195+0000 7ff8277fe700 1 -- 172.21.15.45:0/2628371840 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff838084f00 con 0x7ff8401560a0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.195+0000 7ff83f7fe700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff82c066f30 0x7ff82c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.195+0000 7ff83f7fe700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff82c066f30 0x7ff82c0693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cda20 tx=0x7ff830017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.199+0000 7ff8277fe700 1 -- 172.21.15.45:0/2628371840 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff838051e30 con 0x7ff8401560a0 2023-10-06T05:20:58.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.398+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff840167cb0 con 0x7ff82c066f30 2023-10-06T05:20:58.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.407+0000 7ff8277fe700 1 -- 172.21.15.45:0/2628371840 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300920 (secure 0 0 0) 0x7ff840167cb0 con 0x7ff82c066f30 2023-10-06T05:20:58.412 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff82c066f30 msgr2=0x7ff82c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff82c066f30 0x7ff82c0693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cda20 tx=0x7ff830017000 comp rx=0 tx=0).stop 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8401560a0 msgr2=0x7ff8400a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8401560a0 0x7ff8400a5df0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff838009fa0 tx=0x7ff838009900 comp rx=0 tx=0).stop 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 shutdown_connections 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff82c066f30 0x7ff82c0693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8401560a0 0x7ff8400a5df0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8400ccdc0 0x7ff840166f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 --2- 172.21.15.45:0/2628371840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff840155cd0 0x7ff8400a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 >> 172.21.15.45:0/2628371840 conn(0x7ff84006f090 msgr2=0x7ff840070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 shutdown_connections 2023-10-06T05:20:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.411+0000 7ff846229700 1 -- 172.21.15.45:0/2628371840 wait complete. 2023-10-06T05:20:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:20:58.426 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-10-06T05:20:56.535190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386640,"kb_used_data":8336,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588080,"statfs":{"total":773094113280,"available":770650193920,"internally_reserved":0,"allocated":8536064,"data_stored":4678989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.308264"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.233319+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:14.443405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.231139+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:18.092550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.231957+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:16.581696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.232966+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.111780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.231209+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:19.400261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.231771+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.111780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.232254+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:52.772545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.228949+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:15.627208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234862+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:56.661380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234109+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:50.971856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.233905+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.358350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.235243+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:14.321777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.229355+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:25.680436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.229489+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:14.348273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.235279+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:30.995555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.233125+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:14.348273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234091+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:22.532520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234641+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:12.798790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.233294+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:29.366344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234039+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:45.510798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.232752+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:19.400261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.231208+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:54.675877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234360+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:20.584847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.231132+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:37.159405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234963+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:12.798790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:50.681145+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:20:50.681145+0000","last_peered":"2023-10-06T05:20:50.681145+0000","last_clean":"2023-10-06T05:20:50.681145+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:20:50.681145+0000","last_undegraded":"2023-10-06T05:20:50.681145+0000","last_fullsized":"2023-10-06T05:20:50.681145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.232140+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:37.159405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:49.227364+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:20:49.227364+0000","last_peered":"2023-10-06T05:20:49.227364+0000","last_clean":"2023-10-06T05:20:49.227364+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:20:49.227364+0000","last_undegraded":"2023-10-06T05:20:49.227364+0000","last_fullsized":"2023-10-06T05:20:49.227364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.234332+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:43.784657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:49.773810+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:20:49.773810+0000","last_peered":"2023-10-06T05:20:49.773810+0000","last_clean":"2023-10-06T05:20:49.773810+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:20:49.773810+0000","last_undegraded":"2023-10-06T05:20:49.773810+0000","last_fullsized":"2023-10-06T05:20:49.773810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.236790+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:18.092550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:49.227318+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:20:49.227318+0000","last_peered":"2023-10-06T05:20:49.227318+0000","last_clean":"2023-10-06T05:20:49.227318+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:20:49.227318+0000","last_undegraded":"2023-10-06T05:20:49.227318+0000","last_fullsized":"2023-10-06T05:20:49.227318+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":209,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:49.228256+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:20:49.228256+0000","last_peered":"2023-10-06T05:20:49.228256+0000","last_clean":"2023-10-06T05:20:49.228256+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:20:49.228256+0000","last_undegraded":"2023-10-06T05:20:49.228256+0000","last_fullsized":"2023-10-06T05:20:49.228256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.232009+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:18.092550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:49.228265+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:20:49.228265+0000","last_peered":"2023-10-06T05:20:49.228265+0000","last_clean":"2023-10-06T05:20:49.228265+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:20:49.228265+0000","last_undegraded":"2023-10-06T05:20:49.228265+0000","last_fullsized":"2023-10-06T05:20:49.228265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.228881+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.111780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:49.228262+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:20:49.228262+0000","last_peered":"2023-10-06T05:20:49.228262+0000","last_clean":"2023-10-06T05:20:49.228262+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:20:49.228262+0000","last_undegraded":"2023-10-06T05:20:49.228262+0000","last_fullsized":"2023-10-06T05:20:49.228262+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.228825+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:54.675877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:50.681152+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:20:50.681152+0000","last_peered":"2023-10-06T05:20:50.681152+0000","last_clean":"2023-10-06T05:20:50.681152+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:20:50.681152+0000","last_undegraded":"2023-10-06T05:20:50.681152+0000","last_fullsized":"2023-10-06T05:20:50.681152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:20:53.226501+0000","last_change":"2023-10-06T05:20:53.226501+0000","last_active":"2023-10-06T05:20:53.226501+0000","last_peered":"2023-10-06T05:20:53.226501+0000","last_clean":"2023-10-06T05:20:53.226501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:53.226501+0000","last_undegraded":"2023-10-06T05:20:53.226501+0000","last_fullsized":"2023-10-06T05:20:53.226501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:20:52.221081+0000","last_change":"2023-10-06T05:20:52.235610+0000","last_active":"2023-10-06T05:20:52.221081+0000","last_peered":"2023-10-06T05:20:52.221081+0000","last_clean":"2023-10-06T05:20:52.221081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:20:52.221081+0000","last_undegraded":"2023-10-06T05:20:52.221081+0000","last_fullsized":"2023-10-06T05:20:52.221081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:18.848818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-06T05:20:51.473748+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:20:51.473748+0000","last_peered":"2023-10-06T05:20:51.473748+0000","last_clean":"2023-10-06T05:20:51.473748+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:20:51.473748+0000","last_undegraded":"2023-10-06T05:20:51.473748+0000","last_fullsized":"2023-10-06T05:20:51.473748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298764,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073076,"statfs":{"total":96636764160,"available":96330829824,"internally_reserved":0,"allocated":1511424,"data_stored":1027009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.878,"15min":0.878},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78500000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.61799999999999999},"min":{"1min":0.44400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72599999999999998,"5min":1.476,"15min":1.476},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.42799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78100000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.63},"min":{"1min":0.45900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77400000000000002,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.46800000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.78300000000000003,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.71499999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55500000000000005,"5min":0.754,"15min":0.754},"last":0.5},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.43099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.60399999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.629},{"interface":"front","average":{"1min":0.5,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72599999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73799999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":436641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.73399999999999999},"min":{"1min":0.443,"5min":0.311,"15min":0.311},"max":{"1min":0.72099999999999997,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.502,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78600000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.745}]},{"osd":1,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68400000000000005,"15min":0.94699999999999995},"min":{"1min":0.51400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.75900000000000001,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.76200000000000001},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75600000000000001,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.71599999999999997,"15min":1.004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77500000000000002,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.65600000000000003,"15min":0.79800000000000004},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73899999999999999,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.85499999999999998,"15min":1.5009999999999999},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75900000000000001,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71599999999999997,"15min":0.95399999999999996},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77400000000000002,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.58299999999999996,"15min":0.86299999999999999},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67200000000000004,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.61499999999999999,"15min":0.89100000000000001},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64400000000000002,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53700000000000003,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65700000000000003,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63600000000000001,"15min":0.89100000000000001},"min":{"1min":0.41499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77700000000000002,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.66400000000000003,"15min":0.92800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":9.718,"15min":9.718},"last":0.498},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.68500000000000005,"15min":0.95399999999999996},"min":{"1min":0.436,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77600000000000002,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":436641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.42399999999999999},"min":{"1min":0.33600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.437,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.435,"15min":0.47899999999999998},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":1.673,"15min":1.673},"last":0.34100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.498,"15min":0.57999999999999996},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.437},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.443,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52500000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.52100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.62},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66900000000000004,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93100000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55100000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90800000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90000000000000002,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.53300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.873,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45700000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.91700000000000004,"5min":1.167,"15min":1.167},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.52400000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":1.026,"15min":1.026},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.56299999999999994,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298764,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073076,"statfs":{"total":96636764160,"available":96330829824,"internally_reserved":0,"allocated":1511424,"data_stored":1027009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73199999999999998,"5min":1.038,"15min":1.038},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.433,"15min":0.42099999999999999},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60899999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.32500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.252,"15min":0.252},"max":{"1min":0.57899999999999996,"5min":3.855,"15min":3.855},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65400000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.501,"15min":0.443},"min":{"1min":0.29699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55600000000000005,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":7.7670000000000003,"15min":7.7670000000000003},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63400000000000001,"15min":0.61499999999999999},"min":{"1min":0.46300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79400000000000004,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64600000000000002,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89200000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.66900000000000004},"min":{"1min":0.53500000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80900000000000005,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70399999999999996,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":5.3600000000000003,"15min":5.3600000000000003},"last":0.505}]},{"osd":6,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.501,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82999999999999996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69699999999999995,"15min":0.70799999999999996},"min":{"1min":0.53500000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95499999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52800000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.624},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.66100000000000003},"min":{"1min":0.55200000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.873,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":436641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.432},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.59299999999999997,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42699999999999999,"15min":0.46700000000000003},"min":{"1min":0.30499999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.56200000000000006,"5min":1.899,"15min":1.899},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45900000000000002,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.624,"5min":2.097,"15min":2.097},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.48799999999999999,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66200000000000003,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.44500000000000001},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60799999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.51900000000000002},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.67600000000000005,"5min":2.121,"15min":2.121},"last":0.38700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.71099999999999997},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88200000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.53500000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92500000000000004,"5min":1.643,"15min":1.643},"last":0.73699999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.51600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94399999999999995,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95899999999999996,"5min":1.851,"15min":1.851},"last":0.879}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.51400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.875,"5min":1.571,"15min":1.571},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88700000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68899999999999995,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.90700000000000003,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97399999999999998,"5min":1.613,"15min":1.613},"last":0.81399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":435288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.28199999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40999999999999998,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.25800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.38500000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.41599999999999998},"min":{"1min":0.34000000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.42699999999999999,"15min":0.41699999999999998},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61799999999999999,"5min":0.755,"15min":0.755},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.44400000000000001,"15min":0.44500000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65600000000000003,"15min":0.63500000000000001},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.442,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84299999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.502,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78600000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.626},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.55000000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.497,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67400000000000004,"15min":0.66300000000000003},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67900000000000005,"15min":0.66300000000000003},"min":{"1min":0.48399999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.71899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":436641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.443,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.60599999999999998},"min":{"1min":0.46400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78200000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.48399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75600000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.44400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.5}]},{"osd":2,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.75,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73199999999999998,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83399999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.69799999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.36699999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.753,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.51300000000000001,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75900000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78700000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.44600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.435},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.77500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.51100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":929792,"data_stored":443119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.55700000000000005},"min":{"1min":0.45100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70499999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.55100000000000005},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71799999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75,"5min":0.876,"15min":0.876},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.49099999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78900000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57199999999999995},"min":{"1min":0.501,"5min":0.38,"15min":0.38},"max":{"1min":0.72699999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.36499999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58299999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55900000000000005,"15min":0.53500000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70099999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48699999999999999,"15min":0.46899999999999997},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.72399999999999998,"5min":0.876,"15min":0.876},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68200000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64700000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.54300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:20:58.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:20:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.600+0000 7ff9a942b700 1 Processor -- start 2023-10-06T05:20:58.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.600+0000 7ff9a942b700 1 -- start start 2023-10-06T05:20:58.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a942b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4159170 0x7ff9a4159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a942b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9a415d040 0x7ff9a4068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a942b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a40ca6d0 con 0x7ff9a4069b90 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a942b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a415d550 con 0x7ff9a415d040 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a942b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415d810 con 0x7ff9a4159170 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a27fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9a415d040 0x7ff9a4068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a27fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9a415d040 0x7ff9a4068bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45382/0 (socket says 172.21.15.45:45382) 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a27fc700 1 -- 172.21.15.45:0/162499098 learned_addr learned my addr 172.21.15.45:0/162499098 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a2ffd700 1 --2- 172.21.15.45:0/162499098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4159170 0x7ff9a4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a27fc700 1 -- 172.21.15.45:0/162499098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4159170 msgr2=0x7ff9a4159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a27fc700 1 --2- 172.21.15.45:0/162499098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4159170 0x7ff9a4159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.601+0000 7ff9a27fc700 1 -- 172.21.15.45:0/162499098 >> v1:172.21.15.144:6789/0 conn(0x7ff9a4069b90 legacy=0x7ff9a40aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.602+0000 7ff9a27fc700 1 -- 172.21.15.45:0/162499098 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a415daf0 con 0x7ff9a415d040 2023-10-06T05:20:58.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.602+0000 7ff9a27fc700 1 --2- 172.21.15.45:0/162499098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a415d040 0x7ff9a4068bc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0099b0 tx=0x7ff98c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b4cba38c7b4271 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.602+0000 7ff9a1ffb700 1 -- 172.21.15.45:0/162499098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00b860 con 0x7ff9a415d040 2023-10-06T05:20:58.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.602+0000 7ff9a1ffb700 1 -- 172.21.15.45:0/162499098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c00b9c0 con 0x7ff9a415d040 2023-10-06T05:20:58.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.602+0000 7ff9a1ffb700 1 -- 172.21.15.45:0/162499098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00bd60 con 0x7ff9a415d040 2023-10-06T05:20:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.602+0000 7ff9a942b700 1 -- 172.21.15.45:0/162499098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a415d040 msgr2=0x7ff9a4068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.602+0000 7ff9a942b700 1 --2- 172.21.15.45:0/162499098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a415d040 0x7ff9a4068bc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0099b0 tx=0x7ff98c009250 comp rx=0 tx=0).stop 2023-10-06T05:20:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.603+0000 7ff9a942b700 1 -- 172.21.15.45:0/162499098 shutdown_connections 2023-10-06T05:20:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.603+0000 7ff9a942b700 1 --2- 172.21.15.45:0/162499098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4159170 0x7ff9a4159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.603+0000 7ff9a942b700 1 --2- 172.21.15.45:0/162499098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a415d040 0x7ff9a4068bc0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.603+0000 7ff9a942b700 1 -- 172.21.15.45:0/162499098 >> 172.21.15.45:0/162499098 conn(0x7ff9a406f7c0 msgr2=0x7ff9a40d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.603+0000 7ff9a942b700 1 -- 172.21.15.45:0/162499098 shutdown_connections 2023-10-06T05:20:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.603+0000 7ff9a942b700 1 -- 172.21.15.45:0/162499098 wait complete. 2023-10-06T05:20:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.604+0000 7ff9a942b700 1 Processor -- start 2023-10-06T05:20:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.604+0000 7ff9a942b700 1 -- start start 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.605+0000 7ff9a942b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4069b90 0x7ff9a415e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.605+0000 7ff9a942b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a4159170 0x7ff9a4155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.605+0000 7ff9a942b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a415d040 0x7ff9a41481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.605+0000 7ff9a942b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4159b30 con 0x7ff9a4159170 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.605+0000 7ff9a942b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41567b0 con 0x7ff9a415d040 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.605+0000 7ff9a942b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4156a90 con 0x7ff9a4069b90 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a4159170 0x7ff9a4155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a4159170 0x7ff9a4155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45394/0 (socket says 172.21.15.45:45394) 2023-10-06T05:20:58.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 -- 172.21.15.45:0/1695398914 learned_addr learned my addr 172.21.15.45:0/1695398914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a37fe700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a415d040 0x7ff9a41481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a2ffd700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4069b90 0x7ff9a415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 -- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4069b90 msgr2=0x7ff9a415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4069b90 0x7ff9a415e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 -- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a415d040 msgr2=0x7ff9a41481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a415d040 0x7ff9a41481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 -- 172.21.15.45:0/1695398914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c0094f0 con 0x7ff9a4159170 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a2ffd700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4069b90 0x7ff9a415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a27fc700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a4159170 0x7ff9a4155d80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4088b80 tx=0x7ff98c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff99affd700 1 -- 172.21.15.45:0/1695398914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c024440 con 0x7ff9a4159170 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff99affd700 1 -- 172.21.15.45:0/1695398914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c0245c0 con 0x7ff9a4159170 2023-10-06T05:20:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a4156d70 con 0x7ff9a4159170 2023-10-06T05:20:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.606+0000 7ff99affd700 1 -- 172.21.15.45:0/1695398914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c024980 con 0x7ff9a4159170 2023-10-06T05:20:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.607+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a4155a20 con 0x7ff9a4159170 2023-10-06T05:20:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.608+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4159ca0 con 0x7ff9a4159170 2023-10-06T05:20:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.608+0000 7ff99affd700 1 -- 172.21.15.45:0/1695398914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff98c01a420 con 0x7ff9a4159170 2023-10-06T05:20:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.608+0000 7ff99affd700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff984066f30 0x7ff9840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.609+0000 7ff99affd700 1 -- 172.21.15.45:0/1695398914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff98c0865c0 con 0x7ff9a4159170 2023-10-06T05:20:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.609+0000 7ff9a2ffd700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff984066f30 0x7ff9840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.609+0000 7ff9a2ffd700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff984066f30 0x7ff9840693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff994006fd0 tx=0x7ff994008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.613+0000 7ff99affd700 1 -- 172.21.15.45:0/1695398914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff98c0534f0 con 0x7ff9a4159170 2023-10-06T05:20:58.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.809+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9a4063b50 con 0x7ff984066f30 2023-10-06T05:20:58.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.818+0000 7ff99affd700 1 -- 172.21.15.45:0/1695398914 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311578 (secure 0 0 0) 0x7ff9a4063b50 con 0x7ff984066f30 2023-10-06T05:20:58.823 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff984066f30 msgr2=0x7ff9840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff984066f30 0x7ff9840693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff994006fd0 tx=0x7ff994008040 comp rx=0 tx=0).stop 2023-10-06T05:20:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a4159170 msgr2=0x7ff9a4155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a4159170 0x7ff9a4155d80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4088b80 tx=0x7ff98c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 shutdown_connections 2023-10-06T05:20:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff984066f30 0x7ff9840693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9a415d040 0x7ff9a41481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9a4069b90 0x7ff9a415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 --2- 172.21.15.45:0/1695398914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a4159170 0x7ff9a4155d80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 >> 172.21.15.45:0/1695398914 conn(0x7ff9a406f7c0 msgr2=0x7ff9a41586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 shutdown_connections 2023-10-06T05:20:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:58.822+0000 7ff9a942b700 1 -- 172.21.15.45:0/1695398914 wait complete. 2023-10-06T05:20:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:20:58.837 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2023-10-06T05:20:58.535597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":193,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2387052,"kb_used_data":8748,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752587668,"statfs":{"total":773094113280,"available":770649772032,"internally_reserved":0,"allocated":8957952,"data_stored":4885689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001435"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.260846+0000","last_change":"2023-10-06T05:20:54.257088+0000","last_active":"2023-10-06T05:20:54.260846+0000","last_peered":"2023-10-06T05:20:54.260846+0000","last_clean":"2023-10-06T05:20:54.260846+0000","last_became_active":"2023-10-06T05:20:54.256744+0000","last_became_peered":"2023-10-06T05:20:54.256744+0000","last_unstale":"2023-10-06T05:20:54.260846+0000","last_undegraded":"2023-10-06T05:20:54.260846+0000","last_fullsized":"2023-10-06T05:20:54.260846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.200388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274722+0000","last_change":"2023-10-06T05:20:54.271158+0000","last_active":"2023-10-06T05:20:54.274722+0000","last_peered":"2023-10-06T05:20:54.274722+0000","last_clean":"2023-10-06T05:20:54.274722+0000","last_became_active":"2023-10-06T05:20:54.270654+0000","last_became_peered":"2023-10-06T05:20:54.270654+0000","last_unstale":"2023-10-06T05:20:54.274722+0000","last_undegraded":"2023-10-06T05:20:54.274722+0000","last_fullsized":"2023-10-06T05:20:54.274722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:00.237398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.258047+0000","last_change":"2023-10-06T05:20:54.254135+0000","last_active":"2023-10-06T05:20:54.258047+0000","last_peered":"2023-10-06T05:20:54.258047+0000","last_clean":"2023-10-06T05:20:54.258047+0000","last_became_active":"2023-10-06T05:20:54.253894+0000","last_became_peered":"2023-10-06T05:20:54.253894+0000","last_unstale":"2023-10-06T05:20:54.258047+0000","last_undegraded":"2023-10-06T05:20:54.258047+0000","last_fullsized":"2023-10-06T05:20:54.258047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:25.312380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.281022+0000","last_change":"2023-10-06T05:20:54.279241+0000","last_active":"2023-10-06T05:20:54.281022+0000","last_peered":"2023-10-06T05:20:54.281022+0000","last_clean":"2023-10-06T05:20:54.281022+0000","last_became_active":"2023-10-06T05:20:54.278961+0000","last_became_peered":"2023-10-06T05:20:54.278961+0000","last_unstale":"2023-10-06T05:20:54.281022+0000","last_undegraded":"2023-10-06T05:20:54.281022+0000","last_fullsized":"2023-10-06T05:20:54.281022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:57.666799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.269603+0000","last_change":"2023-10-06T05:20:54.266381+0000","last_active":"2023-10-06T05:20:54.269603+0000","last_peered":"2023-10-06T05:20:54.269603+0000","last_clean":"2023-10-06T05:20:54.269603+0000","last_became_active":"2023-10-06T05:20:54.266108+0000","last_became_peered":"2023-10-06T05:20:54.266108+0000","last_unstale":"2023-10-06T05:20:54.269603+0000","last_undegraded":"2023-10-06T05:20:54.269603+0000","last_fullsized":"2023-10-06T05:20:54.269603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:22.971407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.271923+0000","last_change":"2023-10-06T05:20:54.266343+0000","last_active":"2023-10-06T05:20:54.271923+0000","last_peered":"2023-10-06T05:20:54.271923+0000","last_clean":"2023-10-06T05:20:54.271923+0000","last_became_active":"2023-10-06T05:20:54.265967+0000","last_became_peered":"2023-10-06T05:20:54.265967+0000","last_unstale":"2023-10-06T05:20:54.271923+0000","last_undegraded":"2023-10-06T05:20:54.271923+0000","last_fullsized":"2023-10-06T05:20:54.271923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:19.789660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273122+0000","last_change":"2023-10-06T05:20:54.271282+0000","last_active":"2023-10-06T05:20:54.273122+0000","last_peered":"2023-10-06T05:20:54.273122+0000","last_clean":"2023-10-06T05:20:54.273122+0000","last_became_active":"2023-10-06T05:20:54.271019+0000","last_became_peered":"2023-10-06T05:20:54.271019+0000","last_unstale":"2023-10-06T05:20:54.273122+0000","last_undegraded":"2023-10-06T05:20:54.273122+0000","last_fullsized":"2023-10-06T05:20:54.273122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:43.462013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.272044+0000","last_change":"2023-10-06T05:20:54.266955+0000","last_active":"2023-10-06T05:20:54.272044+0000","last_peered":"2023-10-06T05:20:54.272044+0000","last_clean":"2023-10-06T05:20:54.272044+0000","last_became_active":"2023-10-06T05:20:54.266409+0000","last_became_peered":"2023-10-06T05:20:54.266409+0000","last_unstale":"2023-10-06T05:20:54.272044+0000","last_undegraded":"2023-10-06T05:20:54.272044+0000","last_fullsized":"2023-10-06T05:20:54.272044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:41.141711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.271871+0000","last_change":"2023-10-06T05:20:54.265011+0000","last_active":"2023-10-06T05:20:54.271871+0000","last_peered":"2023-10-06T05:20:54.271871+0000","last_clean":"2023-10-06T05:20:54.271871+0000","last_became_active":"2023-10-06T05:20:54.264740+0000","last_became_peered":"2023-10-06T05:20:54.264740+0000","last_unstale":"2023-10-06T05:20:54.271871+0000","last_undegraded":"2023-10-06T05:20:54.271871+0000","last_fullsized":"2023-10-06T05:20:54.271871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:52.699353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.266037+0000","last_change":"2023-10-06T05:20:54.260135+0000","last_active":"2023-10-06T05:20:54.266037+0000","last_peered":"2023-10-06T05:20:54.266037+0000","last_clean":"2023-10-06T05:20:54.266037+0000","last_became_active":"2023-10-06T05:20:54.259739+0000","last_became_peered":"2023-10-06T05:20:54.259739+0000","last_unstale":"2023-10-06T05:20:54.266037+0000","last_undegraded":"2023-10-06T05:20:54.266037+0000","last_fullsized":"2023-10-06T05:20:54.266037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:32.705266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274024+0000","last_change":"2023-10-06T05:20:54.265259+0000","last_active":"2023-10-06T05:20:54.274024+0000","last_peered":"2023-10-06T05:20:54.274024+0000","last_clean":"2023-10-06T05:20:54.274024+0000","last_became_active":"2023-10-06T05:20:54.263625+0000","last_became_peered":"2023-10-06T05:20:54.263625+0000","last_unstale":"2023-10-06T05:20:54.274024+0000","last_undegraded":"2023-10-06T05:20:54.274024+0000","last_fullsized":"2023-10-06T05:20:54.274024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:21.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.266271+0000","last_change":"2023-10-06T05:20:54.259969+0000","last_active":"2023-10-06T05:20:54.266271+0000","last_peered":"2023-10-06T05:20:54.266271+0000","last_clean":"2023-10-06T05:20:54.266271+0000","last_became_active":"2023-10-06T05:20:54.259685+0000","last_became_peered":"2023-10-06T05:20:54.259685+0000","last_unstale":"2023-10-06T05:20:54.266271+0000","last_undegraded":"2023-10-06T05:20:54.266271+0000","last_fullsized":"2023-10-06T05:20:54.266271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:42.011964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.259416+0000","last_change":"2023-10-06T05:20:54.257081+0000","last_active":"2023-10-06T05:20:54.259416+0000","last_peered":"2023-10-06T05:20:54.259416+0000","last_clean":"2023-10-06T05:20:54.259416+0000","last_became_active":"2023-10-06T05:20:54.256829+0000","last_became_peered":"2023-10-06T05:20:54.256829+0000","last_unstale":"2023-10-06T05:20:54.259416+0000","last_undegraded":"2023-10-06T05:20:54.259416+0000","last_fullsized":"2023-10-06T05:20:54.259416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:44.992062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.280958+0000","last_change":"2023-10-06T05:20:54.278576+0000","last_active":"2023-10-06T05:20:54.280958+0000","last_peered":"2023-10-06T05:20:54.280958+0000","last_clean":"2023-10-06T05:20:54.280958+0000","last_became_active":"2023-10-06T05:20:54.278247+0000","last_became_peered":"2023-10-06T05:20:54.278247+0000","last_unstale":"2023-10-06T05:20:54.280958+0000","last_undegraded":"2023-10-06T05:20:54.280958+0000","last_fullsized":"2023-10-06T05:20:54.280958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:22.077853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.259278+0000","last_change":"2023-10-06T05:20:54.256556+0000","last_active":"2023-10-06T05:20:54.259278+0000","last_peered":"2023-10-06T05:20:54.259278+0000","last_clean":"2023-10-06T05:20:54.259278+0000","last_became_active":"2023-10-06T05:20:54.256247+0000","last_became_peered":"2023-10-06T05:20:54.256247+0000","last_unstale":"2023-10-06T05:20:54.259278+0000","last_undegraded":"2023-10-06T05:20:54.259278+0000","last_fullsized":"2023-10-06T05:20:54.259278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:31.456021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.259784+0000","last_change":"2023-10-06T05:20:54.254473+0000","last_active":"2023-10-06T05:20:54.259784+0000","last_peered":"2023-10-06T05:20:54.259784+0000","last_clean":"2023-10-06T05:20:54.259784+0000","last_became_active":"2023-10-06T05:20:54.254218+0000","last_became_peered":"2023-10-06T05:20:54.254218+0000","last_unstale":"2023-10-06T05:20:54.259784+0000","last_undegraded":"2023-10-06T05:20:54.259784+0000","last_fullsized":"2023-10-06T05:20:54.259784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:02.512613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.271985+0000","last_change":"2023-10-06T05:20:54.267555+0000","last_active":"2023-10-06T05:20:54.271985+0000","last_peered":"2023-10-06T05:20:54.271985+0000","last_clean":"2023-10-06T05:20:54.271985+0000","last_became_active":"2023-10-06T05:20:54.267251+0000","last_became_peered":"2023-10-06T05:20:54.267251+0000","last_unstale":"2023-10-06T05:20:54.271985+0000","last_undegraded":"2023-10-06T05:20:54.271985+0000","last_fullsized":"2023-10-06T05:20:54.271985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:08.577704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.272119+0000","last_change":"2023-10-06T05:20:54.267215+0000","last_active":"2023-10-06T05:20:54.272119+0000","last_peered":"2023-10-06T05:20:54.272119+0000","last_clean":"2023-10-06T05:20:54.272119+0000","last_became_active":"2023-10-06T05:20:54.266900+0000","last_became_peered":"2023-10-06T05:20:54.266900+0000","last_unstale":"2023-10-06T05:20:54.272119+0000","last_undegraded":"2023-10-06T05:20:54.272119+0000","last_fullsized":"2023-10-06T05:20:54.272119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:50.163499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.267996+0000","last_change":"2023-10-06T05:20:54.259932+0000","last_active":"2023-10-06T05:20:54.267996+0000","last_peered":"2023-10-06T05:20:54.267996+0000","last_clean":"2023-10-06T05:20:54.267996+0000","last_became_active":"2023-10-06T05:20:54.259616+0000","last_became_peered":"2023-10-06T05:20:54.259616+0000","last_unstale":"2023-10-06T05:20:54.267996+0000","last_undegraded":"2023-10-06T05:20:54.267996+0000","last_fullsized":"2023-10-06T05:20:54.267996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:25.312380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.281039+0000","last_change":"2023-10-06T05:20:54.279274+0000","last_active":"2023-10-06T05:20:54.281039+0000","last_peered":"2023-10-06T05:20:54.281039+0000","last_clean":"2023-10-06T05:20:54.281039+0000","last_became_active":"2023-10-06T05:20:54.278625+0000","last_became_peered":"2023-10-06T05:20:54.278625+0000","last_unstale":"2023-10-06T05:20:54.281039+0000","last_undegraded":"2023-10-06T05:20:54.281039+0000","last_fullsized":"2023-10-06T05:20:54.281039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.350590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.280996+0000","last_change":"2023-10-06T05:20:54.279164+0000","last_active":"2023-10-06T05:20:54.280996+0000","last_peered":"2023-10-06T05:20:54.280996+0000","last_clean":"2023-10-06T05:20:54.280996+0000","last_became_active":"2023-10-06T05:20:54.278689+0000","last_became_peered":"2023-10-06T05:20:54.278689+0000","last_unstale":"2023-10-06T05:20:54.280996+0000","last_undegraded":"2023-10-06T05:20:54.280996+0000","last_fullsized":"2023-10-06T05:20:54.280996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:12.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.276470+0000","last_change":"2023-10-06T05:20:54.272956+0000","last_active":"2023-10-06T05:20:54.276470+0000","last_peered":"2023-10-06T05:20:54.276470+0000","last_clean":"2023-10-06T05:20:54.276470+0000","last_became_active":"2023-10-06T05:20:54.272191+0000","last_became_peered":"2023-10-06T05:20:54.272191+0000","last_unstale":"2023-10-06T05:20:54.276470+0000","last_undegraded":"2023-10-06T05:20:54.276470+0000","last_fullsized":"2023-10-06T05:20:54.276470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:15.327197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274734+0000","last_change":"2023-10-06T05:20:54.270406+0000","last_active":"2023-10-06T05:20:54.274734+0000","last_peered":"2023-10-06T05:20:54.274734+0000","last_clean":"2023-10-06T05:20:54.274734+0000","last_became_active":"2023-10-06T05:20:54.270142+0000","last_became_peered":"2023-10-06T05:20:54.270142+0000","last_unstale":"2023-10-06T05:20:54.274734+0000","last_undegraded":"2023-10-06T05:20:54.274734+0000","last_fullsized":"2023-10-06T05:20:54.274734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:52.699353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.281082+0000","last_change":"2023-10-06T05:20:54.280178+0000","last_active":"2023-10-06T05:20:54.281082+0000","last_peered":"2023-10-06T05:20:54.281082+0000","last_clean":"2023-10-06T05:20:54.281082+0000","last_became_active":"2023-10-06T05:20:54.278799+0000","last_became_peered":"2023-10-06T05:20:54.278799+0000","last_unstale":"2023-10-06T05:20:54.281082+0000","last_undegraded":"2023-10-06T05:20:54.281082+0000","last_fullsized":"2023-10-06T05:20:54.281082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:47.082860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274592+0000","last_change":"2023-10-06T05:20:54.270099+0000","last_active":"2023-10-06T05:20:54.274592+0000","last_peered":"2023-10-06T05:20:54.274592+0000","last_clean":"2023-10-06T05:20:54.274592+0000","last_became_active":"2023-10-06T05:20:54.269809+0000","last_became_peered":"2023-10-06T05:20:54.269809+0000","last_unstale":"2023-10-06T05:20:54.274592+0000","last_undegraded":"2023-10-06T05:20:54.274592+0000","last_fullsized":"2023-10-06T05:20:54.274592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.350590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273266+0000","last_change":"2023-10-06T05:20:54.265719+0000","last_active":"2023-10-06T05:20:54.273266+0000","last_peered":"2023-10-06T05:20:54.273266+0000","last_clean":"2023-10-06T05:20:54.273266+0000","last_became_active":"2023-10-06T05:20:54.265454+0000","last_became_peered":"2023-10-06T05:20:54.265454+0000","last_unstale":"2023-10-06T05:20:54.273266+0000","last_undegraded":"2023-10-06T05:20:54.273266+0000","last_fullsized":"2023-10-06T05:20:54.273266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:35.301660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.261588+0000","last_change":"2023-10-06T05:20:54.256599+0000","last_active":"2023-10-06T05:20:54.261588+0000","last_peered":"2023-10-06T05:20:54.261588+0000","last_clean":"2023-10-06T05:20:54.261588+0000","last_became_active":"2023-10-06T05:20:54.256310+0000","last_became_peered":"2023-10-06T05:20:54.256310+0000","last_unstale":"2023-10-06T05:20:54.261588+0000","last_undegraded":"2023-10-06T05:20:54.261588+0000","last_fullsized":"2023-10-06T05:20:54.261588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:05.647681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.269329+0000","last_change":"2023-10-06T05:20:54.261028+0000","last_active":"2023-10-06T05:20:54.269329+0000","last_peered":"2023-10-06T05:20:54.269329+0000","last_clean":"2023-10-06T05:20:54.269329+0000","last_became_active":"2023-10-06T05:20:54.260771+0000","last_became_peered":"2023-10-06T05:20:54.260771+0000","last_unstale":"2023-10-06T05:20:54.269329+0000","last_undegraded":"2023-10-06T05:20:54.269329+0000","last_fullsized":"2023-10-06T05:20:54.269329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:41.141711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273092+0000","last_change":"2023-10-06T05:20:54.264367+0000","last_active":"2023-10-06T05:20:54.273092+0000","last_peered":"2023-10-06T05:20:54.273092+0000","last_clean":"2023-10-06T05:20:54.273092+0000","last_became_active":"2023-10-06T05:20:54.262463+0000","last_became_peered":"2023-10-06T05:20:54.262463+0000","last_unstale":"2023-10-06T05:20:54.273092+0000","last_undegraded":"2023-10-06T05:20:54.273092+0000","last_fullsized":"2023-10-06T05:20:54.273092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:50.163499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.275983+0000","last_change":"2023-10-06T05:20:54.272497+0000","last_active":"2023-10-06T05:20:54.275983+0000","last_peered":"2023-10-06T05:20:54.275983+0000","last_clean":"2023-10-06T05:20:54.275983+0000","last_became_active":"2023-10-06T05:20:54.271418+0000","last_became_peered":"2023-10-06T05:20:54.271418+0000","last_unstale":"2023-10-06T05:20:54.275983+0000","last_undegraded":"2023-10-06T05:20:54.275983+0000","last_fullsized":"2023-10-06T05:20:54.275983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:43.462013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.277611+0000","last_change":"2023-10-06T05:20:54.272843+0000","last_active":"2023-10-06T05:20:54.277611+0000","last_peered":"2023-10-06T05:20:54.277611+0000","last_clean":"2023-10-06T05:20:54.277611+0000","last_became_active":"2023-10-06T05:20:54.272284+0000","last_became_peered":"2023-10-06T05:20:54.272284+0000","last_unstale":"2023-10-06T05:20:54.277611+0000","last_undegraded":"2023-10-06T05:20:54.277611+0000","last_fullsized":"2023-10-06T05:20:54.277611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:26.238410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.278062+0000","last_change":"2023-10-06T05:20:54.274437+0000","last_active":"2023-10-06T05:20:54.278062+0000","last_peered":"2023-10-06T05:20:54.278062+0000","last_clean":"2023-10-06T05:20:54.278062+0000","last_became_active":"2023-10-06T05:20:54.274152+0000","last_became_peered":"2023-10-06T05:20:54.274152+0000","last_unstale":"2023-10-06T05:20:54.278062+0000","last_undegraded":"2023-10-06T05:20:54.278062+0000","last_fullsized":"2023-10-06T05:20:54.278062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:20.405681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.261524+0000","last_change":"2023-10-06T05:20:54.256350+0000","last_active":"2023-10-06T05:20:54.261524+0000","last_peered":"2023-10-06T05:20:54.261524+0000","last_clean":"2023-10-06T05:20:54.261524+0000","last_became_active":"2023-10-06T05:20:54.256049+0000","last_became_peered":"2023-10-06T05:20:54.256049+0000","last_unstale":"2023-10-06T05:20:54.261524+0000","last_undegraded":"2023-10-06T05:20:54.261524+0000","last_fullsized":"2023-10-06T05:20:54.261524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:41.287066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"78'4","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273408+0000","last_change":"2023-10-06T05:20:53.263719+0000","last_active":"2023-10-06T05:20:54.273408+0000","last_peered":"2023-10-06T05:20:54.273408+0000","last_clean":"2023-10-06T05:20:54.273408+0000","last_became_active":"2023-10-06T05:20:53.263441+0000","last_became_peered":"2023-10-06T05:20:53.263441+0000","last_unstale":"2023-10-06T05:20:54.273408+0000","last_undegraded":"2023-10-06T05:20:54.273408+0000","last_fullsized":"2023-10-06T05:20:54.273408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:59.231978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274774+0000","last_change":"2023-10-06T05:20:54.270409+0000","last_active":"2023-10-06T05:20:54.274774+0000","last_peered":"2023-10-06T05:20:54.274774+0000","last_clean":"2023-10-06T05:20:54.274774+0000","last_became_active":"2023-10-06T05:20:54.270106+0000","last_became_peered":"2023-10-06T05:20:54.270106+0000","last_unstale":"2023-10-06T05:20:54.274774+0000","last_undegraded":"2023-10-06T05:20:54.274774+0000","last_fullsized":"2023-10-06T05:20:54.274774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:04.153012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.256126+0000","last_change":"2023-10-06T05:20:53.262409+0000","last_active":"2023-10-06T05:20:54.256126+0000","last_peered":"2023-10-06T05:20:54.256126+0000","last_clean":"2023-10-06T05:20:54.256126+0000","last_became_active":"2023-10-06T05:20:53.262164+0000","last_became_peered":"2023-10-06T05:20:53.262164+0000","last_unstale":"2023-10-06T05:20:54.256126+0000","last_undegraded":"2023-10-06T05:20:54.256126+0000","last_fullsized":"2023-10-06T05:20:54.256126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:14.548651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.278053+0000","last_change":"2023-10-06T05:20:54.274399+0000","last_active":"2023-10-06T05:20:54.278053+0000","last_peered":"2023-10-06T05:20:54.278053+0000","last_clean":"2023-10-06T05:20:54.278053+0000","last_became_active":"2023-10-06T05:20:54.274118+0000","last_became_peered":"2023-10-06T05:20:54.274118+0000","last_unstale":"2023-10-06T05:20:54.278053+0000","last_undegraded":"2023-10-06T05:20:54.278053+0000","last_fullsized":"2023-10-06T05:20:54.278053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:20.625207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.270765+0000","last_change":"2023-10-06T05:20:53.256149+0000","last_active":"2023-10-06T05:20:54.270765+0000","last_peered":"2023-10-06T05:20:54.270765+0000","last_clean":"2023-10-06T05:20:54.270765+0000","last_became_active":"2023-10-06T05:20:53.255784+0000","last_became_peered":"2023-10-06T05:20:53.255784+0000","last_unstale":"2023-10-06T05:20:54.270765+0000","last_undegraded":"2023-10-06T05:20:54.270765+0000","last_fullsized":"2023-10-06T05:20:54.270765+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:14.548651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273291+0000","last_change":"2023-10-06T05:20:54.266972+0000","last_active":"2023-10-06T05:20:54.273291+0000","last_peered":"2023-10-06T05:20:54.273291+0000","last_clean":"2023-10-06T05:20:54.273291+0000","last_became_active":"2023-10-06T05:20:54.266711+0000","last_became_peered":"2023-10-06T05:20:54.266711+0000","last_unstale":"2023-10-06T05:20:54.273291+0000","last_undegraded":"2023-10-06T05:20:54.273291+0000","last_fullsized":"2023-10-06T05:20:54.273291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:31.456021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.270618+0000","last_change":"2023-10-06T05:20:53.257993+0000","last_active":"2023-10-06T05:20:54.270618+0000","last_peered":"2023-10-06T05:20:54.270618+0000","last_clean":"2023-10-06T05:20:54.270618+0000","last_became_active":"2023-10-06T05:20:53.255581+0000","last_became_peered":"2023-10-06T05:20:53.255581+0000","last_unstale":"2023-10-06T05:20:54.270618+0000","last_undegraded":"2023-10-06T05:20:54.270618+0000","last_fullsized":"2023-10-06T05:20:54.270618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:19.619788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.276004+0000","last_change":"2023-10-06T05:20:54.272740+0000","last_active":"2023-10-06T05:20:54.276004+0000","last_peered":"2023-10-06T05:20:54.276004+0000","last_clean":"2023-10-06T05:20:54.276004+0000","last_became_active":"2023-10-06T05:20:54.271780+0000","last_became_peered":"2023-10-06T05:20:54.271780+0000","last_unstale":"2023-10-06T05:20:54.276004+0000","last_undegraded":"2023-10-06T05:20:54.276004+0000","last_fullsized":"2023-10-06T05:20:54.276004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:12.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.265873+0000","last_change":"2023-10-06T05:20:53.255214+0000","last_active":"2023-10-06T05:20:54.265873+0000","last_peered":"2023-10-06T05:20:54.265873+0000","last_clean":"2023-10-06T05:20:54.265873+0000","last_became_active":"2023-10-06T05:20:53.254939+0000","last_became_peered":"2023-10-06T05:20:53.254939+0000","last_unstale":"2023-10-06T05:20:54.265873+0000","last_undegraded":"2023-10-06T05:20:54.265873+0000","last_fullsized":"2023-10-06T05:20:54.265873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:14.348273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.272011+0000","last_change":"2023-10-06T05:20:54.267635+0000","last_active":"2023-10-06T05:20:54.272011+0000","last_peered":"2023-10-06T05:20:54.272011+0000","last_clean":"2023-10-06T05:20:54.272011+0000","last_became_active":"2023-10-06T05:20:54.267287+0000","last_became_peered":"2023-10-06T05:20:54.267287+0000","last_unstale":"2023-10-06T05:20:54.272011+0000","last_undegraded":"2023-10-06T05:20:54.272011+0000","last_fullsized":"2023-10-06T05:20:54.272011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:52.002939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.256073+0000","last_change":"2023-10-06T05:20:53.256664+0000","last_active":"2023-10-06T05:20:54.256073+0000","last_peered":"2023-10-06T05:20:54.256073+0000","last_clean":"2023-10-06T05:20:54.256073+0000","last_became_active":"2023-10-06T05:20:53.256424+0000","last_became_peered":"2023-10-06T05:20:53.256424+0000","last_unstale":"2023-10-06T05:20:54.256073+0000","last_undegraded":"2023-10-06T05:20:54.256073+0000","last_fullsized":"2023-10-06T05:20:54.256073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:18.784241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.278008+0000","last_change":"2023-10-06T05:20:54.272927+0000","last_active":"2023-10-06T05:20:54.278008+0000","last_peered":"2023-10-06T05:20:54.278008+0000","last_clean":"2023-10-06T05:20:54.278008+0000","last_became_active":"2023-10-06T05:20:54.272504+0000","last_became_peered":"2023-10-06T05:20:54.272504+0000","last_unstale":"2023-10-06T05:20:54.278008+0000","last_undegraded":"2023-10-06T05:20:54.278008+0000","last_fullsized":"2023-10-06T05:20:54.278008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:04.153012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.246664+0000","last_change":"2023-10-06T05:20:53.255193+0000","last_active":"2023-10-06T05:20:54.246664+0000","last_peered":"2023-10-06T05:20:54.246664+0000","last_clean":"2023-10-06T05:20:54.246664+0000","last_became_active":"2023-10-06T05:20:53.254849+0000","last_became_peered":"2023-10-06T05:20:53.254849+0000","last_unstale":"2023-10-06T05:20:54.246664+0000","last_undegraded":"2023-10-06T05:20:54.246664+0000","last_fullsized":"2023-10-06T05:20:54.246664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:18.784241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.257612+0000","last_change":"2023-10-06T05:20:54.255444+0000","last_active":"2023-10-06T05:20:54.257612+0000","last_peered":"2023-10-06T05:20:54.257612+0000","last_clean":"2023-10-06T05:20:54.257612+0000","last_became_active":"2023-10-06T05:20:54.255184+0000","last_became_peered":"2023-10-06T05:20:54.255184+0000","last_unstale":"2023-10-06T05:20:54.257612+0000","last_undegraded":"2023-10-06T05:20:54.257612+0000","last_fullsized":"2023-10-06T05:20:54.257612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:35.301660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.249594+0000","last_change":"2023-10-06T05:20:53.256614+0000","last_active":"2023-10-06T05:20:54.249594+0000","last_peered":"2023-10-06T05:20:54.249594+0000","last_clean":"2023-10-06T05:20:54.249594+0000","last_became_active":"2023-10-06T05:20:53.256326+0000","last_became_peered":"2023-10-06T05:20:53.256326+0000","last_unstale":"2023-10-06T05:20:54.249594+0000","last_undegraded":"2023-10-06T05:20:54.249594+0000","last_fullsized":"2023-10-06T05:20:54.249594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:37.807223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.275101+0000","last_change":"2023-10-06T05:20:54.271095+0000","last_active":"2023-10-06T05:20:54.275101+0000","last_peered":"2023-10-06T05:20:54.275101+0000","last_clean":"2023-10-06T05:20:54.275101+0000","last_became_active":"2023-10-06T05:20:54.270226+0000","last_became_peered":"2023-10-06T05:20:54.270226+0000","last_unstale":"2023-10-06T05:20:54.275101+0000","last_undegraded":"2023-10-06T05:20:54.275101+0000","last_fullsized":"2023-10-06T05:20:54.275101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:38.836602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.253002+0000","last_change":"2023-10-06T05:20:53.262567+0000","last_active":"2023-10-06T05:20:54.253002+0000","last_peered":"2023-10-06T05:20:54.253002+0000","last_clean":"2023-10-06T05:20:54.253002+0000","last_became_active":"2023-10-06T05:20:53.262290+0000","last_became_peered":"2023-10-06T05:20:53.262290+0000","last_unstale":"2023-10-06T05:20:54.253002+0000","last_undegraded":"2023-10-06T05:20:54.253002+0000","last_fullsized":"2023-10-06T05:20:54.253002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:00.345171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274687+0000","last_change":"2023-10-06T05:20:54.268985+0000","last_active":"2023-10-06T05:20:54.274687+0000","last_peered":"2023-10-06T05:20:54.274687+0000","last_clean":"2023-10-06T05:20:54.274687+0000","last_became_active":"2023-10-06T05:20:54.268727+0000","last_became_peered":"2023-10-06T05:20:54.268727+0000","last_unstale":"2023-10-06T05:20:54.274687+0000","last_undegraded":"2023-10-06T05:20:54.274687+0000","last_fullsized":"2023-10-06T05:20:54.274687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:15.554070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.250429+0000","last_change":"2023-10-06T05:20:53.257222+0000","last_active":"2023-10-06T05:20:54.250429+0000","last_peered":"2023-10-06T05:20:54.250429+0000","last_clean":"2023-10-06T05:20:54.250429+0000","last_became_active":"2023-10-06T05:20:53.256921+0000","last_became_peered":"2023-10-06T05:20:53.256921+0000","last_unstale":"2023-10-06T05:20:54.250429+0000","last_undegraded":"2023-10-06T05:20:54.250429+0000","last_fullsized":"2023-10-06T05:20:54.250429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:21.072433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.275019+0000","last_change":"2023-10-06T05:20:54.271142+0000","last_active":"2023-10-06T05:20:54.275019+0000","last_peered":"2023-10-06T05:20:54.275019+0000","last_clean":"2023-10-06T05:20:54.275019+0000","last_became_active":"2023-10-06T05:20:54.270876+0000","last_became_peered":"2023-10-06T05:20:54.270876+0000","last_unstale":"2023-10-06T05:20:54.275019+0000","last_undegraded":"2023-10-06T05:20:54.275019+0000","last_fullsized":"2023-10-06T05:20:54.275019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:11.640686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"78'3","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.269428+0000","last_change":"2023-10-06T05:20:53.252862+0000","last_active":"2023-10-06T05:20:54.269428+0000","last_peered":"2023-10-06T05:20:54.269428+0000","last_clean":"2023-10-06T05:20:54.269428+0000","last_became_active":"2023-10-06T05:20:53.252593+0000","last_became_peered":"2023-10-06T05:20:53.252593+0000","last_unstale":"2023-10-06T05:20:54.269428+0000","last_undegraded":"2023-10-06T05:20:54.269428+0000","last_fullsized":"2023-10-06T05:20:54.269428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:46.077440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.277595+0000","last_change":"2023-10-06T05:20:54.272528+0000","last_active":"2023-10-06T05:20:54.277595+0000","last_peered":"2023-10-06T05:20:54.277595+0000","last_clean":"2023-10-06T05:20:54.277595+0000","last_became_active":"2023-10-06T05:20:54.272254+0000","last_became_peered":"2023-10-06T05:20:54.272254+0000","last_unstale":"2023-10-06T05:20:54.277595+0000","last_undegraded":"2023-10-06T05:20:54.277595+0000","last_fullsized":"2023-10-06T05:20:54.277595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:50.585984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"78'6","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.275299+0000","last_change":"2023-10-06T05:20:53.258766+0000","last_active":"2023-10-06T05:20:54.275299+0000","last_peered":"2023-10-06T05:20:54.275299+0000","last_clean":"2023-10-06T05:20:54.275299+0000","last_became_active":"2023-10-06T05:20:53.258476+0000","last_became_peered":"2023-10-06T05:20:53.258476+0000","last_unstale":"2023-10-06T05:20:54.275299+0000","last_undegraded":"2023-10-06T05:20:54.275299+0000","last_fullsized":"2023-10-06T05:20:54.275299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:22.638085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273532+0000","last_change":"2023-10-06T05:20:54.264985+0000","last_active":"2023-10-06T05:20:54.273532+0000","last_peered":"2023-10-06T05:20:54.273532+0000","last_clean":"2023-10-06T05:20:54.273532+0000","last_became_active":"2023-10-06T05:20:54.264645+0000","last_became_peered":"2023-10-06T05:20:54.264645+0000","last_unstale":"2023-10-06T05:20:54.273532+0000","last_undegraded":"2023-10-06T05:20:54.273532+0000","last_fullsized":"2023-10-06T05:20:54.273532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:04.153012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273804+0000","last_change":"2023-10-06T05:20:53.257504+0000","last_active":"2023-10-06T05:20:54.273804+0000","last_peered":"2023-10-06T05:20:54.273804+0000","last_clean":"2023-10-06T05:20:54.273804+0000","last_became_active":"2023-10-06T05:20:53.257222+0000","last_became_peered":"2023-10-06T05:20:53.257222+0000","last_unstale":"2023-10-06T05:20:54.273804+0000","last_undegraded":"2023-10-06T05:20:54.273804+0000","last_fullsized":"2023-10-06T05:20:54.273804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:22.355522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.263393+0000","last_change":"2023-10-06T05:20:54.258213+0000","last_active":"2023-10-06T05:20:54.263393+0000","last_peered":"2023-10-06T05:20:54.263393+0000","last_clean":"2023-10-06T05:20:54.263393+0000","last_became_active":"2023-10-06T05:20:54.257943+0000","last_became_peered":"2023-10-06T05:20:54.257943+0000","last_unstale":"2023-10-06T05:20:54.263393+0000","last_undegraded":"2023-10-06T05:20:54.263393+0000","last_fullsized":"2023-10-06T05:20:54.263393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:54.388524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.245920+0000","last_change":"2023-10-06T05:20:53.257924+0000","last_active":"2023-10-06T05:20:54.245920+0000","last_peered":"2023-10-06T05:20:54.245920+0000","last_clean":"2023-10-06T05:20:54.245920+0000","last_became_active":"2023-10-06T05:20:53.257646+0000","last_became_peered":"2023-10-06T05:20:53.257646+0000","last_unstale":"2023-10-06T05:20:54.245920+0000","last_undegraded":"2023-10-06T05:20:54.245920+0000","last_fullsized":"2023-10-06T05:20:54.245920+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:19.619788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274759+0000","last_change":"2023-10-06T05:20:54.269984+0000","last_active":"2023-10-06T05:20:54.274759+0000","last_peered":"2023-10-06T05:20:54.274759+0000","last_clean":"2023-10-06T05:20:54.274759+0000","last_became_active":"2023-10-06T05:20:54.269700+0000","last_became_peered":"2023-10-06T05:20:54.269700+0000","last_unstale":"2023-10-06T05:20:54.274759+0000","last_undegraded":"2023-10-06T05:20:54.274759+0000","last_fullsized":"2023-10-06T05:20:54.274759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:57.666799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.245007+0000","last_change":"2023-10-06T05:20:53.254542+0000","last_active":"2023-10-06T05:20:54.245007+0000","last_peered":"2023-10-06T05:20:54.245007+0000","last_clean":"2023-10-06T05:20:54.245007+0000","last_became_active":"2023-10-06T05:20:53.254305+0000","last_became_peered":"2023-10-06T05:20:53.254305+0000","last_unstale":"2023-10-06T05:20:54.245007+0000","last_undegraded":"2023-10-06T05:20:54.245007+0000","last_fullsized":"2023-10-06T05:20:54.245007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:47.781111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274622+0000","last_change":"2023-10-06T05:20:54.270444+0000","last_active":"2023-10-06T05:20:54.274622+0000","last_peered":"2023-10-06T05:20:54.274622+0000","last_clean":"2023-10-06T05:20:54.274622+0000","last_became_active":"2023-10-06T05:20:54.270150+0000","last_became_peered":"2023-10-06T05:20:54.270150+0000","last_unstale":"2023-10-06T05:20:54.274622+0000","last_undegraded":"2023-10-06T05:20:54.274622+0000","last_fullsized":"2023-10-06T05:20:54.274622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:47.082860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"78'8","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.277438+0000","last_change":"2023-10-06T05:20:53.254559+0000","last_active":"2023-10-06T05:20:54.277438+0000","last_peered":"2023-10-06T05:20:54.277438+0000","last_clean":"2023-10-06T05:20:54.277438+0000","last_became_active":"2023-10-06T05:20:53.254263+0000","last_became_peered":"2023-10-06T05:20:53.254263+0000","last_unstale":"2023-10-06T05:20:54.277438+0000","last_undegraded":"2023-10-06T05:20:54.277438+0000","last_fullsized":"2023-10-06T05:20:54.277438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:51.693934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.261599+0000","last_change":"2023-10-06T05:20:54.258759+0000","last_active":"2023-10-06T05:20:54.261599+0000","last_peered":"2023-10-06T05:20:54.261599+0000","last_clean":"2023-10-06T05:20:54.261599+0000","last_became_active":"2023-10-06T05:20:54.258499+0000","last_became_peered":"2023-10-06T05:20:54.258499+0000","last_unstale":"2023-10-06T05:20:54.261599+0000","last_undegraded":"2023-10-06T05:20:54.261599+0000","last_fullsized":"2023-10-06T05:20:54.261599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:44.992062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.264981+0000","last_change":"2023-10-06T05:20:53.250703+0000","last_active":"2023-10-06T05:20:54.264981+0000","last_peered":"2023-10-06T05:20:54.264981+0000","last_clean":"2023-10-06T05:20:54.264981+0000","last_became_active":"2023-10-06T05:20:53.250450+0000","last_became_peered":"2023-10-06T05:20:53.250450+0000","last_unstale":"2023-10-06T05:20:54.264981+0000","last_undegraded":"2023-10-06T05:20:54.264981+0000","last_fullsized":"2023-10-06T05:20:54.264981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.804388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.269300+0000","last_change":"2023-10-06T05:20:54.261425+0000","last_active":"2023-10-06T05:20:54.269300+0000","last_peered":"2023-10-06T05:20:54.269300+0000","last_clean":"2023-10-06T05:20:54.269300+0000","last_became_active":"2023-10-06T05:20:54.261150+0000","last_became_peered":"2023-10-06T05:20:54.261150+0000","last_unstale":"2023-10-06T05:20:54.269300+0000","last_undegraded":"2023-10-06T05:20:54.269300+0000","last_fullsized":"2023-10-06T05:20:54.269300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:11.640686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.249379+0000","last_change":"2023-10-06T05:20:53.254425+0000","last_active":"2023-10-06T05:20:54.249379+0000","last_peered":"2023-10-06T05:20:54.249379+0000","last_clean":"2023-10-06T05:20:54.249379+0000","last_became_active":"2023-10-06T05:20:53.254157+0000","last_became_peered":"2023-10-06T05:20:53.254157+0000","last_unstale":"2023-10-06T05:20:54.249379+0000","last_undegraded":"2023-10-06T05:20:54.249379+0000","last_fullsized":"2023-10-06T05:20:54.249379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:18.848818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.272045+0000","last_change":"2023-10-06T05:20:54.267005+0000","last_active":"2023-10-06T05:20:54.272045+0000","last_peered":"2023-10-06T05:20:54.272045+0000","last_clean":"2023-10-06T05:20:54.272045+0000","last_became_active":"2023-10-06T05:20:54.266732+0000","last_became_peered":"2023-10-06T05:20:54.266732+0000","last_unstale":"2023-10-06T05:20:54.272045+0000","last_undegraded":"2023-10-06T05:20:54.272045+0000","last_fullsized":"2023-10-06T05:20:54.272045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:35.301660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.251930+0000","last_change":"2023-10-06T05:20:53.253804+0000","last_active":"2023-10-06T05:20:54.251930+0000","last_peered":"2023-10-06T05:20:54.251930+0000","last_clean":"2023-10-06T05:20:54.251930+0000","last_became_active":"2023-10-06T05:20:53.253540+0000","last_became_peered":"2023-10-06T05:20:53.253540+0000","last_unstale":"2023-10-06T05:20:54.251930+0000","last_undegraded":"2023-10-06T05:20:54.251930+0000","last_fullsized":"2023-10-06T05:20:54.251930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:14.321777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.269577+0000","last_change":"2023-10-06T05:20:54.265049+0000","last_active":"2023-10-06T05:20:54.269577+0000","last_peered":"2023-10-06T05:20:54.269577+0000","last_clean":"2023-10-06T05:20:54.269577+0000","last_became_active":"2023-10-06T05:20:54.264714+0000","last_became_peered":"2023-10-06T05:20:54.264714+0000","last_unstale":"2023-10-06T05:20:54.269577+0000","last_undegraded":"2023-10-06T05:20:54.269577+0000","last_fullsized":"2023-10-06T05:20:54.269577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:52.109899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.252667+0000","last_change":"2023-10-06T05:20:53.257110+0000","last_active":"2023-10-06T05:20:54.252667+0000","last_peered":"2023-10-06T05:20:54.252667+0000","last_clean":"2023-10-06T05:20:54.252667+0000","last_became_active":"2023-10-06T05:20:53.256732+0000","last_became_peered":"2023-10-06T05:20:53.256732+0000","last_unstale":"2023-10-06T05:20:54.252667+0000","last_undegraded":"2023-10-06T05:20:54.252667+0000","last_fullsized":"2023-10-06T05:20:54.252667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:11.900764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.259247+0000","last_change":"2023-10-06T05:20:54.255721+0000","last_active":"2023-10-06T05:20:54.259247+0000","last_peered":"2023-10-06T05:20:54.259247+0000","last_clean":"2023-10-06T05:20:54.259247+0000","last_became_active":"2023-10-06T05:20:54.255438+0000","last_became_peered":"2023-10-06T05:20:54.255438+0000","last_unstale":"2023-10-06T05:20:54.259247+0000","last_undegraded":"2023-10-06T05:20:54.259247+0000","last_fullsized":"2023-10-06T05:20:54.259247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:38.836602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.247079+0000","last_change":"2023-10-06T05:20:53.265594+0000","last_active":"2023-10-06T05:20:54.247079+0000","last_peered":"2023-10-06T05:20:54.247079+0000","last_clean":"2023-10-06T05:20:54.247079+0000","last_became_active":"2023-10-06T05:20:53.265240+0000","last_became_peered":"2023-10-06T05:20:53.265240+0000","last_unstale":"2023-10-06T05:20:54.247079+0000","last_undegraded":"2023-10-06T05:20:54.247079+0000","last_fullsized":"2023-10-06T05:20:54.247079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:41.006544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.274959+0000","last_change":"2023-10-06T05:20:54.269627+0000","last_active":"2023-10-06T05:20:54.274959+0000","last_peered":"2023-10-06T05:20:54.274959+0000","last_clean":"2023-10-06T05:20:54.274959+0000","last_became_active":"2023-10-06T05:20:54.265563+0000","last_became_peered":"2023-10-06T05:20:54.265563+0000","last_unstale":"2023-10-06T05:20:54.274959+0000","last_undegraded":"2023-10-06T05:20:54.274959+0000","last_fullsized":"2023-10-06T05:20:54.274959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:54.388524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.248701+0000","last_change":"2023-10-06T05:20:53.256656+0000","last_active":"2023-10-06T05:20:54.248701+0000","last_peered":"2023-10-06T05:20:54.248701+0000","last_clean":"2023-10-06T05:20:54.248701+0000","last_became_active":"2023-10-06T05:20:53.256388+0000","last_became_peered":"2023-10-06T05:20:53.256388+0000","last_unstale":"2023-10-06T05:20:54.248701+0000","last_undegraded":"2023-10-06T05:20:54.248701+0000","last_fullsized":"2023-10-06T05:20:54.248701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:19.619788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.273370+0000","last_change":"2023-10-06T05:20:54.265707+0000","last_active":"2023-10-06T05:20:54.273370+0000","last_peered":"2023-10-06T05:20:54.273370+0000","last_clean":"2023-10-06T05:20:54.273370+0000","last_became_active":"2023-10-06T05:20:54.265427+0000","last_became_peered":"2023-10-06T05:20:54.265427+0000","last_unstale":"2023-10-06T05:20:54.273370+0000","last_undegraded":"2023-10-06T05:20:54.273370+0000","last_fullsized":"2023-10-06T05:20:54.273370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:38.836602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.252192+0000","last_change":"2023-10-06T05:20:53.258419+0000","last_active":"2023-10-06T05:20:54.252192+0000","last_peered":"2023-10-06T05:20:54.252192+0000","last_clean":"2023-10-06T05:20:54.252192+0000","last_became_active":"2023-10-06T05:20:53.258132+0000","last_became_peered":"2023-10-06T05:20:53.258132+0000","last_unstale":"2023-10-06T05:20:54.252192+0000","last_undegraded":"2023-10-06T05:20:54.252192+0000","last_fullsized":"2023-10-06T05:20:54.252192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:42.456593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.272053+0000","last_change":"2023-10-06T05:20:54.266256+0000","last_active":"2023-10-06T05:20:54.272053+0000","last_peered":"2023-10-06T05:20:54.272053+0000","last_clean":"2023-10-06T05:20:54.272053+0000","last_became_active":"2023-10-06T05:20:54.265871+0000","last_became_peered":"2023-10-06T05:20:54.265871+0000","last_unstale":"2023-10-06T05:20:54.272053+0000","last_undegraded":"2023-10-06T05:20:54.272053+0000","last_fullsized":"2023-10-06T05:20:54.272053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:38.836602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.249632+0000","last_change":"2023-10-06T05:20:53.253533+0000","last_active":"2023-10-06T05:20:54.249632+0000","last_peered":"2023-10-06T05:20:54.249632+0000","last_clean":"2023-10-06T05:20:54.249632+0000","last_became_active":"2023-10-06T05:20:53.253280+0000","last_became_peered":"2023-10-06T05:20:53.253280+0000","last_unstale":"2023-10-06T05:20:54.249632+0000","last_undegraded":"2023-10-06T05:20:54.249632+0000","last_fullsized":"2023-10-06T05:20:54.249632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:40.136291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.259270+0000","last_change":"2023-10-06T05:20:54.255841+0000","last_active":"2023-10-06T05:20:54.259270+0000","last_peered":"2023-10-06T05:20:54.259270+0000","last_clean":"2023-10-06T05:20:54.259270+0000","last_became_active":"2023-10-06T05:20:54.255533+0000","last_became_peered":"2023-10-06T05:20:54.255533+0000","last_unstale":"2023-10-06T05:20:54.259270+0000","last_undegraded":"2023-10-06T05:20:54.259270+0000","last_fullsized":"2023-10-06T05:20:54.259270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:32.705266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.260715+0000","last_change":"2023-10-06T05:20:53.257623+0000","last_active":"2023-10-06T05:20:54.260715+0000","last_peered":"2023-10-06T05:20:54.260715+0000","last_clean":"2023-10-06T05:20:54.260715+0000","last_became_active":"2023-10-06T05:20:53.257338+0000","last_became_peered":"2023-10-06T05:20:53.257338+0000","last_unstale":"2023-10-06T05:20:54.260715+0000","last_undegraded":"2023-10-06T05:20:54.260715+0000","last_fullsized":"2023-10-06T05:20:54.260715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:43.173477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.248967+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:20:54.248967+0000","last_peered":"2023-10-06T05:20:54.248967+0000","last_clean":"2023-10-06T05:20:54.248967+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:20:54.248967+0000","last_undegraded":"2023-10-06T05:20:54.248967+0000","last_fullsized":"2023-10-06T05:20:54.248967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.269627+0000","last_change":"2023-10-06T05:20:54.266885+0000","last_active":"2023-10-06T05:20:54.269627+0000","last_peered":"2023-10-06T05:20:54.269627+0000","last_clean":"2023-10-06T05:20:54.269627+0000","last_became_active":"2023-10-06T05:20:54.266622+0000","last_became_peered":"2023-10-06T05:20:54.266622+0000","last_unstale":"2023-10-06T05:20:54.269627+0000","last_undegraded":"2023-10-06T05:20:54.269627+0000","last_fullsized":"2023-10-06T05:20:54.269627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:22.419963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.250150+0000","last_change":"2023-10-06T05:20:53.265884+0000","last_active":"2023-10-06T05:20:54.250150+0000","last_peered":"2023-10-06T05:20:54.250150+0000","last_clean":"2023-10-06T05:20:54.250150+0000","last_became_active":"2023-10-06T05:20:53.265439+0000","last_became_peered":"2023-10-06T05:20:53.265439+0000","last_unstale":"2023-10-06T05:20:54.250150+0000","last_undegraded":"2023-10-06T05:20:54.250150+0000","last_fullsized":"2023-10-06T05:20:54.250150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:40.136291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.244959+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:20:54.244959+0000","last_peered":"2023-10-06T05:20:54.244959+0000","last_clean":"2023-10-06T05:20:54.244959+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:20:54.244959+0000","last_undegraded":"2023-10-06T05:20:54.244959+0000","last_fullsized":"2023-10-06T05:20:54.244959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.252047+0000","last_change":"2023-10-06T05:20:54.248488+0000","last_active":"2023-10-06T05:20:54.252047+0000","last_peered":"2023-10-06T05:20:54.252047+0000","last_clean":"2023-10-06T05:20:54.252047+0000","last_became_active":"2023-10-06T05:20:54.248236+0000","last_became_peered":"2023-10-06T05:20:54.248236+0000","last_unstale":"2023-10-06T05:20:54.252047+0000","last_undegraded":"2023-10-06T05:20:54.252047+0000","last_fullsized":"2023-10-06T05:20:54.252047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.200388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.260760+0000","last_change":"2023-10-06T05:20:53.254308+0000","last_active":"2023-10-06T05:20:54.260760+0000","last_peered":"2023-10-06T05:20:54.260760+0000","last_clean":"2023-10-06T05:20:54.260760+0000","last_became_active":"2023-10-06T05:20:53.253911+0000","last_became_peered":"2023-10-06T05:20:53.253911+0000","last_unstale":"2023-10-06T05:20:54.260760+0000","last_undegraded":"2023-10-06T05:20:54.260760+0000","last_fullsized":"2023-10-06T05:20:54.260760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:49.158080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.264769+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:20:54.264769+0000","last_peered":"2023-10-06T05:20:54.264769+0000","last_clean":"2023-10-06T05:20:54.264769+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:20:54.264769+0000","last_undegraded":"2023-10-06T05:20:54.264769+0000","last_fullsized":"2023-10-06T05:20:54.264769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.276531+0000","last_change":"2023-10-06T05:20:54.273173+0000","last_active":"2023-10-06T05:20:54.276531+0000","last_peered":"2023-10-06T05:20:54.276531+0000","last_clean":"2023-10-06T05:20:54.276531+0000","last_became_active":"2023-10-06T05:20:54.269043+0000","last_became_peered":"2023-10-06T05:20:54.269043+0000","last_unstale":"2023-10-06T05:20:54.276531+0000","last_undegraded":"2023-10-06T05:20:54.276531+0000","last_fullsized":"2023-10-06T05:20:54.276531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:22.077853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.262924+0000","last_change":"2023-10-06T05:20:53.252943+0000","last_active":"2023-10-06T05:20:54.262924+0000","last_peered":"2023-10-06T05:20:54.262924+0000","last_clean":"2023-10-06T05:20:54.262924+0000","last_became_active":"2023-10-06T05:20:53.252691+0000","last_became_peered":"2023-10-06T05:20:53.252691+0000","last_unstale":"2023-10-06T05:20:54.262924+0000","last_undegraded":"2023-10-06T05:20:54.262924+0000","last_fullsized":"2023-10-06T05:20:54.262924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:37.159405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.245122+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:20:54.245122+0000","last_peered":"2023-10-06T05:20:54.245122+0000","last_clean":"2023-10-06T05:20:54.245122+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:20:54.245122+0000","last_undegraded":"2023-10-06T05:20:54.245122+0000","last_fullsized":"2023-10-06T05:20:54.245122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":212,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.242660+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:20:54.242660+0000","last_peered":"2023-10-06T05:20:54.242660+0000","last_clean":"2023-10-06T05:20:54.242660+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:20:54.242660+0000","last_undegraded":"2023-10-06T05:20:54.242660+0000","last_fullsized":"2023-10-06T05:20:54.242660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.260827+0000","last_change":"2023-10-06T05:20:54.256964+0000","last_active":"2023-10-06T05:20:54.260827+0000","last_peered":"2023-10-06T05:20:54.260827+0000","last_clean":"2023-10-06T05:20:54.260827+0000","last_became_active":"2023-10-06T05:20:54.256445+0000","last_became_peered":"2023-10-06T05:20:54.256445+0000","last_unstale":"2023-10-06T05:20:54.260827+0000","last_undegraded":"2023-10-06T05:20:54.260827+0000","last_fullsized":"2023-10-06T05:20:54.260827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:31.456021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.265266+0000","last_change":"2023-10-06T05:20:53.265761+0000","last_active":"2023-10-06T05:20:54.265266+0000","last_peered":"2023-10-06T05:20:54.265266+0000","last_clean":"2023-10-06T05:20:54.265266+0000","last_became_active":"2023-10-06T05:20:53.265491+0000","last_became_peered":"2023-10-06T05:20:53.265491+0000","last_unstale":"2023-10-06T05:20:54.265266+0000","last_undegraded":"2023-10-06T05:20:54.265266+0000","last_fullsized":"2023-10-06T05:20:54.265266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:54.675877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.242758+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:20:54.242758+0000","last_peered":"2023-10-06T05:20:54.242758+0000","last_clean":"2023-10-06T05:20:54.242758+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:20:54.242758+0000","last_undegraded":"2023-10-06T05:20:54.242758+0000","last_fullsized":"2023-10-06T05:20:54.242758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.280925+0000","last_change":"2023-10-06T05:20:54.278846+0000","last_active":"2023-10-06T05:20:54.280925+0000","last_peered":"2023-10-06T05:20:54.280925+0000","last_clean":"2023-10-06T05:20:54.280925+0000","last_became_active":"2023-10-06T05:20:54.278532+0000","last_became_peered":"2023-10-06T05:20:54.278532+0000","last_unstale":"2023-10-06T05:20:54.280925+0000","last_undegraded":"2023-10-06T05:20:54.280925+0000","last_fullsized":"2023-10-06T05:20:54.280925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:21.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.264393+0000","last_change":"2023-10-06T05:20:53.257540+0000","last_active":"2023-10-06T05:20:54.264393+0000","last_peered":"2023-10-06T05:20:54.264393+0000","last_clean":"2023-10-06T05:20:54.264393+0000","last_became_active":"2023-10-06T05:20:53.257232+0000","last_became_peered":"2023-10-06T05:20:53.257232+0000","last_unstale":"2023-10-06T05:20:54.264393+0000","last_undegraded":"2023-10-06T05:20:54.264393+0000","last_fullsized":"2023-10-06T05:20:54.264393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:18.784241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.243383+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:20:54.243383+0000","last_peered":"2023-10-06T05:20:54.243383+0000","last_clean":"2023-10-06T05:20:54.243383+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:20:54.243383+0000","last_undegraded":"2023-10-06T05:20:54.243383+0000","last_fullsized":"2023-10-06T05:20:54.243383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.271856+0000","last_change":"2023-10-06T05:20:54.264991+0000","last_active":"2023-10-06T05:20:54.271856+0000","last_peered":"2023-10-06T05:20:54.271856+0000","last_clean":"2023-10-06T05:20:54.271856+0000","last_became_active":"2023-10-06T05:20:54.264773+0000","last_became_peered":"2023-10-06T05:20:54.264773+0000","last_unstale":"2023-10-06T05:20:54.271856+0000","last_undegraded":"2023-10-06T05:20:54.271856+0000","last_fullsized":"2023-10-06T05:20:54.271856+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:42.011964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.245637+0000","last_change":"2023-10-06T05:20:53.256845+0000","last_active":"2023-10-06T05:20:54.245637+0000","last_peered":"2023-10-06T05:20:54.245637+0000","last_clean":"2023-10-06T05:20:54.245637+0000","last_became_active":"2023-10-06T05:20:53.256575+0000","last_became_peered":"2023-10-06T05:20:53.256575+0000","last_unstale":"2023-10-06T05:20:54.245637+0000","last_undegraded":"2023-10-06T05:20:54.245637+0000","last_fullsized":"2023-10-06T05:20:54.245637+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:14.548651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.249464+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:20:54.249464+0000","last_peered":"2023-10-06T05:20:54.249464+0000","last_clean":"2023-10-06T05:20:54.249464+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:20:54.249464+0000","last_undegraded":"2023-10-06T05:20:54.249464+0000","last_fullsized":"2023-10-06T05:20:54.249464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.271688+0000","last_change":"2023-10-06T05:20:54.266002+0000","last_active":"2023-10-06T05:20:54.271688+0000","last_peered":"2023-10-06T05:20:54.271688+0000","last_clean":"2023-10-06T05:20:54.271688+0000","last_became_active":"2023-10-06T05:20:54.265701+0000","last_became_peered":"2023-10-06T05:20:54.265701+0000","last_unstale":"2023-10-06T05:20:54.271688+0000","last_undegraded":"2023-10-06T05:20:54.271688+0000","last_fullsized":"2023-10-06T05:20:54.271688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:53.226501+0000","last_clean_scrub_stamp":"2023-10-06T05:20:53.226501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:51.977276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.264667+0000","last_change":"2023-10-06T05:20:53.255961+0000","last_active":"2023-10-06T05:20:54.264667+0000","last_peered":"2023-10-06T05:20:54.264667+0000","last_clean":"2023-10-06T05:20:54.264667+0000","last_became_active":"2023-10-06T05:20:53.255666+0000","last_became_peered":"2023-10-06T05:20:53.255666+0000","last_unstale":"2023-10-06T05:20:54.264667+0000","last_undegraded":"2023-10-06T05:20:54.264667+0000","last_fullsized":"2023-10-06T05:20:54.264667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:52.221081+0000","last_clean_scrub_stamp":"2023-10-06T05:20:52.221081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:28.804388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T05:20:54.252218+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:20:54.252218+0000","last_peered":"2023-10-06T05:20:54.252218+0000","last_clean":"2023-10-06T05:20:54.252218+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:20:54.252218+0000","last_undegraded":"2023-10-06T05:20:54.252218+0000","last_fullsized":"2023-10-06T05:20:54.252218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298804,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073036,"statfs":{"total":96636764160,"available":96330788864,"internally_reserved":0,"allocated":1552384,"data_stored":1047929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.878,"15min":0.878},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78500000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.621}]},{"osd":1,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.61799999999999999},"min":{"1min":0.44400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72599999999999998,"5min":1.476,"15min":1.476},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.42799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78100000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.63},"min":{"1min":0.45900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77400000000000002,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.46800000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.78300000000000003,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55500000000000005,"5min":0.754,"15min":0.754},"last":0.44},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.60399999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72599999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73799999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":978944,"data_stored":465118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.73399999999999999},"min":{"1min":0.443,"5min":0.311,"15min":0.311},"max":{"1min":0.72099999999999997,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.502,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78600000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.82799999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68400000000000005,"15min":0.94699999999999995},"min":{"1min":0.51400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.75900000000000001,"5min":9.6300000000000008,"15min":9.6300000000000008},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.76200000000000001},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75600000000000001,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.71599999999999997,"15min":1.004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77500000000000002,"5min":11.595000000000001,"15min":11.595000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.65600000000000003,"15min":0.79800000000000004},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73899999999999999,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.85499999999999998,"15min":1.5009999999999999},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75900000000000001,"5min":23.146000000000001,"15min":23.146000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71599999999999997,"15min":0.95399999999999996},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77400000000000002,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.58299999999999996,"15min":0.86299999999999999},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67200000000000004,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.61499999999999999,"15min":0.89100000000000001},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64400000000000002,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53700000000000003,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65700000000000003,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.63600000000000001,"15min":0.89100000000000001},"min":{"1min":0.41499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77700000000000002,"5min":9.8420000000000005,"15min":9.8420000000000005},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.66400000000000003,"15min":0.92800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":9.718,"15min":9.718},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.68500000000000005,"15min":0.95399999999999996},"min":{"1min":0.436,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77600000000000002,"5min":9.8930000000000007,"15min":9.8930000000000007},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":978944,"data_stored":464646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.42399999999999999},"min":{"1min":0.33600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.437,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.435,"15min":0.47899999999999998},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54100000000000004,"5min":1.673,"15min":1.673},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.498,"15min":0.57999999999999996},"min":{"1min":0.33000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.443,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52500000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.34499999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.52100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66900000000000004,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93100000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90800000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90000000000000002,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.53300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.873,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45700000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.91700000000000004,"5min":1.167,"15min":1.167},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.52400000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93700000000000006,"5min":1.026,"15min":1.026},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.56299999999999994,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298816,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073024,"statfs":{"total":96636764160,"available":96330776576,"internally_reserved":0,"allocated":1564672,"data_stored":1054854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73199999999999998,"5min":1.038,"15min":1.038},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.433,"15min":0.42099999999999999},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60899999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.252,"15min":0.252},"max":{"1min":0.57899999999999996,"5min":3.855,"15min":3.855},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65400000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.501,"15min":0.443},"min":{"1min":0.29699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55600000000000005,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":0.495},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":7.7670000000000003,"15min":7.7670000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63400000000000001,"15min":0.61499999999999999},"min":{"1min":0.46300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79400000000000004,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64600000000000002,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89200000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.66900000000000004},"min":{"1min":0.53500000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80900000000000005,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70399999999999996,"15min":0.66700000000000004},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":5.3600000000000003,"15min":5.3600000000000003},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.501,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82999999999999996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69699999999999995,"15min":0.70799999999999996},"min":{"1min":0.53500000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95499999999999996,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52800000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67400000000000004,"15min":0.66100000000000003},"min":{"1min":0.55200000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.873,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.70999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":465006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.432},"min":{"1min":0.26900000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.59299999999999997,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42699999999999999,"15min":0.46700000000000003},"min":{"1min":0.30499999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.56200000000000006,"5min":1.899,"15min":1.899},"last":0.46600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45900000000000002,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.624,"5min":2.097,"15min":2.097},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.48799999999999999,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66200000000000003,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.44500000000000001},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60799999999999998,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.51900000000000002},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.67600000000000005,"5min":2.121,"15min":2.121},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.71099999999999997},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88200000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.753},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.53500000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92500000000000004,"5min":1.643,"15min":1.643},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.51600000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94399999999999995,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95899999999999996,"5min":1.851,"15min":1.851},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.51400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.875,"5min":1.571,"15min":1.571},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88700000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68899999999999995,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.90700000000000003,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97399999999999998,"5min":1.613,"15min":1.613},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073600,"statfs":{"total":96636764160,"available":96331366400,"internally_reserved":0,"allocated":974848,"data_stored":464635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.28199999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.40999999999999998,"15min":0.39300000000000002},"min":{"1min":0.29499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.38500000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.41599999999999998},"min":{"1min":0.34000000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.42699999999999999,"15min":0.41699999999999998},"min":{"1min":0.316,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61799999999999999,"5min":0.755,"15min":0.755},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.44400000000000001,"15min":0.44500000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.376}]},{"osd":4,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65600000000000003,"15min":0.63500000000000001},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.442,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84299999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.502,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78600000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.505},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.55000000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.47699999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.497,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.433}]},{"osd":7,"last update":"Fri Oct 6 05:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67400000000000004,"15min":0.66300000000000003},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67900000000000005,"15min":0.66300000000000003},"min":{"1min":0.48399999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":954368,"data_stored":457491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.443,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.60599999999999998},"min":{"1min":0.46400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78200000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.48399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75600000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.438},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.44400000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.75,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73199999999999998,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83399999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.624},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.69799999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.753,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.51300000000000001,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75900000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67900000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78700000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.77500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":983040,"data_stored":466010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.746,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.55700000000000005},"min":{"1min":0.45100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.78000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70499999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.55100000000000005},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71799999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72099999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75,"5min":0.876,"15min":0.876},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.49099999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78900000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.75},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57199999999999995},"min":{"1min":0.501,"5min":0.38,"15min":0.38},"max":{"1min":0.72699999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73199999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47499999999999998,"15min":0.46700000000000003},"min":{"1min":0.36499999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58299999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55900000000000005,"15min":0.53500000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70099999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48699999999999999,"15min":0.46899999999999997},"min":{"1min":0.35999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.72399999999999998,"5min":0.876,"15min":0.876},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68200000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64700000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:20:58.845 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:20:58.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.019+0000 7fa331977700 1 Processor -- start 2023-10-06T05:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.019+0000 7fa331977700 1 -- start start 2023-10-06T05:20:59.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa331977700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa32c156780 0x7fa32c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa331977700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa32c0c7ec0 0x7fa32c157090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa331977700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0cc740 con 0x7fa32c06a070 2023-10-06T05:20:59.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa331977700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c0c68b0 con 0x7fa32c0c7ec0 2023-10-06T05:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa331977700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c0c6b90 con 0x7fa32c156780 2023-10-06T05:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa32c0c7ec0 0x7fa32c157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa32c0c7ec0 0x7fa32c157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45396/0 (socket says 172.21.15.45:45396) 2023-10-06T05:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 -- 172.21.15.45:0/2393463439 learned_addr learned my addr 172.21.15.45:0/2393463439 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32affd700 1 --2- 172.21.15.45:0/2393463439 >> v2:172.21.15.144:3300/0 conn(0x7fa32c156780 0x7fa32c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 -- 172.21.15.45:0/2393463439 >> v2:172.21.15.144:3300/0 conn(0x7fa32c156780 msgr2=0x7fa32c156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:59.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 --2- 172.21.15.45:0/2393463439 >> v2:172.21.15.144:3300/0 conn(0x7fa32c156780 0x7fa32c156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 -- 172.21.15.45:0/2393463439 >> v1:172.21.15.144:6789/0 conn(0x7fa32c06a070 legacy=0x7fa32c0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 -- 172.21.15.45:0/2393463439 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0c6e70 con 0x7fa32c0c7ec0 2023-10-06T05:20:59.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.020+0000 7fa32b7fe700 1 --2- 172.21.15.45:0/2393463439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c0c7ec0 0x7fa32c157090 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06da00 tx=0x7fa324009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e02adaa6be45ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa329ffb700 1 -- 172.21.15.45:0/2393463439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32400e070 con 0x7fa32c0c7ec0 2023-10-06T05:20:59.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa329ffb700 1 -- 172.21.15.45:0/2393463439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32400ba20 con 0x7fa32c0c7ec0 2023-10-06T05:20:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa329ffb700 1 -- 172.21.15.45:0/2393463439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3240092e0 con 0x7fa32c0c7ec0 2023-10-06T05:20:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 -- 172.21.15.45:0/2393463439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c0c7ec0 msgr2=0x7fa32c157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 --2- 172.21.15.45:0/2393463439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c0c7ec0 0x7fa32c157090 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06da00 tx=0x7fa324009840 comp rx=0 tx=0).stop 2023-10-06T05:20:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 -- 172.21.15.45:0/2393463439 shutdown_connections 2023-10-06T05:20:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 --2- 172.21.15.45:0/2393463439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c0c7ec0 0x7fa32c157090 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 --2- 172.21.15.45:0/2393463439 >> v2:172.21.15.144:3300/0 conn(0x7fa32c156780 0x7fa32c156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 -- 172.21.15.45:0/2393463439 >> 172.21.15.45:0/2393463439 conn(0x7fa32c0680f0 msgr2=0x7fa32c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 -- 172.21.15.45:0/2393463439 shutdown_connections 2023-10-06T05:20:59.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.021+0000 7fa331977700 1 -- 172.21.15.45:0/2393463439 wait complete. 2023-10-06T05:20:59.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 Processor -- start 2023-10-06T05:20:59.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 -- start start 2023-10-06T05:20:59.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32c06a070 0x7fa32c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32c0c7ec0 0x7fa32c147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c156780 0x7fa32c165970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c1565d0 con 0x7fa32c156780 2023-10-06T05:20:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c151950 con 0x7fa32c06a070 2023-10-06T05:20:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.023+0000 7fa331977700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c165eb0 con 0x7fa32c0c7ec0 2023-10-06T05:20:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c156780 0x7fa32c165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c156780 0x7fa32c165970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45408/0 (socket says 172.21.15.45:45408) 2023-10-06T05:20:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 -- 172.21.15.45:0/1451631770 learned_addr learned my addr 172.21.15.45:0/1451631770 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32affd700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32c06a070 0x7fa32c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32a7fc700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32c0c7ec0 0x7fa32c147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 -- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32c0c7ec0 msgr2=0x7fa32c147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32c0c7ec0 0x7fa32c147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 -- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32c06a070 msgr2=0x7fa32c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32c06a070 0x7fa32c155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 -- 172.21.15.45:0/1451631770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324009ae0 con 0x7fa32c156780 2023-10-06T05:20:59.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa32b7fe700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c156780 0x7fa32c165970 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa324009880 tx=0x7fa324018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa322ffd700 1 -- 172.21.15.45:0/1451631770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324008a30 con 0x7fa32c156780 2023-10-06T05:20:59.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa322ffd700 1 -- 172.21.15.45:0/1451631770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa324008b90 con 0x7fa32c156780 2023-10-06T05:20:59.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa322ffd700 1 -- 172.21.15.45:0/1451631770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324026460 con 0x7fa32c156780 2023-10-06T05:20:59.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c1562f0 con 0x7fa32c156780 2023-10-06T05:20:59.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.024+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c1520f0 con 0x7fa32c156780 2023-10-06T05:20:59.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.025+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c06cc70 con 0x7fa32c156780 2023-10-06T05:20:59.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.026+0000 7fa322ffd700 1 -- 172.21.15.45:0/1451631770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa324003040 con 0x7fa32c156780 2023-10-06T05:20:59.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.026+0000 7fa322ffd700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa30c066f30 0x7fa30c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.026+0000 7fa322ffd700 1 -- 172.21.15.45:0/1451631770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa324086650 con 0x7fa32c156780 2023-10-06T05:20:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.026+0000 7fa32affd700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa30c066f30 0x7fa30c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.027+0000 7fa32affd700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa30c066f30 0x7fa30c0693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009d10 tx=0x7fa31c0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.031+0000 7fa322ffd700 1 -- 172.21.15.45:0/1451631770 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa324053620 con 0x7fa32c156780 2023-10-06T05:20:59.219 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:20:59.217+0000 7f6b20f95700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:20:59.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.313+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa32c0c7150 con 0x7fa32c156780 2023-10-06T05:20:59.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.317+0000 7fa322ffd700 1 -- 172.21.15.45:0/1451631770 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa324008f30 con 0x7fa32c156780 2023-10-06T05:20:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa30c066f30 msgr2=0x7fa30c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa30c066f30 0x7fa30c0693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009d10 tx=0x7fa31c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c156780 msgr2=0x7fa32c165970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c156780 0x7fa32c165970 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa324009880 tx=0x7fa324018000 comp rx=0 tx=0).stop 2023-10-06T05:20:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 shutdown_connections 2023-10-06T05:20:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa30c066f30 0x7fa30c0693e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa32c156780 0x7fa32c165970 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32c06a070 0x7fa32c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 --2- 172.21.15.45:0/1451631770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32c0c7ec0 0x7fa32c147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 >> 172.21.15.45:0/1451631770 conn(0x7fa32c0680f0 msgr2=0x7fa32c0c23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 shutdown_connections 2023-10-06T05:20:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.319+0000 7fa331977700 1 -- 172.21.15.45:0/1451631770 wait complete. 2023-10-06T05:20:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-10-06T05:20:59.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:59.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.465+0000 7f1225d03700 1 Processor -- start 2023-10-06T05:20:59.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.466+0000 7f1225d03700 1 -- start start 2023-10-06T05:20:59.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.466+0000 7f1225d03700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1220156ce0 0x7f12201570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.466+0000 7f1225d03700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12200ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.466+0000 7f1225d03700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122006bf70 con 0x7f122006bad0 2023-10-06T05:20:59.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.466+0000 7f1225d03700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12200b02f0 con 0x7f1220156ce0 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.466+0000 7f1225d03700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122015b380 con 0x7f122015ac10 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12200ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1220156ce0 0x7f12201570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121effd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f122006bad0 0x7f12200afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:41254/0 (socket says 172.21.15.45:41254) 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121effd700 1 -- 172.21.15.45:0/990292406 learned_addr learned my addr 172.21.15.45:0/990292406 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121f7fe700 1 -- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 msgr2=0x7f12200ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121f7fe700 1 --2- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12200ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121f7fe700 1 -- 172.21.15.45:0/990292406 >> v1:172.21.15.144:6789/0 conn(0x7f122006bad0 legacy=0x7f12200afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121f7fe700 1 -- 172.21.15.45:0/990292406 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122015b660 con 0x7f1220156ce0 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121ffff700 1 --2- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12200ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:59.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121e7fc700 1 -- 172.21.15.45:0/990292406 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41342220 0 0) 0x7f122006bf70 con 0x7f122006bad0 2023-10-06T05:20:59.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.467+0000 7f121f7fe700 1 --2- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1220156ce0 0x7f12201570b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12100098e0 tx=0x7f12100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4478b630102086b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f121e7fc700 1 -- 172.21.15.45:0/990292406 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121000b640 con 0x7f1220156ce0 2023-10-06T05:20:59.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f121e7fc700 1 -- 172.21.15.45:0/990292406 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121000b7a0 con 0x7f1220156ce0 2023-10-06T05:20:59.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f121e7fc700 1 -- 172.21.15.45:0/990292406 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121000bb40 con 0x7f1220156ce0 2023-10-06T05:20:59.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 -- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1220156ce0 msgr2=0x7f12201570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 --2- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1220156ce0 0x7f12201570b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12100098e0 tx=0x7f12100091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 -- 172.21.15.45:0/990292406 shutdown_connections 2023-10-06T05:20:59.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 --2- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12200ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 --2- 172.21.15.45:0/990292406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1220156ce0 0x7f12201570b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 -- 172.21.15.45:0/990292406 >> 172.21.15.45:0/990292406 conn(0x7f122006f090 msgr2=0x7f122006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 -- 172.21.15.45:0/990292406 shutdown_connections 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.468+0000 7f1225d03700 1 -- 172.21.15.45:0/990292406 wait complete. 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 Processor -- start 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 -- start start 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f122006bad0 0x7f12200c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1220156ce0 0x7f12200cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12201543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122016e380 con 0x7f1220156ce0 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122016d4d0 con 0x7f122006bad0 2023-10-06T05:20:59.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.470+0000 7f1225d03700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122016d770 con 0x7f122015ac10 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1220156ce0 0x7f12200cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1220156ce0 0x7f12200cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45422/0 (socket says 172.21.15.45:45422) 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 -- 172.21.15.45:0/3656041000 learned_addr learned my addr 172.21.15.45:0/3656041000 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121ffff700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12201543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121f7fe700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f122006bad0 0x7f12200c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 -- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 msgr2=0x7f12201543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12201543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 -- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f122006bad0 msgr2=0x7f12200c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f122006bad0 0x7f12200c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 -- 172.21.15.45:0/3656041000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218009a20 con 0x7f1220156ce0 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f121effd700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1220156ce0 0x7f12200cc0f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f121400c7e0 tx=0x7f1214007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f12077fe700 1 -- 172.21.15.45:0/3656041000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214005b30 con 0x7f1220156ce0 2023-10-06T05:20:59.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f12077fe700 1 -- 172.21.15.45:0/3656041000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1214005c90 con 0x7f1220156ce0 2023-10-06T05:20:59.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1210009420 con 0x7f1220156ce0 2023-10-06T05:20:59.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f12077fe700 1 -- 172.21.15.45:0/3656041000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214007300 con 0x7f1220156ce0 2023-10-06T05:20:59.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.471+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122016dd30 con 0x7f1220156ce0 2023-10-06T05:20:59.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.472+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12200af710 con 0x7f1220156ce0 2023-10-06T05:20:59.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.473+0000 7f12077fe700 1 -- 172.21.15.45:0/3656041000 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12140076b0 con 0x7f1220156ce0 2023-10-06T05:20:59.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.473+0000 7f12077fe700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f120c066f30 0x7f120c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.473+0000 7f12077fe700 1 -- 172.21.15.45:0/3656041000 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1214083c80 con 0x7f1220156ce0 2023-10-06T05:20:59.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.473+0000 7f121f7fe700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f120c066f30 0x7f120c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.474+0000 7f121f7fe700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f120c066f30 0x7f120c0693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f12100093f0 tx=0x7f1210017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.478+0000 7f12077fe700 1 -- 172.21.15.45:0/3656041000 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1214050d10 con 0x7f1220156ce0 2023-10-06T05:20:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.729+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f122015b940 con 0x7f1220156ce0 2023-10-06T05:20:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.730+0000 7f12077fe700 1 -- 172.21.15.45:0/3656041000 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1214017090 con 0x7f1220156ce0 2023-10-06T05:20:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:20:59.732 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:20:57.257456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1530850582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1530850582},{"type":"v1","addr":"172.21.15.45:6834","nonce":1530850582}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:20:59.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f120c066f30 msgr2=0x7f120c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f120c066f30 0x7f120c0693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f12100093f0 tx=0x7f1210017000 comp rx=0 tx=0).stop 2023-10-06T05:20:59.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1220156ce0 msgr2=0x7f12200cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1220156ce0 0x7f12200cc0f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f121400c7e0 tx=0x7f1214007df0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 shutdown_connections 2023-10-06T05:20:59.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f120c066f30 0x7f120c0693e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f122015ac10 0x7f12201543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f122006bad0 0x7f12200c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 --2- 172.21.15.45:0/3656041000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1220156ce0 0x7f12200cc0f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 >> 172.21.15.45:0/3656041000 conn(0x7f122006f090 msgr2=0x7f1220068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 shutdown_connections 2023-10-06T05:20:59.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:20:59.732+0000 7f1225d03700 1 -- 172.21.15.45:0/3656041000 wait complete. 2023-10-06T05:20:59.737 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 71 2023-10-06T05:21:00.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:20:53.763976+0000', 'modified': '2023-10-06T05:20:57.257456+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6702}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6702': {'gid': 6702, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6834/1530850582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 1530850582}, {'type': 'v1', 'addr': '172.21.15.45:6834', 'nonce': 1530850582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:21:00.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:21:00.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:21:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.292+0000 7f8485eb9700 1 Processor -- start 2023-10-06T05:21:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.292+0000 7f8485eb9700 1 -- start start 2023-10-06T05:21:00.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.292+0000 7f8485eb9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 0x7f848006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.292+0000 7f8485eb9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800b2120 con 0x7f848015a490 2023-10-06T05:21:00.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.292+0000 7f8485eb9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800ab7d0 con 0x7f848006add0 2023-10-06T05:21:00.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.292+0000 7f8485eb9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800ae020 con 0x7f8480067830 2023-10-06T05:21:00.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f848015a490 0x7f848015b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58768/0 (socket says 172.21.15.45:58768) 2023-10-06T05:21:00.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847f7fe700 1 -- 172.21.15.45:0/269295966 learned_addr learned my addr 172.21.15.45:0/269295966 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:00.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847effd700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 0x7f848006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973362496 0 0) 0x7f8470001110 con 0x7f848015a490 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8468003570 con 0x7f848015a490 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154668687 0 0) 0x7f84800ab7d0 con 0x7f848006add0 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8470001110 con 0x7f848006add0 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640068140 0 0) 0x7f8468003570 con 0x7f848015a490 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 msgr2=0x7f848006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 0x7f848006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 >> v1:172.21.15.144:6789/0 conn(0x7f848006add0 legacy=0x7f84800abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480067e80 con 0x7f848015a490 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f8470001600 con 0x7f848015a490 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 >> v1:172.21.15.45:6789/0 conn(0x7f848015a490 legacy=0x7f848015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8468003c20 0x7f8468004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 0x7f8468004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8468004d50 0x7f84680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800ae020 con 0x7f8468003c20 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846800bbf0 con 0x7f8468004d50 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846800bd60 con 0x7f8480067830 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.293+0000 7f847f7fe700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8468003c20 0x7f8468004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847effd700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 0x7f8468004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847ffff700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8468004d50 0x7f84680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847f7fe700 1 -- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 msgr2=0x7f8468004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847f7fe700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8480067830 0x7f8468004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847f7fe700 1 -- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8468004d50 msgr2=0x7f84680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847f7fe700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8468004d50 0x7f84680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847f7fe700 1 -- 172.21.15.45:0/269295966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468003570 con 0x7f8468003c20 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847f7fe700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8468003c20 0x7f8468004010 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8470001ed0 tx=0x7f8470007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfc43e2a1a9dd90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000cc50 con 0x7f8468003c20 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847000cdd0 con 0x7f8468003c20 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.294+0000 7f847e7fc700 1 -- 172.21.15.45:0/269295966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470005b70 con 0x7f8468003c20 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.295+0000 7f8485eb9700 1 -- 172.21.15.45:0/269295966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8468003c20 msgr2=0x7f8468004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.295+0000 7f8485eb9700 1 --2- 172.21.15.45:0/269295966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8468003c20 0x7f8468004010 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8470001ed0 tx=0x7f8470007940 comp rx=0 tx=0).stop 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.295+0000 7f847ffff700 1 -- 172.21.15.45:0/269295966 reap_dead start 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.295+0000 7f8485eb9700 1 -- 172.21.15.45:0/269295966 shutdown_connections 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.295+0000 7f8485eb9700 1 -- 172.21.15.45:0/269295966 >> 172.21.15.45:0/269295966 conn(0x7f84800cc5a0 msgr2=0x7f84800cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.295+0000 7f8485eb9700 1 -- 172.21.15.45:0/269295966 shutdown_connections 2023-10-06T05:21:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.295+0000 7f8485eb9700 1 -- 172.21.15.45:0/269295966 wait complete. 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.296+0000 7f8485eb9700 1 Processor -- start 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.296+0000 7f8485eb9700 1 -- start start 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f8485eb9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8468003c20 0x7f84800c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f8485eb9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848014a780 0x7f84800c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f8485eb9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848016cd60 0x7f848016d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f8485eb9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8480067ca0 con 0x7f848014a780 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f8485eb9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848016d740 con 0x7f848016cd60 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f8485eb9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800a9150 con 0x7f8468003c20 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f847effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848014a780 0x7f84800c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f847effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848014a780 0x7f84800c02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45428/0 (socket says 172.21.15.45:45428) 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f847effd700 1 -- 172.21.15.45:0/2957570545 learned_addr learned my addr 172.21.15.45:0/2957570545 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f847ffff700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848016cd60 0x7f848016d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f847f7fe700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8468003c20 0x7f84800c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.297+0000 7f847effd700 1 -- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8468003c20 msgr2=0x7f84800c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f847effd700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8468003c20 0x7f84800c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f847effd700 1 -- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848016cd60 msgr2=0x7f848016d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f847effd700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848016cd60 0x7f848016d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f847effd700 1 -- 172.21.15.45:0/2957570545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800c0800 con 0x7f848014a780 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f847effd700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848014a780 0x7f84800c02c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8474011da0 tx=0x7f84740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f84677fe700 1 -- 172.21.15.45:0/2957570545 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474009e20 con 0x7f848014a780 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f84677fe700 1 -- 172.21.15.45:0/2957570545 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847400fcd0 con 0x7f848014a780 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84800c7c10 con 0x7f848014a780 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f84677fe700 1 -- 172.21.15.45:0/2957570545 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474014650 con 0x7f848014a780 2023-10-06T05:21:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.298+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800c81d0 con 0x7f848014a780 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.300+0000 7f84677fe700 1 -- 172.21.15.45:0/2957570545 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8474014a80 con 0x7f848014a780 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.300+0000 7f84677fe700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f846c066f30 0x7f846c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.300+0000 7f84677fe700 1 -- 172.21.15.45:0/2957570545 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8474018030 con 0x7f848014a780 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.300+0000 7f847f7fe700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f846c066f30 0x7f846c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.301+0000 7f847f7fe700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f846c066f30 0x7f846c0693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8470008fd0 tx=0x7f8470015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.302+0000 7f848f1cdb80 1 Processor -- start 2023-10-06T05:21:00.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.303+0000 7f848f1cdb80 1 -- start start 2023-10-06T05:21:00.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.303+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096489e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.303+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x5609648a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.303+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x5609648a7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.303+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560964771310 con 0x5609648a74d0 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.303+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609647ae130 con 0x560964883f40 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.303+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609647abaf0 con 0x56096489e450 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x5609648a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x5609648a7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45444/0 (socket says 172.21.15.45:45444) 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 -- 172.21.15.45:0/2339907361 learned_addr learned my addr 172.21.15.45:0/2339907361 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847effd700 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x5609648a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847f7fe700 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096489e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 -- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 msgr2=0x56096489e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096489e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 -- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 msgr2=0x5609648a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x5609648a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 -- 172.21.15.45:0/2339907361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609645ad790 con 0x5609648a74d0 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847effd700 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x5609648a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f847ffff700 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x5609648a7940 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f848006dbe0 tx=0x7f84780017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72236cbf1f10402 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f8464ff9700 1 -- 172.21.15.45:0/2339907361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478001ab0 con 0x5609648a74d0 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f8464ff9700 1 -- 172.21.15.45:0/2339907361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478001c10 con 0x5609648a74d0 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.304+0000 7f8464ff9700 1 -- 172.21.15.45:0/2339907361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478010710 con 0x5609648a74d0 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2339907361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 msgr2=0x5609648a7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x5609648a7940 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f848006dbe0 tx=0x7f84780017a0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2339907361 shutdown_connections 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x5609648a7940 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096489e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2339907361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x5609648a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2339907361 >> 172.21.15.45:0/2339907361 conn(0x5609648a1880 msgr2=0x5609648a1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2339907361 shutdown_connections 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.305+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2339907361 wait complete. 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.306+0000 7f848f1cdb80 1 -- start start 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.306+0000 7f848f1cdb80 1 Processor -- start 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.307+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x560964879560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.307+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x560964877bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.307+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x560964858af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.307+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609647a78e0 con 0x5609648a74d0 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.307+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560964753980 con 0x560964883f40 2023-10-06T05:21:00.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.307+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x560964753620 con 0x56096489e450 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.307+0000 7f848f1cdb80 10 client.0 _ll_register_callbacks cb 0x5609645b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x560964858af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x560964858af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45452/0 (socket says 172.21.15.45:45452) 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 -- 172.21.15.45:0/2321118651 learned_addr learned my addr 172.21.15.45:0/2321118651 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847f7fe700 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x560964879560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847effd700 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x560964877bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 -- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 msgr2=0x560964877bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x560964877bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 -- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 msgr2=0x560964879560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x560964879560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 -- 172.21.15.45:0/2321118651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800c81d0 con 0x5609648a74d0 2023-10-06T05:21:00.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f847ffff700 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x560964858af0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f84800636f0 tx=0x7f8478019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66613d19da616721 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f845dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f845dffb700 1 -- 172.21.15.45:0/2321118651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478010710 con 0x5609648a74d0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f845dffb700 1 -- 172.21.15.45:0/2321118651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8478006ba0 con 0x5609648a74d0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56096484bd60 con 0x5609648a74d0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f845dffb700 1 -- 172.21.15.45:0/2321118651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8478002030 con 0x5609648a74d0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.308+0000 7f848f1cdb80 10 client.6834 fetch_fsmap learned FSMap version 71 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x7f847800f040 con 0x5609648a74d0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f845dffb700 1 -- 172.21.15.45:0/2321118651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56096484bd60 con 0x5609648a74d0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f845dffb700 1 -- 172.21.15.45:0/2321118651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f847801b070 con 0x5609648a74d0 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f848f1cdb80 10 client.6834 fetch_fsmap finished waiting for FSMap version 71 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f848f1cdb80 10 client.6834 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:21:00.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f848f1cdb80 20 client.6834 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:00.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x56096487d7a0 0x56096487db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f848f1cdb80 4 client.6834 mds_command: new command op to 6702 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:21:00.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 --> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560964828b30 con 0x56096487d7a0 2023-10-06T05:21:00.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f847f7fe700 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x56096487d7a0 0x56096487db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f845dffb700 10 client.6834 ms_handle_connect on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:00.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.309+0000 7f847f7fe700 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x56096487d7a0 0x56096487db70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35bdf98bbfd66ee server_cookie=f050f65065762d69 in_seq=0 out_seq=0 2023-10-06T05:21:00.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f845dffb700 1 -- 172.21.15.45:0/2321118651 <== mds.0 v2:172.21.15.45:6832/1530850582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560964828b30 con 0x56096487d7a0 2023-10-06T05:21:00.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f845dffb700 10 client.6834 handle_command_reply: tid=0 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 client.6834 shutdown 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 msgr2=0x560964858af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x560964858af0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f84800636f0 tx=0x7f8478019000 comp rx=0 tx=0).stop 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 shutdown_connections 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x56096487d7a0 0x56096487db70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609648a74d0 0x560964858af0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x560964879560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f845dffb700 0 client.6834 ms_handle_reset on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/2321118651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x560964877bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 >> 172.21.15.45:0/2321118651 conn(0x5609648a1880 msgr2=0x560964853c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 shutdown_connections 2023-10-06T05:21:00.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.310+0000 7f848f1cdb80 1 -- 172.21.15.45:0/2321118651 wait complete. 2023-10-06T05:21:00.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.311+0000 7f848f1cdb80 20 client.6834 trim_cache size 0 max 16384 2023-10-06T05:21:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.328+0000 7f848f1cdb80 1 Processor -- start 2023-10-06T05:21:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.328+0000 7f848f1cdb80 1 -- start start 2023-10-06T05:21:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x56096484d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56096489e450 0x56096484ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609648a74d0 0x56096487fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096463f1e0 con 0x56096489e450 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096484d510 con 0x560964883f40 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096487a960 con 0x5609648a74d0 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f847effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56096489e450 0x56096484ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f847effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56096489e450 0x56096484ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45458/0 (socket says 172.21.15.45:45458) 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f847effd700 1 -- 172.21.15.45:0/3357121190 learned_addr learned my addr 172.21.15.45:0/3357121190 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f847f7fe700 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x56096484d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.329+0000 7f847ffff700 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609648a74d0 0x56096487fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f847effd700 1 -- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609648a74d0 msgr2=0x56096487fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f847effd700 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609648a74d0 0x56096487fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f847effd700 1 -- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 msgr2=0x56096484d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f847effd700 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x56096484d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f847effd700 1 -- 172.21.15.45:0/3357121190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096469a0a0 con 0x56096489e450 2023-10-06T05:21:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f847effd700 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56096489e450 0x56096484ddb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84740524c0 tx=0x7f847405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcb93f6dd93a072 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f84657fa700 1 -- 172.21.15.45:0/3357121190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740850f0 con 0x56096489e450 2023-10-06T05:21:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f84657fa700 1 -- 172.21.15.45:0/3357121190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8474055fc0 con 0x56096489e450 2023-10-06T05:21:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f84657fa700 1 -- 172.21.15.45:0/3357121190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474059210 con 0x56096489e450 2023-10-06T05:21:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f848f1cdb80 1 -- 172.21.15.45:0/3357121190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56096489e450 msgr2=0x56096484ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56096489e450 0x56096484ddb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84740524c0 tx=0x7f847405b040 comp rx=0 tx=0).stop 2023-10-06T05:21:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f848f1cdb80 1 -- 172.21.15.45:0/3357121190 shutdown_connections 2023-10-06T05:21:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609648a74d0 0x56096487fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560964883f40 0x56096484d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/3357121190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56096489e450 0x56096484ddb0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.330+0000 7f848f1cdb80 1 -- 172.21.15.45:0/3357121190 >> 172.21.15.45:0/3357121190 conn(0x5609648a1880 msgr2=0x5609648547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.331+0000 7f848f1cdb80 1 -- 172.21.15.45:0/3357121190 shutdown_connections 2023-10-06T05:21:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.331+0000 7f848f1cdb80 1 -- 172.21.15.45:0/3357121190 wait complete. 2023-10-06T05:21:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.331+0000 7f848f1cdb80 1 -- start start 2023-10-06T05:21:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.331+0000 7f848f1cdb80 1 Processor -- start 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560964883f40 0x560964902500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096487fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f848f1cdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609648a74d0 0x5609648800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609648871e0 con 0x560964883f40 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560964857e30 con 0x5609648a74d0 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f848f1cdb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609648a4580 con 0x56096489e450 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f848f1cdb80 10 client.0 _ll_register_callbacks cb 0x5609645b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560964883f40 0x560964902500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560964883f40 0x560964902500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45472/0 (socket says 172.21.15.45:45472) 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 -- 172.21.15.45:0/275921818 learned_addr learned my addr 172.21.15.45:0/275921818 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:00.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847ffff700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609648a74d0 0x5609648800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847effd700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096487fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 -- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 msgr2=0x56096487fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096487fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 -- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609648a74d0 msgr2=0x5609648800c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609648a74d0 0x5609648800c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847f7fe700 1 -- 172.21.15.45:0/275921818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800c7c10 con 0x560964883f40 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847ffff700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609648a74d0 0x5609648800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.332+0000 7f847effd700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096487fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f847f7fe700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560964883f40 0x560964902500 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84680057a0 tx=0x7f8470012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93af854081eee92d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f845dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f845dffb700 1 -- 172.21.15.45:0/275921818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847002e070 con 0x560964883f40 2023-10-06T05:21:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f845dffb700 1 -- 172.21.15.45:0/275921818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8470007070 con 0x560964883f40 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8478001c10 con 0x560964883f40 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f845dffb700 1 -- 172.21.15.45:0/275921818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8470007440 con 0x560964883f40 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 10 client.6840 fetch_fsmap learned FSMap version 71 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5609647763c0 con 0x560964883f40 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f845dffb700 1 -- 172.21.15.45:0/275921818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478001c10 con 0x560964883f40 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f845dffb700 1 -- 172.21.15.45:0/275921818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f84700075a0 con 0x560964883f40 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 10 client.6840 fetch_fsmap finished waiting for FSMap version 71 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 10 client.6840 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 20 client.6840 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x5609648a36d0 0x5609648a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 4 client.6840 mds_command: new command op to 6702 tid=0[{"prefix": "status"}] 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 --> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56096474a8d0 con 0x5609648a36d0 2023-10-06T05:21:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.333+0000 7f847effd700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x5609648a36d0 0x5609648a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f847effd700 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x5609648a36d0 0x5609648a3aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3b92d78f5f77ad server_cookie=67b2ec0d8884d704 in_seq=0 out_seq=0 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f845dffb700 10 client.6840 ms_handle_connect on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f845dffb700 1 -- 172.21.15.45:0/275921818 <== mds.0 v2:172.21.15.45:6832/1530850582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56096474a8d0 con 0x5609648a36d0 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f845dffb700 10 client.6840 handle_command_reply: tid=0 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 client.6840 shutdown 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560964883f40 msgr2=0x560964902500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560964883f40 0x560964902500 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84680057a0 tx=0x7f8470012000 comp rx=0 tx=0).stop 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 shutdown_connections 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x5609648a36d0 0x5609648a3aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609648a74d0 0x5609648800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560964883f40 0x560964902500 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 --2- 172.21.15.45:0/275921818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56096489e450 0x56096487fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f845dffb700 0 client.6840 ms_handle_reset on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 >> 172.21.15.45:0/275921818 conn(0x5609648a1880 msgr2=0x56096484d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 shutdown_connections 2023-10-06T05:21:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 1 -- 172.21.15.45:0/275921818 wait complete. 2023-10-06T05:21:00.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.334+0000 7f848f1cdb80 20 client.6840 trim_cache size 0 max 16384 2023-10-06T05:21:00.346 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6702, 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.0983025609999997, 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 71, 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 79, 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 79, 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 12.083402148999999 2023-10-06T05:21:00.347 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f846c066f30 msgr2=0x7f846c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f846c066f30 0x7f846c0693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8470008fd0 tx=0x7f8470015000 comp rx=0 tx=0).stop 2023-10-06T05:21:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848014a780 msgr2=0x7f84800c02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848014a780 0x7f84800c02c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8474011da0 tx=0x7f84740091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 shutdown_connections 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f846c066f30 0x7f846c0693e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848016cd60 0x7f848016d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8468003c20 0x7f84800c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 --2- 172.21.15.45:0/2957570545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848014a780 0x7f84800c02c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 >> 172.21.15.45:0/2957570545 conn(0x7f84800cc5a0 msgr2=0x7f84800af350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 shutdown_connections 2023-10-06T05:21:00.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.335+0000 7f8485eb9700 1 -- 172.21.15.45:0/2957570545 wait complete. 2023-10-06T05:21:00.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:21:00.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.480+0000 7f1aa7891700 1 Processor -- start 2023-10-06T05:21:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.481+0000 7f1aa7891700 1 -- start start 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.481+0000 7f1aa7891700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1aa0155cd0 0x7f1aa015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.481+0000 7f1aa7891700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa00ae8d0 con 0x7f1aa00ccdc0 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.481+0000 7f1aa7891700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa00aebb0 con 0x7f1aa01560a0 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.481+0000 7f1aa7891700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa00ae6b0 con 0x7f1aa0155cd0 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1aa562d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1aa0155cd0 0x7f1aa015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1aa5e2e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1aa00ccdc0 0x7f1aa00cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58780/0 (socket says 172.21.15.45:58780) 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1aa5e2e700 1 -- 172.21.15.45:0/3226505952 learned_addr learned my addr 172.21.15.45:0/3226505952 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1aa562d700 1 -- 172.21.15.45:0/3226505952 >> v1:172.21.15.144:6789/0 conn(0x7f1aa01560a0 legacy=0x7f1aa00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1aa562d700 1 -- 172.21.15.45:0/3226505952 >> v1:172.21.15.45:6789/0 conn(0x7f1aa00ccdc0 legacy=0x7f1aa00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1aa562d700 1 -- 172.21.15.45:0/3226505952 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa00aee90 con 0x7f1aa0155cd0 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1aa562d700 1 --2- 172.21.15.45:0/3226505952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa015a730 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098e0 tx=0x7f1a9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e299d95e3adacba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1a97fff700 1 -- 172.21.15.45:0/3226505952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c009e80 con 0x7f1aa0155cd0 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1a97fff700 1 -- 172.21.15.45:0/3226505952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c00b860 con 0x7f1aa0155cd0 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.482+0000 7f1a97fff700 1 -- 172.21.15.45:0/3226505952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00bbe0 con 0x7f1aa0155cd0 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.483+0000 7f1aa7891700 1 -- 172.21.15.45:0/3226505952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 msgr2=0x7f1aa015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.483+0000 7f1aa7891700 1 --2- 172.21.15.45:0/3226505952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa015a730 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098e0 tx=0x7f1a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.483+0000 7f1aa7891700 1 -- 172.21.15.45:0/3226505952 shutdown_connections 2023-10-06T05:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.483+0000 7f1aa7891700 1 --2- 172.21.15.45:0/3226505952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa015a730 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.483+0000 7f1aa7891700 1 -- 172.21.15.45:0/3226505952 >> 172.21.15.45:0/3226505952 conn(0x7f1aa006f090 msgr2=0x7f1aa006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.483+0000 7f1aa7891700 1 -- 172.21.15.45:0/3226505952 shutdown_connections 2023-10-06T05:21:00.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.483+0000 7f1aa7891700 1 -- 172.21.15.45:0/3226505952 wait complete. 2023-10-06T05:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.484+0000 7f1aa7891700 1 Processor -- start 2023-10-06T05:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.484+0000 7f1aa7891700 1 -- start start 2023-10-06T05:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa7891700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa00ccdc0 0x7f1aa016a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa7891700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa0158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa7891700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa01560a0 0x7f1aa016c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa7891700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa01597f0 con 0x7f1aa0155cd0 2023-10-06T05:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa7891700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa016a6f0 con 0x7f1aa00ccdc0 2023-10-06T05:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa7891700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0158990 con 0x7f1aa01560a0 2023-10-06T05:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa4e2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa0158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa4e2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa0158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45494/0 (socket says 172.21.15.45:45494) 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa4e2c700 1 -- 172.21.15.45:0/340024929 learned_addr learned my addr 172.21.15.45:0/340024929 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa562d700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa00ccdc0 0x7f1aa016a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.485+0000 7f1aa5e2e700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa01560a0 0x7f1aa016c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa4e2c700 1 -- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa01560a0 msgr2=0x7f1aa016c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa4e2c700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa01560a0 0x7f1aa016c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa4e2c700 1 -- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa00ccdc0 msgr2=0x7f1aa016a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa4e2c700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa00ccdc0 0x7f1aa016a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa4e2c700 1 -- 172.21.15.45:0/340024929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c009420 con 0x7f1aa0155cd0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa4e2c700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa0158450 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c590 tx=0x7f1a9000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1a957fa700 1 -- 172.21.15.45:0/340024929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90011590 con 0x7f1aa0155cd0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1a957fa700 1 -- 172.21.15.45:0/340024929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a90011710 con 0x7f1aa0155cd0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1a957fa700 1 -- 172.21.15.45:0/340024929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90011ad0 con 0x7f1aa0155cd0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa0158be0 con 0x7f1aa0155cd0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.486+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa0159480 con 0x7f1aa0155cd0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.487+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0158fd0 con 0x7f1aa0155cd0 2023-10-06T05:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.488+0000 7f1a957fa700 1 -- 172.21.15.45:0/340024929 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a9001b3b0 con 0x7f1aa0155cd0 2023-10-06T05:21:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.488+0000 7f1a957fa700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.488+0000 7f1a957fa700 1 -- 172.21.15.45:0/340024929 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1a90083c40 con 0x7f1aa0155cd0 2023-10-06T05:21:00.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.488+0000 7f1aa562d700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.489+0000 7f1aa562d700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0093f0 tx=0x7f1a9c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.493+0000 7f1a957fa700 1 -- 172.21.15.45:0/340024929 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a90050c10 con 0x7f1aa0155cd0 2023-10-06T05:21:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.742+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1aa00af170 con 0x7f1aa0155cd0 2023-10-06T05:21:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.742+0000 7f1a957fa700 1 -- 172.21.15.45:0/340024929 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1a90017070 con 0x7f1aa0155cd0 2023-10-06T05:21:00.745 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:21:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.744+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 msgr2=0x7f1a840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.744+0000 7f1aa7891700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0093f0 tx=0x7f1a9c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.744+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 msgr2=0x7f1aa0158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.744+0000 7f1aa7891700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa0158450 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c590 tx=0x7f1a9000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 shutdown_connections 2023-10-06T05:21:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa01560a0 0x7f1aa016c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa00ccdc0 0x7f1aa016a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 --2- 172.21.15.45:0/340024929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa0155cd0 0x7f1aa0158450 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 >> 172.21.15.45:0/340024929 conn(0x7f1aa006f090 msgr2=0x7f1aa00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 shutdown_connections 2023-10-06T05:21:00.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:00.745+0000 7f1aa7891700 1 -- 172.21.15.45:0/340024929 wait complete. 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:21:00.760 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:00.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:00.761 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:21:00.783 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:21:00.783 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:21:00.784 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42080sec preferred_lft 42080sec 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:00.785 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:21:00.786 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:00.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:00.787 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:21:00.855 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:21:00.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:00.857 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:21:00.921 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:21:00.921 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:21:00.921 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:21:00.921 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:21:00.921 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:21:00.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:00.922 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:21:00.922 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:21:00.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:21:00.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:21:00.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:21:00.922 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:21:01.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:01.045 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:21:01.067 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:21:01.067 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:21:01.068 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:01.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:01.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:21:01.143 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:21:01.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:01.144 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:01.207 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:01.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:01.207 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:01.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:21:01.348 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:21:01.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:21:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:21:01.424 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:21:01.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:01.425 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:21:01.488 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:21:01.488 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:21:01.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:21:01.565 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:21:01.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:21:01.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:21:01.630+0000 7f8a60ebf540 -1 init, newargv = 0x56390bc5b000 newargc=15 2023-10-06T05:21:01.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[150983]: starting ceph client 2023-10-06T05:21:01.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[150983]: starting fuse 2023-10-06T05:21:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:21:01.641 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:21:01.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:01.642 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:21:01.704 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:21:01.704 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:21:01.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:21:01.704 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:21:01.704 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:21:01.704 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:01.705 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:21:01.706 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:21:01.844 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.150983.asok 2023-10-06T05:21:02.026 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.150983.asok 2023-10-06T05:21:02.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:02.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.150983.asok status 2023-10-06T05:21:02.201 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:02.201 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "150983", 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:21:02.202 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6855, 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 6855 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4005977135 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:02.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4005977135 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.6855 192.168.0.1:0/4005977135", 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/4005977135", 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 71, 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 79, 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:21:02.204 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:21:02.205 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:21:02.205 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:02.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:02.211 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:21:02.211 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:02.232 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:21:02.232 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:02.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:02.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:02.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.486+0000 7f3d85c93700 1 Processor -- start 2023-10-06T05:21:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.486+0000 7f3d85c93700 1 -- start start 2023-10-06T05:21:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d85c93700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d800c8ff0 0x7f3d800c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d85c93700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8015b000 con 0x7f3d80157200 2023-10-06T05:21:02.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d85c93700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8015b2e0 con 0x7f3d800caad0 2023-10-06T05:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d85c93700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8015b5c0 con 0x7f3d800c8ff0 2023-10-06T05:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d800c8ff0 0x7f3d800c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3d80157200 0x7f3d800cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58788/0 (socket says 172.21.15.45:58788) 2023-10-06T05:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d800c8ff0 0x7f3d800c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45498/0 (socket says 172.21.15.45:45498) 2023-10-06T05:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7ffff700 1 -- 172.21.15.45:0/2938535160 learned_addr learned my addr 172.21.15.45:0/2938535160 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7f7fe700 1 -- 172.21.15.45:0/2938535160 >> v1:172.21.15.144:6789/0 conn(0x7f3d800caad0 legacy=0x7f3d80156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7f7fe700 1 -- 172.21.15.45:0/2938535160 >> v1:172.21.15.45:6789/0 conn(0x7f3d80157200 legacy=0x7f3d800cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7e7fc700 1 -- 172.21.15.45:0/2938535160 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917833882 0 0) 0x7f3d8015b000 con 0x7f3d80157200 2023-10-06T05:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7f7fe700 1 -- 172.21.15.45:0/2938535160 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8015b8a0 con 0x7f3d800c8ff0 2023-10-06T05:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.487+0000 7f3d7f7fe700 1 --2- 172.21.15.45:0/2938535160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800c8ff0 0x7f3d800c93c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3d700098e0 tx=0x7f3d700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32191717523e43e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d7e7fc700 1 -- 172.21.15.45:0/2938535160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70009e80 con 0x7f3d800c8ff0 2023-10-06T05:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d7e7fc700 1 -- 172.21.15.45:0/2938535160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7000b860 con 0x7f3d800c8ff0 2023-10-06T05:21:02.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d7e7fc700 1 -- 172.21.15.45:0/2938535160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000bbe0 con 0x7f3d800c8ff0 2023-10-06T05:21:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d85c93700 1 -- 172.21.15.45:0/2938535160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800c8ff0 msgr2=0x7f3d800c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d85c93700 1 --2- 172.21.15.45:0/2938535160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800c8ff0 0x7f3d800c93c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3d700098e0 tx=0x7f3d700091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d85c93700 1 -- 172.21.15.45:0/2938535160 shutdown_connections 2023-10-06T05:21:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d85c93700 1 --2- 172.21.15.45:0/2938535160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800c8ff0 0x7f3d800c93c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d85c93700 1 -- 172.21.15.45:0/2938535160 >> 172.21.15.45:0/2938535160 conn(0x7f3d800cdf50 msgr2=0x7f3d800ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d85c93700 1 -- 172.21.15.45:0/2938535160 shutdown_connections 2023-10-06T05:21:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.488+0000 7f3d85c93700 1 -- 172.21.15.45:0/2938535160 wait complete. 2023-10-06T05:21:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 Processor -- start 2023-10-06T05:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 -- start start 2023-10-06T05:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d800c8ff0 0x7f3d8016a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800caad0 0x7f3d8006a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d80157200 0x7f3d80157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8016a780 con 0x7f3d800caad0 2023-10-06T05:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d801514c0 con 0x7f3d80157200 2023-10-06T05:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.490+0000 7f3d85c93700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d80159b20 con 0x7f3d800c8ff0 2023-10-06T05:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800caad0 0x7f3d8006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800caad0 0x7f3d8006a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45508/0 (socket says 172.21.15.45:45508) 2023-10-06T05:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 -- 172.21.15.45:0/1663782299 learned_addr learned my addr 172.21.15.45:0/1663782299 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:02.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7f7fe700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d800c8ff0 0x7f3d8016a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7ffff700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d80157200 0x7f3d80157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 -- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d800c8ff0 msgr2=0x7f3d8016a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d800c8ff0 0x7f3d8016a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 -- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d80157200 msgr2=0x7f3d80157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d80157200 0x7f3d80157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 -- 172.21.15.45:0/1663782299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70009420 con 0x7f3d800caad0 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d7effd700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800caad0 0x7f3d8006a180 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c590 tx=0x7f3d7400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d677fe700 1 -- 172.21.15.45:0/1663782299 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74011590 con 0x7f3d800caad0 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d677fe700 1 -- 172.21.15.45:0/1663782299 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d740116f0 con 0x7f3d800caad0 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d80159e00 con 0x7f3d800caad0 2023-10-06T05:21:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d677fe700 1 -- 172.21.15.45:0/1663782299 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74011ab0 con 0x7f3d800caad0 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.491+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8015a3c0 con 0x7f3d800caad0 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.492+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8006f410 con 0x7f3d800caad0 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.493+0000 7f3d677fe700 1 -- 172.21.15.45:0/1663782299 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d7401b3b0 con 0x7f3d800caad0 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.493+0000 7f3d677fe700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d6c066f30 0x7f3d6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.494+0000 7f3d677fe700 1 -- 172.21.15.45:0/1663782299 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3d74083d60 con 0x7f3d800caad0 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.494+0000 7f3d7f7fe700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d6c066f30 0x7f3d6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.494+0000 7f3d7f7fe700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d6c066f30 0x7f3d6c0693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3d700093f0 tx=0x7f3d70017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.498+0000 7f3d677fe700 1 -- 172.21.15.45:0/1663782299 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d74050d30 con 0x7f3d800caad0 2023-10-06T05:21:02.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.747+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d8004bb40 con 0x7f3d800caad0 2023-10-06T05:21:02.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.748+0000 7f3d677fe700 1 -- 172.21.15.45:0/1663782299 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f3d74017070 con 0x7f3d800caad0 2023-10-06T05:21:02.750 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:02.750 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:20:57.257456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1530850582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1530850582},{"type":"v1","addr":"172.21.15.45:6834","nonce":1530850582}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:02.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d6c066f30 msgr2=0x7f3d6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d6c066f30 0x7f3d6c0693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3d700093f0 tx=0x7f3d70017000 comp rx=0 tx=0).stop 2023-10-06T05:21:02.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800caad0 msgr2=0x7f3d8006a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800caad0 0x7f3d8006a180 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c590 tx=0x7f3d7400c950 comp rx=0 tx=0).stop 2023-10-06T05:21:02.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 shutdown_connections 2023-10-06T05:21:02.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d6c066f30 0x7f3d6c0693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d80157200 0x7f3d80157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d800c8ff0 0x7f3d8016a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 --2- 172.21.15.45:0/1663782299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800caad0 0x7f3d8006a180 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 >> 172.21.15.45:0/1663782299 conn(0x7f3d800cdf50 msgr2=0x7f3d800ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 shutdown_connections 2023-10-06T05:21:02.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.750+0000 7f3d85c93700 1 -- 172.21.15.45:0/1663782299 wait complete. 2023-10-06T05:21:02.755 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 71 2023-10-06T05:21:02.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2023-10-06T05:21:02.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.898+0000 7f8bca721700 1 Processor -- start 2023-10-06T05:21:02.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.898+0000 7f8bca721700 1 -- start start 2023-10-06T05:21:02.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bca721700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8bc40c3db0 0x7f8bc40c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bca721700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc40cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bca721700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc40ce6a0 con 0x7f8bc40b0610 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bca721700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40690f0 con 0x7f8bc40c3db0 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bca721700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40ceeb0 con 0x7f8bc4069ed0 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc8cbe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc40cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc37fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8bc40c3db0 0x7f8bc40c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc3fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8bc40b0610 0x7f8bc40697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:41286/0 (socket says 172.21.15.45:41286) 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc3fff700 1 -- 172.21.15.45:0/3089448384 learned_addr learned my addr 172.21.15.45:0/3089448384 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc2ffd700 1 -- 172.21.15.45:0/3089448384 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414668846 0 0) 0x7f8bc40ce6a0 con 0x7f8bc40b0610 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc2ffd700 1 -- 172.21.15.45:0/3089448384 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bac003570 con 0x7f8bc40b0610 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc37fe700 1 -- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 msgr2=0x7f8bc40cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc37fe700 1 --2- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc40cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc37fe700 1 -- 172.21.15.45:0/3089448384 >> v1:172.21.15.144:6789/0 conn(0x7f8bc40b0610 legacy=0x7f8bc40697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc37fe700 1 -- 172.21.15.45:0/3089448384 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4159880 con 0x7f8bc40c3db0 2023-10-06T05:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.899+0000 7f8bc8cbe700 1 --2- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc40cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bc37fe700 1 --2- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c85d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009910 tx=0x7f8bb80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2206d392543d8467 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bc2ffd700 1 -- 172.21.15.45:0/3089448384 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8011630 con 0x7f8bc40c3db0 2023-10-06T05:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bc2ffd700 1 -- 172.21.15.45:0/3089448384 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb8011790 con 0x7f8bc40c3db0 2023-10-06T05:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bc2ffd700 1 -- 172.21.15.45:0/3089448384 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8011b30 con 0x7f8bc40c3db0 2023-10-06T05:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bca721700 1 -- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40c3db0 msgr2=0x7f8bc40c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bca721700 1 --2- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c85d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009910 tx=0x7f8bb80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bca721700 1 -- 172.21.15.45:0/3089448384 shutdown_connections 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bca721700 1 --2- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc40cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bca721700 1 --2- 172.21.15.45:0/3089448384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c85d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bca721700 1 -- 172.21.15.45:0/3089448384 >> 172.21.15.45:0/3089448384 conn(0x7f8bc4067e00 msgr2=0x7f8bc40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.900+0000 7f8bca721700 1 -- 172.21.15.45:0/3089448384 shutdown_connections 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.901+0000 7f8bca721700 1 -- 172.21.15.45:0/3089448384 wait complete. 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.902+0000 7f8bca721700 1 Processor -- start 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.902+0000 7f8bca721700 1 -- start start 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bca721700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc414ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bca721700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b0610 0x7f8bc409b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bca721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bca721700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40c5d00 con 0x7f8bc40c3db0 2023-10-06T05:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bca721700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40c5fe0 con 0x7f8bc40b0610 2023-10-06T05:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bca721700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc409a8a0 con 0x7f8bc4069ed0 2023-10-06T05:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45514/0 (socket says 172.21.15.45:45514) 2023-10-06T05:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 -- 172.21.15.45:0/1019125130 learned_addr learned my addr 172.21.15.45:0/1019125130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc37fe700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b0610 0x7f8bc409b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc3fff700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc414ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 -- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 msgr2=0x7f8bc414ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc414ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 -- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b0610 msgr2=0x7f8bc409b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b0610 0x7f8bc409b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 -- 172.21.15.45:0/1019125130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc009a20 con 0x7f8bc40c3db0 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc37fe700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b0610 0x7f8bc409b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc3fff700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc414ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.903+0000 7f8bc8cbe700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c6c00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009fc0 tx=0x7f8bbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.904+0000 7f8babfff700 1 -- 172.21.15.45:0/1019125130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00cd30 con 0x7f8bc40c3db0 2023-10-06T05:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.904+0000 7f8babfff700 1 -- 172.21.15.45:0/1019125130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc00f040 con 0x7f8bc40c3db0 2023-10-06T05:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.904+0000 7f8babfff700 1 -- 172.21.15.45:0/1019125130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0146d0 con 0x7f8bc40c3db0 2023-10-06T05:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.904+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb8009420 con 0x7f8bc40c3db0 2023-10-06T05:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.904+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc409ae60 con 0x7f8bc40c3db0 2023-10-06T05:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.905+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc409a700 con 0x7f8bc40c3db0 2023-10-06T05:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.905+0000 7f8babfff700 1 -- 172.21.15.45:0/1019125130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bbc003020 con 0x7f8bc40c3db0 2023-10-06T05:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.905+0000 7f8babfff700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bb0066f30 0x7f8bb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.906+0000 7f8babfff700 1 -- 172.21.15.45:0/1019125130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8bbc085260 con 0x7f8bc40c3db0 2023-10-06T05:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.906+0000 7f8bc3fff700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bb0066f30 0x7f8bb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.906+0000 7f8bc3fff700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bb0066f30 0x7f8bb00693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4005920 tx=0x7f8bb4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:02.910+0000 7f8babfff700 1 -- 172.21.15.45:0/1019125130 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bc409a700 con 0x7f8bc40c3db0 2023-10-06T05:21:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.197+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) v1 -- 0x7f8bc409a700 con 0x7f8bc40c3db0 2023-10-06T05:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.598+0000 7f8babfff700 1 -- 172.21.15.45:0/1019125130 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.6702 v72) v1 ==== 112+0+0 (secure 0 0 0) 0x7f8bc409a700 con 0x7f8bc40c3db0 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bb0066f30 msgr2=0x7f8bb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bb0066f30 0x7f8bb00693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4005920 tx=0x7f8bb4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc40c3db0 msgr2=0x7f8bc40c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c6c00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009fc0 tx=0x7f8bbc00b040 comp rx=0 tx=0).stop 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 shutdown_connections 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bb0066f30 0x7f8bb00693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc40c3db0 0x7f8bc40c6c00 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4069ed0 0x7f8bc414ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 --2- 172.21.15.45:0/1019125130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b0610 0x7f8bc409b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.601+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 >> 172.21.15.45:0/1019125130 conn(0x7f8bc4067e00 msgr2=0x7f8bc40cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.602+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 shutdown_connections 2023-10-06T05:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.602+0000 7f8bca721700 1 -- 172.21.15.45:0/1019125130 wait complete. 2023-10-06T05:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:freezing mds.6702 2023-10-06T05:21:03.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:03.619 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p a/{b,c}/d') 2023-10-06T05:21:03.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2023-10-06T05:21:03.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.807+0000 7fd0c60df700 1 Processor -- start 2023-10-06T05:21:03.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.807+0000 7fd0c60df700 1 -- start start 2023-10-06T05:21:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0c60df700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0c0140ab0 0x7fd0c00acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0c60df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c00ad4e0 0x7fd0c00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0c60df700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c00c9940 con 0x7fd0c01406e0 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0c60df700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c006d230 con 0x7fd0c0140ab0 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0c60df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c006d510 con 0x7fd0c00ad4e0 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0beffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0c0140ab0 0x7fd0c00acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0beffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0c0140ab0 0x7fd0c00acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45524/0 (socket says 172.21.15.45:45524) 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0beffd700 1 -- 172.21.15.45:0/1174628849 learned_addr learned my addr 172.21.15.45:0/1174628849 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0bffff700 1 --2- 172.21.15.45:0/1174628849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c00ad4e0 0x7fd0c00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0beffd700 1 -- 172.21.15.45:0/1174628849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c00ad4e0 msgr2=0x7fd0c00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1174628849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c00ad4e0 0x7fd0c00c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0beffd700 1 -- 172.21.15.45:0/1174628849 >> v1:172.21.15.144:6789/0 conn(0x7fd0c01406e0 legacy=0x7fd0c006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0beffd700 1 -- 172.21.15.45:0/1174628849 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c006d7f0 con 0x7fd0c0140ab0 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.808+0000 7fd0bffff700 1 --2- 172.21.15.45:0/1174628849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c00ad4e0 0x7fd0c00c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1174628849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0140ab0 0x7fd0c00acfa0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80099b0 tx=0x7fd0a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0a2a58a29c8baa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0be7fc700 1 -- 172.21.15.45:0/1174628849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800b860 con 0x7fd0c0140ab0 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0be7fc700 1 -- 172.21.15.45:0/1174628849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a800b9c0 con 0x7fd0c0140ab0 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0be7fc700 1 -- 172.21.15.45:0/1174628849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800bd60 con 0x7fd0c0140ab0 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 -- 172.21.15.45:0/1174628849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0140ab0 msgr2=0x7fd0c00acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1174628849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0140ab0 0x7fd0c00acfa0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80099b0 tx=0x7fd0a8009250 comp rx=0 tx=0).stop 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 -- 172.21.15.45:0/1174628849 shutdown_connections 2023-10-06T05:21:03.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1174628849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c00ad4e0 0x7fd0c00c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1174628849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0140ab0 0x7fd0c00acfa0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 -- 172.21.15.45:0/1174628849 >> 172.21.15.45:0/1174628849 conn(0x7fd0c00691c0 msgr2=0x7fd0c00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 -- 172.21.15.45:0/1174628849 shutdown_connections 2023-10-06T05:21:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.809+0000 7fd0c60df700 1 -- 172.21.15.45:0/1174628849 wait complete. 2023-10-06T05:21:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.811+0000 7fd0c60df700 1 Processor -- start 2023-10-06T05:21:03.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.811+0000 7fd0c60df700 1 -- start start 2023-10-06T05:21:03.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.811+0000 7fd0c60df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c00ad4e0 0x7fd0c013fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.811+0000 7fd0c60df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c01406e0 0x7fd0c00a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.811+0000 7fd0c60df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c0140ab0 0x7fd0c0169440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.811+0000 7fd0c60df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00c3ef0 con 0x7fd0c01406e0 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.811+0000 7fd0c60df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00c4140 con 0x7fd0c00ad4e0 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0c60df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00c4420 con 0x7fd0c0140ab0 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c01406e0 0x7fd0c00a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c01406e0 0x7fd0c00a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45536/0 (socket says 172.21.15.45:45536) 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 -- 172.21.15.45:0/1232969587 learned_addr learned my addr 172.21.15.45:0/1232969587 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c00ad4e0 0x7fd0c013fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0bffff700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c0140ab0 0x7fd0c0169440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 -- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c0140ab0 msgr2=0x7fd0c0169440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c0140ab0 0x7fd0c0169440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 -- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c00ad4e0 msgr2=0x7fd0c013fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c00ad4e0 0x7fd0c013fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 -- 172.21.15.45:0/1232969587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a80094f0 con 0x7fd0c01406e0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c01406e0 0x7fd0c00a9270 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009290 tx=0x7fd0a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0b77fe700 1 -- 172.21.15.45:0/1232969587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a801a450 con 0x7fd0c01406e0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0b77fe700 1 -- 172.21.15.45:0/1232969587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a801a5b0 con 0x7fd0c01406e0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c00c4700 con 0x7fd0c01406e0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.812+0000 7fd0b77fe700 1 -- 172.21.15.45:0/1232969587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a801a970 con 0x7fd0c01406e0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.813+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c00c67f0 con 0x7fd0c01406e0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.815+0000 7fd0b77fe700 1 -- 172.21.15.45:0/1232969587 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0a801ad20 con 0x7fd0c01406e0 2023-10-06T05:21:03.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.816+0000 7fd0b77fe700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.816+0000 7fd0b77fe700 1 -- 172.21.15.45:0/1232969587 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd0a8085e20 con 0x7fd0c01406e0 2023-10-06T05:21:03.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.816+0000 7fd0cf3f3b80 1 Processor -- start 2023-10-06T05:21:03.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.816+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0cf3f3b80 1 -- start start 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0003860 tx=0x7fd0b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed848e7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed847b1360 con 0x55ed848c3f90 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed847ee180 con 0x55ed848de4a0 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.817+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed847ebb40 con 0x55ed848e7520 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0bf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0bffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed848e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45552/0 (socket says 172.21.15.45:45552) 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0bffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed848e7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:36938/0 (socket says 172.21.15.45:36938) 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 -- 172.21.15.45:0/873730609 learned_addr learned my addr 172.21.15.45:0/873730609 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 -- 172.21.15.45:0/873730609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 msgr2=0x55ed848e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 --2- 172.21.15.45:0/873730609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed848e7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 -- 172.21.15.45:0/873730609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 msgr2=0x55ed848de870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 --2- 172.21.15.45:0/873730609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848de870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 -- 172.21.15.45:0/873730609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed845ed7e0 con 0x55ed848c3f90 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0beffd700 1 --2- 172.21.15.45:0/873730609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848e6fe0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8012fd0 tx=0x7fd0a8083f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4024e2fb6b0f64e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0b4ff9700 1 -- 172.21.15.45:0/873730609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8057110 con 0x55ed848c3f90 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0b4ff9700 1 -- 172.21.15.45:0/873730609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8056a10 con 0x55ed848c3f90 2023-10-06T05:21:03.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.818+0000 7fd0b4ff9700 1 -- 172.21.15.45:0/873730609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8059c60 con 0x55ed848c3f90 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/873730609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 msgr2=0x55ed848e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/873730609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848e6fe0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8012fd0 tx=0x7fd0a8083f60 comp rx=0 tx=0).stop 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/873730609 shutdown_connections 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/873730609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed848e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/873730609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848de870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/873730609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848e6fe0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/873730609 >> 172.21.15.45:0/873730609 conn(0x55ed848e18d0 msgr2=0x55ed848e1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/873730609 shutdown_connections 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.819+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/873730609 wait complete. 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.820+0000 7fd0cf3f3b80 1 -- start start 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.820+0000 7fd0cf3f3b80 1 Processor -- start 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed848ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed848b90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed847e7930 con 0x55ed848de4a0 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed847939d0 con 0x55ed848e7520 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed84793670 con 0x55ed848c3f90 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0cf3f3b80 10 client.0 _ll_register_callbacks cb 0x55ed845f6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed848ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed848ddd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45558/0 (socket says 172.21.15.45:45558) 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 -- 172.21.15.45:0/3226032421 learned_addr learned my addr 172.21.15.45:0/3226032421 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0bffff700 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed848b90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 -- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 msgr2=0x55ed848e12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848e12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 -- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 msgr2=0x55ed848b90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed848b90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 -- 172.21.15.45:0/3226032421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c00c67f0 con 0x55ed848de4a0 2023-10-06T05:21:03.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd0beffd700 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed848ddd30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8006e40 tx=0x7fd0a806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac32bf17be9a230 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:03.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.821+0000 7fd09dffb700 1 -- 172.21.15.45:0/3226032421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8082070 con 0x55ed848de4a0 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd09dffb700 1 -- 172.21.15.45:0/3226032421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd0a8057110 con 0x55ed848de4a0 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed8488bdb0 con 0x55ed848de4a0 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd09dffb700 1 -- 172.21.15.45:0/3226032421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a805d9b0 con 0x55ed848de4a0 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 10 client.6879 fetch_fsmap learned FSMap version 72 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55ed846da0f0 con 0x55ed848de4a0 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd09dffb700 1 -- 172.21.15.45:0/3226032421 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ed8488bdb0 con 0x55ed848de4a0 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd09dffb700 1 -- 172.21.15.45:0/3226032421 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 72) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd0a805d520 con 0x55ed848de4a0 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 10 client.6879 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 10 client.6879 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-10-06T05:21:03.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 20 client.6879 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed84899220 0x55ed848df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 4 client.6879 mds_command: new command op to 6702 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 --> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed84868b80 con 0x55ed84899220 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.822+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed84899220 0x55ed848df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed84899220 0x55ed848df7d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2447a44725c67a09 server_cookie=850124d4082f7541 in_seq=0 out_seq=0 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd09dffb700 10 client.6879 ms_handle_connect on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd09dffb700 1 -- 172.21.15.45:0/3226032421 <== mds.0 v2:172.21.15.45:6832/1530850582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ed84868b80 con 0x55ed84899220 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd09dffb700 10 client.6879 handle_command_reply: tid=0 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 client.6879 shutdown 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 msgr2=0x55ed848ddd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed848ddd30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8006e40 tx=0x7fd0a806a000 comp rx=0 tx=0).stop 2023-10-06T05:21:03.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 shutdown_connections 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed84899220 0x55ed848df7d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed848b90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd09dffb700 0 client.6879 ms_handle_reset on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/3226032421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed848ddd30 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.823+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 >> 172.21.15.45:0/3226032421 conn(0x55ed848e18d0 msgr2=0x55ed8489a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.824+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 shutdown_connections 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.824+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/3226032421 wait complete. 2023-10-06T05:21:03.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.824+0000 7fd0cf3f3b80 20 client.6879 trim_cache size 0 max 16384 2023-10-06T05:21:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.838+0000 7fd0cf3f3b80 1 Processor -- start 2023-10-06T05:21:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.838+0000 7fd0cf3f3b80 1 -- start start 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed8488d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed8488de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8467f230 con 0x55ed848de4a0 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed848c1ba0 con 0x55ed848e7520 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed848ddf70 con 0x55ed848c3f90 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed8488d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed8488d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45564/0 (socket says 172.21.15.45:45564) 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 -- 172.21.15.45:0/794821138 learned_addr learned my addr 172.21.15.45:0/794821138 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:03.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0bffff700 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed8488de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 -- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 msgr2=0x55ed848c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 -- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 msgr2=0x55ed8488de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed8488de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.839+0000 7fd0beffd700 1 -- 172.21.15.45:0/794821138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed847b6410 con 0x55ed848de4a0 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0beffd700 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed8488d8c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8055000 tx=0x7fd0a8070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170b4e48f2545564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0b57fa700 1 -- 172.21.15.45:0/794821138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8082050 con 0x55ed848de4a0 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0b57fa700 1 -- 172.21.15.45:0/794821138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8057110 con 0x55ed848de4a0 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0b57fa700 1 -- 172.21.15.45:0/794821138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8059c60 con 0x55ed848de4a0 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/794821138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 msgr2=0x55ed8488d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed8488d8c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8055000 tx=0x7fd0a8070000 comp rx=0 tx=0).stop 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/794821138 shutdown_connections 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848e7520 0x55ed8488de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848c3f90 0x55ed848c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/794821138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848de4a0 0x55ed8488d8c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/794821138 >> 172.21.15.45:0/794821138 conn(0x55ed848e18d0 msgr2=0x55ed848e2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/794821138 shutdown_connections 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.840+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/794821138 wait complete. 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.841+0000 7fd0cf3f3b80 1 -- start start 2023-10-06T05:21:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.841+0000 7fd0cf3f3b80 1 Processor -- start 2023-10-06T05:21:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.842+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.842+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848b9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.842+0000 7fd0cf3f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed8488d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.842+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed848bf710 con 0x55ed848c3f90 2023-10-06T05:21:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.842+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed84942b30 con 0x55ed848de4a0 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.842+0000 7fd0cf3f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed848b5220 con 0x55ed848e7520 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.842+0000 7fd0cf3f3b80 10 client.0 _ll_register_callbacks cb 0x55ed845f6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848bf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45570/0 (socket says 172.21.15.45:45570) 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 -- 172.21.15.45:0/1143064827 learned_addr learned my addr 172.21.15.45:0/1143064827 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bffff700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed8488d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 -- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 msgr2=0x55ed8488d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed8488d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 -- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 msgr2=0x55ed848b9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848b9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 -- 172.21.15.45:0/1143064827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed846da0f0 con 0x55ed848c3f90 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bffff700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed8488d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0bf7fe700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848bf1b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0003190 tx=0x7fd0b0027250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e494d13a3a39fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd09dffb700 1 -- 172.21.15.45:0/1143064827 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0027070 con 0x55ed848c3f90 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd09dffb700 1 -- 172.21.15.45:0/1143064827 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd0b001b040 con 0x55ed848c3f90 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0a8056a10 con 0x55ed848c3f90 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd09dffb700 1 -- 172.21.15.45:0/1143064827 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0b000eed0 con 0x55ed848c3f90 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0cf3f3b80 10 client.6885 fetch_fsmap learned FSMap version 72 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.843+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55ed8464e220 con 0x55ed848c3f90 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd09dffb700 1 -- 172.21.15.45:0/1143064827 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8056a10 con 0x55ed848c3f90 2023-10-06T05:21:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd09dffb700 1 -- 172.21.15.45:0/1143064827 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 72) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd0b0013840 con 0x55ed848c3f90 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0cf3f3b80 10 client.6885 fetch_fsmap finished waiting for FSMap version 72 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0cf3f3b80 10 client.6885 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0cf3f3b80 20 client.6885 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed8493f740 0x55ed8488de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0cf3f3b80 4 client.6885 mds_command: new command op to 6702 tid=0[{"prefix": "flush journal"}] 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 --> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed8478a920 con 0x55ed8493f740 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed8493f740 0x55ed8488de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd09dffb700 10 client.6885 ms_handle_connect on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.844+0000 7fd0beffd700 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed8493f740 0x55ed8488de00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7922ca495193da3f server_cookie=4d453b89437ba675 in_seq=0 out_seq=0 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.858+0000 7fd09dffb700 1 -- 172.21.15.45:0/1143064827 <== mds.0 v2:172.21.15.45:6832/1530850582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed8478a920 con 0x55ed8493f740 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.858+0000 7fd09dffb700 10 client.6885 handle_command_reply: tid=0 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.858+0000 7fd0cf3f3b80 1 client.6885 shutdown 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 msgr2=0x55ed848bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848bf1b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0003190 tx=0x7fd0b0027250 comp rx=0 tx=0).stop 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 shutdown_connections 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:6832/1530850582,v1:172.21.15.45:6834/1530850582] conn(0x55ed8493f740 0x55ed8488de00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ed848e7520 0x55ed8488d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed848c3f90 0x55ed848bf1b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd09dffb700 0 client.6885 ms_handle_reset on v2:172.21.15.45:6832/1530850582 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 --2- 172.21.15.45:0/1143064827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed848de4a0 0x55ed848b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 >> 172.21.15.45:0/1143064827 conn(0x55ed848e18d0 msgr2=0x55ed8489a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 shutdown_connections 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 1 -- 172.21.15.45:0/1143064827 wait complete. 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.859+0000 7fd0cf3f3b80 20 client.6885 trim_cache size 0 max 16384 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:03.862 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2023-10-06T05:21:03.863 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2023-10-06T05:21:03.863 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:03.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 msgr2=0x7fd0a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0003860 tx=0x7fd0b0009040 comp rx=0 tx=0).stop 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c01406e0 msgr2=0x7fd0c00a9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c01406e0 0x7fd0c00a9270 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009290 tx=0x7fd0a8017000 comp rx=0 tx=0).stop 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 shutdown_connections 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c0140ab0 0x7fd0c0169440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c00ad4e0 0x7fd0c013fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 --2- 172.21.15.45:0/1232969587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c01406e0 0x7fd0c00a9270 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 >> 172.21.15.45:0/1232969587 conn(0x7fd0c00691c0 msgr2=0x7fd0c0067620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 shutdown_connections 2023-10-06T05:21:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:03.860+0000 7fd0c60df700 1 -- 172.21.15.45:0/1232969587 wait complete. 2023-10-06T05:21:03.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2023-10-06T05:21:04.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.009+0000 7f9406dd4700 1 Processor -- start 2023-10-06T05:21:04.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.009+0000 7f9406dd4700 1 -- start start 2023-10-06T05:21:04.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.009+0000 7f9406dd4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94000d0680 0x7f940015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.009+0000 7f9406dd4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f940015c6f0 0x7f94000caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.009+0000 7f9406dd4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94000cd740 con 0x7f94000d02b0 2023-10-06T05:21:04.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.009+0000 7f9406dd4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94000d0ad0 con 0x7f94000d0680 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.009+0000 7f9406dd4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000ccb30 con 0x7f940015c6f0 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9404b70700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f94000d02b0 0x7f940015d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58802/0 (socket says 172.21.15.45:58802) 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9404b70700 1 -- 172.21.15.45:0/4194420929 learned_addr learned my addr 172.21.15.45:0/4194420929 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9405371700 1 --2- 172.21.15.45:0/4194420929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f940015c6f0 0x7f94000caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f93fffff700 1 --2- 172.21.15.45:0/4194420929 >> v2:172.21.15.144:3300/0 conn(0x7f94000d0680 0x7f940015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f93ff7fe700 1 -- 172.21.15.45:0/4194420929 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269699456 0 0) 0x7f94000cd740 con 0x7f94000d02b0 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f93ff7fe700 1 -- 172.21.15.45:0/4194420929 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e8003570 con 0x7f94000d02b0 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9405371700 1 -- 172.21.15.45:0/4194420929 >> v2:172.21.15.144:3300/0 conn(0x7f94000d0680 msgr2=0x7f940015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9405371700 1 --2- 172.21.15.45:0/4194420929 >> v2:172.21.15.144:3300/0 conn(0x7f94000d0680 0x7f940015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9405371700 1 -- 172.21.15.45:0/4194420929 >> v1:172.21.15.45:6789/0 conn(0x7f94000d02b0 legacy=0x7f940015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9405371700 1 -- 172.21.15.45:0/4194420929 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000cce10 con 0x7f940015c6f0 2023-10-06T05:21:04.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f93fffff700 1 --2- 172.21.15.45:0/4194420929 >> v2:172.21.15.144:3300/0 conn(0x7f94000d0680 0x7f940015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:04.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.010+0000 7f9405371700 1 --2- 172.21.15.45:0/4194420929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f940015c6f0 0x7f94000caf10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009e60 tx=0x7f93f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c186f2cdff5f18b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f93ff7fe700 1 -- 172.21.15.45:0/4194420929 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800e070 con 0x7f940015c6f0 2023-10-06T05:21:04.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f93ff7fe700 1 -- 172.21.15.45:0/4194420929 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800bab0 con 0x7f940015c6f0 2023-10-06T05:21:04.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f93ff7fe700 1 -- 172.21.15.45:0/4194420929 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800be30 con 0x7f940015c6f0 2023-10-06T05:21:04.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 -- 172.21.15.45:0/4194420929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f940015c6f0 msgr2=0x7f94000caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 --2- 172.21.15.45:0/4194420929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f940015c6f0 0x7f94000caf10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009e60 tx=0x7f93f8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:04.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 -- 172.21.15.45:0/4194420929 shutdown_connections 2023-10-06T05:21:04.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 --2- 172.21.15.45:0/4194420929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f940015c6f0 0x7f94000caf10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 --2- 172.21.15.45:0/4194420929 >> v2:172.21.15.144:3300/0 conn(0x7f94000d0680 0x7f940015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 -- 172.21.15.45:0/4194420929 >> 172.21.15.45:0/4194420929 conn(0x7f940006ed50 msgr2=0x7f940006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 -- 172.21.15.45:0/4194420929 shutdown_connections 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.011+0000 7f9406dd4700 1 -- 172.21.15.45:0/4194420929 wait complete. 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.012+0000 7f9406dd4700 1 Processor -- start 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.013+0000 7f9406dd4700 1 -- start start 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.013+0000 7f9406dd4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94000d02b0 0x7f94000c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.013+0000 7f9406dd4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94000d0680 0x7f9400169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.013+0000 7f9406dd4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f940015c6f0 0x7f94000c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.013+0000 7f9406dd4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94000b3010 con 0x7f940015c6f0 2023-10-06T05:21:04.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.013+0000 7f9406dd4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94000b32f0 con 0x7f94000d0680 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9406dd4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000c8860 con 0x7f94000d02b0 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f940015c6f0 0x7f94000c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f940015c6f0 0x7f94000c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45580/0 (socket says 172.21.15.45:45580) 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 -- 172.21.15.45:0/406729827 learned_addr learned my addr 172.21.15.45:0/406729827 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f93fffff700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94000d0680 0x7f9400169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9404b70700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94000d02b0 0x7f94000c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 -- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94000d02b0 msgr2=0x7f94000c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94000d02b0 0x7f94000c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 -- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94000d0680 msgr2=0x7f9400169680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94000d0680 0x7f9400169680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 -- 172.21.15.45:0/406729827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009420 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9405371700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f940015c6f0 0x7f94000c7170 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f93f800b610 tx=0x7f93f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f93fcff9700 1 -- 172.21.15.45:0/406729827 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8027470 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f93fcff9700 1 -- 172.21.15.45:0/406729827 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f80275d0 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f8009a20 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f93fcff9700 1 -- 172.21.15.45:0/406729827 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8027990 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.014+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940016a180 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.015+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9400088730 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.016+0000 7f93fcff9700 1 -- 172.21.15.45:0/406729827 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93f8024070 con 0x7f940015c6f0 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.016+0000 7f93fcff9700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ec066f30 0x7f93ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.016+0000 7f93fcff9700 1 -- 172.21.15.45:0/406729827 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f93f8087160 con 0x7f940015c6f0 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.017+0000 7f9404b70700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ec066f30 0x7f93ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.017+0000 7f9404b70700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ec066f30 0x7f93ec0693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f940015b560 tx=0x7f93f0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.021+0000 7f93fcff9700 1 -- 172.21.15.45:0/406729827 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93f8054130 con 0x7f940015c6f0 2023-10-06T05:21:04.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.221+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) v1 -- 0x7f94000cd0f0 con 0x7f940015c6f0 2023-10-06T05:21:04.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.224+0000 7f93fcff9700 1 -- 172.21.15.45:0/406729827 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f93f8057750 con 0x7f940015c6f0 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ec066f30 msgr2=0x7f93ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ec066f30 0x7f93ec0693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f940015b560 tx=0x7f93f0009040 comp rx=0 tx=0).stop 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f940015c6f0 msgr2=0x7f94000c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f940015c6f0 0x7f94000c7170 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f93f800b610 tx=0x7f93f8019000 comp rx=0 tx=0).stop 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 shutdown_connections 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93ec066f30 0x7f93ec0693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f940015c6f0 0x7f94000c7170 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94000d02b0 0x7f94000c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 --2- 172.21.15.45:0/406729827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94000d0680 0x7f9400169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 >> 172.21.15.45:0/406729827 conn(0x7f940006ed50 msgr2=0x7f940015d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 shutdown_connections 2023-10-06T05:21:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:04.227+0000 7f9406dd4700 1 -- 172.21.15.45:0/406729827 wait complete. 2023-10-06T05:21:07.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:07.320 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:09.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:09.244 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'timeout 60 mv a/b a/z') 2023-10-06T05:21:09.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-10-06T05:21:09.294 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:21:09.293+0000 7f6b2379a700 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5560c446bc00) (dversion lock w=1 last_client=6855) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5560c4506a00] 2023-10-06T05:21:09.294 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:21:09.293+0000 7f6b2379a700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5560c446bc00) (dversion lock w=1 last_client=6855) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5560c4506a00] 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f6b2379a700 time 2023-10-06T05:21:09.294037+0000 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6b2bfb9afd] 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.296 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: clone() 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr:*** Caught signal (Aborted) ** 2023-10-06T05:21:09.297 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f6b2379a700 thread_name:ms_dispatch 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:21:09.294+0000 7f6b2379a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f6b2379a700 time 2023-10-06T05:21:09.294037+0000 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6b2bfb9afd] 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.298 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: clone() 2023-10-06T05:21:09.299 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6b2afb1cf0] 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f6b2bfb9bbd] 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.300 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: clone() 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:21:09.296+0000 7f6b2379a700 -1 *** Caught signal (Aborted) ** 2023-10-06T05:21:09.301 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f6b2379a700 thread_name:ms_dispatch 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6b2afb1cf0] 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f6b2bfb9bbd] 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.302 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: clone() 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T05:21:09.303 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.304 INFO:tasks.ceph.mds.a.smithi045.stderr: -2883> 2023-10-06T05:20:59.217+0000 7f6b20f95700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:09.306 INFO:tasks.ceph.mds.a.smithi045.stderr: -4> 2023-10-06T05:21:09.293+0000 7f6b2379a700 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5560c446bc00) (dversion lock w=1 last_client=6855) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5560c4506a00] 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: -2> 2023-10-06T05:21:09.293+0000 7f6b2379a700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5560c446bc00) (dversion lock w=1 last_client=6855) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5560c4506a00] 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: -1> 2023-10-06T05:21:09.294+0000 7f6b2379a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f6b2379a700 time 2023-10-06T05:21:09.294037+0000 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6b2bfb9afd] 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.307 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: clone() 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: 0> 2023-10-06T05:21:09.296+0000 7f6b2379a700 -1 *** Caught signal (Aborted) ** 2023-10-06T05:21:09.308 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f6b2379a700 thread_name:ms_dispatch 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6b2afb1cf0] 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f6b2bfb9bbd] 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.309 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: clone() 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T05:21:09.310 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.311 INFO:tasks.ceph.mds.a.smithi045.stderr: -2883> 2023-10-06T05:20:59.217+0000 7f6b20f95700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: -4> 2023-10-06T05:21:09.293+0000 7f6b2379a700 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5560c446bc00) (dversion lock w=1 last_client=6855) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5560c4506a00] 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: -2> 2023-10-06T05:21:09.293+0000 7f6b2379a700 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by 0x5560c446bc00) (dversion lock w=1 last_client=6855) pv=11 v=10 ap=2 ino=(nil) state=1073741888|bottomlru | request=1 lock=2 authpin=1 0x5560c4506a00] 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: -1> 2023-10-06T05:21:09.294+0000 7f6b2379a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: In function 'bool CDentry::check_corruption(bool)' thread 7f6b2379a700 time 2023-10-06T05:21:09.294037+0000 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.6-1274-g68e53a4a/rpm/el8/BUILD/ceph-17.2.6-1274-g68e53a4a/src/mds/CDentry.cc: 703: ceph_abort_msg("abort() called") 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6b2bfb9afd] 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.313 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.314 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.314 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.314 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.314 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.314 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.314 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.314 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: clone() 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 0> 2023-10-06T05:21:09.296+0000 7f6b2379a700 -1 *** Caught signal (Aborted) ** 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: in thread 7f6b2379a700 thread_name:ms_dispatch 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f6b2afb1cf0] 2023-10-06T05:21:09.315 INFO:tasks.ceph.mds.a.smithi045.stderr: 2: gsignal() 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 3: abort() 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x7f6b2bfb9bbd] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 5: (CDentry::check_corruption(bool)+0x9a2) [0x5560c1f06402] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 6: (EMetaBlob::add_primary_dentry(EMetaBlob::dirlump&, CDentry*, CInode*, unsigned char)+0x47) [0x5560c1d62be7] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 7: (Server::_rename_prepare(boost::intrusive_ptr&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2685) [0x5560c1d34525] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 8: (Server::handle_client_rename(boost::intrusive_ptr&)+0x25d4) [0x5560c1d37be4] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0x6d5) [0x5560c1d4b285] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x5560c1d4ff2c] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5db) [0x5560c1c9824b] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5560c1c9888c] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5560c1c81c8f] 2023-10-06T05:21:09.316 INFO:tasks.ceph.mds.a.smithi045.stderr: 14: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f6b2c234e28] 2023-10-06T05:21:09.317 INFO:tasks.ceph.mds.a.smithi045.stderr: 15: (DispatchQueue::entry()+0x50f) [0x7f6b2c23226f] 2023-10-06T05:21:09.317 INFO:tasks.ceph.mds.a.smithi045.stderr: 16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6b2c2fc681] 2023-10-06T05:21:09.317 INFO:tasks.ceph.mds.a.smithi045.stderr: 17: /lib64/libpthread.so.0(+0x81ca) [0x7f6b2afa71ca] 2023-10-06T05:21:09.317 INFO:tasks.ceph.mds.a.smithi045.stderr: 18: clone() 2023-10-06T05:21:09.317 INFO:tasks.ceph.mds.a.smithi045.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T05:21:09.317 INFO:tasks.ceph.mds.a.smithi045.stderr: 2023-10-06T05:21:09.437 INFO:teuthology.orchestra.run.smithi045.stdout:{"mds_beacon_grace":"15.000000"} 2023-10-06T05:21:09.437 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-10-06T05:21:09.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.609+0000 7f80b8e36700 1 Processor -- start 2023-10-06T05:21:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.609+0000 7f80b8e36700 1 -- start start 2023-10-06T05:21:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.610+0000 7f80b8e36700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80b4155cd0 0x7f80b415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.610+0000 7f80b8e36700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40ccdc0 0x7f80b40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.610+0000 7f80b8e36700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40cd740 con 0x7f80b41560a0 2023-10-06T05:21:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.610+0000 7f80b8e36700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b40ae6b0 con 0x7f80b4155cd0 2023-10-06T05:21:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.610+0000 7f80b8e36700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40ae8d0 con 0x7f80b40ccdc0 2023-10-06T05:21:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80b4155cd0 0x7f80b415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80b4155cd0 0x7f80b415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57522/0 (socket says 172.21.15.45:57522) 2023-10-06T05:21:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 -- 172.21.15.45:0/2406289232 learned_addr learned my addr 172.21.15.45:0/2406289232 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b2d9d700 1 --2- 172.21.15.45:0/2406289232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40ccdc0 0x7f80b40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 -- 172.21.15.45:0/2406289232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40ccdc0 msgr2=0x7f80b40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 --2- 172.21.15.45:0/2406289232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40ccdc0 0x7f80b40cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 -- 172.21.15.45:0/2406289232 >> v1:172.21.15.144:6789/0 conn(0x7f80b41560a0 legacy=0x7f80b40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 -- 172.21.15.45:0/2406289232 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b40aebb0 con 0x7f80b4155cd0 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b259c700 1 --2- 172.21.15.45:0/2406289232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b4155cd0 0x7f80b415a730 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f80a400bc50 tx=0x7f80a400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54668631dc4fbda0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b159a700 1 -- 172.21.15.45:0/2406289232 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400dbf0 con 0x7f80b4155cd0 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b159a700 1 -- 172.21.15.45:0/2406289232 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a400dd50 con 0x7f80b4155cd0 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.611+0000 7f80b159a700 1 -- 172.21.15.45:0/2406289232 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40196b0 con 0x7f80b4155cd0 2023-10-06T05:21:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 -- 172.21.15.45:0/2406289232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b4155cd0 msgr2=0x7f80b415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 --2- 172.21.15.45:0/2406289232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b4155cd0 0x7f80b415a730 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f80a400bc50 tx=0x7f80a400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 -- 172.21.15.45:0/2406289232 shutdown_connections 2023-10-06T05:21:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 --2- 172.21.15.45:0/2406289232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40ccdc0 0x7f80b40cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 --2- 172.21.15.45:0/2406289232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b4155cd0 0x7f80b415a730 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 -- 172.21.15.45:0/2406289232 >> 172.21.15.45:0/2406289232 conn(0x7f80b406f090 msgr2=0x7f80b406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 -- 172.21.15.45:0/2406289232 shutdown_connections 2023-10-06T05:21:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.612+0000 7f80b8e36700 1 -- 172.21.15.45:0/2406289232 wait complete. 2023-10-06T05:21:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.613+0000 7f80b8e36700 1 Processor -- start 2023-10-06T05:21:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.613+0000 7f80b8e36700 1 -- start start 2023-10-06T05:21:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b8e36700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b40ccdc0 0x7f80b4151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b8e36700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b4155cd0 0x7f80b40b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b8e36700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b41560a0 0x7f80b40b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b8e36700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4151a10 con 0x7f80b41560a0 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b8e36700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b414f6c0 con 0x7f80b40ccdc0 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b8e36700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4150e00 con 0x7f80b4155cd0 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b41560a0 0x7f80b40b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b40ccdc0 0x7f80b4151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b41560a0 0x7f80b40b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57534/0 (socket says 172.21.15.45:57534) 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b2d9d700 1 -- 172.21.15.45:0/227030317 learned_addr learned my addr 172.21.15.45:0/227030317 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.614+0000 7f80b1d9b700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b4155cd0 0x7f80b40b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b2d9d700 1 -- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b4155cd0 msgr2=0x7f80b40b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b2d9d700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b4155cd0 0x7f80b40b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b2d9d700 1 -- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b40ccdc0 msgr2=0x7f80b4151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b2d9d700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b40ccdc0 0x7f80b4151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b2d9d700 1 -- 172.21.15.45:0/227030317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a400b790 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b2d9d700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b41560a0 0x7f80b40b2590 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009e90 tx=0x7f80ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80a27fc700 1 -- 172.21.15.45:0/227030317 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00cb70 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80a27fc700 1 -- 172.21.15.45:0/227030317 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac00f040 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b41510e0 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80a27fc700 1 -- 172.21.15.45:0/227030317 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac014480 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.615+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b41516a0 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.616+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b406e2f0 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.616+0000 7f80a27fc700 1 -- 172.21.15.45:0/227030317 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80ac014830 con 0x7f80b41560a0 2023-10-06T05:21:09.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.617+0000 7f80a27fc700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8094066f30 0x7f80940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.617+0000 7f80a27fc700 1 -- 172.21.15.45:0/227030317 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f80ac085200 con 0x7f80b41560a0 2023-10-06T05:21:09.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.617+0000 7f80b259c700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8094066f30 0x7f80940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.618+0000 7f80b259c700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8094066f30 0x7f80940693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f80a400bc50 tx=0x7f80a401b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.621+0000 7f80a27fc700 1 -- 172.21.15.45:0/227030317 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80ac0521d0 con 0x7f80b41560a0 2023-10-06T05:21:09.659 INFO:tasks.ceph.mds.a.smithi045.stderr:daemon-helper: command crashed with signal 6 2023-10-06T05:21:09.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.877+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80b40aee90 con 0x7f80b41560a0 2023-10-06T05:21:09.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.878+0000 7f80a27fc700 1 -- 172.21.15.45:0/227030317 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f80ac0557f0 con 0x7f80b41560a0 2023-10-06T05:21:09.880 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:09.880 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:03.589791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1530850582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1530850582},{"type":"v1","addr":"172.21.15.45:6834","nonce":1530850582}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8094066f30 msgr2=0x7f80940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8094066f30 0x7f80940693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f80a400bc50 tx=0x7f80a401b000 comp rx=0 tx=0).stop 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b41560a0 msgr2=0x7f80b40b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b41560a0 0x7f80b40b2590 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009e90 tx=0x7f80ac00b040 comp rx=0 tx=0).stop 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 shutdown_connections 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8094066f30 0x7f80940693e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b41560a0 0x7f80b40b2590 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b40ccdc0 0x7f80b4151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 --2- 172.21.15.45:0/227030317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b4155cd0 0x7f80b40b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 >> 172.21.15.45:0/227030317 conn(0x7f80b406f090 msgr2=0x7f80b40cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 shutdown_connections 2023-10-06T05:21:09.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:09.880+0000 7f80b8e36700 1 -- 172.21.15.45:0/227030317 wait complete. 2023-10-06T05:21:09.884 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 72 2023-10-06T05:21:09.896 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-10-06T05:21:10.884 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T05:21:14.896 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:15.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.036+0000 7f8ca68e1700 1 Processor -- start 2023-10-06T05:21:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.036+0000 7f8ca68e1700 1 -- start start 2023-10-06T05:21:15.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.036+0000 7f8ca68e1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca00c8ff0 0x7f8ca00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.036+0000 7f8ca68e1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8ca00caad0 0x7f8ca00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.036+0000 7f8ca68e1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ca0156af0 0x7f8ca0156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.036+0000 7f8ca68e1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca015b000 con 0x7f8ca0156af0 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca68e1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca015b2e0 con 0x7f8ca00caad0 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca68e1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca015b5c0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ca0156af0 0x7f8ca0156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ca0156af0 0x7f8ca0156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57542/0 (socket says 172.21.15.45:57542) 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 -- 172.21.15.45:0/489057976 learned_addr learned my addr 172.21.15.45:0/489057976 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8c9ffff700 1 --2- 172.21.15.45:0/489057976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca00c8ff0 0x7f8ca00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8c9f7fe700 1 --2- 172.21.15.45:0/489057976 >> v2:172.21.15.144:3300/0 conn(0x7f8ca00caad0 0x7f8ca00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 -- 172.21.15.45:0/489057976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca00c8ff0 msgr2=0x7f8ca00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 --2- 172.21.15.45:0/489057976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca00c8ff0 0x7f8ca00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 -- 172.21.15.45:0/489057976 >> v2:172.21.15.144:3300/0 conn(0x7f8ca00caad0 msgr2=0x7f8ca00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 --2- 172.21.15.45:0/489057976 >> v2:172.21.15.144:3300/0 conn(0x7f8ca00caad0 0x7f8ca00caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 -- 172.21.15.45:0/489057976 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca015b8a0 con 0x7f8ca0156af0 2023-10-06T05:21:15.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8ca4e7e700 1 --2- 172.21.15.45:0/489057976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0156af0 0x7f8ca0156f70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00c4fb0 tx=0x7f8c98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59107ed92fe90c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8c9effd700 1 -- 172.21.15.45:0/489057976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800e070 con 0x7f8ca0156af0 2023-10-06T05:21:15.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8c9effd700 1 -- 172.21.15.45:0/489057976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9800b970 con 0x7f8ca0156af0 2023-10-06T05:21:15.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.037+0000 7f8c9effd700 1 -- 172.21.15.45:0/489057976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980092e0 con 0x7f8ca0156af0 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 -- 172.21.15.45:0/489057976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0156af0 msgr2=0x7f8ca0156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/489057976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0156af0 0x7f8ca0156f70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00c4fb0 tx=0x7f8c98009840 comp rx=0 tx=0).stop 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 -- 172.21.15.45:0/489057976 shutdown_connections 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/489057976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0156af0 0x7f8ca0156f70 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/489057976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca00c8ff0 0x7f8ca00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/489057976 >> v2:172.21.15.144:3300/0 conn(0x7f8ca00caad0 0x7f8ca00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 -- 172.21.15.45:0/489057976 >> 172.21.15.45:0/489057976 conn(0x7f8ca00cdf50 msgr2=0x7f8ca00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 -- 172.21.15.45:0/489057976 shutdown_connections 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.038+0000 7f8ca68e1700 1 -- 172.21.15.45:0/489057976 wait complete. 2023-10-06T05:21:15.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.039+0000 7f8ca68e1700 1 Processor -- start 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.039+0000 7f8ca68e1700 1 -- start start 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8ca68e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca00c8ff0 0x7f8ca0154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8ca68e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ca00caad0 0x7f8ca0149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8ca68e1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca0156af0 0x7f8ca00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8ca68e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca0159360 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8ca68e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca0149fe0 con 0x7f8ca00caad0 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8ca68e1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca0167430 con 0x7f8ca0156af0 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8c9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca00c8ff0 0x7f8ca0154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8c9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca00c8ff0 0x7f8ca0154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57550/0 (socket says 172.21.15.45:57550) 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8c9ffff700 1 -- 172.21.15.45:0/3741716420 learned_addr learned my addr 172.21.15.45:0/3741716420 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8c9f7fe700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ca00caad0 0x7f8ca0149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.040+0000 7f8ca4e7e700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca0156af0 0x7f8ca00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c9ffff700 1 -- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca0156af0 msgr2=0x7f8ca00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c9ffff700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca0156af0 0x7f8ca00b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c9ffff700 1 -- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ca00caad0 msgr2=0x7f8ca0149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c9ffff700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ca00caad0 0x7f8ca0149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c9ffff700 1 -- 172.21.15.45:0/3741716420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98009ae0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c9ffff700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca00c8ff0 0x7f8ca0154f20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e960 tx=0x7f8c900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c97fff700 1 -- 172.21.15.45:0/3741716420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90013680 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c97fff700 1 -- 172.21.15.45:0/3741716420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c900137e0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca01679f0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8c97fff700 1 -- 172.21.15.45:0/3741716420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90013b80 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.041+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca00b31e0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.042+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca0168040 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.043+0000 7f8c97fff700 1 -- 172.21.15.45:0/3741716420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c900123d0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.043+0000 7f8c97fff700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c80066f30 0x7f8c800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.043+0000 7f8c97fff700 1 -- 172.21.15.45:0/3741716420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8c900848c0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.043+0000 7f8c9f7fe700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c80066f30 0x7f8c800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.044+0000 7f8c9f7fe700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c80066f30 0x7f8c800693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00b2a80 tx=0x7f8c880091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.047+0000 7f8c97fff700 1 -- 172.21.15.45:0/3741716420 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c90051890 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.294+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ca0071500 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.295+0000 7f8c97fff700 1 -- 172.21.15.45:0/3741716420 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f8c90054eb0 con 0x7f8ca00c8ff0 2023-10-06T05:21:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:15.297 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:03.589791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1530850582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1530850582},{"type":"v1","addr":"172.21.15.45:6834","nonce":1530850582}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c80066f30 msgr2=0x7f8c800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c80066f30 0x7f8c800693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00b2a80 tx=0x7f8c880091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca00c8ff0 msgr2=0x7f8ca0154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca00c8ff0 0x7f8ca0154f20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e960 tx=0x7f8c900091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 shutdown_connections 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8c80066f30 0x7f8c800693e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ca0156af0 0x7f8ca00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca00c8ff0 0x7f8ca0154f20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 --2- 172.21.15.45:0/3741716420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ca00caad0 0x7f8ca0149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 >> 172.21.15.45:0/3741716420 conn(0x7f8ca00cdf50 msgr2=0x7f8ca00adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 shutdown_connections 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:15.297+0000 7f8ca68e1700 1 -- 172.21.15.45:0/3741716420 wait complete. 2023-10-06T05:21:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 72 2023-10-06T05:21:15.313 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-10-06T05:21:17.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-10-06T05:21:20.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:20.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.455+0000 7f508a292700 1 Processor -- start 2023-10-06T05:21:20.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.455+0000 7f508a292700 1 -- start start 2023-10-06T05:21:20.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f508a292700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5084159200 0x7f50840c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f508a292700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50841595d0 0x7f508406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f508a292700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50840c7d00 con 0x7f508406b5c0 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f508a292700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840c7780 con 0x7f50841595d0 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f508a292700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840c86d0 con 0x7f5084159200 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f50837fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50841595d0 0x7f508406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f508882f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f508406b5c0 0x7f50840ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60920/0 (socket says 172.21.15.45:60920) 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f508882f700 1 -- 172.21.15.45:0/2905923440 learned_addr learned my addr 172.21.15.45:0/2905923440 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f5083fff700 1 --2- 172.21.15.45:0/2905923440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5084159200 0x7f50840c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f50837fe700 1 -- 172.21.15.45:0/2905923440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5084159200 msgr2=0x7f50840c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f50837fe700 1 --2- 172.21.15.45:0/2905923440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5084159200 0x7f50840c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f50837fe700 1 -- 172.21.15.45:0/2905923440 >> v1:172.21.15.45:6789/0 conn(0x7f508406b5c0 legacy=0x7f50840ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f50837fe700 1 -- 172.21.15.45:0/2905923440 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508406bb40 con 0x7f50841595d0 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.456+0000 7f5082ffd700 1 -- 172.21.15.45:0/2905923440 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573655469 0 0) 0x7f50840c7d00 con 0x7f508406b5c0 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f50837fe700 1 --2- 172.21.15.45:0/2905923440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50841595d0 0x7f508406b080 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f50780098e0 tx=0x7f50780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34eee0718a117ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f5082ffd700 1 -- 172.21.15.45:0/2905923440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078009e80 con 0x7f50841595d0 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f5082ffd700 1 -- 172.21.15.45:0/2905923440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507800b860 con 0x7f50841595d0 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f5082ffd700 1 -- 172.21.15.45:0/2905923440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507800bbe0 con 0x7f50841595d0 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f508a292700 1 -- 172.21.15.45:0/2905923440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50841595d0 msgr2=0x7f508406b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f508a292700 1 --2- 172.21.15.45:0/2905923440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50841595d0 0x7f508406b080 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f50780098e0 tx=0x7f50780091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f508a292700 1 -- 172.21.15.45:0/2905923440 shutdown_connections 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f508a292700 1 --2- 172.21.15.45:0/2905923440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5084159200 0x7f50840c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f508a292700 1 --2- 172.21.15.45:0/2905923440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50841595d0 0x7f508406b080 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.457+0000 7f508a292700 1 -- 172.21.15.45:0/2905923440 >> 172.21.15.45:0/2905923440 conn(0x7f508406f090 msgr2=0x7f508406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:20.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.458+0000 7f508a292700 1 -- 172.21.15.45:0/2905923440 shutdown_connections 2023-10-06T05:21:20.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.458+0000 7f508a292700 1 -- 172.21.15.45:0/2905923440 wait complete. 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.459+0000 7f508a292700 1 Processor -- start 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.459+0000 7f508a292700 1 -- start start 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.460+0000 7f508a292700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f508406b5c0 0x7f508414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.460+0000 7f508a292700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5084159200 0x7f50840cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.460+0000 7f508a292700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50841595d0 0x7f50840c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.460+0000 7f508a292700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840cca40 con 0x7f5084159200 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.460+0000 7f508a292700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50841586b0 con 0x7f508406b5c0 2023-10-06T05:21:20.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.460+0000 7f508a292700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840cbe30 con 0x7f50841595d0 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.460+0000 7f50837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5084159200 0x7f50840cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5084159200 0x7f50840cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44900/0 (socket says 172.21.15.45:44900) 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 -- 172.21.15.45:0/4055487097 learned_addr learned my addr 172.21.15.45:0/4055487097 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f5083fff700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f508406b5c0 0x7f508414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f508882f700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50841595d0 0x7f50840c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 -- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50841595d0 msgr2=0x7f50840c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50841595d0 0x7f50840c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 -- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f508406b5c0 msgr2=0x7f508414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f508406b5c0 0x7f508414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 -- 172.21.15.45:0/4055487097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078009420 con 0x7f5084159200 2023-10-06T05:21:20.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f50837fe700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5084159200 0x7f50840cb320 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f508406aed0 tx=0x7f5078017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f506bfff700 1 -- 172.21.15.45:0/4055487097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507800bbe0 con 0x7f5084159200 2023-10-06T05:21:20.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f506bfff700 1 -- 172.21.15.45:0/4055487097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078011420 con 0x7f5084159200 2023-10-06T05:21:20.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50840cc110 con 0x7f5084159200 2023-10-06T05:21:20.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f506bfff700 1 -- 172.21.15.45:0/4055487097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780117e0 con 0x7f5084159200 2023-10-06T05:21:20.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.461+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50840cc6d0 con 0x7f5084159200 2023-10-06T05:21:20.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.462+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50840c8510 con 0x7f5084159200 2023-10-06T05:21:20.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.463+0000 7f506bfff700 1 -- 172.21.15.45:0/4055487097 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5078011bd0 con 0x7f5084159200 2023-10-06T05:21:20.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.463+0000 7f506bfff700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5070066f30 0x7f50700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.463+0000 7f506bfff700 1 -- 172.21.15.45:0/4055487097 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5078086200 con 0x7f5084159200 2023-10-06T05:21:20.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.463+0000 7f5083fff700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5070066f30 0x7f50700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.464+0000 7f5083fff700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5070066f30 0x7f50700693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.468+0000 7f506bfff700 1 -- 172.21.15.45:0/4055487097 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50780531d0 con 0x7f5084159200 2023-10-06T05:21:20.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.713+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50840ae1e0 con 0x7f5084159200 2023-10-06T05:21:20.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.714+0000 7f506bfff700 1 -- 172.21.15.45:0/4055487097 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f50780567f0 con 0x7f5084159200 2023-10-06T05:21:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:20.716 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":72,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:03.589791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1530850582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1530850582},{"type":"v1","addr":"172.21.15.45:6834","nonce":1530850582}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5070066f30 msgr2=0x7f50700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5070066f30 0x7f50700693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074008040 comp rx=0 tx=0).stop 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5084159200 msgr2=0x7f50840cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5084159200 0x7f50840cb320 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f508406aed0 tx=0x7f5078017000 comp rx=0 tx=0).stop 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 shutdown_connections 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5070066f30 0x7f50700693e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50841595d0 0x7f50840c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f508406b5c0 0x7f508414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 --2- 172.21.15.45:0/4055487097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5084159200 0x7f50840cb320 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 >> 172.21.15.45:0/4055487097 conn(0x7f508406f090 msgr2=0x7f50840af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 shutdown_connections 2023-10-06T05:21:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:20.716+0000 7f508a292700 1 -- 172.21.15.45:0/4055487097 wait complete. 2023-10-06T05:21:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 72 2023-10-06T05:21:20.733 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-10-06T05:21:24.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-10-06T05:21:25.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.874+0000 7f816ffcb700 1 Processor -- start 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.875+0000 7f816ffcb700 1 -- start start 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816ffcb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f81680ccaa0 0x7f816806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816ffcb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81680cb530 con 0x7f8168158f60 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816ffcb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8168069bd0 con 0x7f8168159330 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816ffcb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8168068fc0 con 0x7f81680ccaa0 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816dd67700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8168158f60 0x7f81680cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60932/0 (socket says 172.21.15.45:60932) 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816dd67700 1 -- 172.21.15.45:0/635206467 learned_addr learned my addr 172.21.15.45:0/635206467 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816e568700 1 --2- 172.21.15.45:0/635206467 >> v2:172.21.15.144:3300/0 conn(0x7f81680ccaa0 0x7f816806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816cd65700 1 -- 172.21.15.45:0/635206467 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575473795 0 0) 0x7f8164001110 con 0x7f8168158f60 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816cd65700 1 -- 172.21.15.45:0/635206467 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8154003570 con 0x7f8168158f60 2023-10-06T05:21:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816cd65700 1 -- 172.21.15.45:0/635206467 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718874745 0 0) 0x7f8168069bd0 con 0x7f8168159330 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816cd65700 1 -- 172.21.15.45:0/635206467 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8164001110 con 0x7f8168159330 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816e568700 1 -- 172.21.15.45:0/635206467 >> v1:172.21.15.144:6789/0 conn(0x7f8168159330 legacy=0x7f81680cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816e568700 1 -- 172.21.15.45:0/635206467 >> v1:172.21.15.45:6789/0 conn(0x7f8168158f60 legacy=0x7f81680cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816e568700 1 -- 172.21.15.45:0/635206467 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680692a0 con 0x7f81680ccaa0 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.876+0000 7f816e568700 1 --2- 172.21.15.45:0/635206467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81680ccaa0 0x7f816806a060 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8160009e60 tx=0x7f8160009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e843dd9dde1c627 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816cd65700 1 -- 172.21.15.45:0/635206467 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816000e070 con 0x7f81680ccaa0 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816cd65700 1 -- 172.21.15.45:0/635206467 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816000bab0 con 0x7f81680ccaa0 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816cd65700 1 -- 172.21.15.45:0/635206467 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816000be70 con 0x7f81680ccaa0 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816ffcb700 1 -- 172.21.15.45:0/635206467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81680ccaa0 msgr2=0x7f816806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816ffcb700 1 --2- 172.21.15.45:0/635206467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81680ccaa0 0x7f816806a060 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8160009e60 tx=0x7f8160009780 comp rx=0 tx=0).stop 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816ffcb700 1 -- 172.21.15.45:0/635206467 shutdown_connections 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816ffcb700 1 --2- 172.21.15.45:0/635206467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81680ccaa0 0x7f816806a060 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816ffcb700 1 -- 172.21.15.45:0/635206467 >> 172.21.15.45:0/635206467 conn(0x7f81680cdf50 msgr2=0x7f81680ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816ffcb700 1 -- 172.21.15.45:0/635206467 shutdown_connections 2023-10-06T05:21:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.877+0000 7f816ffcb700 1 -- 172.21.15.45:0/635206467 wait complete. 2023-10-06T05:21:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.879+0000 7f816ffcb700 1 Processor -- start 2023-10-06T05:21:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.879+0000 7f816ffcb700 1 -- start start 2023-10-06T05:21:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816ffcb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81680ccaa0 0x7f8168154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816ffcb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8168158f60 0x7f81680c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816ffcb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8168159330 0x7f81680c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816ffcb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8168156680 con 0x7f81680ccaa0 2023-10-06T05:21:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816ffcb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680c5a10 con 0x7f8168159330 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816ffcb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81680c5c60 con 0x7f8168158f60 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816dd67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81680ccaa0 0x7f8168154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816dd67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81680ccaa0 0x7f8168154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44910/0 (socket says 172.21.15.45:44910) 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816dd67700 1 -- 172.21.15.45:0/1388762865 learned_addr learned my addr 172.21.15.45:0/1388762865 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816e568700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8168159330 0x7f81680c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.880+0000 7f816d566700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8168158f60 0x7f81680c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816dd67700 1 -- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8168158f60 msgr2=0x7f81680c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816dd67700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8168158f60 0x7f81680c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816dd67700 1 -- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8168159330 msgr2=0x7f81680c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816dd67700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8168159330 0x7f81680c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816dd67700 1 -- 172.21.15.45:0/1388762865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160009a20 con 0x7f81680ccaa0 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816dd67700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81680ccaa0 0x7f8168154a80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f81640058d0 tx=0x7f8164007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f815dffb700 1 -- 172.21.15.45:0/1388762865 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164005b30 con 0x7f81680ccaa0 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f815dffb700 1 -- 172.21.15.45:0/1388762865 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8164005c90 con 0x7f81680ccaa0 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81680c5f40 con 0x7f81680ccaa0 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f815dffb700 1 -- 172.21.15.45:0/1388762865 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164007300 con 0x7f81680ccaa0 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.881+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81680c6500 con 0x7f81680ccaa0 2023-10-06T05:21:25.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.882+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81680c6720 con 0x7f81680ccaa0 2023-10-06T05:21:25.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.883+0000 7f815dffb700 1 -- 172.21.15.45:0/1388762865 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f816401b400 con 0x7f81680ccaa0 2023-10-06T05:21:25.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.883+0000 7f815dffb700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f814c066f30 0x7f814c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.883+0000 7f815dffb700 1 -- 172.21.15.45:0/1388762865 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8164084220 con 0x7f81680ccaa0 2023-10-06T05:21:25.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.884+0000 7f816d566700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f814c066f30 0x7f814c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.884+0000 7f816d566700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f814c066f30 0x7f814c0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8158007fd0 tx=0x7f8158007d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:25.888+0000 7f815dffb700 1 -- 172.21.15.45:0/1388762865 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81640511f0 con 0x7f81680ccaa0 2023-10-06T05:21:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.135+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81680cb530 con 0x7f81680ccaa0 2023-10-06T05:21:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.136+0000 7f815dffb700 1 -- 172.21.15.45:0/1388762865 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4653 (secure 0 0 0) 0x7f81680cb530 con 0x7f81680ccaa0 2023-10-06T05:21:26.138 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:26.138 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:25.841970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1530850582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1530850582},{"type":"v1","addr":"172.21.15.45:6834","nonce":1530850582}]},"join_fscid":-1,"laggy_since":"2023-10-06T05:21:25.841921+0000","export_targets":[],"features":4540138320759226367,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f814c066f30 msgr2=0x7f814c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f814c066f30 0x7f814c0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8158007fd0 tx=0x7f8158007d10 comp rx=0 tx=0).stop 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81680ccaa0 msgr2=0x7f8168154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81680ccaa0 0x7f8168154a80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f81640058d0 tx=0x7f8164007ea0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 shutdown_connections 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f814c066f30 0x7f814c0693e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8168159330 0x7f81680c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81680ccaa0 0x7f8168154a80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 --2- 172.21.15.45:0/1388762865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8168158f60 0x7f81680c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 >> 172.21.15.45:0/1388762865 conn(0x7f81680cdf50 msgr2=0x7f816806d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 shutdown_connections 2023-10-06T05:21:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.138+0000 7f816ffcb700 1 -- 172.21.15.45:0/1388762865 wait complete. 2023-10-06T05:21:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2023-10-06T05:21:26.154 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-10-06T05:21:26.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-10-06T05:21:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.288+0000 7fce86b74700 1 Processor -- start 2023-10-06T05:21:26.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.288+0000 7fce86b74700 1 -- start start 2023-10-06T05:21:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.288+0000 7fce86b74700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce800c8b70 0x7fce800c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.288+0000 7fce86b74700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800c3b50 con 0x7fce800ccb70 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.288+0000 7fce86b74700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce80157700 con 0x7fce80157330 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.288+0000 7fce86b74700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800c98e0 con 0x7fce800c8b70 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce85111700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce800c8b70 0x7fce800c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce7ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fce800ccb70 0x7fce800aaae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60944/0 (socket says 172.21.15.45:60944) 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce7ffff700 1 -- 172.21.15.45:0/1757624622 learned_addr learned my addr 172.21.15.45:0/1757624622 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce85111700 1 -- 172.21.15.45:0/1757624622 >> v1:172.21.15.144:6789/0 conn(0x7fce80157330 legacy=0x7fce800ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce85111700 1 -- 172.21.15.45:0/1757624622 >> v1:172.21.15.45:6789/0 conn(0x7fce800ccb70 legacy=0x7fce800aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce85111700 1 -- 172.21.15.45:0/1757624622 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800c9bc0 con 0x7fce800c8b70 2023-10-06T05:21:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce7f7fe700 1 -- 172.21.15.45:0/1757624622 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401776804 0 0) 0x7fce800c3b50 con 0x7fce800ccb70 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce85111700 1 --2- 172.21.15.45:0/1757624622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce800c8b70 0x7fce800c8f90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fce78009e60 tx=0x7fce78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e172b1a3dc11bd26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce7f7fe700 1 -- 172.21.15.45:0/1757624622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7800e070 con 0x7fce800c8b70 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce7f7fe700 1 -- 172.21.15.45:0/1757624622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7800bc30 con 0x7fce800c8b70 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.289+0000 7fce7f7fe700 1 -- 172.21.15.45:0/1757624622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78012430 con 0x7fce800c8b70 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.290+0000 7fce86b74700 1 -- 172.21.15.45:0/1757624622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce800c8b70 msgr2=0x7fce800c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.290+0000 7fce86b74700 1 --2- 172.21.15.45:0/1757624622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce800c8b70 0x7fce800c8f90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fce78009e60 tx=0x7fce78009780 comp rx=0 tx=0).stop 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.290+0000 7fce86b74700 1 -- 172.21.15.45:0/1757624622 shutdown_connections 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.290+0000 7fce86b74700 1 --2- 172.21.15.45:0/1757624622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce800c8b70 0x7fce800c8f90 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.290+0000 7fce86b74700 1 -- 172.21.15.45:0/1757624622 >> 172.21.15.45:0/1757624622 conn(0x7fce80067e00 msgr2=0x7fce800cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.290+0000 7fce86b74700 1 -- 172.21.15.45:0/1757624622 shutdown_connections 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.290+0000 7fce86b74700 1 -- 172.21.15.45:0/1757624622 wait complete. 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.292+0000 7fce86b74700 1 Processor -- start 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.292+0000 7fce86b74700 1 -- start start 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce86b74700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce800c8b70 0x7fce8014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce86b74700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce800ccb70 0x7fce80159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce86b74700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce80157330 0x7fce80155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce86b74700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce800b1200 con 0x7fce80157330 2023-10-06T05:21:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce86b74700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce800b1d80 con 0x7fce800c8b70 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce86b74700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800b1450 con 0x7fce800ccb70 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce80157330 0x7fce80155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce80157330 0x7fce80155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44918/0 (socket says 172.21.15.45:44918) 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 -- 172.21.15.45:0/2107463796 learned_addr learned my addr 172.21.15.45:0/2107463796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce7ffff700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce800ccb70 0x7fce80159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce84910700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce800c8b70 0x7fce8014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 -- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce800ccb70 msgr2=0x7fce80159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce800ccb70 0x7fce80159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 -- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce800c8b70 msgr2=0x7fce8014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce800c8b70 0x7fce8014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.293+0000 7fce85111700 1 -- 172.21.15.45:0/2107463796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78009a20 con 0x7fce80157330 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.294+0000 7fce85111700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce80157330 0x7fce80155e90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fce78013000 tx=0x7fce78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.294+0000 7fce7cff9700 1 -- 172.21.15.45:0/2107463796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7801b420 con 0x7fce80157330 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.294+0000 7fce7cff9700 1 -- 172.21.15.45:0/2107463796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7801b580 con 0x7fce80157330 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.294+0000 7fce7cff9700 1 -- 172.21.15.45:0/2107463796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7801b900 con 0x7fce80157330 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.294+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce800b1730 con 0x7fce80157330 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.294+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce800acd40 con 0x7fce80157330 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.295+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce800cdb40 con 0x7fce80157330 2023-10-06T05:21:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.295+0000 7fce7cff9700 1 -- 172.21.15.45:0/2107463796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce7801bc70 con 0x7fce80157330 2023-10-06T05:21:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.296+0000 7fce7cff9700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce64066f30 0x7fce640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.296+0000 7fce7cff9700 1 -- 172.21.15.45:0/2107463796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fce78086570 con 0x7fce80157330 2023-10-06T05:21:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.296+0000 7fce84910700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce64066f30 0x7fce640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.297+0000 7fce84910700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce64066f30 0x7fce640693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fce74007fd0 tx=0x7fce74007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.300+0000 7fce7cff9700 1 -- 172.21.15.45:0/2107463796 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce780535f0 con 0x7fce80157330 2023-10-06T05:21:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.502+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7fce800b1ef0 con 0x7fce80157330 2023-10-06T05:21:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.506+0000 7fce7cff9700 1 -- 172.21.15.45:0/2107463796 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v24) v1 ==== 121+0+0 (secure 0 0 0) 0x7fce78056c10 con 0x7fce80157330 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce64066f30 msgr2=0x7fce640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce64066f30 0x7fce640693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fce74007fd0 tx=0x7fce74007eb0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce80157330 msgr2=0x7fce80155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce80157330 0x7fce80155e90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fce78013000 tx=0x7fce78019000 comp rx=0 tx=0).stop 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 shutdown_connections 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce64066f30 0x7fce640693e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce80157330 0x7fce80155e90 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce800c8b70 0x7fce8014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 --2- 172.21.15.45:0/2107463796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce800ccb70 0x7fce80159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 >> 172.21.15.45:0/2107463796 conn(0x7fce80067e00 msgr2=0x7fce8006c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 shutdown_connections 2023-10-06T05:21:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.509+0000 7fce86b74700 1 -- 172.21.15.45:0/2107463796 wait complete. 2023-10-06T05:21:26.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2023-10-06T05:21:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.657+0000 7f7acab05700 1 Processor -- start 2023-10-06T05:21:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.657+0000 7f7acab05700 1 -- start start 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.658+0000 7f7acab05700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ac4159520 0x7f7ac415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.658+0000 7f7acab05700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac40c9c90 con 0x7f7ac415d3d0 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.658+0000 7f7acab05700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac415d8e0 con 0x7f7ac4159150 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.658+0000 7f7acab05700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac415dbc0 con 0x7f7ac4159520 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.658+0000 7f7ac3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ac4159520 0x7f7ac415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.658+0000 7f7ac90a2700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7ac415d3d0 0x7f7ac4067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45326/0 (socket says 172.21.15.45:45326) 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.658+0000 7f7ac90a2700 1 -- 172.21.15.45:0/2328708153 learned_addr learned my addr 172.21.15.45:0/2328708153 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac3fff700 1 -- 172.21.15.45:0/2328708153 >> v1:172.21.15.144:6789/0 conn(0x7f7ac4159150 legacy=0x7f7ac40ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:26.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac3fff700 1 -- 172.21.15.45:0/2328708153 >> v1:172.21.15.45:6789/0 conn(0x7f7ac415d3d0 legacy=0x7f7ac4067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac3fff700 1 -- 172.21.15.45:0/2328708153 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac40c9920 con 0x7f7ac4159520 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac37fe700 1 -- 172.21.15.45:0/2328708153 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244784738 0 0) 0x7f7ac40c9c90 con 0x7f7ac415d3d0 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac3fff700 1 --2- 172.21.15.45:0/2328708153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159520 0x7f7ac415ce90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00098e0 tx=0x7f7ab00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99980a3933594eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac37fe700 1 -- 172.21.15.45:0/2328708153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0009e80 con 0x7f7ac4159520 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac37fe700 1 -- 172.21.15.45:0/2328708153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab000b860 con 0x7f7ac4159520 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.659+0000 7f7ac37fe700 1 -- 172.21.15.45:0/2328708153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000bbe0 con 0x7f7ac4159520 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.660+0000 7f7acab05700 1 -- 172.21.15.45:0/2328708153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159520 msgr2=0x7f7ac415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.660+0000 7f7acab05700 1 --2- 172.21.15.45:0/2328708153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159520 0x7f7ac415ce90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00098e0 tx=0x7f7ab00091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.660+0000 7f7acab05700 1 -- 172.21.15.45:0/2328708153 shutdown_connections 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.660+0000 7f7acab05700 1 --2- 172.21.15.45:0/2328708153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159520 0x7f7ac415ce90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.660+0000 7f7acab05700 1 -- 172.21.15.45:0/2328708153 >> 172.21.15.45:0/2328708153 conn(0x7f7ac40691c0 msgr2=0x7f7ac40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.660+0000 7f7acab05700 1 -- 172.21.15.45:0/2328708153 shutdown_connections 2023-10-06T05:21:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.660+0000 7f7acab05700 1 -- 172.21.15.45:0/2328708153 wait complete. 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.661+0000 7f7acab05700 1 Processor -- start 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.661+0000 7f7acab05700 1 -- start start 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7acab05700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4159150 0x7f7ac4155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7acab05700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac415d3d0 0x7f7ac415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7acab05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cae90 0x7f7ac40cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7acab05700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40ae980 con 0x7f7ac40cae90 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7acab05700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40cb850 con 0x7f7ac415d3d0 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7acab05700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac4159ff0 con 0x7f7ac4159150 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7ac90a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cae90 0x7f7ac40cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7ac90a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cae90 0x7f7ac40cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54718/0 (socket says 172.21.15.45:54718) 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7ac90a2700 1 -- 172.21.15.45:0/3724928694 learned_addr learned my addr 172.21.15.45:0/3724928694 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7ac88a1700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4159150 0x7f7ac4155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.662+0000 7f7ac3fff700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac415d3d0 0x7f7ac415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac90a2700 1 -- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4159150 msgr2=0x7f7ac4155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac90a2700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4159150 0x7f7ac4155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac90a2700 1 -- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac415d3d0 msgr2=0x7f7ac415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac90a2700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac415d3d0 0x7f7ac415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac90a2700 1 -- 172.21.15.45:0/3724928694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0009420 con 0x7f7ac40cae90 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac88a1700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4159150 0x7f7ac4155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac90a2700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cae90 0x7f7ac40cb310 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00cc20 tx=0x7f7abc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac0ff9700 1 -- 172.21.15.45:0/3724928694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc004bb0 con 0x7f7ac40cae90 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac4156040 con 0x7f7ac40cae90 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac0ff9700 1 -- 172.21.15.45:0/3724928694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc004d10 con 0x7f7ac40cae90 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7ac0ff9700 1 -- 172.21.15.45:0/3724928694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc00abd0 con 0x7f7ac40cae90 2023-10-06T05:21:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.663+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac415c4c0 con 0x7f7ac40cae90 2023-10-06T05:21:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.664+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac40c2460 con 0x7f7ac40cae90 2023-10-06T05:21:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.665+0000 7f7ac0ff9700 1 -- 172.21.15.45:0/3724928694 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7abc0070f0 con 0x7f7ac40cae90 2023-10-06T05:21:26.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.665+0000 7f7ac0ff9700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa8066f60 0x7f7aa8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.665+0000 7f7ac0ff9700 1 -- 172.21.15.45:0/3724928694 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7abc0846a0 con 0x7f7ac40cae90 2023-10-06T05:21:26.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.665+0000 7f7ac88a1700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa8066f60 0x7f7aa8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.666+0000 7f7ac88a1700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa8066f60 0x7f7aa8069410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8001880 tx=0x7f7ab8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.670+0000 7f7ac0ff9700 1 -- 172.21.15.45:0/3724928694 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7abc051670 con 0x7f7ac40cae90 2023-10-06T05:21:26.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:26.959+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) v1 -- 0x7f7ac40c8fe0 con 0x7f7ac40cae90 2023-10-06T05:21:27.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.878+0000 7f7ac0ff9700 1 -- 172.21.15.45:0/3724928694 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.6702 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7abc0180a0 con 0x7f7ac40cae90 2023-10-06T05:21:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa8066f60 msgr2=0x7f7aa8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa8066f60 0x7f7aa8069410 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8001880 tx=0x7f7ab8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cae90 msgr2=0x7f7ac40cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cae90 0x7f7ac40cb310 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00cc20 tx=0x7f7abc0019b0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 shutdown_connections 2023-10-06T05:21:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa8066f60 0x7f7aa8069410 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cae90 0x7f7ac40cb310 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4159150 0x7f7ac4155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 --2- 172.21.15.45:0/3724928694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac415d3d0 0x7f7ac415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 >> 172.21.15.45:0/3724928694 conn(0x7f7ac40691c0 msgr2=0x7f7ac40c1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 shutdown_connections 2023-10-06T05:21:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:27.882+0000 7f7acab05700 1 -- 172.21.15.45:0/3724928694 wait complete. 2023-10-06T05:21:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:unfreezing mds.6702 2023-10-06T05:21:27.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -n kernel.core_pattern 2023-10-06T05:21:27.936 INFO:teuthology.orchestra.run.smithi045.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:21:27.937 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:21:27.938 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-10-06T05:21:28.018 INFO:teuthology.orchestra.run.smithi045.stdout:1696569669.149557.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-10-06T05:21:28.019 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1696569669.149557.core'] 2023-10-06T05:21:28.019 INFO:tasks.cephfs.cephfs_test_case:Found core file 1696569669.149557.core, deleting it 2023-10-06T05:21:28.019 DEBUG:teuthology.orchestra.run.smithi045:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1696569669.149557.core 2023-10-06T05:21:28.097 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:21:28.097 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:21:28.097 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:21:28.097 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:28.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:21:28.098 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi045 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-10-06T05:21:28.098 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:21:28.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:21:28.140 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:21:28.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:28.233 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:21:28.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.279+0000 7ff63db22700 1 Processor -- start 2023-10-06T05:21:28.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.279+0000 7ff63db22700 1 -- start start 2023-10-06T05:21:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff63db22700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6381560a0 0x7ff63815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff63db22700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6380ae8d0 con 0x7ff638155cd0 2023-10-06T05:21:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff63db22700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6380aebb0 con 0x7ff6380ccdc0 2023-10-06T05:21:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff63db22700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380ae6b0 con 0x7ff6381560a0 2023-10-06T05:21:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff6377fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff638155cd0 0x7ff6380adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45340/0 (socket says 172.21.15.45:45340) 2023-10-06T05:21:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff6377fe700 1 -- 172.21.15.45:0/2681952059 learned_addr learned my addr 172.21.15.45:0/2681952059 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff636ffd700 1 --2- 172.21.15.45:0/2681952059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6381560a0 0x7ff63815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff6367fc700 1 -- 172.21.15.45:0/2681952059 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319977311 0 0) 0x7ff6380ae8d0 con 0x7ff638155cd0 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff6367fc700 1 -- 172.21.15.45:0/2681952059 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff624003570 con 0x7ff638155cd0 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.280+0000 7ff6367fc700 1 -- 172.21.15.45:0/2681952059 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353747070 0 0) 0x7ff6380aebb0 con 0x7ff6380ccdc0 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff636ffd700 1 -- 172.21.15.45:0/2681952059 >> v1:172.21.15.144:6789/0 conn(0x7ff6380ccdc0 legacy=0x7ff6380c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff636ffd700 1 -- 172.21.15.45:0/2681952059 >> v1:172.21.15.45:6789/0 conn(0x7ff638155cd0 legacy=0x7ff6380adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff636ffd700 1 -- 172.21.15.45:0/2681952059 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380aee90 con 0x7ff6381560a0 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff636ffd700 1 --2- 172.21.15.45:0/2681952059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6381560a0 0x7ff63815a730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff620009910 tx=0x7ff6200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac9581f264eb675 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff6367fc700 1 -- 172.21.15.45:0/2681952059 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000b680 con 0x7ff6381560a0 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff6367fc700 1 -- 172.21.15.45:0/2681952059 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62000b7e0 con 0x7ff6381560a0 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff6367fc700 1 -- 172.21.15.45:0/2681952059 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000bba0 con 0x7ff6381560a0 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff63db22700 1 -- 172.21.15.45:0/2681952059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6381560a0 msgr2=0x7ff63815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.281+0000 7ff63db22700 1 --2- 172.21.15.45:0/2681952059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6381560a0 0x7ff63815a730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff620009910 tx=0x7ff6200091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.282+0000 7ff63db22700 1 -- 172.21.15.45:0/2681952059 shutdown_connections 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.282+0000 7ff63db22700 1 --2- 172.21.15.45:0/2681952059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6381560a0 0x7ff63815a730 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.282+0000 7ff63db22700 1 -- 172.21.15.45:0/2681952059 >> 172.21.15.45:0/2681952059 conn(0x7ff63806f090 msgr2=0x7ff63806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.282+0000 7ff63db22700 1 -- 172.21.15.45:0/2681952059 shutdown_connections 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.282+0000 7ff63db22700 1 -- 172.21.15.45:0/2681952059 wait complete. 2023-10-06T05:21:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.283+0000 7ff63db22700 1 Processor -- start 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.283+0000 7ff63db22700 1 -- start start 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff63db22700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6380ccdc0 0x7ff638159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff63db22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff638155cd0 0x7ff638169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff63db22700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6381560a0 0x7ff63815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff63db22700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63815d310 con 0x7ff638155cd0 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff63db22700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63815cd50 con 0x7ff6381560a0 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff63db22700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63815d030 con 0x7ff6380ccdc0 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff638155cd0 0x7ff638169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff638155cd0 0x7ff638169f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54734/0 (socket says 172.21.15.45:54734) 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 -- 172.21.15.45:0/2402381013 learned_addr learned my addr 172.21.15.45:0/2402381013 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff637fff700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6381560a0 0x7ff63815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff6377fe700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6380ccdc0 0x7ff638159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 -- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6380ccdc0 msgr2=0x7ff638159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6380ccdc0 0x7ff638159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 -- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6381560a0 msgr2=0x7ff63815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6381560a0 0x7ff63815a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 -- 172.21.15.45:0/2402381013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620009420 con 0x7ff638155cd0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.284+0000 7ff636ffd700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff638155cd0 0x7ff638169f50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff63806aed0 tx=0x7ff620017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.285+0000 7ff62f7fe700 1 -- 172.21.15.45:0/2402381013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000bb60 con 0x7ff638155cd0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.285+0000 7ff62f7fe700 1 -- 172.21.15.45:0/2402381013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620027450 con 0x7ff638155cd0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.285+0000 7ff62f7fe700 1 -- 172.21.15.45:0/2402381013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200277f0 con 0x7ff638155cd0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.285+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff638156650 con 0x7ff638155cd0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.285+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff638159770 con 0x7ff638155cd0 2023-10-06T05:21:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.286+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63806e2f0 con 0x7ff638155cd0 2023-10-06T05:21:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.286+0000 7ff62f7fe700 1 -- 172.21.15.45:0/2402381013 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff62001a430 con 0x7ff638155cd0 2023-10-06T05:21:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.286+0000 7ff62f7fe700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff618066f30 0x7ff6180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.287+0000 7ff62f7fe700 1 -- 172.21.15.45:0/2402381013 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ff620086870 con 0x7ff638155cd0 2023-10-06T05:21:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.287+0000 7ff6377fe700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff618066f30 0x7ff6180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.287+0000 7ff6377fe700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff618066f30 0x7ff6180693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff6280058d0 tx=0x7ff628009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.291+0000 7ff62f7fe700 1 -- 172.21.15.45:0/2402381013 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6200538a0 con 0x7ff638155cd0 2023-10-06T05:21:28.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.542+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6380af170 con 0x7ff638155cd0 2023-10-06T05:21:28.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.543+0000 7ff62f7fe700 1 -- 172.21.15.45:0/2402381013 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3849 (secure 0 0 0) 0x7ff620056ec0 con 0x7ff638155cd0 2023-10-06T05:21:28.545 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:28.545 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:27.879702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6514":{"gid":6514,"name":"d","rank":0,"incarnation":75,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:28.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff618066f30 msgr2=0x7ff6180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff618066f30 0x7ff6180693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff6280058d0 tx=0x7ff628009040 comp rx=0 tx=0).stop 2023-10-06T05:21:28.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff638155cd0 msgr2=0x7ff638169f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff638155cd0 0x7ff638169f50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff63806aed0 tx=0x7ff620017000 comp rx=0 tx=0).stop 2023-10-06T05:21:28.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 shutdown_connections 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff618066f30 0x7ff6180693e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6381560a0 0x7ff63815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6380ccdc0 0x7ff638159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 --2- 172.21.15.45:0/2402381013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff638155cd0 0x7ff638169f50 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 >> 172.21.15.45:0/2402381013 conn(0x7ff63806f090 msgr2=0x7ff6380c91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 shutdown_connections 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:28.545+0000 7ff63db22700 1 -- 172.21.15.45:0/2402381013 wait complete. 2023-10-06T05:21:28.548 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 75 2023-10-06T05:21:28.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:20:53.763976+0000', 'modified': '2023-10-06T05:21:27.879702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6514': {'gid': 6514, 'name': 'd', 'rank': 0, 'incarnation': 75, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/2999699668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 2999699668}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 2999699668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:21:29.041 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6514:up:replay 2023-10-06T05:21:30.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:30.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.232+0000 7f2e5143d700 1 Processor -- start 2023-10-06T05:21:30.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.232+0000 7f2e5143d700 1 -- start start 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.232+0000 7f2e5143d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e4c0ae9d0 0x7f2e4c068860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.232+0000 7f2e5143d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c0677c0 con 0x7f2e4c15d470 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e5143d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c067aa0 con 0x7f2e4c068da0 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e5143d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c067d80 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e4affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e4c0ae9d0 0x7f2e4c068860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e4b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2e4c15d470 0x7f2e4c0ac630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45350/0 (socket says 172.21.15.45:45350) 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e4b7fe700 1 -- 172.21.15.45:0/4668407 learned_addr learned my addr 172.21.15.45:0/4668407 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e4affd700 1 -- 172.21.15.45:0/4668407 >> v1:172.21.15.144:6789/0 conn(0x7f2e4c068da0 legacy=0x7f2e4c15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e4affd700 1 -- 172.21.15.45:0/4668407 >> v1:172.21.15.45:6789/0 conn(0x7f2e4c15d470 legacy=0x7f2e4c0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e4affd700 1 -- 172.21.15.45:0/4668407 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c068060 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e4affd700 1 --2- 172.21.15.45:0/4668407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c068860 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0098e0 tx=0x7f2e3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69692fdc4be71df6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e49ffb700 1 -- 172.21.15.45:0/4668407 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c009e80 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e49ffb700 1 -- 172.21.15.45:0/4668407 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c00b860 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.233+0000 7f2e49ffb700 1 -- 172.21.15.45:0/4668407 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00bbe0 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.234+0000 7f2e5143d700 1 -- 172.21.15.45:0/4668407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 msgr2=0x7f2e4c068860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.234+0000 7f2e5143d700 1 --2- 172.21.15.45:0/4668407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c068860 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0098e0 tx=0x7f2e3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.234+0000 7f2e5143d700 1 -- 172.21.15.45:0/4668407 shutdown_connections 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.234+0000 7f2e5143d700 1 --2- 172.21.15.45:0/4668407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c068860 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.234+0000 7f2e5143d700 1 -- 172.21.15.45:0/4668407 >> 172.21.15.45:0/4668407 conn(0x7f2e4c0cd000 msgr2=0x7f2e4c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.234+0000 7f2e5143d700 1 -- 172.21.15.45:0/4668407 shutdown_connections 2023-10-06T05:21:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.234+0000 7f2e5143d700 1 -- 172.21.15.45:0/4668407 wait complete. 2023-10-06T05:21:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.235+0000 7f2e5143d700 1 Processor -- start 2023-10-06T05:21:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.235+0000 7f2e5143d700 1 -- start start 2023-10-06T05:21:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e5143d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c068da0 0x7f2e4c156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e5143d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e5143d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e4c15d470 0x7f2e4c0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e5143d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c44b0 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e5143d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c4790 con 0x7f2e4c068da0 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e5143d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c4a70 con 0x7f2e4c15d470 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e4a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e4a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c0c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54750/0 (socket says 172.21.15.45:54750) 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e4a7fc700 1 -- 172.21.15.45:0/2952628206 learned_addr learned my addr 172.21.15.45:0/2952628206 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e4affd700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c068da0 0x7f2e4c156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.236+0000 7f2e4b7fe700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e4c15d470 0x7f2e4c0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e4a7fc700 1 -- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e4c15d470 msgr2=0x7f2e4c0c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e4a7fc700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e4c15d470 0x7f2e4c0c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e4a7fc700 1 -- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c068da0 msgr2=0x7f2e4c156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e4a7fc700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c068da0 0x7f2e4c156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e4a7fc700 1 -- 172.21.15.45:0/2952628206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c009420 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e4a7fc700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c0c57c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000c4e0 tx=0x7f2e4000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e32ffd700 1 -- 172.21.15.45:0/2952628206 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40011590 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e32ffd700 1 -- 172.21.15.45:0/2952628206 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e40011710 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c0c4d50 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e32ffd700 1 -- 172.21.15.45:0/2952628206 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40011ad0 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.237+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c156f90 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.238+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c0b0120 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.239+0000 7f2e32ffd700 1 -- 172.21.15.45:0/2952628206 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e4001b3b0 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.239+0000 7f2e32ffd700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e38066f30 0x7f2e380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.239+0000 7f2e32ffd700 1 -- 172.21.15.45:0/2952628206 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f2e40083c20 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.239+0000 7f2e4affd700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e38066f30 0x7f2e380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.240+0000 7f2e4affd700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e38066f30 0x7f2e380693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0093f0 tx=0x7f2e3c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.244+0000 7f2e32ffd700 1 -- 172.21.15.45:0/2952628206 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e4c0b0120 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.496+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e4c0b0120 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.496+0000 7f2e32ffd700 1 -- 172.21.15.45:0/2952628206 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f2e4c0b0120 con 0x7f2e4c0ae9d0 2023-10-06T05:21:30.498 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:30.498 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":77,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2705674276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2705674276},{"type":"v1","addr":"172.21.15.45:6834","nonce":2705674276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:29.885417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6514":{"gid":6514,"name":"d","rank":0,"incarnation":75,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:30.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e38066f30 msgr2=0x7f2e380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e38066f30 0x7f2e380693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0093f0 tx=0x7f2e3c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 msgr2=0x7f2e4c0c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c0c57c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000c4e0 tx=0x7f2e4000c7f0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 shutdown_connections 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e38066f30 0x7f2e380693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e4c15d470 0x7f2e4c0c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c068da0 0x7f2e4c156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 --2- 172.21.15.45:0/2952628206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c0ae9d0 0x7f2e4c0c57c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 >> 172.21.15.45:0/2952628206 conn(0x7f2e4c0cd000 msgr2=0x7f2e4c0cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 shutdown_connections 2023-10-06T05:21:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:30.499+0000 7f2e5143d700 1 -- 172.21.15.45:0/2952628206 wait complete. 2023-10-06T05:21:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 77 2023-10-06T05:21:30.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T05:21:30.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:20:53.763976+0000', 'modified': '2023-10-06T05:21:29.885417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6514': {'gid': 6514, 'name': 'd', 'rank': 0, 'incarnation': 75, 'state': 'up:rejoin', 'state_seq': 12, 'addr': '172.21.15.144:6837/2999699668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 2999699668}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 2999699668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:21:30.917 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6514:up:rejoin 2023-10-06T05:21:31.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:32.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.057+0000 7f650215c700 1 Processor -- start 2023-10-06T05:21:32.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.057+0000 7f650215c700 1 -- start start 2023-10-06T05:21:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.058+0000 7f650215c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f64fc0cd3f0 0x7f64fc06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.058+0000 7f650215c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc15b380 con 0x7f64fc1570b0 2023-10-06T05:21:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.058+0000 7f650215c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc15b660 con 0x7f64fc156ce0 2023-10-06T05:21:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.058+0000 7f650215c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc06c010 con 0x7f64fc0cd3f0 2023-10-06T05:21:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.058+0000 7f64faffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f64fc1570b0 0x7f64fc0ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45356/0 (socket says 172.21.15.45:45356) 2023-10-06T05:21:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.058+0000 7f64faffd700 1 -- 172.21.15.45:0/2181698302 learned_addr learned my addr 172.21.15.45:0/2181698302 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.058+0000 7f64fbfff700 1 --2- 172.21.15.45:0/2181698302 >> v2:172.21.15.144:3300/0 conn(0x7f64fc0cd3f0 0x7f64fc06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130571315 0 0) 0x7f64fc15b380 con 0x7f64fc1570b0 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e8003570 con 0x7f64fc1570b0 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856917671 0 0) 0x7f64e8003570 con 0x7f64fc1570b0 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 >> v2:172.21.15.144:3300/0 conn(0x7f64fc0cd3f0 msgr2=0x7f64fc06bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 --2- 172.21.15.45:0/2181698302 >> v2:172.21.15.144:3300/0 conn(0x7f64fc0cd3f0 0x7f64fc06bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 >> v1:172.21.15.144:6789/0 conn(0x7f64fc156ce0 legacy=0x7f64fc0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc15b940 con 0x7f64fc1570b0 2023-10-06T05:21:32.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f64e4001470 con 0x7f64fc1570b0 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 >> v1:172.21.15.45:6789/0 conn(0x7f64fc1570b0 legacy=0x7f64fc0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e8003a60 0x7f64e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64e8004810 0x7f64e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e80051c0 0x7f64e800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc06c010 con 0x7f64e80051c0 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e800c050 con 0x7f64e8003a60 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e800c210 con 0x7f64e8004810 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fbfff700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e80051c0 0x7f64e800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fb7fe700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e8003a60 0x7f64e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64faffd700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64e8004810 0x7f64e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fbfff700 1 -- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64e8004810 msgr2=0x7f64e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fbfff700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64e8004810 0x7f64e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fbfff700 1 -- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e8003a60 msgr2=0x7f64e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fbfff700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e8003a60 0x7f64e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.059+0000 7f64fbfff700 1 -- 172.21.15.45:0/2181698302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e8003570 con 0x7f64e80051c0 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f64fbfff700 1 -- 172.21.15.45:0/2181698302 reap_dead start 2023-10-06T05:21:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f64fbfff700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e80051c0 0x7f64e800ba60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f64f400e210 tx=0x7f64f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72420e9f5495960d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400eac0 con 0x7f64e80051c0 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f400f040 con 0x7f64e80051c0 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f64fa7fc700 1 -- 172.21.15.45:0/2181698302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400ee70 con 0x7f64e80051c0 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f650215c700 1 -- 172.21.15.45:0/2181698302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e80051c0 msgr2=0x7f64e800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f650215c700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e80051c0 0x7f64e800ba60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f64f400e210 tx=0x7f64f400b040 comp rx=0 tx=0).stop 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f650215c700 1 -- 172.21.15.45:0/2181698302 shutdown_connections 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f650215c700 1 --2- 172.21.15.45:0/2181698302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e80051c0 0x7f64e800ba60 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f650215c700 1 -- 172.21.15.45:0/2181698302 >> 172.21.15.45:0/2181698302 conn(0x7f64fc06f090 msgr2=0x7f64fc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.060+0000 7f650215c700 1 -- 172.21.15.45:0/2181698302 shutdown_connections 2023-10-06T05:21:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.061+0000 7f650215c700 1 -- 172.21.15.45:0/2181698302 wait complete. 2023-10-06T05:21:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.062+0000 7f650215c700 1 Processor -- start 2023-10-06T05:21:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.063+0000 7f650215c700 1 -- start start 2023-10-06T05:21:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.063+0000 7f650215c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e80051c0 0x7f64fc14a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.063+0000 7f650215c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64fc067f70 0x7f64fc0683e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.063+0000 7f650215c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64fc0c11c0 0x7f64fc0c1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f650215c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc14a870 con 0x7f64fc067f70 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f650215c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc068920 con 0x7f64e80051c0 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f650215c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc155740 con 0x7f64fc0c11c0 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64fc067f70 0x7f64fc0683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64fc067f70 0x7f64fc0683e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54778/0 (socket says 172.21.15.45:54778) 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 -- 172.21.15.45:0/1844717839 learned_addr learned my addr 172.21.15.45:0/1844717839 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64fb7fe700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e80051c0 0x7f64fc14a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64fbfff700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64fc0c11c0 0x7f64fc0c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 -- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64fc0c11c0 msgr2=0x7f64fc0c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64fc0c11c0 0x7f64fc0c1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 -- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e80051c0 msgr2=0x7f64fc14a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e80051c0 0x7f64fc14a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 -- 172.21.15.45:0/1844717839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc16a8c0 con 0x7f64fc067f70 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64fb7fe700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e80051c0 0x7f64fc14a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64faffd700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64fc067f70 0x7f64fc0683e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f64e40073c0 tx=0x7f64e4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64f37fe700 1 -- 172.21.15.45:0/1844717839 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4007850 con 0x7f64fc067f70 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64f37fe700 1 -- 172.21.15.45:0/1844717839 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400f870 con 0x7f64fc067f70 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc0c1b70 con 0x7f64fc067f70 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f64f37fe700 1 -- 172.21.15.45:0/1844717839 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400fc10 con 0x7f64fc067f70 2023-10-06T05:21:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.064+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc16adf0 con 0x7f64fc067f70 2023-10-06T05:21:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.065+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc06e2f0 con 0x7f64fc067f70 2023-10-06T05:21:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.066+0000 7f64f37fe700 1 -- 172.21.15.45:0/1844717839 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64e40133b0 con 0x7f64fc067f70 2023-10-06T05:21:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.066+0000 7f64f37fe700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64dc066f30 0x7f64dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.066+0000 7f64f37fe700 1 -- 172.21.15.45:0/1844717839 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f64e4084070 con 0x7f64fc067f70 2023-10-06T05:21:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.067+0000 7f64fb7fe700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64dc066f30 0x7f64dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.067+0000 7f64fb7fe700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64dc066f30 0x7f64dc0693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0b0260 tx=0x7f64ec007d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.071+0000 7f64f37fe700 1 -- 172.21.15.45:0/1844717839 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64e40510b0 con 0x7f64fc067f70 2023-10-06T05:21:32.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.322+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64fc04bb40 con 0x7f64fc067f70 2023-10-06T05:21:32.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.322+0000 7f64f37fe700 1 -- 172.21.15.45:0/1844717839 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f64e40546d0 con 0x7f64fc067f70 2023-10-06T05:21:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:32.324 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2705674276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2705674276},{"type":"v1","addr":"172.21.15.45:6834","nonce":2705674276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:30.891138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6514":{"gid":6514,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:32.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64dc066f30 msgr2=0x7f64dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64dc066f30 0x7f64dc0693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0b0260 tx=0x7f64ec007d10 comp rx=0 tx=0).stop 2023-10-06T05:21:32.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64fc067f70 msgr2=0x7f64fc0683e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64fc067f70 0x7f64fc0683e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f64e40073c0 tx=0x7f64e4001e70 comp rx=0 tx=0).stop 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 shutdown_connections 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64dc066f30 0x7f64dc0693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64fc0c11c0 0x7f64fc0c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64e80051c0 0x7f64fc14a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 --2- 172.21.15.45:0/1844717839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64fc067f70 0x7f64fc0683e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 >> 172.21.15.45:0/1844717839 conn(0x7f64fc06f090 msgr2=0x7f64fc156210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 shutdown_connections 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.324+0000 7f650215c700 1 -- 172.21.15.45:0/1844717839 wait complete. 2023-10-06T05:21:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 78 2023-10-06T05:21:32.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:20:53.763976+0000', 'modified': '2023-10-06T05:21:30.891138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6514': {'gid': 6514, 'name': 'd', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.144:6837/2999699668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 2999699668}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 2999699668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:21:32.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:21:32.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:21:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.881+0000 7f966715e700 1 Processor -- start 2023-10-06T05:21:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.881+0000 7f966715e700 1 -- start start 2023-10-06T05:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.882+0000 7f966715e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 0x7f96601570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.882+0000 7f966715e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966006bf40 con 0x7f966006bad0 2023-10-06T05:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.882+0000 7f966715e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966015b380 con 0x7f966015ac10 2023-10-06T05:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.882+0000 7f966715e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96600b02f0 con 0x7f9660156ce0 2023-10-06T05:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.882+0000 7f965ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f966006bad0 0x7f96600afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45360/0 (socket says 172.21.15.45:45360) 2023-10-06T05:21:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.882+0000 7f965ffff700 1 -- 172.21.15.45:0/2563918473 learned_addr learned my addr 172.21.15.45:0/2563918473 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.882+0000 7f9664efa700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 0x7f96601570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804629677 0 0) 0x7f966006bf40 con 0x7f966006bad0 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9650003570 con 0x7f966006bad0 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257362950 0 0) 0x7f9650003570 con 0x7f966006bad0 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 msgr2=0x7f96601570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 0x7f96601570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 >> v1:172.21.15.144:6789/0 conn(0x7f966015ac10 legacy=0x7f96600ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966015b660 con 0x7f966006bad0 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f964c001470 con 0x7f966006bad0 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 >> v1:172.21.15.45:6789/0 conn(0x7f966006bad0 legacy=0x7f96600afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9650003a90 0x7f9650003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650004810 0x7f9650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 0x7f96500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96600b02f0 con 0x7f9650004810 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965000bb10 con 0x7f9650003a90 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965000bcd0 con 0x7f9660156ce0 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965ffff700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650004810 0x7f9650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f9664efa700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9650003a90 0x7f9650003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f96656fb700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 0x7f96500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965ffff700 1 -- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 msgr2=0x7f96500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965ffff700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9660156ce0 0x7f96500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965ffff700 1 -- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9650003a90 msgr2=0x7f9650003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965ffff700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9650003a90 0x7f9650003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.883+0000 7f965ffff700 1 -- 172.21.15.45:0/2563918473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650003570 con 0x7f9650004810 2023-10-06T05:21:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f965ffff700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650004810 0x7f9650004c80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c3c0 tx=0x7f964c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46545e21008f6747 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00cc90 con 0x7f9650004810 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00cdf0 con 0x7f9650004810 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f965f7fe700 1 -- 172.21.15.45:0/2563918473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c005b70 con 0x7f9650004810 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f966715e700 1 -- 172.21.15.45:0/2563918473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650004810 msgr2=0x7f9650004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f966715e700 1 --2- 172.21.15.45:0/2563918473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650004810 0x7f9650004c80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c3c0 tx=0x7f964c001e70 comp rx=0 tx=0).stop 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f96656fb700 1 -- 172.21.15.45:0/2563918473 reap_dead start 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f966715e700 1 -- 172.21.15.45:0/2563918473 shutdown_connections 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f966715e700 1 -- 172.21.15.45:0/2563918473 >> 172.21.15.45:0/2563918473 conn(0x7f966006f090 msgr2=0x7f966006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f966715e700 1 -- 172.21.15.45:0/2563918473 shutdown_connections 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.884+0000 7f966715e700 1 -- 172.21.15.45:0/2563918473 wait complete. 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.886+0000 7f966715e700 1 Processor -- start 2023-10-06T05:21:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.886+0000 7f966715e700 1 -- start start 2023-10-06T05:21:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f966715e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650004810 0x7f9660141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f966715e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96600c25e0 0x7f9660157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f966715e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96600c2a50 0x7f9660157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f966715e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660141e80 con 0x7f96600c25e0 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f966715e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966015d1b0 con 0x7f96600c2a50 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f966715e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966015c5a0 con 0x7f9650004810 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96600c25e0 0x7f9660157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96600c25e0 0x7f9660157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54798/0 (socket says 172.21.15.45:54798) 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 -- 172.21.15.45:0/2924340059 learned_addr learned my addr 172.21.15.45:0/2924340059 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f96656fb700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96600c2a50 0x7f9660157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f9664efa700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650004810 0x7f9660141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 -- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650004810 msgr2=0x7f9660141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650004810 0x7f9660141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 -- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96600c2a50 msgr2=0x7f9660157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96600c2a50 0x7f9660157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 -- 172.21.15.45:0/2924340059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966015c880 con 0x7f96600c25e0 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965ffff700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96600c25e0 0x7f9660157680 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9650004770 tx=0x7f964c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965cff9700 1 -- 172.21.15.45:0/2924340059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0070d0 con 0x7f96600c25e0 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965cff9700 1 -- 172.21.15.45:0/2924340059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c007230 con 0x7f96600c25e0 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966015cb60 con 0x7f96600c25e0 2023-10-06T05:21:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.887+0000 7f965cff9700 1 -- 172.21.15.45:0/2924340059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0075f0 con 0x7f96600c25e0 2023-10-06T05:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.888+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966015c170 con 0x7f96600c25e0 2023-10-06T05:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.890+0000 7f965cff9700 1 -- 172.21.15.45:0/2924340059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f964c019030 con 0x7f96600c25e0 2023-10-06T05:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.890+0000 7f965cff9700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9644066f30 0x7f96440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.890+0000 7f965cff9700 1 -- 172.21.15.45:0/2924340059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f964c085bb0 con 0x7f96600c25e0 2023-10-06T05:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.890+0000 7f9664efa700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9644066f30 0x7f96440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.891+0000 7f9664efa700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9644066f30 0x7f96440693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9654006f00 tx=0x7f96540091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.891+0000 7f9670472b80 1 Processor -- start 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.891+0000 7f9670472b80 1 -- start start 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d177f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f9670472b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d178940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f9670472b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d042310 con 0x56010d1784d0 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d07f130 con 0x56010d154f40 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010d07caf0 con 0x56010d16f450 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f96656fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d178940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f96656fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d178940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54812/0 (socket says 172.21.15.45:54812) 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f96656fb700 1 -- 172.21.15.45:0/3668606199 learned_addr learned my addr 172.21.15.45:0/3668606199 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f9664efa700 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.892+0000 7f965ffff700 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d177f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f96656fb700 1 -- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 msgr2=0x56010d16f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f96656fb700 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f96656fb700 1 -- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 msgr2=0x56010d177f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f96656fb700 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d177f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f96656fb700 1 -- 172.21.15.45:0/3668606199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010ce7e790 con 0x56010d1784d0 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f9664efa700 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:32.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f965ffff700 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d177f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f96656fb700 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d178940 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f966015afe0 tx=0x7f96580016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830c4a7e2bae5463 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f9649ffb700 1 -- 172.21.15.45:0/3668606199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658006e80 con 0x56010d1784d0 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f9649ffb700 1 -- 172.21.15.45:0/3668606199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9658011810 con 0x56010d1784d0 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.893+0000 7f9649ffb700 1 -- 172.21.15.45:0/3668606199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658011b90 con 0x56010d1784d0 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 -- 172.21.15.45:0/3668606199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 msgr2=0x56010d178940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d178940 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f966015afe0 tx=0x7f96580016d0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 -- 172.21.15.45:0/3668606199 shutdown_connections 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d178940 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 --2- 172.21.15.45:0/3668606199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d177f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 -- 172.21.15.45:0/3668606199 >> 172.21.15.45:0/3668606199 conn(0x56010d172880 msgr2=0x56010d172c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 -- 172.21.15.45:0/3668606199 shutdown_connections 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 -- 172.21.15.45:0/3668606199 wait complete. 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.894+0000 7f9670472b80 1 -- start start 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.895+0000 7f9670472b80 1 Processor -- start 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.895+0000 7f9670472b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d154f40 0x56010d149930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.895+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.895+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d1784d0 0x56010d150970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.895+0000 7f9670472b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d0788e0 con 0x56010d154f40 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.895+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d024980 con 0x56010d1784d0 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.895+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010d024620 con 0x56010d16f450 2023-10-06T05:21:32.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9670472b80 10 client.0 _ll_register_callbacks cb 0x56010ce87a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d154f40 0x56010d149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d154f40 0x56010d149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54822/0 (socket says 172.21.15.45:54822) 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 -- 172.21.15.45:0/4203159379 learned_addr learned my addr 172.21.15.45:0/4203159379 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f96656fb700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d1784d0 0x56010d150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f965ffff700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 -- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 msgr2=0x56010d16eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 -- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d1784d0 msgr2=0x56010d150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d1784d0 0x56010d150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 -- 172.21.15.45:0/4203159379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965800f040 con 0x56010d154f40 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f96656fb700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d1784d0 0x56010d150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9664efa700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d154f40 0x56010d149930 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9654015b30 tx=0x7f9654019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d364b92ba5e23266 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f963effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f963effd700 1 -- 172.21.15.45:0/4203159379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654022830 con 0x56010d154f40 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f963effd700 1 -- 172.21.15.45:0/4203159379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f965402dde0 con 0x56010d154f40 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56010d11cd60 con 0x56010d154f40 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.896+0000 7f963effd700 1 -- 172.21.15.45:0/4203159379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9654022ba0 con 0x56010d154f40 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 10 client.6975 fetch_fsmap learned FSMap version 78 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x56010cf6b0a0 con 0x56010d154f40 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f963effd700 1 -- 172.21.15.45:0/4203159379 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56010d11cd60 con 0x56010d154f40 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f963effd700 1 -- 172.21.15.45:0/4203159379 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9654022d00 con 0x56010d154f40 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 10 client.6975 fetch_fsmap finished waiting for FSMap version 78 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 10 client.6975 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:21:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 20 client.6975 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d14f6a0 0x56010d14fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 4 client.6975 mds_command: new command op to 6514 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 --> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010d0f9b30 con 0x56010d14f6a0 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.897+0000 7f965ffff700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d14f6a0 0x56010d14fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.898+0000 7f965ffff700 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d14f6a0 0x56010d14fa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66122964cc70157e server_cookie=b963ba411f2626ca in_seq=0 out_seq=0 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.898+0000 7f963effd700 10 client.6975 ms_handle_connect on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.898+0000 7f963effd700 1 -- 172.21.15.45:0/4203159379 <== mds.0 v2:172.21.15.144:6835/2999699668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56010d0f9b30 con 0x56010d14f6a0 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.898+0000 7f963effd700 10 client.6975 handle_command_reply: tid=0 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.898+0000 7f9670472b80 1 client.6975 shutdown 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d154f40 msgr2=0x56010d149930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d154f40 0x56010d149930 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9654015b30 tx=0x7f9654019000 comp rx=0 tx=0).stop 2023-10-06T05:21:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 shutdown_connections 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d14f6a0 0x56010d14fa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d1784d0 0x56010d150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d154f40 0x56010d149930 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f963effd700 0 client.6975 ms_handle_reset on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 --2- 172.21.15.45:0/4203159379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d16eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 >> 172.21.15.45:0/4203159379 conn(0x56010d172880 msgr2=0x56010d124c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 shutdown_connections 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 1 -- 172.21.15.45:0/4203159379 wait complete. 2023-10-06T05:21:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.899+0000 7f9670472b80 20 client.6975 trim_cache size 0 max 16384 2023-10-06T05:21:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.917+0000 7f9670472b80 1 Processor -- start 2023-10-06T05:21:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.917+0000 7f9670472b80 1 -- start start 2023-10-06T05:21:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d11ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d14ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f9670472b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f9670472b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d1528d0 con 0x56010d1784d0 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010cf101e0 con 0x56010d154f40 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010d14f4d0 con 0x56010d16f450 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d14c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54830/0 (socket says 172.21.15.45:54830) 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 -- 172.21.15.45:0/1158685133 learned_addr learned my addr 172.21.15.45:0/1158685133 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f9664efa700 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d11ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f965ffff700 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d14ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 -- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 msgr2=0x56010d14ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d14ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 -- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 msgr2=0x56010d11ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d11ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f96656fb700 1 -- 172.21.15.45:0/1158685133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010d0473c0 con 0x56010d1784d0 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.918+0000 7f965ffff700 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d14ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f96656fb700 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d14c2a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f96600ade00 tx=0x7f9658019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec73108d3ac28d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f964a7fc700 1 -- 172.21.15.45:0/1158685133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658011980 con 0x56010d1784d0 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f964a7fc700 1 -- 172.21.15.45:0/1158685133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9658011b00 con 0x56010d1784d0 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f964a7fc700 1 -- 172.21.15.45:0/1158685133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96580019e0 con 0x56010d1784d0 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f9670472b80 1 -- 172.21.15.45:0/1158685133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 msgr2=0x56010d14c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f9670472b80 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d14c2a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f96600ade00 tx=0x7f9658019000 comp rx=0 tx=0).stop 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f9670472b80 1 -- 172.21.15.45:0/1158685133 shutdown_connections 2023-10-06T05:21:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f9670472b80 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d14c2a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f9670472b80 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d11ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f9670472b80 1 --2- 172.21.15.45:0/1158685133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d14ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.919+0000 7f9670472b80 1 -- 172.21.15.45:0/1158685133 >> 172.21.15.45:0/1158685133 conn(0x56010d172880 msgr2=0x56010d16fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.920+0000 7f9670472b80 1 -- 172.21.15.45:0/1158685133 shutdown_connections 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.920+0000 7f9670472b80 1 -- 172.21.15.45:0/1158685133 wait complete. 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.920+0000 7f9670472b80 1 -- start start 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.920+0000 7f9670472b80 1 Processor -- start 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d1d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9670472b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d1cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9670472b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d127e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9670472b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d1d4c60 con 0x56010d1784d0 2023-10-06T05:21:32.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d1291a0 con 0x56010d154f40 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9670472b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010d14a4c0 con 0x56010d16f450 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9670472b80 10 client.0 _ll_register_callbacks cb 0x56010ce87a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f96656fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d127e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f96656fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d127e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54846/0 (socket says 172.21.15.45:54846) 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f96656fb700 1 -- 172.21.15.45:0/2693550011 learned_addr learned my addr 172.21.15.45:0/2693550011 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f9664efa700 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d1d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.921+0000 7f965ffff700 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d1cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f96656fb700 1 -- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 msgr2=0x56010d1cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f96656fb700 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d1cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f96656fb700 1 -- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 msgr2=0x56010d1d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f96656fb700 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d1d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f96656fb700 1 -- 172.21.15.45:0/2693550011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010ce7e790 con 0x56010d1784d0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f96656fb700 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d127e20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9658002800 tx=0x7f9658024070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc0649f2d1d3260 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f963effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f963effd700 1 -- 172.21.15.45:0/2693550011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96580019e0 con 0x56010d1784d0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f963effd700 1 -- 172.21.15.45:0/2693550011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f9658001d50 con 0x56010d1784d0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9658011810 con 0x56010d1784d0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f963effd700 1 -- 172.21.15.45:0/2693550011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9658011980 con 0x56010d1784d0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f9670472b80 10 client.6981 fetch_fsmap learned FSMap version 78 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x56010cedf1d0 con 0x56010d1784d0 2023-10-06T05:21:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f963effd700 1 -- 172.21.15.45:0/2693550011 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9658011810 con 0x56010d1784d0 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f963effd700 1 -- 172.21.15.45:0/2693550011 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9658002ce0 con 0x56010d1784d0 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.922+0000 7f9670472b80 10 client.6981 fetch_fsmap finished waiting for FSMap version 78 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9670472b80 10 client.6981 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9670472b80 20 client.6981 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9670472b80 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d1248f0 0x56010d124cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9670472b80 4 client.6981 mds_command: new command op to 6514 tid=0[{"prefix": "status"}] 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 --> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56010d01b8d0 con 0x56010d1248f0 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9664efa700 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d1248f0 0x56010d124cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f963effd700 10 client.6981 ms_handle_connect on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9664efa700 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d1248f0 0x56010d124cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd0556b8df1c72a server_cookie=2a72e98a12b9cebb in_seq=0 out_seq=0 2023-10-06T05:21:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f963effd700 1 -- 172.21.15.45:0/2693550011 <== mds.0 v2:172.21.15.144:6835/2999699668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56010d01b8d0 con 0x56010d1248f0 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f963effd700 10 client.6981 handle_command_reply: tid=0 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.923+0000 7f9670472b80 1 client.6981 shutdown 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 msgr2=0x56010d127e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d127e20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9658002800 tx=0x7f9658024070 comp rx=0 tx=0).stop 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 shutdown_connections 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x56010d1248f0 0x56010d124cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010d1784d0 0x56010d127e20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010d154f40 0x56010d1d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f963effd700 0 client.6981 ms_handle_reset on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 --2- 172.21.15.45:0/2693550011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010d16f450 0x56010d1cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 >> 172.21.15.45:0/2693550011 conn(0x56010d172880 msgr2=0x56010d14c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 shutdown_connections 2023-10-06T05:21:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 1 -- 172.21.15.45:0/2693550011 wait complete. 2023-10-06T05:21:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f9670472b80 20 client.6981 trim_cache size 0 max 16384 2023-10-06T05:21:32.934 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:21:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:21:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6514, 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.0431608629999998, 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 78, 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 80, 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 80, 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 44.670179032 2023-10-06T05:21:32.935 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9644066f30 msgr2=0x7f96440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f966715e700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9644066f30 0x7f96440693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9654006f00 tx=0x7f96540091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96600c25e0 msgr2=0x7f9660157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.924+0000 7f966715e700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96600c25e0 0x7f9660157680 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9650004770 tx=0x7f964c015000 comp rx=0 tx=0).stop 2023-10-06T05:21:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 shutdown_connections 2023-10-06T05:21:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9644066f30 0x7f96440693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96600c2a50 0x7f9660157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650004810 0x7f9660141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 --2- 172.21.15.45:0/2924340059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96600c25e0 0x7f9660157680 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 >> 172.21.15.45:0/2924340059 conn(0x7f966006f090 msgr2=0x7f9660156210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 shutdown_connections 2023-10-06T05:21:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:32.925+0000 7f966715e700 1 -- 172.21.15.45:0/2924340059 wait complete. 2023-10-06T05:21:32.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:32.941 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat a/ && find a/') 2023-10-06T05:21:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: File: a/ 2023-10-06T05:21:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 1 IO Block: 4096 directory 2023-10-06T05:21:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 2fh/47d Inode: 1099511627776 Links: 4 2023-10-06T05:21:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1285/ ubuntu) 2023-10-06T05:21:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:fusefs_t:s0 2023-10-06T05:21:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-10-06 05:21:03.660867847 +0000 2023-10-06T05:21:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-10-06 05:21:09.292650288 +0000 2023-10-06T05:21:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-10-06 05:21:09.292650288 +0000 2023-10-06T05:21:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-10-06T05:21:32.998 INFO:teuthology.orchestra.run.smithi045.stdout:a/ 2023-10-06T05:21:32.998 INFO:teuthology.orchestra.run.smithi045.stdout:a/z 2023-10-06T05:21:32.998 INFO:teuthology.orchestra.run.smithi045.stdout:a/z/d 2023-10-06T05:21:32.999 INFO:teuthology.orchestra.run.smithi045.stdout:a/c 2023-10-06T05:21:32.999 INFO:teuthology.orchestra.run.smithi045.stdout:a/c/d 2023-10-06T05:21:32.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2023-10-06T05:21:33.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.178+0000 7fc45f9c3700 1 Processor -- start 2023-10-06T05:21:33.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.179+0000 7fc45f9c3700 1 -- start start 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.179+0000 7fc45f9c3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc458157330 0x7fc4580ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.179+0000 7fc45f9c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4580ccb70 0x7fc4580c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.179+0000 7fc45f9c3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580c3b50 con 0x7fc4580aaae0 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45f9c3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc458157700 con 0x7fc4580ccb70 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45f9c3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4580c98e0 con 0x7fc458157330 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4580ccb70 0x7fc4580c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4580ccb70 0x7fc4580c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54856/0 (socket says 172.21.15.45:54856) 2023-10-06T05:21:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 -- 172.21.15.45:0/3802371350 learned_addr learned my addr 172.21.15.45:0/3802371350 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45d75f700 1 --2- 172.21.15.45:0/3802371350 >> v2:172.21.15.144:3300/0 conn(0x7fc458157330 0x7fc4580ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 -- 172.21.15.45:0/3802371350 >> v2:172.21.15.144:3300/0 conn(0x7fc458157330 msgr2=0x7fc4580ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 --2- 172.21.15.45:0/3802371350 >> v2:172.21.15.144:3300/0 conn(0x7fc458157330 0x7fc4580ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 -- 172.21.15.45:0/3802371350 >> v1:172.21.15.144:6789/0 conn(0x7fc4580aaae0 legacy=0x7fc4580c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 -- 172.21.15.45:0/3802371350 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580c9bc0 con 0x7fc4580ccb70 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc45df60700 1 --2- 172.21.15.45:0/3802371350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580ccb70 0x7fc4580c9280 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc4580cd1d0 tx=0x7fc450009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994c9bb403d486d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc44ffff700 1 -- 172.21.15.45:0/3802371350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000e070 con 0x7fc4580ccb70 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc44ffff700 1 -- 172.21.15.45:0/3802371350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45000ba50 con 0x7fc4580ccb70 2023-10-06T05:21:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.180+0000 7fc44ffff700 1 -- 172.21.15.45:0/3802371350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500092e0 con 0x7fc4580ccb70 2023-10-06T05:21:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 -- 172.21.15.45:0/3802371350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580ccb70 msgr2=0x7fc4580c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/3802371350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580ccb70 0x7fc4580c9280 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc4580cd1d0 tx=0x7fc450009840 comp rx=0 tx=0).stop 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 -- 172.21.15.45:0/3802371350 shutdown_connections 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/3802371350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580ccb70 0x7fc4580c9280 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/3802371350 >> v2:172.21.15.144:3300/0 conn(0x7fc458157330 0x7fc4580ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 -- 172.21.15.45:0/3802371350 >> 172.21.15.45:0/3802371350 conn(0x7fc458067e00 msgr2=0x7fc4580cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 -- 172.21.15.45:0/3802371350 shutdown_connections 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.181+0000 7fc45f9c3700 1 -- 172.21.15.45:0/3802371350 wait complete. 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.183+0000 7fc45f9c3700 1 Processor -- start 2023-10-06T05:21:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.183+0000 7fc45f9c3700 1 -- start start 2023-10-06T05:21:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45f9c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580aaae0 0x7fc458156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45f9c3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4580ccb70 0x7fc45814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45f9c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc458157330 0x7fc4581520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45f9c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458156b90 con 0x7fc4580aaae0 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45f9c3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580ae7b0 con 0x7fc458157330 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45f9c3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45816aa80 con 0x7fc4580ccb70 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580aaae0 0x7fc458156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580aaae0 0x7fc458156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54868/0 (socket says 172.21.15.45:54868) 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 -- 172.21.15.45:0/2050094388 learned_addr learned my addr 172.21.15.45:0/2050094388 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45df60700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc458157330 0x7fc4581520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4580ccb70 0x7fc45814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 -- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4580ccb70 msgr2=0x7fc45814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4580ccb70 0x7fc45814cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 -- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc458157330 msgr2=0x7fc4581520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc458157330 0x7fc4581520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45d75f700 1 -- 172.21.15.45:0/2050094388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc450009ae0 con 0x7fc4580aaae0 2023-10-06T05:21:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.184+0000 7fc45df60700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc458157330 0x7fc4581520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.185+0000 7fc45d75f700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580aaae0 0x7fc458156650 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d810 tx=0x7fc454008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.185+0000 7fc44d7fa700 1 -- 172.21.15.45:0/2050094388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540125f0 con 0x7fc4580aaae0 2023-10-06T05:21:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.185+0000 7fc44d7fa700 1 -- 172.21.15.45:0/2050094388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454012750 con 0x7fc4580aaae0 2023-10-06T05:21:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.185+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45816ad60 con 0x7fc4580aaae0 2023-10-06T05:21:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.185+0000 7fc44d7fa700 1 -- 172.21.15.45:0/2050094388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454012b10 con 0x7fc4580aaae0 2023-10-06T05:21:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.185+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45816b320 con 0x7fc4580aaae0 2023-10-06T05:21:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.186+0000 7fc44d7fa700 1 -- 172.21.15.45:0/2050094388 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc45401e460 con 0x7fc4580aaae0 2023-10-06T05:21:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.187+0000 7fc44d7fa700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.187+0000 7fc44d7fa700 1 -- 172.21.15.45:0/2050094388 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc454084fa0 con 0x7fc4580aaae0 2023-10-06T05:21:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.187+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.188+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc458150c40 tx=0x7fc448009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.189+0000 7fc468cd7b80 1 Processor -- start 2023-10-06T05:21:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.189+0000 7fc468cd7b80 1 -- start start 2023-10-06T05:21:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981c44a0 0x55d2981c4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d2981ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981cd520 0x55d2981cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d298097360 con 0x55d2981cd520 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2980d4180 con 0x55d2981c44a0 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2980d1b40 con 0x55d2981a9f90 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981cd520 0x55d2981cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981cd520 0x55d2981cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54870/0 (socket says 172.21.15.45:54870) 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 -- 172.21.15.45:0/3014087565 learned_addr learned my addr 172.21.15.45:0/3014087565 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45d75f700 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981c44a0 0x55d2981c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d2981ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 -- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 msgr2=0x55d2981ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d2981ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 -- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981c44a0 msgr2=0x55d2981c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981c44a0 0x55d2981c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.190+0000 7fc45df60700 1 -- 172.21.15.45:0/3014087565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d297ed37e0 con 0x55d2981cd520 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc45df60700 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981cd520 0x55d2981cd990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc45806bac0 tx=0x7fc450018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd15c444cbc27e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc4427fc700 1 -- 172.21.15.45:0/3014087565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450009940 con 0x55d2981cd520 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc4427fc700 1 -- 172.21.15.45:0/3014087565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45000be60 con 0x55d2981cd520 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc4427fc700 1 -- 172.21.15.45:0/3014087565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450007200 con 0x55d2981cd520 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc468cd7b80 1 -- 172.21.15.45:0/3014087565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981cd520 msgr2=0x55d2981cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981cd520 0x55d2981cd990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc45806bac0 tx=0x7fc450018000 comp rx=0 tx=0).stop 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc468cd7b80 1 -- 172.21.15.45:0/3014087565 shutdown_connections 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981cd520 0x55d2981cd990 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981c44a0 0x55d2981c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/3014087565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d2981ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.191+0000 7fc468cd7b80 1 -- 172.21.15.45:0/3014087565 >> 172.21.15.45:0/3014087565 conn(0x55d2981c78d0 msgr2=0x55d2981c7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.192+0000 7fc468cd7b80 1 -- 172.21.15.45:0/3014087565 shutdown_connections 2023-10-06T05:21:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.192+0000 7fc468cd7b80 1 -- 172.21.15.45:0/3014087565 wait complete. 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.192+0000 7fc468cd7b80 1 -- start start 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.193+0000 7fc468cd7b80 1 Processor -- start 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d29819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981cd520 0x55d2981a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2980cd930 con 0x55d2981c44a0 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2980799d0 con 0x55d2981cd520 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d298079670 con 0x55d2981a9f90 2023-10-06T05:21:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc468cd7b80 10 client.0 _ll_register_callbacks cb 0x55d297edca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54876/0 (socket says 172.21.15.45:54876) 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 -- 172.21.15.45:0/1403266461 learned_addr learned my addr 172.21.15.45:0/1403266461 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45d75f700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d29819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45df60700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981cd520 0x55d2981a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 -- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 msgr2=0x55d29819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d29819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 -- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981cd520 msgr2=0x55d2981a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981cd520 0x55d2981a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45cf5e700 1 -- 172.21.15.45:0/1403266461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4580c9bc0 con 0x55d2981c44a0 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.194+0000 7fc45df60700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981cd520 0x55d2981a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c61e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc448017f90 tx=0x7fc448019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa63bebd6bade2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc4377fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc4377fe700 1 -- 172.21.15.45:0/1403266461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44801c070 con 0x55d2981c44a0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc4377fe700 1 -- 172.21.15.45:0/1403266461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc448022d70 con 0x55d2981c44a0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d298171db0 con 0x55d2981c44a0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc4377fe700 1 -- 172.21.15.45:0/1403266461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc448001ea0 con 0x55d2981c44a0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 10 client.6993 fetch_fsmap learned FSMap version 78 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55d297fc00f0 con 0x55d2981c44a0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc4377fe700 1 -- 172.21.15.45:0/1403266461 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d298171db0 con 0x55d2981c44a0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc4377fe700 1 -- 172.21.15.45:0/1403266461 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc448022410 con 0x55d2981c44a0 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 10 client.6993 fetch_fsmap finished waiting for FSMap version 78 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 10 client.6993 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2023-10-06T05:21:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 20 client.6993 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981a1120 0x55d2981a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 4 client.6993 mds_command: new command op to 6514 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.195+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 --> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d29814eb80 con 0x55d2981a1120 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.196+0000 7fc45d75f700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981a1120 0x55d2981a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.196+0000 7fc4377fe700 10 client.6993 ms_handle_connect on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.196+0000 7fc45d75f700 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981a1120 0x55d2981a1520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b436f500374cccd server_cookie=e2c173908a121400 in_seq=0 out_seq=0 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.196+0000 7fc4377fe700 1 -- 172.21.15.45:0/1403266461 <== mds.0 v2:172.21.15.144:6835/2999699668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d29814eb80 con 0x55d2981a1120 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.196+0000 7fc4377fe700 10 client.6993 handle_command_reply: tid=0 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 client.6993 shutdown 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 msgr2=0x55d2981c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c61e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc448017f90 tx=0x7fc448019040 comp rx=0 tx=0).stop 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 shutdown_connections 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981a1120 0x55d2981a1520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981cd520 0x55d2981a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981a9f90 0x55d29819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc4377fe700 0 client.6993 ms_handle_reset on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/1403266461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c61e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 >> 172.21.15.45:0/1403266461 conn(0x55d2981c78d0 msgr2=0x55d298179cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 shutdown_connections 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 1 -- 172.21.15.45:0/1403266461 wait complete. 2023-10-06T05:21:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.197+0000 7fc468cd7b80 20 client.6993 trim_cache size 0 max 16384 2023-10-06T05:21:33.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 Processor -- start 2023-10-06T05:21:33.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 -- start start 2023-10-06T05:21:33.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981acd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298173a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d297f65230 con 0x55d2981c44a0 2023-10-06T05:21:33.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2981a3110 con 0x55d2981a9f90 2023-10-06T05:21:33.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d29817bb50 con 0x55d2981cd520 2023-10-06T05:21:33.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.212+0000 7fc45cf5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981a7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54880/0 (socket says 172.21.15.45:54880) 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 -- 172.21.15.45:0/4033357382 learned_addr learned my addr 172.21.15.45:0/4033357382 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45df60700 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298173a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45d75f700 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 -- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 msgr2=0x55d298173a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298173a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 -- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 msgr2=0x55d2981acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 -- 172.21.15.45:0/4033357382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d29809c410 con 0x55d2981c44a0 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981a7480 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc4480173a0 tx=0x7fc448016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0cae2cd6cdbe49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc442ffd700 1 -- 172.21.15.45:0/4033357382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448012e40 con 0x55d2981c44a0 2023-10-06T05:21:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc442ffd700 1 -- 172.21.15.45:0/4033357382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44801d040 con 0x55d2981c44a0 2023-10-06T05:21:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.213+0000 7fc442ffd700 1 -- 172.21.15.45:0/4033357382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448001de0 con 0x55d2981c44a0 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 -- 172.21.15.45:0/4033357382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 msgr2=0x55d2981a7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981a7480 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc4480173a0 tx=0x7fc448016000 comp rx=0 tx=0).stop 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 -- 172.21.15.45:0/4033357382 shutdown_connections 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298173a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/4033357382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981a7480 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 -- 172.21.15.45:0/4033357382 >> 172.21.15.45:0/4033357382 conn(0x55d2981c78d0 msgr2=0x55d29817a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 -- 172.21.15.45:0/4033357382 shutdown_connections 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 -- 172.21.15.45:0/4033357382 wait complete. 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.214+0000 7fc468cd7b80 1 -- start start 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.215+0000 7fc468cd7b80 1 Processor -- start 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298179120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29818d620 con 0x55d2981c44a0 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2981a3510 con 0x55d2981a9f90 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2981741f0 con 0x55d2981cd520 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 10 client.0 _ll_register_callbacks cb 0x55d297edca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54896/0 (socket says 172.21.15.45:54896) 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 -- 172.21.15.45:0/815707719 learned_addr learned my addr 172.21.15.45:0/815707719 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45d75f700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45df60700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298179120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 -- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 msgr2=0x55d298179120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298179120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 -- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 msgr2=0x55d2981c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 -- 172.21.15.45:0/815707719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d297fc00f0 con 0x55d2981c44a0 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45d75f700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc45cf5e700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c58b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc448000fb0 tx=0x7fc448032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e33858198367e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc4377fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc4377fe700 1 -- 172.21.15.45:0/815707719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448001de0 con 0x55d2981c44a0 2023-10-06T05:21:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc4377fe700 1 -- 172.21.15.45:0/815707719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc448017bc0 con 0x55d2981c44a0 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.216+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc45000be60 con 0x55d2981c44a0 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc4377fe700 1 -- 172.21.15.45:0/815707719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4480228d0 con 0x55d2981c44a0 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 10 client.6999 fetch_fsmap learned FSMap version 78 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55d297f34220 con 0x55d2981c44a0 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc4377fe700 1 -- 172.21.15.45:0/815707719 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45000be60 con 0x55d2981c44a0 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc4377fe700 1 -- 172.21.15.45:0/815707719 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc448035430 con 0x55d2981c44a0 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 10 client.6999 fetch_fsmap finished waiting for FSMap version 78 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 10 client.6999 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 20 client.6999 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981ac7a0 0x55d298179660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 4 client.6999 mds_command: new command op to 6514 tid=0[{"prefix": "flush journal"}] 2023-10-06T05:21:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 --> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d298070920 con 0x55d2981ac7a0 2023-10-06T05:21:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.217+0000 7fc45d75f700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981ac7a0 0x55d298179660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.218+0000 7fc45d75f700 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981ac7a0 0x55d298179660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad902ba0f21be05 server_cookie=fb343b97d79bd5bf in_seq=0 out_seq=0 2023-10-06T05:21:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.218+0000 7fc4377fe700 10 client.6999 ms_handle_connect on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.230+0000 7fc4377fe700 1 -- 172.21.15.45:0/815707719 <== mds.0 v2:172.21.15.144:6835/2999699668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d298070920 con 0x55d2981ac7a0 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.230+0000 7fc4377fe700 10 client.6999 handle_command_reply: tid=0 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.230+0000 7fc468cd7b80 1 client.6999 shutdown 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 msgr2=0x55d2981c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c58b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc448000fb0 tx=0x7fc448032000 comp rx=0 tx=0).stop 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 shutdown_connections 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:6835/2999699668,v1:172.21.15.144:6837/2999699668] conn(0x55d2981ac7a0 0x55d298179660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d2981cd520 0x55d298179120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d2981a9f90 0x55d2981c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc4377fe700 0 client.6999 ms_handle_reset on v2:172.21.15.144:6835/2999699668 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 --2- 172.21.15.45:0/815707719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2981c44a0 0x55d2981c58b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 >> 172.21.15.45:0/815707719 conn(0x55d2981c78d0 msgr2=0x55d29817ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 shutdown_connections 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 1 -- 172.21.15.45:0/815707719 wait complete. 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.231+0000 7fc468cd7b80 20 client.6999 trim_cache size 0 max 16384 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2023-10-06T05:21:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2023-10-06T05:21:33.235 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 msgr2=0x7fc43c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc458150c40 tx=0x7fc448009040 comp rx=0 tx=0).stop 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580aaae0 msgr2=0x7fc458156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580aaae0 0x7fc458156650 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d810 tx=0x7fc454008040 comp rx=0 tx=0).stop 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 shutdown_connections 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc43c066f30 0x7fc43c0693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc458157330 0x7fc4581520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4580aaae0 0x7fc458156650 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 --2- 172.21.15.45:0/2050094388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4580ccb70 0x7fc45814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 >> 172.21.15.45:0/2050094388 conn(0x7fc458067e00 msgr2=0x7fc45806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 shutdown_connections 2023-10-06T05:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.232+0000 7fc45f9c3700 1 -- 172.21.15.45:0/2050094388 wait complete. 2023-10-06T05:21:33.247 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:21:33.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:33.296 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:33.353 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:21:33.353 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:21:33.354 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:21:33.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:33.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[150983]: fuse finished with error 0 and tester_r 0 2023-10-06T05:21:33.400 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:21:33.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:33.400 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:21:33.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:21:33.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:33.468 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:21:33.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:21:33.488 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T05:21:33.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:21:33.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:33.488 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:21:33.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:21:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:21:33.512 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:21:33.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:33.513 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:21:33.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:21:33.549 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:21:33.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.692+0000 7fdbbf39e700 1 Processor -- start 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.692+0000 7fdbbf39e700 1 -- start start 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbf39e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdbb8159330 0x7fdbb806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbf39e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb80cb530 con 0x7fdbb80cc390 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbf39e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb8069bd0 con 0x7fdbb8158f60 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbf39e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb8068fc0 con 0x7fdbb8159330 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbc939700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdbb8159330 0x7fdbb806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbd93b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdbb80cc390 0x7fdbb80c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45364/0 (socket says 172.21.15.45:45364) 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbd93b700 1 -- 172.21.15.45:0/3905517869 learned_addr learned my addr 172.21.15.45:0/3905517869 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbc939700 1 -- 172.21.15.45:0/3905517869 >> v1:172.21.15.144:6789/0 conn(0x7fdbb8158f60 legacy=0x7fdbb80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbc939700 1 -- 172.21.15.45:0/3905517869 >> v1:172.21.15.45:6789/0 conn(0x7fdbb80cc390 legacy=0x7fdbb80c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbc939700 1 -- 172.21.15.45:0/3905517869 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb80692a0 con 0x7fdbb8159330 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbaffff700 1 -- 172.21.15.45:0/3905517869 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215738494 0 0) 0x7fdbb0001490 con 0x7fdbb80cc390 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.693+0000 7fdbbc939700 1 --2- 172.21.15.45:0/3905517869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8159330 0x7fdbb806a060 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098e0 tx=0x7fdba80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7a42ec8998bc94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbaffff700 1 -- 172.21.15.45:0/3905517869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8009e80 con 0x7fdbb8159330 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbaffff700 1 -- 172.21.15.45:0/3905517869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba800b860 con 0x7fdbb8159330 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbaffff700 1 -- 172.21.15.45:0/3905517869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800bbe0 con 0x7fdbb8159330 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbbf39e700 1 -- 172.21.15.45:0/3905517869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8159330 msgr2=0x7fdbb806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/3905517869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8159330 0x7fdbb806a060 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098e0 tx=0x7fdba80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbbf39e700 1 -- 172.21.15.45:0/3905517869 shutdown_connections 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/3905517869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8159330 0x7fdbb806a060 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbbf39e700 1 -- 172.21.15.45:0/3905517869 >> 172.21.15.45:0/3905517869 conn(0x7fdbb80cdf50 msgr2=0x7fdbb80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbbf39e700 1 -- 172.21.15.45:0/3905517869 shutdown_connections 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.694+0000 7fdbbf39e700 1 -- 172.21.15.45:0/3905517869 wait complete. 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.696+0000 7fdbbf39e700 1 Processor -- start 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.696+0000 7fdbbf39e700 1 -- start start 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbf39e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbb80cc390 0x7fdbb8154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbf39e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8158f60 0x7fdbb80c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbf39e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbb8159330 0x7fdbb815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbf39e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80c6440 con 0x7fdbb8158f60 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbf39e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80c6720 con 0x7fdbb8159330 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbf39e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80c6a00 con 0x7fdbb80cc390 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbc939700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8158f60 0x7fdbb80c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbc939700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8158f60 0x7fdbb80c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54916/0 (socket says 172.21.15.45:54916) 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbc939700 1 -- 172.21.15.45:0/653856220 learned_addr learned my addr 172.21.15.45:0/653856220 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbd93b700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbb8159330 0x7fdbb815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.697+0000 7fdbbd13a700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbb80cc390 0x7fdbb8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbc939700 1 -- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbb80cc390 msgr2=0x7fdbb8154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbc939700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbb80cc390 0x7fdbb8154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbc939700 1 -- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbb8159330 msgr2=0x7fdbb815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbc939700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbb8159330 0x7fdbb815a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbc939700 1 -- 172.21.15.45:0/653856220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009420 con 0x7fdbb8158f60 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbd13a700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbb80cc390 0x7fdbb8154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbc939700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8158f60 0x7fdbb80c59c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80c4f80 tx=0x7fdba8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbad7fa700 1 -- 172.21.15.45:0/653856220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba801f070 con 0x7fdbb8158f60 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbad7fa700 1 -- 172.21.15.45:0/653856220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba800b9d0 con 0x7fdbb8158f60 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb80c6ce0 con 0x7fdbb8158f60 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbad7fa700 1 -- 172.21.15.45:0/653856220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8011670 con 0x7fdbb8158f60 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.698+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb8154d60 con 0x7fdbb8158f60 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.699+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb80c7200 con 0x7fdbb8158f60 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.700+0000 7fdbad7fa700 1 -- 172.21.15.45:0/653856220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdba80119e0 con 0x7fdbb8158f60 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.700+0000 7fdbad7fa700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb9c066f30 0x7fdb9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.700+0000 7fdbad7fa700 1 -- 172.21.15.45:0/653856220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fdba8085c20 con 0x7fdbb8158f60 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.700+0000 7fdbbd13a700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb9c066f30 0x7fdb9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.701+0000 7fdbbd13a700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb9c066f30 0x7fdb9c0693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4007fd0 tx=0x7fdbb4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.705+0000 7fdbad7fa700 1 -- 172.21.15.45:0/653856220 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbb80c7200 con 0x7fdbb8158f60 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.952+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbb80c7200 con 0x7fdbb8158f60 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.953+0000 7fdbad7fa700 1 -- 172.21.15.45:0/653856220 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fdbb80c7200 con 0x7fdbb8158f60 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2705674276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2705674276},{"type":"v1","addr":"172.21.15.45:6834","nonce":2705674276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:30.891138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6514":{"gid":6514,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb9c066f30 msgr2=0x7fdb9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb9c066f30 0x7fdb9c0693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4007fd0 tx=0x7fdbb4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8158f60 msgr2=0x7fdbb80c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8158f60 0x7fdbb80c59c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80c4f80 tx=0x7fdba8017000 comp rx=0 tx=0).stop 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 shutdown_connections 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb9c066f30 0x7fdb9c0693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbb8159330 0x7fdbb815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbb80cc390 0x7fdbb8154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 --2- 172.21.15.45:0/653856220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb8158f60 0x7fdbb80c59c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 >> 172.21.15.45:0/653856220 conn(0x7fdbb80cdf50 msgr2=0x7fdbb8070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 shutdown_connections 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:33.955+0000 7fdbbf39e700 1 -- 172.21.15.45:0/653856220 wait complete. 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 78 2023-10-06T05:21:33.971 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.102+0000 7f16ab1c4700 1 Processor -- start 2023-10-06T05:21:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.102+0000 7f16ab1c4700 1 -- start start 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16ab1c4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16a40aed00 0x7f16a40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16ab1c4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16a40af0d0 0x7f16a40c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16ab1c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16ab1c4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40cb3e0 con 0x7f16a40af0d0 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16ab1c4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40cb990 con 0x7f16a40aed00 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16ab1c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40cbe20 con 0x7f16a40c9c90 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16a40af0d0 0x7f16a40c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16a40af0d0 0x7f16a40c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54928/0 (socket says 172.21.15.45:54928) 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 -- 172.21.15.45:0/2741107759 learned_addr learned my addr 172.21.15.45:0/2741107759 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a8f60700 1 --2- 172.21.15.45:0/2741107759 >> v2:172.21.15.144:3300/0 conn(0x7f16a40aed00 0x7f16a40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a9761700 1 --2- 172.21.15.45:0/2741107759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 -- 172.21.15.45:0/2741107759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 msgr2=0x7f16a406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 --2- 172.21.15.45:0/2741107759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a406c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 -- 172.21.15.45:0/2741107759 >> v2:172.21.15.144:3300/0 conn(0x7f16a40aed00 msgr2=0x7f16a40cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 --2- 172.21.15.45:0/2741107759 >> v2:172.21.15.144:3300/0 conn(0x7f16a40aed00 0x7f16a40cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 -- 172.21.15.45:0/2741107759 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40c9190 con 0x7f16a40af0d0 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a8f60700 1 --2- 172.21.15.45:0/2741107759 >> v2:172.21.15.144:3300/0 conn(0x7f16a40aed00 0x7f16a40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.103+0000 7f16a3fff700 1 --2- 172.21.15.45:0/2741107759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40af0d0 0x7f16a40c9750 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f16900099b0 tx=0x7f1690009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85194004e51122de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16a37fe700 1 -- 172.21.15.45:0/2741107759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000b820 con 0x7f16a40af0d0 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16a37fe700 1 -- 172.21.15.45:0/2741107759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169000b980 con 0x7f16a40af0d0 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16a37fe700 1 -- 172.21.15.45:0/2741107759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000bd20 con 0x7f16a40af0d0 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 -- 172.21.15.45:0/2741107759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40af0d0 msgr2=0x7f16a40c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/2741107759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40af0d0 0x7f16a40c9750 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f16900099b0 tx=0x7f1690009250 comp rx=0 tx=0).stop 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 -- 172.21.15.45:0/2741107759 shutdown_connections 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/2741107759 >> v2:172.21.15.144:3300/0 conn(0x7f16a40aed00 0x7f16a40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/2741107759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a406c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/2741107759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40af0d0 0x7f16a40c9750 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 -- 172.21.15.45:0/2741107759 >> 172.21.15.45:0/2741107759 conn(0x7f16a4069200 msgr2=0x7f16a40cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 -- 172.21.15.45:0/2741107759 shutdown_connections 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.104+0000 7f16ab1c4700 1 -- 172.21.15.45:0/2741107759 wait complete. 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.105+0000 7f16ab1c4700 1 Processor -- start 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.106+0000 7f16ab1c4700 1 -- start start 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.106+0000 7f16ab1c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40aed00 0x7f16a4166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16ab1c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a40af0d0 0x7f16a415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16ab1c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a4147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16ab1c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4169980 con 0x7f16a40aed00 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16ab1c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4154f30 con 0x7f16a40af0d0 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16ab1c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4147f40 con 0x7f16a40c9c90 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40aed00 0x7f16a4166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40aed00 0x7f16a4166250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54932/0 (socket says 172.21.15.45:54932) 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 -- 172.21.15.45:0/3982395199 learned_addr learned my addr 172.21.15.45:0/3982395199 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a9761700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a4147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a3fff700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a40af0d0 0x7f16a415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 -- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 msgr2=0x7f16a4147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a4147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 -- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a40af0d0 msgr2=0x7f16a415b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a40af0d0 0x7f16a415b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 -- 172.21.15.45:0/3982395199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900094f0 con 0x7f16a40aed00 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a8f60700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40aed00 0x7f16a4166250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f169800e8d0 tx=0x7f1698008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a0ff9700 1 -- 172.21.15.45:0/3982395199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800ee70 con 0x7f16a40aed00 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a0ff9700 1 -- 172.21.15.45:0/3982395199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800c5e0 con 0x7f16a40aed00 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a4166880 con 0x7f16a40aed00 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.107+0000 7f16a0ff9700 1 -- 172.21.15.45:0/3982395199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800c980 con 0x7f16a40aed00 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.108+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a4166e40 con 0x7f16a40aed00 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.108+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a4169c60 con 0x7f16a40aed00 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.109+0000 7f16a0ff9700 1 -- 172.21.15.45:0/3982395199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1698013410 con 0x7f16a40aed00 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.109+0000 7f16a0ff9700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1688066f30 0x7f16880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.109+0000 7f16a0ff9700 1 -- 172.21.15.45:0/3982395199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f1698084610 con 0x7f16a40aed00 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.109+0000 7f16a3fff700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1688066f30 0x7f16880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.110+0000 7f16a3fff700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1688066f30 0x7f16880693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f16900099b0 tx=0x7f1690017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.114+0000 7f16a0ff9700 1 -- 172.21.15.45:0/3982395199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1698051590 con 0x7f16a40aed00 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.320+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16a40c9470 con 0x7f16a40aed00 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.321+0000 7f16a0ff9700 1 -- 172.21.15.45:0/3982395199 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17299 (secure 0 0 0) 0x7f1698054bb0 con 0x7f16a40aed00 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":80,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:21:27.879512+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:51.554669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:52.645101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/1530850582":"2023-10-07T05:21:27.879438+0000","172.21.15.45:6832/1530850582":"2023-10-07T05:21:27.879438+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1688066f30 msgr2=0x7f16880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1688066f30 0x7f16880693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f16900099b0 tx=0x7f1690017000 comp rx=0 tx=0).stop 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40aed00 msgr2=0x7f16a4166250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40aed00 0x7f16a4166250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f169800e8d0 tx=0x7f1698008040 comp rx=0 tx=0).stop 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 shutdown_connections 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1688066f30 0x7f16880693e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16a40c9c90 0x7f16a4147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a40aed00 0x7f16a4166250 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 --2- 172.21.15.45:0/3982395199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a40af0d0 0x7f16a415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 >> 172.21.15.45:0/3982395199 conn(0x7f16a4069200 msgr2=0x7f16a40ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.323+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 shutdown_connections 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.324+0000 7f16ab1c4700 1 -- 172.21.15.45:0/3982395199 wait complete. 2023-10-06T05:21:34.339 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:20:51.554669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:20:52.645101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:21:34.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.472+0000 7f5433292700 1 Processor -- start 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.472+0000 7f5433292700 1 -- start start 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f5433292700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f542c0d02b0 0x7f542c0d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f5433292700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c15c8c0 0x7f542c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f5433292700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0cd740 con 0x7f542c15d820 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f5433292700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c0ccb30 con 0x7f542c0d02b0 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f5433292700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0cce10 con 0x7f542c15c8c0 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f543082d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f542c15d820 0x7f542c15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45372/0 (socket says 172.21.15.45:45372) 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f543082d700 1 -- 172.21.15.45:0/1422733993 learned_addr learned my addr 172.21.15.45:0/1422733993 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f543102e700 1 --2- 172.21.15.45:0/1422733993 >> v2:172.21.15.144:3300/0 conn(0x7f542c0d02b0 0x7f542c0d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.473+0000 7f543182f700 1 --2- 172.21.15.45:0/1422733993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c15c8c0 0x7f542c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f541ffff700 1 -- 172.21.15.45:0/1422733993 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676987425 0 0) 0x7f542c0cd740 con 0x7f542c15d820 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f541ffff700 1 -- 172.21.15.45:0/1422733993 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5418003570 con 0x7f542c15d820 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f543182f700 1 -- 172.21.15.45:0/1422733993 >> v2:172.21.15.144:3300/0 conn(0x7f542c0d02b0 msgr2=0x7f542c0d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f543182f700 1 --2- 172.21.15.45:0/1422733993 >> v2:172.21.15.144:3300/0 conn(0x7f542c0d02b0 0x7f542c0d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f543182f700 1 -- 172.21.15.45:0/1422733993 >> v1:172.21.15.45:6789/0 conn(0x7f542c15d820 legacy=0x7f542c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f543182f700 1 -- 172.21.15.45:0/1422733993 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0cd0f0 con 0x7f542c15c8c0 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f543102e700 1 --2- 172.21.15.45:0/1422733993 >> v2:172.21.15.144:3300/0 conn(0x7f542c0d02b0 0x7f542c0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f543182f700 1 --2- 172.21.15.45:0/1422733993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c15c8c0 0x7f542c0caf10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5424009e60 tx=0x7f5424009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc95d5a7cfafce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f541ffff700 1 -- 172.21.15.45:0/1422733993 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400e070 con 0x7f542c15c8c0 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f541ffff700 1 -- 172.21.15.45:0/1422733993 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400bab0 con 0x7f542c15c8c0 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.474+0000 7f541ffff700 1 -- 172.21.15.45:0/1422733993 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400be30 con 0x7f542c15c8c0 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 -- 172.21.15.45:0/1422733993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c15c8c0 msgr2=0x7f542c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 --2- 172.21.15.45:0/1422733993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c15c8c0 0x7f542c0caf10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5424009e60 tx=0x7f5424009780 comp rx=0 tx=0).stop 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 -- 172.21.15.45:0/1422733993 shutdown_connections 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 --2- 172.21.15.45:0/1422733993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c15c8c0 0x7f542c0caf10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 --2- 172.21.15.45:0/1422733993 >> v2:172.21.15.144:3300/0 conn(0x7f542c0d02b0 0x7f542c0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 -- 172.21.15.45:0/1422733993 >> 172.21.15.45:0/1422733993 conn(0x7f542c06ed50 msgr2=0x7f542c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 -- 172.21.15.45:0/1422733993 shutdown_connections 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.475+0000 7f5433292700 1 -- 172.21.15.45:0/1422733993 wait complete. 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.476+0000 7f5433292700 1 Processor -- start 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.476+0000 7f5433292700 1 -- start start 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f5433292700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c0d02b0 0x7f542c1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f5433292700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f542c15c8c0 0x7f542c16cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f5433292700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542c15d820 0x7f542c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f5433292700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1495a0 con 0x7f542c15c8c0 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f5433292700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c16d510 con 0x7f542c15d820 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f5433292700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c148990 con 0x7f542c0d02b0 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f543082d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f542c15c8c0 0x7f542c16cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f543082d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f542c15c8c0 0x7f542c16cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54934/0 (socket says 172.21.15.45:54934) 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f543082d700 1 -- 172.21.15.45:0/3484373674 learned_addr learned my addr 172.21.15.45:0/3484373674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f543102e700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c0d02b0 0x7f542c1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.477+0000 7f543182f700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542c15d820 0x7f542c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f543082d700 1 -- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c0d02b0 msgr2=0x7f542c1483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f543082d700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c0d02b0 0x7f542c1483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f543082d700 1 -- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542c15d820 msgr2=0x7f542c0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f543082d700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542c15d820 0x7f542c0b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f543082d700 1 -- 172.21.15.45:0/3484373674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5428009420 con 0x7f542c15c8c0 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f543102e700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c0d02b0 0x7f542c1483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f543082d700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f542c15c8c0 0x7f542c16cfd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f54200011e0 tx=0x7f5420007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f541d7fa700 1 -- 172.21.15.45:0/3484373674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420005b30 con 0x7f542c15c8c0 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f541d7fa700 1 -- 172.21.15.45:0/3484373674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5420005c90 con 0x7f542c15c8c0 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5424009a20 con 0x7f542c15c8c0 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f541d7fa700 1 -- 172.21.15.45:0/3484373674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420007320 con 0x7f542c15c8c0 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.478+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c148f50 con 0x7f542c15c8c0 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.479+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c088730 con 0x7f542c15c8c0 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.479+0000 7f541d7fa700 1 -- 172.21.15.45:0/3484373674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5420007690 con 0x7f542c15c8c0 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.480+0000 7f541d7fa700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5410066f30 0x7f54100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.480+0000 7f541d7fa700 1 -- 172.21.15.45:0/3484373674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5420083d20 con 0x7f542c15c8c0 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.480+0000 7f543102e700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5410066f30 0x7f54100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.481+0000 7f543102e700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5410066f30 0x7f54100693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f54280093f0 tx=0x7f5428008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.485+0000 7f541d7fa700 1 -- 172.21.15.45:0/3484373674 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5420050ca0 con 0x7f542c15c8c0 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.688+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f542c1496e0 con 0x7f542c15c8c0 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.688+0000 7f541d7fa700 1 -- 172.21.15.45:0/3484373674 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5420017020 con 0x7f542c15c8c0 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.690+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5410066f30 msgr2=0x7f54100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.690+0000 7f5433292700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5410066f30 0x7f54100693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f54280093f0 tx=0x7f5428008040 comp rx=0 tx=0).stop 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f542c15c8c0 msgr2=0x7f542c16cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f542c15c8c0 0x7f542c16cfd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f54200011e0 tx=0x7f5420007df0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 shutdown_connections 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5410066f30 0x7f54100693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542c15d820 0x7f542c0b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542c0d02b0 0x7f542c1483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 --2- 172.21.15.45:0/3484373674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f542c15c8c0 0x7f542c16cfd0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 >> 172.21.15.45:0/3484373674 conn(0x7f542c06ed50 msgr2=0x7f542c15a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 shutdown_connections 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.691+0000 7f5433292700 1 -- 172.21.15.45:0/3484373674 wait complete. 2023-10-06T05:21:34.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.886+0000 7efef92cf700 1 Processor -- start 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.886+0000 7efef92cf700 1 -- start start 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.886+0000 7efef92cf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efef40d02b0 0x7efef40d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.886+0000 7efef92cf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef40cd740 con 0x7efef415d820 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.886+0000 7efef92cf700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef40ccb30 con 0x7efef415c8c0 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.886+0000 7efef92cf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef40cce10 con 0x7efef40d02b0 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efeea7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efef415d820 0x7efef415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45380/0 (socket says 172.21.15.45:45380) 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efeea7fc700 1 -- 172.21.15.45:0/2459780401 learned_addr learned my addr 172.21.15.45:0/2459780401 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef2ffd700 1 --2- 172.21.15.45:0/2459780401 >> v2:172.21.15.144:3300/0 conn(0x7efef40d02b0 0x7efef40d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef27fc700 1 -- 172.21.15.45:0/2459780401 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942166052 0 0) 0x7efef40cd740 con 0x7efef415d820 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef27fc700 1 -- 172.21.15.45:0/2459780401 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee0003570 con 0x7efef415d820 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef2ffd700 1 -- 172.21.15.45:0/2459780401 >> v1:172.21.15.144:6789/0 conn(0x7efef415c8c0 legacy=0x7efef40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef2ffd700 1 -- 172.21.15.45:0/2459780401 >> v1:172.21.15.45:6789/0 conn(0x7efef415d820 legacy=0x7efef415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef27fc700 1 -- 172.21.15.45:0/2459780401 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492010718 0 0) 0x7efef40ccb30 con 0x7efef415c8c0 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef2ffd700 1 -- 172.21.15.45:0/2459780401 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40cd0f0 con 0x7efef40d02b0 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.887+0000 7efef2ffd700 1 --2- 172.21.15.45:0/2459780401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40d02b0 0x7efef40d0680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efee40098e0 tx=0x7efee40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=689f8a4861e2c605 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef27fc700 1 -- 172.21.15.45:0/2459780401 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400b640 con 0x7efef40d02b0 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef27fc700 1 -- 172.21.15.45:0/2459780401 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400b7a0 con 0x7efef40d02b0 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef27fc700 1 -- 172.21.15.45:0/2459780401 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400bb40 con 0x7efef40d02b0 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef92cf700 1 -- 172.21.15.45:0/2459780401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40d02b0 msgr2=0x7efef40d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef92cf700 1 --2- 172.21.15.45:0/2459780401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40d02b0 0x7efef40d0680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efee40098e0 tx=0x7efee40091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef92cf700 1 -- 172.21.15.45:0/2459780401 shutdown_connections 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef92cf700 1 --2- 172.21.15.45:0/2459780401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40d02b0 0x7efef40d0680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef92cf700 1 -- 172.21.15.45:0/2459780401 >> 172.21.15.45:0/2459780401 conn(0x7efef406ed50 msgr2=0x7efef406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef92cf700 1 -- 172.21.15.45:0/2459780401 shutdown_connections 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.888+0000 7efef92cf700 1 -- 172.21.15.45:0/2459780401 wait complete. 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.890+0000 7efef92cf700 1 Processor -- start 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.890+0000 7efef92cf700 1 -- start start 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.890+0000 7efef92cf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef40d02b0 0x7efef4157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.890+0000 7efef92cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef415c8c0 0x7efef40afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efef92cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef415d820 0x7efef40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efef92cf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef416eee0 con 0x7efef415c8c0 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efef92cf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40ca210 con 0x7efef415d820 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efef92cf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef40af970 con 0x7efef40d02b0 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef415c8c0 0x7efef40afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef415c8c0 0x7efef40afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54938/0 (socket says 172.21.15.45:54938) 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 -- 172.21.15.45:0/723590061 learned_addr learned my addr 172.21.15.45:0/723590061 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efef2ffd700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef40d02b0 0x7efef4157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efef37fe700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef415d820 0x7efef40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 -- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef40d02b0 msgr2=0x7efef4157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef40d02b0 0x7efef4157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 -- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef415d820 msgr2=0x7efef40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef415d820 0x7efef40b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 -- 172.21.15.45:0/723590061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4009420 con 0x7efef415c8c0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeea7fc700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef415c8c0 0x7efef40afe00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efedc0011e0 tx=0x7efedc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeeb7fe700 1 -- 172.21.15.45:0/723590061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc005b30 con 0x7efef415c8c0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeeb7fe700 1 -- 172.21.15.45:0/723590061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc005c90 con 0x7efef415c8c0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef40af340 con 0x7efef415c8c0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.891+0000 7efeeb7fe700 1 -- 172.21.15.45:0/723590061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc007320 con 0x7efef415c8c0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.892+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef40af600 con 0x7efef415c8c0 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.893+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef4088730 con 0x7efef415c8c0 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.893+0000 7efeeb7fe700 1 -- 172.21.15.45:0/723590061 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efedc007690 con 0x7efef415c8c0 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.893+0000 7efeeb7fe700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efed4066f30 0x7efed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.894+0000 7efeeb7fe700 1 -- 172.21.15.45:0/723590061 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7efedc083c70 con 0x7efef415c8c0 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.894+0000 7efef2ffd700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efed4066f30 0x7efed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.894+0000 7efef2ffd700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efed4066f30 0x7efed40693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efee40093f0 tx=0x7efee4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:34.898+0000 7efeeb7fe700 1 -- 172.21.15.45:0/723590061 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efedc050b70 con 0x7efef415c8c0 2023-10-06T05:21:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.100+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efef40afab0 con 0x7efef415c8c0 2023-10-06T05:21:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.101+0000 7efeeb7fe700 1 -- 172.21.15.45:0/723590061 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7efedc017020 con 0x7efef415c8c0 2023-10-06T05:21:35.103 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:21:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efed4066f30 msgr2=0x7efed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efed4066f30 0x7efed40693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efee40093f0 tx=0x7efee4017000 comp rx=0 tx=0).stop 2023-10-06T05:21:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef415c8c0 msgr2=0x7efef40afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef415c8c0 0x7efef40afe00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efedc0011e0 tx=0x7efedc007df0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 shutdown_connections 2023-10-06T05:21:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efed4066f30 0x7efed40693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef415d820 0x7efef40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef40d02b0 0x7efef4157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 --2- 172.21.15.45:0/723590061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef415c8c0 0x7efef40afe00 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 >> 172.21.15.45:0/723590061 conn(0x7efef406ed50 msgr2=0x7efef40cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 shutdown_connections 2023-10-06T05:21:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.103+0000 7efef92cf700 1 -- 172.21.15.45:0/723590061 wait complete. 2023-10-06T05:21:35.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:21:35.223 INFO:tasks.ceph.mds.d.smithi144.stderr:2023-10-06T05:21:35.222+0000 7f87c8a3f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.252+0000 7faf89dc0700 1 Processor -- start 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.252+0000 7faf89dc0700 1 -- start start 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.253+0000 7faf89dc0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf84157330 0x7faf840ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.253+0000 7faf89dc0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf840aaae0 0x7faf840c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.253+0000 7faf89dc0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf840c3b50 con 0x7faf840c90b0 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.253+0000 7faf89dc0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf84157700 con 0x7faf840aaae0 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.253+0000 7faf89dc0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf840c98e0 con 0x7faf84157330 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf82ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf840aaae0 0x7faf840c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf83fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faf840c90b0 0x7faf840c3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45386/0 (socket says 172.21.15.45:45386) 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf83fff700 1 -- 172.21.15.45:0/3028475633 learned_addr learned my addr 172.21.15.45:0/3028475633 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf837fe700 1 --2- 172.21.15.45:0/3028475633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf84157330 0x7faf840ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf82ffd700 1 -- 172.21.15.45:0/3028475633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf84157330 msgr2=0x7faf840ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf82ffd700 1 --2- 172.21.15.45:0/3028475633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf84157330 0x7faf840ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf82ffd700 1 -- 172.21.15.45:0/3028475633 >> v1:172.21.15.45:6789/0 conn(0x7faf840c90b0 legacy=0x7faf840c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf82ffd700 1 -- 172.21.15.45:0/3028475633 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf840c9bc0 con 0x7faf840aaae0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf827fc700 1 -- 172.21.15.45:0/3028475633 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763717706 0 0) 0x7faf840c3b50 con 0x7faf840c90b0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf837fe700 1 --2- 172.21.15.45:0/3028475633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf84157330 0x7faf840ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf82ffd700 1 --2- 172.21.15.45:0/3028475633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf840aaae0 0x7faf840c8b70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7faf780098e0 tx=0x7faf780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8dbdb07d0ad992 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf827fc700 1 -- 172.21.15.45:0/3028475633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78009e80 con 0x7faf840aaae0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf827fc700 1 -- 172.21.15.45:0/3028475633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7800b860 con 0x7faf840aaae0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.254+0000 7faf827fc700 1 -- 172.21.15.45:0/3028475633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7800bbe0 con 0x7faf840aaae0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 -- 172.21.15.45:0/3028475633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf840aaae0 msgr2=0x7faf840c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 --2- 172.21.15.45:0/3028475633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf840aaae0 0x7faf840c8b70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7faf780098e0 tx=0x7faf780091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 -- 172.21.15.45:0/3028475633 shutdown_connections 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 --2- 172.21.15.45:0/3028475633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf84157330 0x7faf840ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 --2- 172.21.15.45:0/3028475633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf840aaae0 0x7faf840c8b70 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 -- 172.21.15.45:0/3028475633 >> 172.21.15.45:0/3028475633 conn(0x7faf84067e00 msgr2=0x7faf840cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 -- 172.21.15.45:0/3028475633 shutdown_connections 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.255+0000 7faf89dc0700 1 -- 172.21.15.45:0/3028475633 wait complete. 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.256+0000 7faf89dc0700 1 Processor -- start 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.256+0000 7faf89dc0700 1 -- start start 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf89dc0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf840aaae0 0x7faf84151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf89dc0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf840c90b0 0x7faf840c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf89dc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf84157330 0x7faf840c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf89dc0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf840c6a00 con 0x7faf84157330 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf89dc0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf840b1380 con 0x7faf840aaae0 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf89dc0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf840c5c20 con 0x7faf840c90b0 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf83fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf84157330 0x7faf840c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf83fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf84157330 0x7faf840c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54964/0 (socket says 172.21.15.45:54964) 2023-10-06T05:21:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf83fff700 1 -- 172.21.15.45:0/2980556180 learned_addr learned my addr 172.21.15.45:0/2980556180 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf837fe700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf840aaae0 0x7faf84151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.257+0000 7faf82ffd700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf840c90b0 0x7faf840c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf83fff700 1 -- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf840c90b0 msgr2=0x7faf840c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf83fff700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf840c90b0 0x7faf840c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf83fff700 1 -- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf840aaae0 msgr2=0x7faf84151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf83fff700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf840aaae0 0x7faf84151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf83fff700 1 -- 172.21.15.45:0/2980556180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78009420 con 0x7faf84157330 2023-10-06T05:21:35.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf837fe700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf840aaae0 0x7faf84151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:35.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf83fff700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf84157330 0x7faf840c5580 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00cc90 tx=0x7faf7c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf6b7fe700 1 -- 172.21.15.45:0/2980556180 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00cea0 con 0x7faf84157330 2023-10-06T05:21:35.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf6b7fe700 1 -- 172.21.15.45:0/2980556180 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c00ab70 con 0x7faf84157330 2023-10-06T05:21:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf840c5f00 con 0x7faf84157330 2023-10-06T05:21:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf6b7fe700 1 -- 172.21.15.45:0/2980556180 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0070f0 con 0x7faf84157330 2023-10-06T05:21:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.258+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf840b1010 con 0x7faf84157330 2023-10-06T05:21:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.259+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf840b1510 con 0x7faf84157330 2023-10-06T05:21:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.259+0000 7faf6b7fe700 1 -- 172.21.15.45:0/2980556180 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf7c0074a0 con 0x7faf84157330 2023-10-06T05:21:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.260+0000 7faf6b7fe700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faf70066f30 0x7faf700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.260+0000 7faf6b7fe700 1 -- 172.21.15.45:0/2980556180 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7faf7c084d00 con 0x7faf84157330 2023-10-06T05:21:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.260+0000 7faf837fe700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faf70066f30 0x7faf700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.261+0000 7faf837fe700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faf70066f30 0x7faf700693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf7400ab10 tx=0x7faf74008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.264+0000 7faf6b7fe700 1 -- 172.21.15.45:0/2980556180 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf7c051d00 con 0x7faf84157330 2023-10-06T05:21:35.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.464+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf8406e350 con 0x7faf84157330 2023-10-06T05:21:35.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.465+0000 7faf6b7fe700 1 -- 172.21.15.45:0/2980556180 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf7c055320 con 0x7faf84157330 2023-10-06T05:21:35.467 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:21:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faf70066f30 msgr2=0x7faf700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faf70066f30 0x7faf700693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf7400ab10 tx=0x7faf74008040 comp rx=0 tx=0).stop 2023-10-06T05:21:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf84157330 msgr2=0x7faf840c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf84157330 0x7faf840c5580 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00cc90 tx=0x7faf7c001a80 comp rx=0 tx=0).stop 2023-10-06T05:21:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 shutdown_connections 2023-10-06T05:21:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faf70066f30 0x7faf700693e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf84157330 0x7faf840c5580 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf840aaae0 0x7faf84151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 --2- 172.21.15.45:0/2980556180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf840c90b0 0x7faf840c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 >> 172.21.15.45:0/2980556180 conn(0x7faf84067e00 msgr2=0x7faf84153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 shutdown_connections 2023-10-06T05:21:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.467+0000 7faf89dc0700 1 -- 172.21.15.45:0/2980556180 wait complete. 2023-10-06T05:21:35.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:21:35.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.616+0000 7f1da9197700 1 Processor -- start 2023-10-06T05:21:35.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.616+0000 7f1da9197700 1 -- start start 2023-10-06T05:21:35.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da9197700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da9197700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1da406cd60 0x7f1da40acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da9197700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1da40ad4e0 0x7f1da40c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da9197700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da40c9940 con 0x7f1da40ad4e0 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da9197700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da406d230 con 0x7f1da406cd60 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da9197700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da406d510 con 0x7f1da41406e0 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da359e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1da40ad4e0 0x7f1da40c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da359e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1da40ad4e0 0x7f1da40c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54970/0 (socket says 172.21.15.45:54970) 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da359e700 1 -- 172.21.15.45:0/1048883952 learned_addr learned my addr 172.21.15.45:0/1048883952 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da2d9d700 1 --2- 172.21.15.45:0/1048883952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1d9a59c700 1 --2- 172.21.15.45:0/1048883952 >> v2:172.21.15.144:3300/0 conn(0x7f1da406cd60 0x7f1da40acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da359e700 1 -- 172.21.15.45:0/1048883952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 msgr2=0x7f1da406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da359e700 1 --2- 172.21.15.45:0/1048883952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da359e700 1 -- 172.21.15.45:0/1048883952 >> v2:172.21.15.144:3300/0 conn(0x7f1da406cd60 msgr2=0x7f1da40acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:35.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.617+0000 7f1da359e700 1 --2- 172.21.15.45:0/1048883952 >> v2:172.21.15.144:3300/0 conn(0x7f1da406cd60 0x7f1da40acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da359e700 1 -- 172.21.15.45:0/1048883952 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da406d7f0 con 0x7f1da40ad4e0 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da359e700 1 --2- 172.21.15.45:0/1048883952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da40ad4e0 0x7f1da40c9dd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1da40ca910 tx=0x7f1d9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b02be18b613222e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da259c700 1 -- 172.21.15.45:0/1048883952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00e070 con 0x7f1da40ad4e0 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da259c700 1 -- 172.21.15.45:0/1048883952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00ba50 con 0x7f1da40ad4e0 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da259c700 1 -- 172.21.15.45:0/1048883952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0092e0 con 0x7f1da40ad4e0 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da9197700 1 -- 172.21.15.45:0/1048883952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da40ad4e0 msgr2=0x7f1da40c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da9197700 1 --2- 172.21.15.45:0/1048883952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da40ad4e0 0x7f1da40c9dd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1da40ca910 tx=0x7f1d9c009840 comp rx=0 tx=0).stop 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da9197700 1 -- 172.21.15.45:0/1048883952 shutdown_connections 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da9197700 1 --2- 172.21.15.45:0/1048883952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da40ad4e0 0x7f1da40c9dd0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da9197700 1 --2- 172.21.15.45:0/1048883952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da9197700 1 --2- 172.21.15.45:0/1048883952 >> v2:172.21.15.144:3300/0 conn(0x7f1da406cd60 0x7f1da40acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.618+0000 7f1da9197700 1 -- 172.21.15.45:0/1048883952 >> 172.21.15.45:0/1048883952 conn(0x7f1da40691c0 msgr2=0x7f1da40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.619+0000 7f1da9197700 1 -- 172.21.15.45:0/1048883952 shutdown_connections 2023-10-06T05:21:35.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.619+0000 7f1da9197700 1 -- 172.21.15.45:0/1048883952 wait complete. 2023-10-06T05:21:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.620+0000 7f1da9197700 1 Processor -- start 2023-10-06T05:21:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.620+0000 7f1da9197700 1 -- start start 2023-10-06T05:21:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da9197700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da406cd60 0x7f1da413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da9197700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da40ad4e0 0x7f1da4067620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da9197700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da4067b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da9197700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40cb7c0 con 0x7f1da406cd60 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da9197700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40cabb0 con 0x7f1da40ad4e0 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da9197700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40cae90 con 0x7f1da41406e0 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da406cd60 0x7f1da413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da406cd60 0x7f1da413eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54978/0 (socket says 172.21.15.45:54978) 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 -- 172.21.15.45:0/396213362 learned_addr learned my addr 172.21.15.45:0/396213362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1d9a59c700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da40ad4e0 0x7f1da4067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da359e700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da4067b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 -- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 msgr2=0x7f1da4067b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da4067b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 -- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da40ad4e0 msgr2=0x7f1da4067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da40ad4e0 0x7f1da4067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 -- 172.21.15.45:0/396213362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c009ae0 con 0x7f1da406cd60 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.621+0000 7f1da2d9d700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da406cd60 0x7f1da413eb20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400d890 tx=0x7f1d94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.622+0000 7f1d9b7fe700 1 -- 172.21.15.45:0/396213362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940125f0 con 0x7f1da406cd60 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.622+0000 7f1d9b7fe700 1 -- 172.21.15.45:0/396213362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94012770 con 0x7f1da406cd60 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.622+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da40cb170 con 0x7f1da406cd60 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.622+0000 7f1d9b7fe700 1 -- 172.21.15.45:0/396213362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94012b30 con 0x7f1da406cd60 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.622+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da40cbc50 con 0x7f1da406cd60 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.623+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da406e2d0 con 0x7f1da406cd60 2023-10-06T05:21:35.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.623+0000 7f1d9b7fe700 1 -- 172.21.15.45:0/396213362 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d9401e460 con 0x7f1da406cd60 2023-10-06T05:21:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.623+0000 7f1d9b7fe700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d84066f30 0x7f1d840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.623+0000 7f1d9b7fe700 1 -- 172.21.15.45:0/396213362 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f1d94084df0 con 0x7f1da406cd60 2023-10-06T05:21:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.624+0000 7f1d9a59c700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d84066f30 0x7f1d840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.624+0000 7f1d9a59c700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d84066f30 0x7f1d840693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1da40b21a0 tx=0x7f1d8c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.628+0000 7f1d9b7fe700 1 -- 172.21.15.45:0/396213362 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d94051d70 con 0x7f1da406cd60 2023-10-06T05:21:35.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.830+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1da40cbaa0 con 0x7f1da406cd60 2023-10-06T05:21:35.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.830+0000 7f1d9b7fe700 1 -- 172.21.15.45:0/396213362 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d94055390 con 0x7f1da406cd60 2023-10-06T05:21:35.832 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:21:35.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d84066f30 msgr2=0x7f1d840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d84066f30 0x7f1d840693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1da40b21a0 tx=0x7f1d8c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da406cd60 msgr2=0x7f1da413eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da406cd60 0x7f1da413eb20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400d890 tx=0x7f1d94008040 comp rx=0 tx=0).stop 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 shutdown_connections 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d84066f30 0x7f1d840693e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1da41406e0 0x7f1da4067b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da406cd60 0x7f1da413eb20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 --2- 172.21.15.45:0/396213362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da40ad4e0 0x7f1da4067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.832+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 >> 172.21.15.45:0/396213362 conn(0x7f1da40691c0 msgr2=0x7f1da40c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.833+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 shutdown_connections 2023-10-06T05:21:35.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.833+0000 7f1da9197700 1 -- 172.21.15.45:0/396213362 wait complete. 2023-10-06T05:21:35.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:35.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.980+0000 7f4d61d33700 1 Processor -- start 2023-10-06T05:21:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.981+0000 7f4d61d33700 1 -- start start 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.981+0000 7f4d61d33700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d5c159330 0x7f4d5c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.981+0000 7f4d61d33700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d5c0cc390 0x7f4d5c0cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.981+0000 7f4d61d33700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5c0cb530 con 0x7f4d5c158f60 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.981+0000 7f4d61d33700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c069bd0 con 0x7f4d5c0cc390 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.981+0000 7f4d61d33700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c0ccd10 con 0x7f4d5c159330 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4d5c158f60 0x7f4d5c0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45392/0 (socket says 172.21.15.45:45392) 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5b7fe700 1 -- 172.21.15.45:0/2968482201 learned_addr learned my addr 172.21.15.45:0/2968482201 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5bfff700 1 --2- 172.21.15.45:0/2968482201 >> v2:172.21.15.45:3300/0 conn(0x7f4d5c0cc390 0x7f4d5c0cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5affd700 1 --2- 172.21.15.45:0/2968482201 >> v2:172.21.15.144:3300/0 conn(0x7f4d5c159330 0x7f4d5c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5a7fc700 1 -- 172.21.15.45:0/2968482201 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726937753 0 0) 0x7f4d4c001110 con 0x7f4d5c158f60 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5a7fc700 1 -- 172.21.15.45:0/2968482201 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d44003570 con 0x7f4d5c158f60 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5bfff700 1 -- 172.21.15.45:0/2968482201 >> v2:172.21.15.144:3300/0 conn(0x7f4d5c159330 msgr2=0x7f4d5c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5bfff700 1 --2- 172.21.15.45:0/2968482201 >> v2:172.21.15.144:3300/0 conn(0x7f4d5c159330 0x7f4d5c06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5bfff700 1 -- 172.21.15.45:0/2968482201 >> v1:172.21.15.45:6789/0 conn(0x7f4d5c158f60 legacy=0x7f4d5c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5bfff700 1 -- 172.21.15.45:0/2968482201 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c068fc0 con 0x7f4d5c0cc390 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5bfff700 1 --2- 172.21.15.45:0/2968482201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c0cc390 0x7f4d5c0cc7d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0872e0 tx=0x7f4d54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb0396b2e540039 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5a7fc700 1 -- 172.21.15.45:0/2968482201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400e070 con 0x7f4d5c0cc390 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5a7fc700 1 -- 172.21.15.45:0/2968482201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5400bab0 con 0x7f4d5c0cc390 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.982+0000 7f4d5a7fc700 1 -- 172.21.15.45:0/2968482201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400be30 con 0x7f4d5c0cc390 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 -- 172.21.15.45:0/2968482201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c0cc390 msgr2=0x7f4d5c0cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 --2- 172.21.15.45:0/2968482201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c0cc390 0x7f4d5c0cc7d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0872e0 tx=0x7f4d54009780 comp rx=0 tx=0).stop 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 -- 172.21.15.45:0/2968482201 shutdown_connections 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 --2- 172.21.15.45:0/2968482201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c0cc390 0x7f4d5c0cc7d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 --2- 172.21.15.45:0/2968482201 >> v2:172.21.15.144:3300/0 conn(0x7f4d5c159330 0x7f4d5c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 -- 172.21.15.45:0/2968482201 >> 172.21.15.45:0/2968482201 conn(0x7f4d5c0cdf50 msgr2=0x7f4d5c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 -- 172.21.15.45:0/2968482201 shutdown_connections 2023-10-06T05:21:35.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.983+0000 7f4d61d33700 1 -- 172.21.15.45:0/2968482201 wait complete. 2023-10-06T05:21:35.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.984+0000 7f4d61d33700 1 Processor -- start 2023-10-06T05:21:35.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.984+0000 7f4d61d33700 1 -- start start 2023-10-06T05:21:35.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d61d33700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d5c0cc390 0x7f4d5c0a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d61d33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c158f60 0x7f4d5c0a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d61d33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c159330 0x7f4d5c0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d61d33700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1572b0 con 0x7f4d5c159330 2023-10-06T05:21:35.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d61d33700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1566a0 con 0x7f4d5c158f60 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d61d33700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c156980 con 0x7f4d5c0cc390 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d5bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c159330 0x7f4d5c0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d5bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c159330 0x7f4d5c0b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54998/0 (socket says 172.21.15.45:54998) 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.985+0000 7f4d5bfff700 1 -- 172.21.15.45:0/774754474 learned_addr learned my addr 172.21.15.45:0/774754474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5affd700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c158f60 0x7f4d5c0a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5b7fe700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d5c0cc390 0x7f4d5c0a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5bfff700 1 -- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d5c0cc390 msgr2=0x7f4d5c0a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5bfff700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d5c0cc390 0x7f4d5c0a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5bfff700 1 -- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c158f60 msgr2=0x7f4d5c0a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5bfff700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c158f60 0x7f4d5c0a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5bfff700 1 -- 172.21.15.45:0/774754474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009a20 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d5bfff700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c159330 0x7f4d5c0b2a30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0c4f80 tx=0x7f4d54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d437fe700 1 -- 172.21.15.45:0/774754474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54027470 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d437fe700 1 -- 172.21.15.45:0/774754474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d540275d0 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d437fe700 1 -- 172.21.15.45:0/774754474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54027990 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c156c60 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.986+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c147460 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.987+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c157460 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.988+0000 7f4d437fe700 1 -- 172.21.15.45:0/774754474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d54024070 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.988+0000 7f4d437fe700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d48066f30 0x7f4d480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.988+0000 7f4d437fe700 1 -- 172.21.15.45:0/774754474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f4d54087220 con 0x7f4d5c159330 2023-10-06T05:21:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.988+0000 7f4d5b7fe700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d48066f30 0x7f4d480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.989+0000 7f4d5b7fe700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d48066f30 0x7f4d480693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c001270 tx=0x7f4d4c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:35.993+0000 7f4d437fe700 1 -- 172.21.15.45:0/774754474 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d540541a0 con 0x7f4d5c159330 2023-10-06T05:21:36.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.244+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d5c06eb60 con 0x7f4d5c159330 2023-10-06T05:21:36.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.244+0000 7f4d437fe700 1 -- 172.21.15.45:0/774754474 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f4d540577c0 con 0x7f4d5c159330 2023-10-06T05:21:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:36.246 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2705674276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2705674276},{"type":"v1","addr":"172.21.15.45:6834","nonce":2705674276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:30.891138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6514":{"gid":6514,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:36.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d48066f30 msgr2=0x7f4d480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d48066f30 0x7f4d480693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c001270 tx=0x7f4d4c009040 comp rx=0 tx=0).stop 2023-10-06T05:21:36.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c159330 msgr2=0x7f4d5c0b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c159330 0x7f4d5c0b2a30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0c4f80 tx=0x7f4d54019000 comp rx=0 tx=0).stop 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 shutdown_connections 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4d48066f30 0x7f4d480693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d5c159330 0x7f4d5c0b2a30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d5c0cc390 0x7f4d5c0a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 --2- 172.21.15.45:0/774754474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5c158f60 0x7f4d5c0a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 >> 172.21.15.45:0/774754474 conn(0x7f4d5c0cdf50 msgr2=0x7f4d5c0c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 shutdown_connections 2023-10-06T05:21:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.247+0000 7f4d61d33700 1 -- 172.21.15.45:0/774754474 wait complete. 2023-10-06T05:21:36.251 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 78 2023-10-06T05:21:36.262 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.393+0000 7f4e70aa9700 1 Processor -- start 2023-10-06T05:21:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.393+0000 7f4e70aa9700 1 -- start start 2023-10-06T05:21:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e70aa9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4e6c159200 0x7f4e6c0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e70aa9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e6c1595d0 0x7f4e6c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e70aa9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0c7d00 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e70aa9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0c7780 con 0x7f4e6c159200 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e70aa9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0c86d0 con 0x7f4e6c1595d0 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4e6c159200 0x7f4e6c0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4e6c159200 0x7f4e6c0c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43556/0 (socket says 172.21.15.45:43556) 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 -- 172.21.15.45:0/3336191390 learned_addr learned my addr 172.21.15.45:0/3336191390 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e69d9b700 1 --2- 172.21.15.45:0/3336191390 >> v2:172.21.15.144:3300/0 conn(0x7f4e6c1595d0 0x7f4e6c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 -- 172.21.15.45:0/3336191390 >> v2:172.21.15.144:3300/0 conn(0x7f4e6c1595d0 msgr2=0x7f4e6c06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 --2- 172.21.15.45:0/3336191390 >> v2:172.21.15.144:3300/0 conn(0x7f4e6c1595d0 0x7f4e6c06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 -- 172.21.15.45:0/3336191390 >> v1:172.21.15.144:6789/0 conn(0x7f4e6c06b5c0 legacy=0x7f4e6c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 -- 172.21.15.45:0/3336191390 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c06bb40 con 0x7f4e6c159200 2023-10-06T05:21:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e69d9b700 1 --2- 172.21.15.45:0/3336191390 >> v2:172.21.15.144:3300/0 conn(0x7f4e6c1595d0 0x7f4e6c06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.394+0000 7f4e6a59c700 1 --2- 172.21.15.45:0/3336191390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c159200 0x7f4e6c0c8190 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c06e790 tx=0x7f4e5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4937705d8887a53a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e6959a700 1 -- 172.21.15.45:0/3336191390 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00b860 con 0x7f4e6c159200 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e6959a700 1 -- 172.21.15.45:0/3336191390 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c00b9c0 con 0x7f4e6c159200 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e6959a700 1 -- 172.21.15.45:0/3336191390 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00bd60 con 0x7f4e6c159200 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 -- 172.21.15.45:0/3336191390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c159200 msgr2=0x7f4e6c0c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/3336191390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c159200 0x7f4e6c0c8190 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c06e790 tx=0x7f4e5c009250 comp rx=0 tx=0).stop 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 -- 172.21.15.45:0/3336191390 shutdown_connections 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/3336191390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c159200 0x7f4e6c0c8190 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/3336191390 >> v2:172.21.15.144:3300/0 conn(0x7f4e6c1595d0 0x7f4e6c06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 -- 172.21.15.45:0/3336191390 >> 172.21.15.45:0/3336191390 conn(0x7f4e6c06f090 msgr2=0x7f4e6c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 -- 172.21.15.45:0/3336191390 shutdown_connections 2023-10-06T05:21:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.395+0000 7f4e70aa9700 1 -- 172.21.15.45:0/3336191390 wait complete. 2023-10-06T05:21:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 Processor -- start 2023-10-06T05:21:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 -- start start 2023-10-06T05:21:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c06b5c0 0x7f4e6c16d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e6c159200 0x7f4e6c169190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e6c1595d0 0x7f4e6c1696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c169d30 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c16a010 con 0x7f4e6c1595d0 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.397+0000 7f4e70aa9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c16a2f0 con 0x7f4e6c159200 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c06b5c0 0x7f4e6c16d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c06b5c0 0x7f4e6c16d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43564/0 (socket says 172.21.15.45:43564) 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 -- 172.21.15.45:0/1886413609 learned_addr learned my addr 172.21.15.45:0/1886413609 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e69d9b700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e6c159200 0x7f4e6c169190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6ad9d700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e6c1595d0 0x7f4e6c1696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 -- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e6c159200 msgr2=0x7f4e6c169190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e6c159200 0x7f4e6c169190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 -- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e6c1595d0 msgr2=0x7f4e6c1696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e6c1595d0 0x7f4e6c1696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 -- 172.21.15.45:0/1886413609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c0094f0 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6ad9d700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e6c1595d0 0x7f4e6c1696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e6a59c700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c06b5c0 0x7f4e6c16d9a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009290 tx=0x7f4e5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e5a7fc700 1 -- 172.21.15.45:0/1886413609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c011e60 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e5a7fc700 1 -- 172.21.15.45:0/1886413609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c01f070 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c16a5d0 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e5a7fc700 1 -- 172.21.15.45:0/1886413609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c01a870 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.398+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c0c6470 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.399+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c06e2f0 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.400+0000 7f4e5a7fc700 1 -- 172.21.15.45:0/1886413609 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e5c01abe0 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.400+0000 7f4e5a7fc700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e4c066f30 0x7f4e4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.400+0000 7f4e5a7fc700 1 -- 172.21.15.45:0/1886413609 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f4e5c021070 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.400+0000 7f4e69d9b700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e4c066f30 0x7f4e4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.401+0000 7f4e69d9b700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e4c066f30 0x7f4e4c0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4e60006f00 tx=0x7f4e60008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.405+0000 7f4e5a7fc700 1 -- 172.21.15.45:0/1886413609 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e5c0537f0 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.608+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e6c169e70 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.609+0000 7f4e5a7fc700 1 -- 172.21.15.45:0/1886413609 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17299 (secure 0 0 0) 0x7f4e6c169e70 con 0x7f4e6c06b5c0 2023-10-06T05:21:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:36.611 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":80,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:21:27.879512+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:51.554669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:52.645101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/1530850582":"2023-10-07T05:21:27.879438+0000","172.21.15.45:6832/1530850582":"2023-10-07T05:21:27.879438+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e4c066f30 msgr2=0x7f4e4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e4c066f30 0x7f4e4c0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4e60006f00 tx=0x7f4e60008040 comp rx=0 tx=0).stop 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c06b5c0 msgr2=0x7f4e6c16d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c06b5c0 0x7f4e6c16d9a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009290 tx=0x7f4e5c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 shutdown_connections 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e4c066f30 0x7f4e4c0693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e6c1595d0 0x7f4e6c1696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e6c06b5c0 0x7f4e6c16d9a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 --2- 172.21.15.45:0/1886413609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e6c159200 0x7f4e6c169190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 >> 172.21.15.45:0/1886413609 conn(0x7f4e6c06f090 msgr2=0x7f4e6c15a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 shutdown_connections 2023-10-06T05:21:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.611+0000 7f4e70aa9700 1 -- 172.21.15.45:0/1886413609 wait complete. 2023-10-06T05:21:36.626 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:21:36.627 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.756+0000 7ff2f446d700 1 Processor -- start 2023-10-06T05:21:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.756+0000 7ff2f446d700 1 -- start start 2023-10-06T05:21:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.757+0000 7ff2f446d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2ec1539f0 0x7ff2ec15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.757+0000 7ff2f446d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ec0ce690 con 0x7ff2ec153620 2023-10-06T05:21:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.757+0000 7ff2f446d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ec15d510 con 0x7ff2ec0ccfc0 2023-10-06T05:21:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.757+0000 7ff2f446d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ec15a7a0 con 0x7ff2ec1539f0 2023-10-06T05:21:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1a08700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2ec1539f0 0x7ff2ec15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f2209700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff2ec153620 0x7ff2ec0cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60578/0 (socket says 172.21.15.45:60578) 2023-10-06T05:21:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f2209700 1 -- 172.21.15.45:0/2265269162 learned_addr learned my addr 172.21.15.45:0/2265269162 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1a08700 1 -- 172.21.15.45:0/2265269162 >> v1:172.21.15.144:6789/0 conn(0x7ff2ec0ccfc0 legacy=0x7ff2ec0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1a08700 1 -- 172.21.15.45:0/2265269162 >> v1:172.21.15.45:6789/0 conn(0x7ff2ec153620 legacy=0x7ff2ec0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1a08700 1 -- 172.21.15.45:0/2265269162 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec15d7f0 con 0x7ff2ec1539f0 2023-10-06T05:21:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1207700 1 -- 172.21.15.45:0/2265269162 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656606995 0 0) 0x7ff2ec0ce690 con 0x7ff2ec153620 2023-10-06T05:21:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1a08700 1 --2- 172.21.15.45:0/2265269162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec1539f0 0x7ff2ec15a260 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098e0 tx=0x7ff2e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51c65f5e2652213 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1207700 1 -- 172.21.15.45:0/2265269162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0009e80 con 0x7ff2ec1539f0 2023-10-06T05:21:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1207700 1 -- 172.21.15.45:0/2265269162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e000b860 con 0x7ff2ec1539f0 2023-10-06T05:21:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.758+0000 7ff2f1207700 1 -- 172.21.15.45:0/2265269162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000bbe0 con 0x7ff2ec1539f0 2023-10-06T05:21:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.759+0000 7ff2f446d700 1 -- 172.21.15.45:0/2265269162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec1539f0 msgr2=0x7ff2ec15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.759+0000 7ff2f446d700 1 --2- 172.21.15.45:0/2265269162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec1539f0 0x7ff2ec15a260 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098e0 tx=0x7ff2e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.759+0000 7ff2f446d700 1 -- 172.21.15.45:0/2265269162 shutdown_connections 2023-10-06T05:21:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.759+0000 7ff2f446d700 1 --2- 172.21.15.45:0/2265269162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec1539f0 0x7ff2ec15a260 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.759+0000 7ff2f446d700 1 -- 172.21.15.45:0/2265269162 >> 172.21.15.45:0/2265269162 conn(0x7ff2ec06ed50 msgr2=0x7ff2ec06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.759+0000 7ff2f446d700 1 -- 172.21.15.45:0/2265269162 shutdown_connections 2023-10-06T05:21:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.759+0000 7ff2f446d700 1 -- 172.21.15.45:0/2265269162 wait complete. 2023-10-06T05:21:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.760+0000 7ff2f446d700 1 Processor -- start 2023-10-06T05:21:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.760+0000 7ff2f446d700 1 -- start start 2023-10-06T05:21:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f446d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec0ccfc0 0x7ff2ec16db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f446d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec153620 0x7ff2ec0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f446d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ec1546c0 0x7ff2ec0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f446d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec16e0b0 con 0x7ff2ec153620 2023-10-06T05:21:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f446d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec16d120 con 0x7ff2ec0ccfc0 2023-10-06T05:21:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f446d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec0b3430 con 0x7ff2ec1546c0 2023-10-06T05:21:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f1a08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec153620 0x7ff2ec0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f1a08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec153620 0x7ff2ec0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43578/0 (socket says 172.21.15.45:43578) 2023-10-06T05:21:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f1a08700 1 -- 172.21.15.45:0/3746643752 learned_addr learned my addr 172.21.15.45:0/3746643752 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f2209700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec0ccfc0 0x7ff2ec16db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f2a0a700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ec1546c0 0x7ff2ec0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f1a08700 1 -- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ec1546c0 msgr2=0x7ff2ec0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f1a08700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ec1546c0 0x7ff2ec0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f1a08700 1 -- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec0ccfc0 msgr2=0x7ff2ec16db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.761+0000 7ff2f1a08700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec0ccfc0 0x7ff2ec16db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.762+0000 7ff2f1a08700 1 -- 172.21.15.45:0/3746643752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0009420 con 0x7ff2ec153620 2023-10-06T05:21:36.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.762+0000 7ff2f1a08700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec153620 0x7ff2ec0b2ef0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec06f900 tx=0x7ff2e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.762+0000 7ff2de7fc700 1 -- 172.21.15.45:0/3746643752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001f070 con 0x7ff2ec153620 2023-10-06T05:21:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.762+0000 7ff2de7fc700 1 -- 172.21.15.45:0/3746643752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e000b9d0 con 0x7ff2ec153620 2023-10-06T05:21:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.762+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec0b36d0 con 0x7ff2ec153620 2023-10-06T05:21:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.762+0000 7ff2de7fc700 1 -- 172.21.15.45:0/3746643752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0025670 con 0x7ff2ec153620 2023-10-06T05:21:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.762+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec16d370 con 0x7ff2ec153620 2023-10-06T05:21:36.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.763+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec088a60 con 0x7ff2ec153620 2023-10-06T05:21:36.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.763+0000 7ff2de7fc700 1 -- 172.21.15.45:0/3746643752 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2e0023070 con 0x7ff2ec153620 2023-10-06T05:21:36.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.764+0000 7ff2de7fc700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2d0066f30 0x7ff2d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.764+0000 7ff2de7fc700 1 -- 172.21.15.45:0/3746643752 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ff2e0085d20 con 0x7ff2ec153620 2023-10-06T05:21:36.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.764+0000 7ff2f2209700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2d0066f30 0x7ff2d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.765+0000 7ff2f2209700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2d0066f30 0x7ff2d00693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80011e0 tx=0x7ff2e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:36.769+0000 7ff2de7fc700 1 -- 172.21.15.45:0/3746643752 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2e0052ca0 con 0x7ff2ec153620 2023-10-06T05:21:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.018+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2ec15cce0 con 0x7ff2ec153620 2023-10-06T05:21:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.018+0000 7ff2de7fc700 1 -- 172.21.15.45:0/3746643752 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4607 (secure 0 0 0) 0x7ff2e002a020 con 0x7ff2ec153620 2023-10-06T05:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:37.020 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2705674276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2705674276},{"type":"v1","addr":"172.21.15.45:6834","nonce":2705674276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:30.891138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6514":{"gid":6514,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2d0066f30 msgr2=0x7ff2d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2d0066f30 0x7ff2d00693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80011e0 tx=0x7ff2e8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec153620 msgr2=0x7ff2ec0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec153620 0x7ff2ec0b2ef0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec06f900 tx=0x7ff2e0017000 comp rx=0 tx=0).stop 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 shutdown_connections 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2d0066f30 0x7ff2d00693e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ec1546c0 0x7ff2ec0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec0ccfc0 0x7ff2ec16db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 --2- 172.21.15.45:0/3746643752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2ec153620 0x7ff2ec0b2ef0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 >> 172.21.15.45:0/3746643752 conn(0x7ff2ec06ed50 msgr2=0x7ff2ec158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 shutdown_connections 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.020+0000 7ff2f446d700 1 -- 172.21.15.45:0/3746643752 wait complete. 2023-10-06T05:21:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 78 2023-10-06T05:21:37.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:37.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.165+0000 7f9431505700 1 Processor -- start 2023-10-06T05:21:37.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.165+0000 7f9431505700 1 -- start start 2023-10-06T05:21:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f9431505700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f942c0ccfb0 0x7f942c164190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f9431505700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c0cbb10 con 0x7f942c069380 2023-10-06T05:21:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f9431505700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c0caf00 con 0x7f942c068fb0 2023-10-06T05:21:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f9431505700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0cb1e0 con 0x7f942c0ccfb0 2023-10-06T05:21:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f942a7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f942c069380 0x7f942c158d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60584/0 (socket says 172.21.15.45:60584) 2023-10-06T05:21:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f942a7fc700 1 -- 172.21.15.45:0/1302145970 learned_addr learned my addr 172.21.15.45:0/1302145970 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f942b7fe700 1 --2- 172.21.15.45:0/1302145970 >> v2:172.21.15.144:3300/0 conn(0x7f942c0ccfb0 0x7f942c164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648339640 0 0) 0x7f942c0cbb10 con 0x7f942c069380 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.166+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9414003570 con 0x7f942c069380 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112098198 0 0) 0x7f942c0caf00 con 0x7f942c068fb0 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f942c0cbb10 con 0x7f942c068fb0 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3187529684 0 0) 0x7f9414003570 con 0x7f942c069380 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 >> v2:172.21.15.144:3300/0 conn(0x7f942c0ccfb0 msgr2=0x7f942c164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 --2- 172.21.15.45:0/1302145970 >> v2:172.21.15.144:3300/0 conn(0x7f942c0ccfb0 0x7f942c164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 >> v1:172.21.15.144:6789/0 conn(0x7f942c068fb0 legacy=0x7f942c0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0cb4c0 con 0x7f942c069380 2023-10-06T05:21:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f9420001470 con 0x7f942c069380 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 >> v1:172.21.15.45:6789/0 conn(0x7f942c069380 legacy=0x7f942c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9414003bf0 0x7f9414003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9414004810 0x7f9414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94140051c0 0x7f941400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0cb1e0 con 0x7f9414003bf0 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941400c0b0 con 0x7f94140051c0 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941400c270 con 0x7f9414004810 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942affd700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9414003bf0 0x7f9414003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942a7fc700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9414004810 0x7f9414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942b7fe700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94140051c0 0x7f941400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942affd700 1 -- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9414004810 msgr2=0x7f9414004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942affd700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9414004810 0x7f9414004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942affd700 1 -- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94140051c0 msgr2=0x7f941400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942affd700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94140051c0 0x7f941400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942b7fe700 1 -- 172.21.15.45:0/1302145970 reap_dead start 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942affd700 1 -- 172.21.15.45:0/1302145970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9414003570 con 0x7f9414003bf0 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.167+0000 7f942affd700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9414003bf0 0x7f9414003fe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f941c001a10 tx=0x7f941c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aab6160a894df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00c590 con 0x7f9414003bf0 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c00c6f0 con 0x7f9414003bf0 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9429ffb700 1 -- 172.21.15.45:0/1302145970 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00ca90 con 0x7f9414003bf0 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9431505700 1 -- 172.21.15.45:0/1302145970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9414003bf0 msgr2=0x7f9414003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9431505700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9414003bf0 0x7f9414003fe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f941c001a10 tx=0x7f941c007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:37.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9431505700 1 -- 172.21.15.45:0/1302145970 shutdown_connections 2023-10-06T05:21:37.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9431505700 1 --2- 172.21.15.45:0/1302145970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9414003bf0 0x7f9414003fe0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9431505700 1 -- 172.21.15.45:0/1302145970 >> 172.21.15.45:0/1302145970 conn(0x7f942c06ed50 msgr2=0x7f942c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9431505700 1 -- 172.21.15.45:0/1302145970 shutdown_connections 2023-10-06T05:21:37.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.168+0000 7f9431505700 1 -- 172.21.15.45:0/1302145970 wait complete. 2023-10-06T05:21:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 Processor -- start 2023-10-06T05:21:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 -- start start 2023-10-06T05:21:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9414003bf0 0x7f942c16a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c0afff0 0x7f942c0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942c0b0460 0x7f942c0b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c5be0 con 0x7f942c0afff0 2023-10-06T05:21:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c16a5f0 con 0x7f9414003bf0 2023-10-06T05:21:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.170+0000 7f9431505700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c4860 con 0x7f942c0b0460 2023-10-06T05:21:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c0afff0 0x7f942c0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c0afff0 0x7f942c0b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43586/0 (socket says 172.21.15.45:43586) 2023-10-06T05:21:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 -- 172.21.15.45:0/1874182507 learned_addr learned my addr 172.21.15.45:0/1874182507 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942affd700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9414003bf0 0x7f942c16a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942b7fe700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942c0b0460 0x7f942c0b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 -- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942c0b0460 msgr2=0x7f942c0b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942c0b0460 0x7f942c0b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 -- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9414003bf0 msgr2=0x7f942c16a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9414003bf0 0x7f942c16a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 -- 172.21.15.45:0/1874182507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0c4b40 con 0x7f942c0afff0 2023-10-06T05:21:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f942a7fc700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c0afff0 0x7f942c0b2e60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f94200073c0 tx=0x7f9420001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f9412ffd700 1 -- 172.21.15.45:0/1874182507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420018030 con 0x7f942c0afff0 2023-10-06T05:21:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f9412ffd700 1 -- 172.21.15.45:0/1874182507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942000f640 con 0x7f942c0afff0 2023-10-06T05:21:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c0c4e20 con 0x7f942c0afff0 2023-10-06T05:21:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f9412ffd700 1 -- 172.21.15.45:0/1874182507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420001000 con 0x7f942c0afff0 2023-10-06T05:21:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.171+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c0c53e0 con 0x7f942c0afff0 2023-10-06T05:21:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.172+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c0886a0 con 0x7f942c0afff0 2023-10-06T05:21:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.173+0000 7f9412ffd700 1 -- 172.21.15.45:0/1874182507 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f942001b070 con 0x7f942c0afff0 2023-10-06T05:21:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.173+0000 7f9412ffd700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9418066f30 0x7f94180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.173+0000 7f9412ffd700 1 -- 172.21.15.45:0/1874182507 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f9420019070 con 0x7f942c0afff0 2023-10-06T05:21:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.173+0000 7f942affd700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9418066f30 0x7f94180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.174+0000 7f942affd700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9418066f30 0x7f94180693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f941c000c00 tx=0x7f941c015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.178+0000 7f9412ffd700 1 -- 172.21.15.45:0/1874182507 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9420051440 con 0x7f942c0afff0 2023-10-06T05:21:37.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:37.396 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.430+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f942c15d9b0 con 0x7f942c0afff0 2023-10-06T05:21:37.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.430+0000 7f9412ffd700 1 -- 172.21.15.45:0/1874182507 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f9420054a60 con 0x7f942c0afff0 2023-10-06T05:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:37.433 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2705674276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2705674276},{"type":"v1","addr":"172.21.15.45:6834","nonce":2705674276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:53.763976+0000","modified":"2023-10-06T05:21:30.891138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6514":{"gid":6514,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6837/2999699668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2999699668},{"type":"v1","addr":"172.21.15.144:6837","nonce":2999699668}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9418066f30 msgr2=0x7f94180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9418066f30 0x7f94180693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f941c000c00 tx=0x7f941c015000 comp rx=0 tx=0).stop 2023-10-06T05:21:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c0afff0 msgr2=0x7f942c0b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c0afff0 0x7f942c0b2e60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f94200073c0 tx=0x7f9420001e10 comp rx=0 tx=0).stop 2023-10-06T05:21:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 shutdown_connections 2023-10-06T05:21:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9418066f30 0x7f94180693e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942c0b0460 0x7f942c0b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9414003bf0 0x7f942c16a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 --2- 172.21.15.45:0/1874182507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c0afff0 0x7f942c0b2e60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.432+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 >> 172.21.15.45:0/1874182507 conn(0x7f942c06ed50 msgr2=0x7f942c0b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.433+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 shutdown_connections 2023-10-06T05:21:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.433+0000 7f9431505700 1 -- 172.21.15.45:0/1874182507 wait complete. 2023-10-06T05:21:37.437 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 78 2023-10-06T05:21:37.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:37.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.635+0000 7f06581a8700 1 Processor -- start 2023-10-06T05:21:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.635+0000 7f06581a8700 1 -- start start 2023-10-06T05:21:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f06581a8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06500c8ff0 0x7f06500c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f06581a8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06500caad0 0x7f06500caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f06581a8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065015b000 con 0x7f0650156af0 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f06581a8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065015b2e0 con 0x7f06500caad0 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f06581a8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065015b5c0 con 0x7f06500c8ff0 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06500caad0 0x7f06500caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06500caad0 0x7f06500caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43598/0 (socket says 172.21.15.45:43598) 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 -- 172.21.15.45:0/700289054 learned_addr learned my addr 172.21.15.45:0/700289054 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655f44700 1 --2- 172.21.15.45:0/700289054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06500c8ff0 0x7f06500c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 -- 172.21.15.45:0/700289054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06500c8ff0 msgr2=0x7f06500c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 --2- 172.21.15.45:0/700289054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06500c8ff0 0x7f06500c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 -- 172.21.15.45:0/700289054 >> v1:172.21.15.144:6789/0 conn(0x7f0650156af0 legacy=0x7f0650156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 -- 172.21.15.45:0/700289054 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065015b8a0 con 0x7f06500caad0 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.636+0000 7f0655743700 1 --2- 172.21.15.45:0/700289054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500caad0 0x7f06500caf40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f06400099b0 tx=0x7f0640009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10825ac47a211ce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f0654f42700 1 -- 172.21.15.45:0/700289054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064000b860 con 0x7f06500caad0 2023-10-06T05:21:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f0654f42700 1 -- 172.21.15.45:0/700289054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064000b9c0 con 0x7f06500caad0 2023-10-06T05:21:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f0654f42700 1 -- 172.21.15.45:0/700289054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064000bd60 con 0x7f06500caad0 2023-10-06T05:21:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 -- 172.21.15.45:0/700289054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500caad0 msgr2=0x7f06500caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 --2- 172.21.15.45:0/700289054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500caad0 0x7f06500caf40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f06400099b0 tx=0x7f0640009250 comp rx=0 tx=0).stop 2023-10-06T05:21:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 -- 172.21.15.45:0/700289054 shutdown_connections 2023-10-06T05:21:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 --2- 172.21.15.45:0/700289054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06500c8ff0 0x7f06500c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 --2- 172.21.15.45:0/700289054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500caad0 0x7f06500caf40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 -- 172.21.15.45:0/700289054 >> 172.21.15.45:0/700289054 conn(0x7f06500cdf50 msgr2=0x7f06500ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 -- 172.21.15.45:0/700289054 shutdown_connections 2023-10-06T05:21:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.637+0000 7f06581a8700 1 -- 172.21.15.45:0/700289054 wait complete. 2023-10-06T05:21:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 Processor -- start 2023-10-06T05:21:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 -- start start 2023-10-06T05:21:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500c8ff0 0x7f0650140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06500caad0 0x7f06500b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0650156af0 0x7f0650168190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06500b2a70 con 0x7f06500c8ff0 2023-10-06T05:21:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06500b20a0 con 0x7f06500caad0 2023-10-06T05:21:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f06581a8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0650152900 con 0x7f0650156af0 2023-10-06T05:21:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f0655f44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500c8ff0 0x7f0650140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.639+0000 7f0655f44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500c8ff0 0x7f0650140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43608/0 (socket says 172.21.15.45:43608) 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655f44700 1 -- 172.21.15.45:0/1024085829 learned_addr learned my addr 172.21.15.45:0/1024085829 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655743700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06500caad0 0x7f06500b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0656745700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0650156af0 0x7f0650168190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655f44700 1 -- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0650156af0 msgr2=0x7f0650168190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655f44700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0650156af0 0x7f0650168190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655f44700 1 -- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06500caad0 msgr2=0x7f06500b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655f44700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06500caad0 0x7f06500b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655f44700 1 -- 172.21.15.45:0/1024085829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400094f0 con 0x7f06500c8ff0 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0655f44700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500c8ff0 0x7f0650140540 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f064c00d890 tx=0x7f064c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0645ffb700 1 -- 172.21.15.45:0/1024085829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0125f0 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0645ffb700 1 -- 172.21.15.45:0/1024085829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c012750 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06500b1490 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f0645ffb700 1 -- 172.21.15.45:0/1024085829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c012af0 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.640+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06500b1a50 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.642+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06500b2380 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.642+0000 7f0645ffb700 1 -- 172.21.15.45:0/1024085829 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f064c01e460 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.642+0000 7f0645ffb700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0634066f30 0x7f06340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.642+0000 7f0645ffb700 1 -- 172.21.15.45:0/1024085829 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f064c084d90 con 0x7f06500c8ff0 2023-10-06T05:21:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.642+0000 7f0655743700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0634066f30 0x7f06340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.643+0000 7f0655743700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0634066f30 0x7f06340693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f06400099b0 tx=0x7f0640017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.647+0000 7f0645ffb700 1 -- 172.21.15.45:0/1024085829 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f064c051d10 con 0x7f06500c8ff0 2023-10-06T05:21:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.852+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06500cd530 con 0x7f06500c8ff0 2023-10-06T05:21:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.853+0000 7f0645ffb700 1 -- 172.21.15.45:0/1024085829 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17299 (secure 0 0 0) 0x7f064c055330 con 0x7f06500c8ff0 2023-10-06T05:21:37.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":80,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:21:27.879512+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:51.554669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:52.645101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/1530850582":"2023-10-07T05:21:27.879438+0000","172.21.15.45:6832/1530850582":"2023-10-07T05:21:27.879438+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0634066f30 msgr2=0x7f06340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0634066f30 0x7f06340693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f06400099b0 tx=0x7f0640017000 comp rx=0 tx=0).stop 2023-10-06T05:21:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500c8ff0 msgr2=0x7f0650140540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500c8ff0 0x7f0650140540 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f064c00d890 tx=0x7f064c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 shutdown_connections 2023-10-06T05:21:37.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0634066f30 0x7f06340693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0650156af0 0x7f0650168190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06500c8ff0 0x7f0650140540 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 --2- 172.21.15.45:0/1024085829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06500caad0 0x7f06500b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 >> 172.21.15.45:0/1024085829 conn(0x7f06500cdf50 msgr2=0x7f06500aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 shutdown_connections 2023-10-06T05:21:37.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:37.856+0000 7f06581a8700 1 -- 172.21.15.45:0/1024085829 wait complete. 2023-10-06T05:21:37.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:21:38.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.005+0000 7fe0022c6700 1 Processor -- start 2023-10-06T05:21:38.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.005+0000 7fe0022c6700 1 -- start start 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.006+0000 7fe0022c6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdffc153620 0x7fdffc15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.006+0000 7fe0022c6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdffc0ce690 con 0x7fdffc1539f0 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.006+0000 7fe0022c6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdffc15d510 con 0x7fdffc0ccfc0 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.006+0000 7fe0022c6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc15a7a0 con 0x7fdffc153620 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.006+0000 7fdffbfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdffc153620 0x7fdffc15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.006+0000 7fdffb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdffc1539f0 0x7fdffc0cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60588/0 (socket says 172.21.15.45:60588) 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffb7fe700 1 -- 172.21.15.45:0/4021277668 learned_addr learned my addr 172.21.15.45:0/4021277668 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffbfff700 1 -- 172.21.15.45:0/4021277668 >> v1:172.21.15.144:6789/0 conn(0x7fdffc0ccfc0 legacy=0x7fdffc0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffbfff700 1 -- 172.21.15.45:0/4021277668 >> v1:172.21.15.45:6789/0 conn(0x7fdffc1539f0 legacy=0x7fdffc0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffbfff700 1 -- 172.21.15.45:0/4021277668 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc15d7f0 con 0x7fdffc153620 2023-10-06T05:21:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffbfff700 1 --2- 172.21.15.45:0/4021277668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc153620 0x7fdffc15a260 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0098e0 tx=0x7fdfec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2634bfdc0996389 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffaffd700 1 -- 172.21.15.45:0/4021277668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec009e80 con 0x7fdffc153620 2023-10-06T05:21:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffaffd700 1 -- 172.21.15.45:0/4021277668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec00b860 con 0x7fdffc153620 2023-10-06T05:21:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.007+0000 7fdffaffd700 1 -- 172.21.15.45:0/4021277668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00bbe0 con 0x7fdffc153620 2023-10-06T05:21:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.008+0000 7fe0022c6700 1 -- 172.21.15.45:0/4021277668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc153620 msgr2=0x7fdffc15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.008+0000 7fe0022c6700 1 --2- 172.21.15.45:0/4021277668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc153620 0x7fdffc15a260 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0098e0 tx=0x7fdfec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.008+0000 7fe0022c6700 1 -- 172.21.15.45:0/4021277668 shutdown_connections 2023-10-06T05:21:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.008+0000 7fe0022c6700 1 --2- 172.21.15.45:0/4021277668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc153620 0x7fdffc15a260 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.008+0000 7fe0022c6700 1 -- 172.21.15.45:0/4021277668 >> 172.21.15.45:0/4021277668 conn(0x7fdffc06ed50 msgr2=0x7fdffc06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.008+0000 7fe0022c6700 1 -- 172.21.15.45:0/4021277668 shutdown_connections 2023-10-06T05:21:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.008+0000 7fe0022c6700 1 -- 172.21.15.45:0/4021277668 wait complete. 2023-10-06T05:21:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.009+0000 7fe0022c6700 1 Processor -- start 2023-10-06T05:21:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.009+0000 7fe0022c6700 1 -- start start 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe0022c6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdffc0ccfc0 0x7fdffc1402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe0022c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdffc153620 0x7fdffc0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe0022c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b4ff0 0x7fdffc0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe0022c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc16aee0 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe0022c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0b5590 con 0x7fdffc153620 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe0022c6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0b5870 con 0x7fdffc0ccfc0 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b4ff0 0x7fdffc0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b4ff0 0x7fdffc0b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43636/0 (socket says 172.21.15.45:43636) 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 -- 172.21.15.45:0/3112366279 learned_addr learned my addr 172.21.15.45:0/3112366279 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fdffb7fe700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdffc153620 0x7fdffc0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fdffbfff700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdffc0ccfc0 0x7fdffc1402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 -- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdffc0ccfc0 msgr2=0x7fdffc1402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdffc0ccfc0 0x7fdffc1402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 -- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdffc153620 msgr2=0x7fdffc0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdffc153620 0x7fdffc0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.010+0000 7fe000863700 1 -- 172.21.15.45:0/3112366279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec009420 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.011+0000 7fe000863700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b4ff0 0x7fdffc0b0880 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c8e0 tx=0x7fdff4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.011+0000 7fdfe3fff700 1 -- 172.21.15.45:0/3112366279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400ce80 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.011+0000 7fdfe3fff700 1 -- 172.21.15.45:0/3112366279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff400a7e0 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.011+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc0b5b50 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.011+0000 7fdfe3fff700 1 -- 172.21.15.45:0/3112366279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400ab80 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.011+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc1407e0 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.012+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdffc16b370 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.012+0000 7fdfe3fff700 1 -- 172.21.15.45:0/3112366279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdff4012420 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.013+0000 7fdfe3fff700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdfe8066f30 0x7fdfe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.013+0000 7fdfe3fff700 1 -- 172.21.15.45:0/3112366279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fdff4084b10 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.013+0000 7fdffbfff700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdfe8066f30 0x7fdfe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.013+0000 7fdffbfff700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdfe8066f30 0x7fdfe80693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0093f0 tx=0x7fdfec017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.017+0000 7fdfe3fff700 1 -- 172.21.15.45:0/3112366279 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdff4051a90 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.265+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdffc04bb40 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.267 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:21:38.265+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:21:38.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.269+0000 7fdfe3fff700 1 -- 172.21.15.45:0/3112366279 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdff40550b0 con 0x7fdffc0b4ff0 2023-10-06T05:21:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdfe8066f30 msgr2=0x7fdfe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdfe8066f30 0x7fdfe80693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0093f0 tx=0x7fdfec017000 comp rx=0 tx=0).stop 2023-10-06T05:21:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b4ff0 msgr2=0x7fdffc0b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b4ff0 0x7fdffc0b0880 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c8e0 tx=0x7fdff4001710 comp rx=0 tx=0).stop 2023-10-06T05:21:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 shutdown_connections 2023-10-06T05:21:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdfe8066f30 0x7fdfe80693e0 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0093f0 tx=0x7fdfec017000 comp rx=0 tx=0).stop 2023-10-06T05:21:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b4ff0 0x7fdffc0b0880 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdffc0ccfc0 0x7fdffc1402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3112366279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdffc153620 0x7fdffc0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 >> 172.21.15.45:0/3112366279 conn(0x7fdffc06ed50 msgr2=0x7fdffc1529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 shutdown_connections 2023-10-06T05:21:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.272+0000 7fe0022c6700 1 -- 172.21.15.45:0/3112366279 wait complete. 2023-10-06T05:21:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:21:38.277 INFO:tasks.ceph.mds.d.smithi144.stderr: -93> 2023-10-06T05:21:35.222+0000 7f87c8a3f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:38.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:21:38.298 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:21:38.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.422+0000 7ff1ae50a700 1 Processor -- start 2023-10-06T05:21:38.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.422+0000 7ff1ae50a700 1 -- start start 2023-10-06T05:21:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.422+0000 7ff1ae50a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff1a813f900 0x7ff1a80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1ae50a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1a80c9f20 0x7ff1a80affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1ae50a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a80c9630 con 0x7ff1a80c8de0 2023-10-06T05:21:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1ae50a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a80cb550 con 0x7ff1a80c9f20 2023-10-06T05:21:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1ae50a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a80cb830 con 0x7ff1a813f900 2023-10-06T05:21:38.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1acaa7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1a80c9f20 0x7ff1a80affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1a77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff1a80c8de0 0x7ff1a80c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60600/0 (socket says 172.21.15.45:60600) 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1a77fe700 1 -- 172.21.15.45:0/3451475781 learned_addr learned my addr 172.21.15.45:0/3451475781 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1a7fff700 1 --2- 172.21.15.45:0/3451475781 >> v2:172.21.15.144:3300/0 conn(0x7ff1a813f900 0x7ff1a80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1acaa7700 1 -- 172.21.15.45:0/3451475781 >> v2:172.21.15.144:3300/0 conn(0x7ff1a813f900 msgr2=0x7ff1a80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1acaa7700 1 --2- 172.21.15.45:0/3451475781 >> v2:172.21.15.144:3300/0 conn(0x7ff1a813f900 0x7ff1a80cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1acaa7700 1 -- 172.21.15.45:0/3451475781 >> v1:172.21.15.45:6789/0 conn(0x7ff1a80c8de0 legacy=0x7ff1a80c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1acaa7700 1 -- 172.21.15.45:0/3451475781 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80cbb10 con 0x7ff1a80c9f20 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1a6ffd700 1 -- 172.21.15.45:0/3451475781 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286734570 0 0) 0x7ff1a80c9630 con 0x7ff1a80c8de0 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1acaa7700 1 --2- 172.21.15.45:0/3451475781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80c9f20 0x7ff1a80affa0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806aeb0 tx=0x7ff1a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb54b7e5009a248a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1a6ffd700 1 -- 172.21.15.45:0/3451475781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000e070 con 0x7ff1a80c9f20 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.423+0000 7ff1a6ffd700 1 -- 172.21.15.45:0/3451475781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a000bb60 con 0x7ff1a80c9f20 2023-10-06T05:21:38.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1a6ffd700 1 -- 172.21.15.45:0/3451475781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0012430 con 0x7ff1a80c9f20 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 -- 172.21.15.45:0/3451475781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80c9f20 msgr2=0x7ff1a80affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/3451475781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80c9f20 0x7ff1a80affa0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806aeb0 tx=0x7ff1a0009780 comp rx=0 tx=0).stop 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 -- 172.21.15.45:0/3451475781 shutdown_connections 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/3451475781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80c9f20 0x7ff1a80affa0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/3451475781 >> v2:172.21.15.144:3300/0 conn(0x7ff1a813f900 0x7ff1a80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 -- 172.21.15.45:0/3451475781 >> 172.21.15.45:0/3451475781 conn(0x7ff1a80691c0 msgr2=0x7ff1a80cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 -- 172.21.15.45:0/3451475781 shutdown_connections 2023-10-06T05:21:38.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.424+0000 7ff1ae50a700 1 -- 172.21.15.45:0/3451475781 wait complete. 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.425+0000 7ff1ae50a700 1 Processor -- start 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1ae50a700 1 -- start start 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1ae50a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a80c8de0 0x7ff1a8130600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1ae50a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a80c9f20 0x7ff1a80c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1ae50a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a813f900 0x7ff1a80c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1ae50a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8157010 con 0x7ff1a813f900 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1ae50a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8156700 con 0x7ff1a80c9f20 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1ae50a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80b1f00 con 0x7ff1a80c8de0 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.426+0000 7ff1acaa7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a813f900 0x7ff1a80c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a813f900 0x7ff1a80c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43662/0 (socket says 172.21.15.45:43662) 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 -- 172.21.15.45:0/990363177 learned_addr learned my addr 172.21.15.45:0/990363177 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1a77fe700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a80c9f20 0x7ff1a80c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1a7fff700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a80c8de0 0x7ff1a8130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 -- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a80c8de0 msgr2=0x7ff1a8130600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a80c8de0 0x7ff1a8130600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 -- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a80c9f20 msgr2=0x7ff1a80c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a80c9f20 0x7ff1a80c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 -- 172.21.15.45:0/990363177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a0009a20 con 0x7ff1a813f900 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1acaa7700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a813f900 0x7ff1a80c2120 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806e930 tx=0x7ff1a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff18ffff700 1 -- 172.21.15.45:0/990363177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0027470 con 0x7ff1a813f900 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff18ffff700 1 -- 172.21.15.45:0/990363177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a00275d0 con 0x7ff1a813f900 2023-10-06T05:21:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a8156950 con 0x7ff1a813f900 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff18ffff700 1 -- 172.21.15.45:0/990363177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0027990 con 0x7ff1a813f900 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.427+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a8164d20 con 0x7ff1a813f900 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.428+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a8157150 con 0x7ff1a813f900 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.429+0000 7ff18ffff700 1 -- 172.21.15.45:0/990363177 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1a0024070 con 0x7ff1a813f900 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.429+0000 7ff18ffff700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.429+0000 7ff18ffff700 1 -- 172.21.15.45:0/990363177 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6550+0+0 (secure 0 0 0) 0x7ff1a0087130 con 0x7ff1a813f900 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.429+0000 7ff1a7fff700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.430+0000 7ff1a7fff700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff194066f30 0x7ff1940693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff198009d10 tx=0x7ff1980091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.434+0000 7ff18ffff700 1 -- 172.21.15.45:0/990363177 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1a8157150 con 0x7ff1a813f900 2023-10-06T05:21:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:38.685+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1a8157150 con 0x7ff1a813f900 2023-10-06T05:21:39.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.274+0000 7ff18ffff700 1 -- 172.21.15.45:0/990363177 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff1a8157150 con 0x7ff1a813f900 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff194066f30 msgr2=0x7ff1940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff194066f30 0x7ff1940693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff198009d10 tx=0x7ff1980091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a813f900 msgr2=0x7ff1a80c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a813f900 0x7ff1a80c2120 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806e930 tx=0x7ff1a0019000 comp rx=0 tx=0).stop 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 shutdown_connections 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a813f900 0x7ff1a80c2120 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a80c8de0 0x7ff1a8130600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 --2- 172.21.15.45:0/990363177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a80c9f20 0x7ff1a80c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 >> 172.21.15.45:0/990363177 conn(0x7ff1a80691c0 msgr2=0x7ff1a813eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 shutdown_connections 2023-10-06T05:21:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.277+0000 7ff1ae50a700 1 -- 172.21.15.45:0/990363177 wait complete. 2023-10-06T05:21:39.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:21:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.433+0000 7f72952cc700 1 Processor -- start 2023-10-06T05:21:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.433+0000 7f72952cc700 1 -- start start 2023-10-06T05:21:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.433+0000 7f72952cc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f729006a0c0 0x7f72900c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.433+0000 7f72952cc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72900c9320 0x7f7290068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f72952cc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72900c8950 con 0x7f7290069cf0 2023-10-06T05:21:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f72952cc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729015d870 con 0x7f72900c9320 2023-10-06T05:21:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f72952cc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729015db50 con 0x7f729006a0c0 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72900c9320 0x7f7290068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72900c9320 0x7f7290068dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43664/0 (socket says 172.21.15.45:43664) 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 -- 172.21.15.45:0/1451196572 learned_addr learned my addr 172.21.15.45:0/1451196572 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728e7fc700 1 --2- 172.21.15.45:0/1451196572 >> v2:172.21.15.144:3300/0 conn(0x7f729006a0c0 0x7f72900c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 -- 172.21.15.45:0/1451196572 >> v2:172.21.15.144:3300/0 conn(0x7f729006a0c0 msgr2=0x7f72900c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 --2- 172.21.15.45:0/1451196572 >> v2:172.21.15.144:3300/0 conn(0x7f729006a0c0 0x7f72900c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 -- 172.21.15.45:0/1451196572 >> v1:172.21.15.144:6789/0 conn(0x7f7290069cf0 legacy=0x7f729015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 -- 172.21.15.45:0/1451196572 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900c85e0 con 0x7f72900c9320 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728f7fe700 1 --2- 172.21.15.45:0/1451196572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f7290068dd0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f72900b3f70 tx=0x7f7288009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c209fa6135298d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728dffb700 1 -- 172.21.15.45:0/1451196572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800e070 con 0x7f72900c9320 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728dffb700 1 -- 172.21.15.45:0/1451196572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728800ba20 con 0x7f72900c9320 2023-10-06T05:21:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.434+0000 7f728dffb700 1 -- 172.21.15.45:0/1451196572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72880092e0 con 0x7f72900c9320 2023-10-06T05:21:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 -- 172.21.15.45:0/1451196572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 msgr2=0x7f7290068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 --2- 172.21.15.45:0/1451196572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f7290068dd0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f72900b3f70 tx=0x7f7288009840 comp rx=0 tx=0).stop 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 -- 172.21.15.45:0/1451196572 shutdown_connections 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 --2- 172.21.15.45:0/1451196572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f7290068dd0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 --2- 172.21.15.45:0/1451196572 >> v2:172.21.15.144:3300/0 conn(0x7f729006a0c0 0x7f72900c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 -- 172.21.15.45:0/1451196572 >> 172.21.15.45:0/1451196572 conn(0x7f7290067620 msgr2=0x7f7290067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 -- 172.21.15.45:0/1451196572 shutdown_connections 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.435+0000 7f72952cc700 1 -- 172.21.15.45:0/1451196572 wait complete. 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.436+0000 7f72952cc700 1 Processor -- start 2023-10-06T05:21:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f72952cc700 1 -- start start 2023-10-06T05:21:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f72952cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7290069cf0 0x7f729016a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f72952cc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f729006a0c0 0x7f72900b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f72952cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f729016c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f72952cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290158cd0 con 0x7f72900c9320 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f72952cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b0c70 con 0x7f7290069cf0 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f72952cc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b30c0 con 0x7f729006a0c0 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f728f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f729016c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f728f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f729016c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43672/0 (socket says 172.21.15.45:43672) 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.437+0000 7f728f7fe700 1 -- 172.21.15.45:0/598670661 learned_addr learned my addr 172.21.15.45:0/598670661 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728effd700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7290069cf0 0x7f729016a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728e7fc700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f729006a0c0 0x7f72900b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728f7fe700 1 -- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f729006a0c0 msgr2=0x7f72900b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728f7fe700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f729006a0c0 0x7f72900b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728f7fe700 1 -- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7290069cf0 msgr2=0x7f729016a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728f7fe700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7290069cf0 0x7f729016a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728f7fe700 1 -- 172.21.15.45:0/598670661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288009ae0 con 0x7f72900c9320 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f728f7fe700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f729016c6b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7288009880 tx=0x7f7288018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f7286ffd700 1 -- 172.21.15.45:0/598670661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288008be0 con 0x7f72900c9320 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f7286ffd700 1 -- 172.21.15.45:0/598670661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7288008d60 con 0x7f72900c9320 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f7286ffd700 1 -- 172.21.15.45:0/598670661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728801c660 con 0x7f72900c9320 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72900b33a0 con 0x7f72900c9320 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.438+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72900b3960 con 0x7f72900c9320 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.439+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72900b0e20 con 0x7f72900c9320 2023-10-06T05:21:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.440+0000 7f7286ffd700 1 -- 172.21.15.45:0/598670661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7288003040 con 0x7f72900c9320 2023-10-06T05:21:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.440+0000 7f7286ffd700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7270066f30 0x7f72700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.440+0000 7f7286ffd700 1 -- 172.21.15.45:0/598670661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f72880860e0 con 0x7f72900c9320 2023-10-06T05:21:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.440+0000 7f728effd700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7270066f30 0x7f72700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.441+0000 7f728effd700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7270066f30 0x7f72700693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7280003880 tx=0x7f7280009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.445+0000 7f7286ffd700 1 -- 172.21.15.45:0/598670661 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72900b0e20 con 0x7f72900c9320 2023-10-06T05:21:39.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:39.647+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72900b0e20 con 0x7f72900c9320 2023-10-06T05:21:40.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.276+0000 7f7286ffd700 1 -- 172.21.15.45:0/598670661 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) v1 ==== 181+0+0 (secure 0 0 0) 0x7f72900b0e20 con 0x7f72900c9320 2023-10-06T05:21:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7270066f30 msgr2=0x7f72700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7270066f30 0x7f72700693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7280003880 tx=0x7f7280009040 comp rx=0 tx=0).stop 2023-10-06T05:21:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 msgr2=0x7f729016c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f729016c6b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7288009880 tx=0x7f7288018000 comp rx=0 tx=0).stop 2023-10-06T05:21:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 shutdown_connections 2023-10-06T05:21:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7270066f30 0x7f72700693e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72900c9320 0x7f729016c6b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7290069cf0 0x7f729016a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 --2- 172.21.15.45:0/598670661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f729006a0c0 0x7f72900b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 >> 172.21.15.45:0/598670661 conn(0x7f7290067620 msgr2=0x7f72900ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 shutdown_connections 2023-10-06T05:21:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.279+0000 7f72952cc700 1 -- 172.21.15.45:0/598670661 wait complete. 2023-10-06T05:21:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:21:40.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:21:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.435+0000 7f5812c21700 1 Processor -- start 2023-10-06T05:21:40.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.435+0000 7f5812c21700 1 -- start start 2023-10-06T05:21:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f5812c21700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c157150 0x7f580c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f5812c21700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f580c06f890 0x7f580c0c0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f5812c21700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c068680 con 0x7f580c0c0550 2023-10-06T05:21:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f5812c21700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c0ae770 con 0x7f580c06f890 2023-10-06T05:21:40.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f5812c21700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0ae9f0 con 0x7f580c157150 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f580bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f580c06f890 0x7f580c0c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f580bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f580c06f890 0x7f580c0c0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43682/0 (socket says 172.21.15.45:43682) 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.436+0000 7f580bfff700 1 -- 172.21.15.45:0/3941608513 learned_addr learned my addr 172.21.15.45:0/3941608513 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580bfff700 1 -- 172.21.15.45:0/3941608513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c157150 msgr2=0x7f580c068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f58109bd700 1 --2- 172.21.15.45:0/3941608513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c157150 0x7f580c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580bfff700 1 --2- 172.21.15.45:0/3941608513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c157150 0x7f580c068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580bfff700 1 -- 172.21.15.45:0/3941608513 >> v1:172.21.15.144:6789/0 conn(0x7f580c0c0550 legacy=0x7f580c06e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580bfff700 1 -- 172.21.15.45:0/3941608513 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0aecd0 con 0x7f580c06f890 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580bfff700 1 --2- 172.21.15.45:0/3941608513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06f890 0x7f580c0c0010 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f58000099b0 tx=0x7f5800009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ca00eec0df9d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580b7fe700 1 -- 172.21.15.45:0/3941608513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000b820 con 0x7f580c06f890 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580b7fe700 1 -- 172.21.15.45:0/3941608513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580000b980 con 0x7f580c06f890 2023-10-06T05:21:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.437+0000 7f580b7fe700 1 -- 172.21.15.45:0/3941608513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000bd20 con 0x7f580c06f890 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 -- 172.21.15.45:0/3941608513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06f890 msgr2=0x7f580c0c0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 --2- 172.21.15.45:0/3941608513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06f890 0x7f580c0c0010 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f58000099b0 tx=0x7f5800009250 comp rx=0 tx=0).stop 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 -- 172.21.15.45:0/3941608513 shutdown_connections 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 --2- 172.21.15.45:0/3941608513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c157150 0x7f580c068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 --2- 172.21.15.45:0/3941608513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06f890 0x7f580c0c0010 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 -- 172.21.15.45:0/3941608513 >> 172.21.15.45:0/3941608513 conn(0x7f580c0cd060 msgr2=0x7f580c0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 -- 172.21.15.45:0/3941608513 shutdown_connections 2023-10-06T05:21:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.438+0000 7f5812c21700 1 -- 172.21.15.45:0/3941608513 wait complete. 2023-10-06T05:21:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.439+0000 7f5812c21700 1 Processor -- start 2023-10-06T05:21:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.439+0000 7f5812c21700 1 -- start start 2023-10-06T05:21:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f5812c21700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f580c06f890 0x7f580c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f5812c21700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c0c0550 0x7f580c0ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f5812c21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c157150 0x7f580c169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f5812c21700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c169530 con 0x7f580c157150 2023-10-06T05:21:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f5812c21700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0c6f10 con 0x7f580c06f890 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f5812c21700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0c71f0 con 0x7f580c0c0550 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c157150 0x7f580c169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c157150 0x7f580c169960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43686/0 (socket says 172.21.15.45:43686) 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 -- 172.21.15.45:0/306689788 learned_addr learned my addr 172.21.15.45:0/306689788 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58109bd700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f580c06f890 0x7f580c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f580bfff700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c0c0550 0x7f580c0ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 -- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c0c0550 msgr2=0x7f580c0ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c0c0550 0x7f580c0ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 -- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f580c06f890 msgr2=0x7f580c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f580c06f890 0x7f580c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.440+0000 7f58111be700 1 -- 172.21.15.45:0/306689788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000094f0 con 0x7f580c157150 2023-10-06T05:21:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.441+0000 7f58111be700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c157150 0x7f580c169960 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f580400c930 tx=0x7f58040026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.441+0000 7f5808ff9700 1 -- 172.21.15.45:0/306689788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400ce70 con 0x7f580c157150 2023-10-06T05:21:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.441+0000 7f5808ff9700 1 -- 172.21.15.45:0/306689788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580400a7e0 con 0x7f580c157150 2023-10-06T05:21:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.441+0000 7f5808ff9700 1 -- 172.21.15.45:0/306689788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400abc0 con 0x7f580c157150 2023-10-06T05:21:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.441+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c0c74d0 con 0x7f580c157150 2023-10-06T05:21:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.441+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c0c6b30 con 0x7f580c157150 2023-10-06T05:21:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.442+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c169680 con 0x7f580c157150 2023-10-06T05:21:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.442+0000 7f5808ff9700 1 -- 172.21.15.45:0/306689788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5804012420 con 0x7f580c157150 2023-10-06T05:21:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.443+0000 7f5808ff9700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57f8066f30 0x7f57f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.443+0000 7f5808ff9700 1 -- 172.21.15.45:0/306689788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f5804084450 con 0x7f580c157150 2023-10-06T05:21:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.443+0000 7f58109bd700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57f8066f30 0x7f57f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.443+0000 7f58109bd700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57f8066f30 0x7f57f80693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00ab30 tx=0x7f57fc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.447+0000 7f5808ff9700 1 -- 172.21.15.45:0/306689788 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58040514b0 con 0x7f580c157150 2023-10-06T05:21:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:40.646+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f580c04c380 con 0x7f580c157150 2023-10-06T05:21:41.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.281+0000 7f5808ff9700 1 -- 172.21.15.45:0/306689788 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5804054ad0 con 0x7f580c157150 2023-10-06T05:21:41.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.284+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57f8066f30 msgr2=0x7f57f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.284+0000 7f5812c21700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57f8066f30 0x7f57f80693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00ab30 tx=0x7f57fc008040 comp rx=0 tx=0).stop 2023-10-06T05:21:41.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c157150 msgr2=0x7f580c169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c157150 0x7f580c169960 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f580400c930 tx=0x7f58040026e0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 shutdown_connections 2023-10-06T05:21:41.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57f8066f30 0x7f57f80693e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c157150 0x7f580c169960 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f580c06f890 0x7f580c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 --2- 172.21.15.45:0/306689788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f580c0c0550 0x7f580c0ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 >> 172.21.15.45:0/306689788 conn(0x7f580c0cd060 msgr2=0x7f580c0caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 shutdown_connections 2023-10-06T05:21:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.285+0000 7f5812c21700 1 -- 172.21.15.45:0/306689788 wait complete. 2023-10-06T05:21:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-06T05:21:41.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:21:41.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.440+0000 7fd070931700 1 Processor -- start 2023-10-06T05:21:41.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.440+0000 7fd070931700 1 -- start start 2023-10-06T05:21:41.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.441+0000 7fd070931700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd06c157330 0x7fd06c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.441+0000 7fd070931700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0c3b50 con 0x7fd06c0ccb70 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd070931700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c157700 con 0x7fd06c0aaae0 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd070931700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0c98e0 con 0x7fd06c157330 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd06c0ccb70 0x7fd06c0c3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60610/0 (socket says 172.21.15.45:60610) 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06ad9d700 1 -- 172.21.15.45:0/2236788956 learned_addr learned my addr 172.21.15.45:0/2236788956 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06a59c700 1 --2- 172.21.15.45:0/2236788956 >> v2:172.21.15.144:3300/0 conn(0x7fd06c157330 0x7fd06c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568975821 0 0) 0x7fd06c0c3b50 con 0x7fd06c0ccb70 2023-10-06T05:21:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd058003570 con 0x7fd06c0ccb70 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1973820392 0 0) 0x7fd058003570 con 0x7fd06c0ccb70 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 >> v2:172.21.15.144:3300/0 conn(0x7fd06c157330 msgr2=0x7fd06c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 --2- 172.21.15.45:0/2236788956 >> v2:172.21.15.144:3300/0 conn(0x7fd06c157330 0x7fd06c0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 >> v1:172.21.15.144:6789/0 conn(0x7fd06c0aaae0 legacy=0x7fd06c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0c9bc0 con 0x7fd06c0ccb70 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fd0640017f0 con 0x7fd06c0ccb70 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06a59c700 1 --2- 172.21.15.45:0/2236788956 >> v2:172.21.15.144:3300/0 conn(0x7fd06c157330 0x7fd06c0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 >> v1:172.21.15.45:6789/0 conn(0x7fd06c0ccb70 legacy=0x7fd06c0c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd058003a60 0x7fd058003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058004810 0x7fd058004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0580051c0 0x7fd05800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c98e0 con 0x7fd058004810 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05800c0d0 con 0x7fd0580051c0 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.442+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05800c290 con 0x7fd058003a60 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd069d9b700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058004810 0x7fd058004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd06a59c700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd058003a60 0x7fd058003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd06ad9d700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0580051c0 0x7fd05800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd069d9b700 1 -- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd058003a60 msgr2=0x7fd058003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd069d9b700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd058003a60 0x7fd058003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd069d9b700 1 -- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0580051c0 msgr2=0x7fd05800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd069d9b700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0580051c0 0x7fd05800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd06ad9d700 1 -- 172.21.15.45:0/2236788956 reap_dead start 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd069d9b700 1 -- 172.21.15.45:0/2236788956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058003570 con 0x7fd058004810 2023-10-06T05:21:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd069d9b700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058004810 0x7fd058004c80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd0540018b0 tx=0x7fd054007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c829e3a74dd5e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054012420 con 0x7fd058004810 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0540125a0 con 0x7fd058004810 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.443+0000 7fd06959a700 1 -- 172.21.15.45:0/2236788956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054012960 con 0x7fd058004810 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.444+0000 7fd070931700 1 -- 172.21.15.45:0/2236788956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058004810 msgr2=0x7fd058004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.444+0000 7fd070931700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058004810 0x7fd058004c80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd0540018b0 tx=0x7fd054007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.444+0000 7fd070931700 1 -- 172.21.15.45:0/2236788956 shutdown_connections 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.444+0000 7fd070931700 1 --2- 172.21.15.45:0/2236788956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058004810 0x7fd058004c80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.444+0000 7fd070931700 1 -- 172.21.15.45:0/2236788956 >> 172.21.15.45:0/2236788956 conn(0x7fd06c067e00 msgr2=0x7fd06c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.444+0000 7fd070931700 1 -- 172.21.15.45:0/2236788956 shutdown_connections 2023-10-06T05:21:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.444+0000 7fd070931700 1 -- 172.21.15.45:0/2236788956 wait complete. 2023-10-06T05:21:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.446+0000 7fd070931700 1 Processor -- start 2023-10-06T05:21:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.446+0000 7fd070931700 1 -- start start 2023-10-06T05:21:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd070931700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd058004810 0x7fd06c0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd070931700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c153b70 0x7fd06c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd070931700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0ac700 0x7fd06c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd070931700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0b19b0 con 0x7fd06c0ac700 2023-10-06T05:21:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd070931700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0ad090 con 0x7fd058004810 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd070931700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c15d400 con 0x7fd06c153b70 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0ac700 0x7fd06c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0ac700 0x7fd06c0acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43710/0 (socket says 172.21.15.45:43710) 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 -- 172.21.15.45:0/1103497435 learned_addr learned my addr 172.21.15.45:0/1103497435 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06a59c700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd058004810 0x7fd06c0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 -- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c153b70 msgr2=0x7fd06c15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd069d9b700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c153b70 0x7fd06c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c153b70 0x7fd06c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 -- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd058004810 msgr2=0x7fd06c0c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd058004810 0x7fd06c0c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 -- 172.21.15.45:0/1103497435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0b1680 con 0x7fd06c0ac700 2023-10-06T05:21:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06a59c700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd058004810 0x7fd06c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd06ad9d700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0ac700 0x7fd06c0acb50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd06c06eca0 tx=0x7fd0640021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd0627fc700 1 -- 172.21.15.45:0/1103497435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400ad40 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd0627fc700 1 -- 172.21.15.45:0/1103497435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0640133f0 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c15cb60 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.447+0000 7fd0627fc700 1 -- 172.21.15.45:0/1103497435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0640137b0 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.448+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c15d120 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.449+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c06e350 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.449+0000 7fd0627fc700 1 -- 172.21.15.45:0/1103497435 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd064013ba0 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.449+0000 7fd0627fc700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd04c066f30 0x7fd04c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.450+0000 7fd0627fc700 1 -- 172.21.15.45:0/1103497435 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fd064084fd0 con 0x7fd06c0ac700 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.450+0000 7fd06a59c700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd04c066f30 0x7fd04c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.450+0000 7fd06a59c700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd04c066f30 0x7fd04c0693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009390 tx=0x7fd05c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.454+0000 7fd0627fc700 1 -- 172.21.15.45:0/1103497435 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0640521a0 con 0x7fd06c0ac700 2023-10-06T05:21:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.658+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd06c0c37e0 con 0x7fd06c0ac700 2023-10-06T05:21:41.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.662+0000 7fd0627fc700 1 -- 172.21.15.45:0/1103497435 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd0640557c0 con 0x7fd06c0ac700 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.664+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd04c066f30 msgr2=0x7fd04c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.664+0000 7fd070931700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd04c066f30 0x7fd04c0693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009390 tx=0x7fd05c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0ac700 msgr2=0x7fd06c0acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0ac700 0x7fd06c0acb50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd06c06eca0 tx=0x7fd0640021d0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 shutdown_connections 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd04c066f30 0x7fd04c0693e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c0ac700 0x7fd06c0acb50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd058004810 0x7fd06c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 --2- 172.21.15.45:0/1103497435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd06c153b70 0x7fd06c15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 >> 172.21.15.45:0/1103497435 conn(0x7fd06c067e00 msgr2=0x7fd06c06c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 shutdown_connections 2023-10-06T05:21:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.665+0000 7fd070931700 1 -- 172.21.15.45:0/1103497435 wait complete. 2023-10-06T05:21:41.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2023-10-06T05:21:41.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.813+0000 7fb61feca700 1 Processor -- start 2023-10-06T05:21:41.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.813+0000 7fb61feca700 1 -- start start 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61feca700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6180b0610 0x7fb6180c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61feca700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb618069ed0 0x7fb6180cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61feca700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6180ce6a0 con 0x7fb6180c3db0 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61feca700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6180690f0 con 0x7fb6180b0610 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61feca700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6180ceeb0 con 0x7fb618069ed0 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61dc66700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6180b0610 0x7fb6180c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61d465700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb6180c3db0 0x7fb6180697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60624/0 (socket says 172.21.15.45:60624) 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61d465700 1 -- 172.21.15.45:0/1333541402 learned_addr learned my addr 172.21.15.45:0/1333541402 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.814+0000 7fb61e467700 1 --2- 172.21.15.45:0/1333541402 >> v2:172.21.15.144:3300/0 conn(0x7fb618069ed0 0x7fb6180cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61dc66700 1 -- 172.21.15.45:0/1333541402 >> v2:172.21.15.144:3300/0 conn(0x7fb618069ed0 msgr2=0x7fb6180cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61dc66700 1 --2- 172.21.15.45:0/1333541402 >> v2:172.21.15.144:3300/0 conn(0x7fb618069ed0 0x7fb6180cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61dc66700 1 -- 172.21.15.45:0/1333541402 >> v1:172.21.15.45:6789/0 conn(0x7fb6180c3db0 legacy=0x7fb6180697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61dc66700 1 -- 172.21.15.45:0/1333541402 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb618159880 con 0x7fb6180b0610 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61cc64700 1 -- 172.21.15.45:0/1333541402 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167982785 0 0) 0x7fb6180ce6a0 con 0x7fb6180c3db0 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61dc66700 1 --2- 172.21.15.45:0/1333541402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6180b0610 0x7fb6180c85d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb6180886e0 tx=0x7fb614009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b84669154e66fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61cc64700 1 -- 172.21.15.45:0/1333541402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009e70 con 0x7fb6180b0610 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61cc64700 1 -- 172.21.15.45:0/1333541402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614005c10 con 0x7fb6180b0610 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61cc64700 1 -- 172.21.15.45:0/1333541402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400b610 con 0x7fb6180b0610 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61feca700 1 -- 172.21.15.45:0/1333541402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6180b0610 msgr2=0x7fb6180c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.815+0000 7fb61feca700 1 --2- 172.21.15.45:0/1333541402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6180b0610 0x7fb6180c85d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb6180886e0 tx=0x7fb614009120 comp rx=0 tx=0).stop 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.816+0000 7fb61feca700 1 -- 172.21.15.45:0/1333541402 shutdown_connections 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.816+0000 7fb61feca700 1 --2- 172.21.15.45:0/1333541402 >> v2:172.21.15.144:3300/0 conn(0x7fb618069ed0 0x7fb6180cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.816+0000 7fb61feca700 1 --2- 172.21.15.45:0/1333541402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6180b0610 0x7fb6180c85d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.816+0000 7fb61feca700 1 -- 172.21.15.45:0/1333541402 >> 172.21.15.45:0/1333541402 conn(0x7fb618067e00 msgr2=0x7fb6180cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.816+0000 7fb61feca700 1 -- 172.21.15.45:0/1333541402 shutdown_connections 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.816+0000 7fb61feca700 1 -- 172.21.15.45:0/1333541402 wait complete. 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.817+0000 7fb61feca700 1 Processor -- start 2023-10-06T05:21:41.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.817+0000 7fb61feca700 1 -- start start 2023-10-06T05:21:41.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61feca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb618069ed0 0x7fb6180b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61feca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6180b0610 0x7fb61815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61feca700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6180c3db0 0x7fb61806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61feca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61815bbf0 con 0x7fb618069ed0 2023-10-06T05:21:41.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61feca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6180b1d00 con 0x7fb6180b0610 2023-10-06T05:21:41.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61feca700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6180b1fe0 con 0x7fb6180c3db0 2023-10-06T05:21:41.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb618069ed0 0x7fb6180b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb618069ed0 0x7fb6180b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43738/0 (socket says 172.21.15.45:43738) 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 -- 172.21.15.45:0/970255362 learned_addr learned my addr 172.21.15.45:0/970255362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61d465700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6180b0610 0x7fb61815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61e467700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6180c3db0 0x7fb61806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 -- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6180c3db0 msgr2=0x7fb61806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6180c3db0 0x7fb61806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 -- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6180b0610 msgr2=0x7fb61815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6180b0610 0x7fb61815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 -- 172.21.15.45:0/970255362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6140093c0 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61d465700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6180b0610 0x7fb61815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61e467700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6180c3db0 0x7fb61806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61dc66700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb618069ed0 0x7fb6180b16f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb614009160 tx=0x7fb614016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb609ffb700 1 -- 172.21.15.45:0/970255362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614006310 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb609ffb700 1 -- 172.21.15.45:0/970255362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614006490 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6180b22c0 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.818+0000 7fb609ffb700 1 -- 172.21.15.45:0/970255362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614006850 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.819+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6180b2880 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.820+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6180b2a70 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.820+0000 7fb609ffb700 1 -- 172.21.15.45:0/970255362 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb61401b690 con 0x7fb618069ed0 2023-10-06T05:21:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.821+0000 7fb609ffb700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5fc066f30 0x7fb5fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.821+0000 7fb609ffb700 1 -- 172.21.15.45:0/970255362 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fb6140868d0 con 0x7fb618069ed0 2023-10-06T05:21:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.821+0000 7fb61d465700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5fc066f30 0x7fb5fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.821+0000 7fb61d465700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5fc066f30 0x7fb5fc0693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb6180b1c30 tx=0x7fb60c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:41.825+0000 7fb609ffb700 1 -- 172.21.15.45:0/970255362 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb614053b40 con 0x7fb618069ed0 2023-10-06T05:21:42.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.027+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) v1 -- 0x7fb61806e350 con 0x7fb618069ed0 2023-10-06T05:21:42.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.028+0000 7fb609ffb700 1 -- 172.21.15.45:0/970255362 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb614057160 con 0x7fb618069ed0 2023-10-06T05:21:42.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5fc066f30 msgr2=0x7fb5fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5fc066f30 0x7fb5fc0693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb6180b1c30 tx=0x7fb60c009040 comp rx=0 tx=0).stop 2023-10-06T05:21:42.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb618069ed0 msgr2=0x7fb6180b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb618069ed0 0x7fb6180b16f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb614009160 tx=0x7fb614016000 comp rx=0 tx=0).stop 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 shutdown_connections 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5fc066f30 0x7fb5fc0693e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6180c3db0 0x7fb61806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb618069ed0 0x7fb6180b16f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 --2- 172.21.15.45:0/970255362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6180b0610 0x7fb61815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 >> 172.21.15.45:0/970255362 conn(0x7fb618067e00 msgr2=0x7fb618157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 shutdown_connections 2023-10-06T05:21:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.030+0000 7fb61feca700 1 -- 172.21.15.45:0/970255362 wait complete. 2023-10-06T05:21:42.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2023-10-06T05:21:42.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.180+0000 7efc3e3f1700 1 Processor -- start 2023-10-06T05:21:42.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.180+0000 7efc3e3f1700 1 -- start start 2023-10-06T05:21:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc3e3f1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc38158f60 0x7efc38159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc3e3f1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 0x7efc380cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc3e3f1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc38069bd0 con 0x7efc380cc390 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc3e3f1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3806a4d0 con 0x7efc38158f60 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc3e3f1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc38068fc0 con 0x7efc3806a060 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc3c98e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efc380cc390 0x7efc380c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60626/0 (socket says 172.21.15.45:60626) 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc3c98e700 1 -- 172.21.15.45:0/2222105219 learned_addr learned my addr 172.21.15.45:0/2222105219 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc377fe700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 0x7efc380cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc37fff700 1 --2- 172.21.15.45:0/2222105219 >> v2:172.21.15.144:3300/0 conn(0x7efc38158f60 0x7efc38159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157109448 0 0) 0x7efc38069bd0 con 0x7efc380cc390 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.181+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003570 con 0x7efc380cc390 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2882286900 0 0) 0x7efc20003570 con 0x7efc380cc390 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 msgr2=0x7efc380cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 0x7efc380cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 >> v2:172.21.15.144:3300/0 conn(0x7efc38158f60 msgr2=0x7efc38159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 --2- 172.21.15.45:0/2222105219 >> v2:172.21.15.144:3300/0 conn(0x7efc38158f60 0x7efc38159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380692a0 con 0x7efc380cc390 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7efc300017f0 con 0x7efc380cc390 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc377fe700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 0x7efc380cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 >> v1:172.21.15.45:6789/0 conn(0x7efc380cc390 legacy=0x7efc380c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc20003a60 0x7efc20003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20004810 0x7efc20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 0x7efc200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3806a4d0 con 0x7efc20003a60 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc38068fc0 con 0x7efc20004810 2023-10-06T05:21:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000ba60 con 0x7efc3806a060 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 --2- 172.21.15.45:0/2222105219 >> v2:172.21.15.144:3300/0 conn(0x7efc38158f60 0x7efc38159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc20003a60 0x7efc20003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc3c98e700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 0x7efc200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc377fe700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20004810 0x7efc20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 -- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 msgr2=0x7efc200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3806a060 0x7efc200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 -- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20004810 msgr2=0x7efc20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20004810 0x7efc20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 -- 172.21.15.45:0/2222105219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c009420 con 0x7efc20003a60 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.182+0000 7efc37fff700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc20003a60 0x7efc20003e70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7efc28009db0 tx=0x7efc280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc22c8632e9cac07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28009740 con 0x7efc20003a60 2023-10-06T05:21:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2800e920 con 0x7efc20003a60 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc36ffd700 1 -- 172.21.15.45:0/2222105219 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800eca0 con 0x7efc20003a60 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc3e3f1700 1 -- 172.21.15.45:0/2222105219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc20003a60 msgr2=0x7efc20003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc3e3f1700 1 --2- 172.21.15.45:0/2222105219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc20003a60 0x7efc20003e70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7efc28009db0 tx=0x7efc280091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc3c98e700 1 -- 172.21.15.45:0/2222105219 reap_dead start 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc3e3f1700 1 -- 172.21.15.45:0/2222105219 shutdown_connections 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc3e3f1700 1 -- 172.21.15.45:0/2222105219 >> 172.21.15.45:0/2222105219 conn(0x7efc380cdf50 msgr2=0x7efc380ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc3e3f1700 1 -- 172.21.15.45:0/2222105219 shutdown_connections 2023-10-06T05:21:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.183+0000 7efc3e3f1700 1 -- 172.21.15.45:0/2222105219 wait complete. 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.184+0000 7efc3e3f1700 1 Processor -- start 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.185+0000 7efc3e3f1700 1 -- start start 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.185+0000 7efc3e3f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20003a60 0x7efc38156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.185+0000 7efc3e3f1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc38156790 0x7efc380c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.185+0000 7efc3e3f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c3be0 0x7efc380b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.185+0000 7efc3e3f1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380c4050 con 0x7efc380c3be0 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.185+0000 7efc3e3f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc38169140 con 0x7efc20003a60 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.185+0000 7efc3e3f1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc38169410 con 0x7efc38156790 2023-10-06T05:21:42.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c3be0 0x7efc380b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c3be0 0x7efc380b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43754/0 (socket says 172.21.15.45:43754) 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 -- 172.21.15.45:0/3097850880 learned_addr learned my addr 172.21.15.45:0/3097850880 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc37fff700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20003a60 0x7efc38156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc377fe700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc38156790 0x7efc380c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 -- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc38156790 msgr2=0x7efc380c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc38156790 0x7efc380c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 -- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20003a60 msgr2=0x7efc38156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20003a60 0x7efc38156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 -- 172.21.15.45:0/3097850880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc381574a0 con 0x7efc380c3be0 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3c98e700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c3be0 0x7efc380b1e70 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7efc3806bf40 tx=0x7efc30002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc1ffff700 1 -- 172.21.15.45:0/3097850880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000acd0 con 0x7efc380c3be0 2023-10-06T05:21:42.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc1ffff700 1 -- 172.21.15.45:0/3097850880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3000ae50 con 0x7efc380c3be0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc38157760 con 0x7efc380c3be0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc1ffff700 1 -- 172.21.15.45:0/3097850880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30013640 con 0x7efc380c3be0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.186+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc38157d20 con 0x7efc380c3be0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.188+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc381695c0 con 0x7efc380c3be0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.188+0000 7efc1ffff700 1 -- 172.21.15.45:0/3097850880 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc30013a30 con 0x7efc380c3be0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.188+0000 7efc1ffff700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc24066f30 0x7efc240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.189+0000 7efc1ffff700 1 -- 172.21.15.45:0/3097850880 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5698+0+0 (secure 0 0 0) 0x7efc30084c50 con 0x7efc380c3be0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.190+0000 7efc37fff700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc24066f30 0x7efc240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.191+0000 7efc37fff700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc24066f30 0x7efc240693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc280091e0 tx=0x7efc28019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.197+0000 7efc1ffff700 1 -- 172.21.15.45:0/3097850880 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc381695c0 con 0x7efc380c3be0 2023-10-06T05:21:42.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:42.495+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) v1 -- 0x7efc381695c0 con 0x7efc380c3be0 2023-10-06T05:21:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.289+0000 7efc1ffff700 1 -- 172.21.15.45:0/3097850880 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v319) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc381695c0 con 0x7efc380c3be0 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc24066f30 msgr2=0x7efc240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc24066f30 0x7efc240693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc280091e0 tx=0x7efc28019000 comp rx=0 tx=0).stop 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c3be0 msgr2=0x7efc380b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c3be0 0x7efc380b1e70 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7efc3806bf40 tx=0x7efc30002210 comp rx=0 tx=0).stop 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 shutdown_connections 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc24066f30 0x7efc240693e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc380c3be0 0x7efc380b1e70 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc20003a60 0x7efc38156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 --2- 172.21.15.45:0/3097850880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc38156790 0x7efc380c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 >> 172.21.15.45:0/3097850880 conn(0x7efc380cdf50 msgr2=0x7efc380cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 shutdown_connections 2023-10-06T05:21:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.292+0000 7efc3e3f1700 1 -- 172.21.15.45:0/3097850880 wait complete. 2023-10-06T05:21:43.310 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2023-10-06T05:21:43.311 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2023-10-06T05:21:43.311 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2023-10-06T05:21:43.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2023-10-06T05:21:43.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.448+0000 7f2602def700 1 Processor -- start 2023-10-06T05:21:43.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.448+0000 7f2602def700 1 -- start start 2023-10-06T05:21:43.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f2602def700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 0x7f25fc0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f2602def700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f25fc1595d0 0x7f25fc06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f2602def700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25fc0c7d00 con 0x7f25fc06b5c0 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f2602def700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25fc0c7780 con 0x7f25fc1595d0 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f2602def700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0c86d0 con 0x7f25fc159200 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f260138c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f25fc06b5c0 0x7f25fc0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60634/0 (socket says 172.21.15.45:60634) 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f260138c700 1 -- 172.21.15.45:0/510294747 learned_addr learned my addr 172.21.15.45:0/510294747 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f25fbfff700 1 --2- 172.21.15.45:0/510294747 >> v2:172.21.15.144:3300/0 conn(0x7f25fc1595d0 0x7f25fc06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f2600b8b700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 0x7f25fc0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600548251 0 0) 0x7f25fc0c7d00 con 0x7f25fc06b5c0 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.449+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec003570 con 0x7f25fc06b5c0 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714329760 0 0) 0x7f25ec003570 con 0x7f25fc06b5c0 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 msgr2=0x7f25fc0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:43.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 0x7f25fc0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 >> v2:172.21.15.144:3300/0 conn(0x7f25fc1595d0 msgr2=0x7f25fc06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 --2- 172.21.15.45:0/510294747 >> v2:172.21.15.144:3300/0 conn(0x7f25fc1595d0 0x7f25fc06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc06bb40 con 0x7f25fc06b5c0 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f25f40017f0 con 0x7f25fc06b5c0 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 >> v1:172.21.15.45:6789/0 conn(0x7f25fc06b5c0 legacy=0x7f25fc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 0x7f25ec003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25ec003fe0 0x7f25ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ec004d50 0x7f25ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0c7780 con 0x7f25ec003fe0 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0c86d0 con 0x7f25ec004d50 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec00ba90 con 0x7f25fc159200 2023-10-06T05:21:43.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fbfff700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25ec003fe0 0x7f25ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f2600b8b700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 0x7f25ec003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f260138c700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ec004d50 0x7f25ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fbfff700 1 -- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 msgr2=0x7f25ec003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fbfff700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc159200 0x7f25ec003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fbfff700 1 -- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ec004d50 msgr2=0x7f25ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fbfff700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ec004d50 0x7f25ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fbfff700 1 -- 172.21.15.45:0/510294747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec003570 con 0x7f25ec003fe0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.450+0000 7f25fbfff700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25ec003fe0 0x7f25ec004810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009c00 tx=0x7f25e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4c471a34719c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e800e6b0 con 0x7f25ec003fe0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e800e810 con 0x7f25ec003fe0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f25fb7fe700 1 -- 172.21.15.45:0/510294747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e800ebd0 con 0x7f25ec003fe0 2023-10-06T05:21:43.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f2602def700 1 -- 172.21.15.45:0/510294747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25ec003fe0 msgr2=0x7f25ec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f2602def700 1 --2- 172.21.15.45:0/510294747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25ec003fe0 0x7f25ec004810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009c00 tx=0x7f25e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f260138c700 1 -- 172.21.15.45:0/510294747 reap_dead start 2023-10-06T05:21:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f2602def700 1 -- 172.21.15.45:0/510294747 shutdown_connections 2023-10-06T05:21:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f2602def700 1 -- 172.21.15.45:0/510294747 >> 172.21.15.45:0/510294747 conn(0x7f25fc06f090 msgr2=0x7f25fc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f2602def700 1 -- 172.21.15.45:0/510294747 shutdown_connections 2023-10-06T05:21:43.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.451+0000 7f2602def700 1 -- 172.21.15.45:0/510294747 wait complete. 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.452+0000 7f2602def700 1 Processor -- start 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.453+0000 7f2602def700 1 -- start start 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.453+0000 7f2602def700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25ec003fe0 0x7f25fc0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.453+0000 7f2602def700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0b0760 0x7f25fc14ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.453+0000 7f2602def700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25fc0b11d0 0x7f25fc153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.453+0000 7f2602def700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0b02d0 con 0x7f25fc0b11d0 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.453+0000 7f2602def700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc14e330 con 0x7f25fc0b0760 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.453+0000 7f2602def700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0b1a00 con 0x7f25ec003fe0 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25fc0b11d0 0x7f25fc153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25fc0b11d0 0x7f25fc153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43774/0 (socket says 172.21.15.45:43774) 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 -- 172.21.15.45:0/4221443138 learned_addr learned my addr 172.21.15.45:0/4221443138 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f25fbfff700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0b0760 0x7f25fc14ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f2600b8b700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25ec003fe0 0x7f25fc0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 -- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25ec003fe0 msgr2=0x7f25fc0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25ec003fe0 0x7f25fc0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 -- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0b0760 msgr2=0x7f25fc14ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0b0760 0x7f25fc14ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 -- 172.21.15.45:0/4221443138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc0afc80 con 0x7f25fc0b11d0 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f2600b8b700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25ec003fe0 0x7f25fc0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f260138c700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25fc0b11d0 0x7f25fc153010 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0c7380 tx=0x7f25f4002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f25f8ff9700 1 -- 172.21.15.45:0/4221443138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f400ae50 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f25f8ff9700 1 -- 172.21.15.45:0/4221443138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f40133f0 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc0aff60 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f25f8ff9700 1 -- 172.21.15.45:0/4221443138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40137b0 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.454+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc0b1650 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.455+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc06e2f0 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.456+0000 7f25f8ff9700 1 -- 172.21.15.45:0/4221443138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25f4013b60 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.456+0000 7f25f8ff9700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25e0066f30 0x7f25e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.456+0000 7f25f8ff9700 1 -- 172.21.15.45:0/4221443138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f25f4084f00 con 0x7f25fc0b11d0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.457+0000 7f2600b8b700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25e0066f30 0x7f25e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.457+0000 7f2600b8b700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25e0066f30 0x7f25e00693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006dc0 tx=0x7f25f0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.461+0000 7f25f8ff9700 1 -- 172.21.15.45:0/4221443138 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25f4052170 con 0x7f25fc0b11d0 2023-10-06T05:21:43.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:43.748+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) v1 -- 0x7f25fc0b0410 con 0x7f25fc0b11d0 2023-10-06T05:21:44.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.291+0000 7f25f8ff9700 1 -- 172.21.15.45:0/4221443138 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v320) v1 ==== 138+0+0 (secure 0 0 0) 0x7f25f4055790 con 0x7f25fc0b11d0 2023-10-06T05:21:44.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25e0066f30 msgr2=0x7f25e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25e0066f30 0x7f25e00693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006dc0 tx=0x7f25f0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25fc0b11d0 msgr2=0x7f25fc153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25fc0b11d0 0x7f25fc153010 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0c7380 tx=0x7f25f4002210 comp rx=0 tx=0).stop 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 shutdown_connections 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25e0066f30 0x7f25e00693e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25fc0b11d0 0x7f25fc153010 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25ec003fe0 0x7f25fc0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 --2- 172.21.15.45:0/4221443138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0b0760 0x7f25fc14ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.294+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 >> 172.21.15.45:0/4221443138 conn(0x7f25fc06f090 msgr2=0x7f25fc1546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.295+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 shutdown_connections 2023-10-06T05:21:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.295+0000 7f2602def700 1 -- 172.21.15.45:0/4221443138 wait complete. 2023-10-06T05:21:44.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:21:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.454+0000 7fde2d414700 1 Processor -- start 2023-10-06T05:21:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.454+0000 7fde2d414700 1 -- start start 2023-10-06T05:21:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.455+0000 7fde2d414700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde2813f900 0x7fde280cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.455+0000 7fde2d414700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fde280c8de0 0x7fde280c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.455+0000 7fde2d414700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde280c9630 con 0x7fde280c9d50 2023-10-06T05:21:44.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.455+0000 7fde2d414700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280ca150 con 0x7fde2813f900 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.455+0000 7fde2d414700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280cb550 con 0x7fde280c8de0 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.455+0000 7fde26ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde2813f900 0x7fde280cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde277fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fde280c9d50 0x7fde2806c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60638/0 (socket says 172.21.15.45:60638) 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde277fe700 1 -- 172.21.15.45:0/288463613 learned_addr learned my addr 172.21.15.45:0/288463613 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde267fc700 1 --2- 172.21.15.45:0/288463613 >> v2:172.21.15.144:3300/0 conn(0x7fde280c8de0 0x7fde280c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde26ffd700 1 -- 172.21.15.45:0/288463613 >> v2:172.21.15.144:3300/0 conn(0x7fde280c8de0 msgr2=0x7fde280c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde26ffd700 1 --2- 172.21.15.45:0/288463613 >> v2:172.21.15.144:3300/0 conn(0x7fde280c8de0 0x7fde280c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde26ffd700 1 -- 172.21.15.45:0/288463613 >> v1:172.21.15.45:6789/0 conn(0x7fde280c9d50 legacy=0x7fde2806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde26ffd700 1 -- 172.21.15.45:0/288463613 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280cb830 con 0x7fde2813f900 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde25ffb700 1 -- 172.21.15.45:0/288463613 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507151617 0 0) 0x7fde280c9630 con 0x7fde280c9d50 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde26ffd700 1 --2- 172.21.15.45:0/288463613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde2813f900 0x7fde280cc840 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fde280cc620 tx=0x7fde180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53317e30600d7ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde25ffb700 1 -- 172.21.15.45:0/288463613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18011630 con 0x7fde2813f900 2023-10-06T05:21:44.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde25ffb700 1 -- 172.21.15.45:0/288463613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18011790 con 0x7fde2813f900 2023-10-06T05:21:44.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.456+0000 7fde25ffb700 1 -- 172.21.15.45:0/288463613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18011b30 con 0x7fde2813f900 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 -- 172.21.15.45:0/288463613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde2813f900 msgr2=0x7fde280cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 --2- 172.21.15.45:0/288463613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde2813f900 0x7fde280cc840 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fde280cc620 tx=0x7fde180091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 -- 172.21.15.45:0/288463613 shutdown_connections 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 --2- 172.21.15.45:0/288463613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde2813f900 0x7fde280cc840 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 --2- 172.21.15.45:0/288463613 >> v2:172.21.15.144:3300/0 conn(0x7fde280c8de0 0x7fde280c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 -- 172.21.15.45:0/288463613 >> 172.21.15.45:0/288463613 conn(0x7fde280691c0 msgr2=0x7fde280cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 -- 172.21.15.45:0/288463613 shutdown_connections 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.457+0000 7fde2d414700 1 -- 172.21.15.45:0/288463613 wait complete. 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.458+0000 7fde2d414700 1 Processor -- start 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.458+0000 7fde2d414700 1 -- start start 2023-10-06T05:21:44.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde2d414700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde280c8de0 0x7fde28167bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde2d414700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde280c9d50 0x7fde281670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde2d414700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2813f900 0x7fde280affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde2d414700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde28135ed0 con 0x7fde280c8de0 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde2d414700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde281680f0 con 0x7fde280c9d50 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde2d414700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde28134ac0 con 0x7fde2813f900 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde26ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde280c8de0 0x7fde28167bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde26ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde280c8de0 0x7fde28167bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43784/0 (socket says 172.21.15.45:43784) 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde26ffd700 1 -- 172.21.15.45:0/2506007388 learned_addr learned my addr 172.21.15.45:0/2506007388 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.459+0000 7fde267fc700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde280c9d50 0x7fde281670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde277fe700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2813f900 0x7fde280affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde26ffd700 1 -- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2813f900 msgr2=0x7fde280affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde26ffd700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2813f900 0x7fde280affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde26ffd700 1 -- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde280c9d50 msgr2=0x7fde281670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde26ffd700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde280c9d50 0x7fde281670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde26ffd700 1 -- 172.21.15.45:0/2506007388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18009420 con 0x7fde280c8de0 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde277fe700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2813f900 0x7fde280affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde26ffd700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde280c8de0 0x7fde28167bb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fde2806e930 tx=0x7fde18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde1effd700 1 -- 172.21.15.45:0/2506007388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18011b10 con 0x7fde280c8de0 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde1effd700 1 -- 172.21.15.45:0/2506007388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde180253f0 con 0x7fde280c8de0 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde28134da0 con 0x7fde280c8de0 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde1effd700 1 -- 172.21.15.45:0/2506007388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18025790 con 0x7fde280c8de0 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.460+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde28135360 con 0x7fde280c8de0 2023-10-06T05:21:44.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.461+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2806e2d0 con 0x7fde280c8de0 2023-10-06T05:21:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.462+0000 7fde1effd700 1 -- 172.21.15.45:0/2506007388 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde18025b80 con 0x7fde280c8de0 2023-10-06T05:21:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.462+0000 7fde1effd700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.462+0000 7fde1effd700 1 -- 172.21.15.45:0/2506007388 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fde18085d90 con 0x7fde280c8de0 2023-10-06T05:21:44.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.462+0000 7fde267fc700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.463+0000 7fde267fc700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde08066f30 0x7fde080693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fde28169c10 tx=0x7fde10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.467+0000 7fde1effd700 1 -- 172.21.15.45:0/2506007388 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde18053000 con 0x7fde280c8de0 2023-10-06T05:21:44.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.668+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fde280cbb10 con 0x7fde280c8de0 2023-10-06T05:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.672+0000 7fde1effd700 1 -- 172.21.15.45:0/2506007388 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde180150c0 con 0x7fde280c8de0 2023-10-06T05:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde08066f30 msgr2=0x7fde080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde08066f30 0x7fde080693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fde28169c10 tx=0x7fde10008040 comp rx=0 tx=0).stop 2023-10-06T05:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde280c8de0 msgr2=0x7fde28167bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde280c8de0 0x7fde28167bb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fde2806e930 tx=0x7fde18017000 comp rx=0 tx=0).stop 2023-10-06T05:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 shutdown_connections 2023-10-06T05:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2813f900 0x7fde280affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde280c8de0 0x7fde28167bb0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 --2- 172.21.15.45:0/2506007388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde280c9d50 0x7fde281670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 >> 172.21.15.45:0/2506007388 conn(0x7fde280691c0 msgr2=0x7fde280cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 shutdown_connections 2023-10-06T05:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.674+0000 7fde2d414700 1 -- 172.21.15.45:0/2506007388 wait complete. 2023-10-06T05:21:44.690 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:21:44.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:44.732 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:44.789 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:21:44.789 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:21:44.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.924+0000 7fba9e452700 1 Processor -- start 2023-10-06T05:21:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.924+0000 7fba9e452700 1 -- start start 2023-10-06T05:21:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.925+0000 7fba9e452700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fba98156ce0 0x7fba981570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.925+0000 7fba9e452700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba9806bad0 0x7fba9815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.925+0000 7fba9e452700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba980afbe0 0x7fba980ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.925+0000 7fba9e452700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9815b150 con 0x7fba9806bad0 2023-10-06T05:21:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.925+0000 7fba9e452700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9806bf70 con 0x7fba98156ce0 2023-10-06T05:21:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.925+0000 7fba9e452700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9815b380 con 0x7fba980afbe0 2023-10-06T05:21:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.925+0000 7fba977fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba9806bad0 0x7fba9815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba9806bad0 0x7fba9815ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43792/0 (socket says 172.21.15.45:43792) 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 -- 172.21.15.45:0/3435804553 learned_addr learned my addr 172.21.15.45:0/3435804553 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba97fff700 1 --2- 172.21.15.45:0/3435804553 >> v2:172.21.15.144:3300/0 conn(0x7fba98156ce0 0x7fba981570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba9c9ef700 1 --2- 172.21.15.45:0/3435804553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba980afbe0 0x7fba980ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 -- 172.21.15.45:0/3435804553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba980afbe0 msgr2=0x7fba980ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 --2- 172.21.15.45:0/3435804553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba980afbe0 0x7fba980ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 -- 172.21.15.45:0/3435804553 >> v2:172.21.15.144:3300/0 conn(0x7fba98156ce0 msgr2=0x7fba981570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 --2- 172.21.15.45:0/3435804553 >> v2:172.21.15.144:3300/0 conn(0x7fba98156ce0 0x7fba981570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 -- 172.21.15.45:0/3435804553 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9815b660 con 0x7fba9806bad0 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba977fe700 1 --2- 172.21.15.45:0/3435804553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba9806bad0 0x7fba9815ac10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fba980af900 tx=0x7fba8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8185d8d0eec682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba96ffd700 1 -- 172.21.15.45:0/3435804553 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00b640 con 0x7fba9806bad0 2023-10-06T05:21:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba96ffd700 1 -- 172.21.15.45:0/3435804553 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c00b7a0 con 0x7fba9806bad0 2023-10-06T05:21:44.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.926+0000 7fba96ffd700 1 -- 172.21.15.45:0/3435804553 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00bb60 con 0x7fba9806bad0 2023-10-06T05:21:44.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 -- 172.21.15.45:0/3435804553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba9806bad0 msgr2=0x7fba9815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 --2- 172.21.15.45:0/3435804553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba9806bad0 0x7fba9815ac10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fba980af900 tx=0x7fba8c009250 comp rx=0 tx=0).stop 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 -- 172.21.15.45:0/3435804553 shutdown_connections 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 --2- 172.21.15.45:0/3435804553 >> v2:172.21.15.144:3300/0 conn(0x7fba98156ce0 0x7fba981570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 --2- 172.21.15.45:0/3435804553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba980afbe0 0x7fba980ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 --2- 172.21.15.45:0/3435804553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba9806bad0 0x7fba9815ac10 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 -- 172.21.15.45:0/3435804553 >> 172.21.15.45:0/3435804553 conn(0x7fba9806f090 msgr2=0x7fba9806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 -- 172.21.15.45:0/3435804553 shutdown_connections 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.927+0000 7fba9e452700 1 -- 172.21.15.45:0/3435804553 wait complete. 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.928+0000 7fba9e452700 1 Processor -- start 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.928+0000 7fba9e452700 1 -- start start 2023-10-06T05:21:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9e452700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba9806bad0 0x7fba980c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9e452700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba980afbe0 0x7fba981691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9e452700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba98156ce0 0x7fba980c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9e452700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980c9400 con 0x7fba98156ce0 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9e452700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980c87f0 con 0x7fba980afbe0 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9e452700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba980c8ad0 con 0x7fba9806bad0 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9c9ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba98156ce0 0x7fba980c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9c9ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba98156ce0 0x7fba980c11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43798/0 (socket says 172.21.15.45:43798) 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.929+0000 7fba9c9ef700 1 -- 172.21.15.45:0/4104163 learned_addr learned my addr 172.21.15.45:0/4104163 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba977fe700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba980afbe0 0x7fba981691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba97fff700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba9806bad0 0x7fba980c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9c9ef700 1 -- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba9806bad0 msgr2=0x7fba980c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9c9ef700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba9806bad0 0x7fba980c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9c9ef700 1 -- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba980afbe0 msgr2=0x7fba981691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9c9ef700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba980afbe0 0x7fba981691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9c9ef700 1 -- 172.21.15.45:0/4104163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0094f0 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9c9ef700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba98156ce0 0x7fba980c11e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e90 tx=0x7fba9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba7ffff700 1 -- 172.21.15.45:0/4104163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9000cc20 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba7ffff700 1 -- 172.21.15.45:0/4104163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9000f040 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba980c8db0 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba7ffff700 1 -- 172.21.15.45:0/4104163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90014480 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.930+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba980c83e0 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.931+0000 7fba7ffff700 1 -- 172.21.15.45:0/4104163 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba90014830 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.932+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba980c4e10 con 0x7fba98156ce0 2023-10-06T05:21:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.932+0000 7fba7ffff700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba84066f60 0x7fba84069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.932+0000 7fba7ffff700 1 -- 172.21.15.45:0/4104163 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fba90084f80 con 0x7fba98156ce0 2023-10-06T05:21:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.932+0000 7fba97fff700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba84066f60 0x7fba84069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.933+0000 7fba97fff700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba84066f60 0x7fba84069410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fba88009b50 tx=0x7fba88008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:44.936+0000 7fba7ffff700 1 -- 172.21.15.45:0/4104163 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba980c4e10 con 0x7fba98156ce0 2023-10-06T05:21:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.182+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba980c4e10 con 0x7fba98156ce0 2023-10-06T05:21:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.183+0000 7fba7ffff700 1 -- 172.21.15.45:0/4104163 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fba90018de0 con 0x7fba98156ce0 2023-10-06T05:21:45.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:45.185 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4106107470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4106107470},{"type":"v1","addr":"172.21.15.45:6835","nonce":4106107470}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2794318700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2794318700},{"type":"v1","addr":"172.21.15.144:6836","nonce":2794318700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3797968254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3797968254},{"type":"v1","addr":"172.21.15.144:6837","nonce":3797968254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6942,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2705674276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2705674276},{"type":"v1","addr":"172.21.15.45:6834","nonce":2705674276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[]} 2023-10-06T05:21:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba84066f60 msgr2=0x7fba84069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba84066f60 0x7fba84069410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fba88009b50 tx=0x7fba88008040 comp rx=0 tx=0).stop 2023-10-06T05:21:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba98156ce0 msgr2=0x7fba980c11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba98156ce0 0x7fba980c11e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e90 tx=0x7fba9000b040 comp rx=0 tx=0).stop 2023-10-06T05:21:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 shutdown_connections 2023-10-06T05:21:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fba84066f60 0x7fba84069410 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba98156ce0 0x7fba980c11e0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba9806bad0 0x7fba980c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 --2- 172.21.15.45:0/4104163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba980afbe0 0x7fba981691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 >> 172.21.15.45:0/4104163 conn(0x7fba9806f090 msgr2=0x7fba980cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 shutdown_connections 2023-10-06T05:21:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.185+0000 7fba9e452700 1 -- 172.21.15.45:0/4104163 wait complete. 2023-10-06T05:21:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 80 2023-10-06T05:21:45.200 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:21:45.200 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:21:45.201 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:21:45.201 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:45.201 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:21:45.201 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:21:45.201 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:21:45.201 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:45.202 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:21:45.202 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:21:45.202 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:21:45.202 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:45.202 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:21:45.202 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:21:45.202 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:21:45.203 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:45.210 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:21:45.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:21:45.210 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:21:45.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:21:45.212 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:21:45.213 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:21:45.214 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:21:45.214 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:21:45.252 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:21:45.252 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:45.253 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:21:45.254 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:45.254 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:21:45.254 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:45.255 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:21:45.255 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:45.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:21:45.337 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:21:45.338 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:21:45.340 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:21:45.340 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:21:45.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.394+0000 7fd49de9b700 1 Processor -- start 2023-10-06T05:21:45.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.394+0000 7fd49de9b700 1 -- start start 2023-10-06T05:21:45.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd49de9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd498159170 0x7fd498159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd49de9b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4980ca6d0 con 0x7fd49815d040 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd49de9b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49815d550 con 0x7fd4980692d0 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd49de9b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49815d810 con 0x7fd498159170 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd4977fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd498159170 0x7fd498159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd496ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd49815d040 0x7fd498068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60656/0 (socket says 172.21.15.45:60656) 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd496ffd700 1 -- 172.21.15.45:0/3743346792 learned_addr learned my addr 172.21.15.45:0/3743346792 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd4977fe700 1 -- 172.21.15.45:0/3743346792 >> v1:172.21.15.144:6789/0 conn(0x7fd4980692d0 legacy=0x7fd4980aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd4977fe700 1 -- 172.21.15.45:0/3743346792 >> v1:172.21.15.45:6789/0 conn(0x7fd49815d040 legacy=0x7fd498068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.395+0000 7fd4977fe700 1 -- 172.21.15.45:0/3743346792 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49815daf0 con 0x7fd498159170 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd4977fe700 1 --2- 172.21.15.45:0/3743346792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd498159540 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098e0 tx=0x7fd4880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb15e2aa2754466a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd4967fc700 1 -- 172.21.15.45:0/3743346792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488009e80 con 0x7fd498159170 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd4967fc700 1 -- 172.21.15.45:0/3743346792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800b860 con 0x7fd498159170 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd4967fc700 1 -- 172.21.15.45:0/3743346792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800bbe0 con 0x7fd498159170 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd49de9b700 1 -- 172.21.15.45:0/3743346792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 msgr2=0x7fd498159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd49de9b700 1 --2- 172.21.15.45:0/3743346792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd498159540 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098e0 tx=0x7fd4880091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd49de9b700 1 -- 172.21.15.45:0/3743346792 shutdown_connections 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd49de9b700 1 --2- 172.21.15.45:0/3743346792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd498159540 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd49de9b700 1 -- 172.21.15.45:0/3743346792 >> 172.21.15.45:0/3743346792 conn(0x7fd49806f7c0 msgr2=0x7fd4980d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.396+0000 7fd49de9b700 1 -- 172.21.15.45:0/3743346792 shutdown_connections 2023-10-06T05:21:45.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.397+0000 7fd49de9b700 1 -- 172.21.15.45:0/3743346792 wait complete. 2023-10-06T05:21:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.398+0000 7fd49de9b700 1 Processor -- start 2023-10-06T05:21:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.398+0000 7fd49de9b700 1 -- start start 2023-10-06T05:21:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd49de9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4980692d0 0x7fd4980a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd49de9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd4981460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd49de9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49815d040 0x7fd49816a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd49de9b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980c9940 con 0x7fd498159170 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd49de9b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980c9c20 con 0x7fd4980692d0 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd49de9b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980c9f00 con 0x7fd49815d040 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd4981460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd4981460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43854/0 (socket says 172.21.15.45:43854) 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 -- 172.21.15.45:0/1170905764 learned_addr learned my addr 172.21.15.45:0/1170905764 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd4977fe700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4980692d0 0x7fd4980a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd497fff700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49815d040 0x7fd49816a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 -- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49815d040 msgr2=0x7fd49816a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49815d040 0x7fd49816a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 -- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4980692d0 msgr2=0x7fd4980a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4980692d0 0x7fd4980a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 -- 172.21.15.45:0/1170905764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488009420 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd496ffd700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd4981460d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c590 tx=0x7fd480001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.399+0000 7fd48f7fe700 1 -- 172.21.15.45:0/1170905764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480005b30 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.400+0000 7fd48f7fe700 1 -- 172.21.15.45:0/1170905764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480005cb0 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.400+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49816ac10 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.400+0000 7fd48f7fe700 1 -- 172.21.15.45:0/1170905764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480007320 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.400+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49816a820 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.401+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49816ae50 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.401+0000 7fd48f7fe700 1 -- 172.21.15.45:0/1170905764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd48001b3b0 con 0x7fd498159170 2023-10-06T05:21:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.401+0000 7fd48f7fe700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd478066f30 0x7fd4780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.402+0000 7fd48f7fe700 1 -- 172.21.15.45:0/1170905764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fd4800838b0 con 0x7fd498159170 2023-10-06T05:21:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.402+0000 7fd4977fe700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd478066f30 0x7fd4780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.402+0000 7fd4977fe700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd478066f30 0x7fd4780693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd4880093f0 tx=0x7fd488017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.406+0000 7fd48f7fe700 1 -- 172.21.15.45:0/1170905764 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd49816ae50 con 0x7fd498159170 2023-10-06T05:21:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:45.624+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd49816ae50 con 0x7fd498159170 2023-10-06T05:21:46.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.862+0000 7fd48f7fe700 1 -- 172.21.15.45:0/1170905764 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd49816ae50 con 0x7fd498159170 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd478066f30 msgr2=0x7fd4780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd478066f30 0x7fd4780693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd4880093f0 tx=0x7fd488017000 comp rx=0 tx=0).stop 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 msgr2=0x7fd4981460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd4981460d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c590 tx=0x7fd480001020 comp rx=0 tx=0).stop 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 shutdown_connections 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd478066f30 0x7fd4780693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49815d040 0x7fd49816a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4980692d0 0x7fd4980a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 --2- 172.21.15.45:0/1170905764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd498159170 0x7fd4981460d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 >> 172.21.15.45:0/1170905764 conn(0x7fd49806f7c0 msgr2=0x7fd498158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 shutdown_connections 2023-10-06T05:21:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:46.865+0000 7fd49de9b700 1 -- 172.21.15.45:0/1170905764 wait complete. 2023-10-06T05:21:46.868 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-10-06T05:21:46.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:21:47.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.016+0000 7fee1f35c700 1 Processor -- start 2023-10-06T05:21:47.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.016+0000 7fee1f35c700 1 -- start start 2023-10-06T05:21:47.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1f35c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee18158f60 0x7fee18159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1f35c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee180cc390 0x7fee180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1f35c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180cb530 con 0x7fee1806a060 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1f35c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1806a500 con 0x7fee18158f60 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1f35c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee180ccd50 con 0x7fee180cc390 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1d0f8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee18158f60 0x7fee18159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1d0f8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee18158f60 0x7fee18159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36352/0 (socket says 172.21.15.45:36352) 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.017+0000 7fee1d0f8700 1 -- 172.21.15.45:0/1351866500 learned_addr learned my addr 172.21.15.45:0/1351866500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee1d8f9700 1 --2- 172.21.15.45:0/1351866500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee180cc390 0x7fee180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee1d0f8700 1 -- 172.21.15.45:0/1351866500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee180cc390 msgr2=0x7fee180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee1d0f8700 1 --2- 172.21.15.45:0/1351866500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee180cc390 0x7fee180cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee1d0f8700 1 -- 172.21.15.45:0/1351866500 >> v1:172.21.15.144:6789/0 conn(0x7fee1806a060 legacy=0x7fee180cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee1d0f8700 1 -- 172.21.15.45:0/1351866500 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18069bd0 con 0x7fee18158f60 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee1d0f8700 1 --2- 172.21.15.45:0/1351866500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee18158f60 0x7fee18159330 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fee140099e0 tx=0x7fee14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def768e38e34731b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee0ffff700 1 -- 172.21.15.45:0/1351866500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400b820 con 0x7fee18158f60 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee0ffff700 1 -- 172.21.15.45:0/1351866500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1400b980 con 0x7fee18158f60 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.018+0000 7fee0ffff700 1 -- 172.21.15.45:0/1351866500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400bd20 con 0x7fee18158f60 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 -- 172.21.15.45:0/1351866500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee18158f60 msgr2=0x7fee18159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 --2- 172.21.15.45:0/1351866500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee18158f60 0x7fee18159330 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fee140099e0 tx=0x7fee14009250 comp rx=0 tx=0).stop 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 -- 172.21.15.45:0/1351866500 shutdown_connections 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 --2- 172.21.15.45:0/1351866500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee180cc390 0x7fee180cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 --2- 172.21.15.45:0/1351866500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee18158f60 0x7fee18159330 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 -- 172.21.15.45:0/1351866500 >> 172.21.15.45:0/1351866500 conn(0x7fee180cdf50 msgr2=0x7fee180ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 -- 172.21.15.45:0/1351866500 shutdown_connections 2023-10-06T05:21:47.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.019+0000 7fee1f35c700 1 -- 172.21.15.45:0/1351866500 wait complete. 2023-10-06T05:21:47.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.020+0000 7fee1f35c700 1 Processor -- start 2023-10-06T05:21:47.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.020+0000 7fee1f35c700 1 -- start start 2023-10-06T05:21:47.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1f35c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1806a060 0x7fee18152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1f35c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee180cc390 0x7fee180c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1f35c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee18158f60 0x7fee180c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1f35c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180c7720 con 0x7fee180cc390 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1f35c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180c6b10 con 0x7fee1806a060 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1f35c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee180c6df0 con 0x7fee18158f60 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1c8f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee180cc390 0x7fee180c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1c8f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee180cc390 0x7fee180c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36360/0 (socket says 172.21.15.45:36360) 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1c8f7700 1 -- 172.21.15.45:0/2164523147 learned_addr learned my addr 172.21.15.45:0/2164523147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1d8f9700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee18158f60 0x7fee180c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.021+0000 7fee1d0f8700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1806a060 0x7fee18152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1c8f7700 1 -- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee18158f60 msgr2=0x7fee180c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1c8f7700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee18158f60 0x7fee180c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1c8f7700 1 -- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1806a060 msgr2=0x7fee18152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1c8f7700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1806a060 0x7fee18152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1c8f7700 1 -- 172.21.15.45:0/2164523147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee140094f0 con 0x7fee180cc390 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1c8f7700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee180cc390 0x7fee180c5bd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c590 tx=0x7fee0800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee0d7fa700 1 -- 172.21.15.45:0/2164523147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08011590 con 0x7fee180cc390 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee0d7fa700 1 -- 172.21.15.45:0/2164523147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee08011710 con 0x7fee180cc390 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee180c70d0 con 0x7fee180cc390 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee0d7fa700 1 -- 172.21.15.45:0/2164523147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08011ad0 con 0x7fee180cc390 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.022+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee18153000 con 0x7fee180cc390 2023-10-06T05:21:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.023+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1806f410 con 0x7fee180cc390 2023-10-06T05:21:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.024+0000 7fee0d7fa700 1 -- 172.21.15.45:0/2164523147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee0801b3b0 con 0x7fee180cc390 2023-10-06T05:21:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.024+0000 7fee0d7fa700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedfc066f30 0x7fedfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.024+0000 7fee0d7fa700 1 -- 172.21.15.45:0/2164523147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fee08083740 con 0x7fee180cc390 2023-10-06T05:21:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.024+0000 7fee1d0f8700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedfc066f30 0x7fedfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.025+0000 7fee1d0f8700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedfc066f30 0x7fedfc0693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fee140099e0 tx=0x7fee14017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.029+0000 7fee0d7fa700 1 -- 172.21.15.45:0/2164523147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee08050a60 con 0x7fee180cc390 2023-10-06T05:21:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.312+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fee180cb530 con 0x7fee180cc390 2023-10-06T05:21:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.313+0000 7fee0d7fa700 1 -- 172.21.15.45:0/2164523147 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fee180cb530 con 0x7fee180cc390 2023-10-06T05:21:47.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:47.315 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:47.315 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-10-06T05:21:47.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.315 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw==", 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w==", 2023-10-06T05:21:47.316 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg==", 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:21:47.317 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw==", 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.318 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg==", 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.319 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg==", 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w==", 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.320 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw==", 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA==", 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ==", 2023-10-06T05:21:47.322 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw==", 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA==", 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.324 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw==", 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw==", 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.325 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g==", 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.326 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w==", 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:21:47.327 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l/0VtGRAA99fxS1gieVbRro7K6P1/lw==", 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A==", 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg==", 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:21:47.329 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:21:47.330 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:21:47.330 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:21:47.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:47.330 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-06T05:21:47.330 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.315+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedfc066f30 msgr2=0x7fedfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.315+0000 7fee1f35c700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedfc066f30 0x7fedfc0693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fee140099e0 tx=0x7fee14017000 comp rx=0 tx=0).stop 2023-10-06T05:21:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.315+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee180cc390 msgr2=0x7fee180c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.315+0000 7fee1f35c700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee180cc390 0x7fee180c5bd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c590 tx=0x7fee0800c8a0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 shutdown_connections 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedfc066f30 0x7fedfc0693e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee18158f60 0x7fee180c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1806a060 0x7fee18152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 --2- 172.21.15.45:0/2164523147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee180cc390 0x7fee180c5bd0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 >> 172.21.15.45:0/2164523147 conn(0x7fee180cdf50 msgr2=0x7fee1806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 shutdown_connections 2023-10-06T05:21:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.316+0000 7fee1f35c700 1 -- 172.21.15.45:0/2164523147 wait complete. 2023-10-06T05:21:47.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.463+0000 7f89c60c2700 1 Processor -- start 2023-10-06T05:21:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.463+0000 7f89c60c2700 1 -- start start 2023-10-06T05:21:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.463+0000 7f89c60c2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89c0159200 0x7f89c00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.463+0000 7f89c60c2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c01595d0 0x7f89c006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.463+0000 7f89c60c2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f89c006b5c0 0x7f89c00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.463+0000 7f89c60c2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c00c7780 con 0x7f89c0159200 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89c60c2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c00c7d00 con 0x7f89c006b5c0 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89c60c2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00c86d0 con 0x7f89c01595d0 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89c0159200 0x7f89c00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89c0159200 0x7f89c00c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36368/0 (socket says 172.21.15.45:36368) 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 -- 172.21.15.45:0/980205787 learned_addr learned my addr 172.21.15.45:0/980205787 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89beffd700 1 --2- 172.21.15.45:0/980205787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c01595d0 0x7f89c006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bffff700 1 --2- 172.21.15.45:0/980205787 >> v2:172.21.15.144:3300/0 conn(0x7f89c006b5c0 0x7f89c00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 -- 172.21.15.45:0/980205787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c01595d0 msgr2=0x7f89c006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 --2- 172.21.15.45:0/980205787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c01595d0 0x7f89c006b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 -- 172.21.15.45:0/980205787 >> v2:172.21.15.144:3300/0 conn(0x7f89c006b5c0 msgr2=0x7f89c00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 --2- 172.21.15.45:0/980205787 >> v2:172.21.15.144:3300/0 conn(0x7f89c006b5c0 0x7f89c00ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 -- 172.21.15.45:0/980205787 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c006bb40 con 0x7f89c0159200 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bffff700 1 --2- 172.21.15.45:0/980205787 >> v2:172.21.15.144:3300/0 conn(0x7f89c006b5c0 0x7f89c00ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89bf7fe700 1 --2- 172.21.15.45:0/980205787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c0159200 0x7f89c00c8190 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f89c0067b90 tx=0x7f89b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8239a51f246625e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89be7fc700 1 -- 172.21.15.45:0/980205787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000b860 con 0x7f89c0159200 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89be7fc700 1 -- 172.21.15.45:0/980205787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b000b9c0 con 0x7f89c0159200 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.464+0000 7f89be7fc700 1 -- 172.21.15.45:0/980205787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000bd60 con 0x7f89c0159200 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 -- 172.21.15.45:0/980205787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c0159200 msgr2=0x7f89c00c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 --2- 172.21.15.45:0/980205787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c0159200 0x7f89c00c8190 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f89c0067b90 tx=0x7f89b0009250 comp rx=0 tx=0).stop 2023-10-06T05:21:47.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 -- 172.21.15.45:0/980205787 shutdown_connections 2023-10-06T05:21:47.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 --2- 172.21.15.45:0/980205787 >> v2:172.21.15.144:3300/0 conn(0x7f89c006b5c0 0x7f89c00ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 --2- 172.21.15.45:0/980205787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c0159200 0x7f89c00c8190 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 --2- 172.21.15.45:0/980205787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c01595d0 0x7f89c006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 -- 172.21.15.45:0/980205787 >> 172.21.15.45:0/980205787 conn(0x7f89c006f090 msgr2=0x7f89c006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 -- 172.21.15.45:0/980205787 shutdown_connections 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.465+0000 7f89c60c2700 1 -- 172.21.15.45:0/980205787 wait complete. 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.466+0000 7f89c60c2700 1 Processor -- start 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.466+0000 7f89c60c2700 1 -- start start 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89c60c2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c006b5c0 0x7f89c00c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89c60c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c01595d0 0x7f89c00cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89c60c2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89c00ccc30 0x7f89c0169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89c60c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00cd0d0 con 0x7f89c01595d0 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89c60c2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b05b0 con 0x7f89c00ccc30 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89c60c2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c016ab50 con 0x7f89c006b5c0 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c01595d0 0x7f89c00cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c006b5c0 0x7f89c00c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c01595d0 0x7f89c00cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36372/0 (socket says 172.21.15.45:36372) 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.467+0000 7f89beffd700 1 -- 172.21.15.45:0/2034779188 learned_addr learned my addr 172.21.15.45:0/2034779188 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89bffff700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89c00ccc30 0x7f89c0169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89beffd700 1 -- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c006b5c0 msgr2=0x7f89c00c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89beffd700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c006b5c0 0x7f89c00c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89beffd700 1 -- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89c00ccc30 msgr2=0x7f89c0169e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89beffd700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89c00ccc30 0x7f89c0169e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89beffd700 1 -- 172.21.15.45:0/2034779188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00094f0 con 0x7f89c01595d0 2023-10-06T05:21:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89beffd700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c01595d0 0x7f89c00cc6f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f89b400c880 tx=0x7f89b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89a77fe700 1 -- 172.21.15.45:0/2034779188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4011600 con 0x7f89c01595d0 2023-10-06T05:21:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89a77fe700 1 -- 172.21.15.45:0/2034779188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b4011760 con 0x7f89c01595d0 2023-10-06T05:21:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89a77fe700 1 -- 172.21.15.45:0/2034779188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4011b00 con 0x7f89c01595d0 2023-10-06T05:21:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c00afc80 con 0x7f89c01595d0 2023-10-06T05:21:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.468+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c00b0240 con 0x7f89c01595d0 2023-10-06T05:21:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.469+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c00b0740 con 0x7f89c01595d0 2023-10-06T05:21:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.470+0000 7f89a77fe700 1 -- 172.21.15.45:0/2034779188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89b4018360 con 0x7f89c01595d0 2023-10-06T05:21:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.470+0000 7f89a77fe700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f89ac066f30 0x7f89ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.470+0000 7f89a77fe700 1 -- 172.21.15.45:0/2034779188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f89b4084490 con 0x7f89c01595d0 2023-10-06T05:21:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.470+0000 7f89bf7fe700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f89ac066f30 0x7f89ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.471+0000 7f89bf7fe700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f89ac066f30 0x7f89ac0693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f89c0067b90 tx=0x7f89b0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.475+0000 7f89a77fe700 1 -- 172.21.15.45:0/2034779188 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89b4051710 con 0x7f89c01595d0 2023-10-06T05:21:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.675+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89c006be20 con 0x7f89c01595d0 2023-10-06T05:21:47.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.676+0000 7f89a77fe700 1 -- 172.21.15.45:0/2034779188 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13952 (secure 0 0 0) 0x7f89b4054d30 con 0x7f89c01595d0 2023-10-06T05:21:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:47.678 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:21:46.859144+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f89ac066f30 msgr2=0x7f89ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f89ac066f30 0x7f89ac0693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f89c0067b90 tx=0x7f89b0017000 comp rx=0 tx=0).stop 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c01595d0 msgr2=0x7f89c00cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c01595d0 0x7f89c00cc6f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f89b400c880 tx=0x7f89b4008040 comp rx=0 tx=0).stop 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 shutdown_connections 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f89ac066f30 0x7f89ac0693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89c00ccc30 0x7f89c0169e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89c006b5c0 0x7f89c00c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 --2- 172.21.15.45:0/2034779188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c01595d0 0x7f89c00cc6f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 >> 172.21.15.45:0/2034779188 conn(0x7f89c006f090 msgr2=0x7f89c00aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 shutdown_connections 2023-10-06T05:21:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.679+0000 7f89c60c2700 1 -- 172.21.15.45:0/2034779188 wait complete. 2023-10-06T05:21:47.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:21:47.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:21:47.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.828+0000 7f555a2c4700 1 Processor -- start 2023-10-06T05:21:47.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.828+0000 7f555a2c4700 1 -- start start 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f555a2c4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f55541406e0 0x7f555406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f555a2c4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55540c9940 con 0x7f555406cd60 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f555a2c4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555406d230 con 0x7f5554140ab0 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f555a2c4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555406d510 con 0x7f55541406e0 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f55537fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f555406cd60 0x7f55540acfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39116/0 (socket says 172.21.15.45:39116) 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f55537fe700 1 -- 172.21.15.45:0/1003188399 learned_addr learned my addr 172.21.15.45:0/1003188399 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5553fff700 1 --2- 172.21.15.45:0/1003188399 >> v2:172.21.15.144:3300/0 conn(0x7f55541406e0 0x7f555406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17009977 0 0) 0x7f55540c9940 con 0x7f555406cd60 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5540003570 con 0x7f555406cd60 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3923809519 0 0) 0x7f5540003570 con 0x7f555406cd60 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 >> v2:172.21.15.144:3300/0 conn(0x7f55541406e0 msgr2=0x7f555406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 --2- 172.21.15.45:0/1003188399 >> v2:172.21.15.144:3300/0 conn(0x7f55541406e0 0x7f555406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 >> v1:172.21.15.144:6789/0 conn(0x7f5554140ab0 legacy=0x7f55540c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555406d7f0 con 0x7f555406cd60 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f553c001470 con 0x7f555406cd60 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 >> v1:172.21.15.45:6789/0 conn(0x7f555406cd60 legacy=0x7f55540acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540003a60 0x7f5540003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5540004810 0x7f5540004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.830+0000 7f5552ffd700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55400051c0 0x7f554000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555406d510 con 0x7f5540003a60 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554000c160 con 0x7f55400051c0 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554000c320 con 0x7f5540004810 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5553fff700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540003a60 0x7f5540003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5558861700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55400051c0 0x7f554000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f55537fe700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5540004810 0x7f5540004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5553fff700 1 -- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5540004810 msgr2=0x7f5540004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5553fff700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5540004810 0x7f5540004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5553fff700 1 -- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55400051c0 msgr2=0x7f554000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5553fff700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55400051c0 0x7f554000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5558861700 1 -- 172.21.15.45:0/1003188399 reap_dead start 2023-10-06T05:21:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5553fff700 1 -- 172.21.15.45:0/1003188399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5540003570 con 0x7f5540003a60 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5553fff700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540003a60 0x7f5540003e70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5544009c00 tx=0x7f55440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb7558abbc7ad06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400e710 con 0x7f5540003a60 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554400e870 con 0x7f5540003a60 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.831+0000 7f5552ffd700 1 -- 172.21.15.45:0/1003188399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400ec30 con 0x7f5540003a60 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.832+0000 7f555a2c4700 1 -- 172.21.15.45:0/1003188399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540003a60 msgr2=0x7f5540003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.832+0000 7f555a2c4700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540003a60 0x7f5540003e70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5544009c00 tx=0x7f55440091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.832+0000 7f555a2c4700 1 -- 172.21.15.45:0/1003188399 shutdown_connections 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.832+0000 7f555a2c4700 1 --2- 172.21.15.45:0/1003188399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540003a60 0x7f5540003e70 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.832+0000 7f555a2c4700 1 -- 172.21.15.45:0/1003188399 >> 172.21.15.45:0/1003188399 conn(0x7f55540691c0 msgr2=0x7f55540cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.832+0000 7f555a2c4700 1 -- 172.21.15.45:0/1003188399 shutdown_connections 2023-10-06T05:21:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.832+0000 7f555a2c4700 1 -- 172.21.15.45:0/1003188399 wait complete. 2023-10-06T05:21:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.833+0000 7f555a2c4700 1 Processor -- start 2023-10-06T05:21:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.833+0000 7f555a2c4700 1 -- start start 2023-10-06T05:21:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f555a2c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5540003a60 0x7f55540c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f555a2c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55540b1590 0x7f5554156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f555a2c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55540c2cf0 0x7f55540c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f555a2c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c36b0 con 0x7f55540b1590 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f555a2c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c70b0 con 0x7f5540003a60 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f555a2c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c64a0 con 0x7f55540c2cf0 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f55537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55540b1590 0x7f5554156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f55537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55540b1590 0x7f5554156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36390/0 (socket says 172.21.15.45:36390) 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f55537fe700 1 -- 172.21.15.45:0/2489958529 learned_addr learned my addr 172.21.15.45:0/2489958529 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f5553fff700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5540003a60 0x7f55540c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.834+0000 7f5558861700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55540c2cf0 0x7f55540c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f55537fe700 1 -- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55540c2cf0 msgr2=0x7f55540c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f55537fe700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55540c2cf0 0x7f55540c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f55537fe700 1 -- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5540003a60 msgr2=0x7f55540c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f55537fe700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5540003a60 0x7f55540c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f55537fe700 1 -- 172.21.15.45:0/2489958529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55540c6780 con 0x7f55540b1590 2023-10-06T05:21:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f55537fe700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55540b1590 0x7f5554156570 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f553c0015a0 tx=0x7f553c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f554bfff700 1 -- 172.21.15.45:0/2489958529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00f690 con 0x7f55540b1590 2023-10-06T05:21:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f554bfff700 1 -- 172.21.15.45:0/2489958529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c00f7f0 con 0x7f55540b1590 2023-10-06T05:21:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55540c6a60 con 0x7f55540b1590 2023-10-06T05:21:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f554bfff700 1 -- 172.21.15.45:0/2489958529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00fbb0 con 0x7f55540b1590 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.835+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5554156ab0 con 0x7f55540b1590 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.836+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555406e2d0 con 0x7f55540b1590 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.837+0000 7f554bfff700 1 -- 172.21.15.45:0/2489958529 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f553c01d460 con 0x7f55540b1590 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.837+0000 7f554bfff700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5534066f00 0x7f55340693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.837+0000 7f554bfff700 1 -- 172.21.15.45:0/2489958529 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f553c084600 con 0x7f55540b1590 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.837+0000 7f5553fff700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5534066f00 0x7f55340693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.838+0000 7f5553fff700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5534066f00 0x7f55340693b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f55440091e0 tx=0x7f554401a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:47.842+0000 7f554bfff700 1 -- 172.21.15.45:0/2489958529 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f553c051920 con 0x7f55540b1590 2023-10-06T05:21:48.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.041+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f555404bb40 con 0x7f55540b1590 2023-10-06T05:21:48.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.042+0000 7f554bfff700 1 -- 172.21.15.45:0/2489958529 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f553c054f40 con 0x7f55540b1590 2023-10-06T05:21:48.044 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5534066f00 msgr2=0x7f55340693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5534066f00 0x7f55340693b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f55440091e0 tx=0x7f554401a000 comp rx=0 tx=0).stop 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55540b1590 msgr2=0x7f5554156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55540b1590 0x7f5554156570 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f553c0015a0 tx=0x7f553c001e50 comp rx=0 tx=0).stop 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 shutdown_connections 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5534066f00 0x7f55340693b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55540c2cf0 0x7f55540c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5540003a60 0x7f55540c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 --2- 172.21.15.45:0/2489958529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55540b1590 0x7f5554156570 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 >> 172.21.15.45:0/2489958529 conn(0x7f55540691c0 msgr2=0x7f555413f740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 shutdown_connections 2023-10-06T05:21:48.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.044+0000 7f555a2c4700 1 -- 172.21.15.45:0/2489958529 wait complete. 2023-10-06T05:21:48.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.192+0000 7f2f066e3700 1 Processor -- start 2023-10-06T05:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.192+0000 7f2f066e3700 1 -- start start 2023-10-06T05:21:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2f066e3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f000b0610 0x7f2f000c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2f066e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f000cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2f066e3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f000ce6a0 con 0x7f2f000c3db0 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2f066e3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f000690f0 con 0x7f2f000b0610 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2f066e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000ceeb0 con 0x7f2f00069ed0 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2efffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f000b0610 0x7f2f000c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2eff7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2f000c3db0 0x7f2f000697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39124/0 (socket says 172.21.15.45:39124) 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2eff7fe700 1 -- 172.21.15.45:0/3414800218 learned_addr learned my addr 172.21.15.45:0/3414800218 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2f04c80700 1 --2- 172.21.15.45:0/3414800218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f000cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2efffff700 1 -- 172.21.15.45:0/3414800218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 msgr2=0x7f2f000cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2efffff700 1 --2- 172.21.15.45:0/3414800218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f000cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2efffff700 1 -- 172.21.15.45:0/3414800218 >> v1:172.21.15.45:6789/0 conn(0x7f2f000c3db0 legacy=0x7f2f000697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2efffff700 1 -- 172.21.15.45:0/3414800218 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00159880 con 0x7f2f000b0610 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2efeffd700 1 -- 172.21.15.45:0/3414800218 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795629261 0 0) 0x7f2f000ce6a0 con 0x7f2f000c3db0 2023-10-06T05:21:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.193+0000 7f2efffff700 1 --2- 172.21.15.45:0/3414800218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b0610 0x7f2f000c85d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009900 tx=0x7f2ef0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a5365fc295910d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2efeffd700 1 -- 172.21.15.45:0/3414800218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0009ea0 con 0x7f2f000b0610 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2efeffd700 1 -- 172.21.15.45:0/3414800218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef0005c10 con 0x7f2f000b0610 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2efeffd700 1 -- 172.21.15.45:0/3414800218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000b610 con 0x7f2f000b0610 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 -- 172.21.15.45:0/3414800218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b0610 msgr2=0x7f2f000c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 --2- 172.21.15.45:0/3414800218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b0610 0x7f2f000c85d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009900 tx=0x7f2ef0009120 comp rx=0 tx=0).stop 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 -- 172.21.15.45:0/3414800218 shutdown_connections 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 --2- 172.21.15.45:0/3414800218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f000cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 --2- 172.21.15.45:0/3414800218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b0610 0x7f2f000c85d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 -- 172.21.15.45:0/3414800218 >> 172.21.15.45:0/3414800218 conn(0x7f2f00067e00 msgr2=0x7f2f000cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 -- 172.21.15.45:0/3414800218 shutdown_connections 2023-10-06T05:21:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.194+0000 7f2f066e3700 1 -- 172.21.15.45:0/3414800218 wait complete. 2023-10-06T05:21:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.196+0000 7f2f066e3700 1 Processor -- start 2023-10-06T05:21:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.196+0000 7f2f066e3700 1 -- start start 2023-10-06T05:21:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f066e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f0015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f066e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f000b0610 0x7f2f000b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f066e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000c3db0 0x7f2f000b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f066e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0009ba70 con 0x7f2f000c3db0 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f066e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0015adc0 con 0x7f2f000b0610 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f066e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0015b0a0 con 0x7f2f00069ed0 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000c3db0 0x7f2f000b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000c3db0 0x7f2f000b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36410/0 (socket says 172.21.15.45:36410) 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 -- 172.21.15.45:0/4098938824 learned_addr learned my addr 172.21.15.45:0/4098938824 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2efffff700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f0015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2eff7fe700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f000b0610 0x7f2f000b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 -- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 msgr2=0x7f2f0015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f0015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 -- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f000b0610 msgr2=0x7f2f000b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f000b0610 0x7f2f000b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 -- 172.21.15.45:0/4098938824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00093c0 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2eff7fe700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f000b0610 0x7f2f000b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f04c80700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000c3db0 0x7f2f000b20f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009fa0 tx=0x7f2ef8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2ef7fff700 1 -- 172.21.15.45:0/4098938824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800ed20 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2ef7fff700 1 -- 172.21.15.45:0/4098938824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef800ee80 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0015b380 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.197+0000 7f2ef7fff700 1 -- 172.21.15.45:0/4098938824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8014670 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.198+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0015b940 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.199+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0006e350 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.199+0000 7f2ef7fff700 1 -- 172.21.15.45:0/4098938824 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ef80149e0 con 0x7f2f000c3db0 2023-10-06T05:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.199+0000 7f2ef7fff700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ee0066f30 0x7f2ee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.200+0000 7f2ef7fff700 1 -- 172.21.15.45:0/4098938824 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f2ef8084c70 con 0x7f2f000c3db0 2023-10-06T05:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.200+0000 7f2efffff700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ee0066f30 0x7f2ee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.200+0000 7f2efffff700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ee0066f30 0x7f2ee00693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009160 tx=0x7f2ef0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.204+0000 7f2ef7fff700 1 -- 172.21.15.45:0/4098938824 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ef8051f90 con 0x7f2f000c3db0 2023-10-06T05:21:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.407+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2f0009bbc0 con 0x7f2f000c3db0 2023-10-06T05:21:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.407+0000 7f2ef7fff700 1 -- 172.21.15.45:0/4098938824 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2ef80555b0 con 0x7f2f000c3db0 2023-10-06T05:21:48.409 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:21:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.409+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ee0066f30 msgr2=0x7f2ee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ee0066f30 0x7f2ee00693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009160 tx=0x7f2ef0016000 comp rx=0 tx=0).stop 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000c3db0 msgr2=0x7f2f000b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000c3db0 0x7f2f000b20f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009fa0 tx=0x7f2ef8009900 comp rx=0 tx=0).stop 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 shutdown_connections 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ee0066f30 0x7f2ee00693e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000c3db0 0x7f2f000b20f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f00069ed0 0x7f2f0015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 --2- 172.21.15.45:0/4098938824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f000b0610 0x7f2f000b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 >> 172.21.15.45:0/4098938824 conn(0x7f2f00067e00 msgr2=0x7f2f000c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 shutdown_connections 2023-10-06T05:21:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.410+0000 7f2f066e3700 1 -- 172.21.15.45:0/4098938824 wait complete. 2023-10-06T05:21:48.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:21:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.556+0000 7fcad82fd700 1 Processor -- start 2023-10-06T05:21:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.556+0000 7fcad82fd700 1 -- start start 2023-10-06T05:21:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad82fd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcad0155cd0 0x7fcad015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad82fd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcad00ccdc0 0x7fcad00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad82fd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad00cd700 con 0x7fcad01560a0 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad82fd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad00ae6b0 con 0x7fcad0155cd0 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad82fd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad00ae8d0 con 0x7fcad00ccdc0 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad6099700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcad0155cd0 0x7fcad015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad6099700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcad0155cd0 0x7fcad015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36422/0 (socket says 172.21.15.45:36422) 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad6099700 1 -- 172.21.15.45:0/536977890 learned_addr learned my addr 172.21.15.45:0/536977890 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.557+0000 7fcad689a700 1 --2- 172.21.15.45:0/536977890 >> v2:172.21.15.144:3300/0 conn(0x7fcad00ccdc0 0x7fcad00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad6099700 1 -- 172.21.15.45:0/536977890 >> v2:172.21.15.144:3300/0 conn(0x7fcad00ccdc0 msgr2=0x7fcad00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad6099700 1 --2- 172.21.15.45:0/536977890 >> v2:172.21.15.144:3300/0 conn(0x7fcad00ccdc0 0x7fcad00cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad6099700 1 -- 172.21.15.45:0/536977890 >> v1:172.21.15.144:6789/0 conn(0x7fcad01560a0 legacy=0x7fcad00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad6099700 1 -- 172.21.15.45:0/536977890 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad00aebb0 con 0x7fcad0155cd0 2023-10-06T05:21:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad6099700 1 --2- 172.21.15.45:0/536977890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad015a730 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fcad00c3dd0 tx=0x7fcacc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd169ce1dafbe01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad5097700 1 -- 172.21.15.45:0/536977890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00dba0 con 0x7fcad0155cd0 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad5097700 1 -- 172.21.15.45:0/536977890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc00dd00 con 0x7fcad0155cd0 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.558+0000 7fcad5097700 1 -- 172.21.15.45:0/536977890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0196d0 con 0x7fcad0155cd0 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 -- 172.21.15.45:0/536977890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 msgr2=0x7fcad015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 --2- 172.21.15.45:0/536977890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad015a730 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fcad00c3dd0 tx=0x7fcacc00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 -- 172.21.15.45:0/536977890 shutdown_connections 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 --2- 172.21.15.45:0/536977890 >> v2:172.21.15.144:3300/0 conn(0x7fcad00ccdc0 0x7fcad00cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 --2- 172.21.15.45:0/536977890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad015a730 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 -- 172.21.15.45:0/536977890 >> 172.21.15.45:0/536977890 conn(0x7fcad006f090 msgr2=0x7fcad006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 -- 172.21.15.45:0/536977890 shutdown_connections 2023-10-06T05:21:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.559+0000 7fcad82fd700 1 -- 172.21.15.45:0/536977890 wait complete. 2023-10-06T05:21:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.560+0000 7fcad82fd700 1 Processor -- start 2023-10-06T05:21:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.560+0000 7fcad82fd700 1 -- start start 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad82fd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad00ccdc0 0x7fcad014efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad82fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad0151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad82fd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad01560a0 0x7fcad00b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad82fd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad014f500 con 0x7fcad0155cd0 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad82fd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00afd10 con 0x7fcad00ccdc0 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad82fd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00afff0 con 0x7fcad01560a0 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad0151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad0151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36430/0 (socket says 172.21.15.45:36430) 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 -- 172.21.15.45:0/14140330 learned_addr learned my addr 172.21.15.45:0/14140330 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad6099700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad00ccdc0 0x7fcad014efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad689a700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad01560a0 0x7fcad00b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 -- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad01560a0 msgr2=0x7fcad00b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad01560a0 0x7fcad00b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 -- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad00ccdc0 msgr2=0x7fcad014efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad00ccdc0 0x7fcad014efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 -- 172.21.15.45:0/14140330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc00b790 con 0x7fcad0155cd0 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.561+0000 7fcad5898700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad0151990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c4e0 tx=0x7fcac000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.562+0000 7fcac67fc700 1 -- 172.21.15.45:0/14140330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0011590 con 0x7fcad0155cd0 2023-10-06T05:21:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.562+0000 7fcac67fc700 1 -- 172.21.15.45:0/14140330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac0011710 con 0x7fcad0155cd0 2023-10-06T05:21:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.562+0000 7fcac67fc700 1 -- 172.21.15.45:0/14140330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0011ad0 con 0x7fcad0155cd0 2023-10-06T05:21:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.562+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad00b02d0 con 0x7fcad0155cd0 2023-10-06T05:21:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.562+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad00b0890 con 0x7fcad0155cd0 2023-10-06T05:21:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.563+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad00adfa0 con 0x7fcad0155cd0 2023-10-06T05:21:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.564+0000 7fcac67fc700 1 -- 172.21.15.45:0/14140330 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcac001b3b0 con 0x7fcad0155cd0 2023-10-06T05:21:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.564+0000 7fcac67fc700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcab4066f30 0x7fcab40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.564+0000 7fcac67fc700 1 -- 172.21.15.45:0/14140330 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fcac0083d70 con 0x7fcad0155cd0 2023-10-06T05:21:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.564+0000 7fcad6099700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcab4066f30 0x7fcab40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.565+0000 7fcad6099700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcab4066f30 0x7fcab40693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcad00c3dd0 tx=0x7fcacc01b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.569+0000 7fcac67fc700 1 -- 172.21.15.45:0/14140330 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcac0051110 con 0x7fcad0155cd0 2023-10-06T05:21:48.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.777+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcad00b0ab0 con 0x7fcad0155cd0 2023-10-06T05:21:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.777+0000 7fcac67fc700 1 -- 172.21.15.45:0/14140330 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) v1 ==== 84+0+126 (secure 0 0 0) 0x7fcac0017090 con 0x7fcad0155cd0 2023-10-06T05:21:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:48.779 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:21:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:48.780 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:21:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcab4066f30 msgr2=0x7fcab40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcab4066f30 0x7fcab40693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcad00c3dd0 tx=0x7fcacc01b000 comp rx=0 tx=0).stop 2023-10-06T05:21:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 msgr2=0x7fcad0151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad0151990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c4e0 tx=0x7fcac000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 shutdown_connections 2023-10-06T05:21:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcab4066f30 0x7fcab40693e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad01560a0 0x7fcad00b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad00ccdc0 0x7fcad014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 --2- 172.21.15.45:0/14140330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad0155cd0 0x7fcad0151990 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.779+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 >> 172.21.15.45:0/14140330 conn(0x7fcad006f090 msgr2=0x7fcad00cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.780+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 shutdown_connections 2023-10-06T05:21:48.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.780+0000 7fcad82fd700 1 -- 172.21.15.45:0/14140330 wait complete. 2023-10-06T05:21:48.795 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:21:48.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:21:48.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.930+0000 7fe0e3732700 1 Processor -- start 2023-10-06T05:21:48.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.930+0000 7fe0e3732700 1 -- start start 2023-10-06T05:21:48.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e3732700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe0dc15d590 0x7fe0dc0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e3732700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0dc0c91f0 0x7fe0dc167290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e3732700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc0cc4b0 con 0x7fe0dc0cacb0 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e3732700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc0cb130 con 0x7fe0dc0c91f0 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e3732700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc0c9730 con 0x7fe0dc15d590 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e1ccf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0dc0c91f0 0x7fe0dc167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e1ccf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0dc0c91f0 0x7fe0dc167290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36438/0 (socket says 172.21.15.45:36438) 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/2280320652 learned_addr learned my addr 172.21.15.45:0/2280320652 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e14ce700 1 --2- 172.21.15.45:0/2280320652 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc15d590 0x7fe0dc0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/2280320652 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc15d590 msgr2=0x7fe0dc0ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e1ccf700 1 --2- 172.21.15.45:0/2280320652 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc15d590 0x7fe0dc0ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/2280320652 >> v1:172.21.15.144:6789/0 conn(0x7fe0dc0cacb0 legacy=0x7fe0dc0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.931+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/2280320652 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0c9a10 con 0x7fe0dc0c91f0 2023-10-06T05:21:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e1ccf700 1 --2- 172.21.15.45:0/2280320652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc0c91f0 0x7fe0dc167290 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc06fc30 tx=0x7fe0d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06635574ad9f3ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0d3fff700 1 -- 172.21.15.45:0/2280320652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400e070 con 0x7fe0dc0c91f0 2023-10-06T05:21:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0d3fff700 1 -- 172.21.15.45:0/2280320652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d400ba20 con 0x7fe0dc0c91f0 2023-10-06T05:21:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0d3fff700 1 -- 172.21.15.45:0/2280320652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40092e0 con 0x7fe0dc0c91f0 2023-10-06T05:21:48.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 -- 172.21.15.45:0/2280320652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc0c91f0 msgr2=0x7fe0dc167290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 --2- 172.21.15.45:0/2280320652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc0c91f0 0x7fe0dc167290 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc06fc30 tx=0x7fe0d4009840 comp rx=0 tx=0).stop 2023-10-06T05:21:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 -- 172.21.15.45:0/2280320652 shutdown_connections 2023-10-06T05:21:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 --2- 172.21.15.45:0/2280320652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc0c91f0 0x7fe0dc167290 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 --2- 172.21.15.45:0/2280320652 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc15d590 0x7fe0dc0ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 -- 172.21.15.45:0/2280320652 >> 172.21.15.45:0/2280320652 conn(0x7fe0dc0680b0 msgr2=0x7fe0dc0cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 -- 172.21.15.45:0/2280320652 shutdown_connections 2023-10-06T05:21:48.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.932+0000 7fe0e3732700 1 -- 172.21.15.45:0/2280320652 wait complete. 2023-10-06T05:21:48.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.934+0000 7fe0e3732700 1 Processor -- start 2023-10-06T05:21:48.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.934+0000 7fe0e3732700 1 -- start start 2023-10-06T05:21:48.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e3732700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0c91f0 0x7fe0dc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e3732700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc0cacb0 0x7fe0dc1624d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e3732700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc15d590 0x7fe0dc14c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e3732700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc162a10 con 0x7fe0dc15d590 2023-10-06T05:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e3732700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0b1ee0 con 0x7fe0dc0c91f0 2023-10-06T05:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e3732700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0b12d0 con 0x7fe0dc0cacb0 2023-10-06T05:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc15d590 0x7fe0dc14c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc15d590 0x7fe0dc14c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36454/0 (socket says 172.21.15.45:36454) 2023-10-06T05:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/1445709581 learned_addr learned my addr 172.21.15.45:0/1445709581 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e14ce700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0c91f0 0x7fe0dc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e0ccd700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc0cacb0 0x7fe0dc1624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc0cacb0 msgr2=0x7fe0dc1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc0cacb0 0x7fe0dc1624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0c91f0 msgr2=0x7fe0dc0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0c91f0 0x7fe0dc0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 -- 172.21.15.45:0/1445709581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4009ae0 con 0x7fe0dc15d590 2023-10-06T05:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0e1ccf700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc15d590 0x7fe0dc14c0b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009880 tx=0x7fe0d4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.935+0000 7fe0d17fa700 1 -- 172.21.15.45:0/1445709581 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4008a80 con 0x7fe0dc15d590 2023-10-06T05:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.936+0000 7fe0d17fa700 1 -- 172.21.15.45:0/1445709581 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4008be0 con 0x7fe0dc15d590 2023-10-06T05:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.936+0000 7fe0d17fa700 1 -- 172.21.15.45:0/1445709581 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4025460 con 0x7fe0dc15d590 2023-10-06T05:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.936+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc0b15b0 con 0x7fe0dc15d590 2023-10-06T05:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.936+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc0b1b70 con 0x7fe0dc15d590 2023-10-06T05:21:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.937+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc04c380 con 0x7fe0dc15d590 2023-10-06T05:21:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.937+0000 7fe0d17fa700 1 -- 172.21.15.45:0/1445709581 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0d4003040 con 0x7fe0dc15d590 2023-10-06T05:21:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.938+0000 7fe0d17fa700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe0c0066f30 0x7fe0c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.938+0000 7fe0d17fa700 1 -- 172.21.15.45:0/1445709581 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fe0d4085a50 con 0x7fe0dc15d590 2023-10-06T05:21:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.938+0000 7fe0e14ce700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe0c0066f30 0x7fe0c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.938+0000 7fe0e14ce700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe0c0066f30 0x7fe0c00693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009db0 tx=0x7fe0cc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:48.942+0000 7fe0d17fa700 1 -- 172.21.15.45:0/1445709581 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0d4052e20 con 0x7fe0dc15d590 2023-10-06T05:21:49.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.149+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe0dc0b2020 con 0x7fe0dc15d590 2023-10-06T05:21:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.320+0000 7fe0d17fa700 1 -- 172.21.15.45:0/1445709581 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe0d4056440 con 0x7fe0dc15d590 2023-10-06T05:21:49.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe0c0066f30 msgr2=0x7fe0c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe0c0066f30 0x7fe0c00693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009db0 tx=0x7fe0cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc15d590 msgr2=0x7fe0dc14c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc15d590 0x7fe0dc14c0b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009880 tx=0x7fe0d4018000 comp rx=0 tx=0).stop 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 shutdown_connections 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe0c0066f30 0x7fe0c00693e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0dc15d590 0x7fe0dc14c0b0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0c91f0 0x7fe0dc0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 --2- 172.21.15.45:0/1445709581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc0cacb0 0x7fe0dc1624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 >> 172.21.15.45:0/1445709581 conn(0x7fe0dc0680b0 msgr2=0x7fe0dc159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.326+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 shutdown_connections 2023-10-06T05:21:49.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.327+0000 7fe0e3732700 1 -- 172.21.15.45:0/1445709581 wait complete. 2023-10-06T05:21:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-06T05:21:49.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.485+0000 7f9dd18f6700 1 Processor -- start 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.485+0000 7f9dd18f6700 1 -- start start 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dd18f6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9dcc0d02b0 0x7f9dcc0d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dd18f6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc15d820 0x7f9dcc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dd18f6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9dcc15c6f0 0x7f9dcc0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dd18f6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc0cd740 con 0x7f9dcc15c6f0 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dd18f6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc0ccb30 con 0x7f9dcc0d02b0 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dd18f6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0cce10 con 0x7f9dcc15d820 2023-10-06T05:21:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dcb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9dcc15c6f0 0x7f9dcc0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dcb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9dcc15c6f0 0x7f9dcc0afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36460/0 (socket says 172.21.15.45:36460) 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/2202090766 learned_addr learned my addr 172.21.15.45:0/2202090766 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/2202090766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc15d820 0x7f9dcc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.486+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/2202090766 >> v2:172.21.15.144:3300/0 conn(0x7f9dcc0d02b0 0x7f9dcc0d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/2202090766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc15d820 msgr2=0x7f9dcc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/2202090766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc15d820 0x7f9dcc15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/2202090766 >> v2:172.21.15.144:3300/0 conn(0x7f9dcc0d02b0 msgr2=0x7f9dcc0d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/2202090766 >> v2:172.21.15.144:3300/0 conn(0x7f9dcc0d02b0 0x7f9dcc0d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/2202090766 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0cd0f0 con 0x7f9dcc15c6f0 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/2202090766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15c6f0 0x7f9dcc0afd40 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009f20 tx=0x7f9dc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2975c517bfe6ad1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dc9ffb700 1 -- 172.21.15.45:0/2202090766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400e070 con 0x7f9dcc15c6f0 2023-10-06T05:21:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dc9ffb700 1 -- 172.21.15.45:0/2202090766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc400b930 con 0x7f9dcc15c6f0 2023-10-06T05:21:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.487+0000 7f9dc9ffb700 1 -- 172.21.15.45:0/2202090766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40092e0 con 0x7f9dcc15c6f0 2023-10-06T05:21:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 -- 172.21.15.45:0/2202090766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15c6f0 msgr2=0x7f9dcc0afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/2202090766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15c6f0 0x7f9dcc0afd40 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009f20 tx=0x7f9dc4009840 comp rx=0 tx=0).stop 2023-10-06T05:21:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 -- 172.21.15.45:0/2202090766 shutdown_connections 2023-10-06T05:21:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/2202090766 >> v2:172.21.15.144:3300/0 conn(0x7f9dcc0d02b0 0x7f9dcc0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/2202090766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15c6f0 0x7f9dcc0afd40 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/2202090766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc15d820 0x7f9dcc15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 -- 172.21.15.45:0/2202090766 >> 172.21.15.45:0/2202090766 conn(0x7f9dcc06ed50 msgr2=0x7f9dcc06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 -- 172.21.15.45:0/2202090766 shutdown_connections 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.488+0000 7f9dd18f6700 1 -- 172.21.15.45:0/2202090766 wait complete. 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.489+0000 7f9dd18f6700 1 Processor -- start 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.489+0000 7f9dd18f6700 1 -- start start 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dd18f6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc0d02b0 0x7f9dcc159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dd18f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dcc15c6f0 0x7f9dcc0a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dd18f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15d820 0x7f9dcc169390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dd18f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0b05e0 con 0x7f9dcc15d820 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dd18f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0c85c0 con 0x7f9dcc15c6f0 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dd18f6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0c88a0 con 0x7f9dcc0d02b0 2023-10-06T05:21:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15d820 0x7f9dcc169390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15d820 0x7f9dcc169390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36468/0 (socket says 172.21.15.45:36468) 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/3128669499 learned_addr learned my addr 172.21.15.45:0/3128669499 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dca7fc700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dcc15c6f0 0x7f9dcc0a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.490+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc0d02b0 0x7f9dcc159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc0d02b0 msgr2=0x7f9dcc159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc0d02b0 0x7f9dcc159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dcc15c6f0 msgr2=0x7f9dcc0a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dcc15c6f0 0x7f9dcc0a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dcb7fe700 1 -- 172.21.15.45:0/3128669499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009ae0 con 0x7f9dcc15d820 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dcb7fe700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15d820 0x7f9dcc169390 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009880 tx=0x7f9dc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9db2ffd700 1 -- 172.21.15.45:0/3128669499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4008ad0 con 0x7f9dcc15d820 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9db2ffd700 1 -- 172.21.15.45:0/3128669499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4008c70 con 0x7f9dcc15d820 2023-10-06T05:21:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc0c8b80 con 0x7f9dcc15d820 2023-10-06T05:21:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9db2ffd700 1 -- 172.21.15.45:0/3128669499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4026460 con 0x7f9dcc15d820 2023-10-06T05:21:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.491+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc0c9140 con 0x7f9dcc15d820 2023-10-06T05:21:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.492+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc088730 con 0x7f9dcc15d820 2023-10-06T05:21:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.493+0000 7f9db2ffd700 1 -- 172.21.15.45:0/3128669499 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9dc4003040 con 0x7f9dcc15d820 2023-10-06T05:21:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.493+0000 7f9db2ffd700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9db8066f30 0x7f9db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.493+0000 7f9db2ffd700 1 -- 172.21.15.45:0/3128669499 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f9dc4085e50 con 0x7f9dcc15d820 2023-10-06T05:21:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.493+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9db8066f30 0x7f9db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.494+0000 7f9dcaffd700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9db8066f30 0x7f9db80693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc15a9c0 tx=0x7f9dbc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.498+0000 7f9db2ffd700 1 -- 172.21.15.45:0/3128669499 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dc4053000 con 0x7f9dcc15d820 2023-10-06T05:21:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:49.699+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9dcc0cd3d0 con 0x7f9dcc15d820 2023-10-06T05:21:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.326+0000 7f9db2ffd700 1 -- 172.21.15.45:0/3128669499 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9dc4056620 con 0x7f9dcc15d820 2023-10-06T05:21:50.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.330+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9db8066f30 msgr2=0x7f9db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.330+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9db8066f30 0x7f9db80693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc15a9c0 tx=0x7f9dbc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.330+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15d820 msgr2=0x7f9dcc169390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.330+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15d820 0x7f9dcc169390 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009880 tx=0x7f9dc4018000 comp rx=0 tx=0).stop 2023-10-06T05:21:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.334+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 shutdown_connections 2023-10-06T05:21:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.334+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9db8066f30 0x7f9db80693e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.334+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dcc15d820 0x7f9dcc169390 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.334+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dcc0d02b0 0x7f9dcc159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.334+0000 7f9dd18f6700 1 --2- 172.21.15.45:0/3128669499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dcc15c6f0 0x7f9dcc0a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.334+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 >> 172.21.15.45:0/3128669499 conn(0x7f9dcc06ed50 msgr2=0x7f9dcc15b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.335+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 shutdown_connections 2023-10-06T05:21:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.337+0000 7f9dd18f6700 1 -- 172.21.15.45:0/3128669499 wait complete. 2023-10-06T05:21:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-06T05:21:50.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:21:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.501+0000 7f5ecbf50700 1 Processor -- start 2023-10-06T05:21:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.501+0000 7f5ecbf50700 1 -- start start 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ecbf50700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec40c3db0 0x7f5ec40c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ecbf50700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec40ce6a0 con 0x7f5ec40b0610 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ecbf50700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec4159880 con 0x7f5ec4069ed0 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ecbf50700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40690f0 con 0x7f5ec40c3db0 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ec9cec700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5ec40b0610 0x7f5ec40697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39140/0 (socket says 172.21.15.45:39140) 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ec9cec700 1 -- 172.21.15.45:0/1052647081 learned_addr learned my addr 172.21.15.45:0/1052647081 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ec94eb700 1 --2- 172.21.15.45:0/1052647081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec40c3db0 0x7f5ec40c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.502+0000 7f5ec8cea700 1 -- 172.21.15.45:0/1052647081 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160301025 0 0) 0x7f5ec40ce6a0 con 0x7f5ec40b0610 2023-10-06T05:21:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec8cea700 1 -- 172.21.15.45:0/1052647081 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb0003570 con 0x7f5ec40b0610 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec8cea700 1 -- 172.21.15.45:0/1052647081 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556769389 0 0) 0x7f5ec4159880 con 0x7f5ec4069ed0 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec8cea700 1 -- 172.21.15.45:0/1052647081 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ec40ce6a0 con 0x7f5ec4069ed0 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec94eb700 1 -- 172.21.15.45:0/1052647081 >> v1:172.21.15.144:6789/0 conn(0x7f5ec4069ed0 legacy=0x7f5ec40cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec94eb700 1 -- 172.21.15.45:0/1052647081 >> v1:172.21.15.45:6789/0 conn(0x7f5ec40b0610 legacy=0x7f5ec40697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec94eb700 1 -- 172.21.15.45:0/1052647081 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec40692e0 con 0x7f5ec40c3db0 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec94eb700 1 --2- 172.21.15.45:0/1052647081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec40c3db0 0x7f5ec40c85d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009910 tx=0x7f5eb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c93b26816fed3bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec8cea700 1 -- 172.21.15.45:0/1052647081 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8011630 con 0x7f5ec40c3db0 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec8cea700 1 -- 172.21.15.45:0/1052647081 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8011790 con 0x7f5ec40c3db0 2023-10-06T05:21:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.503+0000 7f5ec8cea700 1 -- 172.21.15.45:0/1052647081 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8011b10 con 0x7f5ec40c3db0 2023-10-06T05:21:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.504+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1052647081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec40c3db0 msgr2=0x7f5ec40c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.504+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1052647081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec40c3db0 0x7f5ec40c85d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009910 tx=0x7f5eb80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.504+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1052647081 shutdown_connections 2023-10-06T05:21:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.504+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1052647081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec40c3db0 0x7f5ec40c85d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.504+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1052647081 >> 172.21.15.45:0/1052647081 conn(0x7f5ec4067e00 msgr2=0x7f5ec40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.504+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1052647081 shutdown_connections 2023-10-06T05:21:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.504+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1052647081 wait complete. 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.505+0000 7f5ecbf50700 1 Processor -- start 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.505+0000 7f5ecbf50700 1 -- start start 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ecbf50700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec4069ed0 0x7f5ec4143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ecbf50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec40b0610 0x7f5ec415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ecbf50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec40c3db0 0x7f5ec40b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ecbf50700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40b2e60 con 0x7f5ec40b0610 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ecbf50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec4157350 con 0x7f5ec40c3db0 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ecbf50700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40b18d0 con 0x7f5ec4069ed0 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec40b0610 0x7f5ec415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec40b0610 0x7f5ec415bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36484/0 (socket says 172.21.15.45:36484) 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 -- 172.21.15.45:0/1089695451 learned_addr learned my addr 172.21.15.45:0/1089695451 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec9cec700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec4069ed0 0x7f5ec4143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5eca4ed700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec40c3db0 0x7f5ec40b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 -- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec4069ed0 msgr2=0x7f5ec4143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec4069ed0 0x7f5ec4143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 -- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec40c3db0 msgr2=0x7f5ec40b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec40c3db0 0x7f5ec40b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.506+0000 7f5ec94eb700 1 -- 172.21.15.45:0/1089695451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb8009420 con 0x7f5ec40b0610 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.507+0000 7f5ec94eb700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec40b0610 0x7f5ec415bc10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5ec406bac0 tx=0x7f5eb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.507+0000 7f5eb5ffb700 1 -- 172.21.15.45:0/1089695451 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8011900 con 0x7f5ec40b0610 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.507+0000 7f5eb5ffb700 1 -- 172.21.15.45:0/1089695451 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8011a60 con 0x7f5ec40b0610 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.507+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec40b1bb0 con 0x7f5ec40b0610 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.507+0000 7f5eb5ffb700 1 -- 172.21.15.45:0/1089695451 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8025640 con 0x7f5ec40b0610 2023-10-06T05:21:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.507+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec40b2af0 con 0x7f5ec40b0610 2023-10-06T05:21:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.508+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec40b2fd0 con 0x7f5ec40b0610 2023-10-06T05:21:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.509+0000 7f5eb5ffb700 1 -- 172.21.15.45:0/1089695451 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5eb8025a30 con 0x7f5ec40b0610 2023-10-06T05:21:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.509+0000 7f5eb5ffb700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ea8066f30 0x7f5ea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.510+0000 7f5eb5ffb700 1 -- 172.21.15.45:0/1089695451 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f5eb8085ea0 con 0x7f5ec40b0610 2023-10-06T05:21:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.510+0000 7f5ec9cec700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ea8066f30 0x7f5ea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.510+0000 7f5ec9cec700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ea8066f30 0x7f5ea80693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40cdf20 tx=0x7f5ec0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.514+0000 7f5eb5ffb700 1 -- 172.21.15.45:0/1089695451 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5eb8052e30 con 0x7f5ec40b0610 2023-10-06T05:21:50.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:50.761+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5ec406e350 con 0x7f5ec40b0610 2023-10-06T05:21:51.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.331+0000 7f5eb5ffb700 1 -- 172.21.15.45:0/1089695451 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5eb80150c0 con 0x7f5ec40b0610 2023-10-06T05:21:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ea8066f30 msgr2=0x7f5ea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ea8066f30 0x7f5ea80693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40cdf20 tx=0x7f5ec0009040 comp rx=0 tx=0).stop 2023-10-06T05:21:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec40b0610 msgr2=0x7f5ec415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec40b0610 0x7f5ec415bc10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5ec406bac0 tx=0x7f5eb8017000 comp rx=0 tx=0).stop 2023-10-06T05:21:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 shutdown_connections 2023-10-06T05:21:51.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ea8066f30 0x7f5ea80693e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec40c3db0 0x7f5ec40b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ec4069ed0 0x7f5ec4143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 --2- 172.21.15.45:0/1089695451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec40b0610 0x7f5ec415bc10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.336+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 >> 172.21.15.45:0/1089695451 conn(0x7f5ec4067e00 msgr2=0x7f5ec40cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.337+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 shutdown_connections 2023-10-06T05:21:51.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.337+0000 7f5ecbf50700 1 -- 172.21.15.45:0/1089695451 wait complete. 2023-10-06T05:21:51.341 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 14 and data pool 15 2023-10-06T05:21:51.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:51.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.490+0000 7f090cb82700 1 Processor -- start 2023-10-06T05:21:51.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.490+0000 7f090cb82700 1 -- start start 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090cb82700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0908155cd0 0x7f090815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090cb82700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f09080adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090cb82700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09080ccdc0 0x7f09080cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090cb82700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09080cd740 con 0x7f09080ccdc0 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090cb82700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09080ae8d0 con 0x7f0908155cd0 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090cb82700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09080aebb0 con 0x7f09081560a0 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09080ccdc0 0x7f09080cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09080ccdc0 0x7f09080cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36492/0 (socket says 172.21.15.45:36492) 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 -- 172.21.15.45:0/2304064896 learned_addr learned my addr 172.21.15.45:0/2304064896 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0905d9b700 1 --2- 172.21.15.45:0/2304064896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f09080adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090659c700 1 --2- 172.21.15.45:0/2304064896 >> v2:172.21.15.144:3300/0 conn(0x7f0908155cd0 0x7f090815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 -- 172.21.15.45:0/2304064896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 msgr2=0x7f09080adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 --2- 172.21.15.45:0/2304064896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f09080adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 -- 172.21.15.45:0/2304064896 >> v2:172.21.15.144:3300/0 conn(0x7f0908155cd0 msgr2=0x7f090815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 --2- 172.21.15.45:0/2304064896 >> v2:172.21.15.144:3300/0 conn(0x7f0908155cd0 0x7f090815a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0906d9d700 1 -- 172.21.15.45:0/2304064896 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09080aee90 con 0x7f09080ccdc0 2023-10-06T05:21:51.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f0905d9b700 1 --2- 172.21.15.45:0/2304064896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f09080adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.491+0000 7f090659c700 1 --2- 172.21.15.45:0/2304064896 >> v2:172.21.15.144:3300/0 conn(0x7f0908155cd0 0x7f090815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f0906d9d700 1 --2- 172.21.15.45:0/2304064896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09080ccdc0 0x7f09080cd200 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f090806ae50 tx=0x7f090000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fb3f1a73ec1fa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090559a700 1 -- 172.21.15.45:0/2304064896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900010070 con 0x7f09080ccdc0 2023-10-06T05:21:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090559a700 1 -- 172.21.15.45:0/2304064896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090000de90 con 0x7f09080ccdc0 2023-10-06T05:21:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090559a700 1 -- 172.21.15.45:0/2304064896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090000b320 con 0x7f09080ccdc0 2023-10-06T05:21:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090cb82700 1 -- 172.21.15.45:0/2304064896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09080ccdc0 msgr2=0x7f09080cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090cb82700 1 --2- 172.21.15.45:0/2304064896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09080ccdc0 0x7f09080cd200 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f090806ae50 tx=0x7f090000ba50 comp rx=0 tx=0).stop 2023-10-06T05:21:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090cb82700 1 -- 172.21.15.45:0/2304064896 shutdown_connections 2023-10-06T05:21:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090cb82700 1 --2- 172.21.15.45:0/2304064896 >> v2:172.21.15.144:3300/0 conn(0x7f0908155cd0 0x7f090815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090cb82700 1 --2- 172.21.15.45:0/2304064896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09080ccdc0 0x7f09080cd200 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090cb82700 1 --2- 172.21.15.45:0/2304064896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f09080adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.492+0000 7f090cb82700 1 -- 172.21.15.45:0/2304064896 >> 172.21.15.45:0/2304064896 conn(0x7f090806f090 msgr2=0x7f090806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.493+0000 7f090cb82700 1 -- 172.21.15.45:0/2304064896 shutdown_connections 2023-10-06T05:21:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.493+0000 7f090cb82700 1 -- 172.21.15.45:0/2304064896 wait complete. 2023-10-06T05:21:51.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.494+0000 7f090cb82700 1 Processor -- start 2023-10-06T05:21:51.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.494+0000 7f090cb82700 1 -- start start 2023-10-06T05:21:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090cb82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09080ccdc0 0x7f09080c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090cb82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908155cd0 0x7f09080c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090cb82700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f0908153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090cb82700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09080c6a60 con 0x7f0908155cd0 2023-10-06T05:21:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090cb82700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09080afd80 con 0x7f09080ccdc0 2023-10-06T05:21:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090cb82700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09080b0060 con 0x7f09081560a0 2023-10-06T05:21:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908155cd0 0x7f09080c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908155cd0 0x7f09080c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36496/0 (socket says 172.21.15.45:36496) 2023-10-06T05:21:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 -- 172.21.15.45:0/3937475174 learned_addr learned my addr 172.21.15.45:0/3937475174 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0906d9d700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f0908153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090659c700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09080ccdc0 0x7f09080c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 -- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 msgr2=0x7f0908153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f0908153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 -- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09080ccdc0 msgr2=0x7f09080c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09080ccdc0 0x7f09080c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 -- 172.21.15.45:0/3937475174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090000bcf0 con 0x7f0908155cd0 2023-10-06T05:21:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f0905d9b700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908155cd0 0x7f09080c4850 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c850 tx=0x7f08f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f08f67fc700 1 -- 172.21.15.45:0/3937475174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8011600 con 0x7f0908155cd0 2023-10-06T05:21:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f08f67fc700 1 -- 172.21.15.45:0/3937475174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8011760 con 0x7f0908155cd0 2023-10-06T05:21:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.495+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09080b0340 con 0x7f0908155cd0 2023-10-06T05:21:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.496+0000 7f08f67fc700 1 -- 172.21.15.45:0/3937475174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8011b00 con 0x7f0908155cd0 2023-10-06T05:21:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.496+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09080b0900 con 0x7f0908155cd0 2023-10-06T05:21:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.497+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090806e2f0 con 0x7f0908155cd0 2023-10-06T05:21:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.497+0000 7f08f67fc700 1 -- 172.21.15.45:0/3937475174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08f8018350 con 0x7f0908155cd0 2023-10-06T05:21:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.498+0000 7f08f67fc700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08e8066f30 0x7f08e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.498+0000 7f08f67fc700 1 -- 172.21.15.45:0/3937475174 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f08f8084630 con 0x7f0908155cd0 2023-10-06T05:21:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.498+0000 7f090659c700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08e8066f30 0x7f08e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.498+0000 7f090659c700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08e8066f30 0x7f08e80693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00acc0 tx=0x7f08fc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.502+0000 7f08f67fc700 1 -- 172.21.15.45:0/3937475174 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08f8055090 con 0x7f0908155cd0 2023-10-06T05:21:51.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.702+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09080af170 con 0x7f0908155cd0 2023-10-06T05:21:51.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.703+0000 7f08f67fc700 1 -- 172.21.15.45:0/3937475174 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17183 (secure 0 0 0) 0x7f08f8054c20 con 0x7f0908155cd0 2023-10-06T05:21:51.706 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:51.706 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:21:50.762767+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:49.182551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:49.732208+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08e8066f30 msgr2=0x7f08e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08e8066f30 0x7f08e80693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00acc0 tx=0x7f08fc008040 comp rx=0 tx=0).stop 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908155cd0 msgr2=0x7f09080c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908155cd0 0x7f09080c4850 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c850 tx=0x7f08f8008040 comp rx=0 tx=0).stop 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 shutdown_connections 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f08e8066f30 0x7f08e80693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09081560a0 0x7f0908153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09080ccdc0 0x7f09080c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 --2- 172.21.15.45:0/3937475174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908155cd0 0x7f09080c4850 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 >> 172.21.15.45:0/3937475174 conn(0x7f090806f090 msgr2=0x7f0908151330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 shutdown_connections 2023-10-06T05:21:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.705+0000 7f090cb82700 1 -- 172.21.15.45:0/3937475174 wait complete. 2023-10-06T05:21:51.721 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:51.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.851+0000 7f9fff916700 1 Processor -- start 2023-10-06T05:21:51.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.852+0000 7f9fff916700 1 -- start start 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9fff916700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9fff916700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ff81560a0 0x7f9ff80adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9fff916700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff80ae8d0 con 0x7f9ff80ccdc0 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9fff916700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff80aebb0 con 0x7f9ff81560a0 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9fff916700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80aee90 con 0x7f9ff8155cd0 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9ffceb1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ff81560a0 0x7f9ff80adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9ffceb1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ff81560a0 0x7f9ff80adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36506/0 (socket says 172.21.15.45:36506) 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9ffd6b2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9ffceb1700 1 -- 172.21.15.45:0/2523608617 learned_addr learned my addr 172.21.15.45:0/2523608617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9ffceb1700 1 -- 172.21.15.45:0/2523608617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 msgr2=0x7f9ff815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9ffceb1700 1 --2- 172.21.15.45:0/2523608617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff815a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9ffceb1700 1 -- 172.21.15.45:0/2523608617 >> v1:172.21.15.144:6789/0 conn(0x7f9ff80ccdc0 legacy=0x7f9ff80c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.854+0000 7f9ffceb1700 1 -- 172.21.15.45:0/2523608617 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff80af170 con 0x7f9ff81560a0 2023-10-06T05:21:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.853+0000 7f9feffff700 1 -- 172.21.15.45:0/2523608617 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371324210 0 0) 0x7f9ff80ae8d0 con 0x7f9ff80ccdc0 2023-10-06T05:21:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.854+0000 7f9ffceb1700 1 --2- 172.21.15.45:0/2523608617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff80adfa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009f20 tx=0x7f9ff0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541a4849c29a2d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.854+0000 7f9feffff700 1 -- 172.21.15.45:0/2523608617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000e070 con 0x7f9ff81560a0 2023-10-06T05:21:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.854+0000 7f9feffff700 1 -- 172.21.15.45:0/2523608617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff000b9d0 con 0x7f9ff81560a0 2023-10-06T05:21:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.854+0000 7f9feffff700 1 -- 172.21.15.45:0/2523608617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00092e0 con 0x7f9ff81560a0 2023-10-06T05:21:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 -- 172.21.15.45:0/2523608617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 msgr2=0x7f9ff80adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 --2- 172.21.15.45:0/2523608617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff80adfa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009f20 tx=0x7f9ff0009840 comp rx=0 tx=0).stop 2023-10-06T05:21:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 -- 172.21.15.45:0/2523608617 shutdown_connections 2023-10-06T05:21:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 --2- 172.21.15.45:0/2523608617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 --2- 172.21.15.45:0/2523608617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff80adfa0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 -- 172.21.15.45:0/2523608617 >> 172.21.15.45:0/2523608617 conn(0x7f9ff806f090 msgr2=0x7f9ff806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 -- 172.21.15.45:0/2523608617 shutdown_connections 2023-10-06T05:21:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.855+0000 7f9fff916700 1 -- 172.21.15.45:0/2523608617 wait complete. 2023-10-06T05:21:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.856+0000 7f9fff916700 1 Processor -- start 2023-10-06T05:21:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.856+0000 7f9fff916700 1 -- start start 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9fff916700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ff80ccdc0 0x7f9ff8148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9fff916700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff8158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9fff916700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff8158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9fff916700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff8157000 con 0x7f9ff81560a0 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9fff916700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff8158be0 con 0x7f9ff80ccdc0 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9fff916700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff8168220 con 0x7f9ff8155cd0 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff8158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff8158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36516/0 (socket says 172.21.15.45:36516) 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 -- 172.21.15.45:0/3503053853 learned_addr learned my addr 172.21.15.45:0/3503053853 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffd6b2700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ff80ccdc0 0x7f9ff8148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffceb1700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff8158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 -- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 msgr2=0x7f9ff8158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff8158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 -- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ff80ccdc0 msgr2=0x7f9ff8148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ff80ccdc0 0x7f9ff8148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.857+0000 7f9ffdeb3700 1 -- 172.21.15.45:0/3503053853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009ae0 con 0x7f9ff81560a0 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.858+0000 7f9ffdeb3700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff8158540 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400c5d0 tx=0x7f9ff4007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.858+0000 7f9fed7fa700 1 -- 172.21.15.45:0/3503053853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4011590 con 0x7f9ff81560a0 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.858+0000 7f9fed7fa700 1 -- 172.21.15.45:0/3503053853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff40116f0 con 0x7f9ff81560a0 2023-10-06T05:21:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.858+0000 7f9fed7fa700 1 -- 172.21.15.45:0/3503053853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4011ab0 con 0x7f9ff81560a0 2023-10-06T05:21:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.858+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff81566d0 con 0x7f9ff81560a0 2023-10-06T05:21:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.858+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff8156c90 con 0x7f9ff81560a0 2023-10-06T05:21:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.859+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff806e2f0 con 0x7f9ff81560a0 2023-10-06T05:21:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.859+0000 7f9fed7fa700 1 -- 172.21.15.45:0/3503053853 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ff401b3b0 con 0x7f9ff81560a0 2023-10-06T05:21:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.860+0000 7f9fed7fa700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc066f30 0x7f9fdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.860+0000 7f9fed7fa700 1 -- 172.21.15.45:0/3503053853 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9ff4083d40 con 0x7f9ff81560a0 2023-10-06T05:21:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.860+0000 7f9ffd6b2700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc066f30 0x7f9fdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.861+0000 7f9ffd6b2700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc066f30 0x7f9fdc0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80c6690 tx=0x7f9fe80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:51.864+0000 7f9fed7fa700 1 -- 172.21.15.45:0/3503053853 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ff4050c70 con 0x7f9ff81560a0 2023-10-06T05:21:52.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.065+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ff80ae4e0 con 0x7f9ff81560a0 2023-10-06T05:21:52.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.066+0000 7f9fed7fa700 1 -- 172.21.15.45:0/3503053853 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17170 (secure 0 0 0) 0x7f9ff4017070 con 0x7f9ff81560a0 2023-10-06T05:21:52.068 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:52.068 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:21:51.848703+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:49.182551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:49.732208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:52.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.068+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc066f30 msgr2=0x7f9fdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.068+0000 7f9fff916700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc066f30 0x7f9fdc0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80c6690 tx=0x7f9fe80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.068+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 msgr2=0x7f9ff8158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.068+0000 7f9fff916700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff8158540 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400c5d0 tx=0x7f9ff4007d70 comp rx=0 tx=0).stop 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 shutdown_connections 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc066f30 0x7f9fdc0693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff81560a0 0x7f9ff8158540 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ff80ccdc0 0x7f9ff8148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 --2- 172.21.15.45:0/3503053853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ff8155cd0 0x7f9ff8158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 >> 172.21.15.45:0/3503053853 conn(0x7f9ff806f090 msgr2=0x7f9ff80c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 shutdown_connections 2023-10-06T05:21:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.069+0000 7f9fff916700 1 -- 172.21.15.45:0/3503053853 wait complete. 2023-10-06T05:21:52.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:21:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.218+0000 7f9ad369e700 1 Processor -- start 2023-10-06T05:21:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.218+0000 7f9ad369e700 1 -- start start 2023-10-06T05:21:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad369e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9acc0c8b70 0x7f9acc0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad369e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0c3b50 con 0x7f9acc157330 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad369e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc157700 con 0x7f9acc0ccb70 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad369e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0c98e0 con 0x7f9acc0c8b70 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad1c3b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9acc0c8b70 0x7f9acc0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad1c3b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9acc0c8b70 0x7f9acc0c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36520/0 (socket says 172.21.15.45:36520) 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/274768717 learned_addr learned my addr 172.21.15.45:0/274768717 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad143a700 1 --1- 172.21.15.45:0/274768717 >> v1:172.21.15.45:6789/0 conn(0x7f9acc157330 0x7f9acc0ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39156/0 (socket says 172.21.15.45:39156) 2023-10-06T05:21:52.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/274768717 >> v1:172.21.15.144:6789/0 conn(0x7f9acc0ccb70 legacy=0x7f9acc0aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/274768717 >> v1:172.21.15.45:6789/0 conn(0x7f9acc157330 legacy=0x7f9acc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/274768717 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0c9bc0 con 0x7f9acc0c8b70 2023-10-06T05:21:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.219+0000 7f9ac3fff700 1 -- 172.21.15.45:0/274768717 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896745405 0 0) 0x7f9acc0c3b50 con 0x7f9acc157330 2023-10-06T05:21:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ad1c3b700 1 --2- 172.21.15.45:0/274768717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc0c8b70 0x7f9acc0c8f90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009f20 tx=0x7f9ac4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d461b56767dde0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ac3fff700 1 -- 172.21.15.45:0/274768717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e070 con 0x7f9acc0c8b70 2023-10-06T05:21:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ac3fff700 1 -- 172.21.15.45:0/274768717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400ba40 con 0x7f9acc0c8b70 2023-10-06T05:21:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ac3fff700 1 -- 172.21.15.45:0/274768717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40092e0 con 0x7f9acc0c8b70 2023-10-06T05:21:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ad369e700 1 -- 172.21.15.45:0/274768717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc0c8b70 msgr2=0x7f9acc0c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ad369e700 1 --2- 172.21.15.45:0/274768717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc0c8b70 0x7f9acc0c8f90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009f20 tx=0x7f9ac4009840 comp rx=0 tx=0).stop 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ad369e700 1 -- 172.21.15.45:0/274768717 shutdown_connections 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ad369e700 1 --2- 172.21.15.45:0/274768717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc0c8b70 0x7f9acc0c8f90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.220+0000 7f9ad369e700 1 -- 172.21.15.45:0/274768717 >> 172.21.15.45:0/274768717 conn(0x7f9acc067e00 msgr2=0x7f9acc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.221+0000 7f9ad369e700 1 -- 172.21.15.45:0/274768717 shutdown_connections 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.221+0000 7f9ad369e700 1 -- 172.21.15.45:0/274768717 wait complete. 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.222+0000 7f9ad369e700 1 Processor -- start 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.222+0000 7f9ad369e700 1 -- start start 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad369e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0c8b70 0x7f9acc0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad369e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9acc0ccb70 0x7f9acc06c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad369e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc157330 0x7f9acc06c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad369e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc15a4f0 con 0x7f9acc157330 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad369e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b1700 con 0x7f9acc0c8b70 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad369e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b0dd0 con 0x7f9acc0ccb70 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc157330 0x7f9acc06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc157330 0x7f9acc06c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36528/0 (socket says 172.21.15.45:36528) 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/2967495306 learned_addr learned my addr 172.21.15.45:0/2967495306 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad143a700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0c8b70 0x7f9acc0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad0c39700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9acc0ccb70 0x7f9acc06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9acc0ccb70 msgr2=0x7f9acc06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9acc0ccb70 0x7f9acc06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0c8b70 msgr2=0x7f9acc0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0c8b70 0x7f9acc0aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 -- 172.21.15.45:0/2967495306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009ae0 con 0x7f9acc157330 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.223+0000 7f9ad1c3b700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc157330 0x7f9acc06c840 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9acc06bac0 tx=0x7f9ac4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.224+0000 7f9ac17fa700 1 -- 172.21.15.45:0/2967495306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4012e60 con 0x7f9acc157330 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.224+0000 7f9ac17fa700 1 -- 172.21.15.45:0/2967495306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac40092e0 con 0x7f9acc157330 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.224+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc0b10b0 con 0x7f9acc157330 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.224+0000 7f9ac17fa700 1 -- 172.21.15.45:0/2967495306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac401e660 con 0x7f9acc157330 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.224+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc0b1b90 con 0x7f9acc157330 2023-10-06T05:21:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.225+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc06e350 con 0x7f9acc157330 2023-10-06T05:21:52.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.225+0000 7f9ac17fa700 1 -- 172.21.15.45:0/2967495306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ac401ea10 con 0x7f9acc157330 2023-10-06T05:21:52.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.226+0000 7f9ac17fa700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ab0066f30 0x7f9ab00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.226+0000 7f9ac17fa700 1 -- 172.21.15.45:0/2967495306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9ac4086450 con 0x7f9acc157330 2023-10-06T05:21:52.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.226+0000 7f9ad143a700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ab0066f30 0x7f9ab00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.227+0000 7f9ad143a700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ab0066f30 0x7f9ab00693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b22f0 tx=0x7f9abc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.230+0000 7f9ac17fa700 1 -- 172.21.15.45:0/2967495306 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ac4053420 con 0x7f9acc157330 2023-10-06T05:21:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:52.489+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9acc0c37e0 con 0x7f9acc157330 2023-10-06T05:21:53.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.344+0000 7f9ac17fa700 1 -- 172.21.15.45:0/2967495306 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9ac4056a40 con 0x7f9acc157330 2023-10-06T05:21:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.347+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ab0066f30 msgr2=0x7f9ab00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.347+0000 7f9ad369e700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ab0066f30 0x7f9ab00693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b22f0 tx=0x7f9abc009040 comp rx=0 tx=0).stop 2023-10-06T05:21:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.347+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc157330 msgr2=0x7f9acc06c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.347+0000 7f9ad369e700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc157330 0x7f9acc06c840 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9acc06bac0 tx=0x7f9ac4017000 comp rx=0 tx=0).stop 2023-10-06T05:21:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 shutdown_connections 2023-10-06T05:21:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ab0066f30 0x7f9ab00693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc157330 0x7f9acc06c840 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0c8b70 0x7f9acc0aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 --2- 172.21.15.45:0/2967495306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9acc0ccb70 0x7f9acc06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 >> 172.21.15.45:0/2967495306 conn(0x7f9acc067e00 msgr2=0x7f9acc0cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 shutdown_connections 2023-10-06T05:21:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.348+0000 7f9ad369e700 1 -- 172.21.15.45:0/2967495306 wait complete. 2023-10-06T05:21:53.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:21:53.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.501+0000 7f8ad4795700 1 Processor -- start 2023-10-06T05:21:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.501+0000 7f8ad4795700 1 -- start start 2023-10-06T05:21:53.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.502+0000 7f8ad4795700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8acc1406e0 0x7f8acc06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.502+0000 7f8ad4795700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8acc06cd60 0x7f8acc0acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.502+0000 7f8ad4795700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc0ad4e0 0x7f8acc0c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad4795700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8acc0c9940 con 0x7f8acc06cd60 2023-10-06T05:21:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad4795700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8acc06d230 con 0x7f8acc1406e0 2023-10-06T05:21:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad4795700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc06d510 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8acc06cd60 0x7f8acc0acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8acc06cd60 0x7f8acc0acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36538/0 (socket says 172.21.15.45:36538) 2023-10-06T05:21:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 -- 172.21.15.45:0/2044734433 learned_addr learned my addr 172.21.15.45:0/2044734433 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad2d32700 1 --2- 172.21.15.45:0/2044734433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc0ad4e0 0x7f8acc0c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad2531700 1 --2- 172.21.15.45:0/2044734433 >> v2:172.21.15.144:3300/0 conn(0x7f8acc1406e0 0x7f8acc06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 -- 172.21.15.45:0/2044734433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc0ad4e0 msgr2=0x7f8acc0c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 --2- 172.21.15.45:0/2044734433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc0ad4e0 0x7f8acc0c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 -- 172.21.15.45:0/2044734433 >> v2:172.21.15.144:3300/0 conn(0x7f8acc1406e0 msgr2=0x7f8acc06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 --2- 172.21.15.45:0/2044734433 >> v2:172.21.15.144:3300/0 conn(0x7f8acc1406e0 0x7f8acc06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 -- 172.21.15.45:0/2044734433 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc06d7f0 con 0x7f8acc06cd60 2023-10-06T05:21:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad2d32700 1 --2- 172.21.15.45:0/2044734433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc0ad4e0 0x7f8acc0c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad2531700 1 --2- 172.21.15.45:0/2044734433 >> v2:172.21.15.144:3300/0 conn(0x7f8acc1406e0 0x7f8acc06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad1d30700 1 --2- 172.21.15.45:0/2044734433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc06cd60 0x7f8acc0acfa0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8acc06ad90 tx=0x7f8abc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccf76ab7db4ffd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad152f700 1 -- 172.21.15.45:0/2044734433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc00b860 con 0x7f8acc06cd60 2023-10-06T05:21:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad152f700 1 -- 172.21.15.45:0/2044734433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc00b9c0 con 0x7f8acc06cd60 2023-10-06T05:21:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.503+0000 7f8ad152f700 1 -- 172.21.15.45:0/2044734433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc00bd60 con 0x7f8acc06cd60 2023-10-06T05:21:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 -- 172.21.15.45:0/2044734433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc06cd60 msgr2=0x7f8acc0acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 --2- 172.21.15.45:0/2044734433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc06cd60 0x7f8acc0acfa0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8acc06ad90 tx=0x7f8abc009250 comp rx=0 tx=0).stop 2023-10-06T05:21:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 -- 172.21.15.45:0/2044734433 shutdown_connections 2023-10-06T05:21:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 --2- 172.21.15.45:0/2044734433 >> v2:172.21.15.144:3300/0 conn(0x7f8acc1406e0 0x7f8acc06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 --2- 172.21.15.45:0/2044734433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc0ad4e0 0x7f8acc0c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 --2- 172.21.15.45:0/2044734433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc06cd60 0x7f8acc0acfa0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 -- 172.21.15.45:0/2044734433 >> 172.21.15.45:0/2044734433 conn(0x7f8acc0691c0 msgr2=0x7f8acc0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 -- 172.21.15.45:0/2044734433 shutdown_connections 2023-10-06T05:21:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.504+0000 7f8ad4795700 1 -- 172.21.15.45:0/2044734433 wait complete. 2023-10-06T05:21:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.505+0000 7f8ad4795700 1 Processor -- start 2023-10-06T05:21:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.505+0000 7f8ad4795700 1 -- start start 2023-10-06T05:21:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad4795700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8acc06cd60 0x7f8acc13bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad4795700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc0ad4e0 0x7f8acc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad4795700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc1406e0 0x7f8acc0c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad4795700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc0c6fd0 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad4795700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc0b19d0 con 0x7f8acc06cd60 2023-10-06T05:21:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad4795700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc1578a0 con 0x7f8acc1406e0 2023-10-06T05:21:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad1d30700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc0ad4e0 0x7f8acc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad1d30700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc0ad4e0 0x7f8acc0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36540/0 (socket says 172.21.15.45:36540) 2023-10-06T05:21:53.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad1d30700 1 -- 172.21.15.45:0/3685433592 learned_addr learned my addr 172.21.15.45:0/3685433592 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:53.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad2d32700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc1406e0 0x7f8acc0c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.506+0000 7f8ad2531700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8acc06cd60 0x7f8acc13bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad1d30700 1 -- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc1406e0 msgr2=0x7f8acc0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad1d30700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc1406e0 0x7f8acc0c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad1d30700 1 -- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8acc06cd60 msgr2=0x7f8acc13bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad1d30700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8acc06cd60 0x7f8acc13bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad1d30700 1 -- 172.21.15.45:0/3685433592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0094f0 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad1d30700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc0ad4e0 0x7f8acc0b1e60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009290 tx=0x7f8abc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ac27fc700 1 -- 172.21.15.45:0/3685433592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc026480 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ac27fc700 1 -- 172.21.15.45:0/3685433592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc01f070 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ac27fc700 1 -- 172.21.15.45:0/3685433592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc026830 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc157b80 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.507+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc0b23a0 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.508+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8acc0b1b60 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.509+0000 7f8ac27fc700 1 -- 172.21.15.45:0/3685433592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8abc026ba0 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.509+0000 7f8ac27fc700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ab0066f30 0x7f8ab00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.509+0000 7f8ac27fc700 1 -- 172.21.15.45:0/3685433592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8abc021070 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.509+0000 7f8ad2531700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ab0066f30 0x7f8ab00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.510+0000 7f8ad2531700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ab0066f30 0x7f8ab00693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8006fd0 tx=0x7f8ac8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:53.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.513+0000 7f8ac27fc700 1 -- 172.21.15.45:0/3685433592 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8abc053630 con 0x7f8acc0ad4e0 2023-10-06T05:21:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:53.764+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8acc04bb40 con 0x7f8acc0ad4e0 2023-10-06T05:21:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.346+0000 7f8ac27fc700 1 -- 172.21.15.45:0/3685433592 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8abc056c50 con 0x7f8acc0ad4e0 2023-10-06T05:21:54.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ab0066f30 msgr2=0x7f8ab00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ab0066f30 0x7f8ab00693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8006fd0 tx=0x7f8ac8008040 comp rx=0 tx=0).stop 2023-10-06T05:21:54.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc0ad4e0 msgr2=0x7f8acc0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc0ad4e0 0x7f8acc0b1e60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009290 tx=0x7f8abc017000 comp rx=0 tx=0).stop 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 shutdown_connections 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ab0066f30 0x7f8ab00693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8acc1406e0 0x7f8acc0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8acc06cd60 0x7f8acc13bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 --2- 172.21.15.45:0/3685433592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc0ad4e0 0x7f8acc0b1e60 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 >> 172.21.15.45:0/3685433592 conn(0x7f8acc0691c0 msgr2=0x7f8acc0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 shutdown_connections 2023-10-06T05:21:54.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.349+0000 7f8ad4795700 1 -- 172.21.15.45:0/3685433592 wait complete. 2023-10-06T05:21:54.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:54.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.501+0000 7ff6a2c7a700 1 Processor -- start 2023-10-06T05:21:54.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.501+0000 7ff6a2c7a700 1 -- start start 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a2c7a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff69c159170 0x7ff69c159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a2c7a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c15d040 0x7ff69c068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a2c7a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69c0ca6d0 con 0x7ff69c069b90 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a2c7a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c15d550 con 0x7ff69c159170 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a2c7a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c15d810 con 0x7ff69c15d040 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a0a16700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff69c159170 0x7ff69c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a1217700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff69c069b90 0x7ff69c0aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39170/0 (socket says 172.21.15.45:39170) 2023-10-06T05:21:54.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a1217700 1 -- 172.21.15.45:0/1605951946 learned_addr learned my addr 172.21.15.45:0/1605951946 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a0a16700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff69c159170 0x7ff69c159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36550/0 (socket says 172.21.15.45:36550) 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff69bfff700 1 --2- 172.21.15.45:0/1605951946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c15d040 0x7ff69c068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a0a16700 1 -- 172.21.15.45:0/1605951946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c15d040 msgr2=0x7ff69c068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a0a16700 1 --2- 172.21.15.45:0/1605951946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c15d040 0x7ff69c068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a0a16700 1 -- 172.21.15.45:0/1605951946 >> v1:172.21.15.45:6789/0 conn(0x7ff69c069b90 legacy=0x7ff69c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a0a16700 1 -- 172.21.15.45:0/1605951946 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c15daf0 con 0x7ff69c159170 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff69b7fe700 1 -- 172.21.15.45:0/1605951946 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764328363 0 0) 0x7ff69c0ca6d0 con 0x7ff69c069b90 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff6a0a16700 1 --2- 172.21.15.45:0/1605951946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c159170 0x7ff69c159540 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0098e0 tx=0x7ff68c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bd7fd25f316e71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.502+0000 7ff69b7fe700 1 -- 172.21.15.45:0/1605951946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c009e80 con 0x7ff69c159170 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff69b7fe700 1 -- 172.21.15.45:0/1605951946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c00b860 con 0x7ff69c159170 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff69b7fe700 1 -- 172.21.15.45:0/1605951946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00bbe0 con 0x7ff69c159170 2023-10-06T05:21:54.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1605951946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c159170 msgr2=0x7ff69c159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1605951946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c159170 0x7ff69c159540 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0098e0 tx=0x7ff68c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1605951946 shutdown_connections 2023-10-06T05:21:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1605951946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c159170 0x7ff69c159540 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1605951946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c15d040 0x7ff69c068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1605951946 >> 172.21.15.45:0/1605951946 conn(0x7ff69c06f7c0 msgr2=0x7ff69c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1605951946 shutdown_connections 2023-10-06T05:21:54.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.503+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1605951946 wait complete. 2023-10-06T05:21:54.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.504+0000 7ff6a2c7a700 1 Processor -- start 2023-10-06T05:21:54.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.505+0000 7ff6a2c7a700 1 -- start start 2023-10-06T05:21:54.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.505+0000 7ff6a2c7a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff69c069b90 0x7ff69c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.505+0000 7ff6a2c7a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c159170 0x7ff69c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.505+0000 7ff6a2c7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c15d040 0x7ff69c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.505+0000 7ff6a2c7a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0a6af0 con 0x7ff69c15d040 2023-10-06T05:21:54.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.505+0000 7ff6a2c7a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c16ae70 con 0x7ff69c069b90 2023-10-06T05:21:54.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.505+0000 7ff6a2c7a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c16ab90 con 0x7ff69c159170 2023-10-06T05:21:54.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c15d040 0x7ff69c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c15d040 0x7ff69c0b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36552/0 (socket says 172.21.15.45:36552) 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 -- 172.21.15.45:0/1053370248 learned_addr learned my addr 172.21.15.45:0/1053370248 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff69bfff700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c159170 0x7ff69c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a0a16700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff69c069b90 0x7ff69c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 -- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c159170 msgr2=0x7ff69c155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c159170 0x7ff69c155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 -- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff69c069b90 msgr2=0x7ff69c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff69c069b90 0x7ff69c0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 -- 172.21.15.45:0/1053370248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c009420 con 0x7ff69c15d040 2023-10-06T05:21:54.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a1217700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c15d040 0x7ff69c0b2110 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff69400cc90 tx=0x7ff694001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff698ff9700 1 -- 172.21.15.45:0/1053370248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400a940 con 0x7ff69c15d040 2023-10-06T05:21:54.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff698ff9700 1 -- 172.21.15.45:0/1053370248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69400aaa0 con 0x7ff69c15d040 2023-10-06T05:21:54.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c0c72e0 con 0x7ff69c15d040 2023-10-06T05:21:54.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff698ff9700 1 -- 172.21.15.45:0/1053370248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400ae60 con 0x7ff69c15d040 2023-10-06T05:21:54.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.506+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c0c78a0 con 0x7ff69c15d040 2023-10-06T05:21:54.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.507+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c0c7140 con 0x7ff69c15d040 2023-10-06T05:21:54.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.508+0000 7ff698ff9700 1 -- 172.21.15.45:0/1053370248 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff694007300 con 0x7ff69c15d040 2023-10-06T05:21:54.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.508+0000 7ff698ff9700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff67c066f30 0x7ff67c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.508+0000 7ff698ff9700 1 -- 172.21.15.45:0/1053370248 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff694084bf0 con 0x7ff69c15d040 2023-10-06T05:21:54.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.509+0000 7ff6a0a16700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff67c066f30 0x7ff67c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.509+0000 7ff6a0a16700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff67c066f30 0x7ff67c0693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0091e0 tx=0x7ff68c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.513+0000 7ff698ff9700 1 -- 172.21.15.45:0/1053370248 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff69c0c7140 con 0x7ff69c15d040 2023-10-06T05:21:54.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.760+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff69c0c7140 con 0x7ff69c15d040 2023-10-06T05:21:54.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.761+0000 7ff698ff9700 1 -- 172.21.15.45:0/1053370248 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4602 (secure 0 0 0) 0x7ff69c0c7140 con 0x7ff69c15d040 2023-10-06T05:21:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:54.763 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":86,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:21:54.343327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/821082350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":821082350},{"type":"v1","addr":"172.21.15.45:6833","nonce":821082350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff67c066f30 msgr2=0x7ff67c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff67c066f30 0x7ff67c0693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0091e0 tx=0x7ff68c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c15d040 msgr2=0x7ff69c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c15d040 0x7ff69c0b2110 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff69400cc90 tx=0x7ff694001a80 comp rx=0 tx=0).stop 2023-10-06T05:21:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 shutdown_connections 2023-10-06T05:21:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff67c066f30 0x7ff67c0693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff69c15d040 0x7ff69c0b2110 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff69c069b90 0x7ff69c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 --2- 172.21.15.45:0/1053370248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff69c159170 0x7ff69c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 >> 172.21.15.45:0/1053370248 conn(0x7ff69c06f7c0 msgr2=0x7ff69c0cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 shutdown_connections 2023-10-06T05:21:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.763+0000 7ff6a2c7a700 1 -- 172.21.15.45:0/1053370248 wait complete. 2023-10-06T05:21:54.766 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 86 2023-10-06T05:21:54.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:54.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.909+0000 7f0fa1cb0700 1 Processor -- start 2023-10-06T05:21:54.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.909+0000 7f0fa1cb0700 1 -- start start 2023-10-06T05:21:54.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0fa1cb0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f9c0ccaa0 0x7f0f9c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0fa1cb0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9c0cb530 con 0x7f0f9c158f60 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0fa1cb0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9c069bd0 con 0x7f0f9c159330 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0fa1cb0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9c068fc0 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f9c0ccaa0 0x7f0f9c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0f9c158f60 0x7f0f9c0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39182/0 (socket says 172.21.15.45:39182) 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9b7fe700 1 -- 172.21.15.45:0/1623891500 learned_addr learned my addr 172.21.15.45:0/1623891500 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9bfff700 1 --2- 172.21.15.45:0/1623891500 >> v2:172.21.15.45:3300/0 conn(0x7f0f9c0ccaa0 0x7f0f9c06a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36568/0 (socket says 172.21.15.45:36568) 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9bfff700 1 -- 172.21.15.45:0/1623891500 >> v1:172.21.15.144:6789/0 conn(0x7f0f9c159330 legacy=0x7f0f9c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9bfff700 1 -- 172.21.15.45:0/1623891500 >> v1:172.21.15.45:6789/0 conn(0x7f0f9c158f60 legacy=0x7f0f9c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9a7fc700 1 -- 172.21.15.45:0/1623891500 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117829684 0 0) 0x7f0f8c001110 con 0x7f0f9c158f60 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.910+0000 7f0f9bfff700 1 -- 172.21.15.45:0/1623891500 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0692a0 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0f9bfff700 1 --2- 172.21.15.45:0/1623891500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c06a060 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009e60 tx=0x7f0f94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8181d068dd2e0817 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0f9a7fc700 1 -- 172.21.15.45:0/1623891500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9400e070 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0f9a7fc700 1 -- 172.21.15.45:0/1623891500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9400bc30 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0f9a7fc700 1 -- 172.21.15.45:0/1623891500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f94012430 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1623891500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 msgr2=0x7f0f9c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1623891500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c06a060 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009e60 tx=0x7f0f94009780 comp rx=0 tx=0).stop 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1623891500 shutdown_connections 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1623891500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c06a060 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.911+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1623891500 >> 172.21.15.45:0/1623891500 conn(0x7f0f9c0cdf50 msgr2=0x7f0f9c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.912+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1623891500 shutdown_connections 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.912+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1623891500 wait complete. 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.913+0000 7f0fa1cb0700 1 Processor -- start 2023-10-06T05:21:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.913+0000 7f0fa1cb0700 1 -- start start 2023-10-06T05:21:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0fa1cb0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c16a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0fa1cb0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f9c158f60 0x7f0f9c0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0fa1cb0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f9c159330 0x7f0f9c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0fa1cb0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c15a6f0 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0fa1cb0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c15c420 con 0x7f0f9c159330 2023-10-06T05:21:54.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0fa1cb0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c15c700 con 0x7f0f9c158f60 2023-10-06T05:21:54.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c16a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c16a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36582/0 (socket says 172.21.15.45:36582) 2023-10-06T05:21:54.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 -- 172.21.15.45:0/1532877878 learned_addr learned my addr 172.21.15.45:0/1532877878 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:54.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9affd700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f9c158f60 0x7f0f9c0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9bfff700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f9c159330 0x7f0f9c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 -- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f9c158f60 msgr2=0x7f0f9c0c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f9c158f60 0x7f0f9c0c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 -- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f9c159330 msgr2=0x7f0f9c15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f9c159330 0x7f0f9c15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 -- 172.21.15.45:0/1532877878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94009a20 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f9b7fe700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c16a180 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c810 tx=0x7f0f8c007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.914+0000 7f0f937fe700 1 -- 172.21.15.45:0/1532877878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c005b30 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.915+0000 7f0f937fe700 1 -- 172.21.15.45:0/1532877878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c005c90 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.915+0000 7f0f937fe700 1 -- 172.21.15.45:0/1532877878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c007300 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.915+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c15a0a0 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.915+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c15c130 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.916+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c06f410 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.916+0000 7f0f937fe700 1 -- 172.21.15.45:0/1532877878 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f8c01b400 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.916+0000 7f0f937fe700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f7c066f30 0x7f0f7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.917+0000 7f0f937fe700 1 -- 172.21.15.45:0/1532877878 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f0f8c084250 con 0x7f0f9c0ccaa0 2023-10-06T05:21:54.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.917+0000 7f0f9affd700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f7c066f30 0x7f0f7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.917+0000 7f0f9affd700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f7c066f30 0x7f0f7c0693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c15da00 tx=0x7f0f84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:54.921+0000 7f0f937fe700 1 -- 172.21.15.45:0/1532877878 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f8c0512d0 con 0x7f0f9c0ccaa0 2023-10-06T05:21:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.124+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f9c0cb530 con 0x7f0f9c0ccaa0 2023-10-06T05:21:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.125+0000 7f0f937fe700 1 -- 172.21.15.45:0/1532877878 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17170 (secure 0 0 0) 0x7f0f9c0cb530 con 0x7f0f9c0ccaa0 2023-10-06T05:21:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:55.127 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:21:51.848703+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:49.182551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:49.732208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:55.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f7c066f30 msgr2=0x7f0f7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f7c066f30 0x7f0f7c0693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c15da00 tx=0x7f0f84009040 comp rx=0 tx=0).stop 2023-10-06T05:21:55.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 msgr2=0x7f0f9c16a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c16a180 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c810 tx=0x7f0f8c007c50 comp rx=0 tx=0).stop 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 shutdown_connections 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0f7c066f30 0x7f0f7c0693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f9c159330 0x7f0f9c15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9c0ccaa0 0x7f0f9c16a180 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 --2- 172.21.15.45:0/1532877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f9c158f60 0x7f0f9c0c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 >> 172.21.15.45:0/1532877878 conn(0x7f0f9c0cdf50 msgr2=0x7f0f9c152a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 shutdown_connections 2023-10-06T05:21:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.127+0000 7f0fa1cb0700 1 -- 172.21.15.45:0/1532877878 wait complete. 2023-10-06T05:21:55.144 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:21:55.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:55.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.275+0000 7f187e035700 1 Processor -- start 2023-10-06T05:21:55.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.275+0000 7f187e035700 1 -- start start 2023-10-06T05:21:55.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f187e035700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18781560a0 0x7f187815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f187e035700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18780ccdc0 0x7f18780cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f187e035700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18780cd700 con 0x7f1878155cd0 2023-10-06T05:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f187e035700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18780ae6b0 con 0x7f18781560a0 2023-10-06T05:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f187e035700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18780ae8d0 con 0x7f18780ccdc0 2023-10-06T05:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f186ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18781560a0 0x7f187815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f18777fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1878155cd0 0x7f18780adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39190/0 (socket says 172.21.15.45:39190) 2023-10-06T05:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f18777fe700 1 -- 172.21.15.45:0/2256812359 learned_addr learned my addr 172.21.15.45:0/2256812359 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f1877fff700 1 --2- 172.21.15.45:0/2256812359 >> v2:172.21.15.144:3300/0 conn(0x7f18780ccdc0 0x7f18780cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f186ffff700 1 -- 172.21.15.45:0/2256812359 >> v2:172.21.15.144:3300/0 conn(0x7f18780ccdc0 msgr2=0x7f18780cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f186ffff700 1 --2- 172.21.15.45:0/2256812359 >> v2:172.21.15.144:3300/0 conn(0x7f18780ccdc0 0x7f18780cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f186ffff700 1 -- 172.21.15.45:0/2256812359 >> v1:172.21.15.45:6789/0 conn(0x7f1878155cd0 legacy=0x7f18780adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f186ffff700 1 -- 172.21.15.45:0/2256812359 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780aebb0 con 0x7f18781560a0 2023-10-06T05:21:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.276+0000 7f1876ffd700 1 -- 172.21.15.45:0/2256812359 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499530873 0 0) 0x7f18780cd700 con 0x7f1878155cd0 2023-10-06T05:21:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f186ffff700 1 --2- 172.21.15.45:0/2256812359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18781560a0 0x7f187815a730 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f18780c3dd0 tx=0x7f18600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552a0b1d21d8eeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f1876ffd700 1 -- 172.21.15.45:0/2256812359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000b6b0 con 0x7f18781560a0 2023-10-06T05:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f1876ffd700 1 -- 172.21.15.45:0/2256812359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186000b810 con 0x7f18781560a0 2023-10-06T05:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f1876ffd700 1 -- 172.21.15.45:0/2256812359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000bbb0 con 0x7f18781560a0 2023-10-06T05:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 -- 172.21.15.45:0/2256812359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18781560a0 msgr2=0x7f187815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 --2- 172.21.15.45:0/2256812359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18781560a0 0x7f187815a730 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f18780c3dd0 tx=0x7f18600091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 -- 172.21.15.45:0/2256812359 shutdown_connections 2023-10-06T05:21:55.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 --2- 172.21.15.45:0/2256812359 >> v2:172.21.15.144:3300/0 conn(0x7f18780ccdc0 0x7f18780cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 --2- 172.21.15.45:0/2256812359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18781560a0 0x7f187815a730 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 -- 172.21.15.45:0/2256812359 >> 172.21.15.45:0/2256812359 conn(0x7f187806f090 msgr2=0x7f187806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 -- 172.21.15.45:0/2256812359 shutdown_connections 2023-10-06T05:21:55.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.277+0000 7f187e035700 1 -- 172.21.15.45:0/2256812359 wait complete. 2023-10-06T05:21:55.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.279+0000 7f187e035700 1 Processor -- start 2023-10-06T05:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.279+0000 7f187e035700 1 -- start start 2023-10-06T05:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18780ccdc0 0x7f18781564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878155cd0 0x7f1878169480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781560a0 0x7f18780a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781699c0 con 0x7f18780ccdc0 2023-10-06T05:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1878157790 con 0x7f18781560a0 2023-10-06T05:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1878156b80 con 0x7f1878155cd0 2023-10-06T05:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18780ccdc0 0x7f18781564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18780ccdc0 0x7f18781564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36606/0 (socket says 172.21.15.45:36606) 2023-10-06T05:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 -- 172.21.15.45:0/2458261796 learned_addr learned my addr 172.21.15.45:0/2458261796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f186ffff700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878155cd0 0x7f1878169480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f1877fff700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781560a0 0x7f18780a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 -- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878155cd0 msgr2=0x7f1878169480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878155cd0 0x7f1878169480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 -- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781560a0 msgr2=0x7f18780a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781560a0 0x7f18780a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 -- 172.21.15.45:0/2458261796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860009420 con 0x7f18780ccdc0 2023-10-06T05:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f1877fff700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781560a0 0x7f18780a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f18777fe700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18780ccdc0 0x7f18781564e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f186800c7e0 tx=0x7f1868007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f186f7fe700 1 -- 172.21.15.45:0/2458261796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868005b30 con 0x7f18780ccdc0 2023-10-06T05:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f186f7fe700 1 -- 172.21.15.45:0/2458261796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1868005c90 con 0x7f18780ccdc0 2023-10-06T05:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1878156e60 con 0x7f18780ccdc0 2023-10-06T05:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f186f7fe700 1 -- 172.21.15.45:0/2458261796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868007300 con 0x7f18780ccdc0 2023-10-06T05:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.280+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1878157420 con 0x7f18780ccdc0 2023-10-06T05:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.282+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187806e2f0 con 0x7f18780ccdc0 2023-10-06T05:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.282+0000 7f186f7fe700 1 -- 172.21.15.45:0/2458261796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18680076b0 con 0x7f18780ccdc0 2023-10-06T05:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.283+0000 7f186f7fe700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1858066f30 0x7f18580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.283+0000 7f186f7fe700 1 -- 172.21.15.45:0/2458261796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1868083de0 con 0x7f18780ccdc0 2023-10-06T05:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.283+0000 7f186ffff700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1858066f30 0x7f18580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.283+0000 7f186ffff700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1858066f30 0x7f18580693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f18600093f0 tx=0x7f1860017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.287+0000 7f186f7fe700 1 -- 172.21.15.45:0/2458261796 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1868050d10 con 0x7f18780ccdc0 2023-10-06T05:21:55.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.489+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18780aee90 con 0x7f1858066f30 2023-10-06T05:21:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.498+0000 7f186f7fe700 1 -- 172.21.15.45:0/2458261796 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311841 (secure 0 0 0) 0x7f18780aee90 con 0x7f1858066f30 2023-10-06T05:21:55.503 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1858066f30 msgr2=0x7f18580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1858066f30 0x7f18580693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f18600093f0 tx=0x7f1860017000 comp rx=0 tx=0).stop 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18780ccdc0 msgr2=0x7f18781564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18780ccdc0 0x7f18781564e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f186800c7e0 tx=0x7f1868007df0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 shutdown_connections 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1858066f30 0x7f18580693e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781560a0 0x7f18780a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18780ccdc0 0x7f18781564e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 --2- 172.21.15.45:0/2458261796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878155cd0 0x7f1878169480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 >> 172.21.15.45:0/2458261796 conn(0x7f187806f090 msgr2=0x7f18780c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 shutdown_connections 2023-10-06T05:21:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.502+0000 7f187e035700 1 -- 172.21.15.45:0/2458261796 wait complete. 2023-10-06T05:21:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:21:55.518 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-06T05:21:54.549687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2392412,"kb_used_data":10012,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752582308,"statfs":{"total":773094113280,"available":770644283392,"internally_reserved":0,"allocated":10252288,"data_stored":5528733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.229643"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.864866+0000","last_change":"2023-10-06T05:21:50.864866+0000","last_active":"2023-10-06T05:21:50.864866+0000","last_peered":"2023-10-06T05:21:50.864866+0000","last_clean":"2023-10-06T05:21:50.864866+0000","last_became_active":"2023-10-06T05:21:50.864611+0000","last_became_peered":"2023-10-06T05:21:50.864611+0000","last_unstale":"2023-10-06T05:21:50.864866+0000","last_undegraded":"2023-10-06T05:21:50.864866+0000","last_fullsized":"2023-10-06T05:21:50.864866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:09.038999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875542+0000","last_change":"2023-10-06T05:21:50.873574+0000","last_active":"2023-10-06T05:21:50.875542+0000","last_peered":"2023-10-06T05:21:50.875542+0000","last_clean":"2023-10-06T05:21:50.875542+0000","last_became_active":"2023-10-06T05:21:50.873226+0000","last_became_peered":"2023-10-06T05:21:50.873226+0000","last_unstale":"2023-10-06T05:21:50.875542+0000","last_undegraded":"2023-10-06T05:21:50.875542+0000","last_fullsized":"2023-10-06T05:21:50.875542+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.751901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867302+0000","last_change":"2023-10-06T05:21:50.867302+0000","last_active":"2023-10-06T05:21:50.867302+0000","last_peered":"2023-10-06T05:21:50.867302+0000","last_clean":"2023-10-06T05:21:50.867302+0000","last_became_active":"2023-10-06T05:21:50.867027+0000","last_became_peered":"2023-10-06T05:21:50.867027+0000","last_unstale":"2023-10-06T05:21:50.867302+0000","last_undegraded":"2023-10-06T05:21:50.867302+0000","last_fullsized":"2023-10-06T05:21:50.867302+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:16.766769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.860337+0000","last_change":"2023-10-06T05:21:50.860337+0000","last_active":"2023-10-06T05:21:50.860337+0000","last_peered":"2023-10-06T05:21:50.860337+0000","last_clean":"2023-10-06T05:21:50.860337+0000","last_became_active":"2023-10-06T05:21:50.860046+0000","last_became_peered":"2023-10-06T05:21:50.860046+0000","last_unstale":"2023-10-06T05:21:50.860337+0000","last_undegraded":"2023-10-06T05:21:50.860337+0000","last_fullsized":"2023-10-06T05:21:50.860337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:20.581707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872310+0000","last_change":"2023-10-06T05:21:50.867763+0000","last_active":"2023-10-06T05:21:50.872310+0000","last_peered":"2023-10-06T05:21:50.872310+0000","last_clean":"2023-10-06T05:21:50.872310+0000","last_became_active":"2023-10-06T05:21:50.867202+0000","last_became_peered":"2023-10-06T05:21:50.867202+0000","last_unstale":"2023-10-06T05:21:50.872310+0000","last_undegraded":"2023-10-06T05:21:50.872310+0000","last_fullsized":"2023-10-06T05:21:50.872310+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:08.495108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.862899+0000","last_change":"2023-10-06T05:21:50.859454+0000","last_active":"2023-10-06T05:21:50.862899+0000","last_peered":"2023-10-06T05:21:50.862899+0000","last_clean":"2023-10-06T05:21:50.862899+0000","last_became_active":"2023-10-06T05:21:50.858968+0000","last_became_peered":"2023-10-06T05:21:50.858968+0000","last_unstale":"2023-10-06T05:21:50.862899+0000","last_undegraded":"2023-10-06T05:21:50.862899+0000","last_fullsized":"2023-10-06T05:21:50.862899+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:56.047029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.862935+0000","last_change":"2023-10-06T05:21:50.860997+0000","last_active":"2023-10-06T05:21:50.862935+0000","last_peered":"2023-10-06T05:21:50.862935+0000","last_clean":"2023-10-06T05:21:50.862935+0000","last_became_active":"2023-10-06T05:21:50.860748+0000","last_became_peered":"2023-10-06T05:21:50.860748+0000","last_unstale":"2023-10-06T05:21:50.862935+0000","last_undegraded":"2023-10-06T05:21:50.862935+0000","last_fullsized":"2023-10-06T05:21:50.862935+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:31.764235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872987+0000","last_change":"2023-10-06T05:21:50.863490+0000","last_active":"2023-10-06T05:21:50.872987+0000","last_peered":"2023-10-06T05:21:50.872987+0000","last_clean":"2023-10-06T05:21:50.872987+0000","last_became_active":"2023-10-06T05:21:50.863187+0000","last_became_peered":"2023-10-06T05:21:50.863187+0000","last_unstale":"2023-10-06T05:21:50.872987+0000","last_undegraded":"2023-10-06T05:21:50.872987+0000","last_fullsized":"2023-10-06T05:21:50.872987+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:46.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872613+0000","last_change":"2023-10-06T05:21:50.869197+0000","last_active":"2023-10-06T05:21:50.872613+0000","last_peered":"2023-10-06T05:21:50.872613+0000","last_clean":"2023-10-06T05:21:50.872613+0000","last_became_active":"2023-10-06T05:21:50.868961+0000","last_became_peered":"2023-10-06T05:21:50.868961+0000","last_unstale":"2023-10-06T05:21:50.872613+0000","last_undegraded":"2023-10-06T05:21:50.872613+0000","last_fullsized":"2023-10-06T05:21:50.872613+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:00.480344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872371+0000","last_change":"2023-10-06T05:21:50.868580+0000","last_active":"2023-10-06T05:21:50.872371+0000","last_peered":"2023-10-06T05:21:50.872371+0000","last_clean":"2023-10-06T05:21:50.872371+0000","last_became_active":"2023-10-06T05:21:50.868317+0000","last_became_peered":"2023-10-06T05:21:50.868317+0000","last_unstale":"2023-10-06T05:21:50.872371+0000","last_undegraded":"2023-10-06T05:21:50.872371+0000","last_fullsized":"2023-10-06T05:21:50.872371+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:20.581707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.863038+0000","last_change":"2023-10-06T05:21:50.862080+0000","last_active":"2023-10-06T05:21:50.863038+0000","last_peered":"2023-10-06T05:21:50.863038+0000","last_clean":"2023-10-06T05:21:50.863038+0000","last_became_active":"2023-10-06T05:21:50.861672+0000","last_became_peered":"2023-10-06T05:21:50.861672+0000","last_unstale":"2023-10-06T05:21:50.863038+0000","last_undegraded":"2023-10-06T05:21:50.863038+0000","last_fullsized":"2023-10-06T05:21:50.863038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:08.495108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880852+0000","last_change":"2023-10-06T05:21:50.874220+0000","last_active":"2023-10-06T05:21:50.880852+0000","last_peered":"2023-10-06T05:21:50.880852+0000","last_clean":"2023-10-06T05:21:50.880852+0000","last_became_active":"2023-10-06T05:21:50.873941+0000","last_became_peered":"2023-10-06T05:21:50.873941+0000","last_unstale":"2023-10-06T05:21:50.880852+0000","last_undegraded":"2023-10-06T05:21:50.880852+0000","last_fullsized":"2023-10-06T05:21:50.880852+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.871601+0000","last_change":"2023-10-06T05:21:50.871601+0000","last_active":"2023-10-06T05:21:50.871601+0000","last_peered":"2023-10-06T05:21:50.871601+0000","last_clean":"2023-10-06T05:21:50.871601+0000","last_became_active":"2023-10-06T05:21:50.871318+0000","last_became_peered":"2023-10-06T05:21:50.871318+0000","last_unstale":"2023-10-06T05:21:50.871601+0000","last_undegraded":"2023-10-06T05:21:50.871601+0000","last_fullsized":"2023-10-06T05:21:50.871601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:01.081695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868778+0000","last_change":"2023-10-06T05:21:50.868778+0000","last_active":"2023-10-06T05:21:50.868778+0000","last_peered":"2023-10-06T05:21:50.868778+0000","last_clean":"2023-10-06T05:21:50.868778+0000","last_became_active":"2023-10-06T05:21:50.868303+0000","last_became_peered":"2023-10-06T05:21:50.868303+0000","last_unstale":"2023-10-06T05:21:50.868778+0000","last_undegraded":"2023-10-06T05:21:50.868778+0000","last_fullsized":"2023-10-06T05:21:50.868778+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:08.180389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874706+0000","last_change":"2023-10-06T05:21:50.874706+0000","last_active":"2023-10-06T05:21:50.874706+0000","last_peered":"2023-10-06T05:21:50.874706+0000","last_clean":"2023-10-06T05:21:50.874706+0000","last_became_active":"2023-10-06T05:21:50.874312+0000","last_became_peered":"2023-10-06T05:21:50.874312+0000","last_unstale":"2023-10-06T05:21:50.874706+0000","last_undegraded":"2023-10-06T05:21:50.874706+0000","last_fullsized":"2023-10-06T05:21:50.874706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:31.615237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880745+0000","last_change":"2023-10-06T05:21:50.873913+0000","last_active":"2023-10-06T05:21:50.880745+0000","last_peered":"2023-10-06T05:21:50.880745+0000","last_clean":"2023-10-06T05:21:50.880745+0000","last_became_active":"2023-10-06T05:21:50.873630+0000","last_became_peered":"2023-10-06T05:21:50.873630+0000","last_unstale":"2023-10-06T05:21:50.880745+0000","last_undegraded":"2023-10-06T05:21:50.880745+0000","last_fullsized":"2023-10-06T05:21:50.880745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:20.581707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872451+0000","last_change":"2023-10-06T05:21:50.861817+0000","last_active":"2023-10-06T05:21:50.872451+0000","last_peered":"2023-10-06T05:21:50.872451+0000","last_clean":"2023-10-06T05:21:50.872451+0000","last_became_active":"2023-10-06T05:21:50.861550+0000","last_became_peered":"2023-10-06T05:21:50.861550+0000","last_unstale":"2023-10-06T05:21:50.872451+0000","last_undegraded":"2023-10-06T05:21:50.872451+0000","last_fullsized":"2023-10-06T05:21:50.872451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:31.238031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.865871+0000","last_change":"2023-10-06T05:21:50.865871+0000","last_active":"2023-10-06T05:21:50.865871+0000","last_peered":"2023-10-06T05:21:50.865871+0000","last_clean":"2023-10-06T05:21:50.865871+0000","last_became_active":"2023-10-06T05:21:50.865573+0000","last_became_peered":"2023-10-06T05:21:50.865573+0000","last_unstale":"2023-10-06T05:21:50.865871+0000","last_undegraded":"2023-10-06T05:21:50.865871+0000","last_fullsized":"2023-10-06T05:21:50.865871+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:45.717062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874703+0000","last_change":"2023-10-06T05:21:50.874703+0000","last_active":"2023-10-06T05:21:50.874703+0000","last_peered":"2023-10-06T05:21:50.874703+0000","last_clean":"2023-10-06T05:21:50.874703+0000","last_became_active":"2023-10-06T05:21:50.874395+0000","last_became_peered":"2023-10-06T05:21:50.874395+0000","last_unstale":"2023-10-06T05:21:50.874703+0000","last_undegraded":"2023-10-06T05:21:50.874703+0000","last_fullsized":"2023-10-06T05:21:50.874703+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:01.889644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868729+0000","last_change":"2023-10-06T05:21:50.866326+0000","last_active":"2023-10-06T05:21:50.868729+0000","last_peered":"2023-10-06T05:21:50.868729+0000","last_clean":"2023-10-06T05:21:50.868729+0000","last_became_active":"2023-10-06T05:21:50.866067+0000","last_became_peered":"2023-10-06T05:21:50.866067+0000","last_unstale":"2023-10-06T05:21:50.868729+0000","last_undegraded":"2023-10-06T05:21:50.868729+0000","last_fullsized":"2023-10-06T05:21:50.868729+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:31.238031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.866446+0000","last_change":"2023-10-06T05:21:50.866446+0000","last_active":"2023-10-06T05:21:50.866446+0000","last_peered":"2023-10-06T05:21:50.866446+0000","last_clean":"2023-10-06T05:21:50.866446+0000","last_became_active":"2023-10-06T05:21:50.866168+0000","last_became_peered":"2023-10-06T05:21:50.866168+0000","last_unstale":"2023-10-06T05:21:50.866446+0000","last_undegraded":"2023-10-06T05:21:50.866446+0000","last_fullsized":"2023-10-06T05:21:50.866446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:16.766769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.873990+0000","last_change":"2023-10-06T05:21:50.860201+0000","last_active":"2023-10-06T05:21:50.873990+0000","last_peered":"2023-10-06T05:21:50.873990+0000","last_clean":"2023-10-06T05:21:50.873990+0000","last_became_active":"2023-10-06T05:21:50.859891+0000","last_became_peered":"2023-10-06T05:21:50.859891+0000","last_unstale":"2023-10-06T05:21:50.873990+0000","last_undegraded":"2023-10-06T05:21:50.873990+0000","last_fullsized":"2023-10-06T05:21:50.873990+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:10.249360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874300+0000","last_change":"2023-10-06T05:21:50.862482+0000","last_active":"2023-10-06T05:21:50.874300+0000","last_peered":"2023-10-06T05:21:50.874300+0000","last_clean":"2023-10-06T05:21:50.874300+0000","last_became_active":"2023-10-06T05:21:50.861369+0000","last_became_peered":"2023-10-06T05:21:50.861369+0000","last_unstale":"2023-10-06T05:21:50.874300+0000","last_undegraded":"2023-10-06T05:21:50.874300+0000","last_fullsized":"2023-10-06T05:21:50.874300+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:08.533894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880758+0000","last_change":"2023-10-06T05:21:50.874109+0000","last_active":"2023-10-06T05:21:50.880758+0000","last_peered":"2023-10-06T05:21:50.880758+0000","last_clean":"2023-10-06T05:21:50.880758+0000","last_became_active":"2023-10-06T05:21:50.873863+0000","last_became_peered":"2023-10-06T05:21:50.873863+0000","last_unstale":"2023-10-06T05:21:50.880758+0000","last_undegraded":"2023-10-06T05:21:50.880758+0000","last_fullsized":"2023-10-06T05:21:50.880758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:53.693527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880809+0000","last_change":"2023-10-06T05:21:50.873195+0000","last_active":"2023-10-06T05:21:50.880809+0000","last_peered":"2023-10-06T05:21:50.880809+0000","last_clean":"2023-10-06T05:21:50.880809+0000","last_became_active":"2023-10-06T05:21:50.872923+0000","last_became_peered":"2023-10-06T05:21:50.872923+0000","last_unstale":"2023-10-06T05:21:50.880809+0000","last_undegraded":"2023-10-06T05:21:50.880809+0000","last_fullsized":"2023-10-06T05:21:50.880809+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:57.458595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875506+0000","last_change":"2023-10-06T05:21:50.870662+0000","last_active":"2023-10-06T05:21:50.875506+0000","last_peered":"2023-10-06T05:21:50.875506+0000","last_clean":"2023-10-06T05:21:50.875506+0000","last_became_active":"2023-10-06T05:21:50.869878+0000","last_became_peered":"2023-10-06T05:21:50.869878+0000","last_unstale":"2023-10-06T05:21:50.875506+0000","last_undegraded":"2023-10-06T05:21:50.875506+0000","last_fullsized":"2023-10-06T05:21:50.875506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.863028+0000","last_change":"2023-10-06T05:21:50.861910+0000","last_active":"2023-10-06T05:21:50.863028+0000","last_peered":"2023-10-06T05:21:50.863028+0000","last_clean":"2023-10-06T05:21:50.863028+0000","last_became_active":"2023-10-06T05:21:50.861339+0000","last_became_peered":"2023-10-06T05:21:50.861339+0000","last_unstale":"2023-10-06T05:21:50.863028+0000","last_undegraded":"2023-10-06T05:21:50.863028+0000","last_fullsized":"2023-10-06T05:21:50.863028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:53.970173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872400+0000","last_change":"2023-10-06T05:21:50.867765+0000","last_active":"2023-10-06T05:21:50.872400+0000","last_peered":"2023-10-06T05:21:50.872400+0000","last_clean":"2023-10-06T05:21:50.872400+0000","last_became_active":"2023-10-06T05:21:50.867337+0000","last_became_peered":"2023-10-06T05:21:50.867337+0000","last_unstale":"2023-10-06T05:21:50.872400+0000","last_undegraded":"2023-10-06T05:21:50.872400+0000","last_fullsized":"2023-10-06T05:21:50.872400+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:31.764235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872839+0000","last_change":"2023-10-06T05:21:50.865981+0000","last_active":"2023-10-06T05:21:50.872839+0000","last_peered":"2023-10-06T05:21:50.872839+0000","last_clean":"2023-10-06T05:21:50.872839+0000","last_became_active":"2023-10-06T05:21:50.865688+0000","last_became_peered":"2023-10-06T05:21:50.865688+0000","last_unstale":"2023-10-06T05:21:50.872839+0000","last_undegraded":"2023-10-06T05:21:50.872839+0000","last_fullsized":"2023-10-06T05:21:50.872839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:10.322844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874085+0000","last_change":"2023-10-06T05:21:50.874085+0000","last_active":"2023-10-06T05:21:50.874085+0000","last_peered":"2023-10-06T05:21:50.874085+0000","last_clean":"2023-10-06T05:21:50.874085+0000","last_became_active":"2023-10-06T05:21:50.873757+0000","last_became_peered":"2023-10-06T05:21:50.873757+0000","last_unstale":"2023-10-06T05:21:50.874085+0000","last_undegraded":"2023-10-06T05:21:50.874085+0000","last_fullsized":"2023-10-06T05:21:50.874085+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:01.398734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867717+0000","last_change":"2023-10-06T05:21:50.867717+0000","last_active":"2023-10-06T05:21:50.867717+0000","last_peered":"2023-10-06T05:21:50.867717+0000","last_clean":"2023-10-06T05:21:50.867717+0000","last_became_active":"2023-10-06T05:21:50.867430+0000","last_became_peered":"2023-10-06T05:21:50.867430+0000","last_unstale":"2023-10-06T05:21:50.867717+0000","last_undegraded":"2023-10-06T05:21:50.867717+0000","last_fullsized":"2023-10-06T05:21:50.867717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.307845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.876645+0000","last_change":"2023-10-06T05:21:50.876645+0000","last_active":"2023-10-06T05:21:50.876645+0000","last_peered":"2023-10-06T05:21:50.876645+0000","last_clean":"2023-10-06T05:21:50.876645+0000","last_became_active":"2023-10-06T05:21:50.876167+0000","last_became_peered":"2023-10-06T05:21:50.876167+0000","last_unstale":"2023-10-06T05:21:50.876645+0000","last_undegraded":"2023-10-06T05:21:50.876645+0000","last_fullsized":"2023-10-06T05:21:50.876645+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:52.457661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.864170+0000","last_change":"2023-10-06T05:21:50.864170+0000","last_active":"2023-10-06T05:21:50.864170+0000","last_peered":"2023-10-06T05:21:50.864170+0000","last_clean":"2023-10-06T05:21:50.864170+0000","last_became_active":"2023-10-06T05:21:50.863883+0000","last_became_peered":"2023-10-06T05:21:50.863883+0000","last_unstale":"2023-10-06T05:21:50.864170+0000","last_undegraded":"2023-10-06T05:21:50.864170+0000","last_fullsized":"2023-10-06T05:21:50.864170+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:29.095150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"91'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.345448+0000","last_change":"2023-10-06T05:21:50.346108+0000","last_active":"2023-10-06T05:21:51.345448+0000","last_peered":"2023-10-06T05:21:51.345448+0000","last_clean":"2023-10-06T05:21:51.345448+0000","last_became_active":"2023-10-06T05:21:50.345776+0000","last_became_peered":"2023-10-06T05:21:50.345776+0000","last_unstale":"2023-10-06T05:21:51.345448+0000","last_undegraded":"2023-10-06T05:21:51.345448+0000","last_fullsized":"2023-10-06T05:21:51.345448+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:42.218651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867602+0000","last_change":"2023-10-06T05:21:50.867602+0000","last_active":"2023-10-06T05:21:50.867602+0000","last_peered":"2023-10-06T05:21:50.867602+0000","last_clean":"2023-10-06T05:21:50.867602+0000","last_became_active":"2023-10-06T05:21:50.867365+0000","last_became_peered":"2023-10-06T05:21:50.867365+0000","last_unstale":"2023-10-06T05:21:50.867602+0000","last_undegraded":"2023-10-06T05:21:50.867602+0000","last_fullsized":"2023-10-06T05:21:50.867602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:05.486948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.339761+0000","last_change":"2023-10-06T05:21:50.345812+0000","last_active":"2023-10-06T05:21:51.339761+0000","last_peered":"2023-10-06T05:21:51.339761+0000","last_clean":"2023-10-06T05:21:51.339761+0000","last_became_active":"2023-10-06T05:21:50.345499+0000","last_became_peered":"2023-10-06T05:21:50.345499+0000","last_unstale":"2023-10-06T05:21:51.339761+0000","last_undegraded":"2023-10-06T05:21:51.339761+0000","last_fullsized":"2023-10-06T05:21:51.339761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:00.393363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867533+0000","last_change":"2023-10-06T05:21:50.867533+0000","last_active":"2023-10-06T05:21:50.867533+0000","last_peered":"2023-10-06T05:21:50.867533+0000","last_clean":"2023-10-06T05:21:50.867533+0000","last_became_active":"2023-10-06T05:21:50.867263+0000","last_became_peered":"2023-10-06T05:21:50.867263+0000","last_unstale":"2023-10-06T05:21:50.867533+0000","last_undegraded":"2023-10-06T05:21:50.867533+0000","last_fullsized":"2023-10-06T05:21:50.867533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.112440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.342424+0000","last_change":"2023-10-06T05:21:50.351444+0000","last_active":"2023-10-06T05:21:51.342424+0000","last_peered":"2023-10-06T05:21:51.342424+0000","last_clean":"2023-10-06T05:21:51.342424+0000","last_became_active":"2023-10-06T05:21:50.351154+0000","last_became_peered":"2023-10-06T05:21:50.351154+0000","last_unstale":"2023-10-06T05:21:51.342424+0000","last_undegraded":"2023-10-06T05:21:51.342424+0000","last_fullsized":"2023-10-06T05:21:51.342424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:56.453224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869590+0000","last_change":"2023-10-06T05:21:50.865672+0000","last_active":"2023-10-06T05:21:50.869590+0000","last_peered":"2023-10-06T05:21:50.869590+0000","last_clean":"2023-10-06T05:21:50.869590+0000","last_became_active":"2023-10-06T05:21:50.865394+0000","last_became_peered":"2023-10-06T05:21:50.865394+0000","last_unstale":"2023-10-06T05:21:50.869590+0000","last_undegraded":"2023-10-06T05:21:50.869590+0000","last_fullsized":"2023-10-06T05:21:50.869590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.751901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.345679+0000","last_change":"2023-10-06T05:21:50.346837+0000","last_active":"2023-10-06T05:21:51.345679+0000","last_peered":"2023-10-06T05:21:51.345679+0000","last_clean":"2023-10-06T05:21:51.345679+0000","last_became_active":"2023-10-06T05:21:50.346522+0000","last_became_peered":"2023-10-06T05:21:50.346522+0000","last_unstale":"2023-10-06T05:21:51.345679+0000","last_undegraded":"2023-10-06T05:21:51.345679+0000","last_fullsized":"2023-10-06T05:21:51.345679+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:30.758864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.861128+0000","last_change":"2023-10-06T05:21:50.861128+0000","last_active":"2023-10-06T05:21:50.861128+0000","last_peered":"2023-10-06T05:21:50.861128+0000","last_clean":"2023-10-06T05:21:50.861128+0000","last_became_active":"2023-10-06T05:21:50.860886+0000","last_became_peered":"2023-10-06T05:21:50.860886+0000","last_unstale":"2023-10-06T05:21:50.861128+0000","last_undegraded":"2023-10-06T05:21:50.861128+0000","last_fullsized":"2023-10-06T05:21:50.861128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:05.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"91'1","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.856722+0000","last_change":"2023-10-06T05:21:50.348213+0000","last_active":"2023-10-06T05:21:50.856722+0000","last_peered":"2023-10-06T05:21:50.856722+0000","last_clean":"2023-10-06T05:21:50.856722+0000","last_became_active":"2023-10-06T05:21:50.347763+0000","last_became_peered":"2023-10-06T05:21:50.347763+0000","last_unstale":"2023-10-06T05:21:50.856722+0000","last_undegraded":"2023-10-06T05:21:50.856722+0000","last_fullsized":"2023-10-06T05:21:50.856722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:29.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.870796+0000","last_change":"2023-10-06T05:21:50.868575+0000","last_active":"2023-10-06T05:21:50.870796+0000","last_peered":"2023-10-06T05:21:50.870796+0000","last_clean":"2023-10-06T05:21:50.870796+0000","last_became_active":"2023-10-06T05:21:50.868313+0000","last_became_peered":"2023-10-06T05:21:50.868313+0000","last_unstale":"2023-10-06T05:21:50.870796+0000","last_undegraded":"2023-10-06T05:21:50.870796+0000","last_fullsized":"2023-10-06T05:21:50.870796+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:45.717062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.343321+0000","last_change":"2023-10-06T05:21:50.352216+0000","last_active":"2023-10-06T05:21:51.343321+0000","last_peered":"2023-10-06T05:21:51.343321+0000","last_clean":"2023-10-06T05:21:51.343321+0000","last_became_active":"2023-10-06T05:21:50.351925+0000","last_became_peered":"2023-10-06T05:21:50.351925+0000","last_unstale":"2023-10-06T05:21:51.343321+0000","last_undegraded":"2023-10-06T05:21:51.343321+0000","last_fullsized":"2023-10-06T05:21:51.343321+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:19.576336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868276+0000","last_change":"2023-10-06T05:21:50.868276+0000","last_active":"2023-10-06T05:21:50.868276+0000","last_peered":"2023-10-06T05:21:50.868276+0000","last_clean":"2023-10-06T05:21:50.868276+0000","last_became_active":"2023-10-06T05:21:50.868009+0000","last_became_peered":"2023-10-06T05:21:50.868009+0000","last_unstale":"2023-10-06T05:21:50.868276+0000","last_undegraded":"2023-10-06T05:21:50.868276+0000","last_fullsized":"2023-10-06T05:21:50.868276+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:47.407960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.857642+0000","last_change":"2023-10-06T05:21:50.346181+0000","last_active":"2023-10-06T05:21:50.857642+0000","last_peered":"2023-10-06T05:21:50.857642+0000","last_clean":"2023-10-06T05:21:50.857642+0000","last_became_active":"2023-10-06T05:21:50.345904+0000","last_became_peered":"2023-10-06T05:21:50.345904+0000","last_unstale":"2023-10-06T05:21:50.857642+0000","last_undegraded":"2023-10-06T05:21:50.857642+0000","last_fullsized":"2023-10-06T05:21:50.857642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:51.452290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868445+0000","last_change":"2023-10-06T05:21:50.865683+0000","last_active":"2023-10-06T05:21:50.868445+0000","last_peered":"2023-10-06T05:21:50.868445+0000","last_clean":"2023-10-06T05:21:50.868445+0000","last_became_active":"2023-10-06T05:21:50.865414+0000","last_became_peered":"2023-10-06T05:21:50.865414+0000","last_unstale":"2023-10-06T05:21:50.868445+0000","last_undegraded":"2023-10-06T05:21:50.868445+0000","last_fullsized":"2023-10-06T05:21:50.868445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:44.639551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.356027+0000","last_change":"2023-10-06T05:21:50.354788+0000","last_active":"2023-10-06T05:21:50.356027+0000","last_peered":"2023-10-06T05:21:50.356027+0000","last_clean":"2023-10-06T05:21:50.356027+0000","last_became_active":"2023-10-06T05:21:50.354156+0000","last_became_peered":"2023-10-06T05:21:50.354156+0000","last_unstale":"2023-10-06T05:21:50.356027+0000","last_undegraded":"2023-10-06T05:21:50.356027+0000","last_fullsized":"2023-10-06T05:21:50.356027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:24.178488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875487+0000","last_change":"2023-10-06T05:21:50.870489+0000","last_active":"2023-10-06T05:21:50.875487+0000","last_peered":"2023-10-06T05:21:50.875487+0000","last_clean":"2023-10-06T05:21:50.875487+0000","last_became_active":"2023-10-06T05:21:50.869554+0000","last_became_peered":"2023-10-06T05:21:50.869554+0000","last_unstale":"2023-10-06T05:21:50.875487+0000","last_undegraded":"2023-10-06T05:21:50.875487+0000","last_fullsized":"2023-10-06T05:21:50.875487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:10.672361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.348860+0000","last_change":"2023-10-06T05:21:50.342823+0000","last_active":"2023-10-06T05:21:50.348860+0000","last_peered":"2023-10-06T05:21:50.348860+0000","last_clean":"2023-10-06T05:21:50.348860+0000","last_became_active":"2023-10-06T05:21:50.342607+0000","last_became_peered":"2023-10-06T05:21:50.342607+0000","last_unstale":"2023-10-06T05:21:50.348860+0000","last_undegraded":"2023-10-06T05:21:50.348860+0000","last_fullsized":"2023-10-06T05:21:50.348860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:44.279177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869542+0000","last_change":"2023-10-06T05:21:50.865972+0000","last_active":"2023-10-06T05:21:50.869542+0000","last_peered":"2023-10-06T05:21:50.869542+0000","last_clean":"2023-10-06T05:21:50.869542+0000","last_became_active":"2023-10-06T05:21:50.865684+0000","last_became_peered":"2023-10-06T05:21:50.865684+0000","last_unstale":"2023-10-06T05:21:50.869542+0000","last_undegraded":"2023-10-06T05:21:50.869542+0000","last_fullsized":"2023-10-06T05:21:50.869542+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:10.249360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.356624+0000","last_change":"2023-10-06T05:21:50.355420+0000","last_active":"2023-10-06T05:21:50.356624+0000","last_peered":"2023-10-06T05:21:50.356624+0000","last_clean":"2023-10-06T05:21:50.356624+0000","last_became_active":"2023-10-06T05:21:50.355157+0000","last_became_peered":"2023-10-06T05:21:50.355157+0000","last_unstale":"2023-10-06T05:21:50.356624+0000","last_undegraded":"2023-10-06T05:21:50.356624+0000","last_fullsized":"2023-10-06T05:21:50.356624+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:08.033628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880773+0000","last_change":"2023-10-06T05:21:50.874279+0000","last_active":"2023-10-06T05:21:50.880773+0000","last_peered":"2023-10-06T05:21:50.880773+0000","last_clean":"2023-10-06T05:21:50.880773+0000","last_became_active":"2023-10-06T05:21:50.874021+0000","last_became_peered":"2023-10-06T05:21:50.874021+0000","last_unstale":"2023-10-06T05:21:50.880773+0000","last_undegraded":"2023-10-06T05:21:50.880773+0000","last_fullsized":"2023-10-06T05:21:50.880773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:47.407960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"91'3","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.342890+0000","last_change":"2023-10-06T05:21:50.341544+0000","last_active":"2023-10-06T05:21:51.342890+0000","last_peered":"2023-10-06T05:21:51.342890+0000","last_clean":"2023-10-06T05:21:51.342890+0000","last_became_active":"2023-10-06T05:21:50.341257+0000","last_became_peered":"2023-10-06T05:21:50.341257+0000","last_unstale":"2023-10-06T05:21:51.342890+0000","last_undegraded":"2023-10-06T05:21:51.342890+0000","last_fullsized":"2023-10-06T05:21:51.342890+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:00.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880716+0000","last_change":"2023-10-06T05:21:50.873370+0000","last_active":"2023-10-06T05:21:50.880716+0000","last_peered":"2023-10-06T05:21:50.880716+0000","last_clean":"2023-10-06T05:21:50.880716+0000","last_became_active":"2023-10-06T05:21:50.873067+0000","last_became_peered":"2023-10-06T05:21:50.873067+0000","last_unstale":"2023-10-06T05:21:50.880716+0000","last_undegraded":"2023-10-06T05:21:50.880716+0000","last_fullsized":"2023-10-06T05:21:50.880716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:57.081955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"91'6","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.347943+0000","last_change":"2023-10-06T05:21:50.348092+0000","last_active":"2023-10-06T05:21:51.347943+0000","last_peered":"2023-10-06T05:21:51.347943+0000","last_clean":"2023-10-06T05:21:51.347943+0000","last_became_active":"2023-10-06T05:21:50.347608+0000","last_became_peered":"2023-10-06T05:21:50.347608+0000","last_unstale":"2023-10-06T05:21:51.347943+0000","last_undegraded":"2023-10-06T05:21:51.347943+0000","last_fullsized":"2023-10-06T05:21:51.347943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:52.964802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.863771+0000","last_change":"2023-10-06T05:21:50.863771+0000","last_active":"2023-10-06T05:21:50.863771+0000","last_peered":"2023-10-06T05:21:50.863771+0000","last_clean":"2023-10-06T05:21:50.863771+0000","last_became_active":"2023-10-06T05:21:50.863353+0000","last_became_peered":"2023-10-06T05:21:50.863353+0000","last_unstale":"2023-10-06T05:21:50.863771+0000","last_undegraded":"2023-10-06T05:21:50.863771+0000","last_fullsized":"2023-10-06T05:21:50.863771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:43.224022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"91'2","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340897+0000","last_change":"2023-10-06T05:21:50.342430+0000","last_active":"2023-10-06T05:21:51.340897+0000","last_peered":"2023-10-06T05:21:51.340897+0000","last_clean":"2023-10-06T05:21:51.340897+0000","last_became_active":"2023-10-06T05:21:50.342128+0000","last_became_peered":"2023-10-06T05:21:50.342128+0000","last_unstale":"2023-10-06T05:21:51.340897+0000","last_undegraded":"2023-10-06T05:21:51.340897+0000","last_fullsized":"2023-10-06T05:21:51.340897+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:48.883232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.871747+0000","last_change":"2023-10-06T05:21:50.871747+0000","last_active":"2023-10-06T05:21:50.871747+0000","last_peered":"2023-10-06T05:21:50.871747+0000","last_clean":"2023-10-06T05:21:50.871747+0000","last_became_active":"2023-10-06T05:21:50.871535+0000","last_became_peered":"2023-10-06T05:21:50.871535+0000","last_unstale":"2023-10-06T05:21:50.871747+0000","last_undegraded":"2023-10-06T05:21:50.871747+0000","last_fullsized":"2023-10-06T05:21:50.871747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:51.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.353572+0000","last_change":"2023-10-06T05:21:50.349760+0000","last_active":"2023-10-06T05:21:50.353572+0000","last_peered":"2023-10-06T05:21:50.353572+0000","last_clean":"2023-10-06T05:21:50.353572+0000","last_became_active":"2023-10-06T05:21:50.349483+0000","last_became_peered":"2023-10-06T05:21:50.349483+0000","last_unstale":"2023-10-06T05:21:50.353572+0000","last_undegraded":"2023-10-06T05:21:50.353572+0000","last_fullsized":"2023-10-06T05:21:50.353572+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:09.918824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869568+0000","last_change":"2023-10-06T05:21:50.866451+0000","last_active":"2023-10-06T05:21:50.869568+0000","last_peered":"2023-10-06T05:21:50.869568+0000","last_clean":"2023-10-06T05:21:50.869568+0000","last_became_active":"2023-10-06T05:21:50.866190+0000","last_became_peered":"2023-10-06T05:21:50.866190+0000","last_unstale":"2023-10-06T05:21:50.869568+0000","last_undegraded":"2023-10-06T05:21:50.869568+0000","last_fullsized":"2023-10-06T05:21:50.869568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:46.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.855730+0000","last_change":"2023-10-06T05:21:50.341652+0000","last_active":"2023-10-06T05:21:50.855730+0000","last_peered":"2023-10-06T05:21:50.855730+0000","last_clean":"2023-10-06T05:21:50.855730+0000","last_became_active":"2023-10-06T05:21:50.341148+0000","last_became_peered":"2023-10-06T05:21:50.341148+0000","last_unstale":"2023-10-06T05:21:50.855730+0000","last_undegraded":"2023-10-06T05:21:50.855730+0000","last_fullsized":"2023-10-06T05:21:50.855730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:23.803079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868683+0000","last_change":"2023-10-06T05:21:50.868683+0000","last_active":"2023-10-06T05:21:50.868683+0000","last_peered":"2023-10-06T05:21:50.868683+0000","last_clean":"2023-10-06T05:21:50.868683+0000","last_became_active":"2023-10-06T05:21:50.868401+0000","last_became_peered":"2023-10-06T05:21:50.868401+0000","last_unstale":"2023-10-06T05:21:50.868683+0000","last_undegraded":"2023-10-06T05:21:50.868683+0000","last_fullsized":"2023-10-06T05:21:50.868683+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:09.038999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"91'8","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.345433+0000","last_change":"2023-10-06T05:21:50.349367+0000","last_active":"2023-10-06T05:21:51.345433+0000","last_peered":"2023-10-06T05:21:51.345433+0000","last_clean":"2023-10-06T05:21:51.345433+0000","last_became_active":"2023-10-06T05:21:50.348812+0000","last_became_peered":"2023-10-06T05:21:50.348812+0000","last_unstale":"2023-10-06T05:21:51.345433+0000","last_undegraded":"2023-10-06T05:21:51.345433+0000","last_fullsized":"2023-10-06T05:21:51.345433+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:37.705038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875524+0000","last_change":"2023-10-06T05:21:50.870828+0000","last_active":"2023-10-06T05:21:50.875524+0000","last_peered":"2023-10-06T05:21:50.875524+0000","last_clean":"2023-10-06T05:21:50.875524+0000","last_became_active":"2023-10-06T05:21:50.870202+0000","last_became_peered":"2023-10-06T05:21:50.870202+0000","last_unstale":"2023-10-06T05:21:50.875524+0000","last_undegraded":"2023-10-06T05:21:50.875524+0000","last_fullsized":"2023-10-06T05:21:50.875524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:44.639551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"91'2","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340385+0000","last_change":"2023-10-06T05:21:50.336815+0000","last_active":"2023-10-06T05:21:51.340385+0000","last_peered":"2023-10-06T05:21:51.340385+0000","last_clean":"2023-10-06T05:21:51.340385+0000","last_became_active":"2023-10-06T05:21:50.336521+0000","last_became_peered":"2023-10-06T05:21:50.336521+0000","last_unstale":"2023-10-06T05:21:51.340385+0000","last_undegraded":"2023-10-06T05:21:51.340385+0000","last_fullsized":"2023-10-06T05:21:51.340385+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:28.089779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869375+0000","last_change":"2023-10-06T05:21:50.869375+0000","last_active":"2023-10-06T05:21:50.869375+0000","last_peered":"2023-10-06T05:21:50.869375+0000","last_clean":"2023-10-06T05:21:50.869375+0000","last_became_active":"2023-10-06T05:21:50.869084+0000","last_became_peered":"2023-10-06T05:21:50.869084+0000","last_unstale":"2023-10-06T05:21:50.869375+0000","last_undegraded":"2023-10-06T05:21:50.869375+0000","last_fullsized":"2023-10-06T05:21:50.869375+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:46.930766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.854716+0000","last_change":"2023-10-06T05:21:50.350952+0000","last_active":"2023-10-06T05:21:50.854716+0000","last_peered":"2023-10-06T05:21:50.854716+0000","last_clean":"2023-10-06T05:21:50.854716+0000","last_became_active":"2023-10-06T05:21:50.350605+0000","last_became_peered":"2023-10-06T05:21:50.350605+0000","last_unstale":"2023-10-06T05:21:50.854716+0000","last_undegraded":"2023-10-06T05:21:50.854716+0000","last_fullsized":"2023-10-06T05:21:50.854716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:55.041658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869662+0000","last_change":"2023-10-06T05:21:50.869662+0000","last_active":"2023-10-06T05:21:50.869662+0000","last_peered":"2023-10-06T05:21:50.869662+0000","last_clean":"2023-10-06T05:21:50.869662+0000","last_became_active":"2023-10-06T05:21:50.869374+0000","last_became_peered":"2023-10-06T05:21:50.869374+0000","last_unstale":"2023-10-06T05:21:50.869662+0000","last_undegraded":"2023-10-06T05:21:50.869662+0000","last_fullsized":"2023-10-06T05:21:50.869662+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:29.095150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852182+0000","last_change":"2023-10-06T05:21:50.337563+0000","last_active":"2023-10-06T05:21:50.852182+0000","last_peered":"2023-10-06T05:21:50.852182+0000","last_clean":"2023-10-06T05:21:50.852182+0000","last_became_active":"2023-10-06T05:21:50.337244+0000","last_became_peered":"2023-10-06T05:21:50.337244+0000","last_unstale":"2023-10-06T05:21:50.852182+0000","last_undegraded":"2023-10-06T05:21:50.852182+0000","last_fullsized":"2023-10-06T05:21:50.852182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:00.076324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872282+0000","last_change":"2023-10-06T05:21:50.867574+0000","last_active":"2023-10-06T05:21:50.872282+0000","last_peered":"2023-10-06T05:21:50.872282+0000","last_clean":"2023-10-06T05:21:50.872282+0000","last_became_active":"2023-10-06T05:21:50.867301+0000","last_became_peered":"2023-10-06T05:21:50.867301+0000","last_unstale":"2023-10-06T05:21:50.872282+0000","last_undegraded":"2023-10-06T05:21:50.872282+0000","last_fullsized":"2023-10-06T05:21:50.872282+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:57.081955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.858015+0000","last_change":"2023-10-06T05:21:50.344219+0000","last_active":"2023-10-06T05:21:50.858015+0000","last_peered":"2023-10-06T05:21:50.858015+0000","last_clean":"2023-10-06T05:21:50.858015+0000","last_became_active":"2023-10-06T05:21:50.343983+0000","last_became_peered":"2023-10-06T05:21:50.343983+0000","last_unstale":"2023-10-06T05:21:50.858015+0000","last_undegraded":"2023-10-06T05:21:50.858015+0000","last_fullsized":"2023-10-06T05:21:50.858015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:00.393363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.871034+0000","last_change":"2023-10-06T05:21:50.871034+0000","last_active":"2023-10-06T05:21:50.871034+0000","last_peered":"2023-10-06T05:21:50.871034+0000","last_clean":"2023-10-06T05:21:50.871034+0000","last_became_active":"2023-10-06T05:21:50.870739+0000","last_became_peered":"2023-10-06T05:21:50.870739+0000","last_unstale":"2023-10-06T05:21:50.871034+0000","last_undegraded":"2023-10-06T05:21:50.871034+0000","last_fullsized":"2023-10-06T05:21:50.871034+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:51.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852129+0000","last_change":"2023-10-06T05:21:50.348854+0000","last_active":"2023-10-06T05:21:50.852129+0000","last_peered":"2023-10-06T05:21:50.852129+0000","last_clean":"2023-10-06T05:21:50.852129+0000","last_became_active":"2023-10-06T05:21:50.348389+0000","last_became_peered":"2023-10-06T05:21:50.348389+0000","last_unstale":"2023-10-06T05:21:50.852129+0000","last_undegraded":"2023-10-06T05:21:50.852129+0000","last_fullsized":"2023-10-06T05:21:50.852129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:30.609866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868968+0000","last_change":"2023-10-06T05:21:50.865113+0000","last_active":"2023-10-06T05:21:50.868968+0000","last_peered":"2023-10-06T05:21:50.868968+0000","last_clean":"2023-10-06T05:21:50.868968+0000","last_became_active":"2023-10-06T05:21:50.864828+0000","last_became_peered":"2023-10-06T05:21:50.864828+0000","last_unstale":"2023-10-06T05:21:50.868968+0000","last_undegraded":"2023-10-06T05:21:50.868968+0000","last_fullsized":"2023-10-06T05:21:50.868968+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:10.322844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.861081+0000","last_change":"2023-10-06T05:21:50.349369+0000","last_active":"2023-10-06T05:21:50.861081+0000","last_peered":"2023-10-06T05:21:50.861081+0000","last_clean":"2023-10-06T05:21:50.861081+0000","last_became_active":"2023-10-06T05:21:50.349133+0000","last_became_peered":"2023-10-06T05:21:50.349133+0000","last_unstale":"2023-10-06T05:21:50.861081+0000","last_undegraded":"2023-10-06T05:21:50.861081+0000","last_fullsized":"2023-10-06T05:21:50.861081+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:07.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868993+0000","last_change":"2023-10-06T05:21:50.868993+0000","last_active":"2023-10-06T05:21:50.868993+0000","last_peered":"2023-10-06T05:21:50.868993+0000","last_clean":"2023-10-06T05:21:50.868993+0000","last_became_active":"2023-10-06T05:21:50.868558+0000","last_became_peered":"2023-10-06T05:21:50.868558+0000","last_unstale":"2023-10-06T05:21:50.868993+0000","last_undegraded":"2023-10-06T05:21:50.868993+0000","last_fullsized":"2023-10-06T05:21:50.868993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.112440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.354338+0000","last_change":"2023-10-06T05:21:50.352199+0000","last_active":"2023-10-06T05:21:50.354338+0000","last_peered":"2023-10-06T05:21:50.354338+0000","last_clean":"2023-10-06T05:21:50.354338+0000","last_became_active":"2023-10-06T05:21:50.351953+0000","last_became_peered":"2023-10-06T05:21:50.351953+0000","last_unstale":"2023-10-06T05:21:50.354338+0000","last_undegraded":"2023-10-06T05:21:50.354338+0000","last_fullsized":"2023-10-06T05:21:50.354338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:15.761398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872587+0000","last_change":"2023-10-06T05:21:50.869686+0000","last_active":"2023-10-06T05:21:50.872587+0000","last_peered":"2023-10-06T05:21:50.872587+0000","last_clean":"2023-10-06T05:21:50.872587+0000","last_became_active":"2023-10-06T05:21:50.869079+0000","last_became_peered":"2023-10-06T05:21:50.869079+0000","last_unstale":"2023-10-06T05:21:50.872587+0000","last_undegraded":"2023-10-06T05:21:50.872587+0000","last_fullsized":"2023-10-06T05:21:50.872587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:24.808450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.343571+0000","last_change":"2023-10-06T05:21:50.342350+0000","last_active":"2023-10-06T05:21:50.343571+0000","last_peered":"2023-10-06T05:21:50.343571+0000","last_clean":"2023-10-06T05:21:50.343571+0000","last_became_active":"2023-10-06T05:21:50.342011+0000","last_became_peered":"2023-10-06T05:21:50.342011+0000","last_unstale":"2023-10-06T05:21:50.343571+0000","last_undegraded":"2023-10-06T05:21:50.343571+0000","last_fullsized":"2023-10-06T05:21:50.343571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:21.230000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872568+0000","last_change":"2023-10-06T05:21:50.869587+0000","last_active":"2023-10-06T05:21:50.872568+0000","last_peered":"2023-10-06T05:21:50.872568+0000","last_clean":"2023-10-06T05:21:50.872568+0000","last_became_active":"2023-10-06T05:21:50.868698+0000","last_became_peered":"2023-10-06T05:21:50.868698+0000","last_unstale":"2023-10-06T05:21:50.872568+0000","last_undegraded":"2023-10-06T05:21:50.872568+0000","last_fullsized":"2023-10-06T05:21:50.872568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:53.970173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852357+0000","last_change":"2023-10-06T05:21:50.352343+0000","last_active":"2023-10-06T05:21:50.852357+0000","last_peered":"2023-10-06T05:21:50.852357+0000","last_clean":"2023-10-06T05:21:50.852357+0000","last_became_active":"2023-10-06T05:21:50.352068+0000","last_became_peered":"2023-10-06T05:21:50.352068+0000","last_unstale":"2023-10-06T05:21:50.852357+0000","last_undegraded":"2023-10-06T05:21:50.852357+0000","last_fullsized":"2023-10-06T05:21:50.852357+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:07.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.348108+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:21:50.348108+0000","last_peered":"2023-10-06T05:21:50.348108+0000","last_clean":"2023-10-06T05:21:50.348108+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:21:50.348108+0000","last_undegraded":"2023-10-06T05:21:50.348108+0000","last_fullsized":"2023-10-06T05:21:50.348108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872531+0000","last_change":"2023-10-06T05:21:50.869416+0000","last_active":"2023-10-06T05:21:50.872531+0000","last_peered":"2023-10-06T05:21:50.872531+0000","last_clean":"2023-10-06T05:21:50.872531+0000","last_became_active":"2023-10-06T05:21:50.868501+0000","last_became_peered":"2023-10-06T05:21:50.868501+0000","last_unstale":"2023-10-06T05:21:50.872531+0000","last_undegraded":"2023-10-06T05:21:50.872531+0000","last_fullsized":"2023-10-06T05:21:50.872531+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:56.047029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340070+0000","last_change":"2023-10-06T05:21:50.343236+0000","last_active":"2023-10-06T05:21:51.340070+0000","last_peered":"2023-10-06T05:21:51.340070+0000","last_clean":"2023-10-06T05:21:51.340070+0000","last_became_active":"2023-10-06T05:21:50.342974+0000","last_became_peered":"2023-10-06T05:21:50.342974+0000","last_unstale":"2023-10-06T05:21:51.340070+0000","last_undegraded":"2023-10-06T05:21:51.340070+0000","last_fullsized":"2023-10-06T05:21:51.340070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:56.076584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.333116+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:21:50.333116+0000","last_peered":"2023-10-06T05:21:50.333116+0000","last_clean":"2023-10-06T05:21:50.333116+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:21:50.333116+0000","last_undegraded":"2023-10-06T05:21:50.333116+0000","last_fullsized":"2023-10-06T05:21:50.333116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868989+0000","last_change":"2023-10-06T05:21:50.868989+0000","last_active":"2023-10-06T05:21:50.868989+0000","last_peered":"2023-10-06T05:21:50.868989+0000","last_clean":"2023-10-06T05:21:50.868989+0000","last_became_active":"2023-10-06T05:21:50.868752+0000","last_became_peered":"2023-10-06T05:21:50.868752+0000","last_unstale":"2023-10-06T05:21:50.868989+0000","last_undegraded":"2023-10-06T05:21:50.868989+0000","last_fullsized":"2023-10-06T05:21:50.868989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.307845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340291+0000","last_change":"2023-10-06T05:21:50.340299+0000","last_active":"2023-10-06T05:21:51.340291+0000","last_peered":"2023-10-06T05:21:51.340291+0000","last_clean":"2023-10-06T05:21:51.340291+0000","last_became_active":"2023-10-06T05:21:50.340039+0000","last_became_peered":"2023-10-06T05:21:50.340039+0000","last_unstale":"2023-10-06T05:21:51.340291+0000","last_undegraded":"2023-10-06T05:21:51.340291+0000","last_fullsized":"2023-10-06T05:21:51.340291+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:00.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.334807+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:21:50.334807+0000","last_peered":"2023-10-06T05:21:50.334807+0000","last_clean":"2023-10-06T05:21:50.334807+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:21:50.334807+0000","last_undegraded":"2023-10-06T05:21:50.334807+0000","last_fullsized":"2023-10-06T05:21:50.334807+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869925+0000","last_change":"2023-10-06T05:21:50.869925+0000","last_active":"2023-10-06T05:21:50.869925+0000","last_peered":"2023-10-06T05:21:50.869925+0000","last_clean":"2023-10-06T05:21:50.869925+0000","last_became_active":"2023-10-06T05:21:50.869637+0000","last_became_peered":"2023-10-06T05:21:50.869637+0000","last_unstale":"2023-10-06T05:21:50.869925+0000","last_undegraded":"2023-10-06T05:21:50.869925+0000","last_fullsized":"2023-10-06T05:21:50.869925+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:13.791546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.339582+0000","last_change":"2023-10-06T05:21:50.346294+0000","last_active":"2023-10-06T05:21:51.339582+0000","last_peered":"2023-10-06T05:21:51.339582+0000","last_clean":"2023-10-06T05:21:51.339582+0000","last_became_active":"2023-10-06T05:21:50.346033+0000","last_became_peered":"2023-10-06T05:21:50.346033+0000","last_unstale":"2023-10-06T05:21:51.339582+0000","last_undegraded":"2023-10-06T05:21:51.339582+0000","last_fullsized":"2023-10-06T05:21:51.339582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:12.786175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.333396+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:21:50.333396+0000","last_peered":"2023-10-06T05:21:50.333396+0000","last_clean":"2023-10-06T05:21:50.333396+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:21:50.333396+0000","last_undegraded":"2023-10-06T05:21:50.333396+0000","last_fullsized":"2023-10-06T05:21:50.333396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":224,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852150+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:21:50.852150+0000","last_peered":"2023-10-06T05:21:50.852150+0000","last_clean":"2023-10-06T05:21:50.852150+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:21:50.852150+0000","last_undegraded":"2023-10-06T05:21:50.852150+0000","last_fullsized":"2023-10-06T05:21:50.852150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.862496+0000","last_change":"2023-10-06T05:21:50.858627+0000","last_active":"2023-10-06T05:21:50.862496+0000","last_peered":"2023-10-06T05:21:50.862496+0000","last_clean":"2023-10-06T05:21:50.862496+0000","last_became_active":"2023-10-06T05:21:50.858366+0000","last_became_peered":"2023-10-06T05:21:50.858366+0000","last_unstale":"2023-10-06T05:21:50.862496+0000","last_undegraded":"2023-10-06T05:21:50.862496+0000","last_fullsized":"2023-10-06T05:21:50.862496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:24.808450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852303+0000","last_change":"2023-10-06T05:21:50.347150+0000","last_active":"2023-10-06T05:21:50.852303+0000","last_peered":"2023-10-06T05:21:50.852303+0000","last_clean":"2023-10-06T05:21:50.852303+0000","last_became_active":"2023-10-06T05:21:50.339261+0000","last_became_peered":"2023-10-06T05:21:50.339261+0000","last_unstale":"2023-10-06T05:21:50.852303+0000","last_undegraded":"2023-10-06T05:21:50.852303+0000","last_fullsized":"2023-10-06T05:21:50.852303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:44.711691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852241+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:21:50.852241+0000","last_peered":"2023-10-06T05:21:50.852241+0000","last_clean":"2023-10-06T05:21:50.852241+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:21:50.852241+0000","last_undegraded":"2023-10-06T05:21:50.852241+0000","last_fullsized":"2023-10-06T05:21:50.852241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868816+0000","last_change":"2023-10-06T05:21:50.868816+0000","last_active":"2023-10-06T05:21:50.868816+0000","last_peered":"2023-10-06T05:21:50.868816+0000","last_clean":"2023-10-06T05:21:50.868816+0000","last_became_active":"2023-10-06T05:21:50.868199+0000","last_became_peered":"2023-10-06T05:21:50.868199+0000","last_unstale":"2023-10-06T05:21:50.868816+0000","last_undegraded":"2023-10-06T05:21:50.868816+0000","last_fullsized":"2023-10-06T05:21:50.868816+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:05.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.346504+0000","last_change":"2023-10-06T05:21:50.345538+0000","last_active":"2023-10-06T05:21:50.346504+0000","last_peered":"2023-10-06T05:21:50.346504+0000","last_clean":"2023-10-06T05:21:50.346504+0000","last_became_active":"2023-10-06T05:21:50.345284+0000","last_became_peered":"2023-10-06T05:21:50.345284+0000","last_unstale":"2023-10-06T05:21:50.346504+0000","last_undegraded":"2023-10-06T05:21:50.346504+0000","last_fullsized":"2023-10-06T05:21:50.346504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:00.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852935+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:21:50.852935+0000","last_peered":"2023-10-06T05:21:50.852935+0000","last_clean":"2023-10-06T05:21:50.852935+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:21:50.852935+0000","last_undegraded":"2023-10-06T05:21:50.852935+0000","last_fullsized":"2023-10-06T05:21:50.852935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875690+0000","last_change":"2023-10-06T05:21:50.875690+0000","last_active":"2023-10-06T05:21:50.875690+0000","last_peered":"2023-10-06T05:21:50.875690+0000","last_clean":"2023-10-06T05:21:50.875690+0000","last_became_active":"2023-10-06T05:21:50.875407+0000","last_became_peered":"2023-10-06T05:21:50.875407+0000","last_unstale":"2023-10-06T05:21:50.875690+0000","last_undegraded":"2023-10-06T05:21:50.875690+0000","last_fullsized":"2023-10-06T05:21:50.875690+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:52.457661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.349112+0000","last_change":"2023-10-06T05:21:50.348023+0000","last_active":"2023-10-06T05:21:50.349112+0000","last_peered":"2023-10-06T05:21:50.349112+0000","last_clean":"2023-10-06T05:21:50.349112+0000","last_became_active":"2023-10-06T05:21:50.347746+0000","last_became_peered":"2023-10-06T05:21:50.347746+0000","last_unstale":"2023-10-06T05:21:50.349112+0000","last_undegraded":"2023-10-06T05:21:50.349112+0000","last_fullsized":"2023-10-06T05:21:50.349112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:51.452290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.348235+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:21:50.348235+0000","last_peered":"2023-10-06T05:21:50.348235+0000","last_clean":"2023-10-06T05:21:50.348235+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:21:50.348235+0000","last_undegraded":"2023-10-06T05:21:50.348235+0000","last_fullsized":"2023-10-06T05:21:50.348235+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868597+0000","last_change":"2023-10-06T05:21:50.868597+0000","last_active":"2023-10-06T05:21:50.868597+0000","last_peered":"2023-10-06T05:21:50.868597+0000","last_clean":"2023-10-06T05:21:50.868597+0000","last_became_active":"2023-10-06T05:21:50.867951+0000","last_became_peered":"2023-10-06T05:21:50.867951+0000","last_unstale":"2023-10-06T05:21:50.868597+0000","last_undegraded":"2023-10-06T05:21:50.868597+0000","last_fullsized":"2023-10-06T05:21:50.868597+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:14:16.002507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.342438+0000","last_change":"2023-10-06T05:21:50.351076+0000","last_active":"2023-10-06T05:21:51.342438+0000","last_peered":"2023-10-06T05:21:51.342438+0000","last_clean":"2023-10-06T05:21:51.342438+0000","last_became_active":"2023-10-06T05:21:50.350769+0000","last_became_peered":"2023-10-06T05:21:50.350769+0000","last_unstale":"2023-10-06T05:21:51.342438+0000","last_undegraded":"2023-10-06T05:21:51.342438+0000","last_fullsized":"2023-10-06T05:21:51.342438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:46.402589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.860840+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:21:50.860840+0000","last_peered":"2023-10-06T05:21:50.860840+0000","last_clean":"2023-10-06T05:21:50.860840+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:21:50.860840+0000","last_undegraded":"2023-10-06T05:21:50.860840+0000","last_fullsized":"2023-10-06T05:21:50.860840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298972,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072868,"statfs":{"total":96636764160,"available":96330616832,"internally_reserved":0,"allocated":1724416,"data_stored":1131832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.375,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.878,"15min":0.878},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76900000000000002,"5min":0.79500000000000004,"15min":0.92100000000000004},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72799999999999998,"5min":0.92300000000000004,"15min":1.476},"last":0.496},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.86299999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.626},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83299999999999996,"5min":0.86599999999999999,"15min":1.5129999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.63400000000000001},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79200000000000004,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64900000000000002,"15min":0.66900000000000004},"min":{"1min":0.51300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.439},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.501},"min":{"1min":0.34899999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.93200000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71699999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.752}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.87,"15min":0.92900000000000005},"last":0.78300000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298404,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073436,"statfs":{"total":96636764160,"available":96331198464,"internally_reserved":0,"allocated":1142784,"data_stored":548937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56399999999999995,"15min":0.71899999999999997},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.79000000000000004,"15min":4.6900000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.91500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75900000000000001,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.747},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":5.6239999999999997},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.96899999999999997},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80200000000000005,"5min":0.80500000000000005,"15min":11.595000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.78000000000000003},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78000000000000003,"5min":0.89900000000000002,"15min":5.5800000000000001},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":1.4199999999999999},"min":{"1min":0.43099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76200000000000001,"5min":0.78900000000000003,"15min":23.146000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.92500000000000004},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":9.7539999999999996},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46899999999999997,"15min":0.82499999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.68300000000000005,"15min":9.9969999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.85399999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":9.5850000000000009},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51800000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71199999999999997,"5min":0.77300000000000002,"15min":1.6439999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.85699999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.745,"5min":0.77900000000000003,"15min":9.8420000000000005},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55300000000000005,"15min":0.89100000000000001},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68999999999999995,"5min":0.82899999999999996,"15min":9.718},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57199999999999995,"15min":0.91600000000000004},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.749,"5min":0.85299999999999998,"15min":9.8930000000000007},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":541352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.52200000000000002,"5min":0.59699999999999998,"15min":0.64400000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.41499999999999998,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55900000000000005,"5min":0.67700000000000005,"15min":0.69599999999999995},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.437,"15min":0.441},"min":{"1min":0.308,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61499999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.42099999999999999,"15min":0.47499999999999998},"min":{"1min":0.29399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":1.673},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46600000000000003,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59799999999999998,"5min":0.752,"15min":4.0490000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.437,"15min":0.46100000000000002},"min":{"1min":0.308,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.77700000000000002},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.95199999999999996},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86699999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":1.0369999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82599999999999996,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.91200000000000003,"5min":1.167,"15min":1.167},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83399999999999996,"5min":0.94699999999999995,"15min":1.026},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88400000000000001,"5min":0.92600000000000005,"15min":0.93100000000000005},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1732608,"data_stored":1139235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.65100000000000002,"5min":0.82799999999999996,"15min":1.038},"last":0.372},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70299999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48699999999999999,"15min":0.46200000000000002},"min":{"1min":0.34599999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":3.855,"15min":3.855},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.32500000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.621,"5min":0.91200000000000003,"15min":1.1919999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.44600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.623,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.57299999999999995,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.35599999999999998,"15min":0.32700000000000001},"max":{"1min":0.73399999999999999,"5min":7.7670000000000003,"15min":7.7670000000000003},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64600000000000002,"15min":0.61899999999999999},"min":{"1min":0.53100000000000003,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.628},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86499999999999999,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.621}]},{"osd":5,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71199999999999997,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":5.3600000000000003,"15min":5.3600000000000003},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.55900000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91200000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69699999999999995,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.0940000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.55400000000000005,"5min":0.442,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":0.97399999999999998,"15min":1.0169999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":0.94499999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":541386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.42699999999999999},"min":{"1min":0.28100000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.51000000000000001,"5min":0.59299999999999997,"15min":2.0720000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40999999999999998,"15min":0.46200000000000002},"min":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":1.899},"last":0.34999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63,"5min":0.64300000000000002,"15min":2.097},"last":0.318},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.60599999999999998,"5min":0.71099999999999997,"15min":1.8759999999999999},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56699999999999995,"5min":0.66200000000000003,"15min":1.6719999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.51500000000000001},"min":{"1min":0.379,"5min":0.316,"15min":0.311},"max":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":2.121},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69299999999999995,"15min":0.70899999999999996},"min":{"1min":0.51000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.874,"5min":0.94199999999999995,"15min":1.7789999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.69399999999999995},"min":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.46700000000000003},"max":{"1min":0.998,"5min":1.0329999999999999,"15min":1.643},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92100000000000004,"5min":0.94399999999999995,"15min":2.0129999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.97299999999999998,"5min":0.97399999999999998,"15min":1.851},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.571},"last":0.751},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68300000000000005,"15min":0.70299999999999996},"min":{"1min":0.54600000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89100000000000001,"5min":0.98599999999999999,"15min":2.0449999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.90500000000000003,"5min":1.012,"15min":1.8180000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.48199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.613},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298396,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073444,"statfs":{"total":96636764160,"available":96331206656,"internally_reserved":0,"allocated":1134592,"data_stored":541557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"min":{"1min":0.25800000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.60399999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.38700000000000001},"min":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56599999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.318},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.433,"15min":0.41899999999999998},"min":{"1min":0.32300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.435,"15min":0.41999999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69899999999999995,"5min":0.755,"15min":0.755},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66100000000000003,"15min":0.63700000000000001},"min":{"1min":0.52700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.88300000000000001,"5min":1.143,"15min":1.143},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.64000000000000001},"min":{"1min":0.47299999999999998,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.628,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.40300000000000002,"15min":0.39900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96799999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56100000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90300000000000002,"5min":1.236,"15min":1.236},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67300000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298404,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073436,"statfs":{"total":96636764160,"available":96331198464,"internally_reserved":0,"allocated":1142784,"data_stored":543012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.45300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.76400000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.39800000000000002,"15min":0.33100000000000002},"max":{"1min":0.80800000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61099999999999999,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.32500000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79200000000000004,"5min":0.86599999999999999,"15min":0.89500000000000002},"last":0.55600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.75,"5min":0.83899999999999997,"15min":0.89800000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.61299999999999999},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":2.0489999999999999},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.379,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.85499999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77800000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49199999999999999,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.78600000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.372},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.5,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59199999999999997,"5min":0.76000000000000001,"15min":0.83799999999999997},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.64900000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63800000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":0.79800000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":541422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.29999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79300000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78500000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75800000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.876,"15min":0.876},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.496}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.28499999999999998},"max":{"1min":0.56999999999999995,"5min":0.68999999999999995,"15min":0.70999999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.32100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49099999999999999,"15min":0.47099999999999997},"min":{"1min":0.33400000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78600000000000003,"5min":0.876,"15min":0.876},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.502},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.73399999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83099999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:55.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 Processor -- start 2023-10-06T05:21:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 -- start start 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd00c8ff0 0x7f4cd00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cd00caad0 0x7f4cd00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4cd0156af0 0x7f4cd0156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd015b000 con 0x7f4cd00caad0 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd015b2e0 con 0x7f4cd0156af0 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.692+0000 7f4cd6546700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd015b5c0 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cd00caad0 0x7f4cd00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cd00caad0 0x7f4cd00caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36612/0 (socket says 172.21.15.45:36612) 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 -- 172.21.15.45:0/2339630305 learned_addr learned my addr 172.21.15.45:0/2339630305 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccffff700 1 --2- 172.21.15.45:0/2339630305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd00c8ff0 0x7f4cd00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4cd4ae3700 1 --2- 172.21.15.45:0/2339630305 >> v2:172.21.15.144:3300/0 conn(0x7f4cd0156af0 0x7f4cd0156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 -- 172.21.15.45:0/2339630305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd00c8ff0 msgr2=0x7f4cd00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 --2- 172.21.15.45:0/2339630305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd00c8ff0 0x7f4cd00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 -- 172.21.15.45:0/2339630305 >> v2:172.21.15.144:3300/0 conn(0x7f4cd0156af0 msgr2=0x7f4cd0156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 --2- 172.21.15.45:0/2339630305 >> v2:172.21.15.144:3300/0 conn(0x7f4cd0156af0 0x7f4cd0156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 -- 172.21.15.45:0/2339630305 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd015b8a0 con 0x7f4cd00caad0 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccffff700 1 --2- 172.21.15.45:0/2339630305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd00c8ff0 0x7f4cd00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4ccf7fe700 1 --2- 172.21.15.45:0/2339630305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00caad0 0x7f4cd00caf40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b0a60 tx=0x7f4cc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7370325bc86762c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4cceffd700 1 -- 172.21.15.45:0/2339630305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400b860 con 0x7f4cd00caad0 2023-10-06T05:21:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4cceffd700 1 -- 172.21.15.45:0/2339630305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc400b9c0 con 0x7f4cd00caad0 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.693+0000 7f4cceffd700 1 -- 172.21.15.45:0/2339630305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400bd60 con 0x7f4cd00caad0 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 -- 172.21.15.45:0/2339630305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00caad0 msgr2=0x7f4cd00caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 --2- 172.21.15.45:0/2339630305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00caad0 0x7f4cd00caf40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b0a60 tx=0x7f4cc4009250 comp rx=0 tx=0).stop 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 -- 172.21.15.45:0/2339630305 shutdown_connections 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 --2- 172.21.15.45:0/2339630305 >> v2:172.21.15.144:3300/0 conn(0x7f4cd0156af0 0x7f4cd0156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 --2- 172.21.15.45:0/2339630305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd00c8ff0 0x7f4cd00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 --2- 172.21.15.45:0/2339630305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00caad0 0x7f4cd00caf40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 -- 172.21.15.45:0/2339630305 >> 172.21.15.45:0/2339630305 conn(0x7f4cd00cdf50 msgr2=0x7f4cd00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 -- 172.21.15.45:0/2339630305 shutdown_connections 2023-10-06T05:21:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.694+0000 7f4cd6546700 1 -- 172.21.15.45:0/2339630305 wait complete. 2023-10-06T05:21:55.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.695+0000 7f4cd6546700 1 Processor -- start 2023-10-06T05:21:55.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.695+0000 7f4cd6546700 1 -- start start 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4cd6546700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00c8ff0 0x7f4cd00b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4cd6546700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd00caad0 0x7f4cd00c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4cd6546700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd0156af0 0x7f4cd00b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4cd6546700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd00b0870 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4cd6546700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0157760 con 0x7f4cd00caad0 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4cd6546700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd00b2bb0 con 0x7f4cd0156af0 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4ccffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00c8ff0 0x7f4cd00b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4ccffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00c8ff0 0x7f4cd00b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36620/0 (socket says 172.21.15.45:36620) 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4ccffff700 1 -- 172.21.15.45:0/343539184 learned_addr learned my addr 172.21.15.45:0/343539184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4cd4ae3700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd0156af0 0x7f4cd00b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.696+0000 7f4ccf7fe700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd00caad0 0x7f4cd00c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4ccffff700 1 -- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd0156af0 msgr2=0x7f4cd00b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4ccffff700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd0156af0 0x7f4cd00b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4ccffff700 1 -- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd00caad0 msgr2=0x7f4cd00c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4ccffff700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd00caad0 0x7f4cd00c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4ccffff700 1 -- 172.21.15.45:0/343539184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc40094f0 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4ccffff700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00c8ff0 0x7f4cd00b24f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000d860 tx=0x7f4cc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4cb7fff700 1 -- 172.21.15.45:0/343539184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00125f0 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4cb7fff700 1 -- 172.21.15.45:0/343539184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc0012750 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd0157110 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4cb7fff700 1 -- 172.21.15.45:0/343539184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0012af0 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.697+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd0155610 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.698+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd0156f70 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.699+0000 7f4cb7fff700 1 -- 172.21.15.45:0/343539184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cc0013480 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.699+0000 7f4cb7fff700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cbc066f30 0x7f4cbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.699+0000 7f4cb7fff700 1 -- 172.21.15.45:0/343539184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4cc0084910 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.699+0000 7f4ccf7fe700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cbc066f30 0x7f4cbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.700+0000 7f4ccf7fe700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cbc066f30 0x7f4cbc0693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b0a60 tx=0x7f4cc4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.704+0000 7f4cb7fff700 1 -- 172.21.15.45:0/343539184 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cd0156f70 con 0x7f4cd00c8ff0 2023-10-06T05:21:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.902+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4cd01578a0 con 0x7f4cbc066f30 2023-10-06T05:21:55.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.911+0000 7f4cb7fff700 1 -- 172.21.15.45:0/343539184 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311841 (secure 0 0 0) 0x7f4cc400bb30 con 0x7f4cbc066f30 2023-10-06T05:21:55.916 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cbc066f30 msgr2=0x7f4cbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cbc066f30 0x7f4cbc0693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b0a60 tx=0x7f4cc4017000 comp rx=0 tx=0).stop 2023-10-06T05:21:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00c8ff0 msgr2=0x7f4cd00b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00c8ff0 0x7f4cd00b24f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000d860 tx=0x7f4cc0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 shutdown_connections 2023-10-06T05:21:55.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cbc066f30 0x7f4cbc0693e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd0156af0 0x7f4cd00b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd00c8ff0 0x7f4cd00b24f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 --2- 172.21.15.45:0/343539184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd00caad0 0x7f4cd00c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 >> 172.21.15.45:0/343539184 conn(0x7f4cd00cdf50 msgr2=0x7f4cd0068780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 shutdown_connections 2023-10-06T05:21:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:55.915+0000 7f4cd6546700 1 -- 172.21.15.45:0/343539184 wait complete. 2023-10-06T05:21:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:21:55.930 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-06T05:21:54.549687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2392412,"kb_used_data":10012,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752582308,"statfs":{"total":773094113280,"available":770644283392,"internally_reserved":0,"allocated":10252288,"data_stored":5528733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.229643"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.864866+0000","last_change":"2023-10-06T05:21:50.864866+0000","last_active":"2023-10-06T05:21:50.864866+0000","last_peered":"2023-10-06T05:21:50.864866+0000","last_clean":"2023-10-06T05:21:50.864866+0000","last_became_active":"2023-10-06T05:21:50.864611+0000","last_became_peered":"2023-10-06T05:21:50.864611+0000","last_unstale":"2023-10-06T05:21:50.864866+0000","last_undegraded":"2023-10-06T05:21:50.864866+0000","last_fullsized":"2023-10-06T05:21:50.864866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:09.038999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875542+0000","last_change":"2023-10-06T05:21:50.873574+0000","last_active":"2023-10-06T05:21:50.875542+0000","last_peered":"2023-10-06T05:21:50.875542+0000","last_clean":"2023-10-06T05:21:50.875542+0000","last_became_active":"2023-10-06T05:21:50.873226+0000","last_became_peered":"2023-10-06T05:21:50.873226+0000","last_unstale":"2023-10-06T05:21:50.875542+0000","last_undegraded":"2023-10-06T05:21:50.875542+0000","last_fullsized":"2023-10-06T05:21:50.875542+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.751901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867302+0000","last_change":"2023-10-06T05:21:50.867302+0000","last_active":"2023-10-06T05:21:50.867302+0000","last_peered":"2023-10-06T05:21:50.867302+0000","last_clean":"2023-10-06T05:21:50.867302+0000","last_became_active":"2023-10-06T05:21:50.867027+0000","last_became_peered":"2023-10-06T05:21:50.867027+0000","last_unstale":"2023-10-06T05:21:50.867302+0000","last_undegraded":"2023-10-06T05:21:50.867302+0000","last_fullsized":"2023-10-06T05:21:50.867302+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:16.766769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.860337+0000","last_change":"2023-10-06T05:21:50.860337+0000","last_active":"2023-10-06T05:21:50.860337+0000","last_peered":"2023-10-06T05:21:50.860337+0000","last_clean":"2023-10-06T05:21:50.860337+0000","last_became_active":"2023-10-06T05:21:50.860046+0000","last_became_peered":"2023-10-06T05:21:50.860046+0000","last_unstale":"2023-10-06T05:21:50.860337+0000","last_undegraded":"2023-10-06T05:21:50.860337+0000","last_fullsized":"2023-10-06T05:21:50.860337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:20.581707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872310+0000","last_change":"2023-10-06T05:21:50.867763+0000","last_active":"2023-10-06T05:21:50.872310+0000","last_peered":"2023-10-06T05:21:50.872310+0000","last_clean":"2023-10-06T05:21:50.872310+0000","last_became_active":"2023-10-06T05:21:50.867202+0000","last_became_peered":"2023-10-06T05:21:50.867202+0000","last_unstale":"2023-10-06T05:21:50.872310+0000","last_undegraded":"2023-10-06T05:21:50.872310+0000","last_fullsized":"2023-10-06T05:21:50.872310+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:08.495108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.862899+0000","last_change":"2023-10-06T05:21:50.859454+0000","last_active":"2023-10-06T05:21:50.862899+0000","last_peered":"2023-10-06T05:21:50.862899+0000","last_clean":"2023-10-06T05:21:50.862899+0000","last_became_active":"2023-10-06T05:21:50.858968+0000","last_became_peered":"2023-10-06T05:21:50.858968+0000","last_unstale":"2023-10-06T05:21:50.862899+0000","last_undegraded":"2023-10-06T05:21:50.862899+0000","last_fullsized":"2023-10-06T05:21:50.862899+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:56.047029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.862935+0000","last_change":"2023-10-06T05:21:50.860997+0000","last_active":"2023-10-06T05:21:50.862935+0000","last_peered":"2023-10-06T05:21:50.862935+0000","last_clean":"2023-10-06T05:21:50.862935+0000","last_became_active":"2023-10-06T05:21:50.860748+0000","last_became_peered":"2023-10-06T05:21:50.860748+0000","last_unstale":"2023-10-06T05:21:50.862935+0000","last_undegraded":"2023-10-06T05:21:50.862935+0000","last_fullsized":"2023-10-06T05:21:50.862935+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:31.764235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872987+0000","last_change":"2023-10-06T05:21:50.863490+0000","last_active":"2023-10-06T05:21:50.872987+0000","last_peered":"2023-10-06T05:21:50.872987+0000","last_clean":"2023-10-06T05:21:50.872987+0000","last_became_active":"2023-10-06T05:21:50.863187+0000","last_became_peered":"2023-10-06T05:21:50.863187+0000","last_unstale":"2023-10-06T05:21:50.872987+0000","last_undegraded":"2023-10-06T05:21:50.872987+0000","last_fullsized":"2023-10-06T05:21:50.872987+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:46.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872613+0000","last_change":"2023-10-06T05:21:50.869197+0000","last_active":"2023-10-06T05:21:50.872613+0000","last_peered":"2023-10-06T05:21:50.872613+0000","last_clean":"2023-10-06T05:21:50.872613+0000","last_became_active":"2023-10-06T05:21:50.868961+0000","last_became_peered":"2023-10-06T05:21:50.868961+0000","last_unstale":"2023-10-06T05:21:50.872613+0000","last_undegraded":"2023-10-06T05:21:50.872613+0000","last_fullsized":"2023-10-06T05:21:50.872613+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:00.480344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872371+0000","last_change":"2023-10-06T05:21:50.868580+0000","last_active":"2023-10-06T05:21:50.872371+0000","last_peered":"2023-10-06T05:21:50.872371+0000","last_clean":"2023-10-06T05:21:50.872371+0000","last_became_active":"2023-10-06T05:21:50.868317+0000","last_became_peered":"2023-10-06T05:21:50.868317+0000","last_unstale":"2023-10-06T05:21:50.872371+0000","last_undegraded":"2023-10-06T05:21:50.872371+0000","last_fullsized":"2023-10-06T05:21:50.872371+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:20.581707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.863038+0000","last_change":"2023-10-06T05:21:50.862080+0000","last_active":"2023-10-06T05:21:50.863038+0000","last_peered":"2023-10-06T05:21:50.863038+0000","last_clean":"2023-10-06T05:21:50.863038+0000","last_became_active":"2023-10-06T05:21:50.861672+0000","last_became_peered":"2023-10-06T05:21:50.861672+0000","last_unstale":"2023-10-06T05:21:50.863038+0000","last_undegraded":"2023-10-06T05:21:50.863038+0000","last_fullsized":"2023-10-06T05:21:50.863038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:08.495108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880852+0000","last_change":"2023-10-06T05:21:50.874220+0000","last_active":"2023-10-06T05:21:50.880852+0000","last_peered":"2023-10-06T05:21:50.880852+0000","last_clean":"2023-10-06T05:21:50.880852+0000","last_became_active":"2023-10-06T05:21:50.873941+0000","last_became_peered":"2023-10-06T05:21:50.873941+0000","last_unstale":"2023-10-06T05:21:50.880852+0000","last_undegraded":"2023-10-06T05:21:50.880852+0000","last_fullsized":"2023-10-06T05:21:50.880852+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.871601+0000","last_change":"2023-10-06T05:21:50.871601+0000","last_active":"2023-10-06T05:21:50.871601+0000","last_peered":"2023-10-06T05:21:50.871601+0000","last_clean":"2023-10-06T05:21:50.871601+0000","last_became_active":"2023-10-06T05:21:50.871318+0000","last_became_peered":"2023-10-06T05:21:50.871318+0000","last_unstale":"2023-10-06T05:21:50.871601+0000","last_undegraded":"2023-10-06T05:21:50.871601+0000","last_fullsized":"2023-10-06T05:21:50.871601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:01.081695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868778+0000","last_change":"2023-10-06T05:21:50.868778+0000","last_active":"2023-10-06T05:21:50.868778+0000","last_peered":"2023-10-06T05:21:50.868778+0000","last_clean":"2023-10-06T05:21:50.868778+0000","last_became_active":"2023-10-06T05:21:50.868303+0000","last_became_peered":"2023-10-06T05:21:50.868303+0000","last_unstale":"2023-10-06T05:21:50.868778+0000","last_undegraded":"2023-10-06T05:21:50.868778+0000","last_fullsized":"2023-10-06T05:21:50.868778+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:08.180389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874706+0000","last_change":"2023-10-06T05:21:50.874706+0000","last_active":"2023-10-06T05:21:50.874706+0000","last_peered":"2023-10-06T05:21:50.874706+0000","last_clean":"2023-10-06T05:21:50.874706+0000","last_became_active":"2023-10-06T05:21:50.874312+0000","last_became_peered":"2023-10-06T05:21:50.874312+0000","last_unstale":"2023-10-06T05:21:50.874706+0000","last_undegraded":"2023-10-06T05:21:50.874706+0000","last_fullsized":"2023-10-06T05:21:50.874706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:31.615237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880745+0000","last_change":"2023-10-06T05:21:50.873913+0000","last_active":"2023-10-06T05:21:50.880745+0000","last_peered":"2023-10-06T05:21:50.880745+0000","last_clean":"2023-10-06T05:21:50.880745+0000","last_became_active":"2023-10-06T05:21:50.873630+0000","last_became_peered":"2023-10-06T05:21:50.873630+0000","last_unstale":"2023-10-06T05:21:50.880745+0000","last_undegraded":"2023-10-06T05:21:50.880745+0000","last_fullsized":"2023-10-06T05:21:50.880745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:20.581707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872451+0000","last_change":"2023-10-06T05:21:50.861817+0000","last_active":"2023-10-06T05:21:50.872451+0000","last_peered":"2023-10-06T05:21:50.872451+0000","last_clean":"2023-10-06T05:21:50.872451+0000","last_became_active":"2023-10-06T05:21:50.861550+0000","last_became_peered":"2023-10-06T05:21:50.861550+0000","last_unstale":"2023-10-06T05:21:50.872451+0000","last_undegraded":"2023-10-06T05:21:50.872451+0000","last_fullsized":"2023-10-06T05:21:50.872451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:31.238031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.865871+0000","last_change":"2023-10-06T05:21:50.865871+0000","last_active":"2023-10-06T05:21:50.865871+0000","last_peered":"2023-10-06T05:21:50.865871+0000","last_clean":"2023-10-06T05:21:50.865871+0000","last_became_active":"2023-10-06T05:21:50.865573+0000","last_became_peered":"2023-10-06T05:21:50.865573+0000","last_unstale":"2023-10-06T05:21:50.865871+0000","last_undegraded":"2023-10-06T05:21:50.865871+0000","last_fullsized":"2023-10-06T05:21:50.865871+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:45.717062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874703+0000","last_change":"2023-10-06T05:21:50.874703+0000","last_active":"2023-10-06T05:21:50.874703+0000","last_peered":"2023-10-06T05:21:50.874703+0000","last_clean":"2023-10-06T05:21:50.874703+0000","last_became_active":"2023-10-06T05:21:50.874395+0000","last_became_peered":"2023-10-06T05:21:50.874395+0000","last_unstale":"2023-10-06T05:21:50.874703+0000","last_undegraded":"2023-10-06T05:21:50.874703+0000","last_fullsized":"2023-10-06T05:21:50.874703+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:01.889644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868729+0000","last_change":"2023-10-06T05:21:50.866326+0000","last_active":"2023-10-06T05:21:50.868729+0000","last_peered":"2023-10-06T05:21:50.868729+0000","last_clean":"2023-10-06T05:21:50.868729+0000","last_became_active":"2023-10-06T05:21:50.866067+0000","last_became_peered":"2023-10-06T05:21:50.866067+0000","last_unstale":"2023-10-06T05:21:50.868729+0000","last_undegraded":"2023-10-06T05:21:50.868729+0000","last_fullsized":"2023-10-06T05:21:50.868729+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:31.238031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.866446+0000","last_change":"2023-10-06T05:21:50.866446+0000","last_active":"2023-10-06T05:21:50.866446+0000","last_peered":"2023-10-06T05:21:50.866446+0000","last_clean":"2023-10-06T05:21:50.866446+0000","last_became_active":"2023-10-06T05:21:50.866168+0000","last_became_peered":"2023-10-06T05:21:50.866168+0000","last_unstale":"2023-10-06T05:21:50.866446+0000","last_undegraded":"2023-10-06T05:21:50.866446+0000","last_fullsized":"2023-10-06T05:21:50.866446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:16.766769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.873990+0000","last_change":"2023-10-06T05:21:50.860201+0000","last_active":"2023-10-06T05:21:50.873990+0000","last_peered":"2023-10-06T05:21:50.873990+0000","last_clean":"2023-10-06T05:21:50.873990+0000","last_became_active":"2023-10-06T05:21:50.859891+0000","last_became_peered":"2023-10-06T05:21:50.859891+0000","last_unstale":"2023-10-06T05:21:50.873990+0000","last_undegraded":"2023-10-06T05:21:50.873990+0000","last_fullsized":"2023-10-06T05:21:50.873990+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:10.249360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874300+0000","last_change":"2023-10-06T05:21:50.862482+0000","last_active":"2023-10-06T05:21:50.874300+0000","last_peered":"2023-10-06T05:21:50.874300+0000","last_clean":"2023-10-06T05:21:50.874300+0000","last_became_active":"2023-10-06T05:21:50.861369+0000","last_became_peered":"2023-10-06T05:21:50.861369+0000","last_unstale":"2023-10-06T05:21:50.874300+0000","last_undegraded":"2023-10-06T05:21:50.874300+0000","last_fullsized":"2023-10-06T05:21:50.874300+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:08.533894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880758+0000","last_change":"2023-10-06T05:21:50.874109+0000","last_active":"2023-10-06T05:21:50.880758+0000","last_peered":"2023-10-06T05:21:50.880758+0000","last_clean":"2023-10-06T05:21:50.880758+0000","last_became_active":"2023-10-06T05:21:50.873863+0000","last_became_peered":"2023-10-06T05:21:50.873863+0000","last_unstale":"2023-10-06T05:21:50.880758+0000","last_undegraded":"2023-10-06T05:21:50.880758+0000","last_fullsized":"2023-10-06T05:21:50.880758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:53.693527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880809+0000","last_change":"2023-10-06T05:21:50.873195+0000","last_active":"2023-10-06T05:21:50.880809+0000","last_peered":"2023-10-06T05:21:50.880809+0000","last_clean":"2023-10-06T05:21:50.880809+0000","last_became_active":"2023-10-06T05:21:50.872923+0000","last_became_peered":"2023-10-06T05:21:50.872923+0000","last_unstale":"2023-10-06T05:21:50.880809+0000","last_undegraded":"2023-10-06T05:21:50.880809+0000","last_fullsized":"2023-10-06T05:21:50.880809+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:57.458595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875506+0000","last_change":"2023-10-06T05:21:50.870662+0000","last_active":"2023-10-06T05:21:50.875506+0000","last_peered":"2023-10-06T05:21:50.875506+0000","last_clean":"2023-10-06T05:21:50.875506+0000","last_became_active":"2023-10-06T05:21:50.869878+0000","last_became_peered":"2023-10-06T05:21:50.869878+0000","last_unstale":"2023-10-06T05:21:50.875506+0000","last_undegraded":"2023-10-06T05:21:50.875506+0000","last_fullsized":"2023-10-06T05:21:50.875506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:37.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.863028+0000","last_change":"2023-10-06T05:21:50.861910+0000","last_active":"2023-10-06T05:21:50.863028+0000","last_peered":"2023-10-06T05:21:50.863028+0000","last_clean":"2023-10-06T05:21:50.863028+0000","last_became_active":"2023-10-06T05:21:50.861339+0000","last_became_peered":"2023-10-06T05:21:50.861339+0000","last_unstale":"2023-10-06T05:21:50.863028+0000","last_undegraded":"2023-10-06T05:21:50.863028+0000","last_fullsized":"2023-10-06T05:21:50.863028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:53.970173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872400+0000","last_change":"2023-10-06T05:21:50.867765+0000","last_active":"2023-10-06T05:21:50.872400+0000","last_peered":"2023-10-06T05:21:50.872400+0000","last_clean":"2023-10-06T05:21:50.872400+0000","last_became_active":"2023-10-06T05:21:50.867337+0000","last_became_peered":"2023-10-06T05:21:50.867337+0000","last_unstale":"2023-10-06T05:21:50.872400+0000","last_undegraded":"2023-10-06T05:21:50.872400+0000","last_fullsized":"2023-10-06T05:21:50.872400+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:31.764235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872839+0000","last_change":"2023-10-06T05:21:50.865981+0000","last_active":"2023-10-06T05:21:50.872839+0000","last_peered":"2023-10-06T05:21:50.872839+0000","last_clean":"2023-10-06T05:21:50.872839+0000","last_became_active":"2023-10-06T05:21:50.865688+0000","last_became_peered":"2023-10-06T05:21:50.865688+0000","last_unstale":"2023-10-06T05:21:50.872839+0000","last_undegraded":"2023-10-06T05:21:50.872839+0000","last_fullsized":"2023-10-06T05:21:50.872839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:10.322844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.874085+0000","last_change":"2023-10-06T05:21:50.874085+0000","last_active":"2023-10-06T05:21:50.874085+0000","last_peered":"2023-10-06T05:21:50.874085+0000","last_clean":"2023-10-06T05:21:50.874085+0000","last_became_active":"2023-10-06T05:21:50.873757+0000","last_became_peered":"2023-10-06T05:21:50.873757+0000","last_unstale":"2023-10-06T05:21:50.874085+0000","last_undegraded":"2023-10-06T05:21:50.874085+0000","last_fullsized":"2023-10-06T05:21:50.874085+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:01.398734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867717+0000","last_change":"2023-10-06T05:21:50.867717+0000","last_active":"2023-10-06T05:21:50.867717+0000","last_peered":"2023-10-06T05:21:50.867717+0000","last_clean":"2023-10-06T05:21:50.867717+0000","last_became_active":"2023-10-06T05:21:50.867430+0000","last_became_peered":"2023-10-06T05:21:50.867430+0000","last_unstale":"2023-10-06T05:21:50.867717+0000","last_undegraded":"2023-10-06T05:21:50.867717+0000","last_fullsized":"2023-10-06T05:21:50.867717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.307845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.876645+0000","last_change":"2023-10-06T05:21:50.876645+0000","last_active":"2023-10-06T05:21:50.876645+0000","last_peered":"2023-10-06T05:21:50.876645+0000","last_clean":"2023-10-06T05:21:50.876645+0000","last_became_active":"2023-10-06T05:21:50.876167+0000","last_became_peered":"2023-10-06T05:21:50.876167+0000","last_unstale":"2023-10-06T05:21:50.876645+0000","last_undegraded":"2023-10-06T05:21:50.876645+0000","last_fullsized":"2023-10-06T05:21:50.876645+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:52.457661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.864170+0000","last_change":"2023-10-06T05:21:50.864170+0000","last_active":"2023-10-06T05:21:50.864170+0000","last_peered":"2023-10-06T05:21:50.864170+0000","last_clean":"2023-10-06T05:21:50.864170+0000","last_became_active":"2023-10-06T05:21:50.863883+0000","last_became_peered":"2023-10-06T05:21:50.863883+0000","last_unstale":"2023-10-06T05:21:50.864170+0000","last_undegraded":"2023-10-06T05:21:50.864170+0000","last_fullsized":"2023-10-06T05:21:50.864170+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:29.095150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"91'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.345448+0000","last_change":"2023-10-06T05:21:50.346108+0000","last_active":"2023-10-06T05:21:51.345448+0000","last_peered":"2023-10-06T05:21:51.345448+0000","last_clean":"2023-10-06T05:21:51.345448+0000","last_became_active":"2023-10-06T05:21:50.345776+0000","last_became_peered":"2023-10-06T05:21:50.345776+0000","last_unstale":"2023-10-06T05:21:51.345448+0000","last_undegraded":"2023-10-06T05:21:51.345448+0000","last_fullsized":"2023-10-06T05:21:51.345448+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:42.218651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867602+0000","last_change":"2023-10-06T05:21:50.867602+0000","last_active":"2023-10-06T05:21:50.867602+0000","last_peered":"2023-10-06T05:21:50.867602+0000","last_clean":"2023-10-06T05:21:50.867602+0000","last_became_active":"2023-10-06T05:21:50.867365+0000","last_became_peered":"2023-10-06T05:21:50.867365+0000","last_unstale":"2023-10-06T05:21:50.867602+0000","last_undegraded":"2023-10-06T05:21:50.867602+0000","last_fullsized":"2023-10-06T05:21:50.867602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:05.486948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.339761+0000","last_change":"2023-10-06T05:21:50.345812+0000","last_active":"2023-10-06T05:21:51.339761+0000","last_peered":"2023-10-06T05:21:51.339761+0000","last_clean":"2023-10-06T05:21:51.339761+0000","last_became_active":"2023-10-06T05:21:50.345499+0000","last_became_peered":"2023-10-06T05:21:50.345499+0000","last_unstale":"2023-10-06T05:21:51.339761+0000","last_undegraded":"2023-10-06T05:21:51.339761+0000","last_fullsized":"2023-10-06T05:21:51.339761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:00.393363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.867533+0000","last_change":"2023-10-06T05:21:50.867533+0000","last_active":"2023-10-06T05:21:50.867533+0000","last_peered":"2023-10-06T05:21:50.867533+0000","last_clean":"2023-10-06T05:21:50.867533+0000","last_became_active":"2023-10-06T05:21:50.867263+0000","last_became_peered":"2023-10-06T05:21:50.867263+0000","last_unstale":"2023-10-06T05:21:50.867533+0000","last_undegraded":"2023-10-06T05:21:50.867533+0000","last_fullsized":"2023-10-06T05:21:50.867533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.112440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.342424+0000","last_change":"2023-10-06T05:21:50.351444+0000","last_active":"2023-10-06T05:21:51.342424+0000","last_peered":"2023-10-06T05:21:51.342424+0000","last_clean":"2023-10-06T05:21:51.342424+0000","last_became_active":"2023-10-06T05:21:50.351154+0000","last_became_peered":"2023-10-06T05:21:50.351154+0000","last_unstale":"2023-10-06T05:21:51.342424+0000","last_undegraded":"2023-10-06T05:21:51.342424+0000","last_fullsized":"2023-10-06T05:21:51.342424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:56.453224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869590+0000","last_change":"2023-10-06T05:21:50.865672+0000","last_active":"2023-10-06T05:21:50.869590+0000","last_peered":"2023-10-06T05:21:50.869590+0000","last_clean":"2023-10-06T05:21:50.869590+0000","last_became_active":"2023-10-06T05:21:50.865394+0000","last_became_peered":"2023-10-06T05:21:50.865394+0000","last_unstale":"2023-10-06T05:21:50.869590+0000","last_undegraded":"2023-10-06T05:21:50.869590+0000","last_fullsized":"2023-10-06T05:21:50.869590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:21.751901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.345679+0000","last_change":"2023-10-06T05:21:50.346837+0000","last_active":"2023-10-06T05:21:51.345679+0000","last_peered":"2023-10-06T05:21:51.345679+0000","last_clean":"2023-10-06T05:21:51.345679+0000","last_became_active":"2023-10-06T05:21:50.346522+0000","last_became_peered":"2023-10-06T05:21:50.346522+0000","last_unstale":"2023-10-06T05:21:51.345679+0000","last_undegraded":"2023-10-06T05:21:51.345679+0000","last_fullsized":"2023-10-06T05:21:51.345679+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:30.758864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.861128+0000","last_change":"2023-10-06T05:21:50.861128+0000","last_active":"2023-10-06T05:21:50.861128+0000","last_peered":"2023-10-06T05:21:50.861128+0000","last_clean":"2023-10-06T05:21:50.861128+0000","last_became_active":"2023-10-06T05:21:50.860886+0000","last_became_peered":"2023-10-06T05:21:50.860886+0000","last_unstale":"2023-10-06T05:21:50.861128+0000","last_undegraded":"2023-10-06T05:21:50.861128+0000","last_fullsized":"2023-10-06T05:21:50.861128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:05.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"91'1","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.856722+0000","last_change":"2023-10-06T05:21:50.348213+0000","last_active":"2023-10-06T05:21:50.856722+0000","last_peered":"2023-10-06T05:21:50.856722+0000","last_clean":"2023-10-06T05:21:50.856722+0000","last_became_active":"2023-10-06T05:21:50.347763+0000","last_became_peered":"2023-10-06T05:21:50.347763+0000","last_unstale":"2023-10-06T05:21:50.856722+0000","last_undegraded":"2023-10-06T05:21:50.856722+0000","last_fullsized":"2023-10-06T05:21:50.856722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:29.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.870796+0000","last_change":"2023-10-06T05:21:50.868575+0000","last_active":"2023-10-06T05:21:50.870796+0000","last_peered":"2023-10-06T05:21:50.870796+0000","last_clean":"2023-10-06T05:21:50.870796+0000","last_became_active":"2023-10-06T05:21:50.868313+0000","last_became_peered":"2023-10-06T05:21:50.868313+0000","last_unstale":"2023-10-06T05:21:50.870796+0000","last_undegraded":"2023-10-06T05:21:50.870796+0000","last_fullsized":"2023-10-06T05:21:50.870796+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:45.717062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.343321+0000","last_change":"2023-10-06T05:21:50.352216+0000","last_active":"2023-10-06T05:21:51.343321+0000","last_peered":"2023-10-06T05:21:51.343321+0000","last_clean":"2023-10-06T05:21:51.343321+0000","last_became_active":"2023-10-06T05:21:50.351925+0000","last_became_peered":"2023-10-06T05:21:50.351925+0000","last_unstale":"2023-10-06T05:21:51.343321+0000","last_undegraded":"2023-10-06T05:21:51.343321+0000","last_fullsized":"2023-10-06T05:21:51.343321+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:19.576336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868276+0000","last_change":"2023-10-06T05:21:50.868276+0000","last_active":"2023-10-06T05:21:50.868276+0000","last_peered":"2023-10-06T05:21:50.868276+0000","last_clean":"2023-10-06T05:21:50.868276+0000","last_became_active":"2023-10-06T05:21:50.868009+0000","last_became_peered":"2023-10-06T05:21:50.868009+0000","last_unstale":"2023-10-06T05:21:50.868276+0000","last_undegraded":"2023-10-06T05:21:50.868276+0000","last_fullsized":"2023-10-06T05:21:50.868276+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:47.407960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.857642+0000","last_change":"2023-10-06T05:21:50.346181+0000","last_active":"2023-10-06T05:21:50.857642+0000","last_peered":"2023-10-06T05:21:50.857642+0000","last_clean":"2023-10-06T05:21:50.857642+0000","last_became_active":"2023-10-06T05:21:50.345904+0000","last_became_peered":"2023-10-06T05:21:50.345904+0000","last_unstale":"2023-10-06T05:21:50.857642+0000","last_undegraded":"2023-10-06T05:21:50.857642+0000","last_fullsized":"2023-10-06T05:21:50.857642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:51.452290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868445+0000","last_change":"2023-10-06T05:21:50.865683+0000","last_active":"2023-10-06T05:21:50.868445+0000","last_peered":"2023-10-06T05:21:50.868445+0000","last_clean":"2023-10-06T05:21:50.868445+0000","last_became_active":"2023-10-06T05:21:50.865414+0000","last_became_peered":"2023-10-06T05:21:50.865414+0000","last_unstale":"2023-10-06T05:21:50.868445+0000","last_undegraded":"2023-10-06T05:21:50.868445+0000","last_fullsized":"2023-10-06T05:21:50.868445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:44.639551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.356027+0000","last_change":"2023-10-06T05:21:50.354788+0000","last_active":"2023-10-06T05:21:50.356027+0000","last_peered":"2023-10-06T05:21:50.356027+0000","last_clean":"2023-10-06T05:21:50.356027+0000","last_became_active":"2023-10-06T05:21:50.354156+0000","last_became_peered":"2023-10-06T05:21:50.354156+0000","last_unstale":"2023-10-06T05:21:50.356027+0000","last_undegraded":"2023-10-06T05:21:50.356027+0000","last_fullsized":"2023-10-06T05:21:50.356027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:24.178488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875487+0000","last_change":"2023-10-06T05:21:50.870489+0000","last_active":"2023-10-06T05:21:50.875487+0000","last_peered":"2023-10-06T05:21:50.875487+0000","last_clean":"2023-10-06T05:21:50.875487+0000","last_became_active":"2023-10-06T05:21:50.869554+0000","last_became_peered":"2023-10-06T05:21:50.869554+0000","last_unstale":"2023-10-06T05:21:50.875487+0000","last_undegraded":"2023-10-06T05:21:50.875487+0000","last_fullsized":"2023-10-06T05:21:50.875487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:10.672361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.348860+0000","last_change":"2023-10-06T05:21:50.342823+0000","last_active":"2023-10-06T05:21:50.348860+0000","last_peered":"2023-10-06T05:21:50.348860+0000","last_clean":"2023-10-06T05:21:50.348860+0000","last_became_active":"2023-10-06T05:21:50.342607+0000","last_became_peered":"2023-10-06T05:21:50.342607+0000","last_unstale":"2023-10-06T05:21:50.348860+0000","last_undegraded":"2023-10-06T05:21:50.348860+0000","last_fullsized":"2023-10-06T05:21:50.348860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:44.279177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869542+0000","last_change":"2023-10-06T05:21:50.865972+0000","last_active":"2023-10-06T05:21:50.869542+0000","last_peered":"2023-10-06T05:21:50.869542+0000","last_clean":"2023-10-06T05:21:50.869542+0000","last_became_active":"2023-10-06T05:21:50.865684+0000","last_became_peered":"2023-10-06T05:21:50.865684+0000","last_unstale":"2023-10-06T05:21:50.869542+0000","last_undegraded":"2023-10-06T05:21:50.869542+0000","last_fullsized":"2023-10-06T05:21:50.869542+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:10.249360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.356624+0000","last_change":"2023-10-06T05:21:50.355420+0000","last_active":"2023-10-06T05:21:50.356624+0000","last_peered":"2023-10-06T05:21:50.356624+0000","last_clean":"2023-10-06T05:21:50.356624+0000","last_became_active":"2023-10-06T05:21:50.355157+0000","last_became_peered":"2023-10-06T05:21:50.355157+0000","last_unstale":"2023-10-06T05:21:50.356624+0000","last_undegraded":"2023-10-06T05:21:50.356624+0000","last_fullsized":"2023-10-06T05:21:50.356624+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:08.033628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880773+0000","last_change":"2023-10-06T05:21:50.874279+0000","last_active":"2023-10-06T05:21:50.880773+0000","last_peered":"2023-10-06T05:21:50.880773+0000","last_clean":"2023-10-06T05:21:50.880773+0000","last_became_active":"2023-10-06T05:21:50.874021+0000","last_became_peered":"2023-10-06T05:21:50.874021+0000","last_unstale":"2023-10-06T05:21:50.880773+0000","last_undegraded":"2023-10-06T05:21:50.880773+0000","last_fullsized":"2023-10-06T05:21:50.880773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:47.407960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"91'3","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.342890+0000","last_change":"2023-10-06T05:21:50.341544+0000","last_active":"2023-10-06T05:21:51.342890+0000","last_peered":"2023-10-06T05:21:51.342890+0000","last_clean":"2023-10-06T05:21:51.342890+0000","last_became_active":"2023-10-06T05:21:50.341257+0000","last_became_peered":"2023-10-06T05:21:50.341257+0000","last_unstale":"2023-10-06T05:21:51.342890+0000","last_undegraded":"2023-10-06T05:21:51.342890+0000","last_fullsized":"2023-10-06T05:21:51.342890+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:00.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.880716+0000","last_change":"2023-10-06T05:21:50.873370+0000","last_active":"2023-10-06T05:21:50.880716+0000","last_peered":"2023-10-06T05:21:50.880716+0000","last_clean":"2023-10-06T05:21:50.880716+0000","last_became_active":"2023-10-06T05:21:50.873067+0000","last_became_peered":"2023-10-06T05:21:50.873067+0000","last_unstale":"2023-10-06T05:21:50.880716+0000","last_undegraded":"2023-10-06T05:21:50.880716+0000","last_fullsized":"2023-10-06T05:21:50.880716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:57.081955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"91'6","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.347943+0000","last_change":"2023-10-06T05:21:50.348092+0000","last_active":"2023-10-06T05:21:51.347943+0000","last_peered":"2023-10-06T05:21:51.347943+0000","last_clean":"2023-10-06T05:21:51.347943+0000","last_became_active":"2023-10-06T05:21:50.347608+0000","last_became_peered":"2023-10-06T05:21:50.347608+0000","last_unstale":"2023-10-06T05:21:51.347943+0000","last_undegraded":"2023-10-06T05:21:51.347943+0000","last_fullsized":"2023-10-06T05:21:51.347943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:52.964802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.863771+0000","last_change":"2023-10-06T05:21:50.863771+0000","last_active":"2023-10-06T05:21:50.863771+0000","last_peered":"2023-10-06T05:21:50.863771+0000","last_clean":"2023-10-06T05:21:50.863771+0000","last_became_active":"2023-10-06T05:21:50.863353+0000","last_became_peered":"2023-10-06T05:21:50.863353+0000","last_unstale":"2023-10-06T05:21:50.863771+0000","last_undegraded":"2023-10-06T05:21:50.863771+0000","last_fullsized":"2023-10-06T05:21:50.863771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:43.224022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"91'2","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340897+0000","last_change":"2023-10-06T05:21:50.342430+0000","last_active":"2023-10-06T05:21:51.340897+0000","last_peered":"2023-10-06T05:21:51.340897+0000","last_clean":"2023-10-06T05:21:51.340897+0000","last_became_active":"2023-10-06T05:21:50.342128+0000","last_became_peered":"2023-10-06T05:21:50.342128+0000","last_unstale":"2023-10-06T05:21:51.340897+0000","last_undegraded":"2023-10-06T05:21:51.340897+0000","last_fullsized":"2023-10-06T05:21:51.340897+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:48.883232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.871747+0000","last_change":"2023-10-06T05:21:50.871747+0000","last_active":"2023-10-06T05:21:50.871747+0000","last_peered":"2023-10-06T05:21:50.871747+0000","last_clean":"2023-10-06T05:21:50.871747+0000","last_became_active":"2023-10-06T05:21:50.871535+0000","last_became_peered":"2023-10-06T05:21:50.871535+0000","last_unstale":"2023-10-06T05:21:50.871747+0000","last_undegraded":"2023-10-06T05:21:50.871747+0000","last_fullsized":"2023-10-06T05:21:50.871747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:51.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.353572+0000","last_change":"2023-10-06T05:21:50.349760+0000","last_active":"2023-10-06T05:21:50.353572+0000","last_peered":"2023-10-06T05:21:50.353572+0000","last_clean":"2023-10-06T05:21:50.353572+0000","last_became_active":"2023-10-06T05:21:50.349483+0000","last_became_peered":"2023-10-06T05:21:50.349483+0000","last_unstale":"2023-10-06T05:21:50.353572+0000","last_undegraded":"2023-10-06T05:21:50.353572+0000","last_fullsized":"2023-10-06T05:21:50.353572+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:09.918824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869568+0000","last_change":"2023-10-06T05:21:50.866451+0000","last_active":"2023-10-06T05:21:50.869568+0000","last_peered":"2023-10-06T05:21:50.869568+0000","last_clean":"2023-10-06T05:21:50.869568+0000","last_became_active":"2023-10-06T05:21:50.866190+0000","last_became_peered":"2023-10-06T05:21:50.866190+0000","last_unstale":"2023-10-06T05:21:50.869568+0000","last_undegraded":"2023-10-06T05:21:50.869568+0000","last_fullsized":"2023-10-06T05:21:50.869568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:46.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.855730+0000","last_change":"2023-10-06T05:21:50.341652+0000","last_active":"2023-10-06T05:21:50.855730+0000","last_peered":"2023-10-06T05:21:50.855730+0000","last_clean":"2023-10-06T05:21:50.855730+0000","last_became_active":"2023-10-06T05:21:50.341148+0000","last_became_peered":"2023-10-06T05:21:50.341148+0000","last_unstale":"2023-10-06T05:21:50.855730+0000","last_undegraded":"2023-10-06T05:21:50.855730+0000","last_fullsized":"2023-10-06T05:21:50.855730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:23.803079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868683+0000","last_change":"2023-10-06T05:21:50.868683+0000","last_active":"2023-10-06T05:21:50.868683+0000","last_peered":"2023-10-06T05:21:50.868683+0000","last_clean":"2023-10-06T05:21:50.868683+0000","last_became_active":"2023-10-06T05:21:50.868401+0000","last_became_peered":"2023-10-06T05:21:50.868401+0000","last_unstale":"2023-10-06T05:21:50.868683+0000","last_undegraded":"2023-10-06T05:21:50.868683+0000","last_fullsized":"2023-10-06T05:21:50.868683+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:09.038999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"91'8","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.345433+0000","last_change":"2023-10-06T05:21:50.349367+0000","last_active":"2023-10-06T05:21:51.345433+0000","last_peered":"2023-10-06T05:21:51.345433+0000","last_clean":"2023-10-06T05:21:51.345433+0000","last_became_active":"2023-10-06T05:21:50.348812+0000","last_became_peered":"2023-10-06T05:21:50.348812+0000","last_unstale":"2023-10-06T05:21:51.345433+0000","last_undegraded":"2023-10-06T05:21:51.345433+0000","last_fullsized":"2023-10-06T05:21:51.345433+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:37.705038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875524+0000","last_change":"2023-10-06T05:21:50.870828+0000","last_active":"2023-10-06T05:21:50.875524+0000","last_peered":"2023-10-06T05:21:50.875524+0000","last_clean":"2023-10-06T05:21:50.875524+0000","last_became_active":"2023-10-06T05:21:50.870202+0000","last_became_peered":"2023-10-06T05:21:50.870202+0000","last_unstale":"2023-10-06T05:21:50.875524+0000","last_undegraded":"2023-10-06T05:21:50.875524+0000","last_fullsized":"2023-10-06T05:21:50.875524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:44.639551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"91'2","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340385+0000","last_change":"2023-10-06T05:21:50.336815+0000","last_active":"2023-10-06T05:21:51.340385+0000","last_peered":"2023-10-06T05:21:51.340385+0000","last_clean":"2023-10-06T05:21:51.340385+0000","last_became_active":"2023-10-06T05:21:50.336521+0000","last_became_peered":"2023-10-06T05:21:50.336521+0000","last_unstale":"2023-10-06T05:21:51.340385+0000","last_undegraded":"2023-10-06T05:21:51.340385+0000","last_fullsized":"2023-10-06T05:21:51.340385+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:28.089779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869375+0000","last_change":"2023-10-06T05:21:50.869375+0000","last_active":"2023-10-06T05:21:50.869375+0000","last_peered":"2023-10-06T05:21:50.869375+0000","last_clean":"2023-10-06T05:21:50.869375+0000","last_became_active":"2023-10-06T05:21:50.869084+0000","last_became_peered":"2023-10-06T05:21:50.869084+0000","last_unstale":"2023-10-06T05:21:50.869375+0000","last_undegraded":"2023-10-06T05:21:50.869375+0000","last_fullsized":"2023-10-06T05:21:50.869375+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:46.930766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.854716+0000","last_change":"2023-10-06T05:21:50.350952+0000","last_active":"2023-10-06T05:21:50.854716+0000","last_peered":"2023-10-06T05:21:50.854716+0000","last_clean":"2023-10-06T05:21:50.854716+0000","last_became_active":"2023-10-06T05:21:50.350605+0000","last_became_peered":"2023-10-06T05:21:50.350605+0000","last_unstale":"2023-10-06T05:21:50.854716+0000","last_undegraded":"2023-10-06T05:21:50.854716+0000","last_fullsized":"2023-10-06T05:21:50.854716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:55.041658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869662+0000","last_change":"2023-10-06T05:21:50.869662+0000","last_active":"2023-10-06T05:21:50.869662+0000","last_peered":"2023-10-06T05:21:50.869662+0000","last_clean":"2023-10-06T05:21:50.869662+0000","last_became_active":"2023-10-06T05:21:50.869374+0000","last_became_peered":"2023-10-06T05:21:50.869374+0000","last_unstale":"2023-10-06T05:21:50.869662+0000","last_undegraded":"2023-10-06T05:21:50.869662+0000","last_fullsized":"2023-10-06T05:21:50.869662+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:29.095150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852182+0000","last_change":"2023-10-06T05:21:50.337563+0000","last_active":"2023-10-06T05:21:50.852182+0000","last_peered":"2023-10-06T05:21:50.852182+0000","last_clean":"2023-10-06T05:21:50.852182+0000","last_became_active":"2023-10-06T05:21:50.337244+0000","last_became_peered":"2023-10-06T05:21:50.337244+0000","last_unstale":"2023-10-06T05:21:50.852182+0000","last_undegraded":"2023-10-06T05:21:50.852182+0000","last_fullsized":"2023-10-06T05:21:50.852182+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:00.076324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872282+0000","last_change":"2023-10-06T05:21:50.867574+0000","last_active":"2023-10-06T05:21:50.872282+0000","last_peered":"2023-10-06T05:21:50.872282+0000","last_clean":"2023-10-06T05:21:50.872282+0000","last_became_active":"2023-10-06T05:21:50.867301+0000","last_became_peered":"2023-10-06T05:21:50.867301+0000","last_unstale":"2023-10-06T05:21:50.872282+0000","last_undegraded":"2023-10-06T05:21:50.872282+0000","last_fullsized":"2023-10-06T05:21:50.872282+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:57.081955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.858015+0000","last_change":"2023-10-06T05:21:50.344219+0000","last_active":"2023-10-06T05:21:50.858015+0000","last_peered":"2023-10-06T05:21:50.858015+0000","last_clean":"2023-10-06T05:21:50.858015+0000","last_became_active":"2023-10-06T05:21:50.343983+0000","last_became_peered":"2023-10-06T05:21:50.343983+0000","last_unstale":"2023-10-06T05:21:50.858015+0000","last_undegraded":"2023-10-06T05:21:50.858015+0000","last_fullsized":"2023-10-06T05:21:50.858015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:00.393363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.871034+0000","last_change":"2023-10-06T05:21:50.871034+0000","last_active":"2023-10-06T05:21:50.871034+0000","last_peered":"2023-10-06T05:21:50.871034+0000","last_clean":"2023-10-06T05:21:50.871034+0000","last_became_active":"2023-10-06T05:21:50.870739+0000","last_became_peered":"2023-10-06T05:21:50.870739+0000","last_unstale":"2023-10-06T05:21:50.871034+0000","last_undegraded":"2023-10-06T05:21:50.871034+0000","last_fullsized":"2023-10-06T05:21:50.871034+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:51.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852129+0000","last_change":"2023-10-06T05:21:50.348854+0000","last_active":"2023-10-06T05:21:50.852129+0000","last_peered":"2023-10-06T05:21:50.852129+0000","last_clean":"2023-10-06T05:21:50.852129+0000","last_became_active":"2023-10-06T05:21:50.348389+0000","last_became_peered":"2023-10-06T05:21:50.348389+0000","last_unstale":"2023-10-06T05:21:50.852129+0000","last_undegraded":"2023-10-06T05:21:50.852129+0000","last_fullsized":"2023-10-06T05:21:50.852129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:30.609866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868968+0000","last_change":"2023-10-06T05:21:50.865113+0000","last_active":"2023-10-06T05:21:50.868968+0000","last_peered":"2023-10-06T05:21:50.868968+0000","last_clean":"2023-10-06T05:21:50.868968+0000","last_became_active":"2023-10-06T05:21:50.864828+0000","last_became_peered":"2023-10-06T05:21:50.864828+0000","last_unstale":"2023-10-06T05:21:50.868968+0000","last_undegraded":"2023-10-06T05:21:50.868968+0000","last_fullsized":"2023-10-06T05:21:50.868968+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:10.322844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.861081+0000","last_change":"2023-10-06T05:21:50.349369+0000","last_active":"2023-10-06T05:21:50.861081+0000","last_peered":"2023-10-06T05:21:50.861081+0000","last_clean":"2023-10-06T05:21:50.861081+0000","last_became_active":"2023-10-06T05:21:50.349133+0000","last_became_peered":"2023-10-06T05:21:50.349133+0000","last_unstale":"2023-10-06T05:21:50.861081+0000","last_undegraded":"2023-10-06T05:21:50.861081+0000","last_fullsized":"2023-10-06T05:21:50.861081+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:07.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868993+0000","last_change":"2023-10-06T05:21:50.868993+0000","last_active":"2023-10-06T05:21:50.868993+0000","last_peered":"2023-10-06T05:21:50.868993+0000","last_clean":"2023-10-06T05:21:50.868993+0000","last_became_active":"2023-10-06T05:21:50.868558+0000","last_became_peered":"2023-10-06T05:21:50.868558+0000","last_unstale":"2023-10-06T05:21:50.868993+0000","last_undegraded":"2023-10-06T05:21:50.868993+0000","last_fullsized":"2023-10-06T05:21:50.868993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.112440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.354338+0000","last_change":"2023-10-06T05:21:50.352199+0000","last_active":"2023-10-06T05:21:50.354338+0000","last_peered":"2023-10-06T05:21:50.354338+0000","last_clean":"2023-10-06T05:21:50.354338+0000","last_became_active":"2023-10-06T05:21:50.351953+0000","last_became_peered":"2023-10-06T05:21:50.351953+0000","last_unstale":"2023-10-06T05:21:50.354338+0000","last_undegraded":"2023-10-06T05:21:50.354338+0000","last_fullsized":"2023-10-06T05:21:50.354338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:15.761398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872587+0000","last_change":"2023-10-06T05:21:50.869686+0000","last_active":"2023-10-06T05:21:50.872587+0000","last_peered":"2023-10-06T05:21:50.872587+0000","last_clean":"2023-10-06T05:21:50.872587+0000","last_became_active":"2023-10-06T05:21:50.869079+0000","last_became_peered":"2023-10-06T05:21:50.869079+0000","last_unstale":"2023-10-06T05:21:50.872587+0000","last_undegraded":"2023-10-06T05:21:50.872587+0000","last_fullsized":"2023-10-06T05:21:50.872587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:24.808450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.343571+0000","last_change":"2023-10-06T05:21:50.342350+0000","last_active":"2023-10-06T05:21:50.343571+0000","last_peered":"2023-10-06T05:21:50.343571+0000","last_clean":"2023-10-06T05:21:50.343571+0000","last_became_active":"2023-10-06T05:21:50.342011+0000","last_became_peered":"2023-10-06T05:21:50.342011+0000","last_unstale":"2023-10-06T05:21:50.343571+0000","last_undegraded":"2023-10-06T05:21:50.343571+0000","last_fullsized":"2023-10-06T05:21:50.343571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:21.230000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872568+0000","last_change":"2023-10-06T05:21:50.869587+0000","last_active":"2023-10-06T05:21:50.872568+0000","last_peered":"2023-10-06T05:21:50.872568+0000","last_clean":"2023-10-06T05:21:50.872568+0000","last_became_active":"2023-10-06T05:21:50.868698+0000","last_became_peered":"2023-10-06T05:21:50.868698+0000","last_unstale":"2023-10-06T05:21:50.872568+0000","last_undegraded":"2023-10-06T05:21:50.872568+0000","last_fullsized":"2023-10-06T05:21:50.872568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:53.970173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852357+0000","last_change":"2023-10-06T05:21:50.352343+0000","last_active":"2023-10-06T05:21:50.852357+0000","last_peered":"2023-10-06T05:21:50.852357+0000","last_clean":"2023-10-06T05:21:50.852357+0000","last_became_active":"2023-10-06T05:21:50.352068+0000","last_became_peered":"2023-10-06T05:21:50.352068+0000","last_unstale":"2023-10-06T05:21:50.852357+0000","last_undegraded":"2023-10-06T05:21:50.852357+0000","last_fullsized":"2023-10-06T05:21:50.852357+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:07.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.348108+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:21:50.348108+0000","last_peered":"2023-10-06T05:21:50.348108+0000","last_clean":"2023-10-06T05:21:50.348108+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:21:50.348108+0000","last_undegraded":"2023-10-06T05:21:50.348108+0000","last_fullsized":"2023-10-06T05:21:50.348108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.872531+0000","last_change":"2023-10-06T05:21:50.869416+0000","last_active":"2023-10-06T05:21:50.872531+0000","last_peered":"2023-10-06T05:21:50.872531+0000","last_clean":"2023-10-06T05:21:50.872531+0000","last_became_active":"2023-10-06T05:21:50.868501+0000","last_became_peered":"2023-10-06T05:21:50.868501+0000","last_unstale":"2023-10-06T05:21:50.872531+0000","last_undegraded":"2023-10-06T05:21:50.872531+0000","last_fullsized":"2023-10-06T05:21:50.872531+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:56.047029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340070+0000","last_change":"2023-10-06T05:21:50.343236+0000","last_active":"2023-10-06T05:21:51.340070+0000","last_peered":"2023-10-06T05:21:51.340070+0000","last_clean":"2023-10-06T05:21:51.340070+0000","last_became_active":"2023-10-06T05:21:50.342974+0000","last_became_peered":"2023-10-06T05:21:50.342974+0000","last_unstale":"2023-10-06T05:21:51.340070+0000","last_undegraded":"2023-10-06T05:21:51.340070+0000","last_fullsized":"2023-10-06T05:21:51.340070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:56.076584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.333116+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:21:50.333116+0000","last_peered":"2023-10-06T05:21:50.333116+0000","last_clean":"2023-10-06T05:21:50.333116+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:21:50.333116+0000","last_undegraded":"2023-10-06T05:21:50.333116+0000","last_fullsized":"2023-10-06T05:21:50.333116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868989+0000","last_change":"2023-10-06T05:21:50.868989+0000","last_active":"2023-10-06T05:21:50.868989+0000","last_peered":"2023-10-06T05:21:50.868989+0000","last_clean":"2023-10-06T05:21:50.868989+0000","last_became_active":"2023-10-06T05:21:50.868752+0000","last_became_peered":"2023-10-06T05:21:50.868752+0000","last_unstale":"2023-10-06T05:21:50.868989+0000","last_undegraded":"2023-10-06T05:21:50.868989+0000","last_fullsized":"2023-10-06T05:21:50.868989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:11.307845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.340291+0000","last_change":"2023-10-06T05:21:50.340299+0000","last_active":"2023-10-06T05:21:51.340291+0000","last_peered":"2023-10-06T05:21:51.340291+0000","last_clean":"2023-10-06T05:21:51.340291+0000","last_became_active":"2023-10-06T05:21:50.340039+0000","last_became_peered":"2023-10-06T05:21:50.340039+0000","last_unstale":"2023-10-06T05:21:51.340291+0000","last_undegraded":"2023-10-06T05:21:51.340291+0000","last_fullsized":"2023-10-06T05:21:51.340291+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:00.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.334807+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:21:50.334807+0000","last_peered":"2023-10-06T05:21:50.334807+0000","last_clean":"2023-10-06T05:21:50.334807+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:21:50.334807+0000","last_undegraded":"2023-10-06T05:21:50.334807+0000","last_fullsized":"2023-10-06T05:21:50.334807+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.869925+0000","last_change":"2023-10-06T05:21:50.869925+0000","last_active":"2023-10-06T05:21:50.869925+0000","last_peered":"2023-10-06T05:21:50.869925+0000","last_clean":"2023-10-06T05:21:50.869925+0000","last_became_active":"2023-10-06T05:21:50.869637+0000","last_became_peered":"2023-10-06T05:21:50.869637+0000","last_unstale":"2023-10-06T05:21:50.869925+0000","last_undegraded":"2023-10-06T05:21:50.869925+0000","last_fullsized":"2023-10-06T05:21:50.869925+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:13.791546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"91'1","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.339582+0000","last_change":"2023-10-06T05:21:50.346294+0000","last_active":"2023-10-06T05:21:51.339582+0000","last_peered":"2023-10-06T05:21:51.339582+0000","last_clean":"2023-10-06T05:21:51.339582+0000","last_became_active":"2023-10-06T05:21:50.346033+0000","last_became_peered":"2023-10-06T05:21:50.346033+0000","last_unstale":"2023-10-06T05:21:51.339582+0000","last_undegraded":"2023-10-06T05:21:51.339582+0000","last_fullsized":"2023-10-06T05:21:51.339582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:12.786175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.333396+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:21:50.333396+0000","last_peered":"2023-10-06T05:21:50.333396+0000","last_clean":"2023-10-06T05:21:50.333396+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:21:50.333396+0000","last_undegraded":"2023-10-06T05:21:50.333396+0000","last_fullsized":"2023-10-06T05:21:50.333396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":224,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852150+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:21:50.852150+0000","last_peered":"2023-10-06T05:21:50.852150+0000","last_clean":"2023-10-06T05:21:50.852150+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:21:50.852150+0000","last_undegraded":"2023-10-06T05:21:50.852150+0000","last_fullsized":"2023-10-06T05:21:50.852150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.862496+0000","last_change":"2023-10-06T05:21:50.858627+0000","last_active":"2023-10-06T05:21:50.862496+0000","last_peered":"2023-10-06T05:21:50.862496+0000","last_clean":"2023-10-06T05:21:50.862496+0000","last_became_active":"2023-10-06T05:21:50.858366+0000","last_became_peered":"2023-10-06T05:21:50.858366+0000","last_unstale":"2023-10-06T05:21:50.862496+0000","last_undegraded":"2023-10-06T05:21:50.862496+0000","last_fullsized":"2023-10-06T05:21:50.862496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:24.808450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852303+0000","last_change":"2023-10-06T05:21:50.347150+0000","last_active":"2023-10-06T05:21:50.852303+0000","last_peered":"2023-10-06T05:21:50.852303+0000","last_clean":"2023-10-06T05:21:50.852303+0000","last_became_active":"2023-10-06T05:21:50.339261+0000","last_became_peered":"2023-10-06T05:21:50.339261+0000","last_unstale":"2023-10-06T05:21:50.852303+0000","last_undegraded":"2023-10-06T05:21:50.852303+0000","last_fullsized":"2023-10-06T05:21:50.852303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:44.711691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852241+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:21:50.852241+0000","last_peered":"2023-10-06T05:21:50.852241+0000","last_clean":"2023-10-06T05:21:50.852241+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:21:50.852241+0000","last_undegraded":"2023-10-06T05:21:50.852241+0000","last_fullsized":"2023-10-06T05:21:50.852241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868816+0000","last_change":"2023-10-06T05:21:50.868816+0000","last_active":"2023-10-06T05:21:50.868816+0000","last_peered":"2023-10-06T05:21:50.868816+0000","last_clean":"2023-10-06T05:21:50.868816+0000","last_became_active":"2023-10-06T05:21:50.868199+0000","last_became_peered":"2023-10-06T05:21:50.868199+0000","last_unstale":"2023-10-06T05:21:50.868816+0000","last_undegraded":"2023-10-06T05:21:50.868816+0000","last_fullsized":"2023-10-06T05:21:50.868816+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:05.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.346504+0000","last_change":"2023-10-06T05:21:50.345538+0000","last_active":"2023-10-06T05:21:50.346504+0000","last_peered":"2023-10-06T05:21:50.346504+0000","last_clean":"2023-10-06T05:21:50.346504+0000","last_became_active":"2023-10-06T05:21:50.345284+0000","last_became_peered":"2023-10-06T05:21:50.345284+0000","last_unstale":"2023-10-06T05:21:50.346504+0000","last_undegraded":"2023-10-06T05:21:50.346504+0000","last_fullsized":"2023-10-06T05:21:50.346504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:00.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.852935+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:21:50.852935+0000","last_peered":"2023-10-06T05:21:50.852935+0000","last_clean":"2023-10-06T05:21:50.852935+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:21:50.852935+0000","last_undegraded":"2023-10-06T05:21:50.852935+0000","last_fullsized":"2023-10-06T05:21:50.852935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.875690+0000","last_change":"2023-10-06T05:21:50.875690+0000","last_active":"2023-10-06T05:21:50.875690+0000","last_peered":"2023-10-06T05:21:50.875690+0000","last_clean":"2023-10-06T05:21:50.875690+0000","last_became_active":"2023-10-06T05:21:50.875407+0000","last_became_peered":"2023-10-06T05:21:50.875407+0000","last_unstale":"2023-10-06T05:21:50.875690+0000","last_undegraded":"2023-10-06T05:21:50.875690+0000","last_fullsized":"2023-10-06T05:21:50.875690+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:52.457661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.349112+0000","last_change":"2023-10-06T05:21:50.348023+0000","last_active":"2023-10-06T05:21:50.349112+0000","last_peered":"2023-10-06T05:21:50.349112+0000","last_clean":"2023-10-06T05:21:50.349112+0000","last_became_active":"2023-10-06T05:21:50.347746+0000","last_became_peered":"2023-10-06T05:21:50.347746+0000","last_unstale":"2023-10-06T05:21:50.349112+0000","last_undegraded":"2023-10-06T05:21:50.349112+0000","last_fullsized":"2023-10-06T05:21:50.349112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:51.452290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.348235+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:21:50.348235+0000","last_peered":"2023-10-06T05:21:50.348235+0000","last_clean":"2023-10-06T05:21:50.348235+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:21:50.348235+0000","last_undegraded":"2023-10-06T05:21:50.348235+0000","last_fullsized":"2023-10-06T05:21:50.348235+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.868597+0000","last_change":"2023-10-06T05:21:50.868597+0000","last_active":"2023-10-06T05:21:50.868597+0000","last_peered":"2023-10-06T05:21:50.868597+0000","last_clean":"2023-10-06T05:21:50.868597+0000","last_became_active":"2023-10-06T05:21:50.867951+0000","last_became_peered":"2023-10-06T05:21:50.867951+0000","last_unstale":"2023-10-06T05:21:50.868597+0000","last_undegraded":"2023-10-06T05:21:50.868597+0000","last_fullsized":"2023-10-06T05:21:50.868597+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:50.319194+0000","last_clean_scrub_stamp":"2023-10-06T05:21:50.319194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:14:16.002507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:51.342438+0000","last_change":"2023-10-06T05:21:50.351076+0000","last_active":"2023-10-06T05:21:51.342438+0000","last_peered":"2023-10-06T05:21:51.342438+0000","last_clean":"2023-10-06T05:21:51.342438+0000","last_became_active":"2023-10-06T05:21:50.350769+0000","last_became_peered":"2023-10-06T05:21:50.350769+0000","last_unstale":"2023-10-06T05:21:51.342438+0000","last_undegraded":"2023-10-06T05:21:51.342438+0000","last_fullsized":"2023-10-06T05:21:51.342438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:49.313823+0000","last_clean_scrub_stamp":"2023-10-06T05:21:49.313823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:46.402589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T05:21:50.860840+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:21:50.860840+0000","last_peered":"2023-10-06T05:21:50.860840+0000","last_clean":"2023-10-06T05:21:50.860840+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:21:50.860840+0000","last_undegraded":"2023-10-06T05:21:50.860840+0000","last_fullsized":"2023-10-06T05:21:50.860840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298972,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072868,"statfs":{"total":96636764160,"available":96330616832,"internally_reserved":0,"allocated":1724416,"data_stored":1131832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.375,"5min":0.307,"15min":0.307},"max":{"1min":0.752,"5min":0.878,"15min":0.878},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61399999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76900000000000002,"5min":0.79500000000000004,"15min":0.92100000000000004},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.61199999999999999},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72799999999999998,"5min":0.92300000000000004,"15min":1.476},"last":0.496},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.86299999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.81599999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.626},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83299999999999996,"5min":0.86599999999999999,"15min":1.5129999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.63400000000000001},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79200000000000004,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64900000000000002,"15min":0.66900000000000004},"min":{"1min":0.51300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.439},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.501},"min":{"1min":0.34899999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.93200000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.505,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71699999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.752}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.87,"15min":0.92900000000000005},"last":0.78300000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298404,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073436,"statfs":{"total":96636764160,"available":96331198464,"internally_reserved":0,"allocated":1142784,"data_stored":548937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56399999999999995,"15min":0.71899999999999997},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.77100000000000002,"5min":0.79000000000000004,"15min":4.6900000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.61299999999999999,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.91500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75900000000000001,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.747},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":5.6239999999999997},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.96899999999999997},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80200000000000005,"5min":0.80500000000000005,"15min":11.595000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.78000000000000003},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78000000000000003,"5min":0.89900000000000002,"15min":5.5800000000000001},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":1.4199999999999999},"min":{"1min":0.43099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76200000000000001,"5min":0.78900000000000003,"15min":23.146000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.92500000000000004},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":9.7539999999999996},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46899999999999997,"15min":0.82499999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.68300000000000005,"15min":9.9969999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.85399999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":9.5850000000000009},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51800000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71199999999999997,"5min":0.77300000000000002,"15min":1.6439999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.85699999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.745,"5min":0.77900000000000003,"15min":9.8420000000000005},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55300000000000005,"15min":0.89100000000000001},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68999999999999995,"5min":0.82899999999999996,"15min":9.718},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57199999999999995,"15min":0.91600000000000004},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.749,"5min":0.85299999999999998,"15min":9.8930000000000007},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":541352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.52200000000000002,"5min":0.59699999999999998,"15min":0.64400000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.41499999999999998,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55900000000000005,"5min":0.67700000000000005,"15min":0.69599999999999995},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.437,"15min":0.441},"min":{"1min":0.308,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61499999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.42099999999999999,"15min":0.47499999999999998},"min":{"1min":0.29399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":1.673},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46600000000000003,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59799999999999998,"5min":0.752,"15min":4.0490000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.437,"15min":0.46100000000000002},"min":{"1min":0.308,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.77700000000000002},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.95199999999999996},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86699999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":0.91700000000000004,"15min":1.0369999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82599999999999996,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.91200000000000003,"5min":1.167,"15min":1.167},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83399999999999996,"5min":0.94699999999999995,"15min":1.026},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88400000000000001,"5min":0.92600000000000005,"15min":0.93100000000000005},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1732608,"data_stored":1139235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.65100000000000002,"5min":0.82799999999999996,"15min":1.038},"last":0.372},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70299999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48699999999999999,"15min":0.46200000000000002},"min":{"1min":0.34599999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":3.855,"15min":3.855},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.32500000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.621,"5min":0.91200000000000003,"15min":1.1919999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.44600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28499999999999998},"max":{"1min":0.623,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.57299999999999995,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.35599999999999998,"15min":0.32700000000000001},"max":{"1min":0.73399999999999999,"5min":7.7670000000000003,"15min":7.7670000000000003},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64600000000000002,"15min":0.61899999999999999},"min":{"1min":0.53100000000000003,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.628},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86499999999999999,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.621}]},{"osd":5,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71199999999999997,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":5.3600000000000003,"15min":5.3600000000000003},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.55900000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91200000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69699999999999995,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.0940000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.55400000000000005,"5min":0.442,"15min":0.437},"max":{"1min":0.93300000000000005,"5min":0.97399999999999998,"15min":1.0169999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":0.94499999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":541386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.42699999999999999},"min":{"1min":0.28100000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.51000000000000001,"5min":0.59299999999999997,"15min":2.0720000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40999999999999998,"15min":0.46200000000000002},"min":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":1.899},"last":0.34999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63,"5min":0.64300000000000002,"15min":2.097},"last":0.318},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.60599999999999998,"5min":0.71099999999999997,"15min":1.8759999999999999},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56699999999999995,"5min":0.66200000000000003,"15min":1.6719999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.51500000000000001},"min":{"1min":0.379,"5min":0.316,"15min":0.311},"max":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":2.121},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69299999999999995,"15min":0.70899999999999996},"min":{"1min":0.51000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.874,"5min":0.94199999999999995,"15min":1.7789999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.69399999999999995},"min":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.46700000000000003},"max":{"1min":0.998,"5min":1.0329999999999999,"15min":1.643},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92100000000000004,"5min":0.94399999999999995,"15min":2.0129999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.97299999999999998,"5min":0.97399999999999998,"15min":1.851},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.571},"last":0.751},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68300000000000005,"15min":0.70299999999999996},"min":{"1min":0.54600000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89100000000000001,"5min":0.98599999999999999,"15min":2.0449999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.90500000000000003,"5min":1.012,"15min":1.8180000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.48199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.613},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298396,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073444,"statfs":{"total":96636764160,"available":96331206656,"internally_reserved":0,"allocated":1134592,"data_stored":541557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"min":{"1min":0.25800000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.60399999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.38700000000000001},"min":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56599999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.318},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.433,"15min":0.41899999999999998},"min":{"1min":0.32300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71499999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.435,"15min":0.41999999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69899999999999995,"5min":0.755,"15min":0.755},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66100000000000003,"15min":0.63700000000000001},"min":{"1min":0.52700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.88300000000000001,"5min":1.143,"15min":1.143},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.64000000000000001},"min":{"1min":0.47299999999999998,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.628,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.40300000000000002,"15min":0.39900000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96799999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68000000000000005,"15min":0.66500000000000004},"min":{"1min":0.56100000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90300000000000002,"5min":1.236,"15min":1.236},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.47699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67300000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298404,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073436,"statfs":{"total":96636764160,"available":96331198464,"internally_reserved":0,"allocated":1142784,"data_stored":543012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.45300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.76400000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.47999999999999998,"5min":0.39800000000000002,"15min":0.33100000000000002},"max":{"1min":0.80800000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61099999999999999,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.32500000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79200000000000004,"5min":0.86599999999999999,"15min":0.89500000000000002},"last":0.55600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38},"max":{"1min":0.75,"5min":0.83899999999999997,"15min":0.89800000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.61299999999999999},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":2.0489999999999999},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.379,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.85499999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77800000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49199999999999999,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.32600000000000001},"max":{"1min":0.78600000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.372},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.5,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59199999999999997,"5min":0.76000000000000001,"15min":0.83799999999999997},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.64900000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.63800000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":0.79800000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073452,"statfs":{"total":96636764160,"available":96331214848,"internally_reserved":0,"allocated":1126400,"data_stored":541422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.29999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.56100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79300000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78500000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75800000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.876,"15min":0.876},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.496}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.28499999999999998},"max":{"1min":0.56999999999999995,"5min":0.68999999999999995,"15min":0.70999999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.32100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49099999999999999,"15min":0.47099999999999997},"min":{"1min":0.33400000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78600000000000003,"5min":0.876,"15min":0.876},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.502},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.73399999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83099999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:55.937 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:21:55.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:21:56.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.111+0000 7f50d5100700 1 Processor -- start 2023-10-06T05:21:56.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.111+0000 7f50d5100700 1 -- start start 2023-10-06T05:21:56.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50d5100700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50d013f900 0x7f50d00cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50d5100700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f50d00c9f20 0x7f50d00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50d5100700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d00c9630 con 0x7f50d00c8de0 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50d5100700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d00cb550 con 0x7f50d013f900 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50d5100700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d00cb830 con 0x7f50d00c9f20 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50ced9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50d013f900 0x7f50d00cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50ce59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f50d00c8de0 0x7f50d00c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39206/0 (socket says 172.21.15.45:39206) 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50ce59c700 1 -- 172.21.15.45:0/3222755766 learned_addr learned my addr 172.21.15.45:0/3222755766 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50ced9d700 1 -- 172.21.15.45:0/3222755766 >> v2:172.21.15.144:3300/0 conn(0x7f50d00c9f20 msgr2=0x7f50d00affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50cf59e700 1 --2- 172.21.15.45:0/3222755766 >> v2:172.21.15.144:3300/0 conn(0x7f50d00c9f20 0x7f50d00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50ced9d700 1 --2- 172.21.15.45:0/3222755766 >> v2:172.21.15.144:3300/0 conn(0x7f50d00c9f20 0x7f50d00affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50cdd9b700 1 -- 172.21.15.45:0/3222755766 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268177591 0 0) 0x7f50d00c9630 con 0x7f50d00c8de0 2023-10-06T05:21:56.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50ced9d700 1 -- 172.21.15.45:0/3222755766 >> v1:172.21.15.45:6789/0 conn(0x7f50d00c8de0 legacy=0x7f50d00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50ced9d700 1 -- 172.21.15.45:0/3222755766 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00cbb10 con 0x7f50d013f900 2023-10-06T05:21:56.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.112+0000 7f50cf59e700 1 --2- 172.21.15.45:0/3222755766 >> v2:172.21.15.144:3300/0 conn(0x7f50d00c9f20 0x7f50d00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:56.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50ced9d700 1 --2- 172.21.15.45:0/3222755766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d013f900 0x7f50d00cc840 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f50d00af960 tx=0x7f50b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d768ee74f70b95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50cdd9b700 1 -- 172.21.15.45:0/3222755766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800b6b0 con 0x7f50d013f900 2023-10-06T05:21:56.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50cdd9b700 1 -- 172.21.15.45:0/3222755766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b800b810 con 0x7f50d013f900 2023-10-06T05:21:56.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50cdd9b700 1 -- 172.21.15.45:0/3222755766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800bbb0 con 0x7f50d013f900 2023-10-06T05:21:56.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50d5100700 1 -- 172.21.15.45:0/3222755766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d013f900 msgr2=0x7f50d00cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50d5100700 1 --2- 172.21.15.45:0/3222755766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d013f900 0x7f50d00cc840 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f50d00af960 tx=0x7f50b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50d5100700 1 -- 172.21.15.45:0/3222755766 shutdown_connections 2023-10-06T05:21:56.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50d5100700 1 --2- 172.21.15.45:0/3222755766 >> v2:172.21.15.144:3300/0 conn(0x7f50d00c9f20 0x7f50d00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50d5100700 1 --2- 172.21.15.45:0/3222755766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d013f900 0x7f50d00cc840 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50d5100700 1 -- 172.21.15.45:0/3222755766 >> 172.21.15.45:0/3222755766 conn(0x7f50d00691c0 msgr2=0x7f50d00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:56.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.113+0000 7f50d5100700 1 -- 172.21.15.45:0/3222755766 shutdown_connections 2023-10-06T05:21:56.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.114+0000 7f50d5100700 1 -- 172.21.15.45:0/3222755766 wait complete. 2023-10-06T05:21:56.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.115+0000 7f50d5100700 1 Processor -- start 2023-10-06T05:21:56.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.115+0000 7f50d5100700 1 -- start start 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50d5100700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d00c8de0 0x7f50d00c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50d5100700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50d00c9f20 0x7f50d00c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50d5100700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50d013f900 0x7f50d0167160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50d5100700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d01598a0 con 0x7f50d00c8de0 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50d5100700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d015c360 con 0x7f50d013f900 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50d5100700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d0159af0 con 0x7f50d00c9f20 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d00c8de0 0x7f50d00c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d00c8de0 0x7f50d00c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36638/0 (socket says 172.21.15.45:36638) 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 -- 172.21.15.45:0/1315704326 learned_addr learned my addr 172.21.15.45:0/1315704326 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ce59c700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50d00c9f20 0x7f50d00c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50cf59e700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50d013f900 0x7f50d0167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 -- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50d00c9f20 msgr2=0x7f50d00c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50d00c9f20 0x7f50d00c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 -- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50d013f900 msgr2=0x7f50d0167160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50d013f900 0x7f50d0167160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 -- 172.21.15.45:0/1315704326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8009420 con 0x7f50d00c8de0 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50ced9d700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d00c8de0 0x7f50d00c4900 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f50d006e930 tx=0x7f50b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.116+0000 7f50c6ffd700 1 -- 172.21.15.45:0/1315704326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800bb90 con 0x7f50d00c8de0 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.117+0000 7f50c6ffd700 1 -- 172.21.15.45:0/1315704326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8011420 con 0x7f50d00c8de0 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.117+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d0159dd0 con 0x7f50d00c8de0 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.117+0000 7f50c6ffd700 1 -- 172.21.15.45:0/1315704326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80117e0 con 0x7f50d00c8de0 2023-10-06T05:21:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.117+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d015a390 con 0x7f50d00c8de0 2023-10-06T05:21:56.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.118+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d006e2d0 con 0x7f50d00c8de0 2023-10-06T05:21:56.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.118+0000 7f50c6ffd700 1 -- 172.21.15.45:0/1315704326 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50b8011bd0 con 0x7f50d00c8de0 2023-10-06T05:21:56.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.118+0000 7f50c6ffd700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.119+0000 7f50c6ffd700 1 -- 172.21.15.45:0/1315704326 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f50b80860f0 con 0x7f50d00c8de0 2023-10-06T05:21:56.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.119+0000 7f50ce59c700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.120+0000 7f50ce59c700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50b0066f30 0x7f50b00693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f50d016a640 tx=0x7f50c0007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.123+0000 7f50c6ffd700 1 -- 172.21.15.45:0/1315704326 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50b8053020 con 0x7f50d00c8de0 2023-10-06T05:21:56.313 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:21:56.312+0000 7f2d947f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.411+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f50d013fcd0 con 0x7f50d00c8de0 2023-10-06T05:21:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.415+0000 7f50c6ffd700 1 -- 172.21.15.45:0/1315704326 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f50b8056640 con 0x7f50d00c8de0 2023-10-06T05:21:56.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50b0066f30 msgr2=0x7f50b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50b0066f30 0x7f50b00693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f50d016a640 tx=0x7f50c0007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:56.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d00c8de0 msgr2=0x7f50d00c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d00c8de0 0x7f50d00c4900 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f50d006e930 tx=0x7f50b8017000 comp rx=0 tx=0).stop 2023-10-06T05:21:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 shutdown_connections 2023-10-06T05:21:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50d013f900 0x7f50d0167160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d00c8de0 0x7f50d00c4900 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 --2- 172.21.15.45:0/1315704326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50d00c9f20 0x7f50d00c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 >> 172.21.15.45:0/1315704326 conn(0x7f50d00691c0 msgr2=0x7f50d00c2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 shutdown_connections 2023-10-06T05:21:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.417+0000 7f50d5100700 1 -- 172.21.15.45:0/1315704326 wait complete. 2023-10-06T05:21:56.422 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-10-06T05:21:56.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:56.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.563+0000 7fe6712b8700 1 Processor -- start 2023-10-06T05:21:56.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.563+0000 7fe6712b8700 1 -- start start 2023-10-06T05:21:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe6712b8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe66c15cd60 0x7fe66c15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe6712b8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ac630 0x7fe66c0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe6712b8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c15d710 con 0x7fe66c0ae9d0 2023-10-06T05:21:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe6712b8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0acff0 con 0x7fe66c15cd60 2023-10-06T05:21:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe6712b8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0677c0 con 0x7fe66c0ac630 2023-10-06T05:21:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66a7fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe66c15cd60 0x7fe66c15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66affd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe66c0ae9d0 0x7fe66c068860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:38982/0 (socket says 172.21.15.45:38982) 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66b7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ac630 0x7fe66c0acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66affd700 1 -- 172.21.15.45:0/1207141754 learned_addr learned my addr 172.21.15.45:0/1207141754 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe669ffb700 1 -- 172.21.15.45:0/1207141754 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126806610 0 0) 0x7fe66c15d710 con 0x7fe66c0ae9d0 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe669ffb700 1 -- 172.21.15.45:0/1207141754 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe654003570 con 0x7fe66c0ae9d0 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66b7fe700 1 -- 172.21.15.45:0/1207141754 >> v2:172.21.15.144:3300/0 conn(0x7fe66c15cd60 msgr2=0x7fe66c15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66b7fe700 1 --2- 172.21.15.45:0/1207141754 >> v2:172.21.15.144:3300/0 conn(0x7fe66c15cd60 0x7fe66c15d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66b7fe700 1 -- 172.21.15.45:0/1207141754 >> v1:172.21.15.144:6789/0 conn(0x7fe66c0ae9d0 legacy=0x7fe66c068860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66b7fe700 1 -- 172.21.15.45:0/1207141754 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c067aa0 con 0x7fe66c0ac630 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.564+0000 7fe66a7fc700 1 --2- 172.21.15.45:0/1207141754 >> v2:172.21.15.144:3300/0 conn(0x7fe66c15cd60 0x7fe66c15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe66b7fe700 1 --2- 172.21.15.45:0/1207141754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ac630 0x7fe66c0acab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe664009e60 tx=0x7fe664009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=440fe4a84e9b5cf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe669ffb700 1 -- 172.21.15.45:0/1207141754 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400e070 con 0x7fe66c0ac630 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe669ffb700 1 -- 172.21.15.45:0/1207141754 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66400bab0 con 0x7fe66c0ac630 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe669ffb700 1 -- 172.21.15.45:0/1207141754 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400be70 con 0x7fe66c0ac630 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe6712b8700 1 -- 172.21.15.45:0/1207141754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ac630 msgr2=0x7fe66c0acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1207141754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ac630 0x7fe66c0acab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe664009e60 tx=0x7fe664009780 comp rx=0 tx=0).stop 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe6712b8700 1 -- 172.21.15.45:0/1207141754 shutdown_connections 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1207141754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ac630 0x7fe66c0acab0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1207141754 >> v2:172.21.15.144:3300/0 conn(0x7fe66c15cd60 0x7fe66c15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.565+0000 7fe6712b8700 1 -- 172.21.15.45:0/1207141754 >> 172.21.15.45:0/1207141754 conn(0x7fe66c0cd000 msgr2=0x7fe66c0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:56.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.566+0000 7fe6712b8700 1 -- 172.21.15.45:0/1207141754 shutdown_connections 2023-10-06T05:21:56.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.566+0000 7fe6712b8700 1 -- 172.21.15.45:0/1207141754 wait complete. 2023-10-06T05:21:56.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.567+0000 7fe6712b8700 1 Processor -- start 2023-10-06T05:21:56.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.567+0000 7fe6712b8700 1 -- start start 2023-10-06T05:21:56.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe6712b8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66c0ac630 0x7fe66c0b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe6712b8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ae9d0 0x7fe66c0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe6712b8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe66c15cd60 0x7fe66c1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe6712b8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0b2350 con 0x7fe66c0ac630 2023-10-06T05:21:56.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe6712b8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0b1740 con 0x7fe66c15cd60 2023-10-06T05:21:56.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe6712b8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0b1a20 con 0x7fe66c0ae9d0 2023-10-06T05:21:56.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66c0ac630 0x7fe66c0b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66c0ac630 0x7fe66c0b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49618/0 (socket says 172.21.15.45:49618) 2023-10-06T05:21:56.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 -- 172.21.15.45:0/1502017880 learned_addr learned my addr 172.21.15.45:0/1502017880 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:56.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66a7fc700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ae9d0 0x7fe66c0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66b7fe700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe66c15cd60 0x7fe66c1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 -- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ae9d0 msgr2=0x7fe66c0c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ae9d0 0x7fe66c0c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 -- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe66c15cd60 msgr2=0x7fe66c1548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe66c15cd60 0x7fe66c1548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 -- 172.21.15.45:0/1502017880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe660009420 con 0x7fe66c0ac630 2023-10-06T05:21:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe66affd700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66c0ac630 0x7fe66c0b10f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c7c0 tx=0x7fe65c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.568+0000 7fe652ffd700 1 -- 172.21.15.45:0/1502017880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c005b30 con 0x7fe66c0ac630 2023-10-06T05:21:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.569+0000 7fe652ffd700 1 -- 172.21.15.45:0/1502017880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c005c90 con 0x7fe66c0ac630 2023-10-06T05:21:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.569+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe664009a20 con 0x7fe66c0ac630 2023-10-06T05:21:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.569+0000 7fe652ffd700 1 -- 172.21.15.45:0/1502017880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c007300 con 0x7fe66c0ac630 2023-10-06T05:21:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.569+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c0b1fe0 con 0x7fe66c0ac630 2023-10-06T05:21:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.570+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c0cd400 con 0x7fe66c0ac630 2023-10-06T05:21:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.570+0000 7fe652ffd700 1 -- 172.21.15.45:0/1502017880 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe65c01b3b0 con 0x7fe66c0ac630 2023-10-06T05:21:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.570+0000 7fe652ffd700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe658066f30 0x7fe6580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.571+0000 7fe652ffd700 1 -- 172.21.15.45:0/1502017880 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe65c083d80 con 0x7fe66c0ac630 2023-10-06T05:21:56.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.571+0000 7fe66a7fc700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe658066f30 0x7fe6580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.571+0000 7fe66a7fc700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe658066f30 0x7fe6580693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0c5e20 tx=0x7fe6600091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.575+0000 7fe652ffd700 1 -- 172.21.15.45:0/1502017880 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe65c050d50 con 0x7fe66c0ac630 2023-10-06T05:21:56.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.829+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe66c067d80 con 0x7fe66c0ac630 2023-10-06T05:21:56.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.830+0000 7fe652ffd700 1 -- 172.21.15.45:0/1502017880 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fe65c017020 con 0x7fe66c0ac630 2023-10-06T05:21:56.834 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:21:56.834 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":86,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:21:54.343327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/821082350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":821082350},{"type":"v1","addr":"172.21.15.45:6833","nonce":821082350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:56.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe658066f30 msgr2=0x7fe6580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe658066f30 0x7fe6580693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0c5e20 tx=0x7fe6600091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66c0ac630 msgr2=0x7fe66c0b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66c0ac630 0x7fe66c0b10f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c7c0 tx=0x7fe65c007e00 comp rx=0 tx=0).stop 2023-10-06T05:21:56.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 shutdown_connections 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe658066f30 0x7fe6580693e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe66c15cd60 0x7fe66c1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66c0ac630 0x7fe66c0b10f0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 --2- 172.21.15.45:0/1502017880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe66c0ae9d0 0x7fe66c0c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 >> 172.21.15.45:0/1502017880 conn(0x7fe66c0cd000 msgr2=0x7fe66c157840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 shutdown_connections 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:56.832+0000 7fe6712b8700 1 -- 172.21.15.45:0/1502017880 wait complete. 2023-10-06T05:21:56.836 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 86 2023-10-06T05:21:57.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:21:54.343327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7179': {'gid': 7179, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6833/821082350', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 821082350}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 821082350}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:21:57.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:21:57.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:21:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.393+0000 7fb51d187700 1 Processor -- start 2023-10-06T05:21:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.393+0000 7fb51d187700 1 -- start start 2023-10-06T05:21:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51d187700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb518159330 0x7fb51806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51d187700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5180cc390 0x7fb5180cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51d187700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180cb530 con 0x7fb518158f60 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51d187700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb518069bd0 con 0x7fb5180cc390 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51d187700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5180ccd10 con 0x7fb518159330 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51759e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5180cc390 0x7fb5180cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb516d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb518158f60 0x7fb5180cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45226/0 (socket says 172.21.15.45:45226) 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb516d9d700 1 -- 172.21.15.45:0/1472989700 learned_addr learned my addr 172.21.15.45:0/1472989700 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:21:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51759e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5180cc390 0x7fb5180cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49622/0 (socket says 172.21.15.45:49622) 2023-10-06T05:21:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51659c700 1 --2- 172.21.15.45:0/1472989700 >> v2:172.21.15.144:3300/0 conn(0x7fb518159330 0x7fb51806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51759e700 1 -- 172.21.15.45:0/1472989700 >> v2:172.21.15.144:3300/0 conn(0x7fb518159330 msgr2=0x7fb51806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51759e700 1 --2- 172.21.15.45:0/1472989700 >> v2:172.21.15.144:3300/0 conn(0x7fb518159330 0x7fb51806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51759e700 1 -- 172.21.15.45:0/1472989700 >> v1:172.21.15.45:6789/0 conn(0x7fb518158f60 legacy=0x7fb5180cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.394+0000 7fb51759e700 1 -- 172.21.15.45:0/1472989700 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518068fc0 con 0x7fb5180cc390 2023-10-06T05:21:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51759e700 1 --2- 172.21.15.45:0/1472989700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5180cc390 0x7fb5180cc7d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb5180872e0 tx=0x7fb5100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158da299b5262647 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb515d9b700 1 -- 172.21.15.45:0/1472989700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000b6b0 con 0x7fb5180cc390 2023-10-06T05:21:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb515d9b700 1 -- 172.21.15.45:0/1472989700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51000b810 con 0x7fb5180cc390 2023-10-06T05:21:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb515d9b700 1 -- 172.21.15.45:0/1472989700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000bbb0 con 0x7fb5180cc390 2023-10-06T05:21:57.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 -- 172.21.15.45:0/1472989700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5180cc390 msgr2=0x7fb5180cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 --2- 172.21.15.45:0/1472989700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5180cc390 0x7fb5180cc7d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb5180872e0 tx=0x7fb5100091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 -- 172.21.15.45:0/1472989700 shutdown_connections 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 --2- 172.21.15.45:0/1472989700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5180cc390 0x7fb5180cc7d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 --2- 172.21.15.45:0/1472989700 >> v2:172.21.15.144:3300/0 conn(0x7fb518159330 0x7fb51806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 -- 172.21.15.45:0/1472989700 >> 172.21.15.45:0/1472989700 conn(0x7fb5180cdf50 msgr2=0x7fb5180ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 -- 172.21.15.45:0/1472989700 shutdown_connections 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.395+0000 7fb51d187700 1 -- 172.21.15.45:0/1472989700 wait complete. 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.397+0000 7fb51d187700 1 Processor -- start 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.397+0000 7fb51d187700 1 -- start start 2023-10-06T05:21:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51d187700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5180cc390 0x7fb518149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51d187700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb518158f60 0x7fb5180a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51d187700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518159330 0x7fb5180a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51d187700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5181671f0 con 0x7fb518158f60 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51d187700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51816b390 con 0x7fb518159330 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51d187700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51816a780 con 0x7fb5180cc390 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb518158f60 0x7fb5180a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb518158f60 0x7fb5180a8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49636/0 (socket says 172.21.15.45:49636) 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 -- 172.21.15.45:0/2223925804 learned_addr learned my addr 172.21.15.45:0/2223925804 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb516d9d700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5180cc390 0x7fb518149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51759e700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518159330 0x7fb5180a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 -- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5180cc390 msgr2=0x7fb518149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5180cc390 0x7fb518149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 -- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518159330 msgr2=0x7fb5180a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518159330 0x7fb5180a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 -- 172.21.15.45:0/2223925804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb510009420 con 0x7fb518158f60 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51759e700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518159330 0x7fb5180a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51659c700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb518158f60 0x7fb5180a8c10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c880 tx=0x7fb50c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb4feffd700 1 -- 172.21.15.45:0/2223925804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c011600 con 0x7fb518158f60 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb4feffd700 1 -- 172.21.15.45:0/2223925804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c011780 con 0x7fb518158f60 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51816aa60 con 0x7fb518158f60 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.398+0000 7fb4feffd700 1 -- 172.21.15.45:0/2223925804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c011b40 con 0x7fb518158f60 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.399+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51816b020 con 0x7fb518158f60 2023-10-06T05:21:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.400+0000 7fb4feffd700 1 -- 172.21.15.45:0/2223925804 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb50c018360 con 0x7fb518158f60 2023-10-06T05:21:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.400+0000 7fb4feffd700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb504066f30 0x7fb5040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.401+0000 7fb4feffd700 1 -- 172.21.15.45:0/2223925804 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb50c084670 con 0x7fb518158f60 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.401+0000 7fb516d9d700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb504066f30 0x7fb5040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.401+0000 7fb516d9d700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb504066f30 0x7fb5040693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb50800e160 tx=0x7fb508009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.402+0000 7fb52649bb80 1 Processor -- start 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.402+0000 7fb52649bb80 1 -- start start 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d715cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d715d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb52649bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7027310 con 0x55a4d7139f40 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7064130 con 0x55a4d715d4d0 2023-10-06T05:21:57.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7061af0 con 0x55a4d7154450 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb51659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d715cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb51659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d715cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49648/0 (socket says 172.21.15.45:49648) 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb51659c700 1 -- 172.21.15.45:0/1560587631 learned_addr learned my addr 172.21.15.45:0/1560587631 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.403+0000 7fb516d9d700 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51759e700 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d715d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51659c700 1 -- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 msgr2=0x55a4d7154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51659c700 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51659c700 1 -- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 msgr2=0x55a4d715d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51659c700 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d715d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51659c700 1 -- 172.21.15.45:0/1560587631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4d6e63790 con 0x55a4d7139f40 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51759e700 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d715d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb51659c700 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d715cf90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00a530 tx=0x7fb50c0827b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abc128db7dcc698 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb4f7fff700 1 -- 172.21.15.45:0/1560587631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0584b0 con 0x55a4d7139f40 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb4f7fff700 1 -- 172.21.15.45:0/1560587631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c05c200 con 0x55a4d7139f40 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.404+0000 7fb4f7fff700 1 -- 172.21.15.45:0/1560587631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c053600 con 0x55a4d7139f40 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 -- 172.21.15.45:0/1560587631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 msgr2=0x55a4d715cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d715cf90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00a530 tx=0x7fb50c0827b0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 -- 172.21.15.45:0/1560587631 shutdown_connections 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d715d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1560587631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d715cf90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 -- 172.21.15.45:0/1560587631 >> 172.21.15.45:0/1560587631 conn(0x55a4d7157880 msgr2=0x55a4d7157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 -- 172.21.15.45:0/1560587631 shutdown_connections 2023-10-06T05:21:57.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.405+0000 7fb52649bb80 1 -- 172.21.15.45:0/1560587631 wait complete. 2023-10-06T05:21:57.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.406+0000 7fb52649bb80 1 -- start start 2023-10-06T05:21:57.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.406+0000 7fb52649bb80 1 Processor -- start 2023-10-06T05:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d712dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d7154450 0x55a4d7131300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d715d4d0 0x55a4d7131840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb52649bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d705d8e0 con 0x55a4d7139f40 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7009980 con 0x55a4d7154450 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7009620 con 0x55a4d715d4d0 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb52649bb80 10 client.0 _ll_register_callbacks cb 0x55a4d6e6ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d712dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d712dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49652/0 (socket says 172.21.15.45:49652) 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 -- 172.21.15.45:0/433161834 learned_addr learned my addr 172.21.15.45:0/433161834 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb51759e700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d715d4d0 0x55a4d7131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb51659c700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d7154450 0x55a4d7131300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 -- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d715d4d0 msgr2=0x55a4d7131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d715d4d0 0x55a4d7131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 -- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d7154450 msgr2=0x55a4d7131300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d7154450 0x55a4d7131300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 -- 172.21.15.45:0/433161834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51816b020 con 0x55a4d7139f40 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb51659c700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d7154450 0x55a4d7131300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb516d9d700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d712dac0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb508014b60 tx=0x7fb508013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc69bcbf522a006f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.407+0000 7fb4f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb4f57fa700 1 -- 172.21.15.45:0/433161834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800ed00 con 0x55a4d7139f40 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb4f57fa700 1 -- 172.21.15.45:0/433161834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb5080076f0 con 0x55a4d7139f40 2023-10-06T05:21:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4d7101d60 con 0x55a4d7139f40 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb4f57fa700 1 -- 172.21.15.45:0/433161834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb508007ac0 con 0x55a4d7139f40 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 10 client.7329 fetch_fsmap learned FSMap version 86 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55a4d6f500a0 con 0x55a4d7139f40 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb4f57fa700 1 -- 172.21.15.45:0/433161834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4d7101d60 con 0x55a4d7139f40 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb4f57fa700 1 -- 172.21.15.45:0/433161834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 86) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb508020070 con 0x55a4d7139f40 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 10 client.7329 fetch_fsmap finished waiting for FSMap version 86 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 10 client.7329 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:21:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 20 client.7329 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d7155f40 0x55a4d71340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 4 client.7329 mds_command: new command op to 7179 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 --> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4d70deb30 con 0x55a4d7155f40 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.408+0000 7fb51659c700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d7155f40 0x55a4d71340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.409+0000 7fb4f57fa700 10 client.7329 ms_handle_connect on v2:172.21.15.45:6832/821082350 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.409+0000 7fb51659c700 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d7155f40 0x55a4d71340b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67e254fcd918182 server_cookie=dd98baf1b5d353bf in_seq=0 out_seq=0 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.409+0000 7fb4f57fa700 1 -- 172.21.15.45:0/433161834 <== mds.0 v2:172.21.15.45:6832/821082350 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a4d70deb30 con 0x55a4d7155f40 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.409+0000 7fb4f57fa700 10 client.7329 handle_command_reply: tid=0 2023-10-06T05:21:57.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.409+0000 7fb52649bb80 1 client.7329 shutdown 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.409+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 msgr2=0x55a4d712dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.409+0000 7fb52649bb80 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d712dac0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb508014b60 tx=0x7fb508013000 comp rx=0 tx=0).stop 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 shutdown_connections 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d715d4d0 0x55a4d7131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d7155f40 0x55a4d71340b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d712dac0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 --2- 172.21.15.45:0/433161834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d7154450 0x55a4d7131300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb4f57fa700 0 client.7329 ms_handle_reset on v2:172.21.15.45:6832/821082350 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 >> 172.21.15.45:0/433161834 conn(0x55a4d7157880 msgr2=0x55a4d7109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 shutdown_connections 2023-10-06T05:21:57.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 1 -- 172.21.15.45:0/433161834 wait complete. 2023-10-06T05:21:57.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.410+0000 7fb52649bb80 20 client.7329 trim_cache size 0 max 16384 2023-10-06T05:21:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.428+0000 7fb52649bb80 1 Processor -- start 2023-10-06T05:21:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.428+0000 7fb52649bb80 1 -- start start 2023-10-06T05:21:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.428+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.428+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d710e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.428+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d710eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb52649bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d6ef51e0 con 0x55a4d7139f40 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7130c10 con 0x55a4d715d4d0 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7131c80 con 0x55a4d7154450 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49664/0 (socket says 172.21.15.45:49664) 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 -- 172.21.15.45:0/3357992241 learned_addr learned my addr 172.21.15.45:0/3357992241 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb51659c700 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d710e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb51759e700 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d710eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 -- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 msgr2=0x55a4d710e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d710e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 -- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 msgr2=0x55a4d710eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d710eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 -- 172.21.15.45:0/3357992241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4d702c3c0 con 0x55a4d7139f40 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb516d9d700 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb5080012b0 tx=0x7fb508018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1838bd80095fa17f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb4fcff9700 1 -- 172.21.15.45:0/3357992241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508037070 con 0x55a4d7139f40 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb4fcff9700 1 -- 172.21.15.45:0/3357992241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5080076f0 con 0x55a4d7139f40 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.429+0000 7fb4fcff9700 1 -- 172.21.15.45:0/3357992241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508007ac0 con 0x55a4d7139f40 2023-10-06T05:21:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 -- 172.21.15.45:0/3357992241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 msgr2=0x55a4d7103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb5080012b0 tx=0x7fb508018000 comp rx=0 tx=0).stop 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 -- 172.21.15.45:0/3357992241 shutdown_connections 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d710eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 --2- 172.21.15.45:0/3357992241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d710e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 -- 172.21.15.45:0/3357992241 >> 172.21.15.45:0/3357992241 conn(0x55a4d7157880 msgr2=0x55a4d710afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 -- 172.21.15.45:0/3357992241 shutdown_connections 2023-10-06T05:21:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.430+0000 7fb52649bb80 1 -- 172.21.15.45:0/3357992241 wait complete. 2023-10-06T05:21:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.431+0000 7fb52649bb80 1 -- start start 2023-10-06T05:21:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.431+0000 7fb52649bb80 1 Processor -- start 2023-10-06T05:21:57.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.432+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.432+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7156310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.432+0000 7fb52649bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d7156850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.432+0000 7fb52649bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d7157220 con 0x55a4d7139f40 2023-10-06T05:21:57.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.432+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d713d220 con 0x55a4d715d4d0 2023-10-06T05:21:57.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.432+0000 7fb52649bb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4d713c320 con 0x55a4d7154450 2023-10-06T05:21:57.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.432+0000 7fb52649bb80 10 client.0 _ll_register_callbacks cb 0x55a4d6e6ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:57.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49672/0 (socket says 172.21.15.45:49672) 2023-10-06T05:21:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 -- 172.21.15.45:0/1407878976 learned_addr learned my addr 172.21.15.45:0/1407878976 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb51659c700 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb51759e700 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d7156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 -- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 msgr2=0x55a4d7156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 -- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 msgr2=0x55a4d7156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d7156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 -- 172.21.15.45:0/1407878976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4d6f500a0 con 0x55a4d7139f40 2023-10-06T05:21:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb516d9d700 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb508007880 tx=0x7fb508037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42042ee539360782 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb4f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:21:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb4f57fa700 1 -- 172.21.15.45:0/1407878976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800ed00 con 0x55a4d7139f40 2023-10-06T05:21:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb4f57fa700 1 -- 172.21.15.45:0/1407878976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb508029830 con 0x55a4d7139f40 2023-10-06T05:21:57.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb50c05c200 con 0x55a4d7139f40 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.433+0000 7fb4f57fa700 1 -- 172.21.15.45:0/1407878976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb50803c410 con 0x55a4d7139f40 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 10 client.7335 fetch_fsmap learned FSMap version 86 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55a4d6ec41d0 con 0x55a4d7139f40 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb4f57fa700 1 -- 172.21.15.45:0/1407878976 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c05c200 con 0x55a4d7139f40 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb4f57fa700 1 -- 172.21.15.45:0/1407878976 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 86) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb50803c570 con 0x55a4d7139f40 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 10 client.7335 fetch_fsmap finished waiting for FSMap version 86 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 10 client.7335 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:21:57.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 20 client.7335 populate_metadata read hostname 'smithi045' 2023-10-06T05:21:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d71341f0 0x55a4d71345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 4 client.7335 mds_command: new command op to 7179 tid=0[{"prefix": "status"}] 2023-10-06T05:21:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 --> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a4d70008d0 con 0x55a4d71341f0 2023-10-06T05:21:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb51659c700 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d71341f0 0x55a4d71345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb4f57fa700 10 client.7335 ms_handle_connect on v2:172.21.15.45:6832/821082350 2023-10-06T05:21:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.434+0000 7fb51659c700 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d71341f0 0x55a4d71345c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6440f9903ab9bf8 server_cookie=6215f72e97d7ebcc in_seq=0 out_seq=0 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb4f57fa700 1 -- 172.21.15.45:0/1407878976 <== mds.0 v2:172.21.15.45:6832/821082350 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a4d70008d0 con 0x55a4d71341f0 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb4f57fa700 10 client.7335 handle_command_reply: tid=0 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 client.7335 shutdown 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 msgr2=0x55a4d7103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb508007880 tx=0x7fb508037000 comp rx=0 tx=0).stop 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 shutdown_connections 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4d715d4d0 0x55a4d7156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:6832/821082350,v1:172.21.15.45:6833/821082350] conn(0x55a4d71341f0 0x55a4d71345c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a4d7139f40 0x55a4d7103a40 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 --2- 172.21.15.45:0/1407878976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a4d7154450 0x55a4d7156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb4f57fa700 0 client.7335 ms_handle_reset on v2:172.21.15.45:6832/821082350 2023-10-06T05:21:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 >> 172.21.15.45:0/1407878976 conn(0x55a4d7157880 msgr2=0x55a4d716d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 shutdown_connections 2023-10-06T05:21:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 1 -- 172.21.15.45:0/1407878976 wait complete. 2023-10-06T05:21:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb52649bb80 20 client.7335 trim_cache size 0 max 16384 2023-10-06T05:21:57.447 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:57.447 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:21:57.447 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:21:57.447 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7179, 2023-10-06T05:21:57.447 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:21:57.447 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:21:57.448 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:21:57.448 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.1056963700000004, 2023-10-06T05:21:57.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 86, 2023-10-06T05:21:57.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 92, 2023-10-06T05:21:57.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2023-10-06T05:21:57.448 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 12.094444043999999 2023-10-06T05:21:57.448 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.435+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb504066f30 msgr2=0x7fb5040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb504066f30 0x7fb5040693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb50800e160 tx=0x7fb508009040 comp rx=0 tx=0).stop 2023-10-06T05:21:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb518158f60 msgr2=0x7fb5180a8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb518158f60 0x7fb5180a8c10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c880 tx=0x7fb50c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:57.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 shutdown_connections 2023-10-06T05:21:57.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb504066f30 0x7fb5040693e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb518159330 0x7fb5180a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5180cc390 0x7fb518149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 --2- 172.21.15.45:0/2223925804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb518158f60 0x7fb5180a8c10 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 >> 172.21.15.45:0/2223925804 conn(0x7fb5180cdf50 msgr2=0x7fb5180c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 shutdown_connections 2023-10-06T05:21:57.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.436+0000 7fb51d187700 1 -- 172.21.15.45:0/2223925804 wait complete. 2023-10-06T05:21:57.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:21:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.583+0000 7fb0f48e3700 1 Processor -- start 2023-10-06T05:21:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.583+0000 7fb0f48e3700 1 -- start start 2023-10-06T05:21:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.584+0000 7fb0f48e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f0158f60 0x7fb0f0159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.584+0000 7fb0f48e3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb0f00cc390 0x7fb0f00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.584+0000 7fb0f48e3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00cb530 con 0x7fb0f006a060 2023-10-06T05:21:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.584+0000 7fb0f48e3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00ccd50 con 0x7fb0f00cc390 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.584+0000 7fb0f48e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0069bd0 con 0x7fb0f0158f60 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f0158f60 0x7fb0f0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f0158f60 0x7fb0f0159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:55206/0 (socket says 172.21.15.45:55206) 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0edd9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb0f006a060 0x7fb0f00cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:38996/0 (socket says 172.21.15.45:38996) 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 -- 172.21.15.45:0/1520619401 learned_addr learned my addr 172.21.15.45:0/1520619401 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0eed9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb0f00cc390 0x7fb0f00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 -- 172.21.15.45:0/1520619401 >> v2:172.21.15.144:3300/0 conn(0x7fb0f00cc390 msgr2=0x7fb0f00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 --2- 172.21.15.45:0/1520619401 >> v2:172.21.15.144:3300/0 conn(0x7fb0f00cc390 0x7fb0f00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 -- 172.21.15.45:0/1520619401 >> v1:172.21.15.144:6789/0 conn(0x7fb0f006a060 legacy=0x7fb0f00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 -- 172.21.15.45:0/1520619401 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f006a4d0 con 0x7fb0f0158f60 2023-10-06T05:21:57.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ee59c700 1 --2- 172.21.15.45:0/1520619401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f0158f60 0x7fb0f0159330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00099b0 tx=0x7fb0e0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e535b466bd91f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.585+0000 7fb0ed59a700 1 -- 172.21.15.45:0/1520619401 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0009ea0 con 0x7fb0f0158f60 2023-10-06T05:21:57.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0ed59a700 1 -- 172.21.15.45:0/1520619401 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e000b860 con 0x7fb0f0158f60 2023-10-06T05:21:57.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0ed59a700 1 -- 172.21.15.45:0/1520619401 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000bc00 con 0x7fb0f0158f60 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 -- 172.21.15.45:0/1520619401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f0158f60 msgr2=0x7fb0f0159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/1520619401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f0158f60 0x7fb0f0159330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00099b0 tx=0x7fb0e0009250 comp rx=0 tx=0).stop 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 -- 172.21.15.45:0/1520619401 shutdown_connections 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/1520619401 >> v2:172.21.15.144:3300/0 conn(0x7fb0f00cc390 0x7fb0f00cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/1520619401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f0158f60 0x7fb0f0159330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 -- 172.21.15.45:0/1520619401 >> 172.21.15.45:0/1520619401 conn(0x7fb0f00cdf50 msgr2=0x7fb0f00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 -- 172.21.15.45:0/1520619401 shutdown_connections 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.586+0000 7fb0f48e3700 1 -- 172.21.15.45:0/1520619401 wait complete. 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.587+0000 7fb0f48e3700 1 Processor -- start 2023-10-06T05:21:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.587+0000 7fb0f48e3700 1 -- start start 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.588+0000 7fb0f48e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f006a060 0x7fb0f00a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.588+0000 7fb0f48e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0f00cc390 0x7fb0f00c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.588+0000 7fb0f48e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158f60 0x7fb0f00c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.588+0000 7fb0f48e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00af5e0 con 0x7fb0f0158f60 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.588+0000 7fb0f48e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00af8c0 con 0x7fb0f00cc390 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.588+0000 7fb0f48e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00afba0 con 0x7fb0f006a060 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158f60 0x7fb0f00c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158f60 0x7fb0f00c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49680/0 (socket says 172.21.15.45:49680) 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 -- 172.21.15.45:0/2446813183 learned_addr learned my addr 172.21.15.45:0/2446813183 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:21:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0ee59c700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f006a060 0x7fb0f00a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0edd9b700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0f00cc390 0x7fb0f00c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 -- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f006a060 msgr2=0x7fb0f00a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f006a060 0x7fb0f00a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 -- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0f00cc390 msgr2=0x7fb0f00c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0f00cc390 0x7fb0f00c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 -- 172.21.15.45:0/2446813183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00094f0 con 0x7fb0f0158f60 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0eed9d700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158f60 0x7fb0f00c7620 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e90 tx=0x7fb0e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0de7fc700 1 -- 172.21.15.45:0/2446813183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800caa0 con 0x7fb0f0158f60 2023-10-06T05:21:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0de7fc700 1 -- 172.21.15.45:0/2446813183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e800f040 con 0x7fb0f0158f60 2023-10-06T05:21:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0de7fc700 1 -- 172.21.15.45:0/2446813183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800ce50 con 0x7fb0f0158f60 2023-10-06T05:21:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f00afe80 con 0x7fb0f0158f60 2023-10-06T05:21:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.589+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f01698b0 con 0x7fb0f0158f60 2023-10-06T05:21:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.590+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f00b03a0 con 0x7fb0f0158f60 2023-10-06T05:21:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.591+0000 7fb0de7fc700 1 -- 172.21.15.45:0/2446813183 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0e8014690 con 0x7fb0f0158f60 2023-10-06T05:21:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.591+0000 7fb0de7fc700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0d0066f30 0x7fb0d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.591+0000 7fb0de7fc700 1 -- 172.21.15.45:0/2446813183 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb0e80851f0 con 0x7fb0f0158f60 2023-10-06T05:21:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.592+0000 7fb0ee59c700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0d0066f30 0x7fb0d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.592+0000 7fb0ee59c700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0d0066f30 0x7fb0d00693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00099b0 tx=0x7fb0e0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.596+0000 7fb0de7fc700 1 -- 172.21.15.45:0/2446813183 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0f00b03a0 con 0x7fb0f0158f60 2023-10-06T05:21:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.848+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb0f00b03a0 con 0x7fb0f0158f60 2023-10-06T05:21:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.848+0000 7fb0de7fc700 1 -- 172.21.15.45:0/2446813183 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0f00b03a0 con 0x7fb0f0158f60 2023-10-06T05:21:57.850 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0d0066f30 msgr2=0x7fb0d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0d0066f30 0x7fb0d00693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00099b0 tx=0x7fb0e0017000 comp rx=0 tx=0).stop 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158f60 msgr2=0x7fb0f00c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158f60 0x7fb0f00c7620 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e90 tx=0x7fb0e800b040 comp rx=0 tx=0).stop 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 shutdown_connections 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb0d0066f30 0x7fb0d00693e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158f60 0x7fb0f00c7620 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0f006a060 0x7fb0f00a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 --2- 172.21.15.45:0/2446813183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0f00cc390 0x7fb0f00c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 >> 172.21.15.45:0/2446813183 conn(0x7fb0f00cdf50 msgr2=0x7fb0f00c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 shutdown_connections 2023-10-06T05:21:57.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:21:57.851+0000 7fb0f48e3700 1 -- 172.21.15.45:0/2446813183 wait complete. 2023-10-06T05:21:57.866 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:21:57.867 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:57.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:57.868 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:21:57.888 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:21:57.889 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:21:57.889 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:21:57.889 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:57.889 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:21:57.889 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:57.889 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42023sec preferred_lft 42023sec 2023-10-06T05:21:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:57.891 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:57.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:57.892 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:21:57.960 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:21:57.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:57.961 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:21:58.027 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:21:58.027 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:21:58.027 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:21:58.027 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:21:58.027 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:21:58.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:58.028 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:21:58.028 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:21:58.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:21:58.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:21:58.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:21:58.028 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:21:58.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:58.152 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:21:58.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:21:58.174 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:21:58.174 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:58.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:58.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:21:58.251 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:21:58.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:58.251 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:58.314 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:58.314 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:58.314 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:58.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:21:58.454 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:21:58.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:21:58.529 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:21:58.530 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:21:58.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:58.531 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:21:58.594 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:21:58.594 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:21:58.637 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:21:58.672 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:21:58.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:21:58.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:21:58.736+0000 7fbe24e16540 -1 init, newargv = 0x55a8c198f8b0 newargc=15 2023-10-06T05:21:58.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[154946]: starting ceph client 2023-10-06T05:21:58.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[154946]: starting fuse 2023-10-06T05:21:58.744 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:21:58.745 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:21:58.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:58.746 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:21:58.807 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:21:58.807 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:21:58.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:21:58.807 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:21:58.807 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:21:58.807 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:21:58.807 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:21:58.808 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:21:58.809 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:21:58.948 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.154946.asok 2023-10-06T05:21:59.130 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.154946.asok 2023-10-06T05:21:59.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:59.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.154946.asok status 2023-10-06T05:21:59.306 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:59.306 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "154946", 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:21:59.307 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7344, 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 7344 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2286978847 2023-10-06T05:21:59.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2286978847 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.7344 192.168.0.1:0/2286978847", 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2286978847", 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 86, 2023-10-06T05:21:59.309 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 92, 2023-10-06T05:21:59.310 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:21:59.310 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:21:59.310 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:21:59.310 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:59.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:59.317 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:21:59.317 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:59.338 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:21:59.338 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:59.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:59.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:59.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:59.412 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-10-06T05:21:59.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:59.482 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-10-06T05:21:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records in 2023-10-06T05:21:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records out 2023-10-06T05:21:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0643893 s, 97.7 MB/s 2023-10-06T05:21:59.614 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> import json 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> try: 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T05:21:59.615 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2023-10-06T05:21:59.616 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T05:21:59.616 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2023-10-06T05:21:59.616 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:21:59.697 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:21:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mode": 33204, 2023-10-06T05:21:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ino": 1099511627777, 2023-10-06T05:21:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "st_dev": 47, 2023-10-06T05:21:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "st_nlink": 1, 2023-10-06T05:21:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "st_uid": 1000, 2023-10-06T05:21:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "st_gid": 1285, 2023-10-06T05:21:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "st_size": 6291456, 2023-10-06T05:21:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "st_atime": 1696569719.5467207, 2023-10-06T05:21:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mtime": 1696569719.5986948, 2023-10-06T05:21:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ctime": 1696569719.5986948 2023-10-06T05:21:59.730 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:21:59.882 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:21:59.882 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:21:59.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:59.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[154946]: fuse finished with error 0 and tester_r 0 2023-10-06T05:21:59.969 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:22:06.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:06.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:06.071 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:06.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.273+0000 7f5a4bd07700 1 Processor -- start 2023-10-06T05:22:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.273+0000 7f5a4bd07700 1 -- start start 2023-10-06T05:22:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a4bd07700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5a44158f60 0x7f5a44159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a4bd07700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a4406a060 0x7f5a440cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a4bd07700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a44069bd0 con 0x7f5a440cc390 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a4bd07700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4406a500 con 0x7f5a4406a060 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a4bd07700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a44068fc0 con 0x7f5a44158f60 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a492a2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a4406a060 0x7f5a440cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a4a2a4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5a440cc390 0x7f5a440c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45228/0 (socket says 172.21.15.45:45228) 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a4a2a4700 1 -- 172.21.15.45:0/388972128 learned_addr learned my addr 172.21.15.45:0/388972128 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.274+0000 7f5a49aa3700 1 --2- 172.21.15.45:0/388972128 >> v2:172.21.15.144:3300/0 conn(0x7f5a44158f60 0x7f5a44159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a492a2700 1 -- 172.21.15.45:0/388972128 >> v2:172.21.15.144:3300/0 conn(0x7f5a44158f60 msgr2=0x7f5a44159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a492a2700 1 --2- 172.21.15.45:0/388972128 >> v2:172.21.15.144:3300/0 conn(0x7f5a44158f60 0x7f5a44159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a492a2700 1 -- 172.21.15.45:0/388972128 >> v1:172.21.15.45:6789/0 conn(0x7f5a440cc390 legacy=0x7f5a440c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a492a2700 1 -- 172.21.15.45:0/388972128 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440692a0 con 0x7f5a4406a060 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a48aa1700 1 -- 172.21.15.45:0/388972128 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335963055 0 0) 0x7f5a44069bd0 con 0x7f5a440cc390 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a492a2700 1 --2- 172.21.15.45:0/388972128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4406a060 0x7f5a440cae20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5a4406ed80 tx=0x7f5a380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fd393a117b8de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a48aa1700 1 -- 172.21.15.45:0/388972128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800b6b0 con 0x7f5a4406a060 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a48aa1700 1 -- 172.21.15.45:0/388972128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800b810 con 0x7f5a4406a060 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.275+0000 7f5a48aa1700 1 -- 172.21.15.45:0/388972128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800bbb0 con 0x7f5a4406a060 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 -- 172.21.15.45:0/388972128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4406a060 msgr2=0x7f5a440cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/388972128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4406a060 0x7f5a440cae20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5a4406ed80 tx=0x7f5a380091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 -- 172.21.15.45:0/388972128 shutdown_connections 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/388972128 >> v2:172.21.15.144:3300/0 conn(0x7f5a44158f60 0x7f5a44159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/388972128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a4406a060 0x7f5a440cae20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 -- 172.21.15.45:0/388972128 >> 172.21.15.45:0/388972128 conn(0x7f5a440cdf50 msgr2=0x7f5a440ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 -- 172.21.15.45:0/388972128 shutdown_connections 2023-10-06T05:22:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.276+0000 7f5a4bd07700 1 -- 172.21.15.45:0/388972128 wait complete. 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.277+0000 7f5a4bd07700 1 Processor -- start 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.277+0000 7f5a4bd07700 1 -- start start 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4bd07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a4406a060 0x7f5a440c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4bd07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440cc390 0x7f5a4415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4bd07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a44158f60 0x7f5a44169660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4bd07700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4415a820 con 0x7f5a44158f60 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4bd07700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44156ad0 con 0x7f5a440cc390 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4bd07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44157be0 con 0x7f5a4406a060 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a44158f60 0x7f5a44169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a44158f60 0x7f5a44169660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49692/0 (socket says 172.21.15.45:49692) 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 -- 172.21.15.45:0/4012169398 learned_addr learned my addr 172.21.15.45:0/4012169398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a49aa3700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a4406a060 0x7f5a440c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a492a2700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440cc390 0x7f5a4415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 -- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a4406a060 msgr2=0x7f5a440c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a4406a060 0x7f5a440c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 -- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440cc390 msgr2=0x7f5a4415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440cc390 0x7f5a4415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 -- 172.21.15.45:0/4012169398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009420 con 0x7f5a44158f60 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a492a2700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440cc390 0x7f5a4415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a49aa3700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a4406a060 0x7f5a440c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a4a2a4700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a44158f60 0x7f5a44169660 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00cc90 tx=0x7f5a3c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a35ffb700 1 -- 172.21.15.45:0/4012169398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00cea0 con 0x7f5a44158f60 2023-10-06T05:22:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.278+0000 7f5a35ffb700 1 -- 172.21.15.45:0/4012169398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c00ab70 con 0x7f5a44158f60 2023-10-06T05:22:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.279+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a44157ec0 con 0x7f5a44158f60 2023-10-06T05:22:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.279+0000 7f5a35ffb700 1 -- 172.21.15.45:0/4012169398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c0070f0 con 0x7f5a44158f60 2023-10-06T05:22:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.279+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a441578b0 con 0x7f5a44158f60 2023-10-06T05:22:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.280+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4406f410 con 0x7f5a44158f60 2023-10-06T05:22:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.280+0000 7f5a35ffb700 1 -- 172.21.15.45:0/4012169398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a3c0074a0 con 0x7f5a44158f60 2023-10-06T05:22:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.281+0000 7f5a35ffb700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a28066f30 0x7f5a280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.281+0000 7f5a35ffb700 1 -- 172.21.15.45:0/4012169398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5a3c084c50 con 0x7f5a44158f60 2023-10-06T05:22:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.281+0000 7f5a49aa3700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a28066f30 0x7f5a280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.282+0000 7f5a49aa3700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a28066f30 0x7f5a280693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5a40006fd0 tx=0x7f5a40008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.285+0000 7f5a35ffb700 1 -- 172.21.15.45:0/4012169398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a3c08a020 con 0x7f5a44158f60 2023-10-06T05:22:06.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.535+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a44069580 con 0x7f5a44158f60 2023-10-06T05:22:06.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.536+0000 7f5a35ffb700 1 -- 172.21.15.45:0/4012169398 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f5a3c051ca0 con 0x7f5a44158f60 2023-10-06T05:22:06.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:06.538 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":86,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:21:54.343327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7179":{"gid":7179,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/821082350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":821082350},{"type":"v1","addr":"172.21.15.45:6833","nonce":821082350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a28066f30 msgr2=0x7f5a280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a28066f30 0x7f5a280693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5a40006fd0 tx=0x7f5a40008040 comp rx=0 tx=0).stop 2023-10-06T05:22:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a44158f60 msgr2=0x7f5a44169660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a44158f60 0x7f5a44169660 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00cc90 tx=0x7f5a3c001a80 comp rx=0 tx=0).stop 2023-10-06T05:22:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 shutdown_connections 2023-10-06T05:22:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a28066f30 0x7f5a280693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a44158f60 0x7f5a44169660 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a4406a060 0x7f5a440c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 --2- 172.21.15.45:0/4012169398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a440cc390 0x7f5a4415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 >> 172.21.15.45:0/4012169398 conn(0x7f5a440cdf50 msgr2=0x7f5a44154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 shutdown_connections 2023-10-06T05:22:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.538+0000 7f5a4bd07700 1 -- 172.21.15.45:0/4012169398 wait complete. 2023-10-06T05:22:06.541 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 86 2023-10-06T05:22:06.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-10-06T05:22:06.716 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2023-10-06T05:22:06.717 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-06T05:22:06.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:06.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.848+0000 7fcc108b9700 1 Processor -- start 2023-10-06T05:22:06.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.848+0000 7fcc108b9700 1 -- start start 2023-10-06T05:22:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.849+0000 7fcc108b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c159200 0x7fcc0c0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.849+0000 7fcc108b9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc0c0ccdc0 0x7fcc0c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.849+0000 7fcc108b9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c0c7d00 con 0x7fcc0c1595d0 2023-10-06T05:22:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.849+0000 7fcc108b9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0c7780 con 0x7fcc0c0ccdc0 2023-10-06T05:22:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.849+0000 7fcc108b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cd740 con 0x7fcc0c159200 2023-10-06T05:22:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc0c0ccdc0 0x7fcc0c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc0c0ccdc0 0x7fcc0c0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54610/0 (socket says 172.21.15.45:54610) 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3422246322 learned_addr learned my addr 172.21.15.45:0/3422246322 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0a59c700 1 --2- 172.21.15.45:0/3422246322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c159200 0x7fcc0c0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3422246322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c159200 msgr2=0x7fcc0c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 --2- 172.21.15.45:0/3422246322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c159200 0x7fcc0c0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3422246322 >> v1:172.21.15.144:6789/0 conn(0x7fcc0c1595d0 legacy=0x7fcc0c06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3422246322 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0c86d0 con 0x7fcc0c0ccdc0 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0ad9d700 1 --2- 172.21.15.45:0/3422246322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c0ccdc0 0x7fcc0c0cd200 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcc040099b0 tx=0x7fcc04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28e07ff4b29d94b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0959a700 1 -- 172.21.15.45:0/3422246322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04009ea0 con 0x7fcc0c0ccdc0 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0959a700 1 -- 172.21.15.45:0/3422246322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0400b860 con 0x7fcc0c0ccdc0 2023-10-06T05:22:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.850+0000 7fcc0959a700 1 -- 172.21.15.45:0/3422246322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400bc00 con 0x7fcc0c0ccdc0 2023-10-06T05:22:06.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 -- 172.21.15.45:0/3422246322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c0ccdc0 msgr2=0x7fcc0c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3422246322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c0ccdc0 0x7fcc0c0cd200 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcc040099b0 tx=0x7fcc04009250 comp rx=0 tx=0).stop 2023-10-06T05:22:06.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 -- 172.21.15.45:0/3422246322 shutdown_connections 2023-10-06T05:22:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3422246322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c0ccdc0 0x7fcc0c0cd200 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3422246322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c159200 0x7fcc0c0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 -- 172.21.15.45:0/3422246322 >> 172.21.15.45:0/3422246322 conn(0x7fcc0c06f090 msgr2=0x7fcc0c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 -- 172.21.15.45:0/3422246322 shutdown_connections 2023-10-06T05:22:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.851+0000 7fcc108b9700 1 -- 172.21.15.45:0/3422246322 wait complete. 2023-10-06T05:22:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.852+0000 7fcc108b9700 1 Processor -- start 2023-10-06T05:22:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.852+0000 7fcc108b9700 1 -- start start 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc108b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c0ccdc0 0x7fcc0c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc108b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc0c159200 0x7fcc0c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc108b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c1595d0 0x7fcc0c0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc108b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0b21b0 con 0x7fcc0c1595d0 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc108b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c154310 con 0x7fcc0c159200 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc108b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0afd60 con 0x7fcc0c0ccdc0 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc0ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c1595d0 0x7fcc0c0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc0ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c1595d0 0x7fcc0c0c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54620/0 (socket says 172.21.15.45:54620) 2023-10-06T05:22:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3074117681 learned_addr learned my addr 172.21.15.45:0/3074117681 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc09d9b700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc0c159200 0x7fcc0c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.853+0000 7fcc0a59c700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c0ccdc0 0x7fcc0c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c0ccdc0 msgr2=0x7fcc0c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc0ad9d700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c0ccdc0 0x7fcc0c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc0c159200 msgr2=0x7fcc0c0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc0ad9d700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc0c159200 0x7fcc0c0c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc0ad9d700 1 -- 172.21.15.45:0/3074117681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040094f0 con 0x7fcc0c1595d0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc0ad9d700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c1595d0 0x7fcc0c0c29e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c06aed0 tx=0x7fcc04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc027fc700 1 -- 172.21.15.45:0/3074117681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400bbe0 con 0x7fcc0c1595d0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc027fc700 1 -- 172.21.15.45:0/3074117681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04011420 con 0x7fcc0c1595d0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c0b0040 con 0x7fcc0c1595d0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc027fc700 1 -- 172.21.15.45:0/3074117681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc040117e0 con 0x7fcc0c1595d0 2023-10-06T05:22:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.854+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c0b2710 con 0x7fcc0c1595d0 2023-10-06T05:22:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.855+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c0b2900 con 0x7fcc0c1595d0 2023-10-06T05:22:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.856+0000 7fcc027fc700 1 -- 172.21.15.45:0/3074117681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc04011bd0 con 0x7fcc0c1595d0 2023-10-06T05:22:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.856+0000 7fcc027fc700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbec066f30 0x7fcbec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.856+0000 7fcc027fc700 1 -- 172.21.15.45:0/3074117681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fcc04086260 con 0x7fcc0c1595d0 2023-10-06T05:22:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.856+0000 7fcc0a59c700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbec066f30 0x7fcbec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.857+0000 7fcc0a59c700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbec066f30 0x7fcbec0693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00f570 tx=0x7fcbfc00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:06.861+0000 7fcc027fc700 1 -- 172.21.15.45:0/3074117681 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc04053230 con 0x7fcc0c1595d0 2023-10-06T05:22:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.113+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcc0c06bb40 con 0x7fcc0c1595d0 2023-10-06T05:22:07.115 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:07.113+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:07.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.119+0000 7fcc027fc700 1 -- 172.21.15.45:0/3074117681 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcc04056850 con 0x7fcc0c1595d0 2023-10-06T05:22:07.121 INFO:tasks.ceph.mds.a.smithi045.stderr: -3030> 2023-10-06T05:21:56.312+0000 7f2d947f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbec066f30 msgr2=0x7fcbec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbec066f30 0x7fcbec0693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00f570 tx=0x7fcbfc00b040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c1595d0 msgr2=0x7fcc0c0c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c1595d0 0x7fcc0c0c29e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c06aed0 tx=0x7fcc04017000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 shutdown_connections 2023-10-06T05:22:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbec066f30 0x7fcbec0693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc0c1595d0 0x7fcc0c0c29e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc0c0ccdc0 0x7fcc0c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 --2- 172.21.15.45:0/3074117681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc0c159200 0x7fcc0c0c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.121+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 >> 172.21.15.45:0/3074117681 conn(0x7fcc0c06f090 msgr2=0x7fcc0c0ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.124+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 shutdown_connections 2023-10-06T05:22:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.124+0000 7fcc108b9700 1 -- 172.21.15.45:0/3074117681 wait complete. 2023-10-06T05:22:07.126 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:07.139 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2023-10-06T05:22:07.149 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:22:07.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 Processor -- start 2023-10-06T05:22:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 -- start start 2023-10-06T05:22:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e652070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e5c05f0 0x563a4e5c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e63c970 0x563a4e5c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4e4c7980 con 0x563a4e57b1b0 2023-10-06T05:22:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4e4c7620 con 0x563a4e5c05f0 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda7ae04040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a4e4be400 con 0x563a4e63c970 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.183+0000 7fda74022700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e652070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e652070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54650/0 (socket says 172.21.15.45:54650) 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 -- 172.21.15.45:0/199319130 learned_addr learned my addr 172.21.15.45:0/199319130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74823700 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e63c970 0x563a4e5c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda73821700 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e5c05f0 0x563a4e5c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 -- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e63c970 msgr2=0x563a4e5c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e63c970 0x563a4e5c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 -- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e5c05f0 msgr2=0x563a4e5c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e5c05f0 0x563a4e5c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 -- 172.21.15.45:0/199319130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda68005db0 con 0x563a4e57b1b0 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda74022700 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e652070 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fda6800ad40 tx=0x7fda68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f683a6eaa2e697 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda7281f700 1 -- 172.21.15.45:0/199319130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68000c90 con 0x563a4e57b1b0 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda7281f700 1 -- 172.21.15.45:0/199319130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda68005460 con 0x563a4e57b1b0 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.184+0000 7fda7281f700 1 -- 172.21.15.45:0/199319130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda680055d0 con 0x563a4e57b1b0 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 -- 172.21.15.45:0/199319130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 msgr2=0x563a4e652070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e652070 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fda6800ad40 tx=0x7fda68016000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 -- 172.21.15.45:0/199319130 shutdown_connections 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e63c970 0x563a4e5c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e652070 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 --2- 172.21.15.45:0/199319130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e5c05f0 0x563a4e5c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 -- 172.21.15.45:0/199319130 >> 172.21.15.45:0/199319130 conn(0x563a4e57d4a0 msgr2=0x563a4e639730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 -- 172.21.15.45:0/199319130 shutdown_connections 2023-10-06T05:22:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 -- 172.21.15.45:0/199319130 wait complete. 2023-10-06T05:22:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 Processor -- start 2023-10-06T05:22:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.185+0000 7fda7ae04040 1 -- start start 2023-10-06T05:22:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda7ae04040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e666d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda7ae04040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e5c05f0 0x563a4e5aac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda7ae04040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e63c970 0x563a4e59f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda7ae04040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4e63b0d0 con 0x563a4e57b1b0 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda7ae04040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4e5ab180 con 0x563a4e63c970 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda7ae04040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a4e5ab3f0 con 0x563a4e5c05f0 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda74022700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e666d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda73821700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e5c05f0 0x563a4e5aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda74022700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e666d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54654/0 (socket says 172.21.15.45:54654) 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 learned_addr learned my addr 172.21.15.45:0/4052796629 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.186+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e63c970 0x563a4e59f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e63c970 msgr2=0x563a4e59f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a4e63c970 0x563a4e59f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 msgr2=0x563a4e666d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e666d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a4e4f01c0 con 0x563a4e5c05f0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a4e57b1b0 0x563a4e666d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e5c05f0 0x563a4e5aac40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda64002ec0 tx=0x7fda64000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda7101c700 1 -- 172.21.15.45:0/4052796629 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64016030 con 0x563a4e5c05f0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda7101c700 1 -- 172.21.15.45:0/4052796629 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda64008070 con 0x563a4e5c05f0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda7101c700 1 -- 172.21.15.45:0/4052796629 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400ccc0 con 0x563a4e5c05f0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a4e4d63d0 con 0x563a4e5c05f0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.187+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a4e4c9ae0 con 0x563a4e5c05f0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.189+0000 7fda7101c700 1 -- 172.21.15.45:0/4052796629 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda64019070 con 0x563a4e5c05f0 2023-10-06T05:22:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.189+0000 7fda7101c700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda60065910 0x7fda60067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.189+0000 7fda7101c700 1 -- 172.21.15.45:0/4052796629 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fda64017070 con 0x563a4e5c05f0 2023-10-06T05:22:07.195 INFO:teuthology.orchestra.run.smithi045.stdout:���� 2023-10-06T05:22:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: �� 2023-10-06T05:22:07.195 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.190+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563a4e663d90 0x563a4e664190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.190+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.0 14:00000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x563a4e663d90 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.190+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda60065910 0x7fda60067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.190+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563a4e663d90 0x563a4e664190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.190+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563a4e663d90 0x563a4e664190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.190+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda60065910 0x7fda60067dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fda680030c0 tx=0x7fda6801d000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.192+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6c014420 con 0x563a4e663d90 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.192+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7fda6c014b60 0x7fda6c0138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.192+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:08000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c014b60 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.193+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7fda6c014b60 0x7fda6c0138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.193+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7fda6c014b60 0x7fda6c0138c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.193+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=177b] v92'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fda680089d0 con 0x7fda6c014b60 2023-10-06T05:22:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.193+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7fda6801c040 0x7fda68032970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.193+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:3 14.8 14:10000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.193+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7fda6801c040 0x7fda68032970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.194+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7fda6801c040 0x7fda68032970 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.196+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.196+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.18 14:18000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c014b60 2023-10-06T05:22:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.196+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda680089d0 con 0x7fda6c014b60 2023-10-06T05:22:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.196+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7fda68007030 0x7fda68009ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.196+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:5 14.4 14:20000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda68007030 2023-10-06T05:22:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.197+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7fda68007030 0x7fda68009ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.197+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7fda68007030 0x7fda68009ab0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.200+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6c014420 con 0x7fda68007030 2023-10-06T05:22:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.200+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda6c021390 0x7fda6c0217c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.200+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:6 14.14 14:28000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c021390 2023-10-06T05:22:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.200+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda6c021390 0x7fda6c0217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.201+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda6c021390 0x7fda6c0217c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.203+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(6 [pgnls start_epoch 93 out=149b] v92'19 uv19 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fda680089d0 con 0x7fda6c021390 2023-10-06T05:22:07.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.203+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:7 14.c 14:30000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.204+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.204+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:8 14.1c 14:38000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.204+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(8 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.204+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:9 14.2 14:40000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda68007030 2023-10-06T05:22:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.204+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6c014420 con 0x7fda68007030 2023-10-06T05:22:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.204+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:10 14.12 14:48000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x563a4e663d90 2023-10-06T05:22:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.205+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6c014420 con 0x563a4e663d90 2023-10-06T05:22:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.205+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7fda6c011840 0x7fda6c020320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.205+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:11 14.a 14:50000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c011840 2023-10-06T05:22:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.205+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7fda6c011840 0x7fda6c020320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.205+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7fda6c011840 0x7fda6c020320 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6c011840 2023-10-06T05:22:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:12 14.1a 14:58000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:13 14.6 14:60000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(13 [pgnls start_epoch 93 out=77b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda73821700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fda64001bd0 0x7fda64052890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- osd_op(unknown.0.0:14 14.16 14:68000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda64001bd0 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.208+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fda64001bd0 0x7fda64052890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.209+0000 7fda74823700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fda64001bd0 0x7fda64052890 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.209+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.3 v2:172.21.15.45:6824/119762 1 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda64001bd0 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.209+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:15 14.e 14:70000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c021390 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.209+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(15 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fda680089d0 con 0x7fda6c021390 2023-10-06T05:22:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.209+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7fda6801c410 0x7fda6801c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.209+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:16 14.1e 14:78000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c410 2023-10-06T05:22:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.210+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7fda6801c410 0x7fda6801c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.210+0000 7fda74022700 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7fda6801c410 0x7fda6801c860 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.213+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda680089d0 con 0x7fda6801c410 2023-10-06T05:22:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.213+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- osd_op(unknown.0.0:17 14.1 14:80000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda64001bd0 2023-10-06T05:22:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.213+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.3 v2:172.21.15.45:6824/119762 2 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda64001bd0 2023-10-06T05:22:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.213+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:18 14.11 14:88000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c021390 2023-10-06T05:22:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.213+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6c021390 2023-10-06T05:22:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.214+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:19 14.9 14:90000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c011840 2023-10-06T05:22:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.214+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6c011840 2023-10-06T05:22:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.214+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:20 14.19 14:98000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda68007030 2023-10-06T05:22:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.214+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda68007030 2023-10-06T05:22:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.214+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:21 14.5 14:a0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.215+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.215+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:22 14.15 14:a8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c410 2023-10-06T05:22:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.215+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(22 [pgnls start_epoch 93 out=97b] v91'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6801c410 2023-10-06T05:22:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.215+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:23 14.d 14:b0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda68007030 2023-10-06T05:22:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.215+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(23 [pgnls start_epoch 93 out=74b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda68007030 2023-10-06T05:22:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.216+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:24 14.1d 14:b8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.216+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(24 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.216+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:25 14.3 14:c0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.216+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.216+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:26 14.13 14:c8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x563a4e663d90 2023-10-06T05:22:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.216+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(26 [pgnls start_epoch 93 out=98b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.216+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:27 14.b 14:d0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.217+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(27 [pgnls start_epoch 93 out=76b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.217+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.1b 14:d8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c014b60 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.217+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6c014b60 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.217+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:29 14.7 14:e0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c011840 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.218+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6c011840 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.218+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.17 14:e8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6c014b60 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.218+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6c014b60 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.218+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:31 14.f 14:f0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c410 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.218+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(31 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6801c410 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.218+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:32 14.1f 14:f8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6646d0 con 0x7fda6801c040 2023-10-06T05:22:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(32 [pgnls start_epoch 93 out=101b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:33 14.10 14:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e664b40 con 0x7fda6c014b60 2023-10-06T05:22:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(33 601.00000000 [read 0~0] v0'0 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6c014b60 2023-10-06T05:22:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:34 14.10 14:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e664f20 con 0x7fda6c014b60 2023-10-06T05:22:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(34 601.00000000 [getxattrs out=124b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6c014b60 2023-10-06T05:22:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:35 14.10 14:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e59ec60 con 0x7fda6c014b60 2023-10-06T05:22:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(35 601.00000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda6805a040 con 0x7fda6c014b60 2023-10-06T05:22:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.219+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:36 14.10 14:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e59f090 con 0x7fda6c014b60 2023-10-06T05:22:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.220+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(36 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.220+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:37 14.10 14:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6703b0 con 0x7fda6c014b60 2023-10-06T05:22:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.220+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(37 602.00000000 [read 0~0] v0'0 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.220+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:38 14.10 14:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670760 con 0x7fda6c014b60 2023-10-06T05:22:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.220+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(38 602.00000000 [getxattrs out=124b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.220+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:39 14.10 14:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670b10 con 0x7fda6c014b60 2023-10-06T05:22:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(39 602.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:40 14.10 14:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e671ed0 con 0x7fda6c014b60 2023-10-06T05:22:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(40 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:41 14.10 14:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(41 10000000000.00000000 [read 0~0] v0'0 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:42 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(42 10000000000.00000000 [getxattrs out=124b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+124 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.227 INFO:teuthology.orchestra.run.smithi045.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir 2023-10-06T05:22:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.221+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:43 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.222+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(43 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.222+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:44 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672280 con 0x7fda6c014b60 2023-10-06T05:22:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.222+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals out=513b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+513 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.222+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:45 14.10 14:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.222+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(45 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.222+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:46 14.10 14:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.222+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(46 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.223+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:47 14.10 14:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.223+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(47 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.223+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:48 14.10 14:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.223+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(48 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.223+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:49 14.10 14:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.223+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(49 603.00000000 [read 0~0] v0'0 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.223+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:50 14.10 14:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.224+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(50 603.00000000 [getxattrs out=124b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.224+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:51 14.10 14:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c014b60 2023-10-06T05:22:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.224+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(51 603.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.224+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:52 14.10 14:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672280 con 0x7fda6c014b60 2023-10-06T05:22:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.224+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(52 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c014b60 2023-10-06T05:22:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.224+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:53 14.14 14:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.225+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(53 1.00000000.inode [read 0~554 out=554b] v0'0 uv17 ondisk = 0) v8 ==== 160+0+554 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.225+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:54 14.14 14:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.226+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(54 1.00000000.inode [getxattrs out=4b] v0'0 uv17 ondisk = 0) v8 ==== 160+0+4 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.235 INFO:teuthology.orchestra.run.smithi045.stdout:���� ��  (w�e�K� (w�e�K� 8`w�e�[�#8`w�e�[�#���� �� sixmegs_head�i���w�e�[�#���@@`��������w�e�[�#w�e�K� (8`w�e�[�#8`w�e�[�#��������w�e�K� 0���������������� �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��D>**ceph fs volume v011 �w�eÔ¤o�C@@��������w�eÔ¤o(w�eÔ¤o8`w�e�[�#8o�eG�o�eG�����>8������������ �� 2023-10-06T05:22:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.226+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:55 14.14 14:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.236 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.226+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(55 1.00000000.inode [omap-get-header] v0'0 uv17 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.226+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:56 14.14 14:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672280 con 0x7fda6c021390 2023-10-06T05:22:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.226+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(56 1.00000000.inode [omap-get-vals out=5b] v0'0 uv17 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.226+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:57 14.14 14:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.227+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(57 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.227+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:58 14.14 14:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.238 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@3@3@�6@@@���� �� 2023-10-06T05:22:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.227+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(58 200.00000000 [getxattrs out=4b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.227+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:59 14.14 14:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.239 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.227+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(59 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.227+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:60 14.14 14:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672280 con 0x7fda6c021390 2023-10-06T05:22:07.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.228+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(60 200.00000000 [omap-get-vals out=5b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.228+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:61 14.14 14:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.240 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ��77�6�6��XS&YA1h^To�e�g�2 ((88 2 ( 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:( 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:8 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:o�eښ�8 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:o�eښ� ��������@@��XS&YA1�� 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:�v�e�9,��P���C ceph_sha1(68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph_version]ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable) entity_id0hostname smithi045 mount_point/home/ubuntu/cephtest/mnt.0pid154946root/�� ��|@��XS&YA1���w�e�Wksetattr� '���������w�e^�d�C@@��������o�eG�(8w�e^�d8o�eG�o�eG�����>8�����������������:@��XS&YA1���w�e@�ymkdirY' (w�eÔ¤o(w�eÔ¤o8w�eÔ¤o8w�eÔ¤o �subdir���������w�eÔ¤o�A�����������������w�eÔ¤ow�eÔ¤o(88��������w�eÔ¤o������������2 ((88  '���������w�eÔ¤o�C@@��������w�eÔ¤o(w�eÔ¤o8w�eÔ¤o8o�eG�o�eG�����>8�������������������M@��XS&YA1}siw�e�� openc4' (w�eÔ¤o(w�eÔ¤o8w�e�K� 8w�eÔ¤o �subdir���������w�e�K� �A�����������������w�e�K� w�eÔ¤o(w�e�K� 8w�e�K� 8w�e�K� ��������w�eÔ¤o������������J (w�e�K� (w�e�K� 8w�e�K� 8w�e�K� $ sixmegs���������w�e�K� ���@@��������w�e�K� w�e�K� �@(88��������w�e�K� �������������� ��������@��XS&YA1�xnw�e4H�! 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:cap update4' (w�eÔ¤o(w�eÔ¤o8 w�ed'�!8w�eÔ¤o �subdir���������w�e�K� �A�����������������w�e�K� w�eÔ¤o(w�e�K� 8 w�ed'�!8 w�ed'�!��������w�eÔ¤o������������J (w�e�K� (w�e�K� 8 w�ed'�!8 w�ed'�!$ sixmegs���������w�ed'�!���@@ ��������w�ed'�!w�e�K� ��(8 w�ed'�!8 w�ed'�!��������w�e�K� ����������������������@��XS&YA1�xnw�e!L<# 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:cap update4'  (w�eÔ¤o(w�eÔ¤o8Pw�e��-#8w�eÔ¤o �subdir���������w�e�K� �A�����������������w�e�K� w�eÔ¤o(w�e�K� 8Pw�e��-#8Pw�e��-#��������w�eÔ¤o������������J (w�e�K� (w�e�K� 8Pw�e��-#8Pw�e��-#$ sixmegs���������w�e��-#���@@P��������w�e��-#w�e�K� ��(8Pw�e��-#8Pw�e��-#��������w�e�K� '���������������������*@��XS&YA1\RHw�eBʨ9 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:cap update'  (w�eÔ¤o(w�eÔ¤o8`w�e�[�#8w�eÔ¤o �subdir�������� 2023-10-06T05:22:07.241 INFO:teuthology.orchestra.run.smithi045.stdout:�w�e�K� �A�����������������w�e�K� w�eÔ¤o(w�e�K� 8`w�e�[�#8`w�e�[�# 2023-10-06T05:22:07.242 INFO:teuthology.orchestra.run.smithi045.stdout:��������w�eÔ¤o������������$  (w�e�K� (w�e�K� 8`w�e�[�#8`w�e�[�#� �sixmegs���������w�e�[�#���@@`��������w�e�[�#w�e�K� (8`w�e�[�#8`w�e�[�#��������w�e�K� 0����������������������&@��XS&YA1�� 2023-10-06T05:22:07.242 INFO:teuthology.orchestra.run.smithi045.stdout:�w�e���9��P������0.@��XS&YA18.$��.@��XS&YA1B8.}�e���scatter_writebehind�2  (w�eÔ¤o(w�eÔ¤o8`w�e�[�#8`w�e�[�#  '���������w�eÔ¤o�C@@��������w�eÔ¤o(w�eÔ¤o8`w�e�[�#8o�eG�o�eG�����>8����������������)/@��XS&YA1h^T~�e�0-)2  (w�eÔ¤o(w�eÔ¤o8`w�e�[�#8`w�e�[�# 2 ( 2023-10-06T05:22:07.242 INFO:teuthology.orchestra.run.smithi045.stdout:( 2023-10-06T05:22:07.242 INFO:teuthology.orchestra.run.smithi045.stdout:8 2023-10-06T05:22:07.242 INFO:teuthology.orchestra.run.smithi045.stdout:o�eښ�8 2023-10-06T05:22:07.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.228+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(61 200.00000001 [read 0~14075 out=14075b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+14075 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.228+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:62 14.14 14:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.243 INFO:teuthology.orchestra.run.smithi045.stdout:o�eښ� ��������@ 3@���� �� 2023-10-06T05:22:07.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.229+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(62 200.00000001 [getxattrs out=4b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.229+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.14 14:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.244 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.230+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(63 200.00000001 [omap-get-header] v0'0 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.230+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.14 14:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.230+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(64 200.00000001 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.230+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.14 14:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.230+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(65 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.230+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.14 14:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.231+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(66 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.231+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.14 14:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6c021390 2023-10-06T05:22:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.231+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(67 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.231+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.14 14:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672280 con 0x7fda6c021390 2023-10-06T05:22:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.232+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(68 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.232+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:69 14.1c 14:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e670ec0 con 0x7fda6801c040 2023-10-06T05:22:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.232+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(69 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.232+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:70 14.1c 14:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6712a0 con 0x7fda6801c040 2023-10-06T05:22:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(70 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:71 14.1c 14:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6716a0 con 0x7fda6801c040 2023-10-06T05:22:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(71 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:72 14.1c 14:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e671ad0 con 0x7fda6801c040 2023-10-06T05:22:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(72 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:73 14.6 14:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672280 con 0x7fda6801c040 2023-10-06T05:22:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(73 mds0_openfiles.0 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.233+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:74 14.6 14:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672630 con 0x7fda6801c040 2023-10-06T05:22:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.234+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(74 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+4 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.250 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2023-10-06T05:22:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.234+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:75 14.6 14:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6801c040 2023-10-06T05:22:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.234+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(75 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+42 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.234+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:76 14.6 14:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6739f0 con 0x7fda6801c040 2023-10-06T05:22:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.234+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(76 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.234+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:77 14.e 14:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6c021390 2023-10-06T05:22:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.235+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(77 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.235+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:78 14.e 14:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6c021390 2023-10-06T05:22:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.235+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(78 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.235+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:79 14.e 14:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6c021390 2023-10-06T05:22:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.235+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(79 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.235+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:80 14.e 14:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6739f0 con 0x7fda6c021390 2023-10-06T05:22:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.236+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(80 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6c021390 2023-10-06T05:22:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.236+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:81 14.15 14:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6801c410 2023-10-06T05:22:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.236+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(81 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.236+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:82 14.15 14:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6801c410 2023-10-06T05:22:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.236+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(82 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.236+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:83 14.15 14:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6801c410 2023-10-06T05:22:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.237+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(83 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.237+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:84 14.15 14:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6739f0 con 0x7fda6801c410 2023-10-06T05:22:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.237+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(84 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.237+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:85 14.15 14:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6801c410 2023-10-06T05:22:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.238+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(85 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.238+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:86 14.15 14:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6801c410 2023-10-06T05:22:07.258 INFO:teuthology.orchestra.run.smithi045.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@���� �� 2023-10-06T05:22:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.238+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(86 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.238+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:87 14.15 14:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda6801c410 2023-10-06T05:22:07.258 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.238+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(87 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.239+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:88 14.15 14:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6739f0 con 0x7fda6801c410 2023-10-06T05:22:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.239+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(88 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.239+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:89 14.d 14:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6729e0 con 0x7fda68007030 2023-10-06T05:22:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.239+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(89 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda68007030 2023-10-06T05:22:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.239+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:90 14.d 14:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e672d90 con 0x7fda68007030 2023-10-06T05:22:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.240+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 6 ==== osd_op_reply(90 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+4 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda68007030 2023-10-06T05:22:07.260 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2023-10-06T05:22:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.240+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:91 14.d 14:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e673140 con 0x7fda68007030 2023-10-06T05:22:07.261 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.240+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 7 ==== osd_op_reply(91 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda68007030 2023-10-06T05:22:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.240+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:92 14.d 14:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674500 con 0x7fda68007030 2023-10-06T05:22:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.240+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.6 v2:172.21.15.144:6800/119899 8 ==== osd_op_reply(92 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7fda6c02c040 con 0x7fda68007030 2023-10-06T05:22:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.240+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:93 14.1d 14:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6734f0 con 0x7fda6801c040 2023-10-06T05:22:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.241+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(93 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.241+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:94 14.1d 14:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6734f0 con 0x7fda6801c040 2023-10-06T05:22:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.241+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(94 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.241+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:95 14.1d 14:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6734f0 con 0x7fda6801c040 2023-10-06T05:22:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.241+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(95 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.241+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:96 14.1d 14:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6748b0 con 0x7fda6801c040 2023-10-06T05:22:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(96 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6734f0 con 0x563a4e663d90 2023-10-06T05:22:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6738a0 con 0x563a4e663d90 2023-10-06T05:22:07.265 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2023-10-06T05:22:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(98 mds0_inotable [getxattrs out=4b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+4 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e673ca0 con 0x563a4e663d90 2023-10-06T05:22:07.266 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(99 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.242+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740a0 con 0x563a4e663d90 2023-10-06T05:22:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.243+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(100 mds0_inotable [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.243+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:101 14.13 14:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740a0 con 0x563a4e663d90 2023-10-06T05:22:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.243+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(101 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.243+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:102 14.13 14:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x563a4e663d90 2023-10-06T05:22:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.243+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(102 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.268 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2023-10-06T05:22:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.243+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:103 14.13 14:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x563a4e663d90 2023-10-06T05:22:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.244+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(103 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.269 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ( 2023-10-06T05:22:07.269 INFO:teuthology.orchestra.run.smithi045.stdout:( 2023-10-06T05:22:07.269 INFO:teuthology.orchestra.run.smithi045.stdout:8 2023-10-06T05:22:07.269 INFO:teuthology.orchestra.run.smithi045.stdout:o�eښ�8 2023-10-06T05:22:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.244+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:104 14.13 14:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x563a4e663d90 2023-10-06T05:22:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.244+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 <== osd.2 v2:172.21.15.45:6808/119739 11 ==== osd_op_reply(104 100.00000000 [omap-get-vals out=5075b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5075 (crc 0 0 0) 0x7fda6c02c040 con 0x563a4e663d90 2023-10-06T05:22:07.270 INFO:teuthology.orchestra.run.smithi045.stdout:o�eښ����� ���� 2023-10-06T05:22:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.244+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:105 14.b 14:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6801c040 2023-10-06T05:22:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.244+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(105 mds0_sessionmap [read 0~0] v0'0 uv3 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.244+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:106 14.b 14:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6801c040 2023-10-06T05:22:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.245+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(106 mds0_sessionmap [getxattrs out=4b] v0'0 uv3 ondisk = 0) v8 ==== 159+0+4 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.271 INFO:teuthology.orchestra.run.smithi045.stdout: stray0_head�i���o�eM��@A����������������o�eM��(8o�eM��8o�eM��o�eM�������������� stray1_head�i���o�el��@A����������������o�el��(8o�el��8o�el��o�el�������������� stray2_head�i���o�e ��@A����������������o�e ��(8o�e ��8o�e ��o�e �������������� stray3_head�i���o�e�;�@A����������������o�e�;�(8o�e�;�8o�e�;�o�e�;������������� stray4_head�i���o�e ��@A����������������o�e ��(8o�e ��8o�e ��o�e �������������� stray5_head�i���o�e�N�@A����������������o�e�N�(8o�e�N�8o�e�N�o�e�N������������� stray6_head�i���o�e���@A����������������o�e���(8o�e���8o�e���o�e��������������� stray7_head�i���o�e��@A����������������o�e��(8o�e��8o�e��o�e�������������� stray8_head�i���o�e���@A����������������o�e���(8o�e���8o�e���o�e��������������� stray9_head�i��� o�eښ�@A����������������o�eښ�(8o�eښ�8o�eښ�o�eښ����������������� �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2023-10-06T05:22:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.245+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:107 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6801c040 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.245+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(107 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) v8 ==== 159+0+14 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.245+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:108 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6801c040 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.245+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(108 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 159+0+5 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.245+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6c011840 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(109 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6c011840 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6c011840 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(110 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+124 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6c011840 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:111 14.7 14:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6c011840 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(111 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6c011840 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:112 14.7 14:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e6740d0 con 0x7fda6c011840 2023-10-06T05:22:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.5 v2:172.21.15.144:6816/119917 7 ==== osd_op_reply(112 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6c011840 2023-10-06T05:22:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.246+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c410 2023-10-06T05:22:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.247+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(113 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.247+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c410 2023-10-06T05:22:07.276 INFO:teuthology.orchestra.run.smithi045.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2023-10-06T05:22:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.247+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(114 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+4 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.247+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c410 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.248+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(115 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.248+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:116 14.f 14:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c410 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.248+0000 7fda74022700 1 -- 172.21.15.45:0/4052796629 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(116 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fda68063040 con 0x7fda6801c410 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.248+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.248+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(117 100.00000000.inode [read 0~554 out=554b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+554 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��D>**ceph fs volume v011 �o�e#R�@A����������������o�e#R�( 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.248+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(118 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+4 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi045.stdout:8 o�eښ�8 o�eښ�o�e#R�����>8������������ �� 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(119 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.280 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� 2023-10-06T05:22:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(120 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+5 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.281 INFO:teuthology.orchestra.run.smithi045.stdout:���� �� ��("+ 2023-10-06T05:22:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(121 1.00000000 [read 0~0] v0'0 uv6 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.249+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(122 1.00000000 [getxattrs out=92b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+92 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 33 ==== osd_op_reply(123 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x563a4e674050 con 0x7fda6801c040 2023-10-06T05:22:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda73821700 1 -- 172.21.15.45:0/4052796629 <== osd.0 v2:172.21.15.45:6816/119749 34 ==== osd_op_reply(124 1.00000000 [omap-get-vals out=512b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+512 (crc 0 0 0) 0x7fda64051cd0 con 0x7fda6801c040 2023-10-06T05:22:07.283 INFO:teuthology.orchestra.run.smithi045.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@_parent���� ��  (w�eÔ¤o(w�eÔ¤o8`w�e�[�#8`w�e�[�#���� ��� subdir_head�i���w�e�K� �A�����������������w�e�K� w�eÔ¤o(w�e�K� 8`w�e�[�#8`w�e�[�# 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7fda6801c040 msgr2=0x7fda68032970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7fda6801c040 0x7fda68032970 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7fda6c014b60 msgr2=0x7fda6c0138c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7fda6c014b60 0x7fda6c0138c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563a4e663d90 msgr2=0x563a4e664190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563a4e663d90 0x563a4e664190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fda64001bd0 msgr2=0x7fda64052890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fda64001bd0 0x7fda64052890 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda6c021390 msgr2=0x7fda6c0217c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda6c021390 0x7fda6c0217c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 reap_dead start 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7fda6c011840 msgr2=0x7fda6c020320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7fda6c011840 0x7fda6c020320 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7fda68007030 msgr2=0x7fda68009ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7fda68007030 0x7fda68009ab0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda74823700 1 -- 172.21.15.45:0/4052796629 reap_dead start 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7fda6801c410 msgr2=0x7fda6801c860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7fda6801c410 0x7fda6801c860 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda60065910 msgr2=0x7fda60067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.250+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda60065910 0x7fda60067dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fda680030c0 tx=0x7fda6801d000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e5c05f0 msgr2=0x563a4e5aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e5c05f0 0x563a4e5aac40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda64002ec0 tx=0x7fda64000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 shutdown_connections 2023-10-06T05:22:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7fda6801c410 0x7fda6801c860 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563a4e5c05f0 0x563a4e5aac40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 --2- 172.21.15.45:0/4052796629 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda60065910 0x7fda60067dc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 >> 172.21.15.45:0/4052796629 conn(0x563a4e57d4a0 msgr2=0x563a4e639480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 shutdown_connections 2023-10-06T05:22:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.251+0000 7fda7ae04040 1 -- 172.21.15.45:0/4052796629 wait complete. 2023-10-06T05:22:07.287 INFO:teuthology.orchestra.run.smithi045.stdout:��������w�eÔ¤o���������������� �� �� 2023-10-06T05:22:07.287 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2023-10-06T05:22:07.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.371+0000 7f25e4126040 1 Processor -- start 2023-10-06T05:22:07.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.371+0000 7f25e4126040 1 -- start start 2023-10-06T05:22:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25e4126040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604f4ded0 0x55c604ff3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25e4126040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604ff59f0 0x55c604ff3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25e4126040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c604f52590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25e4126040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c604e6d980 con 0x55c604ff9e40 2023-10-06T05:22:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25e4126040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c604e6d620 con 0x55c604f4ded0 2023-10-06T05:22:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25e4126040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c604e64400 con 0x55c604ff59f0 2023-10-06T05:22:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25ddb45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c604f52590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25ddb45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c604f52590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54678/0 (socket says 172.21.15.45:54678) 2023-10-06T05:22:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.372+0000 7f25ddb45700 1 -- 172.21.15.45:0/3421971298 learned_addr learned my addr 172.21.15.45:0/3421971298 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25dcb43700 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604ff59f0 0x55c604ff3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25dd344700 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604f4ded0 0x55c604ff3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25ddb45700 1 -- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604ff59f0 msgr2=0x55c604ff3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25ddb45700 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604ff59f0 0x55c604ff3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25ddb45700 1 -- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604f4ded0 msgr2=0x55c604ff3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25ddb45700 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604f4ded0 0x55c604ff3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25ddb45700 1 -- 172.21.15.45:0/3421971298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8005cf0 con 0x55c604ff9e40 2023-10-06T05:22:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25dd344700 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604f4ded0 0x55c604ff3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25ddb45700 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c604f52590 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f25d80016e0 tx=0x7f25d8006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c423313bb4610760 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25d77fe700 1 -- 172.21.15.45:0/3421971298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80020f0 con 0x55c604ff9e40 2023-10-06T05:22:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25d77fe700 1 -- 172.21.15.45:0/3421971298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8006320 con 0x55c604ff9e40 2023-10-06T05:22:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.373+0000 7f25d77fe700 1 -- 172.21.15.45:0/3421971298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8004750 con 0x55c604ff9e40 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 -- 172.21.15.45:0/3421971298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 msgr2=0x55c604f52590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c604f52590 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f25d80016e0 tx=0x7f25d8006950 comp rx=0 tx=0).stop 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 -- 172.21.15.45:0/3421971298 shutdown_connections 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c604f52590 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604f4ded0 0x55c604ff3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 --2- 172.21.15.45:0/3421971298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604ff59f0 0x55c604ff3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 -- 172.21.15.45:0/3421971298 >> 172.21.15.45:0/3421971298 conn(0x55c604f234a0 msgr2=0x55c604f24570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 -- 172.21.15.45:0/3421971298 shutdown_connections 2023-10-06T05:22:07.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.374+0000 7f25e4126040 1 -- 172.21.15.45:0/3421971298 wait complete. 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.375+0000 7f25e4126040 1 Processor -- start 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.375+0000 7f25e4126040 1 -- start start 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25e4126040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604f4ded0 0x55c605005250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25e4126040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604ff59f0 0x55c604ff17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25e4126040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c605008830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25e4126040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c605009790 con 0x55c604ff9e40 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25e4126040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c605008d70 con 0x55c604ff59f0 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25e4126040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c604ff87a0 con 0x55c604f4ded0 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25ddb45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c605008830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25ddb45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c605008830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54680/0 (socket says 172.21.15.45:54680) 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25dd344700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604f4ded0 0x55c605005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 learned_addr learned my addr 172.21.15.45:0/2458921982 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.376+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604ff59f0 0x55c604ff17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604f4ded0 msgr2=0x55c605005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c604f4ded0 0x55c605005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604ff59f0 msgr2=0x55c604ff17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c604ff59f0 0x55c604ff17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8006c30 con 0x55c604ff9e40 2023-10-06T05:22:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c605008830 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f25d8001d80 tx=0x7f25d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25d5ffb700 1 -- 172.21.15.45:0/2458921982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8004e80 con 0x55c604ff9e40 2023-10-06T05:22:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25d5ffb700 1 -- 172.21.15.45:0/2458921982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d80020f0 con 0x55c604ff9e40 2023-10-06T05:22:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25d5ffb700 1 -- 172.21.15.45:0/2458921982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8013d60 con 0x55c604ff9e40 2023-10-06T05:22:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c604e7c3d0 con 0x55c604ff9e40 2023-10-06T05:22:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.377+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c604e6fae0 con 0x55c604ff9e40 2023-10-06T05:22:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.378+0000 7f25d5ffb700 1 -- 172.21.15.45:0/2458921982 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25d801e630 con 0x55c604ff9e40 2023-10-06T05:22:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.379+0000 7f25d5ffb700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0065890 0x7f25c0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.379+0000 7f25d5ffb700 1 -- 172.21.15.45:0/2458921982 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f25d8085520 con 0x55c604ff9e40 2023-10-06T05:22:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.379+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c604fefd30 0x55c604ff8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.379+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0065890 0x7f25c0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.379+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x55c604fefd30 2023-10-06T05:22:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.379+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c604fefd30 0x55c604ff8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.380+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0065890 0x7f25c0067d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009c20 tx=0x7f25d000c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.380+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c604fefd30 0x55c604ff8c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.380+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x55c604fefd30 2023-10-06T05:22:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.380+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f25c8007030 0x7f25c801d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.380+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c8007030 2023-10-06T05:22:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.380+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f25c8007030 0x7f25c801d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.381+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f25c8007030 0x7f25c801d420 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.381+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=177b] v92'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f25d0007960 con 0x7f25c8007030 2023-10-06T05:22:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.381+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f25d0007040 0x7f25d0016920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.381+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:3 14.8 14:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.381+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f25d0007040 0x7f25d0016920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.382+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f25d0007040 0x7f25d0016920 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.382+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.382+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.18 14:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c8007030 2023-10-06T05:22:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.382+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d0007960 con 0x7f25c8007030 2023-10-06T05:22:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.382+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f25d0024a50 0x7f25d00153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.382+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:5 14.4 14:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0024a50 2023-10-06T05:22:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.383+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f25d0024a50 0x7f25d00153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.383+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f25d0024a50 0x7f25d00153b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.383+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d8024020 con 0x7f25d0024a50 2023-10-06T05:22:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.383+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f25d8026360 0x7f25d8055040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.383+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:6 14.14 14:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d8026360 2023-10-06T05:22:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.384+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f25d8026360 0x7f25d8055040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.384+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f25d8026360 0x7f25d8055040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(6 [pgnls start_epoch 93 out=149b] v92'19 uv19 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f25d0007960 con 0x7f25d8026360 2023-10-06T05:22:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:7 14.c 14:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:8 14.1c 14:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(8 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:9 14.2 14:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0024a50 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d8024020 con 0x7f25d0024a50 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.385+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:10 14.12 14:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x55c604fefd30 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.386+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x55c604fefd30 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.386+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f25c802c040 0x7f25c802c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.386+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:11 14.a 14:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c802c040 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.386+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f25c802c040 0x7f25c802c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.386+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f25c802c040 0x7f25c802c410 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.387+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25c802c040 2023-10-06T05:22:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.387+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:12 14.1a 14:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.387+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.387+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:13 14.6 14:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.387+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(13 [pgnls start_epoch 93 out=77b] v92'2 uv2 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.387+0000 7f25dcb43700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7f25c803d930 0x7f25c801c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.387+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- osd_op(unknown.0.0:14 14.16 14:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c803d930 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.388+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7f25c803d930 0x7f25c801c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.388+0000 7f25ddb45700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7f25c803d930 0x7f25c801c3b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.388+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 <== osd.3 v2:172.21.15.45:6824/119762 1 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d8024020 con 0x7f25c803d930 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.388+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:15 14.e 14:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d8026360 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.389+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(15 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f25d0007960 con 0x7f25d8026360 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.389+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f25d0015c20 0x7f25d00063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.389+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:16 14.1e 14:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0015c20 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.389+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f25d0015c20 0x7f25d00063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.389+0000 7f25dd344700 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f25d0015c20 0x7f25d00063a0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.390+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d0007960 con 0x7f25d0015c20 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.390+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- osd_op(unknown.0.0:17 14.1 14:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c803d930 2023-10-06T05:22:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.390+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 <== osd.3 v2:172.21.15.45:6824/119762 2 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d8024020 con 0x7f25c803d930 2023-10-06T05:22:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.390+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:18 14.11 14:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d8026360 2023-10-06T05:22:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.390+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d0007960 con 0x7f25d8026360 2023-10-06T05:22:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.390+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:19 14.9 14:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c802c040 2023-10-06T05:22:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.391+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25c802c040 2023-10-06T05:22:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.391+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:20 14.19 14:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0024a50 2023-10-06T05:22:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.391+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d8024020 con 0x7f25d0024a50 2023-10-06T05:22:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.391+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:21 14.5 14:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.391+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.391+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:22 14.15 14:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0015c20 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.392+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(22 [pgnls start_epoch 93 out=97b] v91'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f25d0007960 con 0x7f25d0015c20 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.392+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:23 14.d 14:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0024a50 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.392+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(23 [pgnls start_epoch 93 out=74b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f25d8024020 con 0x7f25d0024a50 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.392+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:24 14.1d 14:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.392+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(24 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.392+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:25 14.3 14:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.393+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.393+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:26 14.13 14:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x55c604fefd30 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.393+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(26 [pgnls start_epoch 93 out=98b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f25c8000bc0 con 0x55c604fefd30 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.393+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:27 14.b 14:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.393+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(27 [pgnls start_epoch 93 out=76b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.393+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.1b 14:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c8007030 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.394+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d0007960 con 0x7f25c8007030 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.394+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:29 14.7 14:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c802c040 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.394+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25c802c040 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.394+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.17 14:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25c8007030 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.394+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f25d0007960 con 0x7f25c8007030 2023-10-06T05:22:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.394+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:31 14.f 14:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0015c20 2023-10-06T05:22:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(31 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f25d0007960 con 0x7f25d0015c20 2023-10-06T05:22:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25dd344700 1 -- 172.21.15.45:0/2458921982 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:32 14.1f 14:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c604ff9140 con 0x7f25d0007040 2023-10-06T05:22:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25dcb43700 1 -- 172.21.15.45:0/2458921982 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(32 [pgnls start_epoch 93 out=101b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f25c8000bc0 con 0x7f25d0007040 2023-10-06T05:22:07.414 INFO:teuthology.orchestra.run.smithi045.stdout:601.00000000 2023-10-06T05:22:07.415 INFO:teuthology.orchestra.run.smithi045.stdout:602.00000000 2023-10-06T05:22:07.415 INFO:teuthology.orchestra.run.smithi045.stdout:10000000000.00000000 2023-10-06T05:22:07.415 INFO:teuthology.orchestra.run.smithi045.stdout:600.00000000 2023-10-06T05:22:07.415 INFO:teuthology.orchestra.run.smithi045.stdout:603.00000000 2023-10-06T05:22:07.415 INFO:teuthology.orchestra.run.smithi045.stdout:1.00000000.inode 2023-10-06T05:22:07.415 INFO:teuthology.orchestra.run.smithi045.stdout:200.00000000 2023-10-06T05:22:07.415 INFO:teuthology.orchestra.run.smithi045.stdout:200.00000001 2023-10-06T05:22:07.416 INFO:teuthology.orchestra.run.smithi045.stdout:606.00000000 2023-10-06T05:22:07.416 INFO:teuthology.orchestra.run.smithi045.stdout:607.00000000 2023-10-06T05:22:07.416 INFO:teuthology.orchestra.run.smithi045.stdout:mds0_openfiles.0 2023-10-06T05:22:07.416 INFO:teuthology.orchestra.run.smithi045.stdout:608.00000000 2023-10-06T05:22:07.416 INFO:teuthology.orchestra.run.smithi045.stdout:604.00000000 2023-10-06T05:22:07.416 INFO:teuthology.orchestra.run.smithi045.stdout:500.00000000 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:mds_snaptable 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:605.00000000 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:mds0_inotable 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:100.00000000 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:mds0_sessionmap 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:609.00000000 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:400.00000000 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:100.00000000.inode 2023-10-06T05:22:07.417 INFO:teuthology.orchestra.run.smithi045.stdout:1.00000000 2023-10-06T05:22:07.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f25d0007040 msgr2=0x7f25d0016920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f25d0007040 0x7f25d0016920 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f25c8007030 msgr2=0x7f25c801d420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f25c8007030 0x7f25c801d420 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c604fefd30 msgr2=0x55c604ff8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c604fefd30 0x55c604ff8c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7f25c803d930 msgr2=0x7f25c801c3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7f25c803d930 0x7f25c801c3b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f25d8026360 msgr2=0x7f25d8055040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 reap_dead start 2023-10-06T05:22:07.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7f25d8026360 0x7f25d8055040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 reap_dead start 2023-10-06T05:22:07.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f25c802c040 msgr2=0x7f25c802c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f25c802c040 0x7f25c802c410 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f25d0024a50 msgr2=0x7f25d00153b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.395+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f25d0024a50 0x7f25d00153b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f25d0015c20 msgr2=0x7f25d00063a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f25d0015c20 0x7f25d00063a0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0065890 msgr2=0x7f25c0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0065890 0x7f25c0067d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009c20 tx=0x7f25d000c040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 msgr2=0x55c605008830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 --2- 172.21.15.45:0/2458921982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c604ff9e40 0x55c605008830 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f25d8001d80 tx=0x7f25d8014000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 reap_dead start 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25ddb45700 1 -- 172.21.15.45:0/2458921982 reap_dead start 2023-10-06T05:22:07.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 shutdown_connections 2023-10-06T05:22:07.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 >> 172.21.15.45:0/2458921982 conn(0x55c604f234a0 msgr2=0x55c604f24570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 shutdown_connections 2023-10-06T05:22:07.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.396+0000 7f25e4126040 1 -- 172.21.15.45:0/2458921982 wait complete. 2023-10-06T05:22:07.425 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2023-10-06T05:22:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.473+0000 7f447608e040 1 Processor -- start 2023-10-06T05:22:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.473+0000 7f447608e040 1 -- start start 2023-10-06T05:22:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f447608e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364780210 0x5623647b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f447608e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 0x56236485de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f447608e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364842c60 0x562364859630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f447608e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623646cd980 con 0x562364780210 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f447608e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623646cd620 con 0x562364842890 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f447608e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623646c4400 con 0x562364842c60 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364780210 0x5623647b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364780210 0x5623647b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54688/0 (socket says 172.21.15.45:54688) 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 -- 172.21.15.45:0/1206449935 learned_addr learned my addr 172.21.15.45:0/1206449935 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446eaab700 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 0x56236485de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446faad700 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364842c60 0x562364859630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 -- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364842c60 msgr2=0x562364859630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364842c60 0x562364859630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 -- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 msgr2=0x56236485de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 0x56236485de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 -- 172.21.15.45:0/1206449935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4460005cf0 con 0x562364780210 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446faad700 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364842c60 0x562364859630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.474+0000 7f446f2ac700 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364780210 0x5623647b16a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x5623647afac0 tx=0x7f4460015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9851b2f0d6317473 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f446daa9700 1 -- 172.21.15.45:0/1206449935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460000bc0 con 0x562364780210 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f446daa9700 1 -- 172.21.15.45:0/1206449935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44600078e0 con 0x562364780210 2023-10-06T05:22:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f446daa9700 1 -- 172.21.15.45:0/1206449935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446001c410 con 0x562364780210 2023-10-06T05:22:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 -- 172.21.15.45:0/1206449935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364780210 msgr2=0x5623647b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364780210 0x5623647b16a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x5623647afac0 tx=0x7f4460015000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 -- 172.21.15.45:0/1206449935 shutdown_connections 2023-10-06T05:22:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364842c60 0x562364859630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364780210 0x5623647b16a0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 --2- 172.21.15.45:0/1206449935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 0x56236485de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 -- 172.21.15.45:0/1206449935 >> 172.21.15.45:0/1206449935 conn(0x562364783450 msgr2=0x562364785830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 -- 172.21.15.45:0/1206449935 shutdown_connections 2023-10-06T05:22:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.475+0000 7f447608e040 1 -- 172.21.15.45:0/1206449935 wait complete. 2023-10-06T05:22:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.476+0000 7f447608e040 1 Processor -- start 2023-10-06T05:22:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.476+0000 7f447608e040 1 -- start start 2023-10-06T05:22:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f447608e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364780210 0x562364840e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f447608e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 0x562364860330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f447608e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364842c60 0x562364861b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f447608e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562364860c40 con 0x562364842c60 2023-10-06T05:22:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f447608e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562364860ee0 con 0x562364842890 2023-10-06T05:22:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f447608e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562364861180 con 0x562364780210 2023-10-06T05:22:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364842c60 0x562364861b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364842c60 0x562364861b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54696/0 (socket says 172.21.15.45:54696) 2023-10-06T05:22:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 -- 172.21.15.45:0/3347099452 learned_addr learned my addr 172.21.15.45:0/3347099452 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446f2ac700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364780210 0x562364840e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446eaab700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 0x562364860330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 -- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364780210 msgr2=0x562364840e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562364780210 0x562364840e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 -- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 msgr2=0x562364860330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562364842890 0x562364860330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.477+0000 7f446faad700 1 -- 172.21.15.45:0/3347099452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623646f61c0 con 0x562364842c60 2023-10-06T05:22:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.478+0000 7f446faad700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364842c60 0x562364861b60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f446800e590 tx=0x7f4468007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.478+0000 7f4457fff700 1 -- 172.21.15.45:0/3347099452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468002490 con 0x562364842c60 2023-10-06T05:22:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.478+0000 7f4457fff700 1 -- 172.21.15.45:0/3347099452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468013670 con 0x562364842c60 2023-10-06T05:22:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.478+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623646dc3d0 con 0x562364842c60 2023-10-06T05:22:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.478+0000 7f4457fff700 1 -- 172.21.15.45:0/3347099452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468013a10 con 0x562364842c60 2023-10-06T05:22:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.478+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623646cfae0 con 0x562364842c60 2023-10-06T05:22:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.479+0000 7f4457fff700 1 -- 172.21.15.45:0/3347099452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4468003040 con 0x562364842c60 2023-10-06T05:22:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.480+0000 7f4457fff700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f445c0658c0 0x7f445c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.480+0000 7f4457fff700 1 -- 172.21.15.45:0/3347099452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f44680836d0 con 0x562364842c60 2023-10-06T05:22:07.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.480+0000 7f447608e040 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562364784910 0x56236485a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.480+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x56236485ab70 con 0x562364784910 2023-10-06T05:22:07.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.480+0000 7f446f2ac700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f445c0658c0 0x7f445c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.480+0000 7f446eaab700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562364784910 0x56236485a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f446eaab700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562364784910 0x56236485a630 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f446f2ac700 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f445c0658c0 0x7f445c067d70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5623647afac0 tx=0x7f446001c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f446eaab700 1 -- 172.21.15.45:0/3347099452 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x7f4464010770 con 0x562364784910 2023-10-06T05:22:07.491 INFO:teuthology.orchestra.run.smithi045.stdout:header (274 bytes) : 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000000 04 03 0c 01 00 00 09 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000020 00 00 00 00 77 99 1f 65 95 4b 96 20 01 00 00 00 |....w..e.K. ....| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000050 00 00 77 99 1f 65 95 4b 96 20 01 00 00 00 00 00 |..w..e.K. ......| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.492 INFO:teuthology.orchestra.run.smithi045.stdout:000000a0 00 00 00 00 00 00 00 00 77 99 1f 65 9a 5b af 23 |........w..e.[.#| 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:000000e0 00 00 00 00 00 00 77 99 1f 65 9a 5b af 23 00 00 |......w..e.[.#..| 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:* 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:00000110 00 00 |..| 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout:00000112 2023-10-06T05:22:07.493 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562364784910 msgr2=0x56236485a630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562364784910 0x56236485a630 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f445c0658c0 msgr2=0x7f445c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f445c0658c0 0x7f445c067d70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5623647afac0 tx=0x7f446001c000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364842c60 msgr2=0x562364861b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 --2- 172.21.15.45:0/3347099452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562364842c60 0x562364861b60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f446800e590 tx=0x7f4468007040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f446faad700 1 -- 172.21.15.45:0/3347099452 reap_dead start 2023-10-06T05:22:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 shutdown_connections 2023-10-06T05:22:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 >> 172.21.15.45:0/3347099452 conn(0x562364783450 msgr2=0x5623647b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.481+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 shutdown_connections 2023-10-06T05:22:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.482+0000 7f447608e040 1 -- 172.21.15.45:0/3347099452 wait complete. 2023-10-06T05:22:07.498 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2023-10-06T05:22:07.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:07.543 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.544+0000 7f7d13b17040 1 Processor -- start 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.544+0000 7f7d13b17040 1 -- start start 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.544+0000 7f7d13b17040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae74b2210 0x55bae74f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.544+0000 7f7d13b17040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae7574890 0x55bae74e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.544+0000 7f7d13b17040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae74e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d13b17040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae73ff980 con 0x55bae7574c60 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d13b17040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae73ff620 con 0x55bae7574890 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d13b17040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae73f6400 con 0x55bae74b2210 2023-10-06T05:22:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d0d536700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae74e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d0cd35700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae74b2210 0x55bae74f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae7574890 0x55bae74e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d0d536700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae74e2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54722/0 (socket says 172.21.15.45:54722) 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d0d536700 1 -- 172.21.15.45:0/2210153583 learned_addr learned my addr 172.21.15.45:0/2210153583 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d07fff700 1 -- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae74b2210 msgr2=0x55bae74f98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d07fff700 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae74b2210 0x55bae74f98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d07fff700 1 -- 172.21.15.45:0/2210153583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 msgr2=0x55bae74e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d07fff700 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae74e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d07fff700 1 -- 172.21.15.45:0/2210153583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8005db0 con 0x55bae7574890 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d0cd35700 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae74b2210 0x55bae74f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:07.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d0d536700 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae74e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:07.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.545+0000 7f7d07fff700 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae7574890 0x55bae74e36c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a6e0 tx=0x7f7d00005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6c8885df3eda47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d06ffd700 1 -- 172.21.15.45:0/2210153583 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000f070 con 0x55bae7574890 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d06ffd700 1 -- 172.21.15.45:0/2210153583 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00008700 con 0x55bae7574890 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d06ffd700 1 -- 172.21.15.45:0/2210153583 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00008ac0 con 0x55bae7574890 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 -- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae7574890 msgr2=0x55bae74e36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae7574890 0x55bae74e36c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a6e0 tx=0x7f7d00005b40 comp rx=0 tx=0).stop 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 -- 172.21.15.45:0/2210153583 shutdown_connections 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae74e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae74b2210 0x55bae74f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 --2- 172.21.15.45:0/2210153583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae7574890 0x55bae74e36c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 -- 172.21.15.45:0/2210153583 >> 172.21.15.45:0/2210153583 conn(0x55bae74b5450 msgr2=0x55bae74f8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 -- 172.21.15.45:0/2210153583 shutdown_connections 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.546+0000 7f7d13b17040 1 -- 172.21.15.45:0/2210153583 wait complete. 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.547+0000 7f7d13b17040 1 Processor -- start 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.547+0000 7f7d13b17040 1 -- start start 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d13b17040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae74b2210 0x55bae758edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d13b17040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae7574890 0x55bae7573500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d13b17040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae7573a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d13b17040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae74e3a90 con 0x55bae7574c60 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d13b17040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae758f310 con 0x55bae74b2210 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d13b17040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae759b960 con 0x55bae7574890 2023-10-06T05:22:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae7573a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae7573a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54724/0 (socket says 172.21.15.45:54724) 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae7574890 0x55bae7573500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 -- 172.21.15.45:0/1919817871 learned_addr learned my addr 172.21.15.45:0/1919817871 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0cd35700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae74b2210 0x55bae758edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 -- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae7574890 msgr2=0x55bae7573500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bae7574890 0x55bae7573500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 -- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae74b2210 msgr2=0x55bae758edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bae74b2210 0x55bae758edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.548+0000 7f7d0d536700 1 -- 172.21.15.45:0/1919817871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bae74281c0 con 0x55bae7574c60 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.549+0000 7f7d0d536700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae7573a40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006670 tx=0x7f7d08008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.549+0000 7f7d057fa700 1 -- 172.21.15.45:0/1919817871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080123f0 con 0x55bae7574c60 2023-10-06T05:22:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.549+0000 7f7d057fa700 1 -- 172.21.15.45:0/1919817871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08012550 con 0x55bae7574c60 2023-10-06T05:22:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.549+0000 7f7d057fa700 1 -- 172.21.15.45:0/1919817871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08012910 con 0x55bae7574c60 2023-10-06T05:22:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.549+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bae7401750 con 0x55bae7574c60 2023-10-06T05:22:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.549+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d00005de0 con 0x55bae7574c60 2023-10-06T05:22:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.551+0000 7f7d057fa700 1 -- 172.21.15.45:0/1919817871 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d08003030 con 0x55bae7574c60 2023-10-06T05:22:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.551+0000 7f7d057fa700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cf40658c0 0x7f7cf4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.551+0000 7f7d057fa700 1 -- 172.21.15.45:0/1919817871 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f7d08082ee0 con 0x55bae7574c60 2023-10-06T05:22:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.551+0000 7f7d0cd35700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cf40658c0 0x7f7cf4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.552+0000 7f7d13b17040 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55bae74f8aa0 0x55bae759f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.552+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55bae755d530 con 0x55bae74f8aa0 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.552+0000 7f7d07fff700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55bae74f8aa0 0x55bae759f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.552+0000 7f7d0cd35700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cf40658c0 0x7f7cf4067d70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800afd0 tx=0x7f7cf8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.552+0000 7f7d07fff700 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55bae74f8aa0 0x55bae759f960 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.552+0000 7f7d07fff700 1 -- 172.21.15.45:0/1919817871 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7f7d00008980 con 0x55bae74f8aa0 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:header (274 bytes) : 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-10-06T05:22:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:* 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:* 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:* 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:07.558 INFO:teuthology.orchestra.run.smithi045.stdout:* 2023-10-06T05:22:07.559 INFO:teuthology.orchestra.run.smithi045.stdout:00000110 00 00 |..| 2023-10-06T05:22:07.559 INFO:teuthology.orchestra.run.smithi045.stdout:00000112 2023-10-06T05:22:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55bae74f8aa0 msgr2=0x55bae759f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55bae74f8aa0 0x55bae759f960 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cf40658c0 msgr2=0x7f7cf4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cf40658c0 0x7f7cf4067d70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800afd0 tx=0x7f7cf8016000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 msgr2=0x55bae7573a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 --2- 172.21.15.45:0/1919817871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae7574c60 0x55bae7573a40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006670 tx=0x7f7d08008040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d0d536700 1 -- 172.21.15.45:0/1919817871 reap_dead start 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 shutdown_connections 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 >> 172.21.15.45:0/1919817871 conn(0x55bae74b5450 msgr2=0x55bae7572dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 shutdown_connections 2023-10-06T05:22:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.553+0000 7f7d13b17040 1 -- 172.21.15.45:0/1919817871 wait complete. 2023-10-06T05:22:07.561 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2023-10-06T05:22:07.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.647+0000 7f90ca2e7040 1 Processor -- start 2023-10-06T05:22:07.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.647+0000 7f90ca2e7040 1 -- start start 2023-10-06T05:22:07.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90ca2e7040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff424bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90ca2e7040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff397c00 0x55c6ff425130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90ca2e7040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff40d970 0x55c6ff432f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90ca2e7040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ff29a980 con 0x55c6ff40d970 2023-10-06T05:22:07.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90ca2e7040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ff29a620 con 0x55c6ff37fbe0 2023-10-06T05:22:07.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90ca2e7040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6ff291400 con 0x55c6ff397c00 2023-10-06T05:22:07.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff40d970 0x55c6ff432f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff40d970 0x55c6ff432f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54744/0 (socket says 172.21.15.45:54744) 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 -- 172.21.15.45:0/753899521 learned_addr learned my addr 172.21.15.45:0/753899521 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c2d04700 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff397c00 0x55c6ff425130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3505700 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff424bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 -- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff397c00 msgr2=0x55c6ff425130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff397c00 0x55c6ff425130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 -- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 msgr2=0x55c6ff424bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff424bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 -- 172.21.15.45:0/753899521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b8005db0 con 0x55c6ff40d970 2023-10-06T05:22:07.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c3d06700 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff40d970 0x55c6ff432f20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c4f0 tx=0x7f90bc007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966dae391bdded52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.648+0000 7f90c1d02700 1 -- 172.21.15.45:0/753899521 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0115f0 con 0x55c6ff40d970 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90c1d02700 1 -- 172.21.15.45:0/753899521 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc011750 con 0x55c6ff40d970 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90c1d02700 1 -- 172.21.15.45:0/753899521 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc011b10 con 0x55c6ff40d970 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 -- 172.21.15.45:0/753899521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff40d970 msgr2=0x55c6ff432f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff40d970 0x55c6ff432f20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c4f0 tx=0x7f90bc007040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 -- 172.21.15.45:0/753899521 shutdown_connections 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff40d970 0x55c6ff432f20 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff424bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/753899521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff397c00 0x55c6ff425130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 -- 172.21.15.45:0/753899521 >> 172.21.15.45:0/753899521 conn(0x55c6ff350450 msgr2=0x55c6ff37f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 -- 172.21.15.45:0/753899521 shutdown_connections 2023-10-06T05:22:07.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.649+0000 7f90ca2e7040 1 -- 172.21.15.45:0/753899521 wait complete. 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.650+0000 7f90ca2e7040 1 Processor -- start 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.650+0000 7f90ca2e7040 1 -- start start 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90ca2e7040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff409420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90ca2e7040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff397c00 0x55c6ff408110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90ca2e7040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff40d970 0x55c6ff436ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90ca2e7040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ff394dd0 con 0x55c6ff397c00 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90ca2e7040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ff395070 con 0x55c6ff37fbe0 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90ca2e7040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6ff395310 con 0x55c6ff40d970 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff397c00 0x55c6ff408110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff397c00 0x55c6ff408110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54752/0 (socket says 172.21.15.45:54752) 2023-10-06T05:22:07.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 -- 172.21.15.45:0/335496382 learned_addr learned my addr 172.21.15.45:0/335496382 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c3505700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff409420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c3d06700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff40d970 0x55c6ff436ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 -- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff40d970 msgr2=0x55c6ff436ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c6ff40d970 0x55c6ff436ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 -- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 msgr2=0x55c6ff409420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff409420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 -- 172.21.15.45:0/335496382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc00d040 con 0x55c6ff397c00 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c3505700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6ff37fbe0 0x55c6ff409420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90c2d04700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff397c00 0x55c6ff408110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f90b8008410 tx=0x7f90b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.651+0000 7f90abfff700 1 -- 172.21.15.45:0/335496382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8005550 con 0x55c6ff397c00 2023-10-06T05:22:07.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.652+0000 7f90abfff700 1 -- 172.21.15.45:0/335496382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b8012d90 con 0x55c6ff397c00 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.652+0000 7f90abfff700 1 -- 172.21.15.45:0/335496382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b801d670 con 0x55c6ff397c00 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.652+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c6ff2a93d0 con 0x55c6ff397c00 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.652+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6ff29cae0 con 0x55c6ff397c00 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.653+0000 7f90abfff700 1 -- 172.21.15.45:0/335496382 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90b801da60 con 0x55c6ff397c00 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.654+0000 7f90abfff700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f90b0065890 0x7f90b0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.654+0000 7f90abfff700 1 -- 172.21.15.45:0/335496382 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f90b80855e0 con 0x55c6ff397c00 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.654+0000 7f90c3505700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f90b0065890 0x7f90b0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.654+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c6ff394850 0x55c6ff395cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.654+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c6ff396230 con 0x55c6ff394850 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.654+0000 7f90c3d06700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c6ff394850 0x55c6ff395cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.654+0000 7f90c3505700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f90b0065890 0x7f90b0067d40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f90b4002fd0 tx=0x7f90b4001780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90c3d06700 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c6ff394850 0x55c6ff395cf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90c3d06700 1 -- 172.21.15.45:0/335496382 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv17 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f90bc018020 con 0x55c6ff394850 2023-10-06T05:22:07.660 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c6ff394850 msgr2=0x55c6ff395cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c6ff394850 0x55c6ff395cf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f90b0065890 msgr2=0x7f90b0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f90b0065890 0x7f90b0067d40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f90b4002fd0 tx=0x7f90b4001780 comp rx=0 tx=0).stop 2023-10-06T05:22:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff397c00 msgr2=0x55c6ff408110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 --2- 172.21.15.45:0/335496382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6ff397c00 0x55c6ff408110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f90b8008410 tx=0x7f90b8016000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90c3d06700 1 -- 172.21.15.45:0/335496382 reap_dead start 2023-10-06T05:22:07.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 shutdown_connections 2023-10-06T05:22:07.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 >> 172.21.15.45:0/335496382 conn(0x55c6ff350450 msgr2=0x55c6ff426c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 shutdown_connections 2023-10-06T05:22:07.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.655+0000 7f90ca2e7040 1 -- 172.21.15.45:0/335496382 wait complete. 2023-10-06T05:22:07.664 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2023-10-06T05:22:07.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.749+0000 7f644a645040 1 Processor -- start 2023-10-06T05:22:07.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.749+0000 7f644a645040 1 -- start start 2023-10-06T05:22:07.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f644a645040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899ddb9b0 0x55c899e5af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f644a645040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e5fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f644a645040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899e69140 0x55c899e60240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f644a645040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c899cdb980 con 0x55c899e69140 2023-10-06T05:22:07.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f644a645040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c899cdb620 con 0x55c899e64b70 2023-10-06T05:22:07.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f644a645040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c899cd2400 con 0x55c899ddb9b0 2023-10-06T05:22:07.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6444064700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899e69140 0x55c899e60240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6444064700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899e69140 0x55c899e60240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54762/0 (socket says 172.21.15.45:54762) 2023-10-06T05:22:07.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6444064700 1 -- 172.21.15.45:0/2678577896 learned_addr learned my addr 172.21.15.45:0/2678577896 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6443863700 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899ddb9b0 0x55c899e5af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6443062700 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e5fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6444064700 1 -- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899ddb9b0 msgr2=0x55c899e5af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6444064700 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899ddb9b0 0x55c899e5af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6444064700 1 -- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 msgr2=0x55c899e5fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.750+0000 7f6444064700 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e5fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f6444064700 1 -- 172.21.15.45:0/2678577896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c00a0e0 con 0x55c899e69140 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f6443062700 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e5fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f6444064700 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899e69140 0x55c899e60240 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f643c00a860 tx=0x7f643c007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e05fa66fcfeacc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f6442060700 1 -- 172.21.15.45:0/2678577896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00d6f0 con 0x55c899e69140 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f6442060700 1 -- 172.21.15.45:0/2678577896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c00d850 con 0x55c899e69140 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f6442060700 1 -- 172.21.15.45:0/2678577896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00dbd0 con 0x55c899e69140 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f644a645040 1 -- 172.21.15.45:0/2678577896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899e69140 msgr2=0x55c899e60240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f644a645040 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899e69140 0x55c899e60240 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f643c00a860 tx=0x7f643c007040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f644a645040 1 -- 172.21.15.45:0/2678577896 shutdown_connections 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f644a645040 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899e69140 0x55c899e60240 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f644a645040 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899ddb9b0 0x55c899e5af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f644a645040 1 --2- 172.21.15.45:0/2678577896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e5fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.751+0000 7f644a645040 1 -- 172.21.15.45:0/2678577896 >> 172.21.15.45:0/2678577896 conn(0x55c899d925f0 msgr2=0x55c899dd88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.752+0000 7f644a645040 1 -- 172.21.15.45:0/2678577896 shutdown_connections 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.752+0000 7f644a645040 1 -- 172.21.15.45:0/2678577896 wait complete. 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.752+0000 7f644a645040 1 Processor -- start 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.752+0000 7f644a645040 1 -- start start 2023-10-06T05:22:07.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f644a645040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899ddb9b0 0x55c899e5d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f644a645040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e76c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f644a645040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899e69140 0x55c899e5bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f644a645040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c899d8fbb0 con 0x55c899ddb9b0 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f644a645040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c899dd5e10 con 0x55c899e64b70 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f644a645040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c899dd60b0 con 0x55c899e69140 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f6443863700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899ddb9b0 0x55c899e5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f6443863700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899ddb9b0 0x55c899e5d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54774/0 (socket says 172.21.15.45:54774) 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f6443863700 1 -- 172.21.15.45:0/248786105 learned_addr learned my addr 172.21.15.45:0/248786105 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f6443062700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e76c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.753+0000 7f6444064700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899e69140 0x55c899e5bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f6443863700 1 -- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899e69140 msgr2=0x55c899e5bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f6443863700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c899e69140 0x55c899e5bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f6443863700 1 -- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 msgr2=0x55c899e76c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f6443863700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c899e64b70 0x55c899e76c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f6443863700 1 -- 172.21.15.45:0/248786105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c00a510 con 0x55c899ddb9b0 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f6443863700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899ddb9b0 0x55c899e5d080 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6434011000 tx=0x7f6434015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f644085d700 1 -- 172.21.15.45:0/248786105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434007e80 con 0x55c899ddb9b0 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f644085d700 1 -- 172.21.15.45:0/248786105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6434005330 con 0x55c899ddb9b0 2023-10-06T05:22:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434005cf0 con 0x55c899ddb9b0 2023-10-06T05:22:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f644085d700 1 -- 172.21.15.45:0/248786105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340054e0 con 0x55c899ddb9b0 2023-10-06T05:22:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.754+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c899cea3d0 con 0x55c899ddb9b0 2023-10-06T05:22:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.756+0000 7f644085d700 1 -- 172.21.15.45:0/248786105 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6434019040 con 0x55c899ddb9b0 2023-10-06T05:22:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.756+0000 7f644085d700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64300658c0 0x7f6430067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.756+0000 7f644085d700 1 -- 172.21.15.45:0/248786105 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f6434084c20 con 0x55c899ddb9b0 2023-10-06T05:22:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.756+0000 7f6443062700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64300658c0 0x7f6430067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.756+0000 7f644a645040 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c899d90010 0x55c899d903e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.756+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c899d90920 con 0x55c899d90010 2023-10-06T05:22:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f6444064700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c899d90010 0x55c899d903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f6443062700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64300658c0 0x7f6430067d70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6438004170 tx=0x7f6438000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f6444064700 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c899d90010 0x55c899d903e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f6444064700 1 -- 172.21.15.45:0/248786105 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f643c019070 con 0x55c899d90010 2023-10-06T05:22:07.769 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:07.769 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c899d90010 msgr2=0x55c899d903e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f644a645040 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c899d90010 0x55c899d903e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64300658c0 msgr2=0x7f6430067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f644a645040 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64300658c0 0x7f6430067d70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6438004170 tx=0x7f6438000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899ddb9b0 msgr2=0x55c899e5d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f644a645040 1 --2- 172.21.15.45:0/248786105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c899ddb9b0 0x55c899e5d080 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6434011000 tx=0x7f6434015000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.757+0000 7f6444064700 1 -- 172.21.15.45:0/248786105 reap_dead start 2023-10-06T05:22:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.758+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 shutdown_connections 2023-10-06T05:22:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.758+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 >> 172.21.15.45:0/248786105 conn(0x55c899d925f0 msgr2=0x55c899dd88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.758+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 shutdown_connections 2023-10-06T05:22:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.758+0000 7f644a645040 1 -- 172.21.15.45:0/248786105 wait complete. 2023-10-06T05:22:07.773 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2023-10-06T05:22:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.819+0000 7f0233f72040 1 Processor -- start 2023-10-06T05:22:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.819+0000 7f0233f72040 1 -- start start 2023-10-06T05:22:07.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f0233f72040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218002300 0x55e8a09e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f0233f72040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a0a78310 0x55e8a09a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f0233f72040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a7aa20 0x55e8a0a77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f0233f72040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a0949320 con 0x7f0218002300 2023-10-06T05:22:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f0233f72040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a08ed980 con 0x55e8a0a78310 2023-10-06T05:22:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f0233f72040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8a08ed620 con 0x55e8a0a7aa20 2023-10-06T05:22:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218002300 0x55e8a09e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218002300 0x55e8a09e8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54796/0 (socket says 172.21.15.45:54796) 2023-10-06T05:22:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 -- 172.21.15.45:0/2673371580 learned_addr learned my addr 172.21.15.45:0/2673371580 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d991700 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a7aa20 0x55e8a0a77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022c98f700 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a0a78310 0x55e8a09a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 -- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a7aa20 msgr2=0x55e8a0a77630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a7aa20 0x55e8a0a77630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 -- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a0a78310 msgr2=0x55e8a09a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a0a78310 0x55e8a09a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d190700 1 -- 172.21.15.45:0/2673371580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8a08efae0 con 0x7f0218002300 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022d991700 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a7aa20 0x55e8a0a77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.820+0000 7f022c98f700 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a0a78310 0x55e8a09a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f022d190700 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218002300 0x55e8a09e8d10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f022000b5d0 tx=0x7f0220017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb17772996cd2d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f021f7fe700 1 -- 172.21.15.45:0/2673371580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022000b7e0 con 0x7f0218002300 2023-10-06T05:22:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f021f7fe700 1 -- 172.21.15.45:0/2673371580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0220000ed0 con 0x7f0218002300 2023-10-06T05:22:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f021f7fe700 1 -- 172.21.15.45:0/2673371580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022000bad0 con 0x7f0218002300 2023-10-06T05:22:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 -- 172.21.15.45:0/2673371580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218002300 msgr2=0x55e8a09e8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218002300 0x55e8a09e8d10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f022000b5d0 tx=0x7f0220017000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 -- 172.21.15.45:0/2673371580 shutdown_connections 2023-10-06T05:22:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a7aa20 0x55e8a0a77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218002300 0x55e8a09e8d10 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 --2- 172.21.15.45:0/2673371580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a0a78310 0x55e8a09a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 -- 172.21.15.45:0/2673371580 >> 172.21.15.45:0/2673371580 conn(0x55e8a09a34a0 msgr2=0x55e8a09a3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 -- 172.21.15.45:0/2673371580 shutdown_connections 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.821+0000 7f0233f72040 1 -- 172.21.15.45:0/2673371580 wait complete. 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.822+0000 7f0233f72040 1 Processor -- start 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.822+0000 7f0233f72040 1 -- start start 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f0233f72040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a78310 0x55e8a09d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f0233f72040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8a0a7aa20 0x55e8a0a59cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f0233f72040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0218002300 0x55e8a0a79660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f0233f72040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a08e4400 con 0x55e8a0a7aa20 2023-10-06T05:22:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f0233f72040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a09d2350 con 0x7f0218002300 2023-10-06T05:22:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f0233f72040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8a09d25f0 con 0x55e8a0a78310 2023-10-06T05:22:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8a0a7aa20 0x55e8a0a59cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8a0a7aa20 0x55e8a0a59cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54806/0 (socket says 172.21.15.45:54806) 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 -- 172.21.15.45:0/561131860 learned_addr learned my addr 172.21.15.45:0/561131860 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022d190700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a78310 0x55e8a09d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022d991700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0218002300 0x55e8a0a79660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 -- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a78310 msgr2=0x55e8a09d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a0a78310 0x55e8a09d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 -- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0218002300 msgr2=0x55e8a0a79660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0218002300 0x55e8a0a79660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022c98f700 1 -- 172.21.15.45:0/561131860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218001e10 con 0x55e8a0a7aa20 2023-10-06T05:22:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.823+0000 7f022d991700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0218002300 0x55e8a0a79660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.824+0000 7f022c98f700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8a0a7aa20 0x55e8a0a59cd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0224005cc0 tx=0x7f022400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.824+0000 7f021dffb700 1 -- 172.21.15.45:0/561131860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022400f070 con 0x55e8a0a7aa20 2023-10-06T05:22:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.824+0000 7f021dffb700 1 -- 172.21.15.45:0/561131860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224010040 con 0x55e8a0a7aa20 2023-10-06T05:22:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.824+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e8a08ef540 con 0x55e8a0a7aa20 2023-10-06T05:22:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.824+0000 7f021dffb700 1 -- 172.21.15.45:0/561131860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224006cf0 con 0x55e8a0a7aa20 2023-10-06T05:22:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.824+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8a09d2b30 con 0x55e8a0a7aa20 2023-10-06T05:22:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.825+0000 7f021dffb700 1 -- 172.21.15.45:0/561131860 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0224016690 con 0x55e8a0a7aa20 2023-10-06T05:22:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.826+0000 7f021dffb700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0210065890 0x7f0210067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.826+0000 7f021dffb700 1 -- 172.21.15.45:0/561131860 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0224084200 con 0x55e8a0a7aa20 2023-10-06T05:22:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.826+0000 7f0233f72040 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e8a0a5d220 0x55e8a0a5d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.826+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55e8a0a5db30 con 0x55e8a0a5d220 2023-10-06T05:22:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.826+0000 7f022d190700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0210065890 0x7f0210067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.826+0000 7f022d991700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e8a0a5d220 0x55e8a0a5d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f022d190700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0210065890 0x7f0210067d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f02200016a0 tx=0x7f0220020000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f022d991700 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e8a0a5d220 0x55e8a0a5d5f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.838 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:07.839 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f022d991700 1 -- 172.21.15.45:0/561131860 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f02280112c0 con 0x55e8a0a5d220 2023-10-06T05:22:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e8a0a5d220 msgr2=0x55e8a0a5d5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e8a0a5d220 0x55e8a0a5d5f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0210065890 msgr2=0x7f0210067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0210065890 0x7f0210067d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f02200016a0 tx=0x7f0220020000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8a0a7aa20 msgr2=0x55e8a0a59cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 --2- 172.21.15.45:0/561131860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8a0a7aa20 0x55e8a0a59cd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0224005cc0 tx=0x7f022400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f022d991700 1 -- 172.21.15.45:0/561131860 reap_dead start 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 shutdown_connections 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 >> 172.21.15.45:0/561131860 conn(0x55e8a09a34a0 msgr2=0x55e8a09a3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 shutdown_connections 2023-10-06T05:22:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.827+0000 7f0233f72040 1 -- 172.21.15.45:0/561131860 wait complete. 2023-10-06T05:22:07.842 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2023-10-06T05:22:07.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.888+0000 7f383e7d4040 1 Processor -- start 2023-10-06T05:22:07.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.888+0000 7f383e7d4040 1 -- start start 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.889+0000 7f383e7d4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38240030b0 0x558f161965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.889+0000 7f383e7d4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1626f1c0 0x558f161c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.889+0000 7f383e7d4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626ec80 0x558f16197290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.889+0000 7f383e7d4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1613a320 con 0x558f1626ec80 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.889+0000 7f383e7d4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f160de980 con 0x7f38240030b0 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.889+0000 7f383e7d4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f160de620 con 0x558f1626f1c0 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.889+0000 7f38381f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626ec80 0x558f16197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626ec80 0x558f16197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54832/0 (socket says 172.21.15.45:54832) 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 -- 172.21.15.45:0/2924598304 learned_addr learned my addr 172.21.15.45:0/2924598304 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38379f2700 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38240030b0 0x558f161965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38371f1700 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1626f1c0 0x558f161c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 -- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1626f1c0 msgr2=0x558f161c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1626f1c0 0x558f161c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 -- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38240030b0 msgr2=0x558f161965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38240030b0 0x558f161965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 -- 172.21.15.45:0/2924598304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f160e0ae0 con 0x558f1626ec80 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38381f3700 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626ec80 0x558f16197290 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3830005b20 tx=0x7f3830007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302cea24e8cd5269 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38361ef700 1 -- 172.21.15.45:0/2924598304 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830012680 con 0x558f1626ec80 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38361ef700 1 -- 172.21.15.45:0/2924598304 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300127e0 con 0x558f1626ec80 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.890+0000 7f38361ef700 1 -- 172.21.15.45:0/2924598304 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830012ba0 con 0x558f1626ec80 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 -- 172.21.15.45:0/2924598304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626ec80 msgr2=0x558f16197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626ec80 0x558f16197290 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3830005b20 tx=0x7f3830007040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 -- 172.21.15.45:0/2924598304 shutdown_connections 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626ec80 0x558f16197290 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38240030b0 0x558f161965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 --2- 172.21.15.45:0/2924598304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1626f1c0 0x558f161c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 -- 172.21.15.45:0/2924598304 >> 172.21.15.45:0/2924598304 conn(0x558f161955f0 msgr2=0x558f16195cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 -- 172.21.15.45:0/2924598304 shutdown_connections 2023-10-06T05:22:07.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.891+0000 7f383e7d4040 1 -- 172.21.15.45:0/2924598304 wait complete. 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.892+0000 7f383e7d4040 1 Processor -- start 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.892+0000 7f383e7d4040 1 -- start start 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f383e7d4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1626ec80 0x558f161de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f383e7d4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626f1c0 0x558f16269830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f383e7d4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38240030b0 0x558f161daff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f383e7d4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f160d5400 con 0x558f1626f1c0 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f383e7d4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f16269d70 con 0x558f1626ec80 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f383e7d4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f161dd8a0 con 0x7f38240030b0 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626f1c0 0x558f16269830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626f1c0 0x558f16269830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54844/0 (socket says 172.21.15.45:54844) 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 -- 172.21.15.45:0/1016423747 learned_addr learned my addr 172.21.15.45:0/1016423747 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38381f3700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38240030b0 0x558f161daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38379f2700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1626ec80 0x558f161de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 -- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38240030b0 msgr2=0x558f161daff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38240030b0 0x558f161daff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 -- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1626ec80 msgr2=0x558f161de7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1626ec80 0x558f161de7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 -- 172.21.15.45:0/1016423747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383000a460 con 0x558f1626f1c0 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.893+0000 7f38371f1700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626f1c0 0x558f16269830 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f382800e7a0 tx=0x7f3828017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.894+0000 7f38349ec700 1 -- 172.21.15.45:0/1016423747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800b7e0 con 0x558f1626f1c0 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.894+0000 7f38349ec700 1 -- 172.21.15.45:0/1016423747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3828009540 con 0x558f1626f1c0 2023-10-06T05:22:07.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.894+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f160e0540 con 0x558f1626f1c0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.894+0000 7f38349ec700 1 -- 172.21.15.45:0/1016423747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800ede0 con 0x558f1626f1c0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.894+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f161ddde0 con 0x558f1626f1c0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.895+0000 7f38349ec700 1 -- 172.21.15.45:0/1016423747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3828017660 con 0x558f1626f1c0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.896+0000 7f38349ec700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3818065890 0x7f3818067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.896+0000 7f38349ec700 1 -- 172.21.15.45:0/1016423747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f3828085080 con 0x558f1626f1c0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.896+0000 7f383e7d4040 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f16252930 0x558f1627b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.896+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.6 14:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x558f1627b540 con 0x558f16252930 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.896+0000 7f38379f2700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3818065890 0x7f3818067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.896+0000 7f38381f3700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f16252930 0x558f1627b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.896+0000 7f38381f3700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f16252930 0x558f1627b000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f38379f2700 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3818065890 0x7f3818067d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3820000c00 tx=0x7f3820001780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f38381f3700 1 -- 172.21.15.45:0/1016423747 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+42 (crc 0 0 0) 0x7f383001b020 con 0x558f16252930 2023-10-06T05:22:07.902 INFO:teuthology.orchestra.run.smithi045.stdout:header (42 bytes) : 2023-10-06T05:22:07.902 INFO:teuthology.orchestra.run.smithi045.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2023-10-06T05:22:07.902 INFO:teuthology.orchestra.run.smithi045.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 02 00 00 |me v011.........| 2023-10-06T05:22:07.902 INFO:teuthology.orchestra.run.smithi045.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2023-10-06T05:22:07.902 INFO:teuthology.orchestra.run.smithi045.stdout:0000002a 2023-10-06T05:22:07.902 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f16252930 msgr2=0x558f1627b000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f16252930 0x558f1627b000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3818065890 msgr2=0x7f3818067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3818065890 0x7f3818067d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3820000c00 tx=0x7f3820001780 comp rx=0 tx=0).stop 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626f1c0 msgr2=0x558f16269830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 --2- 172.21.15.45:0/1016423747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1626f1c0 0x558f16269830 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f382800e7a0 tx=0x7f3828017000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f38381f3700 1 -- 172.21.15.45:0/1016423747 reap_dead start 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 shutdown_connections 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 >> 172.21.15.45:0/1016423747 conn(0x558f161955f0 msgr2=0x558f16196570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 shutdown_connections 2023-10-06T05:22:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.897+0000 7f383e7d4040 1 -- 172.21.15.45:0/1016423747 wait complete. 2023-10-06T05:22:07.906 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2023-10-06T05:22:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.955+0000 7febbdb63040 1 Processor -- start 2023-10-06T05:22:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.955+0000 7febbdb63040 1 -- start start 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febbdb63040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40bf9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febbdb63040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 0x557e40c6d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febbdb63040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40c6d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febbdb63040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e40afe980 con 0x557e40bb1fe0 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febbdb63040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e40afe620 con 0x557e40bf75f0 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febbdb63040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e40af5400 con 0x557e40c73970 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febb6d81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40bf9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febb6d81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40bf9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54852/0 (socket says 172.21.15.45:54852) 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febb6d81700 1 -- 172.21.15.45:0/3845347848 learned_addr learned my addr 172.21.15.45:0/3845347848 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febb6580700 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 0x557e40c6d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.956+0000 7febb7582700 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40c6d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb6d81700 1 -- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 msgr2=0x557e40c6d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb6d81700 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40c6d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb6d81700 1 -- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 msgr2=0x557e40c6d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb6d81700 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 0x557e40c6d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb6d81700 1 -- 172.21.15.45:0/3845347848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8005db0 con 0x557e40bb1fe0 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb6d81700 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40bf9cf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7feba0009830 tx=0x7feba000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a24478f5f26e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb557e700 1 -- 172.21.15.45:0/3845347848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba000f040 con 0x557e40bb1fe0 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb557e700 1 -- 172.21.15.45:0/3845347848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0008790 con 0x557e40bb1fe0 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febb557e700 1 -- 172.21.15.45:0/3845347848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0008b50 con 0x557e40bb1fe0 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.957+0000 7febbdb63040 1 -- 172.21.15.45:0/3845347848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 msgr2=0x557e40bf9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40bf9cf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7feba0009830 tx=0x7feba000c040 comp rx=0 tx=0).stop 2023-10-06T05:22:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 -- 172.21.15.45:0/3845347848 shutdown_connections 2023-10-06T05:22:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40c6d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40bf9cf0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 --2- 172.21.15.45:0/3845347848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 0x557e40c6d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 -- 172.21.15.45:0/3845347848 >> 172.21.15.45:0/3845347848 conn(0x557e40bb44a0 msgr2=0x557e40bdfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 -- 172.21.15.45:0/3845347848 shutdown_connections 2023-10-06T05:22:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.958+0000 7febbdb63040 1 -- 172.21.15.45:0/3845347848 wait complete. 2023-10-06T05:22:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.959+0000 7febbdb63040 1 Processor -- start 2023-10-06T05:22:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.959+0000 7febbdb63040 1 -- start start 2023-10-06T05:22:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febbdb63040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40c8c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febbdb63040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 0x557e40bfbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febbdb63040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40bfc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febbdb63040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e40c8cab0 con 0x557e40bb1fe0 2023-10-06T05:22:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febbdb63040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e40bfa4b0 con 0x557e40bf75f0 2023-10-06T05:22:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febbdb63040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e40bde650 con 0x557e40c73970 2023-10-06T05:22:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40c8c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40c8c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54858/0 (socket says 172.21.15.45:54858) 2023-10-06T05:22:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 -- 172.21.15.45:0/705145833 learned_addr learned my addr 172.21.15.45:0/705145833 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6580700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 0x557e40bfbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb7582700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40bfc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 -- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 msgr2=0x557e40bfc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40bfc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 -- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 msgr2=0x557e40bfbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e40bf75f0 0x557e40bfbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 -- 172.21.15.45:0/705145833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0009430 con 0x557e40bb1fe0 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb7582700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e40c73970 0x557e40bfc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.960+0000 7febb6d81700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40c8c570 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7feba0009e20 tx=0x7feba0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.961+0000 7febaf7fe700 1 -- 172.21.15.45:0/705145833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba000f030 con 0x557e40bb1fe0 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.961+0000 7febaf7fe700 1 -- 172.21.15.45:0/705145833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0013730 con 0x557e40bb1fe0 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.961+0000 7febaf7fe700 1 -- 172.21.15.45:0/705145833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0013ab0 con 0x557e40bb1fe0 2023-10-06T05:22:07.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.961+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557e40b0d3d0 con 0x557e40bb1fe0 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.961+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e40b00ae0 con 0x557e40bb1fe0 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.962+0000 7febaf7fe700 1 -- 172.21.15.45:0/705145833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feba001c420 con 0x557e40bb1fe0 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.963+0000 7febaf7fe700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feb98065890 0x7feb98067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.963+0000 7febaf7fe700 1 -- 172.21.15.45:0/705145833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7feba0085220 con 0x557e40bb1fe0 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.963+0000 7febb6580700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feb98065890 0x7feb98067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.963+0000 7febbdb63040 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557e40bd5770 0x557e40be35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.963+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.15 14:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x557e40be3b20 con 0x557e40bd5770 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.963+0000 7febb7582700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557e40bd5770 0x557e40be35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.963+0000 7febb6580700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feb98065890 0x7feb98067d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feba8008fd0 tx=0x7feba8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febb7582700 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557e40bd5770 0x557e40be35e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.973 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:07.973 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febb7582700 1 -- 172.21.15.45:0/705145833 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7febb0002840 con 0x557e40bd5770 2023-10-06T05:22:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557e40bd5770 msgr2=0x557e40be35e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557e40bd5770 0x557e40be35e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feb98065890 msgr2=0x7feb98067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feb98065890 0x7feb98067d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feba8008fd0 tx=0x7feba8016000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 msgr2=0x557e40c8c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 --2- 172.21.15.45:0/705145833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e40bb1fe0 0x557e40c8c570 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7feba0009e20 tx=0x7feba0018000 comp rx=0 tx=0).stop 2023-10-06T05:22:07.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febb7582700 1 -- 172.21.15.45:0/705145833 reap_dead start 2023-10-06T05:22:07.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 shutdown_connections 2023-10-06T05:22:07.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 >> 172.21.15.45:0/705145833 conn(0x557e40bb44a0 msgr2=0x557e40bdfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 shutdown_connections 2023-10-06T05:22:07.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:07.964+0000 7febbdb63040 1 -- 172.21.15.45:0/705145833 wait complete. 2023-10-06T05:22:07.975 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2023-10-06T05:22:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.019+0000 7f4234ab8040 1 Processor -- start 2023-10-06T05:22:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.019+0000 7f4234ab8040 1 -- start start 2023-10-06T05:22:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f4234ab8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdb327e0 0x555dfdb19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f4234ab8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdb30 0x555dfdbb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f4234ab8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdbbdf00 0x555dfdae8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f4234ab8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfda35980 con 0x555dfdb327e0 2023-10-06T05:22:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f4234ab8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfda35620 con 0x555dfdbbdf00 2023-10-06T05:22:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f4234ab8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dfda2c400 con 0x555dfdbbdb30 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f422dcd6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdb327e0 0x555dfdb19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f422dcd6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdb327e0 0x555dfdb19760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54872/0 (socket says 172.21.15.45:54872) 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f422dcd6700 1 -- 172.21.15.45:0/479839417 learned_addr learned my addr 172.21.15.45:0/479839417 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f422e4d7700 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdbbdf00 0x555dfdae8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f422d4d5700 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdb30 0x555dfdbb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f422dcd6700 1 -- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdb30 msgr2=0x555dfdbb97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.020+0000 7f422dcd6700 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdb30 0x555dfdbb97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f422dcd6700 1 -- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdbbdf00 msgr2=0x555dfdae8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f422dcd6700 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdbbdf00 0x555dfdae8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f422dcd6700 1 -- 172.21.15.45:0/479839417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4220005db0 con 0x555dfdb327e0 2023-10-06T05:22:08.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f422dcd6700 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdb327e0 0x555dfdb19760 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f422000ad70 tx=0x7f4220016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c7575c1755bacc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f421ffff700 1 -- 172.21.15.45:0/479839417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220000bc0 con 0x555dfdb327e0 2023-10-06T05:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f421ffff700 1 -- 172.21.15.45:0/479839417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42200089d0 con 0x555dfdb327e0 2023-10-06T05:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f421ffff700 1 -- 172.21.15.45:0/479839417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422001c440 con 0x555dfdb327e0 2023-10-06T05:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 -- 172.21.15.45:0/479839417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdb327e0 msgr2=0x555dfdb19760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdb327e0 0x555dfdb19760 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f422000ad70 tx=0x7f4220016000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 -- 172.21.15.45:0/479839417 shutdown_connections 2023-10-06T05:22:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdbbdf00 0x555dfdae8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdb327e0 0x555dfdb19760 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 --2- 172.21.15.45:0/479839417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdb30 0x555dfdbb97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 -- 172.21.15.45:0/479839417 >> 172.21.15.45:0/479839417 conn(0x555dfdaeb450 msgr2=0x555dfdae8210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.021+0000 7f4234ab8040 1 -- 172.21.15.45:0/479839417 shutdown_connections 2023-10-06T05:22:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.022+0000 7f4234ab8040 1 -- 172.21.15.45:0/479839417 wait complete. 2023-10-06T05:22:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.022+0000 7f4234ab8040 1 Processor -- start 2023-10-06T05:22:08.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.022+0000 7f4234ab8040 1 -- start start 2023-10-06T05:22:08.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f4234ab8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdb327e0 0x555dfdb2e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f4234ab8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdbbdb30 0x555dfdae8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f4234ab8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdf00 0x555dfdb19760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f4234ab8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfdbd1ad0 con 0x555dfdbbdb30 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f4234ab8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfdbb8620 con 0x555dfdb327e0 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f4234ab8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dfdbb88c0 con 0x555dfdbbdf00 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdbbdb30 0x555dfdae8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdbbdb30 0x555dfdae8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54878/0 (socket says 172.21.15.45:54878) 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 -- 172.21.15.45:0/2088435077 learned_addr learned my addr 172.21.15.45:0/2088435077 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422e4d7700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdf00 0x555dfdb19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422dcd6700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdb327e0 0x555dfdb2e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 -- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdf00 msgr2=0x555dfdb19760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555dfdbbdf00 0x555dfdb19760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 -- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdb327e0 msgr2=0x555dfdb2e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555dfdb327e0 0x555dfdb2e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.023+0000 7f422d4d5700 1 -- 172.21.15.45:0/2088435077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dfda5e1c0 con 0x555dfdbbdb30 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.024+0000 7f422d4d5700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdbbdb30 0x555dfdae8720 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4224002f40 tx=0x7f422400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.024+0000 7f421e7fc700 1 -- 172.21.15.45:0/2088435077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400f050 con 0x555dfdbbdb30 2023-10-06T05:22:08.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.024+0000 7f421e7fc700 1 -- 172.21.15.45:0/2088435077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4224010040 con 0x555dfdbbdb30 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.024+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555dfda443d0 con 0x555dfdbbdb30 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.024+0000 7f421e7fc700 1 -- 172.21.15.45:0/2088435077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224006cd0 con 0x555dfdbbdb30 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.024+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dfda37ae0 con 0x555dfdbbdb30 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.025+0000 7f421e7fc700 1 -- 172.21.15.45:0/2088435077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4224015670 con 0x555dfdbbdb30 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.026+0000 7f421e7fc700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4210065890 0x7f4210067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.026+0000 7f421e7fc700 1 -- 172.21.15.45:0/2088435077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f4224083d70 con 0x555dfdbbdb30 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.026+0000 7f4234ab8040 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x555dfdbabc70 0x555dfdbb7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.026+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x555dfdbb7db0 con 0x555dfdbabc70 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.026+0000 7f422dcd6700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4210065890 0x7f4210067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.026+0000 7f422e4d7700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x555dfdbabc70 0x555dfdbb7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f422dcd6700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4210065890 0x7f4210067d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4220005d30 tx=0x7f422001c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f422e4d7700 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x555dfdbabc70 0x555dfdbb7870 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f422e4d7700 1 -- 172.21.15.45:0/2088435077 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f42280031d0 con 0x555dfdbabc70 2023-10-06T05:22:08.036 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:08.036 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x555dfdbabc70 msgr2=0x555dfdbb7870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x555dfdbabc70 0x555dfdbb7870 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4210065890 msgr2=0x7f4210067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4210065890 0x7f4210067d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4220005d30 tx=0x7f422001c000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdbbdb30 msgr2=0x555dfdae8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 --2- 172.21.15.45:0/2088435077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555dfdbbdb30 0x555dfdae8720 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4224002f40 tx=0x7f422400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f422e4d7700 1 -- 172.21.15.45:0/2088435077 reap_dead start 2023-10-06T05:22:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 shutdown_connections 2023-10-06T05:22:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 >> 172.21.15.45:0/2088435077 conn(0x555dfdaeb450 msgr2=0x555dfdbb9640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 shutdown_connections 2023-10-06T05:22:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.027+0000 7f4234ab8040 1 -- 172.21.15.45:0/2088435077 wait complete. 2023-10-06T05:22:08.039 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2023-10-06T05:22:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.083+0000 7f6a333a6040 1 Processor -- start 2023-10-06T05:22:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.083+0000 7f6a333a6040 1 -- start start 2023-10-06T05:22:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a333a6040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433ebfded0 0x56433ec02ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a333a6040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433eca59f0 0x56433ebfe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a333a6040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a333a6040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56433eb1d980 con 0x56433ecaa010 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a333a6040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56433eb1d620 con 0x56433eca59f0 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a333a6040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56433eb14400 con 0x56433ebfded0 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebd3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54892/0 (socket says 172.21.15.45:54892) 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/2751272618 learned_addr learned my addr 172.21.15.45:0/2751272618 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2c5c4700 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433ebfded0 0x56433ec02ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2bdc3700 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433eca59f0 0x56433ebfe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433ebfded0 msgr2=0x56433ec02ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433ebfded0 0x56433ec02ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433eca59f0 msgr2=0x56433ebfe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433eca59f0 0x56433ebfe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/2751272618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2400c370 con 0x56433ecaa010 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2cdc5700 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebd3870 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c700 tx=0x7f6a24019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80856db16db7632 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.084+0000 7f6a2adc1700 1 -- 172.21.15.45:0/2751272618 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400ea50 con 0x56433ecaa010 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a2adc1700 1 -- 172.21.15.45:0/2751272618 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2400ebb0 con 0x56433ecaa010 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a2adc1700 1 -- 172.21.15.45:0/2751272618 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400f6a0 con 0x56433ecaa010 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 -- 172.21.15.45:0/2751272618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 msgr2=0x56433ebd3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebd3870 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c700 tx=0x7f6a24019000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 -- 172.21.15.45:0/2751272618 shutdown_connections 2023-10-06T05:22:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebd3870 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433ebfded0 0x56433ec02ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 --2- 172.21.15.45:0/2751272618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433eca59f0 0x56433ebfe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 -- 172.21.15.45:0/2751272618 >> 172.21.15.45:0/2751272618 conn(0x56433ebd34a0 msgr2=0x56433ebd0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 -- 172.21.15.45:0/2751272618 shutdown_connections 2023-10-06T05:22:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.085+0000 7f6a333a6040 1 -- 172.21.15.45:0/2751272618 wait complete. 2023-10-06T05:22:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.086+0000 7f6a333a6040 1 Processor -- start 2023-10-06T05:22:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.086+0000 7f6a333a6040 1 -- start start 2023-10-06T05:22:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a333a6040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433ebfded0 0x56433eca1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a333a6040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433eca59f0 0x56433eca3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a333a6040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebfef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a333a6040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56433ebfdae0 con 0x56433ecaa010 2023-10-06T05:22:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a333a6040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56433eca8ed0 con 0x56433ebfded0 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a333a6040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56433eca9170 con 0x56433eca59f0 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebfef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebfef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54906/0 (socket says 172.21.15.45:54906) 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/1939174967 learned_addr learned my addr 172.21.15.45:0/1939174967 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2c5c4700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433ebfded0 0x56433eca1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2bdc3700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433eca59f0 0x56433eca3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433eca59f0 msgr2=0x56433eca3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56433eca59f0 0x56433eca3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433ebfded0 msgr2=0x56433eca1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56433ebfded0 0x56433eca1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/1939174967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56433eb461c0 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a2cdc5700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebfef00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c700 tx=0x7f6a2401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.087+0000 7f6a295be700 1 -- 172.21.15.45:0/1939174967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24026070 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.088+0000 7f6a295be700 1 -- 172.21.15.45:0/1939174967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a240032f0 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.088+0000 7f6a295be700 1 -- 172.21.15.45:0/1939174967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400ea50 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.088+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56433eb2c3d0 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.088+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56433eb1fae0 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.089+0000 7f6a295be700 1 -- 172.21.15.45:0/1939174967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a24003040 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.089+0000 7f6a295be700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a0c065890 0x7f6a0c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.089+0000 7f6a295be700 1 -- 172.21.15.45:0/1939174967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f6a24085910 con 0x56433ecaa010 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a2c5c4700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a0c065890 0x7f6a0c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a333a6040 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56433ebff440 0x56433ebd3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x56433eca2b20 con 0x56433ebff440 2023-10-06T05:22:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a2bdc3700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56433ebff440 0x56433ebd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a2c5c4700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a0c065890 0x7f6a0c067d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005fd0 tx=0x7f6a1c00c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a2bdc3700 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56433ebff440 0x56433ebd3870 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a2bdc3700 1 -- 172.21.15.45:0/1939174967 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6a1400f8c0 con 0x56433ebff440 2023-10-06T05:22:08.096 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:08.096 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56433ebff440 msgr2=0x56433ebd3870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.090+0000 7f6a333a6040 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56433ebff440 0x56433ebd3870 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a0c065890 msgr2=0x7f6a0c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a0c065890 0x7f6a0c067d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005fd0 tx=0x7f6a1c00c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 msgr2=0x56433ebfef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 --2- 172.21.15.45:0/1939174967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56433ecaa010 0x56433ebfef00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c700 tx=0x7f6a2401d000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a2cdc5700 1 -- 172.21.15.45:0/1939174967 reap_dead start 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 shutdown_connections 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 >> 172.21.15.45:0/1939174967 conn(0x56433ebd34a0 msgr2=0x56433ebd4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 shutdown_connections 2023-10-06T05:22:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.091+0000 7f6a333a6040 1 -- 172.21.15.45:0/1939174967 wait complete. 2023-10-06T05:22:08.099 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2023-10-06T05:22:08.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.144+0000 7f877640b040 1 Processor -- start 2023-10-06T05:22:08.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.144+0000 7f877640b040 1 -- start start 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.145+0000 7f877640b040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 0x55fb6b4b1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.145+0000 7f877640b040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b4b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.145+0000 7f877640b040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 0x55fb6b4ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.145+0000 7f877640b040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6b325620 con 0x55fb6b40aab0 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.145+0000 7f877640b040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6b31c400 con 0x55fb6b4aeaf0 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.145+0000 7f877640b040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb6b41af00 con 0x55fb6b406dd0 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.145+0000 7f8766e28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b4b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f876f629700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 0x55fb6b4b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b4b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54922/0 (socket says 172.21.15.45:54922) 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 -- 172.21.15.45:0/3419895807 learned_addr learned my addr 172.21.15.45:0/3419895807 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f876fe2a700 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 0x55fb6b4ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 -- 172.21.15.45:0/3419895807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 msgr2=0x55fb6b4b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 0x55fb6b4b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 -- 172.21.15.45:0/3419895807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 msgr2=0x55fb6b4ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 0x55fb6b4ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 -- 172.21.15.45:0/3419895807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8760005db0 con 0x55fb6b40aab0 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f8766e28700 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b4b1a60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8758009950 tx=0x7f875800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6bf4f7359451c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f876e627700 1 -- 172.21.15.45:0/3419895807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800f040 con 0x55fb6b40aab0 2023-10-06T05:22:08.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f876e627700 1 -- 172.21.15.45:0/3419895807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758008800 con 0x55fb6b40aab0 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.146+0000 7f876e627700 1 -- 172.21.15.45:0/3419895807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758008ba0 con 0x55fb6b40aab0 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 -- 172.21.15.45:0/3419895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 msgr2=0x55fb6b4b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b4b1a60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8758009950 tx=0x7f875800c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 -- 172.21.15.45:0/3419895807 shutdown_connections 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 0x55fb6b4ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 0x55fb6b4b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 --2- 172.21.15.45:0/3419895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b4b1a60 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 -- 172.21.15.45:0/3419895807 >> 172.21.15.45:0/3419895807 conn(0x55fb6b3db4a0 msgr2=0x55fb6b4adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 -- 172.21.15.45:0/3419895807 shutdown_connections 2023-10-06T05:22:08.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.147+0000 7f877640b040 1 -- 172.21.15.45:0/3419895807 wait complete. 2023-10-06T05:22:08.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.148+0000 7f877640b040 1 Processor -- start 2023-10-06T05:22:08.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.148+0000 7f877640b040 1 -- start start 2023-10-06T05:22:08.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f877640b040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 0x55fb6b4a8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f877640b040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b422440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f877640b040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 0x55fb6b422980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f877640b040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6b4abad0 con 0x55fb6b40aab0 2023-10-06T05:22:08.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f877640b040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6b4abd70 con 0x55fb6b4aeaf0 2023-10-06T05:22:08.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f877640b040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb6b4ac010 con 0x55fb6b406dd0 2023-10-06T05:22:08.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b422440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b422440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54936/0 (socket says 172.21.15.45:54936) 2023-10-06T05:22:08.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 -- 172.21.15.45:0/1787653261 learned_addr learned my addr 172.21.15.45:0/1787653261 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f876fe2a700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 0x55fb6b422980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f876f629700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 0x55fb6b4a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 -- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 msgr2=0x55fb6b4a8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fb6b406dd0 0x55fb6b4a8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 -- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 msgr2=0x55fb6b422980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fb6b4aeaf0 0x55fb6b422980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.149+0000 7f8766e28700 1 -- 172.21.15.45:0/1787653261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009580 con 0x55fb6b40aab0 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.150+0000 7f8766e28700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b422440 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8758009f40 tx=0x7f8758018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.150+0000 7f876ce24700 1 -- 172.21.15.45:0/1787653261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800f040 con 0x55fb6b40aab0 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.150+0000 7f876ce24700 1 -- 172.21.15.45:0/1787653261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758013800 con 0x55fb6b40aab0 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.150+0000 7f876ce24700 1 -- 172.21.15.45:0/1787653261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758013990 con 0x55fb6b40aab0 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.150+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb6b34e1c0 con 0x55fb6b40aab0 2023-10-06T05:22:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.150+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb6b3343d0 con 0x55fb6b40aab0 2023-10-06T05:22:08.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.151+0000 7f876ce24700 1 -- 172.21.15.45:0/1787653261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8758023020 con 0x55fb6b40aab0 2023-10-06T05:22:08.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.152+0000 7f876ce24700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8750065890 0x7f8750067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.152+0000 7f876ce24700 1 -- 172.21.15.45:0/1787653261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8758084ac0 con 0x55fb6b40aab0 2023-10-06T05:22:08.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.152+0000 7f876f629700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8750065890 0x7f8750067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.152+0000 7f877640b040 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fb6b4ac750 0x55fb6b41e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.152+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55fb6b41eea0 con 0x55fb6b4ac750 2023-10-06T05:22:08.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.152+0000 7f876fe2a700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fb6b4ac750 0x55fb6b41e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.152+0000 7f876f629700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8750065890 0x7f8750067d40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f876000a650 tx=0x7f8760016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f876fe2a700 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fb6b4ac750 0x55fb6b41e960 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f876fe2a700 1 -- 172.21.15.45:0/1787653261 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+274 (crc 0 0 0) 0x7f8768002d90 con 0x55fb6b4ac750 2023-10-06T05:22:08.162 INFO:teuthology.orchestra.run.smithi045.stdout:header (274 bytes) : 2023-10-06T05:22:08.162 INFO:teuthology.orchestra.run.smithi045.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.163 INFO:teuthology.orchestra.run.smithi045.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-10-06T05:22:08.163 INFO:teuthology.orchestra.run.smithi045.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.163 INFO:teuthology.orchestra.run.smithi045.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.163 INFO:teuthology.orchestra.run.smithi045.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-10-06T05:22:08.164 INFO:teuthology.orchestra.run.smithi045.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:000000a0 00 00 00 00 00 00 00 00 6f 99 1f 65 da 9a d6 13 |........o..e....| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2023-10-06T05:22:08.165 INFO:teuthology.orchestra.run.smithi045.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.167 INFO:teuthology.orchestra.run.smithi045.stdout:000000e0 00 00 00 00 00 00 6f 99 1f 65 da 9a d6 13 00 00 |......o..e......| 2023-10-06T05:22:08.167 INFO:teuthology.orchestra.run.smithi045.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.167 INFO:teuthology.orchestra.run.smithi045.stdout:* 2023-10-06T05:22:08.167 INFO:teuthology.orchestra.run.smithi045.stdout:00000110 00 00 |..| 2023-10-06T05:22:08.168 INFO:teuthology.orchestra.run.smithi045.stdout:00000112 2023-10-06T05:22:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fb6b4ac750 msgr2=0x55fb6b41e960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fb6b4ac750 0x55fb6b41e960 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8750065890 msgr2=0x7f8750067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8750065890 0x7f8750067d40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f876000a650 tx=0x7f8760016000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 msgr2=0x55fb6b422440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 --2- 172.21.15.45:0/1787653261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb6b40aab0 0x55fb6b422440 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8758009f40 tx=0x7f8758018000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f876fe2a700 1 -- 172.21.15.45:0/1787653261 reap_dead start 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 shutdown_connections 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 >> 172.21.15.45:0/1787653261 conn(0x55fb6b3db4a0 msgr2=0x55fb6b4231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 shutdown_connections 2023-10-06T05:22:08.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.153+0000 7f877640b040 1 -- 172.21.15.45:0/1787653261 wait complete. 2023-10-06T05:22:08.205 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2023-10-06T05:22:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.251+0000 7fb89f822040 1 Processor -- start 2023-10-06T05:22:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.251+0000 7fb89f822040 1 -- start start 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb89f822040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57247fe0 0x564a5724bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb89f822040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57293340 0x564a5728ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb89f822040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57309970 0x564a5731e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb89f822040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a57194980 con 0x564a57247fe0 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb89f822040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a57194620 con 0x564a57293340 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb89f822040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a5718b400 con 0x564a57309970 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57247fe0 0x564a5724bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57247fe0 0x564a5724bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54940/0 (socket says 172.21.15.45:54940) 2023-10-06T05:22:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 -- 172.21.15.45:0/1089441197 learned_addr learned my addr 172.21.15.45:0/1089441197 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb899241700 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57309970 0x564a5731e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb893fff700 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57293340 0x564a5728ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 -- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57309970 msgr2=0x564a5731e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57309970 0x564a5731e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 -- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57293340 msgr2=0x564a5728ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57293340 0x564a5728ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 -- 172.21.15.45:0/1089441197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89400a0e0 con 0x564a57247fe0 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb893fff700 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57293340 0x564a5728ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.252+0000 7fb898a40700 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57247fe0 0x564a5724bd50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb884001d70 tx=0x7fb884001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f70a8081c8e0265 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb892ffd700 1 -- 172.21.15.45:0/1089441197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884014070 con 0x564a57247fe0 2023-10-06T05:22:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb892ffd700 1 -- 172.21.15.45:0/1089441197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884008030 con 0x564a57247fe0 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb892ffd700 1 -- 172.21.15.45:0/1089441197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400be30 con 0x564a57247fe0 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 -- 172.21.15.45:0/1089441197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57247fe0 msgr2=0x564a5724bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57247fe0 0x564a5724bd50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb884001d70 tx=0x7fb884001810 comp rx=0 tx=0).stop 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 -- 172.21.15.45:0/1089441197 shutdown_connections 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57309970 0x564a5731e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57247fe0 0x564a5724bd50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 --2- 172.21.15.45:0/1089441197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57293340 0x564a5728ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 -- 172.21.15.45:0/1089441197 >> 172.21.15.45:0/1089441197 conn(0x564a5724a4a0 msgr2=0x564a5724c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 -- 172.21.15.45:0/1089441197 shutdown_connections 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.253+0000 7fb89f822040 1 -- 172.21.15.45:0/1089441197 wait complete. 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.254+0000 7fb89f822040 1 Processor -- start 2023-10-06T05:22:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.254+0000 7fb89f822040 1 -- start start 2023-10-06T05:22:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb89f822040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57247fe0 0x564a57303380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb89f822040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57293340 0x564a5731e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb89f822040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57309970 0x564a57301430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb89f822040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a57303e80 con 0x564a57293340 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb89f822040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a57304120 con 0x564a57309970 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb89f822040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a573043c0 con 0x564a57247fe0 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57293340 0x564a5731e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57293340 0x564a5731e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54948/0 (socket says 172.21.15.45:54948) 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 -- 172.21.15.45:0/3970393359 learned_addr learned my addr 172.21.15.45:0/3970393359 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb898a40700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57247fe0 0x564a57303380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb899241700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57309970 0x564a57301430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 -- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57247fe0 msgr2=0x564a57303380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564a57247fe0 0x564a57303380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 -- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57309970 msgr2=0x564a57301430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a57309970 0x564a57301430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 -- 172.21.15.45:0/3970393359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88400a040 con 0x564a57293340 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.255+0000 7fb893fff700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57293340 0x564a5731e6b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb88c008b50 tx=0x7fb88c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.256+0000 7fb8917fa700 1 -- 172.21.15.45:0/3970393359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c019070 con 0x564a57293340 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.256+0000 7fb8917fa700 1 -- 172.21.15.45:0/3970393359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c010040 con 0x564a57293340 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.256+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564a571a33d0 con 0x564a57293340 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.256+0000 7fb8917fa700 1 -- 172.21.15.45:0/3970393359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00f050 con 0x564a57293340 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.256+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a57196ae0 con 0x564a57293340 2023-10-06T05:22:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.257+0000 7fb8917fa700 1 -- 172.21.15.45:0/3970393359 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb88c015460 con 0x564a57293340 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.258+0000 7fb8917fa700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb880065890 0x7fb880067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.258+0000 7fb8917fa700 1 -- 172.21.15.45:0/3970393359 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fb88c083c30 con 0x564a57293340 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.258+0000 7fb898a40700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb880065890 0x7fb880067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.258+0000 7fb89f822040 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564a5731ebf0 0x564a57278820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.258+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x564a57308a70 con 0x564a5731ebf0 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.258+0000 7fb899241700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564a5731ebf0 0x564a57278820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.258+0000 7fb898a40700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb880065890 0x7fb880067d40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb884000fd0 tx=0x7fb884016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb899241700 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564a5731ebf0 0x564a57278820 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb899241700 1 -- 172.21.15.45:0/3970393359 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) v8 ==== 159+0+14 (crc 0 0 0) 0x7fb894023040 con 0x564a5731ebf0 2023-10-06T05:22:08.265 INFO:teuthology.orchestra.run.smithi045.stdout:header (14 bytes) : 2023-10-06T05:22:08.265 INFO:teuthology.orchestra.run.smithi045.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2023-10-06T05:22:08.265 INFO:teuthology.orchestra.run.smithi045.stdout:0000000e 2023-10-06T05:22:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:08.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564a5731ebf0 msgr2=0x564a57278820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564a5731ebf0 0x564a57278820 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb880065890 msgr2=0x7fb880067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb880065890 0x7fb880067d40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb884000fd0 tx=0x7fb884016000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57293340 msgr2=0x564a5731e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 --2- 172.21.15.45:0/3970393359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a57293340 0x564a5731e6b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb88c008b50 tx=0x7fb88c00c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb899241700 1 -- 172.21.15.45:0/3970393359 reap_dead start 2023-10-06T05:22:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 shutdown_connections 2023-10-06T05:22:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 >> 172.21.15.45:0/3970393359 conn(0x564a5724a4a0 msgr2=0x564a5728fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 shutdown_connections 2023-10-06T05:22:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.259+0000 7fb89f822040 1 -- 172.21.15.45:0/3970393359 wait complete. 2023-10-06T05:22:08.267 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2023-10-06T05:22:08.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.312+0000 7fd19a746040 1 Processor -- start 2023-10-06T05:22:08.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.313+0000 7fd19a746040 1 -- start start 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.313+0000 7fd19a746040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087349b0 0x55880872d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.313+0000 7fd19a746040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087bdb70 0x5588087c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd19a746040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087c1f70 0x55880872ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd19a746040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558808634980 con 0x5588087349b0 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd19a746040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558808634620 con 0x5588087c1f70 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd19a746040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55880862b400 con 0x5588087bdb70 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087349b0 0x55880872d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087349b0 0x55880872d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54958/0 (socket says 172.21.15.45:54958) 2023-10-06T05:22:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 -- 172.21.15.45:0/2351982649 learned_addr learned my addr 172.21.15.45:0/2351982649 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193163700 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087bdb70 0x5588087c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd194165700 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087c1f70 0x55880872ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 -- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087bdb70 msgr2=0x5588087c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087bdb70 0x5588087c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 -- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087c1f70 msgr2=0x55880872ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087c1f70 0x55880872ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 -- 172.21.15.45:0/2351982649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c005db0 con 0x5588087349b0 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193163700 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087bdb70 0x5588087c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd193964700 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087349b0 0x55880872d260 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd184009980 tx=0x7fd18400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f985de3ba1848f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd192161700 1 -- 172.21.15.45:0/2351982649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18400f040 con 0x5588087349b0 2023-10-06T05:22:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd192161700 1 -- 172.21.15.45:0/2351982649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184008b50 con 0x5588087349b0 2023-10-06T05:22:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.314+0000 7fd192161700 1 -- 172.21.15.45:0/2351982649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184013420 con 0x5588087349b0 2023-10-06T05:22:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 -- 172.21.15.45:0/2351982649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087349b0 msgr2=0x55880872d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087349b0 0x55880872d260 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd184009980 tx=0x7fd18400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 -- 172.21.15.45:0/2351982649 shutdown_connections 2023-10-06T05:22:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087c1f70 0x55880872ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087349b0 0x55880872d260 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 --2- 172.21.15.45:0/2351982649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087bdb70 0x5588087c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 -- 172.21.15.45:0/2351982649 >> 172.21.15.45:0/2351982649 conn(0x5588086eb5f0 msgr2=0x5588087318f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 -- 172.21.15.45:0/2351982649 shutdown_connections 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.315+0000 7fd19a746040 1 -- 172.21.15.45:0/2351982649 wait complete. 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 Processor -- start 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 -- start start 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087349b0 0x5588087c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087bdb70 0x5588087ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087c1f70 0x5588087ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588086ecb10 con 0x5588087bdb70 2023-10-06T05:22:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588086ecdb0 con 0x5588087349b0 2023-10-06T05:22:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd19a746040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588086ed050 con 0x5588087c1f70 2023-10-06T05:22:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.316+0000 7fd193163700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087bdb70 0x5588087ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087bdb70 0x5588087ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54960/0 (socket says 172.21.15.45:54960) 2023-10-06T05:22:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 -- 172.21.15.45:0/4059483579 learned_addr learned my addr 172.21.15.45:0/4059483579 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd194165700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087c1f70 0x5588087ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193964700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087349b0 0x5588087c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 -- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087c1f70 msgr2=0x5588087ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5588087c1f70 0x5588087ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 -- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087349b0 msgr2=0x5588087c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588087349b0 0x5588087c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 -- 172.21.15.45:0/4059483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184009580 con 0x5588087bdb70 2023-10-06T05:22:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd193163700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087bdb70 0x5588087ce4a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c010 tx=0x7fd17c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd19095e700 1 -- 172.21.15.45:0/4059483579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c000c90 con 0x5588087bdb70 2023-10-06T05:22:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd19095e700 1 -- 172.21.15.45:0/4059483579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c021070 con 0x5588087bdb70 2023-10-06T05:22:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd19095e700 1 -- 172.21.15.45:0/4059483579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0089d0 con 0x5588087bdb70 2023-10-06T05:22:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5588086433d0 con 0x5588087bdb70 2023-10-06T05:22:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.317+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558808636ae0 con 0x5588087bdb70 2023-10-06T05:22:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.319+0000 7fd19095e700 1 -- 172.21.15.45:0/4059483579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd17c023030 con 0x5588087bdb70 2023-10-06T05:22:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.319+0000 7fd19095e700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1740658c0 0x7fd174067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.319+0000 7fd19095e700 1 -- 172.21.15.45:0/4059483579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fd17c0857b0 con 0x5588087bdb70 2023-10-06T05:22:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.320+0000 7fd193964700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1740658c0 0x7fd174067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.320+0000 7fd19a746040 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5588086ec3a0 0x55880872f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.320+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x5588087b7f00 con 0x5588086ec3a0 2023-10-06T05:22:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.320+0000 7fd194165700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5588086ec3a0 0x55880872f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.320+0000 7fd193964700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1740658c0 0x7fd174067d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd184009ef0 tx=0x7fd184018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.320+0000 7fd194165700 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5588086ec3a0 0x55880872f140 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd194165700 1 -- 172.21.15.45:0/4059483579 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd18c00e730 con 0x5588086ec3a0 2023-10-06T05:22:08.332 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:08.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5588086ec3a0 msgr2=0x55880872f140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5588086ec3a0 0x55880872f140 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1740658c0 msgr2=0x7fd174067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1740658c0 0x7fd174067d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd184009ef0 tx=0x7fd184018000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087bdb70 msgr2=0x5588087ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 --2- 172.21.15.45:0/4059483579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588087bdb70 0x5588087ce4a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c010 tx=0x7fd17c016000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd194165700 1 -- 172.21.15.45:0/4059483579 reap_dead start 2023-10-06T05:22:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 shutdown_connections 2023-10-06T05:22:08.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 >> 172.21.15.45:0/4059483579 conn(0x5588086eb5f0 msgr2=0x55880872d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 shutdown_connections 2023-10-06T05:22:08.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.321+0000 7fd19a746040 1 -- 172.21.15.45:0/4059483579 wait complete. 2023-10-06T05:22:08.335 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2023-10-06T05:22:08.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.380+0000 7fdce6f5e040 1 Processor -- start 2023-10-06T05:22:08.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.380+0000 7fdce6f5e040 1 -- start start 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce6f5e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582643c8f80 0x55826439dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce6f5e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558264459900 0x5582643e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce6f5e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 0x55826447cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce6f5e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582642e4980 con 0x5582643c8f80 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce6f5e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582642e4620 con 0x558264459900 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce6f5e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582642db400 con 0x558264459cd0 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582643c8f80 0x55826439dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582643c8f80 0x55826439dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54972/0 (socket says 172.21.15.45:54972) 2023-10-06T05:22:08.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 -- 172.21.15.45:0/929738773 learned_addr learned my addr 172.21.15.45:0/929738773 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce097d700 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 0x55826447cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdcdf97b700 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558264459900 0x5582643e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 -- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 msgr2=0x55826447cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 0x55826447cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 -- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558264459900 msgr2=0x5582643e2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558264459900 0x5582643e2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 -- 172.21.15.45:0/929738773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0005cf0 con 0x5582643c8f80 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.381+0000 7fdce017c700 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582643c8f80 0x55826439dba0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000acf0 tx=0x7fdcd0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ff8d28a91ce04d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdcde979700 1 -- 172.21.15.45:0/929738773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0002f90 con 0x5582643c8f80 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdcde979700 1 -- 172.21.15.45:0/929738773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0000bc0 con 0x5582643c8f80 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdcde979700 1 -- 172.21.15.45:0/929738773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0008ba0 con 0x5582643c8f80 2023-10-06T05:22:08.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 -- 172.21.15.45:0/929738773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582643c8f80 msgr2=0x55826439dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582643c8f80 0x55826439dba0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000acf0 tx=0x7fdcd0016000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 -- 172.21.15.45:0/929738773 shutdown_connections 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 0x55826447cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582643c8f80 0x55826439dba0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/929738773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558264459900 0x5582643e2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 -- 172.21.15.45:0/929738773 >> 172.21.15.45:0/929738773 conn(0x55826439b5f0 msgr2=0x558264458aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 -- 172.21.15.45:0/929738773 shutdown_connections 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.382+0000 7fdce6f5e040 1 -- 172.21.15.45:0/929738773 wait complete. 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.383+0000 7fdce6f5e040 1 Processor -- start 2023-10-06T05:22:08.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.383+0000 7fdce6f5e040 1 -- start start 2023-10-06T05:22:08.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce6f5e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582643c8f80 0x558264472520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce6f5e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558264459900 0x5582643dea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce6f5e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 0x558264476e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce6f5e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558264472d10 con 0x558264459900 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce6f5e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558264474970 con 0x5582643c8f80 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce6f5e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558264474c10 con 0x558264459cd0 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558264459900 0x5582643dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558264459900 0x5582643dea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54980/0 (socket says 172.21.15.45:54980) 2023-10-06T05:22:08.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 -- 172.21.15.45:0/4171172856 learned_addr learned my addr 172.21.15.45:0/4171172856 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce017c700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582643c8f80 0x558264472520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdce097d700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 0x558264476e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 -- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 msgr2=0x558264476e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558264459cd0 0x558264476e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 -- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582643c8f80 msgr2=0x558264472520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582643c8f80 0x558264472520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 -- 172.21.15.45:0/4171172856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55826430d1c0 con 0x558264459900 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.384+0000 7fdcdf97b700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558264459900 0x5582643dea70 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4008b50 tx=0x7fdcd400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.385+0000 7fdcdd176700 1 -- 172.21.15.45:0/4171172856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400f050 con 0x558264459900 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.385+0000 7fdcdd176700 1 -- 172.21.15.45:0/4171172856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd4010040 con 0x558264459900 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.385+0000 7fdcdd176700 1 -- 172.21.15.45:0/4171172856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4006cf0 con 0x558264459900 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.385+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5582642f33d0 con 0x558264459900 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.385+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582642e6ae0 con 0x558264459900 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.386+0000 7fdcdd176700 1 -- 172.21.15.45:0/4171172856 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcd4015670 con 0x558264459900 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.387+0000 7fdcdd176700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdccc0658c0 0x7fdccc067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.387+0000 7fdcdd176700 1 -- 172.21.15.45:0/4171172856 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fdcd4083eb0 con 0x558264459900 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.387+0000 7fdce017c700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdccc0658c0 0x7fdccc067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.387+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5582643e0810 0x5582643e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.387+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x5582643e1150 con 0x5582643e0810 2023-10-06T05:22:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.387+0000 7fdce097d700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5582643e0810 0x5582643e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.387+0000 7fdce017c700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdccc0658c0 0x7fdccc067d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000a650 tx=0x7fdcd001e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce097d700 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5582643e0810 0x5582643e0c10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce097d700 1 -- 172.21.15.45:0/4171172856 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fdcd8004590 con 0x5582643e0810 2023-10-06T05:22:08.394 INFO:teuthology.orchestra.run.smithi045.stdout:header (0 bytes) : 2023-10-06T05:22:08.395 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5582643e0810 msgr2=0x5582643e0c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5582643e0810 0x5582643e0c10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdccc0658c0 msgr2=0x7fdccc067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdccc0658c0 0x7fdccc067d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000a650 tx=0x7fdcd001e000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558264459900 msgr2=0x5582643dea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 --2- 172.21.15.45:0/4171172856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558264459900 0x5582643dea70 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4008b50 tx=0x7fdcd400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce097d700 1 -- 172.21.15.45:0/4171172856 reap_dead start 2023-10-06T05:22:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 shutdown_connections 2023-10-06T05:22:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 >> 172.21.15.45:0/4171172856 conn(0x55826439b5f0 msgr2=0x55826439d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 shutdown_connections 2023-10-06T05:22:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.388+0000 7fdce6f5e040 1 -- 172.21.15.45:0/4171172856 wait complete. 2023-10-06T05:22:08.401 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2023-10-06T05:22:08.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.446+0000 7f3ae61f9040 1 Processor -- start 2023-10-06T05:22:08.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.446+0000 7f3ae61f9040 1 -- start start 2023-10-06T05:22:08.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3ae61f9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0aaf210 0x5638e0b70be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3ae61f9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0b71890 0x5638e0ae03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3ae61f9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71c60 0x5638e0af5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3ae61f9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638e09fc980 con 0x5638e0b71c60 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3ae61f9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638e09fc620 con 0x5638e0aaf210 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3ae61f9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638e09f3400 con 0x5638e0b71890 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71c60 0x5638e0af5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71c60 0x5638e0af5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55000/0 (socket says 172.21.15.45:55000) 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 -- 172.21.15.45:0/3945885077 learned_addr learned my addr 172.21.15.45:0/3945885077 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adf417700 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0aaf210 0x5638e0b70be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adec16700 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0b71890 0x5638e0ae03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 -- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0b71890 msgr2=0x5638e0ae03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0b71890 0x5638e0ae03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 -- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0aaf210 msgr2=0x5638e0b70be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0aaf210 0x5638e0b70be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 -- 172.21.15.45:0/3945885077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad800a180 con 0x5638e0b71c60 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3adfc18700 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71c60 0x5638e0af5fa0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005e40 tx=0x7f3ad8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed4d01e700684d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3addc14700 1 -- 172.21.15.45:0/3945885077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8005960 con 0x5638e0b71c60 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3addc14700 1 -- 172.21.15.45:0/3945885077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad800ed60 con 0x5638e0b71c60 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.447+0000 7f3addc14700 1 -- 172.21.15.45:0/3945885077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad801c430 con 0x5638e0b71c60 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 -- 172.21.15.45:0/3945885077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71c60 msgr2=0x5638e0af5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71c60 0x5638e0af5fa0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005e40 tx=0x7f3ad8014000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 -- 172.21.15.45:0/3945885077 shutdown_connections 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71c60 0x5638e0af5fa0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0aaf210 0x5638e0b70be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/3945885077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0b71890 0x5638e0ae03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 -- 172.21.15.45:0/3945885077 >> 172.21.15.45:0/3945885077 conn(0x5638e0ab2450 msgr2=0x5638e0aaf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 -- 172.21.15.45:0/3945885077 shutdown_connections 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.448+0000 7f3ae61f9040 1 -- 172.21.15.45:0/3945885077 wait complete. 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.449+0000 7f3ae61f9040 1 Processor -- start 2023-10-06T05:22:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.449+0000 7f3ae61f9040 1 -- start start 2023-10-06T05:22:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ae61f9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0aaf210 0x5638e0ab4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ae61f9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71890 0x5638e0b98600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ae61f9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0b71c60 0x5638e0b6f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ae61f9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638e0ad3e10 con 0x5638e0b71890 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ae61f9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638e0ad40b0 con 0x5638e0b71c60 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ae61f9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638e0ad4350 con 0x5638e0aaf210 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71890 0x5638e0b98600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71890 0x5638e0b98600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55010/0 (socket says 172.21.15.45:55010) 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 -- 172.21.15.45:0/4045632914 learned_addr learned my addr 172.21.15.45:0/4045632914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adf417700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0aaf210 0x5638e0ab4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adfc18700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0b71c60 0x5638e0b6f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 -- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0aaf210 msgr2=0x5638e0ab4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638e0aaf210 0x5638e0ab4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 -- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0b71c60 msgr2=0x5638e0b6f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638e0b71c60 0x5638e0b6f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 -- 172.21.15.45:0/4045632914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638e0a251c0 con 0x5638e0b71890 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3adec16700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71890 0x5638e0b98600 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80087c0 tx=0x7f3ac800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ad7fff700 1 -- 172.21.15.45:0/4045632914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800f040 con 0x5638e0b71890 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.450+0000 7f3ad7fff700 1 -- 172.21.15.45:0/4045632914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac8015680 con 0x5638e0b71890 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.451+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5638e0a0b3d0 con 0x5638e0b71890 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.451+0000 7f3ad7fff700 1 -- 172.21.15.45:0/4045632914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8015a20 con 0x5638e0b71890 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.451+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638e09feae0 con 0x5638e0b71890 2023-10-06T05:22:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.452+0000 7f3ad7fff700 1 -- 172.21.15.45:0/4045632914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ac801e400 con 0x5638e0b71890 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.452+0000 7f3ad7fff700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ac0065890 0x7f3ac0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.453+0000 7f3ad7fff700 1 -- 172.21.15.45:0/4045632914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f3ac80840d0 con 0x5638e0b71890 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.453+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5638e0b98b40 0x5638e0ad4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.453+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x5638e0b8c660 con 0x5638e0b98b40 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.453+0000 7f3adf417700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ac0065890 0x7f3ac0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.453+0000 7f3adfc18700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5638e0b98b40 0x5638e0ad4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.453+0000 7f3adfc18700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5638e0b98b40 0x5638e0ad4fa0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.453+0000 7f3adf417700 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ac0065890 0x7f3ac0067d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000f770 tx=0x7f3ad0000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3adfc18700 1 -- 172.21.15.45:0/4045632914 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x7f3ad800e7e0 con 0x5638e0b98b40 2023-10-06T05:22:08.470 INFO:teuthology.orchestra.run.smithi045.stdout:header (274 bytes) : 2023-10-06T05:22:08.470 INFO:teuthology.orchestra.run.smithi045.stdout:00000000 04 03 0c 01 00 00 0c 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000020 00 00 00 00 77 99 1f 65 d4 a4 6f 1c 00 00 00 00 |....w..e..o.....| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000050 00 00 77 99 1f 65 d4 a4 6f 1c 00 00 00 00 00 00 |..w..e..o.......| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000070 00 00 03 02 38 00 00 00 01 00 00 00 00 00 00 00 |....8...........| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:000000a0 00 00 00 00 00 00 00 00 77 99 1f 65 9a 5b af 23 |........w..e.[.#| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:000000b0 03 02 38 00 00 00 01 00 00 00 00 00 00 00 00 00 |..8.............| 2023-10-06T05:22:08.471 INFO:teuthology.orchestra.run.smithi045.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 |`...............| 2023-10-06T05:22:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:000000e0 00 00 00 00 00 00 77 99 1f 65 9a 5b af 23 00 00 |......w..e.[.#..| 2023-10-06T05:22:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2023-10-06T05:22:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:* 2023-10-06T05:22:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:00000110 00 00 |..| 2023-10-06T05:22:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:00000112 2023-10-06T05:22:08.472 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5638e0b98b40 msgr2=0x5638e0ad4fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5638e0b98b40 0x5638e0ad4fa0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ac0065890 msgr2=0x7f3ac0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ac0065890 0x7f3ac0067d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000f770 tx=0x7f3ad0000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71890 msgr2=0x5638e0b98600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 --2- 172.21.15.45:0/4045632914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5638e0b71890 0x5638e0b98600 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80087c0 tx=0x7f3ac800c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3adfc18700 1 -- 172.21.15.45:0/4045632914 reap_dead start 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 shutdown_connections 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 >> 172.21.15.45:0/4045632914 conn(0x5638e0ab2450 msgr2=0x5638e0b70a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 shutdown_connections 2023-10-06T05:22:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.454+0000 7f3ae61f9040 1 -- 172.21.15.45:0/4045632914 wait complete. 2023-10-06T05:22:08.474 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2023-10-06T05:22:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.519+0000 7fe9ac162040 1 Processor -- start 2023-10-06T05:22:08.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.519+0000 7fe9ac162040 1 -- start start 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9ac162040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1482f0 0x55651a161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9ac162040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 0x55651a1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9ac162040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1f2fc0 0x55651a15ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9ac162040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651a067980 con 0x55651a1482f0 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9ac162040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651a067620 con 0x55651a1f2fc0 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9ac162040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55651a05e400 con 0x55651a14c450 2023-10-06T05:22:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9a5380700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1482f0 0x55651a161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9a5380700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1482f0 0x55651a161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55030/0 (socket says 172.21.15.45:55030) 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9a5380700 1 -- 172.21.15.45:0/4279655462 learned_addr learned my addr 172.21.15.45:0/4279655462 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe9a5b81700 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1f2fc0 0x55651a15ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.520+0000 7fe99ffff700 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 0x55651a1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9a5380700 1 -- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 msgr2=0x55651a1f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9a5380700 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 0x55651a1f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9a5380700 1 -- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1f2fc0 msgr2=0x55651a15ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9a5380700 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1f2fc0 0x55651a15ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9a5380700 1 -- 172.21.15.45:0/4279655462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe998005cf0 con 0x55651a1482f0 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9a5380700 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1482f0 0x55651a161d00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe99800a520 tx=0x7fe998016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e731ee4be163439b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe99f7fe700 1 -- 172.21.15.45:0/4279655462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998000c90 con 0x55651a1482f0 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe99f7fe700 1 -- 172.21.15.45:0/4279655462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998008970 con 0x55651a1482f0 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe99f7fe700 1 -- 172.21.15.45:0/4279655462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99801c410 con 0x55651a1482f0 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9ac162040 1 -- 172.21.15.45:0/4279655462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1482f0 msgr2=0x55651a161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.521+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1482f0 0x55651a161d00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe99800a520 tx=0x7fe998016000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 -- 172.21.15.45:0/4279655462 shutdown_connections 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1f2fc0 0x55651a15ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1482f0 0x55651a161d00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4279655462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 0x55651a1f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 -- 172.21.15.45:0/4279655462 >> 172.21.15.45:0/4279655462 conn(0x55651a11cbd0 msgr2=0x55651a11ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 -- 172.21.15.45:0/4279655462 shutdown_connections 2023-10-06T05:22:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 -- 172.21.15.45:0/4279655462 wait complete. 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 Processor -- start 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.522+0000 7fe9ac162040 1 -- start start 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9ac162040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1482f0 0x55651a1ea520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9ac162040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 0x55651a1618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9ac162040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1f2fc0 0x55651a161e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9ac162040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651a1ec060 con 0x55651a1f2fc0 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9ac162040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651a1eb6c0 con 0x55651a1482f0 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9ac162040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55651a1f6270 con 0x55651a14c450 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9a5b81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1f2fc0 0x55651a161e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9a5b81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1f2fc0 0x55651a161e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55042/0 (socket says 172.21.15.45:55042) 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9a5b81700 1 -- 172.21.15.45:0/4110635785 learned_addr learned my addr 172.21.15.45:0/4110635785 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe99ffff700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 0x55651a1618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9a5380700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1482f0 0x55651a1ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.523+0000 7fe9a5b81700 1 -- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 msgr2=0x55651a1618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe9a5b81700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55651a14c450 0x55651a1618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe9a5b81700 1 -- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1482f0 msgr2=0x55651a1ea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe9a5b81700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a1482f0 0x55651a1ea520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe9a5b81700 1 -- 172.21.15.45:0/4110635785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55651a0901c0 con 0x55651a1f2fc0 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe9a5b81700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1f2fc0 0x55651a161e30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0002fd0 tx=0x7fe9a0005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe99dffb700 1 -- 172.21.15.45:0/4110635785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00025b0 con 0x55651a1f2fc0 2023-10-06T05:22:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe99dffb700 1 -- 172.21.15.45:0/4110635785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000e6e0 con 0x55651a1f2fc0 2023-10-06T05:22:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55651a0763d0 con 0x55651a1f2fc0 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe99dffb700 1 -- 172.21.15.45:0/4110635785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000eaa0 con 0x55651a1f2fc0 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.524+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55651a069ae0 con 0x55651a1f2fc0 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.525+0000 7fe99dffb700 1 -- 172.21.15.45:0/4110635785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9a0003040 con 0x55651a1f2fc0 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.526+0000 7fe99dffb700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe988065890 0x7fe988067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.526+0000 7fe99dffb700 1 -- 172.21.15.45:0/4110635785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fe9a0017030 con 0x55651a1f2fc0 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.526+0000 7fe9a5380700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe988065890 0x7fe988067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.526+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55651a1ec220 0x55651a1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.526+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55651a1f42d0 con 0x55651a1ec220 2023-10-06T05:22:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.526+0000 7fe99ffff700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55651a1ec220 0x55651a1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9a5380700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe988065890 0x7fe988067d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe998008fd0 tx=0x7fe99801d000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe99ffff700 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55651a1ec220 0x55651a1f3d90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe99ffff700 1 -- 172.21.15.45:0/4110635785 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+21 (crc 0 0 0) 0x7fe99000f8f0 con 0x55651a1ec220 2023-10-06T05:22:08.554 INFO:teuthology.orchestra.run.smithi045.stdout:sixmegs_head 2023-10-06T05:22:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55651a1ec220 msgr2=0x55651a1f3d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55651a1ec220 0x55651a1f3d90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe988065890 msgr2=0x7fe988067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe988065890 0x7fe988067d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe998008fd0 tx=0x7fe99801d000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1f2fc0 msgr2=0x55651a161e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 --2- 172.21.15.45:0/4110635785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a1f2fc0 0x55651a161e30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0002fd0 tx=0x7fe9a0005d40 comp rx=0 tx=0).stop 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9a5b81700 1 -- 172.21.15.45:0/4110635785 reap_dead start 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 shutdown_connections 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 >> 172.21.15.45:0/4110635785 conn(0x55651a11cbd0 msgr2=0x55651a148a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 shutdown_connections 2023-10-06T05:22:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.527+0000 7fe9ac162040 1 -- 172.21.15.45:0/4110635785 wait complete. 2023-10-06T05:22:08.556 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2023-10-06T05:22:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.604+0000 7f57f2c04040 1 Processor -- start 2023-10-06T05:22:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.604+0000 7f57f2c04040 1 -- start start 2023-10-06T05:22:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57f2c04040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57f2c04040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894f42e0 0x559a894f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57f2c04040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894fab10 0x559a894e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57f2c04040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a893ca320 con 0x559a894f42e0 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57f2c04040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8936e980 con 0x559a8946b7d0 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57f2c04040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a8936e620 con 0x559a894fab10 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894f42e0 0x559a894f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894f42e0 0x559a894f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55050/0 (socket says 172.21.15.45:55050) 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 -- 172.21.15.45:0/1174303431 learned_addr learned my addr 172.21.15.45:0/1174303431 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57ebe22700 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57ec623700 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894fab10 0x559a894e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 -- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894fab10 msgr2=0x559a894e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894fab10 0x559a894e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 -- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 msgr2=0x559a894f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 -- 172.21.15.45:0/1174303431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc005cf0 con 0x559a894f42e0 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57ebe22700 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.605+0000 7f57eb621700 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894f42e0 0x559a894f2560 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ac70 tx=0x7f57dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbe6f2ef7f2954a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57ea61f700 1 -- 172.21.15.45:0/1174303431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc002f70 con 0x559a894f42e0 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57ea61f700 1 -- 172.21.15.45:0/1174303431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc000bc0 con 0x559a894f42e0 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57ea61f700 1 -- 172.21.15.45:0/1174303431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc007b10 con 0x559a894f42e0 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 -- 172.21.15.45:0/1174303431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894f42e0 msgr2=0x559a894f2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894f42e0 0x559a894f2560 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ac70 tx=0x7f57dc015000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 -- 172.21.15.45:0/1174303431 shutdown_connections 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894fab10 0x559a894e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 --2- 172.21.15.45:0/1174303431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894f42e0 0x559a894f2560 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 -- 172.21.15.45:0/1174303431 >> 172.21.15.45:0/1174303431 conn(0x559a89423bd0 msgr2=0x559a894f3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 -- 172.21.15.45:0/1174303431 shutdown_connections 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.606+0000 7f57f2c04040 1 -- 172.21.15.45:0/1174303431 wait complete. 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.607+0000 7f57f2c04040 1 Processor -- start 2023-10-06T05:22:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.607+0000 7f57f2c04040 1 -- start start 2023-10-06T05:22:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57f2c04040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894efd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57f2c04040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894f42e0 0x559a894f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57f2c04040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894fab10 0x559a894f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57f2c04040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a89365400 con 0x559a894fab10 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57f2c04040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a89425750 con 0x559a8946b7d0 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57f2c04040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a894e6080 con 0x559a894f42e0 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894fab10 0x559a894f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894fab10 0x559a894f9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55066/0 (socket says 172.21.15.45:55066) 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 -- 172.21.15.45:0/3773157674 learned_addr learned my addr 172.21.15.45:0/3773157674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57eb621700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894f42e0 0x559a894f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ebe22700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 -- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894f42e0 msgr2=0x559a894f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559a894f42e0 0x559a894f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 -- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 msgr2=0x559a894efd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894efd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ec623700 1 -- 172.21.15.45:0/3773157674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a8937d3d0 con 0x559a894fab10 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.608+0000 7f57ebe22700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a8946b7d0 0x559a894efd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.609+0000 7f57ec623700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894fab10 0x559a894f9be0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f57e400d2c0 tx=0x7f57e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.609+0000 7f57e8e1c700 1 -- 172.21.15.45:0/3773157674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400deb0 con 0x559a894fab10 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.609+0000 7f57e8e1c700 1 -- 172.21.15.45:0/3773157674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e40030d0 con 0x559a894fab10 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.609+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559a89370ae0 con 0x559a894fab10 2023-10-06T05:22:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.609+0000 7f57e8e1c700 1 -- 172.21.15.45:0/3773157674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4003470 con 0x559a894fab10 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.609+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a89370540 con 0x559a894fab10 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.610+0000 7f57e8e1c700 1 -- 172.21.15.45:0/3773157674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57e4001860 con 0x559a894fab10 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.611+0000 7f57e8e1c700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc065890 0x7f57cc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.611+0000 7f57e8e1c700 1 -- 172.21.15.45:0/3773157674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f57e4082ea0 con 0x559a894fab10 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.611+0000 7f57f2c04040 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x559a89425160 0x559a894f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.611+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559a894f0250 con 0x559a89425160 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.611+0000 7f57ebe22700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc065890 0x7f57cc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.611+0000 7f57eb621700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x559a89425160 0x559a894f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57eb621700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x559a89425160 0x559a894f5760 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57ebe22700 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc065890 0x7f57cc067d40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f57d4001ed0 tx=0x7f57d40017a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57eb621700 1 -- 172.21.15.45:0/3773157674 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7f57dc007490 con 0x559a89425160 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x559a89425160 msgr2=0x559a894f5760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x559a89425160 0x559a894f5760 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc065890 msgr2=0x7f57cc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc065890 0x7f57cc067d40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f57d4001ed0 tx=0x7f57d40017a0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894fab10 msgr2=0x559a894f9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 --2- 172.21.15.45:0/3773157674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a894fab10 0x559a894f9be0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f57e400d2c0 tx=0x7f57e4008040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57ec623700 1 -- 172.21.15.45:0/3773157674 reap_dead start 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 shutdown_connections 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 >> 172.21.15.45:0/3773157674 conn(0x559a89423bd0 msgr2=0x559a894f36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 shutdown_connections 2023-10-06T05:22:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.612+0000 7f57f2c04040 1 -- 172.21.15.45:0/3773157674 wait complete. 2023-10-06T05:22:08.619 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2023-10-06T05:22:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.665+0000 7f38d8da8040 1 Processor -- start 2023-10-06T05:22:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.665+0000 7f38d8da8040 1 -- start start 2023-10-06T05:22:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d8da8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c33c60 0x558976b71850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d8da8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c31250 0x558976c56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d8da8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c57020 0x558976c573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d8da8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976b1a320 con 0x558976c33c60 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d8da8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976abe980 con 0x558976c57020 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d8da8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558976abe620 con 0x558976c31250 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c33c60 0x558976b71850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c33c60 0x558976b71850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55078/0 (socket says 172.21.15.45:55078) 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 -- 172.21.15.45:0/3392468465 learned_addr learned my addr 172.21.15.45:0/3392468465 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d17c5700 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c31250 0x558976c56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d27c7700 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c57020 0x558976c573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 -- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c31250 msgr2=0x558976c56ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c31250 0x558976c56ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 -- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c57020 msgr2=0x558976c573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c57020 0x558976c573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 -- 172.21.15.45:0/3392468465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558976ac0ae0 con 0x558976c33c60 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38d1fc6700 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c33c60 0x558976b71850 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0010c0 tx=0x558976bbbc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7598f18dc1d33deb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38cbfff700 1 -- 172.21.15.45:0/3392468465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00f450 con 0x558976c33c60 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38cbfff700 1 -- 172.21.15.45:0/3392468465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc005030 con 0x558976c33c60 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.666+0000 7f38cbfff700 1 -- 172.21.15.45:0/3392468465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00f610 con 0x558976c33c60 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 -- 172.21.15.45:0/3392468465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c33c60 msgr2=0x558976b71850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c33c60 0x558976b71850 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0010c0 tx=0x558976bbbc00 comp rx=0 tx=0).stop 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 -- 172.21.15.45:0/3392468465 shutdown_connections 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c57020 0x558976c573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c33c60 0x558976b71850 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 --2- 172.21.15.45:0/3392468465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c31250 0x558976c56ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 -- 172.21.15.45:0/3392468465 >> 172.21.15.45:0/3392468465 conn(0x558976b74450 msgr2=0x558976ba26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 -- 172.21.15.45:0/3392468465 shutdown_connections 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.667+0000 7f38d8da8040 1 -- 172.21.15.45:0/3392468465 wait complete. 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 Processor -- start 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 -- start start 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c31250 0x558976ba2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c33c60 0x558976b74850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c57020 0x558976c32860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976ab5400 con 0x558976c31250 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558976c30b60 con 0x558976c33c60 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.668+0000 7f38d8da8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558976c52ea0 con 0x558976c57020 2023-10-06T05:22:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c31250 0x558976ba2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c31250 0x558976ba2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55082/0 (socket says 172.21.15.45:55082) 2023-10-06T05:22:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 -- 172.21.15.45:0/686778700 learned_addr learned my addr 172.21.15.45:0/686778700 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d27c7700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c57020 0x558976c32860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d17c5700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c33c60 0x558976b74850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 -- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c57020 msgr2=0x558976c32860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558976c57020 0x558976c32860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 -- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c33c60 msgr2=0x558976b74850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c33c60 0x558976b74850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 -- 172.21.15.45:0/686778700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558976acd3d0 con 0x558976c31250 2023-10-06T05:22:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d17c5700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558976c33c60 0x558976b74850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d1fc6700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c31250 0x558976ba2620 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f38bc000db0 tx=0x7f38bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38ca7fc700 1 -- 172.21.15.45:0/686778700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc018990 con 0x558976c31250 2023-10-06T05:22:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38ca7fc700 1 -- 172.21.15.45:0/686778700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc018af0 con 0x558976c31250 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38ca7fc700 1 -- 172.21.15.45:0/686778700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc018e90 con 0x558976c31250 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558976ac0540 con 0x558976c31250 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.669+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558976c51270 con 0x558976c31250 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.671+0000 7f38ca7fc700 1 -- 172.21.15.45:0/686778700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38bc015020 con 0x558976c31250 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.671+0000 7f38ca7fc700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38b4065890 0x7f38b4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.671+0000 7f38ca7fc700 1 -- 172.21.15.45:0/686778700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f38bc0853c0 con 0x558976c31250 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.671+0000 7f38d17c5700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38b4065890 0x7f38b4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.671+0000 7f38d8da8040 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558976b74d90 0x558976c2f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.671+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x558976b9f430 con 0x558976b74d90 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d27c7700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558976b74d90 0x558976c2f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d17c5700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38b4065890 0x7f38b4067d40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098d0 tx=0x7f38c4018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d27c7700 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558976b74d90 0x558976c2f2a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d27c7700 1 -- 172.21.15.45:0/686778700 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv17 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7f38cc001aa0 con 0x558976b74d90 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558976b74d90 msgr2=0x558976c2f2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d8da8040 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558976b74d90 0x558976c2f2a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38b4065890 msgr2=0x7f38b4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d8da8040 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38b4065890 0x7f38b4067d40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098d0 tx=0x7f38c4018000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c31250 msgr2=0x558976ba2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d8da8040 1 --2- 172.21.15.45:0/686778700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558976c31250 0x558976ba2620 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f38bc000db0 tx=0x7f38bc017000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.672+0000 7f38d27c7700 1 -- 172.21.15.45:0/686778700 reap_dead start 2023-10-06T05:22:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.673+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 shutdown_connections 2023-10-06T05:22:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.673+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 >> 172.21.15.45:0/686778700 conn(0x558976b74450 msgr2=0x558976bba210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.673+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 shutdown_connections 2023-10-06T05:22:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.673+0000 7f38d8da8040 1 -- 172.21.15.45:0/686778700 wait complete. 2023-10-06T05:22:08.718 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2023-10-06T05:22:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.763+0000 7fa77de40040 1 Processor -- start 2023-10-06T05:22:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.763+0000 7fa77de40040 1 -- start start 2023-10-06T05:22:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77de40040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab8f80 0x556d6ba8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77de40040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bab9da0 0x556d6bb62020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77de40040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bb49900 0x556d6bb62560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77de40040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6b9d4980 con 0x556d6bab9da0 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77de40040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6b9d4620 con 0x556d6bb49900 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77de40040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d6b9cb400 con 0x556d6bab8f80 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bab9da0 0x556d6bb62020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bab9da0 0x556d6bb62020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55090/0 (socket says 172.21.15.45:55090) 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa76ffff700 1 -- 172.21.15.45:0/2167321405 learned_addr learned my addr 172.21.15.45:0/2167321405 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77785f700 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bb49900 0x556d6bb62560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.764+0000 7fa77705e700 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab8f80 0x556d6ba8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa76ffff700 1 -- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab8f80 msgr2=0x556d6ba8d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa76ffff700 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab8f80 0x556d6ba8d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa76ffff700 1 -- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bb49900 msgr2=0x556d6bb62560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa76ffff700 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bb49900 0x556d6bb62560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa76ffff700 1 -- 172.21.15.45:0/2167321405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77000a0e0 con 0x556d6bab9da0 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa76ffff700 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bab9da0 0x556d6bb62020 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x556d6baba170 tx=0x7fa760015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05d93303d60cd8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa77605c700 1 -- 172.21.15.45:0/2167321405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760000bc0 con 0x556d6bab9da0 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa77605c700 1 -- 172.21.15.45:0/2167321405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7600078e0 con 0x556d6bab9da0 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa77605c700 1 -- 172.21.15.45:0/2167321405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76001c410 con 0x556d6bab9da0 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa77de40040 1 -- 172.21.15.45:0/2167321405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bab9da0 msgr2=0x556d6bb62020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.765+0000 7fa77de40040 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bab9da0 0x556d6bb62020 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x556d6baba170 tx=0x7fa760015000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 -- 172.21.15.45:0/2167321405 shutdown_connections 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bb49900 0x556d6bb62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab8f80 0x556d6ba8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 --2- 172.21.15.45:0/2167321405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bab9da0 0x556d6bb62020 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 -- 172.21.15.45:0/2167321405 >> 172.21.15.45:0/2167321405 conn(0x556d6ba8b5f0 msgr2=0x556d6bb62e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 -- 172.21.15.45:0/2167321405 shutdown_connections 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 -- 172.21.15.45:0/2167321405 wait complete. 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 Processor -- start 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.766+0000 7fa77de40040 1 -- start start 2023-10-06T05:22:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77de40040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bab8f80 0x556d6bad0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77de40040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab9da0 0x556d6bad1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77de40040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bb49900 0x556d6bb73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77de40040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6bad1760 con 0x556d6bb49900 2023-10-06T05:22:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77de40040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6bad2d80 con 0x556d6bab8f80 2023-10-06T05:22:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77de40040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d6bacf880 con 0x556d6bab9da0 2023-10-06T05:22:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77785f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bb49900 0x556d6bb73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77705e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bab8f80 0x556d6bad0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab9da0 0x556d6bad1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77785f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bb49900 0x556d6bb73bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55096/0 (socket says 172.21.15.45:55096) 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.767+0000 7fa77785f700 1 -- 172.21.15.45:0/3577002413 learned_addr learned my addr 172.21.15.45:0/3577002413 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77785f700 1 -- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab9da0 msgr2=0x556d6bad1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77785f700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556d6bab9da0 0x556d6bad1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77785f700 1 -- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bab8f80 msgr2=0x556d6bad0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77785f700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d6bab8f80 0x556d6bad0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77785f700 1 -- 172.21.15.45:0/3577002413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d6b9fd1c0 con 0x556d6bb49900 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77785f700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bb49900 0x556d6bb73bf0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa77000afd0 tx=0x7fa770007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa774859700 1 -- 172.21.15.45:0/3577002413 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000e6e0 con 0x556d6bb49900 2023-10-06T05:22:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa774859700 1 -- 172.21.15.45:0/3577002413 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77000e840 con 0x556d6bb49900 2023-10-06T05:22:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa760005cf0 con 0x556d6bb49900 2023-10-06T05:22:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa774859700 1 -- 172.21.15.45:0/3577002413 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000ebe0 con 0x556d6bb49900 2023-10-06T05:22:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.768+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d6b9e33d0 con 0x556d6bb49900 2023-10-06T05:22:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.769+0000 7fa774859700 1 -- 172.21.15.45:0/3577002413 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa770013440 con 0x556d6bb49900 2023-10-06T05:22:08.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.770+0000 7fa774859700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa75c0658c0 0x7fa75c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.770+0000 7fa774859700 1 -- 172.21.15.45:0/3577002413 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fa770083ca0 con 0x556d6bb49900 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.770+0000 7fa77705e700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa75c0658c0 0x7fa75c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.770+0000 7fa77de40040 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556d6bace700 0x556d6bb7ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.770+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x556d6bb65590 con 0x556d6bace700 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.770+0000 7fa76ffff700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556d6bace700 0x556d6bb7ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77705e700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa75c0658c0 0x7fa75c067d70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa768004170 tx=0x7fa768000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa76ffff700 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556d6bace700 0x556d6bb7ccb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa76ffff700 1 -- 172.21.15.45:0/3577002413 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fa760022020 con 0x556d6bace700 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556d6bace700 msgr2=0x556d6bb7ccb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556d6bace700 0x556d6bb7ccb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa75c0658c0 msgr2=0x7fa75c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa75c0658c0 0x7fa75c067d70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa768004170 tx=0x7fa768000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bb49900 msgr2=0x556d6bb73bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 --2- 172.21.15.45:0/3577002413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d6bb49900 0x556d6bb73bf0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa77000afd0 tx=0x7fa770007040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77785f700 1 -- 172.21.15.45:0/3577002413 reap_dead start 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 shutdown_connections 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.771+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 >> 172.21.15.45:0/3577002413 conn(0x556d6ba8b5f0 msgr2=0x556d6bb649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.772+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 shutdown_connections 2023-10-06T05:22:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.772+0000 7fa77de40040 1 -- 172.21.15.45:0/3577002413 wait complete. 2023-10-06T05:22:08.845 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2023-10-06T05:22:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.892+0000 7fed9a9b8040 1 Processor -- start 2023-10-06T05:22:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.892+0000 7fed9a9b8040 1 -- start start 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed9a9b8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee564e87e0 0x55ee5656ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed9a9b8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed9a9b8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee56573f00 0x55ee565724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed9a9b8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee563eb620 con 0x55ee56573f00 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed9a9b8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee563e2400 con 0x55ee564e87e0 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed9a9b8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee564a1820 con 0x55ee56573b30 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed943d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee56573f00 0x55ee565724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed933d5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed93bd6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee564e87e0 0x55ee5656ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed933d5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:39940/0 (socket says 172.21.15.45:39940) 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed933d5700 1 -- 172.21.15.45:0/3879641714 learned_addr learned my addr 172.21.15.45:0/3879641714 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.893+0000 7fed943d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee56573f00 0x55ee565724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55120/0 (socket says 172.21.15.45:55120) 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed933d5700 1 -- 172.21.15.45:0/3879641714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee564e87e0 msgr2=0x55ee5656ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed933d5700 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee564e87e0 0x55ee5656ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed933d5700 1 -- 172.21.15.45:0/3879641714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee56573f00 msgr2=0x55ee565724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed933d5700 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee56573f00 0x55ee565724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed933d5700 1 -- 172.21.15.45:0/3879641714 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed88005db0 con 0x55ee56573b30 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed943d7700 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee56573f00 0x55ee565724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed933d5700 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed8800ac90 tx=0x7fed88016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154998c5d073d2b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed923d3700 1 -- 172.21.15.45:0/3879641714 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88005510 con 0x55ee56573b30 2023-10-06T05:22:08.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed923d3700 1 -- 172.21.15.45:0/3879641714 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed88000bc0 con 0x55ee56573b30 2023-10-06T05:22:08.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.894+0000 7fed923d3700 1 -- 172.21.15.45:0/3879641714 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8801d420 con 0x55ee56573b30 2023-10-06T05:22:08.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 -- 172.21.15.45:0/3879641714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 msgr2=0x55ee5656db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed8800ac90 tx=0x7fed88016000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 -- 172.21.15.45:0/3879641714 shutdown_connections 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee56573f00 0x55ee565724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee564e87e0 0x55ee5656ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/3879641714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 -- 172.21.15.45:0/3879641714 >> 172.21.15.45:0/3879641714 conn(0x55ee564a1450 msgr2=0x55ee5656f740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 -- 172.21.15.45:0/3879641714 shutdown_connections 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 -- 172.21.15.45:0/3879641714 wait complete. 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.895+0000 7fed9a9b8040 1 Processor -- start 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed9a9b8040 1 -- start start 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed9a9b8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee564e87e0 0x55ee5655c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed9a9b8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed9a9b8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee56573f00 0x55ee5656e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed9a9b8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee5656c580 con 0x55ee564e87e0 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed9a9b8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee5656c820 con 0x55ee56573f00 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed9a9b8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee5656cac0 con 0x55ee56573b30 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed93bd6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee564e87e0 0x55ee5655c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed93bd6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee564e87e0 0x55ee5655c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55136/0 (socket says 172.21.15.45:55136) 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.896+0000 7fed93bd6700 1 -- 172.21.15.45:0/469387959 learned_addr learned my addr 172.21.15.45:0/469387959 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed933d5700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed943d7700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee56573f00 0x55ee5656e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed93bd6700 1 -- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 msgr2=0x55ee5656db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed93bd6700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee56573b30 0x55ee5656db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed93bd6700 1 -- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee56573f00 msgr2=0x55ee5656e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed93bd6700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee56573f00 0x55ee5656e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed93bd6700 1 -- 172.21.15.45:0/469387959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0024c0 con 0x55ee564e87e0 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed943d7700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee56573f00 0x55ee5656e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed93bd6700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee564e87e0 0x55ee5655c850 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fed84008ac0 tx=0x7fed8400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed90bd0700 1 -- 172.21.15.45:0/469387959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8400f040 con 0x55ee564e87e0 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed90bd0700 1 -- 172.21.15.45:0/469387959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed840108b0 con 0x55ee564e87e0 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed90bd0700 1 -- 172.21.15.45:0/469387959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84010c70 con 0x55ee564e87e0 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ee563ed750 con 0x55ee564e87e0 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.897+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee564141c0 con 0x55ee564e87e0 2023-10-06T05:22:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.899+0000 7fed90bd0700 1 -- 172.21.15.45:0/469387959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed84015660 con 0x55ee564e87e0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.899+0000 7fed90bd0700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed800658c0 0x7fed80067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.899+0000 7fed90bd0700 1 -- 172.21.15.45:0/469387959 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fed84084240 con 0x55ee564e87e0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.899+0000 7fed933d5700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed800658c0 0x7fed80067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ee565753a0 0x55ee5656b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ee5656ba60 con 0x55ee565753a0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed943d7700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ee565753a0 0x55ee5656b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed933d5700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed800658c0 0x7fed80067d70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed8801d5d0 tx=0x7fed8801e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed943d7700 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ee565753a0 0x55ee5656b520 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed943d7700 1 -- 172.21.15.45:0/469387959 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fed8c014420 con 0x55ee565753a0 2023-10-06T05:22:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ee565753a0 msgr2=0x55ee5656b520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.900+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ee565753a0 0x55ee5656b520 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed800658c0 msgr2=0x7fed80067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fed800658c0 0x7fed80067d70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed8801d5d0 tx=0x7fed8801e000 comp rx=0 tx=0).stop 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee564e87e0 msgr2=0x55ee5655c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 --2- 172.21.15.45:0/469387959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee564e87e0 0x55ee5655c850 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fed84008ac0 tx=0x7fed8400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed943d7700 1 -- 172.21.15.45:0/469387959 reap_dead start 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 shutdown_connections 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 >> 172.21.15.45:0/469387959 conn(0x55ee564a1450 msgr2=0x55ee5656f740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 shutdown_connections 2023-10-06T05:22:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.901+0000 7fed9a9b8040 1 -- 172.21.15.45:0/469387959 wait complete. 2023-10-06T05:22:08.908 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2023-10-06T05:22:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.953+0000 7f85c6139040 1 Processor -- start 2023-10-06T05:22:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.953+0000 7f85c6139040 1 -- start start 2023-10-06T05:22:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85c6139040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34859a210 0x55a348679f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85c6139040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34865c890 0x55a3485e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85c6139040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a3485ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85c6139040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3484e7620 con 0x55a34865cc60 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85c6139040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3484de400 con 0x55a34859a210 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85c6139040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3486779e0 con 0x55a34865c890 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a3485ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a3485ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55156/0 (socket says 172.21.15.45:55156) 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 -- 172.21.15.45:0/1751859071 learned_addr learned my addr 172.21.15.45:0/1751859071 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85beb56700 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34865c890 0x55a3485e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bf357700 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34859a210 0x55a348679f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 -- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34865c890 msgr2=0x55a3485e1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34865c890 0x55a3485e1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 -- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34859a210 msgr2=0x55a348679f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34859a210 0x55a348679f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 -- 172.21.15.45:0/1751859071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b800a0e0 con 0x55a34865cc60 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85beb56700 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34865c890 0x55a3485e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bf357700 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34859a210 0x55a348679f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.954+0000 7f85bfb58700 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a3485ca1c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f85b800a930 tx=0x7f85b8007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47792f95ac6291a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85bdb54700 1 -- 172.21.15.45:0/1751859071 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800d960 con 0x55a34865cc60 2023-10-06T05:22:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85bdb54700 1 -- 172.21.15.45:0/1751859071 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800dac0 con 0x55a34865cc60 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85bdb54700 1 -- 172.21.15.45:0/1751859071 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800de80 con 0x55a34865cc60 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 -- 172.21.15.45:0/1751859071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 msgr2=0x55a3485ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a3485ca1c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f85b800a930 tx=0x7f85b8007040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 -- 172.21.15.45:0/1751859071 shutdown_connections 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a3485ca1c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34859a210 0x55a348679f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 --2- 172.21.15.45:0/1751859071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34865c890 0x55a3485e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 -- 172.21.15.45:0/1751859071 >> 172.21.15.45:0/1751859071 conn(0x55a34859d450 msgr2=0x55a34859f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 -- 172.21.15.45:0/1751859071 shutdown_connections 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.955+0000 7f85c6139040 1 -- 172.21.15.45:0/1751859071 wait complete. 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.956+0000 7f85c6139040 1 Processor -- start 2023-10-06T05:22:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.956+0000 7f85c6139040 1 -- start start 2023-10-06T05:22:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85c6139040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34859a210 0x55a34865b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85c6139040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34865c890 0x55a348678d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85c6139040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a348679250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85c6139040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a348683ad0 con 0x55a34865cc60 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85c6139040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a348683d70 con 0x55a34865c890 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85c6139040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a348684010 con 0x55a34859a210 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a348679250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a348679250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55164/0 (socket says 172.21.15.45:55164) 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 -- 172.21.15.45:0/487894044 learned_addr learned my addr 172.21.15.45:0/487894044 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bf357700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34859a210 0x55a34865b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85beb56700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34865c890 0x55a348678d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 -- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34859a210 msgr2=0x55a34865b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a34859a210 0x55a34865b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 -- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34865c890 msgr2=0x55a348678d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a34865c890 0x55a348678d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 -- 172.21.15.45:0/487894044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b800a5b0 con 0x55a34865cc60 2023-10-06T05:22:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.957+0000 7f85bfb58700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a348679250 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f85b800d4b0 tx=0x7f85b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.958+0000 7f85a7fff700 1 -- 172.21.15.45:0/487894044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8009070 con 0x55a34865cc60 2023-10-06T05:22:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.958+0000 7f85a7fff700 1 -- 172.21.15.45:0/487894044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8021070 con 0x55a34865cc60 2023-10-06T05:22:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.958+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b0005cf0 con 0x55a34865cc60 2023-10-06T05:22:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.958+0000 7f85a7fff700 1 -- 172.21.15.45:0/487894044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b801c660 con 0x55a34865cc60 2023-10-06T05:22:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.958+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b4005db0 con 0x55a34865cc60 2023-10-06T05:22:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.959+0000 7f85a7fff700 1 -- 172.21.15.45:0/487894044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85b8023030 con 0x55a34865cc60 2023-10-06T05:22:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.960+0000 7f85a7fff700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f85ac065890 0x7f85ac067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.960+0000 7f85a7fff700 1 -- 172.21.15.45:0/487894044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f85b80858d0 con 0x55a34865cc60 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.960+0000 7f85c6139040 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a34865bb50 0x55a34868ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.960+0000 7f85bf357700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f85ac065890 0x7f85ac067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.960+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.6 14:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55a34868f3b0 con 0x55a34865bb50 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.960+0000 7f85beb56700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a34865bb50 0x55a34868ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.960+0000 7f85bf357700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f85ac065890 0x7f85ac067d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009dc0 tx=0x7f85b000c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85beb56700 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a34865bb50 0x55a34868ee70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85beb56700 1 -- 172.21.15.45:0/487894044 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+5 (crc 0 0 0) 0x7f85b4008970 con 0x55a34865bb50 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a34865bb50 msgr2=0x55a34868ee70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a34865bb50 0x55a34868ee70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f85ac065890 msgr2=0x7f85ac067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f85ac065890 0x7f85ac067d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009dc0 tx=0x7f85b000c040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 msgr2=0x55a348679250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 --2- 172.21.15.45:0/487894044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a34865cc60 0x55a348679250 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f85b800d4b0 tx=0x7f85b8019000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85bfb58700 1 -- 172.21.15.45:0/487894044 reap_dead start 2023-10-06T05:22:09.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 shutdown_connections 2023-10-06T05:22:09.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 >> 172.21.15.45:0/487894044 conn(0x55a34859d450 msgr2=0x55a34859f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 shutdown_connections 2023-10-06T05:22:09.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:08.961+0000 7f85c6139040 1 -- 172.21.15.45:0/487894044 wait complete. 2023-10-06T05:22:09.044 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2023-10-06T05:22:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.092+0000 7f69d08e0040 1 Processor -- start 2023-10-06T05:22:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.092+0000 7f69d08e0040 1 -- start start 2023-10-06T05:22:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69d08e0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32b0ca0 0x5585a33586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69d08e0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32cc7e0 0x5585a32b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69d08e0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a3367c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69d08e0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a322b320 con 0x5585a32cc7e0 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69d08e0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a31cf980 con 0x5585a32b0ca0 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69d08e0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585a31cf620 con 0x5585a3357b30 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69c92fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32cc7e0 0x5585a32b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69c92fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32cc7e0 0x5585a32b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55184/0 (socket says 172.21.15.45:55184) 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69c92fd700 1 -- 172.21.15.45:0/1828611793 learned_addr learned my addr 172.21.15.45:0/1828611793 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69c9afe700 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32b0ca0 0x5585a33586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.093+0000 7f69ca2ff700 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a3367c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c92fd700 1 -- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 msgr2=0x5585a3367c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c92fd700 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a3367c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c92fd700 1 -- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32b0ca0 msgr2=0x5585a33586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c92fd700 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32b0ca0 0x5585a33586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c92fd700 1 -- 172.21.15.45:0/1828611793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c400c370 con 0x5585a32cc7e0 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c92fd700 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32cc7e0 0x5585a32b2950 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001e20 tx=0x7f69b40018c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1c665bbc970b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c3fff700 1 -- 172.21.15.45:0/1828611793 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4014070 con 0x5585a32cc7e0 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c3fff700 1 -- 172.21.15.45:0/1828611793 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4008030 con 0x5585a32cc7e0 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.094+0000 7f69c3fff700 1 -- 172.21.15.45:0/1828611793 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b400bd70 con 0x5585a32cc7e0 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 -- 172.21.15.45:0/1828611793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32cc7e0 msgr2=0x5585a32b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32cc7e0 0x5585a32b2950 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001e20 tx=0x7f69b40018c0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 -- 172.21.15.45:0/1828611793 shutdown_connections 2023-10-06T05:22:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a3367c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32b0ca0 0x5585a33586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1828611793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32cc7e0 0x5585a32b2950 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 -- 172.21.15.45:0/1828611793 >> 172.21.15.45:0/1828611793 conn(0x5585a3285450 msgr2=0x5585a3357f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 -- 172.21.15.45:0/1828611793 shutdown_connections 2023-10-06T05:22:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.095+0000 7f69d08e0040 1 -- 172.21.15.45:0/1828611793 wait complete. 2023-10-06T05:22:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.096+0000 7f69d08e0040 1 Processor -- start 2023-10-06T05:22:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.096+0000 7f69d08e0040 1 -- start start 2023-10-06T05:22:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69d08e0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32b0ca0 0x5585a3351b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69d08e0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32cc7e0 0x5585a3367e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69d08e0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a334d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69d08e0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a31c6400 con 0x5585a32b0ca0 2023-10-06T05:22:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69d08e0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a32c9520 con 0x5585a32cc7e0 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69d08e0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585a334e280 con 0x5585a3357b30 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32b0ca0 0x5585a3351b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32b0ca0 0x5585a3351b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55192/0 (socket says 172.21.15.45:55192) 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 -- 172.21.15.45:0/1091333503 learned_addr learned my addr 172.21.15.45:0/1091333503 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69ca2ff700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a334d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c92fd700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32cc7e0 0x5585a3367e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 -- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 msgr2=0x5585a334d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a334d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 -- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32cc7e0 msgr2=0x5585a3367e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32cc7e0 0x5585a3367e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 -- 172.21.15.45:0/1091333503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b400a040 con 0x5585a32b0ca0 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69ca2ff700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5585a3357b30 0x5585a334d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c92fd700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585a32cc7e0 0x5585a3367e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.097+0000 7f69c9afe700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32b0ca0 0x5585a3351b20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c6f0 tx=0x7f69bc006840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.098+0000 7f69c27fc700 1 -- 172.21.15.45:0/1091333503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00f450 con 0x5585a32b0ca0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.098+0000 7f69c27fc700 1 -- 172.21.15.45:0/1091333503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc00f5b0 con 0x5585a32b0ca0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.098+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5585a31d1ae0 con 0x5585a32b0ca0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.098+0000 7f69c27fc700 1 -- 172.21.15.45:0/1091333503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00f970 con 0x5585a32b0ca0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.098+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585a31d1540 con 0x5585a32b0ca0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.099+0000 7f69c27fc700 1 -- 172.21.15.45:0/1091333503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69bc00fce0 con 0x5585a32b0ca0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.100+0000 7f69c27fc700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69ac0658c0 0x7f69ac067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.100+0000 7f69c27fc700 1 -- 172.21.15.45:0/1091333503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f69bc083c60 con 0x5585a32b0ca0 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.100+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5585a3368340 0x5585a334f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.100+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.15 14:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x5585a33587a0 con 0x5585a3368340 2023-10-06T05:22:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.100+0000 7f69c92fd700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69ac0658c0 0x7f69ac067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.100+0000 7f69ca2ff700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5585a3368340 0x5585a334f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69c92fd700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69ac0658c0 0x7f69ac067d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001920 tx=0x7f69b4014000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69ca2ff700 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5585a3368340 0x5585a334f100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69ca2ff700 1 -- 172.21.15.45:0/1091333503 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7f69c400ad30 con 0x5585a3368340 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5585a3368340 msgr2=0x5585a334f100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5585a3368340 0x5585a334f100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69ac0658c0 msgr2=0x7f69ac067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69ac0658c0 0x7f69ac067d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001920 tx=0x7f69b4014000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32b0ca0 msgr2=0x5585a3351b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 --2- 172.21.15.45:0/1091333503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585a32b0ca0 0x5585a3351b20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c6f0 tx=0x7f69bc006840 comp rx=0 tx=0).stop 2023-10-06T05:22:09.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69ca2ff700 1 -- 172.21.15.45:0/1091333503 reap_dead start 2023-10-06T05:22:09.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 shutdown_connections 2023-10-06T05:22:09.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 >> 172.21.15.45:0/1091333503 conn(0x5585a3285450 msgr2=0x5585a3357f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 shutdown_connections 2023-10-06T05:22:09.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.101+0000 7f69d08e0040 1 -- 172.21.15.45:0/1091333503 wait complete. 2023-10-06T05:22:09.109 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2023-10-06T05:22:09.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.154+0000 7f4634978040 1 Processor -- start 2023-10-06T05:22:09.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.154+0000 7f4634978040 1 -- start start 2023-10-06T05:22:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f4634978040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd07a7e0 0x55d6bd0624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f4634978040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd105b30 0x55d6bd060f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f4634978040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105f00 0x55d6bd104960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f4634978040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bcf7d980 con 0x55d6bd07a7e0 2023-10-06T05:22:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f4634978040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bcf7d620 con 0x55d6bd105b30 2023-10-06T05:22:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f4634978040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6bcf74400 con 0x55d6bd105f00 2023-10-06T05:22:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f462db96700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd07a7e0 0x55d6bd0624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f462db96700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd07a7e0 0x55d6bd0624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55204/0 (socket says 172.21.15.45:55204) 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f462db96700 1 -- 172.21.15.45:0/571270293 learned_addr learned my addr 172.21.15.45:0/571270293 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.155+0000 7f462d395700 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd105b30 0x55d6bd060f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462e397700 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105f00 0x55d6bd104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462db96700 1 -- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105f00 msgr2=0x55d6bd104960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462db96700 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105f00 0x55d6bd104960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462db96700 1 -- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd105b30 msgr2=0x55d6bd060f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462db96700 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd105b30 0x55d6bd060f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462db96700 1 -- 172.21.15.45:0/571270293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618005cf0 con 0x55d6bd07a7e0 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462d395700 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd105b30 0x55d6bd060f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462e397700 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105f00 0x55d6bd104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f462db96700 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd07a7e0 0x55d6bd0624f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4618009aa0 tx=0x7f461800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bac243f766b7bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f4627fff700 1 -- 172.21.15.45:0/571270293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800f040 con 0x55d6bd07a7e0 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f4627fff700 1 -- 172.21.15.45:0/571270293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618008af0 con 0x55d6bd07a7e0 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.156+0000 7f4627fff700 1 -- 172.21.15.45:0/571270293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618008e90 con 0x55d6bd07a7e0 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 -- 172.21.15.45:0/571270293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd07a7e0 msgr2=0x55d6bd0624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd07a7e0 0x55d6bd0624f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4618009aa0 tx=0x7f461800c040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 -- 172.21.15.45:0/571270293 shutdown_connections 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105f00 0x55d6bd104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd07a7e0 0x55d6bd0624f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 --2- 172.21.15.45:0/571270293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd105b30 0x55d6bd060f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 -- 172.21.15.45:0/571270293 >> 172.21.15.45:0/571270293 conn(0x55d6bd033450 msgr2=0x55d6bd061760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 -- 172.21.15.45:0/571270293 shutdown_connections 2023-10-06T05:22:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.157+0000 7f4634978040 1 -- 172.21.15.45:0/571270293 wait complete. 2023-10-06T05:22:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 Processor -- start 2023-10-06T05:22:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 -- start start 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd07a7e0 0x55d6bd074470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105b30 0x55d6bd0fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd105f00 0x55d6bd102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bd075c90 con 0x55d6bd105f00 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bd072140 con 0x55d6bd07a7e0 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f4634978040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6bd0723e0 con 0x55d6bd105b30 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.158+0000 7f462e397700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd105f00 0x55d6bd102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd105f00 0x55d6bd102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55212/0 (socket says 172.21.15.45:55212) 2023-10-06T05:22:09.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 -- 172.21.15.45:0/1546939660 learned_addr learned my addr 172.21.15.45:0/1546939660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462d395700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105b30 0x55d6bd0fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462db96700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd07a7e0 0x55d6bd074470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 -- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105b30 msgr2=0x55d6bd0fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d6bd105b30 0x55d6bd0fa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 -- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd07a7e0 msgr2=0x55d6bd074470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6bd07a7e0 0x55d6bd074470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 -- 172.21.15.45:0/1546939660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6bcfa61c0 con 0x55d6bd105f00 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f462e397700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd105f00 0x55d6bd102060 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f46280016b0 tx=0x7f46280068b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f46267fc700 1 -- 172.21.15.45:0/1546939660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46280123b0 con 0x55d6bd105f00 2023-10-06T05:22:09.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f46267fc700 1 -- 172.21.15.45:0/1546939660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4628012510 con 0x55d6bd105f00 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4628006b90 con 0x55d6bd105f00 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f46267fc700 1 -- 172.21.15.45:0/1546939660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628012890 con 0x55d6bd105f00 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.159+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4618009670 con 0x55d6bd105f00 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.161+0000 7f46267fc700 1 -- 172.21.15.45:0/1546939660 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4628012c80 con 0x55d6bd105f00 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.161+0000 7f46267fc700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4610065890 0x7f4610067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.161+0000 7f46267fc700 1 -- 172.21.15.45:0/1546939660 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f4628082ef0 con 0x55d6bd105f00 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.161+0000 7f462db96700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4610065890 0x7f4610067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.162+0000 7f4634978040 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d6bd0351b0 0x55d6bd0faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.162+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55d6bd0fb450 con 0x55d6bd0351b0 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.162+0000 7f462d395700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d6bd0351b0 0x55d6bd0faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.162+0000 7f462db96700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4610065890 0x7f4610067d40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4618009640 tx=0x7f4618017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.162+0000 7f462d395700 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d6bd0351b0 0x55d6bd0faf10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f462d395700 1 -- 172.21.15.45:0/1546939660 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7f4620008970 con 0x55d6bd0351b0 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d6bd0351b0 msgr2=0x55d6bd0faf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d6bd0351b0 0x55d6bd0faf10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4610065890 msgr2=0x7f4610067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4610065890 0x7f4610067d40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4618009640 tx=0x7f4618017000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd105f00 msgr2=0x55d6bd102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 --2- 172.21.15.45:0/1546939660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6bd105f00 0x55d6bd102060 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f46280016b0 tx=0x7f46280068b0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f462e397700 1 -- 172.21.15.45:0/1546939660 reap_dead start 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 shutdown_connections 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 >> 172.21.15.45:0/1546939660 conn(0x55d6bd033450 msgr2=0x55d6bd0fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 shutdown_connections 2023-10-06T05:22:09.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.163+0000 7f4634978040 1 -- 172.21.15.45:0/1546939660 wait complete. 2023-10-06T05:22:09.169 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2023-10-06T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.233+0000 7f5c1c8da040 1 Processor -- start 2023-10-06T05:22:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.233+0000 7f5c1c8da040 1 -- start start 2023-10-06T05:22:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c1c8da040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c75700920 0x559c757e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c1c8da040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 0x559c757e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c1c8da040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c757c2c60 0x559c757d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c1c8da040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c756a9320 con 0x559c75700920 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c1c8da040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7564d980 con 0x559c757c2c60 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c1c8da040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c7564d620 con 0x559c757c2890 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c75700920 0x559c757e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c75700920 0x559c757e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55220/0 (socket says 172.21.15.45:55220) 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 -- 172.21.15.45:0/762599277 learned_addr learned my addr 172.21.15.45:0/762599277 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c162f9700 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c757c2c60 0x559c757d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c152f7700 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 0x559c757e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 -- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 msgr2=0x559c757e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 0x559c757e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 -- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c757c2c60 msgr2=0x559c757d9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c757c2c60 0x559c757d9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 -- 172.21.15.45:0/762599277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1000a0e0 con 0x559c75700920 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c162f9700 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c757c2c60 0x559c757d9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c15af8700 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c75700920 0x559c757e6050 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5c08001d70 tx=0x7f5c08001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eba36106b7fdc81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.234+0000 7f5c07fff700 1 -- 172.21.15.45:0/762599277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800bbb0 con 0x559c75700920 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c07fff700 1 -- 172.21.15.45:0/762599277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800bd30 con 0x559c75700920 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c07fff700 1 -- 172.21.15.45:0/762599277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800bea0 con 0x559c75700920 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 -- 172.21.15.45:0/762599277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c75700920 msgr2=0x559c757e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c75700920 0x559c757e6050 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f5c08001d70 tx=0x7f5c08001820 comp rx=0 tx=0).stop 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 -- 172.21.15.45:0/762599277 shutdown_connections 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c757c2c60 0x559c757d9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c75700920 0x559c757e6050 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/762599277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 0x559c757e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 -- 172.21.15.45:0/762599277 >> 172.21.15.45:0/762599277 conn(0x559c75703450 msgr2=0x559c75730f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 -- 172.21.15.45:0/762599277 shutdown_connections 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.235+0000 7f5c1c8da040 1 -- 172.21.15.45:0/762599277 wait complete. 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.236+0000 7f5c1c8da040 1 Processor -- start 2023-10-06T05:22:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.236+0000 7f5c1c8da040 1 -- start start 2023-10-06T05:22:09.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c1c8da040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c75700920 0x559c75705360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c1c8da040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 0x559c757e6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c1c8da040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c757c2c60 0x559c757c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c1c8da040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c75644400 con 0x559c757c2c60 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c1c8da040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c757ddba0 con 0x559c75700920 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c1c8da040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c757dde40 con 0x559c757c2890 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c162f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c757c2c60 0x559c757c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c15af8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c75700920 0x559c75705360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c162f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c757c2c60 0x559c757c0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55236/0 (socket says 172.21.15.45:55236) 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c15af8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c75700920 0x559c75705360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:58868/0 (socket says 172.21.15.45:58868) 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c162f9700 1 -- 172.21.15.45:0/244914362 learned_addr learned my addr 172.21.15.45:0/244914362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.237+0000 7f5c152f7700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 0x559c757e6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c162f9700 1 -- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 msgr2=0x559c757e6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c162f9700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559c757c2890 0x559c757e6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c162f9700 1 -- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c75700920 msgr2=0x559c75705360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c162f9700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c75700920 0x559c75705360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c162f9700 1 -- 172.21.15.45:0/244914362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c00001e10 con 0x559c757c2c60 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c162f9700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c757c2c60 0x559c757c0a70 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x559c75706c20 tx=0x7f5c10015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c067fc700 1 -- 172.21.15.45:0/244914362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100032f0 con 0x559c757c2c60 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c067fc700 1 -- 172.21.15.45:0/244914362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1000f860 con 0x559c757c2c60 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559c7564fae0 con 0x559c757c2c60 2023-10-06T05:22:09.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c067fc700 1 -- 172.21.15.45:0/244914362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10010650 con 0x559c757c2c60 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.238+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c7564f540 con 0x559c757c2c60 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.239+0000 7f5c067fc700 1 -- 172.21.15.45:0/244914362 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c10003040 con 0x559c757c2c60 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.240+0000 7f5c067fc700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bf80658c0 0x7f5bf8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.240+0000 7f5c067fc700 1 -- 172.21.15.45:0/244914362 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5c100852a0 con 0x559c757c2c60 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.240+0000 7f5c15af8700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bf80658c0 0x7f5bf8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.240+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559c757c0fb0 0x559c757dba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.240+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x559c757312f0 con 0x559c757c0fb0 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.240+0000 7f5c152f7700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559c757c0fb0 0x559c757dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c15af8700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bf80658c0 0x7f5bf8067d70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000c00 tx=0x7f5c08016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c152f7700 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559c757c0fb0 0x559c757dba90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c152f7700 1 -- 172.21.15.45:0/244914362 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+5 (crc 0 0 0) 0x7f5c0c0106e0 con 0x559c757c0fb0 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559c757c0fb0 msgr2=0x559c757dba90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559c757c0fb0 0x559c757dba90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bf80658c0 msgr2=0x7f5bf8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bf80658c0 0x7f5bf8067d70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000c00 tx=0x7f5c08016000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c757c2c60 msgr2=0x559c757c0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 --2- 172.21.15.45:0/244914362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c757c2c60 0x559c757c0a70 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x559c75706c20 tx=0x7f5c10015000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c162f9700 1 -- 172.21.15.45:0/244914362 reap_dead start 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 shutdown_connections 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 >> 172.21.15.45:0/244914362 conn(0x559c75703450 msgr2=0x559c757e6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 shutdown_connections 2023-10-06T05:22:09.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.241+0000 7f5c1c8da040 1 -- 172.21.15.45:0/244914362 wait complete. 2023-10-06T05:22:09.250 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2023-10-06T05:22:09.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.298+0000 7f5544711040 1 Processor -- start 2023-10-06T05:22:09.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.298+0000 7f5544711040 1 -- start start 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f5544711040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a61fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f5544711040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627939c1450 0x5627939d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f5544711040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562793a67fc0 0x562793a66b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f5544711040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627938dc980 con 0x5627939c1450 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f5544711040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627938dc620 con 0x562793a67fc0 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f5544711040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627938d3400 con 0x5627939bd2f0 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f553d12e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627939c1450 0x5627939d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f553d12e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627939c1450 0x5627939d7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55258/0 (socket says 172.21.15.45:55258) 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f553d12e700 1 -- 172.21.15.45:0/1975211604 learned_addr learned my addr 172.21.15.45:0/1975211604 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f553d92f700 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a61fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f553e130700 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562793a67fc0 0x562793a66b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f553d12e700 1 -- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 msgr2=0x562793a61fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.299+0000 7f553d12e700 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a61fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f553d12e700 1 -- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562793a67fc0 msgr2=0x562793a66b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f553d12e700 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562793a67fc0 0x562793a66b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f553d12e700 1 -- 172.21.15.45:0/1975211604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534005cf0 con 0x5627939c1450 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f553e130700 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562793a67fc0 0x562793a66b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f553d92f700 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a61fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f553d12e700 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627939c1450 0x5627939d7250 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x5627939953a0 tx=0x7f5534016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3713c1950a18e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f552ffff700 1 -- 172.21.15.45:0/1975211604 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534000bc0 con 0x5627939c1450 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f552ffff700 1 -- 172.21.15.45:0/1975211604 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55340089d0 con 0x5627939c1450 2023-10-06T05:22:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f552ffff700 1 -- 172.21.15.45:0/1975211604 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534000d20 con 0x5627939c1450 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 -- 172.21.15.45:0/1975211604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627939c1450 msgr2=0x5627939d7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627939c1450 0x5627939d7250 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x5627939953a0 tx=0x7f5534016000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 -- 172.21.15.45:0/1975211604 shutdown_connections 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562793a67fc0 0x562793a66b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a61fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 --2- 172.21.15.45:0/1975211604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5627939c1450 0x5627939d7250 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 -- 172.21.15.45:0/1975211604 >> 172.21.15.45:0/1975211604 conn(0x562793991bd0 msgr2=0x562793a66400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.300+0000 7f5544711040 1 -- 172.21.15.45:0/1975211604 shutdown_connections 2023-10-06T05:22:09.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.301+0000 7f5544711040 1 -- 172.21.15.45:0/1975211604 wait complete. 2023-10-06T05:22:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.301+0000 7f5544711040 1 Processor -- start 2023-10-06T05:22:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.301+0000 7f5544711040 1 -- start start 2023-10-06T05:22:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f5544711040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a5f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f5544711040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627939c1450 0x562793a594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f5544711040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562793a67fc0 0x562793a609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f5544711040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562793992f30 con 0x562793a67fc0 2023-10-06T05:22:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f5544711040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627939931d0 con 0x5627939c1450 2023-10-06T05:22:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f5544711040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562793993470 con 0x5627939bd2f0 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562793a67fc0 0x562793a609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562793a67fc0 0x562793a609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55262/0 (socket says 172.21.15.45:55262) 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 -- 172.21.15.45:0/1954546104 learned_addr learned my addr 172.21.15.45:0/1954546104 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553d12e700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627939c1450 0x562793a594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553d92f700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a5f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 -- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 msgr2=0x562793a5f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a5f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 -- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627939c1450 msgr2=0x562793a594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627939c1450 0x562793a594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553e130700 1 -- 172.21.15.45:0/1954546104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627939051c0 con 0x562793a67fc0 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553d92f700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5627939bd2f0 0x562793a5f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.302+0000 7f553d12e700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627939c1450 0x562793a594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.303+0000 7f553e130700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562793a67fc0 0x562793a609f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f553800e560 tx=0x7f5538007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.303+0000 7f552e7fc700 1 -- 172.21.15.45:0/1954546104 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553800ce80 con 0x562793a67fc0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.303+0000 7f552e7fc700 1 -- 172.21.15.45:0/1954546104 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5538013670 con 0x562793a67fc0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.303+0000 7f552e7fc700 1 -- 172.21.15.45:0/1954546104 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538013a10 con 0x562793a67fc0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.303+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627938eb3d0 con 0x562793a67fc0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.303+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627938deae0 con 0x562793a67fc0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.304+0000 7f552e7fc700 1 -- 172.21.15.45:0/1954546104 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5538014410 con 0x562793a67fc0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.305+0000 7f552e7fc700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5520065890 0x7f5520067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.305+0000 7f552e7fc700 1 -- 172.21.15.45:0/1954546104 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5538083f10 con 0x562793a67fc0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.305+0000 7f5544711040 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x562793a695c0 0x562793a699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.305+0000 7f553d92f700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5520065890 0x7f5520067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.305+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x562793a69f00 con 0x562793a695c0 2023-10-06T05:22:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.305+0000 7f553d12e700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x562793a695c0 0x562793a699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f553d12e700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x562793a695c0 0x562793a699c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f553d92f700 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5520065890 0x7f5520067d40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f553d12e700 1 -- 172.21.15.45:0/1954546104 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+155 (crc 0 0 0) 0x7f5534000bc0 con 0x562793a695c0 2023-10-06T05:22:09.313 INFO:teuthology.orchestra.run.smithi045.stdout:stray0_head 2023-10-06T05:22:09.313 INFO:teuthology.orchestra.run.smithi045.stdout:stray1_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray2_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray3_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray4_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray5_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray6_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray7_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray8_head 2023-10-06T05:22:09.314 INFO:teuthology.orchestra.run.smithi045.stdout:stray9_head 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x562793a695c0 msgr2=0x562793a699c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x562793a695c0 0x562793a699c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5520065890 msgr2=0x7f5520067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5520065890 0x7f5520067d40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562793a67fc0 msgr2=0x562793a609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 --2- 172.21.15.45:0/1954546104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562793a67fc0 0x562793a609f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f553800e560 tx=0x7f5538007040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f553e130700 1 -- 172.21.15.45:0/1954546104 reap_dead start 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 shutdown_connections 2023-10-06T05:22:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 >> 172.21.15.45:0/1954546104 conn(0x562793991bd0 msgr2=0x5627939d7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 shutdown_connections 2023-10-06T05:22:09.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.306+0000 7f5544711040 1 -- 172.21.15.45:0/1954546104 wait complete. 2023-10-06T05:22:09.317 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2023-10-06T05:22:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.361+0000 7f66c3aed040 1 Processor -- start 2023-10-06T05:22:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.361+0000 7f66c3aed040 1 -- start start 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66c3aed040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c6cf7e0 0x55958c759180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66c3aed040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c75ab30 0x55958c6b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66c3aed040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 0x55958c6b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66c3aed040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958c5d2980 con 0x55958c6cf7e0 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66c3aed040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958c5d2620 con 0x55958c75ab30 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66c3aed040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55958c5c9400 con 0x55958c75af00 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66bcd0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c6cf7e0 0x55958c759180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66bcd0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c6cf7e0 0x55958c759180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55292/0 (socket says 172.21.15.45:55292) 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66bcd0b700 1 -- 172.21.15.45:0/2843550658 learned_addr learned my addr 172.21.15.45:0/2843550658 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66bd50c700 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 0x55958c6b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.362+0000 7f66b7fff700 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c75ab30 0x55958c6b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66bcd0b700 1 -- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 msgr2=0x55958c6b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66bcd0b700 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 0x55958c6b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66bcd0b700 1 -- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c75ab30 msgr2=0x55958c6b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66bcd0b700 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c75ab30 0x55958c6b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66bcd0b700 1 -- 172.21.15.45:0/2843550658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b800a180 con 0x55958c6cf7e0 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66b7fff700 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c75ab30 0x55958c6b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66bcd0b700 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c6cf7e0 0x55958c759180 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009860 tx=0x7f66b0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff485fe854038ab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66b6ffd700 1 -- 172.21.15.45:0/2843550658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000e070 con 0x55958c6cf7e0 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66b6ffd700 1 -- 172.21.15.45:0/2843550658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b00065c0 con 0x55958c6cf7e0 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66b6ffd700 1 -- 172.21.15.45:0/2843550658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0006960 con 0x55958c6cf7e0 2023-10-06T05:22:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66c3aed040 1 -- 172.21.15.45:0/2843550658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c6cf7e0 msgr2=0x55958c759180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.363+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c6cf7e0 0x55958c759180 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009860 tx=0x7f66b0005c00 comp rx=0 tx=0).stop 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 -- 172.21.15.45:0/2843550658 shutdown_connections 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 0x55958c6b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c6cf7e0 0x55958c759180 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2843550658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c75ab30 0x55958c6b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 -- 172.21.15.45:0/2843550658 >> 172.21.15.45:0/2843550658 conn(0x55958c688450 msgr2=0x55958c759cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 -- 172.21.15.45:0/2843550658 shutdown_connections 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 -- 172.21.15.45:0/2843550658 wait complete. 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 Processor -- start 2023-10-06T05:22:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.364+0000 7f66c3aed040 1 -- start start 2023-10-06T05:22:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66c3aed040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c6cf7e0 0x55958c6b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66c3aed040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c75ab30 0x55958c6aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66c3aed040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 0x55958c754960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66c3aed040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958c6b71b0 con 0x55958c75ab30 2023-10-06T05:22:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66c3aed040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958c6a9a20 con 0x55958c6cf7e0 2023-10-06T05:22:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66c3aed040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55958c6b6560 con 0x55958c75af00 2023-10-06T05:22:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c75ab30 0x55958c6aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c75ab30 0x55958c6aa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55304/0 (socket says 172.21.15.45:55304) 2023-10-06T05:22:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66b7fff700 1 -- 172.21.15.45:0/2203970162 learned_addr learned my addr 172.21.15.45:0/2203970162 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66bd50c700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 0x55958c754960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.365+0000 7f66bcd0b700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c6cf7e0 0x55958c6b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b7fff700 1 -- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 msgr2=0x55958c754960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b7fff700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55958c75af00 0x55958c754960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b7fff700 1 -- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c6cf7e0 msgr2=0x55958c6b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b7fff700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55958c6cf7e0 0x55958c6b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b7fff700 1 -- 172.21.15.45:0/2203970162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b0009430 con 0x55958c75ab30 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b7fff700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c75ab30 0x55958c6aa150 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b2e0 tx=0x7f66a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b57fa700 1 -- 172.21.15.45:0/2203970162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800bb40 con 0x55958c75ab30 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b57fa700 1 -- 172.21.15.45:0/2203970162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a800bca0 con 0x55958c75ab30 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55958c5e13d0 con 0x55958c75ab30 2023-10-06T05:22:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66b57fa700 1 -- 172.21.15.45:0/2203970162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8010680 con 0x55958c75ab30 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.366+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55958c5d4ae0 con 0x55958c75ab30 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.367+0000 7f66b57fa700 1 -- 172.21.15.45:0/2203970162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66a8010a70 con 0x55958c75ab30 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.368+0000 7f66b57fa700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66a00658c0 0x7f66a0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.368+0000 7f66b57fa700 1 -- 172.21.15.45:0/2203970162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f66a8008030 con 0x55958c75ab30 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.368+0000 7f66bcd0b700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66a00658c0 0x7f66a0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.368+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55958c76e4d0 0x55958c76e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.368+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55958c76ede0 con 0x55958c76e4d0 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.368+0000 7f66bd50c700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55958c76e4d0 0x55958c76e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66bcd0b700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66a00658c0 0x7f66a0067d70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005c40 tx=0x7f66b0015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66bd50c700 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55958c76e4d0 0x55958c76e8a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66bd50c700 1 -- 172.21.15.45:0/2203970162 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) v8 ==== 159+0+5 (crc 0 0 0) 0x7f66b800e7e0 con 0x55958c76e4d0 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55958c76e4d0 msgr2=0x55958c76e8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55958c76e4d0 0x55958c76e8a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66a00658c0 msgr2=0x7f66a0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66a00658c0 0x7f66a0067d70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005c40 tx=0x7f66b0015000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c75ab30 msgr2=0x55958c6aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 --2- 172.21.15.45:0/2203970162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55958c75ab30 0x55958c6aa150 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b2e0 tx=0x7f66a8000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66bd50c700 1 -- 172.21.15.45:0/2203970162 reap_dead start 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 shutdown_connections 2023-10-06T05:22:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 >> 172.21.15.45:0/2203970162 conn(0x55958c688450 msgr2=0x55958c759110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 shutdown_connections 2023-10-06T05:22:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.369+0000 7f66c3aed040 1 -- 172.21.15.45:0/2203970162 wait complete. 2023-10-06T05:22:09.378 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2023-10-06T05:22:09.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.422+0000 7fc785265040 1 Processor -- start 2023-10-06T05:22:09.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc785265040 1 -- start start 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc785265040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890991b0 0x563389173b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc785265040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890de5f0 0x5633890e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc785265040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 0x5633890ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc785265040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388fe5980 con 0x5633890991b0 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc785265040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388fe5620 con 0x5633890de5f0 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc785265040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563388fdc400 con 0x56338915a970 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.423+0000 7fc77e483700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890991b0 0x563389173b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890991b0 0x563389173b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55324/0 (socket says 172.21.15.45:55324) 2023-10-06T05:22:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 -- 172.21.15.45:0/2271850964 learned_addr learned my addr 172.21.15.45:0/2271850964 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77ec84700 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 0x5633890ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77dc82700 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890de5f0 0x5633890e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 -- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 msgr2=0x5633890ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 0x5633890ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 -- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890de5f0 msgr2=0x5633890e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890de5f0 0x5633890e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 -- 172.21.15.45:0/2271850964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768005db0 con 0x5633890991b0 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77dc82700 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890de5f0 0x5633890e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77e483700 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890991b0 0x563389173b00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc768005b50 tx=0x7fc768015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4632e3bc5568582 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77cc80700 1 -- 172.21.15.45:0/2271850964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768008a30 con 0x5633890991b0 2023-10-06T05:22:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77cc80700 1 -- 172.21.15.45:0/2271850964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc768008bb0 con 0x5633890991b0 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.424+0000 7fc77cc80700 1 -- 172.21.15.45:0/2271850964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680126a0 con 0x5633890991b0 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 -- 172.21.15.45:0/2271850964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890991b0 msgr2=0x563389173b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890991b0 0x563389173b00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc768005b50 tx=0x7fc768015000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 -- 172.21.15.45:0/2271850964 shutdown_connections 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 0x5633890ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890991b0 0x563389173b00 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 --2- 172.21.15.45:0/2271850964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890de5f0 0x5633890e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 -- 172.21.15.45:0/2271850964 >> 172.21.15.45:0/2271850964 conn(0x56338909b4a0 msgr2=0x56338909b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 -- 172.21.15.45:0/2271850964 shutdown_connections 2023-10-06T05:22:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.425+0000 7fc785265040 1 -- 172.21.15.45:0/2271850964 wait complete. 2023-10-06T05:22:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.426+0000 7fc785265040 1 Processor -- start 2023-10-06T05:22:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.426+0000 7fc785265040 1 -- start start 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc785265040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890991b0 0x563389171660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc785265040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890de5f0 0x563389171ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc785265040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 0x563389181c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc785265040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633891816e0 con 0x5633890de5f0 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc785265040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563389184df0 con 0x5633890991b0 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc785265040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563389185020 con 0x56338915a970 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890de5f0 0x563389171ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890de5f0 0x563389171ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55340/0 (socket says 172.21.15.45:55340) 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 -- 172.21.15.45:0/784907144 learned_addr learned my addr 172.21.15.45:0/784907144 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77ec84700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 0x563389181c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77e483700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890991b0 0x563389171660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 -- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 msgr2=0x563389181c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56338915a970 0x563389181c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 -- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890991b0 msgr2=0x563389171660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633890991b0 0x563389171660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.427+0000 7fc77dc82700 1 -- 172.21.15.45:0/784907144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338900e1c0 con 0x5633890de5f0 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.428+0000 7fc77dc82700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890de5f0 0x563389171ba0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc77000a280 tx=0x7fc770002970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.428+0000 7fc776ffd700 1 -- 172.21.15.45:0/784907144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000adc0 con 0x5633890de5f0 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.428+0000 7fc776ffd700 1 -- 172.21.15.45:0/784907144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc770008040 con 0x5633890de5f0 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.428+0000 7fc776ffd700 1 -- 172.21.15.45:0/784907144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700106c0 con 0x5633890de5f0 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.428+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563388ff43d0 con 0x5633890de5f0 2023-10-06T05:22:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.428+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563388fe7ae0 con 0x5633890de5f0 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.429+0000 7fc776ffd700 1 -- 172.21.15.45:0/784907144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc770010ab0 con 0x5633890de5f0 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.430+0000 7fc776ffd700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc760065890 0x7fc760067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.430+0000 7fc776ffd700 1 -- 172.21.15.45:0/784907144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc7700833f0 con 0x5633890de5f0 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.430+0000 7fc77e483700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc760065890 0x7fc760067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.430+0000 7fc785265040 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56338916d5f0 0x56338916d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.430+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x56338916df30 con 0x56338916d5f0 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.430+0000 7fc77ec84700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56338916d5f0 0x56338916d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.430+0000 7fc77e483700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc760065890 0x7fc760067d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc768020000 tx=0x7fc76801c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc77ec84700 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56338916d5f0 0x56338916d9f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc77ec84700 1 -- 172.21.15.45:0/784907144 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x7fc77800cad0 con 0x56338916d5f0 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56338916d5f0 msgr2=0x56338916d9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56338916d5f0 0x56338916d9f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc760065890 msgr2=0x7fc760067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc760065890 0x7fc760067d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc768020000 tx=0x7fc76801c000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890de5f0 msgr2=0x563389171ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 --2- 172.21.15.45:0/784907144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633890de5f0 0x563389171ba0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc77000a280 tx=0x7fc770002970 comp rx=0 tx=0).stop 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc77ec84700 1 -- 172.21.15.45:0/784907144 reap_dead start 2023-10-06T05:22:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 shutdown_connections 2023-10-06T05:22:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.431+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 >> 172.21.15.45:0/784907144 conn(0x56338909b4a0 msgr2=0x56338909cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.432+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 shutdown_connections 2023-10-06T05:22:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.432+0000 7fc785265040 1 -- 172.21.15.45:0/784907144 wait complete. 2023-10-06T05:22:09.438 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2023-10-06T05:22:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.482+0000 7f0a9842c040 1 Processor -- start 2023-10-06T05:22:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.482+0000 7f0a9842c040 1 -- start start 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9842c040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cfd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9842c040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0890 0x560705cdb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9842c040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0c60 0x560705c65750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9842c040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560705b6b980 con 0x560705c1e210 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9842c040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560705b6b620 con 0x560705ce0890 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9842c040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x560705b62400 con 0x560705ce0c60 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cfd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cfd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55360/0 (socket says 172.21.15.45:55360) 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 -- 172.21.15.45:0/1466165874 learned_addr learned my addr 172.21.15.45:0/1466165874 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a90e49700 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0890 0x560705cdb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a91e4b700 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0c60 0x560705c65750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 -- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0c60 msgr2=0x560705c65750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0c60 0x560705c65750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 -- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0890 msgr2=0x560705cdb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0890 0x560705cdb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.483+0000 7f0a9164a700 1 -- 172.21.15.45:0/1466165874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84005db0 con 0x560705c1e210 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9164a700 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cfd0f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400ad70 tx=0x7f0a84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3544f2bdb21e22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a837fe700 1 -- 172.21.15.45:0/1466165874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840054f0 con 0x560705c1e210 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a837fe700 1 -- 172.21.15.45:0/1466165874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84000bc0 con 0x560705c1e210 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a837fe700 1 -- 172.21.15.45:0/1466165874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84008c20 con 0x560705c1e210 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 -- 172.21.15.45:0/1466165874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 msgr2=0x560705cfd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cfd0f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400ad70 tx=0x7f0a84016000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 -- 172.21.15.45:0/1466165874 shutdown_connections 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0c60 0x560705c65750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cfd0f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 --2- 172.21.15.45:0/1466165874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0890 0x560705cdb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 -- 172.21.15.45:0/1466165874 >> 172.21.15.45:0/1466165874 conn(0x560705c21450 msgr2=0x560705c64550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 -- 172.21.15.45:0/1466165874 shutdown_connections 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.484+0000 7f0a9842c040 1 -- 172.21.15.45:0/1466165874 wait complete. 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.485+0000 7f0a9842c040 1 Processor -- start 2023-10-06T05:22:09.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.485+0000 7f0a9842c040 1 -- start start 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9842c040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cdc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9842c040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0890 0x560705cfa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9842c040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0c60 0x560705c49ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9842c040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560705cdca30 con 0x560705c1e210 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9842c040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560705cdccd0 con 0x560705ce0c60 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9842c040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x560705cdcf70 con 0x560705ce0890 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cdc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cdc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55362/0 (socket says 172.21.15.45:55362) 2023-10-06T05:22:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 -- 172.21.15.45:0/175021114 learned_addr learned my addr 172.21.15.45:0/175021114 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a90e49700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0890 0x560705cfa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a91e4b700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0c60 0x560705c49ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 -- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0890 msgr2=0x560705cfa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560705ce0890 0x560705cfa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 -- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0c60 msgr2=0x560705c49ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560705ce0c60 0x560705c49ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.486+0000 7f0a9164a700 1 -- 172.21.15.45:0/175021114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560705b941c0 con 0x560705c1e210 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.487+0000 7f0a9164a700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cdc080 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400a5a0 tx=0x7f0a8401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.487+0000 7f0a81ffb700 1 -- 172.21.15.45:0/175021114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84026880 con 0x560705c1e210 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.487+0000 7f0a81ffb700 1 -- 172.21.15.45:0/175021114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a840269e0 con 0x560705c1e210 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.487+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560705b7a3d0 con 0x560705c1e210 2023-10-06T05:22:09.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.487+0000 7f0a81ffb700 1 -- 172.21.15.45:0/175021114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84026da0 con 0x560705c1e210 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.487+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560705b6dae0 con 0x560705c1e210 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.488+0000 7f0a81ffb700 1 -- 172.21.15.45:0/175021114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a84009640 con 0x560705c1e210 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a81ffb700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a74065890 0x7f0a74067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a81ffb700 1 -- 172.21.15.45:0/175021114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0a84085d90 con 0x560705c1e210 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a9842c040 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560705cde6c0 0x560705d13580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x560705cfa7d0 con 0x560705cde6c0 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a90e49700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a74065890 0x7f0a74067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a91e4b700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560705cde6c0 0x560705d13580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a90e49700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a74065890 0x7f0a74067d40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0a88000c00 tx=0x7f0a88001780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.489+0000 7f0a91e4b700 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560705cde6c0 0x560705d13580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a91e4b700 1 -- 172.21.15.45:0/175021114 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+5 (crc 0 0 0) 0x7f0a8c002d90 con 0x560705cde6c0 2023-10-06T05:22:09.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560705cde6c0 msgr2=0x560705d13580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560705cde6c0 0x560705d13580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a74065890 msgr2=0x7f0a74067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a74065890 0x7f0a74067d40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0a88000c00 tx=0x7f0a88001780 comp rx=0 tx=0).stop 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 msgr2=0x560705cdc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 --2- 172.21.15.45:0/175021114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560705c1e210 0x560705cdc080 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400a5a0 tx=0x7f0a8401e000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a91e4b700 1 -- 172.21.15.45:0/175021114 reap_dead start 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 shutdown_connections 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 >> 172.21.15.45:0/175021114 conn(0x560705c21450 msgr2=0x560705c64550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 shutdown_connections 2023-10-06T05:22:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.490+0000 7f0a9842c040 1 -- 172.21.15.45:0/175021114 wait complete. 2023-10-06T05:22:09.497 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2023-10-06T05:22:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.541+0000 7fc68d09f040 1 Processor -- start 2023-10-06T05:22:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.541+0000 7fc68d09f040 1 -- start start 2023-10-06T05:22:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc68d09f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593462dcca0 0x559346382be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc68d09f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462f87e0 0x559346387bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc68d09f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559346383b30 0x559346388100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc68d09f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593461fb980 con 0x5593462f87e0 2023-10-06T05:22:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc68d09f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593461fb620 con 0x5593462dcca0 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc68d09f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593461f2400 con 0x559346383b30 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462f87e0 0x559346387bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462f87e0 0x559346387bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55378/0 (socket says 172.21.15.45:55378) 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 -- 172.21.15.45:0/4101245229 learned_addr learned my addr 172.21.15.45:0/4101245229 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc686abe700 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559346383b30 0x559346388100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc6862bd700 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593462dcca0 0x559346382be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 -- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559346383b30 msgr2=0x559346388100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559346383b30 0x559346388100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 -- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593462dcca0 msgr2=0x559346382be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593462dcca0 0x559346382be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 -- 172.21.15.45:0/4101245229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678005cf0 con 0x5593462f87e0 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc686abe700 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559346383b30 0x559346388100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc685abc700 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462f87e0 0x559346387bc0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc6780055b0 tx=0x7fc678015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57421aaf8cc4e555 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc684aba700 1 -- 172.21.15.45:0/4101245229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678000bc0 con 0x5593462f87e0 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc684aba700 1 -- 172.21.15.45:0/4101245229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6780078e0 con 0x5593462f87e0 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.542+0000 7fc684aba700 1 -- 172.21.15.45:0/4101245229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67801c410 con 0x5593462f87e0 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 -- 172.21.15.45:0/4101245229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462f87e0 msgr2=0x559346387bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462f87e0 0x559346387bc0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc6780055b0 tx=0x7fc678015000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 -- 172.21.15.45:0/4101245229 shutdown_connections 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559346383b30 0x559346388100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593462dcca0 0x559346382be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 --2- 172.21.15.45:0/4101245229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462f87e0 0x559346387bc0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 -- 172.21.15.45:0/4101245229 >> 172.21.15.45:0/4101245229 conn(0x5593462b1450 msgr2=0x5593462b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 -- 172.21.15.45:0/4101245229 shutdown_connections 2023-10-06T05:22:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.543+0000 7fc68d09f040 1 -- 172.21.15.45:0/4101245229 wait complete. 2023-10-06T05:22:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.544+0000 7fc68d09f040 1 Processor -- start 2023-10-06T05:22:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.544+0000 7fc68d09f040 1 -- start start 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.544+0000 7fc68d09f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462dcca0 0x5593463829d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.544+0000 7fc68d09f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5593462f87e0 0x55934637d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc68d09f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559346383b30 0x55934637ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc68d09f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593462b3860 con 0x5593462dcca0 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc68d09f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559346386300 con 0x559346383b30 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc68d09f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593462b3a40 con 0x5593462f87e0 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462dcca0 0x5593463829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462dcca0 0x5593463829d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55390/0 (socket says 172.21.15.45:55390) 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 -- 172.21.15.45:0/2846077254 learned_addr learned my addr 172.21.15.45:0/2846077254 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc685abc700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5593462f87e0 0x55934637d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 -- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5593462f87e0 msgr2=0x55934637d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc686abe700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559346383b30 0x55934637ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5593462f87e0 0x55934637d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 -- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559346383b30 msgr2=0x55934637ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559346383b30 0x55934637ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 -- 172.21.15.45:0/2846077254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593462241c0 con 0x5593462dcca0 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc685abc700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5593462f87e0 0x55934637d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc6862bd700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462dcca0 0x5593463829d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc67000df10 tx=0x7fc670000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc67effd700 1 -- 172.21.15.45:0/2846077254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000f450 con 0x5593462dcca0 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc67effd700 1 -- 172.21.15.45:0/2846077254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67000f5b0 con 0x5593462dcca0 2023-10-06T05:22:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55934620a3d0 con 0x5593462dcca0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc67effd700 1 -- 172.21.15.45:0/2846077254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000f970 con 0x5593462dcca0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.545+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593461fdae0 con 0x5593462dcca0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.547+0000 7fc67effd700 1 -- 172.21.15.45:0/2846077254 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc67000fce0 con 0x5593462dcca0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.547+0000 7fc67effd700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6680658c0 0x7fc668067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.547+0000 7fc67effd700 1 -- 172.21.15.45:0/2846077254 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc670083c10 con 0x5593462dcca0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.548+0000 7fc685abc700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6680658c0 0x7fc668067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.548+0000 7fc68d09f040 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x559346382f10 0x5593462f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.548+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x5593462b29a0 con 0x559346382f10 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.548+0000 7fc686abe700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x559346382f10 0x5593462f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.548+0000 7fc685abc700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6680658c0 0x7fc668067d70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc6780055b0 tx=0x7fc67801c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.548+0000 7fc686abe700 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x559346382f10 0x5593462f10d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc686abe700 1 -- 172.21.15.45:0/2846077254 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+20 (crc 0 0 0) 0x7fc68000a9b0 con 0x559346382f10 2023-10-06T05:22:09.554 INFO:teuthology.orchestra.run.smithi045.stdout:subdir_head 2023-10-06T05:22:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x559346382f10 msgr2=0x5593462f10d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x559346382f10 0x5593462f10d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6680658c0 msgr2=0x7fc668067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6680658c0 0x7fc668067d70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc6780055b0 tx=0x7fc67801c000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462dcca0 msgr2=0x5593463829d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 --2- 172.21.15.45:0/2846077254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5593462dcca0 0x5593463829d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc67000df10 tx=0x7fc670000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc686abe700 1 -- 172.21.15.45:0/2846077254 reap_dead start 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 shutdown_connections 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 >> 172.21.15.45:0/2846077254 conn(0x5593462b1450 msgr2=0x55934637f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 shutdown_connections 2023-10-06T05:22:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.549+0000 7fc68d09f040 1 -- 172.21.15.45:0/2846077254 wait complete. 2023-10-06T05:22:09.556 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2023-10-06T05:22:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.601+0000 7ff6ba018040 1 Processor -- start 2023-10-06T05:22:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.601+0000 7ff6ba018040 1 -- start start 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6ba018040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3cdccc50 0x55aa3ce580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6ba018040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce5b900 0x55aa3ce55280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6ba018040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3ce75c20 0x55aa3ce76290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6ba018040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3cce8620 con 0x55aa3cdccc50 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6ba018040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3ccdf400 con 0x55aa3ce5b900 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6ba018040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa3cd9dfa0 con 0x55aa3ce75c20 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6b3236700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3cdccc50 0x55aa3ce580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6b3236700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3cdccc50 0x55aa3ce580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55408/0 (socket says 172.21.15.45:55408) 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6b3236700 1 -- 172.21.15.45:0/2053896632 learned_addr learned my addr 172.21.15.45:0/2053896632 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6aaa35700 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce5b900 0x55aa3ce55280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.602+0000 7ff6b3a37700 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3ce75c20 0x55aa3ce76290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b3236700 1 -- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3ce75c20 msgr2=0x55aa3ce76290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b3236700 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3ce75c20 0x55aa3ce76290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b3236700 1 -- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce5b900 msgr2=0x55aa3ce55280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b3236700 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce5b900 0x55aa3ce55280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b3236700 1 -- 172.21.15.45:0/2053896632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c005cf0 con 0x55aa3cdccc50 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b3236700 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3cdccc50 0x55aa3ce580a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005d40 tx=0x7ff6a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18117e5fe8ef176 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b2234700 1 -- 172.21.15.45:0/2053896632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40055c0 con 0x55aa3cdccc50 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b2234700 1 -- 172.21.15.45:0/2053896632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a4000bc0 con 0x55aa3cdccc50 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6b2234700 1 -- 172.21.15.45:0/2053896632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a401d420 con 0x55aa3cdccc50 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 -- 172.21.15.45:0/2053896632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3cdccc50 msgr2=0x55aa3ce580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3cdccc50 0x55aa3ce580a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005d40 tx=0x7ff6a4016000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 -- 172.21.15.45:0/2053896632 shutdown_connections 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3ce75c20 0x55aa3ce76290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3cdccc50 0x55aa3ce580a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 --2- 172.21.15.45:0/2053896632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce5b900 0x55aa3ce55280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 -- 172.21.15.45:0/2053896632 >> 172.21.15.45:0/2053896632 conn(0x55aa3cd9dbd0 msgr2=0x55aa3ce72690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.603+0000 7ff6ba018040 1 -- 172.21.15.45:0/2053896632 shutdown_connections 2023-10-06T05:22:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.604+0000 7ff6ba018040 1 -- 172.21.15.45:0/2053896632 wait complete. 2023-10-06T05:22:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.604+0000 7ff6ba018040 1 Processor -- start 2023-10-06T05:22:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.604+0000 7ff6ba018040 1 -- start start 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6ba018040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3cdccc50 0x55aa3ce714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6ba018040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3ce5b900 0x55aa3cde5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6ba018040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce75c20 0x55aa3ce56430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6ba018040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3ce85000 con 0x55aa3ce5b900 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6ba018040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3ce85260 con 0x55aa3ce75c20 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6ba018040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa3ce85500 con 0x55aa3cdccc50 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3ce5b900 0x55aa3cde5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3ce5b900 0x55aa3cde5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55424/0 (socket says 172.21.15.45:55424) 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 -- 172.21.15.45:0/3911857014 learned_addr learned my addr 172.21.15.45:0/3911857014 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6b3236700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3cdccc50 0x55aa3ce714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6b3a37700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce75c20 0x55aa3ce56430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 -- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3cdccc50 msgr2=0x55aa3ce714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa3cdccc50 0x55aa3ce714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 -- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce75c20 msgr2=0x55aa3ce56430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa3ce75c20 0x55aa3ce56430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.605+0000 7ff6aaa35700 1 -- 172.21.15.45:0/3911857014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3ccea750 con 0x55aa3ce5b900 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.606+0000 7ff6aaa35700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3ce5b900 0x55aa3cde5800 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff69c008bc0 tx=0x7ff69c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.606+0000 7ff6b0a31700 1 -- 172.21.15.45:0/3911857014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00f040 con 0x55aa3ce5b900 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.606+0000 7ff6b0a31700 1 -- 172.21.15.45:0/3911857014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c0108b0 con 0x55aa3ce5b900 2023-10-06T05:22:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.606+0000 7ff6b0a31700 1 -- 172.21.15.45:0/3911857014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c010c30 con 0x55aa3ce5b900 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.606+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a4005db0 con 0x55aa3ce5b900 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.606+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa3cd111c0 con 0x55aa3ce5b900 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.607+0000 7ff6b0a31700 1 -- 172.21.15.45:0/3911857014 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff69c010d90 con 0x55aa3ce5b900 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6b0a31700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff694065890 0x7ff694067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6b0a31700 1 -- 172.21.15.45:0/3911857014 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ff69c084130 con 0x55aa3ce5b900 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6b3236700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff694065890 0x7ff694067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6ba018040 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55aa3cde5d40 0x55aa3ce84460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55aa3ce849a0 con 0x55aa3cde5d40 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6b3a37700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55aa3cde5d40 0x55aa3ce84460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6b3236700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff694065890 0x7ff694067d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4023000 tx=0x7ff6a401e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.608+0000 7ff6b3a37700 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55aa3cde5d40 0x55aa3ce84460 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6b3a37700 1 -- 172.21.15.45:0/3911857014 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x7ff6ac00bb80 con 0x55aa3cde5d40 2023-10-06T05:22:09.615 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/10000000000.00000000 mtime 2023-10-06T05:22:06.000000+0000, size 0 2023-10-06T05:22:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55aa3cde5d40 msgr2=0x55aa3ce84460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55aa3cde5d40 0x55aa3ce84460 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff694065890 msgr2=0x7ff694067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff694065890 0x7ff694067d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4023000 tx=0x7ff6a401e000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3ce5b900 msgr2=0x55aa3cde5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 --2- 172.21.15.45:0/3911857014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3ce5b900 0x55aa3cde5800 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff69c008bc0 tx=0x7ff69c00c040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6b3a37700 1 -- 172.21.15.45:0/3911857014 reap_dead start 2023-10-06T05:22:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 shutdown_connections 2023-10-06T05:22:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 >> 172.21.15.45:0/3911857014 conn(0x55aa3cd9dbd0 msgr2=0x55aa3ce72690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 shutdown_connections 2023-10-06T05:22:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.609+0000 7ff6ba018040 1 -- 172.21.15.45:0/3911857014 wait complete. 2023-10-06T05:22:09.618 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2023-10-06T05:22:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.661+0000 7ff05390b040 1 Processor -- start 2023-10-06T05:22:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.661+0000 7ff05390b040 1 -- start start 2023-10-06T05:22:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff05390b040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263b211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff05390b040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 0x55c263bd8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff05390b040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bd8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff05390b040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c263a3f620 con 0x55c263bce610 2023-10-06T05:22:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff05390b040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c263a36400 con 0x55c263bcc0b0 2023-10-06T05:22:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff05390b040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c263af8490 con 0x55c263b21b10 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff04d32a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bd8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff04d32a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bd8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55448/0 (socket says 172.21.15.45:55448) 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.662+0000 7ff04d32a700 1 -- 172.21.15.45:0/1739532195 learned_addr learned my addr 172.21.15.45:0/1739532195 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff047fff700 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 0x55c263bd8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04cb29700 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263b211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04d32a700 1 -- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 msgr2=0x55c263b211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04d32a700 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263b211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04d32a700 1 -- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 msgr2=0x55c263bd8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04d32a700 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 0x55c263bd8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04d32a700 1 -- 172.21.15.45:0/1739532195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04800a180 con 0x55c263bce610 2023-10-06T05:22:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04cb29700 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263b211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff04d32a700 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bd8690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff04800a510 tx=0x7ff048011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c74b98954da61a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff046ffd700 1 -- 172.21.15.45:0/1739532195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e510 con 0x55c263bce610 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff046ffd700 1 -- 172.21.15.45:0/1739532195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800ed60 con 0x55c263bce610 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.663+0000 7ff046ffd700 1 -- 172.21.15.45:0/1739532195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e9f0 con 0x55c263bce610 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 -- 172.21.15.45:0/1739532195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 msgr2=0x55c263bd8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bd8690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff04800a510 tx=0x7ff048011000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 -- 172.21.15.45:0/1739532195 shutdown_connections 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bd8690 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263b211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 --2- 172.21.15.45:0/1739532195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 0x55c263bd8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 -- 172.21.15.45:0/1739532195 >> 172.21.15.45:0/1739532195 conn(0x55c263af6750 msgr2=0x55c263af7470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 -- 172.21.15.45:0/1739532195 shutdown_connections 2023-10-06T05:22:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 -- 172.21.15.45:0/1739532195 wait complete. 2023-10-06T05:22:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 Processor -- start 2023-10-06T05:22:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.664+0000 7ff05390b040 1 -- start start 2023-10-06T05:22:09.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff05390b040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263bc1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff05390b040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 0x55c263bc1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff05390b040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bc3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff05390b040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c263bd21d0 con 0x55c263bce610 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff05390b040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c263bd2470 con 0x55c263bcc0b0 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff05390b040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c263bd2710 con 0x55c263b21b10 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff04d32a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bc3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff04d32a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bc3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55452/0 (socket says 172.21.15.45:55452) 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff04d32a700 1 -- 172.21.15.45:0/200003437 learned_addr learned my addr 172.21.15.45:0/200003437 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff04cb29700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263bc1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.665+0000 7ff047fff700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 0x55c263bc1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff04d32a700 1 -- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 msgr2=0x55c263bc1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff04d32a700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c263b21b10 0x55c263bc1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff04d32a700 1 -- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 msgr2=0x55c263bc1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff04d32a700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c263bcc0b0 0x55c263bc1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff04d32a700 1 -- 172.21.15.45:0/200003437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c263a41750 con 0x55c263bce610 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff04d32a700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bc3240 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55c263af71b0 tx=0x7ff04801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff0457fa700 1 -- 172.21.15.45:0/200003437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048029070 con 0x55c263bce610 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff0457fa700 1 -- 172.21.15.45:0/200003437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04802d410 con 0x55c263bce610 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c263a681c0 con 0x55c263bce610 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff0457fa700 1 -- 172.21.15.45:0/200003437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e510 con 0x55c263bce610 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.666+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c263a4e3d0 con 0x55c263bce610 2023-10-06T05:22:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.668+0000 7ff0457fa700 1 -- 172.21.15.45:0/200003437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff048003040 con 0x55c263bce610 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.668+0000 7ff0457fa700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0340658c0 0x7ff034067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.668+0000 7ff0457fa700 1 -- 172.21.15.45:0/200003437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7ff0480866d0 con 0x55c263bce610 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.668+0000 7ff04cb29700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0340658c0 0x7ff034067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.668+0000 7ff05390b040 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c263bcd490 0x55c263bcf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.668+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55c263b3c600 con 0x55c263bcd490 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.668+0000 7ff047fff700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c263bcd490 0x55c263bcf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff04cb29700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0340658c0 0x7ff034067d70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff038009b50 tx=0x7ff038005c60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff047fff700 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c263bcd490 0x55c263bcf9a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff047fff700 1 -- 172.21.15.45:0/200003437 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7ff040009a30 con 0x55c263bcd490 2023-10-06T05:22:09.674 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/600.00000000 mtime 2023-10-06T05:21:51.000000+0000, size 0 2023-10-06T05:22:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c263bcd490 msgr2=0x55c263bcf9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff05390b040 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c263bcd490 0x55c263bcf9a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0340658c0 msgr2=0x7ff034067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff05390b040 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0340658c0 0x7ff034067d70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff038009b50 tx=0x7ff038005c60 comp rx=0 tx=0).stop 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 msgr2=0x55c263bc3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff05390b040 1 --2- 172.21.15.45:0/200003437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c263bce610 0x55c263bc3240 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55c263af71b0 tx=0x7ff04801d000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.669+0000 7ff04d32a700 1 -- 172.21.15.45:0/200003437 reap_dead start 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.670+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 shutdown_connections 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.670+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 >> 172.21.15.45:0/200003437 conn(0x55c263af6750 msgr2=0x55c263b1fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.670+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 shutdown_connections 2023-10-06T05:22:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.670+0000 7ff05390b040 1 -- 172.21.15.45:0/200003437 wait complete. 2023-10-06T05:22:09.678 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2023-10-06T05:22:09.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.722+0000 7f4fd0aac040 1 Processor -- start 2023-10-06T05:22:09.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.722+0000 7f4fd0aac040 1 -- start start 2023-10-06T05:22:09.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.722+0000 7f4fd0aac040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86d02210 0x557e86d06f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.722+0000 7f4fd0aac040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86dc4890 0x557e86d02850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fd0aac040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d48f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fd0aac040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e86c4f980 con 0x557e86dc4c60 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fd0aac040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e86c4f620 con 0x557e86dc4890 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fd0aac040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e86c46400 con 0x557e86d02210 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d48f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d48f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55478/0 (socket says 172.21.15.45:55478) 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 -- 172.21.15.45:0/1340993557 learned_addr learned my addr 172.21.15.45:0/1340993557 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fc9cca700 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86d02210 0x557e86d06f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fc94c9700 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86dc4890 0x557e86d02850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 -- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86d02210 msgr2=0x557e86d06f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86d02210 0x557e86d06f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 -- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86dc4890 msgr2=0x557e86d02850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86dc4890 0x557e86d02850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 -- 172.21.15.45:0/1340993557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc400a180 con 0x557e86dc4c60 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fc9cca700 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86d02210 0x557e86d06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fc94c9700 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86dc4890 0x557e86d02850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fca4cb700 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d48f90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400ace0 tx=0x7f4fc4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b26828d0e1de6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fc3fff700 1 -- 172.21.15.45:0/1340993557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4005960 con 0x557e86dc4c60 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fc3fff700 1 -- 172.21.15.45:0/1340993557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400ed60 con 0x557e86dc4c60 2023-10-06T05:22:09.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.723+0000 7f4fc3fff700 1 -- 172.21.15.45:0/1340993557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc401c430 con 0x557e86dc4c60 2023-10-06T05:22:09.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 -- 172.21.15.45:0/1340993557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 msgr2=0x557e86d48f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d48f90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400ace0 tx=0x7f4fc4014000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 -- 172.21.15.45:0/1340993557 shutdown_connections 2023-10-06T05:22:09.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d48f90 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86d02210 0x557e86d06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/1340993557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86dc4890 0x557e86d02850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 -- 172.21.15.45:0/1340993557 >> 172.21.15.45:0/1340993557 conn(0x557e86d05450 msgr2=0x557e86d077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 -- 172.21.15.45:0/1340993557 shutdown_connections 2023-10-06T05:22:09.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.724+0000 7f4fd0aac040 1 -- 172.21.15.45:0/1340993557 wait complete. 2023-10-06T05:22:09.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 Processor -- start 2023-10-06T05:22:09.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 -- start start 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86d02210 0x557e86d26c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86dc4890 0x557e86dc35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d321c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e86d271c0 con 0x557e86dc4c60 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e86d27400 con 0x557e86d02210 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.725+0000 7f4fd0aac040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e86d276a0 con 0x557e86dc4890 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d321c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55494/0 (socket says 172.21.15.45:55494) 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 -- 172.21.15.45:0/782476089 learned_addr learned my addr 172.21.15.45:0/782476089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fc94c9700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86dc4890 0x557e86dc35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fc9cca700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86d02210 0x557e86d26c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 -- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86dc4890 msgr2=0x557e86dc35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557e86dc4890 0x557e86dc35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 -- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86d02210 msgr2=0x557e86d26c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e86d02210 0x557e86d26c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 -- 172.21.15.45:0/782476089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e86c781c0 con 0x557e86dc4c60 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fca4cb700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d321c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005e40 tx=0x7f4fc401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fc27fc700 1 -- 172.21.15.45:0/782476089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc402a070 con 0x557e86dc4c60 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fc27fc700 1 -- 172.21.15.45:0/782476089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc402e410 con 0x557e86dc4c60 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557e86c5e3d0 con 0x557e86dc4c60 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.726+0000 7f4fc27fc700 1 -- 172.21.15.45:0/782476089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400e9f0 con 0x557e86dc4c60 2023-10-06T05:22:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.727+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e86c51ae0 con 0x557e86dc4c60 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.728+0000 7f4fc27fc700 1 -- 172.21.15.45:0/782476089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fc4003040 con 0x557e86dc4c60 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.728+0000 7f4fc27fc700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4fac065890 0x7f4fac067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.728+0000 7f4fc27fc700 1 -- 172.21.15.45:0/782476089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f4fc40864d0 con 0x557e86dc4c60 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.729+0000 7f4fc9cca700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4fac065890 0x7f4fac067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.729+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557e86de0860 0x557e86de0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.729+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x557e86de1170 con 0x557e86de0860 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.729+0000 7f4fc94c9700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557e86de0860 0x557e86de0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.729+0000 7f4fc9cca700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4fac065890 0x7f4fac067d40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005fd0 tx=0x7f4fb400c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.729+0000 7f4fc94c9700 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557e86de0860 0x557e86de0c30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fc94c9700 1 -- 172.21.15.45:0/782476089 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x7f4fbc00b020 con 0x557e86de0860 2023-10-06T05:22:09.736 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/1.00000000.inode mtime 2023-10-06T05:22:06.000000+0000, size 554 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557e86de0860 msgr2=0x557e86de0c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557e86de0860 0x557e86de0c30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4fac065890 msgr2=0x7f4fac067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4fac065890 0x7f4fac067d40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005fd0 tx=0x7f4fb400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 msgr2=0x557e86d321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 --2- 172.21.15.45:0/782476089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e86dc4c60 0x557e86d321c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005e40 tx=0x7f4fc401d000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fca4cb700 1 -- 172.21.15.45:0/782476089 reap_dead start 2023-10-06T05:22:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 shutdown_connections 2023-10-06T05:22:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 >> 172.21.15.45:0/782476089 conn(0x557e86d05450 msgr2=0x557e86de2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 shutdown_connections 2023-10-06T05:22:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.730+0000 7f4fd0aac040 1 -- 172.21.15.45:0/782476089 wait complete. 2023-10-06T05:22:09.738 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2023-10-06T05:22:09.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.782+0000 7f994a599040 1 Processor -- start 2023-10-06T05:22:09.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f994a599040 1 -- start start 2023-10-06T05:22:09.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f994a599040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56327089b5d0 0x563270945bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f994a599040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270946c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f994a599040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563270943400 0x5632708725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f994a599040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632707b9980 con 0x56327092c9a0 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f994a599040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632707b9620 con 0x563270943400 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f994a599040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632707b0400 con 0x56327089b5d0 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f9942fb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270946c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f9942fb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270946c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55520/0 (socket says 172.21.15.45:55520) 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f9942fb6700 1 -- 172.21.15.45:0/1324014264 learned_addr learned my addr 172.21.15.45:0/1324014264 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f99437b7700 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56327089b5d0 0x563270945bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.783+0000 7f9943fb8700 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563270943400 0x5632708725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9942fb6700 1 -- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56327089b5d0 msgr2=0x563270945bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9942fb6700 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56327089b5d0 0x563270945bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9942fb6700 1 -- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563270943400 msgr2=0x5632708725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9942fb6700 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563270943400 0x5632708725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9942fb6700 1 -- 172.21.15.45:0/1324014264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938005db0 con 0x56327092c9a0 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9942fb6700 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270946c30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f993800ae50 tx=0x7f9938015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c929bc7b81870d08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9941fb4700 1 -- 172.21.15.45:0/1324014264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938008a30 con 0x56327092c9a0 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9941fb4700 1 -- 172.21.15.45:0/1324014264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9938008bb0 con 0x56327092c9a0 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.784+0000 7f9941fb4700 1 -- 172.21.15.45:0/1324014264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380126a0 con 0x56327092c9a0 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 -- 172.21.15.45:0/1324014264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 msgr2=0x563270946c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270946c30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f993800ae50 tx=0x7f9938015000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 -- 172.21.15.45:0/1324014264 shutdown_connections 2023-10-06T05:22:09.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563270943400 0x5632708725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56327089b5d0 0x563270945bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 --2- 172.21.15.45:0/1324014264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270946c30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 -- 172.21.15.45:0/1324014264 >> 172.21.15.45:0/1324014264 conn(0x563270870750 msgr2=0x56327092ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 -- 172.21.15.45:0/1324014264 shutdown_connections 2023-10-06T05:22:09.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 -- 172.21.15.45:0/1324014264 wait complete. 2023-10-06T05:22:09.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.785+0000 7f994a599040 1 Processor -- start 2023-10-06T05:22:09.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f994a599040 1 -- start start 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f994a599040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56327089b5d0 0x56327092b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f994a599040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270957190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f994a599040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563270943400 0x5632709576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f994a599040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563270946470 con 0x56327092c9a0 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f994a599040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563270942a40 con 0x56327089b5d0 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f994a599040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632708717d0 con 0x563270943400 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f9942fb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270957190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f9942fb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270957190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55522/0 (socket says 172.21.15.45:55522) 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f99437b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56327089b5d0 0x56327092b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f9942fb6700 1 -- 172.21.15.45:0/1857888787 learned_addr learned my addr 172.21.15.45:0/1857888787 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.786+0000 7f9943fb8700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563270943400 0x5632709576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f9942fb6700 1 -- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563270943400 msgr2=0x5632709576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f9942fb6700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563270943400 0x5632709576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f9942fb6700 1 -- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56327089b5d0 msgr2=0x56327092b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f9942fb6700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56327089b5d0 0x56327092b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f9942fb6700 1 -- 172.21.15.45:0/1857888787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632707e21c0 con 0x56327092c9a0 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f9942fb6700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270957190 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f993800ae50 tx=0x7f993801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f992bfff700 1 -- 172.21.15.45:0/1857888787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938008a30 con 0x56327092c9a0 2023-10-06T05:22:09.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f992bfff700 1 -- 172.21.15.45:0/1857888787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993800a240 con 0x56327092c9a0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632707c83d0 con 0x56327092c9a0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f992bfff700 1 -- 172.21.15.45:0/1857888787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993802e6d0 con 0x56327092c9a0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.787+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632707bbae0 con 0x56327092c9a0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.789+0000 7f992bfff700 1 -- 172.21.15.45:0/1857888787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9938019070 con 0x56327092c9a0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.789+0000 7f992bfff700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9930065890 0x7f9930067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.789+0000 7f992bfff700 1 -- 172.21.15.45:0/1857888787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f99380863e0 con 0x56327092c9a0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.789+0000 7f99437b7700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9930065890 0x7f9930067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.789+0000 7f994a599040 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56327089bde0 0x563270924b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.789+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x56327092bd70 con 0x56327089bde0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.790+0000 7f9943fb8700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56327089bde0 0x563270924b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.790+0000 7f99437b7700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9930065890 0x7f9930067d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f993c00f850 tx=0x7f993c0027f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.790+0000 7f9943fb8700 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56327089bde0 0x563270924b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.790+0000 7f9943fb8700 1 -- 172.21.15.45:0/1857888787 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f993400cad0 con 0x56327089bde0 2023-10-06T05:22:09.797 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/200.00000000 mtime 2023-10-06T05:22:06.000000+0000, size 90 2023-10-06T05:22:09.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56327089bde0 msgr2=0x563270924b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56327089bde0 0x563270924b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9930065890 msgr2=0x7f9930067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9930065890 0x7f9930067d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f993c00f850 tx=0x7f993c0027f0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 msgr2=0x563270957190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 --2- 172.21.15.45:0/1857888787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56327092c9a0 0x563270957190 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f993800ae50 tx=0x7f993801c000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f9943fb8700 1 -- 172.21.15.45:0/1857888787 reap_dead start 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 shutdown_connections 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 >> 172.21.15.45:0/1857888787 conn(0x563270870750 msgr2=0x563270945b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 shutdown_connections 2023-10-06T05:22:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.791+0000 7f994a599040 1 -- 172.21.15.45:0/1857888787 wait complete. 2023-10-06T05:22:09.800 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2023-10-06T05:22:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.844+0000 7f8e7e503040 1 Processor -- start 2023-10-06T05:22:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.844+0000 7f8e7e503040 1 -- start start 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e7e503040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a3d3ed0 0x555b6a3a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e7e503040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a47b9f0 0x555b6a3d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e7e503040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3d8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e7e503040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6a34f320 con 0x555b6a47f900 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e7e503040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6a2f3980 con 0x555b6a47b9f0 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e7e503040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b6a2f3620 con 0x555b6a3d3ed0 2023-10-06T05:22:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e77f22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3d8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e77f22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3d8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55526/0 (socket says 172.21.15.45:55526) 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e77f22700 1 -- 172.21.15.45:0/3731312520 learned_addr learned my addr 172.21.15.45:0/3731312520 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e76f20700 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a47b9f0 0x555b6a3d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.845+0000 7f8e77721700 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a3d3ed0 0x555b6a3a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e77f22700 1 -- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a3d3ed0 msgr2=0x555b6a3a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e77f22700 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a3d3ed0 0x555b6a3a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e77f22700 1 -- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a47b9f0 msgr2=0x555b6a3d81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e77f22700 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a47b9f0 0x555b6a3d81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e77f22700 1 -- 172.21.15.45:0/3731312520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e60005cf0 con 0x555b6a47f900 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e77f22700 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3d8700 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006f20 tx=0x7f8e700069f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d5bb4dc2a26f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e75f1e700 1 -- 172.21.15.45:0/3731312520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700020a0 con 0x555b6a47f900 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e75f1e700 1 -- 172.21.15.45:0/3731312520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e70006110 con 0x555b6a47f900 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.846+0000 7f8e75f1e700 1 -- 172.21.15.45:0/3731312520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004520 con 0x555b6a47f900 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 -- 172.21.15.45:0/3731312520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 msgr2=0x555b6a3d8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3d8700 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006f20 tx=0x7f8e700069f0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 -- 172.21.15.45:0/3731312520 shutdown_connections 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3d8700 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a3d3ed0 0x555b6a3a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 --2- 172.21.15.45:0/3731312520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a47b9f0 0x555b6a3d81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 -- 172.21.15.45:0/3731312520 >> 172.21.15.45:0/3731312520 conn(0x555b6a3a94a0 msgr2=0x555b6a47ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 -- 172.21.15.45:0/3731312520 shutdown_connections 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 -- 172.21.15.45:0/3731312520 wait complete. 2023-10-06T05:22:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 Processor -- start 2023-10-06T05:22:09.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.847+0000 7f8e7e503040 1 -- start start 2023-10-06T05:22:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.848+0000 7f8e7e503040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a3d3ed0 0x555b6a3d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.848+0000 7f8e7e503040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a47b9f0 0x555b6a3ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.848+0000 7f8e7e503040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.848+0000 7f8e7e503040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6a2ea400 con 0x555b6a47f900 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.848+0000 7f8e7e503040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6a3d3d00 con 0x555b6a3d3ed0 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.848+0000 7f8e7e503040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b6a3abe00 con 0x555b6a47b9f0 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.848+0000 7f8e77f22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3a6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55534/0 (socket says 172.21.15.45:55534) 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 -- 172.21.15.45:0/1183009087 learned_addr learned my addr 172.21.15.45:0/1183009087 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e76f20700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a47b9f0 0x555b6a3ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77721700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a3d3ed0 0x555b6a3d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 -- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a47b9f0 msgr2=0x555b6a3ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a47b9f0 0x555b6a3ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 -- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a3d3ed0 msgr2=0x555b6a3d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b6a3d3ed0 0x555b6a3d37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 -- 172.21.15.45:0/1183009087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e64001e10 con 0x555b6a47f900 2023-10-06T05:22:09.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e76f20700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555b6a47b9f0 0x555b6a3ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:09.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e77f22700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3a6cf0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8e70004bf0 tx=0x7f8e70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e6ffff700 1 -- 172.21.15.45:0/1183009087 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70002120 con 0x555b6a47f900 2023-10-06T05:22:09.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e6ffff700 1 -- 172.21.15.45:0/1183009087 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e70004050 con 0x555b6a47f900 2023-10-06T05:22:09.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b6a2f5ae0 con 0x555b6a47f900 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e6ffff700 1 -- 172.21.15.45:0/1183009087 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70003450 con 0x555b6a47f900 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.849+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b6a2f5540 con 0x555b6a47f900 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.851+0000 7f8e6ffff700 1 -- 172.21.15.45:0/1183009087 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e70003800 con 0x555b6a47f900 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.851+0000 7f8e6ffff700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8e580658c0 0x7f8e58067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.851+0000 7f8e6ffff700 1 -- 172.21.15.45:0/1183009087 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8e70084670 con 0x555b6a47f900 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.851+0000 7f8e77721700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8e580658c0 0x7f8e58067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.851+0000 7f8e7e503040 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x555b6a4796e0 0x555b6a479ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.852+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x555b6a492ed0 con 0x555b6a4796e0 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.852+0000 7f8e76f20700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x555b6a4796e0 0x555b6a479ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.852+0000 7f8e77721700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8e580658c0 0x7f8e58067d70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000c00 tx=0x7f8e68002870 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.852+0000 7f8e76f20700 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x555b6a4796e0 0x555b6a479ae0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.852+0000 7f8e76f20700 1 -- 172.21.15.45:0/1183009087 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f8e60000bc0 con 0x555b6a4796e0 2023-10-06T05:22:09.858 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/200.00000001 mtime 2023-10-06T05:22:06.000000+0000, size 14075 2023-10-06T05:22:09.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x555b6a4796e0 msgr2=0x555b6a479ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x555b6a4796e0 0x555b6a479ae0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8e580658c0 msgr2=0x7f8e58067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8e580658c0 0x7f8e58067d70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000c00 tx=0x7f8e68002870 comp rx=0 tx=0).stop 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 msgr2=0x555b6a3a6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 --2- 172.21.15.45:0/1183009087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b6a47f900 0x555b6a3a6cf0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8e70004bf0 tx=0x7f8e70014000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e77f22700 1 -- 172.21.15.45:0/1183009087 reap_dead start 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 shutdown_connections 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 >> 172.21.15.45:0/1183009087 conn(0x555b6a3a94a0 msgr2=0x555b6a482a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 shutdown_connections 2023-10-06T05:22:09.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.853+0000 7f8e7e503040 1 -- 172.21.15.45:0/1183009087 wait complete. 2023-10-06T05:22:09.860 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2023-10-06T05:22:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.904+0000 7fa9eb3e4040 1 Processor -- start 2023-10-06T05:22:09.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9eb3e4040 1 -- start start 2023-10-06T05:22:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9eb3e4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b07254f80 0x562b07228c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9eb3e4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b072e5900 0x562b072e4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9eb3e4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b0726fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9eb3e4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b07170620 con 0x562b072e5cd0 2023-10-06T05:22:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9eb3e4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b07167400 con 0x562b072e5900 2023-10-06T05:22:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9eb3e4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b072e13c0 con 0x562b07254f80 2023-10-06T05:22:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9e4e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b0726fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9e4602700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b07254f80 0x562b07228c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9e4e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b0726fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55546/0 (socket says 172.21.15.45:55546) 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9e4e03700 1 -- 172.21.15.45:0/3934458610 learned_addr learned my addr 172.21.15.45:0/3934458610 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.905+0000 7fa9e3e01700 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b072e5900 0x562b072e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e4e03700 1 -- 172.21.15.45:0/3934458610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b07254f80 msgr2=0x562b07228c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e4e03700 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b07254f80 0x562b07228c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e4e03700 1 -- 172.21.15.45:0/3934458610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b072e5900 msgr2=0x562b072e4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e4e03700 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b072e5900 0x562b072e4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e4e03700 1 -- 172.21.15.45:0/3934458610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4005db0 con 0x562b072e5cd0 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e4e03700 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b0726fdc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c500 tx=0x7fa9dc007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17497de156ac351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e2dff700 1 -- 172.21.15.45:0/3934458610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc010590 con 0x562b072e5cd0 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e2dff700 1 -- 172.21.15.45:0/3934458610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc0106f0 con 0x562b072e5cd0 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.906+0000 7fa9e2dff700 1 -- 172.21.15.45:0/3934458610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc010ab0 con 0x562b072e5cd0 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/3934458610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 msgr2=0x562b0726fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b0726fdc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c500 tx=0x7fa9dc007040 comp rx=0 tx=0).stop 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/3934458610 shutdown_connections 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b0726fdc0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b07254f80 0x562b07228c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/3934458610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b072e5900 0x562b072e4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/3934458610 >> 172.21.15.45:0/3934458610 conn(0x562b072275f0 msgr2=0x562b072e2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/3934458610 shutdown_connections 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/3934458610 wait complete. 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 Processor -- start 2023-10-06T05:22:09.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.907+0000 7fa9eb3e4040 1 -- start start 2023-10-06T05:22:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9eb3e4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b07254f80 0x562b072fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9eb3e4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b072e5900 0x562b07252450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9eb3e4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b072de760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9eb3e4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0726a700 con 0x562b072e5cd0 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9eb3e4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0726a9a0 con 0x562b07254f80 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9eb3e4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b0726ac40 con 0x562b072e5900 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9e4e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b072de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9e4e03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b072de760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55558/0 (socket says 172.21.15.45:55558) 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9e4e03700 1 -- 172.21.15.45:0/870281960 learned_addr learned my addr 172.21.15.45:0/870281960 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9e4602700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b07254f80 0x562b072fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.908+0000 7fa9e3e01700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b072e5900 0x562b07252450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e4e03700 1 -- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b072e5900 msgr2=0x562b07252450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e4e03700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b072e5900 0x562b07252450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e4e03700 1 -- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b07254f80 msgr2=0x562b072fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e4e03700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b07254f80 0x562b072fbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e4e03700 1 -- 172.21.15.45:0/870281960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc00d040 con 0x562b072e5cd0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e4e03700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b072de760 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c4a0 tx=0x7fa9dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e15fc700 1 -- 172.21.15.45:0/870281960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00cec0 con 0x562b072e5cd0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e15fc700 1 -- 172.21.15.45:0/870281960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc010a70 con 0x562b072e5cd0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b071991c0 con 0x562b072e5cd0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9e15fc700 1 -- 172.21.15.45:0/870281960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0256c0 con 0x562b072e5cd0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.909+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b0717f3d0 con 0x562b072e5cd0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.911+0000 7fa9e15fc700 1 -- 172.21.15.45:0/870281960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9dc022070 con 0x562b072e5cd0 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.911+0000 7fa9e15fc700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9d0065890 0x7fa9d0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.911+0000 7fa9e15fc700 1 -- 172.21.15.45:0/870281960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fa9dc085300 con 0x562b072e5cd0 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.911+0000 7fa9e4602700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9d0065890 0x7fa9d0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.911+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x562b072fd600 0x562b0726b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.911+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.6 14:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x562b0726bb30 con 0x562b072fd600 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.911+0000 7fa9e3e01700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x562b072fd600 0x562b0726b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9e4602700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9d0065890 0x7fa9d0067d40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4008410 tx=0x7fa9d4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9e3e01700 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x562b072fd600 0x562b0726b5f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9e3e01700 1 -- 172.21.15.45:0/870281960 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x7fa9d8009b60 con 0x562b072fd600 2023-10-06T05:22:09.917 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2023-10-06T05:22:06.000000+0000, size 0 2023-10-06T05:22:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x562b072fd600 msgr2=0x562b0726b5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x562b072fd600 0x562b0726b5f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9d0065890 msgr2=0x7fa9d0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa9d0065890 0x7fa9d0067d40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4008410 tx=0x7fa9d4016000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 msgr2=0x562b072de760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 --2- 172.21.15.45:0/870281960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b072e5cd0 0x562b072de760 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c4a0 tx=0x7fa9dc018000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9e4e03700 1 -- 172.21.15.45:0/870281960 reap_dead start 2023-10-06T05:22:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 shutdown_connections 2023-10-06T05:22:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.912+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 >> 172.21.15.45:0/870281960 conn(0x562b072275f0 msgr2=0x562b072e1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.913+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 shutdown_connections 2023-10-06T05:22:09.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.913+0000 7fa9eb3e4040 1 -- 172.21.15.45:0/870281960 wait complete. 2023-10-06T05:22:09.920 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2023-10-06T05:22:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 Processor -- start 2023-10-06T05:22:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 -- start start 2023-10-06T05:22:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db5629ff80 0x55db5632c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db56330900 0x55db562b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db56217320 con 0x55db56330cd0 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db561bb980 con 0x55db5629ff80 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.965+0000 7f0d8b21b040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db561bb620 con 0x55db56330900 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55584/0 (socket says 172.21.15.45:55584) 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1947102684 learned_addr learned my addr 172.21.15.45:0/1947102684 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d83c38700 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db56330900 0x55db562b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84439700 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db5629ff80 0x55db5632c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db56330900 msgr2=0x55db562b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db56330900 0x55db562b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db5629ff80 msgr2=0x55db5632c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:09.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db5629ff80 0x55db5632c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1947102684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c005db0 con 0x55db56330cd0 2023-10-06T05:22:09.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d84c3a700 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632ec50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00a560 tx=0x7f0d7c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2ab706d6022c9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d82c36700 1 -- 172.21.15.45:0/1947102684 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00e510 con 0x55db56330cd0 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d82c36700 1 -- 172.21.15.45:0/1947102684 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00e690 con 0x55db56330cd0 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.966+0000 7f0d82c36700 1 -- 172.21.15.45:0/1947102684 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00ea50 con 0x55db56330cd0 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1947102684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 msgr2=0x55db5632ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632ec50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00a560 tx=0x7f0d7c005c40 comp rx=0 tx=0).stop 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1947102684 shutdown_connections 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632ec50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db5629ff80 0x55db5632c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1947102684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db56330900 0x55db562b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1947102684 >> 172.21.15.45:0/1947102684 conn(0x55db562725f0 msgr2=0x55db56272fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1947102684 shutdown_connections 2023-10-06T05:22:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1947102684 wait complete. 2023-10-06T05:22:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.967+0000 7f0d8b21b040 1 Processor -- start 2023-10-06T05:22:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.968+0000 7f0d8b21b040 1 -- start start 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.968+0000 7f0d8b21b040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db5629ff80 0x55db5632d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.968+0000 7f0d8b21b040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db56330900 0x55db562bb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.968+0000 7f0d8b21b040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.968+0000 7f0d8b21b040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db561b2400 con 0x55db56330cd0 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.968+0000 7f0d8b21b040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db56293bb0 con 0x55db56330900 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.968+0000 7f0d8b21b040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db562ba350 con 0x55db5629ff80 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55596/0 (socket says 172.21.15.45:55596) 2023-10-06T05:22:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1291489621 learned_addr learned my addr 172.21.15.45:0/1291489621 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84439700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db5629ff80 0x55db5632d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d83c38700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db56330900 0x55db562bb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db5629ff80 msgr2=0x55db5632d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55db5629ff80 0x55db5632d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db56330900 msgr2=0x55db562bb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db56330900 0x55db562bb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1291489621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c00a0e0 con 0x55db56330cd0 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d84c3a700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632a880 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c005e00 tx=0x7f0d7c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d81433700 1 -- 172.21.15.45:0/1291489621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c000f40 con 0x55db56330cd0 2023-10-06T05:22:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d81433700 1 -- 172.21.15.45:0/1291489621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c005620 con 0x55db56330cd0 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d81433700 1 -- 172.21.15.45:0/1291489621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0010a0 con 0x55db56330cd0 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55db561bdae0 con 0x55db56330cd0 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.969+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db561bd540 con 0x55db56330cd0 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.971+0000 7f0d81433700 1 -- 172.21.15.45:0/1291489621 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d7c01a650 con 0x55db56330cd0 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.971+0000 7f0d81433700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d64065890 0x7f0d64067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.971+0000 7f0d81433700 1 -- 172.21.15.45:0/1291489621 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0d7c084330 con 0x55db56330cd0 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.971+0000 7f0d84439700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d64065890 0x7f0d64067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.972+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55db562bb220 0x55db563580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.972+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.15 14:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55db5632adc0 con 0x55db562bb220 2023-10-06T05:22:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.972+0000 7f0d83c38700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55db562bb220 0x55db563580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.972+0000 7f0d84439700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d64065890 0x7f0d64067d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c008410 tx=0x7f0d6c012000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.972+0000 7f0d83c38700 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55db562bb220 0x55db563580c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d83c38700 1 -- 172.21.15.45:0/1291489621 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f0d7400e820 con 0x55db562bb220 2023-10-06T05:22:09.979 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/500.00000000 mtime 2023-10-06T05:21:51.000000+0000, size 90 2023-10-06T05:22:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55db562bb220 msgr2=0x55db563580c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55db562bb220 0x55db563580c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d64065890 msgr2=0x7f0d64067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d64065890 0x7f0d64067d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c008410 tx=0x7f0d6c012000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 msgr2=0x55db5632a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 --2- 172.21.15.45:0/1291489621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db56330cd0 0x55db5632a880 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c005e00 tx=0x7f0d7c014000 comp rx=0 tx=0).stop 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d84c3a700 1 -- 172.21.15.45:0/1291489621 reap_dead start 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 shutdown_connections 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 >> 172.21.15.45:0/1291489621 conn(0x55db562725f0 msgr2=0x55db56272d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 shutdown_connections 2023-10-06T05:22:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:09.973+0000 7f0d8b21b040 1 -- 172.21.15.45:0/1291489621 wait complete. 2023-10-06T05:22:09.982 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2023-10-06T05:22:10.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.026+0000 7f4c174ef040 1 Processor -- start 2023-10-06T05:22:10.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.026+0000 7f4c174ef040 1 -- start start 2023-10-06T05:22:10.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c174ef040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921ba1210 0x561921c800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c174ef040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63890 0x561921c5e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c174ef040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921c63c60 0x561921be8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c174ef040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561921aee980 con 0x561921ba1210 2023-10-06T05:22:10.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c174ef040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561921aee620 con 0x561921c63c60 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c174ef040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561921ae5400 con 0x561921c63890 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921ba1210 0x561921c800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921ba1210 0x561921c800f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55618/0 (socket says 172.21.15.45:55618) 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 -- 172.21.15.45:0/2986700503 learned_addr learned my addr 172.21.15.45:0/2986700503 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c10f0e700 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921c63c60 0x561921be8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c0ff0c700 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63890 0x561921c5e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 -- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63890 msgr2=0x561921c5e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63890 0x561921c5e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 -- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921c63c60 msgr2=0x561921be8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921c63c60 0x561921be8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 -- 172.21.15.45:0/2986700503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8005db0 con 0x561921ba1210 2023-10-06T05:22:10.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c10f0e700 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921c63c60 0x561921be8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c1070d700 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921ba1210 0x561921c800f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800ad70 tx=0x7f4bf8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160f4daab52b4cb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c0ef0a700 1 -- 172.21.15.45:0/2986700503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80055f0 con 0x561921ba1210 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.027+0000 7f4c0ef0a700 1 -- 172.21.15.45:0/2986700503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf8000bc0 con 0x561921ba1210 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c0ef0a700 1 -- 172.21.15.45:0/2986700503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8008c00 con 0x561921ba1210 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 -- 172.21.15.45:0/2986700503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921ba1210 msgr2=0x561921c800f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921ba1210 0x561921c800f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800ad70 tx=0x7f4bf8016000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 -- 172.21.15.45:0/2986700503 shutdown_connections 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921c63c60 0x561921be8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921ba1210 0x561921c800f0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 --2- 172.21.15.45:0/2986700503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63890 0x561921c5e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 -- 172.21.15.45:0/2986700503 >> 172.21.15.45:0/2986700503 conn(0x561921ba4450 msgr2=0x561921be7550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 -- 172.21.15.45:0/2986700503 shutdown_connections 2023-10-06T05:22:10.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.028+0000 7f4c174ef040 1 -- 172.21.15.45:0/2986700503 wait complete. 2023-10-06T05:22:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.029+0000 7f4c174ef040 1 Processor -- start 2023-10-06T05:22:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.029+0000 7f4c174ef040 1 -- start start 2023-10-06T05:22:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c174ef040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921ba1210 0x561921c60540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c174ef040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921c63890 0x561921c5e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c174ef040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63c60 0x561921c5e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c174ef040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561921c5ed20 con 0x561921c63890 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c174ef040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561921c5f080 con 0x561921ba1210 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c174ef040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561921c5f2b0 con 0x561921c63c60 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c0ff0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921c63890 0x561921c5e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c0ff0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921c63890 0x561921c5e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55626/0 (socket says 172.21.15.45:55626) 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c0ff0c700 1 -- 172.21.15.45:0/1641198940 learned_addr learned my addr 172.21.15.45:0/1641198940 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c10f0e700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63c60 0x561921c5e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.030+0000 7f4c1070d700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921ba1210 0x561921c60540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0ff0c700 1 -- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63c60 msgr2=0x561921c5e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0ff0c700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63c60 0x561921c5e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0ff0c700 1 -- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921ba1210 msgr2=0x561921c60540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0ff0c700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921ba1210 0x561921c60540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0ff0c700 1 -- 172.21.15.45:0/1641198940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561921b171c0 con 0x561921c63890 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c10f0e700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561921c63c60 0x561921c5e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c1070d700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561921ba1210 0x561921c60540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0ff0c700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921c63890 0x561921c5e2a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f4c00003da0 tx=0x7f4c00003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0d707700 1 -- 172.21.15.45:0/1641198940 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000bd40 con 0x561921c63890 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0d707700 1 -- 172.21.15.45:0/1641198940 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c00008040 con 0x561921c63890 2023-10-06T05:22:10.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561921afd3d0 con 0x561921c63890 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c0d707700 1 -- 172.21.15.45:0/1641198940 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000bea0 con 0x561921c63890 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.031+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561921af0ae0 con 0x561921c63890 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.033+0000 7f4c0d707700 1 -- 172.21.15.45:0/1641198940 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c00001590 con 0x561921c63890 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.033+0000 7f4c0d707700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4bf00658c0 0x7f4bf0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.033+0000 7f4c0d707700 1 -- 172.21.15.45:0/1641198940 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f4c00083260 con 0x561921c63890 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.033+0000 7f4c174ef040 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561921c60a80 0x561921c8d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.033+0000 7f4c1070d700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4bf00658c0 0x7f4bf0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.033+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x561921c5ff30 con 0x561921c60a80 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c10f0e700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561921c60a80 0x561921c8d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c1070d700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4bf00658c0 0x7f4bf0067d70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8025000 tx=0x7f4bf801e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c10f0e700 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561921c60a80 0x561921c8d8a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c10f0e700 1 -- 172.21.15.45:0/1641198940 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+16 (crc 0 0 0) 0x7f4c08002c70 con 0x561921c60a80 2023-10-06T05:22:10.040 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/mds_snaptable mtime 2023-10-06T05:21:51.000000+0000, size 70 2023-10-06T05:22:10.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561921c60a80 msgr2=0x561921c8d8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561921c60a80 0x561921c8d8a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4bf00658c0 msgr2=0x7f4bf0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4bf00658c0 0x7f4bf0067d70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8025000 tx=0x7f4bf801e000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921c63890 msgr2=0x561921c5e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 --2- 172.21.15.45:0/1641198940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561921c63890 0x561921c5e2a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f4c00003da0 tx=0x7f4c00003990 comp rx=0 tx=0).stop 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c10f0e700 1 -- 172.21.15.45:0/1641198940 reap_dead start 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 shutdown_connections 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.034+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 >> 172.21.15.45:0/1641198940 conn(0x561921ba4450 msgr2=0x561921ba1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.035+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 shutdown_connections 2023-10-06T05:22:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.035+0000 7f4c174ef040 1 -- 172.21.15.45:0/1641198940 wait complete. 2023-10-06T05:22:10.042 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2023-10-06T05:22:10.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.087+0000 7fe9e0da1040 1 Processor -- start 2023-10-06T05:22:10.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.087+0000 7fe9e0da1040 1 -- start start 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.087+0000 7fe9e0da1040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0ad2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.087+0000 7fe9e0da1040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0b00450 0x555df0ba9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9e0da1040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0ba6fc0 0x555df0ad0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9e0da1040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df0a1b620 con 0x555df0ba6fc0 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9e0da1040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df0a12400 con 0x555df0afc120 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9e0da1040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555df0ad2010 con 0x555df0b00450 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0ba6fc0 0x555df0ad0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0ba6fc0 0x555df0ad0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55646/0 (socket says 172.21.15.45:55646) 2023-10-06T05:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 -- 172.21.15.45:0/3596387079 learned_addr learned my addr 172.21.15.45:0/3596387079 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9d9fbf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0ad2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9d97be700 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0b00450 0x555df0ba9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 -- 172.21.15.45:0/3596387079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0b00450 msgr2=0x555df0ba9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0b00450 0x555df0ba9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 -- 172.21.15.45:0/3596387079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 msgr2=0x555df0ad2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0ad2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 -- 172.21.15.45:0/3596387079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d400a180 con 0x555df0ba6fc0 2023-10-06T05:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9da7c0700 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0ba6fc0 0x555df0ad0fa0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400a510 tx=0x7fe9d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54199aad69b617c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9d3fff700 1 -- 172.21.15.45:0/3596387079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400e510 con 0x555df0ba6fc0 2023-10-06T05:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9d3fff700 1 -- 172.21.15.45:0/3596387079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d400ed80 con 0x555df0ba6fc0 2023-10-06T05:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.088+0000 7fe9d3fff700 1 -- 172.21.15.45:0/3596387079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400ea10 con 0x555df0ba6fc0 2023-10-06T05:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 -- 172.21.15.45:0/3596387079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0ba6fc0 msgr2=0x555df0ad0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0ba6fc0 0x555df0ad0fa0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400a510 tx=0x7fe9d4014000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 -- 172.21.15.45:0/3596387079 shutdown_connections 2023-10-06T05:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0ba6fc0 0x555df0ad0fa0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0ad2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/3596387079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0b00450 0x555df0ba9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 -- 172.21.15.45:0/3596387079 >> 172.21.15.45:0/3596387079 conn(0x555df0ad0bd0 msgr2=0x555df0ad2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 -- 172.21.15.45:0/3596387079 shutdown_connections 2023-10-06T05:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.089+0000 7fe9e0da1040 1 -- 172.21.15.45:0/3596387079 wait complete. 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.090+0000 7fe9e0da1040 1 Processor -- start 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.090+0000 7fe9e0da1040 1 -- start start 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.090+0000 7fe9e0da1040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0baf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.090+0000 7fe9e0da1040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0b00450 0x555df0ba97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.090+0000 7fe9e0da1040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0ba6fc0 0x555df0bb2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.090+0000 7fe9e0da1040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df0bb1d10 con 0x555df0b00450 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9e0da1040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df0bb1fb0 con 0x555df0afc120 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9e0da1040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555df0bb2250 con 0x555df0ba6fc0 2023-10-06T05:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0b00450 0x555df0ba97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0b00450 0x555df0ba97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55652/0 (socket says 172.21.15.45:55652) 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 -- 172.21.15.45:0/642169457 learned_addr learned my addr 172.21.15.45:0/642169457 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9da7c0700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0ba6fc0 0x555df0bb2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d9fbf700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0baf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 -- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0ba6fc0 msgr2=0x555df0bb2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0ba6fc0 0x555df0bb2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 -- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 msgr2=0x555df0baf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0baf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 -- 172.21.15.45:0/642169457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df0a1d750 con 0x555df0b00450 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9da7c0700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555df0ba6fc0 0x555df0bb2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d9fbf700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555df0afc120 0x555df0baf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d97be700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0b00450 0x555df0ba97f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc008ac0 tx=0x7fe9cc00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d27fc700 1 -- 172.21.15.45:0/642169457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00f040 con 0x555df0b00450 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d27fc700 1 -- 172.21.15.45:0/642169457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc0108b0 con 0x555df0b00450 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9d27fc700 1 -- 172.21.15.45:0/642169457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc010c70 con 0x555df0b00450 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555df0a441c0 con 0x555df0b00450 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.091+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555df0a2a3d0 con 0x555df0b00450 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.093+0000 7fe9d27fc700 1 -- 172.21.15.45:0/642169457 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9cc015660 con 0x555df0b00450 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.093+0000 7fe9d27fc700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe9bc0658c0 0x7fe9bc067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.093+0000 7fe9d27fc700 1 -- 172.21.15.45:0/642169457 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fe9cc084360 con 0x555df0b00450 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.093+0000 7fe9d9fbf700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe9bc0658c0 0x7fe9bc067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x555df0ba9d30 0x555df0bad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x555df0bb1690 con 0x555df0ba9d30 2023-10-06T05:22:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9da7c0700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x555df0ba9d30 0x555df0bad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9d9fbf700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe9bc0658c0 0x7fe9bc067d70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005fd0 tx=0x7fe9c400c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9da7c0700 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x555df0ba9d30 0x555df0bad690 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9da7c0700 1 -- 172.21.15.45:0/642169457 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+16 (crc 0 0 0) 0x7fe9d400e7e0 con 0x555df0ba9d30 2023-10-06T05:22:10.101 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/mds0_inotable mtime 2023-10-06T05:22:06.000000+0000, size 34 2023-10-06T05:22:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x555df0ba9d30 msgr2=0x555df0bad690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.094+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x555df0ba9d30 0x555df0bad690 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe9bc0658c0 msgr2=0x7fe9bc067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe9bc0658c0 0x7fe9bc067d70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005fd0 tx=0x7fe9c400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0b00450 msgr2=0x555df0ba97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 --2- 172.21.15.45:0/642169457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555df0b00450 0x555df0ba97f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc008ac0 tx=0x7fe9cc00c040 comp rx=0 tx=0).stop 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9da7c0700 1 -- 172.21.15.45:0/642169457 reap_dead start 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 shutdown_connections 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 >> 172.21.15.45:0/642169457 conn(0x555df0ad0bd0 msgr2=0x555df0ad0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 shutdown_connections 2023-10-06T05:22:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.095+0000 7fe9e0da1040 1 -- 172.21.15.45:0/642169457 wait complete. 2023-10-06T05:22:10.104 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2023-10-06T05:22:10.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.149+0000 7f37a9a4a040 1 Processor -- start 2023-10-06T05:22:10.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.149+0000 7f37a9a4a040 1 -- start start 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a9a4a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4656be0 0x55a1c46eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a9a4a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 0x55a1c4703c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a9a4a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4702190 0x55a1c462c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a9a4a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c4576620 con 0x55a1c4656be0 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a9a4a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c456d400 con 0x55a1c4702190 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a9a4a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1c46e81c0 con 0x55a1c4701dc0 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4656be0 0x55a1c46eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4656be0 0x55a1c46eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55668/0 (socket says 172.21.15.45:55668) 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 -- 172.21.15.45:0/1515291648 learned_addr learned my addr 172.21.15.45:0/1515291648 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a3469700 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4702190 0x55a1c462c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2467700 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 0x55a1c4703c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 -- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 msgr2=0x55a1c4703c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 0x55a1c4703c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 -- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4702190 msgr2=0x55a1c462c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4702190 0x55a1c462c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 -- 172.21.15.45:0/1515291648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c00a180 con 0x55a1c4656be0 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a2c68700 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4656be0 0x55a1c46eada0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f379400a810 tx=0x7f3794005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d800e84fe727852e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a1465700 1 -- 172.21.15.45:0/1515291648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379400f070 con 0x55a1c4656be0 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a1465700 1 -- 172.21.15.45:0/1515291648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794008760 con 0x55a1c4656be0 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.150+0000 7f37a1465700 1 -- 172.21.15.45:0/1515291648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794008b00 con 0x55a1c4656be0 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 -- 172.21.15.45:0/1515291648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4656be0 msgr2=0x55a1c46eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4656be0 0x55a1c46eada0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f379400a810 tx=0x7f3794005d00 comp rx=0 tx=0).stop 2023-10-06T05:22:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 -- 172.21.15.45:0/1515291648 shutdown_connections 2023-10-06T05:22:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4702190 0x55a1c462c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4656be0 0x55a1c46eada0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/1515291648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 0x55a1c4703c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 -- 172.21.15.45:0/1515291648 >> 172.21.15.45:0/1515291648 conn(0x55a1c462bbd0 msgr2=0x55a1c4659730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 -- 172.21.15.45:0/1515291648 shutdown_connections 2023-10-06T05:22:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.151+0000 7f37a9a4a040 1 -- 172.21.15.45:0/1515291648 wait complete. 2023-10-06T05:22:10.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 Processor -- start 2023-10-06T05:22:10.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 -- start start 2023-10-06T05:22:10.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4656be0 0x55a1c4715bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 0x55a1c4716100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4702190 0x55a1c46ea460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c46eb050 con 0x55a1c4702190 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c4713790 con 0x55a1c4656be0 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.152+0000 7f37a9a4a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1c4713a30 con 0x55a1c4701dc0 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4702190 0x55a1c46ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4702190 0x55a1c46ea460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55682/0 (socket says 172.21.15.45:55682) 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 -- 172.21.15.45:0/2851876250 learned_addr learned my addr 172.21.15.45:0/2851876250 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a2467700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 0x55a1c4716100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a2c68700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4656be0 0x55a1c4715bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 -- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 msgr2=0x55a1c4716100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1c4701dc0 0x55a1c4716100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 -- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4656be0 msgr2=0x55a1c4715bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1c4656be0 0x55a1c4715bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 -- 172.21.15.45:0/2851876250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379400a440 con 0x55a1c4702190 2023-10-06T05:22:10.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a3469700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4702190 0x55a1c46ea460 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f379c010000 tx=0x7f379c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37937fe700 1 -- 172.21.15.45:0/2851876250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00e510 con 0x55a1c4702190 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37937fe700 1 -- 172.21.15.45:0/2851876250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00ed60 con 0x55a1c4702190 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37937fe700 1 -- 172.21.15.45:0/2851876250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c005b30 con 0x55a1c4702190 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a1c459f1c0 con 0x55a1c4702190 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.153+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1c45853d0 con 0x55a1c4702190 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.155+0000 7f37937fe700 1 -- 172.21.15.45:0/2851876250 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f379c023070 con 0x55a1c4702190 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.155+0000 7f37937fe700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3788065890 0x7f3788067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.155+0000 7f37937fe700 1 -- 172.21.15.45:0/2851876250 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f379c085850 con 0x55a1c4702190 2023-10-06T05:22:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.155+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a1c470d6c0 0x55a1c470da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.155+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55a1c4714170 con 0x55a1c470d6c0 2023-10-06T05:22:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a2c68700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3788065890 0x7f3788067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a2467700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a1c470d6c0 0x55a1c470da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a2467700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a1c470d6c0 0x55a1c470da90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a2c68700 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3788065890 0x7f3788067d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f379400afd0 tx=0x7f3794017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a2467700 1 -- 172.21.15.45:0/2851876250 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f3798009b30 con 0x55a1c470d6c0 2023-10-06T05:22:10.162 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/100.00000000 mtime 2023-10-06T05:21:51.000000+0000, size 0 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a1c470d6c0 msgr2=0x55a1c470da90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a1c470d6c0 0x55a1c470da90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3788065890 msgr2=0x7f3788067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3788065890 0x7f3788067d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f379400afd0 tx=0x7f3794017000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.156+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4702190 msgr2=0x55a1c46ea460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.157+0000 7f37a9a4a040 1 --2- 172.21.15.45:0/2851876250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1c4702190 0x55a1c46ea460 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f379c010000 tx=0x7f379c011000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.157+0000 7f37a3469700 1 -- 172.21.15.45:0/2851876250 reap_dead start 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.157+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 shutdown_connections 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.157+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 >> 172.21.15.45:0/2851876250 conn(0x55a1c462bbd0 msgr2=0x55a1c4671f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.157+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 shutdown_connections 2023-10-06T05:22:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.157+0000 7f37a9a4a040 1 -- 172.21.15.45:0/2851876250 wait complete. 2023-10-06T05:22:10.165 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2023-10-06T05:22:10.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.209+0000 7fd30d1f8040 1 Processor -- start 2023-10-06T05:22:10.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.209+0000 7fd30d1f8040 1 -- start start 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd30d1f8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb344e7e0 0x557cb34db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd30d1f8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb34d9b30 0x557cb34d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd30d1f8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9f00 0x557cb34cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd30d1f8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb3351980 con 0x557cb344e7e0 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd30d1f8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb3351620 con 0x557cb34d9b30 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd30d1f8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cb3348400 con 0x557cb34d9f00 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb344e7e0 0x557cb34db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb344e7e0 0x557cb34db500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55696/0 (socket says 172.21.15.45:55696) 2023-10-06T05:22:10.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 -- 172.21.15.45:0/3738550345 learned_addr learned my addr 172.21.15.45:0/3738550345 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306c17700 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9f00 0x557cb34cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd305c15700 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb34d9b30 0x557cb34d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 -- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9f00 msgr2=0x557cb34cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9f00 0x557cb34cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 -- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb34d9b30 msgr2=0x557cb34d8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb34d9b30 0x557cb34d8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 -- 172.21.15.45:0/3738550345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc005db0 con 0x557cb344e7e0 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd305c15700 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb34d9b30 0x557cb34d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd306416700 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb344e7e0 0x557cb34db500 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800a780 tx=0x7fd2f8005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcd9852be63f2f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd304c13700 1 -- 172.21.15.45:0/3738550345 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f800f070 con 0x557cb344e7e0 2023-10-06T05:22:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd304c13700 1 -- 172.21.15.45:0/3738550345 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f8008630 con 0x557cb344e7e0 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.210+0000 7fd304c13700 1 -- 172.21.15.45:0/3738550345 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80089d0 con 0x557cb344e7e0 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3738550345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb344e7e0 msgr2=0x557cb34db500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb344e7e0 0x557cb34db500 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800a780 tx=0x7fd2f8005c80 comp rx=0 tx=0).stop 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3738550345 shutdown_connections 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9f00 0x557cb34cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb344e7e0 0x557cb34db500 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3738550345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb34d9b30 0x557cb34d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3738550345 >> 172.21.15.45:0/3738550345 conn(0x557cb3407450 msgr2=0x557cb3409830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3738550345 shutdown_connections 2023-10-06T05:22:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.211+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3738550345 wait complete. 2023-10-06T05:22:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.212+0000 7fd30d1f8040 1 Processor -- start 2023-10-06T05:22:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.212+0000 7fd30d1f8040 1 -- start start 2023-10-06T05:22:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd30d1f8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb344e7e0 0x557cb34ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd30d1f8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9b30 0x557cb344b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd30d1f8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb34d9f00 0x557cb344cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd30d1f8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb344bc10 con 0x557cb34d9f00 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd30d1f8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cb344c0f0 con 0x557cb344e7e0 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd30d1f8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cb344c390 con 0x557cb34d9b30 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb34d9f00 0x557cb344cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb34d9f00 0x557cb344cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55704/0 (socket says 172.21.15.45:55704) 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 -- 172.21.15.45:0/3670611471 learned_addr learned my addr 172.21.15.45:0/3670611471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd305c15700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9b30 0x557cb344b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306416700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb344e7e0 0x557cb34ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 -- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9b30 msgr2=0x557cb344b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557cb34d9b30 0x557cb344b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 -- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb344e7e0 msgr2=0x557cb34ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cb344e7e0 0x557cb34ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 -- 172.21.15.45:0/3670611471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f800a440 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd306c17700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb34d9f00 0x557cb344cd70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd300003500 tx=0x7fd300008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd2f6ffd700 1 -- 172.21.15.45:0/3670611471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300004040 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.213+0000 7fd2f6ffd700 1 -- 172.21.15.45:0/3670611471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300013480 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.214+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557cb33603d0 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.214+0000 7fd2f6ffd700 1 -- 172.21.15.45:0/3670611471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300013840 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.214+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cb3353ae0 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.215+0000 7fd2f6ffd700 1 -- 172.21.15.45:0/3670611471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd300013c30 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.215+0000 7fd2f6ffd700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2e8065890 0x7fd2e8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd2f6ffd700 1 -- 172.21.15.45:0/3670611471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fd300083650 con 0x557cb34d9f00 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd306416700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2e8065890 0x7fd2e8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557cb34d6060 0x557cb34d6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x557cb34cfb70 con 0x557cb34d6060 2023-10-06T05:22:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd305c15700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557cb34d6060 0x557cb34d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd305c15700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557cb34d6060 0x557cb34d6430 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd306416700 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2e8065890 0x7fd2e8067d40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005ce0 tx=0x7fd2f8018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.216+0000 7fd305c15700 1 -- 172.21.15.45:0/3670611471 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 159+0+16 (crc 0 0 0) 0x7fd2fc008a50 con 0x557cb34d6060 2023-10-06T05:22:10.221 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/mds0_sessionmap mtime 2023-10-06T05:22:06.000000+0000, size 0 2023-10-06T05:22:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557cb34d6060 msgr2=0x557cb34d6430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557cb34d6060 0x557cb34d6430 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2e8065890 msgr2=0x7fd2e8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2e8065890 0x7fd2e8067d40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005ce0 tx=0x7fd2f8018000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb34d9f00 msgr2=0x557cb344cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 --2- 172.21.15.45:0/3670611471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cb34d9f00 0x557cb344cd70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd300003500 tx=0x7fd300008040 comp rx=0 tx=0).stop 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd306c17700 1 -- 172.21.15.45:0/3670611471 reap_dead start 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 shutdown_connections 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 >> 172.21.15.45:0/3670611471 conn(0x557cb3407450 msgr2=0x557cb34dca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 shutdown_connections 2023-10-06T05:22:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.217+0000 7fd30d1f8040 1 -- 172.21.15.45:0/3670611471 wait complete. 2023-10-06T05:22:10.224 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2023-10-06T05:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.268+0000 7f7762a86040 1 Processor -- start 2023-10-06T05:22:10.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.268+0000 7f7762a86040 1 -- start start 2023-10-06T05:22:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f7762a86040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee566fe0 0x5582ee5940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f7762a86040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee5ac5f0 0x5582ee5aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f7762a86040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 0x5582ee63f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f7762a86040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ee4b3980 con 0x5582ee5ac5f0 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f7762a86040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ee4b3620 con 0x5582ee628970 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f7762a86040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582ee4aa400 con 0x5582ee566fe0 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee5ac5f0 0x5582ee5aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee5ac5f0 0x5582ee5aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55720/0 (socket says 172.21.15.45:55720) 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 -- 172.21.15.45:0/1578003957 learned_addr learned my addr 172.21.15.45:0/1578003957 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775c4a5700 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 0x5582ee63f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775bca4700 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee566fe0 0x5582ee5940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 -- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee566fe0 msgr2=0x5582ee5940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee566fe0 0x5582ee5940e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 -- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 msgr2=0x5582ee63f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 0x5582ee63f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 -- 172.21.15.45:0/1578003957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750005db0 con 0x5582ee5ac5f0 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775bca4700 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee566fe0 0x5582ee5940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775b4a3700 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee5ac5f0 0x5582ee5aec60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f775000ad70 tx=0x7f7750016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65f2d0a6b933363 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.269+0000 7f775a4a1700 1 -- 172.21.15.45:0/1578003957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750000bc0 con 0x5582ee5ac5f0 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f775a4a1700 1 -- 172.21.15.45:0/1578003957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750020070 con 0x5582ee5ac5f0 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f775a4a1700 1 -- 172.21.15.45:0/1578003957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750008c20 con 0x5582ee5ac5f0 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 -- 172.21.15.45:0/1578003957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee5ac5f0 msgr2=0x5582ee5aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee5ac5f0 0x5582ee5aec60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f775000ad70 tx=0x7f7750016000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 -- 172.21.15.45:0/1578003957 shutdown_connections 2023-10-06T05:22:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 0x5582ee63f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee566fe0 0x5582ee5940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 --2- 172.21.15.45:0/1578003957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee5ac5f0 0x5582ee5aec60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 -- 172.21.15.45:0/1578003957 >> 172.21.15.45:0/1578003957 conn(0x5582ee5694a0 msgr2=0x5582ee63ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 -- 172.21.15.45:0/1578003957 shutdown_connections 2023-10-06T05:22:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.270+0000 7f7762a86040 1 -- 172.21.15.45:0/1578003957 wait complete. 2023-10-06T05:22:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.271+0000 7f7762a86040 1 Processor -- start 2023-10-06T05:22:10.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.271+0000 7f7762a86040 1 -- start start 2023-10-06T05:22:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f7762a86040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee566fe0 0x5582ee6253f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f7762a86040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee5ac5f0 0x5582ee61f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f7762a86040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 0x5582ee63e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f7762a86040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ee597fe0 con 0x5582ee566fe0 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f7762a86040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ee597300 con 0x5582ee628970 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f7762a86040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582ee625930 con 0x5582ee5ac5f0 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee566fe0 0x5582ee6253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee566fe0 0x5582ee6253f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55724/0 (socket says 172.21.15.45:55724) 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 -- 172.21.15.45:0/3384425111 learned_addr learned my addr 172.21.15.45:0/3384425111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775b4a3700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee5ac5f0 0x5582ee61f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775c4a5700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 0x5582ee63e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 -- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee5ac5f0 msgr2=0x5582ee61f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee5ac5f0 0x5582ee61f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 -- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 msgr2=0x5582ee63e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ee628970 0x5582ee63e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 -- 172.21.15.45:0/3384425111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582ee4dc1c0 con 0x5582ee566fe0 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775b4a3700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5582ee5ac5f0 0x5582ee61f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f775bca4700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee566fe0 0x5582ee6253f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f77540088f0 tx=0x7f7754005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.272+0000 7f7758c9e700 1 -- 172.21.15.45:0/3384425111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754018070 con 0x5582ee566fe0 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.273+0000 7f7758c9e700 1 -- 172.21.15.45:0/3384425111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7754010040 con 0x5582ee566fe0 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.273+0000 7f7758c9e700 1 -- 172.21.15.45:0/3384425111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754006cd0 con 0x5582ee566fe0 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.273+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5582ee4c23d0 con 0x5582ee566fe0 2023-10-06T05:22:10.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.273+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582ee4b5ae0 con 0x5582ee566fe0 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.274+0000 7f7758c9e700 1 -- 172.21.15.45:0/3384425111 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7754014600 con 0x5582ee566fe0 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.275+0000 7f7758c9e700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77480658c0 0x7f7748067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.275+0000 7f7758c9e700 1 -- 172.21.15.45:0/3384425111 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f7754083690 con 0x5582ee566fe0 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.275+0000 7f775b4a3700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77480658c0 0x7f7748067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.275+0000 7f7762a86040 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5582ee642300 0x5582ee596770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.275+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x5582ee596cb0 con 0x5582ee642300 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.275+0000 7f775c4a5700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5582ee642300 0x5582ee596770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.275+0000 7f775b4a3700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77480658c0 0x7f7748067d70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7750008830 tx=0x7f775001e070 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f775c4a5700 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5582ee642300 0x5582ee596770 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f775c4a5700 1 -- 172.21.15.45:0/3384425111 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x7f774c006d00 con 0x5582ee642300 2023-10-06T05:22:10.282 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/400.00000000 mtime 2023-10-06T05:21:51.000000+0000, size 22 2023-10-06T05:22:10.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5582ee642300 msgr2=0x5582ee596770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5582ee642300 0x5582ee596770 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77480658c0 msgr2=0x7f7748067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77480658c0 0x7f7748067d70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7750008830 tx=0x7f775001e070 comp rx=0 tx=0).stop 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee566fe0 msgr2=0x5582ee6253f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 --2- 172.21.15.45:0/3384425111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582ee566fe0 0x5582ee6253f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f77540088f0 tx=0x7f7754005a50 comp rx=0 tx=0).stop 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f775c4a5700 1 -- 172.21.15.45:0/3384425111 reap_dead start 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 shutdown_connections 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 >> 172.21.15.45:0/3384425111 conn(0x5582ee5694a0 msgr2=0x5582ee5946e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 shutdown_connections 2023-10-06T05:22:10.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.276+0000 7f7762a86040 1 -- 172.21.15.45:0/3384425111 wait complete. 2023-10-06T05:22:10.284 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2023-10-06T05:22:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.328+0000 7f8522251040 1 Processor -- start 2023-10-06T05:22:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.328+0000 7f8522251040 1 -- start start 2023-10-06T05:22:10.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f8522251040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d215c50 0x561c7d226ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f8522251040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d2bc0c0 0x561c7d22cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f8522251040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc730 0x561c7d29e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f8522251040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d131980 con 0x561c7d2bc730 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f8522251040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d131620 con 0x561c7d215c50 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f8522251040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c7d128400 con 0x561c7d2bc0c0 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f851bc70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc730 0x561c7d29e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f851bc70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc730 0x561c7d29e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55738/0 (socket says 172.21.15.45:55738) 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f851bc70700 1 -- 172.21.15.45:0/660912570 learned_addr learned my addr 172.21.15.45:0/660912570 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f851ac6e700 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d2bc0c0 0x561c7d22cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.329+0000 7f851b46f700 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d215c50 0x561c7d226ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f851bc70700 1 -- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d2bc0c0 msgr2=0x561c7d22cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f851bc70700 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d2bc0c0 0x561c7d22cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f851bc70700 1 -- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d215c50 msgr2=0x561c7d226ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f851bc70700 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d215c50 0x561c7d226ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f851bc70700 1 -- 172.21.15.45:0/660912570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504005cf0 con 0x561c7d2bc730 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f851b46f700 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d215c50 0x561c7d226ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f851bc70700 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc730 0x561c7d29e5e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x561c7d1ea8a0 tx=0x7f8514015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7626427a3b15a859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f8519c6c700 1 -- 172.21.15.45:0/660912570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851400f860 con 0x561c7d2bc730 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f8519c6c700 1 -- 172.21.15.45:0/660912570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851400f9e0 con 0x561c7d2bc730 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.330+0000 7f8519c6c700 1 -- 172.21.15.45:0/660912570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85140246c0 con 0x561c7d2bc730 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 -- 172.21.15.45:0/660912570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc730 msgr2=0x561c7d29e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc730 0x561c7d29e5e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x561c7d1ea8a0 tx=0x7f8514015000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 -- 172.21.15.45:0/660912570 shutdown_connections 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc730 0x561c7d29e5e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d215c50 0x561c7d226ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 --2- 172.21.15.45:0/660912570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d2bc0c0 0x561c7d22cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 -- 172.21.15.45:0/660912570 >> 172.21.15.45:0/660912570 conn(0x561c7d1e6bd0 msgr2=0x561c7d216020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 -- 172.21.15.45:0/660912570 shutdown_connections 2023-10-06T05:22:10.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.331+0000 7f8522251040 1 -- 172.21.15.45:0/660912570 wait complete. 2023-10-06T05:22:10.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.332+0000 7f8522251040 1 Processor -- start 2023-10-06T05:22:10.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.332+0000 7f8522251040 1 -- start start 2023-10-06T05:22:10.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f8522251040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d215c50 0x561c7d2a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f8522251040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc0c0 0x561c7d2cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f8522251040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d2bc730 0x561c7d2cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f8522251040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d22a7f0 con 0x561c7d2bc0c0 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f8522251040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d22a010 con 0x561c7d2bc730 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f8522251040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c7d22a2b0 con 0x561c7d215c50 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc0c0 0x561c7d2cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc0c0 0x561c7d2cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55752/0 (socket says 172.21.15.45:55752) 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 -- 172.21.15.45:0/3478001907 learned_addr learned my addr 172.21.15.45:0/3478001907 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851b46f700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d215c50 0x561c7d2a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851bc70700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d2bc730 0x561c7d2cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 -- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d215c50 msgr2=0x561c7d2a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c7d215c50 0x561c7d2a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 -- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d2bc730 msgr2=0x561c7d2cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c7d2bc730 0x561c7d2cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.333+0000 7f851ac6e700 1 -- 172.21.15.45:0/3478001907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7d15a1c0 con 0x561c7d2bc0c0 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.334+0000 7f851ac6e700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc0c0 0x561c7d2cb5a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8504008af0 tx=0x7f850400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.334+0000 7f8513fff700 1 -- 172.21.15.45:0/3478001907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400f040 con 0x561c7d2bc0c0 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.334+0000 7f8513fff700 1 -- 172.21.15.45:0/3478001907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8504009570 con 0x561c7d2bc0c0 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.334+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851400a0e0 con 0x561c7d2bc0c0 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.334+0000 7f8513fff700 1 -- 172.21.15.45:0/3478001907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504010ae0 con 0x561c7d2bc0c0 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.334+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c7d1403d0 con 0x561c7d2bc0c0 2023-10-06T05:22:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.335+0000 7f8513fff700 1 -- 172.21.15.45:0/3478001907 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8504015450 con 0x561c7d2bc0c0 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.336+0000 7f8513fff700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84fc065890 0x7f84fc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.336+0000 7f8513fff700 1 -- 172.21.15.45:0/3478001907 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8504084000 con 0x561c7d2bc0c0 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.336+0000 7f851b46f700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84fc065890 0x7f84fc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.336+0000 7f8522251040 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c7d29d950 0x561c7d227880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.336+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x561c7d2297f0 con 0x561c7d29d950 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.336+0000 7f851bc70700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c7d29d950 0x561c7d227880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.336+0000 7f851b46f700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84fc065890 0x7f84fc067d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f850c000c00 tx=0x7f850c001950 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f851bc70700 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c7d29d950 0x561c7d227880 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f851bc70700 1 -- 172.21.15.45:0/3478001907 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x7f8514020030 con 0x561c7d29d950 2023-10-06T05:22:10.343 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/100.00000000.inode mtime 2023-10-06T05:21:51.000000+0000, size 554 2023-10-06T05:22:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c7d29d950 msgr2=0x561c7d227880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c7d29d950 0x561c7d227880 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84fc065890 msgr2=0x7f84fc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f84fc065890 0x7f84fc067d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f850c000c00 tx=0x7f850c001950 comp rx=0 tx=0).stop 2023-10-06T05:22:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc0c0 msgr2=0x561c7d2cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 --2- 172.21.15.45:0/3478001907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c7d2bc0c0 0x561c7d2cb5a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8504008af0 tx=0x7f850400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f851bc70700 1 -- 172.21.15.45:0/3478001907 reap_dead start 2023-10-06T05:22:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 shutdown_connections 2023-10-06T05:22:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 >> 172.21.15.45:0/3478001907 conn(0x561c7d1e6bd0 msgr2=0x561c7d216020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 shutdown_connections 2023-10-06T05:22:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.337+0000 7f8522251040 1 -- 172.21.15.45:0/3478001907 wait complete. 2023-10-06T05:22:10.345 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2023-10-06T05:22:10.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.388+0000 7f7c56567040 1 Processor -- start 2023-10-06T05:22:10.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.388+0000 7f7c56567040 1 -- start start 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c56567040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba057e0 0x564bdb9ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c56567040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90b30 0x564bdb9c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c56567040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba90f00 0x564bdba8c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c56567040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdb908620 con 0x564bdba90b30 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c56567040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdb8ff400 con 0x564bdba057e0 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c56567040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bdba8ce50 con 0x564bdba90f00 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90b30 0x564bdb9c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90b30 0x564bdb9c0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55770/0 (socket says 172.21.15.45:55770) 2023-10-06T05:22:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 -- 172.21.15.45:0/2312616749 learned_addr learned my addr 172.21.15.45:0/2312616749 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4f785700 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba057e0 0x564bdb9ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ff86700 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba90f00 0x564bdba8c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 -- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba90f00 msgr2=0x564bdba8c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba90f00 0x564bdba8c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 -- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba057e0 msgr2=0x564bdb9ecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba057e0 0x564bdb9ecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 -- 172.21.15.45:0/2312616749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c44005db0 con 0x564bdba90b30 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4f785700 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba057e0 0x564bdb9ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ff86700 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba90f00 0x564bdba8c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.389+0000 7f7c4ef84700 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90b30 0x564bdb9c0570 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7c44008410 tx=0x7f7c44016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9289512ea041b8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c4df82700 1 -- 172.21.15.45:0/2312616749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44008a50 con 0x564bdba90b30 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c4df82700 1 -- 172.21.15.45:0/2312616749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c44008bf0 con 0x564bdba90b30 2023-10-06T05:22:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c4df82700 1 -- 172.21.15.45:0/2312616749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4401c690 con 0x564bdba90b30 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 -- 172.21.15.45:0/2312616749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90b30 msgr2=0x564bdb9c0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90b30 0x564bdb9c0570 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7c44008410 tx=0x7f7c44016000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 -- 172.21.15.45:0/2312616749 shutdown_connections 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba90f00 0x564bdba8c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba057e0 0x564bdb9ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 --2- 172.21.15.45:0/2312616749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90b30 0x564bdb9c0570 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 -- 172.21.15.45:0/2312616749 >> 172.21.15.45:0/2312616749 conn(0x564bdb9be450 msgr2=0x564bdb9be820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 -- 172.21.15.45:0/2312616749 shutdown_connections 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.390+0000 7f7c56567040 1 -- 172.21.15.45:0/2312616749 wait complete. 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.391+0000 7f7c56567040 1 Processor -- start 2023-10-06T05:22:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.391+0000 7f7c56567040 1 -- start start 2023-10-06T05:22:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c56567040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba057e0 0x564bdba02a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c56567040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba90b30 0x564bdba03fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c56567040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90f00 0x564bdba010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c56567040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdba03320 con 0x564bdba90f00 2023-10-06T05:22:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c56567040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdba035c0 con 0x564bdba90b30 2023-10-06T05:22:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c56567040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bdba03860 con 0x564bdba057e0 2023-10-06T05:22:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90f00 0x564bdba010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90f00 0x564bdba010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55786/0 (socket says 172.21.15.45:55786) 2023-10-06T05:22:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 -- 172.21.15.45:0/3204705927 learned_addr learned my addr 172.21.15.45:0/3204705927 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ef84700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba90b30 0x564bdba03fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4f785700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba057e0 0x564bdba02a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 -- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba057e0 msgr2=0x564bdba02a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564bdba057e0 0x564bdba02a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 -- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba90b30 msgr2=0x564bdba03fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bdba90b30 0x564bdba03fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.392+0000 7f7c4ff86700 1 -- 172.21.15.45:0/3204705927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bdb90a750 con 0x564bdba90f00 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.393+0000 7f7c4ff86700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90f00 0x564bdba010c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800d4f0 tx=0x7f7c48007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.393+0000 7f7c37fff700 1 -- 172.21.15.45:0/3204705927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800b070 con 0x564bdba90f00 2023-10-06T05:22:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.393+0000 7f7c37fff700 1 -- 172.21.15.45:0/3204705927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4800ddc0 con 0x564bdba90f00 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.393+0000 7f7c37fff700 1 -- 172.21.15.45:0/3204705927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48012640 con 0x564bdba90f00 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.393+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564bdb9311c0 con 0x564bdba90f00 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.393+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bdb9173d0 con 0x564bdba90f00 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.394+0000 7f7c37fff700 1 -- 172.21.15.45:0/3204705927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c480129f0 con 0x564bdba90f00 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.395+0000 7f7c37fff700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c3c0658c0 0x7f7c3c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.395+0000 7f7c37fff700 1 -- 172.21.15.45:0/3204705927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f7c48083280 con 0x564bdba90f00 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.395+0000 7f7c4f785700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c3c0658c0 0x7f7c3c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.395+0000 7f7c56567040 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564bdb9bf160 0x564bdb9bf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.395+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x564bdb9fe2c0 con 0x564bdb9bf160 2023-10-06T05:22:10.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.395+0000 7f7c4ef84700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564bdb9bf160 0x564bdb9bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c4f785700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c3c0658c0 0x7f7c3c067d70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005520 tx=0x7f7c4000d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c4ef84700 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564bdb9bf160 0x564bdb9bf530 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c4ef84700 1 -- 172.21.15.45:0/3204705927 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+16 (crc 0 0 0) 0x7f7c4401c440 con 0x564bdb9bf160 2023-10-06T05:22:10.403 INFO:teuthology.orchestra.run.smithi045.stdout:cephfs_metadata/1.00000000 mtime 2023-10-06T05:22:06.000000+0000, size 0 2023-10-06T05:22:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564bdb9bf160 msgr2=0x564bdb9bf530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x564bdb9bf160 0x564bdb9bf530 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c3c0658c0 msgr2=0x7f7c3c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c3c0658c0 0x7f7c3c067d70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005520 tx=0x7f7c4000d040 comp rx=0 tx=0).stop 2023-10-06T05:22:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90f00 msgr2=0x564bdba010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 --2- 172.21.15.45:0/3204705927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bdba90f00 0x564bdba010c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800d4f0 tx=0x7f7c48007040 comp rx=0 tx=0).stop 2023-10-06T05:22:10.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c4ff86700 1 -- 172.21.15.45:0/3204705927 reap_dead start 2023-10-06T05:22:10.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 shutdown_connections 2023-10-06T05:22:10.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 >> 172.21.15.45:0/3204705927 conn(0x564bdb9be450 msgr2=0x564bdb9edd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 shutdown_connections 2023-10-06T05:22:10.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.396+0000 7f7c56567040 1 -- 172.21.15.45:0/3204705927 wait complete. 2023-10-06T05:22:10.407 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2023-10-06T05:22:10.407 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2023-10-06T05:22:10.407 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2023-10-06T05:22:10.408 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2023-10-06T05:22:10.409 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2023-10-06T05:22:10.409 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2023-10-06T05:22:10.409 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2023-10-06T05:22:10.409 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2023-10-06T05:22:10.409 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2023-10-06T05:22:10.409 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:10.409 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:10.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:10.409 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:10.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:22:10.430 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:22:10.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.607+0000 7faa236f6700 1 Processor -- start 2023-10-06T05:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.607+0000 7faa236f6700 1 -- start start 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa236f6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa1c0c93c0 0x7faa1c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa236f6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c15b000 con 0x7faa1c157030 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa236f6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c15b2e0 con 0x7faa1c0c8ff0 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa236f6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c0c9830 con 0x7faa1c0c93c0 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa21c93700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faa1c157030 0x7faa1c0cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42604/0 (socket says 172.21.15.45:42604) 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa21c93700 1 -- 172.21.15.45:0/2586537690 learned_addr learned my addr 172.21.15.45:0/2586537690 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa20c91700 1 --2- 172.21.15.45:0/2586537690 >> v2:172.21.15.144:3300/0 conn(0x7faa1c0c93c0 0x7faa1c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa13fff700 1 -- 172.21.15.45:0/2586537690 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430823099 0 0) 0x7faa1c15b000 con 0x7faa1c157030 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa13fff700 1 -- 172.21.15.45:0/2586537690 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa08003570 con 0x7faa1c157030 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa13fff700 1 -- 172.21.15.45:0/2586537690 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013460547 0 0) 0x7faa1c15b2e0 con 0x7faa1c0c8ff0 2023-10-06T05:22:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.608+0000 7faa13fff700 1 -- 172.21.15.45:0/2586537690 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa1c15b000 con 0x7faa1c0c8ff0 2023-10-06T05:22:10.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa20c91700 1 -- 172.21.15.45:0/2586537690 >> v1:172.21.15.144:6789/0 conn(0x7faa1c0c8ff0 legacy=0x7faa1c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa20c91700 1 -- 172.21.15.45:0/2586537690 >> v1:172.21.15.45:6789/0 conn(0x7faa1c157030 legacy=0x7faa1c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa20c91700 1 -- 172.21.15.45:0/2586537690 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c15b5c0 con 0x7faa1c0c93c0 2023-10-06T05:22:10.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa20c91700 1 --2- 172.21.15.45:0/2586537690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c93c0 0x7faa1c156af0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0098e0 tx=0x7faa0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a506d1f7b16d0e82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa13fff700 1 -- 172.21.15.45:0/2586537690 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011630 con 0x7faa1c0c93c0 2023-10-06T05:22:10.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa13fff700 1 -- 172.21.15.45:0/2586537690 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c011790 con 0x7faa1c0c93c0 2023-10-06T05:22:10.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa13fff700 1 -- 172.21.15.45:0/2586537690 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011b30 con 0x7faa1c0c93c0 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa236f6700 1 -- 172.21.15.45:0/2586537690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c93c0 msgr2=0x7faa1c156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.609+0000 7faa236f6700 1 --2- 172.21.15.45:0/2586537690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c93c0 0x7faa1c156af0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0098e0 tx=0x7faa0c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.610+0000 7faa236f6700 1 -- 172.21.15.45:0/2586537690 shutdown_connections 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.610+0000 7faa236f6700 1 --2- 172.21.15.45:0/2586537690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c93c0 0x7faa1c156af0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.610+0000 7faa236f6700 1 -- 172.21.15.45:0/2586537690 >> 172.21.15.45:0/2586537690 conn(0x7faa1c0cdf50 msgr2=0x7faa1c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.610+0000 7faa236f6700 1 -- 172.21.15.45:0/2586537690 shutdown_connections 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.610+0000 7faa236f6700 1 -- 172.21.15.45:0/2586537690 wait complete. 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.611+0000 7faa236f6700 1 Processor -- start 2023-10-06T05:22:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.611+0000 7faa236f6700 1 -- start start 2023-10-06T05:22:10.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.612+0000 7faa236f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c8ff0 0x7faa1c151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa236f6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa1c0c93c0 0x7faa1c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa236f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1c157030 0x7faa1c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa236f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c159600 con 0x7faa1c157030 2023-10-06T05:22:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa236f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0b3850 con 0x7faa1c0c8ff0 2023-10-06T05:22:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa236f6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0b3b30 con 0x7faa1c0c93c0 2023-10-06T05:22:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1c157030 0x7faa1c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1c157030 0x7faa1c06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55800/0 (socket says 172.21.15.45:55800) 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 -- 172.21.15.45:0/2862187084 learned_addr learned my addr 172.21.15.45:0/2862187084 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa20c91700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa1c0c93c0 0x7faa1c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21492700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c8ff0 0x7faa1c151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 -- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa1c0c93c0 msgr2=0x7faa1c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa1c0c93c0 0x7faa1c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 -- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c8ff0 msgr2=0x7faa1c151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c8ff0 0x7faa1c151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 -- 172.21.15.45:0/2862187084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c009420 con 0x7faa1c157030 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa21c93700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1c157030 0x7faa1c06dc20 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7faa1400cc60 tx=0x7faa14001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa117fa700 1 -- 172.21.15.45:0/2862187084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1400a8b0 con 0x7faa1c157030 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa117fa700 1 -- 172.21.15.45:0/2862187084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1400aa30 con 0x7faa1c157030 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c158fb0 con 0x7faa1c157030 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.613+0000 7faa117fa700 1 -- 172.21.15.45:0/2862187084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1400adf0 con 0x7faa1c157030 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.614+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c0b1230 con 0x7faa1c157030 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.614+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c158e10 con 0x7faa1c157030 2023-10-06T05:22:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.615+0000 7faa117fa700 1 -- 172.21.15.45:0/2862187084 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa14007300 con 0x7faa1c157030 2023-10-06T05:22:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.615+0000 7faa117fa700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.615+0000 7faa117fa700 1 -- 172.21.15.45:0/2862187084 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7faa14084c60 con 0x7faa1c157030 2023-10-06T05:22:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.616+0000 7faa21492700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.616+0000 7faa21492700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa00066f30 0x7faa000693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0c8920 tx=0x7faa18007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.620+0000 7faa117fa700 1 -- 172.21.15.45:0/2862187084 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa1c158e10 con 0x7faa1c157030 2023-10-06T05:22:10.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:10.869+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faa1c158e10 con 0x7faa1c157030 2023-10-06T05:22:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.861+0000 7faa117fa700 1 -- 172.21.15.45:0/2862187084 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) v1 ==== 161+0+0 (secure 0 0 0) 0x7faa1c158e10 con 0x7faa1c157030 2023-10-06T05:22:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa00066f30 msgr2=0x7faa000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa00066f30 0x7faa000693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0c8920 tx=0x7faa18007d50 comp rx=0 tx=0).stop 2023-10-06T05:22:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1c157030 msgr2=0x7faa1c06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1c157030 0x7faa1c06dc20 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7faa1400cc60 tx=0x7faa14001ae0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 shutdown_connections 2023-10-06T05:22:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1c157030 0x7faa1c06dc20 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa1c0c8ff0 0x7faa1c151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 --2- 172.21.15.45:0/2862187084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa1c0c93c0 0x7faa1c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 >> 172.21.15.45:0/2862187084 conn(0x7faa1c0cdf50 msgr2=0x7faa1c155c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 shutdown_connections 2023-10-06T05:22:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:11.864+0000 7faa236f6700 1 -- 172.21.15.45:0/2862187084 wait complete. 2023-10-06T05:22:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:11.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:22:12.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.016+0000 7f52947df700 1 Processor -- start 2023-10-06T05:22:12.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.016+0000 7f52947df700 1 -- start start 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f52947df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c153650 0x7f528c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f52947df700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f528c06afa0 0x7f528c068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f52947df700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c0676c0 con 0x7f528c153a20 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f52947df700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c0679a0 con 0x7f528c06afa0 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f52947df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c067c80 con 0x7f528c153650 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f5291d7a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f528c153a20 0x7f528c06a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42612/0 (socket says 172.21.15.45:42612) 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f5291d7a700 1 -- 172.21.15.45:0/519269887 learned_addr learned my addr 172.21.15.45:0/519269887 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f529257b700 1 --2- 172.21.15.45:0/519269887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c153650 0x7f528c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.017+0000 7f5292d7c700 1 --2- 172.21.15.45:0/519269887 >> v2:172.21.15.144:3300/0 conn(0x7f528c06afa0 0x7f528c068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f5291579700 1 -- 172.21.15.45:0/519269887 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455260569 0 0) 0x7f528c0676c0 con 0x7f528c153a20 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f5291579700 1 -- 172.21.15.45:0/519269887 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5278003570 con 0x7f528c153a20 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f529257b700 1 -- 172.21.15.45:0/519269887 >> v2:172.21.15.144:3300/0 conn(0x7f528c06afa0 msgr2=0x7f528c068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f529257b700 1 --2- 172.21.15.45:0/519269887 >> v2:172.21.15.144:3300/0 conn(0x7f528c06afa0 0x7f528c068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f529257b700 1 -- 172.21.15.45:0/519269887 >> v1:172.21.15.45:6789/0 conn(0x7f528c153a20 legacy=0x7f528c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f529257b700 1 -- 172.21.15.45:0/519269887 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c067f60 con 0x7f528c153650 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f5292d7c700 1 --2- 172.21.15.45:0/519269887 >> v2:172.21.15.144:3300/0 conn(0x7f528c06afa0 0x7f528c068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f529257b700 1 --2- 172.21.15.45:0/519269887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c153650 0x7f528c15a930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f52880098e0 tx=0x7f52880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a539828c3070173 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f5291579700 1 -- 172.21.15.45:0/519269887 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011630 con 0x7f528c153650 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f5291579700 1 -- 172.21.15.45:0/519269887 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288011790 con 0x7f528c153650 2023-10-06T05:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.018+0000 7f5291579700 1 -- 172.21.15.45:0/519269887 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011b30 con 0x7f528c153650 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 -- 172.21.15.45:0/519269887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c153650 msgr2=0x7f528c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 --2- 172.21.15.45:0/519269887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c153650 0x7f528c15a930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f52880098e0 tx=0x7f52880091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 -- 172.21.15.45:0/519269887 shutdown_connections 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 --2- 172.21.15.45:0/519269887 >> v2:172.21.15.144:3300/0 conn(0x7f528c06afa0 0x7f528c068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 --2- 172.21.15.45:0/519269887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c153650 0x7f528c15a930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 -- 172.21.15.45:0/519269887 >> 172.21.15.45:0/519269887 conn(0x7f528c0cc5a0 msgr2=0x7f528c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 -- 172.21.15.45:0/519269887 shutdown_connections 2023-10-06T05:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.019+0000 7f52947df700 1 -- 172.21.15.45:0/519269887 wait complete. 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.020+0000 7f52947df700 1 Processor -- start 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.020+0000 7f52947df700 1 -- start start 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f52947df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c06afa0 0x7f528c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f52947df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c153650 0x7f528c0c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f52947df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c153a20 0x7f528c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f52947df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0aebf0 con 0x7f528c153a20 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f52947df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0c0c80 con 0x7f528c153650 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f52947df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0c0070 con 0x7f528c06afa0 2023-10-06T05:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c153a20 0x7f528c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c153a20 0x7f528c0c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55812/0 (socket says 172.21.15.45:55812) 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 -- 172.21.15.45:0/1014397400 learned_addr learned my addr 172.21.15.45:0/1014397400 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5291d7a700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c153650 0x7f528c0c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f529257b700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c06afa0 0x7f528c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 -- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c06afa0 msgr2=0x7f528c152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c06afa0 0x7f528c152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 -- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c153650 msgr2=0x7f528c0c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c153650 0x7f528c0c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 -- 172.21.15.45:0/1014397400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5284009a20 con 0x7f528c153a20 2023-10-06T05:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.021+0000 7f5292d7c700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c153a20 0x7f528c0c9480 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f528400ef90 tx=0x7f528400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.022+0000 7f52827fc700 1 -- 172.21.15.45:0/1014397400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528400cd30 con 0x7f528c153a20 2023-10-06T05:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.022+0000 7f52827fc700 1 -- 172.21.15.45:0/1014397400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528400f040 con 0x7f528c153a20 2023-10-06T05:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.022+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5288009420 con 0x7f528c153a20 2023-10-06T05:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.022+0000 7f52827fc700 1 -- 172.21.15.45:0/1014397400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52840146b0 con 0x7f528c153a20 2023-10-06T05:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.022+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c0c0630 con 0x7f528c153a20 2023-10-06T05:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.023+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c0c0e10 con 0x7f528c153a20 2023-10-06T05:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.023+0000 7f52827fc700 1 -- 172.21.15.45:0/1014397400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5284014ae0 con 0x7f528c153a20 2023-10-06T05:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.024+0000 7f52827fc700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5270066f30 0x7f52700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.024+0000 7f52827fc700 1 -- 172.21.15.45:0/1014397400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5284085680 con 0x7f528c153a20 2023-10-06T05:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.024+0000 7f529257b700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5270066f30 0x7f52700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.024+0000 7f529257b700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5270066f30 0x7f52700693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f52880091e0 tx=0x7f5288017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.028+0000 7f52827fc700 1 -- 172.21.15.45:0/1014397400 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5284052600 con 0x7f528c153a20 2023-10-06T05:22:12.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.313+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f528c0cc0e0 con 0x7f528c153a20 2023-10-06T05:22:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.868+0000 7f52827fc700 1 -- 172.21.15.45:0/1014397400 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v90) v1 ==== 107+0+0 (secure 0 0 0) 0x7f5284055c20 con 0x7f528c153a20 2023-10-06T05:22:12.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.870+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5270066f30 msgr2=0x7f52700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.870+0000 7f52947df700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5270066f30 0x7f52700693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f52880091e0 tx=0x7f5288017000 comp rx=0 tx=0).stop 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c153a20 msgr2=0x7f528c0c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c153a20 0x7f528c0c9480 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f528400ef90 tx=0x7f528400b040 comp rx=0 tx=0).stop 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 shutdown_connections 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5270066f30 0x7f52700693e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c153a20 0x7f528c0c9480 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c06afa0 0x7f528c152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 --2- 172.21.15.45:0/1014397400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c153650 0x7f528c0c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 >> 172.21.15.45:0/1014397400 conn(0x7f528c0cc5a0 msgr2=0x7f528c15e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 shutdown_connections 2023-10-06T05:22:12.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.871+0000 7f52947df700 1 -- 172.21.15.45:0/1014397400 wait complete. 2023-10-06T05:22:12.875 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:22:12.889 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:22:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.932+0000 7f5aab90f040 1 Processor -- start 2023-10-06T05:22:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5aab90f040 1 -- start start 2023-10-06T05:22:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5aab90f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e96c9b0 0x55825e962d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5aab90f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f5b70 0x55825e9698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5aab90f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e9f9f70 0x55825e969e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5aab90f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825e86c980 con 0x55825e9f5b70 2023-10-06T05:22:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5aab90f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825e86c620 con 0x55825e96c9b0 2023-10-06T05:22:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5aab90f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55825e863400 con 0x55825e9f9f70 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5a9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f5b70 0x55825e9698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5a9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f5b70 0x55825e9698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55842/0 (socket says 172.21.15.45:55842) 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.933+0000 7f5a9ffff700 1 -- 172.21.15.45:0/3672692398 learned_addr learned my addr 172.21.15.45:0/3672692398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e96c9b0 0x55825e962d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5aa532e700 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e9f9f70 0x55825e969e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9ffff700 1 -- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e9f9f70 msgr2=0x55825e969e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e9f9f70 0x55825e969e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9ffff700 1 -- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e96c9b0 msgr2=0x55825e962d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e96c9b0 0x55825e962d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9ffff700 1 -- 172.21.15.45:0/3672692398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98005db0 con 0x55825e9f5b70 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e96c9b0 0x55825e962d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f5b70 0x55825e9698f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800ad70 tx=0x7f5a98016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbec24a76910bab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9effd700 1 -- 172.21.15.45:0/3672692398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980055f0 con 0x55825e9f5b70 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9effd700 1 -- 172.21.15.45:0/3672692398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a98000bc0 con 0x55825e9f5b70 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.934+0000 7f5a9effd700 1 -- 172.21.15.45:0/3672692398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98008c00 con 0x55825e9f5b70 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 -- 172.21.15.45:0/3672692398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f5b70 msgr2=0x55825e9698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f5b70 0x55825e9698f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800ad70 tx=0x7f5a98016000 comp rx=0 tx=0).stop 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 -- 172.21.15.45:0/3672692398 shutdown_connections 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e9f9f70 0x55825e969e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e96c9b0 0x55825e962d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 --2- 172.21.15.45:0/3672692398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f5b70 0x55825e9698f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 -- 172.21.15.45:0/3672692398 >> 172.21.15.45:0/3672692398 conn(0x55825e9235f0 msgr2=0x55825e925310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:12.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 -- 172.21.15.45:0/3672692398 shutdown_connections 2023-10-06T05:22:12.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 -- 172.21.15.45:0/3672692398 wait complete. 2023-10-06T05:22:12.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 Processor -- start 2023-10-06T05:22:12.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.935+0000 7f5aab90f040 1 -- start start 2023-10-06T05:22:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aab90f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e96c9b0 0x55825e9f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aab90f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e9f9f70 0x55825e9f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aab90f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f2f50 0x55825e94e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aab90f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825ea09930 con 0x55825e9f2f50 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aab90f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825e968d10 con 0x55825e9f9f70 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aab90f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55825e969410 con 0x55825e96c9b0 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aa532e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f2f50 0x55825e94e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aa532e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f2f50 0x55825e94e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55858/0 (socket says 172.21.15.45:55858) 2023-10-06T05:22:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 learned_addr learned my addr 172.21.15.45:0/815956552 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e96c9b0 0x55825e9f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.936+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e9f9f70 0x55825e9f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e96c9b0 msgr2=0x55825e9f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aa532e700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55825e96c9b0 0x55825e9f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e9f9f70 msgr2=0x55825e9f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aa532e700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55825e9f9f70 0x55825e9f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55825e8951c0 con 0x55825e9f2f50 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aa532e700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f2f50 0x55825e94e050 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b440 tx=0x7f5aa0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5a9d7fa700 1 -- 172.21.15.45:0/815956552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000be50 con 0x55825e9f2f50 2023-10-06T05:22:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5a9d7fa700 1 -- 172.21.15.45:0/815956552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa0012420 con 0x55825e9f2f50 2023-10-06T05:22:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55825e87b3d0 con 0x55825e9f2f50 2023-10-06T05:22:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5a9d7fa700 1 -- 172.21.15.45:0/815956552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00127e0 con 0x55825e9f2f50 2023-10-06T05:22:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.937+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55825e86eae0 con 0x55825e9f2f50 2023-10-06T05:22:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.939+0000 7f5a9d7fa700 1 -- 172.21.15.45:0/815956552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5aa0012b50 con 0x55825e9f2f50 2023-10-06T05:22:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.939+0000 7f5a9d7fa700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a8c0658c0 0x7f5a8c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.939+0000 7f5a9d7fa700 1 -- 172.21.15.45:0/815956552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5aa00836c0 con 0x55825e9f2f50 2023-10-06T05:22:12.946 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.939+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a8c0658c0 0x7f5a8c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.939+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55825e9223d0 0x55825ea0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.939+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea0ade0 con 0x55825e9223d0 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.940+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55825e9223d0 0x55825ea0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.940+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a8c0658c0 0x7f5a8c067d70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009c20 tx=0x7f5a9000c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.940+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55825e9223d0 0x55825ea0a8a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.942+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v93'11 uv4 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.942+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea0b1c0 con 0x55825e9223d0 2023-10-06T05:22:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.943+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v93'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.943+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea15550 con 0x55825e9223d0 2023-10-06T05:22:12.949 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:22:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.944+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v93'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.944+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea16910 con 0x55825e9223d0 2023-10-06T05:22:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.945+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v93'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.945+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea16cc0 con 0x55825e9223d0 2023-10-06T05:22:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.947+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v93'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.947+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825e94e7e0 con 0x55825e9223d0 2023-10-06T05:22:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.948+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v93'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.948+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea15df0 con 0x55825e9223d0 2023-10-06T05:22:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.949+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v93'17 uv6 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.949+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.950+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v93'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.950+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.954 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:22:12.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.951+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v93'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.951+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.952+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v93'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.953+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.953+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v93'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.953+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.955+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v93'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.955+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.955+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v93'23 uv10 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.956+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.957+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v93'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.957+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.958+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v93'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.961 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:22:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.958+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.959+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v93'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.959+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.960+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v93'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.960+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea161a0 con 0x55825e9223d0 2023-10-06T05:22:12.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.961+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v93'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.961+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.962+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v93'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.962+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.963+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v93'30 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.963+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.964+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v93'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.964+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.968 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:22:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.966+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v93'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.966+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea18080 con 0x55825e9223d0 2023-10-06T05:22:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.967+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v93'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.967+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea18080 con 0x55825e9223d0 2023-10-06T05:22:12.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.968+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v93'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.968+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.970+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v93'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.970+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.971+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v93'36 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.971+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.973+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v93'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.973+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.976 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:22:12.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.974+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v93'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.974+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea18080 con 0x55825e9223d0 2023-10-06T05:22:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.975+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v93'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.975+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea18080 con 0x55825e9223d0 2023-10-06T05:22:12.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.976+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v93'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.976+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea17070 con 0x55825e9223d0 2023-10-06T05:22:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.977+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v93'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825e9223d0 2023-10-06T05:22:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.977+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55825ea17580 0x55825ea19960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.977+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea19ea0 con 0x55825ea17580 2023-10-06T05:22:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.977+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55825ea17580 0x55825ea19960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.977+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55825ea17580 0x55825ea19960 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.979+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v93'20 uv17 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.979+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1a250 con 0x55825ea17580 2023-10-06T05:22:12.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.980+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v93'21 uv21 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:12.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.980+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1a600 con 0x55825ea17580 2023-10-06T05:22:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.982+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v93'22 uv22 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:12.985 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:22:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.982+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1aea0 con 0x55825ea17580 2023-10-06T05:22:12.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.985+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v93'23 uv23 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:12.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.985+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1b250 con 0x55825ea17580 2023-10-06T05:22:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.990+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v93'24 uv24 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.990+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1b600 con 0x55825ea17580 2023-10-06T05:22:12.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.992+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v93'25 uv19 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:12.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.992+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1b9b0 con 0x55825ea17580 2023-10-06T05:22:12.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.994+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v93'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:12.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.994+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:12.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.997+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v93'27 uv27 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.000 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:22:13.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:12.997+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.003+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v93'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.003+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.004+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v93'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.004+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.007+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v93'30 uv15 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.007+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.009+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v93'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.009+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.012+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v93'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.015 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:22:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.012+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f480 con 0x55825ea17580 2023-10-06T05:22:13.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.014+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v93'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.015+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f480 con 0x55825ea17580 2023-10-06T05:22:13.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.016+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v93'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.016+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.017+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v93'35 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.017+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.019+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v93'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.019+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.023 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:22:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.020+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v93'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.020+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1cd70 con 0x55825ea17580 2023-10-06T05:22:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.021+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v93'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.021+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1cd70 con 0x55825ea17580 2023-10-06T05:22:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.022+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v93'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.023+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1bd60 con 0x55825ea17580 2023-10-06T05:22:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.024+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v93'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.024+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55825ea1c270 0x55825ea1e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.024+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1eb90 con 0x55825ea1c270 2023-10-06T05:22:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.024+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55825ea1c270 0x55825ea1e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.024+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55825ea1c270 0x55825ea1e650 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.025+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.026+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1ebd0 con 0x55825ea1c270 2023-10-06T05:22:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.027+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.027+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea1c270 2023-10-06T05:22:13.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.029+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.032 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:22:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.029+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea200a0 con 0x55825ea1c270 2023-10-06T05:22:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.030+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.030+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea200a0 con 0x55825ea1c270 2023-10-06T05:22:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.031+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.031+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea1c270 2023-10-06T05:22:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.032+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.032+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea1c270 2023-10-06T05:22:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.033+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.033+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea1c270 2023-10-06T05:22:13.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.035+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v93'4 uv4 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.035+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea1c270 2023-10-06T05:22:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.036+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v93'5 uv5 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.039 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:22:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.036+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea1c270 2023-10-06T05:22:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.037+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v93'6 uv6 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.037+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea17580 2023-10-06T05:22:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.039+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.039+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea17580 2023-10-06T05:22:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.040+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.040+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea17580 2023-10-06T05:22:13.044 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:22:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.042+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.042+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea200a0 con 0x55825ea17580 2023-10-06T05:22:13.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.044+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.044+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea200a0 con 0x55825ea17580 2023-10-06T05:22:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.045+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.045+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea1f090 con 0x55825ea17580 2023-10-06T05:22:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.046+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea17580 2023-10-06T05:22:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.046+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55825ea1f710 0x55825ea21af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.046+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea22070 con 0x55825ea1f710 2023-10-06T05:22:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.047+0000 7f5aa532e700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55825ea1f710 0x55825ea21af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.047+0000 7f5aa532e700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55825ea1f710 0x55825ea21af0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.050+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v93'4 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.050+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea22540 con 0x55825ea1f710 2023-10-06T05:22:13.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.050+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.050+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea22a30 con 0x55825ea1f710 2023-10-06T05:22:13.053 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:22:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.052+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.052+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea24330 con 0x55825ea1f710 2023-10-06T05:22:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.053+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.053+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea246e0 con 0x55825ea1f710 2023-10-06T05:22:13.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.054+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.054+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea23320 con 0x55825ea1f710 2023-10-06T05:22:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.057+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v93'9 uv9 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.057+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea23720 con 0x55825ea1f710 2023-10-06T05:22:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.058+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v93'10 uv3 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.058+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea23720 con 0x55825ea1f710 2023-10-06T05:22:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.058+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v93'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.058+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea23720 con 0x55825ea1f710 2023-10-06T05:22:13.062 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:22:13.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.059+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v93'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.059+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea23720 con 0x55825ea1f710 2023-10-06T05:22:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.060+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v93'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.061+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea23720 con 0x55825ea1f710 2023-10-06T05:22:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.064+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v93'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.064+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55825ea24a90 0x55825ea26e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.064+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea24a90 2023-10-06T05:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.064+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55825ea24a90 0x55825ea26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.064+0000 7f5aa4b2d700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55825ea24a90 0x55825ea26e70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.065+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v93'2 uv1 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea24a90 2023-10-06T05:22:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.065+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea24a90 2023-10-06T05:22:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.066+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v93'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea24a90 2023-10-06T05:22:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.067+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea24a90 2023-10-06T05:22:13.072 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:22:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.070+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v93'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea24a90 2023-10-06T05:22:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.070+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea24a90 2023-10-06T05:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.073+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v93'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea24a90 2023-10-06T05:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.073+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea24a90 2023-10-06T05:22:13.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.079+0000 7f5aa4b2d700 1 -- 172.21.15.45:0/815956552 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v93'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a90007960 con 0x55825ea24a90 2023-10-06T05:22:13.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.079+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea1c270 2023-10-06T05:22:13.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.079+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.079+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea1c270 2023-10-06T05:22:13.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.080+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.080+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea1c270 2023-10-06T05:22:13.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.082+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.084 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:22:13.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.082+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea283c0 con 0x55825ea1c270 2023-10-06T05:22:13.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.083+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.083+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea283c0 con 0x55825ea1c270 2023-10-06T05:22:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.084+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.084+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea273b0 con 0x55825ea1c270 2023-10-06T05:22:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.085+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.085+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55825ea27a30 0x55825ea29e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.085+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a3e0 con 0x55825ea27a30 2023-10-06T05:22:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.086+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55825ea27a30 0x55825ea29e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.086+0000 7f5a9ffff700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55825ea27a30 0x55825ea29e10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.087+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v93'4 uv3 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.087+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a420 con 0x55825ea27a30 2023-10-06T05:22:13.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.088+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v93'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.088+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a850 con 0x55825ea27a30 2023-10-06T05:22:13.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.089+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v93'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.092 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:22:13.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.089+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a850 con 0x55825ea27a30 2023-10-06T05:22:13.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.091+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v93'7 uv7 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.091+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a850 con 0x55825ea27a30 2023-10-06T05:22:13.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.094+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.094+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a850 con 0x55825ea27a30 2023-10-06T05:22:13.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.095+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v93'9 uv1 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.095+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a850 con 0x55825ea27a30 2023-10-06T05:22:13.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.096+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v93'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.096+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a850 con 0x55825ea27a30 2023-10-06T05:22:13.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.097+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v93'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.099 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:22:13.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.097+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2a850 con 0x55825ea27a30 2023-10-06T05:22:13.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.098+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v93'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.098+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2d5c0 con 0x55825ea27a30 2023-10-06T05:22:13.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.099+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v93'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea27a30 2023-10-06T05:22:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.099+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.100+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v93'4 uv3 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.100+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.101+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v93'5 uv5 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.101+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.105 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:22:13.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.103+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v93'6 uv6 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.103+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.104+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v93'7 uv7 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.104+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55825ea2b650 0x55825ea2dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.104+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea2b650 2023-10-06T05:22:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.104+0000 7f5aa532e700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55825ea2b650 0x55825ea2dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.105+0000 7f5aa532e700 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55825ea2b650 0x55825ea2dd10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.106+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea2b650 2023-10-06T05:22:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.106+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea2b650 2023-10-06T05:22:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.110+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea2b650 2023-10-06T05:22:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.110+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea2b650 2023-10-06T05:22:13.116 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:22:13.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.114+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea2b650 2023-10-06T05:22:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.114+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea2b650 2023-10-06T05:22:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.114+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea2b650 2023-10-06T05:22:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.114+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea2b650 2023-10-06T05:22:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.118+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea2b650 2023-10-06T05:22:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.118+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea2b650 2023-10-06T05:22:13.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.123+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea2b650 2023-10-06T05:22:13.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.123+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1f710 2023-10-06T05:22:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.124+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v93'2 uv1 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.124+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1f710 2023-10-06T05:22:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.126+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v93'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.126+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1f710 2023-10-06T05:22:13.129 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:22:13.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.127+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v93'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.127+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1f710 2023-10-06T05:22:13.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.128+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v93'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.128+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1f710 2023-10-06T05:22:13.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.130+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v93'6 uv6 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5aa0018020 con 0x55825ea1f710 2023-10-06T05:22:13.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.130+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.131+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v93'7 uv3 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.131+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.132+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v93'8 uv8 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.132+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.133+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v93'9 uv9 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.136 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:22:13.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.133+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.136+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v93'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.136+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.139+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v93'11 uv11 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.139+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.145+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v93'12 uv6 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.145+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.146+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v93'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.146+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.149+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v93'14 uv14 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.152 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:22:13.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.149+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.150+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v93'15 uv15 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.150+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.155+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v93'16 uv16 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.155+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2ad40 con 0x55825ea1c270 2023-10-06T05:22:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.156+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v93'17 uv17 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.157+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55825ea2e250 con 0x55825ea1c270 2023-10-06T05:22:13.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.159+0000 7f5a9ffff700 1 -- 172.21.15.45:0/815956552 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v93'18 uv18 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5a98008ae0 con 0x55825ea1c270 2023-10-06T05:22:13.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55825ea1c270 msgr2=0x55825ea1e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55825ea1c270 0x55825ea1e650 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55825e9223d0 msgr2=0x55825ea0a8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55825e9223d0 0x55825ea0a8a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55825ea27a30 msgr2=0x55825ea29e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55825ea27a30 0x55825ea29e10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55825ea17580 msgr2=0x55825ea19960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55825ea17580 0x55825ea19960 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 reap_dead start 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55825ea2b650 msgr2=0x55825ea2dd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55825ea2b650 0x55825ea2dd10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aa532e700 1 -- 172.21.15.45:0/815956552 reap_dead start 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55825ea24a90 msgr2=0x55825ea26e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55825ea24a90 0x55825ea26e70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55825ea1f710 msgr2=0x55825ea21af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55825ea1f710 0x55825ea21af0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a8c0658c0 msgr2=0x7f5a8c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a8c0658c0 0x7f5a8c067d70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009c20 tx=0x7f5a9000c040 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f2f50 msgr2=0x55825e94e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f2f50 0x55825e94e050 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b440 tx=0x7f5aa0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 shutdown_connections 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55825ea1f710 0x55825ea21af0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55825ea24a90 0x55825ea26e70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55825e9f2f50 0x55825e94e050 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 --2- 172.21.15.45:0/815956552 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a8c0658c0 0x7f5a8c067d70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 >> 172.21.15.45:0/815956552 conn(0x55825e9235f0 msgr2=0x55825e9698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 shutdown_connections 2023-10-06T05:22:13.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.160+0000 7f5aab90f040 1 -- 172.21.15.45:0/815956552 wait complete. 2023-10-06T05:22:13.167 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2023-10-06T05:22:13.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.214+0000 7f58f49d4040 1 Processor -- start 2023-10-06T05:22:13.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.214+0000 7f58f49d4040 1 -- start start 2023-10-06T05:22:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58f49d4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60e651b0 0x555c60e94c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58f49d4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60eaa5f0 0x555c60e68960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58f49d4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60f26970 0x555c60e69610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58f49d4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c60db1980 con 0x555c60f26970 2023-10-06T05:22:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58f49d4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c60db1620 con 0x555c60eaa5f0 2023-10-06T05:22:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58f49d4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c60da8400 con 0x555c60e651b0 2023-10-06T05:22:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60f26970 0x555c60e69610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60f26970 0x555c60e69610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55860/0 (socket says 172.21.15.45:55860) 2023-10-06T05:22:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 -- 172.21.15.45:0/2150775743 learned_addr learned my addr 172.21.15.45:0/2150775743 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58edbf2700 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60e651b0 0x555c60e94c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ed3f1700 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60eaa5f0 0x555c60e68960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 -- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60e651b0 msgr2=0x555c60e94c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60e651b0 0x555c60e94c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 -- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60eaa5f0 msgr2=0x555c60e68960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60eaa5f0 0x555c60e68960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 -- 172.21.15.45:0/2150775743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e800a0e0 con 0x555c60f26970 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.215+0000 7f58ee3f3700 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60f26970 0x555c60e69610 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f58e800a960 tx=0x7f58e8007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f267caaf2ee5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58dffff700 1 -- 172.21.15.45:0/2150775743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800d960 con 0x555c60f26970 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58dffff700 1 -- 172.21.15.45:0/2150775743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e800dac0 con 0x555c60f26970 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58dffff700 1 -- 172.21.15.45:0/2150775743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800de80 con 0x555c60f26970 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 -- 172.21.15.45:0/2150775743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60f26970 msgr2=0x555c60e69610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60f26970 0x555c60e69610 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f58e800a960 tx=0x7f58e8007040 comp rx=0 tx=0).stop 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 -- 172.21.15.45:0/2150775743 shutdown_connections 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60f26970 0x555c60e69610 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60e651b0 0x555c60e94c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 --2- 172.21.15.45:0/2150775743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60eaa5f0 0x555c60e68960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 -- 172.21.15.45:0/2150775743 >> 172.21.15.45:0/2150775743 conn(0x555c60e674a0 msgr2=0x555c60e67870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 -- 172.21.15.45:0/2150775743 shutdown_connections 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.216+0000 7f58f49d4040 1 -- 172.21.15.45:0/2150775743 wait complete. 2023-10-06T05:22:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.217+0000 7f58f49d4040 1 Processor -- start 2023-10-06T05:22:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.217+0000 7f58f49d4040 1 -- start start 2023-10-06T05:22:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58f49d4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60e651b0 0x555c60f19b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58f49d4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60eaa5f0 0x555c60e687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58f49d4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60f26970 0x555c60e68d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58f49d4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c60e8a5b0 con 0x555c60e651b0 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58f49d4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c60e89690 con 0x555c60f26970 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58f49d4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c60e89930 con 0x555c60eaa5f0 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58edbf2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60e651b0 0x555c60f19b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58ee3f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60f26970 0x555c60e68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58edbf2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60e651b0 0x555c60f19b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55876/0 (socket says 172.21.15.45:55876) 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58ed3f1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60eaa5f0 0x555c60e687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58edbf2700 1 -- 172.21.15.45:0/3752870142 learned_addr learned my addr 172.21.15.45:0/3752870142 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58edbf2700 1 -- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60eaa5f0 msgr2=0x555c60e687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58edbf2700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60eaa5f0 0x555c60e687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58edbf2700 1 -- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60f26970 msgr2=0x555c60e68d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.218+0000 7f58edbf2700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60f26970 0x555c60e68d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58edbf2700 1 -- 172.21.15.45:0/3752870142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e800a5b0 con 0x555c60e651b0 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58ed3f1700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c60eaa5f0 0x555c60e687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58ee3f3700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c60f26970 0x555c60e68d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58edbf2700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60e651b0 0x555c60f19b90 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x555c60e65930 tx=0x7f58e0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58de7fc700 1 -- 172.21.15.45:0/3752870142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0007610 con 0x555c60e651b0 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58de7fc700 1 -- 172.21.15.45:0/3752870142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0007e80 con 0x555c60e651b0 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e0005cf0 con 0x555c60e651b0 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58de7fc700 1 -- 172.21.15.45:0/3752870142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00078e0 con 0x555c60e651b0 2023-10-06T05:22:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.219+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c60dc03d0 con 0x555c60e651b0 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.220+0000 7f58de7fc700 1 -- 172.21.15.45:0/3752870142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f58e0019040 con 0x555c60e651b0 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.221+0000 7f58de7fc700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58d0065890 0x7f58d0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.221+0000 7f58de7fc700 1 -- 172.21.15.45:0/3752870142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f58e0084b70 con 0x555c60e651b0 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.221+0000 7f58ed3f1700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58d0065890 0x7f58d0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.221+0000 7f58f49d4040 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x555c60f24d30 0x555c60e96bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.221+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x555c60e89180 con 0x555c60f24d30 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.221+0000 7f58ee3f3700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x555c60f24d30 0x555c60e96bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.222+0000 7f58ed3f1700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58d0065890 0x7f58d0067d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f58e4004170 tx=0x7f58e4000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.222+0000 7f58ee3f3700 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x555c60f24d30 0x555c60e96bd0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58ee3f3700 1 -- 172.21.15.45:0/3752870142 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v93'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f58e8019070 con 0x555c60f24d30 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x555c60f24d30 msgr2=0x555c60e96bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x555c60f24d30 0x555c60e96bd0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58d0065890 msgr2=0x7f58d0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58d0065890 0x7f58d0067d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f58e4004170 tx=0x7f58e4000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60e651b0 msgr2=0x555c60f19b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 --2- 172.21.15.45:0/3752870142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c60e651b0 0x555c60f19b90 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x555c60e65930 tx=0x7f58e0015000 comp rx=0 tx=0).stop 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58ee3f3700 1 -- 172.21.15.45:0/3752870142 reap_dead start 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 shutdown_connections 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 >> 172.21.15.45:0/3752870142 conn(0x555c60e674a0 msgr2=0x555c60e94260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 shutdown_connections 2023-10-06T05:22:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.223+0000 7f58f49d4040 1 -- 172.21.15.45:0/3752870142 wait complete. 2023-10-06T05:22:13.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:22:13.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.369+0000 7f2451f53700 1 Processor -- start 2023-10-06T05:22:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.369+0000 7f2451f53700 1 -- start start 2023-10-06T05:22:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f2451f53700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c159150 0x7f244c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f2451f53700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f244c159520 0x7f244c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f2451f53700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f244c15d3d0 0x7f244c067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f2451f53700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c0c9c90 con 0x7f244c15d3d0 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f2451f53700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c15d8e0 con 0x7f244c159520 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f2451f53700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c15dbc0 con 0x7f244c159150 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f244c15d3d0 0x7f244c067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f244c15d3d0 0x7f244c067620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55888/0 (socket says 172.21.15.45:55888) 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 -- 172.21.15.45:0/312458679 learned_addr learned my addr 172.21.15.45:0/312458679 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244b7fe700 1 --2- 172.21.15.45:0/312458679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c159150 0x7f244c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244affd700 1 --2- 172.21.15.45:0/312458679 >> v2:172.21.15.144:3300/0 conn(0x7f244c159520 0x7f244c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 -- 172.21.15.45:0/312458679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c159150 msgr2=0x7f244c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 --2- 172.21.15.45:0/312458679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c159150 0x7f244c0ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 -- 172.21.15.45:0/312458679 >> v2:172.21.15.144:3300/0 conn(0x7f244c159520 msgr2=0x7f244c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 --2- 172.21.15.45:0/312458679 >> v2:172.21.15.144:3300/0 conn(0x7f244c159520 0x7f244c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 -- 172.21.15.45:0/312458679 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0c9920 con 0x7f244c15d3d0 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244bfff700 1 --2- 172.21.15.45:0/312458679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c15d3d0 0x7f244c067620 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f244c06eaa0 tx=0x7f2444009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa9fffb93230462 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.370+0000 7f244a7fc700 1 -- 172.21.15.45:0/312458679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244400e070 con 0x7f244c15d3d0 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f244a7fc700 1 -- 172.21.15.45:0/312458679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244400b970 con 0x7f244c15d3d0 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f244a7fc700 1 -- 172.21.15.45:0/312458679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440092e0 con 0x7f244c15d3d0 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 -- 172.21.15.45:0/312458679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c15d3d0 msgr2=0x7f244c067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 --2- 172.21.15.45:0/312458679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c15d3d0 0x7f244c067620 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f244c06eaa0 tx=0x7f2444009840 comp rx=0 tx=0).stop 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 -- 172.21.15.45:0/312458679 shutdown_connections 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 --2- 172.21.15.45:0/312458679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c15d3d0 0x7f244c067620 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 --2- 172.21.15.45:0/312458679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c159150 0x7f244c0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 --2- 172.21.15.45:0/312458679 >> v2:172.21.15.144:3300/0 conn(0x7f244c159520 0x7f244c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 -- 172.21.15.45:0/312458679 >> 172.21.15.45:0/312458679 conn(0x7f244c0691c0 msgr2=0x7f244c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 -- 172.21.15.45:0/312458679 shutdown_connections 2023-10-06T05:22:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.371+0000 7f2451f53700 1 -- 172.21.15.45:0/312458679 wait complete. 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.373+0000 7f2451f53700 1 Processor -- start 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.373+0000 7f2451f53700 1 -- start start 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f2451f53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c159150 0x7f244c154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f2451f53700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c15d3d0 0x7f244c1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f2451f53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f244c152d20 0x7f244c153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f2451f53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0afbe0 con 0x7f244c159150 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f2451f53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c154b80 con 0x7f244c152d20 2023-10-06T05:22:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f2451f53700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c154e60 con 0x7f244c15d3d0 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f244b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c159150 0x7f244c154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f244b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c159150 0x7f244c154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55890/0 (socket says 172.21.15.45:55890) 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f244b7fe700 1 -- 172.21.15.45:0/611598043 learned_addr learned my addr 172.21.15.45:0/611598043 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f244affd700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c15d3d0 0x7f244c1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.374+0000 7f244bfff700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f244c152d20 0x7f244c153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f244b7fe700 1 -- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c15d3d0 msgr2=0x7f244c1527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f244b7fe700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c15d3d0 0x7f244c1527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f244b7fe700 1 -- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f244c152d20 msgr2=0x7f244c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f244b7fe700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f244c152d20 0x7f244c153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f244b7fe700 1 -- 172.21.15.45:0/611598043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444009ae0 con 0x7f244c159150 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f244b7fe700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c159150 0x7f244c154210 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f243c00ec80 tx=0x7f243c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f24337fe700 1 -- 172.21.15.45:0/611598043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00c850 con 0x7f244c159150 2023-10-06T05:22:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f24337fe700 1 -- 172.21.15.45:0/611598043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c00c9b0 con 0x7f244c159150 2023-10-06T05:22:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c155140 con 0x7f244c159150 2023-10-06T05:22:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f24337fe700 1 -- 172.21.15.45:0/611598043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00cd50 con 0x7f244c159150 2023-10-06T05:22:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.375+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c154750 con 0x7f244c159150 2023-10-06T05:22:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.376+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c06e2d0 con 0x7f244c159150 2023-10-06T05:22:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.377+0000 7f24337fe700 1 -- 172.21.15.45:0/611598043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f243c013680 con 0x7f244c159150 2023-10-06T05:22:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.377+0000 7f24337fe700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2438066f30 0x7f24380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.377+0000 7f24337fe700 1 -- 172.21.15.45:0/611598043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f243c084a40 con 0x7f244c159150 2023-10-06T05:22:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.377+0000 7f244affd700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2438066f30 0x7f24380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.378+0000 7f244affd700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2438066f30 0x7f24380693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f244c0af000 tx=0x7f24400091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:13.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.382+0000 7f24337fe700 1 -- 172.21.15.45:0/611598043 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f243c086e60 con 0x7f244c159150 2023-10-06T05:22:13.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.635+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f244c0ca660 con 0x7f244c159150 2023-10-06T05:22:13.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.869+0000 7f24337fe700 1 -- 172.21.15.45:0/611598043 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v91) v1 ==== 162+0+0 (secure 0 0 0) 0x7f243c0519c0 con 0x7f244c159150 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2438066f30 msgr2=0x7f24380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2438066f30 0x7f24380693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f244c0af000 tx=0x7f24400091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c159150 msgr2=0x7f244c154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c159150 0x7f244c154210 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f243c00ec80 tx=0x7f243c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 shutdown_connections 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2438066f30 0x7f24380693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f244c152d20 0x7f244c153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f244c159150 0x7f244c154210 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 --2- 172.21.15.45:0/611598043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f244c15d3d0 0x7f244c1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 >> 172.21.15.45:0/611598043 conn(0x7f244c0691c0 msgr2=0x7f244c1562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:13.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 shutdown_connections 2023-10-06T05:22:13.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:13.872+0000 7f2451f53700 1 -- 172.21.15.45:0/611598043 wait complete. 2023-10-06T05:22:13.877 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:22:14.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:14.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.434+0000 7f06075d1700 1 Processor -- start 2023-10-06T05:22:14.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.435+0000 7f06075d1700 1 -- start start 2023-10-06T05:22:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.435+0000 7f06075d1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0600155cd0 0x7f060015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.435+0000 7f06075d1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06001560a0 0x7f06000adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.435+0000 7f06075d1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06000ae8d0 con 0x7f06000ccdc0 2023-10-06T05:22:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f06075d1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06000aebb0 con 0x7f06001560a0 2023-10-06T05:22:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f06075d1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000aee90 con 0x7f0600155cd0 2023-10-06T05:22:14.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06001560a0 0x7f06000adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06001560a0 0x7f06000adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55906/0 (socket says 172.21.15.45:55906) 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 -- 172.21.15.45:0/643807508 learned_addr learned my addr 172.21.15.45:0/643807508 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f060536d700 1 --2- 172.21.15.45:0/643807508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0600155cd0 0x7f060015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 -- 172.21.15.45:0/643807508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0600155cd0 msgr2=0x7f060015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 --2- 172.21.15.45:0/643807508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0600155cd0 0x7f060015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 -- 172.21.15.45:0/643807508 >> v1:172.21.15.144:6789/0 conn(0x7f06000ccdc0 legacy=0x7f06000c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 -- 172.21.15.45:0/643807508 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000af170 con 0x7f06001560a0 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f0604b6c700 1 --2- 172.21.15.45:0/643807508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06001560a0 0x7f06000adfa0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f05f00099b0 tx=0x7f05f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ab887a966fdea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f05f7fff700 1 -- 172.21.15.45:0/643807508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0009ea0 con 0x7f06001560a0 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f05f7fff700 1 -- 172.21.15.45:0/643807508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000b860 con 0x7f06001560a0 2023-10-06T05:22:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.436+0000 7f05f7fff700 1 -- 172.21.15.45:0/643807508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000bc00 con 0x7f06001560a0 2023-10-06T05:22:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 -- 172.21.15.45:0/643807508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06001560a0 msgr2=0x7f06000adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 --2- 172.21.15.45:0/643807508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06001560a0 0x7f06000adfa0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f05f00099b0 tx=0x7f05f0009250 comp rx=0 tx=0).stop 2023-10-06T05:22:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 -- 172.21.15.45:0/643807508 shutdown_connections 2023-10-06T05:22:14.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 --2- 172.21.15.45:0/643807508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0600155cd0 0x7f060015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 --2- 172.21.15.45:0/643807508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06001560a0 0x7f06000adfa0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 -- 172.21.15.45:0/643807508 >> 172.21.15.45:0/643807508 conn(0x7f060006f090 msgr2=0x7f060006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:14.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 -- 172.21.15.45:0/643807508 shutdown_connections 2023-10-06T05:22:14.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.437+0000 7f06075d1700 1 -- 172.21.15.45:0/643807508 wait complete. 2023-10-06T05:22:14.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.438+0000 7f06075d1700 1 Processor -- start 2023-10-06T05:22:14.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.438+0000 7f06075d1700 1 -- start start 2023-10-06T05:22:14.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.439+0000 7f06075d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06000ccdc0 0x7f0600158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.439+0000 7f06075d1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0600155cd0 0x7f0600156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.439+0000 7f06075d1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06001560a0 0x7f0600153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.439+0000 7f06075d1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000a4ba0 con 0x7f0600155cd0 2023-10-06T05:22:14.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.439+0000 7f06075d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0600157b00 con 0x7f06000ccdc0 2023-10-06T05:22:14.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.439+0000 7f06075d1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0600156ef0 con 0x7f06001560a0 2023-10-06T05:22:14.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0600155cd0 0x7f0600156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0600155cd0 0x7f0600156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55916/0 (socket says 172.21.15.45:55916) 2023-10-06T05:22:14.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 -- 172.21.15.45:0/4204489975 learned_addr learned my addr 172.21.15.45:0/4204489975 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:14.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0605b6e700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06001560a0 0x7f0600153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f060536d700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06000ccdc0 0x7f0600158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 -- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06001560a0 msgr2=0x7f0600153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06001560a0 0x7f0600153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 -- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06000ccdc0 msgr2=0x7f0600158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06000ccdc0 0x7f0600158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 -- 172.21.15.45:0/4204489975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f00094f0 con 0x7f0600155cd0 2023-10-06T05:22:14.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f0604b6c700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0600155cd0 0x7f0600156510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f060006aed0 tx=0x7f05f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:14.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f05f57fa700 1 -- 172.21.15.45:0/4204489975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0000d10 con 0x7f0600155cd0 2023-10-06T05:22:14.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f05f57fa700 1 -- 172.21.15.45:0/4204489975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000bbe0 con 0x7f0600155cd0 2023-10-06T05:22:14.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06001571d0 con 0x7f0600155cd0 2023-10-06T05:22:14.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f05f57fa700 1 -- 172.21.15.45:0/4204489975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0011670 con 0x7f0600155cd0 2023-10-06T05:22:14.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.440+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0600157790 con 0x7f0600155cd0 2023-10-06T05:22:14.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.441+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060006e2f0 con 0x7f0600155cd0 2023-10-06T05:22:14.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.442+0000 7f05f57fa700 1 -- 172.21.15.45:0/4204489975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05f0011aa0 con 0x7f0600155cd0 2023-10-06T05:22:14.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.442+0000 7f05f57fa700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f05e4066f30 0x7f05e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:14.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.442+0000 7f05f57fa700 1 -- 172.21.15.45:0/4204489975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f05f0085fa0 con 0x7f0600155cd0 2023-10-06T05:22:14.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.442+0000 7f060536d700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f05e4066f30 0x7f05e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:14.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.443+0000 7f060536d700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f05e4066f30 0x7f05e40693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f060015dd20 tx=0x7f05fc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:14.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.447+0000 7f05f57fa700 1 -- 172.21.15.45:0/4204489975 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05f00b75a0 con 0x7f0600155cd0 2023-10-06T05:22:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.694+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0600157c40 con 0x7f0600155cd0 2023-10-06T05:22:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.695+0000 7f05f57fa700 1 -- 172.21.15.45:0/4204489975 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f05f0052f20 con 0x7f0600155cd0 2023-10-06T05:22:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:14.697 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:13.871208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":92,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6833/2550360138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2550360138},{"type":"v1","addr":"172.21.15.45:6833","nonce":2550360138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f05e4066f30 msgr2=0x7f05e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f05e4066f30 0x7f05e40693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f060015dd20 tx=0x7f05fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0600155cd0 msgr2=0x7f0600156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0600155cd0 0x7f0600156510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f060006aed0 tx=0x7f05f0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 shutdown_connections 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f05e4066f30 0x7f05e40693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06001560a0 0x7f0600153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06000ccdc0 0x7f0600158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 --2- 172.21.15.45:0/4204489975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0600155cd0 0x7f0600156510 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 >> 172.21.15.45:0/4204489975 conn(0x7f060006f090 msgr2=0x7f0600159750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 shutdown_connections 2023-10-06T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:14.697+0000 7f06075d1700 1 -- 172.21.15.45:0/4204489975 wait complete. 2023-10-06T05:22:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 92 2023-10-06T05:22:14.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:22:13.871208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7365}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7365': {'gid': 7365, 'name': 'a', 'rank': 0, 'incarnation': 92, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6833/2550360138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 2550360138}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 2550360138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:14.713 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7365:up:replay 2023-10-06T05:22:14.713 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:22:18.124 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:22:18.122+0000 7f93404b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:20.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.256+0000 7f61a6dcb700 1 Processor -- start 2023-10-06T05:22:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.256+0000 7f61a6dcb700 1 -- start start 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a6dcb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153a20 0x7f61a006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a6dcb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f61a006add0 0x7f61a0068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a6dcb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a00676c0 con 0x7f61a0153650 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a6dcb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a006b250 con 0x7f61a006add0 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a6dcb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00679a0 con 0x7f61a0153a20 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a4b67700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f61a0153650 0x7f61a015a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44186/0 (socket says 172.21.15.45:44186) 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a4b67700 1 -- 172.21.15.45:0/3544238036 learned_addr learned my addr 172.21.15.45:0/3544238036 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f619ffff700 1 --2- 172.21.15.45:0/3544238036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153a20 0x7f61a006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f61a5368700 1 --2- 172.21.15.45:0/3544238036 >> v2:172.21.15.144:3300/0 conn(0x7f61a006add0 0x7f61a0068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f619f7fe700 1 -- 172.21.15.45:0/3544238036 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608336252 0 0) 0x7f61a00676c0 con 0x7f61a0153650 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.257+0000 7f619f7fe700 1 -- 172.21.15.45:0/3544238036 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6190003570 con 0x7f61a0153650 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619ffff700 1 -- 172.21.15.45:0/3544238036 >> v2:172.21.15.144:3300/0 conn(0x7f61a006add0 msgr2=0x7f61a0068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619ffff700 1 --2- 172.21.15.45:0/3544238036 >> v2:172.21.15.144:3300/0 conn(0x7f61a006add0 0x7f61a0068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619ffff700 1 -- 172.21.15.45:0/3544238036 >> v1:172.21.15.45:6789/0 conn(0x7f61a0153650 legacy=0x7f61a015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619f7fe700 1 -- 172.21.15.45:0/3544238036 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283471966 0 0) 0x7f6190003570 con 0x7f61a0153650 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619ffff700 1 -- 172.21.15.45:0/3544238036 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0067c80 con 0x7f61a0153a20 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f61a5368700 1 --2- 172.21.15.45:0/3544238036 >> v2:172.21.15.144:3300/0 conn(0x7f61a006add0 0x7f61a0068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619ffff700 1 --2- 172.21.15.45:0/3544238036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153a20 0x7f61a006a890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f618c0098e0 tx=0x7f618c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1986e64868d6064b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619f7fe700 1 -- 172.21.15.45:0/3544238036 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194001470 con 0x7f61a0153a20 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619f7fe700 1 -- 172.21.15.45:0/3544238036 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c011420 con 0x7f61a0153a20 2023-10-06T05:22:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.258+0000 7f619f7fe700 1 -- 172.21.15.45:0/3544238036 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c0117c0 con 0x7f61a0153a20 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 -- 172.21.15.45:0/3544238036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153a20 msgr2=0x7f61a006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/3544238036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153a20 0x7f61a006a890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f618c0098e0 tx=0x7f618c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 -- 172.21.15.45:0/3544238036 shutdown_connections 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/3544238036 >> v2:172.21.15.144:3300/0 conn(0x7f61a006add0 0x7f61a0068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/3544238036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153a20 0x7f61a006a890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 -- 172.21.15.45:0/3544238036 >> 172.21.15.45:0/3544238036 conn(0x7f61a00cc5a0 msgr2=0x7f61a00cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 -- 172.21.15.45:0/3544238036 shutdown_connections 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.259+0000 7f61a6dcb700 1 -- 172.21.15.45:0/3544238036 wait complete. 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.260+0000 7f61a6dcb700 1 Processor -- start 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.260+0000 7f61a6dcb700 1 -- start start 2023-10-06T05:22:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a6dcb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a006add0 0x7f61a00ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a6dcb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153650 0x7f61a0169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a6dcb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61a0153a20 0x7f61a0151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a6dcb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0152b20 con 0x7f61a0153a20 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a6dcb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00aa4a0 con 0x7f61a006add0 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a6dcb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0152510 con 0x7f61a0153650 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61a0153a20 0x7f61a0151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61a0153a20 0x7f61a0151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56526/0 (socket says 172.21.15.45:56526) 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 -- 172.21.15.45:0/2325358003 learned_addr learned my addr 172.21.15.45:0/2325358003 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f619ffff700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153650 0x7f61a0169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a4b67700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a006add0 0x7f61a00ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 -- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153650 msgr2=0x7f61a0169c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153650 0x7f61a0169c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 -- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a006add0 msgr2=0x7f61a00ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a006add0 0x7f61a00ab3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.261+0000 7f61a5368700 1 -- 172.21.15.45:0/2325358003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198009a20 con 0x7f61a0153a20 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.262+0000 7f61a5368700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61a0153a20 0x7f61a0151fd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6198009fc0 tx=0x7f619800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.262+0000 7f619cff9700 1 -- 172.21.15.45:0/2325358003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800cbd0 con 0x7f61a0153a20 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.262+0000 7f619cff9700 1 -- 172.21.15.45:0/2325358003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619800f040 con 0x7f61a0153a20 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.262+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c009420 con 0x7f61a0153a20 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.262+0000 7f619cff9700 1 -- 172.21.15.45:0/2325358003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198014480 con 0x7f61a0153a20 2023-10-06T05:22:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.262+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a00a9b70 con 0x7f61a0153a20 2023-10-06T05:22:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.263+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a00cc0e0 con 0x7f61a0153a20 2023-10-06T05:22:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.263+0000 7f619cff9700 1 -- 172.21.15.45:0/2325358003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6198003020 con 0x7f61a0153a20 2023-10-06T05:22:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.264+0000 7f619cff9700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6184066f30 0x7f61840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.264+0000 7f619cff9700 1 -- 172.21.15.45:0/2325358003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f61980851b0 con 0x7f61a0153a20 2023-10-06T05:22:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.264+0000 7f61a4b67700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6184066f30 0x7f61840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.265+0000 7f61a4b67700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6184066f30 0x7f61840693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f61940011e0 tx=0x7f6194001aa0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.269+0000 7f619cff9700 1 -- 172.21.15.45:0/2325358003 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6198052130 con 0x7f61a0153a20 2023-10-06T05:22:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.519+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61a0067f60 con 0x7f61a0153a20 2023-10-06T05:22:20.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.520+0000 7f619cff9700 1 -- 172.21.15.45:0/2325358003 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f6198055750 con 0x7f61a0153a20 2023-10-06T05:22:20.522 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:20.522 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:16.892286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6833/2550360138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2550360138},{"type":"v1","addr":"172.21.15.45:6833","nonce":2550360138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6184066f30 msgr2=0x7f61840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6184066f30 0x7f61840693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f61940011e0 tx=0x7f6194001aa0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61a0153a20 msgr2=0x7f61a0151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61a0153a20 0x7f61a0151fd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6198009fc0 tx=0x7f619800b040 comp rx=0 tx=0).stop 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 shutdown_connections 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6184066f30 0x7f61840693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61a0153a20 0x7f61a0151fd0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a006add0 0x7f61a00ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 --2- 172.21.15.45:0/2325358003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a0153650 0x7f61a0169c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 >> 172.21.15.45:0/2325358003 conn(0x7f61a00cc5a0 msgr2=0x7f61a0159010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 shutdown_connections 2023-10-06T05:22:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.522+0000 7f61a6dcb700 1 -- 172.21.15.45:0/2325358003 wait complete. 2023-10-06T05:22:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 95 2023-10-06T05:22:20.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:22:16.892286+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7365}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7365': {'gid': 7365, 'name': 'a', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.45:6833/2550360138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 2550360138}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 2550360138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:20.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:22:20.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:22:20.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.668+0000 7f5ef39fb700 1 Processor -- start 2023-10-06T05:22:20.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.668+0000 7f5ef39fb700 1 -- start start 2023-10-06T05:22:20.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ef39fb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5eec0cd340 0x7f5eec157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ef39fb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec15b000 con 0x7f5eec0c93c0 2023-10-06T05:22:20.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ef39fb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec15b2e0 con 0x7f5eec0c8ff0 2023-10-06T05:22:20.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ef39fb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0c9830 con 0x7f5eec0cd340 2023-10-06T05:22:20.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ef0f96700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5eec0c93c0 0x7f5eec156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44202/0 (socket says 172.21.15.45:44202) 2023-10-06T05:22:20.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1858576629 learned_addr learned my addr 172.21.15.45:0/1858576629 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1858576629 >> v2:172.21.15.144:3300/0 conn(0x7f5eec0cd340 0x7f5eec157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037718126 0 0) 0x7f5eec15b000 con 0x7f5eec0c93c0 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.669+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed8003570 con 0x7f5eec0c93c0 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3852921124 0 0) 0x7f5ed8003570 con 0x7f5eec0c93c0 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 >> v2:172.21.15.144:3300/0 conn(0x7f5eec0cd340 msgr2=0x7f5eec157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 --2- 172.21.15.45:0/1858576629 >> v2:172.21.15.144:3300/0 conn(0x7f5eec0cd340 0x7f5eec157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 >> v1:172.21.15.144:6789/0 conn(0x7f5eec0c8ff0 legacy=0x7f5eec0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec15b5c0 con 0x7f5eec0c93c0 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f5ee8001470 con 0x7f5eec0c93c0 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 >> v1:172.21.15.45:6789/0 conn(0x7f5eec0c93c0 legacy=0x7f5eec156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ed8003a60 0x7f5ed8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed8004810 0x7f5ed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed80051c0 0x7f5ed800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0c9830 con 0x7f5ed8004810 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed800c0d0 con 0x7f5ed8003a60 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed800c290 con 0x7f5ed80051c0 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed8004810 0x7f5ed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef1797700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ed8003a60 0x7f5ed8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed80051c0 0x7f5ed800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed80051c0 msgr2=0x7f5ed800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed80051c0 0x7f5ed800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ed8003a60 msgr2=0x7f5ed8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ed8003a60 0x7f5ed8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1858576629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8003570 con 0x7f5ed8004810 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed80051c0 0x7f5ed800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef1f98700 1 -- 172.21.15.45:0/1858576629 reap_dead start 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.670+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed8004810 0x7f5ed8004c80 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c3c0 tx=0x7f5ee8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30ad11e4a26d9c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800cc90 con 0x7f5ed8004810 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee800cdf0 con 0x7f5ed8004810 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ee3fff700 1 -- 172.21.15.45:0/1858576629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8005b50 con 0x7f5ed8004810 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ef39fb700 1 -- 172.21.15.45:0/1858576629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed8004810 msgr2=0x7f5ed8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed8004810 0x7f5ed8004c80 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c3c0 tx=0x7f5ee8001e50 comp rx=0 tx=0).stop 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ef39fb700 1 -- 172.21.15.45:0/1858576629 shutdown_connections 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/1858576629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed8004810 0x7f5ed8004c80 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ef39fb700 1 -- 172.21.15.45:0/1858576629 >> 172.21.15.45:0/1858576629 conn(0x7f5eec0cdf50 msgr2=0x7f5eec0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ef39fb700 1 -- 172.21.15.45:0/1858576629 shutdown_connections 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.671+0000 7f5ef39fb700 1 -- 172.21.15.45:0/1858576629 wait complete. 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.672+0000 7f5ef39fb700 1 Processor -- start 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.673+0000 7f5ef39fb700 1 -- start start 2023-10-06T05:22:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.673+0000 7f5ef39fb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed8004810 0x7f5eec14acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.673+0000 7f5ef39fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec146720 0x7f5eec146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.673+0000 7f5ef39fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec149190 0x7f5eec149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef39fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0c5a10 con 0x7f5eec149190 2023-10-06T05:22:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef39fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0c7e30 con 0x7f5eec146720 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef39fb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0c5c60 con 0x7f5ed8004810 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec149190 0x7f5eec149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec149190 0x7f5eec149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56536/0 (socket says 172.21.15.45:56536) 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1797700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed8004810 0x7f5eec14acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 -- 172.21.15.45:0/623772026 learned_addr learned my addr 172.21.15.45:0/623772026 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec146720 0x7f5eec146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 -- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed8004810 msgr2=0x7f5eec14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed8004810 0x7f5eec14acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 -- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec146720 msgr2=0x7f5eec146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec146720 0x7f5eec146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 -- 172.21.15.45:0/623772026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0c5f40 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec149190 0x7f5eec149630 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0b0a90 tx=0x7f5ee400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ee17fa700 1 -- 172.21.15.45:0/623772026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee401b070 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ee17fa700 1 -- 172.21.15.45:0/623772026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400f040 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec0c6220 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.674+0000 7f5ee17fa700 1 -- 172.21.15.45:0/623772026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400e050 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.675+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec1470d0 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.676+0000 7f5ee17fa700 1 -- 172.21.15.45:0/623772026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ee4003020 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.676+0000 7f5ee17fa700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ed0066f30 0x7f5ed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.677+0000 7f5ee17fa700 1 -- 172.21.15.45:0/623772026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5ee4084c50 con 0x7f5eec149190 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.677+0000 7f5ef1797700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ed0066f30 0x7f5ed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.677+0000 7f5ef1797700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ed0066f30 0x7f5ed00693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5edc003c30 tx=0x7f5edc007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.678+0000 7f5efcd0fb80 1 Processor -- start 2023-10-06T05:22:20.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.679+0000 7f5efcd0fb80 1 -- start start 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.679+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab9e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.679+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856aba7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856aa71310 con 0x55856ab83f40 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856aaae130 con 0x55856aba74d0 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856aaabaf0 con 0x55856ab9e450 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56540/0 (socket says 172.21.15.45:56540) 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1620025089 learned_addr learned my addr 172.21.15.45:0/1620025089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef1797700 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab9e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856aba7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 msgr2=0x55856ab9e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab9e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 msgr2=0x55856aba7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856aba7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 -- 172.21.15.45:0/1620025089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55856a8ad790 con 0x55856ab83f40 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba6f90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c0a0 tx=0x7f5ee8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e699044af84f8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ed67fc700 1 -- 172.21.15.45:0/1620025089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8007ac0 con 0x55856ab83f40 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ed67fc700 1 -- 172.21.15.45:0/1620025089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8000ed0 con 0x55856ab83f40 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.680+0000 7f5ed67fc700 1 -- 172.21.15.45:0/1620025089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8000c30 con 0x55856ab83f40 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1620025089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 msgr2=0x55856aba6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba6f90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c0a0 tx=0x7f5ee8015000 comp rx=0 tx=0).stop 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1620025089 shutdown_connections 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856aba7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab9e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1620025089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba6f90 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1620025089 >> 172.21.15.45:0/1620025089 conn(0x55856aba1880 msgr2=0x55856aba1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1620025089 shutdown_connections 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.681+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1620025089 wait complete. 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.682+0000 7f5efcd0fb80 1 -- start start 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.682+0000 7f5efcd0fb80 1 Processor -- start 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856ab58d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856aaa78e0 con 0x55856ab83f40 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856aa53980 con 0x55856aba74d0 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856aa53620 con 0x55856ab9e450 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5efcd0fb80 10 client.0 _ll_register_callbacks cb 0x55856a8b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5ef1797700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5ef1797700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56544/0 (socket says 172.21.15.45:56544) 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5ef1797700 1 -- 172.21.15.45:0/2203798907 learned_addr learned my addr 172.21.15.45:0/2203798907 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856ab58d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.683+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ef1797700 1 -- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 msgr2=0x55856ab587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ef1797700 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ef1797700 1 -- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 msgr2=0x55856ab58d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ef1797700 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856ab58d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ef1797700 1 -- 172.21.15.45:0/2203798907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed8003570 con 0x55856ab83f40 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ef1797700 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba11e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5edc015b70 tx=0x7f5edc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f542ff57ae0ba527 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ecb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/2203798907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc01a040 con 0x55856ab83f40 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/2203798907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5edc01ebe0 con 0x55856ab83f40 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55856ab4bd60 con 0x55856ab83f40 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/2203798907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5edc006a30 con 0x55856ab83f40 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5efcd0fb80 10 client.7833 fetch_fsmap learned FSMap version 95 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55856a99a0a0 con 0x55856ab83f40 2023-10-06T05:22:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/2203798907 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55856ab4bd60 con 0x55856ab83f40 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/2203798907 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 95) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5edc006b90 con 0x55856ab83f40 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5efcd0fb80 10 client.7833 fetch_fsmap finished waiting for FSMap version 95 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5efcd0fb80 10 client.7833 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.684+0000 7f5efcd0fb80 20 client.7833 populate_metadata read hostname 'smithi045' 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.685+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856ab7b560 0x55856ab7b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.685+0000 7f5efcd0fb80 4 client.7833 mds_command: new command op to 7365 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.685+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 --> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55856ab28b30 con 0x55856ab7b560 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.685+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856ab7b560 0x55856ab7b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.685+0000 7f5ecb7fe700 10 client.7833 ms_handle_connect on v2:172.21.15.45:6832/2550360138 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.685+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856ab7b560 0x55856ab7b930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e47ed1b8f224bff server_cookie=e5cd5cee83eea64c in_seq=0 out_seq=0 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/2203798907 <== mds.0 v2:172.21.15.45:6832/2550360138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55856ab28b30 con 0x55856ab7b560 2023-10-06T05:22:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5ecb7fe700 10 client.7833 handle_command_reply: tid=0 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 client.7833 shutdown 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 msgr2=0x55856aba11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba11e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5edc015b70 tx=0x7f5edc017000 comp rx=0 tx=0).stop 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 shutdown_connections 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856ab7b560 0x55856ab7b930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5ecb7fe700 0 client.7833 ms_handle_reset on v2:172.21.15.45:6832/2550360138 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856aba74d0 0x55856ab58d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab83f40 0x55856aba11e0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/2203798907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 >> 172.21.15.45:0/2203798907 conn(0x55856aba1880 msgr2=0x55856ab57130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 shutdown_connections 2023-10-06T05:22:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/2203798907 wait complete. 2023-10-06T05:22:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.686+0000 7f5efcd0fb80 20 client.7833 trim_cache size 0 max 16384 2023-10-06T05:22:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.704+0000 7f5efcd0fb80 1 Processor -- start 2023-10-06T05:22:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.704+0000 7f5efcd0fb80 1 -- start start 2023-10-06T05:22:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab7b560 0x55856ab72170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab83f40 0x55856ab81430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab9e450 0x55856aba0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856a93f1e0 con 0x55856ab9e450 2023-10-06T05:22:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856ab580b0 con 0x55856ab7b560 2023-10-06T05:22:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856ab79860 con 0x55856ab83f40 2023-10-06T05:22:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab9e450 0x55856aba0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab9e450 0x55856aba0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56552/0 (socket says 172.21.15.45:56552) 2023-10-06T05:22:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 -- 172.21.15.45:0/1447114328 learned_addr learned my addr 172.21.15.45:0/1447114328 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab83f40 0x55856ab81430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1797700 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab7b560 0x55856ab72170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 -- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab83f40 msgr2=0x55856ab81430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab83f40 0x55856ab81430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 -- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab7b560 msgr2=0x55856ab72170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab7b560 0x55856ab72170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 -- 172.21.15.45:0/1447114328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55856ab57e40 con 0x55856ab9e450 2023-10-06T05:22:20.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1797700 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab7b560 0x55856ab72170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:20.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab9e450 0x55856aba0e40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4055820 tx=0x7f5ee4082de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a0f5fb9c677a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ed6ffd700 1 -- 172.21.15.45:0/1447114328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4055890 con 0x55856ab9e450 2023-10-06T05:22:20.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ed6ffd700 1 -- 172.21.15.45:0/1447114328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee4058ae0 con 0x55856ab9e450 2023-10-06T05:22:20.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.705+0000 7f5ed6ffd700 1 -- 172.21.15.45:0/1447114328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee405c830 con 0x55856ab9e450 2023-10-06T05:22:20.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1447114328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab9e450 msgr2=0x55856aba0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab9e450 0x55856aba0e40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4055820 tx=0x7f5ee4082de0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1447114328 shutdown_connections 2023-10-06T05:22:20.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab9e450 0x55856aba0e40 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab7b560 0x55856ab72170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1447114328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab83f40 0x55856ab81430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1447114328 >> 172.21.15.45:0/1447114328 conn(0x55856aba1880 msgr2=0x55856ab75d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1447114328 shutdown_connections 2023-10-06T05:22:20.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1447114328 wait complete. 2023-10-06T05:22:20.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.706+0000 7f5efcd0fb80 1 -- start start 2023-10-06T05:22:20.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.707+0000 7f5efcd0fb80 1 Processor -- start 2023-10-06T05:22:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab7b560 0x55856ac007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab83f40 0x55856abea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab71e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856ab818b0 con 0x55856ab7b560 2023-10-06T05:22:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856abea580 con 0x55856ab83f40 2023-10-06T05:22:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856ac01060 con 0x55856ab9e450 2023-10-06T05:22:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 10 client.0 _ll_register_callbacks cb 0x7f5edc012f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab7b560 0x55856ac007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab7b560 0x55856ac007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56562/0 (socket says 172.21.15.45:56562) 2023-10-06T05:22:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 -- 172.21.15.45:0/1037265709 learned_addr learned my addr 172.21.15.45:0/1037265709 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1f98700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab71e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab83f40 0x55856abea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 -- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 msgr2=0x55856ab71e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab71e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 -- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab83f40 msgr2=0x55856abea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab83f40 0x55856abea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 -- 172.21.15.45:0/1037265709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee406b040 con 0x55856ab7b560 2023-10-06T05:22:20.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab83f40 0x55856abea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:20.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ef1797700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab7b560 0x55856ac007d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5edc007020 tx=0x7f5edc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b1ea3b504d9363 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ecb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:22:20.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/1037265709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0072b0 con 0x55856ab7b560 2023-10-06T05:22:20.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/1037265709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5edc007410 con 0x55856ab7b560 2023-10-06T05:22:20.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.708+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee8000ed0 con 0x55856ab7b560 2023-10-06T05:22:20.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/1037265709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5edc0075d0 con 0x55856ab7b560 2023-10-06T05:22:20.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 10 client.7839 fetch_fsmap learned FSMap version 95 2023-10-06T05:22:20.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55856aa763c0 con 0x55856ab7b560 2023-10-06T05:22:20.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/1037265709 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8000ed0 con 0x55856ab7b560 2023-10-06T05:22:20.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/1037265709 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 95) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5edc040310 con 0x55856ab7b560 2023-10-06T05:22:20.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 10 client.7839 fetch_fsmap finished waiting for FSMap version 95 2023-10-06T05:22:20.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 10 client.7839 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:22:20.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 20 client.7839 populate_metadata read hostname 'smithi045' 2023-10-06T05:22:20.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856aba74d0 0x55856ab6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 4 client.7839 mds_command: new command op to 7365 tid=0[{"prefix": "status"}] 2023-10-06T05:22:20.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 --> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55856aa4a8d0 con 0x55856aba74d0 2023-10-06T05:22:20.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856aba74d0 0x55856ab6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5ecb7fe700 10 client.7839 ms_handle_connect on v2:172.21.15.45:6832/2550360138 2023-10-06T05:22:20.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.709+0000 7f5ef0f96700 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856aba74d0 0x55856ab6f870 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4766bc1bb5f5c249 server_cookie=e5978271e58f86f2 in_seq=0 out_seq=0 2023-10-06T05:22:20.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5ecb7fe700 1 -- 172.21.15.45:0/1037265709 <== mds.0 v2:172.21.15.45:6832/2550360138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55856aa4a8d0 con 0x55856aba74d0 2023-10-06T05:22:20.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5ecb7fe700 10 client.7839 handle_command_reply: tid=0 2023-10-06T05:22:20.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 client.7839 shutdown 2023-10-06T05:22:20.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab7b560 msgr2=0x55856ac007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab7b560 0x55856ac007d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5edc007020 tx=0x7f5edc019000 comp rx=0 tx=0).stop 2023-10-06T05:22:20.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 shutdown_connections 2023-10-06T05:22:20.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:6832/2550360138,v1:172.21.15.45:6833/2550360138] conn(0x55856aba74d0 0x55856ab6f870 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856ab9e450 0x55856ab71e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55856ab7b560 0x55856ac007d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5ecb7fe700 0 client.7839 ms_handle_reset on v2:172.21.15.45:6832/2550360138 2023-10-06T05:22:20.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 --2- 172.21.15.45:0/1037265709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856ab83f40 0x55856abea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 >> 172.21.15.45:0/1037265709 conn(0x55856aba1880 msgr2=0x55856ab9f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 shutdown_connections 2023-10-06T05:22:20.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 1 -- 172.21.15.45:0/1037265709 wait complete. 2023-10-06T05:22:20.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.710+0000 7f5efcd0fb80 20 client.7839 trim_cache size 0 max 16384 2023-10-06T05:22:20.731 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:22:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:22:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:22:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7365, 2023-10-06T05:22:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:22:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:22:20.732 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:22:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8377147110000003, 2023-10-06T05:22:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 95, 2023-10-06T05:22:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 93, 2023-10-06T05:22:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 93, 2023-10-06T05:22:20.733 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 13.561103231000001 2023-10-06T05:22:20.733 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:22:20.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ed0066f30 msgr2=0x7f5ed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ed0066f30 0x7f5ed00693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5edc003c30 tx=0x7f5edc007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:20.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec149190 msgr2=0x7f5eec149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec149190 0x7f5eec149630 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0b0a90 tx=0x7f5ee400b040 comp rx=0 tx=0).stop 2023-10-06T05:22:20.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 shutdown_connections 2023-10-06T05:22:20.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5ed0066f30 0x7f5ed00693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec149190 0x7f5eec149630 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ed8004810 0x7f5eec14acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 --2- 172.21.15.45:0/623772026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec146720 0x7f5eec146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 >> 172.21.15.45:0/623772026 conn(0x7f5eec0cdf50 msgr2=0x7f5eec147690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 shutdown_connections 2023-10-06T05:22:20.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.711+0000 7f5ef39fb700 1 -- 172.21.15.45:0/623772026 wait complete. 2023-10-06T05:22:20.737 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:22:20.737 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2023-10-06T05:22:20.737 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:20.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.868+0000 7f9cd37f7700 1 Processor -- start 2023-10-06T05:22:20.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.868+0000 7f9cd37f7700 1 -- start start 2023-10-06T05:22:20.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd37f7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ccc0c8ff0 0x7f9ccc0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd37f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0caad0 0x7f9ccc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd37f7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ccc156af0 0x7f9ccc156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd37f7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc15b000 con 0x7f9ccc156af0 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd37f7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc15b2e0 con 0x7f9ccc0c8ff0 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd37f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc15b5c0 con 0x7f9ccc0caad0 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ccc156af0 0x7f9ccc156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ccc156af0 0x7f9ccc156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56578/0 (socket says 172.21.15.45:56578) 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 -- 172.21.15.45:0/1769084048 learned_addr learned my addr 172.21.15.45:0/1769084048 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd0d92700 1 --2- 172.21.15.45:0/1769084048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0caad0 0x7f9ccc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1593700 1 --2- 172.21.15.45:0/1769084048 >> v2:172.21.15.144:3300/0 conn(0x7f9ccc0c8ff0 0x7f9ccc0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 -- 172.21.15.45:0/1769084048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0caad0 msgr2=0x7f9ccc0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 --2- 172.21.15.45:0/1769084048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0caad0 0x7f9ccc0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 -- 172.21.15.45:0/1769084048 >> v2:172.21.15.144:3300/0 conn(0x7f9ccc0c8ff0 msgr2=0x7f9ccc0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 --2- 172.21.15.45:0/1769084048 >> v2:172.21.15.144:3300/0 conn(0x7f9ccc0c8ff0 0x7f9ccc0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.869+0000 7f9cd1d94700 1 -- 172.21.15.45:0/1769084048 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc15b8a0 con 0x7f9ccc156af0 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd1d94700 1 --2- 172.21.15.45:0/1769084048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc156f70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0cefc0 tx=0x7f9cc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7dec6eea668a69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cc3fff700 1 -- 172.21.15.45:0/1769084048 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400e070 con 0x7f9ccc156af0 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cc3fff700 1 -- 172.21.15.45:0/1769084048 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400ba50 con 0x7f9ccc156af0 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cc3fff700 1 -- 172.21.15.45:0/1769084048 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40092e0 con 0x7f9ccc156af0 2023-10-06T05:22:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 -- 172.21.15.45:0/1769084048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 msgr2=0x7f9ccc156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/1769084048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc156f70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0cefc0 tx=0x7f9cc4009840 comp rx=0 tx=0).stop 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 -- 172.21.15.45:0/1769084048 shutdown_connections 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/1769084048 >> v2:172.21.15.144:3300/0 conn(0x7f9ccc0c8ff0 0x7f9ccc0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/1769084048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc156f70 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/1769084048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0caad0 0x7f9ccc0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 -- 172.21.15.45:0/1769084048 >> 172.21.15.45:0/1769084048 conn(0x7f9ccc0cdf50 msgr2=0x7f9ccc0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 -- 172.21.15.45:0/1769084048 shutdown_connections 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.870+0000 7f9cd37f7700 1 -- 172.21.15.45:0/1769084048 wait complete. 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.872+0000 7f9cd37f7700 1 Processor -- start 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.872+0000 7f9cd37f7700 1 -- start start 2023-10-06T05:22:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd37f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0c8ff0 0x7f9ccc14ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd37f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0caad0 0x7f9ccc14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd37f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd37f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0b0e70 con 0x7f9ccc156af0 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd37f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0b1150 con 0x7f9ccc0caad0 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd37f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0b1430 con 0x7f9ccc0c8ff0 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc14def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56590/0 (socket says 172.21.15.45:56590) 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 -- 172.21.15.45:0/2906632559 learned_addr learned my addr 172.21.15.45:0/2906632559 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd0d92700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0caad0 0x7f9ccc14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1593700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0c8ff0 0x7f9ccc14ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 -- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0c8ff0 msgr2=0x7f9ccc14ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0c8ff0 0x7f9ccc14ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 -- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0caad0 msgr2=0x7f9ccc14d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0caad0 0x7f9ccc14d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 -- 172.21.15.45:0/2906632559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4009ae0 con 0x7f9ccc156af0 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cd1d94700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc14def0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0c4f80 tx=0x7f9cc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.873+0000 7f9cc17fa700 1 -- 172.21.15.45:0/2906632559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40032f0 con 0x7f9ccc156af0 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.874+0000 7f9cc17fa700 1 -- 172.21.15.45:0/2906632559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc4008e80 con 0x7f9ccc156af0 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.874+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc0b1710 con 0x7f9ccc156af0 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.874+0000 7f9cc17fa700 1 -- 172.21.15.45:0/2906632559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4026690 con 0x7f9ccc156af0 2023-10-06T05:22:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.874+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc14d3d0 con 0x7f9ccc156af0 2023-10-06T05:22:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.874+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc0b1c30 con 0x7f9ccc156af0 2023-10-06T05:22:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.875+0000 7f9cc17fa700 1 -- 172.21.15.45:0/2906632559 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9cc4003040 con 0x7f9ccc156af0 2023-10-06T05:22:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.876+0000 7f9cc17fa700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.876+0000 7f9cc17fa700 1 -- 172.21.15.45:0/2906632559 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f9cc4086b80 con 0x7f9ccc156af0 2023-10-06T05:22:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.876+0000 7f9cd1593700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.877+0000 7f9cd1593700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9cb0066f30 0x7f9cb00693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8006fd0 tx=0x7f9cc8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:20.880+0000 7f9cc17fa700 1 -- 172.21.15.45:0/2906632559 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cc4053b80 con 0x7f9ccc156af0 2023-10-06T05:22:21.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.135+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ccc0b1c30 con 0x7f9ccc156af0 2023-10-06T05:22:21.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.136+0000 7f9cc17fa700 1 -- 172.21.15.45:0/2906632559 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ccc0b1c30 con 0x7f9ccc156af0 2023-10-06T05:22:21.138 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9cb0066f30 msgr2=0x7f9cb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9cb0066f30 0x7f9cb00693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8006fd0 tx=0x7f9cc8008040 comp rx=0 tx=0).stop 2023-10-06T05:22:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 msgr2=0x7f9ccc14def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc14def0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0c4f80 tx=0x7f9cc4018000 comp rx=0 tx=0).stop 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 shutdown_connections 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc156af0 0x7f9ccc14def0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ccc0c8ff0 0x7f9ccc14ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 --2- 172.21.15.45:0/2906632559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0caad0 0x7f9ccc14d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 >> 172.21.15.45:0/2906632559 conn(0x7f9ccc0cdf50 msgr2=0x7f9ccc0aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 shutdown_connections 2023-10-06T05:22:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:21.138+0000 7f9cd37f7700 1 -- 172.21.15.45:0/2906632559 wait complete. 2023-10-06T05:22:21.153 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:22:21.153 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:21.153 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:22:21.154 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:22:21.154 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:21.154 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:22:21.154 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:22:21.154 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:21.154 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:21.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:21.154 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:22:21.175 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42000sec preferred_lft 42000sec 2023-10-06T05:22:21.176 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:21.177 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:22:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:22:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:21.178 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:22:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:22:21.178 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:21.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:21.178 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:21.247 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:22:21.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:21.248 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:22:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:22:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:22:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:22:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:21.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:21.315 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:22:21.315 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:22:21.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:22:21.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:22:21.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:22:21.315 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:22:21.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:21.439 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:22:21.460 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:22:21.461 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:22:21.461 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:21.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:21.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:22:21.537 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:22:21.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:21.537 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:21.601 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:21.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:21.601 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:21.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:22:21.740 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:22:21.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:22:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:22:21.816 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:22:21.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:21.817 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:22:21.881 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:22:21.881 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:22:21.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:22:21.959 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:22:21.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:22:22.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:22:22.024+0000 7f2ed96f0540 -1 init, newargv = 0x5587dfcf0b00 newargc=15 2023-10-06T05:22:22.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[158121]: starting ceph client 2023-10-06T05:22:22.031 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:22:22.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[158121]: starting fuse 2023-10-06T05:22:22.032 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:22:22.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:22.033 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:22:22.095 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:22:22.095 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:22:22.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:22:22.095 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:22:22.095 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:22:22.095 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:22:22.096 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:22:22.097 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:22:22.235 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.158121.asok 2023-10-06T05:22:22.417 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.158121.asok 2023-10-06T05:22:22.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:22.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.158121.asok status 2023-10-06T05:22:22.591 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "158121", 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:22:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7851, 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 7851 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 112060621 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:22:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 112060621 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.7851 192.168.0.1:0/112060621", 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/112060621", 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 95, 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 93, 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:22:22.594 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:22:22.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:22.602 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:22:22.602 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:22.624 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:22:22.624 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:22.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:22.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:22.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:22.698 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:22:22.740 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:22:22.765 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:22:22.764+0000 7f933ccac700 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2023-10-06T05:22:22.766 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:22:22.766 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:22:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:22.767 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:22:22.767 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2023-10-06T05:22:22.767 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2023-10-06T05:22:22.767 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:22:22.767 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:22.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:22.853 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:22:22.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:22.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:22.854 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:22.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:23.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.098+0000 7f2303f88700 1 Processor -- start 2023-10-06T05:22:23.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.098+0000 7f2303f88700 1 -- start start 2023-10-06T05:22:23.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2303f88700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22fc0cc550 0x7f22fc0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2303f88700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22fc161f40 con 0x7f22fc0caa70 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2303f88700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22fc161330 con 0x7f22fc15d0b0 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2303f88700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22fc0cb5d0 con 0x7f22fc0cc550 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2302525700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f22fc0caa70 0x7f22fc0caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44208/0 (socket says 172.21.15.45:44208) 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2302525700 1 -- 172.21.15.45:0/393307379 learned_addr learned my addr 172.21.15.45:0/393307379 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2301d24700 1 --2- 172.21.15.45:0/393307379 >> v2:172.21.15.144:3300/0 conn(0x7f22fc0cc550 0x7f22fc0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348845471 0 0) 0x7f22fc161f40 con 0x7f22fc0caa70 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.099+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e8003570 con 0x7f22fc0caa70 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246885720 0 0) 0x7f22e8003570 con 0x7f22fc0caa70 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 >> v2:172.21.15.144:3300/0 conn(0x7f22fc0cc550 msgr2=0x7f22fc0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 --2- 172.21.15.45:0/393307379 >> v2:172.21.15.144:3300/0 conn(0x7f22fc0cc550 0x7f22fc0cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 >> v1:172.21.15.144:6789/0 conn(0x7f22fc15d0b0 legacy=0x7f22fc15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc161610 con 0x7f22fc0caa70 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f22f40017f0 con 0x7f22fc0caa70 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 >> v1:172.21.15.45:6789/0 conn(0x7f22fc0caa70 legacy=0x7f22fc0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e8003a60 0x7f22e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22e8004810 0x7f22e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e80051c0 0x7f22e800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0cb5d0 con 0x7f22e80051c0 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e800c0d0 con 0x7f22e8003a60 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e800c290 con 0x7f22e8004810 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2302525700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e80051c0 0x7f22e800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2301d24700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e8003a60 0x7f22e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2301523700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22e8004810 0x7f22e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2302525700 1 -- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22e8004810 msgr2=0x7f22e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2302525700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22e8004810 0x7f22e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2302525700 1 -- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e8003a60 msgr2=0x7f22e8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2302525700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e8003a60 0x7f22e8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2302525700 1 -- 172.21.15.45:0/393307379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e8003570 con 0x7f22e80051c0 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2301d24700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e8003a60 0x7f22e8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.100+0000 7f2302525700 1 -- 172.21.15.45:0/393307379 reap_dead start 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2302525700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e80051c0 0x7f22e800ba60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f22f400a930 tx=0x7f22f40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e50852bf79f6b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40070d0 con 0x7f22e80051c0 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4007230 con 0x7f22e80051c0 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2300d22700 1 -- 172.21.15.45:0/393307379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40075d0 con 0x7f22e80051c0 2023-10-06T05:22:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2303f88700 1 -- 172.21.15.45:0/393307379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e80051c0 msgr2=0x7f22e800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2303f88700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e80051c0 0x7f22e800ba60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f22f400a930 tx=0x7f22f40021d0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2303f88700 1 -- 172.21.15.45:0/393307379 shutdown_connections 2023-10-06T05:22:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2303f88700 1 --2- 172.21.15.45:0/393307379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e80051c0 0x7f22e800ba60 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2303f88700 1 -- 172.21.15.45:0/393307379 >> 172.21.15.45:0/393307379 conn(0x7f22fc0cdf50 msgr2=0x7f22fc0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2303f88700 1 -- 172.21.15.45:0/393307379 shutdown_connections 2023-10-06T05:22:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.101+0000 7f2303f88700 1 -- 172.21.15.45:0/393307379 wait complete. 2023-10-06T05:22:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 Processor -- start 2023-10-06T05:22:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 -- start start 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e80051c0 0x7f22fc163450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22fc155620 0x7f22fc15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc0b2050 0x7f22fc0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0b2a00 con 0x7f22fc155620 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc160060 con 0x7f22e80051c0 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2303f88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc160340 con 0x7f22fc0b2050 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.103+0000 7f2301523700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22fc155620 0x7f22fc15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22fc155620 0x7f22fc15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56606/0 (socket says 172.21.15.45:56606) 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 -- 172.21.15.45:0/113714500 learned_addr learned my addr 172.21.15.45:0/113714500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301d24700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e80051c0 0x7f22fc163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2302525700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc0b2050 0x7f22fc0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 -- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc0b2050 msgr2=0x7f22fc0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc0b2050 0x7f22fc0b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 -- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e80051c0 msgr2=0x7f22fc163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e80051c0 0x7f22fc163450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 -- 172.21.15.45:0/113714500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc160620 con 0x7f22fc155620 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2301523700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22fc155620 0x7f22fc15f820 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f22ec002950 tx=0x7f22ec007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f22f1ffb700 1 -- 172.21.15.45:0/113714500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec002e90 con 0x7f22fc155620 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f22f1ffb700 1 -- 172.21.15.45:0/113714500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec00f8d0 con 0x7f22fc155620 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc0c5710 con 0x7f22fc155620 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f22f1ffb700 1 -- 172.21.15.45:0/113714500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00fc90 con 0x7f22fc155620 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.104+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc15fd60 con 0x7f22fc155620 2023-10-06T05:22:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.105+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc0c5950 con 0x7f22fc155620 2023-10-06T05:22:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.106+0000 7f22f1ffb700 1 -- 172.21.15.45:0/113714500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22ec006820 con 0x7f22fc155620 2023-10-06T05:22:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.106+0000 7f22f1ffb700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22e0066f60 0x7f22e0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.107+0000 7f22f1ffb700 1 -- 172.21.15.45:0/113714500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f22ec084920 con 0x7f22fc155620 2023-10-06T05:22:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.107+0000 7f2301d24700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22e0066f60 0x7f22e0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.107+0000 7f2301d24700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22e0066f60 0x7f22e0069410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009170 tx=0x7f22f8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.111+0000 7f22f1ffb700 1 -- 172.21.15.45:0/113714500 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22fc0c5950 con 0x7f22fc155620 2023-10-06T05:22:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.357+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f22fc1618f0 con 0x7f22fc155620 2023-10-06T05:22:23.360 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:23.358+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.362+0000 7f22f1ffb700 1 -- 172.21.15.45:0/113714500 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) v1 ==== 161+0+0 (secure 0 0 0) 0x7f22ec0518a0 con 0x7f22fc155620 2023-10-06T05:22:23.364 INFO:tasks.ceph.mds.a.smithi045.stderr: -598> 2023-10-06T05:22:18.122+0000 7f93404b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:23.365 INFO:tasks.ceph.mds.a.smithi045.stderr: -87> 2023-10-06T05:22:22.764+0000 7f933ccac700 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2023-10-06T05:22:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22e0066f60 msgr2=0x7f22e0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22e0066f60 0x7f22e0069410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009170 tx=0x7f22f8008040 comp rx=0 tx=0).stop 2023-10-06T05:22:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22fc155620 msgr2=0x7f22fc15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22fc155620 0x7f22fc15f820 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f22ec002950 tx=0x7f22ec007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 shutdown_connections 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22e0066f60 0x7f22e0069410 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc0b2050 0x7f22fc0b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e80051c0 0x7f22fc163450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 --2- 172.21.15.45:0/113714500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22fc155620 0x7f22fc15f820 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 >> 172.21.15.45:0/113714500 conn(0x7f22fc0cdf50 msgr2=0x7f22fc06d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 shutdown_connections 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.364+0000 7f2303f88700 1 -- 172.21.15.45:0/113714500 wait complete. 2023-10-06T05:22:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:23.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:22:23.387 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:22:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.512+0000 7f7ac9f88700 1 Processor -- start 2023-10-06T05:22:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.512+0000 7f7ac9f88700 1 -- start start 2023-10-06T05:22:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.513+0000 7f7ac9f88700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ac4159330 0x7f7ac406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.513+0000 7f7ac9f88700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac40cb530 con 0x7f7ac4158f60 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.513+0000 7f7ac9f88700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac4069bd0 con 0x7f7ac40cc390 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.513+0000 7f7ac9f88700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac4068fc0 con 0x7f7ac4159330 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.513+0000 7f7ac2ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ac4159330 0x7f7ac406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.513+0000 7f7ac37fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7ac4158f60 0x7f7ac40cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44226/0 (socket says 172.21.15.45:44226) 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.513+0000 7f7ac37fe700 1 -- 172.21.15.45:0/1356171008 learned_addr learned my addr 172.21.15.45:0/1356171008 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac2ffd700 1 -- 172.21.15.45:0/1356171008 >> v1:172.21.15.144:6789/0 conn(0x7f7ac40cc390 legacy=0x7f7ac40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac2ffd700 1 -- 172.21.15.45:0/1356171008 >> v1:172.21.15.45:6789/0 conn(0x7f7ac4158f60 legacy=0x7f7ac40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac2ffd700 1 -- 172.21.15.45:0/1356171008 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac40692a0 con 0x7f7ac4159330 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac27fc700 1 -- 172.21.15.45:0/1356171008 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775194037 0 0) 0x7f7ab4001110 con 0x7f7ac4158f60 2023-10-06T05:22:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac2ffd700 1 --2- 172.21.15.45:0/1356171008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159330 0x7f7ac406a060 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0098e0 tx=0x7f7aac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5f55ddc680d28c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac27fc700 1 -- 172.21.15.45:0/1356171008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac009e80 con 0x7f7ac4159330 2023-10-06T05:22:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac27fc700 1 -- 172.21.15.45:0/1356171008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac00b860 con 0x7f7ac4159330 2023-10-06T05:22:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.514+0000 7f7ac27fc700 1 -- 172.21.15.45:0/1356171008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00bbe0 con 0x7f7ac4159330 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.515+0000 7f7ac9f88700 1 -- 172.21.15.45:0/1356171008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159330 msgr2=0x7f7ac406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.515+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/1356171008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159330 0x7f7ac406a060 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0098e0 tx=0x7f7aac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.515+0000 7f7ac9f88700 1 -- 172.21.15.45:0/1356171008 shutdown_connections 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.515+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/1356171008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac4159330 0x7f7ac406a060 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.515+0000 7f7ac9f88700 1 -- 172.21.15.45:0/1356171008 >> 172.21.15.45:0/1356171008 conn(0x7f7ac40cdf50 msgr2=0x7f7ac40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.515+0000 7f7ac9f88700 1 -- 172.21.15.45:0/1356171008 shutdown_connections 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.515+0000 7f7ac9f88700 1 -- 172.21.15.45:0/1356171008 wait complete. 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.516+0000 7f7ac9f88700 1 Processor -- start 2023-10-06T05:22:23.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.516+0000 7f7ac9f88700 1 -- start start 2023-10-06T05:22:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac9f88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cc390 0x7f7ac40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac9f88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4158f60 0x7f7ac41577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac9f88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac4159330 0x7f7ac4157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac9f88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac4169c70 con 0x7f7ac40cc390 2023-10-06T05:22:23.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac9f88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac415c4b0 con 0x7f7ac4159330 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac9f88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac415c790 con 0x7f7ac4158f60 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cc390 0x7f7ac40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cc390 0x7f7ac40b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56658/0 (socket says 172.21.15.45:56658) 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 -- 172.21.15.45:0/4118381982 learned_addr learned my addr 172.21.15.45:0/4118381982 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac2ffd700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4158f60 0x7f7ac41577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac3fff700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac4159330 0x7f7ac4157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 -- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4158f60 msgr2=0x7f7ac41577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4158f60 0x7f7ac41577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 -- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac4159330 msgr2=0x7f7ac4157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac4159330 0x7f7ac4157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac37fe700 1 -- 172.21.15.45:0/4118381982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac009420 con 0x7f7ac40cc390 2023-10-06T05:22:23.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.517+0000 7f7ac2ffd700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4158f60 0x7f7ac41577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:23.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.518+0000 7f7ac37fe700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cc390 0x7f7ac40b21f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c810 tx=0x7f7ab4007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.518+0000 7f7abb7fe700 1 -- 172.21.15.45:0/4118381982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4005b30 con 0x7f7ac40cc390 2023-10-06T05:22:23.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.518+0000 7f7abb7fe700 1 -- 172.21.15.45:0/4118381982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4005cb0 con 0x7f7ac40cc390 2023-10-06T05:22:23.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.518+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac415ca70 con 0x7f7ac40cc390 2023-10-06T05:22:23.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.518+0000 7f7abb7fe700 1 -- 172.21.15.45:0/4118381982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4007340 con 0x7f7ac40cc390 2023-10-06T05:22:23.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.518+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac40b2730 con 0x7f7ac40cc390 2023-10-06T05:22:23.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.519+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac40b29b0 con 0x7f7ac40cc390 2023-10-06T05:22:23.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.519+0000 7f7abb7fe700 1 -- 172.21.15.45:0/4118381982 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ab401b400 con 0x7f7ac40cc390 2023-10-06T05:22:23.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.520+0000 7f7abb7fe700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa4066f30 0x7f7aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.520+0000 7f7abb7fe700 1 -- 172.21.15.45:0/4118381982 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f7ab4084200 con 0x7f7ac40cc390 2023-10-06T05:22:23.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.520+0000 7f7ac2ffd700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa4066f30 0x7f7aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.520+0000 7f7ac2ffd700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa4066f30 0x7f7aa40693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0091e0 tx=0x7f7aac017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.524+0000 7f7abb7fe700 1 -- 172.21.15.45:0/4118381982 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ab4051120 con 0x7f7ac40cc390 2023-10-06T05:22:23.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:23.811+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f7ac40cb530 con 0x7f7ac40cc390 2023-10-06T05:22:24.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.364+0000 7f7abb7fe700 1 -- 172.21.15.45:0/4118381982 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v97) v1 ==== 107+0+0 (secure 0 0 0) 0x7f7ac40cb530 con 0x7f7ac40cc390 2023-10-06T05:22:24.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.367+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa4066f30 msgr2=0x7f7aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.367+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa4066f30 0x7f7aa40693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0091e0 tx=0x7f7aac017000 comp rx=0 tx=0).stop 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.367+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cc390 msgr2=0x7f7ac40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cc390 0x7f7ac40b21f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c810 tx=0x7f7ab4007c10 comp rx=0 tx=0).stop 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 shutdown_connections 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7aa4066f30 0x7f7aa40693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac4159330 0x7f7ac4157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ac40cc390 0x7f7ac40b21f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 --2- 172.21.15.45:0/4118381982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac4158f60 0x7f7ac41577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 >> 172.21.15.45:0/4118381982 conn(0x7f7ac40cdf50 msgr2=0x7f7ac40b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 shutdown_connections 2023-10-06T05:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.368+0000 7f7ac9f88700 1 -- 172.21.15.45:0/4118381982 wait complete. 2023-10-06T05:22:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:22:24.385 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:22:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.473+0000 7f1c265eb040 1 Processor -- start 2023-10-06T05:22:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.473+0000 7f1c265eb040 1 -- start start 2023-10-06T05:22:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c265eb040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce1777a7e0 0x55ce17730480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c265eb040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 0x55ce17760380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c265eb040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce17805f00 0x55ce177608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c265eb040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1767d980 con 0x55ce17805f00 2023-10-06T05:22:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c265eb040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1767d620 con 0x55ce1777a7e0 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c265eb040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce17674400 con 0x55ce17805b30 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce17805f00 0x55ce177608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce17805f00 0x55ce177608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56674/0 (socket says 172.21.15.45:56674) 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 -- 172.21.15.45:0/679705530 learned_addr learned my addr 172.21.15.45:0/679705530 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c1f809700 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce1777a7e0 0x55ce17730480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c1f008700 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 0x55ce17760380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 -- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 msgr2=0x55ce17760380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 0x55ce17760380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 -- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce1777a7e0 msgr2=0x55ce17730480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce1777a7e0 0x55ce17730480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 -- 172.21.15.45:0/679705530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10005db0 con 0x55ce17805f00 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c2000a700 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce17805f00 0x55ce177608c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c4c0 tx=0x7f1c18007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb9c3b4dcda9da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.474+0000 7f1c1e006700 1 -- 172.21.15.45:0/679705530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800ce90 con 0x55ce17805f00 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c1e006700 1 -- 172.21.15.45:0/679705530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c180115f0 con 0x55ce17805f00 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c1e006700 1 -- 172.21.15.45:0/679705530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180119b0 con 0x55ce17805f00 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 -- 172.21.15.45:0/679705530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce17805f00 msgr2=0x55ce177608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce17805f00 0x55ce177608c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c4c0 tx=0x7f1c18007040 comp rx=0 tx=0).stop 2023-10-06T05:22:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 -- 172.21.15.45:0/679705530 shutdown_connections 2023-10-06T05:22:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce17805f00 0x55ce177608c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce1777a7e0 0x55ce17730480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 --2- 172.21.15.45:0/679705530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 0x55ce17760380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 -- 172.21.15.45:0/679705530 >> 172.21.15.45:0/679705530 conn(0x55ce17733450 msgr2=0x55ce17803e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 -- 172.21.15.45:0/679705530 shutdown_connections 2023-10-06T05:22:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.475+0000 7f1c265eb040 1 -- 172.21.15.45:0/679705530 wait complete. 2023-10-06T05:22:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.476+0000 7f1c265eb040 1 Processor -- start 2023-10-06T05:22:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.476+0000 7f1c265eb040 1 -- start start 2023-10-06T05:22:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c265eb040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1777a7e0 0x55ce17777e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c265eb040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 0x55ce177ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c265eb040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce17805f00 0x55ce177ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c265eb040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce178141b0 con 0x55ce1777a7e0 2023-10-06T05:22:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c265eb040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce178150d0 con 0x55ce17805f00 2023-10-06T05:22:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c265eb040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce178143e0 con 0x55ce17805b30 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c1f809700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1777a7e0 0x55ce17777e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c1f809700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1777a7e0 0x55ce17777e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56678/0 (socket says 172.21.15.45:56678) 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 learned_addr learned my addr 172.21.15.45:0/2609985519 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c2000a700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce17805f00 0x55ce177ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.477+0000 7f1c1f008700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 0x55ce177ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 msgr2=0x55ce177ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1f809700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ce17805b30 0x55ce177ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce17805f00 msgr2=0x55ce177ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1f809700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ce17805f00 0x55ce177ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1800d040 con 0x55ce1777a7e0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1f809700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1777a7e0 0x55ce17777e10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f1c10008410 tx=0x7f1c10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1c803700 1 -- 172.21.15.45:0/2609985519 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10008c60 con 0x55ce1777a7e0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1c803700 1 -- 172.21.15.45:0/2609985519 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c10021070 con 0x55ce1777a7e0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce1768c3d0 con 0x55ce1777a7e0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c1c803700 1 -- 172.21.15.45:0/2609985519 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c100286b0 con 0x55ce1777a7e0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.478+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce1767fae0 con 0x55ce1777a7e0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.480+0000 7f1c1c803700 1 -- 172.21.15.45:0/2609985519 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c10028a60 con 0x55ce1777a7e0 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.480+0000 7f1c1c803700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c00065890 0x7f1c00067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.480+0000 7f1c1c803700 1 -- 172.21.15.45:0/2609985519 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f1c10023070 con 0x55ce1777a7e0 2023-10-06T05:22:24.488 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:22:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.480+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ce17778350 0x55ce1781da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.480+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1775f950 con 0x55ce17778350 2023-10-06T05:22:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.480+0000 7f1c1f008700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c00065890 0x7f1c00067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.481+0000 7f1c2000a700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ce17778350 0x55ce1781da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.481+0000 7f1c1f008700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c00065890 0x7f1c00067d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1c08000c00 tx=0x7f1c08001780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.481+0000 7f1c2000a700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ce17778350 0x55ce1781da80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.485+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v94'43 uv16 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.485+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce177ffe30 con 0x55ce17778350 2023-10-06T05:22:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.486+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v94'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.486+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17761d30 con 0x55ce17778350 2023-10-06T05:22:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.487+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v94'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.491 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:22:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.487+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1781dfc0 con 0x55ce17778350 2023-10-06T05:22:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.489+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v94'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.489+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827860 con 0x55ce17778350 2023-10-06T05:22:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.490+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v94'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.490+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826850 con 0x55ce17778350 2023-10-06T05:22:24.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.491+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v94'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.491+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826c00 con 0x55ce17778350 2023-10-06T05:22:24.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.492+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v94'49 uv22 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.492+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827000 con 0x55ce17778350 2023-10-06T05:22:24.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.493+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v94'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.494+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827000 con 0x55ce17778350 2023-10-06T05:22:24.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.495+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v94'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.497 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:22:24.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.495+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826fb0 con 0x55ce17778350 2023-10-06T05:22:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.497+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v94'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.497+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826fb0 con 0x55ce17778350 2023-10-06T05:22:24.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.499+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v94'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.499+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826fb0 con 0x55ce17778350 2023-10-06T05:22:24.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.500+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v94'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.500+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826fb0 con 0x55ce17778350 2023-10-06T05:22:24.505 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:22:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.501+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v94'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.501+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826fb0 con 0x55ce17778350 2023-10-06T05:22:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.502+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v94'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.502+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826fb0 con 0x55ce17778350 2023-10-06T05:22:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.504+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v94'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.504+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17826fb0 con 0x55ce17778350 2023-10-06T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.505+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v94'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.505+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.506+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v94'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.506+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.507+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 600.00000000 [create] v94'60 uv35 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.507+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.508+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 600.00000000 [delete] v94'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.508+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.512 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:22:24.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.511+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v94'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.511+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17828c20 con 0x55ce17778350 2023-10-06T05:22:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.513+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v94'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.513+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17828c20 con 0x55ce17778350 2023-10-06T05:22:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.519+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v94'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.519+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.520+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v94'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.520+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.523+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 603.00000000 [create] v94'66 uv41 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.523+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.529+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 603.00000000 [delete] v94'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.529+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.532 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:22:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.530+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v94'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.530+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17828c20 con 0x55ce17778350 2023-10-06T05:22:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.533+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v94'69 uv69 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.533+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17828c20 con 0x55ce17778350 2023-10-06T05:22:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.537+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v94'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.537+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17827c10 con 0x55ce17778350 2023-10-06T05:22:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.539+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v94'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17778350 2023-10-06T05:22:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.539+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ce17828120 0x55ce1782a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.539+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:30 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782aa40 con 0x55ce17828120 2023-10-06T05:22:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.540+0000 7f1c1f809700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ce17828120 0x55ce1782a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.540+0000 7f1c1f809700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ce17828120 0x55ce1782a500 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.541+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(30 1.00000000.inode [create] v94'44 uv24 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.541+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:31 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782adf0 con 0x55ce17828120 2023-10-06T05:22:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.543+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(31 1.00000000.inode [delete] v94'45 uv45 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.543+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782b1a0 con 0x55ce17828120 2023-10-06T05:22:24.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.544+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(32 1.00000000.inode [create] v94'46 uv46 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.545 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:22:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.544+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782ba40 con 0x55ce17828120 2023-10-06T05:22:24.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.545+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(33 1.00000000.inode [write 0~554] v94'47 uv47 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.545+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782bdf0 con 0x55ce17828120 2023-10-06T05:22:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.546+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v94'48 uv48 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.546+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c1a0 con 0x55ce17828120 2023-10-06T05:22:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.547+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(35 200.00000000 [create] v94'49 uv43 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.547+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.549+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(36 200.00000000 [delete] v94'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.549+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.550+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(37 200.00000000 [create] v94'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.552 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:22:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.550+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.552+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v94'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.552+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.554+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v94'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.554+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.555+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(40 200.00000001 [create] v94'54 uv42 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.555+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.556+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(41 200.00000001 [delete] v94'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.556+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.559 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:22:24.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.557+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(42 200.00000001 [create] v94'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.557+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782fc70 con 0x55ce17828120 2023-10-06T05:22:24.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.559+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(43 200.00000001 [write 0~14075] v94'57 uv57 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.559+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782fc70 con 0x55ce17828120 2023-10-06T05:22:24.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.560+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v94'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.560+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.561+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(45 606.00000000 [create] v94'59 uv40 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.562+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.563+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(46 606.00000000 [delete] v94'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.563+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.566 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:22:24.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.564+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(47 606.00000000 [create] v94'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.564+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782d560 con 0x55ce17828120 2023-10-06T05:22:24.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.565+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v94'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.565+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782d560 con 0x55ce17828120 2023-10-06T05:22:24.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.567+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v94'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.567+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782c550 con 0x55ce17828120 2023-10-06T05:22:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.568+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v94'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.568+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ce1782ca60 0x55ce1782ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.568+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:51 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782f380 con 0x55ce1782ca60 2023-10-06T05:22:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.568+0000 7f1c1f008700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ce1782ca60 0x55ce1782ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.569+0000 7f1c1f008700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ce1782ca60 0x55ce1782ee40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.570+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(51 607.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.570+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:52 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782f730 con 0x55ce1782ca60 2023-10-06T05:22:24.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.571+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(52 607.00000000 [delete] v94'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.571+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1782fae0 con 0x55ce1782ca60 2023-10-06T05:22:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.572+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(53 607.00000000 [create] v94'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.574 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:22:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.572+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17831390 con 0x55ce1782ca60 2023-10-06T05:22:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.574+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.574+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17831740 con 0x55ce1782ca60 2023-10-06T05:22:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.575+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.575+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830380 con 0x55ce1782ca60 2023-10-06T05:22:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.576+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.576+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830780 con 0x55ce1782ca60 2023-10-06T05:22:24.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.577+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(57 mds0_openfiles.0 [create] v94'8 uv7 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.577+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830bb0 con 0x55ce1782ca60 2023-10-06T05:22:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.579+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v94'9 uv9 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.579+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830fe0 con 0x55ce1782ca60 2023-10-06T05:22:24.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.580+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(59 mds0_openfiles.0 [create] v94'10 uv10 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.582 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:22:24.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.580+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830bb0 con 0x55ce1782ca60 2023-10-06T05:22:24.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.581+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v94'11 uv11 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.581+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:61 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830b30 con 0x55ce17828120 2023-10-06T05:22:24.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.583+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(61 608.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.583+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:62 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830b30 con 0x55ce17828120 2023-10-06T05:22:24.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.584+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(62 608.00000000 [delete] v94'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.584+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830b30 con 0x55ce17828120 2023-10-06T05:22:24.586 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:22:24.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.585+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(63 608.00000000 [create] v94'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.585+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830b30 con 0x55ce17828120 2023-10-06T05:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.586+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.586+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830b30 con 0x55ce17828120 2023-10-06T05:22:24.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.587+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.587+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17830b30 con 0x55ce17828120 2023-10-06T05:22:24.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.589+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(66 608.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17828120 2023-10-06T05:22:24.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.589+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ce17830f40 0x55ce17834010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.589+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:67 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17834550 con 0x55ce17830f40 2023-10-06T05:22:24.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.589+0000 7f1c2000a700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ce17830f40 0x55ce17834010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.590+0000 7f1c2000a700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ce17830f40 0x55ce17834010 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.591+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(67 604.00000000 [create] v94'15 uv9 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.591+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:68 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17834550 con 0x55ce17830f40 2023-10-06T05:22:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.594+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(68 604.00000000 [delete] v94'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.594+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17834550 con 0x55ce17830f40 2023-10-06T05:22:24.597 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:22:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.596+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(69 604.00000000 [create] v94'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.596+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17835600 con 0x55ce17830f40 2023-10-06T05:22:24.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.598+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v94'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.598+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce178355c0 con 0x55ce17830f40 2023-10-06T05:22:24.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.601+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v94'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.601+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce178355c0 con 0x55ce17830f40 2023-10-06T05:22:24.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.604+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v94'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.604+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce178355c0 con 0x55ce17830f40 2023-10-06T05:22:24.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.606+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(73 500.00000000 [create] v94'21 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.606+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17835580 con 0x55ce17830f40 2023-10-06T05:22:24.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.609+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(74 500.00000000 [delete] v94'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.609+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17834550 con 0x55ce17830f40 2023-10-06T05:22:24.614 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:22:24.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.612+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(75 500.00000000 [create] v94'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.612+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce178345b0 con 0x55ce17830f40 2023-10-06T05:22:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.617+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v94'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.617+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce178349d0 con 0x55ce17830f40 2023-10-06T05:22:24.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.619+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v94'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.619+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ce17834ee0 0x55ce17835310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.619+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:78 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837a20 con 0x55ce17834ee0 2023-10-06T05:22:24.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.619+0000 7f1c1f809700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ce17834ee0 0x55ce17835310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.619+0000 7f1c1f809700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ce17834ee0 0x55ce17835310 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.620+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(78 mds_snaptable [create] v94'7 uv6 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17834ee0 2023-10-06T05:22:24.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.621+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:79 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837a20 con 0x55ce17834ee0 2023-10-06T05:22:24.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.622+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(79 mds_snaptable [delete] v94'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17834ee0 2023-10-06T05:22:24.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.622+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837a20 con 0x55ce17834ee0 2023-10-06T05:22:24.626 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:22:24.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.624+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(80 mds_snaptable [create] v94'9 uv9 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17834ee0 2023-10-06T05:22:24.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.624+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837a60 con 0x55ce17834ee0 2023-10-06T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.627+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v94'10 uv10 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17834ee0 2023-10-06T05:22:24.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.627+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837e60 con 0x55ce17834ee0 2023-10-06T05:22:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.630+0000 7f1c1f809700 1 -- 172.21.15.45:0/2609985519 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v94'11 uv11 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c10054e00 con 0x55ce17834ee0 2023-10-06T05:22:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.630+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:83 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837e60 con 0x55ce1782ca60 2023-10-06T05:22:24.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.631+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(83 605.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.631+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:84 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837e60 con 0x55ce1782ca60 2023-10-06T05:22:24.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.632+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(84 605.00000000 [delete] v94'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.632+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837e60 con 0x55ce1782ca60 2023-10-06T05:22:24.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.633+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(85 605.00000000 [create] v94'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.635 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:22:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.633+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17838e70 con 0x55ce1782ca60 2023-10-06T05:22:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.635+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.635+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17838e70 con 0x55ce1782ca60 2023-10-06T05:22:24.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.636+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.636+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce17837e60 con 0x55ce1782ca60 2023-10-06T05:22:24.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.637+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.637+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ce17838370 0x55ce1783a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.637+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:89 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783ad70 con 0x55ce17838370 2023-10-06T05:22:24.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.637+0000 7f1c1f008700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ce17838370 0x55ce1783a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.638+0000 7f1c1f008700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ce17838370 0x55ce1783a750 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.639+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(89 mds0_inotable [create] v94'14 uv8 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.639+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:90 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783ae30 con 0x55ce17838370 2023-10-06T05:22:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.640+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(90 mds0_inotable [delete] v94'15 uv15 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.640+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b260 con 0x55ce17838370 2023-10-06T05:22:24.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.641+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(91 mds0_inotable [create] v94'16 uv16 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.643 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:22:24.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.641+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b260 con 0x55ce17838370 2023-10-06T05:22:24.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.643+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(92 mds0_inotable [write 0~34] v94'17 uv17 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.643+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b260 con 0x55ce17838370 2023-10-06T05:22:24.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.644+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v94'18 uv18 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.644+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b260 con 0x55ce17838370 2023-10-06T05:22:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.645+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(94 100.00000000 [create] v94'19 uv13 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.645+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b260 con 0x55ce17838370 2023-10-06T05:22:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.646+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(95 100.00000000 [delete] v94'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.646+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b260 con 0x55ce17838370 2023-10-06T05:22:24.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.647+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(96 100.00000000 [create] v94'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.649 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:22:24.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.647+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b260 con 0x55ce17838370 2023-10-06T05:22:24.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.648+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(97 100.00000000 [omap-set-header] v94'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.648+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783dfd0 con 0x55ce17838370 2023-10-06T05:22:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.649+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v94'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce17838370 2023-10-06T05:22:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.649+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.650+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v94'9 uv7 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.650+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.652+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v94'10 uv10 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.652+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.653+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v94'11 uv11 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.655 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:22:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.653+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.654+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v94'12 uv12 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.654+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ce1783c060 0x55ce1783e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.654+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1783c060 2023-10-06T05:22:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.655+0000 7f1c2000a700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ce1783c060 0x55ce1783e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.655+0000 7f1c2000a700 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ce1783c060 0x55ce1783e720 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.656+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce1783c060 2023-10-06T05:22:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.656+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1783c060 2023-10-06T05:22:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.657+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v94'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce1783c060 2023-10-06T05:22:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.657+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1783c060 2023-10-06T05:22:24.661 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:22:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.658+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v94'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce1783c060 2023-10-06T05:22:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.658+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1783c060 2023-10-06T05:22:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.659+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce1783c060 2023-10-06T05:22:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.660+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1783c060 2023-10-06T05:22:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.661+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce1783c060 2023-10-06T05:22:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.661+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1783c060 2023-10-06T05:22:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.662+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce1783c060 2023-10-06T05:22:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.662+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce17830f40 2023-10-06T05:22:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.663+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v94'7 uv6 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.663+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce17830f40 2023-10-06T05:22:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.667+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v94'8 uv8 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.667+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce17830f40 2023-10-06T05:22:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.668+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v94'9 uv9 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.671 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:22:24.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.668+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce17830f40 2023-10-06T05:22:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.670+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v94'10 uv10 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.670+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce17830f40 2023-10-06T05:22:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.676+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v94'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1c18018020 con 0x55ce17830f40 2023-10-06T05:22:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.676+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.677+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v94'20 uv11 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.677+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.681+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v94'21 uv21 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.681+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.682+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v94'22 uv22 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.684 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:22:24.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.682+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.685+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v94'23 uv23 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.685+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.687+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v94'24 uv24 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.687+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.688+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v94'25 uv18 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.688+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.689+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v94'26 uv26 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.689+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.691+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v94'27 uv27 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.692 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:22:24.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.691+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.692+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v94'28 uv28 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.692+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.693+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v94'29 uv29 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.693+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783b750 con 0x55ce1782ca60 2023-10-06T05:22:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.694+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v94'30 uv30 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.694+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x55ce1783ec60 con 0x55ce1782ca60 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c1f008700 1 -- 172.21.15.45:0/2609985519 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v94'31 uv31 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1c08017040 con 0x55ce1782ca60 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ce1782ca60 msgr2=0x55ce1782ee40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ce1782ca60 0x55ce1782ee40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ce17778350 msgr2=0x55ce1781da80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ce17778350 0x55ce1781da80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ce17838370 msgr2=0x55ce1783a750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ce17838370 0x55ce1783a750 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ce17828120 msgr2=0x55ce1782a500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ce17828120 0x55ce1782a500 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ce1783c060 msgr2=0x55ce1783e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ce1783c060 0x55ce1783e720 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ce17834ee0 msgr2=0x55ce17835310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ce17834ee0 0x55ce17835310 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ce17830f40 msgr2=0x55ce17834010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.695+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ce17830f40 0x55ce17834010 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c00065890 msgr2=0x7f1c00067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c00065890 0x7f1c00067d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1c08000c00 tx=0x7f1c08001780 comp rx=0 tx=0).stop 2023-10-06T05:22:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1777a7e0 msgr2=0x55ce17777e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 --2- 172.21.15.45:0/2609985519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1777a7e0 0x55ce17777e10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f1c10008410 tx=0x7f1c10016000 comp rx=0 tx=0).stop 2023-10-06T05:22:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 reap_dead start 2023-10-06T05:22:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 reap_dead start 2023-10-06T05:22:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 reap_dead start 2023-10-06T05:22:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 reap_dead start 2023-10-06T05:22:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c2000a700 1 -- 172.21.15.45:0/2609985519 reap_dead start 2023-10-06T05:22:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 shutdown_connections 2023-10-06T05:22:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 >> 172.21.15.45:0/2609985519 conn(0x55ce17733450 msgr2=0x55ce17806390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 shutdown_connections 2023-10-06T05:22:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.696+0000 7f1c265eb040 1 -- 172.21.15.45:0/2609985519 wait complete. 2023-10-06T05:22:24.701 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2023-10-06T05:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.749+0000 7f49ff4cd040 1 Processor -- start 2023-10-06T05:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.749+0000 7f49ff4cd040 1 -- start start 2023-10-06T05:22:24.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49ff4cd040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567256532f0 0x556725707dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49ff4cd040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556725657450 0x5567256f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49ff4cd040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256fdfc0 0x55672566bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49ff4cd040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567255ce320 con 0x5567256532f0 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49ff4cd040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556725572980 con 0x556725657450 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49ff4cd040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556725572620 con 0x5567256fdfc0 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567256532f0 0x556725707dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567256532f0 0x556725707dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56694/0 (socket says 172.21.15.45:56694) 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 -- 172.21.15.45:0/542565435 learned_addr learned my addr 172.21.15.45:0/542565435 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49efeea700 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556725657450 0x5567256f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f8eec700 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256fdfc0 0x55672566bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 -- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256fdfc0 msgr2=0x55672566bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256fdfc0 0x55672566bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 -- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556725657450 msgr2=0x5567256f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556725657450 0x5567256f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.750+0000 7f49f86eb700 1 -- 172.21.15.45:0/542565435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0005db0 con 0x5567256532f0 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49f86eb700 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567256532f0 0x556725707dc0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f49e800b230 tx=0x7f49e80017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86814a910b3315 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49f76e9700 1 -- 172.21.15.45:0/542565435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8014070 con 0x5567256532f0 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49f76e9700 1 -- 172.21.15.45:0/542565435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e800bdc0 con 0x5567256532f0 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49f76e9700 1 -- 172.21.15.45:0/542565435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e800f6a0 con 0x5567256532f0 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 -- 172.21.15.45:0/542565435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567256532f0 msgr2=0x556725707dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567256532f0 0x556725707dc0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f49e800b230 tx=0x7f49e80017b0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 -- 172.21.15.45:0/542565435 shutdown_connections 2023-10-06T05:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256fdfc0 0x55672566bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567256532f0 0x556725707dc0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/542565435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556725657450 0x5567256f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 -- 172.21.15.45:0/542565435 >> 172.21.15.45:0/542565435 conn(0x556725627bd0 msgr2=0x5567257003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 -- 172.21.15.45:0/542565435 shutdown_connections 2023-10-06T05:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.751+0000 7f49ff4cd040 1 -- 172.21.15.45:0/542565435 wait complete. 2023-10-06T05:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.752+0000 7f49ff4cd040 1 Processor -- start 2023-10-06T05:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.752+0000 7f49ff4cd040 1 -- start start 2023-10-06T05:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.753+0000 7f49ff4cd040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256532f0 0x55672566cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.753+0000 7f49ff4cd040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556725657450 0x5567256f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.753+0000 7f49ff4cd040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567256fdfc0 0x55672566be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.753+0000 7f49ff4cd040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556725569400 con 0x556725657450 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.753+0000 7f49ff4cd040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556725707c00 con 0x5567256fdfc0 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.753+0000 7f49ff4cd040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567256f5200 con 0x5567256532f0 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556725657450 0x5567256f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556725657450 0x5567256f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56706/0 (socket says 172.21.15.45:56706) 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 -- 172.21.15.45:0/3322236129 learned_addr learned my addr 172.21.15.45:0/3322236129 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49f86eb700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256532f0 0x55672566cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49f8eec700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567256fdfc0 0x55672566be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 -- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256532f0 msgr2=0x55672566cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256532f0 0x55672566cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 -- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567256fdfc0 msgr2=0x55672566be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567256fdfc0 0x55672566be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 -- 172.21.15.45:0/3322236129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e8001a90 con 0x556725657450 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49f86eb700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567256532f0 0x55672566cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49efeea700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556725657450 0x5567256f3140 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f49e0008410 tx=0x7f49e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49f5ee6700 1 -- 172.21.15.45:0/3322236129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00054f0 con 0x556725657450 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49f5ee6700 1 -- 172.21.15.45:0/3322236129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e0012ce0 con 0x556725657450 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556725574ae0 con 0x556725657450 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49f5ee6700 1 -- 172.21.15.45:0/3322236129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e001c660 con 0x556725657450 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.754+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556725574540 con 0x556725657450 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.756+0000 7f49f5ee6700 1 -- 172.21.15.45:0/3322236129 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49e001ca10 con 0x556725657450 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.756+0000 7f49f5ee6700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49d80658c0 0x7f49d8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.756+0000 7f49f5ee6700 1 -- 172.21.15.45:0/3322236129 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f49e00857c0 con 0x556725657450 2023-10-06T05:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.757+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5567256fc850 0x5567256ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.757+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567256ff8a0 con 0x5567256fc850 2023-10-06T05:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.757+0000 7f49f86eb700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49d80658c0 0x7f49d8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.757+0000 7f49f8eec700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5567256fc850 0x5567256ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.757+0000 7f49f86eb700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49d80658c0 0x7f49d8067d70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f49e80017f0 tx=0x7f49e8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.757+0000 7f49f8eec700 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5567256fc850 0x5567256ff360 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49f8eec700 1 -- 172.21.15.45:0/3322236129 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 600.00000000 [delete] v94'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f49f00028a0 con 0x5567256fc850 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5567256fc850 msgr2=0x5567256ff360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5567256fc850 0x5567256ff360 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49d80658c0 msgr2=0x7f49d8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49d80658c0 0x7f49d8067d70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f49e80017f0 tx=0x7f49e8016000 comp rx=0 tx=0).stop 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556725657450 msgr2=0x5567256f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 --2- 172.21.15.45:0/3322236129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556725657450 0x5567256f3140 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f49e0008410 tx=0x7f49e0016000 comp rx=0 tx=0).stop 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49f8eec700 1 -- 172.21.15.45:0/3322236129 reap_dead start 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 shutdown_connections 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 >> 172.21.15.45:0/3322236129 conn(0x556725627bd0 msgr2=0x5567256f4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 shutdown_connections 2023-10-06T05:22:24.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.759+0000 7f49ff4cd040 1 -- 172.21.15.45:0/3322236129 wait complete. 2023-10-06T05:22:24.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:22:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.900+0000 7fc4788c6700 1 Processor -- start 2023-10-06T05:22:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.900+0000 7fc4788c6700 1 -- start start 2023-10-06T05:22:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.901+0000 7fc4788c6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc4740c9dd0 0x7fc4740ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.901+0000 7fc4788c6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4740c9940 con 0x7fc474140ab0 2023-10-06T05:22:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.901+0000 7fc4788c6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4740ca750 con 0x7fc4741406e0 2023-10-06T05:22:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.901+0000 7fc4788c6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47406d230 con 0x7fc4740c9dd0 2023-10-06T05:22:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.901+0000 7fc471d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc474140ab0 0x7fc4740acfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44236/0 (socket says 172.21.15.45:44236) 2023-10-06T05:22:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.901+0000 7fc471d9b700 1 -- 172.21.15.45:0/3677289030 learned_addr learned my addr 172.21.15.45:0/3677289030 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc472d9d700 1 --2- 172.21.15.45:0/3677289030 >> v2:172.21.15.144:3300/0 conn(0x7fc4740c9dd0 0x7fc4740ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663742360 0 0) 0x7fc4740c9940 con 0x7fc474140ab0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc460003570 con 0x7fc474140ab0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031702182 0 0) 0x7fc4740ca750 con 0x7fc4741406e0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4740c9940 con 0x7fc4741406e0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc472d9d700 1 -- 172.21.15.45:0/3677289030 >> v1:172.21.15.144:6789/0 conn(0x7fc4741406e0 legacy=0x7fc47406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc472d9d700 1 -- 172.21.15.45:0/3677289030 >> v1:172.21.15.45:6789/0 conn(0x7fc474140ab0 legacy=0x7fc4740acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3633947547 0 0) 0x7fc460003570 con 0x7fc474140ab0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc472d9d700 1 -- 172.21.15.45:0/3677289030 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47406d510 con 0x7fc4740c9dd0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc472d9d700 1 --2- 172.21.15.45:0/3677289030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4740c9dd0 0x7fc4740ca210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009e60 tx=0x7fc46c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8c1b847c989803 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00e070 con 0x7fc4740c9dd0 2023-10-06T05:22:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c00b7a0 con 0x7fc4740c9dd0 2023-10-06T05:22:24.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.902+0000 7fc47159a700 1 -- 172.21.15.45:0/3677289030 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c001470 con 0x7fc4740c9dd0 2023-10-06T05:22:24.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.903+0000 7fc4788c6700 1 -- 172.21.15.45:0/3677289030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4740c9dd0 msgr2=0x7fc4740ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.903+0000 7fc4788c6700 1 --2- 172.21.15.45:0/3677289030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4740c9dd0 0x7fc4740ca210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009e60 tx=0x7fc46c009780 comp rx=0 tx=0).stop 2023-10-06T05:22:24.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.903+0000 7fc4788c6700 1 -- 172.21.15.45:0/3677289030 shutdown_connections 2023-10-06T05:22:24.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.903+0000 7fc4788c6700 1 --2- 172.21.15.45:0/3677289030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4740c9dd0 0x7fc4740ca210 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.903+0000 7fc4788c6700 1 -- 172.21.15.45:0/3677289030 >> 172.21.15.45:0/3677289030 conn(0x7fc4740691c0 msgr2=0x7fc4740cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.903+0000 7fc4788c6700 1 -- 172.21.15.45:0/3677289030 shutdown_connections 2023-10-06T05:22:24.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.903+0000 7fc4788c6700 1 -- 172.21.15.45:0/3677289030 wait complete. 2023-10-06T05:22:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.904+0000 7fc4788c6700 1 Processor -- start 2023-10-06T05:22:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.904+0000 7fc4788c6700 1 -- start start 2023-10-06T05:22:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc4788c6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4740c9dd0 0x7fc4740c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc4788c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4741406e0 0x7fc474156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc4788c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc474140ab0 0x7fc474156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc4788c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47415c8a0 con 0x7fc474140ab0 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc4788c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47415bc20 con 0x7fc4741406e0 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc4788c6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47415bf00 con 0x7fc4740c9dd0 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc472d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc474140ab0 0x7fc474156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc47259c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4740c9dd0 0x7fc4740c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc472d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc474140ab0 0x7fc474156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56708/0 (socket says 172.21.15.45:56708) 2023-10-06T05:22:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc471d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4741406e0 0x7fc474156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.905+0000 7fc472d9d700 1 -- 172.21.15.45:0/2780969683 learned_addr learned my addr 172.21.15.45:0/2780969683 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc472d9d700 1 -- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4740c9dd0 msgr2=0x7fc4740c2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc472d9d700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4740c9dd0 0x7fc4740c2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc472d9d700 1 -- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4741406e0 msgr2=0x7fc474156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc472d9d700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4741406e0 0x7fc474156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc472d9d700 1 -- 172.21.15.45:0/2780969683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c009a20 con 0x7fc474140ab0 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc47259c700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4740c9dd0 0x7fc4740c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc472d9d700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc474140ab0 0x7fc474156b90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fc47406c4b0 tx=0x7fc46c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc46a7fc700 1 -- 172.21.15.45:0/2780969683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c012c40 con 0x7fc474140ab0 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc46a7fc700 1 -- 172.21.15.45:0/2780969683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c012da0 con 0x7fc474140ab0 2023-10-06T05:22:24.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47415c1e0 con 0x7fc474140ab0 2023-10-06T05:22:24.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc46a7fc700 1 -- 172.21.15.45:0/2780969683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c01e680 con 0x7fc474140ab0 2023-10-06T05:22:24.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.906+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4740c3340 con 0x7fc474140ab0 2023-10-06T05:22:24.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.907+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4740c3530 con 0x7fc474140ab0 2023-10-06T05:22:24.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.908+0000 7fc46a7fc700 1 -- 172.21.15.45:0/2780969683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc46c01e9f0 con 0x7fc474140ab0 2023-10-06T05:22:24.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.908+0000 7fc46a7fc700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc454066f30 0x7fc4540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.908+0000 7fc46a7fc700 1 -- 172.21.15.45:0/2780969683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fc46c0869f0 con 0x7fc474140ab0 2023-10-06T05:22:24.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.908+0000 7fc47259c700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc454066f30 0x7fc4540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.909+0000 7fc47259c700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc454066f30 0x7fc4540693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc464007fd0 tx=0x7fc464007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:24.913+0000 7fc46a7fc700 1 -- 172.21.15.45:0/2780969683 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc46c053910 con 0x7fc474140ab0 2023-10-06T05:22:25.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.165+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fc47404bb40 con 0x7fc474140ab0 2023-10-06T05:22:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.368+0000 7fc46a7fc700 1 -- 172.21.15.45:0/2780969683 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v98) v1 ==== 162+0+0 (secure 0 0 0) 0x7fc46c056f30 con 0x7fc474140ab0 2023-10-06T05:22:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc454066f30 msgr2=0x7fc4540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc454066f30 0x7fc4540693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc464007fd0 tx=0x7fc464007d50 comp rx=0 tx=0).stop 2023-10-06T05:22:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc474140ab0 msgr2=0x7fc474156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc474140ab0 0x7fc474156b90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fc47406c4b0 tx=0x7fc46c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:25.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 shutdown_connections 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc454066f30 0x7fc4540693e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc474140ab0 0x7fc474156b90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4740c9dd0 0x7fc4740c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 --2- 172.21.15.45:0/2780969683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4741406e0 0x7fc474156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 >> 172.21.15.45:0/2780969683 conn(0x7fc4740691c0 msgr2=0x7fc4740ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 shutdown_connections 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.372+0000 7fc4788c6700 1 -- 172.21.15.45:0/2780969683 wait complete. 2023-10-06T05:22:25.375 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:22:25.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:25.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.525+0000 7fdfa60ee700 1 Processor -- start 2023-10-06T05:22:25.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.525+0000 7fdfa60ee700 1 -- start start 2023-10-06T05:22:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.525+0000 7fdfa60ee700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfa00cd340 0x7fdfa0157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.525+0000 7fdfa60ee700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa015b000 con 0x7fdfa00c8ff0 2023-10-06T05:22:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.525+0000 7fdfa60ee700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa015b2e0 con 0x7fdfa00c93c0 2023-10-06T05:22:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.525+0000 7fdfa60ee700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00c9830 con 0x7fdfa00cd340 2023-10-06T05:22:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfa00cd340 0x7fdfa0157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdfa00c8ff0 0x7fdfa00caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44250/0 (socket says 172.21.15.45:44250) 2023-10-06T05:22:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9f7fe700 1 -- 172.21.15.45:0/839217513 learned_addr learned my addr 172.21.15.45:0/839217513 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfa00cd340 0x7fdfa0157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56710/0 (socket says 172.21.15.45:56710) 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9ffff700 1 -- 172.21.15.45:0/839217513 >> v1:172.21.15.144:6789/0 conn(0x7fdfa00c93c0 legacy=0x7fdfa0156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9ffff700 1 -- 172.21.15.45:0/839217513 >> v1:172.21.15.45:6789/0 conn(0x7fdfa00c8ff0 legacy=0x7fdfa00caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9ffff700 1 -- 172.21.15.45:0/839217513 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa015b5c0 con 0x7fdfa00cd340 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9ffff700 1 --2- 172.21.15.45:0/839217513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa0157200 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009e60 tx=0x7fdf98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab67d7f9af9240e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9e7fc700 1 -- 172.21.15.45:0/839217513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800e070 con 0x7fdfa00cd340 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9e7fc700 1 -- 172.21.15.45:0/839217513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9800bb30 con 0x7fdfa00cd340 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.526+0000 7fdf9e7fc700 1 -- 172.21.15.45:0/839217513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98012430 con 0x7fdfa00cd340 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.527+0000 7fdfa60ee700 1 -- 172.21.15.45:0/839217513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 msgr2=0x7fdfa0157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.527+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/839217513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa0157200 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009e60 tx=0x7fdf98009780 comp rx=0 tx=0).stop 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.527+0000 7fdfa60ee700 1 -- 172.21.15.45:0/839217513 shutdown_connections 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.527+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/839217513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa0157200 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.527+0000 7fdfa60ee700 1 -- 172.21.15.45:0/839217513 >> 172.21.15.45:0/839217513 conn(0x7fdfa00cdf50 msgr2=0x7fdfa00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:25.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.527+0000 7fdfa60ee700 1 -- 172.21.15.45:0/839217513 shutdown_connections 2023-10-06T05:22:25.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.527+0000 7fdfa60ee700 1 -- 172.21.15.45:0/839217513 wait complete. 2023-10-06T05:22:25.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.529+0000 7fdfa60ee700 1 Processor -- start 2023-10-06T05:22:25.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.529+0000 7fdfa60ee700 1 -- start start 2023-10-06T05:22:25.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdfa60ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfa00c8ff0 0x7fdfa0148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdfa60ee700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfa00c93c0 0x7fdfa00c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdfa60ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa00c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdfa60ee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0149130 con 0x7fdfa00cd340 2023-10-06T05:22:25.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdfa60ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00b2240 con 0x7fdfa00c8ff0 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdfa60ee700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00c6510 con 0x7fdfa00c93c0 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa00c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa00c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56722/0 (socket says 172.21.15.45:56722) 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 -- 172.21.15.45:0/3797712202 learned_addr learned my addr 172.21.15.45:0/3797712202 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9f7fe700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfa00c8ff0 0x7fdfa0148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9effd700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfa00c93c0 0x7fdfa00c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 -- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfa00c93c0 msgr2=0x7fdfa00c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfa00c93c0 0x7fdfa00c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 -- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfa00c8ff0 msgr2=0x7fdfa0148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfa00c8ff0 0x7fdfa0148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9ffff700 1 -- 172.21.15.45:0/3797712202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98009a20 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9f7fe700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfa00c8ff0 0x7fdfa0148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.530+0000 7fdf9effd700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfa00c93c0 0x7fdfa00c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.531+0000 7fdf9ffff700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa00c40c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00c4f80 tx=0x7fdf98018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.531+0000 7fdf877fe700 1 -- 172.21.15.45:0/3797712202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9801e640 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.531+0000 7fdf877fe700 1 -- 172.21.15.45:0/3797712202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9801e7a0 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.531+0000 7fdf877fe700 1 -- 172.21.15.45:0/3797712202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9801e950 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.531+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa00c67c0 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.531+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa00c6d80 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.532+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa006f410 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.532+0000 7fdf877fe700 1 -- 172.21.15.45:0/3797712202 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf98024020 con 0x7fdfa00cd340 2023-10-06T05:22:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.533+0000 7fdf877fe700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf8c066f30 0x7fdf8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.533+0000 7fdf877fe700 1 -- 172.21.15.45:0/3797712202 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fdf98086cc0 con 0x7fdfa00cd340 2023-10-06T05:22:25.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.533+0000 7fdf9f7fe700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf8c066f30 0x7fdf8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.534+0000 7fdf9f7fe700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf8c066f30 0x7fdf8c0693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf900058d0 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:25.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.537+0000 7fdf877fe700 1 -- 172.21.15.45:0/3797712202 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf98053be0 con 0x7fdfa00cd340 2023-10-06T05:22:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.783+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfa004bb40 con 0x7fdfa00cd340 2023-10-06T05:22:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.784+0000 7fdf877fe700 1 -- 172.21.15.45:0/3797712202 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fdf9802c020 con 0x7fdfa00cd340 2023-10-06T05:22:25.786 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:25.786 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:25.369595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7866":{"gid":7866,"name":"a","rank":0,"incarnation":99,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6833/3755910187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3755910187},{"type":"v1","addr":"172.21.15.45:6833","nonce":3755910187}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf8c066f30 msgr2=0x7fdf8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf8c066f30 0x7fdf8c0693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdf900058d0 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 msgr2=0x7fdfa00c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa00c40c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00c4f80 tx=0x7fdf98018000 comp rx=0 tx=0).stop 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 shutdown_connections 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdf8c066f30 0x7fdf8c0693e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfa00cd340 0x7fdfa00c40c0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfa00c8ff0 0x7fdfa0148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 --2- 172.21.15.45:0/3797712202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfa00c93c0 0x7fdfa00c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 >> 172.21.15.45:0/3797712202 conn(0x7fdfa00cdf50 msgr2=0x7fdfa015bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 shutdown_connections 2023-10-06T05:22:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:25.786+0000 7fdfa60ee700 1 -- 172.21.15.45:0/3797712202 wait complete. 2023-10-06T05:22:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 99 2023-10-06T05:22:25.801 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:22:28.385 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:22:28.383+0000 7f9f9a2a7700 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2023-10-06T05:22:28.385 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:22:28.383+0000 7f9f9a2a7700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-10-06T05:22:28.385 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:28.384+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:28.385 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:28.384+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:22:28.391 INFO:tasks.ceph.mds.a.smithi045.stderr: -15> 2023-10-06T05:22:28.383+0000 7f9f9a2a7700 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2023-10-06T05:22:28.391 INFO:tasks.ceph.mds.a.smithi045.stderr: -13> 2023-10-06T05:22:28.383+0000 7f9f9a2a7700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-10-06T05:22:28.414 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:22:30.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:30.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.945+0000 7f5f75ba5700 1 Processor -- start 2023-10-06T05:22:30.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.945+0000 7f5f75ba5700 1 -- start start 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f75ba5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5f701539f0 0x7f5f7015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f75ba5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700ccfc0 0x7f5f7015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f75ba5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700ce690 con 0x7f5f70153620 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f75ba5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7015a7a0 con 0x7f5f701539f0 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f75ba5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700ce3f0 con 0x7f5f700ccfc0 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f6f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5f70153620 0x7f5f700cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39450/0 (socket says 172.21.15.45:39450) 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f6f7fe700 1 -- 172.21.15.45:0/393538549 learned_addr learned my addr 172.21.15.45:0/393538549 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f6effd700 1 --2- 172.21.15.45:0/393538549 >> v2:172.21.15.144:3300/0 conn(0x7f5f701539f0 0x7f5f7015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f6ffff700 1 --2- 172.21.15.45:0/393538549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700ccfc0 0x7f5f7015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.946+0000 7f5f6e7fc700 1 -- 172.21.15.45:0/393538549 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505367685 0 0) 0x7f5f700ce690 con 0x7f5f70153620 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6e7fc700 1 -- 172.21.15.45:0/393538549 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f58003570 con 0x7f5f70153620 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6ffff700 1 -- 172.21.15.45:0/393538549 >> v2:172.21.15.144:3300/0 conn(0x7f5f701539f0 msgr2=0x7f5f7015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6ffff700 1 --2- 172.21.15.45:0/393538549 >> v2:172.21.15.144:3300/0 conn(0x7f5f701539f0 0x7f5f7015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6ffff700 1 -- 172.21.15.45:0/393538549 >> v1:172.21.15.45:6789/0 conn(0x7f5f70153620 legacy=0x7f5f700cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6ffff700 1 -- 172.21.15.45:0/393538549 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7015d510 con 0x7f5f700ccfc0 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6effd700 1 --2- 172.21.15.45:0/393538549 >> v2:172.21.15.144:3300/0 conn(0x7f5f701539f0 0x7f5f7015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6ffff700 1 --2- 172.21.15.45:0/393538549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700ccfc0 0x7f5f7015cdc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009e60 tx=0x7f5f68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f9115cf5e1f82d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6e7fc700 1 -- 172.21.15.45:0/393538549 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800e070 con 0x7f5f700ccfc0 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6e7fc700 1 -- 172.21.15.45:0/393538549 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800bab0 con 0x7f5f700ccfc0 2023-10-06T05:22:30.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.947+0000 7f5f6e7fc700 1 -- 172.21.15.45:0/393538549 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800be30 con 0x7f5f700ccfc0 2023-10-06T05:22:30.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 -- 172.21.15.45:0/393538549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700ccfc0 msgr2=0x7f5f7015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/393538549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700ccfc0 0x7f5f7015cdc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009e60 tx=0x7f5f68009780 comp rx=0 tx=0).stop 2023-10-06T05:22:30.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 -- 172.21.15.45:0/393538549 shutdown_connections 2023-10-06T05:22:30.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/393538549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700ccfc0 0x7f5f7015cdc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/393538549 >> v2:172.21.15.144:3300/0 conn(0x7f5f701539f0 0x7f5f7015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 -- 172.21.15.45:0/393538549 >> 172.21.15.45:0/393538549 conn(0x7f5f7006ed50 msgr2=0x7f5f7006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:30.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 -- 172.21.15.45:0/393538549 shutdown_connections 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.948+0000 7f5f75ba5700 1 -- 172.21.15.45:0/393538549 wait complete. 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.949+0000 7f5f75ba5700 1 Processor -- start 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.949+0000 7f5f75ba5700 1 -- start start 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f75ba5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f700ccfc0 0x7f5f70150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f75ba5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f70153620 0x7f5f700b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f75ba5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700b3c40 0x7f5f70151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f75ba5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701528b0 con 0x7f5f700ccfc0 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f75ba5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b40e0 con 0x7f5f70153620 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f75ba5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b0aa0 con 0x7f5f700b3c40 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f700ccfc0 0x7f5f70150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f700ccfc0 0x7f5f70150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53952/0 (socket says 172.21.15.45:53952) 2023-10-06T05:22:30.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 -- 172.21.15.45:0/3392334698 learned_addr learned my addr 172.21.15.45:0/3392334698 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6effd700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f70153620 0x7f5f700b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6ffff700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700b3c40 0x7f5f70151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 -- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700b3c40 msgr2=0x7f5f70151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700b3c40 0x7f5f70151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 -- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f70153620 msgr2=0x7f5f700b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f70153620 0x7f5f700b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.950+0000 7f5f6f7fe700 1 -- 172.21.15.45:0/3392334698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009420 con 0x7f5f700ccfc0 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.951+0000 7f5f6ffff700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700b3c40 0x7f5f70151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.951+0000 7f5f6f7fe700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f700ccfc0 0x7f5f70150600 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5f600011e0 tx=0x7f5f60007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.951+0000 7f5f577fe700 1 -- 172.21.15.45:0/3392334698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60005b30 con 0x7f5f700ccfc0 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.951+0000 7f5f577fe700 1 -- 172.21.15.45:0/3392334698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60005c90 con 0x7f5f700ccfc0 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.951+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f68009a20 con 0x7f5f700ccfc0 2023-10-06T05:22:30.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.951+0000 7f5f577fe700 1 -- 172.21.15.45:0/3392334698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60007320 con 0x7f5f700ccfc0 2023-10-06T05:22:30.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.951+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f701525d0 con 0x7f5f700ccfc0 2023-10-06T05:22:30.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.952+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f70088a60 con 0x7f5f700ccfc0 2023-10-06T05:22:30.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.952+0000 7f5f577fe700 1 -- 172.21.15.45:0/3392334698 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f60007690 con 0x7f5f700ccfc0 2023-10-06T05:22:30.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.953+0000 7f5f577fe700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f5c066f30 0x7f5f5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.953+0000 7f5f577fe700 1 -- 172.21.15.45:0/3392334698 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f5f60083d60 con 0x7f5f700ccfc0 2023-10-06T05:22:30.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.953+0000 7f5f6effd700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f5c066f30 0x7f5f5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.954+0000 7f5f6effd700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f5c066f30 0x7f5f5c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5f701572b0 tx=0x7f5f640091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:30.957+0000 7f5f577fe700 1 -- 172.21.15.45:0/3392334698 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f60050c30 con 0x7f5f700ccfc0 2023-10-06T05:22:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.206+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f7004bb40 con 0x7f5f700ccfc0 2023-10-06T05:22:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.207+0000 7f5f577fe700 1 -- 172.21.15.45:0/3392334698 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f5f60017020 con 0x7f5f700ccfc0 2023-10-06T05:22:31.209 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:31.209 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7920,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/4128377575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4128377575},{"type":"v1","addr":"172.21.15.45:6833","nonce":4128377575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:28.384914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f5c066f30 msgr2=0x7f5f5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f5c066f30 0x7f5f5c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5f701572b0 tx=0x7f5f640091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f700ccfc0 msgr2=0x7f5f70150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f700ccfc0 0x7f5f70150600 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5f600011e0 tx=0x7f5f60007df0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 shutdown_connections 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f5c066f30 0x7f5f5c0693e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f700b3c40 0x7f5f70151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f700ccfc0 0x7f5f70150600 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.209+0000 7f5f75ba5700 1 --2- 172.21.15.45:0/3392334698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f70153620 0x7f5f700b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.210+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 >> 172.21.15.45:0/3392334698 conn(0x7f5f7006ed50 msgr2=0x7f5f7006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.210+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 shutdown_connections 2023-10-06T05:22:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.210+0000 7f5f75ba5700 1 -- 172.21.15.45:0/3392334698 wait complete. 2023-10-06T05:22:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 104 2023-10-06T05:22:31.225 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:22:31.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:31.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.355+0000 7f75f5a43700 1 Processor -- start 2023-10-06T05:22:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.356+0000 7f75f5a43700 1 -- start start 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.356+0000 7f75f5a43700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00ac200 0x7f75f0159340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.356+0000 7f75f5a43700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f75f00cb5e0 0x7f75f00cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.356+0000 7f75f5a43700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f015db00 con 0x7f75f0158f70 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75f5a43700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f015cef0 con 0x7f75f00cb5e0 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75f5a43700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f015d1d0 con 0x7f75f00ac200 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75eeffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f75f0158f70 0x7f75f00abaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39456/0 (socket says 172.21.15.45:39456) 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75eeffd700 1 -- 172.21.15.45:0/853817729 learned_addr learned my addr 172.21.15.45:0/853817729 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75ef7fe700 1 --2- 172.21.15.45:0/853817729 >> v2:172.21.15.45:3300/0 conn(0x7f75f00cb5e0 0x7f75f00cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75ee7fc700 1 --2- 172.21.15.45:0/853817729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00ac200 0x7f75f0159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75edffb700 1 -- 172.21.15.45:0/853817729 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311111231 0 0) 0x7f75f015db00 con 0x7f75f0158f70 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75edffb700 1 -- 172.21.15.45:0/853817729 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d8003570 con 0x7f75f0158f70 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75ef7fe700 1 -- 172.21.15.45:0/853817729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00ac200 msgr2=0x7f75f0159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75ef7fe700 1 --2- 172.21.15.45:0/853817729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00ac200 0x7f75f0159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75ef7fe700 1 -- 172.21.15.45:0/853817729 >> v1:172.21.15.45:6789/0 conn(0x7f75f0158f70 legacy=0x7f75f00abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75ef7fe700 1 -- 172.21.15.45:0/853817729 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f015d4b0 con 0x7f75f00cb5e0 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75ef7fe700 1 --2- 172.21.15.45:0/853817729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f00cb5e0 0x7f75f00cba60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009e60 tx=0x7f75e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fee63a257e6af88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75edffb700 1 -- 172.21.15.45:0/853817729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800e070 con 0x7f75f00cb5e0 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75edffb700 1 -- 172.21.15.45:0/853817729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e800bab0 con 0x7f75f00cb5e0 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.357+0000 7f75edffb700 1 -- 172.21.15.45:0/853817729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800be30 con 0x7f75f00cb5e0 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 -- 172.21.15.45:0/853817729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f00cb5e0 msgr2=0x7f75f00cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 --2- 172.21.15.45:0/853817729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f00cb5e0 0x7f75f00cba60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009e60 tx=0x7f75e8009780 comp rx=0 tx=0).stop 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 -- 172.21.15.45:0/853817729 shutdown_connections 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 --2- 172.21.15.45:0/853817729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f00cb5e0 0x7f75f00cba60 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 --2- 172.21.15.45:0/853817729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00ac200 0x7f75f0159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 -- 172.21.15.45:0/853817729 >> 172.21.15.45:0/853817729 conn(0x7f75f0069200 msgr2=0x7f75f00cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 -- 172.21.15.45:0/853817729 shutdown_connections 2023-10-06T05:22:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.358+0000 7f75f5a43700 1 -- 172.21.15.45:0/853817729 wait complete. 2023-10-06T05:22:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.359+0000 7f75f5a43700 1 Processor -- start 2023-10-06T05:22:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75f5a43700 1 -- start start 2023-10-06T05:22:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75f5a43700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75f00ac200 0x7f75f01489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75f5a43700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00cb5e0 0x7f75f016cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75f5a43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f0158f70 0x7f75f016d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75f5a43700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00adbb0 con 0x7f75f0158f70 2023-10-06T05:22:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75f5a43700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00ad280 con 0x7f75f00ac200 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75f5a43700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00ad560 con 0x7f75f00cb5e0 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f0158f70 0x7f75f016d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f0158f70 0x7f75f016d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53960/0 (socket says 172.21.15.45:53960) 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75ef7fe700 1 -- 172.21.15.45:0/2040239610 learned_addr learned my addr 172.21.15.45:0/2040239610 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.360+0000 7f75eeffd700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75f00ac200 0x7f75f01489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75ee7fc700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00cb5e0 0x7f75f016cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75ef7fe700 1 -- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00cb5e0 msgr2=0x7f75f016cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75ef7fe700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00cb5e0 0x7f75f016cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75ef7fe700 1 -- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75f00ac200 msgr2=0x7f75f01489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75ef7fe700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75f00ac200 0x7f75f01489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75ef7fe700 1 -- 172.21.15.45:0/2040239610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009a20 con 0x7f75f0158f70 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75ef7fe700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f0158f70 0x7f75f016d470 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f75f006ae90 tx=0x7f75e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75d6ffd700 1 -- 172.21.15.45:0/2040239610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8012e50 con 0x7f75f0158f70 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75d6ffd700 1 -- 172.21.15.45:0/2040239610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e8027470 con 0x7f75f0158f70 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75d6ffd700 1 -- 172.21.15.45:0/2040239610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8027830 con 0x7f75f0158f70 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f00ad840 con 0x7f75f0158f70 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.361+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f0148ee0 con 0x7f75f0158f70 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.362+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f00add60 con 0x7f75f0158f70 2023-10-06T05:22:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.363+0000 7f75d6ffd700 1 -- 172.21.15.45:0/2040239610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75e8024070 con 0x7f75f0158f70 2023-10-06T05:22:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.363+0000 7f75d6ffd700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f75dc066f30 0x7f75dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.363+0000 7f75d6ffd700 1 -- 172.21.15.45:0/2040239610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f75e8086fa0 con 0x7f75f0158f70 2023-10-06T05:22:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.363+0000 7f75eeffd700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f75dc066f30 0x7f75dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.364+0000 7f75eeffd700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f75dc066f30 0x7f75dc0693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75e0007fd0 tx=0x7f75e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.368+0000 7f75d6ffd700 1 -- 172.21.15.45:0/2040239610 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75e80b85a0 con 0x7f75f0158f70 2023-10-06T05:22:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.624+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75f015d790 con 0x7f75f0158f70 2023-10-06T05:22:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.624+0000 7f75d6ffd700 1 -- 172.21.15.45:0/2040239610 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f75e802c090 con 0x7f75f0158f70 2023-10-06T05:22:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:31.626 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7920,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/4128377575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4128377575},{"type":"v1","addr":"172.21.15.45:6833","nonce":4128377575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:28.384914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:31.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f75dc066f30 msgr2=0x7f75dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f75dc066f30 0x7f75dc0693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75e0007fd0 tx=0x7f75e0009040 comp rx=0 tx=0).stop 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f0158f70 msgr2=0x7f75f016d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f0158f70 0x7f75f016d470 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f75f006ae90 tx=0x7f75e8019000 comp rx=0 tx=0).stop 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 shutdown_connections 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f75dc066f30 0x7f75dc0693e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75f0158f70 0x7f75f016d470 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75f00ac200 0x7f75f01489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 --2- 172.21.15.45:0/2040239610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75f00cb5e0 0x7f75f016cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 >> 172.21.15.45:0/2040239610 conn(0x7f75f0069200 msgr2=0x7f75f00c2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 shutdown_connections 2023-10-06T05:22:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.626+0000 7f75f5a43700 1 -- 172.21.15.45:0/2040239610 wait complete. 2023-10-06T05:22:31.630 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 104 2023-10-06T05:22:31.642 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:22:31.643 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:31.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.773+0000 7fcc031b9700 1 Processor -- start 2023-10-06T05:22:31.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.773+0000 7fcc031b9700 1 -- start start 2023-10-06T05:22:31.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc031b9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcbfc15a490 0x7fcbfc067830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc031b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc031b9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbfc06add0 0x7fcbfc06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc031b9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbfc0ae020 con 0x7fcbfc06add0 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc031b9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbfc06b250 con 0x7fcbfc15a490 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc031b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0b2120 con 0x7fcbfc15b5e0 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbfc06add0 0x7fcbfc06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbfc06add0 0x7fcbfc06e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53962/0 (socket says 172.21.15.45:53962) 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 -- 172.21.15.45:0/1790582219 learned_addr learned my addr 172.21.15.45:0/1790582219 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcbfbfff700 1 --2- 172.21.15.45:0/1790582219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc00f55700 1 --2- 172.21.15.45:0/1790582219 >> v2:172.21.15.144:3300/0 conn(0x7fcbfc15a490 0x7fcbfc067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 -- 172.21.15.45:0/1790582219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 msgr2=0x7fcbfc06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 --2- 172.21.15.45:0/1790582219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc06a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 -- 172.21.15.45:0/1790582219 >> v2:172.21.15.144:3300/0 conn(0x7fcbfc15a490 msgr2=0x7fcbfc067830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 --2- 172.21.15.45:0/1790582219 >> v2:172.21.15.144:3300/0 conn(0x7fcbfc15a490 0x7fcbfc067830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 -- 172.21.15.45:0/1790582219 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0ab7d0 con 0x7fcbfc06add0 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.775+0000 7fcc01756700 1 --2- 172.21.15.45:0/1790582219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc06add0 0x7fcbfc06e6e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0cd3d0 tx=0x7fcbf4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23cf3959647a3ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcbfb7fe700 1 -- 172.21.15.45:0/1790582219 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400e070 con 0x7fcbfc06add0 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcbfb7fe700 1 -- 172.21.15.45:0/1790582219 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf400b970 con 0x7fcbfc06add0 2023-10-06T05:22:31.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcbfb7fe700 1 -- 172.21.15.45:0/1790582219 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40092e0 con 0x7fcbfc06add0 2023-10-06T05:22:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 -- 172.21.15.45:0/1790582219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc06add0 msgr2=0x7fcbfc06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 --2- 172.21.15.45:0/1790582219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc06add0 0x7fcbfc06e6e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0cd3d0 tx=0x7fcbf4009840 comp rx=0 tx=0).stop 2023-10-06T05:22:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 -- 172.21.15.45:0/1790582219 shutdown_connections 2023-10-06T05:22:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 --2- 172.21.15.45:0/1790582219 >> v2:172.21.15.144:3300/0 conn(0x7fcbfc15a490 0x7fcbfc067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 --2- 172.21.15.45:0/1790582219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc06add0 0x7fcbfc06e6e0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 --2- 172.21.15.45:0/1790582219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc06a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 -- 172.21.15.45:0/1790582219 >> 172.21.15.45:0/1790582219 conn(0x7fcbfc0cc5a0 msgr2=0x7fcbfc0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 -- 172.21.15.45:0/1790582219 shutdown_connections 2023-10-06T05:22:31.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.776+0000 7fcc031b9700 1 -- 172.21.15.45:0/1790582219 wait complete. 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.777+0000 7fcc031b9700 1 Processor -- start 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.778+0000 7fcc031b9700 1 -- start start 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.778+0000 7fcc031b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbfc06add0 0x7fcbfc0a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc031b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc15a490 0x7fcbfc0c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc031b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc031b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0a9690 con 0x7fcbfc15a490 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc031b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0a8a80 con 0x7fcbfc06add0 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc031b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0a8d60 con 0x7fcbfc15b5e0 2023-10-06T05:22:31.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc15a490 0x7fcbfc0c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc15a490 0x7fcbfc0c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53970/0 (socket says 172.21.15.45:53970) 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 -- 172.21.15.45:0/4231746697 learned_addr learned my addr 172.21.15.45:0/4231746697 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc00f55700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbfc06add0 0x7fcbfc0a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc01756700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 -- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 msgr2=0x7fcbfc0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc0cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 -- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbfc06add0 msgr2=0x7fcbfc0a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbfc06add0 0x7fcbfc0a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 -- 172.21.15.45:0/4231746697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf4009ae0 con 0x7fcbfc15a490 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbfbfff700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc15a490 0x7fcbfc0c4f40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000ec80 tx=0x7fcbf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbf8ff9700 1 -- 172.21.15.45:0/4231746697 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000c9d0 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbf8ff9700 1 -- 172.21.15.45:0/4231746697 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf000cb30 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcbf8ff9700 1 -- 172.21.15.45:0/4231746697 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0013470 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc0a9040 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.779+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc0c5480 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.781+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc0a9840 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.781+0000 7fcbf8ff9700 1 -- 172.21.15.45:0/4231746697 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbf0013820 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.781+0000 7fcbf8ff9700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbe8066f30 0x7fcbe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.782+0000 7fcbf8ff9700 1 -- 172.21.15.45:0/4231746697 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fcbf0084c90 con 0x7fcbfc15a490 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.782+0000 7fcc00f55700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbe8066f30 0x7fcbe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.782+0000 7fcc00f55700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbe8066f30 0x7fcbe80693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009d40 tx=0x7fcbec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:31.786+0000 7fcbf8ff9700 1 -- 172.21.15.45:0/4231746697 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbfc0a9840 con 0x7fcbfc15a490 2023-10-06T05:22:32.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.040+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbfc0cc0e0 con 0x7fcbfc15a490 2023-10-06T05:22:32.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.040+0000 7fcbf8ff9700 1 -- 172.21.15.45:0/4231746697 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fcbf0051b60 con 0x7fcbfc15a490 2023-10-06T05:22:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:32.042 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7920,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/4128377575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4128377575},{"type":"v1","addr":"172.21.15.45:6833","nonce":4128377575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:28.384914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbe8066f30 msgr2=0x7fcbe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbe8066f30 0x7fcbe80693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009d40 tx=0x7fcbec008040 comp rx=0 tx=0).stop 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc15a490 msgr2=0x7fcbfc0c4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc15a490 0x7fcbfc0c4f40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000ec80 tx=0x7fcbf00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 shutdown_connections 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbe8066f30 0x7fcbe80693e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbfc15b5e0 0x7fcbfc0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbfc06add0 0x7fcbfc0a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 --2- 172.21.15.45:0/4231746697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbfc15a490 0x7fcbfc0c4f40 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 >> 172.21.15.45:0/4231746697 conn(0x7fcbfc0cc5a0 msgr2=0x7fcbfc0cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 shutdown_connections 2023-10-06T05:22:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.043+0000 7fcc031b9700 1 -- 172.21.15.45:0/4231746697 wait complete. 2023-10-06T05:22:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 104 2023-10-06T05:22:32.058 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2023-10-06T05:22:32.058 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2023-10-06T05:22:32.058 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:32.058 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:32.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:32.059 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:32.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:22:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:22:32.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.252+0000 7fdc17cdb700 1 Processor -- start 2023-10-06T05:22:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.253+0000 7fdc17cdb700 1 -- start start 2023-10-06T05:22:32.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc17cdb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc100c8ff0 0x7fdc100c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc17cdb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc10157200 0x7fdc100caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc17cdb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1015b000 con 0x7fdc100caad0 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc17cdb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1015b2e0 con 0x7fdc10157200 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc17cdb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1015b5c0 con 0x7fdc100c8ff0 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc16278700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc10157200 0x7fdc100caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc15276700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdc100caad0 0x7fdc10156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39460/0 (socket says 172.21.15.45:39460) 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc15276700 1 -- 172.21.15.45:0/2430324532 learned_addr learned my addr 172.21.15.45:0/2430324532 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc15a77700 1 --2- 172.21.15.45:0/2430324532 >> v2:172.21.15.144:3300/0 conn(0x7fdc100c8ff0 0x7fdc100c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc16278700 1 -- 172.21.15.45:0/2430324532 >> v2:172.21.15.144:3300/0 conn(0x7fdc100c8ff0 msgr2=0x7fdc100c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc16278700 1 --2- 172.21.15.45:0/2430324532 >> v2:172.21.15.144:3300/0 conn(0x7fdc100c8ff0 0x7fdc100c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc16278700 1 -- 172.21.15.45:0/2430324532 >> v1:172.21.15.45:6789/0 conn(0x7fdc100caad0 legacy=0x7fdc10156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc16278700 1 -- 172.21.15.45:0/2430324532 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1015b8a0 con 0x7fdc10157200 2023-10-06T05:22:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.254+0000 7fdc14a75700 1 -- 172.21.15.45:0/2430324532 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883568376 0 0) 0x7fdc1015b000 con 0x7fdc100caad0 2023-10-06T05:22:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc16278700 1 --2- 172.21.15.45:0/2430324532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc10157200 0x7fdc100caf40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdc100cefc0 tx=0x7fdc08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fe756f91a86621 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc14a75700 1 -- 172.21.15.45:0/2430324532 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800e070 con 0x7fdc10157200 2023-10-06T05:22:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc14a75700 1 -- 172.21.15.45:0/2430324532 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0800bc30 con 0x7fdc10157200 2023-10-06T05:22:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc14a75700 1 -- 172.21.15.45:0/2430324532 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08012430 con 0x7fdc10157200 2023-10-06T05:22:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc17cdb700 1 -- 172.21.15.45:0/2430324532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc10157200 msgr2=0x7fdc100caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/2430324532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc10157200 0x7fdc100caf40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdc100cefc0 tx=0x7fdc08009780 comp rx=0 tx=0).stop 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc17cdb700 1 -- 172.21.15.45:0/2430324532 shutdown_connections 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/2430324532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc10157200 0x7fdc100caf40 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/2430324532 >> v2:172.21.15.144:3300/0 conn(0x7fdc100c8ff0 0x7fdc100c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.255+0000 7fdc17cdb700 1 -- 172.21.15.45:0/2430324532 >> 172.21.15.45:0/2430324532 conn(0x7fdc100cdf50 msgr2=0x7fdc100ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.256+0000 7fdc17cdb700 1 -- 172.21.15.45:0/2430324532 shutdown_connections 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.256+0000 7fdc17cdb700 1 -- 172.21.15.45:0/2430324532 wait complete. 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.257+0000 7fdc17cdb700 1 Processor -- start 2023-10-06T05:22:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.257+0000 7fdc17cdb700 1 -- start start 2023-10-06T05:22:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc17cdb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc100c8ff0 0x7fdc100afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc17cdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100caad0 0x7fdc10068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc17cdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc10157200 0x7fdc1014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc17cdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc10150840 con 0x7fdc100caad0 2023-10-06T05:22:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc17cdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100b0170 con 0x7fdc10157200 2023-10-06T05:22:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc17cdb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100c7cc0 con 0x7fdc100c8ff0 2023-10-06T05:22:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc15276700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100caad0 0x7fdc10068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc15276700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100caad0 0x7fdc10068730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53984/0 (socket says 172.21.15.45:53984) 2023-10-06T05:22:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc15276700 1 -- 172.21.15.45:0/3438101311 learned_addr learned my addr 172.21.15.45:0/3438101311 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc15a77700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc100c8ff0 0x7fdc100afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.258+0000 7fdc16278700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc10157200 0x7fdc1014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc15276700 1 -- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc100c8ff0 msgr2=0x7fdc100afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc15276700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc100c8ff0 0x7fdc100afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc15276700 1 -- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc10157200 msgr2=0x7fdc1014fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc15276700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc10157200 0x7fdc1014fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc15276700 1 -- 172.21.15.45:0/3438101311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08009a20 con 0x7fdc100caad0 2023-10-06T05:22:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc15276700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100caad0 0x7fdc10068730 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000c810 tx=0x7fdc00007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc05ffb700 1 -- 172.21.15.45:0/3438101311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00005b30 con 0x7fdc100caad0 2023-10-06T05:22:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc05ffb700 1 -- 172.21.15.45:0/3438101311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00005c90 con 0x7fdc100caad0 2023-10-06T05:22:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc100c5bf0 con 0x7fdc100caad0 2023-10-06T05:22:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc05ffb700 1 -- 172.21.15.45:0/3438101311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00007300 con 0x7fdc100caad0 2023-10-06T05:22:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.259+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc100c61b0 con 0x7fdc100caad0 2023-10-06T05:22:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.260+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1006f410 con 0x7fdc100caad0 2023-10-06T05:22:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.261+0000 7fdc05ffb700 1 -- 172.21.15.45:0/3438101311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc0001b3b0 con 0x7fdc100caad0 2023-10-06T05:22:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.261+0000 7fdc05ffb700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbf4066f30 0x7fdbf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.261+0000 7fdc05ffb700 1 -- 172.21.15.45:0/3438101311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fdc00083d80 con 0x7fdc100caad0 2023-10-06T05:22:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.261+0000 7fdc15a77700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbf4066f30 0x7fdbf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.262+0000 7fdc15a77700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbf4066f30 0x7fdbf40693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdc1015a6a0 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.266+0000 7fdc05ffb700 1 -- 172.21.15.45:0/3438101311 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc00050c50 con 0x7fdc100caad0 2023-10-06T05:22:32.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.516+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdc100c89a0 con 0x7fdc100caad0 2023-10-06T05:22:32.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.863+0000 7fdc05ffb700 1 -- 172.21.15.45:0/3438101311 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdc00017090 con 0x7fdc100caad0 2023-10-06T05:22:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbf4066f30 msgr2=0x7fdbf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbf4066f30 0x7fdbf40693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdc1015a6a0 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2023-10-06T05:22:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100caad0 msgr2=0x7fdc10068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100caad0 0x7fdc10068730 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000c810 tx=0x7fdc00007df0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 shutdown_connections 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbf4066f30 0x7fdbf40693e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc10157200 0x7fdc1014fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc100c8ff0 0x7fdc100afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 --2- 172.21.15.45:0/3438101311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100caad0 0x7fdc10068730 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 >> 172.21.15.45:0/3438101311 conn(0x7fdc100cdf50 msgr2=0x7fdc1006e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 shutdown_connections 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:32.866+0000 7fdc17cdb700 1 -- 172.21.15.45:0/3438101311 wait complete. 2023-10-06T05:22:32.869 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:32.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:22:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.018+0000 7f2ead8da700 1 Processor -- start 2023-10-06T05:22:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.018+0000 7f2ead8da700 1 -- start start 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ead8da700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea80692d0 0x7f2ea8159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ead8da700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ea815d040 0x7f2ea80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ead8da700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80ca6d0 con 0x7f2ea8159170 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ead8da700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea815d550 con 0x7f2ea815d040 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ead8da700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea815d810 con 0x7f2ea80692d0 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ea77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ea815d040 0x7f2ea80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ea77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ea815d040 0x7f2ea80aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53996/0 (socket says 172.21.15.45:53996) 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ea77fe700 1 -- 172.21.15.45:0/2056268011 learned_addr learned my addr 172.21.15.45:0/2056268011 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.019+0000 7f2ea67fc700 1 --2- 172.21.15.45:0/2056268011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea80692d0 0x7f2ea8159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea77fe700 1 -- 172.21.15.45:0/2056268011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea80692d0 msgr2=0x7f2ea8159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea77fe700 1 --2- 172.21.15.45:0/2056268011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea80692d0 0x7f2ea8159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea77fe700 1 -- 172.21.15.45:0/2056268011 >> v1:172.21.15.144:6789/0 conn(0x7f2ea8159170 legacy=0x7f2ea8068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea77fe700 1 -- 172.21.15.45:0/2056268011 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea815daf0 con 0x7f2ea815d040 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea67fc700 1 --2- 172.21.15.45:0/2056268011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea80692d0 0x7f2ea8159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea77fe700 1 --2- 172.21.15.45:0/2056268011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea815d040 0x7f2ea80aea60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009f20 tx=0x7f2ea0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7af020b46a5957e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea5ffb700 1 -- 172.21.15.45:0/2056268011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000e070 con 0x7f2ea815d040 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea5ffb700 1 -- 172.21.15.45:0/2056268011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea000ba80 con 0x7f2ea815d040 2023-10-06T05:22:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.020+0000 7f2ea5ffb700 1 -- 172.21.15.45:0/2056268011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00092e0 con 0x7f2ea815d040 2023-10-06T05:22:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 -- 172.21.15.45:0/2056268011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea815d040 msgr2=0x7f2ea80aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 --2- 172.21.15.45:0/2056268011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea815d040 0x7f2ea80aea60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009f20 tx=0x7f2ea0009840 comp rx=0 tx=0).stop 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 -- 172.21.15.45:0/2056268011 shutdown_connections 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 --2- 172.21.15.45:0/2056268011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea815d040 0x7f2ea80aea60 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 --2- 172.21.15.45:0/2056268011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea80692d0 0x7f2ea8159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 -- 172.21.15.45:0/2056268011 >> 172.21.15.45:0/2056268011 conn(0x7f2ea806f7c0 msgr2=0x7f2ea80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 -- 172.21.15.45:0/2056268011 shutdown_connections 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.021+0000 7f2ead8da700 1 -- 172.21.15.45:0/2056268011 wait complete. 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.022+0000 7f2ead8da700 1 Processor -- start 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.022+0000 7f2ead8da700 1 -- start start 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ead8da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea80692d0 0x7f2ea81575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ead8da700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea8159170 0x7f2ea80c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ead8da700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea815d040 0x7f2ea80cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ead8da700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80a6b80 con 0x7f2ea8159170 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ead8da700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80c7df0 con 0x7f2ea80692d0 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ead8da700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80c71e0 con 0x7f2ea815d040 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea8159170 0x7f2ea80c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea8159170 0x7f2ea80c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54002/0 (socket says 172.21.15.45:54002) 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 -- 172.21.15.45:0/583715302 learned_addr learned my addr 172.21.15.45:0/583715302 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea77fe700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea815d040 0x7f2ea80cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea6ffd700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea80692d0 0x7f2ea81575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 -- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea815d040 msgr2=0x7f2ea80cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea815d040 0x7f2ea80cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 -- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea80692d0 msgr2=0x7f2ea81575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea80692d0 0x7f2ea81575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.023+0000 7f2ea67fc700 1 -- 172.21.15.45:0/583715302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea0009ae0 con 0x7f2ea8159170 2023-10-06T05:22:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.024+0000 7f2ea67fc700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea8159170 0x7f2ea80c8e30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e8d0 tx=0x7f2e90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.024+0000 7f2e9effd700 1 -- 172.21.15.45:0/583715302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90013410 con 0x7f2ea8159170 2023-10-06T05:22:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.024+0000 7f2e9effd700 1 -- 172.21.15.45:0/583715302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90013570 con 0x7f2ea8159170 2023-10-06T05:22:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.024+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea80c74c0 con 0x7f2ea8159170 2023-10-06T05:22:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.024+0000 7f2e9effd700 1 -- 172.21.15.45:0/583715302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90013910 con 0x7f2ea8159170 2023-10-06T05:22:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.024+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea80c7b00 con 0x7f2ea8159170 2023-10-06T05:22:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.025+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea8157c30 con 0x7f2ea8159170 2023-10-06T05:22:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.025+0000 7f2e9effd700 1 -- 172.21.15.45:0/583715302 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e90013d00 con 0x7f2ea8159170 2023-10-06T05:22:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.026+0000 7f2e9effd700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.026+0000 7f2e9effd700 1 -- 172.21.15.45:0/583715302 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f2e90084660 con 0x7f2ea8159170 2023-10-06T05:22:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.026+0000 7f2ea6ffd700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.027+0000 7f2ea6ffd700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e88066f30 0x7f2e880693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80ccde0 tx=0x7f2e98009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.031+0000 7f2e9effd700 1 -- 172.21.15.45:0/583715302 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e90051530 con 0x7f2ea8159170 2023-10-06T05:22:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.319+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f2ea80c7f60 con 0x7f2ea8159170 2023-10-06T05:22:33.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.868+0000 7f2e9effd700 1 -- 172.21.15.45:0/583715302 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v106) v1 ==== 101+0+0 (secure 0 0 0) 0x7f2e90054b50 con 0x7f2ea8159170 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e88066f30 msgr2=0x7f2e880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e88066f30 0x7f2e880693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80ccde0 tx=0x7f2e98009040 comp rx=0 tx=0).stop 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea8159170 msgr2=0x7f2ea80c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea8159170 0x7f2ea80c8e30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e8d0 tx=0x7f2e90008040 comp rx=0 tx=0).stop 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 shutdown_connections 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ea815d040 0x7f2ea80cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ea80692d0 0x7f2ea81575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 --2- 172.21.15.45:0/583715302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea8159170 0x7f2ea80c8e30 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 >> 172.21.15.45:0/583715302 conn(0x7f2ea806f7c0 msgr2=0x7f2ea80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 shutdown_connections 2023-10-06T05:22:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.871+0000 7f2ead8da700 1 -- 172.21.15.45:0/583715302 wait complete. 2023-10-06T05:22:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:22:33.888 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:22:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 Processor -- start 2023-10-06T05:22:33.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 -- start start 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda53cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 0x55eadd9a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 0x55eadda4e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadd8c8980 con 0x55eadd9c57e0 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadd8c8620 con 0x55eadda50f00 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.933+0000 7f3911737040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eadd8bf400 con 0x55eadda50b30 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda53cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda53cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54018/0 (socket says 172.21.15.45:54018) 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 -- 172.21.15.45:0/994003688 learned_addr learned my addr 172.21.15.45:0/994003688 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390b156700 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 0x55eadda4e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a154700 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 0x55eadd9a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 -- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 msgr2=0x55eadd9a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 0x55eadd9a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 -- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 msgr2=0x55eadda4e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 0x55eadda4e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 -- 172.21.15.45:0/994003688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390400a180 con 0x55eadd9c57e0 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f390a955700 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda53cf0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00a870 tx=0x7f38fc005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec129ec1c4d46c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f3909152700 1 -- 172.21.15.45:0/994003688 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00f070 con 0x55eadd9c57e0 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f3909152700 1 -- 172.21.15.45:0/994003688 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc008800 con 0x55eadd9c57e0 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.934+0000 7f3909152700 1 -- 172.21.15.45:0/994003688 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc008ba0 con 0x55eadd9c57e0 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 -- 172.21.15.45:0/994003688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 msgr2=0x55eadda53cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda53cf0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00a870 tx=0x7f38fc005d00 comp rx=0 tx=0).stop 2023-10-06T05:22:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 -- 172.21.15.45:0/994003688 shutdown_connections 2023-10-06T05:22:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 0x55eadda4e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda53cf0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 --2- 172.21.15.45:0/994003688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 0x55eadd9a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 -- 172.21.15.45:0/994003688 >> 172.21.15.45:0/994003688 conn(0x55eadd97e450 msgr2=0x55eadda4fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 -- 172.21.15.45:0/994003688 shutdown_connections 2023-10-06T05:22:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.935+0000 7f3911737040 1 -- 172.21.15.45:0/994003688 wait complete. 2023-10-06T05:22:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.936+0000 7f3911737040 1 Processor -- start 2023-10-06T05:22:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.936+0000 7f3911737040 1 -- start start 2023-10-06T05:22:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.936+0000 7f3911737040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda52110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.936+0000 7f3911737040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 0x55eadda572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.936+0000 7f3911737040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 0x55eadda4b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.936+0000 7f3911737040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadda52740 con 0x55eadd9c57e0 2023-10-06T05:22:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.936+0000 7f3911737040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eadda529e0 con 0x55eadda50f00 2023-10-06T05:22:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f3911737040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eadda52c80 con 0x55eadda50b30 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda52110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54020/0 (socket says 172.21.15.45:54020) 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 learned_addr learned my addr 172.21.15.45:0/908558762 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a154700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 0x55eadda572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390b156700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 0x55eadda4b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 msgr2=0x55eadda572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eadda50b30 0x55eadda572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 msgr2=0x55eadda4b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eadda50f00 0x55eadda4b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc00a440 con 0x55eadd9c57e0 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f390a955700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda52110 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00bfd0 tx=0x7f38fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f38fb7fe700 1 -- 172.21.15.45:0/908558762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc013740 con 0x55eadd9c57e0 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f38fb7fe700 1 -- 172.21.15.45:0/908558762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc0138a0 con 0x55eadd9c57e0 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f38fb7fe700 1 -- 172.21.15.45:0/908558762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc013c60 con 0x55eadd9c57e0 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.937+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eadd8d73d0 con 0x55eadd9c57e0 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.938+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eadd8caae0 con 0x55eadd9c57e0 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.939+0000 7f38fb7fe700 1 -- 172.21.15.45:0/908558762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38fc01a050 con 0x55eadd9c57e0 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.939+0000 7f38fb7fe700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38f0065890 0x7f38f0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.939+0000 7f38fb7fe700 1 -- 172.21.15.45:0/908558762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f38fc084710 con 0x55eadd9c57e0 2023-10-06T05:22:33.944 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:22:33.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.940+0000 7f390a154700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38f0065890 0x7f38f0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.940+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55eadda53660 0x55eadd9bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.940+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda4b5b0 con 0x55eadda53660 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.940+0000 7f390b156700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55eadda53660 0x55eadd9bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.940+0000 7f390a154700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38f0065890 0x7f38f0067d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3900000c00 tx=0x7f39000038f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.940+0000 7f390b156700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55eadda53660 0x55eadd9bd3e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.942+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v95'73 uv48 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.942+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadd9bd920 con 0x55eadda53660 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.943+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v95'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.943+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadd9bdcd0 con 0x55eadda53660 2023-10-06T05:22:33.947 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:22:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.946+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v95'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.946+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadd9be080 con 0x55eadda53660 2023-10-06T05:22:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.948+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v95'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.948+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda71ed0 con 0x55eadda53660 2023-10-06T05:22:33.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.954+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v95'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.954+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda713b0 con 0x55eadda53660 2023-10-06T05:22:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.955+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v95'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.955+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda717b0 con 0x55eadda53660 2023-10-06T05:22:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.958+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v95'79 uv54 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.958+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.964+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v95'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.964+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.965+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v95'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.967 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:22:33.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.965+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.968+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v95'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.968+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.971+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v95'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.971+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.974+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v95'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.974+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.976+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v95'85 uv59 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.976+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.979+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v95'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.979+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.982+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v95'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.983 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.982+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:33.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.984+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v95'88 uv88 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.984+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:33.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.986+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v95'89 uv89 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.986+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.987+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v95'90 uv90 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.987+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.988+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v95'91 uv91 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.988+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.989+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v95'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.991 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:22:33.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.989+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.991+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v95'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.991+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.992+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v95'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.992+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.993+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v95'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.994+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.995+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 603.00000000 [create] v95'96 uv71 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.995+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.996+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 603.00000000 [delete] v95'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.996+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:33.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.997+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v95'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:33.999 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:22:33.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.998+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:34.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.999+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v95'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:34.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:33.999+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda73290 con 0x55eadda53660 2023-10-06T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.000+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v95'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:34.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.000+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda72280 con 0x55eadda53660 2023-10-06T05:22:34.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.002+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v95'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda53660 2023-10-06T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.002+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55eadda72790 0x55eadda74b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.002+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:30 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda750b0 con 0x55eadda72790 2023-10-06T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.002+0000 7f390a955700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55eadda72790 0x55eadda74b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.002+0000 7f390a955700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55eadda72790 0x55eadda74b70 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.004+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(30 1.00000000.inode [create] v95'66 uv48 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.004+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:31 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda75460 con 0x55eadda72790 2023-10-06T05:22:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.005+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(31 1.00000000.inode [delete] v95'67 uv67 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.005+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda75810 con 0x55eadda72790 2023-10-06T05:22:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.006+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(32 1.00000000.inode [create] v95'68 uv68 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.008 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:22:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.006+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda760b0 con 0x55eadda72790 2023-10-06T05:22:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.007+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(33 1.00000000.inode [write 0~554] v95'69 uv69 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.007+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76460 con 0x55eadda72790 2023-10-06T05:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.009+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v95'70 uv70 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.009+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76810 con 0x55eadda72790 2023-10-06T05:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.010+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(35 200.00000000 [create] v95'71 uv53 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.010+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.012+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(36 200.00000000 [delete] v95'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.012+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.017 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.015+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(37 200.00000000 [create] v95'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.015+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.017+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v95'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.017+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.020+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v95'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.021+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.024+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(40 200.00000001 [create] v95'76 uv58 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.024+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.028+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(41 200.00000001 [delete] v95'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.028+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.031 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:22:34.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.030+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(42 200.00000001 [create] v95'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.030+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7a2e0 con 0x55eadda72790 2023-10-06T05:22:34.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.032+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(43 200.00000001 [write 0~14075] v95'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.032+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7a2e0 con 0x55eadda72790 2023-10-06T05:22:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.038+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v95'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.038+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.039+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(45 606.00000000 [create] v95'81 uv64 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.039+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.041+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(46 606.00000000 [delete] v95'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.041+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.046 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:22:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.044+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(47 606.00000000 [create] v95'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.044+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda77bd0 con 0x55eadda72790 2023-10-06T05:22:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.046+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v95'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.046+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda77bd0 con 0x55eadda72790 2023-10-06T05:22:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.049+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v95'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.049+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda76bc0 con 0x55eadda72790 2023-10-06T05:22:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.052+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v95'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.052+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55eadda770d0 0x55eadda794b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.052+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:51 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda799f0 con 0x55eadda770d0 2023-10-06T05:22:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.052+0000 7f390a154700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55eadda770d0 0x55eadda794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.052+0000 7f390a154700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55eadda770d0 0x55eadda794b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.055+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(51 607.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.055+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:52 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7a2c0 con 0x55eadda770d0 2023-10-06T05:22:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.060+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(52 607.00000000 [delete] v95'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.060+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7a670 con 0x55eadda770d0 2023-10-06T05:22:34.063 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:22:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.061+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(53 607.00000000 [create] v95'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.061+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7bab0 con 0x55eadda770d0 2023-10-06T05:22:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.064+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.064+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7bfc0 con 0x55eadda770d0 2023-10-06T05:22:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.067+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.067+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7aa20 con 0x55eadda770d0 2023-10-06T05:22:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.069+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.069+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7ae50 con 0x55eadda770d0 2023-10-06T05:22:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.071+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(57 mds0_openfiles.0 [create] v95'12 uv11 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.071+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b2a0 con 0x55eadda770d0 2023-10-06T05:22:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.072+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v95'13 uv13 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.072+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b2a0 con 0x55eadda770d0 2023-10-06T05:22:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.073+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(59 mds0_openfiles.0 [create] v95'14 uv14 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.075 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:22:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.073+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b250 con 0x55eadda770d0 2023-10-06T05:22:34.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.074+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v95'15 uv15 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.074+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:61 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b200 con 0x55eadda72790 2023-10-06T05:22:34.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.076+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(61 608.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.076+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:62 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b200 con 0x55eadda72790 2023-10-06T05:22:34.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.077+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(62 608.00000000 [delete] v95'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.077+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b200 con 0x55eadda72790 2023-10-06T05:22:34.081 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:22:34.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.079+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(63 608.00000000 [create] v95'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.079+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b200 con 0x55eadda72790 2023-10-06T05:22:34.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.080+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.080+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b200 con 0x55eadda72790 2023-10-06T05:22:34.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.081+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.081+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b200 con 0x55eadda72790 2023-10-06T05:22:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.083+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(66 608.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda72790 2023-10-06T05:22:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.083+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55eadda7c900 0x55eadda7ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.083+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:67 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7b200 con 0x55eadda7c900 2023-10-06T05:22:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.083+0000 7f390b156700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55eadda7c900 0x55eadda7ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.084+0000 7f390b156700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55eadda7c900 0x55eadda7ece0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.085+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(67 604.00000000 [create] v95'26 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.085+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:68 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.086+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(68 604.00000000 [delete] v95'27 uv27 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.086+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.087+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(69 604.00000000 [create] v95'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.090 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:22:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.088+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda80230 con 0x55eadda7c900 2023-10-06T05:22:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.089+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v95'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.089+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda80230 con 0x55eadda7c900 2023-10-06T05:22:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.090+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v95'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.090+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.091+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v95'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.091+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.092+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(73 500.00000000 [create] v95'32 uv25 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.092+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.094+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(74 500.00000000 [delete] v95'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.094+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.096 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:22:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.095+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(75 500.00000000 [create] v95'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.095+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.096+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v95'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.096+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda7f220 con 0x55eadda7c900 2023-10-06T05:22:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.097+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v95'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.097+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55eadda7f8a0 0x55eadda81c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.097+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:78 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda82380 con 0x55eadda7f8a0 2023-10-06T05:22:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.097+0000 7f390a955700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55eadda7f8a0 0x55eadda81c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.098+0000 7f390a955700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55eadda7f8a0 0x55eadda81c80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.099+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(78 mds_snaptable [create] v95'12 uv11 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda7f8a0 2023-10-06T05:22:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.099+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:79 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda823c0 con 0x55eadda7f8a0 2023-10-06T05:22:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.100+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(79 mds_snaptable [delete] v95'13 uv13 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda7f8a0 2023-10-06T05:22:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.100+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda827f0 con 0x55eadda7f8a0 2023-10-06T05:22:34.103 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:22:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.101+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(80 mds_snaptable [create] v95'14 uv14 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda7f8a0 2023-10-06T05:22:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.101+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda827f0 con 0x55eadda7f8a0 2023-10-06T05:22:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.102+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v95'15 uv15 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda7f8a0 2023-10-06T05:22:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.102+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda827f0 con 0x55eadda7f8a0 2023-10-06T05:22:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.104+0000 7f390a955700 1 -- 172.21.15.45:0/908558762 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v95'16 uv16 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f38fc082900 con 0x55eadda7f8a0 2023-10-06T05:22:34.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.104+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:83 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda827f0 con 0x55eadda770d0 2023-10-06T05:22:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.105+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(83 605.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.105+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:84 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda827f0 con 0x55eadda770d0 2023-10-06T05:22:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.107+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(84 605.00000000 [delete] v95'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.107+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda827f0 con 0x55eadda770d0 2023-10-06T05:22:34.109 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:22:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.108+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(85 605.00000000 [create] v95'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.108+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda83800 con 0x55eadda770d0 2023-10-06T05:22:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.109+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.109+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda83800 con 0x55eadda770d0 2023-10-06T05:22:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.110+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.110+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda827f0 con 0x55eadda770d0 2023-10-06T05:22:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.111+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f39000127a0 con 0x55eadda770d0 2023-10-06T05:22:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.111+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55eadda82e70 0x55eadda85250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.111+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:89 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda857e0 con 0x55eadda82e70 2023-10-06T05:22:34.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.112+0000 7f390a154700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55eadda82e70 0x55eadda85250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.112+0000 7f390a154700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55eadda82e70 0x55eadda85250 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.113+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(89 mds0_inotable [create] v95'24 uv18 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.113+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:90 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda858a0 con 0x55eadda82e70 2023-10-06T05:22:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.115+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(90 mds0_inotable [delete] v95'25 uv25 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.115+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85cd0 con 0x55eadda82e70 2023-10-06T05:22:34.117 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:22:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.116+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(91 mds0_inotable [create] v95'26 uv26 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.116+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85cd0 con 0x55eadda82e70 2023-10-06T05:22:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.117+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(92 mds0_inotable [write 0~34] v95'27 uv27 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.117+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85cd0 con 0x55eadda82e70 2023-10-06T05:22:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.118+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v95'28 uv28 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.118+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85cd0 con 0x55eadda82e70 2023-10-06T05:22:34.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.119+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(94 100.00000000 [create] v95'29 uv23 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.119+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85cd0 con 0x55eadda82e70 2023-10-06T05:22:34.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.121+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(95 100.00000000 [delete] v95'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.121+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85cd0 con 0x55eadda82e70 2023-10-06T05:22:34.123 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:22:34.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.122+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(96 100.00000000 [create] v95'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.122+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85cd0 con 0x55eadda82e70 2023-10-06T05:22:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.123+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(97 100.00000000 [omap-set-header] v95'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.123+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda88a40 con 0x55eadda82e70 2023-10-06T05:22:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.124+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v95'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda82e70 2023-10-06T05:22:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.124+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda861c0 con 0x55eadda770d0 2023-10-06T05:22:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.126+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v95'14 uv12 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.126+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda861c0 con 0x55eadda770d0 2023-10-06T05:22:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.127+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v95'15 uv15 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.127+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda861c0 con 0x55eadda770d0 2023-10-06T05:22:34.129 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.128+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v95'16 uv16 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.128+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda861c0 con 0x55eadda770d0 2023-10-06T05:22:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.129+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v95'17 uv17 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.129+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55eadda866d0 0x55eadda86ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.129+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda866d0 2023-10-06T05:22:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.130+0000 7f390b156700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55eadda866d0 0x55eadda86ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.130+0000 7f390b156700 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55eadda866d0 0x55eadda86ac0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.131+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda866d0 2023-10-06T05:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.131+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda866d0 2023-10-06T05:22:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.132+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v95'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda866d0 2023-10-06T05:22:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.133+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda866d0 2023-10-06T05:22:34.136 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:22:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.134+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v95'17 uv17 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda866d0 2023-10-06T05:22:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.135+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda8a1a0 con 0x55eadda866d0 2023-10-06T05:22:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.135+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda866d0 2023-10-06T05:22:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.136+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda8a1a0 con 0x55eadda866d0 2023-10-06T05:22:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.136+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda866d0 2023-10-06T05:22:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.137+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda866d0 2023-10-06T05:22:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.138+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda866d0 2023-10-06T05:22:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.139+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda7c900 2023-10-06T05:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.140+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v95'12 uv11 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.140+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda7c900 2023-10-06T05:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.142+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v95'13 uv13 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.142+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda7c900 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.143+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v95'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.143+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda7c900 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.145+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v95'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.145+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda7c900 2023-10-06T05:22:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.147+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v95'16 uv16 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f390400e7e0 con 0x55eadda7c900 2023-10-06T05:22:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.147+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda770d0 2023-10-06T05:22:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.148+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v95'33 uv24 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.148+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda770d0 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.149+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v95'34 uv34 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.149+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda770d0 2023-10-06T05:22:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.150+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v95'35 uv35 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:22:34.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.150+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda770d0 2023-10-06T05:22:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.151+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v95'36 uv36 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.151+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85d50 con 0x55eadda770d0 2023-10-06T05:22:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.152+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v95'37 uv37 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.152+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda85d50 con 0x55eadda770d0 2023-10-06T05:22:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.153+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v95'38 uv31 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.153+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda770d0 2023-10-06T05:22:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.154+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v95'39 uv39 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.155+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89190 con 0x55eadda770d0 2023-10-06T05:22:34.157 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.156+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v95'40 uv40 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.156+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda8a1a0 con 0x55eadda770d0 2023-10-06T05:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.157+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v95'41 uv41 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.157+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda8a1a0 con 0x55eadda770d0 2023-10-06T05:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.158+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v95'42 uv42 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.158+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89680 con 0x55eadda770d0 2023-10-06T05:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.159+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v95'43 uv43 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.159+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v8 -- 0x55eadda89680 con 0x55eadda770d0 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f390a154700 1 -- 172.21.15.45:0/908558762 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v95'44 uv44 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3900017040 con 0x55eadda770d0 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55eadda770d0 msgr2=0x55eadda794b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55eadda770d0 0x55eadda794b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55eadda53660 msgr2=0x55eadd9bd3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55eadda53660 0x55eadd9bd3e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55eadda82e70 msgr2=0x55eadda85250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55eadda82e70 0x55eadda85250 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55eadda72790 msgr2=0x55eadda74b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55eadda72790 0x55eadda74b70 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55eadda866d0 msgr2=0x55eadda86ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55eadda866d0 0x55eadda86ac0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55eadda7f8a0 msgr2=0x55eadda81c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55eadda7f8a0 0x55eadda81c80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55eadda7c900 msgr2=0x55eadda7ece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55eadda7c900 0x55eadda7ece0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 reap_dead start 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 reap_dead start 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 reap_dead start 2023-10-06T05:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 reap_dead start 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f390b156700 1 -- 172.21.15.45:0/908558762 reap_dead start 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38f0065890 msgr2=0x7f38f0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38f0065890 0x7f38f0067d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3900000c00 tx=0x7f39000038f0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 msgr2=0x55eadda52110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda52110 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00bfd0 tx=0x7f38fc018000 comp rx=0 tx=0).stop 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 shutdown_connections 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eadd9c57e0 0x55eadda52110 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 --2- 172.21.15.45:0/908558762 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f38f0065890 0x7f38f0067d40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 >> 172.21.15.45:0/908558762 conn(0x55eadd97e450 msgr2=0x55eadd97fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 shutdown_connections 2023-10-06T05:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.160+0000 7f3911737040 1 -- 172.21.15.45:0/908558762 wait complete. 2023-10-06T05:22:34.166 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2023-10-06T05:22:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.215+0000 7fa99a0db040 1 Processor -- start 2023-10-06T05:22:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.215+0000 7fa99a0db040 1 -- start start 2023-10-06T05:22:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa99a0db040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 0x557d9ec6b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa99a0db040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec519a0 0x557d9ec4dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa99a0db040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec68400 0x557d9eb97130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa99a0db040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9eade620 con 0x557d9ec519a0 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa99a0db040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ead5400 con 0x557d9ec68400 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa99a0db040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d9ec50af0 con 0x557d9ebc05d0 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec519a0 0x557d9ec4dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec519a0 0x557d9ec4dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54052/0 (socket says 172.21.15.45:54052) 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 -- 172.21.15.45:0/3005024143 learned_addr learned my addr 172.21.15.45:0/3005024143 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa9932f9700 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 0x557d9ec6b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa993afa700 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec68400 0x557d9eb97130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 -- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 msgr2=0x557d9ec6b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 0x557d9ec6b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 -- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec68400 msgr2=0x557d9eb97130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec68400 0x557d9eb97130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 -- 172.21.15.45:0/3005024143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984005db0 con 0x557d9ec519a0 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.216+0000 7fa992af8700 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec519a0 0x557d9ec4dfb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa988009950 tx=0x7fa98800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d70ef2b0d47b9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa991af6700 1 -- 172.21.15.45:0/3005024143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800f040 con 0x557d9ec519a0 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa991af6700 1 -- 172.21.15.45:0/3005024143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988008930 con 0x557d9ec519a0 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa991af6700 1 -- 172.21.15.45:0/3005024143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988008cd0 con 0x557d9ec519a0 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 -- 172.21.15.45:0/3005024143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec519a0 msgr2=0x557d9ec4dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec519a0 0x557d9ec4dfb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa988009950 tx=0x7fa98800c040 comp rx=0 tx=0).stop 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 -- 172.21.15.45:0/3005024143 shutdown_connections 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec68400 0x557d9eb97130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 0x557d9ec6b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3005024143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec519a0 0x557d9ec4dfb0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 -- 172.21.15.45:0/3005024143 >> 172.21.15.45:0/3005024143 conn(0x557d9eb95750 msgr2=0x557d9ebc3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 -- 172.21.15.45:0/3005024143 shutdown_connections 2023-10-06T05:22:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.217+0000 7fa99a0db040 1 -- 172.21.15.45:0/3005024143 wait complete. 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.218+0000 7fa99a0db040 1 Processor -- start 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.218+0000 7fa99a0db040 1 -- start start 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa99a0db040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 0x557d9ec78820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa99a0db040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec519a0 0x557d9ec4c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa99a0db040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec68400 0x557d9ec4cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa99a0db040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ec4a120 con 0x557d9ec68400 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa99a0db040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d9ec4a3c0 con 0x557d9ec519a0 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa99a0db040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d9ec4a660 con 0x557d9ebc05d0 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec68400 0x557d9ec4cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec68400 0x557d9ec4cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54068/0 (socket says 172.21.15.45:54068) 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 -- 172.21.15.45:0/3619977649 learned_addr learned my addr 172.21.15.45:0/3619977649 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa9932f9700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 0x557d9ec78820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa992af8700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec519a0 0x557d9ec4c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 -- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 msgr2=0x557d9ec78820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557d9ebc05d0 0x557d9ec78820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 -- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec519a0 msgr2=0x557d9ec4c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557d9ec519a0 0x557d9ec4c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.219+0000 7fa993afa700 1 -- 172.21.15.45:0/3619977649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa988009580 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.220+0000 7fa993afa700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec68400 0x557d9ec4cec0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00f750 tx=0x7fa98c007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.220+0000 7fa97bfff700 1 -- 172.21.15.45:0/3619977649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00c3d0 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.220+0000 7fa97bfff700 1 -- 172.21.15.45:0/3619977649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c013640 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.220+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557d9eb071c0 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.220+0000 7fa97bfff700 1 -- 172.21.15.45:0/3619977649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0139e0 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.220+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d9eaed3d0 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.221+0000 7fa97bfff700 1 -- 172.21.15.45:0/3619977649 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa98c011400 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.222+0000 7fa97bfff700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980065890 0x7fa980067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.222+0000 7fa97bfff700 1 -- 172.21.15.45:0/3619977649 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fa98c083b80 con 0x557d9ec68400 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.222+0000 7fa9932f9700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980065890 0x7fa980067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.222+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557d9ec7c540 0x557d9ec7c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.222+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e95) v8 -- 0x557d9ec7ce80 con 0x557d9ec7c540 2023-10-06T05:22:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.222+0000 7fa992af8700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557d9ec7c540 0x557d9ec7c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.223+0000 7fa9932f9700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980065890 0x7fa980067d40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa98400a650 tx=0x7fa984016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.223+0000 7fa992af8700 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557d9ec7c540 0x557d9ec7c940 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa992af8700 1 -- 172.21.15.45:0/3619977649 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 1.00000000.inode [delete] v95'87 uv87 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa988019020 con 0x557d9ec7c540 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557d9ec7c540 msgr2=0x557d9ec7c940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557d9ec7c540 0x557d9ec7c940 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980065890 msgr2=0x7fa980067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980065890 0x7fa980067d40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa98400a650 tx=0x7fa984016000 comp rx=0 tx=0).stop 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec68400 msgr2=0x557d9ec4cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 --2- 172.21.15.45:0/3619977649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d9ec68400 0x557d9ec4cec0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00f750 tx=0x7fa98c007040 comp rx=0 tx=0).stop 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa993afa700 1 -- 172.21.15.45:0/3619977649 reap_dead start 2023-10-06T05:22:34.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 shutdown_connections 2023-10-06T05:22:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 >> 172.21.15.45:0/3619977649 conn(0x557d9eb95750 msgr2=0x557d9eb968a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 shutdown_connections 2023-10-06T05:22:34.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.224+0000 7fa99a0db040 1 -- 172.21.15.45:0/3619977649 wait complete. 2023-10-06T05:22:34.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:22:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.369+0000 7fe5fd570700 1 Processor -- start 2023-10-06T05:22:34.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.369+0000 7fe5fd570700 1 -- start start 2023-10-06T05:22:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5fd570700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5f80d02b0 0x7fe5f80d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5fd570700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5f815c8c0 0x7fe5f80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5fd570700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80cd740 con 0x7fe5f815d820 2023-10-06T05:22:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5fd570700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80ccb30 con 0x7fe5f80d02b0 2023-10-06T05:22:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5fd570700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80cce10 con 0x7fe5f815c8c0 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5f80d02b0 0x7fe5f80d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f67fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe5f815d820 0x7fe5f815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39464/0 (socket says 172.21.15.45:39464) 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5f80d02b0 0x7fe5f80d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54076/0 (socket says 172.21.15.45:54076) 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f67fc700 1 -- 172.21.15.45:0/3231683412 learned_addr learned my addr 172.21.15.45:0/3231683412 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f77fe700 1 --2- 172.21.15.45:0/3231683412 >> v2:172.21.15.144:3300/0 conn(0x7fe5f815c8c0 0x7fe5f80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f5ffb700 1 -- 172.21.15.45:0/3231683412 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480824273 0 0) 0x7fe5f80cd740 con 0x7fe5f815d820 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f5ffb700 1 -- 172.21.15.45:0/3231683412 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e4003570 con 0x7fe5f815d820 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f6ffd700 1 -- 172.21.15.45:0/3231683412 >> v2:172.21.15.144:3300/0 conn(0x7fe5f815c8c0 msgr2=0x7fe5f80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f6ffd700 1 --2- 172.21.15.45:0/3231683412 >> v2:172.21.15.144:3300/0 conn(0x7fe5f815c8c0 0x7fe5f80caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f6ffd700 1 -- 172.21.15.45:0/3231683412 >> v1:172.21.15.45:6789/0 conn(0x7fe5f815d820 legacy=0x7fe5f815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.370+0000 7fe5f6ffd700 1 -- 172.21.15.45:0/3231683412 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80cd0f0 con 0x7fe5f80d02b0 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5f6ffd700 1 --2- 172.21.15.45:0/3231683412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f80d02b0 0x7fe5f80d0680 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80b0e00 tx=0x7fe5e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1249b0bafebd1b77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5f5ffb700 1 -- 172.21.15.45:0/3231683412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011630 con 0x7fe5f80d02b0 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5f5ffb700 1 -- 172.21.15.45:0/3231683412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e00117b0 con 0x7fe5f80d02b0 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5f5ffb700 1 -- 172.21.15.45:0/3231683412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011b70 con 0x7fe5f80d02b0 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 -- 172.21.15.45:0/3231683412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f80d02b0 msgr2=0x7fe5f80d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 --2- 172.21.15.45:0/3231683412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f80d02b0 0x7fe5f80d0680 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80b0e00 tx=0x7fe5e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 -- 172.21.15.45:0/3231683412 shutdown_connections 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 --2- 172.21.15.45:0/3231683412 >> v2:172.21.15.144:3300/0 conn(0x7fe5f815c8c0 0x7fe5f80caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 --2- 172.21.15.45:0/3231683412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f80d02b0 0x7fe5f80d0680 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 -- 172.21.15.45:0/3231683412 >> 172.21.15.45:0/3231683412 conn(0x7fe5f806ed50 msgr2=0x7fe5f806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 -- 172.21.15.45:0/3231683412 shutdown_connections 2023-10-06T05:22:34.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.371+0000 7fe5fd570700 1 -- 172.21.15.45:0/3231683412 wait complete. 2023-10-06T05:22:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.373+0000 7fe5fd570700 1 Processor -- start 2023-10-06T05:22:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.373+0000 7fe5fd570700 1 -- start start 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f80d02b0 0x7fe5f814e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815c8c0 0x7fe5f80ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5f815d820 0x7fe5f80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815a110 con 0x7fe5f815c8c0 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80c99c0 con 0x7fe5f80d02b0 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80c8db0 con 0x7fe5f815d820 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815c8c0 0x7fe5f80ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f80d02b0 0x7fe5f814e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815c8c0 0x7fe5f80ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54086/0 (socket says 172.21.15.45:54086) 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 -- 172.21.15.45:0/907364414 learned_addr learned my addr 172.21.15.45:0/907364414 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f77fe700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5f815d820 0x7fe5f80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 -- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5f815d820 msgr2=0x7fe5f80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5f815d820 0x7fe5f80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 -- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f80d02b0 msgr2=0x7fe5f814e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f80d02b0 0x7fe5f814e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 -- 172.21.15.45:0/907364414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009420 con 0x7fe5f815c8c0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5f67fc700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815c8c0 0x7fe5f80ce580 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c590 tx=0x7fe5e8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5eeffd700 1 -- 172.21.15.45:0/907364414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8005b30 con 0x7fe5f815c8c0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5eeffd700 1 -- 172.21.15.45:0/907364414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e8005cb0 con 0x7fe5f815c8c0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f80c9090 con 0x7fe5f815c8c0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5eeffd700 1 -- 172.21.15.45:0/907364414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8007320 con 0x7fe5f815c8c0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.374+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f80c9650 con 0x7fe5f815c8c0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.375+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f8088730 con 0x7fe5f815c8c0 2023-10-06T05:22:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.376+0000 7fe5eeffd700 1 -- 172.21.15.45:0/907364414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5e801b3b0 con 0x7fe5f815c8c0 2023-10-06T05:22:34.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.376+0000 7fe5eeffd700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5d8066f30 0x7fe5d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.377+0000 7fe5eeffd700 1 -- 172.21.15.45:0/907364414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fe5e8083d50 con 0x7fe5f815c8c0 2023-10-06T05:22:34.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.377+0000 7fe5f6ffd700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5d8066f30 0x7fe5d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.377+0000 7fe5f6ffd700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5d8066f30 0x7fe5d80693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80b0e00 tx=0x7fe5e0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.381+0000 7fe5eeffd700 1 -- 172.21.15.45:0/907364414 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5e8087030 con 0x7fe5f815c8c0 2023-10-06T05:22:34.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.629+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe5f80c9b00 con 0x7fe5f815c8c0 2023-10-06T05:22:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.872+0000 7fe5eeffd700 1 -- 172.21.15.45:0/907364414 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v107) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe5e80170b0 con 0x7fe5f815c8c0 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5d8066f30 msgr2=0x7fe5d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5d8066f30 0x7fe5d80693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80b0e00 tx=0x7fe5e0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815c8c0 msgr2=0x7fe5f80ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815c8c0 0x7fe5f80ce580 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c590 tx=0x7fe5e8007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 shutdown_connections 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5d8066f30 0x7fe5d80693e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5f815d820 0x7fe5f80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f80d02b0 0x7fe5f814e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 --2- 172.21.15.45:0/907364414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815c8c0 0x7fe5f80ce580 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 >> 172.21.15.45:0/907364414 conn(0x7fe5f806ed50 msgr2=0x7fe5f806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 shutdown_connections 2023-10-06T05:22:34.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:34.875+0000 7fe5fd570700 1 -- 172.21.15.45:0/907364414 wait complete. 2023-10-06T05:22:34.878 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:22:34.886 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:22:34.884+0000 7f3e122c9700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-10-06T05:22:34.886 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:22:34.884+0000 7f3e122c9700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:22:34.887 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:34.885+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:34.887 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:34.885+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:22:34.893 INFO:tasks.ceph.mds.a.smithi045.stderr: -15> 2023-10-06T05:22:34.884+0000 7f3e122c9700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-10-06T05:22:34.893 INFO:tasks.ceph.mds.a.smithi045.stderr: -13> 2023-10-06T05:22:34.884+0000 7f3e122c9700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:22:34.896 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:34.915 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:22:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.028+0000 7f49dad00700 1 Processor -- start 2023-10-06T05:22:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.028+0000 7f49dad00700 1 -- start start 2023-10-06T05:22:35.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.029+0000 7f49dad00700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f49d4158f60 0x7f49d4159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.029+0000 7f49dad00700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f49d406a060 0x7f49d40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.029+0000 7f49dad00700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d4069bd0 con 0x7f49d40cc390 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.029+0000 7f49dad00700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d406a500 con 0x7f49d4158f60 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.029+0000 7f49dad00700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d4068fc0 con 0x7f49d406a060 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.029+0000 7f49d8a9c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f49d4158f60 0x7f49d4159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.029+0000 7f49d3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f49d406a060 0x7f49d40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d929d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f49d40cc390 0x7f49d40c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:34090/0 (socket says 172.21.15.45:34090) 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d8a9c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f49d4158f60 0x7f49d4159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54116/0 (socket says 172.21.15.45:54116) 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d929d700 1 -- 172.21.15.45:0/3760729650 learned_addr learned my addr 172.21.15.45:0/3760729650 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f49d406a060 0x7f49d40cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50266/0 (socket says 172.21.15.45:50266) 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d3fff700 1 -- 172.21.15.45:0/3760729650 >> v2:172.21.15.45:3300/0 conn(0x7f49d4158f60 msgr2=0x7f49d4159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d3fff700 1 --2- 172.21.15.45:0/3760729650 >> v2:172.21.15.45:3300/0 conn(0x7f49d4158f60 0x7f49d4159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d3fff700 1 -- 172.21.15.45:0/3760729650 >> v1:172.21.15.144:6789/0 conn(0x7f49d40cc390 legacy=0x7f49d40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d3fff700 1 -- 172.21.15.45:0/3760729650 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40692a0 con 0x7f49d406a060 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d8a9c700 1 --2- 172.21.15.45:0/3760729650 >> v2:172.21.15.45:3300/0 conn(0x7f49d4158f60 0x7f49d4159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d3fff700 1 --2- 172.21.15.45:0/3760729650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 0x7f49d40cae20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f49d40872e0 tx=0x7f49c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b35cc65655e516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d37fe700 1 -- 172.21.15.45:0/3760729650 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0011630 con 0x7f49d406a060 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d37fe700 1 -- 172.21.15.45:0/3760729650 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c0011790 con 0x7f49d406a060 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.030+0000 7f49d37fe700 1 -- 172.21.15.45:0/3760729650 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0011b30 con 0x7f49d406a060 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 -- 172.21.15.45:0/3760729650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 msgr2=0x7f49d40cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 --2- 172.21.15.45:0/3760729650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 0x7f49d40cae20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f49d40872e0 tx=0x7f49c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 -- 172.21.15.45:0/3760729650 shutdown_connections 2023-10-06T05:22:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 --2- 172.21.15.45:0/3760729650 >> v2:172.21.15.45:3300/0 conn(0x7f49d4158f60 0x7f49d4159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 --2- 172.21.15.45:0/3760729650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 0x7f49d40cae20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 -- 172.21.15.45:0/3760729650 >> 172.21.15.45:0/3760729650 conn(0x7f49d40cdf50 msgr2=0x7f49d40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 -- 172.21.15.45:0/3760729650 shutdown_connections 2023-10-06T05:22:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.031+0000 7f49dad00700 1 -- 172.21.15.45:0/3760729650 wait complete. 2023-10-06T05:22:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.032+0000 7f49dad00700 1 Processor -- start 2023-10-06T05:22:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.032+0000 7f49dad00700 1 -- start start 2023-10-06T05:22:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49dad00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 0x7f49d40c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49dad00700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49d40cc390 0x7f49d40c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49dad00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d4158f60 0x7f49d40c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49dad00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40c7ea0 con 0x7f49d4158f60 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49dad00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d4169740 con 0x7f49d406a060 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49dad00700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d4158350 con 0x7f49d40cc390 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49d929d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d4158f60 0x7f49d40c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49d929d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d4158f60 0x7f49d40c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54126/0 (socket says 172.21.15.45:54126) 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49d929d700 1 -- 172.21.15.45:0/1309639844 learned_addr learned my addr 172.21.15.45:0/1309639844 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49d8a9c700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 0x7f49d40c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.033+0000 7f49d3fff700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49d40cc390 0x7f49d40c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d929d700 1 -- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49d40cc390 msgr2=0x7f49d40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d929d700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49d40cc390 0x7f49d40c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d929d700 1 -- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 msgr2=0x7f49d40c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d929d700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 0x7f49d40c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d929d700 1 -- 172.21.15.45:0/1309639844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c8009420 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d929d700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d4158f60 0x7f49d40c9020 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00cc70 tx=0x7f49cc001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d0ff9700 1 -- 172.21.15.45:0/1309639844 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00a850 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d0ff9700 1 -- 172.21.15.45:0/1309639844 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00a9b0 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c0009420 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49d0ff9700 1 -- 172.21.15.45:0/1309639844 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00ad70 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.034+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d4169c90 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.035+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d406f410 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.035+0000 7f49d0ff9700 1 -- 172.21.15.45:0/1309639844 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49cc007300 con 0x7f49d4158f60 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.036+0000 7f49d0ff9700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49b8066f30 0x7f49b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.036+0000 7f49d0ff9700 1 -- 172.21.15.45:0/1309639844 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f49cc084c20 con 0x7f49d4158f60 2023-10-06T05:22:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.036+0000 7f49d8a9c700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49b8066f30 0x7f49b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.037+0000 7f49d8a9c700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49b8066f30 0x7f49b80693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f49d40cc020 tx=0x7f49c80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.040+0000 7f49d0ff9700 1 -- 172.21.15.45:0/1309639844 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49cc051b10 con 0x7f49d4158f60 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.284+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49d4069580 con 0x7f49d4158f60 2023-10-06T05:22:35.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.285+0000 7f49d0ff9700 1 -- 172.21.15.45:0/1309639844 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+3842 (secure 0 0 0) 0x7f49cc055130 con 0x7f49d4158f60 2023-10-06T05:22:35.287 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:35.287 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":109,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:34.886602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:35.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49b8066f30 msgr2=0x7f49b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49b8066f30 0x7f49b80693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f49d40cc020 tx=0x7f49c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d4158f60 msgr2=0x7f49d40c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d4158f60 0x7f49d40c9020 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00cc70 tx=0x7f49cc001a50 comp rx=0 tx=0).stop 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 shutdown_connections 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f49b8066f30 0x7f49b80693e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d4158f60 0x7f49d40c9020 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d406a060 0x7f49d40c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 --2- 172.21.15.45:0/1309639844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49d40cc390 0x7f49d40c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 >> 172.21.15.45:0/1309639844 conn(0x7f49d40cdf50 msgr2=0x7f49d415c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 shutdown_connections 2023-10-06T05:22:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.287+0000 7f49dad00700 1 -- 172.21.15.45:0/1309639844 wait complete. 2023-10-06T05:22:35.291 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 109 2023-10-06T05:22:35.302 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:22:35.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:35.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.475+0000 7f3cc8e66700 1 Processor -- start 2023-10-06T05:22:35.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.475+0000 7f3cc8e66700 1 -- start start 2023-10-06T05:22:35.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc8e66700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 0x7f3cc406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc8e66700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40cb530 con 0x7f3cc4158f60 2023-10-06T05:22:35.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc8e66700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc4069bd0 con 0x7f3cc4159330 2023-10-06T05:22:35.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc8e66700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4068fc0 con 0x7f3cc40ccaa0 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3cc4158f60 0x7f3cc40cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39470/0 (socket says 172.21.15.45:39470) 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc259c700 1 -- 172.21.15.45:0/4040920583 learned_addr learned my addr 172.21.15.45:0/4040920583 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc2d9d700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 0x7f3cc406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514736845 0 0) 0x7f3cb4001110 con 0x7f3cc4158f60 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.476+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb0003570 con 0x7f3cc4158f60 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048424709 0 0) 0x7f3cc4069bd0 con 0x7f3cc4159330 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb4001110 con 0x7f3cc4159330 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173727671 0 0) 0x7f3cb0003570 con 0x7f3cc4158f60 2023-10-06T05:22:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 msgr2=0x7f3cc406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 0x7f3cc406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 >> v1:172.21.15.144:6789/0 conn(0x7f3cc4159330 legacy=0x7f3cc40cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40692a0 con 0x7f3cc4158f60 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f3cb4001600 con 0x7f3cc4158f60 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 >> v1:172.21.15.45:6789/0 conn(0x7f3cc4158f60 legacy=0x7f3cc40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cb0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 0x7f3cb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0004d50 0x7f3cb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4068fc0 con 0x7f3cb0003c20 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb000bbf0 con 0x7f3cb0004d50 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb000bd60 con 0x7f3cc40ccaa0 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc259c700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cb0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cbbfff700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 0x7f3cb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc2d9d700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0004d50 0x7f3cb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc259c700 1 -- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 msgr2=0x7f3cb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc259c700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 0x7f3cb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc259c700 1 -- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0004d50 msgr2=0x7f3cb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc259c700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cb0004d50 0x7f3cb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cc259c700 1 -- 172.21.15.45:0/4040920583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0003570 con 0x7f3cb0003c20 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.477+0000 7f3cbbfff700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40ccaa0 0x7f3cb0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc259c700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cb0004010 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4001ed0 tx=0x7f3cb4007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e185de71b18a1ac6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400cc50 con 0x7f3cb0003c20 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb400cdd0 con 0x7f3cb0003c20 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc1d9b700 1 -- 172.21.15.45:0/4040920583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4005b70 con 0x7f3cb0003c20 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc8e66700 1 -- 172.21.15.45:0/4040920583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 msgr2=0x7f3cb0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc8e66700 1 --2- 172.21.15.45:0/4040920583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cb0004010 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4001ed0 tx=0x7f3cb4007940 comp rx=0 tx=0).stop 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc2d9d700 1 -- 172.21.15.45:0/4040920583 reap_dead start 2023-10-06T05:22:35.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc8e66700 1 -- 172.21.15.45:0/4040920583 shutdown_connections 2023-10-06T05:22:35.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc8e66700 1 -- 172.21.15.45:0/4040920583 >> 172.21.15.45:0/4040920583 conn(0x7f3cc40cdf50 msgr2=0x7f3cc40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc8e66700 1 -- 172.21.15.45:0/4040920583 shutdown_connections 2023-10-06T05:22:35.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.478+0000 7f3cc8e66700 1 -- 172.21.15.45:0/4040920583 wait complete. 2023-10-06T05:22:35.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.479+0000 7f3cc8e66700 1 Processor -- start 2023-10-06T05:22:35.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc8e66700 1 -- start start 2023-10-06T05:22:35.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc8e66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cc414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc8e66700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40a95b0 0x7f3cc40a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc8e66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40c3a40 0x7f3cc40c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc8e66700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c4420 con 0x7f3cb0003c20 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc8e66700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4167710 con 0x7f3cc40c3a40 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc8e66700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc416aa40 con 0x7f3cc40a95b0 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cc414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cc414dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54136/0 (socket says 172.21.15.45:54136) 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cc259c700 1 -- 172.21.15.45:0/3850479531 learned_addr learned my addr 172.21.15.45:0/3850479531 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.480+0000 7f3cbbfff700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40a95b0 0x7f3cc40a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc2d9d700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40c3a40 0x7f3cc40c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc259c700 1 -- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40a95b0 msgr2=0x7f3cc40a9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc259c700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40a95b0 0x7f3cc40a9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc259c700 1 -- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40c3a40 msgr2=0x7f3cc40c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc259c700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40c3a40 0x7f3cc40c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc259c700 1 -- 172.21.15.45:0/3850479531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc416aca0 con 0x7f3cb0003c20 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc259c700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cc414dc00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40010e0 tx=0x7f3cb4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cba7fc700 1 -- 172.21.15.45:0/3850479531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400cc50 con 0x7f3cb0003c20 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cba7fc700 1 -- 172.21.15.45:0/3850479531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4001bf0 con 0x7f3cb0003c20 2023-10-06T05:22:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cba7fc700 1 -- 172.21.15.45:0/3850479531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40074f0 con 0x7f3cb0003c20 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc416af80 con 0x7f3cb0003c20 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.481+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc416b540 con 0x7f3cb0003c20 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.482+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc406f410 con 0x7f3cb0003c20 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.483+0000 7f3cba7fc700 1 -- 172.21.15.45:0/3850479531 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cb4019030 con 0x7f3cb0003c20 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.483+0000 7f3cba7fc700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.483+0000 7f3cba7fc700 1 -- 172.21.15.45:0/3850479531 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f3cb4085d20 con 0x7f3cb0003c20 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.484+0000 7f3cbbfff700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.484+0000 7f3cbbfff700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0010e0 tx=0x7f3cac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.488+0000 7f3cba7fc700 1 -- 172.21.15.45:0/3850479531 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb4052af0 con 0x7f3cb0003c20 2023-10-06T05:22:35.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.739+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cc40cb530 con 0x7f3cb0003c20 2023-10-06T05:22:35.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.740+0000 7f3cba7fc700 1 -- 172.21.15.45:0/3850479531 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+3842 (secure 0 0 0) 0x7f3cc40cb530 con 0x7f3cb0003c20 2023-10-06T05:22:35.741 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:35.742 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":109,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:34.886602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:35.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 msgr2=0x7f3ca40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0010e0 tx=0x7f3cac009040 comp rx=0 tx=0).stop 2023-10-06T05:22:35.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 msgr2=0x7f3cc414dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cc414dc00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40010e0 tx=0x7f3cb4015000 comp rx=0 tx=0).stop 2023-10-06T05:22:35.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 shutdown_connections 2023-10-06T05:22:35.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ca4066f30 0x7f3ca40693e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc40c3a40 0x7f3cc40c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb0003c20 0x7f3cc414dc00 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 --2- 172.21.15.45:0/3850479531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc40a95b0 0x7f3cc40a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 >> 172.21.15.45:0/3850479531 conn(0x7f3cc40cdf50 msgr2=0x7f3cc40b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 shutdown_connections 2023-10-06T05:22:35.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.742+0000 7f3cc8e66700 1 -- 172.21.15.45:0/3850479531 wait complete. 2023-10-06T05:22:35.745 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 109 2023-10-06T05:22:35.757 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:22:35.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.889+0000 7f2824f96700 1 Processor -- start 2023-10-06T05:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.889+0000 7f2824f96700 1 -- start start 2023-10-06T05:22:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.890+0000 7f2824f96700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28200c8ff0 0x7f28200c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.890+0000 7f2824f96700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2820157200 0x7f28200caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.890+0000 7f2824f96700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282015b000 con 0x7f28200caad0 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.890+0000 7f2824f96700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282015b2e0 con 0x7f2820157200 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.890+0000 7f2824f96700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282015b5c0 con 0x7f28200c8ff0 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281e59c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28200c8ff0 0x7f28200c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2820157200 0x7f28200caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281e59c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28200c8ff0 0x7f28200c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50300/0 (socket says 172.21.15.45:50300) 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281e59c700 1 -- 172.21.15.45:0/3714071869 learned_addr learned my addr 172.21.15.45:0/3714071869 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2820157200 0x7f28200caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54150/0 (socket says 172.21.15.45:54150) 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28200caad0 0x7f2820156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39482/0 (socket says 172.21.15.45:39482) 2023-10-06T05:22:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281ed9d700 1 -- 172.21.15.45:0/3714071869 >> v2:172.21.15.144:3300/0 conn(0x7f28200c8ff0 msgr2=0x7f28200c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281ed9d700 1 --2- 172.21.15.45:0/3714071869 >> v2:172.21.15.144:3300/0 conn(0x7f28200c8ff0 0x7f28200c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281ed9d700 1 -- 172.21.15.45:0/3714071869 >> v1:172.21.15.45:6789/0 conn(0x7f28200caad0 legacy=0x7f2820156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281ed9d700 1 -- 172.21.15.45:0/3714071869 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282015b8a0 con 0x7f2820157200 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281d59a700 1 -- 172.21.15.45:0/3714071869 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208720532 0 0) 0x7f282015b000 con 0x7f28200caad0 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.891+0000 7f281ed9d700 1 --2- 172.21.15.45:0/3714071869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200caf40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f28200cefc0 tx=0x7f2818009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66007e30bcb8fc64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f281d59a700 1 -- 172.21.15.45:0/3714071869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281800e070 con 0x7f2820157200 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f281d59a700 1 -- 172.21.15.45:0/3714071869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281800bb60 con 0x7f2820157200 2023-10-06T05:22:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f281d59a700 1 -- 172.21.15.45:0/3714071869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818012430 con 0x7f2820157200 2023-10-06T05:22:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 -- 172.21.15.45:0/3714071869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 msgr2=0x7f28200caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 --2- 172.21.15.45:0/3714071869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200caf40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f28200cefc0 tx=0x7f2818009780 comp rx=0 tx=0).stop 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 -- 172.21.15.45:0/3714071869 shutdown_connections 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 --2- 172.21.15.45:0/3714071869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200caf40 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 --2- 172.21.15.45:0/3714071869 >> v2:172.21.15.144:3300/0 conn(0x7f28200c8ff0 0x7f28200c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 -- 172.21.15.45:0/3714071869 >> 172.21.15.45:0/3714071869 conn(0x7f28200cdf50 msgr2=0x7f28200ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 -- 172.21.15.45:0/3714071869 shutdown_connections 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.892+0000 7f2824f96700 1 -- 172.21.15.45:0/3714071869 wait complete. 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.894+0000 7f2824f96700 1 Processor -- start 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.894+0000 7f2824f96700 1 -- start start 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f2824f96700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28200c8ff0 0x7f282015d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f2824f96700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28200caad0 0x7f28200af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f2824f96700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f2824f96700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28200af790 con 0x7f2820157200 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f2824f96700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28200afa30 con 0x7f28200c8ff0 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f2824f96700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28200afd10 con 0x7f28200caad0 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f281ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f281ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54152/0 (socket says 172.21.15.45:54152) 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f281ed9d700 1 -- 172.21.15.45:0/1974816730 learned_addr learned my addr 172.21.15.45:0/1974816730 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f281e59c700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28200c8ff0 0x7f282015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.895+0000 7f281dd9b700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28200caad0 0x7f28200af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f281ed9d700 1 -- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28200caad0 msgr2=0x7f28200af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f281ed9d700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28200caad0 0x7f28200af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f281ed9d700 1 -- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28200c8ff0 msgr2=0x7f282015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f281ed9d700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28200c8ff0 0x7f282015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f281ed9d700 1 -- 172.21.15.45:0/1974816730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2818009a20 con 0x7f2820157200 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f281dd9b700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28200caad0 0x7f28200af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f281ed9d700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200c5d40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f28180097c0 tx=0x7f2818019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f280e7fc700 1 -- 172.21.15.45:0/1974816730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818027470 con 0x7f2820157200 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f280e7fc700 1 -- 172.21.15.45:0/1974816730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2818027610 con 0x7f2820157200 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28200afff0 con 0x7f2820157200 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f280e7fc700 1 -- 172.21.15.45:0/1974816730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180279d0 con 0x7f2820157200 2023-10-06T05:22:35.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.896+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28200c6280 con 0x7f2820157200 2023-10-06T05:22:35.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.897+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28200b0510 con 0x7f2820157200 2023-10-06T05:22:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.898+0000 7f280e7fc700 1 -- 172.21.15.45:0/1974816730 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2818024070 con 0x7f2820157200 2023-10-06T05:22:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.898+0000 7f280e7fc700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2800066f30 0x7f28000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.898+0000 7f280e7fc700 1 -- 172.21.15.45:0/1974816730 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f2818087120 con 0x7f2820157200 2023-10-06T05:22:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.898+0000 7f281e59c700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2800066f30 0x7f28000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.899+0000 7f281e59c700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2800066f30 0x7f28000693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2810006fd0 tx=0x7f2810009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:35.903+0000 7f280e7fc700 1 -- 172.21.15.45:0/1974816730 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28200b0510 con 0x7f2820157200 2023-10-06T05:22:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.147+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28200b0510 con 0x7f2820157200 2023-10-06T05:22:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.148+0000 7f280e7fc700 1 -- 172.21.15.45:0/1974816730 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f28200b0510 con 0x7f2820157200 2023-10-06T05:22:36.150 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:36.150 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7992,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3860010011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3860010011},{"type":"v1","addr":"172.21.15.45:6833","nonce":3860010011}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:34.886602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2800066f30 msgr2=0x7f28000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2800066f30 0x7f28000693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2810006fd0 tx=0x7f2810009250 comp rx=0 tx=0).stop 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 msgr2=0x7f28200c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200c5d40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f28180097c0 tx=0x7f2818019000 comp rx=0 tx=0).stop 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 shutdown_connections 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2800066f30 0x7f28000693e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2820157200 0x7f28200c5d40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28200c8ff0 0x7f282015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 --2- 172.21.15.45:0/1974816730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28200caad0 0x7f28200af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.150+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 >> 172.21.15.45:0/1974816730 conn(0x7f28200cdf50 msgr2=0x7f2820152390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.151+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 shutdown_connections 2023-10-06T05:22:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.151+0000 7f2824f96700 1 -- 172.21.15.45:0/1974816730 wait complete. 2023-10-06T05:22:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 110 2023-10-06T05:22:36.165 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2023-10-06T05:22:36.166 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2023-10-06T05:22:36.166 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:36.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:36.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:36.166 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:36.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:22:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:22:36.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:36.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.361+0000 7f1842990700 1 Processor -- start 2023-10-06T05:22:36.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.361+0000 7f1842990700 1 -- start start 2023-10-06T05:22:36.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.362+0000 7f1842990700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f183c1595d0 0x7f183c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.362+0000 7f1842990700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c0c7d00 con 0x7f183c159200 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.362+0000 7f1842990700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183c06bb40 con 0x7f183c06b5c0 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.362+0000 7f1842990700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c0c7780 con 0x7f183c1595d0 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f183c159200 0x7f183c0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51180/0 (socket says 172.21.15.45:51180) 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183bfff700 1 -- 172.21.15.45:0/1655633698 learned_addr learned my addr 172.21.15.45:0/1655633698 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183b7fe700 1 --2- 172.21.15.45:0/1655633698 >> v2:172.21.15.144:3300/0 conn(0x7f183c1595d0 0x7f183c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594291138 0 0) 0x7f183c0c7d00 con 0x7f183c159200 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1828003570 con 0x7f183c159200 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008209158 0 0) 0x7f1828003570 con 0x7f183c159200 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 >> v2:172.21.15.144:3300/0 conn(0x7f183c1595d0 msgr2=0x7f183c06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 --2- 172.21.15.45:0/1655633698 >> v2:172.21.15.144:3300/0 conn(0x7f183c1595d0 0x7f183c06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 >> v1:172.21.15.144:6789/0 conn(0x7f183c06b5c0 legacy=0x7f183c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c06be20 con 0x7f183c159200 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f182c001470 con 0x7f183c159200 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 >> v1:172.21.15.45:6789/0 conn(0x7f183c159200 legacy=0x7f183c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1828003a60 0x7f1828003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1828004810 0x7f1828004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18280051c0 0x7f182800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0c7780 con 0x7f1828003a60 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182800c0d0 con 0x7f18280051c0 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182800c290 con 0x7f1828004810 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.363+0000 7f183bfff700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1828003a60 0x7f1828003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f1840f2d700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18280051c0 0x7f182800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183b7fe700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1828004810 0x7f1828004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183bfff700 1 -- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1828004810 msgr2=0x7f1828004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183bfff700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1828004810 0x7f1828004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183bfff700 1 -- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18280051c0 msgr2=0x7f182800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183bfff700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18280051c0 0x7f182800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183bfff700 1 -- 172.21.15.45:0/1655633698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828003570 con 0x7f1828003a60 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f1840f2d700 1 -- 172.21.15.45:0/1655633698 reap_dead start 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183bfff700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1828003a60 0x7f1828003e70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c390 tx=0x7f183c0c7c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325d329f3fd8f791 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00cc60 con 0x7f1828003a60 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00cdc0 con 0x7f1828003a60 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.364+0000 7f183affd700 1 -- 172.21.15.45:0/1655633698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c005b70 con 0x7f1828003a60 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.365+0000 7f1842990700 1 -- 172.21.15.45:0/1655633698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1828003a60 msgr2=0x7f1828003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.365+0000 7f1842990700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1828003a60 0x7f1828003e70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c390 tx=0x7f183c0c7c70 comp rx=0 tx=0).stop 2023-10-06T05:22:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.365+0000 7f1842990700 1 -- 172.21.15.45:0/1655633698 shutdown_connections 2023-10-06T05:22:36.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.365+0000 7f1842990700 1 --2- 172.21.15.45:0/1655633698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1828003a60 0x7f1828003e70 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.365+0000 7f1842990700 1 -- 172.21.15.45:0/1655633698 >> 172.21.15.45:0/1655633698 conn(0x7f183c06f090 msgr2=0x7f183c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.365+0000 7f1842990700 1 -- 172.21.15.45:0/1655633698 shutdown_connections 2023-10-06T05:22:36.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.365+0000 7f1842990700 1 -- 172.21.15.45:0/1655633698 wait complete. 2023-10-06T05:22:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.366+0000 7f1842990700 1 Processor -- start 2023-10-06T05:22:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.366+0000 7f1842990700 1 -- start start 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f1842990700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1828003a60 0x7f183c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f1842990700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f183c14dee0 0x7f183c14e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f1842990700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f183c16d530 0x7f183c16d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f1842990700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0b1a20 con 0x7f183c14dee0 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f1842990700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0b1c70 con 0x7f1828003a60 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f1842990700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0b1f50 con 0x7f183c16d530 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f183c14dee0 0x7f183c14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f183c14dee0 0x7f183c14e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42778/0 (socket says 172.21.15.45:42778) 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 -- 172.21.15.45:0/1127444142 learned_addr learned my addr 172.21.15.45:0/1127444142 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183bfff700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1828003a60 0x7f183c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f1840f2d700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f183c16d530 0x7f183c16d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 -- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f183c16d530 msgr2=0x7f183c16d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f183c16d530 0x7f183c16d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 -- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1828003a60 msgr2=0x7f183c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1828003a60 0x7f183c0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.367+0000 7f183b7fe700 1 -- 172.21.15.45:0/1127444142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c0b2230 con 0x7f183c14dee0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.368+0000 7f183b7fe700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f183c14dee0 0x7f183c14e350 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f1824011d40 tx=0x7f18240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.368+0000 7f1833fff700 1 -- 172.21.15.45:0/1127444142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400fa10 con 0x7f183c14dee0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.368+0000 7f1833fff700 1 -- 172.21.15.45:0/1127444142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182400fb70 con 0x7f183c14dee0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.368+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c0b2510 con 0x7f183c14dee0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.368+0000 7f1833fff700 1 -- 172.21.15.45:0/1127444142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824014440 con 0x7f183c14dee0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.368+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c16def0 con 0x7f183c14dee0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.369+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183c06e2f0 con 0x7f183c14dee0 2023-10-06T05:22:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.370+0000 7f1833fff700 1 -- 172.21.15.45:0/1127444142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1824014870 con 0x7f183c14dee0 2023-10-06T05:22:36.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.370+0000 7f1833fff700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f181c066f30 0x7f181c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.370+0000 7f1833fff700 1 -- 172.21.15.45:0/1127444142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f1824018030 con 0x7f183c14dee0 2023-10-06T05:22:36.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.370+0000 7f183bfff700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f181c066f30 0x7f181c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.371+0000 7f183bfff700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f181c066f30 0x7f181c0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f182c007cd0 tx=0x7f182c015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.374+0000 7f1833fff700 1 -- 172.21.15.45:0/1127444142 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18240520f0 con 0x7f183c14dee0 2023-10-06T05:22:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.625+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f183c0c7990 con 0x7f183c14dee0 2023-10-06T05:22:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.898+0000 7f1833fff700 1 -- 172.21.15.45:0/1127444142 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1824055710 con 0x7f183c14dee0 2023-10-06T05:22:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f181c066f30 msgr2=0x7f181c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f181c066f30 0x7f181c0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f182c007cd0 tx=0x7f182c015000 comp rx=0 tx=0).stop 2023-10-06T05:22:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f183c14dee0 msgr2=0x7f183c14e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f183c14dee0 0x7f183c14e350 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f1824011d40 tx=0x7f18240091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 shutdown_connections 2023-10-06T05:22:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f181c066f30 0x7f181c0693e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f183c16d530 0x7f183c16d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1828003a60 0x7f183c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 --2- 172.21.15.45:0/1127444142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f183c14dee0 0x7f183c14e350 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 >> 172.21.15.45:0/1127444142 conn(0x7f183c06f090 msgr2=0x7f183c15b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 shutdown_connections 2023-10-06T05:22:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:36.901+0000 7f1842990700 1 -- 172.21.15.45:0/1127444142 wait complete. 2023-10-06T05:22:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:36.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:22:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.053+0000 7fa3a7db6700 1 Processor -- start 2023-10-06T05:22:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.053+0000 7fa3a7db6700 1 -- start start 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a7db6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a0158f60 0x7fa3a0159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a7db6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3a00cc390 0x7fa3a00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a7db6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a00cb530 con 0x7fa3a006a060 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a7db6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a00ccd50 con 0x7fa3a00cc390 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a7db6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a0069bd0 con 0x7fa3a0158f60 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a6353700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3a00cc390 0x7fa3a00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a5351700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa3a006a060 0x7fa3a00cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51182/0 (socket says 172.21.15.45:51182) 2023-10-06T05:22:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.054+0000 7fa3a5351700 1 -- 172.21.15.45:0/609364136 learned_addr learned my addr 172.21.15.45:0/609364136 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a5b52700 1 --2- 172.21.15.45:0/609364136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a0158f60 0x7fa3a0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a4b50700 1 -- 172.21.15.45:0/609364136 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87712151 0 0) 0x7fa39c001110 con 0x7fa3a006a060 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a4b50700 1 -- 172.21.15.45:0/609364136 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa38c003570 con 0x7fa3a006a060 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a6353700 1 -- 172.21.15.45:0/609364136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a0158f60 msgr2=0x7fa3a0159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a6353700 1 --2- 172.21.15.45:0/609364136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a0158f60 0x7fa3a0159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a6353700 1 -- 172.21.15.45:0/609364136 >> v1:172.21.15.45:6789/0 conn(0x7fa3a006a060 legacy=0x7fa3a00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a6353700 1 -- 172.21.15.45:0/609364136 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a006a4d0 con 0x7fa3a00cc390 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a6353700 1 --2- 172.21.15.45:0/609364136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00cc810 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa398009e60 tx=0x7fa398009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd87d5950b92b171 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a4b50700 1 -- 172.21.15.45:0/609364136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39800e070 con 0x7fa3a00cc390 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a4b50700 1 -- 172.21.15.45:0/609364136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39800bb60 con 0x7fa3a00cc390 2023-10-06T05:22:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.055+0000 7fa3a4b50700 1 -- 172.21.15.45:0/609364136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398012430 con 0x7fa3a00cc390 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 -- 172.21.15.45:0/609364136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 msgr2=0x7fa3a00cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/609364136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00cc810 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa398009e60 tx=0x7fa398009780 comp rx=0 tx=0).stop 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 -- 172.21.15.45:0/609364136 shutdown_connections 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/609364136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00cc810 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/609364136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a0158f60 0x7fa3a0159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 -- 172.21.15.45:0/609364136 >> 172.21.15.45:0/609364136 conn(0x7fa3a00cdf50 msgr2=0x7fa3a00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 -- 172.21.15.45:0/609364136 shutdown_connections 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.056+0000 7fa3a7db6700 1 -- 172.21.15.45:0/609364136 wait complete. 2023-10-06T05:22:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.057+0000 7fa3a7db6700 1 Processor -- start 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.057+0000 7fa3a7db6700 1 -- start start 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a7db6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a006a060 0x7fa3a00b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a7db6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a7db6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0158f60 0x7fa3a015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a7db6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a00b24e0 con 0x7fa3a00cc390 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a7db6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a00b18d0 con 0x7fa3a0158f60 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a7db6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a00b1bb0 con 0x7fa3a006a060 2023-10-06T05:22:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42804/0 (socket says 172.21.15.45:42804) 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 -- 172.21.15.45:0/1059802207 learned_addr learned my addr 172.21.15.45:0/1059802207 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5b52700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a006a060 0x7fa3a00b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a6353700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0158f60 0x7fa3a015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 -- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a006a060 msgr2=0x7fa3a00b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a006a060 0x7fa3a00b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 -- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0158f60 msgr2=0x7fa3a015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0158f60 0x7fa3a015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5351700 1 -- 172.21.15.45:0/1059802207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa398009a20 con 0x7fa3a00cc390 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a6353700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0158f60 0x7fa3a015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.058+0000 7fa3a5b52700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a006a060 0x7fa3a00b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.059+0000 7fa3a5351700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00b2eb0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c7c0 tx=0x7fa39c007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.059+0000 7fa391ffb700 1 -- 172.21.15.45:0/1059802207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c005b30 con 0x7fa3a00cc390 2023-10-06T05:22:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.059+0000 7fa391ffb700 1 -- 172.21.15.45:0/1059802207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c005c90 con 0x7fa3a00cc390 2023-10-06T05:22:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.059+0000 7fa391ffb700 1 -- 172.21.15.45:0/1059802207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c007340 con 0x7fa3a00cc390 2023-10-06T05:22:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.059+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a00b1e90 con 0x7fa3a00cc390 2023-10-06T05:22:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.059+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a015c1d0 con 0x7fa3a00cc390 2023-10-06T05:22:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.060+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a006f410 con 0x7fa3a00cc390 2023-10-06T05:22:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.061+0000 7fa391ffb700 1 -- 172.21.15.45:0/1059802207 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa39c0076f0 con 0x7fa3a00cc390 2023-10-06T05:22:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.061+0000 7fa391ffb700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa384066f30 0x7fa3840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.061+0000 7fa391ffb700 1 -- 172.21.15.45:0/1059802207 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fa39c084250 con 0x7fa3a00cc390 2023-10-06T05:22:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.061+0000 7fa3a5b52700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa384066f30 0x7fa3840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.062+0000 7fa3a5b52700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa384066f30 0x7fa3840693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa394009990 tx=0x7fa394008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.066+0000 7fa391ffb700 1 -- 172.21.15.45:0/1059802207 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa39c051140 con 0x7fa3a00cc390 2023-10-06T05:22:37.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.354+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fa3a00cb530 con 0x7fa3a00cc390 2023-10-06T05:22:37.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:37.607 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:37.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.902+0000 7fa391ffb700 1 -- 172.21.15.45:0/1059802207 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v112) v1 ==== 101+0+0 (secure 0 0 0) 0x7fa3a00cb530 con 0x7fa3a00cc390 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.905+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa384066f30 msgr2=0x7fa3840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.905+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa384066f30 0x7fa3840693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa394009990 tx=0x7fa394008040 comp rx=0 tx=0).stop 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.905+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 msgr2=0x7fa3a00b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.905+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00b2eb0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c7c0 tx=0x7fa39c007c70 comp rx=0 tx=0).stop 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 shutdown_connections 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa384066f30 0x7fa3840693e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a0158f60 0x7fa3a015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a006a060 0x7fa3a00b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 --2- 172.21.15.45:0/1059802207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a00cc390 0x7fa3a00b2eb0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 >> 172.21.15.45:0/1059802207 conn(0x7fa3a00cdf50 msgr2=0x7fa3a0157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 shutdown_connections 2023-10-06T05:22:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.906+0000 7fa3a7db6700 1 -- 172.21.15.45:0/1059802207 wait complete. 2023-10-06T05:22:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:22:37.923 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:22:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.968+0000 7f258ee29040 1 Processor -- start 2023-10-06T05:22:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.968+0000 7f258ee29040 1 -- start start 2023-10-06T05:22:37.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f258ee29040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f258ee29040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c40f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f258ee29040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 0x5558c417e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f258ee29040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c3ff7980 con 0x5558c4180b70 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f258ee29040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c3ff7620 con 0x5558c40f79b0 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f258ee29040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558c3fee400 con 0x5558c4184f70 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c40f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c40f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42826/0 (socket says 172.21.15.45:42826) 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 -- 172.21.15.45:0/2552517817 learned_addr learned my addr 172.21.15.45:0/2552517817 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2588047700 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2588848700 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 0x5558c417e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 -- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 msgr2=0x5558c417e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 0x5558c417e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 -- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 msgr2=0x5558c40f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 -- 172.21.15.45:0/2552517817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c005cf0 con 0x5558c4180b70 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.969+0000 7f2587846700 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c40f4e30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f257c005a90 tx=0x7f257c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803a2f72d71b9dab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f2586844700 1 -- 172.21.15.45:0/2552517817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c000bc0 con 0x5558c4180b70 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f2586844700 1 -- 172.21.15.45:0/2552517817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c008970 con 0x5558c4180b70 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f2586844700 1 -- 172.21.15.45:0/2552517817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c01d420 con 0x5558c4180b70 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 -- 172.21.15.45:0/2552517817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 msgr2=0x5558c40f4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c40f4e30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f257c005a90 tx=0x7f257c016000 comp rx=0 tx=0).stop 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 -- 172.21.15.45:0/2552517817 shutdown_connections 2023-10-06T05:22:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 0x5558c417e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 --2- 172.21.15.45:0/2552517817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c40f4e30 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 -- 172.21.15.45:0/2552517817 >> 172.21.15.45:0/2552517817 conn(0x5558c40ae5f0 msgr2=0x5558c40b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 -- 172.21.15.45:0/2552517817 shutdown_connections 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.970+0000 7f258ee29040 1 -- 172.21.15.45:0/2552517817 wait complete. 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.971+0000 7f258ee29040 1 Processor -- start 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.971+0000 7f258ee29040 1 -- start start 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f258ee29040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f258ee29040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c417ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f258ee29040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 0x5558c40f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f258ee29040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c40b0be0 con 0x5558c4180b70 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f258ee29040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c417b180 con 0x5558c40f79b0 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f258ee29040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558c40b0940 con 0x5558c4184f70 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c417ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c417ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42834/0 (socket says 172.21.15.45:42834) 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 learned_addr learned my addr 172.21.15.45:0/2601009260 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2588848700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 0x5558c40f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 msgr2=0x5558c40f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5558c4184f70 0x5558c40f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 msgr2=0x5558c40b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c40201c0 con 0x5558c4180b70 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558c40f79b0 0x5558c40b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2587846700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c417ac40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f257c0030c0 tx=0x7f257c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2585041700 1 -- 172.21.15.45:0/2601009260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c01d420 con 0x5558c4180b70 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f2585041700 1 -- 172.21.15.45:0/2601009260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c008260 con 0x5558c4180b70 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.972+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5558c40063d0 con 0x5558c4180b70 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.973+0000 7f2585041700 1 -- 172.21.15.45:0/2601009260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c026450 con 0x5558c4180b70 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.973+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558c3ff9ae0 con 0x5558c4180b70 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.974+0000 7f2585041700 1 -- 172.21.15.45:0/2601009260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f257c0267c0 con 0x5558c4180b70 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.974+0000 7f2585041700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2574065890 0x7f2574067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.974+0000 7f2585041700 1 -- 172.21.15.45:0/2601009260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f257c0868a0 con 0x5558c4180b70 2023-10-06T05:22:37.982 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:22:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.975+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5558c40f3c30 0x5558c4191910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.975+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c417a680 con 0x5558c40f3c30 2023-10-06T05:22:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.975+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2574065890 0x7f2574067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.975+0000 7f2588848700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5558c40f3c30 0x5558c4191910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.975+0000 7f2588848700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5558c40f3c30 0x5558c4191910 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.975+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2574065890 0x7f2574067d40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2578005fd0 tx=0x7f257800c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.977+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v96'102 uv78 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.977+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c4191e50 con 0x5558c40f3c30 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.978+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v96'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.978+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41922c0 con 0x5558c40f3c30 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.981+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v96'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:37.984 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:22:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.981+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419eef0 con 0x5558c40f3c30 2023-10-06T05:22:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.984+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v96'105 uv105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.984+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f2a0 con 0x5558c40f3c30 2023-10-06T05:22:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.987+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v96'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.987+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419dee0 con 0x5558c40f3c30 2023-10-06T05:22:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.990+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v96'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.990+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419e2e0 con 0x5558c40f3c30 2023-10-06T05:22:37.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.992+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v96'108 uv84 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:37.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.992+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419e6e0 con 0x5558c40f3c30 2023-10-06T05:22:38.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.998+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v96'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.998+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419eae0 con 0x5558c40f3c30 2023-10-06T05:22:38.002 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:22:38.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.999+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v96'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:37.999+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419eae0 con 0x5558c40f3c30 2023-10-06T05:22:38.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.002+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v96'111 uv111 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.002+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.008+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v96'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.008+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.009+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v96'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.009+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.012+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v96'114 uv91 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.012+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.017+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v96'115 uv115 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.017+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.020 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:22:38.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.018+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v96'116 uv116 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.019+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.021+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v96'117 uv117 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.021+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.024+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v96'118 uv118 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.024+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419ea90 con 0x5558c40f3c30 2023-10-06T05:22:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.026+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v96'119 uv119 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.027+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.029+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v96'120 uv120 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.029+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.031+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v96'121 uv95 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.031+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.032+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v96'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.032+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.036 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:22:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.033+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v96'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.033+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a0660 con 0x5558c40f3c30 2023-10-06T05:22:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.035+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v96'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.035+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a0660 con 0x5558c40f3c30 2023-10-06T05:22:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.036+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v96'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.036+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.037+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v96'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.037+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.038+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v96'127 uv101 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.038+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.041+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v96'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.041+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.044 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:22:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.043+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v96'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.043+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a0660 con 0x5558c40f3c30 2023-10-06T05:22:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.044+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v96'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.044+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a0660 con 0x5558c40f3c30 2023-10-06T05:22:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.045+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v96'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.045+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c419f650 con 0x5558c40f3c30 2023-10-06T05:22:38.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.046+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v96'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c40f3c30 2023-10-06T05:22:38.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.047+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5558c419fb60 0x5558c41a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.047+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a2480 con 0x5558c419fb60 2023-10-06T05:22:38.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.047+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5558c419fb60 0x5558c41a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.047+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5558c419fb60 0x5558c41a1f40 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.049+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v96'88 uv88 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.051 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:22:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.049+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a2d20 con 0x5558c419fb60 2023-10-06T05:22:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.051+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [write 0~554] v96'89 uv89 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.051+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a30d0 con 0x5558c419fb60 2023-10-06T05:22:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.052+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v96'90 uv90 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.052+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a3480 con 0x5558c419fb60 2023-10-06T05:22:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.055+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 200.00000000 [create] v96'91 uv75 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.055+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a3830 con 0x5558c419fb60 2023-10-06T05:22:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.060+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 200.00000000 [delete] v96'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.060+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a3be0 con 0x5558c419fb60 2023-10-06T05:22:38.063 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:22:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.061+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v96'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.061+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a3f90 con 0x5558c419fb60 2023-10-06T05:22:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.063+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v96'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.063+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.069+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v96'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.069+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.070+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000001 [create] v96'96 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.070+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.072+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000001 [delete] v96'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.072+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.077 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:22:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.075+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v96'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.075+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a7a60 con 0x5558c419fb60 2023-10-06T05:22:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.078+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [write 0~14075] v96'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.078+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a7a60 con 0x5558c419fb60 2023-10-06T05:22:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.083+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v96'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.083+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.085+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 606.00000000 [create] v96'101 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.085+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.087+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 606.00000000 [delete] v96'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.087+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.090 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:22:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.089+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v96'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.089+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a5350 con 0x5558c419fb60 2023-10-06T05:22:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.090+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v96'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.090+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a5350 con 0x5558c419fb60 2023-10-06T05:22:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.091+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v96'105 uv105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.091+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c419fb60 2023-10-06T05:22:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.093+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v96'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.093+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5558c41a4c50 0x5558c41a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.093+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:51 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a4340 con 0x5558c41a4c50 2023-10-06T05:22:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.093+0000 7f2587846700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5558c41a4c50 0x5558c41a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.094+0000 7f2587846700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5558c41a4c50 0x5558c41a5020 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.095+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(51 607.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.095+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:52 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a77b0 con 0x5558c41a4c50 2023-10-06T05:22:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.096+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(52 607.00000000 [delete] v96'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.096+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a7c20 con 0x5558c41a4c50 2023-10-06T05:22:38.099 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:22:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.097+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(53 607.00000000 [create] v96'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.097+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a9520 con 0x5558c41a4c50 2023-10-06T05:22:38.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.099+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.099+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a98d0 con 0x5558c41a4c50 2023-10-06T05:22:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.100+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.100+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8510 con 0x5558c41a4c50 2023-10-06T05:22:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.101+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.101+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8910 con 0x5558c41a4c50 2023-10-06T05:22:38.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.102+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(57 mds0_openfiles.0 [create] v96'16 uv15 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.102+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8d60 con 0x5558c41a4c50 2023-10-06T05:22:38.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.103+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v96'17 uv17 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.103+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8d60 con 0x5558c41a4c50 2023-10-06T05:22:38.106 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:22:38.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.105+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(59 mds0_openfiles.0 [create] v96'18 uv18 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.105+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a9160 con 0x5558c41a4c50 2023-10-06T05:22:38.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.106+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v96'19 uv19 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.106+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:61 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8cc0 con 0x5558c419fb60 2023-10-06T05:22:38.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.107+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(61 608.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.107+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:62 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8cc0 con 0x5558c419fb60 2023-10-06T05:22:38.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.109+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(62 608.00000000 [delete] v96'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.109+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8cc0 con 0x5558c419fb60 2023-10-06T05:22:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:22:38.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.110+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v96'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.110+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8cc0 con 0x5558c419fb60 2023-10-06T05:22:38.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.111+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.111+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8cc0 con 0x5558c419fb60 2023-10-06T05:22:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.113+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.113+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41a8cc0 con 0x5558c419fb60 2023-10-06T05:22:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.114+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c419fb60 2023-10-06T05:22:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.114+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5558c41aa190 0x5558c41ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.114+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:67 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.115+0000 7f2588848700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5558c41aa190 0x5558c41ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.115+0000 7f2588848700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5558c41aa190 0x5558c41ac570 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.116+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(67 604.00000000 [create] v96'37 uv31 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.116+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:68 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.118+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(68 604.00000000 [delete] v96'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.118+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:22:38.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.119+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(69 604.00000000 [create] v96'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.119+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41adac0 con 0x5558c41aa190 2023-10-06T05:22:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.120+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v96'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.120+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41adac0 con 0x5558c41aa190 2023-10-06T05:22:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.121+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v96'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.121+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.123+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v96'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.123+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.124+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(73 500.00000000 [create] v96'43 uv36 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.124+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.125+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(74 500.00000000 [delete] v96'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.125+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.126+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(75 500.00000000 [create] v96'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.126+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41acab0 con 0x5558c41aa190 2023-10-06T05:22:38.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.128+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v96'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.128+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41ace80 con 0x5558c41aa190 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.129+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v96'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.130+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5558c41ad390 0x5558c41ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.130+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:78 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41afea0 con 0x5558c41ad390 2023-10-06T05:22:38.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.130+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5558c41ad390 0x5558c41ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.130+0000 7f2588047700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5558c41ad390 0x5558c41ad7c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.131+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(78 mds_snaptable [create] v96'17 uv16 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c41ad390 2023-10-06T05:22:38.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.131+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:79 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41afea0 con 0x5558c41ad390 2023-10-06T05:22:38.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.132+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(79 mds_snaptable [delete] v96'18 uv18 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c41ad390 2023-10-06T05:22:38.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.132+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41afea0 con 0x5558c41ad390 2023-10-06T05:22:38.135 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:22:38.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.134+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(80 mds_snaptable [create] v96'19 uv19 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c41ad390 2023-10-06T05:22:38.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.134+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b02a0 con 0x5558c41ad390 2023-10-06T05:22:38.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.135+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v96'20 uv20 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c41ad390 2023-10-06T05:22:38.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.135+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b02a0 con 0x5558c41ad390 2023-10-06T05:22:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.137+0000 7f2588047700 1 -- 172.21.15.45:0/2601009260 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v96'21 uv21 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2578008c80 con 0x5558c41ad390 2023-10-06T05:22:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.137+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:83 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b02a0 con 0x5558c41a4c50 2023-10-06T05:22:38.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.138+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(83 605.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.138+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:84 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b02a0 con 0x5558c41a4c50 2023-10-06T05:22:38.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.139+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(84 605.00000000 [delete] v96'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.139+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b02a0 con 0x5558c41a4c50 2023-10-06T05:22:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.140+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(85 605.00000000 [create] v96'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.142 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:22:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.140+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b12b0 con 0x5558c41a4c50 2023-10-06T05:22:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.142+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.142+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b12b0 con 0x5558c41a4c50 2023-10-06T05:22:38.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.143+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.143+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b02a0 con 0x5558c41a4c50 2023-10-06T05:22:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.144+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.144+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5558c41b07b0 0x5558c41b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.144+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:89 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3240 con 0x5558c41b07b0 2023-10-06T05:22:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.144+0000 7f2587846700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5558c41b07b0 0x5558c41b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.145+0000 7f2587846700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5558c41b07b0 0x5558c41b2b90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.146+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(89 mds0_inotable [create] v96'34 uv28 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.146+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:90 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.147+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(90 mds0_inotable [delete] v96'35 uv35 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.147+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.148+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(91 mds0_inotable [create] v96'36 uv36 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.150 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:22:38.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.148+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.149+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(92 mds0_inotable [write 0~34] v96'37 uv37 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.149+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.150+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v96'38 uv38 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.151+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.151+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(94 100.00000000 [create] v96'39 uv33 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.151+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.152+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(95 100.00000000 [delete] v96'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.152+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:22:38.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.153+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(96 100.00000000 [create] v96'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.154+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3640 con 0x5558c41b07b0 2023-10-06T05:22:38.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.155+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(97 100.00000000 [omap-set-header] v96'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.155+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b63b0 con 0x5558c41b07b0 2023-10-06T05:22:38.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.156+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v96'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41b07b0 2023-10-06T05:22:38.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.156+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.157+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v96'18 uv17 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.157+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.158+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v96'19 uv19 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.158+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.159+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v96'20 uv20 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:22:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.159+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.160+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v96'21 uv21 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.160+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5558c41b4440 0x5558c41b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.160+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41b4440 2023-10-06T05:22:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.161+0000 7f2588848700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5558c41b4440 0x5558c41b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.161+0000 7f2588848700 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5558c41b4440 0x5558c41b6b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.162+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41b4440 2023-10-06T05:22:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.162+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41b4440 2023-10-06T05:22:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.164+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v96'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41b4440 2023-10-06T05:22:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.164+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41b4440 2023-10-06T05:22:38.167 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:22:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.165+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v96'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41b4440 2023-10-06T05:22:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.165+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41b4440 2023-10-06T05:22:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.168+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41b4440 2023-10-06T05:22:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.168+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41b4440 2023-10-06T05:22:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.169+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41b4440 2023-10-06T05:22:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.169+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41b4440 2023-10-06T05:22:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.172+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41b4440 2023-10-06T05:22:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.172+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41aa190 2023-10-06T05:22:38.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.173+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v96'17 uv16 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.173+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41aa190 2023-10-06T05:22:38.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.175+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v96'18 uv18 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.175+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41aa190 2023-10-06T05:22:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.176+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v96'19 uv19 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.178 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:22:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.176+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41aa190 2023-10-06T05:22:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.178+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v96'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.178+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41aa190 2023-10-06T05:22:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.179+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v96'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2580003660 con 0x5558c41aa190 2023-10-06T05:22:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.179+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.180+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v96'45 uv37 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.180+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.181+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v96'46 uv46 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.181+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.182+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v96'47 uv47 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.184 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:22:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.182+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.184+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v96'48 uv48 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.184+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b7090 con 0x5558c41a4c50 2023-10-06T05:22:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.185+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v96'49 uv49 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.185+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b7040 con 0x5558c41a4c50 2023-10-06T05:22:38.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.186+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v96'50 uv44 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.186+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.187+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v96'51 uv51 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.187+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.188+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v96'52 uv52 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:22:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.188+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.189+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v96'53 uv53 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.189+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3b30 con 0x5558c41a4c50 2023-10-06T05:22:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.190+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v96'54 uv54 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.190+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b7040 con 0x5558c41a4c50 2023-10-06T05:22:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.191+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v96'55 uv55 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.191+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) v8 -- 0x5558c41b3a40 con 0x5558c41a4c50 2023-10-06T05:22:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f2587846700 1 -- 172.21.15.45:0/2601009260 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v96'56 uv56 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f257c055770 con 0x5558c41a4c50 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5558c41a4c50 msgr2=0x5558c41a5020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5558c41a4c50 0x5558c41a5020 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5558c40f3c30 msgr2=0x5558c4191910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5558c40f3c30 0x5558c4191910 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5558c41b07b0 msgr2=0x5558c41b2b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5558c41b07b0 0x5558c41b2b90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5558c419fb60 msgr2=0x5558c41a1f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5558c419fb60 0x5558c41a1f40 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5558c41b4440 msgr2=0x5558c41b6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5558c41b4440 0x5558c41b6b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5558c41ad390 msgr2=0x5558c41ad7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5558c41ad390 0x5558c41ad7c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5558c41aa190 msgr2=0x5558c41ac570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 reap_dead start 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5558c41aa190 0x5558c41ac570 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 reap_dead start 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 reap_dead start 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f2588848700 1 -- 172.21.15.45:0/2601009260 reap_dead start 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2574065890 msgr2=0x7f2574067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2574065890 0x7f2574067d40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2578005fd0 tx=0x7f257800c040 comp rx=0 tx=0).stop 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 msgr2=0x5558c417ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c417ac40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f257c0030c0 tx=0x7f257c01e000 comp rx=0 tx=0).stop 2023-10-06T05:22:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 shutdown_connections 2023-10-06T05:22:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5558c41aa190 0x5558c41ac570 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558c4180b70 0x5558c417ac40 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 --2- 172.21.15.45:0/2601009260 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2574065890 0x7f2574067d40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 >> 172.21.15.45:0/2601009260 conn(0x5558c40ae5f0 msgr2=0x5558c40aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 shutdown_connections 2023-10-06T05:22:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.192+0000 7f258ee29040 1 -- 172.21.15.45:0/2601009260 wait complete. 2023-10-06T05:22:38.198 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2023-10-06T05:22:38.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.245+0000 7fd43b212040 1 Processor -- start 2023-10-06T05:22:38.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.245+0000 7fd43b212040 1 -- start start 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd43b212040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b29b70 0x5614c9b28df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd43b212040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2af50 0x5614c9b33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd43b212040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b2da30 0x5614c9b23d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd43b212040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c99a0980 con 0x5614c9b29b70 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd43b212040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c99a0620 con 0x5614c9b2af50 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd43b212040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614c9997400 con 0x5614c9b2da30 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434430700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b29b70 0x5614c9b28df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434430700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b29b70 0x5614c9b28df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42848/0 (socket says 172.21.15.45:42848) 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434430700 1 -- 172.21.15.45:0/796674555 learned_addr learned my addr 172.21.15.45:0/796674555 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434c31700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b2da30 0x5614c9b23d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd433c2f700 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2af50 0x5614c9b33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434430700 1 -- 172.21.15.45:0/796674555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b2da30 msgr2=0x5614c9b23d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434430700 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b2da30 0x5614c9b23d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434430700 1 -- 172.21.15.45:0/796674555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2af50 msgr2=0x5614c9b33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.246+0000 7fd434430700 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2af50 0x5614c9b33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd434430700 1 -- 172.21.15.45:0/796674555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428005db0 con 0x5614c9b29b70 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd434430700 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b29b70 0x5614c9b28df0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd424001e00 tx=0x7fd424001850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58f6545dc49e641 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd432c2d700 1 -- 172.21.15.45:0/796674555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424014070 con 0x5614c9b29b70 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd432c2d700 1 -- 172.21.15.45:0/796674555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd424008030 con 0x5614c9b29b70 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd432c2d700 1 -- 172.21.15.45:0/796674555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400bcb0 con 0x5614c9b29b70 2023-10-06T05:22:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd43b212040 1 -- 172.21.15.45:0/796674555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b29b70 msgr2=0x5614c9b28df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd43b212040 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b29b70 0x5614c9b28df0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd424001e00 tx=0x7fd424001850 comp rx=0 tx=0).stop 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd43b212040 1 -- 172.21.15.45:0/796674555 shutdown_connections 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd43b212040 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b2da30 0x5614c9b23d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd43b212040 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b29b70 0x5614c9b28df0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd43b212040 1 --2- 172.21.15.45:0/796674555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2af50 0x5614c9b33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.247+0000 7fd43b212040 1 -- 172.21.15.45:0/796674555 >> 172.21.15.45:0/796674555 conn(0x5614c9a575f0 msgr2=0x5614c9a81d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.248+0000 7fd43b212040 1 -- 172.21.15.45:0/796674555 shutdown_connections 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.248+0000 7fd43b212040 1 -- 172.21.15.45:0/796674555 wait complete. 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.248+0000 7fd43b212040 1 Processor -- start 2023-10-06T05:22:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.248+0000 7fd43b212040 1 -- start start 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd43b212040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b29b70 0x5614c9b28490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd43b212040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b2af50 0x5614c9b21a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd43b212040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2da30 0x5614c9b32240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd43b212040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c9b32d50 con 0x5614c9b2af50 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd43b212040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c9b27ab0 con 0x5614c9b2da30 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd43b212040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614c9b27d50 con 0x5614c9b29b70 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd433c2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b2af50 0x5614c9b21a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd434430700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b29b70 0x5614c9b28490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd433c2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b2af50 0x5614c9b21a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42852/0 (socket says 172.21.15.45:42852) 2023-10-06T05:22:38.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd433c2f700 1 -- 172.21.15.45:0/979782752 learned_addr learned my addr 172.21.15.45:0/979782752 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.249+0000 7fd434c31700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2da30 0x5614c9b32240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd433c2f700 1 -- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b29b70 msgr2=0x5614c9b28490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd433c2f700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b29b70 0x5614c9b28490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd433c2f700 1 -- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2da30 msgr2=0x5614c9b32240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd433c2f700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614c9b2da30 0x5614c9b32240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd433c2f700 1 -- 172.21.15.45:0/979782752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42400a040 con 0x5614c9b2af50 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd434430700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614c9b29b70 0x5614c9b28490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd433c2f700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b2af50 0x5614c9b21a80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd428008fd0 tx=0x7fd428016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd43142a700 1 -- 172.21.15.45:0/979782752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428021070 con 0x5614c9b2af50 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd43142a700 1 -- 172.21.15.45:0/979782752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd428025430 con 0x5614c9b2af50 2023-10-06T05:22:38.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd43142a700 1 -- 172.21.15.45:0/979782752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4280257f0 con 0x5614c9b2af50 2023-10-06T05:22:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5614c99af3d0 con 0x5614c9b2af50 2023-10-06T05:22:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.250+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614c99a2ae0 con 0x5614c9b2af50 2023-10-06T05:22:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.252+0000 7fd43142a700 1 -- 172.21.15.45:0/979782752 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd42801c410 con 0x5614c9b2af50 2023-10-06T05:22:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.252+0000 7fd43142a700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd4200658c0 0x7fd420067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.252+0000 7fd43142a700 1 -- 172.21.15.45:0/979782752 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fd428084be0 con 0x5614c9b2af50 2023-10-06T05:22:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.252+0000 7fd43b212040 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614c9a58550 0x5614c9b251f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.252+0000 7fd434430700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd4200658c0 0x7fd420067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.252+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e96) v8 -- 0x5614c9b25730 con 0x5614c9a58550 2023-10-06T05:22:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.253+0000 7fd434c31700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614c9a58550 0x5614c9b251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.253+0000 7fd434430700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd4200658c0 0x7fd420067d70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd4240018b0 tx=0x7fd424016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.253+0000 7fd434c31700 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614c9a58550 0x5614c9b251f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.254+0000 7fd434c31700 1 -- 172.21.15.45:0/979782752 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000000 [delete] v96'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd42c002d90 con 0x5614c9a58550 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.254+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614c9a58550 msgr2=0x5614c9b251f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.254+0000 7fd43b212040 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614c9a58550 0x5614c9b251f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd4200658c0 msgr2=0x7fd420067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd4200658c0 0x7fd420067d70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd4240018b0 tx=0x7fd424016000 comp rx=0 tx=0).stop 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b2af50 msgr2=0x5614c9b21a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 --2- 172.21.15.45:0/979782752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614c9b2af50 0x5614c9b21a80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd428008fd0 tx=0x7fd428016000 comp rx=0 tx=0).stop 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd434c31700 1 -- 172.21.15.45:0/979782752 reap_dead start 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 shutdown_connections 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 >> 172.21.15.45:0/979782752 conn(0x5614c9a575f0 msgr2=0x5614c9b35b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 shutdown_connections 2023-10-06T05:22:38.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.255+0000 7fd43b212040 1 -- 172.21.15.45:0/979782752 wait complete. 2023-10-06T05:22:38.262 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:22:38.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.400+0000 7f283f584700 1 Processor -- start 2023-10-06T05:22:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.400+0000 7f283f584700 1 -- start start 2023-10-06T05:22:38.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283f584700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28380ccb70 0x7f28380aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283f584700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28380c8b70 0x7f28380c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283f584700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28380c3b50 con 0x7f2838157330 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283f584700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2838157700 con 0x7f28380ccb70 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283f584700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28380c98e0 con 0x7f28380c8b70 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283cb1f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28380ccb70 0x7f28380aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283d320700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2838157330 0x7f28380ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51186/0 (socket says 172.21.15.45:51186) 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283cb1f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28380ccb70 0x7f28380aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42862/0 (socket says 172.21.15.45:42862) 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283d320700 1 -- 172.21.15.45:0/2903087758 learned_addr learned my addr 172.21.15.45:0/2903087758 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283db21700 1 --2- 172.21.15.45:0/2903087758 >> v2:172.21.15.144:3300/0 conn(0x7f28380c8b70 0x7f28380c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283cb1f700 1 -- 172.21.15.45:0/2903087758 >> v2:172.21.15.144:3300/0 conn(0x7f28380c8b70 msgr2=0x7f28380c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283cb1f700 1 --2- 172.21.15.45:0/2903087758 >> v2:172.21.15.144:3300/0 conn(0x7f28380c8b70 0x7f28380c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283cb1f700 1 -- 172.21.15.45:0/2903087758 >> v1:172.21.15.45:6789/0 conn(0x7f2838157330 legacy=0x7f28380ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283cb1f700 1 -- 172.21.15.45:0/2903087758 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28380c9bc0 con 0x7f28380ccb70 2023-10-06T05:22:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f2837fff700 1 -- 172.21.15.45:0/2903087758 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362921346 0 0) 0x7f28380c3b50 con 0x7f2838157330 2023-10-06T05:22:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.401+0000 7f283cb1f700 1 --2- 172.21.15.45:0/2903087758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380ccb70 0x7f28380aaae0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f28380cdf40 tx=0x7f282c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b006d3de607a174 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f2837fff700 1 -- 172.21.15.45:0/2903087758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c011630 con 0x7f28380ccb70 2023-10-06T05:22:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f2837fff700 1 -- 172.21.15.45:0/2903087758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c011790 con 0x7f28380ccb70 2023-10-06T05:22:38.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f2837fff700 1 -- 172.21.15.45:0/2903087758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c011b30 con 0x7f28380ccb70 2023-10-06T05:22:38.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 -- 172.21.15.45:0/2903087758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380ccb70 msgr2=0x7f28380aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 --2- 172.21.15.45:0/2903087758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380ccb70 0x7f28380aaae0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f28380cdf40 tx=0x7f282c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 -- 172.21.15.45:0/2903087758 shutdown_connections 2023-10-06T05:22:38.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 --2- 172.21.15.45:0/2903087758 >> v2:172.21.15.144:3300/0 conn(0x7f28380c8b70 0x7f28380c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 --2- 172.21.15.45:0/2903087758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380ccb70 0x7f28380aaae0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 -- 172.21.15.45:0/2903087758 >> 172.21.15.45:0/2903087758 conn(0x7f2838067e00 msgr2=0x7f28380cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 -- 172.21.15.45:0/2903087758 shutdown_connections 2023-10-06T05:22:38.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.402+0000 7f283f584700 1 -- 172.21.15.45:0/2903087758 wait complete. 2023-10-06T05:22:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.403+0000 7f283f584700 1 Processor -- start 2023-10-06T05:22:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.404+0000 7f283f584700 1 -- start start 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.404+0000 7f283f584700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380c8b70 0x7f283816c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.404+0000 7f283f584700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28380ccb70 0x7f28380b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283f584700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2838157330 0x7f283806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283f584700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283816a8a0 con 0x7f28380c8b70 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283f584700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380b1010 con 0x7f28380ccb70 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283f584700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2838169d20 con 0x7f2838157330 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380c8b70 0x7f283816c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380c8b70 0x7f283816c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42866/0 (socket says 172.21.15.45:42866) 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 -- 172.21.15.45:0/862303197 learned_addr learned my addr 172.21.15.45:0/862303197 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283db21700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2838157330 0x7f283806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283cb1f700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28380ccb70 0x7f28380b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 -- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2838157330 msgr2=0x7f283806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2838157330 0x7f283806d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 -- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28380ccb70 msgr2=0x7f28380b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28380ccb70 0x7f28380b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 -- 172.21.15.45:0/862303197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c009420 con 0x7f28380c8b70 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283d320700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380c8b70 0x7f283816c800 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f282800cb60 tx=0x7f28280017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f28357fa700 1 -- 172.21.15.45:0/862303197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828003120 con 0x7f28380c8b70 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f28357fa700 1 -- 172.21.15.45:0/862303197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2828005d80 con 0x7f28380c8b70 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283816a000 con 0x7f28380c8b70 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f28357fa700 1 -- 172.21.15.45:0/862303197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800a800 con 0x7f28380c8b70 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.405+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283816a5c0 con 0x7f28380c8b70 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.406+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283816ab30 con 0x7f28380c8b70 2023-10-06T05:22:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.407+0000 7f28357fa700 1 -- 172.21.15.45:0/862303197 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f282800abf0 con 0x7f28380c8b70 2023-10-06T05:22:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.407+0000 7f28357fa700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2824066f30 0x7f28240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.408+0000 7f28357fa700 1 -- 172.21.15.45:0/862303197 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f2828084160 con 0x7f28380c8b70 2023-10-06T05:22:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.408+0000 7f283cb1f700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2824066f30 0x7f28240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.408+0000 7f283cb1f700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2824066f30 0x7f28240693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f282c0093f0 tx=0x7f282c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.412+0000 7f28357fa700 1 -- 172.21.15.45:0/862303197 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f283816ab30 con 0x7f28380c8b70 2023-10-06T05:22:38.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.662+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f283816ab30 con 0x7f28380c8b70 2023-10-06T05:22:38.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.906+0000 7f28357fa700 1 -- 172.21.15.45:0/862303197 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v113) v1 ==== 162+0+0 (secure 0 0 0) 0x7f283816ab30 con 0x7f28380c8b70 2023-10-06T05:22:38.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2824066f30 msgr2=0x7f28240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2824066f30 0x7f28240693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f282c0093f0 tx=0x7f282c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380c8b70 msgr2=0x7f283816c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380c8b70 0x7f283816c800 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f282800cb60 tx=0x7f28280017b0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 shutdown_connections 2023-10-06T05:22:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2824066f30 0x7f28240693e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2838157330 0x7f283806d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28380c8b70 0x7f283816c800 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 --2- 172.21.15.45:0/862303197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28380ccb70 0x7f28380b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 >> 172.21.15.45:0/862303197 conn(0x7f2838067e00 msgr2=0x7f28380cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 shutdown_connections 2023-10-06T05:22:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:38.910+0000 7f283f584700 1 -- 172.21.15.45:0/862303197 wait complete. 2023-10-06T05:22:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:22:38.916 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:22:38.913+0000 7f79521f0700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2023-10-06T05:22:38.917 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:38.914+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:38.917 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:38.914+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:22:38.921 INFO:tasks.ceph.mds.a.smithi045.stderr: -15> 2023-10-06T05:22:38.913+0000 7f79521f0700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2023-10-06T05:22:38.930 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:38.944 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:22:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.065+0000 7f28c226a700 1 Processor -- start 2023-10-06T05:22:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.065+0000 7f28c226a700 1 -- start start 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.065+0000 7f28c226a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28bc155cd0 0x7f28bc15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.065+0000 7f28c226a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc0ccdc0 0x7f28bc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28c226a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc0cd740 con 0x7f28bc1560a0 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28c226a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc0ae6b0 con 0x7f28bc155cd0 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28c226a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0ae8d0 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bbfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28bc155cd0 0x7f28bc15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28bc1560a0 0x7f28bc0adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51190/0 (socket says 172.21.15.45:51190) 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bb7fe700 1 -- 172.21.15.45:0/1986451748 learned_addr learned my addr 172.21.15.45:0/1986451748 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28c0807700 1 --2- 172.21.15.45:0/1986451748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc0ccdc0 0x7f28bc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bbfff700 1 -- 172.21.15.45:0/1986451748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc0ccdc0 msgr2=0x7f28bc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bbfff700 1 --2- 172.21.15.45:0/1986451748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc0ccdc0 0x7f28bc0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bbfff700 1 -- 172.21.15.45:0/1986451748 >> v1:172.21.15.45:6789/0 conn(0x7f28bc1560a0 legacy=0x7f28bc0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bbfff700 1 -- 172.21.15.45:0/1986451748 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc0aebb0 con 0x7f28bc155cd0 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28baffd700 1 -- 172.21.15.45:0/1986451748 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885071591 0 0) 0x7f28bc0cd740 con 0x7f28bc1560a0 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28bbfff700 1 --2- 172.21.15.45:0/1986451748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc155cd0 0x7f28bc15a730 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0098e0 tx=0x7f28ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a303f22e243e4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28baffd700 1 -- 172.21.15.45:0/1986451748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00b640 con 0x7f28bc155cd0 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28baffd700 1 -- 172.21.15.45:0/1986451748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac00b7a0 con 0x7f28bc155cd0 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.066+0000 7f28baffd700 1 -- 172.21.15.45:0/1986451748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00bb40 con 0x7f28bc155cd0 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 -- 172.21.15.45:0/1986451748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc155cd0 msgr2=0x7f28bc15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 --2- 172.21.15.45:0/1986451748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc155cd0 0x7f28bc15a730 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0098e0 tx=0x7f28ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 -- 172.21.15.45:0/1986451748 shutdown_connections 2023-10-06T05:22:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 --2- 172.21.15.45:0/1986451748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc0ccdc0 0x7f28bc0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 --2- 172.21.15.45:0/1986451748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc155cd0 0x7f28bc15a730 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 -- 172.21.15.45:0/1986451748 >> 172.21.15.45:0/1986451748 conn(0x7f28bc06f090 msgr2=0x7f28bc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 -- 172.21.15.45:0/1986451748 shutdown_connections 2023-10-06T05:22:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.067+0000 7f28c226a700 1 -- 172.21.15.45:0/1986451748 wait complete. 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.068+0000 7f28c226a700 1 Processor -- start 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28c226a700 1 -- start start 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28c226a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc0ccdc0 0x7f28bc14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28c226a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc155cd0 0x7f28bc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28c226a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28bc1560a0 0x7f28bc0a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28c226a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc153510 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28c226a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc068340 con 0x7f28bc1560a0 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28c226a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc067730 con 0x7f28bc155cd0 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc0ccdc0 0x7f28bc14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc0ccdc0 0x7f28bc14ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42886/0 (socket says 172.21.15.45:42886) 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.069+0000 7f28bbfff700 1 -- 172.21.15.45:0/3208238438 learned_addr learned my addr 172.21.15.45:0/3208238438 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28bb7fe700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc155cd0 0x7f28bc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28c0807700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28bc1560a0 0x7f28bc0a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28bbfff700 1 -- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc155cd0 msgr2=0x7f28bc0687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28bbfff700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc155cd0 0x7f28bc0687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28bbfff700 1 -- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28bc1560a0 msgr2=0x7f28bc0a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28bbfff700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28bc1560a0 0x7f28bc0a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28bbfff700 1 -- 172.21.15.45:0/3208238438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac009420 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28bbfff700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc0ccdc0 0x7f28bc14ba40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f28bc06aed0 tx=0x7f28ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28a3fff700 1 -- 172.21.15.45:0/3208238438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00bb20 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28a3fff700 1 -- 172.21.15.45:0/3208238438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac000c90 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc067a10 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28a3fff700 1 -- 172.21.15.45:0/3208238438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac011630 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.070+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc067fd0 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.071+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc068620 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.072+0000 7f28a3fff700 1 -- 172.21.15.45:0/3208238438 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28ac0119a0 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.072+0000 7f28a3fff700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f28a8066f30 0x7f28a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.072+0000 7f28a3fff700 1 -- 172.21.15.45:0/3208238438 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f28ac085cd0 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.072+0000 7f28bb7fe700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f28a8066f30 0x7f28a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.073+0000 7f28bb7fe700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f28a8066f30 0x7f28a80693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0afe80 tx=0x7f28b0007f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.077+0000 7f28a3fff700 1 -- 172.21.15.45:0/3208238438 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28ac052af0 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.324+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28bc0aee90 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.324+0000 7f28a3fff700 1 -- 172.21.15.45:0/3208238438 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+3842 (secure 0 0 0) 0x7f28ac056110 con 0x7f28bc0ccdc0 2023-10-06T05:22:39.326 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:39.326 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:38.915196+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f28a8066f30 msgr2=0x7f28a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f28a8066f30 0x7f28a80693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0afe80 tx=0x7f28b0007f50 comp rx=0 tx=0).stop 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc0ccdc0 msgr2=0x7f28bc14ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc0ccdc0 0x7f28bc14ba40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f28bc06aed0 tx=0x7f28ac017000 comp rx=0 tx=0).stop 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 shutdown_connections 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f28a8066f30 0x7f28a80693e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28bc1560a0 0x7f28bc0a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28bc0ccdc0 0x7f28bc14ba40 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 --2- 172.21.15.45:0/3208238438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28bc155cd0 0x7f28bc0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 >> 172.21.15.45:0/3208238438 conn(0x7f28bc06f090 msgr2=0x7f28bc1598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 shutdown_connections 2023-10-06T05:22:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.326+0000 7f28c226a700 1 -- 172.21.15.45:0/3208238438 wait complete. 2023-10-06T05:22:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2023-10-06T05:22:39.341 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:22:39.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.514+0000 7fc643a88700 1 Processor -- start 2023-10-06T05:22:39.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.515+0000 7fc643a88700 1 -- start start 2023-10-06T05:22:39.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.515+0000 7fc643a88700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc63c155cd0 0x7fc63c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.515+0000 7fc643a88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc643a88700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c0ae8d0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc643a88700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63c0aebb0 con 0x7fc63c155cd0 2023-10-06T05:22:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc643a88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0aee90 con 0x7fc63c1560a0 2023-10-06T05:22:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641023700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641824700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc63c155cd0 0x7fc63c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc642025700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc63c0ccdc0 0x7fc63c0c73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:46698/0 (socket says 172.21.15.45:46698) 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641824700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc63c155cd0 0x7fc63c15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:50988/0 (socket says 172.21.15.45:50988) 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc642025700 1 -- 172.21.15.45:0/4165078330 learned_addr learned my addr 172.21.15.45:0/4165078330 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641023700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:44110/0 (socket says 172.21.15.45:44110) 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641824700 1 -- 172.21.15.45:0/4165078330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 msgr2=0x7fc63c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641824700 1 --2- 172.21.15.45:0/4165078330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641824700 1 -- 172.21.15.45:0/4165078330 >> v1:172.21.15.144:6789/0 conn(0x7fc63c0ccdc0 legacy=0x7fc63c0c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc641824700 1 -- 172.21.15.45:0/4165078330 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c0af170 con 0x7fc63c155cd0 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.516+0000 7fc640822700 1 -- 172.21.15.45:0/4165078330 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137438601 0 0) 0x7fc63c0ae8d0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc641824700 1 --2- 172.21.15.45:0/4165078330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c15a730 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6380098e0 tx=0x7fc6380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e81f5cde259a3268 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc640822700 1 -- 172.21.15.45:0/4165078330 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63800b640 con 0x7fc63c155cd0 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc640822700 1 -- 172.21.15.45:0/4165078330 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63800b7a0 con 0x7fc63c155cd0 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc640822700 1 -- 172.21.15.45:0/4165078330 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63800bb40 con 0x7fc63c155cd0 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc643a88700 1 -- 172.21.15.45:0/4165078330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 msgr2=0x7fc63c15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc643a88700 1 --2- 172.21.15.45:0/4165078330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c15a730 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6380098e0 tx=0x7fc6380091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc643a88700 1 -- 172.21.15.45:0/4165078330 shutdown_connections 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc643a88700 1 --2- 172.21.15.45:0/4165078330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c15a730 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc643a88700 1 --2- 172.21.15.45:0/4165078330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.517+0000 7fc643a88700 1 -- 172.21.15.45:0/4165078330 >> 172.21.15.45:0/4165078330 conn(0x7fc63c06f090 msgr2=0x7fc63c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.518+0000 7fc643a88700 1 -- 172.21.15.45:0/4165078330 shutdown_connections 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.518+0000 7fc643a88700 1 -- 172.21.15.45:0/4165078330 wait complete. 2023-10-06T05:22:39.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.519+0000 7fc643a88700 1 Processor -- start 2023-10-06T05:22:39.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.519+0000 7fc643a88700 1 -- start start 2023-10-06T05:22:39.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.519+0000 7fc643a88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c0ccdc0 0x7fc63c159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.519+0000 7fc643a88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.519+0000 7fc643a88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.519+0000 7fc643a88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c1452f0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc643a88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c1598c0 con 0x7fc63c155cd0 2023-10-06T05:22:39.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc643a88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c144400 con 0x7fc63c1560a0 2023-10-06T05:22:39.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c0ccdc0 0x7fc63c159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c0ccdc0 0x7fc63c159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42898/0 (socket says 172.21.15.45:42898) 2023-10-06T05:22:39.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 -- 172.21.15.45:0/468073424 learned_addr learned my addr 172.21.15.45:0/468073424 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:39.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641023700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc642025700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 -- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 msgr2=0x7fc63c0c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 -- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 msgr2=0x7fc63c157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 -- 172.21.15.45:0/468073424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc638009420 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641023700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc641824700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c0ccdc0 0x7fc63c159380 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc63c06aed0 tx=0x7fc638017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc62dffb700 1 -- 172.21.15.45:0/468073424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63800bb20 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc62dffb700 1 -- 172.21.15.45:0/468073424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc638000c90 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c1446e0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.520+0000 7fc62dffb700 1 -- 172.21.15.45:0/468073424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638011630 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.521+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c144ca0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.522+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c06e2f0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.522+0000 7fc62dffb700 1 -- 172.21.15.45:0/468073424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6380119a0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.523+0000 7fc62dffb700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc620066f30 0x7fc6200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.523+0000 7fc62dffb700 1 -- 172.21.15.45:0/468073424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fc638085cd0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.523+0000 7fc641023700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc620066f30 0x7fc6200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.524+0000 7fc641023700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc620066f30 0x7fc6200693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc63c1464f0 tx=0x7fc6300091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.527+0000 7fc62dffb700 1 -- 172.21.15.45:0/468073424 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc638052af0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.775+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc63c0ae4e0 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.776+0000 7fc62dffb700 1 -- 172.21.15.45:0/468073424 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+3842 (secure 0 0 0) 0x7fc638056110 con 0x7fc63c0ccdc0 2023-10-06T05:22:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:39.778 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:38.915196+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc620066f30 msgr2=0x7fc6200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc620066f30 0x7fc6200693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc63c1464f0 tx=0x7fc6300091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c0ccdc0 msgr2=0x7fc63c159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c0ccdc0 0x7fc63c159380 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc63c06aed0 tx=0x7fc638017000 comp rx=0 tx=0).stop 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 shutdown_connections 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc620066f30 0x7fc6200693e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc63c1560a0 0x7fc63c0c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c0ccdc0 0x7fc63c159380 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 --2- 172.21.15.45:0/468073424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc63c155cd0 0x7fc63c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 >> 172.21.15.45:0/468073424 conn(0x7fc63c06f090 msgr2=0x7fc63c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 shutdown_connections 2023-10-06T05:22:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.778+0000 7fc643a88700 1 -- 172.21.15.45:0/468073424 wait complete. 2023-10-06T05:22:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2023-10-06T05:22:39.793 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:22:39.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.926+0000 7fa3e868b700 1 Processor -- start 2023-10-06T05:22:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.926+0000 7fa3e868b700 1 -- start start 2023-10-06T05:22:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e868b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa3e0159330 0x7fa3e006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e868b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3e00cc390 0x7fa3e00cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e868b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e00cb530 con 0x7fa3e0158f60 2023-10-06T05:22:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e868b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e0069bd0 con 0x7fa3e00cc390 2023-10-06T05:22:39.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e868b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e00ccd10 con 0x7fa3e0159330 2023-10-06T05:22:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e6c28700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3e00cc390 0x7fa3e00cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e6c28700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3e00cc390 0x7fa3e00cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42910/0 (socket says 172.21.15.45:42910) 2023-10-06T05:22:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e6c28700 1 -- 172.21.15.45:0/621807965 learned_addr learned my addr 172.21.15.45:0/621807965 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e5c26700 1 --2- 172.21.15.45:0/621807965 >> v2:172.21.15.144:3300/0 conn(0x7fa3e0159330 0x7fa3e006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e6c28700 1 -- 172.21.15.45:0/621807965 >> v2:172.21.15.144:3300/0 conn(0x7fa3e0159330 msgr2=0x7fa3e006a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e6c28700 1 --2- 172.21.15.45:0/621807965 >> v2:172.21.15.144:3300/0 conn(0x7fa3e0159330 0x7fa3e006a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e6c28700 1 -- 172.21.15.45:0/621807965 >> v1:172.21.15.144:6789/0 conn(0x7fa3e0158f60 legacy=0x7fa3e00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.927+0000 7fa3e6c28700 1 -- 172.21.15.45:0/621807965 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e0068fc0 con 0x7fa3e00cc390 2023-10-06T05:22:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e6c28700 1 --2- 172.21.15.45:0/621807965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e00cc7d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00872e0 tx=0x7fa3d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f449f6337661521a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e5425700 1 -- 172.21.15.45:0/621807965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d800e070 con 0x7fa3e00cc390 2023-10-06T05:22:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e5425700 1 -- 172.21.15.45:0/621807965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d800ba10 con 0x7fa3e00cc390 2023-10-06T05:22:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e5425700 1 -- 172.21.15.45:0/621807965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80092e0 con 0x7fa3e00cc390 2023-10-06T05:22:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e868b700 1 -- 172.21.15.45:0/621807965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 msgr2=0x7fa3e00cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e868b700 1 --2- 172.21.15.45:0/621807965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e00cc7d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00872e0 tx=0x7fa3d8009840 comp rx=0 tx=0).stop 2023-10-06T05:22:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e868b700 1 -- 172.21.15.45:0/621807965 shutdown_connections 2023-10-06T05:22:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e868b700 1 --2- 172.21.15.45:0/621807965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e00cc7d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e868b700 1 --2- 172.21.15.45:0/621807965 >> v2:172.21.15.144:3300/0 conn(0x7fa3e0159330 0x7fa3e006a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.928+0000 7fa3e868b700 1 -- 172.21.15.45:0/621807965 >> 172.21.15.45:0/621807965 conn(0x7fa3e00cdf50 msgr2=0x7fa3e00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.929+0000 7fa3e868b700 1 -- 172.21.15.45:0/621807965 shutdown_connections 2023-10-06T05:22:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.929+0000 7fa3e868b700 1 -- 172.21.15.45:0/621807965 wait complete. 2023-10-06T05:22:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.930+0000 7fa3e868b700 1 Processor -- start 2023-10-06T05:22:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.930+0000 7fa3e868b700 1 -- start start 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.930+0000 7fa3e868b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.930+0000 7fa3e868b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3e0158f60 0x7fa3e00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e868b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e0159330 0x7fa3e00ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e868b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e014e6e0 con 0x7fa3e00cc390 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e868b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00ab4e0 con 0x7fa3e0159330 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e868b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00aa470 con 0x7fa3e0158f60 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e014e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42918/0 (socket says 172.21.15.45:42918) 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 -- 172.21.15.45:0/2695503200 learned_addr learned my addr 172.21.15.45:0/2695503200 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6c28700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e0159330 0x7fa3e00ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e5c26700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3e0158f60 0x7fa3e00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 -- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3e0158f60 msgr2=0x7fa3e00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3e0158f60 0x7fa3e00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 -- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e0159330 msgr2=0x7fa3e00ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e0159330 0x7fa3e00ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 -- 172.21.15.45:0/2695503200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d8009ae0 con 0x7fa3e00cc390 2023-10-06T05:22:39.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e6427700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e014e1a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c5d0 tx=0x7fa3dc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3d67fc700 1 -- 172.21.15.45:0/2695503200 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc011590 con 0x7fa3e00cc390 2023-10-06T05:22:39.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3d67fc700 1 -- 172.21.15.45:0/2695503200 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc011710 con 0x7fa3e00cc390 2023-10-06T05:22:39.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e00a9860 con 0x7fa3e00cc390 2023-10-06T05:22:39.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3d67fc700 1 -- 172.21.15.45:0/2695503200 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc011ad0 con 0x7fa3e00cc390 2023-10-06T05:22:39.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.931+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e00a9e20 con 0x7fa3e00cc390 2023-10-06T05:22:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.932+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e006f410 con 0x7fa3e00cc390 2023-10-06T05:22:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.933+0000 7fa3d67fc700 1 -- 172.21.15.45:0/2695503200 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3dc01c460 con 0x7fa3e00cc390 2023-10-06T05:22:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.933+0000 7fa3d67fc700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c4066f30 0x7fa3c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.933+0000 7fa3d67fc700 1 -- 172.21.15.45:0/2695503200 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fa3dc084640 con 0x7fa3e00cc390 2023-10-06T05:22:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.934+0000 7fa3e5c26700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c4066f30 0x7fa3c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.934+0000 7fa3e5c26700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c4066f30 0x7fa3c40693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00ab940 tx=0x7fa3d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:39.938+0000 7fa3d67fc700 1 -- 172.21.15.45:0/2695503200 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3dc051460 con 0x7fa3e00cc390 2023-10-06T05:22:40.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.184+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3e004bb40 con 0x7fa3e00cc390 2023-10-06T05:22:40.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.185+0000 7fa3d67fc700 1 -- 172.21.15.45:0/2695503200 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4597 (secure 0 0 0) 0x7fa3dc054a80 con 0x7fa3e00cc390 2023-10-06T05:22:40.187 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:40.187 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":116,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3277956876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3277956876},{"type":"v1","addr":"172.21.15.45:6835","nonce":3277956876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:38.915196+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c4066f30 msgr2=0x7fa3c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c4066f30 0x7fa3c40693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00ab940 tx=0x7fa3d0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 msgr2=0x7fa3e014e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e014e1a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c5d0 tx=0x7fa3dc007f60 comp rx=0 tx=0).stop 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 shutdown_connections 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3c4066f30 0x7fa3c40693e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3e0159330 0x7fa3e00ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3e00cc390 0x7fa3e014e1a0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 --2- 172.21.15.45:0/2695503200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3e0158f60 0x7fa3e00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 >> 172.21.15.45:0/2695503200 conn(0x7fa3e00cdf50 msgr2=0x7fa3e00c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 shutdown_connections 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.187+0000 7fa3e868b700 1 -- 172.21.15.45:0/2695503200 wait complete. 2023-10-06T05:22:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 116 2023-10-06T05:22:40.202 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2023-10-06T05:22:40.202 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2023-10-06T05:22:40.203 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:40.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:40.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:40.203 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:40.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:22:40.223 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:22:40.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:40.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.398+0000 7f21b60fc700 1 Processor -- start 2023-10-06T05:22:40.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.398+0000 7f21b60fc700 1 -- start start 2023-10-06T05:22:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21b60fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21b0156ce0 0x7f21b01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21b60fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b00ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21b60fc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b006bf70 con 0x7f21b006bad0 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21b60fc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b00b02f0 con 0x7f21b0156ce0 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21b60fc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b015b380 con 0x7f21b015ac10 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21af7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21b0156ce0 0x7f21b01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21aeffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f21b006bad0 0x7f21b00afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51196/0 (socket says 172.21.15.45:51196) 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21aeffd700 1 -- 172.21.15.45:0/3748362814 learned_addr learned my addr 172.21.15.45:0/3748362814 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21affff700 1 --2- 172.21.15.45:0/3748362814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b00ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21af7fe700 1 -- 172.21.15.45:0/3748362814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 msgr2=0x7f21b00ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21af7fe700 1 --2- 172.21.15.45:0/3748362814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b00ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21af7fe700 1 -- 172.21.15.45:0/3748362814 >> v1:172.21.15.45:6789/0 conn(0x7f21b006bad0 legacy=0x7f21b00afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21af7fe700 1 -- 172.21.15.45:0/3748362814 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b015b660 con 0x7f21b0156ce0 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.399+0000 7f21ae7fc700 1 -- 172.21.15.45:0/3748362814 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649432091 0 0) 0x7f21b006bf70 con 0x7f21b006bad0 2023-10-06T05:22:40.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21af7fe700 1 --2- 172.21.15.45:0/3748362814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b01570b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f21a00098e0 tx=0x7f21a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81e8cf6bafaa2fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21ae7fc700 1 -- 172.21.15.45:0/3748362814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000b640 con 0x7f21b0156ce0 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21ae7fc700 1 -- 172.21.15.45:0/3748362814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a000b7a0 con 0x7f21b0156ce0 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21ae7fc700 1 -- 172.21.15.45:0/3748362814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000bb40 con 0x7f21b0156ce0 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21b60fc700 1 -- 172.21.15.45:0/3748362814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 msgr2=0x7f21b01570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3748362814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b01570b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f21a00098e0 tx=0x7f21a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21b60fc700 1 -- 172.21.15.45:0/3748362814 shutdown_connections 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3748362814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b00ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3748362814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b01570b0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.400+0000 7f21b60fc700 1 -- 172.21.15.45:0/3748362814 >> 172.21.15.45:0/3748362814 conn(0x7f21b006f090 msgr2=0x7f21b006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.401+0000 7f21b60fc700 1 -- 172.21.15.45:0/3748362814 shutdown_connections 2023-10-06T05:22:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.401+0000 7f21b60fc700 1 -- 172.21.15.45:0/3748362814 wait complete. 2023-10-06T05:22:40.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.402+0000 7f21b60fc700 1 Processor -- start 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.402+0000 7f21b60fc700 1 -- start start 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.403+0000 7f21b60fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21b006bad0 0x7f21b01442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.403+0000 7f21b60fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b00b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.403+0000 7f21b60fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b0068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.403+0000 7f21b60fc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b01447f0 con 0x7f21b0156ce0 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.403+0000 7f21b60fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00c5fa0 con 0x7f21b006bad0 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.403+0000 7f21b60fc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00b1cc0 con 0x7f21b015ac10 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.403+0000 7f21aeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b00b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b00b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42930/0 (socket says 172.21.15.45:42930) 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 -- 172.21.15.45:0/3766536244 learned_addr learned my addr 172.21.15.45:0/3766536244 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21af7fe700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21b006bad0 0x7f21b01442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21affff700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b0068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 -- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 msgr2=0x7f21b0068ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b0068ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 -- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21b006bad0 msgr2=0x7f21b01442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21b006bad0 0x7f21b01442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 -- 172.21.15.45:0/3766536244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0009420 con 0x7f21b0156ce0 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21aeffd700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b00b1780 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f21a400c7e0 tx=0x7f21a4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21977fe700 1 -- 172.21.15.45:0/3766536244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4005b30 con 0x7f21b0156ce0 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21977fe700 1 -- 172.21.15.45:0/3766536244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a4005c90 con 0x7f21b0156ce0 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b00c5390 con 0x7f21b0156ce0 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21977fe700 1 -- 172.21.15.45:0/3766536244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4007300 con 0x7f21b0156ce0 2023-10-06T05:22:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.404+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b00c5950 con 0x7f21b0156ce0 2023-10-06T05:22:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.405+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b006e2f0 con 0x7f21b0156ce0 2023-10-06T05:22:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.406+0000 7f21977fe700 1 -- 172.21.15.45:0/3766536244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f21a401b3b0 con 0x7f21b0156ce0 2023-10-06T05:22:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.406+0000 7f21977fe700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f219c066f30 0x7f219c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:40.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.406+0000 7f21977fe700 1 -- 172.21.15.45:0/3766536244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f21a4083ea0 con 0x7f21b0156ce0 2023-10-06T05:22:40.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.406+0000 7f21af7fe700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f219c066f30 0x7f219c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:40.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.407+0000 7f21af7fe700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f219c066f30 0x7f219c0693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f21b015a7c0 tx=0x7f21a0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:40.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.410+0000 7f21977fe700 1 -- 172.21.15.45:0/3766536244 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21a4050cc0 con 0x7f21b0156ce0 2023-10-06T05:22:40.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.659+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f21b00c60e0 con 0x7f21b0156ce0 2023-10-06T05:22:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.922+0000 7f21977fe700 1 -- 172.21.15.45:0/3766536244 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) v1 ==== 161+0+0 (secure 0 0 0) 0x7f21a4017020 con 0x7f21b0156ce0 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f219c066f30 msgr2=0x7f219c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f219c066f30 0x7f219c0693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f21b015a7c0 tx=0x7f21a0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 msgr2=0x7f21b00b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b00b1780 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f21a400c7e0 tx=0x7f21a4007df0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 shutdown_connections 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f219c066f30 0x7f219c0693e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21b015ac10 0x7f21b0068ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21b006bad0 0x7f21b01442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 --2- 172.21.15.45:0/3766536244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21b0156ce0 0x7f21b00b1780 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 >> 172.21.15.45:0/3766536244 conn(0x7f21b006f090 msgr2=0x7f21b00cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 shutdown_connections 2023-10-06T05:22:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:40.925+0000 7f21b60fc700 1 -- 172.21.15.45:0/3766536244 wait complete. 2023-10-06T05:22:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:40.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:22:41.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.076+0000 7f9f0ce98700 1 Processor -- start 2023-10-06T05:22:41.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.076+0000 7f9f0ce98700 1 -- start start 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f0ce98700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9f08153a20 0x7f9f0806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f0ce98700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f0806add0 0x7f9f08068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f0ce98700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f080676c0 con 0x7f9f08153650 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f0ce98700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0806b270 con 0x7f9f08153a20 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f0ce98700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080679a0 con 0x7f9f0806add0 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f0659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f08153650 0x7f9f0815a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51200/0 (socket says 172.21.15.45:51200) 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f0659c700 1 -- 172.21.15.45:0/2476068151 learned_addr learned my addr 172.21.15.45:0/2476068151 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.077+0000 7f9f05d9b700 1 --2- 172.21.15.45:0/2476068151 >> v2:172.21.15.144:3300/0 conn(0x7f9f08153a20 0x7f9f0806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f06d9d700 1 --2- 172.21.15.45:0/2476068151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f0806add0 0x7f9f08068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f0559a700 1 -- 172.21.15.45:0/2476068151 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055304018 0 0) 0x7f9f080676c0 con 0x7f9f08153650 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f0559a700 1 -- 172.21.15.45:0/2476068151 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef4003570 con 0x7f9f08153650 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f06d9d700 1 -- 172.21.15.45:0/2476068151 >> v2:172.21.15.144:3300/0 conn(0x7f9f08153a20 msgr2=0x7f9f0806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f0559a700 1 -- 172.21.15.45:0/2476068151 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1886278227 0 0) 0x7f9ef4003570 con 0x7f9f08153650 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f06d9d700 1 --2- 172.21.15.45:0/2476068151 >> v2:172.21.15.144:3300/0 conn(0x7f9f08153a20 0x7f9f0806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f06d9d700 1 -- 172.21.15.45:0/2476068151 >> v1:172.21.15.45:6789/0 conn(0x7f9f08153650 legacy=0x7f9f0815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f06d9d700 1 -- 172.21.15.45:0/2476068151 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f08067c80 con 0x7f9f0806add0 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f05d9b700 1 --2- 172.21.15.45:0/2476068151 >> v2:172.21.15.144:3300/0 conn(0x7f9f08153a20 0x7f9f0806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f06d9d700 1 --2- 172.21.15.45:0/2476068151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f0806add0 0x7f9f08068480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009e60 tx=0x7f9f00009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36acfc725363d76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f0559a700 1 -- 172.21.15.45:0/2476068151 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0000e070 con 0x7f9f0806add0 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f0559a700 1 -- 172.21.15.45:0/2476068151 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0000b7a0 con 0x7f9f0806add0 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.078+0000 7f9f0559a700 1 -- 172.21.15.45:0/2476068151 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8001470 con 0x7f9f0806add0 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 -- 172.21.15.45:0/2476068151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f0806add0 msgr2=0x7f9f08068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/2476068151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f0806add0 0x7f9f08068480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009e60 tx=0x7f9f00009780 comp rx=0 tx=0).stop 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 -- 172.21.15.45:0/2476068151 shutdown_connections 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/2476068151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f0806add0 0x7f9f08068480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/2476068151 >> v2:172.21.15.144:3300/0 conn(0x7f9f08153a20 0x7f9f0806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 -- 172.21.15.45:0/2476068151 >> 172.21.15.45:0/2476068151 conn(0x7f9f080cc5a0 msgr2=0x7f9f080cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 -- 172.21.15.45:0/2476068151 shutdown_connections 2023-10-06T05:22:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.079+0000 7f9f0ce98700 1 -- 172.21.15.45:0/2476068151 wait complete. 2023-10-06T05:22:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.080+0000 7f9f0ce98700 1 Processor -- start 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.080+0000 7f9f0ce98700 1 -- start start 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0ce98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f0806add0 0x7f9f08151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0ce98700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08153650 0x7f9f080ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0ce98700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f08153a20 0x7f9f080add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0ce98700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080bfbe0 con 0x7f9f0806add0 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0ce98700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080c0a40 con 0x7f9f08153650 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0ce98700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080bfe30 con 0x7f9f08153a20 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f0806add0 0x7f9f08151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f0806add0 0x7f9f08151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42936/0 (socket says 172.21.15.45:42936) 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.081+0000 7f9f0659c700 1 -- 172.21.15.45:0/1078046555 learned_addr learned my addr 172.21.15.45:0/1078046555 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f05d9b700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08153650 0x7f9f080ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f06d9d700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f08153a20 0x7f9f080add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0659c700 1 -- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f08153a20 msgr2=0x7f9f080add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0659c700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f08153a20 0x7f9f080add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0659c700 1 -- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08153650 msgr2=0x7f9f080ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0659c700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08153650 0x7f9f080ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0659c700 1 -- 172.21.15.45:0/1078046555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0009420 con 0x7f9f0806add0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0659c700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f0806add0 0x7f9f08151cc0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800cb90 tx=0x7f9ef8001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9efe7fc700 1 -- 172.21.15.45:0/1078046555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8001d60 con 0x7f9f0806add0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9efe7fc700 1 -- 172.21.15.45:0/1078046555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8007360 con 0x7f9f0806add0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f00009a20 con 0x7f9f0806add0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9efe7fc700 1 -- 172.21.15.45:0/1078046555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8007700 con 0x7f9f0806add0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.082+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f080c03f0 con 0x7f9f0806add0 2023-10-06T05:22:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.083+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0806f950 con 0x7f9f0806add0 2023-10-06T05:22:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.084+0000 7f9efe7fc700 1 -- 172.21.15.45:0/1078046555 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ef8004a40 con 0x7f9f0806add0 2023-10-06T05:22:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.084+0000 7f9efe7fc700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ee8066f30 0x7f9ee80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.084+0000 7f9efe7fc700 1 -- 172.21.15.45:0/1078046555 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f9ef80841c0 con 0x7f9f0806add0 2023-10-06T05:22:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.084+0000 7f9f05d9b700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ee8066f30 0x7f9ee80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.085+0000 7f9f05d9b700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ee8066f30 0x7f9ee80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9f08068ea0 tx=0x7f9ef00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.089+0000 7f9efe7fc700 1 -- 172.21.15.45:0/1078046555 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ef8051060 con 0x7f9f0806add0 2023-10-06T05:22:41.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.372+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f9f080cc0e0 con 0x7f9f0806add0 2023-10-06T05:22:41.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.923+0000 7f9efe7fc700 1 -- 172.21.15.45:0/1078046555 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v118) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9ef8054680 con 0x7f9f0806add0 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ee8066f30 msgr2=0x7f9ee80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ee8066f30 0x7f9ee80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9f08068ea0 tx=0x7f9ef00091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f0806add0 msgr2=0x7f9f08151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f0806add0 0x7f9f08151cc0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800cb90 tx=0x7f9ef8001ad0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 shutdown_connections 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ee8066f30 0x7f9ee80693e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f08153a20 0x7f9f080add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f0806add0 0x7f9f08151cc0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 --2- 172.21.15.45:0/1078046555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f08153650 0x7f9f080ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 >> 172.21.15.45:0/1078046555 conn(0x7f9f080cc5a0 msgr2=0x7f9f08152a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:41.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 shutdown_connections 2023-10-06T05:22:41.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.927+0000 7f9f0ce98700 1 -- 172.21.15.45:0/1078046555 wait complete. 2023-10-06T05:22:41.931 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:22:41.944 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:22:41.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.989+0000 7fe824e3a040 1 Processor -- start 2023-10-06T05:22:41.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.989+0000 7fe824e3a040 1 -- start start 2023-10-06T05:22:41.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.989+0000 7fe824e3a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d2fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.989+0000 7fe824e3a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377890 0x55758d2fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.989+0000 7fe824e3a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377c60 0x55758d2e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.989+0000 7fe824e3a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758d202980 con 0x55758d377c60 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe824e3a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758d202620 con 0x55758d2b5210 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe824e3a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55758d1f9400 con 0x55758d377890 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377c60 0x55758d2e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377c60 0x55758d2e51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42954/0 (socket says 172.21.15.45:42954) 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 -- 172.21.15.45:0/2589637504 learned_addr learned my addr 172.21.15.45:0/2589637504 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81d857700 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377890 0x55758d2fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e058700 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d2fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 -- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377890 msgr2=0x55758d2fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377890 0x55758d2fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 -- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 msgr2=0x55758d2fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:41.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d2fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 -- 172.21.15.45:0/2589637504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810005cf0 con 0x55758d377c60 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81d857700 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377890 0x55758d2fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e058700 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d2fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81e859700 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377c60 0x55758d2e51c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe818006f10 tx=0x7fe8180069f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c78a05bd5272e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81c855700 1 -- 172.21.15.45:0/2589637504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8180020d0 con 0x55758d377c60 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81c855700 1 -- 172.21.15.45:0/2589637504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818006320 con 0x55758d377c60 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.990+0000 7fe81c855700 1 -- 172.21.15.45:0/2589637504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818004730 con 0x55758d377c60 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 -- 172.21.15.45:0/2589637504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377c60 msgr2=0x55758d2e51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377c60 0x55758d2e51c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe818006f10 tx=0x7fe8180069f0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 -- 172.21.15.45:0/2589637504 shutdown_connections 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377c60 0x55758d2e51c0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d2fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 --2- 172.21.15.45:0/2589637504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377890 0x55758d2fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 -- 172.21.15.45:0/2589637504 >> 172.21.15.45:0/2589637504 conn(0x55758d2b8450 msgr2=0x55758d2ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:41.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 -- 172.21.15.45:0/2589637504 shutdown_connections 2023-10-06T05:22:41.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.991+0000 7fe824e3a040 1 -- 172.21.15.45:0/2589637504 wait complete. 2023-10-06T05:22:41.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 Processor -- start 2023-10-06T05:22:41.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 -- start start 2023-10-06T05:22:41.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d396070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377890 0x55758d394b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377c60 0x55758d2e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758d39a720 con 0x55758d377890 2023-10-06T05:22:41.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758d3965b0 con 0x55758d2b5210 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.992+0000 7fe824e3a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55758d3967d0 con 0x55758d377c60 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377890 0x55758d394b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377890 0x55758d394b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42960/0 (socket says 172.21.15.45:42960) 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 learned_addr learned my addr 172.21.15.45:0/47692237 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81e859700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377c60 0x55758d2e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81e058700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d396070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377c60 msgr2=0x55758d2e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55758d377c60 0x55758d2e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 msgr2=0x55758d396070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758d2b5210 0x55758d396070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c001e10 con 0x55758d377890 2023-10-06T05:22:41.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe81d857700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377890 0x55758d394b70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ac40 tx=0x7fe810015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe816ffd700 1 -- 172.21.15.45:0/47692237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810007610 con 0x55758d377890 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe816ffd700 1 -- 172.21.15.45:0/47692237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810007e80 con 0x55758d377890 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55758d2113d0 con 0x55758d377890 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.993+0000 7fe816ffd700 1 -- 172.21.15.45:0/47692237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810007af0 con 0x55758d377890 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.994+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55758d204ae0 con 0x55758d377890 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.995+0000 7fe816ffd700 1 -- 172.21.15.45:0/47692237 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe810019070 con 0x55758d377890 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.995+0000 7fe816ffd700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8000658c0 0x7fe800067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.995+0000 7fe816ffd700 1 -- 172.21.15.45:0/47692237 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fe810084a00 con 0x55758d377890 2023-10-06T05:22:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.996+0000 7fe81e058700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8000658c0 0x7fe800067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.001 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.996+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55758d374e10 0x55758d39adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.996+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d39b330 con 0x55758d374e10 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.996+0000 7fe81e859700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55758d374e10 0x55758d39adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.996+0000 7fe81e058700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8000658c0 0x7fe800067d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe808005b20 tx=0x7fe808005a50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.996+0000 7fe81e859700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55758d374e10 0x55758d39adf0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.998+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v97'133 uv107 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.998+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d39b6e0 con 0x55758d374e10 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.999+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v97'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:41.999+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3aac90 con 0x55758d374e10 2023-10-06T05:22:42.003 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:22:42.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.000+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v97'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.000+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac050 con 0x55758d374e10 2023-10-06T05:22:42.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.001+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v97'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.001+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac400 con 0x55758d374e10 2023-10-06T05:22:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.002+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v97'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.002+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab040 con 0x55758d374e10 2023-10-06T05:22:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.003+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v97'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.003+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab3f0 con 0x55758d374e10 2023-10-06T05:22:42.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.005+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v97'139 uv113 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.005+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.006+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v97'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.006+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.007+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v97'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.009 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:22:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.007+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.008+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v97'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.008+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.009+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v97'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.009+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.010+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v97'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.010+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.011+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v97'145 uv120 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.011+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.012+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v97'146 uv146 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.012+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.015 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:22:42.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.013+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v97'147 uv147 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.013+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.014+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v97'148 uv148 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.014+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.015+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v97'149 uv149 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.015+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ab7a0 con 0x55758d374e10 2023-10-06T05:22:42.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.021+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v97'150 uv150 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.021+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.022+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v97'151 uv151 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.022+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.025+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v97'152 uv126 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.025+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.028+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v97'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.028+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.034 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:22:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.033+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v97'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.033+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ad7c0 con 0x55758d374e10 2023-10-06T05:22:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.034+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v97'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.034+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ad7c0 con 0x55758d374e10 2023-10-06T05:22:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.037+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v97'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.037+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.042+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v97'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.042+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.044+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v97'158 uv132 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.044+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.046+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v97'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.046+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.051 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:22:42.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.049+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v97'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.049+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ad7c0 con 0x55758d374e10 2023-10-06T05:22:42.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.050+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v97'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.050+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ad7c0 con 0x55758d374e10 2023-10-06T05:22:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.052+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v97'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.052+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ac7b0 con 0x55758d374e10 2023-10-06T05:22:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.053+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v97'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d374e10 2023-10-06T05:22:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.053+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55758d3accc0 0x55758d3af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.053+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3af5e0 con 0x55758d3accc0 2023-10-06T05:22:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.053+0000 7fe81e058700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55758d3accc0 0x55758d3af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.053+0000 7fe81e058700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55758d3accc0 0x55758d3af0a0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.055+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v97'108 uv90 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.055+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3af990 con 0x55758d3accc0 2023-10-06T05:22:42.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.056+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v97'109 uv109 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.056+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3afd40 con 0x55758d3accc0 2023-10-06T05:22:42.059 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:22:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.058+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v97'110 uv110 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.058+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b05e0 con 0x55758d3accc0 2023-10-06T05:22:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.059+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v97'111 uv111 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.059+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b0990 con 0x55758d3accc0 2023-10-06T05:22:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.060+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v97'112 uv112 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.060+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b0d40 con 0x55758d3accc0 2023-10-06T05:22:42.063 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:22:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.061+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v97'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.061+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b10f0 con 0x55758d3accc0 2023-10-06T05:22:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.063+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v97'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.063+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.064+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v97'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.064+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.065+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000001 [create] v97'116 uv100 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.065+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.066+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000001 [delete] v97'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.066+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.068+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v97'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.069 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:22:42.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.068+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4bc0 con 0x55758d3accc0 2023-10-06T05:22:42.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.069+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [write 0~14075] v97'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.069+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4bc0 con 0x55758d3accc0 2023-10-06T05:22:42.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.070+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v97'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.070+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.071+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 606.00000000 [create] v97'121 uv106 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.071+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.073+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 606.00000000 [delete] v97'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.073+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.074+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v97'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.076 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:22:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.074+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b24b0 con 0x55758d3accc0 2023-10-06T05:22:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.076+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v97'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.076+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b24b0 con 0x55758d3accc0 2023-10-06T05:22:42.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.077+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v97'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.077+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b14a0 con 0x55758d3accc0 2023-10-06T05:22:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.078+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v97'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.078+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55758d3b19b0 0x55758d3b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.078+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:51 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b42d0 con 0x55758d3b19b0 2023-10-06T05:22:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.078+0000 7fe81d857700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55758d3b19b0 0x55758d3b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.079+0000 7fe81d857700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55758d3b19b0 0x55758d3b3d90 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.080+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(51 607.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.080+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:52 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4680 con 0x55758d3b19b0 2023-10-06T05:22:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.081+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(52 607.00000000 [delete] v97'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.081+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4a30 con 0x55758d3b19b0 2023-10-06T05:22:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:22:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.082+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(53 607.00000000 [create] v97'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.082+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b5df0 con 0x55758d3b19b0 2023-10-06T05:22:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.083+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.084+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b61a0 con 0x55758d3b19b0 2023-10-06T05:22:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.085+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.085+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b0170 con 0x55758d3b19b0 2023-10-06T05:22:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.086+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.086+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b52d0 con 0x55758d3b19b0 2023-10-06T05:22:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.087+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(57 mds0_openfiles.0 [create] v97'20 uv19 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.087+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b56b0 con 0x55758d3b19b0 2023-10-06T05:22:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.089+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v97'21 uv21 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.089+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b56b0 con 0x55758d3b19b0 2023-10-06T05:22:42.092 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:22:42.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.090+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(59 mds0_openfiles.0 [create] v97'22 uv22 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.090+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b56b0 con 0x55758d3b19b0 2023-10-06T05:22:42.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.091+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v97'23 uv23 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.091+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:61 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b5680 con 0x55758d3accc0 2023-10-06T05:22:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.092+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(61 608.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.092+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:62 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b5680 con 0x55758d3accc0 2023-10-06T05:22:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.094+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(62 608.00000000 [delete] v97'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.094+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b5680 con 0x55758d3accc0 2023-10-06T05:22:42.096 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:22:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.095+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v97'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.095+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b5680 con 0x55758d3accc0 2023-10-06T05:22:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.096+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.096+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b5680 con 0x55758d3accc0 2023-10-06T05:22:42.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.097+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.097+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b5680 con 0x55758d3accc0 2023-10-06T05:22:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.098+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3accc0 2023-10-06T05:22:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.098+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55758d3b5680 0x55758d3b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.098+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:67 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b8fb0 con 0x55758d3b5680 2023-10-06T05:22:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.098+0000 7fe81e859700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55758d3b5680 0x55758d3b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.099+0000 7fe81e859700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55758d3b5680 0x55758d3b8a70 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.100+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(67 604.00000000 [create] v97'48 uv42 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.100+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:68 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9070 con 0x55758d3b5680 2023-10-06T05:22:42.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.101+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(68 604.00000000 [delete] v97'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.101+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9470 con 0x55758d3b5680 2023-10-06T05:22:42.104 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:22:42.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.102+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(69 604.00000000 [create] v97'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.103+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ba480 con 0x55758d3b5680 2023-10-06T05:22:42.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.104+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v97'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.104+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3ba480 con 0x55758d3b5680 2023-10-06T05:22:42.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.105+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v97'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.105+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9470 con 0x55758d3b5680 2023-10-06T05:22:42.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.106+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v97'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.106+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9470 con 0x55758d3b5680 2023-10-06T05:22:42.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.107+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(73 500.00000000 [create] v97'54 uv47 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.108+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9470 con 0x55758d3b5680 2023-10-06T05:22:42.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.109+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(74 500.00000000 [delete] v97'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.109+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9470 con 0x55758d3b5680 2023-10-06T05:22:42.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:22:42.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.110+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(75 500.00000000 [create] v97'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.110+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9470 con 0x55758d3b5680 2023-10-06T05:22:42.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.112+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v97'57 uv57 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.112+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b9470 con 0x55758d3b5680 2023-10-06T05:22:42.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.114+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v97'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.114+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55758d3b9980 0x55758d3bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.114+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:78 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc450 con 0x55758d3b9980 2023-10-06T05:22:42.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.114+0000 7fe81e058700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55758d3b9980 0x55758d3bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.115+0000 7fe81e058700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55758d3b9980 0x55758d3bbda0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.116+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(78 mds_snaptable [create] v97'22 uv21 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3b9980 2023-10-06T05:22:42.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.116+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:79 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b9980 2023-10-06T05:22:42.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.117+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(79 mds_snaptable [delete] v97'23 uv23 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3b9980 2023-10-06T05:22:42.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.117+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b9980 2023-10-06T05:22:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.119+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(80 mds_snaptable [create] v97'24 uv24 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3b9980 2023-10-06T05:22:42.121 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:22:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.120+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b9980 2023-10-06T05:22:42.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.122+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v97'25 uv25 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3b9980 2023-10-06T05:22:42.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.122+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b9980 2023-10-06T05:22:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.125+0000 7fe81e058700 1 -- 172.21.15.45:0/47692237 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v97'26 uv26 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe808007d80 con 0x55758d3b9980 2023-10-06T05:22:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.125+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:83 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b19b0 2023-10-06T05:22:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.127+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(83 605.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.127+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:84 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b19b0 2023-10-06T05:22:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.128+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(84 605.00000000 [delete] v97'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.128+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b19b0 2023-10-06T05:22:42.130 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:22:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.129+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(85 605.00000000 [create] v97'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.129+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bd840 con 0x55758d3b19b0 2023-10-06T05:22:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.130+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.130+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bd840 con 0x55758d3b19b0 2023-10-06T05:22:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.131+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.131+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bc830 con 0x55758d3b19b0 2023-10-06T05:22:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.133+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.133+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55758d3bceb0 0x55758d3bf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.133+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:89 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bf8c0 con 0x55758d3bceb0 2023-10-06T05:22:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.133+0000 7fe81d857700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55758d3bceb0 0x55758d3bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.133+0000 7fe81d857700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55758d3bceb0 0x55758d3bf290 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.134+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(89 mds0_inotable [create] v97'44 uv38 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.134+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:90 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.136+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(90 mds0_inotable [delete] v97'45 uv45 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.136+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.137+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(91 mds0_inotable [create] v97'46 uv46 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.138 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:22:42.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.137+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.138+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(92 mds0_inotable [write 0~34] v97'47 uv47 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.138+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.139+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v97'48 uv48 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.139+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.140+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(94 100.00000000 [create] v97'49 uv43 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.140+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.141+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(95 100.00000000 [delete] v97'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.141+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.144 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:22:42.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.142+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(96 100.00000000 [create] v97'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.142+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3bfca0 con 0x55758d3bceb0 2023-10-06T05:22:42.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.143+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(97 100.00000000 [omap-set-header] v97'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.143+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c2920 con 0x55758d3bceb0 2023-10-06T05:22:42.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.144+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v97'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3bceb0 2023-10-06T05:22:42.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.144+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4e50 con 0x55758d3b19b0 2023-10-06T05:22:42.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.145+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v97'22 uv21 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.145+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4e50 con 0x55758d3b19b0 2023-10-06T05:22:42.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.146+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v97'23 uv23 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.146+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4e50 con 0x55758d3b19b0 2023-10-06T05:22:42.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.148+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v97'24 uv24 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.149 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:22:42.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.148+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4e50 con 0x55758d3b19b0 2023-10-06T05:22:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.149+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v97'25 uv25 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.149+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55758d3c09c0 0x55758d3c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.149+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4e50 con 0x55758d3c09c0 2023-10-06T05:22:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.149+0000 7fe81e859700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55758d3c09c0 0x55758d3c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.150+0000 7fe81e859700 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55758d3c09c0 0x55758d3c2da0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.151+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3c09c0 2023-10-06T05:22:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.151+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3b4e50 con 0x55758d3c09c0 2023-10-06T05:22:42.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.153+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v97'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3c09c0 2023-10-06T05:22:42.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.153+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3c09c0 2023-10-06T05:22:42.158 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:22:42.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.156+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v97'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3c09c0 2023-10-06T05:22:42.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.156+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c42f0 con 0x55758d3c09c0 2023-10-06T05:22:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.159+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3c09c0 2023-10-06T05:22:42.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.159+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c42f0 con 0x55758d3c09c0 2023-10-06T05:22:42.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.162+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3c09c0 2023-10-06T05:22:42.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.162+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3c09c0 2023-10-06T05:22:42.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.164+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3c09c0 2023-10-06T05:22:42.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.164+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b5680 2023-10-06T05:22:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.166+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v97'22 uv21 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.166+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b5680 2023-10-06T05:22:42.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.167+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v97'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.167+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b5680 2023-10-06T05:22:42.170 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:22:42.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.168+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v97'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.168+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b5680 2023-10-06T05:22:42.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.170+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v97'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.170+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b5680 2023-10-06T05:22:42.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.171+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v97'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe818004580 con 0x55758d3b5680 2023-10-06T05:22:42.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.171+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b19b0 2023-10-06T05:22:42.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.172+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v97'57 uv49 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.172+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b19b0 2023-10-06T05:22:42.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.173+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v97'58 uv58 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.173+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b19b0 2023-10-06T05:22:42.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.174+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v97'59 uv59 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.176 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:22:42.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.174+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b19b0 2023-10-06T05:22:42.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.176+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v97'60 uv60 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.176+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c0040 con 0x55758d3b19b0 2023-10-06T05:22:42.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.177+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v97'61 uv61 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.177+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c0040 con 0x55758d3b19b0 2023-10-06T05:22:42.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.178+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v97'62 uv56 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.178+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b19b0 2023-10-06T05:22:42.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.179+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v97'63 uv63 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.179+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c32e0 con 0x55758d3b19b0 2023-10-06T05:22:42.182 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:22:42.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.181+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v97'64 uv64 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.181+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c42f0 con 0x55758d3b19b0 2023-10-06T05:22:42.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.182+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v97'65 uv65 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.182+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c42f0 con 0x55758d3b19b0 2023-10-06T05:22:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.183+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v97'66 uv66 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.183+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c37d0 con 0x55758d3b19b0 2023-10-06T05:22:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.184+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v97'67 uv67 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.184+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v8 -- 0x55758d3c37d0 con 0x55758d3b19b0 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe81d857700 1 -- 172.21.15.45:0/47692237 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v97'68 uv68 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe810082b40 con 0x55758d3b19b0 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55758d3b19b0 msgr2=0x55758d3b3d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55758d3b19b0 0x55758d3b3d90 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55758d374e10 msgr2=0x55758d39adf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55758d374e10 0x55758d39adf0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55758d3bceb0 msgr2=0x55758d3bf290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55758d3bceb0 0x55758d3bf290 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55758d3accc0 msgr2=0x55758d3af0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55758d3accc0 0x55758d3af0a0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55758d3c09c0 msgr2=0x55758d3c2da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55758d3c09c0 0x55758d3c2da0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55758d3b9980 msgr2=0x55758d3bbda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55758d3b9980 0x55758d3bbda0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55758d3b5680 msgr2=0x55758d3b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55758d3b5680 0x55758d3b8a70 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 reap_dead start 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 reap_dead start 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 reap_dead start 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 reap_dead start 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe81e859700 1 -- 172.21.15.45:0/47692237 reap_dead start 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8000658c0 msgr2=0x7fe800067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8000658c0 0x7fe800067d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe808005b20 tx=0x7fe808005a50 comp rx=0 tx=0).stop 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377890 msgr2=0x55758d394b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377890 0x55758d394b70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ac40 tx=0x7fe810015000 comp rx=0 tx=0).stop 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 shutdown_connections 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55758d377890 0x55758d394b70 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 --2- 172.21.15.45:0/47692237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8000658c0 0x7fe800067d70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.185+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 >> 172.21.15.45:0/47692237 conn(0x55758d2b8450 msgr2=0x55758d2fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.186+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 shutdown_connections 2023-10-06T05:22:42.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.186+0000 7fe824e3a040 1 -- 172.21.15.45:0/47692237 wait complete. 2023-10-06T05:22:42.191 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2023-10-06T05:22:42.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 Processor -- start 2023-10-06T05:22:42.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 -- start start 2023-10-06T05:22:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f2eb210 0x55e72f3ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3ad890 0x55e72f3c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f3adc60 0x55e72f3d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72f238980 con 0x55e72f3ad890 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72f238620 con 0x55e72f2eb210 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339ed70040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e72f22f400 con 0x55e72f3adc60 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.239+0000 7f339778d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3ad890 0x55e72f3c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3ad890 0x55e72f3c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42980/0 (socket says 172.21.15.45:42980) 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 -- 172.21.15.45:0/256811882 learned_addr learned my addr 172.21.15.45:0/256811882 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339878f700 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f3adc60 0x55e72f3d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f3397f8e700 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f2eb210 0x55e72f3ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 -- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f3adc60 msgr2=0x55e72f3d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f3adc60 0x55e72f3d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 -- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f2eb210 msgr2=0x55e72f3ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f2eb210 0x55e72f3ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 -- 172.21.15.45:0/256811882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380005cf0 con 0x55e72f3ad890 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339778d700 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3ad890 0x55e72f3c9b70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f338000ac40 tx=0x7f3380015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229cf6e153b526d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339678b700 1 -- 172.21.15.45:0/256811882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380000bc0 con 0x55e72f3ad890 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339678b700 1 -- 172.21.15.45:0/256811882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33800078e0 con 0x55e72f3ad890 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.240+0000 7f339678b700 1 -- 172.21.15.45:0/256811882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380000d20 con 0x55e72f3ad890 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 -- 172.21.15.45:0/256811882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3ad890 msgr2=0x55e72f3c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3ad890 0x55e72f3c9b70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f338000ac40 tx=0x7f3380015000 comp rx=0 tx=0).stop 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 -- 172.21.15.45:0/256811882 shutdown_connections 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f3adc60 0x55e72f3d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f2eb210 0x55e72f3ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 --2- 172.21.15.45:0/256811882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3ad890 0x55e72f3c9b70 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 -- 172.21.15.45:0/256811882 >> 172.21.15.45:0/256811882 conn(0x55e72f2ee450 msgr2=0x55e72f2f0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 -- 172.21.15.45:0/256811882 shutdown_connections 2023-10-06T05:22:42.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 -- 172.21.15.45:0/256811882 wait complete. 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.241+0000 7f339ed70040 1 Processor -- start 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339ed70040 1 -- start start 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339ed70040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f2eb210 0x55e72f3ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339ed70040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f3ad890 0x55e72f2f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339ed70040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3adc60 0x55e72f3ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339ed70040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72f2f0c40 con 0x55e72f3adc60 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339ed70040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72f2f0e80 con 0x55e72f3ad890 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339ed70040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e72f3cc6c0 con 0x55e72f2eb210 2023-10-06T05:22:42.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.242+0000 7f339878f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3adc60 0x55e72f3ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339778d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f3ad890 0x55e72f2f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3adc60 0x55e72f3ac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42984/0 (socket says 172.21.15.45:42984) 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f3397f8e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f2eb210 0x55e72f3ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 -- 172.21.15.45:0/614329870 learned_addr learned my addr 172.21.15.45:0/614329870 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 -- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f2eb210 msgr2=0x55e72f3ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e72f2eb210 0x55e72f3ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 -- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f3ad890 msgr2=0x55e72f2f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e72f3ad890 0x55e72f2f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 -- 172.21.15.45:0/614329870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e72f2611c0 con 0x55e72f3adc60 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339878f700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3adc60 0x55e72f3ac2c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f339000d4c0 tx=0x7f3390008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f3394f88700 1 -- 172.21.15.45:0/614329870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900060d0 con 0x55e72f3adc60 2023-10-06T05:22:42.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f3394f88700 1 -- 172.21.15.45:0/614329870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3390006250 con 0x55e72f3adc60 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e72f2473d0 con 0x55e72f3adc60 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f3394f88700 1 -- 172.21.15.45:0/614329870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390004770 con 0x55e72f3adc60 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.243+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e72f23aae0 con 0x55e72f3adc60 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.245+0000 7f3394f88700 1 -- 172.21.15.45:0/614329870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3390004b20 con 0x55e72f3adc60 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.245+0000 7f3394f88700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3378065890 0x7f3378067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.245+0000 7f3394f88700 1 -- 172.21.15.45:0/614329870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f3390082f50 con 0x55e72f3adc60 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.245+0000 7f3397f8e700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3378065890 0x7f3378067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.245+0000 7f339ed70040 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e72f3cd340 0x55e72f3cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.245+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e97) v8 -- 0x55e72f3ac800 con 0x55e72f3cd340 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.246+0000 7f339778d700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e72f3cd340 0x55e72f3cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.246+0000 7f3397f8e700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3378065890 0x7f3378067d40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3388009c20 tx=0x7f338800c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.246+0000 7f339778d700 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e72f3cd340 0x55e72f3cb610 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.247+0000 7f339778d700 1 -- 172.21.15.45:0/614329870 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000001 [delete] v97'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3380022020 con 0x55e72f3cd340 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.247+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e72f3cd340 msgr2=0x55e72f3cb610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.247+0000 7f339ed70040 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e72f3cd340 0x55e72f3cb610 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3378065890 msgr2=0x7f3378067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3378065890 0x7f3378067d40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3388009c20 tx=0x7f338800c040 comp rx=0 tx=0).stop 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3adc60 msgr2=0x55e72f3ac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 --2- 172.21.15.45:0/614329870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e72f3adc60 0x55e72f3ac2c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f339000d4c0 tx=0x7f3390008040 comp rx=0 tx=0).stop 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339878f700 1 -- 172.21.15.45:0/614329870 reap_dead start 2023-10-06T05:22:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 shutdown_connections 2023-10-06T05:22:42.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 >> 172.21.15.45:0/614329870 conn(0x55e72f2ee450 msgr2=0x55e72f3c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 shutdown_connections 2023-10-06T05:22:42.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.248+0000 7f339ed70040 1 -- 172.21.15.45:0/614329870 wait complete. 2023-10-06T05:22:42.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:22:42.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.394+0000 7ff8ee6e5700 1 Processor -- start 2023-10-06T05:22:42.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.394+0000 7ff8ee6e5700 1 -- start start 2023-10-06T05:22:42.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.394+0000 7ff8ee6e5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e80ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.394+0000 7ff8ee6e5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e80c3b50 con 0x7ff8e80aaae0 2023-10-06T05:22:42.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8ee6e5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e8157700 con 0x7ff8e80ccb70 2023-10-06T05:22:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8ee6e5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80c98e0 con 0x7ff8e8157330 2023-10-06T05:22:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff8e80aaae0 0x7ff8e80c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51222/0 (socket says 172.21.15.45:51222) 2023-10-06T05:22:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e77fe700 1 -- 172.21.15.45:0/4050338261 learned_addr learned my addr 172.21.15.45:0/4050338261 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e7fff700 1 --2- 172.21.15.45:0/4050338261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e80ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e6ffd700 1 -- 172.21.15.45:0/4050338261 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630570832 0 0) 0x7ff8e80c3b50 con 0x7ff8e80aaae0 2023-10-06T05:22:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e7fff700 1 -- 172.21.15.45:0/4050338261 >> v1:172.21.15.144:6789/0 conn(0x7ff8e80ccb70 legacy=0x7ff8e80c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e7fff700 1 -- 172.21.15.45:0/4050338261 >> v1:172.21.15.45:6789/0 conn(0x7ff8e80aaae0 legacy=0x7ff8e80c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e7fff700 1 -- 172.21.15.45:0/4050338261 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e80c9bc0 con 0x7ff8e8157330 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e7fff700 1 --2- 172.21.15.45:0/4050338261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e80ce8b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009880 tx=0x7ff8d8009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4b74329646c0e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e6ffd700 1 -- 172.21.15.45:0/4050338261 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8011630 con 0x7ff8e8157330 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e6ffd700 1 -- 172.21.15.45:0/4050338261 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8011790 con 0x7ff8e8157330 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.395+0000 7ff8e6ffd700 1 -- 172.21.15.45:0/4050338261 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8011b30 con 0x7ff8e8157330 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.396+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/4050338261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 msgr2=0x7ff8e80ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.396+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/4050338261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e80ce8b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009880 tx=0x7ff8d8009120 comp rx=0 tx=0).stop 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.396+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/4050338261 shutdown_connections 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.396+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/4050338261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e80ce8b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.396+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/4050338261 >> 172.21.15.45:0/4050338261 conn(0x7ff8e8067e00 msgr2=0x7ff8e80cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.396+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/4050338261 shutdown_connections 2023-10-06T05:22:42.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.396+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/4050338261 wait complete. 2023-10-06T05:22:42.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.397+0000 7ff8ee6e5700 1 Processor -- start 2023-10-06T05:22:42.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.397+0000 7ff8ee6e5700 1 -- start start 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8ee6e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e80aaae0 0x7ff8e814ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8ee6e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8e80ccb70 0x7ff8e8155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8ee6e5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e8153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8ee6e5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8155ed0 con 0x7ff8e80ccb70 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8ee6e5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e81561b0 con 0x7ff8e80aaae0 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8ee6e5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8156490 con 0x7ff8e8157330 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8e77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8e80ccb70 0x7ff8e8155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8e77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8e80ccb70 0x7ff8e8155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42986/0 (socket says 172.21.15.45:42986) 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8e77fe700 1 -- 172.21.15.45:0/3033402969 learned_addr learned my addr 172.21.15.45:0/3033402969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8e7fff700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e80aaae0 0x7ff8e814ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.398+0000 7ff8ecc82700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e8153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8e77fe700 1 -- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 msgr2=0x7ff8e8153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8e77fe700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e8153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8e77fe700 1 -- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e80aaae0 msgr2=0x7ff8e814ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8e77fe700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e80aaae0 0x7ff8e814ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8e77fe700 1 -- 172.21.15.45:0/3033402969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80093c0 con 0x7ff8e80ccb70 2023-10-06T05:22:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8e77fe700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8e80ccb70 0x7ff8e8155480 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00c560 tx=0x7ff8dc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8cffff700 1 -- 172.21.15.45:0/3033402969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc005b30 con 0x7ff8e80ccb70 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8cffff700 1 -- 172.21.15.45:0/3033402969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc005c90 con 0x7ff8e80ccb70 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e8156770 con 0x7ff8e80ccb70 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8cffff700 1 -- 172.21.15.45:0/3033402969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc007320 con 0x7ff8e80ccb70 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.399+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e814f450 con 0x7ff8e80ccb70 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.400+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e806e350 con 0x7ff8e80ccb70 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.401+0000 7ff8cffff700 1 -- 172.21.15.45:0/3033402969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8dc01b3b0 con 0x7ff8e80ccb70 2023-10-06T05:22:42.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.401+0000 7ff8cffff700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff8d4066f30 0x7ff8d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.401+0000 7ff8cffff700 1 -- 172.21.15.45:0/3033402969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6808+0+0 (secure 0 0 0) 0x7ff8dc083d50 con 0x7ff8e80ccb70 2023-10-06T05:22:42.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.401+0000 7ff8e7fff700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff8d4066f30 0x7ff8d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.402+0000 7ff8e7fff700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff8d4066f30 0x7ff8d40693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009160 tx=0x7ff8d8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.406+0000 7ff8cffff700 1 -- 172.21.15.45:0/3033402969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8dc087030 con 0x7ff8e80ccb70 2023-10-06T05:22:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.663+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff8e80c37e0 con 0x7ff8e80ccb70 2023-10-06T05:22:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.931+0000 7ff8cffff700 1 -- 172.21.15.45:0/3033402969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v119) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff8dc054600 con 0x7ff8e80ccb70 2023-10-06T05:22:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff8d4066f30 msgr2=0x7ff8d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff8d4066f30 0x7ff8d40693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009160 tx=0x7ff8d8016000 comp rx=0 tx=0).stop 2023-10-06T05:22:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8e80ccb70 msgr2=0x7ff8e8155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8e80ccb70 0x7ff8e8155480 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00c560 tx=0x7ff8dc000fe0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 shutdown_connections 2023-10-06T05:22:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff8d4066f30 0x7ff8d40693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e8157330 0x7ff8e8153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e80aaae0 0x7ff8e814ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 --2- 172.21.15.45:0/3033402969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8e80ccb70 0x7ff8e8155480 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 >> 172.21.15.45:0/3033402969 conn(0x7ff8e8067e00 msgr2=0x7ff8e8152e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 shutdown_connections 2023-10-06T05:22:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:42.934+0000 7ff8ee6e5700 1 -- 172.21.15.45:0/3033402969 wait complete. 2023-10-06T05:22:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:22:42.944 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:22:42.943+0000 7f8eafae0700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-10-06T05:22:42.946 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:42.944+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:42.946 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:42.944+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:22:42.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:42.960 INFO:tasks.ceph.mds.c.smithi045.stderr: -14> 2023-10-06T05:22:42.943+0000 7f8eafae0700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-10-06T05:22:42.981 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:22:43.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.087+0000 7faec14c2700 1 Processor -- start 2023-10-06T05:22:43.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.087+0000 7faec14c2700 1 -- start start 2023-10-06T05:22:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faec14c2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faebc159170 0x7faebc159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faec14c2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc15d040 0x7faebc068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faec14c2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faebc069b90 0x7faebc069100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faec14c2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc0ca6d0 con 0x7faebc159170 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faec14c2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc15d550 con 0x7faebc069b90 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faec14c2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15d810 con 0x7faebc15d040 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faebaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faebc159170 0x7faebc159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faebaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faebc159170 0x7faebc159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43030/0 (socket says 172.21.15.45:43030) 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faebaffd700 1 -- 172.21.15.45:0/387153851 learned_addr learned my addr 172.21.15.45:0/387153851 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faebb7fe700 1 --2- 172.21.15.45:0/387153851 >> v2:172.21.15.144:3300/0 conn(0x7faebc069b90 0x7faebc069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.088+0000 7faeba7fc700 1 --2- 172.21.15.45:0/387153851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc15d040 0x7faebc068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faebaffd700 1 -- 172.21.15.45:0/387153851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc15d040 msgr2=0x7faebc068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faebaffd700 1 --2- 172.21.15.45:0/387153851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc15d040 0x7faebc068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faebaffd700 1 -- 172.21.15.45:0/387153851 >> v2:172.21.15.144:3300/0 conn(0x7faebc069b90 msgr2=0x7faebc069100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faebaffd700 1 --2- 172.21.15.45:0/387153851 >> v2:172.21.15.144:3300/0 conn(0x7faebc069b90 0x7faebc069100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faebaffd700 1 -- 172.21.15.45:0/387153851 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc15daf0 con 0x7faebc159170 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faeba7fc700 1 --2- 172.21.15.45:0/387153851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc15d040 0x7faebc068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faebb7fe700 1 --2- 172.21.15.45:0/387153851 >> v2:172.21.15.144:3300/0 conn(0x7faebc069b90 0x7faebc069100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faebaffd700 1 --2- 172.21.15.45:0/387153851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc159540 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7faebc0cca60 tx=0x7faeac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e533db90ca5c2682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faeb9ffb700 1 -- 172.21.15.45:0/387153851 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00b640 con 0x7faebc159170 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faeb9ffb700 1 -- 172.21.15.45:0/387153851 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac00b7a0 con 0x7faebc159170 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faeb9ffb700 1 -- 172.21.15.45:0/387153851 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00bb60 con 0x7faebc159170 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faec14c2700 1 -- 172.21.15.45:0/387153851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 msgr2=0x7faebc159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.089+0000 7faec14c2700 1 --2- 172.21.15.45:0/387153851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc159540 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7faebc0cca60 tx=0x7faeac009250 comp rx=0 tx=0).stop 2023-10-06T05:22:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.090+0000 7faec14c2700 1 -- 172.21.15.45:0/387153851 shutdown_connections 2023-10-06T05:22:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.090+0000 7faec14c2700 1 --2- 172.21.15.45:0/387153851 >> v2:172.21.15.144:3300/0 conn(0x7faebc069b90 0x7faebc069100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.090+0000 7faec14c2700 1 --2- 172.21.15.45:0/387153851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc159540 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.090+0000 7faec14c2700 1 --2- 172.21.15.45:0/387153851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc15d040 0x7faebc068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.090+0000 7faec14c2700 1 -- 172.21.15.45:0/387153851 >> 172.21.15.45:0/387153851 conn(0x7faebc06f7c0 msgr2=0x7faebc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.090+0000 7faec14c2700 1 -- 172.21.15.45:0/387153851 shutdown_connections 2023-10-06T05:22:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.090+0000 7faec14c2700 1 -- 172.21.15.45:0/387153851 wait complete. 2023-10-06T05:22:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.091+0000 7faec14c2700 1 Processor -- start 2023-10-06T05:22:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.091+0000 7faec14c2700 1 -- start start 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faec14c2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc069b90 0x7faebc0ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faec14c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faec14c2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc15d040 0x7faebc0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faec14c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15bc20 con 0x7faebc159170 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faec14c2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15ba30 con 0x7faebc15d040 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faec14c2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0ca330 con 0x7faebc069b90 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc15b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43038/0 (socket says 172.21.15.45:43038) 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 -- 172.21.15.45:0/1556228434 learned_addr learned my addr 172.21.15.45:0/1556228434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faebb7fe700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc15d040 0x7faebc0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faebaffd700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc069b90 0x7faebc0ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 -- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc069b90 msgr2=0x7faebc0ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc069b90 0x7faebc0ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 -- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc15d040 msgr2=0x7faebc0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc15d040 0x7faebc0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 -- 172.21.15.45:0/1556228434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4009a20 con 0x7faebc159170 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.092+0000 7faeba7fc700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc15b4f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7faea400ec80 tx=0x7faea40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.093+0000 7faeb2ffd700 1 -- 172.21.15.45:0/1556228434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400c9d0 con 0x7faebc159170 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.093+0000 7faeb2ffd700 1 -- 172.21.15.45:0/1556228434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea400cb30 con 0x7faebc159170 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.093+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea4009420 con 0x7faebc159170 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.093+0000 7faeb2ffd700 1 -- 172.21.15.45:0/1556228434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400ceb0 con 0x7faebc159170 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.093+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac0094f0 con 0x7faebc159170 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.094+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc063b50 con 0x7faebc159170 2023-10-06T05:22:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.094+0000 7faeb2ffd700 1 -- 172.21.15.45:0/1556228434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faea4013680 con 0x7faebc159170 2023-10-06T05:22:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.095+0000 7faeb2ffd700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fae9c066f30 0x7fae9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.095+0000 7faeb2ffd700 1 -- 172.21.15.45:0/1556228434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7faea4084d70 con 0x7faebc159170 2023-10-06T05:22:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.095+0000 7faebaffd700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fae9c066f30 0x7fae9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.095+0000 7faebaffd700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fae9c066f30 0x7fae9c0693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faebc0cca60 tx=0x7faeac017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.099+0000 7faeb2ffd700 1 -- 172.21.15.45:0/1556228434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faea4051b40 con 0x7faebc159170 2023-10-06T05:22:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.344+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faebc15ddd0 con 0x7faebc159170 2023-10-06T05:22:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.344+0000 7faeb2ffd700 1 -- 172.21.15.45:0/1556228434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+3840 (secure 0 0 0) 0x7faea4055160 con 0x7faebc159170 2023-10-06T05:22:43.346 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:43.346 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:42.945445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fae9c066f30 msgr2=0x7fae9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fae9c066f30 0x7fae9c0693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faebc0cca60 tx=0x7faeac017000 comp rx=0 tx=0).stop 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 msgr2=0x7faebc15b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc15b4f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7faea400ec80 tx=0x7faea40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 shutdown_connections 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fae9c066f30 0x7fae9c0693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc15d040 0x7faebc0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc069b90 0x7faebc0ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 --2- 172.21.15.45:0/1556228434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faebc159170 0x7faebc15b4f0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 >> 172.21.15.45:0/1556228434 conn(0x7faebc06f7c0 msgr2=0x7faebc158510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.346+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 shutdown_connections 2023-10-06T05:22:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.347+0000 7faec14c2700 1 -- 172.21.15.45:0/1556228434 wait complete. 2023-10-06T05:22:43.350 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 121 2023-10-06T05:22:43.362 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:22:43.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:43.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.535+0000 7f26b8dc2700 1 Processor -- start 2023-10-06T05:22:43.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.535+0000 7f26b8dc2700 1 -- start start 2023-10-06T05:22:43.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b8dc2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b41560a0 0x7f26b415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b8dc2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f26b40ccdc0 0x7f26b40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b8dc2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40cd740 con 0x7f26b4155cd0 2023-10-06T05:22:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b8dc2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b40ae6b0 con 0x7f26b40ccdc0 2023-10-06T05:22:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b8dc2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40ae8d0 con 0x7f26b41560a0 2023-10-06T05:22:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b1d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b41560a0 0x7f26b415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b2d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f26b40ccdc0 0x7f26b40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b1d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b41560a0 0x7f26b415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:44282/0 (socket says 172.21.15.45:44282) 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.536+0000 7f26b1d9b700 1 -- 172.21.15.45:0/369377936 learned_addr learned my addr 172.21.15.45:0/369377936 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b1d9b700 1 -- 172.21.15.45:0/369377936 >> v2:172.21.15.144:3300/0 conn(0x7f26b40ccdc0 msgr2=0x7f26b40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b1d9b700 1 --2- 172.21.15.45:0/369377936 >> v2:172.21.15.144:3300/0 conn(0x7f26b40ccdc0 0x7f26b40cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b1d9b700 1 -- 172.21.15.45:0/369377936 >> v1:172.21.15.144:6789/0 conn(0x7f26b4155cd0 legacy=0x7f26b40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b1d9b700 1 -- 172.21.15.45:0/369377936 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40aebb0 con 0x7f26b41560a0 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b159a700 1 -- 172.21.15.45:0/369377936 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679395494 0 0) 0x7f26b40cd740 con 0x7f26b4155cd0 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b2d9d700 1 --2- 172.21.15.45:0/369377936 >> v2:172.21.15.144:3300/0 conn(0x7f26b40ccdc0 0x7f26b40cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b1d9b700 1 --2- 172.21.15.45:0/369377936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b41560a0 0x7f26b415a730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f269c0099b0 tx=0x7f269c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c1e82e0bdcb5689 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b159a700 1 -- 172.21.15.45:0/369377936 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c009ea0 con 0x7f26b41560a0 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b159a700 1 -- 172.21.15.45:0/369377936 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00b860 con 0x7f26b41560a0 2023-10-06T05:22:43.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.537+0000 7f26b159a700 1 -- 172.21.15.45:0/369377936 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00bc00 con 0x7f26b41560a0 2023-10-06T05:22:43.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 -- 172.21.15.45:0/369377936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b41560a0 msgr2=0x7f26b415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/369377936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b41560a0 0x7f26b415a730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f269c0099b0 tx=0x7f269c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:43.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 -- 172.21.15.45:0/369377936 shutdown_connections 2023-10-06T05:22:43.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/369377936 >> v2:172.21.15.144:3300/0 conn(0x7f26b40ccdc0 0x7f26b40cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/369377936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b41560a0 0x7f26b415a730 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 -- 172.21.15.45:0/369377936 >> 172.21.15.45:0/369377936 conn(0x7f26b406f090 msgr2=0x7f26b406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 -- 172.21.15.45:0/369377936 shutdown_connections 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.538+0000 7f26b8dc2700 1 -- 172.21.15.45:0/369377936 wait complete. 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.539+0000 7f26b8dc2700 1 Processor -- start 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.539+0000 7f26b8dc2700 1 -- start start 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b8dc2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b40ccdc0 0x7f26b41687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b8dc2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b4155cd0 0x7f26b416a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b8dc2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b41560a0 0x7f26b415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b8dc2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b41567e0 con 0x7f26b41560a0 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b8dc2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4157080 con 0x7f26b40ccdc0 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b8dc2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b41683f0 con 0x7f26b4155cd0 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b41560a0 0x7f26b415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b41560a0 0x7f26b415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43046/0 (socket says 172.21.15.45:43046) 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 -- 172.21.15.45:0/3312640557 learned_addr learned my addr 172.21.15.45:0/3312640557 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b1d9b700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b4155cd0 0x7f26b416a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b259c700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b40ccdc0 0x7f26b41687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 -- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b4155cd0 msgr2=0x7f26b416a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b4155cd0 0x7f26b416a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 -- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b40ccdc0 msgr2=0x7f26b41687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b40ccdc0 0x7f26b41687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.540+0000 7f26b2d9d700 1 -- 172.21.15.45:0/3312640557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac009a20 con 0x7f26b41560a0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.541+0000 7f26b2d9d700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b41560a0 0x7f26b415c710 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009fc0 tx=0x7f26ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.541+0000 7f26aa7fc700 1 -- 172.21.15.45:0/3312640557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00cbd0 con 0x7f26b41560a0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.541+0000 7f26aa7fc700 1 -- 172.21.15.45:0/3312640557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac00f040 con 0x7f26b41560a0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.541+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c0094f0 con 0x7f26b41560a0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.541+0000 7f26aa7fc700 1 -- 172.21.15.45:0/3312640557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac014480 con 0x7f26b41560a0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.541+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b4156d10 con 0x7f26b41560a0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.542+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b41598e0 con 0x7f26b41560a0 2023-10-06T05:22:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.542+0000 7f26aa7fc700 1 -- 172.21.15.45:0/3312640557 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26ac003020 con 0x7f26b41560a0 2023-10-06T05:22:43.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.543+0000 7f26aa7fc700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2694066f30 0x7f26940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.543+0000 7f26aa7fc700 1 -- 172.21.15.45:0/3312640557 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f26ac085390 con 0x7f26b41560a0 2023-10-06T05:22:43.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.543+0000 7f26b259c700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2694066f30 0x7f26940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.543+0000 7f26b259c700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2694066f30 0x7f26940693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26b415d450 tx=0x7f26a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.547+0000 7f26aa7fc700 1 -- 172.21.15.45:0/3312640557 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26ac0520c0 con 0x7f26b41560a0 2023-10-06T05:22:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.802+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26b41571f0 con 0x7f26b41560a0 2023-10-06T05:22:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.802+0000 7f26aa7fc700 1 -- 172.21.15.45:0/3312640557 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+3840 (secure 0 0 0) 0x7f26ac0556e0 con 0x7f26b41560a0 2023-10-06T05:22:43.804 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:43.804 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:42.945445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2694066f30 msgr2=0x7f26940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2694066f30 0x7f26940693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26b415d450 tx=0x7f26a4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b41560a0 msgr2=0x7f26b415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b41560a0 0x7f26b415c710 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009fc0 tx=0x7f26ac00b040 comp rx=0 tx=0).stop 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 shutdown_connections 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2694066f30 0x7f26940693e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b41560a0 0x7f26b415c710 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b40ccdc0 0x7f26b41687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 --2- 172.21.15.45:0/3312640557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b4155cd0 0x7f26b416a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 >> 172.21.15.45:0/3312640557 conn(0x7f26b406f090 msgr2=0x7f26b40c7120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 shutdown_connections 2023-10-06T05:22:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.805+0000 7f26b8dc2700 1 -- 172.21.15.45:0/3312640557 wait complete. 2023-10-06T05:22:43.808 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 121 2023-10-06T05:22:43.820 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:22:43.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.952+0000 7fea93c53700 1 Processor -- start 2023-10-06T05:22:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.952+0000 7fea93c53700 1 -- start start 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea93c53700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea93c53700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea8c0caad0 0x7fea8c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea93c53700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fea8c156af0 0x7fea8c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea93c53700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8c15b000 con 0x7fea8c0caad0 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea93c53700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8c15b2e0 con 0x7fea8c156af0 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea93c53700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c15b5c0 con 0x7fea8c0c8ff0 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea911ee700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea8c0caad0 0x7fea8c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea911ee700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea8c0caad0 0x7fea8c0caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43060/0 (socket says 172.21.15.45:43060) 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea911ee700 1 -- 172.21.15.45:0/2571637465 learned_addr learned my addr 172.21.15.45:0/2571637465 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea921f0700 1 --2- 172.21.15.45:0/2571637465 >> v2:172.21.15.144:3300/0 conn(0x7fea8c156af0 0x7fea8c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.953+0000 7fea919ef700 1 --2- 172.21.15.45:0/2571637465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea911ee700 1 -- 172.21.15.45:0/2571637465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 msgr2=0x7fea8c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea911ee700 1 --2- 172.21.15.45:0/2571637465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea911ee700 1 -- 172.21.15.45:0/2571637465 >> v2:172.21.15.144:3300/0 conn(0x7fea8c156af0 msgr2=0x7fea8c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea911ee700 1 --2- 172.21.15.45:0/2571637465 >> v2:172.21.15.144:3300/0 conn(0x7fea8c156af0 0x7fea8c156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea911ee700 1 -- 172.21.15.45:0/2571637465 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c15b8a0 con 0x7fea8c0caad0 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea911ee700 1 --2- 172.21.15.45:0/2571637465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c0caad0 0x7fea8c0caf40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0b0a60 tx=0x7fea7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e4eda9e5cd1463 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea909ed700 1 -- 172.21.15.45:0/2571637465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00b640 con 0x7fea8c0caad0 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea909ed700 1 -- 172.21.15.45:0/2571637465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c00b7a0 con 0x7fea8c0caad0 2023-10-06T05:22:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.954+0000 7fea909ed700 1 -- 172.21.15.45:0/2571637465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00bb60 con 0x7fea8c0caad0 2023-10-06T05:22:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 -- 172.21.15.45:0/2571637465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c0caad0 msgr2=0x7fea8c0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 --2- 172.21.15.45:0/2571637465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c0caad0 0x7fea8c0caf40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0b0a60 tx=0x7fea7c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 -- 172.21.15.45:0/2571637465 shutdown_connections 2023-10-06T05:22:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 --2- 172.21.15.45:0/2571637465 >> v2:172.21.15.144:3300/0 conn(0x7fea8c156af0 0x7fea8c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 --2- 172.21.15.45:0/2571637465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 --2- 172.21.15.45:0/2571637465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c0caad0 0x7fea8c0caf40 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 -- 172.21.15.45:0/2571637465 >> 172.21.15.45:0/2571637465 conn(0x7fea8c0cdf50 msgr2=0x7fea8c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 -- 172.21.15.45:0/2571637465 shutdown_connections 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.955+0000 7fea93c53700 1 -- 172.21.15.45:0/2571637465 wait complete. 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.956+0000 7fea93c53700 1 Processor -- start 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.956+0000 7fea93c53700 1 -- start start 2023-10-06T05:22:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea93c53700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea93c53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0caad0 0x7fea8c068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea93c53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c156af0 0x7fea8c068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea93c53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c0b3cd0 con 0x7fea8c156af0 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea93c53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c158b10 con 0x7fea8c0caad0 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea93c53700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c0b1400 con 0x7fea8c0c8ff0 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c156af0 0x7fea8c068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c156af0 0x7fea8c068c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43062/0 (socket says 172.21.15.45:43062) 2023-10-06T05:22:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 -- 172.21.15.45:0/2144230094 learned_addr learned my addr 172.21.15.45:0/2144230094 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea919ef700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea911ee700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0caad0 0x7fea8c068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 -- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 msgr2=0x7fea8c0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 -- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0caad0 msgr2=0x7fea8c068730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0caad0 0x7fea8c068730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea921f0700 1 -- 172.21.15.45:0/2144230094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c0094f0 con 0x7fea8c156af0 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.957+0000 7fea919ef700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.958+0000 7fea921f0700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c156af0 0x7fea8c068c70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fea84009e90 tx=0x7fea8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.958+0000 7fea81ffb700 1 -- 172.21.15.45:0/2144230094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400cc20 con 0x7fea8c156af0 2023-10-06T05:22:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.958+0000 7fea81ffb700 1 -- 172.21.15.45:0/2144230094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8400f040 con 0x7fea8c156af0 2023-10-06T05:22:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.958+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8c157350 con 0x7fea8c156af0 2023-10-06T05:22:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.958+0000 7fea81ffb700 1 -- 172.21.15.45:0/2144230094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84014480 con 0x7fea8c156af0 2023-10-06T05:22:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.958+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8c0b1120 con 0x7fea8c156af0 2023-10-06T05:22:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.959+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8c158c80 con 0x7fea8c156af0 2023-10-06T05:22:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.959+0000 7fea81ffb700 1 -- 172.21.15.45:0/2144230094 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea84014830 con 0x7fea8c156af0 2023-10-06T05:22:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.960+0000 7fea81ffb700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fea70066f30 0x7fea700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.960+0000 7fea81ffb700 1 -- 172.21.15.45:0/2144230094 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7fea840853a0 con 0x7fea8c156af0 2023-10-06T05:22:43.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.960+0000 7fea919ef700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fea70066f30 0x7fea700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.961+0000 7fea919ef700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fea70066f30 0x7fea700693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fea88009c50 tx=0x7fea88008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:43.964+0000 7fea81ffb700 1 -- 172.21.15.45:0/2144230094 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea84055c00 con 0x7fea8c156af0 2023-10-06T05:22:44.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.216+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea8c04bb40 con 0x7fea8c156af0 2023-10-06T05:22:44.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.217+0000 7fea81ffb700 1 -- 172.21.15.45:0/2144230094 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4595 (secure 0 0 0) 0x7fea84055790 con 0x7fea8c156af0 2023-10-06T05:22:44.218 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:44.219 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7804,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/226461717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":226461717},{"type":"v1","addr":"172.21.15.45:6835","nonce":226461717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:42.945445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:44.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fea70066f30 msgr2=0x7fea700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fea70066f30 0x7fea700693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fea88009c50 tx=0x7fea88008040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c156af0 msgr2=0x7fea8c068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c156af0 0x7fea8c068c70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fea84009e90 tx=0x7fea8400b040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 shutdown_connections 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fea70066f30 0x7fea700693e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea8c156af0 0x7fea8c068c70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0c8ff0 0x7fea8c0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 --2- 172.21.15.45:0/2144230094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0caad0 0x7fea8c068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 >> 172.21.15.45:0/2144230094 conn(0x7fea8c0cdf50 msgr2=0x7fea8c151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 shutdown_connections 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.219+0000 7fea93c53700 1 -- 172.21.15.45:0/2144230094 wait complete. 2023-10-06T05:22:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2023-10-06T05:22:44.234 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2023-10-06T05:22:44.234 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2023-10-06T05:22:44.235 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:44.235 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:44.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:44.235 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:44.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:22:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:22:44.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.431+0000 7f766b00a700 1 Processor -- start 2023-10-06T05:22:44.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.431+0000 7f766b00a700 1 -- start start 2023-10-06T05:22:44.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.432+0000 7f766b00a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f76640c93c0 0x7f7664156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.432+0000 7f766b00a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 0x7f76640cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.432+0000 7f766b00a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766415b000 con 0x7f76640c8ff0 2023-10-06T05:22:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.432+0000 7f766b00a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76640c9830 con 0x7f76640c93c0 2023-10-06T05:22:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.432+0000 7f766b00a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766415b2e0 con 0x7f7664157030 2023-10-06T05:22:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f7668da6700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f76640c8ff0 0x7f76640caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51224/0 (socket says 172.21.15.45:51224) 2023-10-06T05:22:44.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f7668da6700 1 -- 172.21.15.45:0/4191485098 learned_addr learned my addr 172.21.15.45:0/4191485098 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151002829 0 0) 0x7f766415b000 con 0x7f76640c8ff0 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76695a7700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 0x7f76640cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f764c003570 con 0x7f76640c8ff0 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f7663fff700 1 --2- 172.21.15.45:0/4191485098 >> v2:172.21.15.144:3300/0 conn(0x7f76640c93c0 0x7f7664156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2299665927 0 0) 0x7f764c003570 con 0x7f76640c8ff0 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 msgr2=0x7f76640cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 0x7f76640cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 >> v2:172.21.15.144:3300/0 conn(0x7f76640c93c0 msgr2=0x7f7664156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 --2- 172.21.15.45:0/4191485098 >> v2:172.21.15.144:3300/0 conn(0x7f76640c93c0 0x7f7664156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766415b5c0 con 0x7f76640c8ff0 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f7654001470 con 0x7f76640c8ff0 2023-10-06T05:22:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 >> v1:172.21.15.45:6789/0 conn(0x7f76640c8ff0 legacy=0x7f76640caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c003a60 0x7f764c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 0x7f764c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764c004d50 0x7f764c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640c9830 con 0x7f764c004d50 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766415b2e0 con 0x7f764c003a60 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c00bb10 con 0x7f7664157030 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f76695a7700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764c004d50 0x7f764c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f7663fff700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 0x7f764c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.433+0000 7f7668da6700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c003a60 0x7f764c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76695a7700 1 -- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 msgr2=0x7f764c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76695a7700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7664157030 0x7f764c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76695a7700 1 -- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c003a60 msgr2=0x7f764c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76695a7700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c003a60 0x7f764c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76695a7700 1 -- 172.21.15.45:0/4191485098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c003570 con 0x7f764c004d50 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76695a7700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764c004d50 0x7f764c0051c0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f765c00e210 tx=0x7f765c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0e34af4d1372cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00eac0 con 0x7f764c004d50 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c00f040 con 0x7f764c004d50 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.434+0000 7f76637fe700 1 -- 172.21.15.45:0/4191485098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00ee70 con 0x7f764c004d50 2023-10-06T05:22:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.435+0000 7f766b00a700 1 -- 172.21.15.45:0/4191485098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764c004d50 msgr2=0x7f764c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.435+0000 7f766b00a700 1 --2- 172.21.15.45:0/4191485098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764c004d50 0x7f764c0051c0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f765c00e210 tx=0x7f765c00b040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.435+0000 7f76695a7700 1 -- 172.21.15.45:0/4191485098 reap_dead start 2023-10-06T05:22:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.435+0000 7f766b00a700 1 -- 172.21.15.45:0/4191485098 shutdown_connections 2023-10-06T05:22:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.435+0000 7f766b00a700 1 -- 172.21.15.45:0/4191485098 >> 172.21.15.45:0/4191485098 conn(0x7f76640cdf50 msgr2=0x7f76640ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.435+0000 7f766b00a700 1 -- 172.21.15.45:0/4191485098 shutdown_connections 2023-10-06T05:22:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.435+0000 7f766b00a700 1 -- 172.21.15.45:0/4191485098 wait complete. 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.436+0000 7f766b00a700 1 Processor -- start 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.436+0000 7f766b00a700 1 -- start start 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f766b00a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7664069da0 0x7f76640afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f766b00a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766414e510 0x7f7664134ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f766b00a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766406a170 0x7f7664135000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f766b00a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7664135540 con 0x7f766406a170 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f766b00a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7664149460 con 0x7f7664069da0 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f766b00a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7664149740 con 0x7f766414e510 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766406a170 0x7f7664135000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766406a170 0x7f7664135000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43074/0 (socket says 172.21.15.45:43074) 2023-10-06T05:22:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 -- 172.21.15.45:0/3470098626 learned_addr learned my addr 172.21.15.45:0/3470098626 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f7663fff700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766414e510 0x7f7664134ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f7668da6700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7664069da0 0x7f76640afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 -- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766414e510 msgr2=0x7f7664134ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766414e510 0x7f7664134ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 -- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7664069da0 msgr2=0x7f76640afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7664069da0 0x7f76640afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.437+0000 7f76695a7700 1 -- 172.21.15.45:0/3470098626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664149a20 con 0x7f766406a170 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.438+0000 7f76695a7700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766406a170 0x7f7664135000 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f765c016000 tx=0x7f765c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.438+0000 7f7660ff9700 1 -- 172.21.15.45:0/3470098626 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c027430 con 0x7f766406a170 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.438+0000 7f7660ff9700 1 -- 172.21.15.45:0/3470098626 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c022070 con 0x7f766406a170 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.438+0000 7f7660ff9700 1 -- 172.21.15.45:0/3470098626 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c027800 con 0x7f766406a170 2023-10-06T05:22:44.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.438+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7664149d00 con 0x7f766406a170 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.438+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76640b0130 con 0x7f766406a170 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.439+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766414a220 con 0x7f766406a170 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.440+0000 7f7660ff9700 1 -- 172.21.15.45:0/3470098626 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f765c024030 con 0x7f766406a170 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.440+0000 7f7660ff9700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.440+0000 7f7660ff9700 1 -- 172.21.15.45:0/3470098626 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f765c086eb0 con 0x7f766406a170 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.440+0000 7f7668da6700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.441+0000 7f7668da6700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7650066f30 0x7f76500693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7654000e10 tx=0x7f7654009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.445+0000 7f7660ff9700 1 -- 172.21.15.45:0/3470098626 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f766414a220 con 0x7f766406a170 2023-10-06T05:22:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.691+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f766414a220 con 0x7f766406a170 2023-10-06T05:22:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.956+0000 7f7660ff9700 1 -- 172.21.15.45:0/3470098626 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v123) v1 ==== 161+0+0 (secure 0 0 0) 0x7f766414a220 con 0x7f766406a170 2023-10-06T05:22:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7650066f30 msgr2=0x7f76500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7650066f30 0x7f76500693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7654000e10 tx=0x7f7654009040 comp rx=0 tx=0).stop 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766406a170 msgr2=0x7f7664135000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766406a170 0x7f7664135000 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f765c016000 tx=0x7f765c01a000 comp rx=0 tx=0).stop 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 shutdown_connections 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766406a170 0x7f7664135000 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7664069da0 0x7f76640afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 --2- 172.21.15.45:0/3470098626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766414e510 0x7f7664134ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 >> 172.21.15.45:0/3470098626 conn(0x7f76640cdf50 msgr2=0x7f766414bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 shutdown_connections 2023-10-06T05:22:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:44.959+0000 7f766b00a700 1 -- 172.21.15.45:0/3470098626 wait complete. 2023-10-06T05:22:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:44.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:22:45.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.112+0000 7f6b7476c700 1 Processor -- start 2023-10-06T05:22:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.112+0000 7f6b7476c700 1 -- start start 2023-10-06T05:22:45.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b7476c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6b6c155cd0 0x7f6b6c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b7476c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b6c1560a0 0x7f6b6c0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b7476c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c0ccdc0 0x7f6b6c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b7476c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0cd740 con 0x7f6b6c1560a0 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b7476c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0ae8d0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b7476c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0aebb0 con 0x7f6b6c0ccdc0 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b6c1560a0 0x7f6b6c0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b6c1560a0 0x7f6b6c0adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43084/0 (socket says 172.21.15.45:43084) 2023-10-06T05:22:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 -- 172.21.15.45:0/160294362 learned_addr learned my addr 172.21.15.45:0/160294362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b72d09700 1 --2- 172.21.15.45:0/160294362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c0ccdc0 0x7f6b6c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b72508700 1 --2- 172.21.15.45:0/160294362 >> v2:172.21.15.144:3300/0 conn(0x7f6b6c155cd0 0x7f6b6c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 -- 172.21.15.45:0/160294362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c0ccdc0 msgr2=0x7f6b6c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 --2- 172.21.15.45:0/160294362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c0ccdc0 0x7f6b6c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 -- 172.21.15.45:0/160294362 >> v2:172.21.15.144:3300/0 conn(0x7f6b6c155cd0 msgr2=0x7f6b6c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 --2- 172.21.15.45:0/160294362 >> v2:172.21.15.144:3300/0 conn(0x7f6b6c155cd0 0x7f6b6c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b71d07700 1 -- 172.21.15.45:0/160294362 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0aee90 con 0x7f6b6c1560a0 2023-10-06T05:22:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b72d09700 1 --2- 172.21.15.45:0/160294362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c0ccdc0 0x7f6b6c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.113+0000 7f6b72508700 1 --2- 172.21.15.45:0/160294362 >> v2:172.21.15.144:3300/0 conn(0x7f6b6c155cd0 0x7f6b6c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b71d07700 1 --2- 172.21.15.45:0/160294362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c1560a0 0x7f6b6c0adfa0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c06ae50 tx=0x7f6b5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37a78b26d2cae54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b71506700 1 -- 172.21.15.45:0/160294362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00b860 con 0x7f6b6c1560a0 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b71506700 1 -- 172.21.15.45:0/160294362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c00b9c0 con 0x7f6b6c1560a0 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b71506700 1 -- 172.21.15.45:0/160294362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00bd60 con 0x7f6b6c1560a0 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 -- 172.21.15.45:0/160294362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c1560a0 msgr2=0x7f6b6c0adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 --2- 172.21.15.45:0/160294362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c1560a0 0x7f6b6c0adfa0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c06ae50 tx=0x7f6b5c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 -- 172.21.15.45:0/160294362 shutdown_connections 2023-10-06T05:22:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 --2- 172.21.15.45:0/160294362 >> v2:172.21.15.144:3300/0 conn(0x7f6b6c155cd0 0x7f6b6c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 --2- 172.21.15.45:0/160294362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c0ccdc0 0x7f6b6c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 --2- 172.21.15.45:0/160294362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c1560a0 0x7f6b6c0adfa0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 -- 172.21.15.45:0/160294362 >> 172.21.15.45:0/160294362 conn(0x7f6b6c06f090 msgr2=0x7f6b6c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.114+0000 7f6b7476c700 1 -- 172.21.15.45:0/160294362 shutdown_connections 2023-10-06T05:22:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.115+0000 7f6b7476c700 1 -- 172.21.15.45:0/160294362 wait complete. 2023-10-06T05:22:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.116+0000 7f6b7476c700 1 Processor -- start 2023-10-06T05:22:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.116+0000 7f6b7476c700 1 -- start start 2023-10-06T05:22:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.116+0000 7f6b7476c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b6c0ccdc0 0x7f6b6c149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.116+0000 7f6b7476c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c155cd0 0x7f6b6c0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.116+0000 7f6b7476c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c1560a0 0x7f6b6c0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b7476c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c167610 con 0x7f6b6c155cd0 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b7476c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c1594e0 con 0x7f6b6c0ccdc0 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b7476c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0c5450 con 0x7f6b6c1560a0 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c155cd0 0x7f6b6c0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c155cd0 0x7f6b6c0b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43100/0 (socket says 172.21.15.45:43100) 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 -- 172.21.15.45:0/477145330 learned_addr learned my addr 172.21.15.45:0/477145330 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b72d09700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c1560a0 0x7f6b6c0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b72508700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b6c0ccdc0 0x7f6b6c149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 -- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c1560a0 msgr2=0x7f6b6c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c1560a0 0x7f6b6c0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 -- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b6c0ccdc0 msgr2=0x7f6b6c149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b6c0ccdc0 0x7f6b6c149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 -- 172.21.15.45:0/477145330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0094f0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b72d09700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c1560a0 0x7f6b6c0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b71d07700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c155cd0 0x7f6b6c0b0c60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0090b0 tx=0x7f6b5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b627fc700 1 -- 172.21.15.45:0/477145330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c01a450 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b627fc700 1 -- 172.21.15.45:0/477145330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c01a5b0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c0c45a0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b627fc700 1 -- 172.21.15.45:0/477145330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c01a970 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.117+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0c4ba0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.118+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c0c55e0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.119+0000 7f6b627fc700 1 -- 172.21.15.45:0/477145330 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b5c01ace0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.120+0000 7f6b627fc700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6b50066f30 0x7f6b500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.120+0000 7f6b627fc700 1 -- 172.21.15.45:0/477145330 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f6b5c086010 con 0x7f6b6c155cd0 2023-10-06T05:22:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.120+0000 7f6b72508700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6b50066f30 0x7f6b500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.120+0000 7f6b72508700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6b50066f30 0x7f6b500693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6b68006fd0 tx=0x7f6b68008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.124+0000 7f6b627fc700 1 -- 172.21.15.45:0/477145330 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b5c052de0 con 0x7f6b6c155cd0 2023-10-06T05:22:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.410+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f6b6c0af170 con 0x7f6b6c155cd0 2023-10-06T05:22:45.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.961+0000 7f6b627fc700 1 -- 172.21.15.45:0/477145330 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v124) v1 ==== 101+0+0 (secure 0 0 0) 0x7f6b5c056400 con 0x7f6b6c155cd0 2023-10-06T05:22:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6b50066f30 msgr2=0x7f6b500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6b50066f30 0x7f6b500693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6b68006fd0 tx=0x7f6b68008040 comp rx=0 tx=0).stop 2023-10-06T05:22:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c155cd0 msgr2=0x7f6b6c0b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c155cd0 0x7f6b6c0b0c60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0090b0 tx=0x7f6b5c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 shutdown_connections 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6b50066f30 0x7f6b500693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6c1560a0 0x7f6b6c0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b6c0ccdc0 0x7f6b6c149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 --2- 172.21.15.45:0/477145330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b6c155cd0 0x7f6b6c0b0c60 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 >> 172.21.15.45:0/477145330 conn(0x7f6b6c06f090 msgr2=0x7f6b6c0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 shutdown_connections 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:45.964+0000 7f6b7476c700 1 -- 172.21.15.45:0/477145330 wait complete. 2023-10-06T05:22:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:22:45.981 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:22:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.026+0000 7f1600349040 1 Processor -- start 2023-10-06T05:22:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.026+0000 7f1600349040 1 -- start start 2023-10-06T05:22:46.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f1600349040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b95fb17e0 0x556b9603e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f1600349040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cb30 0x556b96048be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f1600349040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b9603cf00 0x556b96049120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f1600349040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b95eb4980 con 0x556b9603cf00 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f1600349040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b95eb4620 con 0x556b95fb17e0 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f1600349040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b95eab400 con 0x556b9603cb30 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b9603cf00 0x556b96049120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b9603cf00 0x556b96049120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43118/0 (socket says 172.21.15.45:43118) 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 -- 172.21.15.45:0/3441197866 learned_addr learned my addr 172.21.15.45:0/3441197866 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f8d66700 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cb30 0x556b96048be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9567700 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b95fb17e0 0x556b9603e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 -- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cb30 msgr2=0x556b96048be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cb30 0x556b96048be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 -- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b95fb17e0 msgr2=0x556b9603e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b95fb17e0 0x556b9603e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 -- 172.21.15.45:0/3441197866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec005db0 con 0x556b9603cf00 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.027+0000 7f15f9d68700 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b9603cf00 0x556b96049120 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c4f0 tx=0x7f15f4007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263a24204d4b7bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f15eb7fe700 1 -- 172.21.15.45:0/3441197866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4012480 con 0x556b9603cf00 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f15eb7fe700 1 -- 172.21.15.45:0/3441197866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f40125e0 con 0x556b9603cf00 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f15eb7fe700 1 -- 172.21.15.45:0/3441197866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40129a0 con 0x556b9603cf00 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 -- 172.21.15.45:0/3441197866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b9603cf00 msgr2=0x556b96049120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b9603cf00 0x556b96049120 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c4f0 tx=0x7f15f4007040 comp rx=0 tx=0).stop 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 -- 172.21.15.45:0/3441197866 shutdown_connections 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b9603cf00 0x556b96049120 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b95fb17e0 0x556b9603e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 --2- 172.21.15.45:0/3441197866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cb30 0x556b96048be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 -- 172.21.15.45:0/3441197866 >> 172.21.15.45:0/3441197866 conn(0x556b95f6a450 msgr2=0x556b95f67210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 -- 172.21.15.45:0/3441197866 shutdown_connections 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.028+0000 7f1600349040 1 -- 172.21.15.45:0/3441197866 wait complete. 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.029+0000 7f1600349040 1 Processor -- start 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.029+0000 7f1600349040 1 -- start start 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f1600349040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b95fb17e0 0x556b96021a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f1600349040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b9603cb30 0x556b96036570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f1600349040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cf00 0x556b95f6bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f1600349040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b96035650 con 0x556b95fb17e0 2023-10-06T05:22:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f1600349040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b960358f0 con 0x556b9603cb30 2023-10-06T05:22:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f1600349040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b96035b90 con 0x556b9603cf00 2023-10-06T05:22:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b95fb17e0 0x556b96021a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b95fb17e0 0x556b96021a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43122/0 (socket says 172.21.15.45:43122) 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 learned_addr learned my addr 172.21.15.45:0/2118607558 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f8d66700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b9603cb30 0x556b96036570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9d68700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cf00 0x556b95f6bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cf00 msgr2=0x556b95f6bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556b9603cf00 0x556b95f6bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b9603cb30 msgr2=0x556b96036570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b9603cb30 0x556b96036570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.030+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f400d040 con 0x556b95fb17e0 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.031+0000 7f15f9567700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b95fb17e0 0x556b96021a50 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f15ec013000 tx=0x7f15ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.031+0000 7f15e9ffb700 1 -- 172.21.15.45:0/2118607558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec008050 con 0x556b95fb17e0 2023-10-06T05:22:46.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.031+0000 7f15e9ffb700 1 -- 172.21.15.45:0/2118607558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec021070 con 0x556b95fb17e0 2023-10-06T05:22:46.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.031+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b95ec33d0 con 0x556b95fb17e0 2023-10-06T05:22:46.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.031+0000 7f15e9ffb700 1 -- 172.21.15.45:0/2118607558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec008420 con 0x556b95fb17e0 2023-10-06T05:22:46.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.031+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b95eb6ae0 con 0x556b95fb17e0 2023-10-06T05:22:46.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.032+0000 7f15e9ffb700 1 -- 172.21.15.45:0/2118607558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15ec023030 con 0x556b95fb17e0 2023-10-06T05:22:46.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.033+0000 7f15e9ffb700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15dc065890 0x7f15dc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.033+0000 7f15e9ffb700 1 -- 172.21.15.45:0/2118607558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f15ec0846d0 con 0x556b95fb17e0 2023-10-06T05:22:46.040 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:22:46.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.033+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x556b95f6c100 0x556b96051a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.033+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96051f40 con 0x556b95f6c100 2023-10-06T05:22:46.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.033+0000 7f15f8d66700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15dc065890 0x7f15dc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.033+0000 7f15f9d68700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x556b95f6c100 0x556b96051a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.034+0000 7f15f8d66700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15dc065890 0x7f15dc067d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f0001780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.034+0000 7f15f9d68700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x556b95f6c100 0x556b96051a00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.035+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v98'164 uv138 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.035+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605cf30 con 0x556b95f6c100 2023-10-06T05:22:46.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.037+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v98'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.037+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605d2e0 con 0x556b95f6c100 2023-10-06T05:22:46.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.038+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v98'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.043 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:22:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.038+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605e6a0 con 0x556b95f6c100 2023-10-06T05:22:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.039+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v98'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.039+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605ea50 con 0x556b95f6c100 2023-10-06T05:22:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.040+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v98'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.040+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605d690 con 0x556b95f6c100 2023-10-06T05:22:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.041+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v98'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.041+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605da90 con 0x556b95f6c100 2023-10-06T05:22:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.042+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v98'170 uv144 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.042+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de90 con 0x556b95f6c100 2023-10-06T05:22:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.044+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v98'171 uv171 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.044+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de90 con 0x556b95f6c100 2023-10-06T05:22:46.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.045+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v98'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.049 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:22:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.045+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.046+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v98'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.046+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.047+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v98'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.047+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.049+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v98'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.049+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.050+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v98'176 uv151 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.050+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.051+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v98'177 uv177 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.051+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.054 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:22:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.052+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v98'178 uv178 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.052+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.053+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v98'179 uv179 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.053+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.055+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v98'180 uv180 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.055+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.056+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v98'181 uv181 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.056+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.057+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v98'182 uv182 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.057+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.058+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v98'183 uv157 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.058+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.059+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v98'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.060+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.064 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:22:46.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.061+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v98'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.061+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.063+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v98'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.063+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.064+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v98'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.064+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.065+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v98'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.065+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.066+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v98'189 uv163 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.066+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.068+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v98'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.068+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.070 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:22:46.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.069+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v98'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.069+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.070+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v98'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.070+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.071+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v98'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.071+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b95f6c100 2023-10-06T05:22:46.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.072+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v98'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b95f6c100 2023-10-06T05:22:46.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.072+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556b9605e240 0x556b96061320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.072+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9605de40 con 0x556b9605e240 2023-10-06T05:22:46.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.073+0000 7f15f9567700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556b9605e240 0x556b96061320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.073+0000 7f15f9567700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556b9605e240 0x556b96061320 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.074+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v98'128 uv112 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.075+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96061860 con 0x556b9605e240 2023-10-06T05:22:46.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.076+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v98'129 uv129 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.076+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96061c10 con 0x556b9605e240 2023-10-06T05:22:46.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.077+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v98'130 uv130 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.079 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:22:46.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.077+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96061fc0 con 0x556b9605e240 2023-10-06T05:22:46.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.078+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v98'131 uv131 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.078+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96033760 con 0x556b9605e240 2023-10-06T05:22:46.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.080+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v98'132 uv132 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.080+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062860 con 0x556b9605e240 2023-10-06T05:22:46.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.081+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v98'133 uv115 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.081+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062c10 con 0x556b9605e240 2023-10-06T05:22:46.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.082+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v98'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.082+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062c10 con 0x556b9605e240 2023-10-06T05:22:46.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.084+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v98'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.085 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:22:46.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.084+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062c10 con 0x556b9605e240 2023-10-06T05:22:46.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.085+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v98'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.085+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96063060 con 0x556b9605e240 2023-10-06T05:22:46.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.086+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v98'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.086+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96063060 con 0x556b9605e240 2023-10-06T05:22:46.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.088+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v98'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.089 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:22:46.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.088+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96066330 con 0x556b9605e240 2023-10-06T05:22:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.089+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [write 0~14075] v98'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.089+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96066730 con 0x556b9605e240 2023-10-06T05:22:46.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.090+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v98'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.090+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062c10 con 0x556b9605e240 2023-10-06T05:22:46.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.091+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 606.00000000 [create] v98'141 uv126 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.091+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062c10 con 0x556b9605e240 2023-10-06T05:22:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.092+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 606.00000000 [delete] v98'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.092+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062c10 con 0x556b9605e240 2023-10-06T05:22:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.094+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v98'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.095 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:22:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.094+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96063c40 con 0x556b9605e240 2023-10-06T05:22:46.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.095+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v98'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.095+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96063c40 con 0x556b9605e240 2023-10-06T05:22:46.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.098+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v98'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.098+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96063c40 con 0x556b9605e240 2023-10-06T05:22:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.099+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v98'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.099+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556b96063120 0x556b960634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.099+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:51 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96066060 con 0x556b96063120 2023-10-06T05:22:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.099+0000 7f15f8d66700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556b96063120 0x556b960634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.100+0000 7f15f8d66700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556b96063120 0x556b960634f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.101+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(51 607.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.101+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:52 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96066410 con 0x556b96063120 2023-10-06T05:22:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.103+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(52 607.00000000 [delete] v98'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.103+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b960667c0 con 0x556b96063120 2023-10-06T05:22:46.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.104+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(53 607.00000000 [create] v98'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.106 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:22:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.104+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96062420 con 0x556b96063120 2023-10-06T05:22:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.105+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.105+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96068140 con 0x556b96063120 2023-10-06T05:22:46.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.106+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.106+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067060 con 0x556b96063120 2023-10-06T05:22:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.107+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.107+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067460 con 0x556b96063120 2023-10-06T05:22:46.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.109+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(57 mds0_openfiles.0 [create] v98'24 uv23 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.109+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067890 con 0x556b96063120 2023-10-06T05:22:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.110+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v98'25 uv25 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.110+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067890 con 0x556b96063120 2023-10-06T05:22:46.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:22:46.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.111+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(59 mds0_openfiles.0 [create] v98'26 uv26 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.111+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067840 con 0x556b96063120 2023-10-06T05:22:46.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.112+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v98'27 uv27 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.112+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:61 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067810 con 0x556b9605e240 2023-10-06T05:22:46.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.114+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(61 608.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.114+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:62 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067810 con 0x556b9605e240 2023-10-06T05:22:46.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.116+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(62 608.00000000 [delete] v98'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.116+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067810 con 0x556b9605e240 2023-10-06T05:22:46.118 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:22:46.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.117+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v98'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.117+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067810 con 0x556b9605e240 2023-10-06T05:22:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.118+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.118+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067810 con 0x556b9605e240 2023-10-06T05:22:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.119+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.119+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96067810 con 0x556b9605e240 2023-10-06T05:22:46.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.121+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9605e240 2023-10-06T05:22:46.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.121+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x556b96067810 0x556b9606aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.121+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:67 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b010 con 0x556b96067810 2023-10-06T05:22:46.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.121+0000 7f15f9d68700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x556b96067810 0x556b9606aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.122+0000 7f15f9d68700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x556b96067810 0x556b9606aad0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.123+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(67 604.00000000 [create] v98'59 uv53 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.123+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:68 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b0d0 con 0x556b96067810 2023-10-06T05:22:46.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.125+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(68 604.00000000 [delete] v98'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.126+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b0d0 con 0x556b96067810 2023-10-06T05:22:46.130 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:22:46.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.128+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(69 604.00000000 [create] v98'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.128+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606c530 con 0x556b96067810 2023-10-06T05:22:46.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.131+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v98'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.131+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606c530 con 0x556b96067810 2023-10-06T05:22:46.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.134+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v98'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.134+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b520 con 0x556b96067810 2023-10-06T05:22:46.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.138+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v98'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.138+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b520 con 0x556b96067810 2023-10-06T05:22:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.140+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(73 500.00000000 [create] v98'65 uv58 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.140+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b520 con 0x556b96067810 2023-10-06T05:22:46.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.143+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(74 500.00000000 [delete] v98'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.143+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b520 con 0x556b96067810 2023-10-06T05:22:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:22:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.148+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(75 500.00000000 [create] v98'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.148+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b520 con 0x556b96067810 2023-10-06T05:22:46.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.149+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v98'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.149+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606b520 con 0x556b96067810 2023-10-06T05:22:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.152+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v98'69 uv69 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.152+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x556b9606ba30 0x556b9606de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.152+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:78 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606e550 con 0x556b9606ba30 2023-10-06T05:22:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.152+0000 7f15f9567700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x556b9606ba30 0x556b9606de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.153+0000 7f15f9567700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x556b9606ba30 0x556b9606de30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.154+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(78 mds_snaptable [create] v98'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9606ba30 2023-10-06T05:22:46.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.154+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:79 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606e550 con 0x556b9606ba30 2023-10-06T05:22:46.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.155+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(79 mds_snaptable [delete] v98'28 uv28 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9606ba30 2023-10-06T05:22:46.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.155+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606e550 con 0x556b9606ba30 2023-10-06T05:22:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:22:46.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.158+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(80 mds_snaptable [create] v98'29 uv29 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9606ba30 2023-10-06T05:22:46.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.158+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606ea00 con 0x556b9606ba30 2023-10-06T05:22:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.161+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v98'30 uv30 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9606ba30 2023-10-06T05:22:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.161+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606ea00 con 0x556b9606ba30 2023-10-06T05:22:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.164+0000 7f15f9567700 1 -- 172.21.15.45:0/2118607558 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v98'31 uv31 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15ec0827c0 con 0x556b9606ba30 2023-10-06T05:22:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.164+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:83 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606ea00 con 0x556b96063120 2023-10-06T05:22:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.165+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(83 605.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.165+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:84 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606ea00 con 0x556b96063120 2023-10-06T05:22:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.166+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(84 605.00000000 [delete] v98'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.166+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606ea00 con 0x556b96063120 2023-10-06T05:22:46.169 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:22:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.167+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(85 605.00000000 [create] v98'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.167+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606fa10 con 0x556b96063120 2023-10-06T05:22:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.169+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.169+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606fa10 con 0x556b96063120 2023-10-06T05:22:46.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.170+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.170+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b9606ea00 con 0x556b96063120 2023-10-06T05:22:46.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.172+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.172+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x556b9606f080 0x556b96071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.172+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:89 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071a30 con 0x556b9606f080 2023-10-06T05:22:46.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.172+0000 7f15f8d66700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x556b9606f080 0x556b96071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.172+0000 7f15f8d66700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x556b9606f080 0x556b96071460 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.173+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(89 mds0_inotable [create] v98'54 uv48 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.173+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:90 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071af0 con 0x556b9606f080 2023-10-06T05:22:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.176+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(90 mds0_inotable [delete] v98'55 uv55 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.176+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071af0 con 0x556b9606f080 2023-10-06T05:22:46.180 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:22:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.179+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(91 mds0_inotable [create] v98'56 uv56 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.179+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071f40 con 0x556b9606f080 2023-10-06T05:22:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.182+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(92 mds0_inotable [write 0~34] v98'57 uv57 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.182+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071f40 con 0x556b9606f080 2023-10-06T05:22:46.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.184+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v98'58 uv58 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.185+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071f40 con 0x556b9606f080 2023-10-06T05:22:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.187+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(94 100.00000000 [create] v98'59 uv53 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.187+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071f40 con 0x556b9606f080 2023-10-06T05:22:46.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.193+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(95 100.00000000 [delete] v98'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.193+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071f40 con 0x556b9606f080 2023-10-06T05:22:46.196 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:22:46.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.194+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(96 100.00000000 [create] v98'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.194+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071f40 con 0x556b9606f080 2023-10-06T05:22:46.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.197+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(97 100.00000000 [omap-set-header] v98'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.197+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96074cb0 con 0x556b9606f080 2023-10-06T05:22:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.202+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v98'63 uv63 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b9606f080 2023-10-06T05:22:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.202+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96072430 con 0x556b96063120 2023-10-06T05:22:46.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.204+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v98'26 uv25 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.204+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96072430 con 0x556b96063120 2023-10-06T05:22:46.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.205+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v98'27 uv27 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.205+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96072430 con 0x556b96063120 2023-10-06T05:22:46.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.206+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v98'28 uv28 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.208 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:22:46.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.207+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96072430 con 0x556b96063120 2023-10-06T05:22:46.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.208+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v98'29 uv29 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.208+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x556b96072940 0x556b96074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.208+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96072940 2023-10-06T05:22:46.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.208+0000 7f15f9d68700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x556b96072940 0x556b96074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.209+0000 7f15f9d68700 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x556b96072940 0x556b96074d20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.210+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96072940 2023-10-06T05:22:46.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.210+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96072940 2023-10-06T05:22:46.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.211+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v98'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96072940 2023-10-06T05:22:46.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.211+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96072940 2023-10-06T05:22:46.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.213+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v98'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96072940 2023-10-06T05:22:46.215 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:22:46.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.213+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96076270 con 0x556b96072940 2023-10-06T05:22:46.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.216+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96072940 2023-10-06T05:22:46.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.216+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96076270 con 0x556b96072940 2023-10-06T05:22:46.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.217+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96072940 2023-10-06T05:22:46.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.217+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96072940 2023-10-06T05:22:46.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.220+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96072940 2023-10-06T05:22:46.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.220+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96067810 2023-10-06T05:22:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.221+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v98'27 uv26 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.221+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96067810 2023-10-06T05:22:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.223+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v98'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.223+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96067810 2023-10-06T05:22:46.226 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:22:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.225+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v98'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.225+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075260 con 0x556b96067810 2023-10-06T05:22:46.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.228+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v98'30 uv30 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.228+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96067810 2023-10-06T05:22:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.230+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v98'31 uv31 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f15f4018020 con 0x556b96067810 2023-10-06T05:22:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.230+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.232+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v98'69 uv61 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.232+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.233+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v98'70 uv70 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.233+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.234+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v98'71 uv71 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.236 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:22:46.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.234+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.237+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v98'72 uv72 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.237+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.240+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v98'73 uv73 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.240+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.243+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v98'74 uv68 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.243+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.245+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v98'75 uv75 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.245+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.248+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v98'76 uv76 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.249 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:22:46.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.248+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.250+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v98'77 uv77 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.250+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96071fc0 con 0x556b96063120 2023-10-06T05:22:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.251+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v98'78 uv78 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.251+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075750 con 0x556b96063120 2023-10-06T05:22:46.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.252+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v98'79 uv79 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.252+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v8 -- 0x556b96075750 con 0x556b96063120 2023-10-06T05:22:46.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f15f8d66700 1 -- 172.21.15.45:0/2118607558 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v98'80 uv80 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f15f0017040 con 0x556b96063120 2023-10-06T05:22:46.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556b96063120 msgr2=0x556b960634f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556b96063120 0x556b960634f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x556b95f6c100 msgr2=0x556b96051a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x556b95f6c100 0x556b96051a00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x556b9606f080 msgr2=0x556b96071460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x556b9606f080 0x556b96071460 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556b9605e240 msgr2=0x556b96061320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x556b9605e240 0x556b96061320 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x556b96072940 msgr2=0x556b96074d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x556b96072940 0x556b96074d20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x556b9606ba30 msgr2=0x556b9606de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x556b9606ba30 0x556b9606de30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x556b96067810 msgr2=0x556b9606aad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x556b96067810 0x556b9606aad0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 reap_dead start 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 reap_dead start 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 reap_dead start 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f15f9d68700 1 -- 172.21.15.45:0/2118607558 reap_dead start 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15dc065890 msgr2=0x7f15dc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15dc065890 0x7f15dc067d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f0001780 comp rx=0 tx=0).stop 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b95fb17e0 msgr2=0x556b96021a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b95fb17e0 0x556b96021a50 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f15ec013000 tx=0x7f15ec019000 comp rx=0 tx=0).stop 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 shutdown_connections 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b95fb17e0 0x556b96021a50 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 --2- 172.21.15.45:0/2118607558 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15dc065890 0x7f15dc067d40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 >> 172.21.15.45:0/2118607558 conn(0x556b95f6a450 msgr2=0x556b95f67210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 shutdown_connections 2023-10-06T05:22:46.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.254+0000 7f1600349040 1 -- 172.21.15.45:0/2118607558 wait complete. 2023-10-06T05:22:46.260 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2023-10-06T05:22:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 Processor -- start 2023-10-06T05:22:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 -- start start 2023-10-06T05:22:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411527e0 0x5622411db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddb30 0x5622411eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411ddf00 0x5622411edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562241055980 con 0x5622411527e0 2023-10-06T05:22:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562241055620 con 0x5622411ddf00 2023-10-06T05:22:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.308+0000 7f3680fe7040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56224104c400 con 0x5622411ddb30 2023-10-06T05:22:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411527e0 0x5622411db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411527e0 0x5622411db9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43138/0 (socket says 172.21.15.45:43138) 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 -- 172.21.15.45:0/2065353434 learned_addr learned my addr 172.21.15.45:0/2065353434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f3679a04700 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddb30 0x5622411eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367aa06700 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411ddf00 0x5622411edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 -- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddb30 msgr2=0x5622411eda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddb30 0x5622411eda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 -- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411ddf00 msgr2=0x5622411edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411ddf00 0x5622411edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 -- 172.21.15.45:0/2065353434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c005cf0 con 0x5622411527e0 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367aa06700 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411ddf00 0x5622411edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f367a205700 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411527e0 0x5622411db9d0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f366c00acf0 tx=0x7f366c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb427141215e938 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f3678a02700 1 -- 172.21.15.45:0/2065353434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c000bc0 con 0x5622411527e0 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f3678a02700 1 -- 172.21.15.45:0/2065353434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c008970 con 0x5622411527e0 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.309+0000 7f3678a02700 1 -- 172.21.15.45:0/2065353434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c000d20 con 0x5622411527e0 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 -- 172.21.15.45:0/2065353434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411527e0 msgr2=0x5622411db9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411527e0 0x5622411db9d0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f366c00acf0 tx=0x7f366c016000 comp rx=0 tx=0).stop 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 -- 172.21.15.45:0/2065353434 shutdown_connections 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411ddf00 0x5622411edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411527e0 0x5622411db9d0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2065353434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddb30 0x5622411eda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 -- 172.21.15.45:0/2065353434 >> 172.21.15.45:0/2065353434 conn(0x56224110b450 msgr2=0x562241108210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 -- 172.21.15.45:0/2065353434 shutdown_connections 2023-10-06T05:22:46.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.310+0000 7f3680fe7040 1 -- 172.21.15.45:0/2065353434 wait complete. 2023-10-06T05:22:46.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.311+0000 7f3680fe7040 1 Processor -- start 2023-10-06T05:22:46.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.311+0000 7f3680fe7040 1 -- start start 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.311+0000 7f3680fe7040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411527e0 0x5622411cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.311+0000 7f3680fe7040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411ddb30 0x5622411de390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.311+0000 7f3680fe7040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddf00 0x5622411de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3680fe7040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622411383b0 con 0x5622411ddb30 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3680fe7040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562241138650 con 0x5622411527e0 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3680fe7040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622411388f0 con 0x5622411ddf00 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411ddb30 0x5622411de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411ddb30 0x5622411de390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43152/0 (socket says 172.21.15.45:43152) 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 -- 172.21.15.45:0/2676586035 learned_addr learned my addr 172.21.15.45:0/2676586035 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f367a205700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411527e0 0x5622411cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f367aa06700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddf00 0x5622411de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 -- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddf00 msgr2=0x5622411de8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5622411ddf00 0x5622411de8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 -- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411527e0 msgr2=0x5622411cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622411527e0 0x5622411cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 -- 172.21.15.45:0/2676586035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56224107e1c0 con 0x5622411ddb30 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3679a04700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411ddb30 0x5622411de390 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f3664008ac0 tx=0x7f366400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3672ffd700 1 -- 172.21.15.45:0/2676586035 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400f040 con 0x5622411ddb30 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3672ffd700 1 -- 172.21.15.45:0/2676586035 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3664010620 con 0x5622411ddb30 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3672ffd700 1 -- 172.21.15.45:0/2676586035 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36640109a0 con 0x5622411ddb30 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.312+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5622410643d0 con 0x5622411ddb30 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.313+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562241057ae0 con 0x5622411ddb30 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.314+0000 7f3672ffd700 1 -- 172.21.15.45:0/2676586035 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3664010d50 con 0x5622411ddb30 2023-10-06T05:22:46.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.314+0000 7f3672ffd700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f365c0658c0 0x7f365c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.314+0000 7f3672ffd700 1 -- 172.21.15.45:0/2676586035 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f3664083fb0 con 0x5622411ddb30 2023-10-06T05:22:46.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.315+0000 7f367a205700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f365c0658c0 0x7f365c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.315+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56224114e550 0x5622411d7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.315+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v8 -- 0x5622411d75c0 con 0x56224114e550 2023-10-06T05:22:46.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.315+0000 7f367aa06700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56224114e550 0x5622411d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.315+0000 7f367a205700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f365c0658c0 0x7f365c067d70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c010 tx=0x7f366c01d000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.315+0000 7f367aa06700 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56224114e550 0x5622411d7080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f367aa06700 1 -- 172.21.15.45:0/2676586035 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v98'28 uv28 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3674001630 con 0x56224114e550 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56224114e550 msgr2=0x5622411d7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56224114e550 0x5622411d7080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f365c0658c0 msgr2=0x7f365c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f365c0658c0 0x7f365c067d70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c010 tx=0x7f366c01d000 comp rx=0 tx=0).stop 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411ddb30 msgr2=0x5622411de390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 --2- 172.21.15.45:0/2676586035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5622411ddb30 0x5622411de390 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f3664008ac0 tx=0x7f366400c040 comp rx=0 tx=0).stop 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f367aa06700 1 -- 172.21.15.45:0/2676586035 reap_dead start 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 shutdown_connections 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 >> 172.21.15.45:0/2676586035 conn(0x56224110b450 msgr2=0x5622411e08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 shutdown_connections 2023-10-06T05:22:46.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.317+0000 7f3680fe7040 1 -- 172.21.15.45:0/2676586035 wait complete. 2023-10-06T05:22:46.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:22:46.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.461+0000 7f3a34667700 1 Processor -- start 2023-10-06T05:22:46.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.462+0000 7f3a34667700 1 -- start start 2023-10-06T05:22:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a34667700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a2c0c8b70 0x7f3a2c0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a34667700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c0c3b50 con 0x7f3a2c157330 2023-10-06T05:22:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a34667700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c157700 con 0x7f3a2c0ccb70 2023-10-06T05:22:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a34667700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c0c98e0 con 0x7f3a2c0c8b70 2023-10-06T05:22:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a32403700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3a2c157330 0x7f3a2c0ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50332/0 (socket says 172.21.15.45:50332) 2023-10-06T05:22:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a32403700 1 -- 172.21.15.45:0/1051333963 learned_addr learned my addr 172.21.15.45:0/1051333963 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:46.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a32c04700 1 --2- 172.21.15.45:0/1051333963 >> v2:172.21.15.45:3300/0 conn(0x7f3a2c0c8b70 0x7f3a2c0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.464+0000 7f3a32c04700 1 -- 172.21.15.45:0/1051333963 >> v1:172.21.15.144:6789/0 conn(0x7f3a2c0ccb70 legacy=0x7f3a2c0aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.464+0000 7f3a32c04700 1 -- 172.21.15.45:0/1051333963 >> v1:172.21.15.45:6789/0 conn(0x7f3a2c157330 legacy=0x7f3a2c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.464+0000 7f3a32c04700 1 -- 172.21.15.45:0/1051333963 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0c9bc0 con 0x7f3a2c0c8b70 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.463+0000 7f3a31401700 1 -- 172.21.15.45:0/1051333963 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598259637 0 0) 0x7f3a2c0c3b50 con 0x7f3a2c157330 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.464+0000 7f3a32c04700 1 --2- 172.21.15.45:0/1051333963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c0c8b70 0x7f3a2c0c8f90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009de0 tx=0x7f3a24009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a275d19640510b31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.464+0000 7f3a31401700 1 -- 172.21.15.45:0/1051333963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400e070 con 0x7f3a2c0c8b70 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.464+0000 7f3a31401700 1 -- 172.21.15.45:0/1051333963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2400bbb0 con 0x7f3a2c0c8b70 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.464+0000 7f3a31401700 1 -- 172.21.15.45:0/1051333963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24012430 con 0x7f3a2c0c8b70 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.465+0000 7f3a34667700 1 -- 172.21.15.45:0/1051333963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c0c8b70 msgr2=0x7f3a2c0c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.465+0000 7f3a34667700 1 --2- 172.21.15.45:0/1051333963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c0c8b70 0x7f3a2c0c8f90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009de0 tx=0x7f3a24009780 comp rx=0 tx=0).stop 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.465+0000 7f3a34667700 1 -- 172.21.15.45:0/1051333963 shutdown_connections 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.465+0000 7f3a34667700 1 --2- 172.21.15.45:0/1051333963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c0c8b70 0x7f3a2c0c8f90 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.465+0000 7f3a34667700 1 -- 172.21.15.45:0/1051333963 >> 172.21.15.45:0/1051333963 conn(0x7f3a2c067e00 msgr2=0x7f3a2c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.465+0000 7f3a34667700 1 -- 172.21.15.45:0/1051333963 shutdown_connections 2023-10-06T05:22:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.465+0000 7f3a34667700 1 -- 172.21.15.45:0/1051333963 wait complete. 2023-10-06T05:22:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.466+0000 7f3a34667700 1 Processor -- start 2023-10-06T05:22:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.466+0000 7f3a34667700 1 -- start start 2023-10-06T05:22:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a34667700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2c0c8b70 0x7f3a2c1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a34667700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a2c0ccb70 0x7f3a2c0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a34667700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c157330 0x7f3a2c16abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a34667700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0c7ff0 con 0x7f3a2c157330 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a34667700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15a1e0 con 0x7f3a2c0ccb70 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a34667700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c159140 con 0x7f3a2c0c8b70 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a32c04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c157330 0x7f3a2c16abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a32c04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c157330 0x7f3a2c16abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37590/0 (socket says 172.21.15.45:37590) 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a32c04700 1 -- 172.21.15.45:0/2772708318 learned_addr learned my addr 172.21.15.45:0/2772708318 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a31c02700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a2c0ccb70 0x7f3a2c0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.467+0000 7f3a32403700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2c0c8b70 0x7f3a2c1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a32c04700 1 -- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2c0c8b70 msgr2=0x7f3a2c1588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a32c04700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2c0c8b70 0x7f3a2c1588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a32c04700 1 -- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a2c0ccb70 msgr2=0x7f3a2c0b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a32c04700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a2c0ccb70 0x7f3a2c0b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a32c04700 1 -- 172.21.15.45:0/2772708318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a24009a20 con 0x7f3a2c157330 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a32c04700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c157330 0x7f3a2c16abd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3a240099f0 tx=0x7f3a24018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a1e7fc700 1 -- 172.21.15.45:0/2772708318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2401e430 con 0x7f3a2c157330 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a1e7fc700 1 -- 172.21.15.45:0/2772708318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2401e5b0 con 0x7f3a2c157330 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a1e7fc700 1 -- 172.21.15.45:0/2772708318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2401e970 con 0x7f3a2c157330 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c159420 con 0x7f3a2c157330 2023-10-06T05:22:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.468+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c159a60 con 0x7f3a2c157330 2023-10-06T05:22:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.469+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c06e350 con 0x7f3a2c157330 2023-10-06T05:22:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.469+0000 7f3a1e7fc700 1 -- 172.21.15.45:0/2772708318 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a2401d3c0 con 0x7f3a2c157330 2023-10-06T05:22:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.470+0000 7f3a1e7fc700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a10066f30 0x7f3a100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.470+0000 7f3a1e7fc700 1 -- 172.21.15.45:0/2772708318 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f3a24086b00 con 0x7f3a2c157330 2023-10-06T05:22:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.470+0000 7f3a32403700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a10066f30 0x7f3a100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.471+0000 7f3a32403700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a10066f30 0x7f3a100693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3a28004c50 tx=0x7f3a28007cb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.475+0000 7f3a1e7fc700 1 -- 172.21.15.45:0/2772708318 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a24053980 con 0x7f3a2c157330 2023-10-06T05:22:46.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.723+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f3a2c0c37e0 con 0x7f3a2c157330 2023-10-06T05:22:46.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.965+0000 7f3a1e7fc700 1 -- 172.21.15.45:0/2772708318 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v125) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3a24056fa0 con 0x7f3a2c157330 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a10066f30 msgr2=0x7f3a100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a10066f30 0x7f3a100693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3a28004c50 tx=0x7f3a28007cb0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c157330 msgr2=0x7f3a2c16abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c157330 0x7f3a2c16abd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3a240099f0 tx=0x7f3a24018000 comp rx=0 tx=0).stop 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 shutdown_connections 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3a10066f30 0x7f3a100693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a2c157330 0x7f3a2c16abd0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2c0c8b70 0x7f3a2c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 --2- 172.21.15.45:0/2772708318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a2c0ccb70 0x7f3a2c0b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 >> 172.21.15.45:0/2772708318 conn(0x7f3a2c067e00 msgr2=0x7f3a2c1528a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 shutdown_connections 2023-10-06T05:22:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:46.969+0000 7f3a34667700 1 -- 172.21.15.45:0/2772708318 wait complete. 2023-10-06T05:22:46.972 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:22:46.970+0000 7f5d66d60700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-06T05:22:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:22:47.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.530+0000 7fa0c88b6700 1 Processor -- start 2023-10-06T05:22:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.530+0000 7fa0c88b6700 1 -- start start 2023-10-06T05:22:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c88b6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa0c4159330 0x7fa0c406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c88b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0c40cc390 0x7fa0c40cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c88b6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40cb530 con 0x7fa0c4158f60 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c88b6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c4069bd0 con 0x7fa0c40cc390 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c88b6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40ccd10 con 0x7fa0c4159330 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c2d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0c40cc390 0x7fa0c40cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c2d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0c40cc390 0x7fa0c40cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37594/0 (socket says 172.21.15.45:37594) 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.531+0000 7fa0c2d9d700 1 -- 172.21.15.45:0/2736484371 learned_addr learned my addr 172.21.15.45:0/2736484371 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c1d9b700 1 --2- 172.21.15.45:0/2736484371 >> v2:172.21.15.144:3300/0 conn(0x7fa0c4159330 0x7fa0c406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c2d9d700 1 -- 172.21.15.45:0/2736484371 >> v2:172.21.15.144:3300/0 conn(0x7fa0c4159330 msgr2=0x7fa0c406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c2d9d700 1 --2- 172.21.15.45:0/2736484371 >> v2:172.21.15.144:3300/0 conn(0x7fa0c4159330 0x7fa0c406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c2d9d700 1 -- 172.21.15.45:0/2736484371 >> v1:172.21.15.144:6789/0 conn(0x7fa0c4158f60 legacy=0x7fa0c40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c2d9d700 1 -- 172.21.15.45:0/2736484371 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4068fc0 con 0x7fa0c40cc390 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c2d9d700 1 --2- 172.21.15.45:0/2736484371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cc390 0x7fa0c40cc7d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40872e0 tx=0x7fa0bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43d3d9b103d7996 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c159a700 1 -- 172.21.15.45:0/2736484371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00e070 con 0x7fa0c40cc390 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c159a700 1 -- 172.21.15.45:0/2736484371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00ba50 con 0x7fa0c40cc390 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.532+0000 7fa0c159a700 1 -- 172.21.15.45:0/2736484371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0092e0 con 0x7fa0c40cc390 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 -- 172.21.15.45:0/2736484371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cc390 msgr2=0x7fa0c40cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/2736484371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cc390 0x7fa0c40cc7d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40872e0 tx=0x7fa0bc009840 comp rx=0 tx=0).stop 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 -- 172.21.15.45:0/2736484371 shutdown_connections 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/2736484371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c40cc390 0x7fa0c40cc7d0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/2736484371 >> v2:172.21.15.144:3300/0 conn(0x7fa0c4159330 0x7fa0c406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 -- 172.21.15.45:0/2736484371 >> 172.21.15.45:0/2736484371 conn(0x7fa0c40cdf50 msgr2=0x7fa0c40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:47.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 -- 172.21.15.45:0/2736484371 shutdown_connections 2023-10-06T05:22:47.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.533+0000 7fa0c88b6700 1 -- 172.21.15.45:0/2736484371 wait complete. 2023-10-06T05:22:47.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.535+0000 7fa0c88b6700 1 Processor -- start 2023-10-06T05:22:47.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.535+0000 7fa0c88b6700 1 -- start start 2023-10-06T05:22:47.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c88b6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cc390 0x7fa0c4169460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c88b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c4158f60 0x7fa0c4154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c88b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c4159330 0x7fa0c40b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c88b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b3010 con 0x7fa0c4158f60 2023-10-06T05:22:47.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c88b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b2b80 con 0x7fa0c4159330 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c88b6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b1f70 con 0x7fa0c40cc390 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c4158f60 0x7fa0c4154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c4158f60 0x7fa0c4154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37598/0 (socket says 172.21.15.45:37598) 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 -- 172.21.15.45:0/3359334174 learned_addr learned my addr 172.21.15.45:0/3359334174 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c259c700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cc390 0x7fa0c4169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 -- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cc390 msgr2=0x7fa0c4169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cc390 0x7fa0c4169460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 -- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c4159330 msgr2=0x7fa0c40b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c2d9d700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c4159330 0x7fa0c40b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c4159330 0x7fa0c40b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 -- 172.21.15.45:0/3359334174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009ae0 con 0x7fa0c4158f60 2023-10-06T05:22:47.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c1d9b700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c4158f60 0x7fa0c4154200 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d860 tx=0x7fa0b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0b27fc700 1 -- 172.21.15.45:0/3359334174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8009610 con 0x7fa0c4158f60 2023-10-06T05:22:47.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0b27fc700 1 -- 172.21.15.45:0/3359334174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8009770 con 0x7fa0c4158f60 2023-10-06T05:22:47.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40b2250 con 0x7fa0c4158f60 2023-10-06T05:22:47.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.536+0000 7fa0b27fc700 1 -- 172.21.15.45:0/3359334174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8009b30 con 0x7fa0c4158f60 2023-10-06T05:22:47.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.537+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40b2810 con 0x7fa0c4158f60 2023-10-06T05:22:47.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.537+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c40b2d10 con 0x7fa0c4158f60 2023-10-06T05:22:47.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.538+0000 7fa0b27fc700 1 -- 172.21.15.45:0/3359334174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0b801e460 con 0x7fa0c4158f60 2023-10-06T05:22:47.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.538+0000 7fa0b27fc700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a4066f30 0x7fa0a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.538+0000 7fa0b27fc700 1 -- 172.21.15.45:0/3359334174 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7fa0b8085040 con 0x7fa0c4158f60 2023-10-06T05:22:47.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.538+0000 7fa0c259c700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a4066f30 0x7fa0a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.539+0000 7fa0c259c700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a4066f30 0x7fa0a40693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4063ef0 tx=0x7fa0b4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.543+0000 7fa0b27fc700 1 -- 172.21.15.45:0/3359334174 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b8051e10 con 0x7fa0c4158f60 2023-10-06T05:22:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.789+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0c404bb40 con 0x7fa0c4158f60 2023-10-06T05:22:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.789+0000 7fa0b27fc700 1 -- 172.21.15.45:0/3359334174 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4604 (secure 0 0 0) 0x7fa0b8055430 con 0x7fa0c4158f60 2023-10-06T05:22:47.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:47.791 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:46.966832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7804":{"gid":7804,"name":"c","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/226461717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":226461717},{"type":"v1","addr":"172.21.15.45:6835","nonce":226461717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a4066f30 msgr2=0x7fa0a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a4066f30 0x7fa0a40693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4063ef0 tx=0x7fa0b4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c4158f60 msgr2=0x7fa0c4154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c4158f60 0x7fa0c4154200 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d860 tx=0x7fa0b8008040 comp rx=0 tx=0).stop 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 shutdown_connections 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa0a4066f30 0x7fa0a40693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c4159330 0x7fa0c40b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c40cc390 0x7fa0c4169460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 --2- 172.21.15.45:0/3359334174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c4158f60 0x7fa0c4154200 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 >> 172.21.15.45:0/3359334174 conn(0x7fa0c40cdf50 msgr2=0x7fa0c40ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 shutdown_connections 2023-10-06T05:22:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:47.792+0000 7fa0c88b6700 1 -- 172.21.15.45:0/3359334174 wait complete. 2023-10-06T05:22:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 126 2023-10-06T05:22:47.807 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:22:46.966832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7804}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7804': {'gid': 7804, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/226461717', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 226461717}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 226461717}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:47.807 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7804:up:replay 2023-10-06T05:22:47.807 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:22:53.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.349+0000 7f57ef30b700 1 Processor -- start 2023-10-06T05:22:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.349+0000 7f57ef30b700 1 -- start start 2023-10-06T05:22:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ef30b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f57e80c8440 0x7f57e8160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ef30b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e8067620 con 0x7f57e80ace00 2023-10-06T05:22:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ef30b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e8067be0 con 0x7f57e8155410 2023-10-06T05:22:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ef30b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e8069080 con 0x7f57e80c8440 2023-10-06T05:22:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ec8a6700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f57e80ace00 0x7f57e80c7d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50340/0 (socket says 172.21.15.45:50340) 2023-10-06T05:22:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ec8a6700 1 -- 172.21.15.45:0/1052060431 learned_addr learned my addr 172.21.15.45:0/1052060431 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ed8a8700 1 --2- 172.21.15.45:0/1052060431 >> v2:172.21.15.144:3300/0 conn(0x7f57e80c8440 0x7f57e8160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57dffff700 1 -- 172.21.15.45:0/1052060431 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395217735 0 0) 0x7f57e8067620 con 0x7f57e80ace00 2023-10-06T05:22:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57dffff700 1 -- 172.21.15.45:0/1052060431 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d4003570 con 0x7f57e80ace00 2023-10-06T05:22:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ed8a8700 1 -- 172.21.15.45:0/1052060431 >> v1:172.21.15.144:6789/0 conn(0x7f57e8155410 legacy=0x7f57e8159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ed8a8700 1 -- 172.21.15.45:0/1052060431 >> v1:172.21.15.45:6789/0 conn(0x7f57e80ace00 legacy=0x7f57e80c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.350+0000 7f57ed8a8700 1 -- 172.21.15.45:0/1052060431 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8159ca0 con 0x7f57e80c8440 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57ed8a8700 1 --2- 172.21.15.45:0/1052060431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e80c8440 0x7f57e8160540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009e60 tx=0x7f57e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a638d1ee1fdc307d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57dffff700 1 -- 172.21.15.45:0/1052060431 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000e070 con 0x7f57e80c8440 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57dffff700 1 -- 172.21.15.45:0/1052060431 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000bb60 con 0x7f57e80c8440 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57dffff700 1 -- 172.21.15.45:0/1052060431 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0012430 con 0x7f57e80c8440 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57ef30b700 1 -- 172.21.15.45:0/1052060431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e80c8440 msgr2=0x7f57e8160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57ef30b700 1 --2- 172.21.15.45:0/1052060431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e80c8440 0x7f57e8160540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009e60 tx=0x7f57e0009780 comp rx=0 tx=0).stop 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57ef30b700 1 -- 172.21.15.45:0/1052060431 shutdown_connections 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57ef30b700 1 --2- 172.21.15.45:0/1052060431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e80c8440 0x7f57e8160540 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.351+0000 7f57ef30b700 1 -- 172.21.15.45:0/1052060431 >> 172.21.15.45:0/1052060431 conn(0x7f57e80cd000 msgr2=0x7f57e80c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.352+0000 7f57ef30b700 1 -- 172.21.15.45:0/1052060431 shutdown_connections 2023-10-06T05:22:53.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.352+0000 7f57ef30b700 1 -- 172.21.15.45:0/1052060431 wait complete. 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.353+0000 7f57ef30b700 1 Processor -- start 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.353+0000 7f57ef30b700 1 -- start start 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ef30b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e80ace00 0x7f57e814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ef30b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57e80c8440 0x7f57e814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ef30b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e8155410 0x7f57e80c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ef30b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e814b500 con 0x7f57e80ace00 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ef30b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e814c040 con 0x7f57e8155410 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ef30b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e814b710 con 0x7f57e80c8440 2023-10-06T05:22:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e80ace00 0x7f57e814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e80ace00 0x7f57e814f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37606/0 (socket says 172.21.15.45:37606) 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 -- 172.21.15.45:0/186516903 learned_addr learned my addr 172.21.15.45:0/186516903 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed8a8700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e8155410 0x7f57e80c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ec8a6700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57e80c8440 0x7f57e814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 -- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57e80c8440 msgr2=0x7f57e814ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57e80c8440 0x7f57e814ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 -- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e8155410 msgr2=0x7f57e80c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e8155410 0x7f57e80c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed0a7700 1 -- 172.21.15.45:0/186516903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0009a20 con 0x7f57e80ace00 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.354+0000 7f57ed8a8700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e8155410 0x7f57e80c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.355+0000 7f57ed0a7700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e80ace00 0x7f57e814f910 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f57e400c4e0 tx=0x7f57e400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.355+0000 7f57dd7fa700 1 -- 172.21.15.45:0/186516903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4011590 con 0x7f57e80ace00 2023-10-06T05:22:53.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.355+0000 7f57dd7fa700 1 -- 172.21.15.45:0/186516903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e4011710 con 0x7f57e80ace00 2023-10-06T05:22:53.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.355+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e814b9f0 con 0x7f57e80ace00 2023-10-06T05:22:53.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.355+0000 7f57dd7fa700 1 -- 172.21.15.45:0/186516903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4011ad0 con 0x7f57e80ace00 2023-10-06T05:22:53.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.355+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e814b140 con 0x7f57e80ace00 2023-10-06T05:22:53.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.356+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e814b330 con 0x7f57e80ace00 2023-10-06T05:22:53.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.356+0000 7f57dd7fa700 1 -- 172.21.15.45:0/186516903 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57e401b3b0 con 0x7f57e80ace00 2023-10-06T05:22:53.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.357+0000 7f57dd7fa700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc066f30 0x7f57cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.357+0000 7f57dd7fa700 1 -- 172.21.15.45:0/186516903 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f57e4083f80 con 0x7f57e80ace00 2023-10-06T05:22:53.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.357+0000 7f57ec8a6700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc066f30 0x7f57cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.358+0000 7f57ec8a6700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc066f30 0x7f57cc0693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f57e814ea20 tx=0x7f57d8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.361+0000 7f57dd7fa700 1 -- 172.21.15.45:0/186516903 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57e4050cb0 con 0x7f57e80ace00 2023-10-06T05:22:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.609+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57e80cd400 con 0x7f57e80ace00 2023-10-06T05:22:53.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.610+0000 7f57dd7fa700 1 -- 172.21.15.45:0/186516903 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f57e4017070 con 0x7f57e80ace00 2023-10-06T05:22:53.614 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:53.614 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":129,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:49.975692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7804":{"gid":7804,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/226461717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":226461717},{"type":"v1","addr":"172.21.15.45:6835","nonce":226461717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc066f30 msgr2=0x7f57cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc066f30 0x7f57cc0693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f57e814ea20 tx=0x7f57d8009040 comp rx=0 tx=0).stop 2023-10-06T05:22:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e80ace00 msgr2=0x7f57e814f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e80ace00 0x7f57e814f910 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f57e400c4e0 tx=0x7f57e400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 shutdown_connections 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57cc066f30 0x7f57cc0693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e8155410 0x7f57e80c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e80ace00 0x7f57e814f910 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 --2- 172.21.15.45:0/186516903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57e80c8440 0x7f57e814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 >> 172.21.15.45:0/186516903 conn(0x7f57e80cd000 msgr2=0x7f57e80b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.612+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 shutdown_connections 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.613+0000 7f57ef30b700 1 -- 172.21.15.45:0/186516903 wait complete. 2023-10-06T05:22:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 129 2023-10-06T05:22:53.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 129, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:22:49.975692+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7804}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7804': {'gid': 7804, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/226461717', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 226461717}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 226461717}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:53.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:22:53.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:22:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.760+0000 7f9450ab4700 1 Processor -- start 2023-10-06T05:22:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.760+0000 7f9450ab4700 1 -- start start 2023-10-06T05:22:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f9450ab4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 0x7f944c0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f9450ab4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f944c0ccdc0 0x7f944c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f9450ab4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944c0c7d00 con 0x7f944c1595d0 2023-10-06T05:22:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f9450ab4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c0c7780 con 0x7f944c0ccdc0 2023-10-06T05:22:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f9450ab4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0cd740 con 0x7f944c159200 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f9449d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f944c1595d0 0x7f944c06b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50352/0 (socket says 172.21.15.45:50352) 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f9449d9b700 1 -- 172.21.15.45:0/1839710555 learned_addr learned my addr 172.21.15.45:0/1839710555 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944a59c700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 0x7f944c0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944ad9d700 1 --2- 172.21.15.45:0/1839710555 >> v2:172.21.15.144:3300/0 conn(0x7f944c0ccdc0 0x7f944c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536991689 0 0) 0x7f944c0c7d00 con 0x7f944c1595d0 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9434003570 con 0x7f944c1595d0 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4222846700 0 0) 0x7f9434003570 con 0x7f944c1595d0 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 msgr2=0x7f944c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944959a700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 0x7f944c0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 >> v2:172.21.15.144:3300/0 conn(0x7f944c0ccdc0 msgr2=0x7f944c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.761+0000 7f944959a700 1 --2- 172.21.15.45:0/1839710555 >> v2:172.21.15.144:3300/0 conn(0x7f944c0ccdc0 0x7f944c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c0c86d0 con 0x7f944c1595d0 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f9440001470 con 0x7f944c1595d0 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 >> v1:172.21.15.45:6789/0 conn(0x7f944c1595d0 legacy=0x7f944c06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9434003a60 0x7f9434003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 0x7f9434004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434004d50 0x7f94340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0c7780 con 0x7f9434003a60 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0cd740 con 0x7f9434004d50 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943400ba60 con 0x7f944c159200 2023-10-06T05:22:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944a59c700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9434003a60 0x7f9434003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f9449d9b700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 0x7f9434004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944ad9d700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434004d50 0x7f94340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944a59c700 1 -- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 msgr2=0x7f9434004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944a59c700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f944c159200 0x7f9434004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944a59c700 1 -- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434004d50 msgr2=0x7f94340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944a59c700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434004d50 0x7f94340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944a59c700 1 -- 172.21.15.45:0/1839710555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434003570 con 0x7f9434003a60 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944ad9d700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434004d50 0x7f94340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944a59c700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9434003a60 0x7f9434003e70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f943c009c00 tx=0x7f943c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f97921571cf8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00e6b0 con 0x7f9434003a60 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c00e810 con 0x7f9434003a60 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.762+0000 7f944959a700 1 -- 172.21.15.45:0/1839710555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00ebd0 con 0x7f9434003a60 2023-10-06T05:22:53.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.763+0000 7f9450ab4700 1 -- 172.21.15.45:0/1839710555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9434003a60 msgr2=0x7f9434003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.763+0000 7f9450ab4700 1 --2- 172.21.15.45:0/1839710555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9434003a60 0x7f9434003e70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f943c009c00 tx=0x7f943c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.763+0000 7f944ad9d700 1 -- 172.21.15.45:0/1839710555 reap_dead start 2023-10-06T05:22:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.763+0000 7f9450ab4700 1 -- 172.21.15.45:0/1839710555 shutdown_connections 2023-10-06T05:22:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.763+0000 7f9450ab4700 1 -- 172.21.15.45:0/1839710555 >> 172.21.15.45:0/1839710555 conn(0x7f944c06f090 msgr2=0x7f944c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.763+0000 7f9450ab4700 1 -- 172.21.15.45:0/1839710555 shutdown_connections 2023-10-06T05:22:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.763+0000 7f9450ab4700 1 -- 172.21.15.45:0/1839710555 wait complete. 2023-10-06T05:22:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.764+0000 7f9450ab4700 1 Processor -- start 2023-10-06T05:22:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.764+0000 7f9450ab4700 1 -- start start 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f9450ab4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434003a60 0x7f944c1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f9450ab4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f944c154f70 0x7f944c0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f9450ab4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944c155960 0x7f944c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f9450ab4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c167c10 con 0x7f944c155960 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f9450ab4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c156340 con 0x7f944c154f70 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f9450ab4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0c5420 con 0x7f9434003a60 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f944ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944c155960 0x7f944c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f944ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944c155960 0x7f944c155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37620/0 (socket says 172.21.15.45:37620) 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f944ad9d700 1 -- 172.21.15.45:0/3927221843 learned_addr learned my addr 172.21.15.45:0/3927221843 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:53.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f944a59c700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434003a60 0x7f944c1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.765+0000 7f9449d9b700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f944c154f70 0x7f944c0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f944ad9d700 1 -- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434003a60 msgr2=0x7f944c1676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f944ad9d700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434003a60 0x7f944c1676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f944ad9d700 1 -- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f944c154f70 msgr2=0x7f944c0c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f944ad9d700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f944c154f70 0x7f944c0c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f944ad9d700 1 -- 172.21.15.45:0/3927221843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c158660 con 0x7f944c155960 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f944a59c700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434003a60 0x7f944c1676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f9449d9b700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f944c154f70 0x7f944c0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f944ad9d700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944c155960 0x7f944c155e00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f944c0c3dd0 tx=0x7f944400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f943a7fc700 1 -- 172.21.15.45:0/3927221843 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944401b070 con 0x7f944c155960 2023-10-06T05:22:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f943a7fc700 1 -- 172.21.15.45:0/3927221843 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944400f040 con 0x7f944c155960 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c0c5650 con 0x7f944c155960 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f943a7fc700 1 -- 172.21.15.45:0/3927221843 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400e050 con 0x7f944c155960 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.766+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c16a5b0 con 0x7f944c155960 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.768+0000 7f943a7fc700 1 -- 172.21.15.45:0/3927221843 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9444003020 con 0x7f944c155960 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.768+0000 7f943a7fc700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f942c066f30 0x7f942c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.768+0000 7f943a7fc700 1 -- 172.21.15.45:0/3927221843 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f9444084ea0 con 0x7f944c155960 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.769+0000 7f944a59c700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f942c066f30 0x7f942c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.769+0000 7f944a59c700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f942c066f30 0x7f942c0693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f943c0091e0 tx=0x7f943c01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.770+0000 7f9459dc8b80 1 Processor -- start 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f9459dc8b80 1 -- start start 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0985820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f098df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f098e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f0858310 con 0x5562f098e4d0 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f0895130 con 0x5562f0985450 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562f0892af0 con 0x5562f096af40 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f944ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f098e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f944ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f098e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37628/0 (socket says 172.21.15.45:37628) 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.771+0000 7f944ad9d700 1 -- 172.21.15.45:0/962041068 learned_addr learned my addr 172.21.15.45:0/962041068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f944a59c700 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0985820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f9449d9b700 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f098df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f944ad9d700 1 -- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 msgr2=0x5562f098df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f944ad9d700 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f098df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f944ad9d700 1 -- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 msgr2=0x5562f0985820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f944ad9d700 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0985820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f944ad9d700 1 -- 172.21.15.45:0/962041068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f0694790 con 0x5562f098e4d0 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f944ad9d700 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f098e940 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9444082f50 tx=0x7f9444082e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aca71f2970e7a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f942b7fe700 1 -- 172.21.15.45:0/962041068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444058b80 con 0x5562f098e4d0 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f942b7fe700 1 -- 172.21.15.45:0/962041068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944405c8d0 con 0x5562f098e4d0 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.772+0000 7f942b7fe700 1 -- 172.21.15.45:0/962041068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444053cd0 con 0x5562f098e4d0 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 -- 172.21.15.45:0/962041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 msgr2=0x5562f098e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f098e940 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9444082f50 tx=0x7f9444082e80 comp rx=0 tx=0).stop 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 -- 172.21.15.45:0/962041068 shutdown_connections 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f098e940 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0985820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/962041068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f098df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 -- 172.21.15.45:0/962041068 >> 172.21.15.45:0/962041068 conn(0x5562f0988880 msgr2=0x5562f0988c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 -- 172.21.15.45:0/962041068 shutdown_connections 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.773+0000 7f9459dc8b80 1 -- 172.21.15.45:0/962041068 wait complete. 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.774+0000 7f9459dc8b80 1 -- start start 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.774+0000 7f9459dc8b80 1 Processor -- start 2023-10-06T05:22:53.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f096af40 0x5562f0966b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0961340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f098e4d0 0x5562f0984320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f088e8e0 con 0x5562f096af40 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f083a980 con 0x5562f0985450 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562f083a620 con 0x5562f098e4d0 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f9459dc8b80 10 client.0 _ll_register_callbacks cb 0x5562f069da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f944a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f096af40 0x5562f0966b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f944a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f096af40 0x5562f0966b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37644/0 (socket says 172.21.15.45:37644) 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.775+0000 7f944a59c700 1 -- 172.21.15.45:0/2852126038 learned_addr learned my addr 172.21.15.45:0/2852126038 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9449d9b700 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0961340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f944ad9d700 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f098e4d0 0x5562f0984320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f944a59c700 1 -- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f098e4d0 msgr2=0x5562f0984320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f944a59c700 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f098e4d0 0x5562f0984320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f944a59c700 1 -- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 msgr2=0x5562f0961340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f944a59c700 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0961340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f944a59c700 1 -- 172.21.15.45:0/2852126038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c16a5b0 con 0x5562f096af40 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f944a59c700 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f096af40 0x5562f0966b50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f943c016000 tx=0x7f943c01f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d47b67f691fc81b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9428ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9428ff9700 1 -- 172.21.15.45:0/2852126038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c000c90 con 0x5562f096af40 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9428ff9700 1 -- 172.21.15.45:0/2852126038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f943c013ce0 con 0x5562f096af40 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562f0932d60 con 0x5562f096af40 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9428ff9700 1 -- 172.21.15.45:0/2852126038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f943c013ea0 con 0x5562f096af40 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9459dc8b80 10 client.8211 fetch_fsmap learned FSMap version 129 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x5562f07810a0 con 0x5562f096af40 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.776+0000 7f9428ff9700 1 -- 172.21.15.45:0/2852126038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c03f030 con 0x5562f096af40 2023-10-06T05:22:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9428ff9700 1 -- 172.21.15.45:0/2852126038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 129) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f943c005a00 con 0x5562f096af40 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9459dc8b80 10 client.8211 fetch_fsmap finished waiting for FSMap version 129 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9459dc8b80 10 client.8211 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9459dc8b80 20 client.8211 populate_metadata read hostname 'smithi045' 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0987a00 0x5562f0987dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9459dc8b80 4 client.8211 mds_command: new command op to 7804 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 --> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562f090fb30 con 0x5562f0987a00 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9449d9b700 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0987a00 0x5562f0987dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9449d9b700 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0987a00 0x5562f0987dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc63905de7aaa39a server_cookie=570f2f556f9dcf1d in_seq=0 out_seq=0 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.777+0000 7f9428ff9700 10 client.8211 ms_handle_connect on v2:172.21.15.45:6834/226461717 2023-10-06T05:22:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9428ff9700 1 -- 172.21.15.45:0/2852126038 <== mds.0 v2:172.21.15.45:6834/226461717 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5562f090fb30 con 0x5562f0987a00 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9428ff9700 10 client.8211 handle_command_reply: tid=0 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 client.8211 shutdown 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f096af40 msgr2=0x5562f0966b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f096af40 0x5562f0966b50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f943c016000 tx=0x7f943c01f050 comp rx=0 tx=0).stop 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 shutdown_connections 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0987a00 0x5562f0987dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f098e4d0 0x5562f0984320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f096af40 0x5562f0966b50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9428ff9700 0 client.8211 ms_handle_reset on v2:172.21.15.45:6834/226461717 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/2852126038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f0961340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 >> 172.21.15.45:0/2852126038 conn(0x5562f0988880 msgr2=0x5562f093ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 shutdown_connections 2023-10-06T05:22:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 1 -- 172.21.15.45:0/2852126038 wait complete. 2023-10-06T05:22:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.778+0000 7f9459dc8b80 20 client.8211 trim_cache size 0 max 16384 2023-10-06T05:22:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.796+0000 7f9459dc8b80 1 Processor -- start 2023-10-06T05:22:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.796+0000 7f9459dc8b80 1 -- start start 2023-10-06T05:22:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f0985450 0x5562f093bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f098e4d0 0x5562f0960d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f07261e0 con 0x5562f0985450 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f0941120 con 0x5562f098e4d0 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562f0941b50 con 0x5562f096af40 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f0985450 0x5562f093bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f0985450 0x5562f093bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37654/0 (socket says 172.21.15.45:37654) 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 -- 172.21.15.45:0/3472518892 learned_addr learned my addr 172.21.15.45:0/3472518892 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f944ad9d700 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f098e4d0 0x5562f0960d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f944a59c700 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 -- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 msgr2=0x5562f09880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 -- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f098e4d0 msgr2=0x5562f0960d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f098e4d0 0x5562f0960d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 -- 172.21.15.45:0/3472518892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f085d3c0 con 0x5562f0985450 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f944a59c700 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.797+0000 7f9449d9b700 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f0985450 0x5562f093bfd0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9440001050 tx=0x7f9440006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba481ecac09d93f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f942bfff700 1 -- 172.21.15.45:0/3472518892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440004830 con 0x5562f0985450 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f942bfff700 1 -- 172.21.15.45:0/3472518892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440004db0 con 0x5562f0985450 2023-10-06T05:22:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f942bfff700 1 -- 172.21.15.45:0/3472518892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440007760 con 0x5562f0985450 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 -- 172.21.15.45:0/3472518892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f0985450 msgr2=0x5562f093bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f0985450 0x5562f093bfd0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9440001050 tx=0x7f9440006040 comp rx=0 tx=0).stop 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 -- 172.21.15.45:0/3472518892 shutdown_connections 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f098e4d0 0x5562f0960d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/3472518892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f0985450 0x5562f093bfd0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 -- 172.21.15.45:0/3472518892 >> 172.21.15.45:0/3472518892 conn(0x5562f0988880 msgr2=0x5562f0966b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 -- 172.21.15.45:0/3472518892 shutdown_connections 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.798+0000 7f9459dc8b80 1 -- 172.21.15.45:0/3472518892 wait complete. 2023-10-06T05:22:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.799+0000 7f9459dc8b80 1 -- start start 2023-10-06T05:22:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.799+0000 7f9459dc8b80 1 Processor -- start 2023-10-06T05:22:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.800+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09ea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.800+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f099dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.800+0000 7f9459dc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f099e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.800+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f0941400 con 0x5562f098e4d0 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.800+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f09ea7d0 con 0x5562f0985450 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.800+0000 7f9459dc8b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562f09843a0 con 0x5562f096af40 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9459dc8b80 10 client.0 _ll_register_callbacks cb 0x5562f069da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f099e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f099e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37658/0 (socket says 172.21.15.45:37658) 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 -- 172.21.15.45:0/4141986006 learned_addr learned my addr 172.21.15.45:0/4141986006 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944a59c700 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9449d9b700 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f099dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 -- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 msgr2=0x5562f09ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09ea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 -- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 msgr2=0x5562f099dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f099dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 -- 172.21.15.45:0/4141986006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c0c86d0 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f944ad9d700 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f099e220 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f9444010830 tx=0x7f944406b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2d802e61c02fbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9428ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9428ff9700 1 -- 172.21.15.45:0/4141986006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444058b80 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9428ff9700 1 -- 172.21.15.45:0/4141986006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f9444053a70 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f944405c8d0 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9428ff9700 1 -- 172.21.15.45:0/4141986006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=129) v2 ==== 24+0+0 (secure 0 0 0) 0x7f944405c440 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9459dc8b80 10 client.8217 fetch_fsmap learned FSMap version 129 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.801+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=129,monmap=2+}) v3 -- 0x5562f06f51d0 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9428ff9700 1 -- 172.21.15.45:0/4141986006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944405c8d0 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9428ff9700 1 -- 172.21.15.45:0/4141986006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 129) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9444071280 con 0x5562f098e4d0 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9459dc8b80 10 client.8217 fetch_fsmap finished waiting for FSMap version 129 2023-10-06T05:22:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9459dc8b80 10 client.8217 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9459dc8b80 20 client.8217 populate_metadata read hostname 'smithi045' 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0996990 0x5562f0996d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9459dc8b80 4 client.8217 mds_command: new command op to 7804 tid=0[{"prefix": "status"}] 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 --> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5562f08318d0 con 0x5562f0996990 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f944a59c700 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0996990 0x5562f0996d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9428ff9700 10 client.8217 ms_handle_connect on v2:172.21.15.45:6834/226461717 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f944a59c700 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0996990 0x5562f0996d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6009ac06b610b9e server_cookie=ccaef8afb3d913ac in_seq=0 out_seq=0 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9428ff9700 1 -- 172.21.15.45:0/4141986006 <== mds.0 v2:172.21.15.45:6834/226461717 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5562f08318d0 con 0x5562f0996990 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9428ff9700 10 client.8217 handle_command_reply: tid=0 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.802+0000 7f9459dc8b80 1 client.8217 shutdown 2023-10-06T05:22:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 msgr2=0x5562f099e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f099e220 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f9444010830 tx=0x7f944406b000 comp rx=0 tx=0).stop 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 shutdown_connections 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:6834/226461717,v1:172.21.15.45:6835/226461717] conn(0x5562f0996990 0x5562f0996d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f098e4d0 0x5562f099e220 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562f096af40 0x5562f09ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9428ff9700 0 client.8217 ms_handle_reset on v2:172.21.15.45:6834/226461717 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 --2- 172.21.15.45:0/4141986006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562f0985450 0x5562f099dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 >> 172.21.15.45:0/4141986006 conn(0x5562f0988880 msgr2=0x5562f093bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 shutdown_connections 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 1 -- 172.21.15.45:0/4141986006 wait complete. 2023-10-06T05:22:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9459dc8b80 20 client.8217 trim_cache size 0 max 16384 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7804, 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8347257690000003, 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 129, 2023-10-06T05:22:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 98, 2023-10-06T05:22:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 98, 2023-10-06T05:22:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.821038253999999 2023-10-06T05:22:53.813 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f942c066f30 msgr2=0x7f942c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9450ab4700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f942c066f30 0x7f942c0693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f943c0091e0 tx=0x7f943c01a000 comp rx=0 tx=0).stop 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944c155960 msgr2=0x7f944c155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.803+0000 7f9450ab4700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944c155960 0x7f944c155e00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f944c0c3dd0 tx=0x7f944400b040 comp rx=0 tx=0).stop 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 shutdown_connections 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f942c066f30 0x7f942c0693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944c155960 0x7f944c155e00 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434003a60 0x7f944c1676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 --2- 172.21.15.45:0/3927221843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f944c154f70 0x7f944c0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 >> 172.21.15.45:0/3927221843 conn(0x7f944c06f090 msgr2=0x7f944c0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 shutdown_connections 2023-10-06T05:22:53.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.804+0000 7f9450ab4700 1 -- 172.21.15.45:0/3927221843 wait complete. 2023-10-06T05:22:53.818 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:22:53.819 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2023-10-06T05:22:53.819 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.949+0000 7fc1e0a44700 1 Processor -- start 2023-10-06T05:22:53.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.949+0000 7fc1e0a44700 1 -- start start 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1e0a44700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1dc06c3d0 0x7fc1dc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1e0a44700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc15b890 0x7fc1dc0afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1e0a44700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1dc0b0790 con 0x7fc1dc156b70 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1e0a44700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc0c8e80 con 0x7fc1dc06c3d0 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1e0a44700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc0c9160 con 0x7fc1dc15b890 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1da59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1dc06c3d0 0x7fc1dc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1dad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc1dc156b70 0x7fc1dc156ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50354/0 (socket says 172.21.15.45:50354) 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1dad9d700 1 -- 172.21.15.45:0/1486745417 learned_addr learned my addr 172.21.15.45:0/1486745417 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.950+0000 7fc1d9d9b700 1 --2- 172.21.15.45:0/1486745417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc15b890 0x7fc1dc0afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1da59c700 1 -- 172.21.15.45:0/1486745417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc15b890 msgr2=0x7fc1dc0afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:53.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1da59c700 1 --2- 172.21.15.45:0/1486745417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc15b890 0x7fc1dc0afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1da59c700 1 -- 172.21.15.45:0/1486745417 >> v1:172.21.15.45:6789/0 conn(0x7fc1dc156b70 legacy=0x7fc1dc156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1da59c700 1 -- 172.21.15.45:0/1486745417 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc0c9440 con 0x7fc1dc06c3d0 2023-10-06T05:22:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1d959a700 1 -- 172.21.15.45:0/1486745417 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934273158 0 0) 0x7fc1dc0b0790 con 0x7fc1dc156b70 2023-10-06T05:22:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1da59c700 1 --2- 172.21.15.45:0/1486745417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc06c3d0 0x7fc1dc06c7a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0098e0 tx=0x7fc1cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632225bb212f46fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1d959a700 1 -- 172.21.15.45:0/1486745417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc009e80 con 0x7fc1dc06c3d0 2023-10-06T05:22:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1d959a700 1 -- 172.21.15.45:0/1486745417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc00b860 con 0x7fc1dc06c3d0 2023-10-06T05:22:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1d959a700 1 -- 172.21.15.45:0/1486745417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00bbe0 con 0x7fc1dc06c3d0 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1e0a44700 1 -- 172.21.15.45:0/1486745417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc06c3d0 msgr2=0x7fc1dc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.951+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/1486745417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc06c3d0 0x7fc1dc06c7a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0098e0 tx=0x7fc1cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.952+0000 7fc1e0a44700 1 -- 172.21.15.45:0/1486745417 shutdown_connections 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.952+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/1486745417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc06c3d0 0x7fc1dc06c7a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.952+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/1486745417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc15b890 0x7fc1dc0afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.952+0000 7fc1e0a44700 1 -- 172.21.15.45:0/1486745417 >> 172.21.15.45:0/1486745417 conn(0x7fc1dc067620 msgr2=0x7fc1dc067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.952+0000 7fc1e0a44700 1 -- 172.21.15.45:0/1486745417 shutdown_connections 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.952+0000 7fc1e0a44700 1 -- 172.21.15.45:0/1486745417 wait complete. 2023-10-06T05:22:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.953+0000 7fc1e0a44700 1 Processor -- start 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.953+0000 7fc1e0a44700 1 -- start start 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1e0a44700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc06c3d0 0x7fc1dc167b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1e0a44700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1dc156b70 0x7fc1dc155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1e0a44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc15b890 0x7fc1dc0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1e0a44700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc0ce930 con 0x7fc1dc15b890 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1e0a44700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc0cca00 con 0x7fc1dc156b70 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1e0a44700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc0ccce0 con 0x7fc1dc06c3d0 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc15b890 0x7fc1dc0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc15b890 0x7fc1dc0cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37670/0 (socket says 172.21.15.45:37670) 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 -- 172.21.15.45:0/24491718 learned_addr learned my addr 172.21.15.45:0/24491718 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1d9d9b700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1dc156b70 0x7fc1dc155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1da59c700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc06c3d0 0x7fc1dc167b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 -- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc06c3d0 msgr2=0x7fc1dc167b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc06c3d0 0x7fc1dc167b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 -- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1dc156b70 msgr2=0x7fc1dc155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1dc156b70 0x7fc1dc155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 -- 172.21.15.45:0/24491718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc009420 con 0x7fc1dc15b890 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1d9d9b700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1dc156b70 0x7fc1dc155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1da59c700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc06c3d0 0x7fc1dc167b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1dad9d700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc15b890 0x7fc1dc0cbe90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400cc90 tx=0x7fc1d4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1d27fc700 1 -- 172.21.15.45:0/24491718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400cea0 con 0x7fc1dc15b890 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1d27fc700 1 -- 172.21.15.45:0/24491718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d400ab70 con 0x7fc1dc15b890 2023-10-06T05:22:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1d27fc700 1 -- 172.21.15.45:0/24491718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d40070f0 con 0x7fc1dc15b890 2023-10-06T05:22:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.954+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1dc0ccfc0 con 0x7fc1dc15b890 2023-10-06T05:22:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.955+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1dc0c2f70 con 0x7fc1dc15b890 2023-10-06T05:22:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.955+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1dc0c3160 con 0x7fc1dc15b890 2023-10-06T05:22:53.960 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:22:53.956+0000 7f5d6a567700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.956+0000 7fc1d27fc700 1 -- 172.21.15.45:0/24491718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1d40074a0 con 0x7fc1dc15b890 2023-10-06T05:22:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.956+0000 7fc1d27fc700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.957+0000 7fc1d27fc700 1 -- 172.21.15.45:0/24491718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7fc1d4084e50 con 0x7fc1dc15b890 2023-10-06T05:22:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.957+0000 7fc1da59c700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.957+0000 7fc1da59c700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc1bc066f30 0x7fc1bc0693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc003080 tx=0x7fc1cc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:53.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:53.961+0000 7fc1d27fc700 1 -- 172.21.15.45:0/24491718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1d4051ca0 con 0x7fc1dc15b890 2023-10-06T05:22:54.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.212+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc1dc0c9720 con 0x7fc1dc15b890 2023-10-06T05:22:54.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.213+0000 7fc1d27fc700 1 -- 172.21.15.45:0/24491718 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v129) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc1d40552c0 con 0x7fc1dc15b890 2023-10-06T05:22:54.215 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:54.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc1bc066f30 msgr2=0x7fc1bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc1bc066f30 0x7fc1bc0693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc003080 tx=0x7fc1cc017000 comp rx=0 tx=0).stop 2023-10-06T05:22:54.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc15b890 msgr2=0x7fc1dc0cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:54.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc15b890 0x7fc1dc0cbe90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400cc90 tx=0x7fc1d4001a80 comp rx=0 tx=0).stop 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 shutdown_connections 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1dc15b890 0x7fc1dc0cbe90 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1dc06c3d0 0x7fc1dc167b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 --2- 172.21.15.45:0/24491718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1dc156b70 0x7fc1dc155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 >> 172.21.15.45:0/24491718 conn(0x7fc1dc067620 msgr2=0x7fc1dc070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 shutdown_connections 2023-10-06T05:22:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:54.215+0000 7fc1e0a44700 1 -- 172.21.15.45:0/24491718 wait complete. 2023-10-06T05:22:54.230 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:22:54.230 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:54.230 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:22:54.230 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:22:54.230 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:54.231 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:22:54.231 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:22:54.231 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:54.231 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:54.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:54.231 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:22:54.252 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:54.252 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:54.252 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:54.252 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:54.252 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:54.253 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41967sec preferred_lft 41967sec 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:22:54.254 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:54.255 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:54.256 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:22:54.256 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:22:54.256 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:54.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:54.256 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:54.324 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:22:54.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:54.326 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:22:54.392 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:22:54.392 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:54.392 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:22:54.393 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:22:54.393 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:54.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:54.393 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:22:54.393 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:22:54.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:22:54.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:22:54.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:22:54.393 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:22:54.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:54.515 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:22:54.537 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:22:54.537 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:22:54.537 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:54.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:54.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:22:54.613 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:22:54.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:54.613 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:54.677 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:54.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:54.677 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:54.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:22:54.816 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:22:54.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:22:54.893 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:22:54.893 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:22:54.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.894 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:22:54.958 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:22:54.958 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:22:55.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:22:55.038 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:22:55.038 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:22:55.103 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:22:55.101+0000 7f7b2293a540 -1 init, newargv = 0x556e89cc83d0 newargc=15 2023-10-06T05:22:55.103 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[160985]: starting ceph client 2023-10-06T05:22:55.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[160985]: starting fuse 2023-10-06T05:22:55.110 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:22:55.111 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:22:55.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.112 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:22:55.173 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:22:55.173 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:22:55.174 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:22:55.175 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:22:55.176 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:22:55.176 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:22:55.176 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:22:55.312 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.160985.asok 2023-10-06T05:22:55.494 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.160985.asok 2023-10-06T05:22:55.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.160985.asok status 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:22:55.668 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "160985", 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8232, 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 8232 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:22:55.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2306600543 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2306600543 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.8232 192.168.0.1:0/2306600543", 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2306600543", 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:22:55.670 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 129, 2023-10-06T05:22:55.671 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 98, 2023-10-06T05:22:55.671 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:22:55.671 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:22:55.671 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:22:55.671 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:22:55.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.679 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:22:55.679 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:55.699 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:22:55.700 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:55.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:55.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.772 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:22:55.815 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:22:55.842 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:22:55.842 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:22:55.842 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:55.842 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:22:55.842 INFO:teuthology.orchestra.run.smithi045.stdout:sixmegs 2023-10-06T05:22:55.843 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2023-10-06T05:22:55.843 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2023-10-06T05:22:55.843 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:22:55.844 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:55.844 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:55.926 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:22:55.927 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:22:55.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:55.927 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:55.992 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:56.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.172+0000 7f1d72d39700 1 Processor -- start 2023-10-06T05:22:56.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.172+0000 7f1d72d39700 1 -- start start 2023-10-06T05:22:56.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.172+0000 7f1d72d39700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1d6c0c8ff0 0x7f1d6c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.172+0000 7f1d72d39700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c15b000 con 0x7f1d6c0caad0 2023-10-06T05:22:56.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.172+0000 7f1d72d39700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c15b2e0 con 0x7f1d6c157200 2023-10-06T05:22:56.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.172+0000 7f1d72d39700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c15b5c0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d6bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1d6c0caad0 0x7f1d6c156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50356/0 (socket says 172.21.15.45:50356) 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d6bfff700 1 -- 172.21.15.45:0/2435560218 learned_addr learned my addr 172.21.15.45:0/2435560218 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d70ad5700 1 --2- 172.21.15.45:0/2435560218 >> v2:172.21.15.144:3300/0 conn(0x7f1d6c0c8ff0 0x7f1d6c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559555682 0 0) 0x7f1d6c15b000 con 0x7f1d6c0caad0 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5c003570 con 0x7f1d6c0caad0 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778160199 0 0) 0x7f1d6c15b2e0 con 0x7f1d6c157200 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d6c15b000 con 0x7f1d6c157200 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d70ad5700 1 -- 172.21.15.45:0/2435560218 >> v1:172.21.15.144:6789/0 conn(0x7f1d6c157200 legacy=0x7f1d6c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d70ad5700 1 -- 172.21.15.45:0/2435560218 >> v1:172.21.15.45:6789/0 conn(0x7f1d6c0caad0 legacy=0x7f1d6c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2484710159 0 0) 0x7f1d5c003570 con 0x7f1d6c0caad0 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.173+0000 7f1d70ad5700 1 -- 172.21.15.45:0/2435560218 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c15b8a0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d70ad5700 1 --2- 172.21.15.45:0/2435560218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c93c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1d600098e0 tx=0x7f1d600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8650b2121e764cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58001470 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60011420 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d6b7fe700 1 -- 172.21.15.45:0/2435560218 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d600117c0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d72d39700 1 -- 172.21.15.45:0/2435560218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0c8ff0 msgr2=0x7f1d6c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d72d39700 1 --2- 172.21.15.45:0/2435560218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c93c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1d600098e0 tx=0x7f1d600091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d72d39700 1 -- 172.21.15.45:0/2435560218 shutdown_connections 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d72d39700 1 --2- 172.21.15.45:0/2435560218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c93c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d72d39700 1 -- 172.21.15.45:0/2435560218 >> 172.21.15.45:0/2435560218 conn(0x7f1d6c0cdf50 msgr2=0x7f1d6c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d72d39700 1 -- 172.21.15.45:0/2435560218 shutdown_connections 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.174+0000 7f1d72d39700 1 -- 172.21.15.45:0/2435560218 wait complete. 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.176+0000 7f1d72d39700 1 Processor -- start 2023-10-06T05:22:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.176+0000 7f1d72d39700 1 -- start start 2023-10-06T05:22:56.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d72d39700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d72d39700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0caad0 0x7f1d6c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d72d39700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d6c157200 0x7f1d6c0c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d72d39700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0c61f0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d72d39700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0b3c40 con 0x7f1d6c0caad0 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d72d39700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0c6680 con 0x7f1d6c157200 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37674/0 (socket says 172.21.15.45:37674) 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 -- 172.21.15.45:0/1640187184 learned_addr learned my addr 172.21.15.45:0/1640187184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d712d6700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d6c157200 0x7f1d6c0c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d6bfff700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0caad0 0x7f1d6c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 -- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d6c157200 msgr2=0x7f1d6c0c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d6c157200 0x7f1d6c0c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 -- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0caad0 msgr2=0x7f1d6c0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0caad0 0x7f1d6c0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 -- 172.21.15.45:0/1640187184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60009420 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d70ad5700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c5000 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0c4f80 tx=0x7f1d60016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.177+0000 7f1d68ff9700 1 -- 172.21.15.45:0/1640187184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6001cc90 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.178+0000 7f1d68ff9700 1 -- 172.21.15.45:0/1640187184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6001cdf0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.178+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c0c7950 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.178+0000 7f1d68ff9700 1 -- 172.21.15.45:0/1640187184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6001a6a0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.178+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c0c58c0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.179+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c06f410 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.180+0000 7f1d68ff9700 1 -- 172.21.15.45:0/1640187184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d6001aa90 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.180+0000 7f1d68ff9700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d50066f30 0x7f1d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.180+0000 7f1d68ff9700 1 -- 172.21.15.45:0/1640187184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f1d60086640 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.181+0000 7f1d6bfff700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d50066f30 0x7f1d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.181+0000 7f1d6bfff700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d50066f30 0x7f1d500693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1d580011e0 tx=0x7f1d580015c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.184+0000 7f1d68ff9700 1 -- 172.21.15.45:0/1640187184 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d60053490 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.430+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d6c04bb40 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.432 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:22:56.431+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.434+0000 7f1d68ff9700 1 -- 172.21.15.45:0/1640187184 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v130) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1d60056ab0 con 0x7f1d6c0c8ff0 2023-10-06T05:22:56.436 INFO:tasks.ceph.mds.c.smithi045.stderr: -1613> 2023-10-06T05:22:46.970+0000 7f5d66d60700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-06T05:22:56.439 INFO:tasks.ceph.mds.c.smithi045.stderr: -521> 2023-10-06T05:22:53.956+0000 7f5d6a567700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d50066f30 msgr2=0x7f1d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d50066f30 0x7f1d500693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1d580011e0 tx=0x7f1d580015c0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0c8ff0 msgr2=0x7f1d6c0c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c5000 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0c4f80 tx=0x7f1d60016000 comp rx=0 tx=0).stop 2023-10-06T05:22:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 shutdown_connections 2023-10-06T05:22:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1d50066f30 0x7f1d500693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d6c157200 0x7f1d6c0c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6c0c8ff0 0x7f1d6c0c5000 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 --2- 172.21.15.45:0/1640187184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c0caad0 0x7f1d6c0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 >> 172.21.15.45:0/1640187184 conn(0x7f1d6c0cdf50 msgr2=0x7f1d6c14ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 shutdown_connections 2023-10-06T05:22:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.438+0000 7f1d72d39700 1 -- 172.21.15.45:0/1640187184 wait complete. 2023-10-06T05:22:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:56.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:22:56.461 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:22:56.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.588+0000 7f14de758700 1 Processor -- start 2023-10-06T05:22:56.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.588+0000 7f14de758700 1 -- start start 2023-10-06T05:22:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.588+0000 7f14de758700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14d8158f60 0x7f14d8159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.588+0000 7f14de758700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 0x7f14d80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14de758700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d80cb530 con 0x7f14d806a060 2023-10-06T05:22:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14de758700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d806a500 con 0x7f14d8158f60 2023-10-06T05:22:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14de758700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80ccd50 con 0x7f14d80cc390 2023-10-06T05:22:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f14d806a060 0x7f14d80cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49566/0 (socket says 172.21.15.45:49566) 2023-10-06T05:22:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d77fe700 1 -- 172.21.15.45:0/1874213354 learned_addr learned my addr 172.21.15.45:0/1874213354 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:22:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14dccf5700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 0x7f14d80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d7fff700 1 --2- 172.21.15.45:0/1874213354 >> v2:172.21.15.144:3300/0 conn(0x7f14d8158f60 0x7f14d8159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002789469 0 0) 0x7f14cc001110 con 0x7f14d806a060 2023-10-06T05:22:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c0003570 con 0x7f14d806a060 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2941261805 0 0) 0x7f14c0003570 con 0x7f14d806a060 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 msgr2=0x7f14d80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 0x7f14d80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 >> v2:172.21.15.144:3300/0 conn(0x7f14d8158f60 msgr2=0x7f14d8159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 --2- 172.21.15.45:0/1874213354 >> v2:172.21.15.144:3300/0 conn(0x7f14d8158f60 0x7f14d8159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d8069bd0 con 0x7f14d806a060 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f14cc001600 con 0x7f14d806a060 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 >> v1:172.21.15.45:6789/0 conn(0x7f14d806a060 legacy=0x7f14d80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.589+0000 7f14d6ffd700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 0x7f14c0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c0004010 0x7f14c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c0004d50 0x7f14c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d806a500 con 0x7f14c0004010 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80ccd50 con 0x7f14c0004d50 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c000bab0 con 0x7f14d80cc390 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d77fe700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c0004010 0x7f14c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d7fff700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 0x7f14c0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14dccf5700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c0004d50 0x7f14c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d77fe700 1 -- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 msgr2=0x7f14c0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d77fe700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14d80cc390 0x7f14c0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d77fe700 1 -- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c0004d50 msgr2=0x7f14c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d77fe700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14c0004d50 0x7f14c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d77fe700 1 -- 172.21.15.45:0/1874213354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0003570 con 0x7f14c0004010 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d77fe700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c0004010 0x7f14c0004810 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c5f0 tx=0x7f14d8069b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfb4a48edaaa8b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00cdf0 con 0x7f14c0004010 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc005920 con 0x7f14c0004010 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.590+0000 7f14d6ffd700 1 -- 172.21.15.45:0/1874213354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc005ce0 con 0x7f14c0004010 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.591+0000 7f14de758700 1 -- 172.21.15.45:0/1874213354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c0004010 msgr2=0x7f14c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.591+0000 7f14de758700 1 --2- 172.21.15.45:0/1874213354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c0004010 0x7f14c0004810 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c5f0 tx=0x7f14d8069b40 comp rx=0 tx=0).stop 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.591+0000 7f14dccf5700 1 -- 172.21.15.45:0/1874213354 reap_dead start 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.591+0000 7f14de758700 1 -- 172.21.15.45:0/1874213354 shutdown_connections 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.591+0000 7f14de758700 1 -- 172.21.15.45:0/1874213354 >> 172.21.15.45:0/1874213354 conn(0x7f14d80cdf50 msgr2=0x7f14d80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.591+0000 7f14de758700 1 -- 172.21.15.45:0/1874213354 shutdown_connections 2023-10-06T05:22:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.591+0000 7f14de758700 1 -- 172.21.15.45:0/1874213354 wait complete. 2023-10-06T05:22:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.592+0000 7f14de758700 1 Processor -- start 2023-10-06T05:22:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.592+0000 7f14de758700 1 -- start start 2023-10-06T05:22:56.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14de758700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14c0004010 0x7f14d815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14de758700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14d8157bc0 0x7f14d816cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14de758700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d816d220 0x7f14d815c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14de758700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d816d6c0 con 0x7f14d816d220 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14de758700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d8158030 con 0x7f14d8157bc0 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14de758700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81525b0 con 0x7f14c0004010 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14dccf5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d816d220 0x7f14d815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14dccf5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d816d220 0x7f14d815c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42806/0 (socket says 172.21.15.45:42806) 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14dccf5700 1 -- 172.21.15.45:0/2028487072 learned_addr learned my addr 172.21.15.45:0/2028487072 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14d77fe700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14d8157bc0 0x7f14d816cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.593+0000 7f14d7fff700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14c0004010 0x7f14d815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14dccf5700 1 -- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14c0004010 msgr2=0x7f14d815afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14dccf5700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14c0004010 0x7f14d815afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14dccf5700 1 -- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14d8157bc0 msgr2=0x7f14d816cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14dccf5700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14d8157bc0 0x7f14d816cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14dccf5700 1 -- 172.21.15.45:0/2028487072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d815ab40 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14dccf5700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d816d220 0x7f14d815c740 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f14d00094d0 tx=0x7f14d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14bffff700 1 -- 172.21.15.45:0/2028487072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d000e040 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14bffff700 1 -- 172.21.15.45:0/2028487072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d000f040 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d8159c90 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14bffff700 1 -- 172.21.15.45:0/2028487072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d000fd40 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.594+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d815a210 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.595+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d815acd0 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.596+0000 7f14bffff700 1 -- 172.21.15.45:0/2028487072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14d0015640 con 0x7f14d816d220 2023-10-06T05:22:56.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.596+0000 7f14bffff700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.596+0000 7f14bffff700 1 -- 172.21.15.45:0/2028487072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f14d0085140 con 0x7f14d816d220 2023-10-06T05:22:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.596+0000 7f14d7fff700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.597+0000 7f14d7fff700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f14c4066f30 0x7f14c40693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006f00 tx=0x7f14c80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:56.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.600+0000 7f14bffff700 1 -- 172.21.15.45:0/2028487072 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14d0051eb0 con 0x7f14d816d220 2023-10-06T05:22:56.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:56.890+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f14d8068fc0 con 0x7f14d816d220 2023-10-06T05:22:57.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.439+0000 7f14bffff700 1 -- 172.21.15.45:0/2028487072 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v131) v1 ==== 107+0+0 (secure 0 0 0) 0x7f14d000fea0 con 0x7f14d816d220 2023-10-06T05:22:57.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f14c4066f30 msgr2=0x7f14c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f14c4066f30 0x7f14c40693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f14c8006f00 tx=0x7f14c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d816d220 msgr2=0x7f14d815c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d816d220 0x7f14d815c740 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f14d00094d0 tx=0x7f14d000b040 comp rx=0 tx=0).stop 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 shutdown_connections 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d816d220 0x7f14d815c740 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14c0004010 0x7f14d815afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 --2- 172.21.15.45:0/2028487072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14d8157bc0 0x7f14d816cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 >> 172.21.15.45:0/2028487072 conn(0x7f14d80cdf50 msgr2=0x7f14d80c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 shutdown_connections 2023-10-06T05:22:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.442+0000 7f14de758700 1 -- 172.21.15.45:0/2028487072 wait complete. 2023-10-06T05:22:57.446 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:22:57.459 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:22:57.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.548+0000 7f1ef7819040 1 Processor -- start 2023-10-06T05:22:57.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.548+0000 7f1ef7819040 1 -- start start 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef7819040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563107053920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef7819040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107056770 0x563107057740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef7819040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107057c80 0x563107058ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef7819040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106f2d320 con 0x7f1ee00035f0 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef7819040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106ed1980 con 0x563107056770 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef7819040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563106ed1620 con 0x563107057c80 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563107053920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563107053920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42828/0 (socket says 172.21.15.45:42828) 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 -- 172.21.15.45:0/3684968015 learned_addr learned my addr 172.21.15.45:0/3684968015 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1eebfff700 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107056770 0x563107057740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef1238700 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107057c80 0x563107058ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 -- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107057c80 msgr2=0x563107058ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107057c80 0x563107058ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 -- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107056770 msgr2=0x563107057740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107056770 0x563107057740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.549+0000 7f1ef0a37700 1 -- 172.21.15.45:0/3684968015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563106ed3ae0 con 0x7f1ee00035f0 2023-10-06T05:22:57.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563107053920 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4002fc0 tx=0x7f1ee4001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5faee4602aed10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1eeaffd700 1 -- 172.21.15.45:0/3684968015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400ebd0 con 0x7f1ee00035f0 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1eeaffd700 1 -- 172.21.15.45:0/3684968015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee4008030 con 0x7f1ee00035f0 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1eeaffd700 1 -- 172.21.15.45:0/3684968015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4011450 con 0x7f1ee00035f0 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 -- 172.21.15.45:0/3684968015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 msgr2=0x563107053920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563107053920 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4002fc0 tx=0x7f1ee4001830 comp rx=0 tx=0).stop 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 -- 172.21.15.45:0/3684968015 shutdown_connections 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107057c80 0x563107058ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563107053920 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 --2- 172.21.15.45:0/3684968015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107056770 0x563107057740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 -- 172.21.15.45:0/3684968015 >> 172.21.15.45:0/3684968015 conn(0x563106f87450 msgr2=0x563107056060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 -- 172.21.15.45:0/3684968015 shutdown_connections 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.550+0000 7f1ef7819040 1 -- 172.21.15.45:0/3684968015 wait complete. 2023-10-06T05:22:57.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.551+0000 7f1ef7819040 1 Processor -- start 2023-10-06T05:22:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.551+0000 7f1ef7819040 1 -- start start 2023-10-06T05:22:57.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef7819040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107056770 0x5631070516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef7819040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107057c80 0x563107051be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef7819040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563106fc84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef7819040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106ec8400 con 0x7f1ee00035f0 2023-10-06T05:22:57.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef7819040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106fc6570 con 0x563107057c80 2023-10-06T05:22:57.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef7819040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563106fc6810 con 0x563107056770 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563106fc84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563106fc84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42830/0 (socket says 172.21.15.45:42830) 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 learned_addr learned my addr 172.21.15.45:0/2116669555 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107056770 0x5631070516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1eebfff700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107057c80 0x563107051be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107056770 msgr2=0x5631070516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563107056770 0x5631070516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107057c80 msgr2=0x563107051be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563107057c80 0x563107051be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.552+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee400a040 con 0x7f1ee00035f0 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.553+0000 7f1ef1238700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563106fc84e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1eec002230 tx=0x7f1eec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.553+0000 7f1ee97fa700 1 -- 172.21.15.45:0/2116669555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec021070 con 0x7f1ee00035f0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.553+0000 7f1ee97fa700 1 -- 172.21.15.45:0/2116669555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec00a270 con 0x7f1ee00035f0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.553+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563106ed3540 con 0x7f1ee00035f0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.553+0000 7f1ee97fa700 1 -- 172.21.15.45:0/2116669555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec012450 con 0x7f1ee00035f0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.553+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563106fc6d50 con 0x7f1ee00035f0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.554+0000 7f1ee97fa700 1 -- 172.21.15.45:0/2116669555 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1eec003050 con 0x7f1ee00035f0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.555+0000 7f1ee97fa700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed4065890 0x7f1ed4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.555+0000 7f1ee97fa700 1 -- 172.21.15.45:0/2116669555 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f1eec0849a0 con 0x7f1ee00035f0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.555+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed4065890 0x7f1ed4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.560 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:22:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.555+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x563106fb41c0 0x563106fc7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.555+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563106fc79d0 con 0x563106fb41c0 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.555+0000 7f1eebfff700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x563106fb41c0 0x563106fc7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.556+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed4065890 0x7f1ed4067d40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40010e0 tx=0x7f1ee4019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.556+0000 7f1eebfff700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x563106fb41c0 0x563106fc7490 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.557+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v99'195 uv169 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.557+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563106fc7db0 con 0x563106fb41c0 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.559+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v99'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.559+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076250 con 0x563106fb41c0 2023-10-06T05:22:57.563 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:22:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.562+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v99'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.562+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077610 con 0x563106fb41c0 2023-10-06T05:22:57.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.565+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v99'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.565+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x5631070779c0 con 0x563106fb41c0 2023-10-06T05:22:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.567+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v99'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.567+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076600 con 0x563106fb41c0 2023-10-06T05:22:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.573+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v99'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.573+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076a00 con 0x563106fb41c0 2023-10-06T05:22:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.574+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v99'201 uv175 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.574+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076e00 con 0x563106fb41c0 2023-10-06T05:22:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.577+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v99'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.577+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076e00 con 0x563106fb41c0 2023-10-06T05:22:57.584 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:22:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.583+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v99'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.583+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.584+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v99'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.584+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.587+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v99'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.587+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.593+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v99'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.593+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.594+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v99'207 uv182 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.594+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.597+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v99'208 uv208 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.597+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.601 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:22:57.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.600+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v99'209 uv209 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.600+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.603+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v99'210 uv210 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.603+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.605+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v99'211 uv211 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.606+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107076db0 con 0x563106fb41c0 2023-10-06T05:22:57.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.608+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v99'212 uv212 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.608+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x5631070771f0 con 0x563106fb41c0 2023-10-06T05:22:57.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.609+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v99'213 uv213 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.609+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.610+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v99'214 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.610+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.611+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v99'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.611+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.614 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:22:57.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.613+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v99'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.613+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.614+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v99'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.614+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.615+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v99'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.615+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.616+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v99'219 uv219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.616+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.617+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v99'220 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.617+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.618+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v99'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.619+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.621 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:22:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.620+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v99'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.620+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.621+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v99'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.621+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.622+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v99'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.622+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107077020 con 0x563106fb41c0 2023-10-06T05:22:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.623+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v99'225 uv225 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563106fb41c0 2023-10-06T05:22:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.623+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x563107077020 0x56310707a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.623+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707a7d0 con 0x563107077020 2023-10-06T05:22:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.623+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x563107077020 0x56310707a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.624+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x563107077020 0x56310707a290 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.626+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v99'150 uv132 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.626+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707ab80 con 0x563107077020 2023-10-06T05:22:57.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.627+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v99'151 uv151 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.627+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707af30 con 0x563107077020 2023-10-06T05:22:57.630 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:22:57.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.628+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v99'152 uv152 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.628+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707b7d0 con 0x563107077020 2023-10-06T05:22:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.632+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v99'153 uv153 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.632+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707bb80 con 0x563107077020 2023-10-06T05:22:57.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.634+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v99'154 uv154 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.634+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707bf30 con 0x563107077020 2023-10-06T05:22:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.636+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v99'155 uv149 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.636+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c2e0 con 0x563107077020 2023-10-06T05:22:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.637+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v99'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.637+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.640 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:22:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.638+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v99'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.638+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.640+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v99'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.640+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.641+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v99'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.641+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.642+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v99'160 uv148 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.642+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.644+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v99'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.644+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.647 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:22:57.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.645+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v99'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.646+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707fdb0 con 0x563107077020 2023-10-06T05:22:57.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.647+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v99'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.647+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707fdb0 con 0x563107077020 2023-10-06T05:22:57.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.648+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v99'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.648+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.649+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v99'165 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.650+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.651+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v99'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.651+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.654 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:22:57.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.652+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v99'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.652+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707d6a0 con 0x563107077020 2023-10-06T05:22:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.653+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v99'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.653+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707d6a0 con 0x563107077020 2023-10-06T05:22:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.655+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v99'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.655+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707c690 con 0x563107077020 2023-10-06T05:22:57.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.656+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v99'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.656+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56310707cba0 0x56310707ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.656+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f510 con 0x56310707cba0 2023-10-06T05:22:57.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.656+0000 7f1eebfff700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56310707cba0 0x56310707ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.657+0000 7f1eebfff700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56310707cba0 0x56310707ef80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.658+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.658+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f8f0 con 0x56310707cba0 2023-10-06T05:22:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.659+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v99'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.659+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f8f0 con 0x56310707cba0 2023-10-06T05:22:57.662 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:22:57.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.660+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v99'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.660+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107080900 con 0x56310707cba0 2023-10-06T05:22:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.661+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.661+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107080900 con 0x56310707cba0 2023-10-06T05:22:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.663+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.663+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f8f0 con 0x56310707cba0 2023-10-06T05:22:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.664+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.664+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f8f0 con 0x56310707cba0 2023-10-06T05:22:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.665+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v99'30 uv29 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.665+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f8f0 con 0x56310707cba0 2023-10-06T05:22:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.667+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v99'31 uv31 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.667+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f8f0 con 0x56310707cba0 2023-10-06T05:22:57.669 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:22:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.668+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v99'32 uv32 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.668+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707fd20 con 0x56310707cba0 2023-10-06T05:22:57.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.669+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v99'33 uv33 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.669+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f9c0 con 0x563107077020 2023-10-06T05:22:57.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.671+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.671+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f9c0 con 0x563107077020 2023-10-06T05:22:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.672+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v99'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.672+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f9c0 con 0x563107077020 2023-10-06T05:22:57.675 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:22:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.673+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v99'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.673+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x5631070809d0 con 0x563107077020 2023-10-06T05:22:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.675+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.675+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x5631070809d0 con 0x563107077020 2023-10-06T05:22:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.676+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.676+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310707f9c0 con 0x563107077020 2023-10-06T05:22:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.677+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x563107077020 2023-10-06T05:22:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.677+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x563107080040 0x563107082420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.677+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107082a80 con 0x563107080040 2023-10-06T05:22:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.677+0000 7f1ef1238700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x563107080040 0x563107082420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.678+0000 7f1ef1238700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x563107080040 0x563107082420 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.679+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v99'70 uv64 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.679+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107082f30 con 0x563107080040 2023-10-06T05:22:57.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.680+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v99'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.680+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107083420 con 0x563107080040 2023-10-06T05:22:57.684 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:22:57.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.681+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v99'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.681+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107084d20 con 0x563107080040 2023-10-06T05:22:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.682+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v99'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.683+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x5631070850d0 con 0x563107080040 2023-10-06T05:22:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.684+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v99'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.684+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107083d10 con 0x563107080040 2023-10-06T05:22:57.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.685+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v99'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.685+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107084130 con 0x563107080040 2023-10-06T05:22:57.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.686+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v99'76 uv69 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.686+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107084530 con 0x563107080040 2023-10-06T05:22:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.687+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v99'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.688+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107084530 con 0x563107080040 2023-10-06T05:22:57.695 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:22:57.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.693+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v99'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.693+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107084930 con 0x563107080040 2023-10-06T05:22:57.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.695+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v99'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.695+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x5631070844e0 con 0x563107080040 2023-10-06T05:22:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.697+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v99'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.697+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5631070848e0 0x5631070879a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.698+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107087ee0 con 0x5631070848e0 2023-10-06T05:22:57.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.698+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5631070848e0 0x5631070879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.698+0000 7f1ef0a37700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5631070848e0 0x5631070879a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.699+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v99'32 uv31 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x5631070848e0 2023-10-06T05:22:57.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.699+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107087ee0 con 0x5631070848e0 2023-10-06T05:22:57.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.701+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v99'33 uv33 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x5631070848e0 2023-10-06T05:22:57.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.701+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107087ee0 con 0x5631070848e0 2023-10-06T05:22:57.705 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:22:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.703+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v99'34 uv34 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x5631070848e0 2023-10-06T05:22:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.703+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107087ee0 con 0x5631070848e0 2023-10-06T05:22:57.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.706+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v99'35 uv35 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x5631070848e0 2023-10-06T05:22:57.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.707+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107088300 con 0x5631070848e0 2023-10-06T05:22:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.709+0000 7f1ef0a37700 1 -- 172.21.15.45:0/2116669555 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v99'36 uv36 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1ee4025070 con 0x5631070848e0 2023-10-06T05:22:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.709+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107088300 con 0x56310707cba0 2023-10-06T05:22:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.711+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.711+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107088300 con 0x56310707cba0 2023-10-06T05:22:57.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.712+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v99'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.712+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107088300 con 0x56310707cba0 2023-10-06T05:22:57.717 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:22:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.713+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v99'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.713+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107089310 con 0x56310707cba0 2023-10-06T05:22:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.714+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.714+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107089310 con 0x56310707cba0 2023-10-06T05:22:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.716+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.716+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x563107088300 con 0x56310707cba0 2023-10-06T05:22:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.717+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.717+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563107088810 0x56310708abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.717+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b1c0 con 0x563107088810 2023-10-06T05:22:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.717+0000 7f1eebfff700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563107088810 0x56310708abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.718+0000 7f1eebfff700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563107088810 0x56310708abf0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.719+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v99'64 uv58 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.719+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b200 con 0x563107088810 2023-10-06T05:22:57.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.720+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v99'65 uv65 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.720+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b600 con 0x563107088810 2023-10-06T05:22:57.722 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:22:57.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.721+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v99'66 uv66 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.721+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b600 con 0x563107088810 2023-10-06T05:22:57.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.722+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v99'67 uv67 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.722+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b600 con 0x563107088810 2023-10-06T05:22:57.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.723+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v99'68 uv68 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.723+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b600 con 0x563107088810 2023-10-06T05:22:57.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.724+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v99'69 uv63 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.724+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b600 con 0x563107088810 2023-10-06T05:22:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.725+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v99'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.725+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b600 con 0x563107088810 2023-10-06T05:22:57.728 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:22:57.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.726+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v99'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.726+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b600 con 0x563107088810 2023-10-06T05:22:57.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.727+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v99'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.727+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708e370 con 0x563107088810 2023-10-06T05:22:57.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.729+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v99'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x563107088810 2023-10-06T05:22:57.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.729+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708baf0 con 0x56310707cba0 2023-10-06T05:22:57.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.730+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v99'31 uv29 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.730+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708baf0 con 0x56310707cba0 2023-10-06T05:22:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.731+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v99'32 uv32 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.731+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708baf0 con 0x56310707cba0 2023-10-06T05:22:57.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.733+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v99'33 uv33 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.733+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708baf0 con 0x56310707cba0 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.734+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v99'34 uv34 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.734+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56310708c000 0x56310708c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.734+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x56310708c000 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.734+0000 7f1ef1238700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56310708c000 0x56310708c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.735+0000 7f1ef1238700 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56310708c000 0x56310708c3d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.735+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x56310708c000 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.736+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x56310708c000 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.737+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v99'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x56310708c000 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.737+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x56310708c000 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.738+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v99'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x56310708c000 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.738+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708fad0 con 0x56310708c000 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.739+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x56310708c000 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.739+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708fad0 con 0x56310708c000 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.739+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x56310708c000 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.739+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x56310708c000 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.740+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x56310708c000 2023-10-06T05:22:57.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.741+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x563107080040 2023-10-06T05:22:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.742+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v99'32 uv31 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.742+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x563107080040 2023-10-06T05:22:57.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.743+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v99'33 uv33 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.743+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x563107080040 2023-10-06T05:22:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.746+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v99'34 uv34 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.748 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:22:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.746+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708eac0 con 0x563107080040 2023-10-06T05:22:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.749+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v99'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.749+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x563107080040 2023-10-06T05:22:57.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.755+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v99'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1eec019070 con 0x563107080040 2023-10-06T05:22:57.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.755+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.756+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v99'82 uv73 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.756+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.757+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v99'83 uv83 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.757+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.760 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:22:57.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.758+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v99'84 uv84 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.758+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.760+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v99'85 uv85 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.760+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.761+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v99'86 uv86 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.761+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.762+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v99'87 uv80 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.762+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.763+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v99'88 uv88 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.763+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.765 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:22:57.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.764+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v99'89 uv89 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.764+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.765+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v99'90 uv90 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.765+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708b680 con 0x56310707cba0 2023-10-06T05:22:57.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.766+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v99'91 uv91 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.766+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708efb0 con 0x56310707cba0 2023-10-06T05:22:57.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.767+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v99'92 uv92 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.767+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v8 -- 0x56310708efb0 con 0x56310707cba0 2023-10-06T05:22:57.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1eebfff700 1 -- 172.21.15.45:0/2116669555 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v99'93 uv93 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1edc017040 con 0x56310707cba0 2023-10-06T05:22:57.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56310707cba0 msgr2=0x56310707ef80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56310707cba0 0x56310707ef80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x563106fb41c0 msgr2=0x563106fc7490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x563106fb41c0 0x563106fc7490 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563107088810 msgr2=0x56310708abf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x563107088810 0x56310708abf0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x563107077020 msgr2=0x56310707a290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x563107077020 0x56310707a290 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56310708c000 msgr2=0x56310708c3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56310708c000 0x56310708c3d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 reap_dead start 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5631070848e0 msgr2=0x5631070879a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5631070848e0 0x5631070879a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 reap_dead start 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef1238700 1 -- 172.21.15.45:0/2116669555 reap_dead start 2023-10-06T05:22:57.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x563107080040 msgr2=0x563107082420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x563107080040 0x563107082420 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed4065890 msgr2=0x7f1ed4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed4065890 0x7f1ed4067d40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40010e0 tx=0x7f1ee4019000 comp rx=0 tx=0).stop 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 msgr2=0x563106fc84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.768+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563106fc84e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1eec002230 tx=0x7f1eec019000 comp rx=0 tx=0).stop 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.769+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 shutdown_connections 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.769+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x563107080040 0x563107082420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.769+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ee00035f0 0x563106fc84e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.769+0000 7f1ef7819040 1 --2- 172.21.15.45:0/2116669555 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed4065890 0x7f1ed4067d40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.769+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 >> 172.21.15.45:0/2116669555 conn(0x563106f87450 msgr2=0x56310705ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.769+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 shutdown_connections 2023-10-06T05:22:57.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.769+0000 7f1ef7819040 1 -- 172.21.15.45:0/2116669555 wait complete. 2023-10-06T05:22:57.777 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2023-10-06T05:22:57.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.825+0000 7fc0f2570040 1 Processor -- start 2023-10-06T05:22:57.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.825+0000 7fc0f2570040 1 -- start start 2023-10-06T05:22:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0f2570040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574459917e0 0x557445a1a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0f2570040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cb30 0x557445a22ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0f2570040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557445a1cf00 0x557445a23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0f2570040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445894980 con 0x5574459917e0 2023-10-06T05:22:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0f2570040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445894620 con 0x557445a1cb30 2023-10-06T05:22:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0f2570040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55744588b400 con 0x557445a1cf00 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574459917e0 0x557445a1a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574459917e0 0x557445a1a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42844/0 (socket says 172.21.15.45:42844) 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 -- 172.21.15.45:0/2562461432 learned_addr learned my addr 172.21.15.45:0/2562461432 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0ebf8f700 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557445a1cf00 0x557445a23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eaf8d700 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cb30 0x557445a22ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 -- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557445a1cf00 msgr2=0x557445a23220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557445a1cf00 0x557445a23220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 -- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cb30 msgr2=0x557445a22ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cb30 0x557445a22ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 -- 172.21.15.45:0/2562461432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4005cf0 con 0x5574459917e0 2023-10-06T05:22:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.826+0000 7fc0eb78e700 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574459917e0 0x557445a1a940 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400acc0 tx=0x7fc0e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d7ffef86693550 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0e9f8b700 1 -- 172.21.15.45:0/2562461432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4000bc0 con 0x5574459917e0 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0e9f8b700 1 -- 172.21.15.45:0/2562461432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4008970 con 0x5574459917e0 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0e9f8b700 1 -- 172.21.15.45:0/2562461432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e401d420 con 0x5574459917e0 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 -- 172.21.15.45:0/2562461432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574459917e0 msgr2=0x557445a1a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574459917e0 0x557445a1a940 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400acc0 tx=0x7fc0e4016000 comp rx=0 tx=0).stop 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 -- 172.21.15.45:0/2562461432 shutdown_connections 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557445a1cf00 0x557445a23220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574459917e0 0x557445a1a940 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 --2- 172.21.15.45:0/2562461432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cb30 0x557445a22ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 -- 172.21.15.45:0/2562461432 >> 172.21.15.45:0/2562461432 conn(0x55744594a450 msgr2=0x557445947210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 -- 172.21.15.45:0/2562461432 shutdown_connections 2023-10-06T05:22:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.827+0000 7fc0f2570040 1 -- 172.21.15.45:0/2562461432 wait complete. 2023-10-06T05:22:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.828+0000 7fc0f2570040 1 Processor -- start 2023-10-06T05:22:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.828+0000 7fc0f2570040 1 -- start start 2023-10-06T05:22:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0f2570040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5574459917e0 0x557445a159a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0f2570040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557445a1cb30 0x557445a138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0f2570040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cf00 0x557445a13e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0f2570040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445a15ee0 con 0x557445a1cb30 2023-10-06T05:22:57.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0f2570040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557445a25170 con 0x557445a1cf00 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0f2570040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557445a12f10 con 0x5574459917e0 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eaf8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557445a1cb30 0x557445a138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eaf8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557445a1cb30 0x557445a138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42856/0 (socket says 172.21.15.45:42856) 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eaf8d700 1 -- 172.21.15.45:0/3669328585 learned_addr learned my addr 172.21.15.45:0/3669328585 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eb78e700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5574459917e0 0x557445a159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0ebf8f700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cf00 0x557445a13e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eaf8d700 1 -- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5574459917e0 msgr2=0x557445a159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eaf8d700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5574459917e0 0x557445a159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eaf8d700 1 -- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cf00 msgr2=0x557445a13e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.829+0000 7fc0eaf8d700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cf00 0x557445a13e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0eaf8d700 1 -- 172.21.15.45:0/3669328585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574458bd1c0 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0ebf8f700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557445a1cf00 0x557445a13e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0eaf8d700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557445a1cb30 0x557445a138f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c710 tx=0x7fc0d4006880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0e3fff700 1 -- 172.21.15.45:0/3669328585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400f450 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0e3fff700 1 -- 172.21.15.45:0/3669328585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d4005070 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0e3fff700 1 -- 172.21.15.45:0/3669328585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400f5d0 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5574458a33d0 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.830+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557445896ae0 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.831+0000 7fc0e3fff700 1 -- 172.21.15.45:0/3669328585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0d4015020 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.832+0000 7fc0e3fff700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc065890 0x7fc0cc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.832+0000 7fc0e3fff700 1 -- 172.21.15.45:0/3669328585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fc0d4083630 con 0x557445a1cb30 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.832+0000 7fc0eb78e700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc065890 0x7fc0cc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.832+0000 7fc0f2570040 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557445a11d00 0x557445a24160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.832+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e99) v8 -- 0x557445a21600 con 0x557445a11d00 2023-10-06T05:22:57.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.832+0000 7fc0ebf8f700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557445a11d00 0x557445a24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.833+0000 7fc0eb78e700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc065890 0x7fc0cc067d40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c010 tx=0x7fc0e401e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.833+0000 7fc0ebf8f700 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557445a11d00 0x557445a24160 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0ebf8f700 1 -- 172.21.15.45:0/3669328585 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 500.00000000 [delete] v99'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc0dc013540 con 0x557445a11d00 2023-10-06T05:22:57.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557445a11d00 msgr2=0x557445a24160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557445a11d00 0x557445a24160 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc065890 msgr2=0x7fc0cc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc0cc065890 0x7fc0cc067d40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c010 tx=0x7fc0e401e000 comp rx=0 tx=0).stop 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557445a1cb30 msgr2=0x557445a138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 --2- 172.21.15.45:0/3669328585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557445a1cb30 0x557445a138f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c710 tx=0x7fc0d4006880 comp rx=0 tx=0).stop 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0ebf8f700 1 -- 172.21.15.45:0/3669328585 reap_dead start 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 shutdown_connections 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 >> 172.21.15.45:0/3669328585 conn(0x55744594a450 msgr2=0x557445a1a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 shutdown_connections 2023-10-06T05:22:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.834+0000 7fc0f2570040 1 -- 172.21.15.45:0/3669328585 wait complete. 2023-10-06T05:22:57.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:22:57.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.977+0000 7fc7a816c700 1 Processor -- start 2023-10-06T05:22:57.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.978+0000 7fc7a816c700 1 -- start start 2023-10-06T05:22:57.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.978+0000 7fc7a816c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7a00c3db0 0x7fc7a00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a816c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a00ce6a0 con 0x7fc7a0069ed0 2023-10-06T05:22:57.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a816c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a0159880 con 0x7fc7a00b0610 2023-10-06T05:22:57.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a816c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a00690f0 con 0x7fc7a00c3db0 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a5707700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7a00c3db0 0x7fc7a00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a5707700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7a00c3db0 0x7fc7a00c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42868/0 (socket says 172.21.15.45:42868) 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a5707700 1 -- 172.21.15.45:0/1918635556 learned_addr learned my addr 172.21.15.45:0/1918635556 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a6709700 1 --1- 172.21.15.45:0/1918635556 >> v1:172.21.15.45:6789/0 conn(0x7fc7a0069ed0 0x7fc7a00cc350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49610/0 (socket says 172.21.15.45:49610) 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a4f06700 1 -- 172.21.15.45:0/1918635556 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164213047 0 0) 0x7fc7a0159880 con 0x7fc7a00b0610 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a4f06700 1 -- 172.21.15.45:0/1918635556 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc78c003570 con 0x7fc7a00b0610 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a5707700 1 -- 172.21.15.45:0/1918635556 >> v1:172.21.15.144:6789/0 conn(0x7fc7a00b0610 legacy=0x7fc7a00697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a5707700 1 -- 172.21.15.45:0/1918635556 >> v1:172.21.15.45:6789/0 conn(0x7fc7a0069ed0 legacy=0x7fc7a00cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a4f06700 1 -- 172.21.15.45:0/1918635556 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932666026 0 0) 0x7fc7a00ce6a0 con 0x7fc7a0069ed0 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a5707700 1 -- 172.21.15.45:0/1918635556 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a00692e0 con 0x7fc7a00c3db0 2023-10-06T05:22:57.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.979+0000 7fc7a5707700 1 --2- 172.21.15.45:0/1918635556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a00c3db0 0x7fc7a00c85d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc7900099b0 tx=0x7fc790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14792803eaa8b9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a4f06700 1 -- 172.21.15.45:0/1918635556 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790009ea0 con 0x7fc7a00c3db0 2023-10-06T05:22:57.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a4f06700 1 -- 172.21.15.45:0/1918635556 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79000b860 con 0x7fc7a00c3db0 2023-10-06T05:22:57.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a4f06700 1 -- 172.21.15.45:0/1918635556 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000bc00 con 0x7fc7a00c3db0 2023-10-06T05:22:57.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a816c700 1 -- 172.21.15.45:0/1918635556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a00c3db0 msgr2=0x7fc7a00c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a816c700 1 --2- 172.21.15.45:0/1918635556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a00c3db0 0x7fc7a00c85d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc7900099b0 tx=0x7fc790009250 comp rx=0 tx=0).stop 2023-10-06T05:22:57.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a816c700 1 -- 172.21.15.45:0/1918635556 shutdown_connections 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a816c700 1 --2- 172.21.15.45:0/1918635556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a00c3db0 0x7fc7a00c85d0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a816c700 1 -- 172.21.15.45:0/1918635556 >> 172.21.15.45:0/1918635556 conn(0x7fc7a0067e00 msgr2=0x7fc7a00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a816c700 1 -- 172.21.15.45:0/1918635556 shutdown_connections 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.980+0000 7fc7a816c700 1 -- 172.21.15.45:0/1918635556 wait complete. 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.981+0000 7fc7a816c700 1 Processor -- start 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.981+0000 7fc7a816c700 1 -- start start 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a816c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a0069ed0 0x7fc7a00c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a816c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a00b0610 0x7fc7a016ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a816c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a00c3db0 0x7fc7a016cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a816c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00c6a30 con 0x7fc7a0069ed0 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a816c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00c5990 con 0x7fc7a00c3db0 2023-10-06T05:22:57.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a816c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00c5c70 con 0x7fc7a00b0610 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a5f08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a0069ed0 0x7fc7a00c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a5f08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a0069ed0 0x7fc7a00c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42870/0 (socket says 172.21.15.45:42870) 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a5f08700 1 -- 172.21.15.45:0/4102066350 learned_addr learned my addr 172.21.15.45:0/4102066350 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a6709700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a00c3db0 0x7fc7a016cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.982+0000 7fc7a5707700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a00b0610 0x7fc7a016ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a5f08700 1 -- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a00b0610 msgr2=0x7fc7a016ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a5f08700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a00b0610 0x7fc7a016ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a5f08700 1 -- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a00c3db0 msgr2=0x7fc7a016cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a5f08700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a00c3db0 0x7fc7a016cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a5f08700 1 -- 172.21.15.45:0/4102066350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900094f0 con 0x7fc7a0069ed0 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a5f08700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a0069ed0 0x7fc7a00c5040 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c7c0 tx=0x7fc79c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc795ffb700 1 -- 172.21.15.45:0/4102066350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c001530 con 0x7fc7a0069ed0 2023-10-06T05:22:57.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc795ffb700 1 -- 172.21.15.45:0/4102066350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c0016b0 con 0x7fc7a0069ed0 2023-10-06T05:22:57.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a00c5f50 con 0x7fc7a0069ed0 2023-10-06T05:22:57.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc795ffb700 1 -- 172.21.15.45:0/4102066350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c001a70 con 0x7fc7a0069ed0 2023-10-06T05:22:57.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.983+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a00c6510 con 0x7fc7a0069ed0 2023-10-06T05:22:57.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.985+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a00c6730 con 0x7fc7a0069ed0 2023-10-06T05:22:57.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.985+0000 7fc795ffb700 1 -- 172.21.15.45:0/4102066350 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc79c01b3b0 con 0x7fc7a0069ed0 2023-10-06T05:22:57.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.985+0000 7fc795ffb700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc784066f30 0x7fc7840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.985+0000 7fc795ffb700 1 -- 172.21.15.45:0/4102066350 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fc79c084240 con 0x7fc7a0069ed0 2023-10-06T05:22:57.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.985+0000 7fc7a5707700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc784066f30 0x7fc7840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.986+0000 7fc7a5707700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc784066f30 0x7fc7840693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc7900099b0 tx=0x7fc790017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:57.990+0000 7fc795ffb700 1 -- 172.21.15.45:0/4102066350 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc79c050f10 con 0x7fc7a0069ed0 2023-10-06T05:22:58.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:58.237+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fc7a006e350 con 0x7fc7a0069ed0 2023-10-06T05:22:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.004+0000 7fc795ffb700 1 -- 172.21.15.45:0/4102066350 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v132) v1 ==== 162+0+0 (secure 0 0 0) 0x7fc79c054530 con 0x7fc7a0069ed0 2023-10-06T05:22:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc784066f30 msgr2=0x7fc7840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc784066f30 0x7fc7840693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc7900099b0 tx=0x7fc790017000 comp rx=0 tx=0).stop 2023-10-06T05:22:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a0069ed0 msgr2=0x7fc7a00c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a0069ed0 0x7fc7a00c5040 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c7c0 tx=0x7fc79c007db0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 shutdown_connections 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc784066f30 0x7fc7840693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a00c3db0 0x7fc7a016cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7a0069ed0 0x7fc7a00c5040 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 --2- 172.21.15.45:0/4102066350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a00b0610 0x7fc7a016ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.007+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 >> 172.21.15.45:0/4102066350 conn(0x7fc7a0067e00 msgr2=0x7fc7a00cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.008+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 shutdown_connections 2023-10-06T05:22:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.008+0000 7fc7a816c700 1 -- 172.21.15.45:0/4102066350 wait complete. 2023-10-06T05:22:59.012 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:22:59.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:59.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.569+0000 7f364c00d700 1 Processor -- start 2023-10-06T05:22:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.569+0000 7f364c00d700 1 -- start start 2023-10-06T05:22:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f364c00d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3644158f60 0x7f3644159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f364c00d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f364406a060 0x7f36440cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f364c00d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3644069bd0 con 0x7f36440cc390 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f364c00d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364406a500 con 0x7f364406a060 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f364c00d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3644068fc0 con 0x7f3644158f60 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f364406a060 0x7f36440cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f364406a060 0x7f36440cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42878/0 (socket says 172.21.15.45:42878) 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 -- 172.21.15.45:0/2811834231 learned_addr learned my addr 172.21.15.45:0/2811834231 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f3649da9700 1 --2- 172.21.15.45:0/2811834231 >> v2:172.21.15.144:3300/0 conn(0x7f3644158f60 0x7f3644159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 -- 172.21.15.45:0/2811834231 >> v2:172.21.15.144:3300/0 conn(0x7f3644158f60 msgr2=0x7f3644159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 --2- 172.21.15.45:0/2811834231 >> v2:172.21.15.144:3300/0 conn(0x7f3644158f60 0x7f3644159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 -- 172.21.15.45:0/2811834231 >> v1:172.21.15.144:6789/0 conn(0x7f36440cc390 legacy=0x7f36440c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 -- 172.21.15.45:0/2811834231 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440692a0 con 0x7f364406a060 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.570+0000 7f36495a8700 1 --2- 172.21.15.45:0/2811834231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36440cae20 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f364406ed80 tx=0x7f3634009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc03106c8a8445c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f3648da7700 1 -- 172.21.15.45:0/2811834231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400b820 con 0x7f364406a060 2023-10-06T05:22:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f3648da7700 1 -- 172.21.15.45:0/2811834231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400b980 con 0x7f364406a060 2023-10-06T05:22:59.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f3648da7700 1 -- 172.21.15.45:0/2811834231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400bd20 con 0x7f364406a060 2023-10-06T05:22:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 -- 172.21.15.45:0/2811834231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 msgr2=0x7f36440cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 --2- 172.21.15.45:0/2811834231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36440cae20 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f364406ed80 tx=0x7f3634009250 comp rx=0 tx=0).stop 2023-10-06T05:22:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 -- 172.21.15.45:0/2811834231 shutdown_connections 2023-10-06T05:22:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 --2- 172.21.15.45:0/2811834231 >> v2:172.21.15.144:3300/0 conn(0x7f3644158f60 0x7f3644159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 --2- 172.21.15.45:0/2811834231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36440cae20 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 -- 172.21.15.45:0/2811834231 >> 172.21.15.45:0/2811834231 conn(0x7f36440cdf50 msgr2=0x7f36440ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 -- 172.21.15.45:0/2811834231 shutdown_connections 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.571+0000 7f364c00d700 1 -- 172.21.15.45:0/2811834231 wait complete. 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 Processor -- start 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 -- start start 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36441517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36440cc390 0x7f36440b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3644158f60 0x7f36440c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364416aec0 con 0x7f364406a060 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3644152140 con 0x7f36440cc390 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f364c00d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36440b3cd0 con 0x7f3644158f60 2023-10-06T05:22:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.573+0000 7f3649da9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36441517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36441517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42888/0 (socket says 172.21.15.45:42888) 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 -- 172.21.15.45:0/1588490769 learned_addr learned my addr 172.21.15.45:0/1588490769 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f36495a8700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36440cc390 0x7f36440b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f364a5aa700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3644158f60 0x7f36440c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 -- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3644158f60 msgr2=0x7f36440c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3644158f60 0x7f36440c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 -- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36440cc390 msgr2=0x7f36440b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36440cc390 0x7f36440b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 -- 172.21.15.45:0/1588490769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340094f0 con 0x7f364406a060 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3649da9700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36441517f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f364000d890 tx=0x7f3640008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3639ffb700 1 -- 172.21.15.45:0/1588490769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36400125f0 con 0x7f364406a060 2023-10-06T05:22:59.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3639ffb700 1 -- 172.21.15.45:0/1588490769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640012750 con 0x7f364406a060 2023-10-06T05:22:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364416ab90 con 0x7f364406a060 2023-10-06T05:22:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.574+0000 7f3639ffb700 1 -- 172.21.15.45:0/1588490769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640012b10 con 0x7f364406a060 2023-10-06T05:22:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.575+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364415cf50 con 0x7f364406a060 2023-10-06T05:22:59.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.575+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364416a9f0 con 0x7f364406a060 2023-10-06T05:22:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.576+0000 7f3639ffb700 1 -- 172.21.15.45:0/1588490769 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f364001e460 con 0x7f364406a060 2023-10-06T05:22:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.576+0000 7f3639ffb700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3628066f30 0x7f36280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.577+0000 7f3639ffb700 1 -- 172.21.15.45:0/1588490769 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f3640084f40 con 0x7f364406a060 2023-10-06T05:22:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.577+0000 7f36495a8700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3628066f30 0x7f36280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:59.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.577+0000 7f36495a8700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3628066f30 0x7f36280693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f364406ed80 tx=0x7f3634016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.581+0000 7f3639ffb700 1 -- 172.21.15.45:0/1588490769 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f364416a9f0 con 0x7f364406a060 2023-10-06T05:22:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.829+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f364416a9f0 con 0x7f364406a060 2023-10-06T05:22:59.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.829+0000 7f3639ffb700 1 -- 172.21.15.45:0/1588490769 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f364416a9f0 con 0x7f364406a060 2023-10-06T05:22:59.831 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:22:59.831 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:22:59.005475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"c","rank":0,"incarnation":133,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/2531263654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2531263654},{"type":"v1","addr":"172.21.15.45:6835","nonce":2531263654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3628066f30 msgr2=0x7f36280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3628066f30 0x7f36280693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f364406ed80 tx=0x7f3634016000 comp rx=0 tx=0).stop 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 msgr2=0x7f36441517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36441517f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f364000d890 tx=0x7f3640008040 comp rx=0 tx=0).stop 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 shutdown_connections 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3628066f30 0x7f36280693e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3644158f60 0x7f36440c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f364406a060 0x7f36441517f0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 --2- 172.21.15.45:0/1588490769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36440cc390 0x7f36440b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:59.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 >> 172.21.15.45:0/1588490769 conn(0x7f36440cdf50 msgr2=0x7f3644152a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:59.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 shutdown_connections 2023-10-06T05:22:59.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:22:59.832+0000 7f364c00d700 1 -- 172.21.15.45:0/1588490769 wait complete. 2023-10-06T05:22:59.835 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 133 2023-10-06T05:22:59.847 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 133, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:22:59.005475+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8247': {'gid': 8247, 'name': 'c', 'rank': 0, 'incarnation': 133, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/2531263654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 2531263654}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2531263654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:59.847 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8247:up:replay 2023-10-06T05:22:59.847 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:23:02.445 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:02.444+0000 7ff3f8a78700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:05.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:05.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.390+0000 7fd4262c5700 1 Processor -- start 2023-10-06T05:23:05.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.390+0000 7fd4262c5700 1 -- start start 2023-10-06T05:23:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd4262c5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd420155cd0 0x7fd42015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd4262c5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4201560a0 0x7fd4200adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd4262c5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200ae8d0 con 0x7fd4200ccdc0 2023-10-06T05:23:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd4262c5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4200aebb0 con 0x7fd420155cd0 2023-10-06T05:23:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd4262c5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200aee90 con 0x7fd4201560a0 2023-10-06T05:23:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd41ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd420155cd0 0x7fd42015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4201560a0 0x7fd4200adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd424862700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd4200ccdc0 0x7fd4200c73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:41468/0 (socket says 172.21.15.45:41468) 2023-10-06T05:23:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.391+0000 7fd424862700 1 -- 172.21.15.45:0/3684169807 learned_addr learned my addr 172.21.15.45:0/3684169807 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41effd700 1 -- 172.21.15.45:0/3684169807 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081114512 0 0) 0x7fd4200ae8d0 con 0x7fd4200ccdc0 2023-10-06T05:23:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41effd700 1 -- 172.21.15.45:0/3684169807 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd408003570 con 0x7fd4200ccdc0 2023-10-06T05:23:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41ffff700 1 -- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4201560a0 msgr2=0x7fd4200adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41ffff700 1 --2- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4201560a0 0x7fd4200adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41ffff700 1 -- 172.21.15.45:0/3684169807 >> v1:172.21.15.144:6789/0 conn(0x7fd4200ccdc0 legacy=0x7fd4200c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41ffff700 1 -- 172.21.15.45:0/3684169807 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200af170 con 0x7fd420155cd0 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41f7fe700 1 --2- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4201560a0 0x7fd4200adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41ffff700 1 --2- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 0x7fd42015a730 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd4100098e0 tx=0x7fd4100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7e6d1d0264d618 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41effd700 1 -- 172.21.15.45:0/3684169807 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410011630 con 0x7fd420155cd0 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41effd700 1 -- 172.21.15.45:0/3684169807 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410011790 con 0x7fd420155cd0 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.392+0000 7fd41effd700 1 -- 172.21.15.45:0/3684169807 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410011b30 con 0x7fd420155cd0 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 -- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 msgr2=0x7fd42015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 0x7fd42015a730 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd4100098e0 tx=0x7fd4100091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 -- 172.21.15.45:0/3684169807 shutdown_connections 2023-10-06T05:23:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 0x7fd42015a730 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3684169807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4201560a0 0x7fd4200adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 -- 172.21.15.45:0/3684169807 >> 172.21.15.45:0/3684169807 conn(0x7fd42006f090 msgr2=0x7fd42006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 -- 172.21.15.45:0/3684169807 shutdown_connections 2023-10-06T05:23:05.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.393+0000 7fd4262c5700 1 -- 172.21.15.45:0/3684169807 wait complete. 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.394+0000 7fd4262c5700 1 Processor -- start 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.394+0000 7fd4262c5700 1 -- start start 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd4262c5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4200ccdc0 0x7fd4200cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd4262c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 0x7fd420169820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd4262c5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4201560a0 0x7fd4200c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd4262c5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42016ce60 con 0x7fd4201560a0 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd4262c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200cb630 con 0x7fd420155cd0 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd4262c5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200cacc0 con 0x7fd4200ccdc0 2023-10-06T05:23:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd424862700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4201560a0 0x7fd4200c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd424862700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4201560a0 0x7fd4200c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42902/0 (socket says 172.21.15.45:42902) 2023-10-06T05:23:05.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd424862700 1 -- 172.21.15.45:0/3758549778 learned_addr learned my addr 172.21.15.45:0/3758549778 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd41ffff700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4200ccdc0 0x7fd4200cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.395+0000 7fd41f7fe700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 0x7fd420169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd424862700 1 -- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4200ccdc0 msgr2=0x7fd4200cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd424862700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4200ccdc0 0x7fd4200cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd424862700 1 -- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 msgr2=0x7fd420169820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd424862700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 0x7fd420169820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd424862700 1 -- 172.21.15.45:0/3758549778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009420 con 0x7fd4201560a0 2023-10-06T05:23:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd424862700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4201560a0 0x7fd4200c8ca0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fd418001460 tx=0x7fd418001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd407fff700 1 -- 172.21.15.45:0/3758549778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800a830 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd407fff700 1 -- 172.21.15.45:0/3758549778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41800a990 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd410009420 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd407fff700 1 -- 172.21.15.45:0/3758549778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800ad10 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.396+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd420156dd0 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.397+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42016cfd0 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.398+0000 7fd407fff700 1 -- 172.21.15.45:0/3758549778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd418007300 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.398+0000 7fd407fff700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd40c066f30 0x7fd40c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.398+0000 7fd407fff700 1 -- 172.21.15.45:0/3758549778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fd418084e00 con 0x7fd4201560a0 2023-10-06T05:23:05.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.398+0000 7fd41ffff700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd40c066f30 0x7fd40c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.399+0000 7fd41ffff700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd40c066f30 0x7fd40c0693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd4100091e0 tx=0x7fd410017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.402+0000 7fd407fff700 1 -- 172.21.15.45:0/3758549778 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd42016cfd0 con 0x7fd4201560a0 2023-10-06T05:23:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.653+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd42016cfd0 con 0x7fd4201560a0 2023-10-06T05:23:05.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.653+0000 7fd407fff700 1 -- 172.21.15.45:0/3758549778 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4607 (secure 0 0 0) 0x7fd42016cfd0 con 0x7fd4201560a0 2023-10-06T05:23:05.655 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:05.655 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":136,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:02.015123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"c","rank":0,"incarnation":133,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2531263654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2531263654},{"type":"v1","addr":"172.21.15.45:6835","nonce":2531263654}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd40c066f30 msgr2=0x7fd40c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd40c066f30 0x7fd40c0693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd4100091e0 tx=0x7fd410017000 comp rx=0 tx=0).stop 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4201560a0 msgr2=0x7fd4200c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4201560a0 0x7fd4200c8ca0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fd418001460 tx=0x7fd418001ae0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 shutdown_connections 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd40c066f30 0x7fd40c0693e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4201560a0 0x7fd4200c8ca0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4200ccdc0 0x7fd4200cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 --2- 172.21.15.45:0/3758549778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd420155cd0 0x7fd420169820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.655+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 >> 172.21.15.45:0/3758549778 conn(0x7fd42006f090 msgr2=0x7fd420068350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.656+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 shutdown_connections 2023-10-06T05:23:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.656+0000 7fd4262c5700 1 -- 172.21.15.45:0/3758549778 wait complete. 2023-10-06T05:23:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 136 2023-10-06T05:23:05.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 136, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:23:02.015123+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8247': {'gid': 8247, 'name': 'c', 'rank': 0, 'incarnation': 133, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/2531263654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 2531263654}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2531263654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:23:05.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:23:05.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:23:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.803+0000 7f0591c35700 1 Processor -- start 2023-10-06T05:23:05.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.803+0000 7f0591c35700 1 -- start start 2023-10-06T05:23:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.803+0000 7f0591c35700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f058c156ce0 0x7f058c1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.803+0000 7f0591c35700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f058c15ac10 0x7f058c0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.803+0000 7f0591c35700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c06bf70 con 0x7f058c06bad0 2023-10-06T05:23:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.803+0000 7f0591c35700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c0b02f0 con 0x7f058c15ac10 2023-10-06T05:23:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.803+0000 7f0591c35700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c15b380 con 0x7f058c156ce0 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f058c15ac10 0x7f058c0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f058c15ac10 0x7f058c0ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42906/0 (socket says 172.21.15.45:42906) 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 -- 172.21.15.45:0/3811994111 learned_addr learned my addr 172.21.15.45:0/3811994111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058b7fe700 1 --2- 172.21.15.45:0/3811994111 >> v2:172.21.15.144:3300/0 conn(0x7f058c156ce0 0x7f058c1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 -- 172.21.15.45:0/3811994111 >> v2:172.21.15.144:3300/0 conn(0x7f058c156ce0 msgr2=0x7f058c1570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 --2- 172.21.15.45:0/3811994111 >> v2:172.21.15.144:3300/0 conn(0x7f058c156ce0 0x7f058c1570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 -- 172.21.15.45:0/3811994111 >> v1:172.21.15.144:6789/0 conn(0x7f058c06bad0 legacy=0x7f058c0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 -- 172.21.15.45:0/3811994111 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c15b660 con 0x7f058c15ac10 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058bfff700 1 --2- 172.21.15.45:0/3811994111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c15ac10 0x7f058c0ccce0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f058c0af900 tx=0x7f0584009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fd58daa7b41c9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058a7fc700 1 -- 172.21.15.45:0/3811994111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058400e070 con 0x7f058c15ac10 2023-10-06T05:23:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058a7fc700 1 -- 172.21.15.45:0/3811994111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058400ba20 con 0x7f058c15ac10 2023-10-06T05:23:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.804+0000 7f058a7fc700 1 -- 172.21.15.45:0/3811994111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05840092e0 con 0x7f058c15ac10 2023-10-06T05:23:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 -- 172.21.15.45:0/3811994111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c15ac10 msgr2=0x7f058c0ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 --2- 172.21.15.45:0/3811994111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c15ac10 0x7f058c0ccce0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f058c0af900 tx=0x7f0584009840 comp rx=0 tx=0).stop 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 -- 172.21.15.45:0/3811994111 shutdown_connections 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 --2- 172.21.15.45:0/3811994111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c15ac10 0x7f058c0ccce0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 --2- 172.21.15.45:0/3811994111 >> v2:172.21.15.144:3300/0 conn(0x7f058c156ce0 0x7f058c1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 -- 172.21.15.45:0/3811994111 >> 172.21.15.45:0/3811994111 conn(0x7f058c06f090 msgr2=0x7f058c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 -- 172.21.15.45:0/3811994111 shutdown_connections 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.805+0000 7f0591c35700 1 -- 172.21.15.45:0/3811994111 wait complete. 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 Processor -- start 2023-10-06T05:23:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 -- start start 2023-10-06T05:23:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c06bad0 0x7f058c1684f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f058c156ce0 0x7f058c1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058c15ac10 0x7f058c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c15a380 con 0x7f058c06bad0 2023-10-06T05:23:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c157f80 con 0x7f058c15ac10 2023-10-06T05:23:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.807+0000 7f0591c35700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c168a30 con 0x7f058c156ce0 2023-10-06T05:23:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c06bad0 0x7f058c1684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c06bad0 0x7f058c1684f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42908/0 (socket says 172.21.15.45:42908) 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 -- 172.21.15.45:0/2916366771 learned_addr learned my addr 172.21.15.45:0/2916366771 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058bfff700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058c15ac10 0x7f058c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058affd700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f058c156ce0 0x7f058c1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 -- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f058c156ce0 msgr2=0x7f058c1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f058c156ce0 0x7f058c1533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 -- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058c15ac10 msgr2=0x7f058c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058c15ac10 0x7f058c0c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 -- 172.21.15.45:0/2916366771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0584009ae0 con 0x7f058c06bad0 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f058b7fe700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c06bad0 0x7f058c1684f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f057c00ec80 tx=0x7f057c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f05837fe700 1 -- 172.21.15.45:0/2916366771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c00c850 con 0x7f058c06bad0 2023-10-06T05:23:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f05837fe700 1 -- 172.21.15.45:0/2916366771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c00c9b0 con 0x7f058c06bad0 2023-10-06T05:23:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f05837fe700 1 -- 172.21.15.45:0/2916366771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c00cd50 con 0x7f058c06bad0 2023-10-06T05:23:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c0c5460 con 0x7f058c06bad0 2023-10-06T05:23:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.808+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c157930 con 0x7f058c06bad0 2023-10-06T05:23:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.810+0000 7f05837fe700 1 -- 172.21.15.45:0/2916366771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f057c013680 con 0x7f058c06bad0 2023-10-06T05:23:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.811+0000 7f05837fe700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f056c066f30 0x7f056c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.811+0000 7f05837fe700 1 -- 172.21.15.45:0/2916366771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f057c084d70 con 0x7f058c06bad0 2023-10-06T05:23:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.811+0000 7f058affd700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f056c066f30 0x7f056c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.811+0000 7f058affd700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f056c066f30 0x7f056c0693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f058c15a8d0 tx=0x7f0574009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.813+0000 7f059af49b80 1 Processor -- start 2023-10-06T05:23:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.813+0000 7f059af49b80 1 -- start start 2023-10-06T05:23:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x56175497e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561754963f40 0x561754986f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f059af49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617549874d0 0x561754987940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f059af49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561754851310 con 0x5617549874d0 2023-10-06T05:23:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175488e130 con 0x561754963f40 2023-10-06T05:23:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56175488baf0 con 0x56175497e450 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617549874d0 0x561754987940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617549874d0 0x561754987940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42922/0 (socket says 172.21.15.45:42922) 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 -- 172.21.15.45:0/1011451411 learned_addr learned my addr 172.21.15.45:0/1011451411 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058b7fe700 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x56175497e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058affd700 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561754963f40 0x561754986f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 -- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 msgr2=0x56175497e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x56175497e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 -- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561754963f40 msgr2=0x561754986f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561754963f40 0x561754986f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 -- 172.21.15.45:0/1011451411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56175468d790 con 0x5617549874d0 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.814+0000 7f058bfff700 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617549874d0 0x561754987940 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f058c06aed0 tx=0x7f0584018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b83a8dcb378f94b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f0580ff9700 1 -- 172.21.15.45:0/1011451411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058400be30 con 0x5617549874d0 2023-10-06T05:23:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f0580ff9700 1 -- 172.21.15.45:0/1011451411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058400bb90 con 0x5617549874d0 2023-10-06T05:23:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f0580ff9700 1 -- 172.21.15.45:0/1011451411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584007860 con 0x5617549874d0 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 -- 172.21.15.45:0/1011451411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617549874d0 msgr2=0x561754987940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617549874d0 0x561754987940 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f058c06aed0 tx=0x7f0584018000 comp rx=0 tx=0).stop 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 -- 172.21.15.45:0/1011451411 shutdown_connections 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617549874d0 0x561754987940 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x56175497e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 --2- 172.21.15.45:0/1011451411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561754963f40 0x561754986f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 -- 172.21.15.45:0/1011451411 >> 172.21.15.45:0/1011451411 conn(0x561754981880 msgr2=0x561754981c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 -- 172.21.15.45:0/1011451411 shutdown_connections 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.815+0000 7f059af49b80 1 -- 172.21.15.45:0/1011451411 wait complete. 2023-10-06T05:23:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.816+0000 7f059af49b80 1 -- start start 2023-10-06T05:23:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.816+0000 7f059af49b80 1 Processor -- start 2023-10-06T05:23:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x56175495ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56175497e450 0x56175497d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5617549874d0 0x561754937590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617548878e0 con 0x561754963f40 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561754833980 con 0x56175497e450 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561754833620 con 0x5617549874d0 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 10 client.0 _ll_register_callbacks cb 0x561754696a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x56175495ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x56175495ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42928/0 (socket says 172.21.15.45:42928) 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 -- 172.21.15.45:0/3623745895 learned_addr learned my addr 172.21.15.45:0/3623745895 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058bfff700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5617549874d0 0x561754937590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058affd700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56175497e450 0x56175497d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 -- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5617549874d0 msgr2=0x561754937590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5617549874d0 0x561754937590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 -- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56175497e450 msgr2=0x56175497d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56175497e450 0x56175497d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 -- 172.21.15.45:0/3623745895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c15b660 con 0x561754963f40 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058bfff700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5617549874d0 0x561754937590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058affd700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56175497e450 0x56175497d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f058b7fe700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x56175495ee60 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c430 tx=0x7f057c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200d4ac0111c16d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f0569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f0569ffb700 1 -- 172.21.15.45:0/3623745895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c05c6a0 con 0x561754963f40 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f0569ffb700 1 -- 172.21.15.45:0/3623745895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f057c054bf0 con 0x561754963f40 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.818+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56175492bd60 con 0x561754963f40 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f0569ffb700 1 -- 172.21.15.45:0/3623745895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057c05c210 con 0x561754963f40 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 10 client.8304 fetch_fsmap learned FSMap version 136 2023-10-06T05:23:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x56175477a0a0 con 0x561754963f40 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f0569ffb700 1 -- 172.21.15.45:0/3623745895 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56175492bd60 con 0x561754963f40 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f0569ffb700 1 -- 172.21.15.45:0/3623745895 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 136) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f057c06bd20 con 0x561754963f40 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 10 client.8304 fetch_fsmap finished waiting for FSMap version 136 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 10 client.8304 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 20 client.8304 populate_metadata read hostname 'smithi045' 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x56175495da80 0x56175495de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 4 client.8304 mds_command: new command op to 8247 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 --> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561754908b30 con 0x56175495da80 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f058affd700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x56175495da80 0x56175495de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f058affd700 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x56175495da80 0x56175495de50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c188e90080d1f2a7 server_cookie=565168bc9c22e02 in_seq=0 out_seq=0 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.819+0000 7f0569ffb700 10 client.8304 ms_handle_connect on v2:172.21.15.45:6834/2531263654 2023-10-06T05:23:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f0569ffb700 1 -- 172.21.15.45:0/3623745895 <== mds.0 v2:172.21.15.45:6834/2531263654 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561754908b30 con 0x56175495da80 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f0569ffb700 10 client.8304 handle_command_reply: tid=0 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 client.8304 shutdown 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 msgr2=0x56175495ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x56175495ee60 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c430 tx=0x7f057c015040 comp rx=0 tx=0).stop 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 shutdown_connections 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5617549874d0 0x561754937590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x56175495da80 0x56175495de50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x56175495ee60 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 --2- 172.21.15.45:0/3623745895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56175497e450 0x56175497d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 >> 172.21.15.45:0/3623745895 conn(0x561754981880 msgr2=0x561754933c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f0569ffb700 0 client.8304 ms_handle_reset on v2:172.21.15.45:6834/2531263654 2023-10-06T05:23:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 shutdown_connections 2023-10-06T05:23:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 1 -- 172.21.15.45:0/3623745895 wait complete. 2023-10-06T05:23:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.820+0000 7f059af49b80 20 client.8304 trim_cache size 0 max 16384 2023-10-06T05:23:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.838+0000 7f059af49b80 1 Processor -- start 2023-10-06T05:23:05.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.838+0000 7f059af49b80 1 -- start start 2023-10-06T05:23:05.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561754963f40 0x56175495b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f059af49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56175497e450 0x56175495fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x5617549600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f059af49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175471f1e0 con 0x56175497e450 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175495ad10 con 0x5617549874d0 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56175495af10 con 0x561754963f40 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56175497e450 0x56175495fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56175497e450 0x56175495fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42934/0 (socket says 172.21.15.45:42934) 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 -- 172.21.15.45:0/2074317893 learned_addr learned my addr 172.21.15.45:0/2074317893 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058b7fe700 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561754963f40 0x56175495b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058bfff700 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x5617549600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 -- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561754963f40 msgr2=0x56175495b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561754963f40 0x56175495b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 -- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 msgr2=0x5617549600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x5617549600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.839+0000 7f058affd700 1 -- 172.21.15.45:0/2074317893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617548563c0 con 0x56175497e450 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f058affd700 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56175497e450 0x56175495fb90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f057400e440 tx=0x7f0574013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e60b77d85fc9e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f05817fa700 1 -- 172.21.15.45:0/2074317893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05740250b0 con 0x56175497e450 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f05817fa700 1 -- 172.21.15.45:0/2074317893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574029070 con 0x56175497e450 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f05817fa700 1 -- 172.21.15.45:0/2074317893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574006820 con 0x56175497e450 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 -- 172.21.15.45:0/2074317893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56175497e450 msgr2=0x56175495fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56175497e450 0x56175495fb90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f057400e440 tx=0x7f0574013000 comp rx=0 tx=0).stop 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 -- 172.21.15.45:0/2074317893 shutdown_connections 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x5617549600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561754963f40 0x56175495b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 --2- 172.21.15.45:0/2074317893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56175497e450 0x56175495fb90 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 -- 172.21.15.45:0/2074317893 >> 172.21.15.45:0/2074317893 conn(0x561754981880 msgr2=0x5617549347b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 -- 172.21.15.45:0/2074317893 shutdown_connections 2023-10-06T05:23:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.840+0000 7f059af49b80 1 -- 172.21.15.45:0/2074317893 wait complete. 2023-10-06T05:23:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.841+0000 7f059af49b80 1 -- start start 2023-10-06T05:23:05.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.841+0000 7f059af49b80 1 Processor -- start 2023-10-06T05:23:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f059af49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x5617549602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x561754996ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f059af49b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x56175492d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f059af49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617549e3090 con 0x561754963f40 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175495f6e0 con 0x5617549874d0 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f059af49b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617549e3200 con 0x56175497e450 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f059af49b80 10 client.0 _ll_register_callbacks cb 0x561754939e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x5617549602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x5617549602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42940/0 (socket says 172.21.15.45:42940) 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 -- 172.21.15.45:0/1090496537 learned_addr learned my addr 172.21.15.45:0/1090496537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058affd700 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x561754996ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058bfff700 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x56175492d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 -- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 msgr2=0x561754996ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x561754996ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 -- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 msgr2=0x56175492d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:05.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x56175492d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 -- 172.21.15.45:0/1090496537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0574018040 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f058b7fe700 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x5617549602e0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c00 tx=0x7f057c06a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03c55bb31522a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.842+0000 7f0569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f0569ffb700 1 -- 172.21.15.45:0/1090496537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c05c6a0 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f0569ffb700 1 -- 172.21.15.45:0/1090496537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f057c054bf0 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f058400bb90 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f0569ffb700 1 -- 172.21.15.45:0/1090496537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057c00e040 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 10 client.8310 fetch_fsmap learned FSMap version 136 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x5617546ee1d0 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f0569ffb700 1 -- 172.21.15.45:0/1090496537 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058400bb90 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f0569ffb700 1 -- 172.21.15.45:0/1090496537 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 136) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f057c011070 con 0x561754963f40 2023-10-06T05:23:05.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 10 client.8310 fetch_fsmap finished waiting for FSMap version 136 2023-10-06T05:23:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 10 client.8310 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:23:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 20 client.8310 populate_metadata read hostname 'smithi045' 2023-10-06T05:23:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x561754966810 0x561754966be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 4 client.8310 mds_command: new command op to 8247 tid=0[{"prefix": "status"}] 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 --> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56175482a8d0 con 0x561754966810 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.843+0000 7f058affd700 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x561754966810 0x561754966be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f0569ffb700 10 client.8310 ms_handle_connect on v2:172.21.15.45:6834/2531263654 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f058affd700 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x561754966810 0x561754966be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474318d2ef1a75b0 server_cookie=5ffdbb61163c77bf in_seq=0 out_seq=0 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f0569ffb700 1 -- 172.21.15.45:0/1090496537 <== mds.0 v2:172.21.15.45:6834/2531263654 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56175482a8d0 con 0x561754966810 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f0569ffb700 10 client.8310 handle_command_reply: tid=0 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 client.8310 shutdown 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 msgr2=0x5617549602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x5617549602e0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f057c000c00 tx=0x7f057c06a000 comp rx=0 tx=0).stop 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 shutdown_connections 2023-10-06T05:23:05.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617549874d0 0x56175492d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:6834/2531263654,v1:172.21.15.45:6835/2531263654] conn(0x561754966810 0x561754966be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561754963f40 0x5617549602e0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 --2- 172.21.15.45:0/1090496537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56175497e450 0x561754996ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f0569ffb700 0 client.8310 ms_handle_reset on v2:172.21.15.45:6834/2531263654 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 >> 172.21.15.45:0/1090496537 conn(0x561754981880 msgr2=0x561754958130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 shutdown_connections 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 1 -- 172.21.15.45:0/1090496537 wait complete. 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.844+0000 7f059af49b80 20 client.8310 trim_cache size 0 max 16384 2023-10-06T05:23:05.854 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8247, 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.838218972, 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 136, 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 99, 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 99, 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 9.3825122079999996 2023-10-06T05:23:05.855 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:23:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f056c066f30 msgr2=0x7f056c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f056c066f30 0x7f056c0693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f058c15a8d0 tx=0x7f0574009040 comp rx=0 tx=0).stop 2023-10-06T05:23:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c06bad0 msgr2=0x7f058c1684f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c06bad0 0x7f058c1684f0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f057c00ec80 tx=0x7f057c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 shutdown_connections 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f056c066f30 0x7f056c0693e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058c15ac10 0x7f058c0c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c06bad0 0x7f058c1684f0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 --2- 172.21.15.45:0/2916366771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f058c156ce0 0x7f058c1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 >> 172.21.15.45:0/2916366771 conn(0x7f058c06f090 msgr2=0x7f058c153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 shutdown_connections 2023-10-06T05:23:05.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.845+0000 7f0591c35700 1 -- 172.21.15.45:0/2916366771 wait complete. 2023-10-06T05:23:05.860 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:23:05.860 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2023-10-06T05:23:05.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.990+0000 7ff0ec1e3700 1 Processor -- start 2023-10-06T05:23:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.990+0000 7ff0ec1e3700 1 -- start start 2023-10-06T05:23:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.991+0000 7ff0ec1e3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff0e40c93c0 0x7ff0e4156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.991+0000 7ff0ec1e3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e415b000 con 0x7ff0e4157030 2023-10-06T05:23:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.991+0000 7ff0ec1e3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e415b2e0 con 0x7ff0e40c8ff0 2023-10-06T05:23:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.991+0000 7ff0ec1e3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40c9830 con 0x7ff0e40c93c0 2023-10-06T05:23:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.991+0000 7ff0ea780700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff0e4157030 0x7ff0e40cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49618/0 (socket says 172.21.15.45:49618) 2023-10-06T05:23:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.991+0000 7ff0ea780700 1 -- 172.21.15.45:0/3257129366 learned_addr learned my addr 172.21.15.45:0/3257129366 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.991+0000 7ff0e977e700 1 --2- 172.21.15.45:0/3257129366 >> v2:172.21.15.144:3300/0 conn(0x7ff0e40c93c0 0x7ff0e4156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e8f7d700 1 -- 172.21.15.45:0/3257129366 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926279842 0 0) 0x7ff0e415b000 con 0x7ff0e4157030 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e8f7d700 1 -- 172.21.15.45:0/3257129366 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d0003570 con 0x7ff0e4157030 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e977e700 1 -- 172.21.15.45:0/3257129366 >> v1:172.21.15.144:6789/0 conn(0x7ff0e40c8ff0 legacy=0x7ff0e40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e977e700 1 -- 172.21.15.45:0/3257129366 >> v1:172.21.15.45:6789/0 conn(0x7ff0e4157030 legacy=0x7ff0e40cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e8f7d700 1 -- 172.21.15.45:0/3257129366 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972798045 0 0) 0x7ff0e415b2e0 con 0x7ff0e40c8ff0 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e977e700 1 -- 172.21.15.45:0/3257129366 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e415b5c0 con 0x7ff0e40c93c0 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e977e700 1 --2- 172.21.15.45:0/3257129366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e4156af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40098e0 tx=0x7ff0d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb29b7ae9310c4d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e8f7d700 1 -- 172.21.15.45:0/3257129366 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400b640 con 0x7ff0e40c93c0 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e8f7d700 1 -- 172.21.15.45:0/3257129366 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400b7a0 con 0x7ff0e40c93c0 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.992+0000 7ff0e8f7d700 1 -- 172.21.15.45:0/3257129366 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400bb40 con 0x7ff0e40c93c0 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.993+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/3257129366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 msgr2=0x7ff0e4156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.993+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/3257129366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e4156af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40098e0 tx=0x7ff0d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.993+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/3257129366 shutdown_connections 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.993+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/3257129366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e4156af0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.993+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/3257129366 >> 172.21.15.45:0/3257129366 conn(0x7ff0e40cdf50 msgr2=0x7ff0e40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.993+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/3257129366 shutdown_connections 2023-10-06T05:23:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.993+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/3257129366 wait complete. 2023-10-06T05:23:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.994+0000 7ff0ec1e3700 1 Processor -- start 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.994+0000 7ff0ec1e3700 1 -- start start 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0ec1e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e40c8ff0 0x7ff0e40b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0ec1e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e40aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0ec1e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e4157030 0x7ff0e4068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0ec1e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4069740 con 0x7ff0e40c8ff0 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0ec1e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40ae9a0 con 0x7ff0e40c93c0 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0ec1e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4069180 con 0x7ff0e4157030 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0e9f7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e40c8ff0 0x7ff0e40b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0e9f7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e40c8ff0 0x7ff0e40b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42942/0 (socket says 172.21.15.45:42942) 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0e9f7f700 1 -- 172.21.15.45:0/2013586228 learned_addr learned my addr 172.21.15.45:0/2013586228 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0ea780700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e4157030 0x7ff0e4068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0e9f7f700 1 -- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e4157030 msgr2=0x7ff0e4068730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.995+0000 7ff0e977e700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e40aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0e9f7f700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e4157030 0x7ff0e4068730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0e9f7f700 1 -- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 msgr2=0x7ff0e40aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0e9f7f700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e40aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0e9f7f700 1 -- 172.21.15.45:0/2013586228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d4009420 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0e977e700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e40aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0ea780700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e4157030 0x7ff0e4068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0e9f7f700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e40c8ff0 0x7ff0e40b02e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c530 tx=0x7ff0e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0d9ffb700 1 -- 172.21.15.45:0/2013586228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011590 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e4069460 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0d9ffb700 1 -- 172.21.15.45:0/2013586228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e00116f0 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0d9ffb700 1 -- 172.21.15.45:0/2013586228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0011ab0 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.996+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e4069cf0 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.997+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e40ae490 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.998+0000 7ff0d9ffb700 1 -- 172.21.15.45:0/2013586228 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0e001b3b0 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.998+0000 7ff0d9ffb700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0c8066f30 0x7ff0c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.998+0000 7ff0d9ffb700 1 -- 172.21.15.45:0/2013586228 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7ff0e00840b0 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.998+0000 7ff0e977e700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0c8066f30 0x7ff0c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:05.999+0000 7ff0e977e700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0c8066f30 0x7ff0c80693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff0e41501b0 tx=0x7ff0d4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.003+0000 7ff0d9ffb700 1 -- 172.21.15.45:0/2013586228 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0e40ae490 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.258+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff0e40ae490 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.258+0000 7ff0d9ffb700 1 -- 172.21.15.45:0/2013586228 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v136) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff0e40ae490 con 0x7ff0e40c8ff0 2023-10-06T05:23:06.260 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0c8066f30 msgr2=0x7ff0c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0c8066f30 0x7ff0c80693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff0e41501b0 tx=0x7ff0d4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e40c8ff0 msgr2=0x7ff0e40b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e40c8ff0 0x7ff0e40b02e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c530 tx=0x7ff0e000c8f0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 shutdown_connections 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0c8066f30 0x7ff0c80693e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e4157030 0x7ff0e4068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e40c8ff0 0x7ff0e40b02e0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 --2- 172.21.15.45:0/2013586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e40c93c0 0x7ff0e40aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 >> 172.21.15.45:0/2013586228 conn(0x7ff0e40cdf50 msgr2=0x7ff0e40c45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 shutdown_connections 2023-10-06T05:23:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:06.261+0000 7ff0ec1e3700 1 -- 172.21.15.45:0/2013586228 wait complete. 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:06.276 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:06.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:06.277 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:23:06.298 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:06.298 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:06.298 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:06.299 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41955sec preferred_lft 41955sec 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:23:06.300 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:06.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:06.302 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:06.370 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:23:06.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:06.371 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:23:06.437 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:23:06.438 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:06.438 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:23:06.438 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:23:06.439 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:06.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:06.439 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:23:06.439 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:23:06.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:23:06.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:23:06.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:23:06.440 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:23:06.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:07.126 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:23:07.150 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:23:07.749 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:23:07.749 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:07.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:07.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:23:07.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:07.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:07.788 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:23:07.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:07.901 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:07.966 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:08.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:08.052 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:08.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:23:08.152 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:23:08.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:23:08.270 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:23:08.341 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:23:08.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:08.342 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:23:08.406 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:23:08.406 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:23:08.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:23:08.484 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:23:08.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:23:08.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:23:08.547+0000 7fcc17f72540 -1 init, newargv = 0x5653294099b0 newargc=15 2023-10-06T05:23:08.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[162081]: starting ceph client 2023-10-06T05:23:08.555 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[162081]: starting fuse 2023-10-06T05:23:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:23:08.558 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:23:08.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:08.559 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:23:08.620 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:23:08.620 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:23:08.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:23:08.621 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:23:08.622 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:23:08.761 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.162081.asok 2023-10-06T05:23:08.942 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.162081.asok 2023-10-06T05:23:08.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:08.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.162081.asok status 2023-10-06T05:23:09.118 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:23:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:23:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:23:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:23:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:23:09.118 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "162081", 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8319, 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 8319 2023-10-06T05:23:09.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1331581631 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1331581631 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:09.120 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.8319 192.168.0.1:0/1331581631", 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1331581631", 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 136, 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 99, 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:23:09.121 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:23:09.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:09.129 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:23:09.129 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:09.149 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:23:09.150 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:09.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:09.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:09.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:09.223 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:23:09.267 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:23:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:23:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:23:09.293 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:23:09.294 INFO:teuthology.orchestra.run.smithi045.stdout:sixmegs 2023-10-06T05:23:09.294 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2023-10-06T05:23:09.294 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2023-10-06T05:23:09.295 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:23:09.295 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:23:09.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:09.378 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:23:09.378 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:09.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:09.379 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:09.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.623+0000 7ffbc761b700 1 Processor -- start 2023-10-06T05:23:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.623+0000 7ffbc761b700 1 -- start start 2023-10-06T05:23:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbc761b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc0159200 0x7ffbc00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbc761b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc00c7d00 con 0x7ffbc01595d0 2023-10-06T05:23:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbc761b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc00c86d0 con 0x7ffbc00ccdc0 2023-10-06T05:23:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbc761b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00c7780 con 0x7ffbc0159200 2023-10-06T05:23:09.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbc4bb6700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffbc01595d0 0x7ffbc006b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38194/0 (socket says 172.21.15.45:38194) 2023-10-06T05:23:09.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbc4bb6700 1 -- 172.21.15.45:0/2829799911 learned_addr learned my addr 172.21.15.45:0/2829799911 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:09.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbc53b7700 1 --2- 172.21.15.45:0/2829799911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc0159200 0x7ffbc00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763257712 0 0) 0x7ffbc00c7d00 con 0x7ffbc01595d0 2023-10-06T05:23:09.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.624+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbac003570 con 0x7ffbc01595d0 2023-10-06T05:23:09.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943872242 0 0) 0x7ffbc00c86d0 con 0x7ffbc00ccdc0 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc00c7d00 con 0x7ffbc00ccdc0 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbc53b7700 1 -- 172.21.15.45:0/2829799911 >> v1:172.21.15.144:6789/0 conn(0x7ffbc00ccdc0 legacy=0x7ffbc00acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbc53b7700 1 -- 172.21.15.45:0/2829799911 >> v1:172.21.15.45:6789/0 conn(0x7ffbc01595d0 legacy=0x7ffbc006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2758186749 0 0) 0x7ffbac003570 con 0x7ffbc01595d0 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbc53b7700 1 -- 172.21.15.45:0/2829799911 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc006bb40 con 0x7ffbc0159200 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbc53b7700 1 --2- 172.21.15.45:0/2829799911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc0159200 0x7ffbc00c8190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009910 tx=0x7ffbbc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3979053b8aa9b3f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0001470 con 0x7ffbc0159200 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc011420 con 0x7ffbc0159200 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.625+0000 7ffbb7fff700 1 -- 172.21.15.45:0/2829799911 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0117c0 con 0x7ffbc0159200 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.626+0000 7ffbc761b700 1 -- 172.21.15.45:0/2829799911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc0159200 msgr2=0x7ffbc00c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.626+0000 7ffbc761b700 1 --2- 172.21.15.45:0/2829799911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc0159200 0x7ffbc00c8190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009910 tx=0x7ffbbc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.626+0000 7ffbc761b700 1 -- 172.21.15.45:0/2829799911 shutdown_connections 2023-10-06T05:23:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.626+0000 7ffbc761b700 1 --2- 172.21.15.45:0/2829799911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc0159200 0x7ffbc00c8190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.626+0000 7ffbc761b700 1 -- 172.21.15.45:0/2829799911 >> 172.21.15.45:0/2829799911 conn(0x7ffbc006f090 msgr2=0x7ffbc006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.626+0000 7ffbc761b700 1 -- 172.21.15.45:0/2829799911 shutdown_connections 2023-10-06T05:23:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.626+0000 7ffbc761b700 1 -- 172.21.15.45:0/2829799911 wait complete. 2023-10-06T05:23:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.627+0000 7ffbc761b700 1 Processor -- start 2023-10-06T05:23:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.627+0000 7ffbc761b700 1 -- start start 2023-10-06T05:23:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc761b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc00ccdc0 0x7ffbc0153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc761b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc0159200 0x7ffbc00c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc761b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc01595d0 0x7ffbc00c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc761b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00c62f0 con 0x7ffbc0159200 2023-10-06T05:23:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc761b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00c1730 con 0x7ffbc00ccdc0 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc761b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00c1120 con 0x7ffbc01595d0 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc4bb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc0159200 0x7ffbc00c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc4bb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc0159200 0x7ffbc00c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55758/0 (socket says 172.21.15.45:55758) 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc4bb6700 1 -- 172.21.15.45:0/353371708 learned_addr learned my addr 172.21.15.45:0/353371708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc5bb8700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc01595d0 0x7ffbc00c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.628+0000 7ffbc53b7700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc00ccdc0 0x7ffbc0153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc4bb6700 1 -- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc01595d0 msgr2=0x7ffbc00c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc4bb6700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc01595d0 0x7ffbc00c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc4bb6700 1 -- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc00ccdc0 msgr2=0x7ffbc0153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc4bb6700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc00ccdc0 0x7ffbc0153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc4bb6700 1 -- 172.21.15.45:0/353371708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc009420 con 0x7ffbc0159200 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc4bb6700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc0159200 0x7ffbc00c3e40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c7f0 tx=0x7ffbb0001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbb57fa700 1 -- 172.21.15.45:0/353371708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0007360 con 0x7ffbc0159200 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbb57fa700 1 -- 172.21.15.45:0/353371708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb00074c0 con 0x7ffbc0159200 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbb57fa700 1 -- 172.21.15.45:0/353371708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004830 con 0x7ffbc0159200 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc016a140 con 0x7ffbc0159200 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.629+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc016a420 con 0x7ffbc0159200 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.630+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc00c18e0 con 0x7ffbc0159200 2023-10-06T05:23:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.631+0000 7ffbb57fa700 1 -- 172.21.15.45:0/353371708 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbb0004c20 con 0x7ffbc0159200 2023-10-06T05:23:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.631+0000 7ffbb57fa700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.631+0000 7ffbb57fa700 1 -- 172.21.15.45:0/353371708 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6980+0+0 (secure 0 0 0) 0x7ffbb0084240 con 0x7ffbc0159200 2023-10-06T05:23:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.631+0000 7ffbc53b7700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.632+0000 7ffbc53b7700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00c6f60 tx=0x7ffbbc016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.635+0000 7ffbb57fa700 1 -- 172.21.15.45:0/353371708 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbc00c18e0 con 0x7ffbc0159200 2023-10-06T05:23:09.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.892+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ffbc00c18a0 con 0x7ffbc0159200 2023-10-06T05:23:09.894 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:09.892+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.896+0000 7ffbb57fa700 1 -- 172.21.15.45:0/353371708 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v137) v1 ==== 161+0+0 (secure 0 0 0) 0x7ffbb0051030 con 0x7ffbc0159200 2023-10-06T05:23:09.899 INFO:tasks.ceph.mds.c.smithi045.stderr: -654> 2023-10-06T05:23:02.444+0000 7ff3f8a78700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.899+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffba4066f30 msgr2=0x7ffba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.899+0000 7ffbc761b700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00c6f60 tx=0x7ffbbc016000 comp rx=0 tx=0).stop 2023-10-06T05:23:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.899+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc0159200 msgr2=0x7ffbc00c3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.899+0000 7ffbc761b700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc0159200 0x7ffbc00c3e40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c7f0 tx=0x7ffbb0001a10 comp rx=0 tx=0).stop 2023-10-06T05:23:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 shutdown_connections 2023-10-06T05:23:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc01595d0 0x7ffbc00c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc00ccdc0 0x7ffbc0153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 --2- 172.21.15.45:0/353371708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbc0159200 0x7ffbc00c3e40 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 >> 172.21.15.45:0/353371708 conn(0x7ffbc006f090 msgr2=0x7ffbc00af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 shutdown_connections 2023-10-06T05:23:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:09.900+0000 7ffbc761b700 1 -- 172.21.15.45:0/353371708 wait complete. 2023-10-06T05:23:09.903 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:09.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:09.928 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:10.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.053+0000 7f7184cdb700 1 Processor -- start 2023-10-06T05:23:10.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.053+0000 7f7184cdb700 1 -- start start 2023-10-06T05:23:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.054+0000 7f7184cdb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800cd340 0x7f7180157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.054+0000 7f7184cdb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718015b000 con 0x7f71800c8ff0 2023-10-06T05:23:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.054+0000 7f7184cdb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718015b2e0 con 0x7f71800c93c0 2023-10-06T05:23:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.054+0000 7f7184cdb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800c9830 con 0x7f71800cd340 2023-10-06T05:23:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.054+0000 7f717e59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f71800c8ff0 0x7f71800caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38214/0 (socket says 172.21.15.45:38214) 2023-10-06T05:23:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.054+0000 7f717e59c700 1 -- 172.21.15.45:0/971213802 learned_addr learned my addr 172.21.15.45:0/971213802 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.054+0000 7f717ed9d700 1 --2- 172.21.15.45:0/971213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800cd340 0x7f7180157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285725191 0 0) 0x7f718015b000 con 0x7f71800c8ff0 2023-10-06T05:23:10.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7168003570 con 0x7f71800c8ff0 2023-10-06T05:23:10.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184155958 0 0) 0x7f718015b2e0 con 0x7f71800c93c0 2023-10-06T05:23:10.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718015b000 con 0x7f71800c93c0 2023-10-06T05:23:10.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331927344 0 0) 0x7f7168003570 con 0x7f71800c8ff0 2023-10-06T05:23:10.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717ed9d700 1 -- 172.21.15.45:0/971213802 >> v1:172.21.15.144:6789/0 conn(0x7f71800c93c0 legacy=0x7f7180156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717ed9d700 1 -- 172.21.15.45:0/971213802 >> v1:172.21.15.45:6789/0 conn(0x7f71800c8ff0 legacy=0x7f71800caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717ed9d700 1 -- 172.21.15.45:0/971213802 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718015b5c0 con 0x7f71800cd340 2023-10-06T05:23:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717ed9d700 1 --2- 172.21.15.45:0/971213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800cd340 0x7f7180157200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7178009e90 tx=0x7f7178009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1eca49686a4614b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800e070 con 0x7f71800cd340 2023-10-06T05:23:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800b870 con 0x7f71800cd340 2023-10-06T05:23:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.055+0000 7f717d59a700 1 -- 172.21.15.45:0/971213802 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170001470 con 0x7f71800cd340 2023-10-06T05:23:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.056+0000 7f7184cdb700 1 -- 172.21.15.45:0/971213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800cd340 msgr2=0x7f7180157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.056+0000 7f7184cdb700 1 --2- 172.21.15.45:0/971213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800cd340 0x7f7180157200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7178009e90 tx=0x7f7178009780 comp rx=0 tx=0).stop 2023-10-06T05:23:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.056+0000 7f7184cdb700 1 -- 172.21.15.45:0/971213802 shutdown_connections 2023-10-06T05:23:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.056+0000 7f7184cdb700 1 --2- 172.21.15.45:0/971213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800cd340 0x7f7180157200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.056+0000 7f7184cdb700 1 -- 172.21.15.45:0/971213802 >> 172.21.15.45:0/971213802 conn(0x7f71800cdf50 msgr2=0x7f71800ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.056+0000 7f7184cdb700 1 -- 172.21.15.45:0/971213802 shutdown_connections 2023-10-06T05:23:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.056+0000 7f7184cdb700 1 -- 172.21.15.45:0/971213802 wait complete. 2023-10-06T05:23:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.057+0000 7f7184cdb700 1 Processor -- start 2023-10-06T05:23:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.057+0000 7f7184cdb700 1 -- start start 2023-10-06T05:23:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f7184cdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71800c8ff0 0x7f71800c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f7184cdb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800c93c0 0x7f71800c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f7184cdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cd340 0x7f718016cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f7184cdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718016d7a0 con 0x7f71800c8ff0 2023-10-06T05:23:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f7184cdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718016da80 con 0x7f71800cd340 2023-10-06T05:23:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f7184cdb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b2930 con 0x7f71800c93c0 2023-10-06T05:23:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f717e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71800c8ff0 0x7f71800c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f717e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71800c8ff0 0x7f71800c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55782/0 (socket says 172.21.15.45:55782) 2023-10-06T05:23:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f717e59c700 1 -- 172.21.15.45:0/2152737673 learned_addr learned my addr 172.21.15.45:0/2152737673 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f717ed9d700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cd340 0x7f718016cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.058+0000 7f717dd9b700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800c93c0 0x7f71800c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717e59c700 1 -- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800c93c0 msgr2=0x7f71800c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717e59c700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800c93c0 0x7f71800c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717e59c700 1 -- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cd340 msgr2=0x7f718016cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717e59c700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cd340 0x7f718016cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717e59c700 1 -- 172.21.15.45:0/2152737673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7178009a20 con 0x7f71800c8ff0 2023-10-06T05:23:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717ed9d700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cd340 0x7f718016cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:10.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717dd9b700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800c93c0 0x7f71800c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:10.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f717e59c700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71800c8ff0 0x7f71800c7860 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f717000cb90 tx=0x7f7170001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f716e7fc700 1 -- 172.21.15.45:0/2152737673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170001d20 con 0x7f71800c8ff0 2023-10-06T05:23:10.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f716e7fc700 1 -- 172.21.15.45:0/2152737673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170007360 con 0x7f71800c8ff0 2023-10-06T05:23:10.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71800b2c10 con 0x7f71800c8ff0 2023-10-06T05:23:10.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f716e7fc700 1 -- 172.21.15.45:0/2152737673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170004830 con 0x7f71800c8ff0 2023-10-06T05:23:10.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.059+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71800b25e0 con 0x7f71800c8ff0 2023-10-06T05:23:10.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.060+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718016dca0 con 0x7f71800c8ff0 2023-10-06T05:23:10.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.061+0000 7f716e7fc700 1 -- 172.21.15.45:0/2152737673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7170004ba0 con 0x7f71800c8ff0 2023-10-06T05:23:10.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.061+0000 7f716e7fc700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7160066f30 0x7f71600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.061+0000 7f716e7fc700 1 -- 172.21.15.45:0/2152737673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f7170083f00 con 0x7f71800c8ff0 2023-10-06T05:23:10.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.061+0000 7f717dd9b700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7160066f30 0x7f71600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.062+0000 7f717dd9b700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7160066f30 0x7f71600693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7174007fd0 tx=0x7f7174009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.066+0000 7f716e7fc700 1 -- 172.21.15.45:0/2152737673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7170050c00 con 0x7f71800c8ff0 2023-10-06T05:23:10.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.355+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f71800b2e00 con 0x7f71800c8ff0 2023-10-06T05:23:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.898+0000 7f716e7fc700 1 -- 172.21.15.45:0/2152737673 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v138) v1 ==== 107+0+0 (secure 0 0 0) 0x7f7170018020 con 0x7f71800c8ff0 2023-10-06T05:23:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7160066f30 msgr2=0x7f71600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7160066f30 0x7f71600693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7174007fd0 tx=0x7f7174009040 comp rx=0 tx=0).stop 2023-10-06T05:23:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71800c8ff0 msgr2=0x7f71800c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71800c8ff0 0x7f71800c7860 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f717000cb90 tx=0x7f7170001ab0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 shutdown_connections 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7160066f30 0x7f71600693e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cd340 0x7f718016cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71800c8ff0 0x7f71800c7860 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 --2- 172.21.15.45:0/2152737673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800c93c0 0x7f71800c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 >> 172.21.15.45:0/2152737673 conn(0x7f71800cdf50 msgr2=0x7f71800c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 shutdown_connections 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:10.901+0000 7f7184cdb700 1 -- 172.21.15.45:0/2152737673 wait complete. 2023-10-06T05:23:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:23:10.919 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.007+0000 7f962c0b7040 1 Processor -- start 2023-10-06T05:23:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.007+0000 7f962c0b7040 1 -- start start 2023-10-06T05:23:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.008+0000 7f962c0b7040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614755ea210 0x56147561b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.008+0000 7f962c0b7040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614756ac890 0x5614755ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.008+0000 7f962c0b7040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.008+0000 7f962c0b7040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561475537980 con 0x5614756acc60 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.008+0000 7f962c0b7040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561475537620 con 0x5614756ac890 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.008+0000 7f962c0b7040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56147552e400 con 0x5614755ea210 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.008+0000 7f96252d5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614755ea210 0x56147561b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614755ea210 0x56147561b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:53262/0 (socket says 172.21.15.45:53262) 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 -- 172.21.15.45:0/675562358 learned_addr learned my addr 172.21.15.45:0/675562358 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f9624ad4700 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614756ac890 0x5614755ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f9625ad6700 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 -- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614756ac890 msgr2=0x5614755ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614756ac890 0x5614755ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 -- 172.21.15.45:0/675562358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 msgr2=0x5614756a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756a90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 -- 172.21.15.45:0/675562358 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962000a0e0 con 0x5614755ea210 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f96252d5700 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614755ea210 0x56147561b2c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f961800acc0 tx=0x7f9618016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895456c05ac7ed8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f961f7fe700 1 -- 172.21.15.45:0/675562358 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618000bc0 con 0x5614755ea210 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f961f7fe700 1 -- 172.21.15.45:0/675562358 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9618008970 con 0x5614755ea210 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.009+0000 7f961f7fe700 1 -- 172.21.15.45:0/675562358 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961801c410 con 0x5614755ea210 2023-10-06T05:23:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 -- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614755ea210 msgr2=0x56147561b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614755ea210 0x56147561b2c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f961800acc0 tx=0x7f9618016000 comp rx=0 tx=0).stop 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 -- 172.21.15.45:0/675562358 shutdown_connections 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614755ea210 0x56147561b2c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 --2- 172.21.15.45:0/675562358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614756ac890 0x5614755ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 -- 172.21.15.45:0/675562358 >> 172.21.15.45:0/675562358 conn(0x5614755ed450 msgr2=0x5614755ef830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 -- 172.21.15.45:0/675562358 shutdown_connections 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.010+0000 7f962c0b7040 1 -- 172.21.15.45:0/675562358 wait complete. 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.011+0000 7f962c0b7040 1 Processor -- start 2023-10-06T05:23:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.011+0000 7f962c0b7040 1 -- start start 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.011+0000 7f962c0b7040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614755ea210 0x5614756aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f962c0b7040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614756ac890 0x56147561b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f962c0b7040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f962c0b7040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614756a9fe0 con 0x5614756acc60 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f962c0b7040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614756ab390 con 0x5614755ea210 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f962c0b7040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614756a6dd0 con 0x5614756ac890 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55794/0 (socket says 172.21.15.45:55794) 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 learned_addr learned my addr 172.21.15.45:0/1434982380 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9624ad4700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614756ac890 0x56147561b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614755ea210 0x5614756aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614756ac890 msgr2=0x56147561b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614756ac890 0x56147561b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614755ea210 msgr2=0x5614756aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614755ea210 0x5614756aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614755601c0 con 0x5614756acc60 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614755ea210 0x5614756aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.012+0000 7f9625ad6700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756ceed0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9620010970 tx=0x7f9620007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.013+0000 7f961dffb700 1 -- 172.21.15.45:0/1434982380 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620010e60 con 0x5614756acc60 2023-10-06T05:23:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.013+0000 7f961dffb700 1 -- 172.21.15.45:0/1434982380 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962000e8e0 con 0x5614756acc60 2023-10-06T05:23:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.013+0000 7f961dffb700 1 -- 172.21.15.45:0/1434982380 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962000ec80 con 0x5614756acc60 2023-10-06T05:23:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.013+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9618005db0 con 0x5614756acc60 2023-10-06T05:23:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.013+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614755463d0 con 0x5614756acc60 2023-10-06T05:23:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.014+0000 7f961dffb700 1 -- 172.21.15.45:0/1434982380 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9620013440 con 0x5614756acc60 2023-10-06T05:23:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.015+0000 7f961dffb700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96080658c0 0x7f9608067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.015+0000 7f961dffb700 1 -- 172.21.15.45:0/1434982380 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f9620083f80 con 0x5614756acc60 2023-10-06T05:23:11.023 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.015+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96080658c0 0x7f9608067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.015+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56147561b970 0x5614756ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.015+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756cd730 con 0x56147561b970 2023-10-06T05:23:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.015+0000 7f9624ad4700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56147561b970 0x5614756ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.016+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96080658c0 0x7f9608067d70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9618008cf0 tx=0x7f9618003050 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.016+0000 7f9624ad4700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56147561b970 0x5614756ca8f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.017+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v100'226 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.017+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756cdb10 con 0x56147561b970 2023-10-06T05:23:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.019+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v100'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.019+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756cdec0 con 0x56147561b970 2023-10-06T05:23:11.025 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.020+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v100'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.020+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ce270 con 0x56147561b970 2023-10-06T05:23:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.021+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.021+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e10b0 con 0x56147561b970 2023-10-06T05:23:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.022+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.022+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e00a0 con 0x56147561b970 2023-10-06T05:23:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.024+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'231 uv231 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.024+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e04d0 con 0x56147561b970 2023-10-06T05:23:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.025+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v100'232 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.025+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e08d0 con 0x56147561b970 2023-10-06T05:23:11.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.026+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v100'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.026+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e08d0 con 0x56147561b970 2023-10-06T05:23:11.029 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:11.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.028+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v100'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.028+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0cd0 con 0x56147561b970 2023-10-06T05:23:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.029+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.029+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0cd0 con 0x56147561b970 2023-10-06T05:23:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.032+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.032+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0880 con 0x56147561b970 2023-10-06T05:23:11.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.034+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'237 uv237 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.035+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0880 con 0x56147561b970 2023-10-06T05:23:11.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.037+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v100'238 uv213 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.037+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0880 con 0x56147561b970 2023-10-06T05:23:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.041+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'239 uv239 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.041+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0880 con 0x56147561b970 2023-10-06T05:23:11.045 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.044+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v100'240 uv240 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.044+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0880 con 0x56147561b970 2023-10-06T05:23:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.046+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'241 uv241 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.046+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0880 con 0x56147561b970 2023-10-06T05:23:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.047+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'242 uv242 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.048+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e0880 con 0x56147561b970 2023-10-06T05:23:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.049+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'243 uv243 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.049+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.050+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'244 uv244 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.050+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.051+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v100'245 uv219 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.051+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.054+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v100'246 uv246 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.054+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.058 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:11.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.056+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v100'247 uv247 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.056+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e2470 con 0x56147561b970 2023-10-06T05:23:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.057+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'248 uv248 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.057+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e2470 con 0x56147561b970 2023-10-06T05:23:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.059+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'249 uv249 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.059+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.060+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.060+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.061+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v100'251 uv225 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.061+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.062+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v100'252 uv252 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.062+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.065 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.064+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v100'253 uv253 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.064+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e2470 con 0x56147561b970 2023-10-06T05:23:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.066+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'254 uv254 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.066+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e2470 con 0x56147561b970 2023-10-06T05:23:11.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.067+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'255 uv255 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.067+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e1460 con 0x56147561b970 2023-10-06T05:23:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.068+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'256 uv256 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x56147561b970 2023-10-06T05:23:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.068+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614756e1970 0x5614756e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.068+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e4290 con 0x5614756e1970 2023-10-06T05:23:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.068+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614756e1970 0x5614756e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.069+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614756e1970 0x5614756e3d50 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.070+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'174 uv154 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.070+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e4640 con 0x5614756e1970 2023-10-06T05:23:11.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.072+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v100'175 uv175 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.072+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e49f0 con 0x5614756e1970 2023-10-06T05:23:11.075 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:11.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.073+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v100'176 uv176 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.073+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e5290 con 0x5614756e1970 2023-10-06T05:23:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.074+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v100'177 uv177 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.074+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e5640 con 0x5614756e1970 2023-10-06T05:23:11.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.075+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v100'178 uv178 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.075+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e59f0 con 0x5614756e1970 2023-10-06T05:23:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.077+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v100'179 uv173 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.077+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e5da0 con 0x5614756e1970 2023-10-06T05:23:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.078+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v100'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.078+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.079+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v100'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.081 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.079+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.081+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v100'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.081+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.082+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v100'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.082+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.083+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v100'184 uv172 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.083+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.084+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v100'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.084+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.085+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v100'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.087 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.085+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9870 con 0x5614756e1970 2023-10-06T05:23:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.087+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v100'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.087+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9870 con 0x5614756e1970 2023-10-06T05:23:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.088+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v100'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.088+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.089+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v100'189 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.089+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.090+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v100'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.090+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.093 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.092+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v100'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.092+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e7160 con 0x5614756e1970 2023-10-06T05:23:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.093+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v100'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.093+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e7160 con 0x5614756e1970 2023-10-06T05:23:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.095+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v100'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.095+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e6150 con 0x5614756e1970 2023-10-06T05:23:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.096+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v100'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.096+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5614756e6660 0x5614756e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.096+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e8f80 con 0x5614756e6660 2023-10-06T05:23:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.096+0000 7f9624ad4700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5614756e6660 0x5614756e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.097+0000 7f9624ad4700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5614756e6660 0x5614756e8a40 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.098+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.098+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e8f80 con 0x5614756e6660 2023-10-06T05:23:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.099+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v100'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.099+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e8f80 con 0x5614756e6660 2023-10-06T05:23:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.101+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v100'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.102 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:11.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.101+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ea390 con 0x5614756e6660 2023-10-06T05:23:11.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.102+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.102+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ea390 con 0x5614756e6660 2023-10-06T05:23:11.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.103+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.103+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e6660 2023-10-06T05:23:11.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.104+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.104+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e6660 2023-10-06T05:23:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.105+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'35 uv34 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.105+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e6660 2023-10-06T05:23:11.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.106+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v100'36 uv36 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.106+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e6660 2023-10-06T05:23:11.109 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:11.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.108+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v100'37 uv37 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.108+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e6660 2023-10-06T05:23:11.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.109+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v100'38 uv38 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.109+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e1970 2023-10-06T05:23:11.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.110+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.110+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e1970 2023-10-06T05:23:11.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.112+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v100'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.112+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e1970 2023-10-06T05:23:11.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.113+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v100'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.115 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.113+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ea390 con 0x5614756e1970 2023-10-06T05:23:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.114+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.114+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ea390 con 0x5614756e1970 2023-10-06T05:23:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.115+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.115+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756e9380 con 0x5614756e1970 2023-10-06T05:23:11.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.117+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756e1970 2023-10-06T05:23:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.117+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5614756e9a00 0x5614756ebde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.117+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ec360 con 0x5614756e9a00 2023-10-06T05:23:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.117+0000 7f9625ad6700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5614756e9a00 0x5614756ebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.117+0000 7f9625ad6700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5614756e9a00 0x5614756ebde0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.118+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v100'83 uv75 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.118+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ec810 con 0x5614756e9a00 2023-10-06T05:23:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.119+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v100'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.119+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ecd00 con 0x5614756e9a00 2023-10-06T05:23:11.122 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:11.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.120+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v100'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.120+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ee600 con 0x5614756e9a00 2023-10-06T05:23:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.121+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v100'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.121+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ee9b0 con 0x5614756e9a00 2023-10-06T05:23:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.122+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v100'87 uv87 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.122+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ed5f0 con 0x5614756e9a00 2023-10-06T05:23:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.123+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v100'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.124+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756ed9f0 con 0x5614756e9a00 2023-10-06T05:23:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.125+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v100'89 uv82 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.125+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756eddf0 con 0x5614756e9a00 2023-10-06T05:23:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.126+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v100'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.126+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756eddf0 con 0x5614756e9a00 2023-10-06T05:23:11.128 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.127+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v100'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.127+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756edda0 con 0x5614756e9a00 2023-10-06T05:23:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.128+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v100'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.128+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756edda0 con 0x5614756e9a00 2023-10-06T05:23:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.129+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v100'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620052d00 con 0x5614756e9a00 2023-10-06T05:23:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.129+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5614756ee1a0 0x5614756f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.129+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f17c0 con 0x5614756ee1a0 2023-10-06T05:23:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.129+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5614756ee1a0 0x5614756f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.130+0000 7f96252d5700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5614756ee1a0 0x5614756f1280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.133+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v100'37 uv36 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756ee1a0 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.133+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f17c0 con 0x5614756ee1a0 2023-10-06T05:23:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.134+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v100'38 uv38 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756ee1a0 2023-10-06T05:23:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.134+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f17c0 con 0x5614756ee1a0 2023-10-06T05:23:11.136 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.135+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v100'39 uv39 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756ee1a0 2023-10-06T05:23:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.135+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f17c0 con 0x5614756ee1a0 2023-10-06T05:23:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.137+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v100'40 uv40 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756ee1a0 2023-10-06T05:23:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.137+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f1bc0 con 0x5614756ee1a0 2023-10-06T05:23:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.138+0000 7f96252d5700 1 -- 172.21.15.45:0/1434982380 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v100'41 uv41 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f96180054f0 con 0x5614756ee1a0 2023-10-06T05:23:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.138+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f1bc0 con 0x5614756e6660 2023-10-06T05:23:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.139+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.139+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f1bc0 con 0x5614756e6660 2023-10-06T05:23:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.140+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v100'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.140+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f1bc0 con 0x5614756e6660 2023-10-06T05:23:11.143 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.142+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v100'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.142+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f2bd0 con 0x5614756e6660 2023-10-06T05:23:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.143+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.143+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f2bd0 con 0x5614756e6660 2023-10-06T05:23:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.145+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.145+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f1bc0 con 0x5614756e6660 2023-10-06T05:23:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.146+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.146+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5614756f20d0 0x5614756f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.146+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4a70 con 0x5614756f20d0 2023-10-06T05:23:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.146+0000 7f9624ad4700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5614756f20d0 0x5614756f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.147+0000 7f9624ad4700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5614756f20d0 0x5614756f44b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.148+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v100'74 uv68 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.148+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4ab0 con 0x5614756f20d0 2023-10-06T05:23:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.149+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v100'75 uv75 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.149+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4eb0 con 0x5614756f20d0 2023-10-06T05:23:11.152 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.150+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v100'76 uv76 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.150+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4eb0 con 0x5614756f20d0 2023-10-06T05:23:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.151+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v100'77 uv77 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.151+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4eb0 con 0x5614756f20d0 2023-10-06T05:23:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.153+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v100'78 uv78 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.153+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4eb0 con 0x5614756f20d0 2023-10-06T05:23:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.154+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v100'79 uv73 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.154+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4eb0 con 0x5614756f20d0 2023-10-06T05:23:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.155+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v100'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.155+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4eb0 con 0x5614756f20d0 2023-10-06T05:23:11.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.156+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v100'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.157 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:11.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.156+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f4eb0 con 0x5614756f20d0 2023-10-06T05:23:11.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.157+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v100'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.157+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f7c20 con 0x5614756f20d0 2023-10-06T05:23:11.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.158+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v100'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756f20d0 2023-10-06T05:23:11.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.158+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f53a0 con 0x5614756e6660 2023-10-06T05:23:11.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.159+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v100'36 uv34 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.159+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f53a0 con 0x5614756e6660 2023-10-06T05:23:11.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.160+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v100'37 uv37 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.160+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f53a0 con 0x5614756e6660 2023-10-06T05:23:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.161+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v100'38 uv38 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.163 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.161+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f53a0 con 0x5614756e6660 2023-10-06T05:23:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.162+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v100'39 uv39 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.162+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5614756f58b0 0x5614756f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.162+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756f58b0 2023-10-06T05:23:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.163+0000 7f9625ad6700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5614756f58b0 0x5614756f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.163+0000 7f9625ad6700 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5614756f58b0 0x5614756f5cc0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.164+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756f58b0 2023-10-06T05:23:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.164+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756f58b0 2023-10-06T05:23:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.165+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v100'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756f58b0 2023-10-06T05:23:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.165+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756f58b0 2023-10-06T05:23:11.169 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.167+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v100'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756f58b0 2023-10-06T05:23:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.167+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f9380 con 0x5614756f58b0 2023-10-06T05:23:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.170+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756f58b0 2023-10-06T05:23:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.170+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f9380 con 0x5614756f58b0 2023-10-06T05:23:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.171+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756f58b0 2023-10-06T05:23:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.171+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756f58b0 2023-10-06T05:23:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.174+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756f58b0 2023-10-06T05:23:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.174+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e9a00 2023-10-06T05:23:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.175+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v100'37 uv36 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756e9a00 2023-10-06T05:23:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.175+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e9a00 2023-10-06T05:23:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.176+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v100'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756e9a00 2023-10-06T05:23:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.176+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e9a00 2023-10-06T05:23:11.180 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.178+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v100'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756e9a00 2023-10-06T05:23:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.178+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e9a00 2023-10-06T05:23:11.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.179+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v100'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756e9a00 2023-10-06T05:23:11.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.179+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e9a00 2023-10-06T05:23:11.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.181+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v100'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9620069040 con 0x5614756e9a00 2023-10-06T05:23:11.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.181+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e6660 2023-10-06T05:23:11.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.182+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v100'95 uv86 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.182+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e6660 2023-10-06T05:23:11.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.183+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v100'96 uv96 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.183+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e6660 2023-10-06T05:23:11.187 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:11.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.186+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v100'97 uv97 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.186+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e6660 2023-10-06T05:23:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.188+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v100'98 uv98 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.188+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8790 con 0x5614756e6660 2023-10-06T05:23:11.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.191+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v100'99 uv99 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.191+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8790 con 0x5614756e6660 2023-10-06T05:23:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.194+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v100'100 uv93 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.194+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e6660 2023-10-06T05:23:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.196+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v100'101 uv101 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.197+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e6660 2023-10-06T05:23:11.199 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.198+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v100'102 uv102 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.198+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f9380 con 0x5614756e6660 2023-10-06T05:23:11.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.200+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v100'103 uv103 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.200+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f9380 con 0x5614756e6660 2023-10-06T05:23:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.203+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v100'104 uv104 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.203+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8370 con 0x5614756e6660 2023-10-06T05:23:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.205+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v100'105 uv105 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.205+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) v8 -- 0x5614756f8740 con 0x5614756e6660 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f9624ad4700 1 -- 172.21.15.45:0/1434982380 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v100'106 uv106 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f961000b040 con 0x5614756e6660 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5614756e6660 msgr2=0x5614756e8a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5614756e6660 0x5614756e8a40 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56147561b970 msgr2=0x5614756ca8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56147561b970 0x5614756ca8f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5614756f20d0 msgr2=0x5614756f44b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5614756f20d0 0x5614756f44b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614756e1970 msgr2=0x5614756e3d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5614756e1970 0x5614756e3d50 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 reap_dead start 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5614756f58b0 msgr2=0x5614756f5cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5614756f58b0 0x5614756f5cc0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5614756ee1a0 msgr2=0x5614756f1280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5614756ee1a0 0x5614756f1280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5614756e9a00 msgr2=0x5614756ebde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5614756e9a00 0x5614756ebde0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96080658c0 msgr2=0x7f9608067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96080658c0 0x7f9608067d70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9618008cf0 tx=0x7f9618003050 comp rx=0 tx=0).stop 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 msgr2=0x5614756ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756ceed0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9620010970 tx=0x7f9620007040 comp rx=0 tx=0).stop 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f9625ad6700 1 -- 172.21.15.45:0/1434982380 reap_dead start 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 shutdown_connections 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 --2- 172.21.15.45:0/1434982380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614756acc60 0x5614756ceed0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 >> 172.21.15.45:0/1434982380 conn(0x5614755ed450 msgr2=0x5614755ef830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 shutdown_connections 2023-10-06T05:23:11.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.209+0000 7f962c0b7040 1 -- 172.21.15.45:0/1434982380 wait complete. 2023-10-06T05:23:11.216 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2023-10-06T05:23:11.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.264+0000 7fd1ec32e040 1 Processor -- start 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.264+0000 7fd1ec32e040 1 -- start start 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1ec32e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc0624210 0x558cc06247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1ec32e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc06e6890 0x558cc0627820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1ec32e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6c60 0x558cc0627d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1ec32e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0571980 con 0x558cc06e6c60 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1ec32e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0571620 con 0x558cc06e6890 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1ec32e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cc0568400 con 0x558cc0624210 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6c60 0x558cc0627d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6c60 0x558cc0627d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55814/0 (socket says 172.21.15.45:55814) 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 -- 172.21.15.45:0/4019496878 learned_addr learned my addr 172.21.15.45:0/4019496878 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e4d4b700 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc06e6890 0x558cc0627820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e554c700 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc0624210 0x558cc06247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 -- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc0624210 msgr2=0x558cc06247c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc0624210 0x558cc06247c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 -- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc06e6890 msgr2=0x558cc0627820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc06e6890 0x558cc0627820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 -- 172.21.15.45:0/4019496878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e000a180 con 0x558cc06e6c60 2023-10-06T05:23:11.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1e5d4d700 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6c60 0x558cc0627d60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x558cc062ac20 tx=0x7fd1e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5ba71aa5018be8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.265+0000 7fd1df7fe700 1 -- 172.21.15.45:0/4019496878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000edb0 con 0x558cc06e6c60 2023-10-06T05:23:11.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1df7fe700 1 -- 172.21.15.45:0/4019496878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0005870 con 0x558cc06e6c60 2023-10-06T05:23:11.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1df7fe700 1 -- 172.21.15.45:0/4019496878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e001c430 con 0x558cc06e6c60 2023-10-06T05:23:11.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 -- 172.21.15.45:0/4019496878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6c60 msgr2=0x558cc0627d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6c60 0x558cc0627d60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x558cc062ac20 tx=0x7fd1e0014000 comp rx=0 tx=0).stop 2023-10-06T05:23:11.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 -- 172.21.15.45:0/4019496878 shutdown_connections 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6c60 0x558cc0627d60 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc0624210 0x558cc06247c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/4019496878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc06e6890 0x558cc0627820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 -- 172.21.15.45:0/4019496878 >> 172.21.15.45:0/4019496878 conn(0x558cc0627450 msgr2=0x558cc0629770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 -- 172.21.15.45:0/4019496878 shutdown_connections 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.266+0000 7fd1ec32e040 1 -- 172.21.15.45:0/4019496878 wait complete. 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.267+0000 7fd1ec32e040 1 Processor -- start 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.267+0000 7fd1ec32e040 1 -- start start 2023-10-06T05:23:11.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1ec32e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc0624210 0x558cc0703800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1ec32e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6890 0x558cc06e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1ec32e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc06e6c60 0x558cc07055f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1ec32e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc06e60b0 con 0x558cc06e6890 2023-10-06T05:23:11.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1ec32e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc0702260 con 0x558cc0624210 2023-10-06T05:23:11.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1ec32e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cc0702500 con 0x558cc06e6c60 2023-10-06T05:23:11.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6890 0x558cc06e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6890 0x558cc06e55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55824/0 (socket says 172.21.15.45:55824) 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 -- 172.21.15.45:0/3164177597 learned_addr learned my addr 172.21.15.45:0/3164177597 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e5d4d700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc06e6c60 0x558cc07055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e554c700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc0624210 0x558cc0703800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 -- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc06e6c60 msgr2=0x558cc07055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558cc06e6c60 0x558cc07055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 -- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc0624210 msgr2=0x558cc0703800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558cc0624210 0x558cc0703800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 -- 172.21.15.45:0/3164177597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cc059a1c0 con 0x558cc06e6890 2023-10-06T05:23:11.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.268+0000 7fd1e4d4b700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6890 0x558cc06e55d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00087c0 tx=0x7fd1d000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.269+0000 7fd1ddffb700 1 -- 172.21.15.45:0/3164177597 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000f040 con 0x558cc06e6890 2023-10-06T05:23:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.269+0000 7fd1ddffb700 1 -- 172.21.15.45:0/3164177597 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d0015680 con 0x558cc06e6890 2023-10-06T05:23:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.269+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558cc05803d0 con 0x558cc06e6890 2023-10-06T05:23:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.269+0000 7fd1ddffb700 1 -- 172.21.15.45:0/3164177597 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0015a20 con 0x558cc06e6890 2023-10-06T05:23:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.269+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cc0573ae0 con 0x558cc06e6890 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.270+0000 7fd1ddffb700 1 -- 172.21.15.45:0/3164177597 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1d001e400 con 0x558cc06e6890 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.271+0000 7fd1ddffb700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1c80658c0 0x7fd1c8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.271+0000 7fd1ddffb700 1 -- 172.21.15.45:0/3164177597 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7066+0+0 (secure 0 0 0) 0x7fd1d00842b0 con 0x558cc06e6890 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.271+0000 7fd1e554c700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1c80658c0 0x7fd1c8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.271+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558cc06e5b10 0x558cc0704830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.271+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) v8 -- 0x558cc066d280 con 0x558cc06e5b10 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.271+0000 7fd1e5d4d700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558cc06e5b10 0x558cc0704830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.271+0000 7fd1e554c700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1c80658c0 0x7fd1c8067d70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000c00 tx=0x7fd1d8001810 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.272+0000 7fd1e5d4d700 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558cc06e5b10 0x558cc0704830 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1e5d4d700 1 -- 172.21.15.45:0/3164177597 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(1 mds_snaptable [delete] v100'42 uv42 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd1e000edb0 con 0x558cc06e5b10 2023-10-06T05:23:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558cc06e5b10 msgr2=0x558cc0704830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558cc06e5b10 0x558cc0704830 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1c80658c0 msgr2=0x7fd1c8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1c80658c0 0x7fd1c8067d70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000c00 tx=0x7fd1d8001810 comp rx=0 tx=0).stop 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6890 msgr2=0x558cc06e55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 --2- 172.21.15.45:0/3164177597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc06e6890 0x558cc06e55d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00087c0 tx=0x7fd1d000c040 comp rx=0 tx=0).stop 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1e5d4d700 1 -- 172.21.15.45:0/3164177597 reap_dead start 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 shutdown_connections 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 >> 172.21.15.45:0/3164177597 conn(0x558cc0627450 msgr2=0x558cc06248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 shutdown_connections 2023-10-06T05:23:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.273+0000 7fd1ec32e040 1 -- 172.21.15.45:0/3164177597 wait complete. 2023-10-06T05:23:11.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.421+0000 7f3afd1d5700 1 Processor -- start 2023-10-06T05:23:11.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.421+0000 7f3afd1d5700 1 -- start start 2023-10-06T05:23:11.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3afd1d5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3af8140ab0 0x7f3af80acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3afd1d5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af80c9940 con 0x7f3af80ad4e0 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3afd1d5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af806d230 con 0x7f3af81406e0 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3afd1d5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af806d510 con 0x7f3af8140ab0 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3af759e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3af80ad4e0 0x7f3af80c9dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38230/0 (socket says 172.21.15.45:38230) 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3af759e700 1 -- 172.21.15.45:0/3637951006 learned_addr learned my addr 172.21.15.45:0/3637951006 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3af659c700 1 --2- 172.21.15.45:0/3637951006 >> v2:172.21.15.144:3300/0 conn(0x7f3af8140ab0 0x7f3af80acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3af5d9b700 1 -- 172.21.15.45:0/3637951006 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831782374 0 0) 0x7f3af80c9940 con 0x7f3af80ad4e0 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.422+0000 7f3af5d9b700 1 -- 172.21.15.45:0/3637951006 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae4003570 con 0x7f3af80ad4e0 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af659c700 1 -- 172.21.15.45:0/3637951006 >> v1:172.21.15.144:6789/0 conn(0x7f3af81406e0 legacy=0x7f3af806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af5d9b700 1 -- 172.21.15.45:0/3637951006 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515046873 0 0) 0x7f3af806d230 con 0x7f3af81406e0 2023-10-06T05:23:11.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af659c700 1 -- 172.21.15.45:0/3637951006 >> v1:172.21.15.45:6789/0 conn(0x7f3af80ad4e0 legacy=0x7f3af80c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af659c700 1 -- 172.21.15.45:0/3637951006 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af806d7f0 con 0x7f3af8140ab0 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af659c700 1 --2- 172.21.15.45:0/3637951006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 0x7f3af80acfa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00098e0 tx=0x7f3ae00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4920c7d3e9bc90e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af5d9b700 1 -- 172.21.15.45:0/3637951006 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000b640 con 0x7f3af8140ab0 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af5d9b700 1 -- 172.21.15.45:0/3637951006 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae000b7a0 con 0x7f3af8140ab0 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.423+0000 7f3af5d9b700 1 -- 172.21.15.45:0/3637951006 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000bb40 con 0x7f3af8140ab0 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.424+0000 7f3afd1d5700 1 -- 172.21.15.45:0/3637951006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 msgr2=0x7f3af80acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.424+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/3637951006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 0x7f3af80acfa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00098e0 tx=0x7f3ae00091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.424+0000 7f3afd1d5700 1 -- 172.21.15.45:0/3637951006 shutdown_connections 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.424+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/3637951006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 0x7f3af80acfa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.424+0000 7f3afd1d5700 1 -- 172.21.15.45:0/3637951006 >> 172.21.15.45:0/3637951006 conn(0x7f3af80691c0 msgr2=0x7f3af80cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.424+0000 7f3afd1d5700 1 -- 172.21.15.45:0/3637951006 shutdown_connections 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.424+0000 7f3afd1d5700 1 -- 172.21.15.45:0/3637951006 wait complete. 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.425+0000 7f3afd1d5700 1 Processor -- start 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.425+0000 7f3afd1d5700 1 -- start start 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3afd1d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af80ad4e0 0x7f3af80b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3afd1d5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3af81406e0 0x7f3af8156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3afd1d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 0x7f3af815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3afd1d5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80c5d50 con 0x7f3af80ad4e0 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3afd1d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80c5110 con 0x7f3af8140ab0 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3afd1d5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80c53b0 con 0x7f3af81406e0 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af80ad4e0 0x7f3af80b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af80ad4e0 0x7f3af80b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55838/0 (socket says 172.21.15.45:55838) 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 -- 172.21.15.45:0/678461590 learned_addr learned my addr 172.21.15.45:0/678461590 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af659c700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3af81406e0 0x7f3af8156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af759e700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 0x7f3af815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 -- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3af81406e0 msgr2=0x7f3af8156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3af81406e0 0x7f3af8156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 -- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 msgr2=0x7f3af815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 0x7f3af815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 -- 172.21.15.45:0/678461590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009420 con 0x7f3af80ad4e0 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af659c700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3af81406e0 0x7f3af8156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:11.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.426+0000 7f3af6d9d700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af80ad4e0 0x7f3af80b1590 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c7e0 tx=0x7f3ae8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.427+0000 7f3aeeffd700 1 -- 172.21.15.45:0/678461590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8012700 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.427+0000 7f3aeeffd700 1 -- 172.21.15.45:0/678461590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae8012880 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.427+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af80c5690 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.427+0000 7f3aeeffd700 1 -- 172.21.15.45:0/678461590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8012c40 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.427+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af8156e90 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.428+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af806e2d0 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.428+0000 7f3aeeffd700 1 -- 172.21.15.45:0/678461590 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ae801b400 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.429+0000 7f3aeeffd700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ad8066f30 0x7f3ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.429+0000 7f3aeeffd700 1 -- 172.21.15.45:0/678461590 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f3ae8084680 con 0x7f3af80ad4e0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.429+0000 7f3af659c700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ad8066f30 0x7f3ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.429+0000 7f3af659c700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ad8066f30 0x7f3ad80693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00093f0 tx=0x7f3ae0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.433+0000 7f3aeeffd700 1 -- 172.21.15.45:0/678461590 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ae80513a0 con 0x7f3af80ad4e0 2023-10-06T05:23:11.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:11.688+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f3af80cda70 con 0x7f3af80ad4e0 2023-10-06T05:23:12.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.042+0000 7f3aeeffd700 1 -- 172.21.15.45:0/678461590 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v139) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3ae80549c0 con 0x7f3af80ad4e0 2023-10-06T05:23:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ad8066f30 msgr2=0x7f3ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ad8066f30 0x7f3ad80693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00093f0 tx=0x7f3ae0017000 comp rx=0 tx=0).stop 2023-10-06T05:23:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af80ad4e0 msgr2=0x7f3af80b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af80ad4e0 0x7f3af80b1590 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c7e0 tx=0x7f3ae8007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 shutdown_connections 2023-10-06T05:23:12.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3ad8066f30 0x7f3ad80693e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3af8140ab0 0x7f3af815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af80ad4e0 0x7f3af80b1590 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 --2- 172.21.15.45:0/678461590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3af81406e0 0x7f3af8156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 >> 172.21.15.45:0/678461590 conn(0x7f3af80691c0 msgr2=0x7f3af813f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 shutdown_connections 2023-10-06T05:23:12.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.045+0000 7f3afd1d5700 1 -- 172.21.15.45:0/678461590 wait complete. 2023-10-06T05:23:12.055 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:12.056 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:12.048+0000 7ff3f91c2700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-10-06T05:23:12.056 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:12.048+0000 7ff3f91c2700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-10-06T05:23:12.058 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:12.049+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:12.059 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:12.049+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:23:12.059 INFO:tasks.ceph.mds.c.smithi045.stderr: -27> 2023-10-06T05:23:12.048+0000 7ff3f91c2700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-10-06T05:23:12.059 INFO:tasks.ceph.mds.c.smithi045.stderr: -25> 2023-10-06T05:23:12.048+0000 7ff3f91c2700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-10-06T05:23:12.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:12.080 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:12.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.200+0000 7f95d72fb700 1 Processor -- start 2023-10-06T05:23:12.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.201+0000 7f95d72fb700 1 -- start start 2023-10-06T05:23:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.201+0000 7f95d72fb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95d00c8ff0 0x7f95d00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.201+0000 7f95d72fb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95d00caad0 0x7f95d00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.201+0000 7f95d72fb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d0156af0 0x7f95d0156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.201+0000 7f95d72fb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d015b000 con 0x7f95d00caad0 2023-10-06T05:23:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.201+0000 7f95d72fb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d015b2e0 con 0x7f95d00c8ff0 2023-10-06T05:23:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.201+0000 7f95d72fb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d015b5c0 con 0x7f95d0156af0 2023-10-06T05:23:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95d00caad0 0x7f95d00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95d00caad0 0x7f95d00caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55862/0 (socket says 172.21.15.45:55862) 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 -- 172.21.15.45:0/3172249109 learned_addr learned my addr 172.21.15.45:0/3172249109 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d5097700 1 --2- 172.21.15.45:0/3172249109 >> v2:172.21.15.144:3300/0 conn(0x7f95d00c8ff0 0x7f95d00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d5898700 1 --2- 172.21.15.45:0/3172249109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d0156af0 0x7f95d0156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 -- 172.21.15.45:0/3172249109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d0156af0 msgr2=0x7f95d0156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 --2- 172.21.15.45:0/3172249109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d0156af0 0x7f95d0156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 -- 172.21.15.45:0/3172249109 >> v2:172.21.15.144:3300/0 conn(0x7f95d00c8ff0 msgr2=0x7f95d00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 --2- 172.21.15.45:0/3172249109 >> v2:172.21.15.144:3300/0 conn(0x7f95d00c8ff0 0x7f95d00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 -- 172.21.15.45:0/3172249109 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d015b8a0 con 0x7f95d00caad0 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95d4896700 1 --2- 172.21.15.45:0/3172249109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00caad0 0x7f95d00caf40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f95d00cefc0 tx=0x7f95c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f084bf9f747a838 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95c7fff700 1 -- 172.21.15.45:0/3172249109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000b640 con 0x7f95d00caad0 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95c7fff700 1 -- 172.21.15.45:0/3172249109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c000b7a0 con 0x7f95d00caad0 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.202+0000 7f95c7fff700 1 -- 172.21.15.45:0/3172249109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000bb60 con 0x7f95d00caad0 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 -- 172.21.15.45:0/3172249109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00caad0 msgr2=0x7f95d00caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3172249109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00caad0 0x7f95d00caf40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f95d00cefc0 tx=0x7f95c0009250 comp rx=0 tx=0).stop 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 -- 172.21.15.45:0/3172249109 shutdown_connections 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3172249109 >> v2:172.21.15.144:3300/0 conn(0x7f95d00c8ff0 0x7f95d00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3172249109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d0156af0 0x7f95d0156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3172249109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00caad0 0x7f95d00caf40 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 -- 172.21.15.45:0/3172249109 >> 172.21.15.45:0/3172249109 conn(0x7f95d00cdf50 msgr2=0x7f95d00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 -- 172.21.15.45:0/3172249109 shutdown_connections 2023-10-06T05:23:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.203+0000 7f95d72fb700 1 -- 172.21.15.45:0/3172249109 wait complete. 2023-10-06T05:23:12.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.204+0000 7f95d72fb700 1 Processor -- start 2023-10-06T05:23:12.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.204+0000 7f95d72fb700 1 -- start start 2023-10-06T05:23:12.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.205+0000 7f95d72fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00c8ff0 0x7f95d014a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.205+0000 7f95d72fb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00caad0 0x7f95d00c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.205+0000 7f95d72fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d0156af0 0x7f95d00c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.205+0000 7f95d72fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c5e30 con 0x7f95d0156af0 2023-10-06T05:23:12.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.205+0000 7f95d72fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c60d0 con 0x7f95d00c8ff0 2023-10-06T05:23:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.205+0000 7f95d72fb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c63b0 con 0x7f95d00caad0 2023-10-06T05:23:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d0156af0 0x7f95d00c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d0156af0 0x7f95d00c3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55870/0 (socket says 172.21.15.45:55870) 2023-10-06T05:23:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 -- 172.21.15.45:0/3912792760 learned_addr learned my addr 172.21.15.45:0/3912792760 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d4896700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00caad0 0x7f95d00c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5097700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00c8ff0 0x7f95d014a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 -- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00caad0 msgr2=0x7f95d00c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00caad0 0x7f95d00c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 -- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00c8ff0 msgr2=0x7f95d014a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00c8ff0 0x7f95d014a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 -- 172.21.15.45:0/3912792760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00094f0 con 0x7f95d0156af0 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d5898700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d0156af0 0x7f95d00c3230 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009df0 tx=0x7f95c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95c57fa700 1 -- 172.21.15.45:0/3912792760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800caa0 con 0x7f95d0156af0 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95c57fa700 1 -- 172.21.15.45:0/3912792760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800f040 con 0x7f95d0156af0 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d00c6690 con 0x7f95d0156af0 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95c57fa700 1 -- 172.21.15.45:0/3912792760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800ce70 con 0x7f95d0156af0 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.206+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d014af10 con 0x7f95d0156af0 2023-10-06T05:23:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.207+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d006f410 con 0x7f95d0156af0 2023-10-06T05:23:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.208+0000 7f95c57fa700 1 -- 172.21.15.45:0/3912792760 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95c8014690 con 0x7f95d0156af0 2023-10-06T05:23:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.208+0000 7f95c57fa700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b4066f30 0x7f95b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.208+0000 7f95c57fa700 1 -- 172.21.15.45:0/3912792760 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f95c8085360 con 0x7f95d0156af0 2023-10-06T05:23:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.208+0000 7f95d5097700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b4066f30 0x7f95b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.209+0000 7f95d5097700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b4066f30 0x7f95b40693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009db0 tx=0x7f95cc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.213+0000 7f95c57fa700 1 -- 172.21.15.45:0/3912792760 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95c8055a40 con 0x7f95d0156af0 2023-10-06T05:23:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.461+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95d00b0840 con 0x7f95d0156af0 2023-10-06T05:23:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.461+0000 7f95c57fa700 1 -- 172.21.15.45:0/3912792760 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f95c80555d0 con 0x7f95d0156af0 2023-10-06T05:23:12.463 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:12.463 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:12.050456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b4066f30 msgr2=0x7f95b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b4066f30 0x7f95b40693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009db0 tx=0x7f95cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d0156af0 msgr2=0x7f95d00c3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d0156af0 0x7f95d00c3230 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009df0 tx=0x7f95c800b040 comp rx=0 tx=0).stop 2023-10-06T05:23:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 shutdown_connections 2023-10-06T05:23:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b4066f30 0x7f95b40693e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d0156af0 0x7f95d00c3230 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00c8ff0 0x7f95d014a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 --2- 172.21.15.45:0/3912792760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00caad0 0x7f95d00c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 >> 172.21.15.45:0/3912792760 conn(0x7f95d00cdf50 msgr2=0x7f95d00adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 shutdown_connections 2023-10-06T05:23:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.464+0000 7f95d72fb700 1 -- 172.21.15.45:0/3912792760 wait complete. 2023-10-06T05:23:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 141 2023-10-06T05:23:12.479 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:12.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.653+0000 7fbe44784700 1 Processor -- start 2023-10-06T05:23:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.653+0000 7fbe44784700 1 -- start start 2023-10-06T05:23:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe44784700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe3c158f60 0x7fbe3c159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe44784700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0cb530 con 0x7fbe3c0cc390 2023-10-06T05:23:12.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe44784700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c069bd0 con 0x7fbe3c06a060 2023-10-06T05:23:12.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe44784700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c06a500 con 0x7fbe3c158f60 2023-10-06T05:23:12.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe42d21700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbe3c0cc390 0x7fbe3c0c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38248/0 (socket says 172.21.15.45:38248) 2023-10-06T05:23:12.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe42d21700 1 -- 172.21.15.45:0/911204425 learned_addr learned my addr 172.21.15.45:0/911204425 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe42520700 1 --2- 172.21.15.45:0/911204425 >> v2:172.21.15.144:3300/0 conn(0x7fbe3c158f60 0x7fbe3c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469719185 0 0) 0x7fbe34001490 con 0x7fbe3c0cc390 2023-10-06T05:23:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe28003570 con 0x7fbe3c0cc390 2023-10-06T05:23:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226283118 0 0) 0x7fbe28003570 con 0x7fbe3c0cc390 2023-10-06T05:23:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 >> v2:172.21.15.144:3300/0 conn(0x7fbe3c158f60 msgr2=0x7fbe3c159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe4151e700 1 --2- 172.21.15.45:0/911204425 >> v2:172.21.15.144:3300/0 conn(0x7fbe3c158f60 0x7fbe3c159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.654+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 >> v1:172.21.15.144:6789/0 conn(0x7fbe3c06a060 legacy=0x7fbe3c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c068fc0 con 0x7fbe3c0cc390 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fbe34001980 con 0x7fbe3c0cc390 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 >> v1:172.21.15.45:6789/0 conn(0x7fbe3c0cc390 legacy=0x7fbe3c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe28003a60 0x7fbe28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe28004810 0x7fbe28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe280051c0 0x7fbe2800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c06a500 con 0x7fbe280051c0 2023-10-06T05:23:12.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2800c160 con 0x7fbe28004810 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2800c320 con 0x7fbe28003a60 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe280051c0 0x7fbe2800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42520700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe28003a60 0x7fbe28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe41d1f700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe28004810 0x7fbe28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 -- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe28003a60 msgr2=0x7fbe28003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe28003a60 0x7fbe28003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 -- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe28004810 msgr2=0x7fbe28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe28004810 0x7fbe28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 -- 172.21.15.45:0/911204425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28003570 con 0x7fbe280051c0 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 -- 172.21.15.45:0/911204425 reap_dead start 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.655+0000 7fbe42d21700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe280051c0 0x7fbe2800ba60 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400aaa0 tx=0x7fbe3400a170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611a55810b5f6623 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400c5d0 con 0x7fbe280051c0 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3400c750 con 0x7fbe280051c0 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe4151e700 1 -- 172.21.15.45:0/911204425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400cb10 con 0x7fbe280051c0 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe44784700 1 -- 172.21.15.45:0/911204425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe280051c0 msgr2=0x7fbe2800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe44784700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe280051c0 0x7fbe2800ba60 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400aaa0 tx=0x7fbe3400a170 comp rx=0 tx=0).stop 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe44784700 1 -- 172.21.15.45:0/911204425 shutdown_connections 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe44784700 1 --2- 172.21.15.45:0/911204425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe280051c0 0x7fbe2800ba60 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe44784700 1 -- 172.21.15.45:0/911204425 >> 172.21.15.45:0/911204425 conn(0x7fbe3c0cdf50 msgr2=0x7fbe3c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe44784700 1 -- 172.21.15.45:0/911204425 shutdown_connections 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.656+0000 7fbe44784700 1 -- 172.21.15.45:0/911204425 wait complete. 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.657+0000 7fbe44784700 1 Processor -- start 2023-10-06T05:23:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe44784700 1 -- start start 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe44784700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe280051c0 0x7fbe3c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe44784700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3c0a96d0 0x7fbe3c14d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe44784700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe3c0c7060 0x7fbe3c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe44784700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0c7a20 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe44784700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0b0940 con 0x7fbe3c0c7060 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe44784700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0b04b0 con 0x7fbe280051c0 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe41d1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3c0a96d0 0x7fbe3c14d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe41d1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3c0a96d0 0x7fbe3c14d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55886/0 (socket says 172.21.15.45:55886) 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe41d1f700 1 -- 172.21.15.45:0/1297151125 learned_addr learned my addr 172.21.15.45:0/1297151125 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.658+0000 7fbe42520700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe280051c0 0x7fbe3c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe42d21700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe3c0c7060 0x7fbe3c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe41d1f700 1 -- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe280051c0 msgr2=0x7fbe3c0af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe41d1f700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe280051c0 0x7fbe3c0af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe41d1f700 1 -- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe3c0c7060 msgr2=0x7fbe3c0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe41d1f700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe3c0c7060 0x7fbe3c0c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe41d1f700 1 -- 172.21.15.45:0/1297151125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0af8a0 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe41d1f700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3c0a96d0 0x7fbe3c14d940 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c002950 tx=0x7fbe2c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe327fc700 1 -- 172.21.15.45:0/1297151125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c003390 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe327fc700 1 -- 172.21.15.45:0/1297151125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c006820 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe327fc700 1 -- 172.21.15.45:0/1297151125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c006bc0 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c0afb80 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.659+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c0b0140 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.660+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c06f410 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.661+0000 7fbe327fc700 1 -- 172.21.15.45:0/1297151125 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe2c007030 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.661+0000 7fbe327fc700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe20066f30 0x7fbe200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.661+0000 7fbe327fc700 1 -- 172.21.15.45:0/1297151125 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fbe2c084ae0 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.661+0000 7fbe42520700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe20066f30 0x7fbe200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.662+0000 7fbe42520700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe20066f30 0x7fbe200693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe38006f00 tx=0x7fbe380091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.666+0000 7fbe327fc700 1 -- 172.21.15.45:0/1297151125 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe2c0517b0 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.913+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe3c0b05f0 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.913+0000 7fbe327fc700 1 -- 172.21.15.45:0/1297151125 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fbe2c054dd0 con 0x7fbe3c0a96d0 2023-10-06T05:23:12.915 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:12.915 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:12.050456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe20066f30 msgr2=0x7fbe200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe20066f30 0x7fbe200693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe38006f00 tx=0x7fbe380091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3c0a96d0 msgr2=0x7fbe3c14d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3c0a96d0 0x7fbe3c14d940 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c002950 tx=0x7fbe2c007d30 comp rx=0 tx=0).stop 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 shutdown_connections 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe20066f30 0x7fbe200693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe3c0c7060 0x7fbe3c0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe280051c0 0x7fbe3c0af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 --2- 172.21.15.45:0/1297151125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3c0a96d0 0x7fbe3c14d940 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 >> 172.21.15.45:0/1297151125 conn(0x7fbe3c0cdf50 msgr2=0x7fbe3c0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 shutdown_connections 2023-10-06T05:23:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:12.915+0000 7fbe44784700 1 -- 172.21.15.45:0/1297151125 wait complete. 2023-10-06T05:23:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 141 2023-10-06T05:23:12.931 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:12.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.063+0000 7f713d3f5700 1 Processor -- start 2023-10-06T05:23:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.063+0000 7f713d3f5700 1 -- start start 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f713d3f5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71381595d0 0x7f713806b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f713d3f5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f713806b5c0 0x7f71380ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f713d3f5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380c7d00 con 0x7f7138159200 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f713d3f5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71380c7780 con 0x7f713806b5c0 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f713d3f5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713806bb40 con 0x7f71381595d0 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f71377fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f713806b5c0 0x7f71380ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f71377fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f713806b5c0 0x7f71380ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55898/0 (socket says 172.21.15.45:55898) 2023-10-06T05:23:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f71367fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71381595d0 0x7f713806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.064+0000 7f71377fe700 1 -- 172.21.15.45:0/3421385408 learned_addr learned my addr 172.21.15.45:0/3421385408 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f71377fe700 1 -- 172.21.15.45:0/3421385408 >> v2:172.21.15.144:3300/0 conn(0x7f71381595d0 msgr2=0x7f713806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f71377fe700 1 --2- 172.21.15.45:0/3421385408 >> v2:172.21.15.144:3300/0 conn(0x7f71381595d0 0x7f713806b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f71377fe700 1 -- 172.21.15.45:0/3421385408 >> v1:172.21.15.144:6789/0 conn(0x7f7138159200 legacy=0x7f71380c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f71377fe700 1 -- 172.21.15.45:0/3421385408 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713806be20 con 0x7f713806b5c0 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f71377fe700 1 --2- 172.21.15.45:0/3421385408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f713806b5c0 0x7f71380ccdc0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f713806e790 tx=0x7f7130009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412776b60728bc0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f7135ffb700 1 -- 172.21.15.45:0/3421385408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713000e070 con 0x7f713806b5c0 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f7135ffb700 1 -- 172.21.15.45:0/3421385408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713000ba20 con 0x7f713806b5c0 2023-10-06T05:23:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.065+0000 7f7135ffb700 1 -- 172.21.15.45:0/3421385408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71300092e0 con 0x7f713806b5c0 2023-10-06T05:23:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 -- 172.21.15.45:0/3421385408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f713806b5c0 msgr2=0x7f71380ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 --2- 172.21.15.45:0/3421385408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f713806b5c0 0x7f71380ccdc0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f713806e790 tx=0x7f7130009840 comp rx=0 tx=0).stop 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 -- 172.21.15.45:0/3421385408 shutdown_connections 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 --2- 172.21.15.45:0/3421385408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f713806b5c0 0x7f71380ccdc0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 --2- 172.21.15.45:0/3421385408 >> v2:172.21.15.144:3300/0 conn(0x7f71381595d0 0x7f713806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 -- 172.21.15.45:0/3421385408 >> 172.21.15.45:0/3421385408 conn(0x7f713806f090 msgr2=0x7f713806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 -- 172.21.15.45:0/3421385408 shutdown_connections 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.066+0000 7f713d3f5700 1 -- 172.21.15.45:0/3421385408 wait complete. 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.068+0000 7f713d3f5700 1 Processor -- start 2023-10-06T05:23:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.068+0000 7f713d3f5700 1 -- start start 2023-10-06T05:23:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f713d3f5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713806b5c0 0x7f713814e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f713d3f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7138159200 0x7f71380b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f713d3f5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71381595d0 0x7f71380c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f713d3f5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380c5860 con 0x7f7138159200 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f713d3f5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380b10b0 con 0x7f713806b5c0 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f713d3f5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713814e660 con 0x7f71381595d0 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7138159200 0x7f71380b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7138159200 0x7f71380b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55910/0 (socket says 172.21.15.45:55910) 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 -- 172.21.15.45:0/2119031008 learned_addr learned my addr 172.21.15.45:0/2119031008 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f7136ffd700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713806b5c0 0x7f713814e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71377fe700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71381595d0 0x7f71380c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 -- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71381595d0 msgr2=0x7f71380c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71381595d0 0x7f71380c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 -- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713806b5c0 msgr2=0x7f713814e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713806b5c0 0x7f713814e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 -- 172.21.15.45:0/2119031008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7130009ae0 con 0x7f7138159200 2023-10-06T05:23:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.069+0000 7f71367fc700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7138159200 0x7f71380b07d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f712000e930 tx=0x7f71200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.070+0000 7f712effd700 1 -- 172.21.15.45:0/2119031008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000eed0 con 0x7f7138159200 2023-10-06T05:23:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.070+0000 7f712effd700 1 -- 172.21.15.45:0/2119031008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712000c850 con 0x7f7138159200 2023-10-06T05:23:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.070+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71380c4f30 con 0x7f7138159200 2023-10-06T05:23:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.070+0000 7f712effd700 1 -- 172.21.15.45:0/2119031008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000cbf0 con 0x7f7138159200 2023-10-06T05:23:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.070+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71380c54f0 con 0x7f7138159200 2023-10-06T05:23:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.071+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71380c59d0 con 0x7f7138159200 2023-10-06T05:23:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.071+0000 7f712effd700 1 -- 172.21.15.45:0/2119031008 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7120013470 con 0x7f7138159200 2023-10-06T05:23:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.071+0000 7f712effd700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.072+0000 7f712effd700 1 -- 172.21.15.45:0/2119031008 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f7120084d00 con 0x7f7138159200 2023-10-06T05:23:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.072+0000 7f7136ffd700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.072+0000 7f7136ffd700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7128003860 tx=0x7f7128009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.076+0000 7f712effd700 1 -- 172.21.15.45:0/2119031008 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71200519d0 con 0x7f7138159200 2023-10-06T05:23:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.326+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71380c7990 con 0x7f7138159200 2023-10-06T05:23:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.326+0000 7f712effd700 1 -- 172.21.15.45:0/2119031008 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+4599 (secure 0 0 0) 0x7f7120054ff0 con 0x7f7138159200 2023-10-06T05:23:13.328 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:13.328 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":142,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8367,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2532567321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2532567321},{"type":"v1","addr":"172.21.15.45:6835","nonce":2532567321}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:12.050456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:13.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 msgr2=0x7f71180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7128003860 tx=0x7f7128009040 comp rx=0 tx=0).stop 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7138159200 msgr2=0x7f71380b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7138159200 0x7f71380b07d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f712000e930 tx=0x7f71200091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 shutdown_connections 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71381595d0 0x7f71380c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713806b5c0 0x7f713814e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 --2- 172.21.15.45:0/2119031008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7138159200 0x7f71380b07d0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 >> 172.21.15.45:0/2119031008 conn(0x7f713806f090 msgr2=0x7f713815a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 shutdown_connections 2023-10-06T05:23:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.328+0000 7f713d3f5700 1 -- 172.21.15.45:0/2119031008 wait complete. 2023-10-06T05:23:13.332 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 142 2023-10-06T05:23:13.343 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2023-10-06T05:23:13.344 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2023-10-06T05:23:13.344 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:13.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:13.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:13.344 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:13.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:23:13.364 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:23:13.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.540+0000 7f106d53a700 1 Processor -- start 2023-10-06T05:23:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.540+0000 7f106d53a700 1 -- start start 2023-10-06T05:23:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.541+0000 7f106d53a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f106815d040 0x7f1068159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.541+0000 7f106d53a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10680ca6d0 con 0x7f1068159170 2023-10-06T05:23:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.541+0000 7f106d53a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106815d550 con 0x7f10680692d0 2023-10-06T05:23:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.541+0000 7f106d53a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106815d810 con 0x7f106815d040 2023-10-06T05:23:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.541+0000 7f1066ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1068159170 0x7f1068068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38258/0 (socket says 172.21.15.45:38258) 2023-10-06T05:23:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.541+0000 7f1066ffd700 1 -- 172.21.15.45:0/3528097335 learned_addr learned my addr 172.21.15.45:0/3528097335 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.541+0000 7f10677fe700 1 --2- 172.21.15.45:0/3528097335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f106815d040 0x7f1068159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578007701 0 0) 0x7f10680ca6d0 con 0x7f1068159170 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1050003570 con 0x7f1068159170 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887133182 0 0) 0x7f106815d550 con 0x7f10680692d0 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10680ca6d0 con 0x7f10680692d0 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f10677fe700 1 -- 172.21.15.45:0/3528097335 >> v1:172.21.15.144:6789/0 conn(0x7f10680692d0 legacy=0x7f10680aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 923259674 0 0) 0x7f1050003570 con 0x7f1068159170 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f10677fe700 1 -- 172.21.15.45:0/3528097335 >> v1:172.21.15.45:6789/0 conn(0x7f1068159170 legacy=0x7f1068068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f10677fe700 1 -- 172.21.15.45:0/3528097335 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106815daf0 con 0x7f106815d040 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f10677fe700 1 --2- 172.21.15.45:0/3528097335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f106815d040 0x7f1068159540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1060009e90 tx=0x7f1060009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8414562a96281cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000e070 con 0x7f106815d040 2023-10-06T05:23:13.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106000b870 con 0x7f106815d040 2023-10-06T05:23:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.542+0000 7f1065ffb700 1 -- 172.21.15.45:0/3528097335 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058001470 con 0x7f106815d040 2023-10-06T05:23:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.543+0000 7f106d53a700 1 -- 172.21.15.45:0/3528097335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f106815d040 msgr2=0x7f1068159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.543+0000 7f106d53a700 1 --2- 172.21.15.45:0/3528097335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f106815d040 0x7f1068159540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1060009e90 tx=0x7f1060009780 comp rx=0 tx=0).stop 2023-10-06T05:23:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.543+0000 7f106d53a700 1 -- 172.21.15.45:0/3528097335 shutdown_connections 2023-10-06T05:23:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.543+0000 7f106d53a700 1 --2- 172.21.15.45:0/3528097335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f106815d040 0x7f1068159540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.543+0000 7f106d53a700 1 -- 172.21.15.45:0/3528097335 >> 172.21.15.45:0/3528097335 conn(0x7f106806f7c0 msgr2=0x7f10680d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.543+0000 7f106d53a700 1 -- 172.21.15.45:0/3528097335 shutdown_connections 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.543+0000 7f106d53a700 1 -- 172.21.15.45:0/3528097335 wait complete. 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.544+0000 7f106d53a700 1 Processor -- start 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.544+0000 7f106d53a700 1 -- start start 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f106d53a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10680692d0 0x7f10680c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f106d53a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1068159170 0x7f10680c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f106d53a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106815d040 0x7f106815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f106d53a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680ca460 con 0x7f1068159170 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f106d53a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680c9850 con 0x7f106815d040 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f106d53a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10680c9b30 con 0x7f10680692d0 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f10667fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1068159170 0x7f10680c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f10667fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1068159170 0x7f10680c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55920/0 (socket says 172.21.15.45:55920) 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f10667fc700 1 -- 172.21.15.45:0/3057269510 learned_addr learned my addr 172.21.15.45:0/3057269510 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f1066ffd700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10680692d0 0x7f10680c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.545+0000 7f10677fe700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106815d040 0x7f106815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f10667fc700 1 -- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10680692d0 msgr2=0x7f10680c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f10667fc700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10680692d0 0x7f10680c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f10667fc700 1 -- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106815d040 msgr2=0x7f106815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f10667fc700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106815d040 0x7f106815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f10667fc700 1 -- 172.21.15.45:0/3057269510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1060009a20 con 0x7f1068159170 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f10667fc700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1068159170 0x7f10680c7c10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c7c0 tx=0x7f105c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f104effd700 1 -- 172.21.15.45:0/3057269510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c010510 con 0x7f1068159170 2023-10-06T05:23:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f104effd700 1 -- 172.21.15.45:0/3057269510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c0106b0 con 0x7f1068159170 2023-10-06T05:23:13.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10680c9e10 con 0x7f1068159170 2023-10-06T05:23:13.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f104effd700 1 -- 172.21.15.45:0/3057269510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c010a70 con 0x7f1068159170 2023-10-06T05:23:13.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.546+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106815c410 con 0x7f1068159170 2023-10-06T05:23:13.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.547+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1068063b50 con 0x7f1068159170 2023-10-06T05:23:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.547+0000 7f104effd700 1 -- 172.21.15.45:0/3057269510 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f105c012440 con 0x7f1068159170 2023-10-06T05:23:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.548+0000 7f104effd700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1054066f30 0x7f10540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.548+0000 7f104effd700 1 -- 172.21.15.45:0/3057269510 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f105c084130 con 0x7f1068159170 2023-10-06T05:23:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.548+0000 7f1066ffd700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1054066f30 0x7f10540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.549+0000 7f1066ffd700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1054066f30 0x7f10540693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1058001ed0 tx=0x7f10580019d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.552+0000 7f104effd700 1 -- 172.21.15.45:0/3057269510 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f105c04b5a0 con 0x7f1068159170 2023-10-06T05:23:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:13.801+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f106815ddd0 con 0x7f1068159170 2023-10-06T05:23:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.065+0000 7f104effd700 1 -- 172.21.15.45:0/3057269510 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v143) v1 ==== 161+0+0 (secure 0 0 0) 0x7f105c017090 con 0x7f1068159170 2023-10-06T05:23:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1054066f30 msgr2=0x7f10540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1054066f30 0x7f10540693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1058001ed0 tx=0x7f10580019d0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1068159170 msgr2=0x7f10680c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1068159170 0x7f10680c7c10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c7c0 tx=0x7f105c007de0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 shutdown_connections 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1054066f30 0x7f10540693e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106815d040 0x7f106815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10680692d0 0x7f10680c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 --2- 172.21.15.45:0/3057269510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1068159170 0x7f10680c7c10 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 >> 172.21.15.45:0/3057269510 conn(0x7f106806f7c0 msgr2=0x7f10680b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 shutdown_connections 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.068+0000 7f106d53a700 1 -- 172.21.15.45:0/3057269510 wait complete. 2023-10-06T05:23:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:14.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.220+0000 7f98911cb700 1 Processor -- start 2023-10-06T05:23:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.220+0000 7f98911cb700 1 -- start start 2023-10-06T05:23:14.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f98911cb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f988c1595d0 0x7f988c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f98911cb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f988c06b5c0 0x7f988c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f98911cb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c0c7d00 con 0x7f988c159200 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f98911cb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c0c7780 con 0x7f988c1595d0 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f98911cb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c06bb40 con 0x7f988c06b5c0 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f988c1595d0 0x7f988c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f988c159200 0x7f988c0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38272/0 (socket says 172.21.15.45:38272) 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f988c1595d0 0x7f988c06b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55926/0 (socket says 172.21.15.45:55926) 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988ad9d700 1 -- 172.21.15.45:0/1596650882 learned_addr learned my addr 172.21.15.45:0/1596650882 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988b59e700 1 --2- 172.21.15.45:0/1596650882 >> v2:172.21.15.144:3300/0 conn(0x7f988c06b5c0 0x7f988c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988a59c700 1 -- 172.21.15.45:0/1596650882 >> v2:172.21.15.144:3300/0 conn(0x7f988c06b5c0 msgr2=0x7f988c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988a59c700 1 --2- 172.21.15.45:0/1596650882 >> v2:172.21.15.144:3300/0 conn(0x7f988c06b5c0 0x7f988c0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988a59c700 1 -- 172.21.15.45:0/1596650882 >> v1:172.21.15.45:6789/0 conn(0x7f988c159200 legacy=0x7f988c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f988a59c700 1 -- 172.21.15.45:0/1596650882 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c06be20 con 0x7f988c1595d0 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.221+0000 7f9889d9b700 1 -- 172.21.15.45:0/1596650882 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595821271 0 0) 0x7f988c0c7d00 con 0x7f988c159200 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f988a59c700 1 --2- 172.21.15.45:0/1596650882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c1595d0 0x7f988c06b080 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f988c068680 tx=0x7f98740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9d8230a9292262 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f9889d9b700 1 -- 172.21.15.45:0/1596650882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874011630 con 0x7f988c1595d0 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f9889d9b700 1 -- 172.21.15.45:0/1596650882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874011790 con 0x7f988c1595d0 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f9889d9b700 1 -- 172.21.15.45:0/1596650882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874011b30 con 0x7f988c1595d0 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 -- 172.21.15.45:0/1596650882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c1595d0 msgr2=0x7f988c06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 --2- 172.21.15.45:0/1596650882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c1595d0 0x7f988c06b080 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f988c068680 tx=0x7f98740091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 -- 172.21.15.45:0/1596650882 shutdown_connections 2023-10-06T05:23:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 --2- 172.21.15.45:0/1596650882 >> v2:172.21.15.144:3300/0 conn(0x7f988c06b5c0 0x7f988c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 --2- 172.21.15.45:0/1596650882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c1595d0 0x7f988c06b080 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 -- 172.21.15.45:0/1596650882 >> 172.21.15.45:0/1596650882 conn(0x7f988c06f090 msgr2=0x7f988c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 -- 172.21.15.45:0/1596650882 shutdown_connections 2023-10-06T05:23:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.222+0000 7f98911cb700 1 -- 172.21.15.45:0/1596650882 wait complete. 2023-10-06T05:23:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 Processor -- start 2023-10-06T05:23:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 -- start start 2023-10-06T05:23:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988c06b5c0 0x7f988c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c159200 0x7f988c0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988c1595d0 0x7f988c0c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0c4e50 con 0x7f988c159200 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0c3e40 con 0x7f988c1595d0 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.224+0000 7f98911cb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0c49c0 con 0x7f988c06b5c0 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c159200 0x7f988c0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c159200 0x7f988c0c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55932/0 (socket says 172.21.15.45:55932) 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 -- 172.21.15.45:0/3993842882 learned_addr learned my addr 172.21.15.45:0/3993842882 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988ad9d700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988c06b5c0 0x7f988c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988b59e700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988c1595d0 0x7f988c0c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 -- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988c06b5c0 msgr2=0x7f988c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988c06b5c0 0x7f988c0b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 -- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988c1595d0 msgr2=0x7f988c0c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988c1595d0 0x7f988c0c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 -- 172.21.15.45:0/3993842882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874009420 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f988a59c700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c159200 0x7f988c0c29e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f988c06aed0 tx=0x7f9874017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f9882ffd700 1 -- 172.21.15.45:0/3993842882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874000d10 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f9882ffd700 1 -- 172.21.15.45:0/3993842882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874011900 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c0c4090 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f9882ffd700 1 -- 172.21.15.45:0/3993842882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874025420 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.225+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c0c4650 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.226+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c06e2f0 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.227+0000 7f9882ffd700 1 -- 172.21.15.45:0/3993842882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9874023070 con 0x7f988c159200 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.227+0000 7f9882ffd700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f986c066f30 0x7f986c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.227+0000 7f9882ffd700 1 -- 172.21.15.45:0/3993842882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f9874085e90 con 0x7f988c159200 2023-10-06T05:23:14.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.227+0000 7f988ad9d700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f986c066f30 0x7f986c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.228+0000 7f988ad9d700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f986c066f30 0x7f986c0693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f987c007fd0 tx=0x7f987c007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.232+0000 7f9882ffd700 1 -- 172.21.15.45:0/3993842882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9874052b60 con 0x7f988c159200 2023-10-06T05:23:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:14.520+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f988c0c7990 con 0x7f988c159200 2023-10-06T05:23:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.070+0000 7f9882ffd700 1 -- 172.21.15.45:0/3993842882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v144) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9874056180 con 0x7f988c159200 2023-10-06T05:23:15.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f986c066f30 msgr2=0x7f986c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f986c066f30 0x7f986c0693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f987c007fd0 tx=0x7f987c007d30 comp rx=0 tx=0).stop 2023-10-06T05:23:15.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c159200 msgr2=0x7f988c0c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c159200 0x7f988c0c29e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f988c06aed0 tx=0x7f9874017000 comp rx=0 tx=0).stop 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 shutdown_connections 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f986c066f30 0x7f986c0693e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988c1595d0 0x7f988c0c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988c06b5c0 0x7f988c0b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 --2- 172.21.15.45:0/3993842882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c159200 0x7f988c0c29e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 >> 172.21.15.45:0/3993842882 conn(0x7f988c06f090 msgr2=0x7f988c0c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 shutdown_connections 2023-10-06T05:23:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.073+0000 7f98911cb700 1 -- 172.21.15.45:0/3993842882 wait complete. 2023-10-06T05:23:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:23:15.091 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.136+0000 7f4427af6040 1 Processor -- start 2023-10-06T05:23:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.136+0000 7f4427af6040 1 -- start start 2023-10-06T05:23:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4427af6040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215f96ded0 0x56215f9440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4427af6040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa159f0 0x56215fa14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4427af6040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215fa19e40 0x56215fa14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4427af6040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215f88d980 con 0x56215fa19e40 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4427af6040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215f88d620 con 0x56215fa159f0 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4427af6040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56215f884400 con 0x56215f96ded0 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215fa19e40 0x56215fa14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215fa19e40 0x56215fa14c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55938/0 (socket says 172.21.15.45:55938) 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 -- 172.21.15.45:0/2094126518 learned_addr learned my addr 172.21.15.45:0/2094126518 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f441bfff700 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa159f0 0x56215fa14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4420d14700 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215f96ded0 0x56215f9440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 -- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215f96ded0 msgr2=0x56215f9440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215f96ded0 0x56215f9440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 -- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa159f0 msgr2=0x56215fa14710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa159f0 0x56215fa14710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.137+0000 7f4421515700 1 -- 172.21.15.45:0/2094126518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c005cf0 con 0x56215fa19e40 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f441bfff700 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa159f0 0x56215fa14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4421515700 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215fa19e40 0x56215fa14c50 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f441c006f40 tx=0x7f441c0069f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9e41f55f8389cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f441affd700 1 -- 172.21.15.45:0/2094126518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c006110 con 0x56215fa19e40 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f441affd700 1 -- 172.21.15.45:0/2094126518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c006270 con 0x56215fa19e40 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f441affd700 1 -- 172.21.15.45:0/2094126518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c004730 con 0x56215fa19e40 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 -- 172.21.15.45:0/2094126518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215fa19e40 msgr2=0x56215fa14c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215fa19e40 0x56215fa14c50 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f441c006f40 tx=0x7f441c0069f0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 -- 172.21.15.45:0/2094126518 shutdown_connections 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215fa19e40 0x56215fa14c50 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215f96ded0 0x56215f9440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 --2- 172.21.15.45:0/2094126518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa159f0 0x56215fa14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 -- 172.21.15.45:0/2094126518 >> 172.21.15.45:0/2094126518 conn(0x56215f9434a0 msgr2=0x56215fa105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.138+0000 7f4427af6040 1 -- 172.21.15.45:0/2094126518 shutdown_connections 2023-10-06T05:23:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.139+0000 7f4427af6040 1 -- 172.21.15.45:0/2094126518 wait complete. 2023-10-06T05:23:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.139+0000 7f4427af6040 1 Processor -- start 2023-10-06T05:23:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.139+0000 7f4427af6040 1 -- start start 2023-10-06T05:23:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4427af6040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215f96ded0 0x56215fa0ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4427af6040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215fa159f0 0x56215fa0a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4427af6040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa19e40 0x56215fa10110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4427af6040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215fa0beb0 con 0x56215f96ded0 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4427af6040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215fa0c150 con 0x56215fa19e40 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4427af6040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56215fa17e80 con 0x56215fa159f0 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4420d14700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215f96ded0 0x56215fa0ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4420d14700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215f96ded0 0x56215fa0ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55950/0 (socket says 172.21.15.45:55950) 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 learned_addr learned my addr 172.21.15.45:0/122523820 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa19e40 0x56215fa10110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.140+0000 7f441bfff700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215fa159f0 0x56215fa0a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215fa159f0 msgr2=0x56215fa0a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4420d14700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56215fa159f0 0x56215fa0a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa19e40 msgr2=0x56215fa10110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4420d14700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa19e40 0x56215fa10110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4404001e10 con 0x56215f96ded0 2023-10-06T05:23:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56215fa19e40 0x56215fa10110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4420d14700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215f96ded0 0x56215fa0ae10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f440c008bc0 tx=0x7f440c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f44197fa700 1 -- 172.21.15.45:0/122523820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c00f040 con 0x56215f96ded0 2023-10-06T05:23:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f44197fa700 1 -- 172.21.15.45:0/122523820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c010a10 con 0x56215f96ded0 2023-10-06T05:23:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f44197fa700 1 -- 172.21.15.45:0/122523820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c010d90 con 0x56215f96ded0 2023-10-06T05:23:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4414005cf0 con 0x56215f96ded0 2023-10-06T05:23:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.141+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56215f89c3d0 con 0x56215f96ded0 2023-10-06T05:23:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.142+0000 7f44197fa700 1 -- 172.21.15.45:0/122523820 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f440c015660 con 0x56215f96ded0 2023-10-06T05:23:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.143+0000 7f44197fa700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4408065890 0x7f4408067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.143+0000 7f44197fa700 1 -- 172.21.15.45:0/122523820 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f440c0845a0 con 0x56215f96ded0 2023-10-06T05:23:15.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.143+0000 7f441bfff700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4408065890 0x7f4408067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.149 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.143+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56215fa0a680 0x56215fa35160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.143+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa195d0 con 0x56215fa0a680 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.143+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56215fa0a680 0x56215fa35160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.144+0000 7f441bfff700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4408065890 0x7f4408067d40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4414005a90 tx=0x7f4414015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.144+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56215fa0a680 0x56215fa35160 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.145+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v101'257 uv231 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.145+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa0b690 con 0x56215fa0a680 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.147+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v101'258 uv258 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.147+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa356a0 con 0x56215fa0a680 2023-10-06T05:23:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.147+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v101'259 uv259 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.151 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:15.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.148+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa36a60 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.149+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'260 uv260 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.149+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa36e10 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.150+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.150+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa108a0 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.151+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.151+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa35f40 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.152+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v101'263 uv237 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.152+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa36340 con 0x56215fa0a680 2023-10-06T05:23:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.153+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v101'264 uv264 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.153+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa36340 con 0x56215fa0a680 2023-10-06T05:23:15.157 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:15.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.154+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v101'265 uv265 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.154+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.155+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'266 uv266 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.155+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.156+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'267 uv267 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.157+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.158+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'268 uv268 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.158+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.158+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v101'269 uv244 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.159+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.159+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v101'270 uv270 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.159+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.160+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v101'271 uv271 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.162 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.161+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.162+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v101'272 uv272 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.162+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.163+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v101'273 uv273 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.163+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa362f0 con 0x56215fa0a680 2023-10-06T05:23:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.164+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v101'274 uv274 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.164+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.165+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v101'275 uv275 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.165+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.166+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v101'276 uv250 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.166+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.167+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v101'277 uv277 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.167+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.168+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v101'278 uv278 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.169 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.168+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa381d0 con 0x56215fa0a680 2023-10-06T05:23:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.169+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v101'279 uv279 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.169+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa381d0 con 0x56215fa0a680 2023-10-06T05:23:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.170+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v101'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.170+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.171+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v101'281 uv281 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.171+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.172+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v101'282 uv256 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.172+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.173+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v101'283 uv283 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.173+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.177 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:15.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.174+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v101'284 uv284 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.174+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa381d0 con 0x56215fa0a680 2023-10-06T05:23:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.175+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v101'285 uv285 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.175+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa381d0 con 0x56215fa0a680 2023-10-06T05:23:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.176+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v101'286 uv286 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.176+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa371c0 con 0x56215fa0a680 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.177+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v101'287 uv287 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa0a680 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.177+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56215fa376d0 0x56215fa39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.177+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa39ff0 con 0x56215fa376d0 2023-10-06T05:23:15.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.177+0000 7f4420d14700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56215fa376d0 0x56215fa39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.178+0000 7f4420d14700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56215fa376d0 0x56215fa39ab0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.179+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v101'195 uv178 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.179+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3a400 con 0x56215fa376d0 2023-10-06T05:23:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.181+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v101'196 uv196 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.181+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3a8d0 con 0x56215fa376d0 2023-10-06T05:23:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.182+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v101'197 uv197 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.184 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.182+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3b1e0 con 0x56215fa376d0 2023-10-06T05:23:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.184+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v101'198 uv198 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.184+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3ad70 con 0x56215fa376d0 2023-10-06T05:23:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.186+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v101'199 uv199 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.186+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3ba80 con 0x56215fa376d0 2023-10-06T05:23:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.187+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v101'200 uv183 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.187+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3be30 con 0x56215fa376d0 2023-10-06T05:23:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.188+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v101'201 uv201 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.188+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3c1e0 con 0x56215fa376d0 2023-10-06T05:23:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.189+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v101'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.191 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:15.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.190+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3c1e0 con 0x56215fa376d0 2023-10-06T05:23:15.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.192+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v101'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.192+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3b610 con 0x56215fa376d0 2023-10-06T05:23:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.193+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v101'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.193+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3b610 con 0x56215fa376d0 2023-10-06T05:23:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.195+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v101'205 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.195+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3b590 con 0x56215fa376d0 2023-10-06T05:23:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.196+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v101'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.196+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3b590 con 0x56215fa376d0 2023-10-06T05:23:15.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.198+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v101'207 uv207 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.199 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.198+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3b590 con 0x56215fa376d0 2023-10-06T05:23:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.199+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v101'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.199+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3f900 con 0x56215fa376d0 2023-10-06T05:23:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.201+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v101'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.201+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3c1e0 con 0x56215fa376d0 2023-10-06T05:23:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.202+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v101'210 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.202+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3c1e0 con 0x56215fa376d0 2023-10-06T05:23:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.203+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v101'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.203+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3c1e0 con 0x56215fa376d0 2023-10-06T05:23:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.204+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v101'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.206 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.205+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3c1e0 con 0x56215fa376d0 2023-10-06T05:23:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.206+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v101'213 uv213 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.206+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3d1f0 con 0x56215fa376d0 2023-10-06T05:23:15.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.207+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v101'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.207+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3c1e0 con 0x56215fa376d0 2023-10-06T05:23:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.208+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v101'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.208+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56215fa3c6f0 0x56215fa3ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.209+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3f0b0 con 0x56215fa3c6f0 2023-10-06T05:23:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.209+0000 7f441bfff700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56215fa3c6f0 0x56215fa3ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.209+0000 7f441bfff700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56215fa3c6f0 0x56215fa3ead0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.211+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f44140079f0 con 0x56215fa3c6f0 2023-10-06T05:23:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.211+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3f5b0 con 0x56215fa3c6f0 2023-10-06T05:23:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.212+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v101'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.212+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa3faa0 con 0x56215fa3c6f0 2023-10-06T05:23:15.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.213+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v101'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.215 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:15.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.213+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa413a0 con 0x56215fa3c6f0 2023-10-06T05:23:15.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.214+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.214+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa41750 con 0x56215fa3c6f0 2023-10-06T05:23:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.215+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.215+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa40390 con 0x56215fa3c6f0 2023-10-06T05:23:15.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.216+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.216+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa3c6f0 2023-10-06T05:23:15.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.218+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'39 uv38 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.218+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa3c6f0 2023-10-06T05:23:15.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.219+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v101'40 uv40 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.219+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa3c6f0 2023-10-06T05:23:15.221 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:15.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.220+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v101'41 uv41 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.220+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa3c6f0 2023-10-06T05:23:15.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.221+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v101'42 uv42 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.221+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa376d0 2023-10-06T05:23:15.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.223+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.223+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa376d0 2023-10-06T05:23:15.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.224+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v101'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.224+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa376d0 2023-10-06T05:23:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.226+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v101'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.227 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.226+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa376d0 2023-10-06T05:23:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.227+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.227+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa376d0 2023-10-06T05:23:15.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.228+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.228+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa407b0 con 0x56215fa376d0 2023-10-06T05:23:15.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.230+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa376d0 2023-10-06T05:23:15.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.230+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56215fa41b00 0x56215fa43ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.230+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.230+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56215fa41b00 0x56215fa43ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.231+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56215fa41b00 0x56215fa43ee0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.232+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v101'94 uv88 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.232+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.233+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v101'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.233+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.235 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:15.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.235+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v101'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.235+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa45430 con 0x56215fa41b00 2023-10-06T05:23:15.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.235+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v101'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.235+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa45430 con 0x56215fa41b00 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.236+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v101'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.237+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.238+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v101'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.238+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.239+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v101'100 uv93 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.239+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.240+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v101'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.240+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.243 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:15.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.241+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v101'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.241+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.243+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v101'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.243+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa44420 con 0x56215fa41b00 2023-10-06T05:23:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.244+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v101'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.244+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56215fa44aa0 0x56215fa46e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.244+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa474a0 con 0x56215fa44aa0 2023-10-06T05:23:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.244+0000 7f4420d14700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56215fa44aa0 0x56215fa46e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.245+0000 7f4420d14700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56215fa44aa0 0x56215fa46e80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.247 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:15.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.246+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v101'43 uv43 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa44aa0 2023-10-06T05:23:15.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.246+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa47880 con 0x56215fa44aa0 2023-10-06T05:23:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.247+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [write 0~70] v101'44 uv44 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa44aa0 2023-10-06T05:23:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.248+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa47880 con 0x56215fa44aa0 2023-10-06T05:23:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.249+0000 7f4420d14700 1 -- 172.21.15.45:0/122523820 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v101'45 uv45 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f440c012040 con 0x56215fa44aa0 2023-10-06T05:23:15.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.249+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:83 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa47880 con 0x56215fa3c6f0 2023-10-06T05:23:15.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.250+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(83 605.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.250+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:84 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa47880 con 0x56215fa3c6f0 2023-10-06T05:23:15.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.251+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(84 605.00000000 [delete] v101'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.251+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa47880 con 0x56215fa3c6f0 2023-10-06T05:23:15.254 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:15.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.253+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(85 605.00000000 [create] v101'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.253+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa48890 con 0x56215fa3c6f0 2023-10-06T05:23:15.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.254+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.254+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa48890 con 0x56215fa3c6f0 2023-10-06T05:23:15.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.255+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.255+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa47880 con 0x56215fa3c6f0 2023-10-06T05:23:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.256+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(88 605.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.256+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56215fa48160 0x56215fa48530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.256+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:89 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.257+0000 7f441bfff700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56215fa48160 0x56215fa48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.257+0000 7f441bfff700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56215fa48160 0x56215fa48530 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.258+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(89 mds0_inotable [create] v101'84 uv78 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.258+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:90 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.260+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(90 mds0_inotable [delete] v101'85 uv85 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.260+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:15.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.261+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(91 mds0_inotable [create] v101'86 uv86 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.261+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.262+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(92 mds0_inotable [write 0~34] v101'87 uv87 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.262+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.263+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v101'88 uv88 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.263+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.264+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(94 100.00000000 [create] v101'89 uv83 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.264+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.265+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(95 100.00000000 [delete] v101'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.265+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.267 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:15.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.266+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(96 100.00000000 [create] v101'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.266+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ace0 con 0x56215fa48160 2023-10-06T05:23:15.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.267+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(97 100.00000000 [omap-set-header] v101'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.267+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4da50 con 0x56215fa48160 2023-10-06T05:23:15.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.268+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v101'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa48160 2023-10-06T05:23:15.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.268+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4b1d0 con 0x56215fa3c6f0 2023-10-06T05:23:15.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.269+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v101'40 uv39 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.269+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4b1d0 con 0x56215fa3c6f0 2023-10-06T05:23:15.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.270+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v101'41 uv41 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.270+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4b1d0 con 0x56215fa3c6f0 2023-10-06T05:23:15.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.272+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v101'42 uv42 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.273 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:15.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.272+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4b1d0 con 0x56215fa3c6f0 2023-10-06T05:23:15.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.273+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v101'43 uv43 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.273+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56215fa4b6e0 0x56215fa4bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.273+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa4b6e0 2023-10-06T05:23:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.273+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56215fa4b6e0 0x56215fa4bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.274+0000 7f4421515700 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56215fa4b6e0 0x56215fa4bab0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.275+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa4b6e0 2023-10-06T05:23:15.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.275+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa4b6e0 2023-10-06T05:23:15.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.276+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v101'52 uv52 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa4b6e0 2023-10-06T05:23:15.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.276+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa4b6e0 2023-10-06T05:23:15.278 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.277+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v101'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa4b6e0 2023-10-06T05:23:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.277+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4f1b0 con 0x56215fa4b6e0 2023-10-06T05:23:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.278+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa4b6e0 2023-10-06T05:23:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.278+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4f1b0 con 0x56215fa4b6e0 2023-10-06T05:23:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.279+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa4b6e0 2023-10-06T05:23:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.279+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa4b6e0 2023-10-06T05:23:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.280+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa4b6e0 2023-10-06T05:23:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.280+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa41b00 2023-10-06T05:23:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.281+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v101'42 uv41 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.281+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa41b00 2023-10-06T05:23:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.283+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v101'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.283+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa41b00 2023-10-06T05:23:15.285 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.284+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v101'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.284+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e1a0 con 0x56215fa41b00 2023-10-06T05:23:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.285+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v101'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.285+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa41b00 2023-10-06T05:23:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.286+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v101'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f441c0044f0 con 0x56215fa41b00 2023-10-06T05:23:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.286+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.288+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v101'107 uv99 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.288+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.289+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v101'108 uv108 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.289+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.290+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v101'109 uv109 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.292 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.290+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.292+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v101'110 uv110 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.292+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.293+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v101'111 uv111 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.293+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.294+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v101'112 uv106 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.294+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.295+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v101'113 uv113 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.295+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.296+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v101'114 uv114 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.298 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.296+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.297+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v101'115 uv115 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.297+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.299+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v101'116 uv116 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.299+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4ad60 con 0x56215fa3c6f0 2023-10-06T05:23:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.300+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v101'117 uv117 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.300+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) v8 -- 0x56215fa4e690 con 0x56215fa3c6f0 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f441bfff700 1 -- 172.21.15.45:0/122523820 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v101'118 uv118 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4414013420 con 0x56215fa3c6f0 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56215fa3c6f0 msgr2=0x56215fa3ead0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56215fa3c6f0 0x56215fa3ead0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56215fa0a680 msgr2=0x56215fa35160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56215fa0a680 0x56215fa35160 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56215fa48160 msgr2=0x56215fa48530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56215fa48160 0x56215fa48530 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56215fa376d0 msgr2=0x56215fa39ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56215fa376d0 0x56215fa39ab0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56215fa4b6e0 msgr2=0x56215fa4bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56215fa4b6e0 0x56215fa4bab0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 reap_dead start 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56215fa44aa0 msgr2=0x56215fa46e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56215fa44aa0 0x56215fa46e80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56215fa41b00 msgr2=0x56215fa43ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56215fa41b00 0x56215fa43ee0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4408065890 msgr2=0x7f4408067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4408065890 0x7f4408067d40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4414005a90 tx=0x7f4414015000 comp rx=0 tx=0).stop 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215f96ded0 msgr2=0x56215fa0ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 --2- 172.21.15.45:0/122523820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56215f96ded0 0x56215fa0ae10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f440c008bc0 tx=0x7f440c00c040 comp rx=0 tx=0).stop 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4421515700 1 -- 172.21.15.45:0/122523820 reap_dead start 2023-10-06T05:23:15.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 shutdown_connections 2023-10-06T05:23:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 >> 172.21.15.45:0/122523820 conn(0x56215f9434a0 msgr2=0x56215f9888f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 shutdown_connections 2023-10-06T05:23:15.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.301+0000 7f4427af6040 1 -- 172.21.15.45:0/122523820 wait complete. 2023-10-06T05:23:15.308 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2023-10-06T05:23:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.355+0000 7f809526c040 1 Processor -- start 2023-10-06T05:23:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.355+0000 7f809526c040 1 -- start start 2023-10-06T05:23:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f809526c040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b03857e0 0x55c5b0340830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f809526c040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410b30 0x55c5b036bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f809526c040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b0410f00 0x55c5b040f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f809526c040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b0288980 con 0x55c5b0410f00 2023-10-06T05:23:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f809526c040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b0288620 con 0x55c5b0410b30 2023-10-06T05:23:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f809526c040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5b027f400 con 0x55c5b03857e0 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f808ec8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b0410f00 0x55c5b040f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f808ec8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b0410f00 0x55c5b040f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55970/0 (socket says 172.21.15.45:55970) 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f808ec8b700 1 -- 172.21.15.45:0/3329820932 learned_addr learned my addr 172.21.15.45:0/3329820932 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.356+0000 7f808e48a700 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b03857e0 0x55c5b0340830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808dc89700 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410b30 0x55c5b036bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808ec8b700 1 -- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b03857e0 msgr2=0x55c5b0340830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808ec8b700 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b03857e0 0x55c5b0340830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808ec8b700 1 -- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410b30 msgr2=0x55c5b036bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808ec8b700 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410b30 0x55c5b036bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808ec8b700 1 -- 172.21.15.45:0/3329820932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080005cf0 con 0x55c5b0410f00 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808e48a700 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b03857e0 0x55c5b0340830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:15.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808ec8b700 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b0410f00 0x55c5b040f6f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f80880016e0 tx=0x7f8088006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acea01823a6cb422 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808cc87700 1 -- 172.21.15.45:0/3329820932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088015030 con 0x55c5b0410f00 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808cc87700 1 -- 172.21.15.45:0/3329820932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8088006320 con 0x55c5b0410f00 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f808cc87700 1 -- 172.21.15.45:0/3329820932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088004770 con 0x55c5b0410f00 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f809526c040 1 -- 172.21.15.45:0/3329820932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b0410f00 msgr2=0x55c5b040f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.357+0000 7f809526c040 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b0410f00 0x55c5b040f6f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f80880016e0 tx=0x7f8088006950 comp rx=0 tx=0).stop 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.358+0000 7f809526c040 1 -- 172.21.15.45:0/3329820932 shutdown_connections 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.358+0000 7f809526c040 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b0410f00 0x55c5b040f6f0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.358+0000 7f809526c040 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b03857e0 0x55c5b0340830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.358+0000 7f809526c040 1 --2- 172.21.15.45:0/3329820932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410b30 0x55c5b036bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.358+0000 7f809526c040 1 -- 172.21.15.45:0/3329820932 >> 172.21.15.45:0/3329820932 conn(0x55c5b033e450 msgr2=0x55c5b036c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.358+0000 7f809526c040 1 -- 172.21.15.45:0/3329820932 shutdown_connections 2023-10-06T05:23:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.358+0000 7f809526c040 1 -- 172.21.15.45:0/3329820932 wait complete. 2023-10-06T05:23:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.359+0000 7f809526c040 1 Processor -- start 2023-10-06T05:23:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.359+0000 7f809526c040 1 -- start start 2023-10-06T05:23:15.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f809526c040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b03857e0 0x55c5b040bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f809526c040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b0410b30 0x55c5b040b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f809526c040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410f00 0x55c5b033f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f809526c040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b0407b60 con 0x55c5b03857e0 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f809526c040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b0407620 con 0x55c5b0410f00 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f809526c040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5b0409870 con 0x55c5b0410b30 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b03857e0 0x55c5b040bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b03857e0 0x55c5b040bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55982/0 (socket says 172.21.15.45:55982) 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 -- 172.21.15.45:0/3574304393 learned_addr learned my addr 172.21.15.45:0/3574304393 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808ec8b700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410f00 0x55c5b033f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808dc89700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b0410b30 0x55c5b040b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 -- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b0410b30 msgr2=0x55c5b040b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5b0410b30 0x55c5b040b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 -- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410f00 msgr2=0x55c5b033f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5b0410f00 0x55c5b033f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 -- 172.21.15.45:0/3574304393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8088006c30 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.360+0000 7f808e48a700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b03857e0 0x55c5b040bd70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8078008aa0 tx=0x7f807800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.361+0000 7f8086ffd700 1 -- 172.21.15.45:0/3574304393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078019070 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.361+0000 7f8086ffd700 1 -- 172.21.15.45:0/3574304393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8078010040 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.361+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5b02973d0 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.361+0000 7f8086ffd700 1 -- 172.21.15.45:0/3574304393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800f050 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.361+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5b028aae0 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.362+0000 7f8086ffd700 1 -- 172.21.15.45:0/3574304393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8078015460 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.363+0000 7f8086ffd700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8070065890 0x7f8070067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.363+0000 7f8086ffd700 1 -- 172.21.15.45:0/3574304393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f8078083d10 con 0x55c5b03857e0 2023-10-06T05:23:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.363+0000 7f808dc89700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8070065890 0x7f8070067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.363+0000 7f809526c040 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c5b040b8e0 0x55c5b0406d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.363+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) v8 -- 0x55c5b0413300 con 0x55c5b040b8e0 2023-10-06T05:23:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.363+0000 7f808ec8b700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c5b040b8e0 0x55c5b0406d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.363+0000 7f808dc89700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8070065890 0x7f8070067d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f808000a520 tx=0x7f8080016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.364+0000 7f808ec8b700 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c5b040b8e0 0x55c5b0406d60 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f808ec8b700 1 -- 172.21.15.45:0/3574304393 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 mds0_inotable [delete] v101'94 uv94 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8088004670 con 0x55c5b040b8e0 2023-10-06T05:23:15.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c5b040b8e0 msgr2=0x55c5b0406d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c5b040b8e0 0x55c5b0406d60 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8070065890 msgr2=0x7f8070067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8070065890 0x7f8070067d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f808000a520 tx=0x7f8080016000 comp rx=0 tx=0).stop 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b03857e0 msgr2=0x55c5b040bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 --2- 172.21.15.45:0/3574304393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b03857e0 0x55c5b040bd70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8078008aa0 tx=0x7f807800c040 comp rx=0 tx=0).stop 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f808ec8b700 1 -- 172.21.15.45:0/3574304393 reap_dead start 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 shutdown_connections 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 >> 172.21.15.45:0/3574304393 conn(0x55c5b033e450 msgr2=0x55c5b040d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 shutdown_connections 2023-10-06T05:23:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.365+0000 7f809526c040 1 -- 172.21.15.45:0/3574304393 wait complete. 2023-10-06T05:23:15.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.507+0000 7f57de42c700 1 Processor -- start 2023-10-06T05:23:15.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.507+0000 7f57de42c700 1 -- start start 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57de42c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57d8159150 0x7f57d80ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57de42c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159520 0x7f57d815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57de42c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f57d815d3d0 0x7f57d8067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57de42c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d80c9c90 con 0x7f57d8159150 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57de42c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d815d8e0 con 0x7f57d815d3d0 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57de42c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d815dbc0 con 0x7f57d8159520 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57d7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57d8159150 0x7f57d80ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57d7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57d8159150 0x7f57d80ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55992/0 (socket says 172.21.15.45:55992) 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.508+0000 7f57d7fff700 1 -- 172.21.15.45:0/777158133 learned_addr learned my addr 172.21.15.45:0/777158133 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57dc9c9700 1 --2- 172.21.15.45:0/777158133 >> v2:172.21.15.144:3300/0 conn(0x7f57d815d3d0 0x7f57d8067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d77fe700 1 --2- 172.21.15.45:0/777158133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159520 0x7f57d815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d7fff700 1 -- 172.21.15.45:0/777158133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159520 msgr2=0x7f57d815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d7fff700 1 --2- 172.21.15.45:0/777158133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159520 0x7f57d815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d7fff700 1 -- 172.21.15.45:0/777158133 >> v2:172.21.15.144:3300/0 conn(0x7f57d815d3d0 msgr2=0x7f57d8067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d7fff700 1 --2- 172.21.15.45:0/777158133 >> v2:172.21.15.144:3300/0 conn(0x7f57d815d3d0 0x7f57d8067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d7fff700 1 -- 172.21.15.45:0/777158133 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80c9920 con 0x7f57d8159150 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d7fff700 1 --2- 172.21.15.45:0/777158133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d80ca120 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f57d80c3cf0 tx=0x7f57c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53695d2f072558be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d6ffd700 1 -- 172.21.15.45:0/777158133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800b860 con 0x7f57d8159150 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d6ffd700 1 -- 172.21.15.45:0/777158133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c800b9c0 con 0x7f57d8159150 2023-10-06T05:23:15.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.509+0000 7f57d6ffd700 1 -- 172.21.15.45:0/777158133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800bd60 con 0x7f57d8159150 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 -- 172.21.15.45:0/777158133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 msgr2=0x7f57d80ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 --2- 172.21.15.45:0/777158133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d80ca120 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f57d80c3cf0 tx=0x7f57c8009250 comp rx=0 tx=0).stop 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 -- 172.21.15.45:0/777158133 shutdown_connections 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 --2- 172.21.15.45:0/777158133 >> v2:172.21.15.144:3300/0 conn(0x7f57d815d3d0 0x7f57d8067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 --2- 172.21.15.45:0/777158133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d80ca120 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 --2- 172.21.15.45:0/777158133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159520 0x7f57d815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 -- 172.21.15.45:0/777158133 >> 172.21.15.45:0/777158133 conn(0x7f57d80691c0 msgr2=0x7f57d80cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 -- 172.21.15.45:0/777158133 shutdown_connections 2023-10-06T05:23:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.510+0000 7f57de42c700 1 -- 172.21.15.45:0/777158133 wait complete. 2023-10-06T05:23:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.511+0000 7f57de42c700 1 Processor -- start 2023-10-06T05:23:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.511+0000 7f57de42c700 1 -- start start 2023-10-06T05:23:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57de42c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d81698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57de42c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d815d3d0 0x7f57d80c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57de42c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159a90 0x7f57d80adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57de42c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d816ae90 con 0x7f57d8159150 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57de42c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d816ab60 con 0x7f57d815d3d0 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57de42c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d816a260 con 0x7f57d8159a90 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57d7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d81698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57d7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d81698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56006/0 (socket says 172.21.15.45:56006) 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57d7fff700 1 -- 172.21.15.45:0/4135843627 learned_addr learned my addr 172.21.15.45:0/4135843627 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57dc9c9700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159a90 0x7f57d80adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.512+0000 7f57d77fe700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d815d3d0 0x7f57d80c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57d7fff700 1 -- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159a90 msgr2=0x7f57d80adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57d7fff700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159a90 0x7f57d80adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57d7fff700 1 -- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d815d3d0 msgr2=0x7f57d80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57d7fff700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d815d3d0 0x7f57d80c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57d7fff700 1 -- 172.21.15.45:0/4135843627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80094f0 con 0x7f57d8159150 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57d7fff700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d81698f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f57d806c4b0 tx=0x7f57c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57bffff700 1 -- 172.21.15.45:0/4135843627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8011dc0 con 0x7f57d8159150 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57bffff700 1 -- 172.21.15.45:0/4135843627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c8024440 con 0x7f57d8159150 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d816a540 con 0x7f57d8159150 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57bffff700 1 -- 172.21.15.45:0/4135843627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8024800 con 0x7f57d8159150 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.513+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d8169e30 con 0x7f57d8159150 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.514+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d816a020 con 0x7f57d8159150 2023-10-06T05:23:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.515+0000 7f57bffff700 1 -- 172.21.15.45:0/4135843627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57c801a420 con 0x7f57d8159150 2023-10-06T05:23:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.515+0000 7f57bffff700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57c4066f30 0x7f57c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.515+0000 7f57bffff700 1 -- 172.21.15.45:0/4135843627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f57c80869e0 con 0x7f57d8159150 2023-10-06T05:23:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.515+0000 7f57d77fe700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57c4066f30 0x7f57c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.516+0000 7f57d77fe700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57c4066f30 0x7f57c40693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f57d816ce50 tx=0x7f57cc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.520+0000 7f57bffff700 1 -- 172.21.15.45:0/4135843627 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57d816a020 con 0x7f57d8159150 2023-10-06T05:23:15.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.769+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f57d816a020 con 0x7f57d8159150 2023-10-06T05:23:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.867+0000 7f57bffff700 1 -- 172.21.15.45:0/4135843627 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v145) v1 ==== 162+0+0 (secure 0 0 0) 0x7f57d816a020 con 0x7f57d8159150 2023-10-06T05:23:15.870 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:15.869+0000 7fccd5fb6700 -1 mds.0.inotable: load_2 could not read table: -2 2023-10-06T05:23:15.870 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:15.869+0000 7fccd5fb6700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-10-06T05:23:15.871 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:15.869+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:15.871 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:15.869+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:23:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57c4066f30 msgr2=0x7f57c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57c4066f30 0x7f57c40693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f57d816ce50 tx=0x7f57cc008040 comp rx=0 tx=0).stop 2023-10-06T05:23:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 msgr2=0x7f57d81698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d81698f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f57d806c4b0 tx=0x7f57c8017000 comp rx=0 tx=0).stop 2023-10-06T05:23:15.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 shutdown_connections 2023-10-06T05:23:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57c4066f30 0x7f57c40693e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d8159a90 0x7f57d80adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57d8159150 0x7f57d81698f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 --2- 172.21.15.45:0/4135843627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d815d3d0 0x7f57d80c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 >> 172.21.15.45:0/4135843627 conn(0x7f57d80691c0 msgr2=0x7f57d8067620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 shutdown_connections 2023-10-06T05:23:15.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:15.870+0000 7f57de42c700 1 -- 172.21.15.45:0/4135843627 wait complete. 2023-10-06T05:23:15.874 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:15.878 INFO:tasks.ceph.mds.c.smithi045.stderr: -28> 2023-10-06T05:23:15.869+0000 7fccd5fb6700 -1 mds.0.inotable: load_2 could not read table: -2 2023-10-06T05:23:15.878 INFO:tasks.ceph.mds.c.smithi045.stderr: -27> 2023-10-06T05:23:15.869+0000 7fccd5fb6700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-10-06T05:23:15.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:15.900 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:16.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.020+0000 7f788bf7f700 1 Processor -- start 2023-10-06T05:23:16.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.020+0000 7f788bf7f700 1 -- start start 2023-10-06T05:23:16.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f788bf7f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7884156ce0 0x7f78841570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f788bf7f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788406bf70 con 0x7f788406bad0 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f788bf7f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788415b380 con 0x7f788415ac10 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f788bf7f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78840b02f0 con 0x7f7884156ce0 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f788951a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f788406bad0 0x7f78840afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38308/0 (socket says 172.21.15.45:38308) 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f788951a700 1 -- 172.21.15.45:0/236873797 learned_addr learned my addr 172.21.15.45:0/236873797 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f7889d1b700 1 --2- 172.21.15.45:0/236873797 >> v2:172.21.15.144:3300/0 conn(0x7f7884156ce0 0x7f78841570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f7888d19700 1 -- 172.21.15.45:0/236873797 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378262264 0 0) 0x7f788406bf70 con 0x7f788406bad0 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f7888d19700 1 -- 172.21.15.45:0/236873797 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7870003570 con 0x7f788406bad0 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f7889d1b700 1 -- 172.21.15.45:0/236873797 >> v1:172.21.15.144:6789/0 conn(0x7f788415ac10 legacy=0x7f78840ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f7889d1b700 1 -- 172.21.15.45:0/236873797 >> v1:172.21.15.45:6789/0 conn(0x7f788406bad0 legacy=0x7f78840afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f7888d19700 1 -- 172.21.15.45:0/236873797 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621173224 0 0) 0x7f788415b380 con 0x7f788415ac10 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.021+0000 7f7889d1b700 1 -- 172.21.15.45:0/236873797 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788415b660 con 0x7f7884156ce0 2023-10-06T05:23:16.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.022+0000 7f7889d1b700 1 --2- 172.21.15.45:0/236873797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7884156ce0 0x7f78841570b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f78800098e0 tx=0x7f78800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b9dc259b610a72f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.022+0000 7f7888d19700 1 -- 172.21.15.45:0/236873797 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788000b640 con 0x7f7884156ce0 2023-10-06T05:23:16.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.022+0000 7f7888d19700 1 -- 172.21.15.45:0/236873797 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788000b7a0 con 0x7f7884156ce0 2023-10-06T05:23:16.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.022+0000 7f7888d19700 1 -- 172.21.15.45:0/236873797 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788000bb40 con 0x7f7884156ce0 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.022+0000 7f788bf7f700 1 -- 172.21.15.45:0/236873797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7884156ce0 msgr2=0x7f78841570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.022+0000 7f788bf7f700 1 --2- 172.21.15.45:0/236873797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7884156ce0 0x7f78841570b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f78800098e0 tx=0x7f78800091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.022+0000 7f788bf7f700 1 -- 172.21.15.45:0/236873797 shutdown_connections 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.023+0000 7f788bf7f700 1 --2- 172.21.15.45:0/236873797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7884156ce0 0x7f78841570b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.023+0000 7f788bf7f700 1 -- 172.21.15.45:0/236873797 >> 172.21.15.45:0/236873797 conn(0x7f788406f090 msgr2=0x7f788406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.023+0000 7f788bf7f700 1 -- 172.21.15.45:0/236873797 shutdown_connections 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.023+0000 7f788bf7f700 1 -- 172.21.15.45:0/236873797 wait complete. 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.024+0000 7f788bf7f700 1 Processor -- start 2023-10-06T05:23:16.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.024+0000 7f788bf7f700 1 -- start start 2023-10-06T05:23:16.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.024+0000 7f788bf7f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788406bad0 0x7f7884154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788bf7f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7884156ce0 0x7f78840c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788bf7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f788415ac10 0x7f78840c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788bf7f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c31b0 con 0x7f788415ac10 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788bf7f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c3b80 con 0x7f788406bad0 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788bf7f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c4d90 con 0x7f7884156ce0 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f788415ac10 0x7f78840c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f788415ac10 0x7f78840c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56034/0 (socket says 172.21.15.45:56034) 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 -- 172.21.15.45:0/1887373365 learned_addr learned my addr 172.21.15.45:0/1887373365 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f7889d1b700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788406bad0 0x7f7884154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788951a700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7884156ce0 0x7f78840c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 -- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7884156ce0 msgr2=0x7f78840c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7884156ce0 0x7f78840c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 -- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788406bad0 msgr2=0x7f7884154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788406bad0 0x7f7884154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 -- 172.21.15.45:0/1887373365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009420 con 0x7f788415ac10 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788951a700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7884156ce0 0x7f78840c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788a51c700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f788415ac10 0x7f78840c4850 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f787c00c840 tx=0x7f787c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f7879ffb700 1 -- 172.21.15.45:0/1887373365 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c012630 con 0x7f788415ac10 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f7879ffb700 1 -- 172.21.15.45:0/1887373365 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c012790 con 0x7f788415ac10 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840c1150 con 0x7f788415ac10 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.025+0000 7f7879ffb700 1 -- 172.21.15.45:0/1887373365 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c012b50 con 0x7f788415ac10 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.026+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78840c2e40 con 0x7f788415ac10 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.027+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78840c3360 con 0x7f788415ac10 2023-10-06T05:23:16.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.027+0000 7f7879ffb700 1 -- 172.21.15.45:0/1887373365 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f787c01a3a0 con 0x7f788415ac10 2023-10-06T05:23:16.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.027+0000 7f7879ffb700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7868066f30 0x7f78680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.028+0000 7f7879ffb700 1 -- 172.21.15.45:0/1887373365 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f787c084d20 con 0x7f788415ac10 2023-10-06T05:23:16.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.028+0000 7f7889d1b700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7868066f30 0x7f78680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.028+0000 7f7889d1b700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7868066f30 0x7f78680693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f78800093f0 tx=0x7f7880017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.032+0000 7f7879ffb700 1 -- 172.21.15.45:0/1887373365 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78840c3360 con 0x7f788415ac10 2023-10-06T05:23:16.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.283+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7884157430 con 0x7f788415ac10 2023-10-06T05:23:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.283+0000 7f7879ffb700 1 -- 172.21.15.45:0/1887373365 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f787c055420 con 0x7f788415ac10 2023-10-06T05:23:16.286 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:16.286 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:15.870458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7868066f30 msgr2=0x7f78680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7868066f30 0x7f78680693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f78800093f0 tx=0x7f7880017000 comp rx=0 tx=0).stop 2023-10-06T05:23:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f788415ac10 msgr2=0x7f78840c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f788415ac10 0x7f78840c4850 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f787c00c840 tx=0x7f787c001670 comp rx=0 tx=0).stop 2023-10-06T05:23:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 shutdown_connections 2023-10-06T05:23:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7868066f30 0x7f78680693e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f788415ac10 0x7f78840c4850 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788406bad0 0x7f7884154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 --2- 172.21.15.45:0/1887373365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7884156ce0 0x7f78840c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 >> 172.21.15.45:0/1887373365 conn(0x7f788406f090 msgr2=0x7f78841554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 shutdown_connections 2023-10-06T05:23:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.286+0000 7f788bf7f700 1 -- 172.21.15.45:0/1887373365 wait complete. 2023-10-06T05:23:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2023-10-06T05:23:16.302 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:16.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.473+0000 7fe31dabb700 1 Processor -- start 2023-10-06T05:23:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.473+0000 7fe31dabb700 1 -- start start 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe31dabb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3181570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe31dabb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31806bf40 con 0x7fe31806bad0 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe31dabb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31815b380 con 0x7fe31815ac10 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe31dabb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180b02f0 con 0x7fe318156ce0 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe316ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe31806bad0 0x7fe3180afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52072/0 (socket says 172.21.15.45:52072) 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe316ffd700 1 -- 172.21.15.45:0/2606208278 learned_addr learned my addr 172.21.15.45:0/2606208278 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe3177fe700 1 --2- 172.21.15.45:0/2606208278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3181570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639958702 0 0) 0x7fe31806bf40 con 0x7fe31806bad0 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.474+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe304003570 con 0x7fe31806bad0 2023-10-06T05:23:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525905642 0 0) 0x7fe31815b380 con 0x7fe31815ac10 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe31806bf40 con 0x7fe31815ac10 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012264205 0 0) 0x7fe304003570 con 0x7fe31806bad0 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3177fe700 1 -- 172.21.15.45:0/2606208278 >> v1:172.21.15.144:6789/0 conn(0x7fe31815ac10 legacy=0x7fe3180ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3177fe700 1 -- 172.21.15.45:0/2606208278 >> v1:172.21.15.45:6789/0 conn(0x7fe31806bad0 legacy=0x7fe3180afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3177fe700 1 -- 172.21.15.45:0/2606208278 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31815b660 con 0x7fe318156ce0 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3177fe700 1 --2- 172.21.15.45:0/2606208278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3181570b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe308009910 tx=0x7fe3080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead846b70eb7d8ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300001470 con 0x7fe318156ce0 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe308011420 con 0x7fe318156ce0 2023-10-06T05:23:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.475+0000 7fe3167fc700 1 -- 172.21.15.45:0/2606208278 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3080117c0 con 0x7fe318156ce0 2023-10-06T05:23:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.476+0000 7fe31dabb700 1 -- 172.21.15.45:0/2606208278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 msgr2=0x7fe3181570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.476+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2606208278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3181570b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe308009910 tx=0x7fe3080091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.476+0000 7fe31dabb700 1 -- 172.21.15.45:0/2606208278 shutdown_connections 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.476+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2606208278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3181570b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.476+0000 7fe31dabb700 1 -- 172.21.15.45:0/2606208278 >> 172.21.15.45:0/2606208278 conn(0x7fe31806f090 msgr2=0x7fe31806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.476+0000 7fe31dabb700 1 -- 172.21.15.45:0/2606208278 shutdown_connections 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.476+0000 7fe31dabb700 1 -- 172.21.15.45:0/2606208278 wait complete. 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.477+0000 7fe31dabb700 1 Processor -- start 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.477+0000 7fe31dabb700 1 -- start start 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe31dabb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31806bad0 0x7fe318158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe31dabb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3180abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe31dabb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31815ac10 0x7fe3180c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe31dabb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe318157890 con 0x7fe31806bad0 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe31dabb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe318159290 con 0x7fe31815ac10 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe31dabb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe318159570 con 0x7fe318156ce0 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31806bad0 0x7fe318158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31806bad0 0x7fe318158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35254/0 (socket says 172.21.15.45:35254) 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 -- 172.21.15.45:0/2874915308 learned_addr learned my addr 172.21.15.45:0/2874915308 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe316ffd700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3180abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe317fff700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31815ac10 0x7fe3180c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 -- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 msgr2=0x7fe3180abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3180abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 -- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31815ac10 msgr2=0x7fe3180c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31815ac10 0x7fe3180c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.478+0000 7fe3177fe700 1 -- 172.21.15.45:0/2874915308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009420 con 0x7fe31806bad0 2023-10-06T05:23:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.479+0000 7fe3177fe700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31806bad0 0x7fe318158a40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe3080091e0 tx=0x7fe308016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.479+0000 7fe30f7fe700 1 -- 172.21.15.45:0/2874915308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30801cb30 con 0x7fe31806bad0 2023-10-06T05:23:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.479+0000 7fe30f7fe700 1 -- 172.21.15.45:0/2874915308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30801cc90 con 0x7fe31806bad0 2023-10-06T05:23:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.479+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe318159850 con 0x7fe31806bad0 2023-10-06T05:23:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.479+0000 7fe30f7fe700 1 -- 172.21.15.45:0/2874915308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30801a6a0 con 0x7fe31806bad0 2023-10-06T05:23:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.479+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3180abfe0 con 0x7fe31806bad0 2023-10-06T05:23:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.480+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31806e2f0 con 0x7fe31806bad0 2023-10-06T05:23:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.480+0000 7fe30f7fe700 1 -- 172.21.15.45:0/2874915308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe30801aad0 con 0x7fe31806bad0 2023-10-06T05:23:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.481+0000 7fe30f7fe700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe2f8066f30 0x7fe2f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.481+0000 7fe30f7fe700 1 -- 172.21.15.45:0/2874915308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7fe308086a20 con 0x7fe31806bad0 2023-10-06T05:23:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.481+0000 7fe316ffd700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe2f8066f30 0x7fe2f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.482+0000 7fe316ffd700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe2f8066f30 0x7fe2f80693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe31816a9e0 tx=0x7fe300000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.485+0000 7fe30f7fe700 1 -- 172.21.15.45:0/2874915308 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3080535f0 con 0x7fe31806bad0 2023-10-06T05:23:16.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.735+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe31815b940 con 0x7fe31806bad0 2023-10-06T05:23:16.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.736+0000 7fe30f7fe700 1 -- 172.21.15.45:0/2874915308 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fe308056c10 con 0x7fe31806bad0 2023-10-06T05:23:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:16.739 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:15.870458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe2f8066f30 msgr2=0x7fe2f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe2f8066f30 0x7fe2f80693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe31816a9e0 tx=0x7fe300000bc0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31806bad0 msgr2=0x7fe318158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31806bad0 0x7fe318158a40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe3080091e0 tx=0x7fe308016000 comp rx=0 tx=0).stop 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 shutdown_connections 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe2f8066f30 0x7fe2f80693e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31815ac10 0x7fe3180c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31806bad0 0x7fe318158a40 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 --2- 172.21.15.45:0/2874915308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe318156ce0 0x7fe3180abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 >> 172.21.15.45:0/2874915308 conn(0x7fe31806f090 msgr2=0x7fe3180c3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 shutdown_connections 2023-10-06T05:23:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.738+0000 7fe31dabb700 1 -- 172.21.15.45:0/2874915308 wait complete. 2023-10-06T05:23:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2023-10-06T05:23:16.754 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:16.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:16.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.884+0000 7f4e9bf49700 1 Processor -- start 2023-10-06T05:23:16.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.885+0000 7f4e9bf49700 1 -- start start 2023-10-06T05:23:16.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.885+0000 7f4e9bf49700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e94158f60 0x7f4e94159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e9bf49700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e940cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e9bf49700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e94069bd0 con 0x7f4e940cc390 2023-10-06T05:23:16.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e9bf49700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9406a4d0 con 0x7f4e94158f60 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e9bf49700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e94068fc0 con 0x7f4e9406a060 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e94158f60 0x7f4e94159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e994e4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e940cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e94158f60 0x7f4e94159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:35542/0 (socket says 172.21.15.45:35542) 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e994e4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e940cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:43550/0 (socket says 172.21.15.45:43550) 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 -- 172.21.15.45:0/2231387434 learned_addr learned my addr 172.21.15.45:0/2231387434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e9a4e6700 1 --1- 172.21.15.45:0/2231387434 >> v1:172.21.15.144:6789/0 conn(0x7f4e940cc390 0x7f4e940c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:57944/0 (socket says 172.21.15.45:57944) 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 -- 172.21.15.45:0/2231387434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 msgr2=0x7f4e940cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 --2- 172.21.15.45:0/2231387434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e940cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 -- 172.21.15.45:0/2231387434 >> v1:172.21.15.144:6789/0 conn(0x7f4e940cc390 legacy=0x7f4e940c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 -- 172.21.15.45:0/2231387434 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940692a0 con 0x7f4e94158f60 2023-10-06T05:23:16.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.886+0000 7f4e99ce5700 1 --2- 172.21.15.45:0/2231387434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 0x7f4e94159330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4e900099b0 tx=0x7f4e90009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e443c1fb27dd3fd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e98ce3700 1 -- 172.21.15.45:0/2231387434 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90009ea0 con 0x7f4e94158f60 2023-10-06T05:23:16.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e98ce3700 1 -- 172.21.15.45:0/2231387434 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9000b860 con 0x7f4e94158f60 2023-10-06T05:23:16.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e98ce3700 1 -- 172.21.15.45:0/2231387434 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000bc00 con 0x7f4e94158f60 2023-10-06T05:23:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 -- 172.21.15.45:0/2231387434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 msgr2=0x7f4e94159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/2231387434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 0x7f4e94159330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4e900099b0 tx=0x7f4e90009250 comp rx=0 tx=0).stop 2023-10-06T05:23:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 -- 172.21.15.45:0/2231387434 shutdown_connections 2023-10-06T05:23:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/2231387434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 0x7f4e94159330 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/2231387434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e940cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 -- 172.21.15.45:0/2231387434 >> 172.21.15.45:0/2231387434 conn(0x7f4e940cdf50 msgr2=0x7f4e940ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 -- 172.21.15.45:0/2231387434 shutdown_connections 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.887+0000 7f4e9bf49700 1 -- 172.21.15.45:0/2231387434 wait complete. 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 Processor -- start 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 -- start start 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e94157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e940cc390 0x7f4e9416c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 0x7f4e941578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9416bc30 con 0x7f4e940cc390 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940c9a00 con 0x7f4e94158f60 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e9bf49700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e941576a0 con 0x7f4e9406a060 2023-10-06T05:23:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e994e4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e940cc390 0x7f4e9416c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.889+0000 7f4e994e4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e940cc390 0x7f4e9416c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35268/0 (socket says 172.21.15.45:35268) 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e994e4700 1 -- 172.21.15.45:0/3694275725 learned_addr learned my addr 172.21.15.45:0/3694275725 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e99ce5700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e94157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e9a4e6700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 0x7f4e941578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e994e4700 1 -- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 msgr2=0x7f4e94157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e994e4700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e94157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e994e4700 1 -- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 msgr2=0x7f4e941578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e994e4700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 0x7f4e941578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e994e4700 1 -- 172.21.15.45:0/3694275725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e900094f0 con 0x7f4e940cc390 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e994e4700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e940cc390 0x7f4e9416c0c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400ece0 tx=0x7f4e840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e89ffb700 1 -- 172.21.15.45:0/3694275725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400c850 con 0x7f4e940cc390 2023-10-06T05:23:16.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e89ffb700 1 -- 172.21.15.45:0/3694275725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8400c9b0 con 0x7f4e940cc390 2023-10-06T05:23:16.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e940c9c70 con 0x7f4e940cc390 2023-10-06T05:23:16.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.890+0000 7f4e89ffb700 1 -- 172.21.15.45:0/3694275725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400cd50 con 0x7f4e940cc390 2023-10-06T05:23:16.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.891+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9416b8a0 con 0x7f4e940cc390 2023-10-06T05:23:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.892+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9416bde0 con 0x7f4e940cc390 2023-10-06T05:23:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.892+0000 7f4e89ffb700 1 -- 172.21.15.45:0/3694275725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e84013680 con 0x7f4e940cc390 2023-10-06T05:23:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.892+0000 7f4e89ffb700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.892+0000 7f4e89ffb700 1 -- 172.21.15.45:0/3694275725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f4e84084db0 con 0x7f4e940cc390 2023-10-06T05:23:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.893+0000 7f4e99ce5700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.893+0000 7f4e99ce5700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e78066f30 0x7f4e780693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4e900099b0 tx=0x7f4e90017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:16.897+0000 7f4e89ffb700 1 -- 172.21.15.45:0/3694275725 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e9416bde0 con 0x7f4e940cc390 2023-10-06T05:23:17.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.145+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e9416bde0 con 0x7f4e940cc390 2023-10-06T05:23:17.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.146+0000 7f4e89ffb700 1 -- 172.21.15.45:0/3694275725 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 147 v147) v1 ==== 95+0+4599 (secure 0 0 0) 0x7f4e9416bde0 con 0x7f4e940cc390 2023-10-06T05:23:17.149 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:17.149 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":147,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3141743890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3141743890},{"type":"v1","addr":"172.21.15.45:6835","nonce":3141743890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:15.870458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:17.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e78066f30 msgr2=0x7f4e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e78066f30 0x7f4e780693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4e900099b0 tx=0x7f4e90017000 comp rx=0 tx=0).stop 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e940cc390 msgr2=0x7f4e9416c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e940cc390 0x7f4e9416c0c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400ece0 tx=0x7f4e840091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 shutdown_connections 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e94158f60 0x7f4e941578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e9406a060 0x7f4e94157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 --2- 172.21.15.45:0/3694275725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e940cc390 0x7f4e9416c0c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 >> 172.21.15.45:0/3694275725 conn(0x7f4e940cdf50 msgr2=0x7f4e9415f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 shutdown_connections 2023-10-06T05:23:17.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.148+0000 7f4e9bf49700 1 -- 172.21.15.45:0/3694275725 wait complete. 2023-10-06T05:23:17.153 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 147 2023-10-06T05:23:17.164 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2023-10-06T05:23:17.165 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2023-10-06T05:23:17.165 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:17.165 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:17.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:17.166 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:17.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:23:17.186 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:23:17.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.360+0000 7f6a0121e700 1 Processor -- start 2023-10-06T05:23:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.360+0000 7f6a0121e700 1 -- start start 2023-10-06T05:23:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f6a0121e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69fc155cd0 0x7f69fc15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f6a0121e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 0x7f69fc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f6a0121e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc0cd740 con 0x7f69fc1560a0 2023-10-06T05:23:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f6a0121e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc0ae6b0 con 0x7f69fc155cd0 2023-10-06T05:23:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f6a0121e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0ae8d0 con 0x7f69fc0ccdc0 2023-10-06T05:23:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69fa59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f69fc1560a0 0x7f69fc0adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52088/0 (socket says 172.21.15.45:52088) 2023-10-06T05:23:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69fa59c700 1 -- 172.21.15.45:0/2767449681 learned_addr learned my addr 172.21.15.45:0/2767449681 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69fb59e700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 0x7f69fc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69fad9d700 1 --2- 172.21.15.45:0/2767449681 >> v2:172.21.15.144:3300/0 conn(0x7f69fc155cd0 0x7f69fc15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875509128 0 0) 0x7f69fc0cd740 con 0x7f69fc1560a0 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e8003570 con 0x7f69fc1560a0 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3030817181 0 0) 0x7f69e8003570 con 0x7f69fc1560a0 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 msgr2=0x7f69fc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 0x7f69fc0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 >> v2:172.21.15.144:3300/0 conn(0x7f69fc155cd0 msgr2=0x7f69fc15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 --2- 172.21.15.45:0/2767449681 >> v2:172.21.15.144:3300/0 conn(0x7f69fc155cd0 0x7f69fc15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0aebb0 con 0x7f69fc1560a0 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.361+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f69e4001470 con 0x7f69fc1560a0 2023-10-06T05:23:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 >> v1:172.21.15.45:6789/0 conn(0x7f69fc1560a0 legacy=0x7f69fc0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69e8003a60 0x7f69e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e8004810 0x7f69e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 0x7f69e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0ae6b0 con 0x7f69e8003a60 2023-10-06T05:23:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0ae8d0 con 0x7f69e8004810 2023-10-06T05:23:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e800ba60 con 0x7f69fc0ccdc0 2023-10-06T05:23:17.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fad9d700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69e8003a60 0x7f69e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fa59c700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e8004810 0x7f69e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fb59e700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 0x7f69e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fad9d700 1 -- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 msgr2=0x7f69e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fad9d700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 0x7f69e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fad9d700 1 -- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e8004810 msgr2=0x7f69e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fad9d700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e8004810 0x7f69e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fad9d700 1 -- 172.21.15.45:0/2767449681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e8003570 con 0x7f69e8003a60 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fb59e700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc0ccdc0 0x7f69e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:17.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69fad9d700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69e8003a60 0x7f69e8003e70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009c30 tx=0x7f69ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc086cb8ea158aaa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec00e710 con 0x7f69e8003a60 2023-10-06T05:23:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec00e870 con 0x7f69e8003a60 2023-10-06T05:23:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.362+0000 7f69f9d9b700 1 -- 172.21.15.45:0/2767449681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec00ec30 con 0x7f69e8003a60 2023-10-06T05:23:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.363+0000 7f6a0121e700 1 -- 172.21.15.45:0/2767449681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69e8003a60 msgr2=0x7f69e8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.363+0000 7f6a0121e700 1 --2- 172.21.15.45:0/2767449681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69e8003a60 0x7f69e8003e70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009c30 tx=0x7f69ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.363+0000 7f69fb59e700 1 -- 172.21.15.45:0/2767449681 reap_dead start 2023-10-06T05:23:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.363+0000 7f6a0121e700 1 -- 172.21.15.45:0/2767449681 shutdown_connections 2023-10-06T05:23:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.363+0000 7f6a0121e700 1 -- 172.21.15.45:0/2767449681 >> 172.21.15.45:0/2767449681 conn(0x7f69fc06f090 msgr2=0x7f69fc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.363+0000 7f6a0121e700 1 -- 172.21.15.45:0/2767449681 shutdown_connections 2023-10-06T05:23:17.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.363+0000 7f6a0121e700 1 -- 172.21.15.45:0/2767449681 wait complete. 2023-10-06T05:23:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.365+0000 7f6a0121e700 1 Processor -- start 2023-10-06T05:23:17.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.365+0000 7f6a0121e700 1 -- start start 2023-10-06T05:23:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f6a0121e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc150170 0x7f69fc067620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f6a0121e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69fc0b1a20 0x7f69fc1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f6a0121e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69fc151710 0x7f69fc0afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f6a0121e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc151b60 con 0x7f69fc151710 2023-10-06T05:23:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f6a0121e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc067c90 con 0x7f69fc0b1a20 2023-10-06T05:23:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f6a0121e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc067f70 con 0x7f69fc150170 2023-10-06T05:23:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69fc151710 0x7f69fc0afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69fc151710 0x7f69fc0afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35288/0 (socket says 172.21.15.45:35288) 2023-10-06T05:23:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 -- 172.21.15.45:0/3347687312 learned_addr learned my addr 172.21.15.45:0/3347687312 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fa59c700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69fc0b1a20 0x7f69fc1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fad9d700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc150170 0x7f69fc067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 -- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc150170 msgr2=0x7f69fc067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc150170 0x7f69fc067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 -- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69fc0b1a20 msgr2=0x7f69fc1511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69fc0b1a20 0x7f69fc1511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.366+0000 7f69fb59e700 1 -- 172.21.15.45:0/3347687312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc068250 con 0x7f69fc151710 2023-10-06T05:23:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.367+0000 7f69fb59e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69fc151710 0x7f69fc0afbe0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0af930 tx=0x7f69f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.367+0000 7f69f2ffd700 1 -- 172.21.15.45:0/3347687312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400e050 con 0x7f69fc151710 2023-10-06T05:23:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.367+0000 7f69f2ffd700 1 -- 172.21.15.45:0/3347687312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f400f040 con 0x7f69fc151710 2023-10-06T05:23:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.367+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc068530 con 0x7f69fc151710 2023-10-06T05:23:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.367+0000 7f69f2ffd700 1 -- 172.21.15.45:0/3347687312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4010d40 con 0x7f69fc151710 2023-10-06T05:23:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.367+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc0b1e90 con 0x7f69fc151710 2023-10-06T05:23:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.368+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc0c5570 con 0x7f69fc151710 2023-10-06T05:23:17.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.368+0000 7f69f2ffd700 1 -- 172.21.15.45:0/3347687312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69f4003020 con 0x7f69fc151710 2023-10-06T05:23:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.369+0000 7f69f2ffd700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69dc066f20 0x7f69dc0693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.369+0000 7f69f2ffd700 1 -- 172.21.15.45:0/3347687312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f69f4084de0 con 0x7f69fc151710 2023-10-06T05:23:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.369+0000 7f69fad9d700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69dc066f20 0x7f69dc0693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.369+0000 7f69fad9d700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69dc066f20 0x7f69dc0693d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0af980 tx=0x7f69ec01a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.373+0000 7f69f2ffd700 1 -- 172.21.15.45:0/3347687312 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69fc0c5570 con 0x7f69fc151710 2023-10-06T05:23:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:17.625+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f69fc0c5950 con 0x7f69fc151710 2023-10-06T05:23:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.084+0000 7f69f2ffd700 1 -- 172.21.15.45:0/3347687312 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v148) v1 ==== 161+0+0 (secure 0 0 0) 0x7f69fc0c5950 con 0x7f69fc151710 2023-10-06T05:23:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69dc066f20 msgr2=0x7f69dc0693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69dc066f20 0x7f69dc0693d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0af980 tx=0x7f69ec01a000 comp rx=0 tx=0).stop 2023-10-06T05:23:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69fc151710 msgr2=0x7f69fc0afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69fc151710 0x7f69fc0afbe0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0af930 tx=0x7f69f400b040 comp rx=0 tx=0).stop 2023-10-06T05:23:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 shutdown_connections 2023-10-06T05:23:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f69dc066f20 0x7f69dc0693d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69fc151710 0x7f69fc0afbe0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69fc150170 0x7f69fc067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 --2- 172.21.15.45:0/3347687312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69fc0b1a20 0x7f69fc1511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 >> 172.21.15.45:0/3347687312 conn(0x7f69fc06f090 msgr2=0x7f69fc1541f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 shutdown_connections 2023-10-06T05:23:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.087+0000 7f6a0121e700 1 -- 172.21.15.45:0/3347687312 wait complete. 2023-10-06T05:23:18.092 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:18.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.239+0000 7fa5a44d4700 1 Processor -- start 2023-10-06T05:23:18.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.239+0000 7fa5a44d4700 1 -- start start 2023-10-06T05:23:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a44d4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a44d4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c0b2120 con 0x7fa59c06add0 2023-10-06T05:23:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a44d4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c0ab7d0 con 0x7fa59c15a490 2023-10-06T05:23:18.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a44d4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c0ae020 con 0x7fa59c067830 2023-10-06T05:23:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a2a71700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa59c06add0 0x7fa59c0abc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52104/0 (socket says 172.21.15.45:52104) 2023-10-06T05:23:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a2a71700 1 -- 172.21.15.45:0/1821392375 learned_addr learned my addr 172.21.15.45:0/1821392375 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a1a6f700 1 --2- 172.21.15.45:0/1821392375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a126e700 1 -- 172.21.15.45:0/1821392375 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656969141 0 0) 0x7fa594001490 con 0x7fa59c06add0 2023-10-06T05:23:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.240+0000 7fa5a126e700 1 -- 172.21.15.45:0/1821392375 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa588003570 con 0x7fa59c06add0 2023-10-06T05:23:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a126e700 1 -- 172.21.15.45:0/1821392375 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507508314 0 0) 0x7fa59c0ab7d0 con 0x7fa59c15a490 2023-10-06T05:23:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a126e700 1 -- 172.21.15.45:0/1821392375 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa594001490 con 0x7fa59c15a490 2023-10-06T05:23:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a1a6f700 1 -- 172.21.15.45:0/1821392375 >> v1:172.21.15.144:6789/0 conn(0x7fa59c15a490 legacy=0x7fa59c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a1a6f700 1 -- 172.21.15.45:0/1821392375 >> v1:172.21.15.45:6789/0 conn(0x7fa59c06add0 legacy=0x7fa59c0abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a1a6f700 1 -- 172.21.15.45:0/1821392375 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c067e80 con 0x7fa59c067830 2023-10-06T05:23:18.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a1a6f700 1 --2- 172.21.15.45:0/1821392375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c06a890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa590009910 tx=0x7fa5900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ee25c2ea2d1e45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a126e700 1 -- 172.21.15.45:0/1821392375 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590011630 con 0x7fa59c067830 2023-10-06T05:23:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a126e700 1 -- 172.21.15.45:0/1821392375 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa590011790 con 0x7fa59c067830 2023-10-06T05:23:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.241+0000 7fa5a126e700 1 -- 172.21.15.45:0/1821392375 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590011b30 con 0x7fa59c067830 2023-10-06T05:23:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.242+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1821392375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 msgr2=0x7fa59c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.242+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1821392375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c06a890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa590009910 tx=0x7fa5900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.242+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1821392375 shutdown_connections 2023-10-06T05:23:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.242+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1821392375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c06a890 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.242+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1821392375 >> 172.21.15.45:0/1821392375 conn(0x7fa59c0cc5a0 msgr2=0x7fa59c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.242+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1821392375 shutdown_connections 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.242+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1821392375 wait complete. 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.243+0000 7fa5a44d4700 1 Processor -- start 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.243+0000 7fa5a44d4700 1 -- start start 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a44d4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c0ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a44d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa59c06add0 0x7fa59c166ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a44d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59c15a490 0x7fa59c15b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a44d4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c067ca0 con 0x7fa59c15a490 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a44d4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1697f0 con 0x7fa59c06add0 2023-10-06T05:23:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a44d4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c16a040 con 0x7fa59c067830 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59c15a490 0x7fa59c15b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59c15a490 0x7fa59c15b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35298/0 (socket says 172.21.15.45:35298) 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 -- 172.21.15.45:0/1191861777 learned_addr learned my addr 172.21.15.45:0/1191861777 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2270700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c0ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a1a6f700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa59c06add0 0x7fa59c166ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 -- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 msgr2=0x7fa59c0ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c0ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 -- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa59c06add0 msgr2=0x7fa59c166ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa59c06add0 0x7fa59c166ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.244+0000 7fa5a2a71700 1 -- 172.21.15.45:0/1191861777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa590009420 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.245+0000 7fa5a2a71700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59c15a490 0x7fa59c15b830 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa594002070 tx=0x7fa594001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.245+0000 7fa58e7fc700 1 -- 172.21.15.45:0/1191861777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59400aac0 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.245+0000 7fa58e7fc700 1 -- 172.21.15.45:0/1191861777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59400ac40 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.245+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c0a8df0 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.245+0000 7fa58e7fc700 1 -- 172.21.15.45:0/1191861777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5940070f0 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.245+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c169cd0 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.246+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c0cc0e0 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.246+0000 7fa58e7fc700 1 -- 172.21.15.45:0/1191861777 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5940074a0 con 0x7fa59c15a490 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.247+0000 7fa58e7fc700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.247+0000 7fa58e7fc700 1 -- 172.21.15.45:0/1191861777 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7fa594084f10 con 0x7fa59c15a490 2023-10-06T05:23:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.247+0000 7fa5a2270700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.248+0000 7fa5a2270700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa598007fd0 tx=0x7fa598007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.251+0000 7fa58e7fc700 1 -- 172.21.15.45:0/1191861777 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5940555f0 con 0x7fa59c15a490 2023-10-06T05:23:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:18.539+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fa59c0b2120 con 0x7fa59c15a490 2023-10-06T05:23:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.090+0000 7fa58e7fc700 1 -- 172.21.15.45:0/1191861777 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v149) v1 ==== 101+0+0 (secure 0 0 0) 0x7fa59c0b2120 con 0x7fa59c15a490 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.092+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 msgr2=0x7fa5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa598007fd0 tx=0x7fa598007d50 comp rx=0 tx=0).stop 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59c15a490 msgr2=0x7fa59c15b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59c15a490 0x7fa59c15b830 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa594002070 tx=0x7fa594001c70 comp rx=0 tx=0).stop 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 shutdown_connections 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59c15a490 0x7fa59c15b830 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa59c067830 0x7fa59c0ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 --2- 172.21.15.45:0/1191861777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa59c06add0 0x7fa59c166ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 >> 172.21.15.45:0/1191861777 conn(0x7fa59c0cc5a0 msgr2=0x7fa59c0b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 shutdown_connections 2023-10-06T05:23:19.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.093+0000 7fa5a44d4700 1 -- 172.21.15.45:0/1191861777 wait complete. 2023-10-06T05:23:19.097 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:23:19.111 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.156+0000 7f4b21fe8040 1 Processor -- start 2023-10-06T05:23:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.156+0000 7f4b21fe8040 1 -- start start 2023-10-06T05:23:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b21fe8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x5589256830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b21fe8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 0x558925718d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b21fe8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 0x5589257192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b21fe8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892558a980 con 0x55892566b2f0 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b21fe8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892558a620 con 0x55892566f450 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b21fe8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558925581400 con 0x558925715fc0 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x5589256830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x5589256830c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35314/0 (socket says 172.21.15.45:35314) 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 -- 172.21.15.45:0/3952226928 learned_addr learned my addr 172.21.15.45:0/3952226928 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b12a05700 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 0x558925718d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 0x5589257192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 -- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 msgr2=0x5589257192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 0x5589257192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 -- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 msgr2=0x558925718d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 0x558925718d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 -- 172.21.15.45:0/3952226928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1400a180 con 0x55892566b2f0 2023-10-06T05:23:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1b206700 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x5589256830c0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b260 tx=0x7f4b0c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e5fd07e2f8a4b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1a204700 1 -- 172.21.15.45:0/3952226928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c014070 con 0x55892566b2f0 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1a204700 1 -- 172.21.15.45:0/3952226928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c00f450 con 0x55892566b2f0 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.157+0000 7f4b1a204700 1 -- 172.21.15.45:0/3952226928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00f810 con 0x55892566b2f0 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 -- 172.21.15.45:0/3952226928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 msgr2=0x5589256830c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x5589256830c0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00b260 tx=0x7f4b0c0017b0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 -- 172.21.15.45:0/3952226928 shutdown_connections 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 0x5589257192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x5589256830c0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/3952226928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 0x558925718d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 -- 172.21.15.45:0/3952226928 >> 172.21.15.45:0/3952226928 conn(0x55892563fbd0 msgr2=0x558925641fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 -- 172.21.15.45:0/3952226928 shutdown_connections 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.158+0000 7f4b21fe8040 1 -- 172.21.15.45:0/3952226928 wait complete. 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.159+0000 7f4b21fe8040 1 Processor -- start 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.159+0000 7f4b21fe8040 1 -- start start 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b21fe8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x55892570eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b21fe8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 0x55892571f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b21fe8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 0x55892571f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b21fe8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558925719700 con 0x55892566b2f0 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b21fe8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558925710620 con 0x55892566f450 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b21fe8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589257108a0 con 0x558925715fc0 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x55892570eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x55892570eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35322/0 (socket says 172.21.15.45:35322) 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 learned_addr learned my addr 172.21.15.45:0/2115800876 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b12a05700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 0x55892571f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 0x55892571f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 msgr2=0x55892571f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558925715fc0 0x55892571f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 msgr2=0x55892571f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55892566f450 0x55892571f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c001a90 con 0x55892566b2f0 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b1b206700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x55892570eff0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0017f0 tx=0x7f4b0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.160+0000 7f4b18a01700 1 -- 172.21.15.45:0/2115800876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c001240 con 0x55892566b2f0 2023-10-06T05:23:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.161+0000 7f4b18a01700 1 -- 172.21.15.45:0/2115800876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c01ca20 con 0x55892566b2f0 2023-10-06T05:23:19.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.161+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5589255993d0 con 0x55892566b2f0 2023-10-06T05:23:19.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.161+0000 7f4b18a01700 1 -- 172.21.15.45:0/2115800876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c01cbd0 con 0x55892566b2f0 2023-10-06T05:23:19.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.161+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55892558cae0 con 0x55892566b2f0 2023-10-06T05:23:19.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.162+0000 7f4b18a01700 1 -- 172.21.15.45:0/2115800876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b0c023020 con 0x55892566b2f0 2023-10-06T05:23:19.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.162+0000 7f4b18a01700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4afc065890 0x7f4afc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.163+0000 7f4b18a01700 1 -- 172.21.15.45:0/2115800876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f4b0c008030 con 0x55892566b2f0 2023-10-06T05:23:19.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.163+0000 7f4b12a05700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4afc065890 0x7f4afc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.170 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:19.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.163+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589257178a0 0x5589257219d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.163+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925721f10 con 0x5589257178a0 2023-10-06T05:23:19.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.163+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589257178a0 0x5589257219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.163+0000 7f4b12a05700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4afc065890 0x7f4afc067d40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009c00 tx=0x7f4b0400c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.163+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589257178a0 0x5589257219d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.165+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v102'288 uv262 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.165+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257329b0 con 0x5589257178a0 2023-10-06T05:23:19.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.166+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v102'289 uv289 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.166+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925732d60 con 0x5589257178a0 2023-10-06T05:23:19.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.168+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v102'290 uv290 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.172 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.168+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925734120 con 0x5589257178a0 2023-10-06T05:23:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.169+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.169+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257344d0 con 0x5589257178a0 2023-10-06T05:23:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.170+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.170+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733110 con 0x5589257178a0 2023-10-06T05:23:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.172+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'293 uv293 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.172+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257334c0 con 0x5589257178a0 2023-10-06T05:23:19.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.173+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v102'294 uv268 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.173+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.174+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v102'295 uv295 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.174+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.175+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v102'296 uv296 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.178 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:19.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.175+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.176+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'297 uv297 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.176+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.177+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'298 uv298 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.177+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.179+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'299 uv299 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.179+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.180+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v102'300 uv275 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.180+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.181+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'301 uv301 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.181+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.182+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v102'302 uv302 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.185 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:19.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.182+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.183+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'303 uv303 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.183+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.184+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'304 uv304 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.184+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.185+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'305 uv305 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.185+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.187+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'306 uv306 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.187+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.190+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v102'307 uv281 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.190+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.191+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v102'308 uv308 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.191+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.195 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.192+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v102'309 uv309 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.192+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.194+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v102'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.194+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.195+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v102'311 uv311 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.195+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.196+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v102'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.196+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.198+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v102'313 uv287 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.198+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.199+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v102'314 uv314 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.199+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.203 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.200+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v102'315 uv315 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.200+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.201+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v102'316 uv316 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.202+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.203+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v102'317 uv317 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.203+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x5589257178a0 2023-10-06T05:23:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.204+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v102'318 uv318 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x5589257178a0 2023-10-06T05:23:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.204+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558925733c70 0x558925736da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.204+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925733870 con 0x558925733c70 2023-10-06T05:23:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.204+0000 7f4b1b206700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558925733c70 0x558925736da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.205+0000 7f4b1b206700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558925733c70 0x558925736da0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.206+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v102'216 uv199 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.206+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257372e0 con 0x558925733c70 2023-10-06T05:23:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.208+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v102'217 uv217 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.208+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925737690 con 0x558925733c70 2023-10-06T05:23:19.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.209+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v102'218 uv218 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.212 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:19.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.209+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925737f30 con 0x558925733c70 2023-10-06T05:23:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.210+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v102'219 uv219 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.210+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257382e0 con 0x558925733c70 2023-10-06T05:23:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.212+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v102'220 uv220 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.212+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738690 con 0x558925733c70 2023-10-06T05:23:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.213+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v102'221 uv204 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.213+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738a40 con 0x558925733c70 2023-10-06T05:23:19.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.214+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v102'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.214+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.215+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v102'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.218 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:19.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.215+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.217+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v102'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.217+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.218+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v102'225 uv225 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.218+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.219+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v102'226 uv209 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.219+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.220+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v102'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.220+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.221+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v102'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.224 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:19.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.221+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573c510 con 0x558925733c70 2023-10-06T05:23:19.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.223+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v102'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.223+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573c510 con 0x558925733c70 2023-10-06T05:23:19.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.224+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v102'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.224+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.225+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v102'231 uv215 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.225+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.226+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v102'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.226+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.228+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v102'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.230 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.228+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925739e00 con 0x558925733c70 2023-10-06T05:23:19.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.229+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v102'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.229+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925739e00 con 0x558925733c70 2023-10-06T05:23:19.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.230+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v102'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.230+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925738df0 con 0x558925733c70 2023-10-06T05:23:19.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.231+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v102'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.231+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558925739300 0x55892573b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.232+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573bc20 con 0x558925739300 2023-10-06T05:23:19.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.232+0000 7f4b12a05700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558925739300 0x55892573b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.232+0000 7f4b12a05700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558925739300 0x55892573b6e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.233+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.234+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573bfd0 con 0x558925739300 2023-10-06T05:23:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.235+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v102'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.235+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573c3c0 con 0x558925739300 2023-10-06T05:23:19.238 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.236+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v102'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.236+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573dcf0 con 0x558925739300 2023-10-06T05:23:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.237+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.237+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573e0a0 con 0x558925739300 2023-10-06T05:23:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.238+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.238+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573cce0 con 0x558925739300 2023-10-06T05:23:19.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.239+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.239+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d0e0 con 0x558925739300 2023-10-06T05:23:19.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.240+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'43 uv42 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.240+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d510 con 0x558925739300 2023-10-06T05:23:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.242+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v102'44 uv44 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.242+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d940 con 0x558925739300 2023-10-06T05:23:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.243+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v102'45 uv45 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.246 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.243+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d510 con 0x558925739300 2023-10-06T05:23:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.244+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v102'46 uv46 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.244+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d4e0 con 0x558925733c70 2023-10-06T05:23:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.246+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.246+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d4e0 con 0x558925733c70 2023-10-06T05:23:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.247+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v102'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.247+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d4c0 con 0x558925733c70 2023-10-06T05:23:19.251 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.249+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v102'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.249+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d490 con 0x558925733c70 2023-10-06T05:23:19.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.250+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.250+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d920 con 0x558925733c70 2023-10-06T05:23:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.251+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.251+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892573d920 con 0x558925733c70 2023-10-06T05:23:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.253+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x558925733c70 2023-10-06T05:23:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.253+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55892573d8d0 0x558925740970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.253+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.253+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55892573d8d0 0x558925740970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.253+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55892573d8d0 0x558925740970 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.254+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v102'105 uv99 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.254+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.256+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v102'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.256+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.259 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:19.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.258+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v102'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.258+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925741ec0 con 0x55892573d8d0 2023-10-06T05:23:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.258+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v102'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.258+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925741ec0 con 0x55892573d8d0 2023-10-06T05:23:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.260+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v102'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.260+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.261+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v102'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.261+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.262+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v102'111 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.262+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.263+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v102'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.263+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.266 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.265+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v102'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.265+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925740eb0 con 0x55892573d8d0 2023-10-06T05:23:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.266+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v102'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.266+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257412b0 con 0x55892573d8d0 2023-10-06T05:23:19.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.267+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v102'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.267+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589257417c0 0x558925741bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.267+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257442c0 con 0x5589257417c0 2023-10-06T05:23:19.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.267+0000 7f4b1b206700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589257417c0 0x558925741bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.267+0000 7f4b1b206700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589257417c0 0x558925741bb0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.268+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v102'46 uv45 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x5589257417c0 2023-10-06T05:23:19.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.268+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257442c0 con 0x5589257417c0 2023-10-06T05:23:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.269+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v102'47 uv47 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x5589257417c0 2023-10-06T05:23:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.269+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257442c0 con 0x5589257417c0 2023-10-06T05:23:19.274 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.273+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v102'48 uv48 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x5589257417c0 2023-10-06T05:23:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.273+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257442c0 con 0x5589257417c0 2023-10-06T05:23:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.275+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v102'49 uv49 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x5589257417c0 2023-10-06T05:23:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.275+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257446c0 con 0x5589257417c0 2023-10-06T05:23:19.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.278+0000 7f4b1b206700 1 -- 172.21.15.45:0/2115800876 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v102'50 uv50 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b0c016070 con 0x5589257417c0 2023-10-06T05:23:19.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.278+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257446c0 con 0x558925739300 2023-10-06T05:23:19.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.279+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.279+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257446c0 con 0x558925739300 2023-10-06T05:23:19.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.280+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v102'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.280+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257446c0 con 0x558925739300 2023-10-06T05:23:19.284 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.282+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v102'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.282+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257456d0 con 0x558925739300 2023-10-06T05:23:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.283+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.283+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257456d0 con 0x558925739300 2023-10-06T05:23:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.284+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.284+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x5589257446c0 con 0x558925739300 2023-10-06T05:23:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.286+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.286+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558925744bd0 0x558925746fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.286+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747540 con 0x558925744bd0 2023-10-06T05:23:19.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.286+0000 7f4b12a05700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558925744bd0 0x558925746fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.286+0000 7f4b12a05700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558925744bd0 0x558925746fb0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.290 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:19.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.288+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v102'95 uv95 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.288+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747920 con 0x558925744bd0 2023-10-06T05:23:19.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.289+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [write 0~34] v102'96 uv96 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.289+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747920 con 0x558925744bd0 2023-10-06T05:23:19.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.290+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v102'97 uv97 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.290+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747920 con 0x558925744bd0 2023-10-06T05:23:19.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.291+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 100.00000000 [create] v102'98 uv93 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.291+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747920 con 0x558925744bd0 2023-10-06T05:23:19.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.293+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 100.00000000 [delete] v102'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.293+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747920 con 0x558925744bd0 2023-10-06T05:23:19.296 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:19.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.293+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v102'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.294+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747920 con 0x558925744bd0 2023-10-06T05:23:19.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.295+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [omap-set-header] v102'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.295+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574a690 con 0x558925744bd0 2023-10-06T05:23:19.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.296+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v102'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925744bd0 2023-10-06T05:23:19.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.296+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747e10 con 0x558925739300 2023-10-06T05:23:19.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.297+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v102'44 uv43 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.297+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747e10 con 0x558925739300 2023-10-06T05:23:19.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.298+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v102'45 uv45 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.298+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747e10 con 0x558925739300 2023-10-06T05:23:19.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.300+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v102'46 uv46 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.302 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:19.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.300+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x558925747e10 con 0x558925739300 2023-10-06T05:23:19.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.301+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v102'47 uv47 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.301+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558925748320 0x558925748730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.301+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925748320 2023-10-06T05:23:19.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.301+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558925748320 0x558925748730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.302+0000 7f4b1ba07700 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558925748320 0x558925748730 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.303+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x558925748320 2023-10-06T05:23:19.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.303+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925748320 2023-10-06T05:23:19.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.306+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v102'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x558925748320 2023-10-06T05:23:19.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.306+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925748320 2023-10-06T05:23:19.314 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:19.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.312+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v102'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x558925748320 2023-10-06T05:23:19.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.312+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574bdf0 con 0x558925748320 2023-10-06T05:23:19.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.314+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x558925748320 2023-10-06T05:23:19.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.314+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574bdf0 con 0x558925748320 2023-10-06T05:23:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.315+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x558925748320 2023-10-06T05:23:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.315+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925748320 2023-10-06T05:23:19.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.317+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x558925748320 2023-10-06T05:23:19.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.317+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x55892573d8d0 2023-10-06T05:23:19.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.318+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v102'47 uv46 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.318+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x55892573d8d0 2023-10-06T05:23:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.319+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v102'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.319+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x55892573d8d0 2023-10-06T05:23:19.323 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.320+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v102'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.320+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x55892573d8d0 2023-10-06T05:23:19.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.322+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v102'50 uv50 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.322+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x55892573d8d0 2023-10-06T05:23:19.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.323+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v102'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b1400e7e0 con 0x55892573d8d0 2023-10-06T05:23:19.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.323+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ae00 con 0x558925739300 2023-10-06T05:23:19.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.325+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v102'119 uv111 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.325+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ae00 con 0x558925739300 2023-10-06T05:23:19.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.326+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v102'120 uv120 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.326+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ae00 con 0x558925739300 2023-10-06T05:23:19.330 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.327+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v102'121 uv121 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.327+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ae00 con 0x558925739300 2023-10-06T05:23:19.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.328+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v102'122 uv122 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.328+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574b250 con 0x558925739300 2023-10-06T05:23:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.330+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v102'123 uv123 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.330+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574b250 con 0x558925739300 2023-10-06T05:23:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.331+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v102'124 uv118 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.331+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925739300 2023-10-06T05:23:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.332+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v102'125 uv125 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.332+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925739300 2023-10-06T05:23:19.335 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.333+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v102'126 uv126 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.333+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925739300 2023-10-06T05:23:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.334+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v102'127 uv127 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.334+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574ade0 con 0x558925739300 2023-10-06T05:23:19.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.335+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v102'128 uv128 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.335+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574b200 con 0x558925739300 2023-10-06T05:23:19.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.336+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v102'129 uv129 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.336+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) v8 -- 0x55892574b6f0 con 0x558925739300 2023-10-06T05:23:19.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.337+0000 7f4b12a05700 1 -- 172.21.15.45:0/2115800876 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v102'130 uv130 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b04007d10 con 0x558925739300 2023-10-06T05:23:19.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558925739300 msgr2=0x55892573b6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558925739300 0x55892573b6e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589257178a0 msgr2=0x5589257219d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589257178a0 0x5589257219d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558925744bd0 msgr2=0x558925746fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558925744bd0 0x558925746fb0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558925733c70 msgr2=0x558925736da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558925733c70 0x558925736da0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558925748320 msgr2=0x558925748730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558925748320 0x558925748730 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589257417c0 msgr2=0x558925741bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589257417c0 0x558925741bb0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55892573d8d0 msgr2=0x558925740970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55892573d8d0 0x558925740970 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 reap_dead start 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 reap_dead start 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b1ba07700 1 -- 172.21.15.45:0/2115800876 reap_dead start 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4afc065890 msgr2=0x7f4afc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4afc065890 0x7f4afc067d40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009c00 tx=0x7f4b0400c040 comp rx=0 tx=0).stop 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 msgr2=0x55892570eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x55892570eff0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0017f0 tx=0x7f4b0c016000 comp rx=0 tx=0).stop 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 shutdown_connections 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55892566b2f0 0x55892570eff0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 --2- 172.21.15.45:0/2115800876 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4afc065890 0x7f4afc067d40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 >> 172.21.15.45:0/2115800876 conn(0x55892563fbd0 msgr2=0x55892566b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 shutdown_connections 2023-10-06T05:23:19.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.338+0000 7f4b21fe8040 1 -- 172.21.15.45:0/2115800876 wait complete. 2023-10-06T05:23:19.345 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2023-10-06T05:23:19.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.389+0000 7f6173ab6040 1 Processor -- start 2023-10-06T05:23:19.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.389+0000 7f6173ab6040 1 -- start start 2023-10-06T05:23:19.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6173ab6040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026401b0 0x55e102687800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6173ab6040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e1026855f0 0x55e102642db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6173ab6040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e102701970 0x55e1026432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6173ab6040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10258c980 con 0x55e1026855f0 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6173ab6040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10258c620 con 0x55e1026401b0 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6173ab6040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e102583400 con 0x55e102701970 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6167fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e1026855f0 0x55e102642db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f616ccd4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026401b0 0x55e102687800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6167fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e1026855f0 0x55e102642db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35328/0 (socket says 172.21.15.45:35328) 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f6167fff700 1 -- 172.21.15.45:0/4085400226 learned_addr learned my addr 172.21.15.45:0/4085400226 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.390+0000 7f616d4d5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e102701970 0x55e1026432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f616ccd4700 1 -- 172.21.15.45:0/4085400226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e102701970 msgr2=0x55e1026432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:19.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f616ccd4700 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e102701970 0x55e1026432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f616ccd4700 1 -- 172.21.15.45:0/4085400226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e1026855f0 msgr2=0x55e102642db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f616ccd4700 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e1026855f0 0x55e102642db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f616ccd4700 1 -- 172.21.15.45:0/4085400226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160005db0 con 0x55e1026401b0 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f6167fff700 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e1026855f0 0x55e102642db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f616ccd4700 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026401b0 0x55e102687800 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f61600098c0 tx=0x7f616000c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a3651d4df9509b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f6166ffd700 1 -- 172.21.15.45:0/4085400226 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000f040 con 0x55e1026401b0 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f6166ffd700 1 -- 172.21.15.45:0/4085400226 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160008940 con 0x55e1026401b0 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.391+0000 7f6166ffd700 1 -- 172.21.15.45:0/4085400226 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160008cc0 con 0x55e1026401b0 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 -- 172.21.15.45:0/4085400226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026401b0 msgr2=0x55e102687800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026401b0 0x55e102687800 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f61600098c0 tx=0x7f616000c040 comp rx=0 tx=0).stop 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 -- 172.21.15.45:0/4085400226 shutdown_connections 2023-10-06T05:23:19.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e102701970 0x55e1026432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026401b0 0x55e102687800 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4085400226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e1026855f0 0x55e102642db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 -- 172.21.15.45:0/4085400226 >> 172.21.15.45:0/4085400226 conn(0x55e1026424a0 msgr2=0x55e10266bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 -- 172.21.15.45:0/4085400226 shutdown_connections 2023-10-06T05:23:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 -- 172.21.15.45:0/4085400226 wait complete. 2023-10-06T05:23:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 Processor -- start 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.392+0000 7f6173ab6040 1 -- start start 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f6173ab6040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e1026401b0 0x55e102671840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f6173ab6040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026855f0 0x55e102729000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f6173ab6040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e102701970 0x55e10266fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f6173ab6040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1026fbeb0 con 0x55e102701970 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f6173ab6040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e10266f260 con 0x55e1026855f0 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f6173ab6040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e102729540 con 0x55e1026401b0 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f616d4d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e102701970 0x55e10266fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f616d4d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e102701970 0x55e10266fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35342/0 (socket says 172.21.15.45:35342) 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f616d4d5700 1 -- 172.21.15.45:0/4284504116 learned_addr learned my addr 172.21.15.45:0/4284504116 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f6167fff700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026855f0 0x55e102729000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.393+0000 7f616ccd4700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e1026401b0 0x55e102671840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f616d4d5700 1 -- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e1026401b0 msgr2=0x55e102671840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f616d4d5700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e1026401b0 0x55e102671840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f616d4d5700 1 -- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026855f0 msgr2=0x55e102729000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f616d4d5700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1026855f0 0x55e102729000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f616d4d5700 1 -- 172.21.15.45:0/4284504116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615800a040 con 0x55e102701970 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f616ccd4700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e1026401b0 0x55e102671840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f616d4d5700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e102701970 0x55e10266fe70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x55e10266be70 tx=0x7f6168015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f61657fa700 1 -- 172.21.15.45:0/4284504116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168020070 con 0x55e102701970 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f61657fa700 1 -- 172.21.15.45:0/4284504116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800fa70 con 0x55e102701970 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61600094f0 con 0x55e102701970 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f61657fa700 1 -- 172.21.15.45:0/4284504116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680256b0 con 0x55e102701970 2023-10-06T05:23:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.394+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616800a0e0 con 0x55e102701970 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.396+0000 7f61657fa700 1 -- 172.21.15.45:0/4284504116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6168003040 con 0x55e102701970 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.396+0000 7f61657fa700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61500658c0 0x7f6150067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.396+0000 7f61657fa700 1 -- 172.21.15.45:0/4284504116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f6168005e40 con 0x55e102701970 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.396+0000 7f616ccd4700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61500658c0 0x7f6150067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.396+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e10271bfe0 0x55e102727f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.396+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) v8 -- 0x55e1026703b0 con 0x55e10271bfe0 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.397+0000 7f6167fff700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e10271bfe0 0x55e102727f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.397+0000 7f616ccd4700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61500658c0 0x7f6150067d70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6160008410 tx=0x7f6160017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.397+0000 7f6167fff700 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e10271bfe0 0x55e102727f80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6167fff700 1 -- 172.21.15.45:0/4284504116 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 100.00000000 [delete] v102'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f615800f920 con 0x55e10271bfe0 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e10271bfe0 msgr2=0x55e102727f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e10271bfe0 0x55e102727f80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61500658c0 msgr2=0x7f6150067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61500658c0 0x7f6150067d70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6160008410 tx=0x7f6160017000 comp rx=0 tx=0).stop 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e102701970 msgr2=0x55e10266fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 --2- 172.21.15.45:0/4284504116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e102701970 0x55e10266fe70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x55e10266be70 tx=0x7f6168015000 comp rx=0 tx=0).stop 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f616d4d5700 1 -- 172.21.15.45:0/4284504116 reap_dead start 2023-10-06T05:23:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 shutdown_connections 2023-10-06T05:23:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 >> 172.21.15.45:0/4284504116 conn(0x55e1026424a0 msgr2=0x55e10268b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 shutdown_connections 2023-10-06T05:23:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.398+0000 7f6173ab6040 1 -- 172.21.15.45:0/4284504116 wait complete. 2023-10-06T05:23:19.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:19.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.542+0000 7f0e295e1700 1 Processor -- start 2023-10-06T05:23:19.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.542+0000 7f0e295e1700 1 -- start start 2023-10-06T05:23:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.542+0000 7f0e295e1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d0680 0x7f0e2415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.542+0000 7f0e295e1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cd740 con 0x7f0e240d02b0 2023-10-06T05:23:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.542+0000 7f0e295e1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240ccb30 con 0x7f0e2415c6f0 2023-10-06T05:23:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.542+0000 7f0e295e1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240d0ad0 con 0x7f0e240d0680 2023-10-06T05:23:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e22ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0e240d02b0 0x7f0e2415d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52112/0 (socket says 172.21.15.45:52112) 2023-10-06T05:23:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e22ffd700 1 -- 172.21.15.45:0/1613325379 learned_addr learned my addr 172.21.15.45:0/1613325379 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e227fc700 1 --2- 172.21.15.45:0/1613325379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d0680 0x7f0e2415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e21ffb700 1 -- 172.21.15.45:0/1613325379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832933437 0 0) 0x7f0e240cd740 con 0x7f0e240d02b0 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e21ffb700 1 -- 172.21.15.45:0/1613325379 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c003570 con 0x7f0e240d02b0 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e21ffb700 1 -- 172.21.15.45:0/1613325379 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511952421 0 0) 0x7f0e240ccb30 con 0x7f0e2415c6f0 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e227fc700 1 -- 172.21.15.45:0/1613325379 >> v1:172.21.15.144:6789/0 conn(0x7f0e2415c6f0 legacy=0x7f0e240caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e227fc700 1 -- 172.21.15.45:0/1613325379 >> v1:172.21.15.45:6789/0 conn(0x7f0e240d02b0 legacy=0x7f0e2415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e227fc700 1 -- 172.21.15.45:0/1613325379 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240cce10 con 0x7f0e240d0680 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e227fc700 1 --2- 172.21.15.45:0/1613325379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d0680 0x7f0e2415c1b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009940 tx=0x7f0e180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c60d4fe611350a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.543+0000 7f0e21ffb700 1 -- 172.21.15.45:0/1613325379 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1800b670 con 0x7f0e240d0680 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e21ffb700 1 -- 172.21.15.45:0/1613325379 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1800b7d0 con 0x7f0e240d0680 2023-10-06T05:23:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e21ffb700 1 -- 172.21.15.45:0/1613325379 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1800bb90 con 0x7f0e240d0680 2023-10-06T05:23:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e295e1700 1 -- 172.21.15.45:0/1613325379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d0680 msgr2=0x7f0e2415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e295e1700 1 --2- 172.21.15.45:0/1613325379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d0680 0x7f0e2415c1b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009940 tx=0x7f0e180091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e295e1700 1 -- 172.21.15.45:0/1613325379 shutdown_connections 2023-10-06T05:23:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e295e1700 1 --2- 172.21.15.45:0/1613325379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d0680 0x7f0e2415c1b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e295e1700 1 -- 172.21.15.45:0/1613325379 >> 172.21.15.45:0/1613325379 conn(0x7f0e2406ed50 msgr2=0x7f0e2406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e295e1700 1 -- 172.21.15.45:0/1613325379 shutdown_connections 2023-10-06T05:23:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.544+0000 7f0e295e1700 1 -- 172.21.15.45:0/1613325379 wait complete. 2023-10-06T05:23:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.545+0000 7f0e295e1700 1 Processor -- start 2023-10-06T05:23:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e295e1700 1 -- start start 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e295e1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d02b0 0x7f0e2416a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e295e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e240d0680 0x7f0e240c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e295e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e2415c6f0 0x7f0e2416d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e295e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24169a20 con 0x7f0e2415c6f0 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e295e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24169fc0 con 0x7f0e240d0680 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e295e1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2416a990 con 0x7f0e240d02b0 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e22ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d02b0 0x7f0e2416a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.546+0000 7f0e237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e2415c6f0 0x7f0e2416d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d02b0 0x7f0e2416a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:43656/0 (socket says 172.21.15.45:43656) 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 -- 172.21.15.45:0/2633340198 learned_addr learned my addr 172.21.15.45:0/2633340198 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e227fc700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e240d0680 0x7f0e240c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 -- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e240d0680 msgr2=0x7f0e240c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e240d0680 0x7f0e240c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 -- 172.21.15.45:0/2633340198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e2415c6f0 msgr2=0x7f0e2416d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e2415c6f0 0x7f0e2416d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 -- 172.21.15.45:0/2633340198 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e18009420 con 0x7f0e240d02b0 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e22ffd700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d02b0 0x7f0e2416a450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0e140011e0 tx=0x7f0e14007df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e0affd700 1 -- 172.21.15.45:0/2633340198 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14005b30 con 0x7f0e240d02b0 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e0affd700 1 -- 172.21.15.45:0/2633340198 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14005c90 con 0x7f0e240d02b0 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e0affd700 1 -- 172.21.15.45:0/2633340198 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14007300 con 0x7f0e240d02b0 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2416da80 con 0x7f0e240d02b0 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.547+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e24169c50 con 0x7f0e240d02b0 2023-10-06T05:23:19.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.548+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2416a2a0 con 0x7f0e240d02b0 2023-10-06T05:23:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.549+0000 7f0e0affd700 1 -- 172.21.15.45:0/2633340198 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e14007670 con 0x7f0e240d02b0 2023-10-06T05:23:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.549+0000 7f0e0affd700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0e10066f30 0x7f0e100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.549+0000 7f0e0affd700 1 -- 172.21.15.45:0/2633340198 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f0e14083fe0 con 0x7f0e240d02b0 2023-10-06T05:23:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.549+0000 7f0e227fc700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0e10066f30 0x7f0e100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.550+0000 7f0e227fc700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0e10066f30 0x7f0e100693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009910 tx=0x7f0e18017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.555+0000 7f0e0affd700 1 -- 172.21.15.45:0/2633340198 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e14050bb0 con 0x7f0e240d02b0 2023-10-06T05:23:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:19.807+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f0e240b53f0 con 0x7f0e240d02b0 2023-10-06T05:23:20.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.092+0000 7f0e0affd700 1 -- 172.21.15.45:0/2633340198 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v150) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0e14017020 con 0x7f0e240d02b0 2023-10-06T05:23:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0e10066f30 msgr2=0x7f0e100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0e10066f30 0x7f0e100693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009910 tx=0x7f0e18017000 comp rx=0 tx=0).stop 2023-10-06T05:23:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d02b0 msgr2=0x7f0e2416a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d02b0 0x7f0e2416a450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0e140011e0 tx=0x7f0e14007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 shutdown_connections 2023-10-06T05:23:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0e10066f30 0x7f0e100693e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e2415c6f0 0x7f0e2416d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e240d02b0 0x7f0e2416a450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 --2- 172.21.15.45:0/2633340198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e240d0680 0x7f0e240c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 >> 172.21.15.45:0/2633340198 conn(0x7f0e2406ed50 msgr2=0x7f0e24159080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 shutdown_connections 2023-10-06T05:23:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.096+0000 7f0e295e1700 1 -- 172.21.15.45:0/2633340198 wait complete. 2023-10-06T05:23:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:20.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:20.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.250+0000 7efe7bef8700 1 Processor -- start 2023-10-06T05:23:20.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.250+0000 7efe7bef8700 1 -- start start 2023-10-06T05:23:20.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe74155cd0 0x7efe7415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe740ccdc0 0x7efe740cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe740cd700 con 0x7efe741560a0 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740ae6b0 con 0x7efe740ccdc0 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740ae8d0 con 0x7efe74155cd0 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe740ccdc0 0x7efe740cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe740ccdc0 0x7efe740cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35370/0 (socket says 172.21.15.45:35370) 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 -- 172.21.15.45:0/3640673697 learned_addr learned my addr 172.21.15.45:0/3640673697 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe79c94700 1 --2- 172.21.15.45:0/3640673697 >> v2:172.21.15.144:3300/0 conn(0x7efe74155cd0 0x7efe7415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 -- 172.21.15.45:0/3640673697 >> v2:172.21.15.144:3300/0 conn(0x7efe74155cd0 msgr2=0x7efe7415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 --2- 172.21.15.45:0/3640673697 >> v2:172.21.15.144:3300/0 conn(0x7efe74155cd0 0x7efe7415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 -- 172.21.15.45:0/3640673697 >> v1:172.21.15.144:6789/0 conn(0x7efe741560a0 legacy=0x7efe740adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 -- 172.21.15.45:0/3640673697 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740aebb0 con 0x7efe740ccdc0 2023-10-06T05:23:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe79c94700 1 --2- 172.21.15.45:0/3640673697 >> v2:172.21.15.144:3300/0 conn(0x7efe74155cd0 0x7efe7415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7a495700 1 --2- 172.21.15.45:0/3640673697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe740ccdc0 0x7efe740cd1c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efe7406ae50 tx=0x7efe6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71be08099f0cec29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe78c92700 1 -- 172.21.15.45:0/3640673697 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00e070 con 0x7efe740ccdc0 2023-10-06T05:23:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe78c92700 1 -- 172.21.15.45:0/3640673697 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c00ba50 con 0x7efe740ccdc0 2023-10-06T05:23:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe78c92700 1 -- 172.21.15.45:0/3640673697 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0092e0 con 0x7efe740ccdc0 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- 172.21.15.45:0/3640673697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe740ccdc0 msgr2=0x7efe740cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 --2- 172.21.15.45:0/3640673697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe740ccdc0 0x7efe740cd1c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efe7406ae50 tx=0x7efe6c009840 comp rx=0 tx=0).stop 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- 172.21.15.45:0/3640673697 shutdown_connections 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 --2- 172.21.15.45:0/3640673697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe740ccdc0 0x7efe740cd1c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 --2- 172.21.15.45:0/3640673697 >> v2:172.21.15.144:3300/0 conn(0x7efe74155cd0 0x7efe7415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- 172.21.15.45:0/3640673697 >> 172.21.15.45:0/3640673697 conn(0x7efe7406f090 msgr2=0x7efe7406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- 172.21.15.45:0/3640673697 shutdown_connections 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.252+0000 7efe7bef8700 1 -- 172.21.15.45:0/3640673697 wait complete. 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.253+0000 7efe7bef8700 1 Processor -- start 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7bef8700 1 -- start start 2023-10-06T05:23:20.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7bef8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740ccdc0 0x7efe740a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7bef8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74155cd0 0x7efe7414b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7bef8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe741560a0 0x7efe740c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7bef8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe741675d0 con 0x7efe741560a0 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7bef8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740a59c0 con 0x7efe740ccdc0 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7bef8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe740a4db0 con 0x7efe74155cd0 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7a495700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe741560a0 0x7efe740c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7a495700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe741560a0 0x7efe740c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35386/0 (socket says 172.21.15.45:35386) 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.254+0000 7efe7a495700 1 -- 172.21.15.45:0/1635102452 learned_addr learned my addr 172.21.15.45:0/1635102452 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe79c94700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740ccdc0 0x7efe740a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe79493700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74155cd0 0x7efe7414b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7a495700 1 -- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74155cd0 msgr2=0x7efe7414b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7a495700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74155cd0 0x7efe7414b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7a495700 1 -- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740ccdc0 msgr2=0x7efe740a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7a495700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740ccdc0 0x7efe740a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7a495700 1 -- 172.21.15.45:0/1635102452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c009ae0 con 0x7efe741560a0 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe79493700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74155cd0 0x7efe7414b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7a495700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe741560a0 0x7efe740c78f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009880 tx=0x7efe6c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe69ffb700 1 -- 172.21.15.45:0/1635102452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c008e80 con 0x7efe741560a0 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe69ffb700 1 -- 172.21.15.45:0/1635102452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c0094e0 con 0x7efe741560a0 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe740a5090 con 0x7efe741560a0 2023-10-06T05:23:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe69ffb700 1 -- 172.21.15.45:0/1635102452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c026690 con 0x7efe741560a0 2023-10-06T05:23:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.255+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe740a5650 con 0x7efe741560a0 2023-10-06T05:23:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.256+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe740a5b30 con 0x7efe741560a0 2023-10-06T05:23:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.257+0000 7efe69ffb700 1 -- 172.21.15.45:0/1635102452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe6c003040 con 0x7efe741560a0 2023-10-06T05:23:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.257+0000 7efe69ffb700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe58066f30 0x7efe580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.257+0000 7efe69ffb700 1 -- 172.21.15.45:0/1635102452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7238+0+0 (secure 0 0 0) 0x7efe6c086a70 con 0x7efe741560a0 2023-10-06T05:23:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.258+0000 7efe79c94700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe58066f30 0x7efe580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.258+0000 7efe79c94700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe58066f30 0x7efe580693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe70006f00 tx=0x7efe70008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.262+0000 7efe69ffb700 1 -- 172.21.15.45:0/1635102452 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe6c0536e0 con 0x7efe741560a0 2023-10-06T05:23:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.507+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe7406e390 con 0x7efe741560a0 2023-10-06T05:23:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.508+0000 7efe69ffb700 1 -- 172.21.15.45:0/1635102452 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+4608 (secure 0 0 0) 0x7efe7406e390 con 0x7efe741560a0 2023-10-06T05:23:20.510 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:20.510 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":151,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:20.094200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8436":{"gid":8436,"name":"c","rank":0,"incarnation":151,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/3141743890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3141743890},{"type":"v1","addr":"172.21.15.45:6835","nonce":3141743890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe58066f30 msgr2=0x7efe580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe58066f30 0x7efe580693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe70006f00 tx=0x7efe70008040 comp rx=0 tx=0).stop 2023-10-06T05:23:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe741560a0 msgr2=0x7efe740c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe741560a0 0x7efe740c78f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009880 tx=0x7efe6c018000 comp rx=0 tx=0).stop 2023-10-06T05:23:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 shutdown_connections 2023-10-06T05:23:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe58066f30 0x7efe580693e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe741560a0 0x7efe740c78f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740ccdc0 0x7efe740a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 --2- 172.21.15.45:0/1635102452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74155cd0 0x7efe7414b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 >> 172.21.15.45:0/1635102452 conn(0x7efe7406f090 msgr2=0x7efe740c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 shutdown_connections 2023-10-06T05:23:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:20.510+0000 7efe7bef8700 1 -- 172.21.15.45:0/1635102452 wait complete. 2023-10-06T05:23:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 151 2023-10-06T05:23:20.525 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:23:23.111 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:23.109+0000 7fc06ccb7700 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2023-10-06T05:23:23.112 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:23.111+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:23.113 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:23.111+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:23:23.117 INFO:tasks.ceph.mds.c.smithi045.stderr: -19> 2023-10-06T05:23:23.109+0000 7fc06ccb7700 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2023-10-06T05:23:23.141 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:25.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:25.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.665+0000 7fdbfc373700 1 Processor -- start 2023-10-06T05:23:25.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.665+0000 7fdbfc373700 1 -- start start 2023-10-06T05:23:25.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfc373700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdbf4158f60 0x7fdbf4159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfc373700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdbf406a060 0x7fdbf40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfc373700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf4069bd0 con 0x7fdbf40cc390 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfc373700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf406a500 con 0x7fdbf406a060 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfc373700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf4068fc0 con 0x7fdbf4158f60 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbf990e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdbf406a060 0x7fdbf40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfa910700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdbf40cc390 0x7fdbf40c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52116/0 (socket says 172.21.15.45:52116) 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfa910700 1 -- 172.21.15.45:0/1773829401 learned_addr learned my addr 172.21.15.45:0/1773829401 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.666+0000 7fdbfa10f700 1 --2- 172.21.15.45:0/1773829401 >> v2:172.21.15.144:3300/0 conn(0x7fdbf4158f60 0x7fdbf4159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf990e700 1 -- 172.21.15.45:0/1773829401 >> v2:172.21.15.144:3300/0 conn(0x7fdbf4158f60 msgr2=0x7fdbf4159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf990e700 1 --2- 172.21.15.45:0/1773829401 >> v2:172.21.15.144:3300/0 conn(0x7fdbf4158f60 0x7fdbf4159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf990e700 1 -- 172.21.15.45:0/1773829401 >> v1:172.21.15.45:6789/0 conn(0x7fdbf40cc390 legacy=0x7fdbf40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf990e700 1 -- 172.21.15.45:0/1773829401 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf40692a0 con 0x7fdbf406a060 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf910d700 1 -- 172.21.15.45:0/1773829401 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378379823 0 0) 0x7fdbf4069bd0 con 0x7fdbf40cc390 2023-10-06T05:23:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf990e700 1 --2- 172.21.15.45:0/1773829401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf406a060 0x7fdbf40cae20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fdbf406ed80 tx=0x7fdbe40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0fa03cdb875d52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf910d700 1 -- 172.21.15.45:0/1773829401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe400b6b0 con 0x7fdbf406a060 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf910d700 1 -- 172.21.15.45:0/1773829401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe400b810 con 0x7fdbf406a060 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbf910d700 1 -- 172.21.15.45:0/1773829401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe400bbb0 con 0x7fdbf406a060 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbfc373700 1 -- 172.21.15.45:0/1773829401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf406a060 msgr2=0x7fdbf40cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.667+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1773829401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf406a060 0x7fdbf40cae20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fdbf406ed80 tx=0x7fdbe40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.668+0000 7fdbfc373700 1 -- 172.21.15.45:0/1773829401 shutdown_connections 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.668+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1773829401 >> v2:172.21.15.144:3300/0 conn(0x7fdbf4158f60 0x7fdbf4159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.668+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1773829401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf406a060 0x7fdbf40cae20 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.668+0000 7fdbfc373700 1 -- 172.21.15.45:0/1773829401 >> 172.21.15.45:0/1773829401 conn(0x7fdbf40cdf50 msgr2=0x7fdbf40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.668+0000 7fdbfc373700 1 -- 172.21.15.45:0/1773829401 shutdown_connections 2023-10-06T05:23:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.668+0000 7fdbfc373700 1 -- 172.21.15.45:0/1773829401 wait complete. 2023-10-06T05:23:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.669+0000 7fdbfc373700 1 Processor -- start 2023-10-06T05:23:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.669+0000 7fdbfc373700 1 -- start start 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfc373700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf406a060 0x7fdbf40c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfc373700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf40cc390 0x7fdbf415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfc373700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbf4158f60 0x7fdbf4169660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfc373700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf415a820 con 0x7fdbf40cc390 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfc373700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4156ad0 con 0x7fdbf406a060 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfc373700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4157be0 con 0x7fdbf4158f60 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf40cc390 0x7fdbf415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf40cc390 0x7fdbf415a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35434/0 (socket says 172.21.15.45:35434) 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 -- 172.21.15.45:0/1610546712 learned_addr learned my addr 172.21.15.45:0/1610546712 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfa10f700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf406a060 0x7fdbf40c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfa910700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbf4158f60 0x7fdbf4169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 -- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbf4158f60 msgr2=0x7fdbf4169660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbf4158f60 0x7fdbf4169660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 -- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf406a060 msgr2=0x7fdbf40c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf406a060 0x7fdbf40c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 -- 172.21.15.45:0/1610546712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4009420 con 0x7fdbf40cc390 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfa910700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbf4158f60 0x7fdbf4169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfa10f700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf406a060 0x7fdbf40c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbf990e700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf40cc390 0x7fdbf415a2e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40c4f80 tx=0x7fdbe4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbea7fc700 1 -- 172.21.15.45:0/1610546712 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4011420 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbea7fc700 1 -- 172.21.15.45:0/1610546712 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe4011580 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf4157ec0 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.670+0000 7fdbea7fc700 1 -- 172.21.15.45:0/1610546712 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4011940 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.671+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf41578b0 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.671+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf406f410 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.672+0000 7fdbea7fc700 1 -- 172.21.15.45:0/1610546712 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdbe4011d30 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.672+0000 7fdbea7fc700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbd8066f30 0x7fdbd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.672+0000 7fdbea7fc700 1 -- 172.21.15.45:0/1610546712 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fdbe40860b0 con 0x7fdbf40cc390 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.673+0000 7fdbfa10f700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbd8066f30 0x7fdbd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.673+0000 7fdbfa10f700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbd8066f30 0x7fdbd80693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009990 tx=0x7fdbf0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.677+0000 7fdbea7fc700 1 -- 172.21.15.45:0/1610546712 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbe4052cd0 con 0x7fdbf40cc390 2023-10-06T05:23:25.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.929+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbf4069580 con 0x7fdbf40cc390 2023-10-06T05:23:25.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.930+0000 7fdbea7fc700 1 -- 172.21.15.45:0/1610546712 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+4599 (secure 0 0 0) 0x7fdbe4015090 con 0x7fdbf40cc390 2023-10-06T05:23:25.932 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:25.932 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2620071126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2620071126},{"type":"v1","addr":"172.21.15.45:6835","nonce":2620071126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:23.112112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:25.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbd8066f30 msgr2=0x7fdbd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbd8066f30 0x7fdbd80693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009990 tx=0x7fdbf0008040 comp rx=0 tx=0).stop 2023-10-06T05:23:25.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf40cc390 msgr2=0x7fdbf415a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf40cc390 0x7fdbf415a2e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40c4f80 tx=0x7fdbe4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:25.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 shutdown_connections 2023-10-06T05:23:25.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdbd8066f30 0x7fdbd80693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbf4158f60 0x7fdbf4169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf406a060 0x7fdbf40c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 --2- 172.21.15.45:0/1610546712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf40cc390 0x7fdbf415a2e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 >> 172.21.15.45:0/1610546712 conn(0x7fdbf40cdf50 msgr2=0x7fdbf4154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:25.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 shutdown_connections 2023-10-06T05:23:25.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:25.932+0000 7fdbfc373700 1 -- 172.21.15.45:0/1610546712 wait complete. 2023-10-06T05:23:25.935 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 156 2023-10-06T05:23:25.947 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:23:25.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.078+0000 7f56d948a700 1 Processor -- start 2023-10-06T05:23:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.078+0000 7f56d948a700 1 -- start start 2023-10-06T05:23:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d948a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f56d4159560 0x7f56d40b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d948a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d40b0ae0 con 0x7f56d4159190 2023-10-06T05:23:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d948a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d40cd250 con 0x7f56d40b2560 2023-10-06T05:23:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d948a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d40cc640 con 0x7f56d4159560 2023-10-06T05:23:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d2ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f56d4159190 0x7f56d40b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52132/0 (socket says 172.21.15.45:52132) 2023-10-06T05:23:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d2ffd700 1 -- 172.21.15.45:0/1744724509 learned_addr learned my addr 172.21.15.45:0/1744724509 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d27fc700 1 --2- 172.21.15.45:0/1744724509 >> v2:172.21.15.144:3300/0 conn(0x7f56d4159560 0x7f56d40b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d1ffb700 1 -- 172.21.15.45:0/1744724509 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108479047 0 0) 0x7f56c8001110 con 0x7f56d4159190 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d1ffb700 1 -- 172.21.15.45:0/1744724509 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc003570 con 0x7f56d4159190 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d1ffb700 1 -- 172.21.15.45:0/1744724509 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443010070 0 0) 0x7f56d40cd250 con 0x7f56d40b2560 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d1ffb700 1 -- 172.21.15.45:0/1744724509 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56c8001110 con 0x7f56d40b2560 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d27fc700 1 -- 172.21.15.45:0/1744724509 >> v1:172.21.15.144:6789/0 conn(0x7f56d40b2560 legacy=0x7f56d40cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d27fc700 1 -- 172.21.15.45:0/1744724509 >> v1:172.21.15.45:6789/0 conn(0x7f56d4159190 legacy=0x7f56d40b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.079+0000 7f56d27fc700 1 -- 172.21.15.45:0/1744724509 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40cc920 con 0x7f56d4159560 2023-10-06T05:23:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d27fc700 1 --2- 172.21.15.45:0/1744724509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159560 0x7f56d40b2020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f56c40098e0 tx=0x7f56c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ea50b8b909fc49f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d1ffb700 1 -- 172.21.15.45:0/1744724509 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4011630 con 0x7f56d4159560 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d1ffb700 1 -- 172.21.15.45:0/1744724509 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4011790 con 0x7f56d4159560 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d1ffb700 1 -- 172.21.15.45:0/1744724509 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4011b10 con 0x7f56d4159560 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d948a700 1 -- 172.21.15.45:0/1744724509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159560 msgr2=0x7f56d40b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d948a700 1 --2- 172.21.15.45:0/1744724509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159560 0x7f56d40b2020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f56c40098e0 tx=0x7f56c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d948a700 1 -- 172.21.15.45:0/1744724509 shutdown_connections 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d948a700 1 --2- 172.21.15.45:0/1744724509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159560 0x7f56d40b2020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.080+0000 7f56d948a700 1 -- 172.21.15.45:0/1744724509 >> 172.21.15.45:0/1744724509 conn(0x7f56d406ed50 msgr2=0x7f56d406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.081+0000 7f56d948a700 1 -- 172.21.15.45:0/1744724509 shutdown_connections 2023-10-06T05:23:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.081+0000 7f56d948a700 1 -- 172.21.15.45:0/1744724509 wait complete. 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.082+0000 7f56d948a700 1 Processor -- start 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.082+0000 7f56d948a700 1 -- start start 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d948a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d40b2560 0x7f56d4150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d948a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159190 0x7f56d416a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d948a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4159560 0x7f56d40b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d948a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d41506c0 con 0x7f56d40b2560 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d948a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d416a9c0 con 0x7f56d4159190 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d948a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4159dc0 con 0x7f56d4159560 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d40b2560 0x7f56d4150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d40b2560 0x7f56d4150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35440/0 (socket says 172.21.15.45:35440) 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 -- 172.21.15.45:0/1068803009 learned_addr learned my addr 172.21.15.45:0/1068803009 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d37fe700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4159560 0x7f56d40b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d27fc700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159190 0x7f56d416a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 -- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4159560 msgr2=0x7f56d40b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4159560 0x7f56d40b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 -- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159190 msgr2=0x7f56d416a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159190 0x7f56d416a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 -- 172.21.15.45:0/1068803009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4009420 con 0x7f56d40b2560 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d27fc700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159190 0x7f56d416a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56d2ffd700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d40b2560 0x7f56d4150180 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f56c800c770 tx=0x7f56c8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.083+0000 7f56baffd700 1 -- 172.21.15.45:0/1068803009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8005b30 con 0x7f56d40b2560 2023-10-06T05:23:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.084+0000 7f56baffd700 1 -- 172.21.15.45:0/1068803009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c8005c90 con 0x7f56d40b2560 2023-10-06T05:23:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.084+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d415a0a0 con 0x7f56d40b2560 2023-10-06T05:23:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.084+0000 7f56baffd700 1 -- 172.21.15.45:0/1068803009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8007320 con 0x7f56d40b2560 2023-10-06T05:23:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.084+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d415a660 con 0x7f56d40b2560 2023-10-06T05:23:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.085+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d4088820 con 0x7f56d40b2560 2023-10-06T05:23:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.085+0000 7f56baffd700 1 -- 172.21.15.45:0/1068803009 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56c801b400 con 0x7f56d40b2560 2023-10-06T05:23:26.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.085+0000 7f56baffd700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56c0066f30 0x7f56c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.086+0000 7f56baffd700 1 -- 172.21.15.45:0/1068803009 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f56c80845e0 con 0x7f56d40b2560 2023-10-06T05:23:26.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.086+0000 7f56d27fc700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56c0066f30 0x7f56c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.086+0000 7f56d27fc700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56c0066f30 0x7f56c00693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f56d4154400 tx=0x7f56c4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.090+0000 7f56baffd700 1 -- 172.21.15.45:0/1068803009 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56c8051280 con 0x7f56d40b2560 2023-10-06T05:23:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.338+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56d40b0ae0 con 0x7f56d40b2560 2023-10-06T05:23:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.338+0000 7f56baffd700 1 -- 172.21.15.45:0/1068803009 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+4599 (secure 0 0 0) 0x7f56d40b0ae0 con 0x7f56d40b2560 2023-10-06T05:23:26.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:26.340 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2620071126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2620071126},{"type":"v1","addr":"172.21.15.45:6835","nonce":2620071126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:23.112112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56c0066f30 msgr2=0x7f56c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56c0066f30 0x7f56c00693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f56d4154400 tx=0x7f56c4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d40b2560 msgr2=0x7f56d4150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d40b2560 0x7f56d4150180 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f56c800c770 tx=0x7f56c8007ea0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 shutdown_connections 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56c0066f30 0x7f56c00693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4159560 0x7f56d40b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d40b2560 0x7f56d4150180 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 --2- 172.21.15.45:0/1068803009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d4159190 0x7f56d416a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 >> 172.21.15.45:0/1068803009 conn(0x7f56d406ed50 msgr2=0x7f56d40ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 shutdown_connections 2023-10-06T05:23:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.341+0000 7f56d948a700 1 -- 172.21.15.45:0/1068803009 wait complete. 2023-10-06T05:23:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 156 2023-10-06T05:23:26.357 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:26.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.487+0000 7f7d15506700 1 Processor -- start 2023-10-06T05:23:26.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.487+0000 7f7d15506700 1 -- start start 2023-10-06T05:23:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d15506700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d10140990 0x7f7d10069f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d15506700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7d100b1430 0x7f7d1006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d15506700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1006b1f0 con 0x7f7d100b1060 2023-10-06T05:23:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d15506700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1006f270 con 0x7f7d10140990 2023-10-06T05:23:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d15506700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1006a5e0 con 0x7f7d100b1430 2023-10-06T05:23:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0e7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d10140990 0x7f7d10069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7d100b1060 0x7f7d10140280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48640/0 (socket says 172.21.15.45:48640) 2023-10-06T05:23:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0effd700 1 -- 172.21.15.45:0/4281269312 learned_addr learned my addr 172.21.15.45:0/4281269312 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0f7fe700 1 --2- 172.21.15.45:0/4281269312 >> v2:172.21.15.144:3300/0 conn(0x7f7d100b1430 0x7f7d1006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0e7fc700 1 -- 172.21.15.45:0/4281269312 >> v2:172.21.15.144:3300/0 conn(0x7f7d100b1430 msgr2=0x7f7d1006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0e7fc700 1 --2- 172.21.15.45:0/4281269312 >> v2:172.21.15.144:3300/0 conn(0x7f7d100b1430 0x7f7d1006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0e7fc700 1 -- 172.21.15.45:0/4281269312 >> v1:172.21.15.45:6789/0 conn(0x7f7d100b1060 legacy=0x7f7d10140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0e7fc700 1 -- 172.21.15.45:0/4281269312 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1006a8c0 con 0x7f7d10140990 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0dffb700 1 -- 172.21.15.45:0/4281269312 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651593679 0 0) 0x7f7d1006b1f0 con 0x7f7d100b1060 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.488+0000 7f7d0e7fc700 1 --2- 172.21.15.45:0/4281269312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d10140990 0x7f7d10069f70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7d10157610 tx=0x7f7d000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e0b8a6fb7631c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d0dffb700 1 -- 172.21.15.45:0/4281269312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000b6b0 con 0x7f7d10140990 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d0dffb700 1 -- 172.21.15.45:0/4281269312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0000b810 con 0x7f7d10140990 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d0dffb700 1 -- 172.21.15.45:0/4281269312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000bbb0 con 0x7f7d10140990 2023-10-06T05:23:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 -- 172.21.15.45:0/4281269312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d10140990 msgr2=0x7f7d10069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 --2- 172.21.15.45:0/4281269312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d10140990 0x7f7d10069f70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7d10157610 tx=0x7f7d000091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 -- 172.21.15.45:0/4281269312 shutdown_connections 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 --2- 172.21.15.45:0/4281269312 >> v2:172.21.15.144:3300/0 conn(0x7f7d100b1430 0x7f7d1006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 --2- 172.21.15.45:0/4281269312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d10140990 0x7f7d10069f70 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 -- 172.21.15.45:0/4281269312 >> 172.21.15.45:0/4281269312 conn(0x7f7d10067620 msgr2=0x7f7d10067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 -- 172.21.15.45:0/4281269312 shutdown_connections 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.489+0000 7f7d15506700 1 -- 172.21.15.45:0/4281269312 wait complete. 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 Processor -- start 2023-10-06T05:23:26.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 -- start start 2023-10-06T05:23:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d100b1060 0x7f7d10139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d100b1430 0x7f7d10156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d10140990 0x7f7d1015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10139e70 con 0x7f7d100b1060 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10156da0 con 0x7f7d100b1430 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.491+0000 7f7d15506700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10134570 con 0x7f7d10140990 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d100b1060 0x7f7d10139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d100b1060 0x7f7d10139930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60738/0 (socket says 172.21.15.45:60738) 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 -- 172.21.15.45:0/3389870658 learned_addr learned my addr 172.21.15.45:0/3389870658 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0f7fe700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d10140990 0x7f7d1015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0e7fc700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d100b1430 0x7f7d10156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 -- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d10140990 msgr2=0x7f7d1015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d10140990 0x7f7d1015ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 -- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d100b1430 msgr2=0x7f7d10156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d100b1430 0x7f7d10156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 -- 172.21.15.45:0/3389870658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009420 con 0x7f7d100b1060 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0e7fc700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d100b1430 0x7f7d10156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d0effd700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d100b1060 0x7f7d10139930 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c7e0 tx=0x7f7d04007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7cf6ffd700 1 -- 172.21.15.45:0/3389870658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04005b30 con 0x7f7d100b1060 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7cf6ffd700 1 -- 172.21.15.45:0/3389870658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04005c90 con 0x7f7d100b1060 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d10134850 con 0x7f7d100b1060 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7cf6ffd700 1 -- 172.21.15.45:0/3389870658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04007300 con 0x7f7d100b1060 2023-10-06T05:23:26.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.492+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d10134e10 con 0x7f7d100b1060 2023-10-06T05:23:26.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.493+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1006cec0 con 0x7f7d100b1060 2023-10-06T05:23:26.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.494+0000 7f7cf6ffd700 1 -- 172.21.15.45:0/3389870658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d0401b400 con 0x7f7d100b1060 2023-10-06T05:23:26.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.494+0000 7f7cf6ffd700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cfc066f30 0x7f7cfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.495+0000 7f7cf6ffd700 1 -- 172.21.15.45:0/3389870658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f7d04084770 con 0x7f7d100b1060 2023-10-06T05:23:26.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.495+0000 7f7d0e7fc700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cfc066f30 0x7f7cfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.495+0000 7f7d0e7fc700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cfc066f30 0x7f7cfc0693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7d1015cd30 tx=0x7f7d00017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.499+0000 7f7cf6ffd700 1 -- 172.21.15.45:0/3389870658 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d04051370 con 0x7f7d100b1060 2023-10-06T05:23:26.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.744+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d1006aba0 con 0x7f7d100b1060 2023-10-06T05:23:26.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.745+0000 7f7cf6ffd700 1 -- 172.21.15.45:0/3389870658 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+4599 (secure 0 0 0) 0x7f7d04054990 con 0x7f7d100b1060 2023-10-06T05:23:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:26.747 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":156,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2620071126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2620071126},{"type":"v1","addr":"172.21.15.45:6835","nonce":2620071126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:23.112112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:26.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cfc066f30 msgr2=0x7f7cfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cfc066f30 0x7f7cfc0693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7d1015cd30 tx=0x7f7d00017000 comp rx=0 tx=0).stop 2023-10-06T05:23:26.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d100b1060 msgr2=0x7f7d10139930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d100b1060 0x7f7d10139930 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c7e0 tx=0x7f7d04007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 shutdown_connections 2023-10-06T05:23:26.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cfc066f30 0x7f7cfc0693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d10140990 0x7f7d1015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d100b1060 0x7f7d10139930 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 --2- 172.21.15.45:0/3389870658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d100b1430 0x7f7d10156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 >> 172.21.15.45:0/3389870658 conn(0x7f7d10067620 msgr2=0x7f7d100acc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 shutdown_connections 2023-10-06T05:23:26.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.747+0000 7f7d15506700 1 -- 172.21.15.45:0/3389870658 wait complete. 2023-10-06T05:23:26.750 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 156 2023-10-06T05:23:26.763 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2023-10-06T05:23:26.763 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2023-10-06T05:23:26.763 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:26.763 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:26.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:26.763 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:26.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:23:26.784 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:23:26.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:26.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.963+0000 7fd0b7c32700 1 Processor -- start 2023-10-06T05:23:26.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.963+0000 7fd0b7c32700 1 -- start start 2023-10-06T05:23:26.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b7c32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 0x7fd0b00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b7c32700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b00cd300 0x7fd0b0161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b7c32700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd0b0161a50 0x7fd0b00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b7c32700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b00cd770 con 0x7fd0b00cd300 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b7c32700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b006b080 con 0x7fd0b0161a50 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b7c32700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b006b2d0 con 0x7fd0b00c0170 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b00cd300 0x7fd0b0161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b00cd300 0x7fd0b0161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60744/0 (socket says 172.21.15.45:60744) 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 -- 172.21.15.45:0/3434115853 learned_addr learned my addr 172.21.15.45:0/3434115853 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 -- 172.21.15.45:0/3434115853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 msgr2=0x7fd0b00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 --2- 172.21.15.45:0/3434115853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 0x7fd0b00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 -- 172.21.15.45:0/3434115853 >> v2:172.21.15.144:3300/0 conn(0x7fd0b0161a50 msgr2=0x7fd0b00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 --2- 172.21.15.45:0/3434115853 >> v2:172.21.15.144:3300/0 conn(0x7fd0b0161a50 0x7fd0b00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.964+0000 7fd0b51cd700 1 -- 172.21.15.45:0/3434115853 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b006b5b0 con 0x7fd0b00cd300 2023-10-06T05:23:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b51cd700 1 --2- 172.21.15.45:0/3434115853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b00cd300 0x7fd0b0161510 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00c7360 tx=0x7fd0a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce529f6beb8fcb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b49cc700 1 -- 172.21.15.45:0/3434115853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400b860 con 0x7fd0b00cd300 2023-10-06T05:23:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b49cc700 1 -- 172.21.15.45:0/3434115853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a400b9c0 con 0x7fd0b00cd300 2023-10-06T05:23:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b49cc700 1 -- 172.21.15.45:0/3434115853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400bd60 con 0x7fd0b00cd300 2023-10-06T05:23:26.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 -- 172.21.15.45:0/3434115853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b00cd300 msgr2=0x7fd0b0161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/3434115853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b00cd300 0x7fd0b0161510 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00c7360 tx=0x7fd0a4009250 comp rx=0 tx=0).stop 2023-10-06T05:23:26.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 -- 172.21.15.45:0/3434115853 shutdown_connections 2023-10-06T05:23:26.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/3434115853 >> v2:172.21.15.144:3300/0 conn(0x7fd0b0161a50 0x7fd0b00c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/3434115853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 0x7fd0b00ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/3434115853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b00cd300 0x7fd0b0161510 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 -- 172.21.15.45:0/3434115853 >> 172.21.15.45:0/3434115853 conn(0x7fd0b006f090 msgr2=0x7fd0b006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.965+0000 7fd0b7c32700 1 -- 172.21.15.45:0/3434115853 shutdown_connections 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.966+0000 7fd0b7c32700 1 -- 172.21.15.45:0/3434115853 wait complete. 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 Processor -- start 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 -- start start 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 0x7fd0b00c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b00cd300 0x7fd0b015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b0161a50 0x7fd0b015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b015f310 con 0x7fd0b0161a50 2023-10-06T05:23:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b015bd70 con 0x7fd0b00cd300 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.967+0000 7fd0b7c32700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b015ea70 con 0x7fd0b00c0170 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b0161a50 0x7fd0b015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b0161a50 0x7fd0b015fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60756/0 (socket says 172.21.15.45:60756) 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 -- 172.21.15.45:0/107214674 learned_addr learned my addr 172.21.15.45:0/107214674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b51cd700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b00cd300 0x7fd0b015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b59ce700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 0x7fd0b00c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 -- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 msgr2=0x7fd0b00c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 0x7fd0b00c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 -- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b00cd300 msgr2=0x7fd0b015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b00cd300 0x7fd0b015b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 -- 172.21.15.45:0/107214674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a40094f0 con 0x7fd0b0161a50 2023-10-06T05:23:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b61cf700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b0161a50 0x7fd0b015fd00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800bc30 tx=0x7fd0a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0a1ffb700 1 -- 172.21.15.45:0/107214674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80092e0 con 0x7fd0b0161a50 2023-10-06T05:23:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0a1ffb700 1 -- 172.21.15.45:0/107214674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a800c630 con 0x7fd0b0161a50 2023-10-06T05:23:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b015ed50 con 0x7fd0b0161a50 2023-10-06T05:23:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.968+0000 7fd0a1ffb700 1 -- 172.21.15.45:0/107214674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800ca10 con 0x7fd0b0161a50 2023-10-06T05:23:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.969+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b00c3f70 con 0x7fd0b0161a50 2023-10-06T05:23:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.970+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b006e2f0 con 0x7fd0b0161a50 2023-10-06T05:23:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.970+0000 7fd0a1ffb700 1 -- 172.21.15.45:0/107214674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0a8012070 con 0x7fd0b0161a50 2023-10-06T05:23:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.971+0000 7fd0a1ffb700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd094066f30 0x7fd0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.971+0000 7fd0a1ffb700 1 -- 172.21.15.45:0/107214674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fd0a8084760 con 0x7fd0b0161a50 2023-10-06T05:23:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.971+0000 7fd0b59ce700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd094066f30 0x7fd0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.972+0000 7fd0b59ce700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd094066f30 0x7fd0940693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009bf0 tx=0x7fd0ac0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:26.975+0000 7fd0a1ffb700 1 -- 172.21.15.45:0/107214674 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0a80512e0 con 0x7fd0b0161a50 2023-10-06T05:23:27.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.225+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0b00b2c10 con 0x7fd0b0161a50 2023-10-06T05:23:27.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.881+0000 7fd0a1ffb700 1 -- 172.21.15.45:0/107214674 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v157) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd0a8054900 con 0x7fd0b0161a50 2023-10-06T05:23:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd094066f30 msgr2=0x7fd0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd094066f30 0x7fd0940693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009bf0 tx=0x7fd0ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b0161a50 msgr2=0x7fd0b015fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b0161a50 0x7fd0b015fd00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800bc30 tx=0x7fd0a8009840 comp rx=0 tx=0).stop 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 shutdown_connections 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd094066f30 0x7fd0940693e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b0161a50 0x7fd0b015fd00 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b00c0170 0x7fd0b00c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 --2- 172.21.15.45:0/107214674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b00cd300 0x7fd0b015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 >> 172.21.15.45:0/107214674 conn(0x7fd0b006f090 msgr2=0x7fd0b00caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 shutdown_connections 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:27.884+0000 7fd0b7c32700 1 -- 172.21.15.45:0/107214674 wait complete. 2023-10-06T05:23:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:27.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.036+0000 7f6aaae2e700 1 Processor -- start 2023-10-06T05:23:28.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.036+0000 7f6aaae2e700 1 -- start start 2023-10-06T05:23:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aaae2e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa41595d0 0x7f6aa406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aaae2e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6aa406b5c0 0x7f6aa40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aaae2e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa40c7d00 con 0x7f6aa4159200 2023-10-06T05:23:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aaae2e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa40c7780 con 0x7f6aa41595d0 2023-10-06T05:23:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aaae2e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa406bb40 con 0x7f6aa406b5c0 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa41595d0 0x7f6aa406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa8bca700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6aa4159200 0x7f6aa40c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48650/0 (socket says 172.21.15.45:48650) 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa8bca700 1 -- 172.21.15.45:0/2158010225 learned_addr learned my addr 172.21.15.45:0/2158010225 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa93cb700 1 --2- 172.21.15.45:0/2158010225 >> v2:172.21.15.144:3300/0 conn(0x7f6aa406b5c0 0x7f6aa40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa3fff700 1 -- 172.21.15.45:0/2158010225 >> v2:172.21.15.144:3300/0 conn(0x7f6aa406b5c0 msgr2=0x7f6aa40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa3fff700 1 --2- 172.21.15.45:0/2158010225 >> v2:172.21.15.144:3300/0 conn(0x7f6aa406b5c0 0x7f6aa40ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa3fff700 1 -- 172.21.15.45:0/2158010225 >> v1:172.21.15.45:6789/0 conn(0x7f6aa4159200 legacy=0x7f6aa40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa3fff700 1 -- 172.21.15.45:0/2158010225 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa406be20 con 0x7f6aa41595d0 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa37fe700 1 -- 172.21.15.45:0/2158010225 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105825178 0 0) 0x7f6aa40c7d00 con 0x7f6aa4159200 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa3fff700 1 --2- 172.21.15.45:0/2158010225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa41595d0 0x7f6aa406b080 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4068680 tx=0x7f6a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4c1e5c778c144 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.037+0000 7f6aa37fe700 1 -- 172.21.15.45:0/2158010225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c011630 con 0x7f6aa41595d0 2023-10-06T05:23:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aa37fe700 1 -- 172.21.15.45:0/2158010225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c011790 con 0x7f6aa41595d0 2023-10-06T05:23:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aa37fe700 1 -- 172.21.15.45:0/2158010225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c011b30 con 0x7f6aa41595d0 2023-10-06T05:23:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 -- 172.21.15.45:0/2158010225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa41595d0 msgr2=0x7f6aa406b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/2158010225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa41595d0 0x7f6aa406b080 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4068680 tx=0x7f6a8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 -- 172.21.15.45:0/2158010225 shutdown_connections 2023-10-06T05:23:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/2158010225 >> v2:172.21.15.144:3300/0 conn(0x7f6aa406b5c0 0x7f6aa40ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/2158010225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa41595d0 0x7f6aa406b080 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 -- 172.21.15.45:0/2158010225 >> 172.21.15.45:0/2158010225 conn(0x7f6aa406f090 msgr2=0x7f6aa406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 -- 172.21.15.45:0/2158010225 shutdown_connections 2023-10-06T05:23:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.038+0000 7f6aaae2e700 1 -- 172.21.15.45:0/2158010225 wait complete. 2023-10-06T05:23:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.040+0000 7f6aaae2e700 1 Processor -- start 2023-10-06T05:23:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.040+0000 7f6aaae2e700 1 -- start start 2023-10-06T05:23:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.040+0000 7f6aaae2e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406b5c0 0x7f6aa40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aaae2e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa4159200 0x7f6aa40c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aaae2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa41595d0 0x7f6aa40c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aaae2e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40c4e50 con 0x7f6aa406b5c0 2023-10-06T05:23:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aaae2e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40c3e40 con 0x7f6aa41595d0 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aaae2e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40c49c0 con 0x7f6aa4159200 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406b5c0 0x7f6aa40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406b5c0 0x7f6aa40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60786/0 (socket says 172.21.15.45:60786) 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 -- 172.21.15.45:0/3973039107 learned_addr learned my addr 172.21.15.45:0/3973039107 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa3fff700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa4159200 0x7f6aa40c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 -- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa4159200 msgr2=0x7f6aa40c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa4159200 0x7f6aa40c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 -- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa41595d0 msgr2=0x7f6aa40c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa93cb700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa41595d0 0x7f6aa40c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa41595d0 0x7f6aa40c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 -- 172.21.15.45:0/3973039107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c009420 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa3fff700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa4159200 0x7f6aa40c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa8bca700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406b5c0 0x7f6aa40b2320 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400c7e0 tx=0x7f6a94007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa0ff9700 1 -- 172.21.15.45:0/3973039107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94005b30 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa0ff9700 1 -- 172.21.15.45:0/3973039107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a94005c90 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa40c4090 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aa0ff9700 1 -- 172.21.15.45:0/3973039107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94007300 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.041+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa40c4650 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.043+0000 7f6aa0ff9700 1 -- 172.21.15.45:0/3973039107 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a9401b400 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.043+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa40c4ca0 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.043+0000 7f6aa0ff9700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.044+0000 7f6aa0ff9700 1 -- 172.21.15.45:0/3973039107 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f6a94084680 con 0x7f6aa406b5c0 2023-10-06T05:23:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.044+0000 7f6aa3fff700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.044+0000 7f6aa3fff700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ca2f0 tx=0x7f6a8c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.048+0000 7f6aa0ff9700 1 -- 172.21.15.45:0/3973039107 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a94051320 con 0x7f6aa406b5c0 2023-10-06T05:23:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.343+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f6aa40c7990 con 0x7f6aa406b5c0 2023-10-06T05:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.889+0000 7f6aa0ff9700 1 -- 172.21.15.45:0/3973039107 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v158) v1 ==== 101+0+0 (secure 0 0 0) 0x7f6a94054940 con 0x7f6aa406b5c0 2023-10-06T05:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 msgr2=0x7f6a840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ca2f0 tx=0x7f6a8c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406b5c0 msgr2=0x7f6aa40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406b5c0 0x7f6aa40b2320 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400c7e0 tx=0x7f6a94007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 shutdown_connections 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa41595d0 0x7f6aa40c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406b5c0 0x7f6aa40b2320 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 --2- 172.21.15.45:0/3973039107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa4159200 0x7f6aa40c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.892+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 >> 172.21.15.45:0/3973039107 conn(0x7f6aa406f090 msgr2=0x7f6aa40c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.893+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 shutdown_connections 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.893+0000 7f6aaae2e700 1 -- 172.21.15.45:0/3973039107 wait complete. 2023-10-06T05:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:23:28.910 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:28.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.956+0000 7f2e9b523040 1 Processor -- start 2023-10-06T05:23:28.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.956+0000 7f2e9b523040 1 -- start start 2023-10-06T05:23:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e9b523040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c732fe0 0x55fc5c764e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e9b523040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e9b523040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c7f4970 0x55fc5c77aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e9b523040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc5c67f980 con 0x55fc5c7785f0 2023-10-06T05:23:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e9b523040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc5c67f620 con 0x55fc5c732fe0 2023-10-06T05:23:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e9b523040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc5c676400 con 0x55fc5c7f4970 2023-10-06T05:23:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7358e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60804/0 (socket says 172.21.15.45:60804) 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 -- 172.21.15.45:0/1508411633 learned_addr learned my addr 172.21.15.45:0/1508411633 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e94741700 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c732fe0 0x55fc5c764e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e94f42700 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c7f4970 0x55fc5c77aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 -- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c7f4970 msgr2=0x55fc5c77aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c7f4970 0x55fc5c77aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 -- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c732fe0 msgr2=0x55fc5c764e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c732fe0 0x55fc5c764e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e93f40700 1 -- 172.21.15.45:0/1508411633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e84005db0 con 0x55fc5c7785f0 2023-10-06T05:23:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.957+0000 7f2e94741700 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c732fe0 0x55fc5c764e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e93f40700 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7358e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2e880099b0 tx=0x7f2e8800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263ecf29b22ac906 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e92f3e700 1 -- 172.21.15.45:0/1508411633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800f040 con 0x55fc5c7785f0 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e92f3e700 1 -- 172.21.15.45:0/1508411633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88008bc0 con 0x55fc5c7785f0 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e92f3e700 1 -- 172.21.15.45:0/1508411633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88014480 con 0x55fc5c7785f0 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 -- 172.21.15.45:0/1508411633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 msgr2=0x55fc5c7358e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7358e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2e880099b0 tx=0x7f2e8800c040 comp rx=0 tx=0).stop 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 -- 172.21.15.45:0/1508411633 shutdown_connections 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c7f4970 0x55fc5c77aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c732fe0 0x55fc5c764e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 --2- 172.21.15.45:0/1508411633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7358e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 -- 172.21.15.45:0/1508411633 >> 172.21.15.45:0/1508411633 conn(0x55fc5c7354a0 msgr2=0x55fc5c737530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 -- 172.21.15.45:0/1508411633 shutdown_connections 2023-10-06T05:23:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.958+0000 7f2e9b523040 1 -- 172.21.15.45:0/1508411633 wait complete. 2023-10-06T05:23:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.959+0000 7f2e9b523040 1 Processor -- start 2023-10-06T05:23:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.959+0000 7f2e9b523040 1 -- start start 2023-10-06T05:23:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e9b523040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c732fe0 0x55fc5c81c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e9b523040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e9b523040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c7f4970 0x55fc5c81cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e9b523040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc5c77e630 con 0x55fc5c7785f0 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e9b523040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc5c81ca60 con 0x55fc5c7f4970 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e9b523040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc5c7ec950 con 0x55fc5c732fe0 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60814/0 (socket says 172.21.15.45:60814) 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 learned_addr learned my addr 172.21.15.45:0/3626853455 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e94f42700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c7f4970 0x55fc5c81cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c732fe0 0x55fc5c81c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c732fe0 msgr2=0x55fc5c81c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c732fe0 0x55fc5c81c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c7f4970 msgr2=0x55fc5c81cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc5c7f4970 0x55fc5c81cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88009580 con 0x55fc5c7785f0 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.960+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fc5c732fe0 0x55fc5c81c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.961+0000 7f2e93f40700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7eb930 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2e88005e70 tx=0x7f2e88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.961+0000 7f2e9173b700 1 -- 172.21.15.45:0/3626853455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800f040 con 0x55fc5c7785f0 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.961+0000 7f2e9173b700 1 -- 172.21.15.45:0/3626853455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8801f420 con 0x55fc5c7785f0 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.961+0000 7f2e9173b700 1 -- 172.21.15.45:0/3626853455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8801f7c0 con 0x55fc5c7785f0 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.961+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fc5c68e3d0 con 0x55fc5c7785f0 2023-10-06T05:23:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.961+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc5c681ae0 con 0x55fc5c7785f0 2023-10-06T05:23:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.962+0000 7f2e9173b700 1 -- 172.21.15.45:0/3626853455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e880133e0 con 0x55fc5c7785f0 2023-10-06T05:23:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.963+0000 7f2e9173b700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e80065890 0x7f2e80067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.963+0000 7f2e9173b700 1 -- 172.21.15.45:0/3626853455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f2e880850e0 con 0x55fc5c7785f0 2023-10-06T05:23:28.970 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.963+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e80065890 0x7f2e80067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.963+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55fc5c7ec4b0 0x55fc5c7ed870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.963+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c7eddb0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.963+0000 7f2e94f42700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55fc5c7ec4b0 0x55fc5c7ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.964+0000 7f2e94f42700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55fc5c7ec4b0 0x55fc5c7ed870 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.964+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e80065890 0x7f2e80067d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2e84008fd0 tx=0x7f2e84016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.965+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v103'319 uv293 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.965+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c828120 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.967+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v103'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.967+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8284d0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.973 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.968+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v103'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.968+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829890 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.969+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'322 uv322 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.969+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829c40 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.970+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.970+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c828880 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.971+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.972+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c828c80 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.973+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v103'325 uv299 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.973+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829080 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.974+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v103'326 uv326 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.974+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829080 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.977 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.975+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v103'327 uv327 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.975+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829080 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.976+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'328 uv328 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.976+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829080 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.977+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'329 uv329 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.977+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8294b0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.979+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'330 uv330 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.979+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8294b0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.980+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v103'331 uv306 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.980+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.981+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'332 uv332 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.981+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.984 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.982+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v103'333 uv333 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.982+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.983+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'334 uv334 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.983+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.984+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'335 uv335 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.984+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.985+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'336 uv336 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.985+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.987+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'337 uv337 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.987+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.988+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v103'338 uv312 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.988+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.989+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v103'339 uv339 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.989+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.992 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.990+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v103'340 uv340 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.990+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.992+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'341 uv341 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.992+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.993+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.993+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.994+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.994+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.995+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v103'344 uv318 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.995+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.997+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v103'345 uv345 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.997+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:28.999 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.998+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v103'346 uv346 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.998+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.999+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'347 uv347 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:28.999+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.000+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'348 uv348 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.000+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829030 con 0x55fc5c7ec4b0 2023-10-06T05:23:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.002+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'349 uv349 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c7ec4b0 2023-10-06T05:23:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.002+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55fc5c829030 0x55fc5c82c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.002+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c829420 con 0x55fc5c829030 2023-10-06T05:23:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.002+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55fc5c829030 0x55fc5c82c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.002+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55fc5c829030 0x55fc5c82c510 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.004+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'238 uv220 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.004+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82ca50 con 0x55fc5c829030 2023-10-06T05:23:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.005+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v103'239 uv239 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.005+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82ce00 con 0x55fc5c829030 2023-10-06T05:23:29.008 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.006+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v103'240 uv240 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.006+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82d6a0 con 0x55fc5c829030 2023-10-06T05:23:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.008+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v103'241 uv241 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.008+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82da50 con 0x55fc5c829030 2023-10-06T05:23:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.009+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v103'242 uv242 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.009+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82de00 con 0x55fc5c829030 2023-10-06T05:23:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.010+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v103'243 uv225 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.010+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e1b0 con 0x55fc5c829030 2023-10-06T05:23:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.012+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v103'244 uv244 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.012+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.015 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:29.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.013+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v103'245 uv245 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.013+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.014+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v103'246 uv246 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.015+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.016+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v103'247 uv247 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.016+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.017+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v103'248 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.017+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.018+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v103'249 uv249 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.018+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.021 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.019+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v103'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.019+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c831c80 con 0x55fc5c829030 2023-10-06T05:23:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.021+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v103'251 uv251 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.021+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c831c80 con 0x55fc5c829030 2023-10-06T05:23:29.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.023+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v103'252 uv252 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.023+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.024+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v103'253 uv236 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.024+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.025+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v103'254 uv254 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.025+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.028 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.026+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v103'255 uv255 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.026+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82f570 con 0x55fc5c829030 2023-10-06T05:23:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.028+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v103'256 uv256 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.028+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82f570 con 0x55fc5c829030 2023-10-06T05:23:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.029+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v103'257 uv257 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.029+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c82e560 con 0x55fc5c829030 2023-10-06T05:23:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.030+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v103'258 uv258 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.030+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55fc5c82ea70 0x55fc5c830e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.030+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c831390 con 0x55fc5c82ea70 2023-10-06T05:23:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.031+0000 7f2e93f40700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55fc5c82ea70 0x55fc5c830e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.031+0000 7f2e93f40700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55fc5c82ea70 0x55fc5c830e50 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.032+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.032+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c831740 con 0x55fc5c82ea70 2023-10-06T05:23:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.033+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v103'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.034+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c831c40 con 0x55fc5c82ea70 2023-10-06T05:23:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.035+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v103'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.036 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.035+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c833540 con 0x55fc5c82ea70 2023-10-06T05:23:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.036+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.036+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8338f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.037+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.037+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c82ea70 2023-10-06T05:23:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.038+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.038+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c82ea70 2023-10-06T05:23:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.039+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'47 uv46 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.039+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c82ea70 2023-10-06T05:23:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.041+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v103'48 uv48 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.041+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c82ea70 2023-10-06T05:23:29.043 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:29.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.042+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v103'49 uv49 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.042+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832930 con 0x55fc5c82ea70 2023-10-06T05:23:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.043+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v103'50 uv50 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.043+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832930 con 0x55fc5c829030 2023-10-06T05:23:29.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.044+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.044+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832930 con 0x55fc5c829030 2023-10-06T05:23:29.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.046+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v103'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.046+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c829030 2023-10-06T05:23:29.049 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.047+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v103'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.047+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c829030 2023-10-06T05:23:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.048+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.048+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c829030 2023-10-06T05:23:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.050+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.050+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c832530 con 0x55fc5c829030 2023-10-06T05:23:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.051+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c829030 2023-10-06T05:23:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.051+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55fc5c832e40 0x55fc5c835cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.051+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.051+0000 7f2e94f42700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55fc5c832e40 0x55fc5c835cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.052+0000 7f2e94f42700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55fc5c832e40 0x55fc5c835cb0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.053+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v103'116 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.053+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.054+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v103'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.054+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.057 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.055+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v103'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.055+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c837200 con 0x55fc5c832e40 2023-10-06T05:23:29.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.056+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v103'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.057+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c837200 con 0x55fc5c832e40 2023-10-06T05:23:29.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.058+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v103'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.058+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.059+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v103'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.059+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.060+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v103'122 uv115 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.060+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.061+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v103'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.061+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.064 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:29.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.062+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v103'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.063+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.064+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v103'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.064+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8361f0 con 0x55fc5c832e40 2023-10-06T05:23:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.065+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v103'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.065+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55fc5c836870 0x55fc5c838c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.065+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839290 con 0x55fc5c836870 2023-10-06T05:23:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.065+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55fc5c836870 0x55fc5c838c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.066+0000 7f2e94741700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55fc5c836870 0x55fc5c838c50 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.067+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v103'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c836870 2023-10-06T05:23:29.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.067+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c836870 2023-10-06T05:23:29.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.068+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v103'52 uv52 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c836870 2023-10-06T05:23:29.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.068+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c836870 2023-10-06T05:23:29.074 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.069+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v103'53 uv53 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c836870 2023-10-06T05:23:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.069+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c836870 2023-10-06T05:23:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.072+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v103'54 uv54 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c836870 2023-10-06T05:23:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.072+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c836870 2023-10-06T05:23:29.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.074+0000 7f2e94741700 1 -- 172.21.15.45:0/3626853455 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v103'55 uv55 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8401d420 con 0x55fc5c836870 2023-10-06T05:23:29.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.074+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c82ea70 2023-10-06T05:23:29.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.076+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.076+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c82ea70 2023-10-06T05:23:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.077+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v103'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.077+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c82ea70 2023-10-06T05:23:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.078+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v103'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.080 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.078+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83a680 con 0x55fc5c82ea70 2023-10-06T05:23:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.080+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.080+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83a680 con 0x55fc5c82ea70 2023-10-06T05:23:29.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.081+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.081+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c839670 con 0x55fc5c82ea70 2023-10-06T05:23:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.082+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.082+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fc5c839cf0 0x55fc5c83c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.082+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83c740 con 0x55fc5c839cf0 2023-10-06T05:23:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.082+0000 7f2e93f40700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fc5c839cf0 0x55fc5c83c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.083+0000 7f2e93f40700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fc5c839cf0 0x55fc5c83c0d0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.084+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v103'104 uv97 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.084+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83c800 con 0x55fc5c839cf0 2023-10-06T05:23:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.085+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v103'105 uv105 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.085+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83cc30 con 0x55fc5c839cf0 2023-10-06T05:23:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.087+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v103'106 uv106 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.088 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.087+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83cc30 con 0x55fc5c839cf0 2023-10-06T05:23:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.088+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v103'107 uv107 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.088+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83cc30 con 0x55fc5c839cf0 2023-10-06T05:23:29.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.089+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v103'108 uv108 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.089+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83cc30 con 0x55fc5c839cf0 2023-10-06T05:23:29.091 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.090+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v103'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.090+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83cc30 con 0x55fc5c839cf0 2023-10-06T05:23:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.091+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [omap-set-header] v103'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.091+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83f9a0 con 0x55fc5c839cf0 2023-10-06T05:23:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.092+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v103'111 uv111 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c839cf0 2023-10-06T05:23:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.092+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:99 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83d120 con 0x55fc5c82ea70 2023-10-06T05:23:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.094+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(99 mds0_sessionmap [create] v103'49 uv47 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.094+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:100 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83d120 con 0x55fc5c82ea70 2023-10-06T05:23:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.095+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(100 mds0_sessionmap [delete] v103'50 uv50 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.095+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83d120 con 0x55fc5c82ea70 2023-10-06T05:23:29.098 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.096+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(101 mds0_sessionmap [create] v103'51 uv51 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.096+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c83d120 con 0x55fc5c82ea70 2023-10-06T05:23:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.097+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v103'52 uv52 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.097+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55fc5c83d630 0x55fc5c83da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.097+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c83d630 2023-10-06T05:23:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.098+0000 7f2e94f42700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55fc5c83d630 0x55fc5c83da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.098+0000 7f2e94f42700 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55fc5c83d630 0x55fc5c83da40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.099+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c83d630 2023-10-06T05:23:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.099+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c83d630 2023-10-06T05:23:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.100+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v103'64 uv64 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c83d630 2023-10-06T05:23:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.100+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c83d630 2023-10-06T05:23:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.101+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v103'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c83d630 2023-10-06T05:23:29.103 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.101+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c841100 con 0x55fc5c83d630 2023-10-06T05:23:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.102+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c83d630 2023-10-06T05:23:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.102+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c841100 con 0x55fc5c83d630 2023-10-06T05:23:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.103+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c83d630 2023-10-06T05:23:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.104+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c83d630 2023-10-06T05:23:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.105+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c83d630 2023-10-06T05:23:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.105+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c832e40 2023-10-06T05:23:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.106+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v103'52 uv51 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.106+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c832e40 2023-10-06T05:23:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.107+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v103'53 uv53 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.107+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c832e40 2023-10-06T05:23:29.110 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.108+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v103'54 uv54 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.108+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c832e40 2023-10-06T05:23:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.110+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v103'55 uv55 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.110+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c832e40 2023-10-06T05:23:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.111+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v103'56 uv56 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2e8c0025a0 con 0x55fc5c832e40 2023-10-06T05:23:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.111+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.112+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(114 100.00000000.inode [create] v103'132 uv123 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.112+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.113+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(115 100.00000000.inode [delete] v103'133 uv133 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.113+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.118 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.116+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(116 100.00000000.inode [create] v103'134 uv134 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.116+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.117+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v103'135 uv135 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.117+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8404f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.123+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v103'136 uv136 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.123+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8404f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.125+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(119 1.00000000 [create] v103'137 uv130 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.125+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.128+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(120 1.00000000 [delete] v103'138 uv138 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.128+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8400f0 con 0x55fc5c82ea70 2023-10-06T05:23:29.132 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:29.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.130+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(121 1.00000000 [create] v103'139 uv139 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.130+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c841100 con 0x55fc5c82ea70 2023-10-06T05:23:29.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.133+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v103'140 uv140 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.133+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c841100 con 0x55fc5c82ea70 2023-10-06T05:23:29.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.135+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v103'141 uv141 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.135+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8405e0 con 0x55fc5c82ea70 2023-10-06T05:23:29.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.138+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(124 1.00000000 [omap-set-header] v103'142 uv142 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.138+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) v8 -- 0x55fc5c8405e0 con 0x55fc5c82ea70 2023-10-06T05:23:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e93f40700 1 -- 172.21.15.45:0/3626853455 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v103'143 uv143 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2e8801b040 con 0x55fc5c82ea70 2023-10-06T05:23:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55fc5c82ea70 msgr2=0x55fc5c830e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55fc5c82ea70 0x55fc5c830e50 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55fc5c7ec4b0 msgr2=0x55fc5c7ed870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55fc5c7ec4b0 0x55fc5c7ed870 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fc5c839cf0 msgr2=0x55fc5c83c0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55fc5c839cf0 0x55fc5c83c0d0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55fc5c829030 msgr2=0x55fc5c82c510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55fc5c829030 0x55fc5c82c510 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55fc5c83d630 msgr2=0x55fc5c83da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55fc5c83d630 0x55fc5c83da40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55fc5c836870 msgr2=0x55fc5c838c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55fc5c836870 0x55fc5c838c50 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 reap_dead start 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55fc5c832e40 msgr2=0x55fc5c835cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55fc5c832e40 0x55fc5c835cb0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 reap_dead start 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e94f42700 1 -- 172.21.15.45:0/3626853455 reap_dead start 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e80065890 msgr2=0x7f2e80067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e80065890 0x7f2e80067d40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2e84008fd0 tx=0x7f2e84016000 comp rx=0 tx=0).stop 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 msgr2=0x55fc5c7eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7eb930 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2e88005e70 tx=0x7f2e88017000 comp rx=0 tx=0).stop 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 shutdown_connections 2023-10-06T05:23:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc5c7785f0 0x55fc5c7eb930 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 --2- 172.21.15.45:0/3626853455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2e80065890 0x7f2e80067d40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 >> 172.21.15.45:0/3626853455 conn(0x55fc5c7354a0 msgr2=0x55fc5c762260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 shutdown_connections 2023-10-06T05:23:29.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.141+0000 7f2e9b523040 1 -- 172.21.15.45:0/3626853455 wait complete. 2023-10-06T05:23:29.147 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2023-10-06T05:23:29.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.236+0000 7f7250435040 1 Processor -- start 2023-10-06T05:23:29.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.236+0000 7f7250435040 1 -- start start 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7250435040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e69af80 0x56292e66ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7250435040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e72b900 0x56292e66c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7250435040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72bcd0 0x56292e7286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7250435040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e5b6980 con 0x56292e72b900 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7250435040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e5b6620 con 0x56292e69af80 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7250435040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56292e5ad400 con 0x56292e72bcd0 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e72b900 0x56292e66c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7249e54700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72bcd0 0x56292e7286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7249653700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e69af80 0x56292e66ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e72b900 0x56292e66c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60844/0 (socket says 172.21.15.45:60844) 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 -- 172.21.15.45:0/1796175816 learned_addr learned my addr 172.21.15.45:0/1796175816 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 -- 172.21.15.45:0/1796175816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72bcd0 msgr2=0x56292e7286d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 --2- 172.21.15.45:0/1796175816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72bcd0 0x56292e7286d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 -- 172.21.15.45:0/1796175816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e69af80 msgr2=0x56292e66ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 --2- 172.21.15.45:0/1796175816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e69af80 0x56292e66ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 -- 172.21.15.45:0/1796175816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724400a180 con 0x56292e72b900 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.237+0000 7f7248e52700 1 --2- 172.21.15.45:0/1796175816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e72b900 0x56292e66c680 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7234002e60 tx=0x7f72340028b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ab2bbbaedc94bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f72437fe700 1 -- 172.21.15.45:0/1796175816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400baa0 con 0x56292e72b900 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f72437fe700 1 -- 172.21.15.45:0/1796175816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234008030 con 0x56292e72b900 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f72437fe700 1 -- 172.21.15.45:0/1796175816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400be70 con 0x56292e72b900 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 -- 172.21.15.45:0/1796175816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e72b900 msgr2=0x56292e66c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 --2- 172.21.15.45:0/1796175816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e72b900 0x56292e66c680 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7234002e60 tx=0x7f72340028b0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 -- 172.21.15.45:0/1796175816 shutdown_connections 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 --2- 172.21.15.45:0/1796175816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72bcd0 0x56292e7286d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 --2- 172.21.15.45:0/1796175816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e69af80 0x56292e66ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 --2- 172.21.15.45:0/1796175816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e72b900 0x56292e66c680 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 -- 172.21.15.45:0/1796175816 >> 172.21.15.45:0/1796175816 conn(0x56292e66d5f0 msgr2=0x56292e66f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 -- 172.21.15.45:0/1796175816 shutdown_connections 2023-10-06T05:23:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.238+0000 7f7250435040 1 -- 172.21.15.45:0/1796175816 wait complete. 2023-10-06T05:23:29.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.239+0000 7f7250435040 1 Processor -- start 2023-10-06T05:23:29.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.239+0000 7f7250435040 1 -- start start 2023-10-06T05:23:29.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7250435040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e69af80 0x56292e727640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7250435040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72b900 0x56292e68e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7250435040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e72bcd0 0x56292e68edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7250435040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e6b2910 con 0x56292e69af80 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7250435040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e6b2bb0 con 0x56292e72bcd0 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7250435040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56292e6b2e50 con 0x56292e72b900 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e69af80 0x56292e727640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e69af80 0x56292e727640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60852/0 (socket says 172.21.15.45:60852) 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 -- 172.21.15.45:0/1789985369 learned_addr learned my addr 172.21.15.45:0/1789985369 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249e54700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e72bcd0 0x56292e68edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7248e52700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72b900 0x56292e68e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 -- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72b900 msgr2=0x56292e68e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56292e72b900 0x56292e68e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 -- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e72bcd0 msgr2=0x56292e68edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56292e72bcd0 0x56292e68edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 -- 172.21.15.45:0/1789985369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723400a040 con 0x56292e69af80 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.240+0000 7f7249653700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e69af80 0x56292e727640 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f723c008b50 tx=0x7f723c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.241+0000 7f7241ffb700 1 -- 172.21.15.45:0/1789985369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00f050 con 0x56292e69af80 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.241+0000 7f7241ffb700 1 -- 172.21.15.45:0/1789985369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c010040 con 0x56292e69af80 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.241+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56292e5c53d0 con 0x56292e69af80 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.241+0000 7f7241ffb700 1 -- 172.21.15.45:0/1789985369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c006c30 con 0x56292e69af80 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.241+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56292e5b8ae0 con 0x56292e69af80 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.242+0000 7f7241ffb700 1 -- 172.21.15.45:0/1789985369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f723c015670 con 0x56292e69af80 2023-10-06T05:23:29.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.243+0000 7f7241ffb700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f722c065890 0x7f722c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.243+0000 7f7241ffb700 1 -- 172.21.15.45:0/1789985369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f723c083f20 con 0x56292e69af80 2023-10-06T05:23:29.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.243+0000 7f7248e52700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f722c065890 0x7f722c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.243+0000 7f7250435040 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56292e6b5c90 0x56292e6b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.243+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) v8 -- 0x56292e6b3830 con 0x56292e6b5c90 2023-10-06T05:23:29.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.243+0000 7f7249e54700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56292e6b5c90 0x56292e6b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.244+0000 7f7248e52700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f722c065890 0x7f722c067d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7234002910 tx=0x7f7234017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.244+0000 7f7249e54700 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56292e6b5c90 0x56292e6b1be0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7249e54700 1 -- 172.21.15.45:0/1789985369 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v103'53 uv53 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7244017040 con 0x56292e6b5c90 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56292e6b5c90 msgr2=0x56292e6b1be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56292e6b5c90 0x56292e6b1be0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f722c065890 msgr2=0x7f722c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f722c065890 0x7f722c067d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7234002910 tx=0x7f7234017000 comp rx=0 tx=0).stop 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e69af80 msgr2=0x56292e727640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 --2- 172.21.15.45:0/1789985369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56292e69af80 0x56292e727640 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f723c008b50 tx=0x7f723c005be0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7249e54700 1 -- 172.21.15.45:0/1789985369 reap_dead start 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 shutdown_connections 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 >> 172.21.15.45:0/1789985369 conn(0x56292e66d5f0 msgr2=0x56292e743790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 shutdown_connections 2023-10-06T05:23:29.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.245+0000 7f7250435040 1 -- 172.21.15.45:0/1789985369 wait complete. 2023-10-06T05:23:29.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.391+0000 7f74733b7700 1 Processor -- start 2023-10-06T05:23:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.391+0000 7f74733b7700 1 -- start start 2023-10-06T05:23:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f74733b7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c0ccb70 0x7f746c0aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f74733b7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f746c0c8b70 0x7f746c0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f74733b7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746c0c3b50 con 0x7f746c157330 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f74733b7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746c157700 con 0x7f746c0c8b70 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f74733b7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0c98e0 con 0x7f746c0ccb70 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7471153700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f746c157330 0x7f746c0ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48666/0 (socket says 172.21.15.45:48666) 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7471153700 1 -- 172.21.15.45:0/2758613217 learned_addr learned my addr 172.21.15.45:0/2758613217 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7470952700 1 --2- 172.21.15.45:0/2758613217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c0ccb70 0x7f746c0aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7471954700 1 --2- 172.21.15.45:0/2758613217 >> v2:172.21.15.144:3300/0 conn(0x7f746c0c8b70 0x7f746c0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f746bfff700 1 -- 172.21.15.45:0/2758613217 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489760349 0 0) 0x7f746c0c3b50 con 0x7f746c157330 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f746bfff700 1 -- 172.21.15.45:0/2758613217 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7454003570 con 0x7f746c157330 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7470952700 1 -- 172.21.15.45:0/2758613217 >> v2:172.21.15.144:3300/0 conn(0x7f746c0c8b70 msgr2=0x7f746c0c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7470952700 1 --2- 172.21.15.45:0/2758613217 >> v2:172.21.15.144:3300/0 conn(0x7f746c0c8b70 0x7f746c0c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7470952700 1 -- 172.21.15.45:0/2758613217 >> v1:172.21.15.45:6789/0 conn(0x7f746c157330 legacy=0x7f746c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7470952700 1 -- 172.21.15.45:0/2758613217 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0c9bc0 con 0x7f746c0ccb70 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.392+0000 7f7471954700 1 --2- 172.21.15.45:0/2758613217 >> v2:172.21.15.144:3300/0 conn(0x7f746c0c8b70 0x7f746c0c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f7470952700 1 --2- 172.21.15.45:0/2758613217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c0ccb70 0x7f746c0aaae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74600098e0 tx=0x7f74600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563390c94e0055c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f746bfff700 1 -- 172.21.15.45:0/2758613217 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460011630 con 0x7f746c0ccb70 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f746bfff700 1 -- 172.21.15.45:0/2758613217 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7460011790 con 0x7f746c0ccb70 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f746bfff700 1 -- 172.21.15.45:0/2758613217 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460011b30 con 0x7f746c0ccb70 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 -- 172.21.15.45:0/2758613217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c0ccb70 msgr2=0x7f746c0aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 --2- 172.21.15.45:0/2758613217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c0ccb70 0x7f746c0aaae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74600098e0 tx=0x7f74600091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 -- 172.21.15.45:0/2758613217 shutdown_connections 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 --2- 172.21.15.45:0/2758613217 >> v2:172.21.15.144:3300/0 conn(0x7f746c0c8b70 0x7f746c0c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 --2- 172.21.15.45:0/2758613217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c0ccb70 0x7f746c0aaae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 -- 172.21.15.45:0/2758613217 >> 172.21.15.45:0/2758613217 conn(0x7f746c067e00 msgr2=0x7f746c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 -- 172.21.15.45:0/2758613217 shutdown_connections 2023-10-06T05:23:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.393+0000 7f74733b7700 1 -- 172.21.15.45:0/2758613217 wait complete. 2023-10-06T05:23:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.395+0000 7f74733b7700 1 Processor -- start 2023-10-06T05:23:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.395+0000 7f74733b7700 1 -- start start 2023-10-06T05:23:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.396+0000 7f74733b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f746c0c8b70 0x7f746c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.396+0000 7f74733b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0ccb70 0x7f746c068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.396+0000 7f74733b7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c157330 0x7f746c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.396+0000 7f74733b7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0b0940 con 0x7f746c0ccb70 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.396+0000 7f74733b7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c152fa0 con 0x7f746c0c8b70 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.396+0000 7f74733b7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c0b1e60 con 0x7f746c157330 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7470952700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0ccb70 0x7f746c068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7470952700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0ccb70 0x7f746c068730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60864/0 (socket says 172.21.15.45:60864) 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7470952700 1 -- 172.21.15.45:0/2381838932 learned_addr learned my addr 172.21.15.45:0/2381838932 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471153700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f746c0c8b70 0x7f746c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471954700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c157330 0x7f746c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471153700 1 -- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c157330 msgr2=0x7f746c152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471153700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c157330 0x7f746c152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471153700 1 -- 172.21.15.45:0/2381838932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0ccb70 msgr2=0x7f746c068730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471153700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0ccb70 0x7f746c068730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471153700 1 -- 172.21.15.45:0/2381838932 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464009a20 con 0x7f746c0c8b70 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7470952700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0ccb70 0x7f746c068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f7471153700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f746c0c8b70 0x7f746c06a3b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c7c0 tx=0x7f745c007bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f74697fa700 1 -- 172.21.15.45:0/2381838932 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c0072e0 con 0x7f746c0c8b70 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f74697fa700 1 -- 172.21.15.45:0/2381838932 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c007460 con 0x7f746c0c8b70 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7460009420 con 0x7f746c0c8b70 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f74697fa700 1 -- 172.21.15.45:0/2381838932 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c0057f0 con 0x7f746c0c8b70 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.397+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c0b18e0 con 0x7f746c0c8b70 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.398+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c0b3d40 con 0x7f746c0c8b70 2023-10-06T05:23:29.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.399+0000 7f74697fa700 1 -- 172.21.15.45:0/2381838932 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f745c005c20 con 0x7f746c0c8b70 2023-10-06T05:23:29.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.399+0000 7f74697fa700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7458066f30 0x7f74580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.399+0000 7f74697fa700 1 -- 172.21.15.45:0/2381838932 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f745c083dc0 con 0x7f746c0c8b70 2023-10-06T05:23:29.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.399+0000 7f7470952700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7458066f30 0x7f74580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.400+0000 7f7470952700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7458066f30 0x7f74580693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f746c06a280 tx=0x7f7460017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.404+0000 7f74697fa700 1 -- 172.21.15.45:0/2381838932 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f745c04b5a0 con 0x7f746c0c8b70 2023-10-06T05:23:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.652+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f746c06e350 con 0x7f746c0c8b70 2023-10-06T05:23:29.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.897+0000 7f74697fa700 1 -- 172.21.15.45:0/2381838932 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v159) v1 ==== 162+0+0 (secure 0 0 0) 0x7f745c017020 con 0x7f746c0c8b70 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7458066f30 msgr2=0x7f74580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7458066f30 0x7f74580693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f746c06a280 tx=0x7f7460017000 comp rx=0 tx=0).stop 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f746c0c8b70 msgr2=0x7f746c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f746c0c8b70 0x7f746c06a3b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c7c0 tx=0x7f745c007bc0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 shutdown_connections 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7458066f30 0x7f74580693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f746c157330 0x7f746c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f746c0c8b70 0x7f746c06a3b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 --2- 172.21.15.45:0/2381838932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0ccb70 0x7f746c068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 >> 172.21.15.45:0/2381838932 conn(0x7f746c067e00 msgr2=0x7f746c0cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 shutdown_connections 2023-10-06T05:23:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:29.900+0000 7f74733b7700 1 -- 172.21.15.45:0/2381838932 wait complete. 2023-10-06T05:23:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:29.906 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:29.904+0000 7f722dde0700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-10-06T05:23:29.906 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:29.904+0000 7f722dde0700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-10-06T05:23:29.907 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:29.905+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:29.907 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:29.905+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:23:29.911 INFO:tasks.ceph.mds.c.smithi045.stderr: -32> 2023-10-06T05:23:29.904+0000 7f722dde0700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-10-06T05:23:29.912 INFO:tasks.ceph.mds.c.smithi045.stderr: -31> 2023-10-06T05:23:29.904+0000 7f722dde0700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-10-06T05:23:29.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:29.935 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:30.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.054+0000 7f7692ac4700 1 Processor -- start 2023-10-06T05:23:30.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.054+0000 7f7692ac4700 1 -- start start 2023-10-06T05:23:30.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7692ac4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f768c0ca810 0x7f768c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7692ac4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768c0c84c0 con 0x7f768c0c91f0 2023-10-06T05:23:30.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7692ac4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768c0c8d60 con 0x7f768c0c9640 2023-10-06T05:23:30.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7692ac4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c069d00 con 0x7f768c0ca810 2023-10-06T05:23:30.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7690860700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f768c0ca810 0x7f768c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7690860700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f768c0ca810 0x7f768c0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60882/0 (socket says 172.21.15.45:60882) 2023-10-06T05:23:30.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7690860700 1 -- 172.21.15.45:0/916878476 learned_addr learned my addr 172.21.15.45:0/916878476 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:30.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f768bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f768c0c91f0 0x7f768c0afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48670/0 (socket says 172.21.15.45:48670) 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7690860700 1 -- 172.21.15.45:0/916878476 >> v1:172.21.15.144:6789/0 conn(0x7f768c0c9640 legacy=0x7f768c068fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7690860700 1 -- 172.21.15.45:0/916878476 >> v1:172.21.15.45:6789/0 conn(0x7f768c0c91f0 legacy=0x7f768c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7690860700 1 -- 172.21.15.45:0/916878476 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0c8710 con 0x7f768c0ca810 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f768b7fe700 1 -- 172.21.15.45:0/916878476 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255828960 0 0) 0x7f768c0c84c0 con 0x7f768c0c91f0 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.055+0000 7f7690860700 1 --2- 172.21.15.45:0/916878476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0ca810 0x7f768c0cabe0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f767c0099e0 tx=0x7f767c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb0533e05933630 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f768b7fe700 1 -- 172.21.15.45:0/916878476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c00b7e0 con 0x7f768c0ca810 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f768b7fe700 1 -- 172.21.15.45:0/916878476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c00b940 con 0x7f768c0ca810 2023-10-06T05:23:30.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f768b7fe700 1 -- 172.21.15.45:0/916878476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c00bce0 con 0x7f768c0ca810 2023-10-06T05:23:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f7692ac4700 1 -- 172.21.15.45:0/916878476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0ca810 msgr2=0x7f768c0cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f7692ac4700 1 --2- 172.21.15.45:0/916878476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0ca810 0x7f768c0cabe0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f767c0099e0 tx=0x7f767c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f7692ac4700 1 -- 172.21.15.45:0/916878476 shutdown_connections 2023-10-06T05:23:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f7692ac4700 1 --2- 172.21.15.45:0/916878476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0ca810 0x7f768c0cabe0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f7692ac4700 1 -- 172.21.15.45:0/916878476 >> 172.21.15.45:0/916878476 conn(0x7f768c067620 msgr2=0x7f768c067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f7692ac4700 1 -- 172.21.15.45:0/916878476 shutdown_connections 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.056+0000 7f7692ac4700 1 -- 172.21.15.45:0/916878476 wait complete. 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.057+0000 7f7692ac4700 1 Processor -- start 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.058+0000 7f7692ac4700 1 -- start start 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.058+0000 7f7692ac4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0c91f0 0x7f768c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.058+0000 7f7692ac4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768c0c9640 0x7f768c16a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.058+0000 7f7692ac4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768c0ca810 0x7f768c0c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.058+0000 7f7692ac4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0c7c50 con 0x7f768c0c91f0 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.058+0000 7f7692ac4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0c5470 con 0x7f768c0c9640 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.058+0000 7f7692ac4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0c78b0 con 0x7f768c0ca810 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0c91f0 0x7f768c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0c91f0 0x7f768c0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60888/0 (socket says 172.21.15.45:60888) 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 -- 172.21.15.45:0/2013461791 learned_addr learned my addr 172.21.15.45:0/2013461791 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f768bfff700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768c0c9640 0x7f768c16a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7691061700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768c0ca810 0x7f768c0c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 -- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768c0ca810 msgr2=0x7f768c0c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768c0ca810 0x7f768c0c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 -- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768c0c9640 msgr2=0x7f768c16a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768c0c9640 0x7f768c16a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 -- 172.21.15.45:0/2013461791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0094f0 con 0x7f768c0c91f0 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f768bfff700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768c0c9640 0x7f768c16a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7691061700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768c0ca810 0x7f768c0c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7690860700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0c91f0 0x7f768c0b26d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f767c0099b0 tx=0x7f767c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7688ff9700 1 -- 172.21.15.45:0/2013461791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c01b450 con 0x7f768c0c91f0 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7688ff9700 1 -- 172.21.15.45:0/2013461791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c020070 con 0x7f768c0c91f0 2023-10-06T05:23:30.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c0c6fd0 con 0x7f768c0c91f0 2023-10-06T05:23:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7688ff9700 1 -- 172.21.15.45:0/2013461791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c01b820 con 0x7f768c0c91f0 2023-10-06T05:23:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.059+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c0c7590 con 0x7f768c0c91f0 2023-10-06T05:23:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.060+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c0c7d90 con 0x7f768c0c91f0 2023-10-06T05:23:30.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.061+0000 7f7688ff9700 1 -- 172.21.15.45:0/2013461791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f767c022030 con 0x7f768c0c91f0 2023-10-06T05:23:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.061+0000 7f7688ff9700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f766c066f30 0x7f766c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.061+0000 7f7688ff9700 1 -- 172.21.15.45:0/2013461791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f767c086b10 con 0x7f768c0c91f0 2023-10-06T05:23:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.062+0000 7f768bfff700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f766c066f30 0x7f766c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.062+0000 7f768bfff700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f766c066f30 0x7f766c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f768c0ad200 tx=0x7f7674009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.066+0000 7f7688ff9700 1 -- 172.21.15.45:0/2013461791 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f768c0c7d90 con 0x7f768c0c91f0 2023-10-06T05:23:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.313+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f768c0681e0 con 0x7f768c0c91f0 2023-10-06T05:23:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.314+0000 7f7688ff9700 1 -- 172.21.15.45:0/2013461791 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f767c0536d0 con 0x7f768c0c91f0 2023-10-06T05:23:30.316 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:30.316 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":161,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:29.906286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f766c066f30 msgr2=0x7f766c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f766c066f30 0x7f766c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f768c0ad200 tx=0x7f7674009040 comp rx=0 tx=0).stop 2023-10-06T05:23:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0c91f0 msgr2=0x7f768c0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0c91f0 0x7f768c0b26d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f767c0099b0 tx=0x7f767c016000 comp rx=0 tx=0).stop 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 shutdown_connections 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f766c066f30 0x7f766c0693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768c0ca810 0x7f768c0c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f768c0c91f0 0x7f768c0b26d0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 --2- 172.21.15.45:0/2013461791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f768c0c9640 0x7f768c16a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 >> 172.21.15.45:0/2013461791 conn(0x7f768c067620 msgr2=0x7f768c068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 shutdown_connections 2023-10-06T05:23:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.316+0000 7f7692ac4700 1 -- 172.21.15.45:0/2013461791 wait complete. 2023-10-06T05:23:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 161 2023-10-06T05:23:30.332 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:30.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:30.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.504+0000 7fb63f947700 1 Processor -- start 2023-10-06T05:23:30.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.504+0000 7fb63f947700 1 -- start start 2023-10-06T05:23:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63f947700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb638158f60 0x7fb638159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63f947700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb63806a060 0x7fb6380cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63f947700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb638069bd0 con 0x7fb6380cc390 2023-10-06T05:23:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63f947700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63806a500 con 0x7fb638158f60 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63f947700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb638068fc0 con 0x7fb63806a060 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63d6e3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb638158f60 0x7fb638159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63dee4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb6380cc390 0x7fb6380c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48678/0 (socket says 172.21.15.45:48678) 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63d6e3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb638158f60 0x7fb638159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60900/0 (socket says 172.21.15.45:60900) 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63dee4700 1 -- 172.21.15.45:0/2070357399 learned_addr learned my addr 172.21.15.45:0/2070357399 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63cee2700 1 --2- 172.21.15.45:0/2070357399 >> v2:172.21.15.144:3300/0 conn(0x7fb63806a060 0x7fb6380cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63d6e3700 1 -- 172.21.15.45:0/2070357399 >> v2:172.21.15.144:3300/0 conn(0x7fb63806a060 msgr2=0x7fb6380cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63d6e3700 1 --2- 172.21.15.45:0/2070357399 >> v2:172.21.15.144:3300/0 conn(0x7fb63806a060 0x7fb6380cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63d6e3700 1 -- 172.21.15.45:0/2070357399 >> v1:172.21.15.45:6789/0 conn(0x7fb6380cc390 legacy=0x7fb6380c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb63d6e3700 1 -- 172.21.15.45:0/2070357399 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380692a0 con 0x7fb638158f60 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.505+0000 7fb62ffff700 1 -- 172.21.15.45:0/2070357399 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499360372 0 0) 0x7fb638069bd0 con 0x7fb6380cc390 2023-10-06T05:23:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63d6e3700 1 --2- 172.21.15.45:0/2070357399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb638158f60 0x7fb638159330 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fb6380872e0 tx=0x7fb6340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d742284b13ccbc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb62ffff700 1 -- 172.21.15.45:0/2070357399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400b6b0 con 0x7fb638158f60 2023-10-06T05:23:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb62ffff700 1 -- 172.21.15.45:0/2070357399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63400b810 con 0x7fb638158f60 2023-10-06T05:23:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb62ffff700 1 -- 172.21.15.45:0/2070357399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400bbb0 con 0x7fb638158f60 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 -- 172.21.15.45:0/2070357399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb638158f60 msgr2=0x7fb638159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 --2- 172.21.15.45:0/2070357399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb638158f60 0x7fb638159330 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fb6380872e0 tx=0x7fb6340091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 -- 172.21.15.45:0/2070357399 shutdown_connections 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 --2- 172.21.15.45:0/2070357399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb638158f60 0x7fb638159330 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 --2- 172.21.15.45:0/2070357399 >> v2:172.21.15.144:3300/0 conn(0x7fb63806a060 0x7fb6380cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 -- 172.21.15.45:0/2070357399 >> 172.21.15.45:0/2070357399 conn(0x7fb6380cdf50 msgr2=0x7fb6380ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 -- 172.21.15.45:0/2070357399 shutdown_connections 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.506+0000 7fb63f947700 1 -- 172.21.15.45:0/2070357399 wait complete. 2023-10-06T05:23:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.508+0000 7fb63f947700 1 Processor -- start 2023-10-06T05:23:30.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.508+0000 7fb63f947700 1 -- start start 2023-10-06T05:23:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63f947700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63806a060 0x7fb6380b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63f947700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6380cc390 0x7fb638147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63f947700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb638158f60 0x7fb63815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63f947700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380b3450 con 0x7fb6380cc390 2023-10-06T05:23:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63f947700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638147f50 con 0x7fb63806a060 2023-10-06T05:23:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63f947700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63815bb70 con 0x7fb638158f60 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6380cc390 0x7fb638147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6380cc390 0x7fb638147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60904/0 (socket says 172.21.15.45:60904) 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 -- 172.21.15.45:0/2713238221 learned_addr learned my addr 172.21.15.45:0/2713238221 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63d6e3700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63806a060 0x7fb6380b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63dee4700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb638158f60 0x7fb63815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 -- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb638158f60 msgr2=0x7fb63815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb638158f60 0x7fb63815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 -- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63806a060 msgr2=0x7fb6380b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63806a060 0x7fb6380b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 -- 172.21.15.45:0/2713238221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634009420 con 0x7fb6380cc390 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb63cee2700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6380cc390 0x7fb638147a10 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb62800d8c0 tx=0x7fb628008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.509+0000 7fb62d7fa700 1 -- 172.21.15.45:0/2713238221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6280125f0 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.510+0000 7fb62d7fa700 1 -- 172.21.15.45:0/2713238221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb628012750 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.510+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63815be50 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.510+0000 7fb62d7fa700 1 -- 172.21.15.45:0/2713238221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb628012af0 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.510+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63815c410 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.510+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63806f410 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.511+0000 7fb62d7fa700 1 -- 172.21.15.45:0/2713238221 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb62801e460 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.511+0000 7fb62d7fa700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb61c066f30 0x7fb61c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.511+0000 7fb62d7fa700 1 -- 172.21.15.45:0/2713238221 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fb628085210 con 0x7fb6380cc390 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.512+0000 7fb63d6e3700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb61c066f30 0x7fb61c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.512+0000 7fb63d6e3700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb61c066f30 0x7fb61c0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb6340093f0 tx=0x7fb634017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.516+0000 7fb62d7fa700 1 -- 172.21.15.45:0/2713238221 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6280557c0 con 0x7fb6380cc390 2023-10-06T05:23:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.765+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb638069580 con 0x7fb6380cc390 2023-10-06T05:23:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.765+0000 7fb62d7fa700 1 -- 172.21.15.45:0/2713238221 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fb628055350 con 0x7fb6380cc390 2023-10-06T05:23:30.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:30.767 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":161,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:29.906286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb61c066f30 msgr2=0x7fb61c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb61c066f30 0x7fb61c0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb6340093f0 tx=0x7fb634017000 comp rx=0 tx=0).stop 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6380cc390 msgr2=0x7fb638147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6380cc390 0x7fb638147a10 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb62800d8c0 tx=0x7fb628008040 comp rx=0 tx=0).stop 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 shutdown_connections 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb61c066f30 0x7fb61c0693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb638158f60 0x7fb63815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb63806a060 0x7fb6380b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 --2- 172.21.15.45:0/2713238221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6380cc390 0x7fb638147a10 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 >> 172.21.15.45:0/2713238221 conn(0x7fb6380cdf50 msgr2=0x7fb6380ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 shutdown_connections 2023-10-06T05:23:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.768+0000 7fb63f947700 1 -- 172.21.15.45:0/2713238221 wait complete. 2023-10-06T05:23:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 161 2023-10-06T05:23:30.783 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:30.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:30.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.919+0000 7f4de63a2700 1 Processor -- start 2023-10-06T05:23:30.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.919+0000 7f4de63a2700 1 -- start start 2023-10-06T05:23:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.920+0000 7f4de63a2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4de0159200 0x7f4de00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.920+0000 7f4de63a2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4de01595d0 0x7f4de006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.920+0000 7f4de63a2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de006b5c0 0x7f4de00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.920+0000 7f4de63a2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de00c7780 con 0x7f4de0159200 2023-10-06T05:23:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.920+0000 7f4de63a2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de00c7d00 con 0x7f4de01595d0 2023-10-06T05:23:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.920+0000 7f4de63a2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00c86d0 con 0x7f4de006b5c0 2023-10-06T05:23:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4de0159200 0x7f4de00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4de0159200 0x7f4de00c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60914/0 (socket says 172.21.15.45:60914) 2023-10-06T05:23:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 -- 172.21.15.45:0/1708365713 learned_addr learned my addr 172.21.15.45:0/1708365713 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddf7fe700 1 --2- 172.21.15.45:0/1708365713 >> v2:172.21.15.144:3300/0 conn(0x7f4de01595d0 0x7f4de006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4de493f700 1 --2- 172.21.15.45:0/1708365713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de006b5c0 0x7f4de00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 -- 172.21.15.45:0/1708365713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de006b5c0 msgr2=0x7f4de00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 --2- 172.21.15.45:0/1708365713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de006b5c0 0x7f4de00ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 -- 172.21.15.45:0/1708365713 >> v2:172.21.15.144:3300/0 conn(0x7f4de01595d0 msgr2=0x7f4de006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 --2- 172.21.15.45:0/1708365713 >> v2:172.21.15.144:3300/0 conn(0x7f4de01595d0 0x7f4de006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 -- 172.21.15.45:0/1708365713 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de006bb40 con 0x7f4de0159200 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddffff700 1 --2- 172.21.15.45:0/1708365713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de0159200 0x7f4de00c8190 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4de006e790 tx=0x7f4dd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68131d55dd8aee36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddeffd700 1 -- 172.21.15.45:0/1708365713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000b640 con 0x7f4de0159200 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddeffd700 1 -- 172.21.15.45:0/1708365713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd000b7a0 con 0x7f4de0159200 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.921+0000 7f4ddeffd700 1 -- 172.21.15.45:0/1708365713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000bb60 con 0x7f4de0159200 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 -- 172.21.15.45:0/1708365713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de0159200 msgr2=0x7f4de00c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1708365713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de0159200 0x7f4de00c8190 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4de006e790 tx=0x7f4dd0009250 comp rx=0 tx=0).stop 2023-10-06T05:23:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 -- 172.21.15.45:0/1708365713 shutdown_connections 2023-10-06T05:23:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1708365713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de006b5c0 0x7f4de00ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1708365713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de0159200 0x7f4de00c8190 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1708365713 >> v2:172.21.15.144:3300/0 conn(0x7f4de01595d0 0x7f4de006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 -- 172.21.15.45:0/1708365713 >> 172.21.15.45:0/1708365713 conn(0x7f4de006f090 msgr2=0x7f4de006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 -- 172.21.15.45:0/1708365713 shutdown_connections 2023-10-06T05:23:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.922+0000 7f4de63a2700 1 -- 172.21.15.45:0/1708365713 wait complete. 2023-10-06T05:23:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.924+0000 7f4de63a2700 1 Processor -- start 2023-10-06T05:23:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.924+0000 7f4de63a2700 1 -- start start 2023-10-06T05:23:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de63a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de006b5c0 0x7f4de00b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de63a2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de0159200 0x7f4de00c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de63a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de01595d0 0x7f4de00c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de63a2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00c4de0 con 0x7f4de01595d0 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de63a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00c58d0 con 0x7f4de006b5c0 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de63a2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00c4b00 con 0x7f4de0159200 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de493f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de01595d0 0x7f4de00c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de493f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de01595d0 0x7f4de00c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60930/0 (socket says 172.21.15.45:60930) 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4de493f700 1 -- 172.21.15.45:0/1474578500 learned_addr learned my addr 172.21.15.45:0/1474578500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4ddffff700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de006b5c0 0x7f4de00b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.925+0000 7f4ddf7fe700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de0159200 0x7f4de00c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de493f700 1 -- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de0159200 msgr2=0x7f4de00c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de493f700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de0159200 0x7f4de00c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de493f700 1 -- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de006b5c0 msgr2=0x7f4de00b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de493f700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de006b5c0 0x7f4de00b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de493f700 1 -- 172.21.15.45:0/1474578500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00094f0 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de493f700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de01595d0 0x7f4de00c2f20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009e90 tx=0x7f4dd800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4dd7fff700 1 -- 172.21.15.45:0/1474578500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800cb70 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4dd7fff700 1 -- 172.21.15.45:0/1474578500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd800f040 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4dd7fff700 1 -- 172.21.15.45:0/1474578500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8014480 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de00c5280 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.926+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de00b2ad0 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.927+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de00c5a40 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.928+0000 7f4dd7fff700 1 -- 172.21.15.45:0/1474578500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4dd8014830 con 0x7f4de01595d0 2023-10-06T05:23:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.928+0000 7f4dd7fff700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4dc0066f30 0x7f4dc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.928+0000 7f4dd7fff700 1 -- 172.21.15.45:0/1474578500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f4dd80855b0 con 0x7f4de01595d0 2023-10-06T05:23:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.928+0000 7f4ddffff700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4dc0066f30 0x7f4dc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.929+0000 7f4ddffff700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4dc0066f30 0x7f4dc00693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4de006e790 tx=0x7f4dd0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:30.932+0000 7f4dd7fff700 1 -- 172.21.15.45:0/1474578500 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dd8052170 con 0x7f4de01595d0 2023-10-06T05:23:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.184+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4de006be20 con 0x7f4de01595d0 2023-10-06T05:23:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.184+0000 7f4dd7fff700 1 -- 172.21.15.45:0/1474578500 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4599 (secure 0 0 0) 0x7f4dd8055790 con 0x7f4de01595d0 2023-10-06T05:23:31.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:31.186 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":162,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2405580153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2405580153},{"type":"v1","addr":"172.21.15.45:6835","nonce":2405580153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:29.906286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4dc0066f30 msgr2=0x7f4dc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4dc0066f30 0x7f4dc00693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4de006e790 tx=0x7f4dd0017000 comp rx=0 tx=0).stop 2023-10-06T05:23:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de01595d0 msgr2=0x7f4de00c2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de01595d0 0x7f4de00c2f20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009e90 tx=0x7f4dd800b040 comp rx=0 tx=0).stop 2023-10-06T05:23:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 shutdown_connections 2023-10-06T05:23:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4dc0066f30 0x7f4dc00693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de01595d0 0x7f4de00c2f20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de006b5c0 0x7f4de00b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 --2- 172.21.15.45:0/1474578500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4de0159200 0x7f4de00c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 >> 172.21.15.45:0/1474578500 conn(0x7f4de006f090 msgr2=0x7f4de00aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 shutdown_connections 2023-10-06T05:23:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.187+0000 7f4de63a2700 1 -- 172.21.15.45:0/1474578500 wait complete. 2023-10-06T05:23:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 162 2023-10-06T05:23:31.203 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2023-10-06T05:23:31.203 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2023-10-06T05:23:31.203 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:31.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:31.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:31.204 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:31.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:23:31.224 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:23:31.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.400+0000 7f800e5bf700 1 Processor -- start 2023-10-06T05:23:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.400+0000 7f800e5bf700 1 -- start start 2023-10-06T05:23:31.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f800e5bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8008159200 0x7f80080c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f800e5bf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80081595d0 0x7f800806b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f800e5bf700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080c7d00 con 0x7f800806b5c0 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f800e5bf700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80080c7780 con 0x7f8008159200 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f800e5bf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80080c86d0 con 0x7f80081595d0 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f8007fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8008159200 0x7f80080c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f8007fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8008159200 0x7f80080c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60940/0 (socket says 172.21.15.45:60940) 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.401+0000 7f8007fff700 1 -- 172.21.15.45:0/1252176245 learned_addr learned my addr 172.21.15.45:0/1252176245 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f80077fe700 1 --2- 172.21.15.45:0/1252176245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80081595d0 0x7f800806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8007fff700 1 -- 172.21.15.45:0/1252176245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80081595d0 msgr2=0x7f800806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8007fff700 1 --2- 172.21.15.45:0/1252176245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80081595d0 0x7f800806b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8007fff700 1 -- 172.21.15.45:0/1252176245 >> v1:172.21.15.144:6789/0 conn(0x7f800806b5c0 legacy=0x7f80080ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8007fff700 1 -- 172.21.15.45:0/1252176245 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800806bb40 con 0x7f8008159200 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f80077fe700 1 --2- 172.21.15.45:0/1252176245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80081595d0 0x7f800806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8007fff700 1 --2- 172.21.15.45:0/1252176245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8008159200 0x7f80080c8190 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80099b0 tx=0x7f7ff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba65133c9a129305 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8006ffd700 1 -- 172.21.15.45:0/1252176245 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800b820 con 0x7f8008159200 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8006ffd700 1 -- 172.21.15.45:0/1252176245 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff800b980 con 0x7f8008159200 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.402+0000 7f8006ffd700 1 -- 172.21.15.45:0/1252176245 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800bd20 con 0x7f8008159200 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 -- 172.21.15.45:0/1252176245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8008159200 msgr2=0x7f80080c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 --2- 172.21.15.45:0/1252176245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8008159200 0x7f80080c8190 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80099b0 tx=0x7f7ff8009250 comp rx=0 tx=0).stop 2023-10-06T05:23:31.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 -- 172.21.15.45:0/1252176245 shutdown_connections 2023-10-06T05:23:31.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 --2- 172.21.15.45:0/1252176245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8008159200 0x7f80080c8190 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 --2- 172.21.15.45:0/1252176245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80081595d0 0x7f800806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 -- 172.21.15.45:0/1252176245 >> 172.21.15.45:0/1252176245 conn(0x7f800806f090 msgr2=0x7f800806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:31.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 -- 172.21.15.45:0/1252176245 shutdown_connections 2023-10-06T05:23:31.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.403+0000 7f800e5bf700 1 -- 172.21.15.45:0/1252176245 wait complete. 2023-10-06T05:23:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.404+0000 7f800e5bf700 1 Processor -- start 2023-10-06T05:23:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.405+0000 7f800e5bf700 1 -- start start 2023-10-06T05:23:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800e5bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800806b5c0 0x7f80080cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800e5bf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8008159200 0x7f800814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800e5bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80081595d0 0x7f80080cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800e5bf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080ca960 con 0x7f80081595d0 2023-10-06T05:23:31.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800e5bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800816a640 con 0x7f800806b5c0 2023-10-06T05:23:31.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800e5bf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800816a920 con 0x7f8008159200 2023-10-06T05:23:31.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80081595d0 0x7f80080cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80081595d0 0x7f80080cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60956/0 (socket says 172.21.15.45:60956) 2023-10-06T05:23:31.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 -- 172.21.15.45:0/2775808467 learned_addr learned my addr 172.21.15.45:0/2775808467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f80077fe700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8008159200 0x7f800814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f8007fff700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800806b5c0 0x7f80080cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 -- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8008159200 msgr2=0x7f800814e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8008159200 0x7f800814e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 -- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800806b5c0 msgr2=0x7f80080cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800806b5c0 0x7f80080cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 -- 172.21.15.45:0/2775808467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80094f0 con 0x7f80081595d0 2023-10-06T05:23:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800cb5c700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80081595d0 0x7f80080cc800 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f800000c930 tx=0x7f80000026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f7feffff700 1 -- 172.21.15.45:0/2775808467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000ce70 con 0x7f80081595d0 2023-10-06T05:23:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f7feffff700 1 -- 172.21.15.45:0/2775808467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800000a7e0 con 0x7f80081595d0 2023-10-06T05:23:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800816ac00 con 0x7f80081595d0 2023-10-06T05:23:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.406+0000 7f7feffff700 1 -- 172.21.15.45:0/2775808467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000aba0 con 0x7f80081595d0 2023-10-06T05:23:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.407+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80080cc480 con 0x7f80081595d0 2023-10-06T05:23:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.408+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80080c7580 con 0x7f80081595d0 2023-10-06T05:23:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.408+0000 7f7feffff700 1 -- 172.21.15.45:0/2775808467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8000012420 con 0x7f80081595d0 2023-10-06T05:23:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.408+0000 7f7feffff700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7ff4066f30 0x7f7ff40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.409+0000 7f7feffff700 1 -- 172.21.15.45:0/2775808467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f8000084fb0 con 0x7f80081595d0 2023-10-06T05:23:31.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.409+0000 7f8007fff700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7ff4066f30 0x7f7ff40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.409+0000 7f8007fff700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7ff4066f30 0x7f7ff40693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80099b0 tx=0x7f7ff8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:31.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.413+0000 7f7feffff700 1 -- 172.21.15.45:0/2775808467 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80000556b0 con 0x7f80081595d0 2023-10-06T05:23:31.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.658+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f80080c05f0 con 0x7f80081595d0 2023-10-06T05:23:31.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.921+0000 7f7feffff700 1 -- 172.21.15.45:0/2775808467 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v163) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8000055240 con 0x7f80081595d0 2023-10-06T05:23:31.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7ff4066f30 msgr2=0x7f7ff40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7ff4066f30 0x7f7ff40693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80099b0 tx=0x7f7ff8016000 comp rx=0 tx=0).stop 2023-10-06T05:23:31.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80081595d0 msgr2=0x7f80080cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80081595d0 0x7f80080cc800 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f800000c930 tx=0x7f80000026e0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 shutdown_connections 2023-10-06T05:23:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7ff4066f30 0x7f7ff40693e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80081595d0 0x7f80080cc800 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800806b5c0 0x7f80080cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 --2- 172.21.15.45:0/2775808467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8008159200 0x7f800814e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 >> 172.21.15.45:0/2775808467 conn(0x7f800806f090 msgr2=0x7f80080aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 shutdown_connections 2023-10-06T05:23:31.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:31.924+0000 7f800e5bf700 1 -- 172.21.15.45:0/2775808467 wait complete. 2023-10-06T05:23:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:31.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.077+0000 7f12ea613700 1 Processor -- start 2023-10-06T05:23:32.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.077+0000 7f12ea613700 1 -- start start 2023-10-06T05:23:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12ea613700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12e4155cd0 0x7f12e415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12ea613700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f12e41560a0 0x7f12e40adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12ea613700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e40ae8d0 con 0x7f12e40ccdc0 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12ea613700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e40aebb0 con 0x7f12e4155cd0 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12ea613700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e40aee90 con 0x7f12e41560a0 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12e3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12e4155cd0 0x7f12e415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12e8bb0700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f12e40ccdc0 0x7f12e40cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48692/0 (socket says 172.21.15.45:48692) 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12e8bb0700 1 -- 172.21.15.45:0/3084021897 learned_addr learned my addr 172.21.15.45:0/3084021897 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12e37fe700 1 --2- 172.21.15.45:0/3084021897 >> v2:172.21.15.144:3300/0 conn(0x7f12e41560a0 0x7f12e40adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.078+0000 7f12e2ffd700 1 -- 172.21.15.45:0/3084021897 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706332992 0 0) 0x7f12e40ae8d0 con 0x7f12e40ccdc0 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e2ffd700 1 -- 172.21.15.45:0/3084021897 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d0003570 con 0x7f12e40ccdc0 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e3fff700 1 -- 172.21.15.45:0/3084021897 >> v2:172.21.15.144:3300/0 conn(0x7f12e41560a0 msgr2=0x7f12e40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:32.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e3fff700 1 --2- 172.21.15.45:0/3084021897 >> v2:172.21.15.144:3300/0 conn(0x7f12e41560a0 0x7f12e40adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e3fff700 1 -- 172.21.15.45:0/3084021897 >> v1:172.21.15.45:6789/0 conn(0x7f12e40ccdc0 legacy=0x7f12e40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e3fff700 1 -- 172.21.15.45:0/3084021897 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e40af170 con 0x7f12e4155cd0 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e3fff700 1 --2- 172.21.15.45:0/3084021897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4155cd0 0x7f12e415a730 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cda20 tx=0x7f12d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf3bd3f0e4ebf31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e2ffd700 1 -- 172.21.15.45:0/3084021897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011630 con 0x7f12e4155cd0 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e2ffd700 1 -- 172.21.15.45:0/3084021897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d40117b0 con 0x7f12e4155cd0 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.079+0000 7f12e2ffd700 1 -- 172.21.15.45:0/3084021897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011b70 con 0x7f12e4155cd0 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 -- 172.21.15.45:0/3084021897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4155cd0 msgr2=0x7f12e415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 --2- 172.21.15.45:0/3084021897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4155cd0 0x7f12e415a730 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cda20 tx=0x7f12d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 -- 172.21.15.45:0/3084021897 shutdown_connections 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 --2- 172.21.15.45:0/3084021897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4155cd0 0x7f12e415a730 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 --2- 172.21.15.45:0/3084021897 >> v2:172.21.15.144:3300/0 conn(0x7f12e41560a0 0x7f12e40adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 -- 172.21.15.45:0/3084021897 >> 172.21.15.45:0/3084021897 conn(0x7f12e406f090 msgr2=0x7f12e406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 -- 172.21.15.45:0/3084021897 shutdown_connections 2023-10-06T05:23:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.080+0000 7f12ea613700 1 -- 172.21.15.45:0/3084021897 wait complete. 2023-10-06T05:23:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.081+0000 7f12ea613700 1 Processor -- start 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.082+0000 7f12ea613700 1 -- start start 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.082+0000 7f12ea613700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e40ccdc0 0x7f12e40c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.082+0000 7f12ea613700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4155cd0 0x7f12e40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.082+0000 7f12ea613700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12e41560a0 0x7f12e414fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12ea613700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4158380 con 0x7f12e40ccdc0 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12ea613700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4157ac0 con 0x7f12e4155cd0 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12ea613700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40c63f0 con 0x7f12e41560a0 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e40ccdc0 0x7f12e40c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e40ccdc0 0x7f12e40c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60974/0 (socket says 172.21.15.45:60974) 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 -- 172.21.15.45:0/2840880098 learned_addr learned my addr 172.21.15.45:0/2840880098 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e8bb0700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12e41560a0 0x7f12e414fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e37fe700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4155cd0 0x7f12e40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 -- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12e41560a0 msgr2=0x7f12e414fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12e41560a0 0x7f12e414fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 -- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4155cd0 msgr2=0x7f12e40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4155cd0 0x7f12e40c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 -- 172.21.15.45:0/2840880098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4009420 con 0x7f12e40ccdc0 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e37fe700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4155cd0 0x7f12e40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e8bb0700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12e41560a0 0x7f12e414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12e3fff700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e40ccdc0 0x7f12e40c58d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f12e406aed0 tx=0x7f12d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12dbfff700 1 -- 172.21.15.45:0/2840880098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d401f070 con 0x7f12e40ccdc0 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12dbfff700 1 -- 172.21.15.45:0/2840880098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4011920 con 0x7f12e40ccdc0 2023-10-06T05:23:32.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12dbfff700 1 -- 172.21.15.45:0/2840880098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4023410 con 0x7f12e40ccdc0 2023-10-06T05:23:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e4157d30 con 0x7f12e40ccdc0 2023-10-06T05:23:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.083+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e4158810 con 0x7f12e40ccdc0 2023-10-06T05:23:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.084+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e4158660 con 0x7f12e40ccdc0 2023-10-06T05:23:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.085+0000 7f12dbfff700 1 -- 172.21.15.45:0/2840880098 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12d402c3b0 con 0x7f12e40ccdc0 2023-10-06T05:23:32.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.085+0000 7f12dbfff700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c4066f30 0x7f12c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.086+0000 7f12dbfff700 1 -- 172.21.15.45:0/2840880098 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f12d4085fd0 con 0x7f12e40ccdc0 2023-10-06T05:23:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.086+0000 7f12e37fe700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c4066f30 0x7f12c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.086+0000 7f12e37fe700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c4066f30 0x7f12c40693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cad00 tx=0x7f12cc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.090+0000 7f12dbfff700 1 -- 172.21.15.45:0/2840880098 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12d4052af0 con 0x7f12e40ccdc0 2023-10-06T05:23:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.379+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f12e40ae4e0 con 0x7f12e40ccdc0 2023-10-06T05:23:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.927+0000 7f12dbfff700 1 -- 172.21.15.45:0/2840880098 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v164) v1 ==== 101+0+0 (secure 0 0 0) 0x7f12d4056110 con 0x7f12e40ccdc0 2023-10-06T05:23:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c4066f30 msgr2=0x7f12c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c4066f30 0x7f12c40693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cad00 tx=0x7f12cc008040 comp rx=0 tx=0).stop 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e40ccdc0 msgr2=0x7f12e40c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e40ccdc0 0x7f12e40c58d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f12e406aed0 tx=0x7f12d4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 shutdown_connections 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f12c4066f30 0x7f12c40693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12e41560a0 0x7f12e414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e40ccdc0 0x7f12e40c58d0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 --2- 172.21.15.45:0/2840880098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e4155cd0 0x7f12e40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.929+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 >> 172.21.15.45:0/2840880098 conn(0x7f12e406f090 msgr2=0x7f12e40cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.930+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 shutdown_connections 2023-10-06T05:23:32.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.930+0000 7f12ea613700 1 -- 172.21.15.45:0/2840880098 wait complete. 2023-10-06T05:23:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:23:32.947 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 Processor -- start 2023-10-06T05:23:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 -- start start 2023-10-06T05:23:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da0764d7d0 0x55da0764b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da076dcb10 0x55da076d9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07550980 con 0x55da076d62e0 2023-10-06T05:23:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07550620 con 0x55da076dcb10 2023-10-06T05:23:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.992+0000 7f1b5b3d9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da07547400 con 0x55da0764d7d0 2023-10-06T05:23:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60980/0 (socket says 172.21.15.45:60980) 2023-10-06T05:23:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 -- 172.21.15.45:0/3776518949 learned_addr learned my addr 172.21.15.45:0/3776518949 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b545f7700 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da0764d7d0 0x55da0764b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b54df8700 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da076dcb10 0x55da076d9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 -- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da0764d7d0 msgr2=0x55da0764b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da0764d7d0 0x55da0764b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 -- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da076dcb10 msgr2=0x55da076d9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da076dcb10 0x55da076d9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 -- 172.21.15.45:0/3776518949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c005cf0 con 0x55da076d62e0 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b545f7700 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da0764d7d0 0x55da0764b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b53df6700 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d9910 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00acf0 tx=0x7f1b3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75984fa3eec9b246 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b52df4700 1 -- 172.21.15.45:0/3776518949 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c000c90 con 0x55da076d62e0 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b52df4700 1 -- 172.21.15.45:0/3776518949 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c008970 con 0x55da076d62e0 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.993+0000 7f1b52df4700 1 -- 172.21.15.45:0/3776518949 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c01d420 con 0x55da076d62e0 2023-10-06T05:23:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/3776518949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 msgr2=0x55da076d9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d9910 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00acf0 tx=0x7f1b3c016000 comp rx=0 tx=0).stop 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/3776518949 shutdown_connections 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da076dcb10 0x55da076d9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da0764d7d0 0x55da0764b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/3776518949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d9910 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/3776518949 >> 172.21.15.45:0/3776518949 conn(0x55da07605bd0 msgr2=0x55da0764bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/3776518949 shutdown_connections 2023-10-06T05:23:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/3776518949 wait complete. 2023-10-06T05:23:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.994+0000 7f1b5b3d9040 1 Processor -- start 2023-10-06T05:23:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b5b3d9040 1 -- start start 2023-10-06T05:23:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b5b3d9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0764d7d0 0x55da076d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b5b3d9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b5b3d9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da076dcb10 0x55da076d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b5b3d9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da076df320 con 0x55da076d62e0 2023-10-06T05:23:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b5b3d9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da076df580 con 0x55da0764d7d0 2023-10-06T05:23:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b5b3d9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da076df820 con 0x55da076dcb10 2023-10-06T05:23:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.995+0000 7f1b53df6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60986/0 (socket says 172.21.15.45:60986) 2023-10-06T05:23:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 learned_addr learned my addr 172.21.15.45:0/4226199050 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b545f7700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0764d7d0 0x55da076d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b54df8700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da076dcb10 0x55da076d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da076dcb10 msgr2=0x55da076d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da076dcb10 0x55da076d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0764d7d0 msgr2=0x55da076d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0764d7d0 0x55da076d3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da075791c0 con 0x55da076d62e0 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b53df6700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d43a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0054c0 tx=0x7f1b3c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b515f1700 1 -- 172.21.15.45:0/4226199050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c02b070 con 0x55da076d62e0 2023-10-06T05:23:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b515f1700 1 -- 172.21.15.45:0/4226199050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c030470 con 0x55da076d62e0 2023-10-06T05:23:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55da0755f3d0 con 0x55da076d62e0 2023-10-06T05:23:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b515f1700 1 -- 172.21.15.45:0/4226199050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c030830 con 0x55da076d62e0 2023-10-06T05:23:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.996+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da07552ae0 con 0x55da076d62e0 2023-10-06T05:23:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.998+0000 7f1b515f1700 1 -- 172.21.15.45:0/4226199050 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b3c026450 con 0x55da076d62e0 2023-10-06T05:23:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.998+0000 7f1b515f1700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b340658c0 0x7f1b34067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.998+0000 7f1b515f1700 1 -- 172.21.15.45:0/4226199050 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f1b3c087250 con 0x55da076d62e0 2023-10-06T05:23:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.998+0000 7f1b545f7700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b340658c0 0x7f1b34067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.008 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.999+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55da076d74b0 0x55da076d7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.999+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076d7dc0 con 0x55da076d74b0 2023-10-06T05:23:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.999+0000 7f1b54df8700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55da076d74b0 0x55da076d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.999+0000 7f1b545f7700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b340658c0 0x7f1b34067d70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1b44000c00 tx=0x7f1b44006930 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:32.999+0000 7f1b54df8700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55da076d74b0 0x55da076d7880 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.001+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v104'350 uv324 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.001+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076d81a0 con 0x55da076d74b0 2023-10-06T05:23:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.002+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v104'351 uv351 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.002+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076d8550 con 0x55da076d74b0 2023-10-06T05:23:33.010 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.004+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v104'352 uv352 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.004+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9f70 con 0x55da076d74b0 2023-10-06T05:23:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.005+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'353 uv353 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.005+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa320 con 0x55da076d74b0 2023-10-06T05:23:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.006+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.006+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f8f60 con 0x55da076d74b0 2023-10-06T05:23:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.007+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'355 uv355 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.008+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9390 con 0x55da076d74b0 2023-10-06T05:23:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.009+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v104'356 uv330 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.009+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f97c0 con 0x55da076d74b0 2023-10-06T05:23:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.010+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v104'357 uv357 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.010+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f97c0 con 0x55da076d74b0 2023-10-06T05:23:33.014 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.011+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v104'358 uv358 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.011+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9bc0 con 0x55da076d74b0 2023-10-06T05:23:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.012+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'359 uv359 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.012+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9bc0 con 0x55da076d74b0 2023-10-06T05:23:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.013+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'360 uv360 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.013+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9740 con 0x55da076d74b0 2023-10-06T05:23:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.014+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.014+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9740 con 0x55da076d74b0 2023-10-06T05:23:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.016+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v104'362 uv337 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.016+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9740 con 0x55da076d74b0 2023-10-06T05:23:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.017+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'363 uv363 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.017+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9740 con 0x55da076d74b0 2023-10-06T05:23:33.020 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.018+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v104'364 uv364 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.018+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9740 con 0x55da076d74b0 2023-10-06T05:23:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.019+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'365 uv365 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.019+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9740 con 0x55da076d74b0 2023-10-06T05:23:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.020+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'366 uv366 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.020+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076f9740 con 0x55da076d74b0 2023-10-06T05:23:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.021+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'367 uv367 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.021+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.022+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'368 uv368 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.022+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.024+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v104'369 uv343 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.024+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.025+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v104'370 uv370 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.025+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.028 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.026+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v104'371 uv371 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.026+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fb6e0 con 0x55da076d74b0 2023-10-06T05:23:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.028+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'372 uv372 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.028+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fb6e0 con 0x55da076d74b0 2023-10-06T05:23:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.029+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'373 uv373 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.029+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.030+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.030+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.031+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v104'375 uv349 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.031+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.033+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v104'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.033+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.035 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.034+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v104'377 uv377 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.034+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fb6e0 con 0x55da076d74b0 2023-10-06T05:23:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.035+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'378 uv378 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.035+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fb6e0 con 0x55da076d74b0 2023-10-06T05:23:33.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.036+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'379 uv379 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.036+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fa6d0 con 0x55da076d74b0 2023-10-06T05:23:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.037+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'380 uv380 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da076d74b0 2023-10-06T05:23:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.038+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55da076fabe0 0x55da076fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.038+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fd500 con 0x55da076fabe0 2023-10-06T05:23:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.038+0000 7f1b545f7700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55da076fabe0 0x55da076fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.038+0000 7f1b545f7700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55da076fabe0 0x55da076fcfc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.040+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'259 uv242 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.040+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fd8b0 con 0x55da076fabe0 2023-10-06T05:23:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.041+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'260 uv260 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.041+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fdc60 con 0x55da076fabe0 2023-10-06T05:23:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.042+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'261 uv261 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.044 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.042+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fe500 con 0x55da076fabe0 2023-10-06T05:23:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.044+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v104'262 uv262 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.044+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fe8b0 con 0x55da076fabe0 2023-10-06T05:23:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.045+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'263 uv263 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.045+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.046+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v104'264 uv247 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.046+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.048+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v104'265 uv265 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.048+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.050 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.049+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v104'266 uv266 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.049+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.050+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v104'267 uv267 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.050+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.052+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v104'268 uv268 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.052+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.053+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v104'269 uv252 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.053+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.054+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v104'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.054+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.057 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.056+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v104'271 uv271 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.056+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702380 con 0x55da076fabe0 2023-10-06T05:23:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.057+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v104'272 uv272 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.057+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702380 con 0x55da076fabe0 2023-10-06T05:23:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.059+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v104'273 uv273 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.059+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.060+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v104'274 uv258 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.060+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.061+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v104'275 uv275 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.061+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.064 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.062+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v104'276 uv276 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.063+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076ffc70 con 0x55da076fabe0 2023-10-06T05:23:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.064+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v104'277 uv277 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.064+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076ffc70 con 0x55da076fabe0 2023-10-06T05:23:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.065+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v104'278 uv278 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.065+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fec60 con 0x55da076fabe0 2023-10-06T05:23:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.066+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v104'279 uv279 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.066+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55da076ff170 0x55da07701550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.066+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07701a90 con 0x55da076ff170 2023-10-06T05:23:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.067+0000 7f1b53df6700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55da076ff170 0x55da07701550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.067+0000 7f1b53df6700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55da076ff170 0x55da07701550 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.068+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.068+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07701e40 con 0x55da076ff170 2023-10-06T05:23:33.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.069+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v104'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.069+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da077021f0 con 0x55da076ff170 2023-10-06T05:23:33.072 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:33.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.070+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v104'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.070+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da077035b0 con 0x55da076ff170 2023-10-06T05:23:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.072+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.072+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da077039b0 con 0x55da076ff170 2023-10-06T05:23:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.073+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.073+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da076fe0d0 con 0x55da076ff170 2023-10-06T05:23:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.074+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.074+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702620 con 0x55da076ff170 2023-10-06T05:23:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.075+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'51 uv50 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.075+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076ff170 2023-10-06T05:23:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.077+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v104'52 uv52 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.077+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076ff170 2023-10-06T05:23:33.079 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.078+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v104'53 uv53 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.078+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076ff170 2023-10-06T05:23:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.079+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v104'54 uv54 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.079+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076fabe0 2023-10-06T05:23:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.081+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.081+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076fabe0 2023-10-06T05:23:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.082+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v104'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.082+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076fabe0 2023-10-06T05:23:33.085 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.083+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v104'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.083+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076fabe0 2023-10-06T05:23:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.084+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.084+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076fabe0 2023-10-06T05:23:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.085+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.085+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07702fd0 con 0x55da076fabe0 2023-10-06T05:23:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.087+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da076fabe0 2023-10-06T05:23:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.087+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55da077042e0 0x55da077066c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.087+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.087+0000 7f1b54df8700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55da077042e0 0x55da077066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.087+0000 7f1b54df8700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55da077042e0 0x55da077066c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.088+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v104'127 uv121 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.088+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.089+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v104'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.089+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.091+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v104'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.091+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07707c10 con 0x55da077042e0 2023-10-06T05:23:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.092+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v104'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.092+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07707c10 con 0x55da077042e0 2023-10-06T05:23:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.093+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v104'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.093+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.094+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v104'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.094+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.095+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v104'133 uv126 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.095+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.096+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v104'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.097+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.100 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:33.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.098+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v104'135 uv135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.098+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.099+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v104'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.099+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07706c00 con 0x55da077042e0 2023-10-06T05:23:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.100+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v104'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.100+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55da07707510 0x55da07707900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.100+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a030 con 0x55da07707510 2023-10-06T05:23:33.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.101+0000 7f1b545f7700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55da07707510 0x55da07707900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.101+0000 7f1b545f7700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55da07707510 0x55da07707900 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.102+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v104'56 uv55 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da07707510 2023-10-06T05:23:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.102+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a030 con 0x55da07707510 2023-10-06T05:23:33.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.103+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v104'57 uv57 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da07707510 2023-10-06T05:23:33.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.104+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a030 con 0x55da07707510 2023-10-06T05:23:33.106 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:33.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.105+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v104'58 uv58 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da07707510 2023-10-06T05:23:33.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.105+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a070 con 0x55da07707510 2023-10-06T05:23:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.106+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v104'59 uv59 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da07707510 2023-10-06T05:23:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.106+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a4c0 con 0x55da07707510 2023-10-06T05:23:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.107+0000 7f1b545f7700 1 -- 172.21.15.45:0/4226199050 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v104'60 uv60 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b4400e8d0 con 0x55da07707510 2023-10-06T05:23:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.107+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a4c0 con 0x55da076ff170 2023-10-06T05:23:33.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.109+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.109+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a4c0 con 0x55da076ff170 2023-10-06T05:23:33.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.110+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v104'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.110+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a4c0 con 0x55da076ff170 2023-10-06T05:23:33.113 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.111+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v104'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.111+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770b4d0 con 0x55da076ff170 2023-10-06T05:23:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.113+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.113+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770b4d0 con 0x55da076ff170 2023-10-06T05:23:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.114+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.114+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770a4c0 con 0x55da076ff170 2023-10-06T05:23:33.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.115+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.115+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55da0770a9d0 0x55da0770cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.115+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d380 con 0x55da0770a9d0 2023-10-06T05:23:33.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.115+0000 7f1b53df6700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55da0770a9d0 0x55da0770cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.116+0000 7f1b53df6700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55da0770a9d0 0x55da0770cdb0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.117+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v104'112 uv108 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.117+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d440 con 0x55da0770a9d0 2023-10-06T05:23:33.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.118+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v104'113 uv113 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.118+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d840 con 0x55da0770a9d0 2023-10-06T05:23:33.121 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:33.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.119+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v104'114 uv114 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.119+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d840 con 0x55da0770a9d0 2023-10-06T05:23:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.120+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v104'115 uv115 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.120+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d840 con 0x55da0770a9d0 2023-10-06T05:23:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.122+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v104'116 uv116 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.122+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d840 con 0x55da0770a9d0 2023-10-06T05:23:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.123+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v104'117 uv111 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.123+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d840 con 0x55da0770a9d0 2023-10-06T05:23:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.124+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v104'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.124+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d840 con 0x55da0770a9d0 2023-10-06T05:23:33.126 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.125+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v104'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.125+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770d840 con 0x55da0770a9d0 2023-10-06T05:23:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.126+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v104'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.126+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da077105b0 con 0x55da0770a9d0 2023-10-06T05:23:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.127+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v104'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da0770a9d0 2023-10-06T05:23:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.127+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.128+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v104'54 uv54 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.130 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.128+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.129+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v104'55 uv55 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.129+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55da0770e640 0x55da07710d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.129+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:103 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da0770e640 2023-10-06T05:23:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.129+0000 7f1b54df8700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55da0770e640 0x55da07710d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.130+0000 7f1b54df8700 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55da0770e640 0x55da07710d00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.131+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(103 609.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da0770e640 2023-10-06T05:23:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.131+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:104 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da0770e640 2023-10-06T05:23:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.132+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(104 609.00000000 [delete] v104'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da0770e640 2023-10-06T05:23:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.132+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da0770e640 2023-10-06T05:23:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.135+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(105 609.00000000 [create] v104'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da0770e640 2023-10-06T05:23:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.135+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da0770e640 2023-10-06T05:23:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.140+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da0770e640 2023-10-06T05:23:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.140+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da0770e640 2023-10-06T05:23:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.141+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da0770e640 2023-10-06T05:23:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.141+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da0770e640 2023-10-06T05:23:33.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.142+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(108 609.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da0770e640 2023-10-06T05:23:33.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.142+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:109 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da077042e0 2023-10-06T05:23:33.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.144+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(109 400.00000000 [create] v104'57 uv56 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.144+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:110 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da077042e0 2023-10-06T05:23:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.145+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(110 400.00000000 [delete] v104'58 uv58 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.145+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da077042e0 2023-10-06T05:23:33.148 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.146+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(111 400.00000000 [create] v104'59 uv59 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.146+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da077042e0 2023-10-06T05:23:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.147+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(112 400.00000000 [write 0~22] v104'60 uv60 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.148+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da077042e0 2023-10-06T05:23:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.149+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(113 400.00000000 [omap-set-header] v104'61 uv61 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f1b4c002c30 con 0x55da077042e0 2023-10-06T05:23:33.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.149+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:114 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.151+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(114 100.00000000.inode [create] v104'144 uv136 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.151+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:115 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.153+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(115 100.00000000.inode [delete] v104'145 uv145 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.153+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.155 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:33.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.154+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v104'146 uv146 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.154+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.155+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v104'147 uv147 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.155+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.156+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v104'148 uv148 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.156+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.157+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 1.00000000 [create] v104'149 uv143 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.157+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.158+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 1.00000000 [delete] v104'150 uv150 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.158+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.161 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.159+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v104'151 uv151 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.159+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.161+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v104'152 uv152 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.161+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.162+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v104'153 uv153 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.162+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da0770dd30 con 0x55da076ff170 2023-10-06T05:23:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.163+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [omap-set-header] v104'154 uv154 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.163+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) v8 -- 0x55da07711240 con 0x55da076ff170 2023-10-06T05:23:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b53df6700 1 -- 172.21.15.45:0/4226199050 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v104'155 uv155 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b3c085130 con 0x55da076ff170 2023-10-06T05:23:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55da076ff170 msgr2=0x55da07701550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55da076ff170 0x55da07701550 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55da076d74b0 msgr2=0x55da076d7880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55da076d74b0 0x55da076d7880 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55da0770a9d0 msgr2=0x55da0770cdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55da0770a9d0 0x55da0770cdb0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55da076fabe0 msgr2=0x55da076fcfc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55da076fabe0 0x55da076fcfc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55da0770e640 msgr2=0x55da07710d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55da0770e640 0x55da07710d00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55da07707510 msgr2=0x55da07707900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55da07707510 0x55da07707900 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55da077042e0 msgr2=0x55da077066c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55da077042e0 0x55da077066c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 reap_dead start 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 reap_dead start 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 reap_dead start 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 reap_dead start 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b340658c0 msgr2=0x7f1b34067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b54df8700 1 -- 172.21.15.45:0/4226199050 reap_dead start 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b340658c0 0x7f1b34067d70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1b44000c00 tx=0x7f1b44006930 comp rx=0 tx=0).stop 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 msgr2=0x55da076d43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d43a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0054c0 tx=0x7f1b3c01e000 comp rx=0 tx=0).stop 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 shutdown_connections 2023-10-06T05:23:33.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da076d62e0 0x55da076d43a0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 --2- 172.21.15.45:0/4226199050 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b340658c0 0x7f1b34067d70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 >> 172.21.15.45:0/4226199050 conn(0x55da07605bd0 msgr2=0x55da07607ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 shutdown_connections 2023-10-06T05:23:33.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.164+0000 7f1b5b3d9040 1 -- 172.21.15.45:0/4226199050 wait complete. 2023-10-06T05:23:33.174 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2023-10-06T05:23:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.221+0000 7f77cda4a040 1 Processor -- start 2023-10-06T05:23:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.221+0000 7f77cda4a040 1 -- start start 2023-10-06T05:23:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77cda4a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b82e7e0 0x55b42b8141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77cda4a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9b30 0x55b42b7e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77cda4a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b8b9f00 0x55b42b8bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77cda4a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b731980 con 0x55b42b8b9f00 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77cda4a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b731620 con 0x55b42b8b9b30 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77cda4a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b42b728400 con 0x55b42b82e7e0 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77c7469700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b8b9f00 0x55b42b8bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77c6c68700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b82e7e0 0x55b42b8141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77c6467700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9b30 0x55b42b7e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77c7469700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b8b9f00 0x55b42b8bc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32768/0 (socket says 172.21.15.45:32768) 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77c7469700 1 -- 172.21.15.45:0/508350484 learned_addr learned my addr 172.21.15.45:0/508350484 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77c7469700 1 -- 172.21.15.45:0/508350484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b82e7e0 msgr2=0x55b42b8141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.222+0000 7f77c7469700 1 --2- 172.21.15.45:0/508350484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b82e7e0 0x55b42b8141c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.223+0000 7f77c7469700 1 -- 172.21.15.45:0/508350484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9b30 msgr2=0x55b42b7e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.223+0000 7f77c7469700 1 --2- 172.21.15.45:0/508350484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9b30 0x55b42b7e7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.223+0000 7f77c7469700 1 -- 172.21.15.45:0/508350484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc005db0 con 0x55b42b8b9f00 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.223+0000 7f77c7469700 1 --2- 172.21.15.45:0/508350484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b8b9f00 0x55b42b8bc6d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f77c0001580 tx=0x7f77c0006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992e915f1bdedd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.223+0000 7f77c5465700 1 -- 172.21.15.45:0/508350484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0004150 con 0x55b42b8b9f00 2023-10-06T05:23:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.223+0000 7f77c5465700 1 -- 172.21.15.45:0/508350484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0013460 con 0x55b42b8b9f00 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.223+0000 7f77c5465700 1 -- 172.21.15.45:0/508350484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0013800 con 0x55b42b8b9f00 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 -- 172.21.15.45:0/508350484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b8b9f00 msgr2=0x55b42b8bc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 --2- 172.21.15.45:0/508350484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b8b9f00 0x55b42b8bc6d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f77c0001580 tx=0x7f77c0006b10 comp rx=0 tx=0).stop 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 -- 172.21.15.45:0/508350484 shutdown_connections 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 --2- 172.21.15.45:0/508350484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b8b9f00 0x55b42b8bc6d0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 --2- 172.21.15.45:0/508350484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b82e7e0 0x55b42b8141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 --2- 172.21.15.45:0/508350484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9b30 0x55b42b7e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 -- 172.21.15.45:0/508350484 >> 172.21.15.45:0/508350484 conn(0x55b42b7e7450 msgr2=0x55b42b8b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 -- 172.21.15.45:0/508350484 shutdown_connections 2023-10-06T05:23:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 -- 172.21.15.45:0/508350484 wait complete. 2023-10-06T05:23:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 Processor -- start 2023-10-06T05:23:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.224+0000 7f77cda4a040 1 -- start start 2023-10-06T05:23:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77cda4a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b82e7e0 0x55b42b8b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77cda4a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b8b9b30 0x55b42b8a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77cda4a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9f00 0x55b42b82c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77cda4a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b8b41d0 con 0x55b42b82e7e0 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77cda4a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b42b8a90b0 con 0x55b42b8b9f00 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77cda4a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b42b8b39f0 con 0x55b42b8b9b30 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77c6c68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b82e7e0 0x55b42b8b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77c6c68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b82e7e0 0x55b42b8b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32780/0 (socket says 172.21.15.45:32780) 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77c6c68700 1 -- 172.21.15.45:0/3066503346 learned_addr learned my addr 172.21.15.45:0/3066503346 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77c7469700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9f00 0x55b42b82c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.225+0000 7f77c6467700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b8b9b30 0x55b42b8a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77c6c68700 1 -- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b8b9b30 msgr2=0x55b42b8a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77c6c68700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b8b9b30 0x55b42b8a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77c6c68700 1 -- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9f00 msgr2=0x55b42b82c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77c6c68700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b42b8b9f00 0x55b42b82c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77c6c68700 1 -- 172.21.15.45:0/3066503346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b42b733750 con 0x55b42b82e7e0 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77c6467700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b42b8b9b30 0x55b42b8a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77c6c68700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b82e7e0 0x55b42b8b6570 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f77b8008920 tx=0x7f77b8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77b77fe700 1 -- 172.21.15.45:0/3066503346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800f050 con 0x55b42b82e7e0 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77b77fe700 1 -- 172.21.15.45:0/3066503346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b8010040 con 0x55b42b82e7e0 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b42b7403d0 con 0x55b42b82e7e0 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77b77fe700 1 -- 172.21.15.45:0/3066503346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8006cb0 con 0x55b42b82e7e0 2023-10-06T05:23:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.226+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b42b733ae0 con 0x55b42b82e7e0 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.228+0000 7f77b77fe700 1 -- 172.21.15.45:0/3066503346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77b8015670 con 0x55b42b82e7e0 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.228+0000 7f77b77fe700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77ac065890 0x7f77ac067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.228+0000 7f77b77fe700 1 -- 172.21.15.45:0/3066503346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f77b8084060 con 0x55b42b82e7e0 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.228+0000 7f77c6467700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77ac065890 0x7f77ac067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.228+0000 7f77cda4a040 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b42b8a8990 0x55b42b8d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.229+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) v8 -- 0x55b42b8b3590 con 0x55b42b8a8990 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.229+0000 7f77c7469700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b42b8a8990 0x55b42b8d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.229+0000 7f77c6467700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77ac065890 0x7f77ac067d40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00afd0 tx=0x7f77bc016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.229+0000 7f77c7469700 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b42b8a8990 0x55b42b8d0270 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.230+0000 7f77c7469700 1 -- 172.21.15.45:0/3066503346 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 400.00000000 [delete] v104'62 uv62 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f77c0013700 con 0x55b42b8a8990 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b42b8a8990 msgr2=0x55b42b8d0270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b42b8a8990 0x55b42b8d0270 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77ac065890 msgr2=0x7f77ac067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f77ac065890 0x7f77ac067d40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00afd0 tx=0x7f77bc016000 comp rx=0 tx=0).stop 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b82e7e0 msgr2=0x55b42b8b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 --2- 172.21.15.45:0/3066503346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b42b82e7e0 0x55b42b8b6570 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f77b8008920 tx=0x7f77b8005bc0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77c7469700 1 -- 172.21.15.45:0/3066503346 reap_dead start 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 shutdown_connections 2023-10-06T05:23:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 >> 172.21.15.45:0/3066503346 conn(0x55b42b7e7450 msgr2=0x55b42b7e7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 shutdown_connections 2023-10-06T05:23:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.231+0000 7f77cda4a040 1 -- 172.21.15.45:0/3066503346 wait complete. 2023-10-06T05:23:33.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.375+0000 7f271fdde700 1 Processor -- start 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.375+0000 7f271fdde700 1 -- start start 2023-10-06T05:23:33.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.376+0000 7f271fdde700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2718159150 0x7f27180ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.376+0000 7f271fdde700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27180c9c90 con 0x7f2718067620 2023-10-06T05:23:33.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.376+0000 7f271fdde700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271815d8e0 con 0x7f2718159520 2023-10-06T05:23:33.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.376+0000 7f271fdde700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271815dbc0 con 0x7f2718159150 2023-10-06T05:23:33.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.376+0000 7f271db7a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2718159150 0x7f27180ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.376+0000 7f271e37b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2718067620 0x7f2718067a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48704/0 (socket says 172.21.15.45:48704) 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.376+0000 7f271e37b700 1 -- 172.21.15.45:0/1231419923 learned_addr learned my addr 172.21.15.45:0/1231419923 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271db7a700 1 -- 172.21.15.45:0/1231419923 >> v1:172.21.15.144:6789/0 conn(0x7f2718159520 legacy=0x7f271815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271db7a700 1 -- 172.21.15.45:0/1231419923 >> v1:172.21.15.45:6789/0 conn(0x7f2718067620 legacy=0x7f2718067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271cb78700 1 -- 172.21.15.45:0/1231419923 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651130987 0 0) 0x7f27180c9c90 con 0x7f2718067620 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271db7a700 1 -- 172.21.15.45:0/1231419923 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27180c9920 con 0x7f2718159150 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271db7a700 1 --2- 172.21.15.45:0/1231419923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718159150 0x7f27180ca120 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f27140098e0 tx=0x7f27140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93572de3b3b15296 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271cb78700 1 -- 172.21.15.45:0/1231419923 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714009e80 con 0x7f2718159150 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271cb78700 1 -- 172.21.15.45:0/1231419923 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271400b860 con 0x7f2718159150 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271cb78700 1 -- 172.21.15.45:0/1231419923 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271400bbe0 con 0x7f2718159150 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271fdde700 1 -- 172.21.15.45:0/1231419923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718159150 msgr2=0x7f27180ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.377+0000 7f271fdde700 1 --2- 172.21.15.45:0/1231419923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718159150 0x7f27180ca120 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f27140098e0 tx=0x7f27140091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.378+0000 7f271fdde700 1 -- 172.21.15.45:0/1231419923 shutdown_connections 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.378+0000 7f271fdde700 1 --2- 172.21.15.45:0/1231419923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718159150 0x7f27180ca120 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.378+0000 7f271fdde700 1 -- 172.21.15.45:0/1231419923 >> 172.21.15.45:0/1231419923 conn(0x7f27180691c0 msgr2=0x7f27180cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.378+0000 7f271fdde700 1 -- 172.21.15.45:0/1231419923 shutdown_connections 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.378+0000 7f271fdde700 1 -- 172.21.15.45:0/1231419923 wait complete. 2023-10-06T05:23:33.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.379+0000 7f271fdde700 1 Processor -- start 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.379+0000 7f271fdde700 1 -- start start 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271fdde700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718067620 0x7f271815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271fdde700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2718159150 0x7f27180b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271fdde700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271816d040 0x7f27180b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271fdde700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2718159c30 con 0x7f2718067620 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271fdde700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271816ac50 con 0x7f2718159150 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271fdde700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271816a320 con 0x7f271816d040 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718067620 0x7f271815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718067620 0x7f271815a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32798/0 (socket says 172.21.15.45:32798) 2023-10-06T05:23:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 -- 172.21.15.45:0/2356484267 learned_addr learned my addr 172.21.15.45:0/2356484267 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271e37b700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271816d040 0x7f27180b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271d379700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2718159150 0x7f27180b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 -- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271816d040 msgr2=0x7f27180b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271816d040 0x7f27180b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 -- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2718159150 msgr2=0x7f27180b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2718159150 0x7f27180b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 -- 172.21.15.45:0/2356484267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2714009420 con 0x7f2718067620 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f271db7a700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718067620 0x7f271815a310 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f271806c4b0 tx=0x7f2714017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.380+0000 7f2709ffb700 1 -- 172.21.15.45:0/2356484267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271401f070 con 0x7f2718067620 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.381+0000 7f2709ffb700 1 -- 172.21.15.45:0/2356484267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2714000c90 con 0x7f2718067620 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.381+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271816a600 con 0x7f2718067620 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.381+0000 7f2709ffb700 1 -- 172.21.15.45:0/2356484267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714025420 con 0x7f2718067620 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.381+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2718169fd0 con 0x7f2718067620 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.382+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271806e2d0 con 0x7f2718067620 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.382+0000 7f2709ffb700 1 -- 172.21.15.45:0/2356484267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2714023070 con 0x7f2718067620 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.383+0000 7f2709ffb700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f26fc066f30 0x7f26fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.383+0000 7f2709ffb700 1 -- 172.21.15.45:0/2356484267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f2714086100 con 0x7f2718067620 2023-10-06T05:23:33.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.383+0000 7f271d379700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f26fc066f30 0x7f26fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.384+0000 7f271d379700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f26fc066f30 0x7f26fc0693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f27180b20a0 tx=0x7f270c007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.387+0000 7f2709ffb700 1 -- 172.21.15.45:0/2356484267 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2714052c20 con 0x7f2718067620 2023-10-06T05:23:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.638+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f27180ca660 con 0x7f2718067620 2023-10-06T05:23:33.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.931+0000 7f2709ffb700 1 -- 172.21.15.45:0/2356484267 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v165) v1 ==== 162+0+0 (secure 0 0 0) 0x7f27140150b0 con 0x7f2718067620 2023-10-06T05:23:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f26fc066f30 msgr2=0x7f26fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f26fc066f30 0x7f26fc0693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f27180b20a0 tx=0x7f270c007ec0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718067620 msgr2=0x7f271815a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718067620 0x7f271815a310 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f271806c4b0 tx=0x7f2714017000 comp rx=0 tx=0).stop 2023-10-06T05:23:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 shutdown_connections 2023-10-06T05:23:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f26fc066f30 0x7f26fc0693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271816d040 0x7f27180b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2718067620 0x7f271815a310 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 --2- 172.21.15.45:0/2356484267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2718159150 0x7f27180b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 >> 172.21.15.45:0/2356484267 conn(0x7f27180691c0 msgr2=0x7f27180c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 shutdown_connections 2023-10-06T05:23:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:33.934+0000 7f271fdde700 1 -- 172.21.15.45:0/2356484267 wait complete. 2023-10-06T05:23:33.941 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:34.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.494+0000 7f161f840700 1 Processor -- start 2023-10-06T05:23:34.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.494+0000 7f161f840700 1 -- start start 2023-10-06T05:23:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161f840700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16180c8ff0 0x7f16180c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161f840700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180caad0 0x7f16180caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161f840700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161815b000 con 0x7f1618156af0 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161f840700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161815b2e0 con 0x7f16180c8ff0 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161f840700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161815b5c0 con 0x7f16180caad0 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16180c8ff0 0x7f16180c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16180c8ff0 0x7f16180c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32800/0 (socket says 172.21.15.45:32800) 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 -- 172.21.15.45:0/3783014208 learned_addr learned my addr 172.21.15.45:0/3783014208 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161cddb700 1 --2- 172.21.15.45:0/3783014208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180caad0 0x7f16180caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 -- 172.21.15.45:0/3783014208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180caad0 msgr2=0x7f16180caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 --2- 172.21.15.45:0/3783014208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180caad0 0x7f16180caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 -- 172.21.15.45:0/3783014208 >> v1:172.21.15.144:6789/0 conn(0x7f1618156af0 legacy=0x7f1618156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 -- 172.21.15.45:0/3783014208 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161815b8a0 con 0x7f16180c8ff0 2023-10-06T05:23:34.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.495+0000 7f161d5dc700 1 --2- 172.21.15.45:0/3783014208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180c8ff0 0x7f16180c93c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d3631fe2d5cc75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f160ffff700 1 -- 172.21.15.45:0/3783014208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400b860 con 0x7f16180c8ff0 2023-10-06T05:23:34.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f160ffff700 1 -- 172.21.15.45:0/3783014208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400b9c0 con 0x7f16180c8ff0 2023-10-06T05:23:34.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f160ffff700 1 -- 172.21.15.45:0/3783014208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400bd60 con 0x7f16180c8ff0 2023-10-06T05:23:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 -- 172.21.15.45:0/3783014208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180c8ff0 msgr2=0x7f16180c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 --2- 172.21.15.45:0/3783014208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180c8ff0 0x7f16180c93c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614009250 comp rx=0 tx=0).stop 2023-10-06T05:23:34.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 -- 172.21.15.45:0/3783014208 shutdown_connections 2023-10-06T05:23:34.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 --2- 172.21.15.45:0/3783014208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180c8ff0 0x7f16180c93c0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 --2- 172.21.15.45:0/3783014208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180caad0 0x7f16180caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 -- 172.21.15.45:0/3783014208 >> 172.21.15.45:0/3783014208 conn(0x7f16180cdf50 msgr2=0x7f16180ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 -- 172.21.15.45:0/3783014208 shutdown_connections 2023-10-06T05:23:34.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.496+0000 7f161f840700 1 -- 172.21.15.45:0/3783014208 wait complete. 2023-10-06T05:23:34.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.498+0000 7f161f840700 1 Processor -- start 2023-10-06T05:23:34.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.498+0000 7f161f840700 1 -- start start 2023-10-06T05:23:34.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161f840700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180c8ff0 0x7f1618150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161f840700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16180caad0 0x7f16181673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161f840700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1618156af0 0x7f1618157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161f840700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16180b0cf0 con 0x7f1618156af0 2023-10-06T05:23:34.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161f840700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16180b0f40 con 0x7f16180caad0 2023-10-06T05:23:34.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161f840700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16180b1220 con 0x7f16180c8ff0 2023-10-06T05:23:34.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1618156af0 0x7f1618157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1618156af0 0x7f1618157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32812/0 (socket says 172.21.15.45:32812) 2023-10-06T05:23:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 -- 172.21.15.45:0/3922820630 learned_addr learned my addr 172.21.15.45:0/3922820630 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161d5dc700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180c8ff0 0x7f1618150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161cddb700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16180caad0 0x7f16181673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 -- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180c8ff0 msgr2=0x7f1618150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180c8ff0 0x7f1618150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 -- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16180caad0 msgr2=0x7f16181673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16180caad0 0x7f16181673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 -- 172.21.15.45:0/3922820630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140094f0 con 0x7f1618156af0 2023-10-06T05:23:34.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161dddd700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1618156af0 0x7f1618157950 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f161000c840 tx=0x7f1610001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f160d7fa700 1 -- 172.21.15.45:0/3922820630 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610012630 con 0x7f1618156af0 2023-10-06T05:23:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f160d7fa700 1 -- 172.21.15.45:0/3922820630 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100127b0 con 0x7f1618156af0 2023-10-06T05:23:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.499+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16180b1500 con 0x7f1618156af0 2023-10-06T05:23:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.500+0000 7f160d7fa700 1 -- 172.21.15.45:0/3922820630 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610012b70 con 0x7f1618156af0 2023-10-06T05:23:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.500+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16180b3520 con 0x7f1618156af0 2023-10-06T05:23:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.501+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161806f410 con 0x7f1618156af0 2023-10-06T05:23:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.501+0000 7f160d7fa700 1 -- 172.21.15.45:0/3922820630 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f161001a3a0 con 0x7f1618156af0 2023-10-06T05:23:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.501+0000 7f160d7fa700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15fc066f30 0x7f15fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.502+0000 7f160d7fa700 1 -- 172.21.15.45:0/3922820630 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f1610084cc0 con 0x7f1618156af0 2023-10-06T05:23:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.502+0000 7f161d5dc700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15fc066f30 0x7f15fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.502+0000 7f161d5dc700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15fc066f30 0x7f15fc0693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.506+0000 7f160d7fa700 1 -- 172.21.15.45:0/3922820630 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1610051880 con 0x7f1618156af0 2023-10-06T05:23:34.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.751+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f161804bb40 con 0x7f1618156af0 2023-10-06T05:23:34.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.752+0000 7f160d7fa700 1 -- 172.21.15.45:0/3922820630 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f1610054ea0 con 0x7f1618156af0 2023-10-06T05:23:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:34.754 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":166,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:33.932432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8568},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":166,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/2405580153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2405580153},{"type":"v1","addr":"172.21.15.45:6835","nonce":2405580153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:34.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15fc066f30 msgr2=0x7f15fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15fc066f30 0x7f15fc0693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614017000 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1618156af0 msgr2=0x7f1618157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1618156af0 0x7f1618157950 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f161000c840 tx=0x7f1610001670 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 shutdown_connections 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f15fc066f30 0x7f15fc0693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1618156af0 0x7f1618157950 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16180c8ff0 0x7f1618150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 --2- 172.21.15.45:0/3922820630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16180caad0 0x7f16181673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.754+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 >> 172.21.15.45:0/3922820630 conn(0x7f16180cdf50 msgr2=0x7f16181519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.755+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 shutdown_connections 2023-10-06T05:23:34.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:34.755+0000 7f161f840700 1 -- 172.21.15.45:0/3922820630 wait complete. 2023-10-06T05:23:34.758 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 166 2023-10-06T05:23:34.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 166, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:23:33.932432+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 104, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8568}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 166, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/2405580153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 2405580153}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2405580153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:23:34.771 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8568:up:replay 2023-10-06T05:23:34.771 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:23:37.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:37.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:40.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:40.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.314+0000 7fb228780700 1 Processor -- start 2023-10-06T05:23:40.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.314+0000 7fb228780700 1 -- start start 2023-10-06T05:23:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb228780700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb220159200 0x7fb2200c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb228780700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2201595d0 0x7fb22006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb228780700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2200c7d00 con 0x7fb22006b5c0 2023-10-06T05:23:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb228780700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2200c7780 con 0x7fb220159200 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb228780700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2200c86d0 con 0x7fb2201595d0 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb22651c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb220159200 0x7fb2200c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb226d1d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb22006b5c0 0x7fb2200ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57752/0 (socket says 172.21.15.45:57752) 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb226d1d700 1 -- 172.21.15.45:0/4263870541 learned_addr learned my addr 172.21.15.45:0/4263870541 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb225d1b700 1 --2- 172.21.15.45:0/4263870541 >> v2:172.21.15.144:3300/0 conn(0x7fb2201595d0 0x7fb22006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb22651c700 1 -- 172.21.15.45:0/4263870541 >> v2:172.21.15.144:3300/0 conn(0x7fb2201595d0 msgr2=0x7fb22006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb22651c700 1 --2- 172.21.15.45:0/4263870541 >> v2:172.21.15.144:3300/0 conn(0x7fb2201595d0 0x7fb22006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb22651c700 1 -- 172.21.15.45:0/4263870541 >> v1:172.21.15.45:6789/0 conn(0x7fb22006b5c0 legacy=0x7fb2200ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb22651c700 1 -- 172.21.15.45:0/4263870541 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22006bb40 con 0x7fb220159200 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb225d1b700 1 --2- 172.21.15.45:0/4263870541 >> v2:172.21.15.144:3300/0 conn(0x7fb2201595d0 0x7fb22006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.315+0000 7fb22551a700 1 -- 172.21.15.45:0/4263870541 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432864906 0 0) 0x7fb2200c7d00 con 0x7fb22006b5c0 2023-10-06T05:23:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb22651c700 1 --2- 172.21.15.45:0/4263870541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200c8190 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb22006e790 tx=0x7fb2140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3e4367507fdf41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb22551a700 1 -- 172.21.15.45:0/4263870541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400b6b0 con 0x7fb220159200 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb22551a700 1 -- 172.21.15.45:0/4263870541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21400b810 con 0x7fb220159200 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb22551a700 1 -- 172.21.15.45:0/4263870541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400bbb0 con 0x7fb220159200 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 -- 172.21.15.45:0/4263870541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 msgr2=0x7fb2200c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 --2- 172.21.15.45:0/4263870541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200c8190 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb22006e790 tx=0x7fb2140091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 -- 172.21.15.45:0/4263870541 shutdown_connections 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 --2- 172.21.15.45:0/4263870541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200c8190 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 --2- 172.21.15.45:0/4263870541 >> v2:172.21.15.144:3300/0 conn(0x7fb2201595d0 0x7fb22006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 -- 172.21.15.45:0/4263870541 >> 172.21.15.45:0/4263870541 conn(0x7fb22006f090 msgr2=0x7fb22006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 -- 172.21.15.45:0/4263870541 shutdown_connections 2023-10-06T05:23:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.316+0000 7fb228780700 1 -- 172.21.15.45:0/4263870541 wait complete. 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 Processor -- start 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 -- start start 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22006b5c0 0x7fb22016c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2201595d0 0x7fb2201698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200b0430 con 0x7fb220159200 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200c55b0 con 0x7fb22006b5c0 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.318+0000 7fb228780700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200b1b20 con 0x7fb2201595d0 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38110/0 (socket says 172.21.15.45:38110) 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 -- 172.21.15.45:0/3523974240 learned_addr learned my addr 172.21.15.45:0/3523974240 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb22651c700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22006b5c0 0x7fb22016c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb226d1d700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2201595d0 0x7fb2201698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 -- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2201595d0 msgr2=0x7fb2201698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2201595d0 0x7fb2201698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 -- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22006b5c0 msgr2=0x7fb22016c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22006b5c0 0x7fb22016c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 -- 172.21.15.45:0/3523974240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb214009420 con 0x7fb220159200 2023-10-06T05:23:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb22651c700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22006b5c0 0x7fb22016c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb225d1b700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200afbe0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00d860 tx=0x7fb21c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb2127fc700 1 -- 172.21.15.45:0/3523974240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0125f0 con 0x7fb220159200 2023-10-06T05:23:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb2127fc700 1 -- 172.21.15.45:0/3523974240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c012750 con 0x7fb220159200 2023-10-06T05:23:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2200b1810 con 0x7fb220159200 2023-10-06T05:23:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb2127fc700 1 -- 172.21.15.45:0/3523974240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c012ad0 con 0x7fb220159200 2023-10-06T05:23:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.319+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2200c5ae0 con 0x7fb220159200 2023-10-06T05:23:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.320+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2200b0180 con 0x7fb220159200 2023-10-06T05:23:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.321+0000 7fb2127fc700 1 -- 172.21.15.45:0/3523974240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb21c014430 con 0x7fb220159200 2023-10-06T05:23:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.321+0000 7fb2127fc700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb204066f30 0x7fb2040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.321+0000 7fb2127fc700 1 -- 172.21.15.45:0/3523974240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fb21c084fe0 con 0x7fb220159200 2023-10-06T05:23:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.321+0000 7fb22651c700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb204066f30 0x7fb2040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.322+0000 7fb22651c700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb204066f30 0x7fb2040693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb2140093f0 tx=0x7fb214017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.326+0000 7fb2127fc700 1 -- 172.21.15.45:0/3523974240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb21c055630 con 0x7fb220159200 2023-10-06T05:23:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.573+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2200b1c60 con 0x7fb220159200 2023-10-06T05:23:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.573+0000 7fb2127fc700 1 -- 172.21.15.45:0/3523974240 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+4608 (secure 0 0 0) 0x7fb21c0551c0 con 0x7fb220159200 2023-10-06T05:23:40.575 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:40.575 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":169,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:36.953801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8568},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":166,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2405580153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2405580153},{"type":"v1","addr":"172.21.15.45:6835","nonce":2405580153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb204066f30 msgr2=0x7fb2040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb204066f30 0x7fb2040693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb2140093f0 tx=0x7fb214017000 comp rx=0 tx=0).stop 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 msgr2=0x7fb2200afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200afbe0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00d860 tx=0x7fb21c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 shutdown_connections 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb204066f30 0x7fb2040693e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2201595d0 0x7fb2201698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22006b5c0 0x7fb22016c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 --2- 172.21.15.45:0/3523974240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb220159200 0x7fb2200afbe0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 >> 172.21.15.45:0/3523974240 conn(0x7fb22006f090 msgr2=0x7fb22015b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 shutdown_connections 2023-10-06T05:23:40.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.576+0000 7fb228780700 1 -- 172.21.15.45:0/3523974240 wait complete. 2023-10-06T05:23:40.579 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 169 2023-10-06T05:23:40.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 169, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:23:36.953801+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 104, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8568}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 166, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/2405580153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 2405580153}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2405580153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:23:40.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:23:40.591 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:23:40.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.722+0000 7fdb569f6700 1 Processor -- start 2023-10-06T05:23:40.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.722+0000 7fdb569f6700 1 -- start start 2023-10-06T05:23:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.723+0000 7fdb569f6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb500ccaa0 0x7fdb5006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.723+0000 7fdb569f6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb500cb530 con 0x7fdb50159330 2023-10-06T05:23:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.723+0000 7fdb569f6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb50069bd0 con 0x7fdb50158f60 2023-10-06T05:23:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.723+0000 7fdb569f6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb50068fc0 con 0x7fdb500ccaa0 2023-10-06T05:23:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.723+0000 7fdb4f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdb50159330 0x7fdb500cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57758/0 (socket says 172.21.15.45:57758) 2023-10-06T05:23:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.723+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/2517903361 learned_addr learned my addr 172.21.15.45:0/2517903361 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.723+0000 7fdb54f93700 1 --2- 172.21.15.45:0/2517903361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb500ccaa0 0x7fdb5006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728612629 0 0) 0x7fdb38001110 con 0x7fdb50159330 2023-10-06T05:23:40.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb3c003570 con 0x7fdb50159330 2023-10-06T05:23:40.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283714318 0 0) 0x7fdb50069bd0 con 0x7fdb50158f60 2023-10-06T05:23:40.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb38001110 con 0x7fdb50158f60 2023-10-06T05:23:40.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb54f93700 1 -- 172.21.15.45:0/2517903361 >> v1:172.21.15.144:6789/0 conn(0x7fdb50158f60 legacy=0x7fdb500cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965872706 0 0) 0x7fdb3c003570 con 0x7fdb50159330 2023-10-06T05:23:40.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb54f93700 1 -- 172.21.15.45:0/2517903361 >> v1:172.21.15.45:6789/0 conn(0x7fdb50159330 legacy=0x7fdb500cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb54f93700 1 -- 172.21.15.45:0/2517903361 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500692a0 con 0x7fdb500ccaa0 2023-10-06T05:23:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb54f93700 1 --2- 172.21.15.45:0/2517903361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb500ccaa0 0x7fdb5006a060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009e90 tx=0x7fdb48009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564b2c8569848e34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800e070 con 0x7fdb500ccaa0 2023-10-06T05:23:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4800b870 con 0x7fdb500ccaa0 2023-10-06T05:23:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.724+0000 7fdb4effd700 1 -- 172.21.15.45:0/2517903361 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38001600 con 0x7fdb500ccaa0 2023-10-06T05:23:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.725+0000 7fdb569f6700 1 -- 172.21.15.45:0/2517903361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb500ccaa0 msgr2=0x7fdb5006a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.725+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2517903361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb500ccaa0 0x7fdb5006a060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009e90 tx=0x7fdb48009780 comp rx=0 tx=0).stop 2023-10-06T05:23:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.725+0000 7fdb569f6700 1 -- 172.21.15.45:0/2517903361 shutdown_connections 2023-10-06T05:23:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.725+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2517903361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb500ccaa0 0x7fdb5006a060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.725+0000 7fdb569f6700 1 -- 172.21.15.45:0/2517903361 >> 172.21.15.45:0/2517903361 conn(0x7fdb500cdf50 msgr2=0x7fdb500ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.725+0000 7fdb569f6700 1 -- 172.21.15.45:0/2517903361 shutdown_connections 2023-10-06T05:23:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.725+0000 7fdb569f6700 1 -- 172.21.15.45:0/2517903361 wait complete. 2023-10-06T05:23:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.727+0000 7fdb569f6700 1 Processor -- start 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.727+0000 7fdb569f6700 1 -- start start 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb569f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb500ccaa0 0x7fdb50153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb569f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb50158f60 0x7fdb5015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb569f6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb50159330 0x7fdb50152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb569f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb501547b0 con 0x7fdb50158f60 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb569f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb500b14e0 con 0x7fdb500ccaa0 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb569f6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb501544d0 con 0x7fdb50159330 2023-10-06T05:23:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb50158f60 0x7fdb5015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb50158f60 0x7fdb5015f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38124/0 (socket says 172.21.15.45:38124) 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/2688745102 learned_addr learned my addr 172.21.15.45:0/2688745102 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb54f93700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb50159330 0x7fdb50152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb500ccaa0 0x7fdb50153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb50159330 msgr2=0x7fdb50152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb50159330 0x7fdb50152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb500ccaa0 msgr2=0x7fdb50153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb500ccaa0 0x7fdb50153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/2688745102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48009a20 con 0x7fdb50158f60 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb500ccaa0 0x7fdb50153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.728+0000 7fdb54f93700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb50159330 0x7fdb50152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.729+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb50158f60 0x7fdb5015f0e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800cb60 tx=0x7fdb38001ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.729+0000 7fdb47fff700 1 -- 172.21.15.45:0/2688745102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007430 con 0x7fdb50158f60 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.729+0000 7fdb47fff700 1 -- 172.21.15.45:0/2688745102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb380075b0 con 0x7fdb50158f60 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.729+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb500b0e90 con 0x7fdb50158f60 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.729+0000 7fdb47fff700 1 -- 172.21.15.45:0/2688745102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38004a80 con 0x7fdb50158f60 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.729+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb50154110 con 0x7fdb50158f60 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.730+0000 7fdb47fff700 1 -- 172.21.15.45:0/2688745102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb3800a630 con 0x7fdb50158f60 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.731+0000 7fdb47fff700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb30066f30 0x7fdb300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.731+0000 7fdb47fff700 1 -- 172.21.15.45:0/2688745102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fdb380844f0 con 0x7fdb50158f60 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.731+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb30066f30 0x7fdb300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.732+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb30066f30 0x7fdb300693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdb40007fd0 tx=0x7fdb40009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.733+0000 7fdb5fd0ab80 1 Processor -- start 2023-10-06T05:23:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.733+0000 7fdb5fd0ab80 1 -- start start 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd489450 0x5642dd489820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd46ef40 0x5642dd491f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd4924d0 0x5642dd492940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd35c310 con 0x5642dd489450 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd399130 con 0x5642dd46ef40 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642dd396af0 con 0x5642dd4924d0 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd489450 0x5642dd489820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd489450 0x5642dd489820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38128/0 (socket says 172.21.15.45:38128) 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 -- 172.21.15.45:0/3238944444 learned_addr learned my addr 172.21.15.45:0/3238944444 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd46ef40 0x5642dd491f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb54f93700 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd4924d0 0x5642dd492940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 -- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd4924d0 msgr2=0x5642dd492940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd4924d0 0x5642dd492940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 -- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd46ef40 msgr2=0x5642dd491f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd46ef40 0x5642dd491f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4ffff700 1 -- 172.21.15.45:0/3238944444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642dd198790 con 0x5642dd489450 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.734+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd46ef40 0x5642dd491f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.735+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd489450 0x5642dd489820 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000daf0 tx=0x7fdb40013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abe1e24e463ff8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.735+0000 7fdb457fa700 1 -- 172.21.15.45:0/3238944444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40014b20 con 0x5642dd489450 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.735+0000 7fdb457fa700 1 -- 172.21.15.45:0/3238944444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb40014c80 con 0x5642dd489450 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.735+0000 7fdb457fa700 1 -- 172.21.15.45:0/3238944444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4001f400 con 0x5642dd489450 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.735+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3238944444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd489450 msgr2=0x5642dd489820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.735+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd489450 0x5642dd489820 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000daf0 tx=0x7fdb40013000 comp rx=0 tx=0).stop 2023-10-06T05:23:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3238944444 shutdown_connections 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd4924d0 0x5642dd492940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd489450 0x5642dd489820 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3238944444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd46ef40 0x5642dd491f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3238944444 >> 172.21.15.45:0/3238944444 conn(0x5642dd48c880 msgr2=0x5642dd48cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3238944444 shutdown_connections 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3238944444 wait complete. 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 -- start start 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.736+0000 7fdb5fd0ab80 1 Processor -- start 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd462900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd462fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd467710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd3928e0 con 0x5642dd4924d0 2023-10-06T05:23:40.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd33e980 con 0x5642dd489450 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642dd33e620 con 0x5642dd46ef40 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb5fd0ab80 10 client.0 _ll_register_callbacks cb 0x5642dd1a1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb54f93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd467710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb54f93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd467710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38140/0 (socket says 172.21.15.45:38140) 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb54f93700 1 -- 172.21.15.45:0/1437143553 learned_addr learned my addr 172.21.15.45:0/1437143553 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd462fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.737+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd462900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb54f93700 1 -- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 msgr2=0x5642dd462900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb54f93700 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd462900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb54f93700 1 -- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 msgr2=0x5642dd462fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb54f93700 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd462fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb54f93700 1 -- 172.21.15.45:0/1437143553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb38001110 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb54f93700 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd467710 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b840 tx=0x7fdb48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb83d9da8ca0ad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb2e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/1437143553 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800b9e0 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/1437143553 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fdb4801b450 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642dd436d60 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/1437143553 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=169) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4801b820 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb5fd0ab80 10 client.8664 fetch_fsmap learned FSMap version 169 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=169,monmap=2+}) v3 -- 0x5642dd2850a0 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/1437143553 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5642dd436d60 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.738+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/1437143553 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 169) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdb4801bc50 con 0x5642dd4924d0 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb5fd0ab80 10 client.8664 fetch_fsmap finished waiting for FSMap version 169 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb5fd0ab80 10 client.8664 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:23:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb5fd0ab80 20 client.8664 populate_metadata read hostname 'smithi045' 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd48ab60 0x5642dd48af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb5fd0ab80 4 client.8664 mds_command: new command op to 8568 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 --> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642dd413b30 con 0x5642dd48ab60 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd48ab60 0x5642dd48af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd48ab60 0x5642dd48af30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1bd7e6f588f16a server_cookie=a8d6a907268e2b66 in_seq=0 out_seq=0 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.739+0000 7fdb2e7fc700 10 client.8664 ms_handle_connect on v2:172.21.15.45:6834/2405580153 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/1437143553 <== mds.0 v2:172.21.15.45:6834/2405580153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5642dd413b30 con 0x5642dd48ab60 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb2e7fc700 10 client.8664 handle_command_reply: tid=0 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 client.8664 shutdown 2023-10-06T05:23:40.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 msgr2=0x5642dd467710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd467710 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b840 tx=0x7fdb48017000 comp rx=0 tx=0).stop 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 shutdown_connections 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd48ab60 0x5642dd48af30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd467710 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd462900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/1437143553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd462fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb2e7fc700 0 client.8664 ms_handle_reset on v2:172.21.15.45:6834/2405580153 2023-10-06T05:23:40.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 >> 172.21.15.45:0/1437143553 conn(0x5642dd48c880 msgr2=0x5642dd43ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 shutdown_connections 2023-10-06T05:23:40.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/1437143553 wait complete. 2023-10-06T05:23:40.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.740+0000 7fdb5fd0ab80 20 client.8664 trim_cache size 0 max 16384 2023-10-06T05:23:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.758+0000 7fdb5fd0ab80 1 Processor -- start 2023-10-06T05:23:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.758+0000 7fdb5fd0ab80 1 -- start start 2023-10-06T05:23:40.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd48a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd22a1e0 con 0x5642dd4924d0 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd472770 con 0x5642dd489450 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642dd441640 con 0x5642dd46ef40 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38142/0 (socket says 172.21.15.45:38142) 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 -- 172.21.15.45:0/3397771446 learned_addr learned my addr 172.21.15.45:0/3397771446 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd48a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 -- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 msgr2=0x5642dd48a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd48a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 -- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 msgr2=0x5642dd4683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 -- 172.21.15.45:0/3397771446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642dd3613c0 con 0x5642dd4924d0 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd48a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb54f93700 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468930 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fdb480097c0 tx=0x7fdb48020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32da0f71c8f0846a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb45ffb700 1 -- 172.21.15.45:0/3397771446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4801b820 con 0x5642dd4924d0 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb45ffb700 1 -- 172.21.15.45:0/3397771446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4801bc50 con 0x5642dd4924d0 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.759+0000 7fdb45ffb700 1 -- 172.21.15.45:0/3397771446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800bc30 con 0x5642dd4924d0 2023-10-06T05:23:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3397771446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 msgr2=0x5642dd468930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468930 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fdb480097c0 tx=0x7fdb48020040 comp rx=0 tx=0).stop 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3397771446 shutdown_connections 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468930 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd48a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/3397771446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3397771446 >> 172.21.15.45:0/3397771446 conn(0x5642dd48c880 msgr2=0x5642dd43ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3397771446 shutdown_connections 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.760+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/3397771446 wait complete. 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.761+0000 7fdb5fd0ab80 1 -- start start 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.761+0000 7fdb5fd0ab80 1 Processor -- start 2023-10-06T05:23:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd4ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb5fd0ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd4ef280 con 0x5642dd4924d0 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642dd4edfb0 con 0x5642dd489450 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb5fd0ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642dd441e20 con 0x5642dd46ef40 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb5fd0ab80 10 client.0 _ll_register_callbacks cb 0x5642dd1a1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb54f93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb4ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd4ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb4f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb54f93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38148/0 (socket says 172.21.15.45:38148) 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb54f93700 1 -- 172.21.15.45:0/818692952 learned_addr learned my addr 172.21.15.45:0/818692952 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/818692952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 msgr2=0x5642dd4ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd4ed9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 msgr2=0x5642dd468390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb4f7fe700 1 -- 172.21.15.45:0/818692952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642dd2850a0 con 0x5642dd489450 2023-10-06T05:23:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.762+0000 7fdb54f93700 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb4f7fe700 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4eb070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdb38082440 tx=0x7fdb3801f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ec9f3a7fe46462 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb2e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-10-06T05:23:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/818692952 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3805bd90 con 0x5642dd489450 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb40014c80 con 0x5642dd489450 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/818692952 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fdb38007720 con 0x5642dd489450 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/818692952 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=169) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb3805a0f0 con 0x5642dd489450 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 10 client.8275 fetch_fsmap learned FSMap version 169 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=169,monmap=2+}) v3 -- 0x5642dd1f91d0 con 0x5642dd489450 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/818692952 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb40014c80 con 0x5642dd489450 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/818692952 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 169) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdb3805b900 con 0x5642dd489450 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 10 client.8275 fetch_fsmap finished waiting for FSMap version 169 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 10 client.8275 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 20 client.8275 populate_metadata read hostname 'smithi045' 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd442650 0x5642dd442a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 4 client.8275 mds_command: new command op to 8568 tid=0[{"prefix": "status"}] 2023-10-06T05:23:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.763+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 --> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5642dd3358d0 con 0x5642dd442650 2023-10-06T05:23:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd442650 0x5642dd442a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb2e7fc700 10 client.8275 ms_handle_connect on v2:172.21.15.45:6834/2405580153 2023-10-06T05:23:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb4ffff700 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd442650 0x5642dd442a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8534ea06992386 server_cookie=cdf8f3f9eed2d808 in_seq=0 out_seq=0 2023-10-06T05:23:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb2e7fc700 1 -- 172.21.15.45:0/818692952 <== mds.0 v2:172.21.15.45:6834/2405580153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5642dd3358d0 con 0x5642dd442650 2023-10-06T05:23:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb2e7fc700 10 client.8275 handle_command_reply: tid=0 2023-10-06T05:23:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 client.8275 shutdown 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 msgr2=0x5642dd4eb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4eb070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdb38082440 tx=0x7fdb3801f040 comp rx=0 tx=0).stop 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 shutdown_connections 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:6834/2405580153,v1:172.21.15.45:6835/2405580153] conn(0x5642dd442650 0x5642dd442a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642dd4924d0 0x5642dd468390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5642dd46ef40 0x5642dd4ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb2e7fc700 0 client.8275 ms_handle_reset on v2:172.21.15.45:6834/2405580153 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 --2- 172.21.15.45:0/818692952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5642dd489450 0x5642dd4eb070 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 >> 172.21.15.45:0/818692952 conn(0x5642dd48c880 msgr2=0x5642dd464d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 shutdown_connections 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.764+0000 7fdb5fd0ab80 1 -- 172.21.15.45:0/818692952 wait complete. 2023-10-06T05:23:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb5fd0ab80 20 client.8275 trim_cache size 0 max 16384 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8568, 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:23:40.779 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8278365860000001, 2023-10-06T05:23:40.780 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 169, 2023-10-06T05:23:40.780 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 104, 2023-10-06T05:23:40.780 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 104, 2023-10-06T05:23:40.780 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.828787854 2023-10-06T05:23:40.780 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:23:40.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb30066f30 msgr2=0x7fdb300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb30066f30 0x7fdb300693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdb40007fd0 tx=0x7fdb40009040 comp rx=0 tx=0).stop 2023-10-06T05:23:40.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb50158f60 msgr2=0x7fdb5015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb50158f60 0x7fdb5015f0e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800cb60 tx=0x7fdb38001ce0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 shutdown_connections 2023-10-06T05:23:40.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb30066f30 0x7fdb300693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb50159330 0x7fdb50152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb500ccaa0 0x7fdb50153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 --2- 172.21.15.45:0/2688745102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb50158f60 0x7fdb5015f0e0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 >> 172.21.15.45:0/2688745102 conn(0x7fdb500cdf50 msgr2=0x7fdb50070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 shutdown_connections 2023-10-06T05:23:40.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.765+0000 7fdb569f6700 1 -- 172.21.15.45:0/2688745102 wait complete. 2023-10-06T05:23:40.782 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:23:40.782 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2023-10-06T05:23:40.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.913+0000 7f0a3a744700 1 Processor -- start 2023-10-06T05:23:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.913+0000 7f0a3a744700 1 -- start start 2023-10-06T05:23:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a3a744700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a340c8ff0 0x7f0a340c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a3a744700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3415b000 con 0x7f0a34157200 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a3a744700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3415b2e0 con 0x7f0a340caad0 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a3a744700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3415b5c0 con 0x7f0a340c8ff0 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a33fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a340c8ff0 0x7f0a340c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a38ce1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0a34157200 0x7f0a340cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57766/0 (socket says 172.21.15.45:57766) 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a38ce1700 1 -- 172.21.15.45:0/3294459105 learned_addr learned my addr 172.21.15.45:0/3294459105 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.914+0000 7f0a33fff700 1 --2- 172.21.15.45:0/3294459105 >> v2:172.21.15.45:3300/0 conn(0x7f0a340c8ff0 0x7f0a340c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38154/0 (socket says 172.21.15.45:38154) 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a33fff700 1 -- 172.21.15.45:0/3294459105 >> v1:172.21.15.144:6789/0 conn(0x7f0a340caad0 legacy=0x7f0a34156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a33fff700 1 -- 172.21.15.45:0/3294459105 >> v1:172.21.15.45:6789/0 conn(0x7f0a34157200 legacy=0x7f0a340cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a33fff700 1 -- 172.21.15.45:0/3294459105 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3415b8a0 con 0x7f0a340c8ff0 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a32ffd700 1 -- 172.21.15.45:0/3294459105 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99321998 0 0) 0x7f0a3415b000 con 0x7f0a34157200 2023-10-06T05:23:40.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a33fff700 1 --2- 172.21.15.45:0/3294459105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340c8ff0 0x7f0a340c93c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0a240098e0 tx=0x7f0a240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f1ef1cad4e135f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a32ffd700 1 -- 172.21.15.45:0/3294459105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24009e80 con 0x7f0a340c8ff0 2023-10-06T05:23:40.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a32ffd700 1 -- 172.21.15.45:0/3294459105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2400b820 con 0x7f0a340c8ff0 2023-10-06T05:23:40.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a32ffd700 1 -- 172.21.15.45:0/3294459105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400bba0 con 0x7f0a340c8ff0 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a3a744700 1 -- 172.21.15.45:0/3294459105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340c8ff0 msgr2=0x7f0a340c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.915+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3294459105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340c8ff0 0x7f0a340c93c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0a240098e0 tx=0x7f0a240091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.916+0000 7f0a3a744700 1 -- 172.21.15.45:0/3294459105 shutdown_connections 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.916+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3294459105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340c8ff0 0x7f0a340c93c0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.916+0000 7f0a3a744700 1 -- 172.21.15.45:0/3294459105 >> 172.21.15.45:0/3294459105 conn(0x7f0a340cdf50 msgr2=0x7f0a340ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.916+0000 7f0a3a744700 1 -- 172.21.15.45:0/3294459105 shutdown_connections 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.916+0000 7f0a3a744700 1 -- 172.21.15.45:0/3294459105 wait complete. 2023-10-06T05:23:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.917+0000 7f0a3a744700 1 Processor -- start 2023-10-06T05:23:40.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.917+0000 7f0a3a744700 1 -- start start 2023-10-06T05:23:40.920 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:40.917+0000 7fb8e6037700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a3a744700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a340c8ff0 0x7f0a340b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a3a744700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340caad0 0x7f0a3415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a3a744700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34157200 0x7f0a340b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a3a744700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340b2340 con 0x7f0a340caad0 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a3a744700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340b2570 con 0x7f0a34157200 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a3a744700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340b2850 con 0x7f0a340c8ff0 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340caad0 0x7f0a3415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340caad0 0x7f0a3415a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38166/0 (socket says 172.21.15.45:38166) 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 -- 172.21.15.45:0/3107501671 learned_addr learned my addr 172.21.15.45:0/3107501671 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a38ce1700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34157200 0x7f0a340b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a33fff700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a340c8ff0 0x7f0a340b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 -- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a340c8ff0 msgr2=0x7f0a340b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a340c8ff0 0x7f0a340b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 -- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34157200 msgr2=0x7f0a340b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34157200 0x7f0a340b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 -- 172.21.15.45:0/3107501671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a24009420 con 0x7f0a340caad0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.918+0000 7f0a337fe700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340caad0 0x7f0a3415a5d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800c5d0 tx=0x7f0a28007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.919+0000 7f0a1bfff700 1 -- 172.21.15.45:0/3107501671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28011590 con 0x7f0a340caad0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.919+0000 7f0a1bfff700 1 -- 172.21.15.45:0/3107501671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a28011710 con 0x7f0a340caad0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.919+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a340b2b30 con 0x7f0a340caad0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.919+0000 7f0a1bfff700 1 -- 172.21.15.45:0/3107501671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28011ad0 con 0x7f0a340caad0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.919+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a340b30f0 con 0x7f0a340caad0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.920+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a340b3310 con 0x7f0a340caad0 2023-10-06T05:23:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.920+0000 7f0a1bfff700 1 -- 172.21.15.45:0/3107501671 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a2801c460 con 0x7f0a340caad0 2023-10-06T05:23:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.921+0000 7f0a1bfff700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a20066f30 0x7f0a200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.921+0000 7f0a1bfff700 1 -- 172.21.15.45:0/3107501671 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f0a280848a0 con 0x7f0a340caad0 2023-10-06T05:23:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.921+0000 7f0a33fff700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a20066f30 0x7f0a200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.921+0000 7f0a33fff700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a20066f30 0x7f0a200693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0a240030c0 tx=0x7f0a24016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:40.925+0000 7f0a1bfff700 1 -- 172.21.15.45:0/3107501671 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a28051460 con 0x7f0a340caad0 2023-10-06T05:23:41.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.171+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a3404bb40 con 0x7f0a340caad0 2023-10-06T05:23:41.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.171+0000 7f0a1bfff700 1 -- 172.21.15.45:0/3107501671 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v169) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0a28054a80 con 0x7f0a340caad0 2023-10-06T05:23:41.174 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:41.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a20066f30 msgr2=0x7f0a200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a20066f30 0x7f0a200693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0a240030c0 tx=0x7f0a24016000 comp rx=0 tx=0).stop 2023-10-06T05:23:41.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340caad0 msgr2=0x7f0a3415a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340caad0 0x7f0a3415a5d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800c5d0 tx=0x7f0a28007f60 comp rx=0 tx=0).stop 2023-10-06T05:23:41.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 shutdown_connections 2023-10-06T05:23:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0a20066f30 0x7f0a200693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34157200 0x7f0a340b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a340c8ff0 0x7f0a340b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 --2- 172.21.15.45:0/3107501671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a340caad0 0x7f0a3415a5d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 >> 172.21.15.45:0/3107501671 conn(0x7f0a340cdf50 msgr2=0x7f0a34153bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 shutdown_connections 2023-10-06T05:23:41.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:41.174+0000 7f0a3a744700 1 -- 172.21.15.45:0/3107501671 wait complete. 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:41.189 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:41.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:41.190 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:23:41.210 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41920sec preferred_lft 41920sec 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:23:41.212 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:23:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:23:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:41.214 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:23:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:23:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:41.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:41.215 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:41.282 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:23:41.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:41.283 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:23:41.349 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:23:41.349 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:41.349 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:23:41.349 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:23:41.349 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:41.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:41.350 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:23:41.350 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:23:41.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:23:41.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:23:41.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:23:41.350 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:23:41.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:41.473 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:23:41.495 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:23:41.495 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:23:41.495 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:41.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:41.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:23:41.572 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:23:41.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:41.572 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:41.636 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:41.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:41.637 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:41.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:23:41.776 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:23:41.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:23:41.851 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:23:41.852 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:23:41.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.852 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:23:41.917 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:23:41.917 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:23:41.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:23:41.994 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:23:41.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:23:42.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:23:42.060+0000 7fdafe706540 -1 init, newargv = 0x55cf8461d7a0 newargc=15 2023-10-06T05:23:42.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[164946]: starting ceph client 2023-10-06T05:23:42.067 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:23:42.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[164946]: starting fuse 2023-10-06T05:23:42.069 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:23:42.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:42.069 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:23:42.130 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:23:42.130 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:23:42.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:23:42.130 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:23:42.130 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:23:42.131 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:23:42.132 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:23:42.270 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.164946.asok 2023-10-06T05:23:42.452 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.164946.asok 2023-10-06T05:23:42.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:42.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.164946.asok status 2023-10-06T05:23:42.626 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:23:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:23:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:23:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:23:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:23:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "164946", 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8685, 2023-10-06T05:23:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 8685 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:23:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2078996731 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2078996731 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.8685 192.168.0.1:0/2078996731", 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2078996731", 2023-10-06T05:23:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:23:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 169, 2023-10-06T05:23:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 104, 2023-10-06T05:23:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:23:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:23:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:23:42.634 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:23:42.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:42.638 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:23:42.638 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:42.659 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:23:42.659 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:42.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:42.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:42.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:42.732 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:23:42.775 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:23:42.801 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:23:42.802 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:23:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:42.802 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:23:42.802 INFO:teuthology.orchestra.run.smithi045.stdout:sixmegs 2023-10-06T05:23:42.802 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2023-10-06T05:23:42.802 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2023-10-06T05:23:42.802 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:23:42.803 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:23:42.803 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:42.888 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:23:42.888 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:42.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:42.888 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:42.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.132+0000 7f452f5df700 1 Processor -- start 2023-10-06T05:23:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.133+0000 7f452f5df700 1 -- start start 2023-10-06T05:23:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452f5df700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4528153a20 0x7f452806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452f5df700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280676c0 con 0x7f452806add0 2023-10-06T05:23:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452f5df700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280679a0 con 0x7f4528153650 2023-10-06T05:23:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452f5df700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452806b270 con 0x7f4528153a20 2023-10-06T05:23:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452db7c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f452806add0 0x7f4528068480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57774/0 (socket says 172.21.15.45:57774) 2023-10-06T05:23:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452db7c700 1 -- 172.21.15.45:0/3383331629 learned_addr learned my addr 172.21.15.45:0/3383331629 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452cb7a700 1 --2- 172.21.15.45:0/3383331629 >> v2:172.21.15.144:3300/0 conn(0x7f4528153a20 0x7f452806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958123653 0 0) 0x7f45280676c0 con 0x7f452806add0 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4514003570 con 0x7f452806add0 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786802394 0 0) 0x7f45280679a0 con 0x7f4528153650 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45280676c0 con 0x7f4528153650 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452cb7a700 1 -- 172.21.15.45:0/3383331629 >> v1:172.21.15.144:6789/0 conn(0x7f4528153650 legacy=0x7f452815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2270491375 0 0) 0x7f4514003570 con 0x7f452806add0 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452cb7a700 1 -- 172.21.15.45:0/3383331629 >> v1:172.21.15.45:6789/0 conn(0x7f452806add0 legacy=0x7f4528068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.134+0000 7f452cb7a700 1 -- 172.21.15.45:0/3383331629 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528067c80 con 0x7f4528153a20 2023-10-06T05:23:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452cb7a700 1 --2- 172.21.15.45:0/3383331629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153a20 0x7f452806a890 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098e0 tx=0x7f451c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36be8165cf4954f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200017f0 con 0x7f4528153a20 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c011420 con 0x7f4528153a20 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f451bfff700 1 -- 172.21.15.45:0/3383331629 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0117c0 con 0x7f4528153a20 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452f5df700 1 -- 172.21.15.45:0/3383331629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153a20 msgr2=0x7f452806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452f5df700 1 --2- 172.21.15.45:0/3383331629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153a20 0x7f452806a890 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098e0 tx=0x7f451c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452f5df700 1 -- 172.21.15.45:0/3383331629 shutdown_connections 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452f5df700 1 --2- 172.21.15.45:0/3383331629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153a20 0x7f452806a890 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452f5df700 1 -- 172.21.15.45:0/3383331629 >> 172.21.15.45:0/3383331629 conn(0x7f45280cc5a0 msgr2=0x7f45280cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452f5df700 1 -- 172.21.15.45:0/3383331629 shutdown_connections 2023-10-06T05:23:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.135+0000 7f452f5df700 1 -- 172.21.15.45:0/3383331629 wait complete. 2023-10-06T05:23:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.137+0000 7f452f5df700 1 Processor -- start 2023-10-06T05:23:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.137+0000 7f452f5df700 1 -- start start 2023-10-06T05:23:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452f5df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f452806add0 0x7f45280c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452f5df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153650 0x7f4528158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452f5df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4528153a20 0x7f45280b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452f5df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452816d0e0 con 0x7f452806add0 2023-10-06T05:23:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452f5df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452816df40 con 0x7f4528153650 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452f5df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452816d330 con 0x7f4528153a20 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f452806add0 0x7f45280c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f452806add0 0x7f45280c1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38178/0 (socket says 172.21.15.45:38178) 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 -- 172.21.15.45:0/96029279 learned_addr learned my addr 172.21.15.45:0/96029279 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452cb7a700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153650 0x7f4528158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452db7c700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4528153a20 0x7f45280b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 -- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4528153a20 msgr2=0x7f45280b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4528153a20 0x7f45280b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 -- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153650 msgr2=0x7f4528158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153650 0x7f4528158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 -- 172.21.15.45:0/96029279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c009420 con 0x7f452806add0 2023-10-06T05:23:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452db7c700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4528153a20 0x7f45280b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452d37b700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f452806add0 0x7f45280c1700 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f452400c7c0 tx=0x7f4524007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f45197fa700 1 -- 172.21.15.45:0/96029279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524010510 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f45197fa700 1 -- 172.21.15.45:0/96029279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524010670 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f45197fa700 1 -- 172.21.15.45:0/96029279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524010a10 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.138+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452816d610 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.139+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452816dbd0 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.139+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4528067f60 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.140+0000 7f45197fa700 1 -- 172.21.15.45:0/96029279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f452401b3b0 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.140+0000 7f45197fa700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f450c066f30 0x7f450c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.141+0000 7f45197fa700 1 -- 172.21.15.45:0/96029279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f4524084460 con 0x7f452806add0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.141+0000 7f452cb7a700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f450c066f30 0x7f450c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.141+0000 7f452cb7a700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f450c066f30 0x7f450c0693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4528157350 tx=0x7f451c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.145+0000 7f45197fa700 1 -- 172.21.15.45:0/96029279 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4524051020 con 0x7f452806add0 2023-10-06T05:23:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.391+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f452804bb40 con 0x7f452806add0 2023-10-06T05:23:43.393 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:43.391+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:43.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.395+0000 7f45197fa700 1 -- 172.21.15.45:0/96029279 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v170) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4524054640 con 0x7f452806add0 2023-10-06T05:23:43.399 INFO:tasks.ceph.mds.c.smithi045.stderr: -521> 2023-10-06T05:23:40.917+0000 7fb8e6037700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.398+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f450c066f30 msgr2=0x7f450c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.398+0000 7f452f5df700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f450c066f30 0x7f450c0693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4528157350 tx=0x7f451c016000 comp rx=0 tx=0).stop 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.398+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f452806add0 msgr2=0x7f45280c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.398+0000 7f452f5df700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f452806add0 0x7f45280c1700 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f452400c7c0 tx=0x7f4524007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 shutdown_connections 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f450c066f30 0x7f450c0693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4528153a20 0x7f45280b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f452806add0 0x7f45280c1700 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 --2- 172.21.15.45:0/96029279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4528153650 0x7f4528158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 >> 172.21.15.45:0/96029279 conn(0x7f45280cc5a0 msgr2=0x7f452815b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 shutdown_connections 2023-10-06T05:23:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.399+0000 7f452f5df700 1 -- 172.21.15.45:0/96029279 wait complete. 2023-10-06T05:23:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:43.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:43.421 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.545+0000 7f8b16584700 1 Processor -- start 2023-10-06T05:23:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.546+0000 7f8b16584700 1 -- start start 2023-10-06T05:23:43.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.546+0000 7f8b16584700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8b10159200 0x7f8b100c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.546+0000 7f8b16584700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b1006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.546+0000 7f8b16584700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b100c7d00 con 0x7f8b1006b5c0 2023-10-06T05:23:43.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.546+0000 7f8b16584700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b100c7780 con 0x7f8b10159200 2023-10-06T05:23:43.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.546+0000 7f8b16584700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100c86d0 con 0x7f8b101595d0 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b14b21700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8b1006b5c0 0x7f8b100ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57786/0 (socket says 172.21.15.45:57786) 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b14b21700 1 -- 172.21.15.45:0/1023685458 learned_addr learned my addr 172.21.15.45:0/1023685458 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0f7fe700 1 --2- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b1006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0ffff700 1 --2- 172.21.15.45:0/1023685458 >> v2:172.21.15.144:3300/0 conn(0x7f8b10159200 0x7f8b100c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0effd700 1 -- 172.21.15.45:0/1023685458 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076701046 0 0) 0x7f8b100c7d00 con 0x7f8b1006b5c0 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0effd700 1 -- 172.21.15.45:0/1023685458 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8afc003570 con 0x7f8b1006b5c0 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0ffff700 1 -- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 msgr2=0x7f8b1006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0ffff700 1 --2- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b1006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0ffff700 1 -- 172.21.15.45:0/1023685458 >> v1:172.21.15.45:6789/0 conn(0x7f8b1006b5c0 legacy=0x7f8b100ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0ffff700 1 -- 172.21.15.45:0/1023685458 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1006bb40 con 0x7f8b10159200 2023-10-06T05:23:43.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0f7fe700 1 --2- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b1006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0ffff700 1 --2- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b10159200 0x7f8b100c8190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b000098e0 tx=0x7f8b000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fefdc2478fcb276 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.547+0000 7f8b0effd700 1 -- 172.21.15.45:0/1023685458 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00011630 con 0x7f8b10159200 2023-10-06T05:23:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b0effd700 1 -- 172.21.15.45:0/1023685458 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00011790 con 0x7f8b10159200 2023-10-06T05:23:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b0effd700 1 -- 172.21.15.45:0/1023685458 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00011b30 con 0x7f8b10159200 2023-10-06T05:23:43.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 -- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b10159200 msgr2=0x7f8b100c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 --2- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b10159200 0x7f8b100c8190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b000098e0 tx=0x7f8b000091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 -- 172.21.15.45:0/1023685458 shutdown_connections 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 --2- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b10159200 0x7f8b100c8190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 --2- 172.21.15.45:0/1023685458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b1006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 -- 172.21.15.45:0/1023685458 >> 172.21.15.45:0/1023685458 conn(0x7f8b1006f090 msgr2=0x7f8b1006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 -- 172.21.15.45:0/1023685458 shutdown_connections 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.548+0000 7f8b16584700 1 -- 172.21.15.45:0/1023685458 wait complete. 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.549+0000 7f8b16584700 1 Processor -- start 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b16584700 1 -- start start 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b16584700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1006b5c0 0x7f8b10155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b16584700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b10159200 0x7f8b10157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b16584700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b100aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b16584700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10158450 con 0x7f8b10159200 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b16584700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100cae70 con 0x7f8b1006b5c0 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b16584700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b101561e0 con 0x7f8b101595d0 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.550+0000 7f8b0f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b10159200 0x7f8b10157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b10159200 0x7f8b10157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38200/0 (socket says 172.21.15.45:38200) 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 -- 172.21.15.45:0/14606349 learned_addr learned my addr 172.21.15.45:0/14606349 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b14b21700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b100aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0ffff700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1006b5c0 0x7f8b10155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 -- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 msgr2=0x7f8b100aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b100aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 -- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1006b5c0 msgr2=0x7f8b10155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1006b5c0 0x7f8b10155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 -- 172.21.15.45:0/14606349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af8009420 con 0x7f8b10159200 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b0f7fe700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b10159200 0x7f8b10157620 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8af800ecb0 tx=0x7f8af80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b07fff700 1 -- 172.21.15.45:0/14606349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800c7f0 con 0x7f8b10159200 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b07fff700 1 -- 172.21.15.45:0/14606349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af800c950 con 0x7f8b10159200 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b07fff700 1 -- 172.21.15.45:0/14606349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800ccd0 con 0x7f8b10159200 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b00009420 con 0x7f8b10159200 2023-10-06T05:23:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.551+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b10157e90 con 0x7f8b10159200 2023-10-06T05:23:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.552+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b100cb000 con 0x7f8b10159200 2023-10-06T05:23:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.553+0000 7f8b07fff700 1 -- 172.21.15.45:0/14606349 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8af8013680 con 0x7f8b10159200 2023-10-06T05:23:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.553+0000 7f8b07fff700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8af0066f30 0x7f8af00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.553+0000 7f8b07fff700 1 -- 172.21.15.45:0/14606349 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f8af8084e10 con 0x7f8b10159200 2023-10-06T05:23:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.553+0000 7f8b0ffff700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8af0066f30 0x7f8af00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.554+0000 7f8b0ffff700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8af0066f30 0x7f8af00693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8b000091e0 tx=0x7f8b00017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.558+0000 7f8b07fff700 1 -- 172.21.15.45:0/14606349 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8af8055410 con 0x7f8b10159200 2023-10-06T05:23:43.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:43.844+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f8b1006be20 con 0x7f8b10159200 2023-10-06T05:23:44.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.397+0000 7f8b07fff700 1 -- 172.21.15.45:0/14606349 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v171) v1 ==== 107+0+0 (secure 0 0 0) 0x7f8af8054fa0 con 0x7f8b10159200 2023-10-06T05:23:44.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8af0066f30 msgr2=0x7f8af00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8af0066f30 0x7f8af00693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8b000091e0 tx=0x7f8b00017000 comp rx=0 tx=0).stop 2023-10-06T05:23:44.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b10159200 msgr2=0x7f8b10157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b10159200 0x7f8b10157620 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8af800ecb0 tx=0x7f8af80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 shutdown_connections 2023-10-06T05:23:44.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8af0066f30 0x7f8af00693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b101595d0 0x7f8b100aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1006b5c0 0x7f8b10155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 --2- 172.21.15.45:0/14606349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b10159200 0x7f8b10157620 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 >> 172.21.15.45:0/14606349 conn(0x7f8b1006f090 msgr2=0x7f8b1015ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 shutdown_connections 2023-10-06T05:23:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.400+0000 7f8b16584700 1 -- 172.21.15.45:0/14606349 wait complete. 2023-10-06T05:23:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:23:44.417 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.507+0000 7fb57024e040 1 Processor -- start 2023-10-06T05:23:44.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.507+0000 7fb57024e040 1 -- start start 2023-10-06T05:23:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb57024e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d40316bca0 0x55d4032114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb57024e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 0x55d403213a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb57024e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d403212b30 0x55d403183670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb57024e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4030e6320 con 0x55d40316bca0 2023-10-06T05:23:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb57024e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40308a980 con 0x55d403212b30 2023-10-06T05:23:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb57024e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d40308a620 con 0x55d4031877e0 2023-10-06T05:23:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d40316bca0 0x55d4032114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d40316bca0 0x55d4032114e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38216/0 (socket says 172.21.15.45:38216) 2023-10-06T05:23:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 -- 172.21.15.45:0/2322113041 learned_addr learned my addr 172.21.15.45:0/2322113041 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 0x55d403213a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d403212b30 0x55d403183670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 -- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 msgr2=0x55d403213a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 0x55d403213a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 -- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d403212b30 msgr2=0x55d403183670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d403212b30 0x55d403183670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 -- 172.21.15.45:0/2322113041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56400a0e0 con 0x55d40316bca0 2023-10-06T05:23:44.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.508+0000 7fb56946c700 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d40316bca0 0x55d4032114e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00b320 tx=0x7fb55c0068f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f26de1b64e5aea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb5637fe700 1 -- 172.21.15.45:0/2322113041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00e470 con 0x55d40316bca0 2023-10-06T05:23:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb5637fe700 1 -- 172.21.15.45:0/2322113041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c00e5f0 con 0x55d40316bca0 2023-10-06T05:23:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb5637fe700 1 -- 172.21.15.45:0/2322113041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00e9b0 con 0x55d40316bca0 2023-10-06T05:23:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 -- 172.21.15.45:0/2322113041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d40316bca0 msgr2=0x55d4032114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d40316bca0 0x55d4032114e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00b320 tx=0x7fb55c0068f0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 -- 172.21.15.45:0/2322113041 shutdown_connections 2023-10-06T05:23:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d403212b30 0x55d403183670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d40316bca0 0x55d4032114e0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 --2- 172.21.15.45:0/2322113041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 0x55d403213a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 -- 172.21.15.45:0/2322113041 >> 172.21.15.45:0/2322113041 conn(0x55d403140450 msgr2=0x55d403142850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 -- 172.21.15.45:0/2322113041 shutdown_connections 2023-10-06T05:23:44.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.509+0000 7fb57024e040 1 -- 172.21.15.45:0/2322113041 wait complete. 2023-10-06T05:23:44.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.510+0000 7fb57024e040 1 Processor -- start 2023-10-06T05:23:44.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.510+0000 7fb57024e040 1 -- start start 2023-10-06T05:23:44.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb57024e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d40316bca0 0x55d40316e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb57024e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 0x55d403185670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb57024e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d403212b30 0x55d403185bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb57024e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d403081400 con 0x55d403212b30 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb57024e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40320edc0 con 0x55d40316bca0 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb57024e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d403184520 con 0x55d4031877e0 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d403212b30 0x55d403185bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d403212b30 0x55d403185bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38224/0 (socket says 172.21.15.45:38224) 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 learned_addr learned my addr 172.21.15.45:0/2607130034 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb56946c700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d40316bca0 0x55d40316e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 0x55d403185670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 msgr2=0x55d403185670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4031877e0 0x55d403185670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d40316bca0 msgr2=0x55d40316e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d40316bca0 0x55d40316e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.511+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c006bd0 con 0x55d403212b30 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.512+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d403212b30 0x55d403185bb0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x55d403143b30 tx=0x7fb564015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.512+0000 7fb561ffb700 1 -- 172.21.15.45:0/2607130034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400f860 con 0x55d403212b30 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.512+0000 7fb561ffb700 1 -- 172.21.15.45:0/2607130034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56400f9c0 con 0x55d403212b30 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.512+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d40308cae0 con 0x55d403212b30 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.512+0000 7fb561ffb700 1 -- 172.21.15.45:0/2607130034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564025690 con 0x55d403212b30 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.512+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d40308c540 con 0x55d403212b30 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.513+0000 7fb561ffb700 1 -- 172.21.15.45:0/2607130034 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb564003020 con 0x55d403212b30 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.514+0000 7fb561ffb700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c0658c0 0x7fb54c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.514+0000 7fb561ffb700 1 -- 172.21.15.45:0/2607130034 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fb5640859f0 con 0x55d403212b30 2023-10-06T05:23:44.523 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.514+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d40316e7b0 0x55d40320bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.514+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403215f20 con 0x55d40316e7b0 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.514+0000 7fb56946c700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c0658c0 0x7fb54c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.514+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d40316e7b0 0x55d40320bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.515+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d40316e7b0 0x55d40320bf70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.515+0000 7fb56946c700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c0658c0 0x7fb54c067d70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0063a0 tx=0x7fb55c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.516+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v105'381 uv355 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.516+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40320c4b0 con 0x55d40316e7b0 2023-10-06T05:23:44.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.518+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v105'382 uv382 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.518+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40320c860 con 0x55d40316e7b0 2023-10-06T05:23:44.525 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:44.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.519+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v105'383 uv383 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.519+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40320cc40 con 0x55d40316e7b0 2023-10-06T05:23:44.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.520+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.520+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40320cff0 con 0x55d40316e7b0 2023-10-06T05:23:44.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.522+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'385 uv385 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.522+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233790 con 0x55d40316e7b0 2023-10-06T05:23:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.523+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'386 uv386 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.523+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233b40 con 0x55d40316e7b0 2023-10-06T05:23:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.524+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v105'387 uv361 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.524+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233ef0 con 0x55d40316e7b0 2023-10-06T05:23:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.525+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v105'388 uv388 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.525+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233ef0 con 0x55d40316e7b0 2023-10-06T05:23:44.528 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:44.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.526+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v105'389 uv389 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.526+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403234f00 con 0x55d40316e7b0 2023-10-06T05:23:44.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.528+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'390 uv390 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.528+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403234f00 con 0x55d40316e7b0 2023-10-06T05:23:44.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.529+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'391 uv391 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.529+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233ef0 con 0x55d40316e7b0 2023-10-06T05:23:44.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.530+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'392 uv392 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.530+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233ef0 con 0x55d40316e7b0 2023-10-06T05:23:44.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.531+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v105'393 uv368 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.531+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233ef0 con 0x55d40316e7b0 2023-10-06T05:23:44.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.532+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'394 uv394 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.532+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233ef0 con 0x55d40316e7b0 2023-10-06T05:23:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.534+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v105'395 uv395 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.535 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:44.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.534+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403234f00 con 0x55d40316e7b0 2023-10-06T05:23:44.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.535+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'396 uv396 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.535+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403234f00 con 0x55d40316e7b0 2023-10-06T05:23:44.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.536+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'397 uv397 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.536+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403233ef0 con 0x55d40316e7b0 2023-10-06T05:23:44.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.537+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'398 uv398 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.537+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.538+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'399 uv399 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.538+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.539+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v105'400 uv374 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.540+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.541+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v105'401 uv401 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.541+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.542+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v105'402 uv402 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.544 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.542+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032353f0 con 0x55d40316e7b0 2023-10-06T05:23:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.543+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'403 uv403 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.543+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032353f0 con 0x55d40316e7b0 2023-10-06T05:23:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.544+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'404 uv404 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.544+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.545+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'405 uv405 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.545+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.547+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v105'406 uv380 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.547+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.548+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v105'407 uv407 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.548+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.551 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:44.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.549+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v105'408 uv408 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.549+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032353f0 con 0x55d40316e7b0 2023-10-06T05:23:44.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.550+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'409 uv409 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.550+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032353f0 con 0x55d40316e7b0 2023-10-06T05:23:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.551+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'410 uv410 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.551+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032343e0 con 0x55d40316e7b0 2023-10-06T05:23:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.553+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'411 uv411 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d40316e7b0 2023-10-06T05:23:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.553+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d4032348f0 0x55d403236cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.553+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403237210 con 0x55d4032348f0 2023-10-06T05:23:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.553+0000 7fb56946c700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d4032348f0 0x55d403236cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.553+0000 7fb56946c700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d4032348f0 0x55d403236cd0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.557+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'283 uv263 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.557+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032375c0 con 0x55d4032348f0 2023-10-06T05:23:44.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.558+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'284 uv284 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.559+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403237970 con 0x55d4032348f0 2023-10-06T05:23:44.561 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:44.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.560+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'285 uv285 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.560+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238210 con 0x55d4032348f0 2023-10-06T05:23:44.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.561+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v105'286 uv286 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.561+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032385c0 con 0x55d4032348f0 2023-10-06T05:23:44.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.562+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'287 uv287 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.563+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238970 con 0x55d4032348f0 2023-10-06T05:23:44.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.564+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v105'288 uv282 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.564+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.565+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v105'289 uv289 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.565+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.569 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.567+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v105'290 uv290 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.567+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.568+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.569+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403239120 con 0x55d4032348f0 2023-10-06T05:23:44.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.570+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.570+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403239120 con 0x55d4032348f0 2023-10-06T05:23:44.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.571+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v105'293 uv281 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.571+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.573+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v105'294 uv294 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.573+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.576 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.574+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v105'295 uv295 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.574+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.576+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v105'296 uv296 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.576+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c840 con 0x55d4032348f0 2023-10-06T05:23:44.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.577+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v105'297 uv297 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.577+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238da0 con 0x55d4032348f0 2023-10-06T05:23:44.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.578+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v105'298 uv279 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.578+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.579+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v105'299 uv299 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.579+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.581 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.580+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v105'300 uv300 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.580+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.581+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v105'301 uv301 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.581+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.582+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v105'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.583+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403238d20 con 0x55d4032348f0 2023-10-06T05:23:44.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.584+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v105'303 uv303 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.584+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d403239280 0x55d40323b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.584+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323bc80 con 0x55d403239280 2023-10-06T05:23:44.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.584+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d403239280 0x55d40323b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.584+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d403239280 0x55d40323b660 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.586+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.586+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323bd40 con 0x55d403239280 2023-10-06T05:23:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.587+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v105'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.587+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323bd40 con 0x55d403239280 2023-10-06T05:23:44.589 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:44.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.588+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v105'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.588+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323d260 con 0x55d403239280 2023-10-06T05:23:44.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.589+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.589+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323d260 con 0x55d403239280 2023-10-06T05:23:44.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.590+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.590+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d403239280 2023-10-06T05:23:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.591+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.591+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d403239280 2023-10-06T05:23:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.592+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'56 uv55 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.592+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d403239280 2023-10-06T05:23:44.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.594+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v105'57 uv57 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.594+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d403239280 2023-10-06T05:23:44.596 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.595+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v105'58 uv58 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.595+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d403239280 2023-10-06T05:23:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.596+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v105'59 uv59 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.596+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d4032348f0 2023-10-06T05:23:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.598+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.598+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d4032348f0 2023-10-06T05:23:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.599+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v105'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.599+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d4032348f0 2023-10-06T05:23:44.602 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.600+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v105'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.600+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323d260 con 0x55d4032348f0 2023-10-06T05:23:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.602+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.602+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323d260 con 0x55d4032348f0 2023-10-06T05:23:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.603+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.603+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323c250 con 0x55d4032348f0 2023-10-06T05:23:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.604+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d4032348f0 2023-10-06T05:23:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.604+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d40323c8d0 0x55d40323ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.604+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323f1f0 con 0x55d40323c8d0 2023-10-06T05:23:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.604+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d40323c8d0 0x55d40323ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.605+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d40323c8d0 0x55d40323ecb0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.606+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v105'138 uv132 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.606+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323f660 con 0x55d40323c8d0 2023-10-06T05:23:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.607+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v105'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.607+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40323fb50 con 0x55d40323c8d0 2023-10-06T05:23:44.610 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:44.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.608+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v105'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.608+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403241450 con 0x55d40323c8d0 2023-10-06T05:23:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.609+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v105'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.610+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403241800 con 0x55d40323c8d0 2023-10-06T05:23:44.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.611+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v105'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.611+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403240440 con 0x55d40323c8d0 2023-10-06T05:23:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.612+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v105'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.612+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403240840 con 0x55d40323c8d0 2023-10-06T05:23:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.613+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v105'144 uv137 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.613+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403240840 con 0x55d40323c8d0 2023-10-06T05:23:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.614+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v105'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.615+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403240840 con 0x55d40323c8d0 2023-10-06T05:23:44.617 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:44.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.616+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v105'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.616+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403240840 con 0x55d40323c8d0 2023-10-06T05:23:44.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.617+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v105'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.617+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403240840 con 0x55d40323c8d0 2023-10-06T05:23:44.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.619+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v105'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.619+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d403241bb0 0x55d403243f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.619+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403241bb0 2023-10-06T05:23:44.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.619+0000 7fb56946c700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d403241bb0 0x55d403243f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.619+0000 7fb56946c700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d403241bb0 0x55d403243f90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.621+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v105'61 uv60 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d403241bb0 2023-10-06T05:23:44.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.621+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403241bb0 2023-10-06T05:23:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.623+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v105'62 uv62 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d403241bb0 2023-10-06T05:23:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.623+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403241bb0 2023-10-06T05:23:44.628 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:44.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.626+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v105'63 uv63 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d403241bb0 2023-10-06T05:23:44.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.626+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403241bb0 2023-10-06T05:23:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.629+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v105'64 uv64 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d403241bb0 2023-10-06T05:23:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.629+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403241bb0 2023-10-06T05:23:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.631+0000 7fb56946c700 1 -- 172.21.15.45:0/2607130034 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v105'65 uv65 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55c000bc0 con 0x55d403241bb0 2023-10-06T05:23:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.631+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403239280 2023-10-06T05:23:44.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.633+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.633+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403239280 2023-10-06T05:23:44.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.634+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v105'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.634+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403239280 2023-10-06T05:23:44.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.635+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v105'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.637 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:44.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.635+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032454e0 con 0x55d403239280 2023-10-06T05:23:44.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.636+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.636+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032454e0 con 0x55d403239280 2023-10-06T05:23:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.638+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.638+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032444d0 con 0x55d403239280 2023-10-06T05:23:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.639+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.639+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d403244b50 0x55d403246f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.639+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247500 con 0x55d403244b50 2023-10-06T05:23:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.639+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d403244b50 0x55d403246f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.639+0000 7fb568c6b700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d403244b50 0x55d403246f30 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.641+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v105'122 uv116 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.641+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.642+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v105'123 uv123 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.642+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.644 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.643+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v105'124 uv124 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.643+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.644+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v105'125 uv125 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.644+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.645+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v105'126 uv126 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.645+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.646+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v105'127 uv121 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.646+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.647+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v105'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.647+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.651 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:44.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.648+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v105'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.649+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d4032478e0 con 0x55d403244b50 2023-10-06T05:23:44.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.650+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v105'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.650+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40324a650 con 0x55d403244b50 2023-10-06T05:23:44.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.651+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v105'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403244b50 2023-10-06T05:23:44.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.651+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.652+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v105'57 uv55 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.652+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.653+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v105'58 uv58 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.653+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.656 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:44.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.654+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v105'59 uv59 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.654+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.656+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v105'60 uv60 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.656+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d4032486b0 0x55d40324ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.656+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d4032486b0 2023-10-06T05:23:44.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.656+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d4032486b0 0x55d40324ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.657+0000 7fb569c6d700 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d4032486b0 0x55d40324ada0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.658+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d4032486b0 2023-10-06T05:23:44.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.658+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d4032486b0 2023-10-06T05:23:44.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.659+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v105'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d4032486b0 2023-10-06T05:23:44.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.659+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d4032486b0 2023-10-06T05:23:44.661 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:44.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.660+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v105'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d4032486b0 2023-10-06T05:23:44.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.660+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d4032486b0 2023-10-06T05:23:44.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.661+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d4032486b0 2023-10-06T05:23:44.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.661+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d4032486b0 2023-10-06T05:23:44.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.662+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d4032486b0 2023-10-06T05:23:44.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.662+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d4032486b0 2023-10-06T05:23:44.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.663+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d4032486b0 2023-10-06T05:23:44.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.663+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d40323c8d0 2023-10-06T05:23:44.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.664+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v105'64 uv63 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.664+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d40323c8d0 2023-10-06T05:23:44.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.666+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v105'65 uv65 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.666+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d40323c8d0 2023-10-06T05:23:44.668 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:44.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.667+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v105'66 uv66 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.667+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d40323c8d0 2023-10-06T05:23:44.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.668+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v105'67 uv67 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.668+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d40323c8d0 2023-10-06T05:23:44.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.669+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v105'68 uv68 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fb564016020 con 0x55d40323c8d0 2023-10-06T05:23:44.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.669+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.670+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v105'157 uv148 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.670+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.672+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v105'158 uv158 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.672+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.674 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:44.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.673+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v105'159 uv159 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.673+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247dd0 con 0x55d403239280 2023-10-06T05:23:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.674+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v105'160 uv160 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.674+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247da0 con 0x55d403239280 2023-10-06T05:23:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.675+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v105'161 uv161 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.676+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247da0 con 0x55d403239280 2023-10-06T05:23:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.677+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v105'162 uv155 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.677+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247da0 con 0x55d403239280 2023-10-06T05:23:44.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.678+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v105'163 uv163 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.678+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247da0 con 0x55d403239280 2023-10-06T05:23:44.681 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:44.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.679+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v105'164 uv164 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.679+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247da0 con 0x55d403239280 2023-10-06T05:23:44.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.680+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v105'165 uv165 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.680+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d403247da0 con 0x55d403239280 2023-10-06T05:23:44.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.682+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v105'166 uv166 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.682+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40324b2e0 con 0x55d403239280 2023-10-06T05:23:44.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.683+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v105'167 uv167 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.683+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) v8 -- 0x55d40324b2e0 con 0x55d403239280 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb568c6b700 1 -- 172.21.15.45:0/2607130034 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v105'168 uv168 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb55400e810 con 0x55d403239280 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d403239280 msgr2=0x55d40323b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d403239280 0x55d40323b660 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d40316e7b0 msgr2=0x55d40320bf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d40316e7b0 0x55d40320bf70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d403244b50 msgr2=0x55d403246f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d403244b50 0x55d403246f30 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d4032348f0 msgr2=0x55d403236cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d4032348f0 0x55d403236cd0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 reap_dead start 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d4032486b0 msgr2=0x55d40324ada0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d4032486b0 0x55d40324ada0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb569c6d700 1 -- 172.21.15.45:0/2607130034 reap_dead start 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d403241bb0 msgr2=0x55d403243f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d403241bb0 0x55d403243f90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d40323c8d0 msgr2=0x55d40323ecb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d40323c8d0 0x55d40323ecb0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c0658c0 msgr2=0x7fb54c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c0658c0 0x7fb54c067d70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0063a0 tx=0x7fb55c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d403212b30 msgr2=0x55d403185bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d403212b30 0x55d403185bb0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x55d403143b30 tx=0x7fb564015000 comp rx=0 tx=0).stop 2023-10-06T05:23:44.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 shutdown_connections 2023-10-06T05:23:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d40323c8d0 0x55d40323ecb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d403241bb0 0x55d403243f90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d403212b30 0x55d403185bb0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 --2- 172.21.15.45:0/2607130034 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb54c0658c0 0x7fb54c067d70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 >> 172.21.15.45:0/2607130034 conn(0x55d403140450 msgr2=0x55d40320d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.684+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 shutdown_connections 2023-10-06T05:23:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.685+0000 7fb57024e040 1 -- 172.21.15.45:0/2607130034 wait complete. 2023-10-06T05:23:44.690 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2023-10-06T05:23:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.778+0000 7f5360237040 1 Processor -- start 2023-10-06T05:23:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.778+0000 7f5360237040 1 -- start start 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5360237040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f7030a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5360237040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f6fa47e0 0x5601f6f5a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5360237040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f702fb30 0x5601f6f5a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5360237040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f6ea7980 con 0x5601f702fb30 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5360237040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f6ea7620 con 0x5601f6fa47e0 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5360237040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601f6e9e400 con 0x5601f6f88ca0 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5359c56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f702fb30 0x5601f6f5a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5359455700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f7030a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5359c56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f702fb30 0x5601f6f5a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38240/0 (socket says 172.21.15.45:38240) 2023-10-06T05:23:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5359c56700 1 -- 172.21.15.45:0/1472500071 learned_addr learned my addr 172.21.15.45:0/1472500071 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.779+0000 7f5358c54700 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f6fa47e0 0x5601f6f5a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f5359455700 1 -- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f6fa47e0 msgr2=0x5601f6f5a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f5359455700 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f6fa47e0 0x5601f6f5a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f5359455700 1 -- 172.21.15.45:0/1472500071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f702fb30 msgr2=0x5601f6f5a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f5359455700 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f702fb30 0x5601f6f5a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f5359455700 1 -- 172.21.15.45:0/1472500071 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344005cf0 con 0x5601f6f88ca0 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f5359455700 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f7030a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f534c006e80 tx=0x7f534c006930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa94702c64530588 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f53537fe700 1 -- 172.21.15.45:0/1472500071 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c002100 con 0x5601f6f88ca0 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f53537fe700 1 -- 172.21.15.45:0/1472500071 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c006110 con 0x5601f6f88ca0 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.780+0000 7f53537fe700 1 -- 172.21.15.45:0/1472500071 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c004520 con 0x5601f6f88ca0 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 -- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 msgr2=0x5601f7030a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f7030a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f534c006e80 tx=0x7f534c006930 comp rx=0 tx=0).stop 2023-10-06T05:23:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 -- 172.21.15.45:0/1472500071 shutdown_connections 2023-10-06T05:23:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f702fb30 0x5601f6f5a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f7030a60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 --2- 172.21.15.45:0/1472500071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f6fa47e0 0x5601f6f5a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 -- 172.21.15.45:0/1472500071 >> 172.21.15.45:0/1472500071 conn(0x5601f6f5d450 msgr2=0x5601f6f5f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 -- 172.21.15.45:0/1472500071 shutdown_connections 2023-10-06T05:23:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.781+0000 7f5360237040 1 -- 172.21.15.45:0/1472500071 wait complete. 2023-10-06T05:23:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.782+0000 7f5360237040 1 Processor -- start 2023-10-06T05:23:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.782+0000 7f5360237040 1 -- start start 2023-10-06T05:23:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5360237040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f6fa26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5360237040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f6fa47e0 0x5601f6fa1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5360237040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f702fb30 0x5601f6fa3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5360237040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f6fa3570 con 0x5601f6fa47e0 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5360237040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f6fa1a60 con 0x5601f702fb30 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5360237040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601f6fa1d00 con 0x5601f6f88ca0 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f6fa47e0 0x5601f6fa1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f6fa47e0 0x5601f6fa1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38254/0 (socket says 172.21.15.45:38254) 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 -- 172.21.15.45:0/528462380 learned_addr learned my addr 172.21.15.45:0/528462380 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5359455700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f6fa26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5359c56700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f702fb30 0x5601f6fa3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 -- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 msgr2=0x5601f6fa26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5601f6f88ca0 0x5601f6fa26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 -- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f702fb30 msgr2=0x5601f6fa3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601f702fb30 0x5601f6fa3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 -- 172.21.15.45:0/528462380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348001e10 con 0x5601f6fa47e0 2023-10-06T05:23:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.783+0000 7f5358c54700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f6fa47e0 0x5601f6fa1140 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5344008a40 tx=0x7f534400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.784+0000 7f5351ffb700 1 -- 172.21.15.45:0/528462380 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400f040 con 0x5601f6fa47e0 2023-10-06T05:23:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.784+0000 7f5351ffb700 1 -- 172.21.15.45:0/528462380 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53440094e0 con 0x5601f6fa47e0 2023-10-06T05:23:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.784+0000 7f5351ffb700 1 -- 172.21.15.45:0/528462380 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344010a80 con 0x5601f6fa47e0 2023-10-06T05:23:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.784+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5354005cf0 con 0x5601f6fa47e0 2023-10-06T05:23:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.784+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601f6eb63d0 con 0x5601f6fa47e0 2023-10-06T05:23:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.785+0000 7f5351ffb700 1 -- 172.21.15.45:0/528462380 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5344015450 con 0x5601f6fa47e0 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.786+0000 7f5351ffb700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f533c065890 0x7f533c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.786+0000 7f5351ffb700 1 -- 172.21.15.45:0/528462380 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f5344084330 con 0x5601f6fa47e0 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.786+0000 7f5359455700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f533c065890 0x7f533c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.786+0000 7f5360237040 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5601f7026f70 0x5601f702e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.786+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) v8 -- 0x5601f6f9c140 con 0x5601f7026f70 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.786+0000 7f5359c56700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5601f7026f70 0x5601f702e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.787+0000 7f5359455700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f533c065890 0x7f533c067d40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f534c004650 tx=0x7f534c010000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.787+0000 7f5359c56700 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5601f7026f70 0x5601f702e5e0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5359c56700 1 -- 172.21.15.45:0/528462380 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 100.00000000.inode [delete] v105'169 uv169 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5354012410 con 0x5601f7026f70 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5601f7026f70 msgr2=0x5601f702e5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5601f7026f70 0x5601f702e5e0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f533c065890 msgr2=0x7f533c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f533c065890 0x7f533c067d40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f534c004650 tx=0x7f534c010000 comp rx=0 tx=0).stop 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f6fa47e0 msgr2=0x5601f6fa1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 --2- 172.21.15.45:0/528462380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5601f6fa47e0 0x5601f6fa1140 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5344008a40 tx=0x7f534400c040 comp rx=0 tx=0).stop 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5359c56700 1 -- 172.21.15.45:0/528462380 reap_dead start 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 shutdown_connections 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 >> 172.21.15.45:0/528462380 conn(0x5601f6f5d450 msgr2=0x5601f6f8a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 shutdown_connections 2023-10-06T05:23:44.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.788+0000 7f5360237040 1 -- 172.21.15.45:0/528462380 wait complete. 2023-10-06T05:23:44.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:44.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.930+0000 7f82c419f700 1 Processor -- start 2023-10-06T05:23:44.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.930+0000 7f82c419f700 1 -- start start 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c419f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c419f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82bc0aaae0 0x7f82bc0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c419f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82bc0c3b50 con 0x7f82bc0c90b0 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c419f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82bc157700 con 0x7f82bc0aaae0 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c419f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0c98e0 con 0x7f82bc157330 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c173a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82bc0aaae0 0x7f82bc0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c173a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82bc0aaae0 0x7f82bc0c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38258/0 (socket says 172.21.15.45:38258) 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c173a700 1 -- 172.21.15.45:0/2846584443 learned_addr learned my addr 172.21.15.45:0/2846584443 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.931+0000 7f82c1f3b700 1 --2- 172.21.15.45:0/2846584443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c173a700 1 -- 172.21.15.45:0/2846584443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 msgr2=0x7f82bc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c173a700 1 --2- 172.21.15.45:0/2846584443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c173a700 1 -- 172.21.15.45:0/2846584443 >> v1:172.21.15.144:6789/0 conn(0x7f82bc0c90b0 legacy=0x7f82bc0c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c0f39700 1 -- 172.21.15.45:0/2846584443 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469122768 0 0) 0x7f82bc0c3b50 con 0x7f82bc0c90b0 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c173a700 1 -- 172.21.15.45:0/2846584443 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc0c9bc0 con 0x7f82bc0aaae0 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c173a700 1 --2- 172.21.15.45:0/2846584443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0c8b70 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0099b0 tx=0x7f82ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8620d6ad135aea22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c0f39700 1 -- 172.21.15.45:0/2846584443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac009ea0 con 0x7f82bc0aaae0 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c0f39700 1 -- 172.21.15.45:0/2846584443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac00b860 con 0x7f82bc0aaae0 2023-10-06T05:23:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.932+0000 7f82c0f39700 1 -- 172.21.15.45:0/2846584443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00bc00 con 0x7f82bc0aaae0 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 -- 172.21.15.45:0/2846584443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0aaae0 msgr2=0x7f82bc0c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 --2- 172.21.15.45:0/2846584443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0c8b70 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0099b0 tx=0x7f82ac009250 comp rx=0 tx=0).stop 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 -- 172.21.15.45:0/2846584443 shutdown_connections 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 --2- 172.21.15.45:0/2846584443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 --2- 172.21.15.45:0/2846584443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0c8b70 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 -- 172.21.15.45:0/2846584443 >> 172.21.15.45:0/2846584443 conn(0x7f82bc067e00 msgr2=0x7f82bc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 -- 172.21.15.45:0/2846584443 shutdown_connections 2023-10-06T05:23:44.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.933+0000 7f82c419f700 1 -- 172.21.15.45:0/2846584443 wait complete. 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.934+0000 7f82c419f700 1 Processor -- start 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.934+0000 7f82c419f700 1 -- start start 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c419f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c419f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0c90b0 0x7f82bc15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c419f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c419f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0693e0 con 0x7f82bc0c90b0 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c419f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0687d0 con 0x7f82bc0aaae0 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c419f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc068ab0 con 0x7f82bc157330 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0c90b0 0x7f82bc15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0c90b0 0x7f82bc15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38262/0 (socket says 172.21.15.45:38262) 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 -- 172.21.15.45:0/1352720751 learned_addr learned my addr 172.21.15.45:0/1352720751 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c1f3b700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c273c700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 -- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 msgr2=0x7f82bc0ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 -- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82bc0aaae0 msgr2=0x7f82bc0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c173a700 1 -- 172.21.15.45:0/1352720751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac0094f0 con 0x7f82bc0c90b0 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c1f3b700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.935+0000 7f82c273c700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.936+0000 7f82c173a700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0c90b0 0x7f82bc15c250 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f82ac006fd0 tx=0x7f82ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.936+0000 7f82b1ffb700 1 -- 172.21.15.45:0/1352720751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00b9d0 con 0x7f82bc0c90b0 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.936+0000 7f82b1ffb700 1 -- 172.21.15.45:0/1352720751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac00bb70 con 0x7f82bc0c90b0 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.936+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82bc068d90 con 0x7f82bc0c90b0 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.936+0000 7f82b1ffb700 1 -- 172.21.15.45:0/1352720751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac025670 con 0x7f82bc0c90b0 2023-10-06T05:23:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.936+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82bc069870 con 0x7f82bc0c90b0 2023-10-06T05:23:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.937+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82bc069550 con 0x7f82bc0c90b0 2023-10-06T05:23:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.937+0000 7f82b1ffb700 1 -- 172.21.15.45:0/1352720751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82ac023070 con 0x7f82bc0c90b0 2023-10-06T05:23:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.938+0000 7f82b1ffb700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82a0066f30 0x7f82a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.938+0000 7f82b1ffb700 1 -- 172.21.15.45:0/1352720751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f82ac086100 con 0x7f82bc0c90b0 2023-10-06T05:23:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.938+0000 7f82c1f3b700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82a0066f30 0x7f82a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.938+0000 7f82c1f3b700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82a0066f30 0x7f82a00693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f82b8006ea0 tx=0x7f82b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:44.942+0000 7f82b1ffb700 1 -- 172.21.15.45:0/1352720751 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82ac052d20 con 0x7f82bc0c90b0 2023-10-06T05:23:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.194+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f82bc06e350 con 0x7f82bc0c90b0 2023-10-06T05:23:45.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.400+0000 7f82b1ffb700 1 -- 172.21.15.45:0/1352720751 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v172) v1 ==== 162+0+0 (secure 0 0 0) 0x7f82ac02a0b0 con 0x7f82bc0c90b0 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82a0066f30 msgr2=0x7f82a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82a0066f30 0x7f82a00693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f82b8006ea0 tx=0x7f82b8008040 comp rx=0 tx=0).stop 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0c90b0 msgr2=0x7f82bc15c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0c90b0 0x7f82bc15c250 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f82ac006fd0 tx=0x7f82ac017000 comp rx=0 tx=0).stop 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 shutdown_connections 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f82a0066f30 0x7f82a00693e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82bc157330 0x7f82bc0ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82bc0aaae0 0x7f82bc0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 --2- 172.21.15.45:0/1352720751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc0c90b0 0x7f82bc15c250 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 >> 172.21.15.45:0/1352720751 conn(0x7f82bc067e00 msgr2=0x7f82bc06c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 shutdown_connections 2023-10-06T05:23:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.403+0000 7f82c419f700 1 -- 172.21.15.45:0/1352720751 wait complete. 2023-10-06T05:23:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:45.414 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:45.413+0000 7fec153ff700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-10-06T05:23:45.414 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:45.413+0000 7fec153ff700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:23:45.415 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:45.414+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:45.415 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:45.414+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:23:45.420 INFO:tasks.ceph.mds.c.smithi045.stderr: -23> 2023-10-06T05:23:45.413+0000 7fec153ff700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-10-06T05:23:45.420 INFO:tasks.ceph.mds.c.smithi045.stderr: -13> 2023-10-06T05:23:45.413+0000 7fec153ff700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:23:45.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:45.443 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:45.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.555+0000 7ff38e2e1700 1 Processor -- start 2023-10-06T05:23:45.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.555+0000 7ff38e2e1700 1 -- start start 2023-10-06T05:23:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff38e2e1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff3880c8ff0 0x7ff3880c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff38e2e1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3880caad0 0x7ff3880caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff38e2e1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38815b000 con 0x7ff388156af0 2023-10-06T05:23:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff38e2e1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff388157630 con 0x7ff3880caad0 2023-10-06T05:23:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff38e2e1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38815b2e0 con 0x7ff3880c8ff0 2023-10-06T05:23:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3880caad0 0x7ff3880caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3880caad0 0x7ff3880caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38288/0 (socket says 172.21.15.45:38288) 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 -- 172.21.15.45:0/980545506 learned_addr learned my addr 172.21.15.45:0/980545506 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff387fff700 1 --2- 172.21.15.45:0/980545506 >> v2:172.21.15.144:3300/0 conn(0x7ff3880c8ff0 0x7ff3880c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 -- 172.21.15.45:0/980545506 >> v2:172.21.15.144:3300/0 conn(0x7ff3880c8ff0 msgr2=0x7ff3880c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 --2- 172.21.15.45:0/980545506 >> v2:172.21.15.144:3300/0 conn(0x7ff3880c8ff0 0x7ff3880c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 -- 172.21.15.45:0/980545506 >> v1:172.21.15.144:6789/0 conn(0x7ff388156af0 legacy=0x7ff388156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 -- 172.21.15.45:0/980545506 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38815b5c0 con 0x7ff3880caad0 2023-10-06T05:23:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.556+0000 7ff3877fe700 1 --2- 172.21.15.45:0/980545506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3880caad0 0x7ff3880caf40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff3880cefc0 tx=0x7ff37c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f5e09e7306c4d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff386ffd700 1 -- 172.21.15.45:0/980545506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c009e70 con 0x7ff3880caad0 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff386ffd700 1 -- 172.21.15.45:0/980545506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c00b860 con 0x7ff3880caad0 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff386ffd700 1 -- 172.21.15.45:0/980545506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00bc00 con 0x7ff3880caad0 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 -- 172.21.15.45:0/980545506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3880caad0 msgr2=0x7ff3880caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/980545506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3880caad0 0x7ff3880caf40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff3880cefc0 tx=0x7ff37c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 -- 172.21.15.45:0/980545506 shutdown_connections 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/980545506 >> v2:172.21.15.144:3300/0 conn(0x7ff3880c8ff0 0x7ff3880c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/980545506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3880caad0 0x7ff3880caf40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 -- 172.21.15.45:0/980545506 >> 172.21.15.45:0/980545506 conn(0x7ff3880cdf50 msgr2=0x7ff3880ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 -- 172.21.15.45:0/980545506 shutdown_connections 2023-10-06T05:23:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.557+0000 7ff38e2e1700 1 -- 172.21.15.45:0/980545506 wait complete. 2023-10-06T05:23:45.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.558+0000 7ff38e2e1700 1 Processor -- start 2023-10-06T05:23:45.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38e2e1700 1 -- start start 2023-10-06T05:23:45.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38e2e1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3880c8ff0 0x7ff388145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38e2e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3880caad0 0x7ff388167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38e2e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff388156af0 0x7ff3880af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38e2e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff388168400 con 0x7ff388156af0 2023-10-06T05:23:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38e2e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38816b210 con 0x7ff3880caad0 2023-10-06T05:23:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38e2e1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3881595d0 con 0x7ff3880c8ff0 2023-10-06T05:23:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.559+0000 7ff38c87e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff388156af0 0x7ff3880af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff388156af0 0x7ff3880af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38304/0 (socket says 172.21.15.45:38304) 2023-10-06T05:23:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 -- 172.21.15.45:0/2598445070 learned_addr learned my addr 172.21.15.45:0/2598445070 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff387fff700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3880c8ff0 0x7ff388145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff3877fe700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3880caad0 0x7ff388167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 -- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3880c8ff0 msgr2=0x7ff388145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3880c8ff0 0x7ff388145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 -- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3880caad0 msgr2=0x7ff388167ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3880caad0 0x7ff388167ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 -- 172.21.15.45:0/2598445070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c0094f0 con 0x7ff388156af0 2023-10-06T05:23:45.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38c87e700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff388156af0 0x7ff3880af990 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c8e0 tx=0x7ff380001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff36ffff700 1 -- 172.21.15.45:0/2598445070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380012630 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff36ffff700 1 -- 172.21.15.45:0/2598445070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff380012790 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38816abc0 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff36ffff700 1 -- 172.21.15.45:0/2598445070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380012b50 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.560+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff388159d50 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.561+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38816aa20 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.562+0000 7ff36ffff700 1 -- 172.21.15.45:0/2598445070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff38001a3a0 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.562+0000 7ff36ffff700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.562+0000 7ff36ffff700 1 -- 172.21.15.45:0/2598445070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7ff380084df0 con 0x7ff388156af0 2023-10-06T05:23:45.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.562+0000 7ff387fff700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.563+0000 7ff387fff700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff3880b05b0 tx=0x7ff3780091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.567+0000 7ff36ffff700 1 -- 172.21.15.45:0/2598445070 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff38816aa20 con 0x7ff388156af0 2023-10-06T05:23:45.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.813+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff38816aa20 con 0x7ff388156af0 2023-10-06T05:23:45.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.813+0000 7ff36ffff700 1 -- 172.21.15.45:0/2598445070 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+3841 (secure 0 0 0) 0x7ff38816aa20 con 0x7ff388156af0 2023-10-06T05:23:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:45.815 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":174,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:45.414944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:45.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 msgr2=0x7ff3740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff3880b05b0 tx=0x7ff3780091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff388156af0 msgr2=0x7ff3880af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff388156af0 0x7ff3880af990 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c8e0 tx=0x7ff380001710 comp rx=0 tx=0).stop 2023-10-06T05:23:45.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 shutdown_connections 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff388156af0 0x7ff3880af990 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3880c8ff0 0x7ff388145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 --2- 172.21.15.45:0/2598445070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3880caad0 0x7ff388167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 >> 172.21.15.45:0/2598445070 conn(0x7ff3880cdf50 msgr2=0x7ff388151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 shutdown_connections 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:45.815+0000 7ff38e2e1700 1 -- 172.21.15.45:0/2598445070 wait complete. 2023-10-06T05:23:45.818 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 174 2023-10-06T05:23:45.830 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:45.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.003+0000 7fb70d090700 1 Processor -- start 2023-10-06T05:23:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.003+0000 7fb70d090700 1 -- start start 2023-10-06T05:23:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb70d090700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb708159200 0x7fb7080c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb70d090700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080c7d00 con 0x7fb7081595d0 2023-10-06T05:23:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb70d090700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080c86d0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb70d090700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080c7780 con 0x7fb708159200 2023-10-06T05:23:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb706d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb708159200 0x7fb7080c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb70659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb7081595d0 0x7fb70806b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57822/0 (socket says 172.21.15.45:57822) 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb70659c700 1 -- 172.21.15.45:0/4150451017 learned_addr learned my addr 172.21.15.45:0/4150451017 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb706d9d700 1 -- 172.21.15.45:0/4150451017 >> v1:172.21.15.144:6789/0 conn(0x7fb7080ccdc0 legacy=0x7fb7080cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb706d9d700 1 -- 172.21.15.45:0/4150451017 >> v1:172.21.15.45:6789/0 conn(0x7fb7081595d0 legacy=0x7fb70806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb706d9d700 1 -- 172.21.15.45:0/4150451017 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70806bb40 con 0x7fb708159200 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb705d9b700 1 -- 172.21.15.45:0/4150451017 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235011417 0 0) 0x7fb7080c7d00 con 0x7fb7081595d0 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.004+0000 7fb706d9d700 1 --2- 172.21.15.45:0/4150451017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708159200 0x7fb7080c8190 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80098e0 tx=0x7fb6f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f88fe3637b0c7d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb705d9b700 1 -- 172.21.15.45:0/4150451017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8009e80 con 0x7fb708159200 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb705d9b700 1 -- 172.21.15.45:0/4150451017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f800b860 con 0x7fb708159200 2023-10-06T05:23:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb705d9b700 1 -- 172.21.15.45:0/4150451017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f800bbe0 con 0x7fb708159200 2023-10-06T05:23:46.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb70d090700 1 -- 172.21.15.45:0/4150451017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708159200 msgr2=0x7fb7080c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb70d090700 1 --2- 172.21.15.45:0/4150451017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708159200 0x7fb7080c8190 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80098e0 tx=0x7fb6f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb70d090700 1 -- 172.21.15.45:0/4150451017 shutdown_connections 2023-10-06T05:23:46.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb70d090700 1 --2- 172.21.15.45:0/4150451017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708159200 0x7fb7080c8190 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb70d090700 1 -- 172.21.15.45:0/4150451017 >> 172.21.15.45:0/4150451017 conn(0x7fb70806f090 msgr2=0x7fb70806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb70d090700 1 -- 172.21.15.45:0/4150451017 shutdown_connections 2023-10-06T05:23:46.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.005+0000 7fb70d090700 1 -- 172.21.15.45:0/4150451017 wait complete. 2023-10-06T05:23:46.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 Processor -- start 2023-10-06T05:23:46.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 -- start start 2023-10-06T05:23:46.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7080ccdc0 0x7fb7080c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708159200 0x7fb70816ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7081595d0 0x7fb70816d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080b0fb0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080c6650 con 0x7fb708159200 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.007+0000 7fb70d090700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080c5a40 con 0x7fb7081595d0 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7080ccdc0 0x7fb7080c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7080ccdc0 0x7fb7080c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38308/0 (socket says 172.21.15.45:38308) 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 -- 172.21.15.45:0/3577173205 learned_addr learned my addr 172.21.15.45:0/3577173205 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb70659c700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708159200 0x7fb70816ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb70759e700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7081595d0 0x7fb70816d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 -- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7081595d0 msgr2=0x7fb70816d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7081595d0 0x7fb70816d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 -- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708159200 msgr2=0x7fb70816ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708159200 0x7fb70816ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 -- 172.21.15.45:0/3577173205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f8009420 con 0x7fb7080ccdc0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb70659c700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708159200 0x7fb70816ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb70759e700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7081595d0 0x7fb70816d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb706d9d700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7080ccdc0 0x7fb7080c4fc0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb70806aed0 tx=0x7fb6f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb6eeffd700 1 -- 172.21.15.45:0/3577173205 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801f070 con 0x7fb7080ccdc0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb6eeffd700 1 -- 172.21.15.45:0/3577173205 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f8011420 con 0x7fb7080ccdc0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7080c5d20 con 0x7fb7080ccdc0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb6eeffd700 1 -- 172.21.15.45:0/3577173205 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f800bbe0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.008+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7080c62e0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.009+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70815c7d0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.010+0000 7fb6eeffd700 1 -- 172.21.15.45:0/3577173205 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6f8011a50 con 0x7fb7080ccdc0 2023-10-06T05:23:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.010+0000 7fb6eeffd700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6f4066f30 0x7fb6f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.011+0000 7fb6eeffd700 1 -- 172.21.15.45:0/3577173205 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fb6f80862b0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.011+0000 7fb70659c700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6f4066f30 0x7fb6f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.011+0000 7fb70659c700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6f4066f30 0x7fb6f40693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb7080cba20 tx=0x7fb6fc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.015+0000 7fb6eeffd700 1 -- 172.21.15.45:0/3577173205 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6f8052dd0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.266+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7080c6930 con 0x7fb7080ccdc0 2023-10-06T05:23:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.267+0000 7fb6eeffd700 1 -- 172.21.15.45:0/3577173205 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fb6f80563f0 con 0x7fb7080ccdc0 2023-10-06T05:23:46.269 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:46.269 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":174,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:45.414944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:46.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6f4066f30 msgr2=0x7fb6f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6f4066f30 0x7fb6f40693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb7080cba20 tx=0x7fb6fc009040 comp rx=0 tx=0).stop 2023-10-06T05:23:46.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7080ccdc0 msgr2=0x7fb7080c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7080ccdc0 0x7fb7080c4fc0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb70806aed0 tx=0x7fb6f8017000 comp rx=0 tx=0).stop 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 shutdown_connections 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6f4066f30 0x7fb6f40693e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7081595d0 0x7fb70816d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7080ccdc0 0x7fb7080c4fc0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 --2- 172.21.15.45:0/3577173205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708159200 0x7fb70816ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 >> 172.21.15.45:0/3577173205 conn(0x7fb70806f090 msgr2=0x7fb708070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 shutdown_connections 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.269+0000 7fb70d090700 1 -- 172.21.15.45:0/3577173205 wait complete. 2023-10-06T05:23:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 174 2023-10-06T05:23:46.284 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:46.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:46.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.416+0000 7ff245619700 1 Processor -- start 2023-10-06T05:23:46.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.416+0000 7ff245619700 1 -- start start 2023-10-06T05:23:46.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff245619700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2400c8ff0 0x7ff2400c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff245619700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff2400caad0 0x7ff2400caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff245619700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff240156af0 0x7ff240156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff245619700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24015b000 con 0x7ff2400c8ff0 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff245619700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24015b2e0 con 0x7ff2400caad0 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff245619700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24015b5c0 con 0x7ff240156af0 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2400c8ff0 0x7ff2400c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2400c8ff0 0x7ff2400c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52206/0 (socket says 172.21.15.45:52206) 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 -- 172.21.15.45:0/1177493770 learned_addr learned my addr 172.21.15.45:0/1177493770 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23f7fe700 1 --2- 172.21.15.45:0/1177493770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff240156af0 0x7ff240156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23e7fc700 1 --2- 172.21.15.45:0/1177493770 >> v2:172.21.15.144:3300/0 conn(0x7ff2400caad0 0x7ff2400caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 -- 172.21.15.45:0/1177493770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff240156af0 msgr2=0x7ff240156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 --2- 172.21.15.45:0/1177493770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff240156af0 0x7ff240156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 -- 172.21.15.45:0/1177493770 >> v2:172.21.15.144:3300/0 conn(0x7ff2400caad0 msgr2=0x7ff2400caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 --2- 172.21.15.45:0/1177493770 >> v2:172.21.15.144:3300/0 conn(0x7ff2400caad0 0x7ff2400caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 -- 172.21.15.45:0/1177493770 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24015b8a0 con 0x7ff2400c8ff0 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23e7fc700 1 --2- 172.21.15.45:0/1177493770 >> v2:172.21.15.144:3300/0 conn(0x7ff2400caad0 0x7ff2400caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23f7fe700 1 --2- 172.21.15.45:0/1177493770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff240156af0 0x7ff240156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.418+0000 7ff23effd700 1 --2- 172.21.15.45:0/1177493770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2400c8ff0 0x7ff2400c93c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff24006ed80 tx=0x7ff230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4fdc422acd7d8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.419+0000 7ff23dffb700 1 -- 172.21.15.45:0/1177493770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000b860 con 0x7ff2400c8ff0 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.419+0000 7ff23dffb700 1 -- 172.21.15.45:0/1177493770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000b9c0 con 0x7ff2400c8ff0 2023-10-06T05:23:46.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.419+0000 7ff23dffb700 1 -- 172.21.15.45:0/1177493770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000bd60 con 0x7ff2400c8ff0 2023-10-06T05:23:46.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 -- 172.21.15.45:0/1177493770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2400c8ff0 msgr2=0x7ff2400c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 --2- 172.21.15.45:0/1177493770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2400c8ff0 0x7ff2400c93c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff24006ed80 tx=0x7ff230009250 comp rx=0 tx=0).stop 2023-10-06T05:23:46.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 -- 172.21.15.45:0/1177493770 shutdown_connections 2023-10-06T05:23:46.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 --2- 172.21.15.45:0/1177493770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff240156af0 0x7ff240156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 --2- 172.21.15.45:0/1177493770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2400c8ff0 0x7ff2400c93c0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 --2- 172.21.15.45:0/1177493770 >> v2:172.21.15.144:3300/0 conn(0x7ff2400caad0 0x7ff2400caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 -- 172.21.15.45:0/1177493770 >> 172.21.15.45:0/1177493770 conn(0x7ff2400cdf50 msgr2=0x7ff2400ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 -- 172.21.15.45:0/1177493770 shutdown_connections 2023-10-06T05:23:46.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.420+0000 7ff245619700 1 -- 172.21.15.45:0/1177493770 wait complete. 2023-10-06T05:23:46.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.421+0000 7ff245619700 1 Processor -- start 2023-10-06T05:23:46.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.421+0000 7ff245619700 1 -- start start 2023-10-06T05:23:46.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff245619700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2400c8ff0 0x7ff240159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff245619700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2400caad0 0x7ff24015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff245619700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240156af0 0x7ff2400ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff245619700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2401597b0 con 0x7ff240156af0 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff245619700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240169d00 con 0x7ff2400c8ff0 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff245619700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff240169fe0 con 0x7ff2400caad0 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff23f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240156af0 0x7ff2400ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff23f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240156af0 0x7ff2400ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52214/0 (socket says 172.21.15.45:52214) 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff23f7fe700 1 -- 172.21.15.45:0/1996864341 learned_addr learned my addr 172.21.15.45:0/1996864341 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff23effd700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2400c8ff0 0x7ff240159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff23e7fc700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2400caad0 0x7ff24015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff23f7fe700 1 -- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2400caad0 msgr2=0x7ff24015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.422+0000 7ff23f7fe700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2400caad0 0x7ff24015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff23f7fe700 1 -- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2400c8ff0 msgr2=0x7ff240159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff23f7fe700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2400c8ff0 0x7ff240159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff23f7fe700 1 -- 172.21.15.45:0/1996864341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300094f0 con 0x7ff240156af0 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff23f7fe700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240156af0 0x7ff2400ad200 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff23800cef0 tx=0x7ff238007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff226ffd700 1 -- 172.21.15.45:0/1996864341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23800ad20 con 0x7ff240156af0 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff226ffd700 1 -- 172.21.15.45:0/1996864341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff238016420 con 0x7ff240156af0 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff226ffd700 1 -- 172.21.15.45:0/1996864341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23800aea0 con 0x7ff240156af0 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24016a2c0 con 0x7ff240156af0 2023-10-06T05:23:46.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.423+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24015a550 con 0x7ff240156af0 2023-10-06T05:23:46.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.424+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2400c9ab0 con 0x7ff240156af0 2023-10-06T05:23:46.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.425+0000 7ff226ffd700 1 -- 172.21.15.45:0/1996864341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff238016a70 con 0x7ff240156af0 2023-10-06T05:23:46.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.425+0000 7ff226ffd700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff22c066f30 0x7ff22c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.425+0000 7ff226ffd700 1 -- 172.21.15.45:0/1996864341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7ff238012070 con 0x7ff240156af0 2023-10-06T05:23:46.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.425+0000 7ff23effd700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff22c066f30 0x7ff22c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.426+0000 7ff23effd700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff22c066f30 0x7ff22c0693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff24006ed80 tx=0x7ff230017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.430+0000 7ff226ffd700 1 -- 172.21.15.45:0/1996864341 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff23804e5a0 con 0x7ff240156af0 2023-10-06T05:23:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.678+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2400b1d50 con 0x7ff240156af0 2023-10-06T05:23:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.678+0000 7ff226ffd700 1 -- 172.21.15.45:0/1996864341 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4599 (secure 0 0 0) 0x7ff238051fc0 con 0x7ff240156af0 2023-10-06T05:23:46.680 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:46.680 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":175,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2681459400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2681459400},{"type":"v1","addr":"172.21.15.45:6835","nonce":2681459400}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:45.414944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff22c066f30 msgr2=0x7ff22c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff22c066f30 0x7ff22c0693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff24006ed80 tx=0x7ff230017000 comp rx=0 tx=0).stop 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240156af0 msgr2=0x7ff2400ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240156af0 0x7ff2400ad200 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff23800cef0 tx=0x7ff238007860 comp rx=0 tx=0).stop 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 shutdown_connections 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff22c066f30 0x7ff22c0693e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff240156af0 0x7ff2400ad200 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2400c8ff0 0x7ff240159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 --2- 172.21.15.45:0/1996864341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2400caad0 0x7ff24015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 >> 172.21.15.45:0/1996864341 conn(0x7ff2400cdf50 msgr2=0x7ff240151960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 shutdown_connections 2023-10-06T05:23:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.681+0000 7ff245619700 1 -- 172.21.15.45:0/1996864341 wait complete. 2023-10-06T05:23:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 175 2023-10-06T05:23:46.696 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2023-10-06T05:23:46.696 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2023-10-06T05:23:46.696 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:46.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:46.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:46.696 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:46.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:23:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:23:46.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:46.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.892+0000 7fce527f2700 1 Processor -- start 2023-10-06T05:23:46.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.892+0000 7fce527f2700 1 -- start start 2023-10-06T05:23:46.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce527f2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce527f2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce4c0ccdc0 0x7fce4c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce527f2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0cd740 con 0x7fce4c155cd0 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce527f2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0ae6b0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce527f2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0ae8d0 con 0x7fce4c1560a0 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce50d8f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce4c0ccdc0 0x7fce4c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce50d8f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce4c0ccdc0 0x7fce4c0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52220/0 (socket says 172.21.15.45:52220) 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce50d8f700 1 -- 172.21.15.45:0/410566772 learned_addr learned my addr 172.21.15.45:0/410566772 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.893+0000 7fce4b7fe700 1 --2- 172.21.15.45:0/410566772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce50d8f700 1 -- 172.21.15.45:0/410566772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 msgr2=0x7fce4c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce50d8f700 1 --2- 172.21.15.45:0/410566772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce50d8f700 1 -- 172.21.15.45:0/410566772 >> v1:172.21.15.144:6789/0 conn(0x7fce4c155cd0 legacy=0x7fce4c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce50d8f700 1 -- 172.21.15.45:0/410566772 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0aebb0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce50d8f700 1 --2- 172.21.15.45:0/410566772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c0cd200 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fce44009f20 tx=0x7fce44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4310bee3f487254c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce4affd700 1 -- 172.21.15.45:0/410566772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400e070 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce4affd700 1 -- 172.21.15.45:0/410566772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4400b9d0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce4affd700 1 -- 172.21.15.45:0/410566772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440092e0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce527f2700 1 -- 172.21.15.45:0/410566772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 msgr2=0x7fce4c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.894+0000 7fce527f2700 1 --2- 172.21.15.45:0/410566772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c0cd200 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fce44009f20 tx=0x7fce44009840 comp rx=0 tx=0).stop 2023-10-06T05:23:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.895+0000 7fce527f2700 1 -- 172.21.15.45:0/410566772 shutdown_connections 2023-10-06T05:23:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.895+0000 7fce527f2700 1 --2- 172.21.15.45:0/410566772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c0cd200 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.895+0000 7fce527f2700 1 --2- 172.21.15.45:0/410566772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.895+0000 7fce527f2700 1 -- 172.21.15.45:0/410566772 >> 172.21.15.45:0/410566772 conn(0x7fce4c06f090 msgr2=0x7fce4c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.895+0000 7fce527f2700 1 -- 172.21.15.45:0/410566772 shutdown_connections 2023-10-06T05:23:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.895+0000 7fce527f2700 1 -- 172.21.15.45:0/410566772 wait complete. 2023-10-06T05:23:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.896+0000 7fce527f2700 1 Processor -- start 2023-10-06T05:23:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.896+0000 7fce527f2700 1 -- start start 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.897+0000 7fce527f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.897+0000 7fce527f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce4c155cd0 0x7fce4c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.897+0000 7fce527f2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.897+0000 7fce527f2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c1555c0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.897+0000 7fce527f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c155080 con 0x7fce4c155cd0 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.897+0000 7fce527f2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c1673f0 con 0x7fce4c1560a0 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52222/0 (socket says 172.21.15.45:52222) 2023-10-06T05:23:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 -- 172.21.15.45:0/4159445473 learned_addr learned my addr 172.21.15.45:0/4159445473 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4b7fe700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce4c155cd0 0x7fce4c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce50d8f700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 -- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 msgr2=0x7fce4c1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c1589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 -- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce4c155cd0 msgr2=0x7fce4c158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce4c155cd0 0x7fce4c158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 -- 172.21.15.45:0/4159445473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44009ae0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4b7fe700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce4c155cd0 0x7fce4c158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce50d8f700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce4bfff700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c154b40 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c590 tx=0x7fce3c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce43fff700 1 -- 172.21.15.45:0/4159445473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0106b0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce43fff700 1 -- 172.21.15.45:0/4159445473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c010810 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c1676d0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce43fff700 1 -- 172.21.15.45:0/4159445473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0109c0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.898+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c0c41c0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.899+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c06e2f0 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.900+0000 7fce43fff700 1 -- 172.21.15.45:0/4159445473 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce3c011470 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.900+0000 7fce43fff700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.900+0000 7fce43fff700 1 -- 172.21.15.45:0/4159445473 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fce3c015020 con 0x7fce4c0ccdc0 2023-10-06T05:23:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.901+0000 7fce4b7fe700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.901+0000 7fce4b7fe700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fce34009ce0 tx=0x7fce34008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:46.905+0000 7fce43fff700 1 -- 172.21.15.45:0/4159445473 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce3c0533d0 con 0x7fce4c0ccdc0 2023-10-06T05:23:47.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.151+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fce4c0aee90 con 0x7fce4c0ccdc0 2023-10-06T05:23:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.426+0000 7fce43fff700 1 -- 172.21.15.45:0/4159445473 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v176) v1 ==== 161+0+0 (secure 0 0 0) 0x7fce3c0569f0 con 0x7fce4c0ccdc0 2023-10-06T05:23:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 msgr2=0x7fce2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fce34009ce0 tx=0x7fce34008040 comp rx=0 tx=0).stop 2023-10-06T05:23:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 msgr2=0x7fce4c154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c154b40 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c590 tx=0x7fce3c00c950 comp rx=0 tx=0).stop 2023-10-06T05:23:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 shutdown_connections 2023-10-06T05:23:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce4c1560a0 0x7fce4c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce4c0ccdc0 0x7fce4c154b40 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 --2- 172.21.15.45:0/4159445473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce4c155cd0 0x7fce4c158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 >> 172.21.15.45:0/4159445473 conn(0x7fce4c06f090 msgr2=0x7fce4c0c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 shutdown_connections 2023-10-06T05:23:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.429+0000 7fce527f2700 1 -- 172.21.15.45:0/4159445473 wait complete. 2023-10-06T05:23:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:47.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.582+0000 7f3013af3700 1 Processor -- start 2023-10-06T05:23:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.582+0000 7f3013af3700 1 -- start start 2023-10-06T05:23:47.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f3013af3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f300c153650 0x7f300c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f3013af3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f300c153a20 0x7f300c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f3013af3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c0676c0 con 0x7f300c06add0 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f3013af3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c06b270 con 0x7f300c153a20 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f3013af3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c0679a0 con 0x7f300c153650 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301108e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f300c153a20 0x7f300c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f3012090700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f300c06add0 0x7f300c068480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48698/0 (socket says 172.21.15.45:48698) 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f3012090700 1 -- 172.21.15.45:0/1420019722 learned_addr learned my addr 172.21.15.45:0/1420019722 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301188f700 1 --2- 172.21.15.45:0/1420019722 >> v2:172.21.15.144:3300/0 conn(0x7f300c153650 0x7f300c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301108e700 1 -- 172.21.15.45:0/1420019722 >> v2:172.21.15.144:3300/0 conn(0x7f300c153650 msgr2=0x7f300c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301108e700 1 --2- 172.21.15.45:0/1420019722 >> v2:172.21.15.144:3300/0 conn(0x7f300c153650 0x7f300c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301108e700 1 -- 172.21.15.45:0/1420019722 >> v1:172.21.15.45:6789/0 conn(0x7f300c06add0 legacy=0x7f300c068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301108e700 1 -- 172.21.15.45:0/1420019722 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c067c80 con 0x7f300c153a20 2023-10-06T05:23:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301088d700 1 -- 172.21.15.45:0/1420019722 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790078924 0 0) 0x7f300c0676c0 con 0x7f300c06add0 2023-10-06T05:23:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.583+0000 7f301108e700 1 --2- 172.21.15.45:0/1420019722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c153a20 0x7f300c06a890 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f300c06f7f0 tx=0x7f2ffc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23040a45306df4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f301088d700 1 -- 172.21.15.45:0/1420019722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00b6b0 con 0x7f300c153a20 2023-10-06T05:23:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f301088d700 1 -- 172.21.15.45:0/1420019722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc00b810 con 0x7f300c153a20 2023-10-06T05:23:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f301088d700 1 -- 172.21.15.45:0/1420019722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00bbb0 con 0x7f300c153a20 2023-10-06T05:23:47.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 -- 172.21.15.45:0/1420019722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c153a20 msgr2=0x7f300c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 --2- 172.21.15.45:0/1420019722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c153a20 0x7f300c06a890 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f300c06f7f0 tx=0x7f2ffc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 -- 172.21.15.45:0/1420019722 shutdown_connections 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 --2- 172.21.15.45:0/1420019722 >> v2:172.21.15.144:3300/0 conn(0x7f300c153650 0x7f300c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 --2- 172.21.15.45:0/1420019722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c153a20 0x7f300c06a890 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 -- 172.21.15.45:0/1420019722 >> 172.21.15.45:0/1420019722 conn(0x7f300c0cc5a0 msgr2=0x7f300c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 -- 172.21.15.45:0/1420019722 shutdown_connections 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.584+0000 7f3013af3700 1 -- 172.21.15.45:0/1420019722 wait complete. 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 Processor -- start 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 -- start start 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c06add0 0x7f300c1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c153650 0x7f300c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c153a20 0x7f300c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c154220 con 0x7f300c06add0 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c155ed0 con 0x7f300c153650 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f3013af3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c1561b0 con 0x7f300c153a20 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.586+0000 7f301188f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c06add0 0x7f300c1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c06add0 0x7f300c1546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52248/0 (socket says 172.21.15.45:52248) 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 -- 172.21.15.45:0/3885884962 learned_addr learned my addr 172.21.15.45:0/3885884962 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301108e700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c153650 0x7f300c151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f3012090700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c153a20 0x7f300c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 -- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c153a20 msgr2=0x7f300c156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c153a20 0x7f300c156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 -- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c153650 msgr2=0x7f300c151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c153650 0x7f300c151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 -- 172.21.15.45:0/3885884962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc009420 con 0x7f300c06add0 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301108e700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c153650 0x7f300c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f301188f700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c06add0 0x7f300c1546b0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f300800d890 tx=0x7f3008008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f3001ffb700 1 -- 172.21.15.45:0/3885884962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300800b600 con 0x7f300c06add0 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f3001ffb700 1 -- 172.21.15.45:0/3885884962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300800b760 con 0x7f300c06add0 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c153eb0 con 0x7f300c06add0 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.587+0000 7f3001ffb700 1 -- 172.21.15.45:0/3885884962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300800bae0 con 0x7f300c06add0 2023-10-06T05:23:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.588+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c151d80 con 0x7f300c06add0 2023-10-06T05:23:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.589+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c154500 con 0x7f300c06add0 2023-10-06T05:23:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.589+0000 7f3001ffb700 1 -- 172.21.15.45:0/3885884962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f300801e460 con 0x7f300c06add0 2023-10-06T05:23:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.589+0000 7f3001ffb700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ff0066f30 0x7f2ff00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.589+0000 7f3001ffb700 1 -- 172.21.15.45:0/3885884962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f3008085330 con 0x7f300c06add0 2023-10-06T05:23:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.589+0000 7f301108e700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ff0066f30 0x7f2ff00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.590+0000 7f301108e700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ff0066f30 0x7f2ff00693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f300c157350 tx=0x7f2ffc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.594+0000 7f3001ffb700 1 -- 172.21.15.45:0/3885884962 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3008051e50 con 0x7f300c06add0 2023-10-06T05:23:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:47.883+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f300c0cc0e0 con 0x7f300c06add0 2023-10-06T05:23:48.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.428+0000 7f3001ffb700 1 -- 172.21.15.45:0/3885884962 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v177) v1 ==== 101+0+0 (secure 0 0 0) 0x7f3008055470 con 0x7f300c06add0 2023-10-06T05:23:48.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ff0066f30 msgr2=0x7f2ff00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ff0066f30 0x7f2ff00693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f300c157350 tx=0x7f2ffc017000 comp rx=0 tx=0).stop 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c06add0 msgr2=0x7f300c1546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c06add0 0x7f300c1546b0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f300800d890 tx=0x7f3008008040 comp rx=0 tx=0).stop 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 shutdown_connections 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ff0066f30 0x7f2ff00693e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c153a20 0x7f300c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c06add0 0x7f300c1546b0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 --2- 172.21.15.45:0/3885884962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c153650 0x7f300c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.431+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 >> 172.21.15.45:0/3885884962 conn(0x7f300c0cc5a0 msgr2=0x7f300c0cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.432+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 shutdown_connections 2023-10-06T05:23:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.432+0000 7f3013af3700 1 -- 172.21.15.45:0/3885884962 wait complete. 2023-10-06T05:23:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:23:48.449 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.494+0000 7f5ff3e30040 1 Processor -- start 2023-10-06T05:23:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.494+0000 7f5ff3e30040 1 -- start start 2023-10-06T05:23:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5ff3e30040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5ff3e30040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0890 0x55dfcbc25cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5ff3e30040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0c60 0x55dfcbc26200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5ff3e30040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcbb2b980 con 0x55dfcbbde210 2023-10-06T05:23:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5ff3e30040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcbb2b620 con 0x55dfcbca0c60 2023-10-06T05:23:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5ff3e30040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfcbb22400 con 0x55dfcbca0890 2023-10-06T05:23:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52276/0 (socket says 172.21.15.45:52276) 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 -- 172.21.15.45:0/3105760190 learned_addr learned my addr 172.21.15.45:0/3105760190 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 -- 172.21.15.45:0/3105760190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0890 msgr2=0x55dfcbc25cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 --2- 172.21.15.45:0/3105760190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0890 0x55dfcbc25cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 -- 172.21.15.45:0/3105760190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0c60 msgr2=0x55dfcbc26200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 --2- 172.21.15.45:0/3105760190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0c60 0x55dfcbc26200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 -- 172.21.15.45:0/3105760190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0005cf0 con 0x55dfcbbde210 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.495+0000 7f5fed04e700 1 --2- 172.21.15.45:0/3105760190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9efc0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400a780 tx=0x7f5fe4005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba386d24b431ec3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5fdf7fe700 1 -- 172.21.15.45:0/3105760190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400f070 con 0x55dfcbbde210 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5fdf7fe700 1 -- 172.21.15.45:0/3105760190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe40086c0 con 0x55dfcbbde210 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5fdf7fe700 1 -- 172.21.15.45:0/3105760190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4008a60 con 0x55dfcbbde210 2023-10-06T05:23:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 -- 172.21.15.45:0/3105760190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 msgr2=0x55dfcbc9efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/3105760190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9efc0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400a780 tx=0x7f5fe4005c80 comp rx=0 tx=0).stop 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 -- 172.21.15.45:0/3105760190 shutdown_connections 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/3105760190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0c60 0x55dfcbc26200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/3105760190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9efc0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/3105760190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0890 0x55dfcbc25cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 -- 172.21.15.45:0/3105760190 >> 172.21.15.45:0/3105760190 conn(0x55dfcbbe1450 msgr2=0x55dfcbbe3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 -- 172.21.15.45:0/3105760190 shutdown_connections 2023-10-06T05:23:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.496+0000 7f5ff3e30040 1 -- 172.21.15.45:0/3105760190 wait complete. 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.497+0000 7f5ff3e30040 1 Processor -- start 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.497+0000 7f5ff3e30040 1 -- start start 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5ff3e30040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5ff3e30040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0890 0x55dfcbc9e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5ff3e30040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0c60 0x55dfcbc0f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5ff3e30040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcbc9d780 con 0x55dfcbbde210 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5ff3e30040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcbcba780 con 0x55dfcbca0890 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5ff3e30040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfcbcbaa20 con 0x55dfcbca0c60 2023-10-06T05:23:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5fed04e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5fed04e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52284/0 (socket says 172.21.15.45:52284) 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.498+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 learned_addr learned my addr 172.21.15.45:0/1056187611 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fed84f700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0c60 0x55dfcbc0f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fec84d700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0890 0x55dfcbc9e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0c60 msgr2=0x55dfcbc0f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fed04e700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55dfcbca0c60 0x55dfcbc0f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0890 msgr2=0x55dfcbc9e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fed04e700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dfcbca0890 0x55dfcbc9e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe400a440 con 0x55dfcbbde210 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fed04e700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9d240 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400aed0 tx=0x7f5fe4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fddffb700 1 -- 172.21.15.45:0/1056187611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4008dd0 con 0x55dfcbbde210 2023-10-06T05:23:48.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fddffb700 1 -- 172.21.15.45:0/1056187611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4008830 con 0x55dfcbbde210 2023-10-06T05:23:48.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dfcbb3a3d0 con 0x55dfcbbde210 2023-10-06T05:23:48.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5fddffb700 1 -- 172.21.15.45:0/1056187611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe401f410 con 0x55dfcbbde210 2023-10-06T05:23:48.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.499+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfcbb2dae0 con 0x55dfcbbde210 2023-10-06T05:23:48.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.501+0000 7f5fddffb700 1 -- 172.21.15.45:0/1056187611 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fe401c050 con 0x55dfcbbde210 2023-10-06T05:23:48.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.501+0000 7f5fddffb700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5fd00658c0 0x7f5fd0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.501+0000 7f5fddffb700 1 -- 172.21.15.45:0/1056187611 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f5fe4085190 con 0x55dfcbbde210 2023-10-06T05:23:48.506 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.502+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55dfcbcbd870 0x55dfcbc9bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.502+0000 7f5fec84d700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5fd00658c0 0x7f5fd0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.502+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbc9c1f0 con 0x55dfcbcbd870 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.502+0000 7f5fed84f700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55dfcbcbd870 0x55dfcbc9bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.502+0000 7f5fed84f700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55dfcbcbd870 0x55dfcbc9bcb0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.502+0000 7f5fec84d700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5fd00658c0 0x7f5fd0067d70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00057a0 tx=0x7f5fe0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.504+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v106'412 uv386 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.504+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd4370 con 0x55dfcbcbd870 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.505+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v106'413 uv413 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.505+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd4720 con 0x55dfcbcbd870 2023-10-06T05:23:48.509 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:48.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.507+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v106'414 uv414 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.507+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5ae0 con 0x55dfcbcbd870 2023-10-06T05:23:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.508+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v106'415 uv415 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.508+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5e90 con 0x55dfcbcbd870 2023-10-06T05:23:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.509+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v106'416 uv416 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.509+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd4ad0 con 0x55dfcbcbd870 2023-10-06T05:23:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.510+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v106'417 uv417 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.510+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd4eb0 con 0x55dfcbcbd870 2023-10-06T05:23:48.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.512+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v106'418 uv392 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.512+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd52b0 con 0x55dfcbcbd870 2023-10-06T05:23:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.513+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v106'419 uv419 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.513+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd52b0 con 0x55dfcbcbd870 2023-10-06T05:23:48.515 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.514+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v106'420 uv420 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.514+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.515+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v106'421 uv421 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.515+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.517+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v106'422 uv422 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.517+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.518+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v106'423 uv423 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.518+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.519+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v106'424 uv399 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.519+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.520+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v106'425 uv425 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.520+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.522+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v106'426 uv426 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.523 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.522+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.523+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v106'427 uv427 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.523+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.524+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v106'428 uv428 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.524+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.525+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v106'429 uv429 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.525+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.527+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v106'430 uv430 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.527+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.528+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v106'431 uv405 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.528+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.529+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v106'432 uv432 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.529+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.532 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.531+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v106'433 uv433 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.531+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.532+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v106'434 uv434 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.532+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.533+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v106'435 uv435 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.533+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.535+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v106'436 uv436 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.535+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.536+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v106'437 uv411 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.536+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.537+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v106'438 uv438 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.537+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.540 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:48.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.539+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v106'439 uv439 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.539+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.540+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v106'440 uv440 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.540+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.541+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v106'441 uv441 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.541+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcbd870 2023-10-06T05:23:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.543+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v106'442 uv442 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcbd870 2023-10-06T05:23:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.543+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55dfcbcd5630 0x55dfcbcd8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.543+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd5260 con 0x55dfcbcd5630 2023-10-06T05:23:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.543+0000 7f5fed04e700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55dfcbcd5630 0x55dfcbcd8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.544+0000 7f5fed04e700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55dfcbcd5630 0x55dfcbcd8760 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.545+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v106'304 uv287 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.545+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd8ca0 con 0x55dfcbcd5630 2023-10-06T05:23:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.547+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v106'305 uv305 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.547+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd9050 con 0x55dfcbcd5630 2023-10-06T05:23:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.548+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v106'306 uv306 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.550 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:48.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.548+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd98f0 con 0x55dfcbcd5630 2023-10-06T05:23:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.550+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v106'307 uv307 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.550+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcd9ca0 con 0x55dfcbcd5630 2023-10-06T05:23:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.551+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v106'308 uv308 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.551+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda050 con 0x55dfcbcd5630 2023-10-06T05:23:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.552+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v106'309 uv292 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.552+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.553+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v106'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.553+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.556 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:48.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.555+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v106'311 uv311 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.555+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.556+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v106'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.556+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.558+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v106'313 uv313 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.558+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.559+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v106'314 uv297 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.559+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.560+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v106'315 uv315 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.560+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.563 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.561+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v106'316 uv316 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.561+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcddb20 con 0x55dfcbcd5630 2023-10-06T05:23:48.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.563+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v106'317 uv317 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.563+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcddb20 con 0x55dfcbcd5630 2023-10-06T05:23:48.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.564+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v106'318 uv318 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.564+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.565+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v106'319 uv303 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.565+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.566+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v106'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.566+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.567+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v106'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.569 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:48.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.567+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdb410 con 0x55dfcbcd5630 2023-10-06T05:23:48.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.569+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v106'322 uv322 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.569+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdb410 con 0x55dfcbcd5630 2023-10-06T05:23:48.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.570+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v106'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.570+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcda400 con 0x55dfcbcd5630 2023-10-06T05:23:48.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.571+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v106'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.571+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55dfcbcda910 0x55dfcbcdccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.571+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdd280 con 0x55dfcbcda910 2023-10-06T05:23:48.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.571+0000 7f5fec84d700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55dfcbcda910 0x55dfcbcdccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.572+0000 7f5fec84d700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55dfcbcda910 0x55dfcbcdccf0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.573+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.573+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdd780 con 0x55dfcbcda910 2023-10-06T05:23:48.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.574+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v106'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.574+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcddc70 con 0x55dfcbcda910 2023-10-06T05:23:48.576 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:48.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.575+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v106'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.575+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf570 con 0x55dfcbcda910 2023-10-06T05:23:48.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.576+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.576+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf920 con 0x55dfcbcda910 2023-10-06T05:23:48.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.577+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.577+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcde560 con 0x55dfcbcda910 2023-10-06T05:23:48.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.578+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.578+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcde980 con 0x55dfcbcda910 2023-10-06T05:23:48.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.580+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v106'60 uv59 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.580+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdedd0 con 0x55dfcbcda910 2023-10-06T05:23:48.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.581+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v106'61 uv61 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.581+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdedd0 con 0x55dfcbcda910 2023-10-06T05:23:48.584 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:48.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.582+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v106'62 uv62 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.582+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcded80 con 0x55dfcbcda910 2023-10-06T05:23:48.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.583+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v106'63 uv63 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.583+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf1a0 con 0x55dfcbcd5630 2023-10-06T05:23:48.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.585+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.585+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf1a0 con 0x55dfcbcd5630 2023-10-06T05:23:48.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.586+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v106'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.586+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf1a0 con 0x55dfcbcd5630 2023-10-06T05:23:48.589 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:48.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.587+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v106'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.587+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf1a0 con 0x55dfcbcd5630 2023-10-06T05:23:48.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.589+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.589+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf1a0 con 0x55dfcbcd5630 2023-10-06T05:23:48.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.590+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.590+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcdf1a0 con 0x55dfcbcd5630 2023-10-06T05:23:48.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.591+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbcd5630 2023-10-06T05:23:48.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.591+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55dfcbcded50 0x55dfcbce21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.591+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce2730 con 0x55dfcbcded50 2023-10-06T05:23:48.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.592+0000 7f5fed84f700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55dfcbcded50 0x55dfcbce21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.592+0000 7f5fed84f700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55dfcbcded50 0x55dfcbce21f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.593+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v106'149 uv143 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.593+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce2730 con 0x55dfcbcded50 2023-10-06T05:23:48.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.594+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v106'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.594+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce2730 con 0x55dfcbcded50 2023-10-06T05:23:48.597 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:48.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.596+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v106'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.596+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce37e0 con 0x55dfcbcded50 2023-10-06T05:23:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.597+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v106'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.597+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce37a0 con 0x55dfcbcded50 2023-10-06T05:23:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.598+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v106'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.598+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce37a0 con 0x55dfcbcded50 2023-10-06T05:23:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.599+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v106'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.599+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce37a0 con 0x55dfcbcded50 2023-10-06T05:23:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.600+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v106'155 uv148 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.600+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce37a0 con 0x55dfcbcded50 2023-10-06T05:23:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.601+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v106'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.601+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce37a0 con 0x55dfcbcded50 2023-10-06T05:23:48.605 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:48.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.602+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v106'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.602+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce3760 con 0x55dfcbcded50 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.603+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v106'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.603+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce3760 con 0x55dfcbcded50 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.605+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v106'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.605+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55dfcbce2c40 0x55dfcbce3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.605+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce5c30 con 0x55dfcbce2c40 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.605+0000 7f5fed04e700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55dfcbce2c40 0x55dfcbce3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.605+0000 7f5fed04e700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55dfcbce2c40 0x55dfcbce3030 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.607+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v106'66 uv65 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbce2c40 2023-10-06T05:23:48.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.607+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce5c30 con 0x55dfcbce2c40 2023-10-06T05:23:48.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.608+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v106'67 uv67 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbce2c40 2023-10-06T05:23:48.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.608+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce5c30 con 0x55dfcbce2c40 2023-10-06T05:23:48.615 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:48.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.613+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v106'68 uv68 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbce2c40 2023-10-06T05:23:48.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.613+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce5c30 con 0x55dfcbce2c40 2023-10-06T05:23:48.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.614+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v106'69 uv69 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbce2c40 2023-10-06T05:23:48.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.614+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce6080 con 0x55dfcbce2c40 2023-10-06T05:23:48.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.617+0000 7f5fed04e700 1 -- 172.21.15.45:0/1056187611 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v106'70 uv70 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe4082fd0 con 0x55dfcbce2c40 2023-10-06T05:23:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.617+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce6080 con 0x55dfcbcda910 2023-10-06T05:23:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.618+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.618+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce6080 con 0x55dfcbcda910 2023-10-06T05:23:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.620+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v106'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.620+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce6080 con 0x55dfcbcda910 2023-10-06T05:23:48.622 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:48.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.621+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v106'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.621+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce7090 con 0x55dfcbcda910 2023-10-06T05:23:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.622+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.623+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce7090 con 0x55dfcbcda910 2023-10-06T05:23:48.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.624+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.624+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce6080 con 0x55dfcbcda910 2023-10-06T05:23:48.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.625+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.625+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55dfcbce6590 0x55dfcbce8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.625+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce8eb0 con 0x55dfcbce6590 2023-10-06T05:23:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.625+0000 7f5fec84d700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55dfcbce6590 0x55dfcbce8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.626+0000 7f5fec84d700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55dfcbce6590 0x55dfcbce8970 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.627+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v106'132 uv126 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.627+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce8f70 con 0x55dfcbce6590 2023-10-06T05:23:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.628+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v106'133 uv133 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.628+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9370 con 0x55dfcbce6590 2023-10-06T05:23:48.631 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.630+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v106'134 uv134 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.630+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9370 con 0x55dfcbce6590 2023-10-06T05:23:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.631+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v106'135 uv135 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.631+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9370 con 0x55dfcbce6590 2023-10-06T05:23:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.632+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v106'136 uv136 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.632+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9370 con 0x55dfcbce6590 2023-10-06T05:23:48.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.633+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v106'137 uv131 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.633+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9370 con 0x55dfcbce6590 2023-10-06T05:23:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.634+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v106'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.634+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9370 con 0x55dfcbce6590 2023-10-06T05:23:48.636 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.635+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v106'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.635+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9370 con 0x55dfcbce6590 2023-10-06T05:23:48.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.636+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v106'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.636+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcec0e0 con 0x55dfcbce6590 2023-10-06T05:23:48.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.637+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v106'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbce6590 2023-10-06T05:23:48.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.637+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.638+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v106'61 uv60 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.638+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.639+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v106'62 uv62 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.640+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.642 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:48.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.641+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v106'63 uv63 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.641+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.642+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v106'64 uv64 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.642+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55dfcbcea170 0x55dfcbcec830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.642+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcea170 2023-10-06T05:23:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.642+0000 7f5fed84f700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55dfcbcea170 0x55dfcbcec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.643+0000 7f5fed84f700 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55dfcbcea170 0x55dfcbcec830 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.644+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcea170 2023-10-06T05:23:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.644+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcea170 2023-10-06T05:23:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.645+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v106'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcea170 2023-10-06T05:23:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.645+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcea170 2023-10-06T05:23:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.646+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v106'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcea170 2023-10-06T05:23:48.648 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.646+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcea170 2023-10-06T05:23:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.647+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcea170 2023-10-06T05:23:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.647+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcea170 2023-10-06T05:23:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.648+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcea170 2023-10-06T05:23:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.648+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcea170 2023-10-06T05:23:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.650+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcea170 2023-10-06T05:23:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.650+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcded50 2023-10-06T05:23:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.651+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v106'69 uv68 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.651+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcded50 2023-10-06T05:23:48.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.652+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v106'70 uv70 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.652+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcded50 2023-10-06T05:23:48.655 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:48.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.653+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v106'71 uv71 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.653+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcded50 2023-10-06T05:23:48.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.654+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v106'72 uv72 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.654+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcded50 2023-10-06T05:23:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.656+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v106'73 uv73 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5fe8001940 con 0x55dfcbcded50 2023-10-06T05:23:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.656+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.658 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:48.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.657+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v106'170 uv170 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.657+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.658+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [write 0~554] v106'171 uv171 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.658+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.661+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v106'172 uv172 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.661+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.664+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 1.00000000 [create] v106'173 uv168 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.664+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.669+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 1.00000000 [delete] v106'174 uv174 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.669+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.672 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.671+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v106'175 uv175 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.671+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.672+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v106'176 uv176 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.672+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.675+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v106'177 uv177 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.675+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbce9860 con 0x55dfcbcda910 2023-10-06T05:23:48.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.677+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [omap-set-header] v106'178 uv178 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.677+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) v8 -- 0x55dfcbcecd70 con 0x55dfcbcda910 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5fec84d700 1 -- 172.21.15.45:0/1056187611 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v106'179 uv179 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5fe0008a80 con 0x55dfcbcda910 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55dfcbcda910 msgr2=0x55dfcbcdccf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55dfcbcda910 0x55dfcbcdccf0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55dfcbcbd870 msgr2=0x55dfcbc9bcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55dfcbcbd870 0x55dfcbc9bcb0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55dfcbce6590 msgr2=0x55dfcbce8970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55dfcbce6590 0x55dfcbce8970 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55dfcbcd5630 msgr2=0x55dfcbcd8760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55dfcbcd5630 0x55dfcbcd8760 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55dfcbcea170 msgr2=0x55dfcbcec830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55dfcbcea170 0x55dfcbcec830 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55dfcbce2c40 msgr2=0x55dfcbce3030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55dfcbce2c40 0x55dfcbce3030 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55dfcbcded50 msgr2=0x55dfcbce21f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 reap_dead start 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55dfcbcded50 0x55dfcbce21f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 reap_dead start 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5fed84f700 1 -- 172.21.15.45:0/1056187611 reap_dead start 2023-10-06T05:23:48.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5fd00658c0 msgr2=0x7f5fd0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5fd00658c0 0x7f5fd0067d70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00057a0 tx=0x7f5fe0016000 comp rx=0 tx=0).stop 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 msgr2=0x55dfcbc9d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9d240 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400aed0 tx=0x7f5fe4018000 comp rx=0 tx=0).stop 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.680+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 shutdown_connections 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.681+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfcbbde210 0x55dfcbc9d240 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.681+0000 7f5ff3e30040 1 --2- 172.21.15.45:0/1056187611 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5fd00658c0 0x7f5fd0067d70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.681+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 >> 172.21.15.45:0/1056187611 conn(0x55dfcbbe1450 msgr2=0x55dfcbc27300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.681+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 shutdown_connections 2023-10-06T05:23:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.681+0000 7f5ff3e30040 1 -- 172.21.15.45:0/1056187611 wait complete. 2023-10-06T05:23:48.686 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2023-10-06T05:23:48.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.774+0000 7f164db23040 1 Processor -- start 2023-10-06T05:23:48.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.774+0000 7f164db23040 1 -- start start 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f164db23040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a31c120 0x55aa9a338cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f164db23040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a320450 0x55aa9a3c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f164db23040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a2f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f164db23040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9a23b980 con 0x55aa9a3c6fc0 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f164db23040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9a23b620 con 0x55aa9a320450 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f164db23040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa9a232400 con 0x55aa9a31c120 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f1646d41700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a31c120 0x55aa9a338cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f1647542700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a2f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f1646540700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a320450 0x55aa9a3c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f1646d41700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a31c120 0x55aa9a338cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:56668/0 (socket says 172.21.15.45:56668) 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.775+0000 7f1646d41700 1 -- 172.21.15.45:0/4256448725 learned_addr learned my addr 172.21.15.45:0/4256448725 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1646d41700 1 -- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a320450 msgr2=0x55aa9a3c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1646d41700 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a320450 0x55aa9a3c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1646d41700 1 -- 172.21.15.45:0/4256448725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a3c6fc0 msgr2=0x55aa9a2f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1646d41700 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a2f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1646d41700 1 -- 172.21.15.45:0/4256448725 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c005db0 con 0x55aa9a31c120 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1646540700 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a320450 0x55aa9a3c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1647542700 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a2f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f1646d41700 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a31c120 0x55aa9a338cf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f163800a750 tx=0x7f1638005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383aae634834103d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f164553e700 1 -- 172.21.15.45:0/4256448725 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800f070 con 0x55aa9a31c120 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f164553e700 1 -- 172.21.15.45:0/4256448725 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638008550 con 0x55aa9a31c120 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.776+0000 7f164553e700 1 -- 172.21.15.45:0/4256448725 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380088d0 con 0x55aa9a31c120 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 -- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a31c120 msgr2=0x55aa9a338cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a31c120 0x55aa9a338cf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f163800a750 tx=0x7f1638005c80 comp rx=0 tx=0).stop 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 -- 172.21.15.45:0/4256448725 shutdown_connections 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a2f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a31c120 0x55aa9a338cf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 --2- 172.21.15.45:0/4256448725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a320450 0x55aa9a3c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 -- 172.21.15.45:0/4256448725 >> 172.21.15.45:0/4256448725 conn(0x55aa9a2f0bd0 msgr2=0x55aa9a334d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 -- 172.21.15.45:0/4256448725 shutdown_connections 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 -- 172.21.15.45:0/4256448725 wait complete. 2023-10-06T05:23:48.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 Processor -- start 2023-10-06T05:23:48.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.777+0000 7f164db23040 1 -- start start 2023-10-06T05:23:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f164db23040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a31c120 0x55aa9a3c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f164db23040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a320450 0x55aa9a3c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f164db23040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a3c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f164db23040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9a3cb940 con 0x55aa9a31c120 2023-10-06T05:23:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f164db23040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9a3cbbe0 con 0x55aa9a3c6fc0 2023-10-06T05:23:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f164db23040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa9a3cbe80 con 0x55aa9a320450 2023-10-06T05:23:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f1646d41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a31c120 0x55aa9a3c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f1646d41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a31c120 0x55aa9a3c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52300/0 (socket says 172.21.15.45:52300) 2023-10-06T05:23:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f1646540700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a320450 0x55aa9a3c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f1646d41700 1 -- 172.21.15.45:0/3316488648 learned_addr learned my addr 172.21.15.45:0/3316488648 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.778+0000 7f1647542700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a3c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f1646540700 1 -- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a3c6fc0 msgr2=0x55aa9a3c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f1646540700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa9a3c6fc0 0x55aa9a3c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f1646540700 1 -- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a31c120 msgr2=0x55aa9a3c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f1646540700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a31c120 0x55aa9a3c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f1646540700 1 -- 172.21.15.45:0/3316488648 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa9a2641c0 con 0x55aa9a320450 2023-10-06T05:23:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f1646d41700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9a31c120 0x55aa9a3c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f1646540700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a320450 0x55aa9a3c0520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f163c015000 tx=0x7f163c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f16377fe700 1 -- 172.21.15.45:0/3316488648 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c012ce0 con 0x55aa9a320450 2023-10-06T05:23:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f16377fe700 1 -- 172.21.15.45:0/3316488648 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c021070 con 0x55aa9a320450 2023-10-06T05:23:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f16377fe700 1 -- 172.21.15.45:0/3316488648 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c0286d0 con 0x55aa9a320450 2023-10-06T05:23:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aa9a23d750 con 0x55aa9a320450 2023-10-06T05:23:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.779+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163800a440 con 0x55aa9a320450 2023-10-06T05:23:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.781+0000 7f16377fe700 1 -- 172.21.15.45:0/3316488648 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f163c028a80 con 0x55aa9a320450 2023-10-06T05:23:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.781+0000 7f16377fe700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f162c0658c0 0x7f162c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.781+0000 7f16377fe700 1 -- 172.21.15.45:0/3316488648 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f163c023070 con 0x55aa9a320450 2023-10-06T05:23:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.781+0000 7f1646d41700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f162c0658c0 0x7f162c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.781+0000 7f164db23040 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55aa9a3c99f0 0x55aa9a3c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.781+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e106) v8 -- 0x55aa9a3c0a60 con 0x55aa9a3c99f0 2023-10-06T05:23:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.782+0000 7f1647542700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55aa9a3c99f0 0x55aa9a3c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.782+0000 7f1646d41700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f162c0658c0 0x7f162c067d70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1638005ce0 tx=0x7f1638018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.782+0000 7f1647542700 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55aa9a3c99f0 0x55aa9a3c9dc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f1647542700 1 -- 172.21.15.45:0/3316488648 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 1.00000000 [delete] v106'180 uv180 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f164000f900 con 0x55aa9a3c99f0 2023-10-06T05:23:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55aa9a3c99f0 msgr2=0x55aa9a3c9dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55aa9a3c99f0 0x55aa9a3c9dc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f162c0658c0 msgr2=0x7f162c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f162c0658c0 0x7f162c067d70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1638005ce0 tx=0x7f1638018000 comp rx=0 tx=0).stop 2023-10-06T05:23:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a320450 msgr2=0x55aa9a3c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 --2- 172.21.15.45:0/3316488648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa9a320450 0x55aa9a3c0520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f163c015000 tx=0x7f163c016000 comp rx=0 tx=0).stop 2023-10-06T05:23:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f1647542700 1 -- 172.21.15.45:0/3316488648 reap_dead start 2023-10-06T05:23:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 shutdown_connections 2023-10-06T05:23:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 >> 172.21.15.45:0/3316488648 conn(0x55aa9a2f0bd0 msgr2=0x55aa9a3c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 shutdown_connections 2023-10-06T05:23:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.783+0000 7f164db23040 1 -- 172.21.15.45:0/3316488648 wait complete. 2023-10-06T05:23:48.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.933+0000 7f14978f0700 1 Processor -- start 2023-10-06T05:23:48.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.934+0000 7f14978f0700 1 -- start start 2023-10-06T05:23:48.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.934+0000 7f14978f0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14900ccb70 0x7f14900aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.934+0000 7f14978f0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900c96a0 con 0x7f14900c8b70 2023-10-06T05:23:48.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f14978f0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900c3b50 con 0x7f1490157330 2023-10-06T05:23:48.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f14978f0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1490157700 con 0x7f14900ccb70 2023-10-06T05:23:48.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1495e8d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f14900c8b70 0x7f14900c8f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48724/0 (socket says 172.21.15.45:48724) 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1495e8d700 1 -- 172.21.15.45:0/1507069994 learned_addr learned my addr 172.21.15.45:0/1507069994 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1494e8b700 1 --2- 172.21.15.45:0/1507069994 >> v2:172.21.15.144:3300/0 conn(0x7f14900ccb70 0x7f14900aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1487fff700 1 -- 172.21.15.45:0/1507069994 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320987970 0 0) 0x7f14900c3b50 con 0x7f1490157330 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1487fff700 1 -- 172.21.15.45:0/1507069994 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f147c003570 con 0x7f1490157330 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1487fff700 1 -- 172.21.15.45:0/1507069994 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959909627 0 0) 0x7f1488001490 con 0x7f14900c8b70 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1487fff700 1 -- 172.21.15.45:0/1507069994 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14900c3b50 con 0x7f14900c8b70 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1494e8b700 1 -- 172.21.15.45:0/1507069994 >> v1:172.21.15.144:6789/0 conn(0x7f1490157330 legacy=0x7f14900ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1494e8b700 1 -- 172.21.15.45:0/1507069994 >> v1:172.21.15.45:6789/0 conn(0x7f14900c8b70 legacy=0x7f14900c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1494e8b700 1 -- 172.21.15.45:0/1507069994 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900c98e0 con 0x7f14900ccb70 2023-10-06T05:23:48.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1494e8b700 1 --2- 172.21.15.45:0/1507069994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14900ccb70 0x7f14900aaae0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f14800098e0 tx=0x7f14800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23af5f6b8e863f7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1487fff700 1 -- 172.21.15.45:0/1507069994 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480011630 con 0x7f14900ccb70 2023-10-06T05:23:48.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1487fff700 1 -- 172.21.15.45:0/1507069994 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1480011790 con 0x7f14900ccb70 2023-10-06T05:23:48.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.935+0000 7f1487fff700 1 -- 172.21.15.45:0/1507069994 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480011b30 con 0x7f14900ccb70 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.936+0000 7f14978f0700 1 -- 172.21.15.45:0/1507069994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14900ccb70 msgr2=0x7f14900aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.936+0000 7f14978f0700 1 --2- 172.21.15.45:0/1507069994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14900ccb70 0x7f14900aaae0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f14800098e0 tx=0x7f14800091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.936+0000 7f14978f0700 1 -- 172.21.15.45:0/1507069994 shutdown_connections 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.936+0000 7f14978f0700 1 --2- 172.21.15.45:0/1507069994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14900ccb70 0x7f14900aaae0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.936+0000 7f14978f0700 1 -- 172.21.15.45:0/1507069994 >> 172.21.15.45:0/1507069994 conn(0x7f1490067e00 msgr2=0x7f14900cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.936+0000 7f14978f0700 1 -- 172.21.15.45:0/1507069994 shutdown_connections 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.936+0000 7f14978f0700 1 -- 172.21.15.45:0/1507069994 wait complete. 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.937+0000 7f14978f0700 1 Processor -- start 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.937+0000 7f14978f0700 1 -- start start 2023-10-06T05:23:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f14978f0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14900c8b70 0x7f14900b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f14978f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14900ccb70 0x7f14900b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f14978f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1490157330 0x7f1490152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f14978f0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b3600 con 0x7f14900ccb70 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f14978f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b1270 con 0x7f1490157330 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f14978f0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1490153240 con 0x7f14900c8b70 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f1494e8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14900ccb70 0x7f14900b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f1494e8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14900ccb70 0x7f14900b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52308/0 (socket says 172.21.15.45:52308) 2023-10-06T05:23:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f1494e8b700 1 -- 172.21.15.45:0/3801966618 learned_addr learned my addr 172.21.15.45:0/3801966618 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f149568c700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14900c8b70 0x7f14900b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.938+0000 7f1495e8d700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1490157330 0x7f1490152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f1494e8b700 1 -- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14900c8b70 msgr2=0x7f14900b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f1494e8b700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14900c8b70 0x7f14900b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f1494e8b700 1 -- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1490157330 msgr2=0x7f1490152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f1494e8b700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1490157330 0x7f1490152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f1494e8b700 1 -- 172.21.15.45:0/3801966618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480009420 con 0x7f14900ccb70 2023-10-06T05:23:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f1494e8b700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14900ccb70 0x7f14900b0d30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f149006bac0 tx=0x7f1480017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f14857fa700 1 -- 172.21.15.45:0/3801966618 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148001f070 con 0x7f14900ccb70 2023-10-06T05:23:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f14857fa700 1 -- 172.21.15.45:0/3801966618 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1480000bc0 con 0x7f14900ccb70 2023-10-06T05:23:48.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900b3860 con 0x7f14900ccb70 2023-10-06T05:23:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f14857fa700 1 -- 172.21.15.45:0/3801966618 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480025420 con 0x7f14900ccb70 2023-10-06T05:23:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.939+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14900aef40 con 0x7f14900ccb70 2023-10-06T05:23:48.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.940+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149006e350 con 0x7f14900ccb70 2023-10-06T05:23:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.941+0000 7f14857fa700 1 -- 172.21.15.45:0/3801966618 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1480023070 con 0x7f14900ccb70 2023-10-06T05:23:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.941+0000 7f14857fa700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1474066f30 0x7f14740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.942+0000 7f14857fa700 1 -- 172.21.15.45:0/3801966618 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f14800863e0 con 0x7f14900ccb70 2023-10-06T05:23:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.942+0000 7f149568c700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1474066f30 0x7f14740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.942+0000 7f149568c700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1474066f30 0x7f14740693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f148c007fd0 tx=0x7f148c007db0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:48.946+0000 7f14857fa700 1 -- 172.21.15.45:0/3801966618 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1480052ee0 con 0x7f14900ccb70 2023-10-06T05:23:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.192+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f14900c9bc0 con 0x7f14900ccb70 2023-10-06T05:23:49.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.434+0000 7f14857fa700 1 -- 172.21.15.45:0/3801966618 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v178) v1 ==== 162+0+0 (secure 0 0 0) 0x7f1480056500 con 0x7f14900ccb70 2023-10-06T05:23:49.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.436+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1474066f30 msgr2=0x7f14740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.436+0000 7f14978f0700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1474066f30 0x7f14740693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f148c007fd0 tx=0x7f148c007db0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.436+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14900ccb70 msgr2=0x7f14900b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.436+0000 7f14978f0700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14900ccb70 0x7f14900b0d30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f149006bac0 tx=0x7f1480017000 comp rx=0 tx=0).stop 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 shutdown_connections 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1474066f30 0x7f14740693e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1490157330 0x7f1490152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14900c8b70 0x7f14900b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 --2- 172.21.15.45:0/3801966618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14900ccb70 0x7f14900b0d30 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 >> 172.21.15.45:0/3801966618 conn(0x7f1490067e00 msgr2=0x7f1490156270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 shutdown_connections 2023-10-06T05:23:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.437+0000 7f14978f0700 1 -- 172.21.15.45:0/3801966618 wait complete. 2023-10-06T05:23:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:49.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.589+0000 7f06a6a1c700 1 Processor -- start 2023-10-06T05:23:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.590+0000 7f06a6a1c700 1 -- start start 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.590+0000 7f06a6a1c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a0158f60 0x7f06a0159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.590+0000 7f06a6a1c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06a00cc390 0x7f06a00cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.590+0000 7f06a6a1c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a00cb530 con 0x7f06a006a060 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.590+0000 7f06a6a1c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a006a500 con 0x7f06a0158f60 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.590+0000 7f06a6a1c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a0069bd0 con 0x7f06a00cc390 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a0158f60 0x7f06a0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06a006a060 0x7f06a00cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48734/0 (socket says 172.21.15.45:48734) 2023-10-06T05:23:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069f7fe700 1 -- 172.21.15.45:0/246369265 learned_addr learned my addr 172.21.15.45:0/246369265 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f06a4fb9700 1 --2- 172.21.15.45:0/246369265 >> v2:172.21.15.144:3300/0 conn(0x7f06a00cc390 0x7f06a00cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069ffff700 1 -- 172.21.15.45:0/246369265 >> v2:172.21.15.144:3300/0 conn(0x7f06a00cc390 msgr2=0x7f06a00cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069ffff700 1 --2- 172.21.15.45:0/246369265 >> v2:172.21.15.144:3300/0 conn(0x7f06a00cc390 0x7f06a00cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069ffff700 1 -- 172.21.15.45:0/246369265 >> v1:172.21.15.45:6789/0 conn(0x7f06a006a060 legacy=0x7f06a00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069ffff700 1 -- 172.21.15.45:0/246369265 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00ccd10 con 0x7f06a0158f60 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069effd700 1 -- 172.21.15.45:0/246369265 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127233834 0 0) 0x7f0694001110 con 0x7f06a006a060 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069ffff700 1 --2- 172.21.15.45:0/246369265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0158f60 0x7f06a0159330 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f06a00ca4c0 tx=0x7f06900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb13649e4055557 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069effd700 1 -- 172.21.15.45:0/246369265 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690011630 con 0x7f06a0158f60 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069effd700 1 -- 172.21.15.45:0/246369265 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690011790 con 0x7f06a0158f60 2023-10-06T05:23:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.591+0000 7f069effd700 1 -- 172.21.15.45:0/246369265 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690011b30 con 0x7f06a0158f60 2023-10-06T05:23:49.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 -- 172.21.15.45:0/246369265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0158f60 msgr2=0x7f06a0159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/246369265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0158f60 0x7f06a0159330 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f06a00ca4c0 tx=0x7f06900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 -- 172.21.15.45:0/246369265 shutdown_connections 2023-10-06T05:23:49.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/246369265 >> v2:172.21.15.144:3300/0 conn(0x7f06a00cc390 0x7f06a00cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/246369265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0158f60 0x7f06a0159330 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 -- 172.21.15.45:0/246369265 >> 172.21.15.45:0/246369265 conn(0x7f06a00cdf50 msgr2=0x7f06a00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 -- 172.21.15.45:0/246369265 shutdown_connections 2023-10-06T05:23:49.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.592+0000 7f06a6a1c700 1 -- 172.21.15.45:0/246369265 wait complete. 2023-10-06T05:23:49.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.593+0000 7f06a6a1c700 1 Processor -- start 2023-10-06T05:23:49.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.593+0000 7f06a6a1c700 1 -- start start 2023-10-06T05:23:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f06a6a1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a006a060 0x7f06a014a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f06a6a1c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a00cc390 0x7f06a00c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f06a6a1c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a0158f60 0x7f06a00b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f06a6a1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015d770 con 0x7f06a006a060 2023-10-06T05:23:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f06a6a1c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015caf0 con 0x7f06a0158f60 2023-10-06T05:23:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f06a6a1c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015cdd0 con 0x7f06a00cc390 2023-10-06T05:23:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f069ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a006a060 0x7f06a014a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f069ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a006a060 0x7f06a014a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52320/0 (socket says 172.21.15.45:52320) 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f069ffff700 1 -- 172.21.15.45:0/174487162 learned_addr learned my addr 172.21.15.45:0/174487162 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f069f7fe700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a00cc390 0x7f06a00c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.594+0000 7f06a4fb9700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a0158f60 0x7f06a00b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f069ffff700 1 -- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a00cc390 msgr2=0x7f06a00c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f069ffff700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a00cc390 0x7f06a00c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f069ffff700 1 -- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a0158f60 msgr2=0x7f06a00b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f069ffff700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a0158f60 0x7f06a00b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f069ffff700 1 -- 172.21.15.45:0/174487162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690009420 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f069ffff700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a006a060 0x7f06a014a810 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f06a00c4f80 tx=0x7f0690017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f0687fff700 1 -- 172.21.15.45:0/174487162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690000d10 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f0687fff700 1 -- 172.21.15.45:0/174487162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690011b10 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a015d0b0 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f0687fff700 1 -- 172.21.15.45:0/174487162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690023660 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.595+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a014ad50 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.596+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a006f410 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.597+0000 7f0687fff700 1 -- 172.21.15.45:0/174487162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06900193b0 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.597+0000 7f0687fff700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f068c066f30 0x7f068c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.597+0000 7f0687fff700 1 -- 172.21.15.45:0/174487162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f06900860d0 con 0x7f06a006a060 2023-10-06T05:23:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.597+0000 7f069f7fe700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f068c066f30 0x7f068c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.598+0000 7f069f7fe700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f068c066f30 0x7f068c0693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f06a00c6750 tx=0x7f0694007b90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.602+0000 7f0687fff700 1 -- 172.21.15.45:0/174487162 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0690052bd0 con 0x7f06a006a060 2023-10-06T05:23:49.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.853+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a0068fc0 con 0x7f06a006a060 2023-10-06T05:23:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.853+0000 7f0687fff700 1 -- 172.21.15.45:0/174487162 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f0690015090 con 0x7f06a006a060 2023-10-06T05:23:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:49.855 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":179,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:49.435005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7316":{"gid":7316,"name":"c","rank":0,"incarnation":179,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/2681459400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2681459400},{"type":"v1","addr":"172.21.15.45:6835","nonce":2681459400}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f068c066f30 msgr2=0x7f068c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f068c066f30 0x7f068c0693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f06a00c6750 tx=0x7f0694007b90 comp rx=0 tx=0).stop 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a006a060 msgr2=0x7f06a014a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a006a060 0x7f06a014a810 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f06a00c4f80 tx=0x7f0690017000 comp rx=0 tx=0).stop 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 shutdown_connections 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f068c066f30 0x7f068c0693e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a0158f60 0x7f06a00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a006a060 0x7f06a014a810 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 --2- 172.21.15.45:0/174487162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a00cc390 0x7f06a00c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.855+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 >> 172.21.15.45:0/174487162 conn(0x7f06a00cdf50 msgr2=0x7f06a015f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.856+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 shutdown_connections 2023-10-06T05:23:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:49.856+0000 7f06a6a1c700 1 -- 172.21.15.45:0/174487162 wait complete. 2023-10-06T05:23:49.859 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 179 2023-10-06T05:23:49.871 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:23:52.452 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:52.451+0000 7f1b386c3700 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2023-10-06T05:23:52.452 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:52.451+0000 7f1b386c3700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-10-06T05:23:52.453 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:52.452+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:52.453 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:52.452+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:23:52.464 INFO:tasks.ceph.mds.c.smithi045.stderr: -17> 2023-10-06T05:23:52.451+0000 7f1b386c3700 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2023-10-06T05:23:52.464 INFO:tasks.ceph.mds.c.smithi045.stderr: -15> 2023-10-06T05:23:52.451+0000 7f1b386c3700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-10-06T05:23:52.488 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:54.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.010+0000 7f23bc410700 1 Processor -- start 2023-10-06T05:23:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.010+0000 7f23bc410700 1 -- start start 2023-10-06T05:23:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.011+0000 7f23bc410700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b40c8ff0 0x7f23b40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.011+0000 7f23bc410700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23b40caad0 0x7f23b40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.011+0000 7f23bc410700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b415b000 con 0x7f23b4156af0 2023-10-06T05:23:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.011+0000 7f23bc410700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b415b2e0 con 0x7f23b40caad0 2023-10-06T05:23:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.011+0000 7f23bc410700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b415b5c0 con 0x7f23b40c8ff0 2023-10-06T05:23:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba1ac700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b40c8ff0 0x7f23b40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23b99ab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23b40caad0 0x7f23b40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba9ad700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f23b4156af0 0x7f23b4156f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:53996/0 (socket says 172.21.15.45:53996) 2023-10-06T05:23:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba9ad700 1 -- 172.21.15.45:0/2162252666 learned_addr learned my addr 172.21.15.45:0/2162252666 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba1ac700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b40c8ff0 0x7f23b40c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:56732/0 (socket says 172.21.15.45:56732) 2023-10-06T05:23:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23b99ab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23b40caad0 0x7f23b40caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:36140/0 (socket says 172.21.15.45:36140) 2023-10-06T05:23:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba1ac700 1 -- 172.21.15.45:0/2162252666 >> v2:172.21.15.144:3300/0 conn(0x7f23b40caad0 msgr2=0x7f23b40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba1ac700 1 --2- 172.21.15.45:0/2162252666 >> v2:172.21.15.144:3300/0 conn(0x7f23b40caad0 0x7f23b40caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba1ac700 1 -- 172.21.15.45:0/2162252666 >> v1:172.21.15.144:6789/0 conn(0x7f23b4156af0 legacy=0x7f23b4156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba1ac700 1 -- 172.21.15.45:0/2162252666 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b415b8a0 con 0x7f23b40c8ff0 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23b99ab700 1 --2- 172.21.15.45:0/2162252666 >> v2:172.21.15.144:3300/0 conn(0x7f23b40caad0 0x7f23b40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23b91aa700 1 -- 172.21.15.45:0/2162252666 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031024997 0 0) 0x7f23b415b000 con 0x7f23b4156af0 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23ba1ac700 1 --2- 172.21.15.45:0/2162252666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b40c8ff0 0x7f23b40c93c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f23a40098e0 tx=0x7f23a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572cf44b3402a058 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23b91aa700 1 -- 172.21.15.45:0/2162252666 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400b640 con 0x7f23b40c8ff0 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.012+0000 7f23b91aa700 1 -- 172.21.15.45:0/2162252666 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a400b7a0 con 0x7f23b40c8ff0 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23b91aa700 1 -- 172.21.15.45:0/2162252666 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400bb40 con 0x7f23b40c8ff0 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 -- 172.21.15.45:0/2162252666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b40c8ff0 msgr2=0x7f23b40c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 --2- 172.21.15.45:0/2162252666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b40c8ff0 0x7f23b40c93c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f23a40098e0 tx=0x7f23a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 -- 172.21.15.45:0/2162252666 shutdown_connections 2023-10-06T05:23:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 --2- 172.21.15.45:0/2162252666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b40c8ff0 0x7f23b40c93c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 --2- 172.21.15.45:0/2162252666 >> v2:172.21.15.144:3300/0 conn(0x7f23b40caad0 0x7f23b40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 -- 172.21.15.45:0/2162252666 >> 172.21.15.45:0/2162252666 conn(0x7f23b40cdf50 msgr2=0x7f23b40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 -- 172.21.15.45:0/2162252666 shutdown_connections 2023-10-06T05:23:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.013+0000 7f23bc410700 1 -- 172.21.15.45:0/2162252666 wait complete. 2023-10-06T05:23:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.014+0000 7f23bc410700 1 Processor -- start 2023-10-06T05:23:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.014+0000 7f23bc410700 1 -- start start 2023-10-06T05:23:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23bc410700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40c8ff0 0x7f23b40ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23bc410700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40caad0 0x7f23b4154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23bc410700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4156af0 0x7f23b40b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23bc410700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40ad790 con 0x7f23b40c8ff0 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23bc410700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40c69b0 con 0x7f23b40caad0 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23bc410700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40c6c30 con 0x7f23b4156af0 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23ba1ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40c8ff0 0x7f23b40ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23ba1ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40c8ff0 0x7f23b40ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52330/0 (socket says 172.21.15.45:52330) 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23ba1ac700 1 -- 172.21.15.45:0/2203145868 learned_addr learned my addr 172.21.15.45:0/2203145868 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23b99ab700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40caad0 0x7f23b4154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.015+0000 7f23ba9ad700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4156af0 0x7f23b40b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23ba1ac700 1 -- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4156af0 msgr2=0x7f23b40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23ba1ac700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4156af0 0x7f23b40b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23ba1ac700 1 -- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40caad0 msgr2=0x7f23b4154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23ba1ac700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40caad0 0x7f23b4154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23ba1ac700 1 -- 172.21.15.45:0/2203145868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0009420 con 0x7f23b40c8ff0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23ba9ad700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4156af0 0x7f23b40b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23ba1ac700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40c8ff0 0x7f23b40ad250 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f23b415b700 tx=0x7f23a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23aa7fc700 1 -- 172.21.15.45:0/2203145868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4000d10 con 0x7f23b40c8ff0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23aa7fc700 1 -- 172.21.15.45:0/2203145868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a400bb20 con 0x7f23b40c8ff0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a4009420 con 0x7f23b40c8ff0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23aa7fc700 1 -- 172.21.15.45:0/2203145868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4011650 con 0x7f23b40c8ff0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.016+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b40c7270 con 0x7f23b40c8ff0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.017+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b406f410 con 0x7f23b40c8ff0 2023-10-06T05:23:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.018+0000 7f23aa7fc700 1 -- 172.21.15.45:0/2203145868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23a4011a00 con 0x7f23b40c8ff0 2023-10-06T05:23:55.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.018+0000 7f23aa7fc700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2398066f30 0x7f23980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.018+0000 7f23aa7fc700 1 -- 172.21.15.45:0/2203145868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f23a4086340 con 0x7f23b40c8ff0 2023-10-06T05:23:55.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.018+0000 7f23b99ab700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2398066f30 0x7f23980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.019+0000 7f23b99ab700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2398066f30 0x7f23980693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f23b406a910 tx=0x7f23b00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.023+0000 7f23aa7fc700 1 -- 172.21.15.45:0/2203145868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23a4052e00 con 0x7f23b40c8ff0 2023-10-06T05:23:55.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.274+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23b40c4690 con 0x7f23b40c8ff0 2023-10-06T05:23:55.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.274+0000 7f23aa7fc700 1 -- 172.21.15.45:0/2203145868 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+4599 (secure 0 0 0) 0x7f23a4011cb0 con 0x7f23b40c8ff0 2023-10-06T05:23:55.276 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:55.277 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":184,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3351321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3351321197},{"type":"v1","addr":"172.21.15.45:6835","nonce":3351321197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:52.452922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2398066f30 msgr2=0x7f23980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2398066f30 0x7f23980693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f23b406a910 tx=0x7f23b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40c8ff0 msgr2=0x7f23b40ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40c8ff0 0x7f23b40ad250 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f23b415b700 tx=0x7f23a4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 shutdown_connections 2023-10-06T05:23:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2398066f30 0x7f23980693e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4156af0 0x7f23b40b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40c8ff0 0x7f23b40ad250 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 --2- 172.21.15.45:0/2203145868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40caad0 0x7f23b4154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 >> 172.21.15.45:0/2203145868 conn(0x7f23b40cdf50 msgr2=0x7f23b40cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 shutdown_connections 2023-10-06T05:23:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.277+0000 7f23bc410700 1 -- 172.21.15.45:0/2203145868 wait complete. 2023-10-06T05:23:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 184 2023-10-06T05:23:55.293 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:23:55.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.424+0000 7f06ac03a700 1 Processor -- start 2023-10-06T05:23:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.425+0000 7f06ac03a700 1 -- start start 2023-10-06T05:23:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06ac03a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a4153620 0x7f06a415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06ac03a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a41539f0 0x7f06a415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06ac03a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06a40cc8b0 0x7f06a40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06ac03a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a415a7a0 con 0x7f06a4153620 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06ac03a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40ce3f0 con 0x7f06a40cc8b0 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06ac03a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a415d300 con 0x7f06a41539f0 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a4153620 0x7f06a415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a4153620 0x7f06a415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52338/0 (socket says 172.21.15.45:52338) 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 -- 172.21.15.45:0/2728091840 learned_addr learned my addr 172.21.15.45:0/2728091840 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06aa5d7700 1 --2- 172.21.15.45:0/2728091840 >> v2:172.21.15.144:3300/0 conn(0x7f06a40cc8b0 0x7f06a40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a95d5700 1 --2- 172.21.15.45:0/2728091840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a41539f0 0x7f06a415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 -- 172.21.15.45:0/2728091840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a41539f0 msgr2=0x7f06a415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 --2- 172.21.15.45:0/2728091840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a41539f0 0x7f06a415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 -- 172.21.15.45:0/2728091840 >> v2:172.21.15.144:3300/0 conn(0x7f06a40cc8b0 msgr2=0x7f06a40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 --2- 172.21.15.45:0/2728091840 >> v2:172.21.15.144:3300/0 conn(0x7f06a40cc8b0 0x7f06a40ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 -- 172.21.15.45:0/2728091840 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40ce690 con 0x7f06a4153620 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.426+0000 7f06a9dd6700 1 --2- 172.21.15.45:0/2728091840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a415a260 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f06a4157310 tx=0x7f06a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5f5309990ff6fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06a8dd4700 1 -- 172.21.15.45:0/2728091840 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000b860 con 0x7f06a4153620 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06a8dd4700 1 -- 172.21.15.45:0/2728091840 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a000b9c0 con 0x7f06a4153620 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06a8dd4700 1 -- 172.21.15.45:0/2728091840 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000bd60 con 0x7f06a4153620 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 -- 172.21.15.45:0/2728091840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 msgr2=0x7f06a415a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 --2- 172.21.15.45:0/2728091840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a415a260 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f06a4157310 tx=0x7f06a0009250 comp rx=0 tx=0).stop 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 -- 172.21.15.45:0/2728091840 shutdown_connections 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 --2- 172.21.15.45:0/2728091840 >> v2:172.21.15.144:3300/0 conn(0x7f06a40cc8b0 0x7f06a40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 --2- 172.21.15.45:0/2728091840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a415a260 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 --2- 172.21.15.45:0/2728091840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a41539f0 0x7f06a415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 -- 172.21.15.45:0/2728091840 >> 172.21.15.45:0/2728091840 conn(0x7f06a406ed50 msgr2=0x7f06a406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 -- 172.21.15.45:0/2728091840 shutdown_connections 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.427+0000 7f06ac03a700 1 -- 172.21.15.45:0/2728091840 wait complete. 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 Processor -- start 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 -- start start 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a40cc8b0 0x7f06a416a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a4152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a4149d60 0x7f06a414a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a416a690 con 0x7f06a4153620 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40ceb20 con 0x7f06a40cc8b0 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.429+0000 7f06ac03a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40cee00 con 0x7f06a4149d60 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a4152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a4152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52350/0 (socket says 172.21.15.45:52350) 2023-10-06T05:23:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 -- 172.21.15.45:0/1732757620 learned_addr learned my addr 172.21.15.45:0/1732757620 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a9dd6700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a40cc8b0 0x7f06a416a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06aa5d7700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a4149d60 0x7f06a414a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 -- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a4149d60 msgr2=0x7f06a414a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a4149d60 0x7f06a414a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 -- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a40cc8b0 msgr2=0x7f06a416a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a40cc8b0 0x7f06a416a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 -- 172.21.15.45:0/1732757620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00094f0 con 0x7f06a4153620 2023-10-06T05:23:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06a95d5700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a4152a50 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0694006b00 tx=0x7f0694008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f0699ffb700 1 -- 172.21.15.45:0/1732757620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940125f0 con 0x7f06a4153620 2023-10-06T05:23:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f0699ffb700 1 -- 172.21.15.45:0/1732757620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694012770 con 0x7f06a4153620 2023-10-06T05:23:55.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a40cf0e0 con 0x7f06a4153620 2023-10-06T05:23:55.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.430+0000 7f0699ffb700 1 -- 172.21.15.45:0/1732757620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694012b30 con 0x7f06a4153620 2023-10-06T05:23:55.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.431+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a414a710 con 0x7f06a4153620 2023-10-06T05:23:55.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.432+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a4088a60 con 0x7f06a4153620 2023-10-06T05:23:55.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.432+0000 7f0699ffb700 1 -- 172.21.15.45:0/1732757620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f069401e460 con 0x7f06a4153620 2023-10-06T05:23:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.433+0000 7f0699ffb700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0688066f30 0x7f06880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.433+0000 7f0699ffb700 1 -- 172.21.15.45:0/1732757620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f0694085150 con 0x7f06a4153620 2023-10-06T05:23:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.433+0000 7f06a9dd6700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0688066f30 0x7f06880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.434+0000 7f06a9dd6700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0688066f30 0x7f06880693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f06a4157310 tx=0x7f06a0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.437+0000 7f0699ffb700 1 -- 172.21.15.45:0/1732757620 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0694051c10 con 0x7f06a4153620 2023-10-06T05:23:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.691+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a416a7d0 con 0x7f06a4153620 2023-10-06T05:23:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.691+0000 7f0699ffb700 1 -- 172.21.15.45:0/1732757620 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+4599 (secure 0 0 0) 0x7f0694055230 con 0x7f06a4153620 2023-10-06T05:23:55.693 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:55.693 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":184,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3351321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3351321197},{"type":"v1","addr":"172.21.15.45:6835","nonce":3351321197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:52.452922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:55.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0688066f30 msgr2=0x7f06880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0688066f30 0x7f06880693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f06a4157310 tx=0x7f06a0017000 comp rx=0 tx=0).stop 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 msgr2=0x7f06a4152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a4152a50 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0694006b00 tx=0x7f0694008040 comp rx=0 tx=0).stop 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 shutdown_connections 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0688066f30 0x7f06880693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06a4149d60 0x7f06a414a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06a40cc8b0 0x7f06a416a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 --2- 172.21.15.45:0/1732757620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a4153620 0x7f06a4152a50 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 >> 172.21.15.45:0/1732757620 conn(0x7f06a406ed50 msgr2=0x7f06a41583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 shutdown_connections 2023-10-06T05:23:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.694+0000 7f06ac03a700 1 -- 172.21.15.45:0/1732757620 wait complete. 2023-10-06T05:23:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 184 2023-10-06T05:23:55.709 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:55.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.840+0000 7fe932566700 1 Processor -- start 2023-10-06T05:23:55.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.840+0000 7fe932566700 1 -- start start 2023-10-06T05:23:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.840+0000 7fe932566700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe92c0c8ff0 0x7fe92c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.840+0000 7fe932566700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c157200 0x7fe92c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.840+0000 7fe932566700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c15b000 con 0x7fe92c0caad0 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.840+0000 7fe932566700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c15b2e0 con 0x7fe92c0c8ff0 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.840+0000 7fe932566700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c15b5c0 con 0x7fe92c157200 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe92c0c8ff0 0x7fe92c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe92c0c8ff0 0x7fe92c0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52366/0 (socket says 172.21.15.45:52366) 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 -- 172.21.15.45:0/84931682 learned_addr learned my addr 172.21.15.45:0/84931682 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe930b03700 1 --2- 172.21.15.45:0/84931682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c157200 0x7fe92c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 -- 172.21.15.45:0/84931682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c157200 msgr2=0x7fe92c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 --2- 172.21.15.45:0/84931682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c157200 0x7fe92c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 -- 172.21.15.45:0/84931682 >> v1:172.21.15.144:6789/0 conn(0x7fe92c0caad0 legacy=0x7fe92c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 -- 172.21.15.45:0/84931682 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c15b8a0 con 0x7fe92c0c8ff0 2023-10-06T05:23:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92bfff700 1 --2- 172.21.15.45:0/84931682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c0c8ff0 0x7fe92c0c93c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0099b0 tx=0x7fe91c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3c3562f8b9763b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92affd700 1 -- 172.21.15.45:0/84931682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c009ea0 con 0x7fe92c0c8ff0 2023-10-06T05:23:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92affd700 1 -- 172.21.15.45:0/84931682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c00b860 con 0x7fe92c0c8ff0 2023-10-06T05:23:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.841+0000 7fe92affd700 1 -- 172.21.15.45:0/84931682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00bc00 con 0x7fe92c0c8ff0 2023-10-06T05:23:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 -- 172.21.15.45:0/84931682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c0c8ff0 msgr2=0x7fe92c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 --2- 172.21.15.45:0/84931682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c0c8ff0 0x7fe92c0c93c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0099b0 tx=0x7fe91c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 -- 172.21.15.45:0/84931682 shutdown_connections 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 --2- 172.21.15.45:0/84931682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c157200 0x7fe92c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 --2- 172.21.15.45:0/84931682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c0c8ff0 0x7fe92c0c93c0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 -- 172.21.15.45:0/84931682 >> 172.21.15.45:0/84931682 conn(0x7fe92c0cdf50 msgr2=0x7fe92c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 -- 172.21.15.45:0/84931682 shutdown_connections 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.842+0000 7fe932566700 1 -- 172.21.15.45:0/84931682 wait complete. 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.843+0000 7fe932566700 1 Processor -- start 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe932566700 1 -- start start 2023-10-06T05:23:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe932566700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c0c8ff0 0x7fe92c0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe932566700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe92c0caad0 0x7fe92c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe932566700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c157200 0x7fe92c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe932566700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b19e0 con 0x7fe92c157200 2023-10-06T05:23:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe932566700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b1c10 con 0x7fe92c0caad0 2023-10-06T05:23:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe932566700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b1ef0 con 0x7fe92c0c8ff0 2023-10-06T05:23:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe930b03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c157200 0x7fe92c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe930b03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c157200 0x7fe92c1594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52374/0 (socket says 172.21.15.45:52374) 2023-10-06T05:23:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe930b03700 1 -- 172.21.15.45:0/400657708 learned_addr learned my addr 172.21.15.45:0/400657708 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe92bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c0c8ff0 0x7fe92c0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.844+0000 7fe92b7fe700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe92c0caad0 0x7fe92c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe930b03700 1 -- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c0c8ff0 msgr2=0x7fe92c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe930b03700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c0c8ff0 0x7fe92c0b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe930b03700 1 -- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe92c0caad0 msgr2=0x7fe92c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe930b03700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe92c0caad0 0x7fe92c151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe930b03700 1 -- 172.21.15.45:0/400657708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c0094f0 con 0x7fe92c157200 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe930b03700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c157200 0x7fe92c1594b0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe924009df0 tx=0x7fe92400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe913fff700 1 -- 172.21.15.45:0/400657708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400caa0 con 0x7fe92c157200 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe913fff700 1 -- 172.21.15.45:0/400657708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92400f040 con 0x7fe92c157200 2023-10-06T05:23:55.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe913fff700 1 -- 172.21.15.45:0/400657708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400ce70 con 0x7fe92c157200 2023-10-06T05:23:55.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0b21d0 con 0x7fe92c157200 2023-10-06T05:23:55.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.845+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c159f70 con 0x7fe92c157200 2023-10-06T05:23:55.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.846+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c0b26f0 con 0x7fe92c157200 2023-10-06T05:23:55.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.847+0000 7fe913fff700 1 -- 172.21.15.45:0/400657708 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe924014690 con 0x7fe92c157200 2023-10-06T05:23:55.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.847+0000 7fe913fff700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe918066f30 0x7fe9180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.847+0000 7fe913fff700 1 -- 172.21.15.45:0/400657708 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fe9240854f0 con 0x7fe92c157200 2023-10-06T05:23:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.847+0000 7fe92bfff700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe918066f30 0x7fe9180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.848+0000 7fe92bfff700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe918066f30 0x7fe9180693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0099b0 tx=0x7fe91c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:55.852+0000 7fe913fff700 1 -- 172.21.15.45:0/400657708 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe92c0b26f0 con 0x7fe92c157200 2023-10-06T05:23:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.098+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe92c0b26f0 con 0x7fe92c157200 2023-10-06T05:23:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.098+0000 7fe913fff700 1 -- 172.21.15.45:0/400657708 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+4599 (secure 0 0 0) 0x7fe92c0b26f0 con 0x7fe92c157200 2023-10-06T05:23:56.100 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:56.101 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":184,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3351321197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3351321197},{"type":"v1","addr":"172.21.15.45:6835","nonce":3351321197}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:52.452922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe918066f30 msgr2=0x7fe9180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe918066f30 0x7fe9180693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0099b0 tx=0x7fe91c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c157200 msgr2=0x7fe92c1594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c157200 0x7fe92c1594b0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe924009df0 tx=0x7fe92400b040 comp rx=0 tx=0).stop 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 shutdown_connections 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe918066f30 0x7fe9180693e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe92c157200 0x7fe92c1594b0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe92c0c8ff0 0x7fe92c0b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 --2- 172.21.15.45:0/400657708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe92c0caad0 0x7fe92c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 >> 172.21.15.45:0/400657708 conn(0x7fe92c0cdf50 msgr2=0x7fe92c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 shutdown_connections 2023-10-06T05:23:56.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.101+0000 7fe932566700 1 -- 172.21.15.45:0/400657708 wait complete. 2023-10-06T05:23:56.104 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 184 2023-10-06T05:23:56.119 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2023-10-06T05:23:56.119 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2023-10-06T05:23:56.119 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:56.119 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:23:56.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:56.120 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:56.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:23:56.140 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:23:56.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:56.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.315+0000 7fd0c9ec2700 1 Processor -- start 2023-10-06T05:23:56.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.315+0000 7fd0c9ec2700 1 -- start start 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.316+0000 7fd0c9ec2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd0c4159330 0x7fd0c406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.316+0000 7fd0c9ec2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c40cb530 con 0x7fd0c40cc390 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.316+0000 7fd0c9ec2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c4069bd0 con 0x7fd0c4158f60 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.316+0000 7fd0c9ec2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c4068fc0 con 0x7fd0c4159330 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.316+0000 7fd0c3fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd0c40cc390 0x7fd0c40c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48752/0 (socket says 172.21.15.45:48752) 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.316+0000 7fd0c3fff700 1 -- 172.21.15.45:0/1338151930 learned_addr learned my addr 172.21.15.45:0/1338151930 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.316+0000 7fd0c2ffd700 1 --2- 172.21.15.45:0/1338151930 >> v2:172.21.15.144:3300/0 conn(0x7fd0c4159330 0x7fd0c406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c27fc700 1 -- 172.21.15.45:0/1338151930 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824364643 0 0) 0x7fd0bc001490 con 0x7fd0c40cc390 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c27fc700 1 -- 172.21.15.45:0/1338151930 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b0003570 con 0x7fd0c40cc390 2023-10-06T05:23:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c27fc700 1 -- 172.21.15.45:0/1338151930 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845097680 0 0) 0x7fd0c4069bd0 con 0x7fd0c4158f60 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c27fc700 1 -- 172.21.15.45:0/1338151930 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0bc001490 con 0x7fd0c4158f60 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c2ffd700 1 -- 172.21.15.45:0/1338151930 >> v1:172.21.15.144:6789/0 conn(0x7fd0c4158f60 legacy=0x7fd0c40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c2ffd700 1 -- 172.21.15.45:0/1338151930 >> v1:172.21.15.45:6789/0 conn(0x7fd0c40cc390 legacy=0x7fd0c40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c2ffd700 1 -- 172.21.15.45:0/1338151930 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c40692a0 con 0x7fd0c4159330 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c2ffd700 1 --2- 172.21.15.45:0/1338151930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c406a060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0098e0 tx=0x7fd0ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af6ae66c68ee526 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c27fc700 1 -- 172.21.15.45:0/1338151930 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac011630 con 0x7fd0c4159330 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c27fc700 1 -- 172.21.15.45:0/1338151930 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac011790 con 0x7fd0c4159330 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.317+0000 7fd0c27fc700 1 -- 172.21.15.45:0/1338151930 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac011b30 con 0x7fd0c4159330 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.318+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/1338151930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 msgr2=0x7fd0c406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.318+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/1338151930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c406a060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0098e0 tx=0x7fd0ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.318+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/1338151930 shutdown_connections 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.318+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/1338151930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c406a060 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.318+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/1338151930 >> 172.21.15.45:0/1338151930 conn(0x7fd0c40cdf50 msgr2=0x7fd0c40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.318+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/1338151930 shutdown_connections 2023-10-06T05:23:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.318+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/1338151930 wait complete. 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.319+0000 7fd0c9ec2700 1 Processor -- start 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.319+0000 7fd0c9ec2700 1 -- start start 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c9ec2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c40cc390 0x7fd0c415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c9ec2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c4158f60 0x7fd0c4154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c9ec2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c40c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c9ec2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c416caa0 con 0x7fd0c40cc390 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c9ec2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c415df20 con 0x7fd0c4159330 2023-10-06T05:23:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c9ec2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c415bf10 con 0x7fd0c4158f60 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c40cc390 0x7fd0c415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c40cc390 0x7fd0c415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52386/0 (socket says 172.21.15.45:52386) 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c37fe700 1 -- 172.21.15.45:0/3607608502 learned_addr learned my addr 172.21.15.45:0/3607608502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c3fff700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c40c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c2ffd700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c4158f60 0x7fd0c4154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c37fe700 1 -- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c4158f60 msgr2=0x7fd0c4154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c37fe700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c4158f60 0x7fd0c4154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c37fe700 1 -- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 msgr2=0x7fd0c40c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.320+0000 7fd0c37fe700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c40c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0c37fe700 1 -- 172.21.15.45:0/3607608502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac009420 con 0x7fd0c40cc390 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0c3fff700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c40c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:56.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0c37fe700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c40cc390 0x7fd0c415d9e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c790 tx=0x7fd0b4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0bb7fe700 1 -- 172.21.15.45:0/3607608502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4010510 con 0x7fd0c40cc390 2023-10-06T05:23:56.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0bb7fe700 1 -- 172.21.15.45:0/3607608502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4010670 con 0x7fd0c40cc390 2023-10-06T05:23:56.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c415c1f0 con 0x7fd0c40cc390 2023-10-06T05:23:56.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0bb7fe700 1 -- 172.21.15.45:0/3607608502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4010a10 con 0x7fd0c40cc390 2023-10-06T05:23:56.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.321+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c415c7b0 con 0x7fd0c40cc390 2023-10-06T05:23:56.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.322+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c406f410 con 0x7fd0c40cc390 2023-10-06T05:23:56.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.322+0000 7fd0bb7fe700 1 -- 172.21.15.45:0/3607608502 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0b401b400 con 0x7fd0c40cc390 2023-10-06T05:23:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.323+0000 7fd0bb7fe700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a4066f30 0x7fd0a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.323+0000 7fd0bb7fe700 1 -- 172.21.15.45:0/3607608502 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fd0b4084b50 con 0x7fd0c40cc390 2023-10-06T05:23:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.323+0000 7fd0c2ffd700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a4066f30 0x7fd0a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.324+0000 7fd0c2ffd700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a4066f30 0x7fd0a40693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0091e0 tx=0x7fd0ac017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.327+0000 7fd0bb7fe700 1 -- 172.21.15.45:0/3607608502 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0b4051570 con 0x7fd0c40cc390 2023-10-06T05:23:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:56.580+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0c404bb40 con 0x7fd0c40cc390 2023-10-06T05:23:57.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.479+0000 7fd0bb7fe700 1 -- 172.21.15.45:0/3607608502 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v185) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd0b4054b90 con 0x7fd0c40cc390 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a4066f30 msgr2=0x7fd0a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a4066f30 0x7fd0a40693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0091e0 tx=0x7fd0ac017000 comp rx=0 tx=0).stop 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c40cc390 msgr2=0x7fd0c415d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c40cc390 0x7fd0c415d9e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c790 tx=0x7fd0b4007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 shutdown_connections 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a4066f30 0x7fd0a40693e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4159330 0x7fd0c40c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c40cc390 0x7fd0c415d9e0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 --2- 172.21.15.45:0/3607608502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c4158f60 0x7fd0c4154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 >> 172.21.15.45:0/3607608502 conn(0x7fd0c40cdf50 msgr2=0x7fd0c4157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 shutdown_connections 2023-10-06T05:23:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.482+0000 7fd0c9ec2700 1 -- 172.21.15.45:0/3607608502 wait complete. 2023-10-06T05:23:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:57.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:23:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.633+0000 7fe844473700 1 Processor -- start 2023-10-06T05:23:57.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.634+0000 7fe844473700 1 -- start start 2023-10-06T05:23:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe844473700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe83c0c8ff0 0x7fe83c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe844473700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c15b000 con 0x7fe83c0caad0 2023-10-06T05:23:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe844473700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c15b2e0 con 0x7fe83c157200 2023-10-06T05:23:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe844473700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c15b5c0 con 0x7fe83c0c8ff0 2023-10-06T05:23:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84220f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe83c0c8ff0 0x7fe83c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84220f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe83c0c8ff0 0x7fe83c0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38722/0 (socket says 172.21.15.45:38722) 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84220f700 1 -- 172.21.15.45:0/18181851 learned_addr learned my addr 172.21.15.45:0/18181851 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe841a0e700 1 --1- 172.21.15.45:0/18181851 >> v1:172.21.15.45:6789/0 conn(0x7fe83c0caad0 0x7fe83c156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42294/0 (socket says 172.21.15.45:42294) 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84220f700 1 -- 172.21.15.45:0/18181851 >> v1:172.21.15.144:6789/0 conn(0x7fe83c157200 legacy=0x7fe83c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84220f700 1 -- 172.21.15.45:0/18181851 >> v1:172.21.15.45:6789/0 conn(0x7fe83c0caad0 legacy=0x7fe83c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84220f700 1 -- 172.21.15.45:0/18181851 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c15b8a0 con 0x7fe83c0c8ff0 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84120d700 1 -- 172.21.15.45:0/18181851 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293523173 0 0) 0x7fe83c15b000 con 0x7fe83c0caad0 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.635+0000 7fe84220f700 1 --2- 172.21.15.45:0/18181851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c0c8ff0 0x7fe83c0c93c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe8380099b0 tx=0x7fe838009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3a8581073f0aa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe84120d700 1 -- 172.21.15.45:0/18181851 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83800b820 con 0x7fe83c0c8ff0 2023-10-06T05:23:57.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe84120d700 1 -- 172.21.15.45:0/18181851 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83800b980 con 0x7fe83c0c8ff0 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe84120d700 1 -- 172.21.15.45:0/18181851 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83800bd20 con 0x7fe83c0c8ff0 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe844473700 1 -- 172.21.15.45:0/18181851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c0c8ff0 msgr2=0x7fe83c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe844473700 1 --2- 172.21.15.45:0/18181851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c0c8ff0 0x7fe83c0c93c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe8380099b0 tx=0x7fe838009250 comp rx=0 tx=0).stop 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe844473700 1 -- 172.21.15.45:0/18181851 shutdown_connections 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe844473700 1 --2- 172.21.15.45:0/18181851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c0c8ff0 0x7fe83c0c93c0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe844473700 1 -- 172.21.15.45:0/18181851 >> 172.21.15.45:0/18181851 conn(0x7fe83c0cdf50 msgr2=0x7fe83c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe844473700 1 -- 172.21.15.45:0/18181851 shutdown_connections 2023-10-06T05:23:57.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.636+0000 7fe844473700 1 -- 172.21.15.45:0/18181851 wait complete. 2023-10-06T05:23:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.637+0000 7fe844473700 1 Processor -- start 2023-10-06T05:23:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe844473700 1 -- start start 2023-10-06T05:23:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe844473700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0c8ff0 0x7fe83c0b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe844473700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe83c0caad0 0x7fe83c0c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe844473700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c157200 0x7fe83c1672c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe844473700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c158130 con 0x7fe83c157200 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe844473700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c5f80 con 0x7fe83c0c8ff0 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe844473700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c6260 con 0x7fe83c0caad0 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe842a10700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c157200 0x7fe83c1672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe842a10700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c157200 0x7fe83c1672c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38724/0 (socket says 172.21.15.45:38724) 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.638+0000 7fe842a10700 1 -- 172.21.15.45:0/3446561964 learned_addr learned my addr 172.21.15.45:0/3446561964 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe841a0e700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe83c0caad0 0x7fe83c0c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe84220f700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0c8ff0 0x7fe83c0b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe842a10700 1 -- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe83c0caad0 msgr2=0x7fe83c0c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe842a10700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe83c0caad0 0x7fe83c0c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe842a10700 1 -- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0c8ff0 msgr2=0x7fe83c0b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe842a10700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0c8ff0 0x7fe83c0b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe842a10700 1 -- 172.21.15.45:0/3446561964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8380094f0 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe842a10700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c157200 0x7fe83c1672c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe83400bc30 tx=0x7fe834009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe8327fc700 1 -- 172.21.15.45:0/3446561964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400c840 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe8327fc700 1 -- 172.21.15.45:0/3446561964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83400c9e0 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c0c6540 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe8327fc700 1 -- 172.21.15.45:0/3446561964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400cda0 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.639+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c0b3320 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.640+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c1582d0 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.641+0000 7fe8327fc700 1 -- 172.21.15.45:0/3446561964 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe834014620 con 0x7fe83c157200 2023-10-06T05:23:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.641+0000 7fe8327fc700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe820066f30 0x7fe8200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.641+0000 7fe8327fc700 1 -- 172.21.15.45:0/3446561964 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fe834084c20 con 0x7fe83c157200 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.641+0000 7fe84220f700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe820066f30 0x7fe8200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.642+0000 7fe84220f700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe820066f30 0x7fe8200693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe8380099b0 tx=0x7fe838016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.646+0000 7fe8327fc700 1 -- 172.21.15.45:0/3446561964 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe83404c5a0 con 0x7fe83c157200 2023-10-06T05:23:57.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:57.931+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fe83c151a80 con 0x7fe83c157200 2023-10-06T05:23:58.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.486+0000 7fe8327fc700 1 -- 172.21.15.45:0/3446561964 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v186) v1 ==== 101+0+0 (secure 0 0 0) 0x7fe8340516e0 con 0x7fe83c157200 2023-10-06T05:23:58.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe820066f30 msgr2=0x7fe8200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe820066f30 0x7fe8200693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe8380099b0 tx=0x7fe838016000 comp rx=0 tx=0).stop 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c157200 msgr2=0x7fe83c1672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c157200 0x7fe83c1672c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe83400bc30 tx=0x7fe834009820 comp rx=0 tx=0).stop 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 shutdown_connections 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe820066f30 0x7fe8200693e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe83c157200 0x7fe83c1672c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0c8ff0 0x7fe83c0b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.489+0000 7fe844473700 1 --2- 172.21.15.45:0/3446561964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe83c0caad0 0x7fe83c0c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.490+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 >> 172.21.15.45:0/3446561964 conn(0x7fe83c0cdf50 msgr2=0x7fe83c150470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.490+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 shutdown_connections 2023-10-06T05:23:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.490+0000 7fe844473700 1 -- 172.21.15.45:0/3446561964 wait complete. 2023-10-06T05:23:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:23:58.509 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:23:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.555+0000 7fa08020d040 1 Processor -- start 2023-10-06T05:23:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.555+0000 7fa08020d040 1 -- start start 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.555+0000 7fa08020d040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caa727e0 0x55b0cab050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.555+0000 7fa08020d040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caafc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.555+0000 7fa08020d040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caafdf00 0x55b0cab03ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa08020d040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ca975980 con 0x55b0caafdb30 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa08020d040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ca975620 con 0x55b0caafdf00 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa08020d040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0ca96c400 con 0x55b0caa727e0 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa078c2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caafc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa07942b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caa727e0 0x55b0cab050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa078c2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caafc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38738/0 (socket says 172.21.15.45:38738) 2023-10-06T05:23:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa078c2a700 1 -- 172.21.15.45:0/1089991841 learned_addr learned my addr 172.21.15.45:0/1089991841 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa079c2c700 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caafdf00 0x55b0cab03ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa07942b700 1 -- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caafdf00 msgr2=0x55b0cab03ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa07942b700 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caafdf00 0x55b0cab03ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa07942b700 1 -- 172.21.15.45:0/1089991841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 msgr2=0x55b0caafc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa07942b700 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caafc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa07942b700 1 -- 172.21.15.45:0/1089991841 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c005cf0 con 0x55b0caa727e0 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa078c2a700 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caafc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.556+0000 7fa07942b700 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caa727e0 0x55b0cab050c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009a80 tx=0x7fa06c00c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19be37e3c7962ff7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa06b7fe700 1 -- 172.21.15.45:0/1089991841 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00f040 con 0x55b0caa727e0 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa06b7fe700 1 -- 172.21.15.45:0/1089991841 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c008b50 con 0x55b0caa727e0 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa06b7fe700 1 -- 172.21.15.45:0/1089991841 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c014480 con 0x55b0caa727e0 2023-10-06T05:23:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 -- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caa727e0 msgr2=0x55b0cab050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caa727e0 0x55b0cab050c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009a80 tx=0x7fa06c00c040 comp rx=0 tx=0).stop 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 -- 172.21.15.45:0/1089991841 shutdown_connections 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caafdf00 0x55b0cab03ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caa727e0 0x55b0cab050c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 --2- 172.21.15.45:0/1089991841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caafc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 -- 172.21.15.45:0/1089991841 >> 172.21.15.45:0/1089991841 conn(0x55b0caa2b450 msgr2=0x55b0caa59760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 -- 172.21.15.45:0/1089991841 shutdown_connections 2023-10-06T05:23:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.557+0000 7fa08020d040 1 -- 172.21.15.45:0/1089991841 wait complete. 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.558+0000 7fa08020d040 1 Processor -- start 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.558+0000 7fa08020d040 1 -- start start 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa08020d040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caa727e0 0x55b0caaf4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa08020d040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caaf8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa08020d040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caafdf00 0x55b0caaf5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa08020d040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0caaf5010 con 0x55b0caafdb30 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa08020d040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0caaf8db0 con 0x55b0caa727e0 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa08020d040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0caaf9050 con 0x55b0caafdf00 2023-10-06T05:23:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caaf8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caaf8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38752/0 (socket says 172.21.15.45:38752) 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 learned_addr learned my addr 172.21.15.45:0/3645464703 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa079c2c700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caafdf00 0x55b0caaf5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa07942b700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caa727e0 0x55b0caaf4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caafdf00 msgr2=0x55b0caaf5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0caafdf00 0x55b0caaf5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caa727e0 msgr2=0x55b0caaf4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0caa727e0 0x55b0caaf4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0ca99e1c0 con 0x55b0caafdb30 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa078c2a700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caaf8550 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x55b0caa2eb30 tx=0x7fa070016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa069ffb700 1 -- 172.21.15.45:0/3645464703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070000bc0 con 0x55b0caafdb30 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa069ffb700 1 -- 172.21.15.45:0/3645464703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070021070 con 0x55b0caafdb30 2023-10-06T05:23:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.559+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c009650 con 0x55b0caafdb30 2023-10-06T05:23:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.560+0000 7fa069ffb700 1 -- 172.21.15.45:0/3645464703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070000d20 con 0x55b0caafdb30 2023-10-06T05:23:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.560+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa070005db0 con 0x55b0caafdb30 2023-10-06T05:23:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.561+0000 7fa069ffb700 1 -- 172.21.15.45:0/3645464703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa070023030 con 0x55b0caafdb30 2023-10-06T05:23:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.561+0000 7fa069ffb700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa05c065890 0x7fa05c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.562+0000 7fa069ffb700 1 -- 172.21.15.45:0/3645464703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fa070085b00 con 0x55b0caafdb30 2023-10-06T05:23:58.566 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:23:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.562+0000 7fa07942b700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa05c065890 0x7fa05c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.562+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b0caaf61d0 0x55b0caafa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.562+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0caaffaf0 con 0x55b0caaf61d0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.562+0000 7fa079c2c700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b0caaf61d0 0x55b0caafa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.562+0000 7fa079c2c700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b0caaf61d0 0x55b0caafa060 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.562+0000 7fa07942b700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa05c065890 0x7fa05c067d40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009e90 tx=0x7fa06c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.564+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v107'443 uv417 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.564+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0caafa5a0 con 0x55b0caaf61d0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.565+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v107'444 uv444 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.565+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1bf00 con 0x55b0caaf61d0 2023-10-06T05:23:58.569 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:23:58.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.567+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v107'445 uv445 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.567+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1d2c0 con 0x55b0caaf61d0 2023-10-06T05:23:58.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.568+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'446 uv446 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.568+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1d670 con 0x55b0caaf61d0 2023-10-06T05:23:58.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.569+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'447 uv447 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.569+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1c2b0 con 0x55b0caaf61d0 2023-10-06T05:23:58.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.570+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'448 uv448 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.570+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1c6b0 con 0x55b0caaf61d0 2023-10-06T05:23:58.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.571+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v107'449 uv423 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.572+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.573+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v107'450 uv450 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.573+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.575 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:23:58.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.574+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v107'451 uv451 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.574+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.575+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'452 uv452 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.575+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.576+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'453 uv453 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.576+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cee0 con 0x55b0caaf61d0 2023-10-06T05:23:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.577+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'454 uv454 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.577+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cee0 con 0x55b0caaf61d0 2023-10-06T05:23:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.579+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v107'455 uv430 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.579+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.580+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'456 uv456 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.580+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.581+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v107'457 uv457 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.583 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:23:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.581+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.582+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'458 uv458 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.582+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cab0 con 0x55b0caaf61d0 2023-10-06T05:23:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.583+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'459 uv459 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.583+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1cee0 con 0x55b0caaf61d0 2023-10-06T05:23:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.584+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'460 uv460 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.584+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.586+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'461 uv461 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.586+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.587+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v107'462 uv436 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.587+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.588+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v107'463 uv463 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.588+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.591 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:23:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.589+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v107'464 uv464 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.589+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1ea30 con 0x55b0caaf61d0 2023-10-06T05:23:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.591+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'465 uv465 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.591+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1ea30 con 0x55b0caaf61d0 2023-10-06T05:23:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.592+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'466 uv466 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.592+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.593+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'467 uv467 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.593+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.594+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v107'468 uv442 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.594+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.596+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v107'469 uv469 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.596+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.598 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:23:58.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.597+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v107'470 uv470 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.597+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1ea30 con 0x55b0caaf61d0 2023-10-06T05:23:58.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.598+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'471 uv471 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.598+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1ea30 con 0x55b0caaf61d0 2023-10-06T05:23:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.599+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'472 uv472 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.599+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab1da20 con 0x55b0caaf61d0 2023-10-06T05:23:58.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.600+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'473 uv473 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0caaf61d0 2023-10-06T05:23:58.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.600+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b0cab1df30 0x55b0cab20310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.601+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab20850 con 0x55b0cab1df30 2023-10-06T05:23:58.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.601+0000 7fa07942b700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b0cab1df30 0x55b0cab20310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.601+0000 7fa07942b700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b0cab1df30 0x55b0cab20310 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.603+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'326 uv308 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.603+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab20c00 con 0x55b0cab1df30 2023-10-06T05:23:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.604+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'327 uv327 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.604+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab20fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.605+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'328 uv328 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.607 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:23:58.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.605+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21850 con 0x55b0cab1df30 2023-10-06T05:23:58.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.607+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v107'329 uv329 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.607+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21c00 con 0x55b0cab1df30 2023-10-06T05:23:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.608+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'330 uv330 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.608+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.609+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v107'331 uv313 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.609+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.611+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v107'332 uv332 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.611+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.612+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v107'333 uv333 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.614 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:23:58.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.612+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.613+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'334 uv334 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.613+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.615+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'335 uv335 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.615+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.616+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v107'336 uv318 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.616+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.617+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v107'337 uv337 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.617+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.618+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v107'338 uv338 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.620 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:23:58.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.618+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab256d0 con 0x55b0cab1df30 2023-10-06T05:23:58.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.620+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v107'339 uv339 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.620+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab256d0 con 0x55b0cab1df30 2023-10-06T05:23:58.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.621+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'340 uv340 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.621+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.622+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v107'341 uv324 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.623+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.624+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v107'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.624+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.625+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v107'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.627 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:23:58.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.625+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab22fc0 con 0x55b0cab1df30 2023-10-06T05:23:58.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.626+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'344 uv344 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.626+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab22fc0 con 0x55b0cab1df30 2023-10-06T05:23:58.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.628+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'345 uv345 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.628+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab21fb0 con 0x55b0cab1df30 2023-10-06T05:23:58.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.629+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'346 uv346 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.629+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b0cab224c0 0x55b0cab248a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.629+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab24de0 con 0x55b0cab224c0 2023-10-06T05:23:58.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.630+0000 7fa078c2a700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b0cab224c0 0x55b0cab248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.630+0000 7fa078c2a700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b0cab224c0 0x55b0cab248a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.631+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.631+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25190 con 0x55b0cab224c0 2023-10-06T05:23:58.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.632+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v107'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.632+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25540 con 0x55b0cab224c0 2023-10-06T05:23:58.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.634+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v107'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.635 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:23:58.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.634+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab26900 con 0x55b0cab224c0 2023-10-06T05:23:58.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.635+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.635+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab26cb0 con 0x55b0cab224c0 2023-10-06T05:23:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.636+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.636+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab258f0 con 0x55b0cab224c0 2023-10-06T05:23:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.637+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.637+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25cf0 con 0x55b0cab224c0 2023-10-06T05:23:58.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.638+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'64 uv63 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.638+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab26110 con 0x55b0cab224c0 2023-10-06T05:23:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.640+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'65 uv65 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.640+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab26530 con 0x55b0cab224c0 2023-10-06T05:23:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.641+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'66 uv66 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.643 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:23:58.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.641+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab26530 con 0x55b0cab224c0 2023-10-06T05:23:58.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.642+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'67 uv67 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.642+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25e90 con 0x55b0cab1df30 2023-10-06T05:23:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.644+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.644+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25e90 con 0x55b0cab1df30 2023-10-06T05:23:58.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.645+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v107'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.645+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25e90 con 0x55b0cab1df30 2023-10-06T05:23:58.648 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:23:58.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.646+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v107'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.646+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25e90 con 0x55b0cab1df30 2023-10-06T05:23:58.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.647+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.647+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25e90 con 0x55b0cab1df30 2023-10-06T05:23:58.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.648+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.648+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab25e90 con 0x55b0cab1df30 2023-10-06T05:23:58.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.649+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab1df30 2023-10-06T05:23:58.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.650+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b0cab270f0 0x55b0cab294d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.650+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.650+0000 7fa079c2c700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b0cab270f0 0x55b0cab294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.650+0000 7fa079c2c700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b0cab270f0 0x55b0cab294d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.652+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v107'160 uv154 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.652+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.653+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v107'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.653+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.655 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:23:58.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.654+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v107'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.654+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2aa20 con 0x55b0cab270f0 2023-10-06T05:23:58.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.655+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.655+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2aa20 con 0x55b0cab270f0 2023-10-06T05:23:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.656+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.656+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.657+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.657+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.659+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v107'166 uv159 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.659+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.660+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v107'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.660+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.661+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v107'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.663 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:23:58.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.661+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a10 con 0x55b0cab270f0 2023-10-06T05:23:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.663+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.663+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab29a90 con 0x55b0cab270f0 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.664+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.664+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b0cab2a110 0x55b0cab2a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.664+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2ce30 con 0x55b0cab2a110 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.664+0000 7fa07942b700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b0cab2a110 0x55b0cab2a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.665+0000 7fa07942b700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b0cab2a110 0x55b0cab2a520 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.666+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v107'71 uv70 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab2a110 2023-10-06T05:23:58.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.666+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2ce30 con 0x55b0cab2a110 2023-10-06T05:23:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.668+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v107'72 uv72 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab2a110 2023-10-06T05:23:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.668+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2ce30 con 0x55b0cab2a110 2023-10-06T05:23:58.670 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:23:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.669+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v107'73 uv73 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab2a110 2023-10-06T05:23:58.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.669+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2ce90 con 0x55b0cab2a110 2023-10-06T05:23:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.670+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'74 uv74 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab2a110 2023-10-06T05:23:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.670+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2d2b0 con 0x55b0cab2a110 2023-10-06T05:23:58.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.671+0000 7fa07942b700 1 -- 172.21.15.45:0/3645464703 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'75 uv75 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa06c01b040 con 0x55b0cab2a110 2023-10-06T05:23:58.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.672+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2d2b0 con 0x55b0cab224c0 2023-10-06T05:23:58.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.673+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.673+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2d2b0 con 0x55b0cab224c0 2023-10-06T05:23:58.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.674+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v107'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.674+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2d2b0 con 0x55b0cab224c0 2023-10-06T05:23:58.677 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:23:58.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.675+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v107'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.675+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2e2c0 con 0x55b0cab224c0 2023-10-06T05:23:58.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.677+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.677+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2e2c0 con 0x55b0cab224c0 2023-10-06T05:23:58.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.678+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.678+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab2d2b0 con 0x55b0cab224c0 2023-10-06T05:23:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.679+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.679+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b0cab2d7c0 0x55b0cab2fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.679+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30130 con 0x55b0cab2d7c0 2023-10-06T05:23:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.679+0000 7fa078c2a700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b0cab2d7c0 0x55b0cab2fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.680+0000 7fa078c2a700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b0cab2d7c0 0x55b0cab2fba0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.681+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v107'142 uv136 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.681+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30130 con 0x55b0cab2d7c0 2023-10-06T05:23:58.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.682+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v107'143 uv143 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.682+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30560 con 0x55b0cab2d7c0 2023-10-06T05:23:58.685 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:23:58.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.683+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v107'144 uv144 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.683+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30560 con 0x55b0cab2d7c0 2023-10-06T05:23:58.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.684+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'145 uv145 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.684+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30560 con 0x55b0cab2d7c0 2023-10-06T05:23:58.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.686+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'146 uv146 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.686+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30560 con 0x55b0cab2d7c0 2023-10-06T05:23:58.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.687+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v107'147 uv141 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.687+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30560 con 0x55b0cab2d7c0 2023-10-06T05:23:58.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.688+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v107'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.688+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30560 con 0x55b0cab2d7c0 2023-10-06T05:23:58.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.688+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v107'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.690 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:23:58.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.689+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30560 con 0x55b0cab2d7c0 2023-10-06T05:23:58.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.689+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v107'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.690+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab332d0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.691+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v107'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab2d7c0 2023-10-06T05:23:58.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.691+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30a50 con 0x55b0cab224c0 2023-10-06T05:23:58.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.692+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v107'66 uv64 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.692+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30a50 con 0x55b0cab224c0 2023-10-06T05:23:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.693+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v107'67 uv67 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.693+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30a50 con 0x55b0cab224c0 2023-10-06T05:23:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.694+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v107'68 uv68 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.696 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:23:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.694+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab30a50 con 0x55b0cab224c0 2023-10-06T05:23:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.695+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v107'69 uv69 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.695+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b0cab30f60 0x55b0cab31350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.695+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab30f60 2023-10-06T05:23:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.696+0000 7fa079c2c700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b0cab30f60 0x55b0cab31350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.696+0000 7fa079c2c700 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b0cab30f60 0x55b0cab31350 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.697+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab30f60 2023-10-06T05:23:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.697+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab30f60 2023-10-06T05:23:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.698+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v107'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab30f60 2023-10-06T05:23:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.698+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab30f60 2023-10-06T05:23:58.701 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:23:58.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.699+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v107'89 uv89 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab30f60 2023-10-06T05:23:58.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.700+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab34a30 con 0x55b0cab30f60 2023-10-06T05:23:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.702+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab30f60 2023-10-06T05:23:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.702+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab34a30 con 0x55b0cab30f60 2023-10-06T05:23:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.703+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab30f60 2023-10-06T05:23:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.703+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab30f60 2023-10-06T05:23:58.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.704+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab30f60 2023-10-06T05:23:58.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.705+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab270f0 2023-10-06T05:23:58.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.706+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v107'74 uv73 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.706+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab270f0 2023-10-06T05:23:58.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.707+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v107'75 uv75 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.707+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab270f0 2023-10-06T05:23:58.710 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:23:58.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.708+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v107'76 uv76 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.708+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab270f0 2023-10-06T05:23:58.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.710+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v107'77 uv77 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.710+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab270f0 2023-10-06T05:23:58.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.711+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v107'78 uv78 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa0740027f0 con 0x55b0cab270f0 2023-10-06T05:23:58.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.711+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab224c0 2023-10-06T05:23:58.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.712+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v107'182 uv172 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.712+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab224c0 2023-10-06T05:23:58.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.713+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v107'183 uv183 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.713+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab224c0 2023-10-06T05:23:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:23:58.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.714+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v107'184 uv184 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.714+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab224c0 2023-10-06T05:23:58.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.716+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v107'185 uv185 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.716+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab305e0 con 0x55b0cab224c0 2023-10-06T05:23:58.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.717+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v107'186 uv186 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.717+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab305e0 con 0x55b0cab224c0 2023-10-06T05:23:58.719 INFO:teuthology.orchestra.run.smithi045.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:23:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.718+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v107'187 uv187 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.718+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab34a30 con 0x55b0cab224c0 2023-10-06T05:23:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.719+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v107'188 uv188 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.719+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab34a30 con 0x55b0cab224c0 2023-10-06T05:23:58.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.720+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v107'189 uv189 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.720+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33a20 con 0x55b0cab224c0 2023-10-06T05:23:58.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.721+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [omap-set-header] v107'190 uv190 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.721+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x55b0cab33f10 con 0x55b0cab224c0 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa078c2a700 1 -- 172.21.15.45:0/3645464703 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v107'191 uv191 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa0700838e0 con 0x55b0cab224c0 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b0cab224c0 msgr2=0x55b0cab248a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b0cab224c0 0x55b0cab248a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b0caaf61d0 msgr2=0x55b0caafa060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b0caaf61d0 0x55b0caafa060 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b0cab2d7c0 msgr2=0x55b0cab2fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b0cab2d7c0 0x55b0cab2fba0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b0cab1df30 msgr2=0x55b0cab20310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b0cab1df30 0x55b0cab20310 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b0cab30f60 msgr2=0x55b0cab31350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b0cab30f60 0x55b0cab31350 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b0cab2a110 msgr2=0x55b0cab2a520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b0cab2a110 0x55b0cab2a520 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 reap_dead start 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b0cab270f0 msgr2=0x55b0cab294d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b0cab270f0 0x55b0cab294d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 reap_dead start 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 reap_dead start 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa079c2c700 1 -- 172.21.15.45:0/3645464703 reap_dead start 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa05c065890 msgr2=0x7fa05c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa05c065890 0x7fa05c067d40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009e90 tx=0x7fa06c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 msgr2=0x55b0caaf8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caaf8550 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x55b0caa2eb30 tx=0x7fa070016000 comp rx=0 tx=0).stop 2023-10-06T05:23:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 shutdown_connections 2023-10-06T05:23:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0caafdb30 0x55b0caaf8550 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 --2- 172.21.15.45:0/3645464703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa05c065890 0x7fa05c067d40 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 >> 172.21.15.45:0/3645464703 conn(0x55b0caa2b450 msgr2=0x55b0caa58f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 shutdown_connections 2023-10-06T05:23:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.723+0000 7fa08020d040 1 -- 172.21.15.45:0/3645464703 wait complete. 2023-10-06T05:23:58.729 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2023-10-06T05:23:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.777+0000 7fa3efa18040 1 Processor -- start 2023-10-06T05:23:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.777+0000 7fa3efa18040 1 -- start start 2023-10-06T05:23:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3efa18040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0d5e7e0 0x5609d0debe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3efa18040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 0x5609d0de6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3efa18040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0de9f00 0x5609d0d14480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3efa18040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d0c61980 con 0x5609d0d5e7e0 2023-10-06T05:23:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3efa18040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d0c61620 con 0x5609d0de9b30 2023-10-06T05:23:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3efa18040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609d0c58400 con 0x5609d0de9f00 2023-10-06T05:23:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0d5e7e0 0x5609d0debe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0d5e7e0 0x5609d0debe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38762/0 (socket says 172.21.15.45:38762) 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 -- 172.21.15.45:0/59121958 learned_addr learned my addr 172.21.15.45:0/59121958 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8435700 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 0x5609d0de6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e9437700 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0de9f00 0x5609d0d14480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 -- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0de9f00 msgr2=0x5609d0d14480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0de9f00 0x5609d0d14480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 -- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 msgr2=0x5609d0de6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 0x5609d0de6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 -- 172.21.15.45:0/59121958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e000a0e0 con 0x5609d0d5e7e0 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.778+0000 7fa3e8c36700 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0d5e7e0 0x5609d0debe10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800acf0 tx=0x7fa3d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bde8fdc52215e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3e7433700 1 -- 172.21.15.45:0/59121958 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8000bc0 con 0x5609d0d5e7e0 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3e7433700 1 -- 172.21.15.45:0/59121958 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d8008970 con 0x5609d0d5e7e0 2023-10-06T05:23:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3e7433700 1 -- 172.21.15.45:0/59121958 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8000d20 con 0x5609d0d5e7e0 2023-10-06T05:23:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 -- 172.21.15.45:0/59121958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0d5e7e0 msgr2=0x5609d0debe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0d5e7e0 0x5609d0debe10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800acf0 tx=0x7fa3d8016000 comp rx=0 tx=0).stop 2023-10-06T05:23:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 -- 172.21.15.45:0/59121958 shutdown_connections 2023-10-06T05:23:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0de9f00 0x5609d0d14480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0d5e7e0 0x5609d0debe10 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 --2- 172.21.15.45:0/59121958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 0x5609d0de6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 -- 172.21.15.45:0/59121958 >> 172.21.15.45:0/59121958 conn(0x5609d0d17450 msgr2=0x5609d0d19830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 -- 172.21.15.45:0/59121958 shutdown_connections 2023-10-06T05:23:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.779+0000 7fa3efa18040 1 -- 172.21.15.45:0/59121958 wait complete. 2023-10-06T05:23:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.780+0000 7fa3efa18040 1 Processor -- start 2023-10-06T05:23:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.780+0000 7fa3efa18040 1 -- start start 2023-10-06T05:23:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3efa18040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0d5e7e0 0x5609d0d56140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3efa18040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 0x5609d0d56680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3efa18040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0de9f00 0x5609d0d56bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3efa18040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d0d57570 con 0x5609d0de9f00 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3efa18040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609d0dee210 con 0x5609d0de9b30 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3efa18040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609d0d57810 con 0x5609d0d5e7e0 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0de9f00 0x5609d0d56bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0de9f00 0x5609d0d56bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38772/0 (socket says 172.21.15.45:38772) 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 -- 172.21.15.45:0/789814215 learned_addr learned my addr 172.21.15.45:0/789814215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e8c36700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0d5e7e0 0x5609d0d56140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e8435700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 0x5609d0d56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 -- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0d5e7e0 msgr2=0x5609d0d56140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d0d5e7e0 0x5609d0d56140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 -- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 msgr2=0x5609d0d56680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d0de9b30 0x5609d0d56680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 -- 172.21.15.45:0/789814215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609d0c8a1c0 con 0x5609d0de9f00 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.781+0000 7fa3e9437700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0de9f00 0x5609d0d56bc0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000afd0 tx=0x7fa3e0007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.782+0000 7fa3e5c30700 1 -- 172.21.15.45:0/789814215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0012680 con 0x5609d0de9f00 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.782+0000 7fa3e5c30700 1 -- 172.21.15.45:0/789814215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e00127e0 con 0x5609d0de9f00 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.782+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d8005cf0 con 0x5609d0de9f00 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.782+0000 7fa3e5c30700 1 -- 172.21.15.45:0/789814215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0012b80 con 0x5609d0de9f00 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.782+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609d0c703d0 con 0x5609d0de9f00 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.783+0000 7fa3e5c30700 1 -- 172.21.15.45:0/789814215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3e0013440 con 0x5609d0de9f00 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.784+0000 7fa3e5c30700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3d4065890 0x7fa3d4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.784+0000 7fa3e5c30700 1 -- 172.21.15.45:0/789814215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fa3e0084160 con 0x5609d0de9f00 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.784+0000 7fa3e8c36700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3d4065890 0x7fa3d4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.784+0000 7fa3efa18040 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5609d0de59d0 0x5609d0def0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.784+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) v8 -- 0x5609d0d5d2c0 con 0x5609d0de59d0 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.784+0000 7fa3e8435700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5609d0de59d0 0x5609d0def0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.784+0000 7fa3e8c36700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3d4065890 0x7fa3d4067d40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800a520 tx=0x7fa3d801e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.785+0000 7fa3e8435700 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5609d0de59d0 0x5609d0def0b0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3e8435700 1 -- 172.21.15.45:0/789814215 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v107'347 uv347 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa3dc00f8f0 con 0x5609d0de59d0 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5609d0de59d0 msgr2=0x5609d0def0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5609d0de59d0 0x5609d0def0b0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3d4065890 msgr2=0x7fa3d4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa3d4065890 0x7fa3d4067d40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800a520 tx=0x7fa3d801e000 comp rx=0 tx=0).stop 2023-10-06T05:23:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0de9f00 msgr2=0x5609d0d56bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 --2- 172.21.15.45:0/789814215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609d0de9f00 0x5609d0d56bc0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000afd0 tx=0x7fa3e0007040 comp rx=0 tx=0).stop 2023-10-06T05:23:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3e9437700 1 -- 172.21.15.45:0/789814215 reap_dead start 2023-10-06T05:23:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 shutdown_connections 2023-10-06T05:23:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 >> 172.21.15.45:0/789814215 conn(0x5609d0d17450 msgr2=0x5609d0debae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.786+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 shutdown_connections 2023-10-06T05:23:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.787+0000 7fa3efa18040 1 -- 172.21.15.45:0/789814215 wait complete. 2023-10-06T05:23:58.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:23:58.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.975+0000 7fce0f6d2700 1 Processor -- start 2023-10-06T05:23:58.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.976+0000 7fce0f6d2700 1 -- start start 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0f6d2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce08159150 0x7fce080ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0f6d2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159520 0x7fce0815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0f6d2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fce0815d3d0 0x7fce08067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0f6d2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce080c9c90 con 0x7fce08159150 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0f6d2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0815d8e0 con 0x7fce0815d3d0 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0f6d2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0815dbc0 con 0x7fce08159520 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0d46e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce08159150 0x7fce080ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0cc6d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159520 0x7fce0815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0dc6f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fce0815d3d0 0x7fce08067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0d46e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce08159150 0x7fce080ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38788/0 (socket says 172.21.15.45:38788) 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0d46e700 1 -- 172.21.15.45:0/1586897607 learned_addr learned my addr 172.21.15.45:0/1586897607 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0cc6d700 1 -- 172.21.15.45:0/1586897607 >> v2:172.21.15.144:3300/0 conn(0x7fce0815d3d0 msgr2=0x7fce08067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0cc6d700 1 --2- 172.21.15.45:0/1586897607 >> v2:172.21.15.144:3300/0 conn(0x7fce0815d3d0 0x7fce08067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0cc6d700 1 -- 172.21.15.45:0/1586897607 >> v2:172.21.15.45:3300/0 conn(0x7fce08159150 msgr2=0x7fce080ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0cc6d700 1 --2- 172.21.15.45:0/1586897607 >> v2:172.21.15.45:3300/0 conn(0x7fce08159150 0x7fce080ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0cc6d700 1 -- 172.21.15.45:0/1586897607 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce080c9920 con 0x7fce08159520 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0d46e700 1 --2- 172.21.15.45:0/1586897607 >> v2:172.21.15.45:3300/0 conn(0x7fce08159150 0x7fce080ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0dc6f700 1 --2- 172.21.15.45:0/1586897607 >> v2:172.21.15.144:3300/0 conn(0x7fce0815d3d0 0x7fce08067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fce0cc6d700 1 --2- 172.21.15.45:0/1586897607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159520 0x7fce0815ce90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce080c3cf0 tx=0x7fcdfc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840dfd10548ca99f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.977+0000 7fcdfbfff700 1 -- 172.21.15.45:0/1586897607 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc011630 con 0x7fce08159520 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fcdfbfff700 1 -- 172.21.15.45:0/1586897607 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0117b0 con 0x7fce08159520 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fcdfbfff700 1 -- 172.21.15.45:0/1586897607 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc011b70 con 0x7fce08159520 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1586897607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159520 msgr2=0x7fce0815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1586897607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159520 0x7fce0815ce90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fce080c3cf0 tx=0x7fcdfc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1586897607 shutdown_connections 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1586897607 >> v2:172.21.15.144:3300/0 conn(0x7fce0815d3d0 0x7fce08067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1586897607 >> v2:172.21.15.45:3300/0 conn(0x7fce08159150 0x7fce080ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1586897607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159520 0x7fce0815ce90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1586897607 >> 172.21.15.45:0/1586897607 conn(0x7fce080691c0 msgr2=0x7fce080cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1586897607 shutdown_connections 2023-10-06T05:23:58.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.978+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1586897607 wait complete. 2023-10-06T05:23:58.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.979+0000 7fce0f6d2700 1 Processor -- start 2023-10-06T05:23:58.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.979+0000 7fce0f6d2700 1 -- start start 2023-10-06T05:23:58.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0f6d2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159150 0x7fce0814f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0f6d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce0815d3d0 0x7fce080c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0f6d2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce080c8b40 0x7fce080b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0f6d2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0814fa70 con 0x7fce0815d3d0 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0f6d2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce080c8fc0 con 0x7fce080c8b40 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0f6d2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0816a120 con 0x7fce08159150 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0cc6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce0815d3d0 0x7fce080c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0cc6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce0815d3d0 0x7fce080c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38794/0 (socket says 172.21.15.45:38794) 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.980+0000 7fce0cc6d700 1 -- 172.21.15.45:0/1715650886 learned_addr learned my addr 172.21.15.45:0/1715650886 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0d46e700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159150 0x7fce0814f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0dc6f700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce080c8b40 0x7fce080b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0cc6d700 1 -- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159150 msgr2=0x7fce0814f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0cc6d700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159150 0x7fce0814f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0cc6d700 1 -- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce080c8b40 msgr2=0x7fce080b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0cc6d700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce080c8b40 0x7fce080b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0cc6d700 1 -- 172.21.15.45:0/1715650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040094f0 con 0x7fce0815d3d0 2023-10-06T05:23:58.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0cc6d700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce0815d3d0 0x7fce080c8600 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fce0815dd00 tx=0x7fcdfc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fcdf97fa700 1 -- 172.21.15.45:0/1715650886 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc01f070 con 0x7fce0815d3d0 2023-10-06T05:23:58.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fcdf97fa700 1 -- 172.21.15.45:0/1715650886 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc000bc0 con 0x7fce0815d3d0 2023-10-06T05:23:58.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce00009a20 con 0x7fce0815d3d0 2023-10-06T05:23:58.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fcdf97fa700 1 -- 172.21.15.45:0/1715650886 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc011b70 con 0x7fce0815d3d0 2023-10-06T05:23:58.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.981+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc009420 con 0x7fce0815d3d0 2023-10-06T05:23:58.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.982+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0806e2d0 con 0x7fce0815d3d0 2023-10-06T05:23:58.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.983+0000 7fcdf97fa700 1 -- 172.21.15.45:0/1715650886 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdfc023070 con 0x7fce0815d3d0 2023-10-06T05:23:58.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.983+0000 7fcdf97fa700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdec066f30 0x7fcdec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.983+0000 7fcdf97fa700 1 -- 172.21.15.45:0/1715650886 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fcdfc085e90 con 0x7fce0815d3d0 2023-10-06T05:23:58.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.983+0000 7fce0d46e700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdec066f30 0x7fcdec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.984+0000 7fce0d46e700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdec066f30 0x7fcdec0693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce04006d20 tx=0x7fce04009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:58.988+0000 7fcdf97fa700 1 -- 172.21.15.45:0/1715650886 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdfc052950 con 0x7fce0815d3d0 2023-10-06T05:23:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.237+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fce0816a260 con 0x7fce0815d3d0 2023-10-06T05:23:59.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.488+0000 7fcdf97fa700 1 -- 172.21.15.45:0/1715650886 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v187) v1 ==== 162+0+0 (secure 0 0 0) 0x7fcdfc055f70 con 0x7fce0815d3d0 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdec066f30 msgr2=0x7fcdec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdec066f30 0x7fcdec0693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce04006d20 tx=0x7fce04009250 comp rx=0 tx=0).stop 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce0815d3d0 msgr2=0x7fce080c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce0815d3d0 0x7fce080c8600 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fce0815dd00 tx=0x7fcdfc017000 comp rx=0 tx=0).stop 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 shutdown_connections 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcdec066f30 0x7fcdec0693e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce080c8b40 0x7fce080b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce08159150 0x7fce0814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 --2- 172.21.15.45:0/1715650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce0815d3d0 0x7fce080c8600 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 >> 172.21.15.45:0/1715650886 conn(0x7fce080691c0 msgr2=0x7fce080c3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 shutdown_connections 2023-10-06T05:23:59.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.491+0000 7fce0f6d2700 1 -- 172.21.15.45:0/1715650886 wait complete. 2023-10-06T05:23:59.494 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:23:59.498 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:59.497+0000 7f15c4329700 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer 2023-10-06T05:23:59.498 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:23:59.497+0000 7f15c4329700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:23:59.499 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:59.498+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:59.499 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:23:59.498+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:23:59.504 INFO:tasks.ceph.mds.c.smithi045.stderr: -15> 2023-10-06T05:23:59.497+0000 7f15c4329700 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer 2023-10-06T05:23:59.504 INFO:tasks.ceph.mds.c.smithi045.stderr: -13> 2023-10-06T05:23:59.497+0000 7f15c4329700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:23:59.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:59.527 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:23:59.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.643+0000 7fd2ec8c3700 1 Processor -- start 2023-10-06T05:23:59.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.643+0000 7fd2ec8c3700 1 -- start start 2023-10-06T05:23:59.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2ec8c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2e80cd4d0 0x7fd2e815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2ec8c3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e80ae8d0 con 0x7fd2e81560a0 2023-10-06T05:23:59.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2ec8c3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e80aebb0 con 0x7fd2e8155cd0 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2ec8c3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e80ae6b0 con 0x7fd2e80cd4d0 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2e6d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2e80cd4d0 0x7fd2e815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2e5d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd2e81560a0 0x7fd2e80ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42320/0 (socket says 172.21.15.45:42320) 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2e5d9b700 1 -- 172.21.15.45:0/2403527952 learned_addr learned my addr 172.21.15.45:0/2403527952 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2e6d9d700 1 -- 172.21.15.45:0/2403527952 >> v1:172.21.15.144:6789/0 conn(0x7fd2e8155cd0 legacy=0x7fd2e80adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2e6d9d700 1 -- 172.21.15.45:0/2403527952 >> v1:172.21.15.45:6789/0 conn(0x7fd2e81560a0 legacy=0x7fd2e80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.644+0000 7fd2e6d9d700 1 -- 172.21.15.45:0/2403527952 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80aee90 con 0x7fd2e80cd4d0 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2e6d9d700 1 --2- 172.21.15.45:0/2403527952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e80cd4d0 0x7fd2e815a730 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009e60 tx=0x7fd2e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269d6389b36ecd27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2e559a700 1 -- 172.21.15.45:0/2403527952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000e070 con 0x7fd2e80cd4d0 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2e559a700 1 -- 172.21.15.45:0/2403527952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e000bc30 con 0x7fd2e80cd4d0 2023-10-06T05:23:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2e559a700 1 -- 172.21.15.45:0/2403527952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0012430 con 0x7fd2e80cd4d0 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2403527952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e80cd4d0 msgr2=0x7fd2e815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2403527952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e80cd4d0 0x7fd2e815a730 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009e60 tx=0x7fd2e0009780 comp rx=0 tx=0).stop 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2403527952 shutdown_connections 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2403527952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e80cd4d0 0x7fd2e815a730 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2403527952 >> 172.21.15.45:0/2403527952 conn(0x7fd2e806f090 msgr2=0x7fd2e806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2403527952 shutdown_connections 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.645+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2403527952 wait complete. 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 Processor -- start 2023-10-06T05:23:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 -- start start 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e80cd4d0 0x7fd2e8165c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e8155cd0 0x7fd2e8154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2e81560a0 0x7fd2e8154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8169650 con 0x7fd2e8155cd0 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8168a40 con 0x7fd2e80cd4d0 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.647+0000 7fd2ec8c3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8168d20 con 0x7fd2e81560a0 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e8155cd0 0x7fd2e8154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e8155cd0 0x7fd2e8154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38824/0 (socket says 172.21.15.45:38824) 2023-10-06T05:23:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 -- 172.21.15.45:0/2810867961 learned_addr learned my addr 172.21.15.45:0/2810867961 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:23:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e6d9d700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2e81560a0 0x7fd2e8154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e659c700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e80cd4d0 0x7fd2e8165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 -- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2e81560a0 msgr2=0x7fd2e8154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2e81560a0 0x7fd2e8154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 -- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e80cd4d0 msgr2=0x7fd2e8165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e80cd4d0 0x7fd2e8165c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 -- 172.21.15.45:0/2810867961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e0009a20 con 0x7fd2e8155cd0 2023-10-06T05:23:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2e5d9b700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e8155cd0 0x7fd2e8154000 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c7c0 tx=0x7fd2d8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2de7fc700 1 -- 172.21.15.45:0/2810867961 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8005b30 con 0x7fd2e8155cd0 2023-10-06T05:23:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2de7fc700 1 -- 172.21.15.45:0/2810867961 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8005c90 con 0x7fd2e8155cd0 2023-10-06T05:23:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e8169000 con 0x7fd2e8155cd0 2023-10-06T05:23:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2de7fc700 1 -- 172.21.15.45:0/2810867961 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8004a80 con 0x7fd2e8155cd0 2023-10-06T05:23:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.648+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e81661a0 con 0x7fd2e8155cd0 2023-10-06T05:23:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.649+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e8169800 con 0x7fd2e8155cd0 2023-10-06T05:23:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.650+0000 7fd2de7fc700 1 -- 172.21.15.45:0/2810867961 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2d80070d0 con 0x7fd2e8155cd0 2023-10-06T05:23:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.651+0000 7fd2de7fc700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c8066f30 0x7fd2c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.651+0000 7fd2de7fc700 1 -- 172.21.15.45:0/2810867961 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fd2d8084a90 con 0x7fd2e8155cd0 2023-10-06T05:23:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.651+0000 7fd2e659c700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c8066f30 0x7fd2c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.651+0000 7fd2e659c700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c8066f30 0x7fd2c80693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0003aa0 tx=0x7fd2d0000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.654+0000 7fd2de7fc700 1 -- 172.21.15.45:0/2810867961 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2d8051580 con 0x7fd2e8155cd0 2023-10-06T05:23:59.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.902+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2e8169800 con 0x7fd2e8155cd0 2023-10-06T05:23:59.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.902+0000 7fd2de7fc700 1 -- 172.21.15.45:0/2810867961 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fd2e8169800 con 0x7fd2e8155cd0 2023-10-06T05:23:59.904 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:23:59.904 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":189,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:59.498826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:23:59.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c8066f30 msgr2=0x7fd2c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c8066f30 0x7fd2c80693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0003aa0 tx=0x7fd2d0000bc0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e8155cd0 msgr2=0x7fd2e8154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e8155cd0 0x7fd2e8154000 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c7c0 tx=0x7fd2d8007f70 comp rx=0 tx=0).stop 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 shutdown_connections 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2c8066f30 0x7fd2c80693e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2e81560a0 0x7fd2e8154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2e80cd4d0 0x7fd2e8165c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 --2- 172.21.15.45:0/2810867961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2e8155cd0 0x7fd2e8154000 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 >> 172.21.15.45:0/2810867961 conn(0x7fd2e806f090 msgr2=0x7fd2e80b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 shutdown_connections 2023-10-06T05:23:59.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:23:59.904+0000 7fd2ec8c3700 1 -- 172.21.15.45:0/2810867961 wait complete. 2023-10-06T05:23:59.908 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 189 2023-10-06T05:23:59.919 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:23:59.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.095+0000 7f285f7af700 1 Processor -- start 2023-10-06T05:24:00.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.096+0000 7f285f7af700 1 -- start start 2023-10-06T05:24:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285f7af700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2858155440 0x7f2858155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285f7af700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f285806e6e0 0x7f28580c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285f7af700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f285806a890 0x7f285806acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285f7af700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2858068cd0 con 0x7f285806e6e0 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285f7af700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285806b230 con 0x7f285806a890 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285f7af700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28580c8080 con 0x7f2858155440 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f285806e6e0 0x7f28580c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f285806e6e0 0x7f28580c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38830/0 (socket says 172.21.15.45:38830) 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 -- 172.21.15.45:0/4192186030 learned_addr learned my addr 172.21.15.45:0/4192186030 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285d54b700 1 --2- 172.21.15.45:0/4192186030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2858155440 0x7f2858155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285dd4c700 1 --2- 172.21.15.45:0/4192186030 >> v2:172.21.15.144:3300/0 conn(0x7f285806a890 0x7f285806acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 -- 172.21.15.45:0/4192186030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2858155440 msgr2=0x7f2858155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:00.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 --2- 172.21.15.45:0/4192186030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2858155440 0x7f2858155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 -- 172.21.15.45:0/4192186030 >> v2:172.21.15.144:3300/0 conn(0x7f285806a890 msgr2=0x7f285806acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 --2- 172.21.15.45:0/4192186030 >> v2:172.21.15.144:3300/0 conn(0x7f285806a890 0x7f285806acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 -- 172.21.15.45:0/4192186030 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858156050 con 0x7f285806e6e0 2023-10-06T05:24:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.097+0000 7f285cd4a700 1 --2- 172.21.15.45:0/4192186030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806e6e0 0x7f28580c8320 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f28580c6ad0 tx=0x7f2848009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e345d411edb7fcd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f284ffff700 1 -- 172.21.15.45:0/4192186030 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800b640 con 0x7f285806e6e0 2023-10-06T05:24:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f284ffff700 1 -- 172.21.15.45:0/4192186030 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284800b7a0 con 0x7f285806e6e0 2023-10-06T05:24:00.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f284ffff700 1 -- 172.21.15.45:0/4192186030 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800bb60 con 0x7f285806e6e0 2023-10-06T05:24:00.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 -- 172.21.15.45:0/4192186030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806e6e0 msgr2=0x7f28580c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 --2- 172.21.15.45:0/4192186030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806e6e0 0x7f28580c8320 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f28580c6ad0 tx=0x7f2848009250 comp rx=0 tx=0).stop 2023-10-06T05:24:00.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 -- 172.21.15.45:0/4192186030 shutdown_connections 2023-10-06T05:24:00.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 --2- 172.21.15.45:0/4192186030 >> v2:172.21.15.144:3300/0 conn(0x7f285806a890 0x7f285806acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 --2- 172.21.15.45:0/4192186030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2858155440 0x7f2858155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 --2- 172.21.15.45:0/4192186030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806e6e0 0x7f28580c8320 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 -- 172.21.15.45:0/4192186030 >> 172.21.15.45:0/4192186030 conn(0x7f28580cc5a0 msgr2=0x7f28580cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 -- 172.21.15.45:0/4192186030 shutdown_connections 2023-10-06T05:24:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.098+0000 7f285f7af700 1 -- 172.21.15.45:0/4192186030 wait complete. 2023-10-06T05:24:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.100+0000 7f285f7af700 1 Processor -- start 2023-10-06T05:24:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.100+0000 7f285f7af700 1 -- start start 2023-10-06T05:24:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285f7af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806a890 0x7f28580b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285f7af700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285806e6e0 0x7f28580a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285f7af700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2858155440 0x7f2858156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285f7af700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28580b15c0 con 0x7f285806a890 2023-10-06T05:24:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285f7af700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28580c33d0 con 0x7f2858155440 2023-10-06T05:24:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285f7af700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28580b09b0 con 0x7f285806e6e0 2023-10-06T05:24:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806a890 0x7f28580b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806a890 0x7f28580b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38838/0 (socket says 172.21.15.45:38838) 2023-10-06T05:24:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 -- 172.21.15.45:0/1359037745 learned_addr learned my addr 172.21.15.45:0/1359037745 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285cd4a700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285806e6e0 0x7f28580a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285dd4c700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2858155440 0x7f2858156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 -- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285806e6e0 msgr2=0x7f28580a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285806e6e0 0x7f28580a9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 -- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2858155440 msgr2=0x7f2858156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2858155440 0x7f2858156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 -- 172.21.15.45:0/1359037745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480094f0 con 0x7f285806a890 2023-10-06T05:24:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285dd4c700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2858155440 0x7f2858156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285d54b700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806a890 0x7f28580b0320 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f285400e930 tx=0x7f28540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f284d7fa700 1 -- 172.21.15.45:0/1359037745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854013680 con 0x7f285806a890 2023-10-06T05:24:00.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f284d7fa700 1 -- 172.21.15.45:0/1359037745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28540137e0 con 0x7f285806a890 2023-10-06T05:24:00.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f284d7fa700 1 -- 172.21.15.45:0/1359037745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854013b80 con 0x7f285806a890 2023-10-06T05:24:00.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.101+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28580b0c90 con 0x7f285806a890 2023-10-06T05:24:00.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.102+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28580b1250 con 0x7f285806a890 2023-10-06T05:24:00.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.102+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28580cc0e0 con 0x7f285806a890 2023-10-06T05:24:00.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.103+0000 7f284d7fa700 1 -- 172.21.15.45:0/1359037745 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f285400c460 con 0x7f285806a890 2023-10-06T05:24:00.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.103+0000 7f284d7fa700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f283c066f30 0x7f283c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.103+0000 7f284d7fa700 1 -- 172.21.15.45:0/1359037745 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f2854085250 con 0x7f285806a890 2023-10-06T05:24:00.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.104+0000 7f285cd4a700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f283c066f30 0x7f283c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.104+0000 7f285cd4a700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f283c066f30 0x7f283c0693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f28580c6ad0 tx=0x7f2848018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.108+0000 7f284d7fa700 1 -- 172.21.15.45:0/1359037745 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2854051cc0 con 0x7f285806a890 2023-10-06T05:24:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.353+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2858156330 con 0x7f285806a890 2023-10-06T05:24:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.354+0000 7f284d7fa700 1 -- 172.21.15.45:0/1359037745 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f2854017090 con 0x7f285806a890 2023-10-06T05:24:00.356 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:00.356 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":189,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:59.498826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f283c066f30 msgr2=0x7f283c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f283c066f30 0x7f283c0693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f28580c6ad0 tx=0x7f2848018000 comp rx=0 tx=0).stop 2023-10-06T05:24:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806a890 msgr2=0x7f28580b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806a890 0x7f28580b0320 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f285400e930 tx=0x7f28540091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 shutdown_connections 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f283c066f30 0x7f283c0693e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2858155440 0x7f2858156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285806a890 0x7f28580b0320 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 --2- 172.21.15.45:0/1359037745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285806e6e0 0x7f28580a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.356+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 >> 172.21.15.45:0/1359037745 conn(0x7f28580cc5a0 msgr2=0x7f28580c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.357+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 shutdown_connections 2023-10-06T05:24:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.357+0000 7f285f7af700 1 -- 172.21.15.45:0/1359037745 wait complete. 2023-10-06T05:24:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 189 2023-10-06T05:24:00.372 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:00.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.505+0000 7fd602c83700 1 Processor -- start 2023-10-06T05:24:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.505+0000 7fd602c83700 1 -- start start 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd602c83700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd602c83700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0cd760 con 0x7fd5fc1595d0 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd602c83700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0c7d00 con 0x7fd5fc159200 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd602c83700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0c7780 con 0x7fd5fc0ccdc0 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd5fbfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd5fc1595d0 0x7fd5fc06b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42332/0 (socket says 172.21.15.45:42332) 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd5fbfff700 1 -- 172.21.15.45:0/2305424578 learned_addr learned my addr 172.21.15.45:0/2305424578 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd601220700 1 --2- 172.21.15.45:0/2305424578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd5fb7fe700 1 -- 172.21.15.45:0/2305424578 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98570159 0 0) 0x7fd5e8001110 con 0x7fd5fc1595d0 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.506+0000 7fd5fb7fe700 1 -- 172.21.15.45:0/2305424578 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ec003570 con 0x7fd5fc1595d0 2023-10-06T05:24:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd5fb7fe700 1 -- 172.21.15.45:0/2305424578 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863648603 0 0) 0x7fd5fc0c7d00 con 0x7fd5fc159200 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd5fb7fe700 1 -- 172.21.15.45:0/2305424578 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e8001110 con 0x7fd5fc159200 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd601220700 1 -- 172.21.15.45:0/2305424578 >> v1:172.21.15.144:6789/0 conn(0x7fd5fc159200 legacy=0x7fd5fc0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd601220700 1 -- 172.21.15.45:0/2305424578 >> v1:172.21.15.45:6789/0 conn(0x7fd5fc1595d0 legacy=0x7fd5fc06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd601220700 1 -- 172.21.15.45:0/2305424578 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc06bb40 con 0x7fd5fc0ccdc0 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd601220700 1 --2- 172.21.15.45:0/2305424578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0cd220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009e90 tx=0x7fd5f4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8bfb00dfebd924 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd5fb7fe700 1 -- 172.21.15.45:0/2305424578 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400e070 con 0x7fd5fc0ccdc0 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd5fb7fe700 1 -- 172.21.15.45:0/2305424578 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f400bb80 con 0x7fd5fc0ccdc0 2023-10-06T05:24:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.507+0000 7fd5fb7fe700 1 -- 172.21.15.45:0/2305424578 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4012430 con 0x7fd5fc0ccdc0 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.508+0000 7fd602c83700 1 -- 172.21.15.45:0/2305424578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 msgr2=0x7fd5fc0cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.508+0000 7fd602c83700 1 --2- 172.21.15.45:0/2305424578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0cd220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009e90 tx=0x7fd5f4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.508+0000 7fd602c83700 1 -- 172.21.15.45:0/2305424578 shutdown_connections 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.508+0000 7fd602c83700 1 --2- 172.21.15.45:0/2305424578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0cd220 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.508+0000 7fd602c83700 1 -- 172.21.15.45:0/2305424578 >> 172.21.15.45:0/2305424578 conn(0x7fd5fc06f090 msgr2=0x7fd5fc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.508+0000 7fd602c83700 1 -- 172.21.15.45:0/2305424578 shutdown_connections 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.508+0000 7fd602c83700 1 -- 172.21.15.45:0/2305424578 wait complete. 2023-10-06T05:24:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.509+0000 7fd602c83700 1 Processor -- start 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.509+0000 7fd602c83700 1 -- start start 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd602c83700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd602c83700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc159200 0x7fd5fc0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd602c83700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5fc1595d0 0x7fd5fc0c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd602c83700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0c5060 con 0x7fd5fc1595d0 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd602c83700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b1f00 con 0x7fd5fc159200 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd602c83700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b12f0 con 0x7fd5fc0ccdc0 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5fc1595d0 0x7fd5fc0c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5fc1595d0 0x7fd5fc0c2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38840/0 (socket says 172.21.15.45:38840) 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 -- 172.21.15.45:0/1550394467 learned_addr learned my addr 172.21.15.45:0/1550394467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd5fbfff700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc159200 0x7fd5fc0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd600a1f700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 -- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 msgr2=0x7fd5fc0c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 -- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc159200 msgr2=0x7fd5fc0b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc159200 0x7fd5fc0b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd601220700 1 -- 172.21.15.45:0/1550394467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4009a20 con 0x7fd5fc1595d0 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd5fbfff700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc159200 0x7fd5fc0b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.510+0000 7fd600a1f700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.511+0000 7fd601220700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5fc1595d0 0x7fd5fc0c2c30 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b610 tx=0x7fd5f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.511+0000 7fd5f8ff9700 1 -- 172.21.15.45:0/1550394467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4012e30 con 0x7fd5fc1595d0 2023-10-06T05:24:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.511+0000 7fd5f8ff9700 1 -- 172.21.15.45:0/1550394467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f401c610 con 0x7fd5fc1595d0 2023-10-06T05:24:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.511+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc0b15d0 con 0x7fd5fc1595d0 2023-10-06T05:24:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.511+0000 7fd5f8ff9700 1 -- 172.21.15.45:0/1550394467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f401c9d0 con 0x7fd5fc1595d0 2023-10-06T05:24:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.511+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0b1b90 con 0x7fd5fc1595d0 2023-10-06T05:24:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.512+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc06e2f0 con 0x7fd5fc1595d0 2023-10-06T05:24:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.513+0000 7fd5f8ff9700 1 -- 172.21.15.45:0/1550394467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5f4026480 con 0x7fd5fc1595d0 2023-10-06T05:24:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.513+0000 7fd5f8ff9700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd5e0066f30 0x7fd5e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.513+0000 7fd5f8ff9700 1 -- 172.21.15.45:0/1550394467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fd5f40873f0 con 0x7fd5fc1595d0 2023-10-06T05:24:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.513+0000 7fd600a1f700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd5e0066f30 0x7fd5e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.514+0000 7fd600a1f700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd5e0066f30 0x7fd5e00693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00028c0 tx=0x7fd5f0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.517+0000 7fd5f8ff9700 1 -- 172.21.15.45:0/1550394467 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5f4053e40 con 0x7fd5fc1595d0 2023-10-06T05:24:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.760+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5fc06be20 con 0x7fd5fc1595d0 2023-10-06T05:24:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.761+0000 7fd5f8ff9700 1 -- 172.21.15.45:0/1550394467 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) v1 ==== 95+0+4599 (secure 0 0 0) 0x7fd5f4057460 con 0x7fd5fc1595d0 2023-10-06T05:24:00.763 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:00.763 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":190,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8458,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1915871760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1915871760},{"type":"v1","addr":"172.21.15.45:6835","nonce":1915871760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":190}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:23:59.498826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd5e0066f30 msgr2=0x7fd5e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd5e0066f30 0x7fd5e00693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00028c0 tx=0x7fd5f0009040 comp rx=0 tx=0).stop 2023-10-06T05:24:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5fc1595d0 msgr2=0x7fd5fc0c2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5fc1595d0 0x7fd5fc0c2c30 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b610 tx=0x7fd5f4019000 comp rx=0 tx=0).stop 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 shutdown_connections 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd5e0066f30 0x7fd5e00693e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5fc1595d0 0x7fd5fc0c2c30 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0ccdc0 0x7fd5fc0c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 --2- 172.21.15.45:0/1550394467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc159200 0x7fd5fc0b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 >> 172.21.15.45:0/1550394467 conn(0x7fd5fc06f090 msgr2=0x7fd5fc06b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 shutdown_connections 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.763+0000 7fd602c83700 1 -- 172.21.15.45:0/1550394467 wait complete. 2023-10-06T05:24:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 190 2023-10-06T05:24:00.778 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2023-10-06T05:24:00.778 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2023-10-06T05:24:00.779 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:00.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:00.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:00.779 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:00.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:00.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.974+0000 7fac8d51a700 1 Processor -- start 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.974+0000 7fac8d51a700 1 -- start start 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac8d51a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac88156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac8d51a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac88157030 0x7fac880cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac8d51a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8815b000 con 0x7fac880c8ff0 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac8d51a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac880c9830 con 0x7fac88157030 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac8d51a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8815b2e0 con 0x7fac880c93c0 2023-10-06T05:24:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac88157030 0x7fac880cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac86ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fac880c8ff0 0x7fac880caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42336/0 (socket says 172.21.15.45:42336) 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.975+0000 7fac86ffd700 1 -- 172.21.15.45:0/899434472 learned_addr learned my addr 172.21.15.45:0/899434472 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac867fc700 1 --2- 172.21.15.45:0/899434472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac88156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac877fe700 1 -- 172.21.15.45:0/899434472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 msgr2=0x7fac88156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac877fe700 1 --2- 172.21.15.45:0/899434472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac88156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac877fe700 1 -- 172.21.15.45:0/899434472 >> v1:172.21.15.45:6789/0 conn(0x7fac880c8ff0 legacy=0x7fac880caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac877fe700 1 -- 172.21.15.45:0/899434472 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8815b5c0 con 0x7fac88157030 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac85ffb700 1 -- 172.21.15.45:0/899434472 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684596267 0 0) 0x7fac8815b000 con 0x7fac880c8ff0 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac877fe700 1 --2- 172.21.15.45:0/899434472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880cc330 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fac80009e60 tx=0x7fac80009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d545dfb167ade910 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac85ffb700 1 -- 172.21.15.45:0/899434472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8000e070 con 0x7fac88157030 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac85ffb700 1 -- 172.21.15.45:0/899434472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8000bb60 con 0x7fac88157030 2023-10-06T05:24:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.976+0000 7fac85ffb700 1 -- 172.21.15.45:0/899434472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80012430 con 0x7fac88157030 2023-10-06T05:24:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 -- 172.21.15.45:0/899434472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 msgr2=0x7fac880cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 --2- 172.21.15.45:0/899434472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880cc330 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fac80009e60 tx=0x7fac80009780 comp rx=0 tx=0).stop 2023-10-06T05:24:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 -- 172.21.15.45:0/899434472 shutdown_connections 2023-10-06T05:24:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 --2- 172.21.15.45:0/899434472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880cc330 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 --2- 172.21.15.45:0/899434472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac88156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 -- 172.21.15.45:0/899434472 >> 172.21.15.45:0/899434472 conn(0x7fac880cdf50 msgr2=0x7fac880ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 -- 172.21.15.45:0/899434472 shutdown_connections 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.977+0000 7fac8d51a700 1 -- 172.21.15.45:0/899434472 wait complete. 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.978+0000 7fac8d51a700 1 Processor -- start 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.978+0000 7fac8d51a700 1 -- start start 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac8d51a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac880c8ff0 0x7fac880ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac8d51a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac880af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac8d51a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac8d51a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac880c72e0 con 0x7fac88157030 2023-10-06T05:24:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac8d51a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac880c66d0 con 0x7fac880c8ff0 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac8d51a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac880c69b0 con 0x7fac880c93c0 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38856/0 (socket says 172.21.15.45:38856) 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac877fe700 1 -- 172.21.15.45:0/2643236655 learned_addr learned my addr 172.21.15.45:0/2643236655 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac86ffd700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac880c8ff0 0x7fac880ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.979+0000 7fac867fc700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac880af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac877fe700 1 -- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 msgr2=0x7fac880af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac877fe700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac880af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac877fe700 1 -- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac880c8ff0 msgr2=0x7fac880ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac877fe700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac880c8ff0 0x7fac880ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac877fe700 1 -- 172.21.15.45:0/2643236655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac80009a20 con 0x7fac88157030 2023-10-06T05:24:00.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac877fe700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880af8c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fac880c4f80 tx=0x7fac80019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac7effd700 1 -- 172.21.15.45:0/2643236655 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80012ea0 con 0x7fac88157030 2023-10-06T05:24:00.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac7effd700 1 -- 172.21.15.45:0/2643236655 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac80027470 con 0x7fac88157030 2023-10-06T05:24:00.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac880c6c90 con 0x7fac88157030 2023-10-06T05:24:00.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac7effd700 1 -- 172.21.15.45:0/2643236655 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80027830 con 0x7fac88157030 2023-10-06T05:24:00.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.980+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac880ad5e0 con 0x7fac88157030 2023-10-06T05:24:00.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.981+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8806f410 con 0x7fac88157030 2023-10-06T05:24:00.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.982+0000 7fac7effd700 1 -- 172.21.15.45:0/2643236655 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac80024070 con 0x7fac88157030 2023-10-06T05:24:00.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.982+0000 7fac7effd700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac68066f30 0x7fac680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.982+0000 7fac7effd700 1 -- 172.21.15.45:0/2643236655 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fac800876d0 con 0x7fac88157030 2023-10-06T05:24:00.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.983+0000 7fac86ffd700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac68066f30 0x7fac680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.983+0000 7fac86ffd700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac68066f30 0x7fac680693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fac78007160 tx=0x7fac78007f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:00.986+0000 7fac7effd700 1 -- 172.21.15.45:0/2643236655 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac800541c0 con 0x7fac88157030 2023-10-06T05:24:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.235+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fac8804bb40 con 0x7fac88157030 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.878+0000 7fac7effd700 1 -- 172.21.15.45:0/2643236655 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v191) v1 ==== 161+0+0 (secure 0 0 0) 0x7fac800577e0 con 0x7fac88157030 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac68066f30 msgr2=0x7fac680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac68066f30 0x7fac680693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fac78007160 tx=0x7fac78007f50 comp rx=0 tx=0).stop 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 msgr2=0x7fac880af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880af8c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fac880c4f80 tx=0x7fac80019000 comp rx=0 tx=0).stop 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 shutdown_connections 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac68066f30 0x7fac680693e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac88157030 0x7fac880af8c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac880c8ff0 0x7fac880ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 --2- 172.21.15.45:0/2643236655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880c93c0 0x7fac880af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 >> 172.21.15.45:0/2643236655 conn(0x7fac880cdf50 msgr2=0x7fac880cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 shutdown_connections 2023-10-06T05:24:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:01.881+0000 7fac8d51a700 1 -- 172.21.15.45:0/2643236655 wait complete. 2023-10-06T05:24:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:01.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:02.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.034+0000 7eff0d7fe700 1 Processor -- start 2023-10-06T05:24:02.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.034+0000 7eff0d7fe700 1 -- start start 2023-10-06T05:24:02.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff0d7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 0x7eff080cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff0d7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff080c9640 0x7eff080b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff0d7fe700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff080c84c0 con 0x7eff080c91f0 2023-10-06T05:24:02.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff0d7fe700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff080c8d60 con 0x7eff080c9640 2023-10-06T05:24:02.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff0d7fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff08069d00 con 0x7eff080ca810 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff067fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7eff080c91f0 0x7eff080afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42344/0 (socket says 172.21.15.45:42344) 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff067fc700 1 -- 172.21.15.45:0/4101140338 learned_addr learned my addr 172.21.15.45:0/4101140338 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff06ffd700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 0x7eff080cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff077fe700 1 --2- 172.21.15.45:0/4101140338 >> v2:172.21.15.144:3300/0 conn(0x7eff080c9640 0x7eff080b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925368693 0 0) 0x7eff080c84c0 con 0x7eff080c91f0 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efef0003570 con 0x7eff080c91f0 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4096778301 0 0) 0x7efef0003570 con 0x7eff080c91f0 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 msgr2=0x7eff080cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 0x7eff080cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 >> v2:172.21.15.144:3300/0 conn(0x7eff080c9640 msgr2=0x7eff080b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 --2- 172.21.15.45:0/4101140338 >> v2:172.21.15.144:3300/0 conn(0x7eff080c9640 0x7eff080b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080c8710 con 0x7eff080c91f0 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7efef8001470 con 0x7eff080c91f0 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.035+0000 7eff06ffd700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 0x7eff080cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 >> v1:172.21.15.45:6789/0 conn(0x7eff080c91f0 legacy=0x7eff080afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 0x7efef0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef0003fa0 0x7efef0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef0004d50 0x7efef00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080c8d60 con 0x7efef0004d50 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff08069d00 con 0x7efef0003fa0 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef000ba60 con 0x7eff080ca810 2023-10-06T05:24:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 --2- 172.21.15.45:0/4101140338 >> v2:172.21.15.144:3300/0 conn(0x7eff080c9640 0x7eff080b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff06ffd700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 0x7efef0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef0004d50 0x7efef00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff067fc700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef0003fa0 0x7efef0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 -- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 msgr2=0x7efef0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080ca810 0x7efef0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 -- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef0003fa0 msgr2=0x7efef0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef0003fa0 0x7efef0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 -- 172.21.15.45:0/4101140338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc009420 con 0x7efef0004d50 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff077fe700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef0004d50 0x7efef00051c0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7eff0000e3c0 tx=0x7eff0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ced17b02fcd35a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000ec70 con 0x7efef0004d50 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0000f040 con 0x7efef0004d50 2023-10-06T05:24:02.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.036+0000 7eff05ffb700 1 -- 172.21.15.45:0/4101140338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00014480 con 0x7efef0004d50 2023-10-06T05:24:02.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.037+0000 7eff0d7fe700 1 -- 172.21.15.45:0/4101140338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef0004d50 msgr2=0x7efef00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.037+0000 7eff0d7fe700 1 --2- 172.21.15.45:0/4101140338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef0004d50 0x7efef00051c0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7eff0000e3c0 tx=0x7eff0000b040 comp rx=0 tx=0).stop 2023-10-06T05:24:02.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.037+0000 7eff077fe700 1 -- 172.21.15.45:0/4101140338 reap_dead start 2023-10-06T05:24:02.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.037+0000 7eff0d7fe700 1 -- 172.21.15.45:0/4101140338 shutdown_connections 2023-10-06T05:24:02.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.037+0000 7eff0d7fe700 1 -- 172.21.15.45:0/4101140338 >> 172.21.15.45:0/4101140338 conn(0x7eff08067620 msgr2=0x7eff08067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.037+0000 7eff0d7fe700 1 -- 172.21.15.45:0/4101140338 shutdown_connections 2023-10-06T05:24:02.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.037+0000 7eff0d7fe700 1 -- 172.21.15.45:0/4101140338 wait complete. 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.038+0000 7eff0d7fe700 1 Processor -- start 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.038+0000 7eff0d7fe700 1 -- start start 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff0d7fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080cc190 0x7eff080aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff0d7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff080ce160 0x7eff080cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff0d7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff080ce5d0 0x7eff080ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff0d7fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080c66b0 con 0x7eff080ce5d0 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff0d7fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080aa630 con 0x7eff080ce160 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff0d7fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff080ccfe0 con 0x7eff080cc190 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff080ce5d0 0x7eff080ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff077fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff080ce5d0 0x7eff080ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38872/0 (socket says 172.21.15.45:38872) 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff077fe700 1 -- 172.21.15.45:0/2696016078 learned_addr learned my addr 172.21.15.45:0/2696016078 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff06ffd700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080cc190 0x7eff080aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.039+0000 7eff067fc700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff080ce160 0x7eff080cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff077fe700 1 -- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080cc190 msgr2=0x7eff080aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff077fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080cc190 0x7eff080aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff077fe700 1 -- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff080ce160 msgr2=0x7eff080cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff077fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff080ce160 0x7eff080cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff077fe700 1 -- 172.21.15.45:0/2696016078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080c6080 con 0x7eff080ce5d0 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff077fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff080ce5d0 0x7eff080ccaa0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7efef00046f0 tx=0x7eff00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7efeeeffd700 1 -- 172.21.15.45:0/2696016078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00014480 con 0x7eff080ce5d0 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7efeeeffd700 1 -- 172.21.15.45:0/2696016078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0001f450 con 0x7eff080ce5d0 2023-10-06T05:24:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff081588c0 con 0x7eff080ce5d0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7efeeeffd700 1 -- 172.21.15.45:0/2696016078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0001f7f0 con 0x7eff080ce5d0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.040+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff080c6340 con 0x7eff080ce5d0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.041+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff080c6820 con 0x7eff080ce5d0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.042+0000 7efeeeffd700 1 -- 172.21.15.45:0/2696016078 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff0001fbe0 con 0x7eff080ce5d0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.042+0000 7efeeeffd700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efef4066f60 0x7efef4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.042+0000 7efeeeffd700 1 -- 172.21.15.45:0/2696016078 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7eff000872b0 con 0x7eff080ce5d0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.042+0000 7eff06ffd700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efef4066f60 0x7efef4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.043+0000 7eff06ffd700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efef4066f60 0x7efef4069410 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efefc006f00 tx=0x7efefc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.046+0000 7efeeeffd700 1 -- 172.21.15.45:0/2696016078 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff00053d20 con 0x7eff080ce5d0 2023-10-06T05:24:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.333+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7eff080c89f0 con 0x7eff080ce5d0 2023-10-06T05:24:02.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.886+0000 7efeeeffd700 1 -- 172.21.15.45:0/2696016078 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v192) v1 ==== 101+0+0 (secure 0 0 0) 0x7eff00057340 con 0x7eff080ce5d0 2023-10-06T05:24:02.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efef4066f60 msgr2=0x7efef4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efef4066f60 0x7efef4069410 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efefc006f00 tx=0x7efefc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff080ce5d0 msgr2=0x7eff080ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff080ce5d0 0x7eff080ccaa0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7efef00046f0 tx=0x7eff00019000 comp rx=0 tx=0).stop 2023-10-06T05:24:02.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 shutdown_connections 2023-10-06T05:24:02.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efef4066f60 0x7efef4069410 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff080ce5d0 0x7eff080ccaa0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff080cc190 0x7eff080aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 --2- 172.21.15.45:0/2696016078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff080ce160 0x7eff080cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 >> 172.21.15.45:0/2696016078 conn(0x7eff08067620 msgr2=0x7eff080b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 shutdown_connections 2023-10-06T05:24:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.889+0000 7eff0d7fe700 1 -- 172.21.15.45:0/2696016078 wait complete. 2023-10-06T05:24:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:02.906 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.952+0000 7f7a23609040 1 Processor -- start 2023-10-06T05:24:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.952+0000 7f7a23609040 1 -- start start 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a23609040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0c0030b0 0x55d7aa23d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a23609040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa248e10 0x55d7aa1a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a23609040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa2491e0 0x55d7aa1b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a23609040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa11a320 con 0x55d7aa2491e0 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a23609040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa0be980 con 0x55d7aa248e10 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a23609040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa0be620 con 0x7f7a0c0030b0 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa2491e0 0x55d7aa1b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa2491e0 0x55d7aa1b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38892/0 (socket says 172.21.15.45:38892) 2023-10-06T05:24:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 -- 172.21.15.45:0/579919208 learned_addr learned my addr 172.21.15.45:0/579919208 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a17fff700 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa248e10 0x55d7aa1a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1c827700 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0c0030b0 0x55d7aa23d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 -- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0c0030b0 msgr2=0x55d7aa23d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0c0030b0 0x55d7aa23d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 -- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa248e10 msgr2=0x55d7aa1a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa248e10 0x55d7aa1a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 -- 172.21.15.45:0/579919208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7aa0c0ae0 con 0x55d7aa2491e0 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a1d028700 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa2491e0 0x55d7aa1b3140 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x55d7aa1a00d0 tx=0x7f7a18002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9181ffce957cfdfd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a16ffd700 1 -- 172.21.15.45:0/579919208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003450 con 0x55d7aa2491e0 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a16ffd700 1 -- 172.21.15.45:0/579919208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a180035b0 con 0x55d7aa2491e0 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.953+0000 7f7a16ffd700 1 -- 172.21.15.45:0/579919208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003930 con 0x55d7aa2491e0 2023-10-06T05:24:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 -- 172.21.15.45:0/579919208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa2491e0 msgr2=0x55d7aa1b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa2491e0 0x55d7aa1b3140 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x55d7aa1a00d0 tx=0x7f7a18002170 comp rx=0 tx=0).stop 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 -- 172.21.15.45:0/579919208 shutdown_connections 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa2491e0 0x55d7aa1b3140 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0c0030b0 0x55d7aa23d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 --2- 172.21.15.45:0/579919208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa248e10 0x55d7aa1a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 -- 172.21.15.45:0/579919208 >> 172.21.15.45:0/579919208 conn(0x55d7aa174450 msgr2=0x55d7aa245d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 -- 172.21.15.45:0/579919208 shutdown_connections 2023-10-06T05:24:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.954+0000 7f7a23609040 1 -- 172.21.15.45:0/579919208 wait complete. 2023-10-06T05:24:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.955+0000 7f7a23609040 1 Processor -- start 2023-10-06T05:24:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.955+0000 7f7a23609040 1 -- start start 2023-10-06T05:24:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.955+0000 7f7a23609040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7aa248e10 0x55d7aa25b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.955+0000 7f7a23609040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa2491e0 0x55d7aa2454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.955+0000 7f7a23609040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0030b0 0x55d7aa23dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a23609040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa0b5400 con 0x7f7a0c0030b0 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a23609040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa25bdc0 con 0x55d7aa2491e0 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a23609040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa23d2c0 con 0x55d7aa248e10 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0030b0 0x55d7aa23dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0030b0 0x55d7aa23dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38900/0 (socket says 172.21.15.45:38900) 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 learned_addr learned my addr 172.21.15.45:0/541859889 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1c827700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7aa248e10 0x55d7aa25b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a17fff700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa2491e0 0x55d7aa2454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7aa248e10 msgr2=0x55d7aa25b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7aa248e10 0x55d7aa25b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa2491e0 msgr2=0x55d7aa2454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7aa2491e0 0x55d7aa2454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7aa0cd3d0 con 0x7f7a0c0030b0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a1d028700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0030b0 0x55d7aa23dca0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f7a18013eb0 tx=0x7f7a18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a157fa700 1 -- 172.21.15.45:0/541859889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003ce0 con 0x7f7a0c0030b0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a157fa700 1 -- 172.21.15.45:0/541859889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18003e60 con 0x7f7a0c0030b0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a157fa700 1 -- 172.21.15.45:0/541859889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003970 con 0x7f7a0c0030b0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.956+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d7aa0c0540 con 0x7f7a0c0030b0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.957+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7aa23d800 con 0x7f7a0c0030b0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.958+0000 7f7a157fa700 1 -- 172.21.15.45:0/541859889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a180283b0 con 0x7f7a0c0030b0 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.958+0000 7f7a157fa700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a00065890 0x7f7a00067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.958+0000 7f7a157fa700 1 -- 172.21.15.45:0/541859889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f7a18084d90 con 0x7f7a0c0030b0 2023-10-06T05:24:02.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.959+0000 7f7a1c827700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a00065890 0x7f7a00067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.963 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.959+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d7aa1b8af0 0x55d7aa256820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.959+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa1b7b30 con 0x55d7aa1b8af0 2023-10-06T05:24:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.959+0000 7f7a17fff700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d7aa1b8af0 0x55d7aa256820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.959+0000 7f7a1c827700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a00065890 0x7f7a00067d40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005fd0 tx=0x7f7a1000c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.959+0000 7f7a17fff700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d7aa1b8af0 0x55d7aa256820 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.961+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v108'474 uv448 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.961+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa256d60 con 0x55d7aa1b8af0 2023-10-06T05:24:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.962+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v108'475 uv475 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.962+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa257110 con 0x55d7aa1b8af0 2023-10-06T05:24:02.965 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.964+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v108'476 uv476 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.964+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268420 con 0x55d7aa1b8af0 2023-10-06T05:24:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.965+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'477 uv477 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.965+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2687d0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.966+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'478 uv478 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.966+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267410 con 0x55d7aa1b8af0 2023-10-06T05:24:02.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.967+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'479 uv479 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.967+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267810 con 0x55d7aa1b8af0 2023-10-06T05:24:02.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.969+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v108'480 uv454 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.969+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267c10 con 0x55d7aa1b8af0 2023-10-06T05:24:02.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.970+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v108'481 uv481 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.970+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267c10 con 0x55d7aa1b8af0 2023-10-06T05:24:02.972 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:02.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.971+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v108'482 uv482 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.971+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.972+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'483 uv483 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.972+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.973+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'484 uv484 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.973+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.975+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'485 uv485 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.975+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.976+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v108'486 uv461 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.976+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.977+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'487 uv487 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.977+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.980 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.978+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v108'488 uv488 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.978+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.979+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'489 uv489 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.979+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.981+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'490 uv490 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.981+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa267bc0 con 0x55d7aa1b8af0 2023-10-06T05:24:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.982+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'491 uv491 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.982+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.983+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'492 uv492 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.983+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.984+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v108'493 uv467 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.984+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.986+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v108'494 uv494 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.986+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.988 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.987+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v108'495 uv495 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.987+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa269b90 con 0x55d7aa1b8af0 2023-10-06T05:24:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.988+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'496 uv496 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.988+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa269b90 con 0x55d7aa1b8af0 2023-10-06T05:24:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.989+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'497 uv497 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.989+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.991+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'498 uv498 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.991+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.992+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v108'499 uv473 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.992+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.993+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v108'500 uv500 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.993+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:02.996 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.994+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v108'501 uv501 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.994+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa269b90 con 0x55d7aa1b8af0 2023-10-06T05:24:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.995+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'502 uv502 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.996+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa269b90 con 0x55d7aa1b8af0 2023-10-06T05:24:02.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.997+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'503 uv503 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.997+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa268b80 con 0x55d7aa1b8af0 2023-10-06T05:24:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.998+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'504 uv504 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa1b8af0 2023-10-06T05:24:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.998+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d7aa269090 0x55d7aa26b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.998+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26b9b0 con 0x55d7aa269090 2023-10-06T05:24:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.998+0000 7f7a1c827700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d7aa269090 0x55d7aa26b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:02.999+0000 7f7a1c827700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d7aa269090 0x55d7aa26b470 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.000+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'348 uv347 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.000+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26bd60 con 0x55d7aa269090 2023-10-06T05:24:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.002+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'349 uv349 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.002+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26c110 con 0x55d7aa269090 2023-10-06T05:24:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.003+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'350 uv350 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.005 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.003+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26c9b0 con 0x55d7aa269090 2023-10-06T05:24:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.005+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v108'351 uv351 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.005+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26cd60 con 0x55d7aa269090 2023-10-06T05:24:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.006+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'352 uv352 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.006+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d110 con 0x55d7aa269090 2023-10-06T05:24:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.007+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v108'353 uv335 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.007+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d4c0 con 0x55d7aa269090 2023-10-06T05:24:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.008+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v108'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.009+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.010+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v108'355 uv355 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.012 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.010+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.011+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'356 uv356 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.011+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.013+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'357 uv357 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.013+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.014+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v108'358 uv340 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.015+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.016+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v108'359 uv359 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.016+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.017+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v108'360 uv360 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.019 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:03.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.017+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270f90 con 0x55d7aa269090 2023-10-06T05:24:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.019+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v108'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.019+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270f90 con 0x55d7aa269090 2023-10-06T05:24:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.020+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'362 uv362 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.020+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.021+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v108'363 uv346 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.021+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.023+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v108'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.023+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.026 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.024+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v108'365 uv365 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.024+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26e880 con 0x55d7aa269090 2023-10-06T05:24:03.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.025+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'366 uv366 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.025+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26e880 con 0x55d7aa269090 2023-10-06T05:24:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.026+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'367 uv367 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.026+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26d870 con 0x55d7aa269090 2023-10-06T05:24:03.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.028+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'368 uv368 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.028+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d7aa26dd80 0x55d7aa270160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.028+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270730 con 0x55d7aa26dd80 2023-10-06T05:24:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.028+0000 7f7a17fff700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d7aa26dd80 0x55d7aa270160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.028+0000 7f7a17fff700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d7aa26dd80 0x55d7aa270160 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.030+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.030+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2707f0 con 0x55d7aa26dd80 2023-10-06T05:24:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.031+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v108'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.031+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.033 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:03.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.032+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v108'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.032+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa271cc0 con 0x55d7aa26dd80 2023-10-06T05:24:03.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.033+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.033+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa271cc0 con 0x55d7aa26dd80 2023-10-06T05:24:03.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.034+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.034+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.035+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.035+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.037+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'68 uv67 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.037+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.038+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'69 uv69 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.038+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.039+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'70 uv70 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.041 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.039+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.040+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'71 uv71 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.040+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa269090 2023-10-06T05:24:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.042+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.042+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa269090 2023-10-06T05:24:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.043+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v108'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.043+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa269090 2023-10-06T05:24:03.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.044+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v108'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.046 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:03.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.044+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa271cc0 con 0x55d7aa269090 2023-10-06T05:24:03.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.045+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.045+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa271cc0 con 0x55d7aa269090 2023-10-06T05:24:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.046+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.046+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa270cb0 con 0x55d7aa269090 2023-10-06T05:24:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.047+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa269090 2023-10-06T05:24:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.048+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d7aa271330 0x55d7aa273710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.048+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa273d20 con 0x55d7aa271330 2023-10-06T05:24:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.048+0000 7f7a1d028700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d7aa271330 0x55d7aa273710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.048+0000 7f7a1d028700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d7aa271330 0x55d7aa273710 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.049+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v108'171 uv165 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.049+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2741b0 con 0x55d7aa271330 2023-10-06T05:24:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.051+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v108'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.051+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa274630 con 0x55d7aa271330 2023-10-06T05:24:03.054 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.052+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v108'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.052+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa26c5e0 con 0x55d7aa271330 2023-10-06T05:24:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.053+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.053+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa276050 con 0x55d7aa271330 2023-10-06T05:24:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.054+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.054+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa274fa0 con 0x55d7aa271330 2023-10-06T05:24:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.056+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.056+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa274b30 con 0x55d7aa271330 2023-10-06T05:24:03.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.057+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v108'177 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.057+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2758b0 con 0x55d7aa271330 2023-10-06T05:24:03.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.058+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v108'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.058+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2758b0 con 0x55d7aa271330 2023-10-06T05:24:03.060 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.059+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v108'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.059+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2758b0 con 0x55d7aa271330 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.060+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.060+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2758b0 con 0x55d7aa271330 2023-10-06T05:24:03.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.061+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.062+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d7aa276980 0x55d7aa278d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.062+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2792a0 con 0x55d7aa276980 2023-10-06T05:24:03.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.062+0000 7f7a1c827700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d7aa276980 0x55d7aa278d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.062+0000 7f7a1c827700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d7aa276980 0x55d7aa278d60 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.063+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v108'76 uv75 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa276980 2023-10-06T05:24:03.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.063+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2792a0 con 0x55d7aa276980 2023-10-06T05:24:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.064+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v108'77 uv77 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa276980 2023-10-06T05:24:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.064+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2792a0 con 0x55d7aa276980 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.067+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v108'78 uv78 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa276980 2023-10-06T05:24:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.067+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2792a0 con 0x55d7aa276980 2023-10-06T05:24:03.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.068+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'79 uv79 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa276980 2023-10-06T05:24:03.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.068+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa275440 con 0x55d7aa276980 2023-10-06T05:24:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.071+0000 7f7a1c827700 1 -- 172.21.15.45:0/541859889 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'80 uv80 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a10007c10 con 0x55d7aa276980 2023-10-06T05:24:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.071+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa275440 con 0x55d7aa26dd80 2023-10-06T05:24:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.072+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.072+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa275440 con 0x55d7aa26dd80 2023-10-06T05:24:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.074+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v108'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.074+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa275440 con 0x55d7aa26dd80 2023-10-06T05:24:03.076 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.075+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v108'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.075+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa275440 con 0x55d7aa26dd80 2023-10-06T05:24:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.076+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.076+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa275440 con 0x55d7aa26dd80 2023-10-06T05:24:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.077+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.077+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa275440 con 0x55d7aa26dd80 2023-10-06T05:24:03.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.078+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.078+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d7aa279ca0 0x55d7aa27c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.078+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c5c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.078+0000 7f7a17fff700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d7aa279ca0 0x55d7aa27c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.079+0000 7f7a17fff700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d7aa279ca0 0x55d7aa27c080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.080+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v108'152 uv146 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.080+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c5c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.082+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v108'153 uv153 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.082+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c9c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.083+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v108'154 uv154 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.085 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:03.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.083+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c9c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.084+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'155 uv155 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.084+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c9c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.085+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'156 uv156 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.085+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c9c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.086+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v108'157 uv151 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.086+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c9c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.087+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v108'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.087+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c9c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.088+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v108'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.090 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:03.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.088+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27c9c0 con 0x55d7aa279ca0 2023-10-06T05:24:03.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.089+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.089+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27f730 con 0x55d7aa279ca0 2023-10-06T05:24:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.091+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa279ca0 2023-10-06T05:24:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.091+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.092+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v108'70 uv69 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.092+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.093+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v108'71 uv71 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.093+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.096 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.094+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v108'72 uv72 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.094+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.095+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v108'73 uv73 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.095+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d7aa27d7c0 0x55d7aa27fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.095+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa27d7c0 2023-10-06T05:24:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.095+0000 7f7a1d028700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d7aa27d7c0 0x55d7aa27fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.096+0000 7f7a1d028700 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d7aa27d7c0 0x55d7aa27fe80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.097+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa27d7c0 2023-10-06T05:24:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.097+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa27d7c0 2023-10-06T05:24:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.098+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v108'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa27d7c0 2023-10-06T05:24:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.098+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa27d7c0 2023-10-06T05:24:03.101 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.099+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v108'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa27d7c0 2023-10-06T05:24:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.099+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa27d7c0 2023-10-06T05:24:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.100+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa27d7c0 2023-10-06T05:24:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.100+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa27d7c0 2023-10-06T05:24:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.101+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa27d7c0 2023-10-06T05:24:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.102+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa27d7c0 2023-10-06T05:24:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.103+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa27d7c0 2023-10-06T05:24:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.103+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa271330 2023-10-06T05:24:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.104+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v108'79 uv78 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.104+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa271330 2023-10-06T05:24:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.105+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v108'80 uv80 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.105+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa271330 2023-10-06T05:24:03.108 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.106+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v108'81 uv81 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.106+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa271330 2023-10-06T05:24:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.108+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v108'82 uv82 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.108+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa271330 2023-10-06T05:24:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.109+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v108'83 uv83 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7a18053860 con 0x55d7aa271330 2023-10-06T05:24:03.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.109+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.110+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v108'192 uv186 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.110+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.112+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v108'193 uv193 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.112+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.115 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.113+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v108'194 uv194 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.113+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.114+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v108'195 uv195 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.114+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.119+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v108'196 uv196 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.119+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.120+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v108'197 uv191 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.120+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.123+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v108'198 uv198 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.123+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.128 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.126+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v108'199 uv199 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.126+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.129+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v108'200 uv200 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.129+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.131+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v108'201 uv201 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.131+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa27ceb0 con 0x55d7aa26dd80 2023-10-06T05:24:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.132+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v108'202 uv202 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.132+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x55d7aa2803c0 con 0x55d7aa26dd80 2023-10-06T05:24:03.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a17fff700 1 -- 172.21.15.45:0/541859889 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v108'203 uv203 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7a08016070 con 0x55d7aa26dd80 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d7aa26dd80 msgr2=0x55d7aa270160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d7aa26dd80 0x55d7aa270160 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d7aa1b8af0 msgr2=0x55d7aa256820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d7aa1b8af0 0x55d7aa256820 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d7aa279ca0 msgr2=0x55d7aa27c080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d7aa279ca0 0x55d7aa27c080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d7aa269090 msgr2=0x55d7aa26b470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d7aa269090 0x55d7aa26b470 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.135+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 reap_dead start 2023-10-06T05:24:03.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d7aa27d7c0 msgr2=0x55d7aa27fe80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d7aa27d7c0 0x55d7aa27fe80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d7aa276980 msgr2=0x55d7aa278d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d7aa276980 0x55d7aa278d60 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d7aa271330 msgr2=0x55d7aa273710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d7aa271330 0x55d7aa273710 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a00065890 msgr2=0x7f7a00067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7a00065890 0x7f7a00067d40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7a10005fd0 tx=0x7f7a1000c040 comp rx=0 tx=0).stop 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0030b0 msgr2=0x55d7aa23dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a1d028700 1 -- 172.21.15.45:0/541859889 reap_dead start 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0030b0 0x55d7aa23dca0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f7a18013eb0 tx=0x7f7a18012000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 shutdown_connections 2023-10-06T05:24:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 --2- 172.21.15.45:0/541859889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0030b0 0x55d7aa23dca0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 >> 172.21.15.45:0/541859889 conn(0x55d7aa174450 msgr2=0x55d7aa245c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 shutdown_connections 2023-10-06T05:24:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.136+0000 7f7a23609040 1 -- 172.21.15.45:0/541859889 wait complete. 2023-10-06T05:24:03.142 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2023-10-06T05:24:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.191+0000 7f2f9c160040 1 Processor -- start 2023-10-06T05:24:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.191+0000 7f2f9c160040 1 -- start start 2023-10-06T05:24:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f9c160040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f9c160040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6aeb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f9c160040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f9c160040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e6a8f320 con 0x5636e6ba8c60 2023-10-06T05:24:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f9c160040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e6a33980 con 0x5636e6ba8890 2023-10-06T05:24:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f9c160040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e6a33620 con 0x5636e6ae6210 2023-10-06T05:24:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f95b7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f95b7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b171b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38906/0 (socket says 172.21.15.45:38906) 2023-10-06T05:24:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f95b7f700 1 -- 172.21.15.45:0/81851527 learned_addr learned my addr 172.21.15.45:0/81851527 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6aeb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f9537e700 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f94b7d700 1 -- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 msgr2=0x5636e6bc40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f94b7d700 1 -- 172.21.15.45:0/81851527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 msgr2=0x5636e6b171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f94b7d700 1 -- 172.21.15.45:0/81851527 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88005cf0 con 0x5636e6ba8890 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f95b7f700 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.192+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6aeb830 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006f90 tx=0x7f2f80006860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=515d8b2ce3439c11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/81851527 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000bc80 con 0x5636e6ba8890 2023-10-06T05:24:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/81851527 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8000be00 con 0x5636e6ba8890 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f8f7fe700 1 -- 172.21.15.45:0/81851527 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000e6c0 con 0x5636e6ba8890 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 -- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 msgr2=0x5636e6aeb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6aeb830 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006f90 tx=0x7f2f80006860 comp rx=0 tx=0).stop 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 -- 172.21.15.45:0/81851527 shutdown_connections 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 --2- 172.21.15.45:0/81851527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6aeb830 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 -- 172.21.15.45:0/81851527 >> 172.21.15.45:0/81851527 conn(0x5636e6ae9450 msgr2=0x5636e6b2c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 -- 172.21.15.45:0/81851527 shutdown_connections 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.193+0000 7f2f9c160040 1 -- 172.21.15.45:0/81851527 wait complete. 2023-10-06T05:24:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.194+0000 7f2f9c160040 1 Processor -- start 2023-10-06T05:24:03.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.194+0000 7f2f9c160040 1 -- start start 2023-10-06T05:24:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.195+0000 7f2f9c160040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.195+0000 7f2f9c160040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6b2dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.195+0000 7f2f9c160040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b2e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.195+0000 7f2f9c160040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e6a2a400 con 0x5636e6ba8c60 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.195+0000 7f2f9c160040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e6bc1ad0 con 0x5636e6ba8890 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.195+0000 7f2f9c160040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e6b17930 con 0x5636e6ae6210 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.195+0000 7f2f95b7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b2e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38916/0 (socket says 172.21.15.45:38916) 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f9537e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 -- 172.21.15.45:0/4074546428 learned_addr learned my addr 172.21.15.45:0/4074546428 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6b2dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 -- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 msgr2=0x5636e6bc1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 -- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 msgr2=0x5636e6b2dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6b2dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 -- 172.21.15.45:0/4074546428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90006d40 con 0x5636e6ba8c60 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f9537e700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636e6ae6210 0x5636e6bc1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636e6ba8890 0x5636e6b2dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f95b7f700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b2e260 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000e700 tx=0x7f2f90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f8dffb700 1 -- 172.21.15.45:0/4074546428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f900126d0 con 0x5636e6ba8c60 2023-10-06T05:24:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f8dffb700 1 -- 172.21.15.45:0/4074546428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f90012830 con 0x5636e6ba8c60 2023-10-06T05:24:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f80006b40 con 0x5636e6ba8c60 2023-10-06T05:24:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f8dffb700 1 -- 172.21.15.45:0/4074546428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90012bd0 con 0x5636e6ba8c60 2023-10-06T05:24:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.196+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e6a35ae0 con 0x5636e6ba8c60 2023-10-06T05:24:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.198+0000 7f2f8dffb700 1 -- 172.21.15.45:0/4074546428 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f90003030 con 0x5636e6ba8c60 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.198+0000 7f2f8dffb700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f78065890 0x7f2f78067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.199+0000 7f2f8dffb700 1 -- 172.21.15.45:0/4074546428 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f2f90083c30 con 0x5636e6ba8c60 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.199+0000 7f2f9537e700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f78065890 0x7f2f78067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.199+0000 7f2f9c160040 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5636e6b2c550 0x5636e6bc61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.199+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v8 -- 0x5636e6bc31c0 con 0x5636e6b2c550 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.199+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5636e6b2c550 0x5636e6bc61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.199+0000 7f2f9537e700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f78065890 0x7f2f78067d40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800c010 tx=0x7f2f88014000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.199+0000 7f2f94b7d700 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5636e6b2c550 0x5636e6bc61c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f94b7d700 1 -- 172.21.15.45:0/4074546428 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v108'369 uv369 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2f8000eab0 con 0x5636e6b2c550 2023-10-06T05:24:03.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5636e6b2c550 msgr2=0x5636e6bc61c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5636e6b2c550 0x5636e6bc61c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f78065890 msgr2=0x7f2f78067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2f78065890 0x7f2f78067d40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800c010 tx=0x7f2f88014000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 msgr2=0x5636e6b2e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 --2- 172.21.15.45:0/4074546428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636e6ba8c60 0x5636e6b2e260 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000e700 tx=0x7f2f90008040 comp rx=0 tx=0).stop 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f95b7f700 1 -- 172.21.15.45:0/4074546428 reap_dead start 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 shutdown_connections 2023-10-06T05:24:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 >> 172.21.15.45:0/4074546428 conn(0x5636e6ae9450 msgr2=0x5636e6b2d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 shutdown_connections 2023-10-06T05:24:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.201+0000 7f2f9c160040 1 -- 172.21.15.45:0/4074546428 wait complete. 2023-10-06T05:24:03.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:03.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.391+0000 7fd40b578700 1 Processor -- start 2023-10-06T05:24:03.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.391+0000 7fd40b578700 1 -- start start 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd40b578700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd40406ae40 0x7fd40415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd40b578700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd40b578700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd404067d80 con 0x7fd40406aa70 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd40b578700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4040ac620 con 0x7fd40406ae40 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd40b578700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40415cb70 con 0x7fd40415c6f0 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd408b13700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd40406ae40 0x7fd40415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd409314700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd40406aa70 0x7fd4040c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42382/0 (socket says 172.21.15.45:42382) 2023-10-06T05:24:03.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.392+0000 7fd409314700 1 -- 172.21.15.45:0/1939665423 learned_addr learned my addr 172.21.15.45:0/1939665423 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd409b15700 1 --2- 172.21.15.45:0/1939665423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd408b13700 1 -- 172.21.15.45:0/1939665423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 msgr2=0x7fd4040aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd408b13700 1 --2- 172.21.15.45:0/1939665423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd408b13700 1 -- 172.21.15.45:0/1939665423 >> v1:172.21.15.45:6789/0 conn(0x7fd40406aa70 legacy=0x7fd4040c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd408b13700 1 -- 172.21.15.45:0/1939665423 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4040c75b0 con 0x7fd40406ae40 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd3fbfff700 1 -- 172.21.15.45:0/1939665423 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873111149 0 0) 0x7fd404067d80 con 0x7fd40406aa70 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd408b13700 1 --2- 172.21.15.45:0/1939665423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406ae40 0x7fd40415c1b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40098e0 tx=0x7fd3f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8f191ea53a3b10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd3fbfff700 1 -- 172.21.15.45:0/1939665423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4009e80 con 0x7fd40406ae40 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd3fbfff700 1 -- 172.21.15.45:0/1939665423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f400b860 con 0x7fd40406ae40 2023-10-06T05:24:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.393+0000 7fd3fbfff700 1 -- 172.21.15.45:0/1939665423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f400bbe0 con 0x7fd40406ae40 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 -- 172.21.15.45:0/1939665423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406ae40 msgr2=0x7fd40415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 --2- 172.21.15.45:0/1939665423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406ae40 0x7fd40415c1b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40098e0 tx=0x7fd3f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 -- 172.21.15.45:0/1939665423 shutdown_connections 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 --2- 172.21.15.45:0/1939665423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 --2- 172.21.15.45:0/1939665423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406ae40 0x7fd40415c1b0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 -- 172.21.15.45:0/1939665423 >> 172.21.15.45:0/1939665423 conn(0x7fd4040cc570 msgr2=0x7fd4040cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 -- 172.21.15.45:0/1939665423 shutdown_connections 2023-10-06T05:24:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.394+0000 7fd40b578700 1 -- 172.21.15.45:0/1939665423 wait complete. 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.395+0000 7fd40b578700 1 Processor -- start 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.396+0000 7fd40b578700 1 -- start start 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd40b578700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406aa70 0x7fd4040c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd40b578700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40406ae40 0x7fd404154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd40b578700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd40b578700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040c2980 con 0x7fd40406aa70 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd40b578700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040c1d70 con 0x7fd40406ae40 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd40b578700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040c2050 con 0x7fd40415c6f0 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406aa70 0x7fd4040c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406aa70 0x7fd4040c14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38926/0 (socket says 172.21.15.45:38926) 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 -- 172.21.15.45:0/3659042947 learned_addr learned my addr 172.21.15.45:0/3659042947 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409b15700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd408b13700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40406ae40 0x7fd404154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 -- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 msgr2=0x7fd4040cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 -- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40406ae40 msgr2=0x7fd404154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40406ae40 0x7fd404154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 -- 172.21.15.45:0/3659042947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f4009420 con 0x7fd40406aa70 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd408b13700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40406ae40 0x7fd404154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.397+0000 7fd409314700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406aa70 0x7fd4040c14e0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c7e0 tx=0x7fd400007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.398+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3659042947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400005b30 con 0x7fd40406aa70 2023-10-06T05:24:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.398+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3659042947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd400005c90 con 0x7fd40406aa70 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.398+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4040c2330 con 0x7fd40406aa70 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.398+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3659042947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400007300 con 0x7fd40406aa70 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.398+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4040c1a20 con 0x7fd40406aa70 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.399+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40406e300 con 0x7fd40406aa70 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.399+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3659042947 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd40001b400 con 0x7fd40406aa70 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.400+0000 7fd3f97fa700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e8066f30 0x7fd3e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.400+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3659042947 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fd400084950 con 0x7fd40406aa70 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.400+0000 7fd408b13700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e8066f30 0x7fd3e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.400+0000 7fd408b13700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e8066f30 0x7fd3e80693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd40415bdc0 tx=0x7fd3f4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.404+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3659042947 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4000513a0 con 0x7fd40406aa70 2023-10-06T05:24:03.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.651+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd4040c7890 con 0x7fd40406aa70 2023-10-06T05:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.891+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3659042947 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v193) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd4000549c0 con 0x7fd40406aa70 2023-10-06T05:24:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.894+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e8066f30 msgr2=0x7fd3e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.894+0000 7fd40b578700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e8066f30 0x7fd3e80693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd40415bdc0 tx=0x7fd3f4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.894+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406aa70 msgr2=0x7fd4040c14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.894+0000 7fd40b578700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406aa70 0x7fd4040c14e0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c7e0 tx=0x7fd400007df0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 shutdown_connections 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd3e8066f30 0x7fd3e80693e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40415c6f0 0x7fd4040cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40406aa70 0x7fd4040c14e0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 --2- 172.21.15.45:0/3659042947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40406ae40 0x7fd404154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 >> 172.21.15.45:0/3659042947 conn(0x7fd4040cc570 msgr2=0x7fd404067770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 shutdown_connections 2023-10-06T05:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:03.895+0000 7fd40b578700 1 -- 172.21.15.45:0/3659042947 wait complete. 2023-10-06T05:24:03.898 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:03.900 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:03.899+0000 7fabcad33700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2023-10-06T05:24:03.901 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:03.900+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:03.902 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:03.900+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:03.909 INFO:tasks.ceph.mds.c.smithi045.stderr: -13> 2023-10-06T05:24:03.899+0000 7fabcad33700 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2023-10-06T05:24:03.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:03.933 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:24:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.048+0000 7f4186d8a700 1 Processor -- start 2023-10-06T05:24:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.048+0000 7f4186d8a700 1 -- start start 2023-10-06T05:24:04.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4186d8a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f41800cd4d0 0x7f418015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4186d8a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800ae8d0 con 0x7f4180155cd0 2023-10-06T05:24:04.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4186d8a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800aebb0 con 0x7f41801560a0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4186d8a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41800ae6b0 con 0x7f41800cd4d0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4184b26700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4180155cd0 0x7f41800adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42396/0 (socket says 172.21.15.45:42396) 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4184b26700 1 -- 172.21.15.45:0/1409476643 learned_addr learned my addr 172.21.15.45:0/1409476643 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4185327700 1 --2- 172.21.15.45:0/1409476643 >> v2:172.21.15.144:3300/0 conn(0x7f41800cd4d0 0x7f418015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f417f7fe700 1 -- 172.21.15.45:0/1409476643 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395026023 0 0) 0x7f41800ae8d0 con 0x7f4180155cd0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f417f7fe700 1 -- 172.21.15.45:0/1409476643 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f416c003570 con 0x7f4180155cd0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f417f7fe700 1 -- 172.21.15.45:0/1409476643 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057570845 0 0) 0x7f41800aebb0 con 0x7f41801560a0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f417f7fe700 1 -- 172.21.15.45:0/1409476643 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41800ae8d0 con 0x7f41801560a0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4185327700 1 -- 172.21.15.45:0/1409476643 >> v1:172.21.15.144:6789/0 conn(0x7f41801560a0 legacy=0x7f41800ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4185327700 1 -- 172.21.15.45:0/1409476643 >> v1:172.21.15.45:6789/0 conn(0x7f4180155cd0 legacy=0x7f41800adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.050+0000 7f4185327700 1 -- 172.21.15.45:0/1409476643 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800aee90 con 0x7f41800cd4d0 2023-10-06T05:24:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4185327700 1 --2- 172.21.15.45:0/1409476643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 0x7f418015a730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4178009e60 tx=0x7f4178009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a820a68354080 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f417f7fe700 1 -- 172.21.15.45:0/1409476643 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800e070 con 0x7f41800cd4d0 2023-10-06T05:24:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f417f7fe700 1 -- 172.21.15.45:0/1409476643 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417800bab0 con 0x7f41800cd4d0 2023-10-06T05:24:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f417f7fe700 1 -- 172.21.15.45:0/1409476643 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800be70 con 0x7f41800cd4d0 2023-10-06T05:24:04.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4186d8a700 1 -- 172.21.15.45:0/1409476643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 msgr2=0x7f418015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1409476643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 0x7f418015a730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4178009e60 tx=0x7f4178009780 comp rx=0 tx=0).stop 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4186d8a700 1 -- 172.21.15.45:0/1409476643 shutdown_connections 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1409476643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 0x7f418015a730 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4186d8a700 1 -- 172.21.15.45:0/1409476643 >> 172.21.15.45:0/1409476643 conn(0x7f418006f090 msgr2=0x7f418006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4186d8a700 1 -- 172.21.15.45:0/1409476643 shutdown_connections 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.051+0000 7f4186d8a700 1 -- 172.21.15.45:0/1409476643 wait complete. 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.053+0000 7f4186d8a700 1 Processor -- start 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.053+0000 7f4186d8a700 1 -- start start 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4186d8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 0x7f4180167160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4186d8a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4180155cd0 0x7f4180169df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4186d8a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41801560a0 0x7f41800a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4186d8a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4180155090 con 0x7f4180155cd0 2023-10-06T05:24:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4186d8a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c3e40 con 0x7f41800cd4d0 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4186d8a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c49c0 con 0x7f41801560a0 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4180155cd0 0x7f4180169df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4180155cd0 0x7f4180169df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38932/0 (socket says 172.21.15.45:38932) 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 -- 172.21.15.45:0/1393137673 learned_addr learned my addr 172.21.15.45:0/1393137673 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4185327700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41801560a0 0x7f41800a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f4184b26700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 0x7f4180167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 -- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41801560a0 msgr2=0x7f41800a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41801560a0 0x7f41800a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 -- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 msgr2=0x7f4180167160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 0x7f4180167160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 -- 172.21.15.45:0/1393137673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178009a20 con 0x7f4180155cd0 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417ffff700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4180155cd0 0x7f4180169df0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f416800c7c0 tx=0x7f4168007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417cff9700 1 -- 172.21.15.45:0/1393137673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168010510 con 0x7f4180155cd0 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.054+0000 7f417cff9700 1 -- 172.21.15.45:0/1393137673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4168010670 con 0x7f4180155cd0 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.055+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41801676a0 con 0x7f4180155cd0 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.055+0000 7f417cff9700 1 -- 172.21.15.45:0/1393137673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168010a10 con 0x7f4180155cd0 2023-10-06T05:24:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.055+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41800c43f0 con 0x7f4180155cd0 2023-10-06T05:24:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.056+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418006e2f0 con 0x7f4180155cd0 2023-10-06T05:24:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.056+0000 7f417cff9700 1 -- 172.21.15.45:0/1393137673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f416801b3b0 con 0x7f4180155cd0 2023-10-06T05:24:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.056+0000 7f417cff9700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4160066f30 0x7f41600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.057+0000 7f417cff9700 1 -- 172.21.15.45:0/1393137673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f4168084580 con 0x7f4180155cd0 2023-10-06T05:24:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.057+0000 7f4184b26700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4160066f30 0x7f41600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.057+0000 7f4184b26700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4160066f30 0x7f41600693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4170007fd0 tx=0x7f4170009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.061+0000 7f417cff9700 1 -- 172.21.15.45:0/1393137673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4168050fa0 con 0x7f4180155cd0 2023-10-06T05:24:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.308+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41800c4b00 con 0x7f4180155cd0 2023-10-06T05:24:04.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.308+0000 7f417cff9700 1 -- 172.21.15.45:0/1393137673 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f4168017020 con 0x7f4180155cd0 2023-10-06T05:24:04.310 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:04.311 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":195,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:03.901285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.310+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4160066f30 msgr2=0x7f41600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.310+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4160066f30 0x7f41600693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4170007fd0 tx=0x7f4170009040 comp rx=0 tx=0).stop 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.310+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4180155cd0 msgr2=0x7f4180169df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.310+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4180155cd0 0x7f4180169df0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f416800c7c0 tx=0x7f4168007e10 comp rx=0 tx=0).stop 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 shutdown_connections 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4160066f30 0x7f41600693e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41801560a0 0x7f41800a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41800cd4d0 0x7f4180167160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 --2- 172.21.15.45:0/1393137673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4180155cd0 0x7f4180169df0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 >> 172.21.15.45:0/1393137673 conn(0x7f418006f090 msgr2=0x7f41800c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 shutdown_connections 2023-10-06T05:24:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.311+0000 7f4186d8a700 1 -- 172.21.15.45:0/1393137673 wait complete. 2023-10-06T05:24:04.314 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 195 2023-10-06T05:24:04.326 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:04.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.498+0000 7f6f2a71a700 1 Processor -- start 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.498+0000 7f6f2a71a700 1 -- start start 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f2a71a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f24153620 0x7f6f2415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f2a71a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f241539f0 0x7f6f2415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f2a71a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6f240cc8b0 0x7f6f240ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f2a71a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2415a7a0 con 0x7f6f241539f0 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f2a71a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f240ce3f0 con 0x7f6f240cc8b0 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f2a71a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2415d300 con 0x7f6f24153620 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f237fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f241539f0 0x7f6f2415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f237fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f241539f0 0x7f6f2415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38944/0 (socket says 172.21.15.45:38944) 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.499+0000 7f6f237fe700 1 -- 172.21.15.45:0/1805876238 learned_addr learned my addr 172.21.15.45:0/1805876238 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f23fff700 1 --2- 172.21.15.45:0/1805876238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f24153620 0x7f6f2415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f28cb7700 1 --2- 172.21.15.45:0/1805876238 >> v2:172.21.15.144:3300/0 conn(0x7f6f240cc8b0 0x7f6f240ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f237fe700 1 -- 172.21.15.45:0/1805876238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f24153620 msgr2=0x7f6f2415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f237fe700 1 --2- 172.21.15.45:0/1805876238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f24153620 0x7f6f2415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f237fe700 1 -- 172.21.15.45:0/1805876238 >> v2:172.21.15.144:3300/0 conn(0x7f6f240cc8b0 msgr2=0x7f6f240ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f237fe700 1 --2- 172.21.15.45:0/1805876238 >> v2:172.21.15.144:3300/0 conn(0x7f6f240cc8b0 0x7f6f240ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f237fe700 1 -- 172.21.15.45:0/1805876238 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240ce690 con 0x7f6f241539f0 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f28cb7700 1 --2- 172.21.15.45:0/1805876238 >> v2:172.21.15.144:3300/0 conn(0x7f6f240cc8b0 0x7f6f240ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f237fe700 1 --2- 172.21.15.45:0/1805876238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f241539f0 0x7f6f2415cdc0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f6f24157310 tx=0x7f6f18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33597be822464108 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f22ffd700 1 -- 172.21.15.45:0/1805876238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800b860 con 0x7f6f241539f0 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f22ffd700 1 -- 172.21.15.45:0/1805876238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1800b9c0 con 0x7f6f241539f0 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f22ffd700 1 -- 172.21.15.45:0/1805876238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800bd60 con 0x7f6f241539f0 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f2a71a700 1 -- 172.21.15.45:0/1805876238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f241539f0 msgr2=0x7f6f2415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.500+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/1805876238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f241539f0 0x7f6f2415cdc0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f6f24157310 tx=0x7f6f18009250 comp rx=0 tx=0).stop 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.501+0000 7f6f2a71a700 1 -- 172.21.15.45:0/1805876238 shutdown_connections 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.501+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/1805876238 >> v2:172.21.15.144:3300/0 conn(0x7f6f240cc8b0 0x7f6f240ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.501+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/1805876238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f24153620 0x7f6f2415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.501+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/1805876238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f241539f0 0x7f6f2415cdc0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.501+0000 7f6f2a71a700 1 -- 172.21.15.45:0/1805876238 >> 172.21.15.45:0/1805876238 conn(0x7f6f2406ed50 msgr2=0x7f6f2406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.501+0000 7f6f2a71a700 1 -- 172.21.15.45:0/1805876238 shutdown_connections 2023-10-06T05:24:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.501+0000 7f6f2a71a700 1 -- 172.21.15.45:0/1805876238 wait complete. 2023-10-06T05:24:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.502+0000 7f6f2a71a700 1 Processor -- start 2023-10-06T05:24:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.502+0000 7f6f2a71a700 1 -- start start 2023-10-06T05:24:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f2a71a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f240cc8b0 0x7f6f2416d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f2a71a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24153620 0x7f6f24141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f2a71a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f240b3400 0x7f6f240c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f2a71a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240ca0d0 con 0x7f6f240cc8b0 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f2a71a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2416d350 con 0x7f6f24153620 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f2a71a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240c9b10 con 0x7f6f240b3400 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f240cc8b0 0x7f6f2416d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f240cc8b0 0x7f6f2416d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38952/0 (socket says 172.21.15.45:38952) 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 -- 172.21.15.45:0/214466266 learned_addr learned my addr 172.21.15.45:0/214466266 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f28cb7700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f240b3400 0x7f6f240c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f237fe700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24153620 0x7f6f24141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 -- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f240b3400 msgr2=0x7f6f240c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f240b3400 0x7f6f240c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 -- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24153620 msgr2=0x7f6f24141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24153620 0x7f6f24141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.503+0000 7f6f23fff700 1 -- 172.21.15.45:0/214466266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180094f0 con 0x7f6f240cc8b0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.504+0000 7f6f23fff700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f240cc8b0 0x7f6f2416d7e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c7d0 tx=0x7f6f14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.504+0000 7f6f0bfff700 1 -- 172.21.15.45:0/214466266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14011600 con 0x7f6f240cc8b0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.504+0000 7f6f0bfff700 1 -- 172.21.15.45:0/214466266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14011760 con 0x7f6f240cc8b0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.504+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f240c9df0 con 0x7f6f240cc8b0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.504+0000 7f6f0bfff700 1 -- 172.21.15.45:0/214466266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14011b20 con 0x7f6f240cc8b0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.504+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2416dd20 con 0x7f6f240cc8b0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.505+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f24088a60 con 0x7f6f240cc8b0 2023-10-06T05:24:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.505+0000 7f6f0bfff700 1 -- 172.21.15.45:0/214466266 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f140193f0 con 0x7f6f240cc8b0 2023-10-06T05:24:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.506+0000 7f6f0bfff700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f10066f30 0x7f6f100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.506+0000 7f6f0bfff700 1 -- 172.21.15.45:0/214466266 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f6f14085020 con 0x7f6f240cc8b0 2023-10-06T05:24:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.506+0000 7f6f237fe700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f10066f30 0x7f6f100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.507+0000 7f6f237fe700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f10066f30 0x7f6f100693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6f240b2c90 tx=0x7f6f18017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.510+0000 7f6f0bfff700 1 -- 172.21.15.45:0/214466266 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f14051a40 con 0x7f6f240cc8b0 2023-10-06T05:24:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.757+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f2416d630 con 0x7f6f240cc8b0 2023-10-06T05:24:04.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.758+0000 7f6f0bfff700 1 -- 172.21.15.45:0/214466266 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f6f14055060 con 0x7f6f240cc8b0 2023-10-06T05:24:04.760 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:04.760 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":195,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:03.901285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f10066f30 msgr2=0x7f6f100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f10066f30 0x7f6f100693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6f240b2c90 tx=0x7f6f18017000 comp rx=0 tx=0).stop 2023-10-06T05:24:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f240cc8b0 msgr2=0x7f6f2416d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f240cc8b0 0x7f6f2416d7e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c7d0 tx=0x7f6f14008040 comp rx=0 tx=0).stop 2023-10-06T05:24:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 shutdown_connections 2023-10-06T05:24:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6f10066f30 0x7f6f100693e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f240b3400 0x7f6f240c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f240cc8b0 0x7f6f2416d7e0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 --2- 172.21.15.45:0/214466266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24153620 0x7f6f24141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 >> 172.21.15.45:0/214466266 conn(0x7f6f2406ed50 msgr2=0x7f6f24151cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 shutdown_connections 2023-10-06T05:24:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.760+0000 7f6f2a71a700 1 -- 172.21.15.45:0/214466266 wait complete. 2023-10-06T05:24:04.764 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 195 2023-10-06T05:24:04.775 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:04.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.906+0000 7fd2d4244700 1 Processor -- start 2023-10-06T05:24:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.906+0000 7fd2d4244700 1 -- start start 2023-10-06T05:24:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.907+0000 7fd2d4244700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc155cd0 0x7fd2cc15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.907+0000 7fd2d4244700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2cc0ccdc0 0x7fd2cc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.907+0000 7fd2d4244700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0cd740 con 0x7fd2cc1560a0 2023-10-06T05:24:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.907+0000 7fd2d4244700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2cc0ae6b0 con 0x7fd2cc0ccdc0 2023-10-06T05:24:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.907+0000 7fd2d4244700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0ae8d0 con 0x7fd2cc155cd0 2023-10-06T05:24:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d27e1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2cc0ccdc0 0x7fd2cc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d17df700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd2cc1560a0 0x7fd2cc0adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42398/0 (socket says 172.21.15.45:42398) 2023-10-06T05:24:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d17df700 1 -- 172.21.15.45:0/3258938587 learned_addr learned my addr 172.21.15.45:0/3258938587 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d1fe0700 1 --2- 172.21.15.45:0/3258938587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc155cd0 0x7fd2cc15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d27e1700 1 -- 172.21.15.45:0/3258938587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc155cd0 msgr2=0x7fd2cc15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d27e1700 1 --2- 172.21.15.45:0/3258938587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc155cd0 0x7fd2cc15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d27e1700 1 -- 172.21.15.45:0/3258938587 >> v1:172.21.15.45:6789/0 conn(0x7fd2cc1560a0 legacy=0x7fd2cc0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d27e1700 1 -- 172.21.15.45:0/3258938587 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0aebb0 con 0x7fd2cc0ccdc0 2023-10-06T05:24:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.908+0000 7fd2d27e1700 1 --2- 172.21.15.45:0/3258938587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc0ccdc0 0x7fd2cc0cd200 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009e60 tx=0x7fd2c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ecef9573182387 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.910+0000 7fd2d0fde700 1 -- 172.21.15.45:0/3258938587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400e070 con 0x7fd2cc0ccdc0 2023-10-06T05:24:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.910+0000 7fd2d0fde700 1 -- 172.21.15.45:0/3258938587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c400bb60 con 0x7fd2cc0ccdc0 2023-10-06T05:24:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.910+0000 7fd2d0fde700 1 -- 172.21.15.45:0/3258938587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4012430 con 0x7fd2cc0ccdc0 2023-10-06T05:24:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.911+0000 7fd2d4244700 1 -- 172.21.15.45:0/3258938587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc0ccdc0 msgr2=0x7fd2cc0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.911+0000 7fd2d4244700 1 --2- 172.21.15.45:0/3258938587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc0ccdc0 0x7fd2cc0cd200 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009e60 tx=0x7fd2c4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.911+0000 7fd2d4244700 1 -- 172.21.15.45:0/3258938587 shutdown_connections 2023-10-06T05:24:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.911+0000 7fd2d4244700 1 --2- 172.21.15.45:0/3258938587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc0ccdc0 0x7fd2cc0cd200 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.911+0000 7fd2d4244700 1 --2- 172.21.15.45:0/3258938587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc155cd0 0x7fd2cc15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.911+0000 7fd2d4244700 1 -- 172.21.15.45:0/3258938587 >> 172.21.15.45:0/3258938587 conn(0x7fd2cc06f090 msgr2=0x7fd2cc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.912+0000 7fd2d4244700 1 -- 172.21.15.45:0/3258938587 shutdown_connections 2023-10-06T05:24:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.912+0000 7fd2d4244700 1 -- 172.21.15.45:0/3258938587 wait complete. 2023-10-06T05:24:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.913+0000 7fd2d4244700 1 Processor -- start 2023-10-06T05:24:04.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.913+0000 7fd2d4244700 1 -- start start 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d4244700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc0ccdc0 0x7fd2cc0ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d4244700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc155cd0 0x7fd2cc16a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d4244700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2cc1560a0 0x7fd2cc0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d4244700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc152c80 con 0x7fd2cc155cd0 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d4244700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc156a50 con 0x7fd2cc1560a0 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d4244700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c61b0 con 0x7fd2cc0ccdc0 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc155cd0 0x7fd2cc16a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc155cd0 0x7fd2cc16a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38966/0 (socket says 172.21.15.45:38966) 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 -- 172.21.15.45:0/2573508719 learned_addr learned my addr 172.21.15.45:0/2573508719 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d1fe0700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc0ccdc0 0x7fd2cc0ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d27e1700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2cc1560a0 0x7fd2cc0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 -- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc0ccdc0 msgr2=0x7fd2cc0ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc0ccdc0 0x7fd2cc0ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 -- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2cc1560a0 msgr2=0x7fd2cc0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2cc1560a0 0x7fd2cc0b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.914+0000 7fd2d17df700 1 -- 172.21.15.45:0/2573508719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009a20 con 0x7fd2cc155cd0 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.915+0000 7fd2d17df700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc155cd0 0x7fd2cc16a960 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c560 tx=0x7fd2bc001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.915+0000 7fd2c1ffb700 1 -- 172.21.15.45:0/2573508719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc005b30 con 0x7fd2cc155cd0 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.915+0000 7fd2c1ffb700 1 -- 172.21.15.45:0/2573508719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc005c90 con 0x7fd2cc155cd0 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.915+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc0c5bf0 con 0x7fd2cc155cd0 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.915+0000 7fd2c1ffb700 1 -- 172.21.15.45:0/2573508719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc007320 con 0x7fd2cc155cd0 2023-10-06T05:24:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.915+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc1566e0 con 0x7fd2cc155cd0 2023-10-06T05:24:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.916+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc156bc0 con 0x7fd2cc155cd0 2023-10-06T05:24:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.916+0000 7fd2c1ffb700 1 -- 172.21.15.45:0/2573508719 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2bc007690 con 0x7fd2cc155cd0 2023-10-06T05:24:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.917+0000 7fd2c1ffb700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.917+0000 7fd2c1ffb700 1 -- 172.21.15.45:0/2573508719 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fd2bc0840c0 con 0x7fd2cc155cd0 2023-10-06T05:24:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.917+0000 7fd2d1fe0700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.918+0000 7fd2d1fe0700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8005e30 tx=0x7fd2c800b780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:04.921+0000 7fd2c1ffb700 1 -- 172.21.15.45:0/2573508719 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2bc050ae0 con 0x7fd2cc155cd0 2023-10-06T05:24:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.169+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2cc0aee90 con 0x7fd2cc155cd0 2023-10-06T05:24:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.169+0000 7fd2c1ffb700 1 -- 172.21.15.45:0/2573508719 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) v1 ==== 95+0+4596 (secure 0 0 0) 0x7fd2bc017090 con 0x7fd2cc155cd0 2023-10-06T05:24:05.171 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":196,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/858773096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":858773096},{"type":"v1","addr":"172.21.15.45:6835","nonce":858773096}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:03.901285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 msgr2=0x7fd2b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8005e30 tx=0x7fd2c800b780 comp rx=0 tx=0).stop 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc155cd0 msgr2=0x7fd2cc16a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc155cd0 0x7fd2cc16a960 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c560 tx=0x7fd2bc001020 comp rx=0 tx=0).stop 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 shutdown_connections 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2b0066f30 0x7fd2b00693e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2cc1560a0 0x7fd2cc0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2cc0ccdc0 0x7fd2cc0ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 --2- 172.21.15.45:0/2573508719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc155cd0 0x7fd2cc16a960 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 >> 172.21.15.45:0/2573508719 conn(0x7fd2cc06f090 msgr2=0x7fd2cc0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 shutdown_connections 2023-10-06T05:24:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.172+0000 7fd2d4244700 1 -- 172.21.15.45:0/2573508719 wait complete. 2023-10-06T05:24:05.175 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 196 2023-10-06T05:24:05.187 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2023-10-06T05:24:05.187 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2023-10-06T05:24:05.187 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:05.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:05.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:05.187 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:05.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:05.207 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:05.208 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:05.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.383+0000 7f158143c700 1 Processor -- start 2023-10-06T05:24:05.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.383+0000 7f158143c700 1 -- start start 2023-10-06T05:24:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.384+0000 7f158143c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f157c06c3d0 0x7f157c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.384+0000 7f158143c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.384+0000 7f158143c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c0b0790 con 0x7f157c156b70 2023-10-06T05:24:05.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.384+0000 7f158143c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c0c8e80 con 0x7f157c06c3d0 2023-10-06T05:24:05.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.384+0000 7f158143c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0c9160 con 0x7f157c15b890 2023-10-06T05:24:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f157c06c3d0 0x7f157c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:57812/0 (socket says 172.21.15.45:57812) 2023-10-06T05:24:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 -- 172.21.15.45:0/3139938901 learned_addr learned my addr 172.21.15.45:0/3139938901 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157b7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f157c156b70 0x7f157c156ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:54976/0 (socket says 172.21.15.45:54976) 2023-10-06T05:24:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 -- 172.21.15.45:0/3139938901 >> v2:172.21.15.144:3300/0 conn(0x7f157c06c3d0 msgr2=0x7f157c06c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 --2- 172.21.15.45:0/3139938901 >> v2:172.21.15.144:3300/0 conn(0x7f157c06c3d0 0x7f157c06c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 -- 172.21.15.45:0/3139938901 >> v1:172.21.15.144:6789/0 conn(0x7f157c156b70 legacy=0x7f157c156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 -- 172.21.15.45:0/3139938901 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0c9440 con 0x7f157c15b890 2023-10-06T05:24:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f157a7fc700 1 --2- 172.21.15.45:0/3139938901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0afe60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15640099b0 tx=0x7f1564009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447ab723173125f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f1579ffb700 1 -- 172.21.15.45:0/3139938901 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564009ea0 con 0x7f157c15b890 2023-10-06T05:24:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.385+0000 7f1579ffb700 1 -- 172.21.15.45:0/3139938901 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156400b860 con 0x7f157c15b890 2023-10-06T05:24:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f1579ffb700 1 -- 172.21.15.45:0/3139938901 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400bc00 con 0x7f157c15b890 2023-10-06T05:24:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 -- 172.21.15.45:0/3139938901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 msgr2=0x7f157c0afe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 --2- 172.21.15.45:0/3139938901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0afe60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15640099b0 tx=0x7f1564009250 comp rx=0 tx=0).stop 2023-10-06T05:24:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 -- 172.21.15.45:0/3139938901 shutdown_connections 2023-10-06T05:24:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 --2- 172.21.15.45:0/3139938901 >> v2:172.21.15.144:3300/0 conn(0x7f157c06c3d0 0x7f157c06c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 --2- 172.21.15.45:0/3139938901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0afe60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 -- 172.21.15.45:0/3139938901 >> 172.21.15.45:0/3139938901 conn(0x7f157c067620 msgr2=0x7f157c067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 -- 172.21.15.45:0/3139938901 shutdown_connections 2023-10-06T05:24:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.386+0000 7f158143c700 1 -- 172.21.15.45:0/3139938901 wait complete. 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.388+0000 7f158143c700 1 Processor -- start 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.388+0000 7f158143c700 1 -- start start 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.388+0000 7f158143c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c06c3d0 0x7f157c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.388+0000 7f158143c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c156b70 0x7f157c0c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.388+0000 7f158143c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f158143c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c06a800 con 0x7f157c156b70 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f158143c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0c7e70 con 0x7f157c06c3d0 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f158143c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0c4330 con 0x7f157c15b890 2023-10-06T05:24:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c156b70 0x7f157c0c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c156b70 0x7f157c0c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38968/0 (socket says 172.21.15.45:38968) 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 -- 172.21.15.45:0/4022534052 learned_addr learned my addr 172.21.15.45:0/4022534052 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157affd700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c06c3d0 0x7f157c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157b7fe700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 -- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 msgr2=0x7f157c0c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 -- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c06c3d0 msgr2=0x7f157c0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c06c3d0 0x7f157c0cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 -- 172.21.15.45:0/4022534052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15640094f0 con 0x7f157c156b70 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f157a7fc700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c156b70 0x7f157c0c38b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f157c0cb550 tx=0x7f1564017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f1572ffd700 1 -- 172.21.15.45:0/4022534052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400bbe0 con 0x7f157c156b70 2023-10-06T05:24:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f1572ffd700 1 -- 172.21.15.45:0/4022534052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156401a450 con 0x7f157c156b70 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c0c2af0 con 0x7f157c156b70 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.389+0000 7f1572ffd700 1 -- 172.21.15.45:0/4022534052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156401a7f0 con 0x7f157c156b70 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.390+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c0c30b0 con 0x7f157c156b70 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.391+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c06f160 con 0x7f157c156b70 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.391+0000 7f1572ffd700 1 -- 172.21.15.45:0/4022534052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f156401ab60 con 0x7f157c156b70 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.391+0000 7f1572ffd700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f155c066f30 0x7f155c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.392+0000 7f1572ffd700 1 -- 172.21.15.45:0/4022534052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f1564086770 con 0x7f157c156b70 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.392+0000 7f157affd700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f155c066f30 0x7f155c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.392+0000 7f157affd700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f155c066f30 0x7f155c0693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f157c15a280 tx=0x7f156c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:05.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.396+0000 7f1572ffd700 1 -- 172.21.15.45:0/4022534052 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1564053190 con 0x7f157c156b70 2023-10-06T05:24:05.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.646+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f157c0c97b0 con 0x7f157c156b70 2023-10-06T05:24:05.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.914+0000 7f1572ffd700 1 -- 172.21.15.45:0/4022534052 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v197) v1 ==== 161+0+0 (secure 0 0 0) 0x7f15640567b0 con 0x7f157c156b70 2023-10-06T05:24:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f155c066f30 msgr2=0x7f155c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f155c066f30 0x7f155c0693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f157c15a280 tx=0x7f156c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c156b70 msgr2=0x7f157c0c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c156b70 0x7f157c0c38b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f157c0cb550 tx=0x7f1564017000 comp rx=0 tx=0).stop 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 shutdown_connections 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f155c066f30 0x7f155c0693e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f157c15b890 0x7f157c0c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f157c06c3d0 0x7f157c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 --2- 172.21.15.45:0/4022534052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f157c156b70 0x7f157c0c38b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 >> 172.21.15.45:0/4022534052 conn(0x7f157c067620 msgr2=0x7f157c0acb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.917+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 shutdown_connections 2023-10-06T05:24:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:05.918+0000 7f158143c700 1 -- 172.21.15.45:0/4022534052 wait complete. 2023-10-06T05:24:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:05.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.071+0000 7fc836d9d700 1 Processor -- start 2023-10-06T05:24:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.071+0000 7fc836d9d700 1 -- start start 2023-10-06T05:24:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.072+0000 7fc836d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc830159200 0x7fc8300c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.072+0000 7fc836d9d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8300c7d00 con 0x7fc8301595d0 2023-10-06T05:24:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.072+0000 7fc836d9d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8300c86d0 con 0x7fc8300ccdc0 2023-10-06T05:24:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.072+0000 7fc836d9d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8300c7780 con 0x7fc830159200 2023-10-06T05:24:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.072+0000 7fc82ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc8301595d0 0x7fc83006b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42406/0 (socket says 172.21.15.45:42406) 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.072+0000 7fc82ffff700 1 -- 172.21.15.45:0/6738440 learned_addr learned my addr 172.21.15.45:0/6738440 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc834b39700 1 --2- 172.21.15.45:0/6738440 >> v2:172.21.15.144:3300/0 conn(0x7fc830159200 0x7fc8300c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833050047 0 0) 0x7fc8300c7d00 con 0x7fc8301595d0 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc818003570 con 0x7fc8301595d0 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643467972 0 0) 0x7fc8300c86d0 con 0x7fc8300ccdc0 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8300c7d00 con 0x7fc8300ccdc0 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756072724 0 0) 0x7fc818003570 con 0x7fc8301595d0 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 >> v2:172.21.15.144:3300/0 conn(0x7fc830159200 msgr2=0x7fc8300c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 --2- 172.21.15.45:0/6738440 >> v2:172.21.15.144:3300/0 conn(0x7fc830159200 0x7fc8300c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 >> v1:172.21.15.144:6789/0 conn(0x7fc8300ccdc0 legacy=0x7fc8300cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83006bb40 con 0x7fc8301595d0 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fc824001470 con 0x7fc8301595d0 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc834b39700 1 --2- 172.21.15.45:0/6738440 >> v2:172.21.15.144:3300/0 conn(0x7fc830159200 0x7fc8300c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 >> v1:172.21.15.45:6789/0 conn(0x7fc8301595d0 legacy=0x7fc83006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818003bf0 0x7fc818003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc818004810 0x7fc818004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc81800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300c7780 con 0x7fc8180051c0 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81800c0b0 con 0x7fc818004810 2023-10-06T05:24:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81800c270 con 0x7fc818003bf0 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc83533a700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc81800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.073+0000 7fc82ffff700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc818004810 0x7fc818004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc834b39700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818003bf0 0x7fc818003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc83533a700 1 -- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818003bf0 msgr2=0x7fc818003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc83533a700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818003bf0 0x7fc818003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc83533a700 1 -- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc818004810 msgr2=0x7fc818004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc83533a700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc818004810 0x7fc818004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc83533a700 1 -- 172.21.15.45:0/6738440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818003570 con 0x7fc8180051c0 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc83533a700 1 -- 172.21.15.45:0/6738440 reap_dead start 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc83533a700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc81800ba60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fc828002120 tx=0x7fc8280017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525a52cd867c72f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800c980 con 0x7fc8180051c0 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800cae0 con 0x7fc8180051c0 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.074+0000 7fc82f7fe700 1 -- 172.21.15.45:0/6738440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800ce80 con 0x7fc8180051c0 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.075+0000 7fc836d9d700 1 -- 172.21.15.45:0/6738440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 msgr2=0x7fc81800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.075+0000 7fc836d9d700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc81800ba60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fc828002120 tx=0x7fc8280017a0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.075+0000 7fc836d9d700 1 -- 172.21.15.45:0/6738440 shutdown_connections 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.075+0000 7fc836d9d700 1 --2- 172.21.15.45:0/6738440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc81800ba60 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.075+0000 7fc836d9d700 1 -- 172.21.15.45:0/6738440 >> 172.21.15.45:0/6738440 conn(0x7fc83006f090 msgr2=0x7fc83006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.075+0000 7fc836d9d700 1 -- 172.21.15.45:0/6738440 shutdown_connections 2023-10-06T05:24:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.075+0000 7fc836d9d700 1 -- 172.21.15.45:0/6738440 wait complete. 2023-10-06T05:24:06.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.076+0000 7fc836d9d700 1 Processor -- start 2023-10-06T05:24:06.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.077+0000 7fc836d9d700 1 -- start start 2023-10-06T05:24:06.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.077+0000 7fc836d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc8300c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.077+0000 7fc836d9d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc830169140 0x7fc83016a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.077+0000 7fc836d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8300c0850 0x7fc8300c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.077+0000 7fc836d9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300c49e0 con 0x7fc8180051c0 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.077+0000 7fc836d9d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300c1200 con 0x7fc8300c0850 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.077+0000 7fc836d9d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8301695b0 con 0x7fc830169140 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc8300c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc8300c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38992/0 (socket says 172.21.15.45:38992) 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 -- 172.21.15.45:0/701892155 learned_addr learned my addr 172.21.15.45:0/701892155 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc83533a700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8300c0850 0x7fc8300c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc82ffff700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc830169140 0x7fc83016a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 -- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc830169140 msgr2=0x7fc83016a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc830169140 0x7fc83016a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 -- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8300c0850 msgr2=0x7fc8300c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8300c0850 0x7fc8300c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 -- 172.21.15.45:0/701892155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300c2dc0 con 0x7fc8180051c0 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc82ffff700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc830169140 0x7fc83016a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc83533a700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8300c0850 0x7fc8300c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc834b39700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc8300c2790 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fc820011990 tx=0x7fc820008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc82cff9700 1 -- 172.21.15.45:0/701892155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820011ed0 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc82cff9700 1 -- 172.21.15.45:0/701892155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82000f8e0 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8300c30a0 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc82cff9700 1 -- 172.21.15.45:0/701892155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82000fc60 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.078+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8300c3660 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.079+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83006e2f0 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.080+0000 7fc82cff9700 1 -- 172.21.15.45:0/701892155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc820015460 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.080+0000 7fc82cff9700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc81c066f30 0x7fc81c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.080+0000 7fc82cff9700 1 -- 172.21.15.45:0/701892155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fc8200856c0 con 0x7fc8180051c0 2023-10-06T05:24:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.080+0000 7fc82ffff700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc81c066f30 0x7fc81c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.081+0000 7fc82ffff700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc81c066f30 0x7fc81c0693e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc824001c80 tx=0x7fc824009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.085+0000 7fc82cff9700 1 -- 172.21.15.45:0/701892155 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc820055af0 con 0x7fc8180051c0 2023-10-06T05:24:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.373+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fc83006be20 con 0x7fc8180051c0 2023-10-06T05:24:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.920+0000 7fc82cff9700 1 -- 172.21.15.45:0/701892155 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v198) v1 ==== 101+0+0 (secure 0 0 0) 0x7fc8200556e0 con 0x7fc8180051c0 2023-10-06T05:24:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc81c066f30 msgr2=0x7fc81c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc81c066f30 0x7fc81c0693e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc824001c80 tx=0x7fc824009040 comp rx=0 tx=0).stop 2023-10-06T05:24:06.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 msgr2=0x7fc8300c2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc8300c2790 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fc820011990 tx=0x7fc820008040 comp rx=0 tx=0).stop 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 shutdown_connections 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc81c066f30 0x7fc81c0693e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8300c0850 0x7fc8300c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8180051c0 0x7fc8300c2790 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 --2- 172.21.15.45:0/701892155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc830169140 0x7fc83016a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 >> 172.21.15.45:0/701892155 conn(0x7fc83006f090 msgr2=0x7fc8300cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 shutdown_connections 2023-10-06T05:24:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.923+0000 7fc836d9d700 1 -- 172.21.15.45:0/701892155 wait complete. 2023-10-06T05:24:06.927 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:06.940 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 Processor -- start 2023-10-06T05:24:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 -- start start 2023-10-06T05:24:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 0x55ced25d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669890 0x55ced268c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669c60 0x55ced268cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ced2550320 con 0x55ced2669c60 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ced24f4980 con 0x55ced25a7920 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.986+0000 7efe963cb040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ced24f4620 con 0x55ced2669890 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669c60 0x55ced268cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669c60 0x55ced268cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52566/0 (socket says 172.21.15.45:52566) 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 -- 172.21.15.45:0/2187422492 learned_addr learned my addr 172.21.15.45:0/2187422492 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 0x55ced25d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8ede8700 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669890 0x55ced268c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 -- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669890 msgr2=0x55ced268c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669890 0x55ced268c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 -- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 msgr2=0x55ced25d81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 0x55ced25d81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 -- 172.21.15.45:0/2187422492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8800a0e0 con 0x55ced2669c60 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8ede8700 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669890 0x55ced268c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8fdea700 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669c60 0x55ced268cdd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7efe88005d80 tx=0x7efe88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c2adb4972c45c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8dde6700 1 -- 172.21.15.45:0/2187422492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880032f0 con 0x55ced2669c60 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8dde6700 1 -- 172.21.15.45:0/2187422492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800fe00 con 0x55ced2669c60 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.987+0000 7efe8dde6700 1 -- 172.21.15.45:0/2187422492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88025470 con 0x55ced2669c60 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 -- 172.21.15.45:0/2187422492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669c60 msgr2=0x55ced268cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669c60 0x55ced268cdd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7efe88005d80 tx=0x7efe88014000 comp rx=0 tx=0).stop 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 -- 172.21.15.45:0/2187422492 shutdown_connections 2023-10-06T05:24:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669c60 0x55ced268cdd0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 0x55ced25d81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 --2- 172.21.15.45:0/2187422492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669890 0x55ced268c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 -- 172.21.15.45:0/2187422492 >> 172.21.15.45:0/2187422492 conn(0x55ced25aa450 msgr2=0x55ced25a7210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 -- 172.21.15.45:0/2187422492 shutdown_connections 2023-10-06T05:24:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.988+0000 7efe963cb040 1 -- 172.21.15.45:0/2187422492 wait complete. 2023-10-06T05:24:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 Processor -- start 2023-10-06T05:24:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 -- start start 2023-10-06T05:24:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 0x55ced26862e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669890 0x55ced25edba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669c60 0x55ced25aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ced24eb400 con 0x55ced2669890 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ced2689cf0 con 0x55ced25a7920 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe963cb040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ced25ac160 con 0x55ced2669c60 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.989+0000 7efe8ede8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669890 0x55ced25edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669890 0x55ced25edba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52568/0 (socket says 172.21.15.45:52568) 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 learned_addr learned my addr 172.21.15.45:0/1411630136 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 0x55ced26862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8fdea700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669c60 0x55ced25aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669c60 msgr2=0x55ced25aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced2669c60 0x55ced25aa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 msgr2=0x55ced26862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced25a7920 0x55ced26862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ced25033d0 con 0x55ced2669890 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe8ede8700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669890 0x55ced25edba0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7efe8400a200 tx=0x7efe840018c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe77fff700 1 -- 172.21.15.45:0/1411630136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8400ad90 con 0x55ced2669890 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe77fff700 1 -- 172.21.15.45:0/1411630136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe84008040 con 0x55ced2669890 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ced24f6ae0 con 0x55ced2669890 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe77fff700 1 -- 172.21.15.45:0/1411630136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840106c0 con 0x55ced2669890 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.990+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ced24f6540 con 0x55ced2669890 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.992+0000 7efe77fff700 1 -- 172.21.15.45:0/1411630136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe84010ab0 con 0x55ced2669890 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.992+0000 7efe77fff700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe7c065890 0x7efe7c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.992+0000 7efe77fff700 1 -- 172.21.15.45:0/1411630136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7efe84083990 con 0x55ced2669890 2023-10-06T05:24:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.992+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe7c065890 0x7efe7c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.998 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.993+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ced25abc00 0x55ced268ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.993+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced268b360 con 0x55ced25abc00 2023-10-06T05:24:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.993+0000 7efe8fdea700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ced25abc00 0x55ced268ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.993+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe7c065890 0x7efe7c067d40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efe80000fb0 tx=0x7efe80000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.993+0000 7efe8fdea700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ced25abc00 0x55ced268ae20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.995+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v109'505 uv479 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.995+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced268b740 con 0x55ced25abc00 2023-10-06T05:24:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.996+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v109'506 uv506 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.996+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269cac0 con 0x55ced25abc00 2023-10-06T05:24:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.998+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v109'507 uv507 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.000 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.998+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269de80 con 0x55ced25abc00 2023-10-06T05:24:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.999+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'508 uv508 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:06.999+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269e230 con 0x55ced25abc00 2023-10-06T05:24:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.000+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'509 uv509 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.000+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.001+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'510 uv510 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.002+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.003+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v109'511 uv485 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.003+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.004+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v109'512 uv512 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.004+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.005+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v109'513 uv513 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.007 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.005+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.006+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'514 uv514 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.006+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.007+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'515 uv515 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.008+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.009+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'516 uv516 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.009+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.010+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v109'517 uv492 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.010+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.011+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'518 uv518 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.011+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.013+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v109'519 uv519 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.015 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.013+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.014+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'520 uv520 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.014+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.015+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'521 uv521 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.015+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269ce70 con 0x55ced25abc00 2023-10-06T05:24:07.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.016+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'522 uv522 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.016+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced269d760 con 0x55ced25abc00 2023-10-06T05:24:07.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.018+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'523 uv523 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.018+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.019+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v109'524 uv498 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.019+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.020+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v109'525 uv525 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.020+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.021+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v109'526 uv526 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.023 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:07.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.021+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.023+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'527 uv527 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.023+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.024+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'528 uv528 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.024+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.025+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'529 uv529 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.025+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.027+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v109'530 uv504 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.027+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.028+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v109'531 uv531 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.028+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.031 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:07.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.029+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v109'532 uv532 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.030+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.031+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'533 uv533 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.031+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.032+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'534 uv534 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.032+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced25aafd0 con 0x55ced25abc00 2023-10-06T05:24:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.033+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'535 uv535 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced25abc00 2023-10-06T05:24:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.033+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ced25aafd0 0x55ced26a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.033+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a0b30 con 0x55ced25aafd0 2023-10-06T05:24:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.033+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ced25aafd0 0x55ced26a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.034+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ced25aafd0 0x55ced26a05f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.035+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'370 uv352 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.035+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a0ee0 con 0x55ced25aafd0 2023-10-06T05:24:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.036+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'371 uv371 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.036+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a1290 con 0x55ced25aafd0 2023-10-06T05:24:07.039 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.037+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'372 uv372 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.037+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a1b30 con 0x55ced25aafd0 2023-10-06T05:24:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.039+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v109'373 uv373 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.039+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a1ee0 con 0x55ced25aafd0 2023-10-06T05:24:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.040+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'374 uv374 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.040+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2290 con 0x55ced25aafd0 2023-10-06T05:24:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.041+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v109'375 uv369 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.041+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.042+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v109'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.042+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.043+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v109'377 uv377 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.045 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.043+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.045+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'378 uv378 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.045+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.046+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'379 uv379 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.046+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.047+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v109'380 uv362 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.047+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.048+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v109'381 uv381 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.048+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.049+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v109'382 uv382 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.051 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.049+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a5d60 con 0x55ced25aafd0 2023-10-06T05:24:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.051+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v109'383 uv383 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.051+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a5d60 con 0x55ced25aafd0 2023-10-06T05:24:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.052+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.052+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.053+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v109'385 uv368 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.053+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.054+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v109'386 uv386 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.055+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.057 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.056+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v109'387 uv387 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.056+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a3650 con 0x55ced25aafd0 2023-10-06T05:24:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.057+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'388 uv388 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.057+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a3650 con 0x55ced25aafd0 2023-10-06T05:24:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.058+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'389 uv389 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.058+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a2640 con 0x55ced25aafd0 2023-10-06T05:24:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.060+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'390 uv390 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.060+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ced26a2b50 0x55ced26a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.060+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a54c0 con 0x55ced26a2b50 2023-10-06T05:24:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.060+0000 7efe8ede8700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ced26a2b50 0x55ced26a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.060+0000 7efe8ede8700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ced26a2b50 0x55ced26a4f30 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.061+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.061+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a5940 con 0x55ced26a2b50 2023-10-06T05:24:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.062+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v109'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.063+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a5df0 con 0x55ced26a2b50 2023-10-06T05:24:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.063+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v109'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.065 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.063+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a1760 con 0x55ced26a2b50 2023-10-06T05:24:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.064+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.064+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a7830 con 0x55ced26a2b50 2023-10-06T05:24:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.066+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.066+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a66e0 con 0x55ced26a2b50 2023-10-06T05:24:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.067+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.067+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6b00 con 0x55ced26a2b50 2023-10-06T05:24:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.068+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'72 uv71 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.068+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6f00 con 0x55ced26a2b50 2023-10-06T05:24:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.069+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'73 uv73 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.069+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6f20 con 0x55ced26a2b50 2023-10-06T05:24:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.071+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'74 uv74 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.072 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.071+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6f20 con 0x55ced26a2b50 2023-10-06T05:24:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.072+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'75 uv75 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.072+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6f20 con 0x55ced25aafd0 2023-10-06T05:24:07.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.073+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.073+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6f20 con 0x55ced25aafd0 2023-10-06T05:24:07.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.074+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v109'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.074+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6f20 con 0x55ced25aafd0 2023-10-06T05:24:07.077 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:07.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.076+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v109'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.076+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6f20 con 0x55ced25aafd0 2023-10-06T05:24:07.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.077+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.077+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6eb0 con 0x55ced25aafd0 2023-10-06T05:24:07.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.078+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.078+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6eb0 con 0x55ced25aafd0 2023-10-06T05:24:07.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.079+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced25aafd0 2023-10-06T05:24:07.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.079+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ced26a8140 0x55ced26aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.079+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26a6eb0 con 0x55ced26a8140 2023-10-06T05:24:07.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.080+0000 7efe8fdea700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ced26a8140 0x55ced26aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.080+0000 7efe8fdea700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ced26a8140 0x55ced26aa520 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.081+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v109'182 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.081+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.082+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v109'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.082+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.085 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.083+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v109'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.084+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aba70 con 0x55ced26a8140 2023-10-06T05:24:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.085+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.085+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aba70 con 0x55ced26a8140 2023-10-06T05:24:07.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.086+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.086+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.087+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.087+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.088+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v109'188 uv181 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.088+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.089+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v109'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.089+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.091 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:07.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.090+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v109'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.090+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.091+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.091+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26aaa60 con 0x55ced26a8140 2023-10-06T05:24:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.092+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.093+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ced26ab0e0 0x55ced26ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.093+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26adbc0 con 0x55ced26ab0e0 2023-10-06T05:24:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.093+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ced26ab0e0 0x55ced26ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.093+0000 7efe8f5e9700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ced26ab0e0 0x55ced26ad4c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.094+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v109'81 uv80 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced26ab0e0 2023-10-06T05:24:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.094+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26adbc0 con 0x55ced26ab0e0 2023-10-06T05:24:07.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.096+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v109'82 uv82 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced26ab0e0 2023-10-06T05:24:07.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.096+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26ae020 con 0x55ced26ab0e0 2023-10-06T05:24:07.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.097+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v109'83 uv83 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced26ab0e0 2023-10-06T05:24:07.099 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:07.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.097+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26ae020 con 0x55ced26ab0e0 2023-10-06T05:24:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.098+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v109'84 uv84 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced26ab0e0 2023-10-06T05:24:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.098+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26ae020 con 0x55ced26ab0e0 2023-10-06T05:24:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.099+0000 7efe8f5e9700 1 -- 172.21.15.45:0/1411630136 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v109'85 uv85 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe80016040 con 0x55ced26ab0e0 2023-10-06T05:24:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.100+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26ae020 con 0x55ced26a2b50 2023-10-06T05:24:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.101+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.101+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26ae020 con 0x55ced26a2b50 2023-10-06T05:24:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.102+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v109'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.102+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26ae020 con 0x55ced26a2b50 2023-10-06T05:24:07.105 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.103+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v109'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.103+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26af030 con 0x55ced26a2b50 2023-10-06T05:24:07.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.104+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.104+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26af030 con 0x55ced26a2b50 2023-10-06T05:24:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.106+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.106+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26ae020 con 0x55ced26a2b50 2023-10-06T05:24:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.107+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.107+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ced26ae6a0 0x55ced26b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.107+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1050 con 0x55ced26ae6a0 2023-10-06T05:24:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.107+0000 7efe8ede8700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ced26ae6a0 0x55ced26b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.108+0000 7efe8ede8700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ced26ae6a0 0x55ced26b0a80 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.109+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v109'162 uv156 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.109+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.110+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v109'163 uv163 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.110+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.111+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v109'164 uv164 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.111+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.112+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v109'165 uv165 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.112+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.113+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v109'166 uv166 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.113+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.114+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v109'167 uv161 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.114+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.115+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v109'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.115+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.118 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.116+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v109'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.117+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1430 con 0x55ced26ae6a0 2023-10-06T05:24:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.118+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v109'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.118+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b41a0 con 0x55ced26ae6a0 2023-10-06T05:24:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.119+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v109'171 uv171 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26ae6a0 2023-10-06T05:24:07.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.119+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1920 con 0x55ced26a2b50 2023-10-06T05:24:07.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.120+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v109'74 uv73 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.120+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1920 con 0x55ced26a2b50 2023-10-06T05:24:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.121+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v109'75 uv75 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.121+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1920 con 0x55ced26a2b50 2023-10-06T05:24:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.122+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v109'76 uv76 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.124 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.123+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b1920 con 0x55ced26a2b50 2023-10-06T05:24:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.124+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v109'77 uv77 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.124+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ced26b1e30 0x55ced26b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.124+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26b1e30 2023-10-06T05:24:07.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.124+0000 7efe8fdea700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ced26b1e30 0x55ced26b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.125+0000 7efe8fdea700 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ced26b1e30 0x55ced26b2260 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.125+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26b1e30 2023-10-06T05:24:07.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.126+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26b1e30 2023-10-06T05:24:07.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.127+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v109'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26b1e30 2023-10-06T05:24:07.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.127+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26b1e30 2023-10-06T05:24:07.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.128+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v109'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26b1e30 2023-10-06T05:24:07.129 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:07.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.128+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b5900 con 0x55ced26b1e30 2023-10-06T05:24:07.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.129+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26b1e30 2023-10-06T05:24:07.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.129+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b5900 con 0x55ced26b1e30 2023-10-06T05:24:07.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.130+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26b1e30 2023-10-06T05:24:07.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.130+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26b1e30 2023-10-06T05:24:07.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.131+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26b1e30 2023-10-06T05:24:07.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.131+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26a8140 2023-10-06T05:24:07.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.133+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v109'84 uv83 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.133+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26a8140 2023-10-06T05:24:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.134+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v109'85 uv85 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.134+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26a8140 2023-10-06T05:24:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.135+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v109'86 uv86 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.137 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.135+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b48f0 con 0x55ced26a8140 2023-10-06T05:24:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.136+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v109'87 uv87 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.136+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a8140 2023-10-06T05:24:07.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.138+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v109'88 uv88 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7efe88021030 con 0x55ced26a8140 2023-10-06T05:24:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.138+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.139+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v109'204 uv196 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.139+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.141+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v109'205 uv205 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.141+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.145 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.144+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v109'206 uv206 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.144+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.145+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v109'207 uv207 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.145+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.150+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v109'208 uv208 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.151+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.152+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v109'209 uv203 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.152+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.155+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v109'210 uv210 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.155+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.159 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.158+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v109'211 uv211 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.158+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.161+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v109'212 uv212 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.161+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b14b0 con 0x55ced26a2b50 2023-10-06T05:24:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.164+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v109'213 uv213 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.164+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b4de0 con 0x55ced26a2b50 2023-10-06T05:24:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.166+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v109'214 uv214 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.166+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ced26b5050 con 0x55ced26a2b50 2023-10-06T05:24:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe8ede8700 1 -- 172.21.15.45:0/1411630136 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v109'215 uv215 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7efe840816f0 con 0x55ced26a2b50 2023-10-06T05:24:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ced26a2b50 msgr2=0x55ced26a4f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ced26a2b50 0x55ced26a4f30 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ced25abc00 msgr2=0x55ced268ae20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ced25abc00 0x55ced268ae20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ced26ae6a0 msgr2=0x55ced26b0a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ced26ae6a0 0x55ced26b0a80 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ced25aafd0 msgr2=0x55ced26a05f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ced25aafd0 0x55ced26a05f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ced26b1e30 msgr2=0x55ced26b2260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ced26b1e30 0x55ced26b2260 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ced26ab0e0 msgr2=0x55ced26ad4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ced26ab0e0 0x55ced26ad4c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 reap_dead start 2023-10-06T05:24:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ced26a8140 msgr2=0x55ced26aa520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.169+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ced26a8140 0x55ced26aa520 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 reap_dead start 2023-10-06T05:24:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 reap_dead start 2023-10-06T05:24:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe8fdea700 1 -- 172.21.15.45:0/1411630136 reap_dead start 2023-10-06T05:24:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe7c065890 msgr2=0x7efe7c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe7c065890 0x7efe7c067d40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efe80000fb0 tx=0x7efe80000bc0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669890 msgr2=0x55ced25edba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669890 0x55ced25edba0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7efe8400a200 tx=0x7efe840018c0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 shutdown_connections 2023-10-06T05:24:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ced26a8140 0x55ced26aa520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ced2669890 0x55ced25edba0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 --2- 172.21.15.45:0/1411630136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe7c065890 0x7efe7c067d40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 >> 172.21.15.45:0/1411630136 conn(0x55ced25aa450 msgr2=0x55ced25a7210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 shutdown_connections 2023-10-06T05:24:07.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.170+0000 7efe963cb040 1 -- 172.21.15.45:0/1411630136 wait complete. 2023-10-06T05:24:07.177 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2023-10-06T05:24:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.266+0000 7fe184bf8040 1 Processor -- start 2023-10-06T05:24:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.266+0000 7fe184bf8040 1 -- start start 2023-10-06T05:24:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe184bf8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1a9b3dd0 0x561f1a9cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe184bf8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b7c80 0x561f1aa5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe184bf8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1aa5baf0 0x561f1a9cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe184bf8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a8d2980 con 0x561f1a9b3dd0 2023-10-06T05:24:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe184bf8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a8d2620 con 0x561f1a9b7c80 2023-10-06T05:24:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe184bf8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f1a8c9400 con 0x561f1aa5baf0 2023-10-06T05:24:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1a9b3dd0 0x561f1a9cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1a9b3dd0 0x561f1a9cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52574/0 (socket says 172.21.15.45:52574) 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 -- 172.21.15.45:0/2581944065 learned_addr learned my addr 172.21.15.45:0/2581944065 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17d615700 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b7c80 0x561f1aa5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17e617700 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1aa5baf0 0x561f1a9cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 -- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1aa5baf0 msgr2=0x561f1a9cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1aa5baf0 0x561f1a9cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 -- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b7c80 msgr2=0x561f1aa5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b7c80 0x561f1aa5ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17de16700 1 -- 172.21.15.45:0/2581944065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe170005cf0 con 0x561f1a9b3dd0 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.267+0000 7fe17d615700 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b7c80 0x561f1aa5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe17de16700 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1a9b3dd0 0x561f1a9cda10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe1700099e0 tx=0x7fe17000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b366d3e9fa2cc4da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe16ffff700 1 -- 172.21.15.45:0/2581944065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000f040 con 0x561f1a9b3dd0 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe16ffff700 1 -- 172.21.15.45:0/2581944065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe170008b50 con 0x561f1a9b3dd0 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe16ffff700 1 -- 172.21.15.45:0/2581944065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170013420 con 0x561f1a9b3dd0 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 -- 172.21.15.45:0/2581944065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1a9b3dd0 msgr2=0x561f1a9cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1a9b3dd0 0x561f1a9cda10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe1700099e0 tx=0x7fe17000c040 comp rx=0 tx=0).stop 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 -- 172.21.15.45:0/2581944065 shutdown_connections 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1aa5baf0 0x561f1a9cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1a9b3dd0 0x561f1a9cda10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 --2- 172.21.15.45:0/2581944065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b7c80 0x561f1aa5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 -- 172.21.15.45:0/2581944065 >> 172.21.15.45:0/2581944065 conn(0x561f1a9884a0 msgr2=0x561f1a9b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.268+0000 7fe184bf8040 1 -- 172.21.15.45:0/2581944065 shutdown_connections 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.269+0000 7fe184bf8040 1 -- 172.21.15.45:0/2581944065 wait complete. 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.269+0000 7fe184bf8040 1 Processor -- start 2023-10-06T05:24:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.270+0000 7fe184bf8040 1 -- start start 2023-10-06T05:24:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.270+0000 7fe184bf8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b3dd0 0x561f1aa57d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe184bf8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1a9b7c80 0x561f1aa6c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe184bf8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1aa5baf0 0x561f1aa60580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe184bf8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1aa56210 con 0x561f1aa5baf0 2023-10-06T05:24:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe184bf8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1aa58b80 con 0x561f1a9b3dd0 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe184bf8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f1aa56440 con 0x561f1a9b7c80 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1aa5baf0 0x561f1aa60580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1aa5baf0 0x561f1aa60580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52578/0 (socket says 172.21.15.45:52578) 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 -- 172.21.15.45:0/1792379108 learned_addr learned my addr 172.21.15.45:0/1792379108 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17de16700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b3dd0 0x561f1aa57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17d615700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1a9b7c80 0x561f1aa6c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 -- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1a9b7c80 msgr2=0x561f1aa6c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1a9b7c80 0x561f1aa6c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 -- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b3dd0 msgr2=0x561f1aa57d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f1a9b3dd0 0x561f1aa57d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 -- 172.21.15.45:0/1792379108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700095e0 con 0x561f1aa5baf0 2023-10-06T05:24:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17d615700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f1a9b7c80 0x561f1aa6c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.271+0000 7fe17e617700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1aa5baf0 0x561f1aa60580 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x561f1a9b8380 tx=0x7fe178015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.272+0000 7fe16e7fc700 1 -- 172.21.15.45:0/1792379108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17800fa70 con 0x561f1aa5baf0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.272+0000 7fe16e7fc700 1 -- 172.21.15.45:0/1792379108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17800fe50 con 0x561f1aa5baf0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.272+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17800a0e0 con 0x561f1aa5baf0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.272+0000 7fe16e7fc700 1 -- 172.21.15.45:0/1792379108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178010630 con 0x561f1aa5baf0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.272+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f1a8e13d0 con 0x561f1aa5baf0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.273+0000 7fe16e7fc700 1 -- 172.21.15.45:0/1792379108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe178003040 con 0x561f1aa5baf0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.273+0000 7fe16e7fc700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe160065890 0x7fe160067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.274+0000 7fe16e7fc700 1 -- 172.21.15.45:0/1792379108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fe178085680 con 0x561f1aa5baf0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.274+0000 7fe17de16700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe160065890 0x7fe160067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.274+0000 7fe184bf8040 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561f1aa586e0 0x561f1aa57330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.274+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x561f1aa51040 con 0x561f1aa586e0 2023-10-06T05:24:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.274+0000 7fe17d615700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561f1aa586e0 0x561f1aa57330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.274+0000 7fe17de16700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe160065890 0x7fe160067d40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe170009980 tx=0x7fe170016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.274+0000 7fe17d615700 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561f1aa586e0 0x561f1aa57330 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe17d615700 1 -- 172.21.15.45:0/1792379108 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v109'391 uv391 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe17400d820 con 0x561f1aa586e0 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561f1aa586e0 msgr2=0x561f1aa57330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561f1aa586e0 0x561f1aa57330 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe160065890 msgr2=0x7fe160067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe160065890 0x7fe160067d40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe170009980 tx=0x7fe170016000 comp rx=0 tx=0).stop 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1aa5baf0 msgr2=0x561f1aa60580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 --2- 172.21.15.45:0/1792379108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f1aa5baf0 0x561f1aa60580 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x561f1a9b8380 tx=0x7fe178015000 comp rx=0 tx=0).stop 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe17e617700 1 -- 172.21.15.45:0/1792379108 reap_dead start 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 shutdown_connections 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 >> 172.21.15.45:0/1792379108 conn(0x561f1a9884a0 msgr2=0x561f1a9b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 shutdown_connections 2023-10-06T05:24:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.276+0000 7fe184bf8040 1 -- 172.21.15.45:0/1792379108 wait complete. 2023-10-06T05:24:07.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.462+0000 7f8573fc8700 1 Processor -- start 2023-10-06T05:24:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.462+0000 7f8573fc8700 1 -- start start 2023-10-06T05:24:07.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.463+0000 7f8573fc8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c0aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.463+0000 7f8573fc8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856c0c3b50 con 0x7f856c157330 2023-10-06T05:24:07.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.463+0000 7f8573fc8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856c157700 con 0x7f856c0c8b70 2023-10-06T05:24:07.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.463+0000 7f8573fc8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c0c9700 con 0x7f856c0ccb70 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.463+0000 7f8571d64700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f856c157330 0x7f856c0ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39780/0 (socket says 172.21.15.45:39780) 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.463+0000 7f8571d64700 1 -- 172.21.15.45:0/1313560820 learned_addr learned my addr 172.21.15.45:0/1313560820 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8571563700 1 --2- 172.21.15.45:0/1313560820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c0aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983268723 0 0) 0x7f856c0c3b50 con 0x7f856c157330 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8558003570 con 0x7f856c157330 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491325335 0 0) 0x7f856c157700 con 0x7f856c0c8b70 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f856c0c3b50 con 0x7f856c0c8b70 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8571563700 1 -- 172.21.15.45:0/1313560820 >> v1:172.21.15.144:6789/0 conn(0x7f856c0c8b70 legacy=0x7f856c0c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8571563700 1 -- 172.21.15.45:0/1313560820 >> v1:172.21.15.45:6789/0 conn(0x7f856c157330 legacy=0x7f856c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 215569547 0 0) 0x7f8558003570 con 0x7f856c157330 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8571563700 1 -- 172.21.15.45:0/1313560820 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c0c98e0 con 0x7f856c0ccb70 2023-10-06T05:24:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8571563700 1 --2- 172.21.15.45:0/1313560820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c0aaae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8560009910 tx=0x7f85600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ead230416ff8fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680015f0 con 0x7f856c0ccb70 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560011420 con 0x7f856c0ccb70 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.464+0000 7f8570d62700 1 -- 172.21.15.45:0/1313560820 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600117a0 con 0x7f856c0ccb70 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.465+0000 7f8573fc8700 1 -- 172.21.15.45:0/1313560820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 msgr2=0x7f856c0aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.465+0000 7f8573fc8700 1 --2- 172.21.15.45:0/1313560820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c0aaae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8560009910 tx=0x7f85600091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.465+0000 7f8573fc8700 1 -- 172.21.15.45:0/1313560820 shutdown_connections 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.465+0000 7f8573fc8700 1 --2- 172.21.15.45:0/1313560820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c0aaae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.465+0000 7f8573fc8700 1 -- 172.21.15.45:0/1313560820 >> 172.21.15.45:0/1313560820 conn(0x7f856c067e00 msgr2=0x7f856c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.465+0000 7f8573fc8700 1 -- 172.21.15.45:0/1313560820 shutdown_connections 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.465+0000 7f8573fc8700 1 -- 172.21.15.45:0/1313560820 wait complete. 2023-10-06T05:24:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.466+0000 7f8573fc8700 1 Processor -- start 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.466+0000 7f8573fc8700 1 -- start start 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8573fc8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f856c0c8b70 0x7f856c0aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8573fc8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c168150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8573fc8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856c157330 0x7f856c151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8573fc8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c168690 con 0x7f856c0c8b70 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8573fc8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c1523c0 con 0x7f856c157330 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8573fc8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c0ab0c0 con 0x7f856c0ccb70 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f856c0c8b70 0x7f856c0aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f856c0c8b70 0x7f856c0aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52582/0 (socket says 172.21.15.45:52582) 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 -- 172.21.15.45:0/2234236700 learned_addr learned my addr 172.21.15.45:0/2234236700 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571563700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8572565700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856c157330 0x7f856c151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 -- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 msgr2=0x7f856c168150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c168150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 -- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856c157330 msgr2=0x7f856c151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856c157330 0x7f856c151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8571d64700 1 -- 172.21.15.45:0/2234236700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560009420 con 0x7f856c0c8b70 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.467+0000 7f8572565700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856c157330 0x7f856c151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.468+0000 7f8571d64700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f856c0c8b70 0x7f856c0aaae0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f856800c810 tx=0x7f85680016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.468+0000 7f855dffb700 1 -- 172.21.15.45:0/2234236700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568007300 con 0x7f856c0c8b70 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.468+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856c0ab3a0 con 0x7f856c0c8b70 2023-10-06T05:24:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.468+0000 7f855dffb700 1 -- 172.21.15.45:0/2234236700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568007460 con 0x7f856c0c8b70 2023-10-06T05:24:07.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.468+0000 7f855dffb700 1 -- 172.21.15.45:0/2234236700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680057f0 con 0x7f856c0c8b70 2023-10-06T05:24:07.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.468+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856c0ab960 con 0x7f856c0c8b70 2023-10-06T05:24:07.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.469+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856c06e350 con 0x7f856c0c8b70 2023-10-06T05:24:07.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.469+0000 7f855dffb700 1 -- 172.21.15.45:0/2234236700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8568005c40 con 0x7f856c0c8b70 2023-10-06T05:24:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.470+0000 7f855dffb700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8550066f30 0x7f85500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.470+0000 7f855dffb700 1 -- 172.21.15.45:0/2234236700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f8568084b40 con 0x7f856c0c8b70 2023-10-06T05:24:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.470+0000 7f8571563700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8550066f30 0x7f85500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.471+0000 7f8571563700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8550066f30 0x7f85500693e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f85600091e0 tx=0x7f8560016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.474+0000 7f855dffb700 1 -- 172.21.15.45:0/2234236700 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8568051560 con 0x7f856c0c8b70 2023-10-06T05:24:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.724+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f856c158e00 con 0x7f856c0c8b70 2023-10-06T05:24:07.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.927+0000 7f855dffb700 1 -- 172.21.15.45:0/2234236700 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v199) v1 ==== 162+0+0 (secure 0 0 0) 0x7f8568054b80 con 0x7f856c0c8b70 2023-10-06T05:24:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8550066f30 msgr2=0x7f85500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8550066f30 0x7f85500693e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f85600091e0 tx=0x7f8560016000 comp rx=0 tx=0).stop 2023-10-06T05:24:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f856c0c8b70 msgr2=0x7f856c0aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f856c0c8b70 0x7f856c0aaae0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f856800c810 tx=0x7f85680016e0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 shutdown_connections 2023-10-06T05:24:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8550066f30 0x7f85500693e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856c157330 0x7f856c151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f856c0c8b70 0x7f856c0aaae0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 --2- 172.21.15.45:0/2234236700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856c0ccb70 0x7f856c168150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 >> 172.21.15.45:0/2234236700 conn(0x7f856c067e00 msgr2=0x7f856c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 shutdown_connections 2023-10-06T05:24:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:07.931+0000 7f8573fc8700 1 -- 172.21.15.45:0/2234236700 wait complete. 2023-10-06T05:24:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:07.945 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:07.944+0000 7fd2de930700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-10-06T05:24:07.946 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:07.944+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:07.946 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:07.944+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:07.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:07.952 INFO:tasks.ceph.mds.c.smithi045.stderr: -14> 2023-10-06T05:24:07.944+0000 7fd2de930700 -1 log_channel(cluster) log [ERR] : missing journal object 2023-10-06T05:24:07.974 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:24:08.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:08.005 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.089+0000 7fcd55b93700 1 Processor -- start 2023-10-06T05:24:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.089+0000 7fcd55b93700 1 -- start start 2023-10-06T05:24:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.089+0000 7fcd55b93700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd50159330 0x7fcd5006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.089+0000 7fcd55b93700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd500cc390 0x7fcd500cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.089+0000 7fcd55b93700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500cb530 con 0x7fcd50158f60 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.089+0000 7fcd55b93700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd50069bd0 con 0x7fcd50159330 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.089+0000 7fcd55b93700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500ccd10 con 0x7fcd500cc390 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd50159330 0x7fcd5006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcd50158f60 0x7fcd500cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39796/0 (socket says 172.21.15.45:39796) 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4f7fe700 1 -- 172.21.15.45:0/2302690569 learned_addr learned my addr 172.21.15.45:0/2302690569 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4ffff700 1 --2- 172.21.15.45:0/2302690569 >> v2:172.21.15.144:3300/0 conn(0x7fcd500cc390 0x7fcd500cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4effd700 1 -- 172.21.15.45:0/2302690569 >> v2:172.21.15.144:3300/0 conn(0x7fcd500cc390 msgr2=0x7fcd500cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4effd700 1 --2- 172.21.15.45:0/2302690569 >> v2:172.21.15.144:3300/0 conn(0x7fcd500cc390 0x7fcd500cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4effd700 1 -- 172.21.15.45:0/2302690569 >> v1:172.21.15.45:6789/0 conn(0x7fcd50158f60 legacy=0x7fcd500cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4effd700 1 -- 172.21.15.45:0/2302690569 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50068fc0 con 0x7fcd50159330 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4e7fc700 1 -- 172.21.15.45:0/2302690569 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520653681 0 0) 0x7fcd40001110 con 0x7fcd50158f60 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4effd700 1 --2- 172.21.15.45:0/2302690569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd50159330 0x7fcd5006a060 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fcd500ca4c0 tx=0x7fcd440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a98a7d26e367b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4e7fc700 1 -- 172.21.15.45:0/2302690569 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44011630 con 0x7fcd50159330 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4e7fc700 1 -- 172.21.15.45:0/2302690569 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44011790 con 0x7fcd50159330 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.090+0000 7fcd4e7fc700 1 -- 172.21.15.45:0/2302690569 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44011b30 con 0x7fcd50159330 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 -- 172.21.15.45:0/2302690569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd50159330 msgr2=0x7fcd5006a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 --2- 172.21.15.45:0/2302690569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd50159330 0x7fcd5006a060 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fcd500ca4c0 tx=0x7fcd440091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 -- 172.21.15.45:0/2302690569 shutdown_connections 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 --2- 172.21.15.45:0/2302690569 >> v2:172.21.15.144:3300/0 conn(0x7fcd500cc390 0x7fcd500cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 --2- 172.21.15.45:0/2302690569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd50159330 0x7fcd5006a060 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 -- 172.21.15.45:0/2302690569 >> 172.21.15.45:0/2302690569 conn(0x7fcd500cdf50 msgr2=0x7fcd500ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 -- 172.21.15.45:0/2302690569 shutdown_connections 2023-10-06T05:24:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.091+0000 7fcd55b93700 1 -- 172.21.15.45:0/2302690569 wait complete. 2023-10-06T05:24:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.092+0000 7fcd55b93700 1 Processor -- start 2023-10-06T05:24:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd55b93700 1 -- start start 2023-10-06T05:24:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd55b93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd500cc390 0x7fcd50153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd55b93700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50158f60 0x7fcd500b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd55b93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd50159330 0x7fcd501556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd55b93700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500aff90 con 0x7fcd500cc390 2023-10-06T05:24:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd55b93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50155d90 con 0x7fcd50159330 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd55b93700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5015cf10 con 0x7fcd50158f60 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd500cc390 0x7fcd50153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd500cc390 0x7fcd50153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52608/0 (socket says 172.21.15.45:52608) 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.093+0000 7fcd4f7fe700 1 -- 172.21.15.45:0/3859144059 learned_addr learned my addr 172.21.15.45:0/3859144059 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4ffff700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd50159330 0x7fcd501556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4effd700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50158f60 0x7fcd500b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4f7fe700 1 -- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50158f60 msgr2=0x7fcd500b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4f7fe700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50158f60 0x7fcd500b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4f7fe700 1 -- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd50159330 msgr2=0x7fcd501556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4f7fe700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd50159330 0x7fcd501556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4f7fe700 1 -- 172.21.15.45:0/3859144059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd44009420 con 0x7fcd500cc390 2023-10-06T05:24:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4effd700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50158f60 0x7fcd500b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd4f7fe700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd500cc390 0x7fcd50153ce0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c840 tx=0x7fcd40007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd377fe700 1 -- 172.21.15.45:0/3859144059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40005b30 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd377fe700 1 -- 172.21.15.45:0/3859144059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40005c90 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5015d1f0 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd377fe700 1 -- 172.21.15.45:0/3859144059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400072e0 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.094+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd50157980 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.095+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd500b00d0 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.096+0000 7fcd377fe700 1 -- 172.21.15.45:0/3859144059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd400076d0 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.096+0000 7fcd377fe700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd3c066f30 0x7fcd3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.096+0000 7fcd377fe700 1 -- 172.21.15.45:0/3859144059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7fcd400840b0 con 0x7fcd500cc390 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.096+0000 7fcd4effd700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd3c066f30 0x7fcd3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.097+0000 7fcd4effd700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd3c066f30 0x7fcd3c0693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcd440091e0 tx=0x7fcd44017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.101+0000 7fcd377fe700 1 -- 172.21.15.45:0/3859144059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd4004b5a0 con 0x7fcd500cc390 2023-10-06T05:24:08.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.346+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd500cb530 con 0x7fcd500cc390 2023-10-06T05:24:08.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.346+0000 7fcd377fe700 1 -- 172.21.15.45:0/3859144059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 201 v201) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fcd500cb530 con 0x7fcd500cc390 2023-10-06T05:24:08.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:08.350 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":201,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:07.945682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:08.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd3c066f30 msgr2=0x7fcd3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd3c066f30 0x7fcd3c0693e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcd440091e0 tx=0x7fcd44017000 comp rx=0 tx=0).stop 2023-10-06T05:24:08.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd500cc390 msgr2=0x7fcd50153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd500cc390 0x7fcd50153ce0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c840 tx=0x7fcd40007c50 comp rx=0 tx=0).stop 2023-10-06T05:24:08.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 shutdown_connections 2023-10-06T05:24:08.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd3c066f30 0x7fcd3c0693e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd50159330 0x7fcd501556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd500cc390 0x7fcd50153ce0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 --2- 172.21.15.45:0/3859144059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50158f60 0x7fcd500b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 >> 172.21.15.45:0/3859144059 conn(0x7fcd500cdf50 msgr2=0x7fcd500c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 shutdown_connections 2023-10-06T05:24:08.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.349+0000 7fcd55b93700 1 -- 172.21.15.45:0/3859144059 wait complete. 2023-10-06T05:24:08.352 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 201 2023-10-06T05:24:08.364 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:08.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.537+0000 7f95d3ca6700 1 Processor -- start 2023-10-06T05:24:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.537+0000 7f95d3ca6700 1 -- start start 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d3ca6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc0c8ff0 0x7f95cc0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d3ca6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95cc0caad0 0x7f95cc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d3ca6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95cc156af0 0x7f95cc156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d3ca6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc15b000 con 0x7f95cc156af0 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d3ca6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc15b2e0 con 0x7f95cc0caad0 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d3ca6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc15b5c0 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d2243700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95cc156af0 0x7f95cc156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d2243700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95cc156af0 0x7f95cc156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52620/0 (socket says 172.21.15.45:52620) 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d2243700 1 -- 172.21.15.45:0/1336460057 learned_addr learned my addr 172.21.15.45:0/1336460057 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d1241700 1 --2- 172.21.15.45:0/1336460057 >> v2:172.21.15.144:3300/0 conn(0x7f95cc0caad0 0x7f95cc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.538+0000 7f95d1a42700 1 --2- 172.21.15.45:0/1336460057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc0c8ff0 0x7f95cc0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d2243700 1 -- 172.21.15.45:0/1336460057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc0c8ff0 msgr2=0x7f95cc0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d2243700 1 --2- 172.21.15.45:0/1336460057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc0c8ff0 0x7f95cc0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d2243700 1 -- 172.21.15.45:0/1336460057 >> v2:172.21.15.144:3300/0 conn(0x7f95cc0caad0 msgr2=0x7f95cc0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d2243700 1 --2- 172.21.15.45:0/1336460057 >> v2:172.21.15.144:3300/0 conn(0x7f95cc0caad0 0x7f95cc0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d2243700 1 -- 172.21.15.45:0/1336460057 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc15b8a0 con 0x7f95cc156af0 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d1241700 1 --2- 172.21.15.45:0/1336460057 >> v2:172.21.15.144:3300/0 conn(0x7f95cc0caad0 0x7f95cc0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d2243700 1 --2- 172.21.15.45:0/1336460057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc156af0 0x7f95cc156f70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0c4fb0 tx=0x7f95c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d73e851d38cc70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d0a40700 1 -- 172.21.15.45:0/1336460057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c400e070 con 0x7f95cc156af0 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d0a40700 1 -- 172.21.15.45:0/1336460057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c400ba50 con 0x7f95cc156af0 2023-10-06T05:24:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.539+0000 7f95d0a40700 1 -- 172.21.15.45:0/1336460057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40092e0 con 0x7f95cc156af0 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1336460057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc156af0 msgr2=0x7f95cc156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1336460057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc156af0 0x7f95cc156f70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0c4fb0 tx=0x7f95c4009840 comp rx=0 tx=0).stop 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1336460057 shutdown_connections 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1336460057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc156af0 0x7f95cc156f70 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1336460057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc0c8ff0 0x7f95cc0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1336460057 >> v2:172.21.15.144:3300/0 conn(0x7f95cc0caad0 0x7f95cc0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1336460057 >> 172.21.15.45:0/1336460057 conn(0x7f95cc0cdf50 msgr2=0x7f95cc0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1336460057 shutdown_connections 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.540+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1336460057 wait complete. 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.541+0000 7f95d3ca6700 1 Processor -- start 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.541+0000 7f95d3ca6700 1 -- start start 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d3ca6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc0c8ff0 0x7f95cc069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d3ca6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0caad0 0x7f95cc0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d3ca6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc156af0 0x7f95cc151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d3ca6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0c6180 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d3ca6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc06a490 con 0x7f95cc0caad0 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d3ca6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0b17a0 con 0x7f95cc156af0 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc0c8ff0 0x7f95cc069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc0c8ff0 0x7f95cc069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52626/0 (socket says 172.21.15.45:52626) 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 -- 172.21.15.45:0/1462522739 learned_addr learned my addr 172.21.15.45:0/1462522739 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1241700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0caad0 0x7f95cc0696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d2243700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc156af0 0x7f95cc151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 -- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc156af0 msgr2=0x7f95cc151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc156af0 0x7f95cc151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 -- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0caad0 msgr2=0x7f95cc0696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0caad0 0x7f95cc0696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.542+0000 7f95d1a42700 1 -- 172.21.15.45:0/1462522739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4009ae0 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.543+0000 7f95d1a42700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc0c8ff0 0x7f95cc069d90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f95c800d890 tx=0x7f95c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.543+0000 7f95bdffb700 1 -- 172.21.15.45:0/1462522739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800b600 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.543+0000 7f95bdffb700 1 -- 172.21.15.45:0/1462522739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800b760 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.543+0000 7f95bdffb700 1 -- 172.21.15.45:0/1462522739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800bae0 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.543+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc0b1a80 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.543+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc151e40 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.544+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc152060 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.544+0000 7f95bdffb700 1 -- 172.21.15.45:0/1462522739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95c801e460 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.545+0000 7f95bdffb700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b0066f30 0x7f95b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.545+0000 7f95bdffb700 1 -- 172.21.15.45:0/1462522739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f95c8085560 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.545+0000 7f95d1241700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b0066f30 0x7f95b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.546+0000 7f95d1241700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b0066f30 0x7f95b00693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006f00 tx=0x7f95c0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.549+0000 7f95bdffb700 1 -- 172.21.15.45:0/1462522739 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95c8051e80 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.796+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95cc04bb40 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.796+0000 7f95bdffb700 1 -- 172.21.15.45:0/1462522739 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 201 v201) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f95c80554a0 con 0x7f95cc0c8ff0 2023-10-06T05:24:08.798 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:08.798 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":201,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:07.945682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b0066f30 msgr2=0x7f95b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b0066f30 0x7f95b00693e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006f00 tx=0x7f95c0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc0c8ff0 msgr2=0x7f95cc069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc0c8ff0 0x7f95cc069d90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f95c800d890 tx=0x7f95c8008040 comp rx=0 tx=0).stop 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 shutdown_connections 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b0066f30 0x7f95b00693e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95cc156af0 0x7f95cc151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95cc0c8ff0 0x7f95cc069d90 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 --2- 172.21.15.45:0/1462522739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95cc0caad0 0x7f95cc0696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.798+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 >> 172.21.15.45:0/1462522739 conn(0x7f95cc0cdf50 msgr2=0x7f95cc0c4470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.799+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 shutdown_connections 2023-10-06T05:24:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.799+0000 7f95d3ca6700 1 -- 172.21.15.45:0/1462522739 wait complete. 2023-10-06T05:24:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 201 2023-10-06T05:24:08.814 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:08.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.945+0000 7fc1b5a65700 1 Processor -- start 2023-10-06T05:24:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.945+0000 7fc1b5a65700 1 -- start start 2023-10-06T05:24:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.946+0000 7fc1b5a65700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.946+0000 7fc1b5a65700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b00ce690 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.946+0000 7fc1b5a65700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b015d510 con 0x7fc1b01539f0 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.946+0000 7fc1b5a65700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b015a7a0 con 0x7fc1b0153620 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.946+0000 7fc1affff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc1b00ccfc0 0x7fc1b00ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39812/0 (socket says 172.21.15.45:39812) 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.946+0000 7fc1affff700 1 -- 172.21.15.45:0/2038513591 learned_addr learned my addr 172.21.15.45:0/2038513591 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1af7fe700 1 --2- 172.21.15.45:0/2038513591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1ae7fc700 1 -- 172.21.15.45:0/2038513591 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395363030 0 0) 0x7fc1b00ce690 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1ae7fc700 1 -- 172.21.15.45:0/2038513591 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc198003570 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1af7fe700 1 -- 172.21.15.45:0/2038513591 >> v1:172.21.15.144:6789/0 conn(0x7fc1b01539f0 legacy=0x7fc1b00cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1af7fe700 1 -- 172.21.15.45:0/2038513591 >> v1:172.21.15.45:6789/0 conn(0x7fc1b00ccfc0 legacy=0x7fc1b00ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1ae7fc700 1 -- 172.21.15.45:0/2038513591 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 209412297 0 0) 0x7fc198003570 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1af7fe700 1 -- 172.21.15.45:0/2038513591 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b015d7f0 con 0x7fc1b0153620 2023-10-06T05:24:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.947+0000 7fc1af7fe700 1 --2- 172.21.15.45:0/2038513591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b015a260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009910 tx=0x7fc1a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b7de949e0e572 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.952+0000 7fc1ae7fc700 1 -- 172.21.15.45:0/2038513591 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80017f0 con 0x7fc1b0153620 2023-10-06T05:24:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.952+0000 7fc1ae7fc700 1 -- 172.21.15.45:0/2038513591 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a0011420 con 0x7fc1b0153620 2023-10-06T05:24:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.952+0000 7fc1ae7fc700 1 -- 172.21.15.45:0/2038513591 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00117e0 con 0x7fc1b0153620 2023-10-06T05:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.953+0000 7fc1b5a65700 1 -- 172.21.15.45:0/2038513591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 msgr2=0x7fc1b015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.953+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/2038513591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b015a260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009910 tx=0x7fc1a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.953+0000 7fc1b5a65700 1 -- 172.21.15.45:0/2038513591 shutdown_connections 2023-10-06T05:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.953+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/2038513591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b015a260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.953+0000 7fc1b5a65700 1 -- 172.21.15.45:0/2038513591 >> 172.21.15.45:0/2038513591 conn(0x7fc1b006ed50 msgr2=0x7fc1b006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.953+0000 7fc1b5a65700 1 -- 172.21.15.45:0/2038513591 shutdown_connections 2023-10-06T05:24:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.953+0000 7fc1b5a65700 1 -- 172.21.15.45:0/2038513591 wait complete. 2023-10-06T05:24:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.955+0000 7fc1b5a65700 1 Processor -- start 2023-10-06T05:24:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.955+0000 7fc1b5a65700 1 -- start start 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1b5a65700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00ccfc0 0x7fc1b00b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1b5a65700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1b5a65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b00b59c0 0x7fc1b00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1b5a65700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b01505d0 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1b5a65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00b3540 con 0x7fc1b00b59c0 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1b5a65700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00b1e70 con 0x7fc1b0153620 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00ccfc0 0x7fc1b00b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00ccfc0 0x7fc1b00b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52634/0 (socket says 172.21.15.45:52634) 2023-10-06T05:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1af7fe700 1 -- 172.21.15.45:0/849276294 learned_addr learned my addr 172.21.15.45:0/849276294 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1affff700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b00b59c0 0x7fc1b00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.956+0000 7fc1aeffd700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1af7fe700 1 -- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 msgr2=0x7fc1b00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1af7fe700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b00cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1af7fe700 1 -- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b00b59c0 msgr2=0x7fc1b00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1af7fe700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b00b59c0 0x7fc1b00d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1af7fe700 1 -- 172.21.15.45:0/849276294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0009420 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1af7fe700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00ccfc0 0x7fc1b00b39d0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc1b006f900 tx=0x7fc1a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1977fe700 1 -- 172.21.15.45:0/849276294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a001caf0 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1977fe700 1 -- 172.21.15.45:0/849276294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a001cc70 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b016abe0 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1977fe700 1 -- 172.21.15.45:0/849276294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a001a450 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.957+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b00b2ef0 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.958+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b00b3820 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.959+0000 7fc1977fe700 1 -- 172.21.15.45:0/849276294 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1a001a7c0 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.959+0000 7fc1977fe700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc19c066f30 0x7fc19c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.959+0000 7fc1977fe700 1 -- 172.21.15.45:0/849276294 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7fc1a0086560 con 0x7fc1b00ccfc0 2023-10-06T05:24:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.959+0000 7fc1aeffd700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc19c066f30 0x7fc19c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.960+0000 7fc1aeffd700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc19c066f30 0x7fc19c0693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00b5ec0 tx=0x7fc1a4007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:08.964+0000 7fc1977fe700 1 -- 172.21.15.45:0/849276294 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1a0052fd0 con 0x7fc1b00ccfc0 2023-10-06T05:24:09.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.210+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1b004bb40 con 0x7fc1b00ccfc0 2023-10-06T05:24:09.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.211+0000 7fc1977fe700 1 -- 172.21.15.45:0/849276294 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+4599 (secure 0 0 0) 0x7fc1a00565f0 con 0x7fc1b00ccfc0 2023-10-06T05:24:09.212 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:09.213 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":202,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3031536032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3031536032},{"type":"v1","addr":"172.21.15.45:6835","nonce":3031536032}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:07.945682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc19c066f30 msgr2=0x7fc19c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc19c066f30 0x7fc19c0693e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00b5ec0 tx=0x7fc1a4007ec0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00ccfc0 msgr2=0x7fc1b00b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00ccfc0 0x7fc1b00b39d0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc1b006f900 tx=0x7fc1a0016000 comp rx=0 tx=0).stop 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 shutdown_connections 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc19c066f30 0x7fc19c0693e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b00b59c0 0x7fc1b00d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00ccfc0 0x7fc1b00b39d0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 --2- 172.21.15.45:0/849276294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b0153620 0x7fc1b00cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 >> 172.21.15.45:0/849276294 conn(0x7fc1b006ed50 msgr2=0x7fc1b0159830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 shutdown_connections 2023-10-06T05:24:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.213+0000 7fc1b5a65700 1 -- 172.21.15.45:0/849276294 wait complete. 2023-10-06T05:24:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 202 2023-10-06T05:24:09.228 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2023-10-06T05:24:09.228 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2023-10-06T05:24:09.228 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:09.228 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:09.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:09.228 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:09.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:09.248 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:09.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:09.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.422+0000 7f735ada9700 1 Processor -- start 2023-10-06T05:24:09.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.422+0000 7f735ada9700 1 -- start start 2023-10-06T05:24:09.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f735ada9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7354156ce0 0x7f73541570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f735ada9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f735415ac10 0x7f73540ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f735ada9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735406bf70 con 0x7f735406bad0 2023-10-06T05:24:09.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f735ada9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73540b02f0 con 0x7f735415ac10 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f735ada9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735415b380 con 0x7f7354156ce0 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f735415ac10 0x7f73540ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f735415ac10 0x7f73540ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52650/0 (socket says 172.21.15.45:52650) 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 -- 172.21.15.45:0/2576380764 learned_addr learned my addr 172.21.15.45:0/2576380764 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7358b45700 1 --2- 172.21.15.45:0/2576380764 >> v2:172.21.15.144:3300/0 conn(0x7f7354156ce0 0x7f73541570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 -- 172.21.15.45:0/2576380764 >> v2:172.21.15.144:3300/0 conn(0x7f7354156ce0 msgr2=0x7f73541570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 --2- 172.21.15.45:0/2576380764 >> v2:172.21.15.144:3300/0 conn(0x7f7354156ce0 0x7f73541570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 -- 172.21.15.45:0/2576380764 >> v1:172.21.15.144:6789/0 conn(0x7f735406bad0 legacy=0x7f73540afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 -- 172.21.15.45:0/2576380764 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735415b660 con 0x7f735415ac10 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.424+0000 7f7359346700 1 --2- 172.21.15.45:0/2576380764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ccce0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f73540af900 tx=0x7f734c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6345a4885554d562 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f73537fe700 1 -- 172.21.15.45:0/2576380764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c00e070 con 0x7f735415ac10 2023-10-06T05:24:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f73537fe700 1 -- 172.21.15.45:0/2576380764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c00ba10 con 0x7f735415ac10 2023-10-06T05:24:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f73537fe700 1 -- 172.21.15.45:0/2576380764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0092e0 con 0x7f735415ac10 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 -- 172.21.15.45:0/2576380764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 msgr2=0x7f73540ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 --2- 172.21.15.45:0/2576380764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ccce0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f73540af900 tx=0x7f734c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 -- 172.21.15.45:0/2576380764 shutdown_connections 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 --2- 172.21.15.45:0/2576380764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ccce0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 --2- 172.21.15.45:0/2576380764 >> v2:172.21.15.144:3300/0 conn(0x7f7354156ce0 0x7f73541570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 -- 172.21.15.45:0/2576380764 >> 172.21.15.45:0/2576380764 conn(0x7f735406f090 msgr2=0x7f735406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 -- 172.21.15.45:0/2576380764 shutdown_connections 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.425+0000 7f735ada9700 1 -- 172.21.15.45:0/2576380764 wait complete. 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.426+0000 7f735ada9700 1 Processor -- start 2023-10-06T05:24:09.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.427+0000 7f735ada9700 1 -- start start 2023-10-06T05:24:09.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.427+0000 7f735ada9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f735406bad0 0x7f73541597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.427+0000 7f735ada9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7354156ce0 0x7f7354148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.427+0000 7f735ada9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.427+0000 7f735ada9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354159ce0 con 0x7f735415ac10 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.427+0000 7f735ada9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354168430 con 0x7f735406bad0 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.427+0000 7f735ada9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7354168cd0 con 0x7f7354156ce0 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52656/0 (socket says 172.21.15.45:52656) 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 -- 172.21.15.45:0/3930886242 learned_addr learned my addr 172.21.15.45:0/3930886242 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7358b45700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f735406bad0 0x7f73541597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7353fff700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7354156ce0 0x7f7354148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 -- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7354156ce0 msgr2=0x7f7354148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7354156ce0 0x7f7354148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 -- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f735406bad0 msgr2=0x7f73541597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f735406bad0 0x7f73541597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 -- 172.21.15.45:0/3930886242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c009ae0 con 0x7f735415ac10 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7358b45700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f735406bad0 0x7f73541597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7353fff700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7354156ce0 0x7f7354148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7359346700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ad070 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f735406aed0 tx=0x7f734c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7350ff9700 1 -- 172.21.15.45:0/3930886242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c00bee0 con 0x7f735415ac10 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7350ff9700 1 -- 172.21.15.45:0/3930886242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c0032f0 con 0x7f735415ac10 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73541680c0 con 0x7f735415ac10 2023-10-06T05:24:09.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f7350ff9700 1 -- 172.21.15.45:0/3930886242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c01d650 con 0x7f735415ac10 2023-10-06T05:24:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.428+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7354168960 con 0x7f735415ac10 2023-10-06T05:24:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.429+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735406e2f0 con 0x7f735415ac10 2023-10-06T05:24:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.430+0000 7f7350ff9700 1 -- 172.21.15.45:0/3930886242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f734c01da80 con 0x7f735415ac10 2023-10-06T05:24:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.430+0000 7f7350ff9700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7334066f30 0x7f73340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.430+0000 7f7350ff9700 1 -- 172.21.15.45:0/3930886242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f734c086660 con 0x7f735415ac10 2023-10-06T05:24:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.431+0000 7f7358b45700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7334066f30 0x7f73340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.431+0000 7f7358b45700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7334066f30 0x7f73340693e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7344006fd0 tx=0x7f7344008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.435+0000 7f7350ff9700 1 -- 172.21.15.45:0/3930886242 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f734c053020 con 0x7f735415ac10 2023-10-06T05:24:09.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.680+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f735415b940 con 0x7f735415ac10 2023-10-06T05:24:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.963+0000 7f7350ff9700 1 -- 172.21.15.45:0/3930886242 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v203) v1 ==== 161+0+0 (secure 0 0 0) 0x7f734c012e60 con 0x7f735415ac10 2023-10-06T05:24:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.965+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7334066f30 msgr2=0x7f73340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.965+0000 7f735ada9700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7334066f30 0x7f73340693e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7344006fd0 tx=0x7f7344008040 comp rx=0 tx=0).stop 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.965+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 msgr2=0x7f73540ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.965+0000 7f735ada9700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ad070 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f735406aed0 tx=0x7f734c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 shutdown_connections 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7334066f30 0x7f73340693e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735415ac10 0x7f73540ad070 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f735406bad0 0x7f73541597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 --2- 172.21.15.45:0/3930886242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7354156ce0 0x7f7354148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 >> 172.21.15.45:0/3930886242 conn(0x7f735406f090 msgr2=0x7f73540cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 shutdown_connections 2023-10-06T05:24:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:09.966+0000 7f735ada9700 1 -- 172.21.15.45:0/3930886242 wait complete. 2023-10-06T05:24:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:09.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:10.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.117+0000 7f68ecaf4700 1 Processor -- start 2023-10-06T05:24:10.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.117+0000 7f68ecaf4700 1 -- start start 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.118+0000 7f68ecaf4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68e8153650 0x7f68e815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.118+0000 7f68ecaf4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f68e8153a20 0x7f68e806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.118+0000 7f68ecaf4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80676c0 con 0x7f68e806add0 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.118+0000 7f68ecaf4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e806b270 con 0x7f68e8153650 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.118+0000 7f68ecaf4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e80679a0 con 0x7f68e8153a20 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68e8153650 0x7f68e815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e6d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68e806add0 0x7f68e8068480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39818/0 (socket says 172.21.15.45:39818) 2023-10-06T05:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68e8153650 0x7f68e815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52662/0 (socket says 172.21.15.45:52662) 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e6d9d700 1 -- 172.21.15.45:0/320839872 learned_addr learned my addr 172.21.15.45:0/320839872 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e5d9b700 1 --2- 172.21.15.45:0/320839872 >> v2:172.21.15.144:3300/0 conn(0x7f68e8153a20 0x7f68e806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e659c700 1 -- 172.21.15.45:0/320839872 >> v2:172.21.15.144:3300/0 conn(0x7f68e8153a20 msgr2=0x7f68e806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e659c700 1 --2- 172.21.15.45:0/320839872 >> v2:172.21.15.144:3300/0 conn(0x7f68e8153a20 0x7f68e806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e659c700 1 -- 172.21.15.45:0/320839872 >> v1:172.21.15.45:6789/0 conn(0x7f68e806add0 legacy=0x7f68e8068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e659c700 1 -- 172.21.15.45:0/320839872 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8067c80 con 0x7f68e8153650 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e559a700 1 -- 172.21.15.45:0/320839872 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341002018 0 0) 0x7f68e80676c0 con 0x7f68e806add0 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e5d9b700 1 --2- 172.21.15.45:0/320839872 >> v2:172.21.15.144:3300/0 conn(0x7f68e8153a20 0x7f68e806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e659c700 1 --2- 172.21.15.45:0/320839872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153650 0x7f68e815a930 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f68e80b2660 tx=0x7f68d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6982aa285d13924 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e559a700 1 -- 172.21.15.45:0/320839872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800b6b0 con 0x7f68e8153650 2023-10-06T05:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e559a700 1 -- 172.21.15.45:0/320839872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d800b810 con 0x7f68e8153650 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.119+0000 7f68e559a700 1 -- 172.21.15.45:0/320839872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800bbb0 con 0x7f68e8153650 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 -- 172.21.15.45:0/320839872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153650 msgr2=0x7f68e815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/320839872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153650 0x7f68e815a930 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f68e80b2660 tx=0x7f68d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 -- 172.21.15.45:0/320839872 shutdown_connections 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/320839872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153650 0x7f68e815a930 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/320839872 >> v2:172.21.15.144:3300/0 conn(0x7f68e8153a20 0x7f68e806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 -- 172.21.15.45:0/320839872 >> 172.21.15.45:0/320839872 conn(0x7f68e80cc5a0 msgr2=0x7f68e80cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 -- 172.21.15.45:0/320839872 shutdown_connections 2023-10-06T05:24:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.120+0000 7f68ecaf4700 1 -- 172.21.15.45:0/320839872 wait complete. 2023-10-06T05:24:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.121+0000 7f68ecaf4700 1 Processor -- start 2023-10-06T05:24:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.121+0000 7f68ecaf4700 1 -- start start 2023-10-06T05:24:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68ecaf4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68e806add0 0x7f68e80b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68ecaf4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e8153650 0x7f68e816d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68ecaf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153a20 0x7f68e81488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68ecaf4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80b0ed0 con 0x7f68e8153a20 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68ecaf4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80b11b0 con 0x7f68e8153650 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68ecaf4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80b1490 con 0x7f68e806add0 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153a20 0x7f68e81488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68e6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153a20 0x7f68e81488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52672/0 (socket says 172.21.15.45:52672) 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68e6d9d700 1 -- 172.21.15.45:0/214679431 learned_addr learned my addr 172.21.15.45:0/214679431 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68e5d9b700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e8153650 0x7f68e816d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68e659c700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68e806add0 0x7f68e80b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68e6d9d700 1 -- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68e806add0 msgr2=0x7f68e80b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.122+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68e806add0 0x7f68e80b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68e6d9d700 1 -- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e8153650 msgr2=0x7f68e816d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e8153650 0x7f68e816d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68e6d9d700 1 -- 172.21.15.45:0/214679431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8009420 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68e6d9d700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153a20 0x7f68e81488d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f68e000cbf0 tx=0x7f68e00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68d67fc700 1 -- 172.21.15.45:0/214679431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e000a9c0 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68d67fc700 1 -- 172.21.15.45:0/214679431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e000ab20 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e80b1770 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68d67fc700 1 -- 172.21.15.45:0/214679431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00070f0 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.123+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e8148e10 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.124+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e80cc0e0 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.124+0000 7f68d67fc700 1 -- 172.21.15.45:0/214679431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68e0007520 con 0x7f68e8153a20 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.125+0000 7f68d67fc700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68c8066f30 0x7f68c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.125+0000 7f68d67fc700 1 -- 172.21.15.45:0/214679431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f68e0085230 con 0x7f68e8153a20 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.125+0000 7f68e659c700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68c8066f30 0x7f68c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.126+0000 7f68e659c700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68c8066f30 0x7f68c80693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f68d80093f0 tx=0x7f68d8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.129+0000 7f68d67fc700 1 -- 172.21.15.45:0/214679431 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68e0051c70 con 0x7f68e8153a20 2023-10-06T05:24:10.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.415+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f68e8067f60 con 0x7f68e8153a20 2023-10-06T05:24:10.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.969+0000 7f68d67fc700 1 -- 172.21.15.45:0/214679431 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v204) v1 ==== 101+0+0 (secure 0 0 0) 0x7f68e0055290 con 0x7f68e8153a20 2023-10-06T05:24:10.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68c8066f30 msgr2=0x7f68c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68c8066f30 0x7f68c80693e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f68d80093f0 tx=0x7f68d8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:10.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153a20 msgr2=0x7f68e81488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153a20 0x7f68e81488d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f68e000cbf0 tx=0x7f68e00019b0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 shutdown_connections 2023-10-06T05:24:10.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f68c8066f30 0x7f68c80693e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68e8153a20 0x7f68e81488d0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68e806add0 0x7f68e80b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 --2- 172.21.15.45:0/214679431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e8153650 0x7f68e816d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 >> 172.21.15.45:0/214679431 conn(0x7f68e80cc5a0 msgr2=0x7f68e80cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 shutdown_connections 2023-10-06T05:24:10.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:10.972+0000 7f68ecaf4700 1 -- 172.21.15.45:0/214679431 wait complete. 2023-10-06T05:24:10.976 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:10.989 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:11.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.035+0000 7f61f8282040 1 Processor -- start 2023-10-06T05:24:11.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.035+0000 7f61f8282040 1 -- start start 2023-10-06T05:24:11.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f8282040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1af19b0 0x5589a1aeeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f8282040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b5fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f8282040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1b7f140 0x5589a1b60400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f8282040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a19f1980 con 0x5589a1b7ab70 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f8282040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a19f1620 con 0x5589a1b7f140 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f8282040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589a19e8400 con 0x5589a1af19b0 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b5fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b5fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52692/0 (socket says 172.21.15.45:52692) 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 -- 172.21.15.45:0/1080246143 learned_addr learned my addr 172.21.15.45:0/1080246143 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f14a0700 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1af19b0 0x5589a1aeeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1b7f140 0x5589a1b60400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 -- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1af19b0 msgr2=0x5589a1aeeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1af19b0 0x5589a1aeeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 -- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1b7f140 msgr2=0x5589a1b60400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1b7f140 0x5589a1b60400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.036+0000 7f61f0c9f700 1 -- 172.21.15.45:0/1080246143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4005cf0 con 0x5589a1b7ab70 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b5fec0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f61e8001cc0 tx=0x7f61e8001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a74da8e91dbcfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61e37fe700 1 -- 172.21.15.45:0/1080246143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800bbb0 con 0x5589a1b7ab70 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61e37fe700 1 -- 172.21.15.45:0/1080246143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e800bd30 con 0x5589a1b7ab70 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61e37fe700 1 -- 172.21.15.45:0/1080246143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800bea0 con 0x5589a1b7ab70 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 -- 172.21.15.45:0/1080246143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 msgr2=0x5589a1b5fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b5fec0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f61e8001cc0 tx=0x7f61e8001770 comp rx=0 tx=0).stop 2023-10-06T05:24:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 -- 172.21.15.45:0/1080246143 shutdown_connections 2023-10-06T05:24:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1b7f140 0x5589a1b60400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1af19b0 0x5589a1aeeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 --2- 172.21.15.45:0/1080246143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b5fec0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 -- 172.21.15.45:0/1080246143 >> 172.21.15.45:0/1080246143 conn(0x5589a1aa85f0 msgr2=0x5589a1b79e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 -- 172.21.15.45:0/1080246143 shutdown_connections 2023-10-06T05:24:11.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.037+0000 7f61f8282040 1 -- 172.21.15.45:0/1080246143 wait complete. 2023-10-06T05:24:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.038+0000 7f61f8282040 1 Processor -- start 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.038+0000 7f61f8282040 1 -- start start 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.039+0000 7f61f8282040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1af19b0 0x5589a1ae74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.039+0000 7f61f8282040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b8edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.039+0000 7f61f8282040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1b7f140 0x5589a1b8f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.039+0000 7f61f8282040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a1b5f650 con 0x5589a1b7ab70 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.039+0000 7f61f8282040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a1aee470 con 0x5589a1af19b0 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.039+0000 7f61f8282040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589a1b5f880 con 0x5589a1b7f140 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b8edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b8edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52706/0 (socket says 172.21.15.45:52706) 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 learned_addr learned my addr 172.21.15.45:0/806173346 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1af19b0 0x5589a1ae74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1b7f140 0x5589a1b8f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1b7f140 msgr2=0x5589a1b8f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1b7f140 0x5589a1b8f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1af19b0 msgr2=0x5589a1ae74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1af19b0 0x5589a1ae74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc001e10 con 0x5589a1b7ab70 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589a1af19b0 0x5589a1ae74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5589a1b7f140 0x5589a1b8f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b8edc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f61e800b960 tx=0x7f61e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61e1ffb700 1 -- 172.21.15.45:0/806173346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e801f070 con 0x5589a1b7ab70 2023-10-06T05:24:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61e1ffb700 1 -- 172.21.15.45:0/806173346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e800bbb0 con 0x5589a1b7ab70 2023-10-06T05:24:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5589a1a003d0 con 0x5589a1b7ab70 2023-10-06T05:24:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61e1ffb700 1 -- 172.21.15.45:0/806173346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800bea0 con 0x5589a1b7ab70 2023-10-06T05:24:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.040+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589a19f3ae0 con 0x5589a1b7ab70 2023-10-06T05:24:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.042+0000 7f61e1ffb700 1 -- 172.21.15.45:0/806173346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61e801ad20 con 0x5589a1b7ab70 2023-10-06T05:24:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.042+0000 7f61e1ffb700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61d4065890 0x7f61d4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.042+0000 7f61e1ffb700 1 -- 172.21.15.45:0/806173346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f61e8085a40 con 0x5589a1b7ab70 2023-10-06T05:24:11.047 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.042+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61d4065890 0x7f61d4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.043+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589a1ae7a10 0x5589a1b76820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.043+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b5d340 con 0x5589a1ae7a10 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.043+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589a1ae7a10 0x5589a1b76820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.043+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61d4065890 0x7f61d4067d40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f61e4005a90 tx=0x7f61e4015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.043+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589a1ae7a10 0x5589a1b76820 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.045+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v110'536 uv510 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.045+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b76d60 con 0x5589a1ae7a10 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.046+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v110'537 uv537 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.046+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b77110 con 0x5589a1ae7a10 2023-10-06T05:24:11.050 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:11.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.049+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v110'538 uv538 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.049+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b774c0 con 0x5589a1ae7a10 2023-10-06T05:24:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.050+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'539 uv539 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.050+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b990b0 con 0x5589a1ae7a10 2023-10-06T05:24:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.052+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'540 uv540 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.052+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b980a0 con 0x5589a1ae7a10 2023-10-06T05:24:11.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.053+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'541 uv541 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.053+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b984d0 con 0x5589a1ae7a10 2023-10-06T05:24:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.054+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v110'542 uv516 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.054+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b988d0 con 0x5589a1ae7a10 2023-10-06T05:24:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.055+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v110'543 uv543 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.055+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.056+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v110'544 uv544 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.059 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.056+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.058+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'545 uv545 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.058+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.059+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'546 uv546 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.059+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.060+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'547 uv547 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.060+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.061+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v110'548 uv523 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.061+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.062+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'549 uv549 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.062+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.063+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v110'550 uv550 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.065 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.063+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.065+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'551 uv551 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.065+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.066+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'552 uv552 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.066+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.067+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'553 uv553 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.067+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.069+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'554 uv554 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.069+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.070+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v110'555 uv529 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.070+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.071+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v110'556 uv556 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.072+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.074 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:11.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.073+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v110'557 uv557 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.073+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.074+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'558 uv558 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.074+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.075+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'559 uv559 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.076+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.077+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'560 uv560 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.077+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.078+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v110'561 uv535 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.078+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.080+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v110'562 uv562 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.080+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.082 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.081+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v110'563 uv563 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.081+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.082+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'564 uv564 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.082+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.083+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'565 uv565 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.083+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1ae7a10 2023-10-06T05:24:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.085+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'566 uv566 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ae7a10 2023-10-06T05:24:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.085+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5589a1b99970 0x5589a1b9bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.085+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b98c80 con 0x5589a1b99970 2023-10-06T05:24:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.085+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5589a1b99970 0x5589a1b9bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.086+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5589a1b99970 0x5589a1b9bd50 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.087+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'392 uv374 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e40079f0 con 0x5589a1b99970 2023-10-06T05:24:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.087+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9c290 con 0x5589a1b99970 2023-10-06T05:24:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.088+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'393 uv393 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.088+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9c760 con 0x5589a1b99970 2023-10-06T05:24:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.089+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'394 uv394 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.093 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.089+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d070 con 0x5589a1b99970 2023-10-06T05:24:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.091+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v110'395 uv395 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.091+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9cc00 con 0x5589a1b99970 2023-10-06T05:24:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.092+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'396 uv396 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.092+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d910 con 0x5589a1b99970 2023-10-06T05:24:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.093+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v110'397 uv379 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.093+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9dcc0 con 0x5589a1b99970 2023-10-06T05:24:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.094+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v110'398 uv398 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.094+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9dcc0 con 0x5589a1b99970 2023-10-06T05:24:11.098 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.095+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v110'399 uv399 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.095+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9dcc0 con 0x5589a1b99970 2023-10-06T05:24:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.097+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'400 uv400 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.097+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d4a0 con 0x5589a1b99970 2023-10-06T05:24:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.098+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'401 uv401 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.098+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d4a0 con 0x5589a1b99970 2023-10-06T05:24:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.099+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v110'402 uv391 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.099+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9dcc0 con 0x5589a1b99970 2023-10-06T05:24:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.100+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v110'403 uv403 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.100+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9dcc0 con 0x5589a1b99970 2023-10-06T05:24:11.103 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:11.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.102+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v110'404 uv404 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.102+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba13e0 con 0x5589a1b99970 2023-10-06T05:24:11.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.103+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v110'405 uv405 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.103+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d4a0 con 0x5589a1b99970 2023-10-06T05:24:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.104+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'406 uv406 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.104+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d4a0 con 0x5589a1b99970 2023-10-06T05:24:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.105+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v110'407 uv390 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.105+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d420 con 0x5589a1b99970 2023-10-06T05:24:11.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.106+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v110'408 uv408 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.107+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d420 con 0x5589a1b99970 2023-10-06T05:24:11.110 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:11.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.108+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v110'409 uv409 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.108+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d420 con 0x5589a1b99970 2023-10-06T05:24:11.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.109+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'410 uv410 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.109+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d420 con 0x5589a1b99970 2023-10-06T05:24:11.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.110+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'411 uv411 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.110+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d420 con 0x5589a1b99970 2023-10-06T05:24:11.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.111+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'412 uv412 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.111+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5589a1b9e6c0 0x5589a1b9eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.111+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9d420 con 0x5589a1b9e6c0 2023-10-06T05:24:11.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.112+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5589a1b9e6c0 0x5589a1b9eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.112+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5589a1b9e6c0 0x5589a1b9eab0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.113+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.113+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba1220 con 0x5589a1b9e6c0 2023-10-06T05:24:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.114+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v110'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.114+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba1690 con 0x5589a1b9e6c0 2023-10-06T05:24:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.115+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v110'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.117 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.115+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1b9ddc0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.117+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.117+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba3030 con 0x5589a1b9e6c0 2023-10-06T05:24:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.118+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.118+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba1f80 con 0x5589a1b9e6c0 2023-10-06T05:24:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.119+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.119+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba2380 con 0x5589a1b9e6c0 2023-10-06T05:24:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.120+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'76 uv75 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.120+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba27a0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.121+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'77 uv77 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.121+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba27a0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.124 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.122+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'78 uv78 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.122+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba2bf0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.124+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'79 uv79 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.124+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba3450 con 0x5589a1b99970 2023-10-06T05:24:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.125+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.125+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba3450 con 0x5589a1b99970 2023-10-06T05:24:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.126+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v110'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.126+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba3450 con 0x5589a1b99970 2023-10-06T05:24:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.127+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v110'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.129 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.127+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba4460 con 0x5589a1b99970 2023-10-06T05:24:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.128+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.128+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba4460 con 0x5589a1b99970 2023-10-06T05:24:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.129+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.129+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba3450 con 0x5589a1b99970 2023-10-06T05:24:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.130+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1b99970 2023-10-06T05:24:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.131+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5589a1ba3960 0x5589a1ba5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.131+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6280 con 0x5589a1ba3960 2023-10-06T05:24:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.131+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5589a1ba3960 0x5589a1ba5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.131+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5589a1ba3960 0x5589a1ba5d40 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.132+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v110'193 uv187 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.132+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6280 con 0x5589a1ba3960 2023-10-06T05:24:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.133+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v110'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.133+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6280 con 0x5589a1ba3960 2023-10-06T05:24:11.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.135+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v110'195 uv195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.136 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.135+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba7710 con 0x5589a1ba3960 2023-10-06T05:24:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.136+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.136+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba7710 con 0x5589a1ba3960 2023-10-06T05:24:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.137+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.137+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6700 con 0x5589a1ba3960 2023-10-06T05:24:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.138+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.138+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6700 con 0x5589a1ba3960 2023-10-06T05:24:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.139+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v110'199 uv192 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.139+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6700 con 0x5589a1ba3960 2023-10-06T05:24:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.140+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v110'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.140+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6700 con 0x5589a1ba3960 2023-10-06T05:24:11.143 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.141+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v110'201 uv201 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.141+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6700 con 0x5589a1ba3960 2023-10-06T05:24:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.142+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.142+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba6700 con 0x5589a1ba3960 2023-10-06T05:24:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.144+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.144+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589a1ba6d80 0x5589a1ba9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.144+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba97c0 con 0x5589a1ba6d80 2023-10-06T05:24:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.144+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589a1ba6d80 0x5589a1ba9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.144+0000 7f61f14a0700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589a1ba6d80 0x5589a1ba9160 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.145+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v110'86 uv85 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1ba6d80 2023-10-06T05:24:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.145+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1ba6d80 2023-10-06T05:24:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.147+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v110'87 uv87 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1ba6d80 2023-10-06T05:24:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.147+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1ba6d80 2023-10-06T05:24:11.151 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.149+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v110'88 uv88 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1ba6d80 2023-10-06T05:24:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.150+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1ba6d80 2023-10-06T05:24:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.152+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'89 uv89 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1ba6d80 2023-10-06T05:24:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.152+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1ba6d80 2023-10-06T05:24:11.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.156+0000 7f61f14a0700 1 -- 172.21.15.45:0/806173346 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'90 uv90 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e4013420 con 0x5589a1ba6d80 2023-10-06T05:24:11.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.156+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.156+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.156+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.158+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v110'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.158+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.159+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v110'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.161 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:11.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.159+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1baabb0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.160+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.160+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1baabb0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.161+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.161+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1ba9ba0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.162+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.162+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5589a1baa220 0x5589a1bac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.162+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bacc60 con 0x5589a1baa220 2023-10-06T05:24:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.163+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5589a1baa220 0x5589a1bac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.163+0000 7f61f0c9f700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5589a1baa220 0x5589a1bac600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.164+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v110'172 uv166 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.164+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bacc60 con 0x5589a1baa220 2023-10-06T05:24:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.165+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v110'173 uv173 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.165+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad060 con 0x5589a1baa220 2023-10-06T05:24:11.168 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.166+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v110'174 uv174 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.166+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad060 con 0x5589a1baa220 2023-10-06T05:24:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.168+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v110'175 uv175 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.168+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad060 con 0x5589a1baa220 2023-10-06T05:24:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.169+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v110'176 uv176 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.169+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad060 con 0x5589a1baa220 2023-10-06T05:24:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.170+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v110'177 uv171 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.170+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad060 con 0x5589a1baa220 2023-10-06T05:24:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.171+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v110'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.171+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad060 con 0x5589a1baa220 2023-10-06T05:24:11.173 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.171+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v110'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.172+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad060 con 0x5589a1baa220 2023-10-06T05:24:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.173+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v110'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.173+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bafdd0 con 0x5589a1baa220 2023-10-06T05:24:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.174+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v110'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1baa220 2023-10-06T05:24:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.174+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.175+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v110'78 uv77 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.175+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.176+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v110'79 uv79 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.176+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.177+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v110'80 uv80 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.179 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.177+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.178+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v110'81 uv81 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.179+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5589a1bade60 0x5589a1bb0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.179+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1bade60 2023-10-06T05:24:11.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.179+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5589a1bade60 0x5589a1bb0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.179+0000 7f61f1ca1700 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5589a1bade60 0x5589a1bb0520 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.180+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1bade60 2023-10-06T05:24:11.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.180+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1bade60 2023-10-06T05:24:11.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.181+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v110'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1bade60 2023-10-06T05:24:11.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.181+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1bade60 2023-10-06T05:24:11.184 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:11.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.183+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v110'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1bade60 2023-10-06T05:24:11.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.183+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1bade60 2023-10-06T05:24:11.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.184+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1bade60 2023-10-06T05:24:11.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.184+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1bade60 2023-10-06T05:24:11.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.185+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1bade60 2023-10-06T05:24:11.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.185+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1bade60 2023-10-06T05:24:11.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.187+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1bade60 2023-10-06T05:24:11.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.188+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1ba3960 2023-10-06T05:24:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.189+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v110'89 uv88 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.189+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1ba3960 2023-10-06T05:24:11.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.190+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v110'90 uv90 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.190+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1ba3960 2023-10-06T05:24:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.191+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v110'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.193 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.192+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1ba3960 2023-10-06T05:24:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.193+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v110'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.193+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1ba3960 2023-10-06T05:24:11.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.194+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v110'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f61ec002c70 con 0x5589a1ba3960 2023-10-06T05:24:11.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.194+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.196+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v110'216 uv208 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.196+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.197+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v110'217 uv217 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.197+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.199+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v110'218 uv218 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.201 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.200+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.201+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v110'219 uv219 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.201+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.204+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v110'220 uv220 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.204+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.205+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v110'221 uv215 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.205+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.206+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v110'222 uv222 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.206+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.209+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v110'223 uv223 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.210 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:11.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.209+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.210+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v110'224 uv224 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.210+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.211+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v110'225 uv225 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.211+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bad550 con 0x5589a1b9e6c0 2023-10-06T05:24:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.212+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v110'226 uv226 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.212+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5589a1bb0ab0 con 0x5589a1b9e6c0 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f0c9f700 1 -- 172.21.15.45:0/806173346 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v110'227 uv227 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f61e8014070 con 0x5589a1b9e6c0 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5589a1b9e6c0 msgr2=0x5589a1b9eab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5589a1b9e6c0 0x5589a1b9eab0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589a1ae7a10 msgr2=0x5589a1b76820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5589a1ae7a10 0x5589a1b76820 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5589a1baa220 msgr2=0x5589a1bac600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5589a1baa220 0x5589a1bac600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5589a1b99970 msgr2=0x5589a1b9bd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5589a1b99970 0x5589a1b9bd50 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5589a1bade60 msgr2=0x5589a1bb0520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5589a1bade60 0x5589a1bb0520 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589a1ba6d80 msgr2=0x5589a1ba9160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5589a1ba6d80 0x5589a1ba9160 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5589a1ba3960 msgr2=0x5589a1ba5d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5589a1ba3960 0x5589a1ba5d40 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 reap_dead start 2023-10-06T05:24:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 reap_dead start 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 reap_dead start 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f1ca1700 1 -- 172.21.15.45:0/806173346 reap_dead start 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61d4065890 msgr2=0x7f61d4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.213+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61d4065890 0x7f61d4067d40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f61e4005a90 tx=0x7f61e4015000 comp rx=0 tx=0).stop 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 msgr2=0x5589a1b8edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b8edc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f61e800b960 tx=0x7f61e8014000 comp rx=0 tx=0).stop 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 shutdown_connections 2023-10-06T05:24:11.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589a1b7ab70 0x5589a1b8edc0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 --2- 172.21.15.45:0/806173346 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f61d4065890 0x7f61d4067d40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 >> 172.21.15.45:0/806173346 conn(0x5589a1aa85f0 msgr2=0x5589a1b7dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 shutdown_connections 2023-10-06T05:24:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.214+0000 7f61f8282040 1 -- 172.21.15.45:0/806173346 wait complete. 2023-10-06T05:24:11.221 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2023-10-06T05:24:11.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.311+0000 7fc81c8be040 1 Processor -- start 2023-10-06T05:24:11.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.311+0000 7fc81c8be040 1 -- start start 2023-10-06T05:24:11.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.311+0000 7fc81c8be040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc484fca0 0x557bc48ed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.311+0000 7fc81c8be040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc48f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc81c8be040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc48f6b30 0x557bc4852810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc81c8be040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc47ca320 con 0x557bc486b7e0 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc81c8be040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc476e980 con 0x557bc484fca0 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc81c8be040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bc476e620 con 0x557bc48f6b30 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc48f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc48f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52716/0 (socket says 172.21.15.45:52716) 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 -- 172.21.15.45:0/3085914942 learned_addr learned my addr 172.21.15.45:0/3085914942 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc815adc700 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc484fca0 0x557bc48ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8162dd700 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc48f6b30 0x557bc4852810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 -- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc48f6b30 msgr2=0x557bc4852810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc48f6b30 0x557bc4852810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 -- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc484fca0 msgr2=0x557bc48ed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc484fca0 0x557bc48ed740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 -- 172.21.15.45:0/3085914942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c005cf0 con 0x557bc486b7e0 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc8152db700 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc48f8450 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00acf0 tx=0x7fc80c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4713e400b96d7926 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc807fff700 1 -- 172.21.15.45:0/3085914942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c000bc0 con 0x557bc486b7e0 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.312+0000 7fc807fff700 1 -- 172.21.15.45:0/3085914942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c008970 con 0x557bc486b7e0 2023-10-06T05:24:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc807fff700 1 -- 172.21.15.45:0/3085914942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c026460 con 0x557bc486b7e0 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 -- 172.21.15.45:0/3085914942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 msgr2=0x557bc48f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc48f8450 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00acf0 tx=0x7fc80c016000 comp rx=0 tx=0).stop 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 -- 172.21.15.45:0/3085914942 shutdown_connections 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc48f6b30 0x557bc4852810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc484fca0 0x557bc48ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 --2- 172.21.15.45:0/3085914942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc48f8450 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 -- 172.21.15.45:0/3085914942 >> 172.21.15.45:0/3085914942 conn(0x557bc4824450 msgr2=0x557bc4826830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 -- 172.21.15.45:0/3085914942 shutdown_connections 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.313+0000 7fc81c8be040 1 -- 172.21.15.45:0/3085914942 wait complete. 2023-10-06T05:24:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.314+0000 7fc81c8be040 1 Processor -- start 2023-10-06T05:24:11.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.314+0000 7fc81c8be040 1 -- start start 2023-10-06T05:24:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc81c8be040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc484fca0 0x557bc48ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc81c8be040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc4824d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc81c8be040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc48f6b30 0x557bc48252a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc81c8be040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc4765400 con 0x557bc486b7e0 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc81c8be040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc48f1960 con 0x557bc48f6b30 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc81c8be040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bc48f1c00 con 0x557bc484fca0 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc8152db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc4824d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc8152db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc4824d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52732/0 (socket says 172.21.15.45:52732) 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc8152db700 1 -- 172.21.15.45:0/988538379 learned_addr learned my addr 172.21.15.45:0/988538379 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc815adc700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc484fca0 0x557bc48ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc8162dd700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc48f6b30 0x557bc48252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc815adc700 1 -- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc48f6b30 msgr2=0x557bc48252a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc815adc700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bc48f6b30 0x557bc48252a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc815adc700 1 -- 172.21.15.45:0/988538379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 msgr2=0x557bc4824d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc815adc700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc4824d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc815adc700 1 -- 172.21.15.45:0/988538379 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bc477d3d0 con 0x557bc484fca0 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.315+0000 7fc8152db700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bc486b7e0 0x557bc4824d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.316+0000 7fc815adc700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc484fca0 0x557bc48ec310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc808003cc0 tx=0x7fc8080038e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.316+0000 7fc8067fc700 1 -- 172.21.15.45:0/988538379 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8080032e0 con 0x557bc484fca0 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.316+0000 7fc8067fc700 1 -- 172.21.15.45:0/988538379 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808008040 con 0x557bc484fca0 2023-10-06T05:24:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.316+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557bc4770ae0 con 0x557bc484fca0 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.316+0000 7fc8067fc700 1 -- 172.21.15.45:0/988538379 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800bc00 con 0x557bc484fca0 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.316+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bc4770540 con 0x557bc484fca0 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.317+0000 7fc8067fc700 1 -- 172.21.15.45:0/988538379 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc8080015a0 con 0x557bc484fca0 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.318+0000 7fc8067fc700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7f8065890 0x7fc7f8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.318+0000 7fc8067fc700 1 -- 172.21.15.45:0/988538379 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7fc808083750 con 0x557bc484fca0 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.318+0000 7fc8152db700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7f8065890 0x7fc7f8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.318+0000 7fc81c8be040 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557bc4852d20 0x557bc4867a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.318+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.15 14:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557bc4867fd0 con 0x557bc4852d20 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.318+0000 7fc8162dd700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557bc4852d20 0x557bc4867a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.318+0000 7fc8152db700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7f8065890 0x7fc7f8067d40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00a520 tx=0x7fc80c01e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.319+0000 7fc8162dd700 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557bc4852d20 0x557bc4867a90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc8162dd700 1 -- 172.21.15.45:0/988538379 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v110'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc810001a00 con 0x557bc4852d20 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557bc4852d20 msgr2=0x557bc4867a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557bc4852d20 0x557bc4867a90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7f8065890 msgr2=0x7fc7f8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc7f8065890 0x7fc7f8067d40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00a520 tx=0x7fc80c01e000 comp rx=0 tx=0).stop 2023-10-06T05:24:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc484fca0 msgr2=0x557bc48ec310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 --2- 172.21.15.45:0/988538379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bc484fca0 0x557bc48ec310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc808003cc0 tx=0x7fc8080038e0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc8162dd700 1 -- 172.21.15.45:0/988538379 reap_dead start 2023-10-06T05:24:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 shutdown_connections 2023-10-06T05:24:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 >> 172.21.15.45:0/988538379 conn(0x557bc4824450 msgr2=0x557bc48f6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 shutdown_connections 2023-10-06T05:24:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.320+0000 7fc81c8be040 1 -- 172.21.15.45:0/988538379 wait complete. 2023-10-06T05:24:11.328 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.509+0000 7f625bb8b700 1 Processor -- start 2023-10-06T05:24:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.509+0000 7f625bb8b700 1 -- start start 2023-10-06T05:24:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.510+0000 7f625bb8b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f625413f900 0x7f62540cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.510+0000 7f625bb8b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f62540c9f20 0x7f62540affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.510+0000 7f625bb8b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62540c9630 con 0x7f62540c8de0 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.510+0000 7f625bb8b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62540cb550 con 0x7f62540c9f20 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.510+0000 7f625bb8b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540cb830 con 0x7f625413f900 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f625a128700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f62540c9f20 0x7f62540affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6259927700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f625413f900 0x7f62540cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6259927700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f625413f900 0x7f62540cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:46594/0 (socket says 172.21.15.45:46594) 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f625a128700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f62540c9f20 0x7f62540affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:45480/0 (socket says 172.21.15.45:45480) 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6259126700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f62540c8de0 0x7f62540c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:35382/0 (socket says 172.21.15.45:35382) 2023-10-06T05:24:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6259927700 1 -- 172.21.15.45:0/3820919135 learned_addr learned my addr 172.21.15.45:0/3820919135 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f625a128700 1 -- 172.21.15.45:0/3820919135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f625413f900 msgr2=0x7f62540cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f625a128700 1 --2- 172.21.15.45:0/3820919135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f625413f900 0x7f62540cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f625a128700 1 -- 172.21.15.45:0/3820919135 >> v1:172.21.15.144:6789/0 conn(0x7f62540c8de0 legacy=0x7f62540c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6258925700 1 -- 172.21.15.45:0/3820919135 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280482985 0 0) 0x7f62540c9630 con 0x7f62540c8de0 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f625a128700 1 -- 172.21.15.45:0/3820919135 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540cbb10 con 0x7f62540c9f20 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f625a128700 1 --2- 172.21.15.45:0/3820919135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540affa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f624c009e60 tx=0x7f624c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe7fe83904d90f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6258925700 1 -- 172.21.15.45:0/3820919135 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c00e070 con 0x7f62540c9f20 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6258925700 1 -- 172.21.15.45:0/3820919135 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c00bb60 con 0x7f62540c9f20 2023-10-06T05:24:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.511+0000 7f6258925700 1 -- 172.21.15.45:0/3820919135 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c012430 con 0x7f62540c9f20 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 -- 172.21.15.45:0/3820919135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 msgr2=0x7f62540affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 --2- 172.21.15.45:0/3820919135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540affa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f624c009e60 tx=0x7f624c009780 comp rx=0 tx=0).stop 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 -- 172.21.15.45:0/3820919135 shutdown_connections 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 --2- 172.21.15.45:0/3820919135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540affa0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 --2- 172.21.15.45:0/3820919135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f625413f900 0x7f62540cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 -- 172.21.15.45:0/3820919135 >> 172.21.15.45:0/3820919135 conn(0x7f62540691c0 msgr2=0x7f62540cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 -- 172.21.15.45:0/3820919135 shutdown_connections 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.512+0000 7f625bb8b700 1 -- 172.21.15.45:0/3820919135 wait complete. 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.513+0000 7f625bb8b700 1 Processor -- start 2023-10-06T05:24:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.513+0000 7f625bb8b700 1 -- start start 2023-10-06T05:24:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625bb8b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62540c8de0 0x7f6254153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625bb8b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625bb8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625413f900 0x7f62540c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625bb8b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540c56c0 con 0x7f625413f900 2023-10-06T05:24:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625bb8b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254166ff0 con 0x7f62540c9f20 2023-10-06T05:24:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625bb8b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6254167290 con 0x7f62540c8de0 2023-10-06T05:24:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625a128700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625413f900 0x7f62540c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f6259927700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62540c8de0 0x7f6254153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625a128700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625413f900 0x7f62540c23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52744/0 (socket says 172.21.15.45:52744) 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f625a128700 1 -- 172.21.15.45:0/1432602144 learned_addr learned my addr 172.21.15.45:0/1432602144 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.514+0000 7f6259126700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6259126700 1 -- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62540c8de0 msgr2=0x7f6254153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6259126700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62540c8de0 0x7f6254153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6259126700 1 -- 172.21.15.45:0/1432602144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625413f900 msgr2=0x7f62540c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6259126700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625413f900 0x7f62540c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6259126700 1 -- 172.21.15.45:0/1432602144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c009a20 con 0x7f62540c9f20 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6259927700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62540c8de0 0x7f6254153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f625a128700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625413f900 0x7f62540c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:11.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6259126700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540c1ea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f625000c5b0 tx=0x7f6250007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6249ffb700 1 -- 172.21.15.45:0/1432602144 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250005b40 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6249ffb700 1 -- 172.21.15.45:0/1432602144 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6250005ca0 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6254153f60 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f6249ffb700 1 -- 172.21.15.45:0/1432602144 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250007320 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.515+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62540c5100 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.516+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625406e2d0 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.517+0000 7f6249ffb700 1 -- 172.21.15.45:0/1432602144 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f625001b3b0 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.517+0000 7f6249ffb700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6238066f30 0x7f62380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.517+0000 7f6249ffb700 1 -- 172.21.15.45:0/1432602144 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f6250084270 con 0x7f62540c9f20 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.517+0000 7f6259927700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6238066f30 0x7f62380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.518+0000 7f6259927700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6238066f30 0x7f62380693e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6244006fd0 tx=0x7f6244009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.522+0000 7f6249ffb700 1 -- 172.21.15.45:0/1432602144 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6250050b90 con 0x7f62540c9f20 2023-10-06T05:24:11.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.767+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f625413fcd0 con 0x7f62540c9f20 2023-10-06T05:24:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.977+0000 7f6249ffb700 1 -- 172.21.15.45:0/1432602144 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v205) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6250017020 con 0x7f62540c9f20 2023-10-06T05:24:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6238066f30 msgr2=0x7f62380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6238066f30 0x7f62380693e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6244006fd0 tx=0x7f6244009250 comp rx=0 tx=0).stop 2023-10-06T05:24:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 msgr2=0x7f62540c1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540c1ea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f625000c5b0 tx=0x7f6250007d50 comp rx=0 tx=0).stop 2023-10-06T05:24:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 shutdown_connections 2023-10-06T05:24:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6238066f30 0x7f62380693e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625413f900 0x7f62540c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62540c8de0 0x7f6254153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 --2- 172.21.15.45:0/1432602144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62540c9f20 0x7f62540c1ea0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 >> 172.21.15.45:0/1432602144 conn(0x7f62540691c0 msgr2=0x7f62541557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 shutdown_connections 2023-10-06T05:24:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:11.980+0000 7f625bb8b700 1 -- 172.21.15.45:0/1432602144 wait complete. 2023-10-06T05:24:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:11.984 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:11.982+0000 7f6f9378d700 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2023-10-06T05:24:11.984 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:11.982+0000 7f6f9378d700 -1 mds.0.206 unhandled write error (22) Invalid argument, force readonly... 2023-10-06T05:24:11.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.130+0000 7fe13e850700 1 Processor -- start 2023-10-06T05:24:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.130+0000 7fe13e850700 1 -- start start 2023-10-06T05:24:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe13e850700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe138158f60 0x7fe138159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe13e850700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1380cc390 0x7fe1380cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe13e850700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380cb530 con 0x7fe13806a060 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe13e850700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13806a500 con 0x7fe138158f60 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe13e850700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380ccd50 con 0x7fe1380cc390 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe137fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe138158f60 0x7fe138159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe137fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe138158f60 0x7fe138159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52752/0 (socket says 172.21.15.45:52752) 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe1377fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe13806a060 0x7fe1380cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39850/0 (socket says 172.21.15.45:39850) 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe137fff700 1 -- 172.21.15.45:0/1741887682 learned_addr learned my addr 172.21.15.45:0/1741887682 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe13cded700 1 --2- 172.21.15.45:0/1741887682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1380cc390 0x7fe1380cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe137fff700 1 -- 172.21.15.45:0/1741887682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1380cc390 msgr2=0x7fe1380cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe137fff700 1 --2- 172.21.15.45:0/1741887682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1380cc390 0x7fe1380cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe137fff700 1 -- 172.21.15.45:0/1741887682 >> v1:172.21.15.45:6789/0 conn(0x7fe13806a060 legacy=0x7fe1380cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.131+0000 7fe136ffd700 1 -- 172.21.15.45:0/1741887682 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948192692 0 0) 0x7fe128001110 con 0x7fe13806a060 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.132+0000 7fe137fff700 1 -- 172.21.15.45:0/1741887682 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138069bd0 con 0x7fe138158f60 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.132+0000 7fe137fff700 1 --2- 172.21.15.45:0/1741887682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe138158f60 0x7fe138159330 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe1200099e0 tx=0x7fe120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807dbae07afaf417 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.132+0000 7fe136ffd700 1 -- 172.21.15.45:0/1741887682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000b820 con 0x7fe138158f60 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.132+0000 7fe136ffd700 1 -- 172.21.15.45:0/1741887682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12000b980 con 0x7fe138158f60 2023-10-06T05:24:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.132+0000 7fe136ffd700 1 -- 172.21.15.45:0/1741887682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000bd20 con 0x7fe138158f60 2023-10-06T05:24:12.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.132+0000 7fe13e850700 1 -- 172.21.15.45:0/1741887682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe138158f60 msgr2=0x7fe138159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.132+0000 7fe13e850700 1 --2- 172.21.15.45:0/1741887682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe138158f60 0x7fe138159330 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fe1200099e0 tx=0x7fe120009250 comp rx=0 tx=0).stop 2023-10-06T05:24:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.133+0000 7fe13e850700 1 -- 172.21.15.45:0/1741887682 shutdown_connections 2023-10-06T05:24:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.133+0000 7fe13e850700 1 --2- 172.21.15.45:0/1741887682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1380cc390 0x7fe1380cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.133+0000 7fe13e850700 1 --2- 172.21.15.45:0/1741887682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe138158f60 0x7fe138159330 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.133+0000 7fe13e850700 1 -- 172.21.15.45:0/1741887682 >> 172.21.15.45:0/1741887682 conn(0x7fe1380cdf50 msgr2=0x7fe1380ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.133+0000 7fe13e850700 1 -- 172.21.15.45:0/1741887682 shutdown_connections 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.133+0000 7fe13e850700 1 -- 172.21.15.45:0/1741887682 wait complete. 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.134+0000 7fe13e850700 1 Processor -- start 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.134+0000 7fe13e850700 1 -- start start 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe13e850700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe13806a060 0x7fe138158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe13e850700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1380cc390 0x7fe138153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe13e850700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe138158f60 0x7fe13815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe13e850700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138154130 con 0x7fe13806a060 2023-10-06T05:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe13e850700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13815c120 con 0x7fe1380cc390 2023-10-06T05:24:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe13e850700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13815cf60 con 0x7fe138158f60 2023-10-06T05:24:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe13806a060 0x7fe138158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe13806a060 0x7fe138158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52758/0 (socket says 172.21.15.45:52758) 2023-10-06T05:24:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 -- 172.21.15.45:0/3594799444 learned_addr learned my addr 172.21.15.45:0/3594799444 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe13cded700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe138158f60 0x7fe13815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe1377fe700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1380cc390 0x7fe138153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 -- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe138158f60 msgr2=0x7fe13815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe138158f60 0x7fe13815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 -- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1380cc390 msgr2=0x7fe138153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1380cc390 0x7fe138153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.135+0000 7fe137fff700 1 -- 172.21.15.45:0/3594799444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200094f0 con 0x7fe13806a060 2023-10-06T05:24:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.136+0000 7fe137fff700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe13806a060 0x7fe138158410 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe120006fd0 tx=0x7fe120017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.136+0000 7fe12ffff700 1 -- 172.21.15.45:0/3594799444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12001f070 con 0x7fe13806a060 2023-10-06T05:24:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.136+0000 7fe12ffff700 1 -- 172.21.15.45:0/3594799444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe120011c40 con 0x7fe13806a060 2023-10-06T05:24:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.136+0000 7fe12ffff700 1 -- 172.21.15.45:0/3594799444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120025420 con 0x7fe13806a060 2023-10-06T05:24:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.136+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13815c350 con 0x7fe13806a060 2023-10-06T05:24:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.136+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13815c910 con 0x7fe13806a060 2023-10-06T05:24:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.137+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13806f410 con 0x7fe13806a060 2023-10-06T05:24:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.137+0000 7fe12ffff700 1 -- 172.21.15.45:0/3594799444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe120023070 con 0x7fe13806a060 2023-10-06T05:24:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.138+0000 7fe12ffff700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe118066f30 0x7fe1180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.138+0000 7fe12ffff700 1 -- 172.21.15.45:0/3594799444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7fe1200861e0 con 0x7fe13806a060 2023-10-06T05:24:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.138+0000 7fe1377fe700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe118066f30 0x7fe1180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.139+0000 7fe1377fe700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe118066f30 0x7fe1180693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe13806c540 tx=0x7fe128007a30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.142+0000 7fe12ffff700 1 -- 172.21.15.45:0/3594799444 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe120052ba0 con 0x7fe13806a060 2023-10-06T05:24:12.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.388+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe138068fc0 con 0x7fe13806a060 2023-10-06T05:24:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.388+0000 7fe12ffff700 1 -- 172.21.15.45:0/3594799444 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+4608 (secure 0 0 0) 0x7fe12002a090 con 0x7fe13806a060 2023-10-06T05:24:12.390 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:12.390 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":206,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:11.974773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9003":{"gid":9003,"name":"c","rank":0,"incarnation":206,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/3031536032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3031536032},{"type":"v1","addr":"172.21.15.45:6835","nonce":3031536032}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:12.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.390+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe118066f30 msgr2=0x7fe1180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.390+0000 7fe13e850700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe118066f30 0x7fe1180693e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe13806c540 tx=0x7fe128007a30 comp rx=0 tx=0).stop 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.390+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe13806a060 msgr2=0x7fe138158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.390+0000 7fe13e850700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe13806a060 0x7fe138158410 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fe120006fd0 tx=0x7fe120017000 comp rx=0 tx=0).stop 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 shutdown_connections 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe118066f30 0x7fe1180693e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe138158f60 0x7fe13815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe13806a060 0x7fe138158410 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 --2- 172.21.15.45:0/3594799444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1380cc390 0x7fe138153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 >> 172.21.15.45:0/3594799444 conn(0x7fe1380cdf50 msgr2=0x7fe13806d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 shutdown_connections 2023-10-06T05:24:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:12.391+0000 7fe13e850700 1 -- 172.21.15.45:0/3594799444 wait complete. 2023-10-06T05:24:12.394 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 206 2023-10-06T05:24:12.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.116+0000 7f025a21f700 1 Processor -- start 2023-10-06T05:24:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.117+0000 7f025a21f700 1 -- start start 2023-10-06T05:24:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.117+0000 7f025a21f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02540ca810 0x7f02540cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.117+0000 7f025a21f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540c9640 0x7f02540b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.117+0000 7f025a21f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02540c84c0 con 0x7f02540c91f0 2023-10-06T05:24:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.117+0000 7f025a21f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02540c8d60 con 0x7f02540ca810 2023-10-06T05:24:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.117+0000 7f025a21f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0254069d00 con 0x7f02540c9640 2023-10-06T05:24:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02537fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02540ca810 0x7f02540cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f0252ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f02540c91f0 0x7f02540afa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39860/0 (socket says 172.21.15.45:39860) 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f0252ffd700 1 -- 172.21.15.45:0/2762409714 learned_addr learned my addr 172.21.15.45:0/2762409714 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f0253fff700 1 --2- 172.21.15.45:0/2762409714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540c9640 0x7f02540b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02537fe700 1 -- 172.21.15.45:0/2762409714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540c9640 msgr2=0x7f02540b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02537fe700 1 --2- 172.21.15.45:0/2762409714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540c9640 0x7f02540b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02537fe700 1 -- 172.21.15.45:0/2762409714 >> v1:172.21.15.45:6789/0 conn(0x7f02540c91f0 legacy=0x7f02540afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02527fc700 1 -- 172.21.15.45:0/2762409714 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466432525 0 0) 0x7f02540c84c0 con 0x7f02540c91f0 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02537fe700 1 -- 172.21.15.45:0/2762409714 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540c8710 con 0x7f02540ca810 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02537fe700 1 --2- 172.21.15.45:0/2762409714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540ca810 0x7f02540cabe0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f02440098e0 tx=0x7f02440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e469271a556b57f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02527fc700 1 -- 172.21.15.45:0/2762409714 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244009e80 con 0x7f02540ca810 2023-10-06T05:24:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02527fc700 1 -- 172.21.15.45:0/2762409714 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024400b860 con 0x7f02540ca810 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.118+0000 7f02527fc700 1 -- 172.21.15.45:0/2762409714 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400bbe0 con 0x7f02540ca810 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 -- 172.21.15.45:0/2762409714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540ca810 msgr2=0x7f02540cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 --2- 172.21.15.45:0/2762409714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540ca810 0x7f02540cabe0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f02440098e0 tx=0x7f02440091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 -- 172.21.15.45:0/2762409714 shutdown_connections 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 --2- 172.21.15.45:0/2762409714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540c9640 0x7f02540b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 --2- 172.21.15.45:0/2762409714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540ca810 0x7f02540cabe0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 -- 172.21.15.45:0/2762409714 >> 172.21.15.45:0/2762409714 conn(0x7f0254067620 msgr2=0x7f0254067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 -- 172.21.15.45:0/2762409714 shutdown_connections 2023-10-06T05:24:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.119+0000 7f025a21f700 1 -- 172.21.15.45:0/2762409714 wait complete. 2023-10-06T05:24:13.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.121+0000 7f025a21f700 1 Processor -- start 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.121+0000 7f025a21f700 1 -- start start 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f025a21f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c91f0 0x7f02540c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f025a21f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540c9640 0x7f025415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f025a21f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540ca810 0x7f02540b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f025a21f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540c5e90 con 0x7f02540c9640 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f025a21f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540c5280 con 0x7f02540c91f0 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f025a21f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02540c5560 con 0x7f02540ca810 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540c9640 0x7f025415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540c9640 0x7f025415a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52768/0 (socket says 172.21.15.45:52768) 2023-10-06T05:24:13.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 -- 172.21.15.45:0/3634449298 learned_addr learned my addr 172.21.15.45:0/3634449298 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0253fff700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540ca810 0x7f02540b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f02537fe700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c91f0 0x7f02540c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 -- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540ca810 msgr2=0x7f02540b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540ca810 0x7f02540b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 -- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c91f0 msgr2=0x7f02540c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c91f0 0x7f02540c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.122+0000 7f0252ffd700 1 -- 172.21.15.45:0/3634449298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0244009420 con 0x7f02540c9640 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.123+0000 7f0252ffd700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540c9640 0x7f025415a3f0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f024800c7e0 tx=0x7f0248007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.123+0000 7f023b7fe700 1 -- 172.21.15.45:0/3634449298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248005b30 con 0x7f02540c9640 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.123+0000 7f023b7fe700 1 -- 172.21.15.45:0/3634449298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0248005c90 con 0x7f02540c9640 2023-10-06T05:24:13.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.123+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02540c5840 con 0x7f02540c9640 2023-10-06T05:24:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.123+0000 7f023b7fe700 1 -- 172.21.15.45:0/3634449298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248007300 con 0x7f02540c9640 2023-10-06T05:24:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.123+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02540c4f30 con 0x7f02540c9640 2023-10-06T05:24:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.124+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02540c1980 con 0x7f02540c9640 2023-10-06T05:24:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.124+0000 7f023b7fe700 1 -- 172.21.15.45:0/3634449298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f024801b400 con 0x7f02540c9640 2023-10-06T05:24:13.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.125+0000 7f023b7fe700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0240066f60 0x7f0240069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.125+0000 7f023b7fe700 1 -- 172.21.15.45:0/3634449298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f0248084880 con 0x7f02540c9640 2023-10-06T05:24:13.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.125+0000 7f02537fe700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0240066f60 0x7f0240069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.125+0000 7f02537fe700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0240066f60 0x7f0240069410 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f02440091e0 tx=0x7f0244017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.129+0000 7f023b7fe700 1 -- 172.21.15.45:0/3634449298 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02540c1980 con 0x7f02540c9640 2023-10-06T05:24:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.375+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02540c1980 con 0x7f02540c9640 2023-10-06T05:24:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.376+0000 7f023b7fe700 1 -- 172.21.15.45:0/3634449298 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+4611 (secure 0 0 0) 0x7f02540c1980 con 0x7f02540c9640 2023-10-06T05:24:13.378 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:13.378 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":207,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":207,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:12.978825+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9003":{"gid":9003,"name":"c","rank":0,"incarnation":206,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.45:6835/3031536032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3031536032},{"type":"v1","addr":"172.21.15.45:6835","nonce":3031536032}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0240066f60 msgr2=0x7f0240069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0240066f60 0x7f0240069410 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f02440091e0 tx=0x7f0244017000 comp rx=0 tx=0).stop 2023-10-06T05:24:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540c9640 msgr2=0x7f025415a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540c9640 0x7f025415a3f0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f024800c7e0 tx=0x7f0248007df0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 shutdown_connections 2023-10-06T05:24:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0240066f60 0x7f0240069410 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540ca810 0x7f02540b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c91f0 0x7f02540c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 --2- 172.21.15.45:0/3634449298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02540c9640 0x7f025415a3f0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 >> 172.21.15.45:0/3634449298 conn(0x7f0254067620 msgr2=0x7f02540b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 shutdown_connections 2023-10-06T05:24:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:13.378+0000 7f025a21f700 1 -- 172.21.15.45:0/3634449298 wait complete. 2023-10-06T05:24:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 207 2023-10-06T05:24:13.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 207, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:24:12.978825+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9003}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9003': {'gid': 9003, 'name': 'c', 'rank': 0, 'incarnation': 206, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.45:6835/3031536032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 3031536032}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 3031536032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:24:13.394 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9003:up:reconnect 2023-10-06T05:24:13.394 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:24:18.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.533+0000 7f281878a700 1 Processor -- start 2023-10-06T05:24:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.533+0000 7f281878a700 1 -- start start 2023-10-06T05:24:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.534+0000 7f281878a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28100c93c0 0x7f2810156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.534+0000 7f281878a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281015b000 con 0x7f2810157030 2023-10-06T05:24:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.534+0000 7f281878a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281015b2e0 con 0x7f28100c8ff0 2023-10-06T05:24:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.534+0000 7f281878a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28100c9830 con 0x7f28100c93c0 2023-10-06T05:24:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.534+0000 7f2816d27700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2810157030 0x7f28100cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59504/0 (socket says 172.21.15.45:59504) 2023-10-06T05:24:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.534+0000 7f2816d27700 1 -- 172.21.15.45:0/2685869733 learned_addr learned my addr 172.21.15.45:0/2685869733 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815d25700 1 --2- 172.21.15.45:0/2685869733 >> v2:172.21.15.144:3300/0 conn(0x7f28100c93c0 0x7f2810156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808960150 0 0) 0x7f281015b000 con 0x7f2810157030 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27fc003570 con 0x7f2810157030 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304771751 0 0) 0x7f281015b2e0 con 0x7f28100c8ff0 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f281015b000 con 0x7f28100c8ff0 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922883928 0 0) 0x7f27fc003570 con 0x7f2810157030 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815d25700 1 -- 172.21.15.45:0/2685869733 >> v1:172.21.15.144:6789/0 conn(0x7f28100c8ff0 legacy=0x7f28100caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815d25700 1 -- 172.21.15.45:0/2685869733 >> v1:172.21.15.45:6789/0 conn(0x7f2810157030 legacy=0x7f28100cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815d25700 1 -- 172.21.15.45:0/2685869733 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281015b5c0 con 0x7f28100c93c0 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815d25700 1 --2- 172.21.15.45:0/2685869733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28100c93c0 0x7f2810156af0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f28040098e0 tx=0x7f28040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4cb1fb633f454e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28080017f0 con 0x7f28100c93c0 2023-10-06T05:24:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804011420 con 0x7f28100c93c0 2023-10-06T05:24:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.535+0000 7f2815524700 1 -- 172.21.15.45:0/2685869733 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28040117c0 con 0x7f28100c93c0 2023-10-06T05:24:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.536+0000 7f281878a700 1 -- 172.21.15.45:0/2685869733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28100c93c0 msgr2=0x7f2810156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.536+0000 7f281878a700 1 --2- 172.21.15.45:0/2685869733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28100c93c0 0x7f2810156af0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f28040098e0 tx=0x7f28040091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.536+0000 7f281878a700 1 -- 172.21.15.45:0/2685869733 shutdown_connections 2023-10-06T05:24:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.536+0000 7f281878a700 1 --2- 172.21.15.45:0/2685869733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28100c93c0 0x7f2810156af0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.536+0000 7f281878a700 1 -- 172.21.15.45:0/2685869733 >> 172.21.15.45:0/2685869733 conn(0x7f28100cdf50 msgr2=0x7f28100ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.536+0000 7f281878a700 1 -- 172.21.15.45:0/2685869733 shutdown_connections 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.536+0000 7f281878a700 1 -- 172.21.15.45:0/2685869733 wait complete. 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.537+0000 7f281878a700 1 Processor -- start 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.537+0000 7f281878a700 1 -- start start 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f281878a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28100c8ff0 0x7f28100c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f281878a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28100c93c0 0x7f28100c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f281878a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2810157030 0x7f281014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f281878a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28100c63b0 con 0x7f28100c8ff0 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f281878a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28100c5540 con 0x7f2810157030 2023-10-06T05:24:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f281878a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28100c57e0 con 0x7f28100c93c0 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f2816526700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28100c8ff0 0x7f28100c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f2816526700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28100c8ff0 0x7f28100c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54110/0 (socket says 172.21.15.45:54110) 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.538+0000 7f2816526700 1 -- 172.21.15.45:0/952883460 learned_addr learned my addr 172.21.15.45:0/952883460 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816d27700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2810157030 0x7f281014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2815d25700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28100c93c0 0x7f28100c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816526700 1 -- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28100c93c0 msgr2=0x7f28100c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816526700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28100c93c0 0x7f28100c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816526700 1 -- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2810157030 msgr2=0x7f281014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816526700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2810157030 0x7f281014aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816526700 1 -- 172.21.15.45:0/952883460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2804009420 con 0x7f28100c8ff0 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816d27700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2810157030 0x7f281014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f2816526700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28100c8ff0 0x7f28100c7a80 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c580 tx=0x7f280c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f28027fc700 1 -- 172.21.15.45:0/952883460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c010510 con 0x7f28100c8ff0 2023-10-06T05:24:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f28027fc700 1 -- 172.21.15.45:0/952883460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c010670 con 0x7f28100c8ff0 2023-10-06T05:24:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28100c5ac0 con 0x7f28100c8ff0 2023-10-06T05:24:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f28027fc700 1 -- 172.21.15.45:0/952883460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c010a30 con 0x7f28100c8ff0 2023-10-06T05:24:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.539+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281014b010 con 0x7f28100c8ff0 2023-10-06T05:24:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.540+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28100c5fe0 con 0x7f28100c8ff0 2023-10-06T05:24:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.541+0000 7f28027fc700 1 -- 172.21.15.45:0/952883460 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f280c01b400 con 0x7f28100c8ff0 2023-10-06T05:24:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.541+0000 7f28027fc700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27f4066f30 0x7f27f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.541+0000 7f28027fc700 1 -- 172.21.15.45:0/952883460 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f280c084a40 con 0x7f28100c8ff0 2023-10-06T05:24:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.541+0000 7f2815d25700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27f4066f30 0x7f27f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.542+0000 7f2815d25700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27f4066f30 0x7f27f40693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f28040093f0 tx=0x7f2804016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.546+0000 7f28027fc700 1 -- 172.21.15.45:0/952883460 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28100c5fe0 con 0x7f28100c8ff0 2023-10-06T05:24:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.791+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28100c5fe0 con 0x7f28100c8ff0 2023-10-06T05:24:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.792+0000 7f28027fc700 1 -- 172.21.15.45:0/952883460 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f28100c5fe0 con 0x7f28100c8ff0 2023-10-06T05:24:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:18.794 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":209,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:14.989809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9003":{"gid":9003,"name":"c","rank":0,"incarnation":206,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/3031536032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3031536032},{"type":"v1","addr":"172.21.15.45:6835","nonce":3031536032}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:18.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27f4066f30 msgr2=0x7f27f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27f4066f30 0x7f27f40693e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f28040093f0 tx=0x7f2804016000 comp rx=0 tx=0).stop 2023-10-06T05:24:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28100c8ff0 msgr2=0x7f28100c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28100c8ff0 0x7f28100c7a80 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f280c00c580 tx=0x7f280c007e10 comp rx=0 tx=0).stop 2023-10-06T05:24:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 shutdown_connections 2023-10-06T05:24:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27f4066f30 0x7f27f40693e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2810157030 0x7f281014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28100c8ff0 0x7f28100c7a80 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 --2- 172.21.15.45:0/952883460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28100c93c0 0x7f28100c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 >> 172.21.15.45:0/952883460 conn(0x7f28100cdf50 msgr2=0x7f281015cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:18.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 shutdown_connections 2023-10-06T05:24:18.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:18.794+0000 7f281878a700 1 -- 172.21.15.45:0/952883460 wait complete. 2023-10-06T05:24:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 209 2023-10-06T05:24:19.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.350+0000 7f70e2685700 1 Processor -- start 2023-10-06T05:24:19.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.350+0000 7f70e2685700 1 -- start start 2023-10-06T05:24:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70e2685700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f70dc0cd3f0 0x7f70dc06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70e2685700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc15b380 con 0x7f70dc1570b0 2023-10-06T05:24:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70e2685700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc15b660 con 0x7f70dc156ce0 2023-10-06T05:24:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70e2685700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc06c010 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70db7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f70dc1570b0 0x7f70dc0ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59506/0 (socket says 172.21.15.45:59506) 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70db7fe700 1 -- 172.21.15.45:0/4183034559 learned_addr learned my addr 172.21.15.45:0/4183034559 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70e0c22700 1 --2- 172.21.15.45:0/4183034559 >> v2:172.21.15.144:3300/0 conn(0x7f70dc0cd3f0 0x7f70dc06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70daffd700 1 -- 172.21.15.45:0/4183034559 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906638584 0 0) 0x7f70dc15b380 con 0x7f70dc1570b0 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.351+0000 7f70daffd700 1 -- 172.21.15.45:0/4183034559 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c8003570 con 0x7f70dc1570b0 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.352+0000 7f70e0c22700 1 -- 172.21.15.45:0/4183034559 >> v1:172.21.15.144:6789/0 conn(0x7f70dc156ce0 legacy=0x7f70dc0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.352+0000 7f70e0c22700 1 -- 172.21.15.45:0/4183034559 >> v1:172.21.15.45:6789/0 conn(0x7f70dc1570b0 legacy=0x7f70dc0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.352+0000 7f70e0c22700 1 -- 172.21.15.45:0/4183034559 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc15b940 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.352+0000 7f70e0c22700 1 --2- 172.21.15.45:0/4183034559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc06bad0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009e60 tx=0x7f70d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70639f8f77aec16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.352+0000 7f70daffd700 1 -- 172.21.15.45:0/4183034559 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400e070 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.352+0000 7f70daffd700 1 -- 172.21.15.45:0/4183034559 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d400bb60 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.352+0000 7f70daffd700 1 -- 172.21.15.45:0/4183034559 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4012430 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.353+0000 7f70e2685700 1 -- 172.21.15.45:0/4183034559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc0cd3f0 msgr2=0x7f70dc06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.353+0000 7f70e2685700 1 --2- 172.21.15.45:0/4183034559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc06bad0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009e60 tx=0x7f70d4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.353+0000 7f70e2685700 1 -- 172.21.15.45:0/4183034559 shutdown_connections 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.353+0000 7f70e2685700 1 --2- 172.21.15.45:0/4183034559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc06bad0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.353+0000 7f70e2685700 1 -- 172.21.15.45:0/4183034559 >> 172.21.15.45:0/4183034559 conn(0x7f70dc06f090 msgr2=0x7f70dc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.353+0000 7f70e2685700 1 -- 172.21.15.45:0/4183034559 shutdown_connections 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.353+0000 7f70e2685700 1 -- 172.21.15.45:0/4183034559 wait complete. 2023-10-06T05:24:19.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.354+0000 7f70e2685700 1 Processor -- start 2023-10-06T05:24:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.354+0000 7f70e2685700 1 -- start start 2023-10-06T05:24:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70e2685700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70e2685700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70dc156ce0 0x7f70dc068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70e2685700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc1570b0 0x7f70dc068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70e2685700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc167690 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70e2685700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc167970 con 0x7f70dc1570b0 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70e2685700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc167c50 con 0x7f70dc156ce0 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc0b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54124/0 (socket says 172.21.15.45:54124) 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 -- 172.21.15.45:0/1372005540 learned_addr learned my addr 172.21.15.45:0/1372005540 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70db7fe700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70dc156ce0 0x7f70dc068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70e0c22700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc1570b0 0x7f70dc068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 -- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70dc156ce0 msgr2=0x7f70dc068800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70dc156ce0 0x7f70dc068800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 -- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc1570b0 msgr2=0x7f70dc068d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc1570b0 0x7f70dc068d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70dbfff700 1 -- 172.21.15.45:0/1372005540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4009a20 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.355+0000 7f70db7fe700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70dc156ce0 0x7f70dc068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.356+0000 7f70dbfff700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc0b14a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c590 tx=0x7f70cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.356+0000 7f70d3fff700 1 -- 172.21.15.45:0/1372005540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc011590 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.356+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc167f30 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.356+0000 7f70d3fff700 1 -- 172.21.15.45:0/1372005540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc0116f0 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.356+0000 7f70d3fff700 1 -- 172.21.15.45:0/1372005540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc011ab0 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.356+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc0b1c20 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.357+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc06e2f0 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.357+0000 7f70d3fff700 1 -- 172.21.15.45:0/1372005540 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70cc01b3b0 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.358+0000 7f70d3fff700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70bc066f30 0x7f70bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.358+0000 7f70d3fff700 1 -- 172.21.15.45:0/1372005540 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f70cc084420 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.358+0000 7f70db7fe700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70bc066f30 0x7f70bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.358+0000 7f70db7fe700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70bc066f30 0x7f70bc0693e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f70c40058d0 tx=0x7f70c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.362+0000 7f70d3fff700 1 -- 172.21.15.45:0/1372005540 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70cc050de0 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.613+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70dc04bb40 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.614+0000 7f70d3fff700 1 -- 172.21.15.45:0/1372005540 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f70cc017070 con 0x7f70dc0cd3f0 2023-10-06T05:24:19.616 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:19.616 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":209,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:14.989809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9003":{"gid":9003,"name":"c","rank":0,"incarnation":206,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/3031536032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3031536032},{"type":"v1","addr":"172.21.15.45:6835","nonce":3031536032}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70bc066f30 msgr2=0x7f70bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70bc066f30 0x7f70bc0693e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f70c40058d0 tx=0x7f70c4009040 comp rx=0 tx=0).stop 2023-10-06T05:24:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc0cd3f0 msgr2=0x7f70dc0b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc0b14a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c590 tx=0x7f70cc00c950 comp rx=0 tx=0).stop 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 shutdown_connections 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f70bc066f30 0x7f70bc0693e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70dc1570b0 0x7f70dc068d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc0cd3f0 0x7f70dc0b14a0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 --2- 172.21.15.45:0/1372005540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70dc156ce0 0x7f70dc068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 >> 172.21.15.45:0/1372005540 conn(0x7f70dc06f090 msgr2=0x7f70dc067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 shutdown_connections 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.616+0000 7f70e2685700 1 -- 172.21.15.45:0/1372005540 wait complete. 2023-10-06T05:24:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 209 2023-10-06T05:24:19.632 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 209, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:24:14.989809+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9003}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9003': {'gid': 9003, 'name': 'c', 'rank': 0, 'incarnation': 206, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/3031536032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 3031536032}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 3031536032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:24:19.632 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:24:19.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:24:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.763+0000 7f95721ab700 1 Processor -- start 2023-10-06T05:24:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.763+0000 7f95721ab700 1 -- start start 2023-10-06T05:24:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f95721ab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f956c0c93c0 0x7f956c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f95721ab700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c157030 0x7f956c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f95721ab700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956c15b000 con 0x7f956c0c8ff0 2023-10-06T05:24:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f95721ab700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956c0c9830 con 0x7f956c0c93c0 2023-10-06T05:24:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f95721ab700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c15b2e0 con 0x7f956c157030 2023-10-06T05:24:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f956bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c157030 0x7f956c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f956affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f956c0c93c0 0x7f956c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f956affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f956c0c93c0 0x7f956c156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:46464/0 (socket says 172.21.15.45:46464) 2023-10-06T05:24:19.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f956affd700 1 -- 172.21.15.45:0/3812446253 learned_addr learned my addr 172.21.15.45:0/3812446253 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.764+0000 7f956bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c157030 0x7f956c0cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:39518/0 (socket says 172.21.15.45:39518) 2023-10-06T05:24:19.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956affd700 1 -- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c157030 msgr2=0x7f956c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956affd700 1 --2- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c157030 0x7f956c0cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956affd700 1 -- 172.21.15.45:0/3812446253 >> v1:172.21.15.144:6789/0 conn(0x7f956c0c8ff0 legacy=0x7f956c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956affd700 1 -- 172.21.15.45:0/3812446253 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c15b5c0 con 0x7f956c0c93c0 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956bfff700 1 --2- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c157030 0x7f956c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956affd700 1 --2- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c93c0 0x7f956c156af0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f95600099b0 tx=0x7f9560009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df4e010b773415c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956a7fc700 1 -- 172.21.15.45:0/3812446253 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560009ea0 con 0x7f956c0c93c0 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956a7fc700 1 -- 172.21.15.45:0/3812446253 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000b860 con 0x7f956c0c93c0 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.765+0000 7f956a7fc700 1 -- 172.21.15.45:0/3812446253 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000bc00 con 0x7f956c0c93c0 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 -- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c93c0 msgr2=0x7f956c156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 --2- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c93c0 0x7f956c156af0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f95600099b0 tx=0x7f9560009250 comp rx=0 tx=0).stop 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 -- 172.21.15.45:0/3812446253 shutdown_connections 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 --2- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c157030 0x7f956c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 --2- 172.21.15.45:0/3812446253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c93c0 0x7f956c156af0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 -- 172.21.15.45:0/3812446253 >> 172.21.15.45:0/3812446253 conn(0x7f956c0cdf50 msgr2=0x7f956c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 -- 172.21.15.45:0/3812446253 shutdown_connections 2023-10-06T05:24:19.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.766+0000 7f95721ab700 1 -- 172.21.15.45:0/3812446253 wait complete. 2023-10-06T05:24:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.767+0000 7f95721ab700 1 Processor -- start 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.767+0000 7f95721ab700 1 -- start start 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f95721ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c8ff0 0x7f956c069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f95721ab700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c0c93c0 0x7f956c069320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f95721ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c157030 0x7f956c0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f95721ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c06a6f0 con 0x7f956c157030 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f95721ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c06a410 con 0x7f956c0c8ff0 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f95721ab700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c068840 con 0x7f956c0c93c0 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f956bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c157030 0x7f956c0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f956bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c157030 0x7f956c0af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54140/0 (socket says 172.21.15.45:54140) 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f956bfff700 1 -- 172.21.15.45:0/2276493541 learned_addr learned my addr 172.21.15.45:0/2276493541 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.768+0000 7f956affd700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c0c93c0 0x7f956c069320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956b7fe700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c8ff0 0x7f956c069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956bfff700 1 -- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c0c93c0 msgr2=0x7f956c069320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956bfff700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c0c93c0 0x7f956c069320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956bfff700 1 -- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c8ff0 msgr2=0x7f956c069d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956bfff700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c8ff0 0x7f956c069d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956bfff700 1 -- 172.21.15.45:0/2276493541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9564009a20 con 0x7f956c157030 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956b7fe700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c8ff0 0x7f956c069d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956affd700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c0c93c0 0x7f956c069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f956bfff700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c157030 0x7f956c0af840 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f9564009fc0 tx=0x7f956400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f95537fe700 1 -- 172.21.15.45:0/2276493541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956400cd30 con 0x7f956c157030 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f95537fe700 1 -- 172.21.15.45:0/2276493541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956400f040 con 0x7f956c157030 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95600094f0 con 0x7f956c157030 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f95537fe700 1 -- 172.21.15.45:0/2276493541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640146d0 con 0x7f956c157030 2023-10-06T05:24:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.769+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956c068e00 con 0x7f956c157030 2023-10-06T05:24:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.771+0000 7f95537fe700 1 -- 172.21.15.45:0/2276493541 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9564003020 con 0x7f956c157030 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.771+0000 7f95537fe700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9558066f30 0x7f95580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.771+0000 7f95537fe700 1 -- 172.21.15.45:0/2276493541 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f9564085890 con 0x7f956c157030 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.771+0000 7f956b7fe700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9558066f30 0x7f95580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.772+0000 7f956b7fe700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9558066f30 0x7f95580693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f955c000c00 tx=0x7f955c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.773+0000 7f957b4bfb80 1 Processor -- start 2023-10-06T05:24:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.774+0000 7f957b4bfb80 1 -- start start 2023-10-06T05:24:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a674205820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a6741eaf40 0x55a67420df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a67420e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6740d8310 con 0x55a674205450 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a674115130 con 0x55a67420e4d0 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a674112af0 con 0x55a6741eaf40 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a674205820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a674205820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54148/0 (socket says 172.21.15.45:54148) 2023-10-06T05:24:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 -- 172.21.15.45:0/941557210 learned_addr learned my addr 172.21.15.45:0/941557210 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956affd700 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a6741eaf40 0x55a67420df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956bfff700 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a67420e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 -- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a6741eaf40 msgr2=0x55a67420df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a6741eaf40 0x55a67420df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 -- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 msgr2=0x55a67420e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a67420e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 -- 172.21.15.45:0/941557210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a673f14790 con 0x55a674205450 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.775+0000 7f956b7fe700 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a674205820 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e910 tx=0x7f955c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2edebd2f876ef50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f9550ff9700 1 -- 172.21.15.45:0/941557210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c015510 con 0x55a674205450 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f9550ff9700 1 -- 172.21.15.45:0/941557210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c015670 con 0x55a674205450 2023-10-06T05:24:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f9550ff9700 1 -- 172.21.15.45:0/941557210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c015a10 con 0x55a674205450 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 -- 172.21.15.45:0/941557210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 msgr2=0x55a674205820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a674205820 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e910 tx=0x7f955c013000 comp rx=0 tx=0).stop 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 -- 172.21.15.45:0/941557210 shutdown_connections 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a67420e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a674205820 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/941557210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a6741eaf40 0x55a67420df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 -- 172.21.15.45:0/941557210 >> 172.21.15.45:0/941557210 conn(0x55a674208880 msgr2=0x55a674208c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.776+0000 7f957b4bfb80 1 -- 172.21.15.45:0/941557210 shutdown_connections 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.777+0000 7f957b4bfb80 1 -- 172.21.15.45:0/941557210 wait complete. 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.777+0000 7f957b4bfb80 1 -- start start 2023-10-06T05:24:19.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.777+0000 7f957b4bfb80 1 Processor -- start 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a6741bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67410e8e0 con 0x55a674205450 2023-10-06T05:24:19.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6740ba980 con 0x55a6741eaf40 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6740ba620 con 0x55a67420e4d0 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f957b4bfb80 10 client.0 _ll_register_callbacks cb 0x55a673f1da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54162/0 (socket says 172.21.15.45:54162) 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 -- 172.21.15.45:0/583044474 learned_addr learned my addr 172.21.15.45:0/583044474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956b7fe700 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956bfff700 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a6741bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 -- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 msgr2=0x55a6741bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a6741bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 -- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 msgr2=0x55a6741df330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.778+0000 7f956affd700 1 -- 172.21.15.45:0/583044474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c00f040 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f956affd700 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741c07a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f956000c010 tx=0x7f9560017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14c8d850c8f4e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f9549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f9549ffb700 1 -- 172.21.15.45:0/583044474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560020070 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f9549ffb700 1 -- 172.21.15.45:0/583044474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f956001a700 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6741b2d60 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f9549ffb700 1 -- 172.21.15.45:0/583044474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) v2 ==== 24+0+0 (secure 0 0 0) 0x7f956001aad0 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 10 client.9102 fetch_fsmap learned FSMap version 209 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=2+}) v3 -- 0x55a6740010a0 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f9549ffb700 1 -- 172.21.15.45:0/583044474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560036080 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f9549ffb700 1 -- 172.21.15.45:0/583044474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 209) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f956001ac30 con 0x55a674205450 2023-10-06T05:24:19.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 10 client.9102 fetch_fsmap finished waiting for FSMap version 209 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 10 client.9102 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 20 client.9102 populate_metadata read hostname 'smithi045' 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e2610 0x55a6741e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 4 client.9102 mds_command: new command op to 9003 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.779+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 --> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a67418fb30 con 0x55a6741e2610 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.780+0000 7f956b7fe700 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e2610 0x55a6741e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.780+0000 7f9549ffb700 10 client.9102 ms_handle_connect on v2:172.21.15.45:6834/3031536032 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.780+0000 7f956b7fe700 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e2610 0x55a6741e29e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c574e994e755dd74 server_cookie=7a342c7a85d95081 in_seq=0 out_seq=0 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f9549ffb700 1 -- 172.21.15.45:0/583044474 <== mds.0 v2:172.21.15.45:6834/3031536032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a67418fb30 con 0x55a6741e2610 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f9549ffb700 10 client.9102 handle_command_reply: tid=0 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 client.9102 shutdown 2023-10-06T05:24:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 msgr2=0x55a6741c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741c07a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f956000c010 tx=0x7f9560017000 comp rx=0 tx=0).stop 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 shutdown_connections 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e2610 0x55a6741e29e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a6741bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f9549ffb700 0 client.9102 ms_handle_reset on v2:172.21.15.45:6834/3031536032 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/583044474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741c07a0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 >> 172.21.15.45:0/583044474 conn(0x55a674208880 msgr2=0x55a6741bac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 shutdown_connections 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 1 -- 172.21.15.45:0/583044474 wait complete. 2023-10-06T05:24:19.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.781+0000 7f957b4bfb80 20 client.9102 trim_cache size 0 max 16384 2023-10-06T05:24:19.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.799+0000 7f957b4bfb80 1 Processor -- start 2023-10-06T05:24:19.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.799+0000 7f957b4bfb80 1 -- start start 2023-10-06T05:24:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.799+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.799+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.799+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a67421db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.799+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a673fa61e0 con 0x55a674205450 2023-10-06T05:24:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6741e1390 con 0x55a6741eaf40 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a674208200 con 0x55a67420e4d0 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741e6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54164/0 (socket says 172.21.15.45:54164) 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 -- 172.21.15.45:0/781657659 learned_addr learned my addr 172.21.15.45:0/781657659 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956b7fe700 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956bfff700 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a67421db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 -- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 msgr2=0x55a67421db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a67421db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 -- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 msgr2=0x55a6741df330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 -- 172.21.15.45:0/781657659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6740dd3c0 con 0x55a674205450 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f956affd700 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741e6a40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9560009ea0 tx=0x7f9560023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e530c11033248a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f95517fa700 1 -- 172.21.15.45:0/781657659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560020040 con 0x55a674205450 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f95517fa700 1 -- 172.21.15.45:0/781657659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560036020 con 0x55a674205450 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.800+0000 7f95517fa700 1 -- 172.21.15.45:0/781657659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956001a700 con 0x55a674205450 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 -- 172.21.15.45:0/781657659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 msgr2=0x55a6741e6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741e6a40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9560009ea0 tx=0x7f9560023000 comp rx=0 tx=0).stop 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 -- 172.21.15.45:0/781657659 shutdown_connections 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67420e4d0 0x55a67421db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6741eaf40 0x55a6741df330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/781657659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a674205450 0x55a6741e6a40 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 -- 172.21.15.45:0/781657659 >> 172.21.15.45:0/781657659 conn(0x55a674208880 msgr2=0x55a6741bbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 -- 172.21.15.45:0/781657659 shutdown_connections 2023-10-06T05:24:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.801+0000 7f957b4bfb80 1 -- 172.21.15.45:0/781657659 wait complete. 2023-10-06T05:24:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 -- start start 2023-10-06T05:24:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 Processor -- start 2023-10-06T05:24:19.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6741eaf40 0x55a6741b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a674205450 0x55a6741bac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a6741bb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6742179e0 con 0x55a6741eaf40 2023-10-06T05:24:19.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67426b380 con 0x55a67420e4d0 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.802+0000 7f957b4bfb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a67426acd0 con 0x55a674205450 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f957b4bfb80 10 client.0 _ll_register_callbacks cb 0x55a673f1da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6741eaf40 0x55a6741b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6741eaf40 0x55a6741b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54172/0 (socket says 172.21.15.45:54172) 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 -- 172.21.15.45:0/3708819795 learned_addr learned my addr 172.21.15.45:0/3708819795 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956affd700 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a674205450 0x55a6741bac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956bfff700 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a6741bb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 -- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a674205450 msgr2=0x55a6741bac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a674205450 0x55a6741bac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 -- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 msgr2=0x55a6741bb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a6741bb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 -- 172.21.15.45:0/3708819795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6740010a0 con 0x55a6741eaf40 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f956b7fe700 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6741eaf40 0x55a6741b4870 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e5d0 tx=0x7f955c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88874078ad52153 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f9549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f9549ffb700 1 -- 172.21.15.45:0/3708819795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0157e0 con 0x55a6741eaf40 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f9549ffb700 1 -- 172.21.15.45:0/3708819795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f955c015940 con 0x55a6741eaf40 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f955c015670 con 0x55a6741eaf40 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f9549ffb700 1 -- 172.21.15.45:0/3708819795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955c02f6b0 con 0x55a6741eaf40 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f957b4bfb80 10 client.9108 fetch_fsmap learned FSMap version 209 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.803+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=2+}) v3 -- 0x55a673f751d0 con 0x55a6741eaf40 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f9549ffb700 1 -- 172.21.15.45:0/3708819795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c015670 con 0x55a6741eaf40 2023-10-06T05:24:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f9549ffb700 1 -- 172.21.15.45:0/3708819795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 209) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f955c02f810 con 0x55a6741eaf40 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f957b4bfb80 10 client.9108 fetch_fsmap finished waiting for FSMap version 209 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f957b4bfb80 10 client.9108 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f957b4bfb80 20 client.9108 populate_metadata read hostname 'smithi045' 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e3070 0x55a6741e3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f957b4bfb80 4 client.9108 mds_command: new command op to 9003 tid=0[{"prefix": "status"}] 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 --> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a6740b18d0 con 0x55a6741e3070 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f956affd700 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e3070 0x55a6741e3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f9549ffb700 10 client.9108 ms_handle_connect on v2:172.21.15.45:6834/3031536032 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f956affd700 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e3070 0x55a6741e3440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790cae5c7ae6040 server_cookie=edcf2ee942e8906b in_seq=0 out_seq=0 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f9549ffb700 1 -- 172.21.15.45:0/3708819795 <== mds.0 v2:172.21.15.45:6834/3031536032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55a6740b18d0 con 0x55a6741e3070 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f9549ffb700 10 client.9108 handle_command_reply: tid=0 2023-10-06T05:24:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.804+0000 7f957b4bfb80 1 client.9108 shutdown 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6741eaf40 msgr2=0x55a6741b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6741eaf40 0x55a6741b4870 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f955c00e5d0 tx=0x7f955c01d000 comp rx=0 tx=0).stop 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 shutdown_connections 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:6834/3031536032,v1:172.21.15.45:6835/3031536032] conn(0x55a6741e3070 0x55a6741e3440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67420e4d0 0x55a6741bb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6741eaf40 0x55a6741b4870 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f9549ffb700 0 client.9108 ms_handle_reset on v2:172.21.15.45:6834/3031536032 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 --2- 172.21.15.45:0/3708819795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a674205450 0x55a6741bac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 >> 172.21.15.45:0/3708819795 conn(0x55a674208880 msgr2=0x55a6741c05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 shutdown_connections 2023-10-06T05:24:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 1 -- 172.21.15.45:0/3708819795 wait complete. 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f957b4bfb80 20 client.9108 trim_cache size 0 max 16384 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9003, 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:24:19.814 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:24:19.815 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 7.8265530190000003, 2023-10-06T05:24:19.815 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 209, 2023-10-06T05:24:19.815 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 110, 2023-10-06T05:24:19.815 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 110, 2023-10-06T05:24:19.815 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.830646475 2023-10-06T05:24:19.815 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9558066f30 msgr2=0x7f95580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9558066f30 0x7f95580693e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f955c000c00 tx=0x7f955c009040 comp rx=0 tx=0).stop 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c157030 msgr2=0x7f956c0af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c157030 0x7f956c0af840 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f9564009fc0 tx=0x7f956400b040 comp rx=0 tx=0).stop 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 shutdown_connections 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9558066f30 0x7f95580693e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c157030 0x7f956c0af840 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f956c0c8ff0 0x7f956c069d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 --2- 172.21.15.45:0/2276493541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f956c0c93c0 0x7f956c069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 >> 172.21.15.45:0/2276493541 conn(0x7f956c0cdf50 msgr2=0x7f956c14abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.805+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 shutdown_connections 2023-10-06T05:24:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.806+0000 7f95721ab700 1 -- 172.21.15.45:0/2276493541 wait complete. 2023-10-06T05:24:19.820 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:24:19.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:19.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.950+0000 7f7005cab700 1 Processor -- start 2023-10-06T05:24:19.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.950+0000 7f7005cab700 1 -- start start 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f7005cab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f700006afa0 0x7f7000068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f7005cab700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70000676c0 con 0x7f7000153a20 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f7005cab700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70000679a0 con 0x7f7000153650 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f7005cab700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7000067c80 con 0x7f700006afa0 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f6ffeffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7000153a20 0x7f700006a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59510/0 (socket says 172.21.15.45:59510) 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f6ffeffd700 1 -- 172.21.15.45:0/2566518541 learned_addr learned my addr 172.21.15.45:0/2566518541 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f6ffffff700 1 --2- 172.21.15.45:0/2566518541 >> v2:172.21.15.144:3300/0 conn(0x7f700006afa0 0x7f7000068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410291601 0 0) 0x7f70000676c0 con 0x7f7000153a20 2023-10-06T05:24:19.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.951+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec003570 con 0x7f7000153a20 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572310896 0 0) 0x7f6fec003570 con 0x7f7000153a20 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 >> v2:172.21.15.144:3300/0 conn(0x7f700006afa0 msgr2=0x7f7000068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 --2- 172.21.15.45:0/2566518541 >> v2:172.21.15.144:3300/0 conn(0x7f700006afa0 0x7f7000068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 >> v1:172.21.15.144:6789/0 conn(0x7f7000153650 legacy=0x7f700015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000067f60 con 0x7f7000153a20 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f6fe8001470 con 0x7f7000153a20 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 >> v1:172.21.15.45:6789/0 conn(0x7f7000153a20 legacy=0x7f700006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec003a60 0x7f6fec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fec004810 0x7f6fec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec0051c0 0x7f6fec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000067c80 con 0x7f6fec003a60 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec00c160 con 0x7f6fec004810 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec00c320 con 0x7f6fec0051c0 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6fff7fe700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec003a60 0x7f6fec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffeffd700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fec004810 0x7f6fec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffffff700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec0051c0 0x7f6fec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6fff7fe700 1 -- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec0051c0 msgr2=0x7f6fec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6fff7fe700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec0051c0 0x7f6fec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6fff7fe700 1 -- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fec004810 msgr2=0x7f6fec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6fff7fe700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fec004810 0x7f6fec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6fff7fe700 1 -- 172.21.15.45:0/2566518541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec003570 con 0x7f6fec003a60 2023-10-06T05:24:19.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffffff700 1 -- 172.21.15.45:0/2566518541 reap_dead start 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6ffeffd700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fec004810 0x7f6fec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.952+0000 7f6fff7fe700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec003a60 0x7f6fec003e70 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00018b0 tx=0x7f6ff0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e47751e91d401 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0012420 con 0x7f6fec003a60 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff00125a0 con 0x7f6fec003a60 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f6ffe7fc700 1 -- 172.21.15.45:0/2566518541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0012960 con 0x7f6fec003a60 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f7005cab700 1 -- 172.21.15.45:0/2566518541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec003a60 msgr2=0x7f6fec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f7005cab700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec003a60 0x7f6fec003e70 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00018b0 tx=0x7f6ff0007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f7005cab700 1 -- 172.21.15.45:0/2566518541 shutdown_connections 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f7005cab700 1 --2- 172.21.15.45:0/2566518541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec003a60 0x7f6fec003e70 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f7005cab700 1 -- 172.21.15.45:0/2566518541 >> 172.21.15.45:0/2566518541 conn(0x7f70000cc5a0 msgr2=0x7f70000cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f7005cab700 1 -- 172.21.15.45:0/2566518541 shutdown_connections 2023-10-06T05:24:19.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.953+0000 7f7005cab700 1 -- 172.21.15.45:0/2566518541 wait complete. 2023-10-06T05:24:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 Processor -- start 2023-10-06T05:24:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 -- start start 2023-10-06T05:24:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec003a60 0x7f7000152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70000af350 0x7f70000ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70000ad720 0x7f70000c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70000adbc0 con 0x7f70000ad720 2023-10-06T05:24:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000153050 con 0x7f70000af350 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.955+0000 7f7005cab700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70000c4b30 con 0x7f6fec003a60 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70000ad720 0x7f70000c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70000ad720 0x7f70000c3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54182/0 (socket says 172.21.15.45:54182) 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 -- 172.21.15.45:0/1723558228 learned_addr learned my addr 172.21.15.45:0/1723558228 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6fff7fe700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec003a60 0x7f7000152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffeffd700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70000af350 0x7f70000ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 -- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec003a60 msgr2=0x7f7000152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec003a60 0x7f7000152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 -- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70000af350 msgr2=0x7f70000ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70000af350 0x7f70000ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 -- 172.21.15.45:0/1723558228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000c1480 con 0x7f70000ad720 2023-10-06T05:24:19.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ffffff700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70000ad720 0x7f70000c3db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f700006b3a0 tx=0x7f6ff800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ff77fe700 1 -- 172.21.15.45:0/1723558228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800e040 con 0x7f70000ad720 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ff77fe700 1 -- 172.21.15.45:0/1723558228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff800f040 con 0x7f70000ad720 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70000c1760 con 0x7f70000ad720 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f6ff77fe700 1 -- 172.21.15.45:0/1723558228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800fe50 con 0x7f70000ad720 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.956+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70000c4e10 con 0x7f70000ad720 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.957+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70000c1c80 con 0x7f70000ad720 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.958+0000 7f6ff77fe700 1 -- 172.21.15.45:0/1723558228 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ff8015640 con 0x7f70000ad720 2023-10-06T05:24:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.958+0000 7f6ff77fe700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fe0066f30 0x7f6fe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.965 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:19.958+0000 7f6f95f92700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:19.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.959+0000 7f6ff77fe700 1 -- 172.21.15.45:0/1723558228 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f6ff8085610 con 0x7f70000ad720 2023-10-06T05:24:19.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.959+0000 7f6fff7fe700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fe0066f30 0x7f6fe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.959+0000 7f6fff7fe700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fe0066f30 0x7f6fe00693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0001010 tx=0x7f6ff0015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:19.963+0000 7f6ff77fe700 1 -- 172.21.15.45:0/1723558228 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70000c1c80 con 0x7f70000ad720 2023-10-06T05:24:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.212+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70000c1c80 con 0x7f70000ad720 2023-10-06T05:24:20.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.213+0000 7f6ff77fe700 1 -- 172.21.15.45:0/1723558228 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f70000c1c80 con 0x7f70000ad720 2023-10-06T05:24:20.215 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:20.215 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":209,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:14.989809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9003":{"gid":9003,"name":"c","rank":0,"incarnation":206,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/3031536032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3031536032},{"type":"v1","addr":"172.21.15.45:6835","nonce":3031536032}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:20.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fe0066f30 msgr2=0x7f6fe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fe0066f30 0x7f6fe00693e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0001010 tx=0x7f6ff0015000 comp rx=0 tx=0).stop 2023-10-06T05:24:20.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70000ad720 msgr2=0x7f70000c3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70000ad720 0x7f70000c3db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f700006b3a0 tx=0x7f6ff800b040 comp rx=0 tx=0).stop 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 shutdown_connections 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6fe0066f30 0x7f6fe00693e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70000ad720 0x7f70000c3db0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fec003a60 0x7f7000152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 --2- 172.21.15.45:0/1723558228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70000af350 0x7f70000ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 >> 172.21.15.45:0/1723558228 conn(0x7f70000cc5a0 msgr2=0x7f700015bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 shutdown_connections 2023-10-06T05:24:20.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.215+0000 7f7005cab700 1 -- 172.21.15.45:0/1723558228 wait complete. 2023-10-06T05:24:20.219 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 209 2023-10-06T05:24:20.230 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2023-10-06T05:24:20.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.361+0000 7f9d80264700 1 Processor -- start 2023-10-06T05:24:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.361+0000 7f9d80264700 1 -- start start 2023-10-06T05:24:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d80264700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9d7815d4c0 0x7f9d7806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d80264700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780c76c0 con 0x7f9d780c72c0 2023-10-06T05:24:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d80264700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780ac8d0 con 0x7f9d7815d890 2023-10-06T05:24:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d80264700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d780c81a0 con 0x7f9d7815d4c0 2023-10-06T05:24:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d7e801700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9d780c72c0 0x7f9d780bf130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59522/0 (socket says 172.21.15.45:59522) 2023-10-06T05:24:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d7e801700 1 -- 172.21.15.45:0/4202395084 learned_addr learned my addr 172.21.15.45:0/4202395084 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:20.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d7e000700 1 --2- 172.21.15.45:0/4202395084 >> v2:172.21.15.144:3300/0 conn(0x7f9d7815d4c0 0x7f9d7806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d7cffe700 1 -- 172.21.15.45:0/4202395084 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773819154 0 0) 0x7f9d70001490 con 0x7f9d780c72c0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.362+0000 7f9d7cffe700 1 -- 172.21.15.45:0/4202395084 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d64003570 con 0x7f9d780c72c0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.363+0000 7f9d7e000700 1 -- 172.21.15.45:0/4202395084 >> v1:172.21.15.144:6789/0 conn(0x7f9d7815d890 legacy=0x7f9d780c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.363+0000 7f9d7e000700 1 -- 172.21.15.45:0/4202395084 >> v1:172.21.15.45:6789/0 conn(0x7f9d780c72c0 legacy=0x7f9d780bf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.363+0000 7f9d7e000700 1 -- 172.21.15.45:0/4202395084 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d780ac5a0 con 0x7f9d7815d4c0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.363+0000 7f9d7e000700 1 --2- 172.21.15.45:0/4202395084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7806a890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9d740098e0 tx=0x7f9d740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ca979e24266618 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.363+0000 7f9d7cffe700 1 -- 172.21.15.45:0/4202395084 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400b640 con 0x7f9d7815d4c0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.363+0000 7f9d7cffe700 1 -- 172.21.15.45:0/4202395084 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7400b7a0 con 0x7f9d7815d4c0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.363+0000 7f9d7cffe700 1 -- 172.21.15.45:0/4202395084 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400bb40 con 0x7f9d7815d4c0 2023-10-06T05:24:20.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.364+0000 7f9d80264700 1 -- 172.21.15.45:0/4202395084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 msgr2=0x7f9d7806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.364+0000 7f9d80264700 1 --2- 172.21.15.45:0/4202395084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7806a890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9d740098e0 tx=0x7f9d740091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.364+0000 7f9d80264700 1 -- 172.21.15.45:0/4202395084 shutdown_connections 2023-10-06T05:24:20.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.364+0000 7f9d80264700 1 --2- 172.21.15.45:0/4202395084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7806a890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.364+0000 7f9d80264700 1 -- 172.21.15.45:0/4202395084 >> 172.21.15.45:0/4202395084 conn(0x7f9d780cc5a0 msgr2=0x7f9d780cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.364+0000 7f9d80264700 1 -- 172.21.15.45:0/4202395084 shutdown_connections 2023-10-06T05:24:20.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.364+0000 7f9d80264700 1 -- 172.21.15.45:0/4202395084 wait complete. 2023-10-06T05:24:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.365+0000 7f9d80264700 1 Processor -- start 2023-10-06T05:24:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.365+0000 7f9d80264700 1 -- start start 2023-10-06T05:24:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d80264700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d780c72c0 0x7f9d780c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d80264700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d80264700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7815d890 0x7f9d780c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d80264700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780a50e0 con 0x7f9d7815d890 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d80264700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780c5640 con 0x7f9d7815d4c0 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d80264700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780c5920 con 0x7f9d780c72c0 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7815d890 0x7f9d780c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7815d890 0x7f9d780c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54196/0 (socket says 172.21.15.45:54196) 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 -- 172.21.15.45:0/3037019554 learned_addr learned my addr 172.21.15.45:0/3037019554 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e000700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d780c72c0 0x7f9d780c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7d7ff700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 -- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d780c72c0 msgr2=0x7f9d780c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d780c72c0 0x7f9d780c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 -- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 msgr2=0x7f9d7815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7e801700 1 -- 172.21.15.45:0/3037019554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74009420 con 0x7f9d7815d890 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.366+0000 7f9d7d7ff700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.367+0000 7f9d7e801700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7815d890 0x7f9d780c4ad0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000cd00 tx=0x7f9d70001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.367+0000 7f9d6dffb700 1 -- 172.21.15.45:0/3037019554 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7000ab70 con 0x7f9d7815d890 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.367+0000 7f9d6dffb700 1 -- 172.21.15.45:0/3037019554 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7000acd0 con 0x7f9d7815d890 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.367+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d780c5c00 con 0x7f9d7815d890 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.367+0000 7f9d6dffb700 1 -- 172.21.15.45:0/3037019554 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70007300 con 0x7f9d7815d890 2023-10-06T05:24:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.367+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d780c61c0 con 0x7f9d7815d890 2023-10-06T05:24:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.368+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d780c6c20 con 0x7f9d7815d890 2023-10-06T05:24:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.369+0000 7f9d6dffb700 1 -- 172.21.15.45:0/3037019554 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d700076f0 con 0x7f9d7815d890 2023-10-06T05:24:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.369+0000 7f9d6dffb700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d5c066f30 0x7f9d5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.369+0000 7f9d6dffb700 1 -- 172.21.15.45:0/3037019554 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f9d70085190 con 0x7f9d7815d890 2023-10-06T05:24:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.369+0000 7f9d7e000700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d5c066f30 0x7f9d5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.370+0000 7f9d7e000700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d5c066f30 0x7f9d5c0693e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d740093f0 tx=0x7f9d74017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.374+0000 7f9d6dffb700 1 -- 172.21.15.45:0/3037019554 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d70051b40 con 0x7f9d7815d890 2023-10-06T05:24:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.619+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9d780c76c0 con 0x7f9d7815d890 2023-10-06T05:24:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.620+0000 7f9d6dffb700 1 -- 172.21.15.45:0/3037019554 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v209) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9d780c76c0 con 0x7f9d7815d890 2023-10-06T05:24:20.622 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d5c066f30 msgr2=0x7f9d5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d5c066f30 0x7f9d5c0693e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d740093f0 tx=0x7f9d74017000 comp rx=0 tx=0).stop 2023-10-06T05:24:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7815d890 msgr2=0x7f9d780c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7815d890 0x7f9d780c4ad0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000cd00 tx=0x7f9d70001ad0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 shutdown_connections 2023-10-06T05:24:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d5c066f30 0x7f9d5c0693e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7815d890 0x7f9d780c4ad0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d780c72c0 0x7f9d780c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 --2- 172.21.15.45:0/3037019554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d7815d4c0 0x7f9d7815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 >> 172.21.15.45:0/3037019554 conn(0x7f9d780cc5a0 msgr2=0x7f9d78160470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 shutdown_connections 2023-10-06T05:24:20.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:20.622+0000 7f9d80264700 1 -- 172.21.15.45:0/3037019554 wait complete. 2023-10-06T05:24:20.637 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:24:20.637 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:20.637 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:24:20.637 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:24:20.637 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:20.637 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:24:20.637 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:24:20.638 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:20.638 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:20.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:20.638 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:24:20.659 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:20.659 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:24:20.660 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41881sec preferred_lft 41881sec 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:20.661 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:24:20.662 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:20.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:20.663 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:20.730 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:24:20.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:20.732 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:24:20.797 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:24:20.797 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:20.797 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:24:20.797 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:24:20.797 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:20.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:20.798 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:24:20.798 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:24:20.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:24:20.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:24:20.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:24:20.798 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:24:20.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:20.921 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:24:20.943 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:24:20.943 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:24:20.943 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:20.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:20.943 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:24:21.018 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:24:21.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:21.019 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:21.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:21.082 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:21.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:24:21.222 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:24:21.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:24:21.297 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:24:21.298 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:24:21.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:21.299 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:24:21.363 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:24:21.363 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:24:21.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:24:21.441 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:24:21.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:24:21.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:24:21.505+0000 7fe1fe3f6540 -1 init, newargv = 0x55e12a0379f0 newargc=15 2023-10-06T05:24:21.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[168397]: starting ceph client 2023-10-06T05:24:21.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[168397]: starting fuse 2023-10-06T05:24:21.514 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:24:21.516 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:24:21.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:21.517 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:24:21.577 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:24:21.578 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:24:21.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:24:21.578 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:24:21.578 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:24:21.578 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:24:21.578 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:24:21.578 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:24:21.579 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:24:21.580 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:24:21.580 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:24:21.580 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:24:21.580 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:24:21.580 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:24:21.580 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:24:21.580 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:24:21.717 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.168397.asok 2023-10-06T05:24:21.898 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.168397.asok 2023-10-06T05:24:21.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:21.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.168397.asok status 2023-10-06T05:24:22.075 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:24:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:24:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:24:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:24:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:24:22.075 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "168397", 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9123, 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 9123 2023-10-06T05:24:22.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 911731667 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 911731667 2023-10-06T05:24:22.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.9123 192.168.0.1:0/911731667", 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/911731667", 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 209, 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 110, 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:24:22.078 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:24:22.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:22.086 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:24:22.086 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:22.107 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:24:22.107 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:22.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:22.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:22.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:22.180 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:24:22.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:22.223 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2023-10-06T05:24:22.248 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:24:22.248 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:24:22.248 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:22.248 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:24:22.248 INFO:teuthology.orchestra.run.smithi045.stdout:sixmegs 2023-10-06T05:24:22.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:mkdir: cannot create directory ‘foo’: Read-only file system 2023-10-06T05:24:22.270 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory ‘foo’: Read-only file system 2023-10-06T05:24:22.270 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2023-10-06T05:24:22.271 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:24:22.271 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:22.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:22.354 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:24:22.354 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:22.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:22.355 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:22.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:22.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.600+0000 7f48fc186700 1 Processor -- start 2023-10-06T05:24:22.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.600+0000 7f48fc186700 1 -- start start 2023-10-06T05:24:22.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48fc186700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f48f40c8ff0 0x7f48f40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48fc186700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48f40caad0 0x7f48f40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48fc186700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f415b000 con 0x7f48f4156af0 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48fc186700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f4157630 con 0x7f48f40caad0 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48fc186700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f415b2e0 con 0x7f48f40c8ff0 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9721700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48f40caad0 0x7f48f40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9721700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48f40caad0 0x7f48f40caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54208/0 (socket says 172.21.15.45:54208) 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9721700 1 -- 172.21.15.45:0/169143542 learned_addr learned my addr 172.21.15.45:0/169143542 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9f22700 1 --2- 172.21.15.45:0/169143542 >> v2:172.21.15.144:3300/0 conn(0x7f48f40c8ff0 0x7f48f40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9721700 1 -- 172.21.15.45:0/169143542 >> v2:172.21.15.144:3300/0 conn(0x7f48f40c8ff0 msgr2=0x7f48f40c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9721700 1 --2- 172.21.15.45:0/169143542 >> v2:172.21.15.144:3300/0 conn(0x7f48f40c8ff0 0x7f48f40c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9721700 1 -- 172.21.15.45:0/169143542 >> v1:172.21.15.144:6789/0 conn(0x7f48f4156af0 legacy=0x7f48f4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.601+0000 7f48f9721700 1 -- 172.21.15.45:0/169143542 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f415b5c0 con 0x7f48f40caad0 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48f9721700 1 --2- 172.21.15.45:0/169143542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f40caad0 0x7f48f40caf40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f48f40cefc0 tx=0x7f48f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3562f9611448d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48f8f20700 1 -- 172.21.15.45:0/169143542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0009e70 con 0x7f48f40caad0 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48f8f20700 1 -- 172.21.15.45:0/169143542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f000b860 con 0x7f48f40caad0 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48f8f20700 1 -- 172.21.15.45:0/169143542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000bc00 con 0x7f48f40caad0 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48fc186700 1 -- 172.21.15.45:0/169143542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f40caad0 msgr2=0x7f48f40caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48fc186700 1 --2- 172.21.15.45:0/169143542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f40caad0 0x7f48f40caf40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f48f40cefc0 tx=0x7f48f0009250 comp rx=0 tx=0).stop 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48fc186700 1 -- 172.21.15.45:0/169143542 shutdown_connections 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48fc186700 1 --2- 172.21.15.45:0/169143542 >> v2:172.21.15.144:3300/0 conn(0x7f48f40c8ff0 0x7f48f40c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48fc186700 1 --2- 172.21.15.45:0/169143542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f40caad0 0x7f48f40caf40 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48fc186700 1 -- 172.21.15.45:0/169143542 >> 172.21.15.45:0/169143542 conn(0x7f48f40cdf50 msgr2=0x7f48f40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.602+0000 7f48fc186700 1 -- 172.21.15.45:0/169143542 shutdown_connections 2023-10-06T05:24:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.603+0000 7f48fc186700 1 -- 172.21.15.45:0/169143542 wait complete. 2023-10-06T05:24:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 Processor -- start 2023-10-06T05:24:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 -- start start 2023-10-06T05:24:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48f40c8ff0 0x7f48f4149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48f40caad0 0x7f48f40c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f4156af0 0x7f48f40c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c69f0 con 0x7f48f4156af0 2023-10-06T05:24:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c5950 con 0x7f48f40c8ff0 2023-10-06T05:24:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.604+0000 7f48fc186700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c5c30 con 0x7f48f40caad0 2023-10-06T05:24:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f4156af0 0x7f48f40c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f4156af0 0x7f48f40c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54218/0 (socket says 172.21.15.45:54218) 2023-10-06T05:24:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 -- 172.21.15.45:0/3853689078 learned_addr learned my addr 172.21.15.45:0/3853689078 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48f9f22700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48f40c8ff0 0x7f48f4149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48f9721700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48f40caad0 0x7f48f40c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 -- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48f40caad0 msgr2=0x7f48f40c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48f40caad0 0x7f48f40c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 -- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48f40c8ff0 msgr2=0x7f48f4149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48f40c8ff0 0x7f48f4149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 -- 172.21.15.45:0/3853689078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f00094f0 con 0x7f48f4156af0 2023-10-06T05:24:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fa723700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f4156af0 0x7f48f40c7220 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c8e0 tx=0x7f48ec001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48e5ffb700 1 -- 172.21.15.45:0/3853689078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00ce80 con 0x7f48f4156af0 2023-10-06T05:24:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48e5ffb700 1 -- 172.21.15.45:0/3853689078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec00a7e0 con 0x7f48f4156af0 2023-10-06T05:24:22.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f40c5f10 con 0x7f48f4156af0 2023-10-06T05:24:22.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.605+0000 7f48e5ffb700 1 -- 172.21.15.45:0/3853689078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00ab80 con 0x7f48f4156af0 2023-10-06T05:24:22.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.606+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f40c64d0 con 0x7f48f4156af0 2023-10-06T05:24:22.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.606+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f40c66f0 con 0x7f48f4156af0 2023-10-06T05:24:22.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.607+0000 7f48e5ffb700 1 -- 172.21.15.45:0/3853689078 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48ec012420 con 0x7f48f4156af0 2023-10-06T05:24:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.607+0000 7f48e5ffb700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.608+0000 7f48e5ffb700 1 -- 172.21.15.45:0/3853689078 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f48ec085130 con 0x7f48f4156af0 2023-10-06T05:24:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.608+0000 7f48f9f22700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.608+0000 7f48f9f22700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48d8066f30 0x7f48d80693e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ca0 tx=0x7f48e8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.612+0000 7f48e5ffb700 1 -- 172.21.15.45:0/3853689078 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48ec051a50 con 0x7f48f4156af0 2023-10-06T05:24:22.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.862+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f48f414a430 con 0x7f48f4156af0 2023-10-06T05:24:22.865 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:22.863+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:22.869 INFO:tasks.ceph.mds.c.smithi045.stderr: -1608> 2023-10-06T05:24:11.982+0000 7f6f9378d700 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2023-10-06T05:24:22.869 INFO:tasks.ceph.mds.c.smithi045.stderr: -1605> 2023-10-06T05:24:11.982+0000 7f6f9378d700 -1 mds.0.206 unhandled write error (22) Invalid argument, force readonly... 2023-10-06T05:24:22.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.867+0000 7f48e5ffb700 1 -- 172.21.15.45:0/3853689078 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v210) v1 ==== 161+0+0 (secure 0 0 0) 0x7f48ec055070 con 0x7f48f4156af0 2023-10-06T05:24:22.871 INFO:tasks.ceph.mds.c.smithi045.stderr: -540> 2023-10-06T05:24:19.958+0000 7f6f95f92700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:22.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48d8066f30 msgr2=0x7f48d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48d8066f30 0x7f48d80693e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ca0 tx=0x7f48e8008040 comp rx=0 tx=0).stop 2023-10-06T05:24:22.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f4156af0 msgr2=0x7f48f40c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f4156af0 0x7f48f40c7220 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c8e0 tx=0x7f48ec001710 comp rx=0 tx=0).stop 2023-10-06T05:24:22.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 shutdown_connections 2023-10-06T05:24:22.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48f4156af0 0x7f48f40c7220 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48f40c8ff0 0x7f48f4149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 --2- 172.21.15.45:0/3853689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48f40caad0 0x7f48f40c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 >> 172.21.15.45:0/3853689078 conn(0x7f48f40cdf50 msgr2=0x7f48f406c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 shutdown_connections 2023-10-06T05:24:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:22.871+0000 7f48fc186700 1 -- 172.21.15.45:0/3853689078 wait complete. 2023-10-06T05:24:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:22.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:22.895 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:24:23.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.019+0000 7f0fd4c83700 1 Processor -- start 2023-10-06T05:24:23.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.019+0000 7f0fd4c83700 1 -- start start 2023-10-06T05:24:23.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fd4c83700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fd4c83700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0fd0157200 0x7f0fd00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fd4c83700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd015b000 con 0x7f0fd00caad0 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fd4c83700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd015b2e0 con 0x7f0fd0157200 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fd4c83700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd015b5c0 con 0x7f0fd00c8ff0 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fced9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0fd0157200 0x7f0fd00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fced9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0fd0157200 0x7f0fd00caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54244/0 (socket says 172.21.15.45:54244) 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fced9d700 1 -- 172.21.15.45:0/2786835977 learned_addr learned my addr 172.21.15.45:0/2786835977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.020+0000 7f0fce59c700 1 --2- 172.21.15.45:0/2786835977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fced9d700 1 -- 172.21.15.45:0/2786835977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 msgr2=0x7f0fd00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fced9d700 1 --2- 172.21.15.45:0/2786835977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fced9d700 1 -- 172.21.15.45:0/2786835977 >> v1:172.21.15.144:6789/0 conn(0x7f0fd00caad0 legacy=0x7f0fd0156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fced9d700 1 -- 172.21.15.45:0/2786835977 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd015b8a0 con 0x7f0fd0157200 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fce59c700 1 --2- 172.21.15.45:0/2786835977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fced9d700 1 --2- 172.21.15.45:0/2786835977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd0157200 0x7f0fd00caf40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009f20 tx=0x7f0fc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e903bc694481f280 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fcdd9b700 1 -- 172.21.15.45:0/2786835977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800e070 con 0x7f0fd0157200 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fcdd9b700 1 -- 172.21.15.45:0/2786835977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800ba80 con 0x7f0fd0157200 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fcdd9b700 1 -- 172.21.15.45:0/2786835977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80092e0 con 0x7f0fd0157200 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2786835977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd0157200 msgr2=0x7f0fd00caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.021+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2786835977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd0157200 0x7f0fd00caf40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009f20 tx=0x7f0fc8009840 comp rx=0 tx=0).stop 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.022+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2786835977 shutdown_connections 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.022+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2786835977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd0157200 0x7f0fd00caf40 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.022+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2786835977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.022+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2786835977 >> 172.21.15.45:0/2786835977 conn(0x7f0fd00cdf50 msgr2=0x7f0fd00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.022+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2786835977 shutdown_connections 2023-10-06T05:24:23.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.022+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2786835977 wait complete. 2023-10-06T05:24:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.023+0000 7f0fd4c83700 1 Processor -- start 2023-10-06T05:24:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.023+0000 7f0fd4c83700 1 -- start start 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd00caad0 0x7f0fd00c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd0157200 0x7f0fd016a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00ae7a0 con 0x7f0fd00caad0 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd015a460 con 0x7f0fd0157200 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0159ea0 con 0x7f0fd00c8ff0 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd00caad0 0x7f0fd00c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd00caad0 0x7f0fd00c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54260/0 (socket says 172.21.15.45:54260) 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 -- 172.21.15.45:0/2134892240 learned_addr learned my addr 172.21.15.45:0/2134892240 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fce59c700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fced9d700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd0157200 0x7f0fd016a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 -- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 msgr2=0x7f0fd00c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 -- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd0157200 msgr2=0x7f0fd016a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd0157200 0x7f0fd016a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 -- 172.21.15.45:0/2134892240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8009ae0 con 0x7f0fd00caad0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc7fff700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd00caad0 0x7f0fd00c77d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800c590 tx=0x7f0fb800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc67fc700 1 -- 172.21.15.45:0/2134892240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8011590 con 0x7f0fd00caad0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc67fc700 1 -- 172.21.15.45:0/2134892240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb80116f0 con 0x7f0fd00caad0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd015a180 con 0x7f0fd00caad0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fc67fc700 1 -- 172.21.15.45:0/2134892240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8011ab0 con 0x7f0fd00caad0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.024+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd00c7d10 con 0x7f0fd00caad0 2023-10-06T05:24:23.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.025+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd00ae910 con 0x7f0fd00caad0 2023-10-06T05:24:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.026+0000 7f0fc67fc700 1 -- 172.21.15.45:0/2134892240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fb801b3b0 con 0x7f0fd00caad0 2023-10-06T05:24:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.026+0000 7f0fc67fc700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0fb0066f30 0x7f0fb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.026+0000 7f0fc67fc700 1 -- 172.21.15.45:0/2134892240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f0fb80843e0 con 0x7f0fd00caad0 2023-10-06T05:24:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.027+0000 7f0fce59c700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0fb0066f30 0x7f0fb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.027+0000 7f0fce59c700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0fb0066f30 0x7f0fb00693e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006f00 tx=0x7f0fc0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.031+0000 7f0fc67fc700 1 -- 172.21.15.45:0/2134892240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fb8050d50 con 0x7f0fd00caad0 2023-10-06T05:24:23.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.319+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f0fd004bb40 con 0x7f0fd00caad0 2023-10-06T05:24:23.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.868+0000 7f0fc67fc700 1 -- 172.21.15.45:0/2134892240 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v211) v1 ==== 107+0+0 (secure 0 0 0) 0x7f0fb8017070 con 0x7f0fd00caad0 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0fb0066f30 msgr2=0x7f0fb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0fb0066f30 0x7f0fb00693e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006f00 tx=0x7f0fc0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd00caad0 msgr2=0x7f0fd00c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd00caad0 0x7f0fd00c77d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800c590 tx=0x7f0fb800c950 comp rx=0 tx=0).stop 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 shutdown_connections 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0fb0066f30 0x7f0fb00693e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd0157200 0x7f0fd016a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fd00c8ff0 0x7f0fd00c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 --2- 172.21.15.45:0/2134892240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fd00caad0 0x7f0fd00c77d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 >> 172.21.15.45:0/2134892240 conn(0x7f0fd00cdf50 msgr2=0x7f0fd006d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 shutdown_connections 2023-10-06T05:24:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.872+0000 7f0fd4c83700 1 -- 172.21.15.45:0/2134892240 wait complete. 2023-10-06T05:24:23.876 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:24:23.890 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.978+0000 7f483dba9040 1 Processor -- start 2023-10-06T05:24:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.978+0000 7f483dba9040 1 -- start start 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f483dba9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 0x55ad8ce9d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f483dba9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 0x55ad8ce9d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f483dba9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8ce9e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f483dba9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8cd18620 con 0x55ad8cea0f00 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f483dba9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8cd0f400 con 0x55ad8ce157e0 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f483dba9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad8ce9fa20 con 0x55ad8cea0b30 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f48375c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8ce9e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f48375c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8ce9e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54276/0 (socket says 172.21.15.45:54276) 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f48375c8700 1 -- 172.21.15.45:0/132522672 learned_addr learned my addr 172.21.15.45:0/132522672 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f4836dc7700 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 0x55ad8ce9d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.979+0000 7f482e5c6700 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 0x55ad8ce9d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f48375c8700 1 -- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 msgr2=0x55ad8ce9d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f48375c8700 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 0x55ad8ce9d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f48375c8700 1 -- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 msgr2=0x55ad8ce9d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f48375c8700 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 0x55ad8ce9d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f48375c8700 1 -- 172.21.15.45:0/132522672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4820005db0 con 0x55ad8cea0f00 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f48375c8700 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8ce9e4d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f483000c520 tx=0x7f4830007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e9e9c1ca3e4abb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f4835dc5700 1 -- 172.21.15.45:0/132522672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300115f0 con 0x55ad8cea0f00 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f4835dc5700 1 -- 172.21.15.45:0/132522672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830011750 con 0x55ad8cea0f00 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f4835dc5700 1 -- 172.21.15.45:0/132522672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830011b10 con 0x55ad8cea0f00 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f483dba9040 1 -- 172.21.15.45:0/132522672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 msgr2=0x55ad8ce9e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.980+0000 7f483dba9040 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8ce9e4d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f483000c520 tx=0x7f4830007040 comp rx=0 tx=0).stop 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.981+0000 7f483dba9040 1 -- 172.21.15.45:0/132522672 shutdown_connections 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.981+0000 7f483dba9040 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8ce9e4d0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.981+0000 7f483dba9040 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 0x55ad8ce9d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.981+0000 7f483dba9040 1 --2- 172.21.15.45:0/132522672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 0x55ad8ce9d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.981+0000 7f483dba9040 1 -- 172.21.15.45:0/132522672 >> 172.21.15.45:0/132522672 conn(0x55ad8cdce450 msgr2=0x55ad8cdfcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.981+0000 7f483dba9040 1 -- 172.21.15.45:0/132522672 shutdown_connections 2023-10-06T05:24:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.981+0000 7f483dba9040 1 -- 172.21.15.45:0/132522672 wait complete. 2023-10-06T05:24:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.982+0000 7f483dba9040 1 Processor -- start 2023-10-06T05:24:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.982+0000 7f483dba9040 1 -- start start 2023-10-06T05:24:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f483dba9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 0x55ad8ce0f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f483dba9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 0x55ad8cea33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f483dba9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8cea38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f483dba9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8ce0fe50 con 0x55ad8cea0f00 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f483dba9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8ce100f0 con 0x55ad8ce157e0 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f483dba9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad8ce10390 con 0x55ad8cea0b30 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8cea38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8cea38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54288/0 (socket says 172.21.15.45:54288) 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 learned_addr learned my addr 172.21.15.45:0/1697337910 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f4836dc7700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 0x55ad8ce0f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f482e5c6700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 0x55ad8cea33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 msgr2=0x55ad8cea33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ad8cea0b30 0x55ad8cea33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 msgr2=0x55ad8ce0f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad8ce157e0 0x55ad8ce0f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483000d040 con 0x55ad8cea0f00 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.983+0000 7f48375c8700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8cea38e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f483000c410 tx=0x7f4830019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.984+0000 7f482ffff700 1 -- 172.21.15.45:0/1697337910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830011ad0 con 0x55ad8cea0f00 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.984+0000 7f482ffff700 1 -- 172.21.15.45:0/1697337910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483001d410 con 0x55ad8cea0f00 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.984+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ad8cd411c0 con 0x55ad8cea0f00 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.984+0000 7f482ffff700 1 -- 172.21.15.45:0/1697337910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483001d7d0 con 0x55ad8cea0f00 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.984+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad8cd273d0 con 0x55ad8cea0f00 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.985+0000 7f482ffff700 1 -- 172.21.15.45:0/1697337910 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4830003040 con 0x55ad8cea0f00 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.986+0000 7f482ffff700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48180658c0 0x7f4818067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.986+0000 7f482ffff700 1 -- 172.21.15.45:0/1697337910 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f4830085d00 con 0x55ad8cea0f00 2023-10-06T05:24:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.986+0000 7f4836dc7700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48180658c0 0x7f4818067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.992 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.986+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ad8ce0e4d0 0x55ad8cebba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.986+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebbf40 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.986+0000 7f482e5c6700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ad8ce0e4d0 0x55ad8cebba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.986+0000 7f4836dc7700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48180658c0 0x7f4818067d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f48280057a0 tx=0x7f4828005b40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.987+0000 7f482e5c6700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ad8ce0e4d0 0x55ad8cebba00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.988+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v111'567 uv541 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820000bc0 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.988+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebc2f0 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.990+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v111'568 uv568 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820000bc0 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.990+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebc6a0 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.991+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v111'569 uv569 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.994 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.991+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebda60 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.992+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'570 uv570 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.992+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebde10 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.994+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'571 uv571 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.994+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebcf40 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.995+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'572 uv572 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.995+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebd340 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.996+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v111'573 uv547 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.996+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.998+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v111'574 uv574 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.998+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.001 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:24.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.999+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v111'575 uv575 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:23.999+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.000+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'576 uv576 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.000+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.001+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'577 uv577 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.001+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.002+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'578 uv578 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.003+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.005+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v111'579 uv554 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.005+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.006+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'580 uv580 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.006+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.008+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v111'581 uv581 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.010 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:24.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.008+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.009+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'582 uv582 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.009+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.010+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'583 uv583 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.010+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.011+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'584 uv584 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.011+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.013+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'585 uv585 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.013+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.014+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v111'586 uv560 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.014+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.015+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v111'587 uv587 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.015+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.019 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.016+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v111'588 uv588 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.016+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.017+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'589 uv589 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.017+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.019+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'590 uv590 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.019+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.020+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'591 uv591 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.020+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.023+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v111'592 uv566 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.023+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.024+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v111'593 uv593 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.024+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.028 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.025+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v111'594 uv594 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.025+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.026+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'595 uv595 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.026+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebf1d0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.028+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'596 uv596 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.028+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cebe1c0 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.029+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'597 uv597 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8ce0e4d0 2023-10-06T05:24:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.029+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ad8cebe6d0 0x55ad8cec0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.029+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec0ff0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.029+0000 7f4836dc7700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ad8cebe6d0 0x55ad8cec0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.030+0000 7f4836dc7700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ad8cebe6d0 0x55ad8cec0ab0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.031+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'416 uv396 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.031+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec13a0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.033+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'417 uv417 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.033+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec1750 con 0x55ad8cebe6d0 2023-10-06T05:24:24.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.034+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'418 uv418 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:24.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.034+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec1ff0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.035+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v111'419 uv419 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.036+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec23a0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.037+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'420 uv420 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.037+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2750 con 0x55ad8cebe6d0 2023-10-06T05:24:24.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.038+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v111'421 uv415 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.038+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2b00 con 0x55ad8cebe6d0 2023-10-06T05:24:24.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.040+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v111'422 uv422 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.040+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.041+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v111'423 uv423 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.044 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.041+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.042+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'424 uv424 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.042+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.043+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'425 uv425 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.043+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.045+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v111'426 uv414 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.045+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.046+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v111'427 uv427 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.046+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.047+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v111'428 uv428 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.050 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.047+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec65d0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.049+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v111'429 uv429 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.049+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec65d0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.050+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'430 uv430 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.050+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.051+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v111'431 uv412 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.051+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.052+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v111'432 uv432 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.053+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.054+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v111'433 uv433 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.056 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:24.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.054+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec3ec0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.055+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'434 uv434 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.055+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec3ec0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.056+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'435 uv435 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.056+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec2eb0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.057+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'436 uv436 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.057+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ad8cec33c0 0x55ad8cec57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.057+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec5ce0 con 0x55ad8cec33c0 2023-10-06T05:24:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.057+0000 7f482e5c6700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ad8cec33c0 0x55ad8cec57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.058+0000 7f482e5c6700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ad8cec33c0 0x55ad8cec57a0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.059+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8cec33c0 2023-10-06T05:24:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.059+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec5ce0 con 0x55ad8cec33c0 2023-10-06T05:24:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.060+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v111'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820025040 con 0x55ad8cec33c0 2023-10-06T05:24:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.060+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec5da0 con 0x55ad8cec33c0 2023-10-06T05:24:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.061+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v111'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.064 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.061+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec71b0 con 0x55ad8cec33c0 2023-10-06T05:24:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.063+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.063+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec71b0 con 0x55ad8cec33c0 2023-10-06T05:24:24.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.064+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.064+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cec33c0 2023-10-06T05:24:24.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.065+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.065+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cec33c0 2023-10-06T05:24:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.066+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'80 uv79 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.066+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cec33c0 2023-10-06T05:24:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.067+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'81 uv81 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.067+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cec33c0 2023-10-06T05:24:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.068+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'82 uv82 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.071 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.068+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cec33c0 2023-10-06T05:24:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.070+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'83 uv83 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.070+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.071+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.071+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.072+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v111'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.072+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.073+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v111'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.076 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.073+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec71b0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.075+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.075+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec71b0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.076+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.076+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec61a0 con 0x55ad8cebe6d0 2023-10-06T05:24:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.077+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cebe6d0 2023-10-06T05:24:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.077+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ad8cec6820 0x55ad8cec8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.077+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9140 con 0x55ad8cec6820 2023-10-06T05:24:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.077+0000 7f48375c8700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ad8cec6820 0x55ad8cec8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.078+0000 7f48375c8700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ad8cec6820 0x55ad8cec8c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.079+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v111'205 uv198 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.079+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9610 con 0x55ad8cec6820 2023-10-06T05:24:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.080+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v111'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.080+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9b00 con 0x55ad8cec6820 2023-10-06T05:24:24.084 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.081+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v111'207 uv207 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.081+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cecb400 con 0x55ad8cec6820 2023-10-06T05:24:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.082+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.083+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cecb7b0 con 0x55ad8cec6820 2023-10-06T05:24:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.085+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.085+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ceca3f0 con 0x55ad8cec6820 2023-10-06T05:24:24.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.086+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.086+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ceca810 con 0x55ad8cec6820 2023-10-06T05:24:24.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.086+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v111'211 uv204 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.086+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cecabc0 con 0x55ad8cec6820 2023-10-06T05:24:24.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.088+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v111'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.088+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cecabc0 con 0x55ad8cec6820 2023-10-06T05:24:24.091 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:24.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.088+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v111'213 uv213 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.088+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cecabc0 con 0x55ad8cec6820 2023-10-06T05:24:24.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.089+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.090+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cecabc0 con 0x55ad8cec6820 2023-10-06T05:24:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.092+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.092+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ad8cecaf90 0x55ad8cece080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.092+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cece5c0 con 0x55ad8cecaf90 2023-10-06T05:24:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.092+0000 7f4836dc7700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ad8cecaf90 0x55ad8cece080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.092+0000 7f4836dc7700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ad8cecaf90 0x55ad8cece080 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.093+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v111'91 uv90 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cecaf90 2023-10-06T05:24:24.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.093+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cece5c0 con 0x55ad8cecaf90 2023-10-06T05:24:24.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.094+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v111'92 uv92 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cecaf90 2023-10-06T05:24:24.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.094+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cece5c0 con 0x55ad8cecaf90 2023-10-06T05:24:24.098 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:24.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.096+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v111'93 uv93 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cecaf90 2023-10-06T05:24:24.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.096+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cece5c0 con 0x55ad8cecaf90 2023-10-06T05:24:24.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.097+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'94 uv94 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cecaf90 2023-10-06T05:24:24.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.097+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9f80 con 0x55ad8cecaf90 2023-10-06T05:24:24.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.098+0000 7f4836dc7700 1 -- 172.21.15.45:0/1697337910 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'95 uv95 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4828007b80 con 0x55ad8cecaf90 2023-10-06T05:24:24.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.098+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9f80 con 0x55ad8cec33c0 2023-10-06T05:24:24.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.100+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.100+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9f80 con 0x55ad8cec33c0 2023-10-06T05:24:24.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.101+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v111'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.101+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9f80 con 0x55ad8cec33c0 2023-10-06T05:24:24.104 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.102+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v111'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.102+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9f80 con 0x55ad8cec33c0 2023-10-06T05:24:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.103+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.103+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9f80 con 0x55ad8cec33c0 2023-10-06T05:24:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.104+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.104+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8cec9f80 con 0x55ad8cec33c0 2023-10-06T05:24:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.106+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.106+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ad8cecefc0 0x55ad8ced13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.106+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced18e0 con 0x55ad8cecefc0 2023-10-06T05:24:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.106+0000 7f482e5c6700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ad8cecefc0 0x55ad8ced13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.106+0000 7f482e5c6700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ad8cecefc0 0x55ad8ced13a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.107+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v111'182 uv176 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.108+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced18e0 con 0x55ad8cecefc0 2023-10-06T05:24:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.109+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v111'183 uv183 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.109+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1920 con 0x55ad8cecefc0 2023-10-06T05:24:24.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.110+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v111'184 uv184 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.110+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1d20 con 0x55ad8cecefc0 2023-10-06T05:24:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.111+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'185 uv185 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.111+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1d20 con 0x55ad8cecefc0 2023-10-06T05:24:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.112+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'186 uv186 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.112+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1d20 con 0x55ad8cecefc0 2023-10-06T05:24:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.113+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v111'187 uv181 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.113+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1d20 con 0x55ad8cecefc0 2023-10-06T05:24:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.114+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v111'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.114+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1d20 con 0x55ad8cecefc0 2023-10-06T05:24:24.117 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.115+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v111'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.115+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1d20 con 0x55ad8cecefc0 2023-10-06T05:24:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.116+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v111'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.116+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced4a90 con 0x55ad8cecefc0 2023-10-06T05:24:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.117+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v111'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cecefc0 2023-10-06T05:24:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.117+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced2210 con 0x55ad8cec33c0 2023-10-06T05:24:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.118+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v111'83 uv81 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.118+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced2210 con 0x55ad8cec33c0 2023-10-06T05:24:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.119+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v111'84 uv84 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.119+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced2210 con 0x55ad8cec33c0 2023-10-06T05:24:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.120+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v111'85 uv85 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.123 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.121+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced2210 con 0x55ad8cec33c0 2023-10-06T05:24:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.122+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v111'86 uv86 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.122+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ad8cece6b0 0x55ad8ced4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.122+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced4d40 con 0x55ad8cece6b0 2023-10-06T05:24:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.122+0000 7f48375c8700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ad8cece6b0 0x55ad8ced4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.123+0000 7f48375c8700 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ad8cece6b0 0x55ad8ced4730 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.123+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cece6b0 2023-10-06T05:24:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.124+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced4d40 con 0x55ad8cece6b0 2023-10-06T05:24:24.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.125+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v111'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cece6b0 2023-10-06T05:24:24.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.125+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced4d40 con 0x55ad8cece6b0 2023-10-06T05:24:24.131 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:24.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.128+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v111'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cece6b0 2023-10-06T05:24:24.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.128+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced6150 con 0x55ad8cece6b0 2023-10-06T05:24:24.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.130+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cece6b0 2023-10-06T05:24:24.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.130+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced6150 con 0x55ad8cece6b0 2023-10-06T05:24:24.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.131+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cece6b0 2023-10-06T05:24:24.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.131+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cece6b0 2023-10-06T05:24:24.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.132+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cece6b0 2023-10-06T05:24:24.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.133+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec6820 2023-10-06T05:24:24.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.134+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v111'94 uv93 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.134+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec6820 2023-10-06T05:24:24.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.135+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v111'95 uv95 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.135+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec6820 2023-10-06T05:24:24.139 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:24.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.137+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v111'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.137+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec6820 2023-10-06T05:24:24.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.138+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v111'97 uv97 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.138+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec6820 2023-10-06T05:24:24.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.139+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v111'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f483005b040 con 0x55ad8cec6820 2023-10-06T05:24:24.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.139+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec33c0 2023-10-06T05:24:24.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.141+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v111'229 uv220 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.141+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec33c0 2023-10-06T05:24:24.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.142+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v111'230 uv230 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.142+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec33c0 2023-10-06T05:24:24.145 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:24.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.143+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v111'231 uv231 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.143+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec33c0 2023-10-06T05:24:24.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.144+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v111'232 uv232 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.144+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1da0 con 0x55ad8cec33c0 2023-10-06T05:24:24.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.146+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v111'233 uv233 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.146+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced1da0 con 0x55ad8cec33c0 2023-10-06T05:24:24.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.147+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v111'234 uv227 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.147+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec33c0 2023-10-06T05:24:24.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.148+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v111'235 uv235 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.148+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec33c0 2023-10-06T05:24:24.152 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:24.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.149+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v111'236 uv236 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.149+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced6150 con 0x55ad8cec33c0 2023-10-06T05:24:24.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.151+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v111'237 uv237 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.151+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced6150 con 0x55ad8cec33c0 2023-10-06T05:24:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.152+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v111'238 uv238 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.152+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5140 con 0x55ad8cec33c0 2023-10-06T05:24:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.153+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v111'239 uv239 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.153+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x55ad8ced5630 con 0x55ad8cec33c0 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f482e5c6700 1 -- 172.21.15.45:0/1697337910 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v111'240 uv240 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4820031040 con 0x55ad8cec33c0 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ad8cec33c0 msgr2=0x55ad8cec57a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ad8cec33c0 0x55ad8cec57a0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ad8ce0e4d0 msgr2=0x55ad8cebba00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ad8ce0e4d0 0x55ad8cebba00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ad8cecefc0 msgr2=0x55ad8ced13a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ad8cecefc0 0x55ad8ced13a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ad8cebe6d0 msgr2=0x55ad8cec0ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ad8cebe6d0 0x55ad8cec0ab0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 reap_dead start 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ad8cece6b0 msgr2=0x55ad8ced4730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ad8cece6b0 0x55ad8ced4730 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ad8cecaf90 msgr2=0x55ad8cece080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ad8cecaf90 0x55ad8cece080 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ad8cec6820 msgr2=0x55ad8cec8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.154+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ad8cec6820 0x55ad8cec8c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48180658c0 msgr2=0x7f4818067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48180658c0 0x7f4818067d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f48280057a0 tx=0x7f4828005b40 comp rx=0 tx=0).stop 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f48375c8700 1 -- 172.21.15.45:0/1697337910 reap_dead start 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 msgr2=0x55ad8cea38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8cea38e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f483000c410 tx=0x7f4830019000 comp rx=0 tx=0).stop 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 shutdown_connections 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 --2- 172.21.15.45:0/1697337910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad8cea0f00 0x55ad8cea38e0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 >> 172.21.15.45:0/1697337910 conn(0x55ad8cdce450 msgr2=0x55ad8cdcf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 shutdown_connections 2023-10-06T05:24:24.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.155+0000 7f483dba9040 1 -- 172.21.15.45:0/1697337910 wait complete. 2023-10-06T05:24:24.161 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2023-10-06T05:24:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.208+0000 7fa08ce9e040 1 Processor -- start 2023-10-06T05:24:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.208+0000 7fa08ce9e040 1 -- start start 2023-10-06T05:24:24.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa08ce9e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c0641bca0 0x562c064c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa08ce9e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c063ed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa08ce9e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c064bda00 0x562c063ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa08ce9e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c06396320 con 0x562c064c2b30 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa08ce9e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c0633a980 con 0x562c0641bca0 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa08ce9e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c0633a620 con 0x562c064bda00 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa0858bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c063ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa0858bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c063ed210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54298/0 (socket says 172.21.15.45:54298) 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa0858bb700 1 -- 172.21.15.45:0/1023596368 learned_addr learned my addr 172.21.15.45:0/1023596368 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa0860bc700 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c0641bca0 0x562c064c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa0868bd700 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c064bda00 0x562c063ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.209+0000 7fa0858bb700 1 -- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c064bda00 msgr2=0x562c063ed750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0858bb700 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c064bda00 0x562c063ed750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0858bb700 1 -- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c0641bca0 msgr2=0x562c064c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0858bb700 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c0641bca0 0x562c064c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0858bb700 1 -- 172.21.15.45:0/1023596368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c005cf0 con 0x562c064c2b30 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0860bc700 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c0641bca0 0x562c064c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0858bb700 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c063ed210 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x562c063f3b30 tx=0x7fa07c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b791008a433fde8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0848b9700 1 -- 172.21.15.45:0/1023596368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c002f70 con 0x562c064c2b30 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0848b9700 1 -- 172.21.15.45:0/1023596368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c000bc0 con 0x562c064c2b30 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa0848b9700 1 -- 172.21.15.45:0/1023596368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c007b10 con 0x562c064c2b30 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa08ce9e040 1 -- 172.21.15.45:0/1023596368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 msgr2=0x562c063ed210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c063ed210 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x562c063f3b30 tx=0x7fa07c015000 comp rx=0 tx=0).stop 2023-10-06T05:24:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa08ce9e040 1 -- 172.21.15.45:0/1023596368 shutdown_connections 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c064bda00 0x562c063ed750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c0641bca0 0x562c064c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/1023596368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c063ed210 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.210+0000 7fa08ce9e040 1 -- 172.21.15.45:0/1023596368 >> 172.21.15.45:0/1023596368 conn(0x562c063f0450 msgr2=0x562c064be780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.211+0000 7fa08ce9e040 1 -- 172.21.15.45:0/1023596368 shutdown_connections 2023-10-06T05:24:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.211+0000 7fa08ce9e040 1 -- 172.21.15.45:0/1023596368 wait complete. 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.211+0000 7fa08ce9e040 1 Processor -- start 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.211+0000 7fa08ce9e040 1 -- start start 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa08ce9e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c0641bca0 0x562c063f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa08ce9e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c064bda00 0x562c064ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa08ce9e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c064baa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa08ce9e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c06331400 con 0x562c064c2b30 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa08ce9e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c064d9f40 con 0x562c064bda00 2023-10-06T05:24:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa08ce9e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c063f1710 con 0x562c0641bca0 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa0868bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c064baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa0868bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c064baa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54308/0 (socket says 172.21.15.45:54308) 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa0868bd700 1 -- 172.21.15.45:0/4094550959 learned_addr learned my addr 172.21.15.45:0/4094550959 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa0860bc700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c0641bca0 0x562c063f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.212+0000 7fa0858bb700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c064bda00 0x562c064ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa0868bd700 1 -- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c0641bca0 msgr2=0x562c063f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa0868bd700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562c0641bca0 0x562c063f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa0868bd700 1 -- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c064bda00 msgr2=0x562c064ba4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa0868bd700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c064bda00 0x562c064ba4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa0868bd700 1 -- 172.21.15.45:0/4094550959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa070001e10 con 0x562c064c2b30 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa0858bb700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c064bda00 0x562c064ba4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa0868bd700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c064baa20 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fa08000d490 tx=0x7fa080006910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa076ffd700 1 -- 172.21.15.45:0/4094550959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080013480 con 0x562c064c2b30 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa076ffd700 1 -- 172.21.15.45:0/4094550959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0800135e0 con 0x562c064c2b30 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562c063493d0 con 0x562c064c2b30 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa076ffd700 1 -- 172.21.15.45:0/4094550959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0800139a0 con 0x562c064c2b30 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.213+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c0633cae0 con 0x562c064c2b30 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.215+0000 7fa076ffd700 1 -- 172.21.15.45:0/4094550959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa08001d460 con 0x562c064c2b30 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.215+0000 7fa076ffd700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa068065890 0x7fa068067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.215+0000 7fa076ffd700 1 -- 172.21.15.45:0/4094550959 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fa080083cc0 con 0x562c064c2b30 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.215+0000 7fa0860bc700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa068065890 0x7fa068067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.215+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x562c063ed720 0x562c0642fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.215+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v8 -- 0x562c06430060 con 0x562c063ed720 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.216+0000 7fa0858bb700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x562c063ed720 0x562c0642fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.216+0000 7fa0860bc700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa068065890 0x7fa068067d40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa078004170 tx=0x7fa078000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.216+0000 7fa0858bb700 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x562c063ed720 0x562c0642fb20 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa0858bb700 1 -- 172.21.15.45:0/4094550959 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v111'96 uv96 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa07c0079f0 con 0x562c063ed720 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x562c063ed720 msgr2=0x562c0642fb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x562c063ed720 0x562c0642fb20 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa068065890 msgr2=0x7fa068067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa068065890 0x7fa068067d40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa078004170 tx=0x7fa078000bc0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 msgr2=0x562c064baa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 --2- 172.21.15.45:0/4094550959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c064c2b30 0x562c064baa20 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fa08000d490 tx=0x7fa080006910 comp rx=0 tx=0).stop 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa0868bd700 1 -- 172.21.15.45:0/4094550959 reap_dead start 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 shutdown_connections 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 >> 172.21.15.45:0/4094550959 conn(0x562c063f0450 msgr2=0x562c064c2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 shutdown_connections 2023-10-06T05:24:24.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.217+0000 7fa08ce9e040 1 -- 172.21.15.45:0/4094550959 wait complete. 2023-10-06T05:24:24.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.403+0000 7f60c723c700 1 Processor -- start 2023-10-06T05:24:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.403+0000 7f60c723c700 1 -- start start 2023-10-06T05:24:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c723c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60c00b18e0 0x7f60c015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c723c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c0068750 con 0x7f60c006af30 2023-10-06T05:24:24.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c723c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c0067860 con 0x7f60c00b1cb0 2023-10-06T05:24:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c723c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c0067b40 con 0x7f60c00b18e0 2023-10-06T05:24:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c4fd8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60c00b18e0 0x7f60c015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c57d9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f60c006af30 0x7f60c00ab9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59536/0 (socket says 172.21.15.45:59536) 2023-10-06T05:24:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c4fd8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60c00b18e0 0x7f60c015b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54322/0 (socket says 172.21.15.45:54322) 2023-10-06T05:24:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.404+0000 7f60c57d9700 1 -- 172.21.15.45:0/3909389338 learned_addr learned my addr 172.21.15.45:0/3909389338 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60c4fd8700 1 -- 172.21.15.45:0/3909389338 >> v1:172.21.15.144:6789/0 conn(0x7f60c00b1cb0 legacy=0x7f60c006a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60c4fd8700 1 -- 172.21.15.45:0/3909389338 >> v1:172.21.15.45:6789/0 conn(0x7f60c006af30 legacy=0x7f60c00ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60bf7fe700 1 -- 172.21.15.45:0/3909389338 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139078037 0 0) 0x7f60c0067860 con 0x7f60c00b1cb0 2023-10-06T05:24:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60c4fd8700 1 -- 172.21.15.45:0/3909389338 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c0067e20 con 0x7f60c00b18e0 2023-10-06T05:24:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60c4fd8700 1 --2- 172.21.15.45:0/3909389338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c00b18e0 0x7f60c015b5b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f60b00098e0 tx=0x7f60b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2364acec3f9767d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60bf7fe700 1 -- 172.21.15.45:0/3909389338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0009e80 con 0x7f60c00b18e0 2023-10-06T05:24:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60bf7fe700 1 -- 172.21.15.45:0/3909389338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b000b860 con 0x7f60c00b18e0 2023-10-06T05:24:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60bf7fe700 1 -- 172.21.15.45:0/3909389338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000bbe0 con 0x7f60c00b18e0 2023-10-06T05:24:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60c723c700 1 -- 172.21.15.45:0/3909389338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c00b18e0 msgr2=0x7f60c015b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.405+0000 7f60c723c700 1 --2- 172.21.15.45:0/3909389338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c00b18e0 0x7f60c015b5b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f60b00098e0 tx=0x7f60b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.406+0000 7f60c723c700 1 -- 172.21.15.45:0/3909389338 shutdown_connections 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.406+0000 7f60c723c700 1 --2- 172.21.15.45:0/3909389338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c00b18e0 0x7f60c015b5b0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.406+0000 7f60c723c700 1 -- 172.21.15.45:0/3909389338 >> 172.21.15.45:0/3909389338 conn(0x7f60c00cc570 msgr2=0x7f60c00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.406+0000 7f60c723c700 1 -- 172.21.15.45:0/3909389338 shutdown_connections 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.406+0000 7f60c723c700 1 -- 172.21.15.45:0/3909389338 wait complete. 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.407+0000 7f60c723c700 1 Processor -- start 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.407+0000 7f60c723c700 1 -- start start 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c723c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c006af30 0x7f60c00bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c723c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c00b18e0 0x7f60c00b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c723c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c00b1cb0 0x7f60c00c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c723c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c0159e50 con 0x7f60c006af30 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c723c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00c0660 con 0x7f60c00b18e0 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c723c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00c0940 con 0x7f60c00b1cb0 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c006af30 0x7f60c00bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c006af30 0x7f60c00bffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54328/0 (socket says 172.21.15.45:54328) 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 -- 172.21.15.45:0/2128667016 learned_addr learned my addr 172.21.15.45:0/2128667016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60bffff700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c00b18e0 0x7f60c00b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c57d9700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c00b1cb0 0x7f60c00c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 -- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c00b1cb0 msgr2=0x7f60c00c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c00b1cb0 0x7f60c00c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 -- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c00b18e0 msgr2=0x7f60c00b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c00b18e0 0x7f60c00b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.408+0000 7f60c4fd8700 1 -- 172.21.15.45:0/2128667016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b0009420 con 0x7f60c006af30 2023-10-06T05:24:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.409+0000 7f60c4fd8700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c006af30 0x7f60c00bffd0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f60c006f9f0 tx=0x7f60b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.409+0000 7f60bcff9700 1 -- 172.21.15.45:0/2128667016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000bbe0 con 0x7f60c006af30 2023-10-06T05:24:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.409+0000 7f60bcff9700 1 -- 172.21.15.45:0/2128667016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b0011420 con 0x7f60c006af30 2023-10-06T05:24:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.409+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c00c0c20 con 0x7f60c006af30 2023-10-06T05:24:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.409+0000 7f60bcff9700 1 -- 172.21.15.45:0/2128667016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b00117c0 con 0x7f60c006af30 2023-10-06T05:24:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.409+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c00c11e0 con 0x7f60c006af30 2023-10-06T05:24:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.410+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c006e300 con 0x7f60c006af30 2023-10-06T05:24:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.410+0000 7f60bcff9700 1 -- 172.21.15.45:0/2128667016 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60b0011b30 con 0x7f60c006af30 2023-10-06T05:24:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.411+0000 7f60bcff9700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f60ac066f30 0x7f60ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.411+0000 7f60bcff9700 1 -- 172.21.15.45:0/2128667016 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f60b00867d0 con 0x7f60c006af30 2023-10-06T05:24:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.411+0000 7f60bffff700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f60ac066f30 0x7f60ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.412+0000 7f60bffff700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f60ac066f30 0x7f60ac0693e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f60b4007fd0 tx=0x7f60b4007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.416+0000 7f60bcff9700 1 -- 172.21.15.45:0/2128667016 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60b0053140 con 0x7f60c006af30 2023-10-06T05:24:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:24.666+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f60c004bb40 con 0x7f60c006af30 2023-10-06T05:24:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.011+0000 7f60bcff9700 1 -- 172.21.15.45:0/2128667016 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v212) v1 ==== 162+0+0 (secure 0 0 0) 0x7f60b0056760 con 0x7f60c006af30 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f60ac066f30 msgr2=0x7f60ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f60ac066f30 0x7f60ac0693e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f60b4007fd0 tx=0x7f60b4007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c006af30 msgr2=0x7f60c00bffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c006af30 0x7f60c00bffd0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f60c006f9f0 tx=0x7f60b0017000 comp rx=0 tx=0).stop 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 shutdown_connections 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f60ac066f30 0x7f60ac0693e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c00b1cb0 0x7f60c00c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60c006af30 0x7f60c00bffd0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 --2- 172.21.15.45:0/2128667016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c00b18e0 0x7f60c00b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 >> 172.21.15.45:0/2128667016 conn(0x7f60c00cc570 msgr2=0x7f60c00cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 shutdown_connections 2023-10-06T05:24:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.014+0000 7f60c723c700 1 -- 172.21.15.45:0/2128667016 wait complete. 2023-10-06T05:24:25.018 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:25.023 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:25.021+0000 7f17c371e700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input 2023-10-06T05:24:25.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:25.021+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:25.024 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:25.021+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:25.030 INFO:tasks.ceph.mds.c.smithi045.stderr: -26> 2023-10-06T05:24:25.021+0000 7f17c371e700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input 2023-10-06T05:24:25.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:25.052 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:24:25.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.166+0000 7fe32382a700 1 Processor -- start 2023-10-06T05:24:25.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.167+0000 7fe32382a700 1 -- start start 2023-10-06T05:24:25.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.167+0000 7fe32382a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe31c1595d0 0x7fe31c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.167+0000 7fe32382a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c0c7d00 con 0x7fe31c06b5c0 2023-10-06T05:24:25.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.167+0000 7fe32382a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c06bb40 con 0x7fe31c159200 2023-10-06T05:24:25.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.167+0000 7fe32382a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c0c7780 con 0x7fe31c1595d0 2023-10-06T05:24:25.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe321dc7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe31c06b5c0 0x7fe31c0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59548/0 (socket says 172.21.15.45:59548) 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe321dc7700 1 -- 172.21.15.45:0/2985962042 learned_addr learned my addr 172.21.15.45:0/2985962042 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe320dc5700 1 --2- 172.21.15.45:0/2985962042 >> v2:172.21.15.144:3300/0 conn(0x7fe31c1595d0 0x7fe31c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe313fff700 1 -- 172.21.15.45:0/2985962042 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545883803 0 0) 0x7fe31c0c7d00 con 0x7fe31c06b5c0 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe313fff700 1 -- 172.21.15.45:0/2985962042 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe308003570 con 0x7fe31c06b5c0 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe313fff700 1 -- 172.21.15.45:0/2985962042 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879186548 0 0) 0x7fe31c06bb40 con 0x7fe31c159200 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe313fff700 1 -- 172.21.15.45:0/2985962042 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe31c0c7d00 con 0x7fe31c159200 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe320dc5700 1 -- 172.21.15.45:0/2985962042 >> v1:172.21.15.144:6789/0 conn(0x7fe31c159200 legacy=0x7fe31c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe320dc5700 1 -- 172.21.15.45:0/2985962042 >> v1:172.21.15.45:6789/0 conn(0x7fe31c06b5c0 legacy=0x7fe31c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe320dc5700 1 -- 172.21.15.45:0/2985962042 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c06be20 con 0x7fe31c1595d0 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe320dc5700 1 --2- 172.21.15.45:0/2985962042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c1595d0 0x7fe31c06b080 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0098e0 tx=0x7fe30c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76186ca922aba53b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe313fff700 1 -- 172.21.15.45:0/2985962042 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c011630 con 0x7fe31c1595d0 2023-10-06T05:24:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe313fff700 1 -- 172.21.15.45:0/2985962042 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c011790 con 0x7fe31c1595d0 2023-10-06T05:24:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.168+0000 7fe313fff700 1 -- 172.21.15.45:0/2985962042 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c011b10 con 0x7fe31c1595d0 2023-10-06T05:24:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.169+0000 7fe32382a700 1 -- 172.21.15.45:0/2985962042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c1595d0 msgr2=0x7fe31c06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.169+0000 7fe32382a700 1 --2- 172.21.15.45:0/2985962042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c1595d0 0x7fe31c06b080 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0098e0 tx=0x7fe30c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.169+0000 7fe32382a700 1 -- 172.21.15.45:0/2985962042 shutdown_connections 2023-10-06T05:24:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.169+0000 7fe32382a700 1 --2- 172.21.15.45:0/2985962042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c1595d0 0x7fe31c06b080 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.169+0000 7fe32382a700 1 -- 172.21.15.45:0/2985962042 >> 172.21.15.45:0/2985962042 conn(0x7fe31c06f090 msgr2=0x7fe31c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.169+0000 7fe32382a700 1 -- 172.21.15.45:0/2985962042 shutdown_connections 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.169+0000 7fe32382a700 1 -- 172.21.15.45:0/2985962042 wait complete. 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.170+0000 7fe32382a700 1 Processor -- start 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.170+0000 7fe32382a700 1 -- start start 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe32382a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c06b5c0 0x7fe31c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe32382a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31c159200 0x7fe31c14b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe32382a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe31c1595d0 0x7fe31c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe32382a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c14b8f0 con 0x7fe31c159200 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe32382a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0c2440 con 0x7fe31c06b5c0 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe32382a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0c1830 con 0x7fe31c1595d0 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe320dc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31c159200 0x7fe31c14b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe320dc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31c159200 0x7fe31c14b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54366/0 (socket says 172.21.15.45:54366) 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe320dc5700 1 -- 172.21.15.45:0/1161216085 learned_addr learned my addr 172.21.15.45:0/1161216085 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.171+0000 7fe3215c6700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c06b5c0 0x7fe31c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe321dc7700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe31c1595d0 0x7fe31c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe320dc5700 1 -- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe31c1595d0 msgr2=0x7fe31c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe320dc5700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe31c1595d0 0x7fe31c0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe320dc5700 1 -- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c06b5c0 msgr2=0x7fe31c0c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe320dc5700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c06b5c0 0x7fe31c0c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe320dc5700 1 -- 172.21.15.45:0/1161216085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c009420 con 0x7fe31c159200 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe3215c6700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c06b5c0 0x7fe31c0c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe320dc5700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31c159200 0x7fe31c14b3b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0116b0 tx=0x7fe30c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe3117fa700 1 -- 172.21.15.45:0/1161216085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c011900 con 0x7fe31c159200 2023-10-06T05:24:25.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe3117fa700 1 -- 172.21.15.45:0/1161216085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c011a80 con 0x7fe31c159200 2023-10-06T05:24:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c0c1b10 con 0x7fe31c159200 2023-10-06T05:24:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe3117fa700 1 -- 172.21.15.45:0/1161216085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c01a680 con 0x7fe31c159200 2023-10-06T05:24:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.172+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c0c20d0 con 0x7fe31c159200 2023-10-06T05:24:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.173+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c0c25d0 con 0x7fe31c159200 2023-10-06T05:24:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.174+0000 7fe3117fa700 1 -- 172.21.15.45:0/1161216085 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe30c01a9f0 con 0x7fe31c159200 2023-10-06T05:24:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.174+0000 7fe3117fa700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe300066f30 0x7fe3000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.174+0000 7fe3117fa700 1 -- 172.21.15.45:0/1161216085 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fe30c0869e0 con 0x7fe31c159200 2023-10-06T05:24:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.174+0000 7fe3215c6700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe300066f30 0x7fe3000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.175+0000 7fe3215c6700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe300066f30 0x7fe3000693e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe318007fd0 tx=0x7fe318007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.179+0000 7fe3117fa700 1 -- 172.21.15.45:0/1161216085 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe30c0532f0 con 0x7fe31c159200 2023-10-06T05:24:25.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.424+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe31c0c7990 con 0x7fe31c159200 2023-10-06T05:24:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.425+0000 7fe3117fa700 1 -- 172.21.15.45:0/1161216085 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fe30c015090 con 0x7fe31c159200 2023-10-06T05:24:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:25.428 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":214,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:25.023586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe300066f30 msgr2=0x7fe3000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe300066f30 0x7fe3000693e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe318007fd0 tx=0x7fe318007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31c159200 msgr2=0x7fe31c14b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31c159200 0x7fe31c14b3b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0116b0 tx=0x7fe30c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 shutdown_connections 2023-10-06T05:24:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe300066f30 0x7fe3000693e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe31c1595d0 0x7fe31c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe31c06b5c0 0x7fe31c0c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 --2- 172.21.15.45:0/1161216085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe31c159200 0x7fe31c14b3b0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 >> 172.21.15.45:0/1161216085 conn(0x7fe31c06f090 msgr2=0x7fe31c070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 shutdown_connections 2023-10-06T05:24:25.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.427+0000 7fe32382a700 1 -- 172.21.15.45:0/1161216085 wait complete. 2023-10-06T05:24:25.434 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 214 2023-10-06T05:24:25.443 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:25.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.615+0000 7f7cc04c8700 1 Processor -- start 2023-10-06T05:24:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.615+0000 7f7cc04c8700 1 -- start start 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.616+0000 7f7cc04c8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7cb80cd340 0x7f7cb8157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.616+0000 7f7cc04c8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb815b000 con 0x7f7cb80c8ff0 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.616+0000 7f7cc04c8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb815b2e0 con 0x7f7cb80c93c0 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.616+0000 7f7cc04c8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb80c9830 con 0x7f7cb80cd340 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.616+0000 7f7cbea65700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7cb80cd340 0x7f7cb8157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.616+0000 7f7cbe264700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7cb80c8ff0 0x7f7cb80caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59554/0 (socket says 172.21.15.45:59554) 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.616+0000 7f7cbe264700 1 -- 172.21.15.45:0/3740723301 learned_addr learned my addr 172.21.15.45:0/3740723301 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbea65700 1 -- 172.21.15.45:0/3740723301 >> v1:172.21.15.144:6789/0 conn(0x7f7cb80c93c0 legacy=0x7f7cb8156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbea65700 1 -- 172.21.15.45:0/3740723301 >> v1:172.21.15.45:6789/0 conn(0x7f7cb80c8ff0 legacy=0x7f7cb80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbea65700 1 -- 172.21.15.45:0/3740723301 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb815b5c0 con 0x7f7cb80cd340 2023-10-06T05:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbd262700 1 -- 172.21.15.45:0/3740723301 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002001510 0 0) 0x7f7cb815b000 con 0x7f7cb80c8ff0 2023-10-06T05:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbea65700 1 --2- 172.21.15.45:0/3740723301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8157200 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009e60 tx=0x7f7cb0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36011aa42cf49bbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbd262700 1 -- 172.21.15.45:0/3740723301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb000e070 con 0x7f7cb80cd340 2023-10-06T05:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbd262700 1 -- 172.21.15.45:0/3740723301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb000bc30 con 0x7f7cb80cd340 2023-10-06T05:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cbd262700 1 -- 172.21.15.45:0/3740723301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0012430 con 0x7f7cb80cd340 2023-10-06T05:24:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3740723301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 msgr2=0x7f7cb8157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.617+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3740723301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8157200 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009e60 tx=0x7f7cb0009780 comp rx=0 tx=0).stop 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.618+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3740723301 shutdown_connections 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.618+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3740723301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8157200 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.618+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3740723301 >> 172.21.15.45:0/3740723301 conn(0x7f7cb80cdf50 msgr2=0x7f7cb80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.618+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3740723301 shutdown_connections 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.618+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3740723301 wait complete. 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.619+0000 7f7cc04c8700 1 Processor -- start 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.619+0000 7f7cc04c8700 1 -- start start 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cb80c8ff0 0x7f7cb81536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb80c93c0 0x7f7cb80b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb806a6a0 con 0x7f7cb80cd340 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb8153160 con 0x7f7cb80c93c0 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb8153400 con 0x7f7cb80c8ff0 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8068730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54386/0 (socket says 172.21.15.45:54386) 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 -- 172.21.15.45:0/3352095988 learned_addr learned my addr 172.21.15.45:0/3352095988 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbe264700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cb80c8ff0 0x7f7cb81536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbda63700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb80c93c0 0x7f7cb80b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 -- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cb80c8ff0 msgr2=0x7f7cb81536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cb80c8ff0 0x7f7cb81536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 -- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb80c93c0 msgr2=0x7f7cb80b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb80c93c0 0x7f7cb80b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 -- 172.21.15.45:0/3352095988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0009a20 con 0x7f7cb80cd340 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbe264700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cb80c8ff0 0x7f7cb81536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbda63700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb80c93c0 0x7f7cb80b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cbea65700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8068730 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00099f0 tx=0x7f7cb0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cae7fc700 1 -- 172.21.15.45:0/3352095988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb001d630 con 0x7f7cb80cd340 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cae7fc700 1 -- 172.21.15.45:0/3352095988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb001d7b0 con 0x7f7cb80cd340 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb806a330 con 0x7f7cb80cd340 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cae7fc700 1 -- 172.21.15.45:0/3352095988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb001db70 con 0x7f7cb80cd340 2023-10-06T05:24:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.621+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb8069a50 con 0x7f7cb80cd340 2023-10-06T05:24:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.622+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb806a190 con 0x7f7cb80cd340 2023-10-06T05:24:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.622+0000 7f7cae7fc700 1 -- 172.21.15.45:0/3352095988 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cb0024070 con 0x7f7cb80cd340 2023-10-06T05:24:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.622+0000 7f7cae7fc700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.623+0000 7f7cae7fc700 1 -- 172.21.15.45:0/3352095988 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f7cb0087860 con 0x7f7cb80cd340 2023-10-06T05:24:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.623+0000 7f7cbe264700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.623+0000 7f7cbe264700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4001890 tx=0x7f7cb4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.627+0000 7f7cae7fc700 1 -- 172.21.15.45:0/3352095988 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cb806a190 con 0x7f7cb80cd340 2023-10-06T05:24:25.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.873+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cb806a190 con 0x7f7cb80cd340 2023-10-06T05:24:25.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.873+0000 7f7cae7fc700 1 -- 172.21.15.45:0/3352095988 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f7cb806a190 con 0x7f7cb80cd340 2023-10-06T05:24:25.876 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:25.876 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":214,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:25.023586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:25.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 msgr2=0x7f7c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4001890 tx=0x7f7cb4009040 comp rx=0 tx=0).stop 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 msgr2=0x7f7cb8068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8068730 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00099f0 tx=0x7f7cb0019000 comp rx=0 tx=0).stop 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 shutdown_connections 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cb80cd340 0x7f7cb8068730 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cb80c8ff0 0x7f7cb81536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 --2- 172.21.15.45:0/3352095988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb80c93c0 0x7f7cb80b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 >> 172.21.15.45:0/3352095988 conn(0x7f7cb80cdf50 msgr2=0x7f7cb8152890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.875+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 shutdown_connections 2023-10-06T05:24:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:25.876+0000 7f7cc04c8700 1 -- 172.21.15.45:0/3352095988 wait complete. 2023-10-06T05:24:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 214 2023-10-06T05:24:25.892 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:25.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:26.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.023+0000 7fe6aaff1700 1 Processor -- start 2023-10-06T05:24:26.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.023+0000 7fe6aaff1700 1 -- start start 2023-10-06T05:24:26.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.024+0000 7fe6aaff1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe6a4156ce0 0x7fe6a41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.024+0000 7fe6aaff1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a406bf70 con 0x7fe6a406bad0 2023-10-06T05:24:26.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.024+0000 7fe6aaff1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a415b380 con 0x7fe6a415ac10 2023-10-06T05:24:26.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.024+0000 7fe6aaff1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40b02f0 con 0x7fe6a4156ce0 2023-10-06T05:24:26.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.025+0000 7fe6a3fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe6a406bad0 0x7fe6a40afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59564/0 (socket says 172.21.15.45:59564) 2023-10-06T05:24:26.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.025+0000 7fe6a3fff700 1 -- 172.21.15.45:0/3087524411 learned_addr learned my addr 172.21.15.45:0/3087524411 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:26.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.025+0000 7fe6a8d8d700 1 --2- 172.21.15.45:0/3087524411 >> v2:172.21.15.144:3300/0 conn(0x7fe6a4156ce0 0x7fe6a41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.025+0000 7fe6a8d8d700 1 -- 172.21.15.45:0/3087524411 >> v1:172.21.15.144:6789/0 conn(0x7fe6a415ac10 legacy=0x7fe6a40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.025+0000 7fe6a8d8d700 1 -- 172.21.15.45:0/3087524411 >> v1:172.21.15.45:6789/0 conn(0x7fe6a406bad0 legacy=0x7fe6a40afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.025+0000 7fe6a8d8d700 1 -- 172.21.15.45:0/3087524411 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a415b660 con 0x7fe6a4156ce0 2023-10-06T05:24:26.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.025+0000 7fe6a8d8d700 1 --2- 172.21.15.45:0/3087524411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4156ce0 0x7fe6a41570b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098e0 tx=0x7fe6940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60fa7060afe7891d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.029+0000 7fe6a37fe700 1 -- 172.21.15.45:0/3087524411 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694009e80 con 0x7fe6a4156ce0 2023-10-06T05:24:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.029+0000 7fe6a37fe700 1 -- 172.21.15.45:0/3087524411 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69400b860 con 0x7fe6a4156ce0 2023-10-06T05:24:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.029+0000 7fe6a37fe700 1 -- 172.21.15.45:0/3087524411 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69400bbe0 con 0x7fe6a4156ce0 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.029+0000 7fe6aaff1700 1 -- 172.21.15.45:0/3087524411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4156ce0 msgr2=0x7fe6a41570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.029+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/3087524411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4156ce0 0x7fe6a41570b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098e0 tx=0x7fe6940091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.030+0000 7fe6aaff1700 1 -- 172.21.15.45:0/3087524411 shutdown_connections 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.030+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/3087524411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4156ce0 0x7fe6a41570b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.030+0000 7fe6aaff1700 1 -- 172.21.15.45:0/3087524411 >> 172.21.15.45:0/3087524411 conn(0x7fe6a406f090 msgr2=0x7fe6a406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.030+0000 7fe6aaff1700 1 -- 172.21.15.45:0/3087524411 shutdown_connections 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.030+0000 7fe6aaff1700 1 -- 172.21.15.45:0/3087524411 wait complete. 2023-10-06T05:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.031+0000 7fe6aaff1700 1 Processor -- start 2023-10-06T05:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.032+0000 7fe6aaff1700 1 -- start start 2023-10-06T05:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.032+0000 7fe6aaff1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6a406bad0 0x7fe6a4154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6aaff1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6a4156ce0 0x7fe6a40b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6aaff1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a415ac10 0x7fe6a40b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6aaff1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c90a0 con 0x7fe6a406bad0 2023-10-06T05:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6aaff1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a416acf0 con 0x7fe6a415ac10 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6aaff1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40ac2d0 con 0x7fe6a4156ce0 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6a406bad0 0x7fe6a4154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6a406bad0 0x7fe6a4154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54398/0 (socket says 172.21.15.45:54398) 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 -- 172.21.15.45:0/52839612 learned_addr learned my addr 172.21.15.45:0/52839612 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a958e700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a415ac10 0x7fe6a40b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a3fff700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6a4156ce0 0x7fe6a40b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 -- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6a4156ce0 msgr2=0x7fe6a40b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6a4156ce0 0x7fe6a40b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 -- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a415ac10 msgr2=0x7fe6a40b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a415ac10 0x7fe6a40b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 -- 172.21.15.45:0/52839612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe694009420 con 0x7fe6a406bad0 2023-10-06T05:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a8d8d700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6a406bad0 0x7fe6a4154a70 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe6a406aed0 tx=0x7fe694017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a0ff9700 1 -- 172.21.15.45:0/52839612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69400b9d0 con 0x7fe6a406bad0 2023-10-06T05:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6a0ff9700 1 -- 172.21.15.45:0/52839612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69400bb30 con 0x7fe6a406bad0 2023-10-06T05:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.033+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a416aa10 con 0x7fe6a406bad0 2023-10-06T05:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.034+0000 7fe6a0ff9700 1 -- 172.21.15.45:0/52839612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694025650 con 0x7fe6a406bad0 2023-10-06T05:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.034+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a40c7850 con 0x7fe6a406bad0 2023-10-06T05:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.035+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a406e2f0 con 0x7fe6a406bad0 2023-10-06T05:24:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.035+0000 7fe6a0ff9700 1 -- 172.21.15.45:0/52839612 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe694023070 con 0x7fe6a406bad0 2023-10-06T05:24:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.035+0000 7fe6a0ff9700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe684066f30 0x7fe6840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.036+0000 7fe6a0ff9700 1 -- 172.21.15.45:0/52839612 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fe6940864a0 con 0x7fe6a406bad0 2023-10-06T05:24:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.036+0000 7fe6a3fff700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe684066f30 0x7fe6840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.036+0000 7fe6a3fff700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe684066f30 0x7fe6840693e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40afd50 tx=0x7fe68c007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.041+0000 7fe6a0ff9700 1 -- 172.21.15.45:0/52839612 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe694052d10 con 0x7fe6a406bad0 2023-10-06T05:24:26.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.286+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6a415b940 con 0x7fe6a406bad0 2023-10-06T05:24:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.287+0000 7fe6a0ff9700 1 -- 172.21.15.45:0/52839612 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) v1 ==== 95+0+4599 (secure 0 0 0) 0x7fe694056330 con 0x7fe6a406bad0 2023-10-06T05:24:26.289 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:26.289 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":215,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2475927951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2475927951},{"type":"v1","addr":"172.21.15.45:6835","nonce":2475927951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":215}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:25.023586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe684066f30 msgr2=0x7fe6840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe684066f30 0x7fe6840693e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40afd50 tx=0x7fe68c007eb0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6a406bad0 msgr2=0x7fe6a4154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6a406bad0 0x7fe6a4154a70 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe6a406aed0 tx=0x7fe694017000 comp rx=0 tx=0).stop 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 shutdown_connections 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe684066f30 0x7fe6840693e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a415ac10 0x7fe6a40b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6a406bad0 0x7fe6a4154a70 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 --2- 172.21.15.45:0/52839612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6a4156ce0 0x7fe6a40b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 >> 172.21.15.45:0/52839612 conn(0x7fe6a406f090 msgr2=0x7fe6a40687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 shutdown_connections 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.289+0000 7fe6aaff1700 1 -- 172.21.15.45:0/52839612 wait complete. 2023-10-06T05:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 215 2023-10-06T05:24:26.304 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2023-10-06T05:24:26.305 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2023-10-06T05:24:26.305 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:26.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:26.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:26.305 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:26.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:26.325 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:26.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:26.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.500+0000 7fef72b24700 1 Processor -- start 2023-10-06T05:24:26.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.500+0000 7fef72b24700 1 -- start start 2023-10-06T05:24:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef72b24700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef6c153650 0x7fef6c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef72b24700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c0676c0 con 0x7fef6c153a20 2023-10-06T05:24:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef72b24700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c0679a0 con 0x7fef6c06afa0 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef72b24700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c067c80 con 0x7fef6c153650 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef708c0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef6c153650 0x7fef6c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef708c0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef6c153650 0x7fef6c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58300/0 (socket says 172.21.15.45:58300) 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef6bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef6c153a20 0x7fef6c06a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41546/0 (socket says 172.21.15.45:41546) 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.501+0000 7fef708c0700 1 -- 172.21.15.45:0/2544261872 learned_addr learned my addr 172.21.15.45:0/2544261872 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.502+0000 7fef708c0700 1 -- 172.21.15.45:0/2544261872 >> v1:172.21.15.144:6789/0 conn(0x7fef6c06afa0 legacy=0x7fef6c068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.502+0000 7fef708c0700 1 -- 172.21.15.45:0/2544261872 >> v1:172.21.15.45:6789/0 conn(0x7fef6c153a20 legacy=0x7fef6c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.502+0000 7fef708c0700 1 -- 172.21.15.45:0/2544261872 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c067f60 con 0x7fef6c153650 2023-10-06T05:24:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.502+0000 7fef708c0700 1 --2- 172.21.15.45:0/2544261872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153650 0x7fef6c15a930 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0099b0 tx=0x7fef5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffedbca9a2475fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.502+0000 7fef6b7fe700 1 -- 172.21.15.45:0/2544261872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00b860 con 0x7fef6c153650 2023-10-06T05:24:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.502+0000 7fef6b7fe700 1 -- 172.21.15.45:0/2544261872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c00b9c0 con 0x7fef6c153650 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.502+0000 7fef6b7fe700 1 -- 172.21.15.45:0/2544261872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00bd60 con 0x7fef6c153650 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.503+0000 7fef72b24700 1 -- 172.21.15.45:0/2544261872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153650 msgr2=0x7fef6c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.503+0000 7fef72b24700 1 --2- 172.21.15.45:0/2544261872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153650 0x7fef6c15a930 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0099b0 tx=0x7fef5c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.503+0000 7fef72b24700 1 -- 172.21.15.45:0/2544261872 shutdown_connections 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.503+0000 7fef72b24700 1 --2- 172.21.15.45:0/2544261872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153650 0x7fef6c15a930 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.503+0000 7fef72b24700 1 -- 172.21.15.45:0/2544261872 >> 172.21.15.45:0/2544261872 conn(0x7fef6c0cc5a0 msgr2=0x7fef6c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.503+0000 7fef72b24700 1 -- 172.21.15.45:0/2544261872 shutdown_connections 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.503+0000 7fef72b24700 1 -- 172.21.15.45:0/2544261872 wait complete. 2023-10-06T05:24:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.504+0000 7fef72b24700 1 Processor -- start 2023-10-06T05:24:26.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.504+0000 7fef72b24700 1 -- start start 2023-10-06T05:24:26.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef72b24700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c06afa0 0x7fef6c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef72b24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c153650 0x7fef6c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef72b24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153a20 0x7fef6c169230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef72b24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0b0070 con 0x7fef6c153a20 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef72b24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0c1150 con 0x7fef6c153650 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef72b24700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0c1430 con 0x7fef6c06afa0 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153a20 0x7fef6c169230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153a20 0x7fef6c169230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58316/0 (socket says 172.21.15.45:58316) 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 -- 172.21.15.45:0/148570301 learned_addr learned my addr 172.21.15.45:0/148570301 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef6bfff700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c153650 0x7fef6c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef708c0700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c06afa0 0x7fef6c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 -- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c06afa0 msgr2=0x7fef6c156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c06afa0 0x7fef6c156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 -- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c153650 msgr2=0x7fef6c155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c153650 0x7fef6c155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.505+0000 7fef710c1700 1 -- 172.21.15.45:0/148570301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c0094f0 con 0x7fef6c153a20 2023-10-06T05:24:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.506+0000 7fef710c1700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153a20 0x7fef6c169230 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fef6400c910 tx=0x7fef640016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.506+0000 7fef68ff9700 1 -- 172.21.15.45:0/148570301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64012630 con 0x7fef6c153a20 2023-10-06T05:24:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.506+0000 7fef68ff9700 1 -- 172.21.15.45:0/148570301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef64012790 con 0x7fef6c153a20 2023-10-06T05:24:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.506+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c0c1710 con 0x7fef6c153a20 2023-10-06T05:24:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.506+0000 7fef68ff9700 1 -- 172.21.15.45:0/148570301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64012b50 con 0x7fef6c153a20 2023-10-06T05:24:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.506+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c0c1cd0 con 0x7fef6c153a20 2023-10-06T05:24:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.507+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c0cc0e0 con 0x7fef6c153a20 2023-10-06T05:24:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.507+0000 7fef68ff9700 1 -- 172.21.15.45:0/148570301 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef6401b450 con 0x7fef6c153a20 2023-10-06T05:24:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.508+0000 7fef68ff9700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef4c066f30 0x7fef4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.508+0000 7fef68ff9700 1 -- 172.21.15.45:0/148570301 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fef64084fe0 con 0x7fef6c153a20 2023-10-06T05:24:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.508+0000 7fef708c0700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef4c066f30 0x7fef4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.509+0000 7fef708c0700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef4c066f30 0x7fef4c0693e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0099b0 tx=0x7fef5c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.513+0000 7fef68ff9700 1 -- 172.21.15.45:0/148570301 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef64055340 con 0x7fef6c153a20 2023-10-06T05:24:26.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:26.761+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fef6c15ae70 con 0x7fef6c153a20 2023-10-06T05:24:27.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.034+0000 7fef68ff9700 1 -- 172.21.15.45:0/148570301 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v216) v1 ==== 161+0+0 (secure 0 0 0) 0x7fef64016090 con 0x7fef6c153a20 2023-10-06T05:24:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef4c066f30 msgr2=0x7fef4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef4c066f30 0x7fef4c0693e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0099b0 tx=0x7fef5c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153a20 msgr2=0x7fef6c169230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153a20 0x7fef6c169230 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fef6400c910 tx=0x7fef640016f0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 shutdown_connections 2023-10-06T05:24:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef4c066f30 0x7fef4c0693e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6c153a20 0x7fef6c169230 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c06afa0 0x7fef6c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 --2- 172.21.15.45:0/148570301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c153650 0x7fef6c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 >> 172.21.15.45:0/148570301 conn(0x7fef6c0cc5a0 msgr2=0x7fef6c06ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 shutdown_connections 2023-10-06T05:24:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.037+0000 7fef72b24700 1 -- 172.21.15.45:0/148570301 wait complete. 2023-10-06T05:24:27.042 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:27.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.190+0000 7f9d782e2700 1 Processor -- start 2023-10-06T05:24:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.190+0000 7f9d782e2700 1 -- start start 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.191+0000 7f9d782e2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.191+0000 7f9d782e2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d7006bad0 0x7f9d7015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.191+0000 7f9d782e2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9d700afbe0 0x7f9d700ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.191+0000 7f9d782e2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7015b150 con 0x7f9d7006bad0 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.191+0000 7f9d782e2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7006bf40 con 0x7f9d700afbe0 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.191+0000 7f9d782e2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7015b380 con 0x7f9d70156ce0 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d7006bad0 0x7f9d7015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d7006bad0 0x7f9d7015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58320/0 (socket says 172.21.15.45:58320) 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 -- 172.21.15.45:0/4032344674 learned_addr learned my addr 172.21.15.45:0/4032344674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7607e700 1 --2- 172.21.15.45:0/4032344674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7687f700 1 --2- 172.21.15.45:0/4032344674 >> v2:172.21.15.144:3300/0 conn(0x7f9d700afbe0 0x7f9d700ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 -- 172.21.15.45:0/4032344674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 msgr2=0x7f9d701570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 --2- 172.21.15.45:0/4032344674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 -- 172.21.15.45:0/4032344674 >> v2:172.21.15.144:3300/0 conn(0x7f9d700afbe0 msgr2=0x7f9d700ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 --2- 172.21.15.45:0/4032344674 >> v2:172.21.15.144:3300/0 conn(0x7f9d700afbe0 0x7f9d700ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 -- 172.21.15.45:0/4032344674 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7015b660 con 0x7f9d7006bad0 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7587d700 1 --2- 172.21.15.45:0/4032344674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d7015ac10 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9d700aec60 tx=0x7f9d60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60df86ca62d12fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7507c700 1 -- 172.21.15.45:0/4032344674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000b860 con 0x7f9d7006bad0 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7507c700 1 -- 172.21.15.45:0/4032344674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6000b9c0 con 0x7f9d7006bad0 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.192+0000 7f9d7507c700 1 -- 172.21.15.45:0/4032344674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000bd60 con 0x7f9d7006bad0 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 -- 172.21.15.45:0/4032344674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 msgr2=0x7f9d7015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 --2- 172.21.15.45:0/4032344674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d7015ac10 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9d700aec60 tx=0x7f9d60009250 comp rx=0 tx=0).stop 2023-10-06T05:24:27.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 -- 172.21.15.45:0/4032344674 shutdown_connections 2023-10-06T05:24:27.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 --2- 172.21.15.45:0/4032344674 >> v2:172.21.15.144:3300/0 conn(0x7f9d700afbe0 0x7f9d700ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 --2- 172.21.15.45:0/4032344674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 --2- 172.21.15.45:0/4032344674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d7015ac10 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 -- 172.21.15.45:0/4032344674 >> 172.21.15.45:0/4032344674 conn(0x7f9d7006f090 msgr2=0x7f9d7006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:27.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 -- 172.21.15.45:0/4032344674 shutdown_connections 2023-10-06T05:24:27.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.193+0000 7f9d782e2700 1 -- 172.21.15.45:0/4032344674 wait complete. 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.194+0000 7f9d782e2700 1 Processor -- start 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.194+0000 7f9d782e2700 1 -- start start 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d782e2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d70153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d782e2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d700afbe0 0x7f9d700c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d782e2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d782e2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700c32c0 con 0x7f9d7006bad0 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d782e2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700c73d0 con 0x7f9d700afbe0 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d782e2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70159a10 con 0x7f9d70156ce0 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d7607e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d70153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d7607e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d70153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58322/0 (socket says 172.21.15.45:58322) 2023-10-06T05:24:27.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d7607e700 1 -- 172.21.15.45:0/3386098390 learned_addr learned my addr 172.21.15.45:0/3386098390 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d7587d700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d700afbe0 0x7f9d700c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.195+0000 7f9d7687f700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d7607e700 1 -- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 msgr2=0x7f9d701594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d7607e700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d7607e700 1 -- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d700afbe0 msgr2=0x7f9d700c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d7607e700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d700afbe0 0x7f9d700c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d7607e700 1 -- 172.21.15.45:0/3386098390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600094f0 con 0x7f9d7006bad0 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d7687f700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d7607e700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d70153560 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00d860 tx=0x7f9d6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d667fc700 1 -- 172.21.15.45:0/3386098390 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0125f0 con 0x7f9d7006bad0 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d667fc700 1 -- 172.21.15.45:0/3386098390 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c012750 con 0x7f9d7006bad0 2023-10-06T05:24:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d70159cd0 con 0x7f9d7006bad0 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d667fc700 1 -- 172.21.15.45:0/3386098390 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c012b10 con 0x7f9d7006bad0 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.196+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7015a290 con 0x7f9d7006bad0 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.197+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d700c76b0 con 0x7f9d7006bad0 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.197+0000 7f9d667fc700 1 -- 172.21.15.45:0/3386098390 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d6c01e460 con 0x7f9d7006bad0 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.198+0000 7f9d667fc700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d54066f30 0x7f9d540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.198+0000 7f9d667fc700 1 -- 172.21.15.45:0/3386098390 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f9d6c085430 con 0x7f9d7006bad0 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.198+0000 7f9d7587d700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d54066f30 0x7f9d540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.199+0000 7f9d7587d700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d54066f30 0x7f9d540693e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9d700aec60 tx=0x7f9d60017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.202+0000 7f9d667fc700 1 -- 172.21.15.45:0/3386098390 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d6c051d40 con 0x7f9d7006bad0 2023-10-06T05:24:27.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:27.489+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f9d7015b940 con 0x7f9d7006bad0 2023-10-06T05:24:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.035+0000 7f9d667fc700 1 -- 172.21.15.45:0/3386098390 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v217) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9d6c055360 con 0x7f9d7006bad0 2023-10-06T05:24:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d54066f30 msgr2=0x7f9d540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d54066f30 0x7f9d540693e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9d700aec60 tx=0x7f9d60017000 comp rx=0 tx=0).stop 2023-10-06T05:24:28.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 msgr2=0x7f9d70153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d70153560 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00d860 tx=0x7f9d6c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 shutdown_connections 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d54066f30 0x7f9d540693e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d70156ce0 0x7f9d701594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7006bad0 0x7f9d70153560 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 --2- 172.21.15.45:0/3386098390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d700afbe0 0x7f9d700c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 >> 172.21.15.45:0/3386098390 conn(0x7f9d7006f090 msgr2=0x7f9d700cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 shutdown_connections 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.038+0000 7f9d782e2700 1 -- 172.21.15.45:0/3386098390 wait complete. 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:28.055 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.101+0000 7fe8f8b46040 1 Processor -- start 2023-10-06T05:24:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.101+0000 7fe8f8b46040 1 -- start start 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f8b46040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b22effe0 0x5599b22f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f8b46040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b22f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f8b46040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b23b1970 0x5599b231f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f8b46040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b223c980 con 0x5599b233b340 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f8b46040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b223c620 con 0x5599b22effe0 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f8b46040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599b2233400 con 0x5599b23b1970 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b22f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b22f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58330/0 (socket says 172.21.15.45:58330) 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 -- 172.21.15.45:0/98251147 learned_addr learned my addr 172.21.15.45:0/98251147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b22effe0 0x5599b22f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f2565700 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b23b1970 0x5599b231f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 -- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b23b1970 msgr2=0x5599b231f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b23b1970 0x5599b231f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 -- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b22effe0 msgr2=0x5599b22f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b22effe0 0x5599b22f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 -- 172.21.15.45:0/98251147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec00a0e0 con 0x5599b233b340 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8f1563700 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b22f3760 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400acf0 tx=0x7fe8e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7235a6007775fb0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.102+0000 7fe8e3fff700 1 -- 172.21.15.45:0/98251147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4002f90 con 0x5599b233b340 2023-10-06T05:24:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8e3fff700 1 -- 172.21.15.45:0/98251147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e4000bc0 con 0x5599b233b340 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8e3fff700 1 -- 172.21.15.45:0/98251147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4008ba0 con 0x5599b233b340 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 -- 172.21.15.45:0/98251147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 msgr2=0x5599b22f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b22f3760 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400acf0 tx=0x7fe8e4016000 comp rx=0 tx=0).stop 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 -- 172.21.15.45:0/98251147 shutdown_connections 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b23b1970 0x5599b231f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b22effe0 0x5599b22f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/98251147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b22f3760 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 -- 172.21.15.45:0/98251147 >> 172.21.15.45:0/98251147 conn(0x5599b22f24a0 msgr2=0x5599b22f2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 -- 172.21.15.45:0/98251147 shutdown_connections 2023-10-06T05:24:28.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.103+0000 7fe8f8b46040 1 -- 172.21.15.45:0/98251147 wait complete. 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.104+0000 7fe8f8b46040 1 Processor -- start 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.104+0000 7fe8f8b46040 1 -- start start 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f8b46040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b22effe0 0x5599b22f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f8b46040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b23a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f8b46040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b23b1970 0x5599b23caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f8b46040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b233bc80 con 0x5599b233b340 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f8b46040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b22f4c20 con 0x5599b23b1970 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f8b46040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599b2320460 con 0x5599b22effe0 2023-10-06T05:24:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b23a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b23a83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58338/0 (socket says 172.21.15.45:58338) 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 learned_addr learned my addr 172.21.15.45:0/2041497191 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b22effe0 0x5599b22f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f2565700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b23b1970 0x5599b23caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b22effe0 msgr2=0x5599b22f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5599b22effe0 0x5599b22f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b23b1970 msgr2=0x5599b23caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599b23b1970 0x5599b23caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599b22651c0 con 0x5599b233b340 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8f1563700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b23a83e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4008fd0 tx=0x7fe8e401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8e27fc700 1 -- 172.21.15.45:0/2041497191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e401d420 con 0x5599b233b340 2023-10-06T05:24:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.105+0000 7fe8e27fc700 1 -- 172.21.15.45:0/2041497191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e401d580 con 0x5599b233b340 2023-10-06T05:24:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.106+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e4005cf0 con 0x5599b233b340 2023-10-06T05:24:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.106+0000 7fe8e27fc700 1 -- 172.21.15.45:0/2041497191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e401d920 con 0x5599b233b340 2023-10-06T05:24:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.106+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599b224b3d0 con 0x5599b233b340 2023-10-06T05:24:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.107+0000 7fe8e27fc700 1 -- 172.21.15.45:0/2041497191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8e401dc90 con 0x5599b233b340 2023-10-06T05:24:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.107+0000 7fe8e27fc700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8d4065890 0x7fe8d4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.107+0000 7fe8e27fc700 1 -- 172.21.15.45:0/2041497191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fe8e40865f0 con 0x5599b233b340 2023-10-06T05:24:28.112 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.108+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8d4065890 0x7fe8d4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.108+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5599b2320e40 0x5599b231f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.108+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b231f7a0 con 0x5599b2320e40 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.108+0000 7fe8f2565700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5599b2320e40 0x5599b231f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.108+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8d4065890 0x7fe8d4067d40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e80068d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.108+0000 7fe8f2565700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5599b2320e40 0x5599b231f260 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.110+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v112'598 uv572 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.110+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b231fb80 con 0x5599b2320e40 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.111+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v112'599 uv599 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.111+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b231ff60 con 0x5599b2320e40 2023-10-06T05:24:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.113+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v112'600 uv600 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.118 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.113+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e5f60 con 0x5599b2320e40 2023-10-06T05:24:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.114+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'601 uv601 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.114+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e6310 con 0x5599b2320e40 2023-10-06T05:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.115+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'602 uv602 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.115+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e4f50 con 0x5599b2320e40 2023-10-06T05:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.116+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'603 uv603 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.116+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e5300 con 0x5599b2320e40 2023-10-06T05:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.117+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v112'604 uv578 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.118+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.119+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v112'605 uv605 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.119+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.122 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.120+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v112'606 uv606 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.120+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.121+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'607 uv607 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.121+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.123+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'608 uv608 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.123+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.124+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'609 uv609 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.124+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.125+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v112'610 uv585 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.125+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.126+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'611 uv611 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.126+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.128+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v112'612 uv612 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.129 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:28.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.128+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.129+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'613 uv613 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.129+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.130+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'614 uv614 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.130+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e56b0 con 0x5599b2320e40 2023-10-06T05:24:28.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.131+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'615 uv615 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.131+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.133+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'616 uv616 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.133+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.134+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v112'617 uv591 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.134+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.135+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v112'618 uv618 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.135+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.138 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:28.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.136+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v112'619 uv619 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.136+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e76d0 con 0x5599b2320e40 2023-10-06T05:24:28.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.138+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'620 uv620 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.138+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e76d0 con 0x5599b2320e40 2023-10-06T05:24:28.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.139+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'621 uv621 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.139+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.140+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'622 uv622 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.141+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.142+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v112'623 uv597 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.142+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.144+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v112'624 uv624 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.144+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.146 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:28.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.145+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v112'625 uv625 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.145+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e76d0 con 0x5599b2320e40 2023-10-06T05:24:28.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.146+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'626 uv626 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.146+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e76d0 con 0x5599b2320e40 2023-10-06T05:24:28.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.148+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'627 uv627 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.148+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e66c0 con 0x5599b2320e40 2023-10-06T05:24:28.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.149+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'628 uv628 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b2320e40 2023-10-06T05:24:28.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.149+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5599b23e6bd0 0x5599b23e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.149+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e94f0 con 0x5599b23e6bd0 2023-10-06T05:24:28.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.149+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5599b23e6bd0 0x5599b23e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.150+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5599b23e6bd0 0x5599b23e8fb0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.151+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'437 uv420 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.151+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e98a0 con 0x5599b23e6bd0 2023-10-06T05:24:28.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.153+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'438 uv438 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.153+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23e9c50 con 0x5599b23e6bd0 2023-10-06T05:24:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.154+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'439 uv439 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.272 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:28.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.154+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ea4f0 con 0x5599b23e6bd0 2023-10-06T05:24:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.155+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v112'440 uv440 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.155+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ea8a0 con 0x5599b23e6bd0 2023-10-06T05:24:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.156+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'441 uv441 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.156+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.157+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v112'442 uv425 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.157+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.158+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v112'443 uv443 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.159+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.160+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v112'444 uv444 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.277 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:28.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.160+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.161+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'445 uv445 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.161+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.162+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'446 uv446 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.162+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.163+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v112'447 uv430 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.163+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.165+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v112'448 uv448 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.165+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.280 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:28.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.166+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v112'449 uv449 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.166+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ee370 con 0x5599b23e6bd0 2023-10-06T05:24:28.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.167+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v112'450 uv450 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.167+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ee370 con 0x5599b23e6bd0 2023-10-06T05:24:28.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.168+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'451 uv451 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.168+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.170+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v112'452 uv436 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.170+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.171+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v112'453 uv453 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.171+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.284 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:28.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.172+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v112'454 uv454 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.172+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ebc60 con 0x5599b23e6bd0 2023-10-06T05:24:28.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.174+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'455 uv455 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.174+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ebc60 con 0x5599b23e6bd0 2023-10-06T05:24:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.175+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'456 uv456 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.175+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eac50 con 0x5599b23e6bd0 2023-10-06T05:24:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.176+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'457 uv457 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.176+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5599b23eb160 0x5599b23ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.176+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eda80 con 0x5599b23eb160 2023-10-06T05:24:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.176+0000 7fe8f1563700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5599b23eb160 0x5599b23ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.177+0000 7fe8f1563700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5599b23eb160 0x5599b23ed540 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.178+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.178+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ede30 con 0x5599b23eb160 2023-10-06T05:24:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.179+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v112'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.179+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ee1e0 con 0x5599b23eb160 2023-10-06T05:24:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.180+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v112'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.293 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.181+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ea100 con 0x5599b23eb160 2023-10-06T05:24:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.182+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.182+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23efc00 con 0x5599b23eb160 2023-10-06T05:24:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.183+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.183+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eeb50 con 0x5599b23eb160 2023-10-06T05:24:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.185+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.185+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ee6e0 con 0x5599b23eb160 2023-10-06T05:24:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.186+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'84 uv83 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.186+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23eb160 2023-10-06T05:24:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.187+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'85 uv85 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.187+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23eb160 2023-10-06T05:24:28.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.189+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'86 uv86 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.296 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.189+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23eb160 2023-10-06T05:24:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.190+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'87 uv87 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.190+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23e6bd0 2023-10-06T05:24:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.191+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.191+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23e6bd0 2023-10-06T05:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.193+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v112'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.193+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23e6bd0 2023-10-06T05:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.194+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v112'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.299 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:28.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.194+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23e6bd0 2023-10-06T05:24:28.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.195+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.195+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23e6bd0 2023-10-06T05:24:28.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.196+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.196+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23ef440 con 0x5599b23e6bd0 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.197+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23e6bd0 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.197+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5599b23ef440 0x5599b23f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.197+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ac0 con 0x5599b23ef440 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.198+0000 7fe8f2565700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5599b23ef440 0x5599b23f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.198+0000 7fe8f2565700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5599b23ef440 0x5599b23f2580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.199+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v112'216 uv210 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.199+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ea0 con 0x5599b23ef440 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.200+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v112'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.200+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ea0 con 0x5599b23ef440 2023-10-06T05:24:28.303 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.202+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v112'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.202+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f3eb0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.203+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'219 uv219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.203+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f3eb0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.204+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.204+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ea0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.205+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.205+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ea0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.206+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v112'222 uv215 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.206+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ea0 con 0x5599b23ef440 2023-10-06T05:24:28.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.207+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v112'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.207+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ea0 con 0x5599b23ef440 2023-10-06T05:24:28.306 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:28.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.208+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v112'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.208+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f2ea0 con 0x5599b23ef440 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.209+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'225 uv225 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.210+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23eefd0 con 0x5599b23ef440 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.211+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.211+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5599b23f38a0 0x5599b23f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.211+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f61c0 con 0x5599b23f38a0 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.211+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5599b23f38a0 0x5599b23f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.212+0000 7fe8f1d64700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5599b23f38a0 0x5599b23f5c80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.213+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v112'97 uv96 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23f38a0 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.213+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f61c0 con 0x5599b23f38a0 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.214+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v112'98 uv98 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23f38a0 2023-10-06T05:24:28.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.215+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f61c0 con 0x5599b23f38a0 2023-10-06T05:24:28.309 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.216+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v112'99 uv99 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23f38a0 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.216+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f65c0 con 0x5599b23f38a0 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.217+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'100 uv100 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23f38a0 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.217+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f65c0 con 0x5599b23f38a0 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.218+0000 7fe8f1d64700 1 -- 172.21.15.45:0/2041497191 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'101 uv101 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e800e8b0 con 0x5599b23f38a0 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.218+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f65c0 con 0x5599b23eb160 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.220+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.220+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f65c0 con 0x5599b23eb160 2023-10-06T05:24:28.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.221+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v112'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.221+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f65c0 con 0x5599b23eb160 2023-10-06T05:24:28.311 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:28.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.222+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v112'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.222+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f75d0 con 0x5599b23eb160 2023-10-06T05:24:28.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.223+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.223+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f75d0 con 0x5599b23eb160 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.224+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.224+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f65c0 con 0x5599b23eb160 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.225+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.225+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5599b23f6ad0 0x5599b23f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.225+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9520 con 0x5599b23f6ad0 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.225+0000 7fe8f1563700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5599b23f6ad0 0x5599b23f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.226+0000 7fe8f1563700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5599b23f6ad0 0x5599b23f8eb0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.227+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v112'192 uv186 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.227+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f95a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.229+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v112'193 uv193 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.229+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f99a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.314 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.230+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v112'194 uv194 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.230+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f99a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.231+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'195 uv195 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.231+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f99a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.232+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'196 uv196 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.232+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f99a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.233+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v112'197 uv191 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.233+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f99a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.234+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v112'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.234+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f99a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.235+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v112'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.316 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.235+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f99a0 con 0x5599b23f6ad0 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.237+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.237+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23fc710 con 0x5599b23f6ad0 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.238+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'201 uv201 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23f6ad0 2023-10-06T05:24:28.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.238+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.239+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v112'87 uv86 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.239+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.240+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v112'88 uv88 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.240+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.318 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:28.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.241+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v112'89 uv89 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.241+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.243+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v112'90 uv90 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.243+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5599b23fa770 0x5599b23fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.243+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23fa770 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.243+0000 7fe8f2565700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5599b23fa770 0x5599b23fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.244+0000 7fe8f2565700 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5599b23fa770 0x5599b23fce60 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.245+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23fa770 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.245+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23fa770 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.246+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v112'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23fa770 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.246+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23fa770 2023-10-06T05:24:28.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.247+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v112'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23fa770 2023-10-06T05:24:28.321 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.247+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23fa770 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.248+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23fa770 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.248+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23fa770 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.249+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23fa770 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.249+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23fa770 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.250+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23fa770 2023-10-06T05:24:28.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.251+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23ef440 2023-10-06T05:24:28.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.252+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v112'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.252+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23ef440 2023-10-06T05:24:28.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.253+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v112'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.253+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23ef440 2023-10-06T05:24:28.326 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.254+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v112'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.254+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23ef440 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.255+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v112'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.255+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23ef440 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.256+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v112'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe8ec00acc0 con 0x5599b23ef440 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.256+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.258+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v112'241 uv233 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.258+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.259+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v112'242 uv242 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.259+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.260+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v112'243 uv243 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.328 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.260+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.261+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v112'244 uv244 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.261+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.262+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v112'245 uv245 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.262+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.264+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v112'246 uv240 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.264+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.265+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v112'247 uv247 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.265+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.331 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.266+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v112'248 uv248 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.266+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.267+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v112'249 uv249 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.267+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.268+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v112'250 uv250 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.268+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23f9e90 con 0x5599b23eb160 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.269+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v112'251 uv251 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.269+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x5599b23fd3a0 con 0x5599b23eb160 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.270+0000 7fe8f1563700 1 -- 172.21.15.45:0/2041497191 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v112'252 uv252 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe8e4084220 con 0x5599b23eb160 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.270+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5599b23eb160 msgr2=0x5599b23ed540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5599b23eb160 0x5599b23ed540 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5599b2320e40 msgr2=0x5599b231f260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5599b2320e40 0x5599b231f260 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5599b23f6ad0 msgr2=0x5599b23f8eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5599b23f6ad0 0x5599b23f8eb0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5599b23e6bd0 msgr2=0x5599b23e8fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5599b23e6bd0 0x5599b23e8fb0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5599b23fa770 msgr2=0x5599b23fce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5599b23fa770 0x5599b23fce60 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5599b23f38a0 msgr2=0x5599b23f5c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5599b23f38a0 0x5599b23f5c80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 reap_dead start 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5599b23ef440 msgr2=0x5599b23f2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5599b23ef440 0x5599b23f2580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 reap_dead start 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f2565700 1 -- 172.21.15.45:0/2041497191 reap_dead start 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8d4065890 msgr2=0x7fe8d4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8d4065890 0x7fe8d4067d40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e80068d0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 msgr2=0x5599b23a83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b23a83e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4008fd0 tx=0x7fe8e401e000 comp rx=0 tx=0).stop 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 shutdown_connections 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599b233b340 0x5599b23a83e0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 --2- 172.21.15.45:0/2041497191 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8d4065890 0x7fe8d4067d40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 >> 172.21.15.45:0/2041497191 conn(0x5599b22f24a0 msgr2=0x5599b22f2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 shutdown_connections 2023-10-06T05:24:28.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.271+0000 7fe8f8b46040 1 -- 172.21.15.45:0/2041497191 wait complete. 2023-10-06T05:24:28.337 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2023-10-06T05:24:28.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.424+0000 7f9126f35040 1 Processor -- start 2023-10-06T05:24:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.424+0000 7f9126f35040 1 -- start start 2023-10-06T05:24:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9126f35040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d606d40 0x55823d692be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9126f35040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d60ac50 0x55823d6233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9126f35040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d6b3c20 0x55823d5dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9126f35040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55823d526620 con 0x55823d606d40 2023-10-06T05:24:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9126f35040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55823d51d400 con 0x55823d60ac50 2023-10-06T05:24:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9126f35040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55823d5de4b0 con 0x55823d6b3c20 2023-10-06T05:24:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d606d40 0x55823d692be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d606d40 0x55823d692be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58362/0 (socket says 172.21.15.45:58362) 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 -- 172.21.15.45:0/1860562949 learned_addr learned my addr 172.21.15.45:0/1860562949 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120954700 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d6b3c20 0x55823d5dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f911f952700 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d60ac50 0x55823d6233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 -- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d6b3c20 msgr2=0x55823d5dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d6b3c20 0x55823d5dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 -- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d60ac50 msgr2=0x55823d6233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d60ac50 0x55823d6233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 -- 172.21.15.45:0/1860562949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9114005db0 con 0x55823d606d40 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120954700 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d6b3c20 0x55823d5dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f9120153700 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d606d40 0x55823d692be0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9110009980 tx=0x7f911000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15678f1e2e64395 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f911e950700 1 -- 172.21.15.45:0/1860562949 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000f040 con 0x55823d606d40 2023-10-06T05:24:28.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f911e950700 1 -- 172.21.15.45:0/1860562949 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9110008ac0 con 0x55823d606d40 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.425+0000 7f911e950700 1 -- 172.21.15.45:0/1860562949 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110008e60 con 0x55823d606d40 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 -- 172.21.15.45:0/1860562949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d606d40 msgr2=0x55823d692be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d606d40 0x55823d692be0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9110009980 tx=0x7f911000c040 comp rx=0 tx=0).stop 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 -- 172.21.15.45:0/1860562949 shutdown_connections 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d6b3c20 0x55823d5dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d606d40 0x55823d692be0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 --2- 172.21.15.45:0/1860562949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d60ac50 0x55823d6233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 -- 172.21.15.45:0/1860562949 >> 172.21.15.45:0/1860562949 conn(0x55823d5dbbd0 msgr2=0x55823d698910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 -- 172.21.15.45:0/1860562949 shutdown_connections 2023-10-06T05:24:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.426+0000 7f9126f35040 1 -- 172.21.15.45:0/1860562949 wait complete. 2023-10-06T05:24:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.427+0000 7f9126f35040 1 Processor -- start 2023-10-06T05:24:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.427+0000 7f9126f35040 1 -- start start 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9126f35040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d606d40 0x55823d698850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9126f35040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d60ac50 0x55823d5dcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9126f35040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d6b3c20 0x55823d68f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9126f35040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55823d697930 con 0x55823d6b3c20 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9126f35040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55823d697bd0 con 0x55823d606d40 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9126f35040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55823d697e70 con 0x55823d60ac50 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9120954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d6b3c20 0x55823d68f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9120954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d6b3c20 0x55823d68f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58378/0 (socket says 172.21.15.45:58378) 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.428+0000 7f9120954700 1 -- 172.21.15.45:0/2644900198 learned_addr learned my addr 172.21.15.45:0/2644900198 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120153700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d606d40 0x55823d698850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f911f952700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d60ac50 0x55823d5dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120954700 1 -- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d60ac50 msgr2=0x55823d5dcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120954700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d60ac50 0x55823d5dcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120954700 1 -- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d606d40 msgr2=0x55823d698850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120954700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d606d40 0x55823d698850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120954700 1 -- 172.21.15.45:0/2644900198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110009580 con 0x55823d6b3c20 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f911f952700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55823d60ac50 0x55823d5dcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120153700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55823d606d40 0x55823d698850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9120954700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d6b3c20 0x55823d68f2e0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f911800f980 tx=0x7f9118007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f911d14d700 1 -- 172.21.15.45:0/2644900198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800d3f0 con 0x55823d6b3c20 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f911d14d700 1 -- 172.21.15.45:0/2644900198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911800b640 con 0x55823d6b3c20 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55823d54f1c0 con 0x55823d6b3c20 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f911d14d700 1 -- 172.21.15.45:0/2644900198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800b9e0 con 0x55823d6b3c20 2023-10-06T05:24:28.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.429+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55823d5353d0 con 0x55823d6b3c20 2023-10-06T05:24:28.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.431+0000 7f911d14d700 1 -- 172.21.15.45:0/2644900198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9118013430 con 0x55823d6b3c20 2023-10-06T05:24:28.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.431+0000 7f911d14d700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f910c065890 0x7f910c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.431+0000 7f911d14d700 1 -- 172.21.15.45:0/2644900198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f9118084070 con 0x55823d6b3c20 2023-10-06T05:24:28.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.431+0000 7f9120153700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f910c065890 0x7f910c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.432+0000 7f9126f35040 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55823d5dd440 0x55823d6c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.432+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v8 -- 0x55823d68f820 con 0x55823d5dd440 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.432+0000 7f911f952700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55823d5dd440 0x55823d6c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.432+0000 7f9120153700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f910c065890 0x7f910c067d40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f91100097f0 tx=0x7f9110018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.432+0000 7f911f952700 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55823d5dd440 0x55823d6c11d0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.433+0000 7f911f952700 1 -- 172.21.15.45:0/2644900198 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v112'202 uv202 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9114008970 con 0x55823d5dd440 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55823d5dd440 msgr2=0x55823d6c11d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55823d5dd440 0x55823d6c11d0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f910c065890 msgr2=0x7f910c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f910c065890 0x7f910c067d40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f91100097f0 tx=0x7f9110018000 comp rx=0 tx=0).stop 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d6b3c20 msgr2=0x55823d68f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 --2- 172.21.15.45:0/2644900198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55823d6b3c20 0x55823d68f2e0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f911800f980 tx=0x7f9118007040 comp rx=0 tx=0).stop 2023-10-06T05:24:28.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9120954700 1 -- 172.21.15.45:0/2644900198 reap_dead start 2023-10-06T05:24:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 shutdown_connections 2023-10-06T05:24:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 >> 172.21.15.45:0/2644900198 conn(0x55823d5dbbd0 msgr2=0x55823d6b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 shutdown_connections 2023-10-06T05:24:28.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.434+0000 7f9126f35040 1 -- 172.21.15.45:0/2644900198 wait complete. 2023-10-06T05:24:28.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.620+0000 7f19c7cd7700 1 Processor -- start 2023-10-06T05:24:28.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.620+0000 7f19c7cd7700 1 -- start start 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c7cd7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f19c0157330 0x7f19c00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c7cd7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19c00ccb70 0x7f19c00c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c7cd7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c00c3b50 con 0x7f19c00aaae0 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c7cd7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c0157700 con 0x7f19c00ccb70 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c7cd7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c00c98e0 con 0x7f19c0157330 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c6274700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19c00ccb70 0x7f19c00c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c6274700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19c00ccb70 0x7f19c00c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58394/0 (socket says 172.21.15.45:58394) 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c6274700 1 -- 172.21.15.45:0/4099737117 learned_addr learned my addr 172.21.15.45:0/4099737117 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.621+0000 7f19c5a73700 1 --2- 172.21.15.45:0/4099737117 >> v2:172.21.15.144:3300/0 conn(0x7f19c0157330 0x7f19c00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c6274700 1 -- 172.21.15.45:0/4099737117 >> v2:172.21.15.144:3300/0 conn(0x7f19c0157330 msgr2=0x7f19c00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c6274700 1 --2- 172.21.15.45:0/4099737117 >> v2:172.21.15.144:3300/0 conn(0x7f19c0157330 0x7f19c00ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c6274700 1 -- 172.21.15.45:0/4099737117 >> v1:172.21.15.144:6789/0 conn(0x7f19c00aaae0 legacy=0x7f19c00c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:28.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c6274700 1 -- 172.21.15.45:0/4099737117 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00c9bc0 con 0x7f19c00ccb70 2023-10-06T05:24:28.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c6274700 1 --2- 172.21.15.45:0/4099737117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00c9280 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f19c00cd1d0 tx=0x7f19b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4175bbda2b2a24c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c4a71700 1 -- 172.21.15.45:0/4099737117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800e070 con 0x7f19c00ccb70 2023-10-06T05:24:28.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c4a71700 1 -- 172.21.15.45:0/4099737117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800ba10 con 0x7f19c00ccb70 2023-10-06T05:24:28.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c4a71700 1 -- 172.21.15.45:0/4099737117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80092e0 con 0x7f19c00ccb70 2023-10-06T05:24:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c7cd7700 1 -- 172.21.15.45:0/4099737117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 msgr2=0x7f19c00c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.622+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/4099737117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00c9280 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f19c00cd1d0 tx=0x7f19b8009840 comp rx=0 tx=0).stop 2023-10-06T05:24:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.623+0000 7f19c7cd7700 1 -- 172.21.15.45:0/4099737117 shutdown_connections 2023-10-06T05:24:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.623+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/4099737117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00c9280 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.623+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/4099737117 >> v2:172.21.15.144:3300/0 conn(0x7f19c0157330 0x7f19c00ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.623+0000 7f19c7cd7700 1 -- 172.21.15.45:0/4099737117 >> 172.21.15.45:0/4099737117 conn(0x7f19c0067e00 msgr2=0x7f19c00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.623+0000 7f19c7cd7700 1 -- 172.21.15.45:0/4099737117 shutdown_connections 2023-10-06T05:24:28.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.623+0000 7f19c7cd7700 1 -- 172.21.15.45:0/4099737117 wait complete. 2023-10-06T05:24:28.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.624+0000 7f19c7cd7700 1 Processor -- start 2023-10-06T05:24:28.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.624+0000 7f19c7cd7700 1 -- start start 2023-10-06T05:24:28.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c7cd7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19c00aaae0 0x7f19c01529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c7cd7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c7cd7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19c0157330 0x7f19c00ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c7cd7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0152f20 con 0x7f19c00ccb70 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c7cd7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00b0dc0 con 0x7f19c0157330 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c7cd7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00c7630 con 0x7f19c00aaae0 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58404/0 (socket says 172.21.15.45:58404) 2023-10-06T05:24:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 -- 172.21.15.45:0/1875165276 learned_addr learned my addr 172.21.15.45:0/1875165276 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5a73700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19c00aaae0 0x7f19c01529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c6274700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19c0157330 0x7f19c00ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 -- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19c00aaae0 msgr2=0x7f19c01529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19c00aaae0 0x7f19c01529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 -- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19c0157330 msgr2=0x7f19c00ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19c0157330 0x7f19c00ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 -- 172.21.15.45:0/1875165276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8009ae0 con 0x7f19c00ccb70 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.625+0000 7f19c5272700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00ac430 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f19b000b5e0 tx=0x7f19b0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.626+0000 7f19b5ffb700 1 -- 172.21.15.45:0/1875165276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000f510 con 0x7f19c00ccb70 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.626+0000 7f19b5ffb700 1 -- 172.21.15.45:0/1875165276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b000f670 con 0x7f19c00ccb70 2023-10-06T05:24:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.626+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c00c7910 con 0x7f19c00ccb70 2023-10-06T05:24:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.626+0000 7f19b5ffb700 1 -- 172.21.15.45:0/1875165276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000fa30 con 0x7f19c00ccb70 2023-10-06T05:24:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.626+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c00c7ed0 con 0x7f19c00ccb70 2023-10-06T05:24:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.627+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c006e350 con 0x7f19c00ccb70 2023-10-06T05:24:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.627+0000 7f19b5ffb700 1 -- 172.21.15.45:0/1875165276 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19b001a400 con 0x7f19c00ccb70 2023-10-06T05:24:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.628+0000 7f19b5ffb700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f19a4066f30 0x7f19a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.628+0000 7f19b5ffb700 1 -- 172.21.15.45:0/1875165276 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f19b0084a00 con 0x7f19c00ccb70 2023-10-06T05:24:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.628+0000 7f19c5a73700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f19a4066f30 0x7f19a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.628+0000 7f19c5a73700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f19a4066f30 0x7f19a40693e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006aa0 tx=0x7f19bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.632+0000 7f19b5ffb700 1 -- 172.21.15.45:0/1875165276 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19b00512f0 con 0x7f19c00ccb70 2023-10-06T05:24:28.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:28.884+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f19c00c37e0 con 0x7f19c00ccb70 2023-10-06T05:24:29.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.046+0000 7f19b5ffb700 1 -- 172.21.15.45:0/1875165276 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v218) v1 ==== 162+0+0 (secure 0 0 0) 0x7f19b0054910 con 0x7f19c00ccb70 2023-10-06T05:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f19a4066f30 msgr2=0x7f19a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f19a4066f30 0x7f19a40693e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006aa0 tx=0x7f19bc008040 comp rx=0 tx=0).stop 2023-10-06T05:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 msgr2=0x7f19c00ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00ac430 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f19b000b5e0 tx=0x7f19b0009120 comp rx=0 tx=0).stop 2023-10-06T05:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 shutdown_connections 2023-10-06T05:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f19a4066f30 0x7f19a40693e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19c0157330 0x7f19c00ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19c00aaae0 0x7f19c01529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 --2- 172.21.15.45:0/1875165276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c00ccb70 0x7f19c00ac430 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 >> 172.21.15.45:0/1875165276 conn(0x7f19c0067e00 msgr2=0x7f19c00ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 shutdown_connections 2023-10-06T05:24:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.049+0000 7f19c7cd7700 1 -- 172.21.15.45:0/1875165276 wait complete. 2023-10-06T05:24:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:29.056 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:29.054+0000 7f033a022700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input 2023-10-06T05:24:29.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:29.055+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:29.057 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:29.055+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:29.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.204+0000 7fd48be69700 1 Processor -- start 2023-10-06T05:24:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.204+0000 7fd48be69700 1 -- start start 2023-10-06T05:24:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd48be69700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4840c8ff0 0x7fd4840c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd48be69700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd484157200 0x7fd4840caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd48be69700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48415b000 con 0x7fd4840caad0 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd48be69700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48415b2e0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd48be69700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48415b5c0 con 0x7fd484157200 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd489c05700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4840c8ff0 0x7fd4840c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd489c05700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4840c8ff0 0x7fd4840c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58410/0 (socket says 172.21.15.45:58410) 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd489c05700 1 -- 172.21.15.45:0/3797885005 learned_addr learned my addr 172.21.15.45:0/3797885005 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd48a406700 1 --2- 172.21.15.45:0/3797885005 >> v2:172.21.15.144:3300/0 conn(0x7fd484157200 0x7fd4840caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd489c05700 1 -- 172.21.15.45:0/3797885005 >> v2:172.21.15.144:3300/0 conn(0x7fd484157200 msgr2=0x7fd4840caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd489c05700 1 --2- 172.21.15.45:0/3797885005 >> v2:172.21.15.144:3300/0 conn(0x7fd484157200 0x7fd4840caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd489c05700 1 -- 172.21.15.45:0/3797885005 >> v1:172.21.15.144:6789/0 conn(0x7fd4840caad0 legacy=0x7fd484156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.205+0000 7fd489c05700 1 -- 172.21.15.45:0/3797885005 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48415b8a0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd489c05700 1 --2- 172.21.15.45:0/3797885005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd4840c93c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd48415abf0 tx=0x7fd480009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc44a3f8cbc96b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd488c03700 1 -- 172.21.15.45:0/3797885005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000b860 con 0x7fd4840c8ff0 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd488c03700 1 -- 172.21.15.45:0/3797885005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48000b9c0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd488c03700 1 -- 172.21.15.45:0/3797885005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000bd60 con 0x7fd4840c8ff0 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd48be69700 1 -- 172.21.15.45:0/3797885005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 msgr2=0x7fd4840c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd48be69700 1 --2- 172.21.15.45:0/3797885005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd4840c93c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd48415abf0 tx=0x7fd480009250 comp rx=0 tx=0).stop 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd48be69700 1 -- 172.21.15.45:0/3797885005 shutdown_connections 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd48be69700 1 --2- 172.21.15.45:0/3797885005 >> v2:172.21.15.144:3300/0 conn(0x7fd484157200 0x7fd4840caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.206+0000 7fd48be69700 1 --2- 172.21.15.45:0/3797885005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd4840c93c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.207+0000 7fd48be69700 1 -- 172.21.15.45:0/3797885005 >> 172.21.15.45:0/3797885005 conn(0x7fd4840cdf50 msgr2=0x7fd4840ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.207+0000 7fd48be69700 1 -- 172.21.15.45:0/3797885005 shutdown_connections 2023-10-06T05:24:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.207+0000 7fd48be69700 1 -- 172.21.15.45:0/3797885005 wait complete. 2023-10-06T05:24:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.208+0000 7fd48be69700 1 Processor -- start 2023-10-06T05:24:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.208+0000 7fd48be69700 1 -- start start 2023-10-06T05:24:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48be69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd484159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48be69700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4840caad0 0x7fd4841511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48be69700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd484157200 0x7fd4840af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48be69700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd484159540 con 0x7fd4840c8ff0 2023-10-06T05:24:29.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48be69700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4840b1060 con 0x7fd484157200 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48be69700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48416ca90 con 0x7fd4840caad0 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd484159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd484159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58422/0 (socket says 172.21.15.45:58422) 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 -- 172.21.15.45:0/1980927671 learned_addr learned my addr 172.21.15.45:0/1980927671 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48a406700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd484157200 0x7fd4840af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489404700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4840caad0 0x7fd4841511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 -- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4840caad0 msgr2=0x7fd4841511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4840caad0 0x7fd4841511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 -- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd484157200 msgr2=0x7fd4840af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd484157200 0x7fd4840af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 -- 172.21.15.45:0/1980927671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800094f0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd48a406700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd484157200 0x7fd4840af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.209+0000 7fd489c05700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd484159000 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fd4840c4f80 tx=0x7fd480017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.210+0000 7fd479ffb700 1 -- 172.21.15.45:0/1980927671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480011e60 con 0x7fd4840c8ff0 2023-10-06T05:24:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.210+0000 7fd479ffb700 1 -- 172.21.15.45:0/1980927671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48001a630 con 0x7fd4840c8ff0 2023-10-06T05:24:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.210+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48416d330 con 0x7fd4840c8ff0 2023-10-06T05:24:29.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.210+0000 7fd479ffb700 1 -- 172.21.15.45:0/1980927671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48001a9f0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.210+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48416cce0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.211+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48406f410 con 0x7fd4840c8ff0 2023-10-06T05:24:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.211+0000 7fd479ffb700 1 -- 172.21.15.45:0/1980927671 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd48002d440 con 0x7fd4840c8ff0 2023-10-06T05:24:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.212+0000 7fd479ffb700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.212+0000 7fd479ffb700 1 -- 172.21.15.45:0/1980927671 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fd480086ad0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.212+0000 7fd489404700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.213+0000 7fd489404700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd484159780 tx=0x7fd474009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.216+0000 7fd479ffb700 1 -- 172.21.15.45:0/1980927671 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4800532f0 con 0x7fd4840c8ff0 2023-10-06T05:24:29.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.463+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd48404bb40 con 0x7fd4840c8ff0 2023-10-06T05:24:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.463+0000 7fd479ffb700 1 -- 172.21.15.45:0/1980927671 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) v1 ==== 95+0+3841 (secure 0 0 0) 0x7fd480015090 con 0x7fd4840c8ff0 2023-10-06T05:24:29.465 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:29.465 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":220,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:29.056237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.465+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 msgr2=0x7fd4680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.465+0000 7fd48be69700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd484159780 tx=0x7fd474009040 comp rx=0 tx=0).stop 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.465+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 msgr2=0x7fd484159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.465+0000 7fd48be69700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd484159000 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fd4840c4f80 tx=0x7fd480017000 comp rx=0 tx=0).stop 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.465+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 shutdown_connections 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.465+0000 7fd48be69700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.465+0000 7fd48be69700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd484157200 0x7fd4840af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.466+0000 7fd48be69700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4840c8ff0 0x7fd484159000 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.466+0000 7fd48be69700 1 --2- 172.21.15.45:0/1980927671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4840caad0 0x7fd4841511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.466+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 >> 172.21.15.45:0/1980927671 conn(0x7fd4840cdf50 msgr2=0x7fd4840c2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.466+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 shutdown_connections 2023-10-06T05:24:29.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.466+0000 7fd48be69700 1 -- 172.21.15.45:0/1980927671 wait complete. 2023-10-06T05:24:29.469 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 220 2023-10-06T05:24:29.481 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:29.481 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:29.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.613+0000 7f0501483700 1 Processor -- start 2023-10-06T05:24:29.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.613+0000 7f0501483700 1 -- start start 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.613+0000 7f0501483700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f04fc153650 0x7f04fc15a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.613+0000 7f0501483700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f04fc153a20 0x7f04fc06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.613+0000 7f0501483700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.613+0000 7f0501483700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc06b270 con 0x7f04fc153a20 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f0501483700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc0676c0 con 0x7f04fc153650 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f0501483700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0679a0 con 0x7f04fc06add0 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f04fc153a20 0x7f04fc06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f04fc153a20 0x7f04fc06a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58424/0 (socket says 172.21.15.45:58424) 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 -- 172.21.15.45:0/1301884255 learned_addr learned my addr 172.21.15.45:0/1301884255 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04faffd700 1 --2- 172.21.15.45:0/1301884255 >> v2:172.21.15.144:3300/0 conn(0x7f04fc153650 0x7f04fc15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fb7fe700 1 --2- 172.21.15.45:0/1301884255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 -- 172.21.15.45:0/1301884255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 msgr2=0x7f04fc068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 --2- 172.21.15.45:0/1301884255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 -- 172.21.15.45:0/1301884255 >> v2:172.21.15.144:3300/0 conn(0x7f04fc153650 msgr2=0x7f04fc15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 --2- 172.21.15.45:0/1301884255 >> v2:172.21.15.144:3300/0 conn(0x7f04fc153650 0x7f04fc15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 -- 172.21.15.45:0/1301884255 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc067c80 con 0x7f04fc153a20 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04fa7fc700 1 --2- 172.21.15.45:0/1301884255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc06a890 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f04fc06f7f0 tx=0x7f04f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6f4a0fab1404a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.614+0000 7f04f9ffb700 1 -- 172.21.15.45:0/1301884255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000b640 con 0x7f04fc153a20 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f04f9ffb700 1 -- 172.21.15.45:0/1301884255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f000b7a0 con 0x7f04fc153a20 2023-10-06T05:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f04f9ffb700 1 -- 172.21.15.45:0/1301884255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000bb60 con 0x7f04fc153a20 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 -- 172.21.15.45:0/1301884255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 msgr2=0x7f04fc06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 --2- 172.21.15.45:0/1301884255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc06a890 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f04fc06f7f0 tx=0x7f04f0009250 comp rx=0 tx=0).stop 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 -- 172.21.15.45:0/1301884255 shutdown_connections 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 --2- 172.21.15.45:0/1301884255 >> v2:172.21.15.144:3300/0 conn(0x7f04fc153650 0x7f04fc15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 --2- 172.21.15.45:0/1301884255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 --2- 172.21.15.45:0/1301884255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc06a890 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 -- 172.21.15.45:0/1301884255 >> 172.21.15.45:0/1301884255 conn(0x7f04fc0cc5a0 msgr2=0x7f04fc0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 -- 172.21.15.45:0/1301884255 shutdown_connections 2023-10-06T05:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.615+0000 7f0501483700 1 -- 172.21.15.45:0/1301884255 wait complete. 2023-10-06T05:24:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 Processor -- start 2023-10-06T05:24:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 -- start start 2023-10-06T05:24:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04fc153650 0x7f04fc0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc16ca00 con 0x7f04fc153a20 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc16d860 con 0x7f04fc153650 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.617+0000 7f0501483700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc16cc50 con 0x7f04fc06add0 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc0b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc0b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58438/0 (socket says 172.21.15.45:58438) 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 -- 172.21.15.45:0/3635904203 learned_addr learned my addr 172.21.15.45:0/3635904203 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fa7fc700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04fc153650 0x7f04fc0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04faffd700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 -- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 msgr2=0x7f04fc151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 -- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04fc153650 msgr2=0x7f04fc0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04fc153650 0x7f04fc0aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 -- 172.21.15.45:0/3635904203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00094f0 con 0x7f04fc153a20 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04fb7fe700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc0b0440 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009e90 tx=0x7f04f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04e2ffd700 1 -- 172.21.15.45:0/3635904203 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400cb70 con 0x7f04fc153a20 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04e2ffd700 1 -- 172.21.15.45:0/3635904203 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f400f040 con 0x7f04fc153a20 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f04e2ffd700 1 -- 172.21.15.45:0/3635904203 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4014480 con 0x7f04fc153a20 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.618+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc16cf30 con 0x7f04fc153a20 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.619+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc16d4f0 con 0x7f04fc153a20 2023-10-06T05:24:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.619+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc0cc0e0 con 0x7f04fc153a20 2023-10-06T05:24:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.620+0000 7f04e2ffd700 1 -- 172.21.15.45:0/3635904203 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04f4014830 con 0x7f04fc153a20 2023-10-06T05:24:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.620+0000 7f04e2ffd700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f04e8066f30 0x7f04e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.620+0000 7f04e2ffd700 1 -- 172.21.15.45:0/3635904203 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f04f40858a0 con 0x7f04fc153a20 2023-10-06T05:24:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.621+0000 7f04faffd700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f04e8066f30 0x7f04e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.621+0000 7f04faffd700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f04e8066f30 0x7f04e80693e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009d20 tx=0x7f04ec0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.625+0000 7f04e2ffd700 1 -- 172.21.15.45:0/3635904203 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04f4052120 con 0x7f04fc153a20 2023-10-06T05:24:29.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.874+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04fc067f60 con 0x7f04fc153a20 2023-10-06T05:24:29.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.875+0000 7f04e2ffd700 1 -- 172.21.15.45:0/3635904203 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f04f4018090 con 0x7f04fc153a20 2023-10-06T05:24:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:29.877 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":220,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:29.056237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:29.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f04e8066f30 msgr2=0x7f04e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f04e8066f30 0x7f04e80693e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009d20 tx=0x7f04ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 msgr2=0x7f04fc0b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc0b0440 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009e90 tx=0x7f04f400b040 comp rx=0 tx=0).stop 2023-10-06T05:24:29.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 shutdown_connections 2023-10-06T05:24:29.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f04e8066f30 0x7f04e80693e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04fc153a20 0x7f04fc0b0440 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04fc06add0 0x7f04fc151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 --2- 172.21.15.45:0/3635904203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04fc153650 0x7f04fc0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 >> 172.21.15.45:0/3635904203 conn(0x7f04fc0cc5a0 msgr2=0x7f04fc0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 shutdown_connections 2023-10-06T05:24:29.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:29.877+0000 7f0501483700 1 -- 172.21.15.45:0/3635904203 wait complete. 2023-10-06T05:24:29.883 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 220 2023-10-06T05:24:29.892 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:29.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.022+0000 7f3eb6792700 1 Processor -- start 2023-10-06T05:24:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.022+0000 7f3eb6792700 1 -- start start 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eb6792700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3eb00cd340 0x7f3eb0157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eb6792700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb015b000 con 0x7f3eb00c93c0 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eb6792700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb015b2e0 con 0x7f3eb00c8ff0 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eb6792700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb00c9830 con 0x7f3eb00cd340 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eaf7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3eb00c93c0 0x7f3eb0156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41548/0 (socket says 172.21.15.45:41548) 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eaf7fe700 1 -- 172.21.15.45:0/1797900353 learned_addr learned my addr 172.21.15.45:0/1797900353 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eb4d2f700 1 --2- 172.21.15.45:0/1797900353 >> v2:172.21.15.144:3300/0 conn(0x7f3eb00cd340 0x7f3eb0157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eaeffd700 1 -- 172.21.15.45:0/1797900353 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702989129 0 0) 0x7f3eb015b000 con 0x7f3eb00c93c0 2023-10-06T05:24:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.023+0000 7f3eaeffd700 1 -- 172.21.15.45:0/1797900353 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e9c003570 con 0x7f3eb00c93c0 2023-10-06T05:24:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eb4d2f700 1 -- 172.21.15.45:0/1797900353 >> v1:172.21.15.144:6789/0 conn(0x7f3eb00c8ff0 legacy=0x7f3eb00caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eb4d2f700 1 -- 172.21.15.45:0/1797900353 >> v1:172.21.15.45:6789/0 conn(0x7f3eb00c93c0 legacy=0x7f3eb0156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eaeffd700 1 -- 172.21.15.45:0/1797900353 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2983455932 0 0) 0x7f3e9c003570 con 0x7f3eb00c93c0 2023-10-06T05:24:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eb4d2f700 1 -- 172.21.15.45:0/1797900353 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb015b5c0 con 0x7f3eb00cd340 2023-10-06T05:24:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eb4d2f700 1 --2- 172.21.15.45:0/1797900353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 0x7f3eb0157200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009e60 tx=0x7f3ea8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb381db8757161c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eaeffd700 1 -- 172.21.15.45:0/1797900353 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800e070 con 0x7f3eb00cd340 2023-10-06T05:24:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eaeffd700 1 -- 172.21.15.45:0/1797900353 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea800b850 con 0x7f3eb00cd340 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eaeffd700 1 -- 172.21.15.45:0/1797900353 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98001470 con 0x7f3eb00cd340 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.024+0000 7f3eb6792700 1 -- 172.21.15.45:0/1797900353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 msgr2=0x7f3eb0157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.025+0000 7f3eb6792700 1 --2- 172.21.15.45:0/1797900353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 0x7f3eb0157200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009e60 tx=0x7f3ea8009780 comp rx=0 tx=0).stop 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.025+0000 7f3eb6792700 1 -- 172.21.15.45:0/1797900353 shutdown_connections 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.025+0000 7f3eb6792700 1 --2- 172.21.15.45:0/1797900353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 0x7f3eb0157200 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.025+0000 7f3eb6792700 1 -- 172.21.15.45:0/1797900353 >> 172.21.15.45:0/1797900353 conn(0x7f3eb00cdf50 msgr2=0x7f3eb00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.025+0000 7f3eb6792700 1 -- 172.21.15.45:0/1797900353 shutdown_connections 2023-10-06T05:24:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.025+0000 7f3eb6792700 1 -- 172.21.15.45:0/1797900353 wait complete. 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.026+0000 7f3eb6792700 1 Processor -- start 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.026+0000 7f3eb6792700 1 -- start start 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eb6792700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3eb00c8ff0 0x7f3eb006a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eb6792700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb00c93c0 0x7f3eb00ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eb6792700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 0x7f3eb0169330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eb6792700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c5000 con 0x7f3eb00c93c0 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eb6792700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c52a0 con 0x7f3eb00cd340 2023-10-06T05:24:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eb6792700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0169870 con 0x7f3eb00c8ff0 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb00c93c0 0x7f3eb00ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb00c93c0 0x7f3eb00ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58450/0 (socket says 172.21.15.45:58450) 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 -- 172.21.15.45:0/4277612043 learned_addr learned my addr 172.21.15.45:0/4277612043 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eb4d2f700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 0x7f3eb0169330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaffff700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3eb00c8ff0 0x7f3eb006a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 -- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3eb00c8ff0 msgr2=0x7f3eb006a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3eb00c8ff0 0x7f3eb006a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 -- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 msgr2=0x7f3eb0169330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 0x7f3eb0169330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaf7fe700 1 -- 172.21.15.45:0/4277612043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8009a20 con 0x7f3eb00c93c0 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.027+0000 7f3eaffff700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3eb00c8ff0 0x7f3eb006a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.028+0000 7f3eaf7fe700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb00c93c0 0x7f3eb00ad4c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800cb90 tx=0x7f3e98001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.028+0000 7f3ea7fff700 1 -- 172.21.15.45:0/4277612043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98001d60 con 0x7f3eb00c93c0 2023-10-06T05:24:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.028+0000 7f3ea7fff700 1 -- 172.21.15.45:0/4277612043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98007360 con 0x7f3eb00c93c0 2023-10-06T05:24:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.028+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb0169b40 con 0x7f3eb00c93c0 2023-10-06T05:24:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.028+0000 7f3ea7fff700 1 -- 172.21.15.45:0/4277612043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98007700 con 0x7f3eb00c93c0 2023-10-06T05:24:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.028+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb016a100 con 0x7f3eb00c93c0 2023-10-06T05:24:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.029+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb006f410 con 0x7f3eb00c93c0 2023-10-06T05:24:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.029+0000 7f3ea7fff700 1 -- 172.21.15.45:0/4277612043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3e98004a40 con 0x7f3eb00c93c0 2023-10-06T05:24:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.030+0000 7f3ea7fff700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3e90066f30 0x7f3e900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.030+0000 7f3ea7fff700 1 -- 172.21.15.45:0/4277612043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f3e980846a0 con 0x7f3eb00c93c0 2023-10-06T05:24:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.030+0000 7f3eaffff700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3e90066f30 0x7f3e900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.030+0000 7f3eaffff700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3e90066f30 0x7f3e900693e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0007fd0 tx=0x7f3ea0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.034+0000 7f3ea7fff700 1 -- 172.21.15.45:0/4277612043 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e98050fa0 con 0x7f3eb00c93c0 2023-10-06T05:24:30.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.281+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eb00c53e0 con 0x7f3eb00c93c0 2023-10-06T05:24:30.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.281+0000 7f3ea7fff700 1 -- 172.21.15.45:0/4277612043 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) v1 ==== 95+0+3841 (secure 0 0 0) 0x7f3e98018090 con 0x7f3eb00c93c0 2023-10-06T05:24:30.283 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:30.283 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":220,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3433217004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3433217004},{"type":"v1","addr":"172.21.15.144:6835","nonce":3433217004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:29.056237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.283+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3e90066f30 msgr2=0x7f3e900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.283+0000 7f3eb6792700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3e90066f30 0x7f3e900693e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0007fd0 tx=0x7f3ea0009040 comp rx=0 tx=0).stop 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.283+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb00c93c0 msgr2=0x7f3eb00ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.283+0000 7f3eb6792700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb00c93c0 0x7f3eb00ad4c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800cb90 tx=0x7f3e98001ad0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 shutdown_connections 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3e90066f30 0x7f3e900693e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00cd340 0x7f3eb0169330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3eb00c8ff0 0x7f3eb006a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 --2- 172.21.15.45:0/4277612043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eb00c93c0 0x7f3eb00ad4c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 >> 172.21.15.45:0/4277612043 conn(0x7f3eb00cdf50 msgr2=0x7f3eb006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 shutdown_connections 2023-10-06T05:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.284+0000 7f3eb6792700 1 -- 172.21.15.45:0/4277612043 wait complete. 2023-10-06T05:24:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 220 2023-10-06T05:24:30.299 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2023-10-06T05:24:30.299 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2023-10-06T05:24:30.299 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:30.299 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:30.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:30.300 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:30.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:30.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:30.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.494+0000 7f5ccf023700 1 Processor -- start 2023-10-06T05:24:30.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.494+0000 7f5ccf023700 1 -- start start 2023-10-06T05:24:30.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.495+0000 7f5ccf023700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5cc80ccaa0 0x7f5cc806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.495+0000 7f5ccf023700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80cb530 con 0x7f5cc8159330 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.495+0000 7f5ccf023700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc8069bd0 con 0x7f5cc8158f60 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.495+0000 7f5ccf023700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc8068fc0 con 0x7f5cc80ccaa0 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5cc8159330 0x7f5cc80cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41560/0 (socket says 172.21.15.45:41560) 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc7fff700 1 -- 172.21.15.45:0/1574800867 learned_addr learned my addr 172.21.15.45:0/1574800867 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5ccd5c0700 1 --2- 172.21.15.45:0/1574800867 >> v2:172.21.15.144:3300/0 conn(0x7f5cc80ccaa0 0x7f5cc806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339683523 0 0) 0x7f5cb4001110 con 0x7f5cc8159330 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb8003570 con 0x7f5cc8159330 2023-10-06T05:24:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135644030 0 0) 0x7f5cc8069bd0 con 0x7f5cc8158f60 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb4001110 con 0x7f5cc8158f60 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157785017 0 0) 0x7f5cb8003570 con 0x7f5cc8159330 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 >> v2:172.21.15.144:3300/0 conn(0x7f5cc80ccaa0 msgr2=0x7f5cc806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 --2- 172.21.15.45:0/1574800867 >> v2:172.21.15.144:3300/0 conn(0x7f5cc80ccaa0 0x7f5cc806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 >> v1:172.21.15.144:6789/0 conn(0x7f5cc8158f60 legacy=0x7f5cc80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80692a0 con 0x7f5cc8159330 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f5cb4001600 con 0x7f5cc8159330 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 >> v1:172.21.15.45:6789/0 conn(0x7f5cc8159330 legacy=0x7f5cc80cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb8003c40 0x7f5cb8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cb8004810 0x7f5cb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8005210 0x7f5cb800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8068fc0 con 0x7f5cb8003c40 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb800c1b0 con 0x7f5cb8005210 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb800c370 con 0x7f5cb8004810 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.496+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb8003c40 0x7f5cb8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cc7fff700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cb8004810 0x7f5cb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5ccd5c0700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8005210 0x7f5cb800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cccdbf700 1 -- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cb8004810 msgr2=0x7f5cb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cb8004810 0x7f5cb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cccdbf700 1 -- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8005210 msgr2=0x7f5cb800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8005210 0x7f5cb800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cccdbf700 1 -- 172.21.15.45:0/1574800867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8003570 con 0x7f5cb8003c40 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5ccd5c0700 1 -- 172.21.15.45:0/1574800867 reap_dead start 2023-10-06T05:24:30.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb8003c40 0x7f5cb8004030 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001a20 tx=0x7f5cbc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcc86c0ba16dac8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00c590 con 0x7f5cb8003c40 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc00c6f0 con 0x7f5cb8003c40 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5cc77fe700 1 -- 172.21.15.45:0/1574800867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00ca90 con 0x7f5cb8003c40 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5ccf023700 1 -- 172.21.15.45:0/1574800867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb8003c40 msgr2=0x7f5cb8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.497+0000 7f5ccf023700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb8003c40 0x7f5cb8004030 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001a20 tx=0x7f5cbc007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.498+0000 7f5ccf023700 1 -- 172.21.15.45:0/1574800867 shutdown_connections 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.498+0000 7f5ccf023700 1 --2- 172.21.15.45:0/1574800867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cb8003c40 0x7f5cb8004030 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.498+0000 7f5ccf023700 1 -- 172.21.15.45:0/1574800867 >> 172.21.15.45:0/1574800867 conn(0x7f5cc80cdf50 msgr2=0x7f5cc80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.498+0000 7f5ccf023700 1 -- 172.21.15.45:0/1574800867 shutdown_connections 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.498+0000 7f5ccf023700 1 -- 172.21.15.45:0/1574800867 wait complete. 2023-10-06T05:24:30.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.499+0000 7f5ccf023700 1 Processor -- start 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.499+0000 7f5ccf023700 1 -- start start 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5ccf023700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8003c40 0x7f5cc815f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5ccf023700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cc8156e80 0x7f5cc81572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5ccf023700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cc8155dd0 0x7f5cc8156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5ccf023700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8155380 con 0x7f5cc8156e80 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5ccf023700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80c6cc0 con 0x7f5cb8003c40 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5ccf023700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80c6fa0 con 0x7f5cc8155dd0 2023-10-06T05:24:30.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cc8156e80 0x7f5cc81572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cc8156e80 0x7f5cc81572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58474/0 (socket says 172.21.15.45:58474) 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 -- 172.21.15.45:0/458091373 learned_addr learned my addr 172.21.15.45:0/458091373 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5ccd5c0700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cc8155dd0 0x7f5cc8156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8003c40 0x7f5cc815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 -- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cc8155dd0 msgr2=0x7f5cc8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cc8155dd0 0x7f5cc8156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 -- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8003c40 msgr2=0x7f5cc815f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8003c40 0x7f5cc815f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cc7fff700 1 -- 172.21.15.45:0/458091373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc81555d0 con 0x7f5cc8156e80 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.500+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8003c40 0x7f5cc815f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.501+0000 7f5cc7fff700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cc8156e80 0x7f5cc81572f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e770 tx=0x7f5cb4001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.501+0000 7f5cc4ff9700 1 -- 172.21.15.45:0/458091373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4012590 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.501+0000 7f5cc4ff9700 1 -- 172.21.15.45:0/458091373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb40126f0 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.501+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc81558b0 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.501+0000 7f5cc4ff9700 1 -- 172.21.15.45:0/458091373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4012ab0 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.501+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc8154760 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.502+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc815fcc0 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.502+0000 7f5cc4ff9700 1 -- 172.21.15.45:0/458091373 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5cb401d460 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.503+0000 7f5cc4ff9700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5cac066f30 0x7f5cac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.503+0000 7f5cc4ff9700 1 -- 172.21.15.45:0/458091373 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f5cb4084d80 con 0x7f5cc8156e80 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.503+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5cac066f30 0x7f5cac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.503+0000 7f5cccdbf700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5cac066f30 0x7f5cac0693e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001830 tx=0x7f5cbc015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.507+0000 7f5cc4ff9700 1 -- 172.21.15.45:0/458091373 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cb40515a0 con 0x7f5cc8156e80 2023-10-06T05:24:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:30.754+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5cc80cb530 con 0x7f5cc8156e80 2023-10-06T05:24:31.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.051+0000 7f5cc4ff9700 1 -- 172.21.15.45:0/458091373 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v221) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5cc80cb530 con 0x7f5cc8156e80 2023-10-06T05:24:31.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5cac066f30 msgr2=0x7f5cac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5cac066f30 0x7f5cac0693e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001830 tx=0x7f5cbc015000 comp rx=0 tx=0).stop 2023-10-06T05:24:31.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cc8156e80 msgr2=0x7f5cc81572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cc8156e80 0x7f5cc81572f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e770 tx=0x7f5cb4001cd0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 shutdown_connections 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5cac066f30 0x7f5cac0693e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cc8155dd0 0x7f5cc8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cb8003c40 0x7f5cc815f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 --2- 172.21.15.45:0/458091373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cc8156e80 0x7f5cc81572f0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 >> 172.21.15.45:0/458091373 conn(0x7f5cc80cdf50 msgr2=0x7f5cc80b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 shutdown_connections 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.054+0000 7f5ccf023700 1 -- 172.21.15.45:0/458091373 wait complete. 2023-10-06T05:24:31.057 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:31.071 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.206+0000 7f6c69a1c700 1 Processor -- start 2023-10-06T05:24:31.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.206+0000 7f6c69a1c700 1 -- start start 2023-10-06T05:24:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.207+0000 7f6c69a1c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 0x7f6c64159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.207+0000 7f6c69a1c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c640ca6d0 con 0x7f6c64159170 2023-10-06T05:24:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.207+0000 7f6c69a1c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6415d550 con 0x7f6c6415d040 2023-10-06T05:24:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.207+0000 7f6c69a1c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6415d810 con 0x7f6c640692d0 2023-10-06T05:24:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.207+0000 7f6c62ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6c64159170 0x7f6c64068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41572/0 (socket says 172.21.15.45:41572) 2023-10-06T05:24:31.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.207+0000 7f6c62ffd700 1 -- 172.21.15.45:0/2971042183 learned_addr learned my addr 172.21.15.45:0/2971042183 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c627fc700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 0x7f6c64159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138323536 0 0) 0x7f6c640ca6d0 con 0x7f6c64159170 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c50003570 con 0x7f6c64159170 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 671904846 0 0) 0x7f6c50003570 con 0x7f6c64159170 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 msgr2=0x7f6c64159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 0x7f6c64159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 >> v1:172.21.15.144:6789/0 conn(0x7f6c6415d040 legacy=0x7f6c640aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6415daf0 con 0x7f6c64159170 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f6c54001470 con 0x7f6c64159170 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 >> v1:172.21.15.45:6789/0 conn(0x7f6c64159170 legacy=0x7f6c64068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c50003a90 0x7f6c50003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 0x7f6c50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50004d50 0x7f6c500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6415d810 con 0x7f6c50003a90 2023-10-06T05:24:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5000bb70 con 0x7f6c50004d50 2023-10-06T05:24:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5000bd30 con 0x7f6c640692d0 2023-10-06T05:24:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c62ffd700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c50003a90 0x7f6c50003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c627fc700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 0x7f6c50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.208+0000 7f6c637fe700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50004d50 0x7f6c500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c62ffd700 1 -- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 msgr2=0x7f6c50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c62ffd700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640692d0 0x7f6c50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c62ffd700 1 -- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50004d50 msgr2=0x7f6c500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c62ffd700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50004d50 0x7f6c500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c62ffd700 1 -- 172.21.15.45:0/2971042183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50003570 con 0x7f6c50003a90 2023-10-06T05:24:31.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c62ffd700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c50003a90 0x7f6c50003ea0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c3c0 tx=0x7f6c54001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506bda91f0073d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400cc90 con 0x7f6c50003a90 2023-10-06T05:24:31.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5400cdf0 con 0x7f6c50003a90 2023-10-06T05:24:31.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.209+0000 7f6c61ffb700 1 -- 172.21.15.45:0/2971042183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54005b50 con 0x7f6c50003a90 2023-10-06T05:24:31.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.210+0000 7f6c69a1c700 1 -- 172.21.15.45:0/2971042183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c50003a90 msgr2=0x7f6c50003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.210+0000 7f6c69a1c700 1 --2- 172.21.15.45:0/2971042183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c50003a90 0x7f6c50003ea0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c3c0 tx=0x7f6c54001e50 comp rx=0 tx=0).stop 2023-10-06T05:24:31.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.210+0000 7f6c637fe700 1 -- 172.21.15.45:0/2971042183 reap_dead start 2023-10-06T05:24:31.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.210+0000 7f6c69a1c700 1 -- 172.21.15.45:0/2971042183 shutdown_connections 2023-10-06T05:24:31.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.210+0000 7f6c69a1c700 1 -- 172.21.15.45:0/2971042183 >> 172.21.15.45:0/2971042183 conn(0x7f6c6406f7c0 msgr2=0x7f6c640d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.210+0000 7f6c69a1c700 1 -- 172.21.15.45:0/2971042183 shutdown_connections 2023-10-06T05:24:31.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.210+0000 7f6c69a1c700 1 -- 172.21.15.45:0/2971042183 wait complete. 2023-10-06T05:24:31.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.211+0000 7f6c69a1c700 1 Processor -- start 2023-10-06T05:24:31.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.212+0000 7f6c69a1c700 1 -- start start 2023-10-06T05:24:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.212+0000 7f6c69a1c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50003a90 0x7f6c640c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.212+0000 7f6c69a1c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640b2ee0 0x7f6c640b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c69a1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6415b080 0x7f6c6415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c69a1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640b3890 con 0x7f6c6415b080 2023-10-06T05:24:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c69a1c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6415a2f0 con 0x7f6c50003a90 2023-10-06T05:24:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c69a1c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c641596e0 con 0x7f6c640b2ee0 2023-10-06T05:24:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6415b080 0x7f6c6415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6415b080 0x7f6c6415b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58492/0 (socket says 172.21.15.45:58492) 2023-10-06T05:24:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 -- 172.21.15.45:0/1875796925 learned_addr learned my addr 172.21.15.45:0/1875796925 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c627fc700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640b2ee0 0x7f6c640b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c62ffd700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50003a90 0x7f6c640c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 -- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640b2ee0 msgr2=0x7f6c640b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640b2ee0 0x7f6c640b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 -- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50003a90 msgr2=0x7f6c640c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50003a90 0x7f6c640c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 -- 172.21.15.45:0/1875796925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c641599c0 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c637fe700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6415b080 0x7f6c6415b4f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f6c6415cfa0 tx=0x7f6c5c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c5affd700 1 -- 172.21.15.45:0/1875796925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c006db0 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c5affd700 1 -- 172.21.15.45:0/1875796925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c00c540 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c64159ca0 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.213+0000 7f6c5affd700 1 -- 172.21.15.45:0/1875796925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00c900 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.214+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6415ba30 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.214+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6415a460 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.215+0000 7f6c5affd700 1 -- 172.21.15.45:0/1875796925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c5c00cc70 con 0x7f6c6415b080 2023-10-06T05:24:31.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.215+0000 7f6c5affd700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c44066f30 0x7f6c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.215+0000 7f6c5affd700 1 -- 172.21.15.45:0/1875796925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f6c5c084e60 con 0x7f6c6415b080 2023-10-06T05:24:31.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.216+0000 7f6c62ffd700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c44066f30 0x7f6c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.216+0000 7f6c62ffd700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c44066f30 0x7f6c440693e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6c54001ed0 tx=0x7f6c54015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.220+0000 7f6c5affd700 1 -- 172.21.15.45:0/1875796925 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c5c051680 con 0x7f6c6415b080 2023-10-06T05:24:31.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:31.507+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f6c64063b50 con 0x7f6c6415b080 2023-10-06T05:24:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.057+0000 7f6c5affd700 1 -- 172.21.15.45:0/1875796925 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v222) v1 ==== 101+0+0 (secure 0 0 0) 0x7f6c5c054ca0 con 0x7f6c6415b080 2023-10-06T05:24:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c44066f30 msgr2=0x7f6c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c44066f30 0x7f6c440693e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6c54001ed0 tx=0x7f6c54015000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6415b080 msgr2=0x7f6c6415b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6415b080 0x7f6c6415b4f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f6c6415cfa0 tx=0x7f6c5c001640 comp rx=0 tx=0).stop 2023-10-06T05:24:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 shutdown_connections 2023-10-06T05:24:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c44066f30 0x7f6c440693e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c6415b080 0x7f6c6415b4f0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c50003a90 0x7f6c640c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 --2- 172.21.15.45:0/1875796925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c640b2ee0 0x7f6c640b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 >> 172.21.15.45:0/1875796925 conn(0x7f6c6406f7c0 msgr2=0x7f6c640aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 shutdown_connections 2023-10-06T05:24:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.060+0000 7f6c69a1c700 1 -- 172.21.15.45:0/1875796925 wait complete. 2023-10-06T05:24:32.063 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:32.077 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:32.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.122+0000 7f847c8fa040 1 Processor -- start 2023-10-06T05:24:32.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.122+0000 7f847c8fa040 1 -- start start 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f847c8fa040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d83550570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f847c8fa040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d8351f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f847c8fa040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d83565550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f847c8fa040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d834c8320 con 0x561d835697e0 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f847c8fa040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8346c980 con 0x561d8354dca0 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f847c8fa040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d8346c620 con 0x561d835f4b30 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d8351f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d8351f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58498/0 (socket says 172.21.15.45:58498) 2023-10-06T05:24:32.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 -- 172.21.15.45:0/1944203068 learned_addr learned my addr 172.21.15.45:0/1944203068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475b18700 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d83550570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8476319700 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d83565550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 -- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 msgr2=0x561d83565550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d83565550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 -- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 msgr2=0x561d83550570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d83550570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475317700 1 -- 172.21.15.45:0/1944203068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460005cf0 con 0x561d835697e0 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8475b18700 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d83550570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.123+0000 7f8476319700 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d83565550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f8475317700 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d8351f480 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f846000ac70 tx=0x7f8460015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d5c555abc777d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f846ffff700 1 -- 172.21.15.45:0/1944203068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460002f70 con 0x561d835697e0 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f846ffff700 1 -- 172.21.15.45:0/1944203068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460000bc0 con 0x561d835697e0 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f846ffff700 1 -- 172.21.15.45:0/1944203068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846001c410 con 0x561d835697e0 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 -- 172.21.15.45:0/1944203068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 msgr2=0x561d8351f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d8351f480 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f846000ac70 tx=0x7f8460015000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 -- 172.21.15.45:0/1944203068 shutdown_connections 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d83565550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d83550570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 --2- 172.21.15.45:0/1944203068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d8351f480 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 -- 172.21.15.45:0/1944203068 >> 172.21.15.45:0/1944203068 conn(0x561d83522450 msgr2=0x561d83524830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 -- 172.21.15.45:0/1944203068 shutdown_connections 2023-10-06T05:24:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.124+0000 7f847c8fa040 1 -- 172.21.15.45:0/1944203068 wait complete. 2023-10-06T05:24:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.125+0000 7f847c8fa040 1 Processor -- start 2023-10-06T05:24:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.125+0000 7f847c8fa040 1 -- start start 2023-10-06T05:24:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f847c8fa040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d835ef550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f847c8fa040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d835eba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f847c8fa040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d8351f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f847c8fa040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d83463400 con 0x561d835697e0 2023-10-06T05:24:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f847c8fa040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d835ec660 con 0x561d8354dca0 2023-10-06T05:24:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f847c8fa040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d835ec900 con 0x561d835f4b30 2023-10-06T05:24:32.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d835eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d835eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58500/0 (socket says 172.21.15.45:58500) 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 learned_addr learned my addr 172.21.15.45:0/2308717077 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475b18700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d835ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8476319700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d8351f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 msgr2=0x561d8351f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561d835f4b30 0x561d8351f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 msgr2=0x561d835ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d8354dca0 0x561d835ef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.126+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d8347b3d0 con 0x561d835697e0 2023-10-06T05:24:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.127+0000 7f8475317700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d835eba30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f846001f040 tx=0x7f846001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.127+0000 7f846e7fc700 1 -- 172.21.15.45:0/2308717077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846001c410 con 0x561d835697e0 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.127+0000 7f846e7fc700 1 -- 172.21.15.45:0/2308717077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460007e80 con 0x561d835697e0 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.127+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561d8346eae0 con 0x561d835697e0 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.127+0000 7f846e7fc700 1 -- 172.21.15.45:0/2308717077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84600258b0 con 0x561d835697e0 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.127+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d8346e540 con 0x561d835697e0 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.128+0000 7f846e7fc700 1 -- 172.21.15.45:0/2308717077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8460019070 con 0x561d835697e0 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.129+0000 7f846e7fc700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8458065890 0x7f8458067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.129+0000 7f846e7fc700 1 -- 172.21.15.45:0/2308717077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f8460085c60 con 0x561d835697e0 2023-10-06T05:24:32.135 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.129+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x561d835f3870 0x561d83604f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.129+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d835ebf70 con 0x561d835f3870 2023-10-06T05:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.129+0000 7f8475b18700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8458065890 0x7f8458067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.129+0000 7f8476319700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x561d835f3870 0x561d83604f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.130+0000 7f8476319700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x561d835f3870 0x561d83604f40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.130+0000 7f8475b18700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8458065890 0x7f8458067d40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8468001ed0 tx=0x7f8468001780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.131+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v113'629 uv603 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.131+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83605480 con 0x561d835f3870 2023-10-06T05:24:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.133+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v113'630 uv630 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.133+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8351f9c0 con 0x561d835f3870 2023-10-06T05:24:32.137 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.134+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v113'631 uv631 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.134+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615e80 con 0x561d835f3870 2023-10-06T05:24:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.135+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'632 uv632 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.135+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83616230 con 0x561d835f3870 2023-10-06T05:24:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.136+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'633 uv633 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.136+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83614e70 con 0x561d835f3870 2023-10-06T05:24:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.138+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'634 uv634 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.138+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615270 con 0x561d835f3870 2023-10-06T05:24:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.139+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v113'635 uv609 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.139+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615670 con 0x561d835f3870 2023-10-06T05:24:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.140+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v113'636 uv636 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.140+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615670 con 0x561d835f3870 2023-10-06T05:24:32.143 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:32.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.142+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v113'637 uv637 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.142+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.143+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'638 uv638 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.143+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.144+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'639 uv639 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.144+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.145+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'640 uv640 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.145+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.147+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v113'641 uv616 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.147+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.148+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'642 uv642 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.148+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.150 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.149+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v113'643 uv643 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.149+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.150+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'644 uv644 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.150+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.151+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'645 uv645 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.151+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615620 con 0x561d835f3870 2023-10-06T05:24:32.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.153+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'646 uv646 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.153+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615a00 con 0x561d835f3870 2023-10-06T05:24:32.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.154+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'647 uv647 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.154+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.155+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v113'648 uv622 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.155+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.156+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v113'649 uv649 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.156+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.159 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.158+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v113'650 uv650 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.158+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.159+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'651 uv651 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.159+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.160+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'652 uv652 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.160+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.161+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'653 uv653 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.161+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.163+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v113'654 uv628 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.163+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.164+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v113'655 uv655 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.164+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.167 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.165+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v113'656 uv656 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.165+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.166+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'657 uv657 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.166+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.168+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'658 uv658 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.168+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d835f3870 2023-10-06T05:24:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.169+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'659 uv659 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d835f3870 2023-10-06T05:24:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.169+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561d83616af0 0x561d83618ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.169+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83615890 con 0x561d83616af0 2023-10-06T05:24:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.169+0000 7f8475b18700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561d83616af0 0x561d83618ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.170+0000 7f8475b18700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561d83616af0 0x561d83618ed0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.171+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'458 uv441 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.171+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83619410 con 0x561d83616af0 2023-10-06T05:24:32.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.173+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'459 uv459 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.173+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836197c0 con 0x561d83616af0 2023-10-06T05:24:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.174+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'460 uv460 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.174+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361a060 con 0x561d83616af0 2023-10-06T05:24:32.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.175+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v113'461 uv461 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.175+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361a410 con 0x561d83616af0 2023-10-06T05:24:32.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.176+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'462 uv462 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.176+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361a7c0 con 0x561d83616af0 2023-10-06T05:24:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.177+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v113'463 uv446 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.178+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361ab70 con 0x561d83616af0 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.179+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v113'464 uv464 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.179+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.180+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v113'465 uv465 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.180+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.181+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'466 uv466 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.181+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.182+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'467 uv467 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.182+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.184+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v113'468 uv451 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.184+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.185+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v113'469 uv469 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.185+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.186+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v113'470 uv470 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.186+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361e640 con 0x561d83616af0 2023-10-06T05:24:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.188+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v113'471 uv471 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.188+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361e640 con 0x561d83616af0 2023-10-06T05:24:32.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.189+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'472 uv472 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.189+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.190+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v113'473 uv457 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.190+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.191+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v113'474 uv474 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.191+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.194 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.192+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v113'475 uv475 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.192+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361bf30 con 0x561d83616af0 2023-10-06T05:24:32.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.194+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'476 uv476 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.194+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361bf30 con 0x561d83616af0 2023-10-06T05:24:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.195+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'477 uv477 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.195+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361af20 con 0x561d83616af0 2023-10-06T05:24:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.196+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'478 uv478 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.196+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561d8361b430 0x561d8361d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.196+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361dd50 con 0x561d8361b430 2023-10-06T05:24:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.196+0000 7f8475317700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561d8361b430 0x561d8361d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.197+0000 7f8475317700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561d8361b430 0x561d8361d810 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.198+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.198+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361e200 con 0x561d8361b430 2023-10-06T05:24:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.199+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v113'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.199+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361e780 con 0x561d8361b430 2023-10-06T05:24:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.200+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v113'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.202 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.200+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83620080 con 0x561d8361b430 2023-10-06T05:24:32.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.202+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.202+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83620430 con 0x561d8361b430 2023-10-06T05:24:32.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.203+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.203+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f070 con 0x561d8361b430 2023-10-06T05:24:32.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.204+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.204+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f470 con 0x561d8361b430 2023-10-06T05:24:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.205+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'88 uv87 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.205+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f470 con 0x561d8361b430 2023-10-06T05:24:32.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.206+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'89 uv89 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.206+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f470 con 0x561d8361b430 2023-10-06T05:24:32.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.207+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'90 uv90 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.209 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:32.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.207+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f490 con 0x561d8361b430 2023-10-06T05:24:32.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.209+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'91 uv91 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.209+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f490 con 0x561d83616af0 2023-10-06T05:24:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.210+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.210+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f490 con 0x561d83616af0 2023-10-06T05:24:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.211+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v113'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.211+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f470 con 0x561d83616af0 2023-10-06T05:24:32.214 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:32.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.212+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v113'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.212+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f470 con 0x561d83616af0 2023-10-06T05:24:32.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.214+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.214+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f470 con 0x561d83616af0 2023-10-06T05:24:32.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.215+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.215+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8361f470 con 0x561d83616af0 2023-10-06T05:24:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.216+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83616af0 2023-10-06T05:24:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.216+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x561d836207e0 0x561d83622bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.216+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.216+0000 7f8476319700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x561d836207e0 0x561d83622bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.217+0000 7f8476319700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x561d836207e0 0x561d83622bc0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.218+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v113'227 uv221 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.218+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.219+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v113'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.219+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.222 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.220+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v113'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.220+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83624110 con 0x561d836207e0 2023-10-06T05:24:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.221+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.222+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83624110 con 0x561d836207e0 2023-10-06T05:24:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.223+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'231 uv231 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.223+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.224+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.224+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.225+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v113'233 uv226 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.225+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.226+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v113'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.226+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.229 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.227+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v113'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.227+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.228+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.228+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83623100 con 0x561d836207e0 2023-10-06T05:24:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.230+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'237 uv237 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.230+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561d83623780 0x561d83625b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.230+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83626260 con 0x561d83623780 2023-10-06T05:24:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.230+0000 7f8475b18700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561d83623780 0x561d83625b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.231+0000 7f8475b18700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561d83623780 0x561d83625b60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.232+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v113'102 uv101 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83623780 2023-10-06T05:24:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.232+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836262a0 con 0x561d83623780 2023-10-06T05:24:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.233+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v113'103 uv103 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83623780 2023-10-06T05:24:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.233+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836266d0 con 0x561d83623780 2023-10-06T05:24:32.237 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.236+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v113'104 uv104 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83623780 2023-10-06T05:24:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.236+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836266d0 con 0x561d83623780 2023-10-06T05:24:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.238+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'105 uv105 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83623780 2023-10-06T05:24:32.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.238+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836266d0 con 0x561d83623780 2023-10-06T05:24:32.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.241+0000 7f8475b18700 1 -- 172.21.15.45:0/2308717077 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'106 uv106 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f846800c880 con 0x561d83623780 2023-10-06T05:24:32.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.241+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836266d0 con 0x561d8361b430 2023-10-06T05:24:32.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.242+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.242+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836266d0 con 0x561d8361b430 2023-10-06T05:24:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.243+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v113'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.243+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836266d0 con 0x561d8361b430 2023-10-06T05:24:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.244+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v113'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.246 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.245+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836276e0 con 0x561d8361b430 2023-10-06T05:24:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.246+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.246+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836276e0 con 0x561d8361b430 2023-10-06T05:24:32.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.247+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.247+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836266d0 con 0x561d8361b430 2023-10-06T05:24:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.248+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.248+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561d83626d50 0x561d83629130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.248+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d836296b0 con 0x561d83626d50 2023-10-06T05:24:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.248+0000 7f8475317700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561d83626d50 0x561d83629130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.249+0000 7f8475317700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561d83626d50 0x561d83629130 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.250+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v113'203 uv202 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.250+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629790 con 0x561d83626d50 2023-10-06T05:24:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.251+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v113'204 uv204 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.251+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629bc0 con 0x561d83626d50 2023-10-06T05:24:32.254 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.252+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v113'205 uv205 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.252+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629bc0 con 0x561d83626d50 2023-10-06T05:24:32.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.254+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'206 uv206 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.254+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629bc0 con 0x561d83626d50 2023-10-06T05:24:32.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.255+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'207 uv207 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.255+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629bc0 con 0x561d83626d50 2023-10-06T05:24:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.256+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v113'208 uv201 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.256+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629bc0 con 0x561d83626d50 2023-10-06T05:24:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.257+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v113'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.257+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629bc0 con 0x561d83626d50 2023-10-06T05:24:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.258+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v113'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.260 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.258+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d83629bc0 con 0x561d83626d50 2023-10-06T05:24:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.259+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.259+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362c930 con 0x561d83626d50 2023-10-06T05:24:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.260+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d83626d50 2023-10-06T05:24:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.260+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.262+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v113'91 uv90 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.262+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.263+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'92 uv92 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.263+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.264+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v113'93 uv93 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.264+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.265+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'94 uv94 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.265+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561d8362a9c0 0x561d8362d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.265+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8362a9c0 2023-10-06T05:24:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.266+0000 7f8476319700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561d8362a9c0 0x561d8362d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.266+0000 7f8476319700 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561d8362a9c0 0x561d8362d080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.267+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d8362a9c0 2023-10-06T05:24:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.267+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8362a9c0 2023-10-06T05:24:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.268+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v113'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d8362a9c0 2023-10-06T05:24:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.269+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8362a9c0 2023-10-06T05:24:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.270+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v113'125 uv125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d8362a9c0 2023-10-06T05:24:32.271 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.270+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8362a9c0 2023-10-06T05:24:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.271+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d8362a9c0 2023-10-06T05:24:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.271+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8362a9c0 2023-10-06T05:24:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.272+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d8362a9c0 2023-10-06T05:24:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.272+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8362a9c0 2023-10-06T05:24:32.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.273+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d8362a9c0 2023-10-06T05:24:32.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.273+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d836207e0 2023-10-06T05:24:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.274+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v113'104 uv103 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.274+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d836207e0 2023-10-06T05:24:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.275+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v113'105 uv105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.276+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d836207e0 2023-10-06T05:24:32.278 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.276+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v113'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.277+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d836207e0 2023-10-06T05:24:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.278+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'107 uv107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.278+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d836207e0 2023-10-06T05:24:32.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.279+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'108 uv108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8470002750 con 0x561d836207e0 2023-10-06T05:24:32.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.279+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.281+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v113'253 uv245 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.281+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.282+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v113'254 uv254 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.282+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.283+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v113'255 uv255 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.285 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.283+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.284+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v113'256 uv256 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.284+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.286+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'257 uv257 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.286+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.287+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v113'258 uv252 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.287+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.288+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v113'259 uv259 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.288+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.291 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:32.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.289+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v113'260 uv260 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.289+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.291+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'261 uv261 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.291+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.292+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'262 uv262 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.292+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362a0b0 con 0x561d8361b430 2023-10-06T05:24:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.293+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'263 uv263 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.293+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x561d8362d5c0 con 0x561d8361b430 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f8475317700 1 -- 172.21.15.45:0/2308717077 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'264 uv264 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8460054640 con 0x561d8361b430 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561d8361b430 msgr2=0x561d8361d810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561d8361b430 0x561d8361d810 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x561d835f3870 msgr2=0x561d83604f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x561d835f3870 0x561d83604f40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561d83626d50 msgr2=0x561d83629130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561d83626d50 0x561d83629130 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561d83616af0 msgr2=0x561d83618ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561d83616af0 0x561d83618ed0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561d8362a9c0 msgr2=0x561d8362d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561d8362a9c0 0x561d8362d080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561d83623780 msgr2=0x561d83625b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561d83623780 0x561d83625b60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x561d836207e0 msgr2=0x561d83622bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x561d836207e0 0x561d83622bc0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 reap_dead start 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 reap_dead start 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8458065890 msgr2=0x7f8458067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 reap_dead start 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f8476319700 1 -- 172.21.15.45:0/2308717077 reap_dead start 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.294+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8458065890 0x7f8458067d40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8468001ed0 tx=0x7f8468001780 comp rx=0 tx=0).stop 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 msgr2=0x561d835eba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d835eba30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f846001f040 tx=0x7f846001c000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 shutdown_connections 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d835697e0 0x561d835eba30 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 --2- 172.21.15.45:0/2308717077 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8458065890 0x7f8458067d40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 >> 172.21.15.45:0/2308717077 conn(0x561d83522450 msgr2=0x561d8354f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 shutdown_connections 2023-10-06T05:24:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.295+0000 7f847c8fa040 1 -- 172.21.15.45:0/2308717077 wait complete. 2023-10-06T05:24:32.300 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2023-10-06T05:24:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.391+0000 7f5878a5e040 1 Processor -- start 2023-10-06T05:24:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.391+0000 7f5878a5e040 1 -- start start 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f5878a5e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677542120 0x56467755bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f5878a5e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564677546450 0x564677559f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f5878a5e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5646775ecfc0 0x564677516fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f5878a5e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564677461620 con 0x564677546450 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f5878a5e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564677458400 con 0x564677542120 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f5878a5e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646775eabe0 con 0x5646775ecfc0 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564677546450 0x564677559f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564677546450 0x564677559f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58522/0 (socket says 172.21.15.45:58522) 2023-10-06T05:24:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 -- 172.21.15.45:0/3191498801 learned_addr learned my addr 172.21.15.45:0/3191498801 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f5871c7c700 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677542120 0x56467755bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f587247d700 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5646775ecfc0 0x564677516fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 -- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5646775ecfc0 msgr2=0x564677516fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5646775ecfc0 0x564677516fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 -- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677542120 msgr2=0x56467755bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677542120 0x56467755bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 -- 172.21.15.45:0/3191498801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c00a180 con 0x564677546450 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.392+0000 7f586947b700 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564677546450 0x564677559f40 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f585c009980 tx=0x7f585c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d71dff0c884751d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5870c7a700 1 -- 172.21.15.45:0/3191498801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00f040 con 0x564677546450 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5870c7a700 1 -- 172.21.15.45:0/3191498801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c0089e0 con 0x564677546450 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5870c7a700 1 -- 172.21.15.45:0/3191498801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c008d80 con 0x564677546450 2023-10-06T05:24:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 -- 172.21.15.45:0/3191498801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564677546450 msgr2=0x564677559f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564677546450 0x564677559f40 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f585c009980 tx=0x7f585c00c040 comp rx=0 tx=0).stop 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 -- 172.21.15.45:0/3191498801 shutdown_connections 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5646775ecfc0 0x564677516fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677542120 0x56467755bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3191498801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564677546450 0x564677559f40 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 -- 172.21.15.45:0/3191498801 >> 172.21.15.45:0/3191498801 conn(0x564677516bd0 msgr2=0x564677518e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 -- 172.21.15.45:0/3191498801 shutdown_connections 2023-10-06T05:24:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.393+0000 7f5878a5e040 1 -- 172.21.15.45:0/3191498801 wait complete. 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.394+0000 7f5878a5e040 1 Processor -- start 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.394+0000 7f5878a5e040 1 -- start start 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f5878a5e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564677542120 0x5646775182d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f5878a5e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677546450 0x5646775e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f5878a5e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646775ecfc0 0x5646775e4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f5878a5e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646775e5700 con 0x5646775ecfc0 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f5878a5e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646775e59a0 con 0x564677546450 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f5878a5e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646775e5c40 con 0x564677542120 2023-10-06T05:24:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f587247d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646775ecfc0 0x5646775e4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f586947b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677546450 0x5646775e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f587247d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646775ecfc0 0x5646775e4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58530/0 (socket says 172.21.15.45:58530) 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f587247d700 1 -- 172.21.15.45:0/3227686592 learned_addr learned my addr 172.21.15.45:0/3227686592 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f5871c7c700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564677542120 0x5646775182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f586947b700 1 -- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564677542120 msgr2=0x5646775182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f586947b700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564677542120 0x5646775182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f586947b700 1 -- 172.21.15.45:0/3227686592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646775ecfc0 msgr2=0x5646775e4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f586947b700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646775ecfc0 0x5646775e4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f586947b700 1 -- 172.21.15.45:0/3227686592 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c009580 con 0x564677546450 2023-10-06T05:24:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.395+0000 7f587247d700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646775ecfc0 0x5646775e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.396+0000 7f586947b700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677546450 0x5646775e6620 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f585c009980 tx=0x7f585c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.396+0000 7f586affd700 1 -- 172.21.15.45:0/3227686592 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00f040 con 0x564677546450 2023-10-06T05:24:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.396+0000 7f586affd700 1 -- 172.21.15.45:0/3227686592 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c013cf0 con 0x564677546450 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.396+0000 7f586affd700 1 -- 172.21.15.45:0/3227686592 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c01e660 con 0x564677546450 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.396+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56467748a1c0 con 0x564677546450 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.396+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646774703d0 con 0x564677546450 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.397+0000 7f586affd700 1 -- 172.21.15.45:0/3227686592 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f585c01ea90 con 0x564677546450 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.398+0000 7f586affd700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5854065890 0x7f5854067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.398+0000 7f586affd700 1 -- 172.21.15.45:0/3227686592 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f585c085cb0 con 0x564677546450 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.398+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5646775e51e0 0x5646775f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.398+0000 7f5871c7c700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5854065890 0x7f5854067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.398+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v8 -- 0x564677517520 con 0x5646775e51e0 2023-10-06T05:24:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.398+0000 7f587247d700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5646775e51e0 0x5646775f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.399+0000 7f5871c7c700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5854065890 0x7f5854067d40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5864005fd0 tx=0x7f586400c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.399+0000 7f587247d700 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5646775e51e0 0x5646775f7340 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f587247d700 1 -- 172.21.15.45:0/3227686592 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v113'109 uv109 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f586c00e7e0 con 0x5646775e51e0 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5646775e51e0 msgr2=0x5646775f7340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5646775e51e0 0x5646775f7340 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5854065890 msgr2=0x7f5854067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5854065890 0x7f5854067d40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5864005fd0 tx=0x7f586400c040 comp rx=0 tx=0).stop 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677546450 msgr2=0x5646775e6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 --2- 172.21.15.45:0/3227686592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564677546450 0x5646775e6620 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f585c009980 tx=0x7f585c018000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f587247d700 1 -- 172.21.15.45:0/3227686592 reap_dead start 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 shutdown_connections 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 >> 172.21.15.45:0/3227686592 conn(0x564677516bd0 msgr2=0x564677518c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.400+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 shutdown_connections 2023-10-06T05:24:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.401+0000 7f5878a5e040 1 -- 172.21.15.45:0/3227686592 wait complete. 2023-10-06T05:24:32.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.587+0000 7fb6de02b700 1 Processor -- start 2023-10-06T05:24:32.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.587+0000 7fb6de02b700 1 -- start start 2023-10-06T05:24:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6de02b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d8159150 0x7fb6d80ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6de02b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6d8159520 0x7fb6d815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6de02b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80c9c90 con 0x7fb6d815d3d0 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6de02b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d815d8e0 con 0x7fb6d8159520 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6de02b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d815dbc0 con 0x7fb6d8159150 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb6d815d3d0 0x7fb6d8067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41580/0 (socket says 172.21.15.45:41580) 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d7fff700 1 -- 172.21.15.45:0/505706561 learned_addr learned my addr 172.21.15.45:0/505706561 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d77fe700 1 --2- 172.21.15.45:0/505706561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d8159150 0x7fb6d80ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d6ffd700 1 --2- 172.21.15.45:0/505706561 >> v2:172.21.15.144:3300/0 conn(0x7fb6d8159520 0x7fb6d815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d67fc700 1 -- 172.21.15.45:0/505706561 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202622744 0 0) 0x7fb6d80c9c90 con 0x7fb6d815d3d0 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d67fc700 1 -- 172.21.15.45:0/505706561 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c4003570 con 0x7fb6d815d3d0 2023-10-06T05:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d77fe700 1 -- 172.21.15.45:0/505706561 >> v2:172.21.15.144:3300/0 conn(0x7fb6d8159520 msgr2=0x7fb6d815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d77fe700 1 --2- 172.21.15.45:0/505706561 >> v2:172.21.15.144:3300/0 conn(0x7fb6d8159520 0x7fb6d815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d77fe700 1 -- 172.21.15.45:0/505706561 >> v1:172.21.15.45:6789/0 conn(0x7fb6d815d3d0 legacy=0x7fb6d8067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.588+0000 7fb6d77fe700 1 -- 172.21.15.45:0/505706561 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80c9920 con 0x7fb6d8159150 2023-10-06T05:24:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6d77fe700 1 --2- 172.21.15.45:0/505706561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d8159150 0x7fb6d80ca120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80098e0 tx=0x7fb6c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2fa045b6c837dd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6d67fc700 1 -- 172.21.15.45:0/505706561 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8011630 con 0x7fb6d8159150 2023-10-06T05:24:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6d67fc700 1 -- 172.21.15.45:0/505706561 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8011790 con 0x7fb6d8159150 2023-10-06T05:24:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6d67fc700 1 -- 172.21.15.45:0/505706561 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8011b30 con 0x7fb6d8159150 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 -- 172.21.15.45:0/505706561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d8159150 msgr2=0x7fb6d80ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 --2- 172.21.15.45:0/505706561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d8159150 0x7fb6d80ca120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80098e0 tx=0x7fb6c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 -- 172.21.15.45:0/505706561 shutdown_connections 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 --2- 172.21.15.45:0/505706561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d8159150 0x7fb6d80ca120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 --2- 172.21.15.45:0/505706561 >> v2:172.21.15.144:3300/0 conn(0x7fb6d8159520 0x7fb6d815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 -- 172.21.15.45:0/505706561 >> 172.21.15.45:0/505706561 conn(0x7fb6d80691c0 msgr2=0x7fb6d80cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 -- 172.21.15.45:0/505706561 shutdown_connections 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.589+0000 7fb6de02b700 1 -- 172.21.15.45:0/505706561 wait complete. 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.591+0000 7fb6de02b700 1 Processor -- start 2023-10-06T05:24:32.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.591+0000 7fb6de02b700 1 -- start start 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6de02b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d8159150 0x7fb6d815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6de02b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d815d3d0 0x7fb6d80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6de02b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d80afbe0 0x7fb6d80c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6de02b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80c4a90 con 0x7fb6d815d3d0 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6de02b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80b00a0 con 0x7fb6d8159150 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6de02b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80b0380 con 0x7fb6d80afbe0 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d815d3d0 0x7fb6d80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d815d3d0 0x7fb6d80b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58532/0 (socket says 172.21.15.45:58532) 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 -- 172.21.15.45:0/2967978568 learned_addr learned my addr 172.21.15.45:0/2967978568 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d77fe700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d8159150 0x7fb6d815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d7fff700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d80afbe0 0x7fb6d80c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 -- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d80afbe0 msgr2=0x7fb6d80c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d80afbe0 0x7fb6d80c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 -- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d8159150 msgr2=0x7fb6d815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d8159150 0x7fb6d815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 -- 172.21.15.45:0/2967978568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009420 con 0x7fb6d815d3d0 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.592+0000 7fb6d6ffd700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d815d3d0 0x7fb6d80b2380 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000ec80 tx=0x7fb6c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.593+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/2967978568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000c850 con 0x7fb6d815d3d0 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.593+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/2967978568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000c9b0 con 0x7fb6d815d3d0 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.593+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d80b0660 con 0x7fb6d815d3d0 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.593+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/2967978568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000cd50 con 0x7fb6d815d3d0 2023-10-06T05:24:32.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.593+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d815bf10 con 0x7fb6d815d3d0 2023-10-06T05:24:32.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.594+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d80c48d0 con 0x7fb6d815d3d0 2023-10-06T05:24:32.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.594+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/2967978568 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6c0013680 con 0x7fb6d815d3d0 2023-10-06T05:24:32.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.594+0000 7fb6cf7fe700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.595+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/2967978568 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fb6c0085180 con 0x7fb6d815d3d0 2023-10-06T05:24:32.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.595+0000 7fb6d77fe700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.595+0000 7fb6d77fe700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b8066f30 0x7fb6b80693e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80091e0 tx=0x7fb6c8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.599+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/2967978568 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6d80c48d0 con 0x7fb6d815d3d0 2023-10-06T05:24:32.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:32.845+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fb6d80c48d0 con 0x7fb6d815d3d0 2023-10-06T05:24:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.060+0000 7fb6cf7fe700 1 -- 172.21.15.45:0/2967978568 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v223) v1 ==== 162+0+0 (secure 0 0 0) 0x7fb6d80c48d0 con 0x7fb6d815d3d0 2023-10-06T05:24:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b8066f30 msgr2=0x7fb6b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b8066f30 0x7fb6b80693e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80091e0 tx=0x7fb6c8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d815d3d0 msgr2=0x7fb6d80b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d815d3d0 0x7fb6d80b2380 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000ec80 tx=0x7fb6c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 shutdown_connections 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6d80afbe0 0x7fb6d80c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d8159150 0x7fb6d815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 --2- 172.21.15.45:0/2967978568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d815d3d0 0x7fb6d80b2380 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.063+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 >> 172.21.15.45:0/2967978568 conn(0x7fb6d80691c0 msgr2=0x7fb6d8156690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.064+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 shutdown_connections 2023-10-06T05:24:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.064+0000 7fb6de02b700 1 -- 172.21.15.45:0/2967978568 wait complete. 2023-10-06T05:24:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:33.068 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:24:33.067+0000 7ffb3cef9700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-10-06T05:24:33.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:33.067+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:33.069 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:33.067+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:33.077 INFO:tasks.ceph.mds.b.smithi144.stderr: -30> 2023-10-06T05:24:33.067+0000 7ffb3cef9700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-10-06T05:24:33.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:33.098 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:24:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.216+0000 7fb24e85f700 1 Processor -- start 2023-10-06T05:24:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.216+0000 7fb24e85f700 1 -- start start 2023-10-06T05:24:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.216+0000 7fb24e85f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb248153620 0x7fb24815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.216+0000 7fb24e85f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2481539f0 0x7fb24815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb24e85f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2480ce690 con 0x7fb2480cc8b0 2023-10-06T05:24:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb24e85f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24815a7a0 con 0x7fb248153620 2023-10-06T05:24:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb24e85f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2480ce3f0 con 0x7fb2481539f0 2023-10-06T05:24:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb248153620 0x7fb24815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb248153620 0x7fb24815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58546/0 (socket says 172.21.15.45:58546) 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 -- 172.21.15.45:0/3842934502 learned_addr learned my addr 172.21.15.45:0/3842934502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb2477fe700 1 --2- 172.21.15.45:0/3842934502 >> v2:172.21.15.144:3300/0 conn(0x7fb2481539f0 0x7fb24815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 -- 172.21.15.45:0/3842934502 >> v2:172.21.15.144:3300/0 conn(0x7fb2481539f0 msgr2=0x7fb24815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 --2- 172.21.15.45:0/3842934502 >> v2:172.21.15.144:3300/0 conn(0x7fb2481539f0 0x7fb24815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 -- 172.21.15.45:0/3842934502 >> v1:172.21.15.144:6789/0 conn(0x7fb2480cc8b0 legacy=0x7fb2480ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 -- 172.21.15.45:0/3842934502 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24815d510 con 0x7fb248153620 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb247fff700 1 --2- 172.21.15.45:0/3842934502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb248153620 0x7fb24815a260 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fb248067840 tx=0x7fb238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa886ad23a58f08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb246ffd700 1 -- 172.21.15.45:0/3842934502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23800b860 con 0x7fb248153620 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb246ffd700 1 -- 172.21.15.45:0/3842934502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23800b9c0 con 0x7fb248153620 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.217+0000 7fb246ffd700 1 -- 172.21.15.45:0/3842934502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23800bd60 con 0x7fb248153620 2023-10-06T05:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 -- 172.21.15.45:0/3842934502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb248153620 msgr2=0x7fb24815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 --2- 172.21.15.45:0/3842934502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb248153620 0x7fb24815a260 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fb248067840 tx=0x7fb238009250 comp rx=0 tx=0).stop 2023-10-06T05:24:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 -- 172.21.15.45:0/3842934502 shutdown_connections 2023-10-06T05:24:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 --2- 172.21.15.45:0/3842934502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb248153620 0x7fb24815a260 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 --2- 172.21.15.45:0/3842934502 >> v2:172.21.15.144:3300/0 conn(0x7fb2481539f0 0x7fb24815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 -- 172.21.15.45:0/3842934502 >> 172.21.15.45:0/3842934502 conn(0x7fb24806ed50 msgr2=0x7fb24806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 -- 172.21.15.45:0/3842934502 shutdown_connections 2023-10-06T05:24:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.218+0000 7fb24e85f700 1 -- 172.21.15.45:0/3842934502 wait complete. 2023-10-06T05:24:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.220+0000 7fb24e85f700 1 Processor -- start 2023-10-06T05:24:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.220+0000 7fb24e85f700 1 -- start start 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb24e85f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2480cc8b0 0x7fb2480b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb24e85f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb248153620 0x7fb2480caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb24e85f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2480cb3f0 0x7fb2481553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb24e85f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480b5750 con 0x7fb2480cc8b0 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb24e85f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480b3430 con 0x7fb248153620 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb24e85f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480b4e20 con 0x7fb2480cb3f0 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb247fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2480cc8b0 0x7fb2480b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb247fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2480cc8b0 0x7fb2480b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58552/0 (socket says 172.21.15.45:58552) 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb247fff700 1 -- 172.21.15.45:0/1054104590 learned_addr learned my addr 172.21.15.45:0/1054104590 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb24cdfc700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2480cb3f0 0x7fb2481553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.221+0000 7fb2477fe700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb248153620 0x7fb2480caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb247fff700 1 -- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2480cb3f0 msgr2=0x7fb2481553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb247fff700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2480cb3f0 0x7fb2481553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb247fff700 1 -- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb248153620 msgr2=0x7fb2480caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb247fff700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb248153620 0x7fb2480caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb247fff700 1 -- 172.21.15.45:0/1054104590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2380094f0 con 0x7fb2480cc8b0 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb2477fe700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb248153620 0x7fb2480caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb24cdfc700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2480cb3f0 0x7fb2481553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb247fff700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2480cc8b0 0x7fb2480b2ef0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fb24806f900 tx=0x7fb238017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb23ffff700 1 -- 172.21.15.45:0/1054104590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238011e60 con 0x7fb2480cc8b0 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb23ffff700 1 -- 172.21.15.45:0/1054104590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23801a630 con 0x7fb2480cc8b0 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb23ffff700 1 -- 172.21.15.45:0/1054104590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23801a9f0 con 0x7fb2480cc8b0 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2480b5100 con 0x7fb2480cc8b0 2023-10-06T05:24:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.222+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2480b4ad0 con 0x7fb2480cc8b0 2023-10-06T05:24:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.223+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb248088a60 con 0x7fb2480cc8b0 2023-10-06T05:24:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.223+0000 7fb23ffff700 1 -- 172.21.15.45:0/1054104590 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb23802d440 con 0x7fb2480cc8b0 2023-10-06T05:24:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.224+0000 7fb23ffff700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb228066f30 0x7fb2280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.224+0000 7fb23ffff700 1 -- 172.21.15.45:0/1054104590 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fb238086a30 con 0x7fb2480cc8b0 2023-10-06T05:24:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.224+0000 7fb2477fe700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb228066f30 0x7fb2280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.225+0000 7fb2477fe700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb228066f30 0x7fb2280693e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb248158340 tx=0x7fb230008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.228+0000 7fb23ffff700 1 -- 172.21.15.45:0/1054104590 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2380532e0 con 0x7fb2480cc8b0 2023-10-06T05:24:33.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.486+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb24804bb40 con 0x7fb2480cc8b0 2023-10-06T05:24:33.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.487+0000 7fb23ffff700 1 -- 172.21.15.45:0/1054104590 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 225 v225) v1 ==== 95+0+3081 (secure 0 0 0) 0x7fb238015020 con 0x7fb2480cc8b0 2023-10-06T05:24:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":225,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":225,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:33.068761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb228066f30 msgr2=0x7fb2280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb228066f30 0x7fb2280693e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb248158340 tx=0x7fb230008040 comp rx=0 tx=0).stop 2023-10-06T05:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2480cc8b0 msgr2=0x7fb2480b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2480cc8b0 0x7fb2480b2ef0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fb24806f900 tx=0x7fb238017000 comp rx=0 tx=0).stop 2023-10-06T05:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 shutdown_connections 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb228066f30 0x7fb2280693e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2480cb3f0 0x7fb2481553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2480cc8b0 0x7fb2480b2ef0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 --2- 172.21.15.45:0/1054104590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb248153620 0x7fb2480caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 >> 172.21.15.45:0/1054104590 conn(0x7fb24806ed50 msgr2=0x7fb2480ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 shutdown_connections 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.489+0000 7fb24e85f700 1 -- 172.21.15.45:0/1054104590 wait complete. 2023-10-06T05:24:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 225 2023-10-06T05:24:33.504 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:33.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.636+0000 7f0d7065e700 1 Processor -- start 2023-10-06T05:24:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.636+0000 7f0d7065e700 1 -- start start 2023-10-06T05:24:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d7065e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d6815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d7065e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0d680ccfc0 0x7f0d6815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d7065e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d680ce690 con 0x7f0d681539f0 2023-10-06T05:24:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d7065e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6815a7a0 con 0x7f0d680ccfc0 2023-10-06T05:24:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d7065e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680ce3f0 con 0x7f0d68153620 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d6e3fa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d6815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d6ebfb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0d680ccfc0 0x7f0d6815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d6e3fa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d6815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:50502/0 (socket says 172.21.15.45:50502) 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d6ebfb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0d680ccfc0 0x7f0d6815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:46198/0 (socket says 172.21.15.45:46198) 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.637+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/1371797998 learned_addr learned my addr 172.21.15.45:0/1371797998 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6d3f8700 1 -- 172.21.15.45:0/1371797998 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880975925 0 0) 0x7f0d680ce690 con 0x7f0d681539f0 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/1371797998 >> v2:172.21.15.144:3300/0 conn(0x7f0d680ccfc0 msgr2=0x7f0d6815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6e3fa700 1 --2- 172.21.15.45:0/1371797998 >> v2:172.21.15.144:3300/0 conn(0x7f0d680ccfc0 0x7f0d6815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/1371797998 >> v1:172.21.15.144:6789/0 conn(0x7f0d681539f0 legacy=0x7f0d680cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/1371797998 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6815d510 con 0x7f0d68153620 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6e3fa700 1 --2- 172.21.15.45:0/1371797998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d6815a260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d640099b0 tx=0x7f0d64009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db1ade6d959a0709 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6d3f8700 1 -- 172.21.15.45:0/1371797998 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64009ea0 con 0x7f0d68153620 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6d3f8700 1 -- 172.21.15.45:0/1371797998 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6400b860 con 0x7f0d68153620 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.638+0000 7f0d6d3f8700 1 -- 172.21.15.45:0/1371797998 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400bc00 con 0x7f0d68153620 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 -- 172.21.15.45:0/1371797998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 msgr2=0x7f0d6815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 --2- 172.21.15.45:0/1371797998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d6815a260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d640099b0 tx=0x7f0d64009250 comp rx=0 tx=0).stop 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 -- 172.21.15.45:0/1371797998 shutdown_connections 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 --2- 172.21.15.45:0/1371797998 >> v2:172.21.15.144:3300/0 conn(0x7f0d680ccfc0 0x7f0d6815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 --2- 172.21.15.45:0/1371797998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d6815a260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 -- 172.21.15.45:0/1371797998 >> 172.21.15.45:0/1371797998 conn(0x7f0d6806ed50 msgr2=0x7f0d6806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 -- 172.21.15.45:0/1371797998 shutdown_connections 2023-10-06T05:24:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.639+0000 7f0d7065e700 1 -- 172.21.15.45:0/1371797998 wait complete. 2023-10-06T05:24:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.640+0000 7f0d7065e700 1 Processor -- start 2023-10-06T05:24:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.640+0000 7f0d7065e700 1 -- start start 2023-10-06T05:24:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d7065e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d680ccfc0 0x7f0d680c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d7065e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d680c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d7065e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d6815c480 0x7f0d6814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d7065e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680ca090 con 0x7f0d680ccfc0 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d7065e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68169820 con 0x7f0d6815c480 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d7065e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68169530 con 0x7f0d68153620 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d6e3fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d680ccfc0 0x7f0d680c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d6e3fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d680ccfc0 0x7f0d680c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58554/0 (socket says 172.21.15.45:58554) 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/3622412004 learned_addr learned my addr 172.21.15.45:0/3622412004 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d6ebfb700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d6815c480 0x7f0d6814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.641+0000 7f0d6dbf9700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d680c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 msgr2=0x7f0d680c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d6e3fa700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d680c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d6815c480 msgr2=0x7f0d6814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d6e3fa700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d6815c480 0x7f0d6814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d6e3fa700 1 -- 172.21.15.45:0/3622412004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640094f0 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d6e3fa700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d680ccfc0 0x7f0d680c91f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0d68088fb0 tx=0x7f0d64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d5e7fc700 1 -- 172.21.15.45:0/3622412004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64000d10 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d5e7fc700 1 -- 172.21.15.45:0/3622412004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6400bbe0 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d680c97f0 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d5e7fc700 1 -- 172.21.15.45:0/3622412004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64011670 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.642+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d680c9db0 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.643+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d68088a60 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.644+0000 7f0d5e7fc700 1 -- 172.21.15.45:0/3622412004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d64011aa0 con 0x7f0d680ccfc0 2023-10-06T05:24:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.644+0000 7f0d5e7fc700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d4c066f30 0x7f0d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.644+0000 7f0d5e7fc700 1 -- 172.21.15.45:0/3622412004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f0d64086760 con 0x7f0d680ccfc0 2023-10-06T05:24:33.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.644+0000 7f0d6dbf9700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d4c066f30 0x7f0d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.645+0000 7f0d6dbf9700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d4c066f30 0x7f0d4c0693e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0d68167170 tx=0x7f0d58007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.649+0000 7f0d5e7fc700 1 -- 172.21.15.45:0/3622412004 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d64052f90 con 0x7f0d680ccfc0 2023-10-06T05:24:33.807 INFO:tasks.ceph.mds.c.smithi045.stderr: -48> 2023-10-06T05:24:29.054+0000 7f033a022700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input 2023-10-06T05:24:33.828 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:24:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.906+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d6804bb40 con 0x7f0d680ccfc0 2023-10-06T05:24:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.907+0000 7f0d5e7fc700 1 -- 172.21.15.45:0/3622412004 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 225 v225) v1 ==== 95+0+3081 (secure 0 0 0) 0x7f0d64015c20 con 0x7f0d680ccfc0 2023-10-06T05:24:33.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:33.909 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":225,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":225,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:33.068761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:33.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d4c066f30 msgr2=0x7f0d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d4c066f30 0x7f0d4c0693e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0d68167170 tx=0x7f0d58007ec0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d680ccfc0 msgr2=0x7f0d680c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d680ccfc0 0x7f0d680c91f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0d68088fb0 tx=0x7f0d64017000 comp rx=0 tx=0).stop 2023-10-06T05:24:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 shutdown_connections 2023-10-06T05:24:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0d4c066f30 0x7f0d4c0693e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d6815c480 0x7f0d6814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d680ccfc0 0x7f0d680c91f0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 --2- 172.21.15.45:0/3622412004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d68153620 0x7f0d680c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 >> 172.21.15.45:0/3622412004 conn(0x7f0d6806ed50 msgr2=0x7f0d6814aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 shutdown_connections 2023-10-06T05:24:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:33.909+0000 7f0d7065e700 1 -- 172.21.15.45:0/3622412004 wait complete. 2023-10-06T05:24:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 225 2023-10-06T05:24:33.924 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:33.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.055+0000 7f4320822700 1 Processor -- start 2023-10-06T05:24:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.055+0000 7f4320822700 1 -- start start 2023-10-06T05:24:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f4320822700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f431c1570b0 0x7f431c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f4320822700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f431c15ac10 0x7f431c0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f4320822700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c15b380 con 0x7f431c156ce0 2023-10-06T05:24:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f4320822700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c06c010 con 0x7f431c15ac10 2023-10-06T05:24:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f4320822700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c0b02f0 con 0x7f431c1570b0 2023-10-06T05:24:34.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f431c15ac10 0x7f431c0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431a59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f431c156ce0 0x7f431c0afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41604/0 (socket says 172.21.15.45:41604) 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431a59c700 1 -- 172.21.15.45:0/2795689896 learned_addr learned my addr 172.21.15.45:0/2795689896 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f4319d9b700 1 --2- 172.21.15.45:0/2795689896 >> v2:172.21.15.144:3300/0 conn(0x7f431c1570b0 0x7f431c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431ad9d700 1 -- 172.21.15.45:0/2795689896 >> v2:172.21.15.144:3300/0 conn(0x7f431c1570b0 msgr2=0x7f431c06bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431ad9d700 1 --2- 172.21.15.45:0/2795689896 >> v2:172.21.15.144:3300/0 conn(0x7f431c1570b0 0x7f431c06bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431ad9d700 1 -- 172.21.15.45:0/2795689896 >> v1:172.21.15.45:6789/0 conn(0x7f431c156ce0 legacy=0x7f431c0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431959a700 1 -- 172.21.15.45:0/2795689896 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699582068 0 0) 0x7f431c15b380 con 0x7f431c156ce0 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.057+0000 7f431ad9d700 1 -- 172.21.15.45:0/2795689896 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c15b660 con 0x7f431c15ac10 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f431ad9d700 1 --2- 172.21.15.45:0/2795689896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c15ac10 0x7f431c0ccce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f431c06ab50 tx=0x7f43140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5163b793911eb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f431959a700 1 -- 172.21.15.45:0/2795689896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314011630 con 0x7f431c15ac10 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f431959a700 1 -- 172.21.15.45:0/2795689896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314011790 con 0x7f431c15ac10 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f431959a700 1 -- 172.21.15.45:0/2795689896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314011b30 con 0x7f431c15ac10 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 -- 172.21.15.45:0/2795689896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c15ac10 msgr2=0x7f431c0ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 --2- 172.21.15.45:0/2795689896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c15ac10 0x7f431c0ccce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f431c06ab50 tx=0x7f43140091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 -- 172.21.15.45:0/2795689896 shutdown_connections 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 --2- 172.21.15.45:0/2795689896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c15ac10 0x7f431c0ccce0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 --2- 172.21.15.45:0/2795689896 >> v2:172.21.15.144:3300/0 conn(0x7f431c1570b0 0x7f431c06bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 -- 172.21.15.45:0/2795689896 >> 172.21.15.45:0/2795689896 conn(0x7f431c06f090 msgr2=0x7f431c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 -- 172.21.15.45:0/2795689896 shutdown_connections 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.058+0000 7f4320822700 1 -- 172.21.15.45:0/2795689896 wait complete. 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 Processor -- start 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 -- start start 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431c156ce0 0x7f431c0c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c1570b0 0x7f431c0c0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431c15ac10 0x7f431c0c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0c1760 con 0x7f431c1570b0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0c2130 con 0x7f431c15ac10 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.060+0000 7f4320822700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0c0b50 con 0x7f431c156ce0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c1570b0 0x7f431c0c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c1570b0 0x7f431c0c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58586/0 (socket says 172.21.15.45:58586) 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 -- 172.21.15.45:0/3349187914 learned_addr learned my addr 172.21.15.45:0/3349187914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f431a59c700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431c156ce0 0x7f431c0c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f431ad9d700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431c15ac10 0x7f431c0c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 -- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431c156ce0 msgr2=0x7f431c0c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431c156ce0 0x7f431c0c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 -- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431c15ac10 msgr2=0x7f431c0c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431c15ac10 0x7f431c0c1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 -- 172.21.15.45:0/3349187914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009420 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f431ad9d700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431c15ac10 0x7f431c0c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4319d9b700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c1570b0 0x7f431c0c0450 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f431000ecb0 tx=0x7f43100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f430a7fc700 1 -- 172.21.15.45:0/3349187914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431000c9d0 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f430a7fc700 1 -- 172.21.15.45:0/3349187914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431000cb50 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c0c0e30 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f430a7fc700 1 -- 172.21.15.45:0/3349187914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310013470 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.061+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c0c13f0 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.062+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c0c1910 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.063+0000 7f430a7fc700 1 -- 172.21.15.45:0/3349187914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4310013860 con 0x7f431c1570b0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.063+0000 7f430a7fc700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f42fc066f00 0x7f42fc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.063+0000 7f430a7fc700 1 -- 172.21.15.45:0/3349187914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f4310085300 con 0x7f431c1570b0 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.064+0000 7f431a59c700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f42fc066f00 0x7f42fc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.064+0000 7f431a59c700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f42fc066f00 0x7f42fc0693b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f430c002230 tx=0x7f430c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.068+0000 7f430a7fc700 1 -- 172.21.15.45:0/3349187914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f431c0c1910 con 0x7f431c1570b0 2023-10-06T05:24:34.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.322+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f431c0c1910 con 0x7f431c1570b0 2023-10-06T05:24:34.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.323+0000 7f430a7fc700 1 -- 172.21.15.45:0/3349187914 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f431c0c1910 con 0x7f431c1570b0 2023-10-06T05:24:34.325 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:34.325 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":226,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":8875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2156055156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2156055156},{"type":"v1","addr":"172.21.15.144:6835","nonce":2156055156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226},{"gid":9306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2971431427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2971431427},{"type":"v1","addr":"172.21.15.45:6835","nonce":2971431427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226}],"filesystems":[{"mdsmap":{"epoch":225,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:33.068761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f42fc066f00 msgr2=0x7f42fc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f42fc066f00 0x7f42fc0693b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f430c002230 tx=0x7f430c009040 comp rx=0 tx=0).stop 2023-10-06T05:24:34.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c1570b0 msgr2=0x7f431c0c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c1570b0 0x7f431c0c0450 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f431000ecb0 tx=0x7f43100091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 shutdown_connections 2023-10-06T05:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f42fc066f00 0x7f42fc0693b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431c15ac10 0x7f431c0c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431c156ce0 0x7f431c0c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 --2- 172.21.15.45:0/3349187914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f431c1570b0 0x7f431c0c0450 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 >> 172.21.15.45:0/3349187914 conn(0x7f431c06f090 msgr2=0x7f431c070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 shutdown_connections 2023-10-06T05:24:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.325+0000 7f4320822700 1 -- 172.21.15.45:0/3349187914 wait complete. 2023-10-06T05:24:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 226 2023-10-06T05:24:34.340 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2023-10-06T05:24:34.340 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2023-10-06T05:24:34.341 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:34.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:34.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:34.341 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:34.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:34.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.537+0000 7fb1ea188700 1 Processor -- start 2023-10-06T05:24:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.537+0000 7fb1ea188700 1 -- start start 2023-10-06T05:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.538+0000 7fb1ea188700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4158f60 0x7fb1e4159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.538+0000 7fb1ea188700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb1e406a060 0x7fb1e40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.538+0000 7fb1ea188700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e4069bd0 con 0x7fb1e40cc390 2023-10-06T05:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.538+0000 7fb1ea188700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e406a4d0 con 0x7fb1e406a060 2023-10-06T05:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.538+0000 7fb1ea188700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4068fc0 con 0x7fb1e4158f60 2023-10-06T05:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.538+0000 7fb1e37fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4158f60 0x7fb1e4159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.538+0000 7fb1e2ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb1e406a060 0x7fb1e40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e3fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb1e40cc390 0x7fb1e40c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:60474/0 (socket says 172.21.15.45:60474) 2023-10-06T05:24:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e3fff700 1 -- 172.21.15.45:0/180954090 learned_addr learned my addr 172.21.15.45:0/180954090 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e27fc700 1 -- 172.21.15.45:0/180954090 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869197347 0 0) 0x7fb1e4069bd0 con 0x7fb1e40cc390 2023-10-06T05:24:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e27fc700 1 -- 172.21.15.45:0/180954090 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d0003570 con 0x7fb1e40cc390 2023-10-06T05:24:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e37fe700 1 -- 172.21.15.45:0/180954090 >> v2:172.21.15.144:3300/0 conn(0x7fb1e406a060 msgr2=0x7fb1e40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e37fe700 1 --2- 172.21.15.45:0/180954090 >> v2:172.21.15.144:3300/0 conn(0x7fb1e406a060 0x7fb1e40cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e37fe700 1 -- 172.21.15.45:0/180954090 >> v1:172.21.15.144:6789/0 conn(0x7fb1e40cc390 legacy=0x7fb1e40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e37fe700 1 -- 172.21.15.45:0/180954090 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40692a0 con 0x7fb1e4158f60 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e2ffd700 1 --2- 172.21.15.45:0/180954090 >> v2:172.21.15.144:3300/0 conn(0x7fb1e406a060 0x7fb1e40cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e37fe700 1 --2- 172.21.15.45:0/180954090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4158f60 0x7fb1e4159330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40098e0 tx=0x7fb1d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b21706e743333949 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e27fc700 1 -- 172.21.15.45:0/180954090 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4011630 con 0x7fb1e4158f60 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.539+0000 7fb1e27fc700 1 -- 172.21.15.45:0/180954090 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d4011790 con 0x7fb1e4158f60 2023-10-06T05:24:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1e27fc700 1 -- 172.21.15.45:0/180954090 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4011b30 con 0x7fb1e4158f60 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 -- 172.21.15.45:0/180954090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4158f60 msgr2=0x7fb1e4159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 --2- 172.21.15.45:0/180954090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4158f60 0x7fb1e4159330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40098e0 tx=0x7fb1d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 -- 172.21.15.45:0/180954090 shutdown_connections 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 --2- 172.21.15.45:0/180954090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4158f60 0x7fb1e4159330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 --2- 172.21.15.45:0/180954090 >> v2:172.21.15.144:3300/0 conn(0x7fb1e406a060 0x7fb1e40cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 -- 172.21.15.45:0/180954090 >> 172.21.15.45:0/180954090 conn(0x7fb1e40cdf50 msgr2=0x7fb1e40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 -- 172.21.15.45:0/180954090 shutdown_connections 2023-10-06T05:24:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.540+0000 7fb1ea188700 1 -- 172.21.15.45:0/180954090 wait complete. 2023-10-06T05:24:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.541+0000 7fb1ea188700 1 Processor -- start 2023-10-06T05:24:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1ea188700 1 -- start start 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1ea188700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e406a060 0x7fb1e4153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1ea188700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e40cc390 0x7fb1e41696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1ea188700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4158f60 0x7fb1e40c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1ea188700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c6810 con 0x7fb1e4158f60 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1ea188700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c6a60 con 0x7fb1e406a060 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1ea188700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c6d40 con 0x7fb1e40cc390 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4158f60 0x7fb1e40c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4158f60 0x7fb1e40c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58588/0 (socket says 172.21.15.45:58588) 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1e3fff700 1 -- 172.21.15.45:0/2351421154 learned_addr learned my addr 172.21.15.45:0/2351421154 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.542+0000 7fb1e37fe700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e406a060 0x7fb1e4153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e2ffd700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e40cc390 0x7fb1e41696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e3fff700 1 -- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e40cc390 msgr2=0x7fb1e41696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e3fff700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e40cc390 0x7fb1e41696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e3fff700 1 -- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e406a060 msgr2=0x7fb1e4153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e3fff700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e406a060 0x7fb1e4153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e3fff700 1 -- 172.21.15.45:0/2351421154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc009420 con 0x7fb1e4158f60 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e37fe700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e406a060 0x7fb1e4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1e3fff700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4158f60 0x7fb1e40c7820 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc001460 tx=0x7fb1dc001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1db7fe700 1 -- 172.21.15.45:0/2351421154 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00a890 con 0x7fb1e4158f60 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1db7fe700 1 -- 172.21.15.45:0/2351421154 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00a9f0 con 0x7fb1e4158f60 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d4009420 con 0x7fb1e4158f60 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1db7fe700 1 -- 172.21.15.45:0/2351421154 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00ad70 con 0x7fb1e4158f60 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.543+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40c7380 con 0x7fb1e4158f60 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.544+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e40c7510 con 0x7fb1e4158f60 2023-10-06T05:24:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.545+0000 7fb1db7fe700 1 -- 172.21.15.45:0/2351421154 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1dc007300 con 0x7fb1e4158f60 2023-10-06T05:24:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.546+0000 7fb1db7fe700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb1c4066f30 0x7fb1c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.546+0000 7fb1db7fe700 1 -- 172.21.15.45:0/2351421154 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fb1dc085e70 con 0x7fb1e4158f60 2023-10-06T05:24:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.546+0000 7fb1e37fe700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb1c4066f30 0x7fb1c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.547+0000 7fb1e37fe700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb1c4066f30 0x7fb1c40693e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40091e0 tx=0x7fb1d4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.550+0000 7fb1db7fe700 1 -- 172.21.15.45:0/2351421154 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1dc0526b0 con 0x7fb1e4158f60 2023-10-06T05:24:34.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:34.797+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb1e4069580 con 0x7fb1e4158f60 2023-10-06T05:24:35.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.077+0000 7fb1db7fe700 1 -- 172.21.15.45:0/2351421154 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v227) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb1dc055cd0 con 0x7fb1e4158f60 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb1c4066f30 msgr2=0x7fb1c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb1c4066f30 0x7fb1c40693e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40091e0 tx=0x7fb1d4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4158f60 msgr2=0x7fb1e40c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4158f60 0x7fb1e40c7820 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc001460 tx=0x7fb1dc001ae0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 shutdown_connections 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb1c4066f30 0x7fb1c40693e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4158f60 0x7fb1e40c7820 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e406a060 0x7fb1e4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 --2- 172.21.15.45:0/2351421154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e40cc390 0x7fb1e41696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 >> 172.21.15.45:0/2351421154 conn(0x7fb1e40cdf50 msgr2=0x7fb1e4154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.079+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 shutdown_connections 2023-10-06T05:24:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.080+0000 7fb1ea188700 1 -- 172.21.15.45:0/2351421154 wait complete. 2023-10-06T05:24:35.083 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:35.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.231+0000 7fa2e6906700 1 Processor -- start 2023-10-06T05:24:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.231+0000 7fa2e6906700 1 -- start start 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.231+0000 7fa2e6906700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e0153620 0x7fa2e015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.231+0000 7fa2e6906700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2e01539f0 0x7fa2e015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.231+0000 7fa2e6906700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e00ce690 con 0x7fa2e00cc8b0 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2e6906700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e015a7a0 con 0x7fa2e01539f0 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2e6906700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00ce3f0 con 0x7fa2e0153620 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2e01539f0 0x7fa2e015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2e01539f0 0x7fa2e015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58590/0 (socket says 172.21.15.45:58590) 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 -- 172.21.15.45:0/1963011260 learned_addr learned my addr 172.21.15.45:0/1963011260 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2dffff700 1 --2- 172.21.15.45:0/1963011260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e0153620 0x7fa2e015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 -- 172.21.15.45:0/1963011260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e0153620 msgr2=0x7fa2e015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 --2- 172.21.15.45:0/1963011260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e0153620 0x7fa2e015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 -- 172.21.15.45:0/1963011260 >> v1:172.21.15.144:6789/0 conn(0x7fa2e00cc8b0 legacy=0x7fa2e00ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 -- 172.21.15.45:0/1963011260 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e015d510 con 0x7fa2e01539f0 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2df7fe700 1 --2- 172.21.15.45:0/1963011260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e01539f0 0x7fa2e015cdc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40099b0 tx=0x7fa2d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd4b6ca4d073acf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2deffd700 1 -- 172.21.15.45:0/1963011260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400b820 con 0x7fa2e01539f0 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2deffd700 1 -- 172.21.15.45:0/1963011260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400b980 con 0x7fa2e01539f0 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.232+0000 7fa2deffd700 1 -- 172.21.15.45:0/1963011260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400bd20 con 0x7fa2e01539f0 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 -- 172.21.15.45:0/1963011260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e01539f0 msgr2=0x7fa2e015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 --2- 172.21.15.45:0/1963011260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e01539f0 0x7fa2e015cdc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40099b0 tx=0x7fa2d4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 -- 172.21.15.45:0/1963011260 shutdown_connections 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 --2- 172.21.15.45:0/1963011260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e0153620 0x7fa2e015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 --2- 172.21.15.45:0/1963011260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e01539f0 0x7fa2e015cdc0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 -- 172.21.15.45:0/1963011260 >> 172.21.15.45:0/1963011260 conn(0x7fa2e006ed50 msgr2=0x7fa2e006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:35.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 -- 172.21.15.45:0/1963011260 shutdown_connections 2023-10-06T05:24:35.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.233+0000 7fa2e6906700 1 -- 172.21.15.45:0/1963011260 wait complete. 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.234+0000 7fa2e6906700 1 Processor -- start 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.235+0000 7fa2e6906700 1 -- start start 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.235+0000 7fa2e6906700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e00cc8b0 0x7fa2e00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.235+0000 7fa2e6906700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e0153620 0x7fa2e01694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.235+0000 7fa2e6906700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e015b2b0 0x7fa2e00c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.235+0000 7fa2e6906700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e015bae0 con 0x7fa2e0153620 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.235+0000 7fa2e6906700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0140440 con 0x7fa2e015b2b0 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.235+0000 7fa2e6906700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b4410 con 0x7fa2e00cc8b0 2023-10-06T05:24:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e0153620 0x7fa2e01694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e0153620 0x7fa2e01694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58596/0 (socket says 172.21.15.45:58596) 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 -- 172.21.15.45:0/2209169943 learned_addr learned my addr 172.21.15.45:0/2209169943 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2dffff700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e00cc8b0 0x7fa2e00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2e4ea3700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e015b2b0 0x7fa2e00c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 -- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e00cc8b0 msgr2=0x7fa2e00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e00cc8b0 0x7fa2e00c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 -- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e015b2b0 msgr2=0x7fa2e00c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e015b2b0 0x7fa2e00c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 -- 172.21.15.45:0/2209169943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40094f0 con 0x7fa2e0153620 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2df7fe700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e0153620 0x7fa2e01694b0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009290 tx=0x7fa2d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2c7fff700 1 -- 172.21.15.45:0/2209169943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4011e00 con 0x7fa2e0153620 2023-10-06T05:24:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2c7fff700 1 -- 172.21.15.45:0/2209169943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d401b470 con 0x7fa2e0153620 2023-10-06T05:24:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e00b46b0 con 0x7fa2e0153620 2023-10-06T05:24:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2c7fff700 1 -- 172.21.15.45:0/2209169943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d401b830 con 0x7fa2e0153620 2023-10-06T05:24:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.236+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e00b4c70 con 0x7fa2e0153620 2023-10-06T05:24:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.237+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e0088a60 con 0x7fa2e0153620 2023-10-06T05:24:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.238+0000 7fa2c7fff700 1 -- 172.21.15.45:0/2209169943 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2d401bbe0 con 0x7fa2e0153620 2023-10-06T05:24:35.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.238+0000 7fa2c7fff700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2cc066f30 0x7fa2cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.238+0000 7fa2c7fff700 1 -- 172.21.15.45:0/2209169943 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fa2d40869f0 con 0x7fa2e0153620 2023-10-06T05:24:35.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.238+0000 7fa2dffff700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2cc066f30 0x7fa2cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.239+0000 7fa2dffff700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2cc066f30 0x7fa2cc0693e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006fd0 tx=0x7fa2d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:35.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.243+0000 7fa2c7fff700 1 -- 172.21.15.45:0/2209169943 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2d4053220 con 0x7fa2e0153620 2023-10-06T05:24:35.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:35.532+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fa2e00d0a20 con 0x7fa2e0153620 2023-10-06T05:24:36.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.085+0000 7fa2c7fff700 1 -- 172.21.15.45:0/2209169943 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v228) v1 ==== 101+0+0 (secure 0 0 0) 0x7fa2d402a090 con 0x7fa2e0153620 2023-10-06T05:24:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2cc066f30 msgr2=0x7fa2cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2cc066f30 0x7fa2cc0693e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006fd0 tx=0x7fa2d0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e0153620 msgr2=0x7fa2e01694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e0153620 0x7fa2e01694b0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009290 tx=0x7fa2d4016000 comp rx=0 tx=0).stop 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 shutdown_connections 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2cc066f30 0x7fa2cc0693e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e015b2b0 0x7fa2e00c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e00cc8b0 0x7fa2e00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 --2- 172.21.15.45:0/2209169943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e0153620 0x7fa2e01694b0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.088+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 >> 172.21.15.45:0/2209169943 conn(0x7fa2e006ed50 msgr2=0x7fa2e0070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.089+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 shutdown_connections 2023-10-06T05:24:36.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.089+0000 7fa2e6906700 1 -- 172.21.15.45:0/2209169943 wait complete. 2023-10-06T05:24:36.092 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:36.105 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.151+0000 7f7b2697e040 1 Processor -- start 2023-10-06T05:24:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.151+0000 7f7b2697e040 1 -- start start 2023-10-06T05:24:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2697e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126214e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2697e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 0x56126217ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2697e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x56126217f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2697e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126209b980 con 0x561262210c60 2023-10-06T05:24:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2697e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126209b620 con 0x561262210890 2023-10-06T05:24:36.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2697e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561262092400 con 0x56126214e210 2023-10-06T05:24:36.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2039d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x56126217f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b1fb9c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126214e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2039d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x56126217f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58612/0 (socket says 172.21.15.45:58612) 2023-10-06T05:24:36.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b2039d700 1 -- 172.21.15.45:0/1648118011 learned_addr learned my addr 172.21.15.45:0/1648118011 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:36.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.152+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 0x56126217ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 msgr2=0x56126217ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 0x56126217ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/1648118011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 msgr2=0x56126217f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x56126217f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/1648118011 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08005db0 con 0x56126214e210 2023-10-06T05:24:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126214e850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000b310 tx=0x7f7b10001810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e673974a43d857e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1e399700 1 -- 172.21.15.45:0/1648118011 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10014070 con 0x56126214e210 2023-10-06T05:24:36.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1e399700 1 -- 172.21.15.45:0/1648118011 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1000f450 con 0x56126214e210 2023-10-06T05:24:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.153+0000 7f7b1e399700 1 -- 172.21.15.45:0/1648118011 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1000f7f0 con 0x56126214e210 2023-10-06T05:24:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 -- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 msgr2=0x56126214e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126214e850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000b310 tx=0x7f7b10001810 comp rx=0 tx=0).stop 2023-10-06T05:24:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 -- 172.21.15.45:0/1648118011 shutdown_connections 2023-10-06T05:24:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x56126217f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126214e850 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 --2- 172.21.15.45:0/1648118011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 0x56126217ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 -- 172.21.15.45:0/1648118011 >> 172.21.15.45:0/1648118011 conn(0x561262151450 msgr2=0x561262180770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 -- 172.21.15.45:0/1648118011 shutdown_connections 2023-10-06T05:24:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 -- 172.21.15.45:0/1648118011 wait complete. 2023-10-06T05:24:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 Processor -- start 2023-10-06T05:24:36.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.154+0000 7f7b2697e040 1 -- start start 2023-10-06T05:24:36.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2697e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126223b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2697e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 0x56126223b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2697e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x5612621774c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2697e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126222e730 con 0x561262210c60 2023-10-06T05:24:36.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2697e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561262175330 con 0x561262210890 2023-10-06T05:24:36.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2697e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612621755d0 con 0x56126214e210 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2039d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x5612621774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2039d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x5612621774c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58620/0 (socket says 172.21.15.45:58620) 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.155+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 learned_addr learned my addr 172.21.15.45:0/345828836 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 0x56126223b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126223b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 msgr2=0x56126223b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2039d700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56126214e210 0x56126223b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 msgr2=0x56126223b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2039d700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561262210890 0x56126223b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b10001af0 con 0x561262210c60 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2039d700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x5612621774c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800ac40 tx=0x7f7b18015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b1cb96700 1 -- 172.21.15.45:0/345828836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800fe80 con 0x561262210c60 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b1cb96700 1 -- 172.21.15.45:0/345828836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1800fa70 con 0x561262210c60 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b1cb96700 1 -- 172.21.15.45:0/345828836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18025650 con 0x561262210c60 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1800a0e0 con 0x561262210c60 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.156+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612620aa3d0 con 0x561262210c60 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.158+0000 7f7b1cb96700 1 -- 172.21.15.45:0/345828836 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b18003040 con 0x561262210c60 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.158+0000 7f7b1cb96700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b000658c0 0x7f7b00067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.158+0000 7f7b1cb96700 1 -- 172.21.15.45:0/345828836 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f7b18085e80 con 0x561262210c60 2023-10-06T05:24:36.168 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.158+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b000658c0 0x7f7b00067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.158+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56126214ec30 0x561262178400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.158+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126223bba0 con 0x56126214ec30 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.158+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56126214ec30 0x561262178400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.159+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56126214ec30 0x561262178400 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.159+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b000658c0 0x7f7b00067d70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001850 tx=0x7f7b10016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.161+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v114'660 uv634 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.161+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262173750 con 0x56126214ec30 2023-10-06T05:24:36.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.162+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v114'661 uv661 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.162+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262173b30 con 0x56126214ec30 2023-10-06T05:24:36.170 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.164+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v114'662 uv662 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.164+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262173f10 con 0x56126214ec30 2023-10-06T05:24:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.165+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'663 uv663 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.165+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262174300 con 0x56126214ec30 2023-10-06T05:24:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.166+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'664 uv664 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.166+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244220 con 0x56126214ec30 2023-10-06T05:24:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.168+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'665 uv665 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.168+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.169+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v114'666 uv640 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.169+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.170+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v114'667 uv667 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.170+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.173 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.171+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v114'668 uv668 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.172+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622455e0 con 0x56126214ec30 2023-10-06T05:24:36.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.173+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'669 uv669 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.173+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622455e0 con 0x56126214ec30 2023-10-06T05:24:36.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.174+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'670 uv670 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.174+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.175+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'671 uv671 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.175+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.176+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v114'672 uv647 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.176+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.178+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'673 uv673 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.178+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.180 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.179+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v114'674 uv674 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.179+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622455e0 con 0x56126214ec30 2023-10-06T05:24:36.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.180+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'675 uv675 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.180+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622455e0 con 0x56126214ec30 2023-10-06T05:24:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.181+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'676 uv676 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.182+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622445d0 con 0x56126214ec30 2023-10-06T05:24:36.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.183+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'677 uv677 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.183+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.184+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'678 uv678 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.184+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.186+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v114'679 uv653 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.186+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.187+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v114'680 uv680 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.187+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.190 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:36.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.188+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v114'681 uv681 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.188+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262245ad0 con 0x56126214ec30 2023-10-06T05:24:36.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.189+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'682 uv682 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.189+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262245ad0 con 0x56126214ec30 2023-10-06T05:24:36.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.191+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'683 uv683 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.191+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.192+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'684 uv684 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.192+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.193+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v114'685 uv659 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.194+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.195+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v114'686 uv686 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.195+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.198 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:36.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.196+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v114'687 uv687 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.196+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262245ad0 con 0x56126214ec30 2023-10-06T05:24:36.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.198+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'688 uv688 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.198+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262245ad0 con 0x56126214ec30 2023-10-06T05:24:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.199+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'689 uv689 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.199+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262244ac0 con 0x56126214ec30 2023-10-06T05:24:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.200+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'690 uv690 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126214ec30 2023-10-06T05:24:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.200+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561262244fd0 0x5612622473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.200+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622478f0 con 0x561262244fd0 2023-10-06T05:24:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.200+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561262244fd0 0x5612622473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.201+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561262244fd0 0x5612622473b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.202+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'479 uv462 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.202+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262247df0 con 0x561262244fd0 2023-10-06T05:24:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.204+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'480 uv480 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.204+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622482c0 con 0x561262244fd0 2023-10-06T05:24:36.207 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:36.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.205+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'481 uv481 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.205+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262248c00 con 0x561262244fd0 2023-10-06T05:24:36.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.206+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v114'482 uv482 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.207+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262248790 con 0x561262244fd0 2023-10-06T05:24:36.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.208+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'483 uv483 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.208+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622494a0 con 0x561262244fd0 2023-10-06T05:24:36.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.209+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v114'484 uv467 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.209+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249850 con 0x561262244fd0 2023-10-06T05:24:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.210+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v114'485 uv485 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.210+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.213 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:36.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.212+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v114'486 uv486 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.212+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.213+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'487 uv487 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.213+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.214+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'488 uv488 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.214+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.215+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v114'489 uv472 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.215+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.217+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v114'490 uv490 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.217+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.219 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.218+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v114'491 uv491 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.218+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224d320 con 0x561262244fd0 2023-10-06T05:24:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.219+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v114'492 uv492 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.220+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249030 con 0x561262244fd0 2023-10-06T05:24:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.221+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'493 uv493 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.221+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249030 con 0x561262244fd0 2023-10-06T05:24:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.222+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v114'494 uv478 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.222+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.223+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v114'495 uv495 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.223+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.226 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.224+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v114'496 uv496 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.224+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224ac10 con 0x561262244fd0 2023-10-06T05:24:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.226+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'497 uv497 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.226+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224ac10 con 0x561262244fd0 2023-10-06T05:24:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.227+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'498 uv498 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.227+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262249c00 con 0x561262244fd0 2023-10-06T05:24:36.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.228+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'499 uv499 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.228+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56126224a110 0x56126224c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.228+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224cb10 con 0x56126224a110 2023-10-06T05:24:36.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.229+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56126224a110 0x56126224c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.229+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56126224a110 0x56126224c4f0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.231+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.231+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224cef0 con 0x56126224a110 2023-10-06T05:24:36.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.232+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v114'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.232+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224cef0 con 0x56126224a110 2023-10-06T05:24:36.234 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:36.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.233+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v114'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.233+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224df00 con 0x56126224a110 2023-10-06T05:24:36.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.234+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.234+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224df00 con 0x56126224a110 2023-10-06T05:24:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.235+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.235+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224cef0 con 0x56126224a110 2023-10-06T05:24:36.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.236+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.236+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224cef0 con 0x56126224a110 2023-10-06T05:24:36.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.238+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'92 uv91 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.238+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224cef0 con 0x56126224a110 2023-10-06T05:24:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.239+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'93 uv93 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.239+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224cef0 con 0x56126224a110 2023-10-06T05:24:36.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.240+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'94 uv94 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.242 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.240+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224d2f0 con 0x56126224a110 2023-10-06T05:24:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.241+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'95 uv95 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.241+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224d020 con 0x561262244fd0 2023-10-06T05:24:36.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.242+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.242+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224d020 con 0x561262244fd0 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.244+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v114'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.244+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224d020 con 0x561262244fd0 2023-10-06T05:24:36.246 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:36.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.245+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v114'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.245+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224e030 con 0x561262244fd0 2023-10-06T05:24:36.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.246+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.246+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224e030 con 0x561262244fd0 2023-10-06T05:24:36.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.247+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.247+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126224d020 con 0x561262244fd0 2023-10-06T05:24:36.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.248+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262244fd0 2023-10-06T05:24:36.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.248+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56126224d930 0x56126224fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.248+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262250250 con 0x56126224d930 2023-10-06T05:24:36.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.249+0000 7f7b2039d700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56126224d930 0x56126224fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.249+0000 7f7b2039d700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56126224d930 0x56126224fd10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.250+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v114'238 uv232 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.250+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262250600 con 0x56126224d930 2023-10-06T05:24:36.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.251+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v114'239 uv239 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.252+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262250ac0 con 0x56126224d930 2023-10-06T05:24:36.254 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:36.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.253+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v114'240 uv240 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.253+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622490b0 con 0x56126224d930 2023-10-06T05:24:36.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.254+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'241 uv241 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.254+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622524e0 con 0x56126224d930 2023-10-06T05:24:36.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.255+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'242 uv242 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.255+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262251430 con 0x56126224d930 2023-10-06T05:24:36.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.256+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'243 uv243 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.256+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262251430 con 0x56126224d930 2023-10-06T05:24:36.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.257+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v114'244 uv237 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.257+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622513b0 con 0x56126224d930 2023-10-06T05:24:36.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.258+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v114'245 uv245 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.259+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622513b0 con 0x56126224d930 2023-10-06T05:24:36.261 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:36.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.260+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v114'246 uv246 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.260+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622513b0 con 0x56126224d930 2023-10-06T05:24:36.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.261+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'247 uv247 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.261+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x5612622518d0 con 0x56126224d930 2023-10-06T05:24:36.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.262+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'248 uv248 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.262+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561262251880 0x5612622548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.262+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262254e30 con 0x561262251880 2023-10-06T05:24:36.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.262+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561262251880 0x5612622548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.263+0000 7f7b1fb9c700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561262251880 0x5612622548f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.264+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v114'107 uv106 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262251880 2023-10-06T05:24:36.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.264+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262254e30 con 0x561262251880 2023-10-06T05:24:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.265+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v114'108 uv108 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262251880 2023-10-06T05:24:36.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.265+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262254e30 con 0x561262251880 2023-10-06T05:24:36.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.270+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v114'109 uv109 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262251880 2023-10-06T05:24:36.272 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:36.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.270+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262254e30 con 0x561262251880 2023-10-06T05:24:36.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.272+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'110 uv110 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262251880 2023-10-06T05:24:36.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.272+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262255260 con 0x561262251880 2023-10-06T05:24:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.275+0000 7f7b1fb9c700 1 -- 172.21.15.45:0/345828836 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'111 uv111 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b10016070 con 0x561262251880 2023-10-06T05:24:36.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.275+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262255260 con 0x56126224a110 2023-10-06T05:24:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.276+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.276+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262255260 con 0x56126224a110 2023-10-06T05:24:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.277+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v114'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.277+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262255260 con 0x56126224a110 2023-10-06T05:24:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.278+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v114'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.280 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.279+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262256270 con 0x56126224a110 2023-10-06T05:24:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.280+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.280+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262256270 con 0x56126224a110 2023-10-06T05:24:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.281+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.281+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262255260 con 0x56126224a110 2023-10-06T05:24:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.282+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.282+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561262255770 0x561262257b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.282+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258120 con 0x561262255770 2023-10-06T05:24:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.282+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561262255770 0x561262257b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.283+0000 7f7b1f39b700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561262255770 0x561262257b50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.284+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v114'213 uv207 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.284+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258160 con 0x561262255770 2023-10-06T05:24:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.286+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v114'214 uv214 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.286+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258560 con 0x561262255770 2023-10-06T05:24:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.287+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v114'215 uv215 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.288 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.287+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258560 con 0x561262255770 2023-10-06T05:24:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.288+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'216 uv216 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.288+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258560 con 0x561262255770 2023-10-06T05:24:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.289+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'217 uv217 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.289+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258560 con 0x561262255770 2023-10-06T05:24:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.290+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v114'218 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.290+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258560 con 0x561262255770 2023-10-06T05:24:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.292+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v114'219 uv219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.292+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258560 con 0x561262255770 2023-10-06T05:24:36.294 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.293+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v114'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.293+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258560 con 0x561262255770 2023-10-06T05:24:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.294+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.294+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126225b2d0 con 0x561262255770 2023-10-06T05:24:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.296+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x561262255770 2023-10-06T05:24:36.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.296+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.297+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v114'95 uv94 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.297+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.298+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'96 uv96 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.298+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.301 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:36.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.299+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v114'97 uv97 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.299+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.301+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'98 uv98 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.301+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561262259330 0x56126225ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.301+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x561262259330 2023-10-06T05:24:36.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.301+0000 7f7b2039d700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561262259330 0x56126225ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.301+0000 7f7b2039d700 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561262259330 0x56126225ba20 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.302+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x561262259330 2023-10-06T05:24:36.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.303+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x561262259330 2023-10-06T05:24:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.303+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v114'130 uv130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x561262259330 2023-10-06T05:24:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.304+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x561262259330 2023-10-06T05:24:36.308 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:36.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.306+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v114'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x561262259330 2023-10-06T05:24:36.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.306+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x561262259330 2023-10-06T05:24:36.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.309+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x561262259330 2023-10-06T05:24:36.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.309+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x561262259330 2023-10-06T05:24:36.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.311+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x561262259330 2023-10-06T05:24:36.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.312+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x561262259330 2023-10-06T05:24:36.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.314+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x561262259330 2023-10-06T05:24:36.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.314+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224d930 2023-10-06T05:24:36.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.316+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v114'110 uv109 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.316+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224d930 2023-10-06T05:24:36.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.317+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v114'111 uv111 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.317+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224d930 2023-10-06T05:24:36.320 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:36.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.318+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v114'112 uv112 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.318+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224d930 2023-10-06T05:24:36.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.320+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'113 uv113 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.320+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224d930 2023-10-06T05:24:36.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.321+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'114 uv114 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7b18016020 con 0x56126224d930 2023-10-06T05:24:36.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.321+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.323+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v114'265 uv257 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.323+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.324+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v114'266 uv266 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.324+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.327 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.325+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v114'267 uv267 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.325+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.327+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v114'268 uv268 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.327+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.328+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v114'269 uv269 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.328+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.329+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v114'270 uv264 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.329+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.330+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v114'271 uv271 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.330+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.333 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:36.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.332+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v114'272 uv272 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.332+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.333+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v114'273 uv273 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.333+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.334+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v114'274 uv274 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.334+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x561262258a50 con 0x56126224a110 2023-10-06T05:24:36.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.335+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v114'275 uv275 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.336+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x56126225bf60 con 0x56126224a110 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b1f39b700 1 -- 172.21.15.45:0/345828836 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v114'276 uv276 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7b08009c20 con 0x56126224a110 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56126224a110 msgr2=0x56126224c4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56126224a110 0x56126224c4f0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56126214ec30 msgr2=0x561262178400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56126214ec30 0x561262178400 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561262255770 msgr2=0x561262257b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x561262255770 0x561262257b50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561262244fd0 msgr2=0x5612622473b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x561262244fd0 0x5612622473b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 reap_dead start 2023-10-06T05:24:36.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561262259330 msgr2=0x56126225ba20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x561262259330 0x56126225ba20 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2039d700 1 -- 172.21.15.45:0/345828836 reap_dead start 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561262251880 msgr2=0x5612622548f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561262251880 0x5612622548f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56126224d930 msgr2=0x56126224fd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56126224d930 0x56126224fd10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b000658c0 msgr2=0x7f7b00067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b000658c0 0x7f7b00067d70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001850 tx=0x7f7b10016000 comp rx=0 tx=0).stop 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 msgr2=0x5612621774c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x5612621774c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800ac40 tx=0x7f7b18015000 comp rx=0 tx=0).stop 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 shutdown_connections 2023-10-06T05:24:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56126224d930 0x56126224fd10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x561262251880 0x5612622548f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561262210c60 0x5612621774c0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 --2- 172.21.15.45:0/345828836 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7b000658c0 0x7f7b00067d70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 >> 172.21.15.45:0/345828836 conn(0x561262151450 msgr2=0x561262153830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 shutdown_connections 2023-10-06T05:24:36.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.337+0000 7f7b2697e040 1 -- 172.21.15.45:0/345828836 wait complete. 2023-10-06T05:24:36.343 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2023-10-06T05:24:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.391+0000 7f2b1f39e040 1 Processor -- start 2023-10-06T05:24:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.391+0000 7f2b1f39e040 1 -- start start 2023-10-06T05:24:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b1f39e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1fe4c210 0x558f1fe7cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b1f39e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe7d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b1f39e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1ff08ad0 0x558f1ff08ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b1f39e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1fdf5320 con 0x558f1ff08ad0 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b1f39e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1fd99980 con 0x558f1fe4c210 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b1f39e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f1fd99620 con 0x558f1ff0e890 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1ff08ad0 0x558f1ff08ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1ff08ad0 0x558f1ff08ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42016/0 (socket says 172.21.15.45:42016) 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 -- 172.21.15.45:0/125874816 learned_addr learned my addr 172.21.15.45:0/125874816 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b17dbb700 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe7d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b185bc700 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1fe4c210 0x558f1fe7cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 -- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 msgr2=0x558f1fe7d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe7d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 -- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1fe4c210 msgr2=0x558f1fe7cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1fe4c210 0x558f1fe7cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 -- 172.21.15.45:0/125874816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08005db0 con 0x558f1ff08ad0 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b17dbb700 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe7d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b185bc700 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1fe4c210 0x558f1fe7cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b18dbd700 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1ff08ad0 0x558f1ff08ea0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f2b100016e0 tx=0x7f2b10006930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c75e08dec3303f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b16db9700 1 -- 172.21.15.45:0/125874816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000e470 con 0x558f1ff08ad0 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b16db9700 1 -- 172.21.15.45:0/125874816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1000e5d0 con 0x558f1ff08ad0 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.392+0000 7f2b16db9700 1 -- 172.21.15.45:0/125874816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000e990 con 0x558f1ff08ad0 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 -- 172.21.15.45:0/125874816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1ff08ad0 msgr2=0x558f1ff08ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1ff08ad0 0x558f1ff08ea0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f2b100016e0 tx=0x7f2b10006930 comp rx=0 tx=0).stop 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 -- 172.21.15.45:0/125874816 shutdown_connections 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1ff08ad0 0x558f1ff08ea0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1fe4c210 0x558f1fe7cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/125874816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe7d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 -- 172.21.15.45:0/125874816 >> 172.21.15.45:0/125874816 conn(0x558f1fe4f450 msgr2=0x558f1fe7a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 -- 172.21.15.45:0/125874816 shutdown_connections 2023-10-06T05:24:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.393+0000 7f2b1f39e040 1 -- 172.21.15.45:0/125874816 wait complete. 2023-10-06T05:24:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.394+0000 7f2b1f39e040 1 Processor -- start 2023-10-06T05:24:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.394+0000 7f2b1f39e040 1 -- start start 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b1f39e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1fe4c210 0x558f1ff0d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b1f39e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1ff08ad0 0x558f1ff07d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b1f39e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe931a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b1f39e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1fd90400 con 0x558f1fe4c210 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b1f39e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1ff082c0 con 0x558f1ff08ad0 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b1f39e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f1fe717f0 con 0x558f1ff0e890 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b185bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1fe4c210 0x558f1ff0d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b185bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1fe4c210 0x558f1ff0d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42032/0 (socket says 172.21.15.45:42032) 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b185bc700 1 -- 172.21.15.45:0/2885933060 learned_addr learned my addr 172.21.15.45:0/2885933060 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b17dbb700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1ff08ad0 0x558f1ff07d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.395+0000 7f2b18dbd700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b185bc700 1 -- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 msgr2=0x558f1fe931a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b185bc700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558f1ff0e890 0x558f1fe931a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b185bc700 1 -- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1ff08ad0 msgr2=0x558f1ff07d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b185bc700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f1ff08ad0 0x558f1ff07d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b185bc700 1 -- 172.21.15.45:0/2885933060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b10006bf0 con 0x558f1fe4c210 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b185bc700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1fe4c210 0x558f1ff0d610 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000a320 tx=0x7f2b00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b155b6700 1 -- 172.21.15.45:0/2885933060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000ae40 con 0x558f1fe4c210 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b155b6700 1 -- 172.21.15.45:0/2885933060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0000f420 con 0x558f1fe4c210 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b155b6700 1 -- 172.21.15.45:0/2885933060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000f7c0 con 0x558f1fe4c210 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f1fd9bae0 con 0x558f1fe4c210 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.396+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f1fd9b540 con 0x558f1fe4c210 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.398+0000 7f2b155b6700 1 -- 172.21.15.45:0/2885933060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b0000fbf0 con 0x558f1fe4c210 2023-10-06T05:24:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.398+0000 7f2b155b6700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2af80658c0 0x7f2af8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.398+0000 7f2b155b6700 1 -- 172.21.15.45:0/2885933060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f2b00008040 con 0x558f1fe4c210 2023-10-06T05:24:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.398+0000 7f2b17dbb700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2af80658c0 0x7f2af8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.398+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f1fe70740 0x558f1fe70b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.398+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) v8 -- 0x558f1fe72470 con 0x558f1fe70740 2023-10-06T05:24:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.399+0000 7f2b18dbd700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f1fe70740 0x558f1fe70b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.399+0000 7f2b17dbb700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2af80658c0 0x7f2af8067d70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x558f1fe52e50 tx=0x7f2b08014000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.399+0000 7f2b18dbd700 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f1fe70740 0x558f1fe70b10 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b18dbd700 1 -- 172.21.15.45:0/2885933060 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v114'277 uv277 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2b1000e740 con 0x558f1fe70740 2023-10-06T05:24:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f1fe70740 msgr2=0x558f1fe70b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558f1fe70740 0x558f1fe70b10 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2af80658c0 msgr2=0x7f2af8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2af80658c0 0x7f2af8067d70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x558f1fe52e50 tx=0x7f2b08014000 comp rx=0 tx=0).stop 2023-10-06T05:24:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1fe4c210 msgr2=0x558f1ff0d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 --2- 172.21.15.45:0/2885933060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f1fe4c210 0x558f1ff0d610 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000a320 tx=0x7f2b00000bc0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b18dbd700 1 -- 172.21.15.45:0/2885933060 reap_dead start 2023-10-06T05:24:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 shutdown_connections 2023-10-06T05:24:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 >> 172.21.15.45:0/2885933060 conn(0x558f1fe4f450 msgr2=0x558f1fe7a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 shutdown_connections 2023-10-06T05:24:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.401+0000 7f2b1f39e040 1 -- 172.21.15.45:0/2885933060 wait complete. 2023-10-06T05:24:36.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:36.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.588+0000 7f13f625d700 1 Processor -- start 2023-10-06T05:24:36.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.588+0000 7f13f625d700 1 -- start start 2023-10-06T05:24:36.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13f625d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f13f015a3d0 0x7f13f00ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13f625d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f00cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13f625d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f00ce770 con 0x7f13f0153840 2023-10-06T05:24:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13f625d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f00cd4a0 con 0x7f13f015a3d0 2023-10-06T05:24:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13f625d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0153c10 con 0x7f13f00cd060 2023-10-06T05:24:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13eeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f13f015a3d0 0x7f13f00ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13eeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f13f015a3d0 0x7f13f00ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42044/0 (socket says 172.21.15.45:42044) 2023-10-06T05:24:36.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13ef7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f13f0153840 0x7f13f015cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56424/0 (socket says 172.21.15.45:56424) 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13eeffd700 1 -- 172.21.15.45:0/1420252355 learned_addr learned my addr 172.21.15.45:0/1420252355 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13effff700 1 --2- 172.21.15.45:0/1420252355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f00cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13eeffd700 1 -- 172.21.15.45:0/1420252355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 msgr2=0x7f13f00cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13eeffd700 1 --2- 172.21.15.45:0/1420252355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f00cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13eeffd700 1 -- 172.21.15.45:0/1420252355 >> v1:172.21.15.45:6789/0 conn(0x7f13f0153840 legacy=0x7f13f015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13eeffd700 1 -- 172.21.15.45:0/1420252355 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f015d650 con 0x7f13f015a3d0 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.589+0000 7f13ee7fc700 1 -- 172.21.15.45:0/1420252355 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862577179 0 0) 0x7f13f00ce770 con 0x7f13f0153840 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13eeffd700 1 --2- 172.21.15.45:0/1420252355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f015a3d0 0x7f13f00ccb20 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f13d80099b0 tx=0x7f13d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967e4eada2c7ec90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13ee7fc700 1 -- 172.21.15.45:0/1420252355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8009ea0 con 0x7f13f015a3d0 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13ee7fc700 1 -- 172.21.15.45:0/1420252355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d800b8f0 con 0x7f13f015a3d0 2023-10-06T05:24:36.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13ee7fc700 1 -- 172.21.15.45:0/1420252355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800bc90 con 0x7f13f015a3d0 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13f625d700 1 -- 172.21.15.45:0/1420252355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f015a3d0 msgr2=0x7f13f00ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13f625d700 1 --2- 172.21.15.45:0/1420252355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f015a3d0 0x7f13f00ccb20 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f13d80099b0 tx=0x7f13d8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13f625d700 1 -- 172.21.15.45:0/1420252355 shutdown_connections 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13f625d700 1 --2- 172.21.15.45:0/1420252355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f00cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13f625d700 1 --2- 172.21.15.45:0/1420252355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f015a3d0 0x7f13f00ccb20 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.590+0000 7f13f625d700 1 -- 172.21.15.45:0/1420252355 >> 172.21.15.45:0/1420252355 conn(0x7f13f006ed50 msgr2=0x7f13f006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.591+0000 7f13f625d700 1 -- 172.21.15.45:0/1420252355 shutdown_connections 2023-10-06T05:24:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.591+0000 7f13f625d700 1 -- 172.21.15.45:0/1420252355 wait complete. 2023-10-06T05:24:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.592+0000 7f13f625d700 1 Processor -- start 2023-10-06T05:24:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.592+0000 7f13f625d700 1 -- start start 2023-10-06T05:24:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.593+0000 7f13f625d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f01588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.593+0000 7f13f625d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0153840 0x7f13f00ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.593+0000 7f13f625d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13f015a3d0 0x7f13f00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.593+0000 7f13f625d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f016ad40 con 0x7f13f0153840 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.593+0000 7f13f625d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00b4e30 con 0x7f13f015a3d0 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.593+0000 7f13f625d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00b5110 con 0x7f13f00cd060 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0153840 0x7f13f00ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0153840 0x7f13f00ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42046/0 (socket says 172.21.15.45:42046) 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 -- 172.21.15.45:0/306523481 learned_addr learned my addr 172.21.15.45:0/306523481 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13ef7fe700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f01588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13effff700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13f015a3d0 0x7f13f00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 -- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 msgr2=0x7f13f01588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f01588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 -- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13f015a3d0 msgr2=0x7f13f00ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13f015a3d0 0x7f13f00ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 -- 172.21.15.45:0/306523481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80094f0 con 0x7f13f0153840 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13eeffd700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0153840 0x7f13f00ca390 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006fd0 tx=0x7f13d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13e77fe700 1 -- 172.21.15.45:0/306523481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800bb30 con 0x7f13f0153840 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13e77fe700 1 -- 172.21.15.45:0/306523481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8026430 con 0x7f13f0153840 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f00b53f0 con 0x7f13f0153840 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13e77fe700 1 -- 172.21.15.45:0/306523481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800bc90 con 0x7f13f0153840 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.594+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f0158e00 con 0x7f13f0153840 2023-10-06T05:24:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.595+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f016aeb0 con 0x7f13f0153840 2023-10-06T05:24:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.596+0000 7f13e77fe700 1 -- 172.21.15.45:0/306523481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13d8023030 con 0x7f13f0153840 2023-10-06T05:24:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.597+0000 7f13e77fe700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f13d0066f30 0x7f13d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.597+0000 7f13e77fe700 1 -- 172.21.15.45:0/306523481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f13d8086ab0 con 0x7f13f0153840 2023-10-06T05:24:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.597+0000 7f13ef7fe700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f13d0066f30 0x7f13d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.598+0000 7f13ef7fe700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f13d0066f30 0x7f13d00693e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f13e00011e0 tx=0x7f13e0007bf0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.600+0000 7f13e77fe700 1 -- 172.21.15.45:0/306523481 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13d8053300 con 0x7f13f0153840 2023-10-06T05:24:36.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:36.848+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f13f006f990 con 0x7f13f0153840 2023-10-06T05:24:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.090+0000 7f13e77fe700 1 -- 172.21.15.45:0/306523481 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v229) v1 ==== 162+0+0 (secure 0 0 0) 0x7f13d80150b0 con 0x7f13f0153840 2023-10-06T05:24:37.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f13d0066f30 msgr2=0x7f13d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f13d0066f30 0x7f13d00693e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f13e00011e0 tx=0x7f13e0007bf0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0153840 msgr2=0x7f13f00ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0153840 0x7f13f00ca390 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006fd0 tx=0x7f13d8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 shutdown_connections 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f13d0066f30 0x7f13d00693e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13f015a3d0 0x7f13f00ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00cd060 0x7f13f01588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 --2- 172.21.15.45:0/306523481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0153840 0x7f13f00ca390 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.093+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 >> 172.21.15.45:0/306523481 conn(0x7f13f006ed50 msgr2=0x7f13f00cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.094+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 shutdown_connections 2023-10-06T05:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.094+0000 7f13f625d700 1 -- 172.21.15.45:0/306523481 wait complete. 2023-10-06T05:24:37.097 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:37.101 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:37.099+0000 7fa646322700 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer 2023-10-06T05:24:37.101 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-10-06T05:24:37.099+0000 7fa646322700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:24:37.101 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:37.100+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:37.102 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:37.100+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:37.109 INFO:tasks.ceph.mds.c.smithi045.stderr: -23> 2023-10-06T05:24:37.099+0000 7fa646322700 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer 2023-10-06T05:24:37.109 INFO:tasks.ceph.mds.c.smithi045.stderr: -13> 2023-10-06T05:24:37.099+0000 7fa646322700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:24:37.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:37.132 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:24:37.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.246+0000 7fdae9432700 1 Processor -- start 2023-10-06T05:24:37.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.246+0000 7fdae9432700 1 -- start start 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.246+0000 7fdae9432700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdae40692d0 0x7fdae4159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae9432700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae40ca6d0 con 0x7fdae4159170 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae9432700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae415d550 con 0x7fdae415d040 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae9432700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae415d810 con 0x7fdae40692d0 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae27fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdae40692d0 0x7fdae4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae2ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdae4159170 0x7fdae4068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56448/0 (socket says 172.21.15.45:56448) 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae2ffd700 1 -- 172.21.15.45:0/3980185855 learned_addr learned my addr 172.21.15.45:0/3980185855 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae27fc700 1 -- 172.21.15.45:0/3980185855 >> v1:172.21.15.144:6789/0 conn(0x7fdae415d040 legacy=0x7fdae40aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae27fc700 1 -- 172.21.15.45:0/3980185855 >> v1:172.21.15.45:6789/0 conn(0x7fdae4159170 legacy=0x7fdae4068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae27fc700 1 -- 172.21.15.45:0/3980185855 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae415daf0 con 0x7fdae40692d0 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae1ffb700 1 -- 172.21.15.45:0/3980185855 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371545964 0 0) 0x7fdae40ca6d0 con 0x7fdae4159170 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae27fc700 1 --2- 172.21.15.45:0/3980185855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae40692d0 0x7fdae4159540 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fdad80098e0 tx=0x7fdad80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5619264bc5797d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae1ffb700 1 -- 172.21.15.45:0/3980185855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8009e80 con 0x7fdae40692d0 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae1ffb700 1 -- 172.21.15.45:0/3980185855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad800b860 con 0x7fdae40692d0 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.247+0000 7fdae1ffb700 1 -- 172.21.15.45:0/3980185855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad800bbe0 con 0x7fdae40692d0 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.248+0000 7fdae9432700 1 -- 172.21.15.45:0/3980185855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae40692d0 msgr2=0x7fdae4159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.248+0000 7fdae9432700 1 --2- 172.21.15.45:0/3980185855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae40692d0 0x7fdae4159540 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fdad80098e0 tx=0x7fdad80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.248+0000 7fdae9432700 1 -- 172.21.15.45:0/3980185855 shutdown_connections 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.248+0000 7fdae9432700 1 --2- 172.21.15.45:0/3980185855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae40692d0 0x7fdae4159540 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.248+0000 7fdae9432700 1 -- 172.21.15.45:0/3980185855 >> 172.21.15.45:0/3980185855 conn(0x7fdae406f7c0 msgr2=0x7fdae40d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.248+0000 7fdae9432700 1 -- 172.21.15.45:0/3980185855 shutdown_connections 2023-10-06T05:24:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.248+0000 7fdae9432700 1 -- 172.21.15.45:0/3980185855 wait complete. 2023-10-06T05:24:37.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.249+0000 7fdae9432700 1 Processor -- start 2023-10-06T05:24:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae9432700 1 -- start start 2023-10-06T05:24:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae9432700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdae40692d0 0x7fdae414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae9432700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4159170 0x7fdae4147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae9432700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae415d040 0x7fdae4147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae9432700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4148ec0 con 0x7fdae415d040 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae9432700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4149350 con 0x7fdae4159170 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae9432700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae41482b0 con 0x7fdae40692d0 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.250+0000 7fdae37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae415d040 0x7fdae4147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae415d040 0x7fdae4147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42092/0 (socket says 172.21.15.45:42092) 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 -- 172.21.15.45:0/732222161 learned_addr learned my addr 172.21.15.45:0/732222161 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae27fc700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4159170 0x7fdae4147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae2ffd700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdae40692d0 0x7fdae414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 -- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdae40692d0 msgr2=0x7fdae414c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdae40692d0 0x7fdae414c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 -- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4159170 msgr2=0x7fdae4147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4159170 0x7fdae4147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 -- 172.21.15.45:0/732222161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8009420 con 0x7fdae415d040 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae37fe700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae415d040 0x7fdae4147a80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c840 tx=0x7fdadc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdacaffd700 1 -- 172.21.15.45:0/732222161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc012630 con 0x7fdae415d040 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdacaffd700 1 -- 172.21.15.45:0/732222161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc0127b0 con 0x7fdae415d040 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae4148590 con 0x7fdae415d040 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdacaffd700 1 -- 172.21.15.45:0/732222161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc012b70 con 0x7fdae415d040 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.251+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae4148b50 con 0x7fdae415d040 2023-10-06T05:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.252+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae4149030 con 0x7fdae415d040 2023-10-06T05:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.253+0000 7fdacaffd700 1 -- 172.21.15.45:0/732222161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdadc01a3a0 con 0x7fdae415d040 2023-10-06T05:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.253+0000 7fdacaffd700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdad0066f30 0x7fdad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.253+0000 7fdacaffd700 1 -- 172.21.15.45:0/732222161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7fdadc085160 con 0x7fdae415d040 2023-10-06T05:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.253+0000 7fdae2ffd700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdad0066f30 0x7fdad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.254+0000 7fdae2ffd700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdad0066f30 0x7fdad00693e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdae415d010 tx=0x7fdad4007d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.258+0000 7fdacaffd700 1 -- 172.21.15.45:0/732222161 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdadc0518d0 con 0x7fdae415d040 2023-10-06T05:24:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.508+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdae4063b50 con 0x7fdae415d040 2023-10-06T05:24:37.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.509+0000 7fdacaffd700 1 -- 172.21.15.45:0/732222161 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 231 v231) v1 ==== 95+0+3842 (secure 0 0 0) 0x7fdadc054ef0 con 0x7fdae415d040 2023-10-06T05:24:37.511 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:37.511 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":231,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":8875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2156055156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2156055156},{"type":"v1","addr":"172.21.15.144:6835","nonce":2156055156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226}],"filesystems":[{"mdsmap":{"epoch":231,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:37.101213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdad0066f30 msgr2=0x7fdad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdad0066f30 0x7fdad00693e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdae415d010 tx=0x7fdad4007d10 comp rx=0 tx=0).stop 2023-10-06T05:24:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae415d040 msgr2=0x7fdae4147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae415d040 0x7fdae4147a80 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c840 tx=0x7fdadc001670 comp rx=0 tx=0).stop 2023-10-06T05:24:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 shutdown_connections 2023-10-06T05:24:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdad0066f30 0x7fdad00693e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae415d040 0x7fdae4147a80 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdae40692d0 0x7fdae414c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 --2- 172.21.15.45:0/732222161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4159170 0x7fdae4147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 >> 172.21.15.45:0/732222161 conn(0x7fdae406f7c0 msgr2=0x7fdae4068bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 shutdown_connections 2023-10-06T05:24:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.511+0000 7fdae9432700 1 -- 172.21.15.45:0/732222161 wait complete. 2023-10-06T05:24:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 231 2023-10-06T05:24:37.528 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:37.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:37.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.699+0000 7f01062b2700 1 Processor -- start 2023-10-06T05:24:37.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.699+0000 7f01062b2700 1 -- start start 2023-10-06T05:24:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.700+0000 7f01062b2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0100155cd0 0x7f010015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.700+0000 7f01062b2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01000ae8d0 con 0x7f01000ccdc0 2023-10-06T05:24:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.700+0000 7f01062b2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01000aebb0 con 0x7f01001560a0 2023-10-06T05:24:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.700+0000 7f01062b2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01000ae6b0 con 0x7f0100155cd0 2023-10-06T05:24:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.700+0000 7f00fffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0100155cd0 0x7f010015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.700+0000 7f010484f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01000ccdc0 0x7f01000cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56450/0 (socket says 172.21.15.45:56450) 2023-10-06T05:24:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.700+0000 7f010484f700 1 -- 172.21.15.45:0/606889460 learned_addr learned my addr 172.21.15.45:0/606889460 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.701+0000 7f00fffff700 1 -- 172.21.15.45:0/606889460 >> v1:172.21.15.144:6789/0 conn(0x7f01001560a0 legacy=0x7f01000adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.701+0000 7f00fffff700 1 -- 172.21.15.45:0/606889460 >> v1:172.21.15.45:6789/0 conn(0x7f01000ccdc0 legacy=0x7f01000cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.701+0000 7f00fffff700 1 -- 172.21.15.45:0/606889460 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000aee90 con 0x7f0100155cd0 2023-10-06T05:24:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.701+0000 7f00fffff700 1 --2- 172.21.15.45:0/606889460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100155cd0 0x7f010015a730 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f00f00098e0 tx=0x7f00f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7488129350d898ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.701+0000 7f00feffd700 1 -- 172.21.15.45:0/606889460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0009e80 con 0x7f0100155cd0 2023-10-06T05:24:37.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.701+0000 7f00feffd700 1 -- 172.21.15.45:0/606889460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f000b860 con 0x7f0100155cd0 2023-10-06T05:24:37.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.701+0000 7f00feffd700 1 -- 172.21.15.45:0/606889460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000bbe0 con 0x7f0100155cd0 2023-10-06T05:24:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.702+0000 7f01062b2700 1 -- 172.21.15.45:0/606889460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100155cd0 msgr2=0x7f010015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.702+0000 7f01062b2700 1 --2- 172.21.15.45:0/606889460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100155cd0 0x7f010015a730 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f00f00098e0 tx=0x7f00f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.702+0000 7f01062b2700 1 -- 172.21.15.45:0/606889460 shutdown_connections 2023-10-06T05:24:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.702+0000 7f01062b2700 1 --2- 172.21.15.45:0/606889460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0100155cd0 0x7f010015a730 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.702+0000 7f01062b2700 1 -- 172.21.15.45:0/606889460 >> 172.21.15.45:0/606889460 conn(0x7f010006f090 msgr2=0x7f010006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.702+0000 7f01062b2700 1 -- 172.21.15.45:0/606889460 shutdown_connections 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.702+0000 7f01062b2700 1 -- 172.21.15.45:0/606889460 wait complete. 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.703+0000 7f01062b2700 1 Processor -- start 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.703+0000 7f01062b2700 1 -- start start 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f01062b2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01000ccdc0 0x7f0100156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f01062b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0100155cd0 0x7f0100152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f01062b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001560a0 0x7f01000c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f01062b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0100157470 con 0x7f01001560a0 2023-10-06T05:24:37.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f01062b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001598a0 con 0x7f0100155cd0 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f01062b2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010016d1f0 con 0x7f01000ccdc0 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001560a0 0x7f01000c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001560a0 0x7f01000c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42098/0 (socket says 172.21.15.45:42098) 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 -- 172.21.15.45:0/2490268924 learned_addr learned my addr 172.21.15.45:0/2490268924 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f00fffff700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01000ccdc0 0x7f0100156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f00ff7fe700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0100155cd0 0x7f0100152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 -- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01000ccdc0 msgr2=0x7f0100156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01000ccdc0 0x7f0100156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 -- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0100155cd0 msgr2=0x7f0100152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0100155cd0 0x7f0100152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f010484f700 1 -- 172.21.15.45:0/2490268924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f0009420 con 0x7f01001560a0 2023-10-06T05:24:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f00fffff700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01000ccdc0 0x7f0100156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.704+0000 7f00ff7fe700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0100155cd0 0x7f0100152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:37.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.705+0000 7f010484f700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001560a0 0x7f01000c3e40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f00f800cbc0 tx=0x7f00f8001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.705+0000 7f00e7fff700 1 -- 172.21.15.45:0/2490268924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800a900 con 0x7f01001560a0 2023-10-06T05:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.705+0000 7f00e7fff700 1 -- 172.21.15.45:0/2490268924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f800aa60 con 0x7f01001560a0 2023-10-06T05:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.705+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01001528d0 con 0x7f01001560a0 2023-10-06T05:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.705+0000 7f00e7fff700 1 -- 172.21.15.45:0/2490268924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800ade0 con 0x7f01001560a0 2023-10-06T05:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.705+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0100156eb0 con 0x7f01001560a0 2023-10-06T05:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.706+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010016cfc0 con 0x7f01001560a0 2023-10-06T05:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.706+0000 7f00e7fff700 1 -- 172.21.15.45:0/2490268924 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00f8007300 con 0x7f01001560a0 2023-10-06T05:24:37.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.707+0000 7f00e7fff700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00ec066f30 0x7f00ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.707+0000 7f00e7fff700 1 -- 172.21.15.45:0/2490268924 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7f00f80853a0 con 0x7f01001560a0 2023-10-06T05:24:37.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.707+0000 7f00fffff700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00ec066f30 0x7f00ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.708+0000 7f00fffff700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00ec066f30 0x7f00ec0693e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f00f00093f0 tx=0x7f00f0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.711+0000 7f00e7fff700 1 -- 172.21.15.45:0/2490268924 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f010016cfc0 con 0x7f01001560a0 2023-10-06T05:24:37.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.957+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f010016cfc0 con 0x7f01001560a0 2023-10-06T05:24:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.958+0000 7f00e7fff700 1 -- 172.21.15.45:0/2490268924 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 231 v231) v1 ==== 95+0+3842 (secure 0 0 0) 0x7f010016cfc0 con 0x7f01001560a0 2023-10-06T05:24:37.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:37.960 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":231,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":8875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2156055156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2156055156},{"type":"v1","addr":"172.21.15.144:6835","nonce":2156055156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226}],"filesystems":[{"mdsmap":{"epoch":231,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:37.101213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:37.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00ec066f30 msgr2=0x7f00ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00ec066f30 0x7f00ec0693e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f00f00093f0 tx=0x7f00f0017000 comp rx=0 tx=0).stop 2023-10-06T05:24:37.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001560a0 msgr2=0x7f01000c3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001560a0 0x7f01000c3e40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f00f800cbc0 tx=0x7f00f8001990 comp rx=0 tx=0).stop 2023-10-06T05:24:37.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 shutdown_connections 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00ec066f30 0x7f00ec0693e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01001560a0 0x7f01000c3e40 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01000ccdc0 0x7f0100156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 --2- 172.21.15.45:0/2490268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0100155cd0 0x7f0100152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 >> 172.21.15.45:0/2490268924 conn(0x7f010006f090 msgr2=0x7f01000cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 shutdown_connections 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:37.960+0000 7f01062b2700 1 -- 172.21.15.45:0/2490268924 wait complete. 2023-10-06T05:24:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 231 2023-10-06T05:24:37.975 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:37.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:38.080 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:38.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:38.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.111+0000 7fe5d8569700 1 Processor -- start 2023-10-06T05:24:38.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.111+0000 7fe5d8569700 1 -- start start 2023-10-06T05:24:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d8569700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5d0158f60 0x7fe5d0159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d8569700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5d006a060 0x7fe5d00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d8569700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d0069bd0 con 0x7fe5d00cc390 2023-10-06T05:24:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d8569700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d006a500 con 0x7fe5d006a060 2023-10-06T05:24:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d8569700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d0068fc0 con 0x7fe5d0158f60 2023-10-06T05:24:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d5b04700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5d006a060 0x7fe5d00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d5b04700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5d006a060 0x7fe5d00cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42112/0 (socket says 172.21.15.45:42112) 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.112+0000 7fe5d5b04700 1 -- 172.21.15.45:0/125750612 learned_addr learned my addr 172.21.15.45:0/125750612 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d6305700 1 --2- 172.21.15.45:0/125750612 >> v2:172.21.15.144:3300/0 conn(0x7fe5d0158f60 0x7fe5d0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5b04700 1 -- 172.21.15.45:0/125750612 >> v2:172.21.15.144:3300/0 conn(0x7fe5d0158f60 msgr2=0x7fe5d0159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5b04700 1 --2- 172.21.15.45:0/125750612 >> v2:172.21.15.144:3300/0 conn(0x7fe5d0158f60 0x7fe5d0159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5b04700 1 -- 172.21.15.45:0/125750612 >> v1:172.21.15.144:6789/0 conn(0x7fe5d00cc390 legacy=0x7fe5d00c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5b04700 1 -- 172.21.15.45:0/125750612 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00692a0 con 0x7fe5d006a060 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5b04700 1 --2- 172.21.15.45:0/125750612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d006a060 0x7fe5d00cae20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fe5d006ed80 tx=0x7fe5c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba0ef2b76eb2250 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5303700 1 -- 172.21.15.45:0/125750612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0009e70 con 0x7fe5d006a060 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5303700 1 -- 172.21.15.45:0/125750612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c000b860 con 0x7fe5d006a060 2023-10-06T05:24:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.113+0000 7fe5d5303700 1 -- 172.21.15.45:0/125750612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000bc00 con 0x7fe5d006a060 2023-10-06T05:24:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 -- 172.21.15.45:0/125750612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d006a060 msgr2=0x7fe5d00cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 --2- 172.21.15.45:0/125750612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d006a060 0x7fe5d00cae20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fe5d006ed80 tx=0x7fe5c0009250 comp rx=0 tx=0).stop 2023-10-06T05:24:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 -- 172.21.15.45:0/125750612 shutdown_connections 2023-10-06T05:24:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 --2- 172.21.15.45:0/125750612 >> v2:172.21.15.144:3300/0 conn(0x7fe5d0158f60 0x7fe5d0159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 --2- 172.21.15.45:0/125750612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d006a060 0x7fe5d00cae20 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 -- 172.21.15.45:0/125750612 >> 172.21.15.45:0/125750612 conn(0x7fe5d00cdf50 msgr2=0x7fe5d00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 -- 172.21.15.45:0/125750612 shutdown_connections 2023-10-06T05:24:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.114+0000 7fe5d8569700 1 -- 172.21.15.45:0/125750612 wait complete. 2023-10-06T05:24:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.115+0000 7fe5d8569700 1 Processor -- start 2023-10-06T05:24:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.115+0000 7fe5d8569700 1 -- start start 2023-10-06T05:24:38.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d8569700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d006a060 0x7fe5d016a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d8569700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5d00cc390 0x7fe5d0154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d8569700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d0158f60 0x7fe5d00b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d8569700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d015a4c0 con 0x7fe5d0158f60 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d8569700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d01598d0 con 0x7fe5d006a060 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d8569700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0159b90 con 0x7fe5d00cc390 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d6b06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d0158f60 0x7fe5d00b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d6b06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d0158f60 0x7fe5d00b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42124/0 (socket says 172.21.15.45:42124) 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d6b06700 1 -- 172.21.15.45:0/2973887046 learned_addr learned my addr 172.21.15.45:0/2973887046 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d6305700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d006a060 0x7fe5d016a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.116+0000 7fe5d5b04700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5d00cc390 0x7fe5d0154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d6b06700 1 -- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5d00cc390 msgr2=0x7fe5d0154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d6b06700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5d00cc390 0x7fe5d0154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d6b06700 1 -- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d006a060 msgr2=0x7fe5d016a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d6b06700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d006a060 0x7fe5d016a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d6b06700 1 -- 172.21.15.45:0/2973887046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c00094f0 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d6b06700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d0158f60 0x7fe5d00b3270 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c8e0 tx=0x7fe5c8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5c67fc700 1 -- 172.21.15.45:0/2973887046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8012630 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5c67fc700 1 -- 172.21.15.45:0/2973887046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8012790 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d0159e70 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5c67fc700 1 -- 172.21.15.45:0/2973887046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8012b50 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.117+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d016aa90 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.118+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d015a850 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.119+0000 7fe5c67fc700 1 -- 172.21.15.45:0/2973887046 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5c801a3a0 con 0x7fe5d0158f60 2023-10-06T05:24:38.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.119+0000 7fe5c67fc700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5b4066f30 0x7fe5b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.119+0000 7fe5c67fc700 1 -- 172.21.15.45:0/2973887046 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7fe5c8085140 con 0x7fe5d0158f60 2023-10-06T05:24:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.119+0000 7fe5d6305700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5b4066f30 0x7fe5b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.120+0000 7fe5d6305700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5b4066f30 0x7fe5b40693e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0157b30 tx=0x7fe5cc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.123+0000 7fe5c67fc700 1 -- 172.21.15.45:0/2973887046 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5d015a850 con 0x7fe5d0158f60 2023-10-06T05:24:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.374+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5d015a850 con 0x7fe5d0158f60 2023-10-06T05:24:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.375+0000 7fe5c67fc700 1 -- 172.21.15.45:0/2973887046 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 232 v232) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fe5d015a850 con 0x7fe5d0158f60 2023-10-06T05:24:38.377 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:38.377 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":232,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":8875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2156055156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2156055156},{"type":"v1","addr":"172.21.15.144:6835","nonce":2156055156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226}],"filesystems":[{"mdsmap":{"epoch":231,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:37.101213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:38.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5b4066f30 msgr2=0x7fe5b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5b4066f30 0x7fe5b40693e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0157b30 tx=0x7fe5cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d0158f60 msgr2=0x7fe5d00b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d0158f60 0x7fe5d00b3270 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c8e0 tx=0x7fe5c8001710 comp rx=0 tx=0).stop 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 shutdown_connections 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe5b4066f30 0x7fe5b40693e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d0158f60 0x7fe5d00b3270 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5d006a060 0x7fe5d016a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 --2- 172.21.15.45:0/2973887046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5d00cc390 0x7fe5d0154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 >> 172.21.15.45:0/2973887046 conn(0x7fe5d00cdf50 msgr2=0x7fe5d0153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 shutdown_connections 2023-10-06T05:24:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.377+0000 7fe5d8569700 1 -- 172.21.15.45:0/2973887046 wait complete. 2023-10-06T05:24:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 232 2023-10-06T05:24:38.393 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2023-10-06T05:24:38.393 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2023-10-06T05:24:38.393 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:38.393 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:38.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:38.393 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:38.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:38.414 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.590+0000 7fac9a5fe700 1 Processor -- start 2023-10-06T05:24:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.590+0000 7fac9a5fe700 1 -- start start 2023-10-06T05:24:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac9a5fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac941595d0 0x7fac9406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac9a5fe700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac940c7d00 con 0x7fac9406b5c0 2023-10-06T05:24:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac9a5fe700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9406bb40 con 0x7fac94159200 2023-10-06T05:24:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac9a5fe700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac940c7780 con 0x7fac941595d0 2023-10-06T05:24:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac937fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac941595d0 0x7fac9406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac98b9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fac9406b5c0 0x7fac940ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56456/0 (socket says 172.21.15.45:56456) 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac98b9b700 1 -- 172.21.15.45:0/127008746 learned_addr learned my addr 172.21.15.45:0/127008746 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac937fe700 1 -- 172.21.15.45:0/127008746 >> v1:172.21.15.144:6789/0 conn(0x7fac94159200 legacy=0x7fac940c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac937fe700 1 -- 172.21.15.45:0/127008746 >> v1:172.21.15.45:6789/0 conn(0x7fac9406b5c0 legacy=0x7fac940ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac937fe700 1 -- 172.21.15.45:0/127008746 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9406be20 con 0x7fac941595d0 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac92ffd700 1 -- 172.21.15.45:0/127008746 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166608313 0 0) 0x7fac940c7d00 con 0x7fac9406b5c0 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.591+0000 7fac937fe700 1 --2- 172.21.15.45:0/127008746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac941595d0 0x7fac9406b080 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0098e0 tx=0x7fac7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd178b9a357ec4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac92ffd700 1 -- 172.21.15.45:0/127008746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c009e80 con 0x7fac941595d0 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac92ffd700 1 -- 172.21.15.45:0/127008746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c00b860 con 0x7fac941595d0 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac92ffd700 1 -- 172.21.15.45:0/127008746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00bbe0 con 0x7fac941595d0 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac9a5fe700 1 -- 172.21.15.45:0/127008746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac941595d0 msgr2=0x7fac9406b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/127008746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac941595d0 0x7fac9406b080 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0098e0 tx=0x7fac7c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac9a5fe700 1 -- 172.21.15.45:0/127008746 shutdown_connections 2023-10-06T05:24:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/127008746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac941595d0 0x7fac9406b080 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac9a5fe700 1 -- 172.21.15.45:0/127008746 >> 172.21.15.45:0/127008746 conn(0x7fac9406f090 msgr2=0x7fac9406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac9a5fe700 1 -- 172.21.15.45:0/127008746 shutdown_connections 2023-10-06T05:24:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.592+0000 7fac9a5fe700 1 -- 172.21.15.45:0/127008746 wait complete. 2023-10-06T05:24:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.594+0000 7fac9a5fe700 1 Processor -- start 2023-10-06T05:24:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.594+0000 7fac9a5fe700 1 -- start start 2023-10-06T05:24:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac9a5fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9406b5c0 0x7fac940c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac9a5fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac94159200 0x7fac940c0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac9a5fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac941595d0 0x7fac94171350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac9a5fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac940c3920 con 0x7fac94159200 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac9a5fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac94152bf0 con 0x7fac941595d0 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac9a5fe700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac94171890 con 0x7fac9406b5c0 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac94159200 0x7fac940c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac94159200 0x7fac940c0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42152/0 (socket says 172.21.15.45:42152) 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.595+0000 7fac937fe700 1 -- 172.21.15.45:0/1434668175 learned_addr learned my addr 172.21.15.45:0/1434668175 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac98b9b700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac941595d0 0x7fac94171350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac93fff700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9406b5c0 0x7fac940c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac937fe700 1 -- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9406b5c0 msgr2=0x7fac940c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac937fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9406b5c0 0x7fac940c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac937fe700 1 -- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac941595d0 msgr2=0x7fac94171350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac937fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac941595d0 0x7fac94171350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac937fe700 1 -- 172.21.15.45:0/1434668175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c009420 con 0x7fac94159200 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac93fff700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9406b5c0 0x7fac940c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac98b9b700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac941595d0 0x7fac94171350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac937fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac94159200 0x7fac940c0890 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00b3c0 tx=0x7fac7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac8bfff700 1 -- 172.21.15.45:0/1434668175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c01f070 con 0x7fac94159200 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac8bfff700 1 -- 172.21.15.45:0/1434668175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c00b9d0 con 0x7fac94159200 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac8bfff700 1 -- 172.21.15.45:0/1434668175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c025650 con 0x7fac94159200 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac94172270 con 0x7fac94159200 2023-10-06T05:24:38.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.596+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac94171c20 con 0x7fac94159200 2023-10-06T05:24:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.597+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac94171a80 con 0x7fac94159200 2023-10-06T05:24:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.598+0000 7fac8bfff700 1 -- 172.21.15.45:0/1434668175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac7c023070 con 0x7fac94159200 2023-10-06T05:24:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.598+0000 7fac8bfff700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac74066f30 0x7fac740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.598+0000 7fac8bfff700 1 -- 172.21.15.45:0/1434668175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7fac7c086520 con 0x7fac94159200 2023-10-06T05:24:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.599+0000 7fac93fff700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac74066f30 0x7fac740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.599+0000 7fac93fff700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac74066f30 0x7fac740693e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fac84007fd0 tx=0x7fac84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.603+0000 7fac8bfff700 1 -- 172.21.15.45:0/1434668175 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac94171a80 con 0x7fac94159200 2023-10-06T05:24:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:38.849+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fac94171a80 con 0x7fac94159200 2023-10-06T05:24:39.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.112+0000 7fac8bfff700 1 -- 172.21.15.45:0/1434668175 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v233) v1 ==== 161+0+0 (secure 0 0 0) 0x7fac94171a80 con 0x7fac94159200 2023-10-06T05:24:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac74066f30 msgr2=0x7fac740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac74066f30 0x7fac740693e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fac84007fd0 tx=0x7fac84009040 comp rx=0 tx=0).stop 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac94159200 msgr2=0x7fac940c0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac94159200 0x7fac940c0890 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00b3c0 tx=0x7fac7c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 shutdown_connections 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac74066f30 0x7fac740693e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac941595d0 0x7fac94171350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9406b5c0 0x7fac940c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 --2- 172.21.15.45:0/1434668175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac94159200 0x7fac940c0890 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.115+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 >> 172.21.15.45:0/1434668175 conn(0x7fac9406f090 msgr2=0x7fac940af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.116+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 shutdown_connections 2023-10-06T05:24:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.116+0000 7fac9a5fe700 1 -- 172.21.15.45:0/1434668175 wait complete. 2023-10-06T05:24:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:39.133 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:39.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.267+0000 7f0a203a7700 1 Processor -- start 2023-10-06T05:24:39.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.268+0000 7f0a203a7700 1 -- start start 2023-10-06T05:24:39.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a203a7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a18159200 0x7f0a180c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a203a7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0a181595d0 0x7f0a1806b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a203a7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a180c7d00 con 0x7f0a1806b5c0 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a203a7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180c7780 con 0x7f0a181595d0 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a203a7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180c86d0 con 0x7f0a18159200 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1e944700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0a1806b5c0 0x7f0a180ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56468/0 (socket says 172.21.15.45:56468) 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1e944700 1 -- 172.21.15.45:0/2600577490 learned_addr learned my addr 172.21.15.45:0/2600577490 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1e143700 1 --2- 172.21.15.45:0/2600577490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a18159200 0x7f0a180c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1d942700 1 --2- 172.21.15.45:0/2600577490 >> v2:172.21.15.144:3300/0 conn(0x7f0a181595d0 0x7f0a1806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1d141700 1 -- 172.21.15.45:0/2600577490 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555594014 0 0) 0x7f0a180c7d00 con 0x7f0a1806b5c0 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1d141700 1 -- 172.21.15.45:0/2600577490 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a04003570 con 0x7f0a1806b5c0 2023-10-06T05:24:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1e143700 1 -- 172.21.15.45:0/2600577490 >> v2:172.21.15.144:3300/0 conn(0x7f0a181595d0 msgr2=0x7f0a1806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1e143700 1 --2- 172.21.15.45:0/2600577490 >> v2:172.21.15.144:3300/0 conn(0x7f0a181595d0 0x7f0a1806b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1e143700 1 -- 172.21.15.45:0/2600577490 >> v1:172.21.15.45:6789/0 conn(0x7f0a1806b5c0 legacy=0x7f0a180ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1e143700 1 -- 172.21.15.45:0/2600577490 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1806bb40 con 0x7f0a18159200 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.269+0000 7f0a1d942700 1 --2- 172.21.15.45:0/2600577490 >> v2:172.21.15.144:3300/0 conn(0x7f0a181595d0 0x7f0a1806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a1e143700 1 --2- 172.21.15.45:0/2600577490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a18159200 0x7f0a180c8190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a140098e0 tx=0x7f0a140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=294c7d5614406de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a1d141700 1 -- 172.21.15.45:0/2600577490 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011630 con 0x7f0a18159200 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a1d141700 1 -- 172.21.15.45:0/2600577490 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14011790 con 0x7f0a18159200 2023-10-06T05:24:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a1d141700 1 -- 172.21.15.45:0/2600577490 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011b30 con 0x7f0a18159200 2023-10-06T05:24:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a203a7700 1 -- 172.21.15.45:0/2600577490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a18159200 msgr2=0x7f0a180c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a203a7700 1 --2- 172.21.15.45:0/2600577490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a18159200 0x7f0a180c8190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a140098e0 tx=0x7f0a140091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a203a7700 1 -- 172.21.15.45:0/2600577490 shutdown_connections 2023-10-06T05:24:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a203a7700 1 --2- 172.21.15.45:0/2600577490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a18159200 0x7f0a180c8190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a203a7700 1 --2- 172.21.15.45:0/2600577490 >> v2:172.21.15.144:3300/0 conn(0x7f0a181595d0 0x7f0a1806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.270+0000 7f0a203a7700 1 -- 172.21.15.45:0/2600577490 >> 172.21.15.45:0/2600577490 conn(0x7f0a1806f090 msgr2=0x7f0a1806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.271+0000 7f0a203a7700 1 -- 172.21.15.45:0/2600577490 shutdown_connections 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.271+0000 7f0a203a7700 1 -- 172.21.15.45:0/2600577490 wait complete. 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.272+0000 7f0a203a7700 1 Processor -- start 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.272+0000 7f0a203a7700 1 -- start start 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.272+0000 7f0a203a7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a1806b5c0 0x7f0a180c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.272+0000 7f0a203a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a18159200 0x7f0a180b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a203a7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a181595d0 0x7f0a181584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a203a7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180c1050 con 0x7f0a1806b5c0 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a203a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18158a20 con 0x7f0a18159200 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a203a7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b1500 con 0x7f0a181595d0 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a1806b5c0 0x7f0a180c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a1806b5c0 0x7f0a180c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42168/0 (socket says 172.21.15.45:42168) 2023-10-06T05:24:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 -- 172.21.15.45:0/3535029291 learned_addr learned my addr 172.21.15.45:0/3535029291 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1d942700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a18159200 0x7f0a180b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e944700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a181595d0 0x7f0a181584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 -- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a181595d0 msgr2=0x7f0a181584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a181595d0 0x7f0a181584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 -- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a18159200 msgr2=0x7f0a180b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a18159200 0x7f0a180b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 -- 172.21.15.45:0/3535029291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08009420 con 0x7f0a1806b5c0 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a1e143700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a1806b5c0 0x7f0a180c4850 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f0a180c78c0 tx=0x7f0a14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a0e7fc700 1 -- 172.21.15.45:0/3535029291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011b10 con 0x7f0a1806b5c0 2023-10-06T05:24:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a0e7fc700 1 -- 172.21.15.45:0/3535029291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14000c90 con 0x7f0a1806b5c0 2023-10-06T05:24:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a14009420 con 0x7f0a1806b5c0 2023-10-06T05:24:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a0e7fc700 1 -- 172.21.15.45:0/3535029291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1401a660 con 0x7f0a1806b5c0 2023-10-06T05:24:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.273+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a180b1ac0 con 0x7f0a1806b5c0 2023-10-06T05:24:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.274+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1806e2f0 con 0x7f0a1806b5c0 2023-10-06T05:24:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.275+0000 7f0a0e7fc700 1 -- 172.21.15.45:0/3535029291 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a1401aa10 con 0x7f0a1806b5c0 2023-10-06T05:24:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.275+0000 7f0a0e7fc700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09fc066f30 0x7f09fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.275+0000 7f0a0e7fc700 1 -- 172.21.15.45:0/3535029291 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7f0a14086ba0 con 0x7f0a1806b5c0 2023-10-06T05:24:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.276+0000 7f0a1d942700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09fc066f30 0x7f09fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.276+0000 7f0a1d942700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09fc066f30 0x7f09fc0693e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0a18063f70 tx=0x7f0a080091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.280+0000 7f0a0e7fc700 1 -- 172.21.15.45:0/3535029291 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a14053370 con 0x7f0a1806b5c0 2023-10-06T05:24:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:39.570+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f0a1806be20 con 0x7f0a1806b5c0 2023-10-06T05:24:40.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.114+0000 7f0a0e7fc700 1 -- 172.21.15.45:0/3535029291 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v234) v1 ==== 101+0+0 (secure 0 0 0) 0x7f0a14056990 con 0x7f0a1806b5c0 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09fc066f30 msgr2=0x7f09fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09fc066f30 0x7f09fc0693e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0a18063f70 tx=0x7f0a080091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a1806b5c0 msgr2=0x7f0a180c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a1806b5c0 0x7f0a180c4850 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f0a180c78c0 tx=0x7f0a14017000 comp rx=0 tx=0).stop 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 shutdown_connections 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09fc066f30 0x7f09fc0693e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a181595d0 0x7f0a181584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a1806b5c0 0x7f0a180c4850 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 --2- 172.21.15.45:0/3535029291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a18159200 0x7f0a180b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 >> 172.21.15.45:0/3535029291 conn(0x7f0a1806f090 msgr2=0x7f0a181599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 shutdown_connections 2023-10-06T05:24:40.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.117+0000 7f0a203a7700 1 -- 172.21.15.45:0/3535029291 wait complete. 2023-10-06T05:24:40.120 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:40.135 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:40.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.179+0000 7f7eb0da0040 1 Processor -- start 2023-10-06T05:24:40.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.179+0000 7f7eb0da0040 1 -- start start 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eb0da0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9156210 0x5629c9236530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eb0da0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218890 0x5629c9186430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eb0da0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9218c60 0x5629c9186970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eb0da0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c90a3980 con 0x5629c9218c60 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eb0da0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c90a3620 con 0x5629c9156210 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eb0da0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629c909a400 con 0x5629c9218890 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9218c60 0x5629c9186970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9218c60 0x5629c9186970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42188/0 (socket says 172.21.15.45:42188) 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/3506473624 learned_addr learned my addr 172.21.15.45:0/3506473624 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9156210 0x5629c9236530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218890 0x5629c9186430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218890 msgr2=0x5629c9186430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218890 0x5629c9186430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9156210 msgr2=0x5629c9236530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9156210 0x5629c9236530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.180+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/3506473624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0005db0 con 0x5629c9218c60 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9218c60 0x5629c9186970 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4001550 tx=0x7f7ea4006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf9ac660d67cdb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7e9bfff700 1 -- 172.21.15.45:0/3506473624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4004150 con 0x5629c9218c60 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7e9bfff700 1 -- 172.21.15.45:0/3506473624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea400e470 con 0x5629c9218c60 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7e9bfff700 1 -- 172.21.15.45:0/3506473624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400e810 con 0x5629c9218c60 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 -- 172.21.15.45:0/3506473624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9218c60 msgr2=0x5629c9186970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9218c60 0x5629c9186970 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4001550 tx=0x7f7ea4006b10 comp rx=0 tx=0).stop 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 -- 172.21.15.45:0/3506473624 shutdown_connections 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9218c60 0x5629c9186970 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9156210 0x5629c9236530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/3506473624 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218890 0x5629c9186430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 -- 172.21.15.45:0/3506473624 >> 172.21.15.45:0/3506473624 conn(0x5629c9159450 msgr2=0x5629c919c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 -- 172.21.15.45:0/3506473624 shutdown_connections 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.181+0000 7f7eb0da0040 1 -- 172.21.15.45:0/3506473624 wait complete. 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.182+0000 7f7eb0da0040 1 Processor -- start 2023-10-06T05:24:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.182+0000 7f7eb0da0040 1 -- start start 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7eb0da0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9156210 0x5629c92364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7eb0da0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9218890 0x5629c9215c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7eb0da0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218c60 0x5629c9234ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7eb0da0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9230400 con 0x5629c9156210 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7eb0da0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c92306a0 con 0x5629c9218890 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7eb0da0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629c9230940 con 0x5629c9218c60 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7ea9fbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9156210 0x5629c92364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7ea9fbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9156210 0x5629c92364c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42204/0 (socket says 172.21.15.45:42204) 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 learned_addr learned my addr 172.21.15.45:0/732765759 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218c60 0x5629c9234ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.183+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9218890 0x5629c9215c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218c60 msgr2=0x5629c9234ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218c60 0x5629c9234ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9218890 msgr2=0x5629c9215c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9218890 0x5629c9215c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c90a5750 con 0x5629c9156210 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5629c9218c60 0x5629c9234ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5629c9218890 0x5629c9215c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9156210 0x5629c92364c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c008b20 tx=0x7f7e9c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/732765759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00f050 con 0x5629c9156210 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/732765759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c010040 con 0x5629c9156210 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5629c90b23d0 con 0x5629c9156210 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/732765759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c006cb0 con 0x5629c9156210 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.184+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629c90a5ae0 con 0x5629c9156210 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.186+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/732765759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e9c015670 con 0x5629c9156210 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.186+0000 7f7e9a7fc700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e8c065890 0x7f7e8c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.186+0000 7f7e9a7fc700 1 -- 172.21.15.45:0/732765759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7f7e9c084440 con 0x5629c9156210 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.186+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e8c065890 0x7f7e8c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.195 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.186+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5629c923fa00 0x5629c923fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.186+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9231320 con 0x5629c923fa00 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.187+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5629c923fa00 0x5629c923fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.187+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e8c065890 0x7f7e8c067d40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0008fd0 tx=0x7f7ea0016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.187+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5629c923fa00 0x5629c923fdd0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.189+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v115'691 uv665 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.189+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924bb20 con 0x5629c923fa00 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.190+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v115'692 uv692 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.190+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924bed0 con 0x5629c923fa00 2023-10-06T05:24:40.198 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.191+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v115'693 uv693 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.191+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924d290 con 0x5629c923fa00 2023-10-06T05:24:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.193+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'694 uv694 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.193+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924d640 con 0x5629c923fa00 2023-10-06T05:24:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.194+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'695 uv695 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.194+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924c280 con 0x5629c923fa00 2023-10-06T05:24:40.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.195+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'696 uv696 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.195+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924c680 con 0x5629c923fa00 2023-10-06T05:24:40.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.197+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v115'697 uv671 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.197+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ca80 con 0x5629c923fa00 2023-10-06T05:24:40.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.198+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v115'698 uv698 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.198+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce80 con 0x5629c923fa00 2023-10-06T05:24:40.203 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.199+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v115'699 uv699 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.199+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce80 con 0x5629c923fa00 2023-10-06T05:24:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.200+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'700 uv700 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.200+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.202+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'701 uv701 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.202+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.203+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'702 uv702 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.203+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.204+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v115'703 uv678 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.204+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.205+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'704 uv704 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.205+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.208 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:40.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.206+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v115'705 uv705 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.206+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.208+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'706 uv706 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.208+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.209+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'707 uv707 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.209+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.210+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'708 uv708 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.210+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.212+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'709 uv709 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.212+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.213+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v115'710 uv684 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.213+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.214+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v115'711 uv711 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.214+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.217 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.216+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v115'712 uv712 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.216+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.217+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'713 uv713 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.217+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.218+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'714 uv714 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.218+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.220+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'715 uv715 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.220+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.221+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v115'716 uv690 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.221+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.222+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v115'717 uv717 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.222+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.225 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:40.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.224+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v115'718 uv718 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.224+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.225+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'719 uv719 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.225+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.226+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'720 uv720 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.226+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c924ce30 con 0x5629c923fa00 2023-10-06T05:24:40.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.228+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'721 uv721 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c923fa00 2023-10-06T05:24:40.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.228+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5629c924ce30 0x5629c924ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.228+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9250450 con 0x5629c924ce30 2023-10-06T05:24:40.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.228+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5629c924ce30 0x5629c924ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.228+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5629c924ce30 0x5629c924ff10 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.230+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'500 uv483 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.230+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9250800 con 0x5629c924ce30 2023-10-06T05:24:40.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.231+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'501 uv501 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.231+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9250bb0 con 0x5629c924ce30 2023-10-06T05:24:40.234 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.233+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'502 uv502 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.233+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9251450 con 0x5629c924ce30 2023-10-06T05:24:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.234+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v115'503 uv503 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.234+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9251800 con 0x5629c924ce30 2023-10-06T05:24:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.235+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'504 uv504 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.235+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9251bb0 con 0x5629c924ce30 2023-10-06T05:24:40.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.236+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v115'505 uv488 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.236+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9251f60 con 0x5629c924ce30 2023-10-06T05:24:40.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.237+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v115'506 uv506 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.238+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.240 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.239+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v115'507 uv507 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.239+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.240+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'508 uv508 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.240+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252710 con 0x5629c924ce30 2023-10-06T05:24:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.241+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'509 uv509 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.241+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252710 con 0x5629c924ce30 2023-10-06T05:24:40.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.243+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v115'510 uv493 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.243+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.244+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v115'511 uv511 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.244+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.247 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:40.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.245+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v115'512 uv512 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.245+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.247+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v115'513 uv513 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.247+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9255e80 con 0x5629c924ce30 2023-10-06T05:24:40.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.248+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'514 uv514 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.248+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92523e0 con 0x5629c924ce30 2023-10-06T05:24:40.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.249+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v115'515 uv499 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.249+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92523e0 con 0x5629c924ce30 2023-10-06T05:24:40.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.251+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v115'516 uv516 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.251+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.254 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.252+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v115'517 uv517 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.252+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.253+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'518 uv518 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.253+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.255+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'519 uv519 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.255+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c924ce30 2023-10-06T05:24:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.256+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'520 uv520 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.256+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5629c9252c20 0x5629c9252ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.256+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9252310 con 0x5629c9252c20 2023-10-06T05:24:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.256+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5629c9252c20 0x5629c9252ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.257+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5629c9252c20 0x5629c9252ff0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.258+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.258+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9255790 con 0x5629c9252c20 2023-10-06T05:24:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.259+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v115'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.259+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9255c80 con 0x5629c9252c20 2023-10-06T05:24:40.262 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:40.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.260+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v115'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.260+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9257580 con 0x5629c9252c20 2023-10-06T05:24:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.261+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.261+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9257930 con 0x5629c9252c20 2023-10-06T05:24:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.262+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.262+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256570 con 0x5629c9252c20 2023-10-06T05:24:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.264+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.264+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256970 con 0x5629c9252c20 2023-10-06T05:24:40.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.265+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'96 uv95 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.265+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256970 con 0x5629c9252c20 2023-10-06T05:24:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.266+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'97 uv97 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.266+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256970 con 0x5629c9252c20 2023-10-06T05:24:40.269 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.267+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'98 uv98 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.268+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256990 con 0x5629c9252c20 2023-10-06T05:24:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.269+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'99 uv99 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.269+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256990 con 0x5629c924ce30 2023-10-06T05:24:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.270+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.270+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256990 con 0x5629c924ce30 2023-10-06T05:24:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.271+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v115'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.271+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256970 con 0x5629c924ce30 2023-10-06T05:24:40.274 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.272+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v115'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.273+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256970 con 0x5629c924ce30 2023-10-06T05:24:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.274+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.274+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256970 con 0x5629c924ce30 2023-10-06T05:24:40.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.275+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.275+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9256970 con 0x5629c924ce30 2023-10-06T05:24:40.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.276+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c924ce30 2023-10-06T05:24:40.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.277+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5629c9257ce0 0x5629c925a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.277+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.277+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5629c9257ce0 0x5629c925a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.277+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5629c9257ce0 0x5629c925a0c0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.278+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v115'249 uv243 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.278+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.280+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v115'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.280+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.282 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.281+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v115'251 uv251 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.281+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925b610 con 0x5629c9257ce0 2023-10-06T05:24:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.282+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'252 uv252 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.282+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925b610 con 0x5629c9257ce0 2023-10-06T05:24:40.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.283+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'253 uv253 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.283+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.284+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'254 uv254 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.284+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.285+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v115'255 uv248 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.285+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.286+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v115'256 uv256 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.286+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.289 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:40.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.287+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v115'257 uv257 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.287+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.288+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'258 uv258 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.288+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925a600 con 0x5629c9257ce0 2023-10-06T05:24:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.290+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'259 uv259 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.290+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5629c925ac80 0x5629c925d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.290+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925d670 con 0x5629c925ac80 2023-10-06T05:24:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.290+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5629c925ac80 0x5629c925d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.290+0000 7f7ea9fbe700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5629c925ac80 0x5629c925d060 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.292+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v115'112 uv111 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c925ac80 2023-10-06T05:24:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.292+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c925ac80 2023-10-06T05:24:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.293+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v115'113 uv113 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c925ac80 2023-10-06T05:24:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.293+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c925ac80 2023-10-06T05:24:40.297 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:40.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.296+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v115'114 uv114 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c925ac80 2023-10-06T05:24:40.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.296+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c925ac80 2023-10-06T05:24:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.298+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'115 uv115 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c925ac80 2023-10-06T05:24:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.298+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c925ac80 2023-10-06T05:24:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.301+0000 7f7ea9fbe700 1 -- 172.21.15.45:0/732765759 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'116 uv116 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7e9c01b020 con 0x5629c925ac80 2023-10-06T05:24:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.301+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c9252c20 2023-10-06T05:24:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.302+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.302+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c9252c20 2023-10-06T05:24:40.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.303+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v115'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.303+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c9252c20 2023-10-06T05:24:40.306 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:40.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.305+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v115'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.305+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925ea60 con 0x5629c9252c20 2023-10-06T05:24:40.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.306+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.306+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925ea60 con 0x5629c9252c20 2023-10-06T05:24:40.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.307+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.307+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c925da50 con 0x5629c9252c20 2023-10-06T05:24:40.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.308+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.308+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5629c925e0d0 0x5629c92604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.308+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260ad0 con 0x5629c925e0d0 2023-10-06T05:24:40.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.308+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5629c925e0d0 0x5629c92604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.309+0000 7f7ea97bd700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5629c925e0d0 0x5629c92604b0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.310+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v115'223 uv217 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.310+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.311+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v115'224 uv224 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.311+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.314 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:40.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.312+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v115'225 uv225 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.312+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.313+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'226 uv226 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.313+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.314+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'227 uv227 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.315+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.315+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v115'228 uv222 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.315+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.317+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v115'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.317+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.319 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.317+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v115'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.318+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9260eb0 con 0x5629c925e0d0 2023-10-06T05:24:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.318+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'231 uv231 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.319+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9263c20 con 0x5629c925e0d0 2023-10-06T05:24:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.320+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c925e0d0 2023-10-06T05:24:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.320+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9252c20 2023-10-06T05:24:40.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.321+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v115'99 uv98 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.321+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9252c20 2023-10-06T05:24:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.322+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'100 uv100 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.322+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9252c20 2023-10-06T05:24:40.325 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.323+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v115'101 uv101 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.323+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9252c20 2023-10-06T05:24:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.324+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'102 uv102 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.324+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5629c9261cb0 0x5629c9264370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.324+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9261cb0 2023-10-06T05:24:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.325+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5629c9261cb0 0x5629c9264370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.325+0000 7f7eaa7bf700 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5629c9261cb0 0x5629c9264370 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.326+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9261cb0 2023-10-06T05:24:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.326+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9261cb0 2023-10-06T05:24:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.327+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v115'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9261cb0 2023-10-06T05:24:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.327+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9261cb0 2023-10-06T05:24:40.330 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.329+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v115'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9261cb0 2023-10-06T05:24:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.329+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9261cb0 2023-10-06T05:24:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.331+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9261cb0 2023-10-06T05:24:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.332+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9261cb0 2023-10-06T05:24:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.334+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9261cb0 2023-10-06T05:24:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.334+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9261cb0 2023-10-06T05:24:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.337+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9261cb0 2023-10-06T05:24:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.337+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9257ce0 2023-10-06T05:24:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.338+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v115'115 uv114 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.338+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9257ce0 2023-10-06T05:24:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.340+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v115'116 uv116 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.340+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9257ce0 2023-10-06T05:24:40.342 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.341+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v115'117 uv117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.341+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9257ce0 2023-10-06T05:24:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.342+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'118 uv118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.342+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9257ce0 2023-10-06T05:24:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.343+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'119 uv119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f7ea400ec90 con 0x5629c9257ce0 2023-10-06T05:24:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.343+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9252c20 2023-10-06T05:24:40.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.345+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v115'278 uv277 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.345+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92613a0 con 0x5629c9252c20 2023-10-06T05:24:40.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.346+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v115'279 uv279 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.346+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.349 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.347+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v115'280 uv280 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.347+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.348+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v115'281 uv281 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.348+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.350+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'282 uv282 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.350+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.351+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v115'283 uv276 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.351+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.352+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v115'284 uv284 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.352+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.353+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v115'285 uv285 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.354 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.353+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.354+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v115'286 uv286 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.354+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9261370 con 0x5629c9252c20 2023-10-06T05:24:40.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.355+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v115'287 uv287 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.355+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c92648b0 con 0x5629c9252c20 2023-10-06T05:24:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.356+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v115'288 uv288 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.356+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x5629c9264da0 con 0x5629c9252c20 2023-10-06T05:24:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7ea97bd700 1 -- 172.21.15.45:0/732765759 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v115'289 uv289 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7ea00127d0 con 0x5629c9252c20 2023-10-06T05:24:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5629c9252c20 msgr2=0x5629c9252ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x5629c9252c20 0x5629c9252ff0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5629c923fa00 msgr2=0x5629c923fdd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5629c923fa00 0x5629c923fdd0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5629c925e0d0 msgr2=0x5629c92604b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5629c925e0d0 0x5629c92604b0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5629c924ce30 msgr2=0x5629c924ff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x5629c924ce30 0x5629c924ff10 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5629c9261cb0 msgr2=0x5629c9264370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.358+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x5629c9261cb0 0x5629c9264370 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5629c925ac80 msgr2=0x5629c925d060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x5629c925ac80 0x5629c925d060 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 reap_dead start 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5629c9257ce0 msgr2=0x5629c925a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5629c9257ce0 0x5629c925a0c0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 reap_dead start 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eaa7bf700 1 -- 172.21.15.45:0/732765759 reap_dead start 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e8c065890 msgr2=0x7f7e8c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e8c065890 0x7f7e8c067d40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0008fd0 tx=0x7f7ea0016000 comp rx=0 tx=0).stop 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9156210 msgr2=0x5629c92364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9156210 0x5629c92364c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c008b20 tx=0x7f7e9c00c040 comp rx=0 tx=0).stop 2023-10-06T05:24:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 shutdown_connections 2023-10-06T05:24:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c9156210 0x5629c92364c0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 --2- 172.21.15.45:0/732765759 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7e8c065890 0x7f7e8c067d40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 >> 172.21.15.45:0/732765759 conn(0x5629c9159450 msgr2=0x5629c919c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 shutdown_connections 2023-10-06T05:24:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.359+0000 7f7eb0da0040 1 -- 172.21.15.45:0/732765759 wait complete. 2023-10-06T05:24:40.365 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2023-10-06T05:24:40.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.414+0000 7fd22457f040 1 Processor -- start 2023-10-06T05:24:40.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.414+0000 7fd22457f040 1 -- start start 2023-10-06T05:24:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd22457f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 0x564d6a1d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd22457f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a272bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd22457f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 0x564d6a273130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd22457f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6a0da980 con 0x564d6a24f890 2023-10-06T05:24:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd22457f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6a0da620 con 0x564d6a24fc60 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd22457f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d6a0d1400 con 0x564d6a18d210 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a272bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a272bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42208/0 (socket says 172.21.15.45:42208) 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 -- 172.21.15.45:0/485233617 learned_addr learned my addr 172.21.15.45:0/485233617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21df9e700 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 0x564d6a273130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21d79d700 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 0x564d6a1d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 -- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 msgr2=0x564d6a1d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:40.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 0x564d6a1d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 -- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 msgr2=0x564d6a273130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 0x564d6a273130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 -- 172.21.15.45:0/485233617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210005db0 con 0x564d6a24f890 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.415+0000 7fd21cf9c700 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a272bf0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd20800ac40 tx=0x7fd208015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5e64714ab290de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd2177fe700 1 -- 172.21.15.45:0/485233617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208000bc0 con 0x564d6a24f890 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd2177fe700 1 -- 172.21.15.45:0/485233617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2080078e0 con 0x564d6a24f890 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd2177fe700 1 -- 172.21.15.45:0/485233617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20801c410 con 0x564d6a24f890 2023-10-06T05:24:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 -- 172.21.15.45:0/485233617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 msgr2=0x564d6a272bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a272bf0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd20800ac40 tx=0x7fd208015000 comp rx=0 tx=0).stop 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 -- 172.21.15.45:0/485233617 shutdown_connections 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 0x564d6a273130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 0x564d6a1d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 --2- 172.21.15.45:0/485233617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a272bf0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 -- 172.21.15.45:0/485233617 >> 172.21.15.45:0/485233617 conn(0x564d6a190450 msgr2=0x564d6a192830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 -- 172.21.15.45:0/485233617 shutdown_connections 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.416+0000 7fd22457f040 1 -- 172.21.15.45:0/485233617 wait complete. 2023-10-06T05:24:40.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.417+0000 7fd22457f040 1 Processor -- start 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.417+0000 7fd22457f040 1 -- start start 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.418+0000 7fd22457f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 0x564d6a24d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.418+0000 7fd22457f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a26d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.418+0000 7fd22457f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 0x564d6a2699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.418+0000 7fd22457f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6a24c270 con 0x564d6a24f890 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.418+0000 7fd22457f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d6a24c510 con 0x564d6a24fc60 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.418+0000 7fd22457f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d6a24c7b0 con 0x564d6a18d210 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a26d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a26d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42216/0 (socket says 172.21.15.45:42216) 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 -- 172.21.15.45:0/1530508282 learned_addr learned my addr 172.21.15.45:0/1530508282 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:40.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21d79d700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 0x564d6a24d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21df9e700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 0x564d6a2699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 -- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 msgr2=0x564d6a24d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d6a18d210 0x564d6a24d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 -- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 msgr2=0x564d6a2699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d6a24fc60 0x564d6a2699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 -- 172.21.15.45:0/1530508282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d6a1031c0 con 0x564d6a24f890 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd21cf9c700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a26d770 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fd20801e040 tx=0x7fd20801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd215ffb700 1 -- 172.21.15.45:0/1530508282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20801c410 con 0x564d6a24f890 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd215ffb700 1 -- 172.21.15.45:0/1530508282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208000bc0 con 0x564d6a24f890 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd208005cf0 con 0x564d6a24f890 2023-10-06T05:24:40.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd215ffb700 1 -- 172.21.15.45:0/1530508282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208012680 con 0x564d6a24f890 2023-10-06T05:24:40.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.419+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d6a0e93d0 con 0x564d6a24f890 2023-10-06T05:24:40.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.421+0000 7fd215ffb700 1 -- 172.21.15.45:0/1530508282 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd208019070 con 0x564d6a24f890 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.421+0000 7fd215ffb700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200065890 0x7fd200067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.421+0000 7fd215ffb700 1 -- 172.21.15.45:0/1530508282 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7fd208086380 con 0x564d6a24f890 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.421+0000 7fd21d79d700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200065890 0x7fd200067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.421+0000 7fd22457f040 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x564d6a24a6d0 0x564d6a24aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.422+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) v8 -- 0x564d6a24b010 con 0x564d6a24a6d0 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.422+0000 7fd21df9e700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x564d6a24a6d0 0x564d6a24aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.422+0000 7fd21d79d700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200065890 0x7fd200067d40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd2100055c0 tx=0x7fd21000c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.422+0000 7fd21df9e700 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x564d6a24a6d0 0x564d6a24aad0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.423+0000 7fd21df9e700 1 -- 172.21.15.45:0/1530508282 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v115'521 uv521 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd218003570 con 0x564d6a24a6d0 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x564d6a24a6d0 msgr2=0x564d6a24aad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x564d6a24a6d0 0x564d6a24aad0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200065890 msgr2=0x7fd200067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd200065890 0x7fd200067d40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd2100055c0 tx=0x7fd21000c040 comp rx=0 tx=0).stop 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 msgr2=0x564d6a26d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 --2- 172.21.15.45:0/1530508282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d6a24f890 0x564d6a26d770 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fd20801e040 tx=0x7fd20801c000 comp rx=0 tx=0).stop 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd21df9e700 1 -- 172.21.15.45:0/1530508282 reap_dead start 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 shutdown_connections 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 >> 172.21.15.45:0/1530508282 conn(0x564d6a190450 msgr2=0x564d6a1bd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 shutdown_connections 2023-10-06T05:24:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.424+0000 7fd22457f040 1 -- 172.21.15.45:0/1530508282 wait complete. 2023-10-06T05:24:40.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:40.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.569+0000 7f66e8eaa700 1 Processor -- start 2023-10-06T05:24:40.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.569+0000 7f66e8eaa700 1 -- start start 2023-10-06T05:24:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.570+0000 7f66e8eaa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40ccb70 0x7f66e40aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.570+0000 7f66e8eaa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66e40c8b70 0x7f66e40c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.570+0000 7f66e8eaa700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e40c3b50 con 0x7f66e4157330 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.570+0000 7f66e8eaa700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e4157700 con 0x7f66e40c8b70 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.570+0000 7f66e8eaa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40c98e0 con 0x7f66e40ccb70 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.570+0000 7f66e2d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66e40c8b70 0x7f66e40c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e1d9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40ccb70 0x7f66e40aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e259c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f66e4157330 0x7f66e40ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:52608/0 (socket says 172.21.15.45:52608) 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e2d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66e40c8b70 0x7f66e40c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42218/0 (socket says 172.21.15.45:42218) 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e259c700 1 -- 172.21.15.45:0/3267045641 learned_addr learned my addr 172.21.15.45:0/3267045641 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e1d9b700 1 -- 172.21.15.45:0/3267045641 >> v2:172.21.15.45:3300/0 conn(0x7f66e40c8b70 msgr2=0x7f66e40c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e159a700 1 -- 172.21.15.45:0/3267045641 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349778861 0 0) 0x7f66e40c3b50 con 0x7f66e4157330 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e1d9b700 1 --2- 172.21.15.45:0/3267045641 >> v2:172.21.15.45:3300/0 conn(0x7f66e40c8b70 0x7f66e40c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e1d9b700 1 -- 172.21.15.45:0/3267045641 >> v1:172.21.15.144:6789/0 conn(0x7f66e4157330 legacy=0x7f66e40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e1d9b700 1 -- 172.21.15.45:0/3267045641 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40c9bc0 con 0x7f66e40ccb70 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e2d9d700 1 --2- 172.21.15.45:0/3267045641 >> v2:172.21.15.45:3300/0 conn(0x7f66e40c8b70 0x7f66e40c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e1d9b700 1 --2- 172.21.15.45:0/3267045641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40ccb70 0x7f66e40aaae0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66d40098e0 tx=0x7f66d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e92e6c1629f866 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e159a700 1 -- 172.21.15.45:0/3267045641 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d400b640 con 0x7f66e40ccb70 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e159a700 1 -- 172.21.15.45:0/3267045641 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d400b7a0 con 0x7f66e40ccb70 2023-10-06T05:24:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.571+0000 7f66e159a700 1 -- 172.21.15.45:0/3267045641 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d400bb40 con 0x7f66e40ccb70 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 -- 172.21.15.45:0/3267045641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40ccb70 msgr2=0x7f66e40aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/3267045641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40ccb70 0x7f66e40aaae0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66d40098e0 tx=0x7f66d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 -- 172.21.15.45:0/3267045641 shutdown_connections 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/3267045641 >> v2:172.21.15.45:3300/0 conn(0x7f66e40c8b70 0x7f66e40c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/3267045641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40ccb70 0x7f66e40aaae0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 -- 172.21.15.45:0/3267045641 >> 172.21.15.45:0/3267045641 conn(0x7f66e4067e00 msgr2=0x7f66e40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 -- 172.21.15.45:0/3267045641 shutdown_connections 2023-10-06T05:24:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.572+0000 7f66e8eaa700 1 -- 172.21.15.45:0/3267045641 wait complete. 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.574+0000 7f66e8eaa700 1 Processor -- start 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.574+0000 7f66e8eaa700 1 -- start start 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e8eaa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40c8b70 0x7f66e40b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e8eaa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e40ccb70 0x7f66e4168b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e8eaa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66e4157330 0x7f66e40abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e8eaa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4069170 con 0x7f66e40ccb70 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e8eaa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40ac930 con 0x7f66e4157330 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e8eaa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4068840 con 0x7f66e40c8b70 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e40ccb70 0x7f66e4168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e40ccb70 0x7f66e4168b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42234/0 (socket says 172.21.15.45:42234) 2023-10-06T05:24:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 -- 172.21.15.45:0/1990433651 learned_addr learned my addr 172.21.15.45:0/1990433651 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e259c700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40c8b70 0x7f66e40b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e2d9d700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66e4157330 0x7f66e40abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 -- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40c8b70 msgr2=0x7f66e40b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40c8b70 0x7f66e40b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 -- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66e4157330 msgr2=0x7f66e40abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66e4157330 0x7f66e40abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 -- 172.21.15.45:0/1990433651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc009a20 con 0x7f66e40ccb70 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.575+0000 7f66e1d9b700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e40ccb70 0x7f66e4168b50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f66e40c9a20 tx=0x7f66d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.576+0000 7f66d27fc700 1 -- 172.21.15.45:0/1990433651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d400bb20 con 0x7f66e40ccb70 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.576+0000 7f66d27fc700 1 -- 172.21.15.45:0/1990433651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d4000c90 con 0x7f66e40ccb70 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.576+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d4009420 con 0x7f66e40ccb70 2023-10-06T05:24:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.576+0000 7f66d27fc700 1 -- 172.21.15.45:0/1990433651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4011650 con 0x7f66e40ccb70 2023-10-06T05:24:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.576+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e4068e00 con 0x7f66e40ccb70 2023-10-06T05:24:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.576+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e4069320 con 0x7f66e40ccb70 2023-10-06T05:24:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.577+0000 7f66d27fc700 1 -- 172.21.15.45:0/1990433651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66d4011a80 con 0x7f66e40ccb70 2023-10-06T05:24:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.578+0000 7f66d27fc700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c4066f30 0x7f66c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.578+0000 7f66d27fc700 1 -- 172.21.15.45:0/1990433651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8356+0+0 (secure 0 0 0) 0x7f66d4086830 con 0x7f66e40ccb70 2023-10-06T05:24:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.578+0000 7f66e259c700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c4066f30 0x7f66c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.579+0000 7f66e259c700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c4066f30 0x7f66c40693e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f66d8001230 tx=0x7f66d8007d20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.582+0000 7f66d27fc700 1 -- 172.21.15.45:0/1990433651 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66d4053060 con 0x7f66e40ccb70 2023-10-06T05:24:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.831+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f66e406e350 con 0x7f66e40ccb70 2023-10-06T05:24:40.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.884+0000 7f66d27fc700 1 -- 172.21.15.45:0/1990433651 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v235) v1 ==== 162+0+0 (secure 0 0 0) 0x7f66d4056680 con 0x7f66e40ccb70 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c4066f30 msgr2=0x7f66c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c4066f30 0x7f66c40693e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f66d8001230 tx=0x7f66d8007d20 comp rx=0 tx=0).stop 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e40ccb70 msgr2=0x7f66e4168b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e40ccb70 0x7f66e4168b50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f66e40c9a20 tx=0x7f66d4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 shutdown_connections 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c4066f30 0x7f66c40693e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66e4157330 0x7f66e40abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66e40c8b70 0x7f66e40b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 --2- 172.21.15.45:0/1990433651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e40ccb70 0x7f66e4168b50 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 >> 172.21.15.45:0/1990433651 conn(0x7f66e4067e00 msgr2=0x7f66e40c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 shutdown_connections 2023-10-06T05:24:40.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:40.887+0000 7f66e8eaa700 1 -- 172.21.15.45:0/1990433651 wait complete. 2023-10-06T05:24:40.890 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:40.893 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:24:40.892+0000 7f0a5a441700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-10-06T05:24:40.893 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:24:40.892+0000 7f0a5a441700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:24:40.894 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:40.893+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:40.894 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:40.893+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:40.900 INFO:tasks.ceph.mds.b.smithi144.stderr: -15> 2023-10-06T05:24:40.892+0000 7f0a5a441700 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2023-10-06T05:24:40.900 INFO:tasks.ceph.mds.b.smithi144.stderr: -13> 2023-10-06T05:24:40.892+0000 7f0a5a441700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:24:40.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:40.919 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:24:41.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.034+0000 7f145df98700 1 Processor -- start 2023-10-06T05:24:41.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.034+0000 7f145df98700 1 -- start start 2023-10-06T05:24:41.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.035+0000 7f145df98700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c93c0 0x7f1458156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.035+0000 7f145df98700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145815b000 con 0x7f1458157030 2023-10-06T05:24:41.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.035+0000 7f145df98700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145815b2e0 con 0x7f14580c8ff0 2023-10-06T05:24:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.035+0000 7f145df98700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14580c9830 con 0x7f14580c93c0 2023-10-06T05:24:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.035+0000 7f1457fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1458157030 0x7f14580cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56476/0 (socket says 172.21.15.45:56476) 2023-10-06T05:24:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.035+0000 7f1457fff700 1 -- 172.21.15.45:0/3091415155 learned_addr learned my addr 172.21.15.45:0/3091415155 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.035+0000 7f1456ffd700 1 --2- 172.21.15.45:0/3091415155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c93c0 0x7f1458156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f14567fc700 1 -- 172.21.15.45:0/3091415155 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289088725 0 0) 0x7f145815b2e0 con 0x7f14580c8ff0 2023-10-06T05:24:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f14567fc700 1 -- 172.21.15.45:0/3091415155 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1444003570 con 0x7f14580c8ff0 2023-10-06T05:24:41.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f14567fc700 1 -- 172.21.15.45:0/3091415155 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252077221 0 0) 0x7f145815b000 con 0x7f1458157030 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f14567fc700 1 -- 172.21.15.45:0/3091415155 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f145815b2e0 con 0x7f1458157030 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f1456ffd700 1 -- 172.21.15.45:0/3091415155 >> v1:172.21.15.144:6789/0 conn(0x7f14580c8ff0 legacy=0x7f14580caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f1456ffd700 1 -- 172.21.15.45:0/3091415155 >> v1:172.21.15.45:6789/0 conn(0x7f1458157030 legacy=0x7f14580cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f1456ffd700 1 -- 172.21.15.45:0/3091415155 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145815b5c0 con 0x7f14580c93c0 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f1456ffd700 1 --2- 172.21.15.45:0/3091415155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c93c0 0x7f1458156af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1440009910 tx=0x7f14400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98cf16322462ab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f14567fc700 1 -- 172.21.15.45:0/3091415155 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440011630 con 0x7f14580c93c0 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f14567fc700 1 -- 172.21.15.45:0/3091415155 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1440011790 con 0x7f14580c93c0 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.036+0000 7f14567fc700 1 -- 172.21.15.45:0/3091415155 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440011b30 con 0x7f14580c93c0 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.037+0000 7f145df98700 1 -- 172.21.15.45:0/3091415155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c93c0 msgr2=0x7f1458156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.037+0000 7f145df98700 1 --2- 172.21.15.45:0/3091415155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c93c0 0x7f1458156af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1440009910 tx=0x7f14400091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.037+0000 7f145df98700 1 -- 172.21.15.45:0/3091415155 shutdown_connections 2023-10-06T05:24:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.037+0000 7f145df98700 1 --2- 172.21.15.45:0/3091415155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c93c0 0x7f1458156af0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.037+0000 7f145df98700 1 -- 172.21.15.45:0/3091415155 >> 172.21.15.45:0/3091415155 conn(0x7f14580cdf50 msgr2=0x7f14580ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.037+0000 7f145df98700 1 -- 172.21.15.45:0/3091415155 shutdown_connections 2023-10-06T05:24:41.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.037+0000 7f145df98700 1 -- 172.21.15.45:0/3091415155 wait complete. 2023-10-06T05:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.039+0000 7f145df98700 1 Processor -- start 2023-10-06T05:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.039+0000 7f145df98700 1 -- start start 2023-10-06T05:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f145df98700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c8ff0 0x7f14580c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f145df98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14580c93c0 0x7f1458157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f145df98700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1458157030 0x7f145814aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f145df98700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14580c7b70 con 0x7f14580c93c0 2023-10-06T05:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f145df98700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1458158340 con 0x7f1458157030 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f145df98700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145816b640 con 0x7f14580c8ff0 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14580c93c0 0x7f1458157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14580c93c0 0x7f1458157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42242/0 (socket says 172.21.15.45:42242) 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 -- 172.21.15.45:0/4172874969 learned_addr learned my addr 172.21.15.45:0/4172874969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1457fff700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1458157030 0x7f145814aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f14577fe700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c8ff0 0x7f14580c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 -- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c8ff0 msgr2=0x7f14580c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c8ff0 0x7f14580c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 -- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1458157030 msgr2=0x7f145814aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1458157030 0x7f145814aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 -- 172.21.15.45:0/4172874969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009420 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f14577fe700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c8ff0 0x7f14580c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.040+0000 7f1456ffd700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14580c93c0 0x7f1458157e00 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1440009f00 tx=0x7f1440017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.041+0000 7f144f7fe700 1 -- 172.21.15.45:0/4172874969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144001f070 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.041+0000 7f144f7fe700 1 -- 172.21.15.45:0/4172874969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1440011900 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.041+0000 7f144f7fe700 1 -- 172.21.15.45:0/4172874969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440023410 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.041+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145816b920 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.041+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145816bee0 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.042+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14580b22d0 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.042+0000 7f144f7fe700 1 -- 172.21.15.45:0/4172874969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1440023800 con 0x7f14580c93c0 2023-10-06T05:24:41.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.043+0000 7f144f7fe700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1438066f30 0x7f14380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.043+0000 7f144f7fe700 1 -- 172.21.15.45:0/4172874969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7f1440086220 con 0x7f14580c93c0 2023-10-06T05:24:41.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.043+0000 7f14577fe700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1438066f30 0x7f14380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.043+0000 7f14577fe700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1438066f30 0x7f14380693e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1448007fd0 tx=0x7f1448009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.047+0000 7f144f7fe700 1 -- 172.21.15.45:0/4172874969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14580b22d0 con 0x7f14580c93c0 2023-10-06T05:24:41.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.296+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14580b22d0 con 0x7f14580c93c0 2023-10-06T05:24:41.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.297+0000 7f144f7fe700 1 -- 172.21.15.45:0/4172874969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 236 v236) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f14580b22d0 con 0x7f14580c93c0 2023-10-06T05:24:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:41.299 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":236,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":236,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:40.894017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:41.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1438066f30 msgr2=0x7f14380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1438066f30 0x7f14380693e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1448007fd0 tx=0x7f1448009040 comp rx=0 tx=0).stop 2023-10-06T05:24:41.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14580c93c0 msgr2=0x7f1458157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14580c93c0 0x7f1458157e00 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1440009f00 tx=0x7f1440017000 comp rx=0 tx=0).stop 2023-10-06T05:24:41.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 shutdown_connections 2023-10-06T05:24:41.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1438066f30 0x7f14380693e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1458157030 0x7f145814aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580c8ff0 0x7f14580c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 --2- 172.21.15.45:0/4172874969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14580c93c0 0x7f1458157e00 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 >> 172.21.15.45:0/4172874969 conn(0x7f14580cdf50 msgr2=0x7f14580cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:41.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 shutdown_connections 2023-10-06T05:24:41.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.299+0000 7f145df98700 1 -- 172.21.15.45:0/4172874969 wait complete. 2023-10-06T05:24:41.303 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 236 2023-10-06T05:24:41.314 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:41.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.445+0000 7f65845ae700 1 Processor -- start 2023-10-06T05:24:41.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.445+0000 7f65845ae700 1 -- start start 2023-10-06T05:24:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f65845ae700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f657c1595d0 0x7f657c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f65845ae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f657c06b5c0 0x7f657c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f65845ae700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0c7d00 con 0x7f657c159200 2023-10-06T05:24:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f65845ae700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0c7780 con 0x7f657c1595d0 2023-10-06T05:24:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f65845ae700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c06bb40 con 0x7f657c06b5c0 2023-10-06T05:24:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f6581b49700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f657c1595d0 0x7f657c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f6581b49700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f657c1595d0 0x7f657c06b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42254/0 (socket says 172.21.15.45:42254) 2023-10-06T05:24:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.446+0000 7f6581b49700 1 -- 172.21.15.45:0/2855586450 learned_addr learned my addr 172.21.15.45:0/2855586450 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6582b4b700 1 --2- 172.21.15.45:0/2855586450 >> v2:172.21.15.144:3300/0 conn(0x7f657c06b5c0 0x7f657c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581b49700 1 -- 172.21.15.45:0/2855586450 >> v2:172.21.15.144:3300/0 conn(0x7f657c06b5c0 msgr2=0x7f657c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581b49700 1 --2- 172.21.15.45:0/2855586450 >> v2:172.21.15.144:3300/0 conn(0x7f657c06b5c0 0x7f657c0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581b49700 1 -- 172.21.15.45:0/2855586450 >> v1:172.21.15.144:6789/0 conn(0x7f657c159200 legacy=0x7f657c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581b49700 1 -- 172.21.15.45:0/2855586450 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c06be20 con 0x7f657c1595d0 2023-10-06T05:24:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581b49700 1 --2- 172.21.15.45:0/2855586450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c06b080 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f657c068680 tx=0x7f6570009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf6324bdae7cd54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581348700 1 -- 172.21.15.45:0/2855586450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570009e70 con 0x7f657c1595d0 2023-10-06T05:24:41.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581348700 1 -- 172.21.15.45:0/2855586450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657000b860 con 0x7f657c1595d0 2023-10-06T05:24:41.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.447+0000 7f6581348700 1 -- 172.21.15.45:0/2855586450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000bc00 con 0x7f657c1595d0 2023-10-06T05:24:41.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 -- 172.21.15.45:0/2855586450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 msgr2=0x7f657c06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 --2- 172.21.15.45:0/2855586450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c06b080 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f657c068680 tx=0x7f6570009250 comp rx=0 tx=0).stop 2023-10-06T05:24:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 -- 172.21.15.45:0/2855586450 shutdown_connections 2023-10-06T05:24:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 --2- 172.21.15.45:0/2855586450 >> v2:172.21.15.144:3300/0 conn(0x7f657c06b5c0 0x7f657c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 --2- 172.21.15.45:0/2855586450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c06b080 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 -- 172.21.15.45:0/2855586450 >> 172.21.15.45:0/2855586450 conn(0x7f657c06f090 msgr2=0x7f657c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 -- 172.21.15.45:0/2855586450 shutdown_connections 2023-10-06T05:24:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.448+0000 7f65845ae700 1 -- 172.21.15.45:0/2855586450 wait complete. 2023-10-06T05:24:41.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.449+0000 7f65845ae700 1 Processor -- start 2023-10-06T05:24:41.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.449+0000 7f65845ae700 1 -- start start 2023-10-06T05:24:41.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f65845ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f657c06b5c0 0x7f657c0b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f65845ae700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f657c159200 0x7f657c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f65845ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c0c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f65845ae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0c4910 con 0x7f657c1595d0 2023-10-06T05:24:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f65845ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0cd560 con 0x7f657c06b5c0 2023-10-06T05:24:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f65845ae700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0cbc00 con 0x7f657c159200 2023-10-06T05:24:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c0c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c0c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42264/0 (socket says 172.21.15.45:42264) 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 -- 172.21.15.45:0/2735768141 learned_addr learned my addr 172.21.15.45:0/2735768141 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f658234a700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f657c06b5c0 0x7f657c0b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6581b49700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f657c159200 0x7f657c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 -- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f657c159200 msgr2=0x7f657c0cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f657c159200 0x7f657c0cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 -- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f657c06b5c0 msgr2=0x7f657c0b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f657c06b5c0 0x7f657c0b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 -- 172.21.15.45:0/2735768141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700094f0 con 0x7f657c1595d0 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.450+0000 7f6582b4b700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c0c4da0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6574009ec0 tx=0x7f657400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.451+0000 7f656e7fc700 1 -- 172.21.15.45:0/2735768141 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400cb70 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.451+0000 7f656e7fc700 1 -- 172.21.15.45:0/2735768141 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400f040 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.451+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c169a90 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.451+0000 7f656e7fc700 1 -- 172.21.15.45:0/2735768141 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574014480 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.451+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c0c42c0 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.452+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c0c4ac0 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.452+0000 7f656e7fc700 1 -- 172.21.15.45:0/2735768141 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6574014830 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.453+0000 7f656e7fc700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6560066f30 0x7f65600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.453+0000 7f656e7fc700 1 -- 172.21.15.45:0/2735768141 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7f6574085960 con 0x7f657c1595d0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.453+0000 7f658234a700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6560066f30 0x7f65600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.453+0000 7f658234a700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6560066f30 0x7f65600693e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6578003a80 tx=0x7f6578007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.457+0000 7f656e7fc700 1 -- 172.21.15.45:0/2735768141 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f657c0c4ac0 con 0x7f657c1595d0 2023-10-06T05:24:41.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.706+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f657c0c4ac0 con 0x7f657c1595d0 2023-10-06T05:24:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.707+0000 7f656e7fc700 1 -- 172.21.15.45:0/2735768141 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 236 v236) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f657c0c4ac0 con 0x7f657c1595d0 2023-10-06T05:24:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:41.709 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":236,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":236,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:40.894017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6560066f30 msgr2=0x7f65600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6560066f30 0x7f65600693e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6578003a80 tx=0x7f6578007ec0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 msgr2=0x7f657c0c4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c0c4da0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6574009ec0 tx=0x7f657400b040 comp rx=0 tx=0).stop 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 shutdown_connections 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6560066f30 0x7f65600693e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f657c1595d0 0x7f657c0c4da0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f657c06b5c0 0x7f657c0b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.709+0000 7f65845ae700 1 --2- 172.21.15.45:0/2735768141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f657c159200 0x7f657c0cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.710+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 >> 172.21.15.45:0/2735768141 conn(0x7f657c06f090 msgr2=0x7f657c0aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.710+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 shutdown_connections 2023-10-06T05:24:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.710+0000 7f65845ae700 1 -- 172.21.15.45:0/2735768141 wait complete. 2023-10-06T05:24:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 236 2023-10-06T05:24:41.725 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:41.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:41.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.856+0000 7fe5a680e700 1 Processor -- start 2023-10-06T05:24:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.857+0000 7fe5a680e700 1 -- start start 2023-10-06T05:24:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.857+0000 7fe5a680e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a0159200 0x7fe5a00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.857+0000 7fe5a680e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5a00ccdc0 0x7fe5a00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.857+0000 7fe5a680e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a00c7d00 con 0x7fe5a01595d0 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.857+0000 7fe5a680e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00c7780 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.857+0000 7fe5a680e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00cd740 con 0x7fe5a0159200 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe59ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a0159200 0x7fe5a00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe5a4dab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5a00ccdc0 0x7fe5a00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe5a4dab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5a00ccdc0 0x7fe5a00cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:41840/0 (socket says 172.21.15.45:41840) 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe5a4dab700 1 -- 172.21.15.45:0/2312957229 learned_addr learned my addr 172.21.15.45:0/2312957229 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe59ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a0159200 0x7fe5a00c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:46956/0 (socket says 172.21.15.45:46956) 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe5a4dab700 1 -- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a0159200 msgr2=0x7fe5a00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe5a4dab700 1 --2- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a0159200 0x7fe5a00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe5a4dab700 1 -- 172.21.15.45:0/2312957229 >> v1:172.21.15.144:6789/0 conn(0x7fe5a01595d0 legacy=0x7fe5a006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe5a4dab700 1 -- 172.21.15.45:0/2312957229 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00c86d0 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe59effd700 1 -- 172.21.15.45:0/2312957229 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344040381 0 0) 0x7fe5a00c7d00 con 0x7fe5a01595d0 2023-10-06T05:24:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.858+0000 7fe59ffff700 1 --2- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a0159200 0x7fe5a00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a4dab700 1 --2- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a00cd200 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe598009f20 tx=0x7fe598009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45743d878f176d82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe59effd700 1 -- 172.21.15.45:0/2312957229 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59800e070 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe59effd700 1 -- 172.21.15.45:0/2312957229 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59800b9d0 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe59effd700 1 -- 172.21.15.45:0/2312957229 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5980092e0 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 -- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a00ccdc0 msgr2=0x7fe5a00cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 --2- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a00cd200 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe598009f20 tx=0x7fe598009840 comp rx=0 tx=0).stop 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 -- 172.21.15.45:0/2312957229 shutdown_connections 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 --2- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a00cd200 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 --2- 172.21.15.45:0/2312957229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a0159200 0x7fe5a00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 -- 172.21.15.45:0/2312957229 >> 172.21.15.45:0/2312957229 conn(0x7fe5a006f090 msgr2=0x7fe5a006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 -- 172.21.15.45:0/2312957229 shutdown_connections 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.859+0000 7fe5a680e700 1 -- 172.21.15.45:0/2312957229 wait complete. 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.861+0000 7fe5a680e700 1 Processor -- start 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.861+0000 7fe5a680e700 1 -- start start 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe5a680e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a01599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe5a680e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a0159200 0x7fe5a00b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe5a680e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a01595d0 0x7fe5a00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe5a680e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a015b020 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe5a680e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a015a410 con 0x7fe5a0159200 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe5a680e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a015a6f0 con 0x7fe5a01595d0 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a01599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a01599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42272/0 (socket says 172.21.15.45:42272) 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 -- 172.21.15.45:0/4096961620 learned_addr learned my addr 172.21.15.45:0/4096961620 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe5a4dab700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a01595d0 0x7fe5a00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59f7fe700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a0159200 0x7fe5a00b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 -- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a01595d0 msgr2=0x7fe5a00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a01595d0 0x7fe5a00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 -- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a0159200 msgr2=0x7fe5a00b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a0159200 0x7fe5a00b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 -- 172.21.15.45:0/4096961620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590009420 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe59ffff700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a01599c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ec80 tx=0x7fe5900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.862+0000 7fe597fff700 1 -- 172.21.15.45:0/4096961620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000c920 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.863+0000 7fe597fff700 1 -- 172.21.15.45:0/4096961620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59000ca80 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.863+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe598009ae0 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.863+0000 7fe597fff700 1 -- 172.21.15.45:0/4096961620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000ce00 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.863+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a015acb0 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.864+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a015b300 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.865+0000 7fe597fff700 1 -- 172.21.15.45:0/4096961620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe590013680 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.865+0000 7fe597fff700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe580066f30 0x7fe5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.865+0000 7fe597fff700 1 -- 172.21.15.45:0/4096961620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7fe590085b50 con 0x7fe5a00ccdc0 2023-10-06T05:24:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.865+0000 7fe59f7fe700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe580066f30 0x7fe5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.866+0000 7fe59f7fe700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe580066f30 0x7fe5800693e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe5a015b770 tx=0x7fe588007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:41.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:41.869+0000 7fe597fff700 1 -- 172.21.15.45:0/4096961620 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe590052350 con 0x7fe5a00ccdc0 2023-10-06T05:24:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.120+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5a006bb40 con 0x7fe5a00ccdc0 2023-10-06T05:24:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.121+0000 7fe597fff700 1 -- 172.21.15.45:0/4096961620 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fe590017090 con 0x7fe5a00ccdc0 2023-10-06T05:24:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:42.123 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":237,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":7802,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3602644226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3602644226},{"type":"v1","addr":"172.21.15.144:6835","nonce":3602644226}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":237}],"filesystems":[{"mdsmap":{"epoch":236,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:40.894017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:42.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.123+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe580066f30 msgr2=0x7fe5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.123+0000 7fe5a680e700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe580066f30 0x7fe5800693e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe5a015b770 tx=0x7fe588007ec0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a00ccdc0 msgr2=0x7fe5a01599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a01599c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ec80 tx=0x7fe5900091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 shutdown_connections 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe580066f30 0x7fe5800693e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a01595d0 0x7fe5a00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a00ccdc0 0x7fe5a01599c0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 --2- 172.21.15.45:0/4096961620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a0159200 0x7fe5a00b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 >> 172.21.15.45:0/4096961620 conn(0x7fe5a006f090 msgr2=0x7fe5a00c9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 shutdown_connections 2023-10-06T05:24:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.124+0000 7fe5a680e700 1 -- 172.21.15.45:0/4096961620 wait complete. 2023-10-06T05:24:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 237 2023-10-06T05:24:42.139 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2023-10-06T05:24:42.139 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2023-10-06T05:24:42.139 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:42.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:42.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:42.140 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:42.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:42.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:42.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.336+0000 7fec9bab7700 1 Processor -- start 2023-10-06T05:24:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.336+0000 7fec9bab7700 1 -- start start 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec9bab7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec94158f60 0x7fec94159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec9bab7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec9406a060 0x7fec940cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec9bab7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec94069bd0 con 0x7fec940cc390 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec9bab7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9406a4d0 con 0x7fec94158f60 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec9bab7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec94068fc0 con 0x7fec9406a060 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec99052700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec9406a060 0x7fec940cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec99853700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec94158f60 0x7fec94159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec9a054700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fec940cc390 0x7fec940c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:52648/0 (socket says 172.21.15.45:52648) 2023-10-06T05:24:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec9a054700 1 -- 172.21.15.45:0/2325997587 learned_addr learned my addr 172.21.15.45:0/2325997587 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec99853700 1 -- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec9406a060 msgr2=0x7fec940cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec99853700 1 --2- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec9406a060 0x7fec940cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec99853700 1 -- 172.21.15.45:0/2325997587 >> v1:172.21.15.144:6789/0 conn(0x7fec940cc390 legacy=0x7fec940c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec99853700 1 -- 172.21.15.45:0/2325997587 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec940692a0 con 0x7fec94158f60 2023-10-06T05:24:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec99052700 1 --2- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec9406a060 0x7fec940cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.337+0000 7fec98851700 1 -- 172.21.15.45:0/2325997587 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018768939 0 0) 0x7fec94069bd0 con 0x7fec940cc390 2023-10-06T05:24:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec99853700 1 --2- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec94158f60 0x7fec94159330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fec900098e0 tx=0x7fec900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a539152df98d1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec98851700 1 -- 172.21.15.45:0/2325997587 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9000b640 con 0x7fec94158f60 2023-10-06T05:24:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec98851700 1 -- 172.21.15.45:0/2325997587 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9000b7a0 con 0x7fec94158f60 2023-10-06T05:24:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec98851700 1 -- 172.21.15.45:0/2325997587 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9000bb40 con 0x7fec94158f60 2023-10-06T05:24:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec9bab7700 1 -- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec94158f60 msgr2=0x7fec94159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.338+0000 7fec9bab7700 1 --2- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec94158f60 0x7fec94159330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fec900098e0 tx=0x7fec900091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.339+0000 7fec9bab7700 1 -- 172.21.15.45:0/2325997587 shutdown_connections 2023-10-06T05:24:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.339+0000 7fec9bab7700 1 --2- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec94158f60 0x7fec94159330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.339+0000 7fec9bab7700 1 --2- 172.21.15.45:0/2325997587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec9406a060 0x7fec940cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.339+0000 7fec9bab7700 1 -- 172.21.15.45:0/2325997587 >> 172.21.15.45:0/2325997587 conn(0x7fec940cdf50 msgr2=0x7fec940ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.339+0000 7fec9bab7700 1 -- 172.21.15.45:0/2325997587 shutdown_connections 2023-10-06T05:24:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.339+0000 7fec9bab7700 1 -- 172.21.15.45:0/2325997587 wait complete. 2023-10-06T05:24:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.340+0000 7fec9bab7700 1 Processor -- start 2023-10-06T05:24:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.340+0000 7fec9bab7700 1 -- start start 2023-10-06T05:24:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec9bab7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9406a060 0x7fec9416ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec9bab7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec940cc390 0x7fec94147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec9bab7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec94158f60 0x7fec9415c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec9bab7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940c5d40 con 0x7fec9406a060 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec9bab7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec941480f0 con 0x7fec940cc390 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec9bab7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec940c5130 con 0x7fec94158f60 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9406a060 0x7fec9416ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9406a060 0x7fec9416ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42282/0 (socket says 172.21.15.45:42282) 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 -- 172.21.15.45:0/4044117291 learned_addr learned my addr 172.21.15.45:0/4044117291 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99052700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec940cc390 0x7fec94147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec9a054700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec94158f60 0x7fec9415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 -- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec94158f60 msgr2=0x7fec9415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec94158f60 0x7fec9415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 -- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec940cc390 msgr2=0x7fec94147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec940cc390 0x7fec94147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 -- 172.21.15.45:0/4044117291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec88009420 con 0x7fec9406a060 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.341+0000 7fec99853700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9406a060 0x7fec9416ade0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fec940c4f80 tx=0x7fec90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.342+0000 7fec85ffb700 1 -- 172.21.15.45:0/4044117291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9000bb20 con 0x7fec9406a060 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.342+0000 7fec85ffb700 1 -- 172.21.15.45:0/4044117291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec90000c90 con 0x7fec9406a060 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.342+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec90009420 con 0x7fec9406a060 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.342+0000 7fec85ffb700 1 -- 172.21.15.45:0/4044117291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90011630 con 0x7fec9406a060 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.342+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec940c56f0 con 0x7fec9406a060 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.343+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9406f410 con 0x7fec9406a060 2023-10-06T05:24:42.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.343+0000 7fec85ffb700 1 -- 172.21.15.45:0/4044117291 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec900119a0 con 0x7fec9406a060 2023-10-06T05:24:42.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.344+0000 7fec85ffb700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec78066f30 0x7fec780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.344+0000 7fec85ffb700 1 -- 172.21.15.45:0/4044117291 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7fec90086220 con 0x7fec9406a060 2023-10-06T05:24:42.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.344+0000 7fec99052700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec78066f30 0x7fec780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.345+0000 7fec99052700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec78066f30 0x7fec780693e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec94158840 tx=0x7fec880091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.349+0000 7fec85ffb700 1 -- 172.21.15.45:0/4044117291 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec900529a0 con 0x7fec9406a060 2023-10-06T05:24:42.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:42.599+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fec94069580 con 0x7fec9406a060 2023-10-06T05:24:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.131+0000 7fec85ffb700 1 -- 172.21.15.45:0/4044117291 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v238) v1 ==== 161+0+0 (secure 0 0 0) 0x7fec90055fc0 con 0x7fec9406a060 2023-10-06T05:24:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec78066f30 msgr2=0x7fec780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec78066f30 0x7fec780693e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec94158840 tx=0x7fec880091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9406a060 msgr2=0x7fec9416ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9406a060 0x7fec9416ade0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fec940c4f80 tx=0x7fec90017000 comp rx=0 tx=0).stop 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 shutdown_connections 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec78066f30 0x7fec780693e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec94158f60 0x7fec9415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec9406a060 0x7fec9416ade0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 --2- 172.21.15.45:0/4044117291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec940cc390 0x7fec94147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 >> 172.21.15.45:0/4044117291 conn(0x7fec940cdf50 msgr2=0x7fec940b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 shutdown_connections 2023-10-06T05:24:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.134+0000 7fec9bab7700 1 -- 172.21.15.45:0/4044117291 wait complete. 2023-10-06T05:24:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:43.151 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.286+0000 7f20a1a0b700 1 Processor -- start 2023-10-06T05:24:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.286+0000 7f20a1a0b700 1 -- start start 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f20a1a0b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f209c1539f0 0x7f209c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f20a1a0b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c0ccfc0 0x7f209c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f20a1a0b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c0ce690 con 0x7f209c153620 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f20a1a0b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c15a7a0 con 0x7f209c1539f0 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f20a1a0b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0ce3f0 con 0x7f209c0ccfc0 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f209a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f209c1539f0 0x7f209c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f209affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f209c153620 0x7f209c0cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56486/0 (socket says 172.21.15.45:56486) 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f209a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f209c1539f0 0x7f209c15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42288/0 (socket says 172.21.15.45:42288) 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.287+0000 7f209affd700 1 -- 172.21.15.45:0/2093827883 learned_addr learned my addr 172.21.15.45:0/2093827883 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f209b7fe700 1 --2- 172.21.15.45:0/2093827883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c0ccfc0 0x7f209c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f209a7fc700 1 -- 172.21.15.45:0/2093827883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c0ccfc0 msgr2=0x7f209c15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f209a7fc700 1 --2- 172.21.15.45:0/2093827883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c0ccfc0 0x7f209c15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f209a7fc700 1 -- 172.21.15.45:0/2093827883 >> v1:172.21.15.45:6789/0 conn(0x7f209c153620 legacy=0x7f209c0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f209a7fc700 1 -- 172.21.15.45:0/2093827883 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c15d510 con 0x7f209c1539f0 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f2099ffb700 1 -- 172.21.15.45:0/2093827883 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406605499 0 0) 0x7f209c0ce690 con 0x7f209c153620 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f209a7fc700 1 --2- 172.21.15.45:0/2093827883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c1539f0 0x7f209c15a260 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f20840098e0 tx=0x7f20840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3b0ba034074e37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f2099ffb700 1 -- 172.21.15.45:0/2093827883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084009e80 con 0x7f209c1539f0 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f2099ffb700 1 -- 172.21.15.45:0/2093827883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208400b860 con 0x7f209c1539f0 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.288+0000 7f2099ffb700 1 -- 172.21.15.45:0/2093827883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208400bbe0 con 0x7f209c1539f0 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 -- 172.21.15.45:0/2093827883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c1539f0 msgr2=0x7f209c15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/2093827883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c1539f0 0x7f209c15a260 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f20840098e0 tx=0x7f20840091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 -- 172.21.15.45:0/2093827883 shutdown_connections 2023-10-06T05:24:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/2093827883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c0ccfc0 0x7f209c15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/2093827883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c1539f0 0x7f209c15a260 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 -- 172.21.15.45:0/2093827883 >> 172.21.15.45:0/2093827883 conn(0x7f209c06ed50 msgr2=0x7f209c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 -- 172.21.15.45:0/2093827883 shutdown_connections 2023-10-06T05:24:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.289+0000 7f20a1a0b700 1 -- 172.21.15.45:0/2093827883 wait complete. 2023-10-06T05:24:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.290+0000 7f20a1a0b700 1 Processor -- start 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.290+0000 7f20a1a0b700 1 -- start start 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f20a1a0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c0ccfc0 0x7f209c16ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f20a1a0b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c153620 0x7f209c14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f20a1a0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209c150f20 0x7f209c0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f20a1a0b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c15bb90 con 0x7f209c0ccfc0 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f20a1a0b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c153dd0 con 0x7f209c150f20 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f20a1a0b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c16d3b0 con 0x7f209c153620 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c0ccfc0 0x7f209c16ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c0ccfc0 0x7f209c16ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42296/0 (socket says 172.21.15.45:42296) 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 -- 172.21.15.45:0/1763254819 learned_addr learned my addr 172.21.15.45:0/1763254819 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209a7fc700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c153620 0x7f209c14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209b7fe700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209c150f20 0x7f209c0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 -- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c153620 msgr2=0x7f209c14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c153620 0x7f209c14ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 -- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209c150f20 msgr2=0x7f209c0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209c150f20 0x7f209c0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.291+0000 7f209affd700 1 -- 172.21.15.45:0/1763254819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084009420 con 0x7f209c0ccfc0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.292+0000 7f209affd700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c0ccfc0 0x7f209c16ce70 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7e0 tx=0x7f208c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.292+0000 7f2092ffd700 1 -- 172.21.15.45:0/1763254819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c005b30 con 0x7f209c0ccfc0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.292+0000 7f2092ffd700 1 -- 172.21.15.45:0/1763254819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c005c90 con 0x7f209c0ccfc0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.292+0000 7f2092ffd700 1 -- 172.21.15.45:0/1763254819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c007300 con 0x7f209c0ccfc0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.292+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c154020 con 0x7f209c0ccfc0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.292+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c1545e0 con 0x7f209c0ccfc0 2023-10-06T05:24:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.293+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c088a60 con 0x7f209c0ccfc0 2023-10-06T05:24:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.293+0000 7f2092ffd700 1 -- 172.21.15.45:0/1763254819 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f208c01b400 con 0x7f209c0ccfc0 2023-10-06T05:24:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.294+0000 7f2092ffd700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f207c066f30 0x7f207c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.294+0000 7f2092ffd700 1 -- 172.21.15.45:0/1763254819 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7f208c084b30 con 0x7f209c0ccfc0 2023-10-06T05:24:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.294+0000 7f209a7fc700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f207c066f30 0x7f207c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.295+0000 7f209a7fc700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f207c066f30 0x7f207c0693e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f209c0b2c90 tx=0x7f2084017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.299+0000 7f2092ffd700 1 -- 172.21.15.45:0/1763254819 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f208c051330 con 0x7f209c0ccfc0 2023-10-06T05:24:43.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:43.592+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f209c04bb40 con 0x7f209c0ccfc0 2023-10-06T05:24:44.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.133+0000 7f2092ffd700 1 -- 172.21.15.45:0/1763254819 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v239) v1 ==== 101+0+0 (secure 0 0 0) 0x7f208c054950 con 0x7f209c0ccfc0 2023-10-06T05:24:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f207c066f30 msgr2=0x7f207c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f207c066f30 0x7f207c0693e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f209c0b2c90 tx=0x7f2084017000 comp rx=0 tx=0).stop 2023-10-06T05:24:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c0ccfc0 msgr2=0x7f209c16ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c0ccfc0 0x7f209c16ce70 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7e0 tx=0x7f208c007df0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 shutdown_connections 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f207c066f30 0x7f207c0693e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209c150f20 0x7f209c0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f209c0ccfc0 0x7f209c16ce70 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 --2- 172.21.15.45:0/1763254819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209c153620 0x7f209c14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.137+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 >> 172.21.15.45:0/1763254819 conn(0x7f209c06ed50 msgr2=0x7f209c151fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.138+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 shutdown_connections 2023-10-06T05:24:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.138+0000 7f20a1a0b700 1 -- 172.21.15.45:0/1763254819 wait complete. 2023-10-06T05:24:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:44.156 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.202+0000 7f4ba576e040 1 Processor -- start 2023-10-06T05:24:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.202+0000 7f4ba576e040 1 -- start start 2023-10-06T05:24:44.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4ba576e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f1057ca0 0x55b7f1101460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4ba576e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f10737e0 0x55b7f11019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4ba576e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 0x55b7f10f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4ba576e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f0f76980 con 0x55b7f10737e0 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4ba576e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f0f76620 con 0x55b7f1057ca0 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4ba576e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7f0f6d400 con 0x55b7f10feb30 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4b9e18b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f10737e0 0x55b7f11019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4b9e18b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f10737e0 0x55b7f11019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42316/0 (socket says 172.21.15.45:42316) 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4b9e18b700 1 -- 172.21.15.45:0/4114205678 learned_addr learned my addr 172.21.15.45:0/4114205678 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.203+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f1057ca0 0x55b7f1101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 0x55b7f10f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9e18b700 1 -- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 msgr2=0x55b7f10f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 0x55b7f10f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9e18b700 1 -- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f1057ca0 msgr2=0x55b7f1101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f1057ca0 0x55b7f1101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9e18b700 1 -- 172.21.15.45:0/4114205678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b88005db0 con 0x55b7f10737e0 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f10737e0 0x55b7f11019a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800ad40 tx=0x7f4b88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d715755c27609c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9d189700 1 -- 172.21.15.45:0/4114205678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88020070 con 0x55b7f10737e0 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9d189700 1 -- 172.21.15.45:0/4114205678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b880054f0 con 0x55b7f10737e0 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.204+0000 7f4b9d189700 1 -- 172.21.15.45:0/4114205678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b880089d0 con 0x55b7f10737e0 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 -- 172.21.15.45:0/4114205678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f10737e0 msgr2=0x55b7f11019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f10737e0 0x55b7f11019a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800ad40 tx=0x7f4b88016000 comp rx=0 tx=0).stop 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 -- 172.21.15.45:0/4114205678 shutdown_connections 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 0x55b7f10f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f1057ca0 0x55b7f1101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 --2- 172.21.15.45:0/4114205678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f10737e0 0x55b7f11019a0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 -- 172.21.15.45:0/4114205678 >> 172.21.15.45:0/4114205678 conn(0x55b7f102c450 msgr2=0x55b7f10fa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 -- 172.21.15.45:0/4114205678 shutdown_connections 2023-10-06T05:24:44.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.205+0000 7f4ba576e040 1 -- 172.21.15.45:0/4114205678 wait complete. 2023-10-06T05:24:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.206+0000 7f4ba576e040 1 Processor -- start 2023-10-06T05:24:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.206+0000 7f4ba576e040 1 -- start start 2023-10-06T05:24:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4ba576e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f1057ca0 0x55b7f10fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4ba576e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f10737e0 0x55b7f106be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4ba576e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 0x55b7f106c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4ba576e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f10f6270 con 0x55b7f1057ca0 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4ba576e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f1058a30 con 0x55b7f10737e0 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4ba576e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7f10fdde0 con 0x55b7f10feb30 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f1057ca0 0x55b7f10fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f1057ca0 0x55b7f10fd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42328/0 (socket says 172.21.15.45:42328) 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 learned_addr learned my addr 172.21.15.45:0/3987839669 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f10737e0 0x55b7f106be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 0x55b7f106c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 msgr2=0x55b7f106c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f10feb30 0x55b7f106c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f10737e0 msgr2=0x55b7f106be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f10737e0 0x55b7f106be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.207+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7f0f9f1c0 con 0x55b7f1057ca0 2023-10-06T05:24:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.208+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f1057ca0 0x55b7f10fd4a0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4b90002fd0 tx=0x7f4b90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.208+0000 7f4b977fe700 1 -- 172.21.15.45:0/3987839669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000cd80 con 0x55b7f1057ca0 2023-10-06T05:24:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.208+0000 7f4b977fe700 1 -- 172.21.15.45:0/3987839669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90008030 con 0x55b7f1057ca0 2023-10-06T05:24:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.208+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b7f0f853d0 con 0x55b7f1057ca0 2023-10-06T05:24:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.208+0000 7f4b977fe700 1 -- 172.21.15.45:0/3987839669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000cee0 con 0x55b7f1057ca0 2023-10-06T05:24:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.208+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7f0f78ae0 con 0x55b7f1057ca0 2023-10-06T05:24:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.209+0000 7f4b977fe700 1 -- 172.21.15.45:0/3987839669 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b900118b0 con 0x55b7f1057ca0 2023-10-06T05:24:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.210+0000 7f4b977fe700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b800658c0 0x7f4b80067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.210+0000 7f4b977fe700 1 -- 172.21.15.45:0/3987839669 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7f4b90083f90 con 0x55b7f1057ca0 2023-10-06T05:24:44.214 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.210+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b7f1110b00 0x55b7f1119ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.210+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111a000 con 0x55b7f1110b00 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.210+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b800658c0 0x7f4b80067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.210+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b7f1110b00 0x55b7f1119ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.211+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b7f1110b00 0x55b7f1119ac0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.211+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b800658c0 0x7f4b80067d70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4b88027000 tx=0x7f4b8801d070 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.213+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v116'722 uv696 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.213+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111a3b0 con 0x55b7f1110b00 2023-10-06T05:24:44.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.214+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v116'723 uv723 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.214+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111a760 con 0x55b7f1110b00 2023-10-06T05:24:44.217 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:44.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.215+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v116'724 uv724 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.215+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111bb20 con 0x55b7f1110b00 2023-10-06T05:24:44.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.217+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'725 uv725 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.217+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111bed0 con 0x55b7f1110b00 2023-10-06T05:24:44.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.218+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'726 uv726 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.218+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111b000 con 0x55b7f1110b00 2023-10-06T05:24:44.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.219+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'727 uv727 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.219+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111b430 con 0x55b7f1110b00 2023-10-06T05:24:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.221+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v116'728 uv702 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.221+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.222+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v116'729 uv729 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.222+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.225 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:44.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.223+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v116'730 uv730 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.223+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.225+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'731 uv731 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.225+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.226+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'732 uv732 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.226+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.227+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'733 uv733 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.227+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.228+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v116'734 uv709 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.228+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.230+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'735 uv735 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.230+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.232 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.231+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v116'736 uv736 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.231+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.232+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'737 uv737 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.232+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.233+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'738 uv738 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.233+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.235+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'739 uv739 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.235+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.236+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'740 uv740 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.236+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.237+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v116'741 uv715 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.237+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.239+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v116'742 uv742 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.239+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.242 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.240+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v116'743 uv743 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.240+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.241+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'744 uv744 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.241+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.242+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'745 uv745 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.242+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.243+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'746 uv746 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.243+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.245+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v116'747 uv721 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.245+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.246+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v116'748 uv748 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.246+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.249 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.248+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v116'749 uv749 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.248+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.249+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'750 uv750 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.249+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111d290 con 0x55b7f1110b00 2023-10-06T05:24:44.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.250+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'751 uv751 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.250+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111c280 con 0x55b7f1110b00 2023-10-06T05:24:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.251+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'752 uv752 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1110b00 2023-10-06T05:24:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.251+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b7f111c790 0x55b7f111eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.251+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111f0b0 con 0x55b7f111c790 2023-10-06T05:24:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.251+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b7f111c790 0x55b7f111eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.252+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b7f111c790 0x55b7f111eb70 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.253+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'522 uv521 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.253+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111f460 con 0x55b7f111c790 2023-10-06T05:24:44.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.255+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'523 uv523 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.255+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f111f810 con 0x55b7f111c790 2023-10-06T05:24:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.256+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'524 uv524 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.258 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:44.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.256+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11200b0 con 0x55b7f111c790 2023-10-06T05:24:44.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.257+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v116'525 uv525 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.257+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120460 con 0x55b7f111c790 2023-10-06T05:24:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.258+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'526 uv526 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.259+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120810 con 0x55b7f111c790 2023-10-06T05:24:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.260+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v116'527 uv509 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.260+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120bc0 con 0x55b7f111c790 2023-10-06T05:24:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.261+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v116'528 uv528 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.261+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.263+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v116'529 uv529 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.265 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.263+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.264+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'530 uv530 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.264+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.265+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'531 uv531 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.265+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.266+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v116'532 uv514 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.267+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.268+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v116'533 uv533 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.268+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.269+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v116'534 uv534 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.271 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.269+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1124690 con 0x55b7f111c790 2023-10-06T05:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.271+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v116'535 uv535 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.271+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1124690 con 0x55b7f111c790 2023-10-06T05:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.272+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'536 uv536 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.272+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.274+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v116'537 uv520 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.274+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.275+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v116'538 uv538 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.275+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.278 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.276+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v116'539 uv539 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.276+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1121f80 con 0x55b7f111c790 2023-10-06T05:24:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.277+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'540 uv540 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.277+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1121f80 con 0x55b7f111c790 2023-10-06T05:24:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.278+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'541 uv541 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.279+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1120f70 con 0x55b7f111c790 2023-10-06T05:24:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.280+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'542 uv542 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.280+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b7f1121480 0x55b7f1123860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.280+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1123da0 con 0x55b7f1121480 2023-10-06T05:24:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.280+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b7f1121480 0x55b7f1123860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.280+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b7f1121480 0x55b7f1123860 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.282+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.282+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1124230 con 0x55b7f1121480 2023-10-06T05:24:44.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.283+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v116'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.283+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11247b0 con 0x55b7f1121480 2023-10-06T05:24:44.285 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.284+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v116'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.284+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11260b0 con 0x55b7f1121480 2023-10-06T05:24:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.285+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.285+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1126460 con 0x55b7f1121480 2023-10-06T05:24:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.286+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.287+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11250a0 con 0x55b7f1121480 2023-10-06T05:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.288+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.288+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11254c0 con 0x55b7f1121480 2023-10-06T05:24:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.289+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'100 uv99 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.289+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11258f0 con 0x55b7f1121480 2023-10-06T05:24:44.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.290+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'101 uv101 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.290+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11258f0 con 0x55b7f1121480 2023-10-06T05:24:44.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.292+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'102 uv102 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.293 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.292+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f11258a0 con 0x55b7f1121480 2023-10-06T05:24:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.293+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'103 uv103 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.293+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1125cc0 con 0x55b7f111c790 2023-10-06T05:24:44.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.294+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.294+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1125cc0 con 0x55b7f111c790 2023-10-06T05:24:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.295+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v116'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.295+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1125cc0 con 0x55b7f111c790 2023-10-06T05:24:44.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.296+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v116'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.298 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:44.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.296+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1125cc0 con 0x55b7f111c790 2023-10-06T05:24:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.298+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.298+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1125cc0 con 0x55b7f111c790 2023-10-06T05:24:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.299+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.299+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1125cc0 con 0x55b7f111c790 2023-10-06T05:24:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.300+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f111c790 2023-10-06T05:24:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.300+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b7f1125870 0x55b7f1128d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.300+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.300+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b7f1125870 0x55b7f1128d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.301+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b7f1125870 0x55b7f1128d30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.302+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v116'260 uv254 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.302+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.303+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v116'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.303+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.304+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v116'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.306 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.305+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112a2c0 con 0x55b7f1125870 2023-10-06T05:24:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.306+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'263 uv263 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.306+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112a280 con 0x55b7f1125870 2023-10-06T05:24:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.307+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'264 uv264 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.307+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.308+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'265 uv265 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.308+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.309+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v116'266 uv259 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.309+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.311+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v116'267 uv267 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.311+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.312+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v116'268 uv268 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.313 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:44.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.312+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129270 con 0x55b7f1125870 2023-10-06T05:24:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.313+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'269 uv269 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.313+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1129640 con 0x55b7f1125870 2023-10-06T05:24:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.314+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.315+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b7f1129b50 0x55b7f1129f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.315+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112c710 con 0x55b7f1129b50 2023-10-06T05:24:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.315+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b7f1129b50 0x55b7f1129f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.315+0000 7f4b9e98c700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b7f1129b50 0x55b7f1129f60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.316+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v116'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f1129b50 2023-10-06T05:24:44.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.316+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112c710 con 0x55b7f1129b50 2023-10-06T05:24:44.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.318+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v116'118 uv118 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f1129b50 2023-10-06T05:24:44.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.318+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112c710 con 0x55b7f1129b50 2023-10-06T05:24:44.320 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:44.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.319+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v116'119 uv119 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f1129b50 2023-10-06T05:24:44.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.319+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112c750 con 0x55b7f1129b50 2023-10-06T05:24:44.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.321+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'120 uv120 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f1129b50 2023-10-06T05:24:44.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.321+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112cb50 con 0x55b7f1129b50 2023-10-06T05:24:44.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.322+0000 7f4b9e98c700 1 -- 172.21.15.45:0/3987839669 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'121 uv121 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b90017070 con 0x55b7f1129b50 2023-10-06T05:24:44.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.322+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112cb50 con 0x55b7f1121480 2023-10-06T05:24:44.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.323+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.323+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112cb50 con 0x55b7f1121480 2023-10-06T05:24:44.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.325+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v116'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.325+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112cb50 con 0x55b7f1121480 2023-10-06T05:24:44.327 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:44.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.326+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v116'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.326+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112db60 con 0x55b7f1121480 2023-10-06T05:24:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.327+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.327+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112db60 con 0x55b7f1121480 2023-10-06T05:24:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.328+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.328+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112cb50 con 0x55b7f1121480 2023-10-06T05:24:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.330+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.330+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b7f112d060 0x55b7f112f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.330+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112f9c0 con 0x55b7f112d060 2023-10-06T05:24:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.330+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b7f112d060 0x55b7f112f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.330+0000 7f4b9e18b700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b7f112d060 0x55b7f112f440 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.332+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v116'233 uv227 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.332+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112f9c0 con 0x55b7f112d060 2023-10-06T05:24:44.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.333+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v116'234 uv234 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.333+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112fe20 con 0x55b7f112d060 2023-10-06T05:24:44.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.334+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v116'235 uv235 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.336 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:44.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.334+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112fe20 con 0x55b7f112d060 2023-10-06T05:24:44.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.335+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'236 uv236 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.335+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112fe20 con 0x55b7f112d060 2023-10-06T05:24:44.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.337+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'237 uv237 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.337+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112fe20 con 0x55b7f112d060 2023-10-06T05:24:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.338+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v116'238 uv232 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.338+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112fe20 con 0x55b7f112d060 2023-10-06T05:24:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.339+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v116'239 uv239 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.339+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112fe20 con 0x55b7f112d060 2023-10-06T05:24:44.341 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:44.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.340+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v116'240 uv240 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.340+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f112fe20 con 0x55b7f112d060 2023-10-06T05:24:44.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.341+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'241 uv241 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.341+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1132b90 con 0x55b7f112d060 2023-10-06T05:24:44.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.342+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'242 uv242 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f112d060 2023-10-06T05:24:44.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.342+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.344+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v116'103 uv102 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.344+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.345+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'104 uv104 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.345+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.346+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v116'105 uv105 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.348 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.346+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.347+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'106 uv106 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.347+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b7f1130c40 0x55b7f11332e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.347+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1130c40 2023-10-06T05:24:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.347+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b7f1130c40 0x55b7f11332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.348+0000 7f4b9f18d700 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b7f1130c40 0x55b7f11332e0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.349+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1130c40 2023-10-06T05:24:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.349+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1130c40 2023-10-06T05:24:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.352+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v116'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1130c40 2023-10-06T05:24:44.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.353+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1130c40 2023-10-06T05:24:44.355 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:44.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.354+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v116'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1130c40 2023-10-06T05:24:44.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.354+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1130c40 2023-10-06T05:24:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.356+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1130c40 2023-10-06T05:24:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.356+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1130c40 2023-10-06T05:24:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.359+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1130c40 2023-10-06T05:24:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.359+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1130c40 2023-10-06T05:24:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.362+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1130c40 2023-10-06T05:24:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.362+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1125870 2023-10-06T05:24:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.363+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v116'120 uv119 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.363+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1125870 2023-10-06T05:24:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.365+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v116'121 uv121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.365+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1125870 2023-10-06T05:24:44.367 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.366+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v116'122 uv122 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.366+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1125870 2023-10-06T05:24:44.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.367+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'123 uv123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.367+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1125870 2023-10-06T05:24:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.368+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'124 uv124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b9800bb60 con 0x55b7f1125870 2023-10-06T05:24:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.368+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.370+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v116'290 uv282 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.370+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.371+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v116'291 uv291 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.371+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.372+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v116'292 uv292 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.374 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.372+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.374+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v116'293 uv293 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.374+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.375+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'294 uv294 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.375+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.376+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v116'295 uv289 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.376+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.378+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v116'296 uv296 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.378+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.382+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v116'297 uv297 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.383 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:44.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.382+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.383+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'298 uv298 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.383+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130310 con 0x55b7f1121480 2023-10-06T05:24:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.386+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'299 uv299 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.386+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1133820 con 0x55b7f1121480 2023-10-06T05:24:44.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.387+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'300 uv300 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.387+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b7f1130220 con 0x55b7f1121480 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4b9e18b700 1 -- 172.21.15.45:0/3987839669 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'301 uv301 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4b8800a040 con 0x55b7f1121480 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b7f1121480 msgr2=0x55b7f1123860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b7f1121480 0x55b7f1123860 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b7f1110b00 msgr2=0x55b7f1119ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b7f1110b00 0x55b7f1119ac0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b7f112d060 msgr2=0x55b7f112f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b7f112d060 0x55b7f112f440 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b7f111c790 msgr2=0x55b7f111eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b7f111c790 0x55b7f111eb70 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b7f1130c40 msgr2=0x55b7f11332e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b7f1130c40 0x55b7f11332e0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b7f1129b50 msgr2=0x55b7f1129f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b7f1129b50 0x55b7f1129f60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b7f1125870 msgr2=0x55b7f1128d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b7f1125870 0x55b7f1128d30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 reap_dead start 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 reap_dead start 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 reap_dead start 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 reap_dead start 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4b9f18d700 1 -- 172.21.15.45:0/3987839669 reap_dead start 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b800658c0 msgr2=0x7f4b80067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b800658c0 0x7f4b80067d70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4b88027000 tx=0x7f4b8801d070 comp rx=0 tx=0).stop 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f1057ca0 msgr2=0x55b7f10fd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f1057ca0 0x55b7f10fd4a0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4b90002fd0 tx=0x7f4b90000bc0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 shutdown_connections 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7f1057ca0 0x55b7f10fd4a0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 --2- 172.21.15.45:0/3987839669 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b800658c0 0x7f4b80067d70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 >> 172.21.15.45:0/3987839669 conn(0x55b7f102c450 msgr2=0x55b7f10fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 shutdown_connections 2023-10-06T05:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.390+0000 7f4ba576e040 1 -- 172.21.15.45:0/3987839669 wait complete. 2023-10-06T05:24:44.396 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2023-10-06T05:24:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.444+0000 7ffa4885e040 1 Processor -- start 2023-10-06T05:24:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4885e040 1 -- start start 2023-10-06T05:24:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4885e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a94bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4885e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a91e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4885e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 0x55b40a9f4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4885e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40a8c4320 con 0x55b40a967340 2023-10-06T05:24:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4885e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40a868980 con 0x55b40a9dd970 2023-10-06T05:24:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4885e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b40a868620 con 0x55b40a91c1b0 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4127b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a91e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4127b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a91e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42344/0 (socket says 172.21.15.45:42344) 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.445+0000 7ffa4127b700 1 -- 172.21.15.45:0/985475950 learned_addr learned my addr 172.21.15.45:0/985475950 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa4227d700 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 0x55b40a9f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa41a7c700 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a94bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa4127b700 1 -- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 msgr2=0x55b40a94bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa4127b700 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a94bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa4127b700 1 -- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 msgr2=0x55b40a9f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa4127b700 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 0x55b40a9f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa4127b700 1 -- 172.21.15.45:0/985475950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3400a0e0 con 0x55b40a967340 2023-10-06T05:24:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa41a7c700 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a94bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa4127b700 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a91e870 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b4d0 tx=0x7ffa380018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d41e2daec226670 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa33fff700 1 -- 172.21.15.45:0/985475950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800be80 con 0x55b40a967340 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa33fff700 1 -- 172.21.15.45:0/985475950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3800e470 con 0x55b40a967340 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.446+0000 7ffa33fff700 1 -- 172.21.15.45:0/985475950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800e830 con 0x55b40a967340 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 -- 172.21.15.45:0/985475950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 msgr2=0x55b40a91e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a91e870 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b4d0 tx=0x7ffa380018f0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 -- 172.21.15.45:0/985475950 shutdown_connections 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 0x55b40a9f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a94bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 --2- 172.21.15.45:0/985475950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a91e870 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 -- 172.21.15.45:0/985475950 >> 172.21.15.45:0/985475950 conn(0x55b40a91e4a0 msgr2=0x55b40a9f3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 -- 172.21.15.45:0/985475950 shutdown_connections 2023-10-06T05:24:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.447+0000 7ffa4885e040 1 -- 172.21.15.45:0/985475950 wait complete. 2023-10-06T05:24:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 Processor -- start 2023-10-06T05:24:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 -- start start 2023-10-06T05:24:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a9d54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a9f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 0x55b40a9f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40a85f400 con 0x55b40a967340 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40a9f2ed0 con 0x55b40a9dd970 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.448+0000 7ffa4885e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b40aa046b0 con 0x55b40a91c1b0 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a9f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa41a7c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a9d54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a9f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42356/0 (socket says 172.21.15.45:42356) 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 -- 172.21.15.45:0/3885466879 learned_addr learned my addr 172.21.15.45:0/3885466879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4227d700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 0x55b40a9f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 -- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 msgr2=0x55b40a9d54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40a91c1b0 0x55b40a9d54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 -- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 msgr2=0x55b40a9f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40a9dd970 0x55b40a9f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 -- 172.21.15.45:0/3885466879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38001bd0 con 0x55b40a967340 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4127b700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a9f4d50 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800e910 tx=0x7ffa38014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa327fc700 1 -- 172.21.15.45:0/3885466879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3801f070 con 0x55b40a967340 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa327fc700 1 -- 172.21.15.45:0/3885466879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa38001590 con 0x55b40a967340 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b40a86aae0 con 0x55b40a967340 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa327fc700 1 -- 172.21.15.45:0/3885466879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38018870 con 0x55b40a967340 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.449+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b40a86a540 con 0x55b40a967340 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.451+0000 7ffa327fc700 1 -- 172.21.15.45:0/3885466879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa38018ca0 con 0x55b40a967340 2023-10-06T05:24:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.451+0000 7ffa327fc700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffa24065890 0x7ffa24067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.451+0000 7ffa327fc700 1 -- 172.21.15.45:0/3885466879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7ffa38085900 con 0x55b40a967340 2023-10-06T05:24:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.451+0000 7ffa41a7c700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffa24065890 0x7ffa24067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.452+0000 7ffa4885e040 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b40aa07cb0 0x55b40aa08080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.452+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) v8 -- 0x55b40aa085c0 con 0x55b40aa07cb0 2023-10-06T05:24:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.452+0000 7ffa4227d700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b40aa07cb0 0x55b40aa08080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.452+0000 7ffa41a7c700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffa24065890 0x7ffa24067d40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e6d0 tx=0x7ffa3c0018f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.452+0000 7ffa4227d700 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b40aa07cb0 0x55b40aa08080 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4227d700 1 -- 172.21.15.45:0/3885466879 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v116'543 uv543 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ffa3400f860 con 0x55b40aa07cb0 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b40aa07cb0 msgr2=0x55b40aa08080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b40aa07cb0 0x55b40aa08080 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffa24065890 msgr2=0x7ffa24067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ffa24065890 0x7ffa24067d40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e6d0 tx=0x7ffa3c0018f0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 msgr2=0x55b40a9f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 --2- 172.21.15.45:0/3885466879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b40a967340 0x55b40a9f4d50 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800e910 tx=0x7ffa38014000 comp rx=0 tx=0).stop 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4227d700 1 -- 172.21.15.45:0/3885466879 reap_dead start 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 shutdown_connections 2023-10-06T05:24:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 >> 172.21.15.45:0/3885466879 conn(0x55b40a91e4a0 msgr2=0x55b40a9daeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 shutdown_connections 2023-10-06T05:24:44.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.454+0000 7ffa4885e040 1 -- 172.21.15.45:0/3885466879 wait complete. 2023-10-06T05:24:44.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.597+0000 7f5c7eac5700 1 Processor -- start 2023-10-06T05:24:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.598+0000 7f5c7eac5700 1 -- start start 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.598+0000 7f5c7eac5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c780cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.598+0000 7f5c7eac5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c780c8de0 0x7f5c780c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.598+0000 7f5c7eac5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c780c9d50 0x7f5c7806c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.598+0000 7f5c7eac5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780c9630 con 0x7f5c780c8de0 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.598+0000 7f5c7eac5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780ca150 con 0x7f5c780c9d50 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.598+0000 7f5c7eac5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780cb550 con 0x7f5c7813f900 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c780c8de0 0x7f5c780c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c780c8de0 0x7f5c780c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42368/0 (socket says 172.21.15.45:42368) 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 -- 172.21.15.45:0/2226498878 learned_addr learned my addr 172.21.15.45:0/2226498878 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c7d062700 1 --2- 172.21.15.45:0/2226498878 >> v2:172.21.15.144:3300/0 conn(0x7f5c780c9d50 0x7f5c7806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c7c861700 1 --2- 172.21.15.45:0/2226498878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c780cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 -- 172.21.15.45:0/2226498878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 msgr2=0x7f5c780cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 --2- 172.21.15.45:0/2226498878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c780cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 -- 172.21.15.45:0/2226498878 >> v2:172.21.15.144:3300/0 conn(0x7f5c780c9d50 msgr2=0x7f5c7806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 --2- 172.21.15.45:0/2226498878 >> v2:172.21.15.144:3300/0 conn(0x7f5c780c9d50 0x7f5c7806c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 -- 172.21.15.45:0/2226498878 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780cb830 con 0x7f5c780c8de0 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c7d062700 1 --2- 172.21.15.45:0/2226498878 >> v2:172.21.15.144:3300/0 conn(0x7f5c780c9d50 0x7f5c7806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c7c861700 1 --2- 172.21.15.45:0/2226498878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c780cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c77fff700 1 --2- 172.21.15.45:0/2226498878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c8de0 0x7f5c780c9810 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5c780cc620 tx=0x7f5c64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0409945e1588b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c777fe700 1 -- 172.21.15.45:0/2226498878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400b860 con 0x7f5c780c8de0 2023-10-06T05:24:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c777fe700 1 -- 172.21.15.45:0/2226498878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6400b9c0 con 0x7f5c780c8de0 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.599+0000 7f5c777fe700 1 -- 172.21.15.45:0/2226498878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400bd60 con 0x7f5c780c8de0 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 -- 172.21.15.45:0/2226498878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c8de0 msgr2=0x7f5c780c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/2226498878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c8de0 0x7f5c780c9810 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5c780cc620 tx=0x7f5c64009250 comp rx=0 tx=0).stop 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 -- 172.21.15.45:0/2226498878 shutdown_connections 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/2226498878 >> v2:172.21.15.144:3300/0 conn(0x7f5c780c9d50 0x7f5c7806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/2226498878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c780cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/2226498878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c8de0 0x7f5c780c9810 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 -- 172.21.15.45:0/2226498878 >> 172.21.15.45:0/2226498878 conn(0x7f5c780691c0 msgr2=0x7f5c780cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 -- 172.21.15.45:0/2226498878 shutdown_connections 2023-10-06T05:24:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.600+0000 7f5c7eac5700 1 -- 172.21.15.45:0/2226498878 wait complete. 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.601+0000 7f5c7eac5700 1 Processor -- start 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.601+0000 7f5c7eac5700 1 -- start start 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c7eac5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c780c8de0 0x7f5c780c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c7eac5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c9d50 0x7f5c780c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c7eac5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c78138920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c7eac5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c6950 con 0x7f5c780c9d50 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c7eac5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c64c0 con 0x7f5c780c8de0 2023-10-06T05:24:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c7eac5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c4df0 con 0x7f5c7813f900 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c9d50 0x7f5c780c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c9d50 0x7f5c780c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42384/0 (socket says 172.21.15.45:42384) 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c77fff700 1 -- 172.21.15.45:0/12202057 learned_addr learned my addr 172.21.15.45:0/12202057 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.602+0000 7f5c7d062700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c78138920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c7c861700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c780c8de0 0x7f5c780c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c77fff700 1 -- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 msgr2=0x7f5c78138920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c77fff700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c78138920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c77fff700 1 -- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c780c8de0 msgr2=0x7f5c780c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c77fff700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c780c8de0 0x7f5c780c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c77fff700 1 -- 172.21.15.45:0/12202057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640094f0 con 0x7f5c780c9d50 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c7d062700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c78138920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:44.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c7c861700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c780c8de0 0x7f5c780c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c77fff700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c9d50 0x7f5c780c48b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5c64006fd0 tx=0x7f5c64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c74ff9700 1 -- 172.21.15.45:0/12202057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64024440 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c74ff9700 1 -- 172.21.15.45:0/12202057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640245a0 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c74ff9700 1 -- 172.21.15.45:0/12202057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64024960 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780c58b0 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.603+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780c5e70 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.604+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c780c6670 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.604+0000 7f5c74ff9700 1 -- 172.21.15.45:0/12202057 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c64024d50 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.605+0000 7f5c74ff9700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c5c066f30 0x7f5c5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.605+0000 7f5c74ff9700 1 -- 172.21.15.45:0/12202057 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8442+0+0 (secure 0 0 0) 0x7f5c64086c60 con 0x7f5c780c9d50 2023-10-06T05:24:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.605+0000 7f5c7c861700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c5c066f30 0x7f5c5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.606+0000 7f5c7c861700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c5c066f30 0x7f5c5c0693e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006fd0 tx=0x7f5c6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.610+0000 7f5c74ff9700 1 -- 172.21.15.45:0/12202057 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c780c6670 con 0x7f5c780c9d50 2023-10-06T05:24:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:44.860+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5c780c6670 con 0x7f5c780c9d50 2023-10-06T05:24:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.144+0000 7f5c74ff9700 1 -- 172.21.15.45:0/12202057 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v240) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5c780c6670 con 0x7f5c780c9d50 2023-10-06T05:24:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c5c066f30 msgr2=0x7f5c5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c5c066f30 0x7f5c5c0693e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006fd0 tx=0x7f5c6c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c9d50 msgr2=0x7f5c780c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c9d50 0x7f5c780c48b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5c64006fd0 tx=0x7f5c64017000 comp rx=0 tx=0).stop 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 shutdown_connections 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c5c066f30 0x7f5c5c0693e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c7813f900 0x7f5c78138920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c780c8de0 0x7f5c780c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 --2- 172.21.15.45:0/12202057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c780c9d50 0x7f5c780c48b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 >> 172.21.15.45:0/12202057 conn(0x7f5c780691c0 msgr2=0x7f5c78139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 shutdown_connections 2023-10-06T05:24:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.147+0000 7f5c7eac5700 1 -- 172.21.15.45:0/12202057 wait complete. 2023-10-06T05:24:45.151 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:45.157 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:24:45.156+0000 7f78dcfc9700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:24:45.158 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:45.157+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:45.158 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:45.157+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:45.167 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:45.168 INFO:tasks.ceph.mds.b.smithi144.stderr: -18> 2023-10-06T05:24:45.156+0000 7f78dcfc9700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:24:45.187 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:24:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.301+0000 7f98ed79a700 1 Processor -- start 2023-10-06T05:24:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.301+0000 7f98ed79a700 1 -- start start 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.302+0000 7f98ed79a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e8153650 0x7f98e815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.302+0000 7f98ed79a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e8153a20 0x7f98e806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.302+0000 7f98ed79a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f98e806add0 0x7f98e8068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.302+0000 7f98ed79a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e806b250 con 0x7f98e8153650 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.302+0000 7f98ed79a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e80676c0 con 0x7f98e806add0 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.302+0000 7f98ed79a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80679a0 con 0x7f98e8153a20 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.302+0000 7f98e6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e8153650 0x7f98e815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e8153650 0x7f98e815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42396/0 (socket says 172.21.15.45:42396) 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 -- 172.21.15.45:0/3543170003 learned_addr learned my addr 172.21.15.45:0/3543170003 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e77fe700 1 --2- 172.21.15.45:0/3543170003 >> v2:172.21.15.144:3300/0 conn(0x7f98e806add0 0x7f98e8068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e67fc700 1 --2- 172.21.15.45:0/3543170003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e8153a20 0x7f98e806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 -- 172.21.15.45:0/3543170003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e8153a20 msgr2=0x7f98e806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 --2- 172.21.15.45:0/3543170003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e8153a20 0x7f98e806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 -- 172.21.15.45:0/3543170003 >> v2:172.21.15.144:3300/0 conn(0x7f98e806add0 msgr2=0x7f98e8068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 --2- 172.21.15.45:0/3543170003 >> v2:172.21.15.144:3300/0 conn(0x7f98e806add0 0x7f98e8068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 -- 172.21.15.45:0/3543170003 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8067c80 con 0x7f98e8153650 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e6ffd700 1 --2- 172.21.15.45:0/3543170003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153650 0x7f98e815a930 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f98e80c8e10 tx=0x7f98d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8242a923c41169 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e5ffb700 1 -- 172.21.15.45:0/3543170003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800b640 con 0x7f98e8153650 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e5ffb700 1 -- 172.21.15.45:0/3543170003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d800b7a0 con 0x7f98e8153650 2023-10-06T05:24:45.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.303+0000 7f98e5ffb700 1 -- 172.21.15.45:0/3543170003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800bb60 con 0x7f98e8153650 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 -- 172.21.15.45:0/3543170003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153650 msgr2=0x7f98e815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 --2- 172.21.15.45:0/3543170003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153650 0x7f98e815a930 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f98e80c8e10 tx=0x7f98d8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 -- 172.21.15.45:0/3543170003 shutdown_connections 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 --2- 172.21.15.45:0/3543170003 >> v2:172.21.15.144:3300/0 conn(0x7f98e806add0 0x7f98e8068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 --2- 172.21.15.45:0/3543170003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153650 0x7f98e815a930 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 --2- 172.21.15.45:0/3543170003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e8153a20 0x7f98e806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 -- 172.21.15.45:0/3543170003 >> 172.21.15.45:0/3543170003 conn(0x7f98e80cc5a0 msgr2=0x7f98e80cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 -- 172.21.15.45:0/3543170003 shutdown_connections 2023-10-06T05:24:45.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.304+0000 7f98ed79a700 1 -- 172.21.15.45:0/3543170003 wait complete. 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.305+0000 7f98ed79a700 1 Processor -- start 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.305+0000 7f98ed79a700 1 -- start start 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98ed79a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e806add0 0x7f98e80b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98ed79a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98e8153650 0x7f98e8156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98ed79a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153a20 0x7f98e816d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98ed79a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80ae110 con 0x7f98e8153a20 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98ed79a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80b0b30 con 0x7f98e8153650 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98ed79a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e8156ca0 con 0x7f98e806add0 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98e77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153a20 0x7f98e816d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98e77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153a20 0x7f98e816d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42404/0 (socket says 172.21.15.45:42404) 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98e77fe700 1 -- 172.21.15.45:0/4118528826 learned_addr learned my addr 172.21.15.45:0/4118528826 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98e67fc700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98e8153650 0x7f98e8156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.306+0000 7f98e6ffd700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e806add0 0x7f98e80b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98e77fe700 1 -- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e806add0 msgr2=0x7f98e80b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98e77fe700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e806add0 0x7f98e80b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98e77fe700 1 -- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98e8153650 msgr2=0x7f98e8156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98e77fe700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98e8153650 0x7f98e8156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98e77fe700 1 -- 172.21.15.45:0/4118528826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80094f0 con 0x7f98e8153a20 2023-10-06T05:24:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98e67fc700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98e8153650 0x7f98e8156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98e77fe700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153a20 0x7f98e816d060 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e90 tx=0x7f98e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98ceffd700 1 -- 172.21.15.45:0/4118528826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e000cb70 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98ceffd700 1 -- 172.21.15.45:0/4118528826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e000f040 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e80b0d60 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98ceffd700 1 -- 172.21.15.45:0/4118528826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0014480 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.307+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e80b1320 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.308+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e80cc0e0 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.309+0000 7f98ceffd700 1 -- 172.21.15.45:0/4118528826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98e0014830 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.309+0000 7f98ceffd700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98d4066f30 0x7f98d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.309+0000 7f98ceffd700 1 -- 172.21.15.45:0/4118528826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f98e0085db0 con 0x7f98e8153a20 2023-10-06T05:24:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.309+0000 7f98e6ffd700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98d4066f30 0x7f98d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.310+0000 7f98e6ffd700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98d4066f30 0x7f98d40693e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f98e80c8e10 tx=0x7f98d8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.314+0000 7f98ceffd700 1 -- 172.21.15.45:0/4118528826 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98e008a070 con 0x7f98e8153a20 2023-10-06T05:24:45.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.563+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e8067f60 con 0x7f98e8153a20 2023-10-06T05:24:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.563+0000 7f98ceffd700 1 -- 172.21.15.45:0/4118528826 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 242 v242) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f98e00524a0 con 0x7f98e8153a20 2023-10-06T05:24:45.565 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:45.565 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":242,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":242,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:45.158084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:45.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.565+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98d4066f30 msgr2=0x7f98d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.565+0000 7f98ed79a700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98d4066f30 0x7f98d40693e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f98e80c8e10 tx=0x7f98d8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.565+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153a20 msgr2=0x7f98e816d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.565+0000 7f98ed79a700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153a20 0x7f98e816d060 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009e90 tx=0x7f98e000b040 comp rx=0 tx=0).stop 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 shutdown_connections 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98d4066f30 0x7f98d40693e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e8153a20 0x7f98e816d060 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e806add0 0x7f98e80b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 --2- 172.21.15.45:0/4118528826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98e8153650 0x7f98e8156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 >> 172.21.15.45:0/4118528826 conn(0x7f98e80cc5a0 msgr2=0x7f98e815e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 shutdown_connections 2023-10-06T05:24:45.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.566+0000 7f98ed79a700 1 -- 172.21.15.45:0/4118528826 wait complete. 2023-10-06T05:24:45.569 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 242 2023-10-06T05:24:45.580 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:45.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:45.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 Processor -- start 2023-10-06T05:24:45.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 -- start start 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b500b1060 0x7f9b50069f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1430 0x7f9b5006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9b50140280 0x7f9b501406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5006b1f0 con 0x7f9b50140280 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5006f270 con 0x7f9b500b1060 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.711+0000 7f9b55660700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5006a5e0 con 0x7f9b500b1430 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9b50140280 0x7f9b501406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9b50140280 0x7f9b501406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42410/0 (socket says 172.21.15.45:42410) 2023-10-06T05:24:45.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/847871954 learned_addr learned my addr 172.21.15.45:0/847871954 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b467fc700 1 --2- 172.21.15.45:0/847871954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1430 0x7f9b5006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4effd700 1 --2- 172.21.15.45:0/847871954 >> v2:172.21.15.144:3300/0 conn(0x7f9b500b1060 0x7f9b50069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/847871954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1430 msgr2=0x7f9b5006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 --2- 172.21.15.45:0/847871954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1430 0x7f9b5006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/847871954 >> v2:172.21.15.144:3300/0 conn(0x7f9b500b1060 msgr2=0x7f9b50069f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 --2- 172.21.15.45:0/847871954 >> v2:172.21.15.144:3300/0 conn(0x7f9b500b1060 0x7f9b50069f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/847871954 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5006a8c0 con 0x7f9b50140280 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b467fc700 1 --2- 172.21.15.45:0/847871954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1430 0x7f9b5006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4f7fe700 1 --2- 172.21.15.45:0/847871954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b501406b0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9b5006d820 tx=0x7f9b48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4684150378f02c71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4e7fc700 1 -- 172.21.15.45:0/847871954 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4800e070 con 0x7f9b50140280 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4e7fc700 1 -- 172.21.15.45:0/847871954 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4800b970 con 0x7f9b50140280 2023-10-06T05:24:45.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.712+0000 7f9b4e7fc700 1 -- 172.21.15.45:0/847871954 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480092e0 con 0x7f9b50140280 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 -- 172.21.15.45:0/847871954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 msgr2=0x7f9b501406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 --2- 172.21.15.45:0/847871954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b501406b0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9b5006d820 tx=0x7f9b48009840 comp rx=0 tx=0).stop 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 -- 172.21.15.45:0/847871954 shutdown_connections 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 --2- 172.21.15.45:0/847871954 >> v2:172.21.15.144:3300/0 conn(0x7f9b500b1060 0x7f9b50069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 --2- 172.21.15.45:0/847871954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b501406b0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 --2- 172.21.15.45:0/847871954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1430 0x7f9b5006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 -- 172.21.15.45:0/847871954 >> 172.21.15.45:0/847871954 conn(0x7f9b50067620 msgr2=0x7f9b50067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 -- 172.21.15.45:0/847871954 shutdown_connections 2023-10-06T05:24:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.713+0000 7f9b55660700 1 -- 172.21.15.45:0/847871954 wait complete. 2023-10-06T05:24:45.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.715+0000 7f9b55660700 1 Processor -- start 2023-10-06T05:24:45.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.715+0000 7f9b55660700 1 -- start start 2023-10-06T05:24:45.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b55660700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1060 0x7f9b5013b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b55660700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500b1430 0x7f9b5013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b55660700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b5015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b55660700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5013dc30 con 0x7f9b50140280 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b55660700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5013e0c0 con 0x7f9b500b1430 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b55660700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5013d020 con 0x7f9b500b1060 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b5015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b4f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b5015b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42422/0 (socket says 172.21.15.45:42422) 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/3635829696 learned_addr learned my addr 172.21.15.45:0/3635829696 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b4effd700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1060 0x7f9b5013b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b467fc700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500b1430 0x7f9b5013ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.716+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1060 msgr2=0x7f9b5013b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b4f7fe700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1060 0x7f9b5013b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500b1430 msgr2=0x7f9b5013ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b4f7fe700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500b1430 0x7f9b5013ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b4f7fe700 1 -- 172.21.15.45:0/3635829696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38009420 con 0x7f9b50140280 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b467fc700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500b1430 0x7f9b5013ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b4f7fe700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b5015b910 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9b5006d850 tx=0x7f9b48018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b477fe700 1 -- 172.21.15.45:0/3635829696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48008b80 con 0x7f9b50140280 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b477fe700 1 -- 172.21.15.45:0/3635829696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48008d20 con 0x7f9b50140280 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b48009ae0 con 0x7f9b50140280 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b477fe700 1 -- 172.21.15.45:0/3635829696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480256b0 con 0x7f9b50140280 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.717+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5013d5e0 con 0x7f9b50140280 2023-10-06T05:24:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.718+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5006cec0 con 0x7f9b50140280 2023-10-06T05:24:45.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.718+0000 7f9b477fe700 1 -- 172.21.15.45:0/3635829696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b48003040 con 0x7f9b50140280 2023-10-06T05:24:45.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.719+0000 7f9b477fe700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b30066f30 0x7f9b300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.719+0000 7f9b477fe700 1 -- 172.21.15.45:0/3635829696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f9b48086860 con 0x7f9b50140280 2023-10-06T05:24:45.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.719+0000 7f9b4effd700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b30066f30 0x7f9b300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.720+0000 7f9b4effd700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b30066f30 0x7f9b300693e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9b500ccc80 tx=0x7f9b400091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.723+0000 7f9b477fe700 1 -- 172.21.15.45:0/3635829696 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b48052f90 con 0x7f9b50140280 2023-10-06T05:24:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.967+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b5006aba0 con 0x7f9b50140280 2023-10-06T05:24:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.968+0000 7f9b477fe700 1 -- 172.21.15.45:0/3635829696 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 242 v242) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f9b480565b0 con 0x7f9b50140280 2023-10-06T05:24:45.970 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:45.970 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":242,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":242,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:45.158084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b30066f30 msgr2=0x7f9b300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b30066f30 0x7f9b300693e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9b500ccc80 tx=0x7f9b400091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 msgr2=0x7f9b5015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b5015b910 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9b5006d850 tx=0x7f9b48018000 comp rx=0 tx=0).stop 2023-10-06T05:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 shutdown_connections 2023-10-06T05:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9b30066f30 0x7f9b300693e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b50140280 0x7f9b5015b910 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500b1060 0x7f9b5013b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 --2- 172.21.15.45:0/3635829696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500b1430 0x7f9b5013ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 >> 172.21.15.45:0/3635829696 conn(0x7f9b50067620 msgr2=0x7f9b500c7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 shutdown_connections 2023-10-06T05:24:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:45.970+0000 7f9b55660700 1 -- 172.21.15.45:0/3635829696 wait complete. 2023-10-06T05:24:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 242 2023-10-06T05:24:45.985 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:45.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:46.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.116+0000 7f698ff50700 1 Processor -- start 2023-10-06T05:24:46.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.116+0000 7f698ff50700 1 -- start start 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698ff50700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698ff50700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69880cd300 0x7f6988161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698ff50700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69880cd770 con 0x7f6988161a50 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698ff50700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698806b080 con 0x7f69880cd300 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698ff50700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698806b2d0 con 0x7f69880c0170 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698d4eb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69880cd300 0x7f6988161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698d4eb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69880cd300 0x7f6988161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42432/0 (socket says 172.21.15.45:42432) 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698d4eb700 1 -- 172.21.15.45:0/1278947211 learned_addr learned my addr 172.21.15.45:0/1278947211 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698dcec700 1 --2- 172.21.15.45:0/1278947211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.117+0000 7f698d4eb700 1 -- 172.21.15.45:0/1278947211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 msgr2=0x7f69880ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698d4eb700 1 --2- 172.21.15.45:0/1278947211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698d4eb700 1 -- 172.21.15.45:0/1278947211 >> v1:172.21.15.144:6789/0 conn(0x7f6988161a50 legacy=0x7f69880c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698d4eb700 1 -- 172.21.15.45:0/1278947211 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698806b5b0 con 0x7f69880cd300 2023-10-06T05:24:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698d4eb700 1 --2- 172.21.15.45:0/1278947211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69880cd300 0x7f6988161510 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f697c0099b0 tx=0x7f697c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c29aba9f70f10a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698ccea700 1 -- 172.21.15.45:0/1278947211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c00b820 con 0x7f69880cd300 2023-10-06T05:24:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698ccea700 1 -- 172.21.15.45:0/1278947211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c00b980 con 0x7f69880cd300 2023-10-06T05:24:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698ccea700 1 -- 172.21.15.45:0/1278947211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c00bd20 con 0x7f69880cd300 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698ff50700 1 -- 172.21.15.45:0/1278947211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69880cd300 msgr2=0x7f6988161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.118+0000 7f698ff50700 1 --2- 172.21.15.45:0/1278947211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69880cd300 0x7f6988161510 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f697c0099b0 tx=0x7f697c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.119+0000 7f698ff50700 1 -- 172.21.15.45:0/1278947211 shutdown_connections 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.119+0000 7f698ff50700 1 --2- 172.21.15.45:0/1278947211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.119+0000 7f698ff50700 1 --2- 172.21.15.45:0/1278947211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69880cd300 0x7f6988161510 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.119+0000 7f698ff50700 1 -- 172.21.15.45:0/1278947211 >> 172.21.15.45:0/1278947211 conn(0x7f698806f090 msgr2=0x7f698806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.119+0000 7f698ff50700 1 -- 172.21.15.45:0/1278947211 shutdown_connections 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.119+0000 7f698ff50700 1 -- 172.21.15.45:0/1278947211 wait complete. 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.120+0000 7f698ff50700 1 Processor -- start 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.120+0000 7f698ff50700 1 -- start start 2023-10-06T05:24:46.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698ff50700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698ff50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69880cd300 0x7f69880c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698ff50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6988161a50 0x7f69880c1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698ff50700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880afbe0 con 0x7f6988161a50 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698ff50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b0760 con 0x7f69880cd300 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698ff50700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880afe30 con 0x7f69880c0170 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6988161a50 0x7f69880c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6988161a50 0x7f69880c1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42440/0 (socket says 172.21.15.45:42440) 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 -- 172.21.15.45:0/1942317814 learned_addr learned my addr 172.21.15.45:0/1942317814 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698dcec700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698d4eb700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69880cd300 0x7f69880c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 -- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 msgr2=0x7f69880c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 -- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69880cd300 msgr2=0x7f69880c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69880cd300 0x7f69880c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 -- 172.21.15.45:0/1942317814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0094f0 con 0x7f6988161a50 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f698e4ed700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6988161a50 0x7f69880c1100 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f698000bc30 tx=0x7f6980009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.121+0000 7f6979ffb700 1 -- 172.21.15.45:0/1942317814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000c840 con 0x7f6988161a50 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.122+0000 7f6979ffb700 1 -- 172.21.15.45:0/1942317814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698000c9e0 con 0x7f6988161a50 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.122+0000 7f6979ffb700 1 -- 172.21.15.45:0/1942317814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000cda0 con 0x7f6988161a50 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.122+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69880b0110 con 0x7f6988161a50 2023-10-06T05:24:46.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.122+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69880b21a0 con 0x7f6988161a50 2023-10-06T05:24:46.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.123+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69880b23c0 con 0x7f6988161a50 2023-10-06T05:24:46.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.123+0000 7f6979ffb700 1 -- 172.21.15.45:0/1942317814 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6980014620 con 0x7f6988161a50 2023-10-06T05:24:46.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.123+0000 7f6979ffb700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f696c066f30 0x7f696c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.124+0000 7f6979ffb700 1 -- 172.21.15.45:0/1942317814 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6980084fe0 con 0x7f6988161a50 2023-10-06T05:24:46.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.124+0000 7f698dcec700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f696c066f30 0x7f696c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.124+0000 7f698dcec700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f696c066f30 0x7f696c0693e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f698400acb0 tx=0x7f6984008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.128+0000 7f6979ffb700 1 -- 172.21.15.45:0/1942317814 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f698004c5a0 con 0x7f6988161a50 2023-10-06T05:24:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.380+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69880c7e20 con 0x7f6988161a50 2023-10-06T05:24:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.381+0000 7f6979ffb700 1 -- 172.21.15.45:0/1942317814 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 243 v243) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f69880b23c0 con 0x7f6988161a50 2023-10-06T05:24:46.383 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:46.383 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":243,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":9064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1309688991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1309688991},{"type":"v1","addr":"172.21.15.144:6835","nonce":1309688991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":243}],"filesystems":[{"mdsmap":{"epoch":242,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:45.158084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f696c066f30 msgr2=0x7f696c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f696c066f30 0x7f696c0693e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f698400acb0 tx=0x7f6984008040 comp rx=0 tx=0).stop 2023-10-06T05:24:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6988161a50 msgr2=0x7f69880c1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6988161a50 0x7f69880c1100 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f698000bc30 tx=0x7f6980009820 comp rx=0 tx=0).stop 2023-10-06T05:24:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 shutdown_connections 2023-10-06T05:24:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f696c066f30 0x7f696c0693e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6988161a50 0x7f69880c1100 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69880c0170 0x7f69880c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 --2- 172.21.15.45:0/1942317814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69880cd300 0x7f69880c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 >> 172.21.15.45:0/1942317814 conn(0x7f698806f090 msgr2=0x7f69880af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 shutdown_connections 2023-10-06T05:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.383+0000 7f698ff50700 1 -- 172.21.15.45:0/1942317814 wait complete. 2023-10-06T05:24:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 243 2023-10-06T05:24:46.398 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2023-10-06T05:24:46.398 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2023-10-06T05:24:46.398 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:46.398 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:46.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:46.399 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:46.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:46.418 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:46.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.594+0000 7fa4a0eb1700 1 Processor -- start 2023-10-06T05:24:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.594+0000 7fa4a0eb1700 1 -- start start 2023-10-06T05:24:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa4a0eb1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa49c15a490 0x7fa49c067830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa4a0eb1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c0b2120 con 0x7fa49c15b5e0 2023-10-06T05:24:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa4a0eb1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c0ab7d0 con 0x7fa49c06afa0 2023-10-06T05:24:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa4a0eb1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c0ae020 con 0x7fa49c15a490 2023-10-06T05:24:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa49a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa49c15a490 0x7fa49c067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa499d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa49c15b5e0 0x7fa49c06a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47484/0 (socket says 172.21.15.45:47484) 2023-10-06T05:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa499d9b700 1 -- 172.21.15.45:0/2818505165 learned_addr learned my addr 172.21.15.45:0/2818505165 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa49a59c700 1 -- 172.21.15.45:0/2818505165 >> v1:172.21.15.144:6789/0 conn(0x7fa49c06afa0 legacy=0x7fa49c0abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa49a59c700 1 -- 172.21.15.45:0/2818505165 >> v1:172.21.15.45:6789/0 conn(0x7fa49c15b5e0 legacy=0x7fa49c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa49a59c700 1 -- 172.21.15.45:0/2818505165 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c067e80 con 0x7fa49c15a490 2023-10-06T05:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.595+0000 7fa49959a700 1 -- 172.21.15.45:0/2818505165 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184511261 0 0) 0x7fa490001110 con 0x7fa49c15b5e0 2023-10-06T05:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa49a59c700 1 --2- 172.21.15.45:0/2818505165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15a490 0x7fa49c067830 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0098e0 tx=0x7fa48c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ed67bcd4b6bd3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa49959a700 1 -- 172.21.15.45:0/2818505165 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c009e80 con 0x7fa49c15a490 2023-10-06T05:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa49959a700 1 -- 172.21.15.45:0/2818505165 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c00b860 con 0x7fa49c15a490 2023-10-06T05:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa49959a700 1 -- 172.21.15.45:0/2818505165 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00bbe0 con 0x7fa49c15a490 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/2818505165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15a490 msgr2=0x7fa49c067830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/2818505165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15a490 0x7fa49c067830 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0098e0 tx=0x7fa48c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/2818505165 shutdown_connections 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/2818505165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15a490 0x7fa49c067830 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/2818505165 >> 172.21.15.45:0/2818505165 conn(0x7fa49c0cc5a0 msgr2=0x7fa49c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/2818505165 shutdown_connections 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.596+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/2818505165 wait complete. 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 Processor -- start 2023-10-06T05:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 -- start start 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa49c06afa0 0x7fa49c150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa49c15a490 0x7fa49c0c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15b5e0 0x7fa49c0aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0c45a0 con 0x7fa49c15b5e0 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c150fc0 con 0x7fa49c06afa0 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.598+0000 7fa4a0eb1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0c3990 con 0x7fa49c15a490 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15b5e0 0x7fa49c0aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15b5e0 0x7fa49c0aa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34318/0 (socket says 172.21.15.45:34318) 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 -- 172.21.15.45:0/4239369936 learned_addr learned my addr 172.21.15.45:0/4239369936 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49a59c700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa49c06afa0 0x7fa49c150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa499d9b700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa49c15a490 0x7fa49c0c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 -- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa49c15a490 msgr2=0x7fa49c0c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa49c15a490 0x7fa49c0c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 -- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa49c06afa0 msgr2=0x7fa49c150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa49c06afa0 0x7fa49c150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 -- 172.21.15.45:0/4239369936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c009420 con 0x7fa49c15b5e0 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa49ad9d700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15b5e0 0x7fa49c0aa8d0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c940 tx=0x7fa4940016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa48a7fc700 1 -- 172.21.15.45:0/4239369936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494012630 con 0x7fa49c15b5e0 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa48a7fc700 1 -- 172.21.15.45:0/4239369936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494012790 con 0x7fa49c15b5e0 2023-10-06T05:24:46.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c0c3c70 con 0x7fa49c15b5e0 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa48a7fc700 1 -- 172.21.15.45:0/4239369936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494012b50 con 0x7fa49c15b5e0 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.599+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c0c4230 con 0x7fa49c15b5e0 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.600+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c0c4750 con 0x7fa49c15b5e0 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.601+0000 7fa48a7fc700 1 -- 172.21.15.45:0/4239369936 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa49401b450 con 0x7fa49c15b5e0 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.601+0000 7fa48a7fc700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.601+0000 7fa48a7fc700 1 -- 172.21.15.45:0/4239369936 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fa494085130 con 0x7fa49c15b5e0 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.602+0000 7fa49a59c700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.602+0000 7fa49a59c700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa47c066f30 0x7fa47c0693e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0093f0 tx=0x7fa48c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.606+0000 7fa48a7fc700 1 -- 172.21.15.45:0/4239369936 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa49c0c4750 con 0x7fa49c15b5e0 2023-10-06T05:24:46.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:46.853+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa49c0c4750 con 0x7fa49c15b5e0 2023-10-06T05:24:47.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.167+0000 7fa48a7fc700 1 -- 172.21.15.45:0/4239369936 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v244) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa49c0c4750 con 0x7fa49c15b5e0 2023-10-06T05:24:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa47c066f30 msgr2=0x7fa47c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa47c066f30 0x7fa47c0693e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0093f0 tx=0x7fa48c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15b5e0 msgr2=0x7fa49c0aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15b5e0 0x7fa49c0aa8d0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c940 tx=0x7fa4940016f0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 shutdown_connections 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa49c15b5e0 0x7fa49c0aa8d0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa49c06afa0 0x7fa49c150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 --2- 172.21.15.45:0/4239369936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa49c15a490 0x7fa49c0c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 >> 172.21.15.45:0/4239369936 conn(0x7fa49c0cc5a0 msgr2=0x7fa49c0cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 shutdown_connections 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.170+0000 7fa4a0eb1700 1 -- 172.21.15.45:0/4239369936 wait complete. 2023-10-06T05:24:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:47.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:47.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.322+0000 7f4b27bac700 1 Processor -- start 2023-10-06T05:24:47.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.323+0000 7f4b27bac700 1 -- start start 2023-10-06T05:24:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.323+0000 7f4b27bac700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153650 0x7f4b2015a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.323+0000 7f4b27bac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b2006afa0 0x7f4b20068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.323+0000 7f4b27bac700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b200676c0 con 0x7f4b20153a20 2023-10-06T05:24:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.323+0000 7f4b27bac700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200679a0 con 0x7f4b2006afa0 2023-10-06T05:24:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.323+0000 7f4b27bac700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20067c80 con 0x7f4b20153650 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b2006afa0 0x7f4b20068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b2006afa0 0x7f4b20068480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34320/0 (socket says 172.21.15.45:34320) 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 -- 172.21.15.45:0/2705586438 learned_addr learned my addr 172.21.15.45:0/2705586438 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b25948700 1 --2- 172.21.15.45:0/2705586438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153650 0x7f4b2015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 -- 172.21.15.45:0/2705586438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153650 msgr2=0x7f4b2015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 --2- 172.21.15.45:0/2705586438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153650 0x7f4b2015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 -- 172.21.15.45:0/2705586438 >> v1:172.21.15.144:6789/0 conn(0x7f4b20153a20 legacy=0x7f4b2006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 -- 172.21.15.45:0/2705586438 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20067f60 con 0x7f4b2006afa0 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b26149700 1 --2- 172.21.15.45:0/2705586438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2006afa0 0x7f4b20068480 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009f20 tx=0x7f4b18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f920245b75f90ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b24946700 1 -- 172.21.15.45:0/2705586438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800e070 con 0x7f4b2006afa0 2023-10-06T05:24:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b24946700 1 -- 172.21.15.45:0/2705586438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1800b9d0 con 0x7f4b2006afa0 2023-10-06T05:24:47.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.324+0000 7f4b24946700 1 -- 172.21.15.45:0/2705586438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180092e0 con 0x7f4b2006afa0 2023-10-06T05:24:47.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 -- 172.21.15.45:0/2705586438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2006afa0 msgr2=0x7f4b20068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 --2- 172.21.15.45:0/2705586438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2006afa0 0x7f4b20068480 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009f20 tx=0x7f4b18009840 comp rx=0 tx=0).stop 2023-10-06T05:24:47.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 -- 172.21.15.45:0/2705586438 shutdown_connections 2023-10-06T05:24:47.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 --2- 172.21.15.45:0/2705586438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2006afa0 0x7f4b20068480 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 --2- 172.21.15.45:0/2705586438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153650 0x7f4b2015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 -- 172.21.15.45:0/2705586438 >> 172.21.15.45:0/2705586438 conn(0x7f4b200cc5a0 msgr2=0x7f4b200cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 -- 172.21.15.45:0/2705586438 shutdown_connections 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.325+0000 7f4b27bac700 1 -- 172.21.15.45:0/2705586438 wait complete. 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.326+0000 7f4b27bac700 1 Processor -- start 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.326+0000 7f4b27bac700 1 -- start start 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b27bac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2006afa0 0x7f4b20151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b27bac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b20153650 0x7f4b200c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b27bac700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153a20 0x7f4b2015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b27bac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b15e0 con 0x7f4b20153650 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b27bac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b0710 con 0x7f4b2006afa0 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b27bac700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b09d0 con 0x7f4b20153a20 2023-10-06T05:24:47.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b25147700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b20153650 0x7f4b200c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b25147700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b20153650 0x7f4b200c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34322/0 (socket says 172.21.15.45:34322) 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b25147700 1 -- 172.21.15.45:0/357899010 learned_addr learned my addr 172.21.15.45:0/357899010 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b25948700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2006afa0 0x7f4b20151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b26149700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153a20 0x7f4b2015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.327+0000 7f4b25147700 1 -- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153a20 msgr2=0x7f4b2015b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b25147700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153a20 0x7f4b2015b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b25147700 1 -- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2006afa0 msgr2=0x7f4b20151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b25147700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2006afa0 0x7f4b20151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b25147700 1 -- 172.21.15.45:0/357899010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18009ae0 con 0x7f4b20153650 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b25948700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2006afa0 0x7f4b20151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b26149700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153a20 0x7f4b2015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b25147700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b20153650 0x7f4b200c19c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c4e0 tx=0x7f4b14007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b11ffb700 1 -- 172.21.15.45:0/357899010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011590 con 0x7f4b20153650 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b11ffb700 1 -- 172.21.15.45:0/357899010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b140116f0 con 0x7f4b20153650 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b200b0cb0 con 0x7f4b20153650 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b11ffb700 1 -- 172.21.15.45:0/357899010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011ab0 con 0x7f4b20153650 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.328+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b200b1270 con 0x7f4b20153650 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.329+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b200b1750 con 0x7f4b20153650 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.330+0000 7f4b11ffb700 1 -- 172.21.15.45:0/357899010 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b1401b3b0 con 0x7f4b20153650 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.330+0000 7f4b11ffb700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b04066f30 0x7f4b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.330+0000 7f4b11ffb700 1 -- 172.21.15.45:0/357899010 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f4b14084560 con 0x7f4b20153650 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.330+0000 7f4b25948700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b04066f30 0x7f4b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.331+0000 7f4b25948700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b04066f30 0x7f4b040693e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009d80 tx=0x7f4b1c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.334+0000 7f4b11ffb700 1 -- 172.21.15.45:0/357899010 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b14050c90 con 0x7f4b20153650 2023-10-06T05:24:47.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:47.620+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f4b200cc0e0 con 0x7f4b20153650 2023-10-06T05:24:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.169+0000 7f4b11ffb700 1 -- 172.21.15.45:0/357899010 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v245) v1 ==== 101+0+0 (secure 0 0 0) 0x7f4b14017070 con 0x7f4b20153650 2023-10-06T05:24:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b04066f30 msgr2=0x7f4b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b04066f30 0x7f4b040693e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009d80 tx=0x7f4b1c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b20153650 msgr2=0x7f4b200c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b20153650 0x7f4b200c19c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c4e0 tx=0x7f4b14007f00 comp rx=0 tx=0).stop 2023-10-06T05:24:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 shutdown_connections 2023-10-06T05:24:48.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b04066f30 0x7f4b040693e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b20153a20 0x7f4b2015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2006afa0 0x7f4b20151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 --2- 172.21.15.45:0/357899010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b20153650 0x7f4b200c19c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 >> 172.21.15.45:0/357899010 conn(0x7f4b200cc5a0 msgr2=0x7f4b200cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 shutdown_connections 2023-10-06T05:24:48.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.172+0000 7f4b27bac700 1 -- 172.21.15.45:0/357899010 wait complete. 2023-10-06T05:24:48.178 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:48.191 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.238+0000 7f5b7a68e040 1 Processor -- start 2023-10-06T05:24:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.238+0000 7f5b7a68e040 1 -- start start 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b7a68e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b85c760 0x55a79b859230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b7a68e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 0x55a79b85e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b7a68e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b85e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b7a68e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a79b6d9980 con 0x55a79b85c760 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b7a68e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a79b6d9620 con 0x55a79b85ba30 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b7a68e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a79b6d0400 con 0x55a79b7bcc40 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b85c760 0x55a79b859230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b85c760 0x55a79b859230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34342/0 (socket says 172.21.15.45:34342) 2023-10-06T05:24:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 -- 172.21.15.45:0/3437677031 learned_addr learned my addr 172.21.15.45:0/3437677031 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b730ab700 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 0x55a79b85e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b740ad700 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b85e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 -- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 msgr2=0x55a79b85e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b85e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 -- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 msgr2=0x55a79b85e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 0x55a79b85e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 -- 172.21.15.45:0/3437677031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a79b6e83d0 con 0x55a79b85c760 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b740ad700 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b85e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b738ac700 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b85c760 0x55a79b859230 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f5b640083c0 tx=0x7f5b64018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c86b0a50b55da62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b720a9700 1 -- 172.21.15.45:0/3437677031 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400e040 con 0x55a79b85c760 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b720a9700 1 -- 172.21.15.45:0/3437677031 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640054f0 con 0x55a79b85c760 2023-10-06T05:24:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.239+0000 7f5b720a9700 1 -- 172.21.15.45:0/3437677031 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64000bc0 con 0x55a79b85c760 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 -- 172.21.15.45:0/3437677031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b85c760 msgr2=0x55a79b859230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b85c760 0x55a79b859230 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f5b640083c0 tx=0x7f5b64018000 comp rx=0 tx=0).stop 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 -- 172.21.15.45:0/3437677031 shutdown_connections 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b85e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b85c760 0x55a79b859230 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/3437677031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 0x55a79b85e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 -- 172.21.15.45:0/3437677031 >> 172.21.15.45:0/3437677031 conn(0x55a79b78f450 msgr2=0x55a79b7918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 -- 172.21.15.45:0/3437677031 shutdown_connections 2023-10-06T05:24:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.240+0000 7f5b7a68e040 1 -- 172.21.15.45:0/3437677031 wait complete. 2023-10-06T05:24:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 Processor -- start 2023-10-06T05:24:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 -- start start 2023-10-06T05:24:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b857ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 0x55a79b7b9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b876d30 0x55a79b863450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a79b7b9660 con 0x55a79b876d30 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a79b7bd7d0 con 0x55a79b85ba30 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.241+0000 7f5b7a68e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a79b85a7e0 con 0x55a79b7bcc40 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b876d30 0x55a79b863450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b876d30 0x55a79b863450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34354/0 (socket says 172.21.15.45:34354) 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 learned_addr learned my addr 172.21.15.45:0/2217804540 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b730ab700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 0x55a79b7b9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b738ac700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b857ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 msgr2=0x55a79b857ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a79b7bcc40 0x55a79b857ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 msgr2=0x55a79b7b9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a79b85ba30 0x55a79b7b9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c001e10 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b740ad700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b876d30 0x55a79b863450 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c005b20 tx=0x7f5b6c007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b708a6700 1 -- 172.21.15.45:0/2217804540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c014630 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b708a6700 1 -- 172.21.15.45:0/2217804540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c014790 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a79b6dbae0 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b708a6700 1 -- 172.21.15.45:0/2217804540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c014b30 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.242+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a79b6db540 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.244+0000 7f5b708a6700 1 -- 172.21.15.45:0/2217804540 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b6c003040 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.244+0000 7f5b708a6700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b600658c0 0x7f5b60067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.244+0000 7f5b708a6700 1 -- 172.21.15.45:0/2217804540 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f5b6c084390 con 0x55a79b876d30 2023-10-06T05:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.245+0000 7f5b738ac700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b600658c0 0x7f5b60067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.249 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.245+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55a79b863990 0x55a79b7ba590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.245+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b7baad0 con 0x55a79b863990 2023-10-06T05:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.245+0000 7f5b730ab700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55a79b863990 0x55a79b7ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.245+0000 7f5b738ac700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b600658c0 0x7f5b60067d70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x55a79b7be740 tx=0x7f5b6401e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.245+0000 7f5b730ab700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55a79b863990 0x55a79b7ba590 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.247+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v117'753 uv727 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.247+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8828c0 con 0x55a79b863990 2023-10-06T05:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.249+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v117'754 uv754 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.249+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b882c70 con 0x55a79b863990 2023-10-06T05:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.250+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v117'755 uv755 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.252 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.250+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884030 con 0x55a79b863990 2023-10-06T05:24:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.251+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'756 uv756 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.251+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8843e0 con 0x55a79b863990 2023-10-06T05:24:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.253+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'757 uv757 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.253+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b883510 con 0x55a79b863990 2023-10-06T05:24:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.254+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'758 uv758 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.254+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b883910 con 0x55a79b863990 2023-10-06T05:24:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.255+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v117'759 uv733 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.255+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.256+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v117'760 uv760 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.256+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.258+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v117'761 uv761 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.259 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.258+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.259+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'762 uv762 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.259+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.260+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'763 uv763 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.260+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.261+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'764 uv764 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.261+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.262+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v117'765 uv740 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.262+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.264+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'766 uv766 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.264+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.265+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v117'767 uv767 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.267 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:48.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.265+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.266+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'768 uv768 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.266+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.267+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'769 uv769 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.267+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.269+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'770 uv770 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.269+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.270+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'771 uv771 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.270+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.271+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v117'772 uv746 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.271+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.272+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v117'773 uv773 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.272+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.274+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v117'774 uv774 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.275 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.274+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.275+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'775 uv775 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.275+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.276+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'776 uv776 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.276+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.277+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'777 uv777 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.277+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.279+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v117'778 uv752 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.279+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.280+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v117'779 uv779 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.280+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.281+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v117'780 uv780 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.283 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.281+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.282+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'781 uv781 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.282+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8857a0 con 0x55a79b863990 2023-10-06T05:24:48.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.284+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'782 uv782 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.284+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b884790 con 0x55a79b863990 2023-10-06T05:24:48.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.285+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'783 uv783 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b863990 2023-10-06T05:24:48.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.285+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55a79b884ca0 0x55a79b887080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.285+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8875c0 con 0x55a79b884ca0 2023-10-06T05:24:48.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.285+0000 7f5b738ac700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55a79b884ca0 0x55a79b887080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.285+0000 7f5b738ac700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55a79b884ca0 0x55a79b887080 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.287+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'544 uv526 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.287+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b887970 con 0x55a79b884ca0 2023-10-06T05:24:48.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.289+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'545 uv545 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.289+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b887d20 con 0x55a79b884ca0 2023-10-06T05:24:48.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.290+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'546 uv546 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.292 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:48.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.290+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8885c0 con 0x55a79b884ca0 2023-10-06T05:24:48.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.292+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v117'547 uv547 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.292+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b888970 con 0x55a79b884ca0 2023-10-06T05:24:48.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.293+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'548 uv548 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.293+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b888d20 con 0x55a79b884ca0 2023-10-06T05:24:48.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.294+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v117'549 uv543 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.294+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.295+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v117'550 uv550 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.295+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.297+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v117'551 uv551 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.298 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:48.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.297+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.298+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'552 uv552 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.298+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.300+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'553 uv553 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.300+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.301+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v117'554 uv536 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.301+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.302+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v117'555 uv555 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.302+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.303+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v117'556 uv556 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.305 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:48.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.303+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88c7f0 con 0x55a79b884ca0 2023-10-06T05:24:48.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.305+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v117'557 uv557 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.305+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88c7f0 con 0x55a79b884ca0 2023-10-06T05:24:48.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.306+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'558 uv558 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.306+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.307+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v117'559 uv542 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.307+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.308+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v117'560 uv560 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.308+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.311 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:48.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.309+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v117'561 uv561 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.309+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88a0e0 con 0x55a79b884ca0 2023-10-06T05:24:48.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.311+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'562 uv562 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.311+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88a0e0 con 0x55a79b884ca0 2023-10-06T05:24:48.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.312+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'563 uv563 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.312+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8890d0 con 0x55a79b884ca0 2023-10-06T05:24:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.313+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'564 uv564 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.313+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a79b8895e0 0x55a79b88b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.313+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.314+0000 7f5b730ab700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a79b8895e0 0x55a79b88b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.314+0000 7f5b730ab700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a79b8895e0 0x55a79b88b9c0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.315+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.315+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.316+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v117'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.316+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.317+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v117'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.319 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:48.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.317+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88cf10 con 0x55a79b8895e0 2023-10-06T05:24:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.319+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.319+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88cf10 con 0x55a79b8895e0 2023-10-06T05:24:48.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.320+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.320+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.321+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.321+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.322+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'104 uv103 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.322+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.323+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'105 uv105 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.323+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.324+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'106 uv106 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.326 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:48.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.324+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b8895e0 2023-10-06T05:24:48.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.325+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'107 uv107 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.326+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b884ca0 2023-10-06T05:24:48.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.327+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.327+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b884ca0 2023-10-06T05:24:48.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.328+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v117'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.328+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b884ca0 2023-10-06T05:24:48.331 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:48.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.329+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v117'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.329+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b884ca0 2023-10-06T05:24:48.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.331+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.331+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b884ca0 2023-10-06T05:24:48.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.332+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.332+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88bf00 con 0x55a79b884ca0 2023-10-06T05:24:48.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.333+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b884ca0 2023-10-06T05:24:48.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.333+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55a79b88c900 0x55a79b88ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.333+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88f220 con 0x55a79b88c900 2023-10-06T05:24:48.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.333+0000 7f5b740ad700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55a79b88c900 0x55a79b88ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.334+0000 7f5b740ad700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55a79b88c900 0x55a79b88ece0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.335+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v117'271 uv265 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.335+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88f5d0 con 0x55a79b88c900 2023-10-06T05:24:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.336+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v117'272 uv272 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.336+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b88fa80 con 0x55a79b88c900 2023-10-06T05:24:48.339 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:48.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.337+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v117'273 uv273 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.337+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b891380 con 0x55a79b88c900 2023-10-06T05:24:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.338+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'274 uv274 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.339+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b891730 con 0x55a79b88c900 2023-10-06T05:24:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.340+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'275 uv275 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.340+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b890370 con 0x55a79b88c900 2023-10-06T05:24:48.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.341+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'276 uv276 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.341+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b890790 con 0x55a79b88c900 2023-10-06T05:24:48.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.342+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v117'277 uv270 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.342+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b890b90 con 0x55a79b88c900 2023-10-06T05:24:48.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.343+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v117'278 uv278 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.343+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b890b90 con 0x55a79b88c900 2023-10-06T05:24:48.346 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:48.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.344+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v117'279 uv279 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.344+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b890f90 con 0x55a79b88c900 2023-10-06T05:24:48.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.346+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.346+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b890b40 con 0x55a79b88c900 2023-10-06T05:24:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.347+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'281 uv281 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.347+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55a79b890f40 0x55a79b894000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.347+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894540 con 0x55a79b890f40 2023-10-06T05:24:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.348+0000 7f5b738ac700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55a79b890f40 0x55a79b894000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.348+0000 7f5b738ac700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55a79b890f40 0x55a79b894000 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.349+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v117'122 uv121 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b890f40 2023-10-06T05:24:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.349+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894540 con 0x55a79b890f40 2023-10-06T05:24:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.351+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v117'123 uv123 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b890f40 2023-10-06T05:24:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.351+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894540 con 0x55a79b890f40 2023-10-06T05:24:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.353+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v117'124 uv124 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b890f40 2023-10-06T05:24:48.354 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.353+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894940 con 0x55a79b890f40 2023-10-06T05:24:48.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.355+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'125 uv125 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b890f40 2023-10-06T05:24:48.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.355+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894940 con 0x55a79b890f40 2023-10-06T05:24:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.358+0000 7f5b738ac700 1 -- 172.21.15.45:0/2217804540 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'126 uv126 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b6402f070 con 0x55a79b890f40 2023-10-06T05:24:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.358+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894940 con 0x55a79b8895e0 2023-10-06T05:24:48.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.359+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.359+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894940 con 0x55a79b8895e0 2023-10-06T05:24:48.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.360+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v117'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.360+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894940 con 0x55a79b8895e0 2023-10-06T05:24:48.363 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:48.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.362+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v117'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.362+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b895950 con 0x55a79b8895e0 2023-10-06T05:24:48.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.363+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.363+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b895950 con 0x55a79b8895e0 2023-10-06T05:24:48.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.364+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.364+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b894940 con 0x55a79b8895e0 2023-10-06T05:24:48.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.365+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.365+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a79b894e50 0x55a79b897230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.365+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b8978f0 con 0x55a79b894e50 2023-10-06T05:24:48.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.365+0000 7f5b730ab700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a79b894e50 0x55a79b897230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.366+0000 7f5b730ab700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a79b894e50 0x55a79b897230 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.367+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v117'243 uv237 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.367+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897930 con 0x55a79b894e50 2023-10-06T05:24:48.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.369+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v117'244 uv244 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.369+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897d30 con 0x55a79b894e50 2023-10-06T05:24:48.371 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.370+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v117'245 uv245 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.370+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897d30 con 0x55a79b894e50 2023-10-06T05:24:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.371+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'246 uv246 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.371+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897d30 con 0x55a79b894e50 2023-10-06T05:24:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.372+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'247 uv247 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.372+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897d30 con 0x55a79b894e50 2023-10-06T05:24:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.373+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v117'248 uv242 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.374+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897d30 con 0x55a79b894e50 2023-10-06T05:24:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.375+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v117'249 uv249 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.375+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897d30 con 0x55a79b894e50 2023-10-06T05:24:48.377 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.376+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v117'250 uv250 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.376+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b897d30 con 0x55a79b894e50 2023-10-06T05:24:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.377+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'251 uv251 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.377+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b89aaa0 con 0x55a79b894e50 2023-10-06T05:24:48.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.378+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'252 uv252 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b894e50 2023-10-06T05:24:48.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.378+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.379+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v117'107 uv106 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.379+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.380+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'108 uv108 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.380+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.381+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v117'109 uv109 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.382+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.383+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'110 uv110 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.383+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55a79b898b30 0x55a79b89b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.383+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b898b30 2023-10-06T05:24:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.383+0000 7f5b740ad700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55a79b898b30 0x55a79b89b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.383+0000 7f5b740ad700 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55a79b898b30 0x55a79b89b1f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.384+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b898b30 2023-10-06T05:24:48.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.384+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b898b30 2023-10-06T05:24:48.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.386+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v117'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b898b30 2023-10-06T05:24:48.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.386+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b898b30 2023-10-06T05:24:48.390 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.388+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v117'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b898b30 2023-10-06T05:24:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.389+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b898b30 2023-10-06T05:24:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.392+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b898b30 2023-10-06T05:24:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.392+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b898b30 2023-10-06T05:24:48.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.394+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b898b30 2023-10-06T05:24:48.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.394+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b898b30 2023-10-06T05:24:48.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.397+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b898b30 2023-10-06T05:24:48.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.397+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b88c900 2023-10-06T05:24:48.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.398+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v117'125 uv124 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.398+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b88c900 2023-10-06T05:24:48.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.400+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v117'126 uv126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.400+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b88c900 2023-10-06T05:24:48.403 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:48.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.401+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v117'127 uv127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.401+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b88c900 2023-10-06T05:24:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.402+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'128 uv128 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.402+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b88c900 2023-10-06T05:24:48.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.403+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'129 uv129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5b6c081e40 con 0x55a79b88c900 2023-10-06T05:24:48.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.403+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.405+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v117'302 uv294 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.405+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.406+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v117'303 uv303 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.406+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.409+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v117'304 uv304 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.411 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.409+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.411+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v117'305 uv305 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.411+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.413+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'306 uv306 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.413+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.415+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v117'307 uv301 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.415+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.418+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v117'308 uv308 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.418+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.425 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:48.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.424+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v117'309 uv309 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.424+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.425+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'310 uv310 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.425+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.428+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'311 uv311 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.428+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b898220 con 0x55a79b8895e0 2023-10-06T05:24:48.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.429+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'312 uv312 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.429+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55a79b89b730 con 0x55a79b8895e0 2023-10-06T05:24:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b730ab700 1 -- 172.21.15.45:0/2217804540 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'313 uv313 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5b68017040 con 0x55a79b8895e0 2023-10-06T05:24:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a79b8895e0 msgr2=0x55a79b88b9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55a79b8895e0 0x55a79b88b9c0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55a79b863990 msgr2=0x55a79b7ba590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55a79b863990 0x55a79b7ba590 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a79b894e50 msgr2=0x55a79b897230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55a79b894e50 0x55a79b897230 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55a79b884ca0 msgr2=0x55a79b887080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55a79b884ca0 0x55a79b887080 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55a79b898b30 msgr2=0x55a79b89b1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55a79b898b30 0x55a79b89b1f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 reap_dead start 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55a79b890f40 msgr2=0x55a79b894000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55a79b890f40 0x55a79b894000 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b740ad700 1 -- 172.21.15.45:0/2217804540 reap_dead start 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55a79b88c900 msgr2=0x55a79b88ece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55a79b88c900 0x55a79b88ece0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b600658c0 msgr2=0x7f5b60067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b600658c0 0x7f5b60067d70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x55a79b7be740 tx=0x7f5b6401e000 comp rx=0 tx=0).stop 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b876d30 msgr2=0x55a79b863450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b876d30 0x55a79b863450 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c005b20 tx=0x7f5b6c007040 comp rx=0 tx=0).stop 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 shutdown_connections 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55a79b88c900 0x55a79b88ece0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79b876d30 0x55a79b863450 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 --2- 172.21.15.45:0/2217804540 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b600658c0 0x7f5b60067d70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 >> 172.21.15.45:0/2217804540 conn(0x55a79b78f450 msgr2=0x55a79b85d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 shutdown_connections 2023-10-06T05:24:48.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.432+0000 7f5b7a68e040 1 -- 172.21.15.45:0/2217804540 wait complete. 2023-10-06T05:24:48.437 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2023-10-06T05:24:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.486+0000 7f32eeda3040 1 Processor -- start 2023-10-06T05:24:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.486+0000 7f32eeda3040 1 -- start start 2023-10-06T05:24:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32eeda3040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0a799b0 0x55d5e0a6fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32eeda3040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0b02b70 0x55d5e0aeba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32eeda3040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b06f70 0x55d5e0a768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32eeda3040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e0979980 con 0x55d5e0b02b70 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32eeda3040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e0979620 con 0x55d5e0b06f70 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32eeda3040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5e0970400 con 0x55d5e0a799b0 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0b02b70 0x55d5e0aeba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0b02b70 0x55d5e0aeba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34380/0 (socket says 172.21.15.45:34380) 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 -- 172.21.15.45:0/619217880 learned_addr learned my addr 172.21.15.45:0/619217880 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e87c2700 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b06f70 0x55d5e0a768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e7fc1700 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0a799b0 0x55d5e0a6fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 -- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0a799b0 msgr2=0x55d5e0a6fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0a799b0 0x55d5e0a6fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 -- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b06f70 msgr2=0x55d5e0a768f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b06f70 0x55d5e0a768f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e77c0700 1 -- 172.21.15.45:0/619217880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0005cf0 con 0x55d5e0b02b70 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.487+0000 7f32e87c2700 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b06f70 0x55d5e0a768f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32e77c0700 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0b02b70 0x55d5e0aeba00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x55d5e0a5ce80 tx=0x7f32d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dde87f5c1e7f877 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32e67be700 1 -- 172.21.15.45:0/619217880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0000c90 con 0x55d5e0b02b70 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32e67be700 1 -- 172.21.15.45:0/619217880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d0008970 con 0x55d5e0b02b70 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32e67be700 1 -- 172.21.15.45:0/619217880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d001d420 con 0x55d5e0b02b70 2023-10-06T05:24:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 -- 172.21.15.45:0/619217880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0b02b70 msgr2=0x55d5e0aeba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0b02b70 0x55d5e0aeba00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x55d5e0a5ce80 tx=0x7f32d0016000 comp rx=0 tx=0).stop 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 -- 172.21.15.45:0/619217880 shutdown_connections 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b06f70 0x55d5e0a768f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0a799b0 0x55d5e0a6fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 --2- 172.21.15.45:0/619217880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0b02b70 0x55d5e0aeba00 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 -- 172.21.15.45:0/619217880 >> 172.21.15.45:0/619217880 conn(0x55d5e0a305f0 msgr2=0x55d5e0a321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 -- 172.21.15.45:0/619217880 shutdown_connections 2023-10-06T05:24:48.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.488+0000 7f32eeda3040 1 -- 172.21.15.45:0/619217880 wait complete. 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.489+0000 7f32eeda3040 1 Processor -- start 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.489+0000 7f32eeda3040 1 -- start start 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32eeda3040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0a799b0 0x55d5e0a768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32eeda3040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b02b70 0x55d5e0a76e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32eeda3040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0b06f70 0x55d5e0ad6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32eeda3040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e0ad9400 con 0x55d5e0a799b0 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32eeda3040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e0ad96a0 con 0x55d5e0b02b70 2023-10-06T05:24:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32eeda3040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5e0ad9d30 con 0x55d5e0b06f70 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0a799b0 0x55d5e0a768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0a799b0 0x55d5e0a768f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34384/0 (socket says 172.21.15.45:34384) 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 -- 172.21.15.45:0/1331405340 learned_addr learned my addr 172.21.15.45:0/1331405340 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e77c0700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b02b70 0x55d5e0a76e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e87c2700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0b06f70 0x55d5e0ad6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 -- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0b06f70 msgr2=0x55d5e0ad6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d5e0b06f70 0x55d5e0ad6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 -- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b02b70 msgr2=0x55d5e0a76e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5e0b02b70 0x55d5e0a76e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.490+0000 7f32e7fc1700 1 -- 172.21.15.45:0/1331405340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5e09a21c0 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.491+0000 7f32e7fc1700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0a799b0 0x55d5e0a768f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f32d8008b20 tx=0x7f32d800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.491+0000 7f32e4fbb700 1 -- 172.21.15.45:0/1331405340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d800f050 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.491+0000 7f32e4fbb700 1 -- 172.21.15.45:0/1331405340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d8010040 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.491+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5e09883d0 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.491+0000 7f32e4fbb700 1 -- 172.21.15.45:0/1331405340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8006c30 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.491+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5e097bae0 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.492+0000 7f32e4fbb700 1 -- 172.21.15.45:0/1331405340 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32d8006d90 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.493+0000 7f32e4fbb700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32c8065890 0x7f32c8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.493+0000 7f32e4fbb700 1 -- 172.21.15.45:0/1331405340 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f32d80843c0 con 0x55d5e0a799b0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.493+0000 7f32e77c0700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32c8065890 0x7f32c8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.493+0000 7f32eeda3040 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d5e0ad8020 0x55d5e0ad8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.493+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:1 14.14 14:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) v8 -- 0x55d5e0ad8960 con 0x55d5e0ad8020 2023-10-06T05:24:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.493+0000 7f32e87c2700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d5e0ad8020 0x55d5e0ad8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.494+0000 7f32e77c0700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32c8065890 0x7f32c8067d40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f32d0005cc0 tx=0x7f32d001e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.494+0000 7f32e87c2700 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d5e0ad8020 0x55d5e0ad8420 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32e87c2700 1 -- 172.21.15.45:0/1331405340 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v117'565 uv565 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f32e0004630 con 0x55d5e0ad8020 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d5e0ad8020 msgr2=0x55d5e0ad8420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32eeda3040 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d5e0ad8020 0x55d5e0ad8420 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32c8065890 msgr2=0x7f32c8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32eeda3040 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f32c8065890 0x7f32c8067d40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f32d0005cc0 tx=0x7f32d001e000 comp rx=0 tx=0).stop 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0a799b0 msgr2=0x55d5e0a768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32eeda3040 1 --2- 172.21.15.45:0/1331405340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5e0a799b0 0x55d5e0a768f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f32d8008b20 tx=0x7f32d800c040 comp rx=0 tx=0).stop 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.495+0000 7f32e87c2700 1 -- 172.21.15.45:0/1331405340 reap_dead start 2023-10-06T05:24:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.496+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 shutdown_connections 2023-10-06T05:24:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.496+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 >> 172.21.15.45:0/1331405340 conn(0x55d5e0a305f0 msgr2=0x55d5e0a6f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.496+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 shutdown_connections 2023-10-06T05:24:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.496+0000 7f32eeda3040 1 -- 172.21.15.45:0/1331405340 wait complete. 2023-10-06T05:24:48.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:48.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.641+0000 7f5f4882c700 1 Processor -- start 2023-10-06T05:24:48.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.641+0000 7f5f4882c700 1 -- start start 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f4882c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44159150 0x7f5f440ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f4882c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5f44067620 0x7f5f44067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f4882c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f440c9c90 con 0x7f5f44159520 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f4882c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f44067fe0 con 0x7f5f44067620 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f4882c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4415d8e0 con 0x7f5f44159150 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f41d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5f44159520 0x7f5f4415ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47510/0 (socket says 172.21.15.45:47510) 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f41d9b700 1 -- 172.21.15.45:0/1283878226 learned_addr learned my addr 172.21.15.45:0/1283878226 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f4259c700 1 --2- 172.21.15.45:0/1283878226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44159150 0x7f5f440ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.642+0000 7f5f42d9d700 1 --2- 172.21.15.45:0/1283878226 >> v2:172.21.15.144:3300/0 conn(0x7f5f44067620 0x7f5f44067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4159a700 1 -- 172.21.15.45:0/1283878226 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219423031 0 0) 0x7f5f440c9c90 con 0x7f5f44159520 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4159a700 1 -- 172.21.15.45:0/1283878226 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f2c003570 con 0x7f5f44159520 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4259c700 1 -- 172.21.15.45:0/1283878226 >> v2:172.21.15.144:3300/0 conn(0x7f5f44067620 msgr2=0x7f5f44067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4259c700 1 --2- 172.21.15.45:0/1283878226 >> v2:172.21.15.144:3300/0 conn(0x7f5f44067620 0x7f5f44067aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4259c700 1 -- 172.21.15.45:0/1283878226 >> v1:172.21.15.45:6789/0 conn(0x7f5f44159520 legacy=0x7f5f4415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4259c700 1 -- 172.21.15.45:0/1283878226 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4415dbc0 con 0x7f5f44159150 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f42d9d700 1 --2- 172.21.15.45:0/1283878226 >> v2:172.21.15.144:3300/0 conn(0x7f5f44067620 0x7f5f44067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4259c700 1 --2- 172.21.15.45:0/1283878226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44159150 0x7f5f440ca120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098e0 tx=0x7f5f340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefeb82f6de55384 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4159a700 1 -- 172.21.15.45:0/1283878226 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34011630 con 0x7f5f44159150 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4159a700 1 -- 172.21.15.45:0/1283878226 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34011790 con 0x7f5f44159150 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.643+0000 7f5f4159a700 1 -- 172.21.15.45:0/1283878226 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34011b30 con 0x7f5f44159150 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 -- 172.21.15.45:0/1283878226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44159150 msgr2=0x7f5f440ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 --2- 172.21.15.45:0/1283878226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44159150 0x7f5f440ca120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098e0 tx=0x7f5f340091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 -- 172.21.15.45:0/1283878226 shutdown_connections 2023-10-06T05:24:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 --2- 172.21.15.45:0/1283878226 >> v2:172.21.15.144:3300/0 conn(0x7f5f44067620 0x7f5f44067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 --2- 172.21.15.45:0/1283878226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44159150 0x7f5f440ca120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 -- 172.21.15.45:0/1283878226 >> 172.21.15.45:0/1283878226 conn(0x7f5f440691c0 msgr2=0x7f5f440cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 -- 172.21.15.45:0/1283878226 shutdown_connections 2023-10-06T05:24:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.644+0000 7f5f4882c700 1 -- 172.21.15.45:0/1283878226 wait complete. 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.645+0000 7f5f4882c700 1 Processor -- start 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.645+0000 7f5f4882c700 1 -- start start 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f4882c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44067620 0x7f5f44159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f4882c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f44159150 0x7f5f441575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f4882c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f440cbe60 0x7f5f44157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f4882c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440c6ee0 con 0x7f5f44159150 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f4882c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44169240 con 0x7f5f440cbe60 2023-10-06T05:24:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f4882c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440ae5f0 con 0x7f5f44067620 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f44159150 0x7f5f441575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f44159150 0x7f5f441575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34390/0 (socket says 172.21.15.45:34390) 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 -- 172.21.15.45:0/2897173884 learned_addr learned my addr 172.21.15.45:0/2897173884 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f42d9d700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f440cbe60 0x7f5f44157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f4259c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44067620 0x7f5f44159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 -- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44067620 msgr2=0x7f5f44159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44067620 0x7f5f44159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 -- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f440cbe60 msgr2=0x7f5f44157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f440cbe60 0x7f5f44157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f41d9b700 1 -- 172.21.15.45:0/2897173884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c009a20 con 0x7f5f44159150 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.646+0000 7f5f42d9d700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f440cbe60 0x7f5f44157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.647+0000 7f5f41d9b700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f44159150 0x7f5f441575b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800c590 tx=0x7f5f38007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.647+0000 7f5f327fc700 1 -- 172.21.15.45:0/2897173884 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38005b30 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.647+0000 7f5f327fc700 1 -- 172.21.15.45:0/2897173884 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38005cb0 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.647+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f34009420 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.647+0000 7f5f327fc700 1 -- 172.21.15.45:0/2897173884 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38007320 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.647+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f440c4d50 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.648+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f441693b0 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.648+0000 7f5f327fc700 1 -- 172.21.15.45:0/2897173884 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f3801b3b0 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.649+0000 7f5f327fc700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f24066f30 0x7f5f240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.649+0000 7f5f327fc700 1 -- 172.21.15.45:0/2897173884 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f5f38084630 con 0x7f5f44159150 2023-10-06T05:24:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.649+0000 7f5f4259c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f24066f30 0x7f5f240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.650+0000 7f5f4259c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f24066f30 0x7f5f240693e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5f440c2730 tx=0x7f5f34017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.653+0000 7f5f327fc700 1 -- 172.21.15.45:0/2897173884 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f441693b0 con 0x7f5f44159150 2023-10-06T05:24:48.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:48.903+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5f441693b0 con 0x7f5f44159150 2023-10-06T05:24:49.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.179+0000 7f5f327fc700 1 -- 172.21.15.45:0/2897173884 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v246) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5f441693b0 con 0x7f5f44159150 2023-10-06T05:24:49.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f24066f30 msgr2=0x7f5f240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f24066f30 0x7f5f240693e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5f440c2730 tx=0x7f5f34017000 comp rx=0 tx=0).stop 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f44159150 msgr2=0x7f5f441575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f44159150 0x7f5f441575b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800c590 tx=0x7f5f38007e00 comp rx=0 tx=0).stop 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 shutdown_connections 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5f24066f30 0x7f5f240693e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f440cbe60 0x7f5f44157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f44067620 0x7f5f44159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 --2- 172.21.15.45:0/2897173884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f44159150 0x7f5f441575b0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.182+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 >> 172.21.15.45:0/2897173884 conn(0x7f5f440691c0 msgr2=0x7f5f44070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.183+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 shutdown_connections 2023-10-06T05:24:49.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.183+0000 7f5f4882c700 1 -- 172.21.15.45:0/2897173884 wait complete. 2023-10-06T05:24:49.186 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:49.191 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:24:49.190+0000 7fc3d1ebf700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:24:49.192 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:49.190+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:49.192 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:24:49.190+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:24:49.198 INFO:tasks.ceph.mds.b.smithi144.stderr: -13> 2023-10-06T05:24:49.190+0000 7fc3d1ebf700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2023-10-06T05:24:49.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:49.218 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:24:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.336+0000 7fc5230a9700 1 Processor -- start 2023-10-06T05:24:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.336+0000 7fc5230a9700 1 -- start start 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc5230a9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c159200 0x7fc51c0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc5230a9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc51c1595d0 0x7fc51c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc5230a9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c0c7d00 con 0x7fc51c06b5c0 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc5230a9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c0c7780 con 0x7fc51c1595d0 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc5230a9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0c86d0 con 0x7fc51c159200 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc51bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc51c1595d0 0x7fc51c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc521646700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc51c06b5c0 0x7fc51c0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47522/0 (socket says 172.21.15.45:47522) 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.337+0000 7fc521646700 1 -- 172.21.15.45:0/392167185 learned_addr learned my addr 172.21.15.45:0/392167185 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:49.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc520e45700 1 --2- 172.21.15.45:0/392167185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c159200 0x7fc51c0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51bfff700 1 -- 172.21.15.45:0/392167185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c159200 msgr2=0x7fc51c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51bfff700 1 --2- 172.21.15.45:0/392167185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c159200 0x7fc51c0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51bfff700 1 -- 172.21.15.45:0/392167185 >> v1:172.21.15.45:6789/0 conn(0x7fc51c06b5c0 legacy=0x7fc51c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51bfff700 1 -- 172.21.15.45:0/392167185 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c06bb40 con 0x7fc51c1595d0 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51b7fe700 1 -- 172.21.15.45:0/392167185 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416674772 0 0) 0x7fc51c0c7d00 con 0x7fc51c06b5c0 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51bfff700 1 --2- 172.21.15.45:0/392167185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c1595d0 0x7fc51c06b080 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098e0 tx=0x7fc5100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765acc41acc692ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51b7fe700 1 -- 172.21.15.45:0/392167185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510009e80 con 0x7fc51c1595d0 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51b7fe700 1 -- 172.21.15.45:0/392167185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000b860 con 0x7fc51c1595d0 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.338+0000 7fc51b7fe700 1 -- 172.21.15.45:0/392167185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000bbe0 con 0x7fc51c1595d0 2023-10-06T05:24:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 -- 172.21.15.45:0/392167185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c1595d0 msgr2=0x7fc51c06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 --2- 172.21.15.45:0/392167185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c1595d0 0x7fc51c06b080 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098e0 tx=0x7fc5100091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 -- 172.21.15.45:0/392167185 shutdown_connections 2023-10-06T05:24:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 --2- 172.21.15.45:0/392167185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c159200 0x7fc51c0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 --2- 172.21.15.45:0/392167185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c1595d0 0x7fc51c06b080 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 -- 172.21.15.45:0/392167185 >> 172.21.15.45:0/392167185 conn(0x7fc51c06f090 msgr2=0x7fc51c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 -- 172.21.15.45:0/392167185 shutdown_connections 2023-10-06T05:24:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.339+0000 7fc5230a9700 1 -- 172.21.15.45:0/392167185 wait complete. 2023-10-06T05:24:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.340+0000 7fc5230a9700 1 Processor -- start 2023-10-06T05:24:49.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.341+0000 7fc5230a9700 1 -- start start 2023-10-06T05:24:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc5230a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c06b5c0 0x7fc51c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc5230a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51c159200 0x7fc51c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc5230a9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c1595d0 0x7fc51c0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc5230a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c1698d0 con 0x7fc51c06b5c0 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc5230a9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0cb940 con 0x7fc51c159200 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc5230a9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c153fa0 con 0x7fc51c1595d0 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c06b5c0 0x7fc51c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c06b5c0 0x7fc51c153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34414/0 (socket says 172.21.15.45:34414) 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 -- 172.21.15.45:0/2954500074 learned_addr learned my addr 172.21.15.45:0/2954500074 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc521646700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c1595d0 0x7fc51c0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc51bfff700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51c159200 0x7fc51c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 -- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c1595d0 msgr2=0x7fc51c0b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c1595d0 0x7fc51c0b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 -- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51c159200 msgr2=0x7fc51c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51c159200 0x7fc51c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 -- 172.21.15.45:0/2954500074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510009420 con 0x7fc51c06b5c0 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc521646700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c1595d0 0x7fc51c0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc51bfff700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51c159200 0x7fc51c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc520e45700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c06b5c0 0x7fc51c153a60 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00d890 tx=0x7fc50c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc518ff9700 1 -- 172.21.15.45:0/2954500074 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00b600 con 0x7fc51c06b5c0 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc518ff9700 1 -- 172.21.15.45:0/2954500074 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c00b760 con 0x7fc51c06b5c0 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.342+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c0c61b0 con 0x7fc51c06b5c0 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.343+0000 7fc518ff9700 1 -- 172.21.15.45:0/2954500074 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00bae0 con 0x7fc51c06b5c0 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.343+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c0c6770 con 0x7fc51c06b5c0 2023-10-06T05:24:49.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.343+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c0cbf60 con 0x7fc51c06b5c0 2023-10-06T05:24:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.344+0000 7fc518ff9700 1 -- 172.21.15.45:0/2954500074 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc50c01e460 con 0x7fc51c06b5c0 2023-10-06T05:24:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.344+0000 7fc518ff9700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc508066f30 0x7fc5080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.345+0000 7fc518ff9700 1 -- 172.21.15.45:0/2954500074 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fc50c0857b0 con 0x7fc51c06b5c0 2023-10-06T05:24:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.345+0000 7fc51bfff700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc508066f30 0x7fc5080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.345+0000 7fc51bfff700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc508066f30 0x7fc5080693e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc51c16cf00 tx=0x7fc510017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.349+0000 7fc518ff9700 1 -- 172.21.15.45:0/2954500074 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc50c051e80 con 0x7fc51c06b5c0 2023-10-06T05:24:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.598+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc51c06be20 con 0x7fc51c06b5c0 2023-10-06T05:24:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.598+0000 7fc518ff9700 1 -- 172.21.15.45:0/2954500074 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 248 v248) v1 ==== 95+0+3839 (secure 0 0 0) 0x7fc50c0554a0 con 0x7fc51c06b5c0 2023-10-06T05:24:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:49.600 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":248,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":248,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:49.191526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc508066f30 msgr2=0x7fc5080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc508066f30 0x7fc5080693e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc51c16cf00 tx=0x7fc510017000 comp rx=0 tx=0).stop 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c06b5c0 msgr2=0x7fc51c153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c06b5c0 0x7fc51c153a60 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00d890 tx=0x7fc50c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 shutdown_connections 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc508066f30 0x7fc5080693e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51c1595d0 0x7fc51c0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c06b5c0 0x7fc51c153a60 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 --2- 172.21.15.45:0/2954500074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51c159200 0x7fc51c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 >> 172.21.15.45:0/2954500074 conn(0x7fc51c06f090 msgr2=0x7fc51c0af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 shutdown_connections 2023-10-06T05:24:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.601+0000 7fc5230a9700 1 -- 172.21.15.45:0/2954500074 wait complete. 2023-10-06T05:24:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 248 2023-10-06T05:24:49.616 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:49.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:49.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.746+0000 7f5a90d1d700 1 Processor -- start 2023-10-06T05:24:49.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.747+0000 7f5a90d1d700 1 -- start start 2023-10-06T05:24:49.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a90d1d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c159330 0x7f5a8c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a90d1d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a8c0cc390 0x7f5a8c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a90d1d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c0cb530 con 0x7f5a8c158f60 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a90d1d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c1597a0 con 0x7f5a8c0cc390 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a90d1d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0ccd50 con 0x7f5a8c159330 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a8c0cc390 0x7f5a8c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8a59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5a8c158f60 0x7f5a8c0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47536/0 (socket says 172.21.15.45:47536) 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a8c0cc390 0x7f5a8c0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34416/0 (socket says 172.21.15.45:34416) 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8a59c700 1 -- 172.21.15.45:0/4010833830 learned_addr learned my addr 172.21.15.45:0/4010833830 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a89d9b700 1 --2- 172.21.15.45:0/4010833830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c159330 0x7f5a8c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8ad9d700 1 -- 172.21.15.45:0/4010833830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c159330 msgr2=0x7f5a8c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8ad9d700 1 --2- 172.21.15.45:0/4010833830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c159330 0x7f5a8c06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8ad9d700 1 -- 172.21.15.45:0/4010833830 >> v1:172.21.15.45:6789/0 conn(0x7f5a8c158f60 legacy=0x7f5a8c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8ad9d700 1 -- 172.21.15.45:0/4010833830 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c069bd0 con 0x7f5a8c0cc390 2023-10-06T05:24:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8959a700 1 -- 172.21.15.45:0/4010833830 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804527 0 0) 0x7f5a7c001110 con 0x7f5a8c158f60 2023-10-06T05:24:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a89d9b700 1 --2- 172.21.15.45:0/4010833830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c159330 0x7f5a8c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.748+0000 7f5a8ad9d700 1 --2- 172.21.15.45:0/4010833830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c0cc390 0x7f5a8c0cc810 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009e90 tx=0x7f5a84009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592b276e5ba85381 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a8959a700 1 -- 172.21.15.45:0/4010833830 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400e070 con 0x7f5a8c0cc390 2023-10-06T05:24:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a8959a700 1 -- 172.21.15.45:0/4010833830 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8400bbf0 con 0x7f5a8c0cc390 2023-10-06T05:24:49.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a8959a700 1 -- 172.21.15.45:0/4010833830 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84012430 con 0x7f5a8c0cc390 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 -- 172.21.15.45:0/4010833830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c0cc390 msgr2=0x7f5a8c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/4010833830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c0cc390 0x7f5a8c0cc810 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009e90 tx=0x7f5a84009780 comp rx=0 tx=0).stop 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 -- 172.21.15.45:0/4010833830 shutdown_connections 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/4010833830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c0cc390 0x7f5a8c0cc810 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/4010833830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c159330 0x7f5a8c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 -- 172.21.15.45:0/4010833830 >> 172.21.15.45:0/4010833830 conn(0x7f5a8c0cdf50 msgr2=0x7f5a8c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 -- 172.21.15.45:0/4010833830 shutdown_connections 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.749+0000 7f5a90d1d700 1 -- 172.21.15.45:0/4010833830 wait complete. 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 Processor -- start 2023-10-06T05:24:49.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 -- start start 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c0cc390 0x7f5a8c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c158f60 0x7f5a8c0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c159330 0x7f5a8c0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0c6d10 con 0x7f5a8c159330 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0c6100 con 0x7f5a8c0cc390 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.751+0000 7f5a90d1d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0c63e0 con 0x7f5a8c158f60 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c159330 0x7f5a8c0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c159330 0x7f5a8c0c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34422/0 (socket says 172.21.15.45:34422) 2023-10-06T05:24:49.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 -- 172.21.15.45:0/115554589 learned_addr learned my addr 172.21.15.45:0/115554589 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a89d9b700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c158f60 0x7f5a8c0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8a59c700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c0cc390 0x7f5a8c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 -- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c158f60 msgr2=0x7f5a8c0c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c158f60 0x7f5a8c0c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 -- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c0cc390 msgr2=0x7f5a8c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c0cc390 0x7f5a8c153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 -- 172.21.15.45:0/115554589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84009a20 con 0x7f5a8c159330 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a8ad9d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c159330 0x7f5a8c0c76e0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0c4f80 tx=0x7f5a84019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a827fc700 1 -- 172.21.15.45:0/115554589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84027470 con 0x7f5a8c159330 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a827fc700 1 -- 172.21.15.45:0/115554589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a840275d0 con 0x7f5a8c159330 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c0c66c0 con 0x7f5a8c159330 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a827fc700 1 -- 172.21.15.45:0/115554589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84027990 con 0x7f5a8c159330 2023-10-06T05:24:49.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.752+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c154010 con 0x7f5a8c159330 2023-10-06T05:24:49.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.753+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c0c6ec0 con 0x7f5a8c159330 2023-10-06T05:24:49.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.754+0000 7f5a827fc700 1 -- 172.21.15.45:0/115554589 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a84024070 con 0x7f5a8c159330 2023-10-06T05:24:49.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.754+0000 7f5a827fc700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a6c066f30 0x7f5a6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.754+0000 7f5a827fc700 1 -- 172.21.15.45:0/115554589 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7f5a84087730 con 0x7f5a8c159330 2023-10-06T05:24:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.754+0000 7f5a8a59c700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a6c066f30 0x7f5a6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.755+0000 7f5a8a59c700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a6c066f30 0x7f5a6c0693e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c16c6c0 tx=0x7f5a7c007b90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:49.759+0000 7f5a827fc700 1 -- 172.21.15.45:0/115554589 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a84053e00 con 0x7f5a8c159330 2023-10-06T05:24:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.007+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a8c068fc0 con 0x7f5a8c159330 2023-10-06T05:24:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.007+0000 7f5a827fc700 1 -- 172.21.15.45:0/115554589 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 248 v248) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f5a8402c020 con 0x7f5a8c159330 2023-10-06T05:24:50.010 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:50.010 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":248,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":248,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:49.191526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a6c066f30 msgr2=0x7f5a6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a6c066f30 0x7f5a6c0693e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c16c6c0 tx=0x7f5a7c007b90 comp rx=0 tx=0).stop 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c159330 msgr2=0x7f5a8c0c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c159330 0x7f5a8c0c76e0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0c4f80 tx=0x7f5a84019000 comp rx=0 tx=0).stop 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 shutdown_connections 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5a6c066f30 0x7f5a6c0693e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8c159330 0x7f5a8c0c76e0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c0cc390 0x7f5a8c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 --2- 172.21.15.45:0/115554589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a8c158f60 0x7f5a8c0c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 >> 172.21.15.45:0/115554589 conn(0x7f5a8c0cdf50 msgr2=0x7f5a8c154990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 shutdown_connections 2023-10-06T05:24:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.010+0000 7f5a90d1d700 1 -- 172.21.15.45:0/115554589 wait complete. 2023-10-06T05:24:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 248 2023-10-06T05:24:50.025 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:24:50.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:50.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.155+0000 7fe1bdadd700 1 Processor -- start 2023-10-06T05:24:50.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.155+0000 7fe1bdadd700 1 -- start start 2023-10-06T05:24:50.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.155+0000 7fe1bdadd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1b81570b0 0x7fe1b806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.155+0000 7fe1bdadd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe1b815ac10 0x7fe1b80ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.155+0000 7fe1bdadd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b815b380 con 0x7fe1b8156ce0 2023-10-06T05:24:50.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.155+0000 7fe1bdadd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b806c010 con 0x7fe1b81570b0 2023-10-06T05:24:50.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.155+0000 7fe1bdadd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80b02f0 con 0x7fe1b815ac10 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1b81570b0 0x7fe1b806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe1b8156ce0 0x7fe1b80afbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47550/0 (socket says 172.21.15.45:47550) 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b77fe700 1 -- 172.21.15.45:0/3961921349 learned_addr learned my addr 172.21.15.45:0/3961921349 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b7fff700 1 --2- 172.21.15.45:0/3961921349 >> v2:172.21.15.144:3300/0 conn(0x7fe1b815ac10 0x7fe1b80ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b6ffd700 1 -- 172.21.15.45:0/3961921349 >> v2:172.21.15.144:3300/0 conn(0x7fe1b815ac10 msgr2=0x7fe1b80ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b6ffd700 1 --2- 172.21.15.45:0/3961921349 >> v2:172.21.15.144:3300/0 conn(0x7fe1b815ac10 0x7fe1b80ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b6ffd700 1 -- 172.21.15.45:0/3961921349 >> v1:172.21.15.45:6789/0 conn(0x7fe1b8156ce0 legacy=0x7fe1b80afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b6ffd700 1 -- 172.21.15.45:0/3961921349 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b815b660 con 0x7fe1b81570b0 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b67fc700 1 -- 172.21.15.45:0/3961921349 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749614246 0 0) 0x7fe1b815b380 con 0x7fe1b8156ce0 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b6ffd700 1 --2- 172.21.15.45:0/3961921349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b81570b0 0x7fe1b806bad0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80ae6d0 tx=0x7fe1a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4e94353891085d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b67fc700 1 -- 172.21.15.45:0/3961921349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0009e80 con 0x7fe1b81570b0 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b67fc700 1 -- 172.21.15.45:0/3961921349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a000b820 con 0x7fe1b81570b0 2023-10-06T05:24:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.156+0000 7fe1b67fc700 1 -- 172.21.15.45:0/3961921349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000bba0 con 0x7fe1b81570b0 2023-10-06T05:24:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 -- 172.21.15.45:0/3961921349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b81570b0 msgr2=0x7fe1b806bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/3961921349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b81570b0 0x7fe1b806bad0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80ae6d0 tx=0x7fe1a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 -- 172.21.15.45:0/3961921349 shutdown_connections 2023-10-06T05:24:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/3961921349 >> v2:172.21.15.144:3300/0 conn(0x7fe1b815ac10 0x7fe1b80ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/3961921349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b81570b0 0x7fe1b806bad0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 -- 172.21.15.45:0/3961921349 >> 172.21.15.45:0/3961921349 conn(0x7fe1b806f090 msgr2=0x7fe1b806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 -- 172.21.15.45:0/3961921349 shutdown_connections 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.157+0000 7fe1bdadd700 1 -- 172.21.15.45:0/3961921349 wait complete. 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.158+0000 7fe1bdadd700 1 Processor -- start 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.158+0000 7fe1bdadd700 1 -- start start 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1bdadd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b8156ce0 0x7fe1b80c0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1bdadd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b81570b0 0x7fe1b8154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1bdadd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b815ac10 0x7fe1b8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1bdadd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80c1510 con 0x7fe1b8156ce0 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1bdadd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80abc70 con 0x7fe1b81570b0 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1bdadd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8168480 con 0x7fe1b815ac10 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1b77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b8156ce0 0x7fe1b80c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1b77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b8156ce0 0x7fe1b80c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34440/0 (socket says 172.21.15.45:34440) 2023-10-06T05:24:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1b77fe700 1 -- 172.21.15.45:0/859217660 learned_addr learned my addr 172.21.15.45:0/859217660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1b6ffd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b81570b0 0x7fe1b8154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.159+0000 7fe1b7fff700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b815ac10 0x7fe1b8067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1b77fe700 1 -- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b815ac10 msgr2=0x7fe1b8067f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1b77fe700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b815ac10 0x7fe1b8067f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1b77fe700 1 -- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b81570b0 msgr2=0x7fe1b8154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1b77fe700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b81570b0 0x7fe1b8154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1b77fe700 1 -- 172.21.15.45:0/859217660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0009420 con 0x7fe1b8156ce0 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1b7fff700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b815ac10 0x7fe1b8067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1b77fe700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b8156ce0 0x7fe1b80c0450 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800c5b0 tx=0x7fe1a8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1af7fe700 1 -- 172.21.15.45:0/859217660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8005b30 con 0x7fe1b8156ce0 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1af7fe700 1 -- 172.21.15.45:0/859217660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8005c90 con 0x7fe1b8156ce0 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1af7fe700 1 -- 172.21.15.45:0/859217660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8007300 con 0x7fe1b8156ce0 2023-10-06T05:24:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b80abf10 con 0x7fe1b8156ce0 2023-10-06T05:24:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.160+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b80c1230 con 0x7fe1b8156ce0 2023-10-06T05:24:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.161+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b806e2f0 con 0x7fe1b8156ce0 2023-10-06T05:24:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.162+0000 7fe1af7fe700 1 -- 172.21.15.45:0/859217660 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1a801b3b0 con 0x7fe1b8156ce0 2023-10-06T05:24:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.162+0000 7fe1af7fe700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe198066f60 0x7fe198069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.162+0000 7fe1af7fe700 1 -- 172.21.15.45:0/859217660 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fe1a8084580 con 0x7fe1b8156ce0 2023-10-06T05:24:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.162+0000 7fe1b6ffd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe198066f60 0x7fe198069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.163+0000 7fe1b6ffd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe198066f60 0x7fe198069410 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00093f0 tx=0x7fe1a0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.167+0000 7fe1af7fe700 1 -- 172.21.15.45:0/859217660 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1a8050c50 con 0x7fe1b8156ce0 2023-10-06T05:24:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.422+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1b80c6e70 con 0x7fe1b8156ce0 2023-10-06T05:24:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.422+0000 7fe1af7fe700 1 -- 172.21.15.45:0/859217660 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 249 v249) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fe1a8017020 con 0x7fe1b8156ce0 2023-10-06T05:24:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:50.425 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":249,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":9121,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/4239891954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4239891954},{"type":"v1","addr":"172.21.15.144:6835","nonce":4239891954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":249}],"filesystems":[{"mdsmap":{"epoch":248,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:49.191526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.424+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe198066f60 msgr2=0x7fe198069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.424+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe198066f60 0x7fe198069410 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00093f0 tx=0x7fe1a0017000 comp rx=0 tx=0).stop 2023-10-06T05:24:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.424+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b8156ce0 msgr2=0x7fe1b80c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.424+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b8156ce0 0x7fe1b80c0450 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800c5b0 tx=0x7fe1a8001070 comp rx=0 tx=0).stop 2023-10-06T05:24:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 shutdown_connections 2023-10-06T05:24:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe198066f60 0x7fe198069410 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b815ac10 0x7fe1b8067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b8156ce0 0x7fe1b80c0450 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 --2- 172.21.15.45:0/859217660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b81570b0 0x7fe1b8154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 >> 172.21.15.45:0/859217660 conn(0x7fe1b806f090 msgr2=0x7fe1b81555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 shutdown_connections 2023-10-06T05:24:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.425+0000 7fe1bdadd700 1 -- 172.21.15.45:0/859217660 wait complete. 2023-10-06T05:24:50.429 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 249 2023-10-06T05:24:50.441 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2023-10-06T05:24:50.442 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2023-10-06T05:24:50.442 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:50.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:24:50.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:50.442 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:50.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:24:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:24:50.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.639+0000 7f74bdad0700 1 Processor -- start 2023-10-06T05:24:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.639+0000 7f74bdad0700 1 -- start start 2023-10-06T05:24:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74bdad0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f74b80ccaa0 0x7f74b806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74bdad0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b80cb530 con 0x7f74b8159330 2023-10-06T05:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74bdad0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b8069bd0 con 0x7f74b8158f60 2023-10-06T05:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74bdad0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b8068fc0 con 0x7f74b80ccaa0 2023-10-06T05:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b6ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f74b8159330 0x7f74b80cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47552/0 (socket says 172.21.15.45:47552) 2023-10-06T05:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b6ffd700 1 -- 172.21.15.45:0/4070725819 learned_addr learned my addr 172.21.15.45:0/4070725819 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b7fff700 1 --2- 172.21.15.45:0/4070725819 >> v2:172.21.15.144:3300/0 conn(0x7f74b80ccaa0 0x7f74b806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319143713 0 0) 0x7f74a8001110 con 0x7f74b8159330 2023-10-06T05:24:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a4003570 con 0x7f74b8159330 2023-10-06T05:24:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1420284045 0 0) 0x7f74a4003570 con 0x7f74b8159330 2023-10-06T05:24:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 >> v2:172.21.15.144:3300/0 conn(0x7f74b80ccaa0 msgr2=0x7f74b806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 --2- 172.21.15.45:0/4070725819 >> v2:172.21.15.144:3300/0 conn(0x7f74b80ccaa0 0x7f74b806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 >> v1:172.21.15.144:6789/0 conn(0x7f74b8158f60 legacy=0x7f74b80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80692a0 con 0x7f74b8159330 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f74a8001600 con 0x7f74b8159330 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.640+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 >> v1:172.21.15.45:6789/0 conn(0x7f74b8159330 legacy=0x7f74b80cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a4003a60 0x7f74a4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a4004810 0x7f74a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74a40051c0 0x7f74a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8068fc0 con 0x7f74a40051c0 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400c0e0 con 0x7f74a4004810 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400c2a0 con 0x7f74a4003a60 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74a40051c0 0x7f74a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b6ffd700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a4004810 0x7f74a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b77fe700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a4003a60 0x7f74a4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 -- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a4003a60 msgr2=0x7f74a4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a4003a60 0x7f74a4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 -- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a4004810 msgr2=0x7f74a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a4004810 0x7f74a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 -- 172.21.15.45:0/4070725819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a4003570 con 0x7f74a40051c0 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 -- 172.21.15.45:0/4070725819 reap_dead start 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b7fff700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74a40051c0 0x7f74a400ba60 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f74b80ccea0 tx=0x7f74b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5762b989c77ca020 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000ea60 con 0x7f74a40051c0 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000f040 con 0x7f74a40051c0 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.641+0000 7f74b67fc700 1 -- 172.21.15.45:0/4070725819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000ee30 con 0x7f74a40051c0 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.642+0000 7f74bdad0700 1 -- 172.21.15.45:0/4070725819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74a40051c0 msgr2=0x7f74a400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.642+0000 7f74bdad0700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74a40051c0 0x7f74a400ba60 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f74b80ccea0 tx=0x7f74b000b040 comp rx=0 tx=0).stop 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.642+0000 7f74bdad0700 1 -- 172.21.15.45:0/4070725819 shutdown_connections 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.642+0000 7f74bdad0700 1 --2- 172.21.15.45:0/4070725819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74a40051c0 0x7f74a400ba60 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.642+0000 7f74bdad0700 1 -- 172.21.15.45:0/4070725819 >> 172.21.15.45:0/4070725819 conn(0x7f74b80cdf50 msgr2=0x7f74b80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.642+0000 7f74bdad0700 1 -- 172.21.15.45:0/4070725819 shutdown_connections 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.642+0000 7f74bdad0700 1 -- 172.21.15.45:0/4070725819 wait complete. 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.643+0000 7f74bdad0700 1 Processor -- start 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.643+0000 7f74bdad0700 1 -- start start 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74bdad0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a40051c0 0x7f74b8154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74bdad0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b80b3110 0x7f74b80b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74bdad0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74b815af70 0x7f74b815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74bdad0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b3ac0 con 0x7f74b815af70 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74bdad0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b1300 con 0x7f74b80b3110 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74bdad0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b0cf0 con 0x7f74a40051c0 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74b815af70 0x7f74b815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74b7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74b815af70 0x7f74b815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34464/0 (socket says 172.21.15.45:34464) 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74b7fff700 1 -- 172.21.15.45:0/983146251 learned_addr learned my addr 172.21.15.45:0/983146251 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74b77fe700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a40051c0 0x7f74b8154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.644+0000 7f74b6ffd700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b80b3110 0x7f74b80b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74b7fff700 1 -- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a40051c0 msgr2=0x7f74b8154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74b7fff700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a40051c0 0x7f74b8154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74b7fff700 1 -- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b80b3110 msgr2=0x7f74b80b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74b7fff700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b80b3110 0x7f74b80b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74b7fff700 1 -- 172.21.15.45:0/983146251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b815bcc0 con 0x7f74b815af70 2023-10-06T05:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74b7fff700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74b815af70 0x7f74b815b3c0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f74b80cdeb0 tx=0x7f74b001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74af7fe700 1 -- 172.21.15.45:0/983146251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000eee0 con 0x7f74b815af70 2023-10-06T05:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74af7fe700 1 -- 172.21.15.45:0/983146251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b001d630 con 0x7f74b815af70 2023-10-06T05:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b815bfa0 con 0x7f74b815af70 2023-10-06T05:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74af7fe700 1 -- 172.21.15.45:0/983146251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b001d9d0 con 0x7f74b815af70 2023-10-06T05:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.645+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b81553e0 con 0x7f74b815af70 2023-10-06T05:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.646+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b81555d0 con 0x7f74b815af70 2023-10-06T05:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.647+0000 7f74af7fe700 1 -- 172.21.15.45:0/983146251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74b00323f0 con 0x7f74b815af70 2023-10-06T05:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.647+0000 7f74af7fe700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7498066f30 0x7f74980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.647+0000 7f74af7fe700 1 -- 172.21.15.45:0/983146251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7f74b0087130 con 0x7f74b815af70 2023-10-06T05:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.647+0000 7f74b77fe700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7498066f30 0x7f74980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.648+0000 7f74b77fe700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7498066f30 0x7f74980693e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000c00 tx=0x7f74a0007d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.651+0000 7f74af7fe700 1 -- 172.21.15.45:0/983146251 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74b0053960 con 0x7f74b815af70 2023-10-06T05:24:50.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:50.901+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f74b815c190 con 0x7f74b815af70 2023-10-06T05:24:51.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.888+0000 7f74af7fe700 1 -- 172.21.15.45:0/983146251 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v250) v1 ==== 161+0+0 (secure 0 0 0) 0x7f74b0056f80 con 0x7f74b815af70 2023-10-06T05:24:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7498066f30 msgr2=0x7f74980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7498066f30 0x7f74980693e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000c00 tx=0x7f74a0007d10 comp rx=0 tx=0).stop 2023-10-06T05:24:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74b815af70 msgr2=0x7f74b815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74b815af70 0x7f74b815b3c0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f74b80cdeb0 tx=0x7f74b001a000 comp rx=0 tx=0).stop 2023-10-06T05:24:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 shutdown_connections 2023-10-06T05:24:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7498066f30 0x7f74980693e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74b815af70 0x7f74b815b3c0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a40051c0 0x7f74b8154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 --2- 172.21.15.45:0/983146251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b80b3110 0x7f74b80b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 >> 172.21.15.45:0/983146251 conn(0x7f74b80cdf50 msgr2=0x7f74b80c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 shutdown_connections 2023-10-06T05:24:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:51.891+0000 7f74bdad0700 1 -- 172.21.15.45:0/983146251 wait complete. 2023-10-06T05:24:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:51.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:24:52.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.044+0000 7f229d637700 1 Processor -- start 2023-10-06T05:24:52.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.044+0000 7f229d637700 1 -- start start 2023-10-06T05:24:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f229d637700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2298153620 0x7f229815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f229d637700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22981539f0 0x7f229815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f229d637700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22980ce690 con 0x7f22980cc8b0 2023-10-06T05:24:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f229d637700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229815a7a0 con 0x7f22981539f0 2023-10-06T05:24:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f229d637700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980ce3f0 con 0x7f2298153620 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f22967fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22981539f0 0x7f229815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f2296ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2298153620 0x7f229815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f22977fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f22980cc8b0 0x7f22980ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50704/0 (socket says 172.21.15.45:50704) 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f22967fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22981539f0 0x7f229815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:59444/0 (socket says 172.21.15.45:59444) 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.045+0000 7f22977fe700 1 -- 172.21.15.45:0/926259128 learned_addr learned my addr 172.21.15.45:0/926259128 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f2295ffb700 1 -- 172.21.15.45:0/926259128 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798571571 0 0) 0x7f22980ce690 con 0x7f22980cc8b0 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f2295ffb700 1 -- 172.21.15.45:0/926259128 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2280003570 con 0x7f22980cc8b0 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f22967fc700 1 -- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2298153620 msgr2=0x7f229815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f22967fc700 1 --2- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2298153620 0x7f229815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f22967fc700 1 -- 172.21.15.45:0/926259128 >> v1:172.21.15.144:6789/0 conn(0x7f22980cc8b0 legacy=0x7f22980ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f22967fc700 1 -- 172.21.15.45:0/926259128 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229815d510 con 0x7f22981539f0 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f2296ffd700 1 --2- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2298153620 0x7f229815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f22967fc700 1 --2- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981539f0 0x7f229815cdc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f228c0098e0 tx=0x7f228c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927906bbc5708822 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f2295ffb700 1 -- 172.21.15.45:0/926259128 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011630 con 0x7f22981539f0 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f2295ffb700 1 -- 172.21.15.45:0/926259128 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c011790 con 0x7f22981539f0 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.046+0000 7f2295ffb700 1 -- 172.21.15.45:0/926259128 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011b30 con 0x7f22981539f0 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 -- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981539f0 msgr2=0x7f229815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 --2- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981539f0 0x7f229815cdc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f228c0098e0 tx=0x7f228c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 -- 172.21.15.45:0/926259128 shutdown_connections 2023-10-06T05:24:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 --2- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2298153620 0x7f229815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 --2- 172.21.15.45:0/926259128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22981539f0 0x7f229815cdc0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 -- 172.21.15.45:0/926259128 >> 172.21.15.45:0/926259128 conn(0x7f229806ed50 msgr2=0x7f229806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 -- 172.21.15.45:0/926259128 shutdown_connections 2023-10-06T05:24:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.047+0000 7f229d637700 1 -- 172.21.15.45:0/926259128 wait complete. 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 Processor -- start 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 -- start start 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980cc8b0 0x7f229814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298153620 0x7f22980b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980b5960 0x7f22980cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22980b5de0 con 0x7f22980cc8b0 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229815b8c0 con 0x7f2298153620 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.049+0000 7f229d637700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229815c6e0 con 0x7f22980b5960 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980cc8b0 0x7f229814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980cc8b0 0x7f229814fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34480/0 (socket says 172.21.15.45:34480) 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 -- 172.21.15.45:0/55107305 learned_addr learned my addr 172.21.15.45:0/55107305 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f22977fe700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980b5960 0x7f22980cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f22967fc700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298153620 0x7f22980b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 -- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980b5960 msgr2=0x7f22980cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980b5960 0x7f22980cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 -- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298153620 msgr2=0x7f22980b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298153620 0x7f22980b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 -- 172.21.15.45:0/55107305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288009420 con 0x7f22980cc8b0 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f2296ffd700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980cc8b0 0x7f229814fad0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f228800ec80 tx=0x7f22880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f227effd700 1 -- 172.21.15.45:0/55107305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800c920 con 0x7f22980cc8b0 2023-10-06T05:24:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f227effd700 1 -- 172.21.15.45:0/55107305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228800ca80 con 0x7f22980cc8b0 2023-10-06T05:24:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f227effd700 1 -- 172.21.15.45:0/55107305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800ce00 con 0x7f22980cc8b0 2023-10-06T05:24:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c009420 con 0x7f22980cc8b0 2023-10-06T05:24:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.050+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229815bdb0 con 0x7f22980cc8b0 2023-10-06T05:24:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.051+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2298088a60 con 0x7f22980cc8b0 2023-10-06T05:24:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.052+0000 7f227effd700 1 -- 172.21.15.45:0/55107305 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2288013680 con 0x7f22980cc8b0 2023-10-06T05:24:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.053+0000 7f227effd700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.053+0000 7f227effd700 1 -- 172.21.15.45:0/55107305 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7f22880857c0 con 0x7f22980cc8b0 2023-10-06T05:24:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.053+0000 7f22967fc700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.054+0000 7f22967fc700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f228c0098e0 tx=0x7f228c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.057+0000 7f227effd700 1 -- 172.21.15.45:0/55107305 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2288051e90 con 0x7f22980cc8b0 2023-10-06T05:24:52.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.345+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f22980714e0 con 0x7f22980cc8b0 2023-10-06T05:24:52.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.896+0000 7f227effd700 1 -- 172.21.15.45:0/55107305 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v251) v1 ==== 101+0+0 (secure 0 0 0) 0x7f2288017090 con 0x7f22980cc8b0 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.898+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 msgr2=0x7f22840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f228c0098e0 tx=0x7f228c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980cc8b0 msgr2=0x7f229814fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980cc8b0 0x7f229814fad0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f228800ec80 tx=0x7f22880091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 shutdown_connections 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22980b5960 0x7f22980cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22980cc8b0 0x7f229814fad0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 --2- 172.21.15.45:0/55107305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298153620 0x7f22980b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 >> 172.21.15.45:0/55107305 conn(0x7f229806ed50 msgr2=0x7f2298151d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 shutdown_connections 2023-10-06T05:24:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.899+0000 7f229d637700 1 -- 172.21.15.45:0/55107305 wait complete. 2023-10-06T05:24:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:24:52.916 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:24:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.961+0000 7fcfe4bd4040 1 Processor -- start 2023-10-06T05:24:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.961+0000 7fcfe4bd4040 1 -- start start 2023-10-06T05:24:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfe4bd4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791a65fe0 0x558791a6a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfe4bd4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791aab5f0 0x558791b3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfe4bd4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791aada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfe4bd4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587919b2980 con 0x558791b27970 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfe4bd4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587919b2620 con 0x558791aab5f0 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfe4bd4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587919a9400 con 0x558791a65fe0 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791aada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfdd5f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791aab5f0 0x558791b3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfdddf2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791a65fe0 0x558791a6a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791aada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34486/0 (socket says 172.21.15.45:34486) 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 -- 172.21.15.45:0/398626266 learned_addr learned my addr 172.21.15.45:0/398626266 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 -- 172.21.15.45:0/398626266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791a65fe0 msgr2=0x558791a6a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791a65fe0 0x558791a6a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 -- 172.21.15.45:0/398626266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791aab5f0 msgr2=0x558791b3cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791aab5f0 0x558791b3cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfde5f3700 1 -- 172.21.15.45:0/398626266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8005cf0 con 0x558791b27970 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.962+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791a65fe0 0x558791a6a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791aada80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8001580 tx=0x7fcfd8006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abc6d3fc65fc1fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfd7fff700 1 -- 172.21.15.45:0/398626266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8006320 con 0x558791b27970 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfd7fff700 1 -- 172.21.15.45:0/398626266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8001ee0 con 0x558791b27970 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfd7fff700 1 -- 172.21.15.45:0/398626266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8013420 con 0x558791b27970 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/398626266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 msgr2=0x558791aada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791aada80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8001580 tx=0x7fcfd8006b10 comp rx=0 tx=0).stop 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/398626266 shutdown_connections 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791aada80 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791a65fe0 0x558791a6a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/398626266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791aab5f0 0x558791b3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/398626266 >> 172.21.15.45:0/398626266 conn(0x558791a684a0 msgr2=0x558791a93cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.963+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/398626266 shutdown_connections 2023-10-06T05:24:52.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.964+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/398626266 wait complete. 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.964+0000 7fcfe4bd4040 1 Processor -- start 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.964+0000 7fcfe4bd4040 1 -- start start 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfe4bd4040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791a65fe0 0x558791a960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfe4bd4040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791aab5f0 0x558791b1e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfe4bd4040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791b3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfe4bd4040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558791b3f3b0 con 0x558791b27970 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfe4bd4040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558791b3f650 con 0x558791a65fe0 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfe4bd4040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x558791b3f8f0 con 0x558791aab5f0 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfde5f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791b3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfde5f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791b3b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34494/0 (socket says 172.21.15.45:34494) 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 learned_addr learned my addr 172.21.15.45:0/3484724057 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791a65fe0 0x558791a960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfdd5f1700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791aab5f0 0x558791b1e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791aab5f0 msgr2=0x558791b1e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558791aab5f0 0x558791b1e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791a65fe0 msgr2=0x558791a960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.965+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558791a65fe0 0x558791a960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.966+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80094d0 con 0x558791b27970 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.966+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791b3b4b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800f000 tx=0x7fcfd8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.966+0000 7fcfd67fc700 1 -- 172.21.15.45:0/3484724057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8006600 con 0x558791b27970 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.966+0000 7fcfd67fc700 1 -- 172.21.15.45:0/3484724057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8003660 con 0x558791b27970 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.966+0000 7fcfd67fc700 1 -- 172.21.15.45:0/3484724057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8003a20 con 0x558791b27970 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.966+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5587919b4750 con 0x558791b27970 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.966+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd0005db0 con 0x558791b27970 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.967+0000 7fcfd67fc700 1 -- 172.21.15.45:0/3484724057 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfd80123c0 con 0x558791b27970 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.968+0000 7fcfd67fc700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcfc00658c0 0x7fcfc0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.968+0000 7fcfd67fc700 1 -- 172.21.15.45:0/3484724057 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fcfd8085070 con 0x558791b27970 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.968+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcfc00658c0 0x7fcfc0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:24:52.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.968+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x558791b1eec0 0x558791b402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.968+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791ab0f60 con 0x558791b1eec0 2023-10-06T05:24:52.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.968+0000 7fcfdd5f1700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x558791b1eec0 0x558791b402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.969+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcfc00658c0 0x7fcfc0067d70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005cc0 tx=0x7fcfc800c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.969+0000 7fcfdd5f1700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x558791b1eec0 0x558791b402a0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.971+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v118'784 uv758 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.971+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b3aef0 con 0x558791b1eec0 2023-10-06T05:24:52.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.972+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v118'785 uv785 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.972+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5b060 con 0x558791b1eec0 2023-10-06T05:24:52.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.974+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v118'786 uv786 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.975 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:24:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.974+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5c420 con 0x558791b1eec0 2023-10-06T05:24:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.975+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'787 uv787 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.975+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5c7d0 con 0x558791b1eec0 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.977+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'788 uv788 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.977+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5b410 con 0x558791b1eec0 2023-10-06T05:24:52.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.978+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'789 uv789 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.978+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5b7f0 con 0x558791b1eec0 2023-10-06T05:24:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.980+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v118'790 uv764 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.980+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bbf0 con 0x558791b1eec0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.981+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v118'791 uv791 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.981+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bbf0 con 0x558791b1eec0 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.982+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v118'792 uv792 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.982+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.984+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'793 uv793 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.984+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.985+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'794 uv794 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.985+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.986+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'795 uv795 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.986+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.987+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v118'796 uv771 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.987+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.989+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'797 uv797 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.989+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.990+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v118'798 uv798 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.992 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:24:52.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.990+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.992+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'799 uv799 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.992+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.993+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'800 uv800 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.993+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bba0 con 0x558791b1eec0 2023-10-06T05:24:52.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.994+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'801 uv801 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.994+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5bf80 con 0x558791b1eec0 2023-10-06T05:24:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.996+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'802 uv802 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.996+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.998+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v118'803 uv777 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:52.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.998+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.999+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v118'804 uv804 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:52.999+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.001 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:24:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.000+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v118'805 uv805 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.000+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.001+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'806 uv806 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.001+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.003+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'807 uv807 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.003+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.004+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'808 uv808 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.004+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.005+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v118'809 uv783 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.005+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.006+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v118'810 uv810 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.006+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.009 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:24:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.008+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v118'811 uv811 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.008+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.009+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'812 uv812 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.009+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.010+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'813 uv813 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.010+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5be10 con 0x558791b1eec0 2023-10-06T05:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.011+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'814 uv814 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b1eec0 2023-10-06T05:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.011+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558791b5cb80 0x558791b5ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.011+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5f4a0 con 0x558791b5cb80 2023-10-06T05:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.012+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558791b5cb80 0x558791b5ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.012+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558791b5cb80 0x558791b5ef60 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.014+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'566 uv548 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.014+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5f850 con 0x558791b5cb80 2023-10-06T05:24:53.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.015+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'567 uv567 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.015+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b5fc00 con 0x558791b5cb80 2023-10-06T05:24:53.018 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:24:53.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.017+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'568 uv568 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.017+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b604a0 con 0x558791b5cb80 2023-10-06T05:24:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.018+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v118'569 uv569 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.018+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60850 con 0x558791b5cb80 2023-10-06T05:24:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.020+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'570 uv570 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.020+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.021+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v118'571 uv553 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.021+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.023+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v118'572 uv572 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.023+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.025 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:24:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.024+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v118'573 uv573 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.024+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.025+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'574 uv574 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.025+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.027+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'575 uv575 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.027+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.028+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v118'576 uv565 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.028+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.029+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v118'577 uv577 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.029+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.033 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:24:53.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.031+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v118'578 uv578 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.031+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64320 con 0x558791b5cb80 2023-10-06T05:24:53.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.032+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v118'579 uv579 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.032+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64320 con 0x558791b5cb80 2023-10-06T05:24:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.034+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'580 uv580 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.034+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.035+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v118'581 uv564 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.035+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.036+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v118'582 uv582 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.036+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.039 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:24:53.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.038+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v118'583 uv583 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.038+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b61c10 con 0x558791b5cb80 2023-10-06T05:24:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.039+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'584 uv584 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.039+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b61c10 con 0x558791b5cb80 2023-10-06T05:24:53.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.040+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'585 uv585 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.041+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60c00 con 0x558791b5cb80 2023-10-06T05:24:53.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.042+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'586 uv586 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.042+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558791b61110 0x558791b634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.042+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a30 con 0x558791b61110 2023-10-06T05:24:53.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.042+0000 7fcfdd5f1700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558791b61110 0x558791b634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.043+0000 7fcfdd5f1700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558791b61110 0x558791b634f0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.044+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.044+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a30 con 0x558791b61110 2023-10-06T05:24:53.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.045+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v118'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.046+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a30 con 0x558791b61110 2023-10-06T05:24:53.048 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:24:53.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.047+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v118'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.047+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64aa0 con 0x558791b61110 2023-10-06T05:24:53.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.048+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.048+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64a60 con 0x558791b61110 2023-10-06T05:24:53.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.049+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.049+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64a60 con 0x558791b61110 2023-10-06T05:24:53.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.050+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.050+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64a60 con 0x558791b61110 2023-10-06T05:24:53.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.052+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'108 uv107 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.052+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a30 con 0x558791b61110 2023-10-06T05:24:53.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.053+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'109 uv109 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.053+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a50 con 0x558791b61110 2023-10-06T05:24:53.056 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:24:53.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.055+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'110 uv110 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.055+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a50 con 0x558791b61110 2023-10-06T05:24:53.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.056+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'111 uv111 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.056+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a50 con 0x558791b5cb80 2023-10-06T05:24:53.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.057+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.057+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a50 con 0x558791b5cb80 2023-10-06T05:24:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.059+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v118'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.059+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a50 con 0x558791b5cb80 2023-10-06T05:24:53.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.060+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v118'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.062 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:24:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.060+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64e90 con 0x558791b5cb80 2023-10-06T05:24:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.061+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.061+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b64e90 con 0x558791b5cb80 2023-10-06T05:24:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.062+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.062+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a30 con 0x558791b5cb80 2023-10-06T05:24:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.063+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b5cb80 2023-10-06T05:24:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.063+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x558791b64310 0x558791b66ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.063+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b63a30 con 0x558791b64310 2023-10-06T05:24:53.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.064+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x558791b64310 0x558791b66ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.064+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x558791b64310 0x558791b66ea0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.065+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v118'282 uv276 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.066+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b673e0 con 0x558791b64310 2023-10-06T05:24:53.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.067+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v118'283 uv283 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.067+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b677d0 con 0x558791b64310 2023-10-06T05:24:53.070 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:24:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.068+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v118'284 uv284 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.068+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b60090 con 0x558791b64310 2023-10-06T05:24:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.069+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'285 uv285 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.069+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b691d0 con 0x558791b64310 2023-10-06T05:24:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.070+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'286 uv286 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.071+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b680c0 con 0x558791b64310 2023-10-06T05:24:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.072+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'287 uv287 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.072+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b680c0 con 0x558791b64310 2023-10-06T05:24:53.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.073+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v118'288 uv281 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.073+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b680c0 con 0x558791b64310 2023-10-06T05:24:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.074+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v118'289 uv289 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.074+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b680c0 con 0x558791b64310 2023-10-06T05:24:53.077 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:24:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.075+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v118'290 uv290 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.075+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b680c0 con 0x558791b64310 2023-10-06T05:24:53.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.077+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.077+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b68620 con 0x558791b64310 2023-10-06T05:24:53.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.078+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.078+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558791b69630 0x558791b6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.078+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b68620 con 0x558791b69630 2023-10-06T05:24:53.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.078+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558791b69630 0x558791b6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.079+0000 7fcfdddf2700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558791b69630 0x558791b6ba10 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.080+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v118'127 uv126 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b69630 2023-10-06T05:24:53.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.080+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b68620 con 0x558791b69630 2023-10-06T05:24:53.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.081+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v118'128 uv128 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b69630 2023-10-06T05:24:53.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.081+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b68620 con 0x558791b69630 2023-10-06T05:24:53.086 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:24:53.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.084+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v118'129 uv129 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b69630 2023-10-06T05:24:53.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.084+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b68620 con 0x558791b69630 2023-10-06T05:24:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.087+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'130 uv130 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b69630 2023-10-06T05:24:53.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.087+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6bfa0 con 0x558791b69630 2023-10-06T05:24:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.092+0000 7fcfdddf2700 1 -- 172.21.15.45:0/3484724057 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'131 uv131 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfc8012040 con 0x558791b69630 2023-10-06T05:24:53.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.092+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6bfa0 con 0x558791b61110 2023-10-06T05:24:53.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.093+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.093+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6bfa0 con 0x558791b61110 2023-10-06T05:24:53.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.095+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v118'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.095+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6bfa0 con 0x558791b61110 2023-10-06T05:24:53.097 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:24:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.096+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v118'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.096+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6cfb0 con 0x558791b61110 2023-10-06T05:24:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.097+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.097+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6cfb0 con 0x558791b61110 2023-10-06T05:24:53.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.098+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.098+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6bfa0 con 0x558791b61110 2023-10-06T05:24:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.100+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.100+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558791b6c4b0 0x558791b6e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.100+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6ee20 con 0x558791b6c4b0 2023-10-06T05:24:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.100+0000 7fcfdd5f1700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558791b6c4b0 0x558791b6e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.100+0000 7fcfdd5f1700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558791b6c4b0 0x558791b6e890 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.102+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v118'253 uv247 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.102+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.103+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v118'254 uv254 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.103+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.105 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:24:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.104+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v118'255 uv255 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.104+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.105+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'256 uv256 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.105+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.107+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'257 uv257 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.107+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.108+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v118'258 uv252 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.108+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.109+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v118'259 uv259 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.109+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.113 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:24:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.110+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v118'260 uv260 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.110+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f200 con 0x558791b6c4b0 2023-10-06T05:24:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.112+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'261 uv261 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.112+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b71f70 con 0x558791b6c4b0 2023-10-06T05:24:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.113+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'262 uv262 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b6c4b0 2023-10-06T05:24:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.113+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b61110 2023-10-06T05:24:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.114+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v118'111 uv110 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.114+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b61110 2023-10-06T05:24:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.115+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'112 uv112 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.115+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b61110 2023-10-06T05:24:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.117+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v118'113 uv113 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.118 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:24:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.117+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b61110 2023-10-06T05:24:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.118+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'114 uv114 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.118+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558791b70000 0x558791b726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.118+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b70000 2023-10-06T05:24:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.118+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558791b70000 0x558791b726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.119+0000 7fcfde5f3700 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558791b70000 0x558791b726c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.119+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b70000 2023-10-06T05:24:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.120+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b70000 2023-10-06T05:24:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.121+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v118'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b70000 2023-10-06T05:24:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.121+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b70000 2023-10-06T05:24:53.123 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:24:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.122+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v118'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b70000 2023-10-06T05:24:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.122+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b70000 2023-10-06T05:24:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.123+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b70000 2023-10-06T05:24:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.123+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b70000 2023-10-06T05:24:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.124+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b70000 2023-10-06T05:24:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.124+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b70000 2023-10-06T05:24:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.125+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b70000 2023-10-06T05:24:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.125+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b64310 2023-10-06T05:24:53.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.126+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v118'130 uv129 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.126+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b64310 2023-10-06T05:24:53.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.127+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v118'131 uv131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.127+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b64310 2023-10-06T05:24:53.130 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:24:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.129+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v118'132 uv132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.129+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b64310 2023-10-06T05:24:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.130+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'133 uv133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.130+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b64310 2023-10-06T05:24:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.131+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcfd80537a0 con 0x558791b64310 2023-10-06T05:24:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.131+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b61110 2023-10-06T05:24:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.133+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v118'314 uv306 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.133+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6f0 con 0x558791b61110 2023-10-06T05:24:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.134+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v118'315 uv315 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.134+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6c0 con 0x558791b61110 2023-10-06T05:24:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.135+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v118'316 uv316 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.137 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:24:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.135+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6c0 con 0x558791b61110 2023-10-06T05:24:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.136+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v118'317 uv317 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.136+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b72c50 con 0x558791b61110 2023-10-06T05:24:53.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.138+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'318 uv318 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.138+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b72c00 con 0x558791b61110 2023-10-06T05:24:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.139+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v118'319 uv313 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.139+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6c0 con 0x558791b61110 2023-10-06T05:24:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.140+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v118'320 uv320 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.140+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6c0 con 0x558791b61110 2023-10-06T05:24:53.143 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:24:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.142+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v118'321 uv321 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.142+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6c0 con 0x558791b61110 2023-10-06T05:24:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.143+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'322 uv322 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.143+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6a0 con 0x558791b61110 2023-10-06T05:24:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.144+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'323 uv323 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.144+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b6f6a0 con 0x558791b61110 2023-10-06T05:24:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.145+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'324 uv324 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.145+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x558791b72c00 con 0x558791b61110 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfdd5f1700 1 -- 172.21.15.45:0/3484724057 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'325 uv325 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcfd00089d0 con 0x558791b61110 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558791b61110 msgr2=0x558791b634f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x558791b61110 0x558791b634f0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x558791b1eec0 msgr2=0x558791b402a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x558791b1eec0 0x558791b402a0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558791b6c4b0 msgr2=0x558791b6e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x558791b6c4b0 0x558791b6e890 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558791b5cb80 msgr2=0x558791b5ef60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x558791b5cb80 0x558791b5ef60 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558791b70000 msgr2=0x558791b726c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 reap_dead start 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x558791b70000 0x558791b726c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558791b69630 msgr2=0x558791b6ba10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x558791b69630 0x558791b6ba10 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x558791b64310 msgr2=0x558791b66ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x558791b64310 0x558791b66ea0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcfc00658c0 msgr2=0x7fcfc0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcfc00658c0 0x7fcfc0067d70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005cc0 tx=0x7fcfc800c040 comp rx=0 tx=0).stop 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 msgr2=0x558791b3b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.146+0000 7fcfe4bd4040 1 --2- 172.21.15.45:0/3484724057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558791b27970 0x558791b3b4b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800f000 tx=0x7fcfd8014000 comp rx=0 tx=0).stop 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.147+0000 7fcfde5f3700 1 -- 172.21.15.45:0/3484724057 reap_dead start 2023-10-06T05:24:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.147+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 shutdown_connections 2023-10-06T05:24:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.147+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 >> 172.21.15.45:0/3484724057 conn(0x558791a684a0 msgr2=0x558791ab1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.147+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 shutdown_connections 2023-10-06T05:24:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.147+0000 7fcfe4bd4040 1 -- 172.21.15.45:0/3484724057 wait complete. 2023-10-06T05:24:53.152 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2023-10-06T05:24:53.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.241+0000 7fe72c5e0040 1 Processor -- start 2023-10-06T05:24:53.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.241+0000 7fe72c5e0040 1 -- start start 2023-10-06T05:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe72c5e0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f95bc210 0x5607f95c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe72c5e0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f9603320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe72c5e0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f967ec60 0x5607f967cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe72c5e0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f9509980 con 0x5607f95bc210 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe72c5e0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f9509620 con 0x5607f967e890 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe72c5e0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607f9500400 con 0x5607f967ec60 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f95bc210 0x5607f95c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f95bc210 0x5607f95c0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34514/0 (socket says 172.21.15.45:34514) 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 -- 172.21.15.45:0/2527773160 learned_addr learned my addr 172.21.15.45:0/2527773160 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe725fff700 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f967ec60 0x5607f967cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe724ffd700 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f9603320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 -- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f967ec60 msgr2=0x5607f967cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f967ec60 0x5607f967cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 -- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 msgr2=0x5607f9603320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f9603320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe7257fe700 1 -- 172.21.15.45:0/2527773160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c005cf0 con 0x5607f95bc210 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe725fff700 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f967ec60 0x5607f967cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.242+0000 7fe724ffd700 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f9603320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe7257fe700 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f95bc210 0x5607f95c0310 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe71800ac10 tx=0x7fe718015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212adec90665e66c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe7177fe700 1 -- 172.21.15.45:0/2527773160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718000bc0 con 0x5607f95bc210 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe7177fe700 1 -- 172.21.15.45:0/2527773160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7180078e0 con 0x5607f95bc210 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe7177fe700 1 -- 172.21.15.45:0/2527773160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718000d20 con 0x5607f95bc210 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 -- 172.21.15.45:0/2527773160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f95bc210 msgr2=0x5607f95c0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f95bc210 0x5607f95c0310 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe71800ac10 tx=0x7fe718015000 comp rx=0 tx=0).stop 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 -- 172.21.15.45:0/2527773160 shutdown_connections 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f967ec60 0x5607f967cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f95bc210 0x5607f95c0310 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/2527773160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f9603320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 -- 172.21.15.45:0/2527773160 >> 172.21.15.45:0/2527773160 conn(0x5607f95bf450 msgr2=0x5607f95eca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 -- 172.21.15.45:0/2527773160 shutdown_connections 2023-10-06T05:24:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.243+0000 7fe72c5e0040 1 -- 172.21.15.45:0/2527773160 wait complete. 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.244+0000 7fe72c5e0040 1 Processor -- start 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.244+0000 7fe72c5e0040 1 -- start start 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe72c5e0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f95bc210 0x5607f96a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe72c5e0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f95e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe72c5e0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f967ec60 0x5607f95e2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe72c5e0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f9679530 con 0x5607f967ec60 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe72c5e0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f95eda70 con 0x5607f967e890 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe72c5e0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607f95e1500 con 0x5607f95bc210 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe725fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f967ec60 0x5607f95e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe725fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f967ec60 0x5607f95e2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34530/0 (socket says 172.21.15.45:34530) 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe725fff700 1 -- 172.21.15.45:0/3863511825 learned_addr learned my addr 172.21.15.45:0/3863511825 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe7257fe700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f95bc210 0x5607f96a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.245+0000 7fe724ffd700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f95e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe7257fe700 1 -- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 msgr2=0x5607f95e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe7257fe700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607f967e890 0x5607f95e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe7257fe700 1 -- 172.21.15.45:0/3863511825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f967ec60 msgr2=0x5607f95e2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe7257fe700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f967ec60 0x5607f95e2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe7257fe700 1 -- 172.21.15.45:0/3863511825 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607f95321c0 con 0x5607f95bc210 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe725fff700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607f967ec60 0x5607f95e2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe7257fe700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f95bc210 0x5607f96a57d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe718020000 tx=0x7fe71801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe715ffb700 1 -- 172.21.15.45:0/3863511825 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718005630 con 0x5607f95bc210 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe715ffb700 1 -- 172.21.15.45:0/3863511825 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe718000bc0 con 0x5607f95bc210 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe718005cf0 con 0x5607f95bc210 2023-10-06T05:24:53.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe715ffb700 1 -- 172.21.15.45:0/3863511825 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71802d650 con 0x5607f95bc210 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.246+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607f95183d0 con 0x5607f95bc210 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.248+0000 7fe715ffb700 1 -- 172.21.15.45:0/3863511825 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe718019070 con 0x5607f95bc210 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.248+0000 7fe715ffb700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe708065890 0x7fe708067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.248+0000 7fe715ffb700 1 -- 172.21.15.45:0/3863511825 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fe718029070 con 0x5607f95bc210 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.248+0000 7fe724ffd700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe708065890 0x7fe708067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.248+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5607f9698e40 0x5607f967c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.248+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.15 14:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) v8 -- 0x5607f96a5d10 con 0x5607f9698e40 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.248+0000 7fe725fff700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5607f9698e40 0x5607f967c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.249+0000 7fe724ffd700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe708065890 0x7fe708067d40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005cc0 tx=0x7fe71c005a50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.249+0000 7fe725fff700 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5607f9698e40 0x5607f967c0b0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe725fff700 1 -- 172.21.15.45:0/3863511825 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v118'293 uv293 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe720001ed0 con 0x5607f9698e40 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5607f9698e40 msgr2=0x5607f967c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5607f9698e40 0x5607f967c0b0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe708065890 msgr2=0x7fe708067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe708065890 0x7fe708067d40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005cc0 tx=0x7fe71c005a50 comp rx=0 tx=0).stop 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f95bc210 msgr2=0x5607f96a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 --2- 172.21.15.45:0/3863511825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607f95bc210 0x5607f96a57d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe718020000 tx=0x7fe71801c000 comp rx=0 tx=0).stop 2023-10-06T05:24:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe725fff700 1 -- 172.21.15.45:0/3863511825 reap_dead start 2023-10-06T05:24:53.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 shutdown_connections 2023-10-06T05:24:53.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 >> 172.21.15.45:0/3863511825 conn(0x5607f95bf450 msgr2=0x5607f9602550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 shutdown_connections 2023-10-06T05:24:53.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.250+0000 7fe72c5e0040 1 -- 172.21.15.45:0/3863511825 wait complete. 2023-10-06T05:24:53.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:24:53.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.393+0000 7f97629b1700 1 Processor -- start 2023-10-06T05:24:53.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.393+0000 7f97629b1700 1 -- start start 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f97629b1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f975c159150 0x7f975c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f97629b1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f975c159520 0x7f975c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f97629b1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f975c0c9c90 con 0x7f975c15d3d0 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f97629b1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c15d8e0 con 0x7f975c159150 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f97629b1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c15dbc0 con 0x7f975c159520 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f975bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f975c159150 0x7f975c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f9760f4e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f975c15d3d0 0x7f975c067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47566/0 (socket says 172.21.15.45:47566) 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f9760f4e700 1 -- 172.21.15.45:0/2188009334 learned_addr learned my addr 172.21.15.45:0/2188009334 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.394+0000 7f975b7fe700 1 --2- 172.21.15.45:0/2188009334 >> v2:172.21.15.144:3300/0 conn(0x7f975c159520 0x7f975c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975bfff700 1 -- 172.21.15.45:0/2188009334 >> v2:172.21.15.144:3300/0 conn(0x7f975c159520 msgr2=0x7f975c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975bfff700 1 --2- 172.21.15.45:0/2188009334 >> v2:172.21.15.144:3300/0 conn(0x7f975c159520 0x7f975c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975bfff700 1 -- 172.21.15.45:0/2188009334 >> v1:172.21.15.45:6789/0 conn(0x7f975c15d3d0 legacy=0x7f975c067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975bfff700 1 -- 172.21.15.45:0/2188009334 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c0c9920 con 0x7f975c159150 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975affd700 1 -- 172.21.15.45:0/2188009334 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882558497 0 0) 0x7f975c0c9c90 con 0x7f975c15d3d0 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975bfff700 1 --2- 172.21.15.45:0/2188009334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c0ca120 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f975c0b2e40 tx=0x7f974c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c48738b8013908d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975affd700 1 -- 172.21.15.45:0/2188009334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011630 con 0x7f975c159150 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975affd700 1 -- 172.21.15.45:0/2188009334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c011790 con 0x7f975c159150 2023-10-06T05:24:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.395+0000 7f975affd700 1 -- 172.21.15.45:0/2188009334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011b30 con 0x7f975c159150 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 -- 172.21.15.45:0/2188009334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 msgr2=0x7f975c0ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 --2- 172.21.15.45:0/2188009334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c0ca120 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f975c0b2e40 tx=0x7f974c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 -- 172.21.15.45:0/2188009334 shutdown_connections 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 --2- 172.21.15.45:0/2188009334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c0ca120 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 --2- 172.21.15.45:0/2188009334 >> v2:172.21.15.144:3300/0 conn(0x7f975c159520 0x7f975c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 -- 172.21.15.45:0/2188009334 >> 172.21.15.45:0/2188009334 conn(0x7f975c0691c0 msgr2=0x7f975c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 -- 172.21.15.45:0/2188009334 shutdown_connections 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.396+0000 7f97629b1700 1 -- 172.21.15.45:0/2188009334 wait complete. 2023-10-06T05:24:53.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.397+0000 7f97629b1700 1 Processor -- start 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.397+0000 7f97629b1700 1 -- start start 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f97629b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c14a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f97629b1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f975c15d3d0 0x7f975c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f97629b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f975c159900 0x7f975c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f97629b1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c169790 con 0x7f975c159150 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f97629b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c14ae60 con 0x7f975c159900 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f97629b1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0c6e00 con 0x7f975c15d3d0 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c14a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c14a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34550/0 (socket says 172.21.15.45:34550) 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 -- 172.21.15.45:0/2730738983 learned_addr learned my addr 172.21.15.45:0/2730738983 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:53.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 -- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f975c15d3d0 msgr2=0x7f975c15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975b7fe700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f975c15d3d0 0x7f975c15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f975c15d3d0 0x7f975c15a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f9760f4e700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f975c159900 0x7f975c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 -- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f975c159900 msgr2=0x7f975c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f975c159900 0x7f975c15add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.398+0000 7f975bfff700 1 -- 172.21.15.45:0/2730738983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c009420 con 0x7f975c159150 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.399+0000 7f975bfff700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c14a920 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f975c06c4b0 tx=0x7f974c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.399+0000 7f9753fff700 1 -- 172.21.15.45:0/2730738983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011b10 con 0x7f975c159150 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.399+0000 7f9753fff700 1 -- 172.21.15.45:0/2730738983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c0233f0 con 0x7f975c159150 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.399+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c0c3870 con 0x7f975c159150 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.399+0000 7f9753fff700 1 -- 172.21.15.45:0/2730738983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c0237b0 con 0x7f975c159150 2023-10-06T05:24:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.399+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c0c6040 con 0x7f975c159150 2023-10-06T05:24:53.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.400+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c06e2d0 con 0x7f975c159150 2023-10-06T05:24:53.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.400+0000 7f9753fff700 1 -- 172.21.15.45:0/2730738983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f974c0193b0 con 0x7f975c159150 2023-10-06T05:24:53.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.401+0000 7f9753fff700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c066f30 0x7f973c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.401+0000 7f9753fff700 1 -- 172.21.15.45:0/2730738983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7f974c086630 con 0x7f975c159150 2023-10-06T05:24:53.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.401+0000 7f975b7fe700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c066f30 0x7f973c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.401+0000 7f975b7fe700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c066f30 0x7f973c0693e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f97440093f0 tx=0x7f9744008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.405+0000 7f9753fff700 1 -- 172.21.15.45:0/2730738983 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f974c052d80 con 0x7f975c159150 2023-10-06T05:24:53.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.651+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f975c0ca660 con 0x7f975c159150 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.897+0000 7f9753fff700 1 -- 172.21.15.45:0/2730738983 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v252) v1 ==== 162+0+0 (secure 0 0 0) 0x7f974c0280b0 con 0x7f975c159150 2023-10-06T05:24:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c066f30 msgr2=0x7f973c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c066f30 0x7f973c0693e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f97440093f0 tx=0x7f9744008040 comp rx=0 tx=0).stop 2023-10-06T05:24:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 msgr2=0x7f975c14a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c14a920 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f975c06c4b0 tx=0x7f974c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 shutdown_connections 2023-10-06T05:24:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c066f30 0x7f973c0693e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f975c159900 0x7f975c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c159150 0x7f975c14a920 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 --2- 172.21.15.45:0/2730738983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f975c15d3d0 0x7f975c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.900+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 >> 172.21.15.45:0/2730738983 conn(0x7f975c0691c0 msgr2=0x7f975c1566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.901+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 shutdown_connections 2023-10-06T05:24:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:53.901+0000 7f97629b1700 1 -- 172.21.15.45:0/2730738983 wait complete. 2023-10-06T05:24:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:24:54.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.461+0000 7fc6d9208700 1 Processor -- start 2023-10-06T05:24:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.461+0000 7fc6d9208700 1 -- start start 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d9208700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc6d40cd340 0x7fc6d4157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d9208700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d415b000 con 0x7fc6d40c93c0 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d9208700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d415b2e0 con 0x7fc6d40c8ff0 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d9208700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d40c9830 con 0x7fc6d40cd340 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc6d40c93c0 0x7fc6d4156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47574/0 (socket says 172.21.15.45:47574) 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d259c700 1 -- 172.21.15.45:0/2587463868 learned_addr learned my addr 172.21.15.45:0/2587463868 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d359e700 1 --2- 172.21.15.45:0/2587463868 >> v2:172.21.15.144:3300/0 conn(0x7fc6d40cd340 0x7fc6d4157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45900601 0 0) 0x7fc6d415b000 con 0x7fc6d40c93c0 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.462+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6bc003570 con 0x7fc6d40c93c0 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090077129 0 0) 0x7fc6bc003570 con 0x7fc6d40c93c0 2023-10-06T05:24:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 >> v2:172.21.15.144:3300/0 conn(0x7fc6d40cd340 msgr2=0x7fc6d4157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 --2- 172.21.15.45:0/2587463868 >> v2:172.21.15.144:3300/0 conn(0x7fc6d40cd340 0x7fc6d4157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 >> v1:172.21.15.144:6789/0 conn(0x7fc6d40c8ff0 legacy=0x7fc6d40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d415b5c0 con 0x7fc6d40c93c0 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fc6c8001470 con 0x7fc6d40c93c0 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 >> v1:172.21.15.45:6789/0 conn(0x7fc6d40c93c0 legacy=0x7fc6d4156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6bc004810 0x7fc6bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6bc0051c0 0x7fc6bc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40c9830 con 0x7fc6bc003a60 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc00c0d0 con 0x7fc6bc0051c0 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc00c290 con 0x7fc6bc004810 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d2d9d700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d259c700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6bc004810 0x7fc6bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d359e700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6bc0051c0 0x7fc6bc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d2d9d700 1 -- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6bc004810 msgr2=0x7fc6bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d2d9d700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6bc004810 0x7fc6bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d2d9d700 1 -- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6bc0051c0 msgr2=0x7fc6bc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d2d9d700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6bc0051c0 0x7fc6bc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d359e700 1 -- 172.21.15.45:0/2587463868 reap_dead start 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d2d9d700 1 -- 172.21.15.45:0/2587463868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc003570 con 0x7fc6bc003a60 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.463+0000 7fc6d2d9d700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6bc003e70 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40018e0 tx=0x7fc6c4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1d14f9ac251251 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4012420 con 0x7fc6bc003a60 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c40125a0 con 0x7fc6bc003a60 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d1d9b700 1 -- 172.21.15.45:0/2587463868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4012960 con 0x7fc6bc003a60 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d9208700 1 -- 172.21.15.45:0/2587463868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 msgr2=0x7fc6bc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d9208700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6bc003e70 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40018e0 tx=0x7fc6c4007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:54.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d9208700 1 -- 172.21.15.45:0/2587463868 shutdown_connections 2023-10-06T05:24:54.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d9208700 1 --2- 172.21.15.45:0/2587463868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6bc003e70 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d9208700 1 -- 172.21.15.45:0/2587463868 >> 172.21.15.45:0/2587463868 conn(0x7fc6d40cdf50 msgr2=0x7fc6d40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:54.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d9208700 1 -- 172.21.15.45:0/2587463868 shutdown_connections 2023-10-06T05:24:54.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.464+0000 7fc6d9208700 1 -- 172.21.15.45:0/2587463868 wait complete. 2023-10-06T05:24:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.465+0000 7fc6d9208700 1 Processor -- start 2023-10-06T05:24:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d9208700 1 -- start start 2023-10-06T05:24:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d9208700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6d40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d9208700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d40ad170 0x7fc6d416a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d9208700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6d40aee30 0x7fc6d40af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d9208700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40af7e0 con 0x7fc6bc003a60 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d9208700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d416a6f0 con 0x7fc6d40ad170 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d9208700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40ad5e0 con 0x7fc6d40aee30 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6d40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6d40b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34560/0 (socket says 172.21.15.45:34560) 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.466+0000 7fc6d2d9d700 1 -- 172.21.15.45:0/149620371 learned_addr learned my addr 172.21.15.45:0/149620371 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d259c700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d40ad170 0x7fc6d416a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d359e700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6d40aee30 0x7fc6d40af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d2d9d700 1 -- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6d40aee30 msgr2=0x7fc6d40af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d2d9d700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6d40aee30 0x7fc6d40af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d2d9d700 1 -- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d40ad170 msgr2=0x7fc6d416a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d2d9d700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d40ad170 0x7fc6d416a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d2d9d700 1 -- 172.21.15.45:0/149620371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d416d150 con 0x7fc6bc003a60 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d359e700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6d40aee30 0x7fc6d40af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d259c700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d40ad170 0x7fc6d416a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d2d9d700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6d40b0cf0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c0e0 tx=0x7fc6c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6baffd700 1 -- 172.21.15.45:0/149620371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c400ca90 con 0x7fc6bc003a60 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6baffd700 1 -- 172.21.15.45:0/149620371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c400cbf0 con 0x7fc6bc003a60 2023-10-06T05:24:54.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d416d430 con 0x7fc6bc003a60 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6baffd700 1 -- 172.21.15.45:0/149620371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4006820 con 0x7fc6bc003a60 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.467+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d416d9f0 con 0x7fc6bc003a60 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.468+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d406f410 con 0x7fc6bc003a60 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.469+0000 7fc6baffd700 1 -- 172.21.15.45:0/149620371 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6c4019030 con 0x7fc6bc003a60 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.469+0000 7fc6baffd700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6c0066f30 0x7fc6c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.469+0000 7fc6baffd700 1 -- 172.21.15.45:0/149620371 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fc6c4086880 con 0x7fc6bc003a60 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.469+0000 7fc6d259c700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6c0066f30 0x7fc6c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.470+0000 7fc6d259c700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6c0066f30 0x7fc6c00693e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8001ce0 tx=0x7fc6c8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.474+0000 7fc6baffd700 1 -- 172.21.15.45:0/149620371 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6c4052f50 con 0x7fc6bc003a60 2023-10-06T05:24:54.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.728+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6d40af7e0 con 0x7fc6bc003a60 2023-10-06T05:24:54.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.728+0000 7fc6baffd700 1 -- 172.21.15.45:0/149620371 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 253 v253) v1 ==== 95+0+4609 (secure 0 0 0) 0x7fc6c402c070 con 0x7fc6bc003a60 2023-10-06T05:24:54.731 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:24:54.731 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":253,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:53.898773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9121":{"gid":9121,"name":"b","rank":0,"incarnation":253,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/4239891954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4239891954},{"type":"v1","addr":"172.21.15.144:6835","nonce":4239891954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6c0066f30 msgr2=0x7fc6c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6c0066f30 0x7fc6c00693e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8001ce0 tx=0x7fc6c8009040 comp rx=0 tx=0).stop 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 msgr2=0x7fc6d40b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6d40b0cf0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c0e0 tx=0x7fc6c4015000 comp rx=0 tx=0).stop 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 shutdown_connections 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc6c0066f30 0x7fc6c00693e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6d40aee30 0x7fc6d40af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6bc003a60 0x7fc6d40b0cf0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 --2- 172.21.15.45:0/149620371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d40ad170 0x7fc6d416a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 >> 172.21.15.45:0/149620371 conn(0x7fc6d40cdf50 msgr2=0x7fc6d40c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:54.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 shutdown_connections 2023-10-06T05:24:54.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:24:54.731+0000 7fc6d9208700 1 -- 172.21.15.45:0/149620371 wait complete. 2023-10-06T05:24:54.734 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 253 2023-10-06T05:24:54.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 253, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:24:53.898773+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9121}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9121': {'gid': 9121, 'name': 'b', 'rank': 0, 'incarnation': 253, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/4239891954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 4239891954}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 4239891954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:24:54.746 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9121:up:replay 2023-10-06T05:24:54.746 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:25:00.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:00.201 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:00.200+0000 7fb1d8814700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.289+0000 7f5c6cc80700 1 Processor -- start 2023-10-06T05:25:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.289+0000 7f5c6cc80700 1 -- start start 2023-10-06T05:25:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6cc80700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c68153650 0x7f5c6815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6cc80700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c68153a20 0x7f5c6806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6cc80700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c6806add0 0x7f5c68068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6cc80700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6806b250 con 0x7f5c68153650 2023-10-06T05:25:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6cc80700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c680676c0 con 0x7f5c6806add0 2023-10-06T05:25:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6cc80700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680679a0 con 0x7f5c68153a20 2023-10-06T05:25:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c68153650 0x7f5c6815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c68153650 0x7f5c6815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45172/0 (socket says 172.21.15.45:45172) 2023-10-06T05:25:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c6659c700 1 -- 172.21.15.45:0/1456950849 learned_addr learned my addr 172.21.15.45:0/1456950849 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c66d9d700 1 --2- 172.21.15.45:0/1456950849 >> v2:172.21.15.144:3300/0 conn(0x7f5c6806add0 0x7f5c68068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.290+0000 7f5c65d9b700 1 --2- 172.21.15.45:0/1456950849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c68153a20 0x7f5c6806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6659c700 1 -- 172.21.15.45:0/1456950849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c68153a20 msgr2=0x7f5c6806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6659c700 1 --2- 172.21.15.45:0/1456950849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c68153a20 0x7f5c6806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6659c700 1 -- 172.21.15.45:0/1456950849 >> v2:172.21.15.144:3300/0 conn(0x7f5c6806add0 msgr2=0x7f5c68068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6659c700 1 --2- 172.21.15.45:0/1456950849 >> v2:172.21.15.144:3300/0 conn(0x7f5c6806add0 0x7f5c68068480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6659c700 1 -- 172.21.15.45:0/1456950849 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68067c80 con 0x7f5c68153650 2023-10-06T05:25:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c65d9b700 1 --2- 172.21.15.45:0/1456950849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c68153a20 0x7f5c6806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6659c700 1 --2- 172.21.15.45:0/1456950849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153650 0x7f5c6815a930 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5c680c8e10 tx=0x7f5c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60a5a3f4b319b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6559a700 1 -- 172.21.15.45:0/1456950849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58009e70 con 0x7f5c68153650 2023-10-06T05:25:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6559a700 1 -- 172.21.15.45:0/1456950849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5800b860 con 0x7f5c68153650 2023-10-06T05:25:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6559a700 1 -- 172.21.15.45:0/1456950849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800bc00 con 0x7f5c68153650 2023-10-06T05:25:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6cc80700 1 -- 172.21.15.45:0/1456950849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153650 msgr2=0x7f5c6815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.291+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/1456950849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153650 0x7f5c6815a930 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5c680c8e10 tx=0x7f5c58009250 comp rx=0 tx=0).stop 2023-10-06T05:25:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.292+0000 7f5c6cc80700 1 -- 172.21.15.45:0/1456950849 shutdown_connections 2023-10-06T05:25:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.292+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/1456950849 >> v2:172.21.15.144:3300/0 conn(0x7f5c6806add0 0x7f5c68068480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.292+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/1456950849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153650 0x7f5c6815a930 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.292+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/1456950849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c68153a20 0x7f5c6806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.292+0000 7f5c6cc80700 1 -- 172.21.15.45:0/1456950849 >> 172.21.15.45:0/1456950849 conn(0x7f5c680cc5a0 msgr2=0x7f5c680cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.292+0000 7f5c6cc80700 1 -- 172.21.15.45:0/1456950849 shutdown_connections 2023-10-06T05:25:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.292+0000 7f5c6cc80700 1 -- 172.21.15.45:0/1456950849 wait complete. 2023-10-06T05:25:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.293+0000 7f5c6cc80700 1 Processor -- start 2023-10-06T05:25:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.293+0000 7f5c6cc80700 1 -- start start 2023-10-06T05:25:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c6cc80700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c6806add0 0x7f5c681521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c6cc80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c68153650 0x7f5c6815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c6cc80700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153a20 0x7f5c6815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c6cc80700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681513e0 con 0x7f5c68153a20 2023-10-06T05:25:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c6cc80700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681566e0 con 0x7f5c68153650 2023-10-06T05:25:00.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c6cc80700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68155ad0 con 0x7f5c6806add0 2023-10-06T05:25:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153a20 0x7f5c6815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153a20 0x7f5c6815f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45178/0 (socket says 172.21.15.45:45178) 2023-10-06T05:25:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 -- 172.21.15.45:0/3024308064 learned_addr learned my addr 172.21.15.45:0/3024308064 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c65d9b700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c68153650 0x7f5c6815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c6659c700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c6806add0 0x7f5c681521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 -- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c6806add0 msgr2=0x7f5c681521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c6806add0 0x7f5c681521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 -- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c68153650 msgr2=0x7f5c6815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c68153650 0x7f5c6815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 -- 172.21.15.45:0/3024308064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580094f0 con 0x7f5c68153a20 2023-10-06T05:25:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c65d9b700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c68153650 0x7f5c6815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.294+0000 7f5c66d9d700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153a20 0x7f5c6815f430 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009e90 tx=0x7f5c6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.295+0000 7f5c5e7fc700 1 -- 172.21.15.45:0/3024308064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000cc20 con 0x7f5c68153a20 2023-10-06T05:25:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.295+0000 7f5c5e7fc700 1 -- 172.21.15.45:0/3024308064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6000f040 con 0x7f5c68153a20 2023-10-06T05:25:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.295+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c68155db0 con 0x7f5c68153a20 2023-10-06T05:25:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.295+0000 7f5c5e7fc700 1 -- 172.21.15.45:0/3024308064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60014480 con 0x7f5c68153a20 2023-10-06T05:25:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.295+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c68156370 con 0x7f5c68153a20 2023-10-06T05:25:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.296+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c68156870 con 0x7f5c68153a20 2023-10-06T05:25:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.296+0000 7f5c5e7fc700 1 -- 172.21.15.45:0/3024308064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c60014830 con 0x7f5c68153a20 2023-10-06T05:25:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.297+0000 7f5c5e7fc700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c48066f30 0x7f5c480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.297+0000 7f5c5e7fc700 1 -- 172.21.15.45:0/3024308064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7f5c60085b80 con 0x7f5c68153a20 2023-10-06T05:25:00.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.297+0000 7f5c6659c700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c48066f30 0x7f5c480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.297+0000 7f5c6659c700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c48066f30 0x7f5c480693e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5c680c8e10 tx=0x7f5c58017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.301+0000 7f5c5e7fc700 1 -- 172.21.15.45:0/3024308064 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c600521f0 con 0x7f5c68153a20 2023-10-06T05:25:00.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.547+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c680cc0e0 con 0x7f5c68153a20 2023-10-06T05:25:00.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.547+0000 7f5c5e7fc700 1 -- 172.21.15.45:0/3024308064 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 256 v256) v1 ==== 95+0+4609 (secure 0 0 0) 0x7f5c60018090 con 0x7f5c68153a20 2023-10-06T05:25:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:00.550 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":256,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":256,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:24:56.908854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9121":{"gid":9121,"name":"b","rank":0,"incarnation":253,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6835/4239891954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4239891954},{"type":"v1","addr":"172.21.15.144:6835","nonce":4239891954}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:00.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.549+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c48066f30 msgr2=0x7f5c480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.549+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c48066f30 0x7f5c480693e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5c680c8e10 tx=0x7f5c58017000 comp rx=0 tx=0).stop 2023-10-06T05:25:00.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.549+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153a20 msgr2=0x7f5c6815f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.549+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153a20 0x7f5c6815f430 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009e90 tx=0x7f5c6000b040 comp rx=0 tx=0).stop 2023-10-06T05:25:00.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.549+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 shutdown_connections 2023-10-06T05:25:00.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.550+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5c48066f30 0x7f5c480693e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.550+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c68153a20 0x7f5c6815f430 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.550+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c6806add0 0x7f5c681521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.550+0000 7f5c6cc80700 1 --2- 172.21.15.45:0/3024308064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c68153650 0x7f5c6815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.550+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 >> 172.21.15.45:0/3024308064 conn(0x7f5c680cc5a0 msgr2=0x7f5c680cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.550+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 shutdown_connections 2023-10-06T05:25:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.550+0000 7f5c6cc80700 1 -- 172.21.15.45:0/3024308064 wait complete. 2023-10-06T05:25:00.554 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 256 2023-10-06T05:25:00.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 256, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:24:56.908854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9121}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9121': {'gid': 9121, 'name': 'b', 'rank': 0, 'incarnation': 253, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.144:6835/4239891954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 4239891954}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 4239891954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:25:00.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:25:00.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:25:00.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.696+0000 7fd23befd700 1 Processor -- start 2023-10-06T05:25:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.696+0000 7fd23befd700 1 -- start start 2023-10-06T05:25:00.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd23befd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2340c8ff0 0x7fd2340c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd23befd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd2340caad0 0x7fd2340caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd23befd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23415b000 con 0x7fd234156af0 2023-10-06T05:25:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd23befd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd234157630 con 0x7fd2340c8ff0 2023-10-06T05:25:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd23befd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23415b2e0 con 0x7fd2340caad0 2023-10-06T05:25:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd239c99700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2340c8ff0 0x7fd2340c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd23a49a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd234156af0 0x7fd234156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56830/0 (socket says 172.21.15.45:56830) 2023-10-06T05:25:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd23a49a700 1 -- 172.21.15.45:0/81472836 learned_addr learned my addr 172.21.15.45:0/81472836 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd239498700 1 --2- 172.21.15.45:0/81472836 >> v2:172.21.15.144:3300/0 conn(0x7fd2340caad0 0x7fd2340caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd239c99700 1 -- 172.21.15.45:0/81472836 >> v2:172.21.15.144:3300/0 conn(0x7fd2340caad0 msgr2=0x7fd2340caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd239c99700 1 --2- 172.21.15.45:0/81472836 >> v2:172.21.15.144:3300/0 conn(0x7fd2340caad0 0x7fd2340caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd239c99700 1 -- 172.21.15.45:0/81472836 >> v1:172.21.15.45:6789/0 conn(0x7fd234156af0 legacy=0x7fd234156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd239c99700 1 -- 172.21.15.45:0/81472836 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23415b5c0 con 0x7fd2340c8ff0 2023-10-06T05:25:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd238c97700 1 -- 172.21.15.45:0/81472836 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101932262 0 0) 0x7fd23415b000 con 0x7fd234156af0 2023-10-06T05:25:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.697+0000 7fd239498700 1 --2- 172.21.15.45:0/81472836 >> v2:172.21.15.144:3300/0 conn(0x7fd2340caad0 0x7fd2340caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd239c99700 1 --2- 172.21.15.45:0/81472836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2340c93c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd23406ed80 tx=0x7fd2280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca88d1db0ef31a8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd238c97700 1 -- 172.21.15.45:0/81472836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22800b6b0 con 0x7fd2340c8ff0 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd238c97700 1 -- 172.21.15.45:0/81472836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22800b810 con 0x7fd2340c8ff0 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd238c97700 1 -- 172.21.15.45:0/81472836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22800bbb0 con 0x7fd2340c8ff0 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd23befd700 1 -- 172.21.15.45:0/81472836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 msgr2=0x7fd2340c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd23befd700 1 --2- 172.21.15.45:0/81472836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2340c93c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd23406ed80 tx=0x7fd2280091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd23befd700 1 -- 172.21.15.45:0/81472836 shutdown_connections 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd23befd700 1 --2- 172.21.15.45:0/81472836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2340c93c0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd23befd700 1 --2- 172.21.15.45:0/81472836 >> v2:172.21.15.144:3300/0 conn(0x7fd2340caad0 0x7fd2340caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd23befd700 1 -- 172.21.15.45:0/81472836 >> 172.21.15.45:0/81472836 conn(0x7fd2340cdf50 msgr2=0x7fd2340ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.698+0000 7fd23befd700 1 -- 172.21.15.45:0/81472836 shutdown_connections 2023-10-06T05:25:00.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.699+0000 7fd23befd700 1 -- 172.21.15.45:0/81472836 wait complete. 2023-10-06T05:25:00.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 Processor -- start 2023-10-06T05:25:00.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 -- start start 2023-10-06T05:25:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2341681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2340caad0 0x7fd2340b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd234156af0 0x7fd2340b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2340b3c40 con 0x7fd2340c8ff0 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341677b0 con 0x7fd234156af0 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.700+0000 7fd23befd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341686e0 con 0x7fd2340caad0 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2341681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2341681a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45184/0 (socket says 172.21.15.45:45184) 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 -- 172.21.15.45:0/4213988211 learned_addr learned my addr 172.21.15.45:0/4213988211 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd23a49a700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd234156af0 0x7fd2340b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239498700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2340caad0 0x7fd2340b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 -- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2340caad0 msgr2=0x7fd2340b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2340caad0 0x7fd2340b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 -- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd234156af0 msgr2=0x7fd2340b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd234156af0 0x7fd2340b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 -- 172.21.15.45:0/4213988211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd228009420 con 0x7fd2340c8ff0 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239498700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2340caad0 0x7fd2340b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd239c99700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2341681a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd2340c4f80 tx=0x7fd228017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd225ffb700 1 -- 172.21.15.45:0/4213988211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22800bb90 con 0x7fd2340c8ff0 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd225ffb700 1 -- 172.21.15.45:0/4213988211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228011420 con 0x7fd2340c8ff0 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2340b3680 con 0x7fd2340c8ff0 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.701+0000 7fd225ffb700 1 -- 172.21.15.45:0/4213988211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280117a0 con 0x7fd2340c8ff0 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.702+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2341679c0 con 0x7fd2340c8ff0 2023-10-06T05:25:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.703+0000 7fd225ffb700 1 -- 172.21.15.45:0/4213988211 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd228011b90 con 0x7fd2340c8ff0 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.704+0000 7fd225ffb700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd218066f30 0x7fd2180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.704+0000 7fd225ffb700 1 -- 172.21.15.45:0/4213988211 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fd228086a20 con 0x7fd2340c8ff0 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.704+0000 7fd239498700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd218066f30 0x7fd2180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.704+0000 7fd239498700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd218066f30 0x7fd2180693e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd230006f00 tx=0x7fd230008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.705+0000 7fd245211b80 1 Processor -- start 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd245211b80 1 -- start start 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd245211b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d328820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d30df40 0x56191d330f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d331940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd245211b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d1fb310 con 0x56191d328450 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d238130 con 0x56191d30df40 2023-10-06T05:25:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56191d235af0 con 0x56191d3314d0 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd239c99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d328820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd239c99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d328820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45196/0 (socket says 172.21.15.45:45196) 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.706+0000 7fd239c99700 1 -- 172.21.15.45:0/2581600203 learned_addr learned my addr 172.21.15.45:0/2581600203 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd239498700 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d30df40 0x56191d330f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd23a49a700 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d331940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd239c99700 1 -- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 msgr2=0x56191d331940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd239c99700 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d331940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd239c99700 1 -- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d30df40 msgr2=0x56191d330f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd239c99700 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d30df40 0x56191d330f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd239c99700 1 -- 172.21.15.45:0/2581600203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56191d037790 con 0x56191d328450 2023-10-06T05:25:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd239c99700 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d328820 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd22800b160 tx=0x7fd2280842a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c5be49f18ba602 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd21effd700 1 -- 172.21.15.45:0/2581600203 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22805c0b0 con 0x56191d328450 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd21effd700 1 -- 172.21.15.45:0/2581600203 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228057450 con 0x56191d328450 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.707+0000 7fd21effd700 1 -- 172.21.15.45:0/2581600203 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228056d50 con 0x56191d328450 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 -- 172.21.15.45:0/2581600203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 msgr2=0x56191d328820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d328820 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd22800b160 tx=0x7fd2280842a0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 -- 172.21.15.45:0/2581600203 shutdown_connections 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d331940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d328820 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 --2- 172.21.15.45:0/2581600203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d30df40 0x56191d330f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 -- 172.21.15.45:0/2581600203 >> 172.21.15.45:0/2581600203 conn(0x56191d32b880 msgr2=0x56191d32bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 -- 172.21.15.45:0/2581600203 shutdown_connections 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 -- 172.21.15.45:0/2581600203 wait complete. 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 -- start start 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.708+0000 7fd245211b80 1 Processor -- start 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.709+0000 7fd245211b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d30df40 0x56191d305730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.709+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d328450 0x56191d2e1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.709+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d2e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.709+0000 7fd245211b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d2318e0 con 0x56191d30df40 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.709+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d1dd980 con 0x56191d328450 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.709+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56191d1dd620 con 0x56191d3314d0 2023-10-06T05:25:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd245211b80 10 client.0 _ll_register_callbacks cb 0x56191d040a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d30df40 0x56191d305730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d30df40 0x56191d305730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45198/0 (socket says 172.21.15.45:45198) 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 -- 172.21.15.45:0/309574591 learned_addr learned my addr 172.21.15.45:0/309574591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239498700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d328450 0x56191d2e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd23a49a700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d2e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 -- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 msgr2=0x56191d2e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d2e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 -- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d328450 msgr2=0x56191d2e1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d328450 0x56191d2e1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 -- 172.21.15.45:0/309574591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2341679c0 con 0x56191d30df40 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd23a49a700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d2e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd239c99700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d30df40 0x56191d305730 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd22805c010 tx=0x7fd228027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f408cb050dc52b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd213fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd213fff700 1 -- 172.21.15.45:0/309574591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228059fa0 con 0x56191d30df40 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56191d2d5d60 con 0x56191d30df40 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd213fff700 1 -- 172.21.15.45:0/309574591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fd22805dcf0 con 0x56191d30df40 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd213fff700 1 -- 172.21.15.45:0/309574591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd228056d50 con 0x56191d30df40 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd245211b80 10 client.9639 fetch_fsmap learned FSMap version 256 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.710+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=256,monmap=2+}) v3 -- 0x56191d1240a0 con 0x56191d30df40 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd213fff700 1 -- 172.21.15.45:0/309574591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228061080 con 0x56191d30df40 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd213fff700 1 -- 172.21.15.45:0/309574591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 256) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd22805d860 con 0x56191d30df40 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd245211b80 10 client.9639 fetch_fsmap finished waiting for FSMap version 256 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd245211b80 10 client.9639 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd245211b80 20 client.9639 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd245211b80 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d2ff8e0 0x56191d307ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd245211b80 4 client.9639 mds_command: new command op to 9121 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 --> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56191d2b2b30 con 0x56191d2ff8e0 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.711+0000 7fd239498700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d2ff8e0 0x56191d307ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd213fff700 10 client.9639 ms_handle_connect on v2:172.21.15.144:6834/4239891954 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd239498700 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d2ff8e0 0x56191d307ae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d077d51e14b55aff server_cookie=4b9c9fe58119fcff in_seq=0 out_seq=0 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd213fff700 1 -- 172.21.15.45:0/309574591 <== mds.0 v2:172.21.15.144:6834/4239891954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56191d2b2b30 con 0x56191d2ff8e0 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd213fff700 10 client.9639 handle_command_reply: tid=0 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 client.9639 shutdown 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d30df40 msgr2=0x56191d305730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d30df40 0x56191d305730 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd22805c010 tx=0x7fd228027000 comp rx=0 tx=0).stop 2023-10-06T05:25:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 shutdown_connections 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d2ff8e0 0x56191d307ae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d3314d0 0x56191d2e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d30df40 0x56191d305730 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 --2- 172.21.15.45:0/309574591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d328450 0x56191d2e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd213fff700 0 client.9639 ms_handle_reset on v2:172.21.15.144:6834/4239891954 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.712+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 >> 172.21.15.45:0/309574591 conn(0x56191d32b880 msgr2=0x56191d2ddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.713+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 shutdown_connections 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.713+0000 7fd245211b80 1 -- 172.21.15.45:0/309574591 wait complete. 2023-10-06T05:25:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.713+0000 7fd245211b80 20 client.9639 trim_cache size 0 max 16384 2023-10-06T05:25:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.731+0000 7fd245211b80 1 Processor -- start 2023-10-06T05:25:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.731+0000 7fd245211b80 1 -- start start 2023-10-06T05:25:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d37f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd245211b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d30b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2d7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd245211b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d0c91e0 con 0x56191d328450 2023-10-06T05:25:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d301fc0 con 0x56191d3314d0 2023-10-06T05:25:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56191d2e4420 con 0x56191d30df40 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d30b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d30b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45200/0 (socket says 172.21.15.45:45200) 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 -- 172.21.15.45:0/2142063677 learned_addr learned my addr 172.21.15.45:0/2142063677 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 -- 172.21.15.45:0/2142063677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 msgr2=0x56191d37f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d37f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 -- 172.21.15.45:0/2142063677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 msgr2=0x56191d2d7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd23a49a700 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2d7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2d7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 -- 172.21.15.45:0/2142063677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56191d2003c0 con 0x56191d328450 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd239498700 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d30b470 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd230009fd0 tx=0x7fd230029250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6995b1e49f4f0e19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd21f7fe700 1 -- 172.21.15.45:0/2142063677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230028070 con 0x56191d328450 2023-10-06T05:25:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd21f7fe700 1 -- 172.21.15.45:0/2142063677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23000fe90 con 0x56191d328450 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.732+0000 7fd21f7fe700 1 -- 172.21.15.45:0/2142063677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230014b50 con 0x56191d328450 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 -- 172.21.15.45:0/2142063677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 msgr2=0x56191d30b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d30b470 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd230009fd0 tx=0x7fd230029250 comp rx=0 tx=0).stop 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 -- 172.21.15.45:0/2142063677 shutdown_connections 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2d7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d37f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 --2- 172.21.15.45:0/2142063677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d30b470 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 -- 172.21.15.45:0/2142063677 >> 172.21.15.45:0/2142063677 conn(0x56191d32b880 msgr2=0x56191d3056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 -- 172.21.15.45:0/2142063677 shutdown_connections 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.733+0000 7fd245211b80 1 -- 172.21.15.45:0/2142063677 wait complete. 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.734+0000 7fd245211b80 1 -- start start 2023-10-06T05:25:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.734+0000 7fd245211b80 1 Processor -- start 2023-10-06T05:25:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d2d7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d2e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d305af0 con 0x56191d328450 2023-10-06T05:25:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56191d303fe0 con 0x56191d3314d0 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56191d2fc4b0 con 0x56191d30df40 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 10 client.0 _ll_register_callbacks cb 0x56191d040a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d2e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d2e26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45214/0 (socket says 172.21.15.45:45214) 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 -- 172.21.15.45:0/3194989147 learned_addr learned my addr 172.21.15.45:0/3194989147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd23a49a700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239c99700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d2d7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 -- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 msgr2=0x56191d2d7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d2d7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 -- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 msgr2=0x56191d2e2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2e2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 -- 172.21.15.45:0/3194989147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23000e040 con 0x56191d328450 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd23a49a700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239c99700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d2d7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd239498700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d2e26a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fd230008010 tx=0x7fd230025070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfa18dd1b054a60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd213fff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd213fff700 1 -- 172.21.15.45:0/3194989147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230006250 con 0x56191d328450 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd228057450 con 0x56191d328450 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.735+0000 7fd213fff700 1 -- 172.21.15.45:0/3194989147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7fd2300063b0 con 0x56191d328450 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd213fff700 1 -- 172.21.15.45:0/3194989147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd230006780 con 0x56191d328450 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 10 client.9645 fetch_fsmap learned FSMap version 256 2023-10-06T05:25:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=256,monmap=2+}) v3 -- 0x56191d0981d0 con 0x56191d328450 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd213fff700 1 -- 172.21.15.45:0/3194989147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd230041020 con 0x56191d328450 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd213fff700 1 -- 172.21.15.45:0/3194989147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 256) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd230034070 con 0x56191d328450 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 10 client.9645 fetch_fsmap finished waiting for FSMap version 256 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 10 client.9645 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 20 client.9645 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d394f10 0x56191d3952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 4 client.9645 mds_command: new command op to 9121 tid=0[{"prefix": "status"}] 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 --> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56191d1d48d0 con 0x56191d394f10 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.736+0000 7fd239c99700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d394f10 0x56191d3952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd239c99700 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d394f10 0x56191d3952e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6eeaa7da241af8 server_cookie=6ad18d2d9bd2cd77 in_seq=0 out_seq=0 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd213fff700 10 client.9645 ms_handle_connect on v2:172.21.15.144:6834/4239891954 2023-10-06T05:25:00.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd213fff700 1 -- 172.21.15.45:0/3194989147 <== mds.0 v2:172.21.15.144:6834/4239891954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56191d1d48d0 con 0x56191d394f10 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd213fff700 10 client.9645 handle_command_reply: tid=0 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 client.9645 shutdown 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 msgr2=0x56191d2e26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d2e26a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fd230008010 tx=0x7fd230025070 comp rx=0 tx=0).stop 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 shutdown_connections 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:6834/4239891954,v1:172.21.15.144:6835/4239891954] conn(0x56191d394f10 0x56191d3952e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56191d3314d0 0x56191d2e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56191d30df40 0x56191d2d7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd213fff700 0 client.9645 ms_handle_reset on v2:172.21.15.144:6834/4239891954 2023-10-06T05:25:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 --2- 172.21.15.45:0/3194989147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56191d328450 0x56191d2e26a0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 >> 172.21.15.45:0/3194989147 conn(0x56191d32b880 msgr2=0x56191d330060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 shutdown_connections 2023-10-06T05:25:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.737+0000 7fd245211b80 1 -- 172.21.15.45:0/3194989147 wait complete. 2023-10-06T05:25:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd245211b80 20 client.9645 trim_cache size 0 max 16384 2023-10-06T05:25:00.747 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:25:00.747 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:25:00.747 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:25:00.747 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9121, 2023-10-06T05:25:00.747 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:25:00.747 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:25:00.747 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:25:00.748 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8321084719999998, 2023-10-06T05:25:00.748 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 256, 2023-10-06T05:25:00.748 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 118, 2023-10-06T05:25:00.748 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 118, 2023-10-06T05:25:00.748 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.518730801 2023-10-06T05:25:00.748 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:25:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd218066f30 msgr2=0x7fd2180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd218066f30 0x7fd2180693e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd230006f00 tx=0x7fd230008040 comp rx=0 tx=0).stop 2023-10-06T05:25:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 msgr2=0x7fd2341681a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2341681a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd2340c4f80 tx=0x7fd228017000 comp rx=0 tx=0).stop 2023-10-06T05:25:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 shutdown_connections 2023-10-06T05:25:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd218066f30 0x7fd2180693e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd234156af0 0x7fd2340b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340c8ff0 0x7fd2341681a0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 --2- 172.21.15.45:0/4213988211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2340caad0 0x7fd2340b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 >> 172.21.15.45:0/4213988211 conn(0x7fd2340cdf50 msgr2=0x7fd23414a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 shutdown_connections 2023-10-06T05:25:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.738+0000 7fd23befd700 1 -- 172.21.15.45:0/4213988211 wait complete. 2023-10-06T05:25:00.753 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:25:00.753 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2023-10-06T05:25:00.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.883+0000 7f198a844700 1 Processor -- start 2023-10-06T05:25:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.883+0000 7f198a844700 1 -- start start 2023-10-06T05:25:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f198a844700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19841595d0 0x7f198406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f198a844700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f198406b5c0 0x7f19840ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f198a844700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19840c7d00 con 0x7f1984159200 2023-10-06T05:25:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f198a844700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19840c7780 con 0x7f198406b5c0 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f198a844700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198406bb40 con 0x7f19841595d0 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f1988de1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f198406b5c0 0x7f19840ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f19837fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19841595d0 0x7f198406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f1988de1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f198406b5c0 0x7f19840ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:49296/0 (socket says 172.21.15.45:49296) 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.884+0000 7f1988de1700 1 -- 172.21.15.45:0/3263270915 learned_addr learned my addr 172.21.15.45:0/3263270915 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1988de1700 1 -- 172.21.15.45:0/3263270915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19841595d0 msgr2=0x7f198406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1988de1700 1 --2- 172.21.15.45:0/3263270915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19841595d0 0x7f198406b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1988de1700 1 -- 172.21.15.45:0/3263270915 >> v1:172.21.15.144:6789/0 conn(0x7f1984159200 legacy=0x7f19840c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1982ffd700 1 -- 172.21.15.45:0/3263270915 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998447014 0 0) 0x7f19840c7d00 con 0x7f1984159200 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1988de1700 1 -- 172.21.15.45:0/3263270915 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198406be20 con 0x7f198406b5c0 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1988de1700 1 --2- 172.21.15.45:0/3263270915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 0x7f19840ccdc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f197c009f20 tx=0x7f197c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f027c3994499df59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1982ffd700 1 -- 172.21.15.45:0/3263270915 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c00e070 con 0x7f198406b5c0 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1982ffd700 1 -- 172.21.15.45:0/3263270915 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c00b9d0 con 0x7f198406b5c0 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.885+0000 7f1982ffd700 1 -- 172.21.15.45:0/3263270915 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0092e0 con 0x7f198406b5c0 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 -- 172.21.15.45:0/3263270915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 msgr2=0x7f19840ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 --2- 172.21.15.45:0/3263270915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 0x7f19840ccdc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f197c009f20 tx=0x7f197c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 -- 172.21.15.45:0/3263270915 shutdown_connections 2023-10-06T05:25:00.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 --2- 172.21.15.45:0/3263270915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 0x7f19840ccdc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 --2- 172.21.15.45:0/3263270915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19841595d0 0x7f198406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 -- 172.21.15.45:0/3263270915 >> 172.21.15.45:0/3263270915 conn(0x7f198406f090 msgr2=0x7f198406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 -- 172.21.15.45:0/3263270915 shutdown_connections 2023-10-06T05:25:00.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.886+0000 7f198a844700 1 -- 172.21.15.45:0/3263270915 wait complete. 2023-10-06T05:25:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.887+0000 7f198a844700 1 Processor -- start 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.888+0000 7f198a844700 1 -- start start 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.888+0000 7f198a844700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 0x7f19840cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.888+0000 7f198a844700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1984159200 0x7f19840ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.888+0000 7f198a844700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19841595d0 0x7f19840cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.888+0000 7f198a844700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840cafb0 con 0x7f19841595d0 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.888+0000 7f198a844700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840ccc30 con 0x7f198406b5c0 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.888+0000 7f198a844700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19840ccf10 con 0x7f1984159200 2023-10-06T05:25:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19841595d0 0x7f19840cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f19837fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1984159200 0x7f19840ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 0x7f19840cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19841595d0 0x7f19840cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45230/0 (socket says 172.21.15.45:45230) 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 -- 172.21.15.45:0/1775104292 learned_addr learned my addr 172.21.15.45:0/1775104292 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 -- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1984159200 msgr2=0x7f19840ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1984159200 0x7f19840ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 -- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 msgr2=0x7f19840cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 0x7f19840cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 -- 172.21.15.45:0/1775104292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c009ae0 con 0x7f19841595d0 2023-10-06T05:25:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f1988de1700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19841595d0 0x7f19840cb440 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f198406aed0 tx=0x7f197c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f196bfff700 1 -- 172.21.15.45:0/1775104292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c008b30 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f196bfff700 1 -- 172.21.15.45:0/1775104292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c008cb0 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19840ca960 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f196bfff700 1 -- 172.21.15.45:0/1775104292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0266b0 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.889+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19840cb980 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.890+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19840cb160 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.891+0000 7f196bfff700 1 -- 172.21.15.45:0/1775104292 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f197c003040 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.891+0000 7f196bfff700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1970066f30 0x7f19700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.891+0000 7f196bfff700 1 -- 172.21.15.45:0/1775104292 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7f197c086ff0 con 0x7f19841595d0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.891+0000 7f1983fff700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1970066f30 0x7f19700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.892+0000 7f1983fff700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1970066f30 0x7f19700693e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1974007fd0 tx=0x7f1974007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:00.896+0000 7f196bfff700 1 -- 172.21.15.45:0/1775104292 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19840cb160 con 0x7f19841595d0 2023-10-06T05:25:01.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.149+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f19840cb0f0 con 0x7f19841595d0 2023-10-06T05:25:01.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.150+0000 7f196bfff700 1 -- 172.21.15.45:0/1775104292 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v256) v1 ==== 53+0+73 (secure 0 0 0) 0x7f197c0536c0 con 0x7f19841595d0 2023-10-06T05:25:01.152 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:01.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1970066f30 msgr2=0x7f19700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1970066f30 0x7f19700693e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1974007fd0 tx=0x7f1974007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:01.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19841595d0 msgr2=0x7f19840cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19841595d0 0x7f19840cb440 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f198406aed0 tx=0x7f197c018000 comp rx=0 tx=0).stop 2023-10-06T05:25:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 shutdown_connections 2023-10-06T05:25:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1970066f30 0x7f19700693e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19841595d0 0x7f19840cb440 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f198406b5c0 0x7f19840cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 --2- 172.21.15.45:0/1775104292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1984159200 0x7f19840ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 >> 172.21.15.45:0/1775104292 conn(0x7f198406f090 msgr2=0x7f198415ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.152+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 shutdown_connections 2023-10-06T05:25:01.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:01.153+0000 7f198a844700 1 -- 172.21.15.45:0/1775104292 wait complete. 2023-10-06T05:25:01.167 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:01.168 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:01.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:01.169 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:25:01.191 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:01.191 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:01.191 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:01.191 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:01.191 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:01.191 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41840sec preferred_lft 41840sec 2023-10-06T05:25:01.192 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:01.193 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:25:01.194 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:25:01.194 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:01.194 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:01.194 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:25:01.194 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:25:01.194 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:01.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:01.194 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:01.262 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:25:01.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:01.264 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:25:01.328 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:25:01.328 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:01.328 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:25:01.328 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:25:01.328 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:01.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:01.329 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:25:01.329 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:25:01.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:25:01.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:25:01.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:25:01.329 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:25:01.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:01.452 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:25:01.475 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:25:01.475 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:25:01.475 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:01.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:01.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:25:01.551 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:25:01.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:01.551 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:01.615 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:01.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:01.615 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:01.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:25:01.755 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:25:01.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:25:01.829 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:25:01.830 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:25:01.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:01.831 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:25:01.895 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:25:01.895 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:25:01.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:25:01.973 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:25:01.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:25:02.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:25:02.037+0000 7fc6e9e65540 -1 init, newargv = 0x55887f6626f0 newargc=15 2023-10-06T05:25:02.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[172409]: starting ceph client 2023-10-06T05:25:02.045 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[172409]: starting fuse 2023-10-06T05:25:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:25:02.049 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:25:02.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:02.049 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:25:02.110 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:25:02.111 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:25:02.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:25:02.111 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:25:02.111 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:25:02.111 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:25:02.111 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:25:02.111 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:02.111 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:25:02.112 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:25:02.113 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:25:02.250 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.172409.asok 2023-10-06T05:25:02.432 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.172409.asok 2023-10-06T05:25:02.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:02.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.172409.asok status 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "172409", 2023-10-06T05:25:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9654, 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 9654 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:25:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1264459669 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1264459669 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.9654 192.168.0.1:0/1264459669", 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1264459669", 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:25:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 256, 2023-10-06T05:25:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 118, 2023-10-06T05:25:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:25:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:25:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:25:02.610 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:25:02.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:02.619 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:25:02.619 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:02.639 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:25:02.640 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:02.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:02.640 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:02.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:02.713 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:25:02.756 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:25:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:25:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:25:02.783 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:25:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:sixmegs 2023-10-06T05:25:02.784 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2023-10-06T05:25:02.784 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2023-10-06T05:25:02.784 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:25:02.784 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:25:02.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:02.869 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:25:02.869 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:25:02.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:02.869 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:02.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:03.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.114+0000 7f4cf7a00700 1 Processor -- start 2023-10-06T05:25:03.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.115+0000 7f4cf7a00700 1 -- start start 2023-10-06T05:25:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf7a00700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf00cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf7a00700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf00cd760 con 0x7f4cf0159200 2023-10-06T05:25:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf7a00700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf00c7d00 con 0x7f4cf01595d0 2023-10-06T05:25:03.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf7a00700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00c7780 con 0x7f4cf00ccdc0 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf579c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4cf0159200 0x7f4cf00c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56834/0 (socket says 172.21.15.45:56834) 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf579c700 1 -- 172.21.15.45:0/3133584582 learned_addr learned my addr 172.21.15.45:0/3133584582 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf5f9d700 1 --2- 172.21.15.45:0/3133584582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf00cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4ce7fff700 1 -- 172.21.15.45:0/3133584582 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610797816 0 0) 0x7f4cec001110 con 0x7f4cf0159200 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4ce7fff700 1 -- 172.21.15.45:0/3133584582 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cdc003570 con 0x7f4cf0159200 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4ce7fff700 1 -- 172.21.15.45:0/3133584582 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747755565 0 0) 0x7f4cf00c7d00 con 0x7f4cf01595d0 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4ce7fff700 1 -- 172.21.15.45:0/3133584582 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cec001110 con 0x7f4cf01595d0 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf5f9d700 1 -- 172.21.15.45:0/3133584582 >> v1:172.21.15.144:6789/0 conn(0x7f4cf01595d0 legacy=0x7f4cf006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf5f9d700 1 -- 172.21.15.45:0/3133584582 >> v1:172.21.15.45:6789/0 conn(0x7f4cf0159200 legacy=0x7f4cf00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.116+0000 7f4cf5f9d700 1 -- 172.21.15.45:0/3133584582 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf006bb40 con 0x7f4cf00ccdc0 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf5f9d700 1 --2- 172.21.15.45:0/3133584582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf00cd220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009e90 tx=0x7f4ce8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7dcc8acc085a4dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4ce7fff700 1 -- 172.21.15.45:0/3133584582 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800e070 con 0x7f4cf00ccdc0 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4ce7fff700 1 -- 172.21.15.45:0/3133584582 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce800bb80 con 0x7f4cf00ccdc0 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4ce7fff700 1 -- 172.21.15.45:0/3133584582 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8012430 con 0x7f4cf00ccdc0 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf7a00700 1 -- 172.21.15.45:0/3133584582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 msgr2=0x7f4cf00cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/3133584582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf00cd220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009e90 tx=0x7f4ce8009780 comp rx=0 tx=0).stop 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf7a00700 1 -- 172.21.15.45:0/3133584582 shutdown_connections 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/3133584582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf00cd220 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf7a00700 1 -- 172.21.15.45:0/3133584582 >> 172.21.15.45:0/3133584582 conn(0x7f4cf006f090 msgr2=0x7f4cf006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:03.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf7a00700 1 -- 172.21.15.45:0/3133584582 shutdown_connections 2023-10-06T05:25:03.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.117+0000 7f4cf7a00700 1 -- 172.21.15.45:0/3133584582 wait complete. 2023-10-06T05:25:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.119+0000 7f4cf7a00700 1 Processor -- start 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.119+0000 7f4cf7a00700 1 -- start start 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf7a00700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf016c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf7a00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf0159200 0x7f4cf00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf7a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf01595d0 0x7f4cf00c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf7a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0147ec0 con 0x7f4cf01595d0 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf7a00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00b0480 con 0x7f4cf0159200 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf7a00700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00b01a0 con 0x7f4cf00ccdc0 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf01595d0 0x7f4cf00c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf01595d0 0x7f4cf00c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45232/0 (socket says 172.21.15.45:45232) 2023-10-06T05:25:03.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 -- 172.21.15.45:0/850059885 learned_addr learned my addr 172.21.15.45:0/850059885 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf579c700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf016c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf4f9b700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf0159200 0x7f4cf00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 -- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 msgr2=0x7f4cf016c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf016c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 -- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf0159200 msgr2=0x7f4cf00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf0159200 0x7f4cf00c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 -- 172.21.15.45:0/850059885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8009a20 con 0x7f4cf01595d0 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf579c700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf016c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf5f9d700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf01595d0 0x7f4cf00c9d70 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800b610 tx=0x7f4ce8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4ce57fa700 1 -- 172.21.15.45:0/850059885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8012e30 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4ce57fa700 1 -- 172.21.15.45:0/850059885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce801c610 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf0147870 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.120+0000 7f4ce57fa700 1 -- 172.21.15.45:0/850059885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce801c9d0 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.121+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf01539d0 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.122+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf01476d0 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.122+0000 7f4ce57fa700 1 -- 172.21.15.45:0/850059885 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ce801b400 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.122+0000 7f4ce57fa700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cd4066f30 0x7f4cd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.122+0000 7f4ce57fa700 1 -- 172.21.15.45:0/850059885 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8614+0+0 (secure 0 0 0) 0x7f4ce80872c0 con 0x7f4cf01595d0 2023-10-06T05:25:03.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.123+0000 7f4cf579c700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cd4066f30 0x7f4cd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.123+0000 7f4cf579c700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cd4066f30 0x7f4cd40693e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0058d0 tx=0x7f4cec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.127+0000 7f4ce57fa700 1 -- 172.21.15.45:0/850059885 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cf01476d0 con 0x7f4cf01595d0 2023-10-06T05:25:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.375+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4cf0148070 con 0x7f4cf01595d0 2023-10-06T05:25:03.377 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:03.376+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:03.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.380+0000 7f4ce57fa700 1 -- 172.21.15.45:0/850059885 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v257) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4cf0148070 con 0x7f4cf01595d0 2023-10-06T05:25:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cd4066f30 msgr2=0x7f4cd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cd4066f30 0x7f4cd40693e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0058d0 tx=0x7f4cec009040 comp rx=0 tx=0).stop 2023-10-06T05:25:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf01595d0 msgr2=0x7f4cf00c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf01595d0 0x7f4cf00c9d70 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800b610 tx=0x7f4ce8019000 comp rx=0 tx=0).stop 2023-10-06T05:25:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 shutdown_connections 2023-10-06T05:25:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4cd4066f30 0x7f4cd40693e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf01595d0 0x7f4cf00c9d70 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cf00ccdc0 0x7f4cf016c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 --2- 172.21.15.45:0/850059885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf0159200 0x7f4cf00c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 >> 172.21.15.45:0/850059885 conn(0x7f4cf006f090 msgr2=0x7f4cf00aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:03.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 shutdown_connections 2023-10-06T05:25:03.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.382+0000 7f4cf7a00700 1 -- 172.21.15.45:0/850059885 wait complete. 2023-10-06T05:25:03.385 INFO:tasks.ceph.mds.b.smithi144.stderr: -560> 2023-10-06T05:25:00.200+0000 7fb1d8814700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:03.385 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:03.397 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:25:03.406 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:25:03.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.531+0000 7fac2a5b0700 1 Processor -- start 2023-10-06T05:25:03.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.531+0000 7fac2a5b0700 1 -- start start 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.532+0000 7fac2a5b0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac240c8ff0 0x7fac240c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.532+0000 7fac2a5b0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fac240caad0 0x7fac240caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.532+0000 7fac2a5b0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2415b000 con 0x7fac24156af0 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.532+0000 7fac2a5b0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac24157630 con 0x7fac240c8ff0 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.532+0000 7fac2a5b0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2415b2e0 con 0x7fac240caad0 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.532+0000 7fac23fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac240c8ff0 0x7fac240c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac23fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac240c8ff0 0x7fac240c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45248/0 (socket says 172.21.15.45:45248) 2023-10-06T05:25:03.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac23fff700 1 -- 172.21.15.45:0/1062074510 learned_addr learned my addr 172.21.15.45:0/1062074510 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac237fe700 1 --2- 172.21.15.45:0/1062074510 >> v2:172.21.15.144:3300/0 conn(0x7fac240caad0 0x7fac240caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac23fff700 1 -- 172.21.15.45:0/1062074510 >> v2:172.21.15.144:3300/0 conn(0x7fac240caad0 msgr2=0x7fac240caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac23fff700 1 --2- 172.21.15.45:0/1062074510 >> v2:172.21.15.144:3300/0 conn(0x7fac240caad0 0x7fac240caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac23fff700 1 -- 172.21.15.45:0/1062074510 >> v1:172.21.15.144:6789/0 conn(0x7fac24156af0 legacy=0x7fac24156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac23fff700 1 -- 172.21.15.45:0/1062074510 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2415b5c0 con 0x7fac240c8ff0 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac23fff700 1 --2- 172.21.15.45:0/1062074510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c93c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fac2406ed80 tx=0x7fac14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc54d97bdc10a54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac22ffd700 1 -- 172.21.15.45:0/1062074510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400b820 con 0x7fac240c8ff0 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac22ffd700 1 -- 172.21.15.45:0/1062074510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1400b980 con 0x7fac240c8ff0 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.533+0000 7fac22ffd700 1 -- 172.21.15.45:0/1062074510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400bd20 con 0x7fac240c8ff0 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 -- 172.21.15.45:0/1062074510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 msgr2=0x7fac240c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/1062074510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c93c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fac2406ed80 tx=0x7fac14009250 comp rx=0 tx=0).stop 2023-10-06T05:25:03.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 -- 172.21.15.45:0/1062074510 shutdown_connections 2023-10-06T05:25:03.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/1062074510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c93c0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/1062074510 >> v2:172.21.15.144:3300/0 conn(0x7fac240caad0 0x7fac240caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 -- 172.21.15.45:0/1062074510 >> 172.21.15.45:0/1062074510 conn(0x7fac240cdf50 msgr2=0x7fac240ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:03.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 -- 172.21.15.45:0/1062074510 shutdown_connections 2023-10-06T05:25:03.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.534+0000 7fac2a5b0700 1 -- 172.21.15.45:0/1062074510 wait complete. 2023-10-06T05:25:03.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.535+0000 7fac2a5b0700 1 Processor -- start 2023-10-06T05:25:03.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.536+0000 7fac2a5b0700 1 -- start start 2023-10-06T05:25:03.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac2a5b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac2a5b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac240caad0 0x7fac24145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac2a5b0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac24156af0 0x7fac240b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac2a5b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240c75a0 con 0x7fac240c8ff0 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac2a5b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240b17c0 con 0x7fac240caad0 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac2a5b0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240b1aa0 con 0x7fac24156af0 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45256/0 (socket says 172.21.15.45:45256) 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 -- 172.21.15.45:0/3271120421 learned_addr learned my addr 172.21.15.45:0/3271120421 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac237fe700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac240caad0 0x7fac24145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac28b4d700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac24156af0 0x7fac240b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 -- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac24156af0 msgr2=0x7fac240b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac24156af0 0x7fac240b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 -- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac240caad0 msgr2=0x7fac24145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:03.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac240caad0 0x7fac24145960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 -- 172.21.15.45:0/3271120421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac140094f0 con 0x7fac240c8ff0 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac237fe700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac240caad0 0x7fac24145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.537+0000 7fac23fff700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c7a30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fac140030c0 tx=0x7fac14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.538+0000 7fac0bfff700 1 -- 172.21.15.45:0/3271120421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14011e60 con 0x7fac240c8ff0 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.538+0000 7fac0bfff700 1 -- 172.21.15.45:0/3271120421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14020070 con 0x7fac240c8ff0 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.538+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac240c7230 con 0x7fac240c8ff0 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.538+0000 7fac0bfff700 1 -- 172.21.15.45:0/3271120421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1401b6a0 con 0x7fac240c8ff0 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.538+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac240c5e80 con 0x7fac240c8ff0 2023-10-06T05:25:03.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.539+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2406f410 con 0x7fac240c8ff0 2023-10-06T05:25:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.539+0000 7fac0bfff700 1 -- 172.21.15.45:0/3271120421 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac14022030 con 0x7fac240c8ff0 2023-10-06T05:25:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.540+0000 7fac0bfff700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac10066f30 0x7fac100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.540+0000 7fac0bfff700 1 -- 172.21.15.45:0/3271120421 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8700+0+0 (secure 0 0 0) 0x7fac14087020 con 0x7fac240c8ff0 2023-10-06T05:25:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.540+0000 7fac237fe700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac10066f30 0x7fac100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:03.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.540+0000 7fac237fe700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac10066f30 0x7fac100693e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac241532d0 tx=0x7fac18008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:03.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.544+0000 7fac0bfff700 1 -- 172.21.15.45:0/3271120421 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac140536a0 con 0x7fac240c8ff0 2023-10-06T05:25:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:03.837+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fac2406f410 con 0x7fac240c8ff0 2023-10-06T05:25:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.381+0000 7fac0bfff700 1 -- 172.21.15.45:0/3271120421 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v258) v1 ==== 107+0+0 (secure 0 0 0) 0x7fac14056cc0 con 0x7fac240c8ff0 2023-10-06T05:25:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.384+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac10066f30 msgr2=0x7fac100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.384+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac10066f30 0x7fac100693e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac241532d0 tx=0x7fac18008040 comp rx=0 tx=0).stop 2023-10-06T05:25:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.384+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 msgr2=0x7fac240c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.384+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c7a30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fac140030c0 tx=0x7fac14016000 comp rx=0 tx=0).stop 2023-10-06T05:25:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 shutdown_connections 2023-10-06T05:25:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac10066f30 0x7fac100693e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac24156af0 0x7fac240b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac240c8ff0 0x7fac240c7a30 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 --2- 172.21.15.45:0/3271120421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac240caad0 0x7fac24145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 >> 172.21.15.45:0/3271120421 conn(0x7fac240cdf50 msgr2=0x7fac24155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 shutdown_connections 2023-10-06T05:25:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.385+0000 7fac2a5b0700 1 -- 172.21.15.45:0/3271120421 wait complete. 2023-10-06T05:25:04.389 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:25:04.403 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:25:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.448+0000 7f9ffba7c040 1 Processor -- start 2023-10-06T05:25:04.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.448+0000 7f9ffba7c040 1 -- start start 2023-10-06T05:25:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ffba7c040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61280210 0x55ea612c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ffba7c040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342890 0x55ea612b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ffba7c040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61342c60 0x55ea6135e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ffba7c040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea611cd980 con 0x55ea61342c60 2023-10-06T05:25:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ffba7c040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea611cd620 con 0x55ea61342890 2023-10-06T05:25:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ffba7c040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea611c4400 con 0x55ea61280210 2023-10-06T05:25:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61342c60 0x55ea6135e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61342c60 0x55ea6135e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45264/0 (socket says 172.21.15.45:45264) 2023-10-06T05:25:04.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 -- 172.21.15.45:0/2319477746 learned_addr learned my addr 172.21.15.45:0/2319477746 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61280210 0x55ea612c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9feffff700 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342890 0x55ea612b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 -- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61280210 msgr2=0x55ea612c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61280210 0x55ea612c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 -- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342890 msgr2=0x55ea612b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342890 0x55ea612b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.449+0000 7f9ff549b700 1 -- 172.21.15.45:0/2319477746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe8005cf0 con 0x55ea61342c60 2023-10-06T05:25:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ff549b700 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61342c60 0x55ea6135e8b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0006f40 tx=0x7f9ff00069f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82c9b8c9080a638 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9feeffd700 1 -- 172.21.15.45:0/2319477746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0006110 con 0x55ea61342c60 2023-10-06T05:25:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9feeffd700 1 -- 172.21.15.45:0/2319477746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff0006270 con 0x55ea61342c60 2023-10-06T05:25:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9feeffd700 1 -- 172.21.15.45:0/2319477746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0004730 con 0x55ea61342c60 2023-10-06T05:25:04.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 -- 172.21.15.45:0/2319477746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61342c60 msgr2=0x55ea6135e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61342c60 0x55ea6135e8b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0006f40 tx=0x7f9ff00069f0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 -- 172.21.15.45:0/2319477746 shutdown_connections 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61342c60 0x55ea6135e8b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61280210 0x55ea612c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/2319477746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342890 0x55ea612b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 -- 172.21.15.45:0/2319477746 >> 172.21.15.45:0/2319477746 conn(0x55ea61283450 msgr2=0x55ea61285830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 -- 172.21.15.45:0/2319477746 shutdown_connections 2023-10-06T05:25:04.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.450+0000 7f9ffba7c040 1 -- 172.21.15.45:0/2319477746 wait complete. 2023-10-06T05:25:04.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.451+0000 7f9ffba7c040 1 Processor -- start 2023-10-06T05:25:04.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.451+0000 7f9ffba7c040 1 -- start start 2023-10-06T05:25:04.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ffba7c040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61280210 0x55ea6133e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ffba7c040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61342890 0x55ea6135f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ffba7c040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342c60 0x55ea612847f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ffba7c040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea6133d690 con 0x55ea61280210 2023-10-06T05:25:04.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ffba7c040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea6133d8c0 con 0x55ea61342c60 2023-10-06T05:25:04.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ffba7c040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea6133db60 con 0x55ea61342890 2023-10-06T05:25:04.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61280210 0x55ea6133e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61280210 0x55ea6133e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45272/0 (socket says 172.21.15.45:45272) 2023-10-06T05:25:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 learned_addr learned my addr 172.21.15.45:0/1672506894 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9feffff700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61342890 0x55ea6135f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342c60 0x55ea612847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61342890 msgr2=0x55ea6135f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea61342890 0x55ea6135f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342c60 msgr2=0x55ea612847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342c60 0x55ea612847f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8001e10 con 0x55ea61280210 2023-10-06T05:25:04.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.452+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea61342c60 0x55ea612847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:04.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.453+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61280210 0x55ea6133e540 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8008bc0 tx=0x7f9fe800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.453+0000 7f9fed7fa700 1 -- 172.21.15.45:0/1672506894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800f040 con 0x55ea61280210 2023-10-06T05:25:04.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.453+0000 7f9fed7fa700 1 -- 172.21.15.45:0/1672506894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe8010960 con 0x55ea61280210 2023-10-06T05:25:04.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.453+0000 7f9fed7fa700 1 -- 172.21.15.45:0/1672506894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8010ce0 con 0x55ea61280210 2023-10-06T05:25:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.453+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe0005cf0 con 0x55ea61280210 2023-10-06T05:25:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.453+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea611dc3d0 con 0x55ea61280210 2023-10-06T05:25:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.454+0000 7f9fed7fa700 1 -- 172.21.15.45:0/1672506894 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fe8015660 con 0x55ea61280210 2023-10-06T05:25:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.455+0000 7f9fed7fa700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc065890 0x7f9fdc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.455+0000 7f9fed7fa700 1 -- 172.21.15.45:0/1672506894 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8700+0+0 (secure 0 0 0) 0x7f9fe8084b30 con 0x55ea61280210 2023-10-06T05:25:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.455+0000 7f9feffff700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc065890 0x7f9fdc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.469 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:25:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.455+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ea6135f9d0 0x55ea6135cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.455+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613634f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.455+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ea6135f9d0 0x55ea6135cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.455+0000 7f9feffff700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc065890 0x7f9fdc067d40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0005a90 tx=0x7f9fe0015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.456+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ea6135f9d0 0x55ea6135cd60 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.457+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v119'815 uv789 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.458+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61284d30 con 0x55ea6135f9d0 2023-10-06T05:25:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.459+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v119'816 uv816 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.459+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376090 con 0x55ea6135f9d0 2023-10-06T05:25:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.460+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v119'817 uv817 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.473 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:25:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.460+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61377450 con 0x55ea6135f9d0 2023-10-06T05:25:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.461+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'818 uv818 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.462+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61377800 con 0x55ea6135f9d0 2023-10-06T05:25:04.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.463+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'819 uv819 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.463+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376440 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.464+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'820 uv820 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.464+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376840 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.466+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v119'821 uv795 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.466+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376c40 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.467+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v119'822 uv822 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.467+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376c40 con 0x55ea6135f9d0 2023-10-06T05:25:04.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.469+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v119'823 uv823 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.478 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:25:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.469+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.470+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'824 uv824 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.470+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.471+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'825 uv825 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.471+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.473+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'826 uv826 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.473+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.474+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v119'827 uv802 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.474+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.475+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'828 uv828 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.475+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.477+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v119'829 uv829 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.482 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:25:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.477+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.478+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'830 uv830 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.478+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.480+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'831 uv831 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.480+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376bf0 con 0x55ea6135f9d0 2023-10-06T05:25:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.481+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'832 uv832 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.481+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376fd0 con 0x55ea6135f9d0 2023-10-06T05:25:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.483+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'833 uv833 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.483+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.484+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v119'834 uv808 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.484+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.486+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v119'835 uv835 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.486+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.488+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v119'836 uv836 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.490 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:25:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.488+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.489+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'837 uv837 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.489+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.491+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'838 uv838 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.491+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.492+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'839 uv839 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.492+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.493+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v119'840 uv814 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.493+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.495+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v119'841 uv841 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.495+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.496+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v119'842 uv842 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.499 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:25:04.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.496+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.498+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'843 uv843 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.498+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.499+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'844 uv844 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.499+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61376e60 con 0x55ea6135f9d0 2023-10-06T05:25:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.501+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'845 uv845 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6135f9d0 2023-10-06T05:25:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.501+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ea61377bb0 0x55ea61379f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.501+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137a4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.501+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ea61377bb0 0x55ea61379f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.501+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ea61377bb0 0x55ea61379f90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.503+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'590 uv570 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.503+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137a880 con 0x55ea61377bb0 2023-10-06T05:25:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.505+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'591 uv591 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.505+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137ac30 con 0x55ea61377bb0 2023-10-06T05:25:04.508 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:25:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.506+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'592 uv592 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.506+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137b4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.508+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v119'593 uv593 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.508+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137b880 con 0x55ea61377bb0 2023-10-06T05:25:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.509+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'594 uv594 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.509+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137b060 con 0x55ea61377bb0 2023-10-06T05:25:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.510+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v119'595 uv589 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.510+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c120 con 0x55ea61377bb0 2023-10-06T05:25:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.511+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v119'596 uv596 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.512+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.516 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:25:04.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.513+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v119'597 uv597 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.513+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.514+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'598 uv598 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.514+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.515+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'599 uv599 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.515+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.517+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v119'600 uv588 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.517+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.518+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v119'601 uv601 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.518+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.525 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:25:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.519+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v119'602 uv602 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.519+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137fbf0 con 0x55ea61377bb0 2023-10-06T05:25:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.521+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v119'603 uv603 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.521+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137fbf0 con 0x55ea61377bb0 2023-10-06T05:25:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.522+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'604 uv604 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.522+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.523+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v119'605 uv586 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.523+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.525+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v119'606 uv606 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.525+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.528 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:25:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.526+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v119'607 uv607 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.526+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137d4e0 con 0x55ea61377bb0 2023-10-06T05:25:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.527+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'608 uv608 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.527+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137d4e0 con 0x55ea61377bb0 2023-10-06T05:25:04.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.528+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'609 uv609 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.528+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137c4d0 con 0x55ea61377bb0 2023-10-06T05:25:04.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.530+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'610 uv610 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.530+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ea6137c9e0 0x55ea6137edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.530+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137f350 con 0x55ea6137c9e0 2023-10-06T05:25:04.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.530+0000 7f9feffff700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ea6137c9e0 0x55ea6137edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.530+0000 7f9feffff700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ea6137c9e0 0x55ea6137edc0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.532+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.532+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137fc20 con 0x55ea6137c9e0 2023-10-06T05:25:04.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.533+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v119'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.533+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6137ffd0 con 0x55ea6137c9e0 2023-10-06T05:25:04.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.534+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v119'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.536 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:25:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.534+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61381390 con 0x55ea6137c9e0 2023-10-06T05:25:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.535+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.535+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613817e0 con 0x55ea6137c9e0 2023-10-06T05:25:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.536+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.536+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380380 con 0x55ea6137c9e0 2023-10-06T05:25:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.537+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.537+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380780 con 0x55ea6137c9e0 2023-10-06T05:25:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.539+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'113 uv112 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.539+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380780 con 0x55ea6137c9e0 2023-10-06T05:25:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.540+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'114 uv114 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.540+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380780 con 0x55ea6137c9e0 2023-10-06T05:25:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.541+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'115 uv115 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.543 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:25:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.541+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380c50 con 0x55ea6137c9e0 2023-10-06T05:25:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.542+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'116 uv116 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.542+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380c50 con 0x55ea61377bb0 2023-10-06T05:25:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.544+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.544+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380c50 con 0x55ea61377bb0 2023-10-06T05:25:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.545+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v119'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.545+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380c50 con 0x55ea61377bb0 2023-10-06T05:25:04.548 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:25:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.546+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v119'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.546+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380c50 con 0x55ea61377bb0 2023-10-06T05:25:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.547+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.547+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380c50 con 0x55ea61377bb0 2023-10-06T05:25:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.548+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.548+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61380c50 con 0x55ea61377bb0 2023-10-06T05:25:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.549+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61377bb0 2023-10-06T05:25:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.549+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ea61380c50 0x55ea61384100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.549+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613807d0 con 0x55ea61380c50 2023-10-06T05:25:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.550+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ea61380c50 0x55ea61384100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.550+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ea61380c50 0x55ea61384100 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.551+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v119'294 uv287 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.551+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613807d0 con 0x55ea61380c50 2023-10-06T05:25:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.552+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v119'295 uv295 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.553+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613846c0 con 0x55ea61380c50 2023-10-06T05:25:04.556 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:25:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.554+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v119'296 uv296 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.554+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613856d0 con 0x55ea61380c50 2023-10-06T05:25:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.555+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'297 uv297 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.555+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613856d0 con 0x55ea61380c50 2023-10-06T05:25:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.556+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'298 uv298 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.556+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613846c0 con 0x55ea61380c50 2023-10-06T05:25:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.557+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'299 uv299 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.557+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613846c0 con 0x55ea61380c50 2023-10-06T05:25:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.558+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v119'300 uv293 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.558+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613846c0 con 0x55ea61380c50 2023-10-06T05:25:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.559+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v119'301 uv301 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.559+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613846c0 con 0x55ea61380c50 2023-10-06T05:25:04.562 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:25:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.560+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v119'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.560+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613846c0 con 0x55ea61380c50 2023-10-06T05:25:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.561+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'303 uv303 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.562+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea613846c0 con 0x55ea61380c50 2023-10-06T05:25:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.563+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'304 uv304 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.563+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ea61384d40 0x55ea61387120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.563+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387700 con 0x55ea61384d40 2023-10-06T05:25:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.563+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ea61384d40 0x55ea61387120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.563+0000 7f9ff4c9a700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ea61384d40 0x55ea61387120 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.565+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v119'132 uv131 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61384d40 2023-10-06T05:25:04.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.565+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea61384d40 2023-10-06T05:25:04.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.566+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v119'133 uv133 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61384d40 2023-10-06T05:25:04.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.566+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea61384d40 2023-10-06T05:25:04.569 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:25:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.567+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v119'134 uv134 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61384d40 2023-10-06T05:25:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.567+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea61384d40 2023-10-06T05:25:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.568+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'135 uv135 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61384d40 2023-10-06T05:25:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.568+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea61384d40 2023-10-06T05:25:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.571+0000 7f9ff4c9a700 1 -- 172.21.15.45:0/1672506894 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'136 uv136 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe8012040 con 0x55ea61384d40 2023-10-06T05:25:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.571+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea6137c9e0 2023-10-06T05:25:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.572+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.572+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea6137c9e0 2023-10-06T05:25:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.573+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v119'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.573+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea6137c9e0 2023-10-06T05:25:04.576 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:25:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.575+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v119'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.575+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61388af0 con 0x55ea6137c9e0 2023-10-06T05:25:04.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.576+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.576+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61388af0 con 0x55ea6137c9e0 2023-10-06T05:25:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.577+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.577+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea61387ae0 con 0x55ea6137c9e0 2023-10-06T05:25:04.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.578+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.578+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ea61388160 0x55ea6138a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.578+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138ab70 con 0x55ea61388160 2023-10-06T05:25:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.579+0000 7f9feffff700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ea61388160 0x55ea6138a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.579+0000 7f9feffff700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ea61388160 0x55ea6138a540 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.580+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v119'263 uv257 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.581+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.582+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v119'264 uv264 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.582+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.584 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:25:04.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.583+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v119'265 uv265 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.583+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.584+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'266 uv266 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.584+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.585+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'267 uv267 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.585+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.586+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v119'268 uv262 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.586+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.587+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v119'269 uv269 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.587+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.590 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:25:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.588+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v119'270 uv270 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.588+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af50 con 0x55ea61388160 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.589+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'271 uv271 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.589+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138dcc0 con 0x55ea61388160 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.590+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'272 uv272 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea61388160 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.590+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138b440 con 0x55ea6137c9e0 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.592+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v119'116 uv114 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.592+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138b440 con 0x55ea6137c9e0 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.593+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'117 uv117 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.593+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138b440 con 0x55ea6137c9e0 2023-10-06T05:25:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.594+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v119'118 uv118 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.597 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:25:04.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.594+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138b440 con 0x55ea6137c9e0 2023-10-06T05:25:04.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.595+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'119 uv119 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.595+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ea6138b950 0x55ea6138bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.595+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6138b950 2023-10-06T05:25:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.595+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ea6138b950 0x55ea6138bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.596+0000 7f9ff549b700 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ea6138b950 0x55ea6138bd20 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.597+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6138b950 2023-10-06T05:25:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.597+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6138b950 2023-10-06T05:25:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.600+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v119'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6138b950 2023-10-06T05:25:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.600+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6138b950 2023-10-06T05:25:04.605 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:25:04.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.603+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v119'161 uv161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6138b950 2023-10-06T05:25:04.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.603+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138f420 con 0x55ea6138b950 2023-10-06T05:25:04.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.605+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6138b950 2023-10-06T05:25:04.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.606+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138f420 con 0x55ea6138b950 2023-10-06T05:25:04.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.607+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6138b950 2023-10-06T05:25:04.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.607+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6138b950 2023-10-06T05:25:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.608+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea6138b950 2023-10-06T05:25:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.608+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea61380c50 2023-10-06T05:25:04.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.610+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.610+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea61380c50 2023-10-06T05:25:04.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.611+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v119'136 uv136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.611+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea61380c50 2023-10-06T05:25:04.614 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:25:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.613+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v119'137 uv137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.613+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea61380c50 2023-10-06T05:25:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.614+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'138 uv138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.614+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea61380c50 2023-10-06T05:25:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.615+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'139 uv139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9ff00044f0 con 0x55ea61380c50 2023-10-06T05:25:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.615+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6137c9e0 2023-10-06T05:25:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.617+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v119'327 uv318 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.617+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6137c9e0 2023-10-06T05:25:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.618+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v119'328 uv328 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.618+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6137c9e0 2023-10-06T05:25:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.619+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v119'329 uv329 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.621 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:25:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.619+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e410 con 0x55ea6137c9e0 2023-10-06T05:25:04.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.620+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v119'330 uv330 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.620+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138afd0 con 0x55ea6137c9e0 2023-10-06T05:25:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.621+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'331 uv331 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.621+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138afd0 con 0x55ea6137c9e0 2023-10-06T05:25:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.622+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v119'332 uv325 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.622+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af20 con 0x55ea6137c9e0 2023-10-06T05:25:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.623+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v119'333 uv333 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.623+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af20 con 0x55ea6137c9e0 2023-10-06T05:25:04.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.624+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v119'334 uv334 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.626 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:25:04.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.624+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af00 con 0x55ea6137c9e0 2023-10-06T05:25:04.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.626+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'335 uv335 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.626+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af00 con 0x55ea6137c9e0 2023-10-06T05:25:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.627+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'336 uv336 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.627+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138af00 con 0x55ea6137c9e0 2023-10-06T05:25:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.628+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'337 uv337 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.628+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x55ea6138e900 con 0x55ea6137c9e0 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9feffff700 1 -- 172.21.15.45:0/1672506894 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'338 uv338 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9fe0013420 con 0x55ea6137c9e0 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ea6137c9e0 msgr2=0x55ea6137edc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55ea6137c9e0 0x55ea6137edc0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ea6135f9d0 msgr2=0x55ea6135cd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55ea6135f9d0 0x55ea6135cd60 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ea61388160 msgr2=0x55ea6138a540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55ea61388160 0x55ea6138a540 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ea61377bb0 msgr2=0x55ea61379f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55ea61377bb0 0x55ea61379f90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ea6138b950 msgr2=0x55ea6138bd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55ea6138b950 0x55ea6138bd20 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ea61384d40 msgr2=0x55ea61387120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55ea61384d40 0x55ea61387120 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 reap_dead start 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ea61380c50 msgr2=0x55ea61384100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55ea61380c50 0x55ea61384100 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 reap_dead start 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ff549b700 1 -- 172.21.15.45:0/1672506894 reap_dead start 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc065890 msgr2=0x7f9fdc067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc065890 0x7f9fdc067d40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0005a90 tx=0x7f9fe0015000 comp rx=0 tx=0).stop 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61280210 msgr2=0x55ea6133e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61280210 0x55ea6133e540 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8008bc0 tx=0x7f9fe800c040 comp rx=0 tx=0).stop 2023-10-06T05:25:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 shutdown_connections 2023-10-06T05:25:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea61280210 0x55ea6133e540 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 --2- 172.21.15.45:0/1672506894 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9fdc065890 0x7f9fdc067d40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 >> 172.21.15.45:0/1672506894 conn(0x55ea61283450 msgr2=0x55ea61340ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 shutdown_connections 2023-10-06T05:25:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.629+0000 7f9ffba7c040 1 -- 172.21.15.45:0/1672506894 wait complete. 2023-10-06T05:25:04.635 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2023-10-06T05:25:04.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.727+0000 7f5d5020f040 1 Processor -- start 2023-10-06T05:25:04.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.727+0000 7f5d5020f040 1 -- start start 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d5020f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db84fc210 0x560db8521100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d5020f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db85be890 0x560db8544fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d5020f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 0x560db852d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d5020f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db84a5320 con 0x560db85be890 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d5020f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db8449980 con 0x560db84fc210 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d5020f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x560db8449620 con 0x560db85bec60 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d48c2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db85be890 0x560db8544fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d48c2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db85be890 0x560db8544fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45276/0 (socket says 172.21.15.45:45276) 2023-10-06T05:25:04.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d48c2c700 1 -- 172.21.15.45:0/553949735 learned_addr learned my addr 172.21.15.45:0/553949735 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d4942d700 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db84fc210 0x560db8521100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.728+0000 7f5d49c2e700 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 0x560db852d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d48c2c700 1 -- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 msgr2=0x560db852d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d48c2c700 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 0x560db852d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d48c2c700 1 -- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db84fc210 msgr2=0x560db8521100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d48c2c700 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db84fc210 0x560db8521100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d48c2c700 1 -- 172.21.15.45:0/553949735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40005cf0 con 0x560db85be890 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d48c2c700 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db85be890 0x560db8544fb0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000ac40 tx=0x7f5d40015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6545a8f16b981a6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d3b7fe700 1 -- 172.21.15.45:0/553949735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40002f70 con 0x560db85be890 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d3b7fe700 1 -- 172.21.15.45:0/553949735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40000bc0 con 0x560db85be890 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.729+0000 7f5d3b7fe700 1 -- 172.21.15.45:0/553949735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40007b10 con 0x560db85be890 2023-10-06T05:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 -- 172.21.15.45:0/553949735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db85be890 msgr2=0x560db8544fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db85be890 0x560db8544fb0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000ac40 tx=0x7f5d40015000 comp rx=0 tx=0).stop 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 -- 172.21.15.45:0/553949735 shutdown_connections 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 0x560db852d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db84fc210 0x560db8521100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 --2- 172.21.15.45:0/553949735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db85be890 0x560db8544fb0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 -- 172.21.15.45:0/553949735 >> 172.21.15.45:0/553949735 conn(0x560db84ff450 msgr2=0x560db8542550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 -- 172.21.15.45:0/553949735 shutdown_connections 2023-10-06T05:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.730+0000 7f5d5020f040 1 -- 172.21.15.45:0/553949735 wait complete. 2023-10-06T05:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 Processor -- start 2023-10-06T05:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 -- start start 2023-10-06T05:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db84fc210 0x560db8545120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db85be890 0x560db85deed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 0x560db85e0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db8440400 con 0x560db84fc210 2023-10-06T05:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db8528bc0 con 0x560db85be890 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.731+0000 7f5d5020f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x560db85de7d0 con 0x560db85bec60 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db84fc210 0x560db8545120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db84fc210 0x560db8545120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45292/0 (socket says 172.21.15.45:45292) 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 -- 172.21.15.45:0/3040952313 learned_addr learned my addr 172.21.15.45:0/3040952313 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d48c2c700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db85be890 0x560db85deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d49c2e700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 0x560db85e0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 -- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 msgr2=0x560db85e0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560db85bec60 0x560db85e0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 -- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db85be890 msgr2=0x560db85deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560db85be890 0x560db85deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 -- 172.21.15.45:0/3040952313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db84583d0 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d4942d700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db84fc210 0x560db8545120 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00d490 tx=0x7f5d3c006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d39ffb700 1 -- 172.21.15.45:0/3040952313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c012420 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d39ffb700 1 -- 172.21.15.45:0/3040952313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c012580 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560db844bae0 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d39ffb700 1 -- 172.21.15.45:0/3040952313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c012920 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.732+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db844b540 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.734+0000 7f5d39ffb700 1 -- 172.21.15.45:0/3040952313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d3c012cd0 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.734+0000 7f5d39ffb700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d2c0658c0 0x7f5d2c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.734+0000 7f5d39ffb700 1 -- 172.21.15.45:0/3040952313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8700+0+0 (secure 0 0 0) 0x7f5d3c083b20 con 0x560db84fc210 2023-10-06T05:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.735+0000 7f5d5020f040 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x560db8527e70 0x560db85dc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.735+0000 7f5d48c2c700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d2c0658c0 0x7f5d2c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.735+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) v8 -- 0x560db85bc230 con 0x560db8527e70 2023-10-06T05:25:04.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.735+0000 7f5d49c2e700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x560db8527e70 0x560db85dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.735+0000 7f5d48c2c700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d2c0658c0 0x7f5d2c067d70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5d4001f040 tx=0x7f5d4001c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.735+0000 7f5d49c2e700 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x560db8527e70 0x560db85dc910 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.736+0000 7f5d49c2e700 1 -- 172.21.15.45:0/3040952313 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v119'137 uv137 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5d4400f880 con 0x560db8527e70 2023-10-06T05:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.736+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x560db8527e70 msgr2=0x560db85dc910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.736+0000 7f5d5020f040 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x560db8527e70 0x560db85dc910 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d2c0658c0 msgr2=0x7f5d2c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d2c0658c0 0x7f5d2c067d70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5d4001f040 tx=0x7f5d4001c000 comp rx=0 tx=0).stop 2023-10-06T05:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db84fc210 msgr2=0x560db8545120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 --2- 172.21.15.45:0/3040952313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db84fc210 0x560db8545120 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00d490 tx=0x7f5d3c006950 comp rx=0 tx=0).stop 2023-10-06T05:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d49c2e700 1 -- 172.21.15.45:0/3040952313 reap_dead start 2023-10-06T05:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 shutdown_connections 2023-10-06T05:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 >> 172.21.15.45:0/3040952313 conn(0x560db84ff450 msgr2=0x560db8542550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 shutdown_connections 2023-10-06T05:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.737+0000 7f5d5020f040 1 -- 172.21.15.45:0/3040952313 wait complete. 2023-10-06T05:25:04.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:25:04.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.881+0000 7f289f5c4700 1 Processor -- start 2023-10-06T05:25:04.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.881+0000 7f289f5c4700 1 -- start start 2023-10-06T05:25:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289f5c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28980b0610 0x7f28980c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289f5c4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28980c3db0 0x7f28980697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289f5c4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28980ce6a0 con 0x7f2898069d00 2023-10-06T05:25:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289f5c4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28980690f0 con 0x7f28980c3db0 2023-10-06T05:25:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289f5c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980ceeb0 con 0x7f28980b0610 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289db61700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2898069d00 0x7f28980c2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56866/0 (socket says 172.21.15.45:56866) 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289db61700 1 -- 172.21.15.45:0/3283021281 learned_addr learned my addr 172.21.15.45:0/3283021281 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289cb5f700 1 --2- 172.21.15.45:0/3283021281 >> v2:172.21.15.144:3300/0 conn(0x7f28980c3db0 0x7f28980697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289d360700 1 --2- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28980b0610 0x7f28980c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f288ffff700 1 -- 172.21.15.45:0/3283021281 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533844836 0 0) 0x7f28980ce6a0 con 0x7f2898069d00 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f288ffff700 1 -- 172.21.15.45:0/3283021281 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2884003570 con 0x7f2898069d00 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289cb5f700 1 -- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28980b0610 msgr2=0x7f28980c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289cb5f700 1 --2- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28980b0610 0x7f28980c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289cb5f700 1 -- 172.21.15.45:0/3283021281 >> v1:172.21.15.45:6789/0 conn(0x7f2898069d00 legacy=0x7f28980c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289cb5f700 1 -- 172.21.15.45:0/3283021281 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2898159880 con 0x7f28980c3db0 2023-10-06T05:25:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.882+0000 7f289d360700 1 --2- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28980b0610 0x7f28980c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f289cb5f700 1 --2- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 0x7f28980697c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2888009910 tx=0x7f28880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f06f99149bb631a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f288ffff700 1 -- 172.21.15.45:0/3283021281 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888011630 con 0x7f28980c3db0 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f288ffff700 1 -- 172.21.15.45:0/3283021281 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2888011790 con 0x7f28980c3db0 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f288ffff700 1 -- 172.21.15.45:0/3283021281 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888011b30 con 0x7f28980c3db0 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f289f5c4700 1 -- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 msgr2=0x7f28980697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f289f5c4700 1 --2- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 0x7f28980697c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2888009910 tx=0x7f28880091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f289f5c4700 1 -- 172.21.15.45:0/3283021281 shutdown_connections 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f289f5c4700 1 --2- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28980b0610 0x7f28980c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f289f5c4700 1 --2- 172.21.15.45:0/3283021281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 0x7f28980697c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.883+0000 7f289f5c4700 1 -- 172.21.15.45:0/3283021281 >> 172.21.15.45:0/3283021281 conn(0x7f2898067e00 msgr2=0x7f28980cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.884+0000 7f289f5c4700 1 -- 172.21.15.45:0/3283021281 shutdown_connections 2023-10-06T05:25:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.884+0000 7f289f5c4700 1 -- 172.21.15.45:0/3283021281 wait complete. 2023-10-06T05:25:04.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.885+0000 7f289f5c4700 1 Processor -- start 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.885+0000 7f289f5c4700 1 -- start start 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289f5c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2898069d00 0x7f289815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289f5c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980b0610 0x7f289815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289f5c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 0x7f28980c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289f5c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289815c740 con 0x7f28980b0610 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289f5c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289815c410 con 0x7f28980c3db0 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289f5c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289815ad90 con 0x7f2898069d00 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980b0610 0x7f289815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980b0610 0x7f289815e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45296/0 (socket says 172.21.15.45:45296) 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 -- 172.21.15.45:0/1007332801 learned_addr learned my addr 172.21.15.45:0/1007332801 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:04.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289db61700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 0x7f28980c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289d360700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2898069d00 0x7f289815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 -- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2898069d00 msgr2=0x7f289815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2898069d00 0x7f289815a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 -- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 msgr2=0x7f28980c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 0x7f28980c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 -- 172.21.15.45:0/1007332801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940093c0 con 0x7f28980b0610 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f289cb5f700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980b0610 0x7f289815e600 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2888009f00 tx=0x7f2888017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.886+0000 7f288d7fa700 1 -- 172.21.15.45:0/1007332801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888000d10 con 0x7f28980b0610 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.887+0000 7f288d7fa700 1 -- 172.21.15.45:0/1007332801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2888011b10 con 0x7f28980b0610 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.887+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888009420 con 0x7f28980b0610 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.887+0000 7f288d7fa700 1 -- 172.21.15.45:0/1007332801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888025600 con 0x7f28980b0610 2023-10-06T05:25:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.887+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2898159160 con 0x7f28980b0610 2023-10-06T05:25:04.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.888+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289815b170 con 0x7f28980b0610 2023-10-06T05:25:04.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.888+0000 7f288d7fa700 1 -- 172.21.15.45:0/1007332801 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2888025a30 con 0x7f28980b0610 2023-10-06T05:25:04.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.888+0000 7f288d7fa700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c066f30 0x7f287c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:04.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.889+0000 7f288d7fa700 1 -- 172.21.15.45:0/1007332801 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8700+0+0 (secure 0 0 0) 0x7f2888086840 con 0x7f28980b0610 2023-10-06T05:25:04.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.889+0000 7f289d360700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c066f30 0x7f287c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:04.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.889+0000 7f289d360700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c066f30 0x7f287c0693e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2894009f30 tx=0x7f2894009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:04.893+0000 7f288d7fa700 1 -- 172.21.15.45:0/1007332801 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2888052e60 con 0x7f28980b0610 2023-10-06T05:25:05.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.147+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f289806e350 con 0x7f28980b0610 2023-10-06T05:25:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.388+0000 7f288d7fa700 1 -- 172.21.15.45:0/1007332801 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v259) v1 ==== 162+0+0 (secure 0 0 0) 0x7f28880150b0 con 0x7f28980b0610 2023-10-06T05:25:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c066f30 msgr2=0x7f287c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c066f30 0x7f287c0693e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2894009f30 tx=0x7f2894009120 comp rx=0 tx=0).stop 2023-10-06T05:25:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980b0610 msgr2=0x7f289815e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980b0610 0x7f289815e600 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2888009f00 tx=0x7f2888017000 comp rx=0 tx=0).stop 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 shutdown_connections 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c066f30 0x7f287c0693e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28980c3db0 0x7f28980c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2898069d00 0x7f289815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 --2- 172.21.15.45:0/1007332801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980b0610 0x7f289815e600 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 >> 172.21.15.45:0/1007332801 conn(0x7f2898067e00 msgr2=0x7f28980c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 shutdown_connections 2023-10-06T05:25:05.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.391+0000 7f289f5c4700 1 -- 172.21.15.45:0/1007332801 wait complete. 2023-10-06T05:25:05.395 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:25:05.403 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:05.402+0000 7effb0706700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer 2023-10-06T05:25:05.404 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:05.403+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:05.405 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:05.403+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:25:05.410 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:05.414 INFO:tasks.ceph.mds.b.smithi144.stderr: -29> 2023-10-06T05:25:05.402+0000 7effb0706700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer 2023-10-06T05:25:05.433 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:25:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.543+0000 7f2894a57700 1 Processor -- start 2023-10-06T05:25:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.543+0000 7f2894a57700 1 -- start start 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.544+0000 7f2894a57700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28900c8ff0 0x7f28900c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.544+0000 7f2894a57700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2890157200 0x7f28900caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f2894a57700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289015b000 con 0x7f28900caad0 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f2894a57700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289015b2e0 con 0x7f28900c8ff0 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f2894a57700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289015b5c0 con 0x7f2890157200 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28900c8ff0 0x7f28900c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28900caad0 0x7f2890156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56878/0 (socket says 172.21.15.45:56878) 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288dd9b700 1 -- 172.21.15.45:0/3977743691 learned_addr learned my addr 172.21.15.45:0/3977743691 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288ed9d700 1 --2- 172.21.15.45:0/3977743691 >> v2:172.21.15.144:3300/0 conn(0x7f2890157200 0x7f28900caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288d59a700 1 -- 172.21.15.45:0/3977743691 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759780247 0 0) 0x7f289015b000 con 0x7f28900caad0 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288d59a700 1 -- 172.21.15.45:0/3977743691 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2878003570 con 0x7f28900caad0 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288e59c700 1 -- 172.21.15.45:0/3977743691 >> v2:172.21.15.144:3300/0 conn(0x7f2890157200 msgr2=0x7f28900caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288e59c700 1 --2- 172.21.15.45:0/3977743691 >> v2:172.21.15.144:3300/0 conn(0x7f2890157200 0x7f28900caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288e59c700 1 -- 172.21.15.45:0/3977743691 >> v1:172.21.15.45:6789/0 conn(0x7f28900caad0 legacy=0x7f2890156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288e59c700 1 -- 172.21.15.45:0/3977743691 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289015b8a0 con 0x7f28900c8ff0 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288e59c700 1 --2- 172.21.15.45:0/3977743691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900c93c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f289015abf0 tx=0x7f28800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deddfd9c0515e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.545+0000 7f288d59a700 1 -- 172.21.15.45:0/3977743691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880011630 con 0x7f28900c8ff0 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f288d59a700 1 -- 172.21.15.45:0/3977743691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800117b0 con 0x7f28900c8ff0 2023-10-06T05:25:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f288d59a700 1 -- 172.21.15.45:0/3977743691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880011b70 con 0x7f28900c8ff0 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 -- 172.21.15.45:0/3977743691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 msgr2=0x7f28900c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 --2- 172.21.15.45:0/3977743691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900c93c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f289015abf0 tx=0x7f28800091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 -- 172.21.15.45:0/3977743691 shutdown_connections 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 --2- 172.21.15.45:0/3977743691 >> v2:172.21.15.144:3300/0 conn(0x7f2890157200 0x7f28900caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 --2- 172.21.15.45:0/3977743691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900c93c0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 -- 172.21.15.45:0/3977743691 >> 172.21.15.45:0/3977743691 conn(0x7f28900cdf50 msgr2=0x7f28900ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 -- 172.21.15.45:0/3977743691 shutdown_connections 2023-10-06T05:25:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.546+0000 7f2894a57700 1 -- 172.21.15.45:0/3977743691 wait complete. 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.547+0000 7f2894a57700 1 Processor -- start 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.547+0000 7f2894a57700 1 -- start start 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f2894a57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f2894a57700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28900caad0 0x7f2890159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f2894a57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2890157200 0x7f2890169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f2894a57700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28900b3ba0 con 0x7f28900c8ff0 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f2894a57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28901465a0 con 0x7f2890157200 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f2894a57700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289015a450 con 0x7f28900caad0 2023-10-06T05:25:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f288e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f288e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45322/0 (socket says 172.21.15.45:45322) 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f288e59c700 1 -- 172.21.15.45:0/482957061 learned_addr learned my addr 172.21.15.45:0/482957061 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f288dd9b700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28900caad0 0x7f2890159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.548+0000 7f288ed9d700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2890157200 0x7f2890169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f288e59c700 1 -- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28900caad0 msgr2=0x7f2890159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f288e59c700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28900caad0 0x7f2890159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f288e59c700 1 -- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2890157200 msgr2=0x7f2890169ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f288e59c700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2890157200 0x7f2890169ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f288e59c700 1 -- 172.21.15.45:0/482957061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880009420 con 0x7f28900c8ff0 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f288e59c700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900b2c40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2880013000 tx=0x7f2880017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f287e7fc700 1 -- 172.21.15.45:0/482957061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288001f070 con 0x7f28900c8ff0 2023-10-06T05:25:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f287e7fc700 1 -- 172.21.15.45:0/482957061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2880000bc0 con 0x7f28900c8ff0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f287e7fc700 1 -- 172.21.15.45:0/482957061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880011b50 con 0x7f28900c8ff0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28900b3300 con 0x7f28900c8ff0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.549+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28900b38c0 con 0x7f28900c8ff0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.550+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289006f410 con 0x7f28900c8ff0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.550+0000 7f287e7fc700 1 -- 172.21.15.45:0/482957061 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2880023070 con 0x7f28900c8ff0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.551+0000 7f287e7fc700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2870066f30 0x7f28700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.551+0000 7f287e7fc700 1 -- 172.21.15.45:0/482957061 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f28800862e0 con 0x7f28900c8ff0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.551+0000 7f288dd9b700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2870066f30 0x7f28700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.552+0000 7f288dd9b700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2870066f30 0x7f28700693e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f289014cee0 tx=0x7f2884009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.556+0000 7f287e7fc700 1 -- 172.21.15.45:0/482957061 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2880052900 con 0x7f28900c8ff0 2023-10-06T05:25:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.805+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f289004bb40 con 0x7f28900c8ff0 2023-10-06T05:25:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.806+0000 7f287e7fc700 1 -- 172.21.15.45:0/482957061 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f288002a090 con 0x7f28900c8ff0 2023-10-06T05:25:05.808 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:05.808 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":261,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:05.404039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2870066f30 msgr2=0x7f28700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2870066f30 0x7f28700693e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f289014cee0 tx=0x7f2884009040 comp rx=0 tx=0).stop 2023-10-06T05:25:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 msgr2=0x7f28900b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900b2c40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2880013000 tx=0x7f2880017000 comp rx=0 tx=0).stop 2023-10-06T05:25:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 shutdown_connections 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2870066f30 0x7f28700693e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2890157200 0x7f2890169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28900c8ff0 0x7f28900b2c40 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 --2- 172.21.15.45:0/482957061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28900caad0 0x7f2890159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 >> 172.21.15.45:0/482957061 conn(0x7f28900cdf50 msgr2=0x7f28900afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 shutdown_connections 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.808+0000 7f2894a57700 1 -- 172.21.15.45:0/482957061 wait complete. 2023-10-06T05:25:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 261 2023-10-06T05:25:05.823 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:05.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:05.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.954+0000 7fa9a5674700 1 Processor -- start 2023-10-06T05:25:05.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.954+0000 7fa9a5674700 1 -- start start 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa9a5674700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa9a0159170 0x7fa9a0159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa9a5674700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9a00692d0 0x7fa9a00aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa9a5674700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a00ca6d0 con 0x7fa9a015d040 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa9a5674700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a015d550 con 0x7fa9a00692d0 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa9a5674700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a015d810 con 0x7fa9a0159170 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9a00692d0 0x7fa9a00aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9a00692d0 0x7fa9a00aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45326/0 (socket says 172.21.15.45:45326) 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99f7fe700 1 -- 172.21.15.45:0/3385833472 learned_addr learned my addr 172.21.15.45:0/3385833472 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:05.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99effd700 1 --2- 172.21.15.45:0/3385833472 >> v2:172.21.15.144:3300/0 conn(0x7fa9a0159170 0x7fa9a0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99f7fe700 1 -- 172.21.15.45:0/3385833472 >> v2:172.21.15.144:3300/0 conn(0x7fa9a0159170 msgr2=0x7fa9a0159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99f7fe700 1 --2- 172.21.15.45:0/3385833472 >> v2:172.21.15.144:3300/0 conn(0x7fa9a0159170 0x7fa9a0159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99f7fe700 1 -- 172.21.15.45:0/3385833472 >> v1:172.21.15.144:6789/0 conn(0x7fa9a015d040 legacy=0x7fa9a0068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99f7fe700 1 -- 172.21.15.45:0/3385833472 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a015daf0 con 0x7fa9a00692d0 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.955+0000 7fa99effd700 1 --2- 172.21.15.45:0/3385833472 >> v2:172.21.15.144:3300/0 conn(0x7fa9a0159170 0x7fa9a0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa99f7fe700 1 --2- 172.21.15.45:0/3385833472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a00aea60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0068a90 tx=0x7fa998009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f400015008b01d8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa99dffb700 1 -- 172.21.15.45:0/3385833472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99800e070 con 0x7fa9a00692d0 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa99dffb700 1 -- 172.21.15.45:0/3385833472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99800ba50 con 0x7fa9a00692d0 2023-10-06T05:25:05.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa99dffb700 1 -- 172.21.15.45:0/3385833472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9980092e0 con 0x7fa9a00692d0 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 -- 172.21.15.45:0/3385833472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 msgr2=0x7fa9a00aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 --2- 172.21.15.45:0/3385833472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a00aea60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0068a90 tx=0x7fa998009840 comp rx=0 tx=0).stop 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 -- 172.21.15.45:0/3385833472 shutdown_connections 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 --2- 172.21.15.45:0/3385833472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a00aea60 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 --2- 172.21.15.45:0/3385833472 >> v2:172.21.15.144:3300/0 conn(0x7fa9a0159170 0x7fa9a0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 -- 172.21.15.45:0/3385833472 >> 172.21.15.45:0/3385833472 conn(0x7fa9a006f7c0 msgr2=0x7fa9a00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 -- 172.21.15.45:0/3385833472 shutdown_connections 2023-10-06T05:25:05.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.956+0000 7fa9a5674700 1 -- 172.21.15.45:0/3385833472 wait complete. 2023-10-06T05:25:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.958+0000 7fa9a5674700 1 Processor -- start 2023-10-06T05:25:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.958+0000 7fa9a5674700 1 -- start start 2023-10-06T05:25:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa9a5674700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa9a5674700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a0159170 0x7fa9a00c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa9a5674700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a015d040 0x7fa9a00cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa9a5674700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a015cb10 con 0x7fa9a00692d0 2023-10-06T05:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa9a5674700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00cce10 con 0x7fa9a0159170 2023-10-06T05:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa9a5674700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00cc200 con 0x7fa9a015d040 2023-10-06T05:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45334/0 (socket says 172.21.15.45:45334) 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 -- 172.21.15.45:0/2047056136 learned_addr learned my addr 172.21.15.45:0/2047056136 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99f7fe700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a015d040 0x7fa9a00cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99e7fc700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a0159170 0x7fa9a00c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 -- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a015d040 msgr2=0x7fa9a00cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a015d040 0x7fa9a00cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 -- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a0159170 msgr2=0x7fa9a00c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a0159170 0x7fa9a00c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 -- 172.21.15.45:0/2047056136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998009ae0 con 0x7fa9a00692d0 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa99effd700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a015c5d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa98800e8d0 tx=0x7fa988008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.959+0000 7fa996ffd700 1 -- 172.21.15.45:0/2047056136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800ee70 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.960+0000 7fa996ffd700 1 -- 172.21.15.45:0/2047056136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98800c5e0 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.960+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a00cc4e0 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.960+0000 7fa996ffd700 1 -- 172.21.15.45:0/2047056136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800c980 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.960+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a00cc870 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.960+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a0063b50 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.961+0000 7fa996ffd700 1 -- 172.21.15.45:0/2047056136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa988013410 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.962+0000 7fa996ffd700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980066f30 0x7fa9800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.962+0000 7fa996ffd700 1 -- 172.21.15.45:0/2047056136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7fa988084fe0 con 0x7fa9a00692d0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.962+0000 7fa99e7fc700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980066f30 0x7fa9800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.962+0000 7fa99e7fc700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980066f30 0x7fa9800693e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa9a015c010 tx=0x7fa990009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:05.966+0000 7fa996ffd700 1 -- 172.21.15.45:0/2047056136 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa988055090 con 0x7fa9a00692d0 2023-10-06T05:25:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.218+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9a015ddd0 con 0x7fa9a00692d0 2023-10-06T05:25:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.219+0000 7fa996ffd700 1 -- 172.21.15.45:0/2047056136 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) v1 ==== 95+0+3839 (secure 0 0 0) 0x7fa988054c20 con 0x7fa9a00692d0 2023-10-06T05:25:06.221 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:06.221 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":261,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:05.404039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980066f30 msgr2=0x7fa9800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980066f30 0x7fa9800693e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa9a015c010 tx=0x7fa990009040 comp rx=0 tx=0).stop 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 msgr2=0x7fa9a015c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a015c5d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa98800e8d0 tx=0x7fa988008040 comp rx=0 tx=0).stop 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 shutdown_connections 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa980066f30 0x7fa9800693e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a015d040 0x7fa9a00cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a00692d0 0x7fa9a015c5d0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 --2- 172.21.15.45:0/2047056136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a0159170 0x7fa9a00c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 >> 172.21.15.45:0/2047056136 conn(0x7fa9a006f7c0 msgr2=0x7fa9a00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 shutdown_connections 2023-10-06T05:25:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.221+0000 7fa9a5674700 1 -- 172.21.15.45:0/2047056136 wait complete. 2023-10-06T05:25:06.224 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 261 2023-10-06T05:25:06.236 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:06.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.366+0000 7f24dd2d1700 1 Processor -- start 2023-10-06T05:25:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.367+0000 7f24dd2d1700 1 -- start start 2023-10-06T05:25:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.367+0000 7f24dd2d1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24d80692d0 0x7f24d8159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.367+0000 7f24dd2d1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f24d815d040 0x7f24d80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.367+0000 7f24dd2d1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d80ca6d0 con 0x7f24d8159170 2023-10-06T05:25:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.367+0000 7f24dd2d1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d815d550 con 0x7f24d80692d0 2023-10-06T05:25:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.367+0000 7f24dd2d1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d815d810 con 0x7f24d815d040 2023-10-06T05:25:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24d80692d0 0x7f24d8159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24d80692d0 0x7f24d8159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43262/0 (socket says 172.21.15.45:43262) 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 -- 172.21.15.45:0/3613019267 learned_addr learned my addr 172.21.15.45:0/3613019267 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d77fe700 1 --2- 172.21.15.45:0/3613019267 >> v2:172.21.15.144:3300/0 conn(0x7f24d815d040 0x7f24d80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 -- 172.21.15.45:0/3613019267 >> v2:172.21.15.144:3300/0 conn(0x7f24d815d040 msgr2=0x7f24d80aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 --2- 172.21.15.45:0/3613019267 >> v2:172.21.15.144:3300/0 conn(0x7f24d815d040 0x7f24d80aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 -- 172.21.15.45:0/3613019267 >> v1:172.21.15.144:6789/0 conn(0x7f24d8159170 legacy=0x7f24d8068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 -- 172.21.15.45:0/3613019267 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d815daf0 con 0x7f24d80692d0 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d67fc700 1 --2- 172.21.15.45:0/3613019267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d80692d0 0x7f24d8159540 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f24d80c70c0 tx=0x7f24cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38830f6366c4e62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d5ffb700 1 -- 172.21.15.45:0/3613019267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00b820 con 0x7f24d80692d0 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d5ffb700 1 -- 172.21.15.45:0/3613019267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc00b980 con 0x7f24d80692d0 2023-10-06T05:25:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.368+0000 7f24d5ffb700 1 -- 172.21.15.45:0/3613019267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00bd20 con 0x7f24d80692d0 2023-10-06T05:25:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 -- 172.21.15.45:0/3613019267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d80692d0 msgr2=0x7f24d8159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/3613019267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d80692d0 0x7f24d8159540 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f24d80c70c0 tx=0x7f24cc009250 comp rx=0 tx=0).stop 2023-10-06T05:25:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 -- 172.21.15.45:0/3613019267 shutdown_connections 2023-10-06T05:25:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/3613019267 >> v2:172.21.15.144:3300/0 conn(0x7f24d815d040 0x7f24d80aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/3613019267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d80692d0 0x7f24d8159540 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 -- 172.21.15.45:0/3613019267 >> 172.21.15.45:0/3613019267 conn(0x7f24d806f7c0 msgr2=0x7f24d80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 -- 172.21.15.45:0/3613019267 shutdown_connections 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.369+0000 7f24dd2d1700 1 -- 172.21.15.45:0/3613019267 wait complete. 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.370+0000 7f24dd2d1700 1 Processor -- start 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.370+0000 7f24dd2d1700 1 -- start start 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24dd2d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d80692d0 0x7f24d80c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24dd2d1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8159170 0x7f24d815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24dd2d1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24d815d040 0x7f24d8169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24dd2d1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80c8c90 con 0x7f24d8159170 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24dd2d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80c8080 con 0x7f24d80692d0 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24dd2d1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80c8360 con 0x7f24d815d040 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24d67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8159170 0x7f24d815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24d67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8159170 0x7f24d815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43268/0 (socket says 172.21.15.45:43268) 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24d67fc700 1 -- 172.21.15.45:0/57422089 learned_addr learned my addr 172.21.15.45:0/57422089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24d77fe700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24d815d040 0x7f24d8169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.371+0000 7f24d6ffd700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d80692d0 0x7f24d80c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24d67fc700 1 -- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24d815d040 msgr2=0x7f24d8169520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24d67fc700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24d815d040 0x7f24d8169520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24d67fc700 1 -- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d80692d0 msgr2=0x7f24d80c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24d67fc700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d80692d0 0x7f24d80c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24d67fc700 1 -- 172.21.15.45:0/57422089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0094f0 con 0x7f24d8159170 2023-10-06T05:25:06.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24d67fc700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8159170 0x7f24d815a3d0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0030c0 tx=0x7f24cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24beffd700 1 -- 172.21.15.45:0/57422089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc01b470 con 0x7f24d8159170 2023-10-06T05:25:06.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24beffd700 1 -- 172.21.15.45:0/57422089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc01b5f0 con 0x7f24d8159170 2023-10-06T05:25:06.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d80c8640 con 0x7f24d8159170 2023-10-06T05:25:06.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24beffd700 1 -- 172.21.15.45:0/57422089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc01b9b0 con 0x7f24d8159170 2023-10-06T05:25:06.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.372+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d80c62f0 con 0x7f24d8159170 2023-10-06T05:25:06.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.373+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d80c8e00 con 0x7f24d8159170 2023-10-06T05:25:06.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.374+0000 7f24beffd700 1 -- 172.21.15.45:0/57422089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24cc01c450 con 0x7f24d8159170 2023-10-06T05:25:06.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.374+0000 7f24beffd700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f24c4066f30 0x7f24c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.374+0000 7f24beffd700 1 -- 172.21.15.45:0/57422089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f24cc086da0 con 0x7f24d8159170 2023-10-06T05:25:06.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.374+0000 7f24d6ffd700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f24c4066f30 0x7f24c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.375+0000 7f24d6ffd700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f24c4066f30 0x7f24c40693e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f24c8003a30 tx=0x7f24c8007ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.379+0000 7f24beffd700 1 -- 172.21.15.45:0/57422089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24cc0533c0 con 0x7f24d8159170 2023-10-06T05:25:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.629+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24d8063b50 con 0x7f24d8159170 2023-10-06T05:25:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.629+0000 7f24beffd700 1 -- 172.21.15.45:0/57422089 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 262 v262) v1 ==== 95+0+4597 (secure 0 0 0) 0x7f24cc02a070 con 0x7f24d8159170 2023-10-06T05:25:06.632 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:06.632 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":262,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":9268,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/243455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":243455709},{"type":"v1","addr":"172.21.15.144:6835","nonce":243455709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:05.404039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f24c4066f30 msgr2=0x7f24c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f24c4066f30 0x7f24c40693e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f24c8003a30 tx=0x7f24c8007ea0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8159170 msgr2=0x7f24d815a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8159170 0x7f24d815a3d0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0030c0 tx=0x7f24cc016000 comp rx=0 tx=0).stop 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 shutdown_connections 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f24c4066f30 0x7f24c40693e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24d815d040 0x7f24d8169520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d80692d0 0x7f24d80c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 --2- 172.21.15.45:0/57422089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d8159170 0x7f24d815a3d0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 >> 172.21.15.45:0/57422089 conn(0x7f24d806f7c0 msgr2=0x7f24d8158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 shutdown_connections 2023-10-06T05:25:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.632+0000 7f24dd2d1700 1 -- 172.21.15.45:0/57422089 wait complete. 2023-10-06T05:25:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 262 2023-10-06T05:25:06.649 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2023-10-06T05:25:06.650 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2023-10-06T05:25:06.650 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:06.650 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:25:06.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:06.650 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:06.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:25:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:25:06.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:06.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.846+0000 7f72d1948700 1 Processor -- start 2023-10-06T05:25:06.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.846+0000 7f72d1948700 1 -- start start 2023-10-06T05:25:06.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72d1948700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f72cc159170 0x7f72cc159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72d1948700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc15d040 0x7f72cc068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72d1948700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72cc069b90 0x7f72cc069100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72d1948700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc0ca6d0 con 0x7f72cc069b90 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72d1948700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc15d550 con 0x7f72cc159170 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72d1948700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc15d810 con 0x7f72cc15d040 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72cc069b90 0x7f72cc069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72cc069b90 0x7f72cc069100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43274/0 (socket says 172.21.15.45:43274) 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2216659312 learned_addr learned my addr 172.21.15.45:0/2216659312 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72caffd700 1 --2- 172.21.15.45:0/2216659312 >> v2:172.21.15.144:3300/0 conn(0x7f72cc159170 0x7f72cc159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72ca7fc700 1 --2- 172.21.15.45:0/2216659312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc15d040 0x7f72cc068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2216659312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc15d040 msgr2=0x7f72cc068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 --2- 172.21.15.45:0/2216659312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc15d040 0x7f72cc068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2216659312 >> v2:172.21.15.144:3300/0 conn(0x7f72cc159170 msgr2=0x7f72cc159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 --2- 172.21.15.45:0/2216659312 >> v2:172.21.15.144:3300/0 conn(0x7f72cc159170 0x7f72cc159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2216659312 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc15daf0 con 0x7f72cc069b90 2023-10-06T05:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.847+0000 7f72cb7fe700 1 --2- 172.21.15.45:0/2216659312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc069b90 0x7f72cc069100 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f72cc068a90 tx=0x7f72c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a7dc95b26740b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72c9ffb700 1 -- 172.21.15.45:0/2216659312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400e070 con 0x7f72cc069b90 2023-10-06T05:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72c9ffb700 1 -- 172.21.15.45:0/2216659312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c400b970 con 0x7f72cc069b90 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72c9ffb700 1 -- 172.21.15.45:0/2216659312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c40092e0 con 0x7f72cc069b90 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 -- 172.21.15.45:0/2216659312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc069b90 msgr2=0x7f72cc069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 --2- 172.21.15.45:0/2216659312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc069b90 0x7f72cc069100 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f72cc068a90 tx=0x7f72c4009840 comp rx=0 tx=0).stop 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 -- 172.21.15.45:0/2216659312 shutdown_connections 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 --2- 172.21.15.45:0/2216659312 >> v2:172.21.15.144:3300/0 conn(0x7f72cc159170 0x7f72cc159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 --2- 172.21.15.45:0/2216659312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc069b90 0x7f72cc069100 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 --2- 172.21.15.45:0/2216659312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc15d040 0x7f72cc068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 -- 172.21.15.45:0/2216659312 >> 172.21.15.45:0/2216659312 conn(0x7f72cc06f7c0 msgr2=0x7f72cc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 -- 172.21.15.45:0/2216659312 shutdown_connections 2023-10-06T05:25:06.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.848+0000 7f72d1948700 1 -- 172.21.15.45:0/2216659312 wait complete. 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 Processor -- start 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 -- start start 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc069b90 0x7f72cc151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc159170 0x7f72cc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc15d040 0x7f72cc156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc15a3b0 con 0x7f72cc15d040 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0b3250 con 0x7f72cc069b90 2023-10-06T05:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.850+0000 7f72d1948700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0b3530 con 0x7f72cc159170 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc15d040 0x7f72cc156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc15d040 0x7f72cc156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43280/0 (socket says 172.21.15.45:43280) 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2328575507 learned_addr learned my addr 172.21.15.45:0/2328575507 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72ca7fc700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc159170 0x7f72cc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72caffd700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc069b90 0x7f72cc151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc159170 msgr2=0x7f72cc0b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc159170 0x7f72cc0b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc069b90 msgr2=0x7f72cc151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc069b90 0x7f72cc151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 -- 172.21.15.45:0/2328575507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4009ae0 con 0x7f72cc15d040 2023-10-06T05:25:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72cb7fe700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc15d040 0x7f72cc156640 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f72cc088b80 tx=0x7f72c4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72b2ffd700 1 -- 172.21.15.45:0/2328575507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4008b80 con 0x7f72cc15d040 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72b2ffd700 1 -- 172.21.15.45:0/2328575507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4008ce0 con 0x7f72cc15d040 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc0b3810 con 0x7f72cc15d040 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72b2ffd700 1 -- 172.21.15.45:0/2328575507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c40266b0 con 0x7f72cc15d040 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.851+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc0b2ee0 con 0x7f72cc15d040 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.852+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc063b50 con 0x7f72cc15d040 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.853+0000 7f72b2ffd700 1 -- 172.21.15.45:0/2328575507 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72c4003040 con 0x7f72cc15d040 2023-10-06T05:25:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.853+0000 7f72b2ffd700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.853+0000 7f72b2ffd700 1 -- 172.21.15.45:0/2328575507 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f72c4086fd0 con 0x7f72cc15d040 2023-10-06T05:25:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.853+0000 7f72caffd700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.854+0000 7f72caffd700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72b8066f30 0x7f72b80693e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0af130 tx=0x7f72bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:06.858+0000 7f72b2ffd700 1 -- 172.21.15.45:0/2328575507 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72c40536a0 con 0x7f72cc15d040 2023-10-06T05:25:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.107+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f72cc15ddd0 con 0x7f72cc15d040 2023-10-06T05:25:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.413+0000 7f72b2ffd700 1 -- 172.21.15.45:0/2328575507 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v263) v1 ==== 161+0+0 (secure 0 0 0) 0x7f72c4056cc0 con 0x7f72cc15d040 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72b8066f30 msgr2=0x7f72b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72b8066f30 0x7f72b80693e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0af130 tx=0x7f72bc008040 comp rx=0 tx=0).stop 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc15d040 msgr2=0x7f72cc156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc15d040 0x7f72cc156640 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f72cc088b80 tx=0x7f72c4018000 comp rx=0 tx=0).stop 2023-10-06T05:25:07.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 shutdown_connections 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72cc15d040 0x7f72cc156640 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc069b90 0x7f72cc151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 --2- 172.21.15.45:0/2328575507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72cc159170 0x7f72cc0b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 >> 172.21.15.45:0/2328575507 conn(0x7f72cc06f7c0 msgr2=0x7f72cc068bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 shutdown_connections 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.416+0000 7f72d1948700 1 -- 172.21.15.45:0/2328575507 wait complete. 2023-10-06T05:25:07.419 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:07.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:25:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.569+0000 7f73475e1700 1 Processor -- start 2023-10-06T05:25:07.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.570+0000 7f73475e1700 1 -- start start 2023-10-06T05:25:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.570+0000 7f73475e1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f73400cc550 0x7f73400cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.570+0000 7f73475e1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f734015d0b0 0x7f734015d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.570+0000 7f73475e1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7340161f40 con 0x7f73400caa70 2023-10-06T05:25:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.570+0000 7f73475e1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73400cb5d0 con 0x7f734015d0b0 2023-10-06T05:25:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.570+0000 7f73475e1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7340161330 con 0x7f73400cc550 2023-10-06T05:25:07.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7344b7c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f734015d0b0 0x7f734015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7345b7e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f73400caa70 0x7f73400caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39578/0 (socket says 172.21.15.45:39578) 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7345b7e700 1 -- 172.21.15.45:0/763512955 learned_addr learned my addr 172.21.15.45:0/763512955 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f734537d700 1 --2- 172.21.15.45:0/763512955 >> v2:172.21.15.144:3300/0 conn(0x7f73400cc550 0x7f73400cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7344b7c700 1 -- 172.21.15.45:0/763512955 >> v2:172.21.15.144:3300/0 conn(0x7f73400cc550 msgr2=0x7f73400cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7344b7c700 1 --2- 172.21.15.45:0/763512955 >> v2:172.21.15.144:3300/0 conn(0x7f73400cc550 0x7f73400cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7344b7c700 1 -- 172.21.15.45:0/763512955 >> v1:172.21.15.45:6789/0 conn(0x7f73400caa70 legacy=0x7f73400caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7344b7c700 1 -- 172.21.15.45:0/763512955 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7340161610 con 0x7f734015d0b0 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7344b7c700 1 --2- 172.21.15.45:0/763512955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f734015d520 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f734006ed80 tx=0x7f733c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f213304dcb47aafe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7337fff700 1 -- 172.21.15.45:0/763512955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00b6b0 con 0x7f734015d0b0 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7337fff700 1 -- 172.21.15.45:0/763512955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c00b810 con 0x7f734015d0b0 2023-10-06T05:25:07.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.571+0000 7f7337fff700 1 -- 172.21.15.45:0/763512955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00bbb0 con 0x7f734015d0b0 2023-10-06T05:25:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 -- 172.21.15.45:0/763512955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 msgr2=0x7f734015d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 --2- 172.21.15.45:0/763512955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f734015d520 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f734006ed80 tx=0x7f733c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 -- 172.21.15.45:0/763512955 shutdown_connections 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 --2- 172.21.15.45:0/763512955 >> v2:172.21.15.144:3300/0 conn(0x7f73400cc550 0x7f73400cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 --2- 172.21.15.45:0/763512955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f734015d520 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 -- 172.21.15.45:0/763512955 >> 172.21.15.45:0/763512955 conn(0x7f73400cdf50 msgr2=0x7f73400ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 -- 172.21.15.45:0/763512955 shutdown_connections 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.572+0000 7f73475e1700 1 -- 172.21.15.45:0/763512955 wait complete. 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.573+0000 7f73475e1700 1 Processor -- start 2023-10-06T05:25:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.573+0000 7f73475e1700 1 -- start start 2023-10-06T05:25:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f73475e1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73400caa70 0x7f73400b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f73475e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73400cc550 0x7f73401522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f73475e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f7340156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f73475e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7340069d50 con 0x7f734015d0b0 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f73475e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400b1780 con 0x7f73400cc550 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f73475e1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7340152810 con 0x7f73400caa70 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f7345b7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f7340156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f7345b7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f7340156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43308/0 (socket says 172.21.15.45:43308) 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f7345b7e700 1 -- 172.21.15.45:0/1732597724 learned_addr learned my addr 172.21.15.45:0/1732597724 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.574+0000 7f7344b7c700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73400cc550 0x7f73401522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f734537d700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73400caa70 0x7f73400b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f7345b7e700 1 -- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73400caa70 msgr2=0x7f73400b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f7345b7e700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73400caa70 0x7f73400b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f7345b7e700 1 -- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73400cc550 msgr2=0x7f73401522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f7345b7e700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73400cc550 0x7f73401522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f7345b7e700 1 -- 172.21.15.45:0/1732597724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c009420 con 0x7f734015d0b0 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f734537d700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73400caa70 0x7f73400b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f7345b7e700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f7340156620 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f733800cc70 tx=0x7f7338001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f73357fa700 1 -- 172.21.15.45:0/1732597724 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800a900 con 0x7f734015d0b0 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f73357fa700 1 -- 172.21.15.45:0/1732597724 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733800aa80 con 0x7f734015d0b0 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73400b0e90 con 0x7f734015d0b0 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f73357fa700 1 -- 172.21.15.45:0/1732597724 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800ae40 con 0x7f734015d0b0 2023-10-06T05:25:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.575+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73400b1130 con 0x7f734015d0b0 2023-10-06T05:25:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.576+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73400b18f0 con 0x7f734015d0b0 2023-10-06T05:25:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.577+0000 7f73357fa700 1 -- 172.21.15.45:0/1732597724 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7338007300 con 0x7f734015d0b0 2023-10-06T05:25:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.577+0000 7f73357fa700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7324066f60 0x7f7324069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.577+0000 7f73357fa700 1 -- 172.21.15.45:0/1732597724 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f7338085410 con 0x7f734015d0b0 2023-10-06T05:25:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.577+0000 7f734537d700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7324066f60 0x7f7324069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.578+0000 7f734537d700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7324066f60 0x7f7324069410 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7330006fd0 tx=0x7f7330008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.582+0000 7f73357fa700 1 -- 172.21.15.45:0/1732597724 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7338051ab0 con 0x7f734015d0b0 2023-10-06T05:25:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:07.871+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f73401618f0 con 0x7f734015d0b0 2023-10-06T05:25:08.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:08.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.418+0000 7f73357fa700 1 -- 172.21.15.45:0/1732597724 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v264) v1 ==== 101+0+0 (secure 0 0 0) 0x7f73380550d0 con 0x7f734015d0b0 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7324066f60 msgr2=0x7f7324069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7324066f60 0x7f7324069410 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7330006fd0 tx=0x7f7330008040 comp rx=0 tx=0).stop 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 msgr2=0x7f7340156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f7340156620 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f733800cc70 tx=0x7f7338001a50 comp rx=0 tx=0).stop 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 shutdown_connections 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7324066f60 0x7f7324069410 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734015d0b0 0x7f7340156620 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73400caa70 0x7f73400b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 --2- 172.21.15.45:0/1732597724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73400cc550 0x7f73401522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 >> 172.21.15.45:0/1732597724 conn(0x7f73400cdf50 msgr2=0x7f73400c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 shutdown_connections 2023-10-06T05:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.421+0000 7f73475e1700 1 -- 172.21.15.45:0/1732597724 wait complete. 2023-10-06T05:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:25:08.438 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:25:08.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.484+0000 7fe3dd734040 1 Processor -- start 2023-10-06T05:25:08.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.484+0000 7fe3dd734040 1 -- start start 2023-10-06T05:25:08.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3dd734040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6abfe0 0x55d3ac786630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3dd734040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac6f7340 0x55d3ac6b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3dd734040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac76d970 0x55d3ac7828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3dd734040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ac5f8980 con 0x55d3ac6abfe0 2023-10-06T05:25:08.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3dd734040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ac5f8620 con 0x55d3ac6f7340 2023-10-06T05:25:08.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3dd734040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3ac5ef400 con 0x55d3ac76d970 2023-10-06T05:25:08.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6abfe0 0x55d3ac786630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6abfe0 0x55d3ac786630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43322/0 (socket says 172.21.15.45:43322) 2023-10-06T05:25:08.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 -- 172.21.15.45:0/1212547991 learned_addr learned my addr 172.21.15.45:0/1212547991 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac76d970 0x55d3ac7828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac6f7340 0x55d3ac6b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 -- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac76d970 msgr2=0x55d3ac7828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac76d970 0x55d3ac7828c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 -- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac6f7340 msgr2=0x55d3ac6b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac6f7340 0x55d3ac6b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 -- 172.21.15.45:0/1212547991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc005cf0 con 0x55d3ac6abfe0 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac6f7340 0x55d3ac6b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6abfe0 0x55d3ac786630 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b4d0 tx=0x7fe3c8001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c6d3f4f1f3762a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d514f700 1 -- 172.21.15.45:0/1212547991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8014070 con 0x55d3ac6abfe0 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.485+0000 7fe3d514f700 1 -- 172.21.15.45:0/1212547991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c800d500 con 0x55d3ac6abfe0 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3d514f700 1 -- 172.21.15.45:0/1212547991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800d8c0 con 0x55d3ac6abfe0 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 -- 172.21.15.45:0/1212547991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6abfe0 msgr2=0x55d3ac786630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6abfe0 0x55d3ac786630 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b4d0 tx=0x7fe3c8001a60 comp rx=0 tx=0).stop 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 -- 172.21.15.45:0/1212547991 shutdown_connections 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac76d970 0x55d3ac7828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6abfe0 0x55d3ac786630 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1212547991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac6f7340 0x55d3ac6b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 -- 172.21.15.45:0/1212547991 >> 172.21.15.45:0/1212547991 conn(0x55d3ac6ae4a0 msgr2=0x55d3ac6ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 -- 172.21.15.45:0/1212547991 shutdown_connections 2023-10-06T05:25:08.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.486+0000 7fe3dd734040 1 -- 172.21.15.45:0/1212547991 wait complete. 2023-10-06T05:25:08.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.487+0000 7fe3dd734040 1 Processor -- start 2023-10-06T05:25:08.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.487+0000 7fe3dd734040 1 -- start start 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3dd734040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac6abfe0 0x55d3ac6dd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3dd734040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6f7340 0x55d3ac6aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3dd734040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac76d970 0x55d3ac788080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3dd734040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ac76d260 con 0x55d3ac6f7340 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3dd734040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ac6dd770 con 0x55d3ac76d970 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3dd734040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3ac784be0 con 0x55d3ac6abfe0 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6f7340 0x55d3ac6aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6f7340 0x55d3ac6aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43338/0 (socket says 172.21.15.45:43338) 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 learned_addr learned my addr 172.21.15.45:0/1895934549 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac6abfe0 0x55d3ac6dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac76d970 0x55d3ac788080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac6abfe0 msgr2=0x55d3ac6dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d3ac6abfe0 0x55d3ac6dd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac76d970 msgr2=0x55d3ac788080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3ac76d970 0x55d3ac788080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8001d00 con 0x55d3ac6f7340 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6f7340 0x55d3ac6aea50 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005610 tx=0x7fe3cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3c77fe700 1 -- 172.21.15.45:0/1895934549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc005370 con 0x55d3ac6f7340 2023-10-06T05:25:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3c77fe700 1 -- 172.21.15.45:0/1895934549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc007610 con 0x55d3ac6f7340 2023-10-06T05:25:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3ac6073d0 con 0x55d3ac6f7340 2023-10-06T05:25:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.488+0000 7fe3c77fe700 1 -- 172.21.15.45:0/1895934549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0079b0 con 0x55d3ac6f7340 2023-10-06T05:25:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.489+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3ac5faae0 con 0x55d3ac6f7340 2023-10-06T05:25:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.490+0000 7fe3c77fe700 1 -- 172.21.15.45:0/1895934549 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3cc01e440 con 0x55d3ac6f7340 2023-10-06T05:25:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.490+0000 7fe3c77fe700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe3bc0658c0 0x7fe3bc067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.490+0000 7fe3c77fe700 1 -- 172.21.15.45:0/1895934549 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7fe3cc084a40 con 0x55d3ac6f7340 2023-10-06T05:25:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.491+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe3bc0658c0 0x7fe3bc067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.499 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:25:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.491+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d3ac6aef90 0x55d3ac797e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.491+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac6f1840 con 0x55d3ac6aef90 2023-10-06T05:25:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.491+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d3ac6aef90 0x55d3ac797e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.491+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d3ac6aef90 0x55d3ac797e00 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.491+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe3bc0658c0 0x7fe3bc067d70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8001aa0 tx=0x7fe3c8014000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.493+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v120'846 uv820 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.493+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac788ad0 con 0x55d3ac6aef90 2023-10-06T05:25:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.495+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v120'847 uv847 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.495+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac798340 con 0x55d3ac6aef90 2023-10-06T05:25:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.496+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v120'848 uv848 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.501 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:25:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.496+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac798720 con 0x55d3ac6aef90 2023-10-06T05:25:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.498+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'849 uv849 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.498+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a20f0 con 0x55d3ac6aef90 2023-10-06T05:25:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.499+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'850 uv850 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.499+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a10e0 con 0x55d3ac6aef90 2023-10-06T05:25:08.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.501+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'851 uv851 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.501+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a1510 con 0x55d3ac6aef90 2023-10-06T05:25:08.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.502+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v120'852 uv826 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.502+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.504+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v120'853 uv853 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.504+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.505+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v120'854 uv854 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.507 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:25:08.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.505+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.507+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'855 uv855 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.507+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.508+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'856 uv856 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.508+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.509+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'857 uv857 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.509+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.511+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v120'858 uv833 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.511+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.512+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'859 uv859 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.512+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.515 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:25:08.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.513+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v120'860 uv860 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.513+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.515+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'861 uv861 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.515+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.516+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'862 uv862 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.516+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a18c0 con 0x55d3ac6aef90 2023-10-06T05:25:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.518+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'863 uv863 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.518+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.519+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'864 uv864 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.519+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.520+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v120'865 uv839 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.520+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.522+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v120'866 uv866 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.522+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.525 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:25:08.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.523+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v120'867 uv867 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.523+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a34b0 con 0x55d3ac6aef90 2023-10-06T05:25:08.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.524+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'868 uv868 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.525+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a34b0 con 0x55d3ac6aef90 2023-10-06T05:25:08.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.526+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'869 uv869 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.526+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.527+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'870 uv870 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.527+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.529+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v120'871 uv845 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.529+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.530+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v120'872 uv872 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.530+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.533 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:25:08.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.532+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v120'873 uv873 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.532+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a34b0 con 0x55d3ac6aef90 2023-10-06T05:25:08.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.533+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'874 uv874 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.533+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a34b0 con 0x55d3ac6aef90 2023-10-06T05:25:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.534+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'875 uv875 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.534+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a24a0 con 0x55d3ac6aef90 2023-10-06T05:25:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.536+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'876 uv876 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac6aef90 2023-10-06T05:25:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.536+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d3ac7a29b0 0x55d3ac7a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.536+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a52d0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.536+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d3ac7a29b0 0x55d3ac7a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.536+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d3ac7a29b0 0x55d3ac7a4d90 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.538+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'611 uv594 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.538+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a5680 con 0x55d3ac7a29b0 2023-10-06T05:25:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.539+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'612 uv612 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.539+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a5a30 con 0x55d3ac7a29b0 2023-10-06T05:25:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.540+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'613 uv613 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.542 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:25:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.540+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a62d0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.542+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v120'614 uv614 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.542+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6680 con 0x55d3ac7a29b0 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.543+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'615 uv615 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.543+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6a30 con 0x55d3ac7a29b0 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.544+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v120'616 uv599 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.544+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.545+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v120'617 uv617 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.546+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.548 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.547+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v120'618 uv618 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.547+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.548+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'619 uv619 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.548+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.549+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'620 uv620 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.549+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.551+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v120'621 uv604 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.551+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.552+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v120'622 uv622 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.552+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.555 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:25:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.553+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v120'623 uv623 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.553+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7aa500 con 0x55d3ac7a29b0 2023-10-06T05:25:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.555+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v120'624 uv624 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.555+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7aa500 con 0x55d3ac7a29b0 2023-10-06T05:25:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.556+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'625 uv625 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.556+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.557+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v120'626 uv610 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.557+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.558+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v120'627 uv627 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.558+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.561 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:25:08.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.559+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v120'628 uv628 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.559+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a7df0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.560+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'629 uv629 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.560+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a7df0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.562+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'630 uv630 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.562+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a6de0 con 0x55d3ac7a29b0 2023-10-06T05:25:08.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.563+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'631 uv631 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.563+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d3ac7a72f0 0x55d3ac7a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.563+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a9c10 con 0x55d3ac7a72f0 2023-10-06T05:25:08.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.563+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d3ac7a72f0 0x55d3ac7a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.564+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d3ac7a72f0 0x55d3ac7a96d0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.565+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.565+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a9fc0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.566+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v120'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.566+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7aa370 con 0x55d3ac7a72f0 2023-10-06T05:25:08.568 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:25:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.567+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v120'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.567+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7a5f00 con 0x55d3ac7a72f0 2023-10-06T05:25:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.568+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.568+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7abdb0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.569+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.569+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7aac60 con 0x55d3ac7a72f0 2023-10-06T05:25:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.571+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.571+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab080 con 0x55d3ac7a72f0 2023-10-06T05:25:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.572+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'117 uv116 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.572+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab080 con 0x55d3ac7a72f0 2023-10-06T05:25:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.573+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'118 uv118 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.573+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab500 con 0x55d3ac7a72f0 2023-10-06T05:25:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.574+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'119 uv119 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.576 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:25:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.574+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab500 con 0x55d3ac7a72f0 2023-10-06T05:25:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.575+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'120 uv120 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.576+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab480 con 0x55d3ac7a29b0 2023-10-06T05:25:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.577+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.577+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab480 con 0x55d3ac7a29b0 2023-10-06T05:25:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.578+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v120'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.578+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7a29b0 2023-10-06T05:25:08.581 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:25:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.579+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v120'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.579+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7a29b0 2023-10-06T05:25:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.581+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.581+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7a29b0 2023-10-06T05:25:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.582+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.582+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7a29b0 2023-10-06T05:25:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.583+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7a29b0 2023-10-06T05:25:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.583+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d3ac7ac670 0x55d3ac7aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.583+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.583+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d3ac7ac670 0x55d3ac7aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.584+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d3ac7ac670 0x55d3ac7aea50 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.585+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v120'305 uv299 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.585+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.586+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v120'306 uv306 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.586+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.587+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v120'307 uv307 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.589 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:25:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.588+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.589+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'308 uv308 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.589+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.590+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'309 uv309 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.590+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.591+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.591+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.592+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v120'311 uv304 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.592+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.594+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v120'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.594+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.596 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:25:08.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.595+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v120'313 uv313 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.595+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.596+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'314 uv314 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.596+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7ab150 con 0x55d3ac7ac670 2023-10-06T05:25:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.597+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'315 uv315 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.597+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d3ac7af4a0 0x55d3ac7b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.598+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b1dc0 con 0x55d3ac7af4a0 2023-10-06T05:25:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.598+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d3ac7af4a0 0x55d3ac7b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.598+0000 7fe3d6952700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d3ac7af4a0 0x55d3ac7b1880 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.600+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v120'138 uv137 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7af4a0 2023-10-06T05:25:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.600+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b1dc0 con 0x55d3ac7af4a0 2023-10-06T05:25:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.601+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v120'139 uv139 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7af4a0 2023-10-06T05:25:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.601+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b1dc0 con 0x55d3ac7af4a0 2023-10-06T05:25:08.605 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:25:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.604+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v120'140 uv140 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7af4a0 2023-10-06T05:25:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.604+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b21c0 con 0x55d3ac7af4a0 2023-10-06T05:25:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.606+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'141 uv141 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7af4a0 2023-10-06T05:25:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.606+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b21c0 con 0x55d3ac7af4a0 2023-10-06T05:25:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.611+0000 7fe3d6952700 1 -- 172.21.15.45:0/1895934549 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'142 uv142 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3c8016070 con 0x55d3ac7af4a0 2023-10-06T05:25:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.612+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b21c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.613+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.613+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b21c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.614+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v120'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.614+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b21c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.617 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:25:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.615+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v120'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.615+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b31d0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.617+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.617+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b31d0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.618+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.618+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b21c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.619+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.619+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d3ac7b26d0 0x55d3ac7b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.619+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5160 con 0x55d3ac7b26d0 2023-10-06T05:25:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.619+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d3ac7b26d0 0x55d3ac7b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.620+0000 7fe3d6151700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d3ac7b26d0 0x55d3ac7b4ab0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.621+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v120'273 uv267 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.621+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.622+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v120'274 uv274 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.622+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.623+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v120'275 uv275 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.625 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:25:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.623+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.624+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'276 uv276 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.624+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.626+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'277 uv277 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.626+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.627+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v120'278 uv272 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.627+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.628+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v120'279 uv279 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.628+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.629+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v120'280 uv280 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.631 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:25:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.629+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5540 con 0x55d3ac7b26d0 2023-10-06T05:25:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.630+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'281 uv281 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.630+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b82b0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.631+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'282 uv282 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7b26d0 2023-10-06T05:25:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.631+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7a72f0 2023-10-06T05:25:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.632+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v120'120 uv119 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.632+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7a72f0 2023-10-06T05:25:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.633+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'121 uv121 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.634+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7a72f0 2023-10-06T05:25:08.636 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:25:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.635+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v120'122 uv122 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.635+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7a72f0 2023-10-06T05:25:08.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.636+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'123 uv123 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.636+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d3ac7b6340 0x55d3ac7b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.636+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7b6340 2023-10-06T05:25:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.636+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d3ac7b6340 0x55d3ac7b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.637+0000 7fe3d7153700 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d3ac7b6340 0x55d3ac7b8a00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.638+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7b6340 2023-10-06T05:25:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.638+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7b6340 2023-10-06T05:25:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.639+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v120'166 uv166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7b6340 2023-10-06T05:25:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.639+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7b6340 2023-10-06T05:25:08.641 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:25:08.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.640+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v120'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7b6340 2023-10-06T05:25:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.640+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7b6340 2023-10-06T05:25:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.641+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7b6340 2023-10-06T05:25:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.641+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7b6340 2023-10-06T05:25:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.642+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7b6340 2023-10-06T05:25:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.642+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7b6340 2023-10-06T05:25:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.643+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7b6340 2023-10-06T05:25:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.643+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7ac670 2023-10-06T05:25:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.645+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v120'140 uv139 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.645+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7ac670 2023-10-06T05:25:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.646+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v120'141 uv141 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.646+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7ac670 2023-10-06T05:25:08.649 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:25:08.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.647+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v120'142 uv142 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.647+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7ac670 2023-10-06T05:25:08.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.649+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'143 uv143 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.649+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7ac670 2023-10-06T05:25:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.650+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'144 uv144 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe3d0002ce0 con 0x55d3ac7ac670 2023-10-06T05:25:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.650+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a30 con 0x55d3ac7a72f0 2023-10-06T05:25:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.652+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v120'339 uv331 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.652+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a00 con 0x55d3ac7a72f0 2023-10-06T05:25:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.653+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v120'340 uv340 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.653+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a00 con 0x55d3ac7a72f0 2023-10-06T05:25:08.655 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:25:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.654+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v120'341 uv341 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.654+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b5a00 con 0x55d3ac7a72f0 2023-10-06T05:25:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.655+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v120'342 uv342 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.655+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b8f90 con 0x55d3ac7a72f0 2023-10-06T05:25:08.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.656+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'343 uv343 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.656+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b8f40 con 0x55d3ac7a72f0 2023-10-06T05:25:08.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.657+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v120'344 uv338 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.657+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b8f40 con 0x55d3ac7a72f0 2023-10-06T05:25:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.658+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v120'345 uv345 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.659+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b8f40 con 0x55d3ac7a72f0 2023-10-06T05:25:08.662 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:25:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.660+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v120'346 uv346 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.660+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b59e0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.661+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'347 uv347 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.661+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b59e0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.662+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'348 uv348 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.662+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b8f40 con 0x55d3ac7a72f0 2023-10-06T05:25:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.663+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'349 uv349 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.663+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x55d3ac7b9340 con 0x55d3ac7a72f0 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3d6151700 1 -- 172.21.15.45:0/1895934549 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'350 uv350 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe3cc0823c0 con 0x55d3ac7a72f0 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d3ac7a72f0 msgr2=0x55d3ac7a96d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d3ac7a72f0 0x55d3ac7a96d0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d3ac6aef90 msgr2=0x55d3ac797e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d3ac6aef90 0x55d3ac797e00 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d3ac7b26d0 msgr2=0x55d3ac7b4ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d3ac7b26d0 0x55d3ac7b4ab0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d3ac7a29b0 msgr2=0x55d3ac7a4d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d3ac7a29b0 0x55d3ac7a4d90 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d3ac7b6340 msgr2=0x55d3ac7b8a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d3ac7b6340 0x55d3ac7b8a00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d3ac7af4a0 msgr2=0x55d3ac7b1880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d3ac7af4a0 0x55d3ac7b1880 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d3ac7ac670 msgr2=0x55d3ac7aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d3ac7ac670 0x55d3ac7aea50 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 reap_dead start 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 reap_dead start 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 reap_dead start 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 reap_dead start 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3d7153700 1 -- 172.21.15.45:0/1895934549 reap_dead start 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe3bc0658c0 msgr2=0x7fe3bc067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe3bc0658c0 0x7fe3bc067d70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8001aa0 tx=0x7fe3c8014000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6f7340 msgr2=0x55d3ac6aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6f7340 0x55d3ac6aea50 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005610 tx=0x7fe3cc013000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 shutdown_connections 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d3ac6f7340 0x55d3ac6aea50 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 --2- 172.21.15.45:0/1895934549 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe3bc0658c0 0x7fe3bc067d70 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 >> 172.21.15.45:0/1895934549 conn(0x55d3ac6ae4a0 msgr2=0x55d3ac6b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 shutdown_connections 2023-10-06T05:25:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.664+0000 7fe3dd734040 1 -- 172.21.15.45:0/1895934549 wait complete. 2023-10-06T05:25:08.670 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2023-10-06T05:25:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.760+0000 7f84a4e67040 1 Processor -- start 2023-10-06T05:25:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.760+0000 7f84a4e67040 1 -- start start 2023-10-06T05:25:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f84a4e67040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722ac0210 0x559722b7e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f84a4e67040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722b82890 0x559722af0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f84a4e67040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82c60 0x559722ba5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f84a4e67040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722a0d980 con 0x559722b82c60 2023-10-06T05:25:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f84a4e67040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722a0d620 con 0x559722ac0210 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f84a4e67040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559722a04400 con 0x559722b82890 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82c60 0x559722ba5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82c60 0x559722ba5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43354/0 (socket says 172.21.15.45:43354) 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 -- 172.21.15.45:0/3972270321 learned_addr learned my addr 172.21.15.45:0/3972270321 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e085700 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722ac0210 0x559722b7e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849d884700 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722b82890 0x559722af0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 -- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722b82890 msgr2=0x559722af0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722b82890 0x559722af0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 -- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722ac0210 msgr2=0x559722b7e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722ac0210 0x559722b7e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 -- 172.21.15.45:0/3972270321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490005cf0 con 0x559722b82c60 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.761+0000 7f849e886700 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82c60 0x559722ba5980 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f84980016b0 tx=0x7f8498006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dca531a251151b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f849c882700 1 -- 172.21.15.45:0/3972270321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498006320 con 0x559722b82c60 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f849c882700 1 -- 172.21.15.45:0/3972270321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8498004520 con 0x559722b82c60 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f849c882700 1 -- 172.21.15.45:0/3972270321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980048e0 con 0x559722b82c60 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 -- 172.21.15.45:0/3972270321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82c60 msgr2=0x559722ba5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82c60 0x559722ba5980 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f84980016b0 tx=0x7f8498006950 comp rx=0 tx=0).stop 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 -- 172.21.15.45:0/3972270321 shutdown_connections 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82c60 0x559722ba5980 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722ac0210 0x559722b7e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 --2- 172.21.15.45:0/3972270321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722b82890 0x559722af0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 -- 172.21.15.45:0/3972270321 >> 172.21.15.45:0/3972270321 conn(0x559722ac3450 msgr2=0x559722ac5830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 -- 172.21.15.45:0/3972270321 shutdown_connections 2023-10-06T05:25:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.762+0000 7f84a4e67040 1 -- 172.21.15.45:0/3972270321 wait complete. 2023-10-06T05:25:08.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.763+0000 7f84a4e67040 1 Processor -- start 2023-10-06T05:25:08.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.763+0000 7f84a4e67040 1 -- start start 2023-10-06T05:25:08.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f84a4e67040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722ac0210 0x559722b7efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f84a4e67040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82890 0x559722b9cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f84a4e67040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722b82c60 0x559722b9d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f84a4e67040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722b802f0 con 0x559722b82890 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f84a4e67040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722b80590 con 0x559722b82c60 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f84a4e67040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559722b80830 con 0x559722ac0210 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82890 0x559722b9cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82890 0x559722b9cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43370/0 (socket says 172.21.15.45:43370) 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 -- 172.21.15.45:0/2553185920 learned_addr learned my addr 172.21.15.45:0/2553185920 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849e085700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722ac0210 0x559722b7efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849e886700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722b82c60 0x559722b9d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 -- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722ac0210 msgr2=0x559722b7efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559722ac0210 0x559722b7efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 -- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722b82c60 msgr2=0x559722b9d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559722b82c60 0x559722b9d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 -- 172.21.15.45:0/2553185920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498006c30 con 0x559722b82890 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.764+0000 7f849d884700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82890 0x559722b9cdd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f8494008920 tx=0x7f8494005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.765+0000 7f848effd700 1 -- 172.21.15.45:0/2553185920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84940067c0 con 0x559722b82890 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.765+0000 7f848effd700 1 -- 172.21.15.45:0/2553185920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8494010040 con 0x559722b82890 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.765+0000 7f848effd700 1 -- 172.21.15.45:0/2553185920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494006b50 con 0x559722b82890 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.765+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559722a1c3d0 con 0x559722b82890 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.765+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559722a0fae0 con 0x559722b82890 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.766+0000 7f848effd700 1 -- 172.21.15.45:0/2553185920 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f849401e460 con 0x559722b82890 2023-10-06T05:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.767+0000 7f848effd700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8480065890 0x7f8480067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.767+0000 7f848effd700 1 -- 172.21.15.45:0/2553185920 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f84940841d0 con 0x559722b82890 2023-10-06T05:25:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.767+0000 7f849e085700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8480065890 0x7f8480067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.767+0000 7f84a4e67040 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559722b81210 0x559722ba2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.767+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) v8 -- 0x559722ba03d0 con 0x559722b81210 2023-10-06T05:25:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.767+0000 7f849e886700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559722b81210 0x559722ba2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.767+0000 7f849e085700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8480065890 0x7f8480067d40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f849000c010 tx=0x7f8490016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.768+0000 7f849e886700 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559722b81210 0x559722ba2e00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f849e886700 1 -- 172.21.15.45:0/2553185920 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v120'283 uv283 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8498004ce0 con 0x559722b81210 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559722b81210 msgr2=0x559722ba2e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x559722b81210 0x559722ba2e00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8480065890 msgr2=0x7f8480067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8480065890 0x7f8480067d40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f849000c010 tx=0x7f8490016000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82890 msgr2=0x559722b9cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 --2- 172.21.15.45:0/2553185920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559722b82890 0x559722b9cdd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f8494008920 tx=0x7f8494005d80 comp rx=0 tx=0).stop 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f849e886700 1 -- 172.21.15.45:0/2553185920 reap_dead start 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 shutdown_connections 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 >> 172.21.15.45:0/2553185920 conn(0x559722ac3450 msgr2=0x559722b7d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 shutdown_connections 2023-10-06T05:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.769+0000 7f84a4e67040 1 -- 172.21.15.45:0/2553185920 wait complete. 2023-10-06T05:25:08.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:25:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.912+0000 7f7ce5a92700 1 Processor -- start 2023-10-06T05:25:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.912+0000 7f7ce5a92700 1 -- start start 2023-10-06T05:25:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7ce5a92700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7ce00c3db0 0x7f7ce00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7ce5a92700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ce0069ed0 0x7f7ce00cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7ce5a92700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00ce6a0 con 0x7f7ce00b0610 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7ce5a92700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce00690f0 con 0x7f7ce0069ed0 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7ce5a92700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce00ceeb0 con 0x7f7ce00c3db0 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ce0069ed0 0x7f7ce00cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ce0069ed0 0x7f7ce00cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43384/0 (socket says 172.21.15.45:43384) 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdffff700 1 -- 172.21.15.45:0/1938739095 learned_addr learned my addr 172.21.15.45:0/1938739095 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdeffd700 1 --2- 172.21.15.45:0/1938739095 >> v2:172.21.15.144:3300/0 conn(0x7f7ce00c3db0 0x7f7ce00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdffff700 1 -- 172.21.15.45:0/1938739095 >> v2:172.21.15.144:3300/0 conn(0x7f7ce00c3db0 msgr2=0x7f7ce00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdffff700 1 --2- 172.21.15.45:0/1938739095 >> v2:172.21.15.144:3300/0 conn(0x7f7ce00c3db0 0x7f7ce00c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdffff700 1 -- 172.21.15.45:0/1938739095 >> v1:172.21.15.144:6789/0 conn(0x7f7ce00b0610 legacy=0x7f7ce00697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.913+0000 7f7cdffff700 1 -- 172.21.15.45:0/1938739095 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce0159880 con 0x7f7ce0069ed0 2023-10-06T05:25:08.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7cdffff700 1 --2- 172.21.15.45:0/1938739095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce0069ed0 0x7f7ce00cc350 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0064110 tx=0x7f7cd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9889119c3c3537 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7cde7fc700 1 -- 172.21.15.45:0/1938739095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800e070 con 0x7f7ce0069ed0 2023-10-06T05:25:08.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7cde7fc700 1 -- 172.21.15.45:0/1938739095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd800b990 con 0x7f7ce0069ed0 2023-10-06T05:25:08.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7cde7fc700 1 -- 172.21.15.45:0/1938739095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80092e0 con 0x7f7ce0069ed0 2023-10-06T05:25:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 -- 172.21.15.45:0/1938739095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce0069ed0 msgr2=0x7f7ce00cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/1938739095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce0069ed0 0x7f7ce00cc350 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0064110 tx=0x7f7cd8009840 comp rx=0 tx=0).stop 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 -- 172.21.15.45:0/1938739095 shutdown_connections 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/1938739095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce0069ed0 0x7f7ce00cc350 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/1938739095 >> v2:172.21.15.144:3300/0 conn(0x7f7ce00c3db0 0x7f7ce00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 -- 172.21.15.45:0/1938739095 >> 172.21.15.45:0/1938739095 conn(0x7f7ce0067e00 msgr2=0x7f7ce00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 -- 172.21.15.45:0/1938739095 shutdown_connections 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.914+0000 7f7ce5a92700 1 -- 172.21.15.45:0/1938739095 wait complete. 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.916+0000 7f7ce5a92700 1 Processor -- start 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.916+0000 7f7ce5a92700 1 -- start start 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7ce5a92700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce0069ed0 0x7f7ce0151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7ce5a92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce00b0610 0x7f7ce015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7ce5a92700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce00c3db0 0x7f7ce015a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7ce5a92700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce01523f0 con 0x7f7ce00c3db0 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7ce5a92700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b2ab0 con 0x7f7ce00b0610 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7ce5a92700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b2d90 con 0x7f7ce0069ed0 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce00c3db0 0x7f7ce015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce00c3db0 0x7f7ce015a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43394/0 (socket says 172.21.15.45:43394) 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 -- 172.21.15.45:0/2379019732 learned_addr learned my addr 172.21.15.45:0/2379019732 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdeffd700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce00b0610 0x7f7ce015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdf7fe700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce0069ed0 0x7f7ce0151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 -- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce0069ed0 msgr2=0x7f7ce0151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce0069ed0 0x7f7ce0151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 -- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce00b0610 msgr2=0x7f7ce015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce00b0610 0x7f7ce015a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 -- 172.21.15.45:0/2379019732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd8009ae0 con 0x7f7ce00c3db0 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdf7fe700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce0069ed0 0x7f7ce0151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cdffff700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce00c3db0 0x7f7ce015a880 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009880 tx=0x7f7cd8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cd77fe700 1 -- 172.21.15.45:0/2379019732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8008b80 con 0x7f7ce00c3db0 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cd77fe700 1 -- 172.21.15.45:0/2379019732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8008d40 con 0x7f7ce00c3db0 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7cd77fe700 1 -- 172.21.15.45:0/2379019732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80266b0 con 0x7f7ce00c3db0 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.917+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce00b3070 con 0x7f7ce00c3db0 2023-10-06T05:25:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.918+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce00b3630 con 0x7f7ce00c3db0 2023-10-06T05:25:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.918+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce006e350 con 0x7f7ce00c3db0 2023-10-06T05:25:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.919+0000 7f7cd77fe700 1 -- 172.21.15.45:0/2379019732 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cd8003040 con 0x7f7ce00c3db0 2023-10-06T05:25:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.919+0000 7f7cd77fe700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cc0066f30 0x7f7cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.919+0000 7f7cd77fe700 1 -- 172.21.15.45:0/2379019732 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f7cd8086fe0 con 0x7f7ce00c3db0 2023-10-06T05:25:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.920+0000 7f7cdf7fe700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cc0066f30 0x7f7cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.920+0000 7f7cdf7fe700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cc0066f30 0x7f7cc00693e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00b1c30 tx=0x7f7cc8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:08.924+0000 7f7cd77fe700 1 -- 172.21.15.45:0/2379019732 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cd8053600 con 0x7f7ce00c3db0 2023-10-06T05:25:09.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.180+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f7ce004bb40 con 0x7f7ce00c3db0 2023-10-06T05:25:09.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.422+0000 7f7cd77fe700 1 -- 172.21.15.45:0/2379019732 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v265) v1 ==== 162+0+0 (secure 0 0 0) 0x7f7cd8056c20 con 0x7f7ce00c3db0 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cc0066f30 msgr2=0x7f7cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cc0066f30 0x7f7cc00693e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00b1c30 tx=0x7f7cc8009040 comp rx=0 tx=0).stop 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce00c3db0 msgr2=0x7f7ce015a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce00c3db0 0x7f7ce015a880 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009880 tx=0x7f7cd8018000 comp rx=0 tx=0).stop 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 shutdown_connections 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7cc0066f30 0x7f7cc00693e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce00c3db0 0x7f7ce015a880 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce0069ed0 0x7f7ce0151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 --2- 172.21.15.45:0/2379019732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce00b0610 0x7f7ce015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 >> 172.21.15.45:0/2379019732 conn(0x7f7ce0067e00 msgr2=0x7f7ce0068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 shutdown_connections 2023-10-06T05:25:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.425+0000 7f7ce5a92700 1 -- 172.21.15.45:0/2379019732 wait complete. 2023-10-06T05:25:09.428 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:25:09.429 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:09.427+0000 7f0669374700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer 2023-10-06T05:25:09.430 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:09.429+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:09.430 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:09.429+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:25:09.437 INFO:tasks.ceph.mds.b.smithi144.stderr: -31> 2023-10-06T05:25:09.427+0000 7f0669374700 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer 2023-10-06T05:25:09.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:09.458 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:25:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.580+0000 7f713b8d0700 1 Processor -- start 2023-10-06T05:25:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.580+0000 7f713b8d0700 1 -- start start 2023-10-06T05:25:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713b8d0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7134159200 0x7f71340c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713b8d0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f713406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713b8d0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f713406b5c0 0x7f71340ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713b8d0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340c7780 con 0x7f7134159200 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713b8d0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340c7d00 con 0x7f713406b5c0 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713b8d0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c86d0 con 0x7f71341595d0 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7134159200 0x7f71340c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7134159200 0x7f71340c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43404/0 (socket says 172.21.15.45:43404) 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 -- 172.21.15.45:0/2503727659 learned_addr learned my addr 172.21.15.45:0/2503727659 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f7138e6b700 1 --2- 172.21.15.45:0/2503727659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f713406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f7139e6d700 1 --2- 172.21.15.45:0/2503727659 >> v2:172.21.15.144:3300/0 conn(0x7f713406b5c0 0x7f71340ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 -- 172.21.15.45:0/2503727659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 msgr2=0x7f713406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 --2- 172.21.15.45:0/2503727659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f713406b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 -- 172.21.15.45:0/2503727659 >> v2:172.21.15.144:3300/0 conn(0x7f713406b5c0 msgr2=0x7f71340ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 --2- 172.21.15.45:0/2503727659 >> v2:172.21.15.144:3300/0 conn(0x7f713406b5c0 0x7f71340ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 -- 172.21.15.45:0/2503727659 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713406bb40 con 0x7f7134159200 2023-10-06T05:25:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.581+0000 7f713966c700 1 --2- 172.21.15.45:0/2503727659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c8190 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f7134067b90 tx=0x7f7130009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8645c5528426969f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f712bfff700 1 -- 172.21.15.45:0/2503727659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713000b640 con 0x7f7134159200 2023-10-06T05:25:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f712bfff700 1 -- 172.21.15.45:0/2503727659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713000b7a0 con 0x7f7134159200 2023-10-06T05:25:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f712bfff700 1 -- 172.21.15.45:0/2503727659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713000bb60 con 0x7f7134159200 2023-10-06T05:25:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 -- 172.21.15.45:0/2503727659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 msgr2=0x7f71340c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 --2- 172.21.15.45:0/2503727659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c8190 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f7134067b90 tx=0x7f7130009250 comp rx=0 tx=0).stop 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 -- 172.21.15.45:0/2503727659 shutdown_connections 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 --2- 172.21.15.45:0/2503727659 >> v2:172.21.15.144:3300/0 conn(0x7f713406b5c0 0x7f71340ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 --2- 172.21.15.45:0/2503727659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c8190 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 --2- 172.21.15.45:0/2503727659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f713406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 -- 172.21.15.45:0/2503727659 >> 172.21.15.45:0/2503727659 conn(0x7f713406f090 msgr2=0x7f713406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 -- 172.21.15.45:0/2503727659 shutdown_connections 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.582+0000 7f713b8d0700 1 -- 172.21.15.45:0/2503727659 wait complete. 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.583+0000 7f713b8d0700 1 Processor -- start 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.583+0000 7f713b8d0700 1 -- start start 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.584+0000 7f713b8d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713406b5c0 0x7f71340c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.584+0000 7f713b8d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.584+0000 7f713b8d0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f71340b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.584+0000 7f713b8d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c13a0 con 0x7f7134159200 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.584+0000 7f713b8d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c4d90 con 0x7f713406b5c0 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.584+0000 7f713b8d0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c5860 con 0x7f71341595d0 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43406/0 (socket says 172.21.15.45:43406) 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 -- 172.21.15.45:0/4200694137 learned_addr learned my addr 172.21.15.45:0/4200694137 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f713966c700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713406b5c0 0x7f71340c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7139e6d700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f71340b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 -- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 msgr2=0x7f71340b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f71340b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 -- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713406b5c0 msgr2=0x7f71340c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713406b5c0 0x7f71340c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 -- 172.21.15.45:0/4200694137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300094f0 con 0x7f7134159200 2023-10-06T05:25:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f713966c700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713406b5c0 0x7f71340c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f7138e6b700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c4850 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f712400e930 tx=0x7f71240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f71297fa700 1 -- 172.21.15.45:0/4200694137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400eed0 con 0x7f7134159200 2023-10-06T05:25:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f71297fa700 1 -- 172.21.15.45:0/4200694137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712400c850 con 0x7f7134159200 2023-10-06T05:25:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340c4fc0 con 0x7f7134159200 2023-10-06T05:25:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f71297fa700 1 -- 172.21.15.45:0/4200694137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400cbf0 con 0x7f7134159200 2023-10-06T05:25:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.585+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71340c5580 con 0x7f7134159200 2023-10-06T05:25:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.586+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713406e2f0 con 0x7f7134159200 2023-10-06T05:25:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.587+0000 7f71297fa700 1 -- 172.21.15.45:0/4200694137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7124013470 con 0x7f7134159200 2023-10-06T05:25:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.587+0000 7f71297fa700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.587+0000 7f71297fa700 1 -- 172.21.15.45:0/4200694137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7f7124085400 con 0x7f7134159200 2023-10-06T05:25:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.587+0000 7f713966c700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.588+0000 7f713966c700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7134067b90 tx=0x7f7130017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.592+0000 7f71297fa700 1 -- 172.21.15.45:0/4200694137 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71240519d0 con 0x7f7134159200 2023-10-06T05:25:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.842+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f713406be20 con 0x7f7134159200 2023-10-06T05:25:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.843+0000 7f71297fa700 1 -- 172.21.15.45:0/4200694137 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f7124054ff0 con 0x7f7134159200 2023-10-06T05:25:09.845 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:09.845 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":267,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:09.430008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 msgr2=0x7f71180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7134067b90 tx=0x7f7130017000 comp rx=0 tx=0).stop 2023-10-06T05:25:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 msgr2=0x7f71340c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c4850 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f712400e930 tx=0x7f71240091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 shutdown_connections 2023-10-06T05:25:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71341595d0 0x7f71340b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f713406b5c0 0x7f71340c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 --2- 172.21.15.45:0/4200694137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7134159200 0x7f71340c4850 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 >> 172.21.15.45:0/4200694137 conn(0x7f713406f090 msgr2=0x7f713415a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 shutdown_connections 2023-10-06T05:25:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.845+0000 7f713b8d0700 1 -- 172.21.15.45:0/4200694137 wait complete. 2023-10-06T05:25:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 267 2023-10-06T05:25:09.861 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:09.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:09.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.993+0000 7f152a4b1700 1 Processor -- start 2023-10-06T05:25:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.993+0000 7f152a4b1700 1 -- start start 2023-10-06T05:25:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f152a4b1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f15241570b0 0x7f152406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f152a4b1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f152415ac10 0x7f15240ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f152a4b1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152415b380 con 0x7f1524156ce0 2023-10-06T05:25:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f152a4b1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152406c010 con 0x7f15241570b0 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f152a4b1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15240b02f0 con 0x7f152415ac10 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f15241570b0 0x7f152406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f15241570b0 0x7f152406bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43410/0 (socket says 172.21.15.45:43410) 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 -- 172.21.15.45:0/1630223372 learned_addr learned my addr 172.21.15.45:0/1630223372 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f1528a4e700 1 --2- 172.21.15.45:0/1630223372 >> v2:172.21.15.144:3300/0 conn(0x7f152415ac10 0x7f15240ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 -- 172.21.15.45:0/1630223372 >> v2:172.21.15.144:3300/0 conn(0x7f152415ac10 msgr2=0x7f15240ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 --2- 172.21.15.45:0/1630223372 >> v2:172.21.15.144:3300/0 conn(0x7f152415ac10 0x7f15240ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 -- 172.21.15.45:0/1630223372 >> v1:172.21.15.144:6789/0 conn(0x7f1524156ce0 legacy=0x7f15240afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 -- 172.21.15.45:0/1630223372 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152415b660 con 0x7f15241570b0 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f1528a4e700 1 --2- 172.21.15.45:0/1630223372 >> v2:172.21.15.144:3300/0 conn(0x7f152415ac10 0x7f15240ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f15237fe700 1 --2- 172.21.15.45:0/1630223372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f152406bad0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f15240ae6d0 tx=0x7f150c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a9393c9135de6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f1522ffd700 1 -- 172.21.15.45:0/1630223372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00b7e0 con 0x7f15241570b0 2023-10-06T05:25:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f1522ffd700 1 -- 172.21.15.45:0/1630223372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c00b940 con 0x7f15241570b0 2023-10-06T05:25:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.994+0000 7f1522ffd700 1 -- 172.21.15.45:0/1630223372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00bce0 con 0x7f15241570b0 2023-10-06T05:25:10.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 -- 172.21.15.45:0/1630223372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 msgr2=0x7f152406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 --2- 172.21.15.45:0/1630223372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f152406bad0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f15240ae6d0 tx=0x7f150c009250 comp rx=0 tx=0).stop 2023-10-06T05:25:10.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 -- 172.21.15.45:0/1630223372 shutdown_connections 2023-10-06T05:25:10.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 --2- 172.21.15.45:0/1630223372 >> v2:172.21.15.144:3300/0 conn(0x7f152415ac10 0x7f15240ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 --2- 172.21.15.45:0/1630223372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f152406bad0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 -- 172.21.15.45:0/1630223372 >> 172.21.15.45:0/1630223372 conn(0x7f152406f090 msgr2=0x7f152406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 -- 172.21.15.45:0/1630223372 shutdown_connections 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.995+0000 7f152a4b1700 1 -- 172.21.15.45:0/1630223372 wait complete. 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.996+0000 7f152a4b1700 1 Processor -- start 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f152a4b1700 1 -- start start 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f152a4b1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1524156ce0 0x7f15240c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f152a4b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f15241532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f152a4b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f152415ac10 0x7f1524067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f152a4b1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1524158f30 con 0x7f15241570b0 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f152a4b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240b0430 con 0x7f152415ac10 2023-10-06T05:25:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f152a4b1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15240c7940 con 0x7f1524156ce0 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f15237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f15241532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f15237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f15241532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43424/0 (socket says 172.21.15.45:43424) 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.997+0000 7f15237fe700 1 -- 172.21.15.45:0/3685729584 learned_addr learned my addr 172.21.15.45:0/3685729584 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f1523fff700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1524156ce0 0x7f15240c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f1528a4e700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f152415ac10 0x7f1524067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f15237fe700 1 -- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1524156ce0 msgr2=0x7f15240c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f15237fe700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1524156ce0 0x7f15240c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f15237fe700 1 -- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f152415ac10 msgr2=0x7f1524067f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f15237fe700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f152415ac10 0x7f1524067f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f15237fe700 1 -- 172.21.15.45:0/3685729584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0094f0 con 0x7f15241570b0 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f1528a4e700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f152415ac10 0x7f1524067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f15237fe700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f15241532a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f150c0030c0 tx=0x7f150c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f151bfff700 1 -- 172.21.15.45:0/3685729584 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c011ea0 con 0x7f15241570b0 2023-10-06T05:25:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f151bfff700 1 -- 172.21.15.45:0/3685729584 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c020070 con 0x7f15241570b0 2023-10-06T05:25:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f151bfff700 1 -- 172.21.15.45:0/3685729584 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c01b6a0 con 0x7f15241570b0 2023-10-06T05:25:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15240c7c20 con 0x7f15241570b0 2023-10-06T05:25:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.998+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15240c81e0 con 0x7f15241570b0 2023-10-06T05:25:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:09.999+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152406e2f0 con 0x7f15241570b0 2023-10-06T05:25:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.000+0000 7f151bfff700 1 -- 172.21.15.45:0/3685729584 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f150c022030 con 0x7f15241570b0 2023-10-06T05:25:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.000+0000 7f151bfff700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1504066f30 0x7f15040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.000+0000 7f151bfff700 1 -- 172.21.15.45:0/3685729584 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7f150c087230 con 0x7f15241570b0 2023-10-06T05:25:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.000+0000 7f1523fff700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1504066f30 0x7f15040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.001+0000 7f1523fff700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1504066f30 0x7f15040693e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1514006fd0 tx=0x7f1514008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.005+0000 7f151bfff700 1 -- 172.21.15.45:0/3685729584 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f150c0537a0 con 0x7f15241570b0 2023-10-06T05:25:10.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.255+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f152406e2f0 con 0x7f15241570b0 2023-10-06T05:25:10.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.255+0000 7f151bfff700 1 -- 172.21.15.45:0/3685729584 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f150c056dc0 con 0x7f15241570b0 2023-10-06T05:25:10.257 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:10.257 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":267,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:09.430008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:10.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.257+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1504066f30 msgr2=0x7f15040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.257+0000 7f152a4b1700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1504066f30 0x7f15040693e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1514006fd0 tx=0x7f1514008040 comp rx=0 tx=0).stop 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.257+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 msgr2=0x7f15241532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.257+0000 7f152a4b1700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f15241532a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f150c0030c0 tx=0x7f150c016000 comp rx=0 tx=0).stop 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 shutdown_connections 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1504066f30 0x7f15040693e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f152415ac10 0x7f1524067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1524156ce0 0x7f15240c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 --2- 172.21.15.45:0/3685729584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15241570b0 0x7f15241532a0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 >> 172.21.15.45:0/3685729584 conn(0x7f152406f090 msgr2=0x7f1524155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 shutdown_connections 2023-10-06T05:25:10.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.258+0000 7f152a4b1700 1 -- 172.21.15.45:0/3685729584 wait complete. 2023-10-06T05:25:10.261 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 267 2023-10-06T05:25:10.273 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:10.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.404+0000 7f1911444700 1 Processor -- start 2023-10-06T05:25:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.405+0000 7f1911444700 1 -- start start 2023-10-06T05:25:10.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.405+0000 7f1911444700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1060 0x7f190c069f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.405+0000 7f1911444700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c06b1f0 con 0x7f190c140990 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.405+0000 7f1911444700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c06a5e0 con 0x7f190c0b1430 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.405+0000 7f1911444700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c06a8c0 con 0x7f190c0b1060 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f190b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f190c140990 0x7f190c0ca980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39608/0 (socket says 172.21.15.45:39608) 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f190b7fe700 1 -- 172.21.15.45:0/677468156 learned_addr learned my addr 172.21.15.45:0/677468156 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f190affd700 1 --2- 172.21.15.45:0/677468156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1060 0x7f190c069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f1909ffb700 1 -- 172.21.15.45:0/677468156 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871686671 0 0) 0x7f190c06b1f0 con 0x7f190c140990 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f1909ffb700 1 -- 172.21.15.45:0/677468156 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f4003570 con 0x7f190c140990 2023-10-06T05:25:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f1909ffb700 1 -- 172.21.15.45:0/677468156 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189630188 0 0) 0x7f190c06a5e0 con 0x7f190c0b1430 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f1909ffb700 1 -- 172.21.15.45:0/677468156 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f190c06b1f0 con 0x7f190c0b1430 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f190affd700 1 -- 172.21.15.45:0/677468156 >> v1:172.21.15.144:6789/0 conn(0x7f190c0b1430 legacy=0x7f190c140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f190affd700 1 -- 172.21.15.45:0/677468156 >> v1:172.21.15.45:6789/0 conn(0x7f190c140990 legacy=0x7f190c0ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f190affd700 1 -- 172.21.15.45:0/677468156 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c06aba0 con 0x7f190c0b1060 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.406+0000 7f190affd700 1 --2- 172.21.15.45:0/677468156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1060 0x7f190c069f70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009910 tx=0x7f18fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3784825edb0508b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1909ffb700 1 -- 172.21.15.45:0/677468156 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011630 con 0x7f190c0b1060 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1909ffb700 1 -- 172.21.15.45:0/677468156 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc011790 con 0x7f190c0b1060 2023-10-06T05:25:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1909ffb700 1 -- 172.21.15.45:0/677468156 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011b30 con 0x7f190c0b1060 2023-10-06T05:25:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1911444700 1 -- 172.21.15.45:0/677468156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1060 msgr2=0x7f190c069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1911444700 1 --2- 172.21.15.45:0/677468156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1060 0x7f190c069f70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009910 tx=0x7f18fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1911444700 1 -- 172.21.15.45:0/677468156 shutdown_connections 2023-10-06T05:25:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1911444700 1 --2- 172.21.15.45:0/677468156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1060 0x7f190c069f70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1911444700 1 -- 172.21.15.45:0/677468156 >> 172.21.15.45:0/677468156 conn(0x7f190c067620 msgr2=0x7f190c067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1911444700 1 -- 172.21.15.45:0/677468156 shutdown_connections 2023-10-06T05:25:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.407+0000 7f1911444700 1 -- 172.21.15.45:0/677468156 wait complete. 2023-10-06T05:25:10.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 Processor -- start 2023-10-06T05:25:10.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 -- start start 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0b1060 0x7f190c0c0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1430 0x7f190c0c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c140990 0x7f190c12f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0c10a0 con 0x7f190c0b1060 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0c1380 con 0x7f190c140990 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.409+0000 7f1911444700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0c1660 con 0x7f190c0b1430 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0b1060 0x7f190c0c0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0b1060 0x7f190c0c0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43436/0 (socket says 172.21.15.45:43436) 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 -- 172.21.15.45:0/2151816802 learned_addr learned my addr 172.21.15.45:0/2151816802 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190a7fc700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1430 0x7f190c0c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190b7fe700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c140990 0x7f190c12f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 -- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1430 msgr2=0x7f190c0c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1430 0x7f190c0c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 -- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c140990 msgr2=0x7f190c12f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c140990 0x7f190c12f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 -- 172.21.15.45:0/2151816802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc009420 con 0x7f190c0b1060 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f190affd700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0b1060 0x7f190c0c0ac0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009f00 tx=0x7f18fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f18f2ffd700 1 -- 172.21.15.45:0/2151816802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc01f070 con 0x7f190c0b1060 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f18f2ffd700 1 -- 172.21.15.45:0/2151816802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc002f70 con 0x7f190c0b1060 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f18f2ffd700 1 -- 172.21.15.45:0/2151816802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011900 con 0x7f190c0b1060 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c0c1940 con 0x7f190c0b1060 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.410+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c0c2860 con 0x7f190c0b1060 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.411+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c06cec0 con 0x7f190c0b1060 2023-10-06T05:25:10.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.412+0000 7f18f2ffd700 1 -- 172.21.15.45:0/2151816802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18fc023070 con 0x7f190c0b1060 2023-10-06T05:25:10.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.412+0000 7f18f2ffd700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f18f8066f30 0x7f18f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.413+0000 7f18f2ffd700 1 -- 172.21.15.45:0/2151816802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7f18fc086450 con 0x7f190c0b1060 2023-10-06T05:25:10.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.413+0000 7f190a7fc700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f18f8066f30 0x7f18f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.413+0000 7f190a7fc700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f18f8066f30 0x7f18f80693e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f190c15b5e0 tx=0x7f1900007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.417+0000 7f18f2ffd700 1 -- 172.21.15.45:0/2151816802 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18fc052a20 con 0x7f190c0b1060 2023-10-06T05:25:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.669+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f190c06ae80 con 0x7f190c0b1060 2023-10-06T05:25:10.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.670+0000 7f18f2ffd700 1 -- 172.21.15.45:0/2151816802 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 268 v268) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f18fc02a090 con 0x7f190c0b1060 2023-10-06T05:25:10.672 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:10.672 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":268,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":9331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3983487029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3983487029},{"type":"v1","addr":"172.21.15.144:6835","nonce":3983487029}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":268}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:09.430008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f18f8066f30 msgr2=0x7f18f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f18f8066f30 0x7f18f80693e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f190c15b5e0 tx=0x7f1900007d50 comp rx=0 tx=0).stop 2023-10-06T05:25:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0b1060 msgr2=0x7f190c0c0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0b1060 0x7f190c0c0ac0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009f00 tx=0x7f18fc017000 comp rx=0 tx=0).stop 2023-10-06T05:25:10.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 shutdown_connections 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f18f8066f30 0x7f18f80693e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c140990 0x7f190c12f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0b1060 0x7f190c0c0ac0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 --2- 172.21.15.45:0/2151816802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c0b1430 0x7f190c0c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 >> 172.21.15.45:0/2151816802 conn(0x7f190c067620 msgr2=0x7f190c13f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 shutdown_connections 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.672+0000 7f1911444700 1 -- 172.21.15.45:0/2151816802 wait complete. 2023-10-06T05:25:10.675 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 268 2023-10-06T05:25:10.687 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2023-10-06T05:25:10.687 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2023-10-06T05:25:10.688 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:10.688 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:25:10.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:10.688 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:10.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:25:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:25:10.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.885+0000 7f44c9608700 1 Processor -- start 2023-10-06T05:25:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.885+0000 7f44c9608700 1 -- start start 2023-10-06T05:25:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c9608700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f44c40c8ff0 0x7f44c40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c9608700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40caad0 0x7f44c40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c9608700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44c4156af0 0x7f44c4156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c9608700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c415b000 con 0x7f44c4156af0 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c9608700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c415b2e0 con 0x7f44c40c8ff0 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c9608700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c415b5c0 con 0x7f44c40caad0 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44c4156af0 0x7f44c4156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44c4156af0 0x7f44c4156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43452/0 (socket says 172.21.15.45:43452) 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.886+0000 7f44c37fe700 1 -- 172.21.15.45:0/1427997112 learned_addr learned my addr 172.21.15.45:0/1427997112 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c2ffd700 1 --2- 172.21.15.45:0/1427997112 >> v2:172.21.15.144:3300/0 conn(0x7f44c40c8ff0 0x7f44c40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c27fc700 1 --2- 172.21.15.45:0/1427997112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40caad0 0x7f44c40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c37fe700 1 -- 172.21.15.45:0/1427997112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40caad0 msgr2=0x7f44c40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c37fe700 1 --2- 172.21.15.45:0/1427997112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40caad0 0x7f44c40caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c37fe700 1 -- 172.21.15.45:0/1427997112 >> v2:172.21.15.144:3300/0 conn(0x7f44c40c8ff0 msgr2=0x7f44c40c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c37fe700 1 --2- 172.21.15.45:0/1427997112 >> v2:172.21.15.144:3300/0 conn(0x7f44c40c8ff0 0x7f44c40c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c37fe700 1 -- 172.21.15.45:0/1427997112 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c415b8a0 con 0x7f44c4156af0 2023-10-06T05:25:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c27fc700 1 --2- 172.21.15.45:0/1427997112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40caad0 0x7f44c40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c37fe700 1 --2- 172.21.15.45:0/1427997112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c4156f70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f44c40cefc0 tx=0x7f44bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c6b6f2ef1298a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.887+0000 7f44c1ffb700 1 -- 172.21.15.45:0/1427997112 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00e070 con 0x7f44c4156af0 2023-10-06T05:25:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c1ffb700 1 -- 172.21.15.45:0/1427997112 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc00ba50 con 0x7f44c4156af0 2023-10-06T05:25:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c1ffb700 1 -- 172.21.15.45:0/1427997112 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0092e0 con 0x7f44c4156af0 2023-10-06T05:25:10.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 -- 172.21.15.45:0/1427997112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 msgr2=0x7f44c4156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 --2- 172.21.15.45:0/1427997112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c4156f70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f44c40cefc0 tx=0x7f44bc009840 comp rx=0 tx=0).stop 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 -- 172.21.15.45:0/1427997112 shutdown_connections 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 --2- 172.21.15.45:0/1427997112 >> v2:172.21.15.144:3300/0 conn(0x7f44c40c8ff0 0x7f44c40c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 --2- 172.21.15.45:0/1427997112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c4156f70 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 --2- 172.21.15.45:0/1427997112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40caad0 0x7f44c40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 -- 172.21.15.45:0/1427997112 >> 172.21.15.45:0/1427997112 conn(0x7f44c40cdf50 msgr2=0x7f44c40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 -- 172.21.15.45:0/1427997112 shutdown_connections 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.888+0000 7f44c9608700 1 -- 172.21.15.45:0/1427997112 wait complete. 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.890+0000 7f44c9608700 1 Processor -- start 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.890+0000 7f44c9608700 1 -- start start 2023-10-06T05:25:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.890+0000 7f44c9608700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40c8ff0 0x7f44c40b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.890+0000 7f44c9608700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44c40caad0 0x7f44c40c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.890+0000 7f44c9608700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c9608700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40c7680 con 0x7f44c4156af0 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c9608700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40b2d70 con 0x7f44c40caad0 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c9608700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40c6a70 con 0x7f44c40c8ff0 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c40b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43462/0 (socket says 172.21.15.45:43462) 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 -- 172.21.15.45:0/2005867150 learned_addr learned my addr 172.21.15.45:0/2005867150 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c2ffd700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40c8ff0 0x7f44c40b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c27fc700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44c40caad0 0x7f44c40c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 -- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40c8ff0 msgr2=0x7f44c40b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40c8ff0 0x7f44c40b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 -- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44c40caad0 msgr2=0x7f44c40c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44c40caad0 0x7f44c40c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 -- 172.21.15.45:0/2005867150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc009ae0 con 0x7f44c4156af0 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c37fe700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c40b32d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f44c40c4f80 tx=0x7f44bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44baffd700 1 -- 172.21.15.45:0/2005867150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0032f0 con 0x7f44c4156af0 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44baffd700 1 -- 172.21.15.45:0/2005867150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc008d20 con 0x7f44c4156af0 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c40c6d50 con 0x7f44c4156af0 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.891+0000 7f44baffd700 1 -- 172.21.15.45:0/2005867150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc026690 con 0x7f44c4156af0 2023-10-06T05:25:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.892+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c40c7310 con 0x7f44c4156af0 2023-10-06T05:25:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.893+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c40c7830 con 0x7f44c4156af0 2023-10-06T05:25:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.893+0000 7f44baffd700 1 -- 172.21.15.45:0/2005867150 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44bc003040 con 0x7f44c4156af0 2023-10-06T05:25:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.893+0000 7f44baffd700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44a4066f30 0x7f44a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.893+0000 7f44baffd700 1 -- 172.21.15.45:0/2005867150 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7f44bc0870b0 con 0x7f44c4156af0 2023-10-06T05:25:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.894+0000 7f44c2ffd700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44a4066f30 0x7f44a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.894+0000 7f44c2ffd700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44a4066f30 0x7f44a40693e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006fd0 tx=0x7f44ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:10.898+0000 7f44baffd700 1 -- 172.21.15.45:0/2005867150 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44c40c7830 con 0x7f44c4156af0 2023-10-06T05:25:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.145+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f44c40c77c0 con 0x7f44c4156af0 2023-10-06T05:25:11.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.892+0000 7f44baffd700 1 -- 172.21.15.45:0/2005867150 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v269) v1 ==== 161+0+0 (secure 0 0 0) 0x7f44bc053680 con 0x7f44c4156af0 2023-10-06T05:25:11.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.894+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44a4066f30 msgr2=0x7f44a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.894+0000 7f44c9608700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44a4066f30 0x7f44a40693e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006fd0 tx=0x7f44ac008040 comp rx=0 tx=0).stop 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.894+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 msgr2=0x7f44c40b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.894+0000 7f44c9608700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c40b32d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f44c40c4f80 tx=0x7f44bc018000 comp rx=0 tx=0).stop 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 shutdown_connections 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44a4066f30 0x7f44a40693e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44c4156af0 0x7f44c40b32d0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44c40c8ff0 0x7f44c40b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 --2- 172.21.15.45:0/2005867150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44c40caad0 0x7f44c40c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 >> 172.21.15.45:0/2005867150 conn(0x7f44c40cdf50 msgr2=0x7f44c40ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 shutdown_connections 2023-10-06T05:25:11.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:11.895+0000 7f44c9608700 1 -- 172.21.15.45:0/2005867150 wait complete. 2023-10-06T05:25:11.898 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:11.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:25:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.048+0000 7ff23bc4e700 1 Processor -- start 2023-10-06T05:25:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.048+0000 7ff23bc4e700 1 -- start start 2023-10-06T05:25:12.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff23bc4e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2340c93c0 0x7ff234156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff23bc4e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff234157030 0x7ff2340cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff23bc4e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23415b000 con 0x7ff2340c8ff0 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff23bc4e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2340c9830 con 0x7ff2340c93c0 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff23bc4e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23415b2e0 con 0x7ff234157030 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff2391e9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2340c93c0 0x7ff234156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff2399ea700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff2340c8ff0 0x7ff2340caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39624/0 (socket says 172.21.15.45:39624) 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff2399ea700 1 -- 172.21.15.45:0/1338340470 learned_addr learned my addr 172.21.15.45:0/1338340470 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff23a1eb700 1 --2- 172.21.15.45:0/1338340470 >> v2:172.21.15.144:3300/0 conn(0x7ff234157030 0x7ff2340cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff2389e8700 1 -- 172.21.15.45:0/1338340470 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167769717 0 0) 0x7ff23415b000 con 0x7ff2340c8ff0 2023-10-06T05:25:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff2389e8700 1 -- 172.21.15.45:0/1338340470 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff220003570 con 0x7ff2340c8ff0 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.049+0000 7ff2391e9700 1 -- 172.21.15.45:0/1338340470 >> v2:172.21.15.144:3300/0 conn(0x7ff234157030 msgr2=0x7ff2340cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.050+0000 7ff2391e9700 1 --2- 172.21.15.45:0/1338340470 >> v2:172.21.15.144:3300/0 conn(0x7ff234157030 0x7ff2340cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.050+0000 7ff2391e9700 1 -- 172.21.15.45:0/1338340470 >> v1:172.21.15.45:6789/0 conn(0x7ff2340c8ff0 legacy=0x7ff2340caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.050+0000 7ff2391e9700 1 -- 172.21.15.45:0/1338340470 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23415b5c0 con 0x7ff2340c93c0 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.050+0000 7ff2391e9700 1 --2- 172.21.15.45:0/1338340470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c93c0 0x7ff234156af0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff23415abf0 tx=0x7ff2300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecba0907a06ac19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.050+0000 7ff2389e8700 1 -- 172.21.15.45:0/1338340470 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230011630 con 0x7ff2340c93c0 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.050+0000 7ff2389e8700 1 -- 172.21.15.45:0/1338340470 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230011790 con 0x7ff2340c93c0 2023-10-06T05:25:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.050+0000 7ff2389e8700 1 -- 172.21.15.45:0/1338340470 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230011b30 con 0x7ff2340c93c0 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1338340470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c93c0 msgr2=0x7ff234156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1338340470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c93c0 0x7ff234156af0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff23415abf0 tx=0x7ff2300091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1338340470 shutdown_connections 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1338340470 >> v2:172.21.15.144:3300/0 conn(0x7ff234157030 0x7ff2340cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1338340470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c93c0 0x7ff234156af0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1338340470 >> 172.21.15.45:0/1338340470 conn(0x7ff2340cdf50 msgr2=0x7ff2340ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1338340470 shutdown_connections 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.051+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1338340470 wait complete. 2023-10-06T05:25:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.052+0000 7ff23bc4e700 1 Processor -- start 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.052+0000 7ff23bc4e700 1 -- start start 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23bc4e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c8ff0 0x7ff2341688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23bc4e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2340c93c0 0x7ff234159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23bc4e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff234157030 0x7ff2341574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23bc4e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23415a2e0 con 0x7ff2340c8ff0 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23bc4e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340aed40 con 0x7ff234157030 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23bc4e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23415a000 con 0x7ff2340c93c0 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c8ff0 0x7ff2341688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c8ff0 0x7ff2341688a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43482/0 (socket says 172.21.15.45:43482) 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 -- 172.21.15.45:0/1089569362 learned_addr learned my addr 172.21.15.45:0/1089569362 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23a1eb700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff234157030 0x7ff2341574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2391e9700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2340c93c0 0x7ff234159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 -- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2340c93c0 msgr2=0x7ff234159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2340c93c0 0x7ff234159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 -- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff234157030 msgr2=0x7ff2341574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff234157030 0x7ff2341574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 -- 172.21.15.45:0/1089569362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230009420 con 0x7ff2340c8ff0 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23a1eb700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff234157030 0x7ff2341574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff2399ea700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c8ff0 0x7ff2341688a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c590 tx=0x7ff224007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff229ffb700 1 -- 172.21.15.45:0/1089569362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224005b30 con 0x7ff2340c8ff0 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff229ffb700 1 -- 172.21.15.45:0/1089569362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff224005c90 con 0x7ff2340c8ff0 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.053+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2340ae6f0 con 0x7ff2340c8ff0 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.054+0000 7ff229ffb700 1 -- 172.21.15.45:0/1089569362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224007320 con 0x7ff2340c8ff0 2023-10-06T05:25:12.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.054+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23416a510 con 0x7ff2340c8ff0 2023-10-06T05:25:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.055+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23406f410 con 0x7ff2340c8ff0 2023-10-06T05:25:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.055+0000 7ff229ffb700 1 -- 172.21.15.45:0/1089569362 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff22401b3b0 con 0x7ff2340c8ff0 2023-10-06T05:25:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.055+0000 7ff229ffb700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff218066f30 0x7ff2180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.056+0000 7ff229ffb700 1 -- 172.21.15.45:0/1089569362 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7ff2240846f0 con 0x7ff2340c8ff0 2023-10-06T05:25:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.056+0000 7ff2391e9700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff218066f30 0x7ff2180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.056+0000 7ff2391e9700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff218066f30 0x7ff2180693e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff230003080 tx=0x7ff230017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.060+0000 7ff229ffb700 1 -- 172.21.15.45:0/1089569362 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff224050cc0 con 0x7ff2340c8ff0 2023-10-06T05:25:12.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.350+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7ff23404bb40 con 0x7ff2340c8ff0 2023-10-06T05:25:12.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.897+0000 7ff229ffb700 1 -- 172.21.15.45:0/1089569362 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v270) v1 ==== 101+0+0 (secure 0 0 0) 0x7ff224017070 con 0x7ff2340c8ff0 2023-10-06T05:25:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff218066f30 msgr2=0x7ff2180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff218066f30 0x7ff2180693e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff230003080 tx=0x7ff230017000 comp rx=0 tx=0).stop 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c8ff0 msgr2=0x7ff2341688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c8ff0 0x7ff2341688a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c590 tx=0x7ff224007e00 comp rx=0 tx=0).stop 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 shutdown_connections 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff218066f30 0x7ff2180693e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff234157030 0x7ff2341574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2340c8ff0 0x7ff2341688a0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 --2- 172.21.15.45:0/1089569362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2340c93c0 0x7ff234159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.900+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 >> 172.21.15.45:0/1089569362 conn(0x7ff2340cdf50 msgr2=0x7ff234147460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.901+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 shutdown_connections 2023-10-06T05:25:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.901+0000 7ff23bc4e700 1 -- 172.21.15.45:0/1089569362 wait complete. 2023-10-06T05:25:12.904 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:25:12.918 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:25:12.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.963+0000 7fcb9992a040 1 Processor -- start 2023-10-06T05:25:12.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.963+0000 7fcb9992a040 1 -- start start 2023-10-06T05:25:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb9992a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a8137e0 0x55e54a7ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb9992a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89eb30 0x55e54a7cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb9992a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a89ef00 0x55e54a7ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb9992a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54a716620 con 0x55e54a89eb30 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb9992a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54a70d400 con 0x55e54a89ef00 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb9992a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e54a89db20 con 0x55e54a8137e0 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb92347700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89eb30 0x55e54a7cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb92347700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89eb30 0x55e54a7cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43494/0 (socket says 172.21.15.45:43494) 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb92347700 1 -- 172.21.15.45:0/4285879647 learned_addr learned my addr 172.21.15.45:0/4285879647 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb92b48700 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a8137e0 0x55e54a7ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.964+0000 7fcb93349700 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a89ef00 0x55e54a7ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb92347700 1 -- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a8137e0 msgr2=0x55e54a7ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb92347700 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a8137e0 0x55e54a7ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb92347700 1 -- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a89ef00 msgr2=0x55e54a7ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb92347700 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a89ef00 0x55e54a7ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb92347700 1 -- 172.21.15.45:0/4285879647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8400a180 con 0x55e54a89eb30 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb92347700 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89eb30 0x55e54a7cca30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800a870 tx=0x7fcb88005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db171da5a8ddf535 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb91345700 1 -- 172.21.15.45:0/4285879647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8800f070 con 0x55e54a89eb30 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb91345700 1 -- 172.21.15.45:0/4285879647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88008760 con 0x55e54a89eb30 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.965+0000 7fcb91345700 1 -- 172.21.15.45:0/4285879647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88008b00 con 0x55e54a89eb30 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 -- 172.21.15.45:0/4285879647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89eb30 msgr2=0x55e54a7cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89eb30 0x55e54a7cca30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800a870 tx=0x7fcb88005be0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 -- 172.21.15.45:0/4285879647 shutdown_connections 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a89ef00 0x55e54a7ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a8137e0 0x55e54a7ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 --2- 172.21.15.45:0/4285879647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89eb30 0x55e54a7cca30 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 -- 172.21.15.45:0/4285879647 >> 172.21.15.45:0/4285879647 conn(0x55e54a7cc450 msgr2=0x55e54a8a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 -- 172.21.15.45:0/4285879647 shutdown_connections 2023-10-06T05:25:12.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.966+0000 7fcb9992a040 1 -- 172.21.15.45:0/4285879647 wait complete. 2023-10-06T05:25:12.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.967+0000 7fcb9992a040 1 Processor -- start 2023-10-06T05:25:12.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.967+0000 7fcb9992a040 1 -- start start 2023-10-06T05:25:12.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb9992a040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a8137e0 0x55e54a89d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb9992a040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a89eb30 0x55e54a7cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb9992a040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89ef00 0x55e54a8a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb9992a040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54a7ca130 con 0x55e54a89ef00 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb9992a040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54a7c9210 con 0x55e54a8137e0 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb9992a040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e54a7c94b0 con 0x55e54a89eb30 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89ef00 0x55e54a8a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89ef00 0x55e54a8a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43506/0 (socket says 172.21.15.45:43506) 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 learned_addr learned my addr 172.21.15.45:0/1107516806 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb92347700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a89eb30 0x55e54a7cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb92b48700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a8137e0 0x55e54a89d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a89eb30 msgr2=0x55e54a7cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e54a89eb30 0x55e54a7cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a8137e0 msgr2=0x55e54a89d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e54a8137e0 0x55e54a89d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.968+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8800a440 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.969+0000 7fcb93349700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89ef00 0x55e54a8a13a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c008880 tx=0x7fcb8c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.969+0000 7fcb837fe700 1 -- 172.21.15.45:0/1107516806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00f050 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.969+0000 7fcb837fe700 1 -- 172.21.15.45:0/1107516806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c010040 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.969+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e54a73f1c0 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.969+0000 7fcb837fe700 1 -- 172.21.15.45:0/1107516806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c006990 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.969+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e54a7253d0 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.970+0000 7fcb837fe700 1 -- 172.21.15.45:0/1107516806 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb8c01f460 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.971+0000 7fcb837fe700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb78067aa0 0x7fcb78069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.971+0000 7fcb837fe700 1 -- 172.21.15.45:0/1107516806 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7fcb8c084500 con 0x55e54a89ef00 2023-10-06T05:25:12.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.971+0000 7fcb92b48700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb78067aa0 0x7fcb78069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.976 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:25:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.971+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55e54a80c330 0x55e54a8ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.971+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a7ccfa0 con 0x55e54a80c330 2023-10-06T05:25:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.971+0000 7fcb92347700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55e54a80c330 0x55e54a8ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.971+0000 7fcb92b48700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb78067aa0 0x7fcb78069f50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400a150 tx=0x7fcb84011000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.972+0000 7fcb92347700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55e54a80c330 0x55e54a8ae5a0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.974+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v121'877 uv851 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.974+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8aeae0 con 0x55e54a80c330 2023-10-06T05:25:12.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.975+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v121'878 uv878 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.975+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8aef20 con 0x55e54a80c330 2023-10-06T05:25:12.978 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:25:12.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.976+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v121'879 uv879 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.976+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8af2d0 con 0x55e54a80c330 2023-10-06T05:25:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.978+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'880 uv880 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.978+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bbe50 con 0x55e54a80c330 2023-10-06T05:25:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.979+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'881 uv881 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.979+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bae40 con 0x55e54a80c330 2023-10-06T05:25:12.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.980+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'882 uv882 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.980+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb1f0 con 0x55e54a80c330 2023-10-06T05:25:12.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.982+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v121'883 uv857 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.982+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb5a0 con 0x55e54a80c330 2023-10-06T05:25:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.983+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v121'884 uv884 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.983+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb9a0 con 0x55e54a80c330 2023-10-06T05:25:12.986 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:25:12.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.984+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v121'885 uv885 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.984+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb9a0 con 0x55e54a80c330 2023-10-06T05:25:12.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.986+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'886 uv886 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.986+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.987+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'887 uv887 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.987+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.988+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'888 uv888 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.988+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.989+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v121'889 uv864 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.989+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.991+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'890 uv890 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.991+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.993 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:25:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.992+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v121'891 uv891 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.992+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.993+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'892 uv892 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.993+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.994+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'893 uv893 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.994+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bb950 con 0x55e54a80c330 2023-10-06T05:25:12.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.996+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'894 uv894 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:12.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.996+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.997+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'895 uv895 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.997+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.998+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v121'896 uv870 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:12.999+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.000+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v121'897 uv897 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.000+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.004 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:25:13.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.001+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v121'898 uv898 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.001+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bd700 con 0x55e54a80c330 2023-10-06T05:25:13.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.002+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'899 uv899 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.002+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bd700 con 0x55e54a80c330 2023-10-06T05:25:13.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.004+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'900 uv900 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.004+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.005+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'901 uv901 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.005+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.006+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v121'902 uv876 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.006+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.007+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v121'903 uv903 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.008+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.010 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:25:13.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.009+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v121'904 uv904 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.009+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bd700 con 0x55e54a80c330 2023-10-06T05:25:13.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.010+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'905 uv905 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.010+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bd700 con 0x55e54a80c330 2023-10-06T05:25:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.011+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'906 uv906 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.011+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bc6f0 con 0x55e54a80c330 2023-10-06T05:25:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.013+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'907 uv907 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a80c330 2023-10-06T05:25:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.013+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e54a8bcc00 0x55e54a8befe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.013+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bf520 con 0x55e54a8bcc00 2023-10-06T05:25:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.013+0000 7fcb92b48700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e54a8bcc00 0x55e54a8befe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.013+0000 7fcb92b48700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e54a8bcc00 0x55e54a8befe0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.015+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'632 uv615 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.015+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bf8d0 con 0x55e54a8bcc00 2023-10-06T05:25:13.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.016+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'633 uv633 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.016+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8bfc80 con 0x55e54a8bcc00 2023-10-06T05:25:13.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.017+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'634 uv634 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.019 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:25:13.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.018+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c0520 con 0x55e54a8bcc00 2023-10-06T05:25:13.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.019+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v121'635 uv635 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.019+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c08d0 con 0x55e54a8bcc00 2023-10-06T05:25:13.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.020+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'636 uv636 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.020+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c0c80 con 0x55e54a8bcc00 2023-10-06T05:25:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.021+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v121'637 uv620 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.021+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.022+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v121'638 uv638 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.022+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.025 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:25:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.023+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v121'639 uv639 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.023+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.025+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'640 uv640 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.025+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.026+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'641 uv641 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.026+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.027+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v121'642 uv625 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.027+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.028+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v121'643 uv643 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.028+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.031 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:25:13.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.029+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v121'644 uv644 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.029+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4750 con 0x55e54a8bcc00 2023-10-06T05:25:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.031+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v121'645 uv645 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.031+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4750 con 0x55e54a8bcc00 2023-10-06T05:25:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.032+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'646 uv646 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.032+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.033+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v121'647 uv631 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.033+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.034+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v121'648 uv648 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.034+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.037 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:25:13.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.036+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v121'649 uv649 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.036+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c2040 con 0x55e54a8bcc00 2023-10-06T05:25:13.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.037+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'650 uv650 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.037+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c2040 con 0x55e54a8bcc00 2023-10-06T05:25:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.038+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'651 uv651 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.038+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c1030 con 0x55e54a8bcc00 2023-10-06T05:25:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.039+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'652 uv652 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.039+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55e54a8c1540 0x55e54a8c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.039+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c3e60 con 0x55e54a8c1540 2023-10-06T05:25:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.040+0000 7fcb92347700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55e54a8c1540 0x55e54a8c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.040+0000 7fcb92347700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55e54a8c1540 0x55e54a8c3920 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.041+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.041+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c3e60 con 0x55e54a8c1540 2023-10-06T05:25:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.043+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v121'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.043+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c3e60 con 0x55e54a8c1540 2023-10-06T05:25:13.045 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:25:13.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.044+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v121'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.044+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4e70 con 0x55e54a8c1540 2023-10-06T05:25:13.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.045+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.045+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4e70 con 0x55e54a8c1540 2023-10-06T05:25:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.046+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.046+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c3e60 con 0x55e54a8c1540 2023-10-06T05:25:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.047+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.047+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c3e60 con 0x55e54a8c1540 2023-10-06T05:25:13.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.049+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'121 uv120 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.049+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c3e60 con 0x55e54a8c1540 2023-10-06T05:25:13.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.050+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'122 uv122 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.050+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c3e60 con 0x55e54a8c1540 2023-10-06T05:25:13.053 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:25:13.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.051+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'123 uv123 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.051+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4340 con 0x55e54a8c1540 2023-10-06T05:25:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.052+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'124 uv124 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.052+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4340 con 0x55e54a8bcc00 2023-10-06T05:25:13.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.054+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.054+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4340 con 0x55e54a8bcc00 2023-10-06T05:25:13.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.055+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v121'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.055+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4340 con 0x55e54a8bcc00 2023-10-06T05:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:25:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.056+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v121'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.056+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c5350 con 0x55e54a8bcc00 2023-10-06T05:25:13.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.057+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.057+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c5350 con 0x55e54a8bcc00 2023-10-06T05:25:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.058+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.058+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c4340 con 0x55e54a8bcc00 2023-10-06T05:25:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.060+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8bcc00 2023-10-06T05:25:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.060+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e54a8c4850 0x55e54a8c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.060+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c7170 con 0x55e54a8c4850 2023-10-06T05:25:13.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.060+0000 7fcb93349700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e54a8c4850 0x55e54a8c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.060+0000 7fcb93349700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e54a8c4850 0x55e54a8c6c30 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.062+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v121'316 uv310 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.062+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c7560 con 0x55e54a8c4850 2023-10-06T05:25:13.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.063+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v121'317 uv317 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.063+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c7a50 con 0x55e54a8c4850 2023-10-06T05:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:25:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.064+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v121'318 uv318 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.064+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c0150 con 0x55e54a8c4850 2023-10-06T05:25:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.065+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'319 uv319 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.065+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c9490 con 0x55e54a8c4850 2023-10-06T05:25:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.066+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.066+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c8340 con 0x55e54a8c4850 2023-10-06T05:25:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.068+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.068+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c8740 con 0x55e54a8c4850 2023-10-06T05:25:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.069+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v121'322 uv315 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.069+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c8af0 con 0x55e54a8c4850 2023-10-06T05:25:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.070+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v121'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.070+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c8af0 con 0x55e54a8c4850 2023-10-06T05:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:25:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.071+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v121'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.071+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c8af0 con 0x55e54a8c4850 2023-10-06T05:25:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.072+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'325 uv325 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.072+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8c8f40 con 0x55e54a8c4850 2023-10-06T05:25:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.074+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'326 uv326 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c01b020 con 0x55e54a8c4850 2023-10-06T05:25:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.074+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55e54a8c98d0 0x55e54a8cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.074+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc310 con 0x55e54a8c98d0 2023-10-06T05:25:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.074+0000 7fcb92b48700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55e54a8c98d0 0x55e54a8cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.074+0000 7fcb92b48700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55e54a8c98d0 0x55e54a8cbcb0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.076+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v121'143 uv142 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8c98d0 2023-10-06T05:25:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.076+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc310 con 0x55e54a8c98d0 2023-10-06T05:25:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.077+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v121'144 uv144 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8c98d0 2023-10-06T05:25:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.077+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc770 con 0x55e54a8c98d0 2023-10-06T05:25:13.081 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:25:13.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.080+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v121'145 uv145 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8c98d0 2023-10-06T05:25:13.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.080+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc770 con 0x55e54a8c98d0 2023-10-06T05:25:13.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.082+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'146 uv146 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8c98d0 2023-10-06T05:25:13.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.082+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc770 con 0x55e54a8c98d0 2023-10-06T05:25:13.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.085+0000 7fcb92b48700 1 -- 172.21.15.45:0/1107516806 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'147 uv147 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb84023070 con 0x55e54a8c98d0 2023-10-06T05:25:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.085+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc770 con 0x55e54a8c1540 2023-10-06T05:25:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.086+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.086+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc770 con 0x55e54a8c1540 2023-10-06T05:25:13.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.087+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v121'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.087+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc770 con 0x55e54a8c1540 2023-10-06T05:25:13.090 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:25:13.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.088+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v121'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.088+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cd750 con 0x55e54a8c1540 2023-10-06T05:25:13.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.090+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.090+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cd750 con 0x55e54a8c1540 2023-10-06T05:25:13.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.091+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.091+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cc740 con 0x55e54a8c1540 2023-10-06T05:25:13.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.092+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.092+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e54a8ccdc0 0x55e54a8cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.092+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cf810 con 0x55e54a8ccdc0 2023-10-06T05:25:13.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.092+0000 7fcb92347700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e54a8ccdc0 0x55e54a8cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.093+0000 7fcb92347700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e54a8ccdc0 0x55e54a8cf1a0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.094+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v121'284 uv283 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.094+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.096+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v121'285 uv285 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.096+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.098 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:25:13.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.097+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v121'286 uv286 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.097+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.098+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'287 uv287 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.098+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.099+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'288 uv288 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.099+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.100+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v121'289 uv282 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.100+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.101+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v121'290 uv290 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.101+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.109 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:25:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.102+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v121'291 uv291 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.102+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8cfbf0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.103+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'292 uv292 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.103+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d2960 con 0x55e54a8ccdc0 2023-10-06T05:25:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.105+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'293 uv293 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8ccdc0 2023-10-06T05:25:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.105+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d00e0 con 0x55e54a8c1540 2023-10-06T05:25:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.106+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v121'124 uv123 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.106+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d00e0 con 0x55e54a8c1540 2023-10-06T05:25:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.107+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'125 uv125 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.107+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d00e0 con 0x55e54a8c1540 2023-10-06T05:25:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.108+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v121'126 uv126 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:25:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.108+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d00e0 con 0x55e54a8c1540 2023-10-06T05:25:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.109+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'127 uv127 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.109+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55e54a8d05f0 0x55e54a8d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.109+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8d05f0 2023-10-06T05:25:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.110+0000 7fcb93349700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55e54a8d05f0 0x55e54a8d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.110+0000 7fcb93349700 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55e54a8d05f0 0x55e54a8d0a00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.111+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8d05f0 2023-10-06T05:25:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.111+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8d05f0 2023-10-06T05:25:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.114+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v121'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8d05f0 2023-10-06T05:25:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.114+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8d05f0 2023-10-06T05:25:13.118 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:25:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.116+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v121'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8d05f0 2023-10-06T05:25:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.116+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d40c0 con 0x55e54a8d05f0 2023-10-06T05:25:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.119+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8d05f0 2023-10-06T05:25:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.119+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d40c0 con 0x55e54a8d05f0 2023-10-06T05:25:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.121+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8d05f0 2023-10-06T05:25:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.121+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8d05f0 2023-10-06T05:25:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.122+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8d05f0 2023-10-06T05:25:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.122+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c4850 2023-10-06T05:25:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.123+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v121'145 uv144 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8c4850 2023-10-06T05:25:13.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.123+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c4850 2023-10-06T05:25:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.124+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v121'146 uv146 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8c4850 2023-10-06T05:25:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.124+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c4850 2023-10-06T05:25:13.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.125+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v121'147 uv147 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8c4850 2023-10-06T05:25:13.129 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:25:13.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.125+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c4850 2023-10-06T05:25:13.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.127+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'148 uv148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8c4850 2023-10-06T05:25:13.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.127+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c4850 2023-10-06T05:25:13.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.128+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'149 uv149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fcb8c06adb0 con 0x55e54a8c4850 2023-10-06T05:25:13.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.128+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c1540 2023-10-06T05:25:13.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.129+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v121'351 uv343 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.129+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c1540 2023-10-06T05:25:13.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.131+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v121'352 uv352 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.131+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c1540 2023-10-06T05:25:13.133 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:25:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.132+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v121'353 uv353 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.132+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c1540 2023-10-06T05:25:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.133+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v121'354 uv354 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.133+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d3500 con 0x55e54a8c1540 2023-10-06T05:25:13.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.134+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'355 uv355 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.134+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d3500 con 0x55e54a8c1540 2023-10-06T05:25:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.135+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v121'356 uv350 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.135+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c1540 2023-10-06T05:25:13.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.136+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v121'357 uv357 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.136+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c1540 2023-10-06T05:25:13.139 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:25:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.137+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v121'358 uv358 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.137+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d40c0 con 0x55e54a8c1540 2023-10-06T05:25:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.138+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'359 uv359 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.138+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d40c0 con 0x55e54a8c1540 2023-10-06T05:25:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.139+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'360 uv360 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.139+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d30b0 con 0x55e54a8c1540 2023-10-06T05:25:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.140+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'361 uv361 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.140+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e54a8d35a0 con 0x55e54a8c1540 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb92347700 1 -- 172.21.15.45:0/1107516806 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'362 uv362 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcb880088d0 con 0x55e54a8c1540 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55e54a8c1540 msgr2=0x55e54a8c3920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55e54a8c1540 0x55e54a8c3920 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55e54a80c330 msgr2=0x55e54a8ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55e54a80c330 0x55e54a8ae5a0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e54a8ccdc0 msgr2=0x55e54a8cf1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55e54a8ccdc0 0x55e54a8cf1a0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e54a8bcc00 msgr2=0x55e54a8befe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55e54a8bcc00 0x55e54a8befe0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55e54a8d05f0 msgr2=0x55e54a8d0a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55e54a8d05f0 0x55e54a8d0a00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 reap_dead start 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55e54a8c98d0 msgr2=0x55e54a8cbcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55e54a8c98d0 0x55e54a8cbcb0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e54a8c4850 msgr2=0x55e54a8c6c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e54a8c4850 0x55e54a8c6c30 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb78067aa0 msgr2=0x7fcb78069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb78067aa0 0x7fcb78069f50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400a150 tx=0x7fcb84011000 comp rx=0 tx=0).stop 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89ef00 msgr2=0x55e54a8a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 --2- 172.21.15.45:0/1107516806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e54a89ef00 0x55e54a8a13a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c008880 tx=0x7fcb8c005bb0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb93349700 1 -- 172.21.15.45:0/1107516806 reap_dead start 2023-10-06T05:25:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 shutdown_connections 2023-10-06T05:25:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 >> 172.21.15.45:0/1107516806 conn(0x55e54a7cc450 msgr2=0x55e54a8983e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 shutdown_connections 2023-10-06T05:25:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.142+0000 7fcb9992a040 1 -- 172.21.15.45:0/1107516806 wait complete. 2023-10-06T05:25:13.148 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2023-10-06T05:25:13.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.237+0000 7f0afcf34040 1 Processor -- start 2023-10-06T05:25:13.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.237+0000 7f0afcf34040 1 -- start start 2023-10-06T05:25:13.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0afcf34040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd3b70 0x55e490b40c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0afcf34040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd4f50 0x55e490b2ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0afcf34040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 0x55e490bd1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0afcf34040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e490a4a980 con 0x55e490bd4f50 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0afcf34040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e490a4a620 con 0x55e490bd7a30 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0afcf34040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e490a41400 con 0x55e490bd3b70 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd4f50 0x55e490b2ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd4f50 0x55e490b2ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43516/0 (socket says 172.21.15.45:43516) 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 -- 172.21.15.45:0/2935383748 learned_addr learned my addr 172.21.15.45:0/2935383748 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af6152700 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd3b70 0x55e490b40c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af6953700 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 0x55e490bd1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 -- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd3b70 msgr2=0x55e490b40c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd3b70 0x55e490b40c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 -- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 msgr2=0x55e490bd1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 0x55e490bd1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 -- 172.21.15.45:0/2935383748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af000a0e0 con 0x55e490bd4f50 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.238+0000 7f0af5951700 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd4f50 0x55e490b2ffd0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009980 tx=0x7f0ae000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aee2dff6a8edf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0af494f700 1 -- 172.21.15.45:0/2935383748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000f040 con 0x55e490bd4f50 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0af494f700 1 -- 172.21.15.45:0/2935383748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0008950 con 0x55e490bd4f50 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0af494f700 1 -- 172.21.15.45:0/2935383748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0008cf0 con 0x55e490bd4f50 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 -- 172.21.15.45:0/2935383748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd4f50 msgr2=0x55e490b2ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd4f50 0x55e490b2ffd0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009980 tx=0x7f0ae000c040 comp rx=0 tx=0).stop 2023-10-06T05:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 -- 172.21.15.45:0/2935383748 shutdown_connections 2023-10-06T05:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 0x55e490bd1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd3b70 0x55e490b40c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 --2- 172.21.15.45:0/2935383748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd4f50 0x55e490b2ffd0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 -- 172.21.15.45:0/2935383748 >> 172.21.15.45:0/2935383748 conn(0x55e490b015f0 msgr2=0x55e490bd6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 -- 172.21.15.45:0/2935383748 shutdown_connections 2023-10-06T05:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.239+0000 7f0afcf34040 1 -- 172.21.15.45:0/2935383748 wait complete. 2023-10-06T05:25:13.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.240+0000 7f0afcf34040 1 Processor -- start 2023-10-06T05:25:13.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.240+0000 7f0afcf34040 1 -- start start 2023-10-06T05:25:13.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.241+0000 7f0afcf34040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd3b70 0x55e490bd6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.241+0000 7f0afcf34040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd4f50 0x55e490bdb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.241+0000 7f0afcf34040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 0x55e490bd15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.241+0000 7f0afcf34040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e490bda930 con 0x55e490bd3b70 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.241+0000 7f0afcf34040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e490b2d4a0 con 0x55e490bd7a30 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.241+0000 7f0afcf34040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e490b40f40 con 0x55e490bd4f50 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd3b70 0x55e490bd6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd3b70 0x55e490bd6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43520/0 (socket says 172.21.15.45:43520) 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 -- 172.21.15.45:0/825421787 learned_addr learned my addr 172.21.15.45:0/825421787 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6953700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 0x55e490bd15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af5951700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd4f50 0x55e490bdb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 -- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd4f50 msgr2=0x55e490bdb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e490bd4f50 0x55e490bdb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 -- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 msgr2=0x55e490bd15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e490bd7a30 0x55e490bd15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 -- 172.21.15.45:0/825421787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009580 con 0x55e490bd3b70 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0af6152700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd3b70 0x55e490bd6770 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c6c0 tx=0x7f0ae8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0aeeffd700 1 -- 172.21.15.45:0/825421787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800ce40 con 0x55e490bd3b70 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0aeeffd700 1 -- 172.21.15.45:0/825421787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae800f450 con 0x55e490bd3b70 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e490a593d0 con 0x55e490bd3b70 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0aeeffd700 1 -- 172.21.15.45:0/825421787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800f5e0 con 0x55e490bd3b70 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.242+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e490a4cae0 con 0x55e490bd3b70 2023-10-06T05:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.244+0000 7f0aeeffd700 1 -- 172.21.15.45:0/825421787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ae8015020 con 0x55e490bd3b70 2023-10-06T05:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.244+0000 7f0aeeffd700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0ad8065890 0x7f0ad8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.244+0000 7f0aeeffd700 1 -- 172.21.15.45:0/825421787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7f0ae8084070 con 0x55e490bd3b70 2023-10-06T05:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.245+0000 7f0afcf34040 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e490bd1b00 0x55e490bdcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.245+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v8 -- 0x55e490b2cf20 con 0x55e490bd1b00 2023-10-06T05:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.245+0000 7f0af5951700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0ad8065890 0x7f0ad8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.245+0000 7f0af6953700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e490bd1b00 0x55e490bdcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.245+0000 7f0af5951700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0ad8065890 0x7f0ad8067d40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00098a0 tx=0x7f0ae0018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.245+0000 7f0af6953700 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e490bd1b00 0x55e490bdcfd0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0af6953700 1 -- 172.21.15.45:0/825421787 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v121'150 uv150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f0af0023040 con 0x55e490bd1b00 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e490bd1b00 msgr2=0x55e490bdcfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55e490bd1b00 0x55e490bdcfd0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0ad8065890 msgr2=0x7f0ad8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0ad8065890 0x7f0ad8067d40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00098a0 tx=0x7f0ae0018000 comp rx=0 tx=0).stop 2023-10-06T05:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd3b70 msgr2=0x55e490bd6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 --2- 172.21.15.45:0/825421787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e490bd3b70 0x55e490bd6770 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c6c0 tx=0x7f0ae8000bc0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0af6953700 1 -- 172.21.15.45:0/825421787 reap_dead start 2023-10-06T05:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 shutdown_connections 2023-10-06T05:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 >> 172.21.15.45:0/825421787 conn(0x55e490b015f0 msgr2=0x55e490afebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 shutdown_connections 2023-10-06T05:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.247+0000 7f0afcf34040 1 -- 172.21.15.45:0/825421787 wait complete. 2023-10-06T05:25:13.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:25:13.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.389+0000 7f5beddd4700 1 Processor -- start 2023-10-06T05:25:13.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.389+0000 7f5beddd4700 1 -- start start 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5beddd4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be806aa70 0x7f5be80c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5beddd4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5be80c7050 0x7f5be815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5beddd4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be8067d80 con 0x7f5be806ae40 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5beddd4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be80ac620 con 0x7f5be80c7050 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5beddd4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80c75b0 con 0x7f5be806aa70 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5be6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5be80c7050 0x7f5be815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5be6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5be80c7050 0x7f5be815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43522/0 (socket says 172.21.15.45:43522) 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.390+0000 7f5be6ffd700 1 -- 172.21.15.45:0/3574907969 learned_addr learned my addr 172.21.15.45:0/3574907969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:13.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be77fe700 1 --2- 172.21.15.45:0/3574907969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be806aa70 0x7f5be80c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be6ffd700 1 -- 172.21.15.45:0/3574907969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be806aa70 msgr2=0x7f5be80c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be6ffd700 1 --2- 172.21.15.45:0/3574907969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be806aa70 0x7f5be80c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be6ffd700 1 -- 172.21.15.45:0/3574907969 >> v1:172.21.15.144:6789/0 conn(0x7f5be806ae40 legacy=0x7f5be806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be6ffd700 1 -- 172.21.15.45:0/3574907969 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be80c7890 con 0x7f5be80c7050 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be6ffd700 1 --2- 172.21.15.45:0/3574907969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be80c7050 0x7f5be815c1b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00099b0 tx=0x7f5bd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec31952125a8c2f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be67fc700 1 -- 172.21.15.45:0/3574907969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0009ea0 con 0x7f5be80c7050 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be67fc700 1 -- 172.21.15.45:0/3574907969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000b860 con 0x7f5be80c7050 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.391+0000 7f5be67fc700 1 -- 172.21.15.45:0/3574907969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000bc00 con 0x7f5be80c7050 2023-10-06T05:25:13.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 -- 172.21.15.45:0/3574907969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be80c7050 msgr2=0x7f5be815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 --2- 172.21.15.45:0/3574907969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be80c7050 0x7f5be815c1b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00099b0 tx=0x7f5bd0009250 comp rx=0 tx=0).stop 2023-10-06T05:25:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 -- 172.21.15.45:0/3574907969 shutdown_connections 2023-10-06T05:25:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 --2- 172.21.15.45:0/3574907969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be806aa70 0x7f5be80c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 --2- 172.21.15.45:0/3574907969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be80c7050 0x7f5be815c1b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 -- 172.21.15.45:0/3574907969 >> 172.21.15.45:0/3574907969 conn(0x7f5be80cc570 msgr2=0x7f5be80cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 -- 172.21.15.45:0/3574907969 shutdown_connections 2023-10-06T05:25:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.392+0000 7f5beddd4700 1 -- 172.21.15.45:0/3574907969 wait complete. 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.393+0000 7f5beddd4700 1 Processor -- start 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.393+0000 7f5beddd4700 1 -- start start 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5beddd4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806aa70 0x7f5be815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5beddd4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be806ae40 0x7f5be81686f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5beddd4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be80c7050 0x7f5be80c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5beddd4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be815cd40 con 0x7f5be806ae40 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5beddd4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be815d300 con 0x7f5be806aa70 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5beddd4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be815b7f0 con 0x7f5be80c7050 2023-10-06T05:25:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5be6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be806ae40 0x7f5be81686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5be6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be806ae40 0x7f5be81686f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43528/0 (socket says 172.21.15.45:43528) 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5be6ffd700 1 -- 172.21.15.45:0/2529271098 learned_addr learned my addr 172.21.15.45:0/2529271098 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5be7fff700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be80c7050 0x7f5be80c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.394+0000 7f5be77fe700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806aa70 0x7f5be815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5be6ffd700 1 -- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be80c7050 msgr2=0x7f5be80c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5be6ffd700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be80c7050 0x7f5be80c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5be6ffd700 1 -- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806aa70 msgr2=0x7f5be815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5be6ffd700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806aa70 0x7f5be815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5be6ffd700 1 -- 172.21.15.45:0/2529271098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00094f0 con 0x7f5be806ae40 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5be6ffd700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be806ae40 0x7f5be81686f0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f5be806fbf0 tx=0x7f5bd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5bdf7fe700 1 -- 172.21.15.45:0/2529271098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0000d10 con 0x7f5be806ae40 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5bdf7fe700 1 -- 172.21.15.45:0/2529271098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000bbe0 con 0x7f5be806ae40 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be815bad0 con 0x7f5be806ae40 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5bdf7fe700 1 -- 172.21.15.45:0/2529271098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0011670 con 0x7f5be806ae40 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.395+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be815b500 con 0x7f5be806ae40 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.396+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be815d470 con 0x7f5be806ae40 2023-10-06T05:25:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.397+0000 7f5bdf7fe700 1 -- 172.21.15.45:0/2529271098 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bd0011aa0 con 0x7f5be806ae40 2023-10-06T05:25:13.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.398+0000 7f5bdf7fe700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bc8066f30 0x7f5bc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.398+0000 7f5bdf7fe700 1 -- 172.21.15.45:0/2529271098 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8872+0+0 (secure 0 0 0) 0x7f5bd00871e0 con 0x7f5be806ae40 2023-10-06T05:25:13.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.398+0000 7f5be77fe700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bc8066f30 0x7f5bc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.398+0000 7f5be77fe700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bc8066f30 0x7f5bc80693e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009db0 tx=0x7f5bd80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.401+0000 7f5bdf7fe700 1 -- 172.21.15.45:0/2529271098 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bd0053830 con 0x7f5be806ae40 2023-10-06T05:25:13.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.654+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5be80c7b70 con 0x7f5be806ae40 2023-10-06T05:25:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.899+0000 7f5bdf7fe700 1 -- 172.21.15.45:0/2529271098 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v271) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5bd00150b0 con 0x7f5be806ae40 2023-10-06T05:25:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bc8066f30 msgr2=0x7f5bc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bc8066f30 0x7f5bc80693e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009db0 tx=0x7f5bd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be806ae40 msgr2=0x7f5be81686f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be806ae40 0x7f5be81686f0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f5be806fbf0 tx=0x7f5bd0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 shutdown_connections 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5bc8066f30 0x7f5bc80693e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be80c7050 0x7f5be80c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806aa70 0x7f5be815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 --2- 172.21.15.45:0/2529271098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be806ae40 0x7f5be81686f0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 >> 172.21.15.45:0/2529271098 conn(0x7f5be80cc570 msgr2=0x7f5be8070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.902+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 shutdown_connections 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:13.903+0000 7f5beddd4700 1 -- 172.21.15.45:0/2529271098 wait complete. 2023-10-06T05:25:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:25:13.909 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:13.907+0000 7f06aec0b700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-10-06T05:25:13.910 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:13.909+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:13.910 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:13.909+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:25:13.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:13.921 INFO:tasks.ceph.mds.b.smithi144.stderr: -29> 2023-10-06T05:25:13.907+0000 7f06aec0b700 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2023-10-06T05:25:13.943 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:25:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.056+0000 7f970fe56700 1 Processor -- start 2023-10-06T05:25:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.057+0000 7f970fe56700 1 -- start start 2023-10-06T05:25:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970fe56700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97080ccfc0 0x7f970815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970fe56700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080ce690 con 0x7f9708153620 2023-10-06T05:25:14.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970fe56700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970815d510 con 0x7f97081539f0 2023-10-06T05:25:14.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970fe56700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970815a7a0 con 0x7f97080ccfc0 2023-10-06T05:25:14.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970dbf2700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9708153620 0x7f97080cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39650/0 (socket says 172.21.15.45:39650) 2023-10-06T05:25:14.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970e3f3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97080ccfc0 0x7f970815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970dbf2700 1 -- 172.21.15.45:0/2963772683 learned_addr learned my addr 172.21.15.45:0/2963772683 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970e3f3700 1 -- 172.21.15.45:0/2963772683 >> v1:172.21.15.144:6789/0 conn(0x7f97081539f0 legacy=0x7f97080ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970e3f3700 1 -- 172.21.15.45:0/2963772683 >> v1:172.21.15.45:6789/0 conn(0x7f9708153620 legacy=0x7f97080cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970cbf0700 1 -- 172.21.15.45:0/2963772683 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814563947 0 0) 0x7f97080ce690 con 0x7f9708153620 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970e3f3700 1 -- 172.21.15.45:0/2963772683 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970815d7f0 con 0x7f97080ccfc0 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970e3f3700 1 --2- 172.21.15.45:0/2963772683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080ccfc0 0x7f970815a260 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f9700009e60 tx=0x7f9700009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1841cb36ae2e9aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.058+0000 7f970cbf0700 1 -- 172.21.15.45:0/2963772683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000e070 con 0x7f97080ccfc0 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970cbf0700 1 -- 172.21.15.45:0/2963772683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970000bc30 con 0x7f97080ccfc0 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970cbf0700 1 -- 172.21.15.45:0/2963772683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700012430 con 0x7f97080ccfc0 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970fe56700 1 -- 172.21.15.45:0/2963772683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080ccfc0 msgr2=0x7f970815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970fe56700 1 --2- 172.21.15.45:0/2963772683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080ccfc0 0x7f970815a260 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f9700009e60 tx=0x7f9700009780 comp rx=0 tx=0).stop 2023-10-06T05:25:14.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970fe56700 1 -- 172.21.15.45:0/2963772683 shutdown_connections 2023-10-06T05:25:14.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970fe56700 1 --2- 172.21.15.45:0/2963772683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080ccfc0 0x7f970815a260 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970fe56700 1 -- 172.21.15.45:0/2963772683 >> 172.21.15.45:0/2963772683 conn(0x7f970806ed50 msgr2=0x7f970806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:14.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970fe56700 1 -- 172.21.15.45:0/2963772683 shutdown_connections 2023-10-06T05:25:14.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.059+0000 7f970fe56700 1 -- 172.21.15.45:0/2963772683 wait complete. 2023-10-06T05:25:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.060+0000 7f970fe56700 1 Processor -- start 2023-10-06T05:25:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.060+0000 7f970fe56700 1 -- start start 2023-10-06T05:25:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970fe56700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97080ccfc0 0x7f97080b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970fe56700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708153620 0x7f970815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970fe56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080b56e0 0x7f970814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970fe56700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080b2ef0 con 0x7f97080b56e0 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970fe56700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97081555d0 con 0x7f9708153620 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970fe56700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970815ae30 con 0x7f97080ccfc0 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970e3f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080b56e0 0x7f970814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970e3f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080b56e0 0x7f970814fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43552/0 (socket says 172.21.15.45:43552) 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970e3f3700 1 -- 172.21.15.45:0/3210487744 learned_addr learned my addr 172.21.15.45:0/3210487744 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970dbf2700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97080ccfc0 0x7f97080b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.061+0000 7f970d3f1700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708153620 0x7f970815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970e3f3700 1 -- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97080ccfc0 msgr2=0x7f97080b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970e3f3700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97080ccfc0 0x7f97080b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970e3f3700 1 -- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708153620 msgr2=0x7f970815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970e3f3700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708153620 0x7f970815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970e3f3700 1 -- 172.21.15.45:0/3210487744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700009a20 con 0x7f97080b56e0 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970e3f3700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080b56e0 0x7f970814fae0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f970806f900 tx=0x7f9700019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f96fdffb700 1 -- 172.21.15.45:0/3210487744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970001b420 con 0x7f97080b56e0 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f96fdffb700 1 -- 172.21.15.45:0/3210487744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970001b580 con 0x7f97080b56e0 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970815b0d0 con 0x7f97080b56e0 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f96fdffb700 1 -- 172.21.15.45:0/3210487744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970001b940 con 0x7f97080b56e0 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.062+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970815b690 con 0x7f97080b56e0 2023-10-06T05:25:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.063+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9708088a60 con 0x7f97080b56e0 2023-10-06T05:25:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.064+0000 7f96fdffb700 1 -- 172.21.15.45:0/3210487744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f970001bcb0 con 0x7f97080b56e0 2023-10-06T05:25:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.064+0000 7f96fdffb700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96ec066f30 0x7f96ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.064+0000 7f96fdffb700 1 -- 172.21.15.45:0/3210487744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7f97000870a0 con 0x7f97080b56e0 2023-10-06T05:25:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.064+0000 7f970dbf2700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96ec066f30 0x7f96ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.065+0000 7f970dbf2700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96ec066f30 0x7f96ec0693e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9704007fd0 tx=0x7f9704007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.069+0000 7f96fdffb700 1 -- 172.21.15.45:0/3210487744 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9700053620 con 0x7f97080b56e0 2023-10-06T05:25:14.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.320+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9708155710 con 0x7f97080b56e0 2023-10-06T05:25:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.320+0000 7f96fdffb700 1 -- 172.21.15.45:0/3210487744 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 273 v273) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f9700056c40 con 0x7f97080b56e0 2023-10-06T05:25:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:14.322 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":273,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":273,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:13.909983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96ec066f30 msgr2=0x7f96ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96ec066f30 0x7f96ec0693e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9704007fd0 tx=0x7f9704007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080b56e0 msgr2=0x7f970814fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080b56e0 0x7f970814fae0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f970806f900 tx=0x7f9700019000 comp rx=0 tx=0).stop 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 shutdown_connections 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f96ec066f30 0x7f96ec0693e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97080b56e0 0x7f970814fae0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97080ccfc0 0x7f97080b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 --2- 172.21.15.45:0/3210487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708153620 0x7f970815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 >> 172.21.15.45:0/3210487744 conn(0x7f970806ed50 msgr2=0x7f9708150f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 shutdown_connections 2023-10-06T05:25:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.323+0000 7f970fe56700 1 -- 172.21.15.45:0/3210487744 wait complete. 2023-10-06T05:25:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 273 2023-10-06T05:25:14.338 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:14.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.468+0000 7f97cb11d700 1 Processor -- start 2023-10-06T05:25:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.468+0000 7f97cb11d700 1 -- start start 2023-10-06T05:25:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.469+0000 7f97cb11d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40cd340 0x7f97c4157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.469+0000 7f97cb11d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c415b000 con 0x7f97c40c8ff0 2023-10-06T05:25:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.469+0000 7f97cb11d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c415b2e0 con 0x7f97c40c93c0 2023-10-06T05:25:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.469+0000 7f97cb11d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40c9830 con 0x7f97c40cd340 2023-10-06T05:25:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.469+0000 7f97c8eb9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f97c40c8ff0 0x7f97c40caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39654/0 (socket says 172.21.15.45:39654) 2023-10-06T05:25:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.469+0000 7f97c8eb9700 1 -- 172.21.15.45:0/87737007 learned_addr learned my addr 172.21.15.45:0/87737007 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:14.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.469+0000 7f97c96ba700 1 --2- 172.21.15.45:0/87737007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40cd340 0x7f97c4157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726425970 0 0) 0x7f97c415b000 con 0x7f97c40c8ff0 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b0003570 con 0x7f97c40c8ff0 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017982573 0 0) 0x7f97c415b2e0 con 0x7f97c40c93c0 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97c415b000 con 0x7f97c40c93c0 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065368244 0 0) 0x7f97b0003570 con 0x7f97c40c8ff0 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c96ba700 1 -- 172.21.15.45:0/87737007 >> v1:172.21.15.144:6789/0 conn(0x7f97c40c93c0 legacy=0x7f97c4156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c96ba700 1 -- 172.21.15.45:0/87737007 >> v1:172.21.15.45:6789/0 conn(0x7f97c40c8ff0 legacy=0x7f97c40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c96ba700 1 -- 172.21.15.45:0/87737007 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c415b5c0 con 0x7f97c40cd340 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c96ba700 1 --2- 172.21.15.45:0/87737007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40cd340 0x7f97c4157200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009e90 tx=0x7f97bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22a24a290f0826a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00e070 con 0x7f97c40cd340 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc00b870 con 0x7f97c40cd340 2023-10-06T05:25:14.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.470+0000 7f97c37fe700 1 -- 172.21.15.45:0/87737007 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4001470 con 0x7f97c40cd340 2023-10-06T05:25:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.471+0000 7f97cb11d700 1 -- 172.21.15.45:0/87737007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40cd340 msgr2=0x7f97c4157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.471+0000 7f97cb11d700 1 --2- 172.21.15.45:0/87737007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40cd340 0x7f97c4157200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009e90 tx=0x7f97bc009780 comp rx=0 tx=0).stop 2023-10-06T05:25:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.471+0000 7f97cb11d700 1 -- 172.21.15.45:0/87737007 shutdown_connections 2023-10-06T05:25:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.471+0000 7f97cb11d700 1 --2- 172.21.15.45:0/87737007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40cd340 0x7f97c4157200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.471+0000 7f97cb11d700 1 -- 172.21.15.45:0/87737007 >> 172.21.15.45:0/87737007 conn(0x7f97c40cdf50 msgr2=0x7f97c40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.471+0000 7f97cb11d700 1 -- 172.21.15.45:0/87737007 shutdown_connections 2023-10-06T05:25:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.471+0000 7f97cb11d700 1 -- 172.21.15.45:0/87737007 wait complete. 2023-10-06T05:25:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.472+0000 7f97cb11d700 1 Processor -- start 2023-10-06T05:25:14.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.472+0000 7f97cb11d700 1 -- start start 2023-10-06T05:25:14.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97cb11d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97c40c8ff0 0x7f97c40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97cb11d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40c93c0 0x7f97c40ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97cb11d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c40cd340 0x7f97c40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97cb11d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c416cff0 con 0x7f97c40cd340 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97cb11d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40b1230 con 0x7f97c40c8ff0 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97cb11d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40ad350 con 0x7f97c40c93c0 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97c96ba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c40cd340 0x7f97c40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97c96ba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c40cd340 0x7f97c40cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43562/0 (socket says 172.21.15.45:43562) 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.473+0000 7f97c96ba700 1 -- 172.21.15.45:0/1903979764 learned_addr learned my addr 172.21.15.45:0/1903979764 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c3fff700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40c93c0 0x7f97c40ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c8eb9700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97c40c8ff0 0x7f97c40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c96ba700 1 -- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40c93c0 msgr2=0x7f97c40ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c96ba700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40c93c0 0x7f97c40ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c96ba700 1 -- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97c40c8ff0 msgr2=0x7f97c40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c96ba700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97c40c8ff0 0x7f97c40b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c96ba700 1 -- 172.21.15.45:0/1903979764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc009a20 con 0x7f97c40cd340 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c96ba700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c40cd340 0x7f97c40cae50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009de0 tx=0x7f97bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c0ff9700 1 -- 172.21.15.45:0/1903979764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc012d80 con 0x7f97c40cd340 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c0ff9700 1 -- 172.21.15.45:0/1903979764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc01e450 con 0x7f97c40cd340 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c40ad630 con 0x7f97c40cd340 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97c0ff9700 1 -- 172.21.15.45:0/1903979764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc01e810 con 0x7f97c40cd340 2023-10-06T05:25:14.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.474+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c416d520 con 0x7f97c40cd340 2023-10-06T05:25:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.475+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c40c8c30 con 0x7f97c40cd340 2023-10-06T05:25:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.476+0000 7f97c0ff9700 1 -- 172.21.15.45:0/1903979764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97bc01ec40 con 0x7f97c40cd340 2023-10-06T05:25:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.476+0000 7f97c0ff9700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f97a4066f30 0x7f97a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.476+0000 7f97c0ff9700 1 -- 172.21.15.45:0/1903979764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7f97bc087200 con 0x7f97c40cd340 2023-10-06T05:25:14.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.477+0000 7f97c8eb9700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f97a4066f30 0x7f97a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.477+0000 7f97c8eb9700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f97a4066f30 0x7f97a40693e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f97b40011e0 tx=0x7f97b40015c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.481+0000 7f97c0ff9700 1 -- 172.21.15.45:0/1903979764 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97bc053720 con 0x7f97c40cd340 2023-10-06T05:25:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.731+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97c416d740 con 0x7f97c40cd340 2023-10-06T05:25:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.732+0000 7f97c0ff9700 1 -- 172.21.15.45:0/1903979764 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 273 v273) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f97bc056d40 con 0x7f97c40cd340 2023-10-06T05:25:14.734 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:14.734 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":273,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":273,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:13.909983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f97a4066f30 msgr2=0x7f97a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f97a4066f30 0x7f97a40693e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f97b40011e0 tx=0x7f97b40015c0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c40cd340 msgr2=0x7f97c40cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c40cd340 0x7f97c40cae50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009de0 tx=0x7f97bc017000 comp rx=0 tx=0).stop 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 shutdown_connections 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f97a4066f30 0x7f97a40693e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97c40cd340 0x7f97c40cae50 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97c40c8ff0 0x7f97c40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 --2- 172.21.15.45:0/1903979764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97c40c93c0 0x7f97c40ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 >> 172.21.15.45:0/1903979764 conn(0x7f97c40cdf50 msgr2=0x7f97c41472d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 shutdown_connections 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.734+0000 7f97cb11d700 1 -- 172.21.15.45:0/1903979764 wait complete. 2023-10-06T05:25:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 273 2023-10-06T05:25:14.749 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:14.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.880+0000 7f4ae209e700 1 Processor -- start 2023-10-06T05:25:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.880+0000 7f4ae209e700 1 -- start start 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.880+0000 7f4ae209e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc159170 0x7f4adc159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.880+0000 7f4ae209e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4adc15d040 0x7f4adc068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4ae209e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc0ca6d0 con 0x7f4adc069b90 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4ae209e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc15d550 con 0x7f4adc15d040 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4ae209e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc15d810 con 0x7f4adc159170 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4adc15d040 0x7f4adc068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4adc15d040 0x7f4adc068bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43570/0 (socket says 172.21.15.45:43570) 2023-10-06T05:25:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 -- 172.21.15.45:0/2645810638 learned_addr learned my addr 172.21.15.45:0/2645810638 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adb7fe700 1 --2- 172.21.15.45:0/2645810638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc159170 0x7f4adc159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 -- 172.21.15.45:0/2645810638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc159170 msgr2=0x7f4adc159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 --2- 172.21.15.45:0/2645810638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc159170 0x7f4adc159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 -- 172.21.15.45:0/2645810638 >> v1:172.21.15.144:6789/0 conn(0x7f4adc069b90 legacy=0x7f4adc0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 -- 172.21.15.45:0/2645810638 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc15daf0 con 0x7f4adc15d040 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adb7fe700 1 --2- 172.21.15.45:0/2645810638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc159170 0x7f4adc159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4adaffd700 1 --2- 172.21.15.45:0/2645810638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc15d040 0x7f4adc068bc0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00099b0 tx=0x7f4ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2aeceb9ba7abf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.881+0000 7f4ada7fc700 1 -- 172.21.15.45:0/2645810638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000b860 con 0x7f4adc15d040 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ada7fc700 1 -- 172.21.15.45:0/2645810638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad000b9c0 con 0x7f4adc15d040 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ada7fc700 1 -- 172.21.15.45:0/2645810638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000bd60 con 0x7f4adc15d040 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 -- 172.21.15.45:0/2645810638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc15d040 msgr2=0x7f4adc068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2645810638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc15d040 0x7f4adc068bc0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00099b0 tx=0x7f4ad0009250 comp rx=0 tx=0).stop 2023-10-06T05:25:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 -- 172.21.15.45:0/2645810638 shutdown_connections 2023-10-06T05:25:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2645810638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc159170 0x7f4adc159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2645810638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc15d040 0x7f4adc068bc0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 -- 172.21.15.45:0/2645810638 >> 172.21.15.45:0/2645810638 conn(0x7f4adc06f7c0 msgr2=0x7f4adc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 -- 172.21.15.45:0/2645810638 shutdown_connections 2023-10-06T05:25:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.882+0000 7f4ae209e700 1 -- 172.21.15.45:0/2645810638 wait complete. 2023-10-06T05:25:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 Processor -- start 2023-10-06T05:25:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 -- start start 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc069b90 0x7f4adc0a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc159170 0x7f4adc16cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4adc15d040 0x7f4adc16d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0cc8f0 con 0x7f4adc159170 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0cbd70 con 0x7f4adc15d040 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.884+0000 7f4ae209e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0cc050 con 0x7f4adc069b90 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc159170 0x7f4adc16cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc159170 0x7f4adc16cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43576/0 (socket says 172.21.15.45:43576) 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 -- 172.21.15.45:0/2799887395 learned_addr learned my addr 172.21.15.45:0/2799887395 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adb7fe700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc069b90 0x7f4adc0a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adbfff700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4adc15d040 0x7f4adc16d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 -- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc069b90 msgr2=0x7f4adc0a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc069b90 0x7f4adc0a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 -- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4adc15d040 msgr2=0x7f4adc16d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4adc15d040 0x7f4adc16d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 -- 172.21.15.45:0/2799887395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00094f0 con 0x7f4adc159170 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4adaffd700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc159170 0x7f4adc16cea0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009290 tx=0x7f4ad0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4ac37fe700 1 -- 172.21.15.45:0/2799887395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad001a450 con 0x7f4adc159170 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4ac37fe700 1 -- 172.21.15.45:0/2799887395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad001a5b0 con 0x7f4adc159170 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc0cc330 con 0x7f4adc159170 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4ac37fe700 1 -- 172.21.15.45:0/2799887395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad001a970 con 0x7f4adc159170 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.885+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc0a6f20 con 0x7f4adc159170 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.886+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc063b50 con 0x7f4adc159170 2023-10-06T05:25:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.887+0000 7f4ac37fe700 1 -- 172.21.15.45:0/2799887395 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ad001ad20 con 0x7f4adc159170 2023-10-06T05:25:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.887+0000 7f4ac37fe700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ac8066f30 0x7f4ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.887+0000 7f4ac37fe700 1 -- 172.21.15.45:0/2799887395 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7f4ad0086860 con 0x7f4adc159170 2023-10-06T05:25:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.888+0000 7f4adb7fe700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ac8066f30 0x7f4ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.888+0000 7f4adb7fe700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ac8066f30 0x7f4ac80693e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006f00 tx=0x7f4acc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:14.892+0000 7f4ac37fe700 1 -- 172.21.15.45:0/2799887395 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ad0052de0 con 0x7f4adc159170 2023-10-06T05:25:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.145+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4adc15ddd0 con 0x7f4adc159170 2023-10-06T05:25:15.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.145+0000 7f4ac37fe700 1 -- 172.21.15.45:0/2799887395 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f4ad0015090 con 0x7f4adc159170 2023-10-06T05:25:15.147 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:15.147 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":274,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2237111349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2237111349},{"type":"v1","addr":"172.21.15.144:6835","nonce":2237111349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":274}],"filesystems":[{"mdsmap":{"epoch":273,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:13.909983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.147+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ac8066f30 msgr2=0x7f4ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.147+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ac8066f30 0x7f4ac80693e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006f00 tx=0x7f4acc008040 comp rx=0 tx=0).stop 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.147+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc159170 msgr2=0x7f4adc16cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.147+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc159170 0x7f4adc16cea0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009290 tx=0x7f4ad0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 shutdown_connections 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ac8066f30 0x7f4ac80693e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4adc15d040 0x7f4adc16d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4adc069b90 0x7f4adc0a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 --2- 172.21.15.45:0/2799887395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4adc159170 0x7f4adc16cea0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 >> 172.21.15.45:0/2799887395 conn(0x7f4adc06f7c0 msgr2=0x7f4adc0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 shutdown_connections 2023-10-06T05:25:15.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.148+0000 7f4ae209e700 1 -- 172.21.15.45:0/2799887395 wait complete. 2023-10-06T05:25:15.151 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 274 2023-10-06T05:25:15.163 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2023-10-06T05:25:15.163 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2023-10-06T05:25:15.163 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:15.163 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:25:15.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:15.163 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:15.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:25:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:25:15.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.358+0000 7f72cbfb1700 1 Processor -- start 2023-10-06T05:25:15.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.358+0000 7f72cbfb1700 1 -- start start 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72cbfb1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c4153650 0x7f72c415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72cbfb1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c40676c0 con 0x7f72c4153a20 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72cbfb1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c40679a0 con 0x7f72c406afa0 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72cbfb1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4067c80 con 0x7f72c4153650 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72c954c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f72c4153a20 0x7f72c406a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39662/0 (socket says 172.21.15.45:39662) 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72c954c700 1 -- 172.21.15.45:0/2315479997 learned_addr learned my addr 172.21.15.45:0/2315479997 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72c9d4d700 1 --2- 172.21.15.45:0/2315479997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c4153650 0x7f72c415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72c8d4b700 1 -- 172.21.15.45:0/2315479997 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573373871 0 0) 0x7f72c40676c0 con 0x7f72c4153a20 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.359+0000 7f72c8d4b700 1 -- 172.21.15.45:0/2315479997 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b0003570 con 0x7f72c4153a20 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c8d4b700 1 -- 172.21.15.45:0/2315479997 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99706414 0 0) 0x7f72c40679a0 con 0x7f72c406afa0 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c8d4b700 1 -- 172.21.15.45:0/2315479997 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c40676c0 con 0x7f72c406afa0 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c9d4d700 1 -- 172.21.15.45:0/2315479997 >> v1:172.21.15.144:6789/0 conn(0x7f72c406afa0 legacy=0x7f72c4068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c9d4d700 1 -- 172.21.15.45:0/2315479997 >> v1:172.21.15.45:6789/0 conn(0x7f72c4153a20 legacy=0x7f72c406a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c9d4d700 1 -- 172.21.15.45:0/2315479997 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4067f60 con 0x7f72c4153650 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c9d4d700 1 --2- 172.21.15.45:0/2315479997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c4153650 0x7f72c415a930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009910 tx=0x7f72c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f49fc202768471d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c8d4b700 1 -- 172.21.15.45:0/2315479997 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0011630 con 0x7f72c4153650 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c8d4b700 1 -- 172.21.15.45:0/2315479997 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c0011790 con 0x7f72c4153650 2023-10-06T05:25:15.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.360+0000 7f72c8d4b700 1 -- 172.21.15.45:0/2315479997 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0011b30 con 0x7f72c4153650 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.361+0000 7f72cbfb1700 1 -- 172.21.15.45:0/2315479997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c4153650 msgr2=0x7f72c415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.361+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/2315479997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c4153650 0x7f72c415a930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009910 tx=0x7f72c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.361+0000 7f72cbfb1700 1 -- 172.21.15.45:0/2315479997 shutdown_connections 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.361+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/2315479997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c4153650 0x7f72c415a930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.361+0000 7f72cbfb1700 1 -- 172.21.15.45:0/2315479997 >> 172.21.15.45:0/2315479997 conn(0x7f72c40cc5a0 msgr2=0x7f72c40cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.361+0000 7f72cbfb1700 1 -- 172.21.15.45:0/2315479997 shutdown_connections 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.361+0000 7f72cbfb1700 1 -- 172.21.15.45:0/2315479997 wait complete. 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.362+0000 7f72cbfb1700 1 Processor -- start 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.362+0000 7f72cbfb1700 1 -- start start 2023-10-06T05:25:15.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72cbfb1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c406afa0 0x7f72c40c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72cbfb1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c4153650 0x7f72c40cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72cbfb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72c4153a20 0x7f72c41521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72cbfb1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c416cb60 con 0x7f72c4153a20 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72cbfb1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40c24c0 con 0x7f72c4153650 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72cbfb1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40c18b0 con 0x7f72c406afa0 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72c4153a20 0x7f72c41521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72c4153a20 0x7f72c41521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43584/0 (socket says 172.21.15.45:43584) 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 -- 172.21.15.45:0/1268873068 learned_addr learned my addr 172.21.15.45:0/1268873068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72c9d4d700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c406afa0 0x7f72c40c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72c954c700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c4153650 0x7f72c40cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 -- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c406afa0 msgr2=0x7f72c40c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c406afa0 0x7f72c40c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 -- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c4153650 msgr2=0x7f72c40cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c4153650 0x7f72c40cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 -- 172.21.15.45:0/1268873068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0009420 con 0x7f72c4153a20 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.363+0000 7f72ca54e700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72c4153a20 0x7f72c41521c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c940 tx=0x7f72bc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.364+0000 7f72b9ffb700 1 -- 172.21.15.45:0/1268873068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc012630 con 0x7f72c4153a20 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.364+0000 7f72b9ffb700 1 -- 172.21.15.45:0/1268873068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc0127b0 con 0x7f72c4153a20 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.364+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c40c1b90 con 0x7f72c4153a20 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.364+0000 7f72b9ffb700 1 -- 172.21.15.45:0/1268873068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc012b70 con 0x7f72c4153a20 2023-10-06T05:25:15.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.364+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c40c2150 con 0x7f72c4153a20 2023-10-06T05:25:15.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.365+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c40c2650 con 0x7f72c4153a20 2023-10-06T05:25:15.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.365+0000 7f72b9ffb700 1 -- 172.21.15.45:0/1268873068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72bc01b400 con 0x7f72c4153a20 2023-10-06T05:25:15.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.366+0000 7f72b9ffb700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72a8066f30 0x7f72a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.366+0000 7f72b9ffb700 1 -- 172.21.15.45:0/1268873068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7f72bc0854e0 con 0x7f72c4153a20 2023-10-06T05:25:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.366+0000 7f72c9d4d700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72a8066f30 0x7f72a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.367+0000 7f72c9d4d700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72a8066f30 0x7f72a80693e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009860 tx=0x7f72c0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:15.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.370+0000 7f72b9ffb700 1 -- 172.21.15.45:0/1268873068 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72bc051a60 con 0x7f72c4153a20 2023-10-06T05:25:15.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.618+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f72c40cc0e0 con 0x7f72c4153a20 2023-10-06T05:25:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.928+0000 7f72b9ffb700 1 -- 172.21.15.45:0/1268873068 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v275) v1 ==== 161+0+0 (secure 0 0 0) 0x7f72bc055080 con 0x7f72c4153a20 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72a8066f30 msgr2=0x7f72a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72a8066f30 0x7f72a80693e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009860 tx=0x7f72c0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72c4153a20 msgr2=0x7f72c41521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72c4153a20 0x7f72c41521c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c940 tx=0x7f72bc007f90 comp rx=0 tx=0).stop 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 shutdown_connections 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f72a8066f30 0x7f72a80693e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72c4153a20 0x7f72c41521c0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72c406afa0 0x7f72c40c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 --2- 172.21.15.45:0/1268873068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c4153650 0x7f72c40cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 >> 172.21.15.45:0/1268873068 conn(0x7f72c40cc5a0 msgr2=0x7f72c40ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 shutdown_connections 2023-10-06T05:25:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:15.931+0000 7f72cbfb1700 1 -- 172.21.15.45:0/1268873068 wait complete. 2023-10-06T05:25:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:15.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:25:16.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.082+0000 7fdb67488700 1 Processor -- start 2023-10-06T05:25:16.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.082+0000 7fdb67488700 1 -- start start 2023-10-06T05:25:16.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb67488700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb60156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb67488700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6015b000 con 0x7fdb600c8ff0 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb67488700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6015b2e0 con 0x7fdb60157030 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb67488700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c9830 con 0x7fdb600c93c0 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb65224700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdb600c8ff0 0x7fdb600caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39666/0 (socket says 172.21.15.45:39666) 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb65224700 1 -- 172.21.15.45:0/414554179 learned_addr learned my addr 172.21.15.45:0/414554179 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb64a23700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb60156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439620207 0 0) 0x7fdb6015b000 con 0x7fdb600c8ff0 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb4c003570 con 0x7fdb600c8ff0 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348602357 0 0) 0x7fdb6015b2e0 con 0x7fdb60157030 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb6015b000 con 0x7fdb60157030 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3631068547 0 0) 0x7fdb4c003570 con 0x7fdb600c8ff0 2023-10-06T05:25:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 msgr2=0x7fdb60156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb60156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 >> v1:172.21.15.144:6789/0 conn(0x7fdb60157030 legacy=0x7fdb600cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6015b5c0 con 0x7fdb600c8ff0 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fdb5c001470 con 0x7fdb600c8ff0 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.083+0000 7fdb64a23700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb60156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 >> v1:172.21.15.45:6789/0 conn(0x7fdb600c8ff0 legacy=0x7fdb600caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb4c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c004810 0x7fdb4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c9830 con 0x7fdb4c003c20 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c00bb10 con 0x7fdb4c004810 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c00bcd0 con 0x7fdb600c93c0 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65224700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb4c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb64a23700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c004810 0x7fdb4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65a25700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65224700 1 -- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 msgr2=0x7fdb4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65224700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65224700 1 -- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c004810 msgr2=0x7fdb4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65224700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c004810 0x7fdb4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65224700 1 -- 172.21.15.45:0/414554179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50009420 con 0x7fdb4c003c20 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65a25700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb600c93c0 0x7fdb4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb65224700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb4c004010 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c2d0 tx=0x7fdb5c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea770c3a31aef967 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c00cba0 con 0x7fdb4c003c20 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c00cd00 con 0x7fdb4c003c20 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.084+0000 7fdb57fff700 1 -- 172.21.15.45:0/414554179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c005b70 con 0x7fdb4c003c20 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.085+0000 7fdb67488700 1 -- 172.21.15.45:0/414554179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 msgr2=0x7fdb4c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.085+0000 7fdb67488700 1 --2- 172.21.15.45:0/414554179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb4c004010 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c2d0 tx=0x7fdb5c001e10 comp rx=0 tx=0).stop 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.085+0000 7fdb65a25700 1 -- 172.21.15.45:0/414554179 reap_dead start 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.085+0000 7fdb67488700 1 -- 172.21.15.45:0/414554179 shutdown_connections 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.085+0000 7fdb67488700 1 -- 172.21.15.45:0/414554179 >> 172.21.15.45:0/414554179 conn(0x7fdb600cdf50 msgr2=0x7fdb600ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.085+0000 7fdb67488700 1 -- 172.21.15.45:0/414554179 shutdown_connections 2023-10-06T05:25:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.085+0000 7fdb67488700 1 -- 172.21.15.45:0/414554179 wait complete. 2023-10-06T05:25:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.086+0000 7fdb67488700 1 Processor -- start 2023-10-06T05:25:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.086+0000 7fdb67488700 1 -- start start 2023-10-06T05:25:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb67488700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb600697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb67488700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb60158ae0 0x7fdb60157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb67488700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb60157970 0x7fdb60157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb67488700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6006a710 con 0x7fdb4c003c20 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb67488700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c6980 con 0x7fdb60158ae0 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb67488700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60150060 con 0x7fdb60157970 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb65224700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb600697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb65224700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb600697b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43598/0 (socket says 172.21.15.45:43598) 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb65224700 1 -- 172.21.15.45:0/2603965379 learned_addr learned my addr 172.21.15.45:0/2603965379 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb64a23700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb60158ae0 0x7fdb60157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.087+0000 7fdb65a25700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb60157970 0x7fdb60157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb65224700 1 -- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb60157970 msgr2=0x7fdb60157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb65224700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb60157970 0x7fdb60157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb65224700 1 -- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb60158ae0 msgr2=0x7fdb60157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb65224700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb60158ae0 0x7fdb60157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb65224700 1 -- 172.21.15.45:0/2603965379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6006a150 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb65224700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb600697b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c007c90 tx=0x7fdb5c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb557fa700 1 -- 172.21.15.45:0/2603965379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0070d0 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb557fa700 1 -- 172.21.15.45:0/2603965379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c007230 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6006a430 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb557fa700 1 -- 172.21.15.45:0/2603965379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0075f0 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.088+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb600b0cf0 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.089+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6006a670 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.090+0000 7fdb557fa700 1 -- 172.21.15.45:0/2603965379 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb5c019030 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.090+0000 7fdb557fa700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb44066f30 0x7fdb440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.090+0000 7fdb557fa700 1 -- 172.21.15.45:0/2603965379 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7fdb5c086b60 con 0x7fdb4c003c20 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.090+0000 7fdb64a23700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb44066f30 0x7fdb440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.091+0000 7fdb64a23700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb44066f30 0x7fdb440693e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdb600cc700 tx=0x7fdb500091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.095+0000 7fdb557fa700 1 -- 172.21.15.45:0/2603965379 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb5c053190 con 0x7fdb4c003c20 2023-10-06T05:25:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.384+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fdb6015a9a0 con 0x7fdb4c003c20 2023-10-06T05:25:16.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.933+0000 7fdb557fa700 1 -- 172.21.15.45:0/2603965379 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v276) v1 ==== 101+0+0 (secure 0 0 0) 0x7fdb5c0567b0 con 0x7fdb4c003c20 2023-10-06T05:25:16.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.936+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb44066f30 msgr2=0x7fdb440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.936+0000 7fdb67488700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb44066f30 0x7fdb440693e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdb600cc700 tx=0x7fdb500091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.936+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 msgr2=0x7fdb600697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.936+0000 7fdb67488700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb600697b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c007c90 tx=0x7fdb5c015000 comp rx=0 tx=0).stop 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 shutdown_connections 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdb44066f30 0x7fdb440693e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb60157970 0x7fdb60157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb4c003c20 0x7fdb600697b0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 --2- 172.21.15.45:0/2603965379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb60158ae0 0x7fdb60157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 >> 172.21.15.45:0/2603965379 conn(0x7fdb600cdf50 msgr2=0x7fdb60150490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 shutdown_connections 2023-10-06T05:25:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.937+0000 7fdb67488700 1 -- 172.21.15.45:0/2603965379 wait complete. 2023-10-06T05:25:16.940 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:25:16.954 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:25:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.999+0000 7fd898c50040 1 Processor -- start 2023-10-06T05:25:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:16.999+0000 7fd898c50040 1 -- start start 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd898c50040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58a82ca0 0x55cd58a84810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd898c50040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a9e7e0 0x55cd58b2aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd898c50040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58b29b30 0x55cd58b28c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd898c50040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd589a1980 con 0x55cd58b29b30 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd898c50040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd589a1620 con 0x55cd58a82ca0 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd898c50040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd58998400 con 0x55cd58a9e7e0 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58b29b30 0x55cd58b28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58b29b30 0x55cd58b28c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38312/0 (socket says 172.21.15.45:38312) 2023-10-06T05:25:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 -- 172.21.15.45:0/151080459 learned_addr learned my addr 172.21.15.45:0/151080459 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89166d700 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a9e7e0 0x55cd58b2aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd891e6e700 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58a82ca0 0x55cd58a84810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 -- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a9e7e0 msgr2=0x55cd58b2aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a9e7e0 0x55cd58b2aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 -- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58a82ca0 msgr2=0x55cd58a84810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58a82ca0 0x55cd58a84810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.000+0000 7fd89266f700 1 -- 172.21.15.45:0/151080459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c00a0e0 con 0x55cd58b29b30 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd89266f700 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58b29b30 0x55cd58b28c00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00f640 tx=0x7fd88c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85626cebec329495 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd883fff700 1 -- 172.21.15.45:0/151080459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c00fe50 con 0x55cd58b29b30 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd883fff700 1 -- 172.21.15.45:0/151080459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c00f860 con 0x55cd58b29b30 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd883fff700 1 -- 172.21.15.45:0/151080459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c024470 con 0x55cd58b29b30 2023-10-06T05:25:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 -- 172.21.15.45:0/151080459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58b29b30 msgr2=0x55cd58b28c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58b29b30 0x55cd58b28c00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00f640 tx=0x7fd88c015000 comp rx=0 tx=0).stop 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 -- 172.21.15.45:0/151080459 shutdown_connections 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58b29b30 0x55cd58b28c00 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58a82ca0 0x55cd58a84810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 --2- 172.21.15.45:0/151080459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a9e7e0 0x55cd58b2aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 -- 172.21.15.45:0/151080459 >> 172.21.15.45:0/151080459 conn(0x55cd58a57450 msgr2=0x55cd58a59830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 -- 172.21.15.45:0/151080459 shutdown_connections 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.001+0000 7fd898c50040 1 -- 172.21.15.45:0/151080459 wait complete. 2023-10-06T05:25:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.002+0000 7fd898c50040 1 Processor -- start 2023-10-06T05:25:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.002+0000 7fd898c50040 1 -- start start 2023-10-06T05:25:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd898c50040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a82ca0 0x55cd58b2fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd898c50040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58a9e7e0 0x55cd58b3ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd898c50040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58b29b30 0x55cd58b3d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd898c50040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd58b31500 con 0x55cd58a9e7e0 2023-10-06T05:25:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd898c50040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd58b30fc0 con 0x55cd58b29b30 2023-10-06T05:25:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd898c50040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd58b31260 con 0x55cd58a82ca0 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58a9e7e0 0x55cd58b3ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58a9e7e0 0x55cd58b3ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38322/0 (socket says 172.21.15.45:38322) 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 learned_addr learned my addr 172.21.15.45:0/2361417275 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd891e6e700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a82ca0 0x55cd58b2fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89266f700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58b29b30 0x55cd58b3d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a82ca0 msgr2=0x55cd58b2fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cd58a82ca0 0x55cd58b2fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58b29b30 msgr2=0x55cd58b3d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cd58b29b30 0x55cd58b3d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd589ca1c0 con 0x55cd58a9e7e0 2023-10-06T05:25:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.003+0000 7fd89166d700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58a9e7e0 0x55cd58b3ffa0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fd888008760 tx=0x7fd888005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.004+0000 7fd8827fc700 1 -- 172.21.15.45:0/2361417275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888006680 con 0x55cd58a9e7e0 2023-10-06T05:25:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.004+0000 7fd8827fc700 1 -- 172.21.15.45:0/2361417275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd888010040 con 0x55cd58a9e7e0 2023-10-06T05:25:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.004+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd589b03d0 con 0x55cd58a9e7e0 2023-10-06T05:25:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.004+0000 7fd8827fc700 1 -- 172.21.15.45:0/2361417275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888006a10 con 0x55cd58a9e7e0 2023-10-06T05:25:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.004+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd589a3ae0 con 0x55cd58a9e7e0 2023-10-06T05:25:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.005+0000 7fd8827fc700 1 -- 172.21.15.45:0/2361417275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd88801e460 con 0x55cd58a9e7e0 2023-10-06T05:25:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.006+0000 7fd8827fc700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd8740658c0 0x7fd874067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.006+0000 7fd8827fc700 1 -- 172.21.15.45:0/2361417275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7fd888084130 con 0x55cd58a9e7e0 2023-10-06T05:25:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.006+0000 7fd891e6e700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd8740658c0 0x7fd874067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.014 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:25:17.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.006+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cd58a84d60 0x55cd58b3d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.006+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b3de40 con 0x55cd58a84d60 2023-10-06T05:25:17.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.006+0000 7fd89266f700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cd58a84d60 0x55cd58b3d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.006+0000 7fd891e6e700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd8740658c0 0x7fd874067d70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd884000c00 tx=0x7fd884001900 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.007+0000 7fd89266f700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cd58a84d60 0x55cd58b3d900 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.009+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v122'908 uv882 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.009+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b3e220 con 0x55cd58a84d60 2023-10-06T05:25:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.010+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v122'909 uv909 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.010+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b498b0 con 0x55cd58a84d60 2023-10-06T05:25:17.017 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:25:17.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.011+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v122'910 uv910 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.011+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4ac70 con 0x55cd58a84d60 2023-10-06T05:25:17.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.013+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'911 uv911 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.013+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4b020 con 0x55cd58a84d60 2023-10-06T05:25:17.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.014+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'912 uv912 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.014+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b49c60 con 0x55cd58a84d60 2023-10-06T05:25:17.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.015+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'913 uv913 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.015+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a060 con 0x55cd58a84d60 2023-10-06T05:25:17.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.017+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v122'914 uv888 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.017+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a060 con 0x55cd58a84d60 2023-10-06T05:25:17.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.018+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v122'915 uv915 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.018+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.020+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v122'916 uv916 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.022 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:25:17.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.020+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.021+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'917 uv917 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.021+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.023+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'918 uv918 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.023+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.024+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'919 uv919 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.024+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.026+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v122'920 uv895 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.026+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.027+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'921 uv921 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.027+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.030 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:25:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.028+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v122'922 uv922 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.028+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.029+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'923 uv923 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.029+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.031+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'924 uv924 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.031+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.032+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'925 uv925 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.032+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.033+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'926 uv926 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.033+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.035+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v122'927 uv901 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.035+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.036+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v122'928 uv928 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.036+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.037+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v122'929 uv929 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.039 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:25:17.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.037+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.039+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'930 uv930 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.039+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.040+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'931 uv931 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.040+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.041+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'932 uv932 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.041+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.043+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v122'933 uv907 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.043+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.044+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v122'934 uv934 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.044+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.045+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v122'935 uv935 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.047 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:25:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.045+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.047+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'936 uv936 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.047+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.048+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'937 uv937 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.048+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58a84d60 2023-10-06T05:25:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.049+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'938 uv938 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c020030 con 0x55cd58a84d60 2023-10-06T05:25:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.049+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cd58b4b3d0 0x55cd58b4d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.049+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4a010 con 0x55cd58b4b3d0 2023-10-06T05:25:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.049+0000 7fd891e6e700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cd58b4b3d0 0x55cd58b4d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.050+0000 7fd891e6e700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cd58b4b3d0 0x55cd58b4d7b0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.051+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'653 uv636 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.052+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4dcf0 con 0x55cd58b4b3d0 2023-10-06T05:25:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.053+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'654 uv654 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.053+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e0a0 con 0x55cd58b4b3d0 2023-10-06T05:25:17.056 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:25:17.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.054+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'655 uv655 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.054+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e940 con 0x55cd58b4b3d0 2023-10-06T05:25:17.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.056+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v122'656 uv656 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.056+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4ecf0 con 0x55cd58b4b3d0 2023-10-06T05:25:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.057+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'657 uv657 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.057+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4f0a0 con 0x55cd58b4b3d0 2023-10-06T05:25:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.058+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v122'658 uv641 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.059+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4f450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.060+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v122'659 uv659 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.060+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4f800 con 0x55cd58b4b3d0 2023-10-06T05:25:17.063 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:25:17.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.061+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v122'660 uv660 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.061+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4f800 con 0x55cd58b4b3d0 2023-10-06T05:25:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.063+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'661 uv661 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.063+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e4d0 con 0x55cd58b4b3d0 2023-10-06T05:25:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.064+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'662 uv662 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.064+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e4d0 con 0x55cd58b4b3d0 2023-10-06T05:25:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.065+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v122'663 uv646 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.065+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.067+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v122'664 uv664 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.067+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.069 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:25:17.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.068+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v122'665 uv665 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.068+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.069+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v122'666 uv666 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.069+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.071+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'667 uv667 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.071+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.072+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v122'668 uv652 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.072+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.073+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v122'669 uv669 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.073+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.076 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:25:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.075+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v122'670 uv670 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.075+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.076+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'671 uv671 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.076+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.077+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'672 uv672 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.077+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b4b3d0 2023-10-06T05:25:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.078+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'673 uv673 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.078+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cd58b50200 0x55cd58b525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.078+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b4e450 con 0x55cd58b50200 2023-10-06T05:25:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.079+0000 7fd89166d700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cd58b50200 0x55cd58b525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.079+0000 7fd89166d700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cd58b50200 0x55cd58b525e0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.080+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.080+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b52c50 con 0x55cd58b50200 2023-10-06T05:25:17.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.082+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v122'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.082+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53140 con 0x55cd58b50200 2023-10-06T05:25:17.084 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:25:17.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.083+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v122'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.083+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b54a40 con 0x55cd58b50200 2023-10-06T05:25:17.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.084+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.084+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b54df0 con 0x55cd58b50200 2023-10-06T05:25:17.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.085+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.085+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53a30 con 0x55cd58b50200 2023-10-06T05:25:17.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.086+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.086+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b50200 2023-10-06T05:25:17.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.088+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'125 uv124 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.088+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b50200 2023-10-06T05:25:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.089+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'126 uv126 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.089+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b50200 2023-10-06T05:25:17.092 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:25:17.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.090+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'127 uv127 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.090+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b50200 2023-10-06T05:25:17.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.091+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'128 uv128 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.091+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b4b3d0 2023-10-06T05:25:17.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.093+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.093+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b4b3d0 2023-10-06T05:25:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.094+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v122'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.094+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b4b3d0 2023-10-06T05:25:17.097 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:25:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.095+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v122'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.095+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b4b3d0 2023-10-06T05:25:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.096+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.096+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b4b3d0 2023-10-06T05:25:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.097+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.097+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b53e30 con 0x55cd58b4b3d0 2023-10-06T05:25:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.099+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b4b3d0 2023-10-06T05:25:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.099+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cd58b551a0 0x55cd58b57580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.099+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.099+0000 7fd89266f700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cd58b551a0 0x55cd58b57580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.099+0000 7fd89266f700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cd58b551a0 0x55cd58b57580 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.101+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v122'327 uv321 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.101+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.102+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v122'328 uv328 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.102+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.104 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:25:17.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.103+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v122'329 uv329 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.103+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b58ad0 con 0x55cd58b551a0 2023-10-06T05:25:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.104+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'330 uv330 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.104+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b58ad0 con 0x55cd58b551a0 2023-10-06T05:25:17.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.105+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'331 uv331 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.105+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.106+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'332 uv332 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.106+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.107+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v122'333 uv326 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.107+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.108+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v122'334 uv334 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.108+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.109+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v122'335 uv335 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.111 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:25:17.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.110+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.111+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'336 uv336 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.111+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b57ac0 con 0x55cd58b551a0 2023-10-06T05:25:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.112+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'337 uv337 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.112+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cd58b57fd0 0x55cd58b5a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.112+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aae0 con 0x55cd58b57fd0 2023-10-06T05:25:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.112+0000 7fd891e6e700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cd58b57fd0 0x55cd58b5a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.113+0000 7fd891e6e700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cd58b57fd0 0x55cd58b5a3b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.114+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v122'148 uv147 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b57fd0 2023-10-06T05:25:17.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.114+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b57fd0 2023-10-06T05:25:17.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.115+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v122'149 uv149 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b57fd0 2023-10-06T05:25:17.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.115+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b57fd0 2023-10-06T05:25:17.118 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:25:17.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.116+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v122'150 uv150 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b57fd0 2023-10-06T05:25:17.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.117+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b57fd0 2023-10-06T05:25:17.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.118+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'151 uv151 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b57fd0 2023-10-06T05:25:17.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.118+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b57fd0 2023-10-06T05:25:17.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.119+0000 7fd891e6e700 1 -- 172.21.15.45:0/2361417275 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'152 uv152 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd884010a70 con 0x55cd58b57fd0 2023-10-06T05:25:17.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.119+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b50200 2023-10-06T05:25:17.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.120+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.120+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b50200 2023-10-06T05:25:17.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.121+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v122'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.121+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b50200 2023-10-06T05:25:17.124 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:25:17.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.123+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v122'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.123+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5bed0 con 0x55cd58b50200 2023-10-06T05:25:17.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.124+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.124+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5bed0 con 0x55cd58b50200 2023-10-06T05:25:17.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.125+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.125+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5aec0 con 0x55cd58b50200 2023-10-06T05:25:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.126+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.126+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cd58b5b540 0x55cd58b5d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.126+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5df40 con 0x55cd58b5b540 2023-10-06T05:25:17.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.126+0000 7fd89166d700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cd58b5b540 0x55cd58b5d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.127+0000 7fd89166d700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cd58b5b540 0x55cd58b5d920 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.128+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v122'294 uv288 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.128+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.130+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v122'295 uv295 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.130+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.131+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v122'296 uv296 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.133 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:25:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.131+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.132+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'297 uv297 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.132+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.133+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'298 uv298 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.133+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.134+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v122'299 uv293 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.134+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.135+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v122'300 uv300 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.135+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.136+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v122'301 uv301 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.138 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:25:17.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.136+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e320 con 0x55cd58b5b540 2023-10-06T05:25:17.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.137+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'302 uv302 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.137+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b61090 con 0x55cd58b5b540 2023-10-06T05:25:17.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.139+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'303 uv303 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b5b540 2023-10-06T05:25:17.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.139+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.140+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v122'128 uv127 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.140+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.141+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'129 uv129 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.141+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.142+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v122'130 uv130 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.144 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.142+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.143+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'131 uv131 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.143+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cd58b5f100 0x55cd58b617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.143+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b5f100 2023-10-06T05:25:17.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.144+0000 7fd89266f700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cd58b5f100 0x55cd58b617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.144+0000 7fd89266f700 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cd58b5f100 0x55cd58b617e0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.145+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b5f100 2023-10-06T05:25:17.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.145+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b5f100 2023-10-06T05:25:17.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.146+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v122'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b5f100 2023-10-06T05:25:17.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.146+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b5f100 2023-10-06T05:25:17.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.147+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v122'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b5f100 2023-10-06T05:25:17.149 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:25:17.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.148+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b5f100 2023-10-06T05:25:17.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.149+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b5f100 2023-10-06T05:25:17.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.149+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b5f100 2023-10-06T05:25:17.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.150+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b5f100 2023-10-06T05:25:17.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.150+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b5f100 2023-10-06T05:25:17.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.151+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b5f100 2023-10-06T05:25:17.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.152+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b551a0 2023-10-06T05:25:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.153+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v122'151 uv150 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.153+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b551a0 2023-10-06T05:25:17.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.154+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v122'152 uv152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.154+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b551a0 2023-10-06T05:25:17.157 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:25:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.155+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v122'153 uv153 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.155+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b551a0 2023-10-06T05:25:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.156+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'154 uv154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.156+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b551a0 2023-10-06T05:25:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.157+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'155 uv155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fd88c02d040 con 0x55cd58b551a0 2023-10-06T05:25:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.157+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.159+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v122'363 uv355 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.159+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.160+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v122'364 uv364 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.160+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.161+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v122'365 uv365 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.163 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:25:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.161+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e810 con 0x55cd58b50200 2023-10-06T05:25:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.163+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v122'366 uv366 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.163+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e7e0 con 0x55cd58b50200 2023-10-06T05:25:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.164+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'367 uv367 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.164+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e7a0 con 0x55cd58b50200 2023-10-06T05:25:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.165+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v122'368 uv362 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.165+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e740 con 0x55cd58b50200 2023-10-06T05:25:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.166+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v122'369 uv369 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.166+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e6c0 con 0x55cd58b50200 2023-10-06T05:25:17.168 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:25:17.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.167+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v122'370 uv370 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.167+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e6c0 con 0x55cd58b50200 2023-10-06T05:25:17.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.168+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'371 uv371 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.168+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b5e6c0 con 0x55cd58b50200 2023-10-06T05:25:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.169+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'372 uv372 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.169+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b61d20 con 0x55cd58b50200 2023-10-06T05:25:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.170+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'373 uv373 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.170+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55cd58b61d20 con 0x55cd58b50200 2023-10-06T05:25:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd89166d700 1 -- 172.21.15.45:0/2361417275 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'374 uv374 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd88801a020 con 0x55cd58b50200 2023-10-06T05:25:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cd58b50200 msgr2=0x55cd58b525e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cd58b50200 0x55cd58b525e0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cd58a84d60 msgr2=0x55cd58b3d900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cd58a84d60 0x55cd58b3d900 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cd58b5b540 msgr2=0x55cd58b5d920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cd58b5b540 0x55cd58b5d920 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cd58b4b3d0 msgr2=0x55cd58b4d7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cd58b4b3d0 0x55cd58b4d7b0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cd58b5f100 msgr2=0x55cd58b617e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cd58b5f100 0x55cd58b617e0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cd58b57fd0 msgr2=0x55cd58b5a3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cd58b57fd0 0x55cd58b5a3b0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cd58b551a0 msgr2=0x55cd58b57580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cd58b551a0 0x55cd58b57580 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 reap_dead start 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 reap_dead start 2023-10-06T05:25:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 reap_dead start 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 reap_dead start 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd89266f700 1 -- 172.21.15.45:0/2361417275 reap_dead start 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd8740658c0 msgr2=0x7fd874067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd8740658c0 0x7fd874067d70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd884000c00 tx=0x7fd884001900 comp rx=0 tx=0).stop 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58a9e7e0 msgr2=0x55cd58b3ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58a9e7e0 0x55cd58b3ffa0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fd888008760 tx=0x7fd888005c70 comp rx=0 tx=0).stop 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 shutdown_connections 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd58a9e7e0 0x55cd58b3ffa0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 --2- 172.21.15.45:0/2361417275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd8740658c0 0x7fd874067d70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 >> 172.21.15.45:0/2361417275 conn(0x55cd58a57450 msgr2=0x55cd58a59830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 shutdown_connections 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.172+0000 7fd898c50040 1 -- 172.21.15.45:0/2361417275 wait complete. 2023-10-06T05:25:17.177 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2023-10-06T05:25:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.225+0000 7ff3e5b68040 1 Processor -- start 2023-10-06T05:25:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.225+0000 7ff3e5b68040 1 -- start start 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3e5b68040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1f9f51210 0x55b1f9f98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3e5b68040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013890 0x55b1f9f99cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3e5b68040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1fa013c60 0x55b1fa036e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3e5b68040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1f9e9e980 con 0x55b1fa013890 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3e5b68040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1f9e9e620 con 0x55b1fa013c60 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3e5b68040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1f9e95400 con 0x55b1f9f51210 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3de585700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013890 0x55b1f9f99cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3ded86700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1f9f51210 0x55b1f9f98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3de585700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013890 0x55b1f9f99cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38338/0 (socket says 172.21.15.45:38338) 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3de585700 1 -- 172.21.15.45:0/873370582 learned_addr learned my addr 172.21.15.45:0/873370582 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.226+0000 7ff3df587700 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1fa013c60 0x55b1fa036e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3ded86700 1 -- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1fa013c60 msgr2=0x55b1fa036e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3ded86700 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1fa013c60 0x55b1fa036e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3ded86700 1 -- 172.21.15.45:0/873370582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013890 msgr2=0x55b1f9f99cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3ded86700 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013890 0x55b1f9f99cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3ded86700 1 -- 172.21.15.45:0/873370582 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c8005cf0 con 0x55b1f9f51210 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3ded86700 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1f9f51210 0x55b1f9f98140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009a80 tx=0x7ff3c800c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b6780db22f4aed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3dd583700 1 -- 172.21.15.45:0/873370582 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800f040 con 0x55b1f9f51210 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3dd583700 1 -- 172.21.15.45:0/873370582 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c8008b50 con 0x55b1f9f51210 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.227+0000 7ff3dd583700 1 -- 172.21.15.45:0/873370582 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8014480 con 0x55b1f9f51210 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 -- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1f9f51210 msgr2=0x55b1f9f98140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1f9f51210 0x55b1f9f98140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009a80 tx=0x7ff3c800c040 comp rx=0 tx=0).stop 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 -- 172.21.15.45:0/873370582 shutdown_connections 2023-10-06T05:25:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1fa013c60 0x55b1fa036e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1f9f51210 0x55b1f9f98140 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/873370582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013890 0x55b1f9f99cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 -- 172.21.15.45:0/873370582 >> 172.21.15.45:0/873370582 conn(0x55b1f9f54450 msgr2=0x55b1f9f56830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 -- 172.21.15.45:0/873370582 shutdown_connections 2023-10-06T05:25:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.228+0000 7ff3e5b68040 1 -- 172.21.15.45:0/873370582 wait complete. 2023-10-06T05:25:17.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.229+0000 7ff3e5b68040 1 Processor -- start 2023-10-06T05:25:17.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.229+0000 7ff3e5b68040 1 -- start start 2023-10-06T05:25:17.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3e5b68040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1f9f51210 0x55b1f9f826b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3e5b68040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1fa013890 0x55b1fa02c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3e5b68040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013c60 0x55b1fa00d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3e5b68040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1fa030e70 con 0x55b1fa013c60 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3e5b68040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1fa031110 con 0x55b1f9f51210 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3e5b68040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1fa0313b0 con 0x55b1fa013890 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013c60 0x55b1fa00d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013c60 0x55b1fa00d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38348/0 (socket says 172.21.15.45:38348) 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 -- 172.21.15.45:0/445228747 learned_addr learned my addr 172.21.15.45:0/445228747 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3ded86700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1f9f51210 0x55b1f9f826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3de585700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1fa013890 0x55b1fa02c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 -- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1fa013890 msgr2=0x55b1fa02c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b1fa013890 0x55b1fa02c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 -- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1f9f51210 msgr2=0x55b1f9f826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1f9f51210 0x55b1f9f826b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 -- 172.21.15.45:0/445228747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c8009650 con 0x55b1fa013c60 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.230+0000 7ff3df587700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013c60 0x55b1fa00d750 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8003f80 tx=0x7ff3d80068b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.231+0000 7ff3d77fe700 1 -- 172.21.15.45:0/445228747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d80020e0 con 0x55b1fa013c60 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.231+0000 7ff3d77fe700 1 -- 172.21.15.45:0/445228747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d8006dd0 con 0x55b1fa013c60 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.231+0000 7ff3d77fe700 1 -- 172.21.15.45:0/445228747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8012640 con 0x55b1fa013c60 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.231+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d0005cf0 con 0x55b1fa013c60 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.231+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1f9ead3d0 con 0x55b1fa013c60 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.232+0000 7ff3d77fe700 1 -- 172.21.15.45:0/445228747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3d80129b0 con 0x55b1fa013c60 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.233+0000 7ff3d77fe700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff3c00658c0 0x7ff3c0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.233+0000 7ff3d77fe700 1 -- 172.21.15.45:0/445228747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7ff3d8083cc0 con 0x55b1fa013c60 2023-10-06T05:25:17.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.233+0000 7ff3ded86700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff3c00658c0 0x7ff3c0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.233+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b1fa02cc10 0x55b1fa02fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.233+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x55b1f9f98cc0 con 0x55b1fa02cc10 2023-10-06T05:25:17.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.233+0000 7ff3de585700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b1fa02cc10 0x55b1fa02fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.233+0000 7ff3ded86700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff3c00658c0 0x7ff3c0067d70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009a20 tx=0x7ff3c8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.234+0000 7ff3de585700 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b1fa02cc10 0x55b1fa02fff0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3de585700 1 -- 172.21.15.45:0/445228747 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v122'375 uv375 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff3d0008970 con 0x55b1fa02cc10 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b1fa02cc10 msgr2=0x55b1fa02fff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b1fa02cc10 0x55b1fa02fff0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff3c00658c0 msgr2=0x7ff3c0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff3c00658c0 0x7ff3c0067d70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009a20 tx=0x7ff3c8017000 comp rx=0 tx=0).stop 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013c60 msgr2=0x55b1fa00d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 --2- 172.21.15.45:0/445228747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1fa013c60 0x55b1fa00d750 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8003f80 tx=0x7ff3d80068b0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3df587700 1 -- 172.21.15.45:0/445228747 reap_dead start 2023-10-06T05:25:17.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 shutdown_connections 2023-10-06T05:25:17.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 >> 172.21.15.45:0/445228747 conn(0x55b1f9f54450 msgr2=0x55b1f9f56830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 shutdown_connections 2023-10-06T05:25:17.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.235+0000 7ff3e5b68040 1 -- 172.21.15.45:0/445228747 wait complete. 2023-10-06T05:25:17.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:25:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.379+0000 7fa1f8502700 1 Processor -- start 2023-10-06T05:25:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.379+0000 7fa1f8502700 1 -- start start 2023-10-06T05:25:17.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.380+0000 7fa1f8502700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.380+0000 7fa1f8502700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00c3b50 con 0x7fa1f0157330 2023-10-06T05:25:17.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.380+0000 7fa1f8502700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f0157700 con 0x7fa1f00c8b70 2023-10-06T05:25:17.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.380+0000 7fa1f8502700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00c9700 con 0x7fa1f00ccb70 2023-10-06T05:25:17.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f629e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa1f0157330 0x7fa1f00ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53982/0 (socket says 172.21.15.45:53982) 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f629e700 1 -- 172.21.15.45:0/2631183404 learned_addr learned my addr 172.21.15.45:0/2631183404 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f5a9d700 1 --2- 172.21.15.45:0/2631183404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f529c700 1 -- 172.21.15.45:0/2631183404 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731737144 0 0) 0x7fa1f00c3b50 con 0x7fa1f0157330 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f529c700 1 -- 172.21.15.45:0/2631183404 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1dc003570 con 0x7fa1f0157330 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f529c700 1 -- 172.21.15.45:0/2631183404 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892325894 0 0) 0x7fa1f0157700 con 0x7fa1f00c8b70 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f5a9d700 1 -- 172.21.15.45:0/2631183404 >> v1:172.21.15.144:6789/0 conn(0x7fa1f00c8b70 legacy=0x7fa1f00c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f5a9d700 1 -- 172.21.15.45:0/2631183404 >> v1:172.21.15.45:6789/0 conn(0x7fa1f0157330 legacy=0x7fa1f00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f5a9d700 1 -- 172.21.15.45:0/2631183404 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00c98e0 con 0x7fa1f00ccb70 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f5a9d700 1 --2- 172.21.15.45:0/2631183404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00aaae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009910 tx=0x7fa1e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1fc3fdb079b50a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f529c700 1 -- 172.21.15.45:0/2631183404 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000b680 con 0x7fa1f00ccb70 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f529c700 1 -- 172.21.15.45:0/2631183404 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e000b7e0 con 0x7fa1f00ccb70 2023-10-06T05:25:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.381+0000 7fa1f529c700 1 -- 172.21.15.45:0/2631183404 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000bba0 con 0x7fa1f00ccb70 2023-10-06T05:25:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.382+0000 7fa1f8502700 1 -- 172.21.15.45:0/2631183404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 msgr2=0x7fa1f00aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.382+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2631183404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00aaae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009910 tx=0x7fa1e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.382+0000 7fa1f8502700 1 -- 172.21.15.45:0/2631183404 shutdown_connections 2023-10-06T05:25:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.382+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2631183404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00aaae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.382+0000 7fa1f8502700 1 -- 172.21.15.45:0/2631183404 >> 172.21.15.45:0/2631183404 conn(0x7fa1f0067e00 msgr2=0x7fa1f00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.382+0000 7fa1f8502700 1 -- 172.21.15.45:0/2631183404 shutdown_connections 2023-10-06T05:25:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.382+0000 7fa1f8502700 1 -- 172.21.15.45:0/2631183404 wait complete. 2023-10-06T05:25:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.383+0000 7fa1f8502700 1 Processor -- start 2023-10-06T05:25:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.383+0000 7fa1f8502700 1 -- start start 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f8502700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f00c8b70 0x7fa1f00c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f8502700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f8502700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f0157330 0x7fa1f0151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f8502700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00c8130 con 0x7fa1f0157330 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f8502700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0069010 con 0x7fa1f00c8b70 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f8502700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01522f0 con 0x7fa1f00ccb70 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f6a9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f0157330 0x7fa1f0151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f6a9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f0157330 0x7fa1f0151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38354/0 (socket says 172.21.15.45:38354) 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f6a9f700 1 -- 172.21.15.45:0/2528857543 learned_addr learned my addr 172.21.15.45:0/2528857543 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f5a9d700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.384+0000 7fa1f629e700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f00c8b70 0x7fa1f00c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f6a9f700 1 -- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 msgr2=0x7fa1f00c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f6a9f700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f6a9f700 1 -- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f00c8b70 msgr2=0x7fa1f00c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f6a9f700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f00c8b70 0x7fa1f00c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f6a9f700 1 -- 172.21.15.45:0/2528857543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e0009420 con 0x7fa1f0157330 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f629e700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f00c8b70 0x7fa1f00c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f6a9f700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f0157330 0x7fa1f0151db0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c890 tx=0x7fa1e8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1e67fc700 1 -- 172.21.15.45:0/2528857543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8012630 con 0x7fa1f0157330 2023-10-06T05:25:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1e67fc700 1 -- 172.21.15.45:0/2528857543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8012790 con 0x7fa1f0157330 2023-10-06T05:25:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f00c60e0 con 0x7fa1f0157330 2023-10-06T05:25:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1e67fc700 1 -- 172.21.15.45:0/2528857543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8012b50 con 0x7fa1f0157330 2023-10-06T05:25:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.385+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f0068ca0 con 0x7fa1f0157330 2023-10-06T05:25:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.386+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f00691a0 con 0x7fa1f0157330 2023-10-06T05:25:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.386+0000 7fa1e67fc700 1 -- 172.21.15.45:0/2528857543 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1e801a3a0 con 0x7fa1f0157330 2023-10-06T05:25:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.387+0000 7fa1e67fc700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa1d4066f30 0x7fa1d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.387+0000 7fa1e67fc700 1 -- 172.21.15.45:0/2528857543 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8958+0+0 (secure 0 0 0) 0x7fa1e8085380 con 0x7fa1f0157330 2023-10-06T05:25:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.387+0000 7fa1f629e700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa1d4066f30 0x7fa1d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.388+0000 7fa1f629e700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa1d4066f30 0x7fa1d40693e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.392+0000 7fa1e67fc700 1 -- 172.21.15.45:0/2528857543 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1e8051900 con 0x7fa1f0157330 2023-10-06T05:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.641+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa1f006e350 con 0x7fa1f0157330 2023-10-06T05:25:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.935+0000 7fa1e67fc700 1 -- 172.21.15.45:0/2528857543 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v277) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa1e8054f20 con 0x7fa1f0157330 2023-10-06T05:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa1d4066f30 msgr2=0x7fa1d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa1d4066f30 0x7fa1d40693e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2023-10-06T05:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f0157330 msgr2=0x7fa1f0151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f0157330 0x7fa1f0151db0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c890 tx=0x7fa1e8001670 comp rx=0 tx=0).stop 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 shutdown_connections 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa1d4066f30 0x7fa1d40693e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f0157330 0x7fa1f0151db0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f00c8b70 0x7fa1f00c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 --2- 172.21.15.45:0/2528857543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1f00ccb70 0x7fa1f00c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.938+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 >> 172.21.15.45:0/2528857543 conn(0x7fa1f0067e00 msgr2=0x7fa1f00c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.939+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 shutdown_connections 2023-10-06T05:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:17.939+0000 7fa1f8502700 1 -- 172.21.15.45:0/2528857543 wait complete. 2023-10-06T05:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:25:17.948 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:17.946+0000 7f268ee3f700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-10-06T05:25:17.948 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:17.946+0000 7f268ee3f700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:25:17.949 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:17.947+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:17.949 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:17.947+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:25:17.957 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:17.963 INFO:tasks.ceph.mds.b.smithi144.stderr: -24> 2023-10-06T05:25:17.946+0000 7f268ee3f700 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2023-10-06T05:25:17.963 INFO:tasks.ceph.mds.b.smithi144.stderr: -14> 2023-10-06T05:25:17.946+0000 7f268ee3f700 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2023-10-06T05:25:17.984 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:25:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.091+0000 7f18b4e99700 1 Processor -- start 2023-10-06T05:25:18.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.091+0000 7f18b4e99700 1 -- start start 2023-10-06T05:25:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18b4e99700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18b0159200 0x7f18b00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18b4e99700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18b01595d0 0x7f18b006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18b4e99700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00c7d00 con 0x7f18b006b5c0 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18b4e99700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00c7780 con 0x7f18b0159200 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18b4e99700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00c86d0 con 0x7f18b01595d0 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18ae59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18b0159200 0x7f18b00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18aed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f18b006b5c0 0x7f18b00ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53994/0 (socket says 172.21.15.45:53994) 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18aed9d700 1 -- 172.21.15.45:0/1818543639 learned_addr learned my addr 172.21.15.45:0/1818543639 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.092+0000 7f18add9b700 1 --2- 172.21.15.45:0/1818543639 >> v2:172.21.15.144:3300/0 conn(0x7f18b01595d0 0x7f18b006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ae59c700 1 -- 172.21.15.45:0/1818543639 >> v2:172.21.15.144:3300/0 conn(0x7f18b01595d0 msgr2=0x7f18b006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ae59c700 1 --2- 172.21.15.45:0/1818543639 >> v2:172.21.15.144:3300/0 conn(0x7f18b01595d0 0x7f18b006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ae59c700 1 -- 172.21.15.45:0/1818543639 >> v1:172.21.15.45:6789/0 conn(0x7f18b006b5c0 legacy=0x7f18b00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ae59c700 1 -- 172.21.15.45:0/1818543639 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b006bb40 con 0x7f18b0159200 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ad59a700 1 -- 172.21.15.45:0/1818543639 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851833094 0 0) 0x7f18b00c7d00 con 0x7f18b006b5c0 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ae59c700 1 --2- 172.21.15.45:0/1818543639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0159200 0x7f18b00c8190 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f18b006e790 tx=0x7f18a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c67835b2e37a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ad59a700 1 -- 172.21.15.45:0/1818543639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0011630 con 0x7f18b0159200 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ad59a700 1 -- 172.21.15.45:0/1818543639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a0011790 con 0x7f18b0159200 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.093+0000 7f18ad59a700 1 -- 172.21.15.45:0/1818543639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0011b30 con 0x7f18b0159200 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 -- 172.21.15.45:0/1818543639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0159200 msgr2=0x7f18b00c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1818543639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0159200 0x7f18b00c8190 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f18b006e790 tx=0x7f18a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 -- 172.21.15.45:0/1818543639 shutdown_connections 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1818543639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0159200 0x7f18b00c8190 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1818543639 >> v2:172.21.15.144:3300/0 conn(0x7f18b01595d0 0x7f18b006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 -- 172.21.15.45:0/1818543639 >> 172.21.15.45:0/1818543639 conn(0x7f18b006f090 msgr2=0x7f18b006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 -- 172.21.15.45:0/1818543639 shutdown_connections 2023-10-06T05:25:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.094+0000 7f18b4e99700 1 -- 172.21.15.45:0/1818543639 wait complete. 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.095+0000 7f18b4e99700 1 Processor -- start 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.095+0000 7f18b4e99700 1 -- start start 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18b4e99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b006b5c0 0x7f18b016d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18b4e99700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b0159200 0x7f18b00c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18b4e99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b01595d0 0x7f18b014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18b4e99700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00c5860 con 0x7f18b01595d0 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18b4e99700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0158450 con 0x7f18b006b5c0 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18b4e99700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b016dc20 con 0x7f18b0159200 2023-10-06T05:25:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b01595d0 0x7f18b014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b01595d0 0x7f18b014da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38378/0 (socket says 172.21.15.45:38378) 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 -- 172.21.15.45:0/1586327415 learned_addr learned my addr 172.21.15.45:0/1586327415 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18ae59c700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b006b5c0 0x7f18b016d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18add9b700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b0159200 0x7f18b00c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 -- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b0159200 msgr2=0x7f18b00c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b0159200 0x7f18b00c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 -- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b006b5c0 msgr2=0x7f18b016d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b006b5c0 0x7f18b016d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 -- 172.21.15.45:0/1586327415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a0009420 con 0x7f18b01595d0 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.096+0000 7f18aed9d700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b01595d0 0x7f18b014da00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f18a800ccc0 tx=0x7f18a8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.097+0000 7f18a67fc700 1 -- 172.21.15.45:0/1586327415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800a8a0 con 0x7f18b01595d0 2023-10-06T05:25:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.097+0000 7f18a67fc700 1 -- 172.21.15.45:0/1586327415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a800aa00 con 0x7f18b01595d0 2023-10-06T05:25:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.097+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b0169ce0 con 0x7f18b01595d0 2023-10-06T05:25:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.097+0000 7f18a67fc700 1 -- 172.21.15.45:0/1586327415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800adc0 con 0x7f18b01595d0 2023-10-06T05:25:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.097+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b016a2a0 con 0x7f18b01595d0 2023-10-06T05:25:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.098+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b00c59d0 con 0x7f18b01595d0 2023-10-06T05:25:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.098+0000 7f18a67fc700 1 -- 172.21.15.45:0/1586327415 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18a8007300 con 0x7f18b01595d0 2023-10-06T05:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.099+0000 7f18a67fc700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1890066f30 0x7f18900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.099+0000 7f18a67fc700 1 -- 172.21.15.45:0/1586327415 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f18a8085560 con 0x7f18b01595d0 2023-10-06T05:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.099+0000 7f18ae59c700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1890066f30 0x7f18900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.099+0000 7f18ae59c700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1890066f30 0x7f18900693e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f18a00093f0 tx=0x7f18a0017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.103+0000 7f18a67fc700 1 -- 172.21.15.45:0/1586327415 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18b00c59d0 con 0x7f18b01595d0 2023-10-06T05:25:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.357+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18b00c59a0 con 0x7f18b01595d0 2023-10-06T05:25:18.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.358+0000 7f18a67fc700 1 -- 172.21.15.45:0/1586327415 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) v1 ==== 95+0+3839 (secure 0 0 0) 0x7f18a8051b00 con 0x7f18b01595d0 2023-10-06T05:25:18.360 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:18.360 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":279,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:17.948633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1890066f30 msgr2=0x7f18900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1890066f30 0x7f18900693e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f18a00093f0 tx=0x7f18a0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b01595d0 msgr2=0x7f18b014da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b01595d0 0x7f18b014da00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f18a800ccc0 tx=0x7f18a8001a80 comp rx=0 tx=0).stop 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 shutdown_connections 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1890066f30 0x7f18900693e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b01595d0 0x7f18b014da00 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b006b5c0 0x7f18b016d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 --2- 172.21.15.45:0/1586327415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b0159200 0x7f18b00c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 >> 172.21.15.45:0/1586327415 conn(0x7f18b006f090 msgr2=0x7f18b00aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 shutdown_connections 2023-10-06T05:25:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.360+0000 7f18b4e99700 1 -- 172.21.15.45:0/1586327415 wait complete. 2023-10-06T05:25:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 279 2023-10-06T05:25:18.375 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:18.376 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 Processor -- start 2023-10-06T05:25:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 -- start start 2023-10-06T05:25:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974159170 0x7fb974159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb97415d040 0x7fb974068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb974069b90 0x7fb974069100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9740ca6d0 con 0x7fb974069b90 2023-10-06T05:25:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97415d550 con 0x7fb97415d040 2023-10-06T05:25:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.506+0000 7fb9797e5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97415d810 con 0x7fb974159170 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb974069b90 0x7fb974069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb974069b90 0x7fb974069100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38386/0 (socket says 172.21.15.45:38386) 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 -- 172.21.15.45:0/3816906236 learned_addr learned my addr 172.21.15.45:0/3816906236 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9727fc700 1 --2- 172.21.15.45:0/3816906236 >> v2:172.21.15.144:3300/0 conn(0x7fb97415d040 0x7fb974068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb972ffd700 1 --2- 172.21.15.45:0/3816906236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974159170 0x7fb974159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 -- 172.21.15.45:0/3816906236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974159170 msgr2=0x7fb974159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 --2- 172.21.15.45:0/3816906236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974159170 0x7fb974159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 -- 172.21.15.45:0/3816906236 >> v2:172.21.15.144:3300/0 conn(0x7fb97415d040 msgr2=0x7fb974068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 --2- 172.21.15.45:0/3816906236 >> v2:172.21.15.144:3300/0 conn(0x7fb97415d040 0x7fb974068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 -- 172.21.15.45:0/3816906236 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97415daf0 con 0x7fb974069b90 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb9737fe700 1 --2- 172.21.15.45:0/3816906236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974069b90 0x7fb974069100 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb97406a8b0 tx=0x7fb96c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d13c06f832f0e35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb971ffb700 1 -- 172.21.15.45:0/3816906236 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00e070 con 0x7fb974069b90 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb971ffb700 1 -- 172.21.15.45:0/3816906236 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c00ba50 con 0x7fb974069b90 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.507+0000 7fb971ffb700 1 -- 172.21.15.45:0/3816906236 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0092e0 con 0x7fb974069b90 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 -- 172.21.15.45:0/3816906236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974069b90 msgr2=0x7fb974069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 --2- 172.21.15.45:0/3816906236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974069b90 0x7fb974069100 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb97406a8b0 tx=0x7fb96c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 -- 172.21.15.45:0/3816906236 shutdown_connections 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 --2- 172.21.15.45:0/3816906236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974069b90 0x7fb974069100 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 --2- 172.21.15.45:0/3816906236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974159170 0x7fb974159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 --2- 172.21.15.45:0/3816906236 >> v2:172.21.15.144:3300/0 conn(0x7fb97415d040 0x7fb974068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 -- 172.21.15.45:0/3816906236 >> 172.21.15.45:0/3816906236 conn(0x7fb97406f7c0 msgr2=0x7fb9740d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 -- 172.21.15.45:0/3816906236 shutdown_connections 2023-10-06T05:25:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.508+0000 7fb9797e5700 1 -- 172.21.15.45:0/3816906236 wait complete. 2023-10-06T05:25:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.509+0000 7fb9797e5700 1 Processor -- start 2023-10-06T05:25:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.509+0000 7fb9797e5700 1 -- start start 2023-10-06T05:25:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9797e5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974069b90 0x7fb9740c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9797e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974159170 0x7fb9740c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9797e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb97415d040 0x7fb9740c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9797e5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97415a820 con 0x7fb974159170 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9797e5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740c7e30 con 0x7fb97415d040 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9797e5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740c8110 con 0x7fb974069b90 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974159170 0x7fb9740c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974159170 0x7fb9740c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38400/0 (socket says 172.21.15.45:38400) 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9727fc700 1 -- 172.21.15.45:0/4239552703 learned_addr learned my addr 172.21.15.45:0/4239552703 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb9737fe700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb97415d040 0x7fb9740c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.510+0000 7fb972ffd700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974069b90 0x7fb9740c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9727fc700 1 -- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974069b90 msgr2=0x7fb9740c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9727fc700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974069b90 0x7fb9740c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9727fc700 1 -- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb97415d040 msgr2=0x7fb9740c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9727fc700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb97415d040 0x7fb9740c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9727fc700 1 -- 172.21.15.45:0/4239552703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c009ae0 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9727fc700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974159170 0x7fb9740c9410 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d860 tx=0x7fb968008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb95affd700 1 -- 172.21.15.45:0/4239552703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968009610 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb95affd700 1 -- 172.21.15.45:0/4239552703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb968009770 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9740c83f0 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb95affd700 1 -- 172.21.15.45:0/4239552703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968009b30 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.511+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9740c8a30 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.512+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb974088fd0 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.513+0000 7fb95affd700 1 -- 172.21.15.45:0/4239552703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb96801e460 con 0x7fb974159170 2023-10-06T05:25:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.513+0000 7fb95affd700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb960066f30 0x7fb9600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.513+0000 7fb95affd700 1 -- 172.21.15.45:0/4239552703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7fb9680858f0 con 0x7fb974159170 2023-10-06T05:25:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.513+0000 7fb972ffd700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb960066f30 0x7fb9600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.514+0000 7fb972ffd700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb960066f30 0x7fb9600693e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb964006fd0 tx=0x7fb964008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.517+0000 7fb95affd700 1 -- 172.21.15.45:0/4239552703 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb968051e10 con 0x7fb974159170 2023-10-06T05:25:18.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.769+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9740ca860 con 0x7fb974159170 2023-10-06T05:25:18.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.770+0000 7fb95affd700 1 -- 172.21.15.45:0/4239552703 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) v1 ==== 95+0+3839 (secure 0 0 0) 0x7fb968055430 con 0x7fb974159170 2023-10-06T05:25:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:18.772 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":279,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:17.948633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:18.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb960066f30 msgr2=0x7fb9600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb960066f30 0x7fb9600693e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb964006fd0 tx=0x7fb964008040 comp rx=0 tx=0).stop 2023-10-06T05:25:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974159170 msgr2=0x7fb9740c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974159170 0x7fb9740c9410 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d860 tx=0x7fb968008040 comp rx=0 tx=0).stop 2023-10-06T05:25:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 shutdown_connections 2023-10-06T05:25:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb960066f30 0x7fb9600693e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb97415d040 0x7fb9740c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb974069b90 0x7fb9740c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 --2- 172.21.15.45:0/4239552703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb974159170 0x7fb9740c9410 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 >> 172.21.15.45:0/4239552703 conn(0x7fb97406f7c0 msgr2=0x7fb9740cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 shutdown_connections 2023-10-06T05:25:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.772+0000 7fb9797e5700 1 -- 172.21.15.45:0/4239552703 wait complete. 2023-10-06T05:25:18.777 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 279 2023-10-06T05:25:18.787 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:18.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:18.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.920+0000 7f58c5c91700 1 Processor -- start 2023-10-06T05:25:18.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.920+0000 7f58c5c91700 1 -- start start 2023-10-06T05:25:18.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58c5c91700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f58c00c8ff0 0x7f58c00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58c5c91700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58c00caad0 0x7f58c00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58c5c91700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c015b000 con 0x7f58c0156af0 2023-10-06T05:25:18.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58c5c91700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c0157630 con 0x7f58c00caad0 2023-10-06T05:25:18.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58c5c91700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c015b2e0 con 0x7f58c00c8ff0 2023-10-06T05:25:18.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58beffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58c00caad0 0x7f58c00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58bffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f58c0156af0 0x7f58c0156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53998/0 (socket says 172.21.15.45:53998) 2023-10-06T05:25:18.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58bffff700 1 -- 172.21.15.45:0/2445618597 learned_addr learned my addr 172.21.15.45:0/2445618597 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:18.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58bf7fe700 1 --2- 172.21.15.45:0/2445618597 >> v2:172.21.15.144:3300/0 conn(0x7f58c00c8ff0 0x7f58c00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58beffd700 1 -- 172.21.15.45:0/2445618597 >> v2:172.21.15.144:3300/0 conn(0x7f58c00c8ff0 msgr2=0x7f58c00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:18.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58beffd700 1 --2- 172.21.15.45:0/2445618597 >> v2:172.21.15.144:3300/0 conn(0x7f58c00c8ff0 0x7f58c00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58beffd700 1 -- 172.21.15.45:0/2445618597 >> v1:172.21.15.45:6789/0 conn(0x7f58c0156af0 legacy=0x7f58c0156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58beffd700 1 -- 172.21.15.45:0/2445618597 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c015b5c0 con 0x7f58c00caad0 2023-10-06T05:25:18.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58be7fc700 1 -- 172.21.15.45:0/2445618597 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102177563 0 0) 0x7f58c015b000 con 0x7f58c0156af0 2023-10-06T05:25:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.922+0000 7f58beffd700 1 --2- 172.21.15.45:0/2445618597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00caad0 0x7f58c00caf40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f58c00cefc0 tx=0x7f58a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef704c8b2da508eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58be7fc700 1 -- 172.21.15.45:0/2445618597 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8011630 con 0x7f58c00caad0 2023-10-06T05:25:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58be7fc700 1 -- 172.21.15.45:0/2445618597 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a8011790 con 0x7f58c00caad0 2023-10-06T05:25:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58be7fc700 1 -- 172.21.15.45:0/2445618597 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8011b30 con 0x7f58c00caad0 2023-10-06T05:25:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 -- 172.21.15.45:0/2445618597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00caad0 msgr2=0x7f58c00caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 --2- 172.21.15.45:0/2445618597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00caad0 0x7f58c00caf40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f58c00cefc0 tx=0x7f58a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 -- 172.21.15.45:0/2445618597 shutdown_connections 2023-10-06T05:25:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 --2- 172.21.15.45:0/2445618597 >> v2:172.21.15.144:3300/0 conn(0x7f58c00c8ff0 0x7f58c00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 --2- 172.21.15.45:0/2445618597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00caad0 0x7f58c00caf40 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 -- 172.21.15.45:0/2445618597 >> 172.21.15.45:0/2445618597 conn(0x7f58c00cdf50 msgr2=0x7f58c00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 -- 172.21.15.45:0/2445618597 shutdown_connections 2023-10-06T05:25:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.923+0000 7f58c5c91700 1 -- 172.21.15.45:0/2445618597 wait complete. 2023-10-06T05:25:18.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 Processor -- start 2023-10-06T05:25:18.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 -- start start 2023-10-06T05:25:18.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00c8ff0 0x7f58c0144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c00caad0 0x7f58c0158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58c0156af0 0x7f58c016ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00c7940 con 0x7f58c00c8ff0 2023-10-06T05:25:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00b1f20 con 0x7f58c00caad0 2023-10-06T05:25:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.925+0000 7f58c5c91700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00c7380 con 0x7f58c0156af0 2023-10-06T05:25:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00c8ff0 0x7f58c0144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00c8ff0 0x7f58c0144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38410/0 (socket says 172.21.15.45:38410) 2023-10-06T05:25:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 -- 172.21.15.45:0/899190463 learned_addr learned my addr 172.21.15.45:0/899190463 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bffff700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58c0156af0 0x7f58c016ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58beffd700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c00caad0 0x7f58c0158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 -- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58c0156af0 msgr2=0x7f58c016ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58c0156af0 0x7f58c016ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 -- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c00caad0 msgr2=0x7f58c0158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c00caad0 0x7f58c0158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 -- 172.21.15.45:0/899190463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a8009420 con 0x7f58c00c8ff0 2023-10-06T05:25:18.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58bf7fe700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00c8ff0 0x7f58c0144820 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e960 tx=0x7f58b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58b77fe700 1 -- 172.21.15.45:0/899190463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0013680 con 0x7f58c00c8ff0 2023-10-06T05:25:18.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58b77fe700 1 -- 172.21.15.45:0/899190463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b00137e0 con 0x7f58c00c8ff0 2023-10-06T05:25:18.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58b77fe700 1 -- 172.21.15.45:0/899190463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0013b80 con 0x7f58c00c8ff0 2023-10-06T05:25:18.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c00c7660 con 0x7f58c00c8ff0 2023-10-06T05:25:18.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.926+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c0144d60 con 0x7f58c00c8ff0 2023-10-06T05:25:18.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.927+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c00b2200 con 0x7f58c00c8ff0 2023-10-06T05:25:18.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.928+0000 7f58b77fe700 1 -- 172.21.15.45:0/899190463 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f58b00123d0 con 0x7f58c00c8ff0 2023-10-06T05:25:18.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.928+0000 7f58b77fe700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58a0066f30 0x7f58a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.928+0000 7f58b77fe700 1 -- 172.21.15.45:0/899190463 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f58b00853f0 con 0x7f58c00c8ff0 2023-10-06T05:25:18.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.929+0000 7f58beffd700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58a0066f30 0x7f58a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.929+0000 7f58beffd700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58a0066f30 0x7f58a00693e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f58a80093f0 tx=0x7f58a8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:18.933+0000 7f58b77fe700 1 -- 172.21.15.45:0/899190463 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58b00518b0 con 0x7f58c00c8ff0 2023-10-06T05:25:19.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.183+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c004bb40 con 0x7f58c00c8ff0 2023-10-06T05:25:19.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.183+0000 7f58b77fe700 1 -- 172.21.15.45:0/899190463 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 280 v280) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f58b0054ed0 con 0x7f58c00c8ff0 2023-10-06T05:25:19.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:19.185 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":280,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3552713853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3552713853},{"type":"v1","addr":"172.21.15.144:6835","nonce":3552713853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":280}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:17.948633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58a0066f30 msgr2=0x7f58a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58a0066f30 0x7f58a00693e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f58a80093f0 tx=0x7f58a8017000 comp rx=0 tx=0).stop 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00c8ff0 msgr2=0x7f58c0144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00c8ff0 0x7f58c0144820 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e960 tx=0x7f58b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 shutdown_connections 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f58a0066f30 0x7f58a00693e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58c0156af0 0x7f58c016ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00c8ff0 0x7f58c0144820 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 --2- 172.21.15.45:0/899190463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c00caad0 0x7f58c0158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 >> 172.21.15.45:0/899190463 conn(0x7f58c00cdf50 msgr2=0x7f58c00cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:19.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 shutdown_connections 2023-10-06T05:25:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.186+0000 7f58c5c91700 1 -- 172.21.15.45:0/899190463 wait complete. 2023-10-06T05:25:19.189 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 280 2023-10-06T05:25:19.201 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2023-10-06T05:25:19.201 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2023-10-06T05:25:19.202 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:19.202 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:25:19.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:19.202 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:19.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:25:19.222 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:25:19.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.398+0000 7f388a9ad700 1 Processor -- start 2023-10-06T05:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.398+0000 7f388a9ad700 1 -- start start 2023-10-06T05:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.398+0000 7f388a9ad700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3884159200 0x7f38840c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.398+0000 7f388a9ad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38841595d0 0x7f388406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.398+0000 7f388a9ad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f388406b5c0 0x7f38840ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f388a9ad700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38840c7780 con 0x7f3884159200 2023-10-06T05:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f388a9ad700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38840c7d00 con 0x7f388406b5c0 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f388a9ad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38840c86d0 con 0x7f38841595d0 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3884159200 0x7f38840c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3884159200 0x7f38840c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38416/0 (socket says 172.21.15.45:38416) 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 -- 172.21.15.45:0/1915510800 learned_addr learned my addr 172.21.15.45:0/1915510800 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3888f4a700 1 --2- 172.21.15.45:0/1915510800 >> v2:172.21.15.144:3300/0 conn(0x7f388406b5c0 0x7f38840ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f38837fe700 1 --2- 172.21.15.45:0/1915510800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38841595d0 0x7f388406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 -- 172.21.15.45:0/1915510800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38841595d0 msgr2=0x7f388406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 --2- 172.21.15.45:0/1915510800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38841595d0 0x7f388406b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 -- 172.21.15.45:0/1915510800 >> v2:172.21.15.144:3300/0 conn(0x7f388406b5c0 msgr2=0x7f38840ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 --2- 172.21.15.45:0/1915510800 >> v2:172.21.15.144:3300/0 conn(0x7f388406b5c0 0x7f38840ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 -- 172.21.15.45:0/1915510800 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388406bb40 con 0x7f3884159200 2023-10-06T05:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3883fff700 1 --2- 172.21.15.45:0/1915510800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3884159200 0x7f38840c8190 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3884067b90 tx=0x7f3874009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6d5070e5c77f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3882ffd700 1 -- 172.21.15.45:0/1915510800 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400b860 con 0x7f3884159200 2023-10-06T05:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3882ffd700 1 -- 172.21.15.45:0/1915510800 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387400b9c0 con 0x7f3884159200 2023-10-06T05:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.399+0000 7f3882ffd700 1 -- 172.21.15.45:0/1915510800 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400bd60 con 0x7f3884159200 2023-10-06T05:25:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 -- 172.21.15.45:0/1915510800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3884159200 msgr2=0x7f38840c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 --2- 172.21.15.45:0/1915510800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3884159200 0x7f38840c8190 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3884067b90 tx=0x7f3874009250 comp rx=0 tx=0).stop 2023-10-06T05:25:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 -- 172.21.15.45:0/1915510800 shutdown_connections 2023-10-06T05:25:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 --2- 172.21.15.45:0/1915510800 >> v2:172.21.15.144:3300/0 conn(0x7f388406b5c0 0x7f38840ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 --2- 172.21.15.45:0/1915510800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3884159200 0x7f38840c8190 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 --2- 172.21.15.45:0/1915510800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38841595d0 0x7f388406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 -- 172.21.15.45:0/1915510800 >> 172.21.15.45:0/1915510800 conn(0x7f388406f090 msgr2=0x7f388406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 -- 172.21.15.45:0/1915510800 shutdown_connections 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.400+0000 7f388a9ad700 1 -- 172.21.15.45:0/1915510800 wait complete. 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.401+0000 7f388a9ad700 1 Processor -- start 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.402+0000 7f388a9ad700 1 -- start start 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.402+0000 7f388a9ad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f388406b5c0 0x7f388414e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.402+0000 7f388a9ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3884159200 0x7f38840b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.402+0000 7f388a9ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38841595d0 0x7f38840b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.402+0000 7f388a9ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840c3e40 con 0x7f38841595d0 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.402+0000 7f388a9ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840c46e0 con 0x7f3884159200 2023-10-06T05:25:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.402+0000 7f388a9ad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388414e960 con 0x7f388406b5c0 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38841595d0 0x7f38840b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38841595d0 0x7f38840b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38430/0 (socket says 172.21.15.45:38430) 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 -- 172.21.15.45:0/4011469434 learned_addr learned my addr 172.21.15.45:0/4011469434 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3883fff700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f388406b5c0 0x7f388414e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f38837fe700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3884159200 0x7f38840b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 -- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f388406b5c0 msgr2=0x7f388414e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f388406b5c0 0x7f388414e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 -- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3884159200 msgr2=0x7f38840b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3884159200 0x7f38840b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 -- 172.21.15.45:0/4011469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740094f0 con 0x7f38841595d0 2023-10-06T05:25:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f3888f4a700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38841595d0 0x7f38840b1c00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f387c009fd0 tx=0x7f387c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f386bfff700 1 -- 172.21.15.45:0/4011469434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c00ebf0 con 0x7f38841595d0 2023-10-06T05:25:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f386bfff700 1 -- 172.21.15.45:0/4011469434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c00ed50 con 0x7f38841595d0 2023-10-06T05:25:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f386bfff700 1 -- 172.21.15.45:0/4011469434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c014670 con 0x7f38841595d0 2023-10-06T05:25:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3884154db0 con 0x7f38841595d0 2023-10-06T05:25:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.403+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38840c4090 con 0x7f38841595d0 2023-10-06T05:25:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.404+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38840c4850 con 0x7f38841595d0 2023-10-06T05:25:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.405+0000 7f386bfff700 1 -- 172.21.15.45:0/4011469434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f387c0149e0 con 0x7f38841595d0 2023-10-06T05:25:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.405+0000 7f386bfff700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3870066f30 0x7f38700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.405+0000 7f386bfff700 1 -- 172.21.15.45:0/4011469434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f387c085920 con 0x7f38841595d0 2023-10-06T05:25:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.406+0000 7f3883fff700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3870066f30 0x7f38700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.406+0000 7f3883fff700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3870066f30 0x7f38700693e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3884067b90 tx=0x7f3874017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.410+0000 7f386bfff700 1 -- 172.21.15.45:0/4011469434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f387c051e40 con 0x7f38841595d0 2023-10-06T05:25:19.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.661+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f388406e380 con 0x7f38841595d0 2023-10-06T05:25:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.964+0000 7f386bfff700 1 -- 172.21.15.45:0/4011469434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v281) v1 ==== 161+0+0 (secure 0 0 0) 0x7f387c055460 con 0x7f38841595d0 2023-10-06T05:25:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3870066f30 msgr2=0x7f38700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3870066f30 0x7f38700693e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3884067b90 tx=0x7f3874017000 comp rx=0 tx=0).stop 2023-10-06T05:25:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38841595d0 msgr2=0x7f38840b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38841595d0 0x7f38840b1c00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f387c009fd0 tx=0x7f387c009900 comp rx=0 tx=0).stop 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 shutdown_connections 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3870066f30 0x7f38700693e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38841595d0 0x7f38840b1c00 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f388406b5c0 0x7f388414e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 --2- 172.21.15.45:0/4011469434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3884159200 0x7f38840b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 >> 172.21.15.45:0/4011469434 conn(0x7f388406f090 msgr2=0x7f38841599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 shutdown_connections 2023-10-06T05:25:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:19.967+0000 7f388a9ad700 1 -- 172.21.15.45:0/4011469434 wait complete. 2023-10-06T05:25:19.972 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:19.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:25:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.119+0000 7f58ac590700 1 Processor -- start 2023-10-06T05:25:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.119+0000 7f58ac590700 1 -- start start 2023-10-06T05:25:20.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58ac590700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f58a4159200 0x7f58a40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58ac590700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58ac590700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58a406b5c0 0x7f58a40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58ac590700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40c7780 con 0x7f58a406b5c0 2023-10-06T05:25:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58ac590700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40c7d00 con 0x7f58a4159200 2023-10-06T05:25:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58ac590700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40c86d0 con 0x7f58a41595d0 2023-10-06T05:25:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58aab2d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58a406b5c0 0x7f58a40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58aab2d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58a406b5c0 0x7f58a40ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38438/0 (socket says 172.21.15.45:38438) 2023-10-06T05:25:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58aab2d700 1 -- 172.21.15.45:0/1065246995 learned_addr learned my addr 172.21.15.45:0/1065246995 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58a9b2b700 1 --2- 172.21.15.45:0/1065246995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.120+0000 7f58aa32c700 1 --2- 172.21.15.45:0/1065246995 >> v2:172.21.15.144:3300/0 conn(0x7f58a4159200 0x7f58a40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58aab2d700 1 -- 172.21.15.45:0/1065246995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 msgr2=0x7f58a406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58aab2d700 1 --2- 172.21.15.45:0/1065246995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a406b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58aab2d700 1 -- 172.21.15.45:0/1065246995 >> v2:172.21.15.144:3300/0 conn(0x7f58a4159200 msgr2=0x7f58a40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58aab2d700 1 --2- 172.21.15.45:0/1065246995 >> v2:172.21.15.144:3300/0 conn(0x7f58a4159200 0x7f58a40c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58aab2d700 1 -- 172.21.15.45:0/1065246995 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a406bb40 con 0x7f58a406b5c0 2023-10-06T05:25:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58aab2d700 1 --2- 172.21.15.45:0/1065246995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a40ccdc0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f58a406ae50 tx=0x7f589c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c14cabaa1dac14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58a932a700 1 -- 172.21.15.45:0/1065246995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00e070 con 0x7f58a406b5c0 2023-10-06T05:25:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58a932a700 1 -- 172.21.15.45:0/1065246995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c00b970 con 0x7f58a406b5c0 2023-10-06T05:25:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.121+0000 7f58a932a700 1 -- 172.21.15.45:0/1065246995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0092e0 con 0x7f58a406b5c0 2023-10-06T05:25:20.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 -- 172.21.15.45:0/1065246995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 msgr2=0x7f58a40ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 --2- 172.21.15.45:0/1065246995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a40ccdc0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f58a406ae50 tx=0x7f589c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 -- 172.21.15.45:0/1065246995 shutdown_connections 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 --2- 172.21.15.45:0/1065246995 >> v2:172.21.15.144:3300/0 conn(0x7f58a4159200 0x7f58a40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 --2- 172.21.15.45:0/1065246995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a40ccdc0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 --2- 172.21.15.45:0/1065246995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 -- 172.21.15.45:0/1065246995 >> 172.21.15.45:0/1065246995 conn(0x7f58a406f090 msgr2=0x7f58a406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 -- 172.21.15.45:0/1065246995 shutdown_connections 2023-10-06T05:25:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.122+0000 7f58ac590700 1 -- 172.21.15.45:0/1065246995 wait complete. 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.123+0000 7f58ac590700 1 Processor -- start 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.123+0000 7f58ac590700 1 -- start start 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58ac590700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a4155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58ac590700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a4159200 0x7f58a4153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58ac590700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a40ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58ac590700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40b1a60 con 0x7f58a406b5c0 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58ac590700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40b1480 con 0x7f58a4159200 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58ac590700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40c5530 con 0x7f58a41595d0 2023-10-06T05:25:20.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a4155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a4155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38450/0 (socket says 172.21.15.45:38450) 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 -- 172.21.15.45:0/1208190211 learned_addr learned my addr 172.21.15.45:0/1208190211 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aab2d700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a40ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58a9b2b700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a4159200 0x7f58a4153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 -- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 msgr2=0x7f58a40ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a40ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 -- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a4159200 msgr2=0x7f58a4153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a4159200 0x7f58a4153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 -- 172.21.15.45:0/1208190211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c009ae0 con 0x7f58a406b5c0 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58a9b2b700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a4159200 0x7f58a4153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58aa32c700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a4155650 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f58a000ecb0 tx=0x7f58a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58967fc700 1 -- 172.21.15.45:0/1208190211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a000c9d0 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58967fc700 1 -- 172.21.15.45:0/1208190211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a000cb30 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.124+0000 7f58967fc700 1 -- 172.21.15.45:0/1208190211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a000ced0 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.125+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a40b16f0 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.125+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a40c5810 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.126+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a406e2f0 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.126+0000 7f58967fc700 1 -- 172.21.15.45:0/1208190211 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f58a0013680 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.127+0000 7f58967fc700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5888066f30 0x7f58880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.127+0000 7f58967fc700 1 -- 172.21.15.45:0/1208190211 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f58a00855e0 con 0x7f58a406b5c0 2023-10-06T05:25:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.127+0000 7f58a9b2b700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5888066f30 0x7f58880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.127+0000 7f58a9b2b700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5888066f30 0x7f58880693e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5898006f00 tx=0x7f5898008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.131+0000 7f58967fc700 1 -- 172.21.15.45:0/1208190211 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58a0051b00 con 0x7f58a406b5c0 2023-10-06T05:25:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.424+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f58a406be20 con 0x7f58a406b5c0 2023-10-06T05:25:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.981+0000 7f58967fc700 1 -- 172.21.15.45:0/1208190211 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v282) v1 ==== 101+0+0 (secure 0 0 0) 0x7f58a0055120 con 0x7f58a406b5c0 2023-10-06T05:25:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5888066f30 msgr2=0x7f58880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5888066f30 0x7f58880693e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5898006f00 tx=0x7f5898008040 comp rx=0 tx=0).stop 2023-10-06T05:25:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 msgr2=0x7f58a4155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a4155650 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f58a000ecb0 tx=0x7f58a00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 shutdown_connections 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5888066f30 0x7f58880693e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58a41595d0 0x7f58a40ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58a406b5c0 0x7f58a4155650 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 --2- 172.21.15.45:0/1208190211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58a4159200 0x7f58a4153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 >> 172.21.15.45:0/1208190211 conn(0x7f58a406f090 msgr2=0x7f58a40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 shutdown_connections 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:20.984+0000 7f58ac590700 1 -- 172.21.15.45:0/1208190211 wait complete. 2023-10-06T05:25:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:25:21.001 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:25:21.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 Processor -- start 2023-10-06T05:25:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 -- start start 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960876be0 0x560960924280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960921dc0 0x560960924e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960920630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x560960796980 con 0x560960921dc0 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560960796620 con 0x560960876be0 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.048+0000 7f28a0cf1040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096078d400 con 0x560960922190 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960921dc0 0x560960924e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960921dc0 0x560960924e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38458/0 (socket says 172.21.15.45:38458) 2023-10-06T05:25:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 -- 172.21.15.45:0/1908858377 learned_addr learned my addr 172.21.15.45:0/1908858377 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f2899f0f700 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960876be0 0x560960924280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289a710700 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960920630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 -- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 msgr2=0x560960920630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960920630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 -- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960876be0 msgr2=0x560960924280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960876be0 0x560960924280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 -- 172.21.15.45:0/1908858377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289400a180 con 0x560960921dc0 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f289970e700 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960921dc0 0x560960924e20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2890001d40 tx=0x7f2890001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fbc3e55e078183 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f288bfff700 1 -- 172.21.15.45:0/1908858377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890014070 con 0x560960921dc0 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f288bfff700 1 -- 172.21.15.45:0/1908858377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890008070 con 0x560960921dc0 2023-10-06T05:25:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.049+0000 7f288bfff700 1 -- 172.21.15.45:0/1908858377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890001ea0 con 0x560960921dc0 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 -- 172.21.15.45:0/1908858377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960921dc0 msgr2=0x560960924e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960921dc0 0x560960924e20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2890001d40 tx=0x7f2890001810 comp rx=0 tx=0).stop 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 -- 172.21.15.45:0/1908858377 shutdown_connections 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960920630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960876be0 0x560960924280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/1908858377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960921dc0 0x560960924e20 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 -- 172.21.15.45:0/1908858377 >> 172.21.15.45:0/1908858377 conn(0x56096084bbd0 msgr2=0x560960923b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 -- 172.21.15.45:0/1908858377 shutdown_connections 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.050+0000 7f28a0cf1040 1 -- 172.21.15.45:0/1908858377 wait complete. 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 Processor -- start 2023-10-06T05:25:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 -- start start 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960876be0 0x560960925970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960921dc0 0x5609608914d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960922860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609608923c0 con 0x560960876be0 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096084e280 con 0x560960921dc0 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.051+0000 7f28a0cf1040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x560960891be0 con 0x560960922190 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960876be0 0x560960925970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960876be0 0x560960925970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38460/0 (socket says 172.21.15.45:38460) 2023-10-06T05:25:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 learned_addr learned my addr 172.21.15.45:0/3004971062 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f289970e700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960921dc0 0x5609608914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960922860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 msgr2=0x560960922860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960922860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960921dc0 msgr2=0x5609608914d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560960921dc0 0x5609608914d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289000a040 con 0x560960876be0 2023-10-06T05:25:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560960922190 0x560960922860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f2899f0f700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960876be0 0x560960925970 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f288c008ac0 tx=0x7f288c00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f288a7fc700 1 -- 172.21.15.45:0/3004971062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c00f040 con 0x560960876be0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f288a7fc700 1 -- 172.21.15.45:0/3004971062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c0108b0 con 0x560960876be0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5609607a53d0 con 0x560960876be0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f288a7fc700 1 -- 172.21.15.45:0/3004971062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c010c30 con 0x560960876be0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.052+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560960798ae0 con 0x560960876be0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.054+0000 7f288a7fc700 1 -- 172.21.15.45:0/3004971062 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f288c010d90 con 0x560960876be0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.054+0000 7f288a7fc700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c0658c0 0x7f287c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.054+0000 7f288a7fc700 1 -- 172.21.15.45:0/3004971062 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f288c084d60 con 0x560960876be0 2023-10-06T05:25:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.054+0000 7f289970e700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c0658c0 0x7f287c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.061 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:25:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.055+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x560960922da0 0x56096091b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.055+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096092d820 con 0x560960922da0 2023-10-06T05:25:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.055+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x560960922da0 0x56096091b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.055+0000 7f289970e700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c0658c0 0x7f287c067d70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2890001870 tx=0x7f2890016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.055+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x560960922da0 0x56096091b5d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.057+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v123'939 uv913 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.057+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096092dc00 con 0x560960922da0 2023-10-06T05:25:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.059+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v123'940 uv940 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.059+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096092e070 con 0x560960922da0 2023-10-06T05:25:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.060+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v123'941 uv941 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.063 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:25:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.060+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093fbf0 con 0x560960922da0 2023-10-06T05:25:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.062+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'942 uv942 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.062+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093ffa0 con 0x560960922da0 2023-10-06T05:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.063+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'943 uv943 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.063+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093ebe0 con 0x560960922da0 2023-10-06T05:25:21.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.065+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'944 uv944 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.065+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093efe0 con 0x560960922da0 2023-10-06T05:25:21.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.066+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v123'945 uv919 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.066+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f3e0 con 0x560960922da0 2023-10-06T05:25:21.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.067+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v123'946 uv946 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.067+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f3e0 con 0x560960922da0 2023-10-06T05:25:21.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.069+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v123'947 uv947 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.070 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:25:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.069+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.070+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'948 uv948 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.070+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.071+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'949 uv949 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.071+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.072+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'950 uv950 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.072+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.074+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v123'951 uv926 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.074+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.075+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'952 uv952 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.075+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.077+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v123'953 uv953 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.079 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:25:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.077+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.078+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'954 uv954 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.078+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.080+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'955 uv955 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.080+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096093f390 con 0x560960922da0 2023-10-06T05:25:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.081+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'956 uv956 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.081+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.083+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'957 uv957 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.083+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.084+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v123'958 uv932 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.084+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.085+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v123'959 uv959 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.085+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.087+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v123'960 uv960 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.088 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:25:21.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.087+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960941360 con 0x560960922da0 2023-10-06T05:25:21.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.088+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'961 uv961 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.088+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960941360 con 0x560960922da0 2023-10-06T05:25:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.090+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'962 uv962 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.090+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.091+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'963 uv963 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.091+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.093+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v123'964 uv938 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.093+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.094+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v123'965 uv965 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.094+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.095+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v123'966 uv966 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.097 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:25:21.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.095+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960941360 con 0x560960922da0 2023-10-06T05:25:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.097+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'967 uv967 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.097+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960941360 con 0x560960922da0 2023-10-06T05:25:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.098+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'968 uv968 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.098+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960940350 con 0x560960922da0 2023-10-06T05:25:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.100+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'969 uv969 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960922da0 2023-10-06T05:25:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.100+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x560960940860 0x560960942c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.100+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960943180 con 0x560960940860 2023-10-06T05:25:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.100+0000 7f2899f0f700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x560960940860 0x560960942c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.100+0000 7f2899f0f700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x560960940860 0x560960942c40 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.102+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'674 uv657 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.102+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960943530 con 0x560960940860 2023-10-06T05:25:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.103+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'675 uv675 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.103+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609438e0 con 0x560960940860 2023-10-06T05:25:21.106 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:25:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.105+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'676 uv676 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.105+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960944180 con 0x560960940860 2023-10-06T05:25:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.106+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v123'677 uv677 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.106+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960944530 con 0x560960940860 2023-10-06T05:25:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.107+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'678 uv678 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.108+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609448e0 con 0x560960940860 2023-10-06T05:25:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.109+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v123'679 uv662 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.109+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960944c90 con 0x560960940860 2023-10-06T05:25:21.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.110+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v123'680 uv680 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.110+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:25:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.111+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v123'681 uv681 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.111+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.112+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'682 uv682 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.112+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.114+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'683 uv683 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.114+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.115+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v123'684 uv667 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.115+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.117+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v123'685 uv685 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.117+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.120 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:25:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.118+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v123'686 uv686 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.118+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960948760 con 0x560960940860 2023-10-06T05:25:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.120+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v123'687 uv687 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.120+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960948760 con 0x560960940860 2023-10-06T05:25:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.121+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'688 uv688 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.121+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.122+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v123'689 uv673 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.122+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.123+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v123'690 uv690 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.124+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.126 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:25:21.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.125+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v123'691 uv691 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.125+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960946050 con 0x560960940860 2023-10-06T05:25:21.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.126+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'692 uv692 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.126+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960946050 con 0x560960940860 2023-10-06T05:25:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.127+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'693 uv693 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.127+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960945040 con 0x560960940860 2023-10-06T05:25:21.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.128+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'694 uv694 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.128+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560960945550 0x560960947930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.129+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960947e70 con 0x560960945550 2023-10-06T05:25:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.129+0000 7f289970e700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560960945550 0x560960947930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.129+0000 7f289970e700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560960945550 0x560960947930 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.131+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.131+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960948220 con 0x560960945550 2023-10-06T05:25:21.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.132+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v123'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.132+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960948690 con 0x560960945550 2023-10-06T05:25:21.135 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:25:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.133+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v123'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.133+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960949f90 con 0x560960945550 2023-10-06T05:25:21.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.134+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.134+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094a340 con 0x560960945550 2023-10-06T05:25:21.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.135+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.135+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960948f80 con 0x560960945550 2023-10-06T05:25:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.137+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.137+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960945550 2023-10-06T05:25:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.138+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'129 uv128 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.138+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960945550 2023-10-06T05:25:21.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.140+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'130 uv130 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.140+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960945550 2023-10-06T05:25:21.143 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:25:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.141+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'131 uv131 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.141+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960945550 2023-10-06T05:25:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.142+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'132 uv132 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.142+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960940860 2023-10-06T05:25:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.143+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.143+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960940860 2023-10-06T05:25:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.145+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v123'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.145+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960940860 2023-10-06T05:25:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.146+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v123'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.149 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:25:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.146+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960940860 2023-10-06T05:25:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.147+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.147+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960940860 2023-10-06T05:25:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.148+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.148+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x5609609493a0 con 0x560960940860 2023-10-06T05:25:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.149+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x560960940860 2023-10-06T05:25:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.149+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56096094a6f0 0x56096094cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.150+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.150+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56096094a6f0 0x56096094cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.150+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56096094a6f0 0x56096094cad0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.151+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v123'338 uv332 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.151+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.153+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v123'339 uv339 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.153+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.155 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:25:21.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.154+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v123'340 uv340 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.154+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094e020 con 0x56096094a6f0 2023-10-06T05:25:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.155+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'341 uv341 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.155+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094e020 con 0x56096094a6f0 2023-10-06T05:25:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.156+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.156+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.157+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.157+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.158+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v123'344 uv337 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.158+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.160+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v123'345 uv345 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.160+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.162 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:25:21.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.161+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v123'346 uv346 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.161+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.162+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'347 uv347 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.162+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094d010 con 0x56096094a6f0 2023-10-06T05:25:21.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.163+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'348 uv348 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.163+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56096094d690 0x56096094fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.163+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094fff0 con 0x56096094d690 2023-10-06T05:25:21.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.163+0000 7f2899f0f700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56096094d690 0x56096094fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.164+0000 7f2899f0f700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56096094d690 0x56096094fa70 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.165+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v123'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x56096094d690 2023-10-06T05:25:21.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.165+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094fff0 con 0x56096094d690 2023-10-06T05:25:21.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.166+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v123'154 uv154 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x56096094d690 2023-10-06T05:25:21.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.166+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x56096094fff0 con 0x56096094d690 2023-10-06T05:25:21.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.167+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v123'155 uv155 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x56096094d690 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.168+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960950470 con 0x56096094d690 2023-10-06T05:25:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.169+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'156 uv156 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x56096094d690 2023-10-06T05:25:21.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.169+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960950470 con 0x56096094d690 2023-10-06T05:25:21.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.170+0000 7f2899f0f700 1 -- 172.21.15.45:0/3004971062 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'157 uv157 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f288c01b070 con 0x56096094d690 2023-10-06T05:25:21.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.170+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960950470 con 0x560960945550 2023-10-06T05:25:21.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.171+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.171+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960950470 con 0x560960945550 2023-10-06T05:25:21.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.173+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v123'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.173+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960950470 con 0x560960945550 2023-10-06T05:25:21.175 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:25:21.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.174+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v123'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.174+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960951480 con 0x560960945550 2023-10-06T05:25:21.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.175+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.175+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960951480 con 0x560960945550 2023-10-06T05:25:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.176+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.176+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960950470 con 0x560960945550 2023-10-06T05:25:21.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.178+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.178+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x560960950980 0x560960952d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.178+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953410 con 0x560960950980 2023-10-06T05:25:21.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.178+0000 7f289970e700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x560960950980 0x560960952d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.178+0000 7f289970e700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x560960950980 0x560960952d60 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.180+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v123'304 uv298 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.180+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.181+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v123'305 uv305 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.181+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.184 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:25:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.182+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v123'306 uv306 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.182+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.184+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'307 uv307 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.184+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.185+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'308 uv308 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.185+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.186+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v123'309 uv303 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.186+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.187+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v123'310 uv310 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.187+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.191 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:25:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.189+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v123'311 uv311 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.189+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953810 con 0x560960950980 2023-10-06T05:25:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.190+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'312 uv312 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.190+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960956580 con 0x560960950980 2023-10-06T05:25:21.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.191+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'313 uv313 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960950980 2023-10-06T05:25:21.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.191+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.192+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v123'132 uv131 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.192+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.193+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'133 uv133 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.193+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.197 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.195+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v123'134 uv134 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.195+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.196+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'135 uv135 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.196+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560960954610 0x560960956cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.196+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960954610 2023-10-06T05:25:21.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.196+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560960954610 0x560960956cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.197+0000 7f289a710700 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560960954610 0x560960956cd0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.198+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960954610 2023-10-06T05:25:21.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.198+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960954610 2023-10-06T05:25:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.199+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v123'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960954610 2023-10-06T05:25:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.199+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960954610 2023-10-06T05:25:21.202 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:25:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.200+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v123'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960954610 2023-10-06T05:25:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.200+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960954610 2023-10-06T05:25:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.201+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960954610 2023-10-06T05:25:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.201+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960954610 2023-10-06T05:25:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.203+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960954610 2023-10-06T05:25:21.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.203+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960954610 2023-10-06T05:25:21.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.204+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x560960954610 2023-10-06T05:25:21.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.204+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x56096094a6f0 2023-10-06T05:25:21.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.205+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v123'156 uv155 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.205+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x56096094a6f0 2023-10-06T05:25:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.206+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v123'157 uv157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.206+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x56096094a6f0 2023-10-06T05:25:21.209 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:25:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.207+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v123'158 uv158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.207+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x56096094a6f0 2023-10-06T05:25:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.209+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'159 uv159 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.209+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x56096094a6f0 2023-10-06T05:25:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.210+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'160 uv160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f289400e7e0 con 0x56096094a6f0 2023-10-06T05:25:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.210+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.212+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v123'376 uv375 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.212+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.213+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v123'377 uv377 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.213+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.215 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:25:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.214+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v123'378 uv378 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.214+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.215+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v123'379 uv379 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.215+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960957210 con 0x560960945550 2023-10-06T05:25:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.216+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'380 uv380 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.216+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960957210 con 0x560960945550 2023-10-06T05:25:21.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.218+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v123'381 uv374 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.218+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.219+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v123'382 uv382 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.219+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.221 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:25:21.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.220+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v123'383 uv383 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.220+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.221+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'384 uv384 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.221+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960953d00 con 0x560960945550 2023-10-06T05:25:21.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.222+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'385 uv385 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.222+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960957210 con 0x560960945550 2023-10-06T05:25:21.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.223+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'386 uv386 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.223+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x560960957210 con 0x560960945550 2023-10-06T05:25:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f289970e700 1 -- 172.21.15.45:0/3004971062 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'387 uv387 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2890016070 con 0x560960945550 2023-10-06T05:25:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560960945550 msgr2=0x560960947930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560960945550 0x560960947930 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x560960922da0 msgr2=0x56096091b5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x560960922da0 0x56096091b5d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x560960950980 msgr2=0x560960952d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x560960950980 0x560960952d60 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x560960940860 msgr2=0x560960942c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x560960940860 0x560960942c40 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560960954610 msgr2=0x560960956cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560960954610 0x560960956cd0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56096094d690 msgr2=0x56096094fa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56096094d690 0x56096094fa70 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56096094a6f0 msgr2=0x56096094cad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56096094a6f0 0x56096094cad0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.224+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 reap_dead start 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 reap_dead start 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 reap_dead start 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f289a710700 1 -- 172.21.15.45:0/3004971062 reap_dead start 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c0658c0 msgr2=0x7f287c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c0658c0 0x7f287c067d70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2890001870 tx=0x7f2890016000 comp rx=0 tx=0).stop 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960876be0 msgr2=0x560960925970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960876be0 0x560960925970 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f288c008ac0 tx=0x7f288c00c040 comp rx=0 tx=0).stop 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 shutdown_connections 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560960876be0 0x560960925970 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 --2- 172.21.15.45:0/3004971062 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f287c0658c0 0x7f287c067d70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 >> 172.21.15.45:0/3004971062 conn(0x56096084bbd0 msgr2=0x56096087b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 shutdown_connections 2023-10-06T05:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.225+0000 7f28a0cf1040 1 -- 172.21.15.45:0/3004971062 wait complete. 2023-10-06T05:25:21.230 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:25:21.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.321+0000 7f94ca049040 1 Processor -- start 2023-10-06T05:25:21.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.321+0000 7f94ca049040 1 -- start start 2023-10-06T05:25:21.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94ca049040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c54250 0x55b425cfdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94ca049040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c70d40 0x55b425c55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94ca049040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94ca049040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b425b73980 con 0x55b425d024a0 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94ca049040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b425b73620 con 0x55b425c70d40 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94ca049040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b425b6a400 con 0x55b425c54250 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3a68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c2a66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c70d40 0x55b425c55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c2a66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c70d40 0x55b425c55b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:53110/0 (socket says 172.21.15.45:53110) 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3a68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c295e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38482/0 (socket says 172.21.15.45:38482) 2023-10-06T05:25:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c2a66700 1 -- 172.21.15.45:0/1573236329 learned_addr learned my addr 172.21.15.45:0/1573236329 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3267700 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c54250 0x55b425cfdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3a68700 1 -- 172.21.15.45:0/1573236329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c54250 msgr2=0x55b425cfdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3a68700 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c54250 0x55b425cfdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3a68700 1 -- 172.21.15.45:0/1573236329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c70d40 msgr2=0x55b425c55b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3a68700 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c70d40 0x55b425c55b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.322+0000 7f94c3a68700 1 -- 172.21.15.45:0/1573236329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc00a180 con 0x55b425d024a0 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94c3a68700 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c295e0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x55b425c14ce0 tx=0x7f94bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f128569059f8c0c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94c1a64700 1 -- 172.21.15.45:0/1573236329 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00f680 con 0x55b425d024a0 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94c1a64700 1 -- 172.21.15.45:0/1573236329 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc0032f0 con 0x55b425d024a0 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94c1a64700 1 -- 172.21.15.45:0/1573236329 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00f950 con 0x55b425d024a0 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94ca049040 1 -- 172.21.15.45:0/1573236329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 msgr2=0x55b425c295e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94ca049040 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c295e0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x55b425c14ce0 tx=0x7f94bc015000 comp rx=0 tx=0).stop 2023-10-06T05:25:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94ca049040 1 -- 172.21.15.45:0/1573236329 shutdown_connections 2023-10-06T05:25:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94ca049040 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c295e0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94ca049040 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c54250 0x55b425cfdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94ca049040 1 --2- 172.21.15.45:0/1573236329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c70d40 0x55b425c55b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.323+0000 7f94ca049040 1 -- 172.21.15.45:0/1573236329 >> 172.21.15.45:0/1573236329 conn(0x55b425c27ff0 msgr2=0x55b425d014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.324+0000 7f94ca049040 1 -- 172.21.15.45:0/1573236329 shutdown_connections 2023-10-06T05:25:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.324+0000 7f94ca049040 1 -- 172.21.15.45:0/1573236329 wait complete. 2023-10-06T05:25:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.324+0000 7f94ca049040 1 Processor -- start 2023-10-06T05:25:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.325+0000 7f94ca049040 1 -- start start 2023-10-06T05:25:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94ca049040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c54250 0x55b425cf3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94ca049040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c70d40 0x55b425c4b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94ca049040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c4bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94ca049040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b425c16990 con 0x55b425d024a0 2023-10-06T05:25:21.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94ca049040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b425c534f0 con 0x55b425c54250 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94ca049040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b425c53790 con 0x55b425c70d40 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c4bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c4bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38486/0 (socket says 172.21.15.45:38486) 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 -- 172.21.15.45:0/1518105753 learned_addr learned my addr 172.21.15.45:0/1518105753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3267700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c54250 0x55b425cf3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c2a66700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c70d40 0x55b425c4b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 -- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c70d40 msgr2=0x55b425c4b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c70d40 0x55b425c4b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 -- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c54250 msgr2=0x55b425cf3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b425c54250 0x55b425cf3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 -- 172.21.15.45:0/1518105753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b425b9c1c0 con 0x55b425d024a0 2023-10-06T05:25:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c2a66700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b425c70d40 0x55b425c4b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.326+0000 7f94c3a68700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c4bc80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f94bc007000 tx=0x7f94bc01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.327+0000 7f94bbfff700 1 -- 172.21.15.45:0/1518105753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00fb60 con 0x55b425d024a0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.327+0000 7f94bbfff700 1 -- 172.21.15.45:0/1518105753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc00f640 con 0x55b425d024a0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.327+0000 7f94bbfff700 1 -- 172.21.15.45:0/1518105753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc030680 con 0x55b425d024a0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.327+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b425b823d0 con 0x55b425d024a0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.327+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b425b75ae0 con 0x55b425d024a0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.328+0000 7f94bbfff700 1 -- 172.21.15.45:0/1518105753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f94bc022070 con 0x55b425d024a0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94bbfff700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f94a4065890 0x7f94a4067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94bbfff700 1 -- 172.21.15.45:0/1518105753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f94bc02c070 con 0x55b425d024a0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94c3267700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f94a4065890 0x7f94a4067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94ca049040 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b425cfdbe0 0x55b425d0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v8 -- 0x55b425cfd3d0 con 0x55b425cfdbe0 2023-10-06T05:25:21.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94c2a66700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b425cfdbe0 0x55b425d0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94c3267700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f94a4065890 0x7f94a4067d40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f94b4000c00 tx=0x7f94b4001840 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.329+0000 7f94c2a66700 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b425cfdbe0 0x55b425d0ffa0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94c2a66700 1 -- 172.21.15.45:0/1518105753 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v123'970 uv970 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f94ac009a10 con 0x55b425cfdbe0 2023-10-06T05:25:21.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b425cfdbe0 msgr2=0x55b425d0ffa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b425cfdbe0 0x55b425d0ffa0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f94a4065890 msgr2=0x7f94a4067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f94a4065890 0x7f94a4067d40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f94b4000c00 tx=0x7f94b4001840 comp rx=0 tx=0).stop 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 msgr2=0x55b425c4bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 --2- 172.21.15.45:0/1518105753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b425d024a0 0x55b425c4bc80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f94bc007000 tx=0x7f94bc01e070 comp rx=0 tx=0).stop 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94c3a68700 1 -- 172.21.15.45:0/1518105753 reap_dead start 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 shutdown_connections 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 >> 172.21.15.45:0/1518105753 conn(0x55b425c27ff0 msgr2=0x55b425d00fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 shutdown_connections 2023-10-06T05:25:21.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.331+0000 7f94ca049040 1 -- 172.21.15.45:0/1518105753 wait complete. 2023-10-06T05:25:21.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:25:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.473+0000 7f1b29798700 1 Processor -- start 2023-10-06T05:25:21.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.473+0000 7f1b29798700 1 -- start start 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b29798700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1b2406a040 0x7f1b24068730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b29798700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b240c8ad0 con 0x7f1b2406a410 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b29798700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2415a800 con 0x7f1b2406dff0 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b29798700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240c9350 con 0x7f1b2406a040 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b227fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1b2406a410 0x7f1b2406d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54004/0 (socket says 172.21.15.45:54004) 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b227fc700 1 -- 172.21.15.45:0/2274402208 learned_addr learned my addr 172.21.15.45:0/2274402208 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b22ffd700 1 --2- 172.21.15.45:0/2274402208 >> v2:172.21.15.144:3300/0 conn(0x7f1b2406a040 0x7f1b24068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.474+0000 7f1b21ffb700 1 -- 172.21.15.45:0/2274402208 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789231273 0 0) 0x7f1b240c8ad0 con 0x7f1b2406a410 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b21ffb700 1 -- 172.21.15.45:0/2274402208 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b10003570 con 0x7f1b2406a410 2023-10-06T05:25:21.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b21ffb700 1 -- 172.21.15.45:0/2274402208 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694554324 0 0) 0x7f1b2415a800 con 0x7f1b2406dff0 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b21ffb700 1 -- 172.21.15.45:0/2274402208 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b240c8ad0 con 0x7f1b2406dff0 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b22ffd700 1 -- 172.21.15.45:0/2274402208 >> v1:172.21.15.144:6789/0 conn(0x7f1b2406dff0 legacy=0x7f1b241597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b22ffd700 1 -- 172.21.15.45:0/2274402208 >> v1:172.21.15.45:6789/0 conn(0x7f1b2406a410 legacy=0x7f1b2406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b22ffd700 1 -- 172.21.15.45:0/2274402208 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2415a520 con 0x7f1b2406a040 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b22ffd700 1 --2- 172.21.15.45:0/2274402208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a040 0x7f1b24068730 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1b140098e0 tx=0x7f1b140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a3dc0f86fe7e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b21ffb700 1 -- 172.21.15.45:0/2274402208 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14011630 con 0x7f1b2406a040 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b21ffb700 1 -- 172.21.15.45:0/2274402208 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b14011790 con 0x7f1b2406a040 2023-10-06T05:25:21.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.475+0000 7f1b21ffb700 1 -- 172.21.15.45:0/2274402208 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14011b30 con 0x7f1b2406a040 2023-10-06T05:25:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.476+0000 7f1b29798700 1 -- 172.21.15.45:0/2274402208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a040 msgr2=0x7f1b24068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.476+0000 7f1b29798700 1 --2- 172.21.15.45:0/2274402208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a040 0x7f1b24068730 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1b140098e0 tx=0x7f1b140091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.476+0000 7f1b29798700 1 -- 172.21.15.45:0/2274402208 shutdown_connections 2023-10-06T05:25:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.476+0000 7f1b29798700 1 --2- 172.21.15.45:0/2274402208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a040 0x7f1b24068730 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.476+0000 7f1b29798700 1 -- 172.21.15.45:0/2274402208 >> 172.21.15.45:0/2274402208 conn(0x7f1b240cea10 msgr2=0x7f1b2406c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.476+0000 7f1b29798700 1 -- 172.21.15.45:0/2274402208 shutdown_connections 2023-10-06T05:25:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.476+0000 7f1b29798700 1 -- 172.21.15.45:0/2274402208 wait complete. 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.477+0000 7f1b29798700 1 Processor -- start 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.477+0000 7f1b29798700 1 -- start start 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b29798700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b2406a040 0x7f1b2415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b29798700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a410 0x7f1b24068f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b29798700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2406dff0 0x7f1b240ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b29798700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240c6d10 con 0x7f1b2406a040 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b29798700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240c6ff0 con 0x7f1b2406a410 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b29798700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240c72d0 con 0x7f1b2406dff0 2023-10-06T05:25:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b2406a040 0x7f1b2415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b2406a040 0x7f1b2415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38502/0 (socket says 172.21.15.45:38502) 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 -- 172.21.15.45:0/117603530 learned_addr learned my addr 172.21.15.45:0/117603530 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b227fc700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a410 0x7f1b24068f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b237fe700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2406dff0 0x7f1b240ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 -- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2406dff0 msgr2=0x7f1b240ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2406dff0 0x7f1b240ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 -- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a410 msgr2=0x7f1b24068f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a410 0x7f1b24068f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b22ffd700 1 -- 172.21.15.45:0/117603530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b14009420 con 0x7f1b2406a040 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b227fc700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a410 0x7f1b24068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.478+0000 7f1b237fe700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2406dff0 0x7f1b240ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:21.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.479+0000 7f1b22ffd700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b2406a040 0x7f1b2415b150 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f1b240cbac0 tx=0x7f1b14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.479+0000 7f1b1affd700 1 -- 172.21.15.45:0/117603530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1401f070 con 0x7f1b2406a040 2023-10-06T05:25:21.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.479+0000 7f1b1affd700 1 -- 172.21.15.45:0/117603530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b14000c90 con 0x7f1b2406a040 2023-10-06T05:25:21.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.479+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b240c75b0 con 0x7f1b2406a040 2023-10-06T05:25:21.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.479+0000 7f1b1affd700 1 -- 172.21.15.45:0/117603530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14025420 con 0x7f1b2406a040 2023-10-06T05:25:21.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.479+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b240b0590 con 0x7f1b2406a040 2023-10-06T05:25:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.480+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2406f0b0 con 0x7f1b2406a040 2023-10-06T05:25:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.481+0000 7f1b1affd700 1 -- 172.21.15.45:0/117603530 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b14023070 con 0x7f1b2406a040 2023-10-06T05:25:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.481+0000 7f1b1affd700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b04066f30 0x7f1b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.481+0000 7f1b1affd700 1 -- 172.21.15.45:0/117603530 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f1b14086610 con 0x7f1b2406a040 2023-10-06T05:25:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.481+0000 7f1b227fc700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b04066f30 0x7f1b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.482+0000 7f1b227fc700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b04066f30 0x7f1b040693e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1b240af610 tx=0x7f1b0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.486+0000 7f1b1affd700 1 -- 172.21.15.45:0/117603530 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b14052b30 con 0x7f1b2406a040 2023-10-06T05:25:21.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.736+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f1b240c8d00 con 0x7f1b2406a040 2023-10-06T05:25:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.986+0000 7f1b1affd700 1 -- 172.21.15.45:0/117603530 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v283) v1 ==== 162+0+0 (secure 0 0 0) 0x7f1b1402a020 con 0x7f1b2406a040 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b04066f30 msgr2=0x7f1b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b04066f30 0x7f1b040693e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1b240af610 tx=0x7f1b0c009040 comp rx=0 tx=0).stop 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b2406a040 msgr2=0x7f1b2415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b2406a040 0x7f1b2415b150 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f1b240cbac0 tx=0x7f1b14017000 comp rx=0 tx=0).stop 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 shutdown_connections 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1b04066f30 0x7f1b040693e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2406dff0 0x7f1b240ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b2406a040 0x7f1b2415b150 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 --2- 172.21.15.45:0/117603530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2406a410 0x7f1b24068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 >> 172.21.15.45:0/117603530 conn(0x7f1b240cea10 msgr2=0x7f1b240c2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 shutdown_connections 2023-10-06T05:25:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:21.990+0000 7f1b29798700 1 -- 172.21.15.45:0/117603530 wait complete. 2023-10-06T05:25:21.993 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:25:22.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.550+0000 7fd8741e3700 1 Processor -- start 2023-10-06T05:25:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.550+0000 7fd8741e3700 1 -- start start 2023-10-06T05:25:22.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd8741e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c1539f0 0x7fd86c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd8741e3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd86c0ccfc0 0x7fd86c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd8741e3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c0ce690 con 0x7fd86c153620 2023-10-06T05:25:22.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd8741e3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86c15a7a0 con 0x7fd86c0ccfc0 2023-10-06T05:25:22.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd8741e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c0ce3f0 con 0x7fd86c1539f0 2023-10-06T05:25:22.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd872780700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd86c0ccfc0 0x7fd86c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd872780700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd86c0ccfc0 0x7fd86c15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38506/0 (socket says 172.21.15.45:38506) 2023-10-06T05:25:22.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd872780700 1 -- 172.21.15.45:0/2705847586 learned_addr learned my addr 172.21.15.45:0/2705847586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:22.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd87177e700 1 --2- 172.21.15.45:0/2705847586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c1539f0 0x7fd86c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd872780700 1 -- 172.21.15.45:0/2705847586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c1539f0 msgr2=0x7fd86c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:22.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd872780700 1 --2- 172.21.15.45:0/2705847586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c1539f0 0x7fd86c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd872780700 1 -- 172.21.15.45:0/2705847586 >> v1:172.21.15.144:6789/0 conn(0x7fd86c153620 legacy=0x7fd86c0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd872780700 1 -- 172.21.15.45:0/2705847586 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c15d510 con 0x7fd86c0ccfc0 2023-10-06T05:25:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.551+0000 7fd87177e700 1 --2- 172.21.15.45:0/2705847586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c1539f0 0x7fd86c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd872780700 1 --2- 172.21.15.45:0/2705847586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c0ccfc0 0x7fd86c15cdc0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fd864009f20 tx=0x7fd864009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7890287e9e9c0e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd870f7d700 1 -- 172.21.15.45:0/2705847586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400e070 con 0x7fd86c0ccfc0 2023-10-06T05:25:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd870f7d700 1 -- 172.21.15.45:0/2705847586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86400ba40 con 0x7fd86c0ccfc0 2023-10-06T05:25:22.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd870f7d700 1 -- 172.21.15.45:0/2705847586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640092e0 con 0x7fd86c0ccfc0 2023-10-06T05:25:22.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd8741e3700 1 -- 172.21.15.45:0/2705847586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c0ccfc0 msgr2=0x7fd86c15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2705847586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c0ccfc0 0x7fd86c15cdc0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fd864009f20 tx=0x7fd864009840 comp rx=0 tx=0).stop 2023-10-06T05:25:22.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd8741e3700 1 -- 172.21.15.45:0/2705847586 shutdown_connections 2023-10-06T05:25:22.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2705847586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c0ccfc0 0x7fd86c15cdc0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2705847586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c1539f0 0x7fd86c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.552+0000 7fd8741e3700 1 -- 172.21.15.45:0/2705847586 >> 172.21.15.45:0/2705847586 conn(0x7fd86c06ed50 msgr2=0x7fd86c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.553+0000 7fd8741e3700 1 -- 172.21.15.45:0/2705847586 shutdown_connections 2023-10-06T05:25:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.553+0000 7fd8741e3700 1 -- 172.21.15.45:0/2705847586 wait complete. 2023-10-06T05:25:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.554+0000 7fd8741e3700 1 Processor -- start 2023-10-06T05:25:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.554+0000 7fd8741e3700 1 -- start start 2023-10-06T05:25:22.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.554+0000 7fd8741e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c0ccfc0 0x7fd86c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.554+0000 7fd8741e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c153620 0x7fd86c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.554+0000 7fd8741e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd86c144c50 0x7fd86c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd8741e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c156bf0 con 0x7fd86c153620 2023-10-06T05:25:22.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd8741e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c16cea0 con 0x7fd86c144c50 2023-10-06T05:25:22.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd8741e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c16d180 con 0x7fd86c0ccfc0 2023-10-06T05:25:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c153620 0x7fd86c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c153620 0x7fd86c1566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38508/0 (socket says 172.21.15.45:38508) 2023-10-06T05:25:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 -- 172.21.15.45:0/2987509165 learned_addr learned my addr 172.21.15.45:0/2987509165 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd872780700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd86c144c50 0x7fd86c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd871f7f700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c0ccfc0 0x7fd86c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 -- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c0ccfc0 msgr2=0x7fd86c15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c0ccfc0 0x7fd86c15ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 -- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd86c144c50 msgr2=0x7fd86c157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd86c144c50 0x7fd86c157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 -- 172.21.15.45:0/2987509165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864009ae0 con 0x7fd86c153620 2023-10-06T05:25:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd87177e700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c153620 0x7fd86c1566b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c800 tx=0x7fd85c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd861ffb700 1 -- 172.21.15.45:0/2987509165 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c011600 con 0x7fd86c153620 2023-10-06T05:25:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd861ffb700 1 -- 172.21.15.45:0/2987509165 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c011760 con 0x7fd86c153620 2023-10-06T05:25:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c16d460 con 0x7fd86c153620 2023-10-06T05:25:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.555+0000 7fd861ffb700 1 -- 172.21.15.45:0/2987509165 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c011b20 con 0x7fd86c153620 2023-10-06T05:25:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.556+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c16da20 con 0x7fd86c153620 2023-10-06T05:25:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.557+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86c16dc40 con 0x7fd86c153620 2023-10-06T05:25:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.557+0000 7fd861ffb700 1 -- 172.21.15.45:0/2987509165 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd85c018350 con 0x7fd86c153620 2023-10-06T05:25:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.557+0000 7fd861ffb700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd850066f30 0x7fd8500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.557+0000 7fd861ffb700 1 -- 172.21.15.45:0/2987509165 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7fd85c085010 con 0x7fd86c153620 2023-10-06T05:25:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.557+0000 7fd871f7f700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd850066f30 0x7fd8500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.558+0000 7fd871f7f700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd850066f30 0x7fd8500693e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd868003a30 tx=0x7fd868007ea0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.562+0000 7fd861ffb700 1 -- 172.21.15.45:0/2987509165 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd85c054fc0 con 0x7fd86c153620 2023-10-06T05:25:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.807+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd86c0cdb40 con 0x7fd86c153620 2023-10-06T05:25:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.808+0000 7fd861ffb700 1 -- 172.21.15.45:0/2987509165 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 284 v284) v1 ==== 95+0+4609 (secure 0 0 0) 0x7fd85c01e090 con 0x7fd86c153620 2023-10-06T05:25:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:22.810 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":284,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":284,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:21.987868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8135":{"gid":8135,"name":"b","rank":0,"incarnation":284,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/3552713853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3552713853},{"type":"v1","addr":"172.21.15.144:6835","nonce":3552713853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd850066f30 msgr2=0x7fd8500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd850066f30 0x7fd8500693e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd868003a30 tx=0x7fd868007ea0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c153620 msgr2=0x7fd86c1566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c153620 0x7fd86c1566b0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c800 tx=0x7fd85c008040 comp rx=0 tx=0).stop 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 shutdown_connections 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd850066f30 0x7fd8500693e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd86c144c50 0x7fd86c157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd86c0ccfc0 0x7fd86c15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 --2- 172.21.15.45:0/2987509165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd86c153620 0x7fd86c1566b0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 >> 172.21.15.45:0/2987509165 conn(0x7fd86c06ed50 msgr2=0x7fd86c150560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 shutdown_connections 2023-10-06T05:25:22.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:22.810+0000 7fd8741e3700 1 -- 172.21.15.45:0/2987509165 wait complete. 2023-10-06T05:25:22.814 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 284 2023-10-06T05:25:22.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 284, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:25:21.987868+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8135': {'gid': 8135, 'name': 'b', 'rank': 0, 'incarnation': 284, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/3552713853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 3552713853}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3552713853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:25:22.825 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8135:up:replay 2023-10-06T05:25:22.826 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:25:28.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:28.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.367+0000 7f6846c57700 1 Processor -- start 2023-10-06T05:25:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.367+0000 7f6846c57700 1 -- start start 2023-10-06T05:25:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.367+0000 7f6846c57700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68401539f0 0x7f684015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f6846c57700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68400ce690 con 0x7f68400ccfc0 2023-10-06T05:25:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f6846c57700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684015d510 con 0x7f6840153620 2023-10-06T05:25:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f6846c57700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684015a7a0 con 0x7f68401539f0 2023-10-06T05:25:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68401539f0 0x7f684015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f68451f4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68400ccfc0 0x7f68400ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58650/0 (socket says 172.21.15.45:58650) 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68401539f0 0x7f684015a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48240/0 (socket says 172.21.15.45:48240) 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f68451f4700 1 -- 172.21.15.45:0/621590567 learned_addr learned my addr 172.21.15.45:0/621590567 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683ffff700 1 -- 172.21.15.45:0/621590567 >> v1:172.21.15.144:6789/0 conn(0x7f6840153620 legacy=0x7f68400cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683ffff700 1 -- 172.21.15.45:0/621590567 >> v1:172.21.15.45:6789/0 conn(0x7f68400ccfc0 legacy=0x7f68400ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683ffff700 1 -- 172.21.15.45:0/621590567 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684015d7f0 con 0x7f68401539f0 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683f7fe700 1 -- 172.21.15.45:0/621590567 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360415904 0 0) 0x7f68400ce690 con 0x7f68400ccfc0 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683ffff700 1 --2- 172.21.15.45:0/621590567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68401539f0 0x7f684015a260 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f68280098e0 tx=0x7f68280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9862ad3e8cb43b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683f7fe700 1 -- 172.21.15.45:0/621590567 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828009e80 con 0x7f68401539f0 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683f7fe700 1 -- 172.21.15.45:0/621590567 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800b860 con 0x7f68401539f0 2023-10-06T05:25:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.368+0000 7f683f7fe700 1 -- 172.21.15.45:0/621590567 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800bbe0 con 0x7f68401539f0 2023-10-06T05:25:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.369+0000 7f6846c57700 1 -- 172.21.15.45:0/621590567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68401539f0 msgr2=0x7f684015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.369+0000 7f6846c57700 1 --2- 172.21.15.45:0/621590567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68401539f0 0x7f684015a260 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f68280098e0 tx=0x7f68280091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.369+0000 7f6846c57700 1 -- 172.21.15.45:0/621590567 shutdown_connections 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.369+0000 7f6846c57700 1 --2- 172.21.15.45:0/621590567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68401539f0 0x7f684015a260 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.369+0000 7f6846c57700 1 -- 172.21.15.45:0/621590567 >> 172.21.15.45:0/621590567 conn(0x7f684006ed50 msgr2=0x7f684006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.369+0000 7f6846c57700 1 -- 172.21.15.45:0/621590567 shutdown_connections 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.369+0000 7f6846c57700 1 -- 172.21.15.45:0/621590567 wait complete. 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.370+0000 7f6846c57700 1 Processor -- start 2023-10-06T05:25:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.371+0000 7f6846c57700 1 -- start start 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.371+0000 7f6846c57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68400ccfc0 0x7f68400c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.371+0000 7f6846c57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840153620 0x7f68400cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.371+0000 7f6846c57700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68400d0390 0x7f6840152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.371+0000 7f6846c57700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6840156eb0 con 0x7f68400ccfc0 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.371+0000 7f6846c57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400cff00 con 0x7f6840153620 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.371+0000 7f6846c57700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68401526a0 con 0x7f68400d0390 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68400ccfc0 0x7f68400c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68400ccfc0 0x7f68400c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48254/0 (socket says 172.21.15.45:48254) 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 -- 172.21.15.45:0/252965966 learned_addr learned my addr 172.21.15.45:0/252965966 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68451f4700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68400d0390 0x7f6840152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f683ffff700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840153620 0x7f68400cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 -- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68400d0390 msgr2=0x7f6840152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68400d0390 0x7f6840152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 -- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840153620 msgr2=0x7f68400cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840153620 0x7f68400cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 -- 172.21.15.45:0/252965966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009420 con 0x7f68400ccfc0 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f683ffff700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840153620 0x7f68400cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f68449f3700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68400ccfc0 0x7f68400c7bb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f683000c5c0 tx=0x7f683000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f683cff9700 1 -- 172.21.15.45:0/252965966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830011590 con 0x7f68400ccfc0 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f683cff9700 1 -- 172.21.15.45:0/252965966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6830011710 con 0x7f68400ccfc0 2023-10-06T05:25:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f683cff9700 1 -- 172.21.15.45:0/252965966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830011ad0 con 0x7f68400ccfc0 2023-10-06T05:25:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68400cf2f0 con 0x7f68400ccfc0 2023-10-06T05:25:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.372+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68400cf8b0 con 0x7f68400ccfc0 2023-10-06T05:25:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.373+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68400d01e0 con 0x7f68400ccfc0 2023-10-06T05:25:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.374+0000 7f683cff9700 1 -- 172.21.15.45:0/252965966 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f683001b3b0 con 0x7f68400ccfc0 2023-10-06T05:25:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.375+0000 7f683cff9700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6820066f30 0x7f68200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.375+0000 7f683cff9700 1 -- 172.21.15.45:0/252965966 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f6830084c30 con 0x7f68400ccfc0 2023-10-06T05:25:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.375+0000 7f683ffff700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6820066f30 0x7f68200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.375+0000 7f683ffff700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6820066f30 0x7f68200693e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f68400b2c90 tx=0x7f6828017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.379+0000 7f683cff9700 1 -- 172.21.15.45:0/252965966 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68300511d0 con 0x7f68400ccfc0 2023-10-06T05:25:28.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.626+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68400cbb40 con 0x7f68400ccfc0 2023-10-06T05:25:28.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.627+0000 7f683cff9700 1 -- 172.21.15.45:0/252965966 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 287 v287) v1 ==== 95+0+4609 (secure 0 0 0) 0x7f6830017020 con 0x7f68400ccfc0 2023-10-06T05:25:28.630 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:28.630 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":287,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":287,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:24.996480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8135":{"gid":8135,"name":"b","rank":0,"incarnation":284,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6835/3552713853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3552713853},{"type":"v1","addr":"172.21.15.144:6835","nonce":3552713853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6820066f30 msgr2=0x7f68200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6820066f30 0x7f68200693e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f68400b2c90 tx=0x7f6828017000 comp rx=0 tx=0).stop 2023-10-06T05:25:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68400ccfc0 msgr2=0x7f68400c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68400ccfc0 0x7f68400c7bb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f683000c5c0 tx=0x7f683000c980 comp rx=0 tx=0).stop 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 shutdown_connections 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6820066f30 0x7f68200693e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68400d0390 0x7f6840152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68400ccfc0 0x7f68400c7bb0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 --2- 172.21.15.45:0/252965966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840153620 0x7f68400cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 >> 172.21.15.45:0/252965966 conn(0x7f684006ed50 msgr2=0x7f68400cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 shutdown_connections 2023-10-06T05:25:28.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.629+0000 7f6846c57700 1 -- 172.21.15.45:0/252965966 wait complete. 2023-10-06T05:25:28.634 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 287 2023-10-06T05:25:28.645 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 287, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:25:24.996480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8135': {'gid': 8135, 'name': 'b', 'rank': 0, 'incarnation': 284, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.144:6835/3552713853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 3552713853}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3552713853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:25:28.645 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:25:28.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:25:28.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.775+0000 7f8c16473700 1 Processor -- start 2023-10-06T05:25:28.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.775+0000 7f8c16473700 1 -- start start 2023-10-06T05:25:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.776+0000 7f8c16473700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c10159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.776+0000 7f8c16473700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c100ca6d0 con 0x7f8c10159170 2023-10-06T05:25:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.776+0000 7f8c16473700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1015d550 con 0x7f8c100692d0 2023-10-06T05:25:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.776+0000 7f8c16473700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1015d810 con 0x7f8c1015d040 2023-10-06T05:25:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.776+0000 7f8c0ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8c10159170 0x7f8c10068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58654/0 (socket says 172.21.15.45:58654) 2023-10-06T05:25:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.776+0000 7f8c0ffff700 1 -- 172.21.15.45:0/351214727 learned_addr learned my addr 172.21.15.45:0/351214727 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c14a10700 1 --2- 172.21.15.45:0/351214727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c10159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995063856 0 0) 0x7f8c100ca6d0 con 0x7f8c10159170 2023-10-06T05:25:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf8003570 con 0x7f8c10159170 2023-10-06T05:25:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035718220 0 0) 0x7f8c1015d550 con 0x7f8c100692d0 2023-10-06T05:25:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c100ca6d0 con 0x7f8c100692d0 2023-10-06T05:25:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c14a10700 1 -- 172.21.15.45:0/351214727 >> v1:172.21.15.144:6789/0 conn(0x7f8c100692d0 legacy=0x7f8c100aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c14a10700 1 -- 172.21.15.45:0/351214727 >> v1:172.21.15.45:6789/0 conn(0x7f8c10159170 legacy=0x7f8c10068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1634296527 0 0) 0x7f8bf8003570 con 0x7f8c10159170 2023-10-06T05:25:28.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c14a10700 1 -- 172.21.15.45:0/351214727 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1015daf0 con 0x7f8c1015d040 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.777+0000 7f8c14a10700 1 --2- 172.21.15.45:0/351214727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c10159540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009e90 tx=0x7f8c08009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92bc4b66c5411fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800e070 con 0x7f8c1015d040 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0800b870 con 0x7f8c1015d040 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c0effd700 1 -- 172.21.15.45:0/351214727 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00001470 con 0x7f8c1015d040 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c16473700 1 -- 172.21.15.45:0/351214727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 msgr2=0x7f8c10159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c16473700 1 --2- 172.21.15.45:0/351214727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c10159540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009e90 tx=0x7f8c08009780 comp rx=0 tx=0).stop 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c16473700 1 -- 172.21.15.45:0/351214727 shutdown_connections 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c16473700 1 --2- 172.21.15.45:0/351214727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c10159540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c16473700 1 -- 172.21.15.45:0/351214727 >> 172.21.15.45:0/351214727 conn(0x7f8c1006f7c0 msgr2=0x7f8c100d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c16473700 1 -- 172.21.15.45:0/351214727 shutdown_connections 2023-10-06T05:25:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.778+0000 7f8c16473700 1 -- 172.21.15.45:0/351214727 wait complete. 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.779+0000 7f8c16473700 1 Processor -- start 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c16473700 1 -- start start 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c16473700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c100692d0 0x7f8c1014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c16473700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c10159170 0x7f8c100b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c16473700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c100b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c16473700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c8f90 con 0x7f8c100692d0 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c16473700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c8980 con 0x7f8c10159170 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c16473700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c7f00 con 0x7f8c1015d040 2023-10-06T05:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c100692d0 0x7f8c1014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c100692d0 0x7f8c1014d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48256/0 (socket says 172.21.15.45:48256) 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.780+0000 7f8c0ffff700 1 -- 172.21.15.45:0/951904327 learned_addr learned my addr 172.21.15.45:0/951904327 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c10159170 0x7f8c100b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c14a10700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c100b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c0ffff700 1 -- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 msgr2=0x7f8c100b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c100b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c0ffff700 1 -- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c10159170 msgr2=0x7f8c100b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c10159170 0x7f8c100b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c0ffff700 1 -- 172.21.15.45:0/951904327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08009a20 con 0x7f8c100692d0 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c100692d0 0x7f8c1014d470 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000cb60 tx=0x7f8c00001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8bf7fff700 1 -- 172.21.15.45:0/951904327 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00007150 con 0x7f8c100692d0 2023-10-06T05:25:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8bf7fff700 1 -- 172.21.15.45:0/951904327 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c000072b0 con 0x7f8c100692d0 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c100c8c20 con 0x7f8c100692d0 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8bf7fff700 1 -- 172.21.15.45:0/951904327 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00007670 con 0x7f8c100692d0 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.781+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c10169530 con 0x7f8c100692d0 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.783+0000 7f8bf7fff700 1 -- 172.21.15.45:0/951904327 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c00004a40 con 0x7f8c100692d0 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.784+0000 7f8bf7fff700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bfc066f30 0x7f8bfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.784+0000 7f8bf7fff700 1 -- 172.21.15.45:0/951904327 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f8c000848f0 con 0x7f8c100692d0 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.784+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bfc066f30 0x7f8bfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.784+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bfc066f30 0x7f8bfc0693e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8c100c6cd0 tx=0x7f8c04007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.785+0000 7f8c1f787b80 1 Processor -- start 2023-10-06T05:25:28.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.785+0000 7f8c1f787b80 1 -- start start 2023-10-06T05:25:28.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.786+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c2c450 0x55d785c2c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.786+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c34f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.786+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c354d0 0x55d785c35940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.786+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d785aff310 con 0x55d785c354d0 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.786+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d785b3c130 con 0x55d785c11f40 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.786+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d785b39af0 con 0x55d785c2c450 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c14a10700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c354d0 0x55d785c35940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c2c450 0x55d785c2c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c34f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c2c450 0x55d785c2c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:52304/0 (socket says 172.21.15.45:52304) 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c34f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:42948/0 (socket says 172.21.15.45:42948) 2023-10-06T05:25:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c0ffff700 1 -- 172.21.15.45:0/2120476309 learned_addr learned my addr 172.21.15.45:0/2120476309 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c14a10700 1 -- 172.21.15.45:0/2120476309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c2c450 msgr2=0x55d785c2c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c14a10700 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c2c450 0x55d785c2c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c14a10700 1 -- 172.21.15.45:0/2120476309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 msgr2=0x55d785c34f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c14a10700 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c34f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c14a10700 1 -- 172.21.15.45:0/2120476309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d78593b790 con 0x55d785c354d0 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c2c450 0x55d785c2c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8c14a10700 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c354d0 0x55d785c35940 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8c08014000 tx=0x7f8c08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f49ce792ffef1e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8bf57fa700 1 -- 172.21.15.45:0/2120476309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800be00 con 0x55d785c354d0 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8bf57fa700 1 -- 172.21.15.45:0/2120476309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0801e450 con 0x55d785c354d0 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.787+0000 7f8bf57fa700 1 -- 172.21.15.45:0/2120476309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0801e810 con 0x55d785c354d0 2023-10-06T05:25:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 -- 172.21.15.45:0/2120476309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c354d0 msgr2=0x55d785c35940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c354d0 0x55d785c35940 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8c08014000 tx=0x7f8c08017000 comp rx=0 tx=0).stop 2023-10-06T05:25:28.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 -- 172.21.15.45:0/2120476309 shutdown_connections 2023-10-06T05:25:28.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c354d0 0x55d785c35940 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c2c450 0x55d785c2c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/2120476309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c34f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 -- 172.21.15.45:0/2120476309 >> 172.21.15.45:0/2120476309 conn(0x55d785c2f880 msgr2=0x55d785c2fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 -- 172.21.15.45:0/2120476309 shutdown_connections 2023-10-06T05:25:28.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.788+0000 7f8c1f787b80 1 -- 172.21.15.45:0/2120476309 wait complete. 2023-10-06T05:25:28.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.789+0000 7f8c1f787b80 1 -- start start 2023-10-06T05:25:28.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.789+0000 7f8c1f787b80 1 Processor -- start 2023-10-06T05:25:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c2c450 0x55d785be7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d785b358e0 con 0x55d785c2c450 2023-10-06T05:25:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d785ae1980 con 0x55d785c11f40 2023-10-06T05:25:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d785ae1620 con 0x55d785c354d0 2023-10-06T05:25:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c1f787b80 10 client.0 _ll_register_callbacks cb 0x55d785944a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c2c450 0x55d785be7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c2c450 0x55d785be7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48274/0 (socket says 172.21.15.45:48274) 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 -- 172.21.15.45:0/4008380010 learned_addr learned my addr 172.21.15.45:0/4008380010 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c14a10700 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 -- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 msgr2=0x55d785be7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 -- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 msgr2=0x55d785c07600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c07600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 -- 172.21.15.45:0/4008380010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c10169530 con 0x55d785c2c450 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c2c450 0x55d785be7010 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8c100696d0 tx=0x7f8c04015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595c8cedb0dfe90f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.790+0000 7f8bee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8bee7fc700 1 -- 172.21.15.45:0/4008380010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04006820 con 0x55d785c2c450 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8bee7fc700 1 -- 172.21.15.45:0/4008380010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f8c04019070 con 0x55d785c2c450 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d785bd9d60 con 0x55d785c2c450 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8bee7fc700 1 -- 172.21.15.45:0/4008380010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=287) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c04006b90 con 0x55d785c2c450 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 10 client.9993 fetch_fsmap learned FSMap version 287 2023-10-06T05:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=287,monmap=2+}) v3 -- 0x7f8c1015daf0 con 0x55d785c2c450 2023-10-06T05:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8bee7fc700 1 -- 172.21.15.45:0/4008380010 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d785bd9d60 con 0x55d785c2c450 2023-10-06T05:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8bee7fc700 1 -- 172.21.15.45:0/4008380010 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 287) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8c04007050 con 0x55d785c2c450 2023-10-06T05:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 10 client.9993 fetch_fsmap finished waiting for FSMap version 287 2023-10-06T05:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 10 client.9993 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 20 client.9993 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c3e430 0x55d785c3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 4 client.9993 mds_command: new command op to 8135 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 --> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d785bb6b30 con 0x55d785c3e430 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.791+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c3e430 0x55d785c3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.792+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c3e430 0x55d785c3e800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b649e3da80f774d5 server_cookie=79a03a9da65190ad in_seq=0 out_seq=0 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.792+0000 7f8bee7fc700 10 client.9993 ms_handle_connect on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8bee7fc700 1 -- 172.21.15.45:0/4008380010 <== mds.0 v2:172.21.15.144:6834/3552713853 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d785bb6b30 con 0x55d785c3e430 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8bee7fc700 10 client.9993 handle_command_reply: tid=0 2023-10-06T05:25:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 client.9993 shutdown 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c2c450 msgr2=0x55d785be7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c2c450 0x55d785be7010 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8c100696d0 tx=0x7f8c04015000 comp rx=0 tx=0).stop 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 shutdown_connections 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c3e430 0x55d785c3e800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c11f40 0x55d785c07600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8bee7fc700 0 client.9993 ms_handle_reset on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/4008380010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c2c450 0x55d785be7010 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 >> 172.21.15.45:0/4008380010 conn(0x55d785c2f880 msgr2=0x55d785be1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 shutdown_connections 2023-10-06T05:25:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 1 -- 172.21.15.45:0/4008380010 wait complete. 2023-10-06T05:25:28.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.793+0000 7f8c1f787b80 20 client.9993 trim_cache size 0 max 16384 2023-10-06T05:25:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.811+0000 7f8c1f787b80 1 Processor -- start 2023-10-06T05:25:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.811+0000 7f8c1f787b80 1 -- start start 2023-10-06T05:25:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785be63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785c0c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7859cd1e0 con 0x55d785c11f40 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d785be4190 con 0x55d785c2c450 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d785c15360 con 0x55d785c354d0 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785be63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785be63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48288/0 (socket says 172.21.15.45:48288) 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c0ffff700 1 -- 172.21.15.45:0/773800132 learned_addr learned my addr 172.21.15.45:0/773800132 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785c0c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.812+0000 7f8c14a10700 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c0ffff700 1 -- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 msgr2=0x55d785be6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c0ffff700 1 -- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 msgr2=0x55d785c0c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785c0c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c0ffff700 1 -- 172.21.15.45:0/773800132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d785be8b00 con 0x55d785c11f40 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785c0c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785be63d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8c00050ff0 tx=0x7f8c0001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e354a8fe0909472 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8bf5ffb700 1 -- 172.21.15.45:0/773800132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00054af0 con 0x55d785c11f40 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8bf5ffb700 1 -- 172.21.15.45:0/773800132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00052c30 con 0x55d785c11f40 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8bf5ffb700 1 -- 172.21.15.45:0/773800132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00052e90 con 0x55d785c11f40 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c1f787b80 1 -- 172.21.15.45:0/773800132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 msgr2=0x55d785be63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.813+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785be63d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8c00050ff0 tx=0x7f8c0001f040 comp rx=0 tx=0).stop 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 -- 172.21.15.45:0/773800132 shutdown_connections 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785be6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785be63d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/773800132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785c0c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 -- 172.21.15.45:0/773800132 >> 172.21.15.45:0/773800132 conn(0x55d785c2f880 msgr2=0x55d785c0f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 -- 172.21.15.45:0/773800132 shutdown_connections 2023-10-06T05:25:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 -- 172.21.15.45:0/773800132 wait complete. 2023-10-06T05:25:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 -- start start 2023-10-06T05:25:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.814+0000 7f8c1f787b80 1 Processor -- start 2023-10-06T05:25:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785bdb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785bdbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c1f787b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785c09f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d785c33150 con 0x55d785c11f40 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d785be1280 con 0x55d785c2c450 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c1f787b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d785c08980 con 0x55d785c354d0 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c1f787b80 10 client.0 _ll_register_callbacks cb 0x55d785944a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785bdb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785bdb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48292/0 (socket says 172.21.15.45:48292) 2023-10-06T05:25:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 -- 172.21.15.45:0/3669701224 learned_addr learned my addr 172.21.15.45:0/3669701224 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c14a10700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785c09f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785bdbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 -- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 msgr2=0x55d785c09f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785c09f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 -- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 msgr2=0x55d785bdbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785bdbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 -- 172.21.15.45:0/3669701224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c100c8c20 con 0x55d785c11f40 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785bdbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8c0ffff700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785bdb870 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8c0006b080 tx=0x7f8c0007d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee30de615beff325 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.816+0000 7f8bee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8bee7fc700 1 -- 172.21.15.45:0/3669701224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00082170 con 0x55d785c11f40 2023-10-06T05:25:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8bee7fc700 1 -- 172.21.15.45:0/3669701224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f8c00053fe0 con 0x55d785c11f40 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c0801e450 con 0x55d785c11f40 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8bee7fc700 1 -- 172.21.15.45:0/3669701224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=287) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c00067070 con 0x55d785c11f40 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 10 client.9999 fetch_fsmap learned FSMap version 287 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=287,monmap=2+}) v3 -- 0x55d785a280a0 con 0x55d785c11f40 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8bee7fc700 1 -- 172.21.15.45:0/3669701224 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00071020 con 0x55d785c11f40 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8bee7fc700 1 -- 172.21.15.45:0/3669701224 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 287) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8c0005b600 con 0x55d785c11f40 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 10 client.9999 fetch_fsmap finished waiting for FSMap version 287 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 10 client.9999 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 20 client.9999 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c0a460 0x55d785c0a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 4 client.9999 mds_command: new command op to 8135 tid=0[{"prefix": "status"}] 2023-10-06T05:25:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 --> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d785ad88d0 con 0x55d785c0a460 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.817+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c0a460 0x55d785c0a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8bee7fc700 10 client.9999 ms_handle_connect on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c0f7fe700 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c0a460 0x55d785c0a830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34b024335dafdb5 server_cookie=f4d7bb4fab39a0c3 in_seq=0 out_seq=0 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8bee7fc700 1 -- 172.21.15.45:0/3669701224 <== mds.0 v2:172.21.15.144:6834/3552713853 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55d785ad88d0 con 0x55d785c0a460 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8bee7fc700 10 client.9999 handle_command_reply: tid=0 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 client.9999 shutdown 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 msgr2=0x55d785bdb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785bdb870 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8c0006b080 tx=0x7f8c0007d000 comp rx=0 tx=0).stop 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 shutdown_connections 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55d785c0a460 0x55d785c0a830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d785c354d0 0x55d785c09f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d785c11f40 0x55d785bdb870 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 --2- 172.21.15.45:0/3669701224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d785c2c450 0x55d785bdbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8bee7fc700 0 client.9999 ms_handle_reset on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 >> 172.21.15.45:0/3669701224 conn(0x55d785c2f880 msgr2=0x55d785c0c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 shutdown_connections 2023-10-06T05:25:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 1 -- 172.21.15.45:0/3669701224 wait complete. 2023-10-06T05:25:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.818+0000 7f8c1f787b80 20 client.9999 trim_cache size 0 max 16384 2023-10-06T05:25:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:25:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:25:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:25:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8135, 2023-10-06T05:25:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:25:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:25:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:25:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8293529890000002, 2023-10-06T05:25:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 287, 2023-10-06T05:25:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 123, 2023-10-06T05:25:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 123, 2023-10-06T05:25:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.835478235 2023-10-06T05:25:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bfc066f30 msgr2=0x7f8bfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bfc066f30 0x7f8bfc0693e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8c100c6cd0 tx=0x7f8c04007d50 comp rx=0 tx=0).stop 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c100692d0 msgr2=0x7f8c1014d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c100692d0 0x7f8c1014d470 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000cb60 tx=0x7f8c00001ad0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 shutdown_connections 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8bfc066f30 0x7f8bfc0693e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c1015d040 0x7f8c100b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c100692d0 0x7f8c1014d470 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 --2- 172.21.15.45:0/951904327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c10159170 0x7f8c100b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 >> 172.21.15.45:0/951904327 conn(0x7f8c1006f7c0 msgr2=0x7f8c100cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 shutdown_connections 2023-10-06T05:25:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.819+0000 7f8c16473700 1 -- 172.21.15.45:0/951904327 wait complete. 2023-10-06T05:25:28.835 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:25:28.835 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2023-10-06T05:25:28.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.963+0000 7f968b9b0700 1 Processor -- start 2023-10-06T05:25:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.964+0000 7f968b9b0700 1 -- start start 2023-10-06T05:25:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968b9b0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968406daa0 0x7f968406de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968b9b0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96840cbab0 0x7f96840cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968b9b0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9684156a70 0x7f9684156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968b9b0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968406cbc0 con 0x7f9684156a70 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968b9b0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96840cc460 con 0x7f96840cbab0 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968b9b0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840cd2a0 con 0x7f968406daa0 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9684156a70 0x7f9684156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9684156a70 0x7f9684156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48300/0 (socket says 172.21.15.45:48300) 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 -- 172.21.15.45:0/3036149733 learned_addr learned my addr 172.21.15.45:0/3036149733 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9688f4b700 1 --2- 172.21.15.45:0/3036149733 >> v2:172.21.15.144:3300/0 conn(0x7f96840cbab0 0x7f96840cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968974c700 1 --2- 172.21.15.45:0/3036149733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968406daa0 0x7f968406de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 -- 172.21.15.45:0/3036149733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968406daa0 msgr2=0x7f968406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 --2- 172.21.15.45:0/3036149733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968406daa0 0x7f968406de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 -- 172.21.15.45:0/3036149733 >> v2:172.21.15.144:3300/0 conn(0x7f96840cbab0 msgr2=0x7f96840cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 --2- 172.21.15.45:0/3036149733 >> v2:172.21.15.144:3300/0 conn(0x7f96840cbab0 0x7f96840cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 -- 172.21.15.45:0/3036149733 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840cc690 con 0x7f9684156a70 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f968974c700 1 --2- 172.21.15.45:0/3036149733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968406daa0 0x7f968406de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9688f4b700 1 --2- 172.21.15.45:0/3036149733 >> v2:172.21.15.144:3300/0 conn(0x7f96840cbab0 0x7f96840cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.965+0000 7f9689f4d700 1 --2- 172.21.15.45:0/3036149733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9684156a70 0x7f9684156ed0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f968406b4f0 tx=0x7f967c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f295c174414c5425 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f967bfff700 1 -- 172.21.15.45:0/3036149733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00e070 con 0x7f9684156a70 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f967bfff700 1 -- 172.21.15.45:0/3036149733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c00ba50 con 0x7f9684156a70 2023-10-06T05:25:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f967bfff700 1 -- 172.21.15.45:0/3036149733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0092e0 con 0x7f9684156a70 2023-10-06T05:25:28.971 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:28.966+0000 7fdeb3423700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 -- 172.21.15.45:0/3036149733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9684156a70 msgr2=0x7f9684156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 --2- 172.21.15.45:0/3036149733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9684156a70 0x7f9684156ed0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f968406b4f0 tx=0x7f967c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 -- 172.21.15.45:0/3036149733 shutdown_connections 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 --2- 172.21.15.45:0/3036149733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9684156a70 0x7f9684156ed0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 --2- 172.21.15.45:0/3036149733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968406daa0 0x7f968406de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 --2- 172.21.15.45:0/3036149733 >> v2:172.21.15.144:3300/0 conn(0x7f96840cbab0 0x7f96840cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 -- 172.21.15.45:0/3036149733 >> 172.21.15.45:0/3036149733 conn(0x7f96840ce9b0 msgr2=0x7f968406c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 -- 172.21.15.45:0/3036149733 shutdown_connections 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.966+0000 7f968b9b0700 1 -- 172.21.15.45:0/3036149733 wait complete. 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.967+0000 7f968b9b0700 1 Processor -- start 2023-10-06T05:25:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.967+0000 7f968b9b0700 1 -- start start 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.968+0000 7f968b9b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968406daa0 0x7f9684144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.968+0000 7f968b9b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96840cbab0 0x7f96840a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.968+0000 7f968b9b0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9684156a70 0x7f968416cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.968+0000 7f968b9b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840a4cd0 con 0x7f96840cbab0 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.968+0000 7f968b9b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684069640 con 0x7f968406daa0 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.968+0000 7f968b9b0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9684069920 con 0x7f9684156a70 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96840cbab0 0x7f96840a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96840cbab0 0x7f96840a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48316/0 (socket says 172.21.15.45:48316) 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 -- 172.21.15.45:0/2194306274 learned_addr learned my addr 172.21.15.45:0/2194306274 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f968974c700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968406daa0 0x7f9684144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9689f4d700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9684156a70 0x7f968416cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 -- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9684156a70 msgr2=0x7f968416cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9684156a70 0x7f968416cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 -- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968406daa0 msgr2=0x7f9684144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968406daa0 0x7f9684144740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 -- 172.21.15.45:0/2194306274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c009ae0 con 0x7f96840cbab0 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f9688f4b700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96840cbab0 0x7f96840a4790 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f967400e850 tx=0x7f9674008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f96797fa700 1 -- 172.21.15.45:0/2194306274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674013410 con 0x7f96840cbab0 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f96797fa700 1 -- 172.21.15.45:0/2194306274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674013570 con 0x7f96840cbab0 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9684069c00 con 0x7f96840cbab0 2023-10-06T05:25:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f96797fa700 1 -- 172.21.15.45:0/2194306274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674013910 con 0x7f96840cbab0 2023-10-06T05:25:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.969+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9684069140 con 0x7f96840cbab0 2023-10-06T05:25:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.970+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968406a0a0 con 0x7f96840cbab0 2023-10-06T05:25:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.971+0000 7f96797fa700 1 -- 172.21.15.45:0/2194306274 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9674013d00 con 0x7f96840cbab0 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.971+0000 7f96797fa700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9668066f30 0x7f96680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.971+0000 7f96797fa700 1 -- 172.21.15.45:0/2194306274 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f9674084ef0 con 0x7f96840cbab0 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.972+0000 7f968974c700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9668066f30 0x7f96680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.972+0000 7f968974c700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9668066f30 0x7f96680693e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9680006f00 tx=0x7f9680008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:28.976+0000 7f96797fa700 1 -- 172.21.15.45:0/2194306274 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9674051410 con 0x7f96840cbab0 2023-10-06T05:25:29.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.223+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96840693c0 con 0x7f96840cbab0 2023-10-06T05:25:29.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.224+0000 7f96797fa700 1 -- 172.21.15.45:0/2194306274 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v287) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9674020020 con 0x7f96840cbab0 2023-10-06T05:25:29.227 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:29.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9668066f30 msgr2=0x7f96680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9668066f30 0x7f96680693e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9680006f00 tx=0x7f9680008040 comp rx=0 tx=0).stop 2023-10-06T05:25:29.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96840cbab0 msgr2=0x7f96840a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96840cbab0 0x7f96840a4790 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f967400e850 tx=0x7f9674008040 comp rx=0 tx=0).stop 2023-10-06T05:25:29.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 shutdown_connections 2023-10-06T05:25:29.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9668066f30 0x7f96680693e0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9684156a70 0x7f968416cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968406daa0 0x7f9684144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 --2- 172.21.15.45:0/2194306274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96840cbab0 0x7f96840a4790 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 >> 172.21.15.45:0/2194306274 conn(0x7f96840ce9b0 msgr2=0x7f96840adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:29.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 shutdown_connections 2023-10-06T05:25:29.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:29.226+0000 7f968b9b0700 1 -- 172.21.15.45:0/2194306274 wait complete. 2023-10-06T05:25:29.242 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:25:29.242 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:29.242 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:25:29.242 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:25:29.242 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:29.242 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:25:29.243 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:25:29.243 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:29.243 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:29.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:29.243 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:25:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:29.264 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:29.264 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:29.264 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:29.264 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:29.264 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:29.265 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:29.265 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:25:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:29.265 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:29.265 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:25:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:29.266 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:29.266 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:25:29.266 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:25:29.266 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41812sec preferred_lft 41812sec 2023-10-06T05:25:29.266 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:25:29.266 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:25:29.267 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:25:29.268 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:29.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:29.269 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:29.336 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:25:29.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:29.337 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:25:29.403 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:25:29.403 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:29.403 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:25:29.404 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:25:29.404 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:29.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:29.404 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:25:29.404 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:25:29.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:25:29.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:25:29.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:25:29.404 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:25:29.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:29.527 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:25:29.549 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:25:29.549 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:25:29.549 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:29.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:29.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:25:29.625 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:25:29.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:29.626 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:29.688 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:29.688 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:29.688 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:25:29.827 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:25:29.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:25:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:25:29.906 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:25:29.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:29.906 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:25:29.971 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:25:29.971 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:25:30.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:25:30.054 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:25:30.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:25:30.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:25:30.114+0000 7f62cb31e540 -1 init, newargv = 0x55f4319a5c50 newargc=15 2023-10-06T05:25:30.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[175072]: starting ceph client 2023-10-06T05:25:30.123 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[175072]: starting fuse 2023-10-06T05:25:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:25:30.131 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:25:30.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:30.131 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:25:30.193 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:25:30.194 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:25:30.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:25:30.194 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:25:30.194 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:25:30.194 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:25:30.194 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:25:30.195 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:25:30.196 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:25:30.335 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.175072.asok 2023-10-06T05:25:30.516 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.175072.asok 2023-10-06T05:25:30.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:30.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.175072.asok status 2023-10-06T05:25:30.691 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:25:30.691 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "175072", 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:25:30.692 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 10011, 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 10011 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1079153257 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1079153257 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.10011 192.168.0.1:0/1079153257", 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1079153257", 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 287, 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 123, 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:25:30.695 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:25:30.695 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:25:30.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:30.703 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:25:30.703 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:30.723 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:25:30.723 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:30.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:30.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> import json 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> try: 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2023-10-06T05:25:30.797 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2023-10-06T05:25:30.798 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:30.798 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T05:25:30.798 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2023-10-06T05:25:30.798 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T05:25:30.798 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2023-10-06T05:25:30.798 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:25:30.839 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:25:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:daemon-helper: command failed with exit status 5 2023-10-06T05:25:36.903 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:36.901+0000 7fdb6559f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T05:25:36.940 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-10-06T05:25:36.940 INFO:tasks.cephfs.test_damage:Result: EIO on client 2023-10-06T05:25:36.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.079+0000 7f641808c700 1 Processor -- start 2023-10-06T05:25:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.079+0000 7f641808c700 1 -- start start 2023-10-06T05:25:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f641808c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f641015d4c0 0x7f641006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f641808c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f641015d890 0x7f641006add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f641808c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64100c76c0 con 0x7f64100c6bb0 2023-10-06T05:25:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f641808c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64100ac8d0 con 0x7f641015d4c0 2023-10-06T05:25:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f641808c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64100c81a0 con 0x7f641015d890 2023-10-06T05:25:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f6415e28700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f641015d4c0 0x7f641006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f6415e28700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f641015d4c0 0x7f641006a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34720/0 (socket says 172.21.15.45:34720) 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f6415e28700 1 -- 172.21.15.45:0/3849383276 learned_addr learned my addr 172.21.15.45:0/3849383276 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.080+0000 7f6415627700 1 --2- 172.21.15.45:0/3849383276 >> v2:172.21.15.144:3300/0 conn(0x7f641015d890 0x7f641006add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6415e28700 1 -- 172.21.15.45:0/3849383276 >> v2:172.21.15.144:3300/0 conn(0x7f641015d890 msgr2=0x7f641006add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6415e28700 1 --2- 172.21.15.45:0/3849383276 >> v2:172.21.15.144:3300/0 conn(0x7f641015d890 0x7f641006add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6415e28700 1 -- 172.21.15.45:0/3849383276 >> v1:172.21.15.144:6789/0 conn(0x7f64100c6bb0 legacy=0x7f64100c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6415e28700 1 -- 172.21.15.45:0/3849383276 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64100ac5a0 con 0x7f641015d4c0 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6415e28700 1 --2- 172.21.15.45:0/3849383276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f641006a890 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f64100c3540 tx=0x7f640c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfbae5ca0fbc54e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6414e26700 1 -- 172.21.15.45:0/3849383276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c009e70 con 0x7f641015d4c0 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6414e26700 1 -- 172.21.15.45:0/3849383276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c00b860 con 0x7f641015d4c0 2023-10-06T05:25:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.081+0000 7f6414e26700 1 -- 172.21.15.45:0/3849383276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00bc00 con 0x7f641015d4c0 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 -- 172.21.15.45:0/3849383276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 msgr2=0x7f641006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 --2- 172.21.15.45:0/3849383276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f641006a890 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f64100c3540 tx=0x7f640c009250 comp rx=0 tx=0).stop 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 -- 172.21.15.45:0/3849383276 shutdown_connections 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 --2- 172.21.15.45:0/3849383276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f641006a890 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 --2- 172.21.15.45:0/3849383276 >> v2:172.21.15.144:3300/0 conn(0x7f641015d890 0x7f641006add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 -- 172.21.15.45:0/3849383276 >> 172.21.15.45:0/3849383276 conn(0x7f64100cc5a0 msgr2=0x7f64100cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 -- 172.21.15.45:0/3849383276 shutdown_connections 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.082+0000 7f641808c700 1 -- 172.21.15.45:0/3849383276 wait complete. 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.083+0000 7f641808c700 1 Processor -- start 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.083+0000 7f641808c700 1 -- start start 2023-10-06T05:25:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f641808c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64100c6bb0 0x7f6410171d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f641808c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f64100b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f641808c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641015d890 0x7f64100b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f641808c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641016ef50 con 0x7f641015d4c0 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f641808c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641016df60 con 0x7f641015d890 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f641808c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641016eac0 con 0x7f64100c6bb0 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f64100b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f64100b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34728/0 (socket says 172.21.15.45:34728) 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 -- 172.21.15.45:0/2105651929 learned_addr learned my addr 172.21.15.45:0/2105651929 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415e28700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64100c6bb0 0x7f6410171d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6416629700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641015d890 0x7f64100b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 -- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64100c6bb0 msgr2=0x7f6410171d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64100c6bb0 0x7f6410171d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 -- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641015d890 msgr2=0x7f64100b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641015d890 0x7f64100b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 -- 172.21.15.45:0/2105651929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0094f0 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.084+0000 7f6415627700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f64100b14a0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f640400e930 tx=0x7f64040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.085+0000 7f6401ffb700 1 -- 172.21.15.45:0/2105651929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400eed0 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.085+0000 7f6401ffb700 1 -- 172.21.15.45:0/2105651929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640400c850 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.085+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641016e190 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.085+0000 7f6401ffb700 1 -- 172.21.15.45:0/2105651929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400cbf0 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.085+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641016e750 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.086+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641016ec30 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.086+0000 7f6401ffb700 1 -- 172.21.15.45:0/2105651929 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6404013470 con 0x7f641015d4c0 2023-10-06T05:25:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.087+0000 7f6401ffb700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f63f4066f30 0x7f63f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.087+0000 7f6401ffb700 1 -- 172.21.15.45:0/2105651929 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f64040854a0 con 0x7f641015d4c0 2023-10-06T05:25:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.087+0000 7f6415e28700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f63f4066f30 0x7f63f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.088+0000 7f6415e28700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f63f4066f30 0x7f63f40693e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f64100c3540 tx=0x7f640c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.091+0000 7f6401ffb700 1 -- 172.21.15.45:0/2105651929 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6404051960 con 0x7f641015d4c0 2023-10-06T05:25:37.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.339+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64100cc0e0 con 0x7f641015d4c0 2023-10-06T05:25:37.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.340+0000 7f6401ffb700 1 -- 172.21.15.45:0/2105651929 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 288 v288) v1 ==== 95+0+4609 (secure 0 0 0) 0x7f6404054f80 con 0x7f641015d4c0 2023-10-06T05:25:37.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:37.342 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":288,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":288,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:36.902647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8135":{"gid":8135,"name":"b","rank":0,"incarnation":284,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6835/3552713853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3552713853},{"type":"v1","addr":"172.21.15.144:6835","nonce":3552713853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f63f4066f30 msgr2=0x7f63f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f63f4066f30 0x7f63f40693e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f64100c3540 tx=0x7f640c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 msgr2=0x7f64100b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f64100b14a0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f640400e930 tx=0x7f64040091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 shutdown_connections 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f63f4066f30 0x7f63f40693e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641015d890 0x7f64100b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64100c6bb0 0x7f6410171d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 --2- 172.21.15.45:0/2105651929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f641015d4c0 0x7f64100b14a0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.342+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 >> 172.21.15.45:0/2105651929 conn(0x7f64100cc5a0 msgr2=0x7f64100ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.343+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 shutdown_connections 2023-10-06T05:25:37.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.343+0000 7f641808c700 1 -- 172.21.15.45:0/2105651929 wait complete. 2023-10-06T05:25:37.346 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 288 2023-10-06T05:25:37.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2023-10-06T05:25:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.490+0000 7f5df9355700 1 Processor -- start 2023-10-06T05:25:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.490+0000 7f5df9355700 1 -- start start 2023-10-06T05:25:37.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df9355700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5df40d02b0 0x7f5df40d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df9355700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415d820 0x7f5df415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df9355700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5df415c6f0 0x7f5df40afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df9355700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df40cd740 con 0x7f5df40d02b0 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df9355700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df40ccb30 con 0x7f5df415c6f0 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df9355700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40cce10 con 0x7f5df415d820 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df2ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5df40d02b0 0x7f5df40d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df2ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5df40d02b0 0x7f5df40d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34732/0 (socket says 172.21.15.45:34732) 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df2ffd700 1 -- 172.21.15.45:0/3673134618 learned_addr learned my addr 172.21.15.45:0/3673134618 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df37fe700 1 --2- 172.21.15.45:0/3673134618 >> v2:172.21.15.144:3300/0 conn(0x7f5df415c6f0 0x7f5df40afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.491+0000 7f5df27fc700 1 --2- 172.21.15.45:0/3673134618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415d820 0x7f5df415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df2ffd700 1 -- 172.21.15.45:0/3673134618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415d820 msgr2=0x7f5df415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/3673134618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415d820 0x7f5df415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df2ffd700 1 -- 172.21.15.45:0/3673134618 >> v2:172.21.15.144:3300/0 conn(0x7f5df415c6f0 msgr2=0x7f5df40afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/3673134618 >> v2:172.21.15.144:3300/0 conn(0x7f5df415c6f0 0x7f5df40afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df2ffd700 1 -- 172.21.15.45:0/3673134618 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df40cd0f0 con 0x7f5df40d02b0 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/3673134618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df40d02b0 0x7f5df40d0680 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5de40099b0 tx=0x7f5de4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ed55871f628afb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df1ffb700 1 -- 172.21.15.45:0/3673134618 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de400b640 con 0x7f5df40d02b0 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df1ffb700 1 -- 172.21.15.45:0/3673134618 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de400b7a0 con 0x7f5df40d02b0 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.492+0000 7f5df1ffb700 1 -- 172.21.15.45:0/3673134618 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de400bb40 con 0x7f5df40d02b0 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 -- 172.21.15.45:0/3673134618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df40d02b0 msgr2=0x7f5df40d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 --2- 172.21.15.45:0/3673134618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df40d02b0 0x7f5df40d0680 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5de40099b0 tx=0x7f5de4009250 comp rx=0 tx=0).stop 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 -- 172.21.15.45:0/3673134618 shutdown_connections 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 --2- 172.21.15.45:0/3673134618 >> v2:172.21.15.144:3300/0 conn(0x7f5df415c6f0 0x7f5df40afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 --2- 172.21.15.45:0/3673134618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df40d02b0 0x7f5df40d0680 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 --2- 172.21.15.45:0/3673134618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415d820 0x7f5df415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 -- 172.21.15.45:0/3673134618 >> 172.21.15.45:0/3673134618 conn(0x7f5df406ed50 msgr2=0x7f5df406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 -- 172.21.15.45:0/3673134618 shutdown_connections 2023-10-06T05:25:37.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.493+0000 7f5df9355700 1 -- 172.21.15.45:0/3673134618 wait complete. 2023-10-06T05:25:37.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.494+0000 7f5df9355700 1 Processor -- start 2023-10-06T05:25:37.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.494+0000 7f5df9355700 1 -- start start 2023-10-06T05:25:37.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df9355700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df40d02b0 0x7f5df40c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df9355700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415c6f0 0x7f5df40a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df9355700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df415d820 0x7f5df40c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df9355700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40c5d60 con 0x7f5df415d820 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df9355700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40c5150 con 0x7f5df40d02b0 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df9355700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40c5430 con 0x7f5df415c6f0 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df415d820 0x7f5df40c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df415d820 0x7f5df40c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34742/0 (socket says 172.21.15.45:34742) 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df37fe700 1 -- 172.21.15.45:0/741940226 learned_addr learned my addr 172.21.15.45:0/741940226 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df40d02b0 0x7f5df40c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.495+0000 7f5df27fc700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415c6f0 0x7f5df40a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df37fe700 1 -- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415c6f0 msgr2=0x7f5df40a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df37fe700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415c6f0 0x7f5df40a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df37fe700 1 -- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df40d02b0 msgr2=0x7f5df40c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df37fe700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df40d02b0 0x7f5df40c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df37fe700 1 -- 172.21.15.45:0/741940226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40094f0 con 0x7f5df415d820 2023-10-06T05:25:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df37fe700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df415d820 0x7f5df40c61f0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009e90 tx=0x7f5dec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5deaffd700 1 -- 172.21.15.45:0/741940226 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00cb70 con 0x7f5df415d820 2023-10-06T05:25:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5deaffd700 1 -- 172.21.15.45:0/741940226 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec00f040 con 0x7f5df415d820 2023-10-06T05:25:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df40c5710 con 0x7f5df415d820 2023-10-06T05:25:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5deaffd700 1 -- 172.21.15.45:0/741940226 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec014480 con 0x7f5df415d820 2023-10-06T05:25:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.496+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df40c6730 con 0x7f5df415d820 2023-10-06T05:25:37.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.498+0000 7f5deaffd700 1 -- 172.21.15.45:0/741940226 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dec014830 con 0x7f5df415d820 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.498+0000 7f5deaffd700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5dd4066f30 0x7f5dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.498+0000 7f5deaffd700 1 -- 172.21.15.45:0/741940226 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f5dec085c50 con 0x7f5df415d820 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.498+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5dd4066f30 0x7f5dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.499+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5dd4066f30 0x7f5dd40693e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5de40099b0 tx=0x7f5de4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.500+0000 7f5e02669b80 1 Processor -- start 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.500+0000 7f5e02669b80 1 -- start start 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff6a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff6b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff6b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5e02669b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff57b3b0 con 0x55e4ff6a84f0 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff5b81d0 con 0x55e4ff6b1570 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff5b5b90 con 0x55e4ff68dfe0 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff6a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff6a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34746/0 (socket says 172.21.15.45:34746) 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 -- 172.21.15.45:0/565086620 learned_addr learned my addr 172.21.15.45:0/565086620 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df37fe700 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff6b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df27fc700 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff6b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 -- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 msgr2=0x55e4ff6b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff6b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 -- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 msgr2=0x55e4ff6b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff6b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 -- 172.21.15.45:0/565086620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ff3b7830 con 0x55e4ff6a84f0 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.501+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff6a88c0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f5de4006e80 tx=0x7f5de4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb213d489a185cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.502+0000 7f5dd3fff700 1 -- 172.21.15.45:0/565086620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4021070 con 0x55e4ff6a84f0 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.502+0000 7f5dd3fff700 1 -- 172.21.15.45:0/565086620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de402a890 con 0x55e4ff6a84f0 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.502+0000 7f5dd3fff700 1 -- 172.21.15.45:0/565086620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de402aa20 con 0x55e4ff6a84f0 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.502+0000 7f5e02669b80 1 -- 172.21.15.45:0/565086620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 msgr2=0x55e4ff6a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.502+0000 7f5e02669b80 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff6a88c0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f5de4006e80 tx=0x7f5de4021000 comp rx=0 tx=0).stop 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 -- 172.21.15.45:0/565086620 shutdown_connections 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff6b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff6a88c0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 --2- 172.21.15.45:0/565086620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff6b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 -- 172.21.15.45:0/565086620 >> 172.21.15.45:0/565086620 conn(0x55e4ff6ab920 msgr2=0x55e4ff6abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 -- 172.21.15.45:0/565086620 shutdown_connections 2023-10-06T05:25:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 -- 172.21.15.45:0/565086620 wait complete. 2023-10-06T05:25:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 -- start start 2023-10-06T05:25:37.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.503+0000 7f5e02669b80 1 Processor -- start 2023-10-06T05:25:37.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.504+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff67f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.504+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff688af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.504+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff661720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.504+0000 7f5e02669b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff5b1980 con 0x55e4ff6a84f0 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.504+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff55da20 con 0x55e4ff6b1570 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.504+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff55d6c0 con 0x55e4ff68dfe0 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5e02669b80 10 client.0 _ll_register_callbacks cb 0x55e4ff3c0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff688af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df2ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff67f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff688af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34748/0 (socket says 172.21.15.45:34748) 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 -- 172.21.15.45:0/84976745 learned_addr learned my addr 172.21.15.45:0/84976745 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df37fe700 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff661720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 -- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 msgr2=0x55e4ff67f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff67f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 -- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 msgr2=0x55e4ff661720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff661720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 -- 172.21.15.45:0/84976745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df40cd0f0 con 0x55e4ff6a84f0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5df27fc700 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff688af0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc014880 tx=0x7f5ddc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f9f222ad2a08cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5dd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5dd17fa700 1 -- 172.21.15.45:0/84976745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0097d0 con 0x55e4ff6a84f0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5dd17fa700 1 -- 172.21.15.45:0/84976745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f5ddc015630 con 0x55e4ff6a84f0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.505+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4ff655e00 con 0x55e4ff6a84f0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5dd17fa700 1 -- 172.21.15.45:0/84976745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=288) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ddc015a00 con 0x55e4ff6a84f0 2023-10-06T05:25:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 10 client.10029 fetch_fsmap learned FSMap version 288 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=288,monmap=2+}) v3 -- 0x55e4ff4a4140 con 0x55e4ff6a84f0 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5dd17fa700 1 -- 172.21.15.45:0/84976745 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e4ff655e00 con 0x55e4ff6a84f0 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5dd17fa700 1 -- 172.21.15.45:0/84976745 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 288) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5ddc015b60 con 0x55e4ff6a84f0 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 10 client.10029 fetch_fsmap finished waiting for FSMap version 288 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 10 client.10029 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 20 client.10029 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff6bb150 0x55e4ff6bb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 4 client.10029 mds_command: new command op to 8135 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 --> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4ff632bd0 con 0x55e4ff6bb150 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.506+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff6bb150 0x55e4ff6bb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.507+0000 7f5dd17fa700 10 client.10029 ms_handle_connect on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.507+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff6bb150 0x55e4ff6bb520 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd07ce48c2d41d41 server_cookie=1d49f5354fdba1a9 in_seq=0 out_seq=0 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.507+0000 7f5dd17fa700 1 -- 172.21.15.45:0/84976745 <== mds.0 v2:172.21.15.144:6834/3552713853 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e4ff632bd0 con 0x55e4ff6bb150 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.507+0000 7f5dd17fa700 10 client.10029 handle_command_reply: tid=0 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.507+0000 7f5e02669b80 1 client.10029 shutdown 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.507+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 msgr2=0x55e4ff688af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.507+0000 7f5e02669b80 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff688af0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc014880 tx=0x7f5ddc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 shutdown_connections 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff6bb150 0x55e4ff6bb520 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6b1570 0x55e4ff661720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff67f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 --2- 172.21.15.45:0/84976745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff688af0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5dd17fa700 0 client.10029 ms_handle_reset on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 >> 172.21.15.45:0/84976745 conn(0x55e4ff6ab920 msgr2=0x55e4ff65dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 shutdown_connections 2023-10-06T05:25:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 1 -- 172.21.15.45:0/84976745 wait complete. 2023-10-06T05:25:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.508+0000 7f5e02669b80 20 client.10029 trim_cache size 0 max 16384 2023-10-06T05:25:37.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.521+0000 7f5e02669b80 1 Processor -- start 2023-10-06T05:25:37.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.521+0000 7f5e02669b80 1 -- start start 2023-10-06T05:25:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff68dfe0 0x55e4ff6aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5e02669b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff449280 con 0x55e4ff6a84f0 2023-10-06T05:25:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff6ac200 con 0x55e4ff68dfe0 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff6581a0 con 0x55e4ff6b1570 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34754/0 (socket says 172.21.15.45:34754) 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 -- 172.21.15.45:0/443102482 learned_addr learned my addr 172.21.15.45:0/443102482 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df37fe700 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff68dfe0 0x55e4ff6aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 -- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff6b1570 msgr2=0x55e4ff661400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 -- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff68dfe0 msgr2=0x55e4ff6aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff68dfe0 0x55e4ff6aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 -- 172.21.15.45:0/443102482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ff580460 con 0x55e4ff6a84f0 2023-10-06T05:25:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff68dfe0 0x55e4ff6aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df37fe700 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.522+0000 7f5df27fc700 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006c50 tx=0x7f5ddc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c42349a6c28374 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5de8ff9700 1 -- 172.21.15.45:0/443102482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc014b90 con 0x55e4ff6a84f0 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5de8ff9700 1 -- 172.21.15.45:0/443102482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ddc015630 con 0x55e4ff6a84f0 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5de8ff9700 1 -- 172.21.15.45:0/443102482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc015c10 con 0x55e4ff6a84f0 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 -- 172.21.15.45:0/443102482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 msgr2=0x55e4ff660ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006c50 tx=0x7f5ddc01d000 comp rx=0 tx=0).stop 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 -- 172.21.15.45:0/443102482 shutdown_connections 2023-10-06T05:25:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff68dfe0 0x55e4ff6aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 --2- 172.21.15.45:0/443102482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 -- 172.21.15.45:0/443102482 >> 172.21.15.45:0/443102482 conn(0x55e4ff6ab920 msgr2=0x55e4ff65f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 -- 172.21.15.45:0/443102482 shutdown_connections 2023-10-06T05:25:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.523+0000 7f5e02669b80 1 -- 172.21.15.45:0/443102482 wait complete. 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.524+0000 7f5e02669b80 1 -- start start 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.524+0000 7f5e02669b80 1 Processor -- start 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff70dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff65d3b0 con 0x55e4ff6b1570 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff67e320 con 0x55e4ff6a84f0 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4ff689470 con 0x55e4ff68dfe0 2023-10-06T05:25:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 10 client.0 _ll_register_callbacks cb 0x55e4ff688260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34760/0 (socket says 172.21.15.45:34760) 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 -- 172.21.15.45:0/924943343 learned_addr learned my addr 172.21.15.45:0/924943343 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df27fc700 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff70dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 -- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 msgr2=0x55e4ff70dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff70dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 -- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6a84f0 msgr2=0x55e4ff660ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 -- 172.21.15.45:0/924943343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4ff4a4140 con 0x55e4ff6b1570 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5df37fe700 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6b1570 0x55e4ff661400 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5dec052330 tx=0x7f5dec016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6963f6bf1e2ac48f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5dd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5dd17fa700 1 -- 172.21.15.45:0/924943343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0541d0 con 0x55e4ff6b1570 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5dd17fa700 1 -- 172.21.15.45:0/924943343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7f5dec05b0b0 con 0x55e4ff6b1570 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5de402a890 con 0x55e4ff6b1570 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.526+0000 7f5dd17fa700 1 -- 172.21.15.45:0/924943343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=288) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dec055530 con 0x55e4ff6b1570 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 10 client.10035 fetch_fsmap learned FSMap version 288 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=288,monmap=2+}) v3 -- 0x55e4ff418270 con 0x55e4ff6b1570 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5dd17fa700 1 -- 172.21.15.45:0/924943343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de402a890 con 0x55e4ff6b1570 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5dd17fa700 1 -- 172.21.15.45:0/924943343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 288) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5dec05c940 con 0x55e4ff6b1570 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 10 client.10035 fetch_fsmap finished waiting for FSMap version 288 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 10 client.10035 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 20 client.10035 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff681a40 0x55e4ff681e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 4 client.10035 mds_command: new command op to 8135 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 --> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55e4ff554970 con 0x55e4ff681a40 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff681a40 0x55e4ff681e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5df2ffd700 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff681a40 0x55e4ff681e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8c826c5b5effb1 server_cookie=e42ca1f14bc8ded9 in_seq=0 out_seq=0 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.527+0000 7f5dd17fa700 10 client.10035 ms_handle_connect on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5dd17fa700 1 -- 172.21.15.45:0/924943343 <== mds.0 v2:172.21.15.144:6834/3552713853 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+215 (crc 0 0 0) 0x55e4ff554970 con 0x55e4ff681a40 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5dd17fa700 10 client.10035 handle_command_reply: tid=0 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 client.10035 shutdown 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6b1570 msgr2=0x55e4ff661400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6b1570 0x55e4ff661400 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5dec052330 tx=0x7f5dec016040 comp rx=0 tx=0).stop 2023-10-06T05:25:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 shutdown_connections 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:6834/3552713853,v1:172.21.15.144:6835/3552713853] conn(0x55e4ff681a40 0x55e4ff681e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4ff6b1570 0x55e4ff661400 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e4ff68dfe0 0x55e4ff70dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5dd17fa700 0 client.10035 ms_handle_reset on v2:172.21.15.144:6834/3552713853 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 --2- 172.21.15.45:0/924943343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4ff6a84f0 0x55e4ff660ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 >> 172.21.15.45:0/924943343 conn(0x55e4ff6ab920 msgr2=0x55e4ff65f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 shutdown_connections 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 1 -- 172.21.15.45:0/924943343 wait complete. 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.528+0000 7f5e02669b80 20 client.10035 trim_cache size 0 max 16384 2023-10-06T05:25:37.536 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "dentry", 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 3894230329, 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627776, 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: "frag": "*", 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "sixmegs", 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_id": "head", 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/subdir/sixmegs" 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:25:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:25:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5dd4066f30 msgr2=0x7f5dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5dd4066f30 0x7f5dd40693e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5de40099b0 tx=0x7f5de4017000 comp rx=0 tx=0).stop 2023-10-06T05:25:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df415d820 msgr2=0x7f5df40c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df415d820 0x7f5df40c61f0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009e90 tx=0x7f5dec00b040 comp rx=0 tx=0).stop 2023-10-06T05:25:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 shutdown_connections 2023-10-06T05:25:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5dd4066f30 0x7f5dd40693e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df415d820 0x7f5df40c61f0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df40d02b0 0x7f5df40c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 --2- 172.21.15.45:0/741940226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df415c6f0 0x7f5df40a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 >> 172.21.15.45:0/741940226 conn(0x7f5df406ed50 msgr2=0x7f5df406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 shutdown_connections 2023-10-06T05:25:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.529+0000 7f5df9355700 1 -- 172.21.15.45:0/741940226 wait complete. 2023-10-06T05:25:37.543 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2023-10-06T05:25:37.544 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:25:37.544 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:25:37.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:37.584 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:25:37.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:25:37.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:37.585 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:37.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:37.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.827+0000 7fccfe0b2700 1 Processor -- start 2023-10-06T05:25:37.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.827+0000 7fccfe0b2700 1 -- start start 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccfe0b2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fccf80c8ff0 0x7fccf80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccfe0b2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fccf80caad0 0x7fccf80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccfe0b2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf8156af0 0x7fccf8156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccfe0b2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf815b000 con 0x7fccf80caad0 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccfe0b2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf815b2e0 con 0x7fccf80c8ff0 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccfe0b2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf815b5c0 con 0x7fccf8156af0 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccf6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fccf80caad0 0x7fccf80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.828+0000 7fccf6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fccf80caad0 0x7fccf80caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34772/0 (socket says 172.21.15.45:34772) 2023-10-06T05:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf6ffd700 1 -- 172.21.15.45:0/296350202 learned_addr learned my addr 172.21.15.45:0/296350202 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf7fff700 1 --2- 172.21.15.45:0/296350202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf8156af0 0x7fccf8156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf77fe700 1 --2- 172.21.15.45:0/296350202 >> v2:172.21.15.144:3300/0 conn(0x7fccf80c8ff0 0x7fccf80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf6ffd700 1 -- 172.21.15.45:0/296350202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf8156af0 msgr2=0x7fccf8156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf6ffd700 1 --2- 172.21.15.45:0/296350202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf8156af0 0x7fccf8156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf6ffd700 1 -- 172.21.15.45:0/296350202 >> v2:172.21.15.144:3300/0 conn(0x7fccf80c8ff0 msgr2=0x7fccf80c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf6ffd700 1 --2- 172.21.15.45:0/296350202 >> v2:172.21.15.144:3300/0 conn(0x7fccf80c8ff0 0x7fccf80c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf6ffd700 1 -- 172.21.15.45:0/296350202 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf815b8a0 con 0x7fccf80caad0 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf6ffd700 1 --2- 172.21.15.45:0/296350202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80caad0 0x7fccf80caf40 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fccf80cefc0 tx=0x7fccec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19318fdd24f24a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf67fc700 1 -- 172.21.15.45:0/296350202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00b860 con 0x7fccf80caad0 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf67fc700 1 -- 172.21.15.45:0/296350202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec00b9c0 con 0x7fccf80caad0 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.829+0000 7fccf67fc700 1 -- 172.21.15.45:0/296350202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00bd60 con 0x7fccf80caad0 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 -- 172.21.15.45:0/296350202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80caad0 msgr2=0x7fccf80caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/296350202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80caad0 0x7fccf80caf40 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fccf80cefc0 tx=0x7fccec009250 comp rx=0 tx=0).stop 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 -- 172.21.15.45:0/296350202 shutdown_connections 2023-10-06T05:25:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/296350202 >> v2:172.21.15.144:3300/0 conn(0x7fccf80c8ff0 0x7fccf80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/296350202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf8156af0 0x7fccf8156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/296350202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80caad0 0x7fccf80caf40 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 -- 172.21.15.45:0/296350202 >> 172.21.15.45:0/296350202 conn(0x7fccf80cdf50 msgr2=0x7fccf80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 -- 172.21.15.45:0/296350202 shutdown_connections 2023-10-06T05:25:37.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.830+0000 7fccfe0b2700 1 -- 172.21.15.45:0/296350202 wait complete. 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.831+0000 7fccfe0b2700 1 Processor -- start 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.831+0000 7fccfe0b2700 1 -- start start 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccfe0b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80c8ff0 0x7fccf813d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccfe0b2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf80caad0 0x7fccf816cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccfe0b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccf8156af0 0x7fccf816d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccfe0b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8068940 con 0x7fccf80c8ff0 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccfe0b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80b3430 con 0x7fccf8156af0 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccfe0b2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80b3710 con 0x7fccf80caad0 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80c8ff0 0x7fccf813d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80c8ff0 0x7fccf813d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34786/0 (socket says 172.21.15.45:34786) 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf77fe700 1 -- 172.21.15.45:0/1355434066 learned_addr learned my addr 172.21.15.45:0/1355434066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf7fff700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccf8156af0 0x7fccf816d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf6ffd700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf80caad0 0x7fccf816cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf77fe700 1 -- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf80caad0 msgr2=0x7fccf816cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf77fe700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf80caad0 0x7fccf816cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf77fe700 1 -- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccf8156af0 msgr2=0x7fccf816d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.832+0000 7fccf77fe700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccf8156af0 0x7fccf816d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.833+0000 7fccf77fe700 1 -- 172.21.15.45:0/1355434066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0094f0 con 0x7fccf80c8ff0 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.833+0000 7fccf77fe700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80c8ff0 0x7fccf813d900 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d890 tx=0x7fcce8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.833+0000 7fccdf7fe700 1 -- 172.21.15.45:0/1355434066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce800b600 con 0x7fccf80c8ff0 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.833+0000 7fccdf7fe700 1 -- 172.21.15.45:0/1355434066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce800b760 con 0x7fccf80c8ff0 2023-10-06T05:25:37.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.833+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf80b39f0 con 0x7fccf80c8ff0 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.833+0000 7fccdf7fe700 1 -- 172.21.15.45:0/1355434066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce800bae0 con 0x7fccf80c8ff0 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.833+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf814af30 con 0x7fccf80c8ff0 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.834+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf8068c20 con 0x7fccf80c8ff0 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.835+0000 7fccdf7fe700 1 -- 172.21.15.45:0/1355434066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcce801e460 con 0x7fccf80c8ff0 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.835+0000 7fccdf7fe700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcce4066f30 0x7fcce40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.835+0000 7fccdf7fe700 1 -- 172.21.15.45:0/1355434066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9044+0+0 (secure 0 0 0) 0x7fcce8085900 con 0x7fccf80c8ff0 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.835+0000 7fccf6ffd700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcce4066f30 0x7fcce40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.836+0000 7fccf6ffd700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcce4066f30 0x7fcce40693e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fccf80cefc0 tx=0x7fccec017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:37.840+0000 7fccdf7fe700 1 -- 172.21.15.45:0/1355434066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcce8051dc0 con 0x7fccf80c8ff0 2023-10-06T05:25:38.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.092+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fccf80b3c30 con 0x7fccf80c8ff0 2023-10-06T05:25:38.095 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:38.093+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.097+0000 7fccdf7fe700 1 -- 172.21.15.45:0/1355434066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v289) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcce80553e0 con 0x7fccf80c8ff0 2023-10-06T05:25:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcce4066f30 msgr2=0x7fcce40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcce4066f30 0x7fcce40693e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fccf80cefc0 tx=0x7fccec017000 comp rx=0 tx=0).stop 2023-10-06T05:25:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80c8ff0 msgr2=0x7fccf813d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80c8ff0 0x7fccf813d900 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d890 tx=0x7fcce8008040 comp rx=0 tx=0).stop 2023-10-06T05:25:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 shutdown_connections 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcce4066f30 0x7fcce40693e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccf8156af0 0x7fccf816d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccf80c8ff0 0x7fccf813d900 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 --2- 172.21.15.45:0/1355434066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccf80caad0 0x7fccf816cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 >> 172.21.15.45:0/1355434066 conn(0x7fccf80cdf50 msgr2=0x7fccf80c2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 shutdown_connections 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.099+0000 7fccfe0b2700 1 -- 172.21.15.45:0/1355434066 wait complete. 2023-10-06T05:25:38.102 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:38.104 INFO:tasks.ceph.mds.b.smithi144.stderr: -667> 2023-10-06T05:25:28.966+0000 7fdeb3423700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:38.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:25:38.127 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:25:38.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.245+0000 7fc2a8ab0700 1 Processor -- start 2023-10-06T05:25:38.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.245+0000 7fc2a8ab0700 1 -- start start 2023-10-06T05:25:38.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.246+0000 7fc2a8ab0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 0x7fc2a4159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.246+0000 7fc2a8ab0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a40ca6d0 con 0x7fc2a40692d0 2023-10-06T05:25:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a8ab0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a415d550 con 0x7fc2a415d040 2023-10-06T05:25:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a8ab0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a415d810 con 0x7fc2a4159170 2023-10-06T05:25:38.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a2d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc2a40692d0 0x7fc2a40aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36484/0 (socket says 172.21.15.45:36484) 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a2d9d700 1 -- 172.21.15.45:0/86288447 learned_addr learned my addr 172.21.15.45:0/86288447 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a259c700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 0x7fc2a4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694019939 0 0) 0x7fc2a40ca6d0 con 0x7fc2a40692d0 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc290003570 con 0x7fc2a40692d0 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4200865664 0 0) 0x7fc290003570 con 0x7fc2a40692d0 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 msgr2=0x7fc2a4159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 0x7fc2a4159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 >> v1:172.21.15.144:6789/0 conn(0x7fc2a415d040 legacy=0x7fc2a4068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a415daf0 con 0x7fc2a40692d0 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fc29c0017f0 con 0x7fc2a40692d0 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 >> v1:172.21.15.45:6789/0 conn(0x7fc2a40692d0 legacy=0x7fc2a40aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc290003a90 0x7fc290003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 0x7fc290004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a415d810 con 0x7fc290004d50 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29000bb10 con 0x7fc290003a90 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.247+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29000bcd0 con 0x7fc2a4159170 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a2d9d700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a259c700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc290003a90 0x7fc290003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a1d9b700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 0x7fc290004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a2d9d700 1 -- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 msgr2=0x7fc290004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a2d9d700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a4159170 0x7fc290004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a2d9d700 1 -- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc290003a90 msgr2=0x7fc290003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:38.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a2d9d700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc290003a90 0x7fc290003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a2d9d700 1 -- 172.21.15.45:0/86288447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290003570 con 0x7fc290004d50 2023-10-06T05:25:38.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a2d9d700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2900051c0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00a8a0 tx=0x7fc29c002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac33617a11c480b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.254 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0070d0 con 0x7fc290004d50 2023-10-06T05:25:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c007230 con 0x7fc290004d50 2023-10-06T05:25:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.248+0000 7fc2a159a700 1 -- 172.21.15.45:0/86288447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0075f0 con 0x7fc290004d50 2023-10-06T05:25:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.249+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/86288447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 msgr2=0x7fc2900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.249+0000 7fc2a8ab0700 1 --2- 172.21.15.45:0/86288447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2900051c0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00a8a0 tx=0x7fc29c002170 comp rx=0 tx=0).stop 2023-10-06T05:25:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.249+0000 7fc2a2d9d700 1 -- 172.21.15.45:0/86288447 reap_dead start 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.249+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/86288447 shutdown_connections 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.249+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/86288447 >> 172.21.15.45:0/86288447 conn(0x7fc2a406f7c0 msgr2=0x7fc2a40d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.249+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/86288447 shutdown_connections 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.249+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/86288447 wait complete. 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.251+0000 7fc2a8ab0700 1 Processor -- start 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.251+0000 7fc2a8ab0700 1 -- start start 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a8ab0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2a4158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a8ab0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a41696f0 0x7fc2a4169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a8ab0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a415a150 0x7fc2a415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a8ab0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a415ab00 con 0x7fc290004d50 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a8ab0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40c9220 con 0x7fc2a41696f0 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a8ab0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4159cc0 con 0x7fc2a415a150 2023-10-06T05:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2a4158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2a4158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34798/0 (socket says 172.21.15.45:34798) 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 -- 172.21.15.45:0/4287872335 learned_addr learned my addr 172.21.15.45:0/4287872335 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a1d9b700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a41696f0 0x7fc2a4169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a2d9d700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a415a150 0x7fc2a415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 -- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a415a150 msgr2=0x7fc2a415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a415a150 0x7fc2a415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 -- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a41696f0 msgr2=0x7fc2a4169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a41696f0 0x7fc2a4169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a259c700 1 -- 172.21.15.45:0/4287872335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40c8f40 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.252+0000 7fc2a2d9d700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a415a150 0x7fc2a415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.253+0000 7fc2a259c700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2a4158520 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc294011b10 tx=0x7fc2940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.253+0000 7fc29a7fc700 1 -- 172.21.15.45:0/4287872335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400f940 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.253+0000 7fc29a7fc700 1 -- 172.21.15.45:0/4287872335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29400faa0 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.253+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a4159670 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.253+0000 7fc29a7fc700 1 -- 172.21.15.45:0/4287872335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400fe20 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.253+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a4158a60 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.254+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a4063b50 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.255+0000 7fc29a7fc700 1 -- 172.21.15.45:0/4287872335 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc294014650 con 0x7fc290004d50 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.255+0000 7fc29a7fc700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc284066f60 0x7fc284069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.255+0000 7fc29a7fc700 1 -- 172.21.15.45:0/4287872335 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9130+0+0 (secure 0 0 0) 0x7fc294018030 con 0x7fc290004d50 2023-10-06T05:25:38.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.255+0000 7fc2a1d9b700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc284066f60 0x7fc284069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.256+0000 7fc2a1d9b700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc284066f60 0x7fc284069410 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40cd470 tx=0x7fc28c007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.260+0000 7fc29a7fc700 1 -- 172.21.15.45:0/4287872335 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc294052140 con 0x7fc290004d50 2023-10-06T05:25:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:38.551+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fc2a415ddd0 con 0x7fc290004d50 2023-10-06T05:25:39.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.103+0000 7fc29a7fc700 1 -- 172.21.15.45:0/4287872335 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v290) v1 ==== 107+0+0 (secure 0 0 0) 0x7fc294022090 con 0x7fc290004d50 2023-10-06T05:25:39.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.105+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc284066f60 msgr2=0x7fc284069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.105+0000 7fc2a8ab0700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc284066f60 0x7fc284069410 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40cd470 tx=0x7fc28c007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 msgr2=0x7fc2a4158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2a4158520 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc294011b10 tx=0x7fc2940091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 shutdown_connections 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc284066f60 0x7fc284069410 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a415a150 0x7fc2a415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc290004d50 0x7fc2a4158520 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 --2- 172.21.15.45:0/4287872335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a41696f0 0x7fc2a4169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 >> 172.21.15.45:0/4287872335 conn(0x7fc2a406f7c0 msgr2=0x7fc2a40b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 shutdown_connections 2023-10-06T05:25:39.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.106+0000 7fc2a8ab0700 1 -- 172.21.15.45:0/4287872335 wait complete. 2023-10-06T05:25:39.109 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:25:39.123 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:25:39.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.168+0000 7ff0de80e040 1 Processor -- start 2023-10-06T05:25:39.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.168+0000 7ff0de80e040 1 -- start start 2023-10-06T05:25:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0de80e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f721f80 0x56108f6f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0de80e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f7b2900 0x56108f7c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0de80e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2cd0 0x56108f7cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0de80e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108f63d980 con 0x56108f7b2900 2023-10-06T05:25:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0de80e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108f63d620 con 0x56108f721f80 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0de80e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56108f634400 con 0x56108f7b2cd0 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f7b2900 0x56108f7c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f7b2900 0x56108f7c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34830/0 (socket says 172.21.15.45:34830) 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 -- 172.21.15.45:0/2834475195 learned_addr learned my addr 172.21.15.45:0/2834475195 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f721f80 0x56108f6f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d822d700 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2cd0 0x56108f7cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 -- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2cd0 msgr2=0x56108f7cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2cd0 0x56108f7cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 -- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f721f80 msgr2=0x56108f6f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f721f80 0x56108f6f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 -- 172.21.15.45:0/2834475195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c0005db0 con 0x56108f7b2900 2023-10-06T05:25:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d722b700 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f7b2900 0x56108f7c8b90 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000adf0 tx=0x7ff0c0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e099b3a9937771d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d6229700 1 -- 172.21.15.45:0/2834475195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0008a30 con 0x56108f7b2900 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d6229700 1 -- 172.21.15.45:0/2834475195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0008bd0 con 0x56108f7b2900 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.169+0000 7ff0d6229700 1 -- 172.21.15.45:0/2834475195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00126a0 con 0x56108f7b2900 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 -- 172.21.15.45:0/2834475195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f7b2900 msgr2=0x56108f7c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f7b2900 0x56108f7c8b90 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000adf0 tx=0x7ff0c0015000 comp rx=0 tx=0).stop 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 -- 172.21.15.45:0/2834475195 shutdown_connections 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2cd0 0x56108f7cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f721f80 0x56108f6f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 --2- 172.21.15.45:0/2834475195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f7b2900 0x56108f7c8b90 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 -- 172.21.15.45:0/2834475195 >> 172.21.15.45:0/2834475195 conn(0x56108f6f45f0 msgr2=0x56108f7af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 -- 172.21.15.45:0/2834475195 shutdown_connections 2023-10-06T05:25:39.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.170+0000 7ff0de80e040 1 -- 172.21.15.45:0/2834475195 wait complete. 2023-10-06T05:25:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.171+0000 7ff0de80e040 1 Processor -- start 2023-10-06T05:25:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.171+0000 7ff0de80e040 1 -- start start 2023-10-06T05:25:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0de80e040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f721f80 0x56108f7ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0de80e040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2900 0x56108f7cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0de80e040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f7b2cd0 0x56108f7cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0de80e040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108f7d95f0 con 0x56108f721f80 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0de80e040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108f71f410 con 0x56108f7b2cd0 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0de80e040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56108f71f6b0 con 0x56108f7b2900 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f721f80 0x56108f7ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f721f80 0x56108f7ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34836/0 (socket says 172.21.15.45:34836) 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 learned_addr learned my addr 172.21.15.45:0/1369503589 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f7b2cd0 0x56108f7cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d722b700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2900 0x56108f7cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2900 msgr2=0x56108f7cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56108f7b2900 0x56108f7cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f7b2cd0 msgr2=0x56108f7cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f7b2cd0 0x56108f7cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56108f6661c0 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.172+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56108f7b2cd0 0x56108f7cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.173+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f721f80 0x56108f7ae3b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8008b50 tx=0x7ff0c8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.173+0000 7ff0d4a26700 1 -- 172.21.15.45:0/1369503589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800f050 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.173+0000 7ff0d4a26700 1 -- 172.21.15.45:0/1369503589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8010040 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.173+0000 7ff0d4a26700 1 -- 172.21.15.45:0/1369503589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8006cb0 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.173+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56108f64c3d0 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.173+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56108f63fae0 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.174+0000 7ff0d4a26700 1 -- 172.21.15.45:0/1369503589 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0c8015670 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.175+0000 7ff0d4a26700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0b8065890 0x7ff0b8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.175+0000 7ff0d4a26700 1 -- 172.21.15.45:0/1369503589 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9130+0+0 (secure 0 0 0) 0x7ff0c8084970 con 0x56108f721f80 2023-10-06T05:25:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.175+0000 7ff0d722b700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0b8065890 0x7ff0b8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:25:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.175+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56108f7af350 0x56108f7e5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.175+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f6f5010 con 0x56108f7af350 2023-10-06T05:25:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.175+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56108f7af350 0x56108f7e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.176+0000 7ff0d722b700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0b8065890 0x7ff0b8067d40 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff0c001e040 tx=0x7ff0c0003050 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.176+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56108f7af350 0x56108f7e5690 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.178+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v124'971 uv944 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.178+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f739a60 con 0x56108f7af350 2023-10-06T05:25:39.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.179+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v124'972 uv972 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.179+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e5bd0 con 0x56108f7af350 2023-10-06T05:25:39.182 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:25:39.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.180+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v124'973 uv973 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.181+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6f90 con 0x56108f7af350 2023-10-06T05:25:39.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.182+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'974 uv974 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.182+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e7340 con 0x56108f7af350 2023-10-06T05:25:39.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.183+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'975 uv975 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.183+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e5f80 con 0x56108f7af350 2023-10-06T05:25:39.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.184+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'976 uv976 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.184+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6360 con 0x56108f7af350 2023-10-06T05:25:39.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.186+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v124'977 uv950 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.186+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.187+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v124'978 uv978 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.187+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.190 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:25:39.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.188+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v124'979 uv979 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.188+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.190+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'980 uv980 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.190+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.191+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'981 uv981 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.191+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.193+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'982 uv982 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.193+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.194+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v124'983 uv970 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.194+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.196+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'984 uv984 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.196+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.199 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:25:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.197+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v124'985 uv985 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.197+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.198+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'986 uv986 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.198+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.200+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'987 uv987 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.200+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6760 con 0x56108f7af350 2023-10-06T05:25:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.201+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'988 uv988 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.201+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b90 con 0x56108f7af350 2023-10-06T05:25:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.202+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'989 uv989 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.202+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.204+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v124'990 uv963 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.204+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.205+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v124'991 uv991 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.205+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.208 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:25:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.207+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v124'992 uv992 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.207+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.208+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'993 uv993 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.208+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.209+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'994 uv994 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.209+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.210+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'995 uv995 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.210+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.212+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v124'996 uv969 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.212+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.213+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v124'997 uv997 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.213+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.216 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:25:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.214+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v124'998 uv998 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.214+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.216+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'999 uv999 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.216+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.217+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'1000 uv1000 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.217+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7e6b40 con 0x56108f7af350 2023-10-06T05:25:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.218+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'1001 uv1001 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7af350 2023-10-06T05:25:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.218+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56108f7e6b40 0x56108f7e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.218+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ea150 con 0x56108f7e6b40 2023-10-06T05:25:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.218+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56108f7e6b40 0x56108f7e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.219+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56108f7e6b40 0x56108f7e9c10 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.221+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'698 uv678 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.221+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ea580 con 0x56108f7e6b40 2023-10-06T05:25:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.222+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'699 uv699 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.222+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7eaa50 con 0x56108f7e6b40 2023-10-06T05:25:39.225 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:25:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.223+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'700 uv700 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.223+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7eb360 con 0x56108f7e6b40 2023-10-06T05:25:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.224+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v124'701 uv701 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.225+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7eaef0 con 0x56108f7e6b40 2023-10-06T05:25:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.226+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'702 uv702 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.226+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ebc00 con 0x56108f7e6b40 2023-10-06T05:25:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.227+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v124'703 uv697 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.227+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ebfb0 con 0x56108f7e6b40 2023-10-06T05:25:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.228+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v124'704 uv704 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.228+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.231 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:25:39.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.229+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v124'705 uv705 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.229+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.231+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'706 uv706 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.231+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7eb790 con 0x56108f7e6b40 2023-10-06T05:25:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.232+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'707 uv707 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.232+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7eb790 con 0x56108f7e6b40 2023-10-06T05:25:39.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.233+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v124'708 uv696 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.233+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.235+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v124'709 uv709 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.235+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.237 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:25:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.236+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v124'710 uv710 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.236+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7efa80 con 0x56108f7e6b40 2023-10-06T05:25:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.237+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v124'711 uv711 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.237+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7efa80 con 0x56108f7e6b40 2023-10-06T05:25:39.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.239+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'712 uv712 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.239+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.240+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v124'713 uv694 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.240+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.241+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v124'714 uv714 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.241+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.244 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:25:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.243+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v124'715 uv715 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.243+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ed370 con 0x56108f7e6b40 2023-10-06T05:25:39.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.244+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'716 uv716 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.244+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ed370 con 0x56108f7e6b40 2023-10-06T05:25:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.245+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'717 uv717 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.245+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ec360 con 0x56108f7e6b40 2023-10-06T05:25:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.246+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'718 uv718 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.246+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56108f7ec870 0x56108f7eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.246+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ef220 con 0x56108f7ec870 2023-10-06T05:25:39.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.247+0000 7ff0d722b700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56108f7ec870 0x56108f7eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.247+0000 7ff0d722b700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56108f7ec870 0x56108f7eec50 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.248+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.248+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ef6d0 con 0x56108f7ec870 2023-10-06T05:25:39.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.249+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v124'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.249+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7efc50 con 0x56108f7ec870 2023-10-06T05:25:39.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.250+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v124'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:25:39.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.251+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7eb830 con 0x56108f7ec870 2023-10-06T05:25:39.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.251+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.252+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f1690 con 0x56108f7ec870 2023-10-06T05:25:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.253+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.253+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0540 con 0x56108f7ec870 2023-10-06T05:25:39.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.254+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.254+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0940 con 0x56108f7ec870 2023-10-06T05:25:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.255+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'135 uv134 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.255+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0d40 con 0x56108f7ec870 2023-10-06T05:25:39.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.256+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'136 uv136 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.256+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0d40 con 0x56108f7ec870 2023-10-06T05:25:39.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.257+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'137 uv137 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.259 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:25:39.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.257+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7ec870 2023-10-06T05:25:39.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.258+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'138 uv138 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.258+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7e6b40 2023-10-06T05:25:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.259+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.259+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7e6b40 2023-10-06T05:25:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.261+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v124'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.261+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7e6b40 2023-10-06T05:25:39.265 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:25:39.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.262+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v124'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.262+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7e6b40 2023-10-06T05:25:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.263+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.263+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7e6b40 2023-10-06T05:25:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.264+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.264+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7e6b40 2023-10-06T05:25:39.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.266+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7e6b40 2023-10-06T05:25:39.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.266+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56108f7f1fe0 0x56108f7f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.266+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f0cf0 con 0x56108f7f1fe0 2023-10-06T05:25:39.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.266+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56108f7f1fe0 0x56108f7f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.267+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56108f7f1fe0 0x56108f7f43c0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.268+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v124'349 uv343 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.268+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.269+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v124'350 uv350 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.269+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.270+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v124'351 uv351 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.272 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:25:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.271+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f5910 con 0x56108f7f1fe0 2023-10-06T05:25:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.272+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'352 uv352 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.272+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f5910 con 0x56108f7f1fe0 2023-10-06T05:25:39.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.273+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'353 uv353 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.273+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.274+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.274+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.275+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v124'355 uv348 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.275+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.276+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v124'356 uv356 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.276+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.279 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:25:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.278+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v124'357 uv357 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.278+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.279+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'358 uv358 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.279+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f4900 con 0x56108f7f1fe0 2023-10-06T05:25:39.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.280+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'359 uv359 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.280+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56108f7f4f80 0x56108f7f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.280+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f79c0 con 0x56108f7f4f80 2023-10-06T05:25:39.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.280+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56108f7f4f80 0x56108f7f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.281+0000 7ff0d7a2c700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56108f7f4f80 0x56108f7f7360 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.282+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v124'158 uv157 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7f4f80 2023-10-06T05:25:39.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.282+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7f4f80 2023-10-06T05:25:39.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.283+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v124'159 uv159 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7f4f80 2023-10-06T05:25:39.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.283+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7f4f80 2023-10-06T05:25:39.286 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:25:39.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.284+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v124'160 uv160 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7f4f80 2023-10-06T05:25:39.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.284+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7f4f80 2023-10-06T05:25:39.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.285+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'161 uv161 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7f4f80 2023-10-06T05:25:39.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.286+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7f4f80 2023-10-06T05:25:39.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.287+0000 7ff0d7a2c700 1 -- 172.21.15.45:0/1369503589 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'162 uv162 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c801b020 con 0x56108f7f4f80 2023-10-06T05:25:39.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.287+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7ec870 2023-10-06T05:25:39.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.288+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.288+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7ec870 2023-10-06T05:25:39.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.289+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v124'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.290+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7ec870 2023-10-06T05:25:39.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.291+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v124'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.293 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:25:39.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.291+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f8db0 con 0x56108f7ec870 2023-10-06T05:25:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.292+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.292+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f8db0 con 0x56108f7ec870 2023-10-06T05:25:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.293+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.293+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7f7da0 con 0x56108f7ec870 2023-10-06T05:25:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.294+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.294+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56108f7f8420 0x56108f7fa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.294+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fae60 con 0x56108f7f8420 2023-10-06T05:25:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.295+0000 7ff0d722b700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56108f7f8420 0x56108f7fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.295+0000 7ff0d722b700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56108f7f8420 0x56108f7fa800 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.297+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v124'314 uv308 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.297+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.298+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v124'315 uv315 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.298+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.301 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:25:39.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.299+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v124'316 uv316 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.299+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.300+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'317 uv317 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.300+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.301+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'318 uv318 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.301+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.302+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v124'319 uv313 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.302+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.303+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v124'320 uv320 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.303+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.306 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:25:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.305+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v124'321 uv321 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.305+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb290 con 0x56108f7f8420 2023-10-06T05:25:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.306+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'322 uv322 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.306+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe000 con 0x56108f7f8420 2023-10-06T05:25:39.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.307+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'323 uv323 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7f8420 2023-10-06T05:25:39.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.307+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb780 con 0x56108f7ec870 2023-10-06T05:25:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.308+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v124'137 uv135 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.308+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb780 con 0x56108f7ec870 2023-10-06T05:25:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.309+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'138 uv138 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.309+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb780 con 0x56108f7ec870 2023-10-06T05:25:39.312 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:25:39.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.310+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v124'139 uv139 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.310+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb780 con 0x56108f7ec870 2023-10-06T05:25:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.311+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'140 uv140 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.312+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56108f7fbc90 0x56108f7fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.312+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7fbc90 2023-10-06T05:25:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.312+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56108f7fbc90 0x56108f7fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.312+0000 7ff0d822d700 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56108f7fbc90 0x56108f7fc0a0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.313+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7fbc90 2023-10-06T05:25:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.313+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7fbc90 2023-10-06T05:25:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.314+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v124'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7fbc90 2023-10-06T05:25:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.314+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7fbc90 2023-10-06T05:25:39.317 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:25:39.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.316+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v124'191 uv191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7fbc90 2023-10-06T05:25:39.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.316+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ff760 con 0x56108f7fbc90 2023-10-06T05:25:39.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.317+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7fbc90 2023-10-06T05:25:39.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.317+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7ff760 con 0x56108f7fbc90 2023-10-06T05:25:39.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.318+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7fbc90 2023-10-06T05:25:39.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.318+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7fbc90 2023-10-06T05:25:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.319+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7fbc90 2023-10-06T05:25:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.319+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7f1fe0 2023-10-06T05:25:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.320+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v124'161 uv160 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.320+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7f1fe0 2023-10-06T05:25:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.322+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v124'162 uv162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.322+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7f1fe0 2023-10-06T05:25:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.323+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v124'163 uv163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.325 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:25:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.323+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fe750 con 0x56108f7f1fe0 2023-10-06T05:25:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.324+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'164 uv164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.324+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7f1fe0 2023-10-06T05:25:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.325+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'165 uv165 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff0d000b680 con 0x56108f7f1fe0 2023-10-06T05:25:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.325+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.327+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v124'389 uv380 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.327+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.328+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v124'390 uv390 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.328+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.331 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:25:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.329+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v124'391 uv391 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.329+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.331+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v124'392 uv392 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.331+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.332+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'393 uv393 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.332+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.333+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v124'394 uv387 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.333+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.334+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v124'395 uv395 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.334+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.335+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v124'396 uv396 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.337 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:25:39.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.335+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.336+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'397 uv397 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.336+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fb310 con 0x56108f7ec870 2023-10-06T05:25:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.338+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'398 uv398 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.338+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7fec40 con 0x56108f7ec870 2023-10-06T05:25:39.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.339+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'399 uv399 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.339+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56108f7feeb0 con 0x56108f7ec870 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0d722b700 1 -- 172.21.15.45:0/1369503589 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'400 uv400 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff0c0022020 con 0x56108f7ec870 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56108f7ec870 msgr2=0x56108f7eec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56108f7ec870 0x56108f7eec50 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56108f7af350 msgr2=0x56108f7e5690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56108f7af350 0x56108f7e5690 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56108f7f8420 msgr2=0x56108f7fa800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56108f7f8420 0x56108f7fa800 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56108f7e6b40 msgr2=0x56108f7e9c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56108f7e6b40 0x56108f7e9c10 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56108f7fbc90 msgr2=0x56108f7fc0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56108f7fbc90 0x56108f7fc0a0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56108f7f4f80 msgr2=0x56108f7f7360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56108f7f4f80 0x56108f7f7360 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56108f7f1fe0 msgr2=0x56108f7f43c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56108f7f1fe0 0x56108f7f43c0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 reap_dead start 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 reap_dead start 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 reap_dead start 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 reap_dead start 2023-10-06T05:25:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0d822d700 1 -- 172.21.15.45:0/1369503589 reap_dead start 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0b8065890 msgr2=0x7ff0b8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0b8065890 0x7ff0b8067d40 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff0c001e040 tx=0x7ff0c0003050 comp rx=0 tx=0).stop 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f721f80 msgr2=0x56108f7ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f721f80 0x56108f7ae3b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8008b50 tx=0x7ff0c8005be0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 shutdown_connections 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56108f721f80 0x56108f7ae3b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 --2- 172.21.15.45:0/1369503589 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff0b8065890 0x7ff0b8067d40 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 >> 172.21.15.45:0/1369503589 conn(0x56108f6f45f0 msgr2=0x56108f7af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 shutdown_connections 2023-10-06T05:25:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.340+0000 7ff0de80e040 1 -- 172.21.15.45:0/1369503589 wait complete. 2023-10-06T05:25:39.346 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:25:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.435+0000 7f4c4a070040 1 Processor -- start 2023-10-06T05:25:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.435+0000 7f4c4a070040 1 -- start start 2023-10-06T05:25:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4a070040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e756b20 0x56066e7fd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4a070040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e7e4930 0x56066e807a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4a070040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7fb3c0 0x56066e807fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4a070040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56066e672980 con 0x56066e756b20 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4a070040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56066e672620 con 0x56066e7fb3c0 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4a070040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56066e669400 con 0x56066e7e4930 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e756b20 0x56066e7fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e756b20 0x56066e7fd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34862/0 (socket says 172.21.15.45:34862) 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 -- 172.21.15.45:0/485385558 learned_addr learned my addr 172.21.15.45:0/485385558 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c43a8f700 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7fb3c0 0x56066e807fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c42a8d700 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e7e4930 0x56066e807a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 -- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e7e4930 msgr2=0x56066e807a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e7e4930 0x56066e807a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 -- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7fb3c0 msgr2=0x56066e807fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7fb3c0 0x56066e807fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 -- 172.21.15.45:0/485385558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c00a0e0 con 0x56066e756b20 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c42a8d700 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e7e4930 0x56066e807a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c4328e700 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e756b20 0x56066e7fd670 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400b3d0 tx=0x7f4c340017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf57263827fd1e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.436+0000 7f4c41a8b700 1 -- 172.21.15.45:0/485385558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400e470 con 0x56066e756b20 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c41a8b700 1 -- 172.21.15.45:0/485385558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400e5d0 con 0x56066e756b20 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c41a8b700 1 -- 172.21.15.45:0/485385558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400e990 con 0x56066e756b20 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 -- 172.21.15.45:0/485385558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e756b20 msgr2=0x56066e7fd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e756b20 0x56066e7fd670 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400b3d0 tx=0x7f4c340017f0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 -- 172.21.15.45:0/485385558 shutdown_connections 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7fb3c0 0x56066e807fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e756b20 0x56066e7fd670 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 --2- 172.21.15.45:0/485385558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e7e4930 0x56066e807a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 -- 172.21.15.45:0/485385558 >> 172.21.15.45:0/485385558 conn(0x56066e727870 msgr2=0x56066e7e1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 -- 172.21.15.45:0/485385558 shutdown_connections 2023-10-06T05:25:39.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.437+0000 7f4c4a070040 1 -- 172.21.15.45:0/485385558 wait complete. 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.438+0000 7f4c4a070040 1 Processor -- start 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.438+0000 7f4c4a070040 1 -- start start 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c4a070040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e756b20 0x56066e7f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c4a070040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7e4930 0x56066e7577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c4a070040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e7fb3c0 0x56066e757ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c4a070040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56066e7deb30 con 0x56066e7fb3c0 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c4a070040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56066e7de0b0 con 0x56066e7e4930 2023-10-06T05:25:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c4a070040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56066e7de350 con 0x56066e756b20 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c43a8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e7fb3c0 0x56066e757ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c4328e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e756b20 0x56066e7f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c43a8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e7fb3c0 0x56066e757ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34874/0 (socket says 172.21.15.45:34874) 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c43a8f700 1 -- 172.21.15.45:0/923837467 learned_addr learned my addr 172.21.15.45:0/923837467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.439+0000 7f4c42a8d700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7e4930 0x56066e7577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c43a8f700 1 -- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e756b20 msgr2=0x56066e7f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c43a8f700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56066e756b20 0x56066e7f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c43a8f700 1 -- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7e4930 msgr2=0x56066e7577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c43a8f700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7e4930 0x56066e7577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c43a8f700 1 -- 172.21.15.45:0/923837467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34001ad0 con 0x56066e7fb3c0 2023-10-06T05:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c42a8d700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56066e7e4930 0x56066e7577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c43a8f700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e7fb3c0 0x56066e757ce0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c011000 tx=0x7f4c3c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c2bfff700 1 -- 172.21.15.45:0/923837467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00f860 con 0x56066e7fb3c0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c2bfff700 1 -- 172.21.15.45:0/923837467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00f9c0 con 0x56066e7fb3c0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c2bfff700 1 -- 172.21.15.45:0/923837467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c010670 con 0x56066e7fb3c0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56066e6813d0 con 0x56066e7fb3c0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.440+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56066e674ae0 con 0x56066e7fb3c0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.442+0000 7f4c2bfff700 1 -- 172.21.15.45:0/923837467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c3c003040 con 0x56066e7fb3c0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.442+0000 7f4c2bfff700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c30065890 0x7f4c30067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.442+0000 7f4c2bfff700 1 -- 172.21.15.45:0/923837467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9130+0+0 (secure 0 0 0) 0x7f4c3c005e40 con 0x56066e7fb3c0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.442+0000 7f4c4328e700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c30065890 0x7f4c30067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.442+0000 7f4c4a070040 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56066e76b840 0x56066e76bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.442+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) v8 -- 0x56066e76c180 con 0x56066e76b840 2023-10-06T05:25:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.443+0000 7f4c42a8d700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56066e76b840 0x56066e76bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.443+0000 7f4c4328e700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c30065890 0x7f4c30067d40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4c34001830 tx=0x7f4c34014000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.443+0000 7f4c42a8d700 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56066e76b840 0x56066e76bc40 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c42a8d700 1 -- 172.21.15.45:0/923837467 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v124'324 uv324 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4c38007580 con 0x56066e76b840 2023-10-06T05:25:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56066e76b840 msgr2=0x56066e76bc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c4a070040 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56066e76b840 0x56066e76bc40 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c30065890 msgr2=0x7f4c30067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c4a070040 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4c30065890 0x7f4c30067d40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4c34001830 tx=0x7f4c34014000 comp rx=0 tx=0).stop 2023-10-06T05:25:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e7fb3c0 msgr2=0x56066e757ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c4a070040 1 --2- 172.21.15.45:0/923837467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56066e7fb3c0 0x56066e757ce0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c011000 tx=0x7f4c3c015000 comp rx=0 tx=0).stop 2023-10-06T05:25:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.444+0000 7f4c43a8f700 1 -- 172.21.15.45:0/923837467 reap_dead start 2023-10-06T05:25:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.445+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 shutdown_connections 2023-10-06T05:25:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.445+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 >> 172.21.15.45:0/923837467 conn(0x56066e727870 msgr2=0x56066e7faa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.445+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 shutdown_connections 2023-10-06T05:25:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.445+0000 7f4c4a070040 1 -- 172.21.15.45:0/923837467 wait complete. 2023-10-06T05:25:39.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:25:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.587+0000 7f93f61aa700 1 Processor -- start 2023-10-06T05:25:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.587+0000 7f93f61aa700 1 -- start start 2023-10-06T05:25:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93f61aa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93f0157330 0x7f93f00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93f61aa700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93f00aaae0 0x7f93f00c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93f61aa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f00c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93f61aa700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f00c3b50 con 0x7f93f0157330 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93f61aa700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f0157700 con 0x7f93f00aaae0 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93f61aa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00c98e0 con 0x7f93f00c90b0 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93ef7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93f0157330 0x7f93f00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93ef7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93f0157330 0x7f93f00ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34888/0 (socket says 172.21.15.45:34888) 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93ef7fe700 1 -- 172.21.15.45:0/2691457995 learned_addr learned my addr 172.21.15.45:0/2691457995 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93effff700 1 --2- 172.21.15.45:0/2691457995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f00c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93eeffd700 1 --2- 172.21.15.45:0/2691457995 >> v2:172.21.15.144:3300/0 conn(0x7f93f00aaae0 0x7f93f00c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93ef7fe700 1 -- 172.21.15.45:0/2691457995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 msgr2=0x7f93f00c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93ef7fe700 1 --2- 172.21.15.45:0/2691457995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f00c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93ef7fe700 1 -- 172.21.15.45:0/2691457995 >> v2:172.21.15.144:3300/0 conn(0x7f93f00aaae0 msgr2=0x7f93f00c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.588+0000 7f93ef7fe700 1 --2- 172.21.15.45:0/2691457995 >> v2:172.21.15.144:3300/0 conn(0x7f93f00aaae0 0x7f93f00c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93ef7fe700 1 -- 172.21.15.45:0/2691457995 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00c9bc0 con 0x7f93f0157330 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93ef7fe700 1 --2- 172.21.15.45:0/2691457995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f0157330 0x7f93f00ce8b0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f93f00c8960 tx=0x7f93e0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a36c95b7d1a060 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93ee7fc700 1 -- 172.21.15.45:0/2691457995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0005c10 con 0x7f93f0157330 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93ee7fc700 1 -- 172.21.15.45:0/2691457995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0005d70 con 0x7f93f0157330 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93ee7fc700 1 -- 172.21.15.45:0/2691457995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0006260 con 0x7f93f0157330 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93f61aa700 1 -- 172.21.15.45:0/2691457995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f0157330 msgr2=0x7f93f00ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93f61aa700 1 --2- 172.21.15.45:0/2691457995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f0157330 0x7f93f00ce8b0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f93f00c8960 tx=0x7f93e0009220 comp rx=0 tx=0).stop 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93f61aa700 1 -- 172.21.15.45:0/2691457995 shutdown_connections 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93f61aa700 1 --2- 172.21.15.45:0/2691457995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f00c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.589+0000 7f93f61aa700 1 --2- 172.21.15.45:0/2691457995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f0157330 0x7f93f00ce8b0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.590+0000 7f93f61aa700 1 --2- 172.21.15.45:0/2691457995 >> v2:172.21.15.144:3300/0 conn(0x7f93f00aaae0 0x7f93f00c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.590+0000 7f93f61aa700 1 -- 172.21.15.45:0/2691457995 >> 172.21.15.45:0/2691457995 conn(0x7f93f0067e00 msgr2=0x7f93f00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:39.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.590+0000 7f93f61aa700 1 -- 172.21.15.45:0/2691457995 shutdown_connections 2023-10-06T05:25:39.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.590+0000 7f93f61aa700 1 -- 172.21.15.45:0/2691457995 wait complete. 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.591+0000 7f93f61aa700 1 Processor -- start 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.591+0000 7f93f61aa700 1 -- start start 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93f61aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f00aaae0 0x7f93f0154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93f61aa700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f0149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93f61aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f0157330 0x7f93f01524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93f61aa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f0154fd0 con 0x7f93f00aaae0 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93f61aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f0152030 con 0x7f93f0157330 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93f61aa700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f0157ec0 con 0x7f93f00c90b0 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f00aaae0 0x7f93f0154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f00aaae0 0x7f93f0154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34898/0 (socket says 172.21.15.45:34898) 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 -- 172.21.15.45:0/4193479796 learned_addr learned my addr 172.21.15.45:0/4193479796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:39.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93effff700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f0157330 0x7f93f01524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93eeffd700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f0149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 -- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 msgr2=0x7f93f0149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f0149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 -- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f0157330 msgr2=0x7f93f01524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f0157330 0x7f93f01524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 -- 172.21.15.45:0/4193479796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0009480 con 0x7f93f00aaae0 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93eeffd700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f0149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.592+0000 7f93ef7fe700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f00aaae0 0x7f93f0154a90 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009f50 tx=0x7f93e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.593+0000 7f93e77fe700 1 -- 172.21.15.45:0/4193479796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0011ab0 con 0x7f93f00aaae0 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.593+0000 7f93e77fe700 1 -- 172.21.15.45:0/4193479796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0011c10 con 0x7f93f00aaae0 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.593+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f01519e0 con 0x7f93f00aaae0 2023-10-06T05:25:39.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.593+0000 7f93e77fe700 1 -- 172.21.15.45:0/4193479796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e001b430 con 0x7f93f00aaae0 2023-10-06T05:25:39.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.593+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f014a080 con 0x7f93f00aaae0 2023-10-06T05:25:39.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.594+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f006e350 con 0x7f93f00aaae0 2023-10-06T05:25:39.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.594+0000 7f93e77fe700 1 -- 172.21.15.45:0/4193479796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93e001b860 con 0x7f93f00aaae0 2023-10-06T05:25:39.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.595+0000 7f93e77fe700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93d0066f30 0x7f93d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:39.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.595+0000 7f93e77fe700 1 -- 172.21.15.45:0/4193479796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9130+0+0 (secure 0 0 0) 0x7f93e0086bf0 con 0x7f93f00aaae0 2023-10-06T05:25:39.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.595+0000 7f93eeffd700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93d0066f30 0x7f93d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:39.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.595+0000 7f93eeffd700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93d0066f30 0x7f93d00693e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f93f015a900 tx=0x7f93d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:39.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.600+0000 7f93e77fe700 1 -- 172.21.15.45:0/4193479796 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93e00530c0 con 0x7f93f00aaae0 2023-10-06T05:25:39.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:39.850+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f93f00c37e0 con 0x7f93f00aaae0 2023-10-06T05:25:40.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.104+0000 7f93e77fe700 1 -- 172.21.15.45:0/4193479796 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v291) v1 ==== 162+0+0 (secure 0 0 0) 0x7f93e002a020 con 0x7f93f00aaae0 2023-10-06T05:25:40.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93d0066f30 msgr2=0x7f93d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93d0066f30 0x7f93d00693e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f93f015a900 tx=0x7f93d8008040 comp rx=0 tx=0).stop 2023-10-06T05:25:40.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f00aaae0 msgr2=0x7f93f0154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f00aaae0 0x7f93f0154a90 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009f50 tx=0x7f93e0016000 comp rx=0 tx=0).stop 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 shutdown_connections 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93d0066f30 0x7f93d00693e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f0157330 0x7f93f01524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93f00aaae0 0x7f93f0154a90 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 --2- 172.21.15.45:0/4193479796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f00c90b0 0x7f93f0149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 >> 172.21.15.45:0/4193479796 conn(0x7f93f0067e00 msgr2=0x7f93f015c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 shutdown_connections 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.107+0000 7f93f61aa700 1 -- 172.21.15.45:0/4193479796 wait complete. 2023-10-06T05:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:25:40.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:40.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.259+0000 7f40f3ea8700 1 Processor -- start 2023-10-06T05:25:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.259+0000 7f40f3ea8700 1 -- start start 2023-10-06T05:25:40.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f3ea8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40ec159200 0x7f40ec0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f3ea8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f3ea8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec0c7d00 con 0x7f40ec1595d0 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f3ea8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec0c7780 con 0x7f40ec159200 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f3ea8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0cd720 con 0x7f40ec0ccdc0 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f1c44700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40ec159200 0x7f40ec0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f1443700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f40ec1595d0 0x7f40ec06b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36514/0 (socket says 172.21.15.45:36514) 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f1443700 1 -- 172.21.15.45:0/2663620124 learned_addr learned my addr 172.21.15.45:0/2663620124 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f2445700 1 --2- 172.21.15.45:0/2663620124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f1c44700 1 -- 172.21.15.45:0/2663620124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 msgr2=0x7f40ec0cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f1c44700 1 --2- 172.21.15.45:0/2663620124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec0cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f1c44700 1 -- 172.21.15.45:0/2663620124 >> v1:172.21.15.45:6789/0 conn(0x7f40ec1595d0 legacy=0x7f40ec06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f0c42700 1 -- 172.21.15.45:0/2663620124 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860478539 0 0) 0x7f40ec0c7d00 con 0x7f40ec1595d0 2023-10-06T05:25:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.260+0000 7f40f1c44700 1 -- 172.21.15.45:0/2663620124 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0c86d0 con 0x7f40ec159200 2023-10-06T05:25:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f1c44700 1 --2- 172.21.15.45:0/2663620124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec159200 0x7f40ec0c8190 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098e0 tx=0x7f40e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d1b594a68bd356 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f0c42700 1 -- 172.21.15.45:0/2663620124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8009e80 con 0x7f40ec159200 2023-10-06T05:25:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f0c42700 1 -- 172.21.15.45:0/2663620124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e800b860 con 0x7f40ec159200 2023-10-06T05:25:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f0c42700 1 -- 172.21.15.45:0/2663620124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e800bbe0 con 0x7f40ec159200 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 -- 172.21.15.45:0/2663620124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec159200 msgr2=0x7f40ec0c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/2663620124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec159200 0x7f40ec0c8190 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098e0 tx=0x7f40e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 -- 172.21.15.45:0/2663620124 shutdown_connections 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/2663620124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec0cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/2663620124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec159200 0x7f40ec0c8190 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 -- 172.21.15.45:0/2663620124 >> 172.21.15.45:0/2663620124 conn(0x7f40ec06f090 msgr2=0x7f40ec06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 -- 172.21.15.45:0/2663620124 shutdown_connections 2023-10-06T05:25:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.261+0000 7f40f3ea8700 1 -- 172.21.15.45:0/2663620124 wait complete. 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.263+0000 7f40f3ea8700 1 Processor -- start 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.263+0000 7f40f3ea8700 1 -- start start 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.263+0000 7f40f3ea8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.263+0000 7f40f3ea8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40ec159200 0x7f40ec0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.263+0000 7f40f3ea8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec1595d0 0x7f40ec15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f3ea8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0afbe0 con 0x7f40ec1595d0 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f3ea8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0b1f10 con 0x7f40ec159200 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f3ea8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0afe30 con 0x7f40ec0ccdc0 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec1595d0 0x7f40ec15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec1595d0 0x7f40ec15b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34926/0 (socket says 172.21.15.45:34926) 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 -- 172.21.15.45:0/1117117332 learned_addr learned my addr 172.21.15.45:0/1117117332 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:40.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f1c44700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f1443700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40ec159200 0x7f40ec0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 -- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 msgr2=0x7f40ec14e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec14e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 -- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40ec159200 msgr2=0x7f40ec0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40ec159200 0x7f40ec0b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 -- 172.21.15.45:0/1117117332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8009420 con 0x7f40ec1595d0 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f2445700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec1595d0 0x7f40ec15b700 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009fa0 tx=0x7f40e4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40e1ffb700 1 -- 172.21.15.45:0/1117117332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e400ed20 con 0x7f40ec1595d0 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40e1ffb700 1 -- 172.21.15.45:0/1117117332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e400ee80 con 0x7f40ec1595d0 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec0b0110 con 0x7f40ec1595d0 2023-10-06T05:25:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.264+0000 7f40e1ffb700 1 -- 172.21.15.45:0/1117117332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4014670 con 0x7f40ec1595d0 2023-10-06T05:25:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.265+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec0b06d0 con 0x7f40ec1595d0 2023-10-06T05:25:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.266+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec06e2f0 con 0x7f40ec1595d0 2023-10-06T05:25:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.266+0000 7f40e1ffb700 1 -- 172.21.15.45:0/1117117332 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40e40149e0 con 0x7f40ec1595d0 2023-10-06T05:25:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.267+0000 7f40e1ffb700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f40d0066f60 0x7f40d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.267+0000 7f40e1ffb700 1 -- 172.21.15.45:0/1117117332 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9130+0+0 (secure 0 0 0) 0x7f40e4085a20 con 0x7f40ec1595d0 2023-10-06T05:25:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.267+0000 7f40f1c44700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f40d0066f60 0x7f40d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.267+0000 7f40f1c44700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f40d0066f60 0x7f40d0069410 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f40e80091e0 tx=0x7f40e8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.271+0000 7f40e1ffb700 1 -- 172.21.15.45:0/1117117332 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40e4051ef0 con 0x7f40ec1595d0 2023-10-06T05:25:40.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.518+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40ec06bb40 con 0x7f40ec1595d0 2023-10-06T05:25:40.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.519+0000 7f40e1ffb700 1 -- 172.21.15.45:0/1117117332 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 292 v292) v1 ==== 95+0+4603 (secure 0 0 0) 0x7f40e4055510 con 0x7f40ec1595d0 2023-10-06T05:25:40.521 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:40.521 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":292,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":292,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:40.105639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9568},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9568":{"gid":9568,"name":"b","rank":0,"incarnation":292,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/26720443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":26720443},{"type":"v1","addr":"172.21.15.144:6835","nonce":26720443}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f40d0066f60 msgr2=0x7f40d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f40d0066f60 0x7f40d0069410 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f40e80091e0 tx=0x7f40e8017000 comp rx=0 tx=0).stop 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec1595d0 msgr2=0x7f40ec15b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec1595d0 0x7f40ec15b700 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009fa0 tx=0x7f40e4009900 comp rx=0 tx=0).stop 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 shutdown_connections 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f40d0066f60 0x7f40d0069410 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40ec1595d0 0x7f40ec15b700 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40ec0ccdc0 0x7f40ec14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 --2- 172.21.15.45:0/1117117332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40ec159200 0x7f40ec0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:40.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 >> 172.21.15.45:0/1117117332 conn(0x7f40ec06f090 msgr2=0x7f40ec0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:40.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 shutdown_connections 2023-10-06T05:25:40.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:40.521+0000 7f40f3ea8700 1 -- 172.21.15.45:0/1117117332 wait complete. 2023-10-06T05:25:40.524 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 292 2023-10-06T05:25:40.536 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:25:43.119 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:43.118+0000 7f42170f1700 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2023-10-06T05:25:43.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:43.119+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:43.121 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:43.119+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:25:43.131 INFO:tasks.ceph.mds.b.smithi144.stderr: -17> 2023-10-06T05:25:43.118+0000 7f42170f1700 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2023-10-06T05:25:43.152 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:25:45.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.676+0000 7f6df99ee700 1 Processor -- start 2023-10-06T05:25:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.676+0000 7f6df99ee700 1 -- start start 2023-10-06T05:25:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6df99ee700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6df40ac200 0x7f6df4159340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6df99ee700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df415db00 con 0x7f6df4158f70 2023-10-06T05:25:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6df99ee700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df415cef0 con 0x7f6df40cb5e0 2023-10-06T05:25:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6df99ee700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40cc170 con 0x7f6df40ac200 2023-10-06T05:25:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6dea7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6df40ac200 0x7f6df4159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6df2ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6df4158f70 0x7f6df40abaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36524/0 (socket says 172.21.15.45:36524) 2023-10-06T05:25:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6df2ffd700 1 -- 172.21.15.45:0/1109301320 learned_addr learned my addr 172.21.15.45:0/1109301320 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6dea7fc700 1 -- 172.21.15.45:0/1109301320 >> v1:172.21.15.144:6789/0 conn(0x7f6df40cb5e0 legacy=0x7f6df40cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.677+0000 7f6dea7fc700 1 -- 172.21.15.45:0/1109301320 >> v1:172.21.15.45:6789/0 conn(0x7f6df4158f70 legacy=0x7f6df40abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6dea7fc700 1 -- 172.21.15.45:0/1109301320 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df415d1d0 con 0x7f6df40ac200 2023-10-06T05:25:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6dea7fc700 1 --2- 172.21.15.45:0/1109301320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df4159340 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0098e0 tx=0x7f6ddc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf3dfa89ff4efd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df27fc700 1 -- 172.21.15.45:0/1109301320 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc009e80 con 0x7f6df40ac200 2023-10-06T05:25:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df27fc700 1 -- 172.21.15.45:0/1109301320 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00b820 con 0x7f6df40ac200 2023-10-06T05:25:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df27fc700 1 -- 172.21.15.45:0/1109301320 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00bba0 con 0x7f6df40ac200 2023-10-06T05:25:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df99ee700 1 -- 172.21.15.45:0/1109301320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 msgr2=0x7f6df4159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df99ee700 1 --2- 172.21.15.45:0/1109301320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df4159340 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0098e0 tx=0x7f6ddc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df99ee700 1 -- 172.21.15.45:0/1109301320 shutdown_connections 2023-10-06T05:25:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df99ee700 1 --2- 172.21.15.45:0/1109301320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df4159340 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df99ee700 1 -- 172.21.15.45:0/1109301320 >> 172.21.15.45:0/1109301320 conn(0x7f6df4069200 msgr2=0x7f6df40cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.678+0000 7f6df99ee700 1 -- 172.21.15.45:0/1109301320 shutdown_connections 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.679+0000 7f6df99ee700 1 -- 172.21.15.45:0/1109301320 wait complete. 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.680+0000 7f6df99ee700 1 Processor -- start 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.680+0000 7f6df99ee700 1 -- start start 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df99ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df99ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df40cb5e0 0x7f6df415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df99ee700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df4158f70 0x7f6df415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df99ee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df415b480 con 0x7f6df40ac200 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df99ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df415c290 con 0x7f6df40cb5e0 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df99ee700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df416d020 con 0x7f6df4158f70 2023-10-06T05:25:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df40ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34948/0 (socket says 172.21.15.45:34948) 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 -- 172.21.15.45:0/3193536824 learned_addr learned my addr 172.21.15.45:0/3193536824 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6dea7fc700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df40cb5e0 0x7f6df415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df37fe700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df4158f70 0x7f6df415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 -- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df4158f70 msgr2=0x7f6df415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df4158f70 0x7f6df415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 -- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df40cb5e0 msgr2=0x7f6df415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df40cb5e0 0x7f6df415aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 -- 172.21.15.45:0/3193536824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc009420 con 0x7f6df40ac200 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6dea7fc700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df40cb5e0 0x7f6df415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df37fe700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df4158f70 0x7f6df415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.681+0000 7f6df2ffd700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df40ac880 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c560 tx=0x7f6de4001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.682+0000 7f6deb7fe700 1 -- 172.21.15.45:0/3193536824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4005b30 con 0x7f6df40ac200 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.682+0000 7f6deb7fe700 1 -- 172.21.15.45:0/3193536824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4005cb0 con 0x7f6df40ac200 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.682+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df415b680 con 0x7f6df40ac200 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.682+0000 7f6deb7fe700 1 -- 172.21.15.45:0/3193536824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4007320 con 0x7f6df40ac200 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.682+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df415bc40 con 0x7f6df40ac200 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.683+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df406dfd0 con 0x7f6df40ac200 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.683+0000 7f6deb7fe700 1 -- 172.21.15.45:0/3193536824 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6de401b3b0 con 0x7f6df40ac200 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.684+0000 7f6deb7fe700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.684+0000 7f6deb7fe700 1 -- 172.21.15.45:0/3193536824 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6de4084880 con 0x7f6df40ac200 2023-10-06T05:25:45.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.684+0000 7f6dea7fc700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.684+0000 7f6dea7fc700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6df41577f0 tx=0x7f6ddc016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.688+0000 7f6deb7fe700 1 -- 172.21.15.45:0/3193536824 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6de4050cc0 con 0x7f6df40ac200 2023-10-06T05:25:45.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.937+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6df415d4b0 con 0x7f6df40ac200 2023-10-06T05:25:45.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.938+0000 7f6deb7fe700 1 -- 172.21.15.45:0/3193536824 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f6de4017020 con 0x7f6df40ac200 2023-10-06T05:25:45.940 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:45.940 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":297,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":9607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2062728889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2062728889},{"type":"v1","addr":"172.21.15.144:6835","nonce":2062728889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":297}],"filesystems":[{"mdsmap":{"epoch":296,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:43.120353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:45.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6dd4066f30 msgr2=0x7f6dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6df41577f0 tx=0x7f6ddc016000 comp rx=0 tx=0).stop 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 msgr2=0x7f6df40ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df40ac880 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c560 tx=0x7f6de4001020 comp rx=0 tx=0).stop 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 shutdown_connections 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df4158f70 0x7f6df415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6df40ac200 0x7f6df40ac880 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 --2- 172.21.15.45:0/3193536824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df40cb5e0 0x7f6df415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 >> 172.21.15.45:0/3193536824 conn(0x7f6df4069200 msgr2=0x7f6df40ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 shutdown_connections 2023-10-06T05:25:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:45.940+0000 7f6df99ee700 1 -- 172.21.15.45:0/3193536824 wait complete. 2023-10-06T05:25:45.944 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 297 2023-10-06T05:25:45.955 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:25:45.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:46.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.086+0000 7fb88a9ad700 1 Processor -- start 2023-10-06T05:25:46.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.086+0000 7fb88a9ad700 1 -- start start 2023-10-06T05:25:46.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.086+0000 7fb88a9ad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb884153650 0x7fb88415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.086+0000 7fb88a9ad700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8840676c0 con 0x7fb884153a20 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.086+0000 7fb88a9ad700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8840679a0 con 0x7fb88406afa0 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.086+0000 7fb88a9ad700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb884067c80 con 0x7fb884153650 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb8837fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb884153a20 0x7fb88406a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36526/0 (socket says 172.21.15.45:36526) 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb8837fe700 1 -- 172.21.15.45:0/1078089647 learned_addr learned my addr 172.21.15.45:0/1078089647 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb883fff700 1 --2- 172.21.15.45:0/1078089647 >> v2:172.21.15.144:3300/0 conn(0x7fb884153650 0x7fb88415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb882ffd700 1 -- 172.21.15.45:0/1078089647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458581645 0 0) 0x7fb8840676c0 con 0x7fb884153a20 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb882ffd700 1 -- 172.21.15.45:0/1078089647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86c003570 con 0x7fb884153a20 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb883fff700 1 -- 172.21.15.45:0/1078089647 >> v1:172.21.15.144:6789/0 conn(0x7fb88406afa0 legacy=0x7fb884068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb882ffd700 1 -- 172.21.15.45:0/1078089647 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572196335 0 0) 0x7fb8840679a0 con 0x7fb88406afa0 2023-10-06T05:25:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb883fff700 1 -- 172.21.15.45:0/1078089647 >> v1:172.21.15.45:6789/0 conn(0x7fb884153a20 legacy=0x7fb88406a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb883fff700 1 -- 172.21.15.45:0/1078089647 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb884067f60 con 0x7fb884153650 2023-10-06T05:25:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.087+0000 7fb883fff700 1 --2- 172.21.15.45:0/1078089647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 0x7fb88415a930 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8740098e0 tx=0x7fb8740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be167c2ff40f460c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb882ffd700 1 -- 172.21.15.45:0/1078089647 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400b640 con 0x7fb884153650 2023-10-06T05:25:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb882ffd700 1 -- 172.21.15.45:0/1078089647 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87400b7a0 con 0x7fb884153650 2023-10-06T05:25:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb882ffd700 1 -- 172.21.15.45:0/1078089647 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400bb40 con 0x7fb884153650 2023-10-06T05:25:46.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb88a9ad700 1 -- 172.21.15.45:0/1078089647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 msgr2=0x7fb88415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/1078089647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 0x7fb88415a930 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8740098e0 tx=0x7fb8740091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb88a9ad700 1 -- 172.21.15.45:0/1078089647 shutdown_connections 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/1078089647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 0x7fb88415a930 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb88a9ad700 1 -- 172.21.15.45:0/1078089647 >> 172.21.15.45:0/1078089647 conn(0x7fb8840cc5a0 msgr2=0x7fb8840cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb88a9ad700 1 -- 172.21.15.45:0/1078089647 shutdown_connections 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.088+0000 7fb88a9ad700 1 -- 172.21.15.45:0/1078089647 wait complete. 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.089+0000 7fb88a9ad700 1 Processor -- start 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.090+0000 7fb88a9ad700 1 -- start start 2023-10-06T05:25:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.090+0000 7fb88a9ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88406afa0 0x7fb884154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.090+0000 7fb88a9ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 0x7fb8840c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.090+0000 7fb88a9ad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb884153a20 0x7fb884152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.090+0000 7fb88a9ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb884156280 con 0x7fb88406afa0 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.090+0000 7fb88a9ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840c24e0 con 0x7fb884153650 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.090+0000 7fb88a9ad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb884155670 con 0x7fb884153a20 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88406afa0 0x7fb884154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88406afa0 0x7fb884154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34962/0 (socket says 172.21.15.45:34962) 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 -- 172.21.15.45:0/3971376385 learned_addr learned my addr 172.21.15.45:0/3971376385 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb8837fe700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 0x7fb8840c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb888f4a700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb884153a20 0x7fb884152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 -- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb884153a20 msgr2=0x7fb884152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb884153a20 0x7fb884152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 -- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 msgr2=0x7fb8840c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 0x7fb8840c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 -- 172.21.15.45:0/3971376385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb874009420 con 0x7fb88406afa0 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb888f4a700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb884153a20 0x7fb884152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb883fff700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88406afa0 0x7fb884154ff0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fb874009f80 tx=0x7fb874017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb86bfff700 1 -- 172.21.15.45:0/3971376385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87401f070 con 0x7fb88406afa0 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb86bfff700 1 -- 172.21.15.45:0/3971376385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874000bc0 con 0x7fb88406afa0 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb86bfff700 1 -- 172.21.15.45:0/3971376385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874025420 con 0x7fb88406afa0 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.091+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb884155950 con 0x7fb88406afa0 2023-10-06T05:25:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.092+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb884155f10 con 0x7fb88406afa0 2023-10-06T05:25:46.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.092+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8840cc0e0 con 0x7fb88406afa0 2023-10-06T05:25:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.093+0000 7fb86bfff700 1 -- 172.21.15.45:0/3971376385 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb874023070 con 0x7fb88406afa0 2023-10-06T05:25:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.093+0000 7fb86bfff700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb870066f30 0x7fb8700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.093+0000 7fb86bfff700 1 -- 172.21.15.45:0/3971376385 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb874086700 con 0x7fb88406afa0 2023-10-06T05:25:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.094+0000 7fb8837fe700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb870066f30 0x7fb8700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.094+0000 7fb8837fe700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb870066f30 0x7fb8700693e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb88415c610 tx=0x7fb878009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.098+0000 7fb86bfff700 1 -- 172.21.15.45:0/3971376385 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb874052b80 con 0x7fb88406afa0 2023-10-06T05:25:46.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.345+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb88415ae70 con 0x7fb88406afa0 2023-10-06T05:25:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.345+0000 7fb86bfff700 1 -- 172.21.15.45:0/3971376385 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fb87402a090 con 0x7fb88406afa0 2023-10-06T05:25:46.357 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:46.357 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":297,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":9607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2062728889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2062728889},{"type":"v1","addr":"172.21.15.144:6835","nonce":2062728889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":297}],"filesystems":[{"mdsmap":{"epoch":296,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:43.120353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb870066f30 msgr2=0x7fb8700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb870066f30 0x7fb8700693e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb88415c610 tx=0x7fb878009040 comp rx=0 tx=0).stop 2023-10-06T05:25:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88406afa0 msgr2=0x7fb884154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88406afa0 0x7fb884154ff0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fb874009f80 tx=0x7fb874017000 comp rx=0 tx=0).stop 2023-10-06T05:25:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 shutdown_connections 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb870066f30 0x7fb8700693e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb884153a20 0x7fb884152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb88406afa0 0x7fb884154ff0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 --2- 172.21.15.45:0/3971376385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb884153650 0x7fb8840c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 >> 172.21.15.45:0/3971376385 conn(0x7fb8840cc5a0 msgr2=0x7fb884158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 shutdown_connections 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.348+0000 7fb88a9ad700 1 -- 172.21.15.45:0/3971376385 wait complete. 2023-10-06T05:25:46.359 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 297 2023-10-06T05:25:46.363 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:25:46.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.494+0000 7fc17573c700 1 Processor -- start 2023-10-06T05:25:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.494+0000 7fc17573c700 1 -- start start 2023-10-06T05:25:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc17573c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc170159170 0x7fc170159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc17573c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1700692d0 0x7fc1700aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc17573c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1700ca6d0 con 0x7fc17015d040 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc17573c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17015d550 con 0x7fc1700692d0 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc17573c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17015d810 con 0x7fc170159170 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc16f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1700692d0 0x7fc1700aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc16f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1700692d0 0x7fc1700aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46728/0 (socket says 172.21.15.45:46728) 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc16f7fe700 1 -- 172.21.15.45:0/2113998003 learned_addr learned my addr 172.21.15.45:0/2113998003 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc16e7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc17015d040 0x7fc170068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50618/0 (socket says 172.21.15.45:50618) 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.495+0000 7fc16effd700 1 --2- 172.21.15.45:0/2113998003 >> v2:172.21.15.144:3300/0 conn(0x7fc170159170 0x7fc170159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16f7fe700 1 -- 172.21.15.45:0/2113998003 >> v2:172.21.15.144:3300/0 conn(0x7fc170159170 msgr2=0x7fc170159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16f7fe700 1 --2- 172.21.15.45:0/2113998003 >> v2:172.21.15.144:3300/0 conn(0x7fc170159170 0x7fc170159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16f7fe700 1 -- 172.21.15.45:0/2113998003 >> v1:172.21.15.45:6789/0 conn(0x7fc17015d040 legacy=0x7fc170068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16f7fe700 1 -- 172.21.15.45:0/2113998003 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17015daf0 con 0x7fc1700692d0 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16dffb700 1 -- 172.21.15.45:0/2113998003 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237504598 0 0) 0x7fc1700ca6d0 con 0x7fc17015d040 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16f7fe700 1 --2- 172.21.15.45:0/2113998003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1700692d0 0x7fc1700aea60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fc170068a90 tx=0x7fc168009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9935507b119a9d40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16dffb700 1 -- 172.21.15.45:0/2113998003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16800e070 con 0x7fc1700692d0 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16dffb700 1 -- 172.21.15.45:0/2113998003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16800ba20 con 0x7fc1700692d0 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.496+0000 7fc16dffb700 1 -- 172.21.15.45:0/2113998003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1680092e0 con 0x7fc1700692d0 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 -- 172.21.15.45:0/2113998003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1700692d0 msgr2=0x7fc1700aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 --2- 172.21.15.45:0/2113998003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1700692d0 0x7fc1700aea60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fc170068a90 tx=0x7fc168009840 comp rx=0 tx=0).stop 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 -- 172.21.15.45:0/2113998003 shutdown_connections 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 --2- 172.21.15.45:0/2113998003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1700692d0 0x7fc1700aea60 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 --2- 172.21.15.45:0/2113998003 >> v2:172.21.15.144:3300/0 conn(0x7fc170159170 0x7fc170159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 -- 172.21.15.45:0/2113998003 >> 172.21.15.45:0/2113998003 conn(0x7fc17006f7c0 msgr2=0x7fc1700d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 -- 172.21.15.45:0/2113998003 shutdown_connections 2023-10-06T05:25:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.497+0000 7fc17573c700 1 -- 172.21.15.45:0/2113998003 wait complete. 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.498+0000 7fc17573c700 1 Processor -- start 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.498+0000 7fc17573c700 1 -- start start 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc17573c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1700692d0 0x7fc170159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc17573c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc170159170 0x7fc17015c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc17573c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015d040 0x7fc1700c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc17573c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700cba00 con 0x7fc170159170 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc17573c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700cc260 con 0x7fc1700692d0 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc17573c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700c9900 con 0x7fc17015d040 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc170159170 0x7fc17015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc170159170 0x7fc17015c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46742/0 (socket says 172.21.15.45:46742) 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 -- 172.21.15.45:0/4100954717 learned_addr learned my addr 172.21.15.45:0/4100954717 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16effd700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1700692d0 0x7fc170159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16f7fe700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015d040 0x7fc1700c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 -- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015d040 msgr2=0x7fc1700c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015d040 0x7fc1700c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 -- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1700692d0 msgr2=0x7fc170159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1700692d0 0x7fc170159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16e7fc700 1 -- 172.21.15.45:0/4100954717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc168009ae0 con 0x7fc170159170 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.499+0000 7fc16f7fe700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015d040 0x7fc1700c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.500+0000 7fc16e7fc700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc170159170 0x7fc17015c690 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c510 tx=0x7fc16400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.500+0000 7fc156ffd700 1 -- 172.21.15.45:0/4100954717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164005a70 con 0x7fc170159170 2023-10-06T05:25:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.500+0000 7fc156ffd700 1 -- 172.21.15.45:0/4100954717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc164005bd0 con 0x7fc170159170 2023-10-06T05:25:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.500+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1700cbc10 con 0x7fc170159170 2023-10-06T05:25:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.500+0000 7fc156ffd700 1 -- 172.21.15.45:0/4100954717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640070d0 con 0x7fc170159170 2023-10-06T05:25:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.500+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17015cbd0 con 0x7fc170159170 2023-10-06T05:25:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.501+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc170063b50 con 0x7fc170159170 2023-10-06T05:25:46.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.501+0000 7fc156ffd700 1 -- 172.21.15.45:0/4100954717 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc164007480 con 0x7fc170159170 2023-10-06T05:25:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.502+0000 7fc156ffd700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc15c066f30 0x7fc15c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.502+0000 7fc156ffd700 1 -- 172.21.15.45:0/4100954717 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc1640847e0 con 0x7fc170159170 2023-10-06T05:25:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.502+0000 7fc16effd700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc15c066f30 0x7fc15c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.503+0000 7fc16effd700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc15c066f30 0x7fc15c0693e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc1700c9e60 tx=0x7fc1600091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.506+0000 7fc156ffd700 1 -- 172.21.15.45:0/4100954717 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc164050c60 con 0x7fc170159170 2023-10-06T05:25:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.756+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1700cc3a0 con 0x7fc170159170 2023-10-06T05:25:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.757+0000 7fc156ffd700 1 -- 172.21.15.45:0/4100954717 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fc164017070 con 0x7fc170159170 2023-10-06T05:25:46.759 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:46.759 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":297,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":9607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2062728889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2062728889},{"type":"v1","addr":"172.21.15.144:6835","nonce":2062728889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":297}],"filesystems":[{"mdsmap":{"epoch":296,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:43.120353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:46.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc15c066f30 msgr2=0x7fc15c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc15c066f30 0x7fc15c0693e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc1700c9e60 tx=0x7fc1600091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc170159170 msgr2=0x7fc17015c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc170159170 0x7fc17015c690 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c510 tx=0x7fc16400c820 comp rx=0 tx=0).stop 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 shutdown_connections 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc15c066f30 0x7fc15c0693e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015d040 0x7fc1700c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1700692d0 0x7fc170159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 --2- 172.21.15.45:0/4100954717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc170159170 0x7fc17015c690 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 >> 172.21.15.45:0/4100954717 conn(0x7fc17006f7c0 msgr2=0x7fc1700cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.759+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 shutdown_connections 2023-10-06T05:25:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.760+0000 7fc17573c700 1 -- 172.21.15.45:0/4100954717 wait complete. 2023-10-06T05:25:46.763 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 297 2023-10-06T05:25:46.775 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2023-10-06T05:25:46.775 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2023-10-06T05:25:46.775 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:46.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:25:46.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:46.775 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:46.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:25:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:25:46.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.970+0000 7f80d7ca6700 1 Processor -- start 2023-10-06T05:25:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.971+0000 7f80d7ca6700 1 -- start start 2023-10-06T05:25:46.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.971+0000 7f80d7ca6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f80d00ccaa0 0x7f80d006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.971+0000 7f80d7ca6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d00cb530 con 0x7f80d0159330 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d7ca6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d0069bd0 con 0x7f80d0158f60 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d7ca6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d0068fc0 con 0x7f80d00ccaa0 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d5241700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f80d0159330 0x7f80d00cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50620/0 (socket says 172.21.15.45:50620) 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d5241700 1 -- 172.21.15.45:0/414774368 learned_addr learned my addr 172.21.15.45:0/414774368 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283991654 0 0) 0x7f80c0001110 con 0x7f80d0159330 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d6243700 1 --2- 172.21.15.45:0/414774368 >> v2:172.21.15.144:3300/0 conn(0x7f80d00ccaa0 0x7f80d006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80bc003570 con 0x7f80d0159330 2023-10-06T05:25:46.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199808016 0 0) 0x7f80bc003570 con 0x7f80d0159330 2023-10-06T05:25:46.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 >> v2:172.21.15.144:3300/0 conn(0x7f80d00ccaa0 msgr2=0x7f80d006a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:46.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 --2- 172.21.15.45:0/414774368 >> v2:172.21.15.144:3300/0 conn(0x7f80d00ccaa0 0x7f80d006a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 >> v1:172.21.15.144:6789/0 conn(0x7f80d0158f60 legacy=0x7f80d00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d00692a0 con 0x7f80d0159330 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f80c0001600 con 0x7f80d0159330 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 >> v1:172.21.15.45:6789/0 conn(0x7f80d0159330 legacy=0x7f80d00cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80bc004810 0x7f80bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80bc0051c0 0x7f80bc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d0068fc0 con 0x7f80bc003a60 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc00c0e0 con 0x7f80bc004810 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.972+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc00c2a0 con 0x7f80bc0051c0 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5a42700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5241700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80bc004810 0x7f80bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d6243700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80bc0051c0 0x7f80bc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5a42700 1 -- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80bc0051c0 msgr2=0x7f80bc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5a42700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80bc0051c0 0x7f80bc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5a42700 1 -- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80bc004810 msgr2=0x7f80bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5a42700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80bc004810 0x7f80bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5a42700 1 -- 172.21.15.45:0/414774368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc003570 con 0x7f80bc003a60 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d6243700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80bc0051c0 0x7f80bc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d6243700 1 -- 172.21.15.45:0/414774368 reap_dead start 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d5a42700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80bc003e30 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f80cc001960 tx=0x7f80cc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d466c217fcb715 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00c5d0 con 0x7f80bc003a60 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc00c750 con 0x7f80bc003a60 2023-10-06T05:25:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.973+0000 7f80d4a40700 1 -- 172.21.15.45:0/414774368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00cb10 con 0x7f80bc003a60 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.974+0000 7f80d7ca6700 1 -- 172.21.15.45:0/414774368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 msgr2=0x7f80bc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.974+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80bc003e30 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f80cc001960 tx=0x7f80cc007d10 comp rx=0 tx=0).stop 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.974+0000 7f80d7ca6700 1 -- 172.21.15.45:0/414774368 shutdown_connections 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.974+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/414774368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80bc003e30 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.974+0000 7f80d7ca6700 1 -- 172.21.15.45:0/414774368 >> 172.21.15.45:0/414774368 conn(0x7f80d00cdf50 msgr2=0x7f80d00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.974+0000 7f80d7ca6700 1 -- 172.21.15.45:0/414774368 shutdown_connections 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.974+0000 7f80d7ca6700 1 -- 172.21.15.45:0/414774368 wait complete. 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.976+0000 7f80d7ca6700 1 Processor -- start 2023-10-06T05:25:46.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.976+0000 7f80d7ca6700 1 -- start start 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d7ca6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80d01549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d7ca6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d016d080 0x7f80d016d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d7ca6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d0152800 0x7f80d0152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d7ca6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d01561b0 con 0x7f80bc003a60 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d7ca6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d016da30 con 0x7f80d0152800 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d7ca6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d016df70 con 0x7f80d016d080 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80d01549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80d01549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46760/0 (socket says 172.21.15.45:46760) 2023-10-06T05:25:46.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 -- 172.21.15.45:0/2513486557 learned_addr learned my addr 172.21.15.45:0/2513486557 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 -- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d016d080 msgr2=0x7f80d016d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5241700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d016d080 0x7f80d016d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d6243700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d0152800 0x7f80d0152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d016d080 0x7f80d016d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 -- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d0152800 msgr2=0x7f80d0152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d0152800 0x7f80d0152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 -- 172.21.15.45:0/2513486557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d016dc90 con 0x7f80bc003a60 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d6243700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d0152800 0x7f80d0152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d5a42700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80d01549b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00c0b0 tx=0x7f80cc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80c5ffb700 1 -- 172.21.15.45:0/2513486557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00cad0 con 0x7f80bc003a60 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80c5ffb700 1 -- 172.21.15.45:0/2513486557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc00ce80 con 0x7f80bc003a60 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d00c50a0 con 0x7f80bc003a60 2023-10-06T05:25:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.977+0000 7f80c5ffb700 1 -- 172.21.15.45:0/2513486557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc006a70 con 0x7f80bc003a60 2023-10-06T05:25:46.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.978+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d00c5660 con 0x7f80bc003a60 2023-10-06T05:25:46.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.979+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d0070c60 con 0x7f80bc003a60 2023-10-06T05:25:46.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.979+0000 7f80c5ffb700 1 -- 172.21.15.45:0/2513486557 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80cc019070 con 0x7f80bc003a60 2023-10-06T05:25:46.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.979+0000 7f80c5ffb700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80b4066f30 0x7f80b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.980+0000 7f80c5ffb700 1 -- 172.21.15.45:0/2513486557 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f80cc086860 con 0x7f80bc003a60 2023-10-06T05:25:46.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.980+0000 7f80d5241700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80b4066f30 0x7f80b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.980+0000 7f80d5241700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80b4066f30 0x7f80b40693e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f80d00cb420 tx=0x7f80c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:46.984+0000 7f80c5ffb700 1 -- 172.21.15.45:0/2513486557 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80cc052da0 con 0x7f80bc003a60 2023-10-06T05:25:47.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.234+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f80d00c34c0 con 0x7f80bc003a60 2023-10-06T05:25:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.905+0000 7f80c5ffb700 1 -- 172.21.15.45:0/2513486557 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v298) v1 ==== 161+0+0 (secure 0 0 0) 0x7f80cc023090 con 0x7f80bc003a60 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80b4066f30 msgr2=0x7f80b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80b4066f30 0x7f80b40693e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f80d00cb420 tx=0x7f80c0009040 comp rx=0 tx=0).stop 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 msgr2=0x7f80d01549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80d01549b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00c0b0 tx=0x7f80cc015000 comp rx=0 tx=0).stop 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 shutdown_connections 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f80b4066f30 0x7f80b40693e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d0152800 0x7f80d0152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc003a60 0x7f80d01549b0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 --2- 172.21.15.45:0/2513486557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d016d080 0x7f80d016d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 >> 172.21.15.45:0/2513486557 conn(0x7f80d00cdf50 msgr2=0x7f80d00c36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 shutdown_connections 2023-10-06T05:25:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:47.909+0000 7f80d7ca6700 1 -- 172.21.15.45:0/2513486557 wait complete. 2023-10-06T05:25:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:47.926 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:25:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.061+0000 7f7712121700 1 Processor -- start 2023-10-06T05:25:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.061+0000 7f7712121700 1 -- start start 2023-10-06T05:25:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.062+0000 7f7712121700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1595d0 0x7f770c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.062+0000 7f7712121700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f770c06b5c0 0x7f770c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.062+0000 7f7712121700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c0c7d00 con 0x7f770c159200 2023-10-06T05:25:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.062+0000 7f7712121700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c0c7780 con 0x7f770c06b5c0 2023-10-06T05:25:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.062+0000 7f7712121700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c06bb40 con 0x7f770c1595d0 2023-10-06T05:25:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1595d0 0x7f770c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770bfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f770c06b5c0 0x7f770c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1595d0 0x7f770c06b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:43012/0 (socket says 172.21.15.45:43012) 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 -- 172.21.15.45:0/1762607660 learned_addr learned my addr 172.21.15.45:0/1762607660 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770b7fe700 1 --1- 172.21.15.45:0/1762607660 >> v1:172.21.15.144:6789/0 conn(0x7f770c159200 0x7f770c0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:59164/0 (socket says 172.21.15.45:59164) 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 -- 172.21.15.45:0/1762607660 >> v2:172.21.15.144:3300/0 conn(0x7f770c06b5c0 msgr2=0x7f770c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 --2- 172.21.15.45:0/1762607660 >> v2:172.21.15.144:3300/0 conn(0x7f770c06b5c0 0x7f770c0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 -- 172.21.15.45:0/1762607660 >> v1:172.21.15.144:6789/0 conn(0x7f770c159200 legacy=0x7f770c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 -- 172.21.15.45:0/1762607660 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c06be20 con 0x7f770c1595d0 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770bfff700 1 --2- 172.21.15.45:0/1762607660 >> v2:172.21.15.144:3300/0 conn(0x7f770c06b5c0 0x7f770c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770affd700 1 --2- 172.21.15.45:0/1762607660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1595d0 0x7f770c06b080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f77000099b0 tx=0x7f7700009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be66d213a7470ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770a7fc700 1 -- 172.21.15.45:0/1762607660 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700009ea0 con 0x7f770c1595d0 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770a7fc700 1 -- 172.21.15.45:0/1762607660 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770000b860 con 0x7f770c1595d0 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.063+0000 7f770a7fc700 1 -- 172.21.15.45:0/1762607660 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000bc00 con 0x7f770c1595d0 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 -- 172.21.15.45:0/1762607660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1595d0 msgr2=0x7f770c06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 --2- 172.21.15.45:0/1762607660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1595d0 0x7f770c06b080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f77000099b0 tx=0x7f7700009250 comp rx=0 tx=0).stop 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 -- 172.21.15.45:0/1762607660 shutdown_connections 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 --2- 172.21.15.45:0/1762607660 >> v2:172.21.15.144:3300/0 conn(0x7f770c06b5c0 0x7f770c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 --2- 172.21.15.45:0/1762607660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1595d0 0x7f770c06b080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 -- 172.21.15.45:0/1762607660 >> 172.21.15.45:0/1762607660 conn(0x7f770c06f090 msgr2=0x7f770c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 -- 172.21.15.45:0/1762607660 shutdown_connections 2023-10-06T05:25:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.064+0000 7f7712121700 1 -- 172.21.15.45:0/1762607660 wait complete. 2023-10-06T05:25:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.065+0000 7f7712121700 1 Processor -- start 2023-10-06T05:25:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.065+0000 7f7712121700 1 -- start start 2023-10-06T05:25:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f7712121700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c06b5c0 0x7f770c16dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f7712121700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159200 0x7f770c1695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f7712121700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1595d0 0x7f770c0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f7712121700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c14ebd0 con 0x7f770c159200 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f7712121700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c16e270 con 0x7f770c1595d0 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f7712121700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0ca960 con 0x7f770c06b5c0 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f770affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159200 0x7f770c1695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f770affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159200 0x7f770c1695d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46762/0 (socket says 172.21.15.45:46762) 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f770affd700 1 -- 172.21.15.45:0/2848686316 learned_addr learned my addr 172.21.15.45:0/2848686316 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f770b7fe700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c06b5c0 0x7f770c16dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.066+0000 7f770bfff700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1595d0 0x7f770c0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f770affd700 1 -- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c06b5c0 msgr2=0x7f770c16dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f770affd700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c06b5c0 0x7f770c16dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f770affd700 1 -- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1595d0 msgr2=0x7f770c0c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f770affd700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1595d0 0x7f770c0c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f770affd700 1 -- 172.21.15.45:0/2848686316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704009a20 con 0x7f770c159200 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f770affd700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159200 0x7f770c1695d0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f7700006fd0 tx=0x7f7700017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f76f37fe700 1 -- 172.21.15.45:0/2848686316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000bbe0 con 0x7f770c159200 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f76f37fe700 1 -- 172.21.15.45:0/2848686316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7700011420 con 0x7f770c159200 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77000094f0 con 0x7f770c159200 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f76f37fe700 1 -- 172.21.15.45:0/2848686316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77000117c0 con 0x7f770c159200 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.067+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c1574d0 con 0x7f770c159200 2023-10-06T05:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.068+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c06e2f0 con 0x7f770c159200 2023-10-06T05:25:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.069+0000 7f76f37fe700 1 -- 172.21.15.45:0/2848686316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7700011b70 con 0x7f770c159200 2023-10-06T05:25:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.069+0000 7f76f37fe700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f76f8066f30 0x7f76f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.069+0000 7f76f37fe700 1 -- 172.21.15.45:0/2848686316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f77000869d0 con 0x7f770c159200 2023-10-06T05:25:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.069+0000 7f770b7fe700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f76f8066f30 0x7f76f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.070+0000 7f770b7fe700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f76f8066f30 0x7f76f80693e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f76fc004fd0 tx=0x7f76fc007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.074+0000 7f76f37fe700 1 -- 172.21.15.45:0/2848686316 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7700052f10 con 0x7f770c159200 2023-10-06T05:25:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.358+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f770c0caaa0 con 0x7f770c159200 2023-10-06T05:25:48.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.914+0000 7f76f37fe700 1 -- 172.21.15.45:0/2848686316 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v299) v1 ==== 101+0+0 (secure 0 0 0) 0x7f7700056530 con 0x7f770c159200 2023-10-06T05:25:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f76f8066f30 msgr2=0x7f76f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f76f8066f30 0x7f76f80693e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f76fc004fd0 tx=0x7f76fc007eb0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159200 msgr2=0x7f770c1695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159200 0x7f770c1695d0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f7700006fd0 tx=0x7f7700017000 comp rx=0 tx=0).stop 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 shutdown_connections 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f76f8066f30 0x7f76f80693e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1595d0 0x7f770c0c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c06b5c0 0x7f770c16dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.917+0000 7f7712121700 1 --2- 172.21.15.45:0/2848686316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159200 0x7f770c1695d0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.918+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 >> 172.21.15.45:0/2848686316 conn(0x7f770c06f090 msgr2=0x7f770c1583e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.918+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 shutdown_connections 2023-10-06T05:25:48.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.918+0000 7f7712121700 1 -- 172.21.15.45:0/2848686316 wait complete. 2023-10-06T05:25:48.921 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:25:48.935 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:25:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.980+0000 7fef3721f040 1 Processor -- start 2023-10-06T05:25:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.980+0000 7fef3721f040 1 -- start start 2023-10-06T05:25:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.980+0000 7fef3721f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75d8a210 0x55cc75daf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef3721f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75e4b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef3721f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75e4cc60 0x55cc75e4b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef3721f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc75cd7980 con 0x55cc75e4c890 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef3721f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc75cd7620 con 0x55cc75e4cc60 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef3721f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc75cce400 con 0x55cc75d8a210 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75e4b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75e4b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46782/0 (socket says 172.21.15.45:46782) 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 -- 172.21.15.45:0/4001663972 learned_addr learned my addr 172.21.15.45:0/4001663972 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef30c3e700 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75e4cc60 0x55cc75e4b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef3043d700 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75d8a210 0x55cc75daf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 -- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75d8a210 msgr2=0x55cc75daf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75d8a210 0x55cc75daf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 -- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75e4cc60 msgr2=0x55cc75e4b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75e4cc60 0x55cc75e4b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 -- 172.21.15.45:0/4001663972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20005db0 con 0x55cc75e4c890 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75e4b130 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fef20008410 tx=0x7fef20016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62b4ccc50d3a0aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2ec3a700 1 -- 172.21.15.45:0/4001663972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20000bc0 con 0x55cc75e4c890 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2ec3a700 1 -- 172.21.15.45:0/4001663972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef20008a50 con 0x55cc75e4c890 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.981+0000 7fef2ec3a700 1 -- 172.21.15.45:0/4001663972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20008e10 con 0x55cc75e4c890 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 -- 172.21.15.45:0/4001663972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 msgr2=0x55cc75e4b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75e4b130 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fef20008410 tx=0x7fef20016000 comp rx=0 tx=0).stop 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 -- 172.21.15.45:0/4001663972 shutdown_connections 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75e4cc60 0x55cc75e4b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75d8a210 0x55cc75daf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 --2- 172.21.15.45:0/4001663972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75e4b130 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 -- 172.21.15.45:0/4001663972 >> 172.21.15.45:0/4001663972 conn(0x55cc75d8d450 msgr2=0x55cc75db8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 -- 172.21.15.45:0/4001663972 shutdown_connections 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.982+0000 7fef3721f040 1 -- 172.21.15.45:0/4001663972 wait complete. 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 Processor -- start 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 -- start start 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75d8a210 0x55cc75e6be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75daec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75e4cc60 0x55cc75e495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc75daf1c0 con 0x55cc75e4c890 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc75e6c3f0 con 0x55cc75d8a210 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.983+0000 7fef3721f040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc75e6c7a0 con 0x55cc75e4cc60 2023-10-06T05:25:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75daec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75daec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46786/0 (socket says 172.21.15.45:46786) 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 learned_addr learned my addr 172.21.15.45:0/694202442 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef3043d700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75d8a210 0x55cc75e6be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef30c3e700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75e4cc60 0x55cc75e495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75e4cc60 msgr2=0x55cc75e495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc75e4cc60 0x55cc75e495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75d8a210 msgr2=0x55cc75e6be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc75d8a210 0x55cc75e6be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc75d001c0 con 0x55cc75e4c890 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75daec80 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fef2000a5a0 tx=0x7fef2001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2d437700 1 -- 172.21.15.45:0/694202442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20029070 con 0x55cc75e4c890 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2d437700 1 -- 172.21.15.45:0/694202442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef20012830 con 0x55cc75e4c890 2023-10-06T05:25:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cc75ce63d0 con 0x55cc75e4c890 2023-10-06T05:25:48.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.984+0000 7fef2d437700 1 -- 172.21.15.45:0/694202442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20012bf0 con 0x55cc75e4c890 2023-10-06T05:25:48.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.985+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc75cd9ae0 con 0x55cc75e4c890 2023-10-06T05:25:48.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.986+0000 7fef2d437700 1 -- 172.21.15.45:0/694202442 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef2001b430 con 0x55cc75e4c890 2023-10-06T05:25:48.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.986+0000 7fef2d437700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef100658c0 0x7fef10067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.987+0000 7fef2d437700 1 -- 172.21.15.45:0/694202442 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fef20086520 con 0x55cc75e4c890 2023-10-06T05:25:48.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.987+0000 7fef3043d700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef100658c0 0x7fef10067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.994 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.987+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cc75e49b10 0x55cc75e6d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.987+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e6d960 con 0x55cc75e49b10 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.987+0000 7fef30c3e700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cc75e49b10 0x55cc75e6d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.987+0000 7fef3043d700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef100658c0 0x7fef10067d70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fef18009b00 tx=0x7fef18005d20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.987+0000 7fef30c3e700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cc75e49b10 0x55cc75e6d420 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.989+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v125'1002 uv976 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.990+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e6dd40 con 0x55cc75e49b10 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.991+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v125'1003 uv1003 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.991+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e6e180 con 0x55cc75e49b10 2023-10-06T05:25:48.997 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:25:48.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.992+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v125'1004 uv1004 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.992+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e810d0 con 0x55cc75e49b10 2023-10-06T05:25:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.994+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'1005 uv1005 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.994+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81480 con 0x55cc75e49b10 2023-10-06T05:25:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.995+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'1006 uv1006 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.995+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e800c0 con 0x55cc75e49b10 2023-10-06T05:25:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.996+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'1007 uv1007 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.996+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e804c0 con 0x55cc75e49b10 2023-10-06T05:25:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.998+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v125'1008 uv982 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.998+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e808c0 con 0x55cc75e49b10 2023-10-06T05:25:49.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.999+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v125'1009 uv1009 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:48.999+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e808c0 con 0x55cc75e49b10 2023-10-06T05:25:49.002 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:25:49.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.000+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v125'1010 uv1010 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.000+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.001+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'1011 uv1011 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.002+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.003+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'1012 uv1012 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.003+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.004+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'1013 uv1013 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.004+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.005+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v125'1014 uv989 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.005+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.007+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'1015 uv1015 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.007+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.010 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:25:49.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.008+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v125'1016 uv1016 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.008+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.009+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'1017 uv1017 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.009+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.010+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'1018 uv1018 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.011+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e80870 con 0x55cc75e49b10 2023-10-06T05:25:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.012+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'1019 uv1019 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.012+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.013+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'1020 uv1020 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.013+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.014+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v125'1021 uv995 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.014+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.016+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v125'1022 uv1022 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.016+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.019 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:25:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.017+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v125'1023 uv1023 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.017+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e82840 con 0x55cc75e49b10 2023-10-06T05:25:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.018+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'1024 uv1024 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.018+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e82840 con 0x55cc75e49b10 2023-10-06T05:25:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.020+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'1025 uv1025 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.020+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.021+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'1026 uv1026 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.021+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.022+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v125'1027 uv1001 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.022+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.024+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v125'1028 uv1028 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.024+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.027 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:25:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.025+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v125'1029 uv1029 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.025+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e82840 con 0x55cc75e49b10 2023-10-06T05:25:49.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.026+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'1030 uv1030 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.026+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e82840 con 0x55cc75e49b10 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.028+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'1031 uv1031 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.028+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e81830 con 0x55cc75e49b10 2023-10-06T05:25:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.029+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'1032 uv1032 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e49b10 2023-10-06T05:25:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.029+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cc75e81d40 0x55cc75e84120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.029+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e84660 con 0x55cc75e81d40 2023-10-06T05:25:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.029+0000 7fef3043d700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cc75e81d40 0x55cc75e84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.030+0000 7fef3043d700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cc75e81d40 0x55cc75e84120 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.031+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'720 uv702 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.031+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e84a10 con 0x55cc75e81d40 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.033+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'721 uv721 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.033+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e84dc0 con 0x55cc75e81d40 2023-10-06T05:25:49.036 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:25:49.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.034+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'722 uv722 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.034+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e85660 con 0x55cc75e81d40 2023-10-06T05:25:49.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.036+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v125'723 uv723 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.036+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e85a10 con 0x55cc75e81d40 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.037+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'724 uv724 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.037+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e85dc0 con 0x55cc75e81d40 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.038+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v125'725 uv707 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.038+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.040+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v125'726 uv726 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.040+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.043 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:25:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.041+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v125'727 uv727 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.041+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.043+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'728 uv728 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.043+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.044+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'729 uv729 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.044+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.045+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v125'730 uv712 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.045+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.047+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v125'731 uv731 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.047+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.050 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:25:49.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.048+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v125'732 uv732 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.048+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e89890 con 0x55cc75e81d40 2023-10-06T05:25:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.050+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v125'733 uv733 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.050+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e89890 con 0x55cc75e81d40 2023-10-06T05:25:49.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.051+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'734 uv734 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.051+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.052+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v125'735 uv718 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.052+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.054+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v125'736 uv736 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.054+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.056 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:25:49.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.055+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v125'737 uv737 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.055+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e87180 con 0x55cc75e81d40 2023-10-06T05:25:49.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.056+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'738 uv738 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.056+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e87180 con 0x55cc75e81d40 2023-10-06T05:25:49.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.057+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'739 uv739 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.058+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e86170 con 0x55cc75e81d40 2023-10-06T05:25:49.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.059+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'740 uv740 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.059+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cc75e86680 0x55cc75e88a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.059+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e88fa0 con 0x55cc75e86680 2023-10-06T05:25:49.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.059+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cc75e86680 0x55cc75e88a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.060+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cc75e86680 0x55cc75e88a60 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.061+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.061+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e89350 con 0x55cc75e86680 2023-10-06T05:25:49.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.062+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v125'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.062+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e89700 con 0x55cc75e86680 2023-10-06T05:25:49.065 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:25:49.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.063+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v125'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.063+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8b000 con 0x55cc75e86680 2023-10-06T05:25:49.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.064+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.065+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8b3b0 con 0x55cc75e86680 2023-10-06T05:25:49.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.066+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.066+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e89ff0 con 0x55cc75e86680 2023-10-06T05:25:49.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.067+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.067+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a3f0 con 0x55cc75e86680 2023-10-06T05:25:49.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.068+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'139 uv138 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.069+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7d0 con 0x55cc75e86680 2023-10-06T05:25:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.070+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'140 uv140 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.070+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7d0 con 0x55cc75e86680 2023-10-06T05:25:49.072 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:25:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.071+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'141 uv141 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.071+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7d0 con 0x55cc75e86680 2023-10-06T05:25:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.072+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'142 uv142 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.072+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7a0 con 0x55cc75e81d40 2023-10-06T05:25:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.073+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.073+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7a0 con 0x55cc75e81d40 2023-10-06T05:25:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.075+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v125'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.075+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7a0 con 0x55cc75e81d40 2023-10-06T05:25:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.076+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v125'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:25:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.076+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7a0 con 0x55cc75e81d40 2023-10-06T05:25:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.077+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.077+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7a0 con 0x55cc75e81d40 2023-10-06T05:25:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.078+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.078+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8a7a0 con 0x55cc75e81d40 2023-10-06T05:25:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.079+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e81d40 2023-10-06T05:25:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.079+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cc75e8aba0 0x55cc75e8dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.079+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.079+0000 7fef30c3e700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cc75e8aba0 0x55cc75e8dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.080+0000 7fef30c3e700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cc75e8aba0 0x55cc75e8dc80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.081+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v125'360 uv354 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.081+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.082+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v125'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.082+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.085 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:25:49.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.083+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v125'362 uv362 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.084+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8f210 con 0x55cc75e8aba0 2023-10-06T05:25:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.086+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'363 uv363 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.086+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8f1d0 con 0x55cc75e8aba0 2023-10-06T05:25:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.087+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.087+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.088+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'365 uv365 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.088+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.089+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v125'366 uv359 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.089+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.090+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v125'367 uv367 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.090+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.093 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:25:49.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.091+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v125'368 uv368 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.092+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e1c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.093+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'369 uv369 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.093+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e8e5c0 con 0x55cc75e8aba0 2023-10-06T05:25:49.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.094+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'370 uv370 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.094+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cc75e8e900 0x55cc75e8ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.094+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e915f0 con 0x55cc75e8e900 2023-10-06T05:25:49.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.094+0000 7fef3043d700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cc75e8e900 0x55cc75e8ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.095+0000 7fef3043d700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cc75e8e900 0x55cc75e8ecf0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.096+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v125'163 uv162 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e8e900 2023-10-06T05:25:49.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.096+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e915f0 con 0x55cc75e8e900 2023-10-06T05:25:49.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.097+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v125'164 uv164 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e8e900 2023-10-06T05:25:49.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.097+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e915f0 con 0x55cc75e8e900 2023-10-06T05:25:49.101 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:25:49.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.100+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v125'165 uv165 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e8e900 2023-10-06T05:25:49.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.100+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e919f0 con 0x55cc75e8e900 2023-10-06T05:25:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.103+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'166 uv166 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e8e900 2023-10-06T05:25:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.103+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e919f0 con 0x55cc75e8e900 2023-10-06T05:25:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.105+0000 7fef3043d700 1 -- 172.21.15.45:0/694202442 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'167 uv167 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef18008bf0 con 0x55cc75e8e900 2023-10-06T05:25:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.105+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e919f0 con 0x55cc75e86680 2023-10-06T05:25:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.107+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.107+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e919f0 con 0x55cc75e86680 2023-10-06T05:25:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.108+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v125'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.108+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e919f0 con 0x55cc75e86680 2023-10-06T05:25:49.111 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:25:49.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.109+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v125'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.109+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e92a00 con 0x55cc75e86680 2023-10-06T05:25:49.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.110+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.110+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e92a00 con 0x55cc75e86680 2023-10-06T05:25:49.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.111+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.111+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e919f0 con 0x55cc75e86680 2023-10-06T05:25:49.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.113+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.113+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc75e91f00 0x55cc75e942e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.113+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94910 con 0x55cc75e91f00 2023-10-06T05:25:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.113+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc75e91f00 0x55cc75e942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.113+0000 7fef2fc3c700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc75e91f00 0x55cc75e942e0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.115+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v125'325 uv318 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.115+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.116+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v125'326 uv326 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.116+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.117+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v125'327 uv327 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.119 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:25:49.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.117+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.118+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'328 uv328 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.118+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.119+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'329 uv329 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.119+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.121+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v125'330 uv324 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.121+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.122+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v125'331 uv331 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.122+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.124 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:25:49.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.123+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v125'332 uv332 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.123+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e94cf0 con 0x55cc75e91f00 2023-10-06T05:25:49.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.124+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'333 uv333 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.124+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e97a60 con 0x55cc75e91f00 2023-10-06T05:25:49.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.125+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'334 uv334 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e91f00 2023-10-06T05:25:49.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.125+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.127+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v125'142 uv140 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.127+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.128+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'143 uv143 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.128+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.129+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v125'144 uv144 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.132 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:25:49.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.129+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.130+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'145 uv145 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.130+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cc75e95af0 0x55cc75e981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.130+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e95af0 2023-10-06T05:25:49.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.130+0000 7fef30c3e700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cc75e95af0 0x55cc75e981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.131+0000 7fef30c3e700 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cc75e95af0 0x55cc75e981b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.132+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e95af0 2023-10-06T05:25:49.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.132+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e95af0 2023-10-06T05:25:49.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.133+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v125'196 uv196 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e95af0 2023-10-06T05:25:49.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.133+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e95af0 2023-10-06T05:25:49.136 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:25:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.135+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v125'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e95af0 2023-10-06T05:25:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.135+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e95af0 2023-10-06T05:25:49.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.136+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e95af0 2023-10-06T05:25:49.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.136+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e95af0 2023-10-06T05:25:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.137+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e95af0 2023-10-06T05:25:49.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.137+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e95af0 2023-10-06T05:25:49.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.138+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e95af0 2023-10-06T05:25:49.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.138+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e8aba0 2023-10-06T05:25:49.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.140+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v125'166 uv165 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.140+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e8aba0 2023-10-06T05:25:49.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.141+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v125'167 uv167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.141+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e8aba0 2023-10-06T05:25:49.144 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:25:49.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.142+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v125'168 uv168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.142+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e8aba0 2023-10-06T05:25:49.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.143+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'169 uv169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.143+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e8aba0 2023-10-06T05:25:49.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.144+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'170 uv170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef28003980 con 0x55cc75e8aba0 2023-10-06T05:25:49.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.144+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.146+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v125'402 uv393 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.146+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.147+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v125'403 uv403 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.147+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.148+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v125'404 uv404 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.150 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:25:49.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.148+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.149+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v125'405 uv405 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.149+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.150+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'406 uv406 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.150+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.151+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v125'407 uv400 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.151+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.152+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v125'408 uv408 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.153+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951e0 con 0x55cc75e86680 2023-10-06T05:25:49.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.154+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v125'409 uv409 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.155 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:25:49.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.154+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951b0 con 0x55cc75e86680 2023-10-06T05:25:49.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.155+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'410 uv410 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.155+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951b0 con 0x55cc75e86680 2023-10-06T05:25:49.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.156+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'411 uv411 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.156+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e951b0 con 0x55cc75e86680 2023-10-06T05:25:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.157+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'412 uv412 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.157+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x55cc75e986f0 con 0x55cc75e86680 2023-10-06T05:25:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef2fc3c700 1 -- 172.21.15.45:0/694202442 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'413 uv413 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef20083d20 con 0x55cc75e86680 2023-10-06T05:25:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cc75e86680 msgr2=0x55cc75e88a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55cc75e86680 0x55cc75e88a60 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cc75e49b10 msgr2=0x55cc75e6d420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55cc75e49b10 0x55cc75e6d420 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc75e91f00 msgr2=0x55cc75e942e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc75e91f00 0x55cc75e942e0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cc75e81d40 msgr2=0x55cc75e84120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55cc75e81d40 0x55cc75e84120 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cc75e95af0 msgr2=0x55cc75e981b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55cc75e95af0 0x55cc75e981b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cc75e8e900 msgr2=0x55cc75e8ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55cc75e8e900 0x55cc75e8ecf0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.158+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 reap_dead start 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cc75e8aba0 msgr2=0x55cc75e8dc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55cc75e8aba0 0x55cc75e8dc80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 reap_dead start 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef30c3e700 1 -- 172.21.15.45:0/694202442 reap_dead start 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef100658c0 msgr2=0x7fef10067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef100658c0 0x7fef10067d70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fef18009b00 tx=0x7fef18005d20 comp rx=0 tx=0).stop 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 msgr2=0x55cc75daec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75daec80 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fef2000a5a0 tx=0x7fef2001b000 comp rx=0 tx=0).stop 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 shutdown_connections 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc75e4c890 0x55cc75daec80 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 --2- 172.21.15.45:0/694202442 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef100658c0 0x7fef10067d70 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 >> 172.21.15.45:0/694202442 conn(0x55cc75d8d450 msgr2=0x55cc75db8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 shutdown_connections 2023-10-06T05:25:49.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.159+0000 7fef3721f040 1 -- 172.21.15.45:0/694202442 wait complete. 2023-10-06T05:25:49.165 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:25:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 Processor -- start 2023-10-06T05:25:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 -- start start 2023-10-06T05:25:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea517e0 0x556f9eafdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea53280 0x556f9eafe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 0x556f9eb04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9e96d980 con 0x556f9ea517e0 2023-10-06T05:25:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9e96d620 con 0x556f9ea53280 2023-10-06T05:25:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.215+0000 7f0312782040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f9e964400 con 0x556f9eae0a50 2023-10-06T05:25:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea517e0 0x556f9eafdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea517e0 0x556f9eafdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46800/0 (socket says 172.21.15.45:46800) 2023-10-06T05:25:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 -- 172.21.15.45:0/4268923034 learned_addr learned my addr 172.21.15.45:0/4268923034 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b19f700 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea53280 0x556f9eafe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030c1a1700 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 0x556f9eb04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 -- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 msgr2=0x556f9eb04260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 0x556f9eb04260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 -- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea53280 msgr2=0x556f9eafe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea53280 0x556f9eafe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 -- 172.21.15.45:0/4268923034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300005cf0 con 0x556f9ea517e0 2023-10-06T05:25:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030b9a0700 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea517e0 0x556f9eafdf50 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009830 tx=0x7f02fc00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e046afe50087fb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030a19d700 1 -- 172.21.15.45:0/4268923034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00f040 con 0x556f9ea517e0 2023-10-06T05:25:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030a19d700 1 -- 172.21.15.45:0/4268923034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc008700 con 0x556f9ea517e0 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.216+0000 7f030a19d700 1 -- 172.21.15.45:0/4268923034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc008ac0 con 0x556f9ea517e0 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 -- 172.21.15.45:0/4268923034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea517e0 msgr2=0x556f9eafdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea517e0 0x556f9eafdf50 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009830 tx=0x7f02fc00c040 comp rx=0 tx=0).stop 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 -- 172.21.15.45:0/4268923034 shutdown_connections 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 0x556f9eb04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea517e0 0x556f9eafdf50 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 --2- 172.21.15.45:0/4268923034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea53280 0x556f9eafe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 -- 172.21.15.45:0/4268923034 >> 172.21.15.45:0/4268923034 conn(0x556f9ea247a0 msgr2=0x556f9eadd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 -- 172.21.15.45:0/4268923034 shutdown_connections 2023-10-06T05:25:49.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.217+0000 7f0312782040 1 -- 172.21.15.45:0/4268923034 wait complete. 2023-10-06T05:25:49.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.218+0000 7f0312782040 1 Processor -- start 2023-10-06T05:25:49.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.218+0000 7f0312782040 1 -- start start 2023-10-06T05:25:49.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.218+0000 7f0312782040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea517e0 0x556f9eafe010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.218+0000 7f0312782040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea53280 0x556f9eadc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f0312782040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 0x556f9eadcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f0312782040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9eaf8060 con 0x556f9ea53280 2023-10-06T05:25:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f0312782040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9eaf8300 con 0x556f9ea517e0 2023-10-06T05:25:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f0312782040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f9eaf85a0 con 0x556f9eae0a50 2023-10-06T05:25:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea53280 0x556f9eadc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea53280 0x556f9eadc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46816/0 (socket says 172.21.15.45:46816) 2023-10-06T05:25:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 -- 172.21.15.45:0/3095992023 learned_addr learned my addr 172.21.15.45:0/3095992023 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b9a0700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea517e0 0x556f9eafe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030c1a1700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 0x556f9eadcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 -- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 msgr2=0x556f9eadcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f9eae0a50 0x556f9eadcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 -- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea517e0 msgr2=0x556f9eafe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f9ea517e0 0x556f9eafe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 -- 172.21.15.45:0/3095992023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc009430 con 0x556f9ea53280 2023-10-06T05:25:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030b19f700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea53280 0x556f9eadc5f0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x556f9ea280f0 tx=0x7f0300015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030899a700 1 -- 172.21.15.45:0/3095992023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300007610 con 0x556f9ea53280 2023-10-06T05:25:49.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030899a700 1 -- 172.21.15.45:0/3095992023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0300007e80 con 0x556f9ea53280 2023-10-06T05:25:49.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556f9e97c3d0 con 0x556f9ea53280 2023-10-06T05:25:49.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.219+0000 7f030899a700 1 -- 172.21.15.45:0/3095992023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300007af0 con 0x556f9ea53280 2023-10-06T05:25:49.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.220+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f9e96fae0 con 0x556f9ea53280 2023-10-06T05:25:49.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.221+0000 7f030899a700 1 -- 172.21.15.45:0/3095992023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0300019040 con 0x556f9ea53280 2023-10-06T05:25:49.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.221+0000 7f030899a700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f02f8065890 0x7f02f8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.222+0000 7f030899a700 1 -- 172.21.15.45:0/3095992023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0300085650 con 0x556f9ea53280 2023-10-06T05:25:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.222+0000 7f030b9a0700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f02f8065890 0x7f02f8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.222+0000 7f0312782040 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f9eb07980 0x556f9eb07d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.222+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x556f9eb08290 con 0x556f9eb07980 2023-10-06T05:25:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.222+0000 7f030c1a1700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f9eb07980 0x556f9eb07d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.222+0000 7f030b9a0700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f02f8065890 0x7f02f8067d40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0097d0 tx=0x7f02fc018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.222+0000 7f030c1a1700 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f9eb07980 0x556f9eb07d50 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f030c1a1700 1 -- 172.21.15.45:0/3095992023 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v125'414 uv414 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f0304005590 con 0x556f9eb07980 2023-10-06T05:25:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f9eb07980 msgr2=0x556f9eb07d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f9eb07980 0x556f9eb07d50 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f02f8065890 msgr2=0x7f02f8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f02f8065890 0x7f02f8067d40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0097d0 tx=0x7f02fc018000 comp rx=0 tx=0).stop 2023-10-06T05:25:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea53280 msgr2=0x556f9eadc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 --2- 172.21.15.45:0/3095992023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f9ea53280 0x556f9eadc5f0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x556f9ea280f0 tx=0x7f0300015000 comp rx=0 tx=0).stop 2023-10-06T05:25:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f030c1a1700 1 -- 172.21.15.45:0/3095992023 reap_dead start 2023-10-06T05:25:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 shutdown_connections 2023-10-06T05:25:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 >> 172.21.15.45:0/3095992023 conn(0x556f9ea247a0 msgr2=0x556f9eadd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 shutdown_connections 2023-10-06T05:25:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.224+0000 7f0312782040 1 -- 172.21.15.45:0/3095992023 wait complete. 2023-10-06T05:25:49.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:25:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.372+0000 7f31400f4700 1 Processor -- start 2023-10-06T05:25:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.372+0000 7f31400f4700 1 -- start start 2023-10-06T05:25:49.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f31400f4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f313806aa70 0x7f31380c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f31400f4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f31380c7050 0x7f313815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f31400f4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313806ae40 0x7f313815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f31400f4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313815cc30 con 0x7f313806aa70 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f31400f4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31380ac620 con 0x7f31380c7050 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f31400f4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3138067d80 con 0x7f313806ae40 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f313de90700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f313806aa70 0x7f31380c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f313de90700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f313806aa70 0x7f31380c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46826/0 (socket says 172.21.15.45:46826) 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f313de90700 1 -- 172.21.15.45:0/293495694 learned_addr learned my addr 172.21.15.45:0/293495694 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.373+0000 7f313e691700 1 --2- 172.21.15.45:0/293495694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313806ae40 0x7f313815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313d68f700 1 --2- 172.21.15.45:0/293495694 >> v2:172.21.15.144:3300/0 conn(0x7f31380c7050 0x7f313815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313de90700 1 -- 172.21.15.45:0/293495694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313806ae40 msgr2=0x7f313815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313de90700 1 --2- 172.21.15.45:0/293495694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313806ae40 0x7f313815c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313de90700 1 -- 172.21.15.45:0/293495694 >> v2:172.21.15.144:3300/0 conn(0x7f31380c7050 msgr2=0x7f313815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313de90700 1 --2- 172.21.15.45:0/293495694 >> v2:172.21.15.144:3300/0 conn(0x7f31380c7050 0x7f313815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313de90700 1 -- 172.21.15.45:0/293495694 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31380c75b0 con 0x7f313806aa70 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313d68f700 1 --2- 172.21.15.45:0/293495694 >> v2:172.21.15.144:3300/0 conn(0x7f31380c7050 0x7f313815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313de90700 1 --2- 172.21.15.45:0/293495694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f31380c6b10 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f31380cd5e0 tx=0x7f3134009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34c82031aa876af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313ce8e700 1 -- 172.21.15.45:0/293495694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134009e70 con 0x7f313806aa70 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313ce8e700 1 -- 172.21.15.45:0/293495694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313400b860 con 0x7f313806aa70 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.374+0000 7f313ce8e700 1 -- 172.21.15.45:0/293495694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313400bc00 con 0x7f313806aa70 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 -- 172.21.15.45:0/293495694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 msgr2=0x7f31380c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 --2- 172.21.15.45:0/293495694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f31380c6b10 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f31380cd5e0 tx=0x7f3134009250 comp rx=0 tx=0).stop 2023-10-06T05:25:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 -- 172.21.15.45:0/293495694 shutdown_connections 2023-10-06T05:25:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 --2- 172.21.15.45:0/293495694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313806ae40 0x7f313815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 --2- 172.21.15.45:0/293495694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f31380c6b10 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 --2- 172.21.15.45:0/293495694 >> v2:172.21.15.144:3300/0 conn(0x7f31380c7050 0x7f313815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 -- 172.21.15.45:0/293495694 >> 172.21.15.45:0/293495694 conn(0x7f31380cc570 msgr2=0x7f31380cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 -- 172.21.15.45:0/293495694 shutdown_connections 2023-10-06T05:25:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.375+0000 7f31400f4700 1 -- 172.21.15.45:0/293495694 wait complete. 2023-10-06T05:25:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.376+0000 7f31400f4700 1 Processor -- start 2023-10-06T05:25:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.376+0000 7f31400f4700 1 -- start start 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f31400f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f313815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f31400f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313806ae40 0x7f31380c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f31400f4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31380c7050 0x7f31380c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f31400f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313815b6d0 con 0x7f313806aa70 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f31400f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313816cbf0 con 0x7f313806ae40 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f31400f4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313815a740 con 0x7f31380c7050 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f313815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f313815a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46834/0 (socket says 172.21.15.45:46834) 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 -- 172.21.15.45:0/2396734208 learned_addr learned my addr 172.21.15.45:0/2396734208 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313d68f700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313806ae40 0x7f31380c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313e691700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31380c7050 0x7f31380c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 -- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31380c7050 msgr2=0x7f31380c0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31380c7050 0x7f31380c0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 -- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313806ae40 msgr2=0x7f31380c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313806ae40 0x7f31380c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313de90700 1 -- 172.21.15.45:0/2396734208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31340094f0 con 0x7f313806aa70 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.377+0000 7f313d68f700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313806ae40 0x7f31380c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.378+0000 7f313de90700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f313815a200 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f3134009290 tx=0x7f3134017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.378+0000 7f312dffb700 1 -- 172.21.15.45:0/2396734208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313400bbe0 con 0x7f313806aa70 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.378+0000 7f312dffb700 1 -- 172.21.15.45:0/2396734208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313401f070 con 0x7f313806aa70 2023-10-06T05:25:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.378+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3138169b80 con 0x7f313806aa70 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.378+0000 7f312dffb700 1 -- 172.21.15.45:0/2396734208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340266b0 con 0x7f313806aa70 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.378+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3138169f10 con 0x7f313806aa70 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.379+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31380bfb90 con 0x7f313806aa70 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.379+0000 7f312dffb700 1 -- 172.21.15.45:0/2396734208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3134026a60 con 0x7f313806aa70 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.380+0000 7f312dffb700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f311c066f30 0x7f311c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.380+0000 7f312dffb700 1 -- 172.21.15.45:0/2396734208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3134021070 con 0x7f313806aa70 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.380+0000 7f313d68f700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f311c066f30 0x7f311c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.380+0000 7f313d68f700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f311c066f30 0x7f311c0693e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3128006f00 tx=0x7f3128008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.384+0000 7f312dffb700 1 -- 172.21.15.45:0/2396734208 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3134053450 con 0x7f313806aa70 2023-10-06T05:25:49.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.635+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f313806e390 con 0x7f313806aa70 2023-10-06T05:25:49.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.917+0000 7f312dffb700 1 -- 172.21.15.45:0/2396734208 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v300) v1 ==== 162+0+0 (secure 0 0 0) 0x7f313402b070 con 0x7f313806aa70 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f311c066f30 msgr2=0x7f311c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f311c066f30 0x7f311c0693e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3128006f00 tx=0x7f3128008040 comp rx=0 tx=0).stop 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 msgr2=0x7f313815a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f313815a200 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f3134009290 tx=0x7f3134017000 comp rx=0 tx=0).stop 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 shutdown_connections 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f311c066f30 0x7f311c0693e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31380c7050 0x7f31380c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f313806aa70 0x7f313815a200 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 --2- 172.21.15.45:0/2396734208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313806ae40 0x7f31380c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 >> 172.21.15.45:0/2396734208 conn(0x7f31380cc570 msgr2=0x7f3138157b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 shutdown_connections 2023-10-06T05:25:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:49.920+0000 7f31400f4700 1 -- 172.21.15.45:0/2396734208 wait complete. 2023-10-06T05:25:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:25:50.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.479+0000 7fd47778f700 1 Processor -- start 2023-10-06T05:25:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.480+0000 7fd47778f700 1 -- start start 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.480+0000 7fd47778f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4700c93c0 0x7fd470156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.480+0000 7fd47778f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd470157030 0x7fd4700cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.480+0000 7fd47778f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47015b000 con 0x7fd4700c8ff0 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.480+0000 7fd47778f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700c9830 con 0x7fd470157030 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.480+0000 7fd47778f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47015b2e0 con 0x7fd4700c93c0 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd475d2c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd470157030 0x7fd4700cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd47552b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd4700c8ff0 0x7fd4700caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50636/0 (socket says 172.21.15.45:50636) 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd47552b700 1 -- 172.21.15.45:0/3856645176 learned_addr learned my addr 172.21.15.45:0/3856645176 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd474d2a700 1 --2- 172.21.15.45:0/3856645176 >> v2:172.21.15.144:3300/0 conn(0x7fd4700c93c0 0x7fd470156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd475d2c700 1 -- 172.21.15.45:0/3856645176 >> v2:172.21.15.144:3300/0 conn(0x7fd4700c93c0 msgr2=0x7fd470156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd475d2c700 1 --2- 172.21.15.45:0/3856645176 >> v2:172.21.15.144:3300/0 conn(0x7fd4700c93c0 0x7fd470156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd475d2c700 1 -- 172.21.15.45:0/3856645176 >> v1:172.21.15.45:6789/0 conn(0x7fd4700c8ff0 legacy=0x7fd4700caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd475d2c700 1 -- 172.21.15.45:0/3856645176 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47015b5c0 con 0x7fd470157030 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd463fff700 1 -- 172.21.15.45:0/3856645176 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383087749 0 0) 0x7fd47015b000 con 0x7fd4700c8ff0 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd475d2c700 1 --2- 172.21.15.45:0/3856645176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd470157030 0x7fd4700cc330 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd47006ed80 tx=0x7fd468009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d492d8d29775591 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd463fff700 1 -- 172.21.15.45:0/3856645176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800e070 con 0x7fd470157030 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd463fff700 1 -- 172.21.15.45:0/3856645176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46800bb60 con 0x7fd470157030 2023-10-06T05:25:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.481+0000 7fd463fff700 1 -- 172.21.15.45:0/3856645176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468012430 con 0x7fd470157030 2023-10-06T05:25:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 -- 172.21.15.45:0/3856645176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd470157030 msgr2=0x7fd4700cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 --2- 172.21.15.45:0/3856645176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd470157030 0x7fd4700cc330 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd47006ed80 tx=0x7fd468009780 comp rx=0 tx=0).stop 2023-10-06T05:25:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 -- 172.21.15.45:0/3856645176 shutdown_connections 2023-10-06T05:25:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 --2- 172.21.15.45:0/3856645176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd470157030 0x7fd4700cc330 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 --2- 172.21.15.45:0/3856645176 >> v2:172.21.15.144:3300/0 conn(0x7fd4700c93c0 0x7fd470156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 -- 172.21.15.45:0/3856645176 >> 172.21.15.45:0/3856645176 conn(0x7fd4700cdf50 msgr2=0x7fd4700ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 -- 172.21.15.45:0/3856645176 shutdown_connections 2023-10-06T05:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.482+0000 7fd47778f700 1 -- 172.21.15.45:0/3856645176 wait complete. 2023-10-06T05:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.483+0000 7fd47778f700 1 Processor -- start 2023-10-06T05:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.483+0000 7fd47778f700 1 -- start start 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47778f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4700c8ff0 0x7fd4700c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47778f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4700c93c0 0x7fd4700ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47778f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470157030 0x7fd470069c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47778f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47006a6f0 con 0x7fd4700c8ff0 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47778f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c63e0 con 0x7fd4700c93c0 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47778f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c5ab0 con 0x7fd470157030 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47552b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4700c8ff0 0x7fd4700c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47552b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4700c8ff0 0x7fd4700c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46854/0 (socket says 172.21.15.45:46854) 2023-10-06T05:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.484+0000 7fd47552b700 1 -- 172.21.15.45:0/483025161 learned_addr learned my addr 172.21.15.45:0/483025161 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd474d2a700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4700c93c0 0x7fd4700ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd475d2c700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470157030 0x7fd470069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47552b700 1 -- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470157030 msgr2=0x7fd470069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47552b700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470157030 0x7fd470069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47552b700 1 -- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4700c93c0 msgr2=0x7fd4700ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47552b700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4700c93c0 0x7fd4700ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47552b700 1 -- 172.21.15.45:0/483025161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009a20 con 0x7fd4700c8ff0 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd474d2a700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4700c93c0 0x7fd4700ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47552b700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4700c8ff0 0x7fd4700c6870 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c5e0 tx=0x7fd46c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd4617fa700 1 -- 172.21.15.45:0/483025161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c005b30 con 0x7fd4700c8ff0 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd4617fa700 1 -- 172.21.15.45:0/483025161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c005c90 con 0x7fd4700c8ff0 2023-10-06T05:25:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700c5d90 con 0x7fd4700c8ff0 2023-10-06T05:25:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd4617fa700 1 -- 172.21.15.45:0/483025161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0072e0 con 0x7fd4700c8ff0 2023-10-06T05:25:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.485+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700c6db0 con 0x7fd4700c8ff0 2023-10-06T05:25:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.486+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4700c66c0 con 0x7fd4700c8ff0 2023-10-06T05:25:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.487+0000 7fd4617fa700 1 -- 172.21.15.45:0/483025161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd46c01b3b0 con 0x7fd4700c8ff0 2023-10-06T05:25:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.487+0000 7fd4617fa700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd454066f30 0x7fd4540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.487+0000 7fd4617fa700 1 -- 172.21.15.45:0/483025161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd46c084750 con 0x7fd4700c8ff0 2023-10-06T05:25:50.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.487+0000 7fd474d2a700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd454066f30 0x7fd4540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.488+0000 7fd474d2a700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd454066f30 0x7fd4540693e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd4700c59e0 tx=0x7fd464008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.492+0000 7fd4617fa700 1 -- 172.21.15.45:0/483025161 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd46c04b5a0 con 0x7fd4700c8ff0 2023-10-06T05:25:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.741+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd47004bb40 con 0x7fd4700c8ff0 2023-10-06T05:25:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.741+0000 7fd4617fa700 1 -- 172.21.15.45:0/483025161 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) v1 ==== 95+0+4609 (secure 0 0 0) 0x7fd46c017020 con 0x7fd4700c8ff0 2023-10-06T05:25:50.743 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:50.743 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":301,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:49.918056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9607":{"gid":9607,"name":"b","rank":0,"incarnation":301,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/2062728889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2062728889},{"type":"v1","addr":"172.21.15.144:6835","nonce":2062728889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd454066f30 msgr2=0x7fd4540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd454066f30 0x7fd4540693e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd4700c59e0 tx=0x7fd464008040 comp rx=0 tx=0).stop 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4700c8ff0 msgr2=0x7fd4700c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4700c8ff0 0x7fd4700c6870 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c5e0 tx=0x7fd46c001070 comp rx=0 tx=0).stop 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 shutdown_connections 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd454066f30 0x7fd4540693e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470157030 0x7fd470069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4700c8ff0 0x7fd4700c6870 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 --2- 172.21.15.45:0/483025161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4700c93c0 0x7fd4700ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 >> 172.21.15.45:0/483025161 conn(0x7fd4700cdf50 msgr2=0x7fd4700c4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 shutdown_connections 2023-10-06T05:25:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:50.744+0000 7fd47778f700 1 -- 172.21.15.45:0/483025161 wait complete. 2023-10-06T05:25:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 301 2023-10-06T05:25:50.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 301, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:25:49.918056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9607': {'gid': 9607, 'name': 'b', 'rank': 0, 'incarnation': 301, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/2062728889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 2062728889}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2062728889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:25:50.759 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9607:up:replay 2023-10-06T05:25:50.759 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:25:54.133 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:25:54.131+0000 7f3b81e3a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:56.163 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:56.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.302+0000 7fba08e66700 1 Processor -- start 2023-10-06T05:25:56.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.303+0000 7fba08e66700 1 -- start start 2023-10-06T05:25:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba08e66700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fba0406e580 0x7fba040b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba08e66700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba04069f70 0x7fba041550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba08e66700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba040aeb00 con 0x7fba0406e1b0 2023-10-06T05:25:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba08e66700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040c88a0 con 0x7fba04069f70 2023-10-06T05:25:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba08e66700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040aede0 con 0x7fba0406e580 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba04069f70 0x7fba041550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba04069f70 0x7fba041550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46860/0 (socket says 172.21.15.45:46860) 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 -- 172.21.15.45:0/1294643000 learned_addr learned my addr 172.21.15.45:0/1294643000 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba01d9b700 1 --2- 172.21.15.45:0/1294643000 >> v2:172.21.15.144:3300/0 conn(0x7fba0406e580 0x7fba040b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 -- 172.21.15.45:0/1294643000 >> v2:172.21.15.144:3300/0 conn(0x7fba0406e580 msgr2=0x7fba040b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 --2- 172.21.15.45:0/1294643000 >> v2:172.21.15.144:3300/0 conn(0x7fba0406e580 0x7fba040b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 -- 172.21.15.45:0/1294643000 >> v1:172.21.15.144:6789/0 conn(0x7fba0406e1b0 legacy=0x7fba040c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 -- 172.21.15.45:0/1294643000 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0406a7a0 con 0x7fba04069f70 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba02d9d700 1 --2- 172.21.15.45:0/1294643000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041550d0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fba040cbcf0 tx=0x7fb9fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e716b36b6de5ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.304+0000 7fba0159a700 1 -- 172.21.15.45:0/1294643000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0092e0 con 0x7fba04069f70 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba0159a700 1 -- 172.21.15.45:0/1294643000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00ba50 con 0x7fba04069f70 2023-10-06T05:25:56.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba0159a700 1 -- 172.21.15.45:0/1294643000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00be10 con 0x7fba04069f70 2023-10-06T05:25:56.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 -- 172.21.15.45:0/1294643000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 msgr2=0x7fba041550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 --2- 172.21.15.45:0/1294643000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041550d0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fba040cbcf0 tx=0x7fb9fc009840 comp rx=0 tx=0).stop 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 -- 172.21.15.45:0/1294643000 shutdown_connections 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 --2- 172.21.15.45:0/1294643000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041550d0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 --2- 172.21.15.45:0/1294643000 >> v2:172.21.15.144:3300/0 conn(0x7fba0406e580 0x7fba040b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 -- 172.21.15.45:0/1294643000 >> 172.21.15.45:0/1294643000 conn(0x7fba04067620 msgr2=0x7fba04067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 -- 172.21.15.45:0/1294643000 shutdown_connections 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.305+0000 7fba08e66700 1 -- 172.21.15.45:0/1294643000 wait complete. 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.307+0000 7fba08e66700 1 Processor -- start 2023-10-06T05:25:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.307+0000 7fba08e66700 1 -- start start 2023-10-06T05:25:56.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba08e66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba08e66700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba0406e1b0 0x7fba040c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba08e66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0406e580 0x7fba04157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba08e66700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba040c19b0 con 0x7fba04069f70 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba08e66700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04140ef0 con 0x7fba0406e580 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba08e66700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba040c5330 con 0x7fba0406e1b0 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46864/0 (socket says 172.21.15.45:46864) 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 -- 172.21.15.45:0/1736164445 learned_addr learned my addr 172.21.15.45:0/1736164445 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba01d9b700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba0406e1b0 0x7fba040c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba02d9d700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0406e580 0x7fba04157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 -- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba0406e1b0 msgr2=0x7fba040c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba0406e1b0 0x7fba040c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 -- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0406e580 msgr2=0x7fba04157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0406e580 0x7fba04157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba0259c700 1 -- 172.21.15.45:0/1736164445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc009ae0 con 0x7fba04069f70 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.308+0000 7fba01d9b700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba0406e1b0 0x7fba040c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.309+0000 7fba0259c700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041409b0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c590 tx=0x7fb9ec00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.309+0000 7fb9fa7fc700 1 -- 172.21.15.45:0/1736164445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec011590 con 0x7fba04069f70 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.309+0000 7fb9fa7fc700 1 -- 172.21.15.45:0/1736164445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec011710 con 0x7fba04069f70 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.309+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba040c0ac0 con 0x7fba04069f70 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.309+0000 7fb9fa7fc700 1 -- 172.21.15.45:0/1736164445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec011ad0 con 0x7fba04069f70 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.309+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba040c1080 con 0x7fba04069f70 2023-10-06T05:25:56.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.310+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0406cec0 con 0x7fba04069f70 2023-10-06T05:25:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.310+0000 7fb9fa7fc700 1 -- 172.21.15.45:0/1736164445 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9ec01b3b0 con 0x7fba04069f70 2023-10-06T05:25:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.311+0000 7fb9fa7fc700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb9e4066f30 0x7fb9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.311+0000 7fb9fa7fc700 1 -- 172.21.15.45:0/1736164445 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb9ec084860 con 0x7fba04069f70 2023-10-06T05:25:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.311+0000 7fba01d9b700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb9e4066f30 0x7fb9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.311+0000 7fba01d9b700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb9e4066f30 0x7fb9e40693e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fba040c4390 tx=0x7fb9f4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.315+0000 7fb9fa7fc700 1 -- 172.21.15.45:0/1736164445 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9ec050ce0 con 0x7fba04069f70 2023-10-06T05:25:56.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.562+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba0406aa80 con 0x7fba04069f70 2023-10-06T05:25:56.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.562+0000 7fb9fa7fc700 1 -- 172.21.15.45:0/1736164445 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) v1 ==== 95+0+4609 (secure 0 0 0) 0x7fb9ec017030 con 0x7fba04069f70 2023-10-06T05:25:56.564 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:25:56.564 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":304,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:52.925392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9607":{"gid":9607,"name":"b","rank":0,"incarnation":301,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6835/2062728889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2062728889},{"type":"v1","addr":"172.21.15.144:6835","nonce":2062728889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb9e4066f30 msgr2=0x7fb9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb9e4066f30 0x7fb9e40693e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fba040c4390 tx=0x7fb9f4008040 comp rx=0 tx=0).stop 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 msgr2=0x7fba041409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041409b0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c590 tx=0x7fb9ec00c8a0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 shutdown_connections 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb9e4066f30 0x7fb9e40693e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0406e580 0x7fba04157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04069f70 0x7fba041409b0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 --2- 172.21.15.45:0/1736164445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba0406e1b0 0x7fba040c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 >> 172.21.15.45:0/1736164445 conn(0x7fba04067620 msgr2=0x7fba040b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 shutdown_connections 2023-10-06T05:25:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.565+0000 7fba08e66700 1 -- 172.21.15.45:0/1736164445 wait complete. 2023-10-06T05:25:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 304 2023-10-06T05:25:56.580 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 304, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:25:52.925392+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9607': {'gid': 9607, 'name': 'b', 'rank': 0, 'incarnation': 301, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.144:6835/2062728889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 2062728889}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2062728889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:25:56.580 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:25:56.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:25:56.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.711+0000 7fbd66764700 1 Processor -- start 2023-10-06T05:25:56.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.711+0000 7fbd66764700 1 -- start start 2023-10-06T05:25:56.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd66764700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd60155cd0 0x7fbd6015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd66764700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbd600ccdc0 0x7fbd600cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd66764700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd600cd700 con 0x7fbd601560a0 2023-10-06T05:25:56.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd66764700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600ae6b0 con 0x7fbd60155cd0 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd66764700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600ae8d0 con 0x7fbd600ccdc0 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd5ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd60155cd0 0x7fbd6015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd5ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd60155cd0 0x7fbd6015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37158/0 (socket says 172.21.15.45:37158) 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd5ffff700 1 -- 172.21.15.45:0/1193784904 learned_addr learned my addr 172.21.15.45:0/1193784904 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd64d01700 1 --2- 172.21.15.45:0/1193784904 >> v2:172.21.15.144:3300/0 conn(0x7fbd600ccdc0 0x7fbd600cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd5ffff700 1 -- 172.21.15.45:0/1193784904 >> v2:172.21.15.144:3300/0 conn(0x7fbd600ccdc0 msgr2=0x7fbd600cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/1193784904 >> v2:172.21.15.144:3300/0 conn(0x7fbd600ccdc0 0x7fbd600cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd5ffff700 1 -- 172.21.15.45:0/1193784904 >> v1:172.21.15.144:6789/0 conn(0x7fbd601560a0 legacy=0x7fbd600adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd5ffff700 1 -- 172.21.15.45:0/1193784904 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600aebb0 con 0x7fbd60155cd0 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.712+0000 7fbd64d01700 1 --2- 172.21.15.45:0/1193784904 >> v2:172.21.15.144:3300/0 conn(0x7fbd600ccdc0 0x7fbd600cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/1193784904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd6015a730 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fbd600c3dd0 tx=0x7fbd50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e774df1db846aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd5effd700 1 -- 172.21.15.45:0/1193784904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000b860 con 0x7fbd60155cd0 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd5effd700 1 -- 172.21.15.45:0/1193784904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5000b9c0 con 0x7fbd60155cd0 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd5effd700 1 -- 172.21.15.45:0/1193784904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000bd60 con 0x7fbd60155cd0 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd66764700 1 -- 172.21.15.45:0/1193784904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 msgr2=0x7fbd6015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd66764700 1 --2- 172.21.15.45:0/1193784904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd6015a730 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fbd600c3dd0 tx=0x7fbd50009250 comp rx=0 tx=0).stop 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd66764700 1 -- 172.21.15.45:0/1193784904 shutdown_connections 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd66764700 1 --2- 172.21.15.45:0/1193784904 >> v2:172.21.15.144:3300/0 conn(0x7fbd600ccdc0 0x7fbd600cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd66764700 1 --2- 172.21.15.45:0/1193784904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd6015a730 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd66764700 1 -- 172.21.15.45:0/1193784904 >> 172.21.15.45:0/1193784904 conn(0x7fbd6006f090 msgr2=0x7fbd6006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.713+0000 7fbd66764700 1 -- 172.21.15.45:0/1193784904 shutdown_connections 2023-10-06T05:25:56.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.714+0000 7fbd66764700 1 -- 172.21.15.45:0/1193784904 wait complete. 2023-10-06T05:25:56.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.715+0000 7fbd66764700 1 Processor -- start 2023-10-06T05:25:56.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.715+0000 7fbd66764700 1 -- start start 2023-10-06T05:25:56.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd66764700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd600ccdc0 0x7fbd6014a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd66764700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd600abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd66764700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd601560a0 0x7fbd600a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd66764700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600aac00 con 0x7fbd60155cd0 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd66764700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6014a7c0 con 0x7fbd600ccdc0 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd66764700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600aae80 con 0x7fbd601560a0 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd600abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd600abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37164/0 (socket says 172.21.15.45:37164) 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 -- 172.21.15.45:0/3761684500 learned_addr learned my addr 172.21.15.45:0/3761684500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd600ccdc0 0x7fbd6014a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd64d01700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd601560a0 0x7fbd600a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 -- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd601560a0 msgr2=0x7fbd600a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd601560a0 0x7fbd600a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 -- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd600ccdc0 msgr2=0x7fbd6014a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd600ccdc0 0x7fbd6014a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 -- 172.21.15.45:0/3761684500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500094f0 con 0x7fbd60155cd0 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd600ccdc0 0x7fbd6014a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd600abf20 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800c4e0 tx=0x7fbd4800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.716+0000 7fbd57fff700 1 -- 172.21.15.45:0/3761684500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011590 con 0x7fbd60155cd0 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.717+0000 7fbd57fff700 1 -- 172.21.15.45:0/3761684500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48011710 con 0x7fbd60155cd0 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.717+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd600ab160 con 0x7fbd60155cd0 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.717+0000 7fbd57fff700 1 -- 172.21.15.45:0/3761684500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011ad0 con 0x7fbd60155cd0 2023-10-06T05:25:56.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.717+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd600ab4f0 con 0x7fbd60155cd0 2023-10-06T05:25:56.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.718+0000 7fbd57fff700 1 -- 172.21.15.45:0/3761684500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd4801b3b0 con 0x7fbd60155cd0 2023-10-06T05:25:56.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.718+0000 7fbd57fff700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd40066f30 0x7fbd400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.719+0000 7fbd57fff700 1 -- 172.21.15.45:0/3761684500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbd48084860 con 0x7fbd60155cd0 2023-10-06T05:25:56.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.719+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd40066f30 0x7fbd400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.719+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd40066f30 0x7fbd400693e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbd600c3dd0 tx=0x7fbd50017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.720+0000 7fbd6fa78b80 1 Processor -- start 2023-10-06T05:25:56.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.720+0000 7fbd6fa78b80 1 -- start start 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.721+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f98b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.721+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f993f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.721+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f994940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.721+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f85e310 con 0x556f4f9944d0 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.721+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f89b130 con 0x556f4f970f40 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.721+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f4f898af0 con 0x556f4f98b450 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f994940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f994940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37170/0 (socket says 172.21.15.45:37170) 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 -- 172.21.15.45:0/1581128185 learned_addr learned my addr 172.21.15.45:0/1581128185 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f993f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f98b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 -- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 msgr2=0x556f4f98b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f98b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 -- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 msgr2=0x556f4f993f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f993f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 -- 172.21.15.45:0/1581128185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f4f69a790 con 0x556f4f9944d0 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd64d01700 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f994940 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007610 tx=0x7fbd58009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eac100589f12bdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd557fa700 1 -- 172.21.15.45:0/1581128185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58007e80 con 0x556f4f9944d0 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd557fa700 1 -- 172.21.15.45:0/1581128185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5800f050 con 0x556f4f9944d0 2023-10-06T05:25:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.722+0000 7fbd557fa700 1 -- 172.21.15.45:0/1581128185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58013790 con 0x556f4f9944d0 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/1581128185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 msgr2=0x556f4f994940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f994940 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007610 tx=0x7fbd58009cb0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/1581128185 shutdown_connections 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f994940 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f98b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/1581128185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f993f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/1581128185 >> 172.21.15.45:0/1581128185 conn(0x556f4f98e880 msgr2=0x556f4f98ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/1581128185 shutdown_connections 2023-10-06T05:25:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.723+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/1581128185 wait complete. 2023-10-06T05:25:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.724+0000 7fbd6fa78b80 1 -- start start 2023-10-06T05:25:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.724+0000 7fbd6fa78b80 1 Processor -- start 2023-10-06T05:25:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f96cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f965240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f9944d0 0x556f4f98a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f8948e0 con 0x556f4f970f40 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f840980 con 0x556f4f9944d0 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f4f840620 con 0x556f4f98b450 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd6fa78b80 10 client.0 _ll_register_callbacks cb 0x556f4f6a3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f96cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f96cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37178/0 (socket says 172.21.15.45:37178) 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 -- 172.21.15.45:0/2463942130 learned_addr learned my addr 172.21.15.45:0/2463942130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd64d01700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f9944d0 0x556f4f98a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f965240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 -- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 msgr2=0x556f4f965240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f965240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 -- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f9944d0 msgr2=0x556f4f98a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f9944d0 0x556f4f98a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 -- 172.21.15.45:0/2463942130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58007340 con 0x556f4f970f40 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f965240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f96cdd0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fbd500090b0 tx=0x7fbd5001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a8ab0c744ca228 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.725+0000 7fbd3e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2463942130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5002b070 con 0x556f4f970f40 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2463942130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbd5001a630 con 0x556f4f970f40 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f4f938d60 con 0x556f4f970f40 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2463942130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd5001f070 con 0x556f4f970f40 2023-10-06T05:25:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 10 client.10182 fetch_fsmap learned FSMap version 304 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) v3 -- 0x556f4f7870a0 con 0x556f4f970f40 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2463942130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f4f938d60 con 0x556f4f970f40 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2463942130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 304) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbd5001aa00 con 0x556f4f970f40 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 10 client.10182 fetch_fsmap finished waiting for FSMap version 304 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 10 client.10182 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 20 client.10182 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f98cbb0 0x556f4f98cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 4 client.10182 mds_command: new command op to 9607 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 --> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f4f915b30 con 0x556f4f98cbb0 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.726+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f98cbb0 0x556f4f98cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.727+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f98cbb0 0x556f4f98cf80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c16410b91b7ed756 server_cookie=c9c3efda9663841c in_seq=0 out_seq=0 2023-10-06T05:25:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.727+0000 7fbd3e7fc700 10 client.10182 ms_handle_connect on v2:172.21.15.144:6834/2062728889 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2463942130 <== mds.0 v2:172.21.15.144:6834/2062728889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556f4f915b30 con 0x556f4f98cbb0 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd3e7fc700 10 client.10182 handle_command_reply: tid=0 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 client.10182 shutdown 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 msgr2=0x556f4f96cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f96cdd0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fbd500090b0 tx=0x7fbd5001f000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 shutdown_connections 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f98cbb0 0x556f4f98cf80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f9944d0 0x556f4f98a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f96cdd0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd3e7fc700 0 client.10182 ms_handle_reset on v2:172.21.15.144:6834/2062728889 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2463942130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f965240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 >> 172.21.15.45:0/2463942130 conn(0x556f4f98e880 msgr2=0x556f4f940c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 shutdown_connections 2023-10-06T05:25:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2463942130 wait complete. 2023-10-06T05:25:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.728+0000 7fbd6fa78b80 20 client.10182 trim_cache size 0 max 16384 2023-10-06T05:25:56.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.746+0000 7fbd6fa78b80 1 Processor -- start 2023-10-06T05:25:56.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.746+0000 7fbd6fa78b80 1 -- start start 2023-10-06T05:25:56.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f93acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f98b450 0x556f4f98b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f9944d0 0x556f4f965a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f72c1e0 con 0x556f4f970f40 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f9acf20 con 0x556f4f98b450 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f4f965310 con 0x556f4f9944d0 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f93acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f93acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37192/0 (socket says 172.21.15.45:37192) 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 -- 172.21.15.45:0/611634687 learned_addr learned my addr 172.21.15.45:0/611634687 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f98b450 0x556f4f98b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd64d01700 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f9944d0 0x556f4f965a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 -- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f9944d0 msgr2=0x556f4f965a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f9944d0 0x556f4f965a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 -- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f98b450 msgr2=0x556f4f98b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f98b450 0x556f4f98b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 -- 172.21.15.45:0/611634687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f4f8633c0 con 0x556f4f970f40 2023-10-06T05:25:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd64d01700 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f9944d0 0x556f4f965a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.747+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f93acd0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b3c0 tx=0x7fbd5002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6500de23a3ee109c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd55ffb700 1 -- 172.21.15.45:0/611634687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5002b070 con 0x556f4f970f40 2023-10-06T05:25:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd55ffb700 1 -- 172.21.15.45:0/611634687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5002c040 con 0x556f4f970f40 2023-10-06T05:25:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd55ffb700 1 -- 172.21.15.45:0/611634687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5001f040 con 0x556f4f970f40 2023-10-06T05:25:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/611634687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 msgr2=0x556f4f93acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f93acd0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b3c0 tx=0x7fbd5002e000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/611634687 shutdown_connections 2023-10-06T05:25:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f9944d0 0x556f4f965a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f970f40 0x556f4f93acd0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/611634687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f98b450 0x556f4f98b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/611634687 >> 172.21.15.45:0/611634687 conn(0x556f4f98e880 msgr2=0x556f4f96e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/611634687 shutdown_connections 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.748+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/611634687 wait complete. 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.749+0000 7fbd6fa78b80 1 -- start start 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.749+0000 7fbd6fa78b80 1 Processor -- start 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f9ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f973e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f974340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f990c60 con 0x556f4f9944d0 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4f9910c0 con 0x556f4f970f40 2023-10-06T05:25:56.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f4f9ef7b0 con 0x556f4f98b450 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 10 client.0 _ll_register_callbacks cb 0x556f4f6a3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f974340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f974340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37194/0 (socket says 172.21.15.45:37194) 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 -- 172.21.15.45:0/2057862056 learned_addr learned my addr 172.21.15.45:0/2057862056 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f9ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd5f7fe700 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f973e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 -- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 msgr2=0x556f4f973e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f973e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 -- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 msgr2=0x556f4f9ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f9ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 -- 172.21.15.45:0/2057862056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f4f7870a0 con 0x556f4f9944d0 2023-10-06T05:25:56.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd64d01700 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f974340 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fbd58015000 tx=0x7fbd58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bf885b3a275181 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd3e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2057862056 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58026070 con 0x556f4f9944d0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2057862056 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbd58013d90 con 0x556f4f9944d0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.751+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd5800f050 con 0x556f4f9944d0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2057862056 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=304) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd58020630 con 0x556f4f9944d0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 10 client.10188 fetch_fsmap learned FSMap version 304 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=304,monmap=2+}) v3 -- 0x556f4f6fb1d0 con 0x556f4f9944d0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2057862056 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5800f050 con 0x556f4f9944d0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2057862056 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 304) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbd58020790 con 0x556f4f9944d0 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 10 client.10188 fetch_fsmap finished waiting for FSMap version 304 2023-10-06T05:25:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 10 client.10188 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:25:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 20 client.10188 populate_metadata read hostname 'smithi045' 2023-10-06T05:25:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f940ee0 0x556f4f9412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 4 client.10188 mds_command: new command op to 9607 tid=0[{"prefix": "status"}] 2023-10-06T05:25:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 --> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f4f8378d0 con 0x556f4f940ee0 2023-10-06T05:25:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.752+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f940ee0 0x556f4f9412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd3e7fc700 10 client.10188 ms_handle_connect on v2:172.21.15.144:6834/2062728889 2023-10-06T05:25:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd5ffff700 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f940ee0 0x556f4f9412b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc796c6ebd9945ab server_cookie=cd7978ec0e011b79 in_seq=0 out_seq=0 2023-10-06T05:25:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd3e7fc700 1 -- 172.21.15.45:0/2057862056 <== mds.0 v2:172.21.15.144:6834/2062728889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x556f4f8378d0 con 0x556f4f940ee0 2023-10-06T05:25:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd3e7fc700 10 client.10188 handle_command_reply: tid=0 2023-10-06T05:25:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 client.10188 shutdown 2023-10-06T05:25:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 msgr2=0x556f4f974340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f974340 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fbd58015000 tx=0x7fbd58019000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 shutdown_connections 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x556f4f940ee0 0x556f4f9412b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f4f9944d0 0x556f4f974340 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f4f970f40 0x556f4f9ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 --2- 172.21.15.45:0/2057862056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f4f98b450 0x556f4f973e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd3e7fc700 0 client.10188 ms_handle_reset on v2:172.21.15.144:6834/2062728889 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 >> 172.21.15.45:0/2057862056 conn(0x556f4f98e880 msgr2=0x556f4f98b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 shutdown_connections 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 1 -- 172.21.15.45:0/2057862056 wait complete. 2023-10-06T05:25:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.753+0000 7fbd6fa78b80 20 client.10188 trim_cache size 0 max 16384 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9607, 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8339773639999999, 2023-10-06T05:25:56.761 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 304, 2023-10-06T05:25:56.762 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 125, 2023-10-06T05:25:56.762 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 125, 2023-10-06T05:25:56.762 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 13.601009775 2023-10-06T05:25:56.762 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd40066f30 msgr2=0x7fbd400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd40066f30 0x7fbd400693e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbd600c3dd0 tx=0x7fbd50017000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 msgr2=0x7fbd600abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd600abf20 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800c4e0 tx=0x7fbd4800c8a0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 shutdown_connections 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd40066f30 0x7fbd400693e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd601560a0 0x7fbd600a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd600ccdc0 0x7fbd6014a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 --2- 172.21.15.45:0/3761684500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd60155cd0 0x7fbd600abf20 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 >> 172.21.15.45:0/3761684500 conn(0x7fbd6006f090 msgr2=0x7fbd600cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 shutdown_connections 2023-10-06T05:25:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.754+0000 7fbd66764700 1 -- 172.21.15.45:0/3761684500 wait complete. 2023-10-06T05:25:56.769 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:25:56.769 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2023-10-06T05:25:56.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:56.900 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:25:56.898+0000 7fdb6559f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T05:25:56.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.899+0000 7f1cd5e7f700 1 Processor -- start 2023-10-06T05:25:56.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.899+0000 7f1cd5e7f700 1 -- start start 2023-10-06T05:25:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1cd5e7f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cd0156ce0 0x7f1cd01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1cd5e7f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1cd015ac10 0x7f1cd00ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1cd5e7f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd006bf70 con 0x7f1cd006bad0 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1cd5e7f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd00b02f0 con 0x7f1cd0156ce0 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1cd5e7f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd015b380 con 0x7f1cd015ac10 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1ccf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cd0156ce0 0x7f1cd01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1ccf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cd0156ce0 0x7f1cd01570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37198/0 (socket says 172.21.15.45:37198) 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/2412808480 learned_addr learned my addr 172.21.15.45:0/2412808480 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1ccffff700 1 --2- 172.21.15.45:0/2412808480 >> v2:172.21.15.144:3300/0 conn(0x7f1cd015ac10 0x7f1cd00ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/2412808480 >> v2:172.21.15.144:3300/0 conn(0x7f1cd015ac10 msgr2=0x7f1cd00ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.900+0000 7f1ccf7fe700 1 --2- 172.21.15.45:0/2412808480 >> v2:172.21.15.144:3300/0 conn(0x7f1cd015ac10 0x7f1cd00ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.901+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/2412808480 >> v1:172.21.15.144:6789/0 conn(0x7f1cd006bad0 legacy=0x7f1cd00afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.901+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/2412808480 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd015b660 con 0x7f1cd0156ce0 2023-10-06T05:25:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.901+0000 7f1ccf7fe700 1 --2- 172.21.15.45:0/2412808480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd0156ce0 0x7f1cd01570b0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0157650 tx=0x7f1cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e68ed8f7c1c5ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.902+0000 7f1cce7fc700 1 -- 172.21.15.45:0/2412808480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0009e70 con 0x7f1cd0156ce0 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.902+0000 7f1cce7fc700 1 -- 172.21.15.45:0/2412808480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc000b860 con 0x7f1cd0156ce0 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.902+0000 7f1cce7fc700 1 -- 172.21.15.45:0/2412808480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000bc00 con 0x7f1cd0156ce0 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/2412808480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd0156ce0 msgr2=0x7f1cd01570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/2412808480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd0156ce0 0x7f1cd01570b0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0157650 tx=0x7f1cc0009250 comp rx=0 tx=0).stop 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/2412808480 shutdown_connections 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/2412808480 >> v2:172.21.15.144:3300/0 conn(0x7f1cd015ac10 0x7f1cd00ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/2412808480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd0156ce0 0x7f1cd01570b0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/2412808480 >> 172.21.15.45:0/2412808480 conn(0x7f1cd006f090 msgr2=0x7f1cd006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/2412808480 shutdown_connections 2023-10-06T05:25:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.903+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/2412808480 wait complete. 2023-10-06T05:25:56.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.904+0000 7f1cd5e7f700 1 Processor -- start 2023-10-06T05:25:56.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.904+0000 7f1cd5e7f700 1 -- start start 2023-10-06T05:25:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cd5e7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd006bad0 0x7f1cd00c0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cd5e7f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cd0156ce0 0x7f1cd00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cd5e7f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cd015ac10 0x7f1cd0154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cd5e7f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0158a80 con 0x7f1cd006bad0 2023-10-06T05:25:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cd5e7f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00c0990 con 0x7f1cd0156ce0 2023-10-06T05:25:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cd5e7f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00b0970 con 0x7f1cd015ac10 2023-10-06T05:25:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd006bad0 0x7f1cd00c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd006bad0 0x7f1cd00c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37210/0 (socket says 172.21.15.45:37210) 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/1414101706 learned_addr learned my addr 172.21.15.45:0/1414101706 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cceffd700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cd0156ce0 0x7f1cd00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccffff700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cd015ac10 0x7f1cd0154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cd015ac10 msgr2=0x7f1cd0154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cd015ac10 0x7f1cd0154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cd0156ce0 msgr2=0x7f1cd00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cd0156ce0 0x7f1cd00b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccf7fe700 1 -- 172.21.15.45:0/1414101706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00094f0 con 0x7f1cd006bad0 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1ccffff700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cd015ac10 0x7f1cd0154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.905+0000 7f1cceffd700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cd0156ce0 0x7f1cd00b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.906+0000 7f1ccf7fe700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd006bad0 0x7f1cd00c0450 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.906+0000 7f1cb77fe700 1 -- 172.21.15.45:0/1414101706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000bbe0 con 0x7f1cd006bad0 2023-10-06T05:25:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.906+0000 7f1cb77fe700 1 -- 172.21.15.45:0/1414101706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0011420 con 0x7f1cd006bad0 2023-10-06T05:25:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.906+0000 7f1cb77fe700 1 -- 172.21.15.45:0/1414101706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00117a0 con 0x7f1cd006bad0 2023-10-06T05:25:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.906+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd01587a0 con 0x7f1cd006bad0 2023-10-06T05:25:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.906+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd00c12a0 con 0x7f1cd006bad0 2023-10-06T05:25:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.907+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd00c0e20 con 0x7f1cd006bad0 2023-10-06T05:25:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.907+0000 7f1cb77fe700 1 -- 172.21.15.45:0/1414101706 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1cc0011b90 con 0x7f1cd006bad0 2023-10-06T05:25:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.908+0000 7f1cb77fe700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cbc066f30 0x7f1cbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.908+0000 7f1cb77fe700 1 -- 172.21.15.45:0/1414101706 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1cc0086d70 con 0x7f1cd006bad0 2023-10-06T05:25:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.908+0000 7f1cceffd700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cbc066f30 0x7f1cbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.909+0000 7f1cceffd700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cbc066f30 0x7f1cbc0693e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00c3ce0 tx=0x7f1cc4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:56.913+0000 7f1cb77fe700 1 -- 172.21.15.45:0/1414101706 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cd00c0e20 con 0x7f1cd006bad0 2023-10-06T05:25:57.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.166+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1cd00c0e20 con 0x7f1cd006bad0 2023-10-06T05:25:57.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.166+0000 7f1cb77fe700 1 -- 172.21.15.45:0/1414101706 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v305) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1cd00c0e20 con 0x7f1cd006bad0 2023-10-06T05:25:57.168 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cbc066f30 msgr2=0x7f1cbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cbc066f30 0x7f1cbc0693e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00c3ce0 tx=0x7f1cc4009040 comp rx=0 tx=0).stop 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd006bad0 msgr2=0x7f1cd00c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd006bad0 0x7f1cd00c0450 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 shutdown_connections 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cbc066f30 0x7f1cbc0693e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cd015ac10 0x7f1cd0154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd006bad0 0x7f1cd00c0450 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 --2- 172.21.15.45:0/1414101706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cd0156ce0 0x7f1cd00b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 >> 172.21.15.45:0/1414101706 conn(0x7f1cd006f090 msgr2=0x7f1cd0068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 shutdown_connections 2023-10-06T05:25:57.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:25:57.169+0000 7f1cd5e7f700 1 -- 172.21.15.45:0/1414101706 wait complete. 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:57.184 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:57.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:57.185 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:25:57.205 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:57.206 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41784sec preferred_lft 41784sec 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:57.207 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:25:57.208 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:57.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:57.209 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:57.277 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:25:57.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:57.278 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:25:57.344 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:25:57.344 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:57.344 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:25:57.344 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:25:57.345 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:57.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:57.345 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:25:57.345 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:25:57.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:25:57.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:25:57.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:25:57.345 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:25:57.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:57.468 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:25:57.489 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:25:57.490 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:25:57.490 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:57.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:57.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:25:57.566 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:25:57.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:57.566 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:57.630 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:57.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:57.630 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:57.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:25:57.769 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:25:57.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:25:57.844 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:25:57.845 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:25:57.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:57.846 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:25:57.910 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:25:57.910 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:25:57.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:25:57.988 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:25:57.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:25:58.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:25:58.051+0000 7fdd9e804540 -1 init, newargv = 0x55657d6f6a70 newargc=15 2023-10-06T05:25:58.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[176716]: starting ceph client 2023-10-06T05:25:58.059 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[176716]: starting fuse 2023-10-06T05:25:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:25:58.062 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:25:58.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:58.062 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:25:58.123 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:25:58.124 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:25:58.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:25:58.124 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:25:58.124 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:25:58.124 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:25:58.124 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:25:58.125 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:25:58.126 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:25:58.264 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.176716.asok 2023-10-06T05:25:58.445 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.176716.asok 2023-10-06T05:25:58.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:58.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.176716.asok status 2023-10-06T05:25:58.620 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:25:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:25:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:25:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:25:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "176716", 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 10203, 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:25:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 10203 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2266688502 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2266688502 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:25:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.10203 192.168.0.1:0/2266688502", 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2266688502", 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 305, 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 125, 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:25:58.623 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:25:58.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:58.631 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:25:58.631 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:58.652 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:25:58.652 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:58.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:58.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:58.726 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:25:58.726 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:25:58.726 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:25:58.726 DEBUG:teuthology.orchestra.run.smithi045:> import json 2023-10-06T05:25:58.726 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2023-10-06T05:25:58.726 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> try: 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2023-10-06T05:25:58.727 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:25:58.769 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:25:59.029 INFO:teuthology.orchestra.run.smithi045.stderr:daemon-helper: command failed with exit status 5 2023-10-06T05:26:04.871 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-10-06T05:26:04.871 INFO:tasks.cephfs.test_damage:Result: EIO on client 2023-10-06T05:26:04.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.012+0000 7f8afcb06700 1 Processor -- start 2023-10-06T05:26:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.012+0000 7f8afcb06700 1 -- start start 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.013+0000 7f8afcb06700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8af8159170 0x7f8af8159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.013+0000 7f8afcb06700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8af80692d0 0x7f8af80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.013+0000 7f8afcb06700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af80ca6d0 con 0x7f8af815d040 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.013+0000 7f8afcb06700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af815d550 con 0x7f8af8159170 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.013+0000 7f8afcb06700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af815d810 con 0x7f8af80692d0 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8af8159170 0x7f8af8159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8af8159170 0x7f8af8159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37220/0 (socket says 172.21.15.45:37220) 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 -- 172.21.15.45:0/2356638706 learned_addr learned my addr 172.21.15.45:0/2356638706 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af6d9d700 1 --2- 172.21.15.45:0/2356638706 >> v2:172.21.15.144:3300/0 conn(0x7f8af80692d0 0x7f8af80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 -- 172.21.15.45:0/2356638706 >> v2:172.21.15.144:3300/0 conn(0x7f8af80692d0 msgr2=0x7f8af80aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 --2- 172.21.15.45:0/2356638706 >> v2:172.21.15.144:3300/0 conn(0x7f8af80692d0 0x7f8af80aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 -- 172.21.15.45:0/2356638706 >> v1:172.21.15.144:6789/0 conn(0x7f8af815d040 legacy=0x7f8af8068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 -- 172.21.15.45:0/2356638706 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af815daf0 con 0x7f8af8159170 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af659c700 1 --2- 172.21.15.45:0/2356638706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159170 0x7f8af8159540 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f8af80c70c0 tx=0x7f8ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7e17864e802166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af559a700 1 -- 172.21.15.45:0/2356638706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8009e70 con 0x7f8af8159170 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af559a700 1 -- 172.21.15.45:0/2356638706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae800b860 con 0x7f8af8159170 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.014+0000 7f8af559a700 1 -- 172.21.15.45:0/2356638706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800bc00 con 0x7f8af8159170 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 -- 172.21.15.45:0/2356638706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159170 msgr2=0x7f8af8159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 --2- 172.21.15.45:0/2356638706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159170 0x7f8af8159540 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f8af80c70c0 tx=0x7f8ae8009250 comp rx=0 tx=0).stop 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 -- 172.21.15.45:0/2356638706 shutdown_connections 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 --2- 172.21.15.45:0/2356638706 >> v2:172.21.15.144:3300/0 conn(0x7f8af80692d0 0x7f8af80aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 --2- 172.21.15.45:0/2356638706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159170 0x7f8af8159540 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 -- 172.21.15.45:0/2356638706 >> 172.21.15.45:0/2356638706 conn(0x7f8af806f7c0 msgr2=0x7f8af80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 -- 172.21.15.45:0/2356638706 shutdown_connections 2023-10-06T05:26:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.015+0000 7f8afcb06700 1 -- 172.21.15.45:0/2356638706 wait complete. 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.016+0000 7f8afcb06700 1 Processor -- start 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.016+0000 7f8afcb06700 1 -- start start 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8afcb06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af80692d0 0x7f8af80c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8afcb06700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8af8159170 0x7f8af816aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8afcb06700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af815d040 0x7f8af815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8afcb06700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af815ce10 con 0x7f8af80692d0 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8afcb06700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af816d080 con 0x7f8af815d040 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8afcb06700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8152320 con 0x7f8af8159170 2023-10-06T05:26:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af80692d0 0x7f8af80c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af80692d0 0x7f8af80c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37224/0 (socket says 172.21.15.45:37224) 2023-10-06T05:26:05.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 -- 172.21.15.45:0/3028674263 learned_addr learned my addr 172.21.15.45:0/3028674263 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af6d9d700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af815d040 0x7f8af815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 -- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8af8159170 msgr2=0x7f8af816aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8af8159170 0x7f8af816aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 -- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af815d040 msgr2=0x7f8af815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af815d040 0x7f8af815a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.017+0000 7f8af659c700 1 -- 172.21.15.45:0/3028674263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80094f0 con 0x7f8af80692d0 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.018+0000 7f8af659c700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af80692d0 0x7f8af80c99a0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f8af80d0ed0 tx=0x7f8ae8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.018+0000 7f8ae67fc700 1 -- 172.21.15.45:0/3028674263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8011420 con 0x7f8af80692d0 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.018+0000 7f8ae67fc700 1 -- 172.21.15.45:0/3028674263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae8011580 con 0x7f8af80692d0 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.018+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af815c2d0 con 0x7f8af80692d0 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.018+0000 7f8ae67fc700 1 -- 172.21.15.45:0/3028674263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8011940 con 0x7f8af80692d0 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.018+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af815c570 con 0x7f8af80692d0 2023-10-06T05:26:05.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.019+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af8063b50 con 0x7f8af80692d0 2023-10-06T05:26:05.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.019+0000 7f8ae67fc700 1 -- 172.21.15.45:0/3028674263 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ae802c3b0 con 0x7f8af80692d0 2023-10-06T05:26:05.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.020+0000 7f8ae67fc700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ad8066f30 0x7f8ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.020+0000 7f8ae67fc700 1 -- 172.21.15.45:0/3028674263 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8ae80868b0 con 0x7f8af80692d0 2023-10-06T05:26:05.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.020+0000 7f8af5d9b700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ad8066f30 0x7f8ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.021+0000 7f8af5d9b700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ad8066f30 0x7f8ad80693e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8af80cd5a0 tx=0x7f8aec007b40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.025+0000 7f8ae67fc700 1 -- 172.21.15.45:0/3028674263 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ae8052d30 con 0x7f8af80692d0 2023-10-06T05:26:05.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.273+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8af815ddd0 con 0x7f8af80692d0 2023-10-06T05:26:05.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.274+0000 7f8ae67fc700 1 -- 172.21.15.45:0/3028674263 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 305 v305) v1 ==== 95+0+4609 (secure 0 0 0) 0x7f8ae8028090 con 0x7f8af80692d0 2023-10-06T05:26:05.278 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:05.278 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":305,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":305,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:25:56.899753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9607":{"gid":9607,"name":"b","rank":0,"incarnation":301,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6835/2062728889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2062728889},{"type":"v1","addr":"172.21.15.144:6835","nonce":2062728889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ad8066f30 msgr2=0x7f8ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ad8066f30 0x7f8ad80693e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8af80cd5a0 tx=0x7f8aec007b40 comp rx=0 tx=0).stop 2023-10-06T05:26:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af80692d0 msgr2=0x7f8af80c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af80692d0 0x7f8af80c99a0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f8af80d0ed0 tx=0x7f8ae8017000 comp rx=0 tx=0).stop 2023-10-06T05:26:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 shutdown_connections 2023-10-06T05:26:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8ad8066f30 0x7f8ad80693e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af815d040 0x7f8af815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af80692d0 0x7f8af80c99a0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 --2- 172.21.15.45:0/3028674263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8af8159170 0x7f8af816aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 >> 172.21.15.45:0/3028674263 conn(0x7f8af806f7c0 msgr2=0x7f8af81585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 shutdown_connections 2023-10-06T05:26:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.276+0000 7f8afcb06700 1 -- 172.21.15.45:0/3028674263 wait complete. 2023-10-06T05:26:05.280 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 305 2023-10-06T05:26:05.291 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2023-10-06T05:26:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.423+0000 7fc953bdd700 1 Processor -- start 2023-10-06T05:26:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.423+0000 7fc953bdd700 1 -- start start 2023-10-06T05:26:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc953bdd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc94c157330 0x7fc94c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc953bdd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc94c0aaae0 0x7fc94c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc953bdd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94c0c3b50 con 0x7fc94c0c90b0 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc953bdd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94c157700 con 0x7fc94c157330 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc953bdd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94c0c98e0 con 0x7fc94c0aaae0 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951979700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc94c157330 0x7fc94c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc95217a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc94c0c90b0 0x7fc94c0c3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57918/0 (socket says 172.21.15.45:57918) 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc95217a700 1 -- 172.21.15.45:0/1020949546 learned_addr learned my addr 172.21.15.45:0/1020949546 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951979700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc94c157330 0x7fc94c0ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37236/0 (socket says 172.21.15.45:37236) 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951178700 1 --2- 172.21.15.45:0/1020949546 >> v2:172.21.15.144:3300/0 conn(0x7fc94c0aaae0 0x7fc94c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951979700 1 -- 172.21.15.45:0/1020949546 >> v2:172.21.15.144:3300/0 conn(0x7fc94c0aaae0 msgr2=0x7fc94c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951979700 1 --2- 172.21.15.45:0/1020949546 >> v2:172.21.15.144:3300/0 conn(0x7fc94c0aaae0 0x7fc94c0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951979700 1 -- 172.21.15.45:0/1020949546 >> v1:172.21.15.45:6789/0 conn(0x7fc94c0c90b0 legacy=0x7fc94c0c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951979700 1 -- 172.21.15.45:0/1020949546 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c0c9bc0 con 0x7fc94c157330 2023-10-06T05:26:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc950977700 1 -- 172.21.15.45:0/1020949546 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447062021 0 0) 0x7fc94c0c3b50 con 0x7fc94c0c90b0 2023-10-06T05:26:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951178700 1 --2- 172.21.15.45:0/1020949546 >> v2:172.21.15.144:3300/0 conn(0x7fc94c0aaae0 0x7fc94c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.424+0000 7fc951979700 1 --2- 172.21.15.45:0/1020949546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0ce8b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0c8960 tx=0x7fc948009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8135002378664331 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc950977700 1 -- 172.21.15.45:0/1020949546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948005c10 con 0x7fc94c157330 2023-10-06T05:26:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc950977700 1 -- 172.21.15.45:0/1020949546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc948005d70 con 0x7fc94c157330 2023-10-06T05:26:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc950977700 1 -- 172.21.15.45:0/1020949546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948006260 con 0x7fc94c157330 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 -- 172.21.15.45:0/1020949546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 msgr2=0x7fc94c0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 --2- 172.21.15.45:0/1020949546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0ce8b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0c8960 tx=0x7fc948009120 comp rx=0 tx=0).stop 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 -- 172.21.15.45:0/1020949546 shutdown_connections 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 --2- 172.21.15.45:0/1020949546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0ce8b0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 --2- 172.21.15.45:0/1020949546 >> v2:172.21.15.144:3300/0 conn(0x7fc94c0aaae0 0x7fc94c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 -- 172.21.15.45:0/1020949546 >> 172.21.15.45:0/1020949546 conn(0x7fc94c067e00 msgr2=0x7fc94c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 -- 172.21.15.45:0/1020949546 shutdown_connections 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.425+0000 7fc953bdd700 1 -- 172.21.15.45:0/1020949546 wait complete. 2023-10-06T05:26:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 Processor -- start 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 -- start start 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc94c0aaae0 0x7fc94c1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc94c0c90b0 0x7fc94c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c0c7010 con 0x7fc94c157330 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c0c5830 con 0x7fc94c0aaae0 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc953bdd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c0c6d30 con 0x7fc94c0c90b0 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.427+0000 7fc95217a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37242/0 (socket says 172.21.15.45:37242) 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 -- 172.21.15.45:0/250334247 learned_addr learned my addr 172.21.15.45:0/250334247 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc951979700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc94c0aaae0 0x7fc94c1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc951178700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc94c0c90b0 0x7fc94c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 -- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc94c0c90b0 msgr2=0x7fc94c0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc94c0c90b0 0x7fc94c0c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 -- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc94c0aaae0 msgr2=0x7fc94c1520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc94c0aaae0 0x7fc94c1520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 -- 172.21.15.45:0/250334247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480093c0 con 0x7fc94c157330 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc951979700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc94c0aaae0 0x7fc94c1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc95217a700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0c5cc0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fc94400cc90 tx=0x7fc944001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc941ffb700 1 -- 172.21.15.45:0/250334247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400cea0 con 0x7fc94c157330 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc941ffb700 1 -- 172.21.15.45:0/250334247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94400ab70 con 0x7fc94c157330 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c0c51e0 con 0x7fc94c157330 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc941ffb700 1 -- 172.21.15.45:0/250334247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9440070f0 con 0x7fc94c157330 2023-10-06T05:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.428+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c0c6a20 con 0x7fc94c157330 2023-10-06T05:26:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.430+0000 7fc941ffb700 1 -- 172.21.15.45:0/250334247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc944007460 con 0x7fc94c157330 2023-10-06T05:26:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.430+0000 7fc941ffb700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc930066f30 0x7fc9300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.430+0000 7fc941ffb700 1 -- 172.21.15.45:0/250334247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc9440857d0 con 0x7fc94c157330 2023-10-06T05:26:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.431+0000 7fc951979700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc930066f30 0x7fc9300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.431+0000 7fc951979700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc930066f30 0x7fc9300693e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc948009160 tx=0x7fc948016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.432+0000 7fc95cef1b80 1 Processor -- start 2023-10-06T05:26:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc95cef1b80 1 -- start start 2023-10-06T05:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c5cfe0 0x557b17c80030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17b4a3b0 con 0x557b17c5cfe0 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17b871d0 con 0x557b17c80570 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b17b84b90 con 0x557b17c774f0 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc951178700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c5cfe0 0x557b17c80030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc951178700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c5cfe0 0x557b17c80030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37244/0 (socket says 172.21.15.45:37244) 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc951178700 1 -- 172.21.15.45:0/2761468238 learned_addr learned my addr 172.21.15.45:0/2761468238 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.433+0000 7fc951979700 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc95217a700 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc951178700 1 -- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 msgr2=0x557b17c778c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc951178700 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c778c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc951178700 1 -- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 msgr2=0x557b17c809e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc951178700 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c809e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc951178700 1 -- 172.21.15.45:0/2761468238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b17986830 con 0x557b17c5cfe0 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc951178700 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c5cfe0 0x557b17c80030 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009b90 tx=0x7fc93c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3487a674999d20b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc936ffd700 1 -- 172.21.15.45:0/2761468238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c010960 con 0x557b17c5cfe0 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc936ffd700 1 -- 172.21.15.45:0/2761468238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c00c050 con 0x557b17c5cfe0 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.434+0000 7fc936ffd700 1 -- 172.21.15.45:0/2761468238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c010cd0 con 0x557b17c5cfe0 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 -- 172.21.15.45:0/2761468238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c5cfe0 msgr2=0x557b17c80030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c5cfe0 0x557b17c80030 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009b90 tx=0x7fc93c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 -- 172.21.15.45:0/2761468238 shutdown_connections 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/2761468238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c5cfe0 0x557b17c80030 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 -- 172.21.15.45:0/2761468238 >> 172.21.15.45:0/2761468238 conn(0x557b17c7a920 msgr2=0x557b17c7acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 -- 172.21.15.45:0/2761468238 shutdown_connections 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 -- 172.21.15.45:0/2761468238 wait complete. 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.435+0000 7fc95cef1b80 1 -- start start 2023-10-06T05:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.436+0000 7fc95cef1b80 1 Processor -- start 2023-10-06T05:26:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.436+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c5cfe0 0x557b17c7a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c30630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c80570 0x557b17c30b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17b80980 con 0x557b17c80570 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17b2ca20 con 0x557b17c5cfe0 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b17b2c6c0 con 0x557b17c774f0 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95cef1b80 10 client.0 _ll_register_callbacks cb 0x557b1798faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c80570 0x557b17c30b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c80570 0x557b17c30b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37246/0 (socket says 172.21.15.45:37246) 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 -- 172.21.15.45:0/3041059001 learned_addr learned my addr 172.21.15.45:0/3041059001 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc951979700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c5cfe0 0x557b17c7a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc951178700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c30630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 -- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 msgr2=0x557b17c30630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c30630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 -- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c5cfe0 msgr2=0x557b17c7a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c5cfe0 0x557b17c7a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 -- 172.21.15.45:0/3041059001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c0097c0 con 0x557b17c80570 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc951979700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c5cfe0 0x557b17c7a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95217a700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c80570 0x557b17c30b70 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc944055690 tx=0x7fc944082e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37185dc5652fa6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc92bfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc92bfff700 1 -- 172.21.15.45:0/3041059001 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94405c930 con 0x557b17c80570 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc92bfff700 1 -- 172.21.15.45:0/3041059001 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc944053d30 con 0x557b17c80570 2023-10-06T05:26:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.437+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b17c24e00 con 0x557b17c80570 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc92bfff700 1 -- 172.21.15.45:0/3041059001 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=305) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc944055090 con 0x557b17c80570 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 10 client.10224 fetch_fsmap learned FSMap version 305 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=305,monmap=2+}) v3 -- 0x557b17a73140 con 0x557b17c80570 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc92bfff700 1 -- 172.21.15.45:0/3041059001 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b17c24e00 con 0x557b17c80570 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc92bfff700 1 -- 172.21.15.45:0/3041059001 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 305) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc94405c4a0 con 0x557b17c80570 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 10 client.10224 fetch_fsmap finished waiting for FSMap version 305 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 10 client.10224 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 20 client.10224 populate_metadata read hostname 'smithi045' 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c78d10 0x557b17c790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 4 client.10224 mds_command: new command op to 9607 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:26:05.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 --> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b17c01bd0 con 0x557b17c78d10 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.438+0000 7fc951979700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c78d10 0x557b17c790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.439+0000 7fc92bfff700 10 client.10224 ms_handle_connect on v2:172.21.15.144:6834/2062728889 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.439+0000 7fc951979700 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c78d10 0x557b17c790e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1564cf1ac51f0667 server_cookie=bd8c5afe9d259a55 in_seq=0 out_seq=0 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.439+0000 7fc92bfff700 1 -- 172.21.15.45:0/3041059001 <== mds.0 v2:172.21.15.144:6834/2062728889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x557b17c01bd0 con 0x557b17c78d10 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.439+0000 7fc92bfff700 10 client.10224 handle_command_reply: tid=0 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.439+0000 7fc95cef1b80 1 client.10224 shutdown 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.439+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c80570 msgr2=0x557b17c30b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c80570 0x557b17c30b70 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc944055690 tx=0x7fc944082e60 comp rx=0 tx=0).stop 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 shutdown_connections 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c78d10 0x557b17c790e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c80570 0x557b17c30b70 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c5cfe0 0x557b17c7a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc92bfff700 0 client.10224 ms_handle_reset on v2:172.21.15.144:6834/2062728889 2023-10-06T05:26:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3041059001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c774f0 0x557b17c30630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 >> 172.21.15.45:0/3041059001 conn(0x557b17c7a920 msgr2=0x557b17c2cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 shutdown_connections 2023-10-06T05:26:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3041059001 wait complete. 2023-10-06T05:26:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.440+0000 7fc95cef1b80 20 client.10224 trim_cache size 0 max 16384 2023-10-06T05:26:05.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.453+0000 7fc95cef1b80 1 Processor -- start 2023-10-06T05:26:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.453+0000 7fc95cef1b80 1 -- start start 2023-10-06T05:26:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c50c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c2cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17a18280 con 0x557b17c774f0 2023-10-06T05:26:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17c26bf0 con 0x557b17c80570 2023-10-06T05:26:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b17c2cb50 con 0x557b17c5cfe0 2023-10-06T05:26:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c2cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c2cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37248/0 (socket says 172.21.15.45:37248) 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 -- 172.21.15.45:0/3819670325 learned_addr learned my addr 172.21.15.45:0/3819670325 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951979700 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c50c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95217a700 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 -- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 msgr2=0x557b17c50c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c50c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 -- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 msgr2=0x557b17c792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 -- 172.21.15.45:0/3819670325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b17b4f460 con 0x557b17c774f0 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951979700 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c50c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc95217a700 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc951178700 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c2cfb0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009b60 tx=0x7fc93c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de912bcb1f63401d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc9377fe700 1 -- 172.21.15.45:0/3819670325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c024070 con 0x557b17c774f0 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.454+0000 7fc9377fe700 1 -- 172.21.15.45:0/3819670325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c000bc0 con 0x557b17c774f0 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc9377fe700 1 -- 172.21.15.45:0/3819670325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c006c90 con 0x557b17c774f0 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3819670325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 msgr2=0x557b17c2cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c2cfb0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009b60 tx=0x7fc93c01c000 comp rx=0 tx=0).stop 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3819670325 shutdown_connections 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17c792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c50c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3819670325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c2cfb0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3819670325 >> 172.21.15.45:0/3819670325 conn(0x557b17c7a920 msgr2=0x557b17c2d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3819670325 shutdown_connections 2023-10-06T05:26:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.455+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3819670325 wait complete. 2023-10-06T05:26:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.456+0000 7fc95cef1b80 1 -- start start 2023-10-06T05:26:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.456+0000 7fc95cef1b80 1 Processor -- start 2023-10-06T05:26:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c51ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c52010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95cef1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17cd5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17c76140 con 0x557b17c774f0 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b17c55e10 con 0x557b17c80570 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95cef1b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b17c545f0 con 0x557b17c5cfe0 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95cef1b80 10 client.0 _ll_register_callbacks cb 0x557b1798faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c52010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c52010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37256/0 (socket says 172.21.15.45:37256) 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 -- 172.21.15.45:0/3244304753 learned_addr learned my addr 172.21.15.45:0/3244304753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951979700 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c51ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc95217a700 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17cd5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 -- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 msgr2=0x557b17c51ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c51ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 -- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 msgr2=0x557b17cd5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17cd5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.457+0000 7fc951178700 1 -- 172.21.15.45:0/3244304753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b17986830 con 0x557b17c774f0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc951178700 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c52010 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fc93c002000 tx=0x7fc93c021050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65874b299495b42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc92bfff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc92bfff700 1 -- 172.21.15.45:0/3244304753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c024050 con 0x557b17c774f0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc92bfff700 1 -- 172.21.15.45:0/3244304753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc93c015680 con 0x557b17c774f0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc93c00c050 con 0x557b17c774f0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc92bfff700 1 -- 172.21.15.45:0/3244304753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=305) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc93c006af0 con 0x557b17c774f0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 10 client.10230 fetch_fsmap learned FSMap version 305 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=305,monmap=2+}) v3 -- 0x557b179e7270 con 0x557b17c774f0 2023-10-06T05:26:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc92bfff700 1 -- 172.21.15.45:0/3244304753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c00c050 con 0x557b17c774f0 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc92bfff700 1 -- 172.21.15.45:0/3244304753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 305) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc93c01c070 con 0x557b17c774f0 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 10 client.10230 fetch_fsmap finished waiting for FSMap version 305 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 10 client.10230 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 20 client.10230 populate_metadata read hostname 'smithi045' 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c7c5b0 0x557b17c7c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 4 client.10230 mds_command: new command op to 9607 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 --> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x557b17b23970 con 0x557b17c7c5b0 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.458+0000 7fc951979700 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c7c5b0 0x557b17c7c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc92bfff700 10 client.10230 ms_handle_connect on v2:172.21.15.144:6834/2062728889 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc951979700 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c7c5b0 0x557b17c7c980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0736b0d9128a32e server_cookie=b77d6c68b8bddac2 in_seq=0 out_seq=0 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc92bfff700 1 -- 172.21.15.45:0/3244304753 <== mds.0 v2:172.21.15.144:6834/2062728889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+194 (crc 0 0 0) 0x557b17b23970 con 0x557b17c7c5b0 2023-10-06T05:26:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc92bfff700 10 client.10230 handle_command_reply: tid=0 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 client.10230 shutdown 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 msgr2=0x557b17c52010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c52010 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fc93c002000 tx=0x7fc93c021050 comp rx=0 tx=0).stop 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 shutdown_connections 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:6834/2062728889,v1:172.21.15.144:6835/2062728889] conn(0x557b17c7c5b0 0x557b17c7c980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b17c80570 0x557b17cd5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b17c5cfe0 0x557b17c51ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 --2- 172.21.15.45:0/3244304753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b17c774f0 0x557b17c52010 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc92bfff700 0 client.10230 ms_handle_reset on v2:172.21.15.144:6834/2062728889 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 >> 172.21.15.45:0/3244304753 conn(0x557b17c7a920 msgr2=0x557b17c2d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 shutdown_connections 2023-10-06T05:26:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 1 -- 172.21.15.45:0/3244304753 wait complete. 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.459+0000 7fc95cef1b80 20 client.10230 trim_cache size 0 max 16384 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "dentry", 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 1380817490, 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1, 2023-10-06T05:26:05.468 INFO:teuthology.orchestra.run.smithi045.stdout: "frag": "*", 2023-10-06T05:26:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "subdir", 2023-10-06T05:26:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_id": "head", 2023-10-06T05:26:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/subdir" 2023-10-06T05:26:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:26:05.469 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:26:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc930066f30 msgr2=0x7fc9300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc930066f30 0x7fc9300693e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc948009160 tx=0x7fc948016000 comp rx=0 tx=0).stop 2023-10-06T05:26:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 msgr2=0x7fc94c0c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0c5cc0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fc94400cc90 tx=0x7fc944001a80 comp rx=0 tx=0).stop 2023-10-06T05:26:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 shutdown_connections 2023-10-06T05:26:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc930066f30 0x7fc9300693e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc94c157330 0x7fc94c0c5cc0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc94c0aaae0 0x7fc94c1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 --2- 172.21.15.45:0/250334247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc94c0c90b0 0x7fc94c0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 >> 172.21.15.45:0/250334247 conn(0x7fc94c067e00 msgr2=0x7fc94c06cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 shutdown_connections 2023-10-06T05:26:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.460+0000 7fc953bdd700 1 -- 172.21.15.45:0/250334247 wait complete. 2023-10-06T05:26:05.475 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2023-10-06T05:26:05.475 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:26:05.475 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:05.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:05.511 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:26:05.512 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:26:05.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:05.512 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:05.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.753+0000 7efd301a4700 1 Processor -- start 2023-10-06T05:26:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.753+0000 7efd301a4700 1 -- start start 2023-10-06T05:26:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd301a4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efd280aed00 0x7efd280cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd301a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280c9e60 0x7efd280afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd301a4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280cb3e0 con 0x7efd280af0d0 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd301a4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd280cb990 con 0x7efd280aed00 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd301a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd280cbe20 con 0x7efd280c9e60 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2d73f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efd280af0d0 0x7efd280c9750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57932/0 (socket says 172.21.15.45:57932) 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2d73f700 1 -- 172.21.15.45:0/1617862222 learned_addr learned my addr 172.21.15.45:0/1617862222 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2df40700 1 --2- 172.21.15.45:0/1617862222 >> v2:172.21.15.144:3300/0 conn(0x7efd280aed00 0x7efd280cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2e741700 1 --2- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280c9e60 0x7efd280afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2cf3e700 1 -- 172.21.15.45:0/1617862222 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761248170 0 0) 0x7efd18001110 con 0x7efd280af0d0 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2cf3e700 1 -- 172.21.15.45:0/1617862222 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd14003570 con 0x7efd280af0d0 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2df40700 1 -- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280c9e60 msgr2=0x7efd280afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2cf3e700 1 -- 172.21.15.45:0/1617862222 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1390459464 0 0) 0x7efd14003570 con 0x7efd280af0d0 2023-10-06T05:26:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2df40700 1 --2- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280c9e60 0x7efd280afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2df40700 1 -- 172.21.15.45:0/1617862222 >> v1:172.21.15.45:6789/0 conn(0x7efd280af0d0 legacy=0x7efd280c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2df40700 1 -- 172.21.15.45:0/1617862222 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280c9190 con 0x7efd280aed00 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.754+0000 7efd2e741700 1 --2- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280c9e60 0x7efd280afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd2df40700 1 --2- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280aed00 0x7efd280cc5d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd24009910 tx=0x7efd240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910e55e2441a1b4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd2cf3e700 1 -- 172.21.15.45:0/1617862222 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18001600 con 0x7efd280aed00 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd2cf3e700 1 -- 172.21.15.45:0/1617862222 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24011420 con 0x7efd280aed00 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd2cf3e700 1 -- 172.21.15.45:0/1617862222 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd240117c0 con 0x7efd280aed00 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd301a4700 1 -- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280aed00 msgr2=0x7efd280cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd301a4700 1 --2- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280aed00 0x7efd280cc5d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd24009910 tx=0x7efd240091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd301a4700 1 -- 172.21.15.45:0/1617862222 shutdown_connections 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd301a4700 1 --2- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280c9e60 0x7efd280afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.755+0000 7efd301a4700 1 --2- 172.21.15.45:0/1617862222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280aed00 0x7efd280cc5d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.756+0000 7efd301a4700 1 -- 172.21.15.45:0/1617862222 >> 172.21.15.45:0/1617862222 conn(0x7efd28069200 msgr2=0x7efd280cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.756+0000 7efd301a4700 1 -- 172.21.15.45:0/1617862222 shutdown_connections 2023-10-06T05:26:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.756+0000 7efd301a4700 1 -- 172.21.15.45:0/1617862222 wait complete. 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.757+0000 7efd301a4700 1 Processor -- start 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.757+0000 7efd301a4700 1 -- start start 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd301a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd280aed00 0x7efd2814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd301a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280af0d0 0x7efd28151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd301a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280c9e60 0x7efd2814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd301a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281652f0 con 0x7efd280aed00 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd301a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2814e6b0 con 0x7efd280c9e60 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd301a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd28154320 con 0x7efd280af0d0 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd280aed00 0x7efd2814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd280aed00 0x7efd2814d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37260/0 (socket says 172.21.15.45:37260) 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 -- 172.21.15.45:0/22058378 learned_addr learned my addr 172.21.15.45:0/22058378 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2e741700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280c9e60 0x7efd2814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2d73f700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280af0d0 0x7efd28151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 -- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280af0d0 msgr2=0x7efd28151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280af0d0 0x7efd28151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 -- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280c9e60 msgr2=0x7efd2814f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280c9e60 0x7efd2814f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 -- 172.21.15.45:0/22058378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd200099e0 con 0x7efd280aed00 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd2df40700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd280aed00 0x7efd2814d2d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7efd2806c490 tx=0x7efd24016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.758+0000 7efd1dffb700 1 -- 172.21.15.45:0/22058378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2401bc40 con 0x7efd280aed00 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.759+0000 7efd1dffb700 1 -- 172.21.15.45:0/22058378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2401bda0 con 0x7efd280aed00 2023-10-06T05:26:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.759+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd24009420 con 0x7efd280aed00 2023-10-06T05:26:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.759+0000 7efd1dffb700 1 -- 172.21.15.45:0/22058378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2401c660 con 0x7efd280aed00 2023-10-06T05:26:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.759+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd281545a0 con 0x7efd280aed00 2023-10-06T05:26:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.760+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2806dfd0 con 0x7efd280aed00 2023-10-06T05:26:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.760+0000 7efd1dffb700 1 -- 172.21.15.45:0/22058378 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd2401ca90 con 0x7efd280aed00 2023-10-06T05:26:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.761+0000 7efd1dffb700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efd0c066f30 0x7efd0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.761+0000 7efd1dffb700 1 -- 172.21.15.45:0/22058378 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd24086a00 con 0x7efd280aed00 2023-10-06T05:26:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.761+0000 7efd2d73f700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efd0c066f30 0x7efd0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.761+0000 7efd2d73f700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efd0c066f30 0x7efd0c0693e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efd28168960 tx=0x7efd18009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:05.765+0000 7efd1dffb700 1 -- 172.21.15.45:0/22058378 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd24052e80 con 0x7efd280aed00 2023-10-06T05:26:06.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.020+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efd280c9470 con 0x7efd280aed00 2023-10-06T05:26:06.023 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:06.021+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:06.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.024+0000 7efd1dffb700 1 -- 172.21.15.45:0/22058378 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v306) v1 ==== 161+0+0 (secure 0 0 0) 0x7efd24021090 con 0x7efd280aed00 2023-10-06T05:26:06.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efd0c066f30 msgr2=0x7efd0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efd0c066f30 0x7efd0c0693e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efd28168960 tx=0x7efd18009040 comp rx=0 tx=0).stop 2023-10-06T05:26:06.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd280aed00 msgr2=0x7efd2814d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd280aed00 0x7efd2814d2d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7efd2806c490 tx=0x7efd24016000 comp rx=0 tx=0).stop 2023-10-06T05:26:06.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 shutdown_connections 2023-10-06T05:26:06.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efd0c066f30 0x7efd0c0693e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd280c9e60 0x7efd2814f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd280aed00 0x7efd2814d2d0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 --2- 172.21.15.45:0/22058378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd280af0d0 0x7efd28151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 >> 172.21.15.45:0/22058378 conn(0x7efd28069200 msgr2=0x7efd280c2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:06.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 shutdown_connections 2023-10-06T05:26:06.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.027+0000 7efd301a4700 1 -- 172.21.15.45:0/22058378 wait complete. 2023-10-06T05:26:06.030 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:06.031 INFO:tasks.ceph.mds.b.smithi144.stderr: -597> 2023-10-06T05:25:54.131+0000 7f3b81e3a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:06.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:26:06.054 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:26:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.173+0000 7fec72050700 1 Processor -- start 2023-10-06T05:26:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.173+0000 7fec72050700 1 -- start start 2023-10-06T05:26:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.174+0000 7fec72050700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec6c1595d0 0x7fec6c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.174+0000 7fec72050700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c06b5c0 0x7fec6c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.174+0000 7fec72050700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0c7d00 con 0x7fec6c159200 2023-10-06T05:26:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.174+0000 7fec72050700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0c7780 con 0x7fec6c1595d0 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.174+0000 7fec72050700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c06bb40 con 0x7fec6c06b5c0 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.174+0000 7fec6b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fec6c159200 0x7fec6c0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57946/0 (socket says 172.21.15.45:57946) 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.174+0000 7fec6b7fe700 1 -- 172.21.15.45:0/2214219262 learned_addr learned my addr 172.21.15.45:0/2214219262 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6affd700 1 --2- 172.21.15.45:0/2214219262 >> v2:172.21.15.144:3300/0 conn(0x7fec6c1595d0 0x7fec6c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6bfff700 1 --2- 172.21.15.45:0/2214219262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c06b5c0 0x7fec6c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6a7fc700 1 -- 172.21.15.45:0/2214219262 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013096339 0 0) 0x7fec6c0c7d00 con 0x7fec6c159200 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6a7fc700 1 -- 172.21.15.45:0/2214219262 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec54003570 con 0x7fec6c159200 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6bfff700 1 -- 172.21.15.45:0/2214219262 >> v2:172.21.15.144:3300/0 conn(0x7fec6c1595d0 msgr2=0x7fec6c06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6bfff700 1 --2- 172.21.15.45:0/2214219262 >> v2:172.21.15.144:3300/0 conn(0x7fec6c1595d0 0x7fec6c06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6a7fc700 1 -- 172.21.15.45:0/2214219262 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 822737101 0 0) 0x7fec54003570 con 0x7fec6c159200 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6bfff700 1 -- 172.21.15.45:0/2214219262 >> v1:172.21.15.45:6789/0 conn(0x7fec6c159200 legacy=0x7fec6c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6bfff700 1 -- 172.21.15.45:0/2214219262 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c06be20 con 0x7fec6c06b5c0 2023-10-06T05:26:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6bfff700 1 --2- 172.21.15.45:0/2214219262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c06b5c0 0x7fec6c0ccdc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec64009e60 tx=0x7fec64009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbdf3c122a47f4d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6a7fc700 1 -- 172.21.15.45:0/2214219262 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400e070 con 0x7fec6c06b5c0 2023-10-06T05:26:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6a7fc700 1 -- 172.21.15.45:0/2214219262 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6400b7a0 con 0x7fec6c06b5c0 2023-10-06T05:26:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.175+0000 7fec6a7fc700 1 -- 172.21.15.45:0/2214219262 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c001470 con 0x7fec6c06b5c0 2023-10-06T05:26:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 -- 172.21.15.45:0/2214219262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c06b5c0 msgr2=0x7fec6c0ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 --2- 172.21.15.45:0/2214219262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c06b5c0 0x7fec6c0ccdc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec64009e60 tx=0x7fec64009780 comp rx=0 tx=0).stop 2023-10-06T05:26:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 -- 172.21.15.45:0/2214219262 shutdown_connections 2023-10-06T05:26:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 --2- 172.21.15.45:0/2214219262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c06b5c0 0x7fec6c0ccdc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 --2- 172.21.15.45:0/2214219262 >> v2:172.21.15.144:3300/0 conn(0x7fec6c1595d0 0x7fec6c06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 -- 172.21.15.45:0/2214219262 >> 172.21.15.45:0/2214219262 conn(0x7fec6c06f090 msgr2=0x7fec6c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 -- 172.21.15.45:0/2214219262 shutdown_connections 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.176+0000 7fec72050700 1 -- 172.21.15.45:0/2214219262 wait complete. 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.177+0000 7fec72050700 1 Processor -- start 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec72050700 1 -- start start 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec72050700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6c06b5c0 0x7fec6c0c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec72050700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6c159200 0x7fec6c0c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec72050700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c1595d0 0x7fec6c0c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec72050700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0cb680 con 0x7fec6c06b5c0 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec72050700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0caa70 con 0x7fec6c159200 2023-10-06T05:26:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec72050700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0cad50 con 0x7fec6c1595d0 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6c06b5c0 0x7fec6c0c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec6b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6c06b5c0 0x7fec6c0c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37270/0 (socket says 172.21.15.45:37270) 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec6b7fe700 1 -- 172.21.15.45:0/778995514 learned_addr learned my addr 172.21.15.45:0/778995514 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec6bfff700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c1595d0 0x7fec6c0c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.178+0000 7fec6affd700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6c159200 0x7fec6c0c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec6b7fe700 1 -- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c1595d0 msgr2=0x7fec6c0c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec6b7fe700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c1595d0 0x7fec6c0c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec6b7fe700 1 -- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6c159200 msgr2=0x7fec6c0c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec6b7fe700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6c159200 0x7fec6c0c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec6b7fe700 1 -- 172.21.15.45:0/778995514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64009a20 con 0x7fec6c06b5c0 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec6affd700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6c159200 0x7fec6c0c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec6b7fe700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6c06b5c0 0x7fec6c0c3620 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00cb90 tx=0x7fec5c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec537fe700 1 -- 172.21.15.45:0/778995514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c007360 con 0x7fec6c06b5c0 2023-10-06T05:26:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec537fe700 1 -- 172.21.15.45:0/778995514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c0074e0 con 0x7fec6c06b5c0 2023-10-06T05:26:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c0cb030 con 0x7fec6c06b5c0 2023-10-06T05:26:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec537fe700 1 -- 172.21.15.45:0/778995514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004830 con 0x7fec6c06b5c0 2023-10-06T05:26:06.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.179+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c0b11a0 con 0x7fec6c06b5c0 2023-10-06T05:26:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.180+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c06e2f0 con 0x7fec6c06b5c0 2023-10-06T05:26:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.181+0000 7fec537fe700 1 -- 172.21.15.45:0/778995514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec5c004bc0 con 0x7fec6c06b5c0 2023-10-06T05:26:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.181+0000 7fec537fe700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec58066f30 0x7fec580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.181+0000 7fec537fe700 1 -- 172.21.15.45:0/778995514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7fec5c084bd0 con 0x7fec6c06b5c0 2023-10-06T05:26:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.181+0000 7fec6affd700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec58066f30 0x7fec580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.182+0000 7fec6affd700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec58066f30 0x7fec580693e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fec6c1581d0 tx=0x7fec60008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.186+0000 7fec537fe700 1 -- 172.21.15.45:0/778995514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec5c051070 con 0x7fec6c06b5c0 2023-10-06T05:26:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:06.474+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fec6c0cb960 con 0x7fec6c06b5c0 2023-10-06T05:26:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.032+0000 7fec537fe700 1 -- 172.21.15.45:0/778995514 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v307) v1 ==== 107+0+0 (secure 0 0 0) 0x7fec5c018070 con 0x7fec6c06b5c0 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec58066f30 msgr2=0x7fec580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec58066f30 0x7fec580693e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fec6c1581d0 tx=0x7fec60008040 comp rx=0 tx=0).stop 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6c06b5c0 msgr2=0x7fec6c0c3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6c06b5c0 0x7fec6c0c3620 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00cb90 tx=0x7fec5c001b50 comp rx=0 tx=0).stop 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 shutdown_connections 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fec58066f30 0x7fec580693e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec6c1595d0 0x7fec6c0c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6c06b5c0 0x7fec6c0c3620 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 --2- 172.21.15.45:0/778995514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6c159200 0x7fec6c0c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 >> 172.21.15.45:0/778995514 conn(0x7fec6c06f090 msgr2=0x7fec6c0af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 shutdown_connections 2023-10-06T05:26:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.035+0000 7fec72050700 1 -- 172.21.15.45:0/778995514 wait complete. 2023-10-06T05:26:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:26:07.052 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:26:07.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.097+0000 7f92878c9040 1 Processor -- start 2023-10-06T05:26:07.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f92878c9040 1 -- start start 2023-10-06T05:26:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f92878c9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a2904f920 0x557a290961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f92878c9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a29111890 0x557a29097530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f92878c9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29097a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f92878c9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a28ff8320 con 0x557a29111890 2023-10-06T05:26:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f92878c9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a28f9c980 con 0x557a29111c60 2023-10-06T05:26:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f92878c9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a28f9c620 con 0x557a2904f920 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f927bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a29111890 0x557a29097530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f927bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a29111890 0x557a29097530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54782/0 (socket says 172.21.15.45:54782) 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.098+0000 7f927bfff700 1 -- 172.21.15.45:0/2676139458 learned_addr learned my addr 172.21.15.45:0/2676139458 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f9280ae7700 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a2904f920 0x557a290961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f92812e8700 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29097a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927bfff700 1 -- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a2904f920 msgr2=0x557a290961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927bfff700 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a2904f920 0x557a290961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927bfff700 1 -- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 msgr2=0x557a29097a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927bfff700 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29097a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927bfff700 1 -- 172.21.15.45:0/2676139458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c00a0e0 con 0x557a29111890 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f9280ae7700 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a2904f920 0x557a290961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927bfff700 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a29111890 0x557a29097530 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f926c001da0 tx=0x7f926c001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2c9c8298fcb5d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927affd700 1 -- 172.21.15.45:0/2676139458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c014070 con 0x557a29111890 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927affd700 1 -- 172.21.15.45:0/2676139458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c008070 con 0x557a29111890 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.099+0000 7f927affd700 1 -- 172.21.15.45:0/2676139458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c001ea0 con 0x557a29111890 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 -- 172.21.15.45:0/2676139458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a29111890 msgr2=0x557a29097530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a29111890 0x557a29097530 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f926c001da0 tx=0x7f926c001810 comp rx=0 tx=0).stop 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 -- 172.21.15.45:0/2676139458 shutdown_connections 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29097a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a2904f920 0x557a290961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 --2- 172.21.15.45:0/2676139458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a29111890 0x557a29097530 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 -- 172.21.15.45:0/2676139458 >> 172.21.15.45:0/2676139458 conn(0x557a29052450 msgr2=0x557a2907ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 -- 172.21.15.45:0/2676139458 shutdown_connections 2023-10-06T05:26:07.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 -- 172.21.15.45:0/2676139458 wait complete. 2023-10-06T05:26:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 Processor -- start 2023-10-06T05:26:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.100+0000 7f92878c9040 1 -- start start 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f92878c9040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a2904f920 0x557a2910eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f92878c9040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a29111890 0x557a29052820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f92878c9040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29052d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f92878c9040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a28f93400 con 0x557a2904f920 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f92878c9040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a29097640 con 0x557a29111c60 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f92878c9040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a290986d0 con 0x557a29111890 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f9280ae7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a2904f920 0x557a2910eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f9280ae7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a2904f920 0x557a2910eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54798/0 (socket says 172.21.15.45:54798) 2023-10-06T05:26:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 learned_addr learned my addr 172.21.15.45:0/191712409 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f927bfff700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a29111890 0x557a29052820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.101+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29052d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a29111890 msgr2=0x557a29052820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f9280ae7700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a29111890 0x557a29052820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 msgr2=0x557a29052d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f9280ae7700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29052d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c00a040 con 0x557a2904f920 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29111c60 0x557a29052d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f9280ae7700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a2904f920 0x557a2910eb40 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f927400b640 tx=0x7f9274001c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f92797fa700 1 -- 172.21.15.45:0/191712409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274015030 con 0x557a2904f920 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f92797fa700 1 -- 172.21.15.45:0/191712409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927400f650 con 0x557a2904f920 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557a28f9eae0 con 0x557a2904f920 2023-10-06T05:26:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f92797fa700 1 -- 172.21.15.45:0/191712409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927400f9d0 con 0x557a2904f920 2023-10-06T05:26:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.102+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a28f9e540 con 0x557a2904f920 2023-10-06T05:26:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.104+0000 7f92797fa700 1 -- 172.21.15.45:0/191712409 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9274018070 con 0x557a2904f920 2023-10-06T05:26:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.104+0000 7f92797fa700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92640658c0 0x7f9264067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.104+0000 7f92797fa700 1 -- 172.21.15.45:0/191712409 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f9274016070 con 0x557a2904f920 2023-10-06T05:26:07.109 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.104+0000 7f927bfff700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92640658c0 0x7f9264067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.104+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557a2910bf90 0x557a29144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.104+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29097bb0 con 0x557a2910bf90 2023-10-06T05:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.105+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557a2910bf90 0x557a29144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.105+0000 7f927bfff700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92640658c0 0x7f9264067d70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f926c001870 tx=0x7f926c016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.105+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557a2910bf90 0x557a29144440 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.107+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v126'1033 uv1007 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.107+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29081890 con 0x557a2910bf90 2023-10-06T05:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.109+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v126'1034 uv1034 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.109+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2910f080 con 0x557a2910bf90 2023-10-06T05:26:07.112 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:26:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.110+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v126'1035 uv1035 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.110+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a290532a0 con 0x557a2910bf90 2023-10-06T05:26:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.112+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'1036 uv1036 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.112+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145990 con 0x557a2910bf90 2023-10-06T05:26:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.113+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'1037 uv1037 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.113+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29144e70 con 0x557a2910bf90 2023-10-06T05:26:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.114+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'1038 uv1038 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.114+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.116+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v126'1039 uv1013 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.116+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.117+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v126'1040 uv1040 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.117+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.118+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v126'1041 uv1041 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.120 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:26:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.118+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.120+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'1042 uv1042 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.120+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.121+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'1043 uv1043 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.121+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.122+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'1044 uv1044 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.122+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.124+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v126'1045 uv1020 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.124+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.125+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'1046 uv1046 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.125+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.126+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v126'1047 uv1047 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.128 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:26:07.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.126+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.128+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'1048 uv1048 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.128+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.129+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'1049 uv1049 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.129+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145220 con 0x557a2910bf90 2023-10-06T05:26:07.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.130+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'1050 uv1050 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.130+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.131+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'1051 uv1051 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.131+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.133+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v126'1052 uv1026 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.133+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.134+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v126'1053 uv1053 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.134+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.136+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v126'1054 uv1054 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.137 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:26:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.136+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29146d50 con 0x557a2910bf90 2023-10-06T05:26:07.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.137+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'1055 uv1055 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.137+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29146d50 con 0x557a2910bf90 2023-10-06T05:26:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.138+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'1056 uv1056 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.138+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.139+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'1057 uv1057 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.139+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.141+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v126'1058 uv1032 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.141+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.142+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v126'1059 uv1059 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.142+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.143+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v126'1060 uv1060 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.146 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:26:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.144+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29146d50 con 0x557a2910bf90 2023-10-06T05:26:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.145+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'1061 uv1061 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.145+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29146d50 con 0x557a2910bf90 2023-10-06T05:26:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.146+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'1062 uv1062 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.146+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29145d40 con 0x557a2910bf90 2023-10-06T05:26:07.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.147+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'1063 uv1063 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2910bf90 2023-10-06T05:26:07.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.147+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557a29146250 0x557a29148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.148+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29148b70 con 0x557a29146250 2023-10-06T05:26:07.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.148+0000 7f9280ae7700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557a29146250 0x557a29148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.148+0000 7f9280ae7700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557a29146250 0x557a29148630 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.150+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'744 uv724 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.150+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29148f20 con 0x557a29146250 2023-10-06T05:26:07.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.152+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'745 uv745 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.152+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291492d0 con 0x557a29146250 2023-10-06T05:26:07.155 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:26:07.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.153+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'746 uv746 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.153+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29149b70 con 0x557a29146250 2023-10-06T05:26:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.154+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v126'747 uv747 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.154+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29149f20 con 0x557a29146250 2023-10-06T05:26:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.156+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'748 uv748 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.156+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914a2d0 con 0x557a29146250 2023-10-06T05:26:07.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.157+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v126'749 uv743 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.157+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914a680 con 0x557a29146250 2023-10-06T05:26:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.158+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v126'750 uv750 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.158+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.161 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:26:07.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.159+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v126'751 uv751 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.159+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.161+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'752 uv752 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.161+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.162+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'753 uv753 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.162+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.163+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v126'754 uv742 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.163+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.164+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v126'755 uv755 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.164+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.167 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:26:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.166+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v126'756 uv756 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.166+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914e150 con 0x557a29146250 2023-10-06T05:26:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.167+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v126'757 uv757 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.167+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914e150 con 0x557a29146250 2023-10-06T05:26:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.168+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'758 uv758 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.168+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.170+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v126'759 uv740 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.170+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.171+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v126'760 uv760 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.171+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.174 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:26:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.172+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v126'761 uv761 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.172+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914ba40 con 0x557a29146250 2023-10-06T05:26:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.173+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'762 uv762 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.173+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914ba40 con 0x557a29146250 2023-10-06T05:26:07.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.175+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'763 uv763 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.175+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914aa30 con 0x557a29146250 2023-10-06T05:26:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.176+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'764 uv764 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.176+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557a2914af40 0x557a2914d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.176+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914d860 con 0x557a2914af40 2023-10-06T05:26:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.176+0000 7f927bfff700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557a2914af40 0x557a2914d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.177+0000 7f927bfff700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557a2914af40 0x557a2914d320 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.178+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.178+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914dc10 con 0x557a2914af40 2023-10-06T05:26:07.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.179+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v126'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.179+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914dfc0 con 0x557a2914af40 2023-10-06T05:26:07.181 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:26:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.180+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v126'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.180+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f400 con 0x557a2914af40 2023-10-06T05:26:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.181+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.181+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f910 con 0x557a2914af40 2023-10-06T05:26:07.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.182+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.182+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914e370 con 0x557a2914af40 2023-10-06T05:26:07.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.183+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.183+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914e770 con 0x557a2914af40 2023-10-06T05:26:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.184+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'144 uv143 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.184+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914e770 con 0x557a2914af40 2023-10-06T05:26:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.185+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'145 uv145 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.185+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914ebf0 con 0x557a2914af40 2023-10-06T05:26:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.186+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'146 uv146 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.188 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:26:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.186+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914ebf0 con 0x557a2914af40 2023-10-06T05:26:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.187+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'147 uv147 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.187+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f010 con 0x557a29146250 2023-10-06T05:26:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.189+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.189+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f010 con 0x557a29146250 2023-10-06T05:26:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.190+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v126'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.190+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f010 con 0x557a29146250 2023-10-06T05:26:07.192 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:26:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.191+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v126'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.191+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f010 con 0x557a29146250 2023-10-06T05:26:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.192+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.192+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f010 con 0x557a29146250 2023-10-06T05:26:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.193+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.193+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2914f010 con 0x557a29146250 2023-10-06T05:26:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.194+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29146250 2023-10-06T05:26:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.194+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557a2914fd30 0x557a29152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.194+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.195+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557a2914fd30 0x557a29152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.195+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557a2914fd30 0x557a29152110 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.196+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v126'371 uv365 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.196+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.197+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v126'372 uv372 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.197+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.200 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:26:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.198+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v126'373 uv373 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.198+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29153660 con 0x557a2914fd30 2023-10-06T05:26:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.199+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.199+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29153660 con 0x557a2914fd30 2023-10-06T05:26:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.200+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'375 uv375 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.200+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.202+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'376 uv376 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.202+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.203+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v126'377 uv370 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.203+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.204+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v126'378 uv378 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.204+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.207 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:26:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.205+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v126'379 uv379 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.205+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.206+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'380 uv380 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.207+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29152650 con 0x557a2914fd30 2023-10-06T05:26:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.208+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'381 uv381 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.208+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x557a29152fd0 0x557a291553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.208+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291558f0 con 0x557a29152fd0 2023-10-06T05:26:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.208+0000 7f9280ae7700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x557a29152fd0 0x557a291553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.208+0000 7f9280ae7700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x557a29152fd0 0x557a291553b0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.209+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v126'168 uv167 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29152fd0 2023-10-06T05:26:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.210+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291558f0 con 0x557a29152fd0 2023-10-06T05:26:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.211+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v126'169 uv169 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29152fd0 2023-10-06T05:26:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.211+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291558f0 con 0x557a29152fd0 2023-10-06T05:26:07.213 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:26:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.212+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v126'170 uv170 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29152fd0 2023-10-06T05:26:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.212+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29155d70 con 0x557a29152fd0 2023-10-06T05:26:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.213+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'171 uv171 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29152fd0 2023-10-06T05:26:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.213+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29155d70 con 0x557a29152fd0 2023-10-06T05:26:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.214+0000 7f9280ae7700 1 -- 172.21.15.45:0/191712409 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'172 uv172 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9274082db0 con 0x557a29152fd0 2023-10-06T05:26:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.214+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29155d70 con 0x557a2914af40 2023-10-06T05:26:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.216+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.216+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29155d70 con 0x557a2914af40 2023-10-06T05:26:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.217+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v126'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.217+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29155d70 con 0x557a2914af40 2023-10-06T05:26:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.218+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v126'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.220 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:26:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.218+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29156d80 con 0x557a2914af40 2023-10-06T05:26:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.219+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.219+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29156d80 con 0x557a2914af40 2023-10-06T05:26:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.221+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.221+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29155d70 con 0x557a2914af40 2023-10-06T05:26:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.222+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.222+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x557a29156280 0x557a29158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.222+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a29158cd0 con 0x557a29156280 2023-10-06T05:26:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.222+0000 7f927bfff700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x557a29156280 0x557a29158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.223+0000 7f927bfff700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x557a29156280 0x557a29158660 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.224+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v126'335 uv329 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.224+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.225+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v126'336 uv336 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.225+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.228 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:26:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.226+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v126'337 uv337 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.227+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.228+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'338 uv338 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.228+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.229+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'339 uv339 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.229+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.230+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v126'340 uv334 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.230+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.231+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v126'341 uv341 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.231+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.232+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v126'342 uv342 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.234 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:26:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.232+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291590b0 con 0x557a29156280 2023-10-06T05:26:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.233+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'343 uv343 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.233+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915be20 con 0x557a29156280 2023-10-06T05:26:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.235+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'344 uv344 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a29156280 2023-10-06T05:26:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.235+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291595a0 con 0x557a2914af40 2023-10-06T05:26:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.236+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v126'147 uv145 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.236+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291595a0 con 0x557a2914af40 2023-10-06T05:26:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.237+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'148 uv148 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.237+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291595a0 con 0x557a2914af40 2023-10-06T05:26:07.240 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:26:07.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.238+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v126'149 uv149 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.238+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a291595a0 con 0x557a2914af40 2023-10-06T05:26:07.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.240+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'150 uv150 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.240+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x557a29159ab0 0x557a29159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.240+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a29159ab0 2023-10-06T05:26:07.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.240+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x557a29159ab0 0x557a29159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.240+0000 7f92812e8700 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x557a29159ab0 0x557a29159ec0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.241+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a29159ab0 2023-10-06T05:26:07.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.241+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a29159ab0 2023-10-06T05:26:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.244+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v126'202 uv202 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a29159ab0 2023-10-06T05:26:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.244+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a29159ab0 2023-10-06T05:26:07.248 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:26:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.247+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v126'203 uv203 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a29159ab0 2023-10-06T05:26:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.247+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915d580 con 0x557a29159ab0 2023-10-06T05:26:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.248+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a29159ab0 2023-10-06T05:26:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.248+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915d580 con 0x557a29159ab0 2023-10-06T05:26:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.250+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a29159ab0 2023-10-06T05:26:07.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.250+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a29159ab0 2023-10-06T05:26:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.251+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a29159ab0 2023-10-06T05:26:07.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.251+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914fd30 2023-10-06T05:26:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.252+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v126'171 uv170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.252+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914fd30 2023-10-06T05:26:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.253+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v126'172 uv172 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.253+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914fd30 2023-10-06T05:26:07.256 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:26:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.254+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v126'173 uv173 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.254+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914fd30 2023-10-06T05:26:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.255+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'174 uv174 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.255+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914fd30 2023-10-06T05:26:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.256+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'175 uv175 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f927c010b30 con 0x557a2914fd30 2023-10-06T05:26:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.257+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914af40 2023-10-06T05:26:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.258+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v126'416 uv406 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.258+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914af40 2023-10-06T05:26:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.259+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v126'417 uv417 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.259+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914af40 2023-10-06T05:26:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.260+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v126'418 uv418 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.262 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:26:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.260+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914af40 2023-10-06T05:26:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.261+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v126'419 uv419 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.261+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915ca10 con 0x557a2914af40 2023-10-06T05:26:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.262+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'420 uv420 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.262+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c5f0 con 0x557a2914af40 2023-10-06T05:26:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.263+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v126'421 uv414 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.263+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914af40 2023-10-06T05:26:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.265+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v126'422 uv422 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.265+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915c570 con 0x557a2914af40 2023-10-06T05:26:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.266+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v126'423 uv423 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.267 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:26:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.266+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915d580 con 0x557a2914af40 2023-10-06T05:26:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.267+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'424 uv424 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.267+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915d580 con 0x557a2914af40 2023-10-06T05:26:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.268+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'425 uv425 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.268+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915ca60 con 0x557a2914af40 2023-10-06T05:26:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.269+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'426 uv426 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.269+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x557a2915ca60 con 0x557a2914af40 2023-10-06T05:26:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f927bfff700 1 -- 172.21.15.45:0/191712409 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'427 uv427 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f926c016070 con 0x557a2914af40 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557a2914af40 msgr2=0x557a2914d320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x557a2914af40 0x557a2914d320 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557a2910bf90 msgr2=0x557a29144440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557a2910bf90 0x557a29144440 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x557a29156280 msgr2=0x557a29158660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x557a29156280 0x557a29158660 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557a29146250 msgr2=0x557a29148630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x557a29146250 0x557a29148630 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x557a29159ab0 msgr2=0x557a29159ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x557a29159ab0 0x557a29159ec0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x557a29152fd0 msgr2=0x557a291553b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x557a29152fd0 0x557a291553b0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557a2914fd30 msgr2=0x557a29152110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x557a2914fd30 0x557a29152110 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 reap_dead start 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 reap_dead start 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92812e8700 1 -- 172.21.15.45:0/191712409 reap_dead start 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92640658c0 msgr2=0x7f9264067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92640658c0 0x7f9264067d70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f926c001870 tx=0x7f926c016000 comp rx=0 tx=0).stop 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a2904f920 msgr2=0x557a2910eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a2904f920 0x557a2910eb40 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f927400b640 tx=0x7f9274001c90 comp rx=0 tx=0).stop 2023-10-06T05:26:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 shutdown_connections 2023-10-06T05:26:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a2904f920 0x557a2910eb40 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 --2- 172.21.15.45:0/191712409 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92640658c0 0x7f9264067d70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 >> 172.21.15.45:0/191712409 conn(0x557a29052450 msgr2=0x557a2907ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 shutdown_connections 2023-10-06T05:26:07.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.270+0000 7f92878c9040 1 -- 172.21.15.45:0/191712409 wait complete. 2023-10-06T05:26:07.276 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:26:07.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.323+0000 7f95db6ff040 1 Processor -- start 2023-10-06T05:26:07.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.323+0000 7f95db6ff040 1 -- start start 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95db6ff040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3087ed0 0x562df308cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95db6ff040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df305e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95db6ff040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3133900 0x562df30a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95db6ff040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df2fa7980 con 0x562df3087ed0 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95db6ff040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df2fa7620 con 0x562df312f9f0 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95db6ff040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562df2f9e400 con 0x562df3133900 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95d491d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3087ed0 0x562df308cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95d491d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3087ed0 0x562df308cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54810/0 (socket says 172.21.15.45:54810) 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95d491d700 1 -- 172.21.15.45:0/1119475719 learned_addr learned my addr 172.21.15.45:0/1119475719 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95d511e700 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3133900 0x562df30a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.324+0000 7f95cffff700 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df305e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95d491d700 1 -- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3133900 msgr2=0x562df30a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95d491d700 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3133900 0x562df30a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95d491d700 1 -- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 msgr2=0x562df305e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95d491d700 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df305e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95d491d700 1 -- 172.21.15.45:0/1119475719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0005db0 con 0x562df3087ed0 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95cffff700 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df305e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95d491d700 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3087ed0 0x562df308cf00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f95c8001eb0 tx=0x7f95c8001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21fb579b70a6585 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95ceffd700 1 -- 172.21.15.45:0/1119475719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8014070 con 0x562df3087ed0 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95ceffd700 1 -- 172.21.15.45:0/1119475719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8008030 con 0x562df3087ed0 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.325+0000 7f95ceffd700 1 -- 172.21.15.45:0/1119475719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800be50 con 0x562df3087ed0 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 -- 172.21.15.45:0/1119475719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3087ed0 msgr2=0x562df308cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3087ed0 0x562df308cf00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f95c8001eb0 tx=0x7f95c8001900 comp rx=0 tx=0).stop 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 -- 172.21.15.45:0/1119475719 shutdown_connections 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3133900 0x562df30a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3087ed0 0x562df308cf00 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 --2- 172.21.15.45:0/1119475719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df305e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 -- 172.21.15.45:0/1119475719 >> 172.21.15.45:0/1119475719 conn(0x562df305d4a0 msgr2=0x562df3134210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 -- 172.21.15.45:0/1119475719 shutdown_connections 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 -- 172.21.15.45:0/1119475719 wait complete. 2023-10-06T05:26:07.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 Processor -- start 2023-10-06T05:26:07.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.326+0000 7f95db6ff040 1 -- start start 2023-10-06T05:26:07.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95db6ff040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3087ed0 0x562df3125030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95db6ff040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df313c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95db6ff040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3133900 0x562df313cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95db6ff040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df313b080 con 0x562df3133900 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95db6ff040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df313b320 con 0x562df312f9f0 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95db6ff040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x562df313b5c0 con 0x562df3087ed0 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95d511e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3133900 0x562df313cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95d511e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3133900 0x562df313cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54822/0 (socket says 172.21.15.45:54822) 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95d511e700 1 -- 172.21.15.45:0/3840684879 learned_addr learned my addr 172.21.15.45:0/3840684879 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95cffff700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df313c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.327+0000 7f95d491d700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3087ed0 0x562df3125030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95d511e700 1 -- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3087ed0 msgr2=0x562df3125030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95d511e700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562df3087ed0 0x562df3125030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95d511e700 1 -- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 msgr2=0x562df313c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95d511e700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df313c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95d511e700 1 -- 172.21.15.45:0/3840684879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c800a040 con 0x562df3133900 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95cffff700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562df312f9f0 0x562df313c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95d511e700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3133900 0x562df313cdd0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x562df3134960 tx=0x7f95d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95cd7fa700 1 -- 172.21.15.45:0/3840684879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0020070 con 0x562df3133900 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95cd7fa700 1 -- 172.21.15.45:0/3840684879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d000fa70 con 0x562df3133900 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d000a0e0 con 0x562df3133900 2023-10-06T05:26:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95cd7fa700 1 -- 172.21.15.45:0/3840684879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d001a640 con 0x562df3133900 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.328+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562df2fb63d0 con 0x562df3133900 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.330+0000 7f95cd7fa700 1 -- 172.21.15.45:0/3840684879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95d0003040 con 0x562df3133900 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.330+0000 7f95cd7fa700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b8065890 0x7f95b8067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.330+0000 7f95cd7fa700 1 -- 172.21.15.45:0/3840684879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f95d0085ca0 con 0x562df3133900 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.330+0000 7f95d491d700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b8065890 0x7f95b8067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.330+0000 7f95db6ff040 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562df31273e0 0x562df31277e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.330+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v8 -- 0x562df3127d20 con 0x562df31273e0 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.331+0000 7f95cffff700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562df31273e0 0x562df31277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.331+0000 7f95d491d700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b8065890 0x7f95b8067d40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000fb0 tx=0x7f95c8016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.331+0000 7f95cffff700 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562df31273e0 0x562df31277e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95cffff700 1 -- 172.21.15.45:0/3840684879 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v126'1064 uv1064 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f95c00075d0 con 0x562df31273e0 2023-10-06T05:26:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562df31273e0 msgr2=0x562df31277e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x562df31273e0 0x562df31277e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b8065890 msgr2=0x7f95b8067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f95b8065890 0x7f95b8067d40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000fb0 tx=0x7f95c8016000 comp rx=0 tx=0).stop 2023-10-06T05:26:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3133900 msgr2=0x562df313cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 --2- 172.21.15.45:0/3840684879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562df3133900 0x562df313cdd0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x562df3134960 tx=0x7f95d0015000 comp rx=0 tx=0).stop 2023-10-06T05:26:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95d511e700 1 -- 172.21.15.45:0/3840684879 reap_dead start 2023-10-06T05:26:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 shutdown_connections 2023-10-06T05:26:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 >> 172.21.15.45:0/3840684879 conn(0x562df305d4a0 msgr2=0x562df305f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 shutdown_connections 2023-10-06T05:26:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.333+0000 7f95db6ff040 1 -- 172.21.15.45:0/3840684879 wait complete. 2023-10-06T05:26:07.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:26:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.476+0000 7f74ef025700 1 Processor -- start 2023-10-06T05:26:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.476+0000 7f74ef025700 1 -- start start 2023-10-06T05:26:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ef025700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8159520 0x7f74e815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ef025700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74e815d3d0 0x7f74e8067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ef025700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e80c9c90 con 0x7f74e8159150 2023-10-06T05:26:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ef025700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e815d8e0 con 0x7f74e815d3d0 2023-10-06T05:26:07.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ef025700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e815dbc0 con 0x7f74e8159520 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ed5c2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74e815d3d0 0x7f74e8067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ed5c2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74e815d3d0 0x7f74e8067620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54834/0 (socket says 172.21.15.45:54834) 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ed5c2700 1 -- 172.21.15.45:0/2658975176 learned_addr learned my addr 172.21.15.45:0/2658975176 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74e7fff700 1 --2- 172.21.15.45:0/2658975176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8159520 0x7f74e815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ed5c2700 1 -- 172.21.15.45:0/2658975176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8159520 msgr2=0x7f74e815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ed5c2700 1 --2- 172.21.15.45:0/2658975176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8159520 0x7f74e815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ed5c2700 1 -- 172.21.15.45:0/2658975176 >> v1:172.21.15.144:6789/0 conn(0x7f74e8159150 legacy=0x7f74e80ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74e77fe700 1 -- 172.21.15.45:0/2658975176 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761548663 0 0) 0x7f74e80c9c90 con 0x7f74e8159150 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74ed5c2700 1 -- 172.21.15.45:0/2658975176 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80c9920 con 0x7f74e815d3d0 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.477+0000 7f74e7fff700 1 --2- 172.21.15.45:0/2658975176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8159520 0x7f74e815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:07.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ed5c2700 1 --2- 172.21.15.45:0/2658975176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8067620 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009f20 tx=0x7f74e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901c1a9f09c16e53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74e77fe700 1 -- 172.21.15.45:0/2658975176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000e070 con 0x7f74e815d3d0 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74e77fe700 1 -- 172.21.15.45:0/2658975176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e000b9d0 con 0x7f74e815d3d0 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74e77fe700 1 -- 172.21.15.45:0/2658975176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e00092e0 con 0x7f74e815d3d0 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 -- 172.21.15.45:0/2658975176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 msgr2=0x7f74e8067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 --2- 172.21.15.45:0/2658975176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8067620 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009f20 tx=0x7f74e0009840 comp rx=0 tx=0).stop 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 -- 172.21.15.45:0/2658975176 shutdown_connections 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 --2- 172.21.15.45:0/2658975176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8067620 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 --2- 172.21.15.45:0/2658975176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8159520 0x7f74e815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 -- 172.21.15.45:0/2658975176 >> 172.21.15.45:0/2658975176 conn(0x7f74e80691c0 msgr2=0x7f74e80cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 -- 172.21.15.45:0/2658975176 shutdown_connections 2023-10-06T05:26:07.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.478+0000 7f74ef025700 1 -- 172.21.15.45:0/2658975176 wait complete. 2023-10-06T05:26:07.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.480+0000 7f74ef025700 1 Processor -- start 2023-10-06T05:26:07.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.480+0000 7f74ef025700 1 -- start start 2023-10-06T05:26:07.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ef025700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e8159150 0x7f74e8147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ef025700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8168ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ef025700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8169420 0x7f74e815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ef025700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e815c410 con 0x7f74e815d3d0 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ef025700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8166b20 con 0x7f74e8159150 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ef025700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81699f0 con 0x7f74e8169420 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8168ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8168ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54838/0 (socket says 172.21.15.45:54838) 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 -- 172.21.15.45:0/3582975566 learned_addr learned my addr 172.21.15.45:0/3582975566 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ecdc1700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e8159150 0x7f74e8147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ed5c2700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8169420 0x7f74e815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 -- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8169420 msgr2=0x7f74e815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8169420 0x7f74e815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 -- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e8159150 msgr2=0x7f74e8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e8159150 0x7f74e8147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 -- 172.21.15.45:0/3582975566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4009420 con 0x7f74e815d3d0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e7fff700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8168ee0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f74d400ec80 tx=0x7f74d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e4ff9700 1 -- 172.21.15.45:0/3582975566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d400c920 con 0x7f74e815d3d0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e4ff9700 1 -- 172.21.15.45:0/3582975566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d400ca80 con 0x7f74e815d3d0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e0009ae0 con 0x7f74e815d3d0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.481+0000 7f74e4ff9700 1 -- 172.21.15.45:0/3582975566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d400ce00 con 0x7f74e815d3d0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.482+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e8166dc0 con 0x7f74e815d3d0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.482+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e8063500 con 0x7f74e815d3d0 2023-10-06T05:26:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.483+0000 7f74e4ff9700 1 -- 172.21.15.45:0/3582975566 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74d4013680 con 0x7f74e815d3d0 2023-10-06T05:26:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.483+0000 7f74e4ff9700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f74cc066f30 0x7f74cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.483+0000 7f74e4ff9700 1 -- 172.21.15.45:0/3582975566 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f74d4085730 con 0x7f74e815d3d0 2023-10-06T05:26:07.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.484+0000 7f74ecdc1700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f74cc066f30 0x7f74cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.484+0000 7f74ecdc1700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f74cc066f30 0x7f74cc0693e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f74e806bcb0 tx=0x7f74dc007dc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.488+0000 7f74e4ff9700 1 -- 172.21.15.45:0/3582975566 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74d4051b50 con 0x7f74e815d3d0 2023-10-06T05:26:07.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:07.740+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f74e80cdba0 con 0x7f74e815d3d0 2023-10-06T05:26:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.034+0000 7f74e4ff9700 1 -- 172.21.15.45:0/3582975566 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v308) v1 ==== 162+0+0 (secure 0 0 0) 0x7f74d4055170 con 0x7f74e815d3d0 2023-10-06T05:26:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f74cc066f30 msgr2=0x7f74cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f74cc066f30 0x7f74cc0693e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f74e806bcb0 tx=0x7f74dc007dc0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 msgr2=0x7f74e8168ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8168ee0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f74d400ec80 tx=0x7f74d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 shutdown_connections 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f74cc066f30 0x7f74cc0693e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74e8169420 0x7f74e815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e8159150 0x7f74e8147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 --2- 172.21.15.45:0/3582975566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e815d3d0 0x7f74e8168ee0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.037+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 >> 172.21.15.45:0/3582975566 conn(0x7f74e80691c0 msgr2=0x7f74e80c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.038+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 shutdown_connections 2023-10-06T05:26:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.038+0000 7f74ef025700 1 -- 172.21.15.45:0/3582975566 wait complete. 2023-10-06T05:26:08.041 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:26:08.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:08.318 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:08.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.597+0000 7fdc5f59e700 1 Processor -- start 2023-10-06T05:26:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.597+0000 7fdc5f59e700 1 -- start start 2023-10-06T05:26:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.598+0000 7fdc5f59e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc600c3470 0x7fdc60067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.598+0000 7fdc5f59e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc60159450 0x7fdc60067b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.598+0000 7fdc5f59e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc600a9ff0 con 0x7fdc60159080 2023-10-06T05:26:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.598+0000 7fdc5f59e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc600c3be0 con 0x7fdc60159450 2023-10-06T05:26:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.598+0000 7fdc5f59e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600c3900 con 0x7fdc600c3470 2023-10-06T05:26:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc600c3470 0x7fdc60067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5ed9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc60159450 0x7fdc60067b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc600c3470 0x7fdc60067620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:36274/0 (socket says 172.21.15.45:36274) 2023-10-06T05:26:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 -- 172.21.15.45:0/3631944473 learned_addr learned my addr 172.21.15.45:0/3631944473 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 -- 172.21.15.45:0/3631944473 >> v2:172.21.15.144:3300/0 conn(0x7fdc60159450 msgr2=0x7fdc60067b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 --2- 172.21.15.45:0/3631944473 >> v2:172.21.15.144:3300/0 conn(0x7fdc60159450 0x7fdc60067b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 -- 172.21.15.45:0/3631944473 >> v1:172.21.15.144:6789/0 conn(0x7fdc60159080 legacy=0x7fdc600ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 -- 172.21.15.45:0/3631944473 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc600a99a0 con 0x7fdc600c3470 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5ed9d700 1 --2- 172.21.15.45:0/3631944473 >> v2:172.21.15.144:3300/0 conn(0x7fdc60159450 0x7fdc60067b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5dd9b700 1 --2- 172.21.15.45:0/3631944473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc600c3470 0x7fdc60067620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdc480099b0 tx=0x7fdc48009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7bae93801627ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.599+0000 7fdc5d59a700 1 -- 172.21.15.45:0/3631944473 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48009ea0 con 0x7fdc600c3470 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5d59a700 1 -- 172.21.15.45:0/3631944473 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4800b8f0 con 0x7fdc600c3470 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5d59a700 1 -- 172.21.15.45:0/3631944473 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800bc90 con 0x7fdc600c3470 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3631944473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc600c3470 msgr2=0x7fdc60067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3631944473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc600c3470 0x7fdc60067620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdc480099b0 tx=0x7fdc48009250 comp rx=0 tx=0).stop 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3631944473 shutdown_connections 2023-10-06T05:26:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3631944473 >> v2:172.21.15.144:3300/0 conn(0x7fdc60159450 0x7fdc60067b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3631944473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc600c3470 0x7fdc60067620 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3631944473 >> 172.21.15.45:0/3631944473 conn(0x7fdc60095a40 msgr2=0x7fdc60095e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3631944473 shutdown_connections 2023-10-06T05:26:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.600+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3631944473 wait complete. 2023-10-06T05:26:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.601+0000 7fdc5f59e700 1 Processor -- start 2023-10-06T05:26:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.601+0000 7fdc5f59e700 1 -- start start 2023-10-06T05:26:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5f59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600c3470 0x7fdc6015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5f59e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc60159080 0x7fdc600c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5f59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc60159450 0x7fdc600c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5f59e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600c6d60 con 0x7fdc600c3470 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5f59e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6015d790 con 0x7fdc60159450 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5f59e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6015da70 con 0x7fdc60159080 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600c3470 0x7fdc6015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600c3470 0x7fdc6015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54848/0 (socket says 172.21.15.45:54848) 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5e59c700 1 -- 172.21.15.45:0/3636149104 learned_addr learned my addr 172.21.15.45:0/3636149104 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5ed9d700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc60159450 0x7fdc600c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.602+0000 7fdc5dd9b700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc60159080 0x7fdc600c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5e59c700 1 -- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc60159080 msgr2=0x7fdc600c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5e59c700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc60159080 0x7fdc600c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5e59c700 1 -- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc60159450 msgr2=0x7fdc600c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5e59c700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc60159450 0x7fdc600c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5e59c700 1 -- 172.21.15.45:0/3636149104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58009a20 con 0x7fdc600c3470 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5e59c700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600c3470 0x7fdc6015d100 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c7c0 tx=0x7fdc50007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc567fc700 1 -- 172.21.15.45:0/3636149104 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50011590 con 0x7fdc600c3470 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc567fc700 1 -- 172.21.15.45:0/3636149104 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc50011710 con 0x7fdc600c3470 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480094f0 con 0x7fdc600c3470 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc567fc700 1 -- 172.21.15.45:0/3636149104 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50011ad0 con 0x7fdc600c3470 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.603+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6015e030 con 0x7fdc600c3470 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.604+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6006e980 con 0x7fdc600c3470 2023-10-06T05:26:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.605+0000 7fdc567fc700 1 -- 172.21.15.45:0/3636149104 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc5001b3b0 con 0x7fdc600c3470 2023-10-06T05:26:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.605+0000 7fdc567fc700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc40066f30 0x7fdc400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.605+0000 7fdc567fc700 1 -- 172.21.15.45:0/3636149104 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7fdc50084690 con 0x7fdc600c3470 2023-10-06T05:26:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.605+0000 7fdc5dd9b700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc40066f30 0x7fdc400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.606+0000 7fdc5dd9b700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc40066f30 0x7fdc400693e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdc480099b0 tx=0x7fdc48017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.610+0000 7fdc567fc700 1 -- 172.21.15.45:0/3636149104 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc50050c20 con 0x7fdc600c3470 2023-10-06T05:26:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.859+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc60064a60 con 0x7fdc600c3470 2023-10-06T05:26:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.860+0000 7fdc567fc700 1 -- 172.21.15.45:0/3636149104 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 309 v309) v1 ==== 95+0+4609 (secure 0 0 0) 0x7fdc50017070 con 0x7fdc600c3470 2023-10-06T05:26:08.862 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:08.862 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":309,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":309,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:08.035515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9739":{"gid":9739,"name":"b","rank":0,"incarnation":309,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/2001233142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2001233142},{"type":"v1","addr":"172.21.15.144:6835","nonce":2001233142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc40066f30 msgr2=0x7fdc400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc40066f30 0x7fdc400693e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdc480099b0 tx=0x7fdc48017000 comp rx=0 tx=0).stop 2023-10-06T05:26:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600c3470 msgr2=0x7fdc6015d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600c3470 0x7fdc6015d100 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c7c0 tx=0x7fdc50007f70 comp rx=0 tx=0).stop 2023-10-06T05:26:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 shutdown_connections 2023-10-06T05:26:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc40066f30 0x7fdc400693e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc60159450 0x7fdc600c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600c3470 0x7fdc6015d100 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 --2- 172.21.15.45:0/3636149104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc60159080 0x7fdc600c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 >> 172.21.15.45:0/3636149104 conn(0x7fdc60095a40 msgr2=0x7fdc600ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:08.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 shutdown_connections 2023-10-06T05:26:08.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:08.862+0000 7fdc5f59e700 1 -- 172.21.15.45:0/3636149104 wait complete. 2023-10-06T05:26:08.865 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 309 2023-10-06T05:26:08.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 309, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:26:08.035515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9739': {'gid': 9739, 'name': 'b', 'rank': 0, 'incarnation': 309, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/2001233142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 2001233142}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2001233142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:26:08.877 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9739:up:replay 2023-10-06T05:26:08.877 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:26:13.036 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:13.035+0000 7f42387f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:14.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.421+0000 7f9d8a697700 1 Processor -- start 2023-10-06T05:26:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.421+0000 7f9d8a697700 1 -- start start 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d8a697700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d84159170 0x7f9d84159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d8a697700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d840aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d8a697700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d840ca6d0 con 0x7f9d8415d040 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d8a697700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8415d550 con 0x7f9d84159170 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d8a697700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8415d810 con 0x7f9d840692d0 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d83fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d84159170 0x7f9d84159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d837fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9d8415d040 0x7f9d84068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36552/0 (socket says 172.21.15.45:36552) 2023-10-06T05:26:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d837fe700 1 -- 172.21.15.45:0/3487384295 learned_addr learned my addr 172.21.15.45:0/3487384295 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d88c34700 1 --2- 172.21.15.45:0/3487384295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d840aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d83fff700 1 -- 172.21.15.45:0/3487384295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 msgr2=0x7f9d840aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.422+0000 7f9d83fff700 1 --2- 172.21.15.45:0/3487384295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d840aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d83fff700 1 -- 172.21.15.45:0/3487384295 >> v1:172.21.15.45:6789/0 conn(0x7f9d8415d040 legacy=0x7f9d84068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d83fff700 1 -- 172.21.15.45:0/3487384295 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8415daf0 con 0x7f9d84159170 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d82ffd700 1 -- 172.21.15.45:0/3487384295 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716794850 0 0) 0x7f9d840ca6d0 con 0x7f9d8415d040 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d83fff700 1 --2- 172.21.15.45:0/3487384295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d84159540 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9d740098e0 tx=0x7f9d740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578806f1e8e13856 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d82ffd700 1 -- 172.21.15.45:0/3487384295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74009e80 con 0x7f9d84159170 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d82ffd700 1 -- 172.21.15.45:0/3487384295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7400b860 con 0x7f9d84159170 2023-10-06T05:26:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d82ffd700 1 -- 172.21.15.45:0/3487384295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400bbe0 con 0x7f9d84159170 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d8a697700 1 -- 172.21.15.45:0/3487384295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 msgr2=0x7f9d84159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d8a697700 1 --2- 172.21.15.45:0/3487384295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d84159540 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9d740098e0 tx=0x7f9d740091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d8a697700 1 -- 172.21.15.45:0/3487384295 shutdown_connections 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d8a697700 1 --2- 172.21.15.45:0/3487384295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d840aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.423+0000 7f9d8a697700 1 --2- 172.21.15.45:0/3487384295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d84159540 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.424+0000 7f9d8a697700 1 -- 172.21.15.45:0/3487384295 >> 172.21.15.45:0/3487384295 conn(0x7f9d8406f7c0 msgr2=0x7f9d840d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.424+0000 7f9d8a697700 1 -- 172.21.15.45:0/3487384295 shutdown_connections 2023-10-06T05:26:14.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.424+0000 7f9d8a697700 1 -- 172.21.15.45:0/3487384295 wait complete. 2023-10-06T05:26:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.425+0000 7f9d8a697700 1 Processor -- start 2023-10-06T05:26:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.425+0000 7f9d8a697700 1 -- start start 2023-10-06T05:26:14.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d8a697700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d8415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d8a697700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d840cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d8a697700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8415d040 0x7f9d840cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d8a697700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8415cd30 con 0x7f9d84159170 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d8a697700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d84158be0 con 0x7f9d8415d040 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d8a697700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d84157fd0 con 0x7f9d840692d0 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d840cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d840cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54874/0 (socket says 172.21.15.45:54874) 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 -- 172.21.15.45:0/2681194617 learned_addr learned my addr 172.21.15.45:0/2681194617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:14.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d88c34700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8415d040 0x7f9d840cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d83fff700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d8415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 -- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 msgr2=0x7f9d8415c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d8415c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 -- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8415d040 msgr2=0x7f9d840cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8415d040 0x7f9d840cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 -- 172.21.15.45:0/2681194617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74009420 con 0x7f9d84159170 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d88c34700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8415d040 0x7f9d840cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d83fff700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d8415c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.426+0000 7f9d837fe700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d840cf080 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c5e0 tx=0x7f9d78000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.427+0000 7f9d6bfff700 1 -- 172.21.15.45:0/2681194617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78005b30 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.427+0000 7f9d6bfff700 1 -- 172.21.15.45:0/2681194617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78005c90 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.427+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d841582b0 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.427+0000 7f9d6bfff700 1 -- 172.21.15.45:0/2681194617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78007300 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.427+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d84158870 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.428+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d84063b50 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.428+0000 7f9d6bfff700 1 -- 172.21.15.45:0/2681194617 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d780076b0 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.428+0000 7f9d6bfff700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d70066f30 0x7f9d700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.429+0000 7f9d6bfff700 1 -- 172.21.15.45:0/2681194617 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f9d780848c0 con 0x7f9d84159170 2023-10-06T05:26:14.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.429+0000 7f9d83fff700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d70066f30 0x7f9d700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.429+0000 7f9d83fff700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d70066f30 0x7f9d700693e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9d74003080 tx=0x7f9d74017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.433+0000 7f9d6bfff700 1 -- 172.21.15.45:0/2681194617 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d78050ce0 con 0x7f9d84159170 2023-10-06T05:26:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.682+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d8415ddd0 con 0x7f9d84159170 2023-10-06T05:26:14.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.682+0000 7f9d6bfff700 1 -- 172.21.15.45:0/2681194617 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 312 v312) v1 ==== 95+0+4609 (secure 0 0 0) 0x7f9d78017020 con 0x7f9d84159170 2023-10-06T05:26:14.684 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:14.685 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":312,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":312,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:11.051307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9739":{"gid":9739,"name":"b","rank":0,"incarnation":309,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6835/2001233142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2001233142},{"type":"v1","addr":"172.21.15.144:6835","nonce":2001233142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.684+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d70066f30 msgr2=0x7f9d700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.684+0000 7f9d8a697700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d70066f30 0x7f9d700693e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9d74003080 tx=0x7f9d74017000 comp rx=0 tx=0).stop 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.684+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 msgr2=0x7f9d840cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.684+0000 7f9d8a697700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d840cf080 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c5e0 tx=0x7f9d78000fe0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 shutdown_connections 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d70066f30 0x7f9d700693e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d8415d040 0x7f9d840cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d840692d0 0x7f9d8415c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 --2- 172.21.15.45:0/2681194617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d84159170 0x7f9d840cf080 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 >> 172.21.15.45:0/2681194617 conn(0x7f9d8406f7c0 msgr2=0x7f9d840b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 shutdown_connections 2023-10-06T05:26:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.685+0000 7f9d8a697700 1 -- 172.21.15.45:0/2681194617 wait complete. 2023-10-06T05:26:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 312 2023-10-06T05:26:14.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 312, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:26:11.051307+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9739': {'gid': 9739, 'name': 'b', 'rank': 0, 'incarnation': 309, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.144:6835/2001233142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 2001233142}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2001233142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:26:14.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:26:14.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:26:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.832+0000 7f8badba7700 1 Processor -- start 2023-10-06T05:26:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.832+0000 7f8badba7700 1 -- start start 2023-10-06T05:26:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8badba7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8ba8156ce0 0x7f8ba81570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8badba7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ba806bad0 0x7f8ba815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8badba7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba815b150 con 0x7f8ba80afbe0 2023-10-06T05:26:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8badba7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba806bf70 con 0x7f8ba806bad0 2023-10-06T05:26:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8badba7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba815b380 con 0x7f8ba8156ce0 2023-10-06T05:26:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ba806bad0 0x7f8ba815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8ba80afbe0 0x7f8ba80ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36568/0 (socket says 172.21.15.45:36568) 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba7fff700 1 -- 172.21.15.45:0/350307797 learned_addr learned my addr 172.21.15.45:0/350307797 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/350307797 >> v2:172.21.15.144:3300/0 conn(0x7f8ba8156ce0 0x7f8ba81570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/350307797 >> v2:172.21.15.144:3300/0 conn(0x7f8ba8156ce0 msgr2=0x7f8ba81570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/350307797 >> v2:172.21.15.144:3300/0 conn(0x7f8ba8156ce0 0x7f8ba81570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/350307797 >> v1:172.21.15.45:6789/0 conn(0x7f8ba80afbe0 legacy=0x7f8ba80ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/350307797 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba815b660 con 0x7f8ba806bad0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba67fc700 1 -- 172.21.15.45:0/350307797 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493959680 0 0) 0x7f8ba815b150 con 0x7f8ba80afbe0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.833+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/350307797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba806bad0 0x7f8ba815ac10 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80af900 tx=0x7f8b9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a939357730e8142b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8ba67fc700 1 -- 172.21.15.45:0/350307797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00b6b0 con 0x7f8ba806bad0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8ba67fc700 1 -- 172.21.15.45:0/350307797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c00b810 con 0x7f8ba806bad0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8ba67fc700 1 -- 172.21.15.45:0/350307797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00bbb0 con 0x7f8ba806bad0 2023-10-06T05:26:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 -- 172.21.15.45:0/350307797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba806bad0 msgr2=0x7f8ba815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 --2- 172.21.15.45:0/350307797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba806bad0 0x7f8ba815ac10 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80af900 tx=0x7f8b9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 -- 172.21.15.45:0/350307797 shutdown_connections 2023-10-06T05:26:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 --2- 172.21.15.45:0/350307797 >> v2:172.21.15.144:3300/0 conn(0x7f8ba8156ce0 0x7f8ba81570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 --2- 172.21.15.45:0/350307797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba806bad0 0x7f8ba815ac10 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 -- 172.21.15.45:0/350307797 >> 172.21.15.45:0/350307797 conn(0x7f8ba806f090 msgr2=0x7f8ba806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 -- 172.21.15.45:0/350307797 shutdown_connections 2023-10-06T05:26:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.834+0000 7f8badba7700 1 -- 172.21.15.45:0/350307797 wait complete. 2023-10-06T05:26:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.836+0000 7f8badba7700 1 Processor -- start 2023-10-06T05:26:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.836+0000 7f8badba7700 1 -- start start 2023-10-06T05:26:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.836+0000 7f8badba7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ba806bad0 0x7f8ba80c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.836+0000 7f8badba7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba80afbe0 0x7f8ba80c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.836+0000 7f8badba7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba8156ce0 0x7f8ba8154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8badba7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8153c00 con 0x7f8ba8156ce0 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8badba7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba80c6b90 con 0x7f8ba80afbe0 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8badba7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba816cf60 con 0x7f8ba806bad0 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba8156ce0 0x7f8ba8154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba8156ce0 0x7f8ba8154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54890/0 (socket says 172.21.15.45:54890) 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 -- 172.21.15.45:0/2665333833 learned_addr learned my addr 172.21.15.45:0/2665333833 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba80afbe0 0x7f8ba80c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ba806bad0 0x7f8ba80c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 -- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ba806bad0 msgr2=0x7f8ba80c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ba806bad0 0x7f8ba80c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 -- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba80afbe0 msgr2=0x7f8ba80c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba80afbe0 0x7f8ba80c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 -- 172.21.15.45:0/2665333833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c009420 con 0x7f8ba8156ce0 2023-10-06T05:26:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba8156ce0 0x7f8ba8154fe0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000cc90 tx=0x7f8ba0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8b8f7fe700 1 -- 172.21.15.45:0/2665333833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000a8b0 con 0x7f8ba8156ce0 2023-10-06T05:26:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8b8f7fe700 1 -- 172.21.15.45:0/2665333833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba000aa10 con 0x7f8ba8156ce0 2023-10-06T05:26:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8b8f7fe700 1 -- 172.21.15.45:0/2665333833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000add0 con 0x7f8ba8156ce0 2023-10-06T05:26:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba816dae0 con 0x7f8ba8156ce0 2023-10-06T05:26:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.837+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba816d490 con 0x7f8ba8156ce0 2023-10-06T05:26:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.839+0000 7f8b8f7fe700 1 -- 172.21.15.45:0/2665333833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ba0007300 con 0x7f8ba8156ce0 2023-10-06T05:26:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.839+0000 7f8b8f7fe700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8b94066f60 0x7f8b94069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.840+0000 7f8b8f7fe700 1 -- 172.21.15.45:0/2665333833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f8ba0085680 con 0x7f8ba8156ce0 2023-10-06T05:26:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.840+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8b94066f60 0x7f8b94069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.840+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8b94066f60 0x7f8b94069410 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8155c80 tx=0x7f8b98008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.841+0000 7f8bb6ebbb80 1 Processor -- start 2023-10-06T05:26:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.842+0000 7f8bb6ebbb80 1 -- start start 2023-10-06T05:26:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.842+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d99f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.842+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d984f40 0x56233d9a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.842+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d9a84d0 0x56233d9a8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.842+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d872310 con 0x56233d984f40 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d8af130 con 0x56233d9a84d0 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56233d8acaf0 con 0x56233d99f450 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d984f40 0x56233d9a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d984f40 0x56233d9a7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54900/0 (socket says 172.21.15.45:54900) 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/3104334794 learned_addr learned my addr 172.21.15.45:0/3104334794 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d99f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d9a84d0 0x56233d9a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 msgr2=0x56233d99f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d99f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d9a84d0 msgr2=0x56233d9a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d9a84d0 0x56233d9a8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/3104334794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56233d6ae790 con 0x56233d984f40 2023-10-06T05:26:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d9a84d0 0x56233d9a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d984f40 0x56233d9a7f90 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806aed0 tx=0x7f8b9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137f8536e407fc8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8b8cff9700 1 -- 172.21.15.45:0/3104334794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c021070 con 0x56233d984f40 2023-10-06T05:26:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.843+0000 7f8b8cff9700 1 -- 172.21.15.45:0/3104334794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c000bc0 con 0x56233d984f40 2023-10-06T05:26:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8b8cff9700 1 -- 172.21.15.45:0/3104334794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00b980 con 0x56233d984f40 2023-10-06T05:26:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/3104334794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d984f40 msgr2=0x56233d9a7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d984f40 0x56233d9a7f90 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806aed0 tx=0x7f8b9c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/3104334794 shutdown_connections 2023-10-06T05:26:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d9a84d0 0x56233d9a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d99f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/3104334794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d984f40 0x56233d9a7f90 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/3104334794 >> 172.21.15.45:0/3104334794 conn(0x56233d9a2880 msgr2=0x56233d9a2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/3104334794 shutdown_connections 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.844+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/3104334794 wait complete. 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.845+0000 7f8bb6ebbb80 1 -- start start 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.845+0000 7f8bb6ebbb80 1 Processor -- start 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d979330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d99f450 0x56233d97a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d9a84d0 0x56233d959f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d8a88e0 con 0x56233d99f450 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d854980 con 0x56233d984f40 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56233d854620 con 0x56233d9a84d0 2023-10-06T05:26:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8bb6ebbb80 10 client.0 _ll_register_callbacks cb 0x56233d6b7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d99f450 0x56233d97a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d979330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d99f450 0x56233d97a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54910/0 (socket says 172.21.15.45:54910) 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/2002281224 learned_addr learned my addr 172.21.15.45:0/2002281224 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d9a84d0 0x56233d959f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d9a84d0 msgr2=0x56233d959f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d9a84d0 0x56233d959f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 msgr2=0x56233d979330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d979330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.846+0000 7f8ba6ffd700 1 -- 172.21.15.45:0/2002281224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba816d490 con 0x56233d99f450 2023-10-06T05:26:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d99f450 0x56233d97a520 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c02a040 tx=0x7f8b9c01e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a346cb8c6dea6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8b85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8b85ffb700 1 -- 172.21.15.45:0/2002281224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c021050 con 0x56233d99f450 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8b85ffb700 1 -- 172.21.15.45:0/2002281224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f8b9c01a630 con 0x56233d99f450 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56233d94cd60 con 0x56233d99f450 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8b85ffb700 1 -- 172.21.15.45:0/2002281224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=312) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b9c00bb90 con 0x56233d99f450 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 10 client.10299 fetch_fsmap learned FSMap version 312 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=312,monmap=2+}) v3 -- 0x7f8ba815b660 con 0x56233d99f450 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8b85ffb700 1 -- 172.21.15.45:0/2002281224 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56233d94cd60 con 0x56233d99f450 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8b85ffb700 1 -- 172.21.15.45:0/2002281224 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 312) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8b9c02e030 con 0x56233d99f450 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 10 client.10299 fetch_fsmap finished waiting for FSMap version 312 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 10 client.10299 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:26:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 20 client.10299 populate_metadata read hostname 'smithi045' 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9a15e0 0x56233d9a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 4 client.10299 mds_command: new command op to 9739 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.847+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 --> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56233d929b30 con 0x56233d9a15e0 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.848+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9a15e0 0x56233d9a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.848+0000 7f8b85ffb700 10 client.10299 ms_handle_connect on v2:172.21.15.144:6834/2001233142 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.848+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9a15e0 0x56233d9a19b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b717cb2999d9bf7e server_cookie=9677245ff61009a2 in_seq=0 out_seq=0 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8b85ffb700 1 -- 172.21.15.45:0/2002281224 <== mds.0 v2:172.21.15.144:6834/2001233142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56233d929b30 con 0x56233d9a15e0 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8b85ffb700 10 client.10299 handle_command_reply: tid=0 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 client.10299 shutdown 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d99f450 msgr2=0x56233d97a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d99f450 0x56233d97a520 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c02a040 tx=0x7f8b9c01e050 comp rx=0 tx=0).stop 2023-10-06T05:26:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 shutdown_connections 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9a15e0 0x56233d9a19b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d9a84d0 0x56233d959f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d979330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/2002281224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d99f450 0x56233d97a520 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8b85ffb700 0 client.10299 ms_handle_reset on v2:172.21.15.144:6834/2001233142 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 >> 172.21.15.45:0/2002281224 conn(0x56233d9a2880 msgr2=0x56233d954c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 shutdown_connections 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.849+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/2002281224 wait complete. 2023-10-06T05:26:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.850+0000 7f8bb6ebbb80 20 client.10299 trim_cache size 0 max 16384 2023-10-06T05:26:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.868+0000 7f8bb6ebbb80 1 Processor -- start 2023-10-06T05:26:14.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.868+0000 7f8bb6ebbb80 1 -- start start 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.868+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d984f40 0x56233d9593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.868+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d99f450 0x56233d979330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d978150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d7401e0 con 0x56233d9a84d0 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d97ab60 con 0x56233d99f450 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56233d959f60 con 0x56233d984f40 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d978150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d978150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54920/0 (socket says 172.21.15.45:54920) 2023-10-06T05:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 -- 172.21.15.45:0/110607580 learned_addr learned my addr 172.21.15.45:0/110607580 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d984f40 0x56233d9593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba6ffd700 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d99f450 0x56233d979330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 -- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d984f40 msgr2=0x56233d9593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d984f40 0x56233d9593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 -- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d99f450 msgr2=0x56233d979330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d99f450 0x56233d979330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 -- 172.21.15.45:0/110607580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56233d79b0a0 con 0x56233d9a84d0 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d978150 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0051c80 tx=0x7f8ba001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7f855034161f39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8b8d7fa700 1 -- 172.21.15.45:0/110607580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0055e80 con 0x56233d9a84d0 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8b8d7fa700 1 -- 172.21.15.45:0/110607580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0055780 con 0x56233d9a84d0 2023-10-06T05:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.869+0000 7f8b8d7fa700 1 -- 172.21.15.45:0/110607580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00589d0 con 0x56233d9a84d0 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/110607580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 msgr2=0x56233d978150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d978150 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0051c80 tx=0x7f8ba001e040 comp rx=0 tx=0).stop 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/110607580 shutdown_connections 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d978150 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d984f40 0x56233d9593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/110607580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d99f450 0x56233d979330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/110607580 >> 172.21.15.45:0/110607580 conn(0x56233d9a2880 msgr2=0x56233d955fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/110607580 shutdown_connections 2023-10-06T05:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.870+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/110607580 wait complete. 2023-10-06T05:26:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.871+0000 7f8bb6ebbb80 1 -- start start 2023-10-06T05:26:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.871+0000 7f8bb6ebbb80 1 Processor -- start 2023-10-06T05:26:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.872+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d9f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.872+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d958830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.872+0000 7f8bb6ebbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d94e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.872+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d9fc710 con 0x56233d9a84d0 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.872+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56233d97d730 con 0x56233d984f40 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.872+0000 7f8bb6ebbb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56233d9fa2e0 con 0x56233d99f450 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.872+0000 7f8bb6ebbb80 10 client.0 _ll_register_callbacks cb 0x56233d6b7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d94e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d958830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d9f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d94e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54936/0 (socket says 172.21.15.45:54936) 2023-10-06T05:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 -- 172.21.15.45:0/692917842 learned_addr learned my addr 172.21.15.45:0/692917842 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 -- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 msgr2=0x56233d958830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d958830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 -- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 msgr2=0x56233d9f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d9f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 -- 172.21.15.45:0/692917842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba0020070 con 0x56233d9a84d0 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8ba7fff700 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d94e870 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0055480 tx=0x7f8ba007e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b503034eddb792d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8b85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8b85ffb700 1 -- 172.21.15.45:0/692917842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0055e80 con 0x56233d9a84d0 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8b85ffb700 1 -- 172.21.15.45:0/692917842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f8ba005c720 con 0x56233d9a84d0 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b9c000bc0 con 0x56233d9a84d0 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8b85ffb700 1 -- 172.21.15.45:0/692917842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=312) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ba00589d0 con 0x56233d9a84d0 2023-10-06T05:26:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8bb6ebbb80 10 client.10305 fetch_fsmap learned FSMap version 312 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.873+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=312,monmap=2+}) v3 -- 0x56233d8773c0 con 0x56233d9a84d0 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8b85ffb700 1 -- 172.21.15.45:0/692917842 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c000bc0 con 0x56233d9a84d0 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8b85ffb700 1 -- 172.21.15.45:0/692917842 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 312) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8ba0070020 con 0x56233d9a84d0 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8bb6ebbb80 10 client.10305 fetch_fsmap finished waiting for FSMap version 312 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8bb6ebbb80 10 client.10305 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8bb6ebbb80 20 client.10305 populate_metadata read hostname 'smithi045' 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9fb670 0x56233d9fba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8bb6ebbb80 4 client.10305 mds_command: new command op to 9739 tid=0[{"prefix": "status"}] 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 --> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56233d84b8d0 con 0x56233d9fb670 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.874+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9fb670 0x56233d9fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8b85ffb700 10 client.10305 ms_handle_connect on v2:172.21.15.144:6834/2001233142 2023-10-06T05:26:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8ba77fe700 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9fb670 0x56233d9fba40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b17deb12bdba1aa server_cookie=9c4277232e300697 in_seq=0 out_seq=0 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8b85ffb700 1 -- 172.21.15.45:0/692917842 <== mds.0 v2:172.21.15.144:6834/2001233142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56233d84b8d0 con 0x56233d9fb670 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8b85ffb700 10 client.10305 handle_command_reply: tid=0 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 client.10305 shutdown 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 msgr2=0x56233d94e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d94e870 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0055480 tx=0x7f8ba007e000 comp rx=0 tx=0).stop 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 shutdown_connections 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:6834/2001233142,v1:172.21.15.144:6835/2001233142] conn(0x56233d9fb670 0x56233d9fba40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56233d9a84d0 0x56233d94e870 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233d984f40 0x56233d9f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 --2- 172.21.15.45:0/692917842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56233d99f450 0x56233d958830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 >> 172.21.15.45:0/692917842 conn(0x56233d9a2880 msgr2=0x56233d97f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8b85ffb700 0 client.10305 ms_handle_reset on v2:172.21.15.144:6834/2001233142 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 shutdown_connections 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 1 -- 172.21.15.45:0/692917842 wait complete. 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.875+0000 7f8bb6ebbb80 20 client.10305 trim_cache size 0 max 16384 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9739, 2023-10-06T05:26:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8376644710000001, 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 312, 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 126, 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 126, 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 8.820950839 2023-10-06T05:26:14.883 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:26:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8b94066f60 msgr2=0x7f8b94069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8b94066f60 0x7f8b94069410 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8155c80 tx=0x7f8b98008040 comp rx=0 tx=0).stop 2023-10-06T05:26:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba8156ce0 msgr2=0x7f8ba8154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba8156ce0 0x7f8ba8154fe0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000cc90 tx=0x7f8ba0001a80 comp rx=0 tx=0).stop 2023-10-06T05:26:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 shutdown_connections 2023-10-06T05:26:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8b94066f60 0x7f8b94069410 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba8156ce0 0x7f8ba8154fe0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ba806bad0 0x7f8ba80c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 --2- 172.21.15.45:0/2665333833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba80afbe0 0x7f8ba80c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 >> 172.21.15.45:0/2665333833 conn(0x7f8ba806f090 msgr2=0x7f8ba8155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 shutdown_connections 2023-10-06T05:26:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:14.876+0000 7f8badba7700 1 -- 172.21.15.45:0/2665333833 wait complete. 2023-10-06T05:26:14.891 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:26:14.892 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2023-10-06T05:26:14.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:26:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.022+0000 7f6aa84e5700 1 Processor -- start 2023-10-06T05:26:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.022+0000 7f6aa84e5700 1 -- start start 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa84e5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa0155cd0 0x7f6aa015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa84e5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa84e5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00cd740 con 0x7f6aa01560a0 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa84e5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa00ae6b0 con 0x7f6aa0155cd0 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa84e5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00ae8d0 con 0x7f6aa00ccdc0 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa6281700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa0155cd0 0x7f6aa015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa6281700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa0155cd0 0x7f6aa015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54952/0 (socket says 172.21.15.45:54952) 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa6281700 1 -- 172.21.15.45:0/256006647 learned_addr learned my addr 172.21.15.45:0/256006647 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.023+0000 7f6aa6a82700 1 --2- 172.21.15.45:0/256006647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa6281700 1 -- 172.21.15.45:0/256006647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 msgr2=0x7f6aa00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa6281700 1 --2- 172.21.15.45:0/256006647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa6281700 1 -- 172.21.15.45:0/256006647 >> v1:172.21.15.144:6789/0 conn(0x7f6aa01560a0 legacy=0x7f6aa00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa6281700 1 -- 172.21.15.45:0/256006647 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa00aebb0 con 0x7f6aa0155cd0 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa6a82700 1 --2- 172.21.15.45:0/256006647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa6281700 1 --2- 172.21.15.45:0/256006647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa0155cd0 0x7f6aa015a730 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400bc50 tx=0x7f6a9400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecb40eae8fa6cc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa527f700 1 -- 172.21.15.45:0/256006647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400be60 con 0x7f6aa0155cd0 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa527f700 1 -- 172.21.15.45:0/256006647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9400de00 con 0x7f6aa0155cd0 2023-10-06T05:26:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.024+0000 7f6aa527f700 1 -- 172.21.15.45:0/256006647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a940196b0 con 0x7f6aa0155cd0 2023-10-06T05:26:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 -- 172.21.15.45:0/256006647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa0155cd0 msgr2=0x7f6aa015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/256006647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa0155cd0 0x7f6aa015a730 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400bc50 tx=0x7f6a9400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 -- 172.21.15.45:0/256006647 shutdown_connections 2023-10-06T05:26:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/256006647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/256006647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa0155cd0 0x7f6aa015a730 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 -- 172.21.15.45:0/256006647 >> 172.21.15.45:0/256006647 conn(0x7f6aa006f090 msgr2=0x7f6aa006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 -- 172.21.15.45:0/256006647 shutdown_connections 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.025+0000 7f6aa84e5700 1 -- 172.21.15.45:0/256006647 wait complete. 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.026+0000 7f6aa84e5700 1 Processor -- start 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.026+0000 7f6aa84e5700 1 -- start start 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa84e5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa0152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa84e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0155cd0 0x7f6aa00c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa84e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa01560a0 0x7f6aa0169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa84e5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0157530 con 0x7f6aa01560a0 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa84e5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00b1a10 con 0x7f6aa0155cd0 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa84e5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0156920 con 0x7f6aa00ccdc0 2023-10-06T05:26:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa01560a0 0x7f6aa0169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa01560a0 0x7f6aa0169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54962/0 (socket says 172.21.15.45:54962) 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 -- 172.21.15.45:0/359995462 learned_addr learned my addr 172.21.15.45:0/359995462 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6281700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa0152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa5a80700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0155cd0 0x7f6aa00c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 -- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 msgr2=0x7f6aa0152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa0152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 -- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0155cd0 msgr2=0x7f6aa00c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0155cd0 0x7f6aa00c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 -- 172.21.15.45:0/359995462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9400b790 con 0x7f6aa01560a0 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa5a80700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0155cd0 0x7f6aa00c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.027+0000 7f6aa6a82700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa01560a0 0x7f6aa0169780 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800d180 tx=0x7f6a98009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.028+0000 7f6a927fc700 1 -- 172.21.15.45:0/359995462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800de20 con 0x7f6aa01560a0 2023-10-06T05:26:15.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.028+0000 7f6a927fc700 1 -- 172.21.15.45:0/359995462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9800f040 con 0x7f6aa01560a0 2023-10-06T05:26:15.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.028+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa0156c00 con 0x7f6aa01560a0 2023-10-06T05:26:15.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.028+0000 7f6a927fc700 1 -- 172.21.15.45:0/359995462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98015690 con 0x7f6aa01560a0 2023-10-06T05:26:15.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.028+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01571c0 con 0x7f6aa01560a0 2023-10-06T05:26:15.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.029+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa01576e0 con 0x7f6aa01560a0 2023-10-06T05:26:15.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.029+0000 7f6a927fc700 1 -- 172.21.15.45:0/359995462 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a9801e3b0 con 0x7f6aa01560a0 2023-10-06T05:26:15.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.030+0000 7f6a927fc700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.030+0000 7f6a927fc700 1 -- 172.21.15.45:0/359995462 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7f6a98085990 con 0x7f6aa01560a0 2023-10-06T05:26:15.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.030+0000 7f6aa6281700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.030+0000 7f6aa6281700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400bc50 tx=0x7f6a9401b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.034+0000 7f6a927fc700 1 -- 172.21.15.45:0/359995462 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a98051db0 con 0x7f6aa01560a0 2023-10-06T05:26:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.285+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6aa00aee90 con 0x7f6aa01560a0 2023-10-06T05:26:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.286+0000 7f6a927fc700 1 -- 172.21.15.45:0/359995462 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v312) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6a9801a020 con 0x7f6aa01560a0 2023-10-06T05:26:15.288 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:26:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 msgr2=0x7f6a840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400bc50 tx=0x7f6a9401b000 comp rx=0 tx=0).stop 2023-10-06T05:26:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa01560a0 msgr2=0x7f6aa0169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa01560a0 0x7f6aa0169780 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800d180 tx=0x7f6a98009a50 comp rx=0 tx=0).stop 2023-10-06T05:26:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 shutdown_connections 2023-10-06T05:26:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a84066f30 0x7f6a840693e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa01560a0 0x7f6aa0169780 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa00ccdc0 0x7f6aa0152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 --2- 172.21.15.45:0/359995462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0155cd0 0x7f6aa00c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 >> 172.21.15.45:0/359995462 conn(0x7f6aa006f090 msgr2=0x7f6aa00cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 shutdown_connections 2023-10-06T05:26:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:15.288+0000 7f6aa84e5700 1 -- 172.21.15.45:0/359995462 wait complete. 2023-10-06T05:26:15.303 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:26:15.303 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:26:15.303 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:26:15.303 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:26:15.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:15.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:26:15.304 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:26:15.304 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:26:15.304 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:15.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:15.304 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41766sec preferred_lft 41766sec 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:15.327 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:26:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:26:15.328 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:15.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:15.328 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:26:15.397 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:26:15.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:15.399 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:26:15.465 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:26:15.465 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:26:15.465 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:26:15.465 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:26:15.466 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:26:15.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:15.466 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:26:15.466 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:26:15.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:26:15.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:26:15.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:26:15.466 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:26:15.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:15.589 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:26:15.610 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:26:15.610 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:26:15.610 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:15.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:15.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:26:15.686 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:26:15.686 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:15.686 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:15.750 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:15.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:15.750 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:15.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:26:15.890 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:26:15.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:26:15.965 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:26:15.966 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:26:15.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:15.967 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:26:16.030 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:26:16.030 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:26:16.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:26:16.109 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:26:16.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:26:16.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:26:16.172+0000 7f72ba234540 -1 init, newargv = 0x5575127d1a80 newargc=15 2023-10-06T05:26:16.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[177895]: starting ceph client 2023-10-06T05:26:16.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[177895]: starting fuse 2023-10-06T05:26:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:26:16.184 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:26:16.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:16.185 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:26:16.248 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:26:16.249 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:26:16.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:26:16.249 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:26:16.249 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:26:16.250 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:26:16.250 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:26:16.250 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:16.250 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:26:16.250 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:26:16.251 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:26:16.251 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:26:16.251 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:16.251 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:26:16.251 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:26:16.251 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:26:16.252 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:26:16.252 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:16.252 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:26:16.252 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:26:16.252 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:26:16.252 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:26:16.253 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:26:16.253 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:26:16.253 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:26:16.253 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:26:16.253 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:16.254 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:26:16.254 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:26:16.394 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.177895.asok 2023-10-06T05:26:16.576 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.177895.asok 2023-10-06T05:26:16.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:16.576 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.177895.asok status 2023-10-06T05:26:16.751 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:26:16.751 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:26:16.751 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:26:16.751 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:26:16.751 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:26:16.751 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:26:16.752 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:26:16.752 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "177895", 2023-10-06T05:26:16.752 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:26:16.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:16.752 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:26:16.752 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:26:16.752 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 10320, 2023-10-06T05:26:16.753 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:26:16.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:26:16.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:26:16.753 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 10320 2023-10-06T05:26:16.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:16.753 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:26:16.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:26:16.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:26:16.754 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3635489977 2023-10-06T05:26:16.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:26:16.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:16.754 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:26:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:26:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:26:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3635489977 2023-10-06T05:26:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.10320 192.168.0.1:0/3635489977", 2023-10-06T05:26:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/3635489977", 2023-10-06T05:26:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:26:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 312, 2023-10-06T05:26:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 126, 2023-10-06T05:26:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:26:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:26:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:26:16.756 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:26:16.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:16.762 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:26:16.762 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:16.782 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:26:16.783 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:16.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:16.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:16.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:16.856 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:26:16.898 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:26:16.925 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:16.922+0000 7f4234ff2700 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:16.926 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:26:16.926 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:26:16.926 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:16.926 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:26:16.926 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2023-10-06T05:26:16.927 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2023-10-06T05:26:16.927 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:26:16.927 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:16.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:17.013 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:26:17.013 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:26:17.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:17.014 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:17.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:17.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.256+0000 7fd79ff23700 1 Processor -- start 2023-10-06T05:26:17.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.256+0000 7fd79ff23700 1 -- start start 2023-10-06T05:26:17.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79ff23700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7981560a0 0x7fd79815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79ff23700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7980cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79ff23700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7980cd740 con 0x7fd798155cd0 2023-10-06T05:26:17.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79ff23700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7980ae6b0 con 0x7fd7981560a0 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79ff23700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980ae8d0 con 0x7fd7980ccdc0 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79d4be700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7981560a0 0x7fd79815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79dcbf700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd798155cd0 0x7fd7980adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38774/0 (socket says 172.21.15.45:38774) 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79dcbf700 1 -- 172.21.15.45:0/2161020252 learned_addr learned my addr 172.21.15.45:0/2161020252 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79e4c0700 1 --2- 172.21.15.45:0/2161020252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7980cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79d4be700 1 -- 172.21.15.45:0/2161020252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 msgr2=0x7fd7980cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79d4be700 1 --2- 172.21.15.45:0/2161020252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7980cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79d4be700 1 -- 172.21.15.45:0/2161020252 >> v1:172.21.15.45:6789/0 conn(0x7fd798155cd0 legacy=0x7fd7980adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.257+0000 7fd79ccbd700 1 -- 172.21.15.45:0/2161020252 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924853737 0 0) 0x7fd7980cd740 con 0x7fd798155cd0 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79d4be700 1 -- 172.21.15.45:0/2161020252 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7980aebb0 con 0x7fd7981560a0 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79d4be700 1 --2- 172.21.15.45:0/2161020252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd79815a730 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fd7940098e0 tx=0x7fd7940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1692611cee024230 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79ccbd700 1 -- 172.21.15.45:0/2161020252 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794009e80 con 0x7fd7981560a0 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79ccbd700 1 -- 172.21.15.45:0/2161020252 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79400b860 con 0x7fd7981560a0 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79ccbd700 1 -- 172.21.15.45:0/2161020252 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79400bbe0 con 0x7fd7981560a0 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79ff23700 1 -- 172.21.15.45:0/2161020252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 msgr2=0x7fd79815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.258+0000 7fd79ff23700 1 --2- 172.21.15.45:0/2161020252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd79815a730 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fd7940098e0 tx=0x7fd7940091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.259+0000 7fd79ff23700 1 -- 172.21.15.45:0/2161020252 shutdown_connections 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.259+0000 7fd79ff23700 1 --2- 172.21.15.45:0/2161020252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7980cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.259+0000 7fd79ff23700 1 --2- 172.21.15.45:0/2161020252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd79815a730 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.259+0000 7fd79ff23700 1 -- 172.21.15.45:0/2161020252 >> 172.21.15.45:0/2161020252 conn(0x7fd79806f090 msgr2=0x7fd79806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.259+0000 7fd79ff23700 1 -- 172.21.15.45:0/2161020252 shutdown_connections 2023-10-06T05:26:17.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.259+0000 7fd79ff23700 1 -- 172.21.15.45:0/2161020252 wait complete. 2023-10-06T05:26:17.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.260+0000 7fd79ff23700 1 Processor -- start 2023-10-06T05:26:17.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.260+0000 7fd79ff23700 1 -- start start 2023-10-06T05:26:17.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79ff23700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7981552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79ff23700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798155cd0 0x7fd798168670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79ff23700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd7980c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79ff23700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980a45d0 con 0x7fd7981560a0 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79ff23700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980a4e70 con 0x7fd798155cd0 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79ff23700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980a77d0 con 0x7fd7980ccdc0 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79e4c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd7980c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79e4c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd7980c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34574/0 (socket says 172.21.15.45:34574) 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79dcbf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7981552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.261+0000 7fd79e4c0700 1 -- 172.21.15.45:0/1564329037 learned_addr learned my addr 172.21.15.45:0/1564329037 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79d4be700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798155cd0 0x7fd798168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79e4c0700 1 -- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 msgr2=0x7fd7981552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79e4c0700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7981552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79e4c0700 1 -- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798155cd0 msgr2=0x7fd798168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79e4c0700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798155cd0 0x7fd798168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79e4c0700 1 -- 172.21.15.45:0/1564329037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd794009420 con 0x7fd7981560a0 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79e4c0700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd7980c7120 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd790009fd0 tx=0x7fd790009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd78dffb700 1 -- 172.21.15.45:0/1564329037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79000ebf0 con 0x7fd7981560a0 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd78dffb700 1 -- 172.21.15.45:0/1564329037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79000ed50 con 0x7fd7981560a0 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7980a4820 con 0x7fd7981560a0 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd78dffb700 1 -- 172.21.15.45:0/1564329037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790014670 con 0x7fd7981560a0 2023-10-06T05:26:17.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.262+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd798155820 con 0x7fd7981560a0 2023-10-06T05:26:17.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.263+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79806e2f0 con 0x7fd7981560a0 2023-10-06T05:26:17.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.264+0000 7fd78dffb700 1 -- 172.21.15.45:0/1564329037 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7900149e0 con 0x7fd7981560a0 2023-10-06T05:26:17.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.264+0000 7fd78dffb700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd77c066f30 0x7fd77c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.264+0000 7fd78dffb700 1 -- 172.21.15.45:0/1564329037 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9302+0+0 (secure 0 0 0) 0x7fd790085a70 con 0x7fd7981560a0 2023-10-06T05:26:17.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.264+0000 7fd79dcbf700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd77c066f30 0x7fd77c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.265+0000 7fd79dcbf700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd77c066f30 0x7fd77c0693e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd788007fd0 tx=0x7fd788007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.269+0000 7fd78dffb700 1 -- 172.21.15.45:0/1564329037 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd790051e90 con 0x7fd7981560a0 2023-10-06T05:26:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.516+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd7980a4fb0 con 0x7fd7981560a0 2023-10-06T05:26:17.518 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:17.517+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:17.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.520+0000 7fd78dffb700 1 -- 172.21.15.45:0/1564329037 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v313) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd7900554b0 con 0x7fd7981560a0 2023-10-06T05:26:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd77c066f30 msgr2=0x7fd77c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd77c066f30 0x7fd77c0693e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd788007fd0 tx=0x7fd788007d30 comp rx=0 tx=0).stop 2023-10-06T05:26:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 msgr2=0x7fd7980c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd7980c7120 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fd790009fd0 tx=0x7fd790009900 comp rx=0 tx=0).stop 2023-10-06T05:26:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 shutdown_connections 2023-10-06T05:26:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd77c066f30 0x7fd77c0693e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7981560a0 0x7fd7980c7120 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980ccdc0 0x7fd7981552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 --2- 172.21.15.45:0/1564329037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798155cd0 0x7fd798168670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 >> 172.21.15.45:0/1564329037 conn(0x7fd79806f090 msgr2=0x7fd798159720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 shutdown_connections 2023-10-06T05:26:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.523+0000 7fd79ff23700 1 -- 172.21.15.45:0/1564329037 wait complete. 2023-10-06T05:26:17.527 INFO:tasks.ceph.mds.b.smithi144.stderr: -606> 2023-10-06T05:26:13.035+0000 7f42387f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:17.527 INFO:tasks.ceph.mds.b.smithi144.stderr: -89> 2023-10-06T05:26:16.922+0000 7f4234ff2700 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:17.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:26:17.549 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:26:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.669+0000 7f327a7d0700 1 Processor -- start 2023-10-06T05:26:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.669+0000 7f327a7d0700 1 -- start start 2023-10-06T05:26:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f327a7d0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3274156ce0 0x7f32741570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f327a7d0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f327406bad0 0x7f327415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f327a7d0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327415b150 con 0x7f32740afbe0 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f327a7d0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327406bf40 con 0x7f327406bad0 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f327a7d0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327415b380 con 0x7f3274156ce0 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f3278d6d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32740afbe0 0x7f32740ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38784/0 (socket says 172.21.15.45:38784) 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f32737fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f327406bad0 0x7f327415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f3278d6d700 1 -- 172.21.15.45:0/2798972132 learned_addr learned my addr 172.21.15.45:0/2798972132 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f3273fff700 1 --2- 172.21.15.45:0/2798972132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3274156ce0 0x7f32741570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f32737fe700 1 -- 172.21.15.45:0/2798972132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3274156ce0 msgr2=0x7f32741570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f32737fe700 1 --2- 172.21.15.45:0/2798972132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3274156ce0 0x7f32741570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f32737fe700 1 -- 172.21.15.45:0/2798972132 >> v1:172.21.15.45:6789/0 conn(0x7f32740afbe0 legacy=0x7f32740ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f32737fe700 1 -- 172.21.15.45:0/2798972132 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327415b660 con 0x7f327406bad0 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f3272ffd700 1 -- 172.21.15.45:0/2798972132 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137975231 0 0) 0x7f327415b150 con 0x7f32740afbe0 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.670+0000 7f32737fe700 1 --2- 172.21.15.45:0/2798972132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f327406bad0 0x7f327415ac10 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f32680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cb9cf3162c6d9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f3272ffd700 1 -- 172.21.15.45:0/2798972132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800b640 con 0x7f327406bad0 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f3272ffd700 1 -- 172.21.15.45:0/2798972132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326800b7a0 con 0x7f327406bad0 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f3272ffd700 1 -- 172.21.15.45:0/2798972132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326800bb40 con 0x7f327406bad0 2023-10-06T05:26:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 -- 172.21.15.45:0/2798972132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f327406bad0 msgr2=0x7f327415ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 --2- 172.21.15.45:0/2798972132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f327406bad0 0x7f327415ac10 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f32680091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 -- 172.21.15.45:0/2798972132 shutdown_connections 2023-10-06T05:26:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 --2- 172.21.15.45:0/2798972132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3274156ce0 0x7f32741570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 --2- 172.21.15.45:0/2798972132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f327406bad0 0x7f327415ac10 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 -- 172.21.15.45:0/2798972132 >> 172.21.15.45:0/2798972132 conn(0x7f327406f090 msgr2=0x7f327406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 -- 172.21.15.45:0/2798972132 shutdown_connections 2023-10-06T05:26:17.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.671+0000 7f327a7d0700 1 -- 172.21.15.45:0/2798972132 wait complete. 2023-10-06T05:26:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.672+0000 7f327a7d0700 1 Processor -- start 2023-10-06T05:26:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.673+0000 7f327a7d0700 1 -- start start 2023-10-06T05:26:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.673+0000 7f327a7d0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f327406bad0 0x7f32740c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.673+0000 7f327a7d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32740afbe0 0x7f32740c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.673+0000 7f327a7d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3274156ce0 0x7f32740c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.673+0000 7f327a7d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32740c8ce0 con 0x7f3274156ce0 2023-10-06T05:26:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.673+0000 7f327a7d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32740c7700 con 0x7f32740afbe0 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.673+0000 7f327a7d0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32740c79e0 con 0x7f327406bad0 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3274156ce0 0x7f32740c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3274156ce0 0x7f32740c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34596/0 (socket says 172.21.15.45:34596) 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 -- 172.21.15.45:0/699095737 learned_addr learned my addr 172.21.15.45:0/699095737 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f32737fe700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32740afbe0 0x7f32740c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3273fff700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f327406bad0 0x7f32740c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 -- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f327406bad0 msgr2=0x7f32740c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f327406bad0 0x7f32740c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 -- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32740afbe0 msgr2=0x7f32740c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32740afbe0 0x7f32740c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 -- 172.21.15.45:0/699095737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268009420 con 0x7f3274156ce0 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f3278d6d700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3274156ce0 0x7f32740c1270 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f326c00cc90 tx=0x7f326c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f325bfff700 1 -- 172.21.15.45:0/699095737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c00a870 con 0x7f3274156ce0 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f325bfff700 1 -- 172.21.15.45:0/699095737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c00a9d0 con 0x7f3274156ce0 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32740c7cc0 con 0x7f3274156ce0 2023-10-06T05:26:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f325bfff700 1 -- 172.21.15.45:0/699095737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c00ad90 con 0x7f3274156ce0 2023-10-06T05:26:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.674+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32740c8280 con 0x7f3274156ce0 2023-10-06T05:26:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.675+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32740c84a0 con 0x7f3274156ce0 2023-10-06T05:26:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.676+0000 7f325bfff700 1 -- 172.21.15.45:0/699095737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f326c007300 con 0x7f3274156ce0 2023-10-06T05:26:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.676+0000 7f325bfff700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3260066f30 0x7f32600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.676+0000 7f325bfff700 1 -- 172.21.15.45:0/699095737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9388+0+0 (secure 0 0 0) 0x7f326c085800 con 0x7f3274156ce0 2023-10-06T05:26:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.676+0000 7f3273fff700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3260066f30 0x7f32600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.677+0000 7f3273fff700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3260066f30 0x7f32600693e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3264009d10 tx=0x7f32640091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.681+0000 7f325bfff700 1 -- 172.21.15.45:0/699095737 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f326c051b60 con 0x7f3274156ce0 2023-10-06T05:26:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:17.965+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f3274153fe0 con 0x7f3274156ce0 2023-10-06T05:26:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.522+0000 7f325bfff700 1 -- 172.21.15.45:0/699095737 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v314) v1 ==== 107+0+0 (secure 0 0 0) 0x7f326c055180 con 0x7f3274156ce0 2023-10-06T05:26:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3260066f30 msgr2=0x7f32600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3260066f30 0x7f32600693e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3264009d10 tx=0x7f32640091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3274156ce0 msgr2=0x7f32740c1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3274156ce0 0x7f32740c1270 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f326c00cc90 tx=0x7f326c001a20 comp rx=0 tx=0).stop 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 shutdown_connections 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3260066f30 0x7f32600693e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3274156ce0 0x7f32740c1270 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f327406bad0 0x7f32740c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 --2- 172.21.15.45:0/699095737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32740afbe0 0x7f32740c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.524+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 >> 172.21.15.45:0/699095737 conn(0x7f327406f090 msgr2=0x7f3274068ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.525+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 shutdown_connections 2023-10-06T05:26:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.525+0000 7f327a7d0700 1 -- 172.21.15.45:0/699095737 wait complete. 2023-10-06T05:26:18.528 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:26:18.542 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:26:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.587+0000 7f228e220040 1 Processor -- start 2023-10-06T05:26:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.587+0000 7f228e220040 1 -- start start 2023-10-06T05:26:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f228e220040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099dd52120 0x56099dd55930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f228e220040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd56450 0x56099dd6c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f228e220040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099ddfcfc0 0x56099dd28190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f228e220040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099dc71980 con 0x56099dd56450 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f228e220040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099dc71620 con 0x56099ddfcfc0 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f228e220040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56099dc68400 con 0x56099dd52120 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f2286c3d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd56450 0x56099dd6c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f2286c3d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd56450 0x56099dd6c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34604/0 (socket says 172.21.15.45:34604) 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f2286c3d700 1 -- 172.21.15.45:0/1822283900 learned_addr learned my addr 172.21.15.45:0/1822283900 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f2287c3f700 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099ddfcfc0 0x56099dd28190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.588+0000 7f228743e700 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099dd52120 0x56099dd55930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2286c3d700 1 -- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099dd52120 msgr2=0x56099dd55930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2286c3d700 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099dd52120 0x56099dd55930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2286c3d700 1 -- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099ddfcfc0 msgr2=0x56099dd28190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2286c3d700 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099ddfcfc0 0x56099dd28190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2286c3d700 1 -- 172.21.15.45:0/1822283900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2270005db0 con 0x56099dd56450 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2286c3d700 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd56450 0x56099dd6c260 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2270009a30 tx=0x7f227000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa417704b70d931 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2285c3b700 1 -- 172.21.15.45:0/1822283900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227000f040 con 0x56099dd56450 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2285c3b700 1 -- 172.21.15.45:0/1822283900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2270009e70 con 0x56099dd56450 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.589+0000 7f2285c3b700 1 -- 172.21.15.45:0/1822283900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270008d00 con 0x56099dd56450 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 -- 172.21.15.45:0/1822283900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd56450 msgr2=0x56099dd6c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd56450 0x56099dd6c260 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2270009a30 tx=0x7f227000c040 comp rx=0 tx=0).stop 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 -- 172.21.15.45:0/1822283900 shutdown_connections 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099ddfcfc0 0x56099dd28190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099dd52120 0x56099dd55930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 --2- 172.21.15.45:0/1822283900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd56450 0x56099dd6c260 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 -- 172.21.15.45:0/1822283900 >> 172.21.15.45:0/1822283900 conn(0x56099dd26bd0 msgr2=0x56099ddfb300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 -- 172.21.15.45:0/1822283900 shutdown_connections 2023-10-06T05:26:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 -- 172.21.15.45:0/1822283900 wait complete. 2023-10-06T05:26:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 Processor -- start 2023-10-06T05:26:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.590+0000 7f228e220040 1 -- start start 2023-10-06T05:26:18.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228e220040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd52120 0x56099ddf3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228e220040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099dd56450 0x56099ddf2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228e220040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099ddfcfc0 0x56099ddfa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228e220040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099dd29360 con 0x56099dd52120 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228e220040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099ddfe4e0 con 0x56099dd56450 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228e220040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56099ddfe780 con 0x56099ddfcfc0 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228743e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd52120 0x56099ddf3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228743e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd52120 0x56099ddf3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34612/0 (socket says 172.21.15.45:34612) 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.591+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 learned_addr learned my addr 172.21.15.45:0/3987210936 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099ddfcfc0 0x56099ddfa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099dd56450 0x56099ddf2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099ddfcfc0 msgr2=0x56099ddfa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228743e700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099ddfcfc0 0x56099ddfa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099dd56450 msgr2=0x56099ddf2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228743e700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099dd56450 0x56099ddf2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2270009630 con 0x56099dd52120 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56099ddfcfc0 0x56099ddfa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56099dd56450 0x56099ddf2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228743e700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd52120 0x56099ddf3920 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x56099dd2a3a0 tx=0x7f2280016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f227ffff700 1 -- 172.21.15.45:0/3987210936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280008970 con 0x56099dd52120 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f227ffff700 1 -- 172.21.15.45:0/3987210936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2280008af0 con 0x56099dd52120 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2280005cf0 con 0x56099dd52120 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f227ffff700 1 -- 172.21.15.45:0/3987210936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228001e680 con 0x56099dd52120 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.592+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56099dc803d0 con 0x56099dd52120 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.594+0000 7f227ffff700 1 -- 172.21.15.45:0/3987210936 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f228001ea30 con 0x56099dd52120 2023-10-06T05:26:18.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.594+0000 7f227ffff700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22680658c0 0x7f2268067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.594+0000 7f227ffff700 1 -- 172.21.15.45:0/3987210936 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9388+0+0 (secure 0 0 0) 0x7f2280086410 con 0x56099dd52120 2023-10-06T05:26:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.594+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22680658c0 0x7f2268067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.600 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:26:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.594+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56099ddf3300 0x56099ddff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.594+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099dd51c20 con 0x56099ddf3300 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.595+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56099ddf3300 0x56099ddff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.595+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22680658c0 0x7f2268067d70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2270009950 tx=0x7f2270018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.595+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56099ddf3300 0x56099ddff160 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.597+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v127'1065 uv1038 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.597+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099ddff6a0 con 0x56099ddf3300 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.599+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v127'1066 uv1066 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.599+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099ddf8900 con 0x56099ddf3300 2023-10-06T05:26:18.603 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:26:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.600+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v127'1067 uv1067 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.600+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1acd0 con 0x56099ddf3300 2023-10-06T05:26:18.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.602+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'1068 uv1068 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.602+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b080 con 0x56099ddf3300 2023-10-06T05:26:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.603+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'1069 uv1069 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.603+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1a1b0 con 0x56099ddf3300 2023-10-06T05:26:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.604+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'1070 uv1070 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.604+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1a5b0 con 0x56099ddf3300 2023-10-06T05:26:18.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.606+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v127'1071 uv1044 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.606+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.607+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v127'1072 uv1072 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.607+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.610 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:26:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.608+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v127'1073 uv1073 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.608+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.610+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'1074 uv1074 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.610+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.611+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'1075 uv1075 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.611+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.612+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'1076 uv1076 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.612+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.614+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v127'1077 uv1064 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.614+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.615+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'1078 uv1078 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.615+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.616+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v127'1079 uv1079 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.618 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:26:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.616+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.617+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'1080 uv1080 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.617+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.619+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'1081 uv1081 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.619+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.620+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'1082 uv1082 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.620+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.621+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'1083 uv1083 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.621+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.623+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v127'1084 uv1057 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.623+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.624+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v127'1085 uv1085 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.624+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.629 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:26:18.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.625+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v127'1086 uv1086 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.625+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.627+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'1087 uv1087 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.627+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.628+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'1088 uv1088 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.628+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.629+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'1089 uv1089 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.629+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.631+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v127'1090 uv1063 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.631+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.632+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v127'1091 uv1091 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.632+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.633+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v127'1092 uv1092 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.635 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:26:18.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.633+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.635+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'1093 uv1093 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.635+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1c440 con 0x56099ddf3300 2023-10-06T05:26:18.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.636+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'1094 uv1094 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.636+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1b430 con 0x56099ddf3300 2023-10-06T05:26:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.637+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'1095 uv1095 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099ddf3300 2023-10-06T05:26:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.637+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56099de1b940 0x56099de1dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.637+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1e260 con 0x56099de1b940 2023-10-06T05:26:18.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.637+0000 7f228743e700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56099de1b940 0x56099de1dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.638+0000 7f228743e700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56099de1b940 0x56099de1dd20 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.642+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'768 uv748 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.642+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1e610 con 0x56099de1b940 2023-10-06T05:26:18.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.644+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'769 uv769 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.644+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1e9c0 con 0x56099de1b940 2023-10-06T05:26:18.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.645+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'770 uv770 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.647 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:26:18.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.645+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1f260 con 0x56099de1b940 2023-10-06T05:26:18.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.646+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v127'771 uv771 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.646+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1f610 con 0x56099de1b940 2023-10-06T05:26:18.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.648+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'772 uv772 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.648+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1f9c0 con 0x56099de1b940 2023-10-06T05:26:18.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.649+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v127'773 uv767 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.649+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de1fd70 con 0x56099de1b940 2023-10-06T05:26:18.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.650+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v127'774 uv774 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.650+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.651+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v127'775 uv775 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.654 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:26:18.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.651+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.653+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'776 uv776 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.653+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.654+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'777 uv777 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.654+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.655+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v127'778 uv766 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.655+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.657+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v127'779 uv779 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.657+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.658+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v127'780 uv780 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.660 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:26:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.658+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de23840 con 0x56099de1b940 2023-10-06T05:26:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.659+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v127'781 uv781 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.659+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de23840 con 0x56099de1b940 2023-10-06T05:26:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.661+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'782 uv782 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.661+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.662+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v127'783 uv764 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.662+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.663+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v127'784 uv784 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.663+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.665+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v127'785 uv785 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.667 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:26:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.665+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de21130 con 0x56099de1b940 2023-10-06T05:26:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.666+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'786 uv786 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.666+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de21130 con 0x56099de1b940 2023-10-06T05:26:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.667+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'787 uv787 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.667+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de20120 con 0x56099de1b940 2023-10-06T05:26:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.669+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'788 uv788 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.669+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56099de20630 0x56099de22a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.669+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de22f50 con 0x56099de20630 2023-10-06T05:26:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.669+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56099de20630 0x56099de22a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.669+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56099de20630 0x56099de22a10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.673+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.673+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de23380 con 0x56099de20630 2023-10-06T05:26:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.674+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v127'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.675+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de23900 con 0x56099de20630 2023-10-06T05:26:18.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.676+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v127'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.678 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:26:18.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.676+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de25230 con 0x56099de20630 2023-10-06T05:26:18.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.677+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.677+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de255e0 con 0x56099de20630 2023-10-06T05:26:18.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.678+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.678+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24220 con 0x56099de20630 2023-10-06T05:26:18.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.679+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.679+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24220 con 0x56099de20630 2023-10-06T05:26:18.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.681+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'149 uv148 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.681+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24220 con 0x56099de20630 2023-10-06T05:26:18.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.682+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'150 uv150 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.682+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24220 con 0x56099de20630 2023-10-06T05:26:18.685 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:26:18.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.684+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'151 uv151 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.684+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24640 con 0x56099de20630 2023-10-06T05:26:18.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.685+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'152 uv152 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.685+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24640 con 0x56099de1b940 2023-10-06T05:26:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.686+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.686+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24640 con 0x56099de1b940 2023-10-06T05:26:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.688+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v127'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.688+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24640 con 0x56099de1b940 2023-10-06T05:26:18.691 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.689+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v127'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.689+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24220 con 0x56099de1b940 2023-10-06T05:26:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.690+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.690+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24220 con 0x56099de1b940 2023-10-06T05:26:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.691+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.691+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de24220 con 0x56099de1b940 2023-10-06T05:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.692+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de1b940 2023-10-06T05:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.693+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56099de24b50 0x56099de275f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.693+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.693+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56099de24b50 0x56099de275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.693+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56099de24b50 0x56099de275f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.695+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v127'382 uv376 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.695+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.696+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v127'383 uv383 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.696+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.698 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:26:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.697+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v127'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.697+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de28b40 con 0x56099de24b50 2023-10-06T05:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.698+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'385 uv385 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.698+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de28b40 con 0x56099de24b50 2023-10-06T05:26:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.699+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'386 uv386 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.699+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.701+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'387 uv387 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.701+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.702+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v127'388 uv381 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.702+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.703+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v127'389 uv389 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.703+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.706 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:26:18.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.704+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v127'390 uv390 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.704+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.706+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'391 uv391 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.706+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de27b30 con 0x56099de24b50 2023-10-06T05:26:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.707+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'392 uv392 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.707+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56099de281b0 0x56099de2a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.707+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2ac10 con 0x56099de281b0 2023-10-06T05:26:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.708+0000 7f228743e700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56099de281b0 0x56099de2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.708+0000 7f228743e700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56099de281b0 0x56099de2a590 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.709+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v127'173 uv172 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de281b0 2023-10-06T05:26:18.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.709+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2ac10 con 0x56099de281b0 2023-10-06T05:26:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.711+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v127'174 uv174 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de281b0 2023-10-06T05:26:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.711+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2ac10 con 0x56099de281b0 2023-10-06T05:26:18.713 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:26:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.712+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v127'175 uv175 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de281b0 2023-10-06T05:26:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.712+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2b0c0 con 0x56099de281b0 2023-10-06T05:26:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.713+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'176 uv176 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de281b0 2023-10-06T05:26:18.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.713+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2b0c0 con 0x56099de281b0 2023-10-06T05:26:18.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.715+0000 7f228743e700 1 -- 172.21.15.45:0/3987210936 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'177 uv177 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2280005a50 con 0x56099de281b0 2023-10-06T05:26:18.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.715+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2b0c0 con 0x56099de20630 2023-10-06T05:26:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.715+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.715+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2b0c0 con 0x56099de20630 2023-10-06T05:26:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.717+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v127'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.717+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2b0c0 con 0x56099de20630 2023-10-06T05:26:18.719 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:26:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.718+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v127'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.718+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2c0d0 con 0x56099de20630 2023-10-06T05:26:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.719+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.719+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2c0d0 con 0x56099de20630 2023-10-06T05:26:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.720+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.720+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2b0c0 con 0x56099de20630 2023-10-06T05:26:18.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.722+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.722+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56099de2b5d0 0x56099de2d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.722+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e010 con 0x56099de2b5d0 2023-10-06T05:26:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.722+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56099de2b5d0 0x56099de2d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.722+0000 7f2286c3d700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56099de2b5d0 0x56099de2d9b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.724+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v127'345 uv339 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.724+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e010 con 0x56099de2b5d0 2023-10-06T05:26:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.725+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v127'346 uv346 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.725+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e470 con 0x56099de2b5d0 2023-10-06T05:26:18.728 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:26:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.727+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v127'347 uv347 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.727+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e470 con 0x56099de2b5d0 2023-10-06T05:26:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.728+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'348 uv348 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.728+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e470 con 0x56099de2b5d0 2023-10-06T05:26:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.729+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'349 uv349 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.729+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e470 con 0x56099de2b5d0 2023-10-06T05:26:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.731+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v127'350 uv344 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.731+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e470 con 0x56099de2b5d0 2023-10-06T05:26:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.732+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v127'351 uv351 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.732+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e470 con 0x56099de2b5d0 2023-10-06T05:26:18.734 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:26:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.733+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v127'352 uv352 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.733+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e470 con 0x56099de2b5d0 2023-10-06T05:26:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.734+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'353 uv353 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.734+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de311e0 con 0x56099de2b5d0 2023-10-06T05:26:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.736+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'354 uv354 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de2b5d0 2023-10-06T05:26:18.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.736+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e960 con 0x56099de20630 2023-10-06T05:26:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.737+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v127'152 uv150 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.737+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e960 con 0x56099de20630 2023-10-06T05:26:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.738+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'153 uv153 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.738+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e960 con 0x56099de20630 2023-10-06T05:26:18.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.740+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v127'154 uv154 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.741 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:26:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.740+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de2e960 con 0x56099de20630 2023-10-06T05:26:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.741+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'155 uv155 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.741+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56099de2ee70 0x56099de2f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.741+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de2ee70 2023-10-06T05:26:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.741+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56099de2ee70 0x56099de2f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.742+0000 7f2287c3f700 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56099de2ee70 0x56099de2f260 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.743+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de2ee70 2023-10-06T05:26:18.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.743+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de2ee70 2023-10-06T05:26:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.744+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v127'208 uv208 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de2ee70 2023-10-06T05:26:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.744+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de2ee70 2023-10-06T05:26:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.745+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v127'209 uv209 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de2ee70 2023-10-06T05:26:18.746 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:26:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.745+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de32940 con 0x56099de2ee70 2023-10-06T05:26:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.746+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de2ee70 2023-10-06T05:26:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.746+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de32940 con 0x56099de2ee70 2023-10-06T05:26:18.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.747+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de2ee70 2023-10-06T05:26:18.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.747+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de2ee70 2023-10-06T05:26:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.748+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de2ee70 2023-10-06T05:26:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.748+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de24b50 2023-10-06T05:26:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.749+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v127'176 uv175 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.749+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de24b50 2023-10-06T05:26:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.751+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v127'177 uv177 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.751+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de24b50 2023-10-06T05:26:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.752+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v127'178 uv178 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.754 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:26:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.752+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de24b50 2023-10-06T05:26:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.753+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'179 uv179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.754+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de24b50 2023-10-06T05:26:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.755+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'180 uv180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f2278003850 con 0x56099de24b50 2023-10-06T05:26:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.755+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.756+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v127'429 uv420 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.756+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.758+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v127'430 uv430 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.758+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.759+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v127'431 uv431 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.761 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:26:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.759+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.761+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v127'432 uv432 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.761+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.762+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'433 uv433 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.762+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.763+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v127'434 uv427 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.763+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.765+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v127'435 uv435 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.765+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.766+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v127'436 uv436 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.768 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:26:18.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.766+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de32940 con 0x56099de20630 2023-10-06T05:26:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.767+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'437 uv437 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.767+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de32940 con 0x56099de20630 2023-10-06T05:26:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.769+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'438 uv438 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.769+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31930 con 0x56099de20630 2023-10-06T05:26:18.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.770+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'439 uv439 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.770+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x56099de31e20 con 0x56099de20630 2023-10-06T05:26:18.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.771+0000 7f2286c3d700 1 -- 172.21.15.45:0/3987210936 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'440 uv440 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2270019020 con 0x56099de20630 2023-10-06T05:26:18.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56099de20630 msgr2=0x56099de22a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x56099de20630 0x56099de22a10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56099ddf3300 msgr2=0x56099ddff160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x56099ddf3300 0x56099ddff160 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56099de2b5d0 msgr2=0x56099de2d9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x56099de2b5d0 0x56099de2d9b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56099de1b940 msgr2=0x56099de1dd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x56099de1b940 0x56099de1dd20 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56099de2ee70 msgr2=0x56099de2f260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x56099de2ee70 0x56099de2f260 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56099de281b0 msgr2=0x56099de2a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x56099de281b0 0x56099de2a590 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56099de24b50 msgr2=0x56099de275f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x56099de24b50 0x56099de275f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 reap_dead start 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 reap_dead start 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 reap_dead start 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f2287c3f700 1 -- 172.21.15.45:0/3987210936 reap_dead start 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22680658c0 msgr2=0x7f2268067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22680658c0 0x7f2268067d70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2270009950 tx=0x7f2270018000 comp rx=0 tx=0).stop 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd52120 msgr2=0x56099ddf3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd52120 0x56099ddf3920 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x56099dd2a3a0 tx=0x7f2280016000 comp rx=0 tx=0).stop 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 shutdown_connections 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56099dd52120 0x56099ddf3920 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 --2- 172.21.15.45:0/3987210936 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f22680658c0 0x7f2268067d70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 >> 172.21.15.45:0/3987210936 conn(0x56099dd26bd0 msgr2=0x56099dd52d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 shutdown_connections 2023-10-06T05:26:18.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.772+0000 7f228e220040 1 -- 172.21.15.45:0/3987210936 wait complete. 2023-10-06T05:26:18.777 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:26:18.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.867+0000 7f4b3e167040 1 Processor -- start 2023-10-06T05:26:18.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.867+0000 7f4b3e167040 1 -- start start 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b3e167040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be09789b0 0x557be0930c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b3e167040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a01b70 0x557be0a009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b3e167040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be0a06140 0x557be0a00f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b3e167040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be0878980 con 0x557be0a06140 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b3e167040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be0878620 con 0x557be0a01b70 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b3e167040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557be086f400 con 0x557be09789b0 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b37b86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be0a06140 0x557be0a00f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b37385700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be09789b0 0x557be0930c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b37b86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be0a06140 0x557be0a00f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34622/0 (socket says 172.21.15.45:34622) 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b37385700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be09789b0 0x557be0930c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:52682/0 (socket says 172.21.15.45:52682) 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b37b86700 1 -- 172.21.15.45:0/4239003474 learned_addr learned my addr 172.21.15.45:0/4239003474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.868+0000 7f4b36b84700 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a01b70 0x557be0a009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b37385700 1 -- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a01b70 msgr2=0x557be0a009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b37385700 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a01b70 0x557be0a009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b37385700 1 -- 172.21.15.45:0/4239003474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be0a06140 msgr2=0x557be0a00f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b37385700 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be0a06140 0x557be0a00f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b37385700 1 -- 172.21.15.45:0/4239003474 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3000a180 con 0x557be09789b0 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b37385700 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be09789b0 0x557be0930c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b280098a0 tx=0x7f4b2800c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e68430b84b22e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b35b82700 1 -- 172.21.15.45:0/4239003474 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800f040 con 0x557be09789b0 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b35b82700 1 -- 172.21.15.45:0/4239003474 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b280088b0 con 0x557be09789b0 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.869+0000 7f4b35b82700 1 -- 172.21.15.45:0/4239003474 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28008c70 con 0x557be09789b0 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 -- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be09789b0 msgr2=0x557be0930c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be09789b0 0x557be0930c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b280098a0 tx=0x7f4b2800c040 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 -- 172.21.15.45:0/4239003474 shutdown_connections 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be0a06140 0x557be0a00f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be09789b0 0x557be0930c40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 --2- 172.21.15.45:0/4239003474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a01b70 0x557be0a009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 -- 172.21.15.45:0/4239003474 >> 172.21.15.45:0/4239003474 conn(0x557be092f5f0 msgr2=0x557be095ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 -- 172.21.15.45:0/4239003474 shutdown_connections 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 -- 172.21.15.45:0/4239003474 wait complete. 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.870+0000 7f4b3e167040 1 Processor -- start 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b3e167040 1 -- start start 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b3e167040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be09789b0 0x557be095b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b3e167040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be0a01b70 0x557be095ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b3e167040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a06140 0x557be095e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b3e167040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be095b570 con 0x557be09789b0 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b3e167040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be096fec0 con 0x557be0a06140 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b3e167040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x557be0970160 con 0x557be0a01b70 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.871+0000 7f4b37385700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be09789b0 0x557be095b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be09789b0 0x557be095b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34634/0 (socket says 172.21.15.45:34634) 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 -- 172.21.15.45:0/3581895354 learned_addr learned my addr 172.21.15.45:0/3581895354 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b36b84700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be0a01b70 0x557be095ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37b86700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a06140 0x557be095e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 -- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be0a01b70 msgr2=0x557be095ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557be0a01b70 0x557be095ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 -- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a06140 msgr2=0x557be095e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557be0a06140 0x557be095e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 -- 172.21.15.45:0/3581895354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b280094d0 con 0x557be09789b0 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b37385700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be09789b0 0x557be095b030 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005fd0 tx=0x7f4b28018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b1ffff700 1 -- 172.21.15.45:0/3581895354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800f070 con 0x557be09789b0 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b1ffff700 1 -- 172.21.15.45:0/3581895354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b28008a20 con 0x557be09789b0 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b1ffff700 1 -- 172.21.15.45:0/3581895354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28025630 con 0x557be09789b0 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557be08873d0 con 0x557be09789b0 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.872+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557be087aae0 con 0x557be09789b0 2023-10-06T05:26:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.874+0000 7f4b1ffff700 1 -- 172.21.15.45:0/3581895354 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b28025a60 con 0x557be09789b0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.874+0000 7f4b1ffff700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b24065890 0x7f4b24067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.874+0000 7f4b1ffff700 1 -- 172.21.15.45:0/3581895354 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9388+0+0 (secure 0 0 0) 0x7f4b28086090 con 0x557be09789b0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.875+0000 7f4b36b84700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b24065890 0x7f4b24067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.875+0000 7f4b3e167040 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557be09712f0 0x557be09716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.875+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x557be0971c30 con 0x557be09712f0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.875+0000 7f4b37b86700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557be09712f0 0x557be09716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.875+0000 7f4b36b84700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b24065890 0x7f4b24067d40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c001840 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.875+0000 7f4b37b86700 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557be09712f0 0x557be09716f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b37b86700 1 -- 172.21.15.45:0/3581895354 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v127'1096 uv1096 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4b3000e7e0 con 0x557be09712f0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557be09712f0 msgr2=0x557be09716f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x557be09712f0 0x557be09716f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b24065890 msgr2=0x7f4b24067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b24065890 0x7f4b24067d40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c001840 comp rx=0 tx=0).stop 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be09789b0 msgr2=0x557be095b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 --2- 172.21.15.45:0/3581895354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557be09789b0 0x557be095b030 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005fd0 tx=0x7f4b28018000 comp rx=0 tx=0).stop 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b37b86700 1 -- 172.21.15.45:0/3581895354 reap_dead start 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 shutdown_connections 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 >> 172.21.15.45:0/3581895354 conn(0x557be092f5f0 msgr2=0x557be0974c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 shutdown_connections 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:18.877+0000 7f4b3e167040 1 -- 172.21.15.45:0/3581895354 wait complete. 2023-10-06T05:26:18.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:26:19.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.020+0000 7fc61f8cb700 1 Processor -- start 2023-10-06T05:26:19.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.020+0000 7fc61f8cb700 1 -- start start 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.021+0000 7fc61f8cb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc6180cc840 0x7fc6180c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.021+0000 7fc61f8cb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc6180c9d50 0x7fc6180affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.021+0000 7fc61f8cb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6180ccca0 con 0x7fc61813f900 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.021+0000 7fc61f8cb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180c9630 con 0x7fc6180cc840 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.021+0000 7fc61f8cb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180ca150 con 0x7fc6180c9d50 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.021+0000 7fc61ce66700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc6180cc840 0x7fc6180c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61ce66700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc6180cc840 0x7fc6180c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34646/0 (socket says 172.21.15.45:34646) 2023-10-06T05:26:19.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61ce66700 1 -- 172.21.15.45:0/261220835 learned_addr learned my addr 172.21.15.45:0/261220835 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61de68700 1 --2- 172.21.15.45:0/261220835 >> v2:172.21.15.144:3300/0 conn(0x7fc6180c9d50 0x7fc6180affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61ce66700 1 -- 172.21.15.45:0/261220835 >> v2:172.21.15.144:3300/0 conn(0x7fc6180c9d50 msgr2=0x7fc6180affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61ce66700 1 --2- 172.21.15.45:0/261220835 >> v2:172.21.15.144:3300/0 conn(0x7fc6180c9d50 0x7fc6180affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61ce66700 1 -- 172.21.15.45:0/261220835 >> v1:172.21.15.144:6789/0 conn(0x7fc61813f900 legacy=0x7fc6180c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61ce66700 1 -- 172.21.15.45:0/261220835 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180cb550 con 0x7fc6180cc840 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc60bfff700 1 -- 172.21.15.45:0/261220835 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151180441 0 0) 0x7fc614001110 con 0x7fc61813f900 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61de68700 1 --2- 172.21.15.45:0/261220835 >> v2:172.21.15.144:3300/0 conn(0x7fc6180c9d50 0x7fc6180affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc61ce66700 1 --2- 172.21.15.45:0/261220835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180cc840 0x7fc6180c9810 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc6180af960 tx=0x7fc60c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733877e4b3ae95a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc60bfff700 1 -- 172.21.15.45:0/261220835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c009ea0 con 0x7fc6180cc840 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc60bfff700 1 -- 172.21.15.45:0/261220835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60c00b820 con 0x7fc6180cc840 2023-10-06T05:26:19.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.022+0000 7fc60bfff700 1 -- 172.21.15.45:0/261220835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00bbc0 con 0x7fc6180cc840 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 -- 172.21.15.45:0/261220835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180cc840 msgr2=0x7fc6180c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/261220835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180cc840 0x7fc6180c9810 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc6180af960 tx=0x7fc60c009250 comp rx=0 tx=0).stop 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 -- 172.21.15.45:0/261220835 shutdown_connections 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/261220835 >> v2:172.21.15.144:3300/0 conn(0x7fc6180c9d50 0x7fc6180affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/261220835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180cc840 0x7fc6180c9810 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 -- 172.21.15.45:0/261220835 >> 172.21.15.45:0/261220835 conn(0x7fc6180691c0 msgr2=0x7fc6180cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 -- 172.21.15.45:0/261220835 shutdown_connections 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.023+0000 7fc61f8cb700 1 -- 172.21.15.45:0/261220835 wait complete. 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.024+0000 7fc61f8cb700 1 Processor -- start 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.024+0000 7fc61f8cb700 1 -- start start 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61f8cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180c9d50 0x7fc618155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61f8cb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6180cc840 0x7fc6181695a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61f8cb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc61813f900 0x7fc618158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61f8cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc618156740 con 0x7fc6180c9d50 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61f8cb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc618156a20 con 0x7fc6180cc840 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61f8cb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc618158e00 con 0x7fc61813f900 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61d667700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180c9d50 0x7fc618155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61d667700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180c9d50 0x7fc618155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34656/0 (socket says 172.21.15.45:34656) 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61d667700 1 -- 172.21.15.45:0/893791289 learned_addr learned my addr 172.21.15.45:0/893791289 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61de68700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc61813f900 0x7fc618158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.025+0000 7fc61ce66700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6180cc840 0x7fc6181695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61d667700 1 -- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc61813f900 msgr2=0x7fc618158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61d667700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc61813f900 0x7fc618158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61d667700 1 -- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6180cc840 msgr2=0x7fc6181695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61d667700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6180cc840 0x7fc6181695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61d667700 1 -- 172.21.15.45:0/893791289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610009a20 con 0x7fc6180c9d50 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61d667700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180c9d50 0x7fc618155b60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c560 tx=0x7fc61400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc6097fa700 1 -- 172.21.15.45:0/893791289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614011590 con 0x7fc6180c9d50 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc6097fa700 1 -- 172.21.15.45:0/893791289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614011710 con 0x7fc6180c9d50 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c0094f0 con 0x7fc6180c9d50 2023-10-06T05:26:19.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc6097fa700 1 -- 172.21.15.45:0/893791289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614011ad0 con 0x7fc6180c9d50 2023-10-06T05:26:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.026+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6181593c0 con 0x7fc6180c9d50 2023-10-06T05:26:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.027+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61806e2d0 con 0x7fc6180c9d50 2023-10-06T05:26:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.028+0000 7fc6097fa700 1 -- 172.21.15.45:0/893791289 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc61401b3b0 con 0x7fc6180c9d50 2023-10-06T05:26:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.028+0000 7fc6097fa700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc5fc066f30 0x7fc5fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.028+0000 7fc6097fa700 1 -- 172.21.15.45:0/893791289 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9388+0+0 (secure 0 0 0) 0x7fc6140847b0 con 0x7fc6180c9d50 2023-10-06T05:26:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.028+0000 7fc61ce66700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc5fc066f30 0x7fc5fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.029+0000 7fc61ce66700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc5fc066f30 0x7fc5fc0693e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc6180af960 tx=0x7fc60c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.033+0000 7fc6097fa700 1 -- 172.21.15.45:0/893791289 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc614050be0 con 0x7fc6180c9d50 2023-10-06T05:26:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.283+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fc6180ccca0 con 0x7fc6180c9d50 2023-10-06T05:26:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.527+0000 7fc6097fa700 1 -- 172.21.15.45:0/893791289 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v315) v1 ==== 162+0+0 (secure 0 0 0) 0x7fc6180ccca0 con 0x7fc6180c9d50 2023-10-06T05:26:19.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.530+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc5fc066f30 msgr2=0x7fc5fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.530+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc5fc066f30 0x7fc5fc0693e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc6180af960 tx=0x7fc60c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.530+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180c9d50 msgr2=0x7fc618155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.530+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180c9d50 0x7fc618155b60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c560 tx=0x7fc61400c870 comp rx=0 tx=0).stop 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 shutdown_connections 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc5fc066f30 0x7fc5fc0693e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc61813f900 0x7fc618158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6180c9d50 0x7fc618155b60 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 --2- 172.21.15.45:0/893791289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6180cc840 0x7fc6181695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 >> 172.21.15.45:0/893791289 conn(0x7fc6180691c0 msgr2=0x7fc618138830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 shutdown_connections 2023-10-06T05:26:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.531+0000 7fc61f8cb700 1 -- 172.21.15.45:0/893791289 wait complete. 2023-10-06T05:26:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:26:19.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:19.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.686+0000 7fcd739f3700 1 Processor -- start 2023-10-06T05:26:19.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.686+0000 7fcd739f3700 1 -- start start 2023-10-06T05:26:19.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.687+0000 7fcd739f3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.687+0000 7fcd739f3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd6c161c20 0x7fcd6c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.687+0000 7fcd739f3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6c0cd770 con 0x7fcd6c0cd300 2023-10-06T05:26:19.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.687+0000 7fcd739f3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c162080 con 0x7fcd6c161c20 2023-10-06T05:26:19.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.687+0000 7fcd739f3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c06b080 con 0x7fcd6c0c0170 2023-10-06T05:26:19.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.687+0000 7fcd7178f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.687+0000 7fcd71f90700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd6c161c20 0x7fcd6c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd7178f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c0ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:52696/0 (socket says 172.21.15.45:52696) 2023-10-06T05:26:19.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd71f90700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd6c161c20 0x7fcd6c0acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:34562/0 (socket says 172.21.15.45:34562) 2023-10-06T05:26:19.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd7178f700 1 -- 172.21.15.45:0/3446560018 learned_addr learned my addr 172.21.15.45:0/3446560018 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd70f8e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcd6c0cd300 0x7fcd6c161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:34652/0 (socket says 172.21.15.45:34652) 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd63fff700 1 -- 172.21.15.45:0/3446560018 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123224584 0 0) 0x7fcd6c0cd770 con 0x7fcd6c0cd300 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd63fff700 1 -- 172.21.15.45:0/3446560018 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd58003570 con 0x7fcd6c0cd300 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd71f90700 1 -- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 msgr2=0x7fcd6c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd71f90700 1 --2- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c0ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd71f90700 1 -- 172.21.15.45:0/3446560018 >> v1:172.21.15.144:6789/0 conn(0x7fcd6c0cd300 legacy=0x7fcd6c161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd71f90700 1 -- 172.21.15.45:0/3446560018 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c06b2d0 con 0x7fcd6c161c20 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd7178f700 1 --2- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd71f90700 1 --2- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c161c20 0x7fcd6c0acf00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009e60 tx=0x7fcd64009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a236e4aefae65b2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd63fff700 1 -- 172.21.15.45:0/3446560018 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400e070 con 0x7fcd6c161c20 2023-10-06T05:26:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd63fff700 1 -- 172.21.15.45:0/3446560018 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6400bab0 con 0x7fcd6c161c20 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.688+0000 7fcd63fff700 1 -- 172.21.15.45:0/3446560018 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400be70 con 0x7fcd6c161c20 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 -- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c161c20 msgr2=0x7fcd6c0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c161c20 0x7fcd6c0acf00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009e60 tx=0x7fcd64009780 comp rx=0 tx=0).stop 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 -- 172.21.15.45:0/3446560018 shutdown_connections 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c161c20 0x7fcd6c0acf00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3446560018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 -- 172.21.15.45:0/3446560018 >> 172.21.15.45:0/3446560018 conn(0x7fcd6c06f090 msgr2=0x7fcd6c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 -- 172.21.15.45:0/3446560018 shutdown_connections 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.689+0000 7fcd739f3700 1 -- 172.21.15.45:0/3446560018 wait complete. 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.690+0000 7fcd739f3700 1 Processor -- start 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.690+0000 7fcd739f3700 1 -- start start 2023-10-06T05:26:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.691+0000 7fcd739f3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.691+0000 7fcd739f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c0cd300 0x7fcd6c0c0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.691+0000 7fcd739f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd6c161c20 0x7fcd6c160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.691+0000 7fcd739f3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b1460 con 0x7fcd6c161c20 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.691+0000 7fcd739f3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b26c0 con 0x7fcd6c0cd300 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.691+0000 7fcd739f3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b29a0 con 0x7fcd6c0c0170 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd6c161c20 0x7fcd6c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd6c161c20 0x7fcd6c160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34666/0 (socket says 172.21.15.45:34666) 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 -- 172.21.15.45:0/3813339049 learned_addr learned my addr 172.21.15.45:0/3813339049 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd70f8e700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c0cd300 0x7fcd6c0c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd7178f700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 -- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 msgr2=0x7fcd6c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 -- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c0cd300 msgr2=0x7fcd6c0c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c0cd300 0x7fcd6c0c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 -- 172.21.15.45:0/3813339049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c0094f0 con 0x7fcd6c161c20 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd71f90700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd6c161c20 0x7fcd6c160750 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c06aed0 tx=0x7fcd64019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd617fa700 1 -- 172.21.15.45:0/3813339049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64012d80 con 0x7fcd6c161c20 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd617fa700 1 -- 172.21.15.45:0/3813339049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd64027470 con 0x7fcd6c161c20 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd64009a20 con 0x7fcd6c161c20 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd617fa700 1 -- 172.21.15.45:0/3813339049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64027830 con 0x7fcd6c161c20 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.692+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c15bc20 con 0x7fcd6c161c20 2023-10-06T05:26:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.693+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c06e2f0 con 0x7fcd6c161c20 2023-10-06T05:26:19.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.694+0000 7fcd617fa700 1 -- 172.21.15.45:0/3813339049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd64024070 con 0x7fcd6c161c20 2023-10-06T05:26:19.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.694+0000 7fcd617fa700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd50066f30 0x7fcd500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.695+0000 7fcd617fa700 1 -- 172.21.15.45:0/3813339049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9388+0+0 (secure 0 0 0) 0x7fcd640879e0 con 0x7fcd6c161c20 2023-10-06T05:26:19.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.695+0000 7fcd7178f700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd50066f30 0x7fcd500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.695+0000 7fcd7178f700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd50066f30 0x7fcd500693e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006d20 tx=0x7fcd5c009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.699+0000 7fcd617fa700 1 -- 172.21.15.45:0/3813339049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd64053d50 con 0x7fcd6c161c20 2023-10-06T05:26:19.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.945+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd6c068150 con 0x7fcd6c161c20 2023-10-06T05:26:19.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.945+0000 7fcd617fa700 1 -- 172.21.15.45:0/3813339049 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 316 v316) v1 ==== 95+0+4609 (secure 0 0 0) 0x7fcd6402c020 con 0x7fcd6c161c20 2023-10-06T05:26:19.947 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:19.947 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":316,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":316,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:19.528753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9820},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9820":{"gid":9820,"name":"b","rank":0,"incarnation":316,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/3451214191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3451214191},{"type":"v1","addr":"172.21.15.144:6835","nonce":3451214191}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:19.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd50066f30 msgr2=0x7fcd500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd50066f30 0x7fcd500693e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006d20 tx=0x7fcd5c009250 comp rx=0 tx=0).stop 2023-10-06T05:26:19.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd6c161c20 msgr2=0x7fcd6c160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd6c161c20 0x7fcd6c160750 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c06aed0 tx=0x7fcd64019000 comp rx=0 tx=0).stop 2023-10-06T05:26:19.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 shutdown_connections 2023-10-06T05:26:19.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd50066f30 0x7fcd500693e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd6c161c20 0x7fcd6c160750 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd6c0c0170 0x7fcd6c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 --2- 172.21.15.45:0/3813339049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd6c0cd300 0x7fcd6c0c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 >> 172.21.15.45:0/3813339049 conn(0x7fcd6c06f090 msgr2=0x7fcd6c0cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 shutdown_connections 2023-10-06T05:26:19.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:19.948+0000 7fcd739f3700 1 -- 172.21.15.45:0/3813339049 wait complete. 2023-10-06T05:26:19.951 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 316 2023-10-06T05:26:19.963 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:26:22.545 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:22.542+0000 7f322ec9b700 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:22.545 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:22.542+0000 7f322ec9b700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-10-06T05:26:22.548 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:22.547+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:22.548 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:22.547+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:26:22.556 INFO:tasks.ceph.mds.b.smithi144.stderr: -18> 2023-10-06T05:26:22.542+0000 7f322ec9b700 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:22.557 INFO:tasks.ceph.mds.b.smithi144.stderr: -15> 2023-10-06T05:26:22.542+0000 7f322ec9b700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2023-10-06T05:26:22.578 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:26:24.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:25.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.102+0000 7ff2e3e7e700 1 Processor -- start 2023-10-06T05:26:25.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.103+0000 7ff2e3e7e700 1 -- start start 2023-10-06T05:26:25.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e3e7e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff2dc0c93c0 0x7ff2dc156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e3e7e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc15b000 con 0x7ff2dc0c8ff0 2023-10-06T05:26:25.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e3e7e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc15b2e0 con 0x7ff2dc157030 2023-10-06T05:26:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e3e7e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc0c9830 con 0x7ff2dc0c93c0 2023-10-06T05:26:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e1c1a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff2dc0c8ff0 0x7ff2dc0caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38786/0 (socket says 172.21.15.45:38786) 2023-10-06T05:26:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/1008699762 learned_addr learned my addr 172.21.15.45:0/1008699762 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e1419700 1 --2- 172.21.15.45:0/1008699762 >> v2:172.21.15.144:3300/0 conn(0x7ff2dc0c93c0 0x7ff2dc156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356403924 0 0) 0x7ff2dc15b000 con 0x7ff2dc0c8ff0 2023-10-06T05:26:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8003570 con 0x7ff2dc0c8ff0 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698545415 0 0) 0x7ff2c8003570 con 0x7ff2dc0c8ff0 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 >> v2:172.21.15.144:3300/0 conn(0x7ff2dc0c93c0 msgr2=0x7ff2dc156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 --2- 172.21.15.45:0/1008699762 >> v2:172.21.15.144:3300/0 conn(0x7ff2dc0c93c0 0x7ff2dc156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 >> v1:172.21.15.144:6789/0 conn(0x7ff2dc157030 legacy=0x7ff2dc0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc15b5c0 con 0x7ff2dc0c8ff0 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7ff2d0001470 con 0x7ff2dc0c8ff0 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 >> v1:172.21.15.45:6789/0 conn(0x7ff2dc0c8ff0 legacy=0x7ff2dc0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c8003a60 0x7ff2c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.104+0000 7ff2e0c18700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c8004810 0x7ff2c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e0c18700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80051c0 0x7ff2c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0c9830 con 0x7ff2c80051c0 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c800c050 con 0x7ff2c8003a60 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c800c210 con 0x7ff2c8004810 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1c1a700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c8003a60 0x7ff2c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e241b700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80051c0 0x7ff2c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1419700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c8004810 0x7ff2c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c8004810 msgr2=0x7ff2c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1c1a700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c8004810 0x7ff2c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/1008699762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80051c0 msgr2=0x7ff2c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1c1a700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80051c0 0x7ff2c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/1008699762 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8003570 con 0x7ff2c8003a60 2023-10-06T05:26:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e241b700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80051c0 0x7ff2c800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e241b700 1 -- 172.21.15.45:0/1008699762 reap_dead start 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e1c1a700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c8003a60 0x7ff2c8003e30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c3f0 tx=0x7ff2d0001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d06c040187fa61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d000cc80 con 0x7ff2c8003a60 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d000ce00 con 0x7ff2c8003a60 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.105+0000 7ff2e0c18700 1 -- 172.21.15.45:0/1008699762 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0005b70 con 0x7ff2c8003a60 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.106+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c8003a60 msgr2=0x7ff2c8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.106+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c8003a60 0x7ff2c8003e30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c3f0 tx=0x7ff2d0001e70 comp rx=0 tx=0).stop 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.106+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/1008699762 shutdown_connections 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.106+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/1008699762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c8003a60 0x7ff2c8003e30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.106+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/1008699762 >> 172.21.15.45:0/1008699762 conn(0x7ff2dc0cdf50 msgr2=0x7ff2dc0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.106+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/1008699762 shutdown_connections 2023-10-06T05:26:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.106+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/1008699762 wait complete. 2023-10-06T05:26:25.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.107+0000 7ff2e3e7e700 1 Processor -- start 2023-10-06T05:26:25.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.107+0000 7ff2e3e7e700 1 -- start start 2023-10-06T05:26:25.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e3e7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c8003a60 0x7ff2dc1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e3e7e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2dc0b1f70 0x7ff2dc0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e3e7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2dc0b23e0 0x7ff2dc0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e3e7e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc159f20 con 0x7ff2c8003a60 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e3e7e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0c6970 con 0x7ff2dc0b23e0 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e3e7e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0c5130 con 0x7ff2dc0b1f70 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e1c1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c8003a60 0x7ff2dc1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e1c1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c8003a60 0x7ff2dc1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34676/0 (socket says 172.21.15.45:34676) 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/231949955 learned_addr learned my addr 172.21.15.45:0/231949955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.108+0000 7ff2e241b700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2dc0b23e0 0x7ff2dc0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1419700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2dc0b1f70 0x7ff2dc0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2dc0b1f70 msgr2=0x7ff2dc0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1c1a700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2dc0b1f70 0x7ff2dc0c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2dc0b23e0 msgr2=0x7ff2dc0c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1c1a700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2dc0b23e0 0x7ff2dc0c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1c1a700 1 -- 172.21.15.45:0/231949955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0c5410 con 0x7ff2c8003a60 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1419700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2dc0b1f70 0x7ff2dc0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e241b700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2dc0b23e0 0x7ff2dc0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e1c1a700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c8003a60 0x7ff2dc1595b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0011000 tx=0x7ff2d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2cdffb700 1 -- 172.21.15.45:0/231949955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0022070 con 0x7ff2c8003a60 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2cdffb700 1 -- 172.21.15.45:0/231949955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d000cc80 con 0x7ff2c8003a60 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc0c56f0 con 0x7ff2c8003a60 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2cdffb700 1 -- 172.21.15.45:0/231949955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0007510 con 0x7ff2c8003a60 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.109+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc140320 con 0x7ff2c8003a60 2023-10-06T05:26:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.110+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc0c5b90 con 0x7ff2c8003a60 2023-10-06T05:26:25.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.111+0000 7ff2cdffb700 1 -- 172.21.15.45:0/231949955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2d001e430 con 0x7ff2c8003a60 2023-10-06T05:26:25.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.111+0000 7ff2cdffb700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2c0066f30 0x7ff2c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.111+0000 7ff2cdffb700 1 -- 172.21.15.45:0/231949955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff2d0086590 con 0x7ff2c8003a60 2023-10-06T05:26:25.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.111+0000 7ff2e1419700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2c0066f30 0x7ff2c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.112+0000 7ff2e1419700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2c0066f30 0x7ff2c00693e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006dc0 tx=0x7ff2d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.116+0000 7ff2cdffb700 1 -- 172.21.15.45:0/231949955 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2d0052900 con 0x7ff2c8003a60 2023-10-06T05:26:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.361+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2dc0c5d40 con 0x7ff2c8003a60 2023-10-06T05:26:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.362+0000 7ff2cdffb700 1 -- 172.21.15.45:0/231949955 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 321 v321) v1 ==== 95+0+4597 (secure 0 0 0) 0x7ff2d0017070 con 0x7ff2c8003a60 2023-10-06T05:26:25.364 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:25.364 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":321,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":8447,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/603338967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":603338967},{"type":"v1","addr":"172.21.15.144:6835","nonce":603338967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":321}],"filesystems":[{"mdsmap":{"epoch":320,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:22.548021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2c0066f30 msgr2=0x7ff2c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2c0066f30 0x7ff2c00693e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006dc0 tx=0x7ff2d8008040 comp rx=0 tx=0).stop 2023-10-06T05:26:25.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c8003a60 msgr2=0x7ff2dc1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c8003a60 0x7ff2dc1595b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0011000 tx=0x7ff2d0015000 comp rx=0 tx=0).stop 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 shutdown_connections 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff2c0066f30 0x7ff2c00693e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2dc0b23e0 0x7ff2dc0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c8003a60 0x7ff2dc1595b0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 --2- 172.21.15.45:0/231949955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2dc0b1f70 0x7ff2dc0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 >> 172.21.15.45:0/231949955 conn(0x7ff2dc0cdf50 msgr2=0x7ff2dc0c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 shutdown_connections 2023-10-06T05:26:25.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.364+0000 7ff2e3e7e700 1 -- 172.21.15.45:0/231949955 wait complete. 2023-10-06T05:26:25.370 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 321 2023-10-06T05:26:25.379 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:26:25.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:25.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.510+0000 7f6772a88700 1 Processor -- start 2023-10-06T05:26:25.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.510+0000 7f6772a88700 1 -- start start 2023-10-06T05:26:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.510+0000 7f6772a88700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f676c158f60 0x7f676c159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6772a88700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0cb530 con 0x7f676c0cc390 2023-10-06T05:26:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6772a88700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c069bd0 con 0x7f676c06a060 2023-10-06T05:26:25.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6772a88700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676c06a500 con 0x7f676c158f60 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6771025700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f676c0cc390 0x7f676c0c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38798/0 (socket says 172.21.15.45:38798) 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6771025700 1 -- 172.21.15.45:0/1181721353 learned_addr learned my addr 172.21.15.45:0/1181721353 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6770824700 1 --2- 172.21.15.45:0/1181721353 >> v2:172.21.15.144:3300/0 conn(0x7f676c158f60 0x7f676c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f676b7fe700 1 -- 172.21.15.45:0/1181721353 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161954479 0 0) 0x7f6764001490 con 0x7f676c0cc390 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f676b7fe700 1 -- 172.21.15.45:0/1181721353 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f675c003570 con 0x7f676c0cc390 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f676b7fe700 1 -- 172.21.15.45:0/1181721353 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074459322 0 0) 0x7f676c069bd0 con 0x7f676c06a060 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f676b7fe700 1 -- 172.21.15.45:0/1181721353 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6764001490 con 0x7f676c06a060 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6770824700 1 -- 172.21.15.45:0/1181721353 >> v1:172.21.15.144:6789/0 conn(0x7f676c06a060 legacy=0x7f676c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6770824700 1 -- 172.21.15.45:0/1181721353 >> v1:172.21.15.45:6789/0 conn(0x7f676c0cc390 legacy=0x7f676c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6770824700 1 -- 172.21.15.45:0/1181721353 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c068fc0 con 0x7f676c158f60 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.511+0000 7f6770824700 1 --2- 172.21.15.45:0/1181721353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c158f60 0x7f676c159330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f67600098e0 tx=0x7f67600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cab269dde3e6feb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f676b7fe700 1 -- 172.21.15.45:0/1181721353 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760011630 con 0x7f676c158f60 2023-10-06T05:26:25.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f676b7fe700 1 -- 172.21.15.45:0/1181721353 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760011790 con 0x7f676c158f60 2023-10-06T05:26:25.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f676b7fe700 1 -- 172.21.15.45:0/1181721353 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760011b30 con 0x7f676c158f60 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f6772a88700 1 -- 172.21.15.45:0/1181721353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c158f60 msgr2=0x7f676c159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f6772a88700 1 --2- 172.21.15.45:0/1181721353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c158f60 0x7f676c159330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f67600098e0 tx=0x7f67600091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f6772a88700 1 -- 172.21.15.45:0/1181721353 shutdown_connections 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f6772a88700 1 --2- 172.21.15.45:0/1181721353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c158f60 0x7f676c159330 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f6772a88700 1 -- 172.21.15.45:0/1181721353 >> 172.21.15.45:0/1181721353 conn(0x7f676c0cdf50 msgr2=0x7f676c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f6772a88700 1 -- 172.21.15.45:0/1181721353 shutdown_connections 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.512+0000 7f6772a88700 1 -- 172.21.15.45:0/1181721353 wait complete. 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.513+0000 7f6772a88700 1 Processor -- start 2023-10-06T05:26:25.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.514+0000 7f6772a88700 1 -- start start 2023-10-06T05:26:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.514+0000 7f6772a88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c06a060 0x7f676c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.514+0000 7f6772a88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f676c0cc390 0x7f676c1485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.514+0000 7f6772a88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c158f60 0x7f676c148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.514+0000 7f6772a88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0c79e0 con 0x7f676c158f60 2023-10-06T05:26:25.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.514+0000 7f6772a88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c159cd0 con 0x7f676c06a060 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.514+0000 7f6772a88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c159fb0 con 0x7f676c0cc390 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c158f60 0x7f676c148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c158f60 0x7f676c148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34684/0 (socket says 172.21.15.45:34684) 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 -- 172.21.15.45:0/3071457928 learned_addr learned my addr 172.21.15.45:0/3071457928 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6770824700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c06a060 0x7f676c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f676bfff700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f676c0cc390 0x7f676c1485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 -- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f676c0cc390 msgr2=0x7f676c1485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f676c0cc390 0x7f676c1485a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 -- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c06a060 msgr2=0x7f676c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c06a060 0x7f676c15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 -- 172.21.15.45:0/3071457928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760009420 con 0x7f676c158f60 2023-10-06T05:26:25.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6771025700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c158f60 0x7f676c148ae0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f6764002070 tx=0x7f6764001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6768ff9700 1 -- 172.21.15.45:0/3071457928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400aac0 con 0x7f676c158f60 2023-10-06T05:26:25.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6768ff9700 1 -- 172.21.15.45:0/3071457928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676400ac20 con 0x7f676c158f60 2023-10-06T05:26:25.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c15a290 con 0x7f676c158f60 2023-10-06T05:26:25.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6768ff9700 1 -- 172.21.15.45:0/3071457928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640070f0 con 0x7f676c158f60 2023-10-06T05:26:25.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.515+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c15cd60 con 0x7f676c158f60 2023-10-06T05:26:25.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.516+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c06f410 con 0x7f676c158f60 2023-10-06T05:26:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.517+0000 7f6768ff9700 1 -- 172.21.15.45:0/3071457928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67640074e0 con 0x7f676c158f60 2023-10-06T05:26:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.517+0000 7f6768ff9700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6750066f30 0x7f67500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.518+0000 7f6768ff9700 1 -- 172.21.15.45:0/3071457928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f67640858a0 con 0x7f676c158f60 2023-10-06T05:26:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.518+0000 7f6770824700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6750066f30 0x7f67500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.518+0000 7f6770824700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6750066f30 0x7f67500693e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f67600093f0 tx=0x7f6760017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.522+0000 7f6768ff9700 1 -- 172.21.15.45:0/3071457928 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6764051c10 con 0x7f676c158f60 2023-10-06T05:26:25.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.768+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f676c0cb530 con 0x7f676c158f60 2023-10-06T05:26:25.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.768+0000 7f6768ff9700 1 -- 172.21.15.45:0/3071457928 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 321 v321) v1 ==== 95+0+4597 (secure 0 0 0) 0x7f676c0cb530 con 0x7f676c158f60 2023-10-06T05:26:25.770 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:25.770 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":321,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":8447,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/603338967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":603338967},{"type":"v1","addr":"172.21.15.144:6835","nonce":603338967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":321}],"filesystems":[{"mdsmap":{"epoch":320,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:22.548021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6750066f30 msgr2=0x7f67500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6750066f30 0x7f67500693e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f67600093f0 tx=0x7f6760017000 comp rx=0 tx=0).stop 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c158f60 msgr2=0x7f676c148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c158f60 0x7f676c148ae0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f6764002070 tx=0x7f6764001c70 comp rx=0 tx=0).stop 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 shutdown_connections 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6750066f30 0x7f67500693e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c158f60 0x7f676c148ae0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f676c06a060 0x7f676c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 --2- 172.21.15.45:0/3071457928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f676c0cc390 0x7f676c1485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 >> 172.21.15.45:0/3071457928 conn(0x7f676c0cdf50 msgr2=0x7f676c157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 shutdown_connections 2023-10-06T05:26:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.771+0000 7f6772a88700 1 -- 172.21.15.45:0/3071457928 wait complete. 2023-10-06T05:26:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 321 2023-10-06T05:26:25.786 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:26:25.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.916+0000 7fa68ea2e700 1 Processor -- start 2023-10-06T05:26:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.916+0000 7fa68ea2e700 1 -- start start 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa68ea2e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6880c93c0 0x7fa688156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa68ea2e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa688157030 0x7fa6880cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa68ea2e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68815b000 con 0x7fa6880c8ff0 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa68ea2e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6880c9830 con 0x7fa6880c93c0 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa68ea2e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68815b2e0 con 0x7fa688157030 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa6877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6880c93c0 0x7fa688156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa6877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6880c93c0 0x7fa688156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34690/0 (socket says 172.21.15.45:34690) 2023-10-06T05:26:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.917+0000 7fa6877fe700 1 -- 172.21.15.45:0/1524585225 learned_addr learned my addr 172.21.15.45:0/1524585225 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa6877fe700 1 -- 172.21.15.45:0/1524585225 >> v2:172.21.15.144:3300/0 conn(0x7fa688157030 msgr2=0x7fa6880cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa68cfcb700 1 --2- 172.21.15.45:0/1524585225 >> v2:172.21.15.144:3300/0 conn(0x7fa688157030 0x7fa6880cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa6877fe700 1 --2- 172.21.15.45:0/1524585225 >> v2:172.21.15.144:3300/0 conn(0x7fa688157030 0x7fa6880cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa6877fe700 1 -- 172.21.15.45:0/1524585225 >> v1:172.21.15.144:6789/0 conn(0x7fa6880c8ff0 legacy=0x7fa6880caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa6877fe700 1 -- 172.21.15.45:0/1524585225 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68815b5c0 con 0x7fa6880c93c0 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa68cfcb700 1 --2- 172.21.15.45:0/1524585225 >> v2:172.21.15.144:3300/0 conn(0x7fa688157030 0x7fa6880cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa6877fe700 1 --2- 172.21.15.45:0/1524585225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa688156af0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fa68815abf0 tx=0x7fa670009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dca14ece521b07e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa686ffd700 1 -- 172.21.15.45:0/1524585225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67000b820 con 0x7fa6880c93c0 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa686ffd700 1 -- 172.21.15.45:0/1524585225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67000b980 con 0x7fa6880c93c0 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.918+0000 7fa686ffd700 1 -- 172.21.15.45:0/1524585225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67000bd20 con 0x7fa6880c93c0 2023-10-06T05:26:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1524585225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 msgr2=0x7fa688156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1524585225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa688156af0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fa68815abf0 tx=0x7fa670009250 comp rx=0 tx=0).stop 2023-10-06T05:26:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1524585225 shutdown_connections 2023-10-06T05:26:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1524585225 >> v2:172.21.15.144:3300/0 conn(0x7fa688157030 0x7fa6880cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1524585225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa688156af0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1524585225 >> 172.21.15.45:0/1524585225 conn(0x7fa6880cdf50 msgr2=0x7fa6880ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1524585225 shutdown_connections 2023-10-06T05:26:25.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.919+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1524585225 wait complete. 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.920+0000 7fa68ea2e700 1 Processor -- start 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.920+0000 7fa68ea2e700 1 -- start start 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa68ea2e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880c8ff0 0x7fa68814d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa68ea2e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa6880c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa68ea2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688157030 0x7fa6880c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa68ea2e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881588d0 con 0x7fa6880c93c0 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa68ea2e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880c6940 con 0x7fa688157030 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa68ea2e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880c6c20 con 0x7fa6880c8ff0 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa6880c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa6880c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34700/0 (socket says 172.21.15.45:34700) 2023-10-06T05:26:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 -- 172.21.15.45:0/1751986726 learned_addr learned my addr 172.21.15.45:0/1751986726 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa68cfcb700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688157030 0x7fa6880c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa687fff700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880c8ff0 0x7fa68814d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 -- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880c8ff0 msgr2=0x7fa68814d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880c8ff0 0x7fa68814d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 -- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688157030 msgr2=0x7fa6880c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688157030 0x7fa6880c40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.921+0000 7fa6877fe700 1 -- 172.21.15.45:0/1751986726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700094f0 con 0x7fa6880c93c0 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.922+0000 7fa6877fe700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa6880c5fd0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fa6700030c0 tx=0x7fa670016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.922+0000 7fa67ffff700 1 -- 172.21.15.45:0/1751986726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011e60 con 0x7fa6880c93c0 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.922+0000 7fa67ffff700 1 -- 172.21.15.45:0/1751986726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670020070 con 0x7fa6880c93c0 2023-10-06T05:26:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.922+0000 7fa67ffff700 1 -- 172.21.15.45:0/1751986726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67001b6a0 con 0x7fa6880c93c0 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.922+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6880c6f00 con 0x7fa6880c93c0 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.922+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6880c6510 con 0x7fa6880c93c0 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.923+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6880c6700 con 0x7fa6880c93c0 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.923+0000 7fa67ffff700 1 -- 172.21.15.45:0/1751986726 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa670022030 con 0x7fa6880c93c0 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.924+0000 7fa67ffff700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa668066f30 0x7fa6680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.924+0000 7fa67ffff700 1 -- 172.21.15.45:0/1751986726 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa670087450 con 0x7fa6880c93c0 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.924+0000 7fa687fff700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa668066f30 0x7fa6680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.924+0000 7fa687fff700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa668066f30 0x7fa6680693e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa678006fd0 tx=0x7fa678008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:25.928+0000 7fa67ffff700 1 -- 172.21.15.45:0/1751986726 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6880c6700 con 0x7fa6880c93c0 2023-10-06T05:26:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.180+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6880c6700 con 0x7fa6880c93c0 2023-10-06T05:26:26.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.180+0000 7fa67ffff700 1 -- 172.21.15.45:0/1751986726 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 321 v321) v1 ==== 95+0+4597 (secure 0 0 0) 0x7fa6880c6700 con 0x7fa6880c93c0 2023-10-06T05:26:26.182 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:26.182 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":321,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":8447,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/603338967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":603338967},{"type":"v1","addr":"172.21.15.144:6835","nonce":603338967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":321}],"filesystems":[{"mdsmap":{"epoch":320,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:22.548021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa668066f30 msgr2=0x7fa6680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa668066f30 0x7fa6680693e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa678006fd0 tx=0x7fa678008040 comp rx=0 tx=0).stop 2023-10-06T05:26:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 msgr2=0x7fa6880c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa6880c5fd0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fa6700030c0 tx=0x7fa670016000 comp rx=0 tx=0).stop 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 shutdown_connections 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa668066f30 0x7fa6680693e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa688157030 0x7fa6880c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880c8ff0 0x7fa68814d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 --2- 172.21.15.45:0/1751986726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6880c93c0 0x7fa6880c5fd0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 >> 172.21.15.45:0/1751986726 conn(0x7fa6880cdf50 msgr2=0x7fa6880c2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 shutdown_connections 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.182+0000 7fa68ea2e700 1 -- 172.21.15.45:0/1751986726 wait complete. 2023-10-06T05:26:26.185 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 321 2023-10-06T05:26:26.198 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2023-10-06T05:26:26.199 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2023-10-06T05:26:26.199 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:26.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:26:26.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:26.199 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:26.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:26:26.219 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:26:26.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.394+0000 7f2bebdc2700 1 Processor -- start 2023-10-06T05:26:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.395+0000 7f2bebdc2700 1 -- start start 2023-10-06T05:26:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.395+0000 7f2bebdc2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2be40c93c0 0x7f2be4156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bebdc2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2be4157030 0x7f2be40cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bebdc2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be415b000 con 0x7f2be40c8ff0 2023-10-06T05:26:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bebdc2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40c9830 con 0x7f2be4157030 2023-10-06T05:26:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bebdc2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be415b2e0 con 0x7f2be40c93c0 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2be9b5e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2be40c8ff0 0x7f2be40caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44988/0 (socket says 172.21.15.45:44988) 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2be9b5e700 1 -- 172.21.15.45:0/969656968 learned_addr learned my addr 172.21.15.45:0/969656968 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bea35f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2be4157030 0x7f2be40cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2be935d700 1 --2- 172.21.15.45:0/969656968 >> v2:172.21.15.144:3300/0 conn(0x7f2be40c93c0 0x7f2be4156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2be8b5c700 1 -- 172.21.15.45:0/969656968 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754172993 0 0) 0x7f2be415b000 con 0x7f2be40c8ff0 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2be8b5c700 1 -- 172.21.15.45:0/969656968 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd0003570 con 0x7f2be40c8ff0 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bea35f700 1 -- 172.21.15.45:0/969656968 >> v2:172.21.15.144:3300/0 conn(0x7f2be40c93c0 msgr2=0x7f2be4156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bea35f700 1 --2- 172.21.15.45:0/969656968 >> v2:172.21.15.144:3300/0 conn(0x7f2be40c93c0 0x7f2be4156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bea35f700 1 -- 172.21.15.45:0/969656968 >> v1:172.21.15.45:6789/0 conn(0x7f2be40c8ff0 legacy=0x7f2be40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bea35f700 1 -- 172.21.15.45:0/969656968 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be415b5c0 con 0x7f2be4157030 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2bea35f700 1 --2- 172.21.15.45:0/969656968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be4157030 0x7f2be40cc330 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f2be406ed80 tx=0x7f2bdc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9725840e636dfe46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.396+0000 7f2be8b5c700 1 -- 172.21.15.45:0/969656968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00e070 con 0x7f2be4157030 2023-10-06T05:26:26.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2be8b5c700 1 -- 172.21.15.45:0/969656968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc00bab0 con 0x7f2be4157030 2023-10-06T05:26:26.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2be8b5c700 1 -- 172.21.15.45:0/969656968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00be70 con 0x7f2be4157030 2023-10-06T05:26:26.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 -- 172.21.15.45:0/969656968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be4157030 msgr2=0x7f2be40cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/969656968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be4157030 0x7f2be40cc330 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f2be406ed80 tx=0x7f2bdc009780 comp rx=0 tx=0).stop 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 -- 172.21.15.45:0/969656968 shutdown_connections 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/969656968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be4157030 0x7f2be40cc330 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/969656968 >> v2:172.21.15.144:3300/0 conn(0x7f2be40c93c0 0x7f2be4156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 -- 172.21.15.45:0/969656968 >> 172.21.15.45:0/969656968 conn(0x7f2be40cdf50 msgr2=0x7f2be40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 -- 172.21.15.45:0/969656968 shutdown_connections 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.397+0000 7f2bebdc2700 1 -- 172.21.15.45:0/969656968 wait complete. 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.398+0000 7f2bebdc2700 1 Processor -- start 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.398+0000 7f2bebdc2700 1 -- start start 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2bebdc2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be40c8ff0 0x7f2be40c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2bebdc2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be40c93c0 0x7f2be41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2bebdc2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be4157030 0x7f2be40af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2bebdc2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b1bf0 con 0x7f2be40c8ff0 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2bebdc2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40c5f50 con 0x7f2be4157030 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2bebdc2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4159f10 con 0x7f2be40c93c0 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2be9b5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be40c8ff0 0x7f2be40c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2be9b5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be40c8ff0 0x7f2be40c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35100/0 (socket says 172.21.15.45:35100) 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.399+0000 7f2be9b5e700 1 -- 172.21.15.45:0/2460639411 learned_addr learned my addr 172.21.15.45:0/2460639411 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2bea35f700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be4157030 0x7f2be40af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be935d700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be40c93c0 0x7f2be41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be9b5e700 1 -- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be40c93c0 msgr2=0x7f2be41599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be9b5e700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be40c93c0 0x7f2be41599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be9b5e700 1 -- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be4157030 msgr2=0x7f2be40af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be9b5e700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be4157030 0x7f2be40af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be9b5e700 1 -- 172.21.15.45:0/2460639411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc009a20 con 0x7f2be40c8ff0 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be935d700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be40c93c0 0x7f2be41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2be9b5e700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be40c8ff0 0x7f2be40c5a10 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c590 tx=0x7f2bd8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/2460639411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8005b30 con 0x7f2be40c8ff0 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/2460639411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8005cb0 con 0x7f2be40c8ff0 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be40b36c0 con 0x7f2be40c8ff0 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/2460639411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8007320 con 0x7f2be40c8ff0 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.400+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be40b1880 con 0x7f2be40c8ff0 2023-10-06T05:26:26.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.401+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be40b3be0 con 0x7f2be40c8ff0 2023-10-06T05:26:26.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.402+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/2460639411 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bd801b3b0 con 0x7f2be40c8ff0 2023-10-06T05:26:26.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.403+0000 7f2bd5ffb700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2bc8066f30 0x7f2bc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.403+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/2460639411 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2bd8084e60 con 0x7f2be40c8ff0 2023-10-06T05:26:26.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.403+0000 7f2be935d700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2bc8066f30 0x7f2bc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.404+0000 7f2be935d700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2bc8066f30 0x7f2bc80693e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006f00 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:26.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.407+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/2460639411 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bd80511f0 con 0x7f2be40c8ff0 2023-10-06T05:26:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:26.657+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2be40c87a0 con 0x7f2be40c8ff0 2023-10-06T05:26:27.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.571+0000 7f2bd5ffb700 1 -- 172.21.15.45:0/2460639411 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v322) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2bd8017020 con 0x7f2be40c8ff0 2023-10-06T05:26:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2bc8066f30 msgr2=0x7f2bc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2bc8066f30 0x7f2bc80693e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006f00 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2023-10-06T05:26:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be40c8ff0 msgr2=0x7f2be40c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be40c8ff0 0x7f2be40c5a10 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c590 tx=0x7f2bd8007e00 comp rx=0 tx=0).stop 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 shutdown_connections 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2bc8066f30 0x7f2bc80693e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be4157030 0x7f2be40af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2be40c8ff0 0x7f2be40c5a10 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 --2- 172.21.15.45:0/2460639411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be40c93c0 0x7f2be41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 >> 172.21.15.45:0/2460639411 conn(0x7f2be40cdf50 msgr2=0x7f2be40ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 shutdown_connections 2023-10-06T05:26:27.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.574+0000 7f2bebdc2700 1 -- 172.21.15.45:0/2460639411 wait complete. 2023-10-06T05:26:27.578 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:27.591 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:26:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.727+0000 7f786a42b700 1 Processor -- start 2023-10-06T05:26:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.727+0000 7f786a42b700 1 -- start start 2023-10-06T05:26:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f786a42b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78640c93c0 0x7f7864156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f786a42b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f78640cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f786a42b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786415b000 con 0x7f78640c8ff0 2023-10-06T05:26:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f786a42b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640c9830 con 0x7f78640c93c0 2023-10-06T05:26:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f786a42b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786415b2e0 con 0x7f7864157030 2023-10-06T05:26:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f78637fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78640c93c0 0x7f7864156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f7863fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f78640c8ff0 0x7f78640caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45000/0 (socket says 172.21.15.45:45000) 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f7863fff700 1 -- 172.21.15.45:0/1386603489 learned_addr learned my addr 172.21.15.45:0/1386603489 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f78689c8700 1 --2- 172.21.15.45:0/1386603489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f78640cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f78637fe700 1 -- 172.21.15.45:0/1386603489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 msgr2=0x7f78640cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f78637fe700 1 --2- 172.21.15.45:0/1386603489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f78640cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f78637fe700 1 -- 172.21.15.45:0/1386603489 >> v1:172.21.15.45:6789/0 conn(0x7f78640c8ff0 legacy=0x7f78640caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f78637fe700 1 -- 172.21.15.45:0/1386603489 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786415b5c0 con 0x7f78640c93c0 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f7862ffd700 1 -- 172.21.15.45:0/1386603489 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226964332 0 0) 0x7f786415b000 con 0x7f78640c8ff0 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f78637fe700 1 --2- 172.21.15.45:0/1386603489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f7864156af0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f78580098e0 tx=0x7f78580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d15d732f5726f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.728+0000 7f7862ffd700 1 -- 172.21.15.45:0/1386603489 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858009e80 con 0x7f78640c93c0 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f7862ffd700 1 -- 172.21.15.45:0/1386603489 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785800b860 con 0x7f78640c93c0 2023-10-06T05:26:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f7862ffd700 1 -- 172.21.15.45:0/1386603489 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800bbe0 con 0x7f78640c93c0 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 -- 172.21.15.45:0/1386603489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 msgr2=0x7f7864156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 --2- 172.21.15.45:0/1386603489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f7864156af0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f78580098e0 tx=0x7f78580091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 -- 172.21.15.45:0/1386603489 shutdown_connections 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 --2- 172.21.15.45:0/1386603489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f78640cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 --2- 172.21.15.45:0/1386603489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f7864156af0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 -- 172.21.15.45:0/1386603489 >> 172.21.15.45:0/1386603489 conn(0x7f78640cdf50 msgr2=0x7f78640ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 -- 172.21.15.45:0/1386603489 shutdown_connections 2023-10-06T05:26:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.729+0000 7f786a42b700 1 -- 172.21.15.45:0/1386603489 wait complete. 2023-10-06T05:26:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.730+0000 7f786a42b700 1 Processor -- start 2023-10-06T05:26:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.731+0000 7f786a42b700 1 -- start start 2023-10-06T05:26:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.731+0000 7f786a42b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78640c8ff0 0x7f7864069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.731+0000 7f786a42b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f78640af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f786a42b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f7864068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f786a42b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786406a760 con 0x7f78640c93c0 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f786a42b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78640c7190 con 0x7f78640c8ff0 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f786a42b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786406a480 con 0x7f7864157030 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f78640af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f78640af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35126/0 (socket says 172.21.15.45:35126) 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 -- 172.21.15.45:0/1087805684 learned_addr learned my addr 172.21.15.45:0/1087805684 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f7863fff700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78640c8ff0 0x7f7864069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78689c8700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f7864068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 -- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 msgr2=0x7f7864068730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f7864068730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 -- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78640c8ff0 msgr2=0x7f7864069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78640c8ff0 0x7f7864069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 -- 172.21.15.45:0/1087805684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858009420 con 0x7f78640c93c0 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f78637fe700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f78640af780 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f785800b3c0 tx=0x7f7858017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f784bfff700 1 -- 172.21.15.45:0/1087805684 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800bbe0 con 0x7f78640c93c0 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f784bfff700 1 -- 172.21.15.45:0/1087805684 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858000c90 con 0x7f78640c93c0 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78640c6b40 con 0x7f78640c93c0 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.732+0000 7f784bfff700 1 -- 172.21.15.45:0/1087805684 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858011690 con 0x7f78640c93c0 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.733+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786406a190 con 0x7f78640c93c0 2023-10-06T05:26:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.734+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78640c7320 con 0x7f78640c93c0 2023-10-06T05:26:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.734+0000 7f784bfff700 1 -- 172.21.15.45:0/1087805684 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7858011a80 con 0x7f78640c93c0 2023-10-06T05:26:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.734+0000 7f784bfff700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7850066f30 0x7f78500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.734+0000 7f784bfff700 1 -- 172.21.15.45:0/1087805684 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7858086d20 con 0x7f78640c93c0 2023-10-06T05:26:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.734+0000 7f7863fff700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7850066f30 0x7f78500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.735+0000 7f7863fff700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7850066f30 0x7f78500693e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f78640cb250 tx=0x7f7854007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:27.739+0000 7f784bfff700 1 -- 172.21.15.45:0/1087805684 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7858053090 con 0x7f78640c93c0 2023-10-06T05:26:28.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.025+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f786404bb40 con 0x7f78640c93c0 2023-10-06T05:26:28.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.574+0000 7f784bfff700 1 -- 172.21.15.45:0/1087805684 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v323) v1 ==== 101+0+0 (secure 0 0 0) 0x7f7858015090 con 0x7f78640c93c0 2023-10-06T05:26:28.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7850066f30 msgr2=0x7f78500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7850066f30 0x7f78500693e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f78640cb250 tx=0x7f7854007d30 comp rx=0 tx=0).stop 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 msgr2=0x7f78640af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f78640af780 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f785800b3c0 tx=0x7f7858017000 comp rx=0 tx=0).stop 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 shutdown_connections 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7850066f30 0x7f78500693e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7864157030 0x7f7864068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78640c8ff0 0x7f7864069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 --2- 172.21.15.45:0/1087805684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640c93c0 0x7f78640af780 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 >> 172.21.15.45:0/1087805684 conn(0x7f78640cdf50 msgr2=0x7f78640c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 shutdown_connections 2023-10-06T05:26:28.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.577+0000 7f786a42b700 1 -- 172.21.15.45:0/1087805684 wait complete. 2023-10-06T05:26:28.580 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:26:28.594 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:26:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.639+0000 7ff97ae35040 1 Processor -- start 2023-10-06T05:26:28.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.639+0000 7ff97ae35040 1 -- start start 2023-10-06T05:26:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff97ae35040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b38041e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff97ae35040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b1b70 0x55b3804b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff97ae35040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b6140 0x55b3804aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff97ae35040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b380328980 con 0x55b3804b1b70 2023-10-06T05:26:28.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff97ae35040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b380328620 con 0x55b3804289b0 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff97ae35040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b38031f400 con 0x55b3804b6140 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b1b70 0x55b3804b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b1b70 0x55b3804b67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35142/0 (socket says 172.21.15.45:35142) 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 -- 172.21.15.45:0/3658039760 learned_addr learned my addr 172.21.15.45:0/3658039760 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff974854700 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b6140 0x55b3804aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff974053700 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b38041e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 -- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b6140 msgr2=0x55b3804aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b6140 0x55b3804aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 -- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 msgr2=0x55b38041e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b38041e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff973852700 1 -- 172.21.15.45:0/3658039760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964005cf0 con 0x55b3804b1b70 2023-10-06T05:26:28.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.640+0000 7ff974053700 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b38041e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff973852700 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b1b70 0x55b3804b67e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ff968001c90 tx=0x7ff968001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915d3eeabc28d195 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff972850700 1 -- 172.21.15.45:0/3658039760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800bc80 con 0x55b3804b1b70 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff972850700 1 -- 172.21.15.45:0/3658039760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96800be00 con 0x55b3804b1b70 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff972850700 1 -- 172.21.15.45:0/3658039760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800e6c0 con 0x55b3804b1b70 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 -- 172.21.15.45:0/3658039760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b1b70 msgr2=0x55b3804b67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b1b70 0x55b3804b67e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ff968001c90 tx=0x7ff968001770 comp rx=0 tx=0).stop 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 -- 172.21.15.45:0/3658039760 shutdown_connections 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b6140 0x55b3804aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b38041e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 --2- 172.21.15.45:0/3658039760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b1b70 0x55b3804b67e0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 -- 172.21.15.45:0/3658039760 >> 172.21.15.45:0/3658039760 conn(0x55b3803df5f0 msgr2=0x55b3804b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 -- 172.21.15.45:0/3658039760 shutdown_connections 2023-10-06T05:26:28.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.641+0000 7ff97ae35040 1 -- 172.21.15.45:0/3658039760 wait complete. 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.642+0000 7ff97ae35040 1 Processor -- start 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.642+0000 7ff97ae35040 1 -- start start 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff97ae35040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b380424e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff97ae35040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b1b70 0x55b3803dd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff97ae35040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b6140 0x55b3803dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff97ae35040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3804aed90 con 0x55b3804b6140 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff97ae35040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b380425380 con 0x55b3804289b0 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff97ae35040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b38041fd10 con 0x55b3804b1b70 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b6140 0x55b3803dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b6140 0x55b3803dd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35144/0 (socket says 172.21.15.45:35144) 2023-10-06T05:26:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 learned_addr learned my addr 172.21.15.45:0/2411542938 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974053700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b380424e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b1b70 0x55b3803dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b1b70 msgr2=0x55b3803dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b1b70 0x55b3803dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 msgr2=0x55b380424e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3804289b0 0x55b380424e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c001e10 con 0x55b3804b6140 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.643+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3804b1b70 0x55b3803dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.644+0000 7ff974854700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b6140 0x55b3803dd6b0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x55b3803e2ec0 tx=0x7ff96c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.644+0000 7ff97104d700 1 -- 172.21.15.45:0/2411542938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0032f0 con 0x55b3804b6140 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.644+0000 7ff97104d700 1 -- 172.21.15.45:0/2411542938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96c00fa70 con 0x55b3804b6140 2023-10-06T05:26:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.644+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c00a0e0 con 0x55b3804b6140 2023-10-06T05:26:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.644+0000 7ff97104d700 1 -- 172.21.15.45:0/2411542938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c025680 con 0x55b3804b6140 2023-10-06T05:26:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.644+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3803373d0 con 0x55b3804b6140 2023-10-06T05:26:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.645+0000 7ff97104d700 1 -- 172.21.15.45:0/2411542938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff96c022070 con 0x55b3804b6140 2023-10-06T05:26:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.646+0000 7ff97104d700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff960065890 0x7ff960067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.646+0000 7ff97104d700 1 -- 172.21.15.45:0/2411542938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff96c005e40 con 0x55b3804b6140 2023-10-06T05:26:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.646+0000 7ff974053700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff960065890 0x7ff960067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.650 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:26:28.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.646+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b3804cdce0 0x55b3804d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.646+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d0600 con 0x55b3804cdce0 2023-10-06T05:26:28.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.646+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b3804cdce0 0x55b3804d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.646+0000 7ff974053700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff960065890 0x7ff960067d40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff964009d60 tx=0x7ff964005a50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.647+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b3804cdce0 0x55b3804d00c0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.649+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v128'1097 uv1070 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968014070 con 0x55b3804cdce0 2023-10-06T05:26:28.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.649+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d09b0 con 0x55b3804cdce0 2023-10-06T05:26:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.650+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v128'1098 uv1098 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968014070 con 0x55b3804cdce0 2023-10-06T05:26:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.650+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d0d60 con 0x55b3804cdce0 2023-10-06T05:26:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.652+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v128'1099 uv1099 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.655 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:26:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.652+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d2120 con 0x55b3804cdce0 2023-10-06T05:26:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.653+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'1100 uv1100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.653+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d24d0 con 0x55b3804cdce0 2023-10-06T05:26:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.655+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'1101 uv1101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.655+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d1110 con 0x55b3804cdce0 2023-10-06T05:26:28.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.656+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'1102 uv1102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.656+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d1510 con 0x55b3804cdce0 2023-10-06T05:26:28.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.658+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v128'1103 uv1076 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.658+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d1910 con 0x55b3804cdce0 2023-10-06T05:26:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.660+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v128'1104 uv1104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.660+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d1910 con 0x55b3804cdce0 2023-10-06T05:26:28.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.661+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v128'1105 uv1105 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.663 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:26:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.661+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d1910 con 0x55b3804cdce0 2023-10-06T05:26:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.662+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'1106 uv1106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.662+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d1910 con 0x55b3804cdce0 2023-10-06T05:26:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.664+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'1107 uv1107 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.664+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.665+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'1108 uv1108 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.665+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.667+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v128'1109 uv1083 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.667+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.668+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'1110 uv1110 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.669+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.670+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v128'1111 uv1111 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.672 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:26:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.670+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.671+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'1112 uv1112 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.671+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.673+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'1113 uv1113 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.673+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.675+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'1114 uv1114 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.675+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.676+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'1115 uv1115 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.676+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.678+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v128'1116 uv1096 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.678+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.679+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v128'1117 uv1117 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.679+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.682 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:26:28.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.681+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v128'1118 uv1118 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.681+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.682+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'1119 uv1119 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.682+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.684+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'1120 uv1120 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.684+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.685+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'1121 uv1121 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.685+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.687+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v128'1122 uv1095 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.687+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.689+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v128'1123 uv1123 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.689+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.692 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:26:28.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.690+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v128'1124 uv1124 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.690+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.692+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'1125 uv1125 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.692+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.693+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'1126 uv1126 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.693+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804cdce0 2023-10-06T05:26:28.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.695+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'1127 uv1127 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804cdce0 2023-10-06T05:26:28.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.695+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b3804d1ca0 0x55b3804d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.695+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d18c0 con 0x55b3804d1ca0 2023-10-06T05:26:28.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.695+0000 7ff974053700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b3804d1ca0 0x55b3804d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.695+0000 7ff974053700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b3804d1ca0 0x55b3804d4da0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.697+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'790 uv772 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.697+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d52e0 con 0x55b3804d1ca0 2023-10-06T05:26:28.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.699+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'791 uv791 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.699+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d5690 con 0x55b3804d1ca0 2023-10-06T05:26:28.702 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:26:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.700+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'792 uv792 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.700+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d5a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.701+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v128'793 uv793 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.702+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3803ddc70 con 0x55b3804d1ca0 2023-10-06T05:26:28.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.703+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'794 uv794 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.703+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d62e0 con 0x55b3804d1ca0 2023-10-06T05:26:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.704+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v128'795 uv777 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.704+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6690 con 0x55b3804d1ca0 2023-10-06T05:26:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.705+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v128'796 uv796 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.705+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.708 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:26:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.707+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v128'797 uv797 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.707+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.708+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'798 uv798 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.708+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d5e70 con 0x55b3804d1ca0 2023-10-06T05:26:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.709+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'799 uv799 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.709+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d5e70 con 0x55b3804d1ca0 2023-10-06T05:26:28.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.711+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v128'800 uv782 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.711+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.712+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v128'801 uv801 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.712+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.714+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v128'802 uv802 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.715 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:26:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.714+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804da160 con 0x55b3804d1ca0 2023-10-06T05:26:28.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.715+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v128'803 uv803 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.715+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d5e70 con 0x55b3804d1ca0 2023-10-06T05:26:28.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.716+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'804 uv804 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.717+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d5e70 con 0x55b3804d1ca0 2023-10-06T05:26:28.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.718+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v128'805 uv788 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.718+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.720+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v128'806 uv806 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.720+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.721+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v128'807 uv807 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.723 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:26:28.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.721+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.722+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'808 uv808 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.722+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.724+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'809 uv809 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.724+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d1ca0 2023-10-06T05:26:28.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.725+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'810 uv810 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.725+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b3804d7350 0x55b3804d7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.725+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d7350 2023-10-06T05:26:28.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.725+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b3804d7350 0x55b3804d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.726+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b3804d7350 0x55b3804d7720 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.727+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.727+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d7350 2023-10-06T05:26:28.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.729+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v128'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.729+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d6a40 con 0x55b3804d7350 2023-10-06T05:26:28.731 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:26:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.730+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v128'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.730+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804daf80 con 0x55b3804d7350 2023-10-06T05:26:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.731+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.731+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804daf80 con 0x55b3804d7350 2023-10-06T05:26:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.732+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.732+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d7350 2023-10-06T05:26:28.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.733+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.733+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d7350 2023-10-06T05:26:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.734+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'153 uv152 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.734+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d7350 2023-10-06T05:26:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.736+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'154 uv154 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.736+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d7350 2023-10-06T05:26:28.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.737+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'155 uv155 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.739 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:26:28.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.737+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d7350 2023-10-06T05:26:28.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.738+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'156 uv156 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.738+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d1ca0 2023-10-06T05:26:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.740+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.740+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d1ca0 2023-10-06T05:26:28.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.741+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v128'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.741+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d1ca0 2023-10-06T05:26:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.742+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v128'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.744 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:26:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.742+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804daf80 con 0x55b3804d1ca0 2023-10-06T05:26:28.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.743+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.743+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804daf80 con 0x55b3804d1ca0 2023-10-06T05:26:28.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.744+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.744+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804d9f70 con 0x55b3804d1ca0 2023-10-06T05:26:28.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.746+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804d1ca0 2023-10-06T05:26:28.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.746+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b3804da5f0 0x55b3804dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.746+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804dcf10 con 0x55b3804da5f0 2023-10-06T05:26:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.746+0000 7ff974854700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b3804da5f0 0x55b3804dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.746+0000 7ff974854700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b3804da5f0 0x55b3804dc9d0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.747+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v128'393 uv387 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.748+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804dd380 con 0x55b3804da5f0 2023-10-06T05:26:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.749+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v128'394 uv394 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.749+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804dd870 con 0x55b3804da5f0 2023-10-06T05:26:28.751 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:26:28.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.750+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v128'395 uv395 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.750+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804df170 con 0x55b3804da5f0 2023-10-06T05:26:28.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.751+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'396 uv396 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.751+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804df520 con 0x55b3804da5f0 2023-10-06T05:26:28.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.752+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'397 uv397 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.752+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804de160 con 0x55b3804da5f0 2023-10-06T05:26:28.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.753+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'398 uv398 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.753+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804de560 con 0x55b3804da5f0 2023-10-06T05:26:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.754+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v128'399 uv392 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.754+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804de560 con 0x55b3804da5f0 2023-10-06T05:26:28.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.756+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v128'400 uv400 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.756+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804de560 con 0x55b3804da5f0 2023-10-06T05:26:28.758 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:26:28.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.757+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v128'401 uv401 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.757+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804de560 con 0x55b3804da5f0 2023-10-06T05:26:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.758+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'402 uv402 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.758+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804de560 con 0x55b3804da5f0 2023-10-06T05:26:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.759+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'403 uv403 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.759+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b3804df8d0 0x55b3804e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.759+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804df8d0 2023-10-06T05:26:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.760+0000 7ff974053700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b3804df8d0 0x55b3804e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.760+0000 7ff974053700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b3804df8d0 0x55b3804e1cb0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.761+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v128'178 uv177 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804df8d0 2023-10-06T05:26:28.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.761+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804df8d0 2023-10-06T05:26:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.762+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v128'179 uv179 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804df8d0 2023-10-06T05:26:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.763+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804df8d0 2023-10-06T05:26:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.764+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v128'180 uv180 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804df8d0 2023-10-06T05:26:28.765 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:26:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.764+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804df8d0 2023-10-06T05:26:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.765+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'181 uv181 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804df8d0 2023-10-06T05:26:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.765+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804df8d0 2023-10-06T05:26:28.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.766+0000 7ff974053700 1 -- 172.21.15.45:0/2411542938 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'182 uv182 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff964016440 con 0x55b3804df8d0 2023-10-06T05:26:28.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.766+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804d7350 2023-10-06T05:26:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.767+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.767+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804d7350 2023-10-06T05:26:28.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.769+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v128'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.769+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804d7350 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:26:28.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.770+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v128'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.770+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e3200 con 0x55b3804d7350 2023-10-06T05:26:28.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.771+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.771+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e3200 con 0x55b3804d7350 2023-10-06T05:26:28.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.773+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.773+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e21f0 con 0x55b3804d7350 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.774+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.774+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b3804e2870 0x55b3804e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.774+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5260 con 0x55b3804e2870 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.774+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b3804e2870 0x55b3804e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.775+0000 7ff973852700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b3804e2870 0x55b3804e4c50 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.776+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v128'355 uv349 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.776+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.777+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v128'356 uv356 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.777+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.780 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:26:28.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.778+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v128'357 uv357 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.779+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.780+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'358 uv358 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.780+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.781+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'359 uv359 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.781+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.782+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v128'360 uv354 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.782+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.783+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v128'361 uv361 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.783+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.786 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:26:28.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.784+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v128'362 uv362 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.784+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5640 con 0x55b3804e2870 2023-10-06T05:26:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.786+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'363 uv363 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.786+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e83b0 con 0x55b3804e2870 2023-10-06T05:26:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.787+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'364 uv364 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804e2870 2023-10-06T05:26:28.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.787+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.788+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v128'157 uv155 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.788+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.789+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'158 uv158 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.789+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.792 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:26:28.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.790+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v128'159 uv159 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.790+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.792+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'160 uv160 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.792+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b3804e6440 0x55b3804e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.792+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804e6440 2023-10-06T05:26:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.792+0000 7ff974854700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b3804e6440 0x55b3804e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.793+0000 7ff974854700 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b3804e6440 0x55b3804e8b00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.794+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804e6440 2023-10-06T05:26:28.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.794+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804e6440 2023-10-06T05:26:28.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.795+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v128'214 uv214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804e6440 2023-10-06T05:26:28.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.795+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804e6440 2023-10-06T05:26:28.798 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:26:28.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.796+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v128'215 uv215 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804e6440 2023-10-06T05:26:28.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.796+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804e6440 2023-10-06T05:26:28.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.797+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804e6440 2023-10-06T05:26:28.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.797+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804e6440 2023-10-06T05:26:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.799+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804e6440 2023-10-06T05:26:28.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.799+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804e6440 2023-10-06T05:26:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.800+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804e6440 2023-10-06T05:26:28.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.800+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804da5f0 2023-10-06T05:26:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.801+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v128'181 uv180 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.801+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804da5f0 2023-10-06T05:26:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.802+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v128'182 uv182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.802+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804da5f0 2023-10-06T05:26:28.805 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:26:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.803+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v128'183 uv183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.803+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804da5f0 2023-10-06T05:26:28.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.805+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'184 uv184 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.805+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804da5f0 2023-10-06T05:26:28.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.806+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'185 uv185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7ff96c021030 con 0x55b3804da5f0 2023-10-06T05:26:28.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.806+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.807+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v128'442 uv433 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.807+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.808+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v128'443 uv443 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.808+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.811 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:26:28.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.810+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v128'444 uv444 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.810+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b30 con 0x55b3804d7350 2023-10-06T05:26:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.811+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v128'445 uv445 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.811+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b00 con 0x55b3804d7350 2023-10-06T05:26:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.812+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'446 uv446 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.812+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b00 con 0x55b3804d7350 2023-10-06T05:26:28.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.813+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v128'447 uv440 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.813+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b00 con 0x55b3804d7350 2023-10-06T05:26:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.814+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v128'448 uv448 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.814+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b00 con 0x55b3804d7350 2023-10-06T05:26:28.817 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:26:28.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.816+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v128'449 uv449 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.816+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b00 con 0x55b3804d7350 2023-10-06T05:26:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.817+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'450 uv450 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.817+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b00 con 0x55b3804d7350 2023-10-06T05:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.818+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'451 uv451 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.818+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e5b00 con 0x55b3804d7350 2023-10-06T05:26:28.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.819+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'452 uv452 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.819+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55b3804e9040 con 0x55b3804d7350 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff973852700 1 -- 172.21.15.45:0/2411542938 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'453 uv453 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff968022040 con 0x55b3804d7350 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b3804d7350 msgr2=0x55b3804d7720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55b3804d7350 0x55b3804d7720 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b3804cdce0 msgr2=0x55b3804d00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55b3804cdce0 0x55b3804d00c0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b3804e2870 msgr2=0x55b3804e4c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b3804e2870 0x55b3804e4c50 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b3804d1ca0 msgr2=0x55b3804d4da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55b3804d1ca0 0x55b3804d4da0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b3804e6440 msgr2=0x55b3804e8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55b3804e6440 0x55b3804e8b00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 reap_dead start 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b3804df8d0 msgr2=0x55b3804e1cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55b3804df8d0 0x55b3804e1cb0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff974854700 1 -- 172.21.15.45:0/2411542938 reap_dead start 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b3804da5f0 msgr2=0x55b3804dc9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b3804da5f0 0x55b3804dc9d0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff960065890 msgr2=0x7ff960067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff960065890 0x7ff960067d40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff964009d60 tx=0x7ff964005a50 comp rx=0 tx=0).stop 2023-10-06T05:26:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b6140 msgr2=0x55b3803dd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b6140 0x55b3803dd6b0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x55b3803e2ec0 tx=0x7ff96c015000 comp rx=0 tx=0).stop 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 shutdown_connections 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55b3804da5f0 0x55b3804dc9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b3804b6140 0x55b3803dd6b0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 --2- 172.21.15.45:0/2411542938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff960065890 0x7ff960067d40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.820+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 >> 172.21.15.45:0/2411542938 conn(0x55b3803df5f0 msgr2=0x55b3804b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.821+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 shutdown_connections 2023-10-06T05:26:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.821+0000 7ff97ae35040 1 -- 172.21.15.45:0/2411542938 wait complete. 2023-10-06T05:26:28.826 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:26:28.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 Processor -- start 2023-10-06T05:26:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 -- start start 2023-10-06T05:26:28.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 0x561c53475b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c5345fc80 0x561c53512a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53503af0 0x561c53512fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c533d6320 con 0x561c5345fc80 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5337a980 con 0x561c53503af0 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f24a44b1040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c5337a620 con 0x561c5345bdd0 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.917+0000 7f249cece700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c5345fc80 0x561c53512a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c5345fc80 0x561c53512a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35158/0 (socket says 172.21.15.45:35158) 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 -- 172.21.15.45:0/2849371615 learned_addr learned my addr 172.21.15.45:0/2849371615 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249ded0700 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53503af0 0x561c53512fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249d6cf700 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 0x561c53475b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 -- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 msgr2=0x561c53475b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 0x561c53475b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 -- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53503af0 msgr2=0x561c53512fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53503af0 0x561c53512fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 -- 172.21.15.45:0/2849371615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498005cf0 con 0x561c5345fc80 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249ded0700 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53503af0 0x561c53512fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f249cece700 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c5345fc80 0x561c53512a90 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f249400b660 tx=0x7f2494001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a880e9482ca600d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f248f7fe700 1 -- 172.21.15.45:0/2849371615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494014070 con 0x561c5345fc80 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f248f7fe700 1 -- 172.21.15.45:0/2849371615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249400d5d0 con 0x561c5345fc80 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.918+0000 7f248f7fe700 1 -- 172.21.15.45:0/2849371615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400d970 con 0x561c5345fc80 2023-10-06T05:26:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 -- 172.21.15.45:0/2849371615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c5345fc80 msgr2=0x561c53512a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c5345fc80 0x561c53512a90 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f249400b660 tx=0x7f2494001b30 comp rx=0 tx=0).stop 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 -- 172.21.15.45:0/2849371615 shutdown_connections 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53503af0 0x561c53512fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 0x561c53475b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2849371615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c5345fc80 0x561c53512a90 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 -- 172.21.15.45:0/2849371615 >> 172.21.15.45:0/2849371615 conn(0x561c534304a0 msgr2=0x561c53502df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 -- 172.21.15.45:0/2849371615 shutdown_connections 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 -- 172.21.15.45:0/2849371615 wait complete. 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.919+0000 7f24a44b1040 1 Processor -- start 2023-10-06T05:26:28.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f24a44b1040 1 -- start start 2023-10-06T05:26:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f24a44b1040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 0x561c534f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f24a44b1040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c5345fc80 0x561c53477f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f24a44b1040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c53503af0 0x561c53478480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f24a44b1040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c53371400 con 0x561c53503af0 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f24a44b1040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c53474aa0 con 0x561c5345fc80 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f24a44b1040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c53474d40 con 0x561c5345bdd0 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.920+0000 7f249ded0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c53503af0 0x561c53478480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c53503af0 0x561c53478480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35160/0 (socket says 172.21.15.45:35160) 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 -- 172.21.15.45:0/2015716641 learned_addr learned my addr 172.21.15.45:0/2015716641 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249d6cf700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 0x561c534f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249cece700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c5345fc80 0x561c53477f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 -- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 msgr2=0x561c534f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561c5345bdd0 0x561c534f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 -- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c5345fc80 msgr2=0x561c53477f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c5345fc80 0x561c53477f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 -- 172.21.15.45:0/2015716641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249400b210 con 0x561c53503af0 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f249ded0700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c53503af0 0x561c53478480 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f249000d4a0 tx=0x7f2490008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f248dffb700 1 -- 172.21.15.45:0/2015716641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24900030d0 con 0x561c53503af0 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f248dffb700 1 -- 172.21.15.45:0/2015716641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2490003230 con 0x561c53503af0 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c5337cae0 con 0x561c53503af0 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f248dffb700 1 -- 172.21.15.45:0/2015716641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490001650 con 0x561c53503af0 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.921+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c5337c540 con 0x561c53503af0 2023-10-06T05:26:28.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.923+0000 7f248dffb700 1 -- 172.21.15.45:0/2015716641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2490001a40 con 0x561c53503af0 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.923+0000 7f248dffb700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2480065890 0x7f2480067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.923+0000 7f248dffb700 1 -- 172.21.15.45:0/2015716641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2490083990 con 0x561c53503af0 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.924+0000 7f249d6cf700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2480065890 0x7f2480067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.924+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c53474600 0x561c53514000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.924+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x561c534f9ab0 con 0x561c53474600 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.924+0000 7f249cece700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c53474600 0x561c53514000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.924+0000 7f249d6cf700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2480065890 0x7f2480067d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2498005a90 tx=0x7f2498013000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.924+0000 7f249cece700 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c53474600 0x561c53514000 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f249cece700 1 -- 172.21.15.45:0/2015716641 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v128'157 uv157 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f2494016070 con 0x561c53474600 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c53474600 msgr2=0x561c53514000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561c53474600 0x561c53514000 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2480065890 msgr2=0x7f2480067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2480065890 0x7f2480067d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2498005a90 tx=0x7f2498013000 comp rx=0 tx=0).stop 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c53503af0 msgr2=0x561c53478480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 --2- 172.21.15.45:0/2015716641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c53503af0 0x561c53478480 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f249000d4a0 tx=0x7f2490008040 comp rx=0 tx=0).stop 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f249ded0700 1 -- 172.21.15.45:0/2015716641 reap_dead start 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 shutdown_connections 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 >> 172.21.15.45:0/2015716641 conn(0x561c534304a0 msgr2=0x561c53502df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 shutdown_connections 2023-10-06T05:26:28.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:28.926+0000 7f24a44b1040 1 -- 172.21.15.45:0/2015716641 wait complete. 2023-10-06T05:26:28.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:26:29.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.070+0000 7f1aa8f24700 1 Processor -- start 2023-10-06T05:26:29.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.070+0000 7f1aa8f24700 1 -- start start 2023-10-06T05:26:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.071+0000 7f1aa8f24700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1aa40ccb70 0x7f1aa40aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.071+0000 7f1aa8f24700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa40c8b70 0x7f1aa40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.071+0000 7f1aa8f24700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa40c3b50 con 0x7f1aa4157330 2023-10-06T05:26:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.071+0000 7f1aa8f24700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa4157700 con 0x7f1aa40ccb70 2023-10-06T05:26:29.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.071+0000 7f1aa8f24700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40c98e0 con 0x7f1aa40c8b70 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1aa4157330 0x7f1aa40ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45024/0 (socket says 172.21.15.45:45024) 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa259c700 1 -- 172.21.15.45:0/118331669 learned_addr learned my addr 172.21.15.45:0/118331669 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa1d9b700 1 --2- 172.21.15.45:0/118331669 >> v2:172.21.15.144:3300/0 conn(0x7f1aa40ccb70 0x7f1aa40aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa2d9d700 1 --2- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa40c8b70 0x7f1aa40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa159a700 1 -- 172.21.15.45:0/118331669 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214978598 0 0) 0x7f1aa40c3b50 con 0x7f1aa4157330 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa159a700 1 -- 172.21.15.45:0/118331669 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a90003570 con 0x7f1aa4157330 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa1d9b700 1 -- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa40c8b70 msgr2=0x7f1aa40c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa1d9b700 1 --2- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa40c8b70 0x7f1aa40c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa1d9b700 1 -- 172.21.15.45:0/118331669 >> v1:172.21.15.45:6789/0 conn(0x7f1aa4157330 legacy=0x7f1aa40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa1d9b700 1 -- 172.21.15.45:0/118331669 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40c9bc0 con 0x7f1aa40ccb70 2023-10-06T05:26:29.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa2d9d700 1 --2- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa40c8b70 0x7f1aa40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:29.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa1d9b700 1 --2- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 0x7f1aa40aaae0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0098e0 tx=0x7f1a8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17060e8ac16b616 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa159a700 1 -- 172.21.15.45:0/118331669 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c011630 con 0x7f1aa40ccb70 2023-10-06T05:26:29.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa159a700 1 -- 172.21.15.45:0/118331669 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c011790 con 0x7f1aa40ccb70 2023-10-06T05:26:29.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.072+0000 7f1aa159a700 1 -- 172.21.15.45:0/118331669 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c011b30 con 0x7f1aa40ccb70 2023-10-06T05:26:29.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 -- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 msgr2=0x7f1aa40aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 0x7f1aa40aaae0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0098e0 tx=0x7f1a8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 -- 172.21.15.45:0/118331669 shutdown_connections 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa40c8b70 0x7f1aa40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/118331669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 0x7f1aa40aaae0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 -- 172.21.15.45:0/118331669 >> 172.21.15.45:0/118331669 conn(0x7f1aa4067e00 msgr2=0x7f1aa40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 -- 172.21.15.45:0/118331669 shutdown_connections 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.073+0000 7f1aa8f24700 1 -- 172.21.15.45:0/118331669 wait complete. 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.074+0000 7f1aa8f24700 1 Processor -- start 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.074+0000 7f1aa8f24700 1 -- start start 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa8f24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa40c8b70 0x7f1aa40ab170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa8f24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 0x7f1aa4150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa8f24700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa4157330 0x7f1aa40c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa8f24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415a550 con 0x7f1aa40c8b70 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa8f24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40ab6b0 con 0x7f1aa40ccb70 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa8f24700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40ab8d0 con 0x7f1aa4157330 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa40c8b70 0x7f1aa40ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa40c8b70 0x7f1aa40ab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35164/0 (socket says 172.21.15.45:35164) 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa259c700 1 -- 172.21.15.45:0/2892637063 learned_addr learned my addr 172.21.15.45:0/2892637063 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa2d9d700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa4157330 0x7f1aa40c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.075+0000 7f1aa1d9b700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 0x7f1aa4150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa259c700 1 -- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa4157330 msgr2=0x7f1aa40c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa259c700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa4157330 0x7f1aa40c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa259c700 1 -- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 msgr2=0x7f1aa4150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa259c700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 0x7f1aa4150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa259c700 1 -- 172.21.15.45:0/2892637063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c009a20 con 0x7f1aa40c8b70 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa259c700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa40c8b70 0x7f1aa40ab170 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400c790 tx=0x7f1a94007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2892637063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940072e0 con 0x7f1aa40c8b70 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2892637063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a94007440 con 0x7f1aa40c8b70 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c009420 con 0x7f1aa40c8b70 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2892637063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940057f0 con 0x7f1aa40c8b70 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.076+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa416aef0 con 0x7f1aa40c8b70 2023-10-06T05:26:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.077+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa406e350 con 0x7f1aa40c8b70 2023-10-06T05:26:29.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.078+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2892637063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a94005c20 con 0x7f1aa40c8b70 2023-10-06T05:26:29.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.078+0000 7f1a9a7fc700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.078+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2892637063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1a94084760 con 0x7f1aa40c8b70 2023-10-06T05:26:29.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.078+0000 7f1aa1d9b700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.079+0000 7f1aa1d9b700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0091e0 tx=0x7f1a8c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.083+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2892637063 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a9404b5a0 con 0x7f1aa40c8b70 2023-10-06T05:26:29.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.331+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f1aa404bb40 con 0x7f1aa40c8b70 2023-10-06T05:26:29.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.584+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2892637063 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v324) v1 ==== 162+0+0 (secure 0 0 0) 0x7f1a94017020 con 0x7f1aa40c8b70 2023-10-06T05:26:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.587+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 msgr2=0x7f1a840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.587+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0091e0 tx=0x7f1a8c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.587+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa40c8b70 msgr2=0x7f1aa40ab170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.587+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa40c8b70 0x7f1aa40ab170 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400c790 tx=0x7f1a94007c80 comp rx=0 tx=0).stop 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 shutdown_connections 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1a84066f30 0x7f1a840693e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aa4157330 0x7f1aa40c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1aa40c8b70 0x7f1aa40ab170 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 --2- 172.21.15.45:0/2892637063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa40ccb70 0x7f1aa4150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 >> 172.21.15.45:0/2892637063 conn(0x7f1aa4067e00 msgr2=0x7f1aa40cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 shutdown_connections 2023-10-06T05:26:29.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:29.588+0000 7f1aa8f24700 1 -- 172.21.15.45:0/2892637063 wait complete. 2023-10-06T05:26:29.591 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:26:29.592 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:29.591+0000 7fe648332700 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer 2023-10-06T05:26:30.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.147+0000 7f4b31133700 1 Processor -- start 2023-10-06T05:26:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.147+0000 7f4b31133700 1 -- start start 2023-10-06T05:26:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.148+0000 7f4b31133700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b2c0d0760 0x7f4b2c0b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.148+0000 7f4b31133700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c0cb920 con 0x7f4b2c0b50c0 2023-10-06T05:26:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.148+0000 7f4b31133700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c15de70 con 0x7f4b2c15d120 2023-10-06T05:26:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.148+0000 7f4b31133700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c0b1310 con 0x7f4b2c0d0760 2023-10-06T05:26:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b2a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b2c0d0760 0x7f4b2c0b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b2ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4b2c0b50c0 0x7f4b2c0d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45032/0 (socket says 172.21.15.45:45032) 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b2ad9d700 1 -- 172.21.15.45:0/1888725540 learned_addr learned my addr 172.21.15.45:0/1888725540 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b2a59c700 1 -- 172.21.15.45:0/1888725540 >> v1:172.21.15.144:6789/0 conn(0x7f4b2c15d120 legacy=0x7f4b2c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b2a59c700 1 -- 172.21.15.45:0/1888725540 >> v1:172.21.15.45:6789/0 conn(0x7f4b2c0b50c0 legacy=0x7f4b2c0d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b2a59c700 1 -- 172.21.15.45:0/1888725540 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0cb030 con 0x7f4b2c0d0760 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b29d9b700 1 -- 172.21.15.45:0/1888725540 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086503954 0 0) 0x7f4b2c0cb920 con 0x7f4b2c0b50c0 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b2a59c700 1 --2- 172.21.15.45:0/1888725540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c0b5490 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f4b140098e0 tx=0x7f4b140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83866cb5ceb060 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b29d9b700 1 -- 172.21.15.45:0/1888725540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14009e80 con 0x7f4b2c0d0760 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b29d9b700 1 -- 172.21.15.45:0/1888725540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1400b860 con 0x7f4b2c0d0760 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.149+0000 7f4b29d9b700 1 -- 172.21.15.45:0/1888725540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400bbe0 con 0x7f4b2c0d0760 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.150+0000 7f4b31133700 1 -- 172.21.15.45:0/1888725540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 msgr2=0x7f4b2c0b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.150+0000 7f4b31133700 1 --2- 172.21.15.45:0/1888725540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c0b5490 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f4b140098e0 tx=0x7f4b140091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.150+0000 7f4b31133700 1 -- 172.21.15.45:0/1888725540 shutdown_connections 2023-10-06T05:26:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.150+0000 7f4b31133700 1 --2- 172.21.15.45:0/1888725540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c0b5490 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.150+0000 7f4b31133700 1 -- 172.21.15.45:0/1888725540 >> 172.21.15.45:0/1888725540 conn(0x7f4b2c06ed50 msgr2=0x7f4b2c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.150+0000 7f4b31133700 1 -- 172.21.15.45:0/1888725540 shutdown_connections 2023-10-06T05:26:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.150+0000 7f4b31133700 1 -- 172.21.15.45:0/1888725540 wait complete. 2023-10-06T05:26:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.151+0000 7f4b31133700 1 Processor -- start 2023-10-06T05:26:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.151+0000 7f4b31133700 1 -- start start 2023-10-06T05:26:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b31133700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2c0b50c0 0x7f4b2c15ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b31133700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b31133700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b2c15d120 0x7f4b2c169ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b31133700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15dc90 con 0x7f4b2c0d0760 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b31133700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c0c5540 con 0x7f4b2c0b50c0 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b31133700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c0a6c10 con 0x7f4b2c15d120 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b2a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b2a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c169ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35184/0 (socket says 172.21.15.45:35184) 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b2a59c700 1 -- 172.21.15.45:0/4129336503 learned_addr learned my addr 172.21.15.45:0/4129336503 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b2ad9d700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2c0b50c0 0x7f4b2c15ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.152+0000 7f4b2b59e700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b2c15d120 0x7f4b2c169ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b2a59c700 1 -- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b2c15d120 msgr2=0x7f4b2c169ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b2a59c700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b2c15d120 0x7f4b2c169ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b2a59c700 1 -- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2c0b50c0 msgr2=0x7f4b2c15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b2a59c700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2c0b50c0 0x7f4b2c15ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b2a59c700 1 -- 172.21.15.45:0/4129336503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b14009420 con 0x7f4b2c0d0760 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b2b59e700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b2c15d120 0x7f4b2c169ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b2a59c700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c169ab0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f4b14013000 tx=0x7f4b14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b22ffd700 1 -- 172.21.15.45:0/4129336503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1401f070 con 0x7f4b2c0d0760 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b22ffd700 1 -- 172.21.15.45:0/4129336503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14000c90 con 0x7f4b2c0d0760 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c16a690 con 0x7f4b2c0d0760 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b22ffd700 1 -- 172.21.15.45:0/4129336503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14025420 con 0x7f4b2c0d0760 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.153+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c0c4ef0 con 0x7f4b2c0d0760 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.154+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c0886f0 con 0x7f4b2c0d0760 2023-10-06T05:26:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.155+0000 7f4b22ffd700 1 -- 172.21.15.45:0/4129336503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b14023070 con 0x7f4b2c0d0760 2023-10-06T05:26:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.155+0000 7f4b22ffd700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b0c066f30 0x7f4b0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.155+0000 7f4b22ffd700 1 -- 172.21.15.45:0/4129336503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f4b14086950 con 0x7f4b2c0d0760 2023-10-06T05:26:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.155+0000 7f4b2ad9d700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b0c066f30 0x7f4b0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.156+0000 7f4b2ad9d700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b0c066f30 0x7f4b0c0693e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c007fd0 tx=0x7f4b1c007d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.160+0000 7f4b22ffd700 1 -- 172.21.15.45:0/4129336503 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b14052cc0 con 0x7f4b2c0d0760 2023-10-06T05:26:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.411+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b2c0cb330 con 0x7f4b2c0d0760 2023-10-06T05:26:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.411+0000 7f4b22ffd700 1 -- 172.21.15.45:0/4129336503 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 325 v325) v1 ==== 95+0+4606 (secure 0 0 0) 0x7f4b1402a070 con 0x7f4b2c0d0760 2023-10-06T05:26:30.414 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:30.414 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":325,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":325,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:29.585822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8447":{"gid":8447,"name":"b","rank":0,"incarnation":325,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/603338967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":603338967},{"type":"v1","addr":"172.21.15.144:6835","nonce":603338967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b0c066f30 msgr2=0x7f4b0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b0c066f30 0x7f4b0c0693e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c007fd0 tx=0x7f4b1c007d10 comp rx=0 tx=0).stop 2023-10-06T05:26:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 msgr2=0x7f4b2c169ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c169ab0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f4b14013000 tx=0x7f4b14017000 comp rx=0 tx=0).stop 2023-10-06T05:26:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 shutdown_connections 2023-10-06T05:26:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4b0c066f30 0x7f4b0c0693e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b2c15d120 0x7f4b2c169ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b2c0b50c0 0x7f4b2c15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 --2- 172.21.15.45:0/4129336503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b2c0d0760 0x7f4b2c169ab0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 >> 172.21.15.45:0/4129336503 conn(0x7f4b2c06ed50 msgr2=0x7f4b2c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 shutdown_connections 2023-10-06T05:26:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:30.414+0000 7f4b31133700 1 -- 172.21.15.45:0/4129336503 wait complete. 2023-10-06T05:26:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 325 2023-10-06T05:26:30.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 325, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:26:29.585822+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8447}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8447': {'gid': 8447, 'name': 'b', 'rank': 0, 'incarnation': 325, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/603338967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 603338967}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 603338967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:26:30.429 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8447:up:replay 2023-10-06T05:26:30.429 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:26:33.563 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:33.561+0000 7fe64bb39700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:35.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:35.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.972+0000 7f43f27ec700 1 Processor -- start 2023-10-06T05:26:35.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.972+0000 7f43f27ec700 1 -- start start 2023-10-06T05:26:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43f27ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43ec159200 0x7f43ec0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43f27ec700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43ec1595d0 0x7f43ec06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43f27ec700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ec0c7d00 con 0x7f43ec06b5c0 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43f27ec700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0c7780 con 0x7f43ec159200 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43f27ec700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0c86d0 con 0x7f43ec1595d0 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43ebfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43ec159200 0x7f43ec0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43f0d89700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f43ec06b5c0 0x7f43ec0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45044/0 (socket says 172.21.15.45:45044) 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43f0d89700 1 -- 172.21.15.45:0/3343577498 learned_addr learned my addr 172.21.15.45:0/3343577498 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43eb7fe700 1 --2- 172.21.15.45:0/3343577498 >> v2:172.21.15.144:3300/0 conn(0x7f43ec1595d0 0x7f43ec06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43ebfff700 1 -- 172.21.15.45:0/3343577498 >> v2:172.21.15.144:3300/0 conn(0x7f43ec1595d0 msgr2=0x7f43ec06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43ebfff700 1 --2- 172.21.15.45:0/3343577498 >> v2:172.21.15.144:3300/0 conn(0x7f43ec1595d0 0x7f43ec06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43ebfff700 1 -- 172.21.15.45:0/3343577498 >> v1:172.21.15.45:6789/0 conn(0x7f43ec06b5c0 legacy=0x7f43ec0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43ebfff700 1 -- 172.21.15.45:0/3343577498 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec06bb40 con 0x7f43ec159200 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.973+0000 7f43eaffd700 1 -- 172.21.15.45:0/3343577498 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221328023 0 0) 0x7f43ec0c7d00 con 0x7f43ec06b5c0 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43ebfff700 1 --2- 172.21.15.45:0/3343577498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec159200 0x7f43ec0c8190 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f43ec06e790 tx=0x7f43dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cd2d8213995bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43eaffd700 1 -- 172.21.15.45:0/3343577498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00b6b0 con 0x7f43ec159200 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43eaffd700 1 -- 172.21.15.45:0/3343577498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc00b810 con 0x7f43ec159200 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43eaffd700 1 -- 172.21.15.45:0/3343577498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00bbb0 con 0x7f43ec159200 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 -- 172.21.15.45:0/3343577498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec159200 msgr2=0x7f43ec0c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 --2- 172.21.15.45:0/3343577498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec159200 0x7f43ec0c8190 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f43ec06e790 tx=0x7f43dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 -- 172.21.15.45:0/3343577498 shutdown_connections 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 --2- 172.21.15.45:0/3343577498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec159200 0x7f43ec0c8190 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 --2- 172.21.15.45:0/3343577498 >> v2:172.21.15.144:3300/0 conn(0x7f43ec1595d0 0x7f43ec06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 -- 172.21.15.45:0/3343577498 >> 172.21.15.45:0/3343577498 conn(0x7f43ec06f090 msgr2=0x7f43ec06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:35.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 -- 172.21.15.45:0/3343577498 shutdown_connections 2023-10-06T05:26:35.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.974+0000 7f43f27ec700 1 -- 172.21.15.45:0/3343577498 wait complete. 2023-10-06T05:26:35.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.976+0000 7f43f27ec700 1 Processor -- start 2023-10-06T05:26:35.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.976+0000 7f43f27ec700 1 -- start start 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43f27ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec06b5c0 0x7f43ec16a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43f27ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ec159200 0x7f43ec168f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43f27ec700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ec1595d0 0x7f43ec0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43f27ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0cc7d0 con 0x7f43ec06b5c0 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43f27ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec16a740 con 0x7f43ec159200 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43f27ec700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec16d320 con 0x7f43ec1595d0 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec06b5c0 0x7f43ec16a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec06b5c0 0x7f43ec16a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35204/0 (socket says 172.21.15.45:35204) 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 -- 172.21.15.45:0/1766333536 learned_addr learned my addr 172.21.15.45:0/1766333536 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:35.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43eb7fe700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ec159200 0x7f43ec168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43f0d89700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ec1595d0 0x7f43ec0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 -- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ec1595d0 msgr2=0x7f43ec0c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ec1595d0 0x7f43ec0c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 -- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ec159200 msgr2=0x7f43ec168f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ec159200 0x7f43ec168f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 -- 172.21.15.45:0/1766333536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc009420 con 0x7f43ec06b5c0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43ebfff700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec06b5c0 0x7f43ec16a200 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f43ec06aed0 tx=0x7f43dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43d3fff700 1 -- 172.21.15.45:0/1766333536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00bb90 con 0x7f43ec06b5c0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43d3fff700 1 -- 172.21.15.45:0/1766333536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc011420 con 0x7f43ec06b5c0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.977+0000 7f43d3fff700 1 -- 172.21.15.45:0/1766333536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0117c0 con 0x7f43ec06b5c0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.978+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec16d600 con 0x7f43ec06b5c0 2023-10-06T05:26:35.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.978+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec16dbc0 con 0x7f43ec06b5c0 2023-10-06T05:26:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.978+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec06e2f0 con 0x7f43ec06b5c0 2023-10-06T05:26:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.979+0000 7f43d3fff700 1 -- 172.21.15.45:0/1766333536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43dc011bb0 con 0x7f43ec06b5c0 2023-10-06T05:26:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.979+0000 7f43d3fff700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.979+0000 7f43d3fff700 1 -- 172.21.15.45:0/1766333536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f43dc086c70 con 0x7f43ec06b5c0 2023-10-06T05:26:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.980+0000 7f43eb7fe700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.980+0000 7f43eb7fe700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006fd0 tx=0x7f43e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:35.984+0000 7f43d3fff700 1 -- 172.21.15.45:0/1766333536 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43dc052fe0 con 0x7f43ec06b5c0 2023-10-06T05:26:36.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.228+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43ec06be20 con 0x7f43ec06b5c0 2023-10-06T05:26:36.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.229+0000 7f43d3fff700 1 -- 172.21.15.45:0/1766333536 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 328 v328) v1 ==== 95+0+4606 (secure 0 0 0) 0x7f43dc011e60 con 0x7f43ec06b5c0 2023-10-06T05:26:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:36.232 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":328,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":328,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:32.594042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8447":{"gid":8447,"name":"b","rank":0,"incarnation":325,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6835/603338967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":603338967},{"type":"v1","addr":"172.21.15.144:6835","nonce":603338967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f43d8066f30 msgr2=0x7f43d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006fd0 tx=0x7f43e0008040 comp rx=0 tx=0).stop 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec06b5c0 msgr2=0x7f43ec16a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec06b5c0 0x7f43ec16a200 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f43ec06aed0 tx=0x7f43dc017000 comp rx=0 tx=0).stop 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 shutdown_connections 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ec1595d0 0x7f43ec0c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec06b5c0 0x7f43ec16a200 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 --2- 172.21.15.45:0/1766333536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ec159200 0x7f43ec168f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 >> 172.21.15.45:0/1766333536 conn(0x7f43ec06f090 msgr2=0x7f43ec0aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 shutdown_connections 2023-10-06T05:26:36.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.231+0000 7f43f27ec700 1 -- 172.21.15.45:0/1766333536 wait complete. 2023-10-06T05:26:36.235 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 328 2023-10-06T05:26:36.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 328, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:50.763116+0000', 'modified': '2023-10-06T05:26:32.594042+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8447}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8447': {'gid': 8447, 'name': 'b', 'rank': 0, 'incarnation': 325, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.144:6835/603338967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 603338967}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 603338967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:26:36.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:26:36.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:26:36.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.377+0000 7fc34f3f8700 1 Processor -- start 2023-10-06T05:26:36.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.377+0000 7fc34f3f8700 1 -- start start 2023-10-06T05:26:36.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc34f3f8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc3480c93c0 0x7fc348156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc34f3f8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34815b000 con 0x7fc3480c8ff0 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc34f3f8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34815b2e0 con 0x7fc348157030 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc34f3f8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3480c9830 con 0x7fc3480c93c0 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc34d194700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc3480c8ff0 0x7fc3480caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51506/0 (socket says 172.21.15.45:51506) 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc34d194700 1 -- 172.21.15.45:0/430504366 learned_addr learned my addr 172.21.15.45:0/430504366 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc34c993700 1 --2- 172.21.15.45:0/430504366 >> v2:172.21.15.144:3300/0 conn(0x7fc3480c93c0 0x7fc348156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.378+0000 7fc33ffff700 1 -- 172.21.15.45:0/430504366 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198569722 0 0) 0x7fc34815b000 con 0x7fc3480c8ff0 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc33ffff700 1 -- 172.21.15.45:0/430504366 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc334003570 con 0x7fc3480c8ff0 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc34c993700 1 -- 172.21.15.45:0/430504366 >> v1:172.21.15.144:6789/0 conn(0x7fc348157030 legacy=0x7fc3480cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc33ffff700 1 -- 172.21.15.45:0/430504366 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 18353809 0 0) 0x7fc334003570 con 0x7fc3480c8ff0 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc34c993700 1 -- 172.21.15.45:0/430504366 >> v1:172.21.15.45:6789/0 conn(0x7fc3480c8ff0 legacy=0x7fc3480caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc34c993700 1 -- 172.21.15.45:0/430504366 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34815b5c0 con 0x7fc3480c93c0 2023-10-06T05:26:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc34c993700 1 --2- 172.21.15.45:0/430504366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c93c0 0x7fc348156af0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc3380098e0 tx=0x7fc3380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d5986c32c0ad9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc33ffff700 1 -- 172.21.15.45:0/430504366 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344001470 con 0x7fc3480c93c0 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc33ffff700 1 -- 172.21.15.45:0/430504366 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338011420 con 0x7fc3480c93c0 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.379+0000 7fc33ffff700 1 -- 172.21.15.45:0/430504366 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3380117c0 con 0x7fc3480c93c0 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.380+0000 7fc34f3f8700 1 -- 172.21.15.45:0/430504366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c93c0 msgr2=0x7fc348156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.380+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/430504366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c93c0 0x7fc348156af0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc3380098e0 tx=0x7fc3380091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.380+0000 7fc34f3f8700 1 -- 172.21.15.45:0/430504366 shutdown_connections 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.380+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/430504366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3480c93c0 0x7fc348156af0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.380+0000 7fc34f3f8700 1 -- 172.21.15.45:0/430504366 >> 172.21.15.45:0/430504366 conn(0x7fc3480cdf50 msgr2=0x7fc3480ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.380+0000 7fc34f3f8700 1 -- 172.21.15.45:0/430504366 shutdown_connections 2023-10-06T05:26:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.380+0000 7fc34f3f8700 1 -- 172.21.15.45:0/430504366 wait complete. 2023-10-06T05:26:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.381+0000 7fc34f3f8700 1 Processor -- start 2023-10-06T05:26:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.381+0000 7fc34f3f8700 1 -- start start 2023-10-06T05:26:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34f3f8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c8ff0 0x7fc348148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34f3f8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480c93c0 0x7fc348069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34f3f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc348157030 0x7fc34806a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34f3f8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c7cf0 con 0x7fc3480c93c0 2023-10-06T05:26:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34f3f8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c7120 con 0x7fc348157030 2023-10-06T05:26:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34f3f8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c73c0 con 0x7fc3480c8ff0 2023-10-06T05:26:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34c993700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480c93c0 0x7fc348069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34c993700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480c93c0 0x7fc348069e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56368/0 (socket says 172.21.15.45:56368) 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34c993700 1 -- 172.21.15.45:0/2909688559 learned_addr learned my addr 172.21.15.45:0/2909688559 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34d194700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c8ff0 0x7fc348148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.382+0000 7fc34d995700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc348157030 0x7fc34806a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34c993700 1 -- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c8ff0 msgr2=0x7fc348148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34c993700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c8ff0 0x7fc348148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34c993700 1 -- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc348157030 msgr2=0x7fc34806a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34c993700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc348157030 0x7fc34806a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34c993700 1 -- 172.21.15.45:0/2909688559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009420 con 0x7fc3480c93c0 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34d995700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc348157030 0x7fc34806a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34c993700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480c93c0 0x7fc348069e00 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc3380091e0 tx=0x7fc338016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc33d7fa700 1 -- 172.21.15.45:0/2909688559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33801cc90 con 0x7fc3480c93c0 2023-10-06T05:26:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc33d7fa700 1 -- 172.21.15.45:0/2909688559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33801cdf0 con 0x7fc3480c93c0 2023-10-06T05:26:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc33d7fa700 1 -- 172.21.15.45:0/2909688559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33801a6a0 con 0x7fc3480c93c0 2023-10-06T05:26:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3480c76a0 con 0x7fc3480c93c0 2023-10-06T05:26:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.383+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3480b0cf0 con 0x7fc3480c93c0 2023-10-06T05:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.385+0000 7fc33d7fa700 1 -- 172.21.15.45:0/2909688559 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc33801aa50 con 0x7fc3480c93c0 2023-10-06T05:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.385+0000 7fc33d7fa700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc32c066f30 0x7fc32c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.385+0000 7fc33d7fa700 1 -- 172.21.15.45:0/2909688559 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc338086f50 con 0x7fc3480c93c0 2023-10-06T05:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.385+0000 7fc34d194700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc32c066f30 0x7fc32c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.386+0000 7fc34d194700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc32c066f30 0x7fc32c0693e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc3440011e0 tx=0x7fc344000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.387+0000 7fc35870cb80 1 Processor -- start 2023-10-06T05:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.387+0000 7fc35870cb80 1 -- start start 2023-10-06T05:26:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eae2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eaeaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eaeb4d0 0x563d6eaeb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc35870cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6e9b5310 con 0x563d6eac7f40 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6e9f2130 con 0x563d6eae2450 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6e9efaf0 con 0x563d6eaeb4d0 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eaeaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eaeaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56372/0 (socket says 172.21.15.45:56372) 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 -- 172.21.15.45:0/1315016042 learned_addr learned my addr 172.21.15.45:0/1315016042 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34d194700 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eae2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34d995700 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eaeb4d0 0x563d6eaeb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 -- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eaeb4d0 msgr2=0x563d6eaeb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eaeb4d0 0x563d6eaeb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 -- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 msgr2=0x563d6eae2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eae2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 -- 172.21.15.45:0/1315016042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d6e7f1790 con 0x563d6eac7f40 2023-10-06T05:26:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.388+0000 7fc34c993700 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eaeaf90 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fc338056c80 tx=0x7fc3380844d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382451bf50555cbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc3327fc700 1 -- 172.21.15.45:0/1315016042 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338056f80 con 0x563d6eac7f40 2023-10-06T05:26:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc3327fc700 1 -- 172.21.15.45:0/1315016042 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33805a1d0 con 0x563d6eac7f40 2023-10-06T05:26:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc3327fc700 1 -- 172.21.15.45:0/1315016042 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33805df20 con 0x563d6eac7f40 2023-10-06T05:26:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 -- 172.21.15.45:0/1315016042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 msgr2=0x563d6eaeaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eaeaf90 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fc338056c80 tx=0x7fc3380844d0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 -- 172.21.15.45:0/1315016042 shutdown_connections 2023-10-06T05:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eaeb4d0 0x563d6eaeb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eae2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1315016042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eaeaf90 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 -- 172.21.15.45:0/1315016042 >> 172.21.15.45:0/1315016042 conn(0x563d6eae5880 msgr2=0x563d6eae5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 -- 172.21.15.45:0/1315016042 shutdown_connections 2023-10-06T05:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.389+0000 7fc35870cb80 1 -- 172.21.15.45:0/1315016042 wait complete. 2023-10-06T05:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.390+0000 7fc35870cb80 1 -- start start 2023-10-06T05:26:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.390+0000 7fc35870cb80 1 Processor -- start 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eac8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eae2450 0x563d6eac9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eaeb4d0 0x563d6eae3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc35870cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6e9eb8e0 con 0x563d6eac7f40 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6e997980 con 0x563d6eaeb4d0 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6e997620 con 0x563d6eae2450 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc35870cb80 10 client.0 _ll_register_callbacks cb 0x563d6e7faa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc34d194700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eac8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc34d194700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eac8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56378/0 (socket says 172.21.15.45:56378) 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc34d194700 1 -- 172.21.15.45:0/1279549581 learned_addr learned my addr 172.21.15.45:0/1279549581 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc34c993700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eae2450 0x563d6eac9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.391+0000 7fc34d995700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eaeb4d0 0x563d6eae3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34d194700 1 -- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eae2450 msgr2=0x563d6eac9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34d194700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eae2450 0x563d6eac9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34d194700 1 -- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eaeb4d0 msgr2=0x563d6eae3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34d194700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eaeb4d0 0x563d6eae3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34d194700 1 -- 172.21.15.45:0/1279549581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3480b0cf0 con 0x563d6eac7f40 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34c993700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eae2450 0x563d6eac9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34d995700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eaeb4d0 0x563d6eae3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc34d194700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eac8d50 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fc344013550 tx=0x7fc344013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85dd82030bc99bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc3277fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc3277fe700 1 -- 172.21.15.45:0/1279549581 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344013e50 con 0x563d6eac7f40 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc3277fe700 1 -- 172.21.15.45:0/1279549581 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc344012590 con 0x563d6eac7f40 2023-10-06T05:26:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d6ea8fd60 con 0x563d6eac7f40 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc3277fe700 1 -- 172.21.15.45:0/1279549581 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=328) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc344012960 con 0x563d6eac7f40 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc35870cb80 10 client.10464 fetch_fsmap learned FSMap version 328 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=328,monmap=2+}) v3 -- 0x563d6e8de0a0 con 0x563d6eac7f40 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc3277fe700 1 -- 172.21.15.45:0/1279549581 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d6ea8fd60 con 0x563d6eac7f40 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc3277fe700 1 -- 172.21.15.45:0/1279549581 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 328) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc344012ac0 con 0x563d6eac7f40 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc35870cb80 10 client.10464 fetch_fsmap finished waiting for FSMap version 328 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc35870cb80 10 client.10464 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc35870cb80 20 client.10464 populate_metadata read hostname 'smithi045' 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.392+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eae47f0 0x563d6eac6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.393+0000 7fc35870cb80 4 client.10464 mds_command: new command op to 8447 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.393+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 --> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d6ea6cb30 con 0x563d6eae47f0 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.393+0000 7fc34c993700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eae47f0 0x563d6eac6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.393+0000 7fc34c993700 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eae47f0 0x563d6eac6f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea3b90ca308999f server_cookie=eaf54f6bf290fc5d in_seq=0 out_seq=0 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.393+0000 7fc3277fe700 10 client.10464 ms_handle_connect on v2:172.21.15.144:6834/603338967 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc3277fe700 1 -- 172.21.15.45:0/1279549581 <== mds.0 v2:172.21.15.144:6834/603338967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563d6ea6cb30 con 0x563d6eae47f0 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc3277fe700 10 client.10464 handle_command_reply: tid=0 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 client.10464 shutdown 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 msgr2=0x563d6eac8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eac8d50 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fc344013550 tx=0x7fc344013000 comp rx=0 tx=0).stop 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 shutdown_connections 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eae47f0 0x563d6eac6f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eaeb4d0 0x563d6eae3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eac7f40 0x563d6eac8d50 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1279549581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eae2450 0x563d6eac9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc3277fe700 0 client.10464 ms_handle_reset on v2:172.21.15.144:6834/603338967 2023-10-06T05:26:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 >> 172.21.15.45:0/1279549581 conn(0x563d6eae5880 msgr2=0x563d6ea97c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 shutdown_connections 2023-10-06T05:26:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.394+0000 7fc35870cb80 1 -- 172.21.15.45:0/1279549581 wait complete. 2023-10-06T05:26:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.395+0000 7fc35870cb80 20 client.10464 trim_cache size 0 max 16384 2023-10-06T05:26:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.412+0000 7fc35870cb80 1 Processor -- start 2023-10-06T05:26:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.412+0000 7fc35870cb80 1 -- start start 2023-10-06T05:26:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae5880 0x563d6eae1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eac6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eac73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc35870cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6e8831e0 con 0x563d6eae5880 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6eae3ce0 con 0x563d6eae2450 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6ea99b00 con 0x563d6eac7f40 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae5880 0x563d6eae1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae5880 0x563d6eae1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56390/0 (socket says 172.21.15.45:56390) 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 -- 172.21.15.45:0/272274111 learned_addr learned my addr 172.21.15.45:0/272274111 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34c993700 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eac6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d995700 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eac73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 -- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 msgr2=0x563d6eac6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eac6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 -- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 msgr2=0x563d6eac73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eac73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 -- 172.21.15.45:0/272274111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d6e9ba3c0 con 0x563d6eae5880 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.413+0000 7fc34d194700 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae5880 0x563d6eae1870 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e570 tx=0x7fc344015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9122c520beef3ef5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc332ffd700 1 -- 172.21.15.45:0/272274111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344012590 con 0x563d6eae5880 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc332ffd700 1 -- 172.21.15.45:0/272274111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc344012960 con 0x563d6eae5880 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc332ffd700 1 -- 172.21.15.45:0/272274111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344030480 con 0x563d6eae5880 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 -- 172.21.15.45:0/272274111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae5880 msgr2=0x563d6eae1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae5880 0x563d6eae1870 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e570 tx=0x7fc344015070 comp rx=0 tx=0).stop 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 -- 172.21.15.45:0/272274111 shutdown_connections 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae2450 0x563d6eac73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae5880 0x563d6eae1870 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 --2- 172.21.15.45:0/272274111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eac6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 -- 172.21.15.45:0/272274111 >> 172.21.15.45:0/272274111 conn(0x563d6ea92120 msgr2=0x563d6ea97c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 -- 172.21.15.45:0/272274111 shutdown_connections 2023-10-06T05:26:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.414+0000 7fc35870cb80 1 -- 172.21.15.45:0/272274111 wait complete. 2023-10-06T05:26:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.415+0000 7fc35870cb80 1 -- start start 2023-10-06T05:26:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.415+0000 7fc35870cb80 1 Processor -- start 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eb49ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae2450 0x563d6eb506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc35870cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae5880 0x563d6eac8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc35870cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6eac8ff0 con 0x563d6eae2450 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6eb50250 con 0x563d6eae5880 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc35870cb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6ea9d260 con 0x563d6eac7f40 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc35870cb80 10 client.0 _ll_register_callbacks cb 0x7fc34400e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae2450 0x563d6eb506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae2450 0x563d6eb506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56398/0 (socket says 172.21.15.45:56398) 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 -- 172.21.15.45:0/1265930083 learned_addr learned my addr 172.21.15.45:0/1265930083 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34d194700 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eb49ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34d995700 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae5880 0x563d6eac8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 -- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 msgr2=0x563d6eb49ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eb49ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 -- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae5880 msgr2=0x563d6eac8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae5880 0x563d6eac8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.416+0000 7fc34c993700 1 -- 172.21.15.45:0/1265930083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d6e8de0a0 con 0x563d6eae2450 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc34c993700 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae2450 0x563d6eb506e0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fc338056f10 tx=0x7fc33801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6326bb11faedf727 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc3277fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:26:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc3277fe700 1 -- 172.21.15.45:0/1265930083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338056f80 con 0x563d6eae2450 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc3277fe700 1 -- 172.21.15.45:0/1265930083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc33805df20 con 0x563d6eae2450 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc33805a1d0 con 0x563d6eae2450 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc3277fe700 1 -- 172.21.15.45:0/1265930083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=328) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc338053690 con 0x563d6eae2450 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 10 client.10470 fetch_fsmap learned FSMap version 328 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=328,monmap=2+}) v3 -- 0x563d6e8521d0 con 0x563d6eae2450 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc3277fe700 1 -- 172.21.15.45:0/1265930083 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33805a1d0 con 0x563d6eae2450 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc3277fe700 1 -- 172.21.15.45:0/1265930083 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 328) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc33805c0e0 con 0x563d6eae2450 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 10 client.10470 fetch_fsmap finished waiting for FSMap version 328 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 10 client.10470 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 20 client.10470 populate_metadata read hostname 'smithi045' 2023-10-06T05:26:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eaeb4d0 0x563d6eac6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 4 client.10470 mds_command: new command op to 8447 tid=0[{"prefix": "status"}] 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.417+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 --> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d6e98e8d0 con 0x563d6eaeb4d0 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc34d194700 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eaeb4d0 0x563d6eac6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc3277fe700 10 client.10470 ms_handle_connect on v2:172.21.15.144:6834/603338967 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc34d194700 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eaeb4d0 0x563d6eac6ea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bad7a5605c01f5f server_cookie=86fc3cc71805bc87 in_seq=0 out_seq=0 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc3277fe700 1 -- 172.21.15.45:0/1265930083 <== mds.0 v2:172.21.15.144:6834/603338967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x563d6e98e8d0 con 0x563d6eaeb4d0 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc3277fe700 10 client.10470 handle_command_reply: tid=0 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 client.10470 shutdown 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae2450 msgr2=0x563d6eb506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae2450 0x563d6eb506e0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fc338056f10 tx=0x7fc33801f040 comp rx=0 tx=0).stop 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 shutdown_connections 2023-10-06T05:26:36.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:6834/603338967,v1:172.21.15.144:6835/603338967] conn(0x563d6eaeb4d0 0x563d6eac6ea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d6eae5880 0x563d6eac8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d6eac7f40 0x563d6eb49ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc3277fe700 0 client.10470 ms_handle_reset on v2:172.21.15.144:6834/603338967 2023-10-06T05:26:36.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 --2- 172.21.15.45:0/1265930083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6eae2450 0x563d6eb506e0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 >> 172.21.15.45:0/1265930083 conn(0x563d6ea92120 msgr2=0x563d6ea9e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.418+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 shutdown_connections 2023-10-06T05:26:36.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc35870cb80 1 -- 172.21.15.45:0/1265930083 wait complete. 2023-10-06T05:26:36.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc35870cb80 20 client.10470 trim_cache size 0 max 16384 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8447, 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:26:36.432 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.83253062, 2023-10-06T05:26:36.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 328, 2023-10-06T05:26:36.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 128, 2023-10-06T05:26:36.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 128, 2023-10-06T05:26:36.433 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 13.841122342 2023-10-06T05:26:36.433 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc32c066f30 msgr2=0x7fc32c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc32c066f30 0x7fc32c0693e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc3440011e0 tx=0x7fc344000bc0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480c93c0 msgr2=0x7fc348069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480c93c0 0x7fc348069e00 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc3380091e0 tx=0x7fc338016000 comp rx=0 tx=0).stop 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 shutdown_connections 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc32c066f30 0x7fc32c0693e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc348157030 0x7fc34806a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3480c8ff0 0x7fc348148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 --2- 172.21.15.45:0/2909688559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3480c93c0 0x7fc348069e00 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 >> 172.21.15.45:0/2909688559 conn(0x7fc3480cdf50 msgr2=0x7fc34814a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 shutdown_connections 2023-10-06T05:26:36.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.419+0000 7fc34f3f8700 1 -- 172.21.15.45:0/2909688559 wait complete. 2023-10-06T05:26:36.436 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:26:36.436 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2023-10-06T05:26:36.436 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:26:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.565+0000 7f0b87149700 1 Processor -- start 2023-10-06T05:26:36.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.565+0000 7f0b87149700 1 -- start start 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b87149700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b80159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b87149700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b8015d040 0x7f0b80068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b87149700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b800ca6d0 con 0x7f0b80069b90 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b87149700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8015d550 con 0x7f0b8015d040 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b87149700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8015d810 con 0x7f0b80159170 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b7ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b8015d040 0x7f0b80068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b856e6700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0b80069b90 0x7f0b800aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51514/0 (socket says 172.21.15.45:51514) 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b856e6700 1 -- 172.21.15.45:0/1468148530 learned_addr learned my addr 172.21.15.45:0/1468148530 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.566+0000 7f0b84ee5700 1 --2- 172.21.15.45:0/1468148530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b80159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7ffff700 1 -- 172.21.15.45:0/1468148530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 msgr2=0x7f0b80159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7ffff700 1 --2- 172.21.15.45:0/1468148530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b80159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7ffff700 1 -- 172.21.15.45:0/1468148530 >> v1:172.21.15.45:6789/0 conn(0x7f0b80069b90 legacy=0x7f0b800aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7ffff700 1 -- 172.21.15.45:0/1468148530 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8015daf0 con 0x7f0b8015d040 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7f7fe700 1 -- 172.21.15.45:0/1468148530 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190665751 0 0) 0x7f0b800ca6d0 con 0x7f0b80069b90 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7ffff700 1 --2- 172.21.15.45:0/1468148530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b80068bc0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f0b680098e0 tx=0x7f0b680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c307d61b62f2bf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7f7fe700 1 -- 172.21.15.45:0/1468148530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800b640 con 0x7f0b8015d040 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7f7fe700 1 -- 172.21.15.45:0/1468148530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800b7a0 con 0x7f0b8015d040 2023-10-06T05:26:36.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.567+0000 7f0b7f7fe700 1 -- 172.21.15.45:0/1468148530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800bb40 con 0x7f0b8015d040 2023-10-06T05:26:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 -- 172.21.15.45:0/1468148530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 msgr2=0x7f0b80068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 --2- 172.21.15.45:0/1468148530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b80068bc0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f0b680098e0 tx=0x7f0b680091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 -- 172.21.15.45:0/1468148530 shutdown_connections 2023-10-06T05:26:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 --2- 172.21.15.45:0/1468148530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b80159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 --2- 172.21.15.45:0/1468148530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b80068bc0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 -- 172.21.15.45:0/1468148530 >> 172.21.15.45:0/1468148530 conn(0x7f0b8006f7c0 msgr2=0x7f0b800d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 -- 172.21.15.45:0/1468148530 shutdown_connections 2023-10-06T05:26:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.568+0000 7f0b87149700 1 -- 172.21.15.45:0/1468148530 wait complete. 2023-10-06T05:26:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.569+0000 7f0b87149700 1 Processor -- start 2023-10-06T05:26:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.569+0000 7f0b87149700 1 -- start start 2023-10-06T05:26:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b87149700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b80069b90 0x7f0b800c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b87149700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b8015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b87149700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b8016cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b87149700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8015ad20 con 0x7f0b8015d040 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b87149700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8015aa50 con 0x7f0b80069b90 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b87149700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800c7270 con 0x7f0b80159170 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b856e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b8016cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b856e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b8016cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56408/0 (socket says 172.21.15.45:56408) 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b856e6700 1 -- 172.21.15.45:0/3140351015 learned_addr learned my addr 172.21.15.45:0/3140351015 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b84ee5700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b80069b90 0x7f0b800c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b7ffff700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b8015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b856e6700 1 -- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 msgr2=0x7f0b8015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.570+0000 7f0b856e6700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b8015a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b856e6700 1 -- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b80069b90 msgr2=0x7f0b800c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b856e6700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b80069b90 0x7f0b800c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b856e6700 1 -- 172.21.15.45:0/3140351015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68009420 con 0x7f0b8015d040 2023-10-06T05:26:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b856e6700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b8016cb40 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800cc90 tx=0x7f0b78001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b7cff9700 1 -- 172.21.15.45:0/3140351015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800a8b0 con 0x7f0b8015d040 2023-10-06T05:26:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b7cff9700 1 -- 172.21.15.45:0/3140351015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7800aa10 con 0x7f0b8015d040 2023-10-06T05:26:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b800c7550 con 0x7f0b8015d040 2023-10-06T05:26:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b7cff9700 1 -- 172.21.15.45:0/3140351015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800add0 con 0x7f0b8015d040 2023-10-06T05:26:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.571+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b800c7b10 con 0x7f0b8015d040 2023-10-06T05:26:36.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.572+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b80063b50 con 0x7f0b8015d040 2023-10-06T05:26:36.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.572+0000 7f0b7cff9700 1 -- 172.21.15.45:0/3140351015 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b78007300 con 0x7f0b8015d040 2023-10-06T05:26:36.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.573+0000 7f0b7cff9700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b60066f30 0x7f0b600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.573+0000 7f0b7cff9700 1 -- 172.21.15.45:0/3140351015 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0b78085710 con 0x7f0b8015d040 2023-10-06T05:26:36.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.573+0000 7f0b84ee5700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b60066f30 0x7f0b600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.574+0000 7f0b84ee5700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b60066f30 0x7f0b600693e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009db0 tx=0x7f0b700091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.577+0000 7f0b7cff9700 1 -- 172.21.15.45:0/3140351015 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b78051b00 con 0x7f0b8015d040 2023-10-06T05:26:36.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.827+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b8015ddd0 con 0x7f0b8015d040 2023-10-06T05:26:36.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.828+0000 7f0b7cff9700 1 -- 172.21.15.45:0/3140351015 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v328) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0b78055120 con 0x7f0b8015d040 2023-10-06T05:26:36.831 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:26:36.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.829+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b60066f30 msgr2=0x7f0b600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.829+0000 7f0b87149700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b60066f30 0x7f0b600693e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009db0 tx=0x7f0b700091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.829+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 msgr2=0x7f0b8016cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.829+0000 7f0b87149700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b8016cb40 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800cc90 tx=0x7f0b78001a80 comp rx=0 tx=0).stop 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 shutdown_connections 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0b60066f30 0x7f0b600693e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b8015d040 0x7f0b8016cb40 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b80069b90 0x7f0b800c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 --2- 172.21.15.45:0/3140351015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b80159170 0x7f0b8015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 >> 172.21.15.45:0/3140351015 conn(0x7f0b8006f7c0 msgr2=0x7f0b800cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 shutdown_connections 2023-10-06T05:26:36.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:36.830+0000 7f0b87149700 1 -- 172.21.15.45:0/3140351015 wait complete. 2023-10-06T05:26:36.846 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:26:36.846 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:26:36.847 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:26:36.847 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:26:36.847 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:36.847 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:26:36.847 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:26:36.847 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:26:36.847 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:36.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:36.847 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:26:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:26:36.869 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:26:36.869 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:26:36.869 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:36.869 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:26:36.869 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:36.869 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41744sec preferred_lft 41744sec 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:26:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:26:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:26:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:26:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:36.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:36.872 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:26:36.944 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:26:36.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:36.945 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:26:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:26:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:26:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:26:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:26:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:26:37.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:37.013 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:26:37.013 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:26:37.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:26:37.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:26:37.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:26:37.013 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:26:37.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:37.143 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:26:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:26:37.167 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:26:37.167 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:37.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:37.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:26:37.247 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:26:37.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:37.247 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:37.313 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:37.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:37.313 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:37.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:26:37.455 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:26:37.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:26:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:26:37.532 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:26:37.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:37.533 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:26:37.596 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:26:37.596 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:26:37.640 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:26:37.675 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:26:37.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:26:37.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:26:37.738+0000 7f4bfc7e8540 -1 init, newargv = 0x558b5c2ab530 newargc=15 2023-10-06T05:26:37.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[179379]: starting ceph client 2023-10-06T05:26:37.746 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[179379]: starting fuse 2023-10-06T05:26:37.747 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:26:37.749 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:26:37.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:37.750 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:26:37.811 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:26:37.812 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:26:37.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:26:37.812 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:26:37.812 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:26:37.812 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:26:37.812 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:26:37.812 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:37.812 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:37.813 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:26:37.814 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:26:37.951 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.179379.asok 2023-10-06T05:26:38.132 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.179379.asok 2023-10-06T05:26:38.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:38.133 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.179379.asok status 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "179379", 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:26:38.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 10488, 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 10488 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:26:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2886073771 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2886073771 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.10488 192.168.0.1:0/2886073771", 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2886073771", 2023-10-06T05:26:38.310 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:26:38.311 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 328, 2023-10-06T05:26:38.311 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 128, 2023-10-06T05:26:38.311 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:26:38.311 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:26:38.311 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:26:38.311 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:26:38.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:38.319 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:26:38.319 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:26:38.340 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:38.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:38.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:38.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:38.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:38.413 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2023-10-06T05:26:38.455 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:38.456 INFO:teuthology.orchestra.run:waiting for 20 2023-10-06T05:26:38.482 INFO:teuthology.orchestra.run.smithi045.stdout:.: 2023-10-06T05:26:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:subdir 2023-10-06T05:26:38.483 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:./subdir: 2023-10-06T05:26:38.483 INFO:teuthology.orchestra.run.smithi045.stdout:sixmegs 2023-10-06T05:26:38.484 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2023-10-06T05:26:38.484 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2023-10-06T05:26:38.484 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:26:38.484 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:38.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:38.578 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:26:38.578 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:26:38.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:38.578 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:38.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.822+0000 7f711a40d700 1 Processor -- start 2023-10-06T05:26:38.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.822+0000 7f711a40d700 1 -- start start 2023-10-06T05:26:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f711a40d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71140c8ff0 0x7f71140c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f711a40d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f711a40d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7114156af0 0x7f7114156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f711a40d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711415b000 con 0x7f7114156af0 2023-10-06T05:26:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f711a40d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711415b2e0 con 0x7f71140c8ff0 2023-10-06T05:26:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f711a40d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711415b5c0 con 0x7f71140caad0 2023-10-06T05:26:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7114156af0 0x7f7114156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7114156af0 0x7f7114156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56414/0 (socket says 172.21.15.45:56414) 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 -- 172.21.15.45:0/1130600776 learned_addr learned my addr 172.21.15.45:0/1130600776 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71137fe700 1 --2- 172.21.15.45:0/1130600776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f7113fff700 1 --2- 172.21.15.45:0/1130600776 >> v2:172.21.15.144:3300/0 conn(0x7f71140c8ff0 0x7f71140c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 -- 172.21.15.45:0/1130600776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 msgr2=0x7f71140caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 --2- 172.21.15.45:0/1130600776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 -- 172.21.15.45:0/1130600776 >> v2:172.21.15.144:3300/0 conn(0x7f71140c8ff0 msgr2=0x7f71140c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 --2- 172.21.15.45:0/1130600776 >> v2:172.21.15.144:3300/0 conn(0x7f71140c8ff0 0x7f71140c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 -- 172.21.15.45:0/1130600776 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711415b8a0 con 0x7f7114156af0 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f7113fff700 1 --2- 172.21.15.45:0/1130600776 >> v2:172.21.15.144:3300/0 conn(0x7f71140c8ff0 0x7f71140c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f71189aa700 1 --2- 172.21.15.45:0/1130600776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7114156af0 0x7f7114156f70 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f71140cefc0 tx=0x7f710c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d609a3574d6269d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.823+0000 7f7112ffd700 1 -- 172.21.15.45:0/1130600776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c00e070 con 0x7f7114156af0 2023-10-06T05:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f7112ffd700 1 -- 172.21.15.45:0/1130600776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c00ba20 con 0x7f7114156af0 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f7112ffd700 1 -- 172.21.15.45:0/1130600776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c0092e0 con 0x7f7114156af0 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 -- 172.21.15.45:0/1130600776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7114156af0 msgr2=0x7f7114156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 --2- 172.21.15.45:0/1130600776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7114156af0 0x7f7114156f70 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f71140cefc0 tx=0x7f710c009840 comp rx=0 tx=0).stop 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 -- 172.21.15.45:0/1130600776 shutdown_connections 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 --2- 172.21.15.45:0/1130600776 >> v2:172.21.15.144:3300/0 conn(0x7f71140c8ff0 0x7f71140c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 --2- 172.21.15.45:0/1130600776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7114156af0 0x7f7114156f70 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 --2- 172.21.15.45:0/1130600776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 -- 172.21.15.45:0/1130600776 >> 172.21.15.45:0/1130600776 conn(0x7f71140cdf50 msgr2=0x7f71140ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 -- 172.21.15.45:0/1130600776 shutdown_connections 2023-10-06T05:26:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.824+0000 7f711a40d700 1 -- 172.21.15.45:0/1130600776 wait complete. 2023-10-06T05:26:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.825+0000 7f711a40d700 1 Processor -- start 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.826+0000 7f711a40d700 1 -- start start 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.826+0000 7f711a40d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71140c8ff0 0x7f711416d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.826+0000 7f711a40d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.826+0000 7f711a40d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7114156af0 0x7f71140c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f711a40d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140b1cf0 con 0x7f71140c8ff0 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f711a40d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140ad6b0 con 0x7f7114156af0 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f711a40d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71140ad3d0 con 0x7f71140caad0 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71140c8ff0 0x7f711416d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71140c8ff0 0x7f711416d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56430/0 (socket says 172.21.15.45:56430) 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 -- 172.21.15.45:0/2404324133 learned_addr learned my addr 172.21.15.45:0/2404324133 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f71189aa700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7114156af0 0x7f71140c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f71137fe700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 -- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 msgr2=0x7f71140c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 -- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7114156af0 msgr2=0x7f71140c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7114156af0 0x7f71140c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 -- 172.21.15.45:0/2404324133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c009ae0 con 0x7f71140c8ff0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f7113fff700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71140c8ff0 0x7f711416d230 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f710400d860 tx=0x7f7104008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f70fbfff700 1 -- 172.21.15.45:0/2404324133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040125f0 con 0x7f71140c8ff0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f70fbfff700 1 -- 172.21.15.45:0/2404324133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7104012750 con 0x7f71140c8ff0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71140b1730 con 0x7f71140c8ff0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f70fbfff700 1 -- 172.21.15.45:0/2404324133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104012af0 con 0x7f71140c8ff0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.827+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711416c8d0 con 0x7f71140c8ff0 2023-10-06T05:26:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.828+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711406f410 con 0x7f71140c8ff0 2023-10-06T05:26:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.829+0000 7f70fbfff700 1 -- 172.21.15.45:0/2404324133 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7104013480 con 0x7f71140c8ff0 2023-10-06T05:26:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.829+0000 7f70fbfff700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7100066f30 0x7f71000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.830+0000 7f70fbfff700 1 -- 172.21.15.45:0/2404324133 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7104085490 con 0x7f71140c8ff0 2023-10-06T05:26:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.830+0000 7f71137fe700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7100066f30 0x7f71000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.830+0000 7f71137fe700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7100066f30 0x7f71000693e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f711416c5e0 tx=0x7f71080091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:38.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:38.834+0000 7f70fbfff700 1 -- 172.21.15.45:0/2404324133 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71040518a0 con 0x7f71140c8ff0 2023-10-06T05:26:39.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.087+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f711404bb40 con 0x7f71140c8ff0 2023-10-06T05:26:39.200 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:39.087+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:39.200 INFO:tasks.ceph.mds.b.smithi144.stderr: -1601> 2023-10-06T05:26:29.591+0000 7fe648332700 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer 2023-10-06T05:26:39.200 INFO:tasks.ceph.mds.b.smithi144.stderr: -635> 2023-10-06T05:26:33.561+0000 7fe64bb39700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:39.201 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:26:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.091+0000 7f70fbfff700 1 -- 172.21.15.45:0/2404324133 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v329) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7104054ec0 con 0x7f71140c8ff0 2023-10-06T05:26:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7100066f30 msgr2=0x7f71000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7100066f30 0x7f71000693e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f711416c5e0 tx=0x7f71080091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71140c8ff0 msgr2=0x7f711416d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71140c8ff0 0x7f711416d230 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f710400d860 tx=0x7f7104008040 comp rx=0 tx=0).stop 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 shutdown_connections 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7100066f30 0x7f71000693e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7114156af0 0x7f71140c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71140c8ff0 0x7f711416d230 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 --2- 172.21.15.45:0/2404324133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71140caad0 0x7f71140c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 >> 172.21.15.45:0/2404324133 conn(0x7f71140cdf50 msgr2=0x7f71140ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 shutdown_connections 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.094+0000 7f711a40d700 1 -- 172.21.15.45:0/2404324133 wait complete. 2023-10-06T05:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:39.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:26:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.340+0000 7fe4662ae700 1 Processor -- start 2023-10-06T05:26:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.340+0000 7fe4662ae700 1 -- start start 2023-10-06T05:26:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe4662ae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe460159330 0x7fe46006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe4662ae700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4600cc390 0x7fe4600cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe4662ae700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4600cb530 con 0x7fe460158f60 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe4662ae700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe460069bd0 con 0x7fe4600cc390 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe4662ae700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4600ccd10 con 0x7fe460159330 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe46484b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4600cc390 0x7fe4600cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe46484b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4600cc390 0x7fe4600cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56434/0 (socket says 172.21.15.45:56434) 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe46484b700 1 -- 172.21.15.45:0/1647287559 learned_addr learned my addr 172.21.15.45:0/1647287559 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe45f7fe700 1 --2- 172.21.15.45:0/1647287559 >> v2:172.21.15.144:3300/0 conn(0x7fe460159330 0x7fe46006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe46484b700 1 -- 172.21.15.45:0/1647287559 >> v2:172.21.15.144:3300/0 conn(0x7fe460159330 msgr2=0x7fe46006a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe46484b700 1 --2- 172.21.15.45:0/1647287559 >> v2:172.21.15.144:3300/0 conn(0x7fe460159330 0x7fe46006a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe46484b700 1 -- 172.21.15.45:0/1647287559 >> v1:172.21.15.144:6789/0 conn(0x7fe460158f60 legacy=0x7fe4600cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.341+0000 7fe46484b700 1 -- 172.21.15.45:0/1647287559 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460068fc0 con 0x7fe4600cc390 2023-10-06T05:26:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe46484b700 1 --2- 172.21.15.45:0/1647287559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe4600cc7d0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe4600872e0 tx=0x7fe458009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9e7a618a72114c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe45effd700 1 -- 172.21.15.45:0/1647287559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45800e070 con 0x7fe4600cc390 2023-10-06T05:26:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe45effd700 1 -- 172.21.15.45:0/1647287559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45800ba10 con 0x7fe4600cc390 2023-10-06T05:26:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe45effd700 1 -- 172.21.15.45:0/1647287559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4580092e0 con 0x7fe4600cc390 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 -- 172.21.15.45:0/1647287559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 msgr2=0x7fe4600cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 --2- 172.21.15.45:0/1647287559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe4600cc7d0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe4600872e0 tx=0x7fe458009840 comp rx=0 tx=0).stop 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 -- 172.21.15.45:0/1647287559 shutdown_connections 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 --2- 172.21.15.45:0/1647287559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe4600cc7d0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 --2- 172.21.15.45:0/1647287559 >> v2:172.21.15.144:3300/0 conn(0x7fe460159330 0x7fe46006a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 -- 172.21.15.45:0/1647287559 >> 172.21.15.45:0/1647287559 conn(0x7fe4600cdf50 msgr2=0x7fe4600ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 -- 172.21.15.45:0/1647287559 shutdown_connections 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.342+0000 7fe4662ae700 1 -- 172.21.15.45:0/1647287559 wait complete. 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.344+0000 7fe4662ae700 1 Processor -- start 2023-10-06T05:26:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.344+0000 7fe4662ae700 1 -- start start 2023-10-06T05:26:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe4662ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe46014a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe4662ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe460158f60 0x7fe4600b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe4662ae700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe460159330 0x7fe4600acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe4662ae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600a9930 con 0x7fe4600cc390 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe4662ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600a8d20 con 0x7fe460158f60 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe4662ae700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600a9000 con 0x7fe460159330 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe46014a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe46014a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56446/0 (socket says 172.21.15.45:56446) 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 -- 172.21.15.45:0/2550330750 learned_addr learned my addr 172.21.15.45:0/2550330750 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45f7fe700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe460158f60 0x7fe4600b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe46484b700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe460159330 0x7fe4600acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 -- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe460159330 msgr2=0x7fe4600acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe460159330 0x7fe4600acbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 -- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe460158f60 msgr2=0x7fe4600b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe460158f60 0x7fe4600b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 -- 172.21.15.45:0/2550330750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458009ae0 con 0x7fe4600cc390 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe46484b700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe460159330 0x7fe4600acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe45ffff700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe46014a090 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c5d0 tx=0x7fe450007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe457fff700 1 -- 172.21.15.45:0/2550330750 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450011590 con 0x7fe4600cc390 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe457fff700 1 -- 172.21.15.45:0/2550330750 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450011710 con 0x7fe4600cc390 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.345+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4600a92e0 con 0x7fe4600cc390 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.346+0000 7fe457fff700 1 -- 172.21.15.45:0/2550330750 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450011ad0 con 0x7fe4600cc390 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.346+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4600a9dc0 con 0x7fe4600cc390 2023-10-06T05:26:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.347+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4600a9c10 con 0x7fe4600cc390 2023-10-06T05:26:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.347+0000 7fe457fff700 1 -- 172.21.15.45:0/2550330750 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe45001c460 con 0x7fe4600cc390 2023-10-06T05:26:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.347+0000 7fe457fff700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe440066f30 0x7fe4400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.348+0000 7fe457fff700 1 -- 172.21.15.45:0/2550330750 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9560+0+0 (secure 0 0 0) 0x7fe450085280 con 0x7fe4600cc390 2023-10-06T05:26:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.348+0000 7fe45f7fe700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe440066f30 0x7fe4400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.348+0000 7fe45f7fe700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe440066f30 0x7fe4400693e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe460158980 tx=0x7fe448008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.352+0000 7fe457fff700 1 -- 172.21.15.45:0/2550330750 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4500515a0 con 0x7fe4600cc390 2023-10-06T05:26:39.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:39.643+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fe46004bb40 con 0x7fe4600cc390 2023-10-06T05:26:40.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.092+0000 7fe457fff700 1 -- 172.21.15.45:0/2550330750 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v330) v1 ==== 107+0+0 (secure 0 0 0) 0x7fe450054bc0 con 0x7fe4600cc390 2023-10-06T05:26:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe440066f30 msgr2=0x7fe4400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe440066f30 0x7fe4400693e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe460158980 tx=0x7fe448008040 comp rx=0 tx=0).stop 2023-10-06T05:26:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 msgr2=0x7fe46014a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe46014a090 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c5d0 tx=0x7fe450007f60 comp rx=0 tx=0).stop 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 shutdown_connections 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe440066f30 0x7fe4400693e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe460159330 0x7fe4600acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4600cc390 0x7fe46014a090 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 --2- 172.21.15.45:0/2550330750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe460158f60 0x7fe4600b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 >> 172.21.15.45:0/2550330750 conn(0x7fe4600cdf50 msgr2=0x7fe4600c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 shutdown_connections 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.095+0000 7fe4662ae700 1 -- 172.21.15.45:0/2550330750 wait complete. 2023-10-06T05:26:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:nothing to do: rank is not damaged 2023-10-06T05:26:40.114 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:26:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.159+0000 7fef5f049040 1 Processor -- start 2023-10-06T05:26:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.159+0000 7fef5f049040 1 -- start start 2023-10-06T05:26:40.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef5f049040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c373579210 0x55c3735c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef5f049040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c37363b890 0x55c3735c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef5f049040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363bc60 0x55c37365ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef5f049040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3734c6980 con 0x55c373579210 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef5f049040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3734c6620 con 0x55c37363b890 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef5f049040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3734bd400 con 0x55c37363bc60 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c373579210 0x55c3735c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c373579210 0x55c3735c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56458/0 (socket says 172.21.15.45:56458) 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 -- 172.21.15.45:0/2465725524 learned_addr learned my addr 172.21.15.45:0/2465725524 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef57a66700 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c37363b890 0x55c3735c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58a68700 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363bc60 0x55c37365ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 -- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363bc60 msgr2=0x55c37365ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363bc60 0x55c37365ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 -- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c37363b890 msgr2=0x55c3735c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c37363b890 0x55c3735c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.160+0000 7fef58267700 1 -- 172.21.15.45:0/2465725524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40005cf0 con 0x55c373579210 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef58267700 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c373579210 0x55c3735c0140 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fef40009aa0 tx=0x7fef4000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4091b56239f0dd28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef56a64700 1 -- 172.21.15.45:0/2465725524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000f040 con 0x55c373579210 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef56a64700 1 -- 172.21.15.45:0/2465725524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef40008af0 con 0x55c373579210 2023-10-06T05:26:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef56a64700 1 -- 172.21.15.45:0/2465725524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40008e90 con 0x55c373579210 2023-10-06T05:26:40.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 -- 172.21.15.45:0/2465725524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c373579210 msgr2=0x55c3735c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c373579210 0x55c3735c0140 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fef40009aa0 tx=0x7fef4000c040 comp rx=0 tx=0).stop 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 -- 172.21.15.45:0/2465725524 shutdown_connections 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363bc60 0x55c37365ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c373579210 0x55c3735c0140 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 --2- 172.21.15.45:0/2465725524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c37363b890 0x55c3735c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 -- 172.21.15.45:0/2465725524 >> 172.21.15.45:0/2465725524 conn(0x55c37357c450 msgr2=0x55c37357e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.161+0000 7fef5f049040 1 -- 172.21.15.45:0/2465725524 shutdown_connections 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.162+0000 7fef5f049040 1 -- 172.21.15.45:0/2465725524 wait complete. 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.162+0000 7fef5f049040 1 Processor -- start 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.162+0000 7fef5f049040 1 -- start start 2023-10-06T05:26:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef5f049040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c373579210 0x55c373653d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef5f049040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363b890 0x55c37365db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef5f049040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c37363bc60 0x55c37365e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef5f049040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3735aab70 con 0x55c37363bc60 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef5f049040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c373655130 con 0x55c373579210 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef5f049040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c373655390 con 0x55c37363b890 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c37363bc60 0x55c37365e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c37363bc60 0x55c37365e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56468/0 (socket says 172.21.15.45:56468) 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 learned_addr learned my addr 172.21.15.45:0/4116408295 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef57a66700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363b890 0x55c37365db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58267700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c373579210 0x55c373653d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363b890 msgr2=0x55c37365db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c37363b890 0x55c37365db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c373579210 msgr2=0x55c373653d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c373579210 0x55c373653d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.163+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40009670 con 0x55c37363bc60 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.164+0000 7fef58a68700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c37363bc60 0x55c37365e050 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fef50003f00 tx=0x7fef500068b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.164+0000 7fef55261700 1 -- 172.21.15.45:0/4116408295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50002080 con 0x55c37363bc60 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.164+0000 7fef55261700 1 -- 172.21.15.45:0/4116408295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef500123b0 con 0x55c37363bc60 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.164+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef48005db0 con 0x55c37363bc60 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.164+0000 7fef55261700 1 -- 172.21.15.45:0/4116408295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50012750 con 0x55c37363bc60 2023-10-06T05:26:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.164+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3734d53d0 con 0x55c37363bc60 2023-10-06T05:26:40.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.165+0000 7fef55261700 1 -- 172.21.15.45:0/4116408295 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef50012b00 con 0x55c37363bc60 2023-10-06T05:26:40.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.166+0000 7fef55261700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef38065890 0x7fef38067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.166+0000 7fef55261700 1 -- 172.21.15.45:0/4116408295 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9560+0+0 (secure 0 0 0) 0x7fef50083ae0 con 0x55c37363bc60 2023-10-06T05:26:40.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.166+0000 7fef58267700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef38065890 0x7fef38067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.170 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:26:40.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.166+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c3735aa5f0 0x55c3736382a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.166+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3735c1130 con 0x55c3735aa5f0 2023-10-06T05:26:40.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.166+0000 7fef57a66700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c3735aa5f0 0x55c3736382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.167+0000 7fef58267700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef38065890 0x7fef38067d40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fef40009640 tx=0x7fef40017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.167+0000 7fef57a66700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c3735aa5f0 0x55c3736382a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.169+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v129'1128 uv1102 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48000bc0 con 0x55c3735aa5f0 2023-10-06T05:26:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.169+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736387e0 con 0x55c3735aa5f0 2023-10-06T05:26:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.171+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v129'1129 uv1129 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48000bc0 con 0x55c3735aa5f0 2023-10-06T05:26:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.171+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366f1f0 con 0x55c3735aa5f0 2023-10-06T05:26:40.174 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:26:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.172+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v129'1130 uv1130 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48000bc0 con 0x55c3735aa5f0 2023-10-06T05:26:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.172+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736705b0 con 0x55c3735aa5f0 2023-10-06T05:26:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.174+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v129'1131 uv1131 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48000bc0 con 0x55c3735aa5f0 2023-10-06T05:26:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.174+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373670960 con 0x55c3735aa5f0 2023-10-06T05:26:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.175+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v129'1132 uv1132 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.175+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366f5a0 con 0x55c3735aa5f0 2023-10-06T05:26:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.176+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v129'1133 uv1133 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.176+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366f9d0 con 0x55c3735aa5f0 2023-10-06T05:26:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.178+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v129'1134 uv1108 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.178+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fdd0 con 0x55c3735aa5f0 2023-10-06T05:26:40.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.180+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v129'1135 uv1135 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.180+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fdd0 con 0x55c3735aa5f0 2023-10-06T05:26:40.182 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:26:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.181+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v129'1136 uv1136 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.181+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736701d0 con 0x55c3735aa5f0 2023-10-06T05:26:40.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.182+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v129'1137 uv1137 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.182+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.184+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v129'1138 uv1138 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.184+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.185+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v129'1139 uv1139 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.185+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.187+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v129'1140 uv1115 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.187+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.188+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v129'1141 uv1141 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.188+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.191 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:26:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.189+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v129'1142 uv1142 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.190+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.191+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v129'1143 uv1143 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.191+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.192+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v129'1144 uv1144 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.192+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fd80 con 0x55c3735aa5f0 2023-10-06T05:26:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.194+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v129'1145 uv1145 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.194+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373670180 con 0x55c3735aa5f0 2023-10-06T05:26:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.195+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v129'1146 uv1146 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.195+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.197+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v129'1147 uv1121 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.197+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.198+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v129'1148 uv1148 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.198+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.201 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:26:40.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.200+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v129'1149 uv1149 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.200+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.201+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v129'1150 uv1150 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.201+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.202+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v129'1151 uv1151 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.202+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.204+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v129'1152 uv1152 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.204+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.205+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v129'1153 uv1127 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.205+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.207+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v129'1154 uv1154 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.207+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.210 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:26:40.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.208+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v129'1155 uv1155 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.208+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.210+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v129'1156 uv1156 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.210+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.211+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v129'1157 uv1157 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.211+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37366fff0 con 0x55c3735aa5f0 2023-10-06T05:26:40.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.212+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v129'1158 uv1158 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3735aa5f0 2023-10-06T05:26:40.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.212+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c373670d10 0x55c3736730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.213+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373673630 con 0x55c373670d10 2023-10-06T05:26:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.213+0000 7fef58267700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c373670d10 0x55c3736730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.213+0000 7fef58267700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c373670d10 0x55c3736730f0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.215+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v129'814 uv794 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.215+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736739e0 con 0x55c373670d10 2023-10-06T05:26:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.217+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v129'815 uv815 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.217+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373673d90 con 0x55c373670d10 2023-10-06T05:26:40.220 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:26:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.218+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v129'816 uv816 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.218+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674630 con 0x55c373670d10 2023-10-06T05:26:40.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.220+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v129'817 uv817 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.220+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736749e0 con 0x55c373670d10 2023-10-06T05:26:40.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.221+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v129'818 uv818 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.221+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.223+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v129'819 uv813 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.223+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.224+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v129'820 uv820 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.224+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.225+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v129'821 uv821 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.227 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:26:40.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.225+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.227+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v129'822 uv822 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.227+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.228+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v129'823 uv823 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.228+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.229+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v129'824 uv812 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.229+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.231+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v129'825 uv825 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.231+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.233 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:26:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.232+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v129'826 uv826 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.232+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736784b0 con 0x55c373670d10 2023-10-06T05:26:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.234+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v129'827 uv827 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.234+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736784b0 con 0x55c373670d10 2023-10-06T05:26:40.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.235+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v129'828 uv828 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.235+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.236+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v129'829 uv810 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.236+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.237+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v129'830 uv830 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.237+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.240 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:26:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.239+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v129'831 uv831 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.239+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373675da0 con 0x55c373670d10 2023-10-06T05:26:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.240+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v129'832 uv832 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.240+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373675da0 con 0x55c373670d10 2023-10-06T05:26:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.241+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v129'833 uv833 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.241+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674d90 con 0x55c373670d10 2023-10-06T05:26:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.243+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v129'834 uv834 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.243+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c3736752a0 0x55c373677680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.243+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373677bc0 con 0x55c3736752a0 2023-10-06T05:26:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.243+0000 7fef57a66700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c3736752a0 0x55c373677680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.244+0000 7fef57a66700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c3736752a0 0x55c373677680 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.245+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.245+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373677bc0 con 0x55c3736752a0 2023-10-06T05:26:40.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.247+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v129'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.247+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373677fc0 con 0x55c3736752a0 2023-10-06T05:26:40.250 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:26:40.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.248+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v129'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.248+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373678fd0 con 0x55c3736752a0 2023-10-06T05:26:40.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.250+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.250+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373678fd0 con 0x55c3736752a0 2023-10-06T05:26:40.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.251+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.251+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373677fc0 con 0x55c3736752a0 2023-10-06T05:26:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.252+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.252+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373677fc0 con 0x55c3736752a0 2023-10-06T05:26:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.254+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v129'159 uv158 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.254+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373677fc0 con 0x55c3736752a0 2023-10-06T05:26:40.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.255+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v129'160 uv160 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.255+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373677fc0 con 0x55c3736752a0 2023-10-06T05:26:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.256+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v129'161 uv161 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.258 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:26:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.256+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736741c0 con 0x55c3736752a0 2023-10-06T05:26:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.257+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v129'162 uv162 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.257+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736741c0 con 0x55c373670d10 2023-10-06T05:26:40.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.259+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.259+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736741c0 con 0x55c373670d10 2023-10-06T05:26:40.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.260+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v129'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.260+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674140 con 0x55c373670d10 2023-10-06T05:26:40.263 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:26:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.261+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v129'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.261+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674140 con 0x55c373670d10 2023-10-06T05:26:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.262+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.262+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674140 con 0x55c373670d10 2023-10-06T05:26:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.264+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.264+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373674140 con 0x55c373670d10 2023-10-06T05:26:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.265+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c373670d10 2023-10-06T05:26:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.265+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c3736789c0 0x55c37367ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.265+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367b2e0 con 0x55c3736789c0 2023-10-06T05:26:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.265+0000 7fef58a68700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c3736789c0 0x55c37367ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.266+0000 7fef58a68700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c3736789c0 0x55c37367ada0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.267+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v129'404 uv398 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.267+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367b690 con 0x55c3736789c0 2023-10-06T05:26:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.268+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v129'405 uv405 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.268+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367ba80 con 0x55c3736789c0 2023-10-06T05:26:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.269+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v129'406 uv406 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.271 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:26:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.270+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373678100 con 0x55c3736789c0 2023-10-06T05:26:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.271+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v129'407 uv407 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.271+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367d4e0 con 0x55c3736789c0 2023-10-06T05:26:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.272+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v129'408 uv408 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.272+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367c370 con 0x55c3736789c0 2023-10-06T05:26:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.273+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v129'409 uv409 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.273+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367c770 con 0x55c3736789c0 2023-10-06T05:26:40.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.274+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v129'410 uv403 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.274+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367cb70 con 0x55c3736789c0 2023-10-06T05:26:40.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.276+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v129'411 uv411 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.276+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367cb70 con 0x55c3736789c0 2023-10-06T05:26:40.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.277+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v129'412 uv412 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.279 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:26:40.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.277+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367cb20 con 0x55c3736789c0 2023-10-06T05:26:40.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.278+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v129'413 uv413 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.278+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367cb20 con 0x55c3736789c0 2023-10-06T05:26:40.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.280+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v129'414 uv414 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.280+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c37367ddf0 0x55c3736801d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.280+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367cb20 con 0x55c37367ddf0 2023-10-06T05:26:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.280+0000 7fef58267700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c37367ddf0 0x55c3736801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.280+0000 7fef58267700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c37367ddf0 0x55c3736801d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.282+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v129'183 uv182 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c37367ddf0 2023-10-06T05:26:40.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.282+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367cb20 con 0x55c37367ddf0 2023-10-06T05:26:40.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.283+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v129'184 uv184 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c37367ddf0 2023-10-06T05:26:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.283+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c37367cb20 con 0x55c37367ddf0 2023-10-06T05:26:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.284+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v129'185 uv185 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c37367ddf0 2023-10-06T05:26:40.286 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:26:40.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.284+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736807e0 con 0x55c37367ddf0 2023-10-06T05:26:40.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.285+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v129'186 uv186 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c37367ddf0 2023-10-06T05:26:40.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.286+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736807e0 con 0x55c37367ddf0 2023-10-06T05:26:40.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.287+0000 7fef58267700 1 -- 172.21.15.45:0/4116408295 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v129'187 uv187 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef40014690 con 0x55c37367ddf0 2023-10-06T05:26:40.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.287+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736807e0 con 0x55c3736752a0 2023-10-06T05:26:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.288+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.288+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736807e0 con 0x55c3736752a0 2023-10-06T05:26:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.289+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v129'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.289+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736807e0 con 0x55c3736752a0 2023-10-06T05:26:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.291+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v129'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.292 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:26:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.291+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736817f0 con 0x55c3736752a0 2023-10-06T05:26:40.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.292+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.292+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736817f0 con 0x55c3736752a0 2023-10-06T05:26:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.293+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.293+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736807e0 con 0x55c3736752a0 2023-10-06T05:26:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.294+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.294+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c373680cf0 0x55c3736830d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.294+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683780 con 0x55c373680cf0 2023-10-06T05:26:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.295+0000 7fef57a66700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c373680cf0 0x55c3736830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.295+0000 7fef57a66700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c373680cf0 0x55c3736830d0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.296+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v129'365 uv359 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.297+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.298+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v129'366 uv366 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.298+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.300 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:26:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.299+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v129'367 uv367 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.299+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.300+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v129'368 uv368 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.300+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.301+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v129'369 uv369 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.301+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.302+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v129'370 uv364 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.302+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.304+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v129'371 uv371 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.304+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.306 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:26:40.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.305+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v129'372 uv372 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.305+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683b60 con 0x55c373680cf0 2023-10-06T05:26:40.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.306+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v129'373 uv373 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.306+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c3736868d0 con 0x55c373680cf0 2023-10-06T05:26:40.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.307+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v129'374 uv374 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c373680cf0 2023-10-06T05:26:40.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.307+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373684050 con 0x55c3736752a0 2023-10-06T05:26:40.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.308+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v129'162 uv160 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.308+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373684050 con 0x55c3736752a0 2023-10-06T05:26:40.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.309+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v129'163 uv163 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.309+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373684050 con 0x55c3736752a0 2023-10-06T05:26:40.312 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:26:40.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.311+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v129'164 uv164 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.311+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373684050 con 0x55c3736752a0 2023-10-06T05:26:40.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.312+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v129'165 uv165 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.312+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c373684560 0x55c373684950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.312+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c373684560 2023-10-06T05:26:40.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.312+0000 7fef58a68700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c373684560 0x55c373684950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.312+0000 7fef58a68700 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c373684560 0x55c373684950 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.314+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c373684560 2023-10-06T05:26:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.314+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c373684560 2023-10-06T05:26:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.315+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v129'220 uv220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c373684560 2023-10-06T05:26:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.315+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c373684560 2023-10-06T05:26:40.319 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:26:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.316+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v129'221 uv221 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c373684560 2023-10-06T05:26:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.316+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373688030 con 0x55c373684560 2023-10-06T05:26:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.317+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c373684560 2023-10-06T05:26:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.317+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373688030 con 0x55c373684560 2023-10-06T05:26:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.318+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c373684560 2023-10-06T05:26:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.318+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c373684560 2023-10-06T05:26:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.319+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c373684560 2023-10-06T05:26:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.320+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c3736789c0 2023-10-06T05:26:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.321+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v129'186 uv185 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.321+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c3736789c0 2023-10-06T05:26:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.322+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v129'187 uv187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.322+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c3736789c0 2023-10-06T05:26:40.325 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:26:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.323+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v129'188 uv188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.324+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687020 con 0x55c3736789c0 2023-10-06T05:26:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.325+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v129'189 uv189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.325+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736789c0 2023-10-06T05:26:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.326+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v129'190 uv190 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fef50051ee0 con 0x55c3736789c0 2023-10-06T05:26:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.326+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.327+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v129'455 uv446 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.328+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.329+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v129'456 uv456 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.329+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.330+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v129'457 uv457 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.332 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:26:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.330+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.331+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v129'458 uv458 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.331+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.332+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v129'459 uv459 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.332+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.334+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v129'460 uv453 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.334+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.335+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v129'461 uv461 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.335+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.337 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:26:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.336+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v129'462 uv462 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.336+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.337+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v129'463 uv463 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.337+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.338+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v129'464 uv464 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.338+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373683be0 con 0x55c3736752a0 2023-10-06T05:26:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.339+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v129'465 uv465 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.339+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c373687510 con 0x55c3736752a0 2023-10-06T05:26:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef57a66700 1 -- 172.21.15.45:0/4116408295 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v129'466 uv466 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fef48026040 con 0x55c3736752a0 2023-10-06T05:26:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c3736752a0 msgr2=0x55c373677680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c3736752a0 0x55c373677680 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c3735aa5f0 msgr2=0x55c3736382a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c3735aa5f0 0x55c3736382a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c373680cf0 msgr2=0x55c3736830d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c373680cf0 0x55c3736830d0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c373670d10 msgr2=0x55c3736730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c373670d10 0x55c3736730f0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 reap_dead start 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c373684560 msgr2=0x55c373684950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c373684560 0x55c373684950 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c37367ddf0 msgr2=0x55c3736801d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c37367ddf0 0x55c3736801d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c3736789c0 msgr2=0x55c37367ada0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.340+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c3736789c0 0x55c37367ada0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef38065890 msgr2=0x7fef38067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fef38065890 0x7fef38067d40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fef40009640 tx=0x7fef40017000 comp rx=0 tx=0).stop 2023-10-06T05:26:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c37363bc60 msgr2=0x55c37365e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 --2- 172.21.15.45:0/4116408295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c37363bc60 0x55c37365e050 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fef50003f00 tx=0x7fef500068b0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 reap_dead start 2023-10-06T05:26:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef58a68700 1 -- 172.21.15.45:0/4116408295 reap_dead start 2023-10-06T05:26:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 shutdown_connections 2023-10-06T05:26:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 >> 172.21.15.45:0/4116408295 conn(0x55c37357c450 msgr2=0x55c37357e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 shutdown_connections 2023-10-06T05:26:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.341+0000 7fef5f049040 1 -- 172.21.15.45:0/4116408295 wait complete. 2023-10-06T05:26:40.346 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:26:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.436+0000 7fac5c5ab040 1 Processor -- start 2023-10-06T05:26:40.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.436+0000 7fac5c5ab040 1 -- start start 2023-10-06T05:26:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac5c5ab040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac5c5ab040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 0x55b5704c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac5c5ab040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac5c5ab040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b570398320 con 0x55b5704397e0 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac5c5ab040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57033c980 con 0x55b5704c4f00 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac5c5ab040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b57033c620 con 0x55b5704c4b30 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56480/0 (socket says 172.21.15.45:56480) 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 -- 172.21.15.45:0/2862857744 learned_addr learned my addr 172.21.15.45:0/2862857744 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac55fca700 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac54fc8700 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 0x55b5704c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 -- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 msgr2=0x55b5704c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:40.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 0x55b5704c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 -- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 msgr2=0x55b5704c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 -- 172.21.15.45:0/2862857744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac40005cf0 con 0x55b5704397e0 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac55fca700 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac557c9700 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bbcf0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x55b5703f5b30 tx=0x7fac40015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cf862b97a5c177 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.437+0000 7fac4f7fe700 1 -- 172.21.15.45:0/2862857744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40007610 con 0x55b5704397e0 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac4f7fe700 1 -- 172.21.15.45:0/2862857744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac40002f70 con 0x55b5704397e0 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac4f7fe700 1 -- 172.21.15.45:0/2862857744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac400078e0 con 0x55b5704397e0 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 -- 172.21.15.45:0/2862857744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 msgr2=0x55b5704bbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bbcf0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x55b5703f5b30 tx=0x7fac40015000 comp rx=0 tx=0).stop 2023-10-06T05:26:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 -- 172.21.15.45:0/2862857744 shutdown_connections 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bbcf0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/2862857744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 0x55b5704c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 -- 172.21.15.45:0/2862857744 >> 172.21.15.45:0/2862857744 conn(0x55b5703f2450 msgr2=0x55b5704c0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 -- 172.21.15.45:0/2862857744 shutdown_connections 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.438+0000 7fac5c5ab040 1 -- 172.21.15.45:0/2862857744 wait complete. 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.439+0000 7fac5c5ab040 1 Processor -- start 2023-10-06T05:26:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.439+0000 7fac5c5ab040 1 -- start start 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac5c5ab040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac5c5ab040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 0x55b5704bc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac5c5ab040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac5c5ab040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b570333400 con 0x55b5704397e0 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac5c5ab040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5704d46a0 con 0x55b5704c4f00 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac5c5ab040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5704d4940 con 0x55b5704c4b30 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56494/0 (socket says 172.21.15.45:56494) 2023-10-06T05:26:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 -- 172.21.15.45:0/3692783076 learned_addr learned my addr 172.21.15.45:0/3692783076 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac55fca700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac54fc8700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 0x55b5704bc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 -- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 msgr2=0x55b5704bc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5704c4b30 0x55b5704bc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 -- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 msgr2=0x55b5704d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5704c4f00 0x55b5704d3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 -- 172.21.15.45:0/3692783076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57034b3d0 con 0x55b5704397e0 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac557c9700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bf120 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fac4001e000 tx=0x7fac4001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.440+0000 7fac4dffb700 1 -- 172.21.15.45:0/3692783076 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40007ea0 con 0x55b5704397e0 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.441+0000 7fac4dffb700 1 -- 172.21.15.45:0/3692783076 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac400078e0 con 0x55b5704397e0 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.441+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b57033eae0 con 0x55b5704397e0 2023-10-06T05:26:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.441+0000 7fac4dffb700 1 -- 172.21.15.45:0/3692783076 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40007610 con 0x55b5704397e0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.441+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b57033e540 con 0x55b5704397e0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.442+0000 7fac4dffb700 1 -- 172.21.15.45:0/3692783076 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac40019070 con 0x55b5704397e0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.442+0000 7fac4dffb700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac38067aa0 0x7fac38069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.443+0000 7fac4dffb700 1 -- 172.21.15.45:0/3692783076 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9560+0+0 (secure 0 0 0) 0x7fac40086730 con 0x55b5704397e0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.443+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b5704b96b0 0x55b5704dfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.443+0000 7fac54fc8700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac38067aa0 0x7fac38069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.443+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55b5704e0110 con 0x55b5704b96b0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.443+0000 7fac55fca700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b5704b96b0 0x55b5704dfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.443+0000 7fac55fca700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b5704b96b0 0x55b5704dfbd0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.443+0000 7fac54fc8700 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac38067aa0 0x7fac38069f50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fac48000c00 tx=0x7fac48001800 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac55fca700 1 -- 172.21.15.45:0/3692783076 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v129'375 uv375 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fac50003a40 con 0x55b5704b96b0 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b5704b96b0 msgr2=0x55b5704dfbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55b5704b96b0 0x55b5704dfbd0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac38067aa0 msgr2=0x7fac38069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fac38067aa0 0x7fac38069f50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fac48000c00 tx=0x7fac48001800 comp rx=0 tx=0).stop 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 msgr2=0x55b5704bf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 --2- 172.21.15.45:0/3692783076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5704397e0 0x55b5704bf120 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fac4001e000 tx=0x7fac4001c000 comp rx=0 tx=0).stop 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac55fca700 1 -- 172.21.15.45:0/3692783076 reap_dead start 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 shutdown_connections 2023-10-06T05:26:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 >> 172.21.15.45:0/3692783076 conn(0x55b5703f2450 msgr2=0x55b5704c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:40.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 shutdown_connections 2023-10-06T05:26:40.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.445+0000 7fac5c5ab040 1 -- 172.21.15.45:0/3692783076 wait complete. 2023-10-06T05:26:40.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:26:40.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.590+0000 7f4ab2a69700 1 Processor -- start 2023-10-06T05:26:40.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.590+0000 7f4ab2a69700 1 -- start start 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.590+0000 7f4ab2a69700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4aac06aa70 0x7f4aac0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4ab2a69700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4aac0c7050 0x7f4aac15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4ab2a69700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac067d80 con 0x7f4aac06ae40 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4ab2a69700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac0ac620 con 0x7f4aac0c7050 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4ab2a69700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac0c75b0 con 0x7f4aac06aa70 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4aac0c7050 0x7f4aac15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4aac0c7050 0x7f4aac15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56504/0 (socket says 172.21.15.45:56504) 2023-10-06T05:26:40.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 -- 172.21.15.45:0/3201144832 learned_addr learned my addr 172.21.15.45:0/3201144832 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4ab0805700 1 --2- 172.21.15.45:0/3201144832 >> v2:172.21.15.144:3300/0 conn(0x7f4aac06aa70 0x7f4aac0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 -- 172.21.15.45:0/3201144832 >> v2:172.21.15.144:3300/0 conn(0x7f4aac06aa70 msgr2=0x7f4aac0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 --2- 172.21.15.45:0/3201144832 >> v2:172.21.15.144:3300/0 conn(0x7f4aac06aa70 0x7f4aac0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 -- 172.21.15.45:0/3201144832 >> v1:172.21.15.144:6789/0 conn(0x7f4aac06ae40 legacy=0x7f4aac06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 -- 172.21.15.45:0/3201144832 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0c7890 con 0x7f4aac0c7050 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aabfff700 1 --2- 172.21.15.45:0/3201144832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac15c1b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4aac06f990 tx=0x7f4a94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2a7c4875e1670d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aab7fe700 1 -- 172.21.15.45:0/3201144832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94009e70 con 0x7f4aac0c7050 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aab7fe700 1 -- 172.21.15.45:0/3201144832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9400b860 con 0x7f4aac0c7050 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.591+0000 7f4aab7fe700 1 -- 172.21.15.45:0/3201144832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400bc00 con 0x7f4aac0c7050 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 -- 172.21.15.45:0/3201144832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 msgr2=0x7f4aac15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/3201144832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac15c1b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4aac06f990 tx=0x7f4a94009250 comp rx=0 tx=0).stop 2023-10-06T05:26:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 -- 172.21.15.45:0/3201144832 shutdown_connections 2023-10-06T05:26:40.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/3201144832 >> v2:172.21.15.144:3300/0 conn(0x7f4aac06aa70 0x7f4aac0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/3201144832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac15c1b0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 -- 172.21.15.45:0/3201144832 >> 172.21.15.45:0/3201144832 conn(0x7f4aac0cc570 msgr2=0x7f4aac0cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:40.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 -- 172.21.15.45:0/3201144832 shutdown_connections 2023-10-06T05:26:40.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.592+0000 7f4ab2a69700 1 -- 172.21.15.45:0/3201144832 wait complete. 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.593+0000 7f4ab2a69700 1 Processor -- start 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.593+0000 7f4ab2a69700 1 -- start start 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab2a69700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4aac06aa70 0x7f4aac154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab2a69700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aac06ae40 0x7f4aac0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab2a69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac1565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab2a69700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0c0c60 con 0x7f4aac0c7050 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab2a69700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1550e0 con 0x7f4aac06ae40 2023-10-06T05:26:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab2a69700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0c0050 con 0x7f4aac06aa70 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab1006700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac1565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab1006700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac1565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56516/0 (socket says 172.21.15.45:56516) 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab1006700 1 -- 172.21.15.45:0/2381691753 learned_addr learned my addr 172.21.15.45:0/2381691753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4ab0805700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4aac06aa70 0x7f4aac154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.594+0000 7f4aabfff700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aac06ae40 0x7f4aac0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab1006700 1 -- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4aac06aa70 msgr2=0x7f4aac154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab1006700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4aac06aa70 0x7f4aac154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab1006700 1 -- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aac06ae40 msgr2=0x7f4aac0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab1006700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aac06ae40 0x7f4aac0b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab1006700 1 -- 172.21.15.45:0/2381691753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a940094f0 con 0x7f4aac0c7050 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab1006700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac1565e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c8e0 tx=0x7f4aa4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4aa8ff9700 1 -- 172.21.15.45:0/2381691753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4012630 con 0x7f4aac0c7050 2023-10-06T05:26:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4aa8ff9700 1 -- 172.21.15.45:0/2381691753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4012790 con 0x7f4aac0c7050 2023-10-06T05:26:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac0c0330 con 0x7f4aac0c7050 2023-10-06T05:26:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4aa8ff9700 1 -- 172.21.15.45:0/2381691753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4012b50 con 0x7f4aac0c7050 2023-10-06T05:26:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.595+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac0c08f0 con 0x7f4aac0c7050 2023-10-06T05:26:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.596+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac06e300 con 0x7f4aac0c7050 2023-10-06T05:26:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.597+0000 7f4aa8ff9700 1 -- 172.21.15.45:0/2381691753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4aa401a3a0 con 0x7f4aac0c7050 2023-10-06T05:26:40.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.597+0000 7f4aa8ff9700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a8c066f30 0x7f4a8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:40.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.597+0000 7f4aa8ff9700 1 -- 172.21.15.45:0/2381691753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9560+0+0 (secure 0 0 0) 0x7f4aa4085690 con 0x7f4aac0c7050 2023-10-06T05:26:40.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.597+0000 7f4ab0805700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a8c066f30 0x7f4a8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:40.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.598+0000 7f4ab0805700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a8c066f30 0x7f4a8c0693e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009db0 tx=0x7f4a9c0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:40.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.601+0000 7f4aa8ff9700 1 -- 172.21.15.45:0/2381691753 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4aa40519b0 con 0x7f4aac0c7050 2023-10-06T05:26:40.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.853+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f4aac0c7b70 con 0x7f4aac0c7050 2023-10-06T05:26:40.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.909+0000 7f4aa8ff9700 1 -- 172.21.15.45:0/2381691753 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v331) v1 ==== 162+0+0 (secure 0 0 0) 0x7f4aa4054fd0 con 0x7f4aac0c7050 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a8c066f30 msgr2=0x7f4a8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a8c066f30 0x7f4a8c0693e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009db0 tx=0x7f4a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 msgr2=0x7f4aac1565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac1565e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c8e0 tx=0x7f4aa4001710 comp rx=0 tx=0).stop 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 shutdown_connections 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4a8c066f30 0x7f4a8c0693e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac0c7050 0x7f4aac1565e0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4aac06aa70 0x7f4aac154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 --2- 172.21.15.45:0/2381691753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aac06ae40 0x7f4aac0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:40.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 >> 172.21.15.45:0/2381691753 conn(0x7f4aac0cc570 msgr2=0x7f4aac0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 shutdown_connections 2023-10-06T05:26:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:40.912+0000 7f4ab2a69700 1 -- 172.21.15.45:0/2381691753 wait complete. 2023-10-06T05:26:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:26:40.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:41.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.063+0000 7f1749736700 1 Processor -- start 2023-10-06T05:26:41.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.063+0000 7f1749736700 1 -- start start 2023-10-06T05:26:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1749736700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1744159170 0x7f1744159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1749736700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f17440aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1749736700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17440ca6d0 con 0x7f174415d040 2023-10-06T05:26:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1749736700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174415d550 con 0x7f1744159170 2023-10-06T05:26:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1749736700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174415d810 con 0x7f17440692d0 2023-10-06T05:26:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f17437fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f17440aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:41.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1742ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1744159170 0x7f1744159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:41.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1742ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1744159170 0x7f1744159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:49624/0 (socket says 172.21.15.45:49624) 2023-10-06T05:26:41.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f17437fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f17440aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:60750/0 (socket says 172.21.15.45:60750) 2023-10-06T05:26:41.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f1742ffd700 1 -- 172.21.15.45:0/1400090 learned_addr learned my addr 172.21.15.45:0/1400090 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:41.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.064+0000 7f17427fc700 1 --1- 172.21.15.45:0/1400090 >> v1:172.21.15.144:6789/0 conn(0x7f174415d040 0x7f1744068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:53550/0 (socket says 172.21.15.45:53550) 2023-10-06T05:26:41.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1742ffd700 1 -- 172.21.15.45:0/1400090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 msgr2=0x7f17440aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:41.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1742ffd700 1 --2- 172.21.15.45:0/1400090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f17440aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1742ffd700 1 -- 172.21.15.45:0/1400090 >> v1:172.21.15.144:6789/0 conn(0x7f174415d040 legacy=0x7f1744068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:41.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1742ffd700 1 -- 172.21.15.45:0/1400090 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174415daf0 con 0x7f1744159170 2023-10-06T05:26:41.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1742ffd700 1 --2- 172.21.15.45:0/1400090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1744159170 0x7f1744159540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17340099b0 tx=0x7f1734009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2d31339b3e42f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:41.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1741ffb700 1 -- 172.21.15.45:0/1400090 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734009ea0 con 0x7f1744159170 2023-10-06T05:26:41.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1741ffb700 1 -- 172.21.15.45:0/1400090 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173400b860 con 0x7f1744159170 2023-10-06T05:26:41.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.065+0000 7f1741ffb700 1 -- 172.21.15.45:0/1400090 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400bc00 con 0x7f1744159170 2023-10-06T05:26:41.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 -- 172.21.15.45:0/1400090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1744159170 msgr2=0x7f1744159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:41.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 --2- 172.21.15.45:0/1400090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1744159170 0x7f1744159540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17340099b0 tx=0x7f1734009250 comp rx=0 tx=0).stop 2023-10-06T05:26:41.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 -- 172.21.15.45:0/1400090 shutdown_connections 2023-10-06T05:26:41.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 --2- 172.21.15.45:0/1400090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f17440aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 --2- 172.21.15.45:0/1400090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1744159170 0x7f1744159540 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 -- 172.21.15.45:0/1400090 >> 172.21.15.45:0/1400090 conn(0x7f174406f7c0 msgr2=0x7f17440d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 -- 172.21.15.45:0/1400090 shutdown_connections 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.066+0000 7f1749736700 1 -- 172.21.15.45:0/1400090 wait complete. 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.067+0000 7f1749736700 1 Processor -- start 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.067+0000 7f1749736700 1 -- start start 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f1749736700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f1744158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f1749736700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1744159170 0x7f1744159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f1749736700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174415d040 0x7f174416b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f1749736700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174415b340 con 0x7f1744159170 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f1749736700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174415a430 con 0x7f174415d040 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f1749736700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174415a710 con 0x7f17440692d0 2023-10-06T05:26:41.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1744159170 0x7f1744159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1744159170 0x7f1744159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56530/0 (socket says 172.21.15.45:56530) 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 -- 172.21.15.45:0/34131955 learned_addr learned my addr 172.21.15.45:0/34131955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17437fe700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174415d040 0x7f174416b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f1742ffd700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f1744158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 -- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 msgr2=0x7f1744158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f1744158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 -- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174415d040 msgr2=0x7f174416b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174415d040 0x7f174416b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.068+0000 7f17427fc700 1 -- 172.21.15.45:0/34131955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340094f0 con 0x7f1744159170 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.069+0000 7f17427fc700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1744159170 0x7f1744159b30 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f1738001eb0 tx=0x7f1738007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.069+0000 7f172affd700 1 -- 172.21.15.45:0/34131955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738005a70 con 0x7f1744159170 2023-10-06T05:26:41.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.069+0000 7f172affd700 1 -- 172.21.15.45:0/34131955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738005bd0 con 0x7f1744159170 2023-10-06T05:26:41.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.069+0000 7f172affd700 1 -- 172.21.15.45:0/34131955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380070d0 con 0x7f1744159170 2023-10-06T05:26:41.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.069+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174415a9f0 con 0x7f1744159170 2023-10-06T05:26:41.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.069+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1744158ca0 con 0x7f1744159170 2023-10-06T05:26:41.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.070+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1744063b50 con 0x7f1744159170 2023-10-06T05:26:41.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.070+0000 7f172affd700 1 -- 172.21.15.45:0/34131955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1738007480 con 0x7f1744159170 2023-10-06T05:26:41.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.071+0000 7f172affd700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1730066f30 0x7f17300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:41.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.071+0000 7f172affd700 1 -- 172.21.15.45:0/34131955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9560+0+0 (secure 0 0 0) 0x7f17380847b0 con 0x7f1744159170 2023-10-06T05:26:41.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.071+0000 7f1742ffd700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1730066f30 0x7f17300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:41.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.072+0000 7f1742ffd700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1730066f30 0x7f17300693e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f17340099b0 tx=0x7f1734017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:41.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.076+0000 7f172affd700 1 -- 172.21.15.45:0/34131955 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1738050ad0 con 0x7f1744159170 2023-10-06T05:26:41.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.323+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f174415b480 con 0x7f1744159170 2023-10-06T05:26:41.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.323+0000 7f172affd700 1 -- 172.21.15.45:0/34131955 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 331 v331) v1 ==== 95+0+4609 (secure 0 0 0) 0x7f1738017020 con 0x7f1744159170 2023-10-06T05:26:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:41.325 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":331,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:40.906845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8522":{"gid":8522,"name":"b","rank":0,"incarnation":331,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/1302691811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1302691811},{"type":"v1","addr":"172.21.15.144:6835","nonce":1302691811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:41.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1730066f30 msgr2=0x7f17300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1730066f30 0x7f17300693e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f17340099b0 tx=0x7f1734017000 comp rx=0 tx=0).stop 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1744159170 msgr2=0x7f1744159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1744159170 0x7f1744159b30 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f1738001eb0 tx=0x7f1738007f40 comp rx=0 tx=0).stop 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 shutdown_connections 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1730066f30 0x7f17300693e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174415d040 0x7f174416b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17440692d0 0x7f1744158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 --2- 172.21.15.45:0/34131955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1744159170 0x7f1744159b30 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 >> 172.21.15.45:0/34131955 conn(0x7f174406f7c0 msgr2=0x7f17440b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 shutdown_connections 2023-10-06T05:26:41.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:41.325+0000 7f1749736700 1 -- 172.21.15.45:0/34131955 wait complete. 2023-10-06T05:26:41.329 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 331 2023-10-06T05:26:41.340 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:26:43.922 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:43.921+0000 7f3d46a14700 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:43.924 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:43.922+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:43.924 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:43.922+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:26:43.941 INFO:tasks.ceph.mds.b.smithi144.stderr: -20> 2023-10-06T05:26:43.921+0000 7f3d46a14700 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:43.963 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:26:46.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:46.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.480+0000 7f8ff3f37700 1 Processor -- start 2023-10-06T05:26:46.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.481+0000 7f8ff3f37700 1 -- start start 2023-10-06T05:26:46.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff3f37700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff3f37700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fec157030 0x7f8fec0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff3f37700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec15b000 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff3f37700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec0c9830 con 0x7f8fec157030 2023-10-06T05:26:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff3f37700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec15b2e0 con 0x7f8fec0c93c0 2023-10-06T05:26:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff24d4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fec157030 0x7f8fec0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff1cd3700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8fec0c8ff0 0x7f8fec0caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34058/0 (socket says 172.21.15.45:34058) 2023-10-06T05:26:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff1cd3700 1 -- 172.21.15.45:0/2894379221 learned_addr learned my addr 172.21.15.45:0/2894379221 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff24d4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fec157030 0x7f8fec0cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53010/0 (socket says 172.21.15.45:53010) 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff14d2700 1 --2- 172.21.15.45:0/2894379221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff24d4700 1 -- 172.21.15.45:0/2894379221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 msgr2=0x7f8fec156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff24d4700 1 --2- 172.21.15.45:0/2894379221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff24d4700 1 -- 172.21.15.45:0/2894379221 >> v1:172.21.15.45:6789/0 conn(0x7f8fec0c8ff0 legacy=0x7f8fec0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff24d4700 1 -- 172.21.15.45:0/2894379221 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec15b5c0 con 0x7f8fec157030 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff0cd1700 1 -- 172.21.15.45:0/2894379221 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309662045 0 0) 0x7f8fec15b000 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff24d4700 1 --2- 172.21.15.45:0/2894379221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec157030 0x7f8fec0cc330 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e60 tx=0x7f8fe4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1dd4fa1a9c1d61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.482+0000 7f8ff0cd1700 1 -- 172.21.15.45:0/2894379221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400e070 con 0x7f8fec157030 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff0cd1700 1 -- 172.21.15.45:0/2894379221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400bb60 con 0x7f8fec157030 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff0cd1700 1 -- 172.21.15.45:0/2894379221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4012430 con 0x7f8fec157030 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 -- 172.21.15.45:0/2894379221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec157030 msgr2=0x7f8fec0cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/2894379221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec157030 0x7f8fec0cc330 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e60 tx=0x7f8fe4009780 comp rx=0 tx=0).stop 2023-10-06T05:26:46.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 -- 172.21.15.45:0/2894379221 shutdown_connections 2023-10-06T05:26:46.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/2894379221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec157030 0x7f8fec0cc330 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/2894379221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 -- 172.21.15.45:0/2894379221 >> 172.21.15.45:0/2894379221 conn(0x7f8fec0cdf50 msgr2=0x7f8fec0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:46.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 -- 172.21.15.45:0/2894379221 shutdown_connections 2023-10-06T05:26:46.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.483+0000 7f8ff3f37700 1 -- 172.21.15.45:0/2894379221 wait complete. 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.484+0000 7f8ff3f37700 1 Processor -- start 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.484+0000 7f8ff3f37700 1 -- start start 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff3f37700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0c8ff0 0x7f8fec14f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff3f37700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec0ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff3f37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec157030 0x7f8fec154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff3f37700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c5d70 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff3f37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0ad930 con 0x7f8fec157030 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff3f37700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec153490 con 0x7f8fec0c93c0 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff1cd3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0c8ff0 0x7f8fec14f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff1cd3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0c8ff0 0x7f8fec14f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53012/0 (socket says 172.21.15.45:53012) 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff1cd3700 1 -- 172.21.15.45:0/3278468235 learned_addr learned my addr 172.21.15.45:0/3278468235 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.485+0000 7f8ff14d2700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec0ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff24d4700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec157030 0x7f8fec154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff1cd3700 1 -- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 msgr2=0x7f8fec0ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff1cd3700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec0ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff1cd3700 1 -- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec157030 msgr2=0x7f8fec154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff1cd3700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec157030 0x7f8fec154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff1cd3700 1 -- 172.21.15.45:0/3278468235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4009a20 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff24d4700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec157030 0x7f8fec154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff1cd3700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0c8ff0 0x7f8fec14f400 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c5b0 tx=0x7f8fe8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8fe1ffb700 1 -- 172.21.15.45:0/3278468235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8005b30 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8fe1ffb700 1 -- 172.21.15.45:0/3278468235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8005c90 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec153770 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8fe1ffb700 1 -- 172.21.15.45:0/3278468235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8007300 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.486+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec0c5a00 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.487+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec06f410 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.488+0000 7f8fe1ffb700 1 -- 172.21.15.45:0/3278468235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fe801b3b0 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.488+0000 7f8fe1ffb700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8fd0066f30 0x7f8fd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.488+0000 7f8fe1ffb700 1 -- 172.21.15.45:0/3278468235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7f8fe8084a00 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.489+0000 7f8ff14d2700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8fd0066f30 0x7f8fd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.489+0000 7f8ff14d2700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8fd0066f30 0x7f8fd00693e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8fec16c9b0 tx=0x7f8fdc0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:46.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.493+0000 7f8fe1ffb700 1 -- 172.21.15.45:0/3278468235 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fe8050cd0 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.746+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fec04bb40 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.747+0000 7f8fe1ffb700 1 -- 172.21.15.45:0/3278468235 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 336 v336) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f8fe8017070 con 0x7f8fec0c8ff0 2023-10-06T05:26:46.749 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:46.749 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":336,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1309152247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1309152247},{"type":"v1","addr":"172.21.15.144:6835","nonce":1309152247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:43.923644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:46.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8fd0066f30 msgr2=0x7f8fd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8fd0066f30 0x7f8fd00693e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8fec16c9b0 tx=0x7f8fdc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0c8ff0 msgr2=0x7f8fec14f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0c8ff0 0x7f8fec14f400 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c5b0 tx=0x7f8fe8001070 comp rx=0 tx=0).stop 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 shutdown_connections 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8fd0066f30 0x7f8fd00693e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec157030 0x7f8fec154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0c8ff0 0x7f8fec14f400 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 --2- 172.21.15.45:0/3278468235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0c93c0 0x7f8fec0ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 >> 172.21.15.45:0/3278468235 conn(0x7f8fec0cdf50 msgr2=0x7f8fec150140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 shutdown_connections 2023-10-06T05:26:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.749+0000 7f8ff3f37700 1 -- 172.21.15.45:0/3278468235 wait complete. 2023-10-06T05:26:46.753 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 336 2023-10-06T05:26:46.764 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:26:46.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.896+0000 7f18a5a68700 1 Processor -- start 2023-10-06T05:26:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.897+0000 7f18a5a68700 1 -- start start 2023-10-06T05:26:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.897+0000 7f18a5a68700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18a0158f60 0x7f18a0159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.897+0000 7f18a5a68700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18a00cc390 0x7f18a00cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.897+0000 7f18a5a68700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a00cb530 con 0x7f18a006a060 2023-10-06T05:26:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.897+0000 7f18a5a68700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a006a500 con 0x7f18a0158f60 2023-10-06T05:26:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.897+0000 7f18a5a68700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a0069bd0 con 0x7f18a00cc390 2023-10-06T05:26:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18a0158f60 0x7f18a0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18a0158f60 0x7f18a0159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53022/0 (socket says 172.21.15.45:53022) 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 -- 172.21.15.45:0/1533810283 learned_addr learned my addr 172.21.15.45:0/1533810283 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189ffff700 1 --2- 172.21.15.45:0/1533810283 >> v2:172.21.15.144:3300/0 conn(0x7f18a00cc390 0x7f18a00cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 -- 172.21.15.45:0/1533810283 >> v2:172.21.15.144:3300/0 conn(0x7f18a00cc390 msgr2=0x7f18a00cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 --2- 172.21.15.45:0/1533810283 >> v2:172.21.15.144:3300/0 conn(0x7f18a00cc390 0x7f18a00cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 -- 172.21.15.45:0/1533810283 >> v1:172.21.15.144:6789/0 conn(0x7f18a006a060 legacy=0x7f18a00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 -- 172.21.15.45:0/1533810283 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00ccd10 con 0x7f18a0158f60 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189ffff700 1 --2- 172.21.15.45:0/1533810283 >> v2:172.21.15.144:3300/0 conn(0x7f18a00cc390 0x7f18a00cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189f7fe700 1 --2- 172.21.15.45:0/1533810283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0158f60 0x7f18a0159330 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ca4c0 tx=0x7f1890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3925e0dce731f6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189e7fc700 1 -- 172.21.15.45:0/1533810283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000b860 con 0x7f18a0158f60 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189e7fc700 1 -- 172.21.15.45:0/1533810283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189000b9c0 con 0x7f18a0158f60 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.898+0000 7f189e7fc700 1 -- 172.21.15.45:0/1533810283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000bd60 con 0x7f18a0158f60 2023-10-06T05:26:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 -- 172.21.15.45:0/1533810283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0158f60 msgr2=0x7f18a0159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 --2- 172.21.15.45:0/1533810283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0158f60 0x7f18a0159330 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ca4c0 tx=0x7f1890009250 comp rx=0 tx=0).stop 2023-10-06T05:26:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 -- 172.21.15.45:0/1533810283 shutdown_connections 2023-10-06T05:26:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 --2- 172.21.15.45:0/1533810283 >> v2:172.21.15.144:3300/0 conn(0x7f18a00cc390 0x7f18a00cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 --2- 172.21.15.45:0/1533810283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0158f60 0x7f18a0159330 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 -- 172.21.15.45:0/1533810283 >> 172.21.15.45:0/1533810283 conn(0x7f18a00cdf50 msgr2=0x7f18a00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 -- 172.21.15.45:0/1533810283 shutdown_connections 2023-10-06T05:26:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.899+0000 7f18a5a68700 1 -- 172.21.15.45:0/1533810283 wait complete. 2023-10-06T05:26:46.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.900+0000 7f18a5a68700 1 Processor -- start 2023-10-06T05:26:46.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f18a5a68700 1 -- start start 2023-10-06T05:26:46.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f18a5a68700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a006a060 0x7f18a00c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f18a5a68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a00cc390 0x7f18a015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f18a5a68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a0158f60 0x7f18a0146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f18a5a68700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00c6570 con 0x7f18a00cc390 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f18a5a68700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a015c430 con 0x7f18a0158f60 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f18a5a68700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a0146f40 con 0x7f18a006a060 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.901+0000 7f189effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a00cc390 0x7f18a015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a00cc390 0x7f18a015bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53030/0 (socket says 172.21.15.45:53030) 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 -- 172.21.15.45:0/3698100273 learned_addr learned my addr 172.21.15.45:0/3698100273 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:47.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189f7fe700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a006a060 0x7f18a00c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189ffff700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a0158f60 0x7f18a0146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 -- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a006a060 msgr2=0x7f18a00c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a006a060 0x7f18a00c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 -- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a0158f60 msgr2=0x7f18a0146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a0158f60 0x7f18a0146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 -- 172.21.15.45:0/3698100273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900094f0 con 0x7f18a00cc390 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189f7fe700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a006a060 0x7f18a00c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f189effd700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a00cc390 0x7f18a015bef0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f189400c5c0 tx=0x7f189400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f18877fe700 1 -- 172.21.15.45:0/3698100273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894011590 con 0x7f18a00cc390 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f18877fe700 1 -- 172.21.15.45:0/3698100273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894011710 con 0x7f18a00cc390 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f18877fe700 1 -- 172.21.15.45:0/3698100273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894011ad0 con 0x7f18a00cc390 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.902+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a00c56a0 con 0x7f18a00cc390 2023-10-06T05:26:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.903+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a00c5c60 con 0x7f18a00cc390 2023-10-06T05:26:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.903+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a006f410 con 0x7f18a00cc390 2023-10-06T05:26:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.904+0000 7f18877fe700 1 -- 172.21.15.45:0/3698100273 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1894012420 con 0x7f18a00cc390 2023-10-06T05:26:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.904+0000 7f18877fe700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f188c066f30 0x7f188c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.904+0000 7f18877fe700 1 -- 172.21.15.45:0/3698100273 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7f1894084730 con 0x7f18a00cc390 2023-10-06T05:26:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.904+0000 7f189f7fe700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f188c066f30 0x7f188c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.905+0000 7f189f7fe700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f188c066f30 0x7f188c0693e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ca4c0 tx=0x7f1890017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:46.909+0000 7f18877fe700 1 -- 172.21.15.45:0/3698100273 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f189404b5a0 con 0x7f18a00cc390 2023-10-06T05:26:47.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.155+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18a00cb530 con 0x7f18a00cc390 2023-10-06T05:26:47.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.155+0000 7f18877fe700 1 -- 172.21.15.45:0/3698100273 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 336 v336) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f1894084210 con 0x7f18a00cc390 2023-10-06T05:26:47.157 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:47.158 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":336,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1309152247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1309152247},{"type":"v1","addr":"172.21.15.144:6835","nonce":1309152247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:43.923644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.157+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f188c066f30 msgr2=0x7f188c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.157+0000 7f18a5a68700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f188c066f30 0x7f188c0693e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f18a00ca4c0 tx=0x7f1890017000 comp rx=0 tx=0).stop 2023-10-06T05:26:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.157+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a00cc390 msgr2=0x7f18a015bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.157+0000 7f18a5a68700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a00cc390 0x7f18a015bef0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f189400c5c0 tx=0x7f189400c980 comp rx=0 tx=0).stop 2023-10-06T05:26:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 shutdown_connections 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f188c066f30 0x7f188c0693e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a0158f60 0x7f18a0146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a006a060 0x7f18a00c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 --2- 172.21.15.45:0/3698100273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a00cc390 0x7f18a015bef0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 >> 172.21.15.45:0/3698100273 conn(0x7f18a00cdf50 msgr2=0x7f18a0158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 shutdown_connections 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.158+0000 7f18a5a68700 1 -- 172.21.15.45:0/3698100273 wait complete. 2023-10-06T05:26:47.161 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 336 2023-10-06T05:26:47.173 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:26:47.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.308+0000 7f9d70865700 1 Processor -- start 2023-10-06T05:26:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.308+0000 7f9d70865700 1 -- start start 2023-10-06T05:26:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.309+0000 7f9d70865700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c93c0 0x7f9d6c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.309+0000 7f9d70865700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c15b000 con 0x7f9d6c0c8ff0 2023-10-06T05:26:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.309+0000 7f9d70865700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c15b2e0 con 0x7f9d6c157030 2023-10-06T05:26:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d70865700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0c9830 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d6a59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9d6c0c8ff0 0x7f9d6c0caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34072/0 (socket says 172.21.15.45:34072) 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d6a59c700 1 -- 172.21.15.45:0/2529897880 learned_addr learned my addr 172.21.15.45:0/2529897880 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d69d9b700 1 --2- 172.21.15.45:0/2529897880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c93c0 0x7f9d6c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634235577 0 0) 0x7f9d6c15b000 con 0x7f9d6c0c8ff0 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d58003570 con 0x7f9d6c0c8ff0 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205190086 0 0) 0x7f9d6c15b2e0 con 0x7f9d6c157030 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d6c15b000 con 0x7f9d6c157030 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3324452772 0 0) 0x7f9d58003570 con 0x7f9d6c0c8ff0 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d69d9b700 1 -- 172.21.15.45:0/2529897880 >> v1:172.21.15.144:6789/0 conn(0x7f9d6c157030 legacy=0x7f9d6c0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d69d9b700 1 -- 172.21.15.45:0/2529897880 >> v1:172.21.15.45:6789/0 conn(0x7f9d6c0c8ff0 legacy=0x7f9d6c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d69d9b700 1 -- 172.21.15.45:0/2529897880 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c15b5c0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.310+0000 7f9d69d9b700 1 --2- 172.21.15.45:0/2529897880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c93c0 0x7f9d6c156af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009910 tx=0x7f9d540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81bb08f035a5b5fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c001470 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d54011420 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d6959a700 1 -- 172.21.15.45:0/2529897880 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d540117a0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d70865700 1 -- 172.21.15.45:0/2529897880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c93c0 msgr2=0x7f9d6c156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d70865700 1 --2- 172.21.15.45:0/2529897880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c93c0 0x7f9d6c156af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009910 tx=0x7f9d540091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d70865700 1 -- 172.21.15.45:0/2529897880 shutdown_connections 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d70865700 1 --2- 172.21.15.45:0/2529897880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c93c0 0x7f9d6c156af0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d70865700 1 -- 172.21.15.45:0/2529897880 >> 172.21.15.45:0/2529897880 conn(0x7f9d6c0cdf50 msgr2=0x7f9d6c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d70865700 1 -- 172.21.15.45:0/2529897880 shutdown_connections 2023-10-06T05:26:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.311+0000 7f9d70865700 1 -- 172.21.15.45:0/2529897880 wait complete. 2023-10-06T05:26:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.312+0000 7f9d70865700 1 Processor -- start 2023-10-06T05:26:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d70865700 1 -- start start 2023-10-06T05:26:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d70865700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c8ff0 0x7f9d6c15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d70865700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6c0c93c0 0x7f9d6c0c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d70865700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d6c157030 0x7f9d6c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d70865700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0c6f20 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d70865700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0c6350 con 0x7f9d6c157030 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d70865700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0c65f0 con 0x7f9d6c0c8ff0 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6c0c93c0 0x7f9d6c0c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6c0c93c0 0x7f9d6c0c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53034/0 (socket says 172.21.15.45:53034) 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d69d9b700 1 -- 172.21.15.45:0/4003343066 learned_addr learned my addr 172.21.15.45:0/4003343066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.313+0000 7f9d6a59c700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c8ff0 0x7f9d6c15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d6ad9d700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d6c157030 0x7f9d6c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d69d9b700 1 -- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c8ff0 msgr2=0x7f9d6c15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d69d9b700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c8ff0 0x7f9d6c15c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d69d9b700 1 -- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d6c157030 msgr2=0x7f9d6c0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d69d9b700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d6c157030 0x7f9d6c0b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d69d9b700 1 -- 172.21.15.45:0/4003343066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d54009420 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d69d9b700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6c0c93c0 0x7f9d6c0c73b0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009860 tx=0x7f9d54016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d627fc700 1 -- 172.21.15.45:0/4003343066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5401caf0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d627fc700 1 -- 172.21.15.45:0/4003343066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5401cc50 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d627fc700 1 -- 172.21.15.45:0/4003343066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5401a450 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c0c68d0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.314+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c0b0eb0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.315+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c0c70d0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.316+0000 7f9d627fc700 1 -- 172.21.15.45:0/4003343066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d5401a840 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.316+0000 7f9d627fc700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d4c066f30 0x7f9d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.316+0000 7f9d627fc700 1 -- 172.21.15.45:0/4003343066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7f9d54086d40 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.317+0000 7f9d6a59c700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d4c066f30 0x7f9d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.317+0000 7f9d6a59c700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d4c066f30 0x7f9d4c0693e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0011e0 tx=0x7f9d5c000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.321+0000 7f9d627fc700 1 -- 172.21.15.45:0/4003343066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d6c0c70d0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.570+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d6c0c70d0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.571+0000 7f9d627fc700 1 -- 172.21.15.45:0/4003343066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 336 v336) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f9d6c0c70d0 con 0x7f9d6c0c93c0 2023-10-06T05:26:47.573 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:47.573 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":336,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1309152247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1309152247},{"type":"v1","addr":"172.21.15.144:6835","nonce":1309152247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:43.923644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:47.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d4c066f30 msgr2=0x7f9d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d4c066f30 0x7f9d4c0693e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0011e0 tx=0x7f9d5c000bc0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6c0c93c0 msgr2=0x7f9d6c0c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6c0c93c0 0x7f9d6c0c73b0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009860 tx=0x7f9d54016000 comp rx=0 tx=0).stop 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 shutdown_connections 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9d4c066f30 0x7f9d4c0693e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d6c157030 0x7f9d6c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c0c8ff0 0x7f9d6c15c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 --2- 172.21.15.45:0/4003343066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6c0c93c0 0x7f9d6c0c73b0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 >> 172.21.15.45:0/4003343066 conn(0x7f9d6c0cdf50 msgr2=0x7f9d6c0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 shutdown_connections 2023-10-06T05:26:47.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.573+0000 7f9d70865700 1 -- 172.21.15.45:0/4003343066 wait complete. 2023-10-06T05:26:47.577 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 336 2023-10-06T05:26:47.588 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2023-10-06T05:26:47.588 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2023-10-06T05:26:47.588 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:47.588 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:26:47.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:47.589 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:47.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:26:47.608 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:26:47.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:47.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.783+0000 7fd6cc864700 1 Processor -- start 2023-10-06T05:26:47.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.783+0000 7fd6cc864700 1 -- start start 2023-10-06T05:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6cc864700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 0x7fd6c80cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6cc864700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80c84c0 con 0x7fd6c80c9640 2023-10-06T05:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6cc864700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80c8d60 con 0x7fd6c80c91f0 2023-10-06T05:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6cc864700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c8069d00 con 0x7fd6c80ca810 2023-10-06T05:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6c6d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd6c80c9640 0x7fd6c8068fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34080/0 (socket says 172.21.15.45:34080) 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6c6d9d700 1 -- 172.21.15.45:0/1218184871 learned_addr learned my addr 172.21.15.45:0/1218184871 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6c659c700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 0x7fd6c80cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662035309 0 0) 0x7fd6c80c84c0 con 0x7fd6c80c9640 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.784+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b4003570 con 0x7fd6c80c9640 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1815743840 0 0) 0x7fd6b4003570 con 0x7fd6c80c9640 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 msgr2=0x7fd6c80cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 0x7fd6c80cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 >> v1:172.21.15.144:6789/0 conn(0x7fd6c80c91f0 legacy=0x7fd6c80afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80c8710 con 0x7fd6c80c9640 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fd6c00017f0 con 0x7fd6c80c9640 2023-10-06T05:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 >> v1:172.21.15.45:6789/0 conn(0x7fd6c80c9640 legacy=0x7fd6c8068fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b4003a90 0x7fd6b4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 0x7fd6b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b4004d50 0x7fd6b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c8069d00 con 0x7fd6b4003a90 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b400bb10 con 0x7fd6b4004d50 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b400bcd0 con 0x7fd6c80ca810 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c659c700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b4003a90 0x7fd6b4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c5d9b700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 0x7fd6b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c6d9d700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b4004d50 0x7fd6b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c659c700 1 -- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 msgr2=0x7fd6b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c659c700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80ca810 0x7fd6b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c659c700 1 -- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b4004d50 msgr2=0x7fd6b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c659c700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b4004d50 0x7fd6b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.785+0000 7fd6c659c700 1 -- 172.21.15.45:0/1218184871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4003570 con 0x7fd6b4003a90 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6c659c700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b4003a90 0x7fd6b4003e60 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009c00 tx=0x7fd6b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546b8240eabc46fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800e710 con 0x7fd6b4003a90 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b800e870 con 0x7fd6b4003a90 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6c559a700 1 -- 172.21.15.45:0/1218184871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800ec30 con 0x7fd6b4003a90 2023-10-06T05:26:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6cc864700 1 -- 172.21.15.45:0/1218184871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b4003a90 msgr2=0x7fd6b4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6cc864700 1 --2- 172.21.15.45:0/1218184871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b4003a90 0x7fd6b4003e60 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009c00 tx=0x7fd6b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6c6d9d700 1 -- 172.21.15.45:0/1218184871 reap_dead start 2023-10-06T05:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6cc864700 1 -- 172.21.15.45:0/1218184871 shutdown_connections 2023-10-06T05:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6cc864700 1 -- 172.21.15.45:0/1218184871 >> 172.21.15.45:0/1218184871 conn(0x7fd6c8067620 msgr2=0x7fd6c8067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6cc864700 1 -- 172.21.15.45:0/1218184871 shutdown_connections 2023-10-06T05:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.786+0000 7fd6cc864700 1 -- 172.21.15.45:0/1218184871 wait complete. 2023-10-06T05:26:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 Processor -- start 2023-10-06T05:26:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 -- start start 2023-10-06T05:26:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b4003a90 0x7fd6c80a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cd5e0 0x7fd6c815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6c815c670 0x7fd6c8169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c816a910 con 0x7fd6c815c670 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c814b0d0 con 0x7fd6c80cd5e0 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.788+0000 7fd6cc864700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80cddc0 con 0x7fd6b4003a90 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6c815c670 0x7fd6c8169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6c815c670 0x7fd6c8169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53050/0 (socket says 172.21.15.45:53050) 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 -- 172.21.15.45:0/2934741455 learned_addr learned my addr 172.21.15.45:0/2934741455 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c659c700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b4003a90 0x7fd6c80a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c5d9b700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cd5e0 0x7fd6c815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 -- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b4003a90 msgr2=0x7fd6c80a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b4003a90 0x7fd6c80a9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 -- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cd5e0 msgr2=0x7fd6c815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:47.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cd5e0 0x7fd6c815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 -- 172.21.15.45:0/2934741455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80ce0a0 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6c6d9d700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6c815c670 0x7fd6c8169780 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c806fa20 tx=0x7fd6c00021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6be7fc700 1 -- 172.21.15.45:0/2934741455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0001d90 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6be7fc700 1 -- 172.21.15.45:0/2934741455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c000acb0 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6be7fc700 1 -- 172.21.15.45:0/2934741455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0013640 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c80ce380 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.789+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c80cda50 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.790+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c816ada0 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.791+0000 7fd6be7fc700 1 -- 172.21.15.45:0/2934741455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6c0013a70 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.791+0000 7fd6be7fc700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd6a8066f30 0x7fd6a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.791+0000 7fd6be7fc700 1 -- 172.21.15.45:0/2934741455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7fd6c0085f80 con 0x7fd6c815c670 2023-10-06T05:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.791+0000 7fd6c659c700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd6a8066f30 0x7fd6a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.792+0000 7fd6c659c700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd6a8066f30 0x7fd6a80693e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80091e0 tx=0x7fd6b801a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:47.796+0000 7fd6be7fc700 1 -- 172.21.15.45:0/2934741455 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6c816ada0 con 0x7fd6c815c670 2023-10-06T05:26:48.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.044+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd6c816ada0 con 0x7fd6c815c670 2023-10-06T05:26:48.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.955+0000 7fd6be7fc700 1 -- 172.21.15.45:0/2934741455 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v337) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd6c816ada0 con 0x7fd6c815c670 2023-10-06T05:26:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd6a8066f30 msgr2=0x7fd6a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd6a8066f30 0x7fd6a80693e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80091e0 tx=0x7fd6b801a000 comp rx=0 tx=0).stop 2023-10-06T05:26:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6c815c670 msgr2=0x7fd6c8169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:48.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6c815c670 0x7fd6c8169780 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c806fa20 tx=0x7fd6c00021f0 comp rx=0 tx=0).stop 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 shutdown_connections 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd6a8066f30 0x7fd6a80693e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6c815c670 0x7fd6c8169780 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b4003a90 0x7fd6c80a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 --2- 172.21.15.45:0/2934741455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cd5e0 0x7fd6c815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 >> 172.21.15.45:0/2934741455 conn(0x7fd6c8067620 msgr2=0x7fd6c80b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 shutdown_connections 2023-10-06T05:26:48.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:48.958+0000 7fd6cc864700 1 -- 172.21.15.45:0/2934741455 wait complete. 2023-10-06T05:26:48.962 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:48.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:26:49.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.111+0000 7f7672623700 1 Processor -- start 2023-10-06T05:26:49.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.111+0000 7f7672623700 1 -- start start 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f7672623700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f7672623700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f766c06add0 0x7f766c068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f7672623700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c0676c0 con 0x7f766c153650 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f7672623700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c06b250 con 0x7f766c06add0 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f7672623700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0679a0 con 0x7f766c153a20 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f766bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f766c153650 0x7f766c15a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34084/0 (socket says 172.21.15.45:34084) 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f766bfff700 1 -- 172.21.15.45:0/394852537 learned_addr learned my addr 172.21.15.45:0/394852537 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.112+0000 7f7670bc0700 1 --2- 172.21.15.45:0/394852537 >> v2:172.21.15.45:3300/0 conn(0x7f766c06add0 0x7f766c068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f766b7fe700 1 --2- 172.21.15.45:0/394852537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f766affd700 1 -- 172.21.15.45:0/394852537 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403275002 0 0) 0x7f766c0676c0 con 0x7f766c153650 2023-10-06T05:26:49.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f766affd700 1 -- 172.21.15.45:0/394852537 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7654003570 con 0x7f766c153650 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f7670bc0700 1 -- 172.21.15.45:0/394852537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 msgr2=0x7f766c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f7670bc0700 1 --2- 172.21.15.45:0/394852537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c06a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f7670bc0700 1 -- 172.21.15.45:0/394852537 >> v1:172.21.15.45:6789/0 conn(0x7f766c153650 legacy=0x7f766c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f7670bc0700 1 -- 172.21.15.45:0/394852537 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c067c80 con 0x7f766c06add0 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f7670bc0700 1 --2- 172.21.15.45:0/394852537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c06add0 0x7f766c068480 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f7664009e60 tx=0x7f7664009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9591cd81d851528b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f766affd700 1 -- 172.21.15.45:0/394852537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400e070 con 0x7f766c06add0 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f766affd700 1 -- 172.21.15.45:0/394852537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400bb60 con 0x7f766c06add0 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.113+0000 7f766affd700 1 -- 172.21.15.45:0/394852537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664012430 con 0x7f766c06add0 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 -- 172.21.15.45:0/394852537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c06add0 msgr2=0x7f766c068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 --2- 172.21.15.45:0/394852537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c06add0 0x7f766c068480 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f7664009e60 tx=0x7f7664009780 comp rx=0 tx=0).stop 2023-10-06T05:26:49.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 -- 172.21.15.45:0/394852537 shutdown_connections 2023-10-06T05:26:49.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 --2- 172.21.15.45:0/394852537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c06add0 0x7f766c068480 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 --2- 172.21.15.45:0/394852537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c06a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 -- 172.21.15.45:0/394852537 >> 172.21.15.45:0/394852537 conn(0x7f766c0cc5a0 msgr2=0x7f766c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:49.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 -- 172.21.15.45:0/394852537 shutdown_connections 2023-10-06T05:26:49.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.114+0000 7f7672623700 1 -- 172.21.15.45:0/394852537 wait complete. 2023-10-06T05:26:49.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.115+0000 7f7672623700 1 Processor -- start 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.115+0000 7f7672623700 1 -- start start 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f7672623700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c06add0 0x7f766c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f7672623700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c153650 0x7f766c0c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f7672623700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f7672623700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0aea10 con 0x7f766c153650 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f7672623700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0b2280 con 0x7f766c06add0 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f7672623700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0c0110 con 0x7f766c153a20 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f766b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c153650 0x7f766c0c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f766b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c153650 0x7f766c0c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53066/0 (socket says 172.21.15.45:53066) 2023-10-06T05:26:49.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f766b7fe700 1 -- 172.21.15.45:0/274346914 learned_addr learned my addr 172.21.15.45:0/274346914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f7670bc0700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.116+0000 7f766bfff700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c06add0 0x7f766c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f766b7fe700 1 -- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 msgr2=0x7f766c0ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f766b7fe700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c0ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f766b7fe700 1 -- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c06add0 msgr2=0x7f766c151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f766b7fe700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c06add0 0x7f766c151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f766b7fe700 1 -- 172.21.15.45:0/274346914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664009a20 con 0x7f766c153650 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f766b7fe700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c153650 0x7f766c0c35b0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f7660006b00 tx=0x7f7660008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f7653fff700 1 -- 172.21.15.45:0/274346914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600125f0 con 0x7f766c153650 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f7653fff700 1 -- 172.21.15.45:0/274346914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660012770 con 0x7f766c153650 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f7653fff700 1 -- 172.21.15.45:0/274346914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660012b30 con 0x7f766c153650 2023-10-06T05:26:49.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c0b19e0 con 0x7f766c153650 2023-10-06T05:26:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.117+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c0b1fa0 con 0x7f766c153650 2023-10-06T05:26:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.118+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c15c860 con 0x7f766c153650 2023-10-06T05:26:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.119+0000 7f7653fff700 1 -- 172.21.15.45:0/274346914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f766001e460 con 0x7f766c153650 2023-10-06T05:26:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.119+0000 7f7653fff700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7658066f30 0x7f76580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.119+0000 7f7653fff700 1 -- 172.21.15.45:0/274346914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7f7660085ad0 con 0x7f766c153650 2023-10-06T05:26:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.119+0000 7f766bfff700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7658066f30 0x7f76580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:49.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.120+0000 7f766bfff700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7658066f30 0x7f76580693e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f765c0058d0 tx=0x7f765c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:49.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.123+0000 7f7653fff700 1 -- 172.21.15.45:0/274346914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7660051da0 con 0x7f766c153650 2023-10-06T05:26:49.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.416+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f766c0c0280 con 0x7f766c153650 2023-10-06T05:26:49.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.958+0000 7f7653fff700 1 -- 172.21.15.45:0/274346914 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v338) v1 ==== 101+0+0 (secure 0 0 0) 0x7f76600553c0 con 0x7f766c153650 2023-10-06T05:26:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7658066f30 msgr2=0x7f76580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7658066f30 0x7f76580693e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f765c0058d0 tx=0x7f765c009040 comp rx=0 tx=0).stop 2023-10-06T05:26:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c153650 msgr2=0x7f766c0c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c153650 0x7f766c0c35b0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f7660006b00 tx=0x7f7660008040 comp rx=0 tx=0).stop 2023-10-06T05:26:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 shutdown_connections 2023-10-06T05:26:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7658066f30 0x7f76580693e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c153a20 0x7f766c0ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c06add0 0x7f766c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 --2- 172.21.15.45:0/274346914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f766c153650 0x7f766c0c35b0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 >> 172.21.15.45:0/274346914 conn(0x7f766c0cc5a0 msgr2=0x7f766c152950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 shutdown_connections 2023-10-06T05:26:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:49.961+0000 7f7672623700 1 -- 172.21.15.45:0/274346914 wait complete. 2023-10-06T05:26:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:26:49.979 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:26:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.024+0000 7f66db6e3040 1 Processor -- start 2023-10-06T05:26:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.024+0000 7f66db6e3040 1 -- start start 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66db6e3040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a88ef0c0 0x55d8a897be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66db6e3040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a89782e0 0x55d8a88a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66db6e3040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 0x55d8a88d6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66db6e3040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a87f2620 con 0x55d8a88ef0c0 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66db6e3040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a87e9400 con 0x55d8a89782e0 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66db6e3040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8a897e090 con 0x55d8a897eb10 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a88ef0c0 0x55d8a897be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a88ef0c0 0x55d8a897be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53084/0 (socket says 172.21.15.45:53084) 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 -- 172.21.15.45:0/3542958089 learned_addr learned my addr 172.21.15.45:0/3542958089 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d5102700 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 0x55d8a88d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4100700 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a89782e0 0x55d8a88a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 -- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 msgr2=0x55d8a88d6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 0x55d8a88d6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 -- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a89782e0 msgr2=0x55d8a88a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a89782e0 0x55d8a88a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4901700 1 -- 172.21.15.45:0/3542958089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c400a180 con 0x55d8a88ef0c0 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.025+0000 7f66d4100700 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a89782e0 0x55d8a88a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66d4901700 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a88ef0c0 0x55d8a897be10 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x55d8a88ab3a0 tx=0x7f66c4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a29aaf822b3a45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66d30fe700 1 -- 172.21.15.45:0/3542958089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c401f070 con 0x55d8a88ef0c0 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66d30fe700 1 -- 172.21.15.45:0/3542958089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c400ed60 con 0x55d8a88ef0c0 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66d30fe700 1 -- 172.21.15.45:0/3542958089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400e7e0 con 0x55d8a88ef0c0 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 -- 172.21.15.45:0/3542958089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a88ef0c0 msgr2=0x55d8a897be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a88ef0c0 0x55d8a897be10 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x55d8a88ab3a0 tx=0x7f66c4011000 comp rx=0 tx=0).stop 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 -- 172.21.15.45:0/3542958089 shutdown_connections 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 0x55d8a88d6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a88ef0c0 0x55d8a897be10 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 --2- 172.21.15.45:0/3542958089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a89782e0 0x55d8a88a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 -- 172.21.15.45:0/3542958089 >> 172.21.15.45:0/3542958089 conn(0x55d8a88a7bd0 msgr2=0x55d8a88a8380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.026+0000 7f66db6e3040 1 -- 172.21.15.45:0/3542958089 shutdown_connections 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.027+0000 7f66db6e3040 1 -- 172.21.15.45:0/3542958089 wait complete. 2023-10-06T05:26:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.027+0000 7f66db6e3040 1 Processor -- start 2023-10-06T05:26:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.027+0000 7f66db6e3040 1 -- start start 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66db6e3040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a88ef0c0 0x55d8a898f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66db6e3040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a89782e0 0x55d8a88d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66db6e3040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 0x55d8a88d7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66db6e3040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a88eb720 con 0x55d8a89782e0 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66db6e3040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a88eb9c0 con 0x55d8a88ef0c0 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66db6e3040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8a88ebc60 con 0x55d8a897eb10 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66d4100700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a89782e0 0x55d8a88d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66d4100700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a89782e0 0x55d8a88d6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53094/0 (socket says 172.21.15.45:53094) 2023-10-06T05:26:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 learned_addr learned my addr 172.21.15.45:0/681829574 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66d4901700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a88ef0c0 0x55d8a898f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.028+0000 7f66d5102700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 0x55d8a88d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 msgr2=0x55d8a88d7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d4100700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8a897eb10 0x55d8a88d7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a88ef0c0 msgr2=0x55d8a898f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d4100700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8a88ef0c0 0x55d8a898f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a87f4750 con 0x55d8a89782e0 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d4100700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a89782e0 0x55d8a88d6ff0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f66c80088f0 tx=0x7f66c8005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d18fb700 1 -- 172.21.15.45:0/681829574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8006760 con 0x55d8a89782e0 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d18fb700 1 -- 172.21.15.45:0/681829574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c8010040 con 0x55d8a89782e0 2023-10-06T05:26:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d8a881b1c0 con 0x55d8a89782e0 2023-10-06T05:26:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66d18fb700 1 -- 172.21.15.45:0/681829574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8006af0 con 0x55d8a89782e0 2023-10-06T05:26:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.029+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8a88013d0 con 0x55d8a89782e0 2023-10-06T05:26:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.031+0000 7f66d18fb700 1 -- 172.21.15.45:0/681829574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66c801e460 con 0x55d8a89782e0 2023-10-06T05:26:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.031+0000 7f66d18fb700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c0065890 0x7f66c0067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.031+0000 7f66d18fb700 1 -- 172.21.15.45:0/681829574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7f66c8084490 con 0x55d8a89782e0 2023-10-06T05:26:50.036 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:26:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.031+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d8a88eb210 0x55d8a8983280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.031+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a896a7e0 con 0x55d8a88eb210 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.031+0000 7f66d4901700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c0065890 0x7f66c0067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.032+0000 7f66d5102700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d8a88eb210 0x55d8a8983280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.032+0000 7f66d5102700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d8a88eb210 0x55d8a8983280 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.032+0000 7f66d4901700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c0065890 0x7f66c0067d40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x55d8a88ab3a0 tx=0x7f66c401c050 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.034+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v130'1159 uv1133 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.034+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a896abc0 con 0x55d8a88eb210 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.036+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v130'1160 uv1160 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.036+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a896afa0 con 0x55d8a88eb210 2023-10-06T05:26:50.039 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:26:50.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.037+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v130'1161 uv1161 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.037+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c850 con 0x55d8a88eb210 2023-10-06T05:26:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.039+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v130'1162 uv1162 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.039+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899cc00 con 0x55d8a88eb210 2023-10-06T05:26:50.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.040+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v130'1163 uv1163 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.040+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899b840 con 0x55d8a88eb210 2023-10-06T05:26:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.041+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v130'1164 uv1164 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.041+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899bc70 con 0x55d8a88eb210 2023-10-06T05:26:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.043+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v130'1165 uv1139 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.043+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c0a0 con 0x55d8a88eb210 2023-10-06T05:26:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.044+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v130'1166 uv1166 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.044+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c0a0 con 0x55d8a88eb210 2023-10-06T05:26:50.047 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:26:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.045+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v130'1167 uv1167 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.045+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c0a0 con 0x55d8a88eb210 2023-10-06T05:26:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.047+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v130'1168 uv1168 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.047+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c0a0 con 0x55d8a88eb210 2023-10-06T05:26:50.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.048+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v130'1169 uv1169 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.048+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c050 con 0x55d8a88eb210 2023-10-06T05:26:50.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.049+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v130'1170 uv1170 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.049+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c050 con 0x55d8a88eb210 2023-10-06T05:26:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.051+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v130'1171 uv1146 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.051+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c050 con 0x55d8a88eb210 2023-10-06T05:26:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.052+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v130'1172 uv1172 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.052+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c050 con 0x55d8a88eb210 2023-10-06T05:26:50.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.054+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v130'1173 uv1173 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.055 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:26:50.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.054+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c050 con 0x55d8a88eb210 2023-10-06T05:26:50.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.055+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v130'1174 uv1174 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.055+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.056+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v130'1175 uv1175 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.056+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.057+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v130'1176 uv1176 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.057+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.059+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v130'1177 uv1177 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.059+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.060+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v130'1178 uv1152 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.060+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.062+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v130'1179 uv1179 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.062+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.064 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:26:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.063+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v130'1180 uv1180 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.063+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.064+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v130'1181 uv1181 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.064+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.066+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v130'1182 uv1182 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.066+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.067+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v130'1183 uv1183 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.067+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.069+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v130'1184 uv1158 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.069+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.070+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v130'1185 uv1185 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.070+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.073 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:26:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.072+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v130'1186 uv1186 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.072+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.073+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v130'1187 uv1187 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.073+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.074+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v130'1188 uv1188 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.074+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a88eb210 2023-10-06T05:26:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.076+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v130'1189 uv1189 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a88eb210 2023-10-06T05:26:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.076+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d8a899d4c0 0x55d8a899f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.076+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899c020 con 0x55d8a899d4c0 2023-10-06T05:26:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.076+0000 7f66d4901700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d8a899d4c0 0x55d8a899f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.076+0000 7f66d4901700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d8a899d4c0 0x55d8a899f8a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.078+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v130'836 uv818 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.078+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a899fde0 con 0x55d8a899d4c0 2023-10-06T05:26:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.080+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v130'837 uv837 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.080+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a0190 con 0x55d8a899d4c0 2023-10-06T05:26:50.083 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:26:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.081+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v130'838 uv838 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.081+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a0ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.083+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v130'839 uv839 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.083+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a0660 con 0x55d8a899d4c0 2023-10-06T05:26:50.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.084+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v130'840 uv840 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.084+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1370 con 0x55d8a899d4c0 2023-10-06T05:26:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.085+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v130'841 uv823 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.085+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1720 con 0x55d8a899d4c0 2023-10-06T05:26:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.087+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v130'842 uv842 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.087+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.089 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:26:50.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.088+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v130'843 uv843 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.088+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.089+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v130'844 uv844 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.089+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.090+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v130'845 uv845 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.090+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.092+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v130'846 uv828 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.092+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.093+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v130'847 uv847 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.093+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.096 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:26:50.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.094+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v130'848 uv848 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.094+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a51f0 con 0x55d8a899d4c0 2023-10-06T05:26:50.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.096+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v130'849 uv849 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.096+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a0f00 con 0x55d8a899d4c0 2023-10-06T05:26:50.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.097+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v130'850 uv850 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.097+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a0f00 con 0x55d8a899d4c0 2023-10-06T05:26:50.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.098+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v130'851 uv834 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.099+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.100+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v130'852 uv852 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.100+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.101+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v130'853 uv853 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.103 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:26:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.101+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a2ae0 con 0x55d8a899d4c0 2023-10-06T05:26:50.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.102+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v130'854 uv854 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.102+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a2ae0 con 0x55d8a899d4c0 2023-10-06T05:26:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.103+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v130'855 uv855 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.103+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a1ad0 con 0x55d8a899d4c0 2023-10-06T05:26:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.105+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v130'856 uv856 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.105+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d8a89a1fe0 0x55d8a89a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.105+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a4990 con 0x55d8a89a1fe0 2023-10-06T05:26:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.105+0000 7f66d4100700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d8a89a1fe0 0x55d8a89a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.105+0000 7f66d4100700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d8a89a1fe0 0x55d8a89a43c0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.107+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.107+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a4e40 con 0x55d8a89a1fe0 2023-10-06T05:26:50.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.108+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v130'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.108+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a53c0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.110+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v130'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.111 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:26:50.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.110+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a6cc0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.111+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.111+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a7070 con 0x55d8a89a1fe0 2023-10-06T05:26:50.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.112+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.112+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a5cb0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.114+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.114+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a60b0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.115+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v130'163 uv162 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.115+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a6500 con 0x55d8a89a1fe0 2023-10-06T05:26:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.117+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v130'164 uv164 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.117+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a6500 con 0x55d8a89a1fe0 2023-10-06T05:26:50.119 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:26:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.118+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v130'165 uv165 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.118+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a64b0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.119+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v130'166 uv166 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.119+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a68d0 con 0x55d8a899d4c0 2023-10-06T05:26:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.120+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.120+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a68d0 con 0x55d8a899d4c0 2023-10-06T05:26:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.122+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v130'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.122+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a68d0 con 0x55d8a899d4c0 2023-10-06T05:26:50.124 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:26:50.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.123+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v130'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.123+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a68d0 con 0x55d8a899d4c0 2023-10-06T05:26:50.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.124+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.124+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a68d0 con 0x55d8a899d4c0 2023-10-06T05:26:50.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.125+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.125+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a68d0 con 0x55d8a899d4c0 2023-10-06T05:26:50.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.126+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a899d4c0 2023-10-06T05:26:50.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.126+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d8a89a6460 0x55d8a89a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.126+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a6460 2023-10-06T05:26:50.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.127+0000 7f66d5102700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d8a89a6460 0x55d8a89a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.127+0000 7f66d5102700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d8a89a6460 0x55d8a89a9940 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.128+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v130'415 uv409 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.128+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a6460 2023-10-06T05:26:50.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.130+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v130'416 uv416 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.130+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a6460 2023-10-06T05:26:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.131+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v130'417 uv417 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.133 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:26:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.131+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89aaef0 con 0x55d8a89a6460 2023-10-06T05:26:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.132+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v130'418 uv418 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.132+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89aaeb0 con 0x55d8a89a6460 2023-10-06T05:26:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.133+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v130'419 uv419 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.133+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89aaeb0 con 0x55d8a89a6460 2023-10-06T05:26:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.134+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v130'420 uv420 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.134+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89aaeb0 con 0x55d8a89a6460 2023-10-06T05:26:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.135+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v130'421 uv414 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.136+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89aaeb0 con 0x55d8a89a6460 2023-10-06T05:26:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.137+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v130'422 uv422 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.137+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89aaeb0 con 0x55d8a89a6460 2023-10-06T05:26:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.138+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v130'423 uv423 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.140 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:26:50.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.138+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89aaeb0 con 0x55d8a89a6460 2023-10-06T05:26:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.139+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v130'424 uv424 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.139+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a6460 2023-10-06T05:26:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.141+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v130'425 uv425 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.141+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d8a89aa790 0x55d8a89ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.141+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89aa790 2023-10-06T05:26:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.141+0000 7f66d4901700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d8a89aa790 0x55d8a89ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.141+0000 7f66d4901700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d8a89aa790 0x55d8a89ad310 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.142+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v130'188 uv187 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a89aa790 2023-10-06T05:26:50.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.143+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89aa790 2023-10-06T05:26:50.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.144+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v130'189 uv189 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a89aa790 2023-10-06T05:26:50.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.144+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89aa790 2023-10-06T05:26:50.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.145+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v130'190 uv190 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a89aa790 2023-10-06T05:26:50.146 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:26:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.145+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89aa790 2023-10-06T05:26:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.146+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v130'191 uv191 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a89aa790 2023-10-06T05:26:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.146+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89aa790 2023-10-06T05:26:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.147+0000 7f66d4901700 1 -- 172.21.15.45:0/681829574 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v130'192 uv192 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c400e7e0 con 0x55d8a89aa790 2023-10-06T05:26:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.147+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.148+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.148+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.150+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v130'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.150+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.151+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v130'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.153 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:26:50.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.151+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.152+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.152+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.153+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.153+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89a9e80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.155+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.155+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d8a89addb0 0x55d8a89b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.155+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b06d0 con 0x55d8a89addb0 2023-10-06T05:26:50.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.155+0000 7f66d4100700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d8a89addb0 0x55d8a89b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.155+0000 7f66d4100700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d8a89addb0 0x55d8a89b0190 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.157+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v130'376 uv369 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.157+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b06d0 con 0x55d8a89addb0 2023-10-06T05:26:50.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.158+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v130'377 uv377 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.158+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b06d0 con 0x55d8a89addb0 2023-10-06T05:26:50.161 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:26:50.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.159+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v130'378 uv378 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.160+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b00 con 0x55d8a89addb0 2023-10-06T05:26:50.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.161+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v130'379 uv379 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.161+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b00 con 0x55d8a89addb0 2023-10-06T05:26:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.162+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v130'380 uv380 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.162+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b00 con 0x55d8a89addb0 2023-10-06T05:26:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.163+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v130'381 uv375 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.163+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b00 con 0x55d8a89addb0 2023-10-06T05:26:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.164+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v130'382 uv382 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.165+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b00 con 0x55d8a89addb0 2023-10-06T05:26:50.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.166+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v130'383 uv383 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.168 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:26:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.166+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b00 con 0x55d8a89addb0 2023-10-06T05:26:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.167+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v130'384 uv384 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.167+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3870 con 0x55d8a89addb0 2023-10-06T05:26:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.168+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v130'385 uv385 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89addb0 2023-10-06T05:26:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.168+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0ff0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.169+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v130'167 uv165 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.169+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0ff0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.171+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v130'168 uv168 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.171+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0ff0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.172+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v130'169 uv169 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.174 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:26:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.172+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0ff0 con 0x55d8a89a1fe0 2023-10-06T05:26:50.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.173+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v130'170 uv170 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.173+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d8a89b1500 0x55d8a89b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.173+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89b1500 2023-10-06T05:26:50.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.174+0000 7f66d5102700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d8a89b1500 0x55d8a89b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.174+0000 7f66d5102700 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d8a89b1500 0x55d8a89b38e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.175+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89b1500 2023-10-06T05:26:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.175+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89b1500 2023-10-06T05:26:50.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.176+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v130'226 uv226 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89b1500 2023-10-06T05:26:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.177+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89b1500 2023-10-06T05:26:50.179 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:26:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.178+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v130'227 uv227 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89b1500 2023-10-06T05:26:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.178+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b4e30 con 0x55d8a89b1500 2023-10-06T05:26:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.179+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89b1500 2023-10-06T05:26:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.179+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b4e30 con 0x55d8a89b1500 2023-10-06T05:26:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.180+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89b1500 2023-10-06T05:26:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.180+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89b1500 2023-10-06T05:26:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.181+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89b1500 2023-10-06T05:26:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.181+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89a6460 2023-10-06T05:26:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.182+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v130'191 uv190 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.182+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89a6460 2023-10-06T05:26:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.184+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v130'192 uv192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.184+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89a6460 2023-10-06T05:26:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.185+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v130'193 uv193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.187 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:26:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.185+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b3e20 con 0x55d8a89a6460 2023-10-06T05:26:50.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.186+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v130'194 uv194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.186+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a6460 2023-10-06T05:26:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.187+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v130'195 uv195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f66cc009b70 con 0x55d8a89a6460 2023-10-06T05:26:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.188+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.189+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v130'468 uv459 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.189+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.190+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v130'469 uv469 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.190+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.193 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:26:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.191+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v130'470 uv470 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.191+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.193+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v130'471 uv471 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.193+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b4310 con 0x55d8a89a1fe0 2023-10-06T05:26:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.194+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v130'472 uv472 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.194+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b4310 con 0x55d8a89a1fe0 2023-10-06T05:26:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.195+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v130'473 uv466 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.195+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.196+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v130'474 uv474 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.196+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.197+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v130'475 uv475 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.199 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:26:50.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.197+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.198+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v130'476 uv476 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.198+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.199+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v130'477 uv477 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.199+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b0b80 con 0x55d8a89a1fe0 2023-10-06T05:26:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.200+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v130'478 uv478 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.200+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55d8a89b4310 con 0x55d8a89a1fe0 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66d4100700 1 -- 172.21.15.45:0/681829574 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v130'479 uv479 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f66c801a020 con 0x55d8a89a1fe0 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d8a89a1fe0 msgr2=0x55d8a89a43c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55d8a89a1fe0 0x55d8a89a43c0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d8a88eb210 msgr2=0x55d8a8983280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55d8a88eb210 0x55d8a8983280 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d8a89addb0 msgr2=0x55d8a89b0190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55d8a89addb0 0x55d8a89b0190 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d8a899d4c0 msgr2=0x55d8a899f8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55d8a899d4c0 0x55d8a899f8a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d8a89b1500 msgr2=0x55d8a89b38e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55d8a89b1500 0x55d8a89b38e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d8a89aa790 msgr2=0x55d8a89ad310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55d8a89aa790 0x55d8a89ad310 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 reap_dead start 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d8a89a6460 msgr2=0x55d8a89a9940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d8a89a6460 0x55d8a89a9940 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 reap_dead start 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66d5102700 1 -- 172.21.15.45:0/681829574 reap_dead start 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c0065890 msgr2=0x7f66c0067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c0065890 0x7f66c0067d40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x55d8a88ab3a0 tx=0x7f66c401c050 comp rx=0 tx=0).stop 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a89782e0 msgr2=0x55d8a88d6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a89782e0 0x55d8a88d6ff0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f66c80088f0 tx=0x7f66c8005d10 comp rx=0 tx=0).stop 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 shutdown_connections 2023-10-06T05:26:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55d8a89a6460 0x55d8a89a9940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8a89782e0 0x55d8a88d6ff0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 --2- 172.21.15.45:0/681829574 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f66c0065890 0x7f66c0067d40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 >> 172.21.15.45:0/681829574 conn(0x55d8a88a7bd0 msgr2=0x55d8a89825a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 shutdown_connections 2023-10-06T05:26:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.202+0000 7f66db6e3040 1 -- 172.21.15.45:0/681829574 wait complete. 2023-10-06T05:26:50.208 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:26:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.256+0000 7f573c4e8040 1 Processor -- start 2023-10-06T05:26:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f573c4e8040 1 -- start start 2023-10-06T05:26:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f573c4e8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f5890a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f573c4e8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f5890bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f573c4e8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f573c4e8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58901e320 con 0x55f589075210 2023-10-06T05:26:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f573c4e8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f588fc2980 con 0x55f589137890 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f573c4e8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f588fc2620 con 0x55f589137c60 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f5735706700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f5890a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f5735706700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f5890a61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53122/0 (socket says 172.21.15.45:53122) 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f5735706700 1 -- 172.21.15.45:0/54593029 learned_addr learned my addr 172.21.15.45:0/54593029 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.257+0000 7f5734f05700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f5890bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735f07700 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735706700 1 -- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 msgr2=0x55f5890bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735706700 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735706700 1 -- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 msgr2=0x55f5890bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735706700 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f5890bcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735706700 1 -- 172.21.15.45:0/54593029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720005cf0 con 0x55f589075210 2023-10-06T05:26:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735f07700 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f5735706700 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f5890a61b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f5728001eb0 tx=0x7f5728001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7d992a41b0cdba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f572f7fe700 1 -- 172.21.15.45:0/54593029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728014070 con 0x55f589075210 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f572f7fe700 1 -- 172.21.15.45:0/54593029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728008030 con 0x55f589075210 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.258+0000 7f572f7fe700 1 -- 172.21.15.45:0/54593029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572800be50 con 0x55f589075210 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 -- 172.21.15.45:0/54593029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 msgr2=0x55f5890a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f5890a61b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f5728001eb0 tx=0x7f5728001900 comp rx=0 tx=0).stop 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 -- 172.21.15.45:0/54593029 shutdown_connections 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f5890a61b0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 --2- 172.21.15.45:0/54593029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f5890bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 -- 172.21.15.45:0/54593029 >> 172.21.15.45:0/54593029 conn(0x55f589078450 msgr2=0x55f58907a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 -- 172.21.15.45:0/54593029 shutdown_connections 2023-10-06T05:26:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.259+0000 7f573c4e8040 1 -- 172.21.15.45:0/54593029 wait complete. 2023-10-06T05:26:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.260+0000 7f573c4e8040 1 Processor -- start 2023-10-06T05:26:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.260+0000 7f573c4e8040 1 -- start start 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f573c4e8040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f58915e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f573c4e8040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f589133770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f573c4e8040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f573c4e8040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f588fb9400 con 0x55f589075210 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f573c4e8040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58909b660 con 0x55f589137890 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f573c4e8040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58909b900 con 0x55f589137c60 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f58915e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f58915e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53130/0 (socket says 172.21.15.45:53130) 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 -- 172.21.15.45:0/1575213318 learned_addr learned my addr 172.21.15.45:0/1575213318 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5734f05700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f589133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735f07700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 -- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 msgr2=0x55f5890a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f589137c60 0x55f5890a61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 -- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 msgr2=0x55f589133770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f589133770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5735706700 1 -- 172.21.15.45:0/1575213318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572800a040 con 0x55f589075210 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.261+0000 7f5734f05700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f589137890 0x55f589133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.262+0000 7f5735706700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f58915e6e0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f572800b850 tx=0x7f5728016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.262+0000 7f572dffb700 1 -- 172.21.15.45:0/1575213318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280015e0 con 0x55f589075210 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.262+0000 7f572dffb700 1 -- 172.21.15.45:0/1575213318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728021070 con 0x55f589075210 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.262+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f588fc4ae0 con 0x55f589075210 2023-10-06T05:26:50.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.262+0000 7f572dffb700 1 -- 172.21.15.45:0/1575213318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728001360 con 0x55f589075210 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.262+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f588fc4540 con 0x55f589075210 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.263+0000 7f572dffb700 1 -- 172.21.15.45:0/1575213318 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f572801a480 con 0x55f589075210 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.264+0000 7f572dffb700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57180658c0 0x7f5718067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.264+0000 7f572dffb700 1 -- 172.21.15.45:0/1575213318 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7f57280858d0 con 0x55f589075210 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.264+0000 7f5734f05700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57180658c0 0x7f5718067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.264+0000 7f573c4e8040 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55f589133cb0 0x55f58909c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.264+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) v8 -- 0x55f5890a66f0 con 0x55f589133cb0 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.264+0000 7f5735f07700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55f589133cb0 0x55f58909c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.265+0000 7f5734f05700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57180658c0 0x7f5718067d70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5720005730 tx=0x7f5720016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.265+0000 7f5735f07700 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55f589133cb0 0x55f58909c2e0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f5735f07700 1 -- 172.21.15.45:0/1575213318 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v130'171 uv171 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f57300019f0 con 0x55f589133cb0 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55f589133cb0 msgr2=0x55f58909c2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55f589133cb0 0x55f58909c2e0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57180658c0 msgr2=0x7f5718067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f57180658c0 0x7f5718067d70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5720005730 tx=0x7f5720016000 comp rx=0 tx=0).stop 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 msgr2=0x55f58915e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 --2- 172.21.15.45:0/1575213318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f589075210 0x55f58915e6e0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f572800b850 tx=0x7f5728016000 comp rx=0 tx=0).stop 2023-10-06T05:26:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f5735f07700 1 -- 172.21.15.45:0/1575213318 reap_dead start 2023-10-06T05:26:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 shutdown_connections 2023-10-06T05:26:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.266+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 >> 172.21.15.45:0/1575213318 conn(0x55f589078450 msgr2=0x55f589152900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.267+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 shutdown_connections 2023-10-06T05:26:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.267+0000 7f573c4e8040 1 -- 172.21.15.45:0/1575213318 wait complete. 2023-10-06T05:26:50.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:26:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.408+0000 7f945c6c1700 1 Processor -- start 2023-10-06T05:26:50.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.408+0000 7f945c6c1700 1 -- start start 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f945c6c1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9454159520 0x7f945415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f945c6c1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f945415d3d0 0x7f9454067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f945c6c1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94540c9c90 con 0x7f9454159150 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f945c6c1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945415d8e0 con 0x7f945415d3d0 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f945c6c1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945415dbc0 con 0x7f9454159520 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f9459c5c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9454159520 0x7f945415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f945ac5e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f945415d3d0 0x7f9454067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f9459c5c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9454159520 0x7f945415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:48604/0 (socket says 172.21.15.45:48604) 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.409+0000 7f9459c5c700 1 -- 172.21.15.45:0/1414194346 learned_addr learned my addr 172.21.15.45:0/1414194346 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f9459c5c700 1 -- 172.21.15.45:0/1414194346 >> v2:172.21.15.144:3300/0 conn(0x7f945415d3d0 msgr2=0x7f9454067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:50.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f9459c5c700 1 --2- 172.21.15.45:0/1414194346 >> v2:172.21.15.144:3300/0 conn(0x7f945415d3d0 0x7f9454067620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f9459c5c700 1 -- 172.21.15.45:0/1414194346 >> v1:172.21.15.144:6789/0 conn(0x7f9454159150 legacy=0x7f94540ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f9459c5c700 1 -- 172.21.15.45:0/1414194346 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540c9920 con 0x7f9454159520 2023-10-06T05:26:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f9459c5c700 1 --2- 172.21.15.45:0/1414194346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9454159520 0x7f945415ce90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94500099b0 tx=0x7f9450009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf9117593f13d24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f945945b700 1 -- 172.21.15.45:0/1414194346 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450009ea0 con 0x7f9454159520 2023-10-06T05:26:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f945945b700 1 -- 172.21.15.45:0/1414194346 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945000b860 con 0x7f9454159520 2023-10-06T05:26:50.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.410+0000 7f945945b700 1 -- 172.21.15.45:0/1414194346 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000bc20 con 0x7f9454159520 2023-10-06T05:26:50.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 -- 172.21.15.45:0/1414194346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9454159520 msgr2=0x7f945415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 --2- 172.21.15.45:0/1414194346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9454159520 0x7f945415ce90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94500099b0 tx=0x7f9450009250 comp rx=0 tx=0).stop 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 -- 172.21.15.45:0/1414194346 shutdown_connections 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 --2- 172.21.15.45:0/1414194346 >> v2:172.21.15.144:3300/0 conn(0x7f945415d3d0 0x7f9454067620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 --2- 172.21.15.45:0/1414194346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9454159520 0x7f945415ce90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 -- 172.21.15.45:0/1414194346 >> 172.21.15.45:0/1414194346 conn(0x7f94540691c0 msgr2=0x7f94540cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 -- 172.21.15.45:0/1414194346 shutdown_connections 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.411+0000 7f945c6c1700 1 -- 172.21.15.45:0/1414194346 wait complete. 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.412+0000 7f945c6c1700 1 Processor -- start 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.412+0000 7f945c6c1700 1 -- start start 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945c6c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9454159150 0x7f9454155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945c6c1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f945415d3d0 0x7f94541698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945c6c1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f945415a5b0 0x7f945415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945c6c1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540ae640 con 0x7f945415d3d0 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945c6c1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945415af60 con 0x7f9454159150 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945c6c1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540b0f90 con 0x7f945415a5b0 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f9459c5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f945415d3d0 0x7f94541698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f9459c5c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f945415d3d0 0x7f94541698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53140/0 (socket says 172.21.15.45:53140) 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f9459c5c700 1 -- 172.21.15.45:0/4175811286 learned_addr learned my addr 172.21.15.45:0/4175811286 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945a45d700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9454159150 0x7f9454155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945ac5e700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f945415a5b0 0x7f945415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945a45d700 1 -- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f945415a5b0 msgr2=0x7f945415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945a45d700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f945415a5b0 0x7f945415aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945a45d700 1 -- 172.21.15.45:0/4175811286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f945415d3d0 msgr2=0x7f94541698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945a45d700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f945415d3d0 0x7f94541698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945a45d700 1 -- 172.21.15.45:0/4175811286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500094f0 con 0x7f9454159150 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f9459c5c700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f945415d3d0 0x7f94541698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.413+0000 7f945ac5e700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f945415a5b0 0x7f945415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.414+0000 7f945a45d700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9454159150 0x7f9454155c90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f944800c4c0 tx=0x7f9448007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.414+0000 7f94467fc700 1 -- 172.21.15.45:0/4175811286 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800ce30 con 0x7f9454159150 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.414+0000 7f94467fc700 1 -- 172.21.15.45:0/4175811286 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448011590 con 0x7f9454159150 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.414+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94540b1b10 con 0x7f9454159150 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.414+0000 7f94467fc700 1 -- 172.21.15.45:0/4175811286 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448011950 con 0x7f9454159150 2023-10-06T05:26:50.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.414+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94540b11e0 con 0x7f9454159150 2023-10-06T05:26:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.415+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945406e2d0 con 0x7f9454159150 2023-10-06T05:26:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.415+0000 7f94467fc700 1 -- 172.21.15.45:0/4175811286 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9448011d40 con 0x7f9454159150 2023-10-06T05:26:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.416+0000 7f94467fc700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9438066f30 0x7f94380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.416+0000 7f94467fc700 1 -- 172.21.15.45:0/4175811286 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9646+0+0 (secure 0 0 0) 0x7f9448084760 con 0x7f9454159150 2023-10-06T05:26:50.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.416+0000 7f9459c5c700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9438066f30 0x7f94380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:50.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.416+0000 7f9459c5c700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9438066f30 0x7f94380693e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9450009340 tx=0x7f9450017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.421+0000 7f94467fc700 1 -- 172.21.15.45:0/4175811286 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9448050a30 con 0x7f9454159150 2023-10-06T05:26:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.674+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f94540cdba0 con 0x7f9454159150 2023-10-06T05:26:50.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.913+0000 7f94467fc700 1 -- 172.21.15.45:0/4175811286 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v339) v1 ==== 162+0+0 (secure 0 0 0) 0x7f9448017070 con 0x7f9454159150 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9438066f30 msgr2=0x7f94380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9438066f30 0x7f94380693e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9450009340 tx=0x7f9450017000 comp rx=0 tx=0).stop 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9454159150 msgr2=0x7f9454155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9454159150 0x7f9454155c90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f944800c4c0 tx=0x7f9448007ec0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 shutdown_connections 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9438066f30 0x7f94380693e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f945415a5b0 0x7f945415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9454159150 0x7f9454155c90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 --2- 172.21.15.45:0/4175811286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f945415d3d0 0x7f94541698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 >> 172.21.15.45:0/4175811286 conn(0x7f94540691c0 msgr2=0x7f94541570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:50.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 shutdown_connections 2023-10-06T05:26:50.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:50.916+0000 7f945c6c1700 1 -- 172.21.15.45:0/4175811286 wait complete. 2023-10-06T05:26:50.920 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:50.919+0000 7f1012f96700 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input 2023-10-06T05:26:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:26:50.921 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:50.920+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:50.922 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:50.920+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:26:50.928 INFO:tasks.ceph.mds.b.smithi144.stderr: -26> 2023-10-06T05:26:50.919+0000 7f1012f96700 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input 2023-10-06T05:26:50.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:50.949 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:26:51.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.071+0000 7fd79a558700 1 Processor -- start 2023-10-06T05:26:51.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.071+0000 7fd79a558700 1 -- start start 2023-10-06T05:26:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd79a558700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd7940c8ff0 0x7fd7940c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd79a558700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7940caad0 0x7fd7940caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd79a558700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd794156af0 0x7fd794156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd79a558700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79415b000 con 0x7fd7940caad0 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd79a558700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79415b2e0 con 0x7fd7940c8ff0 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd79a558700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79415b5c0 con 0x7fd794156af0 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd7937fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7940caad0 0x7fd7940caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd7937fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7940caad0 0x7fd7940caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53146/0 (socket says 172.21.15.45:53146) 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd7937fe700 1 -- 172.21.15.45:0/2374717066 learned_addr learned my addr 172.21.15.45:0/2374717066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd793fff700 1 --2- 172.21.15.45:0/2374717066 >> v2:172.21.15.144:3300/0 conn(0x7fd7940c8ff0 0x7fd7940c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.072+0000 7fd798af5700 1 --2- 172.21.15.45:0/2374717066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd794156af0 0x7fd794156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd7937fe700 1 -- 172.21.15.45:0/2374717066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd794156af0 msgr2=0x7fd794156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd7937fe700 1 --2- 172.21.15.45:0/2374717066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd794156af0 0x7fd794156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd7937fe700 1 -- 172.21.15.45:0/2374717066 >> v2:172.21.15.144:3300/0 conn(0x7fd7940c8ff0 msgr2=0x7fd7940c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd7937fe700 1 --2- 172.21.15.45:0/2374717066 >> v2:172.21.15.144:3300/0 conn(0x7fd7940c8ff0 0x7fd7940c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd7937fe700 1 -- 172.21.15.45:0/2374717066 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79415b8a0 con 0x7fd7940caad0 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd7937fe700 1 --2- 172.21.15.45:0/2374717066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7940caf40 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fd7940cefc0 tx=0x7fd788009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322bc545e69a8eff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd792ffd700 1 -- 172.21.15.45:0/2374717066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800b640 con 0x7fd7940caad0 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd792ffd700 1 -- 172.21.15.45:0/2374717066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78800b7a0 con 0x7fd7940caad0 2023-10-06T05:26:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.073+0000 7fd792ffd700 1 -- 172.21.15.45:0/2374717066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800bb60 con 0x7fd7940caad0 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 -- 172.21.15.45:0/2374717066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 msgr2=0x7fd7940caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 --2- 172.21.15.45:0/2374717066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7940caf40 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fd7940cefc0 tx=0x7fd788009250 comp rx=0 tx=0).stop 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 -- 172.21.15.45:0/2374717066 shutdown_connections 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 --2- 172.21.15.45:0/2374717066 >> v2:172.21.15.144:3300/0 conn(0x7fd7940c8ff0 0x7fd7940c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 --2- 172.21.15.45:0/2374717066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd794156af0 0x7fd794156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 --2- 172.21.15.45:0/2374717066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7940caf40 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 -- 172.21.15.45:0/2374717066 >> 172.21.15.45:0/2374717066 conn(0x7fd7940cdf50 msgr2=0x7fd7940ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 -- 172.21.15.45:0/2374717066 shutdown_connections 2023-10-06T05:26:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.074+0000 7fd79a558700 1 -- 172.21.15.45:0/2374717066 wait complete. 2023-10-06T05:26:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.075+0000 7fd79a558700 1 Processor -- start 2023-10-06T05:26:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.075+0000 7fd79a558700 1 -- start start 2023-10-06T05:26:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd79a558700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7940c8ff0 0x7fd794156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd79a558700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7941503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd79a558700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd794156af0 0x7fd7940cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd79a558700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b30b0 con 0x7fd7940caad0 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd79a558700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941575e0 con 0x7fd794156af0 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd79a558700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941578c0 con 0x7fd7940c8ff0 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7941503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7941503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53160/0 (socket says 172.21.15.45:53160) 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 -- 172.21.15.45:0/3194890309 learned_addr learned my addr 172.21.15.45:0/3194890309 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd798af5700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd794156af0 0x7fd7940cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd793fff700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7940c8ff0 0x7fd794156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 -- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7940c8ff0 msgr2=0x7fd794156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7940c8ff0 0x7fd794156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 -- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd794156af0 msgr2=0x7fd7940cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd794156af0 0x7fd7940cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.076+0000 7fd7937fe700 1 -- 172.21.15.45:0/3194890309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880094f0 con 0x7fd7940caad0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.077+0000 7fd7937fe700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7941503f0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fd7940c4f80 tx=0x7fd788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.077+0000 7fd77bfff700 1 -- 172.21.15.45:0/3194890309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800b910 con 0x7fd7940caad0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.077+0000 7fd77bfff700 1 -- 172.21.15.45:0/3194890309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78800ba70 con 0x7fd7940caad0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.077+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd794157ba0 con 0x7fd7940caad0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.077+0000 7fd77bfff700 1 -- 172.21.15.45:0/3194890309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788011630 con 0x7fd7940caad0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.077+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd794167a40 con 0x7fd7940caad0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.078+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79406f410 con 0x7fd7940caad0 2023-10-06T05:26:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.078+0000 7fd77bfff700 1 -- 172.21.15.45:0/3194890309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7880119a0 con 0x7fd7940caad0 2023-10-06T05:26:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.079+0000 7fd77bfff700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd780066f30 0x7fd7800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.079+0000 7fd77bfff700 1 -- 172.21.15.45:0/3194890309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7fd788086710 con 0x7fd7940caad0 2023-10-06T05:26:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.079+0000 7fd793fff700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd780066f30 0x7fd7800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.079+0000 7fd793fff700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd780066f30 0x7fd7800693e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd784009db0 tx=0x7fd7840091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.084+0000 7fd77bfff700 1 -- 172.21.15.45:0/3194890309 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd788052ab0 con 0x7fd7940caad0 2023-10-06T05:26:51.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.331+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd79404bb40 con 0x7fd7940caad0 2023-10-06T05:26:51.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.332+0000 7fd77bfff700 1 -- 172.21.15.45:0/3194890309 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 340 v340) v1 ==== 95+0+3839 (secure 0 0 0) 0x7fd788015090 con 0x7fd7940caad0 2023-10-06T05:26:51.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:51.334 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":340,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:50.920923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:51.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd780066f30 msgr2=0x7fd7800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd780066f30 0x7fd7800693e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd784009db0 tx=0x7fd7840091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 msgr2=0x7fd7941503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7941503f0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fd7940c4f80 tx=0x7fd788017000 comp rx=0 tx=0).stop 2023-10-06T05:26:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 shutdown_connections 2023-10-06T05:26:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd780066f30 0x7fd7800693e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd794156af0 0x7fd7940cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7940c8ff0 0x7fd794156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 --2- 172.21.15.45:0/3194890309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7940caad0 0x7fd7941503f0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 >> 172.21.15.45:0/3194890309 conn(0x7fd7940cdf50 msgr2=0x7fd7940aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 shutdown_connections 2023-10-06T05:26:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.334+0000 7fd79a558700 1 -- 172.21.15.45:0/3194890309 wait complete. 2023-10-06T05:26:51.338 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 340 2023-10-06T05:26:51.349 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:26:51.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:51.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.479+0000 7fbe7592d700 1 Processor -- start 2023-10-06T05:26:51.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.479+0000 7fbe7592d700 1 -- start start 2023-10-06T05:26:51.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe7592d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe70159170 0x7fbe70159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe7592d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe700692d0 0x7fbe700aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe7592d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe700ca6d0 con 0x7fbe7015d040 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe7592d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7015d550 con 0x7fbe70159170 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe7592d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7015d810 con 0x7fbe700692d0 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe6effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe70159170 0x7fbe70159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe6e7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbe7015d040 0x7fbe70068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34106/0 (socket says 172.21.15.45:34106) 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe6e7fc700 1 -- 172.21.15.45:0/2403070544 learned_addr learned my addr 172.21.15.45:0/2403070544 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.480+0000 7fbe6f7fe700 1 --2- 172.21.15.45:0/2403070544 >> v2:172.21.15.144:3300/0 conn(0x7fbe700692d0 0x7fbe700aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6effd700 1 -- 172.21.15.45:0/2403070544 >> v2:172.21.15.144:3300/0 conn(0x7fbe700692d0 msgr2=0x7fbe700aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6effd700 1 --2- 172.21.15.45:0/2403070544 >> v2:172.21.15.144:3300/0 conn(0x7fbe700692d0 0x7fbe700aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6effd700 1 -- 172.21.15.45:0/2403070544 >> v1:172.21.15.45:6789/0 conn(0x7fbe7015d040 legacy=0x7fbe70068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6effd700 1 -- 172.21.15.45:0/2403070544 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7015daf0 con 0x7fbe70159170 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6dffb700 1 -- 172.21.15.45:0/2403070544 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728433007 0 0) 0x7fbe700ca6d0 con 0x7fbe7015d040 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6effd700 1 --2- 172.21.15.45:0/2403070544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe70159170 0x7fbe70159540 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fbe700c70c0 tx=0x7fbe600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4a31da70f898a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6dffb700 1 -- 172.21.15.45:0/2403070544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60011630 con 0x7fbe70159170 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6dffb700 1 -- 172.21.15.45:0/2403070544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe60011790 con 0x7fbe70159170 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.481+0000 7fbe6dffb700 1 -- 172.21.15.45:0/2403070544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60011b30 con 0x7fbe70159170 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 -- 172.21.15.45:0/2403070544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe70159170 msgr2=0x7fbe70159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2403070544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe70159170 0x7fbe70159540 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fbe700c70c0 tx=0x7fbe600091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 -- 172.21.15.45:0/2403070544 shutdown_connections 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2403070544 >> v2:172.21.15.144:3300/0 conn(0x7fbe700692d0 0x7fbe700aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2403070544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe70159170 0x7fbe70159540 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 -- 172.21.15.45:0/2403070544 >> 172.21.15.45:0/2403070544 conn(0x7fbe7006f7c0 msgr2=0x7fbe700d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 -- 172.21.15.45:0/2403070544 shutdown_connections 2023-10-06T05:26:51.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.482+0000 7fbe7592d700 1 -- 172.21.15.45:0/2403070544 wait complete. 2023-10-06T05:26:51.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.483+0000 7fbe7592d700 1 Processor -- start 2023-10-06T05:26:51.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.483+0000 7fbe7592d700 1 -- start start 2023-10-06T05:26:51.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe7592d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe700692d0 0x7fbe700c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe7592d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe70159170 0x7fbe700a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe7592d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe7015d040 0x7fbe700cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe7592d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70169a90 con 0x7fbe7015d040 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe7592d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70169d70 con 0x7fbe70159170 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe7592d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7016a050 con 0x7fbe700692d0 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe7015d040 0x7fbe700cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe7015d040 0x7fbe700cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53168/0 (socket says 172.21.15.45:53168) 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 -- 172.21.15.45:0/2800464674 learned_addr learned my addr 172.21.15.45:0/2800464674 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6effd700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe700692d0 0x7fbe700c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6e7fc700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe70159170 0x7fbe700a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 -- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe700692d0 msgr2=0x7fbe700c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe700692d0 0x7fbe700c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 -- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe70159170 msgr2=0x7fbe700a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe70159170 0x7fbe700a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6f7fe700 1 -- 172.21.15.45:0/2800464674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe60009420 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.484+0000 7fbe6e7fc700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe70159170 0x7fbe700a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.485+0000 7fbe6f7fe700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe7015d040 0x7fbe700cbb30 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009e90 tx=0x7fbe6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.485+0000 7fbe56ffd700 1 -- 172.21.15.45:0/2800464674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800cc20 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.485+0000 7fbe56ffd700 1 -- 172.21.15.45:0/2800464674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6800f040 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.485+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7016a330 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.485+0000 7fbe56ffd700 1 -- 172.21.15.45:0/2800464674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68014480 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.485+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe70169740 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.486+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7016cd20 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.486+0000 7fbe56ffd700 1 -- 172.21.15.45:0/2800464674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe68014830 con 0x7fbe7015d040 2023-10-06T05:26:51.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.487+0000 7fbe56ffd700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe5c066f30 0x7fbe5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.487+0000 7fbe56ffd700 1 -- 172.21.15.45:0/2800464674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7fbe68086260 con 0x7fbe7015d040 2023-10-06T05:26:51.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.487+0000 7fbe6effd700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe5c066f30 0x7fbe5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.487+0000 7fbe6effd700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe5c066f30 0x7fbe5c0693e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbe600093f0 tx=0x7fbe60017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.491+0000 7fbe56ffd700 1 -- 172.21.15.45:0/2800464674 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe7016cd20 con 0x7fbe7015d040 2023-10-06T05:26:51.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.742+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe7016cd20 con 0x7fbe7015d040 2023-10-06T05:26:51.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.742+0000 7fbe56ffd700 1 -- 172.21.15.45:0/2800464674 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 340 v340) v1 ==== 95+0+3839 (secure 0 0 0) 0x7fbe7016cd20 con 0x7fbe7015d040 2023-10-06T05:26:51.744 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:51.744 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":340,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:50.920923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.744+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe5c066f30 msgr2=0x7fbe5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe5c066f30 0x7fbe5c0693e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbe600093f0 tx=0x7fbe60017000 comp rx=0 tx=0).stop 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe7015d040 msgr2=0x7fbe700cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe7015d040 0x7fbe700cbb30 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009e90 tx=0x7fbe6800b040 comp rx=0 tx=0).stop 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 shutdown_connections 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe5c066f30 0x7fbe5c0693e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe7015d040 0x7fbe700cbb30 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe700692d0 0x7fbe700c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 --2- 172.21.15.45:0/2800464674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe70159170 0x7fbe700a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 >> 172.21.15.45:0/2800464674 conn(0x7fbe7006f7c0 msgr2=0x7fbe700b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 shutdown_connections 2023-10-06T05:26:51.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.745+0000 7fbe7592d700 1 -- 172.21.15.45:0/2800464674 wait complete. 2023-10-06T05:26:51.748 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 340 2023-10-06T05:26:51.760 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:26:51.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:51.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.890+0000 7f724c357700 1 Processor -- start 2023-10-06T05:26:51.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.890+0000 7f724c357700 1 -- start start 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.891+0000 7f724c357700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f72441539f0 0x7f724415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.891+0000 7f724c357700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72440ccfc0 0x7f724415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.891+0000 7f724c357700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72440ce690 con 0x7f7244153620 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.891+0000 7f724c357700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724415a7a0 con 0x7f72440ccfc0 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.891+0000 7f724c357700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72440ce3f0 con 0x7f72441539f0 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.891+0000 7f724a0f3700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7244153620 0x7f72440cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34108/0 (socket says 172.21.15.45:34108) 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.891+0000 7f724a0f3700 1 -- 172.21.15.45:0/4193041465 learned_addr learned my addr 172.21.15.45:0/4193041465 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:51.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f724a8f4700 1 --2- 172.21.15.45:0/4193041465 >> v2:172.21.15.45:3300/0 conn(0x7f72440ccfc0 0x7f724415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f72498f2700 1 --2- 172.21.15.45:0/4193041465 >> v2:172.21.15.144:3300/0 conn(0x7f72441539f0 0x7f724415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f72490f1700 1 -- 172.21.15.45:0/4193041465 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398107473 0 0) 0x7f72440ce690 con 0x7f7244153620 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f72490f1700 1 -- 172.21.15.45:0/4193041465 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7230003570 con 0x7f7244153620 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f724a8f4700 1 -- 172.21.15.45:0/4193041465 >> v2:172.21.15.144:3300/0 conn(0x7f72441539f0 msgr2=0x7f724415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f724a8f4700 1 --2- 172.21.15.45:0/4193041465 >> v2:172.21.15.144:3300/0 conn(0x7f72441539f0 0x7f724415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f724a8f4700 1 -- 172.21.15.45:0/4193041465 >> v1:172.21.15.45:6789/0 conn(0x7f7244153620 legacy=0x7f72440cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f724a8f4700 1 -- 172.21.15.45:0/4193041465 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724415d510 con 0x7f72440ccfc0 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f72498f2700 1 --2- 172.21.15.45:0/4193041465 >> v2:172.21.15.144:3300/0 conn(0x7f72441539f0 0x7f724415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f724a8f4700 1 --2- 172.21.15.45:0/4193041465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72440ccfc0 0x7f724415cdc0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f7244067840 tx=0x7f723c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e48274aad15a19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f72490f1700 1 -- 172.21.15.45:0/4193041465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00e070 con 0x7f72440ccfc0 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f72490f1700 1 -- 172.21.15.45:0/4193041465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c00bb60 con 0x7f72440ccfc0 2023-10-06T05:26:51.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.892+0000 7f72490f1700 1 -- 172.21.15.45:0/4193041465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c012430 con 0x7f72440ccfc0 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 -- 172.21.15.45:0/4193041465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72440ccfc0 msgr2=0x7f724415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 --2- 172.21.15.45:0/4193041465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72440ccfc0 0x7f724415cdc0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f7244067840 tx=0x7f723c009780 comp rx=0 tx=0).stop 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 -- 172.21.15.45:0/4193041465 shutdown_connections 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 --2- 172.21.15.45:0/4193041465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72440ccfc0 0x7f724415cdc0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 --2- 172.21.15.45:0/4193041465 >> v2:172.21.15.144:3300/0 conn(0x7f72441539f0 0x7f724415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 -- 172.21.15.45:0/4193041465 >> 172.21.15.45:0/4193041465 conn(0x7f724406ed50 msgr2=0x7f724406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 -- 172.21.15.45:0/4193041465 shutdown_connections 2023-10-06T05:26:51.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.893+0000 7f724c357700 1 -- 172.21.15.45:0/4193041465 wait complete. 2023-10-06T05:26:51.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.894+0000 7f724c357700 1 Processor -- start 2023-10-06T05:26:51.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.894+0000 7f724c357700 1 -- start start 2023-10-06T05:26:51.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724c357700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72440ccfc0 0x7f72440c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724c357700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7244153620 0x7f72440d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724c357700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72440cf3f0 0x7f72440b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724c357700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440b2f90 con 0x7f7244153620 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724c357700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440cf880 con 0x7f72440cf3f0 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724c357700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72440b3230 con 0x7f72440ccfc0 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f72498f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7244153620 0x7f72440d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f72498f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7244153620 0x7f72440d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53192/0 (socket says 172.21.15.45:53192) 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f72498f2700 1 -- 172.21.15.45:0/1823690108 learned_addr learned my addr 172.21.15.45:0/1823690108 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724a8f4700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72440cf3f0 0x7f72440b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.895+0000 7f724a0f3700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72440ccfc0 0x7f72440c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72498f2700 1 -- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72440ccfc0 msgr2=0x7f72440c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72498f2700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72440ccfc0 0x7f72440c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72498f2700 1 -- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72440cf3f0 msgr2=0x7f72440b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:51.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72498f2700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72440cf3f0 0x7f72440b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72498f2700 1 -- 172.21.15.45:0/1823690108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c009a20 con 0x7f7244153620 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f724a8f4700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72440cf3f0 0x7f72440b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f724a0f3700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72440ccfc0 0x7f72440c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72498f2700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7244153620 0x7f72440d0260 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f723800d890 tx=0x7f7238008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72367fc700 1 -- 172.21.15.45:0/1823690108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72380125f0 con 0x7f7244153620 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72367fc700 1 -- 172.21.15.45:0/1823690108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7238012750 con 0x7f7244153620 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72440b3510 con 0x7f7244153620 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f72367fc700 1 -- 172.21.15.45:0/1823690108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238012af0 con 0x7f7244153620 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.896+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72440c67c0 con 0x7f7244153620 2023-10-06T05:26:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.897+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7244088a60 con 0x7f7244153620 2023-10-06T05:26:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.897+0000 7f72367fc700 1 -- 172.21.15.45:0/1823690108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7238014430 con 0x7f7244153620 2023-10-06T05:26:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.898+0000 7f72367fc700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7228066f30 0x7f72280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.898+0000 7f72367fc700 1 -- 172.21.15.45:0/1823690108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7f7238085a40 con 0x7f7244153620 2023-10-06T05:26:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.898+0000 7f724a0f3700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7228066f30 0x7f72280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:51.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.899+0000 7f724a0f3700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7228066f30 0x7f72280693e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f72400058d0 tx=0x7f7240009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:51.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:51.902+0000 7f72367fc700 1 -- 172.21.15.45:0/1823690108 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7238087e60 con 0x7f7244153620 2023-10-06T05:26:52.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.153+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f724404bb40 con 0x7f7244153620 2023-10-06T05:26:52.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.154+0000 7f72367fc700 1 -- 172.21.15.45:0/1823690108 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) v1 ==== 95+0+4598 (secure 0 0 0) 0x7f7238051cb0 con 0x7f7244153620 2023-10-06T05:26:52.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:52.156 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":341,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/913913762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":913913762},{"type":"v1","addr":"172.21.15.144:6835","nonce":913913762}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":341}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:50.920923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7228066f30 msgr2=0x7f72280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7228066f30 0x7f72280693e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f72400058d0 tx=0x7f7240009040 comp rx=0 tx=0).stop 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7244153620 msgr2=0x7f72440d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7244153620 0x7f72440d0260 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f723800d890 tx=0x7f7238008040 comp rx=0 tx=0).stop 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 shutdown_connections 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7228066f30 0x7f72280693e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72440cf3f0 0x7f72440b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72440ccfc0 0x7f72440c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 --2- 172.21.15.45:0/1823690108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7244153620 0x7f72440d0260 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 >> 172.21.15.45:0/1823690108 conn(0x7f724406ed50 msgr2=0x7f7244151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 shutdown_connections 2023-10-06T05:26:52.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.156+0000 7f724c357700 1 -- 172.21.15.45:0/1823690108 wait complete. 2023-10-06T05:26:52.159 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 341 2023-10-06T05:26:52.171 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2023-10-06T05:26:52.171 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2023-10-06T05:26:52.171 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi045.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:52.171 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:26:52.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:52.171 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:52.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:26:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:26:52.191 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.365+0000 7fc8cdb04700 1 Processor -- start 2023-10-06T05:26:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.365+0000 7fc8cdb04700 1 -- start start 2023-10-06T05:26:52.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8cdb04700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc8c8155cd0 0x7fc8c815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8cdb04700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8c81560a0 0x7fc8c80adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8cdb04700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c80ae8d0 con 0x7fc8c80ccdc0 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8cdb04700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c80aebb0 con 0x7fc8c81560a0 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8cdb04700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c80aee90 con 0x7fc8c8155cd0 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8c81560a0 0x7fc8c80adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8c81560a0 0x7fc8c80adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53208/0 (socket says 172.21.15.45:53208) 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/1376626218 learned_addr learned my addr 172.21.15.45:0/1376626218 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c77fe700 1 --2- 172.21.15.45:0/1376626218 >> v2:172.21.15.144:3300/0 conn(0x7fc8c8155cd0 0x7fc8c815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/1376626218 >> v2:172.21.15.144:3300/0 conn(0x7fc8c8155cd0 msgr2=0x7fc8c815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 --2- 172.21.15.45:0/1376626218 >> v2:172.21.15.144:3300/0 conn(0x7fc8c8155cd0 0x7fc8c815a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/1376626218 >> v1:172.21.15.144:6789/0 conn(0x7fc8c80ccdc0 legacy=0x7fc8c80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/1376626218 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80af170 con 0x7fc8c81560a0 2023-10-06T05:26:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.366+0000 7fc8c6ffd700 1 --2- 172.21.15.45:0/1376626218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c81560a0 0x7fc8c80adfa0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806ae50 tx=0x7fc8b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473cf7494a2b25c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8c67fc700 1 -- 172.21.15.45:0/1376626218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0009e70 con 0x7fc8c81560a0 2023-10-06T05:26:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8c67fc700 1 -- 172.21.15.45:0/1376626218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b000b860 con 0x7fc8c81560a0 2023-10-06T05:26:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8c67fc700 1 -- 172.21.15.45:0/1376626218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000bc00 con 0x7fc8c81560a0 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 -- 172.21.15.45:0/1376626218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c81560a0 msgr2=0x7fc8c80adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/1376626218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c81560a0 0x7fc8c80adfa0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806ae50 tx=0x7fc8b0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 -- 172.21.15.45:0/1376626218 shutdown_connections 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/1376626218 >> v2:172.21.15.144:3300/0 conn(0x7fc8c8155cd0 0x7fc8c815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/1376626218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c81560a0 0x7fc8c80adfa0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 -- 172.21.15.45:0/1376626218 >> 172.21.15.45:0/1376626218 conn(0x7fc8c806f090 msgr2=0x7fc8c806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 -- 172.21.15.45:0/1376626218 shutdown_connections 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.367+0000 7fc8cdb04700 1 -- 172.21.15.45:0/1376626218 wait complete. 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.369+0000 7fc8cdb04700 1 Processor -- start 2023-10-06T05:26:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.369+0000 7fc8cdb04700 1 -- start start 2023-10-06T05:26:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.369+0000 7fc8cdb04700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ccdc0 0x7fc8c80b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.369+0000 7fc8cdb04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c8155cd0 0x7fc8c81545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8cdb04700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8c81560a0 0x7fc8c81514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8cdb04700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c3fe0 con 0x7fc8c8155cd0 2023-10-06T05:26:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8cdb04700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c4280 con 0x7fc8c80ccdc0 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8cdb04700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c4560 con 0x7fc8c81560a0 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c8155cd0 0x7fc8c81545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c8155cd0 0x7fc8c81545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53214/0 (socket says 172.21.15.45:53214) 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/892377256 learned_addr learned my addr 172.21.15.45:0/892377256 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c77fe700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ccdc0 0x7fc8c80b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c7fff700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8c81560a0 0x7fc8c81514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8c81560a0 msgr2=0x7fc8c81514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8c81560a0 0x7fc8c81514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ccdc0 msgr2=0x7fc8c80b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ccdc0 0x7fc8c80b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 -- 172.21.15.45:0/892377256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b00094f0 con 0x7fc8c8155cd0 2023-10-06T05:26:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c7fff700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8c81560a0 0x7fc8c81514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8c6ffd700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c8155cd0 0x7fc8c81545e0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806aed0 tx=0x7fc8b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8bf7fe700 1 -- 172.21.15.45:0/892377256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000b9d0 con 0x7fc8c8155cd0 2023-10-06T05:26:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8bf7fe700 1 -- 172.21.15.45:0/892377256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b000bb30 con 0x7fc8c8155cd0 2023-10-06T05:26:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c80c4840 con 0x7fc8c8155cd0 2023-10-06T05:26:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8bf7fe700 1 -- 172.21.15.45:0/892377256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0025670 con 0x7fc8c8155cd0 2023-10-06T05:26:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.370+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c80c5040 con 0x7fc8c8155cd0 2023-10-06T05:26:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.371+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c80c4d60 con 0x7fc8c8155cd0 2023-10-06T05:26:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.372+0000 7fc8bf7fe700 1 -- 172.21.15.45:0/892377256 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc8b0023070 con 0x7fc8c8155cd0 2023-10-06T05:26:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.372+0000 7fc8bf7fe700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc8a8066f30 0x7fc8a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.372+0000 7fc8bf7fe700 1 -- 172.21.15.45:0/892377256 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7fc8b0086970 con 0x7fc8c8155cd0 2023-10-06T05:26:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.373+0000 7fc8c77fe700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc8a8066f30 0x7fc8a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.373+0000 7fc8c77fe700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc8a8066f30 0x7fc8a80693e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009db0 tx=0x7fc8b80091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:52.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.377+0000 7fc8bf7fe700 1 -- 172.21.15.45:0/892377256 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8c80c4d60 con 0x7fc8c8155cd0 2023-10-06T05:26:52.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.627+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc8c80c4d60 con 0x7fc8c8155cd0 2023-10-06T05:26:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.975+0000 7fc8bf7fe700 1 -- 172.21.15.45:0/892377256 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v342) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc8c80c4d60 con 0x7fc8c8155cd0 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc8a8066f30 msgr2=0x7fc8a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc8a8066f30 0x7fc8a80693e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009db0 tx=0x7fc8b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c8155cd0 msgr2=0x7fc8c81545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c8155cd0 0x7fc8c81545e0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fc8c806aed0 tx=0x7fc8b0017000 comp rx=0 tx=0).stop 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 shutdown_connections 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc8a8066f30 0x7fc8a80693e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8c81560a0 0x7fc8c81514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8c80ccdc0 0x7fc8c80b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 --2- 172.21.15.45:0/892377256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c8155cd0 0x7fc8c81545e0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:52.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 >> 172.21.15.45:0/892377256 conn(0x7fc8c806f090 msgr2=0x7fc8c80ca9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:52.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.978+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 shutdown_connections 2023-10-06T05:26:52.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:52.979+0000 7fc8cdb04700 1 -- 172.21.15.45:0/892377256 wait complete. 2023-10-06T05:26:52.982 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:52.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-06T05:26:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.131+0000 7fce4fee8700 1 Processor -- start 2023-10-06T05:26:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.132+0000 7fce4fee8700 1 -- start start 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.132+0000 7fce4fee8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fce480c93c0 0x7fce48156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.132+0000 7fce4fee8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4815b000 con 0x7fce48157030 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.132+0000 7fce4fee8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4815b2e0 con 0x7fce480c8ff0 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.132+0000 7fce4fee8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce480c9830 con 0x7fce480c93c0 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4e485700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fce48157030 0x7fce480cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34124/0 (socket says 172.21.15.45:34124) 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4e485700 1 -- 172.21.15.45:0/662856684 learned_addr learned my addr 172.21.15.45:0/662856684 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4d483700 1 --2- 172.21.15.45:0/662856684 >> v2:172.21.15.144:3300/0 conn(0x7fce480c93c0 0x7fce48156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4cc82700 1 -- 172.21.15.45:0/662856684 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401704839 0 0) 0x7fce4815b000 con 0x7fce48157030 2023-10-06T05:26:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4cc82700 1 -- 172.21.15.45:0/662856684 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce34003570 con 0x7fce48157030 2023-10-06T05:26:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4cc82700 1 -- 172.21.15.45:0/662856684 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574719294 0 0) 0x7fce4815b2e0 con 0x7fce480c8ff0 2023-10-06T05:26:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4cc82700 1 -- 172.21.15.45:0/662856684 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce4815b000 con 0x7fce480c8ff0 2023-10-06T05:26:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4d483700 1 -- 172.21.15.45:0/662856684 >> v1:172.21.15.144:6789/0 conn(0x7fce480c8ff0 legacy=0x7fce480caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4d483700 1 -- 172.21.15.45:0/662856684 >> v1:172.21.15.45:6789/0 conn(0x7fce48157030 legacy=0x7fce480cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4d483700 1 -- 172.21.15.45:0/662856684 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4815b5c0 con 0x7fce480c93c0 2023-10-06T05:26:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.133+0000 7fce4d483700 1 --2- 172.21.15.45:0/662856684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce48156af0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0098e0 tx=0x7fce3c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11acc9b0872b897 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4cc82700 1 -- 172.21.15.45:0/662856684 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c011630 con 0x7fce480c93c0 2023-10-06T05:26:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4cc82700 1 -- 172.21.15.45:0/662856684 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c011790 con 0x7fce480c93c0 2023-10-06T05:26:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4cc82700 1 -- 172.21.15.45:0/662856684 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c011b30 con 0x7fce480c93c0 2023-10-06T05:26:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4fee8700 1 -- 172.21.15.45:0/662856684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 msgr2=0x7fce48156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4fee8700 1 --2- 172.21.15.45:0/662856684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce48156af0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0098e0 tx=0x7fce3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4fee8700 1 -- 172.21.15.45:0/662856684 shutdown_connections 2023-10-06T05:26:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4fee8700 1 --2- 172.21.15.45:0/662856684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce48156af0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4fee8700 1 -- 172.21.15.45:0/662856684 >> 172.21.15.45:0/662856684 conn(0x7fce480cdf50 msgr2=0x7fce480ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4fee8700 1 -- 172.21.15.45:0/662856684 shutdown_connections 2023-10-06T05:26:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.134+0000 7fce4fee8700 1 -- 172.21.15.45:0/662856684 wait complete. 2023-10-06T05:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.136+0000 7fce4fee8700 1 Processor -- start 2023-10-06T05:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.136+0000 7fce4fee8700 1 -- start start 2023-10-06T05:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4fee8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce480c8ff0 0x7fce48159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4fee8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce480b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4fee8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce48157030 0x7fce4814f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4fee8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4815a360 con 0x7fce480c8ff0 2023-10-06T05:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4fee8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce48159990 con 0x7fce480c93c0 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4fee8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480b1d00 con 0x7fce48157030 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4dc84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce480c8ff0 0x7fce48159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4dc84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce480c8ff0 0x7fce48159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53226/0 (socket says 172.21.15.45:53226) 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4dc84700 1 -- 172.21.15.45:0/1609151150 learned_addr learned my addr 172.21.15.45:0/1609151150 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4e485700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce48157030 0x7fce4814f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4d483700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce480b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4dc84700 1 -- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce48157030 msgr2=0x7fce4814f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4dc84700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce48157030 0x7fce4814f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.137+0000 7fce4dc84700 1 -- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 msgr2=0x7fce480b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce4dc84700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce480b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce4dc84700 1 -- 172.21.15.45:0/1609151150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c009420 con 0x7fce480c8ff0 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce4d483700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce480b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce4e485700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce48157030 0x7fce4814f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce4dc84700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce480c8ff0 0x7fce48159e20 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fce4400c580 tx=0x7fce44007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce39ffb700 1 -- 172.21.15.45:0/1609151150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44010510 con 0x7fce480c8ff0 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce39ffb700 1 -- 172.21.15.45:0/1609151150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce44010670 con 0x7fce480c8ff0 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce48158d80 con 0x7fce480c8ff0 2023-10-06T05:26:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce39ffb700 1 -- 172.21.15.45:0/1609151150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44010a30 con 0x7fce480c8ff0 2023-10-06T05:26:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.138+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce48159340 con 0x7fce480c8ff0 2023-10-06T05:26:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.139+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4806f410 con 0x7fce480c8ff0 2023-10-06T05:26:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.140+0000 7fce39ffb700 1 -- 172.21.15.45:0/1609151150 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce44025430 con 0x7fce480c8ff0 2023-10-06T05:26:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.140+0000 7fce39ffb700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.141+0000 7fce39ffb700 1 -- 172.21.15.45:0/1609151150 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7fce44085d90 con 0x7fce480c8ff0 2023-10-06T05:26:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.141+0000 7fce4d483700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.141+0000 7fce4d483700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0093f0 tx=0x7fce3c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.145+0000 7fce39ffb700 1 -- 172.21.15.45:0/1609151150 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce44051fa0 con 0x7fce480c8ff0 2023-10-06T05:26:53.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.433+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fce4804bb40 con 0x7fce480c8ff0 2023-10-06T05:26:53.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.983+0000 7fce39ffb700 1 -- 172.21.15.45:0/1609151150 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v343) v1 ==== 101+0+0 (secure 0 0 0) 0x7fce440555c0 con 0x7fce480c8ff0 2023-10-06T05:26:53.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 msgr2=0x7fce2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0093f0 tx=0x7fce3c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:53.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce480c8ff0 msgr2=0x7fce48159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:53.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce480c8ff0 0x7fce48159e20 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fce4400c580 tx=0x7fce44007e10 comp rx=0 tx=0).stop 2023-10-06T05:26:53.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 shutdown_connections 2023-10-06T05:26:53.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fce2c066f30 0x7fce2c0693e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce48157030 0x7fce4814f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce480c8ff0 0x7fce48159e20 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 --2- 172.21.15.45:0/1609151150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce480c93c0 0x7fce480b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:53.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 >> 172.21.15.45:0/1609151150 conn(0x7fce480cdf50 msgr2=0x7fce48150370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:53.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 shutdown_connections 2023-10-06T05:26:53.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:53.986+0000 7fce4fee8700 1 -- 172.21.15.45:0/1609151150 wait complete. 2023-10-06T05:26:53.989 INFO:teuthology.orchestra.run.smithi045.stderr:repaired: restoring rank 6:0 2023-10-06T05:26:54.003 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2023-10-06T05:26:54.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.048+0000 7f975d9c0040 1 Processor -- start 2023-10-06T05:26:54.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.048+0000 7f975d9c0040 1 -- start start 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f975d9c0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82ad1cce0 0x55c82adc4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f975d9c0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 0x55c82ad35df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f975d9c0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82adc9610 0x55c82ad36330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f975d9c0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c82ac3a980 con 0x55c82adc9610 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f975d9c0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c82ac3a620 con 0x55c82adc70b0 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f975d9c0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c82ac31400 con 0x55c82ad1cce0 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f97573df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82adc9610 0x55c82ad36330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f97573df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82adc9610 0x55c82ad36330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53232/0 (socket says 172.21.15.45:53232) 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f97573df700 1 -- 172.21.15.45:0/2587424880 learned_addr learned my addr 172.21.15.45:0/2587424880 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f9756bde700 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82ad1cce0 0x55c82adc4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.049+0000 7f97563dd700 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 0x55c82ad35df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97573df700 1 -- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82ad1cce0 msgr2=0x55c82adc4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97573df700 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82ad1cce0 0x55c82adc4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97573df700 1 -- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 msgr2=0x55c82ad35df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97573df700 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 0x55c82ad35df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97573df700 1 -- 172.21.15.45:0/2587424880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975000a0e0 con 0x55c82adc9610 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97573df700 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82adc9610 0x55c82ad36330 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f975000a8c0 tx=0x7f9750007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c8c4177c630c1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97553db700 1 -- 172.21.15.45:0/2587424880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750012660 con 0x55c82adc9610 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97553db700 1 -- 172.21.15.45:0/2587424880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97500127c0 con 0x55c82adc9610 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.050+0000 7f97553db700 1 -- 172.21.15.45:0/2587424880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750012b40 con 0x55c82adc9610 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 -- 172.21.15.45:0/2587424880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82adc9610 msgr2=0x55c82ad36330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82adc9610 0x55c82ad36330 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f975000a8c0 tx=0x7f9750007040 comp rx=0 tx=0).stop 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 -- 172.21.15.45:0/2587424880 shutdown_connections 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82adc9610 0x55c82ad36330 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82ad1cce0 0x55c82adc4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 --2- 172.21.15.45:0/2587424880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 0x55c82ad35df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 -- 172.21.15.45:0/2587424880 >> 172.21.15.45:0/2587424880 conn(0x55c82acf1750 msgr2=0x55c82acf2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 -- 172.21.15.45:0/2587424880 shutdown_connections 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.051+0000 7f975d9c0040 1 -- 172.21.15.45:0/2587424880 wait complete. 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.052+0000 7f975d9c0040 1 Processor -- start 2023-10-06T05:26:54.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.052+0000 7f975d9c0040 1 -- start start 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f975d9c0040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82ad1cce0 0x55c82add7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f975d9c0040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 0x55c82ad36f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f975d9c0040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82adc9610 0x55c82add7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f975d9c0040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c82ad1cb50 con 0x55c82ad1cce0 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f975d9c0040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c82ad1c370 con 0x55c82adc70b0 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f975d9c0040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c82ad1c610 con 0x55c82adc9610 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82ad1cce0 0x55c82add7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82ad1cce0 0x55c82add7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53238/0 (socket says 172.21.15.45:53238) 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 learned_addr learned my addr 172.21.15.45:0/1806096494 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f97563dd700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 0x55c82ad36f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f97573df700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82adc9610 0x55c82add7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82adc9610 msgr2=0x55c82add7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c82adc9610 0x55c82add7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 msgr2=0x55c82ad36f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c82adc70b0 0x55c82ad36f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975000a510 con 0x55c82ad1cce0 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.053+0000 7f9756bde700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82ad1cce0 0x55c82add7cc0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f9748011000 tx=0x7f9748015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.054+0000 7f97477fe700 1 -- 172.21.15.45:0/1806096494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748007e80 con 0x55c82ad1cce0 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.054+0000 7f97477fe700 1 -- 172.21.15.45:0/1806096494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97480054f0 con 0x55c82ad1cce0 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.054+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9748005cf0 con 0x55c82ad1cce0 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.054+0000 7f97477fe700 1 -- 172.21.15.45:0/1806096494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748007af0 con 0x55c82ad1cce0 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.054+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c82ac493d0 con 0x55c82ad1cce0 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.055+0000 7f97477fe700 1 -- 172.21.15.45:0/1806096494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9748019040 con 0x55c82ad1cce0 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.056+0000 7f97477fe700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c065890 0x7f973c067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.056+0000 7f97477fe700 1 -- 172.21.15.45:0/1806096494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7f9748085cd0 con 0x55c82ad1cce0 2023-10-06T05:26:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.056+0000 7f97563dd700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c065890 0x7f973c067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.063 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pool 2023-10-06T05:26:54.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.056+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c82add8200 0x55c82addab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.056+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:1 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ad1e810 con 0x55c82add8200 2023-10-06T05:26:54.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.056+0000 7f97573df700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c82add8200 0x55c82addab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.056+0000 7f97563dd700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c065890 0x7f973c067d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f974c004170 tx=0x7f974c000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.057+0000 7f97573df700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c82add8200 0x55c82addab50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.059+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(1 601.00000000 [create] v131'1190 uv1164 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.059+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:2 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ad37470 con 0x55c82add8200 2023-10-06T05:26:54.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.060+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== osd_op_reply(2 601.00000000 [delete] v131'1191 uv1191 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.060+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:3 14.10 14:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adc7990 con 0x55c82add8200 2023-10-06T05:26:54.065 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2023-10-06T05:26:54.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.062+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 3 ==== osd_op_reply(3 601.00000000 [create] v131'1192 uv1192 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.062+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:4 14.10 14:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82add7680 con 0x55c82add8200 2023-10-06T05:26:54.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.063+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v131'1193 uv1193 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.063+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:5 14.10 14:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4930 con 0x55c82add8200 2023-10-06T05:26:54.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.064+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v131'1194 uv1194 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.064+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:6 14.10 14:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade3920 con 0x55c82add8200 2023-10-06T05:26:54.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.066+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v131'1195 uv1195 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.066+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade3d20 con 0x55c82add8200 2023-10-06T05:26:54.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.067+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 7 ==== osd_op_reply(7 602.00000000 [create] v131'1196 uv1170 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.067+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:8 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.069+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 8 ==== osd_op_reply(8 602.00000000 [delete] v131'1197 uv1197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.069+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:9 14.10 14:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.072 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2023-10-06T05:26:54.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.070+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 9 ==== osd_op_reply(9 602.00000000 [create] v131'1198 uv1198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.070+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:10 14.10 14:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.071+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v131'1199 uv1199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.071+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:11 14.10 14:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.073+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v131'1200 uv1200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.073+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:12 14.10 14:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.074+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v131'1201 uv1201 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.074+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:13 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.076+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 13 ==== osd_op_reply(13 10000000000.00000000 [create] v131'1202 uv1177 ondisk = -17 ((17) File exists)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.076+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:14 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.077+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v131'1203 uv1203 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.077+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:15 14.10 14:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.078+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 15 ==== osd_op_reply(15 10000000000.00000000 [create] v131'1204 uv1204 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.080 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2023-10-06T05:26:54.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.079+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:16 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.080+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v131'1205 uv1205 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.080+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:17 14.10 14:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.081+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v131'1206 uv1206 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.081+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:18 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade40d0 con 0x55c82add8200 2023-10-06T05:26:54.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.083+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v131'1207 uv1207 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.083+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:19 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.084+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v131'1208 uv1208 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.084+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:20 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.086+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 20 ==== osd_op_reply(20 600.00000000 [create] v131'1209 uv1183 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.086+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:21 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.087+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 21 ==== osd_op_reply(21 600.00000000 [delete] v131'1210 uv1210 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.087+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:22 14.10 14:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.090 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2023-10-06T05:26:54.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.089+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 22 ==== osd_op_reply(22 600.00000000 [create] v131'1211 uv1211 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.089+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:23 14.10 14:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade5cf0 con 0x55c82add8200 2023-10-06T05:26:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.090+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v131'1212 uv1212 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.090+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:24 14.10 14:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade5cf0 con 0x55c82add8200 2023-10-06T05:26:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.091+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v131'1213 uv1213 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.091+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:25 14.10 14:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.093+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v131'1214 uv1214 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.093+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:26 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.094+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 26 ==== osd_op_reply(26 603.00000000 [create] v131'1215 uv1189 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.094+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:27 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.096+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 27 ==== osd_op_reply(27 603.00000000 [delete] v131'1216 uv1216 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.096+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:28 14.10 14:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.099 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2023-10-06T05:26:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.097+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 28 ==== osd_op_reply(28 603.00000000 [create] v131'1217 uv1217 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.098+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:29 14.10 14:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade5cf0 con 0x55c82add8200 2023-10-06T05:26:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.099+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v131'1218 uv1218 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.099+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:30 14.10 14:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade5cf0 con 0x55c82add8200 2023-10-06T05:26:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.100+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v131'1219 uv1219 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.100+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:31 14.10 14:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade4ce0 con 0x55c82add8200 2023-10-06T05:26:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.101+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.1 v2:172.21.15.45:6800/119738 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v131'1220 uv1220 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82add8200 2023-10-06T05:26:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.101+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c82ade51f0 0x55c82ade75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.101+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:32 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade7b10 con 0x55c82ade51f0 2023-10-06T05:26:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.102+0000 7f9756bde700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c82ade51f0 0x55c82ade75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.102+0000 7f9756bde700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c82ade51f0 0x55c82ade75d0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.104+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== osd_op_reply(32 1.00000000.inode [create] v131'857 uv840 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.104+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:33 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade7ec0 con 0x55c82ade51f0 2023-10-06T05:26:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.105+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== osd_op_reply(33 1.00000000.inode [delete] v131'858 uv858 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.105+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:34 14.14 14:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade8270 con 0x55c82ade51f0 2023-10-06T05:26:54.108 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2023-10-06T05:26:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.107+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 3 ==== osd_op_reply(34 1.00000000.inode [create] v131'859 uv859 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.107+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:35 14.14 14:287034c4:::1.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade8b10 con 0x55c82ade51f0 2023-10-06T05:26:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.108+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 4 ==== osd_op_reply(35 1.00000000.inode [write 0~554] v131'860 uv860 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.108+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:36 14.14 14:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade8ec0 con 0x55c82ade51f0 2023-10-06T05:26:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.109+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v131'861 uv861 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.109+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:37 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9270 con 0x55c82ade51f0 2023-10-06T05:26:54.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.111+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 6 ==== osd_op_reply(37 200.00000000 [create] v131'862 uv845 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.111+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:38 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.112+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 7 ==== osd_op_reply(38 200.00000000 [delete] v131'863 uv863 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.112+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.115 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2023-10-06T05:26:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.114+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 8 ==== osd_op_reply(39 200.00000000 [create] v131'864 uv864 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.114+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:40 14.14 14:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.115+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v131'865 uv865 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.115+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:41 14.14 14:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.116+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v131'866 uv866 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.116+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:42 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.118+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 11 ==== osd_op_reply(42 200.00000001 [create] v131'867 uv850 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.118+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:43 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.119+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 12 ==== osd_op_reply(43 200.00000001 [delete] v131'868 uv868 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.119+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:44 14.14 14:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.122 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2023-10-06T05:26:54.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.121+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 13 ==== osd_op_reply(44 200.00000001 [create] v131'869 uv869 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.121+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:45 14.14 14:2e2fa760:::200.00000001:head [write 0~14075 in=14075b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adecd40 con 0x55c82ade51f0 2023-10-06T05:26:54.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.122+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 14 ==== osd_op_reply(45 200.00000001 [write 0~14075] v131'870 uv870 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.122+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:46 14.14 14:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adecd40 con 0x55c82ade51f0 2023-10-06T05:26:54.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.123+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v131'871 uv871 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.123+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:47 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.125+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 16 ==== osd_op_reply(47 606.00000000 [create] v131'872 uv856 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.125+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:48 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.126+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 17 ==== osd_op_reply(48 606.00000000 [delete] v131'873 uv873 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.126+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:49 14.14 14:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.129 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2023-10-06T05:26:54.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.127+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 18 ==== osd_op_reply(49 606.00000000 [create] v131'874 uv874 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.127+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:50 14.14 14:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adea630 con 0x55c82ade51f0 2023-10-06T05:26:54.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.129+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v131'875 uv875 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.129+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:51 14.14 14:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adea630 con 0x55c82ade51f0 2023-10-06T05:26:54.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.130+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v131'876 uv876 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.130+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:52 14.14 14:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82ade9620 con 0x55c82ade51f0 2023-10-06T05:26:54.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.131+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v131'877 uv877 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.131+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c82ade9b30 0x55c82adebf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.131+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:53 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adec450 con 0x55c82ade9b30 2023-10-06T05:26:54.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.131+0000 7f97563dd700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c82ade9b30 0x55c82adebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.132+0000 7f97563dd700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c82ade9b30 0x55c82adebf10 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.133+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(53 607.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.133+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:54 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adec800 con 0x55c82ade9b30 2023-10-06T05:26:54.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.135+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(54 607.00000000 [delete] v131'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.135+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:55 14.1c 14:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adecbb0 con 0x55c82ade9b30 2023-10-06T05:26:54.137 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2023-10-06T05:26:54.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.136+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(55 607.00000000 [create] v131'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.136+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:56 14.1c 14:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adedf70 con 0x55c82ade9b30 2023-10-06T05:26:54.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.137+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.137+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:57 14.1c 14:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adee320 con 0x55c82ade9b30 2023-10-06T05:26:54.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.138+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.138+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:58 14.1c 14:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adecf60 con 0x55c82ade9b30 2023-10-06T05:26:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.139+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.139+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:59 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded360 con 0x55c82ade9b30 2023-10-06T05:26:54.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.140+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 7 ==== osd_op_reply(59 mds0_openfiles.0 [create] v131'167 uv166 ondisk = -17 ((17) File exists)) v8 ==== 160+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.140+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:60 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded360 con 0x55c82ade9b30 2023-10-06T05:26:54.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.142+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 8 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v131'168 uv168 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.142+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:61 14.6 14:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded360 con 0x55c82ade9b30 2023-10-06T05:26:54.144 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2023-10-06T05:26:54.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.143+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 9 ==== osd_op_reply(61 mds0_openfiles.0 [create] v131'169 uv169 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.143+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:62 14.6 14:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded7e0 con 0x55c82ade9b30 2023-10-06T05:26:54.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.144+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 10 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v131'170 uv170 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.144+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:63 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded3e0 con 0x55c82ade51f0 2023-10-06T05:26:54.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.146+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 22 ==== osd_op_reply(63 608.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.146+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:64 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded3e0 con 0x55c82ade51f0 2023-10-06T05:26:54.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.147+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 23 ==== osd_op_reply(64 608.00000000 [delete] v131'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.147+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:65 14.e 14:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded810 con 0x55c82ade51f0 2023-10-06T05:26:54.150 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2023-10-06T05:26:54.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.148+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 24 ==== osd_op_reply(65 608.00000000 [create] v131'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.148+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:66 14.e 14:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded360 con 0x55c82ade51f0 2023-10-06T05:26:54.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.149+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 25 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.149+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:67 14.e 14:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded360 con 0x55c82ade51f0 2023-10-06T05:26:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.150+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 26 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.150+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- osd_op(unknown.0.0:68 14.e 14:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82aded360 con 0x55c82ade51f0 2023-10-06T05:26:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.151+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.4 v2:172.21.15.144:6824/119910 27 ==== osd_op_reply(68 608.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82ade51f0 2023-10-06T05:26:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.152+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c82aded360 0x55c82adf0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.152+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:69 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1180 con 0x55c82aded360 2023-10-06T05:26:54.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.152+0000 7f97573df700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c82aded360 0x55c82adf0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.152+0000 7f97573df700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c82aded360 0x55c82adf0c40 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.154+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(69 604.00000000 [create] v131'426 uv420 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.154+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:70 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.155+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== osd_op_reply(70 604.00000000 [delete] v131'427 uv427 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.155+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:71 14.15 14:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.157 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2023-10-06T05:26:54.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.156+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 3 ==== osd_op_reply(71 604.00000000 [create] v131'428 uv428 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.156+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:72 14.15 14:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf2570 con 0x55c82aded360 2023-10-06T05:26:54.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.157+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v131'429 uv429 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.157+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:73 14.15 14:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf2570 con 0x55c82aded360 2023-10-06T05:26:54.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.158+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v131'430 uv430 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.158+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:74 14.15 14:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.159+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v131'431 uv431 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.159+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:75 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.162+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 7 ==== osd_op_reply(75 500.00000000 [create] v131'432 uv425 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.162+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:76 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.163+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 8 ==== osd_op_reply(76 500.00000000 [delete] v131'433 uv433 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.163+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:77 14.15 14:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.166 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2023-10-06T05:26:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.164+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 9 ==== osd_op_reply(77 500.00000000 [create] v131'434 uv434 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.164+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:78 14.15 14:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.165+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v131'435 uv435 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.165+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:79 14.15 14:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf1560 con 0x55c82aded360 2023-10-06T05:26:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.167+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v131'436 uv436 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.167+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c82adf1be0 0x55c82adf3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.167+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:80 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf46e0 con 0x55c82adf1be0 2023-10-06T05:26:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.167+0000 7f9756bde700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c82adf1be0 0x55c82adf3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.167+0000 7f9756bde700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c82adf1be0 0x55c82adf3fc0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.168+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== osd_op_reply(80 mds_snaptable [create] v131'193 uv192 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82adf1be0 2023-10-06T05:26:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.169+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:81 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82adf1be0 2023-10-06T05:26:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.170+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== osd_op_reply(81 mds_snaptable [delete] v131'194 uv194 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82adf1be0 2023-10-06T05:26:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.170+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:82 14.d 14:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82adf1be0 2023-10-06T05:26:54.172 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2023-10-06T05:26:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.171+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.6 v2:172.21.15.144:6800/119899 3 ==== osd_op_reply(82 mds_snaptable [create] v131'195 uv195 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82adf1be0 2023-10-06T05:26:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.171+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:83 14.d 14:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82adf1be0 2023-10-06T05:26:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.172+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.6 v2:172.21.15.144:6800/119899 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v131'196 uv196 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82adf1be0 2023-10-06T05:26:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.172+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- osd_op(unknown.0.0:84 14.d 14:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82adf1be0 2023-10-06T05:26:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.173+0000 7f9756bde700 1 -- 172.21.15.45:0/1806096494 <== osd.6 v2:172.21.15.144:6800/119899 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v131'197 uv197 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9748024070 con 0x55c82adf1be0 2023-10-06T05:26:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.173+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:85 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82ade9b30 2023-10-06T05:26:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.175+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 11 ==== osd_op_reply(85 605.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.175+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:86 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82ade9b30 2023-10-06T05:26:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.176+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 12 ==== osd_op_reply(86 605.00000000 [delete] v131'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.176+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:87 14.1d 14:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82ade9b30 2023-10-06T05:26:54.179 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2023-10-06T05:26:54.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.177+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 13 ==== osd_op_reply(87 605.00000000 [create] v131'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.177+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:88 14.1d 14:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf5ad0 con 0x55c82ade9b30 2023-10-06T05:26:54.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.178+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 14 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.178+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:89 14.1d 14:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf5ad0 con 0x55c82ade9b30 2023-10-06T05:26:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.179+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 15 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.179+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:90 14.1d 14:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf4ac0 con 0x55c82ade9b30 2023-10-06T05:26:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.180+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 16 ==== osd_op_reply(90 605.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.181+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c82adf5140 0x55c82adf7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.181+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:91 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7b80 con 0x55c82adf5140 2023-10-06T05:26:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.181+0000 7f97563dd700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c82adf5140 0x55c82adf7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.181+0000 7f97563dd700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c82adf5140 0x55c82adf7520 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.183+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(91 mds0_inotable [create] v131'386 uv380 ondisk = -17 ((17) File exists)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.183+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:92 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.184+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== osd_op_reply(92 mds0_inotable [delete] v131'387 uv387 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.184+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.185+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 3 ==== osd_op_reply(93 mds0_inotable [create] v131'388 uv388 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.187 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2023-10-06T05:26:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.185+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.186+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 4 ==== osd_op_reply(94 mds0_inotable [write 0~34] v131'389 uv389 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.186+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:95 14.13 14:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.187+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 5 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v131'390 uv390 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.188+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:96 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.189+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 6 ==== osd_op_reply(96 100.00000000 [create] v131'391 uv385 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.189+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:97 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.190+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 7 ==== osd_op_reply(97 100.00000000 [delete] v131'392 uv392 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.190+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:98 14.13 14:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.191+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 8 ==== osd_op_reply(98 100.00000000 [create] v131'393 uv393 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.193 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2023-10-06T05:26:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.191+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:99 14.13 14:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7f60 con 0x55c82adf5140 2023-10-06T05:26:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.192+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 9 ==== osd_op_reply(99 100.00000000 [omap-set-header] v131'394 uv394 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.192+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:100 14.13 14:cd5a64a3:::100.00000000:head [omap-set-vals in=5074b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfacd0 con 0x55c82adf5140 2023-10-06T05:26:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.194+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.2 v2:172.21.15.45:6808/119739 10 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v131'395 uv395 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82adf5140 2023-10-06T05:26:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.194+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:101 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf8450 con 0x55c82ade9b30 2023-10-06T05:26:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.195+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 17 ==== osd_op_reply(101 mds0_sessionmap [create] v131'172 uv171 ondisk = -17 ((17) File exists)) v8 ==== 159+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.195+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:102 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf8450 con 0x55c82ade9b30 2023-10-06T05:26:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.196+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 18 ==== osd_op_reply(102 mds0_sessionmap [delete] v131'173 uv173 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.196+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:103 14.b 14:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf8450 con 0x55c82ade9b30 2023-10-06T05:26:54.199 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2023-10-06T05:26:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.197+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 19 ==== osd_op_reply(103 mds0_sessionmap [create] v131'174 uv174 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.197+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:104 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf8450 con 0x55c82ade9b30 2023-10-06T05:26:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.198+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 20 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v131'175 uv175 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.198+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c82adf8960 0x55c82adf8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.198+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:105 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82adf8960 2023-10-06T05:26:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.199+0000 7f97573df700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c82adf8960 0x55c82adf8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.199+0000 7f97573df700 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c82adf8960 0x55c82adf8d30 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.200+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(105 609.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82adf8960 2023-10-06T05:26:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.200+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:106 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82adf8960 2023-10-06T05:26:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.201+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(106 609.00000000 [delete] v131'232 uv232 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82adf8960 2023-10-06T05:26:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.202+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:107 14.7 14:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82adf8960 2023-10-06T05:26:54.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.203+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(107 609.00000000 [create] v131'233 uv233 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82adf8960 2023-10-06T05:26:54.204 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2023-10-06T05:26:54.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.203+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:108 14.7 14:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfc430 con 0x55c82adf8960 2023-10-06T05:26:54.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.204+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82adf8960 2023-10-06T05:26:54.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.204+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:109 14.7 14:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfc430 con 0x55c82adf8960 2023-10-06T05:26:54.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.205+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.5 v2:172.21.15.144:6816/119917 5 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82adf8960 2023-10-06T05:26:54.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.205+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:110 14.7 14:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82adf8960 2023-10-06T05:26:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.206+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.5 v2:172.21.15.144:6816/119917 6 ==== osd_op_reply(110 609.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82adf8960 2023-10-06T05:26:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.206+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:111 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82aded360 2023-10-06T05:26:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.207+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 12 ==== osd_op_reply(111 400.00000000 [create] v131'196 uv195 ondisk = -17 ((17) File exists)) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.207+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:112 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82aded360 2023-10-06T05:26:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.208+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 13 ==== osd_op_reply(112 400.00000000 [delete] v131'197 uv197 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.208+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:113 14.f 14:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82aded360 2023-10-06T05:26:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2023-10-06T05:26:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.209+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 14 ==== osd_op_reply(113 400.00000000 [create] v131'198 uv198 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.210+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:114 14.f 14:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb420 con 0x55c82aded360 2023-10-06T05:26:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.211+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 15 ==== osd_op_reply(114 400.00000000 [write 0~22] v131'199 uv199 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.211+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:115 14.f 14:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82aded360 2023-10-06T05:26:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.212+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 <== osd.7 v2:172.21.15.144:6808/119901 16 ==== osd_op_reply(115 400.00000000 [omap-set-header] v131'200 uv200 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f9750019070 con 0x55c82aded360 2023-10-06T05:26:54.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.212+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:116 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.213+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 21 ==== osd_op_reply(116 100.00000000.inode [create] v131'480 uv472 ondisk = -17 ((17) File exists)) v8 ==== 162+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.213+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:117 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.215+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 22 ==== osd_op_reply(117 100.00000000.inode [delete] v131'481 uv481 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.215+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:118 14.1f 14:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.217 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2023-10-06T05:26:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.216+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 23 ==== osd_op_reply(118 100.00000000.inode [create] v131'482 uv482 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.216+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:119 14.1f 14:fe07bba1:::100.00000000.inode:head [write 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.217+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 24 ==== osd_op_reply(119 100.00000000.inode [write 0~554] v131'483 uv483 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.217+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:120 14.1f 14:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.218+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 25 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v131'484 uv484 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.218+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:121 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.219+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 26 ==== osd_op_reply(121 1.00000000 [create] v131'485 uv479 ondisk = -17 ((17) File exists)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.219+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:122 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.221+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 27 ==== osd_op_reply(122 1.00000000 [delete] v131'486 uv486 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.221+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:123 14.1f 14:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.223 INFO:teuthology.orchestra.run.smithi045.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2023-10-06T05:26:54.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.222+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 28 ==== osd_op_reply(123 1.00000000 [create] v131'487 uv487 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.222+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:124 14.1f 14:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.223+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 29 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v131'488 uv488 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.223+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:125 14.1f 14:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.224+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 30 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v131'489 uv489 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.224+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:126 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adf7fe0 con 0x55c82ade9b30 2023-10-06T05:26:54.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.225+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 31 ==== osd_op_reply(126 1.00000000 [omap-set-header] v131'490 uv490 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.225+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:127 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55c82adfb910 con 0x55c82ade9b30 2023-10-06T05:26:54.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f97563dd700 1 -- 172.21.15.45:0/1806096494 <== osd.0 v2:172.21.15.45:6816/119749 32 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v131'491 uv491 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f974c00e8c0 con 0x55c82ade9b30 2023-10-06T05:26:54.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c82ade9b30 msgr2=0x55c82adebf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x55c82ade9b30 0x55c82adebf10 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c82add8200 msgr2=0x55c82addab50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x55c82add8200 0x55c82addab50 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c82adf5140 msgr2=0x55c82adf7520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55c82adf5140 0x55c82adf7520 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c82ade51f0 msgr2=0x55c82ade75d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x55c82ade51f0 0x55c82ade75d0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c82adf8960 msgr2=0x55c82adf8d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x55c82adf8960 0x55c82adf8d30 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c82adf1be0 msgr2=0x55c82adf3fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x55c82adf1be0 0x55c82adf3fc0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c82aded360 msgr2=0x55c82adf0c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x55c82aded360 0x55c82adf0c40 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 reap_dead start 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 reap_dead start 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 reap_dead start 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 reap_dead start 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f97573df700 1 -- 172.21.15.45:0/1806096494 reap_dead start 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c065890 msgr2=0x7f973c067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c065890 0x7f973c067d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f974c004170 tx=0x7f974c000bc0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82ad1cce0 msgr2=0x55c82add7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82ad1cce0 0x55c82add7cc0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f9748011000 tx=0x7f9748015000 comp rx=0 tx=0).stop 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 shutdown_connections 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c82ad1cce0 0x55c82add7cc0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.226+0000 7f975d9c0040 1 --2- 172.21.15.45:0/1806096494 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f973c065890 0x7f973c067d40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.227+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 >> 172.21.15.45:0/1806096494 conn(0x55c82acf1750 msgr2=0x55c82ad345e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.227+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 shutdown_connections 2023-10-06T05:26:54.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.227+0000 7f975d9c0040 1 -- 172.21.15.45:0/1806096494 wait complete. 2023-10-06T05:26:54.233 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2023-10-06T05:26:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.281+0000 7f444c148040 1 Processor -- start 2023-10-06T05:26:54.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.281+0000 7f444c148040 1 -- start start 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f444c148040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0f67f80 0x561cf0ff7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f444c148040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0ff8900 0x561cf100f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f444c148040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf100fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f444c148040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf0e83620 con 0x561cf0ff8cd0 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f444c148040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf0e7a400 con 0x561cf0ff8900 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f444c148040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf1012400 con 0x561cf0f67f80 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f4445b67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf100fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f4444b65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0ff8900 0x561cf100f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f4445366700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0f67f80 0x561cf0ff7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f4444b65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0ff8900 0x561cf100f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:46040/0 (socket says 172.21.15.45:46040) 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f4445b67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf100fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53262/0 (socket says 172.21.15.45:53262) 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.282+0000 7f4444b65700 1 -- 172.21.15.45:0/761036792 learned_addr learned my addr 172.21.15.45:0/761036792 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f4444b65700 1 -- 172.21.15.45:0/761036792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0f67f80 msgr2=0x561cf0ff7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f4444b65700 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0f67f80 0x561cf0ff7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f4444b65700 1 -- 172.21.15.45:0/761036792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 msgr2=0x561cf100fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f4444b65700 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf100fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f4444b65700 1 -- 172.21.15.45:0/761036792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438005db0 con 0x561cf0ff8900 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f4445b67700 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf100fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:54.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f4444b65700 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0ff8900 0x561cf100f690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4438005cf0 tx=0x7f4438016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39523ba4a64b6acb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f443f7fe700 1 -- 172.21.15.45:0/761036792 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438005510 con 0x561cf0ff8900 2023-10-06T05:26:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f443f7fe700 1 -- 172.21.15.45:0/761036792 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438000bc0 con 0x561cf0ff8900 2023-10-06T05:26:54.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.283+0000 7f443f7fe700 1 -- 172.21.15.45:0/761036792 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438008bc0 con 0x561cf0ff8900 2023-10-06T05:26:54.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 -- 172.21.15.45:0/761036792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0ff8900 msgr2=0x561cf100f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0ff8900 0x561cf100f690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4438005cf0 tx=0x7f4438016000 comp rx=0 tx=0).stop 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 -- 172.21.15.45:0/761036792 shutdown_connections 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf100fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0f67f80 0x561cf0ff7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 --2- 172.21.15.45:0/761036792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0ff8900 0x561cf100f690 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 -- 172.21.15.45:0/761036792 >> 172.21.15.45:0/761036792 conn(0x561cf0f3a5f0 msgr2=0x561cf0ff5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 -- 172.21.15.45:0/761036792 shutdown_connections 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.284+0000 7f444c148040 1 -- 172.21.15.45:0/761036792 wait complete. 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.285+0000 7f444c148040 1 Processor -- start 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.285+0000 7f444c148040 1 -- start start 2023-10-06T05:26:54.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f444c148040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0f67f80 0x561cf100e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f444c148040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0ff8900 0x561cf0ff0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f444c148040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf1012bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f444c148040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf1020b80 con 0x561cf0ff8cd0 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f444c148040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf1020e20 con 0x561cf0f67f80 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f444c148040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cf10210c0 con 0x561cf0ff8900 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf1012bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf1012bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53268/0 (socket says 172.21.15.45:53268) 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 -- 172.21.15.45:0/1223036237 learned_addr learned my addr 172.21.15.45:0/1223036237 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445366700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0f67f80 0x561cf100e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4444b65700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0ff8900 0x561cf0ff0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 -- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0ff8900 msgr2=0x561cf0ff0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561cf0ff8900 0x561cf0ff0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 -- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0f67f80 msgr2=0x561cf100e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf0f67f80 0x561cf100e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.286+0000 7f4445b67700 1 -- 172.21.15.45:0/1223036237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444000b340 con 0x561cf0ff8cd0 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.287+0000 7f4445b67700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf1012bd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f4440010940 tx=0x7f4440005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.287+0000 7f443dffb700 1 -- 172.21.15.45:0/1223036237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440010ee0 con 0x561cf0ff8cd0 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.287+0000 7f443dffb700 1 -- 172.21.15.45:0/1223036237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000b5c0 con 0x561cf0ff8cd0 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.287+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561cf0e85750 con 0x561cf0ff8cd0 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.287+0000 7f443dffb700 1 -- 172.21.15.45:0/1223036237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000eb90 con 0x561cf0ff8cd0 2023-10-06T05:26:54.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.287+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cf0eac1c0 con 0x561cf0ff8cd0 2023-10-06T05:26:54.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.288+0000 7f443dffb700 1 -- 172.21.15.45:0/1223036237 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4440013440 con 0x561cf0ff8cd0 2023-10-06T05:26:54.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.289+0000 7f443dffb700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4428065890 0x7f4428067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.289+0000 7f443dffb700 1 -- 172.21.15.45:0/1223036237 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7f4440017030 con 0x561cf0ff8cd0 2023-10-06T05:26:54.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.289+0000 7f4445366700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4428065890 0x7f4428067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.289+0000 7f444c148040 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561cf1021aa0 0x561cf1021e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.289+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x561cf10223b0 con 0x561cf1021aa0 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.289+0000 7f4444b65700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561cf1021aa0 0x561cf1021e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.289+0000 7f4445366700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4428065890 0x7f4428067d40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4430005fd0 tx=0x7f443000c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.290+0000 7f4444b65700 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561cf1021aa0 0x561cf1021e70 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f4444b65700 1 -- 172.21.15.45:0/1223036237 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v131'492 uv492 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4438008970 con 0x561cf1021aa0 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561cf1021aa0 msgr2=0x561cf1021e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561cf1021aa0 0x561cf1021e70 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4428065890 msgr2=0x7f4428067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4428065890 0x7f4428067d40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4430005fd0 tx=0x7f443000c040 comp rx=0 tx=0).stop 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 msgr2=0x561cf1012bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 --2- 172.21.15.45:0/1223036237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cf0ff8cd0 0x561cf1012bd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f4440010940 tx=0x7f4440005d40 comp rx=0 tx=0).stop 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f4445b67700 1 -- 172.21.15.45:0/1223036237 reap_dead start 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 shutdown_connections 2023-10-06T05:26:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 >> 172.21.15.45:0/1223036237 conn(0x561cf0f3a5f0 msgr2=0x561cf0ff5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 shutdown_connections 2023-10-06T05:26:54.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.291+0000 7f444c148040 1 -- 172.21.15.45:0/1223036237 wait complete. 2023-10-06T05:26:54.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:26:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.435+0000 7f0a0c20c700 1 Processor -- start 2023-10-06T05:26:54.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.435+0000 7f0a0c20c700 1 -- start start 2023-10-06T05:26:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a0c20c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a0c20c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040c8b70 0x7f0a040c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a0c20c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040c3b50 con 0x7f0a04157330 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a0c20c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a04157700 con 0x7f0a040ccb70 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a0c20c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040c98e0 con 0x7f0a040c8b70 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a097a7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a09fa8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0a04157330 0x7f0a040ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34136/0 (socket says 172.21.15.45:34136) 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a097a7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53272/0 (socket says 172.21.15.45:53272) 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a09fa8700 1 -- 172.21.15.45:0/298733617 learned_addr learned my addr 172.21.15.45:0/298733617 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a0a7a9700 1 --2- 172.21.15.45:0/298733617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040c8b70 0x7f0a040c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a097a7700 1 -- 172.21.15.45:0/298733617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040c8b70 msgr2=0x7f0a040c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a097a7700 1 --2- 172.21.15.45:0/298733617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040c8b70 0x7f0a040c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a097a7700 1 -- 172.21.15.45:0/298733617 >> v1:172.21.15.45:6789/0 conn(0x7f0a04157330 legacy=0x7f0a040ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a097a7700 1 -- 172.21.15.45:0/298733617 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040c9bc0 con 0x7f0a040ccb70 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a08fa6700 1 -- 172.21.15.45:0/298733617 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323753204 0 0) 0x7f0a040c3b50 con 0x7f0a04157330 2023-10-06T05:26:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.436+0000 7f0a0a7a9700 1 --2- 172.21.15.45:0/298733617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040c8b70 0x7f0a040c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a097a7700 1 --2- 172.21.15.45:0/298733617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f09f40098e0 tx=0x7f09f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f05fa5046ed3dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a08fa6700 1 -- 172.21.15.45:0/298733617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4009e80 con 0x7f0a040ccb70 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a08fa6700 1 -- 172.21.15.45:0/298733617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f400b860 con 0x7f0a040ccb70 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a08fa6700 1 -- 172.21.15.45:0/298733617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400bbe0 con 0x7f0a040ccb70 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 -- 172.21.15.45:0/298733617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a040ccb70 msgr2=0x7f0a040aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/298733617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f09f40098e0 tx=0x7f09f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 -- 172.21.15.45:0/298733617 shutdown_connections 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/298733617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040c8b70 0x7f0a040c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/298733617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 -- 172.21.15.45:0/298733617 >> 172.21.15.45:0/298733617 conn(0x7f0a04067e00 msgr2=0x7f0a040cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 -- 172.21.15.45:0/298733617 shutdown_connections 2023-10-06T05:26:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.437+0000 7f0a0c20c700 1 -- 172.21.15.45:0/298733617 wait complete. 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 Processor -- start 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 -- start start 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a040c8b70 0x7f0a0416b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a04157330 0x7f0a04151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040ab020 con 0x7f0a04157330 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040c5ea0 con 0x7f0a040c8b70 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.439+0000 7f0a0c20c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040ab230 con 0x7f0a040ccb70 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a04157330 0x7f0a04151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a04157330 0x7f0a04151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53278/0 (socket says 172.21.15.45:53278) 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 -- 172.21.15.45:0/2937155124 learned_addr learned my addr 172.21.15.45:0/2937155124 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a09fa8700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a040c8b70 0x7f0a0416b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a097a7700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 -- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040ccb70 msgr2=0x7f0a040aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 -- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a040c8b70 msgr2=0x7f0a0416b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a040c8b70 0x7f0a0416b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 -- 172.21.15.45:0/2937155124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4009420 con 0x7f0a04157330 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a097a7700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0a7a9700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a04157330 0x7f0a04151c70 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009fc0 tx=0x7f09fc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f09f9ffb700 1 -- 172.21.15.45:0/2937155124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00dbe0 con 0x7f0a04157330 2023-10-06T05:26:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f09f9ffb700 1 -- 172.21.15.45:0/2937155124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc00f040 con 0x7f0a04157330 2023-10-06T05:26:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a040ab510 con 0x7f0a04157330 2023-10-06T05:26:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.440+0000 7f09f9ffb700 1 -- 172.21.15.45:0/2937155124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc015440 con 0x7f0a04157330 2023-10-06T05:26:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.441+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a040abad0 con 0x7f0a04157330 2023-10-06T05:26:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.442+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a040c6060 con 0x7f0a04157330 2023-10-06T05:26:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.442+0000 7f09f9ffb700 1 -- 172.21.15.45:0/2937155124 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09fc01e3b0 con 0x7f0a04157330 2023-10-06T05:26:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.442+0000 7f09f9ffb700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09e8066f30 0x7f09e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.443+0000 7f09f9ffb700 1 -- 172.21.15.45:0/2937155124 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7f09fc085970 con 0x7f0a04157330 2023-10-06T05:26:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.443+0000 7f0a09fa8700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09e8066f30 0x7f09e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.443+0000 7f0a09fa8700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09e8066f30 0x7f09e80693e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0a00001360 tx=0x7f0a00009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.447+0000 7f09f9ffb700 1 -- 172.21.15.45:0/2937155124 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09fc051b80 con 0x7f0a04157330 2023-10-06T05:26:54.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.696+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f0a0406e350 con 0x7f0a04157330 2023-10-06T05:26:54.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.988+0000 7f09f9ffb700 1 -- 172.21.15.45:0/2937155124 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v344) v1 ==== 162+0+0 (secure 0 0 0) 0x7f09fc01a020 con 0x7f0a04157330 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09e8066f30 msgr2=0x7f09e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09e8066f30 0x7f09e80693e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0a00001360 tx=0x7f0a00009040 comp rx=0 tx=0).stop 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a04157330 msgr2=0x7f0a04151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a04157330 0x7f0a04151c70 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009fc0 tx=0x7f09fc0099b0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 shutdown_connections 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f09e8066f30 0x7f09e80693e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a04157330 0x7f0a04151c70 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a040c8b70 0x7f0a0416b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 --2- 172.21.15.45:0/2937155124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a040ccb70 0x7f0a040aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 >> 172.21.15.45:0/2937155124 conn(0x7f0a04067e00 msgr2=0x7f0a04152770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 shutdown_connections 2023-10-06T05:26:54.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:54.991+0000 7f0a0c20c700 1 -- 172.21.15.45:0/2937155124 wait complete. 2023-10-06T05:26:54.994 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:26:55.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:55.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.143+0000 7f1d02e8a700 1 Processor -- start 2023-10-06T05:26:55.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.144+0000 7f1d02e8a700 1 -- start start 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.144+0000 7f1d02e8a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cfc0ae9d0 0x7f1cfc068860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.144+0000 7f1d02e8a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc15d470 0x7f1cfc0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.144+0000 7f1d02e8a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc0677c0 con 0x7f1cfc068da0 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.144+0000 7f1d02e8a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc067aa0 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.144+0000 7f1d02e8a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc067d80 con 0x7f1cfc15d470 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1d00c26700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cfc0ae9d0 0x7f1cfc068860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1cfbfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1cfc068da0 0x7f1cfc15cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34148/0 (socket says 172.21.15.45:34148) 2023-10-06T05:26:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1cfbfff700 1 -- 172.21.15.45:0/4127266367 learned_addr learned my addr 172.21.15.45:0/4127266367 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1d01427700 1 --2- 172.21.15.45:0/4127266367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc15d470 0x7f1cfc0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1d00c26700 1 -- 172.21.15.45:0/4127266367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc15d470 msgr2=0x7f1cfc0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1d00c26700 1 --2- 172.21.15.45:0/4127266367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc15d470 0x7f1cfc0ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1d00c26700 1 -- 172.21.15.45:0/4127266367 >> v1:172.21.15.45:6789/0 conn(0x7f1cfc068da0 legacy=0x7f1cfc15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1d00c26700 1 -- 172.21.15.45:0/4127266367 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc068060 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1cfb7fe700 1 -- 172.21.15.45:0/4127266367 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295572027 0 0) 0x7f1cfc0677c0 con 0x7f1cfc068da0 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1d00c26700 1 --2- 172.21.15.45:0/4127266367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc068860 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0098e0 tx=0x7f1cec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe0347391a6f84b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1cfb7fe700 1 -- 172.21.15.45:0/4127266367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec009e80 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1cfb7fe700 1 -- 172.21.15.45:0/4127266367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec00b860 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.145+0000 7f1cfb7fe700 1 -- 172.21.15.45:0/4127266367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00bbe0 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 -- 172.21.15.45:0/4127266367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 msgr2=0x7f1cfc068860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/4127266367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc068860 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0098e0 tx=0x7f1cec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 -- 172.21.15.45:0/4127266367 shutdown_connections 2023-10-06T05:26:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/4127266367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc15d470 0x7f1cfc0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/4127266367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc068860 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 -- 172.21.15.45:0/4127266367 >> 172.21.15.45:0/4127266367 conn(0x7f1cfc0cd000 msgr2=0x7f1cfc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 -- 172.21.15.45:0/4127266367 shutdown_connections 2023-10-06T05:26:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.146+0000 7f1d02e8a700 1 -- 172.21.15.45:0/4127266367 wait complete. 2023-10-06T05:26:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.147+0000 7f1d02e8a700 1 Processor -- start 2023-10-06T05:26:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.147+0000 7f1d02e8a700 1 -- start start 2023-10-06T05:26:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d02e8a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc068da0 0x7f1cfc156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d02e8a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d02e8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cfc15d470 0x7f1cfc16a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d02e8a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc16a480 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d02e8a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc0af860 con 0x7f1cfc15d470 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d02e8a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc0afb40 con 0x7f1cfc068da0 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1cfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1cfbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc169f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53282/0 (socket says 172.21.15.45:53282) 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1cfbfff700 1 -- 172.21.15.45:0/1573079013 learned_addr learned my addr 172.21.15.45:0/1573079013 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d01427700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cfc15d470 0x7f1cfc16a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.148+0000 7f1d00c26700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc068da0 0x7f1cfc156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cfbfff700 1 -- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc068da0 msgr2=0x7f1cfc156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cfbfff700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc068da0 0x7f1cfc156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cfbfff700 1 -- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cfc15d470 msgr2=0x7f1cfc16a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cfbfff700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cfc15d470 0x7f1cfc16a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cfbfff700 1 -- 172.21.15.45:0/1573079013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec009420 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cfbfff700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc169f40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c5b0 tx=0x7f1ce4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cf8ff9700 1 -- 172.21.15.45:0/1573079013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4005b30 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cf8ff9700 1 -- 172.21.15.45:0/1573079013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce4005c90 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc0afe20 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1cf8ff9700 1 -- 172.21.15.45:0/1573079013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4007300 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.149+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc0b03e0 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.150+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc0c5360 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.151+0000 7f1cf8ff9700 1 -- 172.21.15.45:0/1573079013 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ce401b3b0 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.151+0000 7f1cf8ff9700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cdc066f30 0x7f1cdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.151+0000 7f1cf8ff9700 1 -- 172.21.15.45:0/1573079013 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9732+0+0 (secure 0 0 0) 0x7f1ce4084960 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.151+0000 7f1d00c26700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cdc066f30 0x7f1cdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.152+0000 7f1d00c26700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cdc066f30 0x7f1cdc0693e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0091e0 tx=0x7f1cec017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.156+0000 7f1cf8ff9700 1 -- 172.21.15.45:0/1573079013 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cfc0c5360 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.405+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cfc0c5360 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.406+0000 7f1cf8ff9700 1 -- 172.21.15.45:0/1573079013 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 345 v345) v1 ==== 95+0+4609 (secure 0 0 0) 0x7f1cfc0c5360 con 0x7f1cfc0ae9d0 2023-10-06T05:26:55.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:26:55.408 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":345,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:54.989038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10087},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10087":{"gid":10087,"name":"b","rank":0,"incarnation":345,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/913913762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":913913762},{"type":"v1","addr":"172.21.15.144:6835","nonce":913913762}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cdc066f30 msgr2=0x7f1cdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cdc066f30 0x7f1cdc0693e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0091e0 tx=0x7f1cec017000 comp rx=0 tx=0).stop 2023-10-06T05:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 msgr2=0x7f1cfc169f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc169f40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c5b0 tx=0x7f1ce4001070 comp rx=0 tx=0).stop 2023-10-06T05:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 shutdown_connections 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1cdc066f30 0x7f1cdc0693e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cfc15d470 0x7f1cfc16a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1cfc068da0 0x7f1cfc156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 --2- 172.21.15.45:0/1573079013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc0ae9d0 0x7f1cfc169f40 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 >> 172.21.15.45:0/1573079013 conn(0x7f1cfc0cd000 msgr2=0x7f1cfc158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 shutdown_connections 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:26:55.408+0000 7f1d02e8a700 1 -- 172.21.15.45:0/1573079013 wait complete. 2023-10-06T05:26:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 345 2023-10-06T05:26:55.423 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T05:26:58.002 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:58.001+0000 7f3d712d8700 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:58.002 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-06T05:26:58.001+0000 7f3d712d8700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-10-06T05:26:58.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:58.005+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:58.007 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:26:58.005+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-06T05:26:58.013 INFO:tasks.ceph.mds.b.smithi144.stderr: -16> 2023-10-06T05:26:58.001+0000 7f3d712d8700 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input 2023-10-06T05:26:58.013 INFO:tasks.ceph.mds.b.smithi144.stderr: -13> 2023-10-06T05:26:58.001+0000 7f3d712d8700 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2023-10-06T05:26:58.034 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:27:00.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.563+0000 7f6793ec8700 1 Processor -- start 2023-10-06T05:27:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.564+0000 7f6793ec8700 1 -- start start 2023-10-06T05:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.564+0000 7f6793ec8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f678c1539f0 0x7f678c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.564+0000 7f6793ec8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f678c0ccfc0 0x7f678c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.564+0000 7f6793ec8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0ce690 con 0x7f678c153620 2023-10-06T05:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.564+0000 7f6793ec8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c15a7a0 con 0x7f678c0ccfc0 2023-10-06T05:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.564+0000 7f6793ec8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c0ce3f0 con 0x7f678c1539f0 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f678c0ccfc0 0x7f678c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f678c0ccfc0 0x7f678c15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43736/0 (socket says 172.21.15.45:43736) 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 -- 172.21.15.45:0/1454529060 learned_addr learned my addr 172.21.15.45:0/1454529060 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6791463700 1 --2- 172.21.15.45:0/1454529060 >> v2:172.21.15.144:3300/0 conn(0x7f678c1539f0 0x7f678c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 -- 172.21.15.45:0/1454529060 >> v2:172.21.15.144:3300/0 conn(0x7f678c1539f0 msgr2=0x7f678c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 --2- 172.21.15.45:0/1454529060 >> v2:172.21.15.144:3300/0 conn(0x7f678c1539f0 0x7f678c15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 -- 172.21.15.45:0/1454529060 >> v1:172.21.15.144:6789/0 conn(0x7f678c153620 legacy=0x7f678c0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 -- 172.21.15.45:0/1454529060 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c15d510 con 0x7f678c0ccfc0 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6792465700 1 --2- 172.21.15.45:0/1454529060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0ccfc0 0x7f678c15cdc0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f678c067840 tx=0x7f6784009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc4c2f1dea398c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6790c62700 1 -- 172.21.15.45:0/1454529060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400e070 con 0x7f678c0ccfc0 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6790c62700 1 -- 172.21.15.45:0/1454529060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678400ba10 con 0x7f678c0ccfc0 2023-10-06T05:27:00.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.565+0000 7f6790c62700 1 -- 172.21.15.45:0/1454529060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840092e0 con 0x7f678c0ccfc0 2023-10-06T05:27:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 -- 172.21.15.45:0/1454529060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0ccfc0 msgr2=0x7f678c15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 --2- 172.21.15.45:0/1454529060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0ccfc0 0x7f678c15cdc0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f678c067840 tx=0x7f6784009840 comp rx=0 tx=0).stop 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 -- 172.21.15.45:0/1454529060 shutdown_connections 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 --2- 172.21.15.45:0/1454529060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0ccfc0 0x7f678c15cdc0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 --2- 172.21.15.45:0/1454529060 >> v2:172.21.15.144:3300/0 conn(0x7f678c1539f0 0x7f678c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 -- 172.21.15.45:0/1454529060 >> 172.21.15.45:0/1454529060 conn(0x7f678c06ed50 msgr2=0x7f678c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 -- 172.21.15.45:0/1454529060 shutdown_connections 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.566+0000 7f6793ec8700 1 -- 172.21.15.45:0/1454529060 wait complete. 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.567+0000 7f6793ec8700 1 Processor -- start 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.567+0000 7f6793ec8700 1 -- start start 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6793ec8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c0ccfc0 0x7f678c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6793ec8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f678c153620 0x7f678c151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6793ec8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0c5f90 0x7f678c0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6793ec8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0c6910 con 0x7f678c0c5f90 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6793ec8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0b3430 con 0x7f678c0ccfc0 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6793ec8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c151580 con 0x7f678c153620 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6792465700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0c5f90 0x7f678c0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6792465700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0c5f90 0x7f678c0c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43750/0 (socket says 172.21.15.45:43750) 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.568+0000 7f6792465700 1 -- 172.21.15.45:0/2797201964 learned_addr learned my addr 172.21.15.45:0/2797201964 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6791463700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f678c153620 0x7f678c151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6791c64700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c0ccfc0 0x7f678c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6792465700 1 -- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f678c153620 msgr2=0x7f678c151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6792465700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f678c153620 0x7f678c151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6792465700 1 -- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c0ccfc0 msgr2=0x7f678c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6792465700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c0ccfc0 0x7f678c0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6792465700 1 -- 172.21.15.45:0/2797201964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784009ae0 con 0x7f678c0c5f90 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6791463700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f678c153620 0x7f678c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6791c64700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c0ccfc0 0x7f678c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6792465700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0c5f90 0x7f678c0c63d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f678c06f900 tx=0x7f6784017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f677dffb700 1 -- 172.21.15.45:0/2797201964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840094e0 con 0x7f678c0c5f90 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f677dffb700 1 -- 172.21.15.45:0/2797201964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67840032f0 con 0x7f678c0c5f90 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f677dffb700 1 -- 172.21.15.45:0/2797201964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678401d630 con 0x7f678c0c5f90 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c16a6a0 con 0x7f678c0c5f90 2023-10-06T05:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.569+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c1557a0 con 0x7f678c0c5f90 2023-10-06T05:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.570+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c088a60 con 0x7f678c0c5f90 2023-10-06T05:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.571+0000 7f677dffb700 1 -- 172.21.15.45:0/2797201964 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f678401da60 con 0x7f678c0c5f90 2023-10-06T05:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.571+0000 7f677dffb700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6770066f30 0x7f67700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.571+0000 7f677dffb700 1 -- 172.21.15.45:0/2797201964 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f6784086e60 con 0x7f678c0c5f90 2023-10-06T05:27:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.571+0000 7f6791c64700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6770066f30 0x7f67700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.572+0000 7f6791c64700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6770066f30 0x7f67700693e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6780002880 tx=0x7f6780009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.576+0000 7f677dffb700 1 -- 172.21.15.45:0/2797201964 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6784053020 con 0x7f678c0c5f90 2023-10-06T05:27:00.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.824+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f678c1531b0 con 0x7f678c0c5f90 2023-10-06T05:27:00.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.824+0000 7f677dffb700 1 -- 172.21.15.45:0/2797201964 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f6784012e60 con 0x7f678c0c5f90 2023-10-06T05:27:00.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:00.827 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:58.006287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:27:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.826+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6770066f30 msgr2=0x7f67700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.826+0000 7f6793ec8700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6770066f30 0x7f67700693e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6780002880 tx=0x7f6780009040 comp rx=0 tx=0).stop 2023-10-06T05:27:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0c5f90 msgr2=0x7f678c0c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0c5f90 0x7f678c0c63d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f678c06f900 tx=0x7f6784017000 comp rx=0 tx=0).stop 2023-10-06T05:27:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 shutdown_connections 2023-10-06T05:27:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6770066f30 0x7f67700693e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f678c0c5f90 0x7f678c0c63d0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f678c0ccfc0 0x7f678c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 --2- 172.21.15.45:0/2797201964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f678c153620 0x7f678c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 >> 172.21.15.45:0/2797201964 conn(0x7f678c06ed50 msgr2=0x7f678c15cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:00.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 shutdown_connections 2023-10-06T05:27:00.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.827+0000 7f6793ec8700 1 -- 172.21.15.45:0/2797201964 wait complete. 2023-10-06T05:27:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 350 2023-10-06T05:27:00.842 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T05:27:00.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.972+0000 7f8633f19700 1 Processor -- start 2023-10-06T05:27:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.972+0000 7f8633f19700 1 -- start start 2023-10-06T05:27:00.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8633f19700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f862c158f60 0x7f862c159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8633f19700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f862c0cc390 0x7f862c0cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8633f19700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c0cb530 con 0x7f862c06a060 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8633f19700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c06a500 con 0x7f862c158f60 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8633f19700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c069bd0 con 0x7f862c0cc390 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8631cb5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f862c158f60 0x7f862c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f86314b4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f862c06a060 0x7f862c0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47102/0 (socket says 172.21.15.45:47102) 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f86314b4700 1 -- 172.21.15.45:0/123621025 learned_addr learned my addr 172.21.15.45:0/123621025 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f86324b6700 1 --2- 172.21.15.45:0/123621025 >> v2:172.21.15.144:3300/0 conn(0x7f862c0cc390 0x7f862c0cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8631cb5700 1 -- 172.21.15.45:0/123621025 >> v2:172.21.15.144:3300/0 conn(0x7f862c0cc390 msgr2=0x7f862c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8631cb5700 1 --2- 172.21.15.45:0/123621025 >> v2:172.21.15.144:3300/0 conn(0x7f862c0cc390 0x7f862c0cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.973+0000 7f8631cb5700 1 -- 172.21.15.45:0/123621025 >> v1:172.21.15.45:6789/0 conn(0x7f862c06a060 legacy=0x7f862c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.974+0000 7f8631cb5700 1 -- 172.21.15.45:0/123621025 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0ccd10 con 0x7f862c158f60 2023-10-06T05:27:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.974+0000 7f8631cb5700 1 --2- 172.21.15.45:0/123621025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c158f60 0x7f862c159330 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ca4c0 tx=0x7f86280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d21644ae1fbbafd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.974+0000 7f8630cb3700 1 -- 172.21.15.45:0/123621025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800b6b0 con 0x7f862c158f60 2023-10-06T05:27:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.974+0000 7f8630cb3700 1 -- 172.21.15.45:0/123621025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862800b810 con 0x7f862c158f60 2023-10-06T05:27:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.974+0000 7f8630cb3700 1 -- 172.21.15.45:0/123621025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800bbb0 con 0x7f862c158f60 2023-10-06T05:27:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.974+0000 7f8633f19700 1 -- 172.21.15.45:0/123621025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c158f60 msgr2=0x7f862c159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.974+0000 7f8633f19700 1 --2- 172.21.15.45:0/123621025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c158f60 0x7f862c159330 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ca4c0 tx=0x7f86280091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.975+0000 7f8633f19700 1 -- 172.21.15.45:0/123621025 shutdown_connections 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.975+0000 7f8633f19700 1 --2- 172.21.15.45:0/123621025 >> v2:172.21.15.144:3300/0 conn(0x7f862c0cc390 0x7f862c0cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.975+0000 7f8633f19700 1 --2- 172.21.15.45:0/123621025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c158f60 0x7f862c159330 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.975+0000 7f8633f19700 1 -- 172.21.15.45:0/123621025 >> 172.21.15.45:0/123621025 conn(0x7f862c0cdf50 msgr2=0x7f862c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.975+0000 7f8633f19700 1 -- 172.21.15.45:0/123621025 shutdown_connections 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.975+0000 7f8633f19700 1 -- 172.21.15.45:0/123621025 wait complete. 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.976+0000 7f8633f19700 1 Processor -- start 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.976+0000 7f8633f19700 1 -- start start 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8633f19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c06a060 0x7f862c0c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8633f19700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c0cc390 0x7f862c0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8633f19700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c158f60 0x7f862c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8633f19700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0c6dd0 con 0x7f862c06a060 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8633f19700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c156930 con 0x7f862c158f60 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8633f19700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0c75c0 con 0x7f862c0cc390 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c06a060 0x7f862c0c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c06a060 0x7f862c0c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43766/0 (socket says 172.21.15.45:43766) 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 -- 172.21.15.45:0/1818340439 learned_addr learned my addr 172.21.15.45:0/1818340439 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 -- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c0cc390 msgr2=0x7f862c0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f86324b6700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c158f60 0x7f862c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c0cc390 0x7f862c0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 -- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c158f60 msgr2=0x7f862c155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c158f60 0x7f862c155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 -- 172.21.15.45:0/1818340439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628009420 con 0x7f862c06a060 2023-10-06T05:27:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f86324b6700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c158f60 0x7f862c155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8631cb5700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c06a060 0x7f862c0c5c40 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f862c0c4f80 tx=0x7f8628017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8621ffb700 1 -- 172.21.15.45:0/1818340439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628011420 con 0x7f862c06a060 2023-10-06T05:27:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8621ffb700 1 -- 172.21.15.45:0/1818340439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8628011580 con 0x7f862c06a060 2023-10-06T05:27:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c0c7000 con 0x7f862c06a060 2023-10-06T05:27:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.977+0000 7f8621ffb700 1 -- 172.21.15.45:0/1818340439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628011940 con 0x7f862c06a060 2023-10-06T05:27:00.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.978+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c1565c0 con 0x7f862c06a060 2023-10-06T05:27:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.979+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862c156420 con 0x7f862c06a060 2023-10-06T05:27:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.979+0000 7f8621ffb700 1 -- 172.21.15.45:0/1818340439 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8628011d30 con 0x7f862c06a060 2023-10-06T05:27:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.980+0000 7f8621ffb700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8610066f30 0x7f86100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.980+0000 7f8621ffb700 1 -- 172.21.15.45:0/1818340439 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f86280869a0 con 0x7f862c06a060 2023-10-06T05:27:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.980+0000 7f86314b4700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8610066f30 0x7f86100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:00.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.980+0000 7f86314b4700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8610066f30 0x7f86100693e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f862c06e4d0 tx=0x7f861c007b30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:00.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:00.984+0000 7f8621ffb700 1 -- 172.21.15.45:0/1818340439 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f862c156420 con 0x7f862c06a060 2023-10-06T05:27:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.236+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f862c156420 con 0x7f862c06a060 2023-10-06T05:27:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.237+0000 7f8621ffb700 1 -- 172.21.15.45:0/1818340439 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f862c156420 con 0x7f862c06a060 2023-10-06T05:27:01.239 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:01.239 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:58.006287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:27:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8610066f30 msgr2=0x7f86100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8610066f30 0x7f86100693e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f862c06e4d0 tx=0x7f861c007b30 comp rx=0 tx=0).stop 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c06a060 msgr2=0x7f862c0c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c06a060 0x7f862c0c5c40 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f862c0c4f80 tx=0x7f8628017000 comp rx=0 tx=0).stop 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 shutdown_connections 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8610066f30 0x7f86100693e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c158f60 0x7f862c155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f862c06a060 0x7f862c0c5c40 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 --2- 172.21.15.45:0/1818340439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c0cc390 0x7f862c0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 >> 172.21.15.45:0/1818340439 conn(0x7f862c0cdf50 msgr2=0x7f862c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 shutdown_connections 2023-10-06T05:27:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.239+0000 7f8633f19700 1 -- 172.21.15.45:0/1818340439 wait complete. 2023-10-06T05:27:01.243 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 350 2023-10-06T05:27:01.254 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:27:01.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:01.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.384+0000 7f45a438d700 1 Processor -- start 2023-10-06T05:27:01.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.384+0000 7f45a438d700 1 -- start start 2023-10-06T05:27:01.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.385+0000 7f45a438d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c159170 0x7f459c159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a438d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f459c15d040 0x7f459c068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a438d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f459c0ca6d0 con 0x7f459c069b90 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a438d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459c15d550 con 0x7f459c15d040 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a438d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c15d810 con 0x7f459c159170 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a1928700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f459c15d040 0x7f459c068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a2129700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c159170 0x7f459c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a292a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f459c069b90 0x7f459c0aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:41700/0 (socket says 172.21.15.45:41700) 2023-10-06T05:27:01.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a292a700 1 -- 172.21.15.45:0/924189921 learned_addr learned my addr 172.21.15.45:0/924189921 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a1127700 1 -- 172.21.15.45:0/924189921 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197942614 0 0) 0x7f459c0ca6d0 con 0x7f459c069b90 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a1127700 1 -- 172.21.15.45:0/924189921 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4588003570 con 0x7f459c069b90 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a2129700 1 -- 172.21.15.45:0/924189921 >> v2:172.21.15.144:3300/0 conn(0x7f459c15d040 msgr2=0x7f459c068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a2129700 1 --2- 172.21.15.45:0/924189921 >> v2:172.21.15.144:3300/0 conn(0x7f459c15d040 0x7f459c068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a2129700 1 -- 172.21.15.45:0/924189921 >> v1:172.21.15.144:6789/0 conn(0x7f459c069b90 legacy=0x7f459c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a2129700 1 -- 172.21.15.45:0/924189921 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c15daf0 con 0x7f459c159170 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.386+0000 7f45a1928700 1 --2- 172.21.15.45:0/924189921 >> v2:172.21.15.144:3300/0 conn(0x7f459c15d040 0x7f459c068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a2129700 1 --2- 172.21.15.45:0/924189921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c159170 0x7f459c159540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f45980098e0 tx=0x7f45980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f535c2cb6451463 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a1127700 1 -- 172.21.15.45:0/924189921 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598011630 con 0x7f459c159170 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a1127700 1 -- 172.21.15.45:0/924189921 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4598011790 con 0x7f459c159170 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a1127700 1 -- 172.21.15.45:0/924189921 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598011b30 con 0x7f459c159170 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a438d700 1 -- 172.21.15.45:0/924189921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c159170 msgr2=0x7f459c159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a438d700 1 --2- 172.21.15.45:0/924189921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c159170 0x7f459c159540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f45980098e0 tx=0x7f45980091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a438d700 1 -- 172.21.15.45:0/924189921 shutdown_connections 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a438d700 1 --2- 172.21.15.45:0/924189921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c159170 0x7f459c159540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a438d700 1 --2- 172.21.15.45:0/924189921 >> v2:172.21.15.144:3300/0 conn(0x7f459c15d040 0x7f459c068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.387+0000 7f45a438d700 1 -- 172.21.15.45:0/924189921 >> 172.21.15.45:0/924189921 conn(0x7f459c06f7c0 msgr2=0x7f459c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.388+0000 7f45a438d700 1 -- 172.21.15.45:0/924189921 shutdown_connections 2023-10-06T05:27:01.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.388+0000 7f45a438d700 1 -- 172.21.15.45:0/924189921 wait complete. 2023-10-06T05:27:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.389+0000 7f45a438d700 1 Processor -- start 2023-10-06T05:27:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.389+0000 7f45a438d700 1 -- start start 2023-10-06T05:27:01.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a438d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c069b90 0x7f459c159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a438d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f459c159170 0x7f459c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a438d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c15d040 0x7f459c16cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a438d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c15cdf0 con 0x7f459c15d040 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a438d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0b33f0 con 0x7f459c159170 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a438d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c15c580 con 0x7f459c069b90 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c15d040 0x7f459c16cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c15d040 0x7f459c16cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43768/0 (socket says 172.21.15.45:43768) 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 -- 172.21.15.45:0/581972339 learned_addr learned my addr 172.21.15.45:0/581972339 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a2129700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c069b90 0x7f459c159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a1928700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f459c159170 0x7f459c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 -- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c069b90 msgr2=0x7f459c159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c069b90 0x7f459c159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 -- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f459c159170 msgr2=0x7f459c150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f459c159170 0x7f459c150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 -- 172.21.15.45:0/581972339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c009420 con 0x7f459c15d040 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a292a700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c15d040 0x7f459c16cfe0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f4594001460 tx=0x7f4594001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45927fc700 1 -- 172.21.15.45:0/581972339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400ce80 con 0x7f459c15d040 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45927fc700 1 -- 172.21.15.45:0/581972339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400ab70 con 0x7f459c15d040 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.390+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4598009420 con 0x7f459c15d040 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.391+0000 7f45927fc700 1 -- 172.21.15.45:0/581972339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940070f0 con 0x7f459c15d040 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.391+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459c15cb10 con 0x7f459c15d040 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.392+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459c0b3580 con 0x7f459c15d040 2023-10-06T05:27:01.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.392+0000 7f45927fc700 1 -- 172.21.15.45:0/581972339 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4594007520 con 0x7f459c15d040 2023-10-06T05:27:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.392+0000 7f45927fc700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4580066f30 0x7f45800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.392+0000 7f45927fc700 1 -- 172.21.15.45:0/581972339 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f4594085a70 con 0x7f459c15d040 2023-10-06T05:27:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.392+0000 7f45a2129700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4580066f30 0x7f45800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:01.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.393+0000 7f45a2129700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4580066f30 0x7f45800693e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f45980091e0 tx=0x7f4598017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:01.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.397+0000 7f45927fc700 1 -- 172.21.15.45:0/581972339 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4594051d10 con 0x7f459c15d040 2023-10-06T05:27:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.646+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f459c063b50 con 0x7f459c15d040 2023-10-06T05:27:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.647+0000 7f45927fc700 1 -- 172.21.15.45:0/581972339 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f4594055330 con 0x7f459c15d040 2023-10-06T05:27:01.648 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:01.648 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:58.006287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4580066f30 msgr2=0x7f45800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4580066f30 0x7f45800693e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f45980091e0 tx=0x7f4598017000 comp rx=0 tx=0).stop 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c15d040 msgr2=0x7f459c16cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c15d040 0x7f459c16cfe0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f4594001460 tx=0x7f4594001ae0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 shutdown_connections 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4580066f30 0x7f45800693e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c15d040 0x7f459c16cfe0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f459c069b90 0x7f459c159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 --2- 172.21.15.45:0/581972339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f459c159170 0x7f459c150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 >> 172.21.15.45:0/581972339 conn(0x7f459c06f7c0 msgr2=0x7f459c0cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 shutdown_connections 2023-10-06T05:27:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:01.649+0000 7f45a438d700 1 -- 172.21.15.45:0/581972339 wait complete. 2023-10-06T05:27:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 350 2023-10-06T05:27:01.664 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2023-10-06T05:27:01.665 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2023-10-06T05:27:01.665 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:27:01.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:01.718 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:01.769 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:27:01.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:27:01.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:01.769 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:27:01.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-06T05:27:01.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:27:01.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:01.792 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:27:01.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:01.813 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T05:27:01.813 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:27:01.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:01.813 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:27:01.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:01.856 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:27:01.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:27:01.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:01.856 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:27:01.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:01.876 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:27:01.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.039+0000 7f54dae9b700 1 Processor -- start 2023-10-06T05:27:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.039+0000 7f54dae9b700 1 -- start start 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.040+0000 7f54dae9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d40677e0 0x7f54d4067bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.040+0000 7f54dae9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f54d415b3e0 0x7f54d4140410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.040+0000 7f54dae9b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d40c9ed0 con 0x7f54d40ca360 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.040+0000 7f54dae9b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d40cd870 con 0x7f54d415b3e0 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.040+0000 7f54dae9b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d415bc00 con 0x7f54d40677e0 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.040+0000 7f54d3fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f54d40ca360 0x7f54d415acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47104/0 (socket says 172.21.15.45:47104) 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.040+0000 7f54d3fff700 1 -- 172.21.15.45:0/4269174027 learned_addr learned my addr 172.21.15.45:0/4269174027 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d8c37700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d40677e0 0x7f54d4067bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d9438700 1 --2- 172.21.15.45:0/4269174027 >> v2:172.21.15.144:3300/0 conn(0x7f54d415b3e0 0x7f54d4140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5552669 0 0) 0x7f54d40c9ed0 con 0x7f54d40ca360 2023-10-06T05:27:02.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54bc003570 con 0x7f54d40ca360 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2800427603 0 0) 0x7f54bc003570 con 0x7f54d40ca360 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d40677e0 msgr2=0x7f54d4067bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d40677e0 0x7f54d4067bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 >> v2:172.21.15.144:3300/0 conn(0x7f54d415b3e0 msgr2=0x7f54d4140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 --2- 172.21.15.45:0/4269174027 >> v2:172.21.15.144:3300/0 conn(0x7f54d415b3e0 0x7f54d4140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d415b920 con 0x7f54d40ca360 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f54c8001470 con 0x7f54d40ca360 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 >> v1:172.21.15.45:6789/0 conn(0x7f54d40ca360 legacy=0x7f54d415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54bc003a60 0x7f54bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54bc004810 0x7f54bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54bc0051c0 0x7f54bc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40cd870 con 0x7f54bc0051c0 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d415bc00 con 0x7f54bc003a60 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc00bfa0 con 0x7f54bc004810 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d8c37700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d40677e0 0x7f54d4067bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d9438700 1 --2- 172.21.15.45:0/4269174027 >> v2:172.21.15.144:3300/0 conn(0x7f54d415b3e0 0x7f54d4140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d9438700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54bc0051c0 0x7f54bc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d3fff700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54bc004810 0x7f54bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.041+0000 7f54d8c37700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54bc003a60 0x7f54bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d9438700 1 -- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54bc004810 msgr2=0x7f54bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d9438700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54bc004810 0x7f54bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d9438700 1 -- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54bc003a60 msgr2=0x7f54bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d9438700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54bc003a60 0x7f54bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d9438700 1 -- 172.21.15.45:0/4269174027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc003570 con 0x7f54bc0051c0 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d9438700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54bc0051c0 0x7f54bc00ba60 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00e3f0 tx=0x7f54cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c5ab1dc7b4380b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00ec40 con 0x7f54bc0051c0 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc00f040 con 0x7f54bc0051c0 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d37fe700 1 -- 172.21.15.45:0/4269174027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc014480 con 0x7f54bc0051c0 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54dae9b700 1 -- 172.21.15.45:0/4269174027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54bc0051c0 msgr2=0x7f54bc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54dae9b700 1 --2- 172.21.15.45:0/4269174027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54bc0051c0 0x7f54bc00ba60 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00e3f0 tx=0x7f54cc00b040 comp rx=0 tx=0).stop 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.042+0000 7f54d9438700 1 -- 172.21.15.45:0/4269174027 reap_dead start 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.043+0000 7f54dae9b700 1 -- 172.21.15.45:0/4269174027 shutdown_connections 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.043+0000 7f54dae9b700 1 -- 172.21.15.45:0/4269174027 >> 172.21.15.45:0/4269174027 conn(0x7f54d4069200 msgr2=0x7f54d40cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.043+0000 7f54dae9b700 1 -- 172.21.15.45:0/4269174027 shutdown_connections 2023-10-06T05:27:02.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.043+0000 7f54dae9b700 1 -- 172.21.15.45:0/4269174027 wait complete. 2023-10-06T05:27:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.044+0000 7f54dae9b700 1 Processor -- start 2023-10-06T05:27:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.045+0000 7f54dae9b700 1 -- start start 2023-10-06T05:27:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.045+0000 7f54dae9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54d4153ce0 0x7f54d4159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.045+0000 7f54dae9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d41540b0 0x7f54d40b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.045+0000 7f54dae9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d40b2480 0x7f54d416b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.045+0000 7f54dae9b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40b2900 con 0x7f54d40b2480 2023-10-06T05:27:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.045+0000 7f54dae9b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d4159570 con 0x7f54d4153ce0 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.045+0000 7f54dae9b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d4159f40 con 0x7f54d41540b0 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d40b2480 0x7f54d416b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d40b2480 0x7f54d416b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43790/0 (socket says 172.21.15.45:43790) 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 -- 172.21.15.45:0/2337143672 learned_addr learned my addr 172.21.15.45:0/2337143672 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d8c37700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54d4153ce0 0x7f54d4159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d3fff700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d41540b0 0x7f54d40b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 -- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d41540b0 msgr2=0x7f54d40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d41540b0 0x7f54d40b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 -- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54d4153ce0 msgr2=0x7f54d4159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54d4153ce0 0x7f54d4159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 -- 172.21.15.45:0/2337143672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d41589a0 con 0x7f54d40b2480 2023-10-06T05:27:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d9438700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d40b2480 0x7f54d416b0e0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f54cc003000 tx=0x7f54cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d0ff9700 1 -- 172.21.15.45:0/2337143672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc014480 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d0ff9700 1 -- 172.21.15.45:0/2337143672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc01f450 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d4158c40 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54d0ff9700 1 -- 172.21.15.45:0/2337143672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc01f810 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.046+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d4159200 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.047+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d406dfd0 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.048+0000 7f54d0ff9700 1 -- 172.21.15.45:0/2337143672 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54cc01fc40 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.048+0000 7f54d0ff9700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54c0066f30 0x7f54c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.048+0000 7f54d0ff9700 1 -- 172.21.15.45:0/2337143672 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f54cc087b40 con 0x7f54d40b2480 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.049+0000 7f54d8c37700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54c0066f30 0x7f54c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.049+0000 7f54d8c37700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54c0066f30 0x7f54c00693e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f54d40ca910 tx=0x7f54c40091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.053+0000 7f54d0ff9700 1 -- 172.21.15.45:0/2337143672 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54cc053e20 con 0x7f54d40b2480 2023-10-06T05:27:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.308+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54d41596b0 con 0x7f54d40b2480 2023-10-06T05:27:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.308+0000 7f54d0ff9700 1 -- 172.21.15.45:0/2337143672 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f54cc02d020 con 0x7f54d40b2480 2023-10-06T05:27:02.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:02.311 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:58.006287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:27:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54c0066f30 msgr2=0x7f54c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54c0066f30 0x7f54c00693e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f54d40ca910 tx=0x7f54c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d40b2480 msgr2=0x7f54d416b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d40b2480 0x7f54d416b0e0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f54cc003000 tx=0x7f54cc019000 comp rx=0 tx=0).stop 2023-10-06T05:27:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 shutdown_connections 2023-10-06T05:27:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f54c0066f30 0x7f54c00693e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d40b2480 0x7f54d416b0e0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54d4153ce0 0x7f54d4159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 --2- 172.21.15.45:0/2337143672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54d41540b0 0x7f54d40b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 >> 172.21.15.45:0/2337143672 conn(0x7f54d4069200 msgr2=0x7f54d415ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 shutdown_connections 2023-10-06T05:27:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.310+0000 7f54dae9b700 1 -- 172.21.15.45:0/2337143672 wait complete. 2023-10-06T05:27:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 350 2023-10-06T05:27:02.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:02.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.460+0000 7f59f4661700 1 Processor -- start 2023-10-06T05:27:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.460+0000 7f59f4661700 1 -- start start 2023-10-06T05:27:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f4661700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59ec06c3d0 0x7f59ec06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f4661700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59ec15b890 0x7f59ec0afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f4661700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0b0790 con 0x7f59ec156b70 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f4661700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0c8e80 con 0x7f59ec06c3d0 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f4661700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0c9160 con 0x7f59ec15b890 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f2bfe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59ec156b70 0x7f59ec156fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47120/0 (socket says 172.21.15.45:47120) 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f2bfe700 1 -- 172.21.15.45:0/3461685701 learned_addr learned my addr 172.21.15.45:0/3461685701 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f23fd700 1 --2- 172.21.15.45:0/3461685701 >> v2:172.21.15.45:3300/0 conn(0x7f59ec06c3d0 0x7f59ec06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f1bfc700 1 --2- 172.21.15.45:0/3461685701 >> v2:172.21.15.144:3300/0 conn(0x7f59ec15b890 0x7f59ec0afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f13fb700 1 -- 172.21.15.45:0/3461685701 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257114790 0 0) 0x7f59ec0b0790 con 0x7f59ec156b70 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f13fb700 1 -- 172.21.15.45:0/3461685701 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d8003570 con 0x7f59ec156b70 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f23fd700 1 -- 172.21.15.45:0/3461685701 >> v2:172.21.15.144:3300/0 conn(0x7f59ec15b890 msgr2=0x7f59ec0afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f23fd700 1 --2- 172.21.15.45:0/3461685701 >> v2:172.21.15.144:3300/0 conn(0x7f59ec15b890 0x7f59ec0afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f23fd700 1 -- 172.21.15.45:0/3461685701 >> v1:172.21.15.45:6789/0 conn(0x7f59ec156b70 legacy=0x7f59ec156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f23fd700 1 -- 172.21.15.45:0/3461685701 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0c9440 con 0x7f59ec06c3d0 2023-10-06T05:27:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.461+0000 7f59f23fd700 1 --2- 172.21.15.45:0/3461685701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec06c7a0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0cb880 tx=0x7f59e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca691080c5d3e6c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f13fb700 1 -- 172.21.15.45:0/3461685701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8011630 con 0x7f59ec06c3d0 2023-10-06T05:27:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f13fb700 1 -- 172.21.15.45:0/3461685701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e8011790 con 0x7f59ec06c3d0 2023-10-06T05:27:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f13fb700 1 -- 172.21.15.45:0/3461685701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8011b30 con 0x7f59ec06c3d0 2023-10-06T05:27:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 -- 172.21.15.45:0/3461685701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 msgr2=0x7f59ec06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 --2- 172.21.15.45:0/3461685701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec06c7a0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0cb880 tx=0x7f59e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 -- 172.21.15.45:0/3461685701 shutdown_connections 2023-10-06T05:27:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 --2- 172.21.15.45:0/3461685701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec06c7a0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 --2- 172.21.15.45:0/3461685701 >> v2:172.21.15.144:3300/0 conn(0x7f59ec15b890 0x7f59ec0afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 -- 172.21.15.45:0/3461685701 >> 172.21.15.45:0/3461685701 conn(0x7f59ec067620 msgr2=0x7f59ec067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 -- 172.21.15.45:0/3461685701 shutdown_connections 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.462+0000 7f59f4661700 1 -- 172.21.15.45:0/3461685701 wait complete. 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 Processor -- start 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 -- start start 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec156b70 0x7f59ec16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec15b890 0x7f59ec0c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec06a4e0 con 0x7f59ec06c3d0 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec069d10 con 0x7f59ec15b890 2023-10-06T05:27:02.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.464+0000 7f59f4661700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0c2100 con 0x7f59ec156b70 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43812/0 (socket says 172.21.15.45:43812) 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 -- 172.21.15.45:0/280001627 learned_addr learned my addr 172.21.15.45:0/280001627 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f1bfc700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec156b70 0x7f59ec16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f2bfe700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec15b890 0x7f59ec0c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 -- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec156b70 msgr2=0x7f59ec16a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec156b70 0x7f59ec16a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 -- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec15b890 msgr2=0x7f59ec0c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec15b890 0x7f59ec0c2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 -- 172.21.15.45:0/280001627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8009420 con 0x7f59ec06c3d0 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f23fd700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec155310 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009f50 tx=0x7f59e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59e27fc700 1 -- 172.21.15.45:0/280001627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8011b10 con 0x7f59ec06c3d0 2023-10-06T05:27:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59e27fc700 1 -- 172.21.15.45:0/280001627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e80233f0 con 0x7f59ec06c3d0 2023-10-06T05:27:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59e27fc700 1 -- 172.21.15.45:0/280001627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8023790 con 0x7f59ec06c3d0 2023-10-06T05:27:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec069a30 con 0x7f59ec06c3d0 2023-10-06T05:27:02.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.465+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec0c1d90 con 0x7f59ec06c3d0 2023-10-06T05:27:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.466+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec06f160 con 0x7f59ec06c3d0 2023-10-06T05:27:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.467+0000 7f59e27fc700 1 -- 172.21.15.45:0/280001627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59e8023b40 con 0x7f59ec06c3d0 2023-10-06T05:27:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.467+0000 7f59e27fc700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59d0066f30 0x7f59d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.467+0000 7f59e27fc700 1 -- 172.21.15.45:0/280001627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f59e8086930 con 0x7f59ec06c3d0 2023-10-06T05:27:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.468+0000 7f59f1bfc700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59d0066f30 0x7f59d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.468+0000 7f59f1bfc700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59d0066f30 0x7f59d00693e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0692c0 tx=0x7f59dc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.472+0000 7f59e27fc700 1 -- 172.21.15.45:0/280001627 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59e8052b70 con 0x7f59ec06c3d0 2023-10-06T05:27:02.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.677+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59ec0c9720 con 0x7f59ec06c3d0 2023-10-06T05:27:02.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.678+0000 7f59e27fc700 1 -- 172.21.15.45:0/280001627 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+22388 (secure 0 0 0) 0x7f59e8015070 con 0x7f59ec06c3d0 2023-10-06T05:27:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:02.680 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":132,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:26:58.006054+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:49.182551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:49.732208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2681459400":"2023-10-07T05:23:52.452679+0000","172.21.15.45:6835/2971431427":"2023-10-07T05:24:37.100960+0000","172.21.15.45:6834/2681459400":"2023-10-07T05:23:52.452679+0000","172.21.15.45:6835/1013285853":"2023-10-07T05:23:45.414617+0000","172.21.15.45:6835/858773096":"2023-10-07T05:24:07.945434+0000","172.21.15.144:6834/4239891954":"2023-10-07T05:25:03.377085+0000","172.21.15.45:6834/3031536032":"2023-10-07T05:24:22.864925+0000","172.21.15.45:6834/3141743890":"2023-10-07T05:23:23.111877+0000","172.21.15.45:6834/681980478":"2023-10-07T05:23:12.050135+0000","172.21.15.45:6835/2532567321":"2023-10-07T05:23:15.870223+0000","172.21.15.45:6834/2405580153":"2023-10-07T05:23:43.392415+0000","172.21.15.45:6834/2971431427":"2023-10-07T05:24:37.100960+0000","172.21.15.144:6834/3602644226":"2023-10-07T05:24:45.157829+0000","172.21.15.144:6834/1302691811":"2023-10-07T05:26:43.923386+0000","172.21.15.45:6832/3755910187":"2023-10-07T05:22:28.384687+0000","172.21.15.144:6835/2001233142":"2023-10-07T05:26:17.517661+0000","172.21.15.45:6832/2550360138":"2023-10-07T05:22:23.359215+0000","172.21.15.45:6832/821082350":"2023-10-07T05:22:07.114456+0000","172.21.15.45:6835/2405580153":"2023-10-07T05:23:43.392415+0000","172.21.15.45:6832/3860010011":"2023-10-07T05:22:38.914897+0000","172.21.15.45:6835/3141743890":"2023-10-07T05:23:23.111877+0000","172.21.15.45:6832/4128377575":"2023-10-07T05:22:34.886263+0000","172.21.15.45:6835/2475927951":"2023-10-07T05:24:29.055945+0000","172.21.15.45:6833/4128377575":"2023-10-07T05:22:34.886263+0000","172.21.15.144:6834/3552713853":"2023-10-07T05:25:38.094009+0000","172.21.15.45:6833/2550360138":"2023-10-07T05:22:23.359215+0000","172.21.15.45:6834/858773096":"2023-10-07T05:24:07.945434+0000","172.21.15.45:6834/2475927951":"2023-10-07T05:24:29.055945+0000","172.21.15.45:6833/821082350":"2023-10-07T05:22:07.114456+0000","172.21.15.45:6834/3351321197":"2023-10-07T05:23:59.498576+0000","172.21.15.45:6833/3860010011":"2023-10-07T05:22:38.914897+0000","172.21.15.144:6835/2237111349":"2023-10-07T05:25:17.948375+0000","172.21.15.45:6835/681980478":"2023-10-07T05:23:12.050135+0000","172.21.15.45:6835/1996698078":"2023-10-07T05:24:25.023244+0000","172.21.15.45:6835/1915871760":"2023-10-07T05:24:03.901047+0000","172.21.15.45:6835/3031536032":"2023-10-07T05:24:22.864925+0000","172.21.15.45:6833/3755910187":"2023-10-07T05:22:28.384687+0000","172.21.15.45:6835/3277956876":"2023-10-07T05:22:42.945097+0000","172.21.15.45:6834/1915871760":"2023-10-07T05:24:03.901047+0000","172.21.15.45:6834/226461717":"2023-10-07T05:22:56.431657+0000","172.21.15.144:6835/4239891954":"2023-10-07T05:25:03.377085+0000","172.21.15.45:6834/2620071126":"2023-10-07T05:23:29.906056+0000","172.21.15.144:6835/3580730449":"2023-10-07T05:25:05.403592+0000","172.21.15.45:6834/3277956876":"2023-10-07T05:22:42.945097+0000","172.21.15.45:6835/226461717":"2023-10-07T05:22:56.431657+0000","172.21.15.45:6835/2531263654":"2023-10-07T05:23:09.893451+0000","172.21.15.45:6834/1013285853":"2023-10-07T05:23:45.414617+0000","172.21.15.45:6834/1996698078":"2023-10-07T05:24:25.023244+0000","172.21.15.144:6834/3433217004":"2023-10-07T05:24:33.068465+0000","172.21.15.45:6835/2620071126":"2023-10-07T05:23:29.906056+0000","172.21.15.144:6834/2156055156":"2023-10-07T05:24:40.893717+0000","172.21.15.144:6835/2156055156":"2023-10-07T05:24:40.893717+0000","172.21.15.144:6835/3602644226":"2023-10-07T05:24:45.157829+0000","172.21.15.144:6835/1309688991":"2023-10-07T05:24:49.191270+0000","172.21.15.144:6834/2062728889":"2023-10-07T05:26:06.022104+0000","172.21.15.144:6834/3580730449":"2023-10-07T05:25:05.403592+0000","172.21.15.144:6835/913913762":"2023-10-07T05:26:58.006032+0000","172.21.15.144:6834/3983487029":"2023-10-07T05:25:13.909730+0000","172.21.15.144:6835/3451214191":"2023-10-07T05:26:22.547760+0000","172.21.15.144:6834/243455709":"2023-10-07T05:25:09.429589+0000","172.21.15.144:6835/243455709":"2023-10-07T05:25:09.429589+0000","172.21.15.144:6835/2062728889":"2023-10-07T05:26:06.022104+0000","172.21.15.144:6834/913913762":"2023-10-07T05:26:58.006032+0000","172.21.15.144:6835/3983487029":"2023-10-07T05:25:13.909730+0000","172.21.15.144:6835/26720443":"2023-10-07T05:25:43.120106+0000","172.21.15.144:6835/3552713853":"2023-10-07T05:25:38.094009+0000","172.21.15.144:6834/1309688991":"2023-10-07T05:24:49.191270+0000","172.21.15.144:6834/26720443":"2023-10-07T05:25:43.120106+0000","172.21.15.144:6834/2001233142":"2023-10-07T05:26:17.517661+0000","172.21.15.144:6835/1309152247":"2023-10-07T05:26:50.920566+0000","172.21.15.144:6834/3451214191":"2023-10-07T05:26:22.547760+0000","172.21.15.45:6835/3351321197":"2023-10-07T05:23:59.498576+0000","172.21.15.144:6835/3433217004":"2023-10-07T05:24:33.068465+0000","172.21.15.144:6834/603338967":"2023-10-07T05:26:39.088317+0000","172.21.15.144:6834/2237111349":"2023-10-07T05:25:17.948375+0000","172.21.15.144:6835/603338967":"2023-10-07T05:26:39.088317+0000","172.21.15.144:6835/1302691811":"2023-10-07T05:26:43.923386+0000","172.21.15.45:6834/2531263654":"2023-10-07T05:23:09.893451+0000","172.21.15.45:6834/2532567321":"2023-10-07T05:23:15.870223+0000","172.21.15.144:6834/1309152247":"2023-10-07T05:26:50.920566+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:02.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59d0066f30 msgr2=0x7f59d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59d0066f30 0x7f59d00693e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0692c0 tx=0x7f59dc008040 comp rx=0 tx=0).stop 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 msgr2=0x7f59ec155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec155310 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009f50 tx=0x7f59e8017000 comp rx=0 tx=0).stop 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 shutdown_connections 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f59d0066f30 0x7f59d00693e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec15b890 0x7f59ec0c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59ec06c3d0 0x7f59ec155310 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 --2- 172.21.15.45:0/280001627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec156b70 0x7f59ec16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 >> 172.21.15.45:0/280001627 conn(0x7f59ec067620 msgr2=0x7f59ec1562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 shutdown_connections 2023-10-06T05:27:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.680+0000 7f59f4661700 1 -- 172.21.15.45:0/280001627 wait complete. 2023-10-06T05:27:02.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:21:49.182551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:21:49.732208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:27:02.696 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:27:02.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.830+0000 7feadaec5700 1 Processor -- start 2023-10-06T05:27:02.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.830+0000 7feadaec5700 1 -- start start 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7feadaec5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b1cb0 0x7fead406a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7feadaec5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead4068750 con 0x7fead40b18e0 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7feadaec5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead406b190 con 0x7fead406ad60 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7feadaec5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead4067860 con 0x7fead40b1cb0 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7fead8c61700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fead40b18e0 0x7fead415b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47134/0 (socket says 172.21.15.45:47134) 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7fead8c61700 1 -- 172.21.15.45:0/650607323 learned_addr learned my addr 172.21.15.45:0/650607323 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7fead3fff700 1 --2- 172.21.15.45:0/650607323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b1cb0 0x7fead406a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054086180 0 0) 0x7fead4068750 con 0x7fead40b18e0 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.831+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac4003570 con 0x7fead40b18e0 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681562699 0 0) 0x7fead406b190 con 0x7fead406ad60 2023-10-06T05:27:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead4068750 con 0x7fead406ad60 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083104282 0 0) 0x7feac4003570 con 0x7fead40b18e0 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead3fff700 1 -- 172.21.15.45:0/650607323 >> v1:172.21.15.144:6789/0 conn(0x7fead406ad60 legacy=0x7fead40ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead3fff700 1 -- 172.21.15.45:0/650607323 >> v1:172.21.15.45:6789/0 conn(0x7fead40b18e0 legacy=0x7fead415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead3fff700 1 -- 172.21.15.45:0/650607323 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead4067b40 con 0x7fead40b1cb0 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead3fff700 1 --2- 172.21.15.45:0/650607323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b1cb0 0x7fead406a820 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feac0009910 tx=0x7feac00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad4970c4abfcec2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac8001470 con 0x7fead40b1cb0 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac0011420 con 0x7fead40b1cb0 2023-10-06T05:27:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.832+0000 7fead37fe700 1 -- 172.21.15.45:0/650607323 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00117c0 con 0x7fead40b1cb0 2023-10-06T05:27:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.833+0000 7feadaec5700 1 -- 172.21.15.45:0/650607323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b1cb0 msgr2=0x7fead406a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.833+0000 7feadaec5700 1 --2- 172.21.15.45:0/650607323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b1cb0 0x7fead406a820 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feac0009910 tx=0x7feac00091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.833+0000 7feadaec5700 1 -- 172.21.15.45:0/650607323 shutdown_connections 2023-10-06T05:27:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.833+0000 7feadaec5700 1 --2- 172.21.15.45:0/650607323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b1cb0 0x7fead406a820 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.833+0000 7feadaec5700 1 -- 172.21.15.45:0/650607323 >> 172.21.15.45:0/650607323 conn(0x7fead40cc570 msgr2=0x7fead40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.833+0000 7feadaec5700 1 -- 172.21.15.45:0/650607323 shutdown_connections 2023-10-06T05:27:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.833+0000 7feadaec5700 1 -- 172.21.15.45:0/650607323 wait complete. 2023-10-06T05:27:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.834+0000 7feadaec5700 1 Processor -- start 2023-10-06T05:27:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.834+0000 7feadaec5700 1 -- start start 2023-10-06T05:27:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7feadaec5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead406ad60 0x7fead4154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7feadaec5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead40b18e0 0x7fead41554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7feadaec5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40b1cb0 0x7fead40c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7feadaec5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40c29b0 con 0x7fead40b1cb0 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7feadaec5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40c5260 con 0x7fead40b18e0 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7feadaec5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40c6f20 con 0x7fead406ad60 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40b1cb0 0x7fead40c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40b1cb0 0x7fead40c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43820/0 (socket says 172.21.15.45:43820) 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 -- 172.21.15.45:0/3020763049 learned_addr learned my addr 172.21.15.45:0/3020763049 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead8c61700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead406ad60 0x7fead4154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead3fff700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead40b18e0 0x7fead41554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 -- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead406ad60 msgr2=0x7fead4154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead406ad60 0x7fead4154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 -- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead40b18e0 msgr2=0x7fead41554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead40b18e0 0x7fead41554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead9462700 1 -- 172.21.15.45:0/3020763049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0009420 con 0x7fead40b1cb0 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead8c61700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead406ad60 0x7fead4154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.835+0000 7fead3fff700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead40b18e0 0x7fead41554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.836+0000 7fead9462700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40b1cb0 0x7fead40c3a00 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7feacc00c910 tx=0x7feacc0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.836+0000 7fead0ff9700 1 -- 172.21.15.45:0/3020763049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00a7e0 con 0x7fead40b1cb0 2023-10-06T05:27:02.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.836+0000 7fead0ff9700 1 -- 172.21.15.45:0/3020763049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc00a960 con 0x7fead40b1cb0 2023-10-06T05:27:02.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.836+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead40c4ca0 con 0x7fead40b1cb0 2023-10-06T05:27:02.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.836+0000 7fead0ff9700 1 -- 172.21.15.45:0/3020763049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00ad20 con 0x7fead40b1cb0 2023-10-06T05:27:02.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.836+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead40c6bb0 con 0x7fead40b1cb0 2023-10-06T05:27:02.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.837+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead40c7090 con 0x7fead40b1cb0 2023-10-06T05:27:02.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.837+0000 7fead0ff9700 1 -- 172.21.15.45:0/3020763049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feacc012630 con 0x7fead40b1cb0 2023-10-06T05:27:02.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.838+0000 7fead0ff9700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feab8066f30 0x7feab80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:02.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.838+0000 7fead0ff9700 1 -- 172.21.15.45:0/3020763049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7feacc085a40 con 0x7fead40b1cb0 2023-10-06T05:27:02.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.838+0000 7fead8c61700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feab8066f30 0x7feab80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:02.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.839+0000 7fead8c61700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feab8066f30 0x7feab80693e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7feac80011e0 tx=0x7feac8000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:02.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:02.842+0000 7fead0ff9700 1 -- 172.21.15.45:0/3020763049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feacc087e60 con 0x7fead40b1cb0 2023-10-06T05:27:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.047+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fead404bb40 con 0x7fead40b1cb0 2023-10-06T05:27:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.047+0000 7fead0ff9700 1 -- 172.21.15.45:0/3020763049 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7feacc0160a0 con 0x7fead40b1cb0 2023-10-06T05:27:03.049 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:27:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feab8066f30 msgr2=0x7feab80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feab8066f30 0x7feab80693e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7feac80011e0 tx=0x7feac8000bc0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40b1cb0 msgr2=0x7fead40c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40b1cb0 0x7fead40c3a00 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7feacc00c910 tx=0x7feacc0027a0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 shutdown_connections 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7feab8066f30 0x7feab80693e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40b1cb0 0x7fead40c3a00 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead406ad60 0x7fead4154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 --2- 172.21.15.45:0/3020763049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead40b18e0 0x7fead41554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 >> 172.21.15.45:0/3020763049 conn(0x7fead40cc570 msgr2=0x7fead4156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 shutdown_connections 2023-10-06T05:27:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.049+0000 7feadaec5700 1 -- 172.21.15.45:0/3020763049 wait complete. 2023-10-06T05:27:03.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:27:03.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.199+0000 7fb5d9829700 1 Processor -- start 2023-10-06T05:27:03.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.199+0000 7fb5d9829700 1 -- start start 2023-10-06T05:27:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.200+0000 7fb5d9829700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d40ada80 0x7fb5d4163e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.200+0000 7fb5d9829700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d4067d80 con 0x7fb5d406aa70 2023-10-06T05:27:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.200+0000 7fb5d9829700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d40c75b0 con 0x7fb5d406ae40 2023-10-06T05:27:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.200+0000 7fb5d9829700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40ac620 con 0x7fb5d40ada80 2023-10-06T05:27:03.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.200+0000 7fb5d2ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb5d406aa70 0x7fb5d40c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47148/0 (socket says 172.21.15.45:47148) 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.200+0000 7fb5d2ffd700 1 -- 172.21.15.45:0/2790306078 learned_addr learned my addr 172.21.15.45:0/2790306078 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d37fe700 1 --2- 172.21.15.45:0/2790306078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d40ada80 0x7fb5d4163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718245349 0 0) 0x7fb5d4067d80 con 0x7fb5d406aa70 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5bc003570 con 0x7fb5d406aa70 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448640698 0 0) 0x7fb5d40c75b0 con 0x7fb5d406ae40 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d4067d80 con 0x7fb5d406ae40 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2844098181 0 0) 0x7fb5bc003570 con 0x7fb5d406aa70 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d37fe700 1 -- 172.21.15.45:0/2790306078 >> v1:172.21.15.144:6789/0 conn(0x7fb5d406ae40 legacy=0x7fb5d415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d37fe700 1 -- 172.21.15.45:0/2790306078 >> v1:172.21.15.45:6789/0 conn(0x7fb5d406aa70 legacy=0x7fb5d40c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d37fe700 1 -- 172.21.15.45:0/2790306078 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40c7890 con 0x7fb5d40ada80 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d37fe700 1 --2- 172.21.15.45:0/2790306078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d40ada80 0x7fb5d4163e40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e90 tx=0x7fb5cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3701e3575de23984 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.201+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00e070 con 0x7fb5d40ada80 2023-10-06T05:27:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00b870 con 0x7fb5d40ada80 2023-10-06T05:27:03.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d1ffb700 1 -- 172.21.15.45:0/2790306078 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4001470 con 0x7fb5d40ada80 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d9829700 1 -- 172.21.15.45:0/2790306078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d40ada80 msgr2=0x7fb5d4163e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d9829700 1 --2- 172.21.15.45:0/2790306078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d40ada80 0x7fb5d4163e40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e90 tx=0x7fb5cc009780 comp rx=0 tx=0).stop 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d9829700 1 -- 172.21.15.45:0/2790306078 shutdown_connections 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d9829700 1 --2- 172.21.15.45:0/2790306078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d40ada80 0x7fb5d4163e40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d9829700 1 -- 172.21.15.45:0/2790306078 >> 172.21.15.45:0/2790306078 conn(0x7fb5d40cc570 msgr2=0x7fb5d40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d9829700 1 -- 172.21.15.45:0/2790306078 shutdown_connections 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.202+0000 7fb5d9829700 1 -- 172.21.15.45:0/2790306078 wait complete. 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 Processor -- start 2023-10-06T05:27:03.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 -- start start 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d406aa70 0x7fb5d40cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d406ae40 0x7fb5d40c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d40ada80 0x7fb5d40af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40c1b50 con 0x7fb5d406aa70 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40cb730 con 0x7fb5d40ada80 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.204+0000 7fb5d9829700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40cbd60 con 0x7fb5d406ae40 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d406aa70 0x7fb5d40cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d406aa70 0x7fb5d40cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43824/0 (socket says 172.21.15.45:43824) 2023-10-06T05:27:03.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 -- 172.21.15.45:0/3867000143 learned_addr learned my addr 172.21.15.45:0/3867000143 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d37fe700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d40ada80 0x7fb5d40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d27fc700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d406ae40 0x7fb5d40c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 -- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d406ae40 msgr2=0x7fb5d40c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d406ae40 0x7fb5d40c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 -- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d40ada80 msgr2=0x7fb5d40af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d40ada80 0x7fb5d40af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 -- 172.21.15.45:0/3867000143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc009a20 con 0x7fb5d406aa70 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d2ffd700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d406aa70 0x7fb5d40cb150 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400cb90 tx=0x7fb5c4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5baffd700 1 -- 172.21.15.45:0/3867000143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4007150 con 0x7fb5d406aa70 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5baffd700 1 -- 172.21.15.45:0/3867000143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c40072f0 con 0x7fb5d406aa70 2023-10-06T05:27:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5baffd700 1 -- 172.21.15.45:0/3867000143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40076b0 con 0x7fb5d406aa70 2023-10-06T05:27:03.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d40c1260 con 0x7fb5d406aa70 2023-10-06T05:27:03.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.205+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d40c17e0 con 0x7fb5d406aa70 2023-10-06T05:27:03.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.206+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d406e300 con 0x7fb5d406aa70 2023-10-06T05:27:03.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.207+0000 7fb5baffd700 1 -- 172.21.15.45:0/3867000143 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5c4004a40 con 0x7fb5d406aa70 2023-10-06T05:27:03.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.207+0000 7fb5baffd700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5c0066f30 0x7fb5c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.207+0000 7fb5baffd700 1 -- 172.21.15.45:0/3867000143 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7fb5c4084850 con 0x7fb5d406aa70 2023-10-06T05:27:03.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.208+0000 7fb5d27fc700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5c0066f30 0x7fb5c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.208+0000 7fb5d27fc700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5c0066f30 0x7fb5c00693e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4070f60 tx=0x7fb5c8007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.212+0000 7fb5baffd700 1 -- 172.21.15.45:0/3867000143 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5c4050af0 con 0x7fb5d406aa70 2023-10-06T05:27:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.410+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb5d40c7b70 con 0x7fb5d406aa70 2023-10-06T05:27:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.410+0000 7fb5baffd700 1 -- 172.21.15.45:0/3867000143 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb5c4018070 con 0x7fb5d406aa70 2023-10-06T05:27:03.412 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5c0066f30 msgr2=0x7fb5c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5c0066f30 0x7fb5c00693e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4070f60 tx=0x7fb5c8007d50 comp rx=0 tx=0).stop 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d406aa70 msgr2=0x7fb5d40cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d406aa70 0x7fb5d40cb150 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400cb90 tx=0x7fb5c4001ab0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 shutdown_connections 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb5c0066f30 0x7fb5c00693e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d40ada80 0x7fb5d40af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d406aa70 0x7fb5d40cb150 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 --2- 172.21.15.45:0/3867000143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5d406ae40 0x7fb5d40c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.412+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 >> 172.21.15.45:0/3867000143 conn(0x7fb5d40cc570 msgr2=0x7fb5d4156a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.413+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 shutdown_connections 2023-10-06T05:27:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.413+0000 7fb5d9829700 1 -- 172.21.15.45:0/3867000143 wait complete. 2023-10-06T05:27:03.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:27:03.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.561+0000 7f79b5d88700 1 Processor -- start 2023-10-06T05:27:03.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.561+0000 7f79b5d88700 1 -- start start 2023-10-06T05:27:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.561+0000 7f79b5d88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b0610 0x7f79b00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.561+0000 7f79b5d88700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79b00c3db0 0x7f79b00697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79b5d88700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00ce6a0 con 0x7f79b0069d00 2023-10-06T05:27:03.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79b5d88700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00690f0 con 0x7f79b00c3db0 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79b5d88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00ceeb0 con 0x7f79b00b0610 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79aeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79b00c3db0 0x7f79b00697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79affff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f79b0069d00 0x7f79b00c2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47160/0 (socket says 172.21.15.45:47160) 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79affff700 1 -- 172.21.15.45:0/3159527740 learned_addr learned my addr 172.21.15.45:0/3159527740 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79af7fe700 1 --2- 172.21.15.45:0/3159527740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b0610 0x7f79b00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79aeffd700 1 -- 172.21.15.45:0/3159527740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b0610 msgr2=0x7f79b00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79aeffd700 1 --2- 172.21.15.45:0/3159527740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b0610 0x7f79b00c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79aeffd700 1 -- 172.21.15.45:0/3159527740 >> v1:172.21.15.45:6789/0 conn(0x7f79b0069d00 legacy=0x7f79b00c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79aeffd700 1 -- 172.21.15.45:0/3159527740 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0159880 con 0x7f79b00c3db0 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79ae7fc700 1 -- 172.21.15.45:0/3159527740 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190911428 0 0) 0x7f79b00ce6a0 con 0x7f79b0069d00 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79aeffd700 1 --2- 172.21.15.45:0/3159527740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00697c0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f7998009910 tx=0x7f79980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da5047c0300eb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79ae7fc700 1 -- 172.21.15.45:0/3159527740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800b820 con 0x7f79b00c3db0 2023-10-06T05:27:03.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79ae7fc700 1 -- 172.21.15.45:0/3159527740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799800b980 con 0x7f79b00c3db0 2023-10-06T05:27:03.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.562+0000 7f79ae7fc700 1 -- 172.21.15.45:0/3159527740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800bd20 con 0x7f79b00c3db0 2023-10-06T05:27:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 -- 172.21.15.45:0/3159527740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 msgr2=0x7f79b00697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3159527740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00697c0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f7998009910 tx=0x7f79980091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 -- 172.21.15.45:0/3159527740 shutdown_connections 2023-10-06T05:27:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3159527740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b0610 0x7f79b00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3159527740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00697c0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 -- 172.21.15.45:0/3159527740 >> 172.21.15.45:0/3159527740 conn(0x7f79b0067e00 msgr2=0x7f79b00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 -- 172.21.15.45:0/3159527740 shutdown_connections 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.563+0000 7f79b5d88700 1 -- 172.21.15.45:0/3159527740 wait complete. 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.564+0000 7f79b5d88700 1 Processor -- start 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79b5d88700 1 -- start start 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79b5d88700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b0069d00 0x7f79b00cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79b5d88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b0610 0x7f79b00c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79b5d88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79b5d88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0158070 con 0x7f79b00c3db0 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79b5d88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0098700 con 0x7f79b00b0610 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79b5d88700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0097dd0 con 0x7f79b0069d00 2023-10-06T05:27:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43838/0 (socket says 172.21.15.45:43838) 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79affff700 1 -- 172.21.15.45:0/3125951358 learned_addr learned my addr 172.21.15.45:0/3125951358 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79aeffd700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b0610 0x7f79b00c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.565+0000 7f79af7fe700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b0069d00 0x7f79b00cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79affff700 1 -- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b0069d00 msgr2=0x7f79b00cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79affff700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b0069d00 0x7f79b00cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79affff700 1 -- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b0610 msgr2=0x7f79b00c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79affff700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b0610 0x7f79b00c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79affff700 1 -- 172.21.15.45:0/3125951358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998009420 con 0x7f79b00c3db0 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79affff700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00c7b00 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f79a800cc90 tx=0x7f79a8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79a77fe700 1 -- 172.21.15.45:0/3125951358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800a870 con 0x7f79b00c3db0 2023-10-06T05:27:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79a77fe700 1 -- 172.21.15.45:0/3125951358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800a9d0 con 0x7f79b00c3db0 2023-10-06T05:27:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b00980b0 con 0x7f79b00c3db0 2023-10-06T05:27:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79a77fe700 1 -- 172.21.15.45:0/3125951358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800ad90 con 0x7f79b00c3db0 2023-10-06T05:27:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.566+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b01692a0 con 0x7f79b00c3db0 2023-10-06T05:27:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.567+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b006e350 con 0x7f79b00c3db0 2023-10-06T05:27:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.568+0000 7f79a77fe700 1 -- 172.21.15.45:0/3125951358 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f79a8007300 con 0x7f79b00c3db0 2023-10-06T05:27:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.568+0000 7f79a77fe700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7990066f30 0x7f79900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.568+0000 7f79a77fe700 1 -- 172.21.15.45:0/3125951358 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f79a8085a30 con 0x7f79b00c3db0 2023-10-06T05:27:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.568+0000 7f79af7fe700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7990066f30 0x7f79900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.569+0000 7f79af7fe700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7990066f30 0x7f79900693e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f79a0006aa0 tx=0x7f79a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.573+0000 7f79a77fe700 1 -- 172.21.15.45:0/3125951358 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79a8051b60 con 0x7f79b00c3db0 2023-10-06T05:27:03.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.773+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79b0098840 con 0x7f79b00c3db0 2023-10-06T05:27:03.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.773+0000 7f79a77fe700 1 -- 172.21.15.45:0/3125951358 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79a8055180 con 0x7f79b00c3db0 2023-10-06T05:27:03.775 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:27:03.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.775+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7990066f30 msgr2=0x7f79900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.775+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7990066f30 0x7f79900693e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f79a0006aa0 tx=0x7f79a0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.775+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 msgr2=0x7f79b00c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.775+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00c7b00 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f79a800cc90 tx=0x7f79a8001a20 comp rx=0 tx=0).stop 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 shutdown_connections 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7990066f30 0x7f79900693e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79b00c3db0 0x7f79b00c7b00 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b0069d00 0x7f79b00cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 --2- 172.21.15.45:0/3125951358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b0610 0x7f79b00c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 >> 172.21.15.45:0/3125951358 conn(0x7f79b0067e00 msgr2=0x7f79b00c35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 shutdown_connections 2023-10-06T05:27:03.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.776+0000 7f79b5d88700 1 -- 172.21.15.45:0/3125951358 wait complete. 2023-10-06T05:27:03.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:27:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.923+0000 7f27873c4700 1 Processor -- start 2023-10-06T05:27:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.923+0000 7f27873c4700 1 -- start start 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.924+0000 7f27873c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006aa70 0x7f27800c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.924+0000 7f27873c4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f27800c7050 0x7f278015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.924+0000 7f27873c4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f278006ae40 0x7f278015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.924+0000 7f27873c4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278015cc30 con 0x7f278006ae40 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.924+0000 7f27873c4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27800ac620 con 0x7f27800c7050 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.924+0000 7f27873c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2780067d80 con 0x7f278006aa70 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.924+0000 7f2785961700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f278006ae40 0x7f278015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f278006ae40 0x7f278015c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43842/0 (socket says 172.21.15.45:43842) 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 -- 172.21.15.45:0/3028667050 learned_addr learned my addr 172.21.15.45:0/3028667050 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785160700 1 --2- 172.21.15.45:0/3028667050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006aa70 0x7f27800c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f278495f700 1 --2- 172.21.15.45:0/3028667050 >> v2:172.21.15.144:3300/0 conn(0x7f27800c7050 0x7f278015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 -- 172.21.15.45:0/3028667050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006aa70 msgr2=0x7f27800c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 --2- 172.21.15.45:0/3028667050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006aa70 0x7f27800c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 -- 172.21.15.45:0/3028667050 >> v2:172.21.15.144:3300/0 conn(0x7f27800c7050 msgr2=0x7f278015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 --2- 172.21.15.45:0/3028667050 >> v2:172.21.15.144:3300/0 conn(0x7f27800c7050 0x7f278015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 -- 172.21.15.45:0/3028667050 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27800c75b0 con 0x7f278006ae40 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2785961700 1 --2- 172.21.15.45:0/3028667050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006ae40 0x7f278015c6f0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f278006f8a0 tx=0x7f2778009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea72d2e2873a51a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2773fff700 1 -- 172.21.15.45:0/3028667050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277800e070 con 0x7f278006ae40 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2773fff700 1 -- 172.21.15.45:0/3028667050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277800b970 con 0x7f278006ae40 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.925+0000 7f2773fff700 1 -- 172.21.15.45:0/3028667050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780092e0 con 0x7f278006ae40 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 -- 172.21.15.45:0/3028667050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006ae40 msgr2=0x7f278015c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 --2- 172.21.15.45:0/3028667050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006ae40 0x7f278015c6f0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f278006f8a0 tx=0x7f2778009840 comp rx=0 tx=0).stop 2023-10-06T05:27:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 -- 172.21.15.45:0/3028667050 shutdown_connections 2023-10-06T05:27:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 --2- 172.21.15.45:0/3028667050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006ae40 0x7f278015c6f0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 --2- 172.21.15.45:0/3028667050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006aa70 0x7f27800c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 --2- 172.21.15.45:0/3028667050 >> v2:172.21.15.144:3300/0 conn(0x7f27800c7050 0x7f278015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 -- 172.21.15.45:0/3028667050 >> 172.21.15.45:0/3028667050 conn(0x7f27800cc570 msgr2=0x7f27800cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 -- 172.21.15.45:0/3028667050 shutdown_connections 2023-10-06T05:27:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.926+0000 7f27873c4700 1 -- 172.21.15.45:0/3028667050 wait complete. 2023-10-06T05:27:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.927+0000 7f27873c4700 1 Processor -- start 2023-10-06T05:27:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.927+0000 7f27873c4700 1 -- start start 2023-10-06T05:27:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f27873c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006aa70 0x7f2780155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f27873c4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006ae40 0x7f2780155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f27873c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27800c7050 0x7f2780159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f27873c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27800af4f0 con 0x7f278006aa70 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f27873c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27800af7c0 con 0x7f27800c7050 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f27873c4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27800afa60 con 0x7f278006ae40 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f2785160700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006aa70 0x7f2780155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f2785160700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006aa70 0x7f2780155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43858/0 (socket says 172.21.15.45:43858) 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f2785160700 1 -- 172.21.15.45:0/3967462201 learned_addr learned my addr 172.21.15.45:0/3967462201 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f278495f700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006ae40 0x7f2780155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.928+0000 7f2785961700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27800c7050 0x7f2780159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f2785160700 1 -- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006ae40 msgr2=0x7f2780155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f2785160700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006ae40 0x7f2780155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f2785160700 1 -- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27800c7050 msgr2=0x7f2780159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f2785160700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27800c7050 0x7f2780159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f2785160700 1 -- 172.21.15.45:0/3967462201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778009ae0 con 0x7f278006aa70 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f2785160700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006aa70 0x7f2780155620 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f277c00ec80 tx=0x7f277c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f27717fa700 1 -- 172.21.15.45:0/3967462201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c00c850 con 0x7f278006aa70 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f27717fa700 1 -- 172.21.15.45:0/3967462201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c00c9b0 con 0x7f278006aa70 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27800afd40 con 0x7f278006aa70 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f27717fa700 1 -- 172.21.15.45:0/3967462201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c00cd50 con 0x7f278006aa70 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.929+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278015ad20 con 0x7f278006aa70 2023-10-06T05:27:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.930+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278006e300 con 0x7f278006aa70 2023-10-06T05:27:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.930+0000 7f27717fa700 1 -- 172.21.15.45:0/3967462201 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f277c013680 con 0x7f278006aa70 2023-10-06T05:27:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.931+0000 7f27717fa700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2764066f30 0x7f27640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.931+0000 7f27717fa700 1 -- 172.21.15.45:0/3967462201 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f277c085840 con 0x7f278006aa70 2023-10-06T05:27:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.931+0000 7f278495f700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2764066f30 0x7f27640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.932+0000 7f278495f700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2764066f30 0x7f27640693e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f27800c5bb0 tx=0x7f27740091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:03.936+0000 7f27717fa700 1 -- 172.21.15.45:0/3967462201 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f277c051a00 con 0x7f278006aa70 2023-10-06T05:27:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.137+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27800c7890 con 0x7f278006aa70 2023-10-06T05:27:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.137+0000 7f27717fa700 1 -- 172.21.15.45:0/3967462201 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7f277c055020 con 0x7f278006aa70 2023-10-06T05:27:04.139 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2764066f30 msgr2=0x7f27640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2764066f30 0x7f27640693e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f27800c5bb0 tx=0x7f27740091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006aa70 msgr2=0x7f2780155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006aa70 0x7f2780155620 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f277c00ec80 tx=0x7f277c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 shutdown_connections 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2764066f30 0x7f27640693e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27800c7050 0x7f2780159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f278006aa70 0x7f2780155620 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 --2- 172.21.15.45:0/3967462201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006ae40 0x7f2780155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.139+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 >> 172.21.15.45:0/3967462201 conn(0x7f27800cc570 msgr2=0x7f27800cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.140+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 shutdown_connections 2023-10-06T05:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.140+0000 7f27873c4700 1 -- 172.21.15.45:0/3967462201 wait complete. 2023-10-06T05:27:04.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:04.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.284+0000 7fa5ec7ba700 1 Processor -- start 2023-10-06T05:27:04.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.284+0000 7fa5ec7ba700 1 -- start start 2023-10-06T05:27:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.285+0000 7fa5ec7ba700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5e4159330 0x7fa5e406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.285+0000 7fa5ec7ba700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.285+0000 7fa5ec7ba700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40cb530 con 0x7fa5e4158f60 2023-10-06T05:27:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.285+0000 7fa5ec7ba700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e40ccd50 con 0x7fa5e4159330 2023-10-06T05:27:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.285+0000 7fa5ec7ba700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4069bd0 con 0x7fa5e40cc390 2023-10-06T05:27:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5ead57700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9d55700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5e4159330 0x7fa5e406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5ea556700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa5e4158f60 0x7fa5e40cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:41726/0 (socket says 172.21.15.45:41726) 2023-10-06T05:27:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5ea556700 1 -- 172.21.15.45:0/3692420066 learned_addr learned my addr 172.21.15.45:0/3692420066 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5ead57700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e40cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:53376/0 (socket says 172.21.15.45:53376) 2023-10-06T05:27:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9d55700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5e4159330 0x7fa5e406a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:44258/0 (socket says 172.21.15.45:44258) 2023-10-06T05:27:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9d55700 1 -- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 msgr2=0x7fa5e40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9d55700 1 --2- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9d55700 1 -- 172.21.15.45:0/3692420066 >> v1:172.21.15.144:6789/0 conn(0x7fa5e4158f60 legacy=0x7fa5e40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9d55700 1 -- 172.21.15.45:0/3692420066 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e4068fc0 con 0x7fa5e4159330 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9554700 1 -- 172.21.15.45:0/3692420066 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426209790 0 0) 0x7fa5e0001110 con 0x7fa5e4158f60 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5ead57700 1 --2- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.286+0000 7fa5e9d55700 1 --2- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e406a060 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40098e0 tx=0x7fa5d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814a0ac6c2ed010a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5e9554700 1 -- 172.21.15.45:0/3692420066 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400b640 con 0x7fa5e4159330 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5e9554700 1 -- 172.21.15.45:0/3692420066 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d400b7a0 con 0x7fa5e4159330 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5e9554700 1 -- 172.21.15.45:0/3692420066 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400bb40 con 0x7fa5e4159330 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 msgr2=0x7fa5e406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e406a060 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40098e0 tx=0x7fa5d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/3692420066 shutdown_connections 2023-10-06T05:27:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/3692420066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e406a060 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/3692420066 >> 172.21.15.45:0/3692420066 conn(0x7fa5e40cdf50 msgr2=0x7fa5e40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/3692420066 shutdown_connections 2023-10-06T05:27:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.287+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/3692420066 wait complete. 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 Processor -- start 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 -- start start 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e4153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5e4158f60 0x7fa5e415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4154550 con 0x7fa5e4158f60 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4154000 con 0x7fa5e4159330 2023-10-06T05:27:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5ec7ba700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40b2d70 con 0x7fa5e40cc390 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.289+0000 7fa5e9d55700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5e4158f60 0x7fa5e415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5e4158f60 0x7fa5e415c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43870/0 (socket says 172.21.15.45:43870) 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 -- 172.21.15.45:0/1249990088 learned_addr learned my addr 172.21.15.45:0/1249990088 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5ea556700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e4153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5ead57700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 -- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 msgr2=0x7fa5e4153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e4153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 -- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 msgr2=0x7fa5e415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e415e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 -- 172.21.15.45:0/1249990088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc009a20 con 0x7fa5e4158f60 2023-10-06T05:27:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5ead57700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5e9d55700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5e4158f60 0x7fa5e415c5e0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40093f0 tx=0x7fa5d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5da7fc700 1 -- 172.21.15.45:0/1249990088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400bb20 con 0x7fa5e4158f60 2023-10-06T05:27:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5da7fc700 1 -- 172.21.15.45:0/1249990088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4011420 con 0x7fa5e4158f60 2023-10-06T05:27:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5da7fc700 1 -- 172.21.15.45:0/1249990088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40117c0 con 0x7fa5e4158f60 2023-10-06T05:27:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d4009420 con 0x7fa5e4158f60 2023-10-06T05:27:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.290+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e40b3330 con 0x7fa5e4158f60 2023-10-06T05:27:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.291+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e40c5c50 con 0x7fa5e4158f60 2023-10-06T05:27:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.292+0000 7fa5da7fc700 1 -- 172.21.15.45:0/1249990088 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5d4011bf0 con 0x7fa5e4158f60 2023-10-06T05:27:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.292+0000 7fa5da7fc700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa5c8066f30 0x7fa5c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.292+0000 7fa5da7fc700 1 -- 172.21.15.45:0/1249990088 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7fa5d40869b0 con 0x7fa5e4158f60 2023-10-06T05:27:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.293+0000 7fa5ea556700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa5c8066f30 0x7fa5c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.293+0000 7fa5ea556700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa5c8066f30 0x7fa5c80693e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa5e415bf30 tx=0x7fa5e0007b90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.297+0000 7fa5da7fc700 1 -- 172.21.15.45:0/1249990088 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5e40c5c50 con 0x7fa5e4158f60 2023-10-06T05:27:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.542+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5e40c5c50 con 0x7fa5e4158f60 2023-10-06T05:27:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.543+0000 7fa5da7fc700 1 -- 172.21.15.45:0/1249990088 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4601 (secure 0 0 0) 0x7fa5e40c5c50 con 0x7fa5e4158f60 2023-10-06T05:27:04.545 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:04.545 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:58.006287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa5c8066f30 msgr2=0x7fa5c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa5c8066f30 0x7fa5c80693e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa5e415bf30 tx=0x7fa5e0007b90 comp rx=0 tx=0).stop 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5e4158f60 msgr2=0x7fa5e415c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5e4158f60 0x7fa5e415c5e0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40093f0 tx=0x7fa5d4017000 comp rx=0 tx=0).stop 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 shutdown_connections 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa5c8066f30 0x7fa5c80693e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e4159330 0x7fa5e415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40cc390 0x7fa5e4153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 --2- 172.21.15.45:0/1249990088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5e4158f60 0x7fa5e415c5e0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 >> 172.21.15.45:0/1249990088 conn(0x7fa5e40cdf50 msgr2=0x7fa5e40c40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 shutdown_connections 2023-10-06T05:27:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.545+0000 7fa5ec7ba700 1 -- 172.21.15.45:0/1249990088 wait complete. 2023-10-06T05:27:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 350 2023-10-06T05:27:04.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.689+0000 7f6524440700 1 Processor -- start 2023-10-06T05:27:04.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.689+0000 7f6524440700 1 -- start start 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.690+0000 7f6524440700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f651c159330 0x7f651c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.690+0000 7f6524440700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0cb530 con 0x7f651c158f60 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.690+0000 7f6524440700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c069bd0 con 0x7f651c0cc390 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.690+0000 7f6524440700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c068fc0 con 0x7f651c159330 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.690+0000 7f65219db700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f651c159330 0x7f651c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.690+0000 7f65221dc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f651c158f60 0x7f651c0cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47162/0 (socket says 172.21.15.45:47162) 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.690+0000 7f65221dc700 1 -- 172.21.15.45:0/1279297423 learned_addr learned my addr 172.21.15.45:0/1279297423 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.691+0000 7f65219db700 1 -- 172.21.15.45:0/1279297423 >> v1:172.21.15.144:6789/0 conn(0x7f651c0cc390 legacy=0x7f651c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.691+0000 7f65219db700 1 -- 172.21.15.45:0/1279297423 >> v1:172.21.15.45:6789/0 conn(0x7f651c158f60 legacy=0x7f651c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.691+0000 7f65219db700 1 -- 172.21.15.45:0/1279297423 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0692a0 con 0x7f651c159330 2023-10-06T05:27:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.691+0000 7f65219db700 1 --2- 172.21.15.45:0/1279297423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c159330 0x7f651c06a060 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f65180098e0 tx=0x7f65180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9f5d62dedb56a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.691+0000 7f65211da700 1 -- 172.21.15.45:0/1279297423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518009e80 con 0x7f651c159330 2023-10-06T05:27:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.691+0000 7f65211da700 1 -- 172.21.15.45:0/1279297423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651800b820 con 0x7f651c159330 2023-10-06T05:27:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.691+0000 7f65211da700 1 -- 172.21.15.45:0/1279297423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800bba0 con 0x7f651c159330 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.692+0000 7f6524440700 1 -- 172.21.15.45:0/1279297423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c159330 msgr2=0x7f651c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.692+0000 7f6524440700 1 --2- 172.21.15.45:0/1279297423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c159330 0x7f651c06a060 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f65180098e0 tx=0x7f65180091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.692+0000 7f6524440700 1 -- 172.21.15.45:0/1279297423 shutdown_connections 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.692+0000 7f6524440700 1 --2- 172.21.15.45:0/1279297423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c159330 0x7f651c06a060 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.692+0000 7f6524440700 1 -- 172.21.15.45:0/1279297423 >> 172.21.15.45:0/1279297423 conn(0x7f651c0cdf50 msgr2=0x7f651c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.692+0000 7f6524440700 1 -- 172.21.15.45:0/1279297423 shutdown_connections 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.692+0000 7f6524440700 1 -- 172.21.15.45:0/1279297423 wait complete. 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.693+0000 7f6524440700 1 Processor -- start 2023-10-06T05:27:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.693+0000 7f6524440700 1 -- start start 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f6524440700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c0cc390 0x7f651c147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f6524440700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f651c158f60 0x7f651c147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f6524440700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f651c159330 0x7f651c16d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f6524440700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b2220 con 0x7f651c0cc390 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f6524440700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b2500 con 0x7f651c159330 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f6524440700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b27e0 con 0x7f651c158f60 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f65221dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c0cc390 0x7f651c147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f65221dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c0cc390 0x7f651c147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43896/0 (socket says 172.21.15.45:43896) 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f65221dc700 1 -- 172.21.15.45:0/3626547582 learned_addr learned my addr 172.21.15.45:0/3626547582 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f65219db700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f651c158f60 0x7f651c147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f65229dd700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f651c159330 0x7f651c16d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.694+0000 7f65221dc700 1 -- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f651c158f60 msgr2=0x7f651c147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f65221dc700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f651c158f60 0x7f651c147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f65221dc700 1 -- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f651c159330 msgr2=0x7f651c16d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f65221dc700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f651c159330 0x7f651c16d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f65221dc700 1 -- 172.21.15.45:0/3626547582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6518009420 con 0x7f651c0cc390 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f65219db700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f651c158f60 0x7f651c147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f65229dd700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f651c159330 0x7f651c16d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f65221dc700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c0cc390 0x7f651c147590 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f651000c7c0 tx=0x7f6510007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f650e7fc700 1 -- 172.21.15.45:0/3626547582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510005b30 con 0x7f651c0cc390 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f650e7fc700 1 -- 172.21.15.45:0/3626547582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6510005cb0 con 0x7f651c0cc390 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f650e7fc700 1 -- 172.21.15.45:0/3626547582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510004a80 con 0x7f651c0cc390 2023-10-06T05:27:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c0c50a0 con 0x7f651c0cc390 2023-10-06T05:27:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.695+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c154980 con 0x7f651c0cc390 2023-10-06T05:27:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.696+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c0c55c0 con 0x7f651c0cc390 2023-10-06T05:27:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.696+0000 7f650e7fc700 1 -- 172.21.15.45:0/3626547582 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65100070f0 con 0x7f651c0cc390 2023-10-06T05:27:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.697+0000 7f650e7fc700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6500066f30 0x7f65000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.697+0000 7f650e7fc700 1 -- 172.21.15.45:0/3626547582 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f6510084870 con 0x7f651c0cc390 2023-10-06T05:27:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.697+0000 7f65219db700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6500066f30 0x7f65000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.698+0000 7f65219db700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6500066f30 0x7f65000693e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f651c15f080 tx=0x7f6518016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:04.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.701+0000 7f650e7fc700 1 -- 172.21.15.45:0/3626547582 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f651c0c55c0 con 0x7f651c0cc390 2023-10-06T05:27:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.905+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f651c0cb530 con 0x7f651c0cc390 2023-10-06T05:27:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.907+0000 7f650e7fc700 1 -- 172.21.15.45:0/3626547582 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+22388 (secure 0 0 0) 0x7f651c0cb530 con 0x7f651c0cc390 2023-10-06T05:27:04.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:04.909 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":132,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:26:58.006054+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:49.182551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:49.732208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2681459400":"2023-10-07T05:23:52.452679+0000","172.21.15.45:6835/2971431427":"2023-10-07T05:24:37.100960+0000","172.21.15.45:6834/2681459400":"2023-10-07T05:23:52.452679+0000","172.21.15.45:6835/1013285853":"2023-10-07T05:23:45.414617+0000","172.21.15.45:6835/858773096":"2023-10-07T05:24:07.945434+0000","172.21.15.144:6834/4239891954":"2023-10-07T05:25:03.377085+0000","172.21.15.45:6834/3031536032":"2023-10-07T05:24:22.864925+0000","172.21.15.45:6834/3141743890":"2023-10-07T05:23:23.111877+0000","172.21.15.45:6834/681980478":"2023-10-07T05:23:12.050135+0000","172.21.15.45:6835/2532567321":"2023-10-07T05:23:15.870223+0000","172.21.15.45:6834/2405580153":"2023-10-07T05:23:43.392415+0000","172.21.15.45:6834/2971431427":"2023-10-07T05:24:37.100960+0000","172.21.15.144:6834/3602644226":"2023-10-07T05:24:45.157829+0000","172.21.15.144:6834/1302691811":"2023-10-07T05:26:43.923386+0000","172.21.15.45:6832/3755910187":"2023-10-07T05:22:28.384687+0000","172.21.15.144:6835/2001233142":"2023-10-07T05:26:17.517661+0000","172.21.15.45:6832/2550360138":"2023-10-07T05:22:23.359215+0000","172.21.15.45:6832/821082350":"2023-10-07T05:22:07.114456+0000","172.21.15.45:6835/2405580153":"2023-10-07T05:23:43.392415+0000","172.21.15.45:6832/3860010011":"2023-10-07T05:22:38.914897+0000","172.21.15.45:6835/3141743890":"2023-10-07T05:23:23.111877+0000","172.21.15.45:6832/4128377575":"2023-10-07T05:22:34.886263+0000","172.21.15.45:6835/2475927951":"2023-10-07T05:24:29.055945+0000","172.21.15.45:6833/4128377575":"2023-10-07T05:22:34.886263+0000","172.21.15.144:6834/3552713853":"2023-10-07T05:25:38.094009+0000","172.21.15.45:6833/2550360138":"2023-10-07T05:22:23.359215+0000","172.21.15.45:6834/858773096":"2023-10-07T05:24:07.945434+0000","172.21.15.45:6834/2475927951":"2023-10-07T05:24:29.055945+0000","172.21.15.45:6833/821082350":"2023-10-07T05:22:07.114456+0000","172.21.15.45:6834/3351321197":"2023-10-07T05:23:59.498576+0000","172.21.15.45:6833/3860010011":"2023-10-07T05:22:38.914897+0000","172.21.15.144:6835/2237111349":"2023-10-07T05:25:17.948375+0000","172.21.15.45:6835/681980478":"2023-10-07T05:23:12.050135+0000","172.21.15.45:6835/1996698078":"2023-10-07T05:24:25.023244+0000","172.21.15.45:6835/1915871760":"2023-10-07T05:24:03.901047+0000","172.21.15.45:6835/3031536032":"2023-10-07T05:24:22.864925+0000","172.21.15.45:6833/3755910187":"2023-10-07T05:22:28.384687+0000","172.21.15.45:6835/3277956876":"2023-10-07T05:22:42.945097+0000","172.21.15.45:6834/1915871760":"2023-10-07T05:24:03.901047+0000","172.21.15.45:6834/226461717":"2023-10-07T05:22:56.431657+0000","172.21.15.144:6835/4239891954":"2023-10-07T05:25:03.377085+0000","172.21.15.45:6834/2620071126":"2023-10-07T05:23:29.906056+0000","172.21.15.144:6835/3580730449":"2023-10-07T05:25:05.403592+0000","172.21.15.45:6834/3277956876":"2023-10-07T05:22:42.945097+0000","172.21.15.45:6835/226461717":"2023-10-07T05:22:56.431657+0000","172.21.15.45:6835/2531263654":"2023-10-07T05:23:09.893451+0000","172.21.15.45:6834/1013285853":"2023-10-07T05:23:45.414617+0000","172.21.15.45:6834/1996698078":"2023-10-07T05:24:25.023244+0000","172.21.15.144:6834/3433217004":"2023-10-07T05:24:33.068465+0000","172.21.15.45:6835/2620071126":"2023-10-07T05:23:29.906056+0000","172.21.15.144:6834/2156055156":"2023-10-07T05:24:40.893717+0000","172.21.15.144:6835/2156055156":"2023-10-07T05:24:40.893717+0000","172.21.15.144:6835/3602644226":"2023-10-07T05:24:45.157829+0000","172.21.15.144:6835/1309688991":"2023-10-07T05:24:49.191270+0000","172.21.15.144:6834/2062728889":"2023-10-07T05:26:06.022104+0000","172.21.15.144:6834/3580730449":"2023-10-07T05:25:05.403592+0000","172.21.15.144:6835/913913762":"2023-10-07T05:26:58.006032+0000","172.21.15.144:6834/3983487029":"2023-10-07T05:25:13.909730+0000","172.21.15.144:6835/3451214191":"2023-10-07T05:26:22.547760+0000","172.21.15.144:6834/243455709":"2023-10-07T05:25:09.429589+0000","172.21.15.144:6835/243455709":"2023-10-07T05:25:09.429589+0000","172.21.15.144:6835/2062728889":"2023-10-07T05:26:06.022104+0000","172.21.15.144:6834/913913762":"2023-10-07T05:26:58.006032+0000","172.21.15.144:6835/3983487029":"2023-10-07T05:25:13.909730+0000","172.21.15.144:6835/26720443":"2023-10-07T05:25:43.120106+0000","172.21.15.144:6835/3552713853":"2023-10-07T05:25:38.094009+0000","172.21.15.144:6834/1309688991":"2023-10-07T05:24:49.191270+0000","172.21.15.144:6834/26720443":"2023-10-07T05:25:43.120106+0000","172.21.15.144:6834/2001233142":"2023-10-07T05:26:17.517661+0000","172.21.15.144:6835/1309152247":"2023-10-07T05:26:50.920566+0000","172.21.15.144:6834/3451214191":"2023-10-07T05:26:22.547760+0000","172.21.15.45:6835/3351321197":"2023-10-07T05:23:59.498576+0000","172.21.15.144:6835/3433217004":"2023-10-07T05:24:33.068465+0000","172.21.15.144:6834/603338967":"2023-10-07T05:26:39.088317+0000","172.21.15.144:6834/2237111349":"2023-10-07T05:25:17.948375+0000","172.21.15.144:6835/603338967":"2023-10-07T05:26:39.088317+0000","172.21.15.144:6835/1302691811":"2023-10-07T05:26:43.923386+0000","172.21.15.45:6834/2531263654":"2023-10-07T05:23:09.893451+0000","172.21.15.45:6834/2532567321":"2023-10-07T05:23:15.870223+0000","172.21.15.144:6834/1309152247":"2023-10-07T05:26:50.920566+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6500066f30 msgr2=0x7f65000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6500066f30 0x7f65000693e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f651c15f080 tx=0x7f6518016000 comp rx=0 tx=0).stop 2023-10-06T05:27:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c0cc390 msgr2=0x7f651c147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c0cc390 0x7f651c147590 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f651000c7c0 tx=0x7f6510007f70 comp rx=0 tx=0).stop 2023-10-06T05:27:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 shutdown_connections 2023-10-06T05:27:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6500066f30 0x7f65000693e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f651c159330 0x7f651c16d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f651c0cc390 0x7f651c147590 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 --2- 172.21.15.45:0/3626547582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f651c158f60 0x7f651c147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 >> 172.21.15.45:0/3626547582 conn(0x7f651c0cdf50 msgr2=0x7f651c06c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 shutdown_connections 2023-10-06T05:27:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:04.909+0000 7f6524440700 1 -- 172.21.15.45:0/3626547582 wait complete. 2023-10-06T05:27:04.924 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:27:04.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.057+0000 7efe3efdb700 1 Processor -- start 2023-10-06T05:27:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.057+0000 7efe3efdb700 1 -- start start 2023-10-06T05:27:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe3efdb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 0x7efe3815a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe3efdb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380ae8d0 con 0x7efe38155cd0 2023-10-06T05:27:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe3efdb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380aebb0 con 0x7efe381560a0 2023-10-06T05:27:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe3efdb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe380ae6b0 con 0x7efe380cd4d0 2023-10-06T05:27:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe3cd77700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efe38155cd0 0x7efe380adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47174/0 (socket says 172.21.15.45:47174) 2023-10-06T05:27:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe3cd77700 1 -- 172.21.15.45:0/930084292 learned_addr learned my addr 172.21.15.45:0/930084292 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe3d578700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 0x7efe3815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712577965 0 0) 0x7efe380ae8d0 con 0x7efe38155cd0 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.058+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe28003570 con 0x7efe38155cd0 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880872060 0 0) 0x7efe28003570 con 0x7efe38155cd0 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 msgr2=0x7efe3815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 0x7efe3815a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 >> v1:172.21.15.144:6789/0 conn(0x7efe381560a0 legacy=0x7efe380ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380aee90 con 0x7efe38155cd0 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7efe2c001470 con 0x7efe38155cd0 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 >> v1:172.21.15.45:6789/0 conn(0x7efe38155cd0 legacy=0x7efe380adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 0x7efe28003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe28003fd0 0x7efe28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28004d50 0x7efe280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380ae6b0 con 0x7efe28003fd0 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2800bb40 con 0x7efe28004d50 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2800df10 con 0x7efe380cd4d0 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe37fff700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe28003fd0 0x7efe28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe3cd77700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 0x7efe28003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe3d578700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28004d50 0x7efe280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe37fff700 1 -- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 msgr2=0x7efe28003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe37fff700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380cd4d0 0x7efe28003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe37fff700 1 -- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28004d50 msgr2=0x7efe280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe37fff700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28004d50 0x7efe280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe37fff700 1 -- 172.21.15.45:0/930084292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28003570 con 0x7efe28003fd0 2023-10-06T05:27:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.059+0000 7efe37fff700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe28003fd0 0x7efe28004810 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7efe24001960 tx=0x7efe24007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678eda3108f3d7a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400c5d0 con 0x7efe28003fd0 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2400c750 con 0x7efe28003fd0 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe377fe700 1 -- 172.21.15.45:0/930084292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400cb10 con 0x7efe28003fd0 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe3efdb700 1 -- 172.21.15.45:0/930084292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe28003fd0 msgr2=0x7efe28004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe3efdb700 1 --2- 172.21.15.45:0/930084292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe28003fd0 0x7efe28004810 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7efe24001960 tx=0x7efe24007d10 comp rx=0 tx=0).stop 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe3d578700 1 -- 172.21.15.45:0/930084292 reap_dead start 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe3efdb700 1 -- 172.21.15.45:0/930084292 shutdown_connections 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe3efdb700 1 -- 172.21.15.45:0/930084292 >> 172.21.15.45:0/930084292 conn(0x7efe3806f090 msgr2=0x7efe3806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe3efdb700 1 -- 172.21.15.45:0/930084292 shutdown_connections 2023-10-06T05:27:05.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.060+0000 7efe3efdb700 1 -- 172.21.15.45:0/930084292 wait complete. 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.061+0000 7efe3efdb700 1 Processor -- start 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.062+0000 7efe3efdb700 1 -- start start 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.062+0000 7efe3efdb700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28003fd0 0x7efe380c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.062+0000 7efe3efdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe380c4380 0x7efe381575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.062+0000 7efe3efdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38157af0 0x7efe38157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.062+0000 7efe3efdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380c54a0 con 0x7efe380c4380 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.062+0000 7efe3efdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380c5930 con 0x7efe38157af0 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe3efdb700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe381584b0 con 0x7efe28003fd0 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe380c4380 0x7efe381575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe380c4380 0x7efe381575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43920/0 (socket says 172.21.15.45:43920) 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 -- 172.21.15.45:0/3466014392 learned_addr learned my addr 172.21.15.45:0/3466014392 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe3cd77700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28003fd0 0x7efe380c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe3d578700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38157af0 0x7efe38157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 -- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28003fd0 msgr2=0x7efe380c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28003fd0 0x7efe380c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 -- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38157af0 msgr2=0x7efe38157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38157af0 0x7efe38157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 -- 172.21.15.45:0/3466014392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380c4890 con 0x7efe380c4380 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe3d578700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38157af0 0x7efe38157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe37fff700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe380c4380 0x7efe381575b0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7efe240010e0 tx=0x7efe24015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe34ff9700 1 -- 172.21.15.45:0/3466014392 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400c5d0 con 0x7efe380c4380 2023-10-06T05:27:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe34ff9700 1 -- 172.21.15.45:0/3466014392 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2400cec0 con 0x7efe380c4380 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe380c4b70 con 0x7efe380c4380 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe34ff9700 1 -- 172.21.15.45:0/3466014392 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24026400 con 0x7efe380c4380 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.063+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe380c5130 con 0x7efe380c4380 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.064+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe380c5650 con 0x7efe380c4380 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.065+0000 7efe34ff9700 1 -- 172.21.15.45:0/3466014392 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe24019030 con 0x7efe380c4380 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.065+0000 7efe34ff9700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe1c066f30 0x7efe1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.066+0000 7efe34ff9700 1 -- 172.21.15.45:0/3466014392 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7efe24086d60 con 0x7efe380c4380 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.066+0000 7efe3cd77700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe1c066f30 0x7efe1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.066+0000 7efe3cd77700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe1c066f30 0x7efe1c0693e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efe2c001200 tx=0x7efe2c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.070+0000 7efe34ff9700 1 -- 172.21.15.45:0/3466014392 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe380c5650 con 0x7efe380c4380 2023-10-06T05:27:05.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.320+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe380af170 con 0x7efe380c4380 2023-10-06T05:27:05.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.321+0000 7efe34ff9700 1 -- 172.21.15.45:0/3466014392 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4601 (secure 0 0 0) 0x7efe2402b090 con 0x7efe380c4380 2023-10-06T05:27:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:05.323 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:58.006287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe1c066f30 msgr2=0x7efe1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe1c066f30 0x7efe1c0693e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efe2c001200 tx=0x7efe2c009040 comp rx=0 tx=0).stop 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe380c4380 msgr2=0x7efe381575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe380c4380 0x7efe381575b0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7efe240010e0 tx=0x7efe24015000 comp rx=0 tx=0).stop 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 shutdown_connections 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efe1c066f30 0x7efe1c0693e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38157af0 0x7efe38157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28003fd0 0x7efe380c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 --2- 172.21.15.45:0/3466014392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe380c4380 0x7efe381575b0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 >> 172.21.15.45:0/3466014392 conn(0x7efe3806f090 msgr2=0x7efe38067620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 shutdown_connections 2023-10-06T05:27:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.323+0000 7efe3efdb700 1 -- 172.21.15.45:0/3466014392 wait complete. 2023-10-06T05:27:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 350 2023-10-06T05:27:05.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.468+0000 7f66199d9700 1 Processor -- start 2023-10-06T05:27:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.468+0000 7f66199d9700 1 -- start start 2023-10-06T05:27:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66199d9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6614155cd0 0x7f661415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66199d9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66141560a0 0x7f66140adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66199d9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66140ccdc0 0x7f66140cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66199d9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66140cd740 con 0x7f66141560a0 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66199d9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66140ae8d0 con 0x7f6614155cd0 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66199d9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140aebb0 con 0x7f66140ccdc0 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66141560a0 0x7f66140adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66141560a0 0x7f66140adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43928/0 (socket says 172.21.15.45:43928) 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 -- 172.21.15.45:0/2026444393 learned_addr learned my addr 172.21.15.45:0/2026444393 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66137fe700 1 --2- 172.21.15.45:0/2026444393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66140ccdc0 0x7f66140cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f6612ffd700 1 --2- 172.21.15.45:0/2026444393 >> v2:172.21.15.144:3300/0 conn(0x7f6614155cd0 0x7f661415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 -- 172.21.15.45:0/2026444393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66140ccdc0 msgr2=0x7f66140cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 --2- 172.21.15.45:0/2026444393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66140ccdc0 0x7f66140cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 -- 172.21.15.45:0/2026444393 >> v2:172.21.15.144:3300/0 conn(0x7f6614155cd0 msgr2=0x7f661415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 --2- 172.21.15.45:0/2026444393 >> v2:172.21.15.144:3300/0 conn(0x7f6614155cd0 0x7f661415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.470+0000 7f66127fc700 1 -- 172.21.15.45:0/2026444393 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140aee90 con 0x7f66141560a0 2023-10-06T05:27:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66127fc700 1 --2- 172.21.15.45:0/2026444393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f66140adfa0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f661406ae50 tx=0x7f65fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad85c4779c5ad8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f6611ffb700 1 -- 172.21.15.45:0/2026444393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00b640 con 0x7f66141560a0 2023-10-06T05:27:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f6611ffb700 1 -- 172.21.15.45:0/2026444393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00b7a0 con 0x7f66141560a0 2023-10-06T05:27:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f6611ffb700 1 -- 172.21.15.45:0/2026444393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00bb60 con 0x7f66141560a0 2023-10-06T05:27:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 -- 172.21.15.45:0/2026444393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 msgr2=0x7f66140adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 --2- 172.21.15.45:0/2026444393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f66140adfa0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f661406ae50 tx=0x7f65fc009250 comp rx=0 tx=0).stop 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 -- 172.21.15.45:0/2026444393 shutdown_connections 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 --2- 172.21.15.45:0/2026444393 >> v2:172.21.15.144:3300/0 conn(0x7f6614155cd0 0x7f661415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 --2- 172.21.15.45:0/2026444393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66140ccdc0 0x7f66140cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 --2- 172.21.15.45:0/2026444393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f66140adfa0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 -- 172.21.15.45:0/2026444393 >> 172.21.15.45:0/2026444393 conn(0x7f661406f090 msgr2=0x7f661406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 -- 172.21.15.45:0/2026444393 shutdown_connections 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.471+0000 7f66199d9700 1 -- 172.21.15.45:0/2026444393 wait complete. 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.473+0000 7f66199d9700 1 Processor -- start 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.473+0000 7f66199d9700 1 -- start start 2023-10-06T05:27:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66199d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ccdc0 0x7f661414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66199d9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6614155cd0 0x7f66140b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66199d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f6614156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66199d9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614158f20 con 0x7f66141560a0 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66199d9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66141572f0 con 0x7f66140ccdc0 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66199d9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6614158960 con 0x7f6614155cd0 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f6614156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f6614156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43934/0 (socket says 172.21.15.45:43934) 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 -- 172.21.15.45:0/2467466665 learned_addr learned my addr 172.21.15.45:0/2467466665 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66127fc700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6614155cd0 0x7f66140b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f6612ffd700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ccdc0 0x7f661414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 -- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6614155cd0 msgr2=0x7f66140b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6614155cd0 0x7f66140b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 -- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ccdc0 msgr2=0x7f661414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ccdc0 0x7f661414f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 -- 172.21.15.45:0/2467466665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc0094f0 con 0x7f66141560a0 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66137fe700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f6614156730 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f660c009e90 tx=0x7f660c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f660affd700 1 -- 172.21.15.45:0/2467466665 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00cc20 con 0x7f66141560a0 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f660affd700 1 -- 172.21.15.45:0/2467466665 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c00f040 con 0x7f66141560a0 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6614158c40 con 0x7f66141560a0 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.474+0000 7f660affd700 1 -- 172.21.15.45:0/2467466665 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c014480 con 0x7f66141560a0 2023-10-06T05:27:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.475+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66140b0a00 con 0x7f66141560a0 2023-10-06T05:27:05.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.475+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661406e2f0 con 0x7f66141560a0 2023-10-06T05:27:05.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.476+0000 7f660affd700 1 -- 172.21.15.45:0/2467466665 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f660c014830 con 0x7f66141560a0 2023-10-06T05:27:05.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.476+0000 7f660affd700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f65f4066f30 0x7f65f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.477+0000 7f660affd700 1 -- 172.21.15.45:0/2467466665 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f660c086260 con 0x7f66141560a0 2023-10-06T05:27:05.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.477+0000 7f6612ffd700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f65f4066f30 0x7f65f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.477+0000 7f6612ffd700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f65f4066f30 0x7f65f40693e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d40 tx=0x7f6604008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.481+0000 7f660affd700 1 -- 172.21.15.45:0/2467466665 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f660c055f10 con 0x7f66141560a0 2023-10-06T05:27:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.730+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66140af1a0 con 0x7f66141560a0 2023-10-06T05:27:05.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.730+0000 7f660affd700 1 -- 172.21.15.45:0/2467466665 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) v1 ==== 95+0+4601 (secure 0 0 0) 0x7f660c055aa0 con 0x7f66141560a0 2023-10-06T05:27:05.734 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:05.734 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:50.763116+0000","modified":"2023-10-06T05:26:58.006287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:27:05.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f65f4066f30 msgr2=0x7f65f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f65f4066f30 0x7f65f40693e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d40 tx=0x7f6604008040 comp rx=0 tx=0).stop 2023-10-06T05:27:05.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 msgr2=0x7f6614156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f6614156730 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f660c009e90 tx=0x7f660c00b040 comp rx=0 tx=0).stop 2023-10-06T05:27:05.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 shutdown_connections 2023-10-06T05:27:05.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f65f4066f30 0x7f65f40693e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141560a0 0x7f6614156730 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66140ccdc0 0x7f661414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 --2- 172.21.15.45:0/2467466665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6614155cd0 0x7f66140b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 >> 172.21.15.45:0/2467466665 conn(0x7f661406f090 msgr2=0x7f6614160770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 shutdown_connections 2023-10-06T05:27:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.733+0000 7f66199d9700 1 -- 172.21.15.45:0/2467466665 wait complete. 2023-10-06T05:27:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 350 2023-10-06T05:27:05.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:05.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.878+0000 7ff5c5efc700 1 Processor -- start 2023-10-06T05:27:05.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.878+0000 7ff5c5efc700 1 -- start start 2023-10-06T05:27:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5c5efc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5c01560a0 0x7ff5c015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5c5efc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5c5efc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c00cd740 con 0x7ff5c0155cd0 2023-10-06T05:27:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5c5efc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00ae6b0 con 0x7ff5c01560a0 2023-10-06T05:27:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5c5efc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00ae8d0 con 0x7ff5c00ccdc0 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5c01560a0 0x7ff5c015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5c01560a0 0x7ff5c015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43946/0 (socket says 172.21.15.45:43946) 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 -- 172.21.15.45:0/2543109470 learned_addr learned my addr 172.21.15.45:0/2543109470 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5bffff700 1 --2- 172.21.15.45:0/2543109470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 -- 172.21.15.45:0/2543109470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 msgr2=0x7ff5c00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 --2- 172.21.15.45:0/2543109470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 -- 172.21.15.45:0/2543109470 >> v1:172.21.15.144:6789/0 conn(0x7ff5c0155cd0 legacy=0x7ff5c00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 -- 172.21.15.45:0/2543109470 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00aebb0 con 0x7ff5c01560a0 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5beffd700 1 --2- 172.21.15.45:0/2543109470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c01560a0 0x7ff5c015a730 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40099b0 tx=0x7ff5b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a2699a7d0053d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.879+0000 7ff5be7fc700 1 -- 172.21.15.45:0/2543109470 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400b860 con 0x7ff5c01560a0 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5be7fc700 1 -- 172.21.15.45:0/2543109470 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400b9c0 con 0x7ff5c01560a0 2023-10-06T05:27:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5be7fc700 1 -- 172.21.15.45:0/2543109470 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400bd60 con 0x7ff5c01560a0 2023-10-06T05:27:05.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 -- 172.21.15.45:0/2543109470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c01560a0 msgr2=0x7ff5c015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/2543109470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c01560a0 0x7ff5c015a730 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40099b0 tx=0x7ff5b4009250 comp rx=0 tx=0).stop 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 -- 172.21.15.45:0/2543109470 shutdown_connections 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/2543109470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/2543109470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c01560a0 0x7ff5c015a730 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 -- 172.21.15.45:0/2543109470 >> 172.21.15.45:0/2543109470 conn(0x7ff5c006f090 msgr2=0x7ff5c006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 -- 172.21.15.45:0/2543109470 shutdown_connections 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.880+0000 7ff5c5efc700 1 -- 172.21.15.45:0/2543109470 wait complete. 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 Processor -- start 2023-10-06T05:27:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 -- start start 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c0165df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c0155cd0 0x7ff5c014ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5c01560a0 0x7ff5c015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c0166330 con 0x7ff5c0155cd0 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00b1540 con 0x7ff5c01560a0 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5c5efc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00b23f0 con 0x7ff5c00ccdc0 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.882+0000 7ff5beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c0155cd0 0x7ff5c014ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c0155cd0 0x7ff5c014ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43954/0 (socket says 172.21.15.45:43954) 2023-10-06T05:27:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 -- 172.21.15.45:0/3199931969 learned_addr learned my addr 172.21.15.45:0/3199931969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5bffff700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5c01560a0 0x7ff5c015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5bf7fe700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c0165df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 -- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 msgr2=0x7ff5c0165df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c0165df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 -- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5c01560a0 msgr2=0x7ff5c015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5c01560a0 0x7ff5c015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 -- 172.21.15.45:0/3199931969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b40094f0 con 0x7ff5c0155cd0 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5beffd700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c0155cd0 0x7ff5c014ff40 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009fd0 tx=0x7ff5b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5a77fe700 1 -- 172.21.15.45:0/3199931969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4024440 con 0x7ff5c0155cd0 2023-10-06T05:27:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5a77fe700 1 -- 172.21.15.45:0/3199931969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40245a0 con 0x7ff5c0155cd0 2023-10-06T05:27:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c00b1750 con 0x7ff5c0155cd0 2023-10-06T05:27:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5a77fe700 1 -- 172.21.15.45:0/3199931969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4024940 con 0x7ff5c0155cd0 2023-10-06T05:27:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.883+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c00c4070 con 0x7ff5c0155cd0 2023-10-06T05:27:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.884+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c006e2f0 con 0x7ff5c0155cd0 2023-10-06T05:27:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.885+0000 7ff5a77fe700 1 -- 172.21.15.45:0/3199931969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5b401a420 con 0x7ff5c0155cd0 2023-10-06T05:27:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.885+0000 7ff5a77fe700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5ac066f30 0x7ff5ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.886+0000 7ff5a77fe700 1 -- 172.21.15.45:0/3199931969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7ff5b40873c0 con 0x7ff5c0155cd0 2023-10-06T05:27:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.886+0000 7ff5bf7fe700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5ac066f30 0x7ff5ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.886+0000 7ff5bf7fe700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5ac066f30 0x7ff5ac0693e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0003880 tx=0x7ff5b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:05.890+0000 7ff5a77fe700 1 -- 172.21.15.45:0/3199931969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5b40535e0 con 0x7ff5c0155cd0 2023-10-06T05:27:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.094+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5c00aee90 con 0x7ff5c0155cd0 2023-10-06T05:27:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.095+0000 7ff5a77fe700 1 -- 172.21.15.45:0/3199931969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+22388 (secure 0 0 0) 0x7ff5b4015090 con 0x7ff5c0155cd0 2023-10-06T05:27:06.098 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:06.098 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":132,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:26:58.006054+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:49.182551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:49.732208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2681459400":"2023-10-07T05:23:52.452679+0000","172.21.15.45:6835/2971431427":"2023-10-07T05:24:37.100960+0000","172.21.15.45:6834/2681459400":"2023-10-07T05:23:52.452679+0000","172.21.15.45:6835/1013285853":"2023-10-07T05:23:45.414617+0000","172.21.15.45:6835/858773096":"2023-10-07T05:24:07.945434+0000","172.21.15.144:6834/4239891954":"2023-10-07T05:25:03.377085+0000","172.21.15.45:6834/3031536032":"2023-10-07T05:24:22.864925+0000","172.21.15.45:6834/3141743890":"2023-10-07T05:23:23.111877+0000","172.21.15.45:6834/681980478":"2023-10-07T05:23:12.050135+0000","172.21.15.45:6835/2532567321":"2023-10-07T05:23:15.870223+0000","172.21.15.45:6834/2405580153":"2023-10-07T05:23:43.392415+0000","172.21.15.45:6834/2971431427":"2023-10-07T05:24:37.100960+0000","172.21.15.144:6834/3602644226":"2023-10-07T05:24:45.157829+0000","172.21.15.144:6834/1302691811":"2023-10-07T05:26:43.923386+0000","172.21.15.45:6832/3755910187":"2023-10-07T05:22:28.384687+0000","172.21.15.144:6835/2001233142":"2023-10-07T05:26:17.517661+0000","172.21.15.45:6832/2550360138":"2023-10-07T05:22:23.359215+0000","172.21.15.45:6832/821082350":"2023-10-07T05:22:07.114456+0000","172.21.15.45:6835/2405580153":"2023-10-07T05:23:43.392415+0000","172.21.15.45:6832/3860010011":"2023-10-07T05:22:38.914897+0000","172.21.15.45:6835/3141743890":"2023-10-07T05:23:23.111877+0000","172.21.15.45:6832/4128377575":"2023-10-07T05:22:34.886263+0000","172.21.15.45:6835/2475927951":"2023-10-07T05:24:29.055945+0000","172.21.15.45:6833/4128377575":"2023-10-07T05:22:34.886263+0000","172.21.15.144:6834/3552713853":"2023-10-07T05:25:38.094009+0000","172.21.15.45:6833/2550360138":"2023-10-07T05:22:23.359215+0000","172.21.15.45:6834/858773096":"2023-10-07T05:24:07.945434+0000","172.21.15.45:6834/2475927951":"2023-10-07T05:24:29.055945+0000","172.21.15.45:6833/821082350":"2023-10-07T05:22:07.114456+0000","172.21.15.45:6834/3351321197":"2023-10-07T05:23:59.498576+0000","172.21.15.45:6833/3860010011":"2023-10-07T05:22:38.914897+0000","172.21.15.144:6835/2237111349":"2023-10-07T05:25:17.948375+0000","172.21.15.45:6835/681980478":"2023-10-07T05:23:12.050135+0000","172.21.15.45:6835/1996698078":"2023-10-07T05:24:25.023244+0000","172.21.15.45:6835/1915871760":"2023-10-07T05:24:03.901047+0000","172.21.15.45:6835/3031536032":"2023-10-07T05:24:22.864925+0000","172.21.15.45:6833/3755910187":"2023-10-07T05:22:28.384687+0000","172.21.15.45:6835/3277956876":"2023-10-07T05:22:42.945097+0000","172.21.15.45:6834/1915871760":"2023-10-07T05:24:03.901047+0000","172.21.15.45:6834/226461717":"2023-10-07T05:22:56.431657+0000","172.21.15.144:6835/4239891954":"2023-10-07T05:25:03.377085+0000","172.21.15.45:6834/2620071126":"2023-10-07T05:23:29.906056+0000","172.21.15.144:6835/3580730449":"2023-10-07T05:25:05.403592+0000","172.21.15.45:6834/3277956876":"2023-10-07T05:22:42.945097+0000","172.21.15.45:6835/226461717":"2023-10-07T05:22:56.431657+0000","172.21.15.45:6835/2531263654":"2023-10-07T05:23:09.893451+0000","172.21.15.45:6834/1013285853":"2023-10-07T05:23:45.414617+0000","172.21.15.45:6834/1996698078":"2023-10-07T05:24:25.023244+0000","172.21.15.144:6834/3433217004":"2023-10-07T05:24:33.068465+0000","172.21.15.45:6835/2620071126":"2023-10-07T05:23:29.906056+0000","172.21.15.144:6834/2156055156":"2023-10-07T05:24:40.893717+0000","172.21.15.144:6835/2156055156":"2023-10-07T05:24:40.893717+0000","172.21.15.144:6835/3602644226":"2023-10-07T05:24:45.157829+0000","172.21.15.144:6835/1309688991":"2023-10-07T05:24:49.191270+0000","172.21.15.144:6834/2062728889":"2023-10-07T05:26:06.022104+0000","172.21.15.144:6834/3580730449":"2023-10-07T05:25:05.403592+0000","172.21.15.144:6835/913913762":"2023-10-07T05:26:58.006032+0000","172.21.15.144:6834/3983487029":"2023-10-07T05:25:13.909730+0000","172.21.15.144:6835/3451214191":"2023-10-07T05:26:22.547760+0000","172.21.15.144:6834/243455709":"2023-10-07T05:25:09.429589+0000","172.21.15.144:6835/243455709":"2023-10-07T05:25:09.429589+0000","172.21.15.144:6835/2062728889":"2023-10-07T05:26:06.022104+0000","172.21.15.144:6834/913913762":"2023-10-07T05:26:58.006032+0000","172.21.15.144:6835/3983487029":"2023-10-07T05:25:13.909730+0000","172.21.15.144:6835/26720443":"2023-10-07T05:25:43.120106+0000","172.21.15.144:6835/3552713853":"2023-10-07T05:25:38.094009+0000","172.21.15.144:6834/1309688991":"2023-10-07T05:24:49.191270+0000","172.21.15.144:6834/26720443":"2023-10-07T05:25:43.120106+0000","172.21.15.144:6834/2001233142":"2023-10-07T05:26:17.517661+0000","172.21.15.144:6835/1309152247":"2023-10-07T05:26:50.920566+0000","172.21.15.144:6834/3451214191":"2023-10-07T05:26:22.547760+0000","172.21.15.45:6835/3351321197":"2023-10-07T05:23:59.498576+0000","172.21.15.144:6835/3433217004":"2023-10-07T05:24:33.068465+0000","172.21.15.144:6834/603338967":"2023-10-07T05:26:39.088317+0000","172.21.15.144:6834/2237111349":"2023-10-07T05:25:17.948375+0000","172.21.15.144:6835/603338967":"2023-10-07T05:26:39.088317+0000","172.21.15.144:6835/1302691811":"2023-10-07T05:26:43.923386+0000","172.21.15.45:6834/2531263654":"2023-10-07T05:23:09.893451+0000","172.21.15.45:6834/2532567321":"2023-10-07T05:23:15.870223+0000","172.21.15.144:6834/1309152247":"2023-10-07T05:26:50.920566+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5ac066f30 msgr2=0x7ff5ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5ac066f30 0x7ff5ac0693e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0003880 tx=0x7ff5b0009040 comp rx=0 tx=0).stop 2023-10-06T05:27:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c0155cd0 msgr2=0x7ff5c014ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c0155cd0 0x7ff5c014ff40 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009fd0 tx=0x7ff5b4017000 comp rx=0 tx=0).stop 2023-10-06T05:27:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 shutdown_connections 2023-10-06T05:27:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff5ac066f30 0x7ff5ac0693e0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5c01560a0 0x7ff5c015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5c00ccdc0 0x7ff5c0165df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 --2- 172.21.15.45:0/3199931969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c0155cd0 0x7ff5c014ff40 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 >> 172.21.15.45:0/3199931969 conn(0x7ff5c006f090 msgr2=0x7ff5c00adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 shutdown_connections 2023-10-06T05:27:06.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.098+0000 7ff5c5efc700 1 -- 172.21.15.45:0/3199931969 wait complete. 2023-10-06T05:27:06.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:27:06.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.244+0000 7f6cbce9b700 1 Processor -- start 2023-10-06T05:27:06.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.244+0000 7f6cbce9b700 1 -- start start 2023-10-06T05:27:06.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cbce9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6cb80c8ff0 0x7f6cb80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cbce9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8157200 0x7f6cb80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cbce9b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb815b000 con 0x7f6cb80caad0 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cbce9b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb815b2e0 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cbce9b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb815b5c0 con 0x7f6cb8157200 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cadd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6cb80caad0 0x7f6cb8156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47188/0 (socket says 172.21.15.45:47188) 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cadd9b700 1 -- 172.21.15.45:0/107745980 learned_addr learned my addr 172.21.15.45:0/107745980 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cb659c700 1 --2- 172.21.15.45:0/107745980 >> v2:172.21.15.144:3300/0 conn(0x7f6cb80c8ff0 0x7f6cb80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cb6d9d700 1 --2- 172.21.15.45:0/107745980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8157200 0x7f6cb80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cb5d9b700 1 -- 172.21.15.45:0/107745980 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296591673 0 0) 0x7f6cb815b000 con 0x7f6cb80caad0 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.245+0000 7f6cb5d9b700 1 -- 172.21.15.45:0/107745980 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c98003570 con 0x7f6cb80caad0 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb6d9d700 1 -- 172.21.15.45:0/107745980 >> v2:172.21.15.144:3300/0 conn(0x7f6cb80c8ff0 msgr2=0x7f6cb80c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb6d9d700 1 --2- 172.21.15.45:0/107745980 >> v2:172.21.15.144:3300/0 conn(0x7f6cb80c8ff0 0x7f6cb80c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb6d9d700 1 -- 172.21.15.45:0/107745980 >> v1:172.21.15.45:6789/0 conn(0x7f6cb80caad0 legacy=0x7f6cb8156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb6d9d700 1 -- 172.21.15.45:0/107745980 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb815b8a0 con 0x7f6cb8157200 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb659c700 1 --2- 172.21.15.45:0/107745980 >> v2:172.21.15.144:3300/0 conn(0x7f6cb80c8ff0 0x7f6cb80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb6d9d700 1 --2- 172.21.15.45:0/107745980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8157200 0x7f6cb80caf40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009e60 tx=0x7f6cb0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b91cf776d5043878 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb5d9b700 1 -- 172.21.15.45:0/107745980 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000e070 con 0x7f6cb8157200 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb5d9b700 1 -- 172.21.15.45:0/107745980 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb000bab0 con 0x7f6cb8157200 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.246+0000 7f6cb5d9b700 1 -- 172.21.15.45:0/107745980 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000be70 con 0x7f6cb8157200 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 -- 172.21.15.45:0/107745980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8157200 msgr2=0x7f6cb80caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/107745980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8157200 0x7f6cb80caf40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009e60 tx=0x7f6cb0009780 comp rx=0 tx=0).stop 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 -- 172.21.15.45:0/107745980 shutdown_connections 2023-10-06T05:27:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/107745980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8157200 0x7f6cb80caf40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/107745980 >> v2:172.21.15.144:3300/0 conn(0x7f6cb80c8ff0 0x7f6cb80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 -- 172.21.15.45:0/107745980 >> 172.21.15.45:0/107745980 conn(0x7f6cb80cdf50 msgr2=0x7f6cb80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 -- 172.21.15.45:0/107745980 shutdown_connections 2023-10-06T05:27:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.247+0000 7f6cbce9b700 1 -- 172.21.15.45:0/107745980 wait complete. 2023-10-06T05:27:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.248+0000 7f6cbce9b700 1 Processor -- start 2023-10-06T05:27:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.248+0000 7f6cbce9b700 1 -- start start 2023-10-06T05:27:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cbce9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb80c8ff0 0x7f6cb816aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cbce9b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb80caad0 0x7f6cb80b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cbce9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb8157200 0x7f6cb80b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cbce9b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb8159fe0 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cbce9b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb816b120 con 0x7f6cb8157200 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cbce9b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb816b400 con 0x7f6cb80caad0 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb80c8ff0 0x7f6cb816aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb80c8ff0 0x7f6cb816aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43970/0 (socket says 172.21.15.45:43970) 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 -- 172.21.15.45:0/3053526246 learned_addr learned my addr 172.21.15.45:0/3053526246 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb6d9d700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb8157200 0x7f6cb80b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cadd9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb80caad0 0x7f6cb80b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 -- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb80caad0 msgr2=0x7f6cb80b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb80caad0 0x7f6cb80b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 -- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb8157200 msgr2=0x7f6cb80b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb8157200 0x7f6cb80b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb659c700 1 -- 172.21.15.45:0/3053526246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009420 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cb6d9d700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb8157200 0x7f6cb80b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.249+0000 7f6cadd9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb80caad0 0x7f6cb80b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.250+0000 7f6cb659c700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb80c8ff0 0x7f6cb816aa90 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800ec80 tx=0x7f6ca80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.250+0000 7f6caeffd700 1 -- 172.21.15.45:0/3053526246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800c9d0 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.250+0000 7f6caeffd700 1 -- 172.21.15.45:0/3053526246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca800cb30 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.250+0000 7f6caeffd700 1 -- 172.21.15.45:0/3053526246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800ceb0 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.250+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb0009a20 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.250+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb80ae560 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.251+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb806f410 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.251+0000 7f6caeffd700 1 -- 172.21.15.45:0/3053526246 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ca8013680 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.252+0000 7f6caeffd700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c9c066f30 0x7f6c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.252+0000 7f6caeffd700 1 -- 172.21.15.45:0/3053526246 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f6ca80859d0 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.252+0000 7f6cadd9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c9c066f30 0x7f6c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.253+0000 7f6cadd9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c9c066f30 0x7f6c9c0693e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6cb81509b0 tx=0x7f6ca0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.256+0000 7f6caeffd700 1 -- 172.21.15.45:0/3053526246 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ca8051bf0 con 0x7f6cb80c8ff0 2023-10-06T05:27:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:06.503+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6cb806eb60 con 0x7f6cb80c8ff0 2023-10-06T05:27:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.038+0000 7f6caeffd700 1 -- 172.21.15.45:0/3053526246 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v351) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6ca8055210 con 0x7f6cb80c8ff0 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c9c066f30 msgr2=0x7f6c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c9c066f30 0x7f6c9c0693e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6cb81509b0 tx=0x7f6ca0009040 comp rx=0 tx=0).stop 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb80c8ff0 msgr2=0x7f6cb816aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb80c8ff0 0x7f6cb816aa90 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800ec80 tx=0x7f6ca80091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 shutdown_connections 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c9c066f30 0x7f6c9c0693e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb8157200 0x7f6cb80b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb80c8ff0 0x7f6cb816aa90 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 --2- 172.21.15.45:0/3053526246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb80caad0 0x7f6cb80b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 >> 172.21.15.45:0/3053526246 conn(0x7f6cb80cdf50 msgr2=0x7f6cb806d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 shutdown_connections 2023-10-06T05:27:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.041+0000 7f6cbce9b700 1 -- 172.21.15.45:0/3053526246 wait complete. 2023-10-06T05:27:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:27:07.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:27:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.194+0000 7fa3128e0700 1 Processor -- start 2023-10-06T05:27:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.194+0000 7fa3128e0700 1 -- start start 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa3128e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 0x7fa30c067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa3128e0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c067fc0 con 0x7fa30c159520 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa3128e0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c0c9c90 con 0x7fa30c159150 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa3128e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c15d8e0 con 0x7fa30c067620 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa30c159520 0x7fa30c15ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59050/0 (socket says 172.21.15.45:59050) 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30b7fe700 1 -- 172.21.15.45:0/2542940140 learned_addr learned my addr 172.21.15.45:0/2542940140 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa310e7d700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 0x7fa30c067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636028314 0 0) 0x7fa300001110 con 0x7fa30c159520 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f4003570 con 0x7fa30c159520 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153246002 0 0) 0x7fa2f4003570 con 0x7fa30c159520 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 msgr2=0x7fa30c067a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 0x7fa30c067a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 >> v1:172.21.15.144:6789/0 conn(0x7fa30c159150 legacy=0x7fa30c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c15dbc0 con 0x7fa30c159520 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fa300001600 con 0x7fa30c159520 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.195+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 >> v1:172.21.15.45:6789/0 conn(0x7fa30c159520 legacy=0x7fa30c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 0x7fa2f4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f4003fd0 0x7fa2f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2f4004d50 0x7fa2f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c15d8e0 con 0x7fa2f4004d50 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f400bbf0 con 0x7fa2f4003fd0 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f400bdb0 con 0x7fa30c067620 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa310e7d700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2f4004d50 0x7fa2f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30b7fe700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f4003fd0 0x7fa2f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30bfff700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 0x7fa2f4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa310e7d700 1 -- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 msgr2=0x7fa2f4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa310e7d700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa30c067620 0x7fa2f4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa310e7d700 1 -- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f4003fd0 msgr2=0x7fa2f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa310e7d700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f4003fd0 0x7fa2f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa310e7d700 1 -- 172.21.15.45:0/2542940140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f4003570 con 0x7fa2f4004d50 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa310e7d700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2f4004d50 0x7fa2f40051c0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fa30400e210 tx=0x7fa30400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07381f69ce87d88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304009ac0 con 0x7fa2f4004d50 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30400f040 con 0x7fa2f4004d50 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.196+0000 7fa30affd700 1 -- 172.21.15.45:0/2542940140 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30400ecf0 con 0x7fa2f4004d50 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.197+0000 7fa3128e0700 1 -- 172.21.15.45:0/2542940140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2f4004d50 msgr2=0x7fa2f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.197+0000 7fa3128e0700 1 --2- 172.21.15.45:0/2542940140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2f4004d50 0x7fa2f40051c0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fa30400e210 tx=0x7fa30400b040 comp rx=0 tx=0).stop 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.197+0000 7fa310e7d700 1 -- 172.21.15.45:0/2542940140 reap_dead start 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.197+0000 7fa3128e0700 1 -- 172.21.15.45:0/2542940140 shutdown_connections 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.197+0000 7fa3128e0700 1 -- 172.21.15.45:0/2542940140 >> 172.21.15.45:0/2542940140 conn(0x7fa30c0691c0 msgr2=0x7fa30c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.197+0000 7fa3128e0700 1 -- 172.21.15.45:0/2542940140 shutdown_connections 2023-10-06T05:27:07.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.197+0000 7fa3128e0700 1 -- 172.21.15.45:0/2542940140 wait complete. 2023-10-06T05:27:07.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.198+0000 7fa3128e0700 1 Processor -- start 2023-10-06T05:27:07.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.198+0000 7fa3128e0700 1 -- start start 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa3128e0700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2f4004d50 0x7fa30c0c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa3128e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c159900 0x7fa30c159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa3128e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30c147490 0x7fa30c147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa3128e0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c147e40 con 0x7fa30c159900 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa3128e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0c4220 con 0x7fa30c147490 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa3128e0700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0c4500 con 0x7fa2f4004d50 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa30b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c159900 0x7fa30c159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa30b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c159900 0x7fa30c159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37460/0 (socket says 172.21.15.45:37460) 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa30b7fe700 1 -- 172.21.15.45:0/539726387 learned_addr learned my addr 172.21.15.45:0/539726387 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa310e7d700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30c147490 0x7fa30c147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.199+0000 7fa30bfff700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2f4004d50 0x7fa30c0c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa30b7fe700 1 -- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2f4004d50 msgr2=0x7fa30c0c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa30b7fe700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2f4004d50 0x7fa30c0c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa30b7fe700 1 -- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30c147490 msgr2=0x7fa30c147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa30b7fe700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30c147490 0x7fa30c147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa30b7fe700 1 -- 172.21.15.45:0/539726387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0c47e0 con 0x7fa30c159900 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa30b7fe700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c159900 0x7fa30c159d70 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fa300007390 tx=0x7fa300007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa2f3fff700 1 -- 172.21.15.45:0/539726387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300018030 con 0x7fa30c159900 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa2f3fff700 1 -- 172.21.15.45:0/539726387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300007750 con 0x7fa30c159900 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c0c4ac0 con 0x7fa30c159900 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa2f3fff700 1 -- 172.21.15.45:0/539726387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30000fac0 con 0x7fa30c159900 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.200+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c0c3ef0 con 0x7fa30c159900 2023-10-06T05:27:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.201+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c0c4fe0 con 0x7fa30c159900 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.202+0000 7fa2f3fff700 1 -- 172.21.15.45:0/539726387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa30001b070 con 0x7fa30c159900 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.202+0000 7fa2f3fff700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2f8066f30 0x7fa2f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.202+0000 7fa2f3fff700 1 -- 172.21.15.45:0/539726387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7fa300019070 con 0x7fa30c159900 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.202+0000 7fa30bfff700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2f8066f30 0x7fa2f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.203+0000 7fa30bfff700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2f8066f30 0x7fa2f80693e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc007e20 tx=0x7fa2fc007d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.206+0000 7fa2f3fff700 1 -- 172.21.15.45:0/539726387 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa30c0c4fe0 con 0x7fa30c159900 2023-10-06T05:27:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:07.450+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa30c067fc0 con 0x7fa30c159900 2023-10-06T05:27:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.043+0000 7fa2f3fff700 1 -- 172.21.15.45:0/539726387 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v352) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa30c067fc0 con 0x7fa30c159900 2023-10-06T05:27:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2f8066f30 msgr2=0x7fa2f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2f8066f30 0x7fa2f80693e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc007e20 tx=0x7fa2fc007d10 comp rx=0 tx=0).stop 2023-10-06T05:27:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c159900 msgr2=0x7fa30c159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c159900 0x7fa30c159d70 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fa300007390 tx=0x7fa300007a50 comp rx=0 tx=0).stop 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 shutdown_connections 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2f8066f30 0x7fa2f80693e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30c147490 0x7fa30c147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2f4004d50 0x7fa30c0c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 --2- 172.21.15.45:0/539726387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa30c159900 0x7fa30c159d70 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 >> 172.21.15.45:0/539726387 conn(0x7fa30c0691c0 msgr2=0x7fa30c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 shutdown_connections 2023-10-06T05:27:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.046+0000 7fa3128e0700 1 -- 172.21.15.45:0/539726387 wait complete. 2023-10-06T05:27:08.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:27:08.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.204+0000 7f538415a700 1 Processor -- start 2023-10-06T05:27:08.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.204+0000 7f538415a700 1 -- start start 2023-10-06T05:27:08.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f538415a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f537c15d590 0x7f537c0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f538415a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0c91f0 0x7f537c167290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f538415a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0cc4b0 con 0x7f537c0cacb0 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f538415a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0cb130 con 0x7f537c15d590 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f538415a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0c9730 con 0x7f537c0c91f0 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f53816f5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f537c0cacb0 0x7f537c0c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59054/0 (socket says 172.21.15.45:59054) 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f53816f5700 1 -- 172.21.15.45:0/2712532357 learned_addr learned my addr 172.21.15.45:0/2712532357 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.205+0000 7f5381ef6700 1 --2- 172.21.15.45:0/2712532357 >> v2:172.21.15.144:3300/0 conn(0x7f537c15d590 0x7f537c0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f53826f7700 1 --2- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0c91f0 0x7f537c167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5380ef4700 1 -- 172.21.15.45:0/2712532357 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619593022 0 0) 0x7f537c0cc4b0 con 0x7f537c0cacb0 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5380ef4700 1 -- 172.21.15.45:0/2712532357 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5368003570 con 0x7f537c0cacb0 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5381ef6700 1 -- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0c91f0 msgr2=0x7f537c167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:08.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5381ef6700 1 --2- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0c91f0 0x7f537c167290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5381ef6700 1 -- 172.21.15.45:0/2712532357 >> v1:172.21.15.45:6789/0 conn(0x7f537c0cacb0 legacy=0x7f537c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5380ef4700 1 -- 172.21.15.45:0/2712532357 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2812235802 0 0) 0x7f5368003570 con 0x7f537c0cacb0 2023-10-06T05:27:08.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5381ef6700 1 -- 172.21.15.45:0/2712532357 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0c9a10 con 0x7f537c15d590 2023-10-06T05:27:08.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5381ef6700 1 --2- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c15d590 0x7f537c0ca770 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f53780098e0 tx=0x7f53780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d434151a7dde1351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5380ef4700 1 -- 172.21.15.45:0/2712532357 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370001470 con 0x7f537c15d590 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5380ef4700 1 -- 172.21.15.45:0/2712532357 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5378011420 con 0x7f537c15d590 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.206+0000 7f5380ef4700 1 -- 172.21.15.45:0/2712532357 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780117c0 con 0x7f537c15d590 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 -- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c15d590 msgr2=0x7f537c0ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 --2- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c15d590 0x7f537c0ca770 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f53780098e0 tx=0x7f53780091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 -- 172.21.15.45:0/2712532357 shutdown_connections 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 --2- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0c91f0 0x7f537c167290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 --2- 172.21.15.45:0/2712532357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c15d590 0x7f537c0ca770 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 -- 172.21.15.45:0/2712532357 >> 172.21.15.45:0/2712532357 conn(0x7f537c0680b0 msgr2=0x7f537c0cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 -- 172.21.15.45:0/2712532357 shutdown_connections 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.207+0000 7f538415a700 1 -- 172.21.15.45:0/2712532357 wait complete. 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.208+0000 7f538415a700 1 Processor -- start 2023-10-06T05:27:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.208+0000 7f538415a700 1 -- start start 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f538415a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c0c91f0 0x7f537c149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f538415a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0cacb0 0x7f537c16db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f538415a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f537c15d590 0x7f537c15af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f538415a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c171130 con 0x7f537c15d590 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f538415a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1716b0 con 0x7f537c0c91f0 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f538415a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c16e5d0 con 0x7f537c0cacb0 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f53826f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f537c15d590 0x7f537c15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f53826f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f537c15d590 0x7f537c15af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37464/0 (socket says 172.21.15.45:37464) 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f53826f7700 1 -- 172.21.15.45:0/566461853 learned_addr learned my addr 172.21.15.45:0/566461853 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f53816f5700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0cacb0 0x7f537c16db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.209+0000 7f5381ef6700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c0c91f0 0x7f537c149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f53826f7700 1 -- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0cacb0 msgr2=0x7f537c16db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f53826f7700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0cacb0 0x7f537c16db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f53826f7700 1 -- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c0c91f0 msgr2=0x7f537c149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f53826f7700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c0c91f0 0x7f537c149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f53826f7700 1 -- 172.21.15.45:0/566461853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5378009420 con 0x7f537c15d590 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f53816f5700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0cacb0 0x7f537c16db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f53826f7700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f537c15d590 0x7f537c15af50 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5374009df0 tx=0x7f537400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f536dffb700 1 -- 172.21.15.45:0/566461853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400cb50 con 0x7f537c15d590 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f536dffb700 1 -- 172.21.15.45:0/566461853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537400f040 con 0x7f537c15d590 2023-10-06T05:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c16e8b0 con 0x7f537c15d590 2023-10-06T05:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f536dffb700 1 -- 172.21.15.45:0/566461853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374014480 con 0x7f537c15d590 2023-10-06T05:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.210+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1495c0 con 0x7f537c15d590 2023-10-06T05:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.211+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c04c380 con 0x7f537c15d590 2023-10-06T05:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.212+0000 7f536dffb700 1 -- 172.21.15.45:0/566461853 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5374014830 con 0x7f537c15d590 2023-10-06T05:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.212+0000 7f536dffb700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5360066f30 0x7f53600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.212+0000 7f536dffb700 1 -- 172.21.15.45:0/566461853 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9818+0+0 (secure 0 0 0) 0x7f5374085ed0 con 0x7f537c15d590 2023-10-06T05:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.212+0000 7f5381ef6700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5360066f30 0x7f53600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.213+0000 7f5381ef6700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5360066f30 0x7f53600693e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f53780030c0 tx=0x7f5378016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.216+0000 7f536dffb700 1 -- 172.21.15.45:0/566461853 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53740520f0 con 0x7f537c15d590 2023-10-06T05:27:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:08.419+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f537c0c3d00 con 0x7f537c15d590 2023-10-06T05:27:08.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:08.531 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:09.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.050+0000 7f536dffb700 1 -- 172.21.15.45:0/566461853 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7f53740180d0 con 0x7f537c15d590 2023-10-06T05:27:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5360066f30 msgr2=0x7f53600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5360066f30 0x7f53600693e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f53780030c0 tx=0x7f5378016000 comp rx=0 tx=0).stop 2023-10-06T05:27:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f537c15d590 msgr2=0x7f537c15af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f537c15d590 0x7f537c15af50 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5374009df0 tx=0x7f537400b040 comp rx=0 tx=0).stop 2023-10-06T05:27:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 shutdown_connections 2023-10-06T05:27:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5360066f30 0x7f53600693e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f537c15d590 0x7f537c15af50 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f537c0c91f0 0x7f537c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 --2- 172.21.15.45:0/566461853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c0cacb0 0x7f537c16db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.057+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 >> 172.21.15.45:0/566461853 conn(0x7f537c0680b0 msgr2=0x7f537c0cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.058+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 shutdown_connections 2023-10-06T05:27:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.059+0000 7f538415a700 1 -- 172.21.15.45:0/566461853 wait complete. 2023-10-06T05:27:09.063 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:27:09.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:27:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.217+0000 7f47932b9700 1 Processor -- start 2023-10-06T05:27:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.217+0000 7f47932b9700 1 -- start start 2023-10-06T05:27:09.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.218+0000 7f47932b9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f478c067bb0 0x7f478c1404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.218+0000 7f47932b9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c0ca7f0 con 0x7f478c0677e0 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.218+0000 7f47932b9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c0ccdf0 con 0x7f478c0c7500 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.218+0000 7f47932b9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478c068010 con 0x7f478c067bb0 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.218+0000 7f4791055700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f478c0677e0 0x7f478c0c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59066/0 (socket says 172.21.15.45:59066) 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.218+0000 7f4791055700 1 -- 172.21.15.45:0/2294357486 learned_addr learned my addr 172.21.15.45:0/2294357486 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.218+0000 7f4791856700 1 --2- 172.21.15.45:0/2294357486 >> v2:172.21.15.144:3300/0 conn(0x7f478c067bb0 0x7f478c1404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f477ffff700 1 -- 172.21.15.45:0/2294357486 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067501823 0 0) 0x7f478c0ca7f0 con 0x7f478c0677e0 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f477ffff700 1 -- 172.21.15.45:0/2294357486 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4778003570 con 0x7f478c0677e0 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f4791856700 1 -- 172.21.15.45:0/2294357486 >> v1:172.21.15.144:6789/0 conn(0x7f478c0c7500 legacy=0x7f478c0ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f477ffff700 1 -- 172.21.15.45:0/2294357486 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3297641123 0 0) 0x7f4778003570 con 0x7f478c0677e0 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f4791856700 1 -- 172.21.15.45:0/2294357486 >> v1:172.21.15.45:6789/0 conn(0x7f478c0677e0 legacy=0x7f478c0c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f4791856700 1 -- 172.21.15.45:0/2294357486 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0c9f50 con 0x7f478c067bb0 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f4791856700 1 --2- 172.21.15.45:0/2294357486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 0x7f478c1404c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4784009e90 tx=0x7f4784009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27674d4140654381 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f477ffff700 1 -- 172.21.15.45:0/2294357486 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478400e070 con 0x7f478c067bb0 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f477ffff700 1 -- 172.21.15.45:0/2294357486 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478400b920 con 0x7f478c067bb0 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.219+0000 7f477ffff700 1 -- 172.21.15.45:0/2294357486 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780001470 con 0x7f478c067bb0 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.220+0000 7f47932b9700 1 -- 172.21.15.45:0/2294357486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 msgr2=0x7f478c1404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.220+0000 7f47932b9700 1 --2- 172.21.15.45:0/2294357486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 0x7f478c1404c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4784009e90 tx=0x7f4784009780 comp rx=0 tx=0).stop 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.220+0000 7f47932b9700 1 -- 172.21.15.45:0/2294357486 shutdown_connections 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.220+0000 7f47932b9700 1 --2- 172.21.15.45:0/2294357486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 0x7f478c1404c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.220+0000 7f47932b9700 1 -- 172.21.15.45:0/2294357486 >> 172.21.15.45:0/2294357486 conn(0x7f478c068730 msgr2=0x7f478c068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.220+0000 7f47932b9700 1 -- 172.21.15.45:0/2294357486 shutdown_connections 2023-10-06T05:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.220+0000 7f47932b9700 1 -- 172.21.15.45:0/2294357486 wait complete. 2023-10-06T05:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.221+0000 7f47932b9700 1 Processor -- start 2023-10-06T05:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.221+0000 7f47932b9700 1 -- start start 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f47932b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f478c0677e0 0x7f478c16a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f47932b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 0x7f478c13d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f47932b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f478c0c7500 0x7f478c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f47932b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0c4910 con 0x7f478c0677e0 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f47932b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0c50b0 con 0x7f478c067bb0 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f47932b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0b2680 con 0x7f478c0c7500 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f4791055700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f478c0677e0 0x7f478c16a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f4791055700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f478c0677e0 0x7f478c16a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37466/0 (socket says 172.21.15.45:37466) 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.222+0000 7f4791055700 1 -- 172.21.15.45:0/2374121780 learned_addr learned my addr 172.21.15.45:0/2374121780 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4790854700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 0x7f478c13d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4791856700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f478c0c7500 0x7f478c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4791055700 1 -- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f478c0c7500 msgr2=0x7f478c0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4791055700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f478c0c7500 0x7f478c0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4791055700 1 -- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 msgr2=0x7f478c13d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4791055700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 0x7f478c13d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4791055700 1 -- 172.21.15.45:0/2374121780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784009a20 con 0x7f478c0677e0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f4791055700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f478c0677e0 0x7f478c16a130 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f478000cbc0 tx=0x7f4780001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f477d7fa700 1 -- 172.21.15.45:0/2374121780 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780007360 con 0x7f478c0677e0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f477d7fa700 1 -- 172.21.15.45:0/2374121780 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47800074e0 con 0x7f478c0677e0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c13d790 con 0x7f478c0677e0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f477d7fa700 1 -- 172.21.15.45:0/2374121780 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780004830 con 0x7f478c0677e0 2023-10-06T05:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.223+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c15d110 con 0x7f478c0677e0 2023-10-06T05:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.224+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c04c380 con 0x7f478c0677e0 2023-10-06T05:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.225+0000 7f477d7fa700 1 -- 172.21.15.45:0/2374121780 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4780004c00 con 0x7f478c0677e0 2023-10-06T05:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.225+0000 7f477d7fa700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4770066f30 0x7f47700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.225+0000 7f477d7fa700 1 -- 172.21.15.45:0/2374121780 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 9380+0+0 (secure 0 0 0) 0x7f4780084aa0 con 0x7f478c0677e0 2023-10-06T05:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.225+0000 7f4790854700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4770066f30 0x7f47700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.226+0000 7f4790854700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4770066f30 0x7f47700693e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f478c0abdc0 tx=0x7f4788009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:09.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.230+0000 7f477d7fa700 1 -- 172.21.15.45:0/2374121780 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4780050ef0 con 0x7f478c0677e0 2023-10-06T05:27:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:09.434+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f478c13e910 con 0x7f478c0677e0 2023-10-06T05:27:10.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.053+0000 7f477d7fa700 1 -- 172.21.15.45:0/2374121780 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 169+0+0 (secure 0 0 0) 0x7f478c13e910 con 0x7f478c0677e0 2023-10-06T05:27:10.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.057+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4770066f30 msgr2=0x7f47700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.057+0000 7f47932b9700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4770066f30 0x7f47700693e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f478c0abdc0 tx=0x7f4788009040 comp rx=0 tx=0).stop 2023-10-06T05:27:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.057+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f478c0677e0 msgr2=0x7f478c16a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.057+0000 7f47932b9700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f478c0677e0 0x7f478c16a130 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f478000cbc0 tx=0x7f4780001ad0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.057+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 shutdown_connections 2023-10-06T05:27:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.057+0000 7f47932b9700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4770066f30 0x7f47700693e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.057+0000 7f47932b9700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f478c0c7500 0x7f478c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.058+0000 7f47932b9700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f478c0677e0 0x7f478c16a130 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.058+0000 7f47932b9700 1 --2- 172.21.15.45:0/2374121780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f478c067bb0 0x7f478c13d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.058+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 >> 172.21.15.45:0/2374121780 conn(0x7f478c068730 msgr2=0x7f478c0c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.058+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 shutdown_connections 2023-10-06T05:27:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.058+0000 7f47932b9700 1 -- 172.21.15.45:0/2374121780 wait complete. 2023-10-06T05:27:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-06T05:27:10.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.221+0000 7f6a843d9700 1 Processor -- start 2023-10-06T05:27:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.221+0000 7f6a843d9700 1 -- start start 2023-10-06T05:27:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.222+0000 7f6a843d9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a7c153840 0x7f6a7c15a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.222+0000 7f6a843d9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 0x7f6a7c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.222+0000 7f6a843d9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0ce770 con 0x7f6a7c15d1c0 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a843d9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c153c10 con 0x7f6a7c153840 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a843d9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15d650 con 0x7f6a7c15cd70 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a82976700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6a7c15d1c0 0x7f6a7c068bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59078/0 (socket says 172.21.15.45:59078) 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a82976700 1 -- 172.21.15.45:0/3004331564 learned_addr learned my addr 172.21.15.45:0/3004331564 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a82175700 1 --2- 172.21.15.45:0/3004331564 >> v2:172.21.15.144:3300/0 conn(0x7f6a7c153840 0x7f6a7c15a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600282318 0 0) 0x7f6a7c0ce770 con 0x7f6a7c15d1c0 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81974700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 0x7f6a7c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a68003570 con 0x7f6a7c15d1c0 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1571708075 0 0) 0x7f6a68003570 con 0x7f6a7c15d1c0 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 msgr2=0x7f6a7c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 0x7f6a7c0ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 >> v2:172.21.15.144:3300/0 conn(0x7f6a7c153840 msgr2=0x7f6a7c15a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 --2- 172.21.15.45:0/3004331564 >> v2:172.21.15.144:3300/0 conn(0x7f6a7c153840 0x7f6a7c15a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c15d930 con 0x7f6a7c15d1c0 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f6a740017f0 con 0x7f6a7c15d1c0 2023-10-06T05:27:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 >> v1:172.21.15.45:6789/0 conn(0x7f6a7c15d1c0 legacy=0x7f6a7c068bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a68003a60 0x7f6a68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a68004810 0x7f6a68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 0x7f6a680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c153c10 con 0x7f6a68004810 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15d650 con 0x7f6a68003a60 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.223+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6800ba60 con 0x7f6a7c15cd70 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a82175700 1 --2- 172.21.15.45:0/3004331564 >> v2:172.21.15.144:3300/0 conn(0x7f6a7c153840 0x7f6a7c15a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81974700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a68004810 0x7f6a68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a82976700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 0x7f6a680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a82175700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a68003a60 0x7f6a68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81974700 1 -- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 msgr2=0x7f6a680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81974700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a7c15cd70 0x7f6a680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81974700 1 -- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a68003a60 msgr2=0x7f6a68003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81974700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a68003a60 0x7f6a68003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81974700 1 -- 172.21.15.45:0/3004331564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a68003570 con 0x7f6a68004810 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a82175700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a68003a60 0x7f6a68003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81974700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a68004810 0x7f6a68004c80 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009c30 tx=0x7f6a6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cd86a7bb5149ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00e710 con 0x7f6a68004810 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c00e870 con 0x7f6a68004810 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.224+0000 7f6a81173700 1 -- 172.21.15.45:0/3004331564 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00ec30 con 0x7f6a68004810 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.225+0000 7f6a843d9700 1 -- 172.21.15.45:0/3004331564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a68004810 msgr2=0x7f6a68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.225+0000 7f6a843d9700 1 --2- 172.21.15.45:0/3004331564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a68004810 0x7f6a68004c80 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009c30 tx=0x7f6a6c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.225+0000 7f6a82976700 1 -- 172.21.15.45:0/3004331564 reap_dead start 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.225+0000 7f6a843d9700 1 -- 172.21.15.45:0/3004331564 shutdown_connections 2023-10-06T05:27:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.225+0000 7f6a843d9700 1 -- 172.21.15.45:0/3004331564 >> 172.21.15.45:0/3004331564 conn(0x7f6a7c06ed50 msgr2=0x7f6a7c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.225+0000 7f6a843d9700 1 -- 172.21.15.45:0/3004331564 shutdown_connections 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.225+0000 7f6a843d9700 1 -- 172.21.15.45:0/3004331564 wait complete. 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.226+0000 7f6a843d9700 1 Processor -- start 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.226+0000 7f6a843d9700 1 -- start start 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a843d9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a68004810 0x7f6a7c150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a843d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a7c158bf0 0x7f6a7c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a843d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a7c0b2ee0 0x7f6a7c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a843d9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b34a0 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a843d9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b3780 con 0x7f6a7c158bf0 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a843d9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b3a60 con 0x7f6a68004810 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a7c0b2ee0 0x7f6a7c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a7c0b2ee0 0x7f6a7c15c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37498/0 (socket says 172.21.15.45:37498) 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 -- 172.21.15.45:0/351586770 learned_addr learned my addr 172.21.15.45:0/351586770 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82175700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a68004810 0x7f6a7c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a81974700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a7c158bf0 0x7f6a7c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 -- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a68004810 msgr2=0x7f6a7c150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a68004810 0x7f6a7c150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 -- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a7c158bf0 msgr2=0x7f6a7c15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a7c158bf0 0x7f6a7c15bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82976700 1 -- 172.21.15.45:0/351586770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0c8060 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a82175700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a68004810 0x7f6a7c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.227+0000 7f6a81974700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a7c158bf0 0x7f6a7c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:10.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.228+0000 7f6a82976700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a7c0b2ee0 0x7f6a7c15c300 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f6a74000f80 tx=0x7f6a74002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.228+0000 7f6a727fc700 1 -- 172.21.15.45:0/351586770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400adf0 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.228+0000 7f6a727fc700 1 -- 172.21.15.45:0/351586770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a740133f0 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.228+0000 7f6a727fc700 1 -- 172.21.15.45:0/351586770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a740137b0 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.228+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c0c8340 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.228+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c0b5d80 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.229+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c0c8530 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.229+0000 7f6a727fc700 1 -- 172.21.15.45:0/351586770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a74013ba0 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.230+0000 7f6a727fc700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a60066f30 0x7f6a600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.230+0000 7f6a727fc700 1 -- 172.21.15.45:0/351586770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8966+0+0 (secure 0 0 0) 0x7f6a74085b70 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.230+0000 7f6a82175700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a60066f30 0x7f6a600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.230+0000 7f6a82175700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a60066f30 0x7f6a600693e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006f00 tx=0x7f6a78008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.234+0000 7f6a727fc700 1 -- 172.21.15.45:0/351586770 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a740520e0 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.436+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6a7c15dc10 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.439+0000 7f6a727fc700 1 -- 172.21.15.45:0/351586770 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6a74055700 con 0x7f6a7c0b2ee0 2023-10-06T05:27:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a60066f30 msgr2=0x7f6a600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a60066f30 0x7f6a600693e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006f00 tx=0x7f6a78008040 comp rx=0 tx=0).stop 2023-10-06T05:27:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a7c0b2ee0 msgr2=0x7f6a7c15c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a7c0b2ee0 0x7f6a7c15c300 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f6a74000f80 tx=0x7f6a74002210 comp rx=0 tx=0).stop 2023-10-06T05:27:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 shutdown_connections 2023-10-06T05:27:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a60066f30 0x7f6a600693e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a7c0b2ee0 0x7f6a7c15c300 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a68004810 0x7f6a7c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 --2- 172.21.15.45:0/351586770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a7c158bf0 0x7f6a7c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 >> 172.21.15.45:0/351586770 conn(0x7f6a7c06ed50 msgr2=0x7f6a7c15ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:10.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 shutdown_connections 2023-10-06T05:27:10.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.442+0000 7f6a843d9700 1 -- 172.21.15.45:0/351586770 wait complete. 2023-10-06T05:27:10.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2023-10-06T05:27:10.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.587+0000 7f06500b9700 1 Processor -- start 2023-10-06T05:27:10.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.587+0000 7f06500b9700 1 -- start start 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.588+0000 7f06500b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0648153620 0x7f064815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.588+0000 7f06500b9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06480ce690 con 0x7f06480ccfc0 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.588+0000 7f06500b9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064815d510 con 0x7f06481539f0 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.588+0000 7f06500b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064815a7a0 con 0x7f0648153620 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.588+0000 7f064e656700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06480ccfc0 0x7f06480ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59090/0 (socket says 172.21.15.45:59090) 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.588+0000 7f064e656700 1 -- 172.21.15.45:0/1041686934 learned_addr learned my addr 172.21.15.45:0/1041686934 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064de55700 1 --2- 172.21.15.45:0/1041686934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0648153620 0x7f064815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064ce53700 1 -- 172.21.15.45:0/1041686934 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712542042 0 0) 0x7f06480ce690 con 0x7f06480ccfc0 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064ce53700 1 -- 172.21.15.45:0/1041686934 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0634003570 con 0x7f06480ccfc0 2023-10-06T05:27:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064ce53700 1 -- 172.21.15.45:0/1041686934 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875013382 0 0) 0x7f064815d510 con 0x7f06481539f0 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064ce53700 1 -- 172.21.15.45:0/1041686934 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06480ce690 con 0x7f06481539f0 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064de55700 1 -- 172.21.15.45:0/1041686934 >> v1:172.21.15.144:6789/0 conn(0x7f06481539f0 legacy=0x7f06480cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064de55700 1 -- 172.21.15.45:0/1041686934 >> v1:172.21.15.45:6789/0 conn(0x7f06480ccfc0 legacy=0x7f06480ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064de55700 1 -- 172.21.15.45:0/1041686934 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064815d7f0 con 0x7f0648153620 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064de55700 1 --2- 172.21.15.45:0/1041686934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0648153620 0x7f064815a260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0638009910 tx=0x7f06380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7c91f586b486f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064ce53700 1 -- 172.21.15.45:0/1041686934 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638011630 con 0x7f0648153620 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064ce53700 1 -- 172.21.15.45:0/1041686934 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0638011790 con 0x7f0648153620 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.589+0000 7f064ce53700 1 -- 172.21.15.45:0/1041686934 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638011b30 con 0x7f0648153620 2023-10-06T05:27:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.590+0000 7f06500b9700 1 -- 172.21.15.45:0/1041686934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0648153620 msgr2=0x7f064815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.590+0000 7f06500b9700 1 --2- 172.21.15.45:0/1041686934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0648153620 0x7f064815a260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0638009910 tx=0x7f06380091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.590+0000 7f06500b9700 1 -- 172.21.15.45:0/1041686934 shutdown_connections 2023-10-06T05:27:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.590+0000 7f06500b9700 1 --2- 172.21.15.45:0/1041686934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0648153620 0x7f064815a260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.590+0000 7f06500b9700 1 -- 172.21.15.45:0/1041686934 >> 172.21.15.45:0/1041686934 conn(0x7f064806ed50 msgr2=0x7f064806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.590+0000 7f06500b9700 1 -- 172.21.15.45:0/1041686934 shutdown_connections 2023-10-06T05:27:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.590+0000 7f06500b9700 1 -- 172.21.15.45:0/1041686934 wait complete. 2023-10-06T05:27:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.591+0000 7f06500b9700 1 Processor -- start 2023-10-06T05:27:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.591+0000 7f06500b9700 1 -- start start 2023-10-06T05:27:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f06500b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480ccfc0 0x7f064814e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f06500b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0648153620 0x7f0648156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f06500b9700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f064814eaf0 0x7f06481565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f06500b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648154c30 con 0x7f06480ccfc0 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f06500b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481547a0 con 0x7f0648153620 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f06500b9700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0648156b90 con 0x7f064814eaf0 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f064de55700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480ccfc0 0x7f064814e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f064de55700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480ccfc0 0x7f064814e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37506/0 (socket says 172.21.15.45:37506) 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f064de55700 1 -- 172.21.15.45:0/1793950053 learned_addr learned my addr 172.21.15.45:0/1793950053 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f064e656700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f064814eaf0 0x7f06481565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f064d654700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0648153620 0x7f0648156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.592+0000 7f064de55700 1 -- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f064814eaf0 msgr2=0x7f06481565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f064de55700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f064814eaf0 0x7f06481565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f064de55700 1 -- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0648153620 msgr2=0x7f0648156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f064de55700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0648153620 0x7f0648156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f064de55700 1 -- 172.21.15.45:0/1793950053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009420 con 0x7f06480ccfc0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f064d654700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0648153620 0x7f0648156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f064de55700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480ccfc0 0x7f064814e5b0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f0638009fd0 tx=0x7f0638017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f063dffb700 1 -- 172.21.15.45:0/1793950053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063801f070 con 0x7f06480ccfc0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f063dffb700 1 -- 172.21.15.45:0/1793950053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0638011900 con 0x7f06480ccfc0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0648153e70 con 0x7f06480ccfc0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f063dffb700 1 -- 172.21.15.45:0/1793950053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638023410 con 0x7f06480ccfc0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.593+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0648154430 con 0x7f06480ccfc0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.594+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0648154a80 con 0x7f06480ccfc0 2023-10-06T05:27:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.595+0000 7f063dffb700 1 -- 172.21.15.45:0/1793950053 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0638023860 con 0x7f06480ccfc0 2023-10-06T05:27:10.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.595+0000 7f063dffb700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f062c066f30 0x7f062c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:10.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.595+0000 7f063dffb700 1 -- 172.21.15.45:0/1793950053 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8966+0+0 (secure 0 0 0) 0x7f06380863a0 con 0x7f06480ccfc0 2023-10-06T05:27:10.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.595+0000 7f064d654700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f062c066f30 0x7f062c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:10.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.596+0000 7f064d654700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f062c066f30 0x7f062c0693e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f06480c6940 tx=0x7f0644009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:10.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.600+0000 7f063dffb700 1 -- 172.21.15.45:0/1793950053 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0638052910 con 0x7f06480ccfc0 2023-10-06T05:27:10.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:10.890+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) v1 -- 0x7f064804bb40 con 0x7f06480ccfc0 2023-10-06T05:27:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.060+0000 7f063dffb700 1 -- 172.21.15.45:0/1793950053 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v610) v1 ==== 135+0+0 (secure 0 0 0) 0x7f06380150c0 con 0x7f06480ccfc0 2023-10-06T05:27:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f062c066f30 msgr2=0x7f062c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f062c066f30 0x7f062c0693e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f06480c6940 tx=0x7f0644009040 comp rx=0 tx=0).stop 2023-10-06T05:27:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480ccfc0 msgr2=0x7f064814e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480ccfc0 0x7f064814e5b0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f0638009fd0 tx=0x7f0638017000 comp rx=0 tx=0).stop 2023-10-06T05:27:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 shutdown_connections 2023-10-06T05:27:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f062c066f30 0x7f062c0693e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f064814eaf0 0x7f06481565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480ccfc0 0x7f064814e5b0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 --2- 172.21.15.45:0/1793950053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0648153620 0x7f0648156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 >> 172.21.15.45:0/1793950053 conn(0x7f064806ed50 msgr2=0x7f06480c5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 shutdown_connections 2023-10-06T05:27:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.063+0000 7f06500b9700 1 -- 172.21.15.45:0/1793950053 wait complete. 2023-10-06T05:27:11.080 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2023-10-06T05:27:11.080 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2023-10-06T05:27:11.081 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2023-10-06T05:27:11.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.216+0000 7fdcc9e85700 1 Processor -- start 2023-10-06T05:27:11.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.216+0000 7fdcc9e85700 1 -- start start 2023-10-06T05:27:11.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc9e85700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdcc4159200 0x7fdcc40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc9e85700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc40c7d00 con 0x7fdcc41595d0 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc9e85700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc40c86d0 con 0x7fdcc40ccdc0 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc9e85700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc40c7780 con 0x7fdcc4159200 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc2ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdcc41595d0 0x7fdcc406b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59094/0 (socket says 172.21.15.45:59094) 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc2ffd700 1 -- 172.21.15.45:0/1909293406 learned_addr learned my addr 172.21.15.45:0/1909293406 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc37fe700 1 --2- 172.21.15.45:0/1909293406 >> v2:172.21.15.144:3300/0 conn(0x7fdcc4159200 0x7fdcc40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.217+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872954371 0 0) 0x7fdcc40c7d00 con 0x7fdcc41595d0 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb0003570 con 0x7fdcc41595d0 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80800685 0 0) 0x7fdcb0003570 con 0x7fdcc41595d0 2023-10-06T05:27:11.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 >> v2:172.21.15.144:3300/0 conn(0x7fdcc4159200 msgr2=0x7fdcc40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 --2- 172.21.15.45:0/1909293406 >> v2:172.21.15.144:3300/0 conn(0x7fdcc4159200 0x7fdcc40c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 >> v1:172.21.15.144:6789/0 conn(0x7fdcc40ccdc0 legacy=0x7fdcc40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc406bb40 con 0x7fdcc41595d0 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fdcac001470 con 0x7fdcc41595d0 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc37fe700 1 --2- 172.21.15.45:0/1909293406 >> v2:172.21.15.144:3300/0 conn(0x7fdcc4159200 0x7fdcc40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 >> v1:172.21.15.45:6789/0 conn(0x7fdcc41595d0 legacy=0x7fdcc406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb0003a60 0x7fdcb0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00051c0 0x7fdcb000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40c7780 con 0x7fdcb0004810 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb000c0d0 con 0x7fdcb0003a60 2023-10-06T05:27:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb000c290 con 0x7fdcb00051c0 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc2ffd700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc37fe700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb0003a60 0x7fdcb0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc3fff700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00051c0 0x7fdcb000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc2ffd700 1 -- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00051c0 msgr2=0x7fdcb000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc2ffd700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00051c0 0x7fdcb000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc2ffd700 1 -- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb0003a60 msgr2=0x7fdcb0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc2ffd700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb0003a60 0x7fdcb0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc2ffd700 1 -- 172.21.15.45:0/1909293406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0003570 con 0x7fdcb0004810 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc3fff700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00051c0 0x7fdcb000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.218+0000 7fdcc3fff700 1 -- 172.21.15.45:0/1909293406 reap_dead start 2023-10-06T05:27:11.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc2ffd700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcb0004c80 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c3c0 tx=0x7fdcac001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1cb9bf675e242d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00cc90 con 0x7fdcb0004810 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac00cdf0 con 0x7fdcb0004810 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc27fc700 1 -- 172.21.15.45:0/1909293406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac005b50 con 0x7fdcb0004810 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc9e85700 1 -- 172.21.15.45:0/1909293406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 msgr2=0x7fdcb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcb0004c80 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c3c0 tx=0x7fdcac001e50 comp rx=0 tx=0).stop 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc9e85700 1 -- 172.21.15.45:0/1909293406 shutdown_connections 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/1909293406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcb0004c80 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc9e85700 1 -- 172.21.15.45:0/1909293406 >> 172.21.15.45:0/1909293406 conn(0x7fdcc406f090 msgr2=0x7fdcc406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc9e85700 1 -- 172.21.15.45:0/1909293406 shutdown_connections 2023-10-06T05:27:11.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.219+0000 7fdcc9e85700 1 -- 172.21.15.45:0/1909293406 wait complete. 2023-10-06T05:27:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.221+0000 7fdcc9e85700 1 Processor -- start 2023-10-06T05:27:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.221+0000 7fdcc9e85700 1 -- start start 2023-10-06T05:27:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc9e85700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcc40b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc9e85700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcc4158870 0x7fdcc416d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc9e85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc40b0580 0x7fdcc416d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc9e85700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40b2b40 con 0x7fdcb0004810 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc9e85700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc415b680 con 0x7fdcc40b0580 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc9e85700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc415c400 con 0x7fdcc4158870 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcc40b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcc40b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37518/0 (socket says 172.21.15.45:37518) 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 -- 172.21.15.45:0/4286847293 learned_addr learned my addr 172.21.15.45:0/4286847293 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc2ffd700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcc4158870 0x7fdcc416d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc3fff700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc40b0580 0x7fdcc416d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 -- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcc4158870 msgr2=0x7fdcc416d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcc4158870 0x7fdcc416d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 -- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc40b0580 msgr2=0x7fdcc416d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc40b0580 0x7fdcc416d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 -- 172.21.15.45:0/4286847293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc415b920 con 0x7fdcb0004810 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc3fff700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc40b0580 0x7fdcc416d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcc37fe700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcc40b2260 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011d70 tx=0x7fdcb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.222+0000 7fdcbb7fe700 1 -- 172.21.15.45:0/4286847293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4011e70 con 0x7fdcb0004810 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.223+0000 7fdcbb7fe700 1 -- 172.21.15.45:0/4286847293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb400fcd0 con 0x7fdcb0004810 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.223+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc415bc00 con 0x7fdcb0004810 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.223+0000 7fdcbb7fe700 1 -- 172.21.15.45:0/4286847293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4014670 con 0x7fdcb0004810 2023-10-06T05:27:11.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.223+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc40b27a0 con 0x7fdcb0004810 2023-10-06T05:27:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.224+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc415c0a0 con 0x7fdcb0004810 2023-10-06T05:27:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.224+0000 7fdcbb7fe700 1 -- 172.21.15.45:0/4286847293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcb4014aa0 con 0x7fdcb0004810 2023-10-06T05:27:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.224+0000 7fdcbb7fe700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdca4066f30 0x7fdca40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.224+0000 7fdcbb7fe700 1 -- 172.21.15.45:0/4286847293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8966+0+0 (secure 0 0 0) 0x7fdcb4018030 con 0x7fdcb0004810 2023-10-06T05:27:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.225+0000 7fdcc2ffd700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdca4066f30 0x7fdca40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.225+0000 7fdcc2ffd700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdca4066f30 0x7fdca40693e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdcac007cd0 tx=0x7fdcac015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:11.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.229+0000 7fdcbb7fe700 1 -- 172.21.15.45:0/4286847293 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcb4052410 con 0x7fdcb0004810 2023-10-06T05:27:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:11.514+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) v1 -- 0x7fdcc415c250 con 0x7fdcb0004810 2023-10-06T05:27:12.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.061+0000 7fdcbb7fe700 1 -- 172.21.15.45:0/4286847293 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v611) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdcb4022070 con 0x7fdcb0004810 2023-10-06T05:27:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdca4066f30 msgr2=0x7fdca40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdca4066f30 0x7fdca40693e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdcac007cd0 tx=0x7fdcac015000 comp rx=0 tx=0).stop 2023-10-06T05:27:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 msgr2=0x7fdcc40b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcc40b2260 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011d70 tx=0x7fdcb40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 shutdown_connections 2023-10-06T05:27:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdca4066f30 0x7fdca40693e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc40b0580 0x7fdcc416d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb0004810 0x7fdcc40b2260 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 --2- 172.21.15.45:0/4286847293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcc4158870 0x7fdcc416d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 >> 172.21.15.45:0/4286847293 conn(0x7fdcc406f090 msgr2=0x7fdcc40af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 shutdown_connections 2023-10-06T05:27:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.064+0000 7fdcc9e85700 1 -- 172.21.15.45:0/4286847293 wait complete. 2023-10-06T05:27:12.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:27:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.218+0000 7fcc1b57f700 1 Processor -- start 2023-10-06T05:27:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.218+0000 7fcc1b57f700 1 -- start start 2023-10-06T05:27:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc1b57f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc141406e0 0x7fcc1406c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc1b57f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc1406cd60 0x7fcc140acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc1b57f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc1b57f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc140c9940 con 0x7fcc1406cd60 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc1b57f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1406d230 con 0x7fcc141406e0 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc1b57f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1406d510 con 0x7fcc140ad4e0 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc18b1a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc1406cd60 0x7fcc140acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc18b1a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc1406cd60 0x7fcc140acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37532/0 (socket says 172.21.15.45:37532) 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.219+0000 7fcc18b1a700 1 -- 172.21.15.45:0/2427958786 learned_addr learned my addr 172.21.15.45:0/2427958786 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc19b1c700 1 --2- 172.21.15.45:0/2427958786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc1931b700 1 --2- 172.21.15.45:0/2427958786 >> v2:172.21.15.144:3300/0 conn(0x7fcc141406e0 0x7fcc1406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc18b1a700 1 -- 172.21.15.45:0/2427958786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 msgr2=0x7fcc140c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc18b1a700 1 --2- 172.21.15.45:0/2427958786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc18b1a700 1 -- 172.21.15.45:0/2427958786 >> v2:172.21.15.144:3300/0 conn(0x7fcc141406e0 msgr2=0x7fcc1406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc18b1a700 1 --2- 172.21.15.45:0/2427958786 >> v2:172.21.15.144:3300/0 conn(0x7fcc141406e0 0x7fcc1406c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc18b1a700 1 -- 172.21.15.45:0/2427958786 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1406d7f0 con 0x7fcc1406cd60 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc1931b700 1 --2- 172.21.15.45:0/2427958786 >> v2:172.21.15.144:3300/0 conn(0x7fcc141406e0 0x7fcc1406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc19b1c700 1 --2- 172.21.15.45:0/2427958786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc18b1a700 1 --2- 172.21.15.45:0/2427958786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1406cd60 0x7fcc140acfa0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fcc1406ad90 tx=0x7fcc04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cfbc085347c99a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc0bfff700 1 -- 172.21.15.45:0/2427958786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400b860 con 0x7fcc1406cd60 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc0bfff700 1 -- 172.21.15.45:0/2427958786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0400b9c0 con 0x7fcc1406cd60 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.220+0000 7fcc0bfff700 1 -- 172.21.15.45:0/2427958786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400bd60 con 0x7fcc1406cd60 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2427958786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1406cd60 msgr2=0x7fcc140acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2427958786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1406cd60 0x7fcc140acfa0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fcc1406ad90 tx=0x7fcc04009250 comp rx=0 tx=0).stop 2023-10-06T05:27:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2427958786 shutdown_connections 2023-10-06T05:27:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2427958786 >> v2:172.21.15.144:3300/0 conn(0x7fcc141406e0 0x7fcc1406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2427958786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2427958786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc1406cd60 0x7fcc140acfa0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2427958786 >> 172.21.15.45:0/2427958786 conn(0x7fcc140691c0 msgr2=0x7fcc140cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2427958786 shutdown_connections 2023-10-06T05:27:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.221+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2427958786 wait complete. 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.222+0000 7fcc1b57f700 1 Processor -- start 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.222+0000 7fcc1b57f700 1 -- start start 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1b57f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1406cd60 0x7fcc1413c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1b57f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1b57f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc141406e0 0x7fcc1415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1b57f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140cb190 con 0x7fcc141406e0 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1b57f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140c68c0 con 0x7fcc1406cd60 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1b57f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140c6ba0 con 0x7fcc140ad4e0 2023-10-06T05:27:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc141406e0 0x7fcc1415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc141406e0 0x7fcc1415bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37548/0 (socket says 172.21.15.45:37548) 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 -- 172.21.15.45:0/2658927918 learned_addr learned my addr 172.21.15.45:0/2658927918 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc18b1a700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1931b700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1406cd60 0x7fcc1413c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 -- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 msgr2=0x7fcc140c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 -- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1406cd60 msgr2=0x7fcc1413c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1406cd60 0x7fcc1413c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc19b1c700 1 -- 172.21.15.45:0/2658927918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040094f0 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.223+0000 7fcc1931b700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1406cd60 0x7fcc1413c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.224+0000 7fcc19b1c700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc141406e0 0x7fcc1415bb10 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00d180 tx=0x7fcc0c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.224+0000 7fcc097fa700 1 -- 172.21.15.45:0/2658927918 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00dd70 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.224+0000 7fcc097fa700 1 -- 172.21.15.45:0/2658927918 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c00f040 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.224+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc140c6e80 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.224+0000 7fcc097fa700 1 -- 172.21.15.45:0/2658927918 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c015690 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.224+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1413cea0 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.225+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc140cb300 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.225+0000 7fcc097fa700 1 -- 172.21.15.45:0/2658927918 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc0c01e3b0 con 0x7fcc141406e0 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.226+0000 7fcc097fa700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbf8066f30 0x7fcbf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.226+0000 7fcc097fa700 1 -- 172.21.15.45:0/2658927918 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8966+0+0 (secure 0 0 0) 0x7fcc0c0857c0 con 0x7fcc141406e0 2023-10-06T05:27:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.226+0000 7fcc1931b700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbf8066f30 0x7fcbf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.227+0000 7fcc1931b700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbf8066f30 0x7fcbf80693e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcc10006fd0 tx=0x7fcc10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.230+0000 7fcc097fa700 1 -- 172.21.15.45:0/2658927918 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc0c051cd0 con 0x7fcc141406e0 2023-10-06T05:27:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.432+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcc1404bb40 con 0x7fcc141406e0 2023-10-06T05:27:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.436+0000 7fcc097fa700 1 -- 172.21.15.45:0/2658927918 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcc0c01a0c0 con 0x7fcc141406e0 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbf8066f30 msgr2=0x7fcbf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbf8066f30 0x7fcbf80693e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcc10006fd0 tx=0x7fcc10008040 comp rx=0 tx=0).stop 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc141406e0 msgr2=0x7fcc1415bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc141406e0 0x7fcc1415bb10 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00d180 tx=0x7fcc0c009a50 comp rx=0 tx=0).stop 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 shutdown_connections 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbf8066f30 0x7fcbf80693e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc141406e0 0x7fcc1415bb10 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1406cd60 0x7fcc1413c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 --2- 172.21.15.45:0/2658927918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc140ad4e0 0x7fcc140c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 >> 172.21.15.45:0/2658927918 conn(0x7fcc140691c0 msgr2=0x7fcc140ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 shutdown_connections 2023-10-06T05:27:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.438+0000 7fcc1b57f700 1 -- 172.21.15.45:0/2658927918 wait complete. 2023-10-06T05:27:12.453 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:27:12.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:12.495 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:12.540 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:27:12.540 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:27:12.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:12.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.676+0000 7f4ee44f7700 1 Processor -- start 2023-10-06T05:27:12.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.676+0000 7f4ee44f7700 1 -- start start 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee44f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc1560a0 0x7f4edc15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee44f7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4edc0ccdc0 0x7f4edc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee44f7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0cd740 con 0x7f4edc155cd0 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee44f7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc0ae6b0 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee44f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0ae8d0 con 0x7f4edc1560a0 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee2a94700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4edc0ccdc0 0x7f4edc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee2293700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4edc155cd0 0x7f4edc0adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59110/0 (socket says 172.21.15.45:59110) 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee2293700 1 -- 172.21.15.45:0/2671838397 learned_addr learned my addr 172.21.15.45:0/2671838397 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.677+0000 7f4ee1a92700 1 --2- 172.21.15.45:0/2671838397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc1560a0 0x7f4edc15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee2a94700 1 -- 172.21.15.45:0/2671838397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc1560a0 msgr2=0x7f4edc15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee2a94700 1 --2- 172.21.15.45:0/2671838397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc1560a0 0x7f4edc15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee2a94700 1 -- 172.21.15.45:0/2671838397 >> v1:172.21.15.45:6789/0 conn(0x7f4edc155cd0 legacy=0x7f4edc0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee2a94700 1 -- 172.21.15.45:0/2671838397 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0aebb0 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee1291700 1 -- 172.21.15.45:0/2671838397 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524383869 0 0) 0x7f4edc0cd740 con 0x7f4edc155cd0 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee2a94700 1 --2- 172.21.15.45:0/2671838397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0cd200 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009e60 tx=0x7f4ed4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0fe937def2dcc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee1291700 1 -- 172.21.15.45:0/2671838397 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400e070 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee1291700 1 -- 172.21.15.45:0/2671838397 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400bb60 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.678+0000 7f4ee1291700 1 -- 172.21.15.45:0/2671838397 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4012430 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2671838397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 msgr2=0x7f4edc0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2671838397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0cd200 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009e60 tx=0x7f4ed4009780 comp rx=0 tx=0).stop 2023-10-06T05:27:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2671838397 shutdown_connections 2023-10-06T05:27:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2671838397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0cd200 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2671838397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc1560a0 0x7f4edc15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2671838397 >> 172.21.15.45:0/2671838397 conn(0x7f4edc06f090 msgr2=0x7f4edc06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2671838397 shutdown_connections 2023-10-06T05:27:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.679+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2671838397 wait complete. 2023-10-06T05:27:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.680+0000 7f4ee44f7700 1 Processor -- start 2023-10-06T05:27:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.680+0000 7f4ee44f7700 1 -- start start 2023-10-06T05:27:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee44f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee44f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc155cd0 0x7f4edc0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee44f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc1560a0 0x7f4edc168650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee44f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc158b40 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee44f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0a4550 con 0x7f4edc1560a0 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee44f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1680d0 con 0x7f4edc155cd0 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37566/0 (socket says 172.21.15.45:37566) 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 -- 172.21.15.45:0/2806837271 learned_addr learned my addr 172.21.15.45:0/2806837271 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee1a92700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc155cd0 0x7f4edc0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2a94700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc1560a0 0x7f4edc168650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 -- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc155cd0 msgr2=0x7f4edc0a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc155cd0 0x7f4edc0a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 -- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc1560a0 msgr2=0x7f4edc168650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc1560a0 0x7f4edc168650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 -- 172.21.15.45:0/2806837271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4009a20 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.681+0000 7f4ee2293700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0a60f0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c7e0 tx=0x7f4ed8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.682+0000 7f4ece7fc700 1 -- 172.21.15.45:0/2806837271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8005b30 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.682+0000 7f4ece7fc700 1 -- 172.21.15.45:0/2806837271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed8005c90 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.682+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc0a47f0 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.682+0000 7f4ece7fc700 1 -- 172.21.15.45:0/2806837271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8007300 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.682+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1683b0 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.683+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc0a4d10 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.683+0000 7f4ece7fc700 1 -- 172.21.15.45:0/2806837271 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ed801b400 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.683+0000 7f4ece7fc700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ec0066f30 0x7f4ec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.684+0000 7f4ece7fc700 1 -- 172.21.15.45:0/2806837271 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8966+0+0 (secure 0 0 0) 0x7f4ed8084d40 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.684+0000 7f4ee1a92700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ec0066f30 0x7f4ec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.684+0000 7f4ee1a92700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ec0066f30 0x7f4ec00693e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4edc1596b0 tx=0x7f4ed0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.688+0000 7f4ece7fc700 1 -- 172.21.15.45:0/2806837271 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4edc0a4d10 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.932+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4edc0a4d10 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.933+0000 7f4ece7fc700 1 -- 172.21.15.45:0/2806837271 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) v1 ==== 95+0+3537 (secure 0 0 0) 0x7f4edc0a4d10 con 0x7f4edc0ccdc0 2023-10-06T05:27:12.935 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:12.935 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":352,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3247121058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3247121058},{"type":"v1","addr":"172.21.15.144:6837","nonce":3247121058}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6860,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/181238359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":181238359},{"type":"v1","addr":"172.21.15.45:6833","nonce":181238359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116},{"gid":7754,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1901959320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1901959320},{"type":"v1","addr":"172.21.15.45:6835","nonce":1901959320}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":232},{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3865742643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3865742643},{"type":"v1","addr":"172.21.15.144:6835","nonce":3865742643}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":350}],"filesystems":[]} 2023-10-06T05:27:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ec0066f30 msgr2=0x7f4ec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ec0066f30 0x7f4ec00693e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4edc1596b0 tx=0x7f4ed0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 msgr2=0x7f4edc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0a60f0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c7e0 tx=0x7f4ed8007df0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 shutdown_connections 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f4ec0066f30 0x7f4ec00693e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc1560a0 0x7f4edc168650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4edc0ccdc0 0x7f4edc0a60f0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 --2- 172.21.15.45:0/2806837271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4edc155cd0 0x7f4edc0a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 >> 172.21.15.45:0/2806837271 conn(0x7f4edc06f090 msgr2=0x7f4edc0ca030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 shutdown_connections 2023-10-06T05:27:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:12.935+0000 7f4ee44f7700 1 -- 172.21.15.45:0/2806837271 wait complete. 2023-10-06T05:27:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 352 2023-10-06T05:27:12.950 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:27:12.950 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:27:12.951 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:27:12.951 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:12.951 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:27:12.951 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:27:12.951 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:27:12.951 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:12.952 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:27:12.952 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:27:12.952 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:27:12.952 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:12.952 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:27:12.952 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:27:12.952 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:27:12.953 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:12.960 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:27:12.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:27:12.961 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:27:12.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:27:12.963 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:27:12.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:27:12.964 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:27:12.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:27:13.003 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:27:13.003 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:13.004 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:27:13.004 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:13.007 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:27:13.007 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:13.008 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:27:13.008 DEBUG:teuthology.parallel:result is None 2023-10-06T05:27:13.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:27:13.090 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-06T05:27:13.091 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-06T05:27:13.092 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:27:13.093 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-10-06T05:27:13.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.149+0000 7f00c2ab4700 1 Processor -- start 2023-10-06T05:27:13.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.150+0000 7f00c2ab4700 1 -- start start 2023-10-06T05:27:13.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.150+0000 7f00c2ab4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00bc0ae9d0 0x7f00bc068860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.150+0000 7f00c2ab4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc0677c0 con 0x7f00bc068da0 2023-10-06T05:27:13.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.150+0000 7f00c2ab4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc067aa0 con 0x7f00bc15d470 2023-10-06T05:27:13.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.150+0000 7f00c2ab4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc067d80 con 0x7f00bc0ae9d0 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00c0850700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00bc0ae9d0 0x7f00bc068860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00bbfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f00bc068da0 0x7f00bc15cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59124/0 (socket says 172.21.15.45:59124) 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00c0850700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00bc0ae9d0 0x7f00bc068860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37602/0 (socket says 172.21.15.45:37602) 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00bbfff700 1 -- 172.21.15.45:0/1642665208 learned_addr learned my addr 172.21.15.45:0/1642665208 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00c0850700 1 -- 172.21.15.45:0/1642665208 >> v1:172.21.15.144:6789/0 conn(0x7f00bc15d470 legacy=0x7f00bc0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00c0850700 1 -- 172.21.15.45:0/1642665208 >> v1:172.21.15.45:6789/0 conn(0x7f00bc068da0 legacy=0x7f00bc15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00c0850700 1 -- 172.21.15.45:0/1642665208 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc068060 con 0x7f00bc0ae9d0 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00bb7fe700 1 -- 172.21.15.45:0/1642665208 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365463749 0 0) 0x7f00bc0677c0 con 0x7f00bc068da0 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00c0850700 1 --2- 172.21.15.45:0/1642665208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc068860 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0098e0 tx=0x7f00ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd600ce3e45f3559 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00bb7fe700 1 -- 172.21.15.45:0/1642665208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac009e80 con 0x7f00bc0ae9d0 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00bb7fe700 1 -- 172.21.15.45:0/1642665208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac00b860 con 0x7f00bc0ae9d0 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.151+0000 7f00bb7fe700 1 -- 172.21.15.45:0/1642665208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00bbe0 con 0x7f00bc0ae9d0 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.152+0000 7f00c2ab4700 1 -- 172.21.15.45:0/1642665208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc0ae9d0 msgr2=0x7f00bc068860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.152+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/1642665208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc068860 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0098e0 tx=0x7f00ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.152+0000 7f00c2ab4700 1 -- 172.21.15.45:0/1642665208 shutdown_connections 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.152+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/1642665208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc068860 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.152+0000 7f00c2ab4700 1 -- 172.21.15.45:0/1642665208 >> 172.21.15.45:0/1642665208 conn(0x7f00bc0cd000 msgr2=0x7f00bc0c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.152+0000 7f00c2ab4700 1 -- 172.21.15.45:0/1642665208 shutdown_connections 2023-10-06T05:27:13.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.152+0000 7f00c2ab4700 1 -- 172.21.15.45:0/1642665208 wait complete. 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.153+0000 7f00c2ab4700 1 Processor -- start 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.153+0000 7f00c2ab4700 1 -- start start 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c2ab4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00bc068da0 0x7f00bc156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c2ab4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc0af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c2ab4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc15d470 0x7f00bc0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c2ab4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0b03e0 con 0x7f00bc15d470 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c2ab4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0c3c30 con 0x7f00bc0ae9d0 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c2ab4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc156870 con 0x7f00bc068da0 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c1051700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc15d470 0x7f00bc0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c1051700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc15d470 0x7f00bc0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37606/0 (socket says 172.21.15.45:37606) 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.154+0000 7f00c1051700 1 -- 172.21.15.45:0/463382610 learned_addr learned my addr 172.21.15.45:0/463382610 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c0850700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00bc068da0 0x7f00bc156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00bbfff700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c1051700 1 -- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00bc068da0 msgr2=0x7f00bc156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c1051700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00bc068da0 0x7f00bc156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c1051700 1 -- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc0ae9d0 msgr2=0x7f00bc0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c1051700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc0af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c1051700 1 -- 172.21.15.45:0/463382610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac009420 con 0x7f00bc15d470 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00bbfff700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c1051700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc15d470 0x7f00bc0cb780 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f00b400c910 tx=0x7f00b40016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00b8ff9700 1 -- 172.21.15.45:0/463382610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4012630 con 0x7f00bc15d470 2023-10-06T05:27:13.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00b8ff9700 1 -- 172.21.15.45:0/463382610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b4012790 con 0x7f00bc15d470 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc0afe20 con 0x7f00bc15d470 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00b8ff9700 1 -- 172.21.15.45:0/463382610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4012b10 con 0x7f00bc15d470 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.155+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc0c38c0 con 0x7f00bc15d470 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.156+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc0c6500 con 0x7f00bc15d470 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.157+0000 7f00b8ff9700 1 -- 172.21.15.45:0/463382610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00b401a3a0 con 0x7f00bc15d470 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.157+0000 7f00b8ff9700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f009c066f30 0x7f009c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.157+0000 7f00b8ff9700 1 -- 172.21.15.45:0/463382610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8966+0+0 (secure 0 0 0) 0x7f00b4085460 con 0x7f00bc15d470 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.157+0000 7f00c0850700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f009c066f30 0x7f009c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:13.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.158+0000 7f00c0850700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f009c066f30 0x7f009c0693e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0093f0 tx=0x7f00ac017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:13.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.162+0000 7f00b8ff9700 1 -- 172.21.15.45:0/463382610 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00b4055460 con 0x7f00bc15d470 2023-10-06T05:27:13.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T05:27:13.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-06T05:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:13.379+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f00bc0cd400 con 0x7f00bc15d470 2023-10-06T05:27:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.458+0000 7f00b8ff9700 1 -- 172.21.15.45:0/463382610 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) v1 ==== 97+0+0 (secure 0 0 0) 0x7f00b4054ff0 con 0x7f00bc15d470 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f009c066f30 msgr2=0x7f009c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f009c066f30 0x7f009c0693e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0093f0 tx=0x7f00ac017000 comp rx=0 tx=0).stop 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc15d470 msgr2=0x7f00bc0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc15d470 0x7f00bc0cb780 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f00b400c910 tx=0x7f00b40016b0 comp rx=0 tx=0).stop 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 shutdown_connections 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f009c066f30 0x7f009c0693e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00bc15d470 0x7f00bc0cb780 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:15.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00bc068da0 0x7f00bc156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:15.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 --2- 172.21.15.45:0/463382610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc0ae9d0 0x7f00bc0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:15.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.461+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 >> 172.21.15.45:0/463382610 conn(0x7f00bc0cd000 msgr2=0x7f00bc0ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:15.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.462+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 shutdown_connections 2023-10-06T05:27:15.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:14.462+0000 7f00c2ab4700 1 -- 172.21.15.45:0/463382610 wait complete. 2023-10-06T05:27:15.310 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-10-06T05:27:15.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:27:15.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.449+0000 7f2684482700 1 Processor -- start 2023-10-06T05:27:15.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.449+0000 7f2684482700 1 -- start start 2023-10-06T05:27:15.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2684482700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c067830 0x7f267c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2684482700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f267c06add0 0x7f267c0abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2684482700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f267c0b2120 con 0x7f267c15a490 2023-10-06T05:27:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2684482700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267c0ae020 con 0x7f267c06add0 2023-10-06T05:27:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2684482700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c067ca0 con 0x7f267c067830 2023-10-06T05:27:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2681a1d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c067830 0x7f267c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2682a1f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f267c06add0 0x7f267c0abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2681a1d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c067830 0x7f267c06a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:55472/0 (socket says 172.21.15.45:55472) 2023-10-06T05:27:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.450+0000 7f2681a1d700 1 -- 172.21.15.45:0/236450749 learned_addr learned my addr 172.21.15.45:0/236450749 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f2681a1d700 1 -- 172.21.15.45:0/236450749 >> v2:172.21.15.144:3300/0 conn(0x7f267c06add0 msgr2=0x7f267c0abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f2681a1d700 1 --2- 172.21.15.45:0/236450749 >> v2:172.21.15.144:3300/0 conn(0x7f267c06add0 0x7f267c0abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f268121c700 1 -- 172.21.15.45:0/236450749 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596250343 0 0) 0x7f2678001110 con 0x7f267c15a490 2023-10-06T05:27:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f2681a1d700 1 -- 172.21.15.45:0/236450749 >> v1:172.21.15.144:6789/0 conn(0x7f267c15a490 legacy=0x7f267c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f2681a1d700 1 -- 172.21.15.45:0/236450749 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c0ab7d0 con 0x7f267c067830 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f2681a1d700 1 --2- 172.21.15.45:0/236450749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c067830 0x7f267c06a890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f26700099b0 tx=0x7f2670009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=833ad036878e6568 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f268121c700 1 -- 172.21.15.45:0/236450749 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670009ea0 con 0x7f267c067830 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f268121c700 1 -- 172.21.15.45:0/236450749 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267000b860 con 0x7f267c067830 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.451+0000 7f268121c700 1 -- 172.21.15.45:0/236450749 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267000bc00 con 0x7f267c067830 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 -- 172.21.15.45:0/236450749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c067830 msgr2=0x7f267c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 --2- 172.21.15.45:0/236450749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c067830 0x7f267c06a890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f26700099b0 tx=0x7f2670009250 comp rx=0 tx=0).stop 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 -- 172.21.15.45:0/236450749 shutdown_connections 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 --2- 172.21.15.45:0/236450749 >> v2:172.21.15.144:3300/0 conn(0x7f267c06add0 0x7f267c0abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 --2- 172.21.15.45:0/236450749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c067830 0x7f267c06a890 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 -- 172.21.15.45:0/236450749 >> 172.21.15.45:0/236450749 conn(0x7f267c0cc5a0 msgr2=0x7f267c0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 -- 172.21.15.45:0/236450749 shutdown_connections 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.452+0000 7f2684482700 1 -- 172.21.15.45:0/236450749 wait complete. 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.453+0000 7f2684482700 1 Processor -- start 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.454+0000 7f2684482700 1 -- start start 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.454+0000 7f2684482700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267c067830 0x7f267c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2684482700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c06add0 0x7f267c1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2684482700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c15a490 0x7f267c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2684482700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c1695b0 con 0x7f267c06add0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2684482700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c15f600 con 0x7f267c067830 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2684482700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c1689a0 con 0x7f267c15a490 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c06add0 0x7f267c1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c06add0 0x7f267c1539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37620/0 (socket says 172.21.15.45:37620) 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 -- 172.21.15.45:0/3491371638 learned_addr learned my addr 172.21.15.45:0/3491371638 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2682a1f700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c15a490 0x7f267c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f268221e700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267c067830 0x7f267c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 -- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c15a490 msgr2=0x7f267c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c15a490 0x7f267c0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 -- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267c067830 msgr2=0x7f267c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267c067830 0x7f267c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 -- 172.21.15.45:0/3491371638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700094f0 con 0x7f267c06add0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2681a1d700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c06add0 0x7f267c1539a0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f267c0c8f00 tx=0x7f2670017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f266e7fc700 1 -- 172.21.15.45:0/3491371638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670000d10 con 0x7f267c06add0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f266e7fc700 1 -- 172.21.15.45:0/3491371638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267000bbe0 con 0x7f267c06add0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c168c80 con 0x7f267c06add0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.455+0000 7f266e7fc700 1 -- 172.21.15.45:0/3491371638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670011690 con 0x7f267c06add0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.456+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c169240 con 0x7f267c06add0 2023-10-06T05:27:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.457+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c071490 con 0x7f267c06add0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.457+0000 7f266e7fc700 1 -- 172.21.15.45:0/3491371638 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2670011ac0 con 0x7f267c06add0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.457+0000 7f266e7fc700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2660066f30 0x7f26600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.458+0000 7f266e7fc700 1 -- 172.21.15.45:0/3491371638 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f2670085c90 con 0x7f267c06add0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.458+0000 7f268221e700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2660066f30 0x7f26600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.458+0000 7f268221e700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2660066f30 0x7f26600693e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2678003a40 tx=0x7f2678007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.464+0000 7f266e7fc700 1 -- 172.21.15.45:0/3491371638 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2670052fb0 con 0x7f267c06add0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.748+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f267c0cc0e0 con 0x7f267c06add0 2023-10-06T05:27:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.749+0000 7f266e7fc700 1 -- 172.21.15.45:0/3491371638 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2670015090 con 0x7f267c06add0 2023-10-06T05:27:16.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:16.094 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:27:16.094 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-10-06T05:27:16.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.210 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-10-06T05:27:16.210 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l+aifOBAA61RWq8uA9G2QoWNUeClzAw==", 2023-10-06T05:27:16.210 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-10-06T05:27:16.211 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l684/ChAAu6jVBjWYsEfCYiP84Z1a4w==", 2023-10-06T05:27:16.212 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lBkEtBRAAhxHreSCgLMIyEhzwzLPLPg==", 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-10-06T05:27:16.243 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lKjhaERAAUCWjOrmuPSuzy0E8nFVhiw==", 2023-10-06T05:27:16.244 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCilx9ljLuEBBAAT/f/8rYay6kRmXN7cfFvGg==", 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.248 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.250 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCllx9li3CJAxAAgnIqJkphg4+ksosgfR0/Eg==", 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.252 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQColx9lgd0fAxAAY0XERsgG69K77Clq8P306w==", 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-10-06T05:27:16.255 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrlx9ll23pARAANYPv/YPR1B2qc34ApnzKhw==", 2023-10-06T05:27:16.255 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.365 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.366 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.366 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxlx9lYkviBBAAmAB+/VXWdL4y1KyuC/MRmA==", 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.368 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-10-06T05:27:16.368 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0lx9lIOx3AxAASffKxPif5ot+Iwf0/AWToQ==", 2023-10-06T05:27:16.368 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.368 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.369 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.369 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.369 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-10-06T05:27:16.369 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC3lx9lf7EVAhAAHzTfxhwDRjERMnuiWbEhIw==", 2023-10-06T05:27:16.370 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.370 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.371 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-10-06T05:27:16.371 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC6lx9lsbTrABAAUsYGZhEaWgGuLN4c6DNLpA==", 2023-10-06T05:27:16.371 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-10-06T05:27:16.371 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9ljs9AFhAAriCOchpHjOtyOnjhVb2qAw==", 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.372 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-10-06T05:27:16.373 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9l9RlTHRAAK98CQFUvRJX30zYrxcV4lw==", 2023-10-06T05:27:16.374 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:27:16.374 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:27:16.374 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:27:16.374 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-10-06T05:27:16.392 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lW+QKGhAAILqqhEtM7K3hwCGBlMr31g==", 2023-10-06T05:27:16.394 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:27:16.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:27:16.394 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:27:16.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCglx9lpNHfIhAAWqaHVuUp/hg8OzUiBBH00w==", 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l/0VtGRAA99fxS1gieVbRro7K6P1/lw==", 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-10-06T05:27:16.396 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9l0JK/NBAALjJSQy3J1fH8NBnanE5G6A==", 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCflx9lzTgdMRAAKgZwD3oRdHmZe1ylgryQpg==", 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-10-06T05:27:16.397 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-10-06T05:27:16.398 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.398 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:16.398 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-10-06T05:27:16.398 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:27:16.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2660066f30 msgr2=0x7f26600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2660066f30 0x7f26600693e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2678003a40 tx=0x7f2678007ec0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c06add0 msgr2=0x7f267c1539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c06add0 0x7f267c1539a0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f267c0c8f00 tx=0x7f2670017000 comp rx=0 tx=0).stop 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 shutdown_connections 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2660066f30 0x7f26600693e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f267c15a490 0x7f267c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267c067830 0x7f267c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 --2- 172.21.15.45:0/3491371638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f267c06add0 0x7f267c1539a0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 >> 172.21.15.45:0/3491371638 conn(0x7f267c0cc5a0 msgr2=0x7f267c159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 shutdown_connections 2023-10-06T05:27:16.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:15.751+0000 7f2684482700 1 -- 172.21.15.45:0/3491371638 wait complete. 2023-10-06T05:27:16.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:16.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.546+0000 7faabc258700 1 Processor -- start 2023-10-06T05:27:16.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.546+0000 7faabc258700 1 -- start start 2023-10-06T05:27:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faabc258700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faab41539f0 0x7faab415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faabc258700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faab40ccfc0 0x7faab415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faabc258700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab40ce690 con 0x7faab4153620 2023-10-06T05:27:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faabc258700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab415a7a0 con 0x7faab40ccfc0 2023-10-06T05:27:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faabc258700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab40ce3f0 con 0x7faab41539f0 2023-10-06T05:27:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faaba7f5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faab40ccfc0 0x7faab415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faaba7f5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faab40ccfc0 0x7faab415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58558/0 (socket says 172.21.15.45:58558) 2023-10-06T05:27:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faaba7f5700 1 -- 172.21.15.45:0/2756123539 learned_addr learned my addr 172.21.15.45:0/2756123539 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faab97f3700 1 --2- 172.21.15.45:0/2756123539 >> v2:172.21.15.144:3300/0 conn(0x7faab41539f0 0x7faab415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faaba7f5700 1 -- 172.21.15.45:0/2756123539 >> v2:172.21.15.144:3300/0 conn(0x7faab41539f0 msgr2=0x7faab415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faaba7f5700 1 --2- 172.21.15.45:0/2756123539 >> v2:172.21.15.144:3300/0 conn(0x7faab41539f0 0x7faab415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faaba7f5700 1 -- 172.21.15.45:0/2756123539 >> v1:172.21.15.144:6789/0 conn(0x7faab4153620 legacy=0x7faab40cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.547+0000 7faaba7f5700 1 -- 172.21.15.45:0/2756123539 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab415d510 con 0x7faab40ccfc0 2023-10-06T05:27:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faaba7f5700 1 --2- 172.21.15.45:0/2756123539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab40ccfc0 0x7faab415cdc0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7faab4067840 tx=0x7faaac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5076a2ebf7ab355a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faab8ff2700 1 -- 172.21.15.45:0/2756123539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac00e070 con 0x7faab40ccfc0 2023-10-06T05:27:16.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faab8ff2700 1 -- 172.21.15.45:0/2756123539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaac00ba20 con 0x7faab40ccfc0 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faab8ff2700 1 -- 172.21.15.45:0/2756123539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0092e0 con 0x7faab40ccfc0 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faabc258700 1 -- 172.21.15.45:0/2756123539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab40ccfc0 msgr2=0x7faab415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faabc258700 1 --2- 172.21.15.45:0/2756123539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab40ccfc0 0x7faab415cdc0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7faab4067840 tx=0x7faaac009840 comp rx=0 tx=0).stop 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faabc258700 1 -- 172.21.15.45:0/2756123539 shutdown_connections 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faabc258700 1 --2- 172.21.15.45:0/2756123539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab40ccfc0 0x7faab415cdc0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faabc258700 1 --2- 172.21.15.45:0/2756123539 >> v2:172.21.15.144:3300/0 conn(0x7faab41539f0 0x7faab415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.548+0000 7faabc258700 1 -- 172.21.15.45:0/2756123539 >> 172.21.15.45:0/2756123539 conn(0x7faab406ed50 msgr2=0x7faab406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.549+0000 7faabc258700 1 -- 172.21.15.45:0/2756123539 shutdown_connections 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.549+0000 7faabc258700 1 -- 172.21.15.45:0/2756123539 wait complete. 2023-10-06T05:27:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.550+0000 7faabc258700 1 Processor -- start 2023-10-06T05:27:16.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.550+0000 7faabc258700 1 -- start start 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faabc258700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40ccfc0 0x7faab41549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faabc258700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab4153620 0x7faab414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faabc258700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40ca440 0x7faab40b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faabc258700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4154ee0 con 0x7faab4153620 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faabc258700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab41563e0 con 0x7faab40ca440 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faabc258700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab416d910 con 0x7faab40ccfc0 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab4153620 0x7faab414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab4153620 0x7faab414e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58564/0 (socket says 172.21.15.45:58564) 2023-10-06T05:27:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 -- 172.21.15.45:0/347416241 learned_addr learned my addr 172.21.15.45:0/347416241 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faaba7f5700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40ca440 0x7faab40b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab9ff4700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40ccfc0 0x7faab41549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 -- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40ccfc0 msgr2=0x7faab41549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40ccfc0 0x7faab41549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 -- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40ca440 msgr2=0x7faab40b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40ca440 0x7faab40b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 -- 172.21.15.45:0/347416241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac009ae0 con 0x7faab4153620 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faaba7f5700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40ca440 0x7faab40b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faab97f3700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab4153620 0x7faab414e8a0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e930 tx=0x7faaa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faaa9ffb700 1 -- 172.21.15.45:0/347416241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400eed0 con 0x7faab4153620 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faaa9ffb700 1 -- 172.21.15.45:0/347416241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa400c850 con 0x7faab4153620 2023-10-06T05:27:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faaa9ffb700 1 -- 172.21.15.45:0/347416241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400cc10 con 0x7faab4153620 2023-10-06T05:27:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.551+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab416de40 con 0x7faab4153620 2023-10-06T05:27:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.552+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab416db60 con 0x7faab4153620 2023-10-06T05:27:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.553+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab406f120 con 0x7faab4153620 2023-10-06T05:27:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.553+0000 7faaa9ffb700 1 -- 172.21.15.45:0/347416241 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faaa4013470 con 0x7faab4153620 2023-10-06T05:27:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.554+0000 7faaa9ffb700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa98066f30 0x7faa980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.554+0000 7faaa9ffb700 1 -- 172.21.15.45:0/347416241 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7faaa40846e0 con 0x7faab4153620 2023-10-06T05:27:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.554+0000 7faab9ff4700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa98066f30 0x7faa980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.555+0000 7faab9ff4700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa98066f30 0x7faa980693e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7faab0003860 tx=0x7faab0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.558+0000 7faaa9ffb700 1 -- 172.21.15.45:0/347416241 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faaa4051960 con 0x7faab4153620 2023-10-06T05:27:16.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.764+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faab404bb40 con 0x7faab4153620 2023-10-06T05:27:16.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.765+0000 7faaa9ffb700 1 -- 172.21.15.45:0/347416241 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+13953 (secure 0 0 0) 0x7faaa4054f80 con 0x7faab4153620 2023-10-06T05:27:16.832 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:16.832 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":139,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:27:14.455533+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa98066f30 msgr2=0x7faa980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa98066f30 0x7faa980693e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7faab0003860 tx=0x7faab0009040 comp rx=0 tx=0).stop 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab4153620 msgr2=0x7faab414e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab4153620 0x7faab414e8a0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e930 tx=0x7faaa40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 shutdown_connections 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faa98066f30 0x7faa980693e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40ca440 0x7faab40b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40ccfc0 0x7faab41549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 --2- 172.21.15.45:0/347416241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab4153620 0x7faab414e8a0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 >> 172.21.15.45:0/347416241 conn(0x7faab406ed50 msgr2=0x7faab4152720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 shutdown_connections 2023-10-06T05:27:16.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.768+0000 7faabc258700 1 -- 172.21.15.45:0/347416241 wait complete. 2023-10-06T05:27:16.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T05:27:16.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:27:16.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.972+0000 7f24b4445700 1 Processor -- start 2023-10-06T05:27:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.972+0000 7f24b4445700 1 -- start start 2023-10-06T05:27:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b4445700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f24ac0c3db0 0x7f24ac0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b4445700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b4445700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac0ce6a0 con 0x7f24ac0b0610 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b4445700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac0690f0 con 0x7f24ac0c3db0 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b4445700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0ceeb0 con 0x7f24ac069ed0 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b21e1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f24ac0b0610 0x7f24ac0697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39040/0 (socket says 172.21.15.45:39040) 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b21e1700 1 -- 172.21.15.45:0/1101944598 learned_addr learned my addr 172.21.15.45:0/1101944598 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b29e2700 1 --2- 172.21.15.45:0/1101944598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b19e0700 1 --2- 172.21.15.45:0/1101944598 >> v2:172.21.15.144:3300/0 conn(0x7f24ac0c3db0 0x7f24ac0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b11df700 1 -- 172.21.15.45:0/1101944598 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015130301 0 0) 0x7f24ac0ce6a0 con 0x7f24ac0b0610 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b11df700 1 -- 172.21.15.45:0/1101944598 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2498003570 con 0x7f24ac0b0610 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b29e2700 1 -- 172.21.15.45:0/1101944598 >> v2:172.21.15.144:3300/0 conn(0x7f24ac0c3db0 msgr2=0x7f24ac0c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b29e2700 1 --2- 172.21.15.45:0/1101944598 >> v2:172.21.15.144:3300/0 conn(0x7f24ac0c3db0 0x7f24ac0c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b11df700 1 -- 172.21.15.45:0/1101944598 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389893617 0 0) 0x7f2498003570 con 0x7f24ac0b0610 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b29e2700 1 -- 172.21.15.45:0/1101944598 >> v1:172.21.15.45:6789/0 conn(0x7f24ac0b0610 legacy=0x7f24ac0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.973+0000 7f24b29e2700 1 -- 172.21.15.45:0/1101944598 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac159880 con 0x7f24ac069ed0 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.974+0000 7f24b19e0700 1 --2- 172.21.15.45:0/1101944598 >> v2:172.21.15.144:3300/0 conn(0x7f24ac0c3db0 0x7f24ac0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.974+0000 7f24b29e2700 1 --2- 172.21.15.45:0/1101944598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0cc350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009e60 tx=0x7f24a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d148fb4ad7cf1de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.974+0000 7f24b11df700 1 -- 172.21.15.45:0/1101944598 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a400e070 con 0x7f24ac069ed0 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.974+0000 7f24b11df700 1 -- 172.21.15.45:0/1101944598 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a400b7a0 con 0x7f24ac069ed0 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.974+0000 7f24b11df700 1 -- 172.21.15.45:0/1101944598 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00015f0 con 0x7f24ac069ed0 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 -- 172.21.15.45:0/1101944598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 msgr2=0x7f24ac0cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 --2- 172.21.15.45:0/1101944598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0cc350 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009e60 tx=0x7f24a4009780 comp rx=0 tx=0).stop 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 -- 172.21.15.45:0/1101944598 shutdown_connections 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 --2- 172.21.15.45:0/1101944598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0cc350 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 --2- 172.21.15.45:0/1101944598 >> v2:172.21.15.144:3300/0 conn(0x7f24ac0c3db0 0x7f24ac0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 -- 172.21.15.45:0/1101944598 >> 172.21.15.45:0/1101944598 conn(0x7f24ac067e00 msgr2=0x7f24ac0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 -- 172.21.15.45:0/1101944598 shutdown_connections 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.975+0000 7f24b4445700 1 -- 172.21.15.45:0/1101944598 wait complete. 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.976+0000 7f24b4445700 1 Processor -- start 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.976+0000 7f24b4445700 1 -- start start 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b4445700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b4445700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24ac0b0610 0x7f24ac145840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b4445700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ac0c3db0 0x7f24ac145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b4445700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0c6e40 con 0x7f24ac0b0610 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b4445700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0c70e0 con 0x7f24ac0c3db0 2023-10-06T05:27:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b4445700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0c73c0 con 0x7f24ac069ed0 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24ac0b0610 0x7f24ac145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24ac0b0610 0x7f24ac145840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58566/0 (socket says 172.21.15.45:58566) 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 -- 172.21.15.45:0/2603648846 learned_addr learned my addr 172.21.15.45:0/2603648846 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b21e1700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b29e2700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ac0c3db0 0x7f24ac145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 -- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 msgr2=0x7f24ac0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 -- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ac0c3db0 msgr2=0x7f24ac145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ac0c3db0 0x7f24ac145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.977+0000 7f24b19e0700 1 -- 172.21.15.45:0/2603648846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8009420 con 0x7f24ac0b0610 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.978+0000 7f24b19e0700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24ac0b0610 0x7f24ac145840 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f24a800ec80 tx=0x7f24a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.978+0000 7f249e7fc700 1 -- 172.21.15.45:0/2603648846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800c850 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.978+0000 7f249e7fc700 1 -- 172.21.15.45:0/2603648846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a800c9b0 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.978+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a4009a20 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.978+0000 7f249e7fc700 1 -- 172.21.15.45:0/2603648846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800cd30 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.978+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac0c7980 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.979+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac06caf0 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.979+0000 7f249e7fc700 1 -- 172.21.15.45:0/2603648846 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24a8013680 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.980+0000 7f249e7fc700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2490066f30 0x7f24900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.980+0000 7f249e7fc700 1 -- 172.21.15.45:0/2603648846 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f24a80846c0 con 0x7f24ac0b0610 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.980+0000 7f24b21e1700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2490066f30 0x7f24900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.981+0000 7f24b21e1700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2490066f30 0x7f24900693e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f24a0002c70 tx=0x7f24a0002c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:16.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:16.984+0000 7f249e7fc700 1 -- 172.21.15.45:0/2603648846 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24a8055470 con 0x7f24ac0b0610 2023-10-06T05:27:17.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.183+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24ac0cdd00 con 0x7f24ac0b0610 2023-10-06T05:27:17.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.183+0000 7f249e7fc700 1 -- 172.21.15.45:0/2603648846 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24a8055000 con 0x7f24ac0b0610 2023-10-06T05:27:17.298 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:27:17.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.185+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2490066f30 msgr2=0x7f24900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.185+0000 7f24b4445700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2490066f30 0x7f24900693e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f24a0002c70 tx=0x7f24a0002c00 comp rx=0 tx=0).stop 2023-10-06T05:27:17.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24ac0b0610 msgr2=0x7f24ac145840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24ac0b0610 0x7f24ac145840 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f24a800ec80 tx=0x7f24a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 shutdown_connections 2023-10-06T05:27:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2490066f30 0x7f24900693e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ac0c3db0 0x7f24ac145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24ac069ed0 0x7f24ac0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 --2- 172.21.15.45:0/2603648846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24ac0b0610 0x7f24ac145840 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 >> 172.21.15.45:0/2603648846 conn(0x7f24ac067e00 msgr2=0x7f24ac06c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 shutdown_connections 2023-10-06T05:27:17.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.186+0000 7f24b4445700 1 -- 172.21.15.45:0/2603648846 wait complete. 2023-10-06T05:27:17.301 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:27:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.438+0000 7f5049522700 1 Processor -- start 2023-10-06T05:27:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.438+0000 7f5049522700 1 -- start start 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f5049522700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f5049522700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5044159520 0x7f504415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f5049522700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50440c9c90 con 0x7f504415d3d0 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f5049522700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504415d8e0 con 0x7f5044159520 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f5049522700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504415dbc0 con 0x7f5044159150 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f50427fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5044159520 0x7f504415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f50427fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5044159520 0x7f504415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58582/0 (socket says 172.21.15.45:58582) 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.439+0000 7f50427fc700 1 -- 172.21.15.45:0/4153077784 learned_addr learned my addr 172.21.15.45:0/4153077784 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:17.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f5042ffd700 1 --2- 172.21.15.45:0/4153077784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f50427fc700 1 -- 172.21.15.45:0/4153077784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 msgr2=0x7f50440ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f50427fc700 1 --2- 172.21.15.45:0/4153077784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f50427fc700 1 -- 172.21.15.45:0/4153077784 >> v1:172.21.15.144:6789/0 conn(0x7f504415d3d0 legacy=0x7f5044067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f50427fc700 1 -- 172.21.15.45:0/4153077784 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440c9920 con 0x7f5044159520 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f50427fc700 1 --2- 172.21.15.45:0/4153077784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5044159520 0x7f504415ce90 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f50340099b0 tx=0x7f5034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a5d58f8038e13b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f5041ffb700 1 -- 172.21.15.45:0/4153077784 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400b860 con 0x7f5044159520 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f5041ffb700 1 -- 172.21.15.45:0/4153077784 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503400b9c0 con 0x7f5044159520 2023-10-06T05:27:17.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.440+0000 7f5041ffb700 1 -- 172.21.15.45:0/4153077784 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400bd60 con 0x7f5044159520 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 -- 172.21.15.45:0/4153077784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5044159520 msgr2=0x7f504415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 --2- 172.21.15.45:0/4153077784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5044159520 0x7f504415ce90 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f50340099b0 tx=0x7f5034009250 comp rx=0 tx=0).stop 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 -- 172.21.15.45:0/4153077784 shutdown_connections 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 --2- 172.21.15.45:0/4153077784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 --2- 172.21.15.45:0/4153077784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5044159520 0x7f504415ce90 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 -- 172.21.15.45:0/4153077784 >> 172.21.15.45:0/4153077784 conn(0x7f50440691c0 msgr2=0x7f50440cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 -- 172.21.15.45:0/4153077784 shutdown_connections 2023-10-06T05:27:17.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.441+0000 7f5049522700 1 -- 172.21.15.45:0/4153077784 wait complete. 2023-10-06T05:27:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.442+0000 7f5049522700 1 Processor -- start 2023-10-06T05:27:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.442+0000 7f5049522700 1 -- start start 2023-10-06T05:27:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f5049522700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f5049522700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f504415d3d0 0x7f504414fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f5049522700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50440b2500 0x7f504415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f5049522700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c44c0 con 0x7f50440b2500 2023-10-06T05:27:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f5049522700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c5770 con 0x7f504415d3d0 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f5049522700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c46d0 con 0x7f5044159150 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f50437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50440b2500 0x7f504415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f50437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50440b2500 0x7f504415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58596/0 (socket says 172.21.15.45:58596) 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f50437fe700 1 -- 172.21.15.45:0/2223745172 learned_addr learned my addr 172.21.15.45:0/2223745172 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f5042ffd700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.443+0000 7f50427fc700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f504415d3d0 0x7f504414fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f50437fe700 1 -- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 msgr2=0x7f50440c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f50437fe700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f50437fe700 1 -- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f504415d3d0 msgr2=0x7f504414fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f50437fe700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f504415d3d0 0x7f504414fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f50437fe700 1 -- 172.21.15.45:0/2223745172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50340094f0 con 0x7f50440b2500 2023-10-06T05:27:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f50437fe700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50440b2500 0x7f504415c210 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c840 tx=0x7f503c001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f503affd700 1 -- 172.21.15.45:0/2223745172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c012630 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f503affd700 1 -- 172.21.15.45:0/2223745172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c0127b0 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50440c49b0 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f503affd700 1 -- 172.21.15.45:0/2223745172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c012b70 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.444+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50440c4f70 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.445+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504406e2d0 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.446+0000 7f503affd700 1 -- 172.21.15.45:0/2223745172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f503c01a3a0 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.446+0000 7f503affd700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5024066f30 0x7f50240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.446+0000 7f503affd700 1 -- 172.21.15.45:0/2223745172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f503c084590 con 0x7f50440b2500 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.446+0000 7f5042ffd700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5024066f30 0x7f50240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.447+0000 7f5042ffd700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5024066f30 0x7f50240693e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f502c006fd0 tx=0x7f502c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:17.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.450+0000 7f503affd700 1 -- 172.21.15.45:0/2223745172 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f503c0518b0 con 0x7f50440b2500 2023-10-06T05:27:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.654+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f50440ca660 con 0x7f50440b2500 2023-10-06T05:27:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.654+0000 7f503affd700 1 -- 172.21.15.45:0/2223745172 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7f503c054ed0 con 0x7f50440b2500 2023-10-06T05:27:17.656 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:27:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5024066f30 msgr2=0x7f50240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5024066f30 0x7f50240693e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f502c006fd0 tx=0x7f502c008040 comp rx=0 tx=0).stop 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50440b2500 msgr2=0x7f504415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50440b2500 0x7f504415c210 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c840 tx=0x7f503c001610 comp rx=0 tx=0).stop 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 shutdown_connections 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5024066f30 0x7f50240693e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50440b2500 0x7f504415c210 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5044159150 0x7f50440c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 --2- 172.21.15.45:0/2223745172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f504415d3d0 0x7f504414fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 >> 172.21.15.45:0/2223745172 conn(0x7f50440691c0 msgr2=0x7f5044156690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 shutdown_connections 2023-10-06T05:27:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.657+0000 7f5049522700 1 -- 172.21.15.45:0/2223745172 wait complete. 2023-10-06T05:27:17.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:27:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.891+0000 7fab5dc30700 1 Processor -- start 2023-10-06T05:27:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.891+0000 7fab5dc30700 1 -- start start 2023-10-06T05:27:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.891+0000 7fab5dc30700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab580c8ff0 0x7fab580c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.891+0000 7fab5dc30700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580caad0 0x7fab580caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab5dc30700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab58156af0 0x7fab58156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab5dc30700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5815b000 con 0x7fab58156af0 2023-10-06T05:27:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab5dc30700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5815b2e0 con 0x7fab580c8ff0 2023-10-06T05:27:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab5dc30700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5815b5c0 con 0x7fab580caad0 2023-10-06T05:27:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab58156af0 0x7fab58156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab58156af0 0x7fab58156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58598/0 (socket says 172.21.15.45:58598) 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 -- 172.21.15.45:0/2646348609 learned_addr learned my addr 172.21.15.45:0/2646348609 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab577fe700 1 --2- 172.21.15.45:0/2646348609 >> v2:172.21.15.144:3300/0 conn(0x7fab580c8ff0 0x7fab580c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab56ffd700 1 --2- 172.21.15.45:0/2646348609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580caad0 0x7fab580caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 -- 172.21.15.45:0/2646348609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580caad0 msgr2=0x7fab580caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 --2- 172.21.15.45:0/2646348609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580caad0 0x7fab580caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 -- 172.21.15.45:0/2646348609 >> v2:172.21.15.144:3300/0 conn(0x7fab580c8ff0 msgr2=0x7fab580c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 --2- 172.21.15.45:0/2646348609 >> v2:172.21.15.144:3300/0 conn(0x7fab580c8ff0 0x7fab580c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 -- 172.21.15.45:0/2646348609 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5815b8a0 con 0x7fab58156af0 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab57fff700 1 --2- 172.21.15.45:0/2646348609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab58156af0 0x7fab58156f70 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fab580cefc0 tx=0x7fab50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a38abaa943a3b55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab567fc700 1 -- 172.21.15.45:0/2646348609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000e070 con 0x7fab58156af0 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.892+0000 7fab567fc700 1 -- 172.21.15.45:0/2646348609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5000b970 con 0x7fab58156af0 2023-10-06T05:27:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab567fc700 1 -- 172.21.15.45:0/2646348609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500092e0 con 0x7fab58156af0 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 -- 172.21.15.45:0/2646348609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab58156af0 msgr2=0x7fab58156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 --2- 172.21.15.45:0/2646348609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab58156af0 0x7fab58156f70 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fab580cefc0 tx=0x7fab50009840 comp rx=0 tx=0).stop 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 -- 172.21.15.45:0/2646348609 shutdown_connections 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 --2- 172.21.15.45:0/2646348609 >> v2:172.21.15.144:3300/0 conn(0x7fab580c8ff0 0x7fab580c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 --2- 172.21.15.45:0/2646348609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab58156af0 0x7fab58156f70 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 --2- 172.21.15.45:0/2646348609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580caad0 0x7fab580caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 -- 172.21.15.45:0/2646348609 >> 172.21.15.45:0/2646348609 conn(0x7fab580cdf50 msgr2=0x7fab580ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 -- 172.21.15.45:0/2646348609 shutdown_connections 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.893+0000 7fab5dc30700 1 -- 172.21.15.45:0/2646348609 wait complete. 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.894+0000 7fab5dc30700 1 Processor -- start 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab5dc30700 1 -- start start 2023-10-06T05:27:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab5dc30700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580c8ff0 0x7fab58150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab5dc30700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab580caad0 0x7fab58141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab5dc30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab58156af0 0x7fab580b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab5dc30700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580b3ce0 con 0x7fab580caad0 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab5dc30700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab581596d0 con 0x7fab58156af0 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab5dc30700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580b1230 con 0x7fab580c8ff0 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab56ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab580caad0 0x7fab58141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab56ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab580caad0 0x7fab58141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58614/0 (socket says 172.21.15.45:58614) 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.895+0000 7fab56ffd700 1 -- 172.21.15.45:0/1687980486 learned_addr learned my addr 172.21.15.45:0/1687980486 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab57fff700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab58156af0 0x7fab580b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab577fe700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580c8ff0 0x7fab58150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab56ffd700 1 -- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580c8ff0 msgr2=0x7fab58150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab56ffd700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580c8ff0 0x7fab58150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab56ffd700 1 -- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab58156af0 msgr2=0x7fab580b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab56ffd700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab58156af0 0x7fab580b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab56ffd700 1 -- 172.21.15.45:0/1687980486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009ae0 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab56ffd700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab580caad0 0x7fab58141020 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00ec80 tx=0x7fab4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab3f7fe700 1 -- 172.21.15.45:0/1687980486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00c850 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab3f7fe700 1 -- 172.21.15.45:0/1687980486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c00c9b0 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab580b3720 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab3f7fe700 1 -- 172.21.15.45:0/1687980486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00cd50 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.896+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab58159360 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.897+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab581591c0 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.898+0000 7fab3f7fe700 1 -- 172.21.15.45:0/1687980486 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab4c013680 con 0x7fab580caad0 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.898+0000 7fab3f7fe700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.898+0000 7fab3f7fe700 1 -- 172.21.15.45:0/1687980486 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fab4c0846b0 con 0x7fab580caad0 2023-10-06T05:27:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.898+0000 7fab577fe700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.899+0000 7fab577fe700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab44066f30 0x7fab440693e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fab48009d10 tx=0x7fab480091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:17.903+0000 7fab3f7fe700 1 -- 172.21.15.45:0/1687980486 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab581591c0 con 0x7fab580caad0 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.108+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fab581591c0 con 0x7fab580caad0 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.109+0000 7fab3f7fe700 1 -- 172.21.15.45:0/1687980486 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) v1 ==== 84+0+126 (secure 0 0 0) 0x7fab581591c0 con 0x7fab580caad0 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-10-06T05:27:18.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-10-06T05:27:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-10-06T05:27:18.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:18.113 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab44066f30 msgr2=0x7fab440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab44066f30 0x7fab440693e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fab48009d10 tx=0x7fab480091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab580caad0 msgr2=0x7fab58141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab580caad0 0x7fab58141020 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00ec80 tx=0x7fab4c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 shutdown_connections 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab58156af0 0x7fab580b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab580c8ff0 0x7fab58150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 --2- 172.21.15.45:0/1687980486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab580caad0 0x7fab58141020 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 >> 172.21.15.45:0/1687980486 conn(0x7fab580cdf50 msgr2=0x7fab58151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:18.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 shutdown_connections 2023-10-06T05:27:18.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.111+0000 7fab5dc30700 1 -- 172.21.15.45:0/1687980486 wait complete. 2023-10-06T05:27:18.128 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:27:18.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:27:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.271+0000 7fb77e258700 1 Processor -- start 2023-10-06T05:27:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.271+0000 7fb77e258700 1 -- start start 2023-10-06T05:27:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb77e258700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7780cfd50 0x7fb778068c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb77e258700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb7780ce7d0 0x7fb7780691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb77e258700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7780cf8c0 con 0x7fb7781536e0 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb77e258700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780cecb0 con 0x7fb7780cfd50 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb77e258700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780cef90 con 0x7fb7780ce7d0 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb776ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7780cfd50 0x7fb778068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb776ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7780cfd50 0x7fb778068c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58628/0 (socket says 172.21.15.45:58628) 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb776ffd700 1 -- 172.21.15.45:0/71075855 learned_addr learned my addr 172.21.15.45:0/71075855 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb777fff700 1 --2- 172.21.15.45:0/71075855 >> v2:172.21.15.144:3300/0 conn(0x7fb7780ce7d0 0x7fb7780691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb776ffd700 1 -- 172.21.15.45:0/71075855 >> v2:172.21.15.144:3300/0 conn(0x7fb7780ce7d0 msgr2=0x7fb7780691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb776ffd700 1 --2- 172.21.15.45:0/71075855 >> v2:172.21.15.144:3300/0 conn(0x7fb7780ce7d0 0x7fb7780691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.272+0000 7fb776ffd700 1 -- 172.21.15.45:0/71075855 >> v1:172.21.15.144:6789/0 conn(0x7fb7781536e0 legacy=0x7fb77815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:18.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.273+0000 7fb776ffd700 1 -- 172.21.15.45:0/71075855 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780cf270 con 0x7fb7780cfd50 2023-10-06T05:27:18.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.273+0000 7fb776ffd700 1 --2- 172.21.15.45:0/71075855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780cfd50 0x7fb778068c70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fb778064d40 tx=0x7fb760009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd19977841eb417e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.273+0000 7fb7767fc700 1 -- 172.21.15.45:0/71075855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000b860 con 0x7fb7780cfd50 2023-10-06T05:27:18.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.273+0000 7fb7767fc700 1 -- 172.21.15.45:0/71075855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76000b9c0 con 0x7fb7780cfd50 2023-10-06T05:27:18.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.273+0000 7fb7767fc700 1 -- 172.21.15.45:0/71075855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000bd60 con 0x7fb7780cfd50 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.273+0000 7fb77e258700 1 -- 172.21.15.45:0/71075855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780cfd50 msgr2=0x7fb778068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.273+0000 7fb77e258700 1 --2- 172.21.15.45:0/71075855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780cfd50 0x7fb778068c70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fb778064d40 tx=0x7fb760009250 comp rx=0 tx=0).stop 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.274+0000 7fb77e258700 1 -- 172.21.15.45:0/71075855 shutdown_connections 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.274+0000 7fb77e258700 1 --2- 172.21.15.45:0/71075855 >> v2:172.21.15.144:3300/0 conn(0x7fb7780ce7d0 0x7fb7780691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.274+0000 7fb77e258700 1 --2- 172.21.15.45:0/71075855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780cfd50 0x7fb778068c70 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.274+0000 7fb77e258700 1 -- 172.21.15.45:0/71075855 >> 172.21.15.45:0/71075855 conn(0x7fb77806f820 msgr2=0x7fb7780b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.274+0000 7fb77e258700 1 -- 172.21.15.45:0/71075855 shutdown_connections 2023-10-06T05:27:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.274+0000 7fb77e258700 1 -- 172.21.15.45:0/71075855 wait complete. 2023-10-06T05:27:18.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.275+0000 7fb77e258700 1 Processor -- start 2023-10-06T05:27:18.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.275+0000 7fb77e258700 1 -- start start 2023-10-06T05:27:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb77e258700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7780ce7d0 0x7fb77814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb77e258700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7780cfd50 0x7fb7781695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb77e258700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7781536e0 0x7fb778158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb77e258700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778159280 con 0x7fb7781536e0 2023-10-06T05:27:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb77e258700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778154bb0 con 0x7fb7780cfd50 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb77e258700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb778154e90 con 0x7fb7780ce7d0 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7781536e0 0x7fb778158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7781536e0 0x7fb778158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58636/0 (socket says 172.21.15.45:58636) 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 -- 172.21.15.45:0/3354277429 learned_addr learned my addr 172.21.15.45:0/3354277429 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb776ffd700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7780cfd50 0x7fb7781695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb7777fe700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7780ce7d0 0x7fb77814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 -- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7780ce7d0 msgr2=0x7fb77814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7780ce7d0 0x7fb77814e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 -- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7780cfd50 msgr2=0x7fb7781695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7780cfd50 0x7fb7781695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.276+0000 7fb777fff700 1 -- 172.21.15.45:0/3354277429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7600094f0 con 0x7fb7781536e0 2023-10-06T05:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.277+0000 7fb777fff700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7781536e0 0x7fb778158850 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb770007f30 tx=0x7fb7700098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.277+0000 7fb76f7fe700 1 -- 172.21.15.45:0/3354277429 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77000eb90 con 0x7fb7781536e0 2023-10-06T05:27:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.277+0000 7fb76f7fe700 1 -- 172.21.15.45:0/3354277429 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77000ecf0 con 0x7fb7781536e0 2023-10-06T05:27:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.277+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb778155170 con 0x7fb7781536e0 2023-10-06T05:27:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.277+0000 7fb76f7fe700 1 -- 172.21.15.45:0/3354277429 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770014670 con 0x7fb7781536e0 2023-10-06T05:27:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.277+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77814e650 con 0x7fb7781536e0 2023-10-06T05:27:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.278+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77804c380 con 0x7fb7781536e0 2023-10-06T05:27:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.278+0000 7fb76f7fe700 1 -- 172.21.15.45:0/3354277429 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb770014a20 con 0x7fb7781536e0 2023-10-06T05:27:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.279+0000 7fb76f7fe700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb758066f30 0x7fb7580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.279+0000 7fb76f7fe700 1 -- 172.21.15.45:0/3354277429 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fb770084b10 con 0x7fb7781536e0 2023-10-06T05:27:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.279+0000 7fb7777fe700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb758066f30 0x7fb7580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.280+0000 7fb7777fe700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb758066f30 0x7fb7580693e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb7680039e0 tx=0x7fb768007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:18.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.283+0000 7fb76f7fe700 1 -- 172.21.15.45:0/3354277429 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb770051e30 con 0x7fb7781536e0 2023-10-06T05:27:18.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:18.488+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb7780cf550 con 0x7fb7781536e0 2023-10-06T05:27:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.089+0000 7fb76f7fe700 1 -- 172.21.15.45:0/3354277429 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb77000ee60 con 0x7fb7781536e0 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb758066f30 msgr2=0x7fb7580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb758066f30 0x7fb7580693e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb7680039e0 tx=0x7fb768007ec0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7781536e0 msgr2=0x7fb778158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7781536e0 0x7fb778158850 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb770007f30 tx=0x7fb7700098e0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 shutdown_connections 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb758066f30 0x7fb7580693e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7781536e0 0x7fb778158850 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7780ce7d0 0x7fb77814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 --2- 172.21.15.45:0/3354277429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7780cfd50 0x7fb7781695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 >> 172.21.15.45:0/3354277429 conn(0x7fb77806f820 msgr2=0x7fb7780d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:19.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 shutdown_connections 2023-10-06T05:27:19.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.092+0000 7fb77e258700 1 -- 172.21.15.45:0/3354277429 wait complete. 2023-10-06T05:27:19.095 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-10-06T05:27:19.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:27:19.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.257+0000 7efddd64e700 1 Processor -- start 2023-10-06T05:27:19.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.257+0000 7efddd64e700 1 -- start start 2023-10-06T05:27:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efddd64e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efdd80cd1c0 0x7efdd81594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efddd64e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd815a800 con 0x7efdd80cae00 2023-10-06T05:27:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efddd64e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd80cb280 con 0x7efdd81590f0 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efddd64e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd8159a00 con 0x7efdd80cd1c0 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efdd80cd1c0 0x7efdd81594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd67fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efdd80cae00 0x7efdd80ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39048/0 (socket says 172.21.15.45:39048) 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd77fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efdd80cd1c0 0x7efdd81594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58652/0 (socket says 172.21.15.45:58652) 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd67fc700 1 -- 172.21.15.45:0/1474620135 learned_addr learned my addr 172.21.15.45:0/1474620135 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd77fe700 1 -- 172.21.15.45:0/1474620135 >> v1:172.21.15.144:6789/0 conn(0x7efdd81590f0 legacy=0x7efdd80b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd77fe700 1 -- 172.21.15.45:0/1474620135 >> v1:172.21.15.45:6789/0 conn(0x7efdd80cae00 legacy=0x7efdd80ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd5ffb700 1 -- 172.21.15.45:0/1474620135 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633474048 0 0) 0x7efdd815a800 con 0x7efdd80cae00 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd77fe700 1 -- 172.21.15.45:0/1474620135 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd8159bf0 con 0x7efdd80cd1c0 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.258+0000 7efdd77fe700 1 --2- 172.21.15.45:0/1474620135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd81594c0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009e60 tx=0x7efdd0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb05c89ea57b9bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efdd5ffb700 1 -- 172.21.15.45:0/1474620135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000e070 con 0x7efdd80cd1c0 2023-10-06T05:27:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efdd5ffb700 1 -- 172.21.15.45:0/1474620135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd000bc30 con 0x7efdd80cd1c0 2023-10-06T05:27:19.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efdd5ffb700 1 -- 172.21.15.45:0/1474620135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0012430 con 0x7efdd80cd1c0 2023-10-06T05:27:19.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efddd64e700 1 -- 172.21.15.45:0/1474620135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 msgr2=0x7efdd81594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efddd64e700 1 --2- 172.21.15.45:0/1474620135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd81594c0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009e60 tx=0x7efdd0009780 comp rx=0 tx=0).stop 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efddd64e700 1 -- 172.21.15.45:0/1474620135 shutdown_connections 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efddd64e700 1 --2- 172.21.15.45:0/1474620135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd81594c0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efddd64e700 1 -- 172.21.15.45:0/1474620135 >> 172.21.15.45:0/1474620135 conn(0x7efdd806ed50 msgr2=0x7efdd806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efddd64e700 1 -- 172.21.15.45:0/1474620135 shutdown_connections 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.259+0000 7efddd64e700 1 -- 172.21.15.45:0/1474620135 wait complete. 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.261+0000 7efddd64e700 1 Processor -- start 2023-10-06T05:27:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.261+0000 7efddd64e700 1 -- start start 2023-10-06T05:27:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efddd64e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdd80cae00 0x7efdd81584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efddd64e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd80ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efddd64e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdd81590f0 0x7efdd815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efddd64e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd815d500 con 0x7efdd80cd1c0 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efddd64e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80cf040 con 0x7efdd81590f0 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efddd64e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80cf320 con 0x7efdd80cae00 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd80ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd80ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58656/0 (socket says 172.21.15.45:58656) 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 -- 172.21.15.45:0/3148969256 learned_addr learned my addr 172.21.15.45:0/3148969256 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd77fe700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdd81590f0 0x7efdd815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd6ffd700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdd80cae00 0x7efdd81584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 -- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdd80cae00 msgr2=0x7efdd81584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdd80cae00 0x7efdd81584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 -- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdd81590f0 msgr2=0x7efdd815f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdd81590f0 0x7efdd815f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 -- 172.21.15.45:0/3148969256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0009a20 con 0x7efdd80cd1c0 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdd67fc700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd80ce5c0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c5b0 tx=0x7efdcc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdbeffd700 1 -- 172.21.15.45:0/3148969256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc005b30 con 0x7efdd80cd1c0 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdbeffd700 1 -- 172.21.15.45:0/3148969256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc005c90 con 0x7efdd80cd1c0 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efdbeffd700 1 -- 172.21.15.45:0/3148969256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc007300 con 0x7efdd80cd1c0 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.262+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd815d190 con 0x7efdd80cd1c0 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.263+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd80c5ff0 con 0x7efdd80cd1c0 2023-10-06T05:27:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.263+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd815cff0 con 0x7efdd80cd1c0 2023-10-06T05:27:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.264+0000 7efdbeffd700 1 -- 172.21.15.45:0/3148969256 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdcc01b3b0 con 0x7efdd80cd1c0 2023-10-06T05:27:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.265+0000 7efdbeffd700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdc4066f30 0x7efdc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.265+0000 7efdbeffd700 1 -- 172.21.15.45:0/3148969256 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5896+0+0 (secure 0 0 0) 0x7efdcc083da0 con 0x7efdd80cd1c0 2023-10-06T05:27:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.265+0000 7efdd6ffd700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdc4066f30 0x7efdc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.265+0000 7efdd6ffd700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdc4066f30 0x7efdc40693e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efdc8003880 tx=0x7efdc8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:19.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.269+0000 7efdbeffd700 1 -- 172.21.15.45:0/3148969256 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdcc050fd0 con 0x7efdd80cd1c0 2023-10-06T05:27:19.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:19.475+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efdd815cff0 con 0x7efdd80cd1c0 2023-10-06T05:27:19.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T05:27:19.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-06T05:27:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.095+0000 7efdbeffd700 1 -- 172.21.15.45:0/3148969256 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) v1 ==== 170+0+0 (secure 0 0 0) 0x7efdd815cff0 con 0x7efdd80cd1c0 2023-10-06T05:27:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.100+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdc4066f30 msgr2=0x7efdc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.100+0000 7efddd64e700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdc4066f30 0x7efdc40693e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efdc8003880 tx=0x7efdc8009040 comp rx=0 tx=0).stop 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.100+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 msgr2=0x7efdd80ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.100+0000 7efddd64e700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd80ce5c0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c5b0 tx=0x7efdcc001070 comp rx=0 tx=0).stop 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.100+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 shutdown_connections 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.101+0000 7efddd64e700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efdc4066f30 0x7efdc40693e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.101+0000 7efddd64e700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdd81590f0 0x7efdd815f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.101+0000 7efddd64e700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdd80cae00 0x7efdd81584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.101+0000 7efddd64e700 1 --2- 172.21.15.45:0/3148969256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdd80cd1c0 0x7efdd80ce5c0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.101+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 >> 172.21.15.45:0/3148969256 conn(0x7efdd806ed50 msgr2=0x7efdd80b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.103+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 shutdown_connections 2023-10-06T05:27:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.103+0000 7efddd64e700 1 -- 172.21.15.45:0/3148969256 wait complete. 2023-10-06T05:27:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-10-06T05:27:20.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 Processor -- start 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 -- start start 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf0c159030 0x7faf0c068b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159400 0x7faf0c0690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faf0c0cafa0 0x7faf0c0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0c06a010 con 0x7faf0c159030 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0c0aefd0 con 0x7faf0c0cafa0 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.269+0000 7faf11497700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0cdee0 con 0x7faf0c159400 2023-10-06T05:27:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf0c159030 0x7faf0c068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf0c159030 0x7faf0c068b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58660/0 (socket says 172.21.15.45:58660) 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 -- 172.21.15.45:0/1684490284 learned_addr learned my addr 172.21.15.45:0/1684490284 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0b7fe700 1 --2- 172.21.15.45:0/1684490284 >> v2:172.21.15.144:3300/0 conn(0x7faf0c0cafa0 0x7faf0c0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0a7fc700 1 --2- 172.21.15.45:0/1684490284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159400 0x7faf0c0690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 -- 172.21.15.45:0/1684490284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159400 msgr2=0x7faf0c0690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 --2- 172.21.15.45:0/1684490284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159400 0x7faf0c0690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 -- 172.21.15.45:0/1684490284 >> v2:172.21.15.144:3300/0 conn(0x7faf0c0cafa0 msgr2=0x7faf0c0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 --2- 172.21.15.45:0/1684490284 >> v2:172.21.15.144:3300/0 conn(0x7faf0c0cafa0 0x7faf0c0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 -- 172.21.15.45:0/1684490284 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0cb960 con 0x7faf0c159030 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0a7fc700 1 --2- 172.21.15.45:0/1684490284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159400 0x7faf0c0690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf0affd700 1 --2- 172.21.15.45:0/1684490284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c159030 0x7faf0c068b90 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0c71b0 tx=0x7faefc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e015c78feb6dd5f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.270+0000 7faf09ffb700 1 -- 172.21.15.45:0/1684490284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc009e70 con 0x7faf0c159030 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf09ffb700 1 -- 172.21.15.45:0/1684490284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc00b860 con 0x7faf0c159030 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf09ffb700 1 -- 172.21.15.45:0/1684490284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00bc00 con 0x7faf0c159030 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 -- 172.21.15.45:0/1684490284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c159030 msgr2=0x7faf0c068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 --2- 172.21.15.45:0/1684490284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c159030 0x7faf0c068b90 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0c71b0 tx=0x7faefc009250 comp rx=0 tx=0).stop 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 -- 172.21.15.45:0/1684490284 shutdown_connections 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 --2- 172.21.15.45:0/1684490284 >> v2:172.21.15.144:3300/0 conn(0x7faf0c0cafa0 0x7faf0c0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 --2- 172.21.15.45:0/1684490284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c159030 0x7faf0c068b90 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 --2- 172.21.15.45:0/1684490284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159400 0x7faf0c0690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 -- 172.21.15.45:0/1684490284 >> 172.21.15.45:0/1684490284 conn(0x7faf0c06f7c0 msgr2=0x7faf0c0d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 -- 172.21.15.45:0/1684490284 shutdown_connections 2023-10-06T05:27:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.271+0000 7faf11497700 1 -- 172.21.15.45:0/1684490284 wait complete. 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.273+0000 7faf11497700 1 Processor -- start 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.273+0000 7faf11497700 1 -- start start 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.273+0000 7faf11497700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c0cafa0 0x7faf0c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.273+0000 7faf11497700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159030 0x7faf0c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.273+0000 7faf11497700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf0c159400 0x7faf0c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf11497700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1598f0 con 0x7faf0c0cafa0 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf11497700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c15a750 con 0x7faf0c159400 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf11497700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c15abe0 con 0x7faf0c159030 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c0cafa0 0x7faf0c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c0cafa0 0x7faf0c153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58668/0 (socket says 172.21.15.45:58668) 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 -- 172.21.15.45:0/979292592 learned_addr learned my addr 172.21.15.45:0/979292592 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0b7fe700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf0c159400 0x7faf0c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0a7fc700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159030 0x7faf0c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 -- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159030 msgr2=0x7faf0c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159030 0x7faf0c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 -- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf0c159400 msgr2=0x7faf0c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf0c159400 0x7faf0c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 -- 172.21.15.45:0/979292592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0094f0 con 0x7faf0c0cafa0 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf0affd700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c0cafa0 0x7faf0c153cc0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7faefc009290 tx=0x7faefc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faef2ffd700 1 -- 172.21.15.45:0/979292592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00bbe0 con 0x7faf0c0cafa0 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faef2ffd700 1 -- 172.21.15.45:0/979292592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc000c90 con 0x7faf0c0cafa0 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c159b40 con 0x7faf0c0cafa0 2023-10-06T05:27:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faef2ffd700 1 -- 172.21.15.45:0/979292592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc01a680 con 0x7faf0c0cafa0 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.274+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c15a100 con 0x7faf0c0cafa0 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.275+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c15aa30 con 0x7faf0c0cafa0 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.276+0000 7faef2ffd700 1 -- 172.21.15.45:0/979292592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faefc01aa30 con 0x7faf0c0cafa0 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.276+0000 7faef2ffd700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faef8066f30 0x7faef80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.276+0000 7faef2ffd700 1 -- 172.21.15.45:0/979292592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6278+0+0 (secure 0 0 0) 0x7faefc085d30 con 0x7faf0c0cafa0 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.277+0000 7faf0a7fc700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faef8066f30 0x7faef80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.277+0000 7faf0a7fc700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faef8066f30 0x7faef80693e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faf0c16d080 tx=0x7faf00008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.281+0000 7faef2ffd700 1 -- 172.21.15.45:0/979292592 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faefc052cc0 con 0x7faf0c0cafa0 2023-10-06T05:27:20.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:20.532+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faf0c069770 con 0x7faf0c0cafa0 2023-10-06T05:27:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.101+0000 7faef2ffd700 1 -- 172.21.15.45:0/979292592 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v354) v1 ==== 174+0+0 (secure 0 0 0) 0x7faefc0562e0 con 0x7faf0c0cafa0 2023-10-06T05:27:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faef8066f30 msgr2=0x7faef80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faef8066f30 0x7faef80693e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faf0c16d080 tx=0x7faf00008040 comp rx=0 tx=0).stop 2023-10-06T05:27:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c0cafa0 msgr2=0x7faf0c153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c0cafa0 0x7faf0c153cc0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7faefc009290 tx=0x7faefc017000 comp rx=0 tx=0).stop 2023-10-06T05:27:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 shutdown_connections 2023-10-06T05:27:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7faef8066f30 0x7faef80693e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf0c159400 0x7faf0c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c0cafa0 0x7faf0c153cc0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 --2- 172.21.15.45:0/979292592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf0c159030 0x7faf0c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.106+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 >> 172.21.15.45:0/979292592 conn(0x7faf0c06f7c0 msgr2=0x7faf0c157dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.107+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 shutdown_connections 2023-10-06T05:27:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.107+0000 7faf11497700 1 -- 172.21.15.45:0/979292592 wait complete. 2023-10-06T05:27:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 16 and data pool 17 2023-10-06T05:27:21.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.263+0000 7f6cb34b1700 1 Processor -- start 2023-10-06T05:27:21.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.263+0000 7f6cb34b1700 1 -- start start 2023-10-06T05:27:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb34b1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cac06f070 0x7f6cac06a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb34b1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac0ac5a0 con 0x7f6cac0c72c0 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb34b1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac159690 con 0x7f6cac06add0 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb34b1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac0c7700 con 0x7f6cac06f070 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb124d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cac06f070 0x7f6cac06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb1a4e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6cac0c72c0 0x7f6cac0acd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39050/0 (socket says 172.21.15.45:39050) 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb124d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cac06f070 0x7f6cac06a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58680/0 (socket says 172.21.15.45:58680) 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.264+0000 7f6cb124d700 1 -- 172.21.15.45:0/4019959495 learned_addr learned my addr 172.21.15.45:0/4019959495 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6cb124d700 1 -- 172.21.15.45:0/4019959495 >> v1:172.21.15.144:6789/0 conn(0x7f6cac06add0 legacy=0x7f6cac0c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6cb124d700 1 -- 172.21.15.45:0/4019959495 >> v1:172.21.15.45:6789/0 conn(0x7f6cac0c72c0 legacy=0x7f6cac0acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6ca3fff700 1 -- 172.21.15.45:0/4019959495 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350064827 0 0) 0x7f6cac0ac5a0 con 0x7f6cac0c72c0 2023-10-06T05:27:21.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6cb124d700 1 -- 172.21.15.45:0/4019959495 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0c78e0 con 0x7f6cac06f070 2023-10-06T05:27:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6cb124d700 1 --2- 172.21.15.45:0/4019959495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac06a890 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0099b0 tx=0x7f6c9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31cc6e6e01c26a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6ca3fff700 1 -- 172.21.15.45:0/4019959495 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00b860 con 0x7f6cac06f070 2023-10-06T05:27:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6ca3fff700 1 -- 172.21.15.45:0/4019959495 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c00b9c0 con 0x7f6cac06f070 2023-10-06T05:27:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6ca3fff700 1 -- 172.21.15.45:0/4019959495 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00bd60 con 0x7f6cac06f070 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6cb34b1700 1 -- 172.21.15.45:0/4019959495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 msgr2=0x7f6cac06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.265+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/4019959495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac06a890 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0099b0 tx=0x7f6c9c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.266+0000 7f6cb34b1700 1 -- 172.21.15.45:0/4019959495 shutdown_connections 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.266+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/4019959495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac06a890 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.266+0000 7f6cb34b1700 1 -- 172.21.15.45:0/4019959495 >> 172.21.15.45:0/4019959495 conn(0x7f6cac0cc5a0 msgr2=0x7f6cac0cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.266+0000 7f6cb34b1700 1 -- 172.21.15.45:0/4019959495 shutdown_connections 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.266+0000 7f6cb34b1700 1 -- 172.21.15.45:0/4019959495 wait complete. 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.267+0000 7f6cb34b1700 1 Processor -- start 2023-10-06T05:27:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.267+0000 7f6cb34b1700 1 -- start start 2023-10-06T05:27:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb34b1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac06add0 0x7f6cac16acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb34b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb34b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0c72c0 0x7f6cac0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb34b1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac16aa70 con 0x7f6cac06f070 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb34b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0aaa00 con 0x7f6cac0c72c0 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb34b1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac16b200 con 0x7f6cac06add0 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac1583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58694/0 (socket says 172.21.15.45:58694) 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 -- 172.21.15.45:0/255120030 learned_addr learned my addr 172.21.15.45:0/255120030 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb124d700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac06add0 0x7f6cac16acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb1a4e700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0c72c0 0x7f6cac0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 -- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac06add0 msgr2=0x7f6cac16acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac06add0 0x7f6cac16acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 -- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0c72c0 msgr2=0x7f6cac0c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0c72c0 0x7f6cac0c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb0a4c700 1 -- 172.21.15.45:0/255120030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0094f0 con 0x7f6cac06f070 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.268+0000 7f6cb1a4e700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0c72c0 0x7f6cac0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.269+0000 7f6cb0a4c700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac1583f0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800c4e0 tx=0x7f6ca800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.269+0000 7f6ca17fa700 1 -- 172.21.15.45:0/255120030 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8011590 con 0x7f6cac06f070 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.269+0000 7f6ca17fa700 1 -- 172.21.15.45:0/255120030 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8011710 con 0x7f6cac06f070 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.269+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac158930 con 0x7f6cac06f070 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.269+0000 7f6ca17fa700 1 -- 172.21.15.45:0/255120030 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8011ad0 con 0x7f6cac06f070 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.269+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac0aa3b0 con 0x7f6cac06f070 2023-10-06T05:27:21.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.270+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac0cc0e0 con 0x7f6cac06f070 2023-10-06T05:27:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.270+0000 7f6ca17fa700 1 -- 172.21.15.45:0/255120030 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ca8001020 con 0x7f6cac06f070 2023-10-06T05:27:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.271+0000 7f6ca17fa700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c90066f30 0x7f6c900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.271+0000 7f6ca17fa700 1 -- 172.21.15.45:0/255120030 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6ca8083c10 con 0x7f6cac06f070 2023-10-06T05:27:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.271+0000 7f6cb124d700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c90066f30 0x7f6c900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.271+0000 7f6cb124d700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c90066f30 0x7f6c900693e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0099b0 tx=0x7f6c9c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:21.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.275+0000 7f6ca17fa700 1 -- 172.21.15.45:0/255120030 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ca8050b40 con 0x7f6cac06f070 2023-10-06T05:27:21.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.479+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cac0aab40 con 0x7f6cac06f070 2023-10-06T05:27:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.480+0000 7f6ca17fa700 1 -- 172.21.15.45:0/255120030 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17194 (secure 0 0 0) 0x7f6ca8017070 con 0x7f6cac06f070 2023-10-06T05:27:21.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:21.483 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":142,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:27:20.533607+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:18.521771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:19.508254+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c90066f30 msgr2=0x7f6c900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c90066f30 0x7f6c900693e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0099b0 tx=0x7f6c9c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 msgr2=0x7f6cac1583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac1583f0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800c4e0 tx=0x7f6ca800c7f0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 shutdown_connections 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6c90066f30 0x7f6c900693e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0c72c0 0x7f6cac0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac06add0 0x7f6cac16acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 --2- 172.21.15.45:0/255120030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cac06f070 0x7f6cac1583f0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 >> 172.21.15.45:0/255120030 conn(0x7f6cac0cc5a0 msgr2=0x7f6cac0b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:21.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 shutdown_connections 2023-10-06T05:27:21.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.483+0000 7f6cb34b1700 1 -- 172.21.15.45:0/255120030 wait complete. 2023-10-06T05:27:21.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.628+0000 7f46f36c1700 1 Processor -- start 2023-10-06T05:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.629+0000 7f46f36c1700 1 -- start start 2023-10-06T05:27:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.629+0000 7f46f36c1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f46ec1595d0 0x7f46ec06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.629+0000 7f46f36c1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46ec06b5c0 0x7f46ec0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.629+0000 7f46f36c1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ec0c7d00 con 0x7f46ec159200 2023-10-06T05:27:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.629+0000 7f46f36c1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ec0c7780 con 0x7f46ec06b5c0 2023-10-06T05:27:21.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.629+0000 7f46f36c1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ec06bb40 con 0x7f46ec1595d0 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46ec06b5c0 0x7f46ec0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46ec06b5c0 0x7f46ec0ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58704/0 (socket says 172.21.15.45:58704) 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 -- 172.21.15.45:0/949444224 learned_addr learned my addr 172.21.15.45:0/949444224 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f0c5c700 1 --2- 172.21.15.45:0/949444224 >> v2:172.21.15.144:3300/0 conn(0x7f46ec1595d0 0x7f46ec06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 -- 172.21.15.45:0/949444224 >> v2:172.21.15.144:3300/0 conn(0x7f46ec1595d0 msgr2=0x7f46ec06b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 --2- 172.21.15.45:0/949444224 >> v2:172.21.15.144:3300/0 conn(0x7f46ec1595d0 0x7f46ec06b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 -- 172.21.15.45:0/949444224 >> v1:172.21.15.144:6789/0 conn(0x7f46ec159200 legacy=0x7f46ec0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 -- 172.21.15.45:0/949444224 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec06be20 con 0x7f46ec06b5c0 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f0c5c700 1 --2- 172.21.15.45:0/949444224 >> v2:172.21.15.144:3300/0 conn(0x7f46ec1595d0 0x7f46ec06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46f1c5e700 1 --2- 172.21.15.45:0/949444224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec06b5c0 0x7f46ec0ccdc0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f46ec06e790 tx=0x7f46e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91c68c86bd8a4b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46dffff700 1 -- 172.21.15.45:0/949444224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400e070 con 0x7f46ec06b5c0 2023-10-06T05:27:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46dffff700 1 -- 172.21.15.45:0/949444224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400ba50 con 0x7f46ec06b5c0 2023-10-06T05:27:21.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.630+0000 7f46dffff700 1 -- 172.21.15.45:0/949444224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40092e0 con 0x7f46ec06b5c0 2023-10-06T05:27:21.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 -- 172.21.15.45:0/949444224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec06b5c0 msgr2=0x7f46ec0ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 --2- 172.21.15.45:0/949444224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec06b5c0 0x7f46ec0ccdc0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f46ec06e790 tx=0x7f46e4009840 comp rx=0 tx=0).stop 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 -- 172.21.15.45:0/949444224 shutdown_connections 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 --2- 172.21.15.45:0/949444224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec06b5c0 0x7f46ec0ccdc0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 --2- 172.21.15.45:0/949444224 >> v2:172.21.15.144:3300/0 conn(0x7f46ec1595d0 0x7f46ec06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 -- 172.21.15.45:0/949444224 >> 172.21.15.45:0/949444224 conn(0x7f46ec06f090 msgr2=0x7f46ec06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 -- 172.21.15.45:0/949444224 shutdown_connections 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.631+0000 7f46f36c1700 1 -- 172.21.15.45:0/949444224 wait complete. 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.632+0000 7f46f36c1700 1 Processor -- start 2023-10-06T05:27:21.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.632+0000 7f46f36c1700 1 -- start start 2023-10-06T05:27:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f36c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec06b5c0 0x7f46ec0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f36c1700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec159200 0x7f46ec14e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f36c1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec1595d0 0x7f46ec15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f36c1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0ccc20 con 0x7f46ec1595d0 2023-10-06T05:27:21.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f36c1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0cc010 con 0x7f46ec06b5c0 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f36c1700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0cc2f0 con 0x7f46ec159200 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f1c5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec1595d0 0x7f46ec15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f1c5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec1595d0 0x7f46ec15a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58716/0 (socket says 172.21.15.45:58716) 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.633+0000 7f46f1c5e700 1 -- 172.21.15.45:0/2059145008 learned_addr learned my addr 172.21.15.45:0/2059145008 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f145d700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec06b5c0 0x7f46ec0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f0c5c700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec159200 0x7f46ec14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f1c5e700 1 -- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec159200 msgr2=0x7f46ec14e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f1c5e700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec159200 0x7f46ec14e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f1c5e700 1 -- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec06b5c0 msgr2=0x7f46ec0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f1c5e700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec06b5c0 0x7f46ec0cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f1c5e700 1 -- 172.21.15.45:0/2059145008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4009ae0 con 0x7f46ec1595d0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f1c5e700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec1595d0 0x7f46ec15a1c0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f46ec06aed0 tx=0x7f46e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46dd7fa700 1 -- 172.21.15.45:0/2059145008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4008e40 con 0x7f46ec1595d0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46dd7fa700 1 -- 172.21.15.45:0/2059145008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400be60 con 0x7f46ec1595d0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46dd7fa700 1 -- 172.21.15.45:0/2059145008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40266b0 con 0x7f46ec1595d0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec0cc5d0 con 0x7f46ec1595d0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.634+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec14e890 con 0x7f46ec1595d0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.635+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec0ccdd0 con 0x7f46ec1595d0 2023-10-06T05:27:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.636+0000 7f46dd7fa700 1 -- 172.21.15.45:0/2059145008 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46e4003040 con 0x7f46ec1595d0 2023-10-06T05:27:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.636+0000 7f46dd7fa700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46d0066f30 0x7f46d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.636+0000 7f46dd7fa700 1 -- 172.21.15.45:0/2059145008 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f46e4086780 con 0x7f46ec1595d0 2023-10-06T05:27:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.636+0000 7f46f145d700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46d0066f30 0x7f46d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:21.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.637+0000 7f46f145d700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46d0066f30 0x7f46d00693e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f46e8003860 tx=0x7f46e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.640+0000 7f46dd7fa700 1 -- 172.21.15.45:0/2059145008 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46ec0ccdd0 con 0x7f46ec1595d0 2023-10-06T05:27:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.844+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46ec0ccd60 con 0x7f46ec1595d0 2023-10-06T05:27:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.845+0000 7f46dd7fa700 1 -- 172.21.15.45:0/2059145008 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+17194 (secure 0 0 0) 0x7f46e4053750 con 0x7f46ec1595d0 2023-10-06T05:27:21.847 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:21.847 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":142,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:27:20.533607+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:18.521771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:19.508254+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.847+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46d0066f30 msgr2=0x7f46d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.847+0000 7f46f36c1700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46d0066f30 0x7f46d00693e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f46e8003860 tx=0x7f46e8009040 comp rx=0 tx=0).stop 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.847+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec1595d0 msgr2=0x7f46ec15a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.847+0000 7f46f36c1700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec1595d0 0x7f46ec15a1c0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f46ec06aed0 tx=0x7f46e4018000 comp rx=0 tx=0).stop 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 shutdown_connections 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f46d0066f30 0x7f46d00693e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46ec1595d0 0x7f46ec15a1c0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec06b5c0 0x7f46ec0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 --2- 172.21.15.45:0/2059145008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec159200 0x7f46ec14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 >> 172.21.15.45:0/2059145008 conn(0x7f46ec06f090 msgr2=0x7f46ec15b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 shutdown_connections 2023-10-06T05:27:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.848+0000 7f46f36c1700 1 -- 172.21.15.45:0/2059145008 wait complete. 2023-10-06T05:27:21.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:27:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.999+0000 7ff397f75700 1 Processor -- start 2023-10-06T05:27:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:21.999+0000 7ff397f75700 1 -- start start 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.000+0000 7ff397f75700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff39013f900 0x7ff3900cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.000+0000 7ff397f75700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3900c9630 con 0x7ff3900c9f20 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.000+0000 7ff397f75700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3900cb550 con 0x7ff3900c8de0 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.000+0000 7ff397f75700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3900cb830 con 0x7ff39013f900 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.000+0000 7ff396512700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff3900c9f20 0x7ff3900affa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39054/0 (socket says 172.21.15.45:39054) 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.000+0000 7ff396512700 1 -- 172.21.15.45:0/3977996421 learned_addr learned my addr 172.21.15.45:0/3977996421 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff395d11700 1 --2- 172.21.15.45:0/3977996421 >> v2:172.21.15.144:3300/0 conn(0x7ff39013f900 0x7ff3900cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624984253 0 0) 0x7ff3900c9630 con 0x7ff3900c9f20 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff37c003570 con 0x7ff3900c9f20 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 597523932 0 0) 0x7ff37c003570 con 0x7ff3900c9f20 2023-10-06T05:27:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 >> v2:172.21.15.144:3300/0 conn(0x7ff39013f900 msgr2=0x7ff3900cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 --2- 172.21.15.45:0/3977996421 >> v2:172.21.15.144:3300/0 conn(0x7ff39013f900 0x7ff3900cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 >> v1:172.21.15.144:6789/0 conn(0x7ff3900c8de0 legacy=0x7ff3900c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900cbb10 con 0x7ff3900c9f20 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7ff3880017f0 con 0x7ff3900c9f20 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff395d11700 1 --2- 172.21.15.45:0/3977996421 >> v2:172.21.15.144:3300/0 conn(0x7ff39013f900 0x7ff3900cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 >> v1:172.21.15.45:6789/0 conn(0x7ff3900c9f20 legacy=0x7ff3900affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff37c003a60 0x7ff37c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c004810 0x7ff37c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff37c0051c0 0x7ff37c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900cb830 con 0x7ff37c0051c0 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c00c0e0 con 0x7ff37c004810 2023-10-06T05:27:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c00c2a0 con 0x7ff37c003a60 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff396512700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff37c0051c0 0x7ff37c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff395510700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c004810 0x7ff37c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.001+0000 7ff395d11700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff37c003a60 0x7ff37c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff396512700 1 -- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff37c003a60 msgr2=0x7ff37c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff396512700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff37c003a60 0x7ff37c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff396512700 1 -- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c004810 msgr2=0x7ff37c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff396512700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c004810 0x7ff37c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff396512700 1 -- 172.21.15.45:0/3977996421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c003570 con 0x7ff37c0051c0 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff395d11700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff37c003a60 0x7ff37c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff396512700 1 -- 172.21.15.45:0/3977996421 reap_dead start 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff396512700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff37c0051c0 0x7ff37c00ba60 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7ff38800a950 tx=0x7ff3880021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58a91414264f6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880104a0 con 0x7ff37c0051c0 2023-10-06T05:27:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff388010640 con 0x7ff37c0051c0 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.002+0000 7ff394d0f700 1 -- 172.21.15.45:0/3977996421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388010a00 con 0x7ff37c0051c0 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.003+0000 7ff397f75700 1 -- 172.21.15.45:0/3977996421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff37c0051c0 msgr2=0x7ff37c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.003+0000 7ff397f75700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff37c0051c0 0x7ff37c00ba60 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7ff38800a950 tx=0x7ff3880021f0 comp rx=0 tx=0).stop 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.003+0000 7ff397f75700 1 -- 172.21.15.45:0/3977996421 shutdown_connections 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.003+0000 7ff397f75700 1 --2- 172.21.15.45:0/3977996421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff37c0051c0 0x7ff37c00ba60 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.003+0000 7ff397f75700 1 -- 172.21.15.45:0/3977996421 >> 172.21.15.45:0/3977996421 conn(0x7ff3900691c0 msgr2=0x7ff3900cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.003+0000 7ff397f75700 1 -- 172.21.15.45:0/3977996421 shutdown_connections 2023-10-06T05:27:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.003+0000 7ff397f75700 1 -- 172.21.15.45:0/3977996421 wait complete. 2023-10-06T05:27:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.004+0000 7ff397f75700 1 Processor -- start 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.004+0000 7ff397f75700 1 -- start start 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff397f75700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c0051c0 0x7ff390164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff397f75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3900c7230 0x7ff390140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff397f75700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3900b1e00 0x7ff3900b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff397f75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390164e80 con 0x7ff3900c7230 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff397f75700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900b27e0 con 0x7ff37c0051c0 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff397f75700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff390141d50 con 0x7ff3900b1e00 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3900c7230 0x7ff390140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3900c7230 0x7ff390140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58742/0 (socket says 172.21.15.45:58742) 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 -- 172.21.15.45:0/2771776233 learned_addr learned my addr 172.21.15.45:0/2771776233 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395d11700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c0051c0 0x7ff390164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff396512700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3900b1e00 0x7ff3900b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 -- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3900b1e00 msgr2=0x7ff3900b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3900b1e00 0x7ff3900b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 -- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c0051c0 msgr2=0x7ff390164940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c0051c0 0x7ff390164940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 -- 172.21.15.45:0/2771776233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390141140 con 0x7ff3900c7230 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.005+0000 7ff395510700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3900c7230 0x7ff390140910 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7ff380002ab0 tx=0x7ff380007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.006+0000 7ff385ffb700 1 -- 172.21.15.45:0/2771776233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380002e90 con 0x7ff3900c7230 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.006+0000 7ff385ffb700 1 -- 172.21.15.45:0/2771776233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000f870 con 0x7ff3900c7230 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.006+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff390141420 con 0x7ff3900c7230 2023-10-06T05:27:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.006+0000 7ff385ffb700 1 -- 172.21.15.45:0/2771776233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000fc30 con 0x7ff3900c7230 2023-10-06T05:27:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.006+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3901419e0 con 0x7ff3900c7230 2023-10-06T05:27:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.007+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3901420e0 con 0x7ff3900c7230 2023-10-06T05:27:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.007+0000 7ff385ffb700 1 -- 172.21.15.45:0/2771776233 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff38001c400 con 0x7ff3900c7230 2023-10-06T05:27:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.008+0000 7ff385ffb700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.008+0000 7ff385ffb700 1 -- 172.21.15.45:0/2771776233 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff380084c50 con 0x7ff3900c7230 2023-10-06T05:27:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.008+0000 7ff395d11700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:22.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.008+0000 7ff395d11700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0093f0 tx=0x7ff38c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.012+0000 7ff385ffb700 1 -- 172.21.15.45:0/2771776233 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3901420e0 con 0x7ff3900c7230 2023-10-06T05:27:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:22.264+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff3901420e0 con 0x7ff3900c7230 2023-10-06T05:27:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.112+0000 7ff385ffb700 1 -- 172.21.15.45:0/2771776233 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v357) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff3901420e0 con 0x7ff3900c7230 2023-10-06T05:27:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 msgr2=0x7ff3740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0093f0 tx=0x7ff38c008040 comp rx=0 tx=0).stop 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3900c7230 msgr2=0x7ff390140910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3900c7230 0x7ff390140910 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7ff380002ab0 tx=0x7ff380007d10 comp rx=0 tx=0).stop 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 shutdown_connections 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff374066f30 0x7ff3740693e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3900b1e00 0x7ff3900b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37c0051c0 0x7ff390164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 --2- 172.21.15.45:0/2771776233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3900c7230 0x7ff390140910 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 >> 172.21.15.45:0/2771776233 conn(0x7ff3900691c0 msgr2=0x7ff390067620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 shutdown_connections 2023-10-06T05:27:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.115+0000 7ff397f75700 1 -- 172.21.15.45:0/2771776233 wait complete. 2023-10-06T05:27:23.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:27:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.269+0000 7f6ee24f7700 1 Processor -- start 2023-10-06T05:27:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.269+0000 7f6ee24f7700 1 -- start start 2023-10-06T05:27:23.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.270+0000 7f6ee24f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc159520 0x7f6edc15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:23.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.270+0000 7f6ee24f7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc0c9c90 con 0x7f6edc159150 2023-10-06T05:27:23.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.270+0000 7f6ee24f7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc15d8e0 con 0x7f6edc15d3d0 2023-10-06T05:27:23.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.270+0000 7f6ee24f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc15dbc0 con 0x7f6edc159520 2023-10-06T05:27:23.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.270+0000 7f6edbfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6edc159150 0x7f6edc0ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39070/0 (socket says 172.21.15.45:39070) 2023-10-06T05:27:23.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.270+0000 7f6edbfff700 1 -- 172.21.15.45:0/1191125514 learned_addr learned my addr 172.21.15.45:0/1191125514 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:23.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.270+0000 7f6edb7fe700 1 --2- 172.21.15.45:0/1191125514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc159520 0x7f6edc15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976956002 0 0) 0x7f6edc0c9c90 con 0x7f6edc159150 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec8003570 con 0x7f6edc159150 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675899778 0 0) 0x7f6edc15d8e0 con 0x7f6edc15d3d0 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6edc0c9c90 con 0x7f6edc15d3d0 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edb7fe700 1 -- 172.21.15.45:0/1191125514 >> v1:172.21.15.144:6789/0 conn(0x7f6edc15d3d0 legacy=0x7f6edc067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edb7fe700 1 -- 172.21.15.45:0/1191125514 >> v1:172.21.15.45:6789/0 conn(0x7f6edc159150 legacy=0x7f6edc0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3144149512 0 0) 0x7f6ec8003570 con 0x7f6edc159150 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edb7fe700 1 -- 172.21.15.45:0/1191125514 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0c9920 con 0x7f6edc159520 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edb7fe700 1 --2- 172.21.15.45:0/1191125514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc159520 0x7f6edc15ce90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009910 tx=0x7f6ec40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a51638321915ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc001470 con 0x7f6edc159520 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec4011420 con 0x7f6edc159520 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.271+0000 7f6edaffd700 1 -- 172.21.15.45:0/1191125514 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40117c0 con 0x7f6edc159520 2023-10-06T05:27:23.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.272+0000 7f6ee24f7700 1 -- 172.21.15.45:0/1191125514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc159520 msgr2=0x7f6edc15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.272+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/1191125514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc159520 0x7f6edc15ce90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009910 tx=0x7f6ec40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.272+0000 7f6ee24f7700 1 -- 172.21.15.45:0/1191125514 shutdown_connections 2023-10-06T05:27:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.272+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/1191125514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc159520 0x7f6edc15ce90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.272+0000 7f6ee24f7700 1 -- 172.21.15.45:0/1191125514 >> 172.21.15.45:0/1191125514 conn(0x7f6edc0691c0 msgr2=0x7f6edc0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.272+0000 7f6ee24f7700 1 -- 172.21.15.45:0/1191125514 shutdown_connections 2023-10-06T05:27:23.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.272+0000 7f6ee24f7700 1 -- 172.21.15.45:0/1191125514 wait complete. 2023-10-06T05:27:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.273+0000 7f6ee24f7700 1 Processor -- start 2023-10-06T05:27:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.273+0000 7f6ee24f7700 1 -- start start 2023-10-06T05:27:23.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6ee24f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6edc159150 0x7f6edc0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6ee24f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6edc15d3d0 0x7f6edc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6ee24f7700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc16d4c0 0x7f6edc157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6ee24f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0c6050 con 0x7f6edc159150 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6ee24f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0c6330 con 0x7f6edc15d3d0 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6ee24f7700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0c6610 con 0x7f6edc16d4c0 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6edbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6edc159150 0x7f6edc0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6edbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6edc159150 0x7f6edc0c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58746/0 (socket says 172.21.15.45:58746) 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6edbfff700 1 -- 172.21.15.45:0/2726195961 learned_addr learned my addr 172.21.15.45:0/2726195961 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6edb7fe700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6edc15d3d0 0x7f6edc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.274+0000 7f6ee0a94700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc16d4c0 0x7f6edc157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6edbfff700 1 -- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc16d4c0 msgr2=0x7f6edc157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:23.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6edbfff700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc16d4c0 0x7f6edc157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6edbfff700 1 -- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6edc15d3d0 msgr2=0x7f6edc0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6edbfff700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6edc15d3d0 0x7f6edc0b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6edbfff700 1 -- 172.21.15.45:0/2726195961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4009420 con 0x7f6edc159150 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6edbfff700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6edc159150 0x7f6edc0c5a70 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cb70 tx=0x7f6ecc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6ed3fff700 1 -- 172.21.15.45:0/2726195961 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc007e90 con 0x7f6edc159150 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6ed3fff700 1 -- 172.21.15.45:0/2726195961 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc007360 con 0x7f6edc159150 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc16ccc0 con 0x7f6edc159150 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6ed3fff700 1 -- 172.21.15.45:0/2726195961 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004830 con 0x7f6edc159150 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.275+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc0b1f80 con 0x7f6edc159150 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.276+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6edc16d1c0 con 0x7f6edc159150 2023-10-06T05:27:23.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.277+0000 7f6ed3fff700 1 -- 172.21.15.45:0/2726195961 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ecc004ba0 con 0x7f6edc159150 2023-10-06T05:27:23.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.277+0000 7f6ed3fff700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ebc066f30 0x7f6ebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:23.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.277+0000 7f6ed3fff700 1 -- 172.21.15.45:0/2726195961 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6ecc084030 con 0x7f6edc159150 2023-10-06T05:27:23.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.277+0000 7f6edb7fe700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ebc066f30 0x7f6ebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:23.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.278+0000 7f6edb7fe700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ebc066f30 0x7f6ebc0693e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40030c0 tx=0x7f6ec4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:23.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.281+0000 7f6ed3fff700 1 -- 172.21.15.45:0/2726195961 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ecc051080 con 0x7f6edc159150 2023-10-06T05:27:23.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:23.531+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6edc0cdba0 con 0x7f6edc159150 2023-10-06T05:27:24.071 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:27:24.069+0000 7f1bc853e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.117+0000 7f6ed3fff700 1 -- 172.21.15.45:0/2726195961 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v358) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ecc0546a0 con 0x7f6edc159150 2023-10-06T05:27:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ebc066f30 msgr2=0x7f6ebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ebc066f30 0x7f6ebc0693e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40030c0 tx=0x7f6ec4016000 comp rx=0 tx=0).stop 2023-10-06T05:27:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6edc159150 msgr2=0x7f6edc0c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6edc159150 0x7f6edc0c5a70 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cb70 tx=0x7f6ecc001b50 comp rx=0 tx=0).stop 2023-10-06T05:27:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 shutdown_connections 2023-10-06T05:27:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6ebc066f30 0x7f6ebc0693e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6edc16d4c0 0x7f6edc157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6edc159150 0x7f6edc0c5a70 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 --2- 172.21.15.45:0/2726195961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6edc15d3d0 0x7f6edc0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 >> 172.21.15.45:0/2726195961 conn(0x7f6edc0691c0 msgr2=0x7f6edc0c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 shutdown_connections 2023-10-06T05:27:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.120+0000 7f6ee24f7700 1 -- 172.21.15.45:0/2726195961 wait complete. 2023-10-06T05:27:24.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.271+0000 7f1ef4af5700 1 Processor -- start 2023-10-06T05:27:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.271+0000 7f1ef4af5700 1 -- start start 2023-10-06T05:27:24.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1ef4af5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ef01595d0 0x7f1ef006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1ef4af5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef00c7d00 con 0x7f1ef0159200 2023-10-06T05:27:24.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1ef4af5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef006bb40 con 0x7f1ef006b5c0 2023-10-06T05:27:24.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1ef4af5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00c7780 con 0x7f1ef01595d0 2023-10-06T05:27:24.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1eedd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ef01595d0 0x7f1ef006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1eee59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1ef0159200 0x7f1ef00c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39074/0 (socket says 172.21.15.45:39074) 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1eedd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ef01595d0 0x7f1ef006b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58750/0 (socket says 172.21.15.45:58750) 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.272+0000 7f1eee59c700 1 -- 172.21.15.45:0/2440530880 learned_addr learned my addr 172.21.15.45:0/2440530880 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eedd9b700 1 -- 172.21.15.45:0/2440530880 >> v1:172.21.15.144:6789/0 conn(0x7f1ef006b5c0 legacy=0x7f1ef00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eedd9b700 1 -- 172.21.15.45:0/2440530880 >> v1:172.21.15.45:6789/0 conn(0x7f1ef0159200 legacy=0x7f1ef00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eedd9b700 1 -- 172.21.15.45:0/2440530880 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef006be20 con 0x7f1ef01595d0 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eed59a700 1 -- 172.21.15.45:0/2440530880 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056659620 0 0) 0x7f1ef00c7d00 con 0x7f1ef0159200 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eedd9b700 1 --2- 172.21.15.45:0/2440530880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef01595d0 0x7f1ef006b080 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80098e0 tx=0x7f1ed80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2296744afddb22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:24.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eed59a700 1 -- 172.21.15.45:0/2440530880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8009e80 con 0x7f1ef01595d0 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eed59a700 1 -- 172.21.15.45:0/2440530880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed800b860 con 0x7f1ef01595d0 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.273+0000 7f1eed59a700 1 -- 172.21.15.45:0/2440530880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed800bbe0 con 0x7f1ef01595d0 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.274+0000 7f1ef4af5700 1 -- 172.21.15.45:0/2440530880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef01595d0 msgr2=0x7f1ef006b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.274+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/2440530880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef01595d0 0x7f1ef006b080 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80098e0 tx=0x7f1ed80091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.274+0000 7f1ef4af5700 1 -- 172.21.15.45:0/2440530880 shutdown_connections 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.274+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/2440530880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef01595d0 0x7f1ef006b080 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.274+0000 7f1ef4af5700 1 -- 172.21.15.45:0/2440530880 >> 172.21.15.45:0/2440530880 conn(0x7f1ef006f090 msgr2=0x7f1ef006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.274+0000 7f1ef4af5700 1 -- 172.21.15.45:0/2440530880 shutdown_connections 2023-10-06T05:27:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.274+0000 7f1ef4af5700 1 -- 172.21.15.45:0/2440530880 wait complete. 2023-10-06T05:27:24.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.275+0000 7f1ef4af5700 1 Processor -- start 2023-10-06T05:27:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.275+0000 7f1ef4af5700 1 -- start start 2023-10-06T05:27:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1ef4af5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef006b5c0 0x7f1ef0169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1ef4af5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0159200 0x7f1ef00c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1ef4af5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ef01595d0 0x7f1ef00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1ef4af5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef014ec40 con 0x7f1ef006b5c0 2023-10-06T05:27:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1ef4af5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00cb850 con 0x7f1ef0159200 2023-10-06T05:27:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1ef4af5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c2d80 con 0x7f1ef01595d0 2023-10-06T05:27:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef006b5c0 0x7f1ef0169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef006b5c0 0x7f1ef0169510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58754/0 (socket says 172.21.15.45:58754) 2023-10-06T05:27:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 -- 172.21.15.45:0/3350260996 learned_addr learned my addr 172.21.15.45:0/3350260996 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eedd9b700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0159200 0x7f1ef00c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eeed9d700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ef01595d0 0x7f1ef00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 -- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ef01595d0 msgr2=0x7f1ef00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ef01595d0 0x7f1ef00cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 -- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0159200 msgr2=0x7f1ef00c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0159200 0x7f1ef00c1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eee59c700 1 -- 172.21.15.45:0/3350260996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed8009420 con 0x7f1ef006b5c0 2023-10-06T05:27:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.276+0000 7f1eedd9b700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0159200 0x7f1ef00c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.277+0000 7f1eee59c700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef006b5c0 0x7f1ef0169510 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000c810 tx=0x7f1ee0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.277+0000 7f1ee67fc700 1 -- 172.21.15.45:0/3350260996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0005c00 con 0x7f1ef006b5c0 2023-10-06T05:27:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.277+0000 7f1ee67fc700 1 -- 172.21.15.45:0/3350260996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee0005d60 con 0x7f1ef006b5c0 2023-10-06T05:27:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.277+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00c2170 con 0x7f1ef006b5c0 2023-10-06T05:27:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.277+0000 7f1ee67fc700 1 -- 172.21.15.45:0/3350260996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0007340 con 0x7f1ef006b5c0 2023-10-06T05:27:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.277+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef00c2730 con 0x7f1ef006b5c0 2023-10-06T05:27:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.278+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef006e2f0 con 0x7f1ef006b5c0 2023-10-06T05:27:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.278+0000 7f1ee67fc700 1 -- 172.21.15.45:0/3350260996 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ee001b400 con 0x7f1ef006b5c0 2023-10-06T05:27:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.279+0000 7f1ee67fc700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed0066f30 0x7f1ed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.279+0000 7f1ee67fc700 1 -- 172.21.15.45:0/3350260996 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1ee00842b0 con 0x7f1ef006b5c0 2023-10-06T05:27:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.279+0000 7f1eedd9b700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed0066f30 0x7f1ed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.280+0000 7f1eedd9b700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed0066f30 0x7f1ed00693e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00ae470 tx=0x7f1ed8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:24.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.283+0000 7f1ee67fc700 1 -- 172.21.15.45:0/3350260996 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ee0051300 con 0x7f1ef006b5c0 2023-10-06T05:27:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.536+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ef00b03d0 con 0x7f1ef006b5c0 2023-10-06T05:27:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.537+0000 7f1ee67fc700 1 -- 172.21.15.45:0/3350260996 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 358 v358) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f1ee0054920 con 0x7f1ef006b5c0 2023-10-06T05:27:24.539 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:24.539 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":358,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":358,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:24.111609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10842":{"gid":10842,"name":"a","rank":0,"incarnation":354,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6833/2716751458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2716751458},{"type":"v1","addr":"172.21.15.45:6833","nonce":2716751458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed0066f30 msgr2=0x7f1ed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed0066f30 0x7f1ed00693e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00ae470 tx=0x7f1ed8017000 comp rx=0 tx=0).stop 2023-10-06T05:27:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef006b5c0 msgr2=0x7f1ef0169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef006b5c0 0x7f1ef0169510 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000c810 tx=0x7f1ee0007d90 comp rx=0 tx=0).stop 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 shutdown_connections 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1ed0066f30 0x7f1ed00693e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ef01595d0 0x7f1ef00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ef006b5c0 0x7f1ef0169510 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 --2- 172.21.15.45:0/3350260996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0159200 0x7f1ef00c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 >> 172.21.15.45:0/3350260996 conn(0x7f1ef006f090 msgr2=0x7f1ef00c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 shutdown_connections 2023-10-06T05:27:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.539+0000 7f1ef4af5700 1 -- 172.21.15.45:0/3350260996 wait complete. 2023-10-06T05:27:24.544 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 358 2023-10-06T05:27:24.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.689+0000 7f9740711700 1 Processor -- start 2023-10-06T05:27:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.689+0000 7f9740711700 1 -- start start 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f9740711700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97380c93c0 0x7f9738156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f9740711700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9738157030 0x7f97380cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f9740711700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973815b000 con 0x7f97380c8ff0 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f9740711700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97380c9830 con 0x7f97380c93c0 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f9740711700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973815b2e0 con 0x7f9738157030 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f973dcac700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97380c93c0 0x7f9738156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f973e4ad700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f97380c8ff0 0x7f97380caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:34970/0 (socket says 172.21.15.45:34970) 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f973e4ad700 1 -- 172.21.15.45:0/3496240814 learned_addr learned my addr 172.21.15.45:0/3496240814 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.690+0000 7f973ecae700 1 --2- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9738157030 0x7f97380cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973dcac700 1 -- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9738157030 msgr2=0x7f97380cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973dcac700 1 --2- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9738157030 0x7f97380cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973dcac700 1 -- 172.21.15.45:0/3496240814 >> v1:172.21.15.144:6789/0 conn(0x7f97380c8ff0 legacy=0x7f97380caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973dcac700 1 -- 172.21.15.45:0/3496240814 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973815b5c0 con 0x7f97380c93c0 2023-10-06T05:27:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973d4ab700 1 -- 172.21.15.45:0/3496240814 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345975611 0 0) 0x7f973815b000 con 0x7f97380c8ff0 2023-10-06T05:27:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973ecae700 1 --2- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9738157030 0x7f97380cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973dcac700 1 --2- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97380c93c0 0x7f9738156af0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f97280098e0 tx=0x7f97280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d58281bd0020a321 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973d4ab700 1 -- 172.21.15.45:0/3496240814 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800b640 con 0x7f97380c93c0 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973d4ab700 1 -- 172.21.15.45:0/3496240814 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972800b7a0 con 0x7f97380c93c0 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.691+0000 7f973d4ab700 1 -- 172.21.15.45:0/3496240814 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800bb40 con 0x7f97380c93c0 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 -- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97380c93c0 msgr2=0x7f9738156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 --2- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97380c93c0 0x7f9738156af0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f97280098e0 tx=0x7f97280091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 -- 172.21.15.45:0/3496240814 shutdown_connections 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 --2- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9738157030 0x7f97380cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 --2- 172.21.15.45:0/3496240814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97380c93c0 0x7f9738156af0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 -- 172.21.15.45:0/3496240814 >> 172.21.15.45:0/3496240814 conn(0x7f97380cdf50 msgr2=0x7f97380ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 -- 172.21.15.45:0/3496240814 shutdown_connections 2023-10-06T05:27:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.692+0000 7f9740711700 1 -- 172.21.15.45:0/3496240814 wait complete. 2023-10-06T05:27:24.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.693+0000 7f9740711700 1 Processor -- start 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.693+0000 7f9740711700 1 -- start start 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f9740711700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97380c8ff0 0x7f973814d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f9740711700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97380c93c0 0x7f97380c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f9740711700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9738157030 0x7f97380c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f9740711700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380b2630 con 0x7f97380c8ff0 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f9740711700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c5130 con 0x7f9738157030 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f9740711700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973816aea0 con 0x7f97380c93c0 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97380c8ff0 0x7f973814d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97380c8ff0 0x7f973814d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58768/0 (socket says 172.21.15.45:58768) 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 -- 172.21.15.45:0/3006080703 learned_addr learned my addr 172.21.15.45:0/3006080703 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973ecae700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9738157030 0x7f97380c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973dcac700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97380c93c0 0x7f97380c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 -- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97380c93c0 msgr2=0x7f97380c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97380c93c0 0x7f97380c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 -- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9738157030 msgr2=0x7f97380c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9738157030 0x7f97380c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 -- 172.21.15.45:0/3006080703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730009a20 con 0x7f97380c8ff0 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973ecae700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9738157030 0x7f97380c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f973e4ad700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97380c8ff0 0x7f973814d030 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f973400c7e0 tx=0x7f9734007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.694+0000 7f972e7fc700 1 -- 172.21.15.45:0/3006080703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734005b30 con 0x7f97380c8ff0 2023-10-06T05:27:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.695+0000 7f972e7fc700 1 -- 172.21.15.45:0/3006080703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734005c90 con 0x7f97380c8ff0 2023-10-06T05:27:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.695+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728009420 con 0x7f97380c8ff0 2023-10-06T05:27:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.695+0000 7f972e7fc700 1 -- 172.21.15.45:0/3006080703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340072e0 con 0x7f97380c8ff0 2023-10-06T05:27:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.695+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973816b460 con 0x7f97380c8ff0 2023-10-06T05:27:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.695+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97380c5410 con 0x7f97380c8ff0 2023-10-06T05:27:24.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.696+0000 7f972e7fc700 1 -- 172.21.15.45:0/3006080703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97340076d0 con 0x7f97380c8ff0 2023-10-06T05:27:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.697+0000 7f972e7fc700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f971c066f30 0x7f971c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.697+0000 7f972e7fc700 1 -- 172.21.15.45:0/3006080703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9734083cb0 con 0x7f97380c8ff0 2023-10-06T05:27:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.697+0000 7f973dcac700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f971c066f30 0x7f971c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.697+0000 7f973dcac700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f971c066f30 0x7f971c0693e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f97280093f0 tx=0x7f9728017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.701+0000 7f972e7fc700 1 -- 172.21.15.45:0/3006080703 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9734050d30 con 0x7f97380c8ff0 2023-10-06T05:27:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.908+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f973804bb40 con 0x7f97380c8ff0 2023-10-06T05:27:24.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.909+0000 7f972e7fc700 1 -- 172.21.15.45:0/3006080703 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+17181 (secure 0 0 0) 0x7f9734017090 con 0x7f97380c8ff0 2023-10-06T05:27:24.911 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:24.911 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":143,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:27:21.917914+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:18.521771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:19.508254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f971c066f30 msgr2=0x7f971c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f971c066f30 0x7f971c0693e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f97280093f0 tx=0x7f9728017000 comp rx=0 tx=0).stop 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97380c8ff0 msgr2=0x7f973814d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97380c8ff0 0x7f973814d030 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f973400c7e0 tx=0x7f9734007df0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 shutdown_connections 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f971c066f30 0x7f971c0693e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9738157030 0x7f97380c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97380c8ff0 0x7f973814d030 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 --2- 172.21.15.45:0/3006080703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97380c93c0 0x7f97380c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 >> 172.21.15.45:0/3006080703 conn(0x7f97380cdf50 msgr2=0x7f97380afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 shutdown_connections 2023-10-06T05:27:24.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:24.912+0000 7f9740711700 1 -- 172.21.15.45:0/3006080703 wait complete. 2023-10-06T05:27:24.927 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:27:24.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:27:25.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.060+0000 7fc124529700 1 Processor -- start 2023-10-06T05:27:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.060+0000 7fc124529700 1 -- start start 2023-10-06T05:27:26.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc124529700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc11c1595d0 0x7fc11c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc124529700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0c7d00 con 0x7fc11c159200 2023-10-06T05:27:26.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc124529700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c06bb40 con 0x7fc11c06b5c0 2023-10-06T05:27:26.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc124529700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0c7780 con 0x7fc11c1595d0 2023-10-06T05:27:26.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc121ac4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc11c1595d0 0x7fc11c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc1222c5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc11c159200 0x7fc11c0c8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39088/0 (socket says 172.21.15.45:39088) 2023-10-06T05:27:26.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc1222c5700 1 -- 172.21.15.45:0/766915022 learned_addr learned my addr 172.21.15.45:0/766915022 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:26.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc121ac4700 1 -- 172.21.15.45:0/766915022 >> v1:172.21.15.144:6789/0 conn(0x7fc11c06b5c0 legacy=0x7fc11c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:26.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc121ac4700 1 -- 172.21.15.45:0/766915022 >> v1:172.21.15.45:6789/0 conn(0x7fc11c159200 legacy=0x7fc11c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc121ac4700 1 -- 172.21.15.45:0/766915022 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c06be20 con 0x7fc11c1595d0 2023-10-06T05:27:26.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.061+0000 7fc121ac4700 1 --2- 172.21.15.45:0/766915022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c1595d0 0x7fc11c06b080 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098e0 tx=0x7fc10c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10893bb20137aa11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc1212c3700 1 -- 172.21.15.45:0/766915022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c009e80 con 0x7fc11c1595d0 2023-10-06T05:27:26.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc1212c3700 1 -- 172.21.15.45:0/766915022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c00b820 con 0x7fc11c1595d0 2023-10-06T05:27:26.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc1212c3700 1 -- 172.21.15.45:0/766915022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00bba0 con 0x7fc11c1595d0 2023-10-06T05:27:26.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc124529700 1 -- 172.21.15.45:0/766915022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c1595d0 msgr2=0x7fc11c06b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc124529700 1 --2- 172.21.15.45:0/766915022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c1595d0 0x7fc11c06b080 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098e0 tx=0x7fc10c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc124529700 1 -- 172.21.15.45:0/766915022 shutdown_connections 2023-10-06T05:27:26.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc124529700 1 --2- 172.21.15.45:0/766915022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c1595d0 0x7fc11c06b080 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc124529700 1 -- 172.21.15.45:0/766915022 >> 172.21.15.45:0/766915022 conn(0x7fc11c06f090 msgr2=0x7fc11c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc124529700 1 -- 172.21.15.45:0/766915022 shutdown_connections 2023-10-06T05:27:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.062+0000 7fc124529700 1 -- 172.21.15.45:0/766915022 wait complete. 2023-10-06T05:27:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.063+0000 7fc124529700 1 Processor -- start 2023-10-06T05:27:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc124529700 1 -- start start 2023-10-06T05:27:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc124529700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c06b5c0 0x7fc11c1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc124529700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c159200 0x7fc11c16dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc124529700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc11c1595d0 0x7fc11c0c0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc124529700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0ca960 con 0x7fc11c159200 2023-10-06T05:27:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc124529700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0c0dd0 con 0x7fc11c06b5c0 2023-10-06T05:27:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc124529700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0cabb0 con 0x7fc11c1595d0 2023-10-06T05:27:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc121ac4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c159200 0x7fc11c16dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc121ac4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c159200 0x7fc11c16dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58786/0 (socket says 172.21.15.45:58786) 2023-10-06T05:27:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.064+0000 7fc121ac4700 1 -- 172.21.15.45:0/3829629823 learned_addr learned my addr 172.21.15.45:0/3829629823 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc1222c5700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c06b5c0 0x7fc11c1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc122ac6700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc11c1595d0 0x7fc11c0c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc121ac4700 1 -- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc11c1595d0 msgr2=0x7fc11c0c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc121ac4700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc11c1595d0 0x7fc11c0c0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc121ac4700 1 -- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c06b5c0 msgr2=0x7fc11c1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc121ac4700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c06b5c0 0x7fc11c1583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc121ac4700 1 -- 172.21.15.45:0/3829629823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009420 con 0x7fc11c159200 2023-10-06T05:27:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc122ac6700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc11c1595d0 0x7fc11c0c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc121ac4700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c159200 0x7fc11c16dc70 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fc11c06aed0 tx=0x7fc10c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc1127fc700 1 -- 172.21.15.45:0/3829629823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00ba60 con 0x7fc11c159200 2023-10-06T05:27:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc1127fc700 1 -- 172.21.15.45:0/3829629823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c020070 con 0x7fc11c159200 2023-10-06T05:27:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c0cae90 con 0x7fc11c159200 2023-10-06T05:27:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc1127fc700 1 -- 172.21.15.45:0/3829629823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00bbe0 con 0x7fc11c159200 2023-10-06T05:27:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.065+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c0cb450 con 0x7fc11c159200 2023-10-06T05:27:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.066+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0cb640 con 0x7fc11c159200 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.067+0000 7fc1127fc700 1 -- 172.21.15.45:0/3829629823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc10c022030 con 0x7fc11c159200 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.067+0000 7fc1127fc700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.067+0000 7fc1127fc700 1 -- 172.21.15.45:0/3829629823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc10c086730 con 0x7fc11c159200 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.067+0000 7fc1222c5700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.068+0000 7fc1222c5700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc100066f30 0x7fc1000693e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc118007130 tx=0x7fc118007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.072+0000 7fc1127fc700 1 -- 172.21.15.45:0/3829629823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc10c053660 con 0x7fc11c159200 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.272+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc11c0cb640 con 0x7fc100066f30 2023-10-06T05:27:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.282+0000 7fc1127fc700 1 -- 172.21.15.45:0/3829629823 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312267 (secure 0 0 0) 0x7fc11c0cb640 con 0x7fc100066f30 2023-10-06T05:27:26.133 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc100066f30 msgr2=0x7fc1000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc100066f30 0x7fc1000693e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc118007130 tx=0x7fc118007eb0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c159200 msgr2=0x7fc11c16dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c159200 0x7fc11c16dc70 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fc11c06aed0 tx=0x7fc10c016000 comp rx=0 tx=0).stop 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 shutdown_connections 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc11c1595d0 0x7fc11c0c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c06b5c0 0x7fc11c1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 --2- 172.21.15.45:0/3829629823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc11c159200 0x7fc11c16dc70 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 >> 172.21.15.45:0/3829629823 conn(0x7fc11c06f090 msgr2=0x7fc11c0acf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 shutdown_connections 2023-10-06T05:27:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:25.286+0000 7fc124529700 1 -- 172.21.15.45:0/3829629823 wait complete. 2023-10-06T05:27:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:27:26.137 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2023-10-06T05:27:24.633768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2455528,"kb_used_data":15784,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752519192,"statfs":{"total":773094113280,"available":770579652608,"internally_reserved":0,"allocated":16162816,"data_stored":8970246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.540709"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942026+0000","last_change":"2023-10-06T05:27:20.942026+0000","last_active":"2023-10-06T05:27:20.942026+0000","last_peered":"2023-10-06T05:27:20.942026+0000","last_clean":"2023-10-06T05:27:20.942026+0000","last_became_active":"2023-10-06T05:27:20.940430+0000","last_became_peered":"2023-10-06T05:27:20.940430+0000","last_unstale":"2023-10-06T05:27:20.942026+0000","last_undegraded":"2023-10-06T05:27:20.942026+0000","last_fullsized":"2023-10-06T05:27:20.942026+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:59.901425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940463+0000","last_change":"2023-10-06T05:27:20.935099+0000","last_active":"2023-10-06T05:27:20.940463+0000","last_peered":"2023-10-06T05:27:20.940463+0000","last_clean":"2023-10-06T05:27:20.940463+0000","last_became_active":"2023-10-06T05:27:20.934804+0000","last_became_peered":"2023-10-06T05:27:20.934804+0000","last_unstale":"2023-10-06T05:27:20.940463+0000","last_undegraded":"2023-10-06T05:27:20.940463+0000","last_fullsized":"2023-10-06T05:27:20.940463+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:35.285755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.930390+0000","last_change":"2023-10-06T05:27:20.930390+0000","last_active":"2023-10-06T05:27:20.930390+0000","last_peered":"2023-10-06T05:27:20.930390+0000","last_clean":"2023-10-06T05:27:20.930390+0000","last_became_active":"2023-10-06T05:27:20.930074+0000","last_became_peered":"2023-10-06T05:27:20.930074+0000","last_unstale":"2023-10-06T05:27:20.930390+0000","last_undegraded":"2023-10-06T05:27:20.930390+0000","last_fullsized":"2023-10-06T05:27:20.930390+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:59.357444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940043+0000","last_change":"2023-10-06T05:27:20.940043+0000","last_active":"2023-10-06T05:27:20.940043+0000","last_peered":"2023-10-06T05:27:20.940043+0000","last_clean":"2023-10-06T05:27:20.940043+0000","last_became_active":"2023-10-06T05:27:20.939323+0000","last_became_peered":"2023-10-06T05:27:20.939323+0000","last_unstale":"2023-10-06T05:27:20.940043+0000","last_undegraded":"2023-10-06T05:27:20.940043+0000","last_fullsized":"2023-10-06T05:27:20.940043+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:22.797307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.937459+0000","last_change":"2023-10-06T05:27:20.937459+0000","last_active":"2023-10-06T05:27:20.937459+0000","last_peered":"2023-10-06T05:27:20.937459+0000","last_clean":"2023-10-06T05:27:20.937459+0000","last_became_active":"2023-10-06T05:27:20.937158+0000","last_became_peered":"2023-10-06T05:27:20.937158+0000","last_unstale":"2023-10-06T05:27:20.937459+0000","last_undegraded":"2023-10-06T05:27:20.937459+0000","last_fullsized":"2023-10-06T05:27:20.937459+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:00.094940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943368+0000","last_change":"2023-10-06T05:27:20.938405+0000","last_active":"2023-10-06T05:27:20.943368+0000","last_peered":"2023-10-06T05:27:20.943368+0000","last_clean":"2023-10-06T05:27:20.943368+0000","last_became_active":"2023-10-06T05:27:20.938135+0000","last_became_peered":"2023-10-06T05:27:20.938135+0000","last_unstale":"2023-10-06T05:27:20.943368+0000","last_undegraded":"2023-10-06T05:27:20.943368+0000","last_fullsized":"2023-10-06T05:27:20.943368+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:00.601747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.934017+0000","last_change":"2023-10-06T05:27:20.934017+0000","last_active":"2023-10-06T05:27:20.934017+0000","last_peered":"2023-10-06T05:27:20.934017+0000","last_clean":"2023-10-06T05:27:20.934017+0000","last_became_active":"2023-10-06T05:27:20.933779+0000","last_became_peered":"2023-10-06T05:27:20.933779+0000","last_unstale":"2023-10-06T05:27:20.934017+0000","last_undegraded":"2023-10-06T05:27:20.934017+0000","last_fullsized":"2023-10-06T05:27:20.934017+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:09.024160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.937557+0000","last_change":"2023-10-06T05:27:20.937557+0000","last_active":"2023-10-06T05:27:20.937557+0000","last_peered":"2023-10-06T05:27:20.937557+0000","last_clean":"2023-10-06T05:27:20.937557+0000","last_became_active":"2023-10-06T05:27:20.935748+0000","last_became_peered":"2023-10-06T05:27:20.935748+0000","last_unstale":"2023-10-06T05:27:20.937557+0000","last_undegraded":"2023-10-06T05:27:20.937557+0000","last_fullsized":"2023-10-06T05:27:20.937557+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:29.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944198+0000","last_change":"2023-10-06T05:27:20.944198+0000","last_active":"2023-10-06T05:27:20.944198+0000","last_peered":"2023-10-06T05:27:20.944198+0000","last_clean":"2023-10-06T05:27:20.944198+0000","last_became_active":"2023-10-06T05:27:20.943579+0000","last_became_peered":"2023-10-06T05:27:20.943579+0000","last_unstale":"2023-10-06T05:27:20.944198+0000","last_undegraded":"2023-10-06T05:27:20.944198+0000","last_fullsized":"2023-10-06T05:27:20.944198+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:38.975075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943260+0000","last_change":"2023-10-06T05:27:20.943260+0000","last_active":"2023-10-06T05:27:20.943260+0000","last_peered":"2023-10-06T05:27:20.943260+0000","last_clean":"2023-10-06T05:27:20.943260+0000","last_became_active":"2023-10-06T05:27:20.942925+0000","last_became_peered":"2023-10-06T05:27:20.942925+0000","last_unstale":"2023-10-06T05:27:20.943260+0000","last_undegraded":"2023-10-06T05:27:20.943260+0000","last_fullsized":"2023-10-06T05:27:20.943260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:20.632237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940502+0000","last_change":"2023-10-06T05:27:20.936410+0000","last_active":"2023-10-06T05:27:20.940502+0000","last_peered":"2023-10-06T05:27:20.940502+0000","last_clean":"2023-10-06T05:27:20.940502+0000","last_became_active":"2023-10-06T05:27:20.936137+0000","last_became_peered":"2023-10-06T05:27:20.936137+0000","last_unstale":"2023-10-06T05:27:20.940502+0000","last_undegraded":"2023-10-06T05:27:20.940502+0000","last_fullsized":"2023-10-06T05:27:20.940502+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:32.575068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.941900+0000","last_change":"2023-10-06T05:27:20.941900+0000","last_active":"2023-10-06T05:27:20.941900+0000","last_peered":"2023-10-06T05:27:20.941900+0000","last_clean":"2023-10-06T05:27:20.941900+0000","last_became_active":"2023-10-06T05:27:20.939644+0000","last_became_peered":"2023-10-06T05:27:20.939644+0000","last_unstale":"2023-10-06T05:27:20.941900+0000","last_undegraded":"2023-10-06T05:27:20.941900+0000","last_fullsized":"2023-10-06T05:27:20.941900+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:09.262099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932483+0000","last_change":"2023-10-06T05:27:20.932483+0000","last_active":"2023-10-06T05:27:20.932483+0000","last_peered":"2023-10-06T05:27:20.932483+0000","last_clean":"2023-10-06T05:27:20.932483+0000","last_became_active":"2023-10-06T05:27:20.931908+0000","last_became_peered":"2023-10-06T05:27:20.931908+0000","last_unstale":"2023-10-06T05:27:20.932483+0000","last_undegraded":"2023-10-06T05:27:20.932483+0000","last_fullsized":"2023-10-06T05:27:20.932483+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:57.093884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.935465+0000","last_change":"2023-10-06T05:27:20.935465+0000","last_active":"2023-10-06T05:27:20.935465+0000","last_peered":"2023-10-06T05:27:20.935465+0000","last_clean":"2023-10-06T05:27:20.935465+0000","last_became_active":"2023-10-06T05:27:20.935156+0000","last_became_peered":"2023-10-06T05:27:20.935156+0000","last_unstale":"2023-10-06T05:27:20.935465+0000","last_undegraded":"2023-10-06T05:27:20.935465+0000","last_fullsized":"2023-10-06T05:27:20.935465+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:37.607374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943552+0000","last_change":"2023-10-06T05:27:20.939762+0000","last_active":"2023-10-06T05:27:20.943552+0000","last_peered":"2023-10-06T05:27:20.943552+0000","last_clean":"2023-10-06T05:27:20.943552+0000","last_became_active":"2023-10-06T05:27:20.939493+0000","last_became_peered":"2023-10-06T05:27:20.939493+0000","last_unstale":"2023-10-06T05:27:20.943552+0000","last_undegraded":"2023-10-06T05:27:20.943552+0000","last_fullsized":"2023-10-06T05:27:20.943552+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:24.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940825+0000","last_change":"2023-10-06T05:27:20.940825+0000","last_active":"2023-10-06T05:27:20.940825+0000","last_peered":"2023-10-06T05:27:20.940825+0000","last_clean":"2023-10-06T05:27:20.940825+0000","last_became_active":"2023-10-06T05:27:20.940346+0000","last_became_peered":"2023-10-06T05:27:20.940346+0000","last_unstale":"2023-10-06T05:27:20.940825+0000","last_undegraded":"2023-10-06T05:27:20.940825+0000","last_fullsized":"2023-10-06T05:27:20.940825+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:51.451391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940804+0000","last_change":"2023-10-06T05:27:20.936886+0000","last_active":"2023-10-06T05:27:20.940804+0000","last_peered":"2023-10-06T05:27:20.940804+0000","last_clean":"2023-10-06T05:27:20.940804+0000","last_became_active":"2023-10-06T05:27:20.936630+0000","last_became_peered":"2023-10-06T05:27:20.936630+0000","last_unstale":"2023-10-06T05:27:20.940804+0000","last_undegraded":"2023-10-06T05:27:20.940804+0000","last_fullsized":"2023-10-06T05:27:20.940804+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:57:06.787188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.941132+0000","last_change":"2023-10-06T05:27:20.941132+0000","last_active":"2023-10-06T05:27:20.941132+0000","last_peered":"2023-10-06T05:27:20.941132+0000","last_clean":"2023-10-06T05:27:20.941132+0000","last_became_active":"2023-10-06T05:27:20.940872+0000","last_became_peered":"2023-10-06T05:27:20.940872+0000","last_unstale":"2023-10-06T05:27:20.941132+0000","last_undegraded":"2023-10-06T05:27:20.941132+0000","last_fullsized":"2023-10-06T05:27:20.941132+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:53.364001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932459+0000","last_change":"2023-10-06T05:27:20.932459+0000","last_active":"2023-10-06T05:27:20.932459+0000","last_peered":"2023-10-06T05:27:20.932459+0000","last_clean":"2023-10-06T05:27:20.932459+0000","last_became_active":"2023-10-06T05:27:20.932170+0000","last_became_peered":"2023-10-06T05:27:20.932170+0000","last_unstale":"2023-10-06T05:27:20.932459+0000","last_undegraded":"2023-10-06T05:27:20.932459+0000","last_fullsized":"2023-10-06T05:27:20.932459+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:39.161796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940484+0000","last_change":"2023-10-06T05:27:20.936149+0000","last_active":"2023-10-06T05:27:20.940484+0000","last_peered":"2023-10-06T05:27:20.940484+0000","last_clean":"2023-10-06T05:27:20.940484+0000","last_became_active":"2023-10-06T05:27:20.935882+0000","last_became_peered":"2023-10-06T05:27:20.935882+0000","last_unstale":"2023-10-06T05:27:20.940484+0000","last_undegraded":"2023-10-06T05:27:20.940484+0000","last_fullsized":"2023-10-06T05:27:20.940484+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:11.927694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940464+0000","last_change":"2023-10-06T05:27:20.935521+0000","last_active":"2023-10-06T05:27:20.940464+0000","last_peered":"2023-10-06T05:27:20.940464+0000","last_clean":"2023-10-06T05:27:20.940464+0000","last_became_active":"2023-10-06T05:27:20.935086+0000","last_became_peered":"2023-10-06T05:27:20.935086+0000","last_unstale":"2023-10-06T05:27:20.940464+0000","last_undegraded":"2023-10-06T05:27:20.940464+0000","last_fullsized":"2023-10-06T05:27:20.940464+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:20.632237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940186+0000","last_change":"2023-10-06T05:27:20.936637+0000","last_active":"2023-10-06T05:27:20.940186+0000","last_peered":"2023-10-06T05:27:20.940186+0000","last_clean":"2023-10-06T05:27:20.940186+0000","last_became_active":"2023-10-06T05:27:20.936319+0000","last_became_peered":"2023-10-06T05:27:20.936319+0000","last_unstale":"2023-10-06T05:27:20.940186+0000","last_undegraded":"2023-10-06T05:27:20.940186+0000","last_fullsized":"2023-10-06T05:27:20.940186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:17.377569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940382+0000","last_change":"2023-10-06T05:27:20.934423+0000","last_active":"2023-10-06T05:27:20.940382+0000","last_peered":"2023-10-06T05:27:20.940382+0000","last_clean":"2023-10-06T05:27:20.940382+0000","last_became_active":"2023-10-06T05:27:20.931395+0000","last_became_peered":"2023-10-06T05:27:20.931395+0000","last_unstale":"2023-10-06T05:27:20.940382+0000","last_undegraded":"2023-10-06T05:27:20.940382+0000","last_fullsized":"2023-10-06T05:27:20.940382+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:02.070913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940738+0000","last_change":"2023-10-06T05:27:20.933991+0000","last_active":"2023-10-06T05:27:20.940738+0000","last_peered":"2023-10-06T05:27:20.940738+0000","last_clean":"2023-10-06T05:27:20.940738+0000","last_became_active":"2023-10-06T05:27:20.933532+0000","last_became_peered":"2023-10-06T05:27:20.933532+0000","last_unstale":"2023-10-06T05:27:20.940738+0000","last_undegraded":"2023-10-06T05:27:20.940738+0000","last_fullsized":"2023-10-06T05:27:20.940738+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:02.277528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940785+0000","last_change":"2023-10-06T05:27:20.936611+0000","last_active":"2023-10-06T05:27:20.940785+0000","last_peered":"2023-10-06T05:27:20.940785+0000","last_clean":"2023-10-06T05:27:20.940785+0000","last_became_active":"2023-10-06T05:27:20.936337+0000","last_became_peered":"2023-10-06T05:27:20.936337+0000","last_unstale":"2023-10-06T05:27:20.940785+0000","last_undegraded":"2023-10-06T05:27:20.940785+0000","last_fullsized":"2023-10-06T05:27:20.940785+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:38.297380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943242+0000","last_change":"2023-10-06T05:27:20.943242+0000","last_active":"2023-10-06T05:27:20.943242+0000","last_peered":"2023-10-06T05:27:20.943242+0000","last_clean":"2023-10-06T05:27:20.943242+0000","last_became_active":"2023-10-06T05:27:20.940034+0000","last_became_peered":"2023-10-06T05:27:20.940034+0000","last_unstale":"2023-10-06T05:27:20.943242+0000","last_undegraded":"2023-10-06T05:27:20.943242+0000","last_fullsized":"2023-10-06T05:27:20.943242+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:05.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945737+0000","last_change":"2023-10-06T05:27:20.943276+0000","last_active":"2023-10-06T05:27:20.945737+0000","last_peered":"2023-10-06T05:27:20.945737+0000","last_clean":"2023-10-06T05:27:20.945737+0000","last_became_active":"2023-10-06T05:27:20.942997+0000","last_became_peered":"2023-10-06T05:27:20.942997+0000","last_unstale":"2023-10-06T05:27:20.945737+0000","last_undegraded":"2023-10-06T05:27:20.945737+0000","last_fullsized":"2023-10-06T05:27:20.945737+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:19.110346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945186+0000","last_change":"2023-10-06T05:27:20.944345+0000","last_active":"2023-10-06T05:27:20.945186+0000","last_peered":"2023-10-06T05:27:20.945186+0000","last_clean":"2023-10-06T05:27:20.945186+0000","last_became_active":"2023-10-06T05:27:20.944035+0000","last_became_peered":"2023-10-06T05:27:20.944035+0000","last_unstale":"2023-10-06T05:27:20.945186+0000","last_undegraded":"2023-10-06T05:27:20.945186+0000","last_fullsized":"2023-10-06T05:27:20.945186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:52.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.939058+0000","last_change":"2023-10-06T05:27:20.931587+0000","last_active":"2023-10-06T05:27:20.939058+0000","last_peered":"2023-10-06T05:27:20.939058+0000","last_clean":"2023-10-06T05:27:20.939058+0000","last_became_active":"2023-10-06T05:27:20.931294+0000","last_became_peered":"2023-10-06T05:27:20.931294+0000","last_unstale":"2023-10-06T05:27:20.939058+0000","last_undegraded":"2023-10-06T05:27:20.939058+0000","last_fullsized":"2023-10-06T05:27:20.939058+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:07.373004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.935915+0000","last_change":"2023-10-06T05:27:20.935915+0000","last_active":"2023-10-06T05:27:20.935915+0000","last_peered":"2023-10-06T05:27:20.935915+0000","last_clean":"2023-10-06T05:27:20.935915+0000","last_became_active":"2023-10-06T05:27:20.935653+0000","last_became_peered":"2023-10-06T05:27:20.935653+0000","last_unstale":"2023-10-06T05:27:20.935915+0000","last_undegraded":"2023-10-06T05:27:20.935915+0000","last_fullsized":"2023-10-06T05:27:20.935915+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:41.312664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940393+0000","last_change":"2023-10-06T05:27:20.940393+0000","last_active":"2023-10-06T05:27:20.940393+0000","last_peered":"2023-10-06T05:27:20.940393+0000","last_clean":"2023-10-06T05:27:20.940393+0000","last_became_active":"2023-10-06T05:27:20.939809+0000","last_became_peered":"2023-10-06T05:27:20.939809+0000","last_unstale":"2023-10-06T05:27:20.940393+0000","last_undegraded":"2023-10-06T05:27:20.940393+0000","last_fullsized":"2023-10-06T05:27:20.940393+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:07.373004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943717+0000","last_change":"2023-10-06T05:27:20.935384+0000","last_active":"2023-10-06T05:27:20.943717+0000","last_peered":"2023-10-06T05:27:20.943717+0000","last_clean":"2023-10-06T05:27:20.943717+0000","last_became_active":"2023-10-06T05:27:20.935095+0000","last_became_peered":"2023-10-06T05:27:20.935095+0000","last_unstale":"2023-10-06T05:27:20.943717+0000","last_undegraded":"2023-10-06T05:27:20.943717+0000","last_fullsized":"2023-10-06T05:27:20.943717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:59.314820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943400+0000","last_change":"2023-10-06T05:27:20.934554+0000","last_active":"2023-10-06T05:27:20.943400+0000","last_peered":"2023-10-06T05:27:20.943400+0000","last_clean":"2023-10-06T05:27:20.943400+0000","last_became_active":"2023-10-06T05:27:20.934274+0000","last_became_peered":"2023-10-06T05:27:20.934274+0000","last_unstale":"2023-10-06T05:27:20.943400+0000","last_undegraded":"2023-10-06T05:27:20.943400+0000","last_fullsized":"2023-10-06T05:27:20.943400+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:17.824716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"142'4","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115546+0000","last_change":"2023-10-06T05:27:20.109545+0000","last_active":"2023-10-06T05:27:21.115546+0000","last_peered":"2023-10-06T05:27:21.115546+0000","last_clean":"2023-10-06T05:27:21.115546+0000","last_became_active":"2023-10-06T05:27:20.109052+0000","last_became_peered":"2023-10-06T05:27:20.109052+0000","last_unstale":"2023-10-06T05:27:21.115546+0000","last_undegraded":"2023-10-06T05:27:21.115546+0000","last_fullsized":"2023-10-06T05:27:21.115546+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:23.637667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.933728+0000","last_change":"2023-10-06T05:27:20.933728+0000","last_active":"2023-10-06T05:27:20.933728+0000","last_peered":"2023-10-06T05:27:20.933728+0000","last_clean":"2023-10-06T05:27:20.933728+0000","last_became_active":"2023-10-06T05:27:20.933505+0000","last_became_peered":"2023-10-06T05:27:20.933505+0000","last_unstale":"2023-10-06T05:27:20.933728+0000","last_undegraded":"2023-10-06T05:27:20.933728+0000","last_fullsized":"2023-10-06T05:27:20.933728+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:33.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.921111+0000","last_change":"2023-10-06T05:27:20.108743+0000","last_active":"2023-10-06T05:27:20.921111+0000","last_peered":"2023-10-06T05:27:20.921111+0000","last_clean":"2023-10-06T05:27:20.921111+0000","last_became_active":"2023-10-06T05:27:20.108469+0000","last_became_peered":"2023-10-06T05:27:20.108469+0000","last_unstale":"2023-10-06T05:27:20.921111+0000","last_undegraded":"2023-10-06T05:27:20.921111+0000","last_fullsized":"2023-10-06T05:27:20.921111+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:32.300938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943628+0000","last_change":"2023-10-06T05:27:20.937837+0000","last_active":"2023-10-06T05:27:20.943628+0000","last_peered":"2023-10-06T05:27:20.943628+0000","last_clean":"2023-10-06T05:27:20.943628+0000","last_became_active":"2023-10-06T05:27:20.937502+0000","last_became_peered":"2023-10-06T05:27:20.937502+0000","last_unstale":"2023-10-06T05:27:20.943628+0000","last_undegraded":"2023-10-06T05:27:20.943628+0000","last_fullsized":"2023-10-06T05:27:20.943628+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:09.024160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"142'2","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.109301+0000","last_change":"2023-10-06T05:27:20.120271+0000","last_active":"2023-10-06T05:27:21.109301+0000","last_peered":"2023-10-06T05:27:21.109301+0000","last_clean":"2023-10-06T05:27:21.109301+0000","last_became_active":"2023-10-06T05:27:20.119970+0000","last_became_peered":"2023-10-06T05:27:20.119970+0000","last_unstale":"2023-10-06T05:27:21.109301+0000","last_undegraded":"2023-10-06T05:27:21.109301+0000","last_fullsized":"2023-10-06T05:27:21.109301+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:24.584977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.933339+0000","last_change":"2023-10-06T05:27:20.933339+0000","last_active":"2023-10-06T05:27:20.933339+0000","last_peered":"2023-10-06T05:27:20.933339+0000","last_clean":"2023-10-06T05:27:20.933339+0000","last_became_active":"2023-10-06T05:27:20.933053+0000","last_became_peered":"2023-10-06T05:27:20.933053+0000","last_unstale":"2023-10-06T05:27:20.933339+0000","last_undegraded":"2023-10-06T05:27:20.933339+0000","last_fullsized":"2023-10-06T05:27:20.933339+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:36.233871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"142'2","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.109152+0000","last_change":"2023-10-06T05:27:20.119082+0000","last_active":"2023-10-06T05:27:21.109152+0000","last_peered":"2023-10-06T05:27:21.109152+0000","last_clean":"2023-10-06T05:27:21.109152+0000","last_became_active":"2023-10-06T05:27:20.118788+0000","last_became_peered":"2023-10-06T05:27:20.118788+0000","last_unstale":"2023-10-06T05:27:21.109152+0000","last_undegraded":"2023-10-06T05:27:21.109152+0000","last_fullsized":"2023-10-06T05:27:21.109152+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:45.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942814+0000","last_change":"2023-10-06T05:27:20.942814+0000","last_active":"2023-10-06T05:27:20.942814+0000","last_peered":"2023-10-06T05:27:20.942814+0000","last_clean":"2023-10-06T05:27:20.942814+0000","last_became_active":"2023-10-06T05:27:20.939983+0000","last_became_peered":"2023-10-06T05:27:20.939983+0000","last_unstale":"2023-10-06T05:27:20.942814+0000","last_undegraded":"2023-10-06T05:27:20.942814+0000","last_fullsized":"2023-10-06T05:27:20.942814+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:54.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.920611+0000","last_change":"2023-10-06T05:27:20.117927+0000","last_active":"2023-10-06T05:27:20.920611+0000","last_peered":"2023-10-06T05:27:20.920611+0000","last_clean":"2023-10-06T05:27:20.920611+0000","last_became_active":"2023-10-06T05:27:20.117616+0000","last_became_peered":"2023-10-06T05:27:20.117616+0000","last_unstale":"2023-10-06T05:27:20.920611+0000","last_undegraded":"2023-10-06T05:27:20.920611+0000","last_fullsized":"2023-10-06T05:27:20.920611+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:58.825906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940443+0000","last_change":"2023-10-06T05:27:20.935351+0000","last_active":"2023-10-06T05:27:20.940443+0000","last_peered":"2023-10-06T05:27:20.940443+0000","last_clean":"2023-10-06T05:27:20.940443+0000","last_became_active":"2023-10-06T05:27:20.934744+0000","last_became_peered":"2023-10-06T05:27:20.934744+0000","last_unstale":"2023-10-06T05:27:20.940443+0000","last_undegraded":"2023-10-06T05:27:20.940443+0000","last_fullsized":"2023-10-06T05:27:20.940443+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:09.262099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115655+0000","last_change":"2023-10-06T05:27:20.111509+0000","last_active":"2023-10-06T05:27:21.115655+0000","last_peered":"2023-10-06T05:27:21.115655+0000","last_clean":"2023-10-06T05:27:21.115655+0000","last_became_active":"2023-10-06T05:27:20.111248+0000","last_became_peered":"2023-10-06T05:27:20.111248+0000","last_unstale":"2023-10-06T05:27:21.115655+0000","last_undegraded":"2023-10-06T05:27:21.115655+0000","last_fullsized":"2023-10-06T05:27:21.115655+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.143977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.941697+0000","last_change":"2023-10-06T05:27:20.941697+0000","last_active":"2023-10-06T05:27:20.941697+0000","last_peered":"2023-10-06T05:27:20.941697+0000","last_clean":"2023-10-06T05:27:20.941697+0000","last_became_active":"2023-10-06T05:27:20.940701+0000","last_became_peered":"2023-10-06T05:27:20.940701+0000","last_unstale":"2023-10-06T05:27:20.941697+0000","last_undegraded":"2023-10-06T05:27:20.941697+0000","last_fullsized":"2023-10-06T05:27:20.941697+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:00.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.924885+0000","last_change":"2023-10-06T05:27:20.115627+0000","last_active":"2023-10-06T05:27:20.924885+0000","last_peered":"2023-10-06T05:27:20.924885+0000","last_clean":"2023-10-06T05:27:20.924885+0000","last_became_active":"2023-10-06T05:27:20.115383+0000","last_became_peered":"2023-10-06T05:27:20.115383+0000","last_unstale":"2023-10-06T05:27:20.924885+0000","last_undegraded":"2023-10-06T05:27:20.924885+0000","last_fullsized":"2023-10-06T05:27:20.924885+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:56.088343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.933911+0000","last_change":"2023-10-06T05:27:20.933911+0000","last_active":"2023-10-06T05:27:20.933911+0000","last_peered":"2023-10-06T05:27:20.933911+0000","last_clean":"2023-10-06T05:27:20.933911+0000","last_became_active":"2023-10-06T05:27:20.933661+0000","last_became_peered":"2023-10-06T05:27:20.933661+0000","last_unstale":"2023-10-06T05:27:20.933911+0000","last_undegraded":"2023-10-06T05:27:20.933911+0000","last_fullsized":"2023-10-06T05:27:20.933911+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:59.314820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.126549+0000","last_change":"2023-10-06T05:27:20.125933+0000","last_active":"2023-10-06T05:27:20.126549+0000","last_peered":"2023-10-06T05:27:20.126549+0000","last_clean":"2023-10-06T05:27:20.126549+0000","last_became_active":"2023-10-06T05:27:20.125551+0000","last_became_peered":"2023-10-06T05:27:20.125551+0000","last_unstale":"2023-10-06T05:27:20.126549+0000","last_undegraded":"2023-10-06T05:27:20.126549+0000","last_fullsized":"2023-10-06T05:27:20.126549+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:30.607114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945064+0000","last_change":"2023-10-06T05:27:20.945064+0000","last_active":"2023-10-06T05:27:20.945064+0000","last_peered":"2023-10-06T05:27:20.945064+0000","last_clean":"2023-10-06T05:27:20.945064+0000","last_became_active":"2023-10-06T05:27:20.944643+0000","last_became_peered":"2023-10-06T05:27:20.944643+0000","last_unstale":"2023-10-06T05:27:20.945064+0000","last_undegraded":"2023-10-06T05:27:20.945064+0000","last_fullsized":"2023-10-06T05:27:20.945064+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:16.149518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929327+0000","last_change":"2023-10-06T05:27:20.108073+0000","last_active":"2023-10-06T05:27:20.929327+0000","last_peered":"2023-10-06T05:27:20.929327+0000","last_clean":"2023-10-06T05:27:20.929327+0000","last_became_active":"2023-10-06T05:27:20.107766+0000","last_became_peered":"2023-10-06T05:27:20.107766+0000","last_unstale":"2023-10-06T05:27:20.929327+0000","last_undegraded":"2023-10-06T05:27:20.929327+0000","last_fullsized":"2023-10-06T05:27:20.929327+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.931848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945724+0000","last_change":"2023-10-06T05:27:20.943259+0000","last_active":"2023-10-06T05:27:20.945724+0000","last_peered":"2023-10-06T05:27:20.945724+0000","last_clean":"2023-10-06T05:27:20.945724+0000","last_became_active":"2023-10-06T05:27:20.942989+0000","last_became_peered":"2023-10-06T05:27:20.942989+0000","last_unstale":"2023-10-06T05:27:20.945724+0000","last_undegraded":"2023-10-06T05:27:20.945724+0000","last_fullsized":"2023-10-06T05:27:20.945724+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:41.312664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"142'2","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.109423+0000","last_change":"2023-10-06T05:27:20.111279+0000","last_active":"2023-10-06T05:27:21.109423+0000","last_peered":"2023-10-06T05:27:21.109423+0000","last_clean":"2023-10-06T05:27:21.109423+0000","last_became_active":"2023-10-06T05:27:20.110878+0000","last_became_peered":"2023-10-06T05:27:20.110878+0000","last_unstale":"2023-10-06T05:27:21.109423+0000","last_undegraded":"2023-10-06T05:27:21.109423+0000","last_fullsized":"2023-10-06T05:27:21.109423+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:16.819176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942824+0000","last_change":"2023-10-06T05:27:20.942824+0000","last_active":"2023-10-06T05:27:20.942824+0000","last_peered":"2023-10-06T05:27:20.942824+0000","last_clean":"2023-10-06T05:27:20.942824+0000","last_became_active":"2023-10-06T05:27:20.942518+0000","last_became_peered":"2023-10-06T05:27:20.942518+0000","last_unstale":"2023-10-06T05:27:20.942824+0000","last_undegraded":"2023-10-06T05:27:20.942824+0000","last_fullsized":"2023-10-06T05:27:20.942824+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:16.240554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929399+0000","last_change":"2023-10-06T05:27:20.115884+0000","last_active":"2023-10-06T05:27:20.929399+0000","last_peered":"2023-10-06T05:27:20.929399+0000","last_clean":"2023-10-06T05:27:20.929399+0000","last_became_active":"2023-10-06T05:27:20.115315+0000","last_became_peered":"2023-10-06T05:27:20.115315+0000","last_unstale":"2023-10-06T05:27:20.929399+0000","last_undegraded":"2023-10-06T05:27:20.929399+0000","last_fullsized":"2023-10-06T05:27:20.929399+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:18.589955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.924715+0000","last_change":"2023-10-06T05:27:20.924715+0000","last_active":"2023-10-06T05:27:20.924715+0000","last_peered":"2023-10-06T05:27:20.924715+0000","last_clean":"2023-10-06T05:27:20.924715+0000","last_became_active":"2023-10-06T05:27:20.924431+0000","last_became_peered":"2023-10-06T05:27:20.924431+0000","last_unstale":"2023-10-06T05:27:20.924715+0000","last_undegraded":"2023-10-06T05:27:20.924715+0000","last_fullsized":"2023-10-06T05:27:20.924715+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:52.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.122455+0000","last_change":"2023-10-06T05:27:20.122025+0000","last_active":"2023-10-06T05:27:20.122455+0000","last_peered":"2023-10-06T05:27:20.122455+0000","last_clean":"2023-10-06T05:27:20.122455+0000","last_became_active":"2023-10-06T05:27:20.121770+0000","last_became_peered":"2023-10-06T05:27:20.121770+0000","last_unstale":"2023-10-06T05:27:20.122455+0000","last_undegraded":"2023-10-06T05:27:20.122455+0000","last_fullsized":"2023-10-06T05:27:20.122455+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:03.266305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.930285+0000","last_change":"2023-10-06T05:27:20.930285+0000","last_active":"2023-10-06T05:27:20.930285+0000","last_peered":"2023-10-06T05:27:20.930285+0000","last_clean":"2023-10-06T05:27:20.930285+0000","last_became_active":"2023-10-06T05:27:20.929997+0000","last_became_peered":"2023-10-06T05:27:20.929997+0000","last_unstale":"2023-10-06T05:27:20.930285+0000","last_undegraded":"2023-10-06T05:27:20.930285+0000","last_fullsized":"2023-10-06T05:27:20.930285+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:45.872372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115443+0000","last_change":"2023-10-06T05:27:20.112622+0000","last_active":"2023-10-06T05:27:21.115443+0000","last_peered":"2023-10-06T05:27:21.115443+0000","last_clean":"2023-10-06T05:27:21.115443+0000","last_became_active":"2023-10-06T05:27:20.112294+0000","last_became_peered":"2023-10-06T05:27:20.112294+0000","last_unstale":"2023-10-06T05:27:21.115443+0000","last_undegraded":"2023-10-06T05:27:21.115443+0000","last_fullsized":"2023-10-06T05:27:21.115443+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:57.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.939540+0000","last_change":"2023-10-06T05:27:20.939540+0000","last_active":"2023-10-06T05:27:20.939540+0000","last_peered":"2023-10-06T05:27:20.939540+0000","last_clean":"2023-10-06T05:27:20.939540+0000","last_became_active":"2023-10-06T05:27:20.939189+0000","last_became_peered":"2023-10-06T05:27:20.939189+0000","last_unstale":"2023-10-06T05:27:20.939540+0000","last_undegraded":"2023-10-06T05:27:20.939540+0000","last_fullsized":"2023-10-06T05:27:20.939540+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:11.272889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121588+0000","last_change":"2023-10-06T05:27:20.118400+0000","last_active":"2023-10-06T05:27:20.121588+0000","last_peered":"2023-10-06T05:27:20.121588+0000","last_clean":"2023-10-06T05:27:20.121588+0000","last_became_active":"2023-10-06T05:27:20.118135+0000","last_became_peered":"2023-10-06T05:27:20.118135+0000","last_unstale":"2023-10-06T05:27:20.121588+0000","last_undegraded":"2023-10-06T05:27:20.121588+0000","last_fullsized":"2023-10-06T05:27:20.121588+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:18.104805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932275+0000","last_change":"2023-10-06T05:27:20.932275+0000","last_active":"2023-10-06T05:27:20.932275+0000","last_peered":"2023-10-06T05:27:20.932275+0000","last_clean":"2023-10-06T05:27:20.932275+0000","last_became_active":"2023-10-06T05:27:20.931990+0000","last_became_peered":"2023-10-06T05:27:20.931990+0000","last_unstale":"2023-10-06T05:27:20.932275+0000","last_undegraded":"2023-10-06T05:27:20.932275+0000","last_fullsized":"2023-10-06T05:27:20.932275+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:31.612654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.931593+0000","last_change":"2023-10-06T05:27:20.116874+0000","last_active":"2023-10-06T05:27:20.931593+0000","last_peered":"2023-10-06T05:27:20.931593+0000","last_clean":"2023-10-06T05:27:20.931593+0000","last_became_active":"2023-10-06T05:27:20.113196+0000","last_became_peered":"2023-10-06T05:27:20.113196+0000","last_unstale":"2023-10-06T05:27:20.931593+0000","last_undegraded":"2023-10-06T05:27:20.931593+0000","last_fullsized":"2023-10-06T05:27:20.931593+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:11.343731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944257+0000","last_change":"2023-10-06T05:27:20.942355+0000","last_active":"2023-10-06T05:27:20.944257+0000","last_peered":"2023-10-06T05:27:20.944257+0000","last_clean":"2023-10-06T05:27:20.944257+0000","last_became_active":"2023-10-06T05:27:20.941779+0000","last_became_peered":"2023-10-06T05:27:20.941779+0000","last_unstale":"2023-10-06T05:27:20.944257+0000","last_undegraded":"2023-10-06T05:27:20.944257+0000","last_fullsized":"2023-10-06T05:27:20.944257+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:59.357444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.126107+0000","last_change":"2023-10-06T05:27:20.123106+0000","last_active":"2023-10-06T05:27:20.126107+0000","last_peered":"2023-10-06T05:27:20.126107+0000","last_clean":"2023-10-06T05:27:20.126107+0000","last_became_active":"2023-10-06T05:27:20.122803+0000","last_became_peered":"2023-10-06T05:27:20.122803+0000","last_unstale":"2023-10-06T05:27:20.126107+0000","last_undegraded":"2023-10-06T05:27:20.126107+0000","last_fullsized":"2023-10-06T05:27:20.126107+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:51.380550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944131+0000","last_change":"2023-10-06T05:27:20.944131+0000","last_active":"2023-10-06T05:27:20.944131+0000","last_peered":"2023-10-06T05:27:20.944131+0000","last_clean":"2023-10-06T05:27:20.944131+0000","last_became_active":"2023-10-06T05:27:20.943949+0000","last_became_peered":"2023-10-06T05:27:20.943949+0000","last_unstale":"2023-10-06T05:27:20.944131+0000","last_undegraded":"2023-10-06T05:27:20.944131+0000","last_fullsized":"2023-10-06T05:27:20.944131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:52.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.113317+0000","last_change":"2023-10-06T05:27:20.123262+0000","last_active":"2023-10-06T05:27:21.113317+0000","last_peered":"2023-10-06T05:27:21.113317+0000","last_clean":"2023-10-06T05:27:21.113317+0000","last_became_active":"2023-10-06T05:27:20.122963+0000","last_became_peered":"2023-10-06T05:27:20.122963+0000","last_unstale":"2023-10-06T05:27:21.113317+0000","last_undegraded":"2023-10-06T05:27:21.113317+0000","last_fullsized":"2023-10-06T05:27:21.113317+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:04.913075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942230+0000","last_change":"2023-10-06T05:27:20.942230+0000","last_active":"2023-10-06T05:27:20.942230+0000","last_peered":"2023-10-06T05:27:20.942230+0000","last_clean":"2023-10-06T05:27:20.942230+0000","last_became_active":"2023-10-06T05:27:20.940755+0000","last_became_peered":"2023-10-06T05:27:20.940755+0000","last_unstale":"2023-10-06T05:27:20.942230+0000","last_undegraded":"2023-10-06T05:27:20.942230+0000","last_fullsized":"2023-10-06T05:27:20.942230+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:16.826861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.110353+0000","last_change":"2023-10-06T05:27:20.108787+0000","last_active":"2023-10-06T05:27:20.110353+0000","last_peered":"2023-10-06T05:27:20.110353+0000","last_clean":"2023-10-06T05:27:20.110353+0000","last_became_active":"2023-10-06T05:27:20.108401+0000","last_became_peered":"2023-10-06T05:27:20.108401+0000","last_unstale":"2023-10-06T05:27:20.110353+0000","last_undegraded":"2023-10-06T05:27:20.110353+0000","last_fullsized":"2023-10-06T05:27:20.110353+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:45.990714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942247+0000","last_change":"2023-10-06T05:27:20.942247+0000","last_active":"2023-10-06T05:27:20.942247+0000","last_peered":"2023-10-06T05:27:20.942247+0000","last_clean":"2023-10-06T05:27:20.942247+0000","last_became_active":"2023-10-06T05:27:20.940358+0000","last_became_peered":"2023-10-06T05:27:20.940358+0000","last_unstale":"2023-10-06T05:27:20.942247+0000","last_undegraded":"2023-10-06T05:27:20.942247+0000","last_fullsized":"2023-10-06T05:27:20.942247+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:36.907052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.125104+0000","last_change":"2023-10-06T05:27:20.120510+0000","last_active":"2023-10-06T05:27:20.125104+0000","last_peered":"2023-10-06T05:27:20.125104+0000","last_clean":"2023-10-06T05:27:20.125104+0000","last_became_active":"2023-10-06T05:27:20.120171+0000","last_became_peered":"2023-10-06T05:27:20.120171+0000","last_unstale":"2023-10-06T05:27:20.125104+0000","last_undegraded":"2023-10-06T05:27:20.125104+0000","last_fullsized":"2023-10-06T05:27:20.125104+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:35.228330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943351+0000","last_change":"2023-10-06T05:27:20.943351+0000","last_active":"2023-10-06T05:27:20.943351+0000","last_peered":"2023-10-06T05:27:20.943351+0000","last_clean":"2023-10-06T05:27:20.943351+0000","last_became_active":"2023-10-06T05:27:20.943061+0000","last_became_peered":"2023-10-06T05:27:20.943061+0000","last_unstale":"2023-10-06T05:27:20.943351+0000","last_undegraded":"2023-10-06T05:27:20.943351+0000","last_fullsized":"2023-10-06T05:27:20.943351+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:59.357444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.925242+0000","last_change":"2023-10-06T05:27:20.114794+0000","last_active":"2023-10-06T05:27:20.925242+0000","last_peered":"2023-10-06T05:27:20.925242+0000","last_clean":"2023-10-06T05:27:20.925242+0000","last_became_active":"2023-10-06T05:27:20.114503+0000","last_became_peered":"2023-10-06T05:27:20.114503+0000","last_unstale":"2023-10-06T05:27:20.925242+0000","last_undegraded":"2023-10-06T05:27:20.925242+0000","last_fullsized":"2023-10-06T05:27:20.925242+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:45.138057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.938784+0000","last_change":"2023-10-06T05:27:20.931487+0000","last_active":"2023-10-06T05:27:20.938784+0000","last_peered":"2023-10-06T05:27:20.938784+0000","last_clean":"2023-10-06T05:27:20.938784+0000","last_became_active":"2023-10-06T05:27:20.931212+0000","last_became_peered":"2023-10-06T05:27:20.931212+0000","last_unstale":"2023-10-06T05:27:20.938784+0000","last_undegraded":"2023-10-06T05:27:20.938784+0000","last_fullsized":"2023-10-06T05:27:20.938784+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:05.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.107715+0000","last_change":"2023-10-06T05:27:20.106451+0000","last_active":"2023-10-06T05:27:20.107715+0000","last_peered":"2023-10-06T05:27:20.107715+0000","last_clean":"2023-10-06T05:27:20.107715+0000","last_became_active":"2023-10-06T05:27:20.106186+0000","last_became_peered":"2023-10-06T05:27:20.106186+0000","last_unstale":"2023-10-06T05:27:20.107715+0000","last_undegraded":"2023-10-06T05:27:20.107715+0000","last_fullsized":"2023-10-06T05:27:20.107715+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:57.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.939071+0000","last_change":"2023-10-06T05:27:20.933402+0000","last_active":"2023-10-06T05:27:20.939071+0000","last_peered":"2023-10-06T05:27:20.939071+0000","last_clean":"2023-10-06T05:27:20.939071+0000","last_became_active":"2023-10-06T05:27:20.933098+0000","last_became_peered":"2023-10-06T05:27:20.933098+0000","last_unstale":"2023-10-06T05:27:20.939071+0000","last_undegraded":"2023-10-06T05:27:20.939071+0000","last_fullsized":"2023-10-06T05:27:20.939071+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:47.618195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.925597+0000","last_change":"2023-10-06T05:27:20.118479+0000","last_active":"2023-10-06T05:27:20.925597+0000","last_peered":"2023-10-06T05:27:20.925597+0000","last_clean":"2023-10-06T05:27:20.925597+0000","last_became_active":"2023-10-06T05:27:20.118218+0000","last_became_peered":"2023-10-06T05:27:20.118218+0000","last_unstale":"2023-10-06T05:27:20.925597+0000","last_undegraded":"2023-10-06T05:27:20.925597+0000","last_fullsized":"2023-10-06T05:27:20.925597+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:30.607114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.934125+0000","last_change":"2023-10-06T05:27:20.934125+0000","last_active":"2023-10-06T05:27:20.934125+0000","last_peered":"2023-10-06T05:27:20.934125+0000","last_clean":"2023-10-06T05:27:20.934125+0000","last_became_active":"2023-10-06T05:27:20.933784+0000","last_became_peered":"2023-10-06T05:27:20.933784+0000","last_unstale":"2023-10-06T05:27:20.934125+0000","last_undegraded":"2023-10-06T05:27:20.934125+0000","last_fullsized":"2023-10-06T05:27:20.934125+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:17.937389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121680+0000","last_change":"2023-10-06T05:27:20.119179+0000","last_active":"2023-10-06T05:27:20.121680+0000","last_peered":"2023-10-06T05:27:20.121680+0000","last_clean":"2023-10-06T05:27:20.121680+0000","last_became_active":"2023-10-06T05:27:20.118836+0000","last_became_peered":"2023-10-06T05:27:20.118836+0000","last_unstale":"2023-10-06T05:27:20.121680+0000","last_undegraded":"2023-10-06T05:27:20.121680+0000","last_fullsized":"2023-10-06T05:27:20.121680+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:58.309280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.946731+0000","last_change":"2023-10-06T05:27:20.946731+0000","last_active":"2023-10-06T05:27:20.946731+0000","last_peered":"2023-10-06T05:27:20.946731+0000","last_clean":"2023-10-06T05:27:20.946731+0000","last_became_active":"2023-10-06T05:27:20.946489+0000","last_became_peered":"2023-10-06T05:27:20.946489+0000","last_unstale":"2023-10-06T05:27:20.946731+0000","last_undegraded":"2023-10-06T05:27:20.946731+0000","last_fullsized":"2023-10-06T05:27:20.946731+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:58.213731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.126092+0000","last_change":"2023-10-06T05:27:20.119642+0000","last_active":"2023-10-06T05:27:20.126092+0000","last_peered":"2023-10-06T05:27:20.126092+0000","last_clean":"2023-10-06T05:27:20.126092+0000","last_became_active":"2023-10-06T05:27:20.118049+0000","last_became_peered":"2023-10-06T05:27:20.118049+0000","last_unstale":"2023-10-06T05:27:20.126092+0000","last_undegraded":"2023-10-06T05:27:20.126092+0000","last_fullsized":"2023-10-06T05:27:20.126092+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:58.351904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943490+0000","last_change":"2023-10-06T05:27:20.943490+0000","last_active":"2023-10-06T05:27:20.943490+0000","last_peered":"2023-10-06T05:27:20.943490+0000","last_clean":"2023-10-06T05:27:20.943490+0000","last_became_active":"2023-10-06T05:27:20.943252+0000","last_became_peered":"2023-10-06T05:27:20.943252+0000","last_unstale":"2023-10-06T05:27:20.943490+0000","last_undegraded":"2023-10-06T05:27:20.943490+0000","last_fullsized":"2023-10-06T05:27:20.943490+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:02.277528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.924109+0000","last_change":"2023-10-06T05:27:20.112005+0000","last_active":"2023-10-06T05:27:20.924109+0000","last_peered":"2023-10-06T05:27:20.924109+0000","last_clean":"2023-10-06T05:27:20.924109+0000","last_became_active":"2023-10-06T05:27:20.111732+0000","last_became_peered":"2023-10-06T05:27:20.111732+0000","last_unstale":"2023-10-06T05:27:20.924109+0000","last_undegraded":"2023-10-06T05:27:20.924109+0000","last_fullsized":"2023-10-06T05:27:20.924109+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:59.089399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929778+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:27:20.929778+0000","last_peered":"2023-10-06T05:27:20.929778+0000","last_clean":"2023-10-06T05:27:20.929778+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:27:20.929778+0000","last_undegraded":"2023-10-06T05:27:20.929778+0000","last_fullsized":"2023-10-06T05:27:20.929778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944234+0000","last_change":"2023-10-06T05:27:20.942128+0000","last_active":"2023-10-06T05:27:20.944234+0000","last_peered":"2023-10-06T05:27:20.944234+0000","last_clean":"2023-10-06T05:27:20.944234+0000","last_became_active":"2023-10-06T05:27:20.941338+0000","last_became_peered":"2023-10-06T05:27:20.941338+0000","last_unstale":"2023-10-06T05:27:20.944234+0000","last_undegraded":"2023-10-06T05:27:20.944234+0000","last_fullsized":"2023-10-06T05:27:20.944234+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:19.000419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"142'8","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.116883+0000","last_change":"2023-10-06T05:27:20.124721+0000","last_active":"2023-10-06T05:27:21.116883+0000","last_peered":"2023-10-06T05:27:21.116883+0000","last_clean":"2023-10-06T05:27:21.116883+0000","last_became_active":"2023-10-06T05:27:20.124368+0000","last_became_peered":"2023-10-06T05:27:20.124368+0000","last_unstale":"2023-10-06T05:27:21.116883+0000","last_undegraded":"2023-10-06T05:27:21.116883+0000","last_fullsized":"2023-10-06T05:27:21.116883+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:08.018620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.099217+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:27:20.099217+0000","last_peered":"2023-10-06T05:27:20.099217+0000","last_clean":"2023-10-06T05:27:20.099217+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:27:20.099217+0000","last_undegraded":"2023-10-06T05:27:20.099217+0000","last_fullsized":"2023-10-06T05:27:20.099217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940166+0000","last_change":"2023-10-06T05:27:20.935335+0000","last_active":"2023-10-06T05:27:20.940166+0000","last_peered":"2023-10-06T05:27:20.940166+0000","last_clean":"2023-10-06T05:27:20.940166+0000","last_became_active":"2023-10-06T05:27:20.934906+0000","last_became_peered":"2023-10-06T05:27:20.934906+0000","last_unstale":"2023-10-06T05:27:20.940166+0000","last_undegraded":"2023-10-06T05:27:20.940166+0000","last_fullsized":"2023-10-06T05:27:20.940166+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:53.364001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115987+0000","last_change":"2023-10-06T05:27:20.112960+0000","last_active":"2023-10-06T05:27:21.115987+0000","last_peered":"2023-10-06T05:27:21.115987+0000","last_clean":"2023-10-06T05:27:21.115987+0000","last_became_active":"2023-10-06T05:27:20.112651+0000","last_became_peered":"2023-10-06T05:27:20.112651+0000","last_unstale":"2023-10-06T05:27:21.115987+0000","last_undegraded":"2023-10-06T05:27:21.115987+0000","last_fullsized":"2023-10-06T05:27:21.115987+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:44.096170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":124,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.104141+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:27:20.104141+0000","last_peered":"2023-10-06T05:27:20.104141+0000","last_clean":"2023-10-06T05:27:20.104141+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:27:20.104141+0000","last_undegraded":"2023-10-06T05:27:20.104141+0000","last_fullsized":"2023-10-06T05:27:20.104141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940717+0000","last_change":"2023-10-06T05:27:20.933807+0000","last_active":"2023-10-06T05:27:20.940717+0000","last_peered":"2023-10-06T05:27:20.940717+0000","last_clean":"2023-10-06T05:27:20.940717+0000","last_became_active":"2023-10-06T05:27:20.933243+0000","last_became_peered":"2023-10-06T05:27:20.933243+0000","last_unstale":"2023-10-06T05:27:20.940717+0000","last_undegraded":"2023-10-06T05:27:20.940717+0000","last_fullsized":"2023-10-06T05:27:20.940717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:33.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.122625+0000","last_change":"2023-10-06T05:27:20.122080+0000","last_active":"2023-10-06T05:27:20.122625+0000","last_peered":"2023-10-06T05:27:20.122625+0000","last_clean":"2023-10-06T05:27:20.122625+0000","last_became_active":"2023-10-06T05:27:20.121820+0000","last_became_peered":"2023-10-06T05:27:20.121820+0000","last_unstale":"2023-10-06T05:27:20.122625+0000","last_undegraded":"2023-10-06T05:27:20.122625+0000","last_fullsized":"2023-10-06T05:27:20.122625+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:58.480434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.100100+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:27:20.100100+0000","last_peered":"2023-10-06T05:27:20.100100+0000","last_clean":"2023-10-06T05:27:20.100100+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:27:20.100100+0000","last_undegraded":"2023-10-06T05:27:20.100100+0000","last_fullsized":"2023-10-06T05:27:20.100100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.103292+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:27:20.103292+0000","last_peered":"2023-10-06T05:27:20.103292+0000","last_clean":"2023-10-06T05:27:20.103292+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:27:20.103292+0000","last_undegraded":"2023-10-06T05:27:20.103292+0000","last_fullsized":"2023-10-06T05:27:20.103292+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.936901+0000","last_change":"2023-10-06T05:27:20.933054+0000","last_active":"2023-10-06T05:27:20.936901+0000","last_peered":"2023-10-06T05:27:20.936901+0000","last_clean":"2023-10-06T05:27:20.936901+0000","last_became_active":"2023-10-06T05:27:20.932725+0000","last_became_peered":"2023-10-06T05:27:20.932725+0000","last_unstale":"2023-10-06T05:27:20.936901+0000","last_undegraded":"2023-10-06T05:27:20.936901+0000","last_fullsized":"2023-10-06T05:27:20.936901+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.775287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121005+0000","last_change":"2023-10-06T05:27:20.120545+0000","last_active":"2023-10-06T05:27:20.121005+0000","last_peered":"2023-10-06T05:27:20.121005+0000","last_clean":"2023-10-06T05:27:20.121005+0000","last_became_active":"2023-10-06T05:27:20.120282+0000","last_became_peered":"2023-10-06T05:27:20.120282+0000","last_unstale":"2023-10-06T05:27:20.121005+0000","last_undegraded":"2023-10-06T05:27:20.121005+0000","last_fullsized":"2023-10-06T05:27:20.121005+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:37.541610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.103377+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:27:20.103377+0000","last_peered":"2023-10-06T05:27:20.103377+0000","last_clean":"2023-10-06T05:27:20.103377+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:27:20.103377+0000","last_undegraded":"2023-10-06T05:27:20.103377+0000","last_fullsized":"2023-10-06T05:27:20.103377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932722+0000","last_change":"2023-10-06T05:27:20.932722+0000","last_active":"2023-10-06T05:27:20.932722+0000","last_peered":"2023-10-06T05:27:20.932722+0000","last_clean":"2023-10-06T05:27:20.932722+0000","last_became_active":"2023-10-06T05:27:20.932427+0000","last_became_peered":"2023-10-06T05:27:20.932427+0000","last_unstale":"2023-10-06T05:27:20.932722+0000","last_undegraded":"2023-10-06T05:27:20.932722+0000","last_fullsized":"2023-10-06T05:27:20.932722+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:00.601747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121541+0000","last_change":"2023-10-06T05:27:20.117995+0000","last_active":"2023-10-06T05:27:20.121541+0000","last_peered":"2023-10-06T05:27:20.121541+0000","last_clean":"2023-10-06T05:27:20.121541+0000","last_became_active":"2023-10-06T05:27:20.117697+0000","last_became_peered":"2023-10-06T05:27:20.117697+0000","last_unstale":"2023-10-06T05:27:20.121541+0000","last_undegraded":"2023-10-06T05:27:20.121541+0000","last_fullsized":"2023-10-06T05:27:20.121541+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:45.092670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.103888+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:27:20.103888+0000","last_peered":"2023-10-06T05:27:20.103888+0000","last_clean":"2023-10-06T05:27:20.103888+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:27:20.103888+0000","last_undegraded":"2023-10-06T05:27:20.103888+0000","last_fullsized":"2023-10-06T05:27:20.103888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.926379+0000","last_change":"2023-10-06T05:27:20.926379+0000","last_active":"2023-10-06T05:27:20.926379+0000","last_peered":"2023-10-06T05:27:20.926379+0000","last_clean":"2023-10-06T05:27:20.926379+0000","last_became_active":"2023-10-06T05:27:20.926136+0000","last_became_peered":"2023-10-06T05:27:20.926136+0000","last_unstale":"2023-10-06T05:27:20.926379+0000","last_undegraded":"2023-10-06T05:27:20.926379+0000","last_fullsized":"2023-10-06T05:27:20.926379+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:24.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"142'6","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.113537+0000","last_change":"2023-10-06T05:27:20.125761+0000","last_active":"2023-10-06T05:27:21.113537+0000","last_peered":"2023-10-06T05:27:21.113537+0000","last_clean":"2023-10-06T05:27:21.113537+0000","last_became_active":"2023-10-06T05:27:20.125503+0000","last_became_peered":"2023-10-06T05:27:20.125503+0000","last_unstale":"2023-10-06T05:27:21.113537+0000","last_undegraded":"2023-10-06T05:27:21.113537+0000","last_fullsized":"2023-10-06T05:27:21.113537+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:36.601834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929545+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:27:20.929545+0000","last_peered":"2023-10-06T05:27:20.929545+0000","last_clean":"2023-10-06T05:27:20.929545+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:27:20.929545+0000","last_undegraded":"2023-10-06T05:27:20.929545+0000","last_fullsized":"2023-10-06T05:27:20.929545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944220+0000","last_change":"2023-10-06T05:27:20.941557+0000","last_active":"2023-10-06T05:27:20.944220+0000","last_peered":"2023-10-06T05:27:20.944220+0000","last_clean":"2023-10-06T05:27:20.944220+0000","last_became_active":"2023-10-06T05:27:20.941284+0000","last_became_peered":"2023-10-06T05:27:20.941284+0000","last_unstale":"2023-10-06T05:27:20.944220+0000","last_undegraded":"2023-10-06T05:27:20.944220+0000","last_fullsized":"2023-10-06T05:27:20.944220+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:49.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"142'3","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.112975+0000","last_change":"2023-10-06T05:27:20.118823+0000","last_active":"2023-10-06T05:27:21.112975+0000","last_peered":"2023-10-06T05:27:21.112975+0000","last_clean":"2023-10-06T05:27:21.112975+0000","last_became_active":"2023-10-06T05:27:20.118522+0000","last_became_peered":"2023-10-06T05:27:20.118522+0000","last_unstale":"2023-10-06T05:27:21.112975+0000","last_undegraded":"2023-10-06T05:27:21.112975+0000","last_fullsized":"2023-10-06T05:27:21.112975+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:27.995944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":119,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.102797+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:27:20.102797+0000","last_peered":"2023-10-06T05:27:20.102797+0000","last_clean":"2023-10-06T05:27:20.102797+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:27:20.102797+0000","last_undegraded":"2023-10-06T05:27:20.102797+0000","last_fullsized":"2023-10-06T05:27:20.102797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307884,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063956,"statfs":{"total":96636764160,"available":96321490944,"internally_reserved":0,"allocated":2461696,"data_stored":1561627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.878},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":1.476},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":1.5129999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":2.0190000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0059999999999998},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58699999999999997,"5min":0.76800000000000002,"15min":0.90300000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.504,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.76800000000000002,"15min":0.93500000000000005},"last":0.442}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68999999999999995,"5min":0.81699999999999995,"15min":0.90000000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.84899999999999998},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.91000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.314},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.92900000000000005},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1863680,"data_stored":970741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":4.6900000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":0.86799999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.76400000000000001,"15min":5.6239999999999997},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.73899999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":11.595000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":5.5800000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.93600000000000005},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":23.146000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.73699999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":9.7539999999999996},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.61099999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":9.9969999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.64600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.751,"5min":0.752,"15min":9.5850000000000009},"last":0.434}]},{"osd":5,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.496,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.6439999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":9.8420000000000005},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":9.718},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":9.8930000000000007},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307316,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064524,"statfs":{"total":96636764160,"available":96322072576,"internally_reserved":0,"allocated":1880064,"data_stored":977756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.64400000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.40699999999999997,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.73699999999999999},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.3999999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58199999999999996,"5min":0.67100000000000004,"15min":1.673},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.503},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":4.0490000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77700000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.92600000000000005,"15min":0.94699999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.95199999999999996},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":1.0369999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.44},"max":{"1min":0.871,"5min":0.95599999999999996,"15min":1.167},"last":0.69599999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.94499999999999995,"15min":1.026},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92100000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.66000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307896,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063944,"statfs":{"total":96636764160,"available":96321478656,"internally_reserved":0,"allocated":2473984,"data_stored":1562687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.30599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56799999999999995,"5min":0.82099999999999995,"15min":1.038},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.30299999999999999,"5min":0.27400000000000002,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":1.0609999999999999},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.252},"max":{"1min":0.60799999999999998,"5min":0.67800000000000005,"15min":3.855},"last":0.439},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1919999999999999},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.45200000000000001},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":7.5880000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":7.7670000000000003},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.4039999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.97199999999999998,"15min":2.0870000000000002},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":2.4940000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":5.3600000000000003},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.01,"15min":1.1759999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.45900000000000002,"15min":0.45400000000000001},"max":{"1min":0.874,"5min":0.99199999999999999,"15min":1.0940000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.97899999999999998,"15min":1.0169999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.47499999999999998,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":1.2150000000000001},"last":0.72899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307296,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064544,"statfs":{"total":96636764160,"available":96322093056,"internally_reserved":0,"allocated":1859584,"data_stored":970671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.39500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":2.0720000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.899},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":2.097},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.48699999999999999},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":1.8759999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":1.6719999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.48299999999999998},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":0.70499999999999996,"15min":2.121},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":1.012,"15min":1.7789999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.018,"15min":1.643},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66300000000000003,"15min":0.68999999999999995},"min":{"1min":0.495,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":1.2130000000000001,"15min":2.0129999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.69499999999999995},"min":{"1min":0.51000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.851},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81999999999999995,"5min":0.88300000000000001,"15min":1.571},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.0800000000000001,"15min":2.0449999999999999},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0569999999999999,"15min":1.8180000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.44500000000000001},"max":{"1min":1.0029999999999999,"5min":1.131,"15min":1.613},"last":0.86399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307320,"kb_used_data":1840,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064520,"statfs":{"total":96636764160,"available":96322068480,"internally_reserved":0,"allocated":1884160,"data_stored":977885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26900000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64500000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40799999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.22900000000000001},"max":{"1min":0.63900000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40999999999999998,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61399999999999999,"5min":1.75,"15min":1.75},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60099999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72699999999999998,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.433}]},{"osd":4,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.0469999999999999,"15min":1.143},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":1.855,"15min":1.855},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.43099999999999999},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":1.885,"15min":1.885},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":1.236},"last":0.621},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":1.1950000000000001},"last":0.80300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311404,"kb_used_data":1828,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060436,"statfs":{"total":96636764160,"available":96317886464,"internally_reserved":0,"allocated":1871872,"data_stored":977632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":12.983000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.33400000000000002,"15min":0.33100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":1.7430000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.8080000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":12.911},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.73099999999999998,"5min":0.84299999999999997,"15min":5.0890000000000004},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":12.999000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":13.289999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.95099999999999996,"15min":1.5760000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.88200000000000001,"15min":1.552},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.54600000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.85699999999999998,"15min":12.949},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.92900000000000005,"15min":5.2549999999999999},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":2.871},"last":0.439},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.91500000000000004,"15min":12.859},"last":0.39000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":1867776,"data_stored":971247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.79400000000000004,"15min":0.82299999999999995},"last":0.5},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.89400000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.82199999999999995,"15min":0.84599999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.81599999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.83299999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.876},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.67600000000000005,"5min":0.85999999999999999,"15min":0.875},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.49199999999999999,"5min":0.432,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.499,"15min":0.48399999999999999},"min":{"1min":0.36499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.92800000000000005},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.317,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.86699999999999999,"15min":0.876},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63}]},{"osd":7,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84899999999999998,"15min":0.89300000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.875,"15min":0.90900000000000003},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:27:26.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:27:26.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.453+0000 7f45fa172700 1 Processor -- start 2023-10-06T05:27:26.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.453+0000 7f45fa172700 1 -- start start 2023-10-06T05:27:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45fa172700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 0x7f45f415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45fa172700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f45f41560a0 0x7f45f40adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45fa172700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40ae8d0 con 0x7f45f40ccdc0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45fa172700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f40aebb0 con 0x7f45f41560a0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45fa172700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40aee90 con 0x7f45f4155cd0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f3fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45f40ccdc0 0x7f45f40c73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40126/0 (socket says 172.21.15.45:40126) 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f3fff700 1 -- 172.21.15.45:0/3098810486 learned_addr learned my addr 172.21.15.45:0/3098810486 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 0x7f45f415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f2ffd700 1 --2- 172.21.15.45:0/3098810486 >> v2:172.21.15.144:3300/0 conn(0x7f45f41560a0 0x7f45f40adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736307683 0 0) 0x7f45f40ae8d0 con 0x7f45f40ccdc0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e0003570 con 0x7f45f40ccdc0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988589297 0 0) 0x7f45e0003570 con 0x7f45f40ccdc0 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.454+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 msgr2=0x7f45f415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 0x7f45f415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 >> v2:172.21.15.144:3300/0 conn(0x7f45f41560a0 msgr2=0x7f45f40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 --2- 172.21.15.45:0/3098810486 >> v2:172.21.15.144:3300/0 conn(0x7f45f41560a0 0x7f45f40adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40af170 con 0x7f45f40ccdc0 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f45ec0017f0 con 0x7f45f40ccdc0 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f2ffd700 1 --2- 172.21.15.45:0/3098810486 >> v2:172.21.15.144:3300/0 conn(0x7f45f41560a0 0x7f45f40adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 >> v1:172.21.15.45:6789/0 conn(0x7f45f40ccdc0 legacy=0x7f45f40c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e0003a60 0x7f45e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0004810 0x7f45e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 0x7f45e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40aebb0 con 0x7f45e0003a60 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40aee90 con 0x7f45e0004810 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e000ba60 con 0x7f45f4155cd0 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e0003a60 0x7f45e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f2ffd700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0004810 0x7f45e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f3fff700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 0x7f45e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f37fe700 1 -- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 msgr2=0x7f45e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f4155cd0 0x7f45e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f37fe700 1 -- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0004810 msgr2=0x7f45e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0004810 0x7f45e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f37fe700 1 -- 172.21.15.45:0/3098810486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e0003570 con 0x7f45e0003a60 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e0003a60 0x7f45e0003e70 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009c30 tx=0x7f45dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e1ec35bd31b2d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00e710 con 0x7f45e0003a60 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc00e870 con 0x7f45e0003a60 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.455+0000 7f45f27fc700 1 -- 172.21.15.45:0/3098810486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00ec30 con 0x7f45e0003a60 2023-10-06T05:27:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.456+0000 7f45fa172700 1 -- 172.21.15.45:0/3098810486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e0003a60 msgr2=0x7f45e0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.456+0000 7f45fa172700 1 --2- 172.21.15.45:0/3098810486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e0003a60 0x7f45e0003e70 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009c30 tx=0x7f45dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.456+0000 7f45f3fff700 1 -- 172.21.15.45:0/3098810486 reap_dead start 2023-10-06T05:27:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.456+0000 7f45fa172700 1 -- 172.21.15.45:0/3098810486 shutdown_connections 2023-10-06T05:27:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.456+0000 7f45fa172700 1 -- 172.21.15.45:0/3098810486 >> 172.21.15.45:0/3098810486 conn(0x7f45f406f090 msgr2=0x7f45f406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.456+0000 7f45fa172700 1 -- 172.21.15.45:0/3098810486 shutdown_connections 2023-10-06T05:27:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.456+0000 7f45fa172700 1 -- 172.21.15.45:0/3098810486 wait complete. 2023-10-06T05:27:26.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.457+0000 7f45fa172700 1 Processor -- start 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.457+0000 7f45fa172700 1 -- start start 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45fa172700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0003a60 0x7f45f40afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45fa172700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40b1b70 0x7f45f40c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45fa172700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f40c5c30 0x7f45f40c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45fa172700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40c60a0 con 0x7f45f40b1b70 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45fa172700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4157250 con 0x7f45e0003a60 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45fa172700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b0410 con 0x7f45f40c5c30 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45f2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40b1b70 0x7f45f40c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45f37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0003a60 0x7f45f40afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45f37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0003a60 0x7f45f40afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:46754/0 (socket says 172.21.15.45:46754) 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45f2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40b1b70 0x7f45f40c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39120/0 (socket says 172.21.15.45:39120) 2023-10-06T05:27:26.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45f37fe700 1 -- 172.21.15.45:0/3448198066 learned_addr learned my addr 172.21.15.45:0/3448198066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.458+0000 7f45f3fff700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f40c5c30 0x7f45f40c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45f2ffd700 1 -- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f40c5c30 msgr2=0x7f45f40c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45f2ffd700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f40c5c30 0x7f45f40c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45f2ffd700 1 -- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0003a60 msgr2=0x7f45f40afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45f2ffd700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0003a60 0x7f45f40afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45f2ffd700 1 -- 172.21.15.45:0/3448198066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40b06f0 con 0x7f45f40b1b70 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0003a60 0x7f45f40afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45f2ffd700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40b1b70 0x7f45f40c56f0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f45e4011d40 tx=0x7f45e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3448198066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4011e40 con 0x7f45f40b1b70 2023-10-06T05:27:26.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3448198066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e400fc20 con 0x7f45f40b1b70 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f40b09d0 con 0x7f45f40b1b70 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3448198066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4014440 con 0x7f45f40b1b70 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.459+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4156e00 con 0x7f45f40b1b70 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.460+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f40b1970 con 0x7f45f40b1b70 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.461+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3448198066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45e4014830 con 0x7f45f40b1b70 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.461+0000 7f45eb7fe700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f45d4066f30 0x7f45d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.461+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3448198066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f45e4018030 con 0x7f45f40b1b70 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.461+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f45d4066f30 0x7f45d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.462+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f45d4066f30 0x7f45d40693e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0091e0 tx=0x7f45dc019000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.465+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3448198066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45e4055bf0 con 0x7f45f40b1b70 2023-10-06T05:27:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.666+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45f41573c0 con 0x7f45d4066f30 2023-10-06T05:27:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.675+0000 7f45eb7fe700 1 -- 172.21.15.45:0/3448198066 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312267 (secure 0 0 0) 0x7f45f41573c0 con 0x7f45d4066f30 2023-10-06T05:27:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f45d4066f30 msgr2=0x7f45d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f45d4066f30 0x7f45d40693e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0091e0 tx=0x7f45dc019000 comp rx=0 tx=0).stop 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40b1b70 msgr2=0x7f45f40c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40b1b70 0x7f45f40c56f0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f45e4011d40 tx=0x7f45e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 shutdown_connections 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f45d4066f30 0x7f45d40693e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45f40c5c30 0x7f45f40c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0003a60 0x7f45f40afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 --2- 172.21.15.45:0/3448198066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40b1b70 0x7f45f40c56f0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 >> 172.21.15.45:0/3448198066 conn(0x7f45f406f090 msgr2=0x7f45f4154060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 shutdown_connections 2023-10-06T05:27:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.679+0000 7f45fa172700 1 -- 172.21.15.45:0/3448198066 wait complete. 2023-10-06T05:27:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:27:26.694 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2023-10-06T05:27:26.634160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2455528,"kb_used_data":15784,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752519192,"statfs":{"total":773094113280,"available":770579652608,"internally_reserved":0,"allocated":16162816,"data_stored":8970246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.538535"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942026+0000","last_change":"2023-10-06T05:27:20.942026+0000","last_active":"2023-10-06T05:27:20.942026+0000","last_peered":"2023-10-06T05:27:20.942026+0000","last_clean":"2023-10-06T05:27:20.942026+0000","last_became_active":"2023-10-06T05:27:20.940430+0000","last_became_peered":"2023-10-06T05:27:20.940430+0000","last_unstale":"2023-10-06T05:27:20.942026+0000","last_undegraded":"2023-10-06T05:27:20.942026+0000","last_fullsized":"2023-10-06T05:27:20.942026+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:59.901425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940463+0000","last_change":"2023-10-06T05:27:20.935099+0000","last_active":"2023-10-06T05:27:20.940463+0000","last_peered":"2023-10-06T05:27:20.940463+0000","last_clean":"2023-10-06T05:27:20.940463+0000","last_became_active":"2023-10-06T05:27:20.934804+0000","last_became_peered":"2023-10-06T05:27:20.934804+0000","last_unstale":"2023-10-06T05:27:20.940463+0000","last_undegraded":"2023-10-06T05:27:20.940463+0000","last_fullsized":"2023-10-06T05:27:20.940463+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:35.285755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.930390+0000","last_change":"2023-10-06T05:27:20.930390+0000","last_active":"2023-10-06T05:27:20.930390+0000","last_peered":"2023-10-06T05:27:20.930390+0000","last_clean":"2023-10-06T05:27:20.930390+0000","last_became_active":"2023-10-06T05:27:20.930074+0000","last_became_peered":"2023-10-06T05:27:20.930074+0000","last_unstale":"2023-10-06T05:27:20.930390+0000","last_undegraded":"2023-10-06T05:27:20.930390+0000","last_fullsized":"2023-10-06T05:27:20.930390+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:59.357444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940043+0000","last_change":"2023-10-06T05:27:20.940043+0000","last_active":"2023-10-06T05:27:20.940043+0000","last_peered":"2023-10-06T05:27:20.940043+0000","last_clean":"2023-10-06T05:27:20.940043+0000","last_became_active":"2023-10-06T05:27:20.939323+0000","last_became_peered":"2023-10-06T05:27:20.939323+0000","last_unstale":"2023-10-06T05:27:20.940043+0000","last_undegraded":"2023-10-06T05:27:20.940043+0000","last_fullsized":"2023-10-06T05:27:20.940043+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:22.797307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.937459+0000","last_change":"2023-10-06T05:27:20.937459+0000","last_active":"2023-10-06T05:27:20.937459+0000","last_peered":"2023-10-06T05:27:20.937459+0000","last_clean":"2023-10-06T05:27:20.937459+0000","last_became_active":"2023-10-06T05:27:20.937158+0000","last_became_peered":"2023-10-06T05:27:20.937158+0000","last_unstale":"2023-10-06T05:27:20.937459+0000","last_undegraded":"2023-10-06T05:27:20.937459+0000","last_fullsized":"2023-10-06T05:27:20.937459+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:00.094940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943368+0000","last_change":"2023-10-06T05:27:20.938405+0000","last_active":"2023-10-06T05:27:20.943368+0000","last_peered":"2023-10-06T05:27:20.943368+0000","last_clean":"2023-10-06T05:27:20.943368+0000","last_became_active":"2023-10-06T05:27:20.938135+0000","last_became_peered":"2023-10-06T05:27:20.938135+0000","last_unstale":"2023-10-06T05:27:20.943368+0000","last_undegraded":"2023-10-06T05:27:20.943368+0000","last_fullsized":"2023-10-06T05:27:20.943368+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:00.601747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.934017+0000","last_change":"2023-10-06T05:27:20.934017+0000","last_active":"2023-10-06T05:27:20.934017+0000","last_peered":"2023-10-06T05:27:20.934017+0000","last_clean":"2023-10-06T05:27:20.934017+0000","last_became_active":"2023-10-06T05:27:20.933779+0000","last_became_peered":"2023-10-06T05:27:20.933779+0000","last_unstale":"2023-10-06T05:27:20.934017+0000","last_undegraded":"2023-10-06T05:27:20.934017+0000","last_fullsized":"2023-10-06T05:27:20.934017+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:09.024160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.937557+0000","last_change":"2023-10-06T05:27:20.937557+0000","last_active":"2023-10-06T05:27:20.937557+0000","last_peered":"2023-10-06T05:27:20.937557+0000","last_clean":"2023-10-06T05:27:20.937557+0000","last_became_active":"2023-10-06T05:27:20.935748+0000","last_became_peered":"2023-10-06T05:27:20.935748+0000","last_unstale":"2023-10-06T05:27:20.937557+0000","last_undegraded":"2023-10-06T05:27:20.937557+0000","last_fullsized":"2023-10-06T05:27:20.937557+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:29.001485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944198+0000","last_change":"2023-10-06T05:27:20.944198+0000","last_active":"2023-10-06T05:27:20.944198+0000","last_peered":"2023-10-06T05:27:20.944198+0000","last_clean":"2023-10-06T05:27:20.944198+0000","last_became_active":"2023-10-06T05:27:20.943579+0000","last_became_peered":"2023-10-06T05:27:20.943579+0000","last_unstale":"2023-10-06T05:27:20.944198+0000","last_undegraded":"2023-10-06T05:27:20.944198+0000","last_fullsized":"2023-10-06T05:27:20.944198+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:38.975075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943260+0000","last_change":"2023-10-06T05:27:20.943260+0000","last_active":"2023-10-06T05:27:20.943260+0000","last_peered":"2023-10-06T05:27:20.943260+0000","last_clean":"2023-10-06T05:27:20.943260+0000","last_became_active":"2023-10-06T05:27:20.942925+0000","last_became_peered":"2023-10-06T05:27:20.942925+0000","last_unstale":"2023-10-06T05:27:20.943260+0000","last_undegraded":"2023-10-06T05:27:20.943260+0000","last_fullsized":"2023-10-06T05:27:20.943260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:20.632237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940502+0000","last_change":"2023-10-06T05:27:20.936410+0000","last_active":"2023-10-06T05:27:20.940502+0000","last_peered":"2023-10-06T05:27:20.940502+0000","last_clean":"2023-10-06T05:27:20.940502+0000","last_became_active":"2023-10-06T05:27:20.936137+0000","last_became_peered":"2023-10-06T05:27:20.936137+0000","last_unstale":"2023-10-06T05:27:20.940502+0000","last_undegraded":"2023-10-06T05:27:20.940502+0000","last_fullsized":"2023-10-06T05:27:20.940502+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:32.575068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.941900+0000","last_change":"2023-10-06T05:27:20.941900+0000","last_active":"2023-10-06T05:27:20.941900+0000","last_peered":"2023-10-06T05:27:20.941900+0000","last_clean":"2023-10-06T05:27:20.941900+0000","last_became_active":"2023-10-06T05:27:20.939644+0000","last_became_peered":"2023-10-06T05:27:20.939644+0000","last_unstale":"2023-10-06T05:27:20.941900+0000","last_undegraded":"2023-10-06T05:27:20.941900+0000","last_fullsized":"2023-10-06T05:27:20.941900+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:09.262099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932483+0000","last_change":"2023-10-06T05:27:20.932483+0000","last_active":"2023-10-06T05:27:20.932483+0000","last_peered":"2023-10-06T05:27:20.932483+0000","last_clean":"2023-10-06T05:27:20.932483+0000","last_became_active":"2023-10-06T05:27:20.931908+0000","last_became_peered":"2023-10-06T05:27:20.931908+0000","last_unstale":"2023-10-06T05:27:20.932483+0000","last_undegraded":"2023-10-06T05:27:20.932483+0000","last_fullsized":"2023-10-06T05:27:20.932483+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:57.093884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.935465+0000","last_change":"2023-10-06T05:27:20.935465+0000","last_active":"2023-10-06T05:27:20.935465+0000","last_peered":"2023-10-06T05:27:20.935465+0000","last_clean":"2023-10-06T05:27:20.935465+0000","last_became_active":"2023-10-06T05:27:20.935156+0000","last_became_peered":"2023-10-06T05:27:20.935156+0000","last_unstale":"2023-10-06T05:27:20.935465+0000","last_undegraded":"2023-10-06T05:27:20.935465+0000","last_fullsized":"2023-10-06T05:27:20.935465+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:37.607374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943552+0000","last_change":"2023-10-06T05:27:20.939762+0000","last_active":"2023-10-06T05:27:20.943552+0000","last_peered":"2023-10-06T05:27:20.943552+0000","last_clean":"2023-10-06T05:27:20.943552+0000","last_became_active":"2023-10-06T05:27:20.939493+0000","last_became_peered":"2023-10-06T05:27:20.939493+0000","last_unstale":"2023-10-06T05:27:20.943552+0000","last_undegraded":"2023-10-06T05:27:20.943552+0000","last_fullsized":"2023-10-06T05:27:20.943552+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:24.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940825+0000","last_change":"2023-10-06T05:27:20.940825+0000","last_active":"2023-10-06T05:27:20.940825+0000","last_peered":"2023-10-06T05:27:20.940825+0000","last_clean":"2023-10-06T05:27:20.940825+0000","last_became_active":"2023-10-06T05:27:20.940346+0000","last_became_peered":"2023-10-06T05:27:20.940346+0000","last_unstale":"2023-10-06T05:27:20.940825+0000","last_undegraded":"2023-10-06T05:27:20.940825+0000","last_fullsized":"2023-10-06T05:27:20.940825+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:51.451391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940804+0000","last_change":"2023-10-06T05:27:20.936886+0000","last_active":"2023-10-06T05:27:20.940804+0000","last_peered":"2023-10-06T05:27:20.940804+0000","last_clean":"2023-10-06T05:27:20.940804+0000","last_became_active":"2023-10-06T05:27:20.936630+0000","last_became_peered":"2023-10-06T05:27:20.936630+0000","last_unstale":"2023-10-06T05:27:20.940804+0000","last_undegraded":"2023-10-06T05:27:20.940804+0000","last_fullsized":"2023-10-06T05:27:20.940804+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:57:06.787188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.941132+0000","last_change":"2023-10-06T05:27:20.941132+0000","last_active":"2023-10-06T05:27:20.941132+0000","last_peered":"2023-10-06T05:27:20.941132+0000","last_clean":"2023-10-06T05:27:20.941132+0000","last_became_active":"2023-10-06T05:27:20.940872+0000","last_became_peered":"2023-10-06T05:27:20.940872+0000","last_unstale":"2023-10-06T05:27:20.941132+0000","last_undegraded":"2023-10-06T05:27:20.941132+0000","last_fullsized":"2023-10-06T05:27:20.941132+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:53.364001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932459+0000","last_change":"2023-10-06T05:27:20.932459+0000","last_active":"2023-10-06T05:27:20.932459+0000","last_peered":"2023-10-06T05:27:20.932459+0000","last_clean":"2023-10-06T05:27:20.932459+0000","last_became_active":"2023-10-06T05:27:20.932170+0000","last_became_peered":"2023-10-06T05:27:20.932170+0000","last_unstale":"2023-10-06T05:27:20.932459+0000","last_undegraded":"2023-10-06T05:27:20.932459+0000","last_fullsized":"2023-10-06T05:27:20.932459+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:39.161796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940484+0000","last_change":"2023-10-06T05:27:20.936149+0000","last_active":"2023-10-06T05:27:20.940484+0000","last_peered":"2023-10-06T05:27:20.940484+0000","last_clean":"2023-10-06T05:27:20.940484+0000","last_became_active":"2023-10-06T05:27:20.935882+0000","last_became_peered":"2023-10-06T05:27:20.935882+0000","last_unstale":"2023-10-06T05:27:20.940484+0000","last_undegraded":"2023-10-06T05:27:20.940484+0000","last_fullsized":"2023-10-06T05:27:20.940484+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:11.927694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940464+0000","last_change":"2023-10-06T05:27:20.935521+0000","last_active":"2023-10-06T05:27:20.940464+0000","last_peered":"2023-10-06T05:27:20.940464+0000","last_clean":"2023-10-06T05:27:20.940464+0000","last_became_active":"2023-10-06T05:27:20.935086+0000","last_became_peered":"2023-10-06T05:27:20.935086+0000","last_unstale":"2023-10-06T05:27:20.940464+0000","last_undegraded":"2023-10-06T05:27:20.940464+0000","last_fullsized":"2023-10-06T05:27:20.940464+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:20.632237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940186+0000","last_change":"2023-10-06T05:27:20.936637+0000","last_active":"2023-10-06T05:27:20.940186+0000","last_peered":"2023-10-06T05:27:20.940186+0000","last_clean":"2023-10-06T05:27:20.940186+0000","last_became_active":"2023-10-06T05:27:20.936319+0000","last_became_peered":"2023-10-06T05:27:20.936319+0000","last_unstale":"2023-10-06T05:27:20.940186+0000","last_undegraded":"2023-10-06T05:27:20.940186+0000","last_fullsized":"2023-10-06T05:27:20.940186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:17.377569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940382+0000","last_change":"2023-10-06T05:27:20.934423+0000","last_active":"2023-10-06T05:27:20.940382+0000","last_peered":"2023-10-06T05:27:20.940382+0000","last_clean":"2023-10-06T05:27:20.940382+0000","last_became_active":"2023-10-06T05:27:20.931395+0000","last_became_peered":"2023-10-06T05:27:20.931395+0000","last_unstale":"2023-10-06T05:27:20.940382+0000","last_undegraded":"2023-10-06T05:27:20.940382+0000","last_fullsized":"2023-10-06T05:27:20.940382+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:02.070913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940738+0000","last_change":"2023-10-06T05:27:20.933991+0000","last_active":"2023-10-06T05:27:20.940738+0000","last_peered":"2023-10-06T05:27:20.940738+0000","last_clean":"2023-10-06T05:27:20.940738+0000","last_became_active":"2023-10-06T05:27:20.933532+0000","last_became_peered":"2023-10-06T05:27:20.933532+0000","last_unstale":"2023-10-06T05:27:20.940738+0000","last_undegraded":"2023-10-06T05:27:20.940738+0000","last_fullsized":"2023-10-06T05:27:20.940738+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:02.277528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940785+0000","last_change":"2023-10-06T05:27:20.936611+0000","last_active":"2023-10-06T05:27:20.940785+0000","last_peered":"2023-10-06T05:27:20.940785+0000","last_clean":"2023-10-06T05:27:20.940785+0000","last_became_active":"2023-10-06T05:27:20.936337+0000","last_became_peered":"2023-10-06T05:27:20.936337+0000","last_unstale":"2023-10-06T05:27:20.940785+0000","last_undegraded":"2023-10-06T05:27:20.940785+0000","last_fullsized":"2023-10-06T05:27:20.940785+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:38.297380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943242+0000","last_change":"2023-10-06T05:27:20.943242+0000","last_active":"2023-10-06T05:27:20.943242+0000","last_peered":"2023-10-06T05:27:20.943242+0000","last_clean":"2023-10-06T05:27:20.943242+0000","last_became_active":"2023-10-06T05:27:20.940034+0000","last_became_peered":"2023-10-06T05:27:20.940034+0000","last_unstale":"2023-10-06T05:27:20.943242+0000","last_undegraded":"2023-10-06T05:27:20.943242+0000","last_fullsized":"2023-10-06T05:27:20.943242+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:05.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945737+0000","last_change":"2023-10-06T05:27:20.943276+0000","last_active":"2023-10-06T05:27:20.945737+0000","last_peered":"2023-10-06T05:27:20.945737+0000","last_clean":"2023-10-06T05:27:20.945737+0000","last_became_active":"2023-10-06T05:27:20.942997+0000","last_became_peered":"2023-10-06T05:27:20.942997+0000","last_unstale":"2023-10-06T05:27:20.945737+0000","last_undegraded":"2023-10-06T05:27:20.945737+0000","last_fullsized":"2023-10-06T05:27:20.945737+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:19.110346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945186+0000","last_change":"2023-10-06T05:27:20.944345+0000","last_active":"2023-10-06T05:27:20.945186+0000","last_peered":"2023-10-06T05:27:20.945186+0000","last_clean":"2023-10-06T05:27:20.945186+0000","last_became_active":"2023-10-06T05:27:20.944035+0000","last_became_peered":"2023-10-06T05:27:20.944035+0000","last_unstale":"2023-10-06T05:27:20.945186+0000","last_undegraded":"2023-10-06T05:27:20.945186+0000","last_fullsized":"2023-10-06T05:27:20.945186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:52.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.939058+0000","last_change":"2023-10-06T05:27:20.931587+0000","last_active":"2023-10-06T05:27:20.939058+0000","last_peered":"2023-10-06T05:27:20.939058+0000","last_clean":"2023-10-06T05:27:20.939058+0000","last_became_active":"2023-10-06T05:27:20.931294+0000","last_became_peered":"2023-10-06T05:27:20.931294+0000","last_unstale":"2023-10-06T05:27:20.939058+0000","last_undegraded":"2023-10-06T05:27:20.939058+0000","last_fullsized":"2023-10-06T05:27:20.939058+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:07.373004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.935915+0000","last_change":"2023-10-06T05:27:20.935915+0000","last_active":"2023-10-06T05:27:20.935915+0000","last_peered":"2023-10-06T05:27:20.935915+0000","last_clean":"2023-10-06T05:27:20.935915+0000","last_became_active":"2023-10-06T05:27:20.935653+0000","last_became_peered":"2023-10-06T05:27:20.935653+0000","last_unstale":"2023-10-06T05:27:20.935915+0000","last_undegraded":"2023-10-06T05:27:20.935915+0000","last_fullsized":"2023-10-06T05:27:20.935915+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:41.312664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940393+0000","last_change":"2023-10-06T05:27:20.940393+0000","last_active":"2023-10-06T05:27:20.940393+0000","last_peered":"2023-10-06T05:27:20.940393+0000","last_clean":"2023-10-06T05:27:20.940393+0000","last_became_active":"2023-10-06T05:27:20.939809+0000","last_became_peered":"2023-10-06T05:27:20.939809+0000","last_unstale":"2023-10-06T05:27:20.940393+0000","last_undegraded":"2023-10-06T05:27:20.940393+0000","last_fullsized":"2023-10-06T05:27:20.940393+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:07.373004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943717+0000","last_change":"2023-10-06T05:27:20.935384+0000","last_active":"2023-10-06T05:27:20.943717+0000","last_peered":"2023-10-06T05:27:20.943717+0000","last_clean":"2023-10-06T05:27:20.943717+0000","last_became_active":"2023-10-06T05:27:20.935095+0000","last_became_peered":"2023-10-06T05:27:20.935095+0000","last_unstale":"2023-10-06T05:27:20.943717+0000","last_undegraded":"2023-10-06T05:27:20.943717+0000","last_fullsized":"2023-10-06T05:27:20.943717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:59.314820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943400+0000","last_change":"2023-10-06T05:27:20.934554+0000","last_active":"2023-10-06T05:27:20.943400+0000","last_peered":"2023-10-06T05:27:20.943400+0000","last_clean":"2023-10-06T05:27:20.943400+0000","last_became_active":"2023-10-06T05:27:20.934274+0000","last_became_peered":"2023-10-06T05:27:20.934274+0000","last_unstale":"2023-10-06T05:27:20.943400+0000","last_undegraded":"2023-10-06T05:27:20.943400+0000","last_fullsized":"2023-10-06T05:27:20.943400+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:17.824716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"142'4","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115546+0000","last_change":"2023-10-06T05:27:20.109545+0000","last_active":"2023-10-06T05:27:21.115546+0000","last_peered":"2023-10-06T05:27:21.115546+0000","last_clean":"2023-10-06T05:27:21.115546+0000","last_became_active":"2023-10-06T05:27:20.109052+0000","last_became_peered":"2023-10-06T05:27:20.109052+0000","last_unstale":"2023-10-06T05:27:21.115546+0000","last_undegraded":"2023-10-06T05:27:21.115546+0000","last_fullsized":"2023-10-06T05:27:21.115546+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:23.637667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.933728+0000","last_change":"2023-10-06T05:27:20.933728+0000","last_active":"2023-10-06T05:27:20.933728+0000","last_peered":"2023-10-06T05:27:20.933728+0000","last_clean":"2023-10-06T05:27:20.933728+0000","last_became_active":"2023-10-06T05:27:20.933505+0000","last_became_peered":"2023-10-06T05:27:20.933505+0000","last_unstale":"2023-10-06T05:27:20.933728+0000","last_undegraded":"2023-10-06T05:27:20.933728+0000","last_fullsized":"2023-10-06T05:27:20.933728+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:33.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.921111+0000","last_change":"2023-10-06T05:27:20.108743+0000","last_active":"2023-10-06T05:27:20.921111+0000","last_peered":"2023-10-06T05:27:20.921111+0000","last_clean":"2023-10-06T05:27:20.921111+0000","last_became_active":"2023-10-06T05:27:20.108469+0000","last_became_peered":"2023-10-06T05:27:20.108469+0000","last_unstale":"2023-10-06T05:27:20.921111+0000","last_undegraded":"2023-10-06T05:27:20.921111+0000","last_fullsized":"2023-10-06T05:27:20.921111+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:32.300938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943628+0000","last_change":"2023-10-06T05:27:20.937837+0000","last_active":"2023-10-06T05:27:20.943628+0000","last_peered":"2023-10-06T05:27:20.943628+0000","last_clean":"2023-10-06T05:27:20.943628+0000","last_became_active":"2023-10-06T05:27:20.937502+0000","last_became_peered":"2023-10-06T05:27:20.937502+0000","last_unstale":"2023-10-06T05:27:20.943628+0000","last_undegraded":"2023-10-06T05:27:20.943628+0000","last_fullsized":"2023-10-06T05:27:20.943628+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:09.024160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"142'2","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.109301+0000","last_change":"2023-10-06T05:27:20.120271+0000","last_active":"2023-10-06T05:27:21.109301+0000","last_peered":"2023-10-06T05:27:21.109301+0000","last_clean":"2023-10-06T05:27:21.109301+0000","last_became_active":"2023-10-06T05:27:20.119970+0000","last_became_peered":"2023-10-06T05:27:20.119970+0000","last_unstale":"2023-10-06T05:27:21.109301+0000","last_undegraded":"2023-10-06T05:27:21.109301+0000","last_fullsized":"2023-10-06T05:27:21.109301+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:24.584977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.933339+0000","last_change":"2023-10-06T05:27:20.933339+0000","last_active":"2023-10-06T05:27:20.933339+0000","last_peered":"2023-10-06T05:27:20.933339+0000","last_clean":"2023-10-06T05:27:20.933339+0000","last_became_active":"2023-10-06T05:27:20.933053+0000","last_became_peered":"2023-10-06T05:27:20.933053+0000","last_unstale":"2023-10-06T05:27:20.933339+0000","last_undegraded":"2023-10-06T05:27:20.933339+0000","last_fullsized":"2023-10-06T05:27:20.933339+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:36.233871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"142'2","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.109152+0000","last_change":"2023-10-06T05:27:20.119082+0000","last_active":"2023-10-06T05:27:21.109152+0000","last_peered":"2023-10-06T05:27:21.109152+0000","last_clean":"2023-10-06T05:27:21.109152+0000","last_became_active":"2023-10-06T05:27:20.118788+0000","last_became_peered":"2023-10-06T05:27:20.118788+0000","last_unstale":"2023-10-06T05:27:21.109152+0000","last_undegraded":"2023-10-06T05:27:21.109152+0000","last_fullsized":"2023-10-06T05:27:21.109152+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:45.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942814+0000","last_change":"2023-10-06T05:27:20.942814+0000","last_active":"2023-10-06T05:27:20.942814+0000","last_peered":"2023-10-06T05:27:20.942814+0000","last_clean":"2023-10-06T05:27:20.942814+0000","last_became_active":"2023-10-06T05:27:20.939983+0000","last_became_peered":"2023-10-06T05:27:20.939983+0000","last_unstale":"2023-10-06T05:27:20.942814+0000","last_undegraded":"2023-10-06T05:27:20.942814+0000","last_fullsized":"2023-10-06T05:27:20.942814+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:54.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.920611+0000","last_change":"2023-10-06T05:27:20.117927+0000","last_active":"2023-10-06T05:27:20.920611+0000","last_peered":"2023-10-06T05:27:20.920611+0000","last_clean":"2023-10-06T05:27:20.920611+0000","last_became_active":"2023-10-06T05:27:20.117616+0000","last_became_peered":"2023-10-06T05:27:20.117616+0000","last_unstale":"2023-10-06T05:27:20.920611+0000","last_undegraded":"2023-10-06T05:27:20.920611+0000","last_fullsized":"2023-10-06T05:27:20.920611+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:58.825906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940443+0000","last_change":"2023-10-06T05:27:20.935351+0000","last_active":"2023-10-06T05:27:20.940443+0000","last_peered":"2023-10-06T05:27:20.940443+0000","last_clean":"2023-10-06T05:27:20.940443+0000","last_became_active":"2023-10-06T05:27:20.934744+0000","last_became_peered":"2023-10-06T05:27:20.934744+0000","last_unstale":"2023-10-06T05:27:20.940443+0000","last_undegraded":"2023-10-06T05:27:20.940443+0000","last_fullsized":"2023-10-06T05:27:20.940443+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:09.262099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"142'1","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115655+0000","last_change":"2023-10-06T05:27:20.111509+0000","last_active":"2023-10-06T05:27:21.115655+0000","last_peered":"2023-10-06T05:27:21.115655+0000","last_clean":"2023-10-06T05:27:21.115655+0000","last_became_active":"2023-10-06T05:27:20.111248+0000","last_became_peered":"2023-10-06T05:27:20.111248+0000","last_unstale":"2023-10-06T05:27:21.115655+0000","last_undegraded":"2023-10-06T05:27:21.115655+0000","last_fullsized":"2023-10-06T05:27:21.115655+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.143977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.941697+0000","last_change":"2023-10-06T05:27:20.941697+0000","last_active":"2023-10-06T05:27:20.941697+0000","last_peered":"2023-10-06T05:27:20.941697+0000","last_clean":"2023-10-06T05:27:20.941697+0000","last_became_active":"2023-10-06T05:27:20.940701+0000","last_became_peered":"2023-10-06T05:27:20.940701+0000","last_unstale":"2023-10-06T05:27:20.941697+0000","last_undegraded":"2023-10-06T05:27:20.941697+0000","last_fullsized":"2023-10-06T05:27:20.941697+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:00.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.924885+0000","last_change":"2023-10-06T05:27:20.115627+0000","last_active":"2023-10-06T05:27:20.924885+0000","last_peered":"2023-10-06T05:27:20.924885+0000","last_clean":"2023-10-06T05:27:20.924885+0000","last_became_active":"2023-10-06T05:27:20.115383+0000","last_became_peered":"2023-10-06T05:27:20.115383+0000","last_unstale":"2023-10-06T05:27:20.924885+0000","last_undegraded":"2023-10-06T05:27:20.924885+0000","last_fullsized":"2023-10-06T05:27:20.924885+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:56.088343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.933911+0000","last_change":"2023-10-06T05:27:20.933911+0000","last_active":"2023-10-06T05:27:20.933911+0000","last_peered":"2023-10-06T05:27:20.933911+0000","last_clean":"2023-10-06T05:27:20.933911+0000","last_became_active":"2023-10-06T05:27:20.933661+0000","last_became_peered":"2023-10-06T05:27:20.933661+0000","last_unstale":"2023-10-06T05:27:20.933911+0000","last_undegraded":"2023-10-06T05:27:20.933911+0000","last_fullsized":"2023-10-06T05:27:20.933911+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:59.314820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.126549+0000","last_change":"2023-10-06T05:27:20.125933+0000","last_active":"2023-10-06T05:27:20.126549+0000","last_peered":"2023-10-06T05:27:20.126549+0000","last_clean":"2023-10-06T05:27:20.126549+0000","last_became_active":"2023-10-06T05:27:20.125551+0000","last_became_peered":"2023-10-06T05:27:20.125551+0000","last_unstale":"2023-10-06T05:27:20.126549+0000","last_undegraded":"2023-10-06T05:27:20.126549+0000","last_fullsized":"2023-10-06T05:27:20.126549+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:30.607114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945064+0000","last_change":"2023-10-06T05:27:20.945064+0000","last_active":"2023-10-06T05:27:20.945064+0000","last_peered":"2023-10-06T05:27:20.945064+0000","last_clean":"2023-10-06T05:27:20.945064+0000","last_became_active":"2023-10-06T05:27:20.944643+0000","last_became_peered":"2023-10-06T05:27:20.944643+0000","last_unstale":"2023-10-06T05:27:20.945064+0000","last_undegraded":"2023-10-06T05:27:20.945064+0000","last_fullsized":"2023-10-06T05:27:20.945064+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:16.149518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929327+0000","last_change":"2023-10-06T05:27:20.108073+0000","last_active":"2023-10-06T05:27:20.929327+0000","last_peered":"2023-10-06T05:27:20.929327+0000","last_clean":"2023-10-06T05:27:20.929327+0000","last_became_active":"2023-10-06T05:27:20.107766+0000","last_became_peered":"2023-10-06T05:27:20.107766+0000","last_unstale":"2023-10-06T05:27:20.929327+0000","last_undegraded":"2023-10-06T05:27:20.929327+0000","last_fullsized":"2023-10-06T05:27:20.929327+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:16.931848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.945724+0000","last_change":"2023-10-06T05:27:20.943259+0000","last_active":"2023-10-06T05:27:20.945724+0000","last_peered":"2023-10-06T05:27:20.945724+0000","last_clean":"2023-10-06T05:27:20.945724+0000","last_became_active":"2023-10-06T05:27:20.942989+0000","last_became_peered":"2023-10-06T05:27:20.942989+0000","last_unstale":"2023-10-06T05:27:20.945724+0000","last_undegraded":"2023-10-06T05:27:20.945724+0000","last_fullsized":"2023-10-06T05:27:20.945724+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:41.312664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"142'2","reported_seq":11,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.109423+0000","last_change":"2023-10-06T05:27:20.111279+0000","last_active":"2023-10-06T05:27:21.109423+0000","last_peered":"2023-10-06T05:27:21.109423+0000","last_clean":"2023-10-06T05:27:21.109423+0000","last_became_active":"2023-10-06T05:27:20.110878+0000","last_became_peered":"2023-10-06T05:27:20.110878+0000","last_unstale":"2023-10-06T05:27:21.109423+0000","last_undegraded":"2023-10-06T05:27:21.109423+0000","last_fullsized":"2023-10-06T05:27:21.109423+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:16.819176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942824+0000","last_change":"2023-10-06T05:27:20.942824+0000","last_active":"2023-10-06T05:27:20.942824+0000","last_peered":"2023-10-06T05:27:20.942824+0000","last_clean":"2023-10-06T05:27:20.942824+0000","last_became_active":"2023-10-06T05:27:20.942518+0000","last_became_peered":"2023-10-06T05:27:20.942518+0000","last_unstale":"2023-10-06T05:27:20.942824+0000","last_undegraded":"2023-10-06T05:27:20.942824+0000","last_fullsized":"2023-10-06T05:27:20.942824+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:16.240554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"142'1","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929399+0000","last_change":"2023-10-06T05:27:20.115884+0000","last_active":"2023-10-06T05:27:20.929399+0000","last_peered":"2023-10-06T05:27:20.929399+0000","last_clean":"2023-10-06T05:27:20.929399+0000","last_became_active":"2023-10-06T05:27:20.115315+0000","last_became_peered":"2023-10-06T05:27:20.115315+0000","last_unstale":"2023-10-06T05:27:20.929399+0000","last_undegraded":"2023-10-06T05:27:20.929399+0000","last_fullsized":"2023-10-06T05:27:20.929399+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:18.589955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.924715+0000","last_change":"2023-10-06T05:27:20.924715+0000","last_active":"2023-10-06T05:27:20.924715+0000","last_peered":"2023-10-06T05:27:20.924715+0000","last_clean":"2023-10-06T05:27:20.924715+0000","last_became_active":"2023-10-06T05:27:20.924431+0000","last_became_peered":"2023-10-06T05:27:20.924431+0000","last_unstale":"2023-10-06T05:27:20.924715+0000","last_undegraded":"2023-10-06T05:27:20.924715+0000","last_fullsized":"2023-10-06T05:27:20.924715+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:52.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.122455+0000","last_change":"2023-10-06T05:27:20.122025+0000","last_active":"2023-10-06T05:27:20.122455+0000","last_peered":"2023-10-06T05:27:20.122455+0000","last_clean":"2023-10-06T05:27:20.122455+0000","last_became_active":"2023-10-06T05:27:20.121770+0000","last_became_peered":"2023-10-06T05:27:20.121770+0000","last_unstale":"2023-10-06T05:27:20.122455+0000","last_undegraded":"2023-10-06T05:27:20.122455+0000","last_fullsized":"2023-10-06T05:27:20.122455+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:03.266305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.930285+0000","last_change":"2023-10-06T05:27:20.930285+0000","last_active":"2023-10-06T05:27:20.930285+0000","last_peered":"2023-10-06T05:27:20.930285+0000","last_clean":"2023-10-06T05:27:20.930285+0000","last_became_active":"2023-10-06T05:27:20.929997+0000","last_became_peered":"2023-10-06T05:27:20.929997+0000","last_unstale":"2023-10-06T05:27:20.930285+0000","last_undegraded":"2023-10-06T05:27:20.930285+0000","last_fullsized":"2023-10-06T05:27:20.930285+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:45.872372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115443+0000","last_change":"2023-10-06T05:27:20.112622+0000","last_active":"2023-10-06T05:27:21.115443+0000","last_peered":"2023-10-06T05:27:21.115443+0000","last_clean":"2023-10-06T05:27:21.115443+0000","last_became_active":"2023-10-06T05:27:20.112294+0000","last_became_peered":"2023-10-06T05:27:20.112294+0000","last_unstale":"2023-10-06T05:27:21.115443+0000","last_undegraded":"2023-10-06T05:27:21.115443+0000","last_fullsized":"2023-10-06T05:27:21.115443+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:57.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.939540+0000","last_change":"2023-10-06T05:27:20.939540+0000","last_active":"2023-10-06T05:27:20.939540+0000","last_peered":"2023-10-06T05:27:20.939540+0000","last_clean":"2023-10-06T05:27:20.939540+0000","last_became_active":"2023-10-06T05:27:20.939189+0000","last_became_peered":"2023-10-06T05:27:20.939189+0000","last_unstale":"2023-10-06T05:27:20.939540+0000","last_undegraded":"2023-10-06T05:27:20.939540+0000","last_fullsized":"2023-10-06T05:27:20.939540+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:11.272889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121588+0000","last_change":"2023-10-06T05:27:20.118400+0000","last_active":"2023-10-06T05:27:20.121588+0000","last_peered":"2023-10-06T05:27:20.121588+0000","last_clean":"2023-10-06T05:27:20.121588+0000","last_became_active":"2023-10-06T05:27:20.118135+0000","last_became_peered":"2023-10-06T05:27:20.118135+0000","last_unstale":"2023-10-06T05:27:20.121588+0000","last_undegraded":"2023-10-06T05:27:20.121588+0000","last_fullsized":"2023-10-06T05:27:20.121588+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:18.104805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932275+0000","last_change":"2023-10-06T05:27:20.932275+0000","last_active":"2023-10-06T05:27:20.932275+0000","last_peered":"2023-10-06T05:27:20.932275+0000","last_clean":"2023-10-06T05:27:20.932275+0000","last_became_active":"2023-10-06T05:27:20.931990+0000","last_became_peered":"2023-10-06T05:27:20.931990+0000","last_unstale":"2023-10-06T05:27:20.932275+0000","last_undegraded":"2023-10-06T05:27:20.932275+0000","last_fullsized":"2023-10-06T05:27:20.932275+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:31.612654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.931593+0000","last_change":"2023-10-06T05:27:20.116874+0000","last_active":"2023-10-06T05:27:20.931593+0000","last_peered":"2023-10-06T05:27:20.931593+0000","last_clean":"2023-10-06T05:27:20.931593+0000","last_became_active":"2023-10-06T05:27:20.113196+0000","last_became_peered":"2023-10-06T05:27:20.113196+0000","last_unstale":"2023-10-06T05:27:20.931593+0000","last_undegraded":"2023-10-06T05:27:20.931593+0000","last_fullsized":"2023-10-06T05:27:20.931593+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:11.343731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944257+0000","last_change":"2023-10-06T05:27:20.942355+0000","last_active":"2023-10-06T05:27:20.944257+0000","last_peered":"2023-10-06T05:27:20.944257+0000","last_clean":"2023-10-06T05:27:20.944257+0000","last_became_active":"2023-10-06T05:27:20.941779+0000","last_became_peered":"2023-10-06T05:27:20.941779+0000","last_unstale":"2023-10-06T05:27:20.944257+0000","last_undegraded":"2023-10-06T05:27:20.944257+0000","last_fullsized":"2023-10-06T05:27:20.944257+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:59.357444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.126107+0000","last_change":"2023-10-06T05:27:20.123106+0000","last_active":"2023-10-06T05:27:20.126107+0000","last_peered":"2023-10-06T05:27:20.126107+0000","last_clean":"2023-10-06T05:27:20.126107+0000","last_became_active":"2023-10-06T05:27:20.122803+0000","last_became_peered":"2023-10-06T05:27:20.122803+0000","last_unstale":"2023-10-06T05:27:20.126107+0000","last_undegraded":"2023-10-06T05:27:20.126107+0000","last_fullsized":"2023-10-06T05:27:20.126107+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:51.380550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944131+0000","last_change":"2023-10-06T05:27:20.944131+0000","last_active":"2023-10-06T05:27:20.944131+0000","last_peered":"2023-10-06T05:27:20.944131+0000","last_clean":"2023-10-06T05:27:20.944131+0000","last_became_active":"2023-10-06T05:27:20.943949+0000","last_became_peered":"2023-10-06T05:27:20.943949+0000","last_unstale":"2023-10-06T05:27:20.944131+0000","last_undegraded":"2023-10-06T05:27:20.944131+0000","last_fullsized":"2023-10-06T05:27:20.944131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:52.386090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.113317+0000","last_change":"2023-10-06T05:27:20.123262+0000","last_active":"2023-10-06T05:27:21.113317+0000","last_peered":"2023-10-06T05:27:21.113317+0000","last_clean":"2023-10-06T05:27:21.113317+0000","last_became_active":"2023-10-06T05:27:20.122963+0000","last_became_peered":"2023-10-06T05:27:20.122963+0000","last_unstale":"2023-10-06T05:27:21.113317+0000","last_undegraded":"2023-10-06T05:27:21.113317+0000","last_fullsized":"2023-10-06T05:27:21.113317+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:04.913075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942230+0000","last_change":"2023-10-06T05:27:20.942230+0000","last_active":"2023-10-06T05:27:20.942230+0000","last_peered":"2023-10-06T05:27:20.942230+0000","last_clean":"2023-10-06T05:27:20.942230+0000","last_became_active":"2023-10-06T05:27:20.940755+0000","last_became_peered":"2023-10-06T05:27:20.940755+0000","last_unstale":"2023-10-06T05:27:20.942230+0000","last_undegraded":"2023-10-06T05:27:20.942230+0000","last_fullsized":"2023-10-06T05:27:20.942230+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:16.826861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.110353+0000","last_change":"2023-10-06T05:27:20.108787+0000","last_active":"2023-10-06T05:27:20.110353+0000","last_peered":"2023-10-06T05:27:20.110353+0000","last_clean":"2023-10-06T05:27:20.110353+0000","last_became_active":"2023-10-06T05:27:20.108401+0000","last_became_peered":"2023-10-06T05:27:20.108401+0000","last_unstale":"2023-10-06T05:27:20.110353+0000","last_undegraded":"2023-10-06T05:27:20.110353+0000","last_fullsized":"2023-10-06T05:27:20.110353+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:45.990714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.942247+0000","last_change":"2023-10-06T05:27:20.942247+0000","last_active":"2023-10-06T05:27:20.942247+0000","last_peered":"2023-10-06T05:27:20.942247+0000","last_clean":"2023-10-06T05:27:20.942247+0000","last_became_active":"2023-10-06T05:27:20.940358+0000","last_became_peered":"2023-10-06T05:27:20.940358+0000","last_unstale":"2023-10-06T05:27:20.942247+0000","last_undegraded":"2023-10-06T05:27:20.942247+0000","last_fullsized":"2023-10-06T05:27:20.942247+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:36.907052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.125104+0000","last_change":"2023-10-06T05:27:20.120510+0000","last_active":"2023-10-06T05:27:20.125104+0000","last_peered":"2023-10-06T05:27:20.125104+0000","last_clean":"2023-10-06T05:27:20.125104+0000","last_became_active":"2023-10-06T05:27:20.120171+0000","last_became_peered":"2023-10-06T05:27:20.120171+0000","last_unstale":"2023-10-06T05:27:20.125104+0000","last_undegraded":"2023-10-06T05:27:20.125104+0000","last_fullsized":"2023-10-06T05:27:20.125104+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:35.228330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943351+0000","last_change":"2023-10-06T05:27:20.943351+0000","last_active":"2023-10-06T05:27:20.943351+0000","last_peered":"2023-10-06T05:27:20.943351+0000","last_clean":"2023-10-06T05:27:20.943351+0000","last_became_active":"2023-10-06T05:27:20.943061+0000","last_became_peered":"2023-10-06T05:27:20.943061+0000","last_unstale":"2023-10-06T05:27:20.943351+0000","last_undegraded":"2023-10-06T05:27:20.943351+0000","last_fullsized":"2023-10-06T05:27:20.943351+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:59.357444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.925242+0000","last_change":"2023-10-06T05:27:20.114794+0000","last_active":"2023-10-06T05:27:20.925242+0000","last_peered":"2023-10-06T05:27:20.925242+0000","last_clean":"2023-10-06T05:27:20.925242+0000","last_became_active":"2023-10-06T05:27:20.114503+0000","last_became_peered":"2023-10-06T05:27:20.114503+0000","last_unstale":"2023-10-06T05:27:20.925242+0000","last_undegraded":"2023-10-06T05:27:20.925242+0000","last_fullsized":"2023-10-06T05:27:20.925242+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:45.138057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.938784+0000","last_change":"2023-10-06T05:27:20.931487+0000","last_active":"2023-10-06T05:27:20.938784+0000","last_peered":"2023-10-06T05:27:20.938784+0000","last_clean":"2023-10-06T05:27:20.938784+0000","last_became_active":"2023-10-06T05:27:20.931212+0000","last_became_peered":"2023-10-06T05:27:20.931212+0000","last_unstale":"2023-10-06T05:27:20.938784+0000","last_undegraded":"2023-10-06T05:27:20.938784+0000","last_fullsized":"2023-10-06T05:27:20.938784+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:05.699330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.107715+0000","last_change":"2023-10-06T05:27:20.106451+0000","last_active":"2023-10-06T05:27:20.107715+0000","last_peered":"2023-10-06T05:27:20.107715+0000","last_clean":"2023-10-06T05:27:20.107715+0000","last_became_active":"2023-10-06T05:27:20.106186+0000","last_became_peered":"2023-10-06T05:27:20.106186+0000","last_unstale":"2023-10-06T05:27:20.107715+0000","last_undegraded":"2023-10-06T05:27:20.107715+0000","last_fullsized":"2023-10-06T05:27:20.107715+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:57.913135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.939071+0000","last_change":"2023-10-06T05:27:20.933402+0000","last_active":"2023-10-06T05:27:20.939071+0000","last_peered":"2023-10-06T05:27:20.939071+0000","last_clean":"2023-10-06T05:27:20.939071+0000","last_became_active":"2023-10-06T05:27:20.933098+0000","last_became_peered":"2023-10-06T05:27:20.933098+0000","last_unstale":"2023-10-06T05:27:20.939071+0000","last_undegraded":"2023-10-06T05:27:20.939071+0000","last_fullsized":"2023-10-06T05:27:20.939071+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:10:47.618195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.925597+0000","last_change":"2023-10-06T05:27:20.118479+0000","last_active":"2023-10-06T05:27:20.925597+0000","last_peered":"2023-10-06T05:27:20.925597+0000","last_clean":"2023-10-06T05:27:20.925597+0000","last_became_active":"2023-10-06T05:27:20.118218+0000","last_became_peered":"2023-10-06T05:27:20.118218+0000","last_unstale":"2023-10-06T05:27:20.925597+0000","last_undegraded":"2023-10-06T05:27:20.925597+0000","last_fullsized":"2023-10-06T05:27:20.925597+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:30.607114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.934125+0000","last_change":"2023-10-06T05:27:20.934125+0000","last_active":"2023-10-06T05:27:20.934125+0000","last_peered":"2023-10-06T05:27:20.934125+0000","last_clean":"2023-10-06T05:27:20.934125+0000","last_became_active":"2023-10-06T05:27:20.933784+0000","last_became_peered":"2023-10-06T05:27:20.933784+0000","last_unstale":"2023-10-06T05:27:20.934125+0000","last_undegraded":"2023-10-06T05:27:20.934125+0000","last_fullsized":"2023-10-06T05:27:20.934125+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:17.937389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121680+0000","last_change":"2023-10-06T05:27:20.119179+0000","last_active":"2023-10-06T05:27:20.121680+0000","last_peered":"2023-10-06T05:27:20.121680+0000","last_clean":"2023-10-06T05:27:20.121680+0000","last_became_active":"2023-10-06T05:27:20.118836+0000","last_became_peered":"2023-10-06T05:27:20.118836+0000","last_unstale":"2023-10-06T05:27:20.121680+0000","last_undegraded":"2023-10-06T05:27:20.121680+0000","last_fullsized":"2023-10-06T05:27:20.121680+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:58.309280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.946731+0000","last_change":"2023-10-06T05:27:20.946731+0000","last_active":"2023-10-06T05:27:20.946731+0000","last_peered":"2023-10-06T05:27:20.946731+0000","last_clean":"2023-10-06T05:27:20.946731+0000","last_became_active":"2023-10-06T05:27:20.946489+0000","last_became_peered":"2023-10-06T05:27:20.946489+0000","last_unstale":"2023-10-06T05:27:20.946731+0000","last_undegraded":"2023-10-06T05:27:20.946731+0000","last_fullsized":"2023-10-06T05:27:20.946731+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:58.213731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.126092+0000","last_change":"2023-10-06T05:27:20.119642+0000","last_active":"2023-10-06T05:27:20.126092+0000","last_peered":"2023-10-06T05:27:20.126092+0000","last_clean":"2023-10-06T05:27:20.126092+0000","last_became_active":"2023-10-06T05:27:20.118049+0000","last_became_peered":"2023-10-06T05:27:20.118049+0000","last_unstale":"2023-10-06T05:27:20.126092+0000","last_undegraded":"2023-10-06T05:27:20.126092+0000","last_fullsized":"2023-10-06T05:27:20.126092+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:58.351904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.943490+0000","last_change":"2023-10-06T05:27:20.943490+0000","last_active":"2023-10-06T05:27:20.943490+0000","last_peered":"2023-10-06T05:27:20.943490+0000","last_clean":"2023-10-06T05:27:20.943490+0000","last_became_active":"2023-10-06T05:27:20.943252+0000","last_became_peered":"2023-10-06T05:27:20.943252+0000","last_unstale":"2023-10-06T05:27:20.943490+0000","last_undegraded":"2023-10-06T05:27:20.943490+0000","last_fullsized":"2023-10-06T05:27:20.943490+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:02.277528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.924109+0000","last_change":"2023-10-06T05:27:20.112005+0000","last_active":"2023-10-06T05:27:20.924109+0000","last_peered":"2023-10-06T05:27:20.924109+0000","last_clean":"2023-10-06T05:27:20.924109+0000","last_became_active":"2023-10-06T05:27:20.111732+0000","last_became_peered":"2023-10-06T05:27:20.111732+0000","last_unstale":"2023-10-06T05:27:20.924109+0000","last_undegraded":"2023-10-06T05:27:20.924109+0000","last_fullsized":"2023-10-06T05:27:20.924109+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:59.089399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929778+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:27:20.929778+0000","last_peered":"2023-10-06T05:27:20.929778+0000","last_clean":"2023-10-06T05:27:20.929778+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:27:20.929778+0000","last_undegraded":"2023-10-06T05:27:20.929778+0000","last_fullsized":"2023-10-06T05:27:20.929778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944234+0000","last_change":"2023-10-06T05:27:20.942128+0000","last_active":"2023-10-06T05:27:20.944234+0000","last_peered":"2023-10-06T05:27:20.944234+0000","last_clean":"2023-10-06T05:27:20.944234+0000","last_became_active":"2023-10-06T05:27:20.941338+0000","last_became_peered":"2023-10-06T05:27:20.941338+0000","last_unstale":"2023-10-06T05:27:20.944234+0000","last_undegraded":"2023-10-06T05:27:20.944234+0000","last_fullsized":"2023-10-06T05:27:20.944234+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:19.000419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"142'8","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.116883+0000","last_change":"2023-10-06T05:27:20.124721+0000","last_active":"2023-10-06T05:27:21.116883+0000","last_peered":"2023-10-06T05:27:21.116883+0000","last_clean":"2023-10-06T05:27:21.116883+0000","last_became_active":"2023-10-06T05:27:20.124368+0000","last_became_peered":"2023-10-06T05:27:20.124368+0000","last_unstale":"2023-10-06T05:27:21.116883+0000","last_undegraded":"2023-10-06T05:27:21.116883+0000","last_fullsized":"2023-10-06T05:27:21.116883+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:08.018620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.099217+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:27:20.099217+0000","last_peered":"2023-10-06T05:27:20.099217+0000","last_clean":"2023-10-06T05:27:20.099217+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:27:20.099217+0000","last_undegraded":"2023-10-06T05:27:20.099217+0000","last_fullsized":"2023-10-06T05:27:20.099217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940166+0000","last_change":"2023-10-06T05:27:20.935335+0000","last_active":"2023-10-06T05:27:20.940166+0000","last_peered":"2023-10-06T05:27:20.940166+0000","last_clean":"2023-10-06T05:27:20.940166+0000","last_became_active":"2023-10-06T05:27:20.934906+0000","last_became_peered":"2023-10-06T05:27:20.934906+0000","last_unstale":"2023-10-06T05:27:20.940166+0000","last_undegraded":"2023-10-06T05:27:20.940166+0000","last_fullsized":"2023-10-06T05:27:20.940166+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:01:53.364001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"142'2","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.115987+0000","last_change":"2023-10-06T05:27:20.112960+0000","last_active":"2023-10-06T05:27:21.115987+0000","last_peered":"2023-10-06T05:27:21.115987+0000","last_clean":"2023-10-06T05:27:21.115987+0000","last_became_active":"2023-10-06T05:27:20.112651+0000","last_became_peered":"2023-10-06T05:27:20.112651+0000","last_unstale":"2023-10-06T05:27:21.115987+0000","last_undegraded":"2023-10-06T05:27:21.115987+0000","last_fullsized":"2023-10-06T05:27:21.115987+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:44.096170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":124,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.104141+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:27:20.104141+0000","last_peered":"2023-10-06T05:27:20.104141+0000","last_clean":"2023-10-06T05:27:20.104141+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:27:20.104141+0000","last_undegraded":"2023-10-06T05:27:20.104141+0000","last_fullsized":"2023-10-06T05:27:20.104141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.940717+0000","last_change":"2023-10-06T05:27:20.933807+0000","last_active":"2023-10-06T05:27:20.940717+0000","last_peered":"2023-10-06T05:27:20.940717+0000","last_clean":"2023-10-06T05:27:20.940717+0000","last_became_active":"2023-10-06T05:27:20.933243+0000","last_became_peered":"2023-10-06T05:27:20.933243+0000","last_unstale":"2023-10-06T05:27:20.940717+0000","last_undegraded":"2023-10-06T05:27:20.940717+0000","last_fullsized":"2023-10-06T05:27:20.940717+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:33.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.122625+0000","last_change":"2023-10-06T05:27:20.122080+0000","last_active":"2023-10-06T05:27:20.122625+0000","last_peered":"2023-10-06T05:27:20.122625+0000","last_clean":"2023-10-06T05:27:20.122625+0000","last_became_active":"2023-10-06T05:27:20.121820+0000","last_became_peered":"2023-10-06T05:27:20.121820+0000","last_unstale":"2023-10-06T05:27:20.122625+0000","last_undegraded":"2023-10-06T05:27:20.122625+0000","last_fullsized":"2023-10-06T05:27:20.122625+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:58.480434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.100100+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:27:20.100100+0000","last_peered":"2023-10-06T05:27:20.100100+0000","last_clean":"2023-10-06T05:27:20.100100+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:27:20.100100+0000","last_undegraded":"2023-10-06T05:27:20.100100+0000","last_fullsized":"2023-10-06T05:27:20.100100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.103292+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:27:20.103292+0000","last_peered":"2023-10-06T05:27:20.103292+0000","last_clean":"2023-10-06T05:27:20.103292+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:27:20.103292+0000","last_undegraded":"2023-10-06T05:27:20.103292+0000","last_fullsized":"2023-10-06T05:27:20.103292+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.936901+0000","last_change":"2023-10-06T05:27:20.933054+0000","last_active":"2023-10-06T05:27:20.936901+0000","last_peered":"2023-10-06T05:27:20.936901+0000","last_clean":"2023-10-06T05:27:20.936901+0000","last_became_active":"2023-10-06T05:27:20.932725+0000","last_became_peered":"2023-10-06T05:27:20.932725+0000","last_unstale":"2023-10-06T05:27:20.936901+0000","last_undegraded":"2023-10-06T05:27:20.936901+0000","last_fullsized":"2023-10-06T05:27:20.936901+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:56.775287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121005+0000","last_change":"2023-10-06T05:27:20.120545+0000","last_active":"2023-10-06T05:27:20.121005+0000","last_peered":"2023-10-06T05:27:20.121005+0000","last_clean":"2023-10-06T05:27:20.121005+0000","last_became_active":"2023-10-06T05:27:20.120282+0000","last_became_peered":"2023-10-06T05:27:20.120282+0000","last_unstale":"2023-10-06T05:27:20.121005+0000","last_undegraded":"2023-10-06T05:27:20.121005+0000","last_fullsized":"2023-10-06T05:27:20.121005+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:37.541610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.103377+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:27:20.103377+0000","last_peered":"2023-10-06T05:27:20.103377+0000","last_clean":"2023-10-06T05:27:20.103377+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:27:20.103377+0000","last_undegraded":"2023-10-06T05:27:20.103377+0000","last_fullsized":"2023-10-06T05:27:20.103377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.932722+0000","last_change":"2023-10-06T05:27:20.932722+0000","last_active":"2023-10-06T05:27:20.932722+0000","last_peered":"2023-10-06T05:27:20.932722+0000","last_clean":"2023-10-06T05:27:20.932722+0000","last_became_active":"2023-10-06T05:27:20.932427+0000","last_became_peered":"2023-10-06T05:27:20.932427+0000","last_unstale":"2023-10-06T05:27:20.932722+0000","last_undegraded":"2023-10-06T05:27:20.932722+0000","last_fullsized":"2023-10-06T05:27:20.932722+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:00.601747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.121541+0000","last_change":"2023-10-06T05:27:20.117995+0000","last_active":"2023-10-06T05:27:20.121541+0000","last_peered":"2023-10-06T05:27:20.121541+0000","last_clean":"2023-10-06T05:27:20.121541+0000","last_became_active":"2023-10-06T05:27:20.117697+0000","last_became_peered":"2023-10-06T05:27:20.117697+0000","last_unstale":"2023-10-06T05:27:20.121541+0000","last_undegraded":"2023-10-06T05:27:20.121541+0000","last_fullsized":"2023-10-06T05:27:20.121541+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:45.092670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.103888+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:27:20.103888+0000","last_peered":"2023-10-06T05:27:20.103888+0000","last_clean":"2023-10-06T05:27:20.103888+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:27:20.103888+0000","last_undegraded":"2023-10-06T05:27:20.103888+0000","last_fullsized":"2023-10-06T05:27:20.103888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":8,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.926379+0000","last_change":"2023-10-06T05:27:20.926379+0000","last_active":"2023-10-06T05:27:20.926379+0000","last_peered":"2023-10-06T05:27:20.926379+0000","last_clean":"2023-10-06T05:27:20.926379+0000","last_became_active":"2023-10-06T05:27:20.926136+0000","last_became_peered":"2023-10-06T05:27:20.926136+0000","last_unstale":"2023-10-06T05:27:20.926379+0000","last_undegraded":"2023-10-06T05:27:20.926379+0000","last_fullsized":"2023-10-06T05:27:20.926379+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:24.709473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"142'6","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.113537+0000","last_change":"2023-10-06T05:27:20.125761+0000","last_active":"2023-10-06T05:27:21.113537+0000","last_peered":"2023-10-06T05:27:21.113537+0000","last_clean":"2023-10-06T05:27:21.113537+0000","last_became_active":"2023-10-06T05:27:20.125503+0000","last_became_peered":"2023-10-06T05:27:20.125503+0000","last_unstale":"2023-10-06T05:27:21.113537+0000","last_undegraded":"2023-10-06T05:27:21.113537+0000","last_fullsized":"2023-10-06T05:27:21.113537+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:36.601834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.929545+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:27:20.929545+0000","last_peered":"2023-10-06T05:27:20.929545+0000","last_clean":"2023-10-06T05:27:20.929545+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:27:20.929545+0000","last_undegraded":"2023-10-06T05:27:20.929545+0000","last_fullsized":"2023-10-06T05:27:20.929545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.944220+0000","last_change":"2023-10-06T05:27:20.941557+0000","last_active":"2023-10-06T05:27:20.944220+0000","last_peered":"2023-10-06T05:27:20.944220+0000","last_clean":"2023-10-06T05:27:20.944220+0000","last_became_active":"2023-10-06T05:27:20.941284+0000","last_became_peered":"2023-10-06T05:27:20.941284+0000","last_unstale":"2023-10-06T05:27:20.944220+0000","last_undegraded":"2023-10-06T05:27:20.944220+0000","last_fullsized":"2023-10-06T05:27:20.944220+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:20.088256+0000","last_clean_scrub_stamp":"2023-10-06T05:27:20.088256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:49.163901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"142'3","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-06T05:27:21.112975+0000","last_change":"2023-10-06T05:27:20.118823+0000","last_active":"2023-10-06T05:27:21.112975+0000","last_peered":"2023-10-06T05:27:21.112975+0000","last_clean":"2023-10-06T05:27:21.112975+0000","last_became_active":"2023-10-06T05:27:20.118522+0000","last_became_peered":"2023-10-06T05:27:20.118522+0000","last_unstale":"2023-10-06T05:27:21.112975+0000","last_undegraded":"2023-10-06T05:27:21.112975+0000","last_fullsized":"2023-10-06T05:27:21.112975+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:27:19.082716+0000","last_clean_scrub_stamp":"2023-10-06T05:27:19.082716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:27.995944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":119,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-06T05:27:20.102797+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:27:20.102797+0000","last_peered":"2023-10-06T05:27:20.102797+0000","last_clean":"2023-10-06T05:27:20.102797+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:27:20.102797+0000","last_undegraded":"2023-10-06T05:27:20.102797+0000","last_fullsized":"2023-10-06T05:27:20.102797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307884,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063956,"statfs":{"total":96636764160,"available":96321490944,"internally_reserved":0,"allocated":2461696,"data_stored":1561627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.878},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":1.476},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":1.5129999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":2.0190000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0059999999999998},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58699999999999997,"5min":0.76800000000000002,"15min":0.90300000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.504,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.76800000000000002,"15min":0.93500000000000005},"last":0.442}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68999999999999995,"5min":0.81699999999999995,"15min":0.90000000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.84899999999999998},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.91000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.314},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.92900000000000005},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":1863680,"data_stored":970741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":4.6900000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":0.86799999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.76400000000000001,"15min":5.6239999999999997},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.73899999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":11.595000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":5.5800000000000001},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.93600000000000005},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":23.146000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.73699999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":9.7539999999999996},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.61099999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":9.9969999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.64600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.751,"5min":0.752,"15min":9.5850000000000009},"last":0.434}]},{"osd":5,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.496,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.6439999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":9.8420000000000005},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":9.718},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":9.8930000000000007},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307316,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064524,"statfs":{"total":96636764160,"available":96322072576,"internally_reserved":0,"allocated":1880064,"data_stored":977756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.64400000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.40699999999999997,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.73699999999999999},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.3999999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58199999999999996,"5min":0.67100000000000004,"15min":1.673},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.503},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":4.0490000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77700000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.92600000000000005,"15min":0.94699999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.95199999999999996},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":1.0369999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.44},"max":{"1min":0.871,"5min":0.95599999999999996,"15min":1.167},"last":0.69599999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.94499999999999995,"15min":1.026},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92100000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.66000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307896,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063944,"statfs":{"total":96636764160,"available":96321478656,"internally_reserved":0,"allocated":2473984,"data_stored":1562687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.30599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56799999999999995,"5min":0.82099999999999995,"15min":1.038},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.30299999999999999,"5min":0.27400000000000002,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":1.0609999999999999},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.252},"max":{"1min":0.60799999999999998,"5min":0.67800000000000005,"15min":3.855},"last":0.439},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1919999999999999},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.45200000000000001},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":7.5880000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":7.7670000000000003},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.4039999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.97199999999999998,"15min":2.0870000000000002},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":2.4940000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":5.3600000000000003},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.01,"15min":1.1759999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.45900000000000002,"15min":0.45400000000000001},"max":{"1min":0.874,"5min":0.99199999999999999,"15min":1.0940000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.97899999999999998,"15min":1.0169999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.47499999999999998,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":1.2150000000000001},"last":0.72899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307296,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064544,"statfs":{"total":96636764160,"available":96322093056,"internally_reserved":0,"allocated":1859584,"data_stored":970671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.39500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":2.0720000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.899},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":2.097},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.48699999999999999},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":1.8759999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":1.6719999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.48299999999999998},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":0.70499999999999996,"15min":2.121},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":1.012,"15min":1.7789999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.018,"15min":1.643},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66300000000000003,"15min":0.68999999999999995},"min":{"1min":0.495,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":1.2130000000000001,"15min":2.0129999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.69499999999999995},"min":{"1min":0.51000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.851},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81999999999999995,"5min":0.88300000000000001,"15min":1.571},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.0800000000000001,"15min":2.0449999999999999},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0569999999999999,"15min":1.8180000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.44500000000000001},"max":{"1min":1.0029999999999999,"5min":1.131,"15min":1.613},"last":0.86399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307320,"kb_used_data":1840,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064520,"statfs":{"total":96636764160,"available":96322068480,"internally_reserved":0,"allocated":1884160,"data_stored":977885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26900000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64500000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40799999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.22900000000000001},"max":{"1min":0.63900000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40999999999999998,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61399999999999999,"5min":1.75,"15min":1.75},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60099999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72699999999999998,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.433}]},{"osd":4,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.0469999999999999,"15min":1.143},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":1.855,"15min":1.855},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.43099999999999999},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":1.885,"15min":1.885},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":1.236},"last":0.621},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":1.1950000000000001},"last":0.80300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311404,"kb_used_data":1828,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060436,"statfs":{"total":96636764160,"available":96317886464,"internally_reserved":0,"allocated":1871872,"data_stored":977632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":12.983000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.33400000000000002,"15min":0.33100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":1.7430000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.8080000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":12.911},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.73099999999999998,"5min":0.84299999999999997,"15min":5.0890000000000004},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":12.999000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":13.289999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.95099999999999996,"15min":1.5760000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.88200000000000001,"15min":1.552},"last":0.53300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.54600000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.85699999999999998,"15min":12.949},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.92900000000000005,"15min":5.2549999999999999},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":2.871},"last":0.439},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.91500000000000004,"15min":12.859},"last":0.39000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":1867776,"data_stored":971247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.79400000000000004,"15min":0.82299999999999995},"last":0.5},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.89400000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.82199999999999995,"15min":0.84599999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.81599999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.83299999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.876},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.67600000000000005,"5min":0.85999999999999999,"15min":0.875},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.49199999999999999,"5min":0.432,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.499,"15min":0.48399999999999999},"min":{"1min":0.36499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.92800000000000005},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.317,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.86699999999999999,"15min":0.876},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63}]},{"osd":7,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84899999999999998,"15min":0.89300000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.875,"15min":0.90900000000000003},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:27:26.781 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:27:26.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:27:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.925+0000 7f279e358700 1 Processor -- start 2023-10-06T05:27:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.925+0000 7f279e358700 1 -- start start 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.926+0000 7f279e358700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2798158a60 0x7f279806a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.926+0000 7f279e358700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f279806ee30 0x7f279806ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.926+0000 7f279e358700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2798159a10 con 0x7f2798159ea0 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.926+0000 7f279e358700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27980ac2e0 con 0x7f279806ee30 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.926+0000 7f279e358700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279806f410 con 0x7f2798158a60 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.926+0000 7f279c8f5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f279806ee30 0x7f279806ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.926+0000 7f27977fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2798159ea0 0x7f279806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40140/0 (socket says 172.21.15.45:40140) 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f27977fe700 1 -- 172.21.15.45:0/2160438111 learned_addr learned my addr 172.21.15.45:0/2160438111 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f2797fff700 1 --2- 172.21.15.45:0/2160438111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2798158a60 0x7f279806a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f279c8f5700 1 -- 172.21.15.45:0/2160438111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2798158a60 msgr2=0x7f279806a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f279c8f5700 1 --2- 172.21.15.45:0/2160438111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2798158a60 0x7f279806a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f279c8f5700 1 -- 172.21.15.45:0/2160438111 >> v1:172.21.15.45:6789/0 conn(0x7f2798159ea0 legacy=0x7f279806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f279c8f5700 1 -- 172.21.15.45:0/2160438111 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27981593c0 con 0x7f279806ee30 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f2796ffd700 1 -- 172.21.15.45:0/2160438111 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646478905 0 0) 0x7f2798159a10 con 0x7f2798159ea0 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f279c8f5700 1 --2- 172.21.15.45:0/2160438111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f279806ee30 0x7f279806ad60 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f279000af70 tx=0x7f279000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568895f1e8378d50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f2796ffd700 1 -- 172.21.15.45:0/2160438111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000f070 con 0x7f279806ee30 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f2796ffd700 1 -- 172.21.15.45:0/2160438111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000a2e0 con 0x7f279806ee30 2023-10-06T05:27:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.927+0000 7f2796ffd700 1 -- 172.21.15.45:0/2160438111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000cc70 con 0x7f279806ee30 2023-10-06T05:27:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 -- 172.21.15.45:0/2160438111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f279806ee30 msgr2=0x7f279806ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 --2- 172.21.15.45:0/2160438111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f279806ee30 0x7f279806ad60 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f279000af70 tx=0x7f279000a890 comp rx=0 tx=0).stop 2023-10-06T05:27:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 -- 172.21.15.45:0/2160438111 shutdown_connections 2023-10-06T05:27:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 --2- 172.21.15.45:0/2160438111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f279806ee30 0x7f279806ad60 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 --2- 172.21.15.45:0/2160438111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2798158a60 0x7f279806a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 -- 172.21.15.45:0/2160438111 >> 172.21.15.45:0/2160438111 conn(0x7f27980cc570 msgr2=0x7f27980cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 -- 172.21.15.45:0/2160438111 shutdown_connections 2023-10-06T05:27:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.928+0000 7f279e358700 1 -- 172.21.15.45:0/2160438111 wait complete. 2023-10-06T05:27:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.930+0000 7f279e358700 1 Processor -- start 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.930+0000 7f279e358700 1 -- start start 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279e358700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f279806ee30 0x7f27980c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279e358700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2798158a60 0x7f2798168cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279e358700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2798159ea0 0x7f27980a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279e358700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2798153340 con 0x7f2798159ea0 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279e358700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980bffd0 con 0x7f2798158a60 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279e358700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980a9d00 con 0x7f279806ee30 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2798159ea0 0x7f27980a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2798159ea0 0x7f27980a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39144/0 (socket says 172.21.15.45:39144) 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 -- 172.21.15.45:0/2981847066 learned_addr learned my addr 172.21.15.45:0/2981847066 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f27977fe700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2798158a60 0x7f2798168cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f2797fff700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f279806ee30 0x7f27980c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 -- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f279806ee30 msgr2=0x7f27980c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f279806ee30 0x7f27980c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 -- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2798158a60 msgr2=0x7f2798168cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2798158a60 0x7f2798168cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 -- 172.21.15.45:0/2981847066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279000ab30 con 0x7f2798159ea0 2023-10-06T05:27:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.931+0000 7f279c8f5700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2798159ea0 0x7f27980a88c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f279806fbf0 tx=0x7f2790019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.932+0000 7f277ffff700 1 -- 172.21.15.45:0/2981847066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790009c80 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.932+0000 7f277ffff700 1 -- 172.21.15.45:0/2981847066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2790009de0 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.932+0000 7f277ffff700 1 -- 172.21.15.45:0/2981847066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279001d690 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.932+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2798169200 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.932+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980a93d0 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.933+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279806e300 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.933+0000 7f277ffff700 1 -- 172.21.15.45:0/2981847066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2790003020 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.934+0000 7f277ffff700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2784066f60 0x7f2784069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.934+0000 7f277ffff700 1 -- 172.21.15.45:0/2981847066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2790086520 con 0x7f2798159ea0 2023-10-06T05:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.934+0000 7f2797fff700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2784066f60 0x7f2784069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.935+0000 7f2797fff700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2784066f60 0x7f2784069410 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2788009d40 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:26.939+0000 7f277ffff700 1 -- 172.21.15.45:0/2981847066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27900534f0 con 0x7f2798159ea0 2023-10-06T05:27:27.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.220+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f279804bb40 con 0x7f2798159ea0 2023-10-06T05:27:27.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.225+0000 7f277ffff700 1 -- 172.21.15.45:0/2981847066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2790056b10 con 0x7f2798159ea0 2023-10-06T05:27:27.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2784066f60 msgr2=0x7f2784069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2784066f60 0x7f2784069410 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2788009d40 tx=0x7f2788008040 comp rx=0 tx=0).stop 2023-10-06T05:27:27.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2798159ea0 msgr2=0x7f27980a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2798159ea0 0x7f27980a88c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f279806fbf0 tx=0x7f2790019000 comp rx=0 tx=0).stop 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 shutdown_connections 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2784066f60 0x7f2784069410 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2798159ea0 0x7f27980a88c0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f279806ee30 0x7f27980c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 --2- 172.21.15.45:0/2981847066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2798158a60 0x7f2798168cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 >> 172.21.15.45:0/2981847066 conn(0x7f27980cc570 msgr2=0x7f27980c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 shutdown_connections 2023-10-06T05:27:27.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.227+0000 7f279e358700 1 -- 172.21.15.45:0/2981847066 wait complete. 2023-10-06T05:27:27.231 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-10-06T05:27:27.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:27.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.372+0000 7f568e9ef700 1 Processor -- start 2023-10-06T05:27:27.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.372+0000 7f568e9ef700 1 -- start start 2023-10-06T05:27:27.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f568e9ef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153650 0x7f568815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f568e9ef700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f568806afa0 0x7f5688068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f568e9ef700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880676c0 con 0x7f5688153a20 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f568e9ef700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880679a0 con 0x7f568806afa0 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f568e9ef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5688067c80 con 0x7f5688153650 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f56877fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5688153a20 0x7f568806a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:38420/0 (socket says 172.21.15.45:38420) 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f56877fe700 1 -- 172.21.15.45:0/2379653113 learned_addr learned my addr 172.21.15.45:0/2379653113 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f5687fff700 1 --2- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153650 0x7f568815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f568cf8c700 1 --2- 172.21.15.45:0/2379653113 >> v2:172.21.15.144:3300/0 conn(0x7f568806afa0 0x7f5688068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f5686ffd700 1 -- 172.21.15.45:0/2379653113 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852770399 0 0) 0x7f56880676c0 con 0x7f5688153a20 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.373+0000 7f5686ffd700 1 -- 172.21.15.45:0/2379653113 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5670003570 con 0x7f5688153a20 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f568cf8c700 1 -- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153650 msgr2=0x7f568815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f568cf8c700 1 --2- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153650 0x7f568815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f568cf8c700 1 -- 172.21.15.45:0/2379653113 >> v1:172.21.15.144:6789/0 conn(0x7f5688153a20 legacy=0x7f568806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f568cf8c700 1 -- 172.21.15.45:0/2379653113 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688067f60 con 0x7f568806afa0 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f5687fff700 1 --2- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153650 0x7f568815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f568cf8c700 1 --2- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 0x7f5688068480 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5680009e60 tx=0x7f5680009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b146aef66a02738 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f5686ffd700 1 -- 172.21.15.45:0/2379653113 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568000e070 con 0x7f568806afa0 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f5686ffd700 1 -- 172.21.15.45:0/2379653113 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568000bab0 con 0x7f568806afa0 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f5686ffd700 1 -- 172.21.15.45:0/2379653113 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568000be70 con 0x7f568806afa0 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f568e9ef700 1 -- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 msgr2=0x7f5688068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.374+0000 7f568e9ef700 1 --2- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 0x7f5688068480 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5680009e60 tx=0x7f5680009780 comp rx=0 tx=0).stop 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.375+0000 7f568e9ef700 1 -- 172.21.15.45:0/2379653113 shutdown_connections 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.375+0000 7f568e9ef700 1 --2- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 0x7f5688068480 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.375+0000 7f568e9ef700 1 --2- 172.21.15.45:0/2379653113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153650 0x7f568815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.375+0000 7f568e9ef700 1 -- 172.21.15.45:0/2379653113 >> 172.21.15.45:0/2379653113 conn(0x7f56880cc5a0 msgr2=0x7f56880cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.375+0000 7f568e9ef700 1 -- 172.21.15.45:0/2379653113 shutdown_connections 2023-10-06T05:27:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.375+0000 7f568e9ef700 1 -- 172.21.15.45:0/2379653113 wait complete. 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.376+0000 7f568e9ef700 1 Processor -- start 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.376+0000 7f568e9ef700 1 -- start start 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568e9ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 0x7f5688156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568e9ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688153650 0x7f56880c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568e9ef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153a20 0x7f56880adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568e9ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688154f80 con 0x7f5688153650 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568e9ef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688155de0 con 0x7f568806afa0 2023-10-06T05:27:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568e9ef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56881551d0 con 0x7f5688153a20 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688153650 0x7f56880c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688153650 0x7f56880c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39152/0 (socket says 172.21.15.45:39152) 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 -- 172.21.15.45:0/3446121967 learned_addr learned my addr 172.21.15.45:0/3446121967 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f5687fff700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 0x7f5688156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568cf8c700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153a20 0x7f56880adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 -- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153a20 msgr2=0x7f56880adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153a20 0x7f56880adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 -- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 msgr2=0x7f5688156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 0x7f5688156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f56877fe700 1 -- 172.21.15.45:0/3446121967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c009420 con 0x7f5688153650 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.377+0000 7f568cf8c700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153a20 0x7f56880adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.378+0000 7f56877fe700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688153650 0x7f56880c29e0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f567800c590 tx=0x7f5678007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.378+0000 7f566ffff700 1 -- 172.21.15.45:0/3446121967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678005c00 con 0x7f5688153650 2023-10-06T05:27:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.378+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5680009a20 con 0x7f5688153650 2023-10-06T05:27:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.378+0000 7f566ffff700 1 -- 172.21.15.45:0/3446121967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678005d60 con 0x7f5688153650 2023-10-06T05:27:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.378+0000 7f566ffff700 1 -- 172.21.15.45:0/3446121967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678007300 con 0x7f5688153650 2023-10-06T05:27:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.378+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5688155790 con 0x7f5688153650 2023-10-06T05:27:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.379+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568815c860 con 0x7f5688153650 2023-10-06T05:27:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.379+0000 7f566ffff700 1 -- 172.21.15.45:0/3446121967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f567801b3b0 con 0x7f5688153650 2023-10-06T05:27:27.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.380+0000 7f566ffff700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5674066f30 0x7f56740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:27.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.380+0000 7f566ffff700 1 -- 172.21.15.45:0/3446121967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5678083ce0 con 0x7f5688153650 2023-10-06T05:27:27.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.380+0000 7f5687fff700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5674066f30 0x7f56740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:27.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.381+0000 7f5687fff700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5674066f30 0x7f56740693e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f567c009f40 tx=0x7f567c0091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:27.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.385+0000 7f566ffff700 1 -- 172.21.15.45:0/3446121967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5678050cb0 con 0x7f5688153650 2023-10-06T05:27:27.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.633+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56880cc0e0 con 0x7f5688153650 2023-10-06T05:27:27.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.634+0000 7f566ffff700 1 -- 172.21.15.45:0/3446121967 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 358 v358) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f5678017070 con 0x7f5688153650 2023-10-06T05:27:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:27.636 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":358,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":358,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:24.111609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10842":{"gid":10842,"name":"a","rank":0,"incarnation":354,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6833/2716751458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2716751458},{"type":"v1","addr":"172.21.15.45:6833","nonce":2716751458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:27.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5674066f30 msgr2=0x7f56740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5674066f30 0x7f56740693e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f567c009f40 tx=0x7f567c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688153650 msgr2=0x7f56880c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688153650 0x7f56880c29e0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f567800c590 tx=0x7f5678007dd0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 shutdown_connections 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5674066f30 0x7f56740693e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688153a20 0x7f56880adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568806afa0 0x7f5688156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 --2- 172.21.15.45:0/3446121967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5688153650 0x7f56880c29e0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 >> 172.21.15.45:0/3446121967 conn(0x7f56880cc5a0 msgr2=0x7f568815b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:27.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 shutdown_connections 2023-10-06T05:27:27.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:27.636+0000 7f568e9ef700 1 -- 172.21.15.45:0/3446121967 wait complete. 2023-10-06T05:27:27.640 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 358 2023-10-06T05:27:28.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 358, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:27:20.533957+0000', 'modified': '2023-10-06T05:27:24.111609+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10842}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10842': {'gid': 10842, 'name': 'a', 'rank': 0, 'incarnation': 354, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6833/2716751458', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 2716751458}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 2716751458}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:27:28.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:27:28.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:27:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.196+0000 7f5d92dd2700 1 Processor -- start 2023-10-06T05:27:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.196+0000 7f5d92dd2700 1 -- start start 2023-10-06T05:27:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d92dd2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d8c159170 0x7f5d8c159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d92dd2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c15d040 0x7f5d8c068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d92dd2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c0ca6d0 con 0x7f5d8c069b90 2023-10-06T05:27:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d92dd2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c15d550 con 0x7f5d8c159170 2023-10-06T05:27:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d92dd2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c15d810 con 0x7f5d8c15d040 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d90b6e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d8c159170 0x7f5d8c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d90b6e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d8c159170 0x7f5d8c159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39168/0 (socket says 172.21.15.45:39168) 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3086080296 learned_addr learned my addr 172.21.15.45:0/3086080296 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.197+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/3086080296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c15d040 0x7f5d8c068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3086080296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c15d040 msgr2=0x7f5d8c068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/3086080296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c15d040 0x7f5d8c068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3086080296 >> v1:172.21.15.144:6789/0 conn(0x7f5d8c069b90 legacy=0x7f5d8c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3086080296 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c15daf0 con 0x7f5d8c159170 2023-10-06T05:27:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/3086080296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c159170 0x7f5d8c159540 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0099b0 tx=0x7f5d7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4ba6b6e42f3b2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d8b7fe700 1 -- 172.21.15.45:0/3086080296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c009ea0 con 0x7f5d8c159170 2023-10-06T05:27:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d8b7fe700 1 -- 172.21.15.45:0/3086080296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c00b860 con 0x7f5d8c159170 2023-10-06T05:27:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.198+0000 7f5d8b7fe700 1 -- 172.21.15.45:0/3086080296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00bc00 con 0x7f5d8c159170 2023-10-06T05:27:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 -- 172.21.15.45:0/3086080296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c159170 msgr2=0x7f5d8c159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/3086080296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c159170 0x7f5d8c159540 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0099b0 tx=0x7f5d7c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 -- 172.21.15.45:0/3086080296 shutdown_connections 2023-10-06T05:27:28.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/3086080296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c159170 0x7f5d8c159540 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/3086080296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c15d040 0x7f5d8c068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 -- 172.21.15.45:0/3086080296 >> 172.21.15.45:0/3086080296 conn(0x7f5d8c06f7c0 msgr2=0x7f5d8c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 -- 172.21.15.45:0/3086080296 shutdown_connections 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.199+0000 7f5d92dd2700 1 -- 172.21.15.45:0/3086080296 wait complete. 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.200+0000 7f5d92dd2700 1 Processor -- start 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.200+0000 7f5d92dd2700 1 -- start start 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d92dd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d8c069b90 0x7f5d8c14c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d92dd2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c159170 0x7f5d8c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d92dd2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c15d040 0x7f5d8c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d92dd2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c14cad0 con 0x7f5d8c15d040 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d92dd2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c169660 con 0x7f5d8c069b90 2023-10-06T05:27:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d92dd2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c15b500 con 0x7f5d8c159170 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c15d040 0x7f5d8c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c15d040 0x7f5d8c0cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39182/0 (socket says 172.21.15.45:39182) 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d9136f700 1 -- 172.21.15.45:0/1815035096 learned_addr learned my addr 172.21.15.45:0/1815035096 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d8c069b90 0x7f5d8c14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.201+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c159170 0x7f5d8c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d9136f700 1 -- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c159170 msgr2=0x7f5d8c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d9136f700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c159170 0x7f5d8c0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d9136f700 1 -- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d8c069b90 msgr2=0x7f5d8c14c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d9136f700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d8c069b90 0x7f5d8c14c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d9136f700 1 -- 172.21.15.45:0/1815035096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0094f0 con 0x7f5d8c15d040 2023-10-06T05:27:28.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d8c069b90 0x7f5d8c14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d9136f700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c15d040 0x7f5d8c0cc310 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c8e0 tx=0x7f5d84001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d88ff9700 1 -- 172.21.15.45:0/1815035096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84012630 con 0x7f5d8c15d040 2023-10-06T05:27:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d88ff9700 1 -- 172.21.15.45:0/1815035096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84012790 con 0x7f5d8c15d040 2023-10-06T05:27:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c15b7e0 con 0x7f5d8c15d040 2023-10-06T05:27:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d88ff9700 1 -- 172.21.15.45:0/1815035096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84012b50 con 0x7f5d8c15d040 2023-10-06T05:27:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.202+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c15bda0 con 0x7f5d8c15d040 2023-10-06T05:27:28.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.204+0000 7f5d88ff9700 1 -- 172.21.15.45:0/1815035096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d8401a3a0 con 0x7f5d8c15d040 2023-10-06T05:27:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.204+0000 7f5d88ff9700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d78066f30 0x7f5d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.204+0000 7f5d88ff9700 1 -- 172.21.15.45:0/1815035096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5d84084930 con 0x7f5d8c15d040 2023-10-06T05:27:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.205+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d78066f30 0x7f5d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.205+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d78066f30 0x7f5d780693e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0099b0 tx=0x7f5d7c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.206+0000 7f5d9c0e6b80 1 Processor -- start 2023-10-06T05:27:28.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.206+0000 7f5d9c0e6b80 1 -- start start 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c02455f450 0x55c02455f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c024567f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c0245684d0 0x55c024568940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c024432310 con 0x55c02455f450 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c02446f130 con 0x55c024544f40 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c02446caf0 con 0x55c0245684d0 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d90b6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c02455f450 0x55c02455f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d90b6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c02455f450 0x55c02455f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39192/0 (socket says 172.21.15.45:39192) 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3849012118 learned_addr learned my addr 172.21.15.45:0/3849012118 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c024567f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.207+0000 7f5d9136f700 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c0245684d0 0x55c024568940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c0245684d0 msgr2=0x55c024568940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c0245684d0 0x55c024568940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 msgr2=0x55c024567f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c024567f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d90b6e700 1 -- 172.21.15.45:0/3849012118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c02426e790 con 0x55c02455f450 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c02455f450 0x55c02455f820 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c006fd0 tx=0x7f5d7c01f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b5c720e37228d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d71ffb700 1 -- 172.21.15.45:0/3849012118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00bbe0 con 0x55c02455f450 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d71ffb700 1 -- 172.21.15.45:0/3849012118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c011420 con 0x55c02455f450 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.208+0000 7f5d71ffb700 1 -- 172.21.15.45:0/3849012118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0117a0 con 0x55c02455f450 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/3849012118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c02455f450 msgr2=0x55c02455f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c02455f450 0x55c02455f820 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c006fd0 tx=0x7f5d7c01f050 comp rx=0 tx=0).stop 2023-10-06T05:27:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/3849012118 shutdown_connections 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c0245684d0 0x55c024568940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c02455f450 0x55c02455f820 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/3849012118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c024567f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/3849012118 >> 172.21.15.45:0/3849012118 conn(0x55c024562880 msgr2=0x55c024562c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/3849012118 shutdown_connections 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/3849012118 wait complete. 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.209+0000 7f5d9c0e6b80 1 -- start start 2023-10-06T05:27:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.210+0000 7f5d9c0e6b80 1 Processor -- start 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02455f450 0x55c02453c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0245684d0 0x55c024518590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0244688e0 con 0x55c0245684d0 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c024414980 con 0x55c024544f40 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c024414620 con 0x55c02455f450 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9c0e6b80 10 client.0 _ll_register_callbacks cb 0x55c024277a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0245684d0 0x55c024518590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d90b6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0245684d0 0x55c024518590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39194/0 (socket says 172.21.15.45:39194) 2023-10-06T05:27:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d9136f700 1 -- 172.21.15.45:0/2239324250 learned_addr learned my addr 172.21.15.45:0/2239324250 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.211+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02455f450 0x55c02453c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d90b6e700 1 -- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02455f450 msgr2=0x55c02453c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02455f450 0x55c02453c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d90b6e700 1 -- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0245684d0 msgr2=0x55c024518590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0245684d0 0x55c024518590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d90b6e700 1 -- 172.21.15.45:0/2239324250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c024070 con 0x55c024544f40 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d9136f700 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0245684d0 0x55c024518590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453fdb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c006fd0 tx=0x7f5d7c02a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf44f395b2974ecf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d6affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d6affd700 1 -- 172.21.15.45:0/2239324250 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c011780 con 0x55c024544f40 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c02450cd60 con 0x55c024544f40 2023-10-06T05:27:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d6affd700 1 -- 172.21.15.45:0/2239324250 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5d7c039070 con 0x55c024544f40 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d6affd700 1 -- 172.21.15.45:0/2239324250 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=358) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7c033400 con 0x55c024544f40 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d9c0e6b80 10 client.10432 fetch_fsmap learned FSMap version 358 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.212+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=358,monmap=2+}) v3 -- 0x55c02435b0a0 con 0x55c024544f40 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d6affd700 1 -- 172.21.15.45:0/2239324250 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c02450cd60 con 0x55c024544f40 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d6affd700 1 -- 172.21.15.45:0/2239324250 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 358) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5d7c031030 con 0x55c024544f40 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d9c0e6b80 10 client.10432 fetch_fsmap finished waiting for FSMap version 358 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d9c0e6b80 10 client.10432 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d9c0e6b80 20 client.10432 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c0245609c0 0x55c024560d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d9c0e6b80 4 client.10432 mds_command: new command op to 10842 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 --> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0244e9b30 con 0x55c0245609c0 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c0245609c0 0x55c024560d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c0245609c0 0x55c024560d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b5e94c1af9abbe server_cookie=359e3b06f7f25f52 in_seq=0 out_seq=0 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.213+0000 7f5d6affd700 10 client.10432 ms_handle_connect on v2:172.21.15.45:6832/2716751458 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d6affd700 1 -- 172.21.15.45:0/2239324250 <== mds.0 v2:172.21.15.45:6832/2716751458 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c0244e9b30 con 0x55c0245609c0 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d6affd700 10 client.10432 handle_command_reply: tid=0 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 client.10432 shutdown 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 msgr2=0x55c02453fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453fdb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c006fd0 tx=0x7f5d7c02a000 comp rx=0 tx=0).stop 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 shutdown_connections 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c0245609c0 0x55c024560d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0245684d0 0x55c024518590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d6affd700 0 client.10432 ms_handle_reset on v2:172.21.15.45:6832/2716751458 2023-10-06T05:27:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453fdb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2239324250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02455f450 0x55c02453c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 >> 172.21.15.45:0/2239324250 conn(0x55c024562880 msgr2=0x55c024514c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 shutdown_connections 2023-10-06T05:27:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2239324250 wait complete. 2023-10-06T05:27:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.214+0000 7f5d9c0e6b80 20 client.10432 trim_cache size 0 max 16384 2023-10-06T05:27:28.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.232+0000 7f5d9c0e6b80 1 Processor -- start 2023-10-06T05:27:28.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.232+0000 7f5d9c0e6b80 1 -- start start 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c0245633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024562880 0x55c02453fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024544f40 0x55c024560d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0243001e0 con 0x55c024544f40 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c024515520 con 0x55c024562880 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c024562c50 con 0x55c02453b5b0 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024544f40 0x55c024560d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024544f40 0x55c024560d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39198/0 (socket says 172.21.15.45:39198) 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.233+0000 7f5d9136f700 1 -- 172.21.15.45:0/2901399057 learned_addr learned my addr 172.21.15.45:0/2901399057 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024562880 0x55c02453fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c0245633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d9136f700 1 -- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 msgr2=0x55c0245633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d9136f700 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c0245633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d9136f700 1 -- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024562880 msgr2=0x55c02453fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d9136f700 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024562880 0x55c02453fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d9136f700 1 -- 172.21.15.45:0/2901399057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0244373c0 con 0x55c024544f40 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024562880 0x55c02453fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d9136f700 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024544f40 0x55c024560d90 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f5d84082b60 tx=0x7f5d84084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c379096773f1eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d727fc700 1 -- 172.21.15.45:0/2901399057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840711c0 con 0x55c024544f40 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d727fc700 1 -- 172.21.15.45:0/2901399057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8405c540 con 0x55c024544f40 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.234+0000 7f5d727fc700 1 -- 172.21.15.45:0/2901399057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84053940 con 0x55c024544f40 2023-10-06T05:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2901399057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024544f40 msgr2=0x55c024560d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024544f40 0x55c024560d90 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f5d84082b60 tx=0x7f5d84084040 comp rx=0 tx=0).stop 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2901399057 shutdown_connections 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024544f40 0x55c024560d90 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c0245633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/2901399057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024562880 0x55c02453fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2901399057 >> 172.21.15.45:0/2901399057 conn(0x55c024542430 msgr2=0x55c024567d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2901399057 shutdown_connections 2023-10-06T05:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/2901399057 wait complete. 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.235+0000 7f5d9c0e6b80 1 -- start start 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.236+0000 7f5d9c0e6b80 1 Processor -- start 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c02453e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9c0e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024562880 0x55c0245c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0245c5290 con 0x55c024562880 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c02453c7d0 con 0x55c024544f40 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9c0e6b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c02453d6a0 con 0x55c02453b5b0 2023-10-06T05:27:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9c0e6b80 10 client.0 _ll_register_callbacks cb 0x55c024277a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024562880 0x55c0245c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024562880 0x55c0245c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39200/0 (socket says 172.21.15.45:39200) 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d9136f700 1 -- 172.21.15.45:0/1652789173 learned_addr learned my addr 172.21.15.45:0/1652789173 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c02453e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.237+0000 7f5d8bfff700 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9136f700 1 -- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 msgr2=0x55c02453e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9136f700 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c02453e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9136f700 1 -- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 msgr2=0x55c02453e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9136f700 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9136f700 1 -- 172.21.15.45:0/1652789173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c15bda0 con 0x55c024562880 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9136f700 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024562880 0x55c0245c3ee0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f5d84055530 tx=0x7f5d84070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4d04461776c258 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d6affd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d6affd700 1 -- 172.21.15.45:0/1652789173 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84002230 con 0x55c024562880 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d6affd700 1 -- 172.21.15.45:0/1652789173 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5d84002390 con 0x55c024562880 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d7c011420 con 0x55c024562880 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d6affd700 1 -- 172.21.15.45:0/1652789173 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=358) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d84055a10 con 0x55c024562880 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 10 client.10986 fetch_fsmap learned FSMap version 358 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=358,monmap=2+}) v3 -- 0x55c0242cf1d0 con 0x55c024562880 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d6affd700 1 -- 172.21.15.45:0/1652789173 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c011420 con 0x55c024562880 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d6affd700 1 -- 172.21.15.45:0/1652789173 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 358) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5d84054a90 con 0x55c024562880 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 10 client.10986 fetch_fsmap finished waiting for FSMap version 358 2023-10-06T05:27:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 10 client.10986 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 20 client.10986 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c02455f450 0x55c0245c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 4 client.10986 mds_command: new command op to 10842 tid=0[{"prefix": "status"}] 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.238+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 --> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c02440b8d0 con 0x55c02455f450 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c02455f450 0x55c0245c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d6affd700 10 client.10986 ms_handle_connect on v2:172.21.15.45:6832/2716751458 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d90b6e700 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c02455f450 0x55c0245c4420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52cef1e5d5c8159 server_cookie=de0506a73fdf1725 in_seq=0 out_seq=0 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d6affd700 1 -- 172.21.15.45:0/1652789173 <== mds.0 v2:172.21.15.45:6832/2716751458 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55c02440b8d0 con 0x55c02455f450 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d6affd700 10 client.10986 handle_command_reply: tid=0 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 client.10986 shutdown 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024562880 msgr2=0x55c0245c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024562880 0x55c0245c3ee0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f5d84055530 tx=0x7f5d84070000 comp rx=0 tx=0).stop 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 shutdown_connections 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:6832/2716751458,v1:172.21.15.45:6833/2716751458] conn(0x55c02455f450 0x55c0245c4420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c024562880 0x55c0245c3ee0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c02453b5b0 0x55c02453e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 --2- 172.21.15.45:0/1652789173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c024544f40 0x55c02453e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d6affd700 0 client.10986 ms_handle_reset on v2:172.21.15.45:6832/2716751458 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.239+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 >> 172.21.15.45:0/1652789173 conn(0x55c024542430 msgr2=0x55c024560d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 shutdown_connections 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d9c0e6b80 1 -- 172.21.15.45:0/1652789173 wait complete. 2023-10-06T05:27:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d9c0e6b80 20 client.10986 trim_cache size 0 max 16384 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 10842, 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 7.1406256160000003, 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 358, 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 143, 2023-10-06T05:27:28.248 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 143, 2023-10-06T05:27:28.249 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 15.146757446000001 2023-10-06T05:27:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:27:28.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d78066f30 msgr2=0x7f5d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d78066f30 0x7f5d780693e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0099b0 tx=0x7f5d7c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c15d040 msgr2=0x7f5d8c0cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c15d040 0x7f5d8c0cc310 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c8e0 tx=0x7f5d84001710 comp rx=0 tx=0).stop 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 shutdown_connections 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5d78066f30 0x7f5d780693e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d8c15d040 0x7f5d8c0cc310 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d8c069b90 0x7f5d8c14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 --2- 172.21.15.45:0/1815035096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d8c159170 0x7f5d8c0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 >> 172.21.15.45:0/1815035096 conn(0x7f5d8c06f7c0 msgr2=0x7f5d8c1586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 shutdown_connections 2023-10-06T05:27:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.240+0000 7f5d92dd2700 1 -- 172.21.15.45:0/1815035096 wait complete. 2023-10-06T05:27:28.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.385+0000 7f9aebc2f700 1 Processor -- start 2023-10-06T05:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.385+0000 7f9aebc2f700 1 -- start start 2023-10-06T05:27:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.386+0000 7f9aebc2f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ae4159200 0x7f9ae40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.386+0000 7f9aebc2f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ae41595d0 0x7f9ae406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.386+0000 7f9aebc2f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae406b5c0 0x7f9ae40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.386+0000 7f9aebc2f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae40c7780 con 0x7f9ae4159200 2023-10-06T05:27:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.386+0000 7f9aebc2f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae40c7d00 con 0x7f9ae41595d0 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.386+0000 7f9aebc2f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c86d0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.386+0000 7f9ae99cb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ae4159200 0x7f9ae40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ae4159200 0x7f9ae40c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39204/0 (socket says 172.21.15.45:39204) 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 -- 172.21.15.45:0/1194464663 learned_addr learned my addr 172.21.15.45:0/1194464663 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae91ca700 1 --2- 172.21.15.45:0/1194464663 >> v2:172.21.15.144:3300/0 conn(0x7f9ae41595d0 0x7f9ae406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9aea1cc700 1 --2- 172.21.15.45:0/1194464663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae406b5c0 0x7f9ae40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 -- 172.21.15.45:0/1194464663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae406b5c0 msgr2=0x7f9ae40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 --2- 172.21.15.45:0/1194464663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae406b5c0 0x7f9ae40ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 -- 172.21.15.45:0/1194464663 >> v2:172.21.15.144:3300/0 conn(0x7f9ae41595d0 msgr2=0x7f9ae406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 --2- 172.21.15.45:0/1194464663 >> v2:172.21.15.144:3300/0 conn(0x7f9ae41595d0 0x7f9ae406b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 -- 172.21.15.45:0/1194464663 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae406bb40 con 0x7f9ae4159200 2023-10-06T05:27:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae99cb700 1 --2- 172.21.15.45:0/1194464663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae4159200 0x7f9ae40c8190 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f9ae406e790 tx=0x7f9ae0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a894b472d371f1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae89c9700 1 -- 172.21.15.45:0/1194464663 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0009e70 con 0x7f9ae4159200 2023-10-06T05:27:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae89c9700 1 -- 172.21.15.45:0/1194464663 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae000b860 con 0x7f9ae4159200 2023-10-06T05:27:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.387+0000 7f9ae89c9700 1 -- 172.21.15.45:0/1194464663 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000bc00 con 0x7f9ae4159200 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 -- 172.21.15.45:0/1194464663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae4159200 msgr2=0x7f9ae40c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/1194464663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae4159200 0x7f9ae40c8190 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f9ae406e790 tx=0x7f9ae0009250 comp rx=0 tx=0).stop 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 -- 172.21.15.45:0/1194464663 shutdown_connections 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/1194464663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae406b5c0 0x7f9ae40ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/1194464663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae4159200 0x7f9ae40c8190 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/1194464663 >> v2:172.21.15.144:3300/0 conn(0x7f9ae41595d0 0x7f9ae406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 -- 172.21.15.45:0/1194464663 >> 172.21.15.45:0/1194464663 conn(0x7f9ae406f090 msgr2=0x7f9ae406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 -- 172.21.15.45:0/1194464663 shutdown_connections 2023-10-06T05:27:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.388+0000 7f9aebc2f700 1 -- 172.21.15.45:0/1194464663 wait complete. 2023-10-06T05:27:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.390+0000 7f9aebc2f700 1 Processor -- start 2023-10-06T05:27:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.390+0000 7f9aebc2f700 1 -- start start 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aebc2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae406b5c0 0x7f9ae40cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aebc2f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae4159200 0x7f9ae40c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aebc2f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae41595d0 0x7f9ae40c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aebc2f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c2d70 con 0x7f9ae406b5c0 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aebc2f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40ccbc0 con 0x7f9ae4159200 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aebc2f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae416a000 con 0x7f9ae41595d0 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae406b5c0 0x7f9ae40cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae406b5c0 0x7f9ae40cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39216/0 (socket says 172.21.15.45:39216) 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 -- 172.21.15.45:0/54731070 learned_addr learned my addr 172.21.15.45:0/54731070 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae91ca700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae4159200 0x7f9ae40c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aea1cc700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae41595d0 0x7f9ae40c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 -- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae41595d0 msgr2=0x7f9ae40c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae41595d0 0x7f9ae40c1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 -- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae4159200 msgr2=0x7f9ae40c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae4159200 0x7f9ae40c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 -- 172.21.15.45:0/54731070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae00094f0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9aea1cc700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae41595d0 0x7f9ae40c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.391+0000 7f9ae99cb700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae406b5c0 0x7f9ae40cb370 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f9ae406aed0 tx=0x7f9ae0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.392+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/54731070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000bbe0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.392+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/54731070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae0011420 con 0x7f9ae406b5c0 2023-10-06T05:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.392+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae416dac0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.392+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/54731070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00117c0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.392+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae416e080 con 0x7f9ae406b5c0 2023-10-06T05:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.393+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae406e2f0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.393+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/54731070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ae0011bf0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.394+0000 7f9ad5ffb700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac8066f30 0x7f9ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.394+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/54731070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9ae00861d0 con 0x7f9ae406b5c0 2023-10-06T05:27:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.394+0000 7f9ae91ca700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac8066f30 0x7f9ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.395+0000 7f9ae91ca700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac8066f30 0x7f9ac80693e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40c5f90 tx=0x7f9ad8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:28.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.398+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/54731070 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ae0053100 con 0x7f9ae406b5c0 2023-10-06T05:27:28.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.645+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ae406be20 con 0x7f9ae406b5c0 2023-10-06T05:27:28.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.645+0000 7f9ad5ffb700 1 -- 172.21.15.45:0/54731070 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v358) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ae0056720 con 0x7f9ae406b5c0 2023-10-06T05:27:28.647 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:27:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac8066f30 msgr2=0x7f9ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac8066f30 0x7f9ac80693e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40c5f90 tx=0x7f9ad8008040 comp rx=0 tx=0).stop 2023-10-06T05:27:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae406b5c0 msgr2=0x7f9ae40cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:28.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae406b5c0 0x7f9ae40cb370 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f9ae406aed0 tx=0x7f9ae0017000 comp rx=0 tx=0).stop 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 shutdown_connections 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9ac8066f30 0x7f9ac80693e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ae41595d0 0x7f9ae40c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae406b5c0 0x7f9ae40cb370 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 --2- 172.21.15.45:0/54731070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae4159200 0x7f9ae40c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 >> 172.21.15.45:0/54731070 conn(0x7f9ae406f090 msgr2=0x7f9ae40aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 shutdown_connections 2023-10-06T05:27:28.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:28.647+0000 7f9aebc2f700 1 -- 172.21.15.45:0/54731070 wait complete. 2023-10-06T05:27:28.662 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:27:28.663 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:28.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:28.663 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:27:28.684 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:27:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:27:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:27:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:27:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41693sec preferred_lft 41693sec 2023-10-06T05:27:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:27:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:27:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:28.687 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:27:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:27:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:28.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:28.687 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:27:28.756 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:27:28.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:28.759 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:27:28.825 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:27:28.825 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:27:28.826 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:27:28.826 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:27:28.826 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:27:28.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:28.827 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:27:28.827 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:27:28.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:27:28.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:27:28.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:27:28.828 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:27:28.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:28.953 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:27:28.974 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:27:28.975 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:27:28.975 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:28.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:28.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:27:29.050 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:27:29.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:29.051 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:29.114 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:29.114 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:29.115 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:29.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:27:29.254 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:27:29.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:27:29.329 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:27:29.330 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:27:29.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:29.331 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:27:29.395 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:27:29.395 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:27:29.438 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:27:29.473 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:27:29.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:27:29.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:27:29.537+0000 7fd8000c0540 -1 init, newargv = 0x55af0d319ba0 newargc=15 2023-10-06T05:27:29.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[183539]: starting ceph client 2023-10-06T05:27:29.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[183539]: starting fuse 2023-10-06T05:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:27:29.549 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:27:29.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:29.549 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:27:29.611 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:27:29.611 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:27:29.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:27:29.612 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:27:29.613 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:27:29.614 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:27:29.751 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.183539.asok 2023-10-06T05:27:29.932 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.183539.asok 2023-10-06T05:27:29.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:29.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.183539.asok status 2023-10-06T05:27:30.106 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "183539", 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:27:30.107 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 11001, 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 11001 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3856836427 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:30.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3856836427 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.11001 192.168.0.1:0/3856836427", 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/3856836427", 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 358, 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 143, 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:27:30.109 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:27:30.110 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:27:30.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.117 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:27:30.118 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:30.138 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:27:30.138 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:30.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:30.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.212 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2023-10-06T05:27:30.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.282 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2023-10-06T05:27:30.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.349 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2023-10-06T05:27:30.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.415 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2023-10-06T05:27:30.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.485 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2023-10-06T05:27:30.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.552 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2023-10-06T05:27:30.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:30.624 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2023-10-06T05:27:30.692 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:27:30.692 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:27:30.692 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:27:30.693 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:30.693 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2023-10-06T05:27:30.693 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:27:30.816 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2023-10-06T05:27:31.004 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:27:31.004 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:27:31.004 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:27:31.004 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:31.004 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2023-10-06T05:27:31.004 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:27:31.126 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628279 2023-10-06T05:27:31.315 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:27:31.315 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:27:31.315 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2023-10-06T05:27:31.315 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:31.315 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2023-10-06T05:27:31.315 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:27:31.437 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628278 2023-10-06T05:27:31.625 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:27:31.626 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:27:31.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:31.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[183539]: fuse finished with error 0 and tester_r 0 2023-10-06T05:27:31.710 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T05:27:37.812 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:27:37.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:37.812 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:27:37.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:38.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.015+0000 7f55b04a4700 1 Processor -- start 2023-10-06T05:27:38.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.016+0000 7f55b04a4700 1 -- start start 2023-10-06T05:27:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55b04a4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f55a806c3d0 0x7f55a806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55b04a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a8156b70 0x7f55a8156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55b04a4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a80b0790 con 0x7f55a815b890 2023-10-06T05:27:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55b04a4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a80b0570 con 0x7f55a806c3d0 2023-10-06T05:27:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55b04a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80c8e80 con 0x7f55a8156b70 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55aea41700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a8156b70 0x7f55a8156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55ae240700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f55a806c3d0 0x7f55a806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55ada3f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f55a815b890 0x7f55a80afe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:58282/0 (socket says 172.21.15.45:58282) 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.017+0000 7f55ada3f700 1 -- 172.21.15.45:0/1528371734 learned_addr learned my addr 172.21.15.45:0/1528371734 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55aea41700 1 -- 172.21.15.45:0/1528371734 >> v2:172.21.15.144:3300/0 conn(0x7f55a806c3d0 msgr2=0x7f55a806c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55aea41700 1 --2- 172.21.15.45:0/1528371734 >> v2:172.21.15.144:3300/0 conn(0x7f55a806c3d0 0x7f55a806c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55aea41700 1 -- 172.21.15.45:0/1528371734 >> v1:172.21.15.144:6789/0 conn(0x7f55a815b890 legacy=0x7f55a80afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55aea41700 1 -- 172.21.15.45:0/1528371734 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80c9160 con 0x7f55a8156b70 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55ad23e700 1 -- 172.21.15.45:0/1528371734 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134340937 0 0) 0x7f55a80b0790 con 0x7f55a815b890 2023-10-06T05:27:38.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55ae240700 1 --2- 172.21.15.45:0/1528371734 >> v2:172.21.15.144:3300/0 conn(0x7f55a806c3d0 0x7f55a806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55aea41700 1 --2- 172.21.15.45:0/1528371734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a8156b70 0x7f55a8156fb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009e60 tx=0x7f55a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321ab81727edb257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55ad23e700 1 -- 172.21.15.45:0/1528371734 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000e070 con 0x7f55a8156b70 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55ad23e700 1 -- 172.21.15.45:0/1528371734 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a000bb60 con 0x7f55a8156b70 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.018+0000 7f55ad23e700 1 -- 172.21.15.45:0/1528371734 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0012430 con 0x7f55a8156b70 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 -- 172.21.15.45:0/1528371734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a8156b70 msgr2=0x7f55a8156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 --2- 172.21.15.45:0/1528371734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a8156b70 0x7f55a8156fb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009e60 tx=0x7f55a0009780 comp rx=0 tx=0).stop 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 -- 172.21.15.45:0/1528371734 shutdown_connections 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 --2- 172.21.15.45:0/1528371734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a8156b70 0x7f55a8156fb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 --2- 172.21.15.45:0/1528371734 >> v2:172.21.15.144:3300/0 conn(0x7f55a806c3d0 0x7f55a806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 -- 172.21.15.45:0/1528371734 >> 172.21.15.45:0/1528371734 conn(0x7f55a8067620 msgr2=0x7f55a8067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 -- 172.21.15.45:0/1528371734 shutdown_connections 2023-10-06T05:27:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.019+0000 7f55b04a4700 1 -- 172.21.15.45:0/1528371734 wait complete. 2023-10-06T05:27:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.020+0000 7f55b04a4700 1 Processor -- start 2023-10-06T05:27:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.020+0000 7f55b04a4700 1 -- start start 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55b04a4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a806c3d0 0x7f55a80cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55b04a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55a8156b70 0x7f55a8158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55b04a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a815b890 0x7f55a8155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55b04a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80cea40 con 0x7f55a8156b70 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55b04a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80cdec0 con 0x7f55a815b890 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55b04a4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80ce1a0 con 0x7f55a806c3d0 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55ada3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55a8156b70 0x7f55a8158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55ada3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55a8156b70 0x7f55a8158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41310/0 (socket says 172.21.15.45:41310) 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55ada3f700 1 -- 172.21.15.45:0/2998944323 learned_addr learned my addr 172.21.15.45:0/2998944323 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55ae240700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a806c3d0 0x7f55a80cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.021+0000 7f55aea41700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a815b890 0x7f55a8155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55ada3f700 1 -- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a806c3d0 msgr2=0x7f55a80cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55ada3f700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a806c3d0 0x7f55a80cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55ada3f700 1 -- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a815b890 msgr2=0x7f55a8155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55ada3f700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a815b890 0x7f55a8155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55ada3f700 1 -- 172.21.15.45:0/2998944323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4009420 con 0x7f55a8156b70 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55ada3f700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55a8156b70 0x7f55a8158400 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f559800c7e0 tx=0x7f5598007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f559e7fc700 1 -- 172.21.15.45:0/2998944323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598005b30 con 0x7f55a8156b70 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f559e7fc700 1 -- 172.21.15.45:0/2998944323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598005c90 con 0x7f55a8156b70 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f559e7fc700 1 -- 172.21.15.45:0/2998944323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598007300 con 0x7f55a8156b70 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a0009a20 con 0x7f55a8156b70 2023-10-06T05:27:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.022+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a80ce7e0 con 0x7f55a8156b70 2023-10-06T05:27:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.023+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a80cbb30 con 0x7f55a8156b70 2023-10-06T05:27:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.024+0000 7f559e7fc700 1 -- 172.21.15.45:0/2998944323 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f559801b3b0 con 0x7f55a8156b70 2023-10-06T05:27:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.024+0000 7f559e7fc700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f558c066f30 0x7f558c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.024+0000 7f559e7fc700 1 -- 172.21.15.45:0/2998944323 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5598083cf0 con 0x7f55a8156b70 2023-10-06T05:27:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.024+0000 7f55ae240700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f558c066f30 0x7f558c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.025+0000 7f55ae240700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f558c066f30 0x7f558c0693e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009ea0 tx=0x7f55a40091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.029+0000 7f559e7fc700 1 -- 172.21.15.45:0/2998944323 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55a80cbb30 con 0x7f55a8156b70 2023-10-06T05:27:38.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.277+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55a80cdc50 con 0x7f55a8156b70 2023-10-06T05:27:38.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.278+0000 7f559e7fc700 1 -- 172.21.15.45:0/2998944323 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 358 v358) v1 ==== 95+0+4608 (secure 0 0 0) 0x7f5598017020 con 0x7f55a8156b70 2023-10-06T05:27:38.280 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:38.280 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":358,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":358,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:24.111609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10842":{"gid":10842,"name":"a","rank":0,"incarnation":354,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6833/2716751458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2716751458},{"type":"v1","addr":"172.21.15.45:6833","nonce":2716751458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f558c066f30 msgr2=0x7f558c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f558c066f30 0x7f558c0693e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009ea0 tx=0x7f55a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55a8156b70 msgr2=0x7f55a8158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55a8156b70 0x7f55a8158400 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f559800c7e0 tx=0x7f5598007df0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 shutdown_connections 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f558c066f30 0x7f558c0693e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55a815b890 0x7f55a8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55a806c3d0 0x7f55a80cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 --2- 172.21.15.45:0/2998944323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55a8156b70 0x7f55a8158400 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 >> 172.21.15.45:0/2998944323 conn(0x7f55a8067620 msgr2=0x7f55a80c56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 shutdown_connections 2023-10-06T05:27:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.280+0000 7f55b04a4700 1 -- 172.21.15.45:0/2998944323 wait complete. 2023-10-06T05:27:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 358 2023-10-06T05:27:38.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-10-06T05:27:38.452 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2023-10-06T05:27:38.452 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-06T05:27:38.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:27:38.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.583+0000 7f1c860b6700 1 Processor -- start 2023-10-06T05:27:38.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.583+0000 7f1c860b6700 1 -- start start 2023-10-06T05:27:38.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c860b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c80155cd0 0x7f1c8015a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c860b6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c800adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c860b6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c800ae8d0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c860b6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c800aebb0 con 0x7f1c80155cd0 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c860b6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800aee90 con 0x7f1c801560a0 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c80155cd0 0x7f1c8015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c80155cd0 0x7f1c8015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41314/0 (socket says 172.21.15.45:41314) 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/185643214 learned_addr learned my addr 172.21.15.45:0/185643214 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7effd700 1 --2- 172.21.15.45:0/185643214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c800adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/185643214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 msgr2=0x7f1c800adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7f7fe700 1 --2- 172.21.15.45:0/185643214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c800adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/185643214 >> v1:172.21.15.144:6789/0 conn(0x7f1c800ccdc0 legacy=0x7f1c800c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.584+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/185643214 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800af170 con 0x7f1c80155cd0 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c7f7fe700 1 --2- 172.21.15.45:0/185643214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c80155cd0 0x7f1c8015a730 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f1c700099b0 tx=0x7f1c70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e0c773390cdbb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c7e7fc700 1 -- 172.21.15.45:0/185643214 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70009ea0 con 0x7f1c80155cd0 2023-10-06T05:27:38.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c7e7fc700 1 -- 172.21.15.45:0/185643214 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7000b860 con 0x7f1c80155cd0 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c7e7fc700 1 -- 172.21.15.45:0/185643214 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000bc00 con 0x7f1c80155cd0 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c860b6700 1 -- 172.21.15.45:0/185643214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c80155cd0 msgr2=0x7f1c8015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c860b6700 1 --2- 172.21.15.45:0/185643214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c80155cd0 0x7f1c8015a730 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f1c700099b0 tx=0x7f1c70009250 comp rx=0 tx=0).stop 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c860b6700 1 -- 172.21.15.45:0/185643214 shutdown_connections 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c860b6700 1 --2- 172.21.15.45:0/185643214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c80155cd0 0x7f1c8015a730 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c860b6700 1 --2- 172.21.15.45:0/185643214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c800adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c860b6700 1 -- 172.21.15.45:0/185643214 >> 172.21.15.45:0/185643214 conn(0x7f1c8006f090 msgr2=0x7f1c8006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.585+0000 7f1c860b6700 1 -- 172.21.15.45:0/185643214 shutdown_connections 2023-10-06T05:27:38.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.586+0000 7f1c860b6700 1 -- 172.21.15.45:0/185643214 wait complete. 2023-10-06T05:27:38.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.587+0000 7f1c860b6700 1 Processor -- start 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.587+0000 7f1c860b6700 1 -- start start 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c860b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c800ccdc0 0x7f1c800c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c860b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c80155cd0 0x7f1c800c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c860b6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c801645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c860b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800c51d0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c860b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800c5b60 con 0x7f1c80155cd0 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c860b6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800b2990 con 0x7f1c801560a0 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c800ccdc0 0x7f1c800c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c800ccdc0 0x7f1c800c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41316/0 (socket says 172.21.15.45:41316) 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/560440699 learned_addr learned my addr 172.21.15.45:0/560440699 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7effd700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c80155cd0 0x7f1c800c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7ffff700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c801645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 msgr2=0x7f1c801645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c801645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c80155cd0 msgr2=0x7f1c800c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c80155cd0 0x7f1c800c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 -- 172.21.15.45:0/560440699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c700094f0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7ffff700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c801645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c7f7fe700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c800ccdc0 0x7f1c800c4be0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f1c8006aed0 tx=0x7f1c70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c777fe700 1 -- 172.21.15.45:0/560440699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000bbe0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c777fe700 1 -- 172.21.15.45:0/560440699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c70011420 con 0x7f1c800ccdc0 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.588+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c800c5e00 con 0x7f1c800ccdc0 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.589+0000 7f1c777fe700 1 -- 172.21.15.45:0/560440699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700117c0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.589+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c800c63c0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.590+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8006e2f0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.590+0000 7f1c777fe700 1 -- 172.21.15.45:0/560440699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c70011bf0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.590+0000 7f1c777fe700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c60066f30 0x7f1c600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.591+0000 7f1c777fe700 1 -- 172.21.15.45:0/560440699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1c70085cc0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.591+0000 7f1c7effd700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c60066f30 0x7f1c600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.591+0000 7f1c7effd700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c60066f30 0x7f1c600693e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009d10 tx=0x7f1c680091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.595+0000 7f1c777fe700 1 -- 172.21.15.45:0/560440699 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c70052d10 con 0x7f1c800ccdc0 2023-10-06T05:27:38.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:38.601 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.841+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1c800ae4e0 con 0x7f1c800ccdc0 2023-10-06T05:27:38.844 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:27:38.842+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:27:38.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.846+0000 7f1c777fe700 1 -- 172.21.15.45:0/560440699 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v359) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1c70056330 con 0x7f1c800ccdc0 2023-10-06T05:27:38.851 INFO:tasks.ceph.mds.a.smithi045.stderr: -5047> 2023-10-06T05:27:24.069+0000 7f1bc853e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:38.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c60066f30 msgr2=0x7f1c600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c60066f30 0x7f1c600693e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009d10 tx=0x7f1c680091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c800ccdc0 msgr2=0x7f1c800c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c800ccdc0 0x7f1c800c4be0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f1c8006aed0 tx=0x7f1c70017000 comp rx=0 tx=0).stop 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 shutdown_connections 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1c60066f30 0x7f1c600693e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c801560a0 0x7f1c801645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c800ccdc0 0x7f1c800c4be0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 --2- 172.21.15.45:0/560440699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c80155cd0 0x7f1c800c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 >> 172.21.15.45:0/560440699 conn(0x7f1c8006f090 msgr2=0x7f1c800cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 shutdown_connections 2023-10-06T05:27:38.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.859+0000 7f1c860b6700 1 -- 172.21.15.45:0/560440699 wait complete. 2023-10-06T05:27:38.863 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:27:38.876 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2023-10-06T05:27:38.880 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:27:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.919+0000 7fcd8fc6c280 1 -- start start 2023-10-06T05:27:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.919+0000 7fcd8fc6c280 1 Processor -- start 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72c400 0x560110372580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72c800 0x560110372b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010f72cc00 0x560110373080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f6dc0 con 0x56010f72c400 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f7cc0 con 0x56010f72cc00 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f7b80 con 0x56010f72c800 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8911d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72c400 0x560110372580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8911d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72c400 0x560110372580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41362/0 (socket says 172.21.15.45:41362) 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8911d700 1 -- 172.21.15.45:0/1187645677 learned_addr learned my addr 172.21.15.45:0/1187645677 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010f72cc00 0x560110373080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.920+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72c800 0x560110372b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8911d700 1 -- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72c800 msgr2=0x560110372b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72c800 0x560110372b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8911d700 1 -- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010f72cc00 msgr2=0x560110373080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010f72cc00 0x560110373080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8911d700 1 -- 172.21.15.45:0/1187645677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010f5da340 con 0x56010f72c400 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010f72cc00 0x560110373080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72c400 0x560110372580 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x56010f72e0f0 tx=0x560110342ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5cfe677d113346 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd86918700 1 -- 172.21.15.45:0/1187645677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5ef600 con 0x56010f72c400 2023-10-06T05:27:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd86918700 1 -- 172.21.15.45:0/1187645677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56010f60c600 con 0x56010f72c400 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd86918700 1 -- 172.21.15.45:0/1187645677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5ef1e0 con 0x56010f72c400 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1187645677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010f5da000 con 0x56010f72c400 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1187645677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56010f5daea0 con 0x56010f72c400 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8fc6c280 4 waiting for MDS map... 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd86918700 1 -- 172.21.15.45:0/1187645677 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x56010f5fac40 con 0x56010f72c400 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd86918700 1 -- 172.21.15.45:0/1187645677 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 359) v1 ==== 2559+0+0 (secure 0 0 0) 0x5601103b4600 con 0x56010f72c400 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8fc6c280 4 Got MDS map 359 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.921+0000 7fcd8fc6c280 10 main: JournalTool::main 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.922+0000 7fcd8fc6c280 4 main: JournalTool: connecting to RADOS... 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.922+0000 7fcd8fc6c280 1 Processor -- start 2023-10-06T05:27:38.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.922+0000 7fcd8fc6c280 1 -- start start 2023-10-06T05:27:38.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72dc00 0x560110373600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041e000 0x560110373b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e400 0x560110374100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f63c0 con 0x56011041e000 2023-10-06T05:27:38.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f7cc0 con 0x56011041e400 2023-10-06T05:27:38.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f6000 con 0x56010f72dc00 2023-10-06T05:27:38.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041e000 0x560110373b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041e000 0x560110373b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41364/0 (socket says 172.21.15.45:41364) 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 -- 172.21.15.45:0/4009502599 learned_addr learned my addr 172.21.15.45:0/4009502599 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8891c700 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e400 0x560110374100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8991e700 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72dc00 0x560110373600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 -- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72dc00 msgr2=0x560110373600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72dc00 0x560110373600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 -- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e400 msgr2=0x560110374100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e400 0x560110374100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 -- 172.21.15.45:0/4009502599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010f5db860 con 0x56011041e000 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd8911d700 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041e000 0x560110373b80 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x56010f72f560 tx=0x560110342b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e591e2e204b560 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd84914700 1 -- 172.21.15.45:0/4009502599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5eef20 con 0x56011041e000 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd84914700 1 -- 172.21.15.45:0/4009502599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56010f60cc00 con 0x56011041e000 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.923+0000 7fcd84914700 1 -- 172.21.15.45:0/4009502599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5ef600 con 0x56011041e000 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/4009502599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041e000 msgr2=0x560110373b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041e000 0x560110373b80 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x56010f72f560 tx=0x560110342b40 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/4009502599 shutdown_connections 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e400 0x560110374100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72dc00 0x560110373600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/4009502599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041e000 0x560110373b80 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/4009502599 >> 172.21.15.45:0/4009502599 conn(0x56010f72d000 msgr2=0x56010f6a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/4009502599 shutdown_connections 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/4009502599 wait complete. 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.924+0000 7fcd8fc6c280 1 Processor -- start 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8fc6c280 1 -- start start 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72dc00 0x560110374100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041e400 0x560110374680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e800 0x560110374c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f7180 con 0x56010f72dc00 2023-10-06T05:27:38.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f7040 con 0x56011041e800 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f6f00 con 0x56011041e400 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8991e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72dc00 0x560110374100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8991e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72dc00 0x560110374100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41380/0 (socket says 172.21.15.45:41380) 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8991e700 1 -- 172.21.15.45:0/1916419018 learned_addr learned my addr 172.21.15.45:0/1916419018 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e800 0x560110374c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.925+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041e400 0x560110374680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8991e700 1 -- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041e400 msgr2=0x560110374680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041e400 0x560110374680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8991e700 1 -- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e800 msgr2=0x560110374c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e800 0x560110374c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8991e700 1 -- 172.21.15.45:0/1916419018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010f5daea0 con 0x56010f72dc00 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e800 0x560110374c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72dc00 0x560110374100 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x5601104320c0 tx=0x5601103d2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd83111700 1 -- 172.21.15.45:0/1916419018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5eef20 con 0x56010f72dc00 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd83111700 1 -- 172.21.15.45:0/1916419018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601103a4300 con 0x56010f72dc00 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56010f5dba00 con 0x56010f72dc00 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd83111700 1 -- 172.21.15.45:0/1916419018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5ef600 con 0x56010f72dc00 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010f5daea0 con 0x56010f72dc00 2023-10-06T05:27:38.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.926+0000 7fcd8fc6c280 4 main: JournalTool: resolving pool 16 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.928+0000 7fcd83111700 1 -- 172.21.15.45:0/1916419018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x56011032d080 con 0x56010f72dc00 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.928+0000 7fcd83111700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x560110436400 0x560110375700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.928+0000 7fcd83111700 1 -- 172.21.15.45:0/1916419018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x56010f5faa80 con 0x56010f72dc00 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.928+0000 7fcd8fc6c280 4 main: JournalTool: creating IoCtx.. 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.928+0000 7fcd8fc6c280 4 main: Executing for rank 0 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.928+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x560110436400 0x560110375700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.929+0000 7fcd8fc6c280 1 -- start start 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.929+0000 7fcd8fc6c280 1 Processor -- start 2023-10-06T05:27:38.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.929+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x560110436400 0x560110375700 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x560110337a10 tx=0x5601103d2cc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041f000 0x560110578000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041f400 0x560110578580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8fc6c280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041f800 0x560110578b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f67f040 con 0x56011041f000 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f67f180 con 0x56011041f400 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8fc6c280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010f5f7040 con 0x56011041f800 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041f000 0x560110578000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041f000 0x560110578000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41396/0 (socket says 172.21.15.45:41396) 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 -- 172.21.15.45:0/1323558231 learned_addr learned my addr 172.21.15.45:0/1323558231 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041f400 0x560110578580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041f800 0x560110578b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041f800 msgr2=0x560110578b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041f800 0x560110578b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041f400 msgr2=0x560110578580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041f400 0x560110578580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010f5da340 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.930+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041f000 0x560110578000 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x560110433e00 tx=0x5601103d36e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e71408960c3b72f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8010b700 1 -- 172.21.15.45:0/1323558231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5ee420 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8010b700 1 -- 172.21.15.45:0/1323558231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601103a4180 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8010b700 1 -- 172.21.15.45:0/1323558231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56010f5ee6e0 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010f5da000 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560110471a00 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8fc6c280 4 waiting for MDS map... 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8010b700 1 -- 172.21.15.45:0/1323558231 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x56010f5fa8c0 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8010b700 1 -- 172.21.15.45:0/1323558231 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 359) v1 ==== 2559+0+0 (secure 0 0 0) 0x560110582300 con 0x56011041f000 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8fc6c280 4 Got MDS map 359 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560110437400 0x560110579080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e144) v8 -- 0x560110437800 con 0x560110437400 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.931+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560110437400 0x560110579080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.932+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560110437400 0x560110579080 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=f433e9ca27a7bf04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.934+0000 7fcd8911d700 1 -- 172.21.15.45:0/1323558231 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x560110395200 con 0x560110437400 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.934+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e144) v8 -- 0x560110437c00 con 0x560110437400 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.934+0000 7fcd8911d700 1 -- 172.21.15.45:0/1323558231 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv21 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x560110395200 con 0x560110437400 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.935+0000 7fcd7e908700 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e144) v8 -- 0x56011041e000 con 0x560110437400 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.935+0000 7fcd7e908700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560110437800 0x560110579b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.935+0000 7fcd7e908700 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e144) v8 -- 0x560110437c00 con 0x560110437800 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.935+0000 7fcd8911d700 1 -- 172.21.15.45:0/1323558231 <== osd.0 v2:172.21.15.45:6816/119749 3 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv18 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x560110395200 con 0x560110437400 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.935+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560110437800 0x560110579b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.935+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560110437800 0x560110579b80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=1b3e66154e46e050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.936+0000 7fcd8891c700 1 -- 172.21.15.45:0/1323558231 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560110395440 con 0x560110437800 2023-10-06T05:27:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:old journal was 4219291~892 2023-10-06T05:27:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:new journal start will be 8388608 (4168425 bytes past old end) 2023-10-06T05:27:38.945 INFO:teuthology.orchestra.run.smithi045.stdout:writing journal head 2023-10-06T05:27:38.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.936+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:5 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x5601105a4000 con 0x560110437400 2023-10-06T05:27:38.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8911d700 1 -- 172.21.15.45:0/1323558231 <== osd.0 v2:172.21.15.45:6816/119749 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v144'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x560110395200 con 0x560110437400 2023-10-06T05:27:38.946 INFO:teuthology.orchestra.run.smithi045.stdout:writing EResetJournal entry 2023-10-06T05:27:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:6 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x5601105a4400 con 0x560110437800 2023-10-06T05:27:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5601105a4800 0x560110579600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- osd_op(unknown.0.0:7 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x5601105a4c00 con 0x5601105a4800 2023-10-06T05:27:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:8 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x5601105a5000 con 0x560110437800 2023-10-06T05:27:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x5601105a5400 0x56011057a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- osd_op(unknown.0.0:9 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x5601105a5800 con 0x5601105a5400 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5601105a5c00 0x56011057a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- osd_op(unknown.0.0:10 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x5601105a4400 con 0x5601105a5c00 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5601105a5000 0x56011057ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:11 16.1d 16:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x560110437c00 con 0x5601105a5000 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x5601105a5400 0x56011057a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5601105a4800 0x560110579600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5601105a5c00 0x56011057a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.937+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5601105a5000 0x56011057ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.938+0000 7fcd8911d700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x5601105a5400 0x56011057a100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=3f51fe0e42b59ff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.938+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5601105a5000 0x56011057ac00 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=2b75d86e5dd1cb52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.938+0000 7fcd8991e700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5601105a4800 0x560110579600 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=2cdaa0015713b63e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.939+0000 7fcd8891c700 1 -- 172.21.15.45:0/1323558231 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== osd_op_reply(8 200.00000004 [delete] v144'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560110395440 con 0x560110437800 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.939+0000 7fcd8891c700 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5601105a5c00 0x56011057a680 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=12cb3535eaf8588a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.939+0000 7fcd8911d700 1 -- 172.21.15.45:0/1323558231 <== osd.3 v2:172.21.15.45:6824/119762 1 ==== osd_op_reply(9 200.00000005 [delete] v144'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560110395200 con 0x5601105a5400 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.939+0000 7fcd8991e700 1 -- 172.21.15.45:0/1323558231 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== osd_op_reply(7 200.00000003 [delete] v144'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5601105c2000 con 0x5601105a4800 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.941+0000 7fcd8891c700 1 -- 172.21.15.45:0/1323558231 <== osd.5 v2:172.21.15.144:6816/119917 3 ==== osd_op_reply(6 200.00000002 [delete] v144'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560110395440 con 0x560110437800 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.941+0000 7fcd7e908700 1 -- 172.21.15.45:0/1323558231 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- osd_op(unknown.0.0:12 16.1b 16:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) v8 -- 0x5601105c4000 con 0x560110437800 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.941+0000 7fcd8991e700 1 -- 172.21.15.45:0/1323558231 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(11 200.00000007 [delete] v144'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5601105c2000 con 0x5601105a5000 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.942+0000 7fcd8891c700 1 -- 172.21.15.45:0/1323558231 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== osd_op_reply(10 200.00000006 [delete] v144'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560110395440 con 0x5601105a5c00 2023-10-06T05:27:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8891c700 1 -- 172.21.15.45:0/1323558231 <== osd.5 v2:172.21.15.144:6816/119917 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v144'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x560110395440 con 0x560110437800 2023-10-06T05:27:38.951 INFO:teuthology.orchestra.run.smithi045.stdout:done 2023-10-06T05:27:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560110437400 msgr2=0x560110579080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x560110437400 0x560110579080 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5601105a4800 msgr2=0x560110579600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x5601105a4800 0x560110579600 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5601105a5000 msgr2=0x56011057ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x5601105a5000 0x56011057ac00 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x5601105a5400 msgr2=0x56011057a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x5601105a5400 0x56011057a100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.944+0000 7fcd8891c700 1 -- 172.21.15.45:0/1323558231 reap_dead start 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560110437800 msgr2=0x560110579b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560110437800 0x560110579b80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5601105a5c00 msgr2=0x56011057a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5601105a5c00 0x56011057a680 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041f000 msgr2=0x560110578000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041f000 0x560110578000 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x560110433e00 tx=0x5601103d36e0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 shutdown_connections 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x5601105a5400 0x56011057a100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x560110437800 0x560110579b80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56011041f000 0x560110578000 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1323558231 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x5601105a5c00 0x56011057a680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 >> 172.21.15.45:0/1323558231 conn(0x56011041ec00 msgr2=0x56010f6a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 shutdown_connections 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1323558231 wait complete. 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x560110436400 msgr2=0x560110375700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.945+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x560110436400 0x560110375700 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x560110337a10 tx=0x5601103d2cc0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72dc00 msgr2=0x560110374100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72dc00 0x560110374100 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x5601104320c0 tx=0x5601103d2e40 comp rx=0 tx=0).stop 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 shutdown_connections 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x560110436400 0x560110375700 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56011041e800 0x560110374c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72dc00 0x560110374100 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1916419018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56011041e400 0x560110374680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 >> 172.21.15.45:0/1916419018 conn(0x56010f72d000 msgr2=0x56010f6a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 shutdown_connections 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1916419018 wait complete. 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1187645677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72c400 msgr2=0x560110372580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72c400 0x560110372580 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x56010f72e0f0 tx=0x560110342ea0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1187645677 shutdown_connections 2023-10-06T05:27:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56010f72cc00 0x560110373080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56010f72c400 0x560110372580 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 --2- 172.21.15.45:0/1187645677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56010f72c800 0x560110372b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1187645677 >> 172.21.15.45:0/1187645677 conn(0x56010f72c000 msgr2=0x56010f6a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1187645677 shutdown_connections 2023-10-06T05:27:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:38.946+0000 7fcd8fc6c280 1 -- 172.21.15.45:0/1187645677 wait complete. 2023-10-06T05:27:38.957 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.081675, result: old journal was 4219291~892 new journal start will be 8388608 (4168425 bytes past old end) writing journal head writing EResetJournal entry done 2023-10-06T05:27:38.957 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-06T05:27:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.129+0000 7fa81fb7e700 1 Processor -- start 2023-10-06T05:27:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.129+0000 7fa81fb7e700 1 -- start start 2023-10-06T05:27:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81fb7e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa818157330 0x7fa8180ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81fb7e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa8180ccb70 0x7fa8180c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81fb7e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180c3b50 con 0x7fa8180aaae0 2023-10-06T05:27:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81fb7e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa818157700 con 0x7fa818157330 2023-10-06T05:27:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81fb7e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8180c98e0 con 0x7fa8180ccb70 2023-10-06T05:27:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa818157330 0x7fa8180ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa818157330 0x7fa8180ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41398/0 (socket says 172.21.15.45:41398) 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d119700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa8180aaae0 0x7fa8180c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59514/0 (socket says 172.21.15.45:59514) 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 -- 172.21.15.45:0/783243412 learned_addr learned my addr 172.21.15.45:0/783243412 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81e11b700 1 --2- 172.21.15.45:0/783243412 >> v2:172.21.15.144:3300/0 conn(0x7fa8180ccb70 0x7fa8180c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 -- 172.21.15.45:0/783243412 >> v2:172.21.15.144:3300/0 conn(0x7fa8180ccb70 msgr2=0x7fa8180c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 --2- 172.21.15.45:0/783243412 >> v2:172.21.15.144:3300/0 conn(0x7fa8180ccb70 0x7fa8180c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 -- 172.21.15.45:0/783243412 >> v1:172.21.15.45:6789/0 conn(0x7fa8180aaae0 legacy=0x7fa8180c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 -- 172.21.15.45:0/783243412 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8180c9bc0 con 0x7fa818157330 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81d91a700 1 --2- 172.21.15.45:0/783243412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa8180ce8b0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fa8180cdf40 tx=0x7fa814009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168a72745c970bf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.131+0000 7fa81c918700 1 -- 172.21.15.45:0/783243412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814005c10 con 0x7fa818157330 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81c918700 1 -- 172.21.15.45:0/783243412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814005d70 con 0x7fa818157330 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81c918700 1 -- 172.21.15.45:0/783243412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81400b880 con 0x7fa818157330 2023-10-06T05:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 -- 172.21.15.45:0/783243412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 msgr2=0x7fa8180ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/783243412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa8180ce8b0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fa8180cdf40 tx=0x7fa814009220 comp rx=0 tx=0).stop 2023-10-06T05:27:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 -- 172.21.15.45:0/783243412 shutdown_connections 2023-10-06T05:27:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/783243412 >> v2:172.21.15.144:3300/0 conn(0x7fa8180ccb70 0x7fa8180c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/783243412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa8180ce8b0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 -- 172.21.15.45:0/783243412 >> 172.21.15.45:0/783243412 conn(0x7fa818067e00 msgr2=0x7fa8180cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 -- 172.21.15.45:0/783243412 shutdown_connections 2023-10-06T05:27:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.132+0000 7fa81fb7e700 1 -- 172.21.15.45:0/783243412 wait complete. 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.134+0000 7fa81fb7e700 1 Processor -- start 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.134+0000 7fa81fb7e700 1 -- start start 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81fb7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8180aaae0 0x7fa818069fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81fb7e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8180ccb70 0x7fa81814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81fb7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa818150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81fb7e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818151500 con 0x7fa818157330 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81fb7e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180ac920 con 0x7fa8180aaae0 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81fb7e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81806a580 con 0x7fa8180ccb70 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa818150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa818150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41400/0 (socket says 172.21.15.45:41400) 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 -- 172.21.15.45:0/2368508343 learned_addr learned my addr 172.21.15.45:0/2368508343 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81d91a700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8180aaae0 0x7fa818069fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81d119700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8180ccb70 0x7fa81814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 -- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8180ccb70 msgr2=0x7fa81814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8180ccb70 0x7fa81814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 -- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8180aaae0 msgr2=0x7fa818069fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8180aaae0 0x7fa818069fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 -- 172.21.15.45:0/2368508343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814009480 con 0x7fa818157330 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81d119700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8180ccb70 0x7fa81814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81e11b700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa818150fc0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fa810009fa0 tx=0x7fa810009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa809ffb700 1 -- 172.21.15.45:0/2368508343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81000ed20 con 0x7fa818157330 2023-10-06T05:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa809ffb700 1 -- 172.21.15.45:0/2368508343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81000ee80 con 0x7fa818157330 2023-10-06T05:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180acb70 con 0x7fa818157330 2023-10-06T05:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.135+0000 7fa809ffb700 1 -- 172.21.15.45:0/2368508343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810014670 con 0x7fa818157330 2023-10-06T05:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.136+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8180abcd0 con 0x7fa818157330 2023-10-06T05:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.137+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81806e350 con 0x7fa818157330 2023-10-06T05:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.137+0000 7fa809ffb700 1 -- 172.21.15.45:0/2368508343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8100149e0 con 0x7fa818157330 2023-10-06T05:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.137+0000 7fa809ffb700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7fc066f30 0x7fa7fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.137+0000 7fa809ffb700 1 -- 172.21.15.45:0/2368508343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fa810085010 con 0x7fa818157330 2023-10-06T05:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.138+0000 7fa81d91a700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7fc066f30 0x7fa7fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.138+0000 7fa81d91a700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7fc066f30 0x7fa7fc0693e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa814009260 tx=0x7fa814016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.142+0000 7fa809ffb700 1 -- 172.21.15.45:0/2368508343 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa810051f90 con 0x7fa818157330 2023-10-06T05:27:39.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.393+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa8180c37e0 con 0x7fa818157330 2023-10-06T05:27:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.847+0000 7fa809ffb700 1 -- 172.21.15.45:0/2368508343 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v360) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa8100555b0 con 0x7fa818157330 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7fc066f30 msgr2=0x7fa7fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7fc066f30 0x7fa7fc0693e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa814009260 tx=0x7fa814016000 comp rx=0 tx=0).stop 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 msgr2=0x7fa818150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa818150fc0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fa810009fa0 tx=0x7fa810009900 comp rx=0 tx=0).stop 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 shutdown_connections 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa7fc066f30 0x7fa7fc0693e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa818157330 0x7fa818150fc0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8180aaae0 0x7fa818069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 --2- 172.21.15.45:0/2368508343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8180ccb70 0x7fa81814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.851+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 >> 172.21.15.45:0/2368508343 conn(0x7fa818067e00 msgr2=0x7fa8180cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.852+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 shutdown_connections 2023-10-06T05:27:39.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:39.852+0000 7fa81fb7e700 1 -- 172.21.15.45:0/2368508343 wait complete. 2023-10-06T05:27:39.856 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-06T05:27:39.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:40.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.005+0000 7f328e15a700 1 Processor -- start 2023-10-06T05:27:40.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.005+0000 7f328e15a700 1 -- start start 2023-10-06T05:27:40.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f328e15a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3288153620 0x7f328815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f328e15a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32880ccfc0 0x7f328815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f328e15a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32880ce690 con 0x7f32881539f0 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f328e15a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328815a7a0 con 0x7f3288153620 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f328e15a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32880ce3f0 con 0x7f32880ccfc0 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f32877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3288153620 0x7f328815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f32877fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3288153620 0x7f328815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41406/0 (socket says 172.21.15.45:41406) 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f32877fe700 1 -- 172.21.15.45:0/3460941207 learned_addr learned my addr 172.21.15.45:0/3460941207 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f3287fff700 1 --2- 172.21.15.45:0/3460941207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32880ccfc0 0x7f328815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f32877fe700 1 -- 172.21.15.45:0/3460941207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32880ccfc0 msgr2=0x7f328815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f32877fe700 1 --2- 172.21.15.45:0/3460941207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32880ccfc0 0x7f328815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f32877fe700 1 -- 172.21.15.45:0/3460941207 >> v1:172.21.15.144:6789/0 conn(0x7f32881539f0 legacy=0x7f32880cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f32877fe700 1 -- 172.21.15.45:0/3460941207 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328815d510 con 0x7f3288153620 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.006+0000 7f3287fff700 1 --2- 172.21.15.45:0/3460941207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32880ccfc0 0x7f328815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f32877fe700 1 --2- 172.21.15.45:0/3460941207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3288153620 0x7f328815a260 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f32780099b0 tx=0x7f3278009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a21e56447ab14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f32867fc700 1 -- 172.21.15.45:0/3460941207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800b860 con 0x7f3288153620 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f32867fc700 1 -- 172.21.15.45:0/3460941207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327800b9c0 con 0x7f3288153620 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f32867fc700 1 -- 172.21.15.45:0/3460941207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800bd60 con 0x7f3288153620 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f328e15a700 1 -- 172.21.15.45:0/3460941207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3288153620 msgr2=0x7f328815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f328e15a700 1 --2- 172.21.15.45:0/3460941207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3288153620 0x7f328815a260 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f32780099b0 tx=0x7f3278009250 comp rx=0 tx=0).stop 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f328e15a700 1 -- 172.21.15.45:0/3460941207 shutdown_connections 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f328e15a700 1 --2- 172.21.15.45:0/3460941207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32880ccfc0 0x7f328815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f328e15a700 1 --2- 172.21.15.45:0/3460941207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3288153620 0x7f328815a260 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.007+0000 7f328e15a700 1 -- 172.21.15.45:0/3460941207 >> 172.21.15.45:0/3460941207 conn(0x7f328806ed50 msgr2=0x7f328806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:40.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.008+0000 7f328e15a700 1 -- 172.21.15.45:0/3460941207 shutdown_connections 2023-10-06T05:27:40.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.008+0000 7f328e15a700 1 -- 172.21.15.45:0/3460941207 wait complete. 2023-10-06T05:27:40.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.009+0000 7f328e15a700 1 Processor -- start 2023-10-06T05:27:40.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.009+0000 7f328e15a700 1 -- start start 2023-10-06T05:27:40.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f328e15a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32880ccfc0 0x7f32880c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f328e15a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3288153620 0x7f32880b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f328e15a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328816a570 0x7f3288150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f328e15a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32880b2f80 con 0x7f32880ccfc0 2023-10-06T05:27:40.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f328e15a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32880b5280 con 0x7f328816a570 2023-10-06T05:27:40.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f328e15a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32880b5560 con 0x7f3288153620 2023-10-06T05:27:40.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32880ccfc0 0x7f32880c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32880ccfc0 0x7f32880c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41416/0 (socket says 172.21.15.45:41416) 2023-10-06T05:27:40.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 -- 172.21.15.45:0/1698403801 learned_addr learned my addr 172.21.15.45:0/1698403801 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:40.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f3287fff700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328816a570 0x7f3288150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f3286ffd700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3288153620 0x7f32880b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 -- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3288153620 msgr2=0x7f32880b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3288153620 0x7f32880b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 -- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328816a570 msgr2=0x7f3288150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328816a570 0x7f3288150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 -- 172.21.15.45:0/1698403801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32780094f0 con 0x7f32880ccfc0 2023-10-06T05:27:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f3287fff700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328816a570 0x7f3288150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f32877fe700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32880ccfc0 0x7f32880c83f0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f3278009290 tx=0x7f3278017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.010+0000 7f327f7fe700 1 -- 172.21.15.45:0/1698403801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278024440 con 0x7f32880ccfc0 2023-10-06T05:27:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.011+0000 7f327f7fe700 1 -- 172.21.15.45:0/1698403801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32780245a0 con 0x7f32880ccfc0 2023-10-06T05:27:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.011+0000 7f327f7fe700 1 -- 172.21.15.45:0/1698403801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278024960 con 0x7f32880ccfc0 2023-10-06T05:27:40.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.011+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32880b5840 con 0x7f32880ccfc0 2023-10-06T05:27:40.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.011+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32880b4e90 con 0x7f32880ccfc0 2023-10-06T05:27:40.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.012+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32880c60d0 con 0x7f32880ccfc0 2023-10-06T05:27:40.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.012+0000 7f327f7fe700 1 -- 172.21.15.45:0/1698403801 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f327801a420 con 0x7f32880ccfc0 2023-10-06T05:27:40.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.012+0000 7f327f7fe700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3268066f30 0x7f32680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:40.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.013+0000 7f327f7fe700 1 -- 172.21.15.45:0/1698403801 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f3278086770 con 0x7f32880ccfc0 2023-10-06T05:27:40.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.013+0000 7f3286ffd700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3268066f30 0x7f32680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:40.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.013+0000 7f3286ffd700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3268066f30 0x7f32680693e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f32880afb50 tx=0x7f3270009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:40.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.017+0000 7f327f7fe700 1 -- 172.21.15.45:0/1698403801 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32780536f0 con 0x7f32880ccfc0 2023-10-06T05:27:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.268+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32880b5020 con 0x7f32880ccfc0 2023-10-06T05:27:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.269+0000 7f327f7fe700 1 -- 172.21.15.45:0/1698403801 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 361 v361) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f3278056d10 con 0x7f32880ccfc0 2023-10-06T05:27:40.272 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:40.272 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":361,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":361,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:39.849551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3268066f30 msgr2=0x7f32680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3268066f30 0x7f32680693e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f32880afb50 tx=0x7f3270009040 comp rx=0 tx=0).stop 2023-10-06T05:27:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32880ccfc0 msgr2=0x7f32880c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32880ccfc0 0x7f32880c83f0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f3278009290 tx=0x7f3278017000 comp rx=0 tx=0).stop 2023-10-06T05:27:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 shutdown_connections 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3268066f30 0x7f32680693e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328816a570 0x7f3288150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32880ccfc0 0x7f32880c83f0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 --2- 172.21.15.45:0/1698403801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3288153620 0x7f32880b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 >> 172.21.15.45:0/1698403801 conn(0x7f328806ed50 msgr2=0x7f3288151e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 shutdown_connections 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:40.271+0000 7f328e15a700 1 -- 172.21.15.45:0/1698403801 wait complete. 2023-10-06T05:27:40.275 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 361 2023-10-06T05:27:40.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 361, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:27:20.533957+0000', 'modified': '2023-10-06T05:27:39.849551+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 144, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11019}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11019': {'gid': 11019, 'name': 'a', 'rank': 0, 'incarnation': 361, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6833/4232324777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 4232324777}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 4232324777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:27:40.691 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11019:up:replay 2023-10-06T05:27:41.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:42.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.029+0000 7f2a3814e700 1 Processor -- start 2023-10-06T05:27:42.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.029+0000 7f2a3814e700 1 -- start start 2023-10-06T05:27:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.030+0000 7f2a3814e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a300692d0 0x7f2a30159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.030+0000 7f2a3814e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a300ca6d0 con 0x7f2a3015d040 2023-10-06T05:27:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.030+0000 7f2a3814e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3015d550 con 0x7f2a30159170 2023-10-06T05:27:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.030+0000 7f2a3814e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015d810 con 0x7f2a300692d0 2023-10-06T05:27:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.030+0000 7f2a366eb700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2a3015d040 0x7f2a300aea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59526/0 (socket says 172.21.15.45:59526) 2023-10-06T05:27:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.030+0000 7f2a366eb700 1 -- 172.21.15.45:0/3631167019 learned_addr learned my addr 172.21.15.45:0/3631167019 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a356e9700 1 --2- 172.21.15.45:0/3631167019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a300692d0 0x7f2a30159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a34ee8700 1 -- 172.21.15.45:0/3631167019 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90312997 0 0) 0x7f2a300ca6d0 con 0x7f2a3015d040 2023-10-06T05:27:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a34ee8700 1 -- 172.21.15.45:0/3631167019 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a1c003570 con 0x7f2a3015d040 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a34ee8700 1 -- 172.21.15.45:0/3631167019 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937162982 0 0) 0x7f2a3015d550 con 0x7f2a30159170 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a34ee8700 1 -- 172.21.15.45:0/3631167019 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a300ca6d0 con 0x7f2a30159170 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a356e9700 1 -- 172.21.15.45:0/3631167019 >> v1:172.21.15.144:6789/0 conn(0x7f2a30159170 legacy=0x7f2a30068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a356e9700 1 -- 172.21.15.45:0/3631167019 >> v1:172.21.15.45:6789/0 conn(0x7f2a3015d040 legacy=0x7f2a300aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a356e9700 1 -- 172.21.15.45:0/3631167019 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3015daf0 con 0x7f2a300692d0 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a356e9700 1 --2- 172.21.15.45:0/3631167019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a300692d0 0x7f2a30159540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009910 tx=0x7f2a2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b522984111691f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a34ee8700 1 -- 172.21.15.45:0/3631167019 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c011630 con 0x7f2a300692d0 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a34ee8700 1 -- 172.21.15.45:0/3631167019 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c011790 con 0x7f2a300692d0 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.031+0000 7f2a34ee8700 1 -- 172.21.15.45:0/3631167019 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c011b30 con 0x7f2a300692d0 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.032+0000 7f2a3814e700 1 -- 172.21.15.45:0/3631167019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a300692d0 msgr2=0x7f2a30159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.032+0000 7f2a3814e700 1 --2- 172.21.15.45:0/3631167019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a300692d0 0x7f2a30159540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009910 tx=0x7f2a2c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.032+0000 7f2a3814e700 1 -- 172.21.15.45:0/3631167019 shutdown_connections 2023-10-06T05:27:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.032+0000 7f2a3814e700 1 --2- 172.21.15.45:0/3631167019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a300692d0 0x7f2a30159540 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.032+0000 7f2a3814e700 1 -- 172.21.15.45:0/3631167019 >> 172.21.15.45:0/3631167019 conn(0x7f2a3006f7c0 msgr2=0x7f2a300d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.032+0000 7f2a3814e700 1 -- 172.21.15.45:0/3631167019 shutdown_connections 2023-10-06T05:27:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.032+0000 7f2a3814e700 1 -- 172.21.15.45:0/3631167019 wait complete. 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.033+0000 7f2a3814e700 1 Processor -- start 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.033+0000 7f2a3814e700 1 -- start start 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a3814e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a300692d0 0x7f2a300cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a3814e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a30159170 0x7f2a3016d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a3814e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a3015d040 0x7f2a3016d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a3814e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300ccf90 con 0x7f2a300692d0 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a3814e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300cd270 con 0x7f2a3015d040 2023-10-06T05:27:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a3814e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015a330 con 0x7f2a30159170 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a35eea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a300692d0 0x7f2a300cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a35eea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a300692d0 0x7f2a300cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41426/0 (socket says 172.21.15.45:41426) 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.034+0000 7f2a35eea700 1 -- 172.21.15.45:0/624124819 learned_addr learned my addr 172.21.15.45:0/624124819 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a356e9700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a30159170 0x7f2a3016d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a366eb700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a3015d040 0x7f2a3016d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a35eea700 1 -- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a30159170 msgr2=0x7f2a3016d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a35eea700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a30159170 0x7f2a3016d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a35eea700 1 -- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a3015d040 msgr2=0x7f2a3016d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a35eea700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a3015d040 0x7f2a3016d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a35eea700 1 -- 172.21.15.45:0/624124819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c009420 con 0x7f2a300692d0 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a356e9700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a30159170 0x7f2a3016d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a35eea700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a300692d0 0x7f2a300cc540 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400c790 tx=0x7f2a24007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a21ffb700 1 -- 172.21.15.45:0/624124819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24010510 con 0x7f2a300692d0 2023-10-06T05:27:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a21ffb700 1 -- 172.21.15.45:0/624124819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24010670 con 0x7f2a300692d0 2023-10-06T05:27:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a21ffb700 1 -- 172.21.15.45:0/624124819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24010a10 con 0x7f2a300692d0 2023-10-06T05:27:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3015a610 con 0x7f2a300692d0 2023-10-06T05:27:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.035+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a301509d0 con 0x7f2a300692d0 2023-10-06T05:27:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.036+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3015a190 con 0x7f2a300692d0 2023-10-06T05:27:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.037+0000 7f2a21ffb700 1 -- 172.21.15.45:0/624124819 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a2401b400 con 0x7f2a300692d0 2023-10-06T05:27:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.037+0000 7f2a21ffb700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a14066f30 0x7f2a140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.037+0000 7f2a21ffb700 1 -- 172.21.15.45:0/624124819 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f2a240845f0 con 0x7f2a300692d0 2023-10-06T05:27:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.037+0000 7f2a356e9700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a14066f30 0x7f2a140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.038+0000 7f2a356e9700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a14066f30 0x7f2a140693e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009860 tx=0x7f2a2c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.042+0000 7f2a21ffb700 1 -- 172.21.15.45:0/624124819 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a3015a190 con 0x7f2a300692d0 2023-10-06T05:27:42.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.288+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a3015a190 con 0x7f2a300692d0 2023-10-06T05:27:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.289+0000 7f2a21ffb700 1 -- 172.21.15.45:0/624124819 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 363 v363) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f2a3015a190 con 0x7f2a300692d0 2023-10-06T05:27:43.488 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:43.488 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":363,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":363,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:41.853244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a14066f30 msgr2=0x7f2a140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a14066f30 0x7f2a140693e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009860 tx=0x7f2a2c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a300692d0 msgr2=0x7f2a300cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a300692d0 0x7f2a300cc540 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400c790 tx=0x7f2a24007e10 comp rx=0 tx=0).stop 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 shutdown_connections 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2a14066f30 0x7f2a140693e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a3015d040 0x7f2a3016d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a300692d0 0x7f2a300cc540 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 --2- 172.21.15.45:0/624124819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a30159170 0x7f2a3016d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 >> 172.21.15.45:0/624124819 conn(0x7f2a3006f7c0 msgr2=0x7f2a300aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 shutdown_connections 2023-10-06T05:27:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:42.291+0000 7f2a3814e700 1 -- 172.21.15.45:0/624124819 wait complete. 2023-10-06T05:27:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 363 2023-10-06T05:27:43.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 363, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:27:20.533957+0000', 'modified': '2023-10-06T05:27:41.853244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 144, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11019}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11019': {'gid': 11019, 'name': 'a', 'rank': 0, 'incarnation': 361, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.45:6833/4232324777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 4232324777}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 4232324777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:27:43.896 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11019:up:rejoin 2023-10-06T05:27:44.855 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:27:44.854+0000 7f798a167700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:44.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.038+0000 7fedd70f2700 1 Processor -- start 2023-10-06T05:27:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.038+0000 7fedd70f2700 1 -- start start 2023-10-06T05:27:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.039+0000 7fedd70f2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd0159200 0x7fedd00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.039+0000 7fedd70f2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fedd01595d0 0x7fedd006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.039+0000 7fedd70f2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd00c7d00 con 0x7fedd006b5c0 2023-10-06T05:27:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.039+0000 7fedd70f2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00c7780 con 0x7fedd01595d0 2023-10-06T05:27:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.039+0000 7fedd70f2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c86d0 con 0x7fedd0159200 2023-10-06T05:27:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.039+0000 7fedcffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fedd01595d0 0x7fedd006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fedd01595d0 0x7fedd006b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41438/0 (socket says 172.21.15.45:41438) 2023-10-06T05:27:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedd568f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fedd006b5c0 0x7fedd00ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59532/0 (socket says 172.21.15.45:59532) 2023-10-06T05:27:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcffff700 1 -- 172.21.15.45:0/1422110869 learned_addr learned my addr 172.21.15.45:0/1422110869 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedd4e8e700 1 --2- 172.21.15.45:0/1422110869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd0159200 0x7fedd00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcffff700 1 -- 172.21.15.45:0/1422110869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd0159200 msgr2=0x7fedd00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcffff700 1 --2- 172.21.15.45:0/1422110869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd0159200 0x7fedd00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcffff700 1 -- 172.21.15.45:0/1422110869 >> v1:172.21.15.45:6789/0 conn(0x7fedd006b5c0 legacy=0x7fedd00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcffff700 1 -- 172.21.15.45:0/1422110869 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd006bb40 con 0x7fedd01595d0 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcf7fe700 1 -- 172.21.15.45:0/1422110869 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214609258 0 0) 0x7fedd00c7d00 con 0x7fedd006b5c0 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcffff700 1 --2- 172.21.15.45:0/1422110869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd01595d0 0x7fedd006b080 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fedb80099b0 tx=0x7fedb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aac535c6479d1d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcf7fe700 1 -- 172.21.15.45:0/1422110869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800b860 con 0x7fedd01595d0 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcf7fe700 1 -- 172.21.15.45:0/1422110869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800b9c0 con 0x7fedd01595d0 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.040+0000 7fedcf7fe700 1 -- 172.21.15.45:0/1422110869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800bd60 con 0x7fedd01595d0 2023-10-06T05:27:45.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 -- 172.21.15.45:0/1422110869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd01595d0 msgr2=0x7fedd006b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 --2- 172.21.15.45:0/1422110869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd01595d0 0x7fedd006b080 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fedb80099b0 tx=0x7fedb8009250 comp rx=0 tx=0).stop 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 -- 172.21.15.45:0/1422110869 shutdown_connections 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 --2- 172.21.15.45:0/1422110869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd0159200 0x7fedd00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 --2- 172.21.15.45:0/1422110869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd01595d0 0x7fedd006b080 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 -- 172.21.15.45:0/1422110869 >> 172.21.15.45:0/1422110869 conn(0x7fedd006f090 msgr2=0x7fedd006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 -- 172.21.15.45:0/1422110869 shutdown_connections 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.041+0000 7fedd70f2700 1 -- 172.21.15.45:0/1422110869 wait complete. 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.042+0000 7fedd70f2700 1 Processor -- start 2023-10-06T05:27:45.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.042+0000 7fedd70f2700 1 -- start start 2023-10-06T05:27:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd70f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd006b5c0 0x7fedd016a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd70f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd0159200 0x7fedd0169240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd70f2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd01595d0 0x7fedd00c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd70f2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd016a9f0 con 0x7fedd0159200 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd70f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c58e0 con 0x7fedd006b5c0 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd70f2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c4fb0 con 0x7fedd01595d0 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedcffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd0159200 0x7fedd0169240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedcffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd0159200 0x7fedd0169240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41442/0 (socket says 172.21.15.45:41442) 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedcffff700 1 -- 172.21.15.45:0/3938853858 learned_addr learned my addr 172.21.15.45:0/3938853858 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd4e8e700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd006b5c0 0x7fedd016a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.043+0000 7fedd568f700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd01595d0 0x7fedd00c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedcffff700 1 -- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd01595d0 msgr2=0x7fedd00c11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedcffff700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd01595d0 0x7fedd00c11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedcffff700 1 -- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd006b5c0 msgr2=0x7fedd016a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedcffff700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd006b5c0 0x7fedd016a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedcffff700 1 -- 172.21.15.45:0/3938853858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80094f0 con 0x7fedd0159200 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedcffff700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd0159200 0x7fedd0169240 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fedd006aed0 tx=0x7fedb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedccff9700 1 -- 172.21.15.45:0/3938853858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8011d70 con 0x7fedd0159200 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedccff9700 1 -- 172.21.15.45:0/3938853858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8024440 con 0x7fedd0159200 2023-10-06T05:27:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedccff9700 1 -- 172.21.15.45:0/3938853858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8024800 con 0x7fedd0159200 2023-10-06T05:27:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd00c5290 con 0x7fedd0159200 2023-10-06T05:27:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.044+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd0169780 con 0x7fedd0159200 2023-10-06T05:27:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.045+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd006e2f0 con 0x7fedd0159200 2023-10-06T05:27:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.046+0000 7fedccff9700 1 -- 172.21.15.45:0/3938853858 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedb801a420 con 0x7fedd0159200 2023-10-06T05:27:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.046+0000 7fedccff9700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedb0066f30 0x7fedb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.046+0000 7fedccff9700 1 -- 172.21.15.45:0/3938853858 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fedb8086690 con 0x7fedd0159200 2023-10-06T05:27:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.046+0000 7fedd4e8e700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedb0066f30 0x7fedb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.047+0000 7fedd4e8e700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedb0066f30 0x7fedb00693e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006fd0 tx=0x7fedc0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.051+0000 7fedccff9700 1 -- 172.21.15.45:0/3938853858 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedb8053610 con 0x7fedd0159200 2023-10-06T05:27:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.297+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedd00c5a20 con 0x7fedd0159200 2023-10-06T05:27:45.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.298+0000 7fedccff9700 1 -- 172.21.15.45:0/3938853858 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 364 v364) v1 ==== 95+0+4610 (secure 0 0 0) 0x7fedb8056c30 con 0x7fedd0159200 2023-10-06T05:27:45.300 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:45.300 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":364,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":364,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:42.854757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:45.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedb0066f30 msgr2=0x7fedb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedb0066f30 0x7fedb00693e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006fd0 tx=0x7fedc0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd0159200 msgr2=0x7fedd0169240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd0159200 0x7fedd0169240 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fedd006aed0 tx=0x7fedb8017000 comp rx=0 tx=0).stop 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 shutdown_connections 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fedb0066f30 0x7fedb00693e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd01595d0 0x7fedd00c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd006b5c0 0x7fedd016a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 --2- 172.21.15.45:0/3938853858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedd0159200 0x7fedd0169240 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 >> 172.21.15.45:0/3938853858 conn(0x7fedd006f090 msgr2=0x7fedd00cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 shutdown_connections 2023-10-06T05:27:45.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.300+0000 7fedd70f2700 1 -- 172.21.15.45:0/3938853858 wait complete. 2023-10-06T05:27:45.304 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 364 2023-10-06T05:27:45.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 364, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:27:20.533957+0000', 'modified': '2023-10-06T05:27:42.854757+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 144, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11019}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11019': {'gid': 11019, 'name': 'a', 'rank': 0, 'incarnation': 361, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6833/4232324777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 4232324777}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 4232324777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:27:45.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:27:45.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:27:45.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.859+0000 7f98495df700 1 Processor -- start 2023-10-06T05:27:45.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.859+0000 7f98495df700 1 -- start start 2023-10-06T05:27:45.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98495df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f9844159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98495df700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f984415d040 0x7f9844068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98495df700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9844069b90 0x7f9844069100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98495df700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98440ca6d0 con 0x7f9844069b90 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98495df700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984415d550 con 0x7f984415d040 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98495df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984415d810 con 0x7f9844159170 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9844069b90 0x7f9844069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9844069b90 0x7f9844069100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41448/0 (socket says 172.21.15.45:41448) 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 -- 172.21.15.45:0/215659391 learned_addr learned my addr 172.21.15.45:0/215659391 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98427fc700 1 --2- 172.21.15.45:0/215659391 >> v2:172.21.15.144:3300/0 conn(0x7f984415d040 0x7f9844068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f9842ffd700 1 --2- 172.21.15.45:0/215659391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f9844159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 -- 172.21.15.45:0/215659391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 msgr2=0x7f9844159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 --2- 172.21.15.45:0/215659391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f9844159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 -- 172.21.15.45:0/215659391 >> v2:172.21.15.144:3300/0 conn(0x7f984415d040 msgr2=0x7f9844068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 --2- 172.21.15.45:0/215659391 >> v2:172.21.15.144:3300/0 conn(0x7f984415d040 0x7f9844068bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 -- 172.21.15.45:0/215659391 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984415daf0 con 0x7f9844069b90 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f9842ffd700 1 --2- 172.21.15.45:0/215659391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f9844159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f98437fe700 1 --2- 172.21.15.45:0/215659391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844069100 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f984406a8b0 tx=0x7f983c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb6332abcdb4397 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.860+0000 7f9841ffb700 1 -- 172.21.15.45:0/215659391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00e070 con 0x7f9844069b90 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f9841ffb700 1 -- 172.21.15.45:0/215659391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c00ba20 con 0x7f9844069b90 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f9841ffb700 1 -- 172.21.15.45:0/215659391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0092e0 con 0x7f9844069b90 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 -- 172.21.15.45:0/215659391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 msgr2=0x7f9844069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 --2- 172.21.15.45:0/215659391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844069100 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f984406a8b0 tx=0x7f983c009840 comp rx=0 tx=0).stop 2023-10-06T05:27:45.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 -- 172.21.15.45:0/215659391 shutdown_connections 2023-10-06T05:27:45.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 --2- 172.21.15.45:0/215659391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844069100 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 --2- 172.21.15.45:0/215659391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f9844159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 --2- 172.21.15.45:0/215659391 >> v2:172.21.15.144:3300/0 conn(0x7f984415d040 0x7f9844068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 -- 172.21.15.45:0/215659391 >> 172.21.15.45:0/215659391 conn(0x7f984406f7c0 msgr2=0x7f98440d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 -- 172.21.15.45:0/215659391 shutdown_connections 2023-10-06T05:27:45.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.861+0000 7f98495df700 1 -- 172.21.15.45:0/215659391 wait complete. 2023-10-06T05:27:45.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 Processor -- start 2023-10-06T05:27:45.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 -- start start 2023-10-06T05:27:45.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f98440c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f984415d040 0x7f98440b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98440c7140 con 0x7f9844069b90 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98440c7420 con 0x7f984415d040 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.863+0000 7f98495df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98440c7700 con 0x7f9844159170 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41464/0 (socket says 172.21.15.45:41464) 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 -- 172.21.15.45:0/139342542 learned_addr learned my addr 172.21.15.45:0/139342542 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f98437fe700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f984415d040 0x7f98440b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f98427fc700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f98440c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 -- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 msgr2=0x7f98440c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f98440c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 -- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f984415d040 msgr2=0x7f98440b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f984415d040 0x7f98440b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 -- 172.21.15.45:0/139342542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c009ae0 con 0x7f9844069b90 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f98427fc700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f98440c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f9842ffd700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844158880 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f983400e8d0 tx=0x7f9834008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f982affd700 1 -- 172.21.15.45:0/139342542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400ee70 con 0x7f9844069b90 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f982affd700 1 -- 172.21.15.45:0/139342542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983400c5e0 con 0x7f9844069b90 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98440c79e0 con 0x7f9844069b90 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f982affd700 1 -- 172.21.15.45:0/139342542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400c980 con 0x7f9844069b90 2023-10-06T05:27:45.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.864+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98440c96e0 con 0x7f9844069b90 2023-10-06T05:27:45.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.866+0000 7f982affd700 1 -- 172.21.15.45:0/139342542 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9834013410 con 0x7f9844069b90 2023-10-06T05:27:45.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.867+0000 7f982affd700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9830066f30 0x7f98300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.867+0000 7f982affd700 1 -- 172.21.15.45:0/139342542 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f9834084610 con 0x7f9844069b90 2023-10-06T05:27:45.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.867+0000 7f98427fc700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9830066f30 0x7f98300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.867+0000 7f98427fc700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9830066f30 0x7f98300693e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f98440cecd0 tx=0x7f9838008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.869+0000 7f98528f3b80 1 Processor -- start 2023-10-06T05:27:45.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.869+0000 7f98528f3b80 1 -- start start 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.869+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456482820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45648af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45648b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98528f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456355310 con 0x55c456467f40 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456392130 con 0x55c45648b4d0 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45638faf0 con 0x55c456482450 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45648af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45648af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41474/0 (socket says 172.21.15.45:41474) 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 -- 172.21.15.45:0/3110492911 learned_addr learned my addr 172.21.15.45:0/3110492911 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98437fe700 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45648b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f9842ffd700 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456482820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 -- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 msgr2=0x55c456482820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456482820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 -- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 msgr2=0x55c45648b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45648b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 -- 172.21.15.45:0/3110492911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c456191790 con 0x55c456467f40 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f98427fc700 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45648af90 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f9838011d50 tx=0x7f9838018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7e162a0bcd9b79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f9823fff700 1 -- 172.21.15.45:0/3110492911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838011d80 con 0x55c456467f40 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f9823fff700 1 -- 172.21.15.45:0/3110492911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9838015bb0 con 0x55c456467f40 2023-10-06T05:27:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.870+0000 7f9823fff700 1 -- 172.21.15.45:0/3110492911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838011ee0 con 0x55c456467f40 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 -- 172.21.15.45:0/3110492911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 msgr2=0x55c45648af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45648af90 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f9838011d50 tx=0x7f9838018000 comp rx=0 tx=0).stop 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 -- 172.21.15.45:0/3110492911 shutdown_connections 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45648b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456482820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3110492911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45648af90 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 -- 172.21.15.45:0/3110492911 >> 172.21.15.45:0/3110492911 conn(0x55c456485880 msgr2=0x55c456485c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 -- 172.21.15.45:0/3110492911 shutdown_connections 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.871+0000 7f98528f3b80 1 -- 172.21.15.45:0/3110492911 wait complete. 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.872+0000 7f98528f3b80 1 -- start start 2023-10-06T05:27:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.872+0000 7f98528f3b80 1 Processor -- start 2023-10-06T05:27:45.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456467f40 0x55c45645c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456482450 0x55c45643cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45643d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98528f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45638b8e0 con 0x55c456482450 2023-10-06T05:27:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456337980 con 0x55c45648b4d0 2023-10-06T05:27:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c456337620 con 0x55c456467f40 2023-10-06T05:27:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98528f3b80 10 client.0 _ll_register_callbacks cb 0x55c45619aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456482450 0x55c45643cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456482450 0x55c45643cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41478/0 (socket says 172.21.15.45:41478) 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 -- 172.21.15.45:0/4058450475 learned_addr learned my addr 172.21.15.45:0/4058450475 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98437fe700 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45643d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f9842ffd700 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456467f40 0x55c45645c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 -- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456467f40 msgr2=0x55c45645c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456467f40 0x55c45645c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 -- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 msgr2=0x55c45643d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45643d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.874+0000 7f98427fc700 1 -- 172.21.15.45:0/4058450475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9838011a40 con 0x55c456482450 2023-10-06T05:27:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98427fc700 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456482450 0x55c45643cd30 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f9838022000 tx=0x7f983801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bc3a619a4c3069 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98217fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98217fa700 1 -- 172.21.15.45:0/4058450475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838011d80 con 0x55c456482450 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98217fa700 1 -- 172.21.15.45:0/4058450475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f9838031070 con 0x55c456482450 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c45642fd60 con 0x55c456482450 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98217fa700 1 -- 172.21.15.45:0/4058450475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9838011ee0 con 0x55c456482450 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 10 client.11076 fetch_fsmap learned FSMap version 364 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) v3 -- 0x55c45627e0a0 con 0x55c456482450 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98217fa700 1 -- 172.21.15.45:0/4058450475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c45642fd60 con 0x55c456482450 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98217fa700 1 -- 172.21.15.45:0/4058450475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 364) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9838036480 con 0x55c456482450 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 10 client.11076 fetch_fsmap finished waiting for FSMap version 364 2023-10-06T05:27:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 10 client.11076 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 20 client.11076 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645f080 0x55c4564621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 4 client.11076 mds_command: new command op to 11019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c45640cb30 con 0x55c45645f080 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.875+0000 7f9842ffd700 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645f080 0x55c4564621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.876+0000 7f98217fa700 10 client.11076 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.876+0000 7f9842ffd700 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645f080 0x55c4564621d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25bf525fd0a02b7 server_cookie=4c85d8331c53ecd5 in_seq=0 out_seq=0 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.876+0000 7f98217fa700 1 -- 172.21.15.45:0/4058450475 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c45640cb30 con 0x55c45645f080 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.876+0000 7f98217fa700 10 client.11076 handle_command_reply: tid=0 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.876+0000 7f98528f3b80 1 client.11076 shutdown 2023-10-06T05:27:45.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456482450 msgr2=0x55c45643cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456482450 0x55c45643cd30 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f9838022000 tx=0x7f983801b000 comp rx=0 tx=0).stop 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 shutdown_connections 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645f080 0x55c4564621d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c45643d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98217fa700 0 client.11076 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456467f40 0x55c45645c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 --2- 172.21.15.45:0/4058450475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456482450 0x55c45643cd30 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 >> 172.21.15.45:0/4058450475 conn(0x55c456485880 msgr2=0x55c456437c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 shutdown_connections 2023-10-06T05:27:45.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 1 -- 172.21.15.45:0/4058450475 wait complete. 2023-10-06T05:27:45.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.877+0000 7f98528f3b80 20 client.11076 trim_cache size 0 max 16384 2023-10-06T05:27:45.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 Processor -- start 2023-10-06T05:27:45.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 -- start start 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45645dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456462630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c456460a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45645e9c0 con 0x55c456467f40 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4562231e0 con 0x55c45648b4d0 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.895+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45643e240 con 0x55c456482450 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45645dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45645dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41482/0 (socket says 172.21.15.45:41482) 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 -- 172.21.15.45:0/3142417345 learned_addr learned my addr 172.21.15.45:0/3142417345 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f98437fe700 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c456460a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f98427fc700 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456462630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 -- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 msgr2=0x55c456462630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456462630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 -- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 msgr2=0x55c456460a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c456460a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 -- 172.21.15.45:0/3142417345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45635a3c0 con 0x55c456467f40 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9842ffd700 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45645dc30 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f9834054f50 tx=0x7f983401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1261c3e157a9669 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9828ff9700 1 -- 172.21.15.45:0/3142417345 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340584a0 con 0x55c456467f40 2023-10-06T05:27:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9828ff9700 1 -- 172.21.15.45:0/3142417345 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983405c1f0 con 0x55c456467f40 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.896+0000 7f9828ff9700 1 -- 172.21.15.45:0/3142417345 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834055250 con 0x55c456467f40 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 -- 172.21.15.45:0/3142417345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 msgr2=0x55c45645dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45645dc30 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f9834054f50 tx=0x7f983401d040 comp rx=0 tx=0).stop 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 -- 172.21.15.45:0/3142417345 shutdown_connections 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c45648b4d0 0x55c456460a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45645dc30 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 --2- 172.21.15.45:0/3142417345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c456482450 0x55c456462630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 -- 172.21.15.45:0/3142417345 >> 172.21.15.45:0/3142417345 conn(0x55c456485880 msgr2=0x55c456484c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 -- 172.21.15.45:0/3142417345 shutdown_connections 2023-10-06T05:27:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.897+0000 7f98528f3b80 1 -- 172.21.15.45:0/3142417345 wait complete. 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.898+0000 7f98528f3b80 1 -- start start 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.898+0000 7f98528f3b80 1 Processor -- start 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45643d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c456482450 0x55c45645f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98528f3b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c45648b4d0 0x55c45645f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98528f3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c456464d70 con 0x55c456467f40 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4564e8b10 con 0x55c456482450 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98528f3b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c456464b20 con 0x55c45648b4d0 2023-10-06T05:27:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98528f3b80 10 client.0 _ll_register_callbacks cb 0x55c45619aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f9842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45643d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f9842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45643d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41498/0 (socket says 172.21.15.45:41498) 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f9842ffd700 1 -- 172.21.15.45:0/2507096275 learned_addr learned my addr 172.21.15.45:0/2507096275 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.899+0000 7f98427fc700 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c456482450 0x55c45645f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98437fe700 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c45648b4d0 0x55c45645f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f9842ffd700 1 -- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c45648b4d0 msgr2=0x55c45645f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f9842ffd700 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c45648b4d0 0x55c45645f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f9842ffd700 1 -- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c456482450 msgr2=0x55c45645f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f9842ffd700 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c456482450 0x55c45645f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f9842ffd700 1 -- 172.21.15.45:0/2507096275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98440c96e0 con 0x55c456467f40 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f9842ffd700 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45643d780 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f983400efd0 tx=0x7f9834015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eab3ef3feb97cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98217fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98217fa700 1 -- 172.21.15.45:0/2507096275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340584a0 con 0x55c456467f40 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98217fa700 1 -- 172.21.15.45:0/2507096275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f9834055250 con 0x55c456467f40 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9838015bb0 con 0x55c456467f40 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98217fa700 1 -- 172.21.15.45:0/2507096275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9834054950 con 0x55c456467f40 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98528f3b80 10 client.11082 fetch_fsmap learned FSMap version 364 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) v3 -- 0x55c4561f21d0 con 0x55c456467f40 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98217fa700 1 -- 172.21.15.45:0/2507096275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9838015bb0 con 0x55c456467f40 2023-10-06T05:27:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.900+0000 7f98217fa700 1 -- 172.21.15.45:0/2507096275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 364) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f983405bd60 con 0x55c456467f40 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 10 client.11082 fetch_fsmap finished waiting for FSMap version 364 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 10 client.11082 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 20 client.11082 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645fed0 0x55c4564eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 4 client.11082 mds_command: new command op to 11019 tid=0[{"prefix": "status"}] 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c45632e8d0 con 0x55c45645fed0 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98427fc700 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645fed0 0x55c4564eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98217fa700 10 client.11082 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98427fc700 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645fed0 0x55c4564eacc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7823cbe90f83b7 server_cookie=686f81b53f9cd0b0 in_seq=0 out_seq=0 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98217fa700 1 -- 172.21.15.45:0/2507096275 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55c45632e8d0 con 0x55c45645fed0 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98217fa700 10 client.11082 handle_command_reply: tid=0 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 client.11082 shutdown 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 msgr2=0x55c45643d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45643d780 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f983400efd0 tx=0x7f9834015070 comp rx=0 tx=0).stop 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 shutdown_connections 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c45645fed0 0x55c4564eacc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c45648b4d0 0x55c45645f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c456467f40 0x55c45643d780 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98217fa700 0 client.11082 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:45.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.901+0000 7f98528f3b80 1 --2- 172.21.15.45:0/2507096275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c456482450 0x55c45645f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 >> 172.21.15.45:0/2507096275 conn(0x55c456485880 msgr2=0x55c456436ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 shutdown_connections 2023-10-06T05:27:45.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98528f3b80 1 -- 172.21.15.45:0/2507096275 wait complete. 2023-10-06T05:27:45.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98528f3b80 20 client.11082 trim_cache size 0 max 16384 2023-10-06T05:27:45.911 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:27:45.911 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "158572ea-864f-4150-9c7e-6d3b4fe3c21a", 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 11019, 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.0490012450000004, 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 364, 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 144, 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 144, 2023-10-06T05:27:45.912 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 7.0211338239999996 2023-10-06T05:27:45.913 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:27:45.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9830066f30 msgr2=0x7f98300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9830066f30 0x7f98300693e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f98440cecd0 tx=0x7f9838008040 comp rx=0 tx=0).stop 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 msgr2=0x7f9844158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844158880 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f983400e8d0 tx=0x7f9834008040 comp rx=0 tx=0).stop 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 shutdown_connections 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9830066f30 0x7f98300693e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f984415d040 0x7f98440b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844069b90 0x7f9844158880 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 --2- 172.21.15.45:0/139342542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9844159170 0x7f98440c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 >> 172.21.15.45:0/139342542 conn(0x7f984406f7c0 msgr2=0x7f98440cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:45.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 shutdown_connections 2023-10-06T05:27:45.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:45.902+0000 7f98495df700 1 -- 172.21.15.45:0/139342542 wait complete. 2023-10-06T05:27:45.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:27:46.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 Processor -- start 2023-10-06T05:27:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 -- start start 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37a815a490 0x7f37a8067830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a815b5e0 0x7f37a806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37a806add0 0x7f37a806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a80ae020 con 0x7f37a806add0 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a806b250 con 0x7f37a815a490 2023-10-06T05:27:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.048+0000 7f37aef07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80b2120 con 0x7f37a815b5e0 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37a806add0 0x7f37a806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37a806add0 0x7f37a806e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41512/0 (socket says 172.21.15.45:41512) 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 -- 172.21.15.45:0/3603903209 learned_addr learned my addr 172.21.15.45:0/3603903209 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37a7fff700 1 --2- 172.21.15.45:0/3603903209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a815b5e0 0x7f37a806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37acca3700 1 --2- 172.21.15.45:0/3603903209 >> v2:172.21.15.144:3300/0 conn(0x7f37a815a490 0x7f37a8067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 -- 172.21.15.45:0/3603903209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a815b5e0 msgr2=0x7f37a806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 --2- 172.21.15.45:0/3603903209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a815b5e0 0x7f37a806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 -- 172.21.15.45:0/3603903209 >> v2:172.21.15.144:3300/0 conn(0x7f37a815a490 msgr2=0x7f37a8067830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:46.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 --2- 172.21.15.45:0/3603903209 >> v2:172.21.15.144:3300/0 conn(0x7f37a815a490 0x7f37a8067830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 -- 172.21.15.45:0/3603903209 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80ab7d0 con 0x7f37a806add0 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37ad4a4700 1 --2- 172.21.15.45:0/3603903209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a806add0 0x7f37a806e6e0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f37a80cd3d0 tx=0x7f37a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0e15270ee87726 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37a77fe700 1 -- 172.21.15.45:0/3603903209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a000e070 con 0x7f37a806add0 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.049+0000 7f37a77fe700 1 -- 172.21.15.45:0/3603903209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a000ba50 con 0x7f37a806add0 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37a77fe700 1 -- 172.21.15.45:0/3603903209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a00092e0 con 0x7f37a806add0 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 -- 172.21.15.45:0/3603903209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a806add0 msgr2=0x7f37a806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 --2- 172.21.15.45:0/3603903209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a806add0 0x7f37a806e6e0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f37a80cd3d0 tx=0x7f37a0009840 comp rx=0 tx=0).stop 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 -- 172.21.15.45:0/3603903209 shutdown_connections 2023-10-06T05:27:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 --2- 172.21.15.45:0/3603903209 >> v2:172.21.15.144:3300/0 conn(0x7f37a815a490 0x7f37a8067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 --2- 172.21.15.45:0/3603903209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a806add0 0x7f37a806e6e0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 --2- 172.21.15.45:0/3603903209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a815b5e0 0x7f37a806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 -- 172.21.15.45:0/3603903209 >> 172.21.15.45:0/3603903209 conn(0x7f37a80cc5a0 msgr2=0x7f37a80cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 -- 172.21.15.45:0/3603903209 shutdown_connections 2023-10-06T05:27:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.050+0000 7f37aef07700 1 -- 172.21.15.45:0/3603903209 wait complete. 2023-10-06T05:27:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.051+0000 7f37aef07700 1 Processor -- start 2023-10-06T05:27:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37aef07700 1 -- start start 2023-10-06T05:27:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37aef07700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a806add0 0x7f37a8168ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37aef07700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a815a490 0x7f37a80c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37aef07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37a815b5e0 0x7f37a80c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37aef07700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80aa760 con 0x7f37a815a490 2023-10-06T05:27:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37aef07700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80a88c0 con 0x7f37a815b5e0 2023-10-06T05:27:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37aef07700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a81695e0 con 0x7f37a806add0 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a815a490 0x7f37a80c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a815a490 0x7f37a80c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41526/0 (socket says 172.21.15.45:41526) 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.052+0000 7f37a7fff700 1 -- 172.21.15.45:0/2853780951 learned_addr learned my addr 172.21.15.45:0/2853780951 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37acca3700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a806add0 0x7f37a8168ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37ad4a4700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37a815b5e0 0x7f37a80c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a7fff700 1 -- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a806add0 msgr2=0x7f37a8168ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a7fff700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a806add0 0x7f37a8168ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a7fff700 1 -- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37a815b5e0 msgr2=0x7f37a80c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a7fff700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37a815b5e0 0x7f37a80c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a7fff700 1 -- 172.21.15.45:0/2853780951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0009ae0 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37acca3700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a806add0 0x7f37a8168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a7fff700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a815a490 0x7f37a80c86a0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f379c00d890 tx=0x7f379c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a4ff9700 1 -- 172.21.15.45:0/2853780951 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00b600 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a4ff9700 1 -- 172.21.15.45:0/2853780951 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00b760 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37a4ff9700 1 -- 172.21.15.45:0/2853780951 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00bae0 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a8169880 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.053+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a8169e40 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.054+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a80a8a70 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.055+0000 7f37a4ff9700 1 -- 172.21.15.45:0/2853780951 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f379c01e460 con 0x7f37a815a490 2023-10-06T05:27:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.055+0000 7f37a4ff9700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3794066f30 0x7f37940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:46.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.056+0000 7f37a4ff9700 1 -- 172.21.15.45:0/2853780951 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f379c084f00 con 0x7f37a815a490 2023-10-06T05:27:46.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.056+0000 7f37acca3700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3794066f30 0x7f37940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:46.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.056+0000 7f37acca3700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3794066f30 0x7f37940693e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3798006fd0 tx=0x7f3798008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:46.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.060+0000 7f37a4ff9700 1 -- 172.21.15.45:0/2853780951 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37a80a8a70 con 0x7f37a815a490 2023-10-06T05:27:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.310+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37a80a8a70 con 0x7f37a815a490 2023-10-06T05:27:46.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.311+0000 7f37a4ff9700 1 -- 172.21.15.45:0/2853780951 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v364) v1 ==== 53+0+73 (secure 0 0 0) 0x7f37a80a8a70 con 0x7f37a815a490 2023-10-06T05:27:46.313 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:27:46.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3794066f30 msgr2=0x7f37940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3794066f30 0x7f37940693e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3798006fd0 tx=0x7f3798008040 comp rx=0 tx=0).stop 2023-10-06T05:27:46.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a815a490 msgr2=0x7f37a80c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a815a490 0x7f37a80c86a0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f379c00d890 tx=0x7f379c008040 comp rx=0 tx=0).stop 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 shutdown_connections 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3794066f30 0x7f37940693e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37a815b5e0 0x7f37a80c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37a806add0 0x7f37a8168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 --2- 172.21.15.45:0/2853780951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37a815a490 0x7f37a80c86a0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 >> 172.21.15.45:0/2853780951 conn(0x7f37a80cc5a0 msgr2=0x7f37a80cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 shutdown_connections 2023-10-06T05:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:46.313+0000 7f37aef07700 1 -- 172.21.15.45:0/2853780951 wait complete. 2023-10-06T05:27:46.328 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:27:46.328 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:27:46.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-10-06T05:27:46.329 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:27:46.329 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:46.329 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:27:46.329 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:27:46.329 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:27:46.329 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:46.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:46.329 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-10-06T05:27:46.350 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:27:46.350 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:27:46.350 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:27:46.350 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:27:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41675sec preferred_lft 41675sec 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:27:46.352 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::f837:54ff:fe30:a86c/64 scope link 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 1e:21:99:73:dc:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::1c21:99ff:fe73:dcf1/64 scope link 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:46.353 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:27:46.354 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c2:66:b1:07:4b:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:27:46.354 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c066:b1ff:fe07:4b16/64 scope link 2023-10-06T05:27:46.354 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:27:46.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:46.354 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:27:46.421 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-10-06T05:27:46.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:46.423 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-10-06T05:27:46.488 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-10-06T05:27:46.488 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:27:46.488 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:27:46.488 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:27:46.488 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:27:46.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:46.488 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-10-06T05:27:46.489 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-10-06T05:27:46.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:27:46.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:27:46.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:27:46.489 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-10-06T05:27:46.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:46.611 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:27:46.634 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:27:46.634 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:27:46.634 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:46.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:46.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-10-06T05:27:46.711 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T05:27:46.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:46.711 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:46.775 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:27:46.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:27:46.775 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:46.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:27:46.915 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:27:46.915 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:27:46.991 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:27:46.992 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:27:46.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:46.992 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:27:47.056 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T05:27:47.056 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T05:27:47.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-10-06T05:27:47.134 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T05:27:47.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T05:27:47.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2023-10-06T05:27:47.198+0000 7f4dea77c540 -1 init, newargv = 0x56457951a870 newargc=15 2023-10-06T05:27:47.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[184881]: starting ceph client 2023-10-06T05:27:47.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[184881]: starting fuse 2023-10-06T05:27:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T05:27:47.209 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T05:27:47.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:47.210 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-10-06T05:27:47.271 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-10-06T05:27:47.271 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T05:27:47.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-10-06T05:27:47.272 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> 2023-10-06T05:27:47.273 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2023-10-06T05:27:47.274 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-10-06T05:27:47.412 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.184881.asok 2023-10-06T05:27:47.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.184881.asok 2023-10-06T05:27:47.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:47.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.184881.asok status 2023-10-06T05:27:47.768 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:27:47.768 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-10-06T05:27:47.768 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "184881", 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 11094, 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-10-06T05:27:47.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 11094 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1589314850 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-10-06T05:27:47.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1589314850 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.11094 192.168.0.1:0/1589314850", 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1589314850", 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 364, 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 144, 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-10-06T05:27:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-10-06T05:27:47.772 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-10-06T05:27:47.772 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:27:47.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:47.779 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-10-06T05:27:47.779 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:47.800 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-10-06T05:27:47.800 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:47.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:47.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:47.873 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-10-06T05:27:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.963+0000 7f0534a48040 1 Processor -- start 2023-10-06T05:27:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.963+0000 7f0534a48040 1 -- start start 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0534a48040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b3213d210 0x561b32184440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0534a48040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b321ff890 0x561b3216df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0534a48040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ffc60 0x561b3216e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0534a48040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b320e6320 con 0x561b321ff890 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0534a48040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b3208a980 con 0x561b3213d210 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0534a48040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b3208a620 con 0x561b321ffc60 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b321ff890 0x561b3216df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b321ff890 0x561b3216df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54328/0 (socket says 172.21.15.45:54328) 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 -- 172.21.15.45:0/1353176955 learned_addr learned my addr 172.21.15.45:0/1353176955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f052dc66700 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b3213d210 0x561b32184440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f052e467700 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ffc60 0x561b3216e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 -- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ffc60 msgr2=0x561b3216e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ffc60 0x561b3216e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 -- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b3213d210 msgr2=0x561b32184440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b3213d210 0x561b32184440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 -- 172.21.15.45:0/1353176955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518005cf0 con 0x561b321ff890 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f052e467700 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ffc60 0x561b3216e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f0527fff700 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b321ff890 0x561b3216df40 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f051800ac40 tx=0x7f0518013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc029d39af39f567 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f052cc64700 1 -- 172.21.15.45:0/1353176955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051801a070 con 0x561b321ff890 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f052cc64700 1 -- 172.21.15.45:0/1353176955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0518013670 con 0x561b321ff890 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.964+0000 7f052cc64700 1 -- 172.21.15.45:0/1353176955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518013a30 con 0x561b321ff890 2023-10-06T05:27:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 -- 172.21.15.45:0/1353176955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b321ff890 msgr2=0x561b3216df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b321ff890 0x561b3216df40 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f051800ac40 tx=0x7f0518013000 comp rx=0 tx=0).stop 2023-10-06T05:27:47.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 -- 172.21.15.45:0/1353176955 shutdown_connections 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ffc60 0x561b3216e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b3213d210 0x561b32184440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 --2- 172.21.15.45:0/1353176955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b321ff890 0x561b3216df40 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 -- 172.21.15.45:0/1353176955 >> 172.21.15.45:0/1353176955 conn(0x561b32140450 msgr2=0x561b32142830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 -- 172.21.15.45:0/1353176955 shutdown_connections 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.965+0000 7f0534a48040 1 -- 172.21.15.45:0/1353176955 wait complete. 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.966+0000 7f0534a48040 1 Processor -- start 2023-10-06T05:27:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.966+0000 7f0534a48040 1 -- start start 2023-10-06T05:27:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f0534a48040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b3213d210 0x561b321fc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f0534a48040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ff890 0x561b32165c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f0534a48040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b321ffc60 0x561b32166150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f0534a48040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b32081400 con 0x561b3213d210 2023-10-06T05:27:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f0534a48040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b32169b60 con 0x561b321ffc60 2023-10-06T05:27:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f0534a48040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b32169620 con 0x561b321ff890 2023-10-06T05:27:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b3213d210 0x561b321fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b3213d210 0x561b321fc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54334/0 (socket says 172.21.15.45:54334) 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 -- 172.21.15.45:0/1151130795 learned_addr learned my addr 172.21.15.45:0/1151130795 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052e467700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b321ffc60 0x561b32166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f0527fff700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ff890 0x561b32165c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 -- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ff890 msgr2=0x561b32165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b321ff890 0x561b32165c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 -- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b321ffc60 msgr2=0x561b32166150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b321ffc60 0x561b32166150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 -- 172.21.15.45:0/1151130795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b320993d0 con 0x561b3213d210 2023-10-06T05:27:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.967+0000 7f052dc66700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b3213d210 0x561b321fc800 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f052000c5e0 tx=0x7f0520000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.968+0000 7f05267fc700 1 -- 172.21.15.45:0/1151130795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520016030 con 0x561b3213d210 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.968+0000 7f05267fc700 1 -- 172.21.15.45:0/1151130795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520005070 con 0x561b3213d210 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.968+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561b3208cae0 con 0x561b3213d210 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.968+0000 7f05267fc700 1 -- 172.21.15.45:0/1151130795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520011690 con 0x561b3213d210 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.968+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b3208c540 con 0x561b3213d210 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.969+0000 7f05267fc700 1 -- 172.21.15.45:0/1151130795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0520019070 con 0x561b3213d210 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.970+0000 7f05267fc700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0510065890 0x7f0510067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.970+0000 7f05267fc700 1 -- 172.21.15.45:0/1151130795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0520017070 con 0x561b3213d210 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.970+0000 7f0534a48040 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561b321fed80 0x561b32163980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.970+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e144) v8 -- 0x561b32163ec0 con 0x561b321fed80 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.970+0000 7f0527fff700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0510065890 0x7f0510067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.970+0000 7f052e467700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561b321fed80 0x561b32163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.970+0000 7f052e467700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561b321fed80 0x561b32163980 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0527fff700 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0510065890 0x7f0510067d40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f051800ac40 tx=0x7f0518017050 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f052e467700 1 -- 172.21.15.45:0/1151130795 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+91 (crc 0 0 0) 0x7f0528003a40 con 0x561b321fed80 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561b321fed80 msgr2=0x561b32163980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x561b321fed80 0x561b32163980 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0510065890 msgr2=0x7f0510067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f0510065890 0x7f0510067d40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f051800ac40 tx=0x7f0518017050 comp rx=0 tx=0).stop 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b3213d210 msgr2=0x561b321fc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 --2- 172.21.15.45:0/1151130795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b3213d210 0x561b321fc800 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f052000c5e0 tx=0x7f0520000bc0 comp rx=0 tx=0).stop 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f052e467700 1 -- 172.21.15.45:0/1151130795 reap_dead start 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 shutdown_connections 2023-10-06T05:27:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 >> 172.21.15.45:0/1151130795 conn(0x561b32140450 msgr2=0x561b32142830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 shutdown_connections 2023-10-06T05:27:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:47.971+0000 7f0534a48040 1 -- 172.21.15.45:0/1151130795 wait complete. 2023-10-06T05:27:47.979 INFO:teuthology.orchestra.run.smithi045.stdout:Ufile1dir1 2023-10-06T05:27:47.980 DEBUG:teuthology.orchestra.run.smithi045:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "ancestors": [ 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1099511627776, 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "file1", 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 6 2023-10-06T05:27:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1, 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "dir1", 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 18 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 17, 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "old_pools": [] 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:27:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:48.110 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent rhubarb 2023-10-06T05:27:48.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.160+0000 7f5042f1d040 1 Processor -- start 2023-10-06T05:27:48.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.160+0000 7f5042f1d040 1 -- start start 2023-10-06T05:27:48.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f5042f1d040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b6a9b70 0x556f1b7389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f5042f1d040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b68ef30 0x556f1b6a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f5042f1d040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b735900 0x556f1b68d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f5042f1d040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1b609320 con 0x556f1b68ef30 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f5042f1d040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1b5ad980 con 0x556f1b6a9b70 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f5042f1d040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f1b5ad620 con 0x556f1b735900 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b68ef30 0x556f1b6a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b68ef30 0x556f1b6a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54354/0 (socket says 172.21.15.45:54354) 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 -- 172.21.15.45:0/2245507309 learned_addr learned my addr 172.21.15.45:0/2245507309 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503c13b700 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b6a9b70 0x556f1b7389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503c93c700 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b735900 0x556f1b68d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 -- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b735900 msgr2=0x556f1b68d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b735900 0x556f1b68d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 -- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b6a9b70 msgr2=0x556f1b7389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b6a9b70 0x556f1b7389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 -- 172.21.15.45:0/2245507309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f1b5afae0 con 0x556f1b68ef30 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.161+0000 7f503b93a700 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b68ef30 0x556f1b6a9f40 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f50300097d0 tx=0x7f503000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9707c5a0a4ff74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f503a938700 1 -- 172.21.15.45:0/2245507309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503000f040 con 0x556f1b68ef30 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f503a938700 1 -- 172.21.15.45:0/2245507309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5030008850 con 0x556f1b68ef30 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f503a938700 1 -- 172.21.15.45:0/2245507309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030008bd0 con 0x556f1b68ef30 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 -- 172.21.15.45:0/2245507309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b68ef30 msgr2=0x556f1b6a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b68ef30 0x556f1b6a9f40 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f50300097d0 tx=0x7f503000c040 comp rx=0 tx=0).stop 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 -- 172.21.15.45:0/2245507309 shutdown_connections 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b735900 0x556f1b68d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b6a9b70 0x556f1b7389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 --2- 172.21.15.45:0/2245507309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b68ef30 0x556f1b6a9f40 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 -- 172.21.15.45:0/2245507309 >> 172.21.15.45:0/2245507309 conn(0x556f1b662870 msgr2=0x556f1b71c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 -- 172.21.15.45:0/2245507309 shutdown_connections 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.162+0000 7f5042f1d040 1 -- 172.21.15.45:0/2245507309 wait complete. 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.163+0000 7f5042f1d040 1 Processor -- start 2023-10-06T05:27:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.163+0000 7f5042f1d040 1 -- start start 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f5042f1d040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b68ef30 0x556f1b6a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f5042f1d040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b6a9b70 0x556f1b719960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f5042f1d040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b735900 0x556f1b71b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f5042f1d040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1b5a4400 con 0x556f1b735900 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f5042f1d040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1b735220 con 0x556f1b68ef30 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f5042f1d040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f1b71a510 con 0x556f1b6a9b70 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b735900 0x556f1b71b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b735900 0x556f1b71b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54366/0 (socket says 172.21.15.45:54366) 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 -- 172.21.15.45:0/3047300303 learned_addr learned my addr 172.21.15.45:0/3047300303 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c13b700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b68ef30 0x556f1b6a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503b93a700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b6a9b70 0x556f1b719960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 -- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b6a9b70 msgr2=0x556f1b719960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f1b6a9b70 0x556f1b719960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 -- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b68ef30 msgr2=0x556f1b6a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f1b68ef30 0x556f1b6a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 -- 172.21.15.45:0/3047300303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300094b0 con 0x556f1b735900 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.164+0000 7f503c93c700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b735900 0x556f1b71b190 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f5034003000 tx=0x7f5034015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.165+0000 7f5039135700 1 -- 172.21.15.45:0/3047300303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034001ef0 con 0x556f1b735900 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.165+0000 7f5039135700 1 -- 172.21.15.45:0/3047300303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034022070 con 0x556f1b735900 2023-10-06T05:27:48.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.165+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556f1b5af540 con 0x556f1b735900 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.165+0000 7f5039135700 1 -- 172.21.15.45:0/3047300303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034001360 con 0x556f1b735900 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.165+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f1b71aa50 con 0x556f1b735900 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.166+0000 7f5039135700 1 -- 172.21.15.45:0/3047300303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5034019070 con 0x556f1b735900 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f5039135700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5028065890 0x7f5028067d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f5039135700 1 -- 172.21.15.45:0/3047300303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f5034084090 con 0x556f1b735900 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f503c13b700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5028065890 0x7f5028067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f5042f1d040 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f1b6a2850 0x556f1b7489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [setxattr parent (7) in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145) v8 -- 0x556f1b664970 con 0x556f1b6a2850 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f503b93a700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f1b6a2850 0x556f1b7489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f503c13b700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5028065890 0x7f5028067d40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f502c001eb0 tx=0x7f502c011040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.167+0000 7f503b93a700 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f1b6a2850 0x556f1b7489d0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f503b93a700 1 -- 172.21.15.45:0/3047300303 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (7)] v145'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f503001b020 con 0x556f1b6a2850 2023-10-06T05:27:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f1b6a2850 msgr2=0x556f1b7489d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x556f1b6a2850 0x556f1b7489d0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5028065890 msgr2=0x7f5028067d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5028065890 0x7f5028067d40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f502c001eb0 tx=0x7f502c011040 comp rx=0 tx=0).stop 2023-10-06T05:27:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b735900 msgr2=0x556f1b71b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 --2- 172.21.15.45:0/3047300303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556f1b735900 0x556f1b71b190 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f5034003000 tx=0x7f5034015000 comp rx=0 tx=0).stop 2023-10-06T05:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f503c93c700 1 -- 172.21.15.45:0/3047300303 reap_dead start 2023-10-06T05:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 shutdown_connections 2023-10-06T05:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 >> 172.21.15.45:0/3047300303 conn(0x556f1b662870 msgr2=0x556f1b68d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 shutdown_connections 2023-10-06T05:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.169+0000 7f5042f1d040 1 -- 172.21.15.45:0/3047300303 wait complete. 2023-10-06T05:27:48.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:48.175 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink1) 2023-10-06T05:27:48.200 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:27:48.198+0000 7f7986960700 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input 2023-10-06T05:27:48.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:stat: cannot statx 'testdir/hardlink1': Input/output error 2023-10-06T05:27:48.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:48.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.378+0000 7fdcaf93f700 1 Processor -- start 2023-10-06T05:27:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.379+0000 7fdcaf93f700 1 -- start start 2023-10-06T05:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.379+0000 7fdcaf93f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdca8158f60 0x7fdca8159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcaf93f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdca80cc390 0x7fdca80cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcaf93f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca80cb530 con 0x7fdca806a060 2023-10-06T05:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcaf93f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca806a500 con 0x7fdca80cc390 2023-10-06T05:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcaf93f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca8069bd0 con 0x7fdca8158f60 2023-10-06T05:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdca80cc390 0x7fdca80cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdca80cc390 0x7fdca80cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54372/0 (socket says 172.21.15.45:54372) 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 -- 172.21.15.45:0/401845609 learned_addr learned my addr 172.21.15.45:0/401845609 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcad6db700 1 --2- 172.21.15.45:0/401845609 >> v2:172.21.15.144:3300/0 conn(0x7fdca8158f60 0x7fdca8159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 -- 172.21.15.45:0/401845609 >> v2:172.21.15.144:3300/0 conn(0x7fdca8158f60 msgr2=0x7fdca8159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 --2- 172.21.15.45:0/401845609 >> v2:172.21.15.144:3300/0 conn(0x7fdca8158f60 0x7fdca8159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 -- 172.21.15.45:0/401845609 >> v1:172.21.15.144:6789/0 conn(0x7fdca806a060 legacy=0x7fdca80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 -- 172.21.15.45:0/401845609 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80ccd10 con 0x7fdca80cc390 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcad6db700 1 --2- 172.21.15.45:0/401845609 >> v2:172.21.15.144:3300/0 conn(0x7fdca8158f60 0x7fdca8159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdcadedc700 1 --2- 172.21.15.45:0/401845609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca80cc390 0x7fdca80cc7d0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fdca806ed80 tx=0x7fdca0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2169d44e503bee1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdc9ffff700 1 -- 172.21.15.45:0/401845609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000e070 con 0x7fdca80cc390 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdc9ffff700 1 -- 172.21.15.45:0/401845609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000ba50 con 0x7fdca80cc390 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.380+0000 7fdc9ffff700 1 -- 172.21.15.45:0/401845609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00092e0 con 0x7fdca80cc390 2023-10-06T05:27:48.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 -- 172.21.15.45:0/401845609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca80cc390 msgr2=0x7fdca80cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/401845609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca80cc390 0x7fdca80cc7d0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fdca806ed80 tx=0x7fdca0009840 comp rx=0 tx=0).stop 2023-10-06T05:27:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 -- 172.21.15.45:0/401845609 shutdown_connections 2023-10-06T05:27:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/401845609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca80cc390 0x7fdca80cc7d0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/401845609 >> v2:172.21.15.144:3300/0 conn(0x7fdca8158f60 0x7fdca8159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 -- 172.21.15.45:0/401845609 >> 172.21.15.45:0/401845609 conn(0x7fdca80cdf50 msgr2=0x7fdca80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 -- 172.21.15.45:0/401845609 shutdown_connections 2023-10-06T05:27:48.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.381+0000 7fdcaf93f700 1 -- 172.21.15.45:0/401845609 wait complete. 2023-10-06T05:27:48.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.382+0000 7fdcaf93f700 1 Processor -- start 2023-10-06T05:27:48.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.382+0000 7fdcaf93f700 1 -- start start 2023-10-06T05:27:48.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.383+0000 7fdcaf93f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca806a060 0x7fdca815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.383+0000 7fdcaf93f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdca80cc390 0x7fdca8154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.383+0000 7fdcaf93f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdca8158f60 0x7fdca80c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.383+0000 7fdcaf93f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca81556d0 con 0x7fdca806a060 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.383+0000 7fdcaf93f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca816d8f0 con 0x7fdca80cc390 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.383+0000 7fdcaf93f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8156230 con 0x7fdca8158f60 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca806a060 0x7fdca815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca806a060 0x7fdca815c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54376/0 (socket says 172.21.15.45:54376) 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 -- 172.21.15.45:0/2898874661 learned_addr learned my addr 172.21.15.45:0/2898874661 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcaceda700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdca80cc390 0x7fdca8154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcadedc700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdca8158f60 0x7fdca80c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 -- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdca8158f60 msgr2=0x7fdca80c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdca8158f60 0x7fdca80c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 -- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdca80cc390 msgr2=0x7fdca8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdca80cc390 0x7fdca8154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 -- 172.21.15.45:0/2898874661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009ae0 con 0x7fdca806a060 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcadedc700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdca8158f60 0x7fdca80c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcaceda700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdca80cc390 0x7fdca8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcad6db700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca806a060 0x7fdca815c4d0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fdca400d890 tx=0x7fdca4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdc9d7fa700 1 -- 172.21.15.45:0/2898874661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca400b600 con 0x7fdca806a060 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdc9d7fa700 1 -- 172.21.15.45:0/2898874661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca400b760 con 0x7fdca806a060 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca8155900 con 0x7fdca806a060 2023-10-06T05:27:48.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdc9d7fa700 1 -- 172.21.15.45:0/2898874661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca400bb00 con 0x7fdca806a060 2023-10-06T05:27:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.384+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca8155ec0 con 0x7fdca806a060 2023-10-06T05:27:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.385+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca806f410 con 0x7fdca806a060 2023-10-06T05:27:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.386+0000 7fdc9d7fa700 1 -- 172.21.15.45:0/2898874661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdca4014430 con 0x7fdca806a060 2023-10-06T05:27:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.386+0000 7fdc9d7fa700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc8c066f30 0x7fdc8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.387+0000 7fdc9d7fa700 1 -- 172.21.15.45:0/2898874661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fdca4084cc0 con 0x7fdca806a060 2023-10-06T05:27:48.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.387+0000 7fdcaceda700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc8c066f30 0x7fdc8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.387+0000 7fdcaceda700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc8c066f30 0x7fdc8c0693e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdc98007fd0 tx=0x7fdc98009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.391+0000 7fdc9d7fa700 1 -- 172.21.15.45:0/2898874661 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdca408b080 con 0x7fdca806a060 2023-10-06T05:27:48.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.635+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdca8068fc0 con 0x7fdca806a060 2023-10-06T05:27:48.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.636+0000 7fdc9d7fa700 1 -- 172.21.15.45:0/2898874661 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 364 v364) v1 ==== 95+0+4610 (secure 0 0 0) 0x7fdca4051b60 con 0x7fdca806a060 2023-10-06T05:27:48.638 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:48.638 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":364,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":364,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:42.854757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc8c066f30 msgr2=0x7fdc8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc8c066f30 0x7fdc8c0693e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdc98007fd0 tx=0x7fdc98009040 comp rx=0 tx=0).stop 2023-10-06T05:27:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca806a060 msgr2=0x7fdca815c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca806a060 0x7fdca815c4d0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fdca400d890 tx=0x7fdca4008040 comp rx=0 tx=0).stop 2023-10-06T05:27:48.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 shutdown_connections 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc8c066f30 0x7fdc8c0693e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdca8158f60 0x7fdca80c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdca806a060 0x7fdca815c4d0 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 --2- 172.21.15.45:0/2898874661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdca80cc390 0x7fdca8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 >> 172.21.15.45:0/2898874661 conn(0x7fdca80cdf50 msgr2=0x7fdca80c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 shutdown_connections 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.638+0000 7fdcaf93f700 1 -- 172.21.15.45:0/2898874661 wait complete. 2023-10-06T05:27:48.641 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 364 2023-10-06T05:27:48.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-10-06T05:27:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.785+0000 7f07d44ef700 1 Processor -- start 2023-10-06T05:27:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.785+0000 7f07d44ef700 1 -- start start 2023-10-06T05:27:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d44ef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0ccb70 0x7f07cc0aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d44ef700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07cc0c8b70 0x7f07cc0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d44ef700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0c3b50 con 0x7f07cc157330 2023-10-06T05:27:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d44ef700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc157700 con 0x7f07cc0c8b70 2023-10-06T05:27:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d44ef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c98e0 con 0x7f07cc0ccb70 2023-10-06T05:27:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d1a8a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0ccb70 0x7f07cc0aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d2a8c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07cc0c8b70 0x7f07cc0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d1a8a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0ccb70 0x7f07cc0aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:52942/0 (socket says 172.21.15.45:52942) 2023-10-06T05:27:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d2a8c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07cc0c8b70 0x7f07cc0c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:48954/0 (socket says 172.21.15.45:48954) 2023-10-06T05:27:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d1a8a700 1 -- 172.21.15.45:0/2409221257 learned_addr learned my addr 172.21.15.45:0/2409221257 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.786+0000 7f07d228b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f07cc157330 0x7f07cc0ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:49926/0 (socket says 172.21.15.45:49926) 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d2a8c700 1 -- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0ccb70 msgr2=0x7f07cc0aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0ccb70 0x7f07cc0aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d2a8c700 1 -- 172.21.15.45:0/2409221257 >> v1:172.21.15.144:6789/0 conn(0x7f07cc157330 legacy=0x7f07cc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d1289700 1 -- 172.21.15.45:0/2409221257 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507356553 0 0) 0x7f07cc0c3b50 con 0x7f07cc157330 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d2a8c700 1 -- 172.21.15.45:0/2409221257 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0c9bc0 con 0x7f07cc0c8b70 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0ccb70 0x7f07cc0aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0c8b70 0x7f07cc0c8ff0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009e60 tx=0x7f07c4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f0abcad77df19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d1289700 1 -- 172.21.15.45:0/2409221257 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c400e070 con 0x7f07cc0c8b70 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d1289700 1 -- 172.21.15.45:0/2409221257 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c400bb60 con 0x7f07cc0c8b70 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.787+0000 7f07d1289700 1 -- 172.21.15.45:0/2409221257 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4012430 con 0x7f07cc0c8b70 2023-10-06T05:27:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 -- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0c8b70 msgr2=0x7f07cc0c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 --2- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0c8b70 0x7f07cc0c8ff0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009e60 tx=0x7f07c4009780 comp rx=0 tx=0).stop 2023-10-06T05:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 -- 172.21.15.45:0/2409221257 shutdown_connections 2023-10-06T05:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 --2- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0c8b70 0x7f07cc0c8ff0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 --2- 172.21.15.45:0/2409221257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0ccb70 0x7f07cc0aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 -- 172.21.15.45:0/2409221257 >> 172.21.15.45:0/2409221257 conn(0x7f07cc067e00 msgr2=0x7f07cc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 -- 172.21.15.45:0/2409221257 shutdown_connections 2023-10-06T05:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.788+0000 7f07d44ef700 1 -- 172.21.15.45:0/2409221257 wait complete. 2023-10-06T05:27:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.789+0000 7f07d44ef700 1 Processor -- start 2023-10-06T05:27:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.789+0000 7f07d44ef700 1 -- start start 2023-10-06T05:27:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d44ef700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0c8b70 0x7f07cc14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d44ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0ccb70 0x7f07cc0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d44ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc157330 0x7f07cc0690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d44ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c74a0 con 0x7f07cc157330 2023-10-06T05:27:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d44ef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c5420 con 0x7f07cc0ccb70 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d44ef700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c5700 con 0x7f07cc0c8b70 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d2a8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc157330 0x7f07cc0690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d2a8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc157330 0x7f07cc0690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54378/0 (socket says 172.21.15.45:54378) 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d228b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0c8b70 0x7f07cc14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1719121052 learned_addr learned my addr 172.21.15.45:0/1719121052 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.790+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0ccb70 0x7f07cc0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0c8b70 msgr2=0x7f07cc14fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0c8b70 0x7f07cc14fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0ccb70 msgr2=0x7f07cc0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0ccb70 0x7f07cc0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1719121052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c00094f0 con 0x7f07cc157330 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0ccb70 0x7f07cc0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc157330 0x7f07cc0690f0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f07c400b6c0 tx=0x7f07c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07be7fc700 1 -- 172.21.15.45:0/1719121052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4027470 con 0x7f07cc157330 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07be7fc700 1 -- 172.21.15.45:0/1719121052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c40275f0 con 0x7f07cc157330 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c4009a20 con 0x7f07cc157330 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07be7fc700 1 -- 172.21.15.45:0/1719121052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c40279b0 con 0x7f07cc157330 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.791+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc0c7130 con 0x7f07cc157330 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.792+0000 7f07be7fc700 1 -- 172.21.15.45:0/1719121052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07c4024070 con 0x7f07cc157330 2023-10-06T05:27:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.793+0000 7f07be7fc700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f07b0066f30 0x7f07b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.793+0000 7f07be7fc700 1 -- 172.21.15.45:0/1719121052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f07c4087200 con 0x7f07cc157330 2023-10-06T05:27:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.793+0000 7f07d228b700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f07b0066f30 0x7f07b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.794+0000 7f07d228b700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f07b0066f30 0x7f07b00693e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001230 tx=0x7f07c8007d20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.795+0000 7f07dd803b80 1 Processor -- start 2023-10-06T05:27:48.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.795+0000 7f07dd803b80 1 -- start start 2023-10-06T05:27:48.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb654f0 0x55c65bb658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb6e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07dd803b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65ba383b0 con 0x55c65bb654f0 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65ba751d0 con 0x55c65bb4afe0 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c65ba72b90 con 0x55c65bb6e570 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb654f0 0x55c65bb658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb654f0 0x55c65bb658c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54382/0 (socket says 172.21.15.45:54382) 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 -- 172.21.15.45:0/473980210 learned_addr learned my addr 172.21.15.45:0/473980210 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb6e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 -- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 msgr2=0x55c65bb6e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb6e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 -- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 msgr2=0x55c65bb6e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb6e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 -- 172.21.15.45:0/473980210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c65b874830 con 0x55c65bb654f0 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.796+0000 7f07d228b700 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb654f0 0x55c65bb658c0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001880 tx=0x7f07c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6a9f60ba62d105 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07af7fe700 1 -- 172.21.15.45:0/473980210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8013600 con 0x55c65bb654f0 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07af7fe700 1 -- 172.21.15.45:0/473980210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c8013760 con 0x55c65bb654f0 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07af7fe700 1 -- 172.21.15.45:0/473980210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8013b00 con 0x55c65bb654f0 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 -- 172.21.15.45:0/473980210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb654f0 msgr2=0x55c65bb658c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb654f0 0x55c65bb658c0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001880 tx=0x7f07c8014000 comp rx=0 tx=0).stop 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 -- 172.21.15.45:0/473980210 shutdown_connections 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb6e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb654f0 0x55c65bb658c0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 --2- 172.21.15.45:0/473980210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb6e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 -- 172.21.15.45:0/473980210 >> 172.21.15.45:0/473980210 conn(0x55c65bb68920 msgr2=0x55c65bb68cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 -- 172.21.15.45:0/473980210 shutdown_connections 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.797+0000 7f07dd803b80 1 -- 172.21.15.45:0/473980210 wait complete. 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.798+0000 7f07dd803b80 1 -- start start 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.798+0000 7f07dd803b80 1 Processor -- start 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb67ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07dd803b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65ba6e980 con 0x55c65bb6e570 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65ba1aa20 con 0x55c65bb4afe0 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c65ba1a6c0 con 0x55c65bb654f0 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07dd803b80 10 client.0 _ll_register_callbacks cb 0x55c65b87daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07d2a8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb67ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07d2a8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb67ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54384/0 (socket says 172.21.15.45:54384) 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07d228b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1626692190 learned_addr learned my addr 172.21.15.45:0/1626692190 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.799+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1626692190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 msgr2=0x55c65bb677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1626692190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 msgr2=0x55c65bb4bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb4bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07d2a8c700 1 -- 172.21.15.45:0/1626692190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c80014d0 con 0x55c65bb6e570 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb67ce0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f07c4057d90 tx=0x7f07c400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6636de689fad74cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07acff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07acff9700 1 -- 172.21.15.45:0/1626692190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c400e040 con 0x55c65bb6e570 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07acff9700 1 -- 172.21.15.45:0/1626692190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f07c4058500 con 0x55c65bb6e570 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c65bb12e00 con 0x55c65bb6e570 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07acff9700 1 -- 172.21.15.45:0/1626692190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07c4057e00 con 0x55c65bb6e570 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07dd803b80 10 client.11127 fetch_fsmap learned FSMap version 364 2023-10-06T05:27:48.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) v3 -- 0x55c65b961140 con 0x55c65bb6e570 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07acff9700 1 -- 172.21.15.45:0/1626692190 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c4084020 con 0x55c65bb6e570 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07acff9700 1 -- 172.21.15.45:0/1626692190 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 364) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f07c4058060 con 0x55c65bb6e570 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.800+0000 7f07dd803b80 10 client.11127 fetch_fsmap finished waiting for FSMap version 364 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07dd803b80 10 client.11127 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07dd803b80 20 client.11127 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07dd803b80 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb3f2c0 0x55c65bb3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07dd803b80 4 client.11127 mds_command: new command op to 11019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c65baefbd0 con 0x55c65bb3f2c0 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07d228b700 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb3f2c0 0x55c65bb3f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07acff9700 10 client.11127 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.801+0000 7f07d228b700 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb3f2c0 0x55c65bb3f690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8ea162a9d3c04b server_cookie=2e81d94063b58126 in_seq=0 out_seq=0 2023-10-06T05:27:48.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07acff9700 1 -- 172.21.15.45:0/1626692190 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c65baefbd0 con 0x55c65bb3f2c0 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07acff9700 10 client.11127 handle_command_reply: tid=0 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 client.11127 shutdown 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 msgr2=0x55c65bb67ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb67ce0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f07c4057d90 tx=0x7f07c400bfd0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 shutdown_connections 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb3f2c0 0x55c65bb3f690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb67ce0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07acff9700 0 client.11127 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 --2- 172.21.15.45:0/1626692190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 >> 172.21.15.45:0/1626692190 conn(0x55c65bb68920 msgr2=0x55c65bb1ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 shutdown_connections 2023-10-06T05:27:48.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 1 -- 172.21.15.45:0/1626692190 wait complete. 2023-10-06T05:27:48.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.802+0000 7f07dd803b80 20 client.11127 trim_cache size 0 max 16384 2023-10-06T05:27:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.815+0000 7f07dd803b80 1 Processor -- start 2023-10-06T05:27:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.815+0000 7f07dd803b80 1 -- start start 2023-10-06T05:27:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb67780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb3e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb3e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07dd803b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65b906280 con 0x55c65bb6e570 2023-10-06T05:27:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65bb46170 con 0x55c65bb4afe0 2023-10-06T05:27:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c65bb66020 con 0x55c65bb654f0 2023-10-06T05:27:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07d2a8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb3e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07d2a8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb3e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54400/0 (socket says 172.21.15.45:54400) 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07d2a8c700 1 -- 172.21.15.45:0/3872905744 learned_addr learned my addr 172.21.15.45:0/3872905744 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.816+0000 7f07d228b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb67780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb3e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d2a8c700 1 -- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 msgr2=0x55c65bb3e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb3e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d2a8c700 1 -- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 msgr2=0x55c65bb67780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb67780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d2a8c700 1 -- 172.21.15.45:0/3872905744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c65ba3d460 con 0x55c65bb6e570 2023-10-06T05:27:48.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb3e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:48.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d228b700 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:48.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb3e7e0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f07c40033e0 tx=0x7f07c406e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55285b8444809fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07affff700 1 -- 172.21.15.45:0/3872905744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c400e050 con 0x55c65bb6e570 2023-10-06T05:27:48.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07affff700 1 -- 172.21.15.45:0/3872905744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c4084030 con 0x55c65bb6e570 2023-10-06T05:27:48.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.817+0000 7f07affff700 1 -- 172.21.15.45:0/3872905744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4054140 con 0x55c65bb6e570 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 -- 172.21.15.45:0/3872905744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 msgr2=0x55c65bb3e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb3e7e0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f07c40033e0 tx=0x7f07c406e000 comp rx=0 tx=0).stop 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 -- 172.21.15.45:0/3872905744 shutdown_connections 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb6e570 0x55c65bb3e7e0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb4afe0 0x55c65bb67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 --2- 172.21.15.45:0/3872905744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb654f0 0x55c65bb3e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 -- 172.21.15.45:0/3872905744 >> 172.21.15.45:0/3872905744 conn(0x55c65bb68920 msgr2=0x55c65bb1c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 -- 172.21.15.45:0/3872905744 shutdown_connections 2023-10-06T05:27:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 -- 172.21.15.45:0/3872905744 wait complete. 2023-10-06T05:27:48.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 -- start start 2023-10-06T05:27:48.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.818+0000 7f07dd803b80 1 Processor -- start 2023-10-06T05:27:48.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.819+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb4afe0 0x55c65bbc4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.819+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb654f0 0x55c65bb465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.819+0000 7f07dd803b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb46b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.819+0000 7f07dd803b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65bb1bcc0 con 0x55c65bb4afe0 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.819+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c65bb47430 con 0x55c65bb654f0 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.819+0000 7f07dd803b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c65bb1f380 con 0x55c65bb6e570 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.819+0000 7f07dd803b80 10 client.0 _ll_register_callbacks cb 0x55c65b87daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb4afe0 0x55c65bbc4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb4afe0 0x55c65bbc4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54414/0 (socket says 172.21.15.45:54414) 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 -- 172.21.15.45:0/2179490246 learned_addr learned my addr 172.21.15.45:0/2179490246 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb654f0 0x55c65bb465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d2a8c700 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb46b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 -- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 msgr2=0x55c65bb46b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb46b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 -- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb654f0 msgr2=0x55c65bb465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb654f0 0x55c65bb465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 -- 172.21.15.45:0/2179490246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c65b874830 con 0x55c65bb4afe0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07d228b700 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb4afe0 0x55c65bbc4200 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001c40 tx=0x7f07c8034250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d25662310141050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07acff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07acff9700 1 -- 172.21.15.45:0/2179490246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8033040 con 0x55c65bb4afe0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07acff9700 1 -- 172.21.15.45:0/2179490246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f07c801b050 con 0x55c65bb4afe0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.820+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c65bb21bc0 con 0x55c65bb4afe0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07acff9700 1 -- 172.21.15.45:0/2179490246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07c8013ae0 con 0x55c65bb4afe0 2023-10-06T05:27:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 10 client.11133 fetch_fsmap learned FSMap version 364 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) v3 -- 0x55c65b961140 con 0x55c65bb4afe0 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07acff9700 1 -- 172.21.15.45:0/2179490246 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c65bb21bc0 con 0x55c65bb4afe0 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07acff9700 1 -- 172.21.15.45:0/2179490246 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 364) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f07c803e020 con 0x55c65bb4afe0 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 10 client.11133 fetch_fsmap finished waiting for FSMap version 364 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 10 client.11133 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 20 client.11133 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb1ad50 0x55c65bb1b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 4 client.11133 mds_command: new command op to 11019 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x7f07c8013760 con 0x55c65bb1ad50 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb1ad50 0x55c65bb1b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07acff9700 10 client.11133 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:48.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.821+0000 7f07d1a8a700 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb1ad50 0x55c65bb1b120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b06f150fd81f7b server_cookie=eaffc9e1d24282de in_seq=0 out_seq=0 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07acff9700 1 -- 172.21.15.45:0/2179490246 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+145 (crc 0 0 0) 0x7f07c8013760 con 0x55c65bb1ad50 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07acff9700 10 client.11133 handle_command_reply: tid=0 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 client.11133 shutdown 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb4afe0 msgr2=0x55c65bbc4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb4afe0 0x55c65bbc4200 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001c40 tx=0x7f07c8034250 comp rx=0 tx=0).stop 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 shutdown_connections 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55c65bb1ad50 0x55c65bb1b120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c65bb6e570 0x55c65bb46b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c65bb4afe0 0x55c65bbc4200 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07acff9700 0 client.11133 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:48.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 --2- 172.21.15.45:0/2179490246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c65bb654f0 0x55c65bb465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 >> 172.21.15.45:0/2179490246 conn(0x55c65bb68920 msgr2=0x55c65bb20ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 shutdown_connections 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 1 -- 172.21.15.45:0/2179490246 wait complete. 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.822+0000 7f07dd803b80 20 client.11133 trim_cache size 0 max 16384 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2023-10-06T05:27:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 2157878309, 2023-10-06T05:27:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2023-10-06T05:27:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/testdir/hardlink1" 2023-10-06T05:27:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:27:48.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f07b0066f30 msgr2=0x7f07b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f07b0066f30 0x7f07b00693e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001230 tx=0x7f07c8007d20 comp rx=0 tx=0).stop 2023-10-06T05:27:48.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc157330 msgr2=0x7f07cc0690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc157330 0x7f07cc0690f0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f07c400b6c0 tx=0x7f07c4019000 comp rx=0 tx=0).stop 2023-10-06T05:27:48.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 shutdown_connections 2023-10-06T05:27:48.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f07b0066f30 0x7f07b00693e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc157330 0x7f07cc0690f0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07cc0c8b70 0x7f07cc14fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 --2- 172.21.15.45:0/1719121052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07cc0ccb70 0x7f07cc0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 >> 172.21.15.45:0/1719121052 conn(0x7f07cc067e00 msgr2=0x7f07cc0c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 shutdown_connections 2023-10-06T05:27:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.823+0000 7f07d44ef700 1 -- 172.21.15.45:0/1719121052 wait complete. 2023-10-06T05:27:48.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:48.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.967+0000 7f3c05159700 1 Processor -- start 2023-10-06T05:27:48.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.967+0000 7f3c05159700 1 -- start start 2023-10-06T05:27:48.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.967+0000 7f3c05159700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3c000cd3f0 0x7f3c0006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.967+0000 7f3c05159700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0015b380 con 0x7f3c001570b0 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.967+0000 7f3c05159700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0015b660 con 0x7f3c00156ce0 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3c05159700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0006c010 con 0x7f3c000cd3f0 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfe59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c001570b0 0x7f3c000ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53218/0 (socket says 172.21.15.45:53218) 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfe59c700 1 -- 172.21.15.45:0/3031751901 learned_addr learned my addr 172.21.15.45:0/3031751901 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bff59e700 1 --2- 172.21.15.45:0/3031751901 >> v2:172.21.15.144:3300/0 conn(0x7f3c000cd3f0 0x7f3c0006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703562262 0 0) 0x7f3c0015b380 con 0x7f3c001570b0 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be8003570 con 0x7f3c001570b0 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1628493837 0 0) 0x7f3be8003570 con 0x7f3c001570b0 2023-10-06T05:27:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 >> v2:172.21.15.144:3300/0 conn(0x7f3c000cd3f0 msgr2=0x7f3c0006bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 --2- 172.21.15.45:0/3031751901 >> v2:172.21.15.144:3300/0 conn(0x7f3c000cd3f0 0x7f3c0006bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 >> v1:172.21.15.144:6789/0 conn(0x7f3c00156ce0 legacy=0x7f3c000afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0015b940 con 0x7f3c001570b0 2023-10-06T05:27:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f3bf4001470 con 0x7f3c001570b0 2023-10-06T05:27:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 >> v1:172.21.15.45:6789/0 conn(0x7f3c001570b0 legacy=0x7f3c000ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3be8003a60 0x7f3be8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be8004810 0x7f3be8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be80051c0 0x7f3be800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0006c010 con 0x7f3be8003a60 2023-10-06T05:27:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be800c0d0 con 0x7f3be80051c0 2023-10-06T05:27:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.968+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be800c290 con 0x7f3be8004810 2023-10-06T05:27:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfed9d700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3be8003a60 0x7f3be8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bff59e700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be80051c0 0x7f3be800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfe59c700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be8004810 0x7f3be8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfed9d700 1 -- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be8004810 msgr2=0x7f3be8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfed9d700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be8004810 0x7f3be8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfed9d700 1 -- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be80051c0 msgr2=0x7f3be800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfed9d700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be80051c0 0x7f3be800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfed9d700 1 -- 172.21.15.45:0/3031751901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8003570 con 0x7f3be8003a60 2023-10-06T05:27:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bff59e700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be80051c0 0x7f3be800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bff59e700 1 -- 172.21.15.45:0/3031751901 reap_dead start 2023-10-06T05:27:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfed9d700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3be8003a60 0x7f3be8003e70 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00018b0 tx=0x7f3bf0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4368022db52f3564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0012420 con 0x7f3be8003a60 2023-10-06T05:27:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf00125a0 con 0x7f3be8003a60 2023-10-06T05:27:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.969+0000 7f3bfdd9b700 1 -- 172.21.15.45:0/3031751901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0012960 con 0x7f3be8003a60 2023-10-06T05:27:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.970+0000 7f3c05159700 1 -- 172.21.15.45:0/3031751901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3be8003a60 msgr2=0x7f3be8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.970+0000 7f3c05159700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3be8003a60 0x7f3be8003e70 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00018b0 tx=0x7f3bf0007d30 comp rx=0 tx=0).stop 2023-10-06T05:27:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.970+0000 7f3c05159700 1 -- 172.21.15.45:0/3031751901 shutdown_connections 2023-10-06T05:27:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.970+0000 7f3c05159700 1 --2- 172.21.15.45:0/3031751901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3be8003a60 0x7f3be8003e70 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.970+0000 7f3c05159700 1 -- 172.21.15.45:0/3031751901 >> 172.21.15.45:0/3031751901 conn(0x7f3c0006f090 msgr2=0x7f3c0006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.970+0000 7f3c05159700 1 -- 172.21.15.45:0/3031751901 shutdown_connections 2023-10-06T05:27:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.970+0000 7f3c05159700 1 -- 172.21.15.45:0/3031751901 wait complete. 2023-10-06T05:27:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.971+0000 7f3c05159700 1 Processor -- start 2023-10-06T05:27:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.971+0000 7f3c05159700 1 -- start start 2023-10-06T05:27:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3c05159700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be8003a60 0x7f3c00154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3c05159700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c000c5610 0x7f3c000c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3c05159700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c000c5a80 0x7f3c000c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3c05159700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000c85a0 con 0x7f3c000c5610 2023-10-06T05:27:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3c05159700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000c9300 con 0x7f3be8003a60 2023-10-06T05:27:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3c05159700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c001551e0 con 0x7f3c000c5a80 2023-10-06T05:27:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c000c5610 0x7f3c000c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c000c5610 0x7f3c000c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54426/0 (socket says 172.21.15.45:54426) 2023-10-06T05:27:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 -- 172.21.15.45:0/890038036 learned_addr learned my addr 172.21.15.45:0/890038036 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfed9d700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be8003a60 0x7f3c00154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bff59e700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c000c5a80 0x7f3c000c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 -- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c000c5a80 msgr2=0x7f3c000c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c000c5a80 0x7f3c000c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 -- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be8003a60 msgr2=0x7f3c00154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be8003a60 0x7f3c00154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.972+0000 7f3bfe59c700 1 -- 172.21.15.45:0/890038036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c000c7fe0 con 0x7f3c000c5610 2023-10-06T05:27:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.973+0000 7f3bfe59c700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c000c5610 0x7f3c000c47e0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4007310 tx=0x7f3bf4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.973+0000 7f3be6ffd700 1 -- 172.21.15.45:0/890038036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4018030 con 0x7f3c000c5610 2023-10-06T05:27:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.973+0000 7f3be6ffd700 1 -- 172.21.15.45:0/890038036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf400f8d0 con 0x7f3c000c5610 2023-10-06T05:27:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.973+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c000c82c0 con 0x7f3c000c5610 2023-10-06T05:27:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.973+0000 7f3be6ffd700 1 -- 172.21.15.45:0/890038036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf400fc90 con 0x7f3c000c5610 2023-10-06T05:27:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.973+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c000c42c0 con 0x7f3c000c5610 2023-10-06T05:27:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.974+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c000c3e40 con 0x7f3c000c5610 2023-10-06T05:27:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.974+0000 7f3be6ffd700 1 -- 172.21.15.45:0/890038036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bf401b070 con 0x7f3c000c5610 2023-10-06T05:27:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.975+0000 7f3be6ffd700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3bec066f30 0x7f3bec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.975+0000 7f3be6ffd700 1 -- 172.21.15.45:0/890038036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f3bf4019070 con 0x7f3c000c5610 2023-10-06T05:27:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.975+0000 7f3bfed9d700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3bec066f30 0x7f3bec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.976+0000 7f3bfed9d700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3bec066f30 0x7f3bec0693e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00018b0 tx=0x7f3bf0015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:48.980+0000 7f3be6ffd700 1 -- 172.21.15.45:0/890038036 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c000c3e40 con 0x7f3c000c5610 2023-10-06T05:27:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.231+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c000c3e40 con 0x7f3c000c5610 2023-10-06T05:27:49.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.232+0000 7f3be6ffd700 1 -- 172.21.15.45:0/890038036 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 364 v364) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f3c000c3e40 con 0x7f3c000c5610 2023-10-06T05:27:49.735 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:49.735 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":364,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":364,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:42.854757+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3bec066f30 msgr2=0x7f3bec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3bec066f30 0x7f3bec0693e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00018b0 tx=0x7f3bf0015000 comp rx=0 tx=0).stop 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c000c5610 msgr2=0x7f3c000c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c000c5610 0x7f3c000c47e0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4007310 tx=0x7f3bf4001e50 comp rx=0 tx=0).stop 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 shutdown_connections 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3bec066f30 0x7f3bec0693e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c000c5a80 0x7f3c000c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be8003a60 0x7f3c00154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 --2- 172.21.15.45:0/890038036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c000c5610 0x7f3c000c47e0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 >> 172.21.15.45:0/890038036 conn(0x7f3c0006f090 msgr2=0x7f3c00070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 shutdown_connections 2023-10-06T05:27:49.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.234+0000 7f3c05159700 1 -- 172.21.15.45:0/890038036 wait complete. 2023-10-06T05:27:49.738 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 364 2023-10-06T05:27:49.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2157878309 2023-10-06T05:27:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.880+0000 7efec5c16700 1 Processor -- start 2023-10-06T05:27:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.880+0000 7efec5c16700 1 -- start start 2023-10-06T05:27:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efec5c16700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efec0159150 0x7efec00ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efec5c16700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec0159520 0x7efec015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efec5c16700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efec015d3d0 0x7efec0067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efec5c16700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec00c9c90 con 0x7efec015d3d0 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efec5c16700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec015d8e0 con 0x7efec0159150 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efec5c16700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec015dbc0 con 0x7efec0159520 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efec015d3d0 0x7efec0067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efec015d3d0 0x7efec0067620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54440/0 (socket says 172.21.15.45:54440) 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 -- 172.21.15.45:0/1121277158 learned_addr learned my addr 172.21.15.45:0/1121277158 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebeffd700 1 --2- 172.21.15.45:0/1121277158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec0159520 0x7efec015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebf7fe700 1 --2- 172.21.15.45:0/1121277158 >> v2:172.21.15.144:3300/0 conn(0x7efec0159150 0x7efec00ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 -- 172.21.15.45:0/1121277158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec0159520 msgr2=0x7efec015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 --2- 172.21.15.45:0/1121277158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec0159520 0x7efec015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 -- 172.21.15.45:0/1121277158 >> v2:172.21.15.144:3300/0 conn(0x7efec0159150 msgr2=0x7efec00ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 --2- 172.21.15.45:0/1121277158 >> v2:172.21.15.144:3300/0 conn(0x7efec0159150 0x7efec00ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.881+0000 7efebffff700 1 -- 172.21.15.45:0/1121277158 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec00c9920 con 0x7efec015d3d0 2023-10-06T05:27:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efebffff700 1 --2- 172.21.15.45:0/1121277158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec0067620 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7efec00c3d80 tx=0x7efeb8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd47054c4e412114 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efebe7fc700 1 -- 172.21.15.45:0/1121277158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800e070 con 0x7efec015d3d0 2023-10-06T05:27:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efebe7fc700 1 -- 172.21.15.45:0/1121277158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb800ba50 con 0x7efec015d3d0 2023-10-06T05:27:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efebe7fc700 1 -- 172.21.15.45:0/1121277158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb80092e0 con 0x7efec015d3d0 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efec5c16700 1 -- 172.21.15.45:0/1121277158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 msgr2=0x7efec0067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efec5c16700 1 --2- 172.21.15.45:0/1121277158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec0067620 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7efec00c3d80 tx=0x7efeb8009840 comp rx=0 tx=0).stop 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efec5c16700 1 -- 172.21.15.45:0/1121277158 shutdown_connections 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efec5c16700 1 --2- 172.21.15.45:0/1121277158 >> v2:172.21.15.144:3300/0 conn(0x7efec0159150 0x7efec00ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efec5c16700 1 --2- 172.21.15.45:0/1121277158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec0067620 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efec5c16700 1 --2- 172.21.15.45:0/1121277158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec0159520 0x7efec015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.882+0000 7efec5c16700 1 -- 172.21.15.45:0/1121277158 >> 172.21.15.45:0/1121277158 conn(0x7efec00691c0 msgr2=0x7efec00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.883+0000 7efec5c16700 1 -- 172.21.15.45:0/1121277158 shutdown_connections 2023-10-06T05:27:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.883+0000 7efec5c16700 1 -- 172.21.15.45:0/1121277158 wait complete. 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.884+0000 7efec5c16700 1 Processor -- start 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.884+0000 7efec5c16700 1 -- start start 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efec5c16700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec0159150 0x7efec0150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efec5c16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec00c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efec5c16700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec016a3b0 0x7efec00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efec5c16700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00c4740 con 0x7efec015d3d0 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efec5c16700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00c4a20 con 0x7efec0159150 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efec5c16700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec00c4d00 con 0x7efec016a3b0 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec00c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec00c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54446/0 (socket says 172.21.15.45:54446) 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 -- 172.21.15.45:0/4094051622 learned_addr learned my addr 172.21.15.45:0/4094051622 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebffff700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec016a3b0 0x7efec00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebf7fe700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec0159150 0x7efec0150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 -- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec016a3b0 msgr2=0x7efec00c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec016a3b0 0x7efec00c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 -- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec0159150 msgr2=0x7efec0150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec0159150 0x7efec0150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 -- 172.21.15.45:0/4094051622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8009ae0 con 0x7efec015d3d0 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.885+0000 7efebeffd700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec00c3ef0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7efeb000d890 tx=0x7efeb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.886+0000 7efeb77fe700 1 -- 172.21.15.45:0/4094051622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00125f0 con 0x7efec015d3d0 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.886+0000 7efeb77fe700 1 -- 172.21.15.45:0/4094051622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0012750 con 0x7efec015d3d0 2023-10-06T05:27:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.886+0000 7efeb77fe700 1 -- 172.21.15.45:0/4094051622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0012af0 con 0x7efec015d3d0 2023-10-06T05:27:49.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.886+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec00c4fe0 con 0x7efec015d3d0 2023-10-06T05:27:49.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.886+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec00c8e30 con 0x7efec015d3d0 2023-10-06T05:27:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.888+0000 7efeb77fe700 1 -- 172.21.15.45:0/4094051622 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efeb001e460 con 0x7efec015d3d0 2023-10-06T05:27:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.888+0000 7efeb77fe700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efea0066f30 0x7efea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.888+0000 7efeb77fe700 1 -- 172.21.15.45:0/4094051622 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7efeb0084dd0 con 0x7efec015d3d0 2023-10-06T05:27:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.888+0000 7efebf7fe700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efea0066f30 0x7efea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.889+0000 7efebf7fe700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efea0066f30 0x7efea00693e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efea8006fd0 tx=0x7efea8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.890+0000 7efecef2ab80 1 Processor -- start 2023-10-06T05:27:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.890+0000 7efecef2ab80 1 -- start start 2023-10-06T05:27:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb5788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb55dfc0 0x5635fb581010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb5819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efecef2ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb44b390 con 0x5635fb55dfc0 2023-10-06T05:27:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb4881b0 con 0x5635fb581550 2023-10-06T05:27:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635fb485b70 con 0x5635fb5784d0 2023-10-06T05:27:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb55dfc0 0x5635fb581010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb55dfc0 0x5635fb581010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54458/0 (socket says 172.21.15.45:54458) 2023-10-06T05:27:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efebeffd700 1 -- 172.21.15.45:0/1615264715 learned_addr learned my addr 172.21.15.45:0/1615264715 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efebffff700 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb5819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.891+0000 7efebf7fe700 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb5788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efebeffd700 1 -- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 msgr2=0x5635fb5788a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efebeffd700 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb5788a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efebeffd700 1 -- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 msgr2=0x5635fb5819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efebeffd700 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb5819c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efebeffd700 1 -- 172.21.15.45:0/1615264715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635fb287810 con 0x5635fb55dfc0 2023-10-06T05:27:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efebeffd700 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb55dfc0 0x5635fb581010 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7efeb0082ff0 tx=0x7efeb0082f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f596dfb5322e59a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efeb4ff9700 1 -- 172.21.15.45:0/1615264715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0058c20 con 0x5635fb55dfc0 2023-10-06T05:27:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efeb4ff9700 1 -- 172.21.15.45:0/1615264715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb005c970 con 0x5635fb55dfc0 2023-10-06T05:27:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.892+0000 7efeb4ff9700 1 -- 172.21.15.45:0/1615264715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0053b10 con 0x5635fb55dfc0 2023-10-06T05:27:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 -- 172.21.15.45:0/1615264715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb55dfc0 msgr2=0x5635fb581010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb55dfc0 0x5635fb581010 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7efeb0082ff0 tx=0x7efeb0082f20 comp rx=0 tx=0).stop 2023-10-06T05:27:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 -- 172.21.15.45:0/1615264715 shutdown_connections 2023-10-06T05:27:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb5819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb5788a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 --2- 172.21.15.45:0/1615264715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb55dfc0 0x5635fb581010 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 -- 172.21.15.45:0/1615264715 >> 172.21.15.45:0/1615264715 conn(0x5635fb57b900 msgr2=0x5635fb57bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 -- 172.21.15.45:0/1615264715 shutdown_connections 2023-10-06T05:27:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.893+0000 7efecef2ab80 1 -- 172.21.15.45:0/1615264715 wait complete. 2023-10-06T05:27:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 -- start start 2023-10-06T05:27:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 Processor -- start 2023-10-06T05:27:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb55dfc0 0x5635fb5331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb579800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb579d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb481960 con 0x5635fb5784d0 2023-10-06T05:27:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb42da00 con 0x5635fb581550 2023-10-06T05:27:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.894+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635fb42d6a0 con 0x5635fb55dfc0 2023-10-06T05:27:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efecef2ab80 10 client.0 _ll_register_callbacks cb 0x5635fb290a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb579800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb579800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54460/0 (socket says 172.21.15.45:54460) 2023-10-06T05:27:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 -- 172.21.15.45:0/2450338015 learned_addr learned my addr 172.21.15.45:0/2450338015 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebf7fe700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb55dfc0 0x5635fb5331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebffff700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb579d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 -- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb55dfc0 msgr2=0x5635fb5331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb55dfc0 0x5635fb5331c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 -- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 msgr2=0x5635fb579d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb579d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 -- 172.21.15.45:0/2450338015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec00c8e30 con 0x5635fb5784d0 2023-10-06T05:27:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebf7fe700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb55dfc0 0x5635fb5331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebffff700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb579d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efebeffd700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb579800 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7efeb0051ed0 tx=0x7efeb006b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c859783fb2689eda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efe9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efe9dffb700 1 -- 172.21.15.45:0/2450338015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0012ee0 con 0x5635fb5784d0 2023-10-06T05:27:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efe9dffb700 1 -- 172.21.15.45:0/2450338015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7efeb0055c30 con 0x5635fb5784d0 2023-10-06T05:27:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635fb525de0 con 0x5635fb5784d0 2023-10-06T05:27:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efe9dffb700 1 -- 172.21.15.45:0/2450338015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb0054ec0 con 0x5635fb5784d0 2023-10-06T05:27:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efecef2ab80 10 client.11151 fetch_fsmap learned FSMap version 364 2023-10-06T05:27:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.895+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) v3 -- 0x5635fb374120 con 0x5635fb5784d0 2023-10-06T05:27:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efe9dffb700 1 -- 172.21.15.45:0/2450338015 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5635fb525de0 con 0x5635fb5784d0 2023-10-06T05:27:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efe9dffb700 1 -- 172.21.15.45:0/2450338015 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 364) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efeb005c4e0 con 0x5635fb5784d0 2023-10-06T05:27:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efecef2ab80 10 client.11151 fetch_fsmap finished waiting for FSMap version 364 2023-10-06T05:27:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efecef2ab80 10 client.11151 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efecef2ab80 20 client.11151 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb58aca0 0x5635fb58b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efecef2ab80 4 client.11151 mds_command: new command op to 11019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635fb502bb0 con 0x5635fb58aca0 2023-10-06T05:27:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efebf7fe700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb58aca0 0x5635fb58b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efe9dffb700 10 client.11151 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.896+0000 7efebf7fe700 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb58aca0 0x5635fb58b070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179ba187f6a5a32e server_cookie=342b6744ca64e1c2 in_seq=0 out_seq=0 2023-10-06T05:27:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efe9dffb700 1 -- 172.21.15.45:0/2450338015 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5635fb502bb0 con 0x5635fb58aca0 2023-10-06T05:27:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efe9dffb700 10 client.11151 handle_command_reply: tid=0 2023-10-06T05:27:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 client.11151 shutdown 2023-10-06T05:27:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 msgr2=0x5635fb579800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb579800 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7efeb0051ed0 tx=0x7efeb006b000 comp rx=0 tx=0).stop 2023-10-06T05:27:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 shutdown_connections 2023-10-06T05:27:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb58aca0 0x5635fb58b070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb581550 0x5635fb579d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb55dfc0 0x5635fb5331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2450338015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb579800 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efe9dffb700 0 client.11151 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 >> 172.21.15.45:0/2450338015 conn(0x5635fb57b900 msgr2=0x5635fb5344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 shutdown_connections 2023-10-06T05:27:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 1 -- 172.21.15.45:0/2450338015 wait complete. 2023-10-06T05:27:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.897+0000 7efecef2ab80 20 client.11151 trim_cache size 0 max 16384 2023-10-06T05:27:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 Processor -- start 2023-10-06T05:27:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 -- start start 2023-10-06T05:27:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb5531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb527e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb581550 0x5635fb531120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb319260 con 0x5635fb5784d0 2023-10-06T05:27:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb57c600 con 0x5635fb55dfc0 2023-10-06T05:27:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635fb5552f0 con 0x5635fb581550 2023-10-06T05:27:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb527e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efebeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb527e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54464/0 (socket says 172.21.15.45:54464) 2023-10-06T05:27:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.911+0000 7efebeffd700 1 -- 172.21.15.45:0/94483257 learned_addr learned my addr 172.21.15.45:0/94483257 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebf7fe700 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb5531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebffff700 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb581550 0x5635fb531120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebeffd700 1 -- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb581550 msgr2=0x5635fb531120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebeffd700 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb581550 0x5635fb531120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebeffd700 1 -- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 msgr2=0x5635fb5531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebeffd700 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb5531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebeffd700 1 -- 172.21.15.45:0/94483257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635fb450440 con 0x5635fb5784d0 2023-10-06T05:27:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efebeffd700 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb527e40 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb006e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c997730cc75d34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efeb57fa700 1 -- 172.21.15.45:0/94483257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0054ec0 con 0x5635fb5784d0 2023-10-06T05:27:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efeb57fa700 1 -- 172.21.15.45:0/94483257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0012ee0 con 0x5635fb5784d0 2023-10-06T05:27:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.912+0000 7efeb57fa700 1 -- 172.21.15.45:0/94483257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0055c30 con 0x5635fb5784d0 2023-10-06T05:27:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 -- 172.21.15.45:0/94483257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 msgr2=0x5635fb527e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb527e40 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb006e000 comp rx=0 tx=0).stop 2023-10-06T05:27:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 -- 172.21.15.45:0/94483257 shutdown_connections 2023-10-06T05:27:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb581550 0x5635fb531120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb5531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 --2- 172.21.15.45:0/94483257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb5784d0 0x5635fb527e40 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 -- 172.21.15.45:0/94483257 >> 172.21.15.45:0/94483257 conn(0x5635fb57b900 msgr2=0x5635fb5331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 -- 172.21.15.45:0/94483257 shutdown_connections 2023-10-06T05:27:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 -- 172.21.15.45:0/94483257 wait complete. 2023-10-06T05:27:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 -- start start 2023-10-06T05:27:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.913+0000 7efecef2ab80 1 Processor -- start 2023-10-06T05:27:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb52d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb52dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efecef2ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb581550 0x5635fb579150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efecef2ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb5617e0 con 0x5635fb581550 2023-10-06T05:27:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fb556120 con 0x5635fb55dfc0 2023-10-06T05:27:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efecef2ab80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635fb556400 con 0x5635fb5784d0 2023-10-06T05:27:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efecef2ab80 10 client.0 _ll_register_callbacks cb 0x5635fb290a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efebffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb581550 0x5635fb579150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efebffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb581550 0x5635fb579150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54474/0 (socket says 172.21.15.45:54474) 2023-10-06T05:27:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efebffff700 1 -- 172.21.15.45:0/2869704928 learned_addr learned my addr 172.21.15.45:0/2869704928 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efebeffd700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb52dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.914+0000 7efebf7fe700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb52d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efebffff700 1 -- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 msgr2=0x5635fb52dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efebffff700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb52dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efebffff700 1 -- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 msgr2=0x5635fb52d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efebffff700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb52d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efebffff700 1 -- 172.21.15.45:0/2869704928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635fb374120 con 0x5635fb581550 2023-10-06T05:27:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efebeffd700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb52dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efebffff700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb581550 0x5635fb579150 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7efec006c4b0 tx=0x7efeb8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73d41e0b06eabb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efe9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efe9dffb700 1 -- 172.21.15.45:0/2869704928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8023070 con 0x5635fb581550 2023-10-06T05:27:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efeb005c970 con 0x5635fb581550 2023-10-06T05:27:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efe9dffb700 1 -- 172.21.15.45:0/2869704928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7efeb8007200 con 0x5635fb581550 2023-10-06T05:27:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efe9dffb700 1 -- 172.21.15.45:0/2869704928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb8007e90 con 0x5635fb581550 2023-10-06T05:27:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 10 client.11157 fetch_fsmap learned FSMap version 364 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) v3 -- 0x5635fb2e8250 con 0x5635fb581550 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efe9dffb700 1 -- 172.21.15.45:0/2869704928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb005c970 con 0x5635fb581550 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efe9dffb700 1 -- 172.21.15.45:0/2869704928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 364) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efeb800b050 con 0x5635fb581550 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 10 client.11157 fetch_fsmap finished waiting for FSMap version 364 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 10 client.11157 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 20 client.11157 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb579750 0x5635fb579b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 4 client.11157 mds_command: new command op to 11019 tid=0[{"prefix": "damage rm", "damage_id": 2157878309}] 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.915+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2157878309}) v1 -- 0x5635fb424950 con 0x5635fb579750 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efebf7fe700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb579750 0x5635fb579b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efe9dffb700 10 client.11157 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efebf7fe700 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb579750 0x5635fb579b20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3960b94316feee server_cookie=dca2fc442e376c1c in_seq=0 out_seq=0 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efe9dffb700 1 -- 172.21.15.45:0/2869704928 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x5635fb424950 con 0x5635fb579750 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efe9dffb700 10 client.11157 handle_command_reply: tid=0 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 client.11157 shutdown 2023-10-06T05:27:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb581550 msgr2=0x5635fb579150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb581550 0x5635fb579150 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7efec006c4b0 tx=0x7efeb8018000 comp rx=0 tx=0).stop 2023-10-06T05:27:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 shutdown_connections 2023-10-06T05:27:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x5635fb579750 0x5635fb579b20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635fb581550 0x5635fb579150 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635fb55dfc0 0x5635fb52d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 --2- 172.21.15.45:0/2869704928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5635fb5784d0 0x5635fb52dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efe9dffb700 0 client.11157 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 >> 172.21.15.45:0/2869704928 conn(0x5635fb57b900 msgr2=0x5635fb57bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 shutdown_connections 2023-10-06T05:27:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.916+0000 7efecef2ab80 1 -- 172.21.15.45:0/2869704928 wait complete. 2023-10-06T05:27:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efecef2ab80 20 client.11157 trim_cache size 0 max 16384 2023-10-06T05:27:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efea0066f30 msgr2=0x7efea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efea0066f30 0x7efea00693e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efea8006fd0 tx=0x7efea8008040 comp rx=0 tx=0).stop 2023-10-06T05:27:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 msgr2=0x7efec00c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec00c3ef0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7efeb000d890 tx=0x7efeb0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 shutdown_connections 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efea0066f30 0x7efea00693e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec016a3b0 0x7efec00c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec0159150 0x7efec0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 --2- 172.21.15.45:0/4094051622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec015d3d0 0x7efec00c3ef0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 >> 172.21.15.45:0/4094051622 conn(0x7efec00691c0 msgr2=0x7efec00c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 shutdown_connections 2023-10-06T05:27:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.917+0000 7efec5c16700 1 -- 172.21.15.45:0/4094051622 wait complete. 2023-10-06T05:27:49.942 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100000001f6.00000000 2023-10-06T05:27:49.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.987+0000 7fd2979b3040 1 Processor -- start 2023-10-06T05:27:49.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.987+0000 7fd2979b3040 1 -- start start 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2979b3040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2800030b0 0x55cc31b96a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2979b3040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc31b9c8b0 0x55cc31ad3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2979b3040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b14c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2979b3040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc31a1b980 con 0x55cc31b16f10 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2979b3040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc31a1b620 con 0x55cc31b9c8b0 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2979b3040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc31a12400 con 0x7fd2800030b0 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b14c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b14c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54494/0 (socket says 172.21.15.45:54494) 2023-10-06T05:27:49.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 -- 172.21.15.45:0/662679211 learned_addr learned my addr 172.21.15.45:0/662679211 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd290bd1700 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2800030b0 0x55cc31b96a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd28bfff700 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc31b9c8b0 0x55cc31ad3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 -- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2800030b0 msgr2=0x55cc31b96a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2800030b0 0x55cc31b96a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 -- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc31b9c8b0 msgr2=0x55cc31ad3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc31b9c8b0 0x55cc31ad3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 -- 172.21.15.45:0/662679211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc31a2a3d0 con 0x55cc31b16f10 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd2913d2700 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b14c20 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x55cc31ad3760 tx=0x55cc31ba5340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a3256c0b51d798 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd28affd700 1 -- 172.21.15.45:0/662679211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c006e90 con 0x55cc31b16f10 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd28affd700 1 -- 172.21.15.45:0/662679211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c003450 con 0x55cc31b16f10 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.988+0000 7fd28affd700 1 -- 172.21.15.45:0/662679211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c003830 con 0x55cc31b16f10 2023-10-06T05:27:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 -- 172.21.15.45:0/662679211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 msgr2=0x55cc31b14c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b14c20 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x55cc31ad3760 tx=0x55cc31ba5340 comp rx=0 tx=0).stop 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 -- 172.21.15.45:0/662679211 shutdown_connections 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b14c20 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2800030b0 0x55cc31b96a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 --2- 172.21.15.45:0/662679211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc31b9c8b0 0x55cc31ad3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 -- 172.21.15.45:0/662679211 >> 172.21.15.45:0/662679211 conn(0x55cc31ad0bd0 msgr2=0x55cc31b9cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 -- 172.21.15.45:0/662679211 shutdown_connections 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.989+0000 7fd2979b3040 1 -- 172.21.15.45:0/662679211 wait complete. 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.990+0000 7fd2979b3040 1 Processor -- start 2023-10-06T05:27:49.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.990+0000 7fd2979b3040 1 -- start start 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd2979b3040 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b96170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd2979b3040 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc31b9c8b0 0x55cc31ba2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd2979b3040 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800030b0 0x55cc31ba3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd2979b3040 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc31afd940 con 0x55cc31b16f10 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd2979b3040 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc31ba1950 con 0x7fd2800030b0 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd2979b3040 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc31b00370 con 0x55cc31b9c8b0 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b96170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b96170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54502/0 (socket says 172.21.15.45:54502) 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 -- 172.21.15.45:0/442324670 learned_addr learned my addr 172.21.15.45:0/442324670 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd28bfff700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc31b9c8b0 0x55cc31ba2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd2913d2700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800030b0 0x55cc31ba3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:49.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 -- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc31b9c8b0 msgr2=0x55cc31ba2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc31b9c8b0 0x55cc31ba2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 -- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800030b0 msgr2=0x55cc31ba3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:49.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800030b0 0x55cc31ba3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:49.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.991+0000 7fd290bd1700 1 -- 172.21.15.45:0/442324670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc31a441c0 con 0x55cc31b16f10 2023-10-06T05:27:49.997 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:27:49.992+0000 7fdb6559f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-06T05:27:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.992+0000 7fd290bd1700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b96170 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fd284008af0 tx=0x7fd28400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.992+0000 7fd2897fa700 1 -- 172.21.15.45:0/442324670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28400f040 con 0x55cc31b16f10 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.992+0000 7fd2897fa700 1 -- 172.21.15.45:0/442324670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd284008e00 con 0x55cc31b16f10 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.992+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cc31a1dae0 con 0x55cc31b16f10 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.992+0000 7fd2897fa700 1 -- 172.21.15.45:0/442324670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284013bb0 con 0x55cc31b16f10 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.992+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc31a1d540 con 0x55cc31b16f10 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.994+0000 7fd2897fa700 1 -- 172.21.15.45:0/442324670 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd28401f480 con 0x55cc31b16f10 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.994+0000 7fd2897fa700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2740658c0 0x7fd274067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.994+0000 7fd2897fa700 1 -- 172.21.15.45:0/442324670 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fd28401a070 con 0x55cc31b16f10 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.995+0000 7fd28bfff700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2740658c0 0x7fd274067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.995+0000 7fd2979b3040 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc31b9ba50 0x55cc31b9be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.995+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- osd_op(unknown.0.0:1 16.a 16:57fe7535:::100000001f6.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e146) v8 -- 0x55cc31b9c390 con 0x55cc31b9ba50 2023-10-06T05:27:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.995+0000 7fd2913d2700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc31b9ba50 0x55cc31b9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.995+0000 7fd2913d2700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc31b9ba50 0x55cc31b9be50 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.995+0000 7fd28bfff700 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2740658c0 0x7fd274067d70 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0096e0 tx=0x7fd27c018000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.996+0000 7fd2913d2700 1 -- 172.21.15.45:0/442324670 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== osd_op_reply(1 100000001f6.00000000 [delete] v146'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd28c0035c0 con 0x55cc31b9ba50 2023-10-06T05:27:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.996+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc31b9ba50 msgr2=0x55cc31b9be50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.996+0000 7fd2979b3040 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x55cc31b9ba50 0x55cc31b9be50 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.996+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2740658c0 msgr2=0x7fd274067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.996+0000 7fd2979b3040 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd2740658c0 0x7fd274067d70 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0096e0 tx=0x7fd27c018000 comp rx=0 tx=0).stop 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.996+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 msgr2=0x55cc31b96170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.996+0000 7fd2979b3040 1 --2- 172.21.15.45:0/442324670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc31b16f10 0x55cc31b96170 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fd284008af0 tx=0x7fd28400c040 comp rx=0 tx=0).stop 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.997+0000 7fd2913d2700 1 -- 172.21.15.45:0/442324670 reap_dead start 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.997+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 shutdown_connections 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.997+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 >> 172.21.15.45:0/442324670 conn(0x55cc31ad0bd0 msgr2=0x55cc31ad2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.997+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 shutdown_connections 2023-10-06T05:27:50.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:49.997+0000 7fd2979b3040 1 -- 172.21.15.45:0/442324670 wait complete. 2023-10-06T05:27:50.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:27:50.004 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink2) 2023-10-06T05:27:50.027 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-10-06T05:27:50.024+0000 7f7986960700 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2023-10-06T05:27:50.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:50.028 INFO:teuthology.orchestra.run.smithi045.stderr:stat: cannot statx 'testdir/hardlink2': Input/output error 2023-10-06T05:27:50.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.204+0000 7f00c9f9e700 1 Processor -- start 2023-10-06T05:27:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.204+0000 7f00c9f9e700 1 -- start start 2023-10-06T05:27:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c9f9e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c40b0460 0x7f00c40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c9f9e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00c40c2b80 0x7f00c40c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c9f9e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c4068ee0 con 0x7f00c4069370 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c9f9e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c40cdc60 con 0x7f00c40c2b80 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c9f9e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c3510 con 0x7f00c40b0460 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00c40c2b80 0x7f00c40c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00c40c2b80 0x7f00c40c2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54514/0 (socket says 172.21.15.45:54514) 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c3fff700 1 -- 172.21.15.45:0/2168397135 learned_addr learned my addr 172.21.15.45:0/2168397135 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.205+0000 7f00c8f9c700 1 --2- 172.21.15.45:0/2168397135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c40b0460 0x7f00c40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c3fff700 1 -- 172.21.15.45:0/2168397135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c40b0460 msgr2=0x7f00c40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c3fff700 1 --2- 172.21.15.45:0/2168397135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c40b0460 0x7f00c40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c3fff700 1 -- 172.21.15.45:0/2168397135 >> v1:172.21.15.144:6789/0 conn(0x7f00c4069370 legacy=0x7f00c40697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c3fff700 1 -- 172.21.15.45:0/2168397135 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40688d0 con 0x7f00c40c2b80 2023-10-06T05:27:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c3fff700 1 --2- 172.21.15.45:0/2168397135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40c2b80 0x7f00c40c2fd0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0099b0 tx=0x7f00ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2182b6e2935f6aa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c37fe700 1 -- 172.21.15.45:0/2168397135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac009ea0 con 0x7f00c40c2b80 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c37fe700 1 -- 172.21.15.45:0/2168397135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac00b860 con 0x7f00c40c2b80 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c37fe700 1 -- 172.21.15.45:0/2168397135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00bc00 con 0x7f00c40c2b80 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c9f9e700 1 -- 172.21.15.45:0/2168397135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40c2b80 msgr2=0x7f00c40c2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.206+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/2168397135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40c2b80 0x7f00c40c2fd0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0099b0 tx=0x7f00ac009250 comp rx=0 tx=0).stop 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.207+0000 7f00c9f9e700 1 -- 172.21.15.45:0/2168397135 shutdown_connections 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.207+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/2168397135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c40b0460 0x7f00c40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.207+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/2168397135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40c2b80 0x7f00c40c2fd0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.207+0000 7f00c9f9e700 1 -- 172.21.15.45:0/2168397135 >> 172.21.15.45:0/2168397135 conn(0x7f00c4067e00 msgr2=0x7f00c40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.207+0000 7f00c9f9e700 1 -- 172.21.15.45:0/2168397135 shutdown_connections 2023-10-06T05:27:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.207+0000 7f00c9f9e700 1 -- 172.21.15.45:0/2168397135 wait complete. 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.208+0000 7f00c9f9e700 1 Processor -- start 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.208+0000 7f00c9f9e700 1 -- start start 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c9f9e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c4069370 0x7f00c415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c9f9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40b0460 0x7f00c40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c9f9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00c40c2b80 0x7f00c414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c9f9e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c415e460 con 0x7f00c40b0460 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c9f9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40b1e00 con 0x7f00c40c2b80 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c9f9e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c415eca0 con 0x7f00c4069370 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40b0460 0x7f00c40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40b0460 0x7f00c40b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54518/0 (socket says 172.21.15.45:54518) 2023-10-06T05:27:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 -- 172.21.15.45:0/654617514 learned_addr learned my addr 172.21.15.45:0/654617514 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c8f9c700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c4069370 0x7f00c415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c979d700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00c40c2b80 0x7f00c414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 -- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c4069370 msgr2=0x7f00c415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c4069370 0x7f00c415e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 -- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00c40c2b80 msgr2=0x7f00c414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00c40c2b80 0x7f00c414ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 -- 172.21.15.45:0/654617514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0094f0 con 0x7f00c40b0460 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.209+0000 7f00c3fff700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40b0460 0x7f00c40b2290 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f00ac006fd0 tx=0x7f00ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.210+0000 7f00c0ff9700 1 -- 172.21.15.45:0/654617514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00bbe0 con 0x7f00c40b0460 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.210+0000 7f00c0ff9700 1 -- 172.21.15.45:0/654617514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac000cc0 con 0x7f00c40b0460 2023-10-06T05:27:50.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.210+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c40b11f0 con 0x7f00c40b0460 2023-10-06T05:27:50.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.210+0000 7f00c0ff9700 1 -- 172.21.15.45:0/654617514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac01a680 con 0x7f00c40b0460 2023-10-06T05:27:50.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.210+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c40b17b0 con 0x7f00c40b0460 2023-10-06T05:27:50.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.211+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c40b1fb0 con 0x7f00c40b0460 2023-10-06T05:27:50.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.212+0000 7f00c0ff9700 1 -- 172.21.15.45:0/654617514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00ac01a9f0 con 0x7f00c40b0460 2023-10-06T05:27:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.212+0000 7f00c0ff9700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00a4066f30 0x7f00a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.212+0000 7f00c0ff9700 1 -- 172.21.15.45:0/654617514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f00ac086ab0 con 0x7f00c40b0460 2023-10-06T05:27:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.212+0000 7f00c8f9c700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00a4066f30 0x7f00a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.213+0000 7f00c8f9c700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00a4066f30 0x7f00a40693e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009da0 tx=0x7f00b4009120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.216+0000 7f00c0ff9700 1 -- 172.21.15.45:0/654617514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00ac053ab0 con 0x7f00c40b0460 2023-10-06T05:27:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.467+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00c40b1fb0 con 0x7f00c40b0460 2023-10-06T05:27:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.467+0000 7f00c0ff9700 1 -- 172.21.15.45:0/654617514 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f00c40b1fb0 con 0x7f00c40b0460 2023-10-06T05:27:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:50.469 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":365,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":365,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:49.992996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:50.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00a4066f30 msgr2=0x7f00a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00a4066f30 0x7f00a40693e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009da0 tx=0x7f00b4009120 comp rx=0 tx=0).stop 2023-10-06T05:27:50.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40b0460 msgr2=0x7f00c40b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40b0460 0x7f00c40b2290 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f00ac006fd0 tx=0x7f00ac017000 comp rx=0 tx=0).stop 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 shutdown_connections 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f00a4066f30 0x7f00a40693e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00c40c2b80 0x7f00c414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00c4069370 0x7f00c415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 --2- 172.21.15.45:0/654617514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c40b0460 0x7f00c40b2290 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 >> 172.21.15.45:0/654617514 conn(0x7f00c4067e00 msgr2=0x7f00c406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.469+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 shutdown_connections 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.470+0000 7f00c9f9e700 1 -- 172.21.15.45:0/654617514 wait complete. 2023-10-06T05:27:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 365 2023-10-06T05:27:50.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-10-06T05:27:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.616+0000 7f1ac2532700 1 Processor -- start 2023-10-06T05:27:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.616+0000 7f1ac2532700 1 -- start start 2023-10-06T05:27:50.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac2532700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1abc0cd360 0x7f1abc0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac2532700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1abc0ce6a0 con 0x7f1abc0c3db0 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac2532700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1abc159880 con 0x7f1abc0b0610 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac2532700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc0690f0 con 0x7f1abc0cd360 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac0acf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1abc0cd360 0x7f1abc0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1abb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1abc0c3db0 0x7f1abc069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53236/0 (socket says 172.21.15.45:53236) 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1abb7fe700 1 -- 172.21.15.45:0/3724058384 learned_addr learned my addr 172.21.15.45:0/3724058384 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac0acf700 1 -- 172.21.15.45:0/3724058384 >> v1:172.21.15.144:6789/0 conn(0x7f1abc0b0610 legacy=0x7f1abc0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac0acf700 1 -- 172.21.15.45:0/3724058384 >> v1:172.21.15.45:6789/0 conn(0x7f1abc0c3db0 legacy=0x7f1abc069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1ac0acf700 1 -- 172.21.15.45:0/3724058384 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc0692e0 con 0x7f1abc0cd360 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.617+0000 7f1abaffd700 1 -- 172.21.15.45:0/3724058384 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722017457 0 0) 0x7f1abc0ce6a0 con 0x7f1abc0c3db0 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/3724058384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0cd360 0x7f1abc0c85d0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e60 tx=0x7f1ab4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b335bec84e4f231 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1abaffd700 1 -- 172.21.15.45:0/3724058384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400e070 con 0x7f1abc0cd360 2023-10-06T05:27:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1abaffd700 1 -- 172.21.15.45:0/3724058384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab400bc30 con 0x7f1abc0cd360 2023-10-06T05:27:50.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1abaffd700 1 -- 172.21.15.45:0/3724058384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4012430 con 0x7f1abc0cd360 2023-10-06T05:27:50.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac2532700 1 -- 172.21.15.45:0/3724058384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0cd360 msgr2=0x7f1abc0c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac2532700 1 --2- 172.21.15.45:0/3724058384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0cd360 0x7f1abc0c85d0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e60 tx=0x7f1ab4009780 comp rx=0 tx=0).stop 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac2532700 1 -- 172.21.15.45:0/3724058384 shutdown_connections 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac2532700 1 --2- 172.21.15.45:0/3724058384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0cd360 0x7f1abc0c85d0 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac2532700 1 -- 172.21.15.45:0/3724058384 >> 172.21.15.45:0/3724058384 conn(0x7f1abc067e00 msgr2=0x7f1abc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac2532700 1 -- 172.21.15.45:0/3724058384 shutdown_connections 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.618+0000 7f1ac2532700 1 -- 172.21.15.45:0/3724058384 wait complete. 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.620+0000 7f1ac2532700 1 Processor -- start 2023-10-06T05:27:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.620+0000 7f1ac2532700 1 -- start start 2023-10-06T05:27:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1ac2532700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1abc0b0610 0x7f1abc14f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1ac2532700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0c3db0 0x7f1abc0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1ac2532700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1abc0cd360 0x7f1abc158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1ac2532700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc15d5e0 con 0x7f1abc0c3db0 2023-10-06T05:27:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1ac2532700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc14f560 con 0x7f1abc0b0610 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1ac2532700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0b3850 con 0x7f1abc0cd360 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0c3db0 0x7f1abc0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0c3db0 0x7f1abc0b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54536/0 (socket says 172.21.15.45:54536) 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 -- 172.21.15.45:0/1822395012 learned_addr learned my addr 172.21.15.45:0/1822395012 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abbfff700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1abc0b0610 0x7f1abc14f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1abc0cd360 0x7f1abc158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 -- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1abc0cd360 msgr2=0x7f1abc158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1abc0cd360 0x7f1abc158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 -- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1abc0b0610 msgr2=0x7f1abc14f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1abc0b0610 0x7f1abc14f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 -- 172.21.15.45:0/1822395012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009a20 con 0x7f1abc0c3db0 2023-10-06T05:27:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0c3db0 0x7f1abc0b2fa0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000c7e0 tx=0x7f1ab0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.621+0000 7f1aa3fff700 1 -- 172.21.15.45:0/1822395012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0005b30 con 0x7f1abc0c3db0 2023-10-06T05:27:50.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.622+0000 7f1aa3fff700 1 -- 172.21.15.45:0/1822395012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab0005c90 con 0x7f1abc0c3db0 2023-10-06T05:27:50.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.622+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc0b3b30 con 0x7f1abc0c3db0 2023-10-06T05:27:50.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.622+0000 7f1aa3fff700 1 -- 172.21.15.45:0/1822395012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0007300 con 0x7f1abc0c3db0 2023-10-06T05:27:50.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.622+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc15d270 con 0x7f1abc0c3db0 2023-10-06T05:27:50.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.623+0000 7f1aa3fff700 1 -- 172.21.15.45:0/1822395012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ab001b3b0 con 0x7f1abc0c3db0 2023-10-06T05:27:50.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.623+0000 7f1aa3fff700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1aa8066f30 0x7f1aa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.624+0000 7f1aa3fff700 1 -- 172.21.15.45:0/1822395012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f1ab0083e60 con 0x7f1abc0c3db0 2023-10-06T05:27:50.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.624+0000 7f1abbfff700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1aa8066f30 0x7f1aa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.624+0000 7f1abbfff700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1aa8066f30 0x7f1aa80693e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007fd0 tx=0x7f1aac007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.625+0000 7f1acb846b80 1 Processor -- start 2023-10-06T05:27:50.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.625+0000 7f1acb846b80 1 -- start start 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.626+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def4e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.626+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def57030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.626+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.626+0000 7f1acb846b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dee213b0 con 0x55f8def4e4f0 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.626+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dee5e1d0 con 0x55f8def57570 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.626+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8dee5bb90 con 0x55f8def33fe0 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def4e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def4e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54550/0 (socket says 172.21.15.45:54550) 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 -- 172.21.15.45:0/1727489315 learned_addr learned my addr 172.21.15.45:0/1727489315 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def57030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 -- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 msgr2=0x55f8def57030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def57030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 -- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 msgr2=0x55f8def579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 -- 172.21.15.45:0/1727489315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dec5d830 con 0x55f8def4e4f0 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1abbfff700 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def4e8c0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f1aac002a70 tx=0x7f1aac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523b9ebb47772fa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1aa17fa700 1 -- 172.21.15.45:0/1727489315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0037c0 con 0x55f8def4e4f0 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1aa17fa700 1 -- 172.21.15.45:0/1727489315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac014660 con 0x55f8def4e4f0 2023-10-06T05:27:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.627+0000 7f1aa17fa700 1 -- 172.21.15.45:0/1727489315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0149e0 con 0x55f8def4e4f0 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 -- 172.21.15.45:0/1727489315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 msgr2=0x55f8def4e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def4e8c0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f1aac002a70 tx=0x7f1aac015000 comp rx=0 tx=0).stop 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 -- 172.21.15.45:0/1727489315 shutdown_connections 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def4e8c0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 --2- 172.21.15.45:0/1727489315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def57030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 -- 172.21.15.45:0/1727489315 >> 172.21.15.45:0/1727489315 conn(0x55f8def51920 msgr2=0x55f8def51cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 -- 172.21.15.45:0/1727489315 shutdown_connections 2023-10-06T05:27:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.628+0000 7f1acb846b80 1 -- 172.21.15.45:0/1727489315 wait complete. 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.629+0000 7f1acb846b80 1 -- start start 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.629+0000 7f1acb846b80 1 Processor -- start 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def28ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1acb846b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dee57980 con 0x55f8def4e4f0 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dee03a20 con 0x55f8def57570 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8dee036c0 con 0x55f8def33fe0 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1acb846b80 10 client.0 _ll_register_callbacks cb 0x55f8dec66aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def2bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54552/0 (socket says 172.21.15.45:54552) 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 -- 172.21.15.45:0/2442534377 learned_addr learned my addr 172.21.15.45:0/2442534377 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abbfff700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def28ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 -- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 msgr2=0x55f8def28ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def28ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 -- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 msgr2=0x55f8def091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 -- 172.21.15.45:0/2442534377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc159880 con 0x55f8def4e4f0 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abbfff700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def28ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def2bcb0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000cdd0 tx=0x7f1ab0081f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09a00d0daf2f859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.630+0000 7f1a9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2442534377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0005e00 con 0x55f8def4e4f0 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2442534377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f1ab00076f0 con 0x55f8def4e4f0 2023-10-06T05:27:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8deefbe00 con 0x55f8def4e4f0 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2442534377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=365) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab005b6e0 con 0x55f8def4e4f0 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 10 client.11187 fetch_fsmap learned FSMap version 365 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=365,monmap=2+}) v3 -- 0x55f8ded4a140 con 0x55f8def4e4f0 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2442534377 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab005f080 con 0x55f8def4e4f0 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2442534377 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 365) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1ab0069030 con 0x55f8def4e4f0 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 10 client.11187 fetch_fsmap finished waiting for FSMap version 365 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 10 client.11187 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 20 client.11187 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def513a0 0x55f8def280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 4 client.11187 mds_command: new command op to 11019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8deed8bd0 con 0x55f8def513a0 2023-10-06T05:27:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.631+0000 7f1abbfff700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def513a0 0x55f8def280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1abbfff700 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def513a0 0x55f8def280a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca17f808af2268 server_cookie=12ba708fd17e8e7d in_seq=0 out_seq=0 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1a9a7fc700 10 client.11187 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/2442534377 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f8deed8bd0 con 0x55f8def513a0 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1a9a7fc700 10 client.11187 handle_command_reply: tid=0 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 client.11187 shutdown 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 msgr2=0x55f8def2bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def2bcb0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000cdd0 tx=0x7f1ab0081f10 comp rx=0 tx=0).stop 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 shutdown_connections 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def513a0 0x55f8def280a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def33fe0 0x55f8def28ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1a9a7fc700 0 client.11187 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2442534377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def4e4f0 0x55f8def2bcb0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.632+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 >> 172.21.15.45:0/2442534377 conn(0x55f8def51920 msgr2=0x55f8def0a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.633+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 shutdown_connections 2023-10-06T05:27:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.633+0000 7f1acb846b80 1 -- 172.21.15.45:0/2442534377 wait complete. 2023-10-06T05:27:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.633+0000 7f1acb846b80 20 client.11187 trim_cache size 0 max 16384 2023-10-06T05:27:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.646+0000 7f1acb846b80 1 Processor -- start 2023-10-06T05:27:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.646+0000 7f1acb846b80 1 -- start start 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def33fe0 0x55f8def30850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8def04aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def57570 0x55f8deefd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1acb846b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8decef280 con 0x55f8def57570 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8def03160 con 0x55f8def33fe0 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8def2bcd0 con 0x55f8def4e4f0 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def57570 0x55f8deefd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def57570 0x55f8deefd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54558/0 (socket says 172.21.15.45:54558) 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 -- 172.21.15.45:0/2994277499 learned_addr learned my addr 172.21.15.45:0/2994277499 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1abbfff700 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def33fe0 0x55f8def30850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 -- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 msgr2=0x55f8def04aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8def04aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8def04aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 -- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def33fe0 msgr2=0x55f8def30850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def33fe0 0x55f8def30850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1ac0acf700 1 -- 172.21.15.45:0/2994277499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dee26460 con 0x55f8def57570 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1abbfff700 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def33fe0 0x55f8def30850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.647+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8def04aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def57570 0x55f8deefd910 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f1abc06bac0 tx=0x7f1ab4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2275d6ea43457f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1aa1ffb700 1 -- 172.21.15.45:0/2994277499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4012910 con 0x55f8def57570 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1aa1ffb700 1 -- 172.21.15.45:0/2994277499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab4023040 con 0x55f8def57570 2023-10-06T05:27:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1aa1ffb700 1 -- 172.21.15.45:0/2994277499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4012ce0 con 0x55f8def57570 2023-10-06T05:27:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 -- 172.21.15.45:0/2994277499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def57570 msgr2=0x55f8deefd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def57570 0x55f8deefd910 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f1abc06bac0 tx=0x7f1ab4019000 comp rx=0 tx=0).stop 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 -- 172.21.15.45:0/2994277499 shutdown_connections 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def57570 0x55f8deefd910 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def33fe0 0x55f8def30850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 --2- 172.21.15.45:0/2994277499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8def04aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 -- 172.21.15.45:0/2994277499 >> 172.21.15.45:0/2994277499 conn(0x55f8def51920 msgr2=0x55f8def28d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.648+0000 7f1acb846b80 1 -- 172.21.15.45:0/2994277499 shutdown_connections 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.649+0000 7f1acb846b80 1 -- 172.21.15.45:0/2994277499 wait complete. 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.649+0000 7f1acb846b80 1 -- start start 2023-10-06T05:27:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.649+0000 7f1acb846b80 1 Processor -- start 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def33fe0 0x55f8deefd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8deefde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1acb846b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def53d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1acb846b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8def54c20 con 0x55f8def33fe0 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8def275b0 con 0x55f8def57570 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1acb846b80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8defb29a0 con 0x55f8def4e4f0 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1acb846b80 10 client.0 _ll_register_callbacks cb 0x55f8def2af80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def33fe0 0x55f8deefd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def33fe0 0x55f8deefd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54568/0 (socket says 172.21.15.45:54568) 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 -- 172.21.15.45:0/705213681 learned_addr learned my addr 172.21.15.45:0/705213681 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1ac0acf700 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def53d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8deefde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 -- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 msgr2=0x55f8deefde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8deefde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 -- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 msgr2=0x55f8def53d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def53d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.650+0000 7f1abbfff700 1 -- 172.21.15.45:0/705213681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc0692e0 con 0x55f8def33fe0 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1abbfff700 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def33fe0 0x55f8deefd910 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007c40 tx=0x7f1aac032250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467c204a64688249 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1a9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/705213681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac032070 con 0x55f8def33fe0 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/705213681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f1aac01c040 con 0x55f8def33fe0 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1aac014660 con 0x55f8def33fe0 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/705213681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=365) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aac0149e0 con 0x55f8def33fe0 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 10 client.11193 fetch_fsmap learned FSMap version 365 2023-10-06T05:27:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=365,monmap=2+}) v3 -- 0x55f8decbe270 con 0x55f8def33fe0 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/705213681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac014660 con 0x55f8def33fe0 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/705213681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 365) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1aac012070 con 0x55f8def33fe0 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 10 client.11193 fetch_fsmap finished waiting for FSMap version 365 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 10 client.11193 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 20 client.11193 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def54300 0x55f8def2f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 4 client.11193 mds_command: new command op to 11019 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55f8dedfa970 con 0x55f8def54300 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.651+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def54300 0x55f8def2f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1a9a7fc700 10 client.11193 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1abb7fe700 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def54300 0x55f8def2f2e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa3d158377ad3e53 server_cookie=191cb4bf72ca9944 in_seq=0 out_seq=0 2023-10-06T05:27:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1a9a7fc700 1 -- 172.21.15.45:0/705213681 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+294 (crc 0 0 0) 0x55f8dedfa970 con 0x55f8def54300 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1a9a7fc700 10 client.11193 handle_command_reply: tid=0 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 client.11193 shutdown 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def33fe0 msgr2=0x55f8deefd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def33fe0 0x55f8deefd910 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007c40 tx=0x7f1aac032250 comp rx=0 tx=0).stop 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 shutdown_connections 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55f8def54300 0x55f8def2f2e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8def57570 0x55f8def53d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8def33fe0 0x55f8deefd910 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1a9a7fc700 0 client.11193 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 --2- 172.21.15.45:0/705213681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f8def4e4f0 0x55f8deefde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 >> 172.21.15.45:0/705213681 conn(0x55f8def51920 msgr2=0x55f8def30620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 shutdown_connections 2023-10-06T05:27:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 1 -- 172.21.15.45:0/705213681 wait complete. 2023-10-06T05:27:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.652+0000 7f1acb846b80 20 client.11193 trim_cache size 0 max 16384 2023-10-06T05:27:50.661 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:50.661 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-10-06T05:27:50.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:50.661 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2023-10-06T05:27:50.661 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 2179480999, 2023-10-06T05:27:50.661 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511628279, 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/testdir/hardlink2" 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "dir_frag", 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 2385964704, 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511628278, 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: "frag": "*", 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/dir2" 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-10-06T05:27:50.662 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-10-06T05:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1aa8066f30 msgr2=0x7f1aa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1aa8066f30 0x7f1aa80693e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007fd0 tx=0x7f1aac007eb0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0c3db0 msgr2=0x7f1abc0b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0c3db0 0x7f1abc0b2fa0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000c7e0 tx=0x7f1ab0007df0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 shutdown_connections 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1aa8066f30 0x7f1aa80693e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1abc0cd360 0x7f1abc158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1abc0b0610 0x7f1abc14f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 --2- 172.21.15.45:0/1822395012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0c3db0 0x7f1abc0b2fa0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 >> 172.21.15.45:0/1822395012 conn(0x7f1abc067e00 msgr2=0x7f1abc151dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 shutdown_connections 2023-10-06T05:27:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.653+0000 7f1ac2532700 1 -- 172.21.15.45:0/1822395012 wait complete. 2023-10-06T05:27:50.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.797+0000 7fa2d7844700 1 Processor -- start 2023-10-06T05:27:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.797+0000 7fa2d7844700 1 -- start start 2023-10-06T05:27:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.798+0000 7fa2d7844700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2d00cac50 0x7fa2d006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.798+0000 7fa2d7844700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d00b0790 con 0x7fa2d006c3d0 2023-10-06T05:27:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.798+0000 7fa2d7844700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d00c8e80 con 0x7fa2d015b890 2023-10-06T05:27:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.798+0000 7fa2d7844700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d00b0570 con 0x7fa2d00cac50 2023-10-06T05:27:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.798+0000 7fa2d5de1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2d00cac50 0x7fa2d006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.798+0000 7fa2d55e0700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa2d006c3d0 0x7fa2d00afe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53252/0 (socket says 172.21.15.45:53252) 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.798+0000 7fa2d55e0700 1 -- 172.21.15.45:0/1254144647 learned_addr learned my addr 172.21.15.45:0/1254144647 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2c3fff700 1 -- 172.21.15.45:0/1254144647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780918849 0 0) 0x7fa2d00b0790 con 0x7fa2d006c3d0 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2c3fff700 1 -- 172.21.15.45:0/1254144647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2bc003570 con 0x7fa2d006c3d0 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2d5de1700 1 -- 172.21.15.45:0/1254144647 >> v1:172.21.15.144:6789/0 conn(0x7fa2d015b890 legacy=0x7fa2d0156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2d5de1700 1 -- 172.21.15.45:0/1254144647 >> v1:172.21.15.45:6789/0 conn(0x7fa2d006c3d0 legacy=0x7fa2d00afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2d5de1700 1 -- 172.21.15.45:0/1254144647 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00c9160 con 0x7fa2d00cac50 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2d5de1700 1 --2- 172.21.15.45:0/1254144647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d006c7a0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009e60 tx=0x7fa2c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3836d7b6503d7b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2c3fff700 1 -- 172.21.15.45:0/1254144647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800e070 con 0x7fa2d00cac50 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2c3fff700 1 -- 172.21.15.45:0/1254144647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c800bb60 con 0x7fa2d00cac50 2023-10-06T05:27:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.799+0000 7fa2c3fff700 1 -- 172.21.15.45:0/1254144647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8012430 con 0x7fa2d00cac50 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.800+0000 7fa2d7844700 1 -- 172.21.15.45:0/1254144647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 msgr2=0x7fa2d006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.800+0000 7fa2d7844700 1 --2- 172.21.15.45:0/1254144647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d006c7a0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009e60 tx=0x7fa2c8009780 comp rx=0 tx=0).stop 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.800+0000 7fa2d7844700 1 -- 172.21.15.45:0/1254144647 shutdown_connections 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.800+0000 7fa2d7844700 1 --2- 172.21.15.45:0/1254144647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d006c7a0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.800+0000 7fa2d7844700 1 -- 172.21.15.45:0/1254144647 >> 172.21.15.45:0/1254144647 conn(0x7fa2d0067620 msgr2=0x7fa2d0067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.800+0000 7fa2d7844700 1 -- 172.21.15.45:0/1254144647 shutdown_connections 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.800+0000 7fa2d7844700 1 -- 172.21.15.45:0/1254144647 wait complete. 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.801+0000 7fa2d7844700 1 Processor -- start 2023-10-06T05:27:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.801+0000 7fa2d7844700 1 -- start start 2023-10-06T05:27:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d7844700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2d006c3d0 0x7fa2d01695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d7844700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d00c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d7844700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d015b890 0x7fa2d0158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d7844700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d01590c0 con 0x7fa2d00cac50 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d7844700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d00c4d10 con 0x7fa2d015b890 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d7844700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d0169c30 con 0x7fa2d006c3d0 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d00c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d00c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54586/0 (socket says 172.21.15.45:54586) 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 -- 172.21.15.45:0/3263225938 learned_addr learned my addr 172.21.15.45:0/3263225938 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d55e0700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2d006c3d0 0x7fa2d01695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d5de1700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d015b890 0x7fa2d0158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 -- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2d006c3d0 msgr2=0x7fa2d01695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2d006c3d0 0x7fa2d01695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 -- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d015b890 msgr2=0x7fa2d0158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d015b890 0x7fa2d0158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 -- 172.21.15.45:0/3263225938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c8009a20 con 0x7fa2d00cac50 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d5de1700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d015b890 0x7fa2d0158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.802+0000 7fa2d4ddf700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d00c47d0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c590 tx=0x7fa2c400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.803+0000 7fa2c17fa700 1 -- 172.21.15.45:0/3263225938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4011590 con 0x7fa2d00cac50 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.803+0000 7fa2c17fa700 1 -- 172.21.15.45:0/3263225938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c40116f0 con 0x7fa2d00cac50 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.803+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d0169f10 con 0x7fa2d00cac50 2023-10-06T05:27:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.803+0000 7fa2c17fa700 1 -- 172.21.15.45:0/3263225938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4011ab0 con 0x7fa2d00cac50 2023-10-06T05:27:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.803+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d016a4d0 con 0x7fa2d00cac50 2023-10-06T05:27:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.804+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d006f160 con 0x7fa2d00cac50 2023-10-06T05:27:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.804+0000 7fa2c17fa700 1 -- 172.21.15.45:0/3263225938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2c401b3b0 con 0x7fa2d00cac50 2023-10-06T05:27:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.805+0000 7fa2c17fa700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2b4066f30 0x7fa2b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.805+0000 7fa2c17fa700 1 -- 172.21.15.45:0/3263225938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fa2c4083e10 con 0x7fa2d00cac50 2023-10-06T05:27:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.805+0000 7fa2d55e0700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2b4066f30 0x7fa2b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.806+0000 7fa2d55e0700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2b4066f30 0x7fa2b40693e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0058d0 tx=0x7fa2cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:50.809+0000 7fa2c17fa700 1 -- 172.21.15.45:0/3263225938 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2c4050d50 con 0x7fa2d00cac50 2023-10-06T05:27:51.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.063+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2d00c9440 con 0x7fa2d00cac50 2023-10-06T05:27:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.063+0000 7fa2c17fa700 1 -- 172.21.15.45:0/3263225938 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) v1 ==== 95+0+4610 (secure 0 0 0) 0x7fa2c4017070 con 0x7fa2d00cac50 2023-10-06T05:27:51.065 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:51.065 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":365,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":365,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:49.992996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2b4066f30 msgr2=0x7fa2b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2b4066f30 0x7fa2b40693e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0058d0 tx=0x7fa2cc009040 comp rx=0 tx=0).stop 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 msgr2=0x7fa2d00c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d00c47d0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c590 tx=0x7fa2c400c950 comp rx=0 tx=0).stop 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 shutdown_connections 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fa2b4066f30 0x7fa2b40693e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d015b890 0x7fa2d0158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2d006c3d0 0x7fa2d01695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 --2- 172.21.15.45:0/3263225938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d00cac50 0x7fa2d00c47d0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 >> 172.21.15.45:0/3263225938 conn(0x7fa2d0067620 msgr2=0x7fa2d006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 shutdown_connections 2023-10-06T05:27:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.066+0000 7fa2d7844700 1 -- 172.21.15.45:0/3263225938 wait complete. 2023-10-06T05:27:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 365 2023-10-06T05:27:51.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2179480999 2023-10-06T05:27:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.213+0000 7fde8707a700 1 Processor -- start 2023-10-06T05:27:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.213+0000 7fde8707a700 1 -- start start 2023-10-06T05:27:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.214+0000 7fde8707a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde80140ab0 0x7fde800acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.214+0000 7fde8707a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800c9940 con 0x7fde800ad4e0 2023-10-06T05:27:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.214+0000 7fde8707a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8006d230 con 0x7fde801406e0 2023-10-06T05:27:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.214+0000 7fde8707a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8006d510 con 0x7fde80140ab0 2023-10-06T05:27:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde80140ab0 0x7fde800acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde85617700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fde800ad4e0 0x7fde800c9dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53258/0 (socket says 172.21.15.45:53258) 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde85617700 1 -- 172.21.15.45:0/2576593933 learned_addr learned my addr 172.21.15.45:0/2576593933 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7ffff700 1 -- 172.21.15.45:0/2576593933 >> v1:172.21.15.144:6789/0 conn(0x7fde801406e0 legacy=0x7fde8006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7ffff700 1 -- 172.21.15.45:0/2576593933 >> v1:172.21.15.45:6789/0 conn(0x7fde800ad4e0 legacy=0x7fde800c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7ffff700 1 -- 172.21.15.45:0/2576593933 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8006d7f0 con 0x7fde80140ab0 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7f7fe700 1 -- 172.21.15.45:0/2576593933 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068617756 0 0) 0x7fde800c9940 con 0x7fde800ad4e0 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7ffff700 1 --2- 172.21.15.45:0/2576593933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde80140ab0 0x7fde800acfa0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0098e0 tx=0x7fde6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941f520458086648 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7f7fe700 1 -- 172.21.15.45:0/2576593933 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c009e80 con 0x7fde80140ab0 2023-10-06T05:27:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7f7fe700 1 -- 172.21.15.45:0/2576593933 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c00b860 con 0x7fde80140ab0 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.215+0000 7fde7f7fe700 1 -- 172.21.15.45:0/2576593933 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00bbe0 con 0x7fde80140ab0 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.216+0000 7fde8707a700 1 -- 172.21.15.45:0/2576593933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde80140ab0 msgr2=0x7fde800acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.216+0000 7fde8707a700 1 --2- 172.21.15.45:0/2576593933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde80140ab0 0x7fde800acfa0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0098e0 tx=0x7fde6c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.216+0000 7fde8707a700 1 -- 172.21.15.45:0/2576593933 shutdown_connections 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.216+0000 7fde8707a700 1 --2- 172.21.15.45:0/2576593933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde80140ab0 0x7fde800acfa0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.216+0000 7fde8707a700 1 -- 172.21.15.45:0/2576593933 >> 172.21.15.45:0/2576593933 conn(0x7fde800691c0 msgr2=0x7fde800cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.216+0000 7fde8707a700 1 -- 172.21.15.45:0/2576593933 shutdown_connections 2023-10-06T05:27:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.216+0000 7fde8707a700 1 -- 172.21.15.45:0/2576593933 wait complete. 2023-10-06T05:27:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.218+0000 7fde8707a700 1 Processor -- start 2023-10-06T05:27:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.218+0000 7fde8707a700 1 -- start start 2023-10-06T05:27:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde8707a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde800ad4e0 0x7fde800caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde8707a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde801406e0 0x7fde80169b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde8707a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde80140ab0 0x7fde8016a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde8707a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800cb4d0 con 0x7fde801406e0 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde8707a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800cb770 con 0x7fde800ad4e0 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde8707a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800cba50 con 0x7fde80140ab0 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde801406e0 0x7fde80169b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde801406e0 0x7fde80169b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54612/0 (socket says 172.21.15.45:54612) 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 -- 172.21.15.45:0/3909596431 learned_addr learned my addr 172.21.15.45:0/3909596431 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde84e16700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde800ad4e0 0x7fde800caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde85617700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde80140ab0 0x7fde8016a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 -- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde80140ab0 msgr2=0x7fde8016a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde80140ab0 0x7fde8016a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 -- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde800ad4e0 msgr2=0x7fde800caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde800ad4e0 0x7fde800caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 -- 172.21.15.45:0/3909596431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c009420 con 0x7fde801406e0 2023-10-06T05:27:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.219+0000 7fde7ffff700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde801406e0 0x7fde80169b40 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fde8006c4b0 tx=0x7fde6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.220+0000 7fde7cff9700 1 -- 172.21.15.45:0/3909596431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c01f070 con 0x7fde801406e0 2023-10-06T05:27:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.220+0000 7fde7cff9700 1 -- 172.21.15.45:0/3909596431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c011420 con 0x7fde801406e0 2023-10-06T05:27:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.220+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde800cbd30 con 0x7fde801406e0 2023-10-06T05:27:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.220+0000 7fde7cff9700 1 -- 172.21.15.45:0/3909596431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00bbe0 con 0x7fde801406e0 2023-10-06T05:27:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.220+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde800cc2f0 con 0x7fde801406e0 2023-10-06T05:27:51.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.222+0000 7fde7cff9700 1 -- 172.21.15.45:0/3909596431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde6c011a50 con 0x7fde801406e0 2023-10-06T05:27:51.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.222+0000 7fde7cff9700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde64066f30 0x7fde640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.222+0000 7fde7cff9700 1 -- 172.21.15.45:0/3909596431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fde6c085dc0 con 0x7fde801406e0 2023-10-06T05:27:51.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.222+0000 7fde84e16700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde64066f30 0x7fde640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.223+0000 7fde84e16700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde64066f30 0x7fde640693e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fde74003860 tx=0x7fde74009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.224+0000 7fde9038eb80 1 Processor -- start 2023-10-06T05:27:51.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.224+0000 7fde9038eb80 1 -- start start 2023-10-06T05:27:51.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce9198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce922010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce9229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde9038eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce7ec390 con 0x55d4ce922550 2023-10-06T05:27:51.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce8291b0 con 0x55d4ce8fefc0 2023-10-06T05:27:51.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce826b70 con 0x55d4ce9194d0 2023-10-06T05:27:51.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce9229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce9229c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54624/0 (socket says 172.21.15.45:54624) 2023-10-06T05:27:51.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 -- 172.21.15.45:0/4062032071 learned_addr learned my addr 172.21.15.45:0/4062032071 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde7ffff700 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce922010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde84e16700 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce9198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 -- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 msgr2=0x55d4ce9198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce9198a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 -- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 msgr2=0x55d4ce922010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce922010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 -- 172.21.15.45:0/4062032071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ce628810 con 0x55d4ce922550 2023-10-06T05:27:51.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde85617700 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce9229c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fde780020c0 tx=0x7fde78009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bcb88ca2fff969 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde69ffb700 1 -- 172.21.15.45:0/4062032071 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78004bb0 con 0x55d4ce922550 2023-10-06T05:27:51.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde69ffb700 1 -- 172.21.15.45:0/4062032071 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde78004d30 con 0x55d4ce922550 2023-10-06T05:27:51.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.225+0000 7fde69ffb700 1 -- 172.21.15.45:0/4062032071 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78004ea0 con 0x55d4ce922550 2023-10-06T05:27:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 -- 172.21.15.45:0/4062032071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 msgr2=0x55d4ce9229c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce9229c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fde780020c0 tx=0x7fde78009040 comp rx=0 tx=0).stop 2023-10-06T05:27:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 -- 172.21.15.45:0/4062032071 shutdown_connections 2023-10-06T05:27:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce9229c0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce9198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 --2- 172.21.15.45:0/4062032071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce922010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 -- 172.21.15.45:0/4062032071 >> 172.21.15.45:0/4062032071 conn(0x55d4ce91c900 msgr2=0x55d4ce91ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 -- 172.21.15.45:0/4062032071 shutdown_connections 2023-10-06T05:27:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.226+0000 7fde9038eb80 1 -- 172.21.15.45:0/4062032071 wait complete. 2023-10-06T05:27:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.227+0000 7fde9038eb80 1 -- start start 2023-10-06T05:27:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.227+0000 7fde9038eb80 1 Processor -- start 2023-10-06T05:27:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce8f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce8f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde9038eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce822960 con 0x55d4ce922550 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce7cea00 con 0x55d4ce8fefc0 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce7ce6a0 con 0x55d4ce9194d0 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde9038eb80 10 client.0 _ll_register_callbacks cb 0x55d4ce631a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce8f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce8f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54626/0 (socket says 172.21.15.45:54626) 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 -- 172.21.15.45:0/3618140436 learned_addr learned my addr 172.21.15.45:0/3618140436 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde84e16700 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce8f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde7ffff700 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 -- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 msgr2=0x55d4ce91a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 -- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 msgr2=0x55d4ce8f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce8f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.228+0000 7fde85617700 1 -- 172.21.15.45:0/3618140436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7800f040 con 0x55d4ce922550 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde85617700 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce8f66b0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fde78004fa0 tx=0x7fde78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75eb88e19d24c42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde5effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde5effd700 1 -- 172.21.15.45:0/3618140436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78001710 con 0x55d4ce922550 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde5effd700 1 -- 172.21.15.45:0/3618140436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fde78013420 con 0x55d4ce922550 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4ce8c6de0 con 0x55d4ce922550 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde5effd700 1 -- 172.21.15.45:0/3618140436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=365) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde780137f0 con 0x55d4ce922550 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 10 client.11217 fetch_fsmap learned FSMap version 365 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=365,monmap=2+}) v3 -- 0x55d4ce715120 con 0x55d4ce922550 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde5effd700 1 -- 172.21.15.45:0/3618140436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4ce8c6de0 con 0x55d4ce922550 2023-10-06T05:27:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde5effd700 1 -- 172.21.15.45:0/3618140436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 365) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fde78013950 con 0x55d4ce922550 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 10 client.11217 fetch_fsmap finished waiting for FSMap version 365 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 10 client.11217 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 20 client.11217 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8f7660 0x55d4ce8f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 4 client.11217 mds_command: new command op to 11019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4ce8a3bb0 con 0x55d4ce8f7660 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.229+0000 7fde84e16700 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8f7660 0x55d4ce8f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.230+0000 7fde5effd700 10 client.11217 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.230+0000 7fde84e16700 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8f7660 0x55d4ce8f7a30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d33af21e38f656f server_cookie=1affc742b0e73fb7 in_seq=0 out_seq=0 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.230+0000 7fde5effd700 1 -- 172.21.15.45:0/3618140436 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d4ce8a3bb0 con 0x55d4ce8f7660 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.230+0000 7fde5effd700 10 client.11217 handle_command_reply: tid=0 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.230+0000 7fde9038eb80 1 client.11217 shutdown 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 msgr2=0x55d4ce8f66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce8f66b0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fde78004fa0 tx=0x7fde78019000 comp rx=0 tx=0).stop 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 shutdown_connections 2023-10-06T05:27:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8f7660 0x55d4ce8f7a30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce922550 0x55d4ce8f66b0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce8fefc0 0x55d4ce8f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde5effd700 0 client.11217 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3618140436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 >> 172.21.15.45:0/3618140436 conn(0x55d4ce91c900 msgr2=0x55d4ce8cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 shutdown_connections 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 1 -- 172.21.15.45:0/3618140436 wait complete. 2023-10-06T05:27:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.231+0000 7fde9038eb80 20 client.11217 trim_cache size 0 max 16384 2023-10-06T05:27:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.244+0000 7fde9038eb80 1 Processor -- start 2023-10-06T05:27:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.244+0000 7fde9038eb80 1 -- start start 2023-10-06T05:27:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce8fefc0 0x55d4ce8c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce9194d0 0x55d4ce8d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde9038eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce8fbc00 con 0x55d4ce9194d0 2023-10-06T05:27:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce6ba260 con 0x55d4ce922550 2023-10-06T05:27:51.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce8fc560 con 0x55d4ce8fefc0 2023-10-06T05:27:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce9194d0 0x55d4ce8d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce9194d0 0x55d4ce8d5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54634/0 (socket says 172.21.15.45:54634) 2023-10-06T05:27:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 -- 172.21.15.45:0/3439024867 learned_addr learned my addr 172.21.15.45:0/3439024867 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde85617700 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde84e16700 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce8fefc0 0x55d4ce8c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 -- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce8fefc0 msgr2=0x55d4ce8c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce8fefc0 0x55d4ce8c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 -- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 msgr2=0x55d4ce8d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 -- 172.21.15.45:0/3439024867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ce7f1440 con 0x55d4ce9194d0 2023-10-06T05:27:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde85617700 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.245+0000 7fde7ffff700 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce9194d0 0x55d4ce8d5400 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00b620 tx=0x7fde6c059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabdae01eae20ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde6a7fc700 1 -- 172.21.15.45:0/3439024867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0569c0 con 0x55d4ce9194d0 2023-10-06T05:27:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde6a7fc700 1 -- 172.21.15.45:0/3439024867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c059c10 con 0x55d4ce9194d0 2023-10-06T05:27:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde6a7fc700 1 -- 172.21.15.45:0/3439024867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c05d960 con 0x55d4ce9194d0 2023-10-06T05:27:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 -- 172.21.15.45:0/3439024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce9194d0 msgr2=0x55d4ce8d5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce9194d0 0x55d4ce8d5400 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00b620 tx=0x7fde6c059040 comp rx=0 tx=0).stop 2023-10-06T05:27:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 -- 172.21.15.45:0/3439024867 shutdown_connections 2023-10-06T05:27:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce8fefc0 0x55d4ce8c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 --2- 172.21.15.45:0/3439024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce9194d0 0x55d4ce8d5400 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 -- 172.21.15.45:0/3439024867 >> 172.21.15.45:0/3439024867 conn(0x55d4ce91c900 msgr2=0x55d4ce8f7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 -- 172.21.15.45:0/3439024867 shutdown_connections 2023-10-06T05:27:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.246+0000 7fde9038eb80 1 -- 172.21.15.45:0/3439024867 wait complete. 2023-10-06T05:27:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.247+0000 7fde9038eb80 1 -- start start 2023-10-06T05:27:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.247+0000 7fde9038eb80 1 Processor -- start 2023-10-06T05:27:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce8fefc0 0x55d4ce975c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde9038eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde9038eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce97def0 con 0x55d4ce8fefc0 2023-10-06T05:27:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce91b7e0 con 0x55d4ce922550 2023-10-06T05:27:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde9038eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4ce8f9be0 con 0x55d4ce9194d0 2023-10-06T05:27:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde9038eb80 10 client.0 _ll_register_callbacks cb 0x55d4ce631a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce8fefc0 0x55d4ce975c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce8fefc0 0x55d4ce975c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54638/0 (socket says 172.21.15.45:54638) 2023-10-06T05:27:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 -- 172.21.15.45:0/761514341 learned_addr learned my addr 172.21.15.45:0/761514341 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde85617700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde7ffff700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 -- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 msgr2=0x55d4ce91b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 -- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 msgr2=0x55d4ce8c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde84e16700 1 -- 172.21.15.45:0/761514341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde800cc2f0 con 0x55d4ce8fefc0 2023-10-06T05:27:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde85617700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.248+0000 7fde7ffff700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde84e16700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce8fefc0 0x55d4ce975c50 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fde74007fd0 tx=0x7fde74028250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab81297aab9e778 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde5effd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde5effd700 1 -- 172.21.15.45:0/761514341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74027070 con 0x55d4ce8fefc0 2023-10-06T05:27:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde5effd700 1 -- 172.21.15.45:0/761514341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fde7401b040 con 0x55d4ce8fefc0 2023-10-06T05:27:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde78004d30 con 0x55d4ce8fefc0 2023-10-06T05:27:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde5effd700 1 -- 172.21.15.45:0/761514341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=365) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde74022ea0 con 0x55d4ce8fefc0 2023-10-06T05:27:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 10 client.11223 fetch_fsmap learned FSMap version 365 2023-10-06T05:27:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=365,monmap=2+}) v3 -- 0x55d4ce689250 con 0x55d4ce8fefc0 2023-10-06T05:27:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde5effd700 1 -- 172.21.15.45:0/761514341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde78004d30 con 0x55d4ce8fefc0 2023-10-06T05:27:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde5effd700 1 -- 172.21.15.45:0/761514341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 365) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fde74006a30 con 0x55d4ce8fefc0 2023-10-06T05:27:51.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 10 client.11223 fetch_fsmap finished waiting for FSMap version 365 2023-10-06T05:27:51.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 10 client.11223 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:51.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 20 client.11223 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:51.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8fa670 0x55d4ce8faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 4 client.11223 mds_command: new command op to 11019 tid=0[{"prefix": "damage rm", "damage_id": 2179480999}] 2023-10-06T05:27:51.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2179480999}) v1 -- 0x55d4ce7c5950 con 0x55d4ce8fa670 2023-10-06T05:27:51.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.249+0000 7fde7ffff700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8fa670 0x55d4ce8faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde5effd700 10 client.11223 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde7ffff700 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8fa670 0x55d4ce8faa40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9301cf55eba15332 server_cookie=d25855c801718c10 in_seq=0 out_seq=0 2023-10-06T05:27:51.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde5effd700 1 -- 172.21.15.45:0/761514341 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55d4ce7c5950 con 0x55d4ce8fa670 2023-10-06T05:27:51.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde5effd700 10 client.11223 handle_command_reply: tid=0 2023-10-06T05:27:51.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 client.11223 shutdown 2023-10-06T05:27:51.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce8fefc0 msgr2=0x55d4ce975c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce8fefc0 0x55d4ce975c50 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fde74007fd0 tx=0x7fde74028250 comp rx=0 tx=0).stop 2023-10-06T05:27:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 shutdown_connections 2023-10-06T05:27:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x55d4ce8fa670 0x55d4ce8faa40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ce922550 0x55d4ce8c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d4ce8fefc0 0x55d4ce975c50 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde5effd700 0 client.11223 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 --2- 172.21.15.45:0/761514341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ce9194d0 0x55d4ce91b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 >> 172.21.15.45:0/761514341 conn(0x55d4ce91c900 msgr2=0x55d4ce8f6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 shutdown_connections 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 1 -- 172.21.15.45:0/761514341 wait complete. 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.250+0000 7fde9038eb80 20 client.11223 trim_cache size 0 max 16384 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde64066f30 msgr2=0x7fde640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde64066f30 0x7fde640693e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fde74003860 tx=0x7fde74009040 comp rx=0 tx=0).stop 2023-10-06T05:27:51.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde801406e0 msgr2=0x7fde80169b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde801406e0 0x7fde80169b40 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fde8006c4b0 tx=0x7fde6c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 shutdown_connections 2023-10-06T05:27:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fde64066f30 0x7fde640693e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde80140ab0 0x7fde8016a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde800ad4e0 0x7fde800caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 --2- 172.21.15.45:0/3909596431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde801406e0 0x7fde80169b40 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 >> 172.21.15.45:0/3909596431 conn(0x7fde800691c0 msgr2=0x7fde8013faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 shutdown_connections 2023-10-06T05:27:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.251+0000 7fde8707a700 1 -- 172.21.15.45:0/3909596431 wait complete. 2023-10-06T05:27:51.276 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.406+0000 7f2b07c91700 1 Processor -- start 2023-10-06T05:27:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.406+0000 7f2b07c91700 1 -- start start 2023-10-06T05:27:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b07c91700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b00158f60 0x7f2b00159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b07c91700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b07c91700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b00069bd0 con 0x7f2b000cc390 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b07c91700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0006a4d0 con 0x7f2b00158f60 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b07c91700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b00068fc0 con 0x7f2b0006a060 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b05a2d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b00158f60 0x7f2b00159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b05a2d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b00158f60 0x7f2b00159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54650/0 (socket says 172.21.15.45:54650) 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b05a2d700 1 -- 172.21.15.45:0/1313077833 learned_addr learned my addr 172.21.15.45:0/1313077833 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b0522c700 1 --2- 172.21.15.45:0/1313077833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b05a2d700 1 -- 172.21.15.45:0/1313077833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 msgr2=0x7f2b000cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b05a2d700 1 --2- 172.21.15.45:0/1313077833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.407+0000 7f2b05a2d700 1 -- 172.21.15.45:0/1313077833 >> v1:172.21.15.144:6789/0 conn(0x7f2b000cc390 legacy=0x7f2b000c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b05a2d700 1 -- 172.21.15.45:0/1313077833 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000692a0 con 0x7f2b00158f60 2023-10-06T05:27:51.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b05a2d700 1 --2- 172.21.15.45:0/1313077833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b00159330 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0099b0 tx=0x7f2afc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f310338a8d1290fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b04a2b700 1 -- 172.21.15.45:0/1313077833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc009ea0 con 0x7f2b00158f60 2023-10-06T05:27:51.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b04a2b700 1 -- 172.21.15.45:0/1313077833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc00b860 con 0x7f2b00158f60 2023-10-06T05:27:51.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b04a2b700 1 -- 172.21.15.45:0/1313077833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00bc00 con 0x7f2b00158f60 2023-10-06T05:27:51.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b07c91700 1 -- 172.21.15.45:0/1313077833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 msgr2=0x7f2b00159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1313077833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b00159330 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0099b0 tx=0x7f2afc009250 comp rx=0 tx=0).stop 2023-10-06T05:27:51.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b07c91700 1 -- 172.21.15.45:0/1313077833 shutdown_connections 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1313077833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b00159330 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.408+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1313077833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.409+0000 7f2b07c91700 1 -- 172.21.15.45:0/1313077833 >> 172.21.15.45:0/1313077833 conn(0x7f2b000cdf50 msgr2=0x7f2b000ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.409+0000 7f2b07c91700 1 -- 172.21.15.45:0/1313077833 shutdown_connections 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.409+0000 7f2b07c91700 1 -- 172.21.15.45:0/1313077833 wait complete. 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.410+0000 7f2b07c91700 1 Processor -- start 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.410+0000 7f2b07c91700 1 -- start start 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.410+0000 7f2b07c91700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b07c91700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b000cc390 0x7f2b00157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b07c91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b0016a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b07c91700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0016a990 con 0x7f2b00158f60 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b07c91700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0016d430 con 0x7f2b000cc390 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b07c91700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000c7790 con 0x7f2b0006a060 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b0016a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b0016a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54664/0 (socket says 172.21.15.45:54664) 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 -- 172.21.15.45:0/1154249055 learned_addr learned my addr 172.21.15.45:0/1154249055 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b05a2d700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0522c700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b000cc390 0x7f2b00157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 -- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 msgr2=0x7f2b000c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 -- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b000cc390 msgr2=0x7f2b00157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b000cc390 0x7f2b00157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 -- 172.21.15.45:0/1154249055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0094f0 con 0x7f2b00158f60 2023-10-06T05:27:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b05a2d700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b0622e700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b0016a450 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c910 tx=0x7f2af8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2af5ffb700 1 -- 172.21.15.45:0/1154249055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af800a7a0 con 0x7f2b00158f60 2023-10-06T05:27:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2af5ffb700 1 -- 172.21.15.45:0/1154249055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af800a900 con 0x7f2b00158f60 2023-10-06T05:27:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b000c71d0 con 0x7f2b00158f60 2023-10-06T05:27:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.411+0000 7f2af5ffb700 1 -- 172.21.15.45:0/1154249055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af800acc0 con 0x7f2b00158f60 2023-10-06T05:27:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.412+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0016d0c0 con 0x7f2b00158f60 2023-10-06T05:27:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.413+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0006f410 con 0x7f2b00158f60 2023-10-06T05:27:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.413+0000 7f2af5ffb700 1 -- 172.21.15.45:0/1154249055 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2af8012630 con 0x7f2b00158f60 2023-10-06T05:27:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.414+0000 7f2af5ffb700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ae4066f30 0x7f2ae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.414+0000 7f2af5ffb700 1 -- 172.21.15.45:0/1154249055 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f2af8084bf0 con 0x7f2b00158f60 2023-10-06T05:27:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.414+0000 7f2b05a2d700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ae4066f30 0x7f2ae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.414+0000 7f2b05a2d700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ae4066f30 0x7f2ae40693e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0099b0 tx=0x7f2afc017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.418+0000 7f2af5ffb700 1 -- 172.21.15.45:0/1154249055 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2af8051a90 con 0x7f2b00158f60 2023-10-06T05:27:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.670+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b00069580 con 0x7f2b00158f60 2023-10-06T05:27:51.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.670+0000 7f2af5ffb700 1 -- 172.21.15.45:0/1154249055 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f2af80550b0 con 0x7f2b00158f60 2023-10-06T05:27:51.672 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:51.673 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":365,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":365,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:49.992996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ae4066f30 msgr2=0x7f2ae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ae4066f30 0x7f2ae40693e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0099b0 tx=0x7f2afc017000 comp rx=0 tx=0).stop 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 msgr2=0x7f2b0016a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b0016a450 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c910 tx=0x7f2af8001710 comp rx=0 tx=0).stop 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 shutdown_connections 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f2ae4066f30 0x7f2ae40693e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00158f60 0x7f2b0016a450 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b0006a060 0x7f2b000c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 --2- 172.21.15.45:0/1154249055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b000cc390 0x7f2b00157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 >> 172.21.15.45:0/1154249055 conn(0x7f2b000cdf50 msgr2=0x7f2b0006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 shutdown_connections 2023-10-06T05:27:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.673+0000 7f2b07c91700 1 -- 172.21.15.45:0/1154249055 wait complete. 2023-10-06T05:27:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 365 2023-10-06T05:27:51.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2385964704 2023-10-06T05:27:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.821+0000 7fcc5393a700 1 Processor -- start 2023-10-06T05:27:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.821+0000 7fcc5393a700 1 -- start start 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc5393a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c146e40 0x7fcc4c147210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc5393a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc4c067d30 0x7fcc4c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc5393a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c0c8770 con 0x7fcc4c0cb650 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc5393a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c0cc030 con 0x7fcc4c067d30 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc5393a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c8120 con 0x7fcc4c146e40 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc50ed5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcc4c0cb650 0x7fcc4c067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53264/0 (socket says 172.21.15.45:53264) 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc50ed5700 1 -- 172.21.15.45:0/1070953350 learned_addr learned my addr 172.21.15.45:0/1070953350 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:51.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/1070953350 >> v2:172.21.15.144:3300/0 conn(0x7fcc4c067d30 0x7fcc4c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.822+0000 7fcc516d6700 1 --2- 172.21.15.45:0/1070953350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c146e40 0x7fcc4c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc3ffff700 1 -- 172.21.15.45:0/1070953350 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892797062 0 0) 0x7fcc4c0c8770 con 0x7fcc4c0cb650 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc3ffff700 1 -- 172.21.15.45:0/1070953350 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc38003570 con 0x7fcc4c0cb650 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc3ffff700 1 -- 172.21.15.45:0/1070953350 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281813607 0 0) 0x7fcc38003570 con 0x7fcc4c0cb650 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc516d6700 1 -- 172.21.15.45:0/1070953350 >> v2:172.21.15.144:3300/0 conn(0x7fcc4c067d30 msgr2=0x7fcc4c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc516d6700 1 --2- 172.21.15.45:0/1070953350 >> v2:172.21.15.144:3300/0 conn(0x7fcc4c067d30 0x7fcc4c0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc516d6700 1 -- 172.21.15.45:0/1070953350 >> v1:172.21.15.45:6789/0 conn(0x7fcc4c0cb650 legacy=0x7fcc4c067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc516d6700 1 -- 172.21.15.45:0/1070953350 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0c8400 con 0x7fcc4c146e40 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/1070953350 >> v2:172.21.15.144:3300/0 conn(0x7fcc4c067d30 0x7fcc4c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc516d6700 1 --2- 172.21.15.45:0/1070953350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c146e40 0x7fcc4c147210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc480098e0 tx=0x7fcc480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d383c2329079638c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc3ffff700 1 -- 172.21.15.45:0/1070953350 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40001470 con 0x7fcc4c146e40 2023-10-06T05:27:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc3ffff700 1 -- 172.21.15.45:0/1070953350 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48011420 con 0x7fcc4c146e40 2023-10-06T05:27:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.823+0000 7fcc3ffff700 1 -- 172.21.15.45:0/1070953350 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc480117a0 con 0x7fcc4c146e40 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 -- 172.21.15.45:0/1070953350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c146e40 msgr2=0x7fcc4c147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 --2- 172.21.15.45:0/1070953350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c146e40 0x7fcc4c147210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc480098e0 tx=0x7fcc480091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 -- 172.21.15.45:0/1070953350 shutdown_connections 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 --2- 172.21.15.45:0/1070953350 >> v2:172.21.15.144:3300/0 conn(0x7fcc4c067d30 0x7fcc4c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 --2- 172.21.15.45:0/1070953350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c146e40 0x7fcc4c147210 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 -- 172.21.15.45:0/1070953350 >> 172.21.15.45:0/1070953350 conn(0x7fcc4c0691c0 msgr2=0x7fcc4c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 -- 172.21.15.45:0/1070953350 shutdown_connections 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.824+0000 7fcc5393a700 1 -- 172.21.15.45:0/1070953350 wait complete. 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.826+0000 7fcc5393a700 1 Processor -- start 2023-10-06T05:27:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.826+0000 7fcc5393a700 1 -- start start 2023-10-06T05:27:51.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc5393a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c067d30 0x7fcc4c171ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc5393a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0cb650 0x7fcc4c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc5393a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c146e40 0x7fcc4c15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc5393a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c4c30 con 0x7fcc4c067d30 2023-10-06T05:27:51.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc5393a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c1723e0 con 0x7fcc4c146e40 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc5393a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c4020 con 0x7fcc4c0cb650 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c067d30 0x7fcc4c171ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c067d30 0x7fcc4c171ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54670/0 (socket says 172.21.15.45:54670) 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 -- 172.21.15.45:0/4182089761 learned_addr learned my addr 172.21.15.45:0/4182089761 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c146e40 0x7fcc4c15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0cb650 0x7fcc4c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 -- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0cb650 msgr2=0x7fcc4c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0cb650 0x7fcc4c15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 -- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c146e40 msgr2=0x7fcc4c15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c146e40 0x7fcc4c15c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc516d6700 1 -- 172.21.15.45:0/4182089761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44009a20 con 0x7fcc4c067d30 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c146e40 0x7fcc4c15c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.827+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0cb650 0x7fcc4c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.828+0000 7fcc516d6700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c067d30 0x7fcc4c171ea0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0c8260 tx=0x7fcc48016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.828+0000 7fcc3d7fa700 1 -- 172.21.15.45:0/4182089761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4801cba0 con 0x7fcc4c067d30 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.828+0000 7fcc3d7fa700 1 -- 172.21.15.45:0/4182089761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4801cd20 con 0x7fcc4c067d30 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.828+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc48009420 con 0x7fcc4c067d30 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.828+0000 7fcc3d7fa700 1 -- 172.21.15.45:0/4182089761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4801a6a0 con 0x7fcc4c067d30 2023-10-06T05:27:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.828+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c0c45e0 con 0x7fcc4c067d30 2023-10-06T05:27:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.829+0000 7fcc3d7fa700 1 -- 172.21.15.45:0/4182089761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc4801aa50 con 0x7fcc4c067d30 2023-10-06T05:27:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.830+0000 7fcc3d7fa700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc30066f30 0x7fcc300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.830+0000 7fcc3d7fa700 1 -- 172.21.15.45:0/4182089761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fcc48086420 con 0x7fcc4c067d30 2023-10-06T05:27:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.830+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc30066f30 0x7fcc300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.831+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc30066f30 0x7fcc300693e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcc40001ed0 tx=0x7fcc40009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.832+0000 7fcc5cc4eb80 1 Processor -- start 2023-10-06T05:27:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.832+0000 7fcc5cc4eb80 1 -- start start 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fb218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fb2a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551f9f4390 con 0x56551fb2a550 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551fa311b0 con 0x56551fb214d0 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56551fa2eb70 con 0x56551fb06fc0 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fb2a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fb2a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54676/0 (socket says 172.21.15.45:54676) 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 -- 172.21.15.45:0/177693810 learned_addr learned my addr 172.21.15.45:0/177693810 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc516d6700 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fb218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 -- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 msgr2=0x56551fb2a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb2a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 -- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 msgr2=0x56551fb218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fb218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 -- 172.21.15.45:0/177693810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551f830810 con 0x56551fb2a550 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fb2a9c0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fcc440094d0 tx=0x7fcc4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4f57c51ed9ddab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.833+0000 7fcc367fc700 1 -- 172.21.15.45:0/177693810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440145e0 con 0x56551fb2a550 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc367fc700 1 -- 172.21.15.45:0/177693810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4400f050 con 0x56551fb2a550 2023-10-06T05:27:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc367fc700 1 -- 172.21.15.45:0/177693810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44014990 con 0x56551fb2a550 2023-10-06T05:27:51.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/177693810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 msgr2=0x56551fb2a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fb2a9c0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fcc440094d0 tx=0x7fcc4400b040 comp rx=0 tx=0).stop 2023-10-06T05:27:51.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/177693810 shutdown_connections 2023-10-06T05:27:51.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fb2a9c0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fb218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/177693810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/177693810 >> 172.21.15.45:0/177693810 conn(0x56551fb24900 msgr2=0x56551fb24cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/177693810 shutdown_connections 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.834+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/177693810 wait complete. 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.835+0000 7fcc5cc4eb80 1 -- start start 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.835+0000 7fcc5cc4eb80 1 Processor -- start 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb06fc0 0x56551fafc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb214d0 0x56551fadbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551fa2a960 con 0x56551fb2a550 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551f9d6a00 con 0x56551fb06fc0 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56551f9d66a0 con 0x56551fb214d0 2023-10-06T05:27:51.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc5cc4eb80 10 client.0 _ll_register_callbacks cb 0x56551f839a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54690/0 (socket says 172.21.15.45:54690) 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2149759546 learned_addr learned my addr 172.21.15.45:0/2149759546 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb214d0 0x56551fadbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc516d6700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb06fc0 0x56551fafc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb214d0 msgr2=0x56551fadbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb214d0 0x56551fadbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb06fc0 msgr2=0x56551fafc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb06fc0 0x56551fafc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2149759546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc44007420 con 0x56551fb2a550 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc516d6700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb06fc0 0x56551fafc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:51.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.836+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb214d0 0x56551fadbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc380 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009f80 tx=0x7fcc4401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6dadb9b39a78abd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/2149759546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440070a0 con 0x56551fb2a550 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/2149759546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fcc44015680 con 0x56551fb2a550 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56551facede0 con 0x56551fb2a550 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/2149759546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=365) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc44015a50 con 0x56551fb2a550 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 10 client.11241 fetch_fsmap learned FSMap version 365 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=365,monmap=2+}) v3 -- 0x56551f91d120 con 0x56551fb2a550 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/2149759546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56551facede0 con 0x56551fb2a550 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/2149759546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 365) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcc44015bb0 con 0x56551fb2a550 2023-10-06T05:27:51.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 10 client.11241 fetch_fsmap finished waiting for FSMap version 365 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 10 client.11241 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 20 client.11241 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fafe820 0x56551fafebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 4 client.11241 mds_command: new command op to 11019 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56551faabbb0 con 0x56551fafe820 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.837+0000 7fcc516d6700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fafe820 0x56551fafebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.838+0000 7fcc516d6700 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fafe820 0x56551fafebf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ef0c138c4b4376 server_cookie=fb018d6e59d6410c in_seq=0 out_seq=0 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.838+0000 7fcc2b7fe700 10 client.11241 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.838+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/2149759546 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56551faabbb0 con 0x56551fafe820 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.838+0000 7fcc2b7fe700 10 client.11241 handle_command_reply: tid=0 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.838+0000 7fcc5cc4eb80 1 client.11241 shutdown 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 msgr2=0x56551fadc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc380 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009f80 tx=0x7fcc4401a000 comp rx=0 tx=0).stop 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 shutdown_connections 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fafe820 0x56551fafebf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc380 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb06fc0 0x56551fafc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2149759546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb214d0 0x56551fadbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc2b7fe700 0 client.11241 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 >> 172.21.15.45:0/2149759546 conn(0x56551fb24900 msgr2=0x56551fad6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 shutdown_connections 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2149759546 wait complete. 2023-10-06T05:27:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.839+0000 7fcc5cc4eb80 20 client.11241 trim_cache size 0 max 16384 2023-10-06T05:27:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.852+0000 7fcc5cc4eb80 1 Processor -- start 2023-10-06T05:27:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.852+0000 7fcc5cc4eb80 1 -- start start 2023-10-06T05:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb24ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fad5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fad64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551f8c2260 con 0x56551fb2a550 2023-10-06T05:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551fad8c20 con 0x56551fb214d0 2023-10-06T05:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56551fb3aa40 con 0x56551fb06fc0 2023-10-06T05:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fad64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fad64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54696/0 (socket says 172.21.15.45:54696) 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2633364061 learned_addr learned my addr 172.21.15.45:0/2633364061 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fad5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc516d6700 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb24ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 msgr2=0x56551fb24ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb24ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 msgr2=0x56551fad5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fad5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 -- 172.21.15.45:0/2633364061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551f9f9440 con 0x56551fb2a550 2023-10-06T05:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.853+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fad64b0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009f00 tx=0x7fcc4400b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60517f440bb040b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc36ffd700 1 -- 172.21.15.45:0/2633364061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440143d0 con 0x56551fb2a550 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc36ffd700 1 -- 172.21.15.45:0/2633364061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc44000f40 con 0x56551fb2a550 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc36ffd700 1 -- 172.21.15.45:0/2633364061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44007000 con 0x56551fb2a550 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2633364061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 msgr2=0x56551fad64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fad64b0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009f00 tx=0x7fcc4400b070 comp rx=0 tx=0).stop 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2633364061 shutdown_connections 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fad64b0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb24ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/2633364061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fad5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2633364061 >> 172.21.15.45:0/2633364061 conn(0x56551fb24900 msgr2=0x56551fafa000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2633364061 shutdown_connections 2023-10-06T05:27:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.854+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/2633364061 wait complete. 2023-10-06T05:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.855+0000 7fcc5cc4eb80 1 -- start start 2023-10-06T05:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.855+0000 7fcc5cc4eb80 1 Processor -- start 2023-10-06T05:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb7f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fadbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc5cc4eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551fb86ae0 con 0x56551fb2a550 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551fb85bb0 con 0x56551fb214d0 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc5cc4eb80 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56551fb866a0 con 0x56551fb06fc0 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc5cc4eb80 10 client.0 _ll_register_callbacks cb 0x56551f839a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54702/0 (socket says 172.21.15.45:54702) 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 -- 172.21.15.45:0/981545178 learned_addr learned my addr 172.21.15.45:0/981545178 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc50ed5700 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fadbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc516d6700 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb7f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 -- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 msgr2=0x56551fb7f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb7f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 -- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 msgr2=0x56551fadbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fadbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.856+0000 7fcc51ed7700 1 -- 172.21.15.45:0/981545178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56551f91d120 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc51ed7700 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc260 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcc44034040 tx=0x7fcc44010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c95d43f4f18c64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/981545178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44014e10 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/981545178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fcc44007000 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc4400f050 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/981545178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=365) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc4401e8d0 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 10 client.11247 fetch_fsmap learned FSMap version 365 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=365,monmap=2+}) v3 -- 0x56551f891250 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/981545178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4400f050 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/981545178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 365) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcc4401ea30 con 0x56551fb2a550 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 10 client.11247 fetch_fsmap finished waiting for FSMap version 365 2023-10-06T05:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 10 client.11247 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 20 client.11247 populate_metadata read hostname 'smithi045' 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fb094e0 0x56551fb098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 4 client.11247 mds_command: new command op to 11019 tid=0[{"prefix": "damage rm", "damage_id": 2385964704}] 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 --> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2385964704}) v1 -- 0x56551f9cd950 con 0x56551fb094e0 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.857+0000 7fcc516d6700 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fb094e0 0x56551fb098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc2b7fe700 10 client.11247 ms_handle_connect on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc516d6700 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fb094e0 0x56551fb098b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde3308c45871025 server_cookie=2af03097524bfffd in_seq=0 out_seq=0 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc2b7fe700 1 -- 172.21.15.45:0/981545178 <== mds.0 v2:172.21.15.45:6832/4232324777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x56551f9cd950 con 0x56551fb094e0 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc2b7fe700 10 client.11247 handle_command_reply: tid=0 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 client.11247 shutdown 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 msgr2=0x56551fadc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc260 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcc44034040 tx=0x7fcc44010000 comp rx=0 tx=0).stop 2023-10-06T05:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 shutdown_connections 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:6832/4232324777,v1:172.21.15.45:6833/4232324777] conn(0x56551fb094e0 0x56551fb098b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56551fb2a550 0x56551fadc260 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56551fb06fc0 0x56551fb7f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 --2- 172.21.15.45:0/981545178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551fb214d0 0x56551fadbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc2b7fe700 0 client.11247 ms_handle_reset on v2:172.21.15.45:6832/4232324777 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 >> 172.21.15.45:0/981545178 conn(0x56551fb24900 msgr2=0x56551fafa000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 shutdown_connections 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 1 -- 172.21.15.45:0/981545178 wait complete. 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.858+0000 7fcc5cc4eb80 20 client.11247 trim_cache size 0 max 16384 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc30066f30 msgr2=0x7fcc300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc30066f30 0x7fcc300693e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcc40001ed0 tx=0x7fcc40009040 comp rx=0 tx=0).stop 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c067d30 msgr2=0x7fcc4c171ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c067d30 0x7fcc4c171ea0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0c8260 tx=0x7fcc48016000 comp rx=0 tx=0).stop 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 shutdown_connections 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcc30066f30 0x7fcc300693e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c146e40 0x7fcc4c15c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c067d30 0x7fcc4c171ea0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 --2- 172.21.15.45:0/4182089761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0cb650 0x7fcc4c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 >> 172.21.15.45:0/4182089761 conn(0x7fcc4c0691c0 msgr2=0x7fcc4c0c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 shutdown_connections 2023-10-06T05:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:51.859+0000 7fcc5393a700 1 -- 172.21.15.45:0/4182089761 wait complete. 2023-10-06T05:27:51.874 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:27:51.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:51.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:27:51.965 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:27:51.965 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-10-06T05:27:51.965 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:27:51.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T05:27:52.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[184881]: fuse finished with error 0 and tester_r 0 2023-10-06T05:27:52.013 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:27:52.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:52.013 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:27:52.146 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:27:52.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:52.147 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:27:52.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:52.169 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T05:27:52.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:27:52.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:52.170 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:27:52.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T05:27:52.211 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:27:52.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:27:52.211 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:27:52.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:52.232 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T05:27:52.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.390+0000 7f7fc07e3700 1 Processor -- start 2023-10-06T05:27:52.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.390+0000 7f7fc07e3700 1 -- start start 2023-10-06T05:27:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fc07e3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7fb8159200 0x7f7fb80c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fc07e3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb80c7d00 con 0x7f7fb80ccdc0 2023-10-06T05:27:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fc07e3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb80c86d0 con 0x7f7fb81595d0 2023-10-06T05:27:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fc07e3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb80c7780 con 0x7f7fb8159200 2023-10-06T05:27:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fbed80700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7fb80ccdc0 0x7f7fb80cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53278/0 (socket says 172.21.15.45:53278) 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fbed80700 1 -- 172.21.15.45:0/4051554806 learned_addr learned my addr 172.21.15.45:0/4051554806 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fbe57f700 1 --2- 172.21.15.45:0/4051554806 >> v2:172.21.15.144:3300/0 conn(0x7f7fb8159200 0x7f7fb80c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670650859 0 0) 0x7f7fb80c7d00 con 0x7f7fb80ccdc0 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.391+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fa4003570 con 0x7f7fb80ccdc0 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809262227 0 0) 0x7f7fb80c86d0 con 0x7f7fb81595d0 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb80c7d00 con 0x7f7fb81595d0 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165512640 0 0) 0x7f7fa4003570 con 0x7f7fb80ccdc0 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbe57f700 1 -- 172.21.15.45:0/4051554806 >> v1:172.21.15.144:6789/0 conn(0x7f7fb81595d0 legacy=0x7f7fb806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbe57f700 1 -- 172.21.15.45:0/4051554806 >> v1:172.21.15.45:6789/0 conn(0x7f7fb80ccdc0 legacy=0x7f7fb80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbe57f700 1 -- 172.21.15.45:0/4051554806 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb806bb40 con 0x7f7fb8159200 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbe57f700 1 --2- 172.21.15.45:0/4051554806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb8159200 0x7f7fb80c8190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40098e0 tx=0x7f7fb40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28efd60c901abfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00017f0 con 0x7f7fb8159200 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb4011420 con 0x7f7fb8159200 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.392+0000 7f7fbd57d700 1 -- 172.21.15.45:0/4051554806 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40117c0 con 0x7f7fb8159200 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.393+0000 7f7fc07e3700 1 -- 172.21.15.45:0/4051554806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb8159200 msgr2=0x7f7fb80c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.393+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/4051554806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb8159200 0x7f7fb80c8190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40098e0 tx=0x7f7fb40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.393+0000 7f7fc07e3700 1 -- 172.21.15.45:0/4051554806 shutdown_connections 2023-10-06T05:27:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.393+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/4051554806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb8159200 0x7f7fb80c8190 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.393+0000 7f7fc07e3700 1 -- 172.21.15.45:0/4051554806 >> 172.21.15.45:0/4051554806 conn(0x7f7fb806f090 msgr2=0x7f7fb806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.393+0000 7f7fc07e3700 1 -- 172.21.15.45:0/4051554806 shutdown_connections 2023-10-06T05:27:52.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.393+0000 7f7fc07e3700 1 -- 172.21.15.45:0/4051554806 wait complete. 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.394+0000 7f7fc07e3700 1 Processor -- start 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.394+0000 7f7fc07e3700 1 -- start start 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fc07e3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fb80ccdc0 0x7f7fb815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fc07e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb8159200 0x7f7fb80b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fc07e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb81595d0 0x7f7fb81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fc07e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb815ae90 con 0x7f7fb8159200 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fc07e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb815a280 con 0x7f7fb81595d0 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fc07e3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb815a560 con 0x7f7fb80ccdc0 2023-10-06T05:27:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb8159200 0x7f7fb80b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb8159200 0x7f7fb80b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54716/0 (socket says 172.21.15.45:54716) 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 -- 172.21.15.45:0/1097796813 learned_addr learned my addr 172.21.15.45:0/1097796813 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbe57f700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fb80ccdc0 0x7f7fb815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbed80700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb81595d0 0x7f7fb81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 -- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fb80ccdc0 msgr2=0x7f7fb815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fb80ccdc0 0x7f7fb815b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 -- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb81595d0 msgr2=0x7f7fb81599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb81595d0 0x7f7fb81599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 -- 172.21.15.45:0/1097796813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4009420 con 0x7f7fb8159200 2023-10-06T05:27:52.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbe57f700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fb80ccdc0 0x7f7fb815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbed80700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb81595d0 0x7f7fb81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.395+0000 7f7fbdd7e700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb8159200 0x7f7fb80b25b0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c580 tx=0x7f7fa8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.396+0000 7f7fae7fc700 1 -- 172.21.15.45:0/1097796813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8010510 con 0x7f7fb8159200 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.396+0000 7f7fae7fc700 1 -- 172.21.15.45:0/1097796813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8010670 con 0x7f7fb8159200 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.396+0000 7f7fae7fc700 1 -- 172.21.15.45:0/1097796813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8010a30 con 0x7f7fb8159200 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.396+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb815a840 con 0x7f7fb8159200 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.396+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb80b2af0 con 0x7f7fb8159200 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.397+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb815ace0 con 0x7f7fb8159200 2023-10-06T05:27:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.397+0000 7f7fae7fc700 1 -- 172.21.15.45:0/1097796813 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fa801b400 con 0x7f7fb8159200 2023-10-06T05:27:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.398+0000 7f7fae7fc700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7f9c066f30 0x7f7f9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.398+0000 7f7fae7fc700 1 -- 172.21.15.45:0/1097796813 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f7fa80845c0 con 0x7f7fb8159200 2023-10-06T05:27:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.398+0000 7f7fbe57f700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7f9c066f30 0x7f7f9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.399+0000 7f7fbe57f700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7f9c066f30 0x7f7f9c0693e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40091e0 tx=0x7f7fb4016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.402+0000 7f7fae7fc700 1 -- 172.21.15.45:0/1097796813 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fa8051460 con 0x7f7fb8159200 2023-10-06T05:27:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.650+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fb806be20 con 0x7f7fb8159200 2023-10-06T05:27:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.651+0000 7f7fae7fc700 1 -- 172.21.15.45:0/1097796813 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f7fa8054a80 con 0x7f7fb8159200 2023-10-06T05:27:52.652 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:52.653 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":365,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":365,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:49.992996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7f9c066f30 msgr2=0x7f7f9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7f9c066f30 0x7f7f9c0693e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40091e0 tx=0x7f7fb4016000 comp rx=0 tx=0).stop 2023-10-06T05:27:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb8159200 msgr2=0x7f7fb80b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb8159200 0x7f7fb80b25b0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c580 tx=0x7f7fa8007e10 comp rx=0 tx=0).stop 2023-10-06T05:27:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 shutdown_connections 2023-10-06T05:27:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7f9c066f30 0x7f7f9c0693e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fb81595d0 0x7f7fb81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fb80ccdc0 0x7f7fb815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 --2- 172.21.15.45:0/1097796813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb8159200 0x7f7fb80b25b0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 >> 172.21.15.45:0/1097796813 conn(0x7f7fb806f090 msgr2=0x7f7fb8153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 shutdown_connections 2023-10-06T05:27:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.653+0000 7f7fc07e3700 1 -- 172.21.15.45:0/1097796813 wait complete. 2023-10-06T05:27:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 365 2023-10-06T05:27:52.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:52.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.798+0000 7fd8992cd700 1 Processor -- start 2023-10-06T05:27:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.799+0000 7fd8992cd700 1 -- start start 2023-10-06T05:27:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.799+0000 7fd8992cd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd89406c3d0 0x7fd89406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.799+0000 7fd8992cd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd894156b70 0x7fd894156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.799+0000 7fd8992cd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8940b0790 con 0x7fd89415b890 2023-10-06T05:27:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.799+0000 7fd8992cd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8940b0570 con 0x7fd89406c3d0 2023-10-06T05:27:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.799+0000 7fd8992cd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8940c8e80 con 0x7fd894156b70 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd89406c3d0 0x7fd89406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd89406c3d0 0x7fd89406c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54726/0 (socket says 172.21.15.45:54726) 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 -- 172.21.15.45:0/91826363 learned_addr learned my addr 172.21.15.45:0/91826363 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd8937fe700 1 --2- 172.21.15.45:0/91826363 >> v2:172.21.15.144:3300/0 conn(0x7fd894156b70 0x7fd894156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 -- 172.21.15.45:0/91826363 >> v2:172.21.15.144:3300/0 conn(0x7fd894156b70 msgr2=0x7fd894156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 --2- 172.21.15.45:0/91826363 >> v2:172.21.15.144:3300/0 conn(0x7fd894156b70 0x7fd894156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 -- 172.21.15.45:0/91826363 >> v1:172.21.15.144:6789/0 conn(0x7fd89415b890 legacy=0x7fd8940afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 -- 172.21.15.45:0/91826363 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8940c9160 con 0x7fd89406c3d0 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd892ffd700 1 --2- 172.21.15.45:0/91826363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89406c3d0 0x7fd89406c7a0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fd8940c1950 tx=0x7fd884009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af80805d7cf19b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd891ffb700 1 -- 172.21.15.45:0/91826363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884009e70 con 0x7fd89406c3d0 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd891ffb700 1 -- 172.21.15.45:0/91826363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88400b860 con 0x7fd89406c3d0 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.800+0000 7fd891ffb700 1 -- 172.21.15.45:0/91826363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400bc00 con 0x7fd89406c3d0 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 -- 172.21.15.45:0/91826363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89406c3d0 msgr2=0x7fd89406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 --2- 172.21.15.45:0/91826363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89406c3d0 0x7fd89406c7a0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fd8940c1950 tx=0x7fd884009250 comp rx=0 tx=0).stop 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 -- 172.21.15.45:0/91826363 shutdown_connections 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 --2- 172.21.15.45:0/91826363 >> v2:172.21.15.144:3300/0 conn(0x7fd894156b70 0x7fd894156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 --2- 172.21.15.45:0/91826363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89406c3d0 0x7fd89406c7a0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 -- 172.21.15.45:0/91826363 >> 172.21.15.45:0/91826363 conn(0x7fd894067620 msgr2=0x7fd894067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 -- 172.21.15.45:0/91826363 shutdown_connections 2023-10-06T05:27:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.801+0000 7fd8992cd700 1 -- 172.21.15.45:0/91826363 wait complete. 2023-10-06T05:27:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 Processor -- start 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 -- start start 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd89406c3d0 0x7fd8941556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd894156b70 0x7fd894068730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd89415b890 0x7fd894169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940c1980 con 0x7fd894156b70 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940c1c60 con 0x7fd89406c3d0 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.803+0000 7fd8992cd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd894155c30 con 0x7fd89415b890 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd894156b70 0x7fd894068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd894156b70 0x7fd894068730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54738/0 (socket says 172.21.15.45:54738) 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 -- 172.21.15.45:0/1814640352 learned_addr learned my addr 172.21.15.45:0/1814640352 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd892ffd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd89406c3d0 0x7fd8941556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8937fe700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd89415b890 0x7fd894169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 -- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd89415b890 msgr2=0x7fd894169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd89415b890 0x7fd894169890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 -- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd89406c3d0 msgr2=0x7fd8941556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd89406c3d0 0x7fd8941556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 -- 172.21.15.45:0/1814640352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840094f0 con 0x7fd894156b70 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd892ffd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd89406c3d0 0x7fd8941556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8927fc700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd894156b70 0x7fd894068730 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c5f0 tx=0x7fd87c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd88affd700 1 -- 172.21.15.45:0/1814640352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011590 con 0x7fd894156b70 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd88affd700 1 -- 172.21.15.45:0/1814640352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c011710 con 0x7fd894156b70 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd88affd700 1 -- 172.21.15.45:0/1814640352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011ad0 con 0x7fd894156b70 2023-10-06T05:27:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd894068c70 con 0x7fd894156b70 2023-10-06T05:27:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.804+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89416a070 con 0x7fd894156b70 2023-10-06T05:27:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.805+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89406f160 con 0x7fd894156b70 2023-10-06T05:27:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.806+0000 7fd88affd700 1 -- 172.21.15.45:0/1814640352 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd87c01b3b0 con 0x7fd894156b70 2023-10-06T05:27:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.806+0000 7fd88affd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.806+0000 7fd88affd700 1 -- 172.21.15.45:0/1814640352 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fd87c083c60 con 0x7fd894156b70 2023-10-06T05:27:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.807+0000 7fd892ffd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.807+0000 7fd892ffd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd874066f30 0x7fd8740693e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd8940c1950 tx=0x7fd884017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:52.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:52.811+0000 7fd88affd700 1 -- 172.21.15.45:0/1814640352 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd87c050ba0 con 0x7fd894156b70 2023-10-06T05:27:53.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.017+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8940c1da0 con 0x7fd894156b70 2023-10-06T05:27:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.018+0000 7fd88affd700 1 -- 172.21.15.45:0/1814640352 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+17407 (secure 0 0 0) 0x7fd87c017070 con 0x7fd894156b70 2023-10-06T05:27:53.021 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:53.021 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:27:48.988520+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:18.521771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:19.508254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/2716751458":"2023-10-07T05:27:38.843219+0000","172.21.15.45:6832/2716751458":"2023-10-07T05:27:38.843219+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd874066f30 msgr2=0x7fd8740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd874066f30 0x7fd8740693e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd8940c1950 tx=0x7fd884017000 comp rx=0 tx=0).stop 2023-10-06T05:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd894156b70 msgr2=0x7fd894068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd894156b70 0x7fd894068730 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c5f0 tx=0x7fd87c00c900 comp rx=0 tx=0).stop 2023-10-06T05:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 shutdown_connections 2023-10-06T05:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd89415b890 0x7fd894169890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd89406c3d0 0x7fd8941556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 --2- 172.21.15.45:0/1814640352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd894156b70 0x7fd894068730 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 >> 172.21.15.45:0/1814640352 conn(0x7fd894067620 msgr2=0x7fd8940c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 shutdown_connections 2023-10-06T05:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.020+0000 7fd8992cd700 1 -- 172.21.15.45:0/1814640352 wait complete. 2023-10-06T05:27:53.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:14:52.494626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:14:52.632144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:27:18.521771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:27:19.508254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:27:53.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:27:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.170+0000 7f6a738c8700 1 Processor -- start 2023-10-06T05:27:53.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.170+0000 7f6a738c8700 1 -- start start 2023-10-06T05:27:53.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a738c8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a6c157330 0x7f6a6c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a738c8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0c3b50 con 0x7f6a6c0aaae0 2023-10-06T05:27:53.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a738c8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c157700 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a738c8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c98e0 con 0x7f6a6c157330 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a71664700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a6c157330 0x7f6a6c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a70e63700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6a6c0aaae0 0x7f6a6c0c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53280/0 (socket says 172.21.15.45:53280) 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a70e63700 1 -- 172.21.15.45:0/757982867 learned_addr learned my addr 172.21.15.45:0/757982867 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a71664700 1 -- 172.21.15.45:0/757982867 >> v1:172.21.15.144:6789/0 conn(0x7f6a6c0ccb70 legacy=0x7f6a6c0c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a71664700 1 -- 172.21.15.45:0/757982867 >> v1:172.21.15.45:6789/0 conn(0x7f6a6c0aaae0 legacy=0x7f6a6c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a71664700 1 -- 172.21.15.45:0/757982867 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0c9bc0 con 0x7f6a6c157330 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.171+0000 7f6a63fff700 1 -- 172.21.15.45:0/757982867 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479717593 0 0) 0x7f6a6c0c3b50 con 0x7f6a6c0aaae0 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a71664700 1 --2- 172.21.15.45:0/757982867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c157330 0x7f6a6c0ce8b0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f6a680098d0 tx=0x7f6a68009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53179cb81a06d104 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a63fff700 1 -- 172.21.15.45:0/757982867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68009e70 con 0x7f6a6c157330 2023-10-06T05:27:53.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a63fff700 1 -- 172.21.15.45:0/757982867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a68005c10 con 0x7f6a6c157330 2023-10-06T05:27:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a63fff700 1 -- 172.21.15.45:0/757982867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6800b650 con 0x7f6a6c157330 2023-10-06T05:27:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a738c8700 1 -- 172.21.15.45:0/757982867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c157330 msgr2=0x7f6a6c0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a738c8700 1 --2- 172.21.15.45:0/757982867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c157330 0x7f6a6c0ce8b0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f6a680098d0 tx=0x7f6a68009120 comp rx=0 tx=0).stop 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a738c8700 1 -- 172.21.15.45:0/757982867 shutdown_connections 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a738c8700 1 --2- 172.21.15.45:0/757982867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c157330 0x7f6a6c0ce8b0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a738c8700 1 -- 172.21.15.45:0/757982867 >> 172.21.15.45:0/757982867 conn(0x7f6a6c067e00 msgr2=0x7f6a6c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a738c8700 1 -- 172.21.15.45:0/757982867 shutdown_connections 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.172+0000 7f6a738c8700 1 -- 172.21.15.45:0/757982867 wait complete. 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.174+0000 7f6a738c8700 1 Processor -- start 2023-10-06T05:27:53.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.174+0000 7f6a738c8700 1 -- start start 2023-10-06T05:27:53.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.174+0000 7f6a738c8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6c0aaae0 0x7f6a6c0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.174+0000 7f6a738c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c0ccb70 0x7f6a6c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a738c8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a6c157330 0x7f6a6c167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a738c8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c156ba0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a738c8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c150d90 con 0x7f6a6c0aaae0 2023-10-06T05:27:53.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a738c8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c155b00 con 0x7f6a6c157330 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c0ccb70 0x7f6a6c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c0ccb70 0x7f6a6c1552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54762/0 (socket says 172.21.15.45:54762) 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 -- 172.21.15.45:0/3564443628 learned_addr learned my addr 172.21.15.45:0/3564443628 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a71664700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6c0aaae0 0x7f6a6c0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a71e65700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a6c157330 0x7f6a6c167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 -- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a6c157330 msgr2=0x7f6a6c167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a6c157330 0x7f6a6c167aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 -- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6c0aaae0 msgr2=0x7f6a6c0aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6c0aaae0 0x7f6a6c0aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 -- 172.21.15.45:0/3564443628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680093c0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a70e63700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c0ccb70 0x7f6a6c1552d0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c560 tx=0x7f6a5c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a617fa700 1 -- 172.21.15.45:0/3564443628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c005b30 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a617fa700 1 -- 172.21.15.45:0/3564443628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c005c90 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c155de0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a617fa700 1 -- 172.21.15.45:0/3564443628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c007320 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.175+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c1563a0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.176+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c06e350 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.177+0000 7f6a617fa700 1 -- 172.21.15.45:0/3564443628 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a5c01b3b0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.177+0000 7f6a617fa700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a50066f30 0x7f6a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.177+0000 7f6a617fa700 1 -- 172.21.15.45:0/3564443628 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f6a5c083d90 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.178+0000 7f6a71664700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a50066f30 0x7f6a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.178+0000 7f6a71664700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a50066f30 0x7f6a500693e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009160 tx=0x7f6a68016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.182+0000 7f6a617fa700 1 -- 172.21.15.45:0/3564443628 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a5c050cd0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.384+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a6c0c37e0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.384+0000 7f6a617fa700 1 -- 172.21.15.45:0/3564443628 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a5c0170a0 con 0x7f6a6c0ccb70 2023-10-06T05:27:53.386 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-10-06T05:27:53.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.386+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a50066f30 msgr2=0x7f6a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.386+0000 7f6a738c8700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a50066f30 0x7f6a500693e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009160 tx=0x7f6a68016000 comp rx=0 tx=0).stop 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.386+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c0ccb70 msgr2=0x7f6a6c1552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.386+0000 7f6a738c8700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c0ccb70 0x7f6a6c1552d0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c560 tx=0x7f6a5c000fe0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 shutdown_connections 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6a50066f30 0x7f6a500693e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a6c157330 0x7f6a6c167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a6c0aaae0 0x7f6a6c0aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 --2- 172.21.15.45:0/3564443628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a6c0ccb70 0x7f6a6c1552d0 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 >> 172.21.15.45:0/3564443628 conn(0x7f6a6c067e00 msgr2=0x7f6a6c0c2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 shutdown_connections 2023-10-06T05:27:53.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.387+0000 7f6a738c8700 1 -- 172.21.15.45:0/3564443628 wait complete. 2023-10-06T05:27:53.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:27:53.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.535+0000 7f98beb6d700 1 Processor -- start 2023-10-06T05:27:53.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.535+0000 7f98beb6d700 1 -- start start 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98beb6d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98b80ccb70 0x7f98b80aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98beb6d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b80c96a0 con 0x7f98b8157330 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98beb6d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b80c3b50 con 0x7f98b80c8b70 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98beb6d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b8157700 con 0x7f98b80ccb70 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98b7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98b80ccb70 0x7f98b80aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98bc909700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f98b8157330 0x7f98b80ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53284/0 (socket says 172.21.15.45:53284) 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98b7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98b80ccb70 0x7f98b80aaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54770/0 (socket says 172.21.15.45:54770) 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.536+0000 7f98bc909700 1 -- 172.21.15.45:0/493934981 learned_addr learned my addr 172.21.15.45:0/493934981 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b7fff700 1 -- 172.21.15.45:0/493934981 >> v1:172.21.15.144:6789/0 conn(0x7f98b80c8b70 legacy=0x7f98b80c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b7fff700 1 -- 172.21.15.45:0/493934981 >> v1:172.21.15.45:6789/0 conn(0x7f98b8157330 legacy=0x7f98b80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b7fff700 1 -- 172.21.15.45:0/493934981 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80c98e0 con 0x7f98b80ccb70 2023-10-06T05:27:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b77fe700 1 -- 172.21.15.45:0/493934981 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599736254 0 0) 0x7f98a8001290 con 0x7f98b8157330 2023-10-06T05:27:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b7fff700 1 --2- 172.21.15.45:0/493934981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b80ccb70 0x7f98b80aaae0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0098e0 tx=0x7f98ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53e58598b3ec198 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b77fe700 1 -- 172.21.15.45:0/493934981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac009e80 con 0x7f98b80ccb70 2023-10-06T05:27:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b77fe700 1 -- 172.21.15.45:0/493934981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac00b860 con 0x7f98b80ccb70 2023-10-06T05:27:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.537+0000 7f98b77fe700 1 -- 172.21.15.45:0/493934981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00bbe0 con 0x7f98b80ccb70 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.538+0000 7f98beb6d700 1 -- 172.21.15.45:0/493934981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b80ccb70 msgr2=0x7f98b80aaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.538+0000 7f98beb6d700 1 --2- 172.21.15.45:0/493934981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b80ccb70 0x7f98b80aaae0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0098e0 tx=0x7f98ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.538+0000 7f98beb6d700 1 -- 172.21.15.45:0/493934981 shutdown_connections 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.538+0000 7f98beb6d700 1 --2- 172.21.15.45:0/493934981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b80ccb70 0x7f98b80aaae0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.538+0000 7f98beb6d700 1 -- 172.21.15.45:0/493934981 >> 172.21.15.45:0/493934981 conn(0x7f98b8067e00 msgr2=0x7f98b80cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.538+0000 7f98beb6d700 1 -- 172.21.15.45:0/493934981 shutdown_connections 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.538+0000 7f98beb6d700 1 -- 172.21.15.45:0/493934981 wait complete. 2023-10-06T05:27:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.539+0000 7f98beb6d700 1 Processor -- start 2023-10-06T05:27:53.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.539+0000 7f98beb6d700 1 -- start start 2023-10-06T05:27:53.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98beb6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b80c8b70 0x7f98b80ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98beb6d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b80ccb70 0x7f98b80c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98beb6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b8157330 0x7f98b80b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98beb6d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c60a0 con 0x7f98b8157330 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98beb6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c5490 con 0x7f98b80c8b70 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98beb6d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c5770 con 0x7f98b80ccb70 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b8157330 0x7f98b80b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b8157330 0x7f98b80b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54772/0 (socket says 172.21.15.45:54772) 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 -- 172.21.15.45:0/3066956014 learned_addr learned my addr 172.21.15.45:0/3066956014 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bc909700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b80c8b70 0x7f98b80ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98b7fff700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b80ccb70 0x7f98b80c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 -- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b80ccb70 msgr2=0x7f98b80c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b80ccb70 0x7f98b80c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 -- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b80c8b70 msgr2=0x7f98b80ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b80c8b70 0x7f98b80ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bd10a700 1 -- 172.21.15.45:0/3066956014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac009420 con 0x7f98b8157330 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.540+0000 7f98bc909700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b80c8b70 0x7f98b80ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.541+0000 7f98bd10a700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b8157330 0x7f98b80b2150 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c840 tx=0x7f98b0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.541+0000 7f98b4ff9700 1 -- 172.21.15.45:0/3066956014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0012630 con 0x7f98b8157330 2023-10-06T05:27:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.541+0000 7f98b4ff9700 1 -- 172.21.15.45:0/3066956014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00127b0 con 0x7f98b8157330 2023-10-06T05:27:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.541+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b80c5a50 con 0x7f98b8157330 2023-10-06T05:27:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.541+0000 7f98b4ff9700 1 -- 172.21.15.45:0/3066956014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0012b70 con 0x7f98b8157330 2023-10-06T05:27:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.541+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b80ad770 con 0x7f98b8157330 2023-10-06T05:27:53.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.542+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b80c6250 con 0x7f98b8157330 2023-10-06T05:27:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.542+0000 7f98b4ff9700 1 -- 172.21.15.45:0/3066956014 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98b0003030 con 0x7f98b8157330 2023-10-06T05:27:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.543+0000 7f98b4ff9700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98a4066f30 0x7f98a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.543+0000 7f98b4ff9700 1 -- 172.21.15.45:0/3066956014 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f98b0084820 con 0x7f98b8157330 2023-10-06T05:27:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.543+0000 7f98bc909700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98a4066f30 0x7f98a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.543+0000 7f98bc909700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98a4066f30 0x7f98a40693e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000c00 tx=0x7f98a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.547+0000 7f98b4ff9700 1 -- 172.21.15.45:0/3066956014 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98b80c6250 con 0x7f98b8157330 2023-10-06T05:27:53.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.745+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98b80c6250 con 0x7f98b8157330 2023-10-06T05:27:53.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.745+0000 7f98b4ff9700 1 -- 172.21.15.45:0/3066956014 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v146) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98b80c6250 con 0x7f98b8157330 2023-10-06T05:27:53.747 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-10-06T05:27:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98a4066f30 msgr2=0x7f98a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98a4066f30 0x7f98a40693e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000c00 tx=0x7f98a8009040 comp rx=0 tx=0).stop 2023-10-06T05:27:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b8157330 msgr2=0x7f98b80b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b8157330 0x7f98b80b2150 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c840 tx=0x7f98b0001670 comp rx=0 tx=0).stop 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 shutdown_connections 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f98a4066f30 0x7f98a40693e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98b8157330 0x7f98b80b2150 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98b80c8b70 0x7f98b80ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 --2- 172.21.15.45:0/3066956014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b80ccb70 0x7f98b80c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 >> 172.21.15.45:0/3066956014 conn(0x7f98b8067e00 msgr2=0x7f98b80c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 shutdown_connections 2023-10-06T05:27:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.747+0000 7f98beb6d700 1 -- 172.21.15.45:0/3066956014 wait complete. 2023-10-06T05:27:53.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:27:53.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.896+0000 7f25e4f42700 1 Processor -- start 2023-10-06T05:27:53.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.896+0000 7f25e4f42700 1 -- start start 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25e4f42700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f25e00ccb70 0x7f25e00aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25e4f42700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00c8b70 0x7f25e00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25e4f42700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e00c3b50 con 0x7f25e0157330 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25e4f42700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e0157700 con 0x7f25e00ccb70 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25e4f42700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00c98e0 con 0x7f25e00c8b70 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25de59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f25e0157330 0x7f25e00ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:50018/0 (socket says 172.21.15.45:50018) 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25de59c700 1 -- 172.21.15.45:0/1432980789 learned_addr learned my addr 172.21.15.45:0/1432980789 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25ded9d700 1 --2- 172.21.15.45:0/1432980789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00c8b70 0x7f25e00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25ddd9b700 1 --2- 172.21.15.45:0/1432980789 >> v2:172.21.15.144:3300/0 conn(0x7f25e00ccb70 0x7f25e00aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.897+0000 7f25dd59a700 1 -- 172.21.15.45:0/1432980789 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224610486 0 0) 0x7f25e00c3b50 con 0x7f25e0157330 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25dd59a700 1 -- 172.21.15.45:0/1432980789 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25c8003570 con 0x7f25e0157330 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25ded9d700 1 -- 172.21.15.45:0/1432980789 >> v2:172.21.15.144:3300/0 conn(0x7f25e00ccb70 msgr2=0x7f25e00aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25ded9d700 1 --2- 172.21.15.45:0/1432980789 >> v2:172.21.15.144:3300/0 conn(0x7f25e00ccb70 0x7f25e00aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25ded9d700 1 -- 172.21.15.45:0/1432980789 >> v1:172.21.15.144:6789/0 conn(0x7f25e0157330 legacy=0x7f25e00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25ded9d700 1 -- 172.21.15.45:0/1432980789 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00c9bc0 con 0x7f25e00c8b70 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25ded9d700 1 --2- 172.21.15.45:0/1432980789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00c8b70 0x7f25e00c8fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009e60 tx=0x7f25d8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9908ae71af3699 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25dd59a700 1 -- 172.21.15.45:0/1432980789 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800e070 con 0x7f25e00c8b70 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25dd59a700 1 -- 172.21.15.45:0/1432980789 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800bab0 con 0x7f25e00c8b70 2023-10-06T05:27:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.898+0000 7f25dd59a700 1 -- 172.21.15.45:0/1432980789 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800be70 con 0x7f25e00c8b70 2023-10-06T05:27:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 -- 172.21.15.45:0/1432980789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00c8b70 msgr2=0x7f25e00c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 --2- 172.21.15.45:0/1432980789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00c8b70 0x7f25e00c8fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009e60 tx=0x7f25d8009780 comp rx=0 tx=0).stop 2023-10-06T05:27:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 -- 172.21.15.45:0/1432980789 shutdown_connections 2023-10-06T05:27:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 --2- 172.21.15.45:0/1432980789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00c8b70 0x7f25e00c8fd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 --2- 172.21.15.45:0/1432980789 >> v2:172.21.15.144:3300/0 conn(0x7f25e00ccb70 0x7f25e00aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 -- 172.21.15.45:0/1432980789 >> 172.21.15.45:0/1432980789 conn(0x7f25e0067e00 msgr2=0x7f25e00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 -- 172.21.15.45:0/1432980789 shutdown_connections 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.899+0000 7f25e4f42700 1 -- 172.21.15.45:0/1432980789 wait complete. 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.900+0000 7f25e4f42700 1 Processor -- start 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.900+0000 7f25e4f42700 1 -- start start 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25e4f42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25e00c8b70 0x7f25e0144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25e4f42700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00ccb70 0x7f25e00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25e4f42700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e0157330 0x7f25e00abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25e4f42700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00b0490 con 0x7f25e00c8b70 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25e4f42700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00af880 con 0x7f25e0157330 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25e4f42700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00afb60 con 0x7f25e00ccb70 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25e00c8b70 0x7f25e0144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25e00c8b70 0x7f25e0144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54786/0 (socket says 172.21.15.45:54786) 2023-10-06T05:27:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 -- 172.21.15.45:0/559263334 learned_addr learned my addr 172.21.15.45:0/559263334 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25ded9d700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e0157330 0x7f25e00abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25ddd9b700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00ccb70 0x7f25e00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 -- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00ccb70 msgr2=0x7f25e00aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00ccb70 0x7f25e00aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 -- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e0157330 msgr2=0x7f25e00abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e0157330 0x7f25e00abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 -- 172.21.15.45:0/559263334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8009a20 con 0x7f25e00c8b70 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25ded9d700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e0157330 0x7f25e00abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.901+0000 7f25de59c700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25e00c8b70 0x7f25e0144860 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c790 tx=0x7f25d0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.902+0000 7f25ce7fc700 1 -- 172.21.15.45:0/559263334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0005b80 con 0x7f25e00c8b70 2023-10-06T05:27:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.902+0000 7f25ce7fc700 1 -- 172.21.15.45:0/559263334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d0005ce0 con 0x7f25e00c8b70 2023-10-06T05:27:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.902+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e00afe40 con 0x7f25e00c8b70 2023-10-06T05:27:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.902+0000 7f25ce7fc700 1 -- 172.21.15.45:0/559263334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0007300 con 0x7f25e00c8b70 2023-10-06T05:27:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.902+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e015a4a0 con 0x7f25e00c8b70 2023-10-06T05:27:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.903+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e006e350 con 0x7f25e00c8b70 2023-10-06T05:27:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.903+0000 7f25ce7fc700 1 -- 172.21.15.45:0/559263334 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25d00076b0 con 0x7f25e00c8b70 2023-10-06T05:27:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.903+0000 7f25ce7fc700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0066f60 0x7f25c0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.904+0000 7f25ce7fc700 1 -- 172.21.15.45:0/559263334 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f25d0083d20 con 0x7f25e00c8b70 2023-10-06T05:27:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.904+0000 7f25ddd9b700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0066f60 0x7f25c0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:53.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.905+0000 7f25ddd9b700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0066f60 0x7f25c0069410 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f25e006d4d0 tx=0x7f25d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:53.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:53.908+0000 7f25ce7fc700 1 -- 172.21.15.45:0/559263334 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25d0050c60 con 0x7f25e00c8b70 2023-10-06T05:27:54.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.113+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25e00c37e0 con 0x7f25e00c8b70 2023-10-06T05:27:54.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.113+0000 7f25ce7fc700 1 -- 172.21.15.45:0/559263334 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v146) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25d00170a0 con 0x7f25e00c8b70 2023-10-06T05:27:54.115 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0066f60 msgr2=0x7f25c0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0066f60 0x7f25c0069410 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f25e006d4d0 tx=0x7f25d4008040 comp rx=0 tx=0).stop 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25e00c8b70 msgr2=0x7f25e0144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25e00c8b70 0x7f25e0144860 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c790 tx=0x7f25d0007e00 comp rx=0 tx=0).stop 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 shutdown_connections 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f25c0066f60 0x7f25c0069410 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e0157330 0x7f25e00abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25e00c8b70 0x7f25e0144860 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 --2- 172.21.15.45:0/559263334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e00ccb70 0x7f25e00aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 >> 172.21.15.45:0/559263334 conn(0x7f25e0067e00 msgr2=0x7f25e0156640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 shutdown_connections 2023-10-06T05:27:54.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.116+0000 7f25e4f42700 1 -- 172.21.15.45:0/559263334 wait complete. 2023-10-06T05:27:54.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:27:54.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.264+0000 7fc004fae700 1 Processor -- start 2023-10-06T05:27:54.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.264+0000 7fc004fae700 1 -- start start 2023-10-06T05:27:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fc004fae700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc000156e20 0x7fc00006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fc004fae700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0000cc4e0 0x7fc00006de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fc004fae700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0000cc900 con 0x7fc0001571f0 2023-10-06T05:27:54.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fc004fae700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0000ccb50 con 0x7fc0000cc4e0 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fc004fae700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000cce30 con 0x7fc000156e20 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fbffed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0000cc4e0 0x7fc00006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fbffed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0000cc4e0 0x7fc00006de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54788/0 (socket says 172.21.15.45:54788) 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fbffed9d700 1 -- 172.21.15.45:0/3495208875 learned_addr learned my addr 172.21.15.45:0/3495208875 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.265+0000 7fbffe59c700 1 --2- 172.21.15.45:0/3495208875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc000156e20 0x7fc00006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffed9d700 1 -- 172.21.15.45:0/3495208875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc000156e20 msgr2=0x7fc00006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffed9d700 1 --2- 172.21.15.45:0/3495208875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc000156e20 0x7fc00006d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffed9d700 1 -- 172.21.15.45:0/3495208875 >> v1:172.21.15.144:6789/0 conn(0x7fc0001571f0 legacy=0x7fc0000cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffed9d700 1 -- 172.21.15.45:0/3495208875 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000cd110 con 0x7fc0000cc4e0 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffed9d700 1 --2- 172.21.15.45:0/3495208875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc00006de20 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009f20 tx=0x7fbff8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6042e3950dccb30e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffdd9b700 1 -- 172.21.15.45:0/3495208875 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800e070 con 0x7fc0000cc4e0 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffdd9b700 1 -- 172.21.15.45:0/3495208875 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff800ba80 con 0x7fc0000cc4e0 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fbffdd9b700 1 -- 172.21.15.45:0/3495208875 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80092e0 con 0x7fc0000cc4e0 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fc004fae700 1 -- 172.21.15.45:0/3495208875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 msgr2=0x7fc00006de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.266+0000 7fc004fae700 1 --2- 172.21.15.45:0/3495208875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc00006de20 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009f20 tx=0x7fbff8009840 comp rx=0 tx=0).stop 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.267+0000 7fc004fae700 1 -- 172.21.15.45:0/3495208875 shutdown_connections 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.267+0000 7fc004fae700 1 --2- 172.21.15.45:0/3495208875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc00006de20 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.267+0000 7fc004fae700 1 --2- 172.21.15.45:0/3495208875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc000156e20 0x7fc00006d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.267+0000 7fc004fae700 1 -- 172.21.15.45:0/3495208875 >> 172.21.15.45:0/3495208875 conn(0x7fc0000cea10 msgr2=0x7fc00006c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.267+0000 7fc004fae700 1 -- 172.21.15.45:0/3495208875 shutdown_connections 2023-10-06T05:27:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.267+0000 7fc004fae700 1 -- 172.21.15.45:0/3495208875 wait complete. 2023-10-06T05:27:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.268+0000 7fc004fae700 1 Processor -- start 2023-10-06T05:27:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.268+0000 7fc004fae700 1 -- start start 2023-10-06T05:27:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fc004fae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc000149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fc004fae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc000156e20 0x7fc0000b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fc004fae700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0001571f0 0x7fc0001537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fc004fae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00014dd20 con 0x7fc0000cc4e0 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fc004fae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000b1760 con 0x7fc000156e20 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fc004fae700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000b1a40 con 0x7fc0001571f0 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc000149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc000149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54798/0 (socket says 172.21.15.45:54798) 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 -- 172.21.15.45:0/1335958114 learned_addr learned my addr 172.21.15.45:0/1335958114 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffed9d700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0001571f0 0x7fc0001537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbff7fff700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc000156e20 0x7fc0000b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 -- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0001571f0 msgr2=0x7fc0001537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0001571f0 0x7fc0001537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 -- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc000156e20 msgr2=0x7fc0000b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc000156e20 0x7fc0000b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 -- 172.21.15.45:0/1335958114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff8009ae0 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbffe59c700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc000149b10 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d890 tx=0x7fbff0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.269+0000 7fbff67fc700 1 -- 172.21.15.45:0/1335958114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff00125f0 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.270+0000 7fbff67fc700 1 -- 172.21.15.45:0/1335958114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0012750 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.270+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0000b1d20 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.270+0000 7fbff67fc700 1 -- 172.21.15.45:0/1335958114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0012af0 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.270+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0000b2cb0 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.270+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00014e050 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.271+0000 7fbff67fc700 1 -- 172.21.15.45:0/1335958114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbff001e460 con 0x7fc0000cc4e0 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.271+0000 7fbff67fc700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbfe0066f30 0x7fbfe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.272+0000 7fbff67fc700 1 -- 172.21.15.45:0/1335958114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fbff0084e00 con 0x7fc0000cc4e0 2023-10-06T05:27:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.272+0000 7fbff7fff700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbfe0066f30 0x7fbfe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.272+0000 7fbff7fff700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbfe0066f30 0x7fbfe00693e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc00014f830 tx=0x7fbfe8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.276+0000 7fbff67fc700 1 -- 172.21.15.45:0/1335958114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbff00557d0 con 0x7fc0000cc4e0 2023-10-06T05:27:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.476+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0000b2240 con 0x7fc0000cc4e0 2023-10-06T05:27:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.476+0000 7fbff67fc700 1 -- 172.21.15.45:0/1335958114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v146) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0000b2240 con 0x7fc0000cc4e0 2023-10-06T05:27:54.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-10-06T05:27:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbfe0066f30 msgr2=0x7fbfe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbfe0066f30 0x7fbfe00693e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc00014f830 tx=0x7fbfe8009040 comp rx=0 tx=0).stop 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 msgr2=0x7fc000149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc000149b10 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d890 tx=0x7fbff0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 shutdown_connections 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbfe0066f30 0x7fbfe00693e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0001571f0 0x7fc0001537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0000cc4e0 0x7fc000149b10 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 --2- 172.21.15.45:0/1335958114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc000156e20 0x7fc0000b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.478+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 >> 172.21.15.45:0/1335958114 conn(0x7fc0000cea10 msgr2=0x7fc0000afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.479+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 shutdown_connections 2023-10-06T05:27:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.479+0000 7fc004fae700 1 -- 172.21.15.45:0/1335958114 wait complete. 2023-10-06T05:27:54.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:54.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.624+0000 7fd48d9b3700 1 Processor -- start 2023-10-06T05:27:54.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.624+0000 7fd48d9b3700 1 -- start start 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd48d9b3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd48815b3e0 0x7fd488067bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd48d9b3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4880ca360 0x7fd488140410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd48d9b3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4880c9ed0 con 0x7fd4880677e0 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd48d9b3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4880cd870 con 0x7fd48815b3e0 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd48d9b3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48815bc00 con 0x7fd4880ca360 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd486ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd4880677e0 0x7fd48815acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53300/0 (socket says 172.21.15.45:53300) 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd486ffd700 1 -- 172.21.15.45:0/1605749189 learned_addr learned my addr 172.21.15.45:0/1605749189 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd47e7fc700 1 --2- 172.21.15.45:0/1605749189 >> v2:172.21.15.144:3300/0 conn(0x7fd48815b3e0 0x7fd488067bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd4877fe700 1 --2- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4880ca360 0x7fd488140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd4867fc700 1 -- 172.21.15.45:0/1605749189 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190800722 0 0) 0x7fd4880c9ed0 con 0x7fd4880677e0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.625+0000 7fd4867fc700 1 -- 172.21.15.45:0/1605749189 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd474003570 con 0x7fd4880677e0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd47e7fc700 1 -- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4880ca360 msgr2=0x7fd488140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd47e7fc700 1 --2- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4880ca360 0x7fd488140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd47e7fc700 1 -- 172.21.15.45:0/1605749189 >> v1:172.21.15.45:6789/0 conn(0x7fd4880677e0 legacy=0x7fd48815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd47e7fc700 1 -- 172.21.15.45:0/1605749189 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48815b920 con 0x7fd48815b3e0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd47e7fc700 1 --2- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48815b3e0 0x7fd488067bb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd4700098e0 tx=0x7fd4700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38680c840c7be750 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd4867fc700 1 -- 172.21.15.45:0/1605749189 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470011630 con 0x7fd48815b3e0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd4867fc700 1 -- 172.21.15.45:0/1605749189 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd470011790 con 0x7fd48815b3e0 2023-10-06T05:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.626+0000 7fd4867fc700 1 -- 172.21.15.45:0/1605749189 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470011b30 con 0x7fd48815b3e0 2023-10-06T05:27:54.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 -- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48815b3e0 msgr2=0x7fd488067bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48815b3e0 0x7fd488067bb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd4700098e0 tx=0x7fd4700091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 -- 172.21.15.45:0/1605749189 shutdown_connections 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4880ca360 0x7fd488140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/1605749189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48815b3e0 0x7fd488067bb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 -- 172.21.15.45:0/1605749189 >> 172.21.15.45:0/1605749189 conn(0x7fd488069200 msgr2=0x7fd4880cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 -- 172.21.15.45:0/1605749189 shutdown_connections 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.627+0000 7fd48d9b3700 1 -- 172.21.15.45:0/1605749189 wait complete. 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.628+0000 7fd48d9b3700 1 Processor -- start 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.628+0000 7fd48d9b3700 1 -- start start 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd48d9b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4880677e0 0x7fd48816dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd48d9b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4880ca360 0x7fd48816e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd48d9b3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48815b3e0 0x7fd48816ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd48d9b3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880c4970 con 0x7fd4880677e0 2023-10-06T05:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd48d9b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880cb3c0 con 0x7fd4880ca360 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd48d9b3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48816ef90 con 0x7fd48815b3e0 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4880677e0 0x7fd48816dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4880677e0 0x7fd48816dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54800/0 (socket says 172.21.15.45:54800) 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 -- 172.21.15.45:0/2162730276 learned_addr learned my addr 172.21.15.45:0/2162730276 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd47e7fc700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4880ca360 0x7fd48816e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd4877fe700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48815b3e0 0x7fd48816ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 -- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48815b3e0 msgr2=0x7fd48816ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48815b3e0 0x7fd48816ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 -- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4880ca360 msgr2=0x7fd48816e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4880ca360 0x7fd48816e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.629+0000 7fd486ffd700 1 -- 172.21.15.45:0/2162730276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd470009420 con 0x7fd4880677e0 2023-10-06T05:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.630+0000 7fd486ffd700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4880677e0 0x7fd48816dfd0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c590 tx=0x7fd478007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.630+0000 7fd47f7fe700 1 -- 172.21.15.45:0/2162730276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478005b30 con 0x7fd4880677e0 2023-10-06T05:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.630+0000 7fd47f7fe700 1 -- 172.21.15.45:0/2162730276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478005cb0 con 0x7fd4880677e0 2023-10-06T05:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.630+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4880c71e0 con 0x7fd4880677e0 2023-10-06T05:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.630+0000 7fd47f7fe700 1 -- 172.21.15.45:0/2162730276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478007320 con 0x7fd4880677e0 2023-10-06T05:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.630+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4880cb940 con 0x7fd4880677e0 2023-10-06T05:27:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.631+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48806dfd0 con 0x7fd4880677e0 2023-10-06T05:27:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.631+0000 7fd47f7fe700 1 -- 172.21.15.45:0/2162730276 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd47801b3b0 con 0x7fd4880677e0 2023-10-06T05:27:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.632+0000 7fd47f7fe700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.632+0000 7fd47f7fe700 1 -- 172.21.15.45:0/2162730276 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7fd478083ce0 con 0x7fd4880677e0 2023-10-06T05:27:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.632+0000 7fd47e7fc700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.633+0000 7fd47e7fc700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd4700091e0 tx=0x7fd470017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:54.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.636+0000 7fd47f7fe700 1 -- 172.21.15.45:0/2162730276 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd478050c20 con 0x7fd4880677e0 2023-10-06T05:27:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.883+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4880c3350 con 0x7fd4880677e0 2023-10-06T05:27:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.884+0000 7fd47f7fe700 1 -- 172.21.15.45:0/2162730276 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) v1 ==== 95+0+4610 (secure 0 0 0) 0x7fd478017020 con 0x7fd4880677e0 2023-10-06T05:27:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:54.886 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":365,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":365,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:49.992996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 msgr2=0x7fd4680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd4700091e0 tx=0x7fd470017000 comp rx=0 tx=0).stop 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4880677e0 msgr2=0x7fd48816dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4880677e0 0x7fd48816dfd0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c590 tx=0x7fd478007e00 comp rx=0 tx=0).stop 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 shutdown_connections 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd468066f30 0x7fd4680693e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48815b3e0 0x7fd48816ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4880677e0 0x7fd48816dfd0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 --2- 172.21.15.45:0/2162730276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4880ca360 0x7fd48816e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 >> 172.21.15.45:0/2162730276 conn(0x7fd488069200 msgr2=0x7fd488159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 shutdown_connections 2023-10-06T05:27:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:54.886+0000 7fd48d9b3700 1 -- 172.21.15.45:0/2162730276 wait complete. 2023-10-06T05:27:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 365 2023-10-06T05:27:54.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:55.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.031+0000 7f451bec6700 1 Processor -- start 2023-10-06T05:27:55.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.031+0000 7f451bec6700 1 -- start start 2023-10-06T05:27:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451bec6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f45140c8ff0 0x7f45140c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451bec6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4514157200 0x7f45140caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451bec6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451415b000 con 0x7f45140caad0 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451bec6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451415b2e0 con 0x7f4514157200 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451bec6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451415b5c0 con 0x7f45140c8ff0 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451a463700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4514157200 0x7f45140caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451a463700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4514157200 0x7f45140caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54812/0 (socket says 172.21.15.45:54812) 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451a463700 1 -- 172.21.15.45:0/516953072 learned_addr learned my addr 172.21.15.45:0/516953072 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f4519c62700 1 --2- 172.21.15.45:0/516953072 >> v2:172.21.15.144:3300/0 conn(0x7f45140c8ff0 0x7f45140c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451a463700 1 -- 172.21.15.45:0/516953072 >> v2:172.21.15.144:3300/0 conn(0x7f45140c8ff0 msgr2=0x7f45140c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.032+0000 7f451a463700 1 --2- 172.21.15.45:0/516953072 >> v2:172.21.15.144:3300/0 conn(0x7f45140c8ff0 0x7f45140c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451a463700 1 -- 172.21.15.45:0/516953072 >> v1:172.21.15.144:6789/0 conn(0x7f45140caad0 legacy=0x7f4514156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451a463700 1 -- 172.21.15.45:0/516953072 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451415b8a0 con 0x7f4514157200 2023-10-06T05:27:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451a463700 1 --2- 172.21.15.45:0/516953072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514157200 0x7f45140caf40 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f45140cefc0 tx=0x7f450c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11f4e1dd809039c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f4518c60700 1 -- 172.21.15.45:0/516953072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00e070 con 0x7f4514157200 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f4518c60700 1 -- 172.21.15.45:0/516953072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c00ba20 con 0x7f4514157200 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f4518c60700 1 -- 172.21.15.45:0/516953072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c0092e0 con 0x7f4514157200 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451bec6700 1 -- 172.21.15.45:0/516953072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514157200 msgr2=0x7f45140caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451bec6700 1 --2- 172.21.15.45:0/516953072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514157200 0x7f45140caf40 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f45140cefc0 tx=0x7f450c009840 comp rx=0 tx=0).stop 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451bec6700 1 -- 172.21.15.45:0/516953072 shutdown_connections 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451bec6700 1 --2- 172.21.15.45:0/516953072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514157200 0x7f45140caf40 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451bec6700 1 --2- 172.21.15.45:0/516953072 >> v2:172.21.15.144:3300/0 conn(0x7f45140c8ff0 0x7f45140c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.033+0000 7f451bec6700 1 -- 172.21.15.45:0/516953072 >> 172.21.15.45:0/516953072 conn(0x7f45140cdf50 msgr2=0x7f45140ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.034+0000 7f451bec6700 1 -- 172.21.15.45:0/516953072 shutdown_connections 2023-10-06T05:27:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.034+0000 7f451bec6700 1 -- 172.21.15.45:0/516953072 wait complete. 2023-10-06T05:27:55.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.035+0000 7f451bec6700 1 Processor -- start 2023-10-06T05:27:55.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.035+0000 7f451bec6700 1 -- start start 2023-10-06T05:27:55.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451bec6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45140c8ff0 0x7f451414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451bec6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45140caad0 0x7f451414fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451bec6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514157200 0x7f451414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451bec6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45140aeef0 con 0x7f45140caad0 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451bec6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45140ae5c0 con 0x7f4514157200 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451bec6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45140ae8a0 con 0x7f45140c8ff0 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45140caad0 0x7f451414fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45140caad0 0x7f451414fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54814/0 (socket says 172.21.15.45:54814) 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 -- 172.21.15.45:0/2087824393 learned_addr learned my addr 172.21.15.45:0/2087824393 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519c62700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45140c8ff0 0x7f451414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451a463700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514157200 0x7f451414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 -- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45140c8ff0 msgr2=0x7f451414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45140c8ff0 0x7f451414a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 -- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514157200 msgr2=0x7f451414ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514157200 0x7f451414ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 -- 172.21.15.45:0/2087824393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c009ae0 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4519461700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45140caad0 0x7f451414fa30 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f450800c790 tx=0x7f4508007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4505ffb700 1 -- 172.21.15.45:0/2087824393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508011590 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f4505ffb700 1 -- 172.21.15.45:0/2087824393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4508011710 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.036+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45140aeb80 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.037+0000 7f4505ffb700 1 -- 172.21.15.45:0/2087824393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508011ad0 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.037+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45141676c0 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.038+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45140af060 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.038+0000 7f4505ffb700 1 -- 172.21.15.45:0/2087824393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f450801b3b0 con 0x7f45140caad0 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.039+0000 7f4505ffb700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44f8066f30 0x7f44f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.039+0000 7f4505ffb700 1 -- 172.21.15.45:0/2087824393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f4508083d80 con 0x7f45140caad0 2023-10-06T05:27:55.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.039+0000 7f4519c62700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44f8066f30 0x7f44f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.039+0000 7f4519c62700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44f8066f30 0x7f44f80693e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f45140c9720 tx=0x7f4510008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.043+0000 7f4505ffb700 1 -- 172.21.15.45:0/2087824393 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4508050cc0 con 0x7f45140caad0 2023-10-06T05:27:55.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.247+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f451404bb40 con 0x7f45140caad0 2023-10-06T05:27:55.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.248+0000 7f4505ffb700 1 -- 172.21.15.45:0/2087824393 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f4508017070 con 0x7f45140caad0 2023-10-06T05:27:55.250 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:55.250 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:27:48.988520+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:18.521771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:19.508254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/2716751458":"2023-10-07T05:27:38.843219+0000","172.21.15.45:6832/2716751458":"2023-10-07T05:27:38.843219+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:55.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44f8066f30 msgr2=0x7f44f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44f8066f30 0x7f44f80693e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f45140c9720 tx=0x7f4510008040 comp rx=0 tx=0).stop 2023-10-06T05:27:55.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45140caad0 msgr2=0x7f451414fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45140caad0 0x7f451414fa30 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f450800c790 tx=0x7f4508007f40 comp rx=0 tx=0).stop 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 shutdown_connections 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f44f8066f30 0x7f44f80693e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4514157200 0x7f451414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45140c8ff0 0x7f451414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 --2- 172.21.15.45:0/2087824393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45140caad0 0x7f451414fa30 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 >> 172.21.15.45:0/2087824393 conn(0x7f45140cdf50 msgr2=0x7f451406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 shutdown_connections 2023-10-06T05:27:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.250+0000 7f451bec6700 1 -- 172.21.15.45:0/2087824393 wait complete. 2023-10-06T05:27:55.266 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:27:55.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:55.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.396+0000 7f8fb6663700 1 Processor -- start 2023-10-06T05:27:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.397+0000 7f8fb6663700 1 -- start start 2023-10-06T05:27:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.397+0000 7f8fb6663700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fb0159200 0x7f8fb00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.397+0000 7f8fb6663700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fb01595d0 0x7f8fb006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.397+0000 7f8fb6663700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb00c7d00 con 0x7f8fb006b5c0 2023-10-06T05:27:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.397+0000 7f8fb6663700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb00c7780 con 0x7f8fb01595d0 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.397+0000 7f8fb6663700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb00c86d0 con 0x7f8fb0159200 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fb01595d0 0x7f8fb006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8fb4c00700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8fb006b5c0 0x7f8fb00ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53314/0 (socket says 172.21.15.45:53314) 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8fb4c00700 1 -- 172.21.15.45:0/2947411081 learned_addr learned my addr 172.21.15.45:0/2947411081 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faffff700 1 --2- 172.21.15.45:0/2947411081 >> v2:172.21.15.144:3300/0 conn(0x7f8fb0159200 0x7f8fb00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faf7fe700 1 -- 172.21.15.45:0/2947411081 >> v2:172.21.15.144:3300/0 conn(0x7f8fb0159200 msgr2=0x7f8fb00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faf7fe700 1 --2- 172.21.15.45:0/2947411081 >> v2:172.21.15.144:3300/0 conn(0x7f8fb0159200 0x7f8fb00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faf7fe700 1 -- 172.21.15.45:0/2947411081 >> v1:172.21.15.45:6789/0 conn(0x7f8fb006b5c0 legacy=0x7f8fb00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faf7fe700 1 -- 172.21.15.45:0/2947411081 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb006bb40 con 0x7f8fb01595d0 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faeffd700 1 -- 172.21.15.45:0/2947411081 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942923372 0 0) 0x7f8fb00c7d00 con 0x7f8fb006b5c0 2023-10-06T05:27:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faf7fe700 1 --2- 172.21.15.45:0/2947411081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb01595d0 0x7f8fb006b080 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f8fb006ae50 tx=0x7f8fa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd82abd3e5cba23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faeffd700 1 -- 172.21.15.45:0/2947411081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4011630 con 0x7f8fb01595d0 2023-10-06T05:27:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faeffd700 1 -- 172.21.15.45:0/2947411081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa4011790 con 0x7f8fb01595d0 2023-10-06T05:27:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.398+0000 7f8faeffd700 1 -- 172.21.15.45:0/2947411081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4011b30 con 0x7f8fb01595d0 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 -- 172.21.15.45:0/2947411081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb01595d0 msgr2=0x7f8fb006b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 --2- 172.21.15.45:0/2947411081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb01595d0 0x7f8fb006b080 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f8fb006ae50 tx=0x7f8fa40091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 -- 172.21.15.45:0/2947411081 shutdown_connections 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 --2- 172.21.15.45:0/2947411081 >> v2:172.21.15.144:3300/0 conn(0x7f8fb0159200 0x7f8fb00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 --2- 172.21.15.45:0/2947411081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb01595d0 0x7f8fb006b080 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 -- 172.21.15.45:0/2947411081 >> 172.21.15.45:0/2947411081 conn(0x7f8fb006f090 msgr2=0x7f8fb006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 -- 172.21.15.45:0/2947411081 shutdown_connections 2023-10-06T05:27:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.399+0000 7f8fb6663700 1 -- 172.21.15.45:0/2947411081 wait complete. 2023-10-06T05:27:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.401+0000 7f8fb6663700 1 Processor -- start 2023-10-06T05:27:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.401+0000 7f8fb6663700 1 -- start start 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8fb6663700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fb006b5c0 0x7f8fb01697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8fb6663700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb0159200 0x7f8fb00c0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8fb6663700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb01595d0 0x7f8fb016d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8fb6663700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00ccf80 con 0x7f8fb0159200 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8fb6663700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0169cf0 con 0x7f8fb01595d0 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8fb6663700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00cd410 con 0x7f8fb006b5c0 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb0159200 0x7f8fb00c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb0159200 0x7f8fb00c0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54834/0 (socket says 172.21.15.45:54834) 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faf7fe700 1 -- 172.21.15.45:0/356775084 learned_addr learned my addr 172.21.15.45:0/356775084 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faffff700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fb006b5c0 0x7f8fb01697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8fb4c00700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb01595d0 0x7f8fb016d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faf7fe700 1 -- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fb006b5c0 msgr2=0x7f8fb01697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faf7fe700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fb006b5c0 0x7f8fb01697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faf7fe700 1 -- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb01595d0 msgr2=0x7f8fb016d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.402+0000 7f8faf7fe700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb01595d0 0x7f8fb016d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8faf7fe700 1 -- 172.21.15.45:0/356775084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa4009420 con 0x7f8fb0159200 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8faffff700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fb006b5c0 0x7f8fb01697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8fb4c00700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb01595d0 0x7f8fb016d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8faf7fe700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb0159200 0x7f8fb00c0890 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f8fb006aed0 tx=0x7f8fa4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8f97fff700 1 -- 172.21.15.45:0/356775084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4011b10 con 0x7f8fb0159200 2023-10-06T05:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8f97fff700 1 -- 172.21.15.45:0/356775084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa40253f0 con 0x7f8fb0159200 2023-10-06T05:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb00cc670 con 0x7f8fb0159200 2023-10-06T05:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8f97fff700 1 -- 172.21.15.45:0/356775084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4025790 con 0x7f8fb0159200 2023-10-06T05:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.403+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb00ccc10 con 0x7f8fb0159200 2023-10-06T05:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.404+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb006e2f0 con 0x7f8fb0159200 2023-10-06T05:27:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.404+0000 7f8f97fff700 1 -- 172.21.15.45:0/356775084 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fa4025b80 con 0x7f8fb0159200 2023-10-06T05:27:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.405+0000 7f8f97fff700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f9c066f30 0x7f8f9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.405+0000 7f8f97fff700 1 -- 172.21.15.45:0/356775084 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f8fa40860c0 con 0x7f8fb0159200 2023-10-06T05:27:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.405+0000 7f8faffff700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f9c066f30 0x7f8f9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.406+0000 7f8faffff700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f9c066f30 0x7f8f9c0693e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009d80 tx=0x7f8fa0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.409+0000 7f8f97fff700 1 -- 172.21.15.45:0/356775084 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fa4053000 con 0x7f8fb0159200 2023-10-06T05:27:55.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.655+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fb006be20 con 0x7f8fb0159200 2023-10-06T05:27:55.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.655+0000 7f8f97fff700 1 -- 172.21.15.45:0/356775084 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f8fa4056620 con 0x7f8fb0159200 2023-10-06T05:27:55.657 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:55.657 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":365,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":365,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:49.992996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:55.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f9c066f30 msgr2=0x7f8f9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f9c066f30 0x7f8f9c0693e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009d80 tx=0x7f8fa0008040 comp rx=0 tx=0).stop 2023-10-06T05:27:55.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb0159200 msgr2=0x7f8fb00c0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb0159200 0x7f8fb00c0890 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f8fb006aed0 tx=0x7f8fa4017000 comp rx=0 tx=0).stop 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 shutdown_connections 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f8f9c066f30 0x7f8f9c0693e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb01595d0 0x7f8fb016d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fb006b5c0 0x7f8fb01697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 --2- 172.21.15.45:0/356775084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fb0159200 0x7f8fb00c0890 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 >> 172.21.15.45:0/356775084 conn(0x7f8fb006f090 msgr2=0x7f8fb00ad7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 shutdown_connections 2023-10-06T05:27:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.657+0000 7f8fb6663700 1 -- 172.21.15.45:0/356775084 wait complete. 2023-10-06T05:27:55.661 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 365 2023-10-06T05:27:55.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:27:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.803+0000 7f182c09e700 1 Processor -- start 2023-10-06T05:27:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.803+0000 7f182c09e700 1 -- start start 2023-10-06T05:27:55.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f182c09e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18240cd340 0x7f1824157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f182c09e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182415b000 con 0x7f18240c93c0 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f182c09e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182415b2e0 con 0x7f18240c8ff0 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f182c09e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240c9830 con 0x7f18240cd340 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f182a63b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18240cd340 0x7f1824157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f1829639700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f18240c93c0 0x7f1824156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53320/0 (socket says 172.21.15.45:53320) 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f1829639700 1 -- 172.21.15.45:0/3757137100 learned_addr learned my addr 172.21.15.45:0/3757137100 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.804+0000 7f182a63b700 1 -- 172.21.15.45:0/3757137100 >> v1:172.21.15.144:6789/0 conn(0x7f18240c8ff0 legacy=0x7f18240caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f182a63b700 1 -- 172.21.15.45:0/3757137100 >> v1:172.21.15.45:6789/0 conn(0x7f18240c93c0 legacy=0x7f1824156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f1828e38700 1 -- 172.21.15.45:0/3757137100 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120187733 0 0) 0x7f182415b000 con 0x7f18240c93c0 2023-10-06T05:27:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f182a63b700 1 -- 172.21.15.45:0/3757137100 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182415b5c0 con 0x7f18240cd340 2023-10-06T05:27:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f182a63b700 1 --2- 172.21.15.45:0/3757137100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f1824157200 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f181c009e60 tx=0x7f181c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddcc7ea3c204093 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f1828e38700 1 -- 172.21.15.45:0/3757137100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00e070 con 0x7f18240cd340 2023-10-06T05:27:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f1828e38700 1 -- 172.21.15.45:0/3757137100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00bc30 con 0x7f18240cd340 2023-10-06T05:27:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f1828e38700 1 -- 172.21.15.45:0/3757137100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c012430 con 0x7f18240cd340 2023-10-06T05:27:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f182c09e700 1 -- 172.21.15.45:0/3757137100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 msgr2=0x7f1824157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.805+0000 7f182c09e700 1 --2- 172.21.15.45:0/3757137100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f1824157200 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f181c009e60 tx=0x7f181c009780 comp rx=0 tx=0).stop 2023-10-06T05:27:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.806+0000 7f182c09e700 1 -- 172.21.15.45:0/3757137100 shutdown_connections 2023-10-06T05:27:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.806+0000 7f182c09e700 1 --2- 172.21.15.45:0/3757137100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f1824157200 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.806+0000 7f182c09e700 1 -- 172.21.15.45:0/3757137100 >> 172.21.15.45:0/3757137100 conn(0x7f18240cdf50 msgr2=0x7f18240ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.806+0000 7f182c09e700 1 -- 172.21.15.45:0/3757137100 shutdown_connections 2023-10-06T05:27:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.806+0000 7f182c09e700 1 -- 172.21.15.45:0/3757137100 wait complete. 2023-10-06T05:27:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.807+0000 7f182c09e700 1 Processor -- start 2023-10-06T05:27:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.807+0000 7f182c09e700 1 -- start start 2023-10-06T05:27:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182c09e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18240c8ff0 0x7f18240c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182c09e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18240c93c0 0x7f1824146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182c09e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f18240b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182c09e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18241468b0 con 0x7f18240cd340 2023-10-06T05:27:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182c09e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240b2450 con 0x7f18240c8ff0 2023-10-06T05:27:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182c09e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240b2730 con 0x7f18240c93c0 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182a63b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f18240b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182a63b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f18240b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54846/0 (socket says 172.21.15.45:54846) 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.808+0000 7f182a63b700 1 -- 172.21.15.45:0/3575032490 learned_addr learned my addr 172.21.15.45:0/3575032490 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f1829e3a700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18240c8ff0 0x7f18240c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f1829639700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18240c93c0 0x7f1824146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182a63b700 1 -- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18240c93c0 msgr2=0x7f1824146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182a63b700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18240c93c0 0x7f1824146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182a63b700 1 -- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18240c8ff0 msgr2=0x7f18240c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182a63b700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18240c8ff0 0x7f18240c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182a63b700 1 -- 172.21.15.45:0/3575032490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c009a20 con 0x7f18240cd340 2023-10-06T05:27:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182a63b700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f18240b0cf0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f181c013000 tx=0x7f181c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f1819ffb700 1 -- 172.21.15.45:0/3575032490 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c01b420 con 0x7f18240cd340 2023-10-06T05:27:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f1819ffb700 1 -- 172.21.15.45:0/3575032490 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c01b580 con 0x7f18240cd340 2023-10-06T05:27:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18240b2a10 con 0x7f18240cd340 2023-10-06T05:27:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f1819ffb700 1 -- 172.21.15.45:0/3575032490 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c01b920 con 0x7f18240cd340 2023-10-06T05:27:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.809+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18240b2fd0 con 0x7f18240cd340 2023-10-06T05:27:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.810+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182406f410 con 0x7f18240cd340 2023-10-06T05:27:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.811+0000 7f1819ffb700 1 -- 172.21.15.45:0/3575032490 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f181c01bc90 con 0x7f18240cd340 2023-10-06T05:27:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.811+0000 7f1819ffb700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1808066f30 0x7f18080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.811+0000 7f1819ffb700 1 -- 172.21.15.45:0/3575032490 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f181c086670 con 0x7f18240cd340 2023-10-06T05:27:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.812+0000 7f1829e3a700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1808066f30 0x7f18080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.812+0000 7f1829e3a700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1808066f30 0x7f18080693e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1820003a10 tx=0x7f1820007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:55.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:55.816+0000 7f1819ffb700 1 -- 172.21.15.45:0/3575032490 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f181c0535b0 con 0x7f18240cd340 2023-10-06T05:27:56.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.065+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f182404bb40 con 0x7f18240cd340 2023-10-06T05:27:56.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.065+0000 7f1819ffb700 1 -- 172.21.15.45:0/3575032490 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 365 v365) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f181c056bd0 con 0x7f18240cd340 2023-10-06T05:27:56.067 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:56.067 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":365,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353}],"filesystems":[{"mdsmap":{"epoch":365,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:27:20.533957+0000","modified":"2023-10-06T05:27:49.992996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11019":{"gid":11019,"name":"a","rank":0,"incarnation":361,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6833/4232324777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4232324777},{"type":"v1","addr":"172.21.15.45:6833","nonce":4232324777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T05:27:56.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1808066f30 msgr2=0x7f18080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1808066f30 0x7f18080693e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1820003a10 tx=0x7f1820007ec0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 msgr2=0x7f18240b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f18240b0cf0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f181c013000 tx=0x7f181c019000 comp rx=0 tx=0).stop 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 shutdown_connections 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f1808066f30 0x7f18080693e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18240cd340 0x7f18240b0cf0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18240c8ff0 0x7f18240c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 --2- 172.21.15.45:0/3575032490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18240c93c0 0x7f1824146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.067+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 >> 172.21.15.45:0/3575032490 conn(0x7f18240cdf50 msgr2=0x7f18240ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.068+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 shutdown_connections 2023-10-06T05:27:56.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.068+0000 7f182c09e700 1 -- 172.21.15.45:0/3575032490 wait complete. 2023-10-06T05:27:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 365 2023-10-06T05:27:56.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:27:56.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.212+0000 7f93162cc700 1 Processor -- start 2023-10-06T05:27:56.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.212+0000 7f93162cc700 1 -- start start 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f93162cc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93100692d0 0x7f9310159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f93162cc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f931015d040 0x7f93100aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f93162cc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93100ca6d0 con 0x7f9310159170 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f93162cc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931015d550 con 0x7f93100692d0 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f93162cc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931015d810 con 0x7f931015d040 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f930f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93100692d0 0x7f9310159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f930ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9310159170 0x7f9310068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53324/0 (socket says 172.21.15.45:53324) 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f930ffff700 1 -- 172.21.15.45:0/1937405327 learned_addr learned my addr 172.21.15.45:0/1937405327 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.213+0000 7f930f7fe700 1 --2- 172.21.15.45:0/1937405327 >> v2:172.21.15.45:3300/0 conn(0x7f93100692d0 0x7f9310159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54852/0 (socket says 172.21.15.45:54852) 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f9314869700 1 --2- 172.21.15.45:0/1937405327 >> v2:172.21.15.144:3300/0 conn(0x7f931015d040 0x7f93100aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930f7fe700 1 -- 172.21.15.45:0/1937405327 >> v2:172.21.15.144:3300/0 conn(0x7f931015d040 msgr2=0x7f93100aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930f7fe700 1 --2- 172.21.15.45:0/1937405327 >> v2:172.21.15.144:3300/0 conn(0x7f931015d040 0x7f93100aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930f7fe700 1 -- 172.21.15.45:0/1937405327 >> v1:172.21.15.45:6789/0 conn(0x7f9310159170 legacy=0x7f9310068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930f7fe700 1 -- 172.21.15.45:0/1937405327 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931015daf0 con 0x7f93100692d0 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930f7fe700 1 --2- 172.21.15.45:0/1937405327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f9310159540 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f93100c70c0 tx=0x7f93040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3d742b1c39497a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930effd700 1 -- 172.21.15.45:0/1937405327 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304011630 con 0x7f93100692d0 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930effd700 1 -- 172.21.15.45:0/1937405327 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9304011790 con 0x7f93100692d0 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.214+0000 7f930effd700 1 -- 172.21.15.45:0/1937405327 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304011b30 con 0x7f93100692d0 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 -- 172.21.15.45:0/1937405327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 msgr2=0x7f9310159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 --2- 172.21.15.45:0/1937405327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f9310159540 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f93100c70c0 tx=0x7f93040091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 -- 172.21.15.45:0/1937405327 shutdown_connections 2023-10-06T05:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 --2- 172.21.15.45:0/1937405327 >> v2:172.21.15.144:3300/0 conn(0x7f931015d040 0x7f93100aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 --2- 172.21.15.45:0/1937405327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f9310159540 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 -- 172.21.15.45:0/1937405327 >> 172.21.15.45:0/1937405327 conn(0x7f931006f7c0 msgr2=0x7f93100d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 -- 172.21.15.45:0/1937405327 shutdown_connections 2023-10-06T05:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.215+0000 7f93162cc700 1 -- 172.21.15.45:0/1937405327 wait complete. 2023-10-06T05:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.216+0000 7f93162cc700 1 Processor -- start 2023-10-06T05:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.216+0000 7f93162cc700 1 -- start start 2023-10-06T05:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f93162cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f93100a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f93162cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9310159170 0x7f93100aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f93162cc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f931015d040 0x7f93100af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f93162cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100ca020 con 0x7f93100692d0 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f93162cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100c9d40 con 0x7f9310159170 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f93162cc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931015a8d0 con 0x7f931015d040 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f93100a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f93100a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54866/0 (socket says 172.21.15.45:54866) 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 -- 172.21.15.45:0/87985525 learned_addr learned my addr 172.21.15.45:0/87985525 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930f7fe700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9310159170 0x7f93100aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f9314869700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f931015d040 0x7f93100af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 -- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f931015d040 msgr2=0x7f93100af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f931015d040 0x7f93100af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 -- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9310159170 msgr2=0x7f93100aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9310159170 0x7f93100aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.217+0000 7f930ffff700 1 -- 172.21.15.45:0/87985525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304009420 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.218+0000 7f930ffff700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f93100a6c30 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f930000c810 tx=0x7f9300007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.218+0000 7f92f7fff700 1 -- 172.21.15.45:0/87985525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300005b30 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.218+0000 7f92f7fff700 1 -- 172.21.15.45:0/87985525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300005c90 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.218+0000 7f92f7fff700 1 -- 172.21.15.45:0/87985525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300007320 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.218+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931015abb0 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.218+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93100a7170 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.219+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931015a730 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.219+0000 7f92f7fff700 1 -- 172.21.15.45:0/87985525 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f930001b3b0 con 0x7f93100692d0 2023-10-06T05:27:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.220+0000 7f92f7fff700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92fc066f30 0x7f92fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.220+0000 7f92f7fff700 1 -- 172.21.15.45:0/87985525 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f9300083e60 con 0x7f93100692d0 2023-10-06T05:27:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.220+0000 7f930f7fe700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92fc066f30 0x7f92fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.220+0000 7f930f7fe700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92fc066f30 0x7f92fc0693e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f93100696d0 tx=0x7f9304016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.224+0000 7f92f7fff700 1 -- 172.21.15.45:0/87985525 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f931015a730 con 0x7f93100692d0 2023-10-06T05:27:56.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.427+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f931015a730 con 0x7f93100692d0 2023-10-06T05:27:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.428+0000 7f92f7fff700 1 -- 172.21.15.45:0/87985525 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f931015a730 con 0x7f93100692d0 2023-10-06T05:27:56.430 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:27:56.430 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:27:48.988520+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:27:18.521771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-06T05:27:19.508254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/2716751458":"2023-10-07T05:27:38.843219+0000","172.21.15.45:6832/2716751458":"2023-10-07T05:27:38.843219+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92fc066f30 msgr2=0x7f92fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92fc066f30 0x7f92fc0693e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f93100696d0 tx=0x7f9304016000 comp rx=0 tx=0).stop 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 msgr2=0x7f93100a6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f93100a6c30 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f930000c810 tx=0x7f9300007f30 comp rx=0 tx=0).stop 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 shutdown_connections 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f92fc066f30 0x7f92fc0693e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f931015d040 0x7f93100af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93100692d0 0x7f93100a6c30 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 --2- 172.21.15.45:0/87985525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9310159170 0x7f93100aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 >> 172.21.15.45:0/87985525 conn(0x7f931006f7c0 msgr2=0x7f93100cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 shutdown_connections 2023-10-06T05:27:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.430+0000 7f93162cc700 1 -- 172.21.15.45:0/87985525 wait complete. 2023-10-06T05:27:56.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:27:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.576+0000 7f7c28325700 1 Processor -- start 2023-10-06T05:27:56.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.576+0000 7f7c28325700 1 -- start start 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.577+0000 7f7c28325700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7c200c8ff0 0x7f7c200c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.577+0000 7f7c28325700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c200caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.577+0000 7f7c28325700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2015b000 con 0x7f7c20156af0 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.577+0000 7f7c28325700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2015b2e0 con 0x7f7c200c8ff0 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.577+0000 7f7c28325700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2015b5c0 con 0x7f7c200caad0 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.577+0000 7f7c260c1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7c200c8ff0 0x7f7c200c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c268c2700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7c20156af0 0x7f7c20156f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59624/0 (socket says 172.21.15.45:59624) 2023-10-06T05:27:56.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c268c2700 1 -- 172.21.15.45:0/1803846700 learned_addr learned my addr 172.21.15.45:0/1803846700 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c258c0700 1 --2- 172.21.15.45:0/1803846700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c200caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c260c1700 1 -- 172.21.15.45:0/1803846700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 msgr2=0x7f7c200caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c260c1700 1 --2- 172.21.15.45:0/1803846700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c200caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c260c1700 1 -- 172.21.15.45:0/1803846700 >> v1:172.21.15.45:6789/0 conn(0x7f7c20156af0 legacy=0x7f7c20156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c250bf700 1 -- 172.21.15.45:0/1803846700 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213827968 0 0) 0x7f7c2015b000 con 0x7f7c20156af0 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c260c1700 1 -- 172.21.15.45:0/1803846700 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2015b8a0 con 0x7f7c200c8ff0 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c260c1700 1 --2- 172.21.15.45:0/1803846700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c200c8ff0 0x7f7c200c93c0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0098e0 tx=0x7f7c1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdcd05ed4779a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c250bf700 1 -- 172.21.15.45:0/1803846700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c009e80 con 0x7f7c200c8ff0 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c250bf700 1 -- 172.21.15.45:0/1803846700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00b860 con 0x7f7c200c8ff0 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.578+0000 7f7c250bf700 1 -- 172.21.15.45:0/1803846700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00bbe0 con 0x7f7c200c8ff0 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 -- 172.21.15.45:0/1803846700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c200c8ff0 msgr2=0x7f7c200c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 --2- 172.21.15.45:0/1803846700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c200c8ff0 0x7f7c200c93c0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0098e0 tx=0x7f7c1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 -- 172.21.15.45:0/1803846700 shutdown_connections 2023-10-06T05:27:56.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 --2- 172.21.15.45:0/1803846700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c200c8ff0 0x7f7c200c93c0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 --2- 172.21.15.45:0/1803846700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c200caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 -- 172.21.15.45:0/1803846700 >> 172.21.15.45:0/1803846700 conn(0x7f7c200cdf50 msgr2=0x7f7c200ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:56.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 -- 172.21.15.45:0/1803846700 shutdown_connections 2023-10-06T05:27:56.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.579+0000 7f7c28325700 1 -- 172.21.15.45:0/1803846700 wait complete. 2023-10-06T05:27:56.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.580+0000 7f7c28325700 1 Processor -- start 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.580+0000 7f7c28325700 1 -- start start 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c28325700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200c8ff0 0x7f7c20069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c28325700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c2016cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c28325700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c20156af0 0x7f7c201583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c28325700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20069c70 con 0x7f7c20156af0 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c28325700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20157330 con 0x7f7c200c8ff0 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c28325700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20157610 con 0x7f7c200caad0 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c268c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c20156af0 0x7f7c201583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c268c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c20156af0 0x7f7c201583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51386/0 (socket says 172.21.15.45:51386) 2023-10-06T05:27:56.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c268c2700 1 -- 172.21.15.45:0/832996500 learned_addr learned my addr 172.21.15.45:0/832996500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c260c1700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200c8ff0 0x7f7c20069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.581+0000 7f7c258c0700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c2016cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c268c2700 1 -- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 msgr2=0x7f7c2016cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c268c2700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c2016cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c268c2700 1 -- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200c8ff0 msgr2=0x7f7c20069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c268c2700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200c8ff0 0x7f7c20069730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c268c2700 1 -- 172.21.15.45:0/832996500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c009420 con 0x7f7c20156af0 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c258c0700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c2016cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c268c2700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c20156af0 0x7f7c201583d0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cbf0 tx=0x7f7c180019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c127fc700 1 -- 172.21.15.45:0/832996500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800a9a0 con 0x7f7c20156af0 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c127fc700 1 -- 172.21.15.45:0/832996500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1800ab00 con 0x7f7c20156af0 2023-10-06T05:27:56.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c127fc700 1 -- 172.21.15.45:0/832996500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c180070f0 con 0x7f7c20156af0 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c201578f0 con 0x7f7c20156af0 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.582+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c20157f30 con 0x7f7c20156af0 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.583+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c200694f0 con 0x7f7c20156af0 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.583+0000 7f7c127fc700 1 -- 172.21.15.45:0/832996500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c18007520 con 0x7f7c20156af0 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.584+0000 7f7c127fc700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c04066f30 0x7f7c040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.584+0000 7f7c127fc700 1 -- 172.21.15.45:0/832996500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6522+0+0 (secure 0 0 0) 0x7f7c18084c30 con 0x7f7c20156af0 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.584+0000 7f7c260c1700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c04066f30 0x7f7c040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.585+0000 7f7c260c1700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c04066f30 0x7f7c040693e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0091e0 tx=0x7f7c1c017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.588+0000 7f7c127fc700 1 -- 172.21.15.45:0/832996500 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c18051bf0 con 0x7f7c20156af0 2023-10-06T05:27:56.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.837+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7c201580c0 con 0x7f7c20156af0 2023-10-06T05:27:56.839 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-10-06T05:27:56.838+0000 7fdb62d9a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:27:56.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.841+0000 7f7c127fc700 1 -- 172.21.15.45:0/832996500 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v366) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7c18055210 con 0x7f7c20156af0 2023-10-06T05:27:56.845 INFO:tasks.ceph.mds.a.smithi045.stderr: -1048> 2023-10-06T05:27:44.854+0000 7f798a167700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:27:56.846 INFO:tasks.ceph.mds.a.smithi045.stderr: -649> 2023-10-06T05:27:48.198+0000 7f7986960700 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c04066f30 msgr2=0x7f7c040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c04066f30 0x7f7c040693e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0091e0 tx=0x7f7c1c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c20156af0 msgr2=0x7f7c201583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c20156af0 0x7f7c201583d0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cbf0 tx=0x7f7c180019b0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 shutdown_connections 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7c04066f30 0x7f7c040693e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c20156af0 0x7f7c201583d0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200c8ff0 0x7f7c20069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 --2- 172.21.15.45:0/832996500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200caad0 0x7f7c2016cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.844+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 >> 172.21.15.45:0/832996500 conn(0x7f7c200cdf50 msgr2=0x7f7c2015a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:56.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.845+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 shutdown_connections 2023-10-06T05:27:56.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.845+0000 7f7c28325700 1 -- 172.21.15.45:0/832996500 wait complete. 2023-10-06T05:27:56.850 INFO:tasks.ceph.mds.a.smithi045.stderr: -376> 2023-10-06T05:27:50.024+0000 7f7986960700 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2023-10-06T05:27:56.850 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:27:56.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:27:56.869 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-10-06T05:27:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.994+0000 7fd02cedf700 1 Processor -- start 2023-10-06T05:27:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.994+0000 7fd02cedf700 1 -- start start 2023-10-06T05:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd02cedf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd028140ab0 0x7fd0280acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd02cedf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0280ad4e0 0x7fd0280c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd02cedf700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0280c9940 con 0x7fd0281406e0 2023-10-06T05:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd02cedf700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02806d230 con 0x7fd0280ad4e0 2023-10-06T05:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd02cedf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02806d510 con 0x7fd028140ab0 2023-10-06T05:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0280ad4e0 0x7fd0280c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0280ad4e0 0x7fd0280c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51418/0 (socket says 172.21.15.45:51418) 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 -- 172.21.15.45:0/2779509846 learned_addr learned my addr 172.21.15.45:0/2779509846 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd025d9b700 1 --2- 172.21.15.45:0/2779509846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd028140ab0 0x7fd0280acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 -- 172.21.15.45:0/2779509846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd028140ab0 msgr2=0x7fd0280acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 --2- 172.21.15.45:0/2779509846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd028140ab0 0x7fd0280acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 -- 172.21.15.45:0/2779509846 >> v1:172.21.15.144:6789/0 conn(0x7fd0281406e0 legacy=0x7fd02806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 -- 172.21.15.45:0/2779509846 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02806d7f0 con 0x7fd0280ad4e0 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.995+0000 7fd026d9d700 1 --2- 172.21.15.45:0/2779509846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0280ad4e0 0x7fd0280c9dd0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fd020009f20 tx=0x7fd020009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdec61f97cb7ac5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02559a700 1 -- 172.21.15.45:0/2779509846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02000e070 con 0x7fd0280ad4e0 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02559a700 1 -- 172.21.15.45:0/2779509846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02000ba80 con 0x7fd0280ad4e0 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02559a700 1 -- 172.21.15.45:0/2779509846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200092e0 con 0x7fd0280ad4e0 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 -- 172.21.15.45:0/2779509846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0280ad4e0 msgr2=0x7fd0280c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 --2- 172.21.15.45:0/2779509846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0280ad4e0 0x7fd0280c9dd0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fd020009f20 tx=0x7fd020009840 comp rx=0 tx=0).stop 2023-10-06T05:27:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 -- 172.21.15.45:0/2779509846 shutdown_connections 2023-10-06T05:27:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 --2- 172.21.15.45:0/2779509846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0280ad4e0 0x7fd0280c9dd0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 --2- 172.21.15.45:0/2779509846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd028140ab0 0x7fd0280acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 -- 172.21.15.45:0/2779509846 >> 172.21.15.45:0/2779509846 conn(0x7fd0280691c0 msgr2=0x7fd0280cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 -- 172.21.15.45:0/2779509846 shutdown_connections 2023-10-06T05:27:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.996+0000 7fd02cedf700 1 -- 172.21.15.45:0/2779509846 wait complete. 2023-10-06T05:27:57.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.997+0000 7fd02cedf700 1 Processor -- start 2023-10-06T05:27:57.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.997+0000 7fd02cedf700 1 -- start start 2023-10-06T05:27:57.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd02cedf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0280ad4e0 0x7fd0280c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd02cedf700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0281406e0 0x7fd0280ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd02cedf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd028140ab0 0x7fd0280c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd02cedf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0281350d0 con 0x7fd028140ab0 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd02cedf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280cae20 con 0x7fd0280ad4e0 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd02cedf700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280c6bb0 con 0x7fd0281406e0 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd026d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd028140ab0 0x7fd0280c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd02659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0280ad4e0 0x7fd0280c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd026d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd028140ab0 0x7fd0280c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51432/0 (socket says 172.21.15.45:51432) 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd026d9d700 1 -- 172.21.15.45:0/3197749431 learned_addr learned my addr 172.21.15.45:0/3197749431 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.998+0000 7fd025d9b700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0281406e0 0x7fd0280ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd026d9d700 1 -- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0281406e0 msgr2=0x7fd0280ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd026d9d700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0281406e0 0x7fd0280ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd026d9d700 1 -- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0280ad4e0 msgr2=0x7fd0280c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd026d9d700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0280ad4e0 0x7fd0280c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd026d9d700 1 -- 172.21.15.45:0/3197749431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd020009ae0 con 0x7fd028140ab0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd026d9d700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd028140ab0 0x7fd0280c5e60 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fd02806c4b0 tx=0x7fd020018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd0167fc700 1 -- 172.21.15.45:0/3197749431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200032f0 con 0x7fd028140ab0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd0167fc700 1 -- 172.21.15.45:0/3197749431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd020008d20 con 0x7fd028140ab0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0280c6e90 con 0x7fd028140ab0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd0167fc700 1 -- 172.21.15.45:0/3197749431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020026690 con 0x7fd028140ab0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:56.999+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0280caab0 con 0x7fd028140ab0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.000+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0280cafd0 con 0x7fd028140ab0 2023-10-06T05:27:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.000+0000 7fd0167fc700 1 -- 172.21.15.45:0/3197749431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd020003040 con 0x7fd028140ab0 2023-10-06T05:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.001+0000 7fd0167fc700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd008066f30 0x7fd0080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.001+0000 7fd0167fc700 1 -- 172.21.15.45:0/3197749431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fd020086750 con 0x7fd028140ab0 2023-10-06T05:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.001+0000 7fd02659c700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd008066f30 0x7fd0080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.002+0000 7fd02659c700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd008066f30 0x7fd0080693e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd018003a80 tx=0x7fd018007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:57.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.006+0000 7fd0167fc700 1 -- 172.21.15.45:0/3197749431 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0280cafd0 con 0x7fd028140ab0 2023-10-06T05:27:57.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.260+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd02806e890 con 0x7fd028140ab0 2023-10-06T05:27:57.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.843+0000 7fd0167fc700 1 -- 172.21.15.45:0/3197749431 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v367) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd020053640 con 0x7fd028140ab0 2023-10-06T05:27:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd008066f30 msgr2=0x7fd0080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd008066f30 0x7fd0080693e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fd018003a80 tx=0x7fd018007ec0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd028140ab0 msgr2=0x7fd0280c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd028140ab0 0x7fd0280c5e60 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fd02806c4b0 tx=0x7fd020018000 comp rx=0 tx=0).stop 2023-10-06T05:27:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 shutdown_connections 2023-10-06T05:27:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd008066f30 0x7fd0080693e0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd028140ab0 0x7fd0280c5e60 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0280ad4e0 0x7fd0280c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 --2- 172.21.15.45:0/3197749431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0281406e0 0x7fd0280ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 >> 172.21.15.45:0/3197749431 conn(0x7fd0280691c0 msgr2=0x7fd02813f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 shutdown_connections 2023-10-06T05:27:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:57.847+0000 7fd02cedf700 1 -- 172.21.15.45:0/3197749431 wait complete. 2023-10-06T05:27:57.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:27:58.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.048+0000 7f90723d8700 1 Processor -- start 2023-10-06T05:27:58.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.048+0000 7f90723d8700 1 -- start start 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.049+0000 7f90723d8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159160 0x7f906c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.049+0000 7f90723d8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f906c067620 0x7f906c067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.049+0000 7f90723d8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c0c8140 con 0x7f906c159530 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.049+0000 7f90723d8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906c067fe0 con 0x7f906c067620 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.049+0000 7f90723d8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0ca1e0 con 0x7f906c159160 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.049+0000 7f906bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159160 0x7f906c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.049+0000 7f9070975700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f906c067620 0x7f906c067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906bfff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159160 0x7f906c0cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:37066/0 (socket says 172.21.15.45:37066) 2023-10-06T05:27:58.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906bfff700 1 -- 172.21.15.45:0/4137543243 learned_addr learned my addr 172.21.15.45:0/4137543243 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906b7fe700 1 --1- 172.21.15.45:0/4137543243 >> v1:172.21.15.144:6789/0 conn(0x7f906c159530 0x7f906c0c9780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:57246/0 (socket says 172.21.15.45:57246) 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906bfff700 1 -- 172.21.15.45:0/4137543243 >> v2:172.21.15.144:3300/0 conn(0x7f906c067620 msgr2=0x7f906c067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906bfff700 1 --2- 172.21.15.45:0/4137543243 >> v2:172.21.15.144:3300/0 conn(0x7f906c067620 0x7f906c067aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906bfff700 1 -- 172.21.15.45:0/4137543243 >> v1:172.21.15.144:6789/0 conn(0x7f906c159530 legacy=0x7f906c0c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906bfff700 1 -- 172.21.15.45:0/4137543243 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0ca4c0 con 0x7f906c159160 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f9070975700 1 --2- 172.21.15.45:0/4137543243 >> v2:172.21.15.144:3300/0 conn(0x7f906c067620 0x7f906c067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906bfff700 1 --2- 172.21.15.45:0/4137543243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159160 0x7f906c0cadf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f905c0099b0 tx=0x7f905c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35c02052da7a19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906affd700 1 -- 172.21.15.45:0/4137543243 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c009ea0 con 0x7f906c159160 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906affd700 1 -- 172.21.15.45:0/4137543243 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c00b860 con 0x7f906c159160 2023-10-06T05:27:58.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.050+0000 7f906affd700 1 -- 172.21.15.45:0/4137543243 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c00bc00 con 0x7f906c159160 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 -- 172.21.15.45:0/4137543243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159160 msgr2=0x7f906c0cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 --2- 172.21.15.45:0/4137543243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159160 0x7f906c0cadf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f905c0099b0 tx=0x7f905c009250 comp rx=0 tx=0).stop 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 -- 172.21.15.45:0/4137543243 shutdown_connections 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 --2- 172.21.15.45:0/4137543243 >> v2:172.21.15.144:3300/0 conn(0x7f906c067620 0x7f906c067aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 --2- 172.21.15.45:0/4137543243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159160 0x7f906c0cadf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 -- 172.21.15.45:0/4137543243 >> 172.21.15.45:0/4137543243 conn(0x7f906c068730 msgr2=0x7f906c068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 -- 172.21.15.45:0/4137543243 shutdown_connections 2023-10-06T05:27:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.051+0000 7f90723d8700 1 -- 172.21.15.45:0/4137543243 wait complete. 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.052+0000 7f90723d8700 1 Processor -- start 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.052+0000 7f90723d8700 1 -- start start 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f90723d8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906c067620 0x7f906c1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f90723d8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f906c159160 0x7f906c0c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f90723d8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159530 0x7f906c0b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f90723d8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0b0ae0 con 0x7f906c067620 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f90723d8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c155610 con 0x7f906c159160 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f90723d8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c144e30 con 0x7f906c159530 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f906bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906c067620 0x7f906c1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f906bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906c067620 0x7f906c1448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51444/0 (socket says 172.21.15.45:51444) 2023-10-06T05:27:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f906bfff700 1 -- 172.21.15.45:0/430519724 learned_addr learned my addr 172.21.15.45:0/430519724 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f9070975700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159530 0x7f906c0b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.053+0000 7f906b7fe700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f906c159160 0x7f906c0c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f906bfff700 1 -- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159530 msgr2=0x7f906c0b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f906bfff700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159530 0x7f906c0b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f906bfff700 1 -- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f906c159160 msgr2=0x7f906c0c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f906bfff700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f906c159160 0x7f906c0c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f906bfff700 1 -- 172.21.15.45:0/430519724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064009a20 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f906b7fe700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f906c159160 0x7f906c0c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f906bfff700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906c067620 0x7f906c1448f0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f906c0ca320 tx=0x7f905c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f9053fff700 1 -- 172.21.15.45:0/430519724 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c00bbe0 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f9053fff700 1 -- 172.21.15.45:0/430519724 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c011420 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c0094f0 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f9053fff700 1 -- 172.21.15.45:0/430519724 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0117c0 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.054+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c0afed0 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.055+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c04c380 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.056+0000 7f9053fff700 1 -- 172.21.15.45:0/430519724 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f905c011bf0 con 0x7f906c067620 2023-10-06T05:27:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.056+0000 7f9053fff700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9058066f30 0x7f90580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.056+0000 7f9053fff700 1 -- 172.21.15.45:0/430519724 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f905c086130 con 0x7f906c067620 2023-10-06T05:27:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.057+0000 7f906b7fe700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9058066f30 0x7f90580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:58.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.060+0000 7f906b7fe700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9058066f30 0x7f90580693e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9060007fd0 tx=0x7f9060007e10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:58.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.060+0000 7f9053fff700 1 -- 172.21.15.45:0/430519724 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f905c0530e0 con 0x7f906c067620 2023-10-06T05:27:58.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:58.264+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f906c0c83e0 con 0x7f906c067620 2023-10-06T05:27:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.033+0000 7f9053fff700 1 -- 172.21.15.45:0/430519724 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) v1 ==== 181+0+0 (secure 0 0 0) 0x7f905c056700 con 0x7f906c067620 2023-10-06T05:27:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.037+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9058066f30 msgr2=0x7f90580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.037+0000 7f90723d8700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9058066f30 0x7f90580693e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9060007fd0 tx=0x7f9060007e10 comp rx=0 tx=0).stop 2023-10-06T05:27:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.037+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906c067620 msgr2=0x7f906c1448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.037+0000 7f90723d8700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906c067620 0x7f906c1448f0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f906c0ca320 tx=0x7f905c017000 comp rx=0 tx=0).stop 2023-10-06T05:27:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 shutdown_connections 2023-10-06T05:27:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f9058066f30 0x7f90580693e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f906c159530 0x7f906c0b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f906c067620 0x7f906c1448f0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 --2- 172.21.15.45:0/430519724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f906c159160 0x7f906c0c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 >> 172.21.15.45:0/430519724 conn(0x7f906c068730 msgr2=0x7f906c0c2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:59.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 shutdown_connections 2023-10-06T05:27:59.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.038+0000 7f90723d8700 1 -- 172.21.15.45:0/430519724 wait complete. 2023-10-06T05:27:59.041 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:27:59.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:27:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.195+0000 7efc6daad700 1 Processor -- start 2023-10-06T05:27:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.195+0000 7efc6daad700 1 -- start start 2023-10-06T05:27:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc6daad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc680cd460 0x7efc68161c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc6daad700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc680cbfc0 con 0x7efc68154c80 2023-10-06T05:27:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc6daad700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc68155050 con 0x7efc680ce740 2023-10-06T05:27:59.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc6daad700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc681585e0 con 0x7efc680cd460 2023-10-06T05:27:59.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc677fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efc68154c80 0x7efc68157640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59630/0 (socket says 172.21.15.45:59630) 2023-10-06T05:27:59.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc677fe700 1 -- 172.21.15.45:0/238981915 learned_addr learned my addr 172.21.15.45:0/238981915 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:27:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc67fff700 1 --2- 172.21.15.45:0/238981915 >> v2:172.21.15.144:3300/0 conn(0x7efc680cd460 0x7efc68161c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220756571 0 0) 0x7efc680cbfc0 con 0x7efc68154c80 2023-10-06T05:27:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc54003570 con 0x7efc68154c80 2023-10-06T05:27:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134095650 0 0) 0x7efc54003570 con 0x7efc68154c80 2023-10-06T05:27:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 >> v2:172.21.15.144:3300/0 conn(0x7efc680cd460 msgr2=0x7efc68161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 --2- 172.21.15.45:0/238981915 >> v2:172.21.15.144:3300/0 conn(0x7efc680cd460 0x7efc68161c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 >> v1:172.21.15.144:6789/0 conn(0x7efc680ce740 legacy=0x7efc680ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68157d50 con 0x7efc68154c80 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7efc58001470 con 0x7efc68154c80 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 >> v1:172.21.15.45:6789/0 conn(0x7efc68154c80 legacy=0x7efc68157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc54003a60 0x7efc54003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc54004810 0x7efc54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc5400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc681585e0 con 0x7efc540051c0 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5400c0e0 con 0x7efc54004810 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.197+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5400c2a0 con 0x7efc54003a60 2023-10-06T05:27:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc5400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc66ffd700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc54004810 0x7efc54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc677fe700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc54003a60 0x7efc54003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 -- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc54003a60 msgr2=0x7efc54003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc54003a60 0x7efc54003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 -- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc54004810 msgr2=0x7efc54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc54004810 0x7efc54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 -- 172.21.15.45:0/238981915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc54003570 con 0x7efc540051c0 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 -- 172.21.15.45:0/238981915 reap_dead start 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc67fff700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc5400ba60 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7efc6000e210 tx=0x7efc6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d72f8fc58a58e59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6000eac0 con 0x7efc540051c0 2023-10-06T05:27:59.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6000f040 con 0x7efc540051c0 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.198+0000 7efc667fc700 1 -- 172.21.15.45:0/238981915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6000ee70 con 0x7efc540051c0 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.199+0000 7efc6daad700 1 -- 172.21.15.45:0/238981915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 msgr2=0x7efc5400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.199+0000 7efc6daad700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc5400ba60 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7efc6000e210 tx=0x7efc6000b040 comp rx=0 tx=0).stop 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.199+0000 7efc6daad700 1 -- 172.21.15.45:0/238981915 shutdown_connections 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.199+0000 7efc6daad700 1 --2- 172.21.15.45:0/238981915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc5400ba60 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.199+0000 7efc6daad700 1 -- 172.21.15.45:0/238981915 >> 172.21.15.45:0/238981915 conn(0x7efc680680b0 msgr2=0x7efc680cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.199+0000 7efc6daad700 1 -- 172.21.15.45:0/238981915 shutdown_connections 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.199+0000 7efc6daad700 1 -- 172.21.15.45:0/238981915 wait complete. 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.200+0000 7efc6daad700 1 Processor -- start 2023-10-06T05:27:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.200+0000 7efc6daad700 1 -- start start 2023-10-06T05:27:59.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc6daad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc680c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc6daad700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc6814f0e0 0x7efc68151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc6daad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc680b1740 0x7efc680c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc6daad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680b0cf0 con 0x7efc540051c0 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc6daad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680b12b0 con 0x7efc680b1740 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc6daad700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc68152490 con 0x7efc6814f0e0 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc680c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc680c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51462/0 (socket says 172.21.15.45:51462) 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 -- 172.21.15.45:0/402177567 learned_addr learned my addr 172.21.15.45:0/402177567 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc66ffd700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc6814f0e0 0x7efc68151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc67fff700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc680b1740 0x7efc680c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 -- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc6814f0e0 msgr2=0x7efc68151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc6814f0e0 0x7efc68151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 -- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc680b1740 msgr2=0x7efc680c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc680b1740 0x7efc680c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc677fe700 1 -- 172.21.15.45:0/402177567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68152770 con 0x7efc540051c0 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc66ffd700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc6814f0e0 0x7efc68151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.201+0000 7efc67fff700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc680b1740 0x7efc680c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.202+0000 7efc677fe700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc680c68e0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7efc580073c0 tx=0x7efc680cbf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.202+0000 7efc5f7fe700 1 -- 172.21.15.45:0/402177567 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58018030 con 0x7efc540051c0 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.202+0000 7efc5f7fe700 1 -- 172.21.15.45:0/402177567 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc58007830 con 0x7efc540051c0 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.202+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc680b0f40 con 0x7efc540051c0 2023-10-06T05:27:59.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.202+0000 7efc5f7fe700 1 -- 172.21.15.45:0/402177567 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5800faa0 con 0x7efc540051c0 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.202+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc680c3fc0 con 0x7efc540051c0 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.203+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc681529b0 con 0x7efc540051c0 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.204+0000 7efc5f7fe700 1 -- 172.21.15.45:0/402177567 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc5801b070 con 0x7efc540051c0 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.204+0000 7efc5f7fe700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc48066f30 0x7efc480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.204+0000 7efc5f7fe700 1 -- 172.21.15.45:0/402177567 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6170+0+0 (secure 0 0 0) 0x7efc58019070 con 0x7efc540051c0 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.204+0000 7efc66ffd700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc48066f30 0x7efc480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.205+0000 7efc66ffd700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc48066f30 0x7efc480693e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efc680c6570 tx=0x7efc50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:27:59.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.208+0000 7efc5f7fe700 1 -- 172.21.15.45:0/402177567 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc58052110 con 0x7efc540051c0 2023-10-06T05:27:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:27:59.412+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efc6804c380 con 0x7efc540051c0 2023-10-06T05:28:00.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.037+0000 7efc5f7fe700 1 -- 172.21.15.45:0/402177567 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) v1 ==== 169+0+0 (secure 0 0 0) 0x7efc58055730 con 0x7efc540051c0 2023-10-06T05:28:00.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.041+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc48066f30 msgr2=0x7efc480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.041+0000 7efc6daad700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc48066f30 0x7efc480693e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efc680c6570 tx=0x7efc50009040 comp rx=0 tx=0).stop 2023-10-06T05:28:00.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.041+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 msgr2=0x7efc680c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.041+0000 7efc6daad700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc680c68e0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7efc580073c0 tx=0x7efc680cbf30 comp rx=0 tx=0).stop 2023-10-06T05:28:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 shutdown_connections 2023-10-06T05:28:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7efc48066f30 0x7efc480693e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc680b1740 0x7efc680c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc540051c0 0x7efc680c68e0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 --2- 172.21.15.45:0/402177567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc6814f0e0 0x7efc68151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 >> 172.21.15.45:0/402177567 conn(0x7efc680680b0 msgr2=0x7efc680cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 shutdown_connections 2023-10-06T05:28:00.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.042+0000 7efc6daad700 1 -- 172.21.15.45:0/402177567 wait complete. 2023-10-06T05:28:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-10-06T05:28:00.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:28:00.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.197+0000 7f69810e8700 1 Processor -- start 2023-10-06T05:28:00.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.197+0000 7f69810e8700 1 -- start start 2023-10-06T05:28:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f69810e8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f69810e8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f697c15d2f0 0x7f697c067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f69810e8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0c7b90 con 0x7f697c0ca4c0 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f69810e8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c15dd50 con 0x7f697c15d2f0 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f69810e8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0c8130 con 0x7f697c0ca0f0 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f697c15d2f0 0x7f697c067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f697c15d2f0 0x7f697c067620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51472/0 (socket says 172.21.15.45:51472) 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697a59c700 1 -- 172.21.15.45:0/829904028 learned_addr learned my addr 172.21.15.45:0/829904028 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697ad9d700 1 --2- 172.21.15.45:0/829904028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697a59c700 1 -- 172.21.15.45:0/829904028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 msgr2=0x7f697c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697a59c700 1 --2- 172.21.15.45:0/829904028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697a59c700 1 -- 172.21.15.45:0/829904028 >> v1:172.21.15.144:6789/0 conn(0x7f697c0ca4c0 legacy=0x7f697c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.198+0000 7f697a59c700 1 -- 172.21.15.45:0/829904028 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c159690 con 0x7f697c15d2f0 2023-10-06T05:28:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f697a59c700 1 --2- 172.21.15.45:0/829904028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c067620 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f696c0099b0 tx=0x7f696c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3071c75bb9e827f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f6979d9b700 1 -- 172.21.15.45:0/829904028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c009ea0 con 0x7f697c15d2f0 2023-10-06T05:28:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f6979d9b700 1 -- 172.21.15.45:0/829904028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c00b860 con 0x7f697c15d2f0 2023-10-06T05:28:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f6979d9b700 1 -- 172.21.15.45:0/829904028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00bc00 con 0x7f697c15d2f0 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 -- 172.21.15.45:0/829904028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 msgr2=0x7f697c067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 --2- 172.21.15.45:0/829904028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c067620 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f696c0099b0 tx=0x7f696c009250 comp rx=0 tx=0).stop 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 -- 172.21.15.45:0/829904028 shutdown_connections 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 --2- 172.21.15.45:0/829904028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 --2- 172.21.15.45:0/829904028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c067620 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 -- 172.21.15.45:0/829904028 >> 172.21.15.45:0/829904028 conn(0x7f697c0691c0 msgr2=0x7f697c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 -- 172.21.15.45:0/829904028 shutdown_connections 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.199+0000 7f69810e8700 1 -- 172.21.15.45:0/829904028 wait complete. 2023-10-06T05:28:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.201+0000 7f69810e8700 1 Processor -- start 2023-10-06T05:28:00.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.201+0000 7f69810e8700 1 -- start start 2023-10-06T05:28:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f69810e8700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f69810e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f697c0ca4c0 0x7f697c158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f69810e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f69810e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c159950 con 0x7f697c15d2f0 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f69810e8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c159c30 con 0x7f697c0ca4c0 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f69810e8700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c159f10 con 0x7f697c0ca0f0 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51478/0 (socket says 172.21.15.45:51478) 2023-10-06T05:28:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 -- 172.21.15.45:0/3580817275 learned_addr learned my addr 172.21.15.45:0/3580817275 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697ad9d700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697a59c700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f697c0ca4c0 0x7f697c158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 -- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 msgr2=0x7f697c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 -- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f697c0ca4c0 msgr2=0x7f697c158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f697c0ca4c0 0x7f697c158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 -- 172.21.15.45:0/3580817275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c0094f0 con 0x7f697c15d2f0 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f697b59e700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c1556e0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f697400c8e0 tx=0x7f6974001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f6962ffd700 1 -- 172.21.15.45:0/3580817275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974012630 con 0x7f697c15d2f0 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f6962ffd700 1 -- 172.21.15.45:0/3580817275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974012790 con 0x7f697c15d2f0 2023-10-06T05:28:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c15a1f0 con 0x7f697c15d2f0 2023-10-06T05:28:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.202+0000 7f6962ffd700 1 -- 172.21.15.45:0/3580817275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974012b50 con 0x7f697c15d2f0 2023-10-06T05:28:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.203+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c15af30 con 0x7f697c15d2f0 2023-10-06T05:28:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.203+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c15a710 con 0x7f697c15d2f0 2023-10-06T05:28:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.204+0000 7f6962ffd700 1 -- 172.21.15.45:0/3580817275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f697401a3a0 con 0x7f697c15d2f0 2023-10-06T05:28:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.204+0000 7f6962ffd700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.205+0000 7f6962ffd700 1 -- 172.21.15.45:0/3580817275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f69740846a0 con 0x7f697c15d2f0 2023-10-06T05:28:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.205+0000 7f697ad9d700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.205+0000 7f697ad9d700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6968066f30 0x7f69680693e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6970009db0 tx=0x7f69700091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.209+0000 7f6962ffd700 1 -- 172.21.15.45:0/3580817275 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f697c15a710 con 0x7f697c15d2f0 2023-10-06T05:28:00.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.411+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f697c15a710 con 0x7f697c15d2f0 2023-10-06T05:28:00.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.418+0000 7f6962ffd700 1 -- 172.21.15.45:0/3580817275 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f697c15a710 con 0x7f697c15d2f0 2023-10-06T05:28:00.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6968066f30 msgr2=0x7f69680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6968066f30 0x7f69680693e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6970009db0 tx=0x7f69700091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 msgr2=0x7f697c1556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c1556e0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f697400c8e0 tx=0x7f6974001710 comp rx=0 tx=0).stop 2023-10-06T05:28:00.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 shutdown_connections 2023-10-06T05:28:00.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f697c15d2f0 0x7f697c1556e0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f697c0ca0f0 0x7f697c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 --2- 172.21.15.45:0/3580817275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f697c0ca4c0 0x7f697c158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 >> 172.21.15.45:0/3580817275 conn(0x7f697c0691c0 msgr2=0x7f697c0c31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 shutdown_connections 2023-10-06T05:28:00.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.420+0000 7f69810e8700 1 -- 172.21.15.45:0/3580817275 wait complete. 2023-10-06T05:28:00.435 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2023-10-06T05:28:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.565+0000 7fbe6779b700 1 Processor -- start 2023-10-06T05:28:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.565+0000 7fbe6779b700 1 -- start start 2023-10-06T05:28:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.566+0000 7fbe6779b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600c8ff0 0x7fbe600c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.566+0000 7fbe6779b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe60157200 0x7fbe600caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.566+0000 7fbe6779b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6015b000 con 0x7fbe600caad0 2023-10-06T05:28:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.566+0000 7fbe6779b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6015b2e0 con 0x7fbe60157200 2023-10-06T05:28:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.566+0000 7fbe6779b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6015b5c0 con 0x7fbe600c8ff0 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe60157200 0x7fbe600caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe60157200 0x7fbe600caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51484/0 (socket says 172.21.15.45:51484) 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 -- 172.21.15.45:0/4201029371 learned_addr learned my addr 172.21.15.45:0/4201029371 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65537700 1 --2- 172.21.15.45:0/4201029371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600c8ff0 0x7fbe600c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe64d36700 1 --1- 172.21.15.45:0/4201029371 >> v1:172.21.15.144:6789/0 conn(0x7fbe600caad0 0x7fbe60156af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:57274/0 (socket says 172.21.15.45:57274) 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 -- 172.21.15.45:0/4201029371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600c8ff0 msgr2=0x7fbe600c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 --2- 172.21.15.45:0/4201029371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600c8ff0 0x7fbe600c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 -- 172.21.15.45:0/4201029371 >> v1:172.21.15.144:6789/0 conn(0x7fbe600caad0 legacy=0x7fbe60156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 -- 172.21.15.45:0/4201029371 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6015b8a0 con 0x7fbe60157200 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe57fff700 1 -- 172.21.15.45:0/4201029371 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844184668 0 0) 0x7fbe6015b000 con 0x7fbe600caad0 2023-10-06T05:28:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe65d38700 1 --2- 172.21.15.45:0/4201029371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe60157200 0x7fbe600caf40 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009f20 tx=0x7fbe58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230e02cd4d7c5d83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe57fff700 1 -- 172.21.15.45:0/4201029371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800e070 con 0x7fbe60157200 2023-10-06T05:28:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe57fff700 1 -- 172.21.15.45:0/4201029371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5800b9d0 con 0x7fbe60157200 2023-10-06T05:28:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.567+0000 7fbe57fff700 1 -- 172.21.15.45:0/4201029371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580092e0 con 0x7fbe60157200 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 -- 172.21.15.45:0/4201029371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe60157200 msgr2=0x7fbe600caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 --2- 172.21.15.45:0/4201029371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe60157200 0x7fbe600caf40 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009f20 tx=0x7fbe58009840 comp rx=0 tx=0).stop 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 -- 172.21.15.45:0/4201029371 shutdown_connections 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 --2- 172.21.15.45:0/4201029371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe60157200 0x7fbe600caf40 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 --2- 172.21.15.45:0/4201029371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600c8ff0 0x7fbe600c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 -- 172.21.15.45:0/4201029371 >> 172.21.15.45:0/4201029371 conn(0x7fbe600cdf50 msgr2=0x7fbe600ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 -- 172.21.15.45:0/4201029371 shutdown_connections 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.568+0000 7fbe6779b700 1 -- 172.21.15.45:0/4201029371 wait complete. 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.570+0000 7fbe6779b700 1 Processor -- start 2023-10-06T05:28:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.570+0000 7fbe6779b700 1 -- start start 2023-10-06T05:28:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe6779b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe600c8ff0 0x7fbe601579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe6779b700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600caad0 0x7fbe600aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe6779b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60157200 0x7fbe601592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe6779b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600af510 con 0x7fbe600c8ff0 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe6779b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600af7f0 con 0x7fbe60157200 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe6779b700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600afad0 con 0x7fbe600caad0 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe600c8ff0 0x7fbe601579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe600c8ff0 0x7fbe601579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51496/0 (socket says 172.21.15.45:51496) 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 -- 172.21.15.45:0/2964408334 learned_addr learned my addr 172.21.15.45:0/2964408334 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe64d36700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600caad0 0x7fbe600aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65d38700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60157200 0x7fbe601592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 -- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600caad0 msgr2=0x7fbe600aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600caad0 0x7fbe600aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 -- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60157200 msgr2=0x7fbe601592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60157200 0x7fbe601592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65537700 1 -- 172.21.15.45:0/2964408334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58009ae0 con 0x7fbe600c8ff0 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe65d38700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60157200 0x7fbe601592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.571+0000 7fbe64d36700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600caad0 0x7fbe600aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.572+0000 7fbe65537700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe600c8ff0 0x7fbe601579d0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ec80 tx=0x7fbe500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.572+0000 7fbe557fa700 1 -- 172.21.15.45:0/2964408334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000c9d0 con 0x7fbe600c8ff0 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.572+0000 7fbe557fa700 1 -- 172.21.15.45:0/2964408334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5000cb30 con 0x7fbe600c8ff0 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.572+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe600b0cf0 con 0x7fbe600c8ff0 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.572+0000 7fbe557fa700 1 -- 172.21.15.45:0/2964408334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50013470 con 0x7fbe600c8ff0 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.572+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe600af200 con 0x7fbe600c8ff0 2023-10-06T05:28:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.573+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe600b1340 con 0x7fbe600c8ff0 2023-10-06T05:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.574+0000 7fbe557fa700 1 -- 172.21.15.45:0/2964408334 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe50013820 con 0x7fbe600c8ff0 2023-10-06T05:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.574+0000 7fbe557fa700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe44066f30 0x7fbe440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.574+0000 7fbe557fa700 1 -- 172.21.15.45:0/2964408334 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fbe500848c0 con 0x7fbe600c8ff0 2023-10-06T05:28:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.574+0000 7fbe64d36700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe44066f30 0x7fbe440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:00.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.575+0000 7fbe64d36700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe44066f30 0x7fbe440693e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbe60157f10 tx=0x7fbe5c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.578+0000 7fbe557fa700 1 -- 172.21.15.45:0/2964408334 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe50051b30 con 0x7fbe600c8ff0 2023-10-06T05:28:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:00.861+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) v1 -- 0x7fbe601507c0 con 0x7fbe600c8ff0 2023-10-06T05:28:01.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.042+0000 7fbe557fa700 1 -- 172.21.15.45:0/2964408334 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v654) v1 ==== 135+0+0 (secure 0 0 0) 0x7fbe50055150 con 0x7fbe600c8ff0 2023-10-06T05:28:01.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe44066f30 msgr2=0x7fbe440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:01.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe44066f30 0x7fbe440693e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbe60157f10 tx=0x7fbe5c009040 comp rx=0 tx=0).stop 2023-10-06T05:28:01.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe600c8ff0 msgr2=0x7fbe601579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:01.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe600c8ff0 0x7fbe601579d0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ec80 tx=0x7fbe500091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 shutdown_connections 2023-10-06T05:28:01.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbe44066f30 0x7fbe440693e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60157200 0x7fbe601592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe600c8ff0 0x7fbe601579d0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 --2- 172.21.15.45:0/2964408334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe600caad0 0x7fbe600aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 >> 172.21.15.45:0/2964408334 conn(0x7fbe600cdf50 msgr2=0x7fbe6006d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:01.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 shutdown_connections 2023-10-06T05:28:01.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.045+0000 7fbe6779b700 1 -- 172.21.15.45:0/2964408334 wait complete. 2023-10-06T05:28:01.062 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ok 2023-10-06T05:28:01.062 INFO:tasks.cephfs_test_runner: 2023-10-06T05:28:01.063 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T05:28:01.063 INFO:tasks.cephfs_test_runner:Ran 6 tests in 729.692s 2023-10-06T05:28:01.063 INFO:tasks.cephfs_test_runner: 2023-10-06T05:28:01.063 INFO:tasks.cephfs_test_runner:OK 2023-10-06T05:28:01.063 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T05:28:01.072 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-06T05:28:01.080 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-06T05:28:01.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.081 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-06T05:28:01.106 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:28:01.106 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:28:01.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.106 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:28:01.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.187 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:28:01.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.275 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-10-06T05:28:01.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.355 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-10-06T05:28:01.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:28:01.379 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:28:01.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:28:01.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:28:01.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:28:01.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-10-06T05:28:01.602 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T05:28:01.611 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T05:28:01.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:28:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.744+0000 7fd0ba730700 1 Processor -- start 2023-10-06T05:28:01.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.744+0000 7fd0ba730700 1 -- start start 2023-10-06T05:28:01.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0ba730700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd0b40b11a0 0x7fd0b406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0ba730700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b415a240 0x7fd0b4158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0ba730700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b41583b0 con 0x7fd0b40b1570 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0ba730700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b4139460 con 0x7fd0b415a240 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0ba730700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b41577a0 con 0x7fd0b40b11a0 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0b8ccd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b415a240 0x7fd0b4158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0b37fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd0b40b1570 0x7fd0b4159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59644/0 (socket says 172.21.15.45:59644) 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0b37fe700 1 -- 172.21.15.45:0/2630764289 learned_addr learned my addr 172.21.15.45:0/2630764289 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.745+0000 7fd0b3fff700 1 --2- 172.21.15.45:0/2630764289 >> v2:172.21.15.144:3300/0 conn(0x7fd0b40b11a0 0x7fd0b406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b8ccd700 1 -- 172.21.15.45:0/2630764289 >> v2:172.21.15.144:3300/0 conn(0x7fd0b40b11a0 msgr2=0x7fd0b406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b8ccd700 1 --2- 172.21.15.45:0/2630764289 >> v2:172.21.15.144:3300/0 conn(0x7fd0b40b11a0 0x7fd0b406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b8ccd700 1 -- 172.21.15.45:0/2630764289 >> v1:172.21.15.45:6789/0 conn(0x7fd0b40b1570 legacy=0x7fd0b4159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b8ccd700 1 -- 172.21.15.45:0/2630764289 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4157a80 con 0x7fd0b415a240 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b2ffd700 1 -- 172.21.15.45:0/2630764289 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616733788 0 0) 0x7fd0b41583b0 con 0x7fd0b40b1570 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b8ccd700 1 --2- 172.21.15.45:0/2630764289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b415a240 0x7fd0b4158840 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fd0b406db30 tx=0x7fd0ac00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60fab2725203cce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b2ffd700 1 -- 172.21.15.45:0/2630764289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac010070 con 0x7fd0b415a240 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b2ffd700 1 -- 172.21.15.45:0/2630764289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac00b2e0 con 0x7fd0b415a240 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.746+0000 7fd0b2ffd700 1 -- 172.21.15.45:0/2630764289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00dd50 con 0x7fd0b415a240 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 -- 172.21.15.45:0/2630764289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b415a240 msgr2=0x7fd0b4158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 --2- 172.21.15.45:0/2630764289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b415a240 0x7fd0b4158840 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fd0b406db30 tx=0x7fd0ac00b990 comp rx=0 tx=0).stop 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 -- 172.21.15.45:0/2630764289 shutdown_connections 2023-10-06T05:28:01.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 --2- 172.21.15.45:0/2630764289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b415a240 0x7fd0b4158840 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 --2- 172.21.15.45:0/2630764289 >> v2:172.21.15.144:3300/0 conn(0x7fd0b40b11a0 0x7fd0b406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 -- 172.21.15.45:0/2630764289 >> 172.21.15.45:0/2630764289 conn(0x7fd0b4067620 msgr2=0x7fd0b4067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 -- 172.21.15.45:0/2630764289 shutdown_connections 2023-10-06T05:28:01.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.747+0000 7fd0ba730700 1 -- 172.21.15.45:0/2630764289 wait complete. 2023-10-06T05:28:01.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.748+0000 7fd0ba730700 1 Processor -- start 2023-10-06T05:28:01.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.748+0000 7fd0ba730700 1 -- start start 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0ba730700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b40b11a0 0x7fd0b414c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0ba730700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b40b1570 0x7fd0b40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0ba730700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b415a240 0x7fd0b40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0ba730700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4153600 con 0x7fd0b40b11a0 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0ba730700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4170f50 con 0x7fd0b415a240 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0ba730700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4171230 con 0x7fd0b40b1570 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0b3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b40b11a0 0x7fd0b414c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0b3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b40b11a0 0x7fd0b414c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51506/0 (socket says 172.21.15.45:51506) 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0b3fff700 1 -- 172.21.15.45:0/4122728913 learned_addr learned my addr 172.21.15.45:0/4122728913 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:01.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0b8ccd700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b415a240 0x7fd0b40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.749+0000 7fd0b37fe700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b40b1570 0x7fd0b40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0b3fff700 1 -- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b40b1570 msgr2=0x7fd0b40b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0b3fff700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b40b1570 0x7fd0b40b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0b3fff700 1 -- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b415a240 msgr2=0x7fd0b40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0b3fff700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b415a240 0x7fd0b40aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0b3fff700 1 -- 172.21.15.45:0/4122728913 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac00bc30 con 0x7fd0b40b11a0 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0b3fff700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b40b11a0 0x7fd0b414c6d0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400ec80 tx=0x7fd0a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd09bfff700 1 -- 172.21.15.45:0/4122728913 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4009690 con 0x7fd0b40b11a0 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd09bfff700 1 -- 172.21.15.45:0/4122728913 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a400c850 con 0x7fd0b40b11a0 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b4171510 con 0x7fd0b40b11a0 2023-10-06T05:28:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd09bfff700 1 -- 172.21.15.45:0/4122728913 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400cc10 con 0x7fd0b40b11a0 2023-10-06T05:28:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.750+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b416d470 con 0x7fd0b40b11a0 2023-10-06T05:28:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.751+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b416d6c0 con 0x7fd0b40b11a0 2023-10-06T05:28:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.751+0000 7fd09bfff700 1 -- 172.21.15.45:0/4122728913 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0a4013470 con 0x7fd0b40b11a0 2023-10-06T05:28:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.752+0000 7fd09bfff700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.752+0000 7fd09bfff700 1 -- 172.21.15.45:0/4122728913 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fd0a40846d0 con 0x7fd0b40b11a0 2023-10-06T05:28:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.752+0000 7fd0b37fe700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.753+0000 7fd0b37fe700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40cc6d0 tx=0x7fd0a8007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.756+0000 7fd09bfff700 1 -- 172.21.15.45:0/4122728913 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0a40553d0 con 0x7fd0b40b11a0 2023-10-06T05:28:01.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.959+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0b4157d60 con 0x7fd0b40b11a0 2023-10-06T05:28:02.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.960+0000 7fd09bfff700 1 -- 172.21.15.45:0/4122728913 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+14220 (secure 0 0 0) 0x7fd0a4054f60 con 0x7fd0b40b11a0 2023-10-06T05:28:02.102 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:02.102 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":149,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:28:00.031787+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/4232324777":"2023-10-07T05:27:56.838619+0000","172.21.15.45:6832/4232324777":"2023-10-07T05:27:56.838619+0000","172.21.15.45:6833/2716751458":"2023-10-07T05:27:38.843219+0000","172.21.15.45:6832/2716751458":"2023-10-07T05:27:38.843219+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:28:02.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 msgr2=0x7fd0a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40cc6d0 tx=0x7fd0a8007d30 comp rx=0 tx=0).stop 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b40b11a0 msgr2=0x7fd0b414c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b40b11a0 0x7fd0b414c6d0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400ec80 tx=0x7fd0a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 shutdown_connections 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0b415a240 0x7fd0b40aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b40b11a0 0x7fd0b414c6d0 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 --2- 172.21.15.45:0/4122728913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0b40b1570 0x7fd0b40b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.962+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 >> 172.21.15.45:0/4122728913 conn(0x7fd0b4067620 msgr2=0x7fd0b4138160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.963+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 shutdown_connections 2023-10-06T05:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:01.963+0000 7fd0ba730700 1 -- 172.21.15.45:0/4122728913 wait complete. 2023-10-06T05:28:02.105 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:28:02.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.242+0000 7f93d5172700 1 Processor -- start 2023-10-06T05:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.242+0000 7f93d5172700 1 -- start start 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93d5172700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93d006e1b0 0x7f93d00b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93d5172700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d006e580 0x7f93d00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93d5172700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93d0069f70 0x7f93d01550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93d5172700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d00c88a0 con 0x7f93d006e1b0 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93d5172700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d00aeb00 con 0x7f93d0069f70 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93d5172700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d00aede0 con 0x7f93d006e580 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93ced9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93d006e1b0 0x7f93d00b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93ced9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93d006e1b0 0x7f93d00b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51508/0 (socket says 172.21.15.45:51508) 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93ced9d700 1 -- 172.21.15.45:0/4054997109 learned_addr learned my addr 172.21.15.45:0/4054997109 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93cf59e700 1 --2- 172.21.15.45:0/4054997109 >> v2:172.21.15.144:3300/0 conn(0x7f93d0069f70 0x7f93d01550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93ce59c700 1 --2- 172.21.15.45:0/4054997109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d006e580 0x7f93d00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93ced9d700 1 -- 172.21.15.45:0/4054997109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d006e580 msgr2=0x7f93d00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.243+0000 7f93ced9d700 1 --2- 172.21.15.45:0/4054997109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d006e580 0x7f93d00c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93ced9d700 1 -- 172.21.15.45:0/4054997109 >> v2:172.21.15.144:3300/0 conn(0x7f93d0069f70 msgr2=0x7f93d01550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93ced9d700 1 --2- 172.21.15.45:0/4054997109 >> v2:172.21.15.144:3300/0 conn(0x7f93d0069f70 0x7f93d01550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93ced9d700 1 -- 172.21.15.45:0/4054997109 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d00c9340 con 0x7f93d006e1b0 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93ced9d700 1 --2- 172.21.15.45:0/4054997109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e1b0 0x7f93d00b0bd0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f93d00aed50 tx=0x7f93c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dabbef61673d51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93cdd9b700 1 -- 172.21.15.45:0/4054997109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c400b640 con 0x7f93d006e1b0 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93cdd9b700 1 -- 172.21.15.45:0/4054997109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c400b7a0 con 0x7f93d006e1b0 2023-10-06T05:28:02.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93cdd9b700 1 -- 172.21.15.45:0/4054997109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c400bb60 con 0x7f93d006e1b0 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93d5172700 1 -- 172.21.15.45:0/4054997109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e1b0 msgr2=0x7f93d00b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93d5172700 1 --2- 172.21.15.45:0/4054997109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e1b0 0x7f93d00b0bd0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f93d00aed50 tx=0x7f93c4009250 comp rx=0 tx=0).stop 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93d5172700 1 -- 172.21.15.45:0/4054997109 shutdown_connections 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93d5172700 1 --2- 172.21.15.45:0/4054997109 >> v2:172.21.15.144:3300/0 conn(0x7f93d0069f70 0x7f93d01550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93d5172700 1 --2- 172.21.15.45:0/4054997109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e1b0 0x7f93d00b0bd0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93d5172700 1 --2- 172.21.15.45:0/4054997109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d006e580 0x7f93d00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.244+0000 7f93d5172700 1 -- 172.21.15.45:0/4054997109 >> 172.21.15.45:0/4054997109 conn(0x7f93d0067620 msgr2=0x7f93d0067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.245+0000 7f93d5172700 1 -- 172.21.15.45:0/4054997109 shutdown_connections 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.245+0000 7f93d5172700 1 -- 172.21.15.45:0/4054997109 wait complete. 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.246+0000 7f93d5172700 1 Processor -- start 2023-10-06T05:28:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.246+0000 7f93d5172700 1 -- start start 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93d5172700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0069f70 0x7f93d0135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93d5172700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d006e1b0 0x7f93d0148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93d5172700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e580 0x7f93d00c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93d5172700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0149060 con 0x7f93d006e580 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93d5172700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0149ea0 con 0x7f93d006e1b0 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93d5172700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d00c7990 con 0x7f93d0069f70 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e580 0x7f93d00c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e580 0x7f93d00c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51514/0 (socket says 172.21.15.45:51514) 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 -- 172.21.15.45:0/4249031351 learned_addr learned my addr 172.21.15.45:0/4249031351 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93ce59c700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d006e1b0 0x7f93d0148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93ced9d700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0069f70 0x7f93d0135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 -- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0069f70 msgr2=0x7f93d0135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0069f70 0x7f93d0135400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 -- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d006e1b0 msgr2=0x7f93d0148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d006e1b0 0x7f93d0148b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 -- 172.21.15.45:0/4249031351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40094f0 con 0x7f93d006e580 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93ced9d700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0069f70 0x7f93d0135400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.247+0000 7f93cf59e700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e580 0x7f93d00c7450 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009e90 tx=0x7f93c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.248+0000 7f93b6ffd700 1 -- 172.21.15.45:0/4249031351 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c800cc20 con 0x7f93d006e580 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.248+0000 7f93b6ffd700 1 -- 172.21.15.45:0/4249031351 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c800f040 con 0x7f93d006e580 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.248+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d016a560 con 0x7f93d006e580 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.248+0000 7f93b6ffd700 1 -- 172.21.15.45:0/4249031351 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8014480 con 0x7f93d006e580 2023-10-06T05:28:02.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.248+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d01495f0 con 0x7f93d006e580 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.249+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d014a030 con 0x7f93d006e580 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.249+0000 7f93b6ffd700 1 -- 172.21.15.45:0/4249031351 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93c8014830 con 0x7f93d006e580 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.250+0000 7f93b6ffd700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93bc066f30 0x7f93bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.250+0000 7f93b6ffd700 1 -- 172.21.15.45:0/4249031351 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f93c8085330 con 0x7f93d006e580 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.250+0000 7f93ced9d700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93bc066f30 0x7f93bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.251+0000 7f93ced9d700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93bc066f30 0x7f93bc0693e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f93d00aed50 tx=0x7f93c4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.254+0000 7f93b6ffd700 1 -- 172.21.15.45:0/4249031351 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93c8055f90 con 0x7f93d006e580 2023-10-06T05:28:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.448+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93d006a7a0 con 0x7f93bc066f30 2023-10-06T05:28:02.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.452+0000 7f93b6ffd700 1 -- 172.21.15.45:0/4249031351 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73310 (secure 0 0 0) 0x7f93d006a7a0 con 0x7f93bc066f30 2023-10-06T05:28:02.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93bc066f30 msgr2=0x7f93bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93bc066f30 0x7f93bc0693e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f93d00aed50 tx=0x7f93c4017000 comp rx=0 tx=0).stop 2023-10-06T05:28:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e580 msgr2=0x7f93d00c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e580 0x7f93d00c7450 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009e90 tx=0x7f93c800b040 comp rx=0 tx=0).stop 2023-10-06T05:28:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 shutdown_connections 2023-10-06T05:28:02.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f93bc066f30 0x7f93bc0693e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d006e580 0x7f93d00c7450 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0069f70 0x7f93d0135400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 --2- 172.21.15.45:0/4249031351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d006e1b0 0x7f93d0148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 >> 172.21.15.45:0/4249031351 conn(0x7f93d0067620 msgr2=0x7f93d00b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 shutdown_connections 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.454+0000 7f93d5172700 1 -- 172.21.15.45:0/4249031351 wait complete. 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:28:02.556 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-10-06T05:28:00.643286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2455956,"kb_used_data":16212,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752518764,"statfs":{"total":773094113280,"available":770579214336,"internally_reserved":0,"allocated":16601088,"data_stored":9232051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27944,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-94,"num_write_kb":-60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001663"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":128,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T05:27:47.995658+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:27:47.995658+0000","last_peered":"2023-10-06T05:27:47.995658+0000","last_clean":"2023-10-06T05:27:47.995658+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:27:47.995658+0000","last_undegraded":"2023-10-06T05:27:47.995658+0000","last_fullsized":"2023-10-06T05:27:47.995658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":127,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-06T05:27:49.165100+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:27:49.165100+0000","last_peered":"2023-10-06T05:27:49.165100+0000","last_clean":"2023-10-06T05:27:49.165100+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:27:49.165100+0000","last_undegraded":"2023-10-06T05:27:49.165100+0000","last_fullsized":"2023-10-06T05:27:49.165100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T05:27:47.998490+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:27:47.998490+0000","last_peered":"2023-10-06T05:27:47.998490+0000","last_clean":"2023-10-06T05:27:47.998490+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:27:47.998490+0000","last_undegraded":"2023-10-06T05:27:47.998490+0000","last_fullsized":"2023-10-06T05:27:47.998490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":127,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-06T05:27:49.165249+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:27:49.165249+0000","last_peered":"2023-10-06T05:27:49.165249+0000","last_clean":"2023-10-06T05:27:49.165249+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:27:49.165249+0000","last_undegraded":"2023-10-06T05:27:49.165249+0000","last_fullsized":"2023-10-06T05:27:49.165249+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":277,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-06T05:27:49.722918+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:27:49.722918+0000","last_peered":"2023-10-06T05:27:49.722918+0000","last_clean":"2023-10-06T05:27:49.722918+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:27:49.722918+0000","last_undegraded":"2023-10-06T05:27:49.722918+0000","last_fullsized":"2023-10-06T05:27:49.722918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":127,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-06T05:27:49.722821+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:27:49.722821+0000","last_peered":"2023-10-06T05:27:49.722821+0000","last_clean":"2023-10-06T05:27:49.722821+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:27:49.722821+0000","last_undegraded":"2023-10-06T05:27:49.722821+0000","last_fullsized":"2023-10-06T05:27:49.722821+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":127,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-06T05:27:49.723266+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:27:49.723266+0000","last_peered":"2023-10-06T05:27:49.723266+0000","last_clean":"2023-10-06T05:27:49.723266+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:27:49.723266+0000","last_undegraded":"2023-10-06T05:27:49.723266+0000","last_fullsized":"2023-10-06T05:27:49.723266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T05:27:47.996068+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:27:47.996068+0000","last_peered":"2023-10-06T05:27:47.996068+0000","last_clean":"2023-10-06T05:27:47.996068+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:27:47.996068+0000","last_undegraded":"2023-10-06T05:27:47.996068+0000","last_fullsized":"2023-10-06T05:27:47.996068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T05:27:48.002257+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:27:48.002257+0000","last_peered":"2023-10-06T05:27:48.002257+0000","last_clean":"2023-10-06T05:27:48.002257+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:27:48.002257+0000","last_undegraded":"2023-10-06T05:27:48.002257+0000","last_fullsized":"2023-10-06T05:27:48.002257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307944,"kb_used_data":2464,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063896,"statfs":{"total":96636764160,"available":96321429504,"internally_reserved":0,"allocated":2523136,"data_stored":1595734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.878},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":1.476},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.44}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":1.5129999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":2.0190000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0059999999999998},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58699999999999997,"5min":0.76800000000000002,"15min":0.90300000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.504,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.76800000000000002,"15min":0.93500000000000005},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68999999999999995,"5min":0.81699999999999995,"15min":0.90000000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.84899999999999998},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.91000000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.314},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.92900000000000005},"last":0.72099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":1912832,"data_stored":997946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":4.6900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":0.86799999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.495},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.76400000000000001,"15min":5.6239999999999997},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.73899999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":11.595000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":5.5800000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.93600000000000005},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":23.146000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.73699999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":9.7539999999999996},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.61099999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":9.9969999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.64600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.751,"5min":0.752,"15min":9.5850000000000009},"last":0.42399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.496,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.6439999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":9.8420000000000005},"last":0.59399999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":9.718},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":9.8930000000000007},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307356,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064484,"statfs":{"total":96636764160,"available":96322031616,"internally_reserved":0,"allocated":1921024,"data_stored":998651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.64400000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.373,"5min":0.40699999999999997,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.73699999999999999},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.3999999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58199999999999996,"5min":0.67100000000000004,"15min":1.673},"last":0.35399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.503},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":4.0490000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77700000000000002},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.92600000000000005,"15min":0.94699999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.95199999999999996},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.751},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":1.0369999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.44},"max":{"1min":0.871,"5min":0.95599999999999996,"15min":1.167},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.94499999999999995,"15min":1.026},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92100000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.72899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307980,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063860,"statfs":{"total":96636764160,"available":96321392640,"internally_reserved":0,"allocated":2560000,"data_stored":1621765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.30599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56799999999999995,"5min":0.82099999999999995,"15min":1.038},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.30299999999999999,"5min":0.27400000000000002,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":1.0609999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.252},"max":{"1min":0.60799999999999998,"5min":0.67800000000000005,"15min":3.855},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1919999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.45200000000000001},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":7.5880000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":7.7670000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.4039999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.97199999999999998,"15min":2.0870000000000002},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":2.4940000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":5.3600000000000003},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.01,"15min":1.1759999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.45900000000000002,"15min":0.45400000000000001},"max":{"1min":0.874,"5min":0.99199999999999999,"15min":1.0940000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.97899999999999998,"15min":1.0169999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.47499999999999998,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":1.2150000000000001},"last":0.64900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":1908736,"data_stored":997876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.39500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":2.0720000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.899},"last":0.32300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":2.097},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.48699999999999999},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":1.8759999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":1.6719999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.48299999999999998},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":0.70499999999999996,"15min":2.121},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":1.012,"15min":1.7789999999999999},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.018,"15min":1.643},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66300000000000003,"15min":0.68999999999999995},"min":{"1min":0.495,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":1.2130000000000001,"15min":2.0129999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.69499999999999995},"min":{"1min":0.51000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.851},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81999999999999995,"5min":0.88300000000000001,"15min":1.571},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.0800000000000001,"15min":2.0449999999999999},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0569999999999999,"15min":1.8180000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.44500000000000001},"max":{"1min":1.0029999999999999,"5min":1.131,"15min":1.613},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307380,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064460,"statfs":{"total":96636764160,"available":96322007040,"internally_reserved":0,"allocated":1945600,"data_stored":1023116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26900000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64500000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40799999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.22900000000000001},"max":{"1min":0.63900000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.33200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40999999999999998,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61399999999999999,"5min":1.75,"15min":1.75},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60099999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72699999999999998,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.42499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.0469999999999999,"15min":1.143},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":1.855,"15min":1.855},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.745}]},{"osd":6,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.43099999999999999},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":1.885,"15min":1.885},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":1.236},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":1.1950000000000001},"last":0.77300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311440,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060400,"statfs":{"total":96636764160,"available":96317849600,"internally_reserved":0,"allocated":1908736,"data_stored":997876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":12.983000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.33400000000000002,"15min":0.33100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":1.7430000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.8080000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":12.911},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.73099999999999998,"5min":0.84299999999999997,"15min":5.0890000000000004},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":12.999000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":13.289999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.95099999999999996,"15min":1.5760000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.88200000000000001,"15min":1.552},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.54600000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.85699999999999998,"15min":12.949},"last":0.497},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.92900000000000005,"15min":5.2549999999999999},"last":0.621}]},{"osd":7,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":2.871},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.91500000000000004,"15min":12.859},"last":0.68300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":1921024,"data_stored":999087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.79400000000000004,"15min":0.82299999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.89400000000000002},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.82199999999999995,"15min":0.84599999999999997},"last":0.505},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.81599999999999995},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.83299999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.876},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.67600000000000005,"5min":0.85999999999999999,"15min":0.875},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.49199999999999999,"5min":0.432,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.499,"15min":0.48399999999999999},"min":{"1min":0.36499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.92800000000000005},"last":0.443}]},{"osd":6,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.317,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.86699999999999999,"15min":0.876},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84899999999999998,"15min":0.89300000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.875,"15min":0.90900000000000003},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:28:02.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:28:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.723+0000 7fb8c97ea700 1 Processor -- start 2023-10-06T05:28:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.723+0000 7fb8c97ea700 1 -- start start 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c97ea700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c40c8ff0 0x7fb8c40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c97ea700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8c40caad0 0x7fb8c40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c97ea700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb8c4156af0 0x7fb8c4156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c97ea700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c415b000 con 0x7fb8c40caad0 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c97ea700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c415b2e0 con 0x7fb8c4156af0 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c97ea700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c415b5c0 con 0x7fb8c40c8ff0 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c27fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8c40caad0 0x7fb8c40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c27fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8c40caad0 0x7fb8c40caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51520/0 (socket says 172.21.15.45:51520) 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.724+0000 7fb8c27fc700 1 -- 172.21.15.45:0/2432122123 learned_addr learned my addr 172.21.15.45:0/2432122123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c37fe700 1 --2- 172.21.15.45:0/2432122123 >> v2:172.21.15.144:3300/0 conn(0x7fb8c4156af0 0x7fb8c4156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c2ffd700 1 --2- 172.21.15.45:0/2432122123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c40c8ff0 0x7fb8c40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c27fc700 1 -- 172.21.15.45:0/2432122123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c40c8ff0 msgr2=0x7fb8c40c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c27fc700 1 --2- 172.21.15.45:0/2432122123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c40c8ff0 0x7fb8c40c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c27fc700 1 -- 172.21.15.45:0/2432122123 >> v2:172.21.15.144:3300/0 conn(0x7fb8c4156af0 msgr2=0x7fb8c4156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c27fc700 1 --2- 172.21.15.45:0/2432122123 >> v2:172.21.15.144:3300/0 conn(0x7fb8c4156af0 0x7fb8c4156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c27fc700 1 -- 172.21.15.45:0/2432122123 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c415b8a0 con 0x7fb8c40caad0 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c2ffd700 1 --2- 172.21.15.45:0/2432122123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c40c8ff0 0x7fb8c40c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c27fc700 1 --2- 172.21.15.45:0/2432122123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40caf40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40b0a60 tx=0x7fb8ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540c5f6ca06977ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c1ffb700 1 -- 172.21.15.45:0/2432122123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac009e70 con 0x7fb8c40caad0 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c1ffb700 1 -- 172.21.15.45:0/2432122123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac00b860 con 0x7fb8c40caad0 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.725+0000 7fb8c1ffb700 1 -- 172.21.15.45:0/2432122123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00bc00 con 0x7fb8c40caad0 2023-10-06T05:28:02.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 -- 172.21.15.45:0/2432122123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 msgr2=0x7fb8c40caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/2432122123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40caf40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40b0a60 tx=0x7fb8ac009250 comp rx=0 tx=0).stop 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 -- 172.21.15.45:0/2432122123 shutdown_connections 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/2432122123 >> v2:172.21.15.144:3300/0 conn(0x7fb8c4156af0 0x7fb8c4156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/2432122123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c40c8ff0 0x7fb8c40c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/2432122123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40caf40 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 -- 172.21.15.45:0/2432122123 >> 172.21.15.45:0/2432122123 conn(0x7fb8c40cdf50 msgr2=0x7fb8c40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 -- 172.21.15.45:0/2432122123 shutdown_connections 2023-10-06T05:28:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.726+0000 7fb8c97ea700 1 -- 172.21.15.45:0/2432122123 wait complete. 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.727+0000 7fb8c97ea700 1 Processor -- start 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.727+0000 7fb8c97ea700 1 -- start start 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c97ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8c40c8ff0 0x7fb8c41578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c97ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c97ea700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c4156af0 0x7fb8c4150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c97ea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b2ee0 con 0x7fb8c40caad0 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c97ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4157e00 con 0x7fb8c40c8ff0 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c97ea700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b2920 con 0x7fb8c4156af0 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51536/0 (socket says 172.21.15.45:51536) 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c27fc700 1 -- 172.21.15.45:0/271706478 learned_addr learned my addr 172.21.15.45:0/271706478 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c37fe700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c4156af0 0x7fb8c4150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.728+0000 7fb8c2ffd700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8c40c8ff0 0x7fb8c41578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c27fc700 1 -- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c4156af0 msgr2=0x7fb8c4150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c27fc700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c4156af0 0x7fb8c4150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c27fc700 1 -- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8c40c8ff0 msgr2=0x7fb8c41578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c27fc700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8c40c8ff0 0x7fb8c41578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c27fc700 1 -- 172.21.15.45:0/271706478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0094f0 con 0x7fb8c40caad0 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c27fc700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40cc580 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009290 tx=0x7fb8ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8baffd700 1 -- 172.21.15.45:0/271706478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00bbe0 con 0x7fb8c40caad0 2023-10-06T05:28:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8baffd700 1 -- 172.21.15.45:0/271706478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac01f070 con 0x7fb8c40caad0 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8baffd700 1 -- 172.21.15.45:0/271706478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0266b0 con 0x7fb8c40caad0 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c40b2c00 con 0x7fb8c40caad0 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.729+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c40ad5a0 con 0x7fb8c40caad0 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.730+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c40ad120 con 0x7fb8c40caad0 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.730+0000 7fb8baffd700 1 -- 172.21.15.45:0/271706478 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8ac026a60 con 0x7fb8c40caad0 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.731+0000 7fb8baffd700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8a4066f30 0x7fb8a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.731+0000 7fb8baffd700 1 -- 172.21.15.45:0/271706478 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fb8ac021070 con 0x7fb8c40caad0 2023-10-06T05:28:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.731+0000 7fb8c2ffd700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8a4066f30 0x7fb8a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.732+0000 7fb8c2ffd700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8a4066f30 0x7fb8a40693e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006f00 tx=0x7fb8b4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.735+0000 7fb8baffd700 1 -- 172.21.15.45:0/271706478 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8c40ad120 con 0x7fb8c40caad0 2023-10-06T05:28:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.936+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8c40ad120 con 0x7fb8a4066f30 2023-10-06T05:28:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.939+0000 7fb8baffd700 1 -- 172.21.15.45:0/271706478 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73411 (secure 0 0 0) 0x7fb8c40ad120 con 0x7fb8a4066f30 2023-10-06T05:28:02.943 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8a4066f30 msgr2=0x7fb8a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8a4066f30 0x7fb8a40693e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006f00 tx=0x7fb8b4008040 comp rx=0 tx=0).stop 2023-10-06T05:28:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 msgr2=0x7fb8c40cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40cc580 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009290 tx=0x7fb8ac017000 comp rx=0 tx=0).stop 2023-10-06T05:28:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 shutdown_connections 2023-10-06T05:28:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb8a4066f30 0x7fb8a40693e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8c4156af0 0x7fb8c4150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8c40c8ff0 0x7fb8c41578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 --2- 172.21.15.45:0/271706478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c40caad0 0x7fb8c40cc580 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 >> 172.21.15.45:0/271706478 conn(0x7fb8c40cdf50 msgr2=0x7fb8c41519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 shutdown_connections 2023-10-06T05:28:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:02.941+0000 7fb8c97ea700 1 -- 172.21.15.45:0/271706478 wait complete. 2023-10-06T05:28:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:28:02.956 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":537,"stamp":"2023-10-06T05:28:02.643782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2456176,"kb_used_data":16432,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752518544,"statfs":{"total":773094113280,"available":770578989056,"internally_reserved":0,"allocated":16826368,"data_stored":9323486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27944,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-94,"num_write_kb":-60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001625"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.544369+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:27:57.544369+0000","last_peered":"2023-10-06T05:27:57.544369+0000","last_clean":"2023-10-06T05:27:57.544369+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:27:57.544369+0000","last_undegraded":"2023-10-06T05:27:57.544369+0000","last_fullsized":"2023-10-06T05:27:57.544369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.495351+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:27:57.495351+0000","last_peered":"2023-10-06T05:27:57.495351+0000","last_clean":"2023-10-06T05:27:57.495351+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:27:57.495351+0000","last_undegraded":"2023-10-06T05:27:57.495351+0000","last_fullsized":"2023-10-06T05:27:57.495351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":131,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:27:59.035023+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:27:59.035023+0000","last_peered":"2023-10-06T05:27:59.035023+0000","last_clean":"2023-10-06T05:27:59.035023+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:27:59.035023+0000","last_undegraded":"2023-10-06T05:27:59.035023+0000","last_fullsized":"2023-10-06T05:27:59.035023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.495323+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:27:57.495323+0000","last_peered":"2023-10-06T05:27:57.495323+0000","last_clean":"2023-10-06T05:27:57.495323+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:27:57.495323+0000","last_undegraded":"2023-10-06T05:27:57.495323+0000","last_fullsized":"2023-10-06T05:27:57.495323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.843534+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:27:56.843534+0000","last_peered":"2023-10-06T05:27:56.843534+0000","last_clean":"2023-10-06T05:27:56.843534+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:27:56.843534+0000","last_undegraded":"2023-10-06T05:27:56.843534+0000","last_fullsized":"2023-10-06T05:27:56.843534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.843419+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:27:56.843419+0000","last_peered":"2023-10-06T05:27:56.843419+0000","last_clean":"2023-10-06T05:27:56.843419+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:27:56.843419+0000","last_undegraded":"2023-10-06T05:27:56.843419+0000","last_fullsized":"2023-10-06T05:27:56.843419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.844094+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:27:56.844094+0000","last_peered":"2023-10-06T05:27:56.844094+0000","last_clean":"2023-10-06T05:27:56.844094+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:27:56.844094+0000","last_undegraded":"2023-10-06T05:27:56.844094+0000","last_fullsized":"2023-10-06T05:27:56.844094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":130,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.544465+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:27:57.544465+0000","last_peered":"2023-10-06T05:27:57.544465+0000","last_clean":"2023-10-06T05:27:57.544465+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:27:57.544465+0000","last_undegraded":"2023-10-06T05:27:57.544465+0000","last_fullsized":"2023-10-06T05:27:57.544465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":125,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.924935+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:27:57.924935+0000","last_peered":"2023-10-06T05:27:57.924935+0000","last_clean":"2023-10-06T05:27:57.924935+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:27:57.924935+0000","last_undegraded":"2023-10-06T05:27:57.924935+0000","last_fullsized":"2023-10-06T05:27:57.924935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.878},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":1.476},"last":0.37},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":1.5129999999999999},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":2.0190000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0059999999999998},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58699999999999997,"5min":0.76800000000000002,"15min":0.90300000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.504,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.76800000000000002,"15min":0.93500000000000005},"last":0.436}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68999999999999995,"5min":0.81699999999999995,"15min":0.90000000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.84899999999999998},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.91000000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.314},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.92900000000000005},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":4.6900000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":0.86799999999999999},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.76400000000000001,"15min":5.6239999999999997},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.73899999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":11.595000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":5.5800000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.93600000000000005},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":23.146000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.73699999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":9.7539999999999996},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.61099999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":9.9969999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.64600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.751,"5min":0.752,"15min":9.5850000000000009},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.496,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.6439999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":9.8420000000000005},"last":0.46300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":9.718},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":9.8930000000000007},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307400,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064440,"statfs":{"total":96636764160,"available":96321986560,"internally_reserved":0,"allocated":1966080,"data_stored":1018691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.64400000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.373,"5min":0.40699999999999997,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.73699999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.3999999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58199999999999996,"5min":0.67100000000000004,"15min":1.673},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.503},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":4.0490000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77700000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.92600000000000005,"15min":0.94699999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.95199999999999996},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":1.0369999999999999},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.44},"max":{"1min":0.871,"5min":0.95599999999999996,"15min":1.167},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.94499999999999995,"15min":1.026},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92100000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.82799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.30599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56799999999999995,"5min":0.82099999999999995,"15min":1.038},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.30299999999999999,"5min":0.27400000000000002,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":1.0609999999999999},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.252},"max":{"1min":0.60799999999999998,"5min":0.67800000000000005,"15min":3.855},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1919999999999999},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.45200000000000001},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":7.5880000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":7.7670000000000003},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.4039999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.97199999999999998,"15min":2.0870000000000002},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":2.4940000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":5.3600000000000003},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.01,"15min":1.1759999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.45900000000000002,"15min":0.45400000000000001},"max":{"1min":0.874,"5min":0.99199999999999999,"15min":1.0940000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.97899999999999998,"15min":1.0169999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.47499999999999998,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":1.2150000000000001},"last":0.64400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.39500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":2.0720000000000001},"last":0.38},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.899},"last":0.42299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":2.097},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.48699999999999999},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":1.8759999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":1.6719999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.48299999999999998},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":0.70499999999999996,"15min":2.121},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":1.012,"15min":1.7789999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.018,"15min":1.643},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66300000000000003,"15min":0.68999999999999995},"min":{"1min":0.495,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":1.2130000000000001,"15min":2.0129999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.69499999999999995},"min":{"1min":0.51000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.851},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81999999999999995,"5min":0.88300000000000001,"15min":1.571},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.0800000000000001,"15min":2.0449999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0569999999999999,"15min":1.8180000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.44500000000000001},"max":{"1min":1.0029999999999999,"5min":1.131,"15min":1.613},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307384,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064456,"statfs":{"total":96636764160,"available":96322002944,"internally_reserved":0,"allocated":1949696,"data_stored":1016563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26900000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64500000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.311},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40799999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.22900000000000001},"max":{"1min":0.63900000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40999999999999998,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61399999999999999,"5min":1.75,"15min":1.75},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60099999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.378}]},{"osd":2,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72699999999999998,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.0469999999999999,"15min":1.143},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":1.855,"15min":1.855},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.43099999999999999},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":1.885,"15min":1.885},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":1.236},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":1.1950000000000001},"last":0.72899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":311484,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060356,"statfs":{"total":96636764160,"available":96317804544,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":12.983000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.33400000000000002,"15min":0.33100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":1.7430000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.8080000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":12.911},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.73099999999999998,"5min":0.84299999999999997,"15min":5.0890000000000004},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":12.999000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":13.289999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.95099999999999996,"15min":1.5760000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.88200000000000001,"15min":1.552},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.54600000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.85699999999999998,"15min":12.949},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.92900000000000005,"15min":5.2549999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":2.871},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.91500000000000004,"15min":12.859},"last":0.62}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.79400000000000004,"15min":0.82299999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.89400000000000002},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.82199999999999995,"15min":0.84599999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.81599999999999995},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.83299999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.876},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.67600000000000005,"5min":0.85999999999999999,"15min":0.875},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.49199999999999999,"5min":0.432,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.499,"15min":0.48399999999999999},"min":{"1min":0.36499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.92800000000000005},"last":0.505}]},{"osd":6,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.317,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.86699999999999999,"15min":0.876},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84899999999999998,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.875,"15min":0.90900000000000003},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:28:02.959 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:28:02.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:28:03.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.091+0000 7febf85dc700 1 Processor -- start 2023-10-06T05:28:03.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.091+0000 7febf85dc700 1 -- start start 2023-10-06T05:28:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.092+0000 7febf85dc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf85dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febf015ac10 0x7febf00ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf85dc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf015b380 con 0x7febf0156ce0 2023-10-06T05:28:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf85dc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf0157520 con 0x7febf015ac10 2023-10-06T05:28:03.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf85dc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf006c010 con 0x7febf01570b0 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febf015ac10 0x7febf00ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febf015ac10 0x7febf00ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51544/0 (socket says 172.21.15.45:51544) 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 -- 172.21.15.45:0/3766873657 learned_addr learned my addr 172.21.15.45:0/3766873657 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf5b77700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 -- 172.21.15.45:0/3766873657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 msgr2=0x7febf006bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 --2- 172.21.15.45:0/3766873657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf006bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 -- 172.21.15.45:0/3766873657 >> v1:172.21.15.144:6789/0 conn(0x7febf0156ce0 legacy=0x7febf00afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 -- 172.21.15.45:0/3766873657 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf015b660 con 0x7febf015ac10 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf6b79700 1 --2- 172.21.15.45:0/3766873657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf015ac10 0x7febf00ccce0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7febe8009f50 tx=0x7febe8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b914a7071435e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf5376700 1 -- 172.21.15.45:0/3766873657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800e070 con 0x7febf015ac10 2023-10-06T05:28:03.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf5376700 1 -- 172.21.15.45:0/3766873657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe800b9c0 con 0x7febf015ac10 2023-10-06T05:28:03.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.093+0000 7febf5376700 1 -- 172.21.15.45:0/3766873657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80092e0 con 0x7febf015ac10 2023-10-06T05:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 -- 172.21.15.45:0/3766873657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf015ac10 msgr2=0x7febf00ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 --2- 172.21.15.45:0/3766873657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf015ac10 0x7febf00ccce0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7febe8009f50 tx=0x7febe8009840 comp rx=0 tx=0).stop 2023-10-06T05:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 -- 172.21.15.45:0/3766873657 shutdown_connections 2023-10-06T05:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 --2- 172.21.15.45:0/3766873657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf015ac10 0x7febf00ccce0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 --2- 172.21.15.45:0/3766873657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf006bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 -- 172.21.15.45:0/3766873657 >> 172.21.15.45:0/3766873657 conn(0x7febf006f090 msgr2=0x7febf006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 -- 172.21.15.45:0/3766873657 shutdown_connections 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.094+0000 7febf85dc700 1 -- 172.21.15.45:0/3766873657 wait complete. 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.095+0000 7febf85dc700 1 Processor -- start 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.095+0000 7febf85dc700 1 -- start start 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf85dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf0156ce0 0x7febf0158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf85dc700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf01590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf85dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febf015ac10 0x7febf0147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf85dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00c3ee0 con 0x7febf0156ce0 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf85dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00c4180 con 0x7febf015ac10 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf85dc700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf00c4460 con 0x7febf01570b0 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf6378700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf0156ce0 0x7febf0158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf6378700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf0156ce0 0x7febf0158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51548/0 (socket says 172.21.15.45:51548) 2023-10-06T05:28:03.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf6378700 1 -- 172.21.15.45:0/2048290386 learned_addr learned my addr 172.21.15.45:0/2048290386 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf5b77700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf01590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.096+0000 7febf6b79700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febf015ac10 0x7febf0147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf6378700 1 -- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 msgr2=0x7febf01590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf6378700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf01590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf6378700 1 -- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febf015ac10 msgr2=0x7febf0147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf6378700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febf015ac10 0x7febf0147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf6378700 1 -- 172.21.15.45:0/2048290386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009ae0 con 0x7febf0156ce0 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf6378700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf0156ce0 0x7febf0158040 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7febec00c4e0 tx=0x7febec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febe67fc700 1 -- 172.21.15.45:0/2048290386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec011590 con 0x7febf0156ce0 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febe67fc700 1 -- 172.21.15.45:0/2048290386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec011710 con 0x7febf0156ce0 2023-10-06T05:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf00c4740 con 0x7febf0156ce0 2023-10-06T05:28:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febe67fc700 1 -- 172.21.15.45:0/2048290386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec011ad0 con 0x7febf0156ce0 2023-10-06T05:28:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.097+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf00c6100 con 0x7febf0156ce0 2023-10-06T05:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.098+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf00c62f0 con 0x7febf0156ce0 2023-10-06T05:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.099+0000 7febe67fc700 1 -- 172.21.15.45:0/2048290386 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febec01b3b0 con 0x7febf0156ce0 2023-10-06T05:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.099+0000 7febe67fc700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7febd4066f30 0x7febd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.099+0000 7febe67fc700 1 -- 172.21.15.45:0/2048290386 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7febec083930 con 0x7febf0156ce0 2023-10-06T05:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.099+0000 7febf5b77700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7febd4066f30 0x7febd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.100+0000 7febf5b77700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7febd4066f30 0x7febd40693e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7febf00c3da0 tx=0x7febe00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.103+0000 7febe67fc700 1 -- 172.21.15.45:0/2048290386 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febec050ba0 con 0x7febf0156ce0 2023-10-06T05:28:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.302+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febf00c4930 con 0x7febd4066f30 2023-10-06T05:28:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.305+0000 7febe67fc700 1 -- 172.21.15.45:0/2048290386 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73411 (secure 0 0 0) 0x7febf00c4930 con 0x7febd4066f30 2023-10-06T05:28:03.308 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7febd4066f30 msgr2=0x7febd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7febd4066f30 0x7febd40693e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7febf00c3da0 tx=0x7febe00091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf0156ce0 msgr2=0x7febf0158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf0156ce0 0x7febf0158040 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7febec00c4e0 tx=0x7febec00c7f0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 shutdown_connections 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7febd4066f30 0x7febd40693e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febf015ac10 0x7febf0147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febf0156ce0 0x7febf0158040 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 --2- 172.21.15.45:0/2048290386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febf01570b0 0x7febf01590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 >> 172.21.15.45:0/2048290386 conn(0x7febf006f090 msgr2=0x7febf0068e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 shutdown_connections 2023-10-06T05:28:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.308+0000 7febf85dc700 1 -- 172.21.15.45:0/2048290386 wait complete. 2023-10-06T05:28:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:28:03.323 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":537,"stamp":"2023-10-06T05:28:02.643782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2456176,"kb_used_data":16432,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752518544,"statfs":{"total":773094113280,"available":770578989056,"internally_reserved":0,"allocated":16826368,"data_stored":9323486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27944,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-60,"num_read_kb":-61,"num_write":-94,"num_write_kb":-60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001625"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.544369+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:27:57.544369+0000","last_peered":"2023-10-06T05:27:57.544369+0000","last_clean":"2023-10-06T05:27:57.544369+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:27:57.544369+0000","last_undegraded":"2023-10-06T05:27:57.544369+0000","last_fullsized":"2023-10-06T05:27:57.544369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:13.020222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.495351+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:27:57.495351+0000","last_peered":"2023-10-06T05:27:57.495351+0000","last_clean":"2023-10-06T05:27:57.495351+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:27:57.495351+0000","last_undegraded":"2023-10-06T05:27:57.495351+0000","last_fullsized":"2023-10-06T05:27:57.495351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":131,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:27:59.035023+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:27:59.035023+0000","last_peered":"2023-10-06T05:27:59.035023+0000","last_clean":"2023-10-06T05:27:59.035023+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:27:59.035023+0000","last_undegraded":"2023-10-06T05:27:59.035023+0000","last_fullsized":"2023-10-06T05:27:59.035023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.495323+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:27:57.495323+0000","last_peered":"2023-10-06T05:27:57.495323+0000","last_clean":"2023-10-06T05:27:57.495323+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:27:57.495323+0000","last_undegraded":"2023-10-06T05:27:57.495323+0000","last_fullsized":"2023-10-06T05:27:57.495323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.843534+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:27:56.843534+0000","last_peered":"2023-10-06T05:27:56.843534+0000","last_clean":"2023-10-06T05:27:56.843534+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:27:56.843534+0000","last_undegraded":"2023-10-06T05:27:56.843534+0000","last_fullsized":"2023-10-06T05:27:56.843534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.843419+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:27:56.843419+0000","last_peered":"2023-10-06T05:27:56.843419+0000","last_clean":"2023-10-06T05:27:56.843419+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:27:56.843419+0000","last_undegraded":"2023-10-06T05:27:56.843419+0000","last_fullsized":"2023-10-06T05:27:56.843419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.844094+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:27:56.844094+0000","last_peered":"2023-10-06T05:27:56.844094+0000","last_clean":"2023-10-06T05:27:56.844094+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:27:56.844094+0000","last_undegraded":"2023-10-06T05:27:56.844094+0000","last_fullsized":"2023-10-06T05:27:56.844094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":130,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.544465+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:27:57.544465+0000","last_peered":"2023-10-06T05:27:57.544465+0000","last_clean":"2023-10-06T05:27:57.544465+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:27:57.544465+0000","last_undegraded":"2023-10-06T05:27:57.544465+0000","last_fullsized":"2023-10-06T05:27:57.544465+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":125,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.924935+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:27:57.924935+0000","last_peered":"2023-10-06T05:27:57.924935+0000","last_clean":"2023-10-06T05:27:57.924935+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:27:57.924935+0000","last_undegraded":"2023-10-06T05:27:57.924935+0000","last_fullsized":"2023-10-06T05:27:57.924935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.878},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":1.476},"last":0.37},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":1.5129999999999999},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":2.0190000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0059999999999998},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58699999999999997,"5min":0.76800000000000002,"15min":0.90300000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.504,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.76800000000000002,"15min":0.93500000000000005},"last":0.436}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68999999999999995,"5min":0.81699999999999995,"15min":0.90000000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.84899999999999998},"last":0.70799999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.91000000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.314},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.92900000000000005},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":4.6900000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":0.86799999999999999},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.76400000000000001,"15min":5.6239999999999997},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.73899999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":11.595000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":5.5800000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.93600000000000005},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":23.146000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.73699999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":9.7539999999999996},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.61099999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":9.9969999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.64600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.751,"5min":0.752,"15min":9.5850000000000009},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.496,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.6439999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":9.8420000000000005},"last":0.46300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":9.718},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":9.8930000000000007},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307400,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064440,"statfs":{"total":96636764160,"available":96321986560,"internally_reserved":0,"allocated":1966080,"data_stored":1018691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.64400000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.373,"5min":0.40699999999999997,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.73699999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.3999999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58199999999999996,"5min":0.67100000000000004,"15min":1.673},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.503},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":4.0490000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77700000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.92600000000000005,"15min":0.94699999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.95199999999999996},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":1.0369999999999999},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.44},"max":{"1min":0.871,"5min":0.95599999999999996,"15min":1.167},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.94499999999999995,"15min":1.026},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92100000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.82799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.30599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56799999999999995,"5min":0.82099999999999995,"15min":1.038},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.30299999999999999,"5min":0.27400000000000002,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":1.0609999999999999},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.252},"max":{"1min":0.60799999999999998,"5min":0.67800000000000005,"15min":3.855},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1919999999999999},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.45200000000000001},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":7.5880000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":7.7670000000000003},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.4039999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.97199999999999998,"15min":2.0870000000000002},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":2.4940000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":5.3600000000000003},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.01,"15min":1.1759999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.45900000000000002,"15min":0.45400000000000001},"max":{"1min":0.874,"5min":0.99199999999999999,"15min":1.0940000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.97899999999999998,"15min":1.0169999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.47499999999999998,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":1.2150000000000001},"last":0.64400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.39500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":2.0720000000000001},"last":0.38},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.899},"last":0.42299999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":2.097},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.48699999999999999},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":1.8759999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":1.6719999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.48299999999999998},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":0.70499999999999996,"15min":2.121},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":1.012,"15min":1.7789999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.018,"15min":1.643},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66300000000000003,"15min":0.68999999999999995},"min":{"1min":0.495,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":1.2130000000000001,"15min":2.0129999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.69499999999999995},"min":{"1min":0.51000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.851},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81999999999999995,"5min":0.88300000000000001,"15min":1.571},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.0800000000000001,"15min":2.0449999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0569999999999999,"15min":1.8180000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.44500000000000001},"max":{"1min":1.0029999999999999,"5min":1.131,"15min":1.613},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307384,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064456,"statfs":{"total":96636764160,"available":96322002944,"internally_reserved":0,"allocated":1949696,"data_stored":1016563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26900000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64500000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.311},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40799999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.22900000000000001},"max":{"1min":0.63900000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40999999999999998,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61399999999999999,"5min":1.75,"15min":1.75},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60099999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.378}]},{"osd":2,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72699999999999998,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.0469999999999999,"15min":1.143},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":1.855,"15min":1.855},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.43099999999999999},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":1.885,"15min":1.885},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":1.236},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":1.1950000000000001},"last":0.72899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":311484,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060356,"statfs":{"total":96636764160,"available":96317804544,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":12.983000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.33400000000000002,"15min":0.33100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":1.7430000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.8080000000000001},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":12.911},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.73099999999999998,"5min":0.84299999999999997,"15min":5.0890000000000004},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":12.999000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":13.289999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.95099999999999996,"15min":1.5760000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.88200000000000001,"15min":1.552},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.54600000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.85699999999999998,"15min":12.949},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.92900000000000005,"15min":5.2549999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":2.871},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.91500000000000004,"15min":12.859},"last":0.62}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.79400000000000004,"15min":0.82299999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.89400000000000002},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.82199999999999995,"15min":0.84599999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.81599999999999995},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.83299999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.876},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.67600000000000005,"5min":0.85999999999999999,"15min":0.875},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.49199999999999999,"5min":0.432,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.499,"15min":0.48399999999999999},"min":{"1min":0.36499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.92800000000000005},"last":0.505}]},{"osd":6,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.317,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.86699999999999999,"15min":0.876},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84899999999999998,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.875,"15min":0.90900000000000003},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:28:03.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.455+0000 7f48bb0f4700 1 Processor -- start 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.455+0000 7f48bb0f4700 1 -- start start 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48bb0f4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48b4155cd0 0x7f48b415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48bb0f4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f48b40ccdc0 0x7f48b40cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48bb0f4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40cd700 con 0x7f48b41560a0 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48bb0f4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40ae6b0 con 0x7f48b4155cd0 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48bb0f4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40ae8d0 con 0x7f48b40ccdc0 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48b8e90700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48b4155cd0 0x7f48b415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48b8e90700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48b4155cd0 0x7f48b415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51556/0 (socket says 172.21.15.45:51556) 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48b3fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f48b41560a0 0x7f48b40adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59658/0 (socket says 172.21.15.45:59658) 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48b8e90700 1 -- 172.21.15.45:0/2272210424 learned_addr learned my addr 172.21.15.45:0/2272210424 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.456+0000 7f48b9691700 1 --2- 172.21.15.45:0/2272210424 >> v2:172.21.15.144:3300/0 conn(0x7f48b40ccdc0 0x7f48b40cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b8e90700 1 -- 172.21.15.45:0/2272210424 >> v2:172.21.15.144:3300/0 conn(0x7f48b40ccdc0 msgr2=0x7f48b40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b8e90700 1 --2- 172.21.15.45:0/2272210424 >> v2:172.21.15.144:3300/0 conn(0x7f48b40ccdc0 0x7f48b40cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b8e90700 1 -- 172.21.15.45:0/2272210424 >> v1:172.21.15.45:6789/0 conn(0x7f48b41560a0 legacy=0x7f48b40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b8e90700 1 -- 172.21.15.45:0/2272210424 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40aebb0 con 0x7f48b4155cd0 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b37fe700 1 -- 172.21.15.45:0/2272210424 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827935984 0 0) 0x7f48b40cd700 con 0x7f48b41560a0 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b8e90700 1 --2- 172.21.15.45:0/2272210424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155cd0 0x7f48b415a730 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c3dd0 tx=0x7f48a400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf177c6cc1788fb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b37fe700 1 -- 172.21.15.45:0/2272210424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400dba0 con 0x7f48b4155cd0 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b37fe700 1 -- 172.21.15.45:0/2272210424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a400dd00 con 0x7f48b4155cd0 2023-10-06T05:28:03.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.457+0000 7f48b37fe700 1 -- 172.21.15.45:0/2272210424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40196d0 con 0x7f48b4155cd0 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 -- 172.21.15.45:0/2272210424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155cd0 msgr2=0x7f48b415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/2272210424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155cd0 0x7f48b415a730 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c3dd0 tx=0x7f48a400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 -- 172.21.15.45:0/2272210424 shutdown_connections 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/2272210424 >> v2:172.21.15.144:3300/0 conn(0x7f48b40ccdc0 0x7f48b40cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/2272210424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155cd0 0x7f48b415a730 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 -- 172.21.15.45:0/2272210424 >> 172.21.15.45:0/2272210424 conn(0x7f48b406f090 msgr2=0x7f48b406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 -- 172.21.15.45:0/2272210424 shutdown_connections 2023-10-06T05:28:03.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.458+0000 7f48bb0f4700 1 -- 172.21.15.45:0/2272210424 wait complete. 2023-10-06T05:28:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.459+0000 7f48bb0f4700 1 Processor -- start 2023-10-06T05:28:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.459+0000 7f48bb0f4700 1 -- start start 2023-10-06T05:28:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48bb0f4700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48b40ccdc0 0x7f48b40c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48bb0f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b4155cd0 0x7f48b4068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48bb0f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b41560a0 0x7f48b4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48bb0f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4165e90 con 0x7f48b41560a0 2023-10-06T05:28:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48bb0f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41669d0 con 0x7f48b4155cd0 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48bb0f4700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41660a0 con 0x7f48b40ccdc0 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48b9691700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b41560a0 0x7f48b4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48b9691700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b41560a0 0x7f48b4151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51558/0 (socket says 172.21.15.45:51558) 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48b9691700 1 -- 172.21.15.45:0/21952471 learned_addr learned my addr 172.21.15.45:0/21952471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48b3fff700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b4155cd0 0x7f48b4068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48b8e90700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48b40ccdc0 0x7f48b40c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.460+0000 7f48b9691700 1 -- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48b40ccdc0 msgr2=0x7f48b40c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b9691700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48b40ccdc0 0x7f48b40c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b9691700 1 -- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b4155cd0 msgr2=0x7f48b4068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b9691700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b4155cd0 0x7f48b4068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b9691700 1 -- 172.21.15.45:0/21952471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a400b790 con 0x7f48b41560a0 2023-10-06T05:28:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b8e90700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48b40ccdc0 0x7f48b40c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b9691700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b41560a0 0x7f48b4151a40 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009e90 tx=0x7f48ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b0ff9700 1 -- 172.21.15.45:0/21952471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac00cc20 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b0ff9700 1 -- 172.21.15.45:0/21952471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac00f040 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48b0ff9700 1 -- 172.21.15.45:0/21952471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac014480 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b4166380 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.461+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b40c6210 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.462+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b406e2f0 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.462+0000 7f48b0ff9700 1 -- 172.21.15.45:0/21952471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48ac014830 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.463+0000 7f48b0ff9700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48a0066f30 0x7f48a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.463+0000 7f48b0ff9700 1 -- 172.21.15.45:0/21952471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f48ac084f50 con 0x7f48b41560a0 2023-10-06T05:28:03.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.463+0000 7f48b8e90700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48a0066f30 0x7f48a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:03.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.464+0000 7f48b8e90700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48a0066f30 0x7f48a00693e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c3dd0 tx=0x7f48a401b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:03.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.467+0000 7f48b0ff9700 1 -- 172.21.15.45:0/21952471 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48ac055c50 con 0x7f48b41560a0 2023-10-06T05:28:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.670+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48b40aee90 con 0x7f48b41560a0 2023-10-06T05:28:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.671+0000 7f48b0ff9700 1 -- 172.21.15.45:0/21952471 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+14220 (secure 0 0 0) 0x7f48ac0557e0 con 0x7f48b41560a0 2023-10-06T05:28:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:03.673 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":149,"fsid":"158572ea-864f-4150-9c7e-6d3b4fe3c21a","created":"2023-10-06T05:14:40.791337+0000","modified":"2023-10-06T05:28:00.031787+0000","last_up_change":"2023-10-06T05:14:51.823626+0000","last_in_change":"2023-10-06T05:14:43.727684+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T05:14:52.494626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T05:14:52.632144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c3c47b6a-60d4-46ff-a7fc-2e09430ec9a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":119749},{"type":"v1","addr":"172.21.15.45:6817","nonce":119749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":119749},{"type":"v1","addr":"172.21.15.45:6819","nonce":119749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":119749},{"type":"v1","addr":"172.21.15.45:6823","nonce":119749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":119749},{"type":"v1","addr":"172.21.15.45:6821","nonce":119749}]},"public_addr":"172.21.15.45:6817/119749","cluster_addr":"172.21.15.45:6819/119749","heartbeat_back_addr":"172.21.15.45:6823/119749","heartbeat_front_addr":"172.21.15.45:6821/119749","state":["exists","up"]},{"osd":1,"uuid":"8efcdf9d-7f7d-4c07-98f5-1d752cb42495","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":119738},{"type":"v1","addr":"172.21.15.45:6801","nonce":119738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":119738},{"type":"v1","addr":"172.21.15.45:6803","nonce":119738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":119738},{"type":"v1","addr":"172.21.15.45:6807","nonce":119738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":119738},{"type":"v1","addr":"172.21.15.45:6805","nonce":119738}]},"public_addr":"172.21.15.45:6801/119738","cluster_addr":"172.21.15.45:6803/119738","heartbeat_back_addr":"172.21.15.45:6807/119738","heartbeat_front_addr":"172.21.15.45:6805/119738","state":["exists","up"]},{"osd":2,"uuid":"0540e79e-e321-4dcf-8ed6-1d023b47ab28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":119739},{"type":"v1","addr":"172.21.15.45:6809","nonce":119739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":119739},{"type":"v1","addr":"172.21.15.45:6811","nonce":119739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":119739},{"type":"v1","addr":"172.21.15.45:6815","nonce":119739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":119739},{"type":"v1","addr":"172.21.15.45:6813","nonce":119739}]},"public_addr":"172.21.15.45:6809/119739","cluster_addr":"172.21.15.45:6811/119739","heartbeat_back_addr":"172.21.15.45:6815/119739","heartbeat_front_addr":"172.21.15.45:6813/119739","state":["exists","up"]},{"osd":3,"uuid":"7500eeb7-53cf-41c3-bc9d-6bc9a6165384","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":119762},{"type":"v1","addr":"172.21.15.45:6825","nonce":119762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":119762},{"type":"v1","addr":"172.21.15.45:6827","nonce":119762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":119762},{"type":"v1","addr":"172.21.15.45:6831","nonce":119762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":119762},{"type":"v1","addr":"172.21.15.45:6829","nonce":119762}]},"public_addr":"172.21.15.45:6825/119762","cluster_addr":"172.21.15.45:6827/119762","heartbeat_back_addr":"172.21.15.45:6831/119762","heartbeat_front_addr":"172.21.15.45:6829/119762","state":["exists","up"]},{"osd":4,"uuid":"8fb00fd5-c06a-4ce9-adba-cf22ef411339","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":119910},{"type":"v1","addr":"172.21.15.144:6825","nonce":119910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":119910},{"type":"v1","addr":"172.21.15.144:6827","nonce":119910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":119910},{"type":"v1","addr":"172.21.15.144:6831","nonce":119910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":119910},{"type":"v1","addr":"172.21.15.144:6829","nonce":119910}]},"public_addr":"172.21.15.144:6825/119910","cluster_addr":"172.21.15.144:6827/119910","heartbeat_back_addr":"172.21.15.144:6831/119910","heartbeat_front_addr":"172.21.15.144:6829/119910","state":["exists","up"]},{"osd":5,"uuid":"b9b96788-94eb-4910-97ca-3469eb4fd873","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":119917},{"type":"v1","addr":"172.21.15.144:6817","nonce":119917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":119917},{"type":"v1","addr":"172.21.15.144:6819","nonce":119917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":119917},{"type":"v1","addr":"172.21.15.144:6823","nonce":119917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":119917},{"type":"v1","addr":"172.21.15.144:6821","nonce":119917}]},"public_addr":"172.21.15.144:6817/119917","cluster_addr":"172.21.15.144:6819/119917","heartbeat_back_addr":"172.21.15.144:6823/119917","heartbeat_front_addr":"172.21.15.144:6821/119917","state":["exists","up"]},{"osd":6,"uuid":"fb1fe043-d4b9-4353-b923-f39108e85b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":119899},{"type":"v1","addr":"172.21.15.144:6801","nonce":119899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":119899},{"type":"v1","addr":"172.21.15.144:6803","nonce":119899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":119899},{"type":"v1","addr":"172.21.15.144:6807","nonce":119899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":119899},{"type":"v1","addr":"172.21.15.144:6805","nonce":119899}]},"public_addr":"172.21.15.144:6801/119899","cluster_addr":"172.21.15.144:6803/119899","heartbeat_back_addr":"172.21.15.144:6807/119899","heartbeat_front_addr":"172.21.15.144:6805/119899","state":["exists","up"]},{"osd":7,"uuid":"818a9fdc-9978-46ce-8993-1f4ff8d9c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":119901},{"type":"v1","addr":"172.21.15.144:6809","nonce":119901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":119901},{"type":"v1","addr":"172.21.15.144:6811","nonce":119901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":119901},{"type":"v1","addr":"172.21.15.144:6815","nonce":119901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":119901},{"type":"v1","addr":"172.21.15.144:6813","nonce":119901}]},"public_addr":"172.21.15.144:6809/119901","cluster_addr":"172.21.15.144:6811/119901","heartbeat_back_addr":"172.21.15.144:6815/119901","heartbeat_front_addr":"172.21.15.144:6813/119901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.847143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.604995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.929157+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.890663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.944140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.892204+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.830872+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:14:48.904797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/4232324777":"2023-10-07T05:27:56.838619+0000","172.21.15.45:6832/4232324777":"2023-10-07T05:27:56.838619+0000","172.21.15.45:6833/2716751458":"2023-10-07T05:27:38.843219+0000","172.21.15.45:6832/2716751458":"2023-10-07T05:27:38.843219+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:28:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48a0066f30 msgr2=0x7f48a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48a0066f30 0x7f48a00693e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c3dd0 tx=0x7f48a401b000 comp rx=0 tx=0).stop 2023-10-06T05:28:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b41560a0 msgr2=0x7f48b4151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b41560a0 0x7f48b4151a40 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009e90 tx=0x7f48ac00b040 comp rx=0 tx=0).stop 2023-10-06T05:28:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 shutdown_connections 2023-10-06T05:28:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f48a0066f30 0x7f48a00693e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b41560a0 0x7f48b4151a40 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48b40ccdc0 0x7f48b40c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 --2- 172.21.15.45:0/21952471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48b4155cd0 0x7f48b4068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 >> 172.21.15.45:0/21952471 conn(0x7f48b406f090 msgr2=0x7f48b4154c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 shutdown_connections 2023-10-06T05:28:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:03.673+0000 7f48bb0f4700 1 -- 172.21.15.45:0/21952471 wait complete. 2023-10-06T05:28:04.689 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T05:28:04.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.833+0000 7f656f0cd700 1 Processor -- start 2023-10-06T05:28:04.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.833+0000 7f656f0cd700 1 -- start start 2023-10-06T05:28:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.834+0000 7f656f0cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159530 0x7f65680c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.834+0000 7f656f0cd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65680c8140 con 0x7f65680c9cc0 2023-10-06T05:28:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.834+0000 7f656f0cd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65680ca1e0 con 0x7f6568159160 2023-10-06T05:28:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.834+0000 7f656f0cd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65680cb500 con 0x7f6568159530 2023-10-06T05:28:04.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.834+0000 7f656d66a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f65680c9cc0 0x7f6568067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59666/0 (socket says 172.21.15.45:59666) 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.834+0000 7f656d66a700 1 -- 172.21.15.45:0/1317511113 learned_addr learned my addr 172.21.15.45:0/1317511113 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.834+0000 7f655ffff700 1 --2- 172.21.15.45:0/1317511113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159530 0x7f65680c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132832206 0 0) 0x7f65680c8140 con 0x7f65680c9cc0 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6550003570 con 0x7f65680c9cc0 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798861466 0 0) 0x7f65680ca1e0 con 0x7f6568159160 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65680c8140 con 0x7f6568159160 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655ffff700 1 -- 172.21.15.45:0/1317511113 >> v1:172.21.15.144:6789/0 conn(0x7f6568159160 legacy=0x7f65680cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996464927 0 0) 0x7f6550003570 con 0x7f65680c9cc0 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655ffff700 1 -- 172.21.15.45:0/1317511113 >> v1:172.21.15.45:6789/0 conn(0x7f65680c9cc0 legacy=0x7f6568067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655ffff700 1 -- 172.21.15.45:0/1317511113 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680ca4c0 con 0x7f6568159530 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655ffff700 1 --2- 172.21.15.45:0/1317511113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159530 0x7f65680c9780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6558009910 tx=0x7f65580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0dbe492458fc2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600017f0 con 0x7f6568159530 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6558011420 con 0x7f6568159530 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.835+0000 7f655f7fe700 1 -- 172.21.15.45:0/1317511113 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65580117a0 con 0x7f6568159530 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.836+0000 7f656f0cd700 1 -- 172.21.15.45:0/1317511113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159530 msgr2=0x7f65680c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.836+0000 7f656f0cd700 1 --2- 172.21.15.45:0/1317511113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159530 0x7f65680c9780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6558009910 tx=0x7f65580091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.836+0000 7f656f0cd700 1 -- 172.21.15.45:0/1317511113 shutdown_connections 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.836+0000 7f656f0cd700 1 --2- 172.21.15.45:0/1317511113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159530 0x7f65680c9780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.836+0000 7f656f0cd700 1 -- 172.21.15.45:0/1317511113 >> 172.21.15.45:0/1317511113 conn(0x7f6568068730 msgr2=0x7f6568068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.836+0000 7f656f0cd700 1 -- 172.21.15.45:0/1317511113 shutdown_connections 2023-10-06T05:28:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.836+0000 7f656f0cd700 1 -- 172.21.15.45:0/1317511113 wait complete. 2023-10-06T05:28:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.837+0000 7f656f0cd700 1 Processor -- start 2023-10-06T05:28:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.837+0000 7f656f0cd700 1 -- start start 2023-10-06T05:28:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656f0cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9cc0 0x7f6568168980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656f0cd700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159160 0x7f65680c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656f0cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6568159530 0x7f65680b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656f0cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65680c6510 con 0x7f65680c9cc0 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656f0cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65680aeea0 con 0x7f6568159530 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656f0cd700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65680aebf0 con 0x7f6568159160 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656ce69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9cc0 0x7f6568168980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656ce69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9cc0 0x7f6568168980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51570/0 (socket says 172.21.15.45:51570) 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656ce69700 1 -- 172.21.15.45:0/2600672796 learned_addr learned my addr 172.21.15.45:0/2600672796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f656d66a700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6568159530 0x7f65680b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.838+0000 7f655ffff700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159160 0x7f65680c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656ce69700 1 -- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159160 msgr2=0x7f65680c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656ce69700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568159160 0x7f65680c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656ce69700 1 -- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6568159530 msgr2=0x7f65680b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656ce69700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6568159530 0x7f65680b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656ce69700 1 -- 172.21.15.45:0/2600672796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6558009420 con 0x7f65680c9cc0 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656ce69700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9cc0 0x7f6568168980 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f656400c790 tx=0x7f6564007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564010510 con 0x7f65680c9cc0 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564010670 con 0x7f65680c9cc0 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65680c69f0 con 0x7f65680c9cc0 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564010a30 con 0x7f65680c9cc0 2023-10-06T05:28:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.839+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6568168ec0 con 0x7f65680c9cc0 2023-10-06T05:28:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.840+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f656804c380 con 0x7f65680c9cc0 2023-10-06T05:28:04.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.840+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f656401b3b0 con 0x7f65680c9cc0 2023-10-06T05:28:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.841+0000 7f655cff9700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6554066f30 0x7f65540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.841+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f6564083da0 con 0x7f65680c9cc0 2023-10-06T05:28:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.841+0000 7f655cff9700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f655406d8f0 0x7f655406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.841+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65540703f0 con 0x7f655406d8f0 2023-10-06T05:28:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.841+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65640863e0 con 0x7f65680c9cc0 2023-10-06T05:28:04.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.841+0000 7f655ffff700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6554066f30 0x7f65540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.841+0000 7f656d66a700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f655406d8f0 0x7f655406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.842+0000 7f656d66a700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f655406d8f0 0x7f655406fd20 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.842+0000 7f655ffff700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6554066f30 0x7f65540693e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6558009910 tx=0x7f6558016000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.842+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== osd.0 v2:172.21.15.45:6816/119749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f65540703f0 con 0x7f655406d8f0 2023-10-06T05:28:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 --> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65680c83e0 con 0x7f655406d8f0 2023-10-06T05:28:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f655cff9700 1 -- 172.21.15.45:0/2600672796 <== osd.0 v2:172.21.15.45:6816/119749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f65680c83e0 con 0x7f655406d8f0 2023-10-06T05:28:04.859 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:04.859 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:04.859 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f655406d8f0 msgr2=0x7f655406fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:6816/119749,v1:172.21.15.45:6817/119749] conn(0x7f655406d8f0 0x7f655406fd20 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6554066f30 msgr2=0x7f65540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6554066f30 0x7f65540693e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6558009910 tx=0x7f6558016000 comp rx=0 tx=0).stop 2023-10-06T05:28:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9cc0 msgr2=0x7f6568168980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 --2- 172.21.15.45:0/2600672796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65680c9cc0 0x7f6568168980 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f656400c790 tx=0x7f6564007e10 comp rx=0 tx=0).stop 2023-10-06T05:28:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656d66a700 1 -- 172.21.15.45:0/2600672796 reap_dead start 2023-10-06T05:28:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 shutdown_connections 2023-10-06T05:28:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.857+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 >> 172.21.15.45:0/2600672796 conn(0x7f6568068730 msgr2=0x7f65680c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.858+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 shutdown_connections 2023-10-06T05:28:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:04.858+0000 7f656f0cd700 1 -- 172.21.15.45:0/2600672796 wait complete. 2023-10-06T05:28:04.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T05:28:05.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.004+0000 7f56fdb5a700 1 Processor -- start 2023-10-06T05:28:05.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.004+0000 7f56fdb5a700 1 -- start start 2023-10-06T05:28:05.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.004+0000 7f56fdb5a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f815a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.004+0000 7f56fdb5a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f56f8153a20 0x7f56f806a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.004+0000 7f56fdb5a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80676c0 con 0x7f56f806add0 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.004+0000 7f56fdb5a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f806b270 con 0x7f56f8153a20 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.004+0000 7f56fdb5a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80679a0 con 0x7f56f8153650 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f77fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f6ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f56f8153a20 0x7f56f806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f6ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f56f8153a20 0x7f56f806a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.45:52344/0 (socket says 172.21.15.45:52344) 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f6ffd700 1 -- 172.21.15.45:0/2940732149 learned_addr learned my addr 172.21.15.45:0/2940732149 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f7fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f56f806add0 0x7f56f8068480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:57288/0 (socket says 172.21.15.45:57288) 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f77fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f815a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:37198/0 (socket says 172.21.15.45:37198) 2023-10-06T05:28:05.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f77fe700 1 -- 172.21.15.45:0/2940732149 >> v2:172.21.15.144:3300/0 conn(0x7f56f8153a20 msgr2=0x7f56f806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f77fe700 1 --2- 172.21.15.45:0/2940732149 >> v2:172.21.15.144:3300/0 conn(0x7f56f8153a20 0x7f56f806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f77fe700 1 -- 172.21.15.45:0/2940732149 >> v1:172.21.15.144:6789/0 conn(0x7f56f806add0 legacy=0x7f56f8068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f77fe700 1 -- 172.21.15.45:0/2940732149 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8067c80 con 0x7f56f8153650 2023-10-06T05:28:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.005+0000 7f56f77fe700 1 --2- 172.21.15.45:0/2940732149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f815a930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f56e80098e0 tx=0x7f56e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b74d6571974b3085 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56f67fc700 1 -- 172.21.15.45:0/2940732149 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800b640 con 0x7f56f8153650 2023-10-06T05:28:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56f67fc700 1 -- 172.21.15.45:0/2940732149 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e800b7a0 con 0x7f56f8153650 2023-10-06T05:28:05.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56f67fc700 1 -- 172.21.15.45:0/2940732149 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800bb40 con 0x7f56f8153650 2023-10-06T05:28:05.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 -- 172.21.15.45:0/2940732149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 msgr2=0x7f56f815a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/2940732149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f815a930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f56e80098e0 tx=0x7f56e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 -- 172.21.15.45:0/2940732149 shutdown_connections 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/2940732149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f815a930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/2940732149 >> v2:172.21.15.144:3300/0 conn(0x7f56f8153a20 0x7f56f806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 -- 172.21.15.45:0/2940732149 >> 172.21.15.45:0/2940732149 conn(0x7f56f80cc5a0 msgr2=0x7f56f80cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 -- 172.21.15.45:0/2940732149 shutdown_connections 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.006+0000 7f56fdb5a700 1 -- 172.21.15.45:0/2940732149 wait complete. 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.008+0000 7f56fdb5a700 1 Processor -- start 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.008+0000 7f56fdb5a700 1 -- start start 2023-10-06T05:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.008+0000 7f56fdb5a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f806add0 0x7f56f8155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56fdb5a700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f80b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56fdb5a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8153a20 0x7f56f8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56fdb5a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f81566a0 con 0x7f56f8153a20 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56fdb5a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80af7f0 con 0x7f56f806add0 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56fdb5a700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80b22f0 con 0x7f56f8153650 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8153a20 0x7f56f8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8153a20 0x7f56f8151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51584/0 (socket says 172.21.15.45:51584) 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 -- 172.21.15.45:0/3484891190 learned_addr learned my addr 172.21.15.45:0/3484891190 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f6ffd700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f80b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 -- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 msgr2=0x7f56f80b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f80b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 -- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f806add0 msgr2=0x7f56f8155a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f806add0 0x7f56f8155a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 -- 172.21.15.45:0/3484891190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e8009420 con 0x7f56f8153a20 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56f7fff700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8153a20 0x7f56f8151bb0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f56f000cc70 tx=0x7f56f0001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56df7fe700 1 -- 172.21.15.45:0/3484891190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0005df0 con 0x7f56f8153a20 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56df7fe700 1 -- 172.21.15.45:0/3484891190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f000a660 con 0x7f56f8153a20 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f81520f0 con 0x7f56f8153a20 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.009+0000 7f56df7fe700 1 -- 172.21.15.45:0/3484891190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f000aa20 con 0x7f56f8153a20 2023-10-06T05:28:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.010+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f81563c0 con 0x7f56f8153a20 2023-10-06T05:28:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.011+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f80cc0e0 con 0x7f56f8153a20 2023-10-06T05:28:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.011+0000 7f56df7fe700 1 -- 172.21.15.45:0/3484891190 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56f00070f0 con 0x7f56f8153a20 2023-10-06T05:28:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.011+0000 7f56df7fe700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56e4066f30 0x7f56e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.012+0000 7f56df7fe700 1 -- 172.21.15.45:0/3484891190 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f56f0084490 con 0x7f56f8153a20 2023-10-06T05:28:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.012+0000 7f56f77fe700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56e4066f30 0x7f56e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.012+0000 7f56f77fe700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56e4066f30 0x7f56e40693e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f56e800a010 tx=0x7f56e8017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.016+0000 7f56df7fe700 1 -- 172.21.15.45:0/3484891190 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56f0051830 con 0x7f56f8153a20 2023-10-06T05:28:05.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.224+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f56f8067f60 con 0x7f56e4066f30 2023-10-06T05:28:05.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.225+0000 7f56df7fe700 1 -- 172.21.15.45:0/3484891190 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56f8067f60 con 0x7f56e4066f30 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.227+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56e4066f30 msgr2=0x7f56e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.227+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56e4066f30 0x7f56e40693e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f56e800a010 tx=0x7f56e8017000 comp rx=0 tx=0).stop 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8153a20 msgr2=0x7f56f8151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8153a20 0x7f56f8151bb0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f56f000cc70 tx=0x7f56f0001a70 comp rx=0 tx=0).stop 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 shutdown_connections 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f56e4066f30 0x7f56e40693e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8153a20 0x7f56f8151bb0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56f806add0 0x7f56f8155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 --2- 172.21.15.45:0/3484891190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56f8153650 0x7f56f80b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 >> 172.21.15.45:0/3484891190 conn(0x7f56f80cc5a0 msgr2=0x7f56f806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 shutdown_connections 2023-10-06T05:28:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.228+0000 7f56fdb5a700 1 -- 172.21.15.45:0/3484891190 wait complete. 2023-10-06T05:28:05.231 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T05:28:05.243 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T05:28:05.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:05.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.377+0000 7f3b9c7df700 1 Processor -- start 2023-10-06T05:28:05.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.377+0000 7f3b9c7df700 1 -- start start 2023-10-06T05:28:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b9c7df700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3b940677e0 0x7f3b940c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b9c7df700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b94067bb0 0x7f3b940c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b9c7df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 0x7f3b941404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b9c7df700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b940ca7f0 con 0x7f3b94067bb0 2023-10-06T05:28:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b9c7df700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b940ccdf0 con 0x7f3b940677e0 2023-10-06T05:28:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b9c7df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940c9f50 con 0x7f3b940ad1f0 2023-10-06T05:28:05.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b99d7a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b94067bb0 0x7f3b940c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b99d7a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b94067bb0 0x7f3b940c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51596/0 (socket says 172.21.15.45:51596) 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b99d7a700 1 -- 172.21.15.45:0/3525120949 learned_addr learned my addr 172.21.15.45:0/3525120949 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.378+0000 7f3b9a57b700 1 --2- 172.21.15.45:0/3525120949 >> v2:172.21.15.144:3300/0 conn(0x7f3b940677e0 0x7f3b940c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b9ad7c700 1 --2- 172.21.15.45:0/3525120949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 0x7f3b941404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99d7a700 1 -- 172.21.15.45:0/3525120949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 msgr2=0x7f3b941404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99d7a700 1 --2- 172.21.15.45:0/3525120949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 0x7f3b941404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99d7a700 1 -- 172.21.15.45:0/3525120949 >> v2:172.21.15.144:3300/0 conn(0x7f3b940677e0 msgr2=0x7f3b940c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99d7a700 1 --2- 172.21.15.45:0/3525120949 >> v2:172.21.15.144:3300/0 conn(0x7f3b940677e0 0x7f3b940c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99d7a700 1 -- 172.21.15.45:0/3525120949 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940ca230 con 0x7f3b94067bb0 2023-10-06T05:28:05.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99d7a700 1 --2- 172.21.15.45:0/3525120949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b940c9720 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f3b840099b0 tx=0x7f3b84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1266d557f11b1d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99579700 1 -- 172.21.15.45:0/3525120949 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400b820 con 0x7f3b94067bb0 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99579700 1 -- 172.21.15.45:0/3525120949 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8400b980 con 0x7f3b94067bb0 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.379+0000 7f3b99579700 1 -- 172.21.15.45:0/3525120949 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400bd40 con 0x7f3b94067bb0 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 -- 172.21.15.45:0/3525120949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 msgr2=0x7f3b940c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 --2- 172.21.15.45:0/3525120949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b940c9720 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f3b840099b0 tx=0x7f3b84009250 comp rx=0 tx=0).stop 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 -- 172.21.15.45:0/3525120949 shutdown_connections 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 --2- 172.21.15.45:0/3525120949 >> v2:172.21.15.144:3300/0 conn(0x7f3b940677e0 0x7f3b940c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 --2- 172.21.15.45:0/3525120949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 0x7f3b941404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 --2- 172.21.15.45:0/3525120949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b940c9720 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 -- 172.21.15.45:0/3525120949 >> 172.21.15.45:0/3525120949 conn(0x7f3b94068730 msgr2=0x7f3b94068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 -- 172.21.15.45:0/3525120949 shutdown_connections 2023-10-06T05:28:05.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.380+0000 7f3b9c7df700 1 -- 172.21.15.45:0/3525120949 wait complete. 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.381+0000 7f3b9c7df700 1 Processor -- start 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.381+0000 7f3b9c7df700 1 -- start start 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9c7df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b940677e0 0x7f3b9413d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9c7df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b9415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9c7df700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 0x7f3b941591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9c7df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940c5a50 con 0x7f3b94067bb0 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9c7df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940c49b0 con 0x7f3b940677e0 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9c7df700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940c4c90 con 0x7f3b940ad1f0 2023-10-06T05:28:05.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b9415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b9415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51598/0 (socket says 172.21.15.45:51598) 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 -- 172.21.15.45:0/764334837 learned_addr learned my addr 172.21.15.45:0/764334837 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9a57b700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b940677e0 0x7f3b9413d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b9ad7c700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 0x7f3b941591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 -- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 msgr2=0x7f3b941591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b940ad1f0 0x7f3b941591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 -- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b940677e0 msgr2=0x7f3b9413d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b940677e0 0x7f3b9413d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 -- 172.21.15.45:0/764334837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840094f0 con 0x7f3b94067bb0 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b99d7a700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b9415b380 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f3b9406c1e0 tx=0x7f3b84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.382+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401f070 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.383+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b84002f30 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.383+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b940c4f70 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.383+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84011b90 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.383+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b940c5530 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.383+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b9404c380 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.384+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b84023070 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.384+0000 7f3b8a7fc700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3b78066f30 0x7f3b780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.384+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f3b84085fd0 con 0x7f3b94067bb0 2023-10-06T05:28:05.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.385+0000 7f3b8a7fc700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f3b7806d8f0 0x7f3b7806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.385+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b780703f0 con 0x7f3b7806d8f0 2023-10-06T05:28:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.385+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b840862c0 con 0x7f3b94067bb0 2023-10-06T05:28:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.385+0000 7f3b9a57b700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3b78066f30 0x7f3b780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.385+0000 7f3b9ad7c700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f3b7806d8f0 0x7f3b7806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.385+0000 7f3b9ad7c700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f3b7806d8f0 0x7f3b7806fd20 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.385+0000 7f3b9a57b700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3b78066f30 0x7f3b780693e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3b90006fd0 tx=0x7f3b90008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.386+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== osd.1 v2:172.21.15.45:6800/119738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3b780703f0 con 0x7f3b7806d8f0 2023-10-06T05:28:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.400+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 --> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3b9406a170 con 0x7f3b7806d8f0 2023-10-06T05:28:05.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/764334837 <== osd.1 v2:172.21.15.45:6800/119738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3b9406a170 con 0x7f3b7806d8f0 2023-10-06T05:28:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:05.403 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f3b7806d8f0 msgr2=0x7f3b7806fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:6800/119738,v1:172.21.15.45:6801/119738] conn(0x7f3b7806d8f0 0x7f3b7806fd20 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3b78066f30 msgr2=0x7f3b780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3b78066f30 0x7f3b780693e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3b90006fd0 tx=0x7f3b90008040 comp rx=0 tx=0).stop 2023-10-06T05:28:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 msgr2=0x7f3b9415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 --2- 172.21.15.45:0/764334837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b94067bb0 0x7f3b9415b380 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f3b9406c1e0 tx=0x7f3b84017000 comp rx=0 tx=0).stop 2023-10-06T05:28:05.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9ad7c700 1 -- 172.21.15.45:0/764334837 reap_dead start 2023-10-06T05:28:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 shutdown_connections 2023-10-06T05:28:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 >> 172.21.15.45:0/764334837 conn(0x7f3b94068730 msgr2=0x7f3b940c1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 shutdown_connections 2023-10-06T05:28:05.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.401+0000 7f3b9c7df700 1 -- 172.21.15.45:0/764334837 wait complete. 2023-10-06T05:28:05.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T05:28:05.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.547+0000 7fe8cfb01700 1 Processor -- start 2023-10-06T05:28:05.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.547+0000 7fe8cfb01700 1 -- start start 2023-10-06T05:28:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cfb01700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c8159200 0x7fe8c80c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cfb01700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8c81595d0 0x7fe8c806b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cfb01700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c80c7d00 con 0x7fe8c806b5c0 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cfb01700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c80c7780 con 0x7fe8c81595d0 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cfb01700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80c86d0 con 0x7fe8c8159200 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8c81595d0 0x7fe8c806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8c81595d0 0x7fe8c806b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51612/0 (socket says 172.21.15.45:51612) 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 -- 172.21.15.45:0/3995637591 learned_addr learned my addr 172.21.15.45:0/3995637591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd89d700 1 --2- 172.21.15.45:0/3995637591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c8159200 0x7fe8c80c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 -- 172.21.15.45:0/3995637591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c8159200 msgr2=0x7fe8c80c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 --2- 172.21.15.45:0/3995637591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c8159200 0x7fe8c80c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 -- 172.21.15.45:0/3995637591 >> v1:172.21.15.144:6789/0 conn(0x7fe8c806b5c0 legacy=0x7fe8c80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 -- 172.21.15.45:0/3995637591 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c806bb40 con 0x7fe8c81595d0 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cd09c700 1 --2- 172.21.15.45:0/3995637591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c81595d0 0x7fe8c806b080 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80099b0 tx=0x7fe8b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83437c217a3c1ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.548+0000 7fe8cc89b700 1 -- 172.21.15.45:0/3995637591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800b820 con 0x7fe8c81595d0 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cc89b700 1 -- 172.21.15.45:0/3995637591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b800b980 con 0x7fe8c81595d0 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cc89b700 1 -- 172.21.15.45:0/3995637591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800bd20 con 0x7fe8c81595d0 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 -- 172.21.15.45:0/3995637591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c81595d0 msgr2=0x7fe8c806b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/3995637591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c81595d0 0x7fe8c806b080 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80099b0 tx=0x7fe8b8009250 comp rx=0 tx=0).stop 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 -- 172.21.15.45:0/3995637591 shutdown_connections 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/3995637591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c8159200 0x7fe8c80c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/3995637591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c81595d0 0x7fe8c806b080 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 -- 172.21.15.45:0/3995637591 >> 172.21.15.45:0/3995637591 conn(0x7fe8c806f090 msgr2=0x7fe8c806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 -- 172.21.15.45:0/3995637591 shutdown_connections 2023-10-06T05:28:05.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.549+0000 7fe8cfb01700 1 -- 172.21.15.45:0/3995637591 wait complete. 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.551+0000 7fe8cfb01700 1 Processor -- start 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.551+0000 7fe8cfb01700 1 -- start start 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cfb01700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8c806b5c0 0x7fe8c8169400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cfb01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8159200 0x7fe8c80b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cfb01700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c81595d0 0x7fe8c80afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cfb01700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80c43c0 con 0x7fe8c8159200 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cfb01700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80c5ca0 con 0x7fe8c806b5c0 2023-10-06T05:28:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cfb01700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8169940 con 0x7fe8c81595d0 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8159200 0x7fe8c80b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8159200 0x7fe8c80b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51614/0 (socket says 172.21.15.45:51614) 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 -- 172.21.15.45:0/2266294117 learned_addr learned my addr 172.21.15.45:0/2266294117 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd89d700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8c806b5c0 0x7fe8c8169400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8ce09e700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c81595d0 0x7fe8c80afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 -- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c81595d0 msgr2=0x7fe8c80afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c81595d0 0x7fe8c80afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 -- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8c806b5c0 msgr2=0x7fe8c8169400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8c806b5c0 0x7fe8c8169400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd09c700 1 -- 172.21.15.45:0/2266294117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80094f0 con 0x7fe8c8159200 2023-10-06T05:28:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.552+0000 7fe8cd89d700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8c806b5c0 0x7fe8c8169400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.553+0000 7fe8cd09c700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8159200 0x7fe8c80b17b0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009290 tx=0x7fe8b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.553+0000 7fe8bdffb700 1 -- 172.21.15.45:0/2266294117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011d70 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.553+0000 7fe8bdffb700 1 -- 172.21.15.45:0/2266294117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8020070 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.553+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c80c5ef0 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.553+0000 7fe8bdffb700 1 -- 172.21.15.45:0/2266294117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b801b6a0 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.553+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c80c3fe0 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.554+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c80c3e40 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.555+0000 7fe8bdffb700 1 -- 172.21.15.45:0/2266294117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8b8022030 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.555+0000 7fe8bdffb700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8ac066f30 0x7fe8ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.555+0000 7fe8bdffb700 1 -- 172.21.15.45:0/2266294117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fe8b8086490 con 0x7fe8c8159200 2023-10-06T05:28:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.555+0000 7fe8cd89d700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8ac066f30 0x7fe8ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.556+0000 7fe8cd89d700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8ac066f30 0x7fe8ac0693e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400ab70 tx=0x7fe8c4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.560+0000 7fe8bdffb700 1 -- 172.21.15.45:0/2266294117 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8c80c3e40 con 0x7fe8c8159200 2023-10-06T05:28:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.761+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8c80c3e40 con 0x7fe8ac066f30 2023-10-06T05:28:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.762+0000 7fe8bdffb700 1 -- 172.21.15.45:0/2266294117 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe8c80c3e40 con 0x7fe8ac066f30 2023-10-06T05:28:05.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8ac066f30 msgr2=0x7fe8ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8ac066f30 0x7fe8ac0693e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400ab70 tx=0x7fe8c4008040 comp rx=0 tx=0).stop 2023-10-06T05:28:05.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8159200 msgr2=0x7fe8c80b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8159200 0x7fe8c80b17b0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009290 tx=0x7fe8b8016000 comp rx=0 tx=0).stop 2023-10-06T05:28:05.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 shutdown_connections 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fe8ac066f30 0x7fe8ac0693e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8c81595d0 0x7fe8c80afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8c806b5c0 0x7fe8c8169400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 --2- 172.21.15.45:0/2266294117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8159200 0x7fe8c80b17b0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 >> 172.21.15.45:0/2266294117 conn(0x7fe8c806f090 msgr2=0x7fe8c815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 shutdown_connections 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.764+0000 7fe8cfb01700 1 -- 172.21.15.45:0/2266294117 wait complete. 2023-10-06T05:28:05.767 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T05:28:05.779 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T05:28:05.779 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.913+0000 7fcbd8d1c700 1 Processor -- start 2023-10-06T05:28:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.914+0000 7fcbd8d1c700 1 -- start start 2023-10-06T05:28:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd8d1c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbd4159490 0x7fcbd415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd8d1c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd415d3d0 0x7fcbd40c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd8d1c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd4068040 con 0x7fcbd41590c0 2023-10-06T05:28:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd8d1c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd415d850 con 0x7fcbd4159490 2023-10-06T05:28:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd8d1c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415dcf0 con 0x7fcbd415d3d0 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbd4159490 0x7fcbd415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbd4159490 0x7fcbd415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51628/0 (socket says 172.21.15.45:51628) 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 -- 172.21.15.45:0/2357020798 learned_addr learned my addr 172.21.15.45:0/2357020798 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd2d9d700 1 --2- 172.21.15.45:0/2357020798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd415d3d0 0x7fcbd40c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 -- 172.21.15.45:0/2357020798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd415d3d0 msgr2=0x7fcbd40c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 --2- 172.21.15.45:0/2357020798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd415d3d0 0x7fcbd40c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 -- 172.21.15.45:0/2357020798 >> v1:172.21.15.144:6789/0 conn(0x7fcbd41590c0 legacy=0x7fcbd40ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 -- 172.21.15.45:0/2357020798 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4067750 con 0x7fcbd4159490 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd1d9b700 1 --2- 172.21.15.45:0/2357020798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd4159490 0x7fcbd415ce90 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0099b0 tx=0x7fcbbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53025c38340bc0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.915+0000 7fcbd159a700 1 -- 172.21.15.45:0/2357020798 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00b860 con 0x7fcbd4159490 2023-10-06T05:28:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd159a700 1 -- 172.21.15.45:0/2357020798 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc00b9c0 con 0x7fcbd4159490 2023-10-06T05:28:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd159a700 1 -- 172.21.15.45:0/2357020798 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00bd60 con 0x7fcbd4159490 2023-10-06T05:28:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/2357020798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd4159490 msgr2=0x7fcbd415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 --2- 172.21.15.45:0/2357020798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd4159490 0x7fcbd415ce90 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0099b0 tx=0x7fcbbc009250 comp rx=0 tx=0).stop 2023-10-06T05:28:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/2357020798 shutdown_connections 2023-10-06T05:28:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 --2- 172.21.15.45:0/2357020798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd415d3d0 0x7fcbd40c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 --2- 172.21.15.45:0/2357020798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd4159490 0x7fcbd415ce90 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/2357020798 >> 172.21.15.45:0/2357020798 conn(0x7fcbd40cd060 msgr2=0x7fcbd40c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/2357020798 shutdown_connections 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.916+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/2357020798 wait complete. 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.917+0000 7fcbd8d1c700 1 Processor -- start 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.917+0000 7fcbd8d1c700 1 -- start start 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd8d1c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd41590c0 0x7fcbd414b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd8d1c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd4159490 0x7fcbd40af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd8d1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd415d3d0 0x7fcbd40ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd8d1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40afce0 con 0x7fcbd415d3d0 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd8d1c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40affc0 con 0x7fcbd41590c0 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd8d1c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40b02a0 con 0x7fcbd4159490 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd415d3d0 0x7fcbd40ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd415d3d0 0x7fcbd40ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51638/0 (socket says 172.21.15.45:51638) 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd2d9d700 1 -- 172.21.15.45:0/4111808027 learned_addr learned my addr 172.21.15.45:0/4111808027 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd259c700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd41590c0 0x7fcbd414b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.918+0000 7fcbd1d9b700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd4159490 0x7fcbd40af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd2d9d700 1 -- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd4159490 msgr2=0x7fcbd40af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd2d9d700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd4159490 0x7fcbd40af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd2d9d700 1 -- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd41590c0 msgr2=0x7fcbd414b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd2d9d700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd41590c0 0x7fcbd414b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd2d9d700 1 -- 172.21.15.45:0/4111808027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc0094f0 con 0x7fcbd415d3d0 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd2d9d700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd415d3d0 0x7fcbd40ca720 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc007f20 tx=0x7fcbcc009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00eb90 con 0x7fcbd415d3d0 2023-10-06T05:28:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc00ed10 con 0x7fcbd415d3d0 2023-10-06T05:28:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc014670 con 0x7fcbd415d3d0 2023-10-06T05:28:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40b0580 con 0x7fcbd415d3d0 2023-10-06T05:28:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.919+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd414be40 con 0x7fcbd415d3d0 2023-10-06T05:28:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.920+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcbd404c380 con 0x7fcbd415d3d0 2023-10-06T05:28:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.920+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbcc0149e0 con 0x7fcbd415d3d0 2023-10-06T05:28:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.921+0000 7fcbca7fc700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbb4066f30 0x7fcbb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.921+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fcbcc084b30 con 0x7fcbd415d3d0 2023-10-06T05:28:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.921+0000 7fcbca7fc700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7fcbb406d8f0 0x7fcbb406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.921+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbb40703f0 con 0x7fcbb406d8f0 2023-10-06T05:28:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.921+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbcc089240 con 0x7fcbd415d3d0 2023-10-06T05:28:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.921+0000 7fcbd259c700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbb4066f30 0x7fcbb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.921+0000 7fcbd1d9b700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7fcbb406d8f0 0x7fcbb406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.922+0000 7fcbd1d9b700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7fcbb406d8f0 0x7fcbb406fd20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.922+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== osd.2 v2:172.21.15.45:6808/119739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fcbb40703f0 con 0x7fcbb406d8f0 2023-10-06T05:28:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.923+0000 7fcbd259c700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbb4066f30 0x7fcbb40693e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4003a80 tx=0x7fcbc4007ec0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:05.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.937+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 --> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcbd40c9a90 con 0x7fcbb406d8f0 2023-10-06T05:28:05.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbca7fc700 1 -- 172.21.15.45:0/4111808027 <== osd.2 v2:172.21.15.45:6808/119739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcbd40c9a90 con 0x7fcbb406d8f0 2023-10-06T05:28:05.940 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:05.940 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:05.940 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7fcbb406d8f0 msgr2=0x7fcbb406fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:6808/119739,v1:172.21.15.45:6809/119739] conn(0x7fcbb406d8f0 0x7fcbb406fd20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbb4066f30 msgr2=0x7fcbb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcbb4066f30 0x7fcbb40693e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4003a80 tx=0x7fcbc4007ec0 comp rx=0 tx=0).stop 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd415d3d0 msgr2=0x7fcbd40ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 --2- 172.21.15.45:0/4111808027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbd415d3d0 0x7fcbd40ca720 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc007f20 tx=0x7fcbcc009860 comp rx=0 tx=0).stop 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd2d9d700 1 -- 172.21.15.45:0/4111808027 reap_dead start 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 shutdown_connections 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 >> 172.21.15.45:0/4111808027 conn(0x7fcbd40cd060 msgr2=0x7fcbd415c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:05.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 shutdown_connections 2023-10-06T05:28:05.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:05.938+0000 7fcbd8d1c700 1 -- 172.21.15.45:0/4111808027 wait complete. 2023-10-06T05:28:05.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T05:28:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.085+0000 7fcb71b23700 1 Processor -- start 2023-10-06T05:28:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.085+0000 7fcb71b23700 1 -- start start 2023-10-06T05:28:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb71b23700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcb6c155cd0 0x7fcb6c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb71b23700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb6c1560a0 0x7fcb6c0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb71b23700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c0ccdc0 0x7fcb6c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb71b23700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c0cd740 con 0x7fcb6c1560a0 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb71b23700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c0ae8d0 con 0x7fcb6c155cd0 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb71b23700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0aebb0 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb6c1560a0 0x7fcb6c0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb6c1560a0 0x7fcb6c0adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51642/0 (socket says 172.21.15.45:51642) 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 -- 172.21.15.45:0/4062452478 learned_addr learned my addr 172.21.15.45:0/4062452478 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6b7fe700 1 --2- 172.21.15.45:0/4062452478 >> v2:172.21.15.144:3300/0 conn(0x7fcb6c155cd0 0x7fcb6c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6bfff700 1 --2- 172.21.15.45:0/4062452478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c0ccdc0 0x7fcb6c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 -- 172.21.15.45:0/4062452478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c0ccdc0 msgr2=0x7fcb6c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 --2- 172.21.15.45:0/4062452478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c0ccdc0 0x7fcb6c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 -- 172.21.15.45:0/4062452478 >> v2:172.21.15.144:3300/0 conn(0x7fcb6c155cd0 msgr2=0x7fcb6c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 --2- 172.21.15.45:0/4062452478 >> v2:172.21.15.144:3300/0 conn(0x7fcb6c155cd0 0x7fcb6c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.086+0000 7fcb6affd700 1 -- 172.21.15.45:0/4062452478 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c0aee90 con 0x7fcb6c1560a0 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb6affd700 1 --2- 172.21.15.45:0/4062452478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c1560a0 0x7fcb6c0adfa0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c06ae50 tx=0x7fcb5c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2011a1c84b0e584 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb6a7fc700 1 -- 172.21.15.45:0/4062452478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00dc50 con 0x7fcb6c1560a0 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb6a7fc700 1 -- 172.21.15.45:0/4062452478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c00ddb0 con 0x7fcb6c1560a0 2023-10-06T05:28:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb6a7fc700 1 -- 172.21.15.45:0/4062452478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0196d0 con 0x7fcb6c1560a0 2023-10-06T05:28:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 -- 172.21.15.45:0/4062452478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c1560a0 msgr2=0x7fcb6c0adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 --2- 172.21.15.45:0/4062452478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c1560a0 0x7fcb6c0adfa0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c06ae50 tx=0x7fcb5c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 -- 172.21.15.45:0/4062452478 shutdown_connections 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 --2- 172.21.15.45:0/4062452478 >> v2:172.21.15.144:3300/0 conn(0x7fcb6c155cd0 0x7fcb6c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 --2- 172.21.15.45:0/4062452478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c0ccdc0 0x7fcb6c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 --2- 172.21.15.45:0/4062452478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c1560a0 0x7fcb6c0adfa0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 -- 172.21.15.45:0/4062452478 >> 172.21.15.45:0/4062452478 conn(0x7fcb6c06f090 msgr2=0x7fcb6c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 -- 172.21.15.45:0/4062452478 shutdown_connections 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.087+0000 7fcb71b23700 1 -- 172.21.15.45:0/4062452478 wait complete. 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.089+0000 7fcb71b23700 1 Processor -- start 2023-10-06T05:28:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.089+0000 7fcb71b23700 1 -- start start 2023-10-06T05:28:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb71b23700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c0ccdc0 0x7fcb6c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb71b23700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c155cd0 0x7fcb6c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb71b23700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c1560a0 0x7fcb6c153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb71b23700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c164f60 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb71b23700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c165a40 con 0x7fcb6c155cd0 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb71b23700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c152b20 con 0x7fcb6c1560a0 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c0ccdc0 0x7fcb6c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c0ccdc0 0x7fcb6c1525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51652/0 (socket says 172.21.15.45:51652) 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.090+0000 7fcb6b7fe700 1 -- 172.21.15.45:0/541088317 learned_addr learned my addr 172.21.15.45:0/541088317 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6affd700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c155cd0 0x7fcb6c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6bfff700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c1560a0 0x7fcb6c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6b7fe700 1 -- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c1560a0 msgr2=0x7fcb6c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6b7fe700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c1560a0 0x7fcb6c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6b7fe700 1 -- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c155cd0 msgr2=0x7fcb6c152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6b7fe700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c155cd0 0x7fcb6c152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6b7fe700 1 -- 172.21.15.45:0/541088317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c00b790 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6affd700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c155cd0 0x7fcb6c152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb6b7fe700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c0ccdc0 0x7fcb6c1525e0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000d8c0 tx=0x7fcb60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb537fe700 1 -- 172.21.15.45:0/541088317 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000de60 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb537fe700 1 -- 172.21.15.45:0/541088317 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6000b780 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c165150 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb537fe700 1 -- 172.21.15.45:0/541088317 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000bb20 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.091+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c157500 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.092+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c165d20 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.093+0000 7fcb537fe700 1 -- 172.21.15.45:0/541088317 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb6001e460 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.093+0000 7fcb537fe700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb58066f30 0x7fcb580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.093+0000 7fcb537fe700 1 -- 172.21.15.45:0/541088317 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fcb60084bb0 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.093+0000 7fcb6affd700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb58066f30 0x7fcb580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.094+0000 7fcb6affd700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb58066f30 0x7fcb580693e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c06ae50 tx=0x7fcb5c01b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.098+0000 7fcb537fe700 1 -- 172.21.15.45:0/541088317 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb60051e20 con 0x7fcb6c0ccdc0 2023-10-06T05:28:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.299+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb6c0af170 con 0x7fcb58066f30 2023-10-06T05:28:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.300+0000 7fcb537fe700 1 -- 172.21.15.45:0/541088317 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcb6c0af170 con 0x7fcb58066f30 2023-10-06T05:28:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb58066f30 msgr2=0x7fcb580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb58066f30 0x7fcb580693e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c06ae50 tx=0x7fcb5c01b000 comp rx=0 tx=0).stop 2023-10-06T05:28:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c0ccdc0 msgr2=0x7fcb6c1525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c0ccdc0 0x7fcb6c1525e0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000d8c0 tx=0x7fcb60008040 comp rx=0 tx=0).stop 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 shutdown_connections 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcb58066f30 0x7fcb580693e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb6c1560a0 0x7fcb6c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb6c0ccdc0 0x7fcb6c1525e0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 --2- 172.21.15.45:0/541088317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c155cd0 0x7fcb6c152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 >> 172.21.15.45:0/541088317 conn(0x7fcb6c06f090 msgr2=0x7fcb6c154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 shutdown_connections 2023-10-06T05:28:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.302+0000 7fcb71b23700 1 -- 172.21.15.45:0/541088317 wait complete. 2023-10-06T05:28:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T05:28:06.317 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T05:28:06.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.452+0000 7fbd9d9e5700 1 Processor -- start 2023-10-06T05:28:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.452+0000 7fbd9d9e5700 1 -- start start 2023-10-06T05:28:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd9d9e5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd9815cb80 0x7fbd980cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd9d9e5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd980b0140 con 0x7fbd980afcd0 2023-10-06T05:28:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd9d9e5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd980b0420 con 0x7fbd980b2340 2023-10-06T05:28:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd9d9e5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd980b0700 con 0x7fbd9815cb80 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd977fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd9815cb80 0x7fbd980cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd967fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbd980afcd0 0x7fbd980cabb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50004/0 (socket says 172.21.15.45:50004) 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd977fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd9815cb80 0x7fbd980cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33366/0 (socket says 172.21.15.45:33366) 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.453+0000 7fbd967fc700 1 -- 172.21.15.45:0/2320468006 learned_addr learned my addr 172.21.15.45:0/2320468006 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd977fe700 1 -- 172.21.15.45:0/2320468006 >> v1:172.21.15.144:6789/0 conn(0x7fbd980b2340 legacy=0x7fbd9815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd977fe700 1 -- 172.21.15.45:0/2320468006 >> v1:172.21.15.45:6789/0 conn(0x7fbd980afcd0 legacy=0x7fbd980cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd977fe700 1 -- 172.21.15.45:0/2320468006 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980b09e0 con 0x7fbd9815cb80 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd95ffb700 1 -- 172.21.15.45:0/2320468006 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504781633 0 0) 0x7fbd980b0140 con 0x7fbd980afcd0 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd977fe700 1 --2- 172.21.15.45:0/2320468006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd9815cb80 0x7fbd980cc390 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009e60 tx=0x7fbd90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0a0a430a2aa019 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd95ffb700 1 -- 172.21.15.45:0/2320468006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000e070 con 0x7fbd9815cb80 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd95ffb700 1 -- 172.21.15.45:0/2320468006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000bcc0 con 0x7fbd9815cb80 2023-10-06T05:28:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd95ffb700 1 -- 172.21.15.45:0/2320468006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90012680 con 0x7fbd9815cb80 2023-10-06T05:28:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/2320468006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd9815cb80 msgr2=0x7fbd980cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.454+0000 7fbd9d9e5700 1 --2- 172.21.15.45:0/2320468006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd9815cb80 0x7fbd980cc390 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009e60 tx=0x7fbd90009780 comp rx=0 tx=0).stop 2023-10-06T05:28:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.455+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/2320468006 shutdown_connections 2023-10-06T05:28:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.455+0000 7fbd9d9e5700 1 --2- 172.21.15.45:0/2320468006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd9815cb80 0x7fbd980cc390 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.455+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/2320468006 >> 172.21.15.45:0/2320468006 conn(0x7fbd9806f820 msgr2=0x7fbd980d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.455+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/2320468006 shutdown_connections 2023-10-06T05:28:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.455+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/2320468006 wait complete. 2023-10-06T05:28:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.456+0000 7fbd9d9e5700 1 Processor -- start 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.456+0000 7fbd9d9e5700 1 -- start start 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd9d9e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd980afcd0 0x7fbd98152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd9d9e5700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd980b2340 0x7fbd9815f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd9d9e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd9815cb80 0x7fbd98158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd9d9e5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98152a80 con 0x7fbd980afcd0 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd9d9e5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9815f0f0 con 0x7fbd9815cb80 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd9d9e5700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9815e200 con 0x7fbd980b2340 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd980afcd0 0x7fbd98152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd980afcd0 0x7fbd98152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33376/0 (socket says 172.21.15.45:33376) 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 -- 172.21.15.45:0/3781381718 learned_addr learned my addr 172.21.15.45:0/3781381718 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd977fe700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd9815cb80 0x7fbd98158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd967fc700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd980b2340 0x7fbd9815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 -- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd980b2340 msgr2=0x7fbd9815f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd980b2340 0x7fbd9815f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 -- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd9815cb80 msgr2=0x7fbd98158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd9815cb80 0x7fbd98158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 -- 172.21.15.45:0/3781381718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90009a20 con 0x7fbd980afcd0 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd977fe700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd9815cb80 0x7fbd98158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.457+0000 7fbd96ffd700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd980afcd0 0x7fbd98152540 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c590 tx=0x7fbd8800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.458+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011590 con 0x7fbd980afcd0 2023-10-06T05:28:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.458+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd880116f0 con 0x7fbd980afcd0 2023-10-06T05:28:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.458+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9815e4e0 con 0x7fbd980afcd0 2023-10-06T05:28:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.458+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011ab0 con 0x7fbd980afcd0 2023-10-06T05:28:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.458+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9815ed80 con 0x7fbd980afcd0 2023-10-06T05:28:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.458+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd9815e8d0 con 0x7fbd980afcd0 2023-10-06T05:28:06.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.459+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd8801b3b0 con 0x7fbd980afcd0 2023-10-06T05:28:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.459+0000 7fbd8effd700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd78066f60 0x7fbd78069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fbd88083ae0 con 0x7fbd980afcd0 2023-10-06T05:28:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd8effd700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fbd7806d920 0x7fbd7806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd78070420 con 0x7fbd7806d920 2023-10-06T05:28:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd88085ea0 con 0x7fbd980afcd0 2023-10-06T05:28:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd967fc700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd78066f60 0x7fbd78069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd977fe700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fbd7806d920 0x7fbd7806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd977fe700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fbd7806d920 0x7fbd7806fd50 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.460+0000 7fbd967fc700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd78066f60 0x7fbd78069410 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbd80007fd0 tx=0x7fbd80009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.461+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== osd.3 v2:172.21.15.45:6824/119762 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fbd78070420 con 0x7fbd7806d920 2023-10-06T05:28:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.475+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 --> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbd9804c380 con 0x7fbd7806d920 2023-10-06T05:28:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd8effd700 1 -- 172.21.15.45:0/3781381718 <== osd.3 v2:172.21.15.45:6824/119762 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbd9804c380 con 0x7fbd7806d920 2023-10-06T05:28:06.478 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:06.478 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fbd7806d920 msgr2=0x7fbd7806fd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:6824/119762,v1:172.21.15.45:6825/119762] conn(0x7fbd7806d920 0x7fbd7806fd50 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd78066f60 msgr2=0x7fbd78069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fbd78066f60 0x7fbd78069410 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbd80007fd0 tx=0x7fbd80009040 comp rx=0 tx=0).stop 2023-10-06T05:28:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd980afcd0 msgr2=0x7fbd98152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 --2- 172.21.15.45:0/3781381718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd980afcd0 0x7fbd98152540 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c590 tx=0x7fbd8800c950 comp rx=0 tx=0).stop 2023-10-06T05:28:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd977fe700 1 -- 172.21.15.45:0/3781381718 reap_dead start 2023-10-06T05:28:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 shutdown_connections 2023-10-06T05:28:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 >> 172.21.15.45:0/3781381718 conn(0x7fbd9806f820 msgr2=0x7fbd980d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 shutdown_connections 2023-10-06T05:28:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.476+0000 7fbd9d9e5700 1 -- 172.21.15.45:0/3781381718 wait complete. 2023-10-06T05:28:06.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T05:28:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.622+0000 7fcd90918700 1 Processor -- start 2023-10-06T05:28:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.622+0000 7fcd90918700 1 -- start start 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd90918700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 0x7fcd8c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd90918700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd8c0ccdc0 0x7fcd8c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd90918700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0cd740 con 0x7fcd8c155cd0 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd90918700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c0ae6b0 con 0x7fcd8c0ccdc0 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd90918700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0ae8d0 con 0x7fcd8c1560a0 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd8a59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcd8c155cd0 0x7fcd8c0adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50018/0 (socket says 172.21.15.45:50018) 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd8a59c700 1 -- 172.21.15.45:0/3639079776 learned_addr learned my addr 172.21.15.45:0/3639079776 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8ad9d700 1 --2- 172.21.15.45:0/3639079776 >> v2:172.21.15.144:3300/0 conn(0x7fcd8c0ccdc0 0x7fcd8c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.623+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136741666 0 0) 0x7fcd8c0cd740 con 0x7fcd8c155cd0 2023-10-06T05:28:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 0x7fcd8c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd74003570 con 0x7fcd8c155cd0 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3875342340 0 0) 0x7fcd74003570 con 0x7fcd8c155cd0 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 msgr2=0x7fcd8c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 0x7fcd8c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 >> v2:172.21.15.144:3300/0 conn(0x7fcd8c0ccdc0 msgr2=0x7fcd8c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 --2- 172.21.15.45:0/3639079776 >> v2:172.21.15.144:3300/0 conn(0x7fcd8c0ccdc0 0x7fcd8c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0aebb0 con 0x7fcd8c155cd0 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7fcd7c001470 con 0x7fcd8c155cd0 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 >> v1:172.21.15.45:6789/0 conn(0x7fcd8c155cd0 legacy=0x7fcd8c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 0x7fcd74003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd74004d50 0x7fcd740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0ae6b0 con 0x7fcd740040b0 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0ae8d0 con 0x7fcd74004d50 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7400bb60 con 0x7fcd8c1560a0 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8a59c700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 0x7fcd74003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd8ad9d700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd74004d50 0x7fcd740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd89d9b700 1 -- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 msgr2=0x7fcd74003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c1560a0 0x7fcd74003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd89d9b700 1 -- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd74004d50 msgr2=0x7fcd740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd74004d50 0x7fcd740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.624+0000 7fcd89d9b700 1 -- 172.21.15.45:0/3639079776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74003570 con 0x7fcd740040b0 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd74004810 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006f00 tx=0x7fcd8000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f146360f2bf916f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80010b30 con 0x7fcd740040b0 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd80010c90 con 0x7fcd740040b0 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd8959a700 1 -- 172.21.15.45:0/3639079776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80011450 con 0x7fcd740040b0 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd90918700 1 -- 172.21.15.45:0/3639079776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 msgr2=0x7fcd74004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd90918700 1 --2- 172.21.15.45:0/3639079776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd74004810 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006f00 tx=0x7fcd8000b430 comp rx=0 tx=0).stop 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd8ad9d700 1 -- 172.21.15.45:0/3639079776 reap_dead start 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd90918700 1 -- 172.21.15.45:0/3639079776 shutdown_connections 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd90918700 1 -- 172.21.15.45:0/3639079776 >> 172.21.15.45:0/3639079776 conn(0x7fcd8c06f090 msgr2=0x7fcd8c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.625+0000 7fcd90918700 1 -- 172.21.15.45:0/3639079776 shutdown_connections 2023-10-06T05:28:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.626+0000 7fcd90918700 1 -- 172.21.15.45:0/3639079776 wait complete. 2023-10-06T05:28:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.627+0000 7fcd90918700 1 Processor -- start 2023-10-06T05:28:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.627+0000 7fcd90918700 1 -- start start 2023-10-06T05:28:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.627+0000 7fcd90918700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd8c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.627+0000 7fcd90918700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c0c8a50 0x7fcd8c0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd90918700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd8c0b1fc0 0x7fcd8c0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd90918700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0c9740 con 0x7fcd740040b0 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd90918700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0c9a20 con 0x7fcd8c0b1fc0 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd90918700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0c9d00 con 0x7fcd8c0c8a50 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd8c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd8c1511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33400/0 (socket says 172.21.15.45:33400) 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 -- 172.21.15.45:0/3469234197 learned_addr learned my addr 172.21.15.45:0/3469234197 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8ad9d700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd8c0b1fc0 0x7fcd8c0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c0c8a50 0x7fcd8c0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 -- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c0c8a50 msgr2=0x7fcd8c0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c0c8a50 0x7fcd8c0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 -- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd8c0b1fc0 msgr2=0x7fcd8c0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd8c0b1fc0 0x7fcd8c0b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 -- 172.21.15.45:0/3469234197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0c9fe0 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c0c8a50 0x7fcd8c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd8a59c700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd8c1511f0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0073c0 tx=0x7fcd7c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd7a7fc700 1 -- 172.21.15.45:0/3469234197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c018030 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd7a7fc700 1 -- 172.21.15.45:0/3469234197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c007780 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c0ca2c0 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd7a7fc700 1 -- 172.21.15.45:0/3469234197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00fa80 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.628+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c0b2950 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.629+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c06e2f0 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.630+0000 7fcd7a7fc700 1 -- 172.21.15.45:0/3469234197 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd7c01b070 con 0x7fcd740040b0 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.630+0000 7fcd7a7fc700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd6c066f30 0x7fcd6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.630+0000 7fcd7a7fc700 1 -- 172.21.15.45:0/3469234197 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fcd7c019070 con 0x7fcd740040b0 2023-10-06T05:28:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.631+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd6c066f30 0x7fcd6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.631+0000 7fcd89d9b700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd6c066f30 0x7fcd6c0693e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006f00 tx=0x7fcd8001c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.635+0000 7fcd7a7fc700 1 -- 172.21.15.45:0/3469234197 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd7c0515c0 con 0x7fcd740040b0 2023-10-06T05:28:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.836+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd8c0aee90 con 0x7fcd6c066f30 2023-10-06T05:28:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.836+0000 7fcd7a7fc700 1 -- 172.21.15.45:0/3469234197 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcd8c0aee90 con 0x7fcd6c066f30 2023-10-06T05:28:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.838+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd6c066f30 msgr2=0x7fcd6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.838+0000 7fcd90918700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd6c066f30 0x7fcd6c0693e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006f00 tx=0x7fcd8001c000 comp rx=0 tx=0).stop 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.838+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 msgr2=0x7fcd8c1511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.838+0000 7fcd90918700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd8c1511f0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0073c0 tx=0x7fcd7c001dc0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 shutdown_connections 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fcd6c066f30 0x7fcd6c0693e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd8c0b1fc0 0x7fcd8c0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd740040b0 0x7fcd8c1511f0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 --2- 172.21.15.45:0/3469234197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd8c0c8a50 0x7fcd8c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 >> 172.21.15.45:0/3469234197 conn(0x7fcd8c06f090 msgr2=0x7fcd8c0ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 shutdown_connections 2023-10-06T05:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.839+0000 7fcd90918700 1 -- 172.21.15.45:0/3469234197 wait complete. 2023-10-06T05:28:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T05:28:06.854 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T05:28:06.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.988+0000 7fda9bbd2700 1 Processor -- start 2023-10-06T05:28:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.988+0000 7fda9bbd2700 1 -- start start 2023-10-06T05:28:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.989+0000 7fda9bbd2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda940caa90 0x7fda940cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.989+0000 7fda9bbd2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940ca020 0x7fda940c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.989+0000 7fda9bbd2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda94161ce0 con 0x7fda940cba10 2023-10-06T05:28:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.989+0000 7fda9bbd2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda94161af0 con 0x7fda940caa90 2023-10-06T05:28:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.989+0000 7fda9bbd2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda94161fc0 con 0x7fda940ca020 2023-10-06T05:28:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9a16f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940ca020 0x7fda940c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9916d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda940caa90 0x7fda940cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9996e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fda940cba10 0x7fda941613e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:60538/0 (socket says 172.21.15.45:60538) 2023-10-06T05:28:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9996e700 1 -- 172.21.15.45:0/3503952021 learned_addr learned my addr 172.21.15.45:0/3503952021 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9a16f700 1 -- 172.21.15.45:0/3503952021 >> v2:172.21.15.144:3300/0 conn(0x7fda940caa90 msgr2=0x7fda940cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9a16f700 1 --2- 172.21.15.45:0/3503952021 >> v2:172.21.15.144:3300/0 conn(0x7fda940caa90 0x7fda940cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9a16f700 1 -- 172.21.15.45:0/3503952021 >> v1:172.21.15.144:6789/0 conn(0x7fda940cba10 legacy=0x7fda941613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9a16f700 1 -- 172.21.15.45:0/3503952021 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda941622a0 con 0x7fda940ca020 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9916d700 1 --2- 172.21.15.45:0/3503952021 >> v2:172.21.15.144:3300/0 conn(0x7fda940caa90 0x7fda940cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9a16f700 1 --2- 172.21.15.45:0/3503952021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940ca020 0x7fda940c7fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009e60 tx=0x7fda8c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7734c609f69a1f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.990+0000 7fda9896c700 1 -- 172.21.15.45:0/3503952021 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00e070 con 0x7fda940ca020 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9896c700 1 -- 172.21.15.45:0/3503952021 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c00bb60 con 0x7fda940ca020 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9896c700 1 -- 172.21.15.45:0/3503952021 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c012430 con 0x7fda940ca020 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 -- 172.21.15.45:0/3503952021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940ca020 msgr2=0x7fda940c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 --2- 172.21.15.45:0/3503952021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940ca020 0x7fda940c7fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009e60 tx=0x7fda8c009780 comp rx=0 tx=0).stop 2023-10-06T05:28:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 -- 172.21.15.45:0/3503952021 shutdown_connections 2023-10-06T05:28:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 --2- 172.21.15.45:0/3503952021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940ca020 0x7fda940c7fa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 --2- 172.21.15.45:0/3503952021 >> v2:172.21.15.144:3300/0 conn(0x7fda940caa90 0x7fda940cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 -- 172.21.15.45:0/3503952021 >> 172.21.15.45:0/3503952021 conn(0x7fda94068730 msgr2=0x7fda94068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 -- 172.21.15.45:0/3503952021 shutdown_connections 2023-10-06T05:28:06.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.991+0000 7fda9bbd2700 1 -- 172.21.15.45:0/3503952021 wait complete. 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.992+0000 7fda9bbd2700 1 Processor -- start 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.992+0000 7fda9bbd2700 1 -- start start 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9bbd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda940ca020 0x7fda940c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9bbd2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda940caa90 0x7fda9415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9bbd2700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940cba10 0x7fda940c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9bbd2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda940c50f0 con 0x7fda940caa90 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9bbd2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda940c69e0 con 0x7fda940ca020 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9bbd2700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda940c44e0 con 0x7fda940cba10 2023-10-06T05:28:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9916d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda940caa90 0x7fda9415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9916d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda940caa90 0x7fda9415d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33408/0 (socket says 172.21.15.45:33408) 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.993+0000 7fda9916d700 1 -- 172.21.15.45:0/2127768146 learned_addr learned my addr 172.21.15.45:0/2127768146 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9a16f700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940cba10 0x7fda940c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9996e700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda940ca020 0x7fda940c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9916d700 1 -- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940cba10 msgr2=0x7fda940c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9916d700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda940cba10 0x7fda940c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9916d700 1 -- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda940ca020 msgr2=0x7fda940c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9916d700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda940ca020 0x7fda940c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9916d700 1 -- 172.21.15.45:0/2127768146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90009420 con 0x7fda940caa90 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9916d700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda940caa90 0x7fda9415d4d0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fda9000ec80 tx=0x7fda900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000c9d0 con 0x7fda940caa90 2023-10-06T05:28:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9000cb30 con 0x7fda940caa90 2023-10-06T05:28:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8c009a20 con 0x7fda940caa90 2023-10-06T05:28:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000ceb0 con 0x7fda940caa90 2023-10-06T05:28:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.994+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda940c4aa0 con 0x7fda940caa90 2023-10-06T05:28:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.995+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda9404c380 con 0x7fda940caa90 2023-10-06T05:28:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.996+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda90013680 con 0x7fda940caa90 2023-10-06T05:28:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.996+0000 7fda85ffb700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda78066f30 0x7fda780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.996+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fda90084e10 con 0x7fda940caa90 2023-10-06T05:28:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.997+0000 7fda85ffb700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda7806d8f0 0x7fda7806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.997+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda780703f0 con 0x7fda7806d8f0 2023-10-06T05:28:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.997+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda90085100 con 0x7fda940caa90 2023-10-06T05:28:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.997+0000 7fda9996e700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda78066f30 0x7fda780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.997+0000 7fda9a16f700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda7806d8f0 0x7fda7806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.997+0000 7fda9996e700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda78066f30 0x7fda780693e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fda88001780 tx=0x7fda88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.997+0000 7fda9a16f700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda7806d8f0 0x7fda7806fd20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:06.998+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== osd.4 v2:172.21.15.144:6824/119910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fda780703f0 con 0x7fda7806d8f0 2023-10-06T05:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.013+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 --> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fda940c9820 con 0x7fda7806d8f0 2023-10-06T05:28:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.013+0000 7fda85ffb700 1 -- 172.21.15.45:0/2127768146 <== osd.4 v2:172.21.15.144:6824/119910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fda940c9820 con 0x7fda7806d8f0 2023-10-06T05:28:07.015 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:07.015 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:07.015 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda7806d8f0 msgr2=0x7fda7806fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6824/119910,v1:172.21.15.144:6825/119910] conn(0x7fda7806d8f0 0x7fda7806fd20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda78066f30 msgr2=0x7fda780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fda78066f30 0x7fda780693e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fda88001780 tx=0x7fda88009040 comp rx=0 tx=0).stop 2023-10-06T05:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda940caa90 msgr2=0x7fda9415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 --2- 172.21.15.45:0/2127768146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda940caa90 0x7fda9415d4d0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fda9000ec80 tx=0x7fda900091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9a16f700 1 -- 172.21.15.45:0/2127768146 reap_dead start 2023-10-06T05:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 shutdown_connections 2023-10-06T05:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 >> 172.21.15.45:0/2127768146 conn(0x7fda94068730 msgr2=0x7fda94068b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 shutdown_connections 2023-10-06T05:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.014+0000 7fda9bbd2700 1 -- 172.21.15.45:0/2127768146 wait complete. 2023-10-06T05:28:07.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T05:28:07.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.160+0000 7f6e04412700 1 Processor -- start 2023-10-06T05:28:07.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.160+0000 7f6e04412700 1 -- start start 2023-10-06T05:28:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e04412700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 0x7f6dfc159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e04412700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0ca6d0 con 0x7f6dfc159170 2023-10-06T05:28:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e04412700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc15d550 con 0x7f6dfc15d040 2023-10-06T05:28:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e04412700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15d810 con 0x7f6dfc0692d0 2023-10-06T05:28:07.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e021ae700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6dfc159170 0x7f6dfc068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50022/0 (socket says 172.21.15.45:50022) 2023-10-06T05:28:07.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e021ae700 1 -- 172.21.15.45:0/3451977848 learned_addr learned my addr 172.21.15.45:0/3451977848 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:07.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e019ad700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 0x7f6dfc159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.161+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983700921 0 0) 0x7f6dfc0ca6d0 con 0x7f6dfc159170 2023-10-06T05:28:07.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de8003570 con 0x7f6dfc159170 2023-10-06T05:28:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2678906949 0 0) 0x7f6de8003570 con 0x7f6dfc159170 2023-10-06T05:28:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 msgr2=0x7f6dfc159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 0x7f6dfc159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 >> v1:172.21.15.144:6789/0 conn(0x7f6dfc15d040 legacy=0x7f6dfc0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc15daf0 con 0x7f6dfc159170 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7f6df8001470 con 0x7f6dfc159170 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 >> v1:172.21.15.45:6789/0 conn(0x7f6dfc159170 legacy=0x7f6dfc068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6de8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 0x7f6de8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6de8004d50 0x7f6de80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15d810 con 0x7f6de8003a90 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de800bb10 con 0x7f6de8004d50 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de800bcd0 con 0x7f6dfc0692d0 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e021ae700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6de8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e019ad700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 0x7f6de8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e029af700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6de8004d50 0x7f6de80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e021ae700 1 -- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 msgr2=0x7f6de8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e021ae700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0692d0 0x7f6de8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e021ae700 1 -- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6de8004d50 msgr2=0x7f6de80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e021ae700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6de8004d50 0x7f6de80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.162+0000 7f6e021ae700 1 -- 172.21.15.45:0/3451977848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de8003570 con 0x7f6de8003a90 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e021ae700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6de8003e60 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c3c0 tx=0x7f6df8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf667434c12d95a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800cc90 con 0x7f6de8003a90 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800ce10 con 0x7f6de8003a90 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e011ac700 1 -- 172.21.15.45:0/3451977848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8005b70 con 0x7f6de8003a90 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e04412700 1 -- 172.21.15.45:0/3451977848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 msgr2=0x7f6de8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e04412700 1 --2- 172.21.15.45:0/3451977848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6de8003e60 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c3c0 tx=0x7f6df8001e70 comp rx=0 tx=0).stop 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e029af700 1 -- 172.21.15.45:0/3451977848 reap_dead start 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e04412700 1 -- 172.21.15.45:0/3451977848 shutdown_connections 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e04412700 1 -- 172.21.15.45:0/3451977848 >> 172.21.15.45:0/3451977848 conn(0x7f6dfc06f7c0 msgr2=0x7f6dfc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e04412700 1 -- 172.21.15.45:0/3451977848 shutdown_connections 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.163+0000 7f6e04412700 1 -- 172.21.15.45:0/3451977848 wait complete. 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 Processor -- start 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 -- start start 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6dfc15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0aec80 0x7f6dfc0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dfc0af0f0 0x7f6dfc0ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15a370 con 0x7f6de8003a90 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15a650 con 0x7f6dfc0af0f0 2023-10-06T05:28:07.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e04412700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15a930 con 0x7f6dfc0aec80 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.165+0000 7f6e021ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6dfc15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6dfc15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33434/0 (socket says 172.21.15.45:33434) 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 -- 172.21.15.45:0/3200527105 learned_addr learned my addr 172.21.15.45:0/3200527105 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e019ad700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0aec80 0x7f6dfc0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e029af700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dfc0af0f0 0x7f6dfc0ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 -- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0aec80 msgr2=0x7f6dfc0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0aec80 0x7f6dfc0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 -- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dfc0af0f0 msgr2=0x7f6dfc0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dfc0af0f0 0x7f6dfc0ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 -- 172.21.15.45:0/3200527105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc15ac10 con 0x7f6de8003a90 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e021ae700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6dfc15b3f0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f6de8004770 tx=0x7f6df8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6dee7fc700 1 -- 172.21.15.45:0/3200527105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800cc90 con 0x7f6de8003a90 2023-10-06T05:28:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6dee7fc700 1 -- 172.21.15.45:0/3200527105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df80070d0 con 0x7f6de8003a90 2023-10-06T05:28:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc15aef0 con 0x7f6de8003a90 2023-10-06T05:28:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6dee7fc700 1 -- 172.21.15.45:0/3200527105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8007470 con 0x7f6de8003a90 2023-10-06T05:28:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.166+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc159860 con 0x7f6de8003a90 2023-10-06T05:28:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.167+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc15b260 con 0x7f6de8003a90 2023-10-06T05:28:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.168+0000 7f6dee7fc700 1 -- 172.21.15.45:0/3200527105 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6df8019030 con 0x7f6de8003a90 2023-10-06T05:28:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.168+0000 7f6dee7fc700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.168+0000 7f6dee7fc700 1 -- 172.21.15.45:0/3200527105 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f6df8085b40 con 0x7f6de8003a90 2023-10-06T05:28:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.168+0000 7f6e019ad700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.169+0000 7f6e019ad700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6de0066f30 0x7f6de00693e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006f00 tx=0x7f6df00091a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.173+0000 7f6dee7fc700 1 -- 172.21.15.45:0/3200527105 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6df8052dc0 con 0x7f6de8003a90 2023-10-06T05:28:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.377+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dfc15a7c0 con 0x7f6de0066f30 2023-10-06T05:28:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.377+0000 7f6dee7fc700 1 -- 172.21.15.45:0/3200527105 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6dfc15a7c0 con 0x7f6de0066f30 2023-10-06T05:28:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6de0066f30 msgr2=0x7f6de00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6de0066f30 0x7f6de00693e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006f00 tx=0x7f6df00091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 msgr2=0x7f6dfc15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6dfc15b3f0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f6de8004770 tx=0x7f6df8015000 comp rx=0 tx=0).stop 2023-10-06T05:28:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 shutdown_connections 2023-10-06T05:28:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f6de0066f30 0x7f6de00693e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dfc0af0f0 0x7f6dfc0ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6de8003a90 0x7f6dfc15b3f0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 --2- 172.21.15.45:0/3200527105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dfc0aec80 0x7f6dfc0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 >> 172.21.15.45:0/3200527105 conn(0x7f6dfc06f7c0 msgr2=0x7f6dfc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 shutdown_connections 2023-10-06T05:28:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.380+0000 7f6e04412700 1 -- 172.21.15.45:0/3200527105 wait complete. 2023-10-06T05:28:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T05:28:07.395 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T05:28:07.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.529+0000 7f3d4727c700 1 Processor -- start 2023-10-06T05:28:07.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.529+0000 7f3d4727c700 1 -- start start 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d4727c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d401535e0 0x7f3d401539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d4727c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d40159f70 0x7f3d400ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d4727c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d400c9230 con 0x7f3d40068730 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d4727c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d400cc4b0 con 0x7f3d40159f70 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d4727c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4015a420 con 0x7f3d401535e0 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d45819700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d40159f70 0x7f3d400ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d44817700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3d40068730 0x7f3d40159860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50026/0 (socket says 172.21.15.45:50026) 2023-10-06T05:28:07.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d45819700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d40159f70 0x7f3d400ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33442/0 (socket says 172.21.15.45:33442) 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.530+0000 7f3d44817700 1 -- 172.21.15.45:0/364981379 learned_addr learned my addr 172.21.15.45:0/364981379 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d45018700 1 --2- 172.21.15.45:0/364981379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d401535e0 0x7f3d401539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d45819700 1 -- 172.21.15.45:0/364981379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d401535e0 msgr2=0x7f3d401539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d45819700 1 --2- 172.21.15.45:0/364981379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d401535e0 0x7f3d401539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d45819700 1 -- 172.21.15.45:0/364981379 >> v1:172.21.15.45:6789/0 conn(0x7f3d40068730 legacy=0x7f3d40159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d33fff700 1 -- 172.21.15.45:0/364981379 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649666923 0 0) 0x7f3d400c9230 con 0x7f3d40068730 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d45819700 1 -- 172.21.15.45:0/364981379 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4015a700 con 0x7f3d40159f70 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d45819700 1 --2- 172.21.15.45:0/364981379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40159f70 0x7f3d400ca770 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009e60 tx=0x7f3d38009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c399cb583f5757 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d33fff700 1 -- 172.21.15.45:0/364981379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800e070 con 0x7f3d40159f70 2023-10-06T05:28:07.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d33fff700 1 -- 172.21.15.45:0/364981379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3800bb60 con 0x7f3d40159f70 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.531+0000 7f3d33fff700 1 -- 172.21.15.45:0/364981379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38012430 con 0x7f3d40159f70 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 -- 172.21.15.45:0/364981379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40159f70 msgr2=0x7f3d400ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 --2- 172.21.15.45:0/364981379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40159f70 0x7f3d400ca770 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009e60 tx=0x7f3d38009780 comp rx=0 tx=0).stop 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 -- 172.21.15.45:0/364981379 shutdown_connections 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 --2- 172.21.15.45:0/364981379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40159f70 0x7f3d400ca770 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 --2- 172.21.15.45:0/364981379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d401535e0 0x7f3d401539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 -- 172.21.15.45:0/364981379 >> 172.21.15.45:0/364981379 conn(0x7f3d400680b0 msgr2=0x7f3d400cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 -- 172.21.15.45:0/364981379 shutdown_connections 2023-10-06T05:28:07.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.532+0000 7f3d4727c700 1 -- 172.21.15.45:0/364981379 wait complete. 2023-10-06T05:28:07.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.534+0000 7f3d4727c700 1 Processor -- start 2023-10-06T05:28:07.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.534+0000 7f3d4727c700 1 -- start start 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d4727c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40068730 0x7f3d40069630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d4727c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d401535e0 0x7f3d400b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d4727c700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d40159f70 0x7f3d400b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d4727c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d400b1a80 con 0x7f3d40068730 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d4727c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d400b0e00 con 0x7f3d401535e0 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d4727c700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d400b10e0 con 0x7f3d40159f70 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40068730 0x7f3d40069630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40068730 0x7f3d40069630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33448/0 (socket says 172.21.15.45:33448) 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 -- 172.21.15.45:0/3143801550 learned_addr learned my addr 172.21.15.45:0/3143801550 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45819700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d40159f70 0x7f3d400b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d44817700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d401535e0 0x7f3d400b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 -- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d40159f70 msgr2=0x7f3d400b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d40159f70 0x7f3d400b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 -- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d401535e0 msgr2=0x7f3d400b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d401535e0 0x7f3d400b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 -- 172.21.15.45:0/3143801550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d38009a20 con 0x7f3d40068730 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.535+0000 7f3d45018700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40068730 0x7f3d40069630 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00ec80 tx=0x7f3d3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.536+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00c9d0 con 0x7f3d40068730 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.536+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c00cb30 con 0x7f3d40068730 2023-10-06T05:28:07.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.536+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d400b13c0 con 0x7f3d40068730 2023-10-06T05:28:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.536+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c013470 con 0x7f3d40068730 2023-10-06T05:28:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.536+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d400b2890 con 0x7f3d40068730 2023-10-06T05:28:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.537+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d4006a820 con 0x7f3d40068730 2023-10-06T05:28:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.537+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d3c0137e0 con 0x7f3d40068730 2023-10-06T05:28:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.538+0000 7f3d317fa700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d24066f30 0x7f3d240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.538+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f3d3c0848a0 con 0x7f3d40068730 2023-10-06T05:28:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.538+0000 7f3d317fa700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f3d2406d8f0 0x7f3d2406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.538+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d240703f0 con 0x7f3d2406d8f0 2023-10-06T05:28:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.538+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d3c089240 con 0x7f3d40068730 2023-10-06T05:28:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.538+0000 7f3d44817700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d24066f30 0x7f3d240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.538+0000 7f3d45819700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f3d2406d8f0 0x7f3d2406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.539+0000 7f3d44817700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d24066f30 0x7f3d240693e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3d4014c360 tx=0x7f3d34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.539+0000 7f3d45819700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f3d2406d8f0 0x7f3d2406fd20 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.539+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== osd.5 v2:172.21.15.144:6816/119917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3d240703f0 con 0x7f3d2406d8f0 2023-10-06T05:28:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.554+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 --> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3d4004c380 con 0x7f3d2406d8f0 2023-10-06T05:28:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d317fa700 1 -- 172.21.15.45:0/3143801550 <== osd.5 v2:172.21.15.144:6816/119917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3d4004c380 con 0x7f3d2406d8f0 2023-10-06T05:28:07.556 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:07.557 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f3d2406d8f0 msgr2=0x7f3d2406fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6816/119917,v1:172.21.15.144:6817/119917] conn(0x7f3d2406d8f0 0x7f3d2406fd20 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d24066f30 msgr2=0x7f3d240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f3d24066f30 0x7f3d240693e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3d4014c360 tx=0x7f3d34009040 comp rx=0 tx=0).stop 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40068730 msgr2=0x7f3d40069630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 --2- 172.21.15.45:0/3143801550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d40068730 0x7f3d40069630 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00ec80 tx=0x7f3d3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d45819700 1 -- 172.21.15.45:0/3143801550 reap_dead start 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 shutdown_connections 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 >> 172.21.15.45:0/3143801550 conn(0x7f3d400680b0 msgr2=0x7f3d4015cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 shutdown_connections 2023-10-06T05:28:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.555+0000 7f3d4727c700 1 -- 172.21.15.45:0/3143801550 wait complete. 2023-10-06T05:28:07.570 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T05:28:07.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.701+0000 7fb87af15700 1 Processor -- start 2023-10-06T05:28:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.701+0000 7fb87af15700 1 -- start start 2023-10-06T05:28:07.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb87af15700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb8740ca810 0x7fb8740cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb87af15700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb8740afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb87af15700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740c84c0 con 0x7fb8740c9640 2023-10-06T05:28:07.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb87af15700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8740c8d60 con 0x7fb8740ca810 2023-10-06T05:28:07.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb87af15700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb874069d00 con 0x7fb8740c91f0 2023-10-06T05:28:07.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb873fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb8740afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb878cb1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb8740ca810 0x7fb8740cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb873fff700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb8740afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.45:35758/0 (socket says 172.21.15.45:35758) 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb8794b2700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb8740c9640 0x7fb8740affc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:60552/0 (socket says 172.21.15.45:60552) 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.702+0000 7fb873fff700 1 -- 172.21.15.45:0/4057286611 learned_addr learned my addr 172.21.15.45:0/4057286611 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb878cb1700 1 -- 172.21.15.45:0/4057286611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 msgr2=0x7fb8740afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb878cb1700 1 --2- 172.21.15.45:0/4057286611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb8740afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb878cb1700 1 -- 172.21.15.45:0/4057286611 >> v1:172.21.15.144:6789/0 conn(0x7fb8740c9640 legacy=0x7fb8740affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb878cb1700 1 -- 172.21.15.45:0/4057286611 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740c8710 con 0x7fb8740ca810 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb8737fe700 1 -- 172.21.15.45:0/4057286611 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366066677 0 0) 0x7fb8740c84c0 con 0x7fb8740c9640 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb878cb1700 1 --2- 172.21.15.45:0/4057286611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740ca810 0x7fb8740cabe0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb8680098e0 tx=0x7fb8680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6dbd64b1a37bf8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb8737fe700 1 -- 172.21.15.45:0/4057286611 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86800b640 con 0x7fb8740ca810 2023-10-06T05:28:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb8737fe700 1 -- 172.21.15.45:0/4057286611 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86800b7a0 con 0x7fb8740ca810 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.703+0000 7fb8737fe700 1 -- 172.21.15.45:0/4057286611 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86800bb40 con 0x7fb8740ca810 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 -- 172.21.15.45:0/4057286611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740ca810 msgr2=0x7fb8740cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 --2- 172.21.15.45:0/4057286611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740ca810 0x7fb8740cabe0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb8680098e0 tx=0x7fb8680091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 -- 172.21.15.45:0/4057286611 shutdown_connections 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 --2- 172.21.15.45:0/4057286611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740ca810 0x7fb8740cabe0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 --2- 172.21.15.45:0/4057286611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb8740afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 -- 172.21.15.45:0/4057286611 >> 172.21.15.45:0/4057286611 conn(0x7fb874067620 msgr2=0x7fb874067a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 -- 172.21.15.45:0/4057286611 shutdown_connections 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.704+0000 7fb87af15700 1 -- 172.21.15.45:0/4057286611 wait complete. 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.705+0000 7fb87af15700 1 Processor -- start 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.705+0000 7fb87af15700 1 -- start start 2023-10-06T05:28:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb87af15700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb87415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb87af15700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740c9640 0x7fb8740c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb87af15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8740ca810 0x7fb8740c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb87af15700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740c2350 con 0x7fb8740ca810 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb87af15700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87416ad10 con 0x7fb8740c9640 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb87af15700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740c1a20 con 0x7fb8740c91f0 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb8794b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8740ca810 0x7fb8740c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb8794b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8740ca810 0x7fb8740c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33454/0 (socket says 172.21.15.45:33454) 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb8794b2700 1 -- 172.21.15.45:0/3453517951 learned_addr learned my addr 172.21.15.45:0/3453517951 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb878cb1700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb87415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb873fff700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740c9640 0x7fb8740c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb8794b2700 1 -- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 msgr2=0x7fb87415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb8794b2700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb87415a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb8794b2700 1 -- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740c9640 msgr2=0x7fb8740c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.706+0000 7fb8794b2700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740c9640 0x7fb8740c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.707+0000 7fb8794b2700 1 -- 172.21.15.45:0/3453517951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb868009420 con 0x7fb8740ca810 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.707+0000 7fb8794b2700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8740ca810 0x7fb8740c6f40 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00cd50 tx=0x7fb86c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.707+0000 7fb870ff9700 1 -- 172.21.15.45:0/3453517951 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c005df0 con 0x7fb8740ca810 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.707+0000 7fb870ff9700 1 -- 172.21.15.45:0/3453517951 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00a960 con 0x7fb8740ca810 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.707+0000 7fb870ff9700 1 -- 172.21.15.45:0/3453517951 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00ad20 con 0x7fb8740ca810 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.707+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8740c1d00 con 0x7fb8740ca810 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.707+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87415a950 con 0x7fb8740ca810 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.708+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87406f160 con 0x7fb8740ca810 2023-10-06T05:28:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.709+0000 7fb870ff9700 1 -- 172.21.15.45:0/3453517951 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb86c007300 con 0x7fb8740ca810 2023-10-06T05:28:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.709+0000 7fb870ff9700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb858066f30 0x7fb8580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.709+0000 7fb870ff9700 1 -- 172.21.15.45:0/3453517951 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fb86c084740 con 0x7fb8740ca810 2023-10-06T05:28:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.709+0000 7fb878cb1700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb858066f30 0x7fb8580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.710+0000 7fb878cb1700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb858066f30 0x7fb8580693e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb8680093f0 tx=0x7fb868017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.713+0000 7fb870ff9700 1 -- 172.21.15.45:0/3453517951 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb86c051a30 con 0x7fb8740ca810 2023-10-06T05:28:07.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.919+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8740c2490 con 0x7fb858066f30 2023-10-06T05:28:07.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.920+0000 7fb870ff9700 1 -- 172.21.15.45:0/3453517951 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb8740c2490 con 0x7fb858066f30 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb858066f30 msgr2=0x7fb8580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb858066f30 0x7fb8580693e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb8680093f0 tx=0x7fb868017000 comp rx=0 tx=0).stop 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8740ca810 msgr2=0x7fb8740c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8740ca810 0x7fb8740c6f40 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00cd50 tx=0x7fb86c001b10 comp rx=0 tx=0).stop 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 shutdown_connections 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fb858066f30 0x7fb8580693e0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8740ca810 0x7fb8740c6f40 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8740c91f0 0x7fb87415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 --2- 172.21.15.45:0/3453517951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8740c9640 0x7fb8740c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 >> 172.21.15.45:0/3453517951 conn(0x7fb874067620 msgr2=0x7fb87415c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 shutdown_connections 2023-10-06T05:28:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:07.922+0000 7fb87af15700 1 -- 172.21.15.45:0/3453517951 wait complete. 2023-10-06T05:28:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T05:28:07.937 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T05:28:07.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.072+0000 7f5b6f8c3700 1 Processor -- start 2023-10-06T05:28:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.072+0000 7f5b6f8c3700 1 -- start start 2023-10-06T05:28:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6f8c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5b68159160 0x7f5b680cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6f8c3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5b68159530 0x7f5b680c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6f8c3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680c8140 con 0x7f5b680c9cc0 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6f8c3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680ca1e0 con 0x7f5b68159160 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6f8c3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680ca4c0 con 0x7f5b68159530 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6d65f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5b68159160 0x7f5b680cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6de60700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5b680c9cc0 0x7f5b68067620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50034/0 (socket says 172.21.15.45:50034) 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6de60700 1 -- 172.21.15.45:0/366710909 learned_addr learned my addr 172.21.15.45:0/366710909 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6d65f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5b68159160 0x7f5b680cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33456/0 (socket says 172.21.15.45:33456) 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6ce5e700 1 --2- 172.21.15.45:0/366710909 >> v2:172.21.15.144:3300/0 conn(0x7f5b68159530 0x7f5b680c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6d65f700 1 -- 172.21.15.45:0/366710909 >> v2:172.21.15.144:3300/0 conn(0x7f5b68159530 msgr2=0x7f5b680c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6d65f700 1 --2- 172.21.15.45:0/366710909 >> v2:172.21.15.144:3300/0 conn(0x7f5b68159530 0x7f5b680c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6d65f700 1 -- 172.21.15.45:0/366710909 >> v1:172.21.15.45:6789/0 conn(0x7f5b680c9cc0 legacy=0x7f5b68067620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6d65f700 1 -- 172.21.15.45:0/366710909 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680c83e0 con 0x7f5b68159160 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b5ffff700 1 -- 172.21.15.45:0/366710909 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742089495 0 0) 0x7f5b680c8140 con 0x7f5b680c9cc0 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.073+0000 7f5b6ce5e700 1 --2- 172.21.15.45:0/366710909 >> v2:172.21.15.144:3300/0 conn(0x7f5b68159530 0x7f5b680c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b6d65f700 1 --2- 172.21.15.45:0/366710909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b680cadf0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f5b68064cd0 tx=0x7f5b640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75714ff0eee5e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b5ffff700 1 -- 172.21.15.45:0/366710909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400b6b0 con 0x7f5b68159160 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b5ffff700 1 -- 172.21.15.45:0/366710909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6400b810 con 0x7f5b68159160 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b5ffff700 1 -- 172.21.15.45:0/366710909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400bbb0 con 0x7f5b68159160 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/366710909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 msgr2=0x7f5b680cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b6f8c3700 1 --2- 172.21.15.45:0/366710909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b680cadf0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f5b68064cd0 tx=0x7f5b640091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/366710909 shutdown_connections 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b6f8c3700 1 --2- 172.21.15.45:0/366710909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b680cadf0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b6f8c3700 1 --2- 172.21.15.45:0/366710909 >> v2:172.21.15.144:3300/0 conn(0x7f5b68159530 0x7f5b680c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.074+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/366710909 >> 172.21.15.45:0/366710909 conn(0x7f5b68068730 msgr2=0x7f5b68068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.075+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/366710909 shutdown_connections 2023-10-06T05:28:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.075+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/366710909 wait complete. 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.076+0000 7f5b6f8c3700 1 Processor -- start 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.076+0000 7f5b6f8c3700 1 -- start start 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6f8c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b680c9cc0 0x7f5b68155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6f8c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b68147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6f8c3700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b68159530 0x7f5b680b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6f8c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680b1590 con 0x7f5b68159160 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6f8c3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680b0600 con 0x7f5b680c9cc0 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6f8c3700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680b2190 con 0x7f5b68159530 2023-10-06T05:28:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b68147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b68147aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33458/0 (socket says 172.21.15.45:33458) 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 -- 172.21.15.45:0/543311644 learned_addr learned my addr 172.21.15.45:0/543311644 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6d65f700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b680c9cc0 0x7f5b68155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6de60700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b68159530 0x7f5b680b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 -- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b68159530 msgr2=0x7f5b680b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b68159530 0x7f5b680b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 -- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b680c9cc0 msgr2=0x7f5b68155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b680c9cc0 0x7f5b68155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 -- 172.21.15.45:0/543311644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009420 con 0x7f5b68159160 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6ce5e700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b68147aa0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800d860 tx=0x7f5b58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580125f0 con 0x7f5b68159160 2023-10-06T05:28:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b58012750 con 0x7f5b68159160 2023-10-06T05:28:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.077+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b680b2430 con 0x7f5b68159160 2023-10-06T05:28:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.078+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58012af0 con 0x7f5b68159160 2023-10-06T05:28:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.078+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b680b0b30 con 0x7f5b68159160 2023-10-06T05:28:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.078+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b6804c380 con 0x7f5b68159160 2023-10-06T05:28:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.079+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b58013480 con 0x7f5b68159160 2023-10-06T05:28:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.080+0000 7f5b5d7fa700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b4c066f30 0x7f5b4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.080+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f5b58085140 con 0x7f5b68159160 2023-10-06T05:28:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.080+0000 7f5b5d7fa700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f5b4c06d8f0 0x7f5b4c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.080+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b4c0703f0 con 0x7f5b4c06d8f0 2023-10-06T05:28:08.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.080+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b58085430 con 0x7f5b68159160 2023-10-06T05:28:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.080+0000 7f5b6d65f700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b4c066f30 0x7f5b4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.080+0000 7f5b6de60700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f5b4c06d8f0 0x7f5b4c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.081+0000 7f5b6d65f700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b4c066f30 0x7f5b4c0693e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5b640093f0 tx=0x7f5b64017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.081+0000 7f5b6de60700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f5b4c06d8f0 0x7f5b4c06fd20 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.081+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== osd.6 v2:172.21.15.144:6800/119899 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f5b4c0703f0 con 0x7f5b4c06d8f0 2023-10-06T05:28:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.096+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 --> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5b680ca7a0 con 0x7f5b4c06d8f0 2023-10-06T05:28:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b5d7fa700 1 -- 172.21.15.45:0/543311644 <== osd.6 v2:172.21.15.144:6800/119899 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5b680ca7a0 con 0x7f5b4c06d8f0 2023-10-06T05:28:08.098 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:08.099 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:08.099 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f5b4c06d8f0 msgr2=0x7f5b4c06fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6800/119899,v1:172.21.15.144:6801/119899] conn(0x7f5b4c06d8f0 0x7f5b4c06fd20 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b4c066f30 msgr2=0x7f5b4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f5b4c066f30 0x7f5b4c0693e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5b640093f0 tx=0x7f5b64017000 comp rx=0 tx=0).stop 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 msgr2=0x7f5b68147aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 --2- 172.21.15.45:0/543311644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68159160 0x7f5b68147aa0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800d860 tx=0x7f5b58008040 comp rx=0 tx=0).stop 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6de60700 1 -- 172.21.15.45:0/543311644 reap_dead start 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 shutdown_connections 2023-10-06T05:28:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 >> 172.21.15.45:0/543311644 conn(0x7f5b68068730 msgr2=0x7f5b680c2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 shutdown_connections 2023-10-06T05:28:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.097+0000 7f5b6f8c3700 1 -- 172.21.15.45:0/543311644 wait complete. 2023-10-06T05:28:08.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T05:28:08.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.243+0000 7f40925d6700 1 Processor -- start 2023-10-06T05:28:08.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.243+0000 7f40925d6700 1 -- start start 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f40925d6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f408c0c8ff0 0x7f408c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f40925d6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f408c0caad0 0x7f408c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f40925d6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c15b000 con 0x7f408c156af0 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f40925d6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c157630 con 0x7f408c0caad0 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f40925d6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c15b2e0 con 0x7f408c0c8ff0 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f408b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f408c0caad0 0x7f408c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f4090b73700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f408c156af0 0x7f408c156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50048/0 (socket says 172.21.15.45:50048) 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f4090b73700 1 -- 172.21.15.45:0/2776008031 learned_addr learned my addr 172.21.15.45:0/2776008031 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f408bfff700 1 --2- 172.21.15.45:0/2776008031 >> v2:172.21.15.144:3300/0 conn(0x7f408c0c8ff0 0x7f408c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.244+0000 7f408affd700 1 -- 172.21.15.45:0/2776008031 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459268733 0 0) 0x7f408c15b000 con 0x7f408c156af0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408affd700 1 -- 172.21.15.45:0/2776008031 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4078003570 con 0x7f408c156af0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408b7fe700 1 -- 172.21.15.45:0/2776008031 >> v2:172.21.15.144:3300/0 conn(0x7f408c0c8ff0 msgr2=0x7f408c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408b7fe700 1 --2- 172.21.15.45:0/2776008031 >> v2:172.21.15.144:3300/0 conn(0x7f408c0c8ff0 0x7f408c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408b7fe700 1 -- 172.21.15.45:0/2776008031 >> v1:172.21.15.45:6789/0 conn(0x7f408c156af0 legacy=0x7f408c156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408b7fe700 1 -- 172.21.15.45:0/2776008031 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c15b5c0 con 0x7f408c0caad0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408b7fe700 1 --2- 172.21.15.45:0/2776008031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c0caad0 0x7f408c0caf40 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f408c0cefc0 tx=0x7f40740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bf5bcb1349c4b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408affd700 1 -- 172.21.15.45:0/2776008031 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074011630 con 0x7f408c0caad0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408affd700 1 -- 172.21.15.45:0/2776008031 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40740117b0 con 0x7f408c0caad0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f408affd700 1 -- 172.21.15.45:0/2776008031 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074011b70 con 0x7f408c0caad0 2023-10-06T05:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f40925d6700 1 -- 172.21.15.45:0/2776008031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c0caad0 msgr2=0x7f408c0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.245+0000 7f40925d6700 1 --2- 172.21.15.45:0/2776008031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c0caad0 0x7f408c0caf40 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f408c0cefc0 tx=0x7f40740091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.246+0000 7f40925d6700 1 -- 172.21.15.45:0/2776008031 shutdown_connections 2023-10-06T05:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.246+0000 7f40925d6700 1 --2- 172.21.15.45:0/2776008031 >> v2:172.21.15.144:3300/0 conn(0x7f408c0c8ff0 0x7f408c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.246+0000 7f40925d6700 1 --2- 172.21.15.45:0/2776008031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c0caad0 0x7f408c0caf40 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.246+0000 7f40925d6700 1 -- 172.21.15.45:0/2776008031 >> 172.21.15.45:0/2776008031 conn(0x7f408c0cdf50 msgr2=0x7f408c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.246+0000 7f40925d6700 1 -- 172.21.15.45:0/2776008031 shutdown_connections 2023-10-06T05:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.246+0000 7f40925d6700 1 -- 172.21.15.45:0/2776008031 wait complete. 2023-10-06T05:28:08.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.247+0000 7f40925d6700 1 Processor -- start 2023-10-06T05:28:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.247+0000 7f40925d6700 1 -- start start 2023-10-06T05:28:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f40925d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f408c0c8ff0 0x7f408c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f40925d6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408c0caad0 0x7f408c1499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f40925d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c156af0 0x7f408c0c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f40925d6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c157e30 con 0x7f408c156af0 2023-10-06T05:28:08.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f40925d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0b1230 con 0x7f408c0c8ff0 2023-10-06T05:28:08.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f40925d6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c1578b0 con 0x7f408c0caad0 2023-10-06T05:28:08.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c156af0 0x7f408c0c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c156af0 0x7f408c0c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33468/0 (socket says 172.21.15.45:33468) 2023-10-06T05:28:08.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 -- 172.21.15.45:0/742331313 learned_addr learned my addr 172.21.15.45:0/742331313 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:08.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f408bfff700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f408c0c8ff0 0x7f408c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 -- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408c0caad0 msgr2=0x7f408c1499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408c0caad0 0x7f408c1499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 -- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f408c0c8ff0 msgr2=0x7f408c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f408c0c8ff0 0x7f408c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 -- 172.21.15.45:0/742331313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4074009420 con 0x7f408c156af0 2023-10-06T05:28:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f408bfff700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f408c0c8ff0 0x7f408c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.248+0000 7f4090b73700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c156af0 0x7f408c0c6340 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f4084001460 tx=0x7f4084001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.249+0000 7f4083fff700 1 -- 172.21.15.45:0/742331313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408400ce80 con 0x7f408c156af0 2023-10-06T05:28:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.249+0000 7f4083fff700 1 -- 172.21.15.45:0/742331313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408400ab70 con 0x7f408c156af0 2023-10-06T05:28:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.249+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c157b50 con 0x7f408c156af0 2023-10-06T05:28:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.249+0000 7f4083fff700 1 -- 172.21.15.45:0/742331313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840070f0 con 0x7f408c156af0 2023-10-06T05:28:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.249+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c0c5e20 con 0x7f408c156af0 2023-10-06T05:28:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.250+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408c0c6040 con 0x7f408c156af0 2023-10-06T05:28:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.250+0000 7f4083fff700 1 -- 172.21.15.45:0/742331313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4084007520 con 0x7f408c156af0 2023-10-06T05:28:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.251+0000 7f4083fff700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f406c066f30 0x7f406c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.251+0000 7f4083fff700 1 -- 172.21.15.45:0/742331313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f4084084a00 con 0x7f408c156af0 2023-10-06T05:28:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.251+0000 7f408bfff700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f406c066f30 0x7f406c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.251+0000 7f408bfff700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f406c066f30 0x7f406c0693e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f407c006f00 tx=0x7f407c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.255+0000 7f4083fff700 1 -- 172.21.15.45:0/742331313 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4084051cf0 con 0x7f408c156af0 2023-10-06T05:28:08.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.459+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f408c0ca5f0 con 0x7f406c066f30 2023-10-06T05:28:08.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.459+0000 7f4083fff700 1 -- 172.21.15.45:0/742331313 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f408c0ca5f0 con 0x7f406c066f30 2023-10-06T05:28:08.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f406c066f30 msgr2=0x7f406c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f406c066f30 0x7f406c0693e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f407c006f00 tx=0x7f407c008040 comp rx=0 tx=0).stop 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c156af0 msgr2=0x7f408c0c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c156af0 0x7f408c0c6340 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f4084001460 tx=0x7f4084001ae0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 shutdown_connections 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f406c066f30 0x7f406c0693e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408c156af0 0x7f408c0c6340 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f408c0c8ff0 0x7f408c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.461+0000 7f40925d6700 1 --2- 172.21.15.45:0/742331313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408c0caad0 0x7f408c1499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.462+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 >> 172.21.15.45:0/742331313 conn(0x7f408c0cdf50 msgr2=0x7f408c0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.462+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 shutdown_connections 2023-10-06T05:28:08.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.462+0000 7f40925d6700 1 -- 172.21.15.45:0/742331313 wait complete. 2023-10-06T05:28:08.465 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T05:28:08.477 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T05:28:08.477 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:28:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.611+0000 7f64d9ba6700 1 Processor -- start 2023-10-06T05:28:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.611+0000 7f64d9ba6700 1 -- start start 2023-10-06T05:28:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.612+0000 7f64d9ba6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d40c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.612+0000 7f64d9ba6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f64d4067bb0 0x7f64d40c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.612+0000 7f64d9ba6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d40ca7f0 con 0x7f64d40ad1f0 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.612+0000 7f64d9ba6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d4068000 con 0x7f64d4067bb0 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.612+0000 7f64d9ba6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40ccdf0 con 0x7f64d40677e0 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d37fe700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d40c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d2ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f64d4067bb0 0x7f64d40c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d3fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f64d40ad1f0 0x7f64d41404c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.45:60568/0 (socket says 172.21.15.45:60568) 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d3fff700 1 -- 172.21.15.45:0/2447104200 learned_addr learned my addr 172.21.15.45:0/2447104200 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d27fc700 1 -- 172.21.15.45:0/2447104200 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536878353 0 0) 0x7f64d40ca7f0 con 0x7f64d40ad1f0 2023-10-06T05:28:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d27fc700 1 -- 172.21.15.45:0/2447104200 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64bc003570 con 0x7f64d40ad1f0 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d37fe700 1 -- 172.21.15.45:0/2447104200 >> v2:172.21.15.144:3300/0 conn(0x7f64d4067bb0 msgr2=0x7f64d40c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d37fe700 1 --2- 172.21.15.45:0/2447104200 >> v2:172.21.15.144:3300/0 conn(0x7f64d4067bb0 0x7f64d40c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d37fe700 1 -- 172.21.15.45:0/2447104200 >> v1:172.21.15.144:6789/0 conn(0x7f64d40ad1f0 legacy=0x7f64d41404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d37fe700 1 -- 172.21.15.45:0/2447104200 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40c9f50 con 0x7f64d40677e0 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.613+0000 7f64d2ffd700 1 --2- 172.21.15.45:0/2447104200 >> v2:172.21.15.144:3300/0 conn(0x7f64d4067bb0 0x7f64d40c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d37fe700 1 --2- 172.21.15.45:0/2447104200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d40c7500 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64c40098e0 tx=0x7f64c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bcf77e3145324fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d27fc700 1 -- 172.21.15.45:0/2447104200 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011630 con 0x7f64d40677e0 2023-10-06T05:28:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d27fc700 1 -- 172.21.15.45:0/2447104200 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4011790 con 0x7f64d40677e0 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d27fc700 1 -- 172.21.15.45:0/2447104200 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011b30 con 0x7f64d40677e0 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d9ba6700 1 -- 172.21.15.45:0/2447104200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 msgr2=0x7f64d40c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d9ba6700 1 --2- 172.21.15.45:0/2447104200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d40c7500 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64c40098e0 tx=0x7f64c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d9ba6700 1 -- 172.21.15.45:0/2447104200 shutdown_connections 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d9ba6700 1 --2- 172.21.15.45:0/2447104200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d40c7500 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d9ba6700 1 --2- 172.21.15.45:0/2447104200 >> v2:172.21.15.144:3300/0 conn(0x7f64d4067bb0 0x7f64d40c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d9ba6700 1 -- 172.21.15.45:0/2447104200 >> 172.21.15.45:0/2447104200 conn(0x7f64d4068730 msgr2=0x7f64d4068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.614+0000 7f64d9ba6700 1 -- 172.21.15.45:0/2447104200 shutdown_connections 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.615+0000 7f64d9ba6700 1 -- 172.21.15.45:0/2447104200 wait complete. 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.616+0000 7f64d9ba6700 1 Processor -- start 2023-10-06T05:28:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.616+0000 7f64d9ba6700 1 -- start start 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d9ba6700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d4156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d9ba6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64d4067bb0 0x7f64d40c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d9ba6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64d40ad1f0 0x7f64d413cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d9ba6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40c4260 con 0x7f64d4067bb0 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d9ba6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40c61e0 con 0x7f64d40ad1f0 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d9ba6700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b17d0 con 0x7f64d40677e0 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64d4067bb0 0x7f64d40c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64d4067bb0 0x7f64d40c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33474/0 (socket says 172.21.15.45:33474) 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 -- 172.21.15.45:0/490017033 learned_addr learned my addr 172.21.15.45:0/490017033 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d3fff700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64d40ad1f0 0x7f64d413cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d37fe700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d4156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 -- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 msgr2=0x7f64d4156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64d40677e0 0x7f64d4156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 -- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64d40ad1f0 msgr2=0x7f64d413cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64d40ad1f0 0x7f64d413cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 -- 172.21.15.45:0/490017033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009420 con 0x7f64d4067bb0 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.617+0000 7f64d2ffd700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64d4067bb0 0x7f64d40c6670 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f64c800ec80 tx=0x7f64c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.618+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800c920 con 0x7f64d4067bb0 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.618+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800ca80 con 0x7f64d4067bb0 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.618+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c4009420 con 0x7f64d4067bb0 2023-10-06T05:28:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.618+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800ce00 con 0x7f64d4067bb0 2023-10-06T05:28:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.618+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d40c5ef0 con 0x7f64d4067bb0 2023-10-06T05:28:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.619+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64d40b1bd0 con 0x7f64d4067bb0 2023-10-06T05:28:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.619+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64c8013680 con 0x7f64d4067bb0 2023-10-06T05:28:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.620+0000 7f64bb7fe700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64c0066f30 0x7f64c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.620+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f64c8084cc0 con 0x7f64d4067bb0 2023-10-06T05:28:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.620+0000 7f64bb7fe700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f64c006d8f0 0x7f64c006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.620+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64c00703f0 con 0x7f64c006d8f0 2023-10-06T05:28:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.620+0000 7f64d37fe700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64c0066f30 0x7f64c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.620+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64c8085090 con 0x7f64d4067bb0 2023-10-06T05:28:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.621+0000 7f64d3fff700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f64c006d8f0 0x7f64c006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.622+0000 7f64d37fe700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64c0066f30 0x7f64c00693e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f64c40098e0 tx=0x7f64c4017000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.623+0000 7f64d3fff700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f64c006d8f0 0x7f64c006fd20 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.625+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== osd.7 v2:172.21.15.144:6808/119901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f64c00703f0 con 0x7f64c006d8f0 2023-10-06T05:28:08.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.640+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 --> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f64d40c59f0 con 0x7f64c006d8f0 2023-10-06T05:28:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.640+0000 7f64bb7fe700 1 -- 172.21.15.45:0/490017033 <== osd.7 v2:172.21.15.144:6808/119901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f64d40c59f0 con 0x7f64c006d8f0 2023-10-06T05:28:08.642 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-10-06T05:28:08.643 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:28:08.643 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-10-06T05:28:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.640+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f64c006d8f0 msgr2=0x7f64c006fd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6808/119901,v1:172.21.15.144:6809/119901] conn(0x7f64c006d8f0 0x7f64c006fd20 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64c0066f30 msgr2=0x7f64c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f64c0066f30 0x7f64c00693e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f64c40098e0 tx=0x7f64c4017000 comp rx=0 tx=0).stop 2023-10-06T05:28:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64d4067bb0 msgr2=0x7f64d40c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 --2- 172.21.15.45:0/490017033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64d4067bb0 0x7f64d40c6670 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f64c800ec80 tx=0x7f64c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d3fff700 1 -- 172.21.15.45:0/490017033 reap_dead start 2023-10-06T05:28:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 shutdown_connections 2023-10-06T05:28:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 >> 172.21.15.45:0/490017033 conn(0x7f64d4068730 msgr2=0x7f64d40c1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 shutdown_connections 2023-10-06T05:28:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.641+0000 7f64d9ba6700 1 -- 172.21.15.45:0/490017033 wait complete. 2023-10-06T05:28:08.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T05:28:08.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:08.702 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.789+0000 7f738a10d700 1 Processor -- start 2023-10-06T05:28:08.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.789+0000 7f738a10d700 1 -- start start 2023-10-06T05:28:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f738a10d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7384155cd0 0x7f738415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f738a10d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73841560a0 0x7f73840adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f738a10d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f73840cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f738a10d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840cd740 con 0x7f73841560a0 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f738a10d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840ae8d0 con 0x7f7384155cd0 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f738a10d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840aebb0 con 0x7f73840ccdc0 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7382ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73841560a0 0x7f73840adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7382ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73841560a0 0x7f73840adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33478/0 (socket says 172.21.15.45:33478) 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7382ffd700 1 -- 172.21.15.45:0/872892950 learned_addr learned my addr 172.21.15.45:0/872892950 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7383fff700 1 --2- 172.21.15.45:0/872892950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f73840cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7382ffd700 1 -- 172.21.15.45:0/872892950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 msgr2=0x7f73840cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7382ffd700 1 --2- 172.21.15.45:0/872892950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f73840cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7382ffd700 1 -- 172.21.15.45:0/872892950 >> v2:172.21.15.144:3300/0 conn(0x7f7384155cd0 msgr2=0x7f738415a730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.790+0000 7f7382ffd700 1 --2- 172.21.15.45:0/872892950 >> v2:172.21.15.144:3300/0 conn(0x7f7384155cd0 0x7f738415a730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f7382ffd700 1 -- 172.21.15.45:0/872892950 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840aee90 con 0x7f73841560a0 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f7382ffd700 1 --2- 172.21.15.45:0/872892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73841560a0 0x7f73840adfa0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f738406ae50 tx=0x7f736c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f600f0b817778ba9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f73827fc700 1 -- 172.21.15.45:0/872892950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00b860 con 0x7f73841560a0 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f73827fc700 1 -- 172.21.15.45:0/872892950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c00b9c0 con 0x7f73841560a0 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f73827fc700 1 -- 172.21.15.45:0/872892950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00bd60 con 0x7f73841560a0 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f738a10d700 1 -- 172.21.15.45:0/872892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73841560a0 msgr2=0x7f73840adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f738a10d700 1 --2- 172.21.15.45:0/872892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73841560a0 0x7f73840adfa0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f738406ae50 tx=0x7f736c009250 comp rx=0 tx=0).stop 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f738a10d700 1 -- 172.21.15.45:0/872892950 shutdown_connections 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f738a10d700 1 --2- 172.21.15.45:0/872892950 >> v2:172.21.15.144:3300/0 conn(0x7f7384155cd0 0x7f738415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f738a10d700 1 --2- 172.21.15.45:0/872892950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f73840cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.791+0000 7f738a10d700 1 --2- 172.21.15.45:0/872892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73841560a0 0x7f73840adfa0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.792+0000 7f738a10d700 1 -- 172.21.15.45:0/872892950 >> 172.21.15.45:0/872892950 conn(0x7f738406f090 msgr2=0x7f738406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.792+0000 7f738a10d700 1 -- 172.21.15.45:0/872892950 shutdown_connections 2023-10-06T05:28:08.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.792+0000 7f738a10d700 1 -- 172.21.15.45:0/872892950 wait complete. 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.793+0000 7f738a10d700 1 Processor -- start 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.793+0000 7f738a10d700 1 -- start start 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f738a10d700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f7384151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f738a10d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7384155cd0 0x7f7384158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f738a10d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73841560a0 0x7f73841582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f738a10d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840c4f60 con 0x7f7384155cd0 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f738a10d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840b2130 con 0x7f73841560a0 2023-10-06T05:28:08.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f738a10d700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840b2410 con 0x7f73840ccdc0 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7384155cd0 0x7f7384158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7384155cd0 0x7f7384158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33484/0 (socket says 172.21.15.45:33484) 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 -- 172.21.15.45:0/3086547710 learned_addr learned my addr 172.21.15.45:0/3086547710 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f73837fe700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f7384151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7383fff700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73841560a0 0x7f73841582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 -- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 msgr2=0x7f7384151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f7384151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 -- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73841560a0 msgr2=0x7f73841582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73841560a0 0x7f73841582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 -- 172.21.15.45:0/3086547710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c0094f0 con 0x7f7384155cd0 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.794+0000 7f7382ffd700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7384155cd0 0x7f7384158fb0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f736c009290 tx=0x7f736c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.795+0000 7f737b7fe700 1 -- 172.21.15.45:0/3086547710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c01e070 con 0x7f7384155cd0 2023-10-06T05:28:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.795+0000 7f737b7fe700 1 -- 172.21.15.45:0/3086547710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c019d00 con 0x7f7384155cd0 2023-10-06T05:28:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.795+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73840c4910 con 0x7f7384155cd0 2023-10-06T05:28:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.795+0000 7f737b7fe700 1 -- 172.21.15.45:0/3086547710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c0106c0 con 0x7f7384155cd0 2023-10-06T05:28:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.795+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73841594f0 con 0x7f7384155cd0 2023-10-06T05:28:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.796+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73840c5110 con 0x7f7384155cd0 2023-10-06T05:28:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.796+0000 7f737b7fe700 1 -- 172.21.15.45:0/3086547710 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f736c022070 con 0x7f7384155cd0 2023-10-06T05:28:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.797+0000 7f737b7fe700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7364066f60 0x7f7364069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.797+0000 7f737b7fe700 1 -- 172.21.15.45:0/3086547710 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f736c014070 con 0x7f7384155cd0 2023-10-06T05:28:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.797+0000 7f73837fe700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7364066f60 0x7f7364069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.797+0000 7f73837fe700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7364066f60 0x7f7364069410 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7374006fd0 tx=0x7f73740090f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:08.801+0000 7f737b7fe700 1 -- 172.21.15.45:0/3086547710 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73840c5110 con 0x7f7384155cd0 2023-10-06T05:28:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.002+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f73841518c0 con 0x7f7364066f60 2023-10-06T05:28:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.003+0000 7f737b7fe700 1 -- 172.21.15.45:0/3086547710 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f737400e650 con 0x7f7364066f60 2023-10-06T05:28:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7364066f60 msgr2=0x7f7364069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7364066f60 0x7f7364069410 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7374006fd0 tx=0x7f73740090f0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7384155cd0 msgr2=0x7f7384158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7384155cd0 0x7f7384158fb0 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f736c009290 tx=0x7f736c019000 comp rx=0 tx=0).stop 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 shutdown_connections 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f7364066f60 0x7f7364069410 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73841560a0 0x7f73841582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73840ccdc0 0x7f7384151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 --2- 172.21.15.45:0/3086547710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7384155cd0 0x7f7384158fb0 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 >> 172.21.15.45:0/3086547710 conn(0x7f738406f090 msgr2=0x7f7384070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 shutdown_connections 2023-10-06T05:28:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.005+0000 7f738a10d700 1 -- 172.21.15.45:0/3086547710 wait complete. 2023-10-06T05:28:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T05:28:09.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:28:09.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.191+0000 7ff88af4e700 1 Processor -- start 2023-10-06T05:28:09.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.192+0000 7ff88af4e700 1 -- start start 2023-10-06T05:28:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.192+0000 7ff88af4e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff8841560a0 0x7ff88415a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.192+0000 7ff88af4e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840ccdc0 0x7ff8840cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.192+0000 7ff88af4e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8840cd740 con 0x7ff884155cd0 2023-10-06T05:28:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.192+0000 7ff88af4e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8840ae6b0 con 0x7ff8841560a0 2023-10-06T05:28:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.192+0000 7ff88af4e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840ae8d0 con 0x7ff8840ccdc0 2023-10-06T05:28:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff888cea700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff884155cd0 0x7ff8840adfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50052/0 (socket says 172.21.15.45:50052) 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff888cea700 1 -- 172.21.15.45:0/3595057672 learned_addr learned my addr 172.21.15.45:0/3595057672 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8894eb700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840ccdc0 0x7ff8840cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff883fff700 1 --2- 172.21.15.45:0/3595057672 >> v2:172.21.15.144:3300/0 conn(0x7ff8841560a0 0x7ff88415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441634102 0 0) 0x7ff8840cd740 con 0x7ff884155cd0 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870003570 con 0x7ff884155cd0 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2848446508 0 0) 0x7ff870003570 con 0x7ff884155cd0 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840ccdc0 msgr2=0x7ff8840cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840ccdc0 0x7ff8840cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 >> v2:172.21.15.144:3300/0 conn(0x7ff8841560a0 msgr2=0x7ff88415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 --2- 172.21.15.45:0/3595057672 >> v2:172.21.15.144:3300/0 conn(0x7ff8841560a0 0x7ff88415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840aebb0 con 0x7ff884155cd0 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4258700163 0 0) 0x7ff86c001470 con 0x7ff884155cd0 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 >> v1:172.21.15.45:6789/0 conn(0x7ff884155cd0 legacy=0x7ff8840adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff870003a60 0x7ff870003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff870004810 0x7ff870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8700051c0 0x7ff87000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840ae6b0 con 0x7ff870003a60 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840ae8d0 con 0x7ff8700051c0 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000e1b0 con 0x7ff870004810 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff8894eb700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840ccdc0 0x7ff8840cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff883fff700 1 --2- 172.21.15.45:0/3595057672 >> v2:172.21.15.144:3300/0 conn(0x7ff8841560a0 0x7ff88415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.193+0000 7ff888cea700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff870003a60 0x7ff870003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff8894eb700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8700051c0 0x7ff87000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff883fff700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff870004810 0x7ff870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff888cea700 1 -- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff870004810 msgr2=0x7ff870004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff888cea700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff870004810 0x7ff870004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff888cea700 1 -- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8700051c0 msgr2=0x7ff87000dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff888cea700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8700051c0 0x7ff87000dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff888cea700 1 -- 172.21.15.45:0/3595057672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870003570 con 0x7ff870003a60 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff888cea700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff870003a60 0x7ff870003e70 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c3f0 tx=0x7ff86c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ba1267a1349d47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00cc20 con 0x7ff870003a60 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c00cdc0 con 0x7ff870003a60 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.194+0000 7ff8837fe700 1 -- 172.21.15.45:0/3595057672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c005b70 con 0x7ff870003a60 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.195+0000 7ff88af4e700 1 -- 172.21.15.45:0/3595057672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff870003a60 msgr2=0x7ff870003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.195+0000 7ff88af4e700 1 --2- 172.21.15.45:0/3595057672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff870003a60 0x7ff870003e70 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c3f0 tx=0x7ff86c001e90 comp rx=0 tx=0).stop 2023-10-06T05:28:09.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.195+0000 7ff8894eb700 1 -- 172.21.15.45:0/3595057672 reap_dead start 2023-10-06T05:28:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.195+0000 7ff88af4e700 1 -- 172.21.15.45:0/3595057672 shutdown_connections 2023-10-06T05:28:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.195+0000 7ff88af4e700 1 -- 172.21.15.45:0/3595057672 >> 172.21.15.45:0/3595057672 conn(0x7ff88406f090 msgr2=0x7ff88406f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.195+0000 7ff88af4e700 1 -- 172.21.15.45:0/3595057672 shutdown_connections 2023-10-06T05:28:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.195+0000 7ff88af4e700 1 -- 172.21.15.45:0/3595057672 wait complete. 2023-10-06T05:28:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.196+0000 7ff88af4e700 1 Processor -- start 2023-10-06T05:28:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.196+0000 7ff88af4e700 1 -- start start 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff88af4e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff870003a60 0x7ff8841525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff88af4e700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840c55d0 0x7ff8840c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff88af4e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8840b1fb0 0x7ff8840b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff88af4e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884169c70 con 0x7ff8840b1fb0 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff88af4e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840c5f60 con 0x7ff870003a60 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff88af4e700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b2990 con 0x7ff8840c55d0 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff8894eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8840b1fb0 0x7ff8840b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff8894eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8840b1fb0 0x7ff8840b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33498/0 (socket says 172.21.15.45:33498) 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff8894eb700 1 -- 172.21.15.45:0/2683554718 learned_addr learned my addr 172.21.15.45:0/2683554718 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff883fff700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840c55d0 0x7ff8840c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff888cea700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff870003a60 0x7ff8841525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.197+0000 7ff8894eb700 1 -- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840c55d0 msgr2=0x7ff8840c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff8894eb700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840c55d0 0x7ff8840c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff8894eb700 1 -- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff870003a60 msgr2=0x7ff8841525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff8894eb700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff870003a60 0x7ff8841525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff8894eb700 1 -- 172.21.15.45:0/2683554718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884165970 con 0x7ff8840b1fb0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff8894eb700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8840b1fb0 0x7ff8840b2450 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7ff8840cd190 tx=0x7ff87c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff880ff9700 1 -- 172.21.15.45:0/2683554718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00e040 con 0x7ff8840b1fb0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff880ff9700 1 -- 172.21.15.45:0/2683554718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c00f040 con 0x7ff8840b1fb0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff884169060 con 0x7ff8840b1fb0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff880ff9700 1 -- 172.21.15.45:0/2683554718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00fde0 con 0x7ff8840b1fb0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.198+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884169620 con 0x7ff8840b1fb0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.199+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884169e20 con 0x7ff8840b1fb0 2023-10-06T05:28:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.200+0000 7ff880ff9700 1 -- 172.21.15.45:0/2683554718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff87c015640 con 0x7ff8840b1fb0 2023-10-06T05:28:09.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.200+0000 7ff880ff9700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff864066f30 0x7ff8640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:09.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.200+0000 7ff880ff9700 1 -- 172.21.15.45:0/2683554718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7ff87c0846c0 con 0x7ff8840b1fb0 2023-10-06T05:28:09.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.200+0000 7ff888cea700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff864066f30 0x7ff8640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:09.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.201+0000 7ff888cea700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff864066f30 0x7ff8640693e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c3f0 tx=0x7ff86c015000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:09.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.204+0000 7ff880ff9700 1 -- 172.21.15.45:0/2683554718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff87c0519b0 con 0x7ff8840b1fb0 2023-10-06T05:28:09.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.402+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8840aee90 con 0x7ff864066f30 2023-10-06T05:28:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.405+0000 7ff880ff9700 1 -- 172.21.15.45:0/2683554718 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73138 (secure 0 0 0) 0x7ff8840aee90 con 0x7ff864066f30 2023-10-06T05:28:09.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff864066f30 msgr2=0x7ff8640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff864066f30 0x7ff8640693e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c3f0 tx=0x7ff86c015000 comp rx=0 tx=0).stop 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8840b1fb0 msgr2=0x7ff8840b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8840b1fb0 0x7ff8840b2450 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7ff8840cd190 tx=0x7ff87c00b040 comp rx=0 tx=0).stop 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 shutdown_connections 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7ff864066f30 0x7ff8640693e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8840b1fb0 0x7ff8840b2450 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff870003a60 0x7ff8841525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 --2- 172.21.15.45:0/2683554718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8840c55d0 0x7ff8840c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:09.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 >> 172.21.15.45:0/2683554718 conn(0x7ff88406f090 msgr2=0x7ff884068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:09.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 shutdown_connections 2023-10-06T05:28:09.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:09.408+0000 7ff88af4e700 1 -- 172.21.15.45:0/2683554718 wait complete. 2023-10-06T05:28:09.411 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:28:09.423 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2023-10-06T05:28:08.644998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2456164,"kb_used_data":16420,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752518556,"statfs":{"total":773094113280,"available":770579001344,"internally_reserved":0,"allocated":16814080,"data_stored":9322711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.612850"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":131,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:01.572404+0000","last_change":"2023-10-06T05:14:53.984894+0000","last_active":"2023-10-06T05:28:01.572404+0000","last_peered":"2023-10-06T05:28:01.572404+0000","last_clean":"2023-10-06T05:28:01.572404+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:28:01.572404+0000","last_undegraded":"2023-10-06T05:28:01.572404+0000","last_fullsized":"2023-10-06T05:28:01.572404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.495351+0000","last_change":"2023-10-06T05:14:53.840339+0000","last_active":"2023-10-06T05:27:57.495351+0000","last_peered":"2023-10-06T05:27:57.495351+0000","last_clean":"2023-10-06T05:27:57.495351+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:27:57.495351+0000","last_undegraded":"2023-10-06T05:27:57.495351+0000","last_fullsized":"2023-10-06T05:27:57.495351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:00.039471+0000","last_change":"2023-10-06T05:14:54.057253+0000","last_active":"2023-10-06T05:28:00.039471+0000","last_peered":"2023-10-06T05:28:00.039471+0000","last_clean":"2023-10-06T05:28:00.039471+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:28:00.039471+0000","last_undegraded":"2023-10-06T05:28:00.039471+0000","last_fullsized":"2023-10-06T05:28:00.039471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.495323+0000","last_change":"2023-10-06T05:14:53.840063+0000","last_active":"2023-10-06T05:27:57.495323+0000","last_peered":"2023-10-06T05:27:57.495323+0000","last_clean":"2023-10-06T05:27:57.495323+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:27:57.495323+0000","last_undegraded":"2023-10-06T05:27:57.495323+0000","last_fullsized":"2023-10-06T05:27:57.495323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.843534+0000","last_change":"2023-10-06T05:14:53.985766+0000","last_active":"2023-10-06T05:27:56.843534+0000","last_peered":"2023-10-06T05:27:56.843534+0000","last_clean":"2023-10-06T05:27:56.843534+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:27:56.843534+0000","last_undegraded":"2023-10-06T05:27:56.843534+0000","last_fullsized":"2023-10-06T05:27:56.843534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:57.722376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.843419+0000","last_change":"2023-10-06T05:14:53.841077+0000","last_active":"2023-10-06T05:27:56.843419+0000","last_peered":"2023-10-06T05:27:56.843419+0000","last_clean":"2023-10-06T05:27:56.843419+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:27:56.843419+0000","last_undegraded":"2023-10-06T05:27:56.843419+0000","last_fullsized":"2023-10-06T05:27:56.843419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:26.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:56.844094+0000","last_change":"2023-10-06T05:14:53.985119+0000","last_active":"2023-10-06T05:27:56.844094+0000","last_peered":"2023-10-06T05:27:56.844094+0000","last_clean":"2023-10-06T05:27:56.844094+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:27:56.844094+0000","last_undegraded":"2023-10-06T05:27:56.844094+0000","last_fullsized":"2023-10-06T05:27:56.844094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.646783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":131,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:01.573993+0000","last_change":"2023-10-06T05:14:53.841450+0000","last_active":"2023-10-06T05:28:01.573993+0000","last_peered":"2023-10-06T05:28:01.573993+0000","last_clean":"2023-10-06T05:28:01.573993+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:28:01.573993+0000","last_undegraded":"2023-10-06T05:28:01.573993+0000","last_fullsized":"2023-10-06T05:28:01.573993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":125,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T05:27:57.924935+0000","last_change":"2023-10-06T05:14:53.840034+0000","last_active":"2023-10-06T05:27:57.924935+0000","last_peered":"2023-10-06T05:27:57.924935+0000","last_clean":"2023-10-06T05:27:57.924935+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:27:57.924935+0000","last_undegraded":"2023-10-06T05:27:57.924935+0000","last_fullsized":"2023-10-06T05:27:57.924935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:14:52.825207+0000","last_clean_scrub_stamp":"2023-10-06T05:14:52.825207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:30.554245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.878},"last":0.621},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":1.476},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84999999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":1.5129999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":2.0190000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":2.0059999999999998},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58699999999999997,"5min":0.76800000000000002,"15min":0.90300000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.504,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.76800000000000002,"15min":0.93500000000000005},"last":0.39700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.505},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68999999999999995,"5min":0.81699999999999995,"15min":0.90000000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.84899999999999998},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.91000000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.314},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.92900000000000005},"last":0.496}]}]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":4.6900000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":0.86799999999999999},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.82599999999999996,"15min":9.6300000000000008},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.76400000000000001,"15min":5.6239999999999997},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.73899999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":11.595000000000001},"last":0.499},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.78100000000000003,"15min":5.5800000000000001},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.93600000000000005},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.39000000000000001},"max":{"1min":0.745,"5min":0.75900000000000001,"15min":23.146000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.73699999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":9.7539999999999996},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.61099999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":9.9969999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.64600000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.751,"5min":0.752,"15min":9.5850000000000009},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.496,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.6439999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":9.8420000000000005},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":9.718},"last":0.629},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":9.8930000000000007},"last":0.62}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.64400000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.40699999999999997,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.73699999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.3999999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58199999999999996,"5min":0.67100000000000004,"15min":1.673},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.503},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67800000000000005,"5min":0.72399999999999998,"15min":4.0490000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.45200000000000001},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77700000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":0.92600000000000005,"15min":0.94699999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.95199999999999996},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.54000000000000004,"5min":0.45000000000000001,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68000000000000005,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":1.0369999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77200000000000002,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66300000000000003,"15min":0.68400000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.44},"max":{"1min":0.871,"5min":0.95599999999999996,"15min":1.167},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63300000000000001,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.46600000000000003,"15min":0.42899999999999999},"max":{"1min":0.88300000000000001,"5min":0.94499999999999995,"15min":1.026},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92100000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.69299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.30599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56799999999999995,"5min":0.82099999999999995,"15min":1.038},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.30299999999999999,"5min":0.27400000000000002,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":1.0609999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.252},"max":{"1min":0.60799999999999998,"5min":0.67800000000000005,"15min":3.855},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44800000000000001},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.1919999999999999},"last":0.313}]},{"osd":3,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.45200000000000001},"min":{"1min":0.32100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58799999999999997,"5min":0.66900000000000004,"15min":7.5880000000000001},"last":0.374},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":7.7670000000000003},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.438,"15min":0.35899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":2.4039999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.97199999999999998,"15min":2.0870000000000002},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":0.98199999999999998,"15min":2.4940000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":5.3600000000000003},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.01,"15min":1.1759999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.45900000000000002,"15min":0.45400000000000001},"max":{"1min":0.874,"5min":0.99199999999999999,"15min":1.0940000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.97899999999999998,"15min":1.0169999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.54700000000000004,"5min":0.47499999999999998,"15min":0.46500000000000002},"max":{"1min":0.85299999999999998,"5min":0.92400000000000004,"15min":1.2150000000000001},"last":0.70199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.39500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":2.0720000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.899},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66600000000000004,"15min":2.097},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.48699999999999999},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":1.8759999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.26700000000000002,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":1.6719999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.48299999999999998},"min":{"1min":0.33700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61199999999999999,"5min":0.70499999999999996,"15min":2.121},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":1.012,"15min":1.7789999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.47999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":1.018,"15min":1.643},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66300000000000003,"15min":0.68999999999999995},"min":{"1min":0.495,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":1.2130000000000001,"15min":2.0129999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.69499999999999995},"min":{"1min":0.51000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.851},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81999999999999995,"5min":0.88300000000000001,"15min":1.571},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.0800000000000001,"15min":2.0449999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.68000000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0569999999999999,"15min":1.8180000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.44500000000000001},"max":{"1min":1.0029999999999999,"5min":1.131,"15min":1.613},"last":0.745}]}]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307384,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064456,"statfs":{"total":96636764160,"available":96322002944,"internally_reserved":0,"allocated":1949696,"data_stored":1016563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26900000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64500000000000002,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.309},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.40799999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.22900000000000001},"max":{"1min":0.63900000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40999999999999998,"15min":0.39600000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61399999999999999,"5min":1.75,"15min":1.75},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60099999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.437},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72699999999999998,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.39800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.47399999999999998,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.0469999999999999,"15min":1.143},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.52600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85599999999999998,"5min":1.855,"15min":1.855},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.626}]},{"osd":6,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.43099999999999999},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.879,"5min":1.885,"15min":1.885},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.89900000000000002,"15min":1.236},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.98399999999999999,"15min":1.1950000000000001},"last":0.63400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":311484,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060356,"statfs":{"total":96636764160,"available":96317804544,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":12.983000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.33400000000000002,"15min":0.33100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":1.7430000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.8080000000000001},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":12.911},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.374,"15min":0.374},"max":{"1min":0.73099999999999998,"5min":0.84299999999999997,"15min":5.0890000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":12.999000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":13.289999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.95099999999999996,"15min":1.5760000000000001},"last":0.434},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.88200000000000001,"15min":1.552},"last":0.46999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.54600000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.85699999999999998,"15min":12.949},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.92900000000000005,"15min":5.2549999999999999},"last":0.622}]},{"osd":7,"last update":"Fri Oct 6 05:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":2.871},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.61699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.91500000000000004,"15min":12.859},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76400000000000001,"5min":0.79400000000000004,"15min":0.82299999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.89400000000000002},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.82199999999999995,"15min":0.84599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.77200000000000002,"15min":0.81599999999999995},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.83299999999999996},"last":0.62},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.876},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.67600000000000005,"5min":0.85999999999999999,"15min":0.875},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.49199999999999999,"5min":0.432,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.499,"15min":0.48399999999999999},"min":{"1min":0.36499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.92800000000000005},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.317,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.86699999999999999,"15min":0.876},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.84899999999999998,"15min":0.89300000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.754,"5min":0.875,"15min":0.90900000000000003},"last":0.74399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:28:09.427 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.427 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T05:14:52.825207+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=14, tm_sec=52, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=28, tm_sec=3, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:28:09.428 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T05:28:29.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:28:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.570+0000 7fd1cde31700 1 Processor -- start 2023-10-06T05:28:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.570+0000 7fd1cde31700 1 -- start start 2023-10-06T05:28:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.570+0000 7fd1cde31700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1c8159330 0x7fd1c806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.570+0000 7fd1cde31700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c80cb530 con 0x7fd1c80cc390 2023-10-06T05:28:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1cde31700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c8069bd0 con 0x7fd1c8158f60 2023-10-06T05:28:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1cde31700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c8068fc0 con 0x7fd1c8159330 2023-10-06T05:28:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1c8159330 0x7fd1c806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd1c80cc390 0x7fd1c80c8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45700/0 (socket says 172.21.15.45:45700) 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1c8159330 0x7fd1c806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40432/0 (socket says 172.21.15.45:40432) 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c7fff700 1 -- 172.21.15.45:0/380482053 learned_addr learned my addr 172.21.15.45:0/380482053 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c6ffd700 1 -- 172.21.15.45:0/380482053 >> v1:172.21.15.144:6789/0 conn(0x7fd1c8158f60 legacy=0x7fd1c80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c6ffd700 1 -- 172.21.15.45:0/380482053 >> v1:172.21.15.45:6789/0 conn(0x7fd1c80cc390 legacy=0x7fd1c80c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c6ffd700 1 -- 172.21.15.45:0/380482053 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80692a0 con 0x7fd1c8159330 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c67fc700 1 -- 172.21.15.45:0/380482053 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528748612 0 0) 0x7fd1c0001490 con 0x7fd1c80cc390 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c6ffd700 1 --2- 172.21.15.45:0/380482053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c806a060 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098e0 tx=0x7fd1b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a313765d9934c3fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c67fc700 1 -- 172.21.15.45:0/380482053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4009e80 con 0x7fd1c8159330 2023-10-06T05:28:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c67fc700 1 -- 172.21.15.45:0/380482053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400b860 con 0x7fd1c8159330 2023-10-06T05:28:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.571+0000 7fd1c67fc700 1 -- 172.21.15.45:0/380482053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400bbe0 con 0x7fd1c8159330 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.572+0000 7fd1cde31700 1 -- 172.21.15.45:0/380482053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 msgr2=0x7fd1c806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.572+0000 7fd1cde31700 1 --2- 172.21.15.45:0/380482053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c806a060 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098e0 tx=0x7fd1b40091a0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.572+0000 7fd1cde31700 1 -- 172.21.15.45:0/380482053 shutdown_connections 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.572+0000 7fd1cde31700 1 --2- 172.21.15.45:0/380482053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c806a060 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.572+0000 7fd1cde31700 1 -- 172.21.15.45:0/380482053 >> 172.21.15.45:0/380482053 conn(0x7fd1c80cdf50 msgr2=0x7fd1c80ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.572+0000 7fd1cde31700 1 -- 172.21.15.45:0/380482053 shutdown_connections 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.572+0000 7fd1cde31700 1 -- 172.21.15.45:0/380482053 wait complete. 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.574+0000 7fd1cde31700 1 Processor -- start 2023-10-06T05:28:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.574+0000 7fd1cde31700 1 -- start start 2023-10-06T05:28:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1cde31700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c80cc390 0x7fd1c8155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1cde31700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1c8158f60 0x7fd1c80b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1cde31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c80c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1cde31700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c80b2210 con 0x7fd1c8159330 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1cde31700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c815be30 con 0x7fd1c80cc390 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1cde31700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c815c110 con 0x7fd1c8158f60 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c80c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c80c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40442/0 (socket says 172.21.15.45:40442) 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.575+0000 7fd1c7fff700 1 -- 172.21.15.45:0/3678340513 learned_addr learned my addr 172.21.15.45:0/3678340513 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c77fe700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c80cc390 0x7fd1c8155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c6ffd700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1c8158f60 0x7fd1c80b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c7fff700 1 -- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1c8158f60 msgr2=0x7fd1c80b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c7fff700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1c8158f60 0x7fd1c80b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c7fff700 1 -- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c80cc390 msgr2=0x7fd1c8155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c7fff700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c80cc390 0x7fd1c8155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c7fff700 1 -- 172.21.15.45:0/3678340513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b4009420 con 0x7fd1c8159330 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c6ffd700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1c8158f60 0x7fd1c80b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1c7fff700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c80c6260 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000ccc0 tx=0x7fd1c0001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1bf7fe700 1 -- 172.21.15.45:0/3678340513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000ce00 con 0x7fd1c8159330 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1bf7fe700 1 -- 172.21.15.45:0/3678340513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c000ab70 con 0x7fd1c8159330 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c815c3f0 con 0x7fd1c8159330 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1bf7fe700 1 -- 172.21.15.45:0/3678340513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00070f0 con 0x7fd1c8159330 2023-10-06T05:28:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.576+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8155750 con 0x7fd1c8159330 2023-10-06T05:28:29.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.577+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c80cd1d0 con 0x7fd1c8159330 2023-10-06T05:28:29.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.578+0000 7fd1bf7fe700 1 -- 172.21.15.45:0/3678340513 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1c0007520 con 0x7fd1c8159330 2023-10-06T05:28:29.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.578+0000 7fd1bf7fe700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1b0066f30 0x7fd1b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.578+0000 7fd1bf7fe700 1 -- 172.21.15.45:0/3678340513 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fd1c0084860 con 0x7fd1c8159330 2023-10-06T05:28:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.578+0000 7fd1c77fe700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1b0066f30 0x7fd1b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.579+0000 7fd1c77fe700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1b0066f30 0x7fd1b00693e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8003860 tx=0x7fd1b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:29.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.583+0000 7fd1bf7fe700 1 -- 172.21.15.45:0/3678340513 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1c0051b50 con 0x7fd1c8159330 2023-10-06T05:28:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.779+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 --> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1c80cb530 con 0x7fd1b0066f30 2023-10-06T05:28:29.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.783+0000 7fd1bf7fe700 1 -- 172.21.15.45:0/3678340513 <== mgr.4109 v2:172.21.15.144:6832/119196 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73519 (secure 0 0 0) 0x7fd1c80cb530 con 0x7fd1b0066f30 2023-10-06T05:28:29.785 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1b0066f30 msgr2=0x7fd1b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1b0066f30 0x7fd1b00693e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8003860 tx=0x7fd1b8009040 comp rx=0 tx=0).stop 2023-10-06T05:28:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 msgr2=0x7fd1c80c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c80c6260 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000ccc0 tx=0x7fd1c0001a70 comp rx=0 tx=0).stop 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 shutdown_connections 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fd1b0066f30 0x7fd1b00693e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1c8159330 0x7fd1c80c6260 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c80cc390 0x7fd1c8155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 --2- 172.21.15.45:0/3678340513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1c8158f60 0x7fd1c80b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 >> 172.21.15.45:0/3678340513 conn(0x7fd1c80cdf50 msgr2=0x7fd1c81580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 shutdown_connections 2023-10-06T05:28:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.784+0000 7fd1cde31700 1 -- 172.21.15.45:0/3678340513 wait complete. 2023-10-06T05:28:29.788 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-10-06T05:28:29.799 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":550,"stamp":"2023-10-06T05:28:28.649788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2456164,"kb_used_data":16420,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752518556,"statfs":{"total":773094113280,"available":770579001344,"internally_reserved":0,"allocated":16814080,"data_stored":9322711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002765"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:14.516914+0000","last_change":"2023-10-06T05:28:14.516887+0000","last_active":"2023-10-06T05:28:14.516914+0000","last_peered":"2023-10-06T05:28:14.516914+0000","last_clean":"2023-10-06T05:28:14.516914+0000","last_became_active":"2023-10-06T05:14:53.984613+0000","last_became_peered":"2023-10-06T05:14:53.984613+0000","last_unstale":"2023-10-06T05:28:14.516914+0000","last_undegraded":"2023-10-06T05:28:14.516914+0000","last_fullsized":"2023-10-06T05:28:14.516914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:14.516775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:14.516775+0000","last_clean_scrub_stamp":"2023-10-06T05:28:14.516775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:35.373897+0000","scrub_duration":0.0018200320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":139,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:13.933128+0000","last_change":"2023-10-06T05:28:13.933102+0000","last_active":"2023-10-06T05:28:13.933128+0000","last_peered":"2023-10-06T05:28:13.933128+0000","last_clean":"2023-10-06T05:28:13.933128+0000","last_became_active":"2023-10-06T05:14:53.840030+0000","last_became_peered":"2023-10-06T05:14:53.840030+0000","last_unstale":"2023-10-06T05:28:13.933128+0000","last_undegraded":"2023-10-06T05:28:13.933128+0000","last_fullsized":"2023-10-06T05:28:13.933128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:13.933006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:13.933006+0000","last_clean_scrub_stamp":"2023-10-06T05:28:13.933006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:34.790128+0000","scrub_duration":0.002139855,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:09.526344+0000","last_change":"2023-10-06T05:28:09.526318+0000","last_active":"2023-10-06T05:28:09.526344+0000","last_peered":"2023-10-06T05:28:09.526344+0000","last_clean":"2023-10-06T05:28:09.526344+0000","last_became_active":"2023-10-06T05:14:54.056919+0000","last_became_peered":"2023-10-06T05:14:54.056919+0000","last_unstale":"2023-10-06T05:28:09.526344+0000","last_undegraded":"2023-10-06T05:28:09.526344+0000","last_fullsized":"2023-10-06T05:28:09.526344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:09.526227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:09.526227+0000","last_clean_scrub_stamp":"2023-10-06T05:28:09.526227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:30.383349+0000","scrub_duration":0.001737094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":139,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:08.021218+0000","last_change":"2023-10-06T05:28:08.021173+0000","last_active":"2023-10-06T05:28:08.021218+0000","last_peered":"2023-10-06T05:28:08.021218+0000","last_clean":"2023-10-06T05:28:08.021218+0000","last_became_active":"2023-10-06T05:14:53.839755+0000","last_became_peered":"2023-10-06T05:14:53.839755+0000","last_unstale":"2023-10-06T05:28:08.021218+0000","last_undegraded":"2023-10-06T05:28:08.021218+0000","last_fullsized":"2023-10-06T05:28:08.021218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:08.021070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:08.021070+0000","last_clean_scrub_stamp":"2023-10-06T05:28:08.021070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:08.143099+0000","scrub_duration":0.0016313879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"149'119","reported_seq":291,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:11.957047+0000","last_change":"2023-10-06T05:28:11.957019+0000","last_active":"2023-10-06T05:28:11.957047+0000","last_peered":"2023-10-06T05:28:11.957047+0000","last_clean":"2023-10-06T05:28:11.957047+0000","last_became_active":"2023-10-06T05:14:53.984798+0000","last_became_peered":"2023-10-06T05:14:53.984798+0000","last_unstale":"2023-10-06T05:28:11.957047+0000","last_undegraded":"2023-10-06T05:28:11.957047+0000","last_fullsized":"2023-10-06T05:28:11.957047+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'119","last_scrub_stamp":"2023-10-06T05:28:11.956926+0000","last_deep_scrub":"149'119","last_deep_scrub_stamp":"2023-10-06T05:28:11.956926+0000","last_clean_scrub_stamp":"2023-10-06T05:28:11.956926+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:48.707494+0000","scrub_duration":0.0047260000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:21.004105+0000","last_change":"2023-10-06T05:28:21.004077+0000","last_active":"2023-10-06T05:28:21.004105+0000","last_peered":"2023-10-06T05:28:21.004105+0000","last_clean":"2023-10-06T05:28:21.004105+0000","last_became_active":"2023-10-06T05:14:53.840760+0000","last_became_peered":"2023-10-06T05:14:53.840760+0000","last_unstale":"2023-10-06T05:28:21.004105+0000","last_undegraded":"2023-10-06T05:28:21.004105+0000","last_fullsized":"2023-10-06T05:28:21.004105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:21.003982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:21.003982+0000","last_clean_scrub_stamp":"2023-10-06T05:28:21.003982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:22.241331+0000","scrub_duration":0.0023013700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:18.038571+0000","last_change":"2023-10-06T05:28:18.038534+0000","last_active":"2023-10-06T05:28:18.038571+0000","last_peered":"2023-10-06T05:28:18.038571+0000","last_clean":"2023-10-06T05:28:18.038571+0000","last_became_active":"2023-10-06T05:14:53.984795+0000","last_became_peered":"2023-10-06T05:14:53.984795+0000","last_unstale":"2023-10-06T05:28:18.038571+0000","last_undegraded":"2023-10-06T05:28:18.038571+0000","last_fullsized":"2023-10-06T05:28:18.038571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:18.038433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:18.038433+0000","last_clean_scrub_stamp":"2023-10-06T05:28:18.038433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:40.967890+0000","scrub_duration":0.0023837810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:08.549316+0000","last_change":"2023-10-06T05:28:08.549278+0000","last_active":"2023-10-06T05:28:08.549316+0000","last_peered":"2023-10-06T05:28:08.549316+0000","last_clean":"2023-10-06T05:28:08.549316+0000","last_became_active":"2023-10-06T05:14:53.841119+0000","last_became_peered":"2023-10-06T05:14:53.841119+0000","last_unstale":"2023-10-06T05:28:08.549316+0000","last_undegraded":"2023-10-06T05:28:08.549316+0000","last_fullsized":"2023-10-06T05:28:08.549316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:08.549140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:08.549140+0000","last_clean_scrub_stamp":"2023-10-06T05:28:08.549140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:08.671170+0000","scrub_duration":0.0019870959999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:28:08.490021+0000","last_change":"2023-10-06T05:28:08.489995+0000","last_active":"2023-10-06T05:28:08.490021+0000","last_peered":"2023-10-06T05:28:08.490021+0000","last_clean":"2023-10-06T05:28:08.490021+0000","last_became_active":"2023-10-06T05:14:53.839732+0000","last_became_peered":"2023-10-06T05:14:53.839732+0000","last_unstale":"2023-10-06T05:28:08.490021+0000","last_undegraded":"2023-10-06T05:28:08.490021+0000","last_fullsized":"2023-10-06T05:28:08.490021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:28:08.489897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:28:08.489897+0000","last_clean_scrub_stamp":"2023-10-06T05:28:08.489897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:40.324462+0000","scrub_duration":0.0016728229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.878},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.627,"15min":0.61899999999999999},"min":{"1min":0.51200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.78000000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55100000000000005,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75700000000000001,"5min":0.84099999999999997,"15min":1.476},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57299999999999995,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.78100000000000003,"5min":0.872,"15min":0.872},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76300000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":1.5129999999999999},"last":0.377}]},{"osd":3,"last update":"Fri Oct 6 05:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.502,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81200000000000006,"5min":0.83999999999999997,"15min":2.0190000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.38800000000000001},"max":{"1min":0.79600000000000004,"5min":0.89600000000000002,"15min":2.0059999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.74299999999999999,"5min":0.76800000000000002,"15min":0.90300000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69999999999999996,"5min":0.76800000000000002,"15min":0.93500000000000005},"last":0.33500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.372,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.90000000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.41999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70799999999999996,"5min":0.755,"15min":0.84899999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.49099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.91000000000000003},"last":0.44},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.58699999999999997},"min":{"1min":0.46400000000000002,"5min":0.374,"15min":0.314},"max":{"1min":0.72899999999999998,"5min":0.91300000000000003,"15min":0.92900000000000005},"last":0.58999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575013,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.59799999999999998},"min":{"1min":0.40100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":4.6900000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.80000000000000004,"15min":0.86799999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.67300000000000004},"min":{"1min":0.42099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":9.6300000000000008},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55500000000000005,"15min":0.623},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73599999999999999,"5min":0.76400000000000001,"15min":5.6239999999999997},"last":0.629}]},{"osd":2,"last update":"Fri Oct 6 05:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.70099999999999996},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71399999999999997,"5min":0.75700000000000001,"15min":11.595000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.64400000000000002},"min":{"1min":0.49199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77000000000000002,"5min":0.78100000000000003,"15min":5.5800000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.85599999999999998},"min":{"1min":0.47499999999999998,"5min":0.41699999999999998,"15min":0.39000000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":23.146000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58199999999999996,"15min":0.70699999999999996},"min":{"1min":0.41199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":9.7539999999999996},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.47399999999999998,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70299999999999996,"5min":0.77800000000000002,"15min":9.9969999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.74399999999999999,"5min":0.751,"15min":9.5850000000000009},"last":0.745}]},{"osd":5,"last update":"Fri Oct 6 05:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50600000000000001,"15min":0.52800000000000002},"min":{"1min":0.434,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.748,"5min":0.76200000000000001,"15min":1.6439999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.623},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":9.8420000000000005},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":9.718},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.67600000000000005},"min":{"1min":0.52200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":9.8930000000000007},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38600000000000001,"15min":0.38500000000000001},"min":{"1min":0.26300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40699999999999997,"15min":0.41599999999999998},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.24199999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.73699999999999999},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.436},"min":{"1min":0.30599999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":1.3999999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":1.673},"last":0.371}]},{"osd":3,"last update":"Fri Oct 6 05:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.46000000000000002,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":4.0490000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45700000000000002,"15min":0.45300000000000001},"min":{"1min":0.36799999999999999,"5min":0.32600000000000001,"15min":0.29699999999999999},"max":{"1min":0.627,"5min":0.72199999999999998,"15min":0.77700000000000002},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92600000000000005,"15min":0.94699999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65400000000000003,"15min":0.66100000000000003},"min":{"1min":0.53000000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.95199999999999996},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.39500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.4159999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.499,"5min":0.45600000000000002,"15min":0.436},"max":{"1min":0.96899999999999997,"5min":0.98899999999999999,"15min":1.0369999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81599999999999995,"5min":0.92400000000000004,"15min":1.4810000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66800000000000004,"15min":0.68400000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.167},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.65500000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.42899999999999999},"max":{"1min":0.80300000000000005,"5min":0.94499999999999995,"15min":1.026},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.60699999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93200000000000005,"5min":0.97599999999999998,"15min":1.5649999999999999},"last":0.60099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575014,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1608284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.55600000000000005,"5min":0.82099999999999995,"15min":1.038},"last":0.439},{"interface":"front","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42499999999999999},"min":{"1min":0.317,"5min":0.27400000000000002,"15min":0.25800000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":1.0609999999999999},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.33200000000000002,"5min":0.26000000000000001,"15min":0.252},"max":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":3.855},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.27700000000000002},"max":{"1min":0.65400000000000003,"5min":0.71199999999999997,"15min":1.1919999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42999999999999999,"15min":0.45400000000000001},"min":{"1min":0.33200000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":7.5880000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.49199999999999999,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":7.7670000000000003},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.48699999999999999,"5min":0.438,"15min":0.35899999999999999},"max":{"1min":0.82899999999999996,"5min":0.95899999999999996,"15min":2.4039999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66900000000000004,"15min":0.64800000000000002},"min":{"1min":0.54800000000000004,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83199999999999996,"5min":0.97199999999999998,"15min":2.0870000000000002},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.92300000000000004,"5min":0.98199999999999998,"15min":2.4940000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":5.3600000000000003},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45500000000000002,"15min":0.40799999999999997},"max":{"1min":0.97299999999999998,"5min":1.01,"15min":1.1759999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.56899999999999995,"5min":0.45900000000000002,"15min":0.45400000000000001},"max":{"1min":0.92200000000000004,"5min":0.99199999999999999,"15min":1.0940000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.497,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0169999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.501,"5min":0.47499999999999998,"15min":0.46500000000000002},"max":{"1min":0.88900000000000001,"5min":0.92400000000000004,"15min":1.2150000000000001},"last":0.71699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575013,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.26600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.45100000000000001,"15min":0.46100000000000002},"min":{"1min":0.30099999999999999,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.8759999999999999},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":0.64200000000000002,"15min":0.499},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.23499999999999999},"max":{"1min":19.614000000000001,"5min":19.614000000000001,"15min":19.614000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.54200000000000004,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.0050000000000008,"5min":8.0050000000000008,"15min":8.0050000000000008},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.503,"5min":0.38500000000000001,"15min":0.33900000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.7789999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66600000000000004,"15min":0.68600000000000005},"min":{"1min":0.504,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.643},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.434,"15min":0.432},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.851},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.44700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.2889999999999999,"5min":0.78500000000000003,"15min":0.72399999999999998},"min":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.443},"max":{"1min":10.882999999999999,"5min":10.882999999999999,"15min":10.882999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.69299999999999995,"15min":0.68999999999999995},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":3.6110000000000002,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.70399999999999996,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.45800000000000002,"15min":0.44500000000000001},"max":{"1min":4.0620000000000003,"5min":4.0620000000000003,"15min":4.0620000000000003},"last":0.71099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307384,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064456,"statfs":{"total":96636764160,"available":96322002944,"internally_reserved":0,"allocated":1949696,"data_stored":1016563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53200000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.33200000000000002,"5min":0.247,"15min":0.22900000000000001},"max":{"1min":0.61599999999999999,"5min":0.64000000000000001,"15min":0.94399999999999995},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.41599999999999998,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.21099999999999999},"max":{"1min":0.68000000000000005,"5min":1.75,"15min":1.75},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.437,"15min":0.42899999999999999},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63400000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45200000000000001,"15min":0.439},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.24399999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.89800000000000002},"last":0.317},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46700000000000003,"15min":0.45400000000000001},"min":{"1min":0.36299999999999999,"5min":0.30499999999999999,"15min":0.28599999999999998},"max":{"1min":0.58799999999999997,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68500000000000005,"15min":0.66100000000000003},"min":{"1min":0.52400000000000002,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.81399999999999995,"5min":0.93200000000000005,"15min":1.143},"last":0.629},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.47799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66700000000000004,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83699999999999997,"5min":1.855,"15min":1.855},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.56000000000000005,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.89900000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.52500000000000002,"5min":0.44400000000000001,"15min":0.43099999999999999},"max":{"1min":0.875,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88600000000000001,"5min":1.885,"15min":1.885},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65100000000000002,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85899999999999999,"5min":0.89900000000000002,"15min":1.236},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.68000000000000005},"min":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91400000000000003,"5min":0.94399999999999995,"15min":1.1950000000000001},"last":0.68200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":311484,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060356,"statfs":{"total":96636764160,"available":96317804544,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57999999999999996,"15min":0.623},"min":{"1min":0.35699999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":12.983000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.45700000000000002,"5min":0.33400000000000002,"15min":0.33100000000000002},"max":{"1min":0.84599999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.47199999999999998,"5min":0.39900000000000002,"15min":0.32500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.7430000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.5,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.8080000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.46999999999999997,"5min":0.36399999999999999,"15min":0.34899999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":12.911},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77200000000000002,"5min":0.82699999999999996,"15min":5.0890000000000004},"last":0.751}]},{"osd":3,"last update":"Fri Oct 6 05:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.60599999999999998},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":12.999000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60299999999999998,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.81000000000000005,"5min":0.86699999999999999,"15min":13.289999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76100000000000001,"5min":0.95099999999999996,"15min":1.5760000000000001},"last":0.432},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.32400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.70399999999999996,"5min":0.85599999999999998,"15min":1.552},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":12.949},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.33400000000000002},"max":{"1min":0.83599999999999997,"5min":0.90200000000000002,"15min":5.2549999999999999},"last":0.47999999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69299999999999995,"5min":0.873,"15min":2.871},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":12.859},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":1017916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.754,"5min":0.79400000000000004,"15min":0.82299999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.89400000000000002},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.45100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66700000000000004,"5min":0.82199999999999995,"15min":0.84599999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.371,"15min":0.33700000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.81599999999999995},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.312,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":0.71799999999999997,"15min":0.83299999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.47199999999999998,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":0.70999999999999996,"5min":0.80900000000000005,"15min":0.876},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.36899999999999999},"max":{"1min":0.72899999999999998,"5min":0.85999999999999999,"15min":0.875},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38},"max":{"1min":0.76500000000000001,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.755}]},{"osd":4,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.498,"15min":0.48499999999999999},"min":{"1min":0.33400000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68200000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.92800000000000005},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77600000000000002,"5min":0.86699999999999999,"15min":0.876},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.39500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":0.83399999999999996,"15min":0.89300000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.36699999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74399999999999999,"5min":0.82899999999999996,"15min":0.90900000000000003},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:28:29.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T05:28:29.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.932+0000 7fdc88b7f700 1 Processor -- start 2023-10-06T05:28:29.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.932+0000 7fdc88b7f700 1 -- start start 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc88b7f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc840c8b70 0x7fdc840c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc88b7f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc840c3b50 con 0x7fdc84157330 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc88b7f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc84157700 con 0x7fdc840ccb70 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc88b7f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc840c98e0 con 0x7fdc840c8b70 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc82d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc840c8b70 0x7fdc840c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc8259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdc84157330 0x7fdc840ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45712/0 (socket says 172.21.15.45:45712) 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc82d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc840c8b70 0x7fdc840c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40454/0 (socket says 172.21.15.45:40454) 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.933+0000 7fdc82d9d700 1 -- 172.21.15.45:0/16406052 learned_addr learned my addr 172.21.15.45:0/16406052 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:29.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc82d9d700 1 -- 172.21.15.45:0/16406052 >> v1:172.21.15.144:6789/0 conn(0x7fdc840ccb70 legacy=0x7fdc840aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc82d9d700 1 -- 172.21.15.45:0/16406052 >> v1:172.21.15.45:6789/0 conn(0x7fdc84157330 legacy=0x7fdc840ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc8159a700 1 -- 172.21.15.45:0/16406052 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520571822 0 0) 0x7fdc840c3b50 con 0x7fdc84157330 2023-10-06T05:28:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc82d9d700 1 -- 172.21.15.45:0/16406052 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc840c9bc0 con 0x7fdc840c8b70 2023-10-06T05:28:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc82d9d700 1 --2- 172.21.15.45:0/16406052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c8f90 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009f20 tx=0x7fdc7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a97c843fb5b5ea5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc8159a700 1 -- 172.21.15.45:0/16406052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00e070 con 0x7fdc840c8b70 2023-10-06T05:28:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc8159a700 1 -- 172.21.15.45:0/16406052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c00ba80 con 0x7fdc840c8b70 2023-10-06T05:28:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.934+0000 7fdc8159a700 1 -- 172.21.15.45:0/16406052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0092e0 con 0x7fdc840c8b70 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.935+0000 7fdc88b7f700 1 -- 172.21.15.45:0/16406052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 msgr2=0x7fdc840c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.935+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/16406052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c8f90 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009f20 tx=0x7fdc7c009840 comp rx=0 tx=0).stop 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.935+0000 7fdc88b7f700 1 -- 172.21.15.45:0/16406052 shutdown_connections 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.935+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/16406052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c8f90 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.935+0000 7fdc88b7f700 1 -- 172.21.15.45:0/16406052 >> 172.21.15.45:0/16406052 conn(0x7fdc84067e00 msgr2=0x7fdc840cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.935+0000 7fdc88b7f700 1 -- 172.21.15.45:0/16406052 shutdown_connections 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.935+0000 7fdc88b7f700 1 -- 172.21.15.45:0/16406052 wait complete. 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.936+0000 7fdc88b7f700 1 Processor -- start 2023-10-06T05:28:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.936+0000 7fdc88b7f700 1 -- start start 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc88b7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc88b7f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc840ccb70 0x7fdc841521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc88b7f700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc84157330 0x7fdc8414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc88b7f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840ab820 con 0x7fdc840c8b70 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc88b7f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840abb00 con 0x7fdc840ccb70 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc88b7f700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840abde0 con 0x7fdc84157330 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40460/0 (socket says 172.21.15.45:40460) 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 -- 172.21.15.45:0/3546258393 learned_addr learned my addr 172.21.15.45:0/3546258393 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc81d9b700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc840ccb70 0x7fdc841521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 -- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc84157330 msgr2=0x7fdc8414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc82d9d700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc84157330 0x7fdc8414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc84157330 0x7fdc8414fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 -- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc840ccb70 msgr2=0x7fdc841521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc840ccb70 0x7fdc841521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 -- 172.21.15.45:0/3546258393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c009ae0 con 0x7fdc840c8b70 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc81d9b700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc840ccb70 0x7fdc841521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.937+0000 7fdc8259c700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c5f60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c810 tx=0x7fdc74007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.938+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74007300 con 0x7fdc840c8b70 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.938+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74007480 con 0x7fdc840c8b70 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.938+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc840ac0c0 con 0x7fdc840c8b70 2023-10-06T05:28:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.938+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740057f0 con 0x7fdc840c8b70 2023-10-06T05:28:29.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.938+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc840c64a0 con 0x7fdc840c8b70 2023-10-06T05:28:29.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.939+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8406e350 con 0x7fdc840c8b70 2023-10-06T05:28:29.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.939+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc74005b60 con 0x7fdc840c8b70 2023-10-06T05:28:29.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.940+0000 7fdc7a7fc700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc64066f30 0x7fdc640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:29.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.940+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7fdc74084020 con 0x7fdc840c8b70 2023-10-06T05:28:29.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.940+0000 7fdc81d9b700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc64066f30 0x7fdc640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:29.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.940+0000 7fdc81d9b700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc64066f30 0x7fdc640693e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c001840 tx=0x7fdc6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:29.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:29.944+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc74051310 con 0x7fdc840c8b70 2023-10-06T05:28:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.148+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdc840c37e0 con 0x7fdc840c8b70 2023-10-06T05:28:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.152+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdc74051310 con 0x7fdc840c8b70 2023-10-06T05:28:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.152+0000 7fdc7a7fc700 1 -- 172.21.15.45:0/3546258393 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdc74054930 con 0x7fdc840c8b70 2023-10-06T05:28:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc64066f30 msgr2=0x7fdc640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc64066f30 0x7fdc640693e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c001840 tx=0x7fdc6c009040 comp rx=0 tx=0).stop 2023-10-06T05:28:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 msgr2=0x7fdc840c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c5f60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c810 tx=0x7fdc74007cc0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 shutdown_connections 2023-10-06T05:28:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7fdc64066f30 0x7fdc640693e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc84157330 0x7fdc8414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc840c8b70 0x7fdc840c5f60 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 --2- 172.21.15.45:0/3546258393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc840ccb70 0x7fdc841521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 >> 172.21.15.45:0/3546258393 conn(0x7fdc84067e00 msgr2=0x7fdc840c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 shutdown_connections 2023-10-06T05:28:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.155+0000 7fdc88b7f700 1 -- 172.21.15.45:0/3546258393 wait complete. 2023-10-06T05:28:30.171 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:28:30.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:28:30.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.302+0000 7f27e5294700 1 Processor -- start 2023-10-06T05:28:30.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.302+0000 7f27e5294700 1 -- start start 2023-10-06T05:28:30.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.302+0000 7f27e5294700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e00692d0 0x7f27e0159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27e5294700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27e015d040 0x7f27e00aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27e5294700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e00ca6d0 con 0x7f27e0159170 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27e5294700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e015d550 con 0x7f27e015d040 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27e5294700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e015d810 con 0x7f27e00692d0 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27deffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f27e0159170 0x7f27e0068bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45728/0 (socket says 172.21.15.45:45728) 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27df7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27e015d040 0x7f27e00aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27deffd700 1 -- 172.21.15.45:0/435213608 learned_addr learned my addr 172.21.15.45:0/435213608 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27de7fc700 1 --2- 172.21.15.45:0/435213608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e00692d0 0x7f27e0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27df7fe700 1 -- 172.21.15.45:0/435213608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e00692d0 msgr2=0x7f27e0159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27df7fe700 1 --2- 172.21.15.45:0/435213608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e00692d0 0x7f27e0159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27df7fe700 1 -- 172.21.15.45:0/435213608 >> v1:172.21.15.45:6789/0 conn(0x7f27e0159170 legacy=0x7f27e0068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27df7fe700 1 -- 172.21.15.45:0/435213608 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e015daf0 con 0x7f27e015d040 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27ddffb700 1 -- 172.21.15.45:0/435213608 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429519724 0 0) 0x7f27e00ca6d0 con 0x7f27e0159170 2023-10-06T05:28:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27df7fe700 1 --2- 172.21.15.45:0/435213608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e015d040 0x7f27e00aea60 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009e60 tx=0x7f27d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2b15b99c95e60b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27ddffb700 1 -- 172.21.15.45:0/435213608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d800e070 con 0x7f27e015d040 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27ddffb700 1 -- 172.21.15.45:0/435213608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f27d800bc30 con 0x7f27e015d040 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.303+0000 7f27ddffb700 1 -- 172.21.15.45:0/435213608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8012430 con 0x7f27e015d040 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 -- 172.21.15.45:0/435213608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e015d040 msgr2=0x7f27e00aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 --2- 172.21.15.45:0/435213608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e015d040 0x7f27e00aea60 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009e60 tx=0x7f27d8009780 comp rx=0 tx=0).stop 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 -- 172.21.15.45:0/435213608 shutdown_connections 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 --2- 172.21.15.45:0/435213608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e015d040 0x7f27e00aea60 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 --2- 172.21.15.45:0/435213608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e00692d0 0x7f27e0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 -- 172.21.15.45:0/435213608 >> 172.21.15.45:0/435213608 conn(0x7f27e006f7c0 msgr2=0x7f27e00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 -- 172.21.15.45:0/435213608 shutdown_connections 2023-10-06T05:28:30.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.304+0000 7f27e5294700 1 -- 172.21.15.45:0/435213608 wait complete. 2023-10-06T05:28:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 Processor -- start 2023-10-06T05:28:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 -- start start 2023-10-06T05:28:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e00692d0 0x7f27e01585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0159170 0x7f27e00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e015d040 0x7f27e00c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00c7140 con 0x7f27e00692d0 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00c7700 con 0x7f27e0159170 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.306+0000 7f27e5294700 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e015bdc0 con 0x7f27e015d040 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e00692d0 0x7f27e01585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e00692d0 0x7f27e01585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40490/0 (socket says 172.21.15.45:40490) 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 -- 172.21.15.45:0/2592583215 learned_addr learned my addr 172.21.15.45:0/2592583215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27de7fc700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0159170 0x7f27e00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27df7fe700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e015d040 0x7f27e00c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 -- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e015d040 msgr2=0x7f27e00c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e015d040 0x7f27e00c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 -- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0159170 msgr2=0x7f27e00c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0159170 0x7f27e00c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 -- 172.21.15.45:0/2592583215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8009a20 con 0x7f27e00692d0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27deffd700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e00692d0 0x7f27e01585b0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f27d000c810 tx=0x7f27d0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27c6ffd700 1 -- 172.21.15.45:0/2592583215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0005b30 con 0x7f27e00692d0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27c6ffd700 1 -- 172.21.15.45:0/2592583215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f27d0005c90 con 0x7f27e00692d0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e015c0a0 con 0x7f27e00692d0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27c6ffd700 1 -- 172.21.15.45:0/2592583215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0007320 con 0x7f27e00692d0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.307+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e0158af0 con 0x7f27e00692d0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.308+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e0063b50 con 0x7f27e00692d0 2023-10-06T05:28:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.309+0000 7f27c6ffd700 1 -- 172.21.15.45:0/2592583215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27d001b400 con 0x7f27e00692d0 2023-10-06T05:28:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.309+0000 7f27c6ffd700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27cc066f30 0x7f27cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:28:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.309+0000 7f27c6ffd700 1 -- 172.21.15.45:0/2592583215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5698+0+0 (secure 0 0 0) 0x7f27d0083ff0 con 0x7f27e00692d0 2023-10-06T05:28:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.310+0000 7f27de7fc700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27cc066f30 0x7f27cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:28:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.310+0000 7f27de7fc700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27cc066f30 0x7f27cc0693e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f27d4006fd0 tx=0x7f27d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:28:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.314+0000 7f27c6ffd700 1 -- 172.21.15.45:0/2592583215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27d00513c0 con 0x7f27e00692d0 2023-10-06T05:28:30.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.566+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27e015ddd0 con 0x7f27e00692d0 2023-10-06T05:28:30.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.567+0000 7f27c6ffd700 1 -- 172.21.15.45:0/2592583215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 367 v367) v1 ==== 95+0+3532 (secure 0 0 0) 0x7f27d00549e0 con 0x7f27e00692d0 2023-10-06T05:28:30.569 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-10-06T05:28:30.569 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":367,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8771,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3836311272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3836311272},{"type":"v1","addr":"172.21.15.144:6836","nonce":3836311272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/749343025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":749343025},{"type":"v1","addr":"172.21.15.144:6837","nonce":749343025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":8780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/490759433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":490759433},{"type":"v1","addr":"172.21.15.45:6835","nonce":490759433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":353},{"gid":11343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/849889156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":849889156},{"type":"v1","addr":"172.21.15.45:6833","nonce":849889156}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":367}],"filesystems":[]} 2023-10-06T05:28:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27cc066f30 msgr2=0x7f27cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27cc066f30 0x7f27cc0693e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f27d4006fd0 tx=0x7f27d4008040 comp rx=0 tx=0).stop 2023-10-06T05:28:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e00692d0 msgr2=0x7f27e01585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:28:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e00692d0 0x7f27e01585b0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f27d000c810 tx=0x7f27d0007df0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 shutdown_connections 2023-10-06T05:28:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:6832/119196,v1:172.21.15.144:6833/119196] conn(0x7f27cc066f30 0x7f27cc0693e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27e015d040 0x7f27e00c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e00692d0 0x7f27e01585b0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 --2- 172.21.15.45:0/2592583215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27e0159170 0x7f27e00c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:28:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 >> 172.21.15.45:0/2592583215 conn(0x7f27e006f7c0 msgr2=0x7f27e00cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:28:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 shutdown_connections 2023-10-06T05:28:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-10-06T05:28:30.569+0000 7f27e5294700 1 -- 172.21.15.45:0/2592583215 wait complete. 2023-10-06T05:28:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 367 2023-10-06T05:28:30.584 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T05:28:30.584 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T05:28:30.585 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:28:30.585 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.593 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:28:30.594 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:28:30.594 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.643 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:28:30.643 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:28:30.643 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.654 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:28:30.654 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:28:30.654 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.705 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:28:30.706 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T05:28:30.706 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T05:28:30.706 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.737 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T05:28:30.737 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T05:28:30.737 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.806 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T05:28:30.806 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T05:28:30.806 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.877 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T05:28:30.877 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T05:28:30.877 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.953 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T05:28:30.953 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T05:28:30.953 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:30.984 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T05:28:30.985 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T05:28:30.985 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.048 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T05:28:31.048 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T05:28:31.048 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.123 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T05:28:31.124 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T05:28:31.124 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.192 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T05:28:31.192 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T05:28:31.192 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T05:28:31.193 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.219 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T05:28:31.219 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T05:28:31.219 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.257 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T05:28:31.257 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T05:28:31.258 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T05:28:31.258 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.273 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T05:28:31.273 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T05:28:31.273 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.308 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T05:28:31.309 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T05:28:31.309 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:28:31.360 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T05:28:31.360 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T05:28:31.360 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-10-06T05:28:31.408 INFO:teuthology.orchestra.run.smithi045.stdout:1696569298.9273782 mon.a (mon.0) 134 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:28:31.408 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T05:28:31.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-10-06T05:28:31.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-10-06T05:28:31.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | head -n 1 2023-10-06T05:28:31.583 INFO:teuthology.orchestra.run.smithi045.stdout:1696569298.9273782 mon.a (mon.0) 134 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:28:31.583 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:28:31.584 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T05:28:31.818 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:28:31.818 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T05:28:31.915 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:28:31.916 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T05:28:31.995 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:28:31.995 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T05:28:32.071 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:28:32.071 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T05:28:32.242 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:28:32.242 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T05:28:32.347 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:28:32.347 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T05:28:32.422 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:28:32.422 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T05:28:32.495 INFO:tasks.ceph:Archiving mon data... 2023-10-06T05:28:32.496 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/data/mon.a.tgz 2023-10-06T05:28:32.496 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2023-10-06T05:28:32.520 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.jbi9lSupxs 2023-10-06T05:28:32.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.jbi9lSupxs 2023-10-06T05:28:32.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.jbi9lSupxs 2023-10-06T05:28:32.837 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.jbi9lSupxs is 686KB 2023-10-06T05:28:32.915 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.jbi9lSupxs 2023-10-06T05:28:32.938 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/data/mon.b.tgz 2023-10-06T05:28:32.938 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-10-06T05:28:32.960 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.o5Fm3qP92h 2023-10-06T05:28:32.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.o5Fm3qP92h 2023-10-06T05:28:33.182 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.o5Fm3qP92h 2023-10-06T05:28:33.270 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.o5Fm3qP92h is 728KB 2023-10-06T05:28:33.369 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.o5Fm3qP92h 2023-10-06T05:28:33.387 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/data/mon.c.tgz 2023-10-06T05:28:33.387 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-10-06T05:28:33.444 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.rKjFILnEWw 2023-10-06T05:28:33.444 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rKjFILnEWw 2023-10-06T05:28:33.645 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.rKjFILnEWw 2023-10-06T05:28:33.732 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.rKjFILnEWw is 729KB 2023-10-06T05:28:33.835 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.rKjFILnEWw 2023-10-06T05:28:33.853 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T05:28:33.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:28:33.856 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:28:33.920 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T05:28:33.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:28:33.931 DEBUG:teuthology.orchestra.run.smithi144:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:28:33.964 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:28:33.987 INFO:teuthology.orchestra.run.smithi144.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:28:34.522 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T05:28:34.525 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/remote/smithi045/crash 2023-10-06T05:28:34.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:28:34.593 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/remote/smithi144/crash 2023-10-06T05:28:34.594 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:28:34.623 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T05:28:34.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:34.626 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:34.716 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:34.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:28:34.778 INFO:tasks.ceph:Compressing logs... 2023-10-06T05:28:34.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:28:34.793 DEBUG:teuthology.orchestra.run.smithi144:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:28:44.127 INFO:tasks.ceph:Archiving logs... 2023-10-06T05:28:44.128 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/remote/smithi045/log 2023-10-06T05:28:44.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:28:45.864 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/remote/smithi144/log 2023-10-06T05:28:45.865 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:28:46.800 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T05:28:46.809 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T05:28:46.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:28:46.812 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:28:46.899 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T05:28:46.909 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T05:28:46.909 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:28:46.912 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:28:46.931 INFO:teuthology.orchestra.run.smithi144.stderr:bash: ntpq: command not found 2023-10-06T05:28:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-10-06T05:28:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:28:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2023-10-06T05:28:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +116us[ +103us] +/- 73ms 2023-10-06T05:28:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -321us[ -335us] +/- 78ms 2023-10-06T05:28:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +254us[ +241us] +/- 84ms 2023-10-06T05:28:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:28:46.939 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:28:46.939 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-10-06T05:28:46.940 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +102us[ +103us] +/- 73ms 2023-10-06T05:28:46.940 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -360us[ -359us] +/- 78ms 2023-10-06T05:28:46.940 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +249us[ +249us] +/- 84ms 2023-10-06T05:28:46.940 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:28:46.940 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T05:28:46.955 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T05:28:46.956 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T05:28:46.971 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:28:47.038 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:28:47.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:28:47.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:28:47.174 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:28:47.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:28:47.208 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T05:28:47.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T05:28:47.229 INFO:teuthology.task.internal:Duration was 1546.283714 seconds 2023-10-06T05:28:47.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T05:28:47.238 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T05:28:47.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:28:47.241 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:28:47.295 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:28:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:28:47.597 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T05:28:47.597 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2023-10-06T05:28:47.598 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:28:47.627 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2023-10-06T05:28:47.628 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:28:47.651 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T05:28:47.651 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:28:47.671 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:28:47.731 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T05:28:47.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:28:47.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:28:47.845 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T05:28:47.856 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T05:28:47.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:28:47.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:28:47.915 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T05:28:47.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:28:47.931 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:28:47.961 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2023-10-06T05:28:47.981 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2023-10-06T05:28:47.995 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:28:48.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:28:48.040 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:28:48.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:28:48.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T05:28:48.069 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T05:28:48.070 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/remote/smithi045 2023-10-06T05:28:48.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:28:48.123 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322/remote/smithi144 2023-10-06T05:28:48.124 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:28:48.162 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T05:28:48.162 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:28:48.166 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:28:48.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T05:28:48.225 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T05:28:48.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T05:28:48.235 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T05:28:48.236 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:28:48.238 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:28:48.259 INFO:teuthology.orchestra.run.smithi045.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:28 /home/ubuntu/cephtest 2023-10-06T05:28:48.272 INFO:teuthology.orchestra.run.smithi144.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:28 /home/ubuntu/cephtest 2023-10-06T05:28:48.273 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T05:28:48.284 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T05:28:48.309 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T05:28:48.331 DEBUG:teuthology.nuke:shortname: smithi045 2023-10-06T05:28:48.331 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:28:48.350 DEBUG:teuthology.nuke:shortname: smithi144 2023-10-06T05:28:48.351 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:28:48.354 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:53:42.380397', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKBa3lfTt2DZu9P4yZRPDyvyQf+QSD5SAI+sU0pgiHdmC0MYDM0qgHDrd8YHKd1B8IXkPBZAihL7kQmYTYjiR1Y='} 2023-10-06T05:28:48.375 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:53:42.381794', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIWK9WT+5apjDhQm97J+fYPfqyp6XWshyveO/zIdWuVysyXRHbrCqwG5j7SWBKH3wdTQQOfl3zYtPbrFPgBzFrQ='} 2023-10-06T05:28:48.403 INFO:teuthology.orchestra.console.smithi045:Power off 2023-10-06T05:28:48.403 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:28:48.425 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2023-10-06T05:28:48.426 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:28:48.441 INFO:teuthology.orchestra.console.smithi144:Power off 2023-10-06T05:28:48.442 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:28:48.450 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-10-06T05:28:48.465 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-10-06T05:28:48.465 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:28:48.489 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-06T05:28:52.452 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:28:52.490 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:28:52.561 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2023-10-06T05:28:52.600 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-06T05:28:56.566 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:28:56.601 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:28:56.676 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2023-10-06T05:28:56.710 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-10-06T05:28:56.778 INFO:teuthology.orchestra.console.smithi045:Power off completed 2023-10-06T05:28:56.811 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-10-06T05:28:56.915 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} duration: 1546.283714056015 failure_reason: '"1696569298.9273782 mon.a (mon.0) 134 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T05:28:56.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T05:28:57.017 INFO:teuthology.run:FAIL